In this episode Federico and Matthew talk about their experience testing installers. Taking from their experience testing the ASP.NET MVC installer, they share tips and advice for those faced with the challenge of testing the very first experience of many products: the installer.

News:

• Coding QA “Studio”
• Visual Studio 2010 and .NET Framework 4 released
• Microsoft KIN released

Installer Testing

• Before somebody gets to use a product, they have to get it into their machines. It is the first experience with a product.
• When building an installer test plan start by brainstorming 4 general areas:
  • Pre-requisites. How does the installer check for requirements? Does it provide enough information if requirements are not met? Does it fail as soon as possible?
  • Installation: What files are copies and where? Registry changes? GAC changes? Folders created? IIS changes? Does it need to check the web?
  • Uninstall: Is everything rolled back completely? Is anything is supposed to be left behind?
  • Machine state: OS, bitness, other applications running, hardware, disk space, network access, permissions.
• Based on these variables, start building combinations of scenarios.
• Use additional heuristics for identifying and discussing special cases:
  • Interruptions: Machine goes into hibernation, unplug network cable, user cancels the installer.
  • Full circle: Snapshot the state of the machine before installing, make sure it is the same after uninstalling.
  • Side by Side: Previous versions of the software installed, multiple versions installed.
  • Reinstall/Repair: Reinstall multiple times, break the application and repair it.
  • Temp folder: what is written in the temp folder.
  • Unintended changes: Additional files in placed installation directory.
  • Partial installs: some of files that are meant to be installed are already present, some of the files that are meant to be removed are not there.

In our experience, installer testing is closer to scripted testing than exploratory testing.

In this episode Matthew and Federico sit down with Damian Edwards to talk about his experience working for a company that writes functional tests as part of their development cycle. Damian is a new Program Manager in the Visual Web Developer team and used to work for Readify, a consulting company.

Interview with Damian Edwards

• Introduction
• Types of testing that an agile team can do.
  • Unit testing
  • Integration testing.
  • Functional testing
  • Performance/Load testing.
• Working on a team that releases to web every week.
  • Who is responsible for what type of testing?
  • One week is too small of a time to do dev and test work.
  • Developers implement features in one week, testers test it the next.
  • What happens if a tester finds a bug?
• Using functional automated tests.
  • Is there enough time for testers to write them?
  • What automation framework did you use?
  • How did you deal with fragile tests?
  • Making markup easier to automate vs. making markup minimal for client.
• Links
  • http://www.graysonline.com/
  • http://webassert.codeplex.com/
  • http://webformsmvp.com/
  • http://damianedwards.wordpress.com/
  • http://twitter.com/DamianEdwards

In this episode Matthew and Federico talk about the quality of test code. They share what to look for when reviewing test code and their experience dealing with automated test cases written by a large team.

News

• One year anniversary of the Coding QA podcast!!!

Test Code Quality

• A sense of pride on the code you write.
• Reviewing tests is more effective when done early.
• What to look for when reviewing tests?
  • How easy it is to figure out why a test failed from the log?
  • How good is the test at catching the intended bug?
  • How resilient is the test to other changes?
  • Over verifying and under verifying.
  • Does it align to .NET coding guidelines?
• Common excuses by testers
  • I don't have time.
  • I don't understand what is the value.
  • Test code is not the same as product code.
• Test engineers produce code, as professionals we must strive for the best code.