Monday, November 4, 2013

The Dangers of Ad Hoc Testing and the Need for a Test Plan

There's an old anecdote that when a bug is found in an application by an end user, often developers will reply, "Well it works for me." Obviously this isn't a sustainable approach for the health of an application or the developer's job security. Developers are sometimes guilty of only testing the happy path, or don't test their application as a less-privileged user. These failures can become evident in user acceptance testing (UAT).

But sometimes UAT doesn't have a formal test plan and bugs slip through the cracks and are promoted into a production environment and then it's time to develop fixes. When these bugs are fixed, testing can continue to be disregarded in what I call “testing apathy”. Testing apathy can impair both developers and testers as they tend to test the new functionality almost exclusively, completely overlooking the importance of regression testing the application.

The lack of a test plan leaves testers wandering through an application like sheep without a shepherd. A cursory test may lead one to believe that 95% of the application may be working as designed, but the remaining 5% is still important to the health and stability of the application.

Asides from reflecting poorly on the development team, the lack of testing impacts both the end user and the application. Broken functionality can impair an end user's faith in the application which can lead to curtailed adoption of the application. It's hard to escape the headlines in the news about Healthcare.gov and its lack of testing which is causing a lot of frustration among users. It's best to have functionality that's been thoroughly tested before it is made available to end users.

Let's say you have an application in place, but never created a formal, functional testing plan. Fret not; it's never too late to create a test plan.

Why create a test plan now if your application has been doing well so far? Testing applicable business scenarios will ultimately give a better holistic view of the application. For existing applications, the creation of a test plan will make you the proud owner of a regression test plan. When new functionality is promoted between environments, it will be easy to pull up existing test cases and validate that prior functionality still operates as intended.

Patching the application? The test plan can help you validate that everything is copacetic after the patch's installation.

Thinking of doing a migration from one platform to another? The user stories within the test plan can help define requirements for the new platform.

From a developer's point a view, a test plan helps the application's deployment cycle break out of this Sisyphean task of working on nothing but production bug fixes. By testing up front, more time and energy can be focused on developing new functionality.

From a tester's perspective, testing is a chance to get dirty with the application and provide feedback to the development team before the application goes live. Testing not only validates the application from a technical perspective, but provides a channel of input to improve the user experience, all the while aligning to the original requirements specified in the user stories.

Whether a test plan is kept in an Excel spreadsheet, OneNote, SharePoint, TFS, or another application, you need to find an approach for managing a test plan in your organization that is easy to follow and provides a means to collaborate on bugs and issues when they arise.