Maximise test coverage by writing effective Test Cases

Once your Test Plan has been structured and completed with all the relevant and consolidated information, based on the requirements gathering and specification process, the real creative work can begin, i.e. thinking about and designing the Test Cases that will be executed on the software system under examination

.

Although this phase might seem very boring, especially when it is not undertaken with the right spirit, it represents one of the most important of the test process, as only through a proper Test Case design is it  possible to achieve maximum test coverage.

There are of course several approaches and techniques, but before describing those that have provided me with a high return on investment, I would like to explore briefly the most used by the testing community. Obviously we need to distinguish between the different scenarios like a new project, a redesign of an existing one and last but not least a project under maintenance which can be subject to bug-fixing and/or small enhancements. 

In the case of new projects, life should be straightforward, as in theory there should be enough time to do everything well. However this rarely happens  because when there is too much time available for thinking, people tend to spend it on planning and discussing what might be the right approach, which test types and levels have to be forecasted. They then realise  too late that what was planned cannot be put in practise due to a shortage of time. 

However, if the available time is used properly,  a proactive approach is usually the best one because it is possible to concentrate on the test design effort before any code has been written. Then refining the test cases as soon as the first version of the software is ready for testing (even though I strongly recommended getting access to review the implementation during development) and at the same time giving the developers the chance to verify that they have taken into account all the conditions stated for each test case together with the expected output, will result in a higher quality release that will be put through formal testing at the end of the development phase. 

Obviously this approach is not only proactive but also iterative, as it is developed over the project life cycle rather than either upstream or downstream of the coding phase; but of course the availability and cooperation of the development team is vital.

In such a case the Test Design techniques I would suggest, also to prepare test driven scenarios, are the Equivalent Class Partitioning together with Boundary Value Analysis and a further refinement using Decision Tables. In this way the Test Cases will ensure most conditions are covered, but this is still not enough, as the objective is to cover all the business processes and the possible scenarios as well. To do that a good approach would be to use Mind Maps, which allows us to draw what test/business analysts have in their heads, and since it is usually based both on the experience in the testing field (which is very important) and on the business domain (having correctly understood all the functionalities and their interactions) it gives us a clear picture of the system behaviour.

An additional benefit is provided through the ease of translating such a representation into the Test Management platform, as the structure of the Test Scenarios is already prepared – The Mind Map –.

The contrary approach, i.e. a reactive one,  would normally be adopted for projects under maintenance. This is because on the one hand it is not possible to determine when and where software failures will occur, on the other hand for small enhancements (which does not merit a new project) there is not usually enough time available to follow a proper test process. In this case the existing test cases are typically updated to take the changes into account.

A more complex sub-scenario must be faced when no Test Cases are available for the existing functionalities (which is very common for projects developed when the organisation was not yet well organised from the testing perspective), one which requires a very fast design process in order to react quickly to changes.

In such cases the best approach is to start creating the specific test cases to test the functionality/feature subject to changes and incrementally add cases every time new defects/small enhancements occur. But of course this requires more resources as the resources that have already been allocated will be busy with test execution.

To summarise, for existing projects the test design is mostly based on the experience of the test personnel involved and will usually require additional testing resources.

For redesign projects the situation is slightly different, because even though the existing Test Cases, if any, can be properly updated as the project plan should take into account the whole Test Process, forecasting the right time for Test Case Design and validation, can become more complex if the system behaviour is likely to differ heavily from the existing one. In this case the approach mentioned above for band new projects can provide some help.

If this is not the case the advantage is represented by the possibility of doing Exploratory Testing as the actual implementation is available both for investigation and to get familiar with the processes; moreover the key users feedback and suggestions can be exploited to better design effective Test Cases.

I have briefly explored the most common approaches, now I would like to describe the approach I typically apply and that for recent projects has allowed me to maximize the return on investment, a topic that is very important for organisations.

I have not reinvented the wheel, as it is a combination of the approaches mentioned previously.

Basically I always follow an iterative approach, no matter whether or not I am working for a brand-new project or for an existing one. My Test Strategy and the consequent Test Design approach, is essentially the same with slight adjustments in order to fit with the organisation standards, methodologies and techniques.

My first requirement is to have access to the entire set of project documentation, both business and technical.  Then, as soon as I can get all the answers/explanations I need, I organise some sessions with key users and specifically:

• For a new project I meet directly with them in order to understand their vision (especially from the usability perspective) and how they would like to interact with the new application, which functionalities represent the core and which are desired but very important to maximise their effectiveness in performing operational duties
• For an existing project redesign/under maintenance it is even better as I am typically in a position to observe how they work (basically doing exploratory testing together) and to understand if the application is forcing users in to unnatural usage patterns and/or if there is space for improvement. I usually spend a lot time with them to be able to quickly report back to the project manager and verify if the feedback/learnings can be considered

Spending time with the key users/subject matter experts/final users will bring you many benefits that are hard to imagine. Moreover they really appreciate this kind of involvement/contribution and this makes them more willing  and open to changes, a subject to which there is always a strong resistance.

Where a project has a prototype or set of navigable wireframes that is available before the implementation commences  (i.e. during the functional analysis phase) it is a good practice to organise some Usability Testing sessions, as doing so makes it possible to anticipate problems which if missed could result in expensive rework if the key users are only able to look at the application immediately after development has been completed (this is true even if an Agile approach is being used, as also in case of intermediate validations – UATs – carried out at the end of each sprint, if the key users do not have the possibility to see a prototype before development it could result in changes being needed in order to provide the desired behaviour).

When the working session and the Usability Testing are performed together with the key users, it allows the Test Cases to be better organises and designed. This is because by focussing on the functionality and interaction and how the final users will use them, allows us to prepare business oriented Test Cases and to think of all the possible related cases in advance.

Once the skeleton of the Test Cases has been prepared, it is now time to apply the techniques mentioned above, i.e. Equivalent Class Partitioning with Boundary Value Analysis and Decision Tables (there are many other techniques but these three are the techniques I prefer and those that can be automatically implemented once experience has been acquired).

At this stage things are almost complete but our Test Cases need further refinement; how can we do that? Easy, we need to push developers to show the units, modules and user stories that have been implemented. This is an activity that needs to be carried out over the entire length of the project, by doing this we do not need to wait until the end of the development/sprint to see the actual implementation with our own eyes and we refine the Test Cases incrementally during the project life cycle. An additional benefit of this approach is having the possibility to anticipate the Test Execution as well, removing the main issues that could result in a costly rework if undertaken downstream of the development phase.   

Last but not least, by following such approach it is possible, both in case of manual and automated testing, to estimate the Test Execution time and to add it as a reference for each Test Case. This provides valuable information for the Project and Test Managers, who can easily adjust their plans based on the Test Execution estimation.

To summarise, the approach I suggest requires a high level of proactivity and interaction/collaboration  with the key users but this will definitely help you in achieving the desired result, i.e. the highest quality for your projects. 


Massimo Forno - IxmaSoft CEO