Once your Test Plan
has been structured and completed with all the relevant and consolidated information,
based on the requirements gathering and specification process, the real
creative work can begin, i.e. thinking about and designing the Test Cases that
will be executed on the software system under examination
.
Although this phase
might seem very boring, especially when it is not undertaken with the right
spirit, it represents one of the most important of the test process, as only
through a proper Test Case design is it
possible to achieve maximum test coverage.
There are of course
several approaches and techniques, but before describing those that have
provided me with a high return on investment, I would like to explore briefly
the most used by the testing community. Obviously we need to distinguish
between the different scenarios like a new project, a redesign of an existing
one and last but not least a project under maintenance which can be subject to
bug-fixing and/or small enhancements.
In the case of new
projects, life should be straightforward, as in theory there should be enough
time to do everything well. However this rarely happens because when there is too much time available
for thinking, people tend to spend it on planning and discussing what might be
the right approach, which test types and levels have to be forecasted. They
then realise too late that what was
planned cannot be put in practise due to a shortage of time.
However, if the
available time is used properly, a
proactive approach is usually the best one because it is possible to
concentrate on the test design effort before any code has been written. Then
refining the test cases as soon as the first version of the software is ready
for testing (even though I strongly recommended getting access to review the
implementation during development) and at the same time giving the developers
the chance to verify that they have taken into account all the conditions
stated for each test case together with the expected output, will result in a
higher quality release that will be put through formal testing at the end of
the development phase.
Obviously this
approach is not only proactive but also iterative, as it is developed over the
project life cycle rather than either upstream or downstream of the coding
phase; but of course the availability and cooperation of the development team
is vital.
In such a case the
Test Design techniques I would suggest, also to prepare test driven scenarios,
are the Equivalent Class Partitioning together with Boundary Value Analysis and
a further refinement using Decision Tables. In this way the Test Cases will
ensure most conditions are covered, but this is still not enough, as the
objective is to cover all the business processes and the possible scenarios as
well. To do that a good approach would be to use Mind Maps, which allows us to
draw what test/business analysts have in their heads, and since it is usually
based both on the experience in the testing field (which is very important) and
on the business domain (having correctly understood all the functionalities and
their interactions) it gives us a clear picture of the system behaviour.
An additional benefit
is provided through the ease of translating such a representation into the Test
Management platform, as the structure of the Test Scenarios is already prepared
– The Mind Map –.
The contrary approach,
i.e. a reactive one, would normally be
adopted for projects under maintenance. This is because on the one hand it is
not possible to determine when and where software failures will occur, on the
other hand for small enhancements (which does not merit a new project) there is
not usually enough time available to follow a proper test process. In this case
the existing test cases are typically updated to take the changes into account.
A more complex
sub-scenario must be faced when no Test Cases are available for the existing
functionalities (which is very common for projects developed when the
organisation was not yet well organised from the testing perspective), one
which requires a very fast design process in order to react quickly to changes.
In such cases the best
approach is to start creating the specific test cases to test the
functionality/feature subject to changes and incrementally add cases every time
new defects/small enhancements occur. But of course this requires more
resources as the resources that have already been allocated will be busy with
test execution.
To summarise, for
existing projects the test design is mostly based on the experience of the test
personnel involved and will usually require additional testing resources.
For redesign projects
the situation is slightly different, because even though the existing Test
Cases, if any, can be properly updated as the project plan should take into
account the whole Test Process, forecasting the right time for Test Case Design
and validation, can become more complex if the system behaviour is likely to
differ heavily from the existing one. In this case the approach mentioned above
for band new projects can provide some help.
If this is not the
case the advantage is represented by the possibility of doing Exploratory
Testing as the actual implementation is available both for investigation and to
get familiar with the processes; moreover the key users feedback and
suggestions can be exploited to better design effective Test Cases.
I have briefly
explored the most common approaches, now I would like to describe the approach
I typically apply and that for recent projects has allowed me to maximize the
return on investment, a topic that is very important for organisations.
I have not reinvented the
wheel, as it is a combination of the approaches mentioned previously.
Basically I always
follow an iterative approach, no matter whether or not I am working for a
brand-new project or for an existing one. My Test Strategy and the consequent
Test Design approach, is essentially the same with slight adjustments in order
to fit with the organisation standards, methodologies and techniques.
My first requirement
is to have access to the entire set of project documentation, both business and
technical. Then, as soon as I can get
all the answers/explanations I need, I organise some sessions with key users
and specifically:
- For a new project I meet directly with them in order to understand their vision (especially from the usability perspective) and how they would like to interact with the new application, which functionalities represent the core and which are desired but very important to maximise their effectiveness in performing operational duties
- For an existing project redesign/under maintenance it is even better as I am typically in a position to observe how they work (basically doing exploratory testing together) and to understand if the application is forcing users in to unnatural usage patterns and/or if there is space for improvement. I usually spend a lot time with them to be able to quickly report back to the project manager and verify if the feedback/learnings can be considered
Spending time with the
key users/subject matter experts/final users will bring you many benefits that
are hard to imagine. Moreover they really appreciate this kind of
involvement/contribution and this makes them more willing and open to changes, a subject to which there
is always a strong resistance.
Where a project has a
prototype or set of navigable wireframes that is available before the
implementation commences (i.e. during
the functional analysis phase) it is a good practice to organise some Usability
Testing sessions, as doing so makes it possible to anticipate problems which if
missed could result in expensive rework if the key users are only able to look
at the application immediately after development has been completed (this is
true even if an Agile approach is being used, as also in case of intermediate
validations – UATs – carried out at the end of each sprint, if the key users do
not have the possibility to see a prototype before development it could result
in changes being needed in order to provide the desired behaviour).
When the working
session and the Usability Testing are performed together with the key users, it
allows the Test Cases to be better organises and designed. This is because by
focussing on the functionality and interaction and how the final users will use
them, allows us to prepare business oriented Test Cases and to think of all the
possible related cases in advance.
Once the skeleton of
the Test Cases has been prepared, it is now time to apply the techniques
mentioned above, i.e. Equivalent Class Partitioning with Boundary Value
Analysis and Decision Tables (there are many other techniques but these three
are the techniques I prefer and those that can be automatically implemented
once experience has been acquired).
At this stage things
are almost complete but our Test Cases need further refinement; how can we do
that? Easy, we need to push developers to show the units, modules and user
stories that have been implemented. This is an activity that needs to be
carried out over the entire length of the project, by doing this we do not need
to wait until the end of the development/sprint to see the actual
implementation with our own eyes and we refine the Test Cases incrementally
during the project life cycle. An additional benefit of this approach is having
the possibility to anticipate the Test Execution as well, removing the main
issues that could result in a costly rework if undertaken downstream of the
development phase.
Last but not least, by
following such approach it is possible, both in case of manual and automated
testing, to estimate the Test Execution time and to add it as a reference for each
Test Case. This provides valuable information for the Project and Test
Managers, who can easily adjust their plans based on the Test Execution
estimation.
To summarise, the approach I suggest requires a high level of proactivity and interaction/collaboration with the key users but this will definitely help you in achieving the desired result, i.e. the highest quality for your projects.
Massimo Forno - IxmaSoft CEO
