Agile Performance Testing - Testing Experience

Mobile Testing - A Toolkit for Advanced Testing -
The Techie, The Simple and The Obvious
The field of mobile1 testing poses unique challenges to the advanced
software tester given its diversity and fast change pace.
It is a world of many devices, ever changing operating systems,
APIs, software components and screen resolutions as well as evolving
customer demands, device designs and mobile network carriers.
These characteristics of the Mobile software world aggravate
the regular challenges faced by software testers. Testing has to
be performed in a small window of time - between the tight and
much abused deadlines of the coders and the short planned time
to market of the marketing department. In order to cope with the
high levels of uncertainty and to perform our work with high certainty
(covering the important things) and deliver reproducible
results, we must establish a stable testing platform with which
to navigate through the stormy waters of new mobile platforms,
application releases and tight schedules
While testing world-leading mobile applications in our work
places, our teams have developed over the years (since 2000)
various techniques which allow us to cope with short release
timelines, while only minimally compromising the continuous
quality improvement for which the our products are known. It is
important to remember that ‚testing technology‘ isn‘t just about
the hardware/software technology used, but also about the techniques
of the testing, i.e. the methodology and the way things are
done.
In this article we aim to present a range of useful techniques
(though not a comprehensive one) we have gathered over the years
in our SQA teams. We hope to offer a simple toolkit that will
provide a jumpstart to making the life of mobile testers easier
and help focus on the important things - the “bugs’ hideouts” -
which lay behind the platforms’ built-in impediments.
The Techie - Pure technique tools
Simulating real-world scenarios by means of configurable data
manipulation.
The communication protocols connecting clients with servers are
routed to a proxy mediator in order to simulate various server responses
to the mobile device.
A growing portion of mobile applications aims to provide information
to customers. Two kinds of information are usually provided:
static information hard-coded into the client or online
1 Phones, PDAs and Smartphones
by Dario Uriel Estrin & Bernard Shai Lelchuk
© iStockphoto.com/shironosov
updated information. 2The backend systems provide updated information
upon request. This information changes as a function
of various parameters – reality-related and user- related. Testers
can set the user-related parameters, check the reality- related
ones, and see whether the resulting information/application
behavior is correct. However, such methodology impedes them
from testing all possible results as reality is reality, i.e. chaotic and
unpredictable. For instance, testing an instant messaging mobile
client will require testing interactions with other users, server responses,
network problems, etc. The best way we came up with to
cover the system-centric scenarios was ‘controlling the world’, i.e.
simulating server responses according to real- world input/triggers/responses.
Simulating a server’s response for a given request
(response which is for itself a function of responses provided by
another system) allows us to isolate the mobile client behavior
from real- world input and to focus on inter-system testing.
In our case the response to the mobile application was sent via a
comma- delimited data stream from one of the servers. The testing
team learned how the stream looks like and behaves. Later,
we built a testing server environment which generates responses
according to variables set by the tester (this could be done by manually
manipulating a text file with values or getting an HTML
page to pick values from and which creates the file for the tester).
This resulted in a valid comma-delimited response that was fed
to the mobile client to be tested. This allowed us full control of
the data that in its turn triggered the various logical paths of the
application. Being able to easily simulate any possible scenario
helped discovering flaws in the system, which may not have been
discovered, if we had to count on the uncontrolled real-world
data that is the input to the servers and application.
Technically, we routed the device to a ‘server’ that was getting the
status from our testing environment:
1. We configured the device for data connections via a PC machine
or used a device simulator on a PC machine. 3 This allowed
us to control the routing path of data networking.
2. We manipulated the address of the servers which the client
was looking for and routed it to the testing environment
described above. This was done in the Windows ‚hosts‘ file
(usually located in the ‚C:\WINDOWS\system32\drivers\etc‘
2 Static information on client or server consists of two types of
data. One which is totally static and never changes (e.g. the number
of players in a soccer team), and the other which is rarely updated by
an external source (e.g. the number of teams in the soccer league).
3 In which case the connections are obviously routed via the PC
too.
10 The Magazine for Professional Testers www.testingexperience.com