Agile Performance Testing - Testing Experience

Code example:
• End-to-end functionality via data integrity testing.
There are two major cases in which we would like to trace the
information flow through its whole life cycle.
In the first testing phases of a new product, we would like to confirm
the data integrity in order to confirm that the basic product
infrastructures work before moving forward to testing the product
at a higher level. Later on, during the testing, when encountering
unexplained problems, we would also like to trace the data
from its inception (e.g. input by user) to the processor (e.g. server)
and back to the client. In both cases, we learned to appreciate the
usefulness of various “sniffing” and data interception tools.
In order to confirm that the basic product’s infrastructure works
correctly, we used a simple, yet straight-forward sniffing tool (e.g.
‘WireShark’) 5 . We verified the data integrity on the server, while
it was being passed to the client, on the client and vice versa. The
same was performed for authentication requests and their respective
responses passed between the clients and the servers.
At a later stage, when encountering a problem, we monitored all
the incoming/outgoing client/server requests/responses, in order
to trace the complete information flow.
The testing was divided into small functionality & data validity
chunks:
• Initiate a client request
◦ Client sends to server initial values for authentication
such as product identifiers and authentication data.
• Verify server’s initial response
◦ Server validates client’s initial authentication request &
returns respective response:
◦ Authentication failure: Error code & message returns
to client & connection ends.
◦ Authentication success: Positive response (e.g. OK)
returns to client & connection continues.
• Verify client additional requests
◦ Client now sends additional requests to the server - either
together or one at a time
• Verify server’s final response
◦ Server returns a response to the client per each sent request.
Certain modules of the systems communicated via XML for easier
5 http://www.wireshark.org
cross- platform coding. Remember we are talking about the mobile
world, where there is a plethora of different platforms. This
leads us to a deeper level of data validation tests, which include
thorough XML testing of client/server responses upon various requests
& data changes.
As data changes can originate either from a client or a server,
root-cause tracing for problems is done by intercepting the communication
between them. The intercepted data is later analyzed
by the testing team at the data integrity level as well as the XML
validity. Mobile applications create a lot of data traffic, which
presents an additional challenge – the tester must know the application
well enough in order to know where to look for the problem
and not spend precious time trying to sort out were exactly in
the data stream he should be looking. On the other hand, when
a new feature or communication-producing feature is added, the
best thing to ensure a bullet-proof infrastructure is to intrusively
and thoroughly test the added data exchange and not only a
small chunk of it.
For more advanced data sniffing and interception, we used tools
such as BurpSuite. 6 Its set of tools gave us extended flexibility,
such as advanced data scanning and filtering, interception, inspection
and modification capabilities of all the traffic passing in
both directions from client to server and vice versa. These tools
proved to be useful as well in security testing, such as penetration,
SQL injection and various other vulnerability attacks.
The ‘Simple’ – Some simpler tools
As the iPhone rocked the market, mobile web applications finally
became part of the mainstream mobile applications market. In
our quest for tools that will support our functionally testing of
mobile web applications, we ran into Firefox’s extensions. A
combination of these enabled us to simulate mobile client browsers,
allow faster testing of mobile web applications and easier debugging
on a PC, while saving a substantial amount of time and
effort.
Using FireFox add-ons, such as Live HTTP Headers, XHTML Mobile
Profile & Modify Headers, we were able to simulate mobile client
browsers of any type (especially clients we do not have access to).
Testing using these tools proved to be time efficient, bugs were
discovered at early stages and fixed on-the-go, thus test coverage
was greater than we would have achieved with regular testing.
To improve the development process, we’ve introduced these
tools to the Development, Graphics Designers and Web Masters
teams which espoused it. Thus we gained a shift in product quality
and maturity due to preliminary testing (or unit testing) by
the teams using these productive tools. By doing so, we managed
to release products earlier than scheduled with higher maturity
and coverage levels.
One of the biggest pains of mobile testers is the number of deliverables
to be tested according to various platforms, sales channels,
customizations, etc. Externalizing relevant parameters allows
focusing on testing this externalization module once and
then later on being able to focus on the core functionalities of
the application without having to worry about the multiple release
packages. Examples of such externalized parameters could
be versions identifiers, paths of graphic images or even names to
be embedded into the applications.
The externalized parameters are placed in a file nearby the installation
file. The latter takes the relevant information on the fly,
thus eventually customizing the application to be installed. Once
this mechanism is proof-tested, it spares the need to test multiple
installers. Moreover, these parameters can at any time be
manually changed and tested by non R&D personnel (e.g. sales
and production). On many mobile platforms, the installation-reboot-test-uninstall-reboot
process is known to be extremely time
consuming, which emphasizes the advantage of this technique.
6 http://www.portswigger.net
12 The Magazine for Professional Testers www.testingexperience.com