Agile Performance Testing - Testing Experience

init_server
The init server job helps to clean up and restart the application
server (websphere, weblogic, jboss, tomcat). It was not easy to get
this one stable, as the ANT job that starts the server on the remote
node got killed once the parent Hudson job returned to server.
The solution here is to add an environment variable BUILD_
ID=dontKillMe to this job, so that child jobs stays alive.
The logic that defines which server needs to be started, and where
the server is located, is actually not visible here, but hidden in our
test framework. We just call the ANT target clean-start-server,
and the rest will be handled locally on the node. In future, we
want to integrate Cargo into our ANT task, as it allows for easy
server start and deployments.
Install
The install job installs the software, compiles the updated test
software, configures the client-server set-up and deploys the
server part on the application server. All these tasks have been
integrated into a single ANT job “ant testenvir”, so the Hudson job
just needs to call this ANT target, with some additional environment
specific ANT properties.
Runtest
Our runtest jobs do the real test execution. We use separate build
Test report
Image8: SVN repositories
Image9: Generate report
The result of the testing is sent by mail. The report shows failures
and errors per test suite, for each individual environment.
steps in one single job:
• On Windows we call a Windows batch script to map network
drives
• We call ANT target “setup-minimal-audittrail” to set up our
database
• We call ANT target “runsuite” to start our test
• We call ANT target “load-all” to load the results of the unit
tests into a database. We have some Java jobs that parse the
unit .xml files, store them into a database schema to facilitate
good comparison and reporting afterwards.
The ANT jobs above all require their own specific environment
settings, which we supply as a parameter in the ANT properties.
Each node will get its own environment setting.
Report
The Hudson job “report_reg_trunk” has a script execution build
step, that starts a Python script. The script reads test results from
the database, both for the current and for the previous version,
and writes output to the console. The job also has an additional
email configuration that creates a mail with the console output
and sends it to the QA department.
It also compares the results of this build against the results of
the previous build. Analysis of the report still takes time, but the
comparison helps.
30 The Magazine for Professional Testers www.testingexperience.com