Exposure to Exploratory Testing - QAI

Exposure to Exploratory Testing
Gaurav Luthra
Archana Mantripragada
Bala Kumaran Anbu Raja

Abstract
• Exploratory Testing (ET), useful for discovering new information can be defined as learning simultaneously while planning
and execution.
• Exploratory Testing is not a testing technique, but it is the thought that can be applied on any technique. This type of testing
is performed under strict timelines with minimal resource usage. It requires highly skillful and experienced testers with good
working knowledge on the domain and technology.
• Test Case Based (TCB) is a script-based approach focusing on a predefined test case design technique.
• In exploratory testing, a skilled tester will cover all the related aspects of any ad-hoc changes/requirements.
• ET helps a No/Ad-hoc requirement project where an Exploratory Tester can add his thought to the requirement if beneficiary
to the Project. It also benefits the short term and fixed bid projects.
• Thus in short, with minimum time and effort, tester can maximize the ability to find more critical defects in an efficient way.
• In this paper we will discuss and suggest strategies using ET to overcome the challenges/hindrances faced in TCB and non-
Requirement based project.
• Test Managers, Test Leads and BAs will benefit from this paper.
BFS-US SSDU Research
2

Test Case Based Approach Testing
• Script Driven Testing follows Analyzing-Planning-Implementing-Reporting approach.
• BRD, LLD and HLD are taken as the source documents and Test plan is prepared.
• Test plan is followed by Test case preparation and Execution covering all the functionalities.
• In TCB, application is not explored and the chances of missing important aspects are high.
• Requirements are covered as per the LLD/HLD given and the testing team follows the same

What is ET ?
Learn
• ET is defined as Interactive process of simultaneous
planning, learning, running and reporting.
• Tester explores the product to fulfil the testing mission
Exploratory
Analysis
Testing
and report the results.
Test
Design
Test
Execution
BFS-US SSDU Research
4

What is ET ? Contd..
Factors affecting ET
• Product specifications
• Tester’s knowledge on the product
• Testing Strategy
• Available Test data and the documents
• Requirements.
Basics needs to be taken care of by the Exploratory tester:
• Careful Observation
• Critical Thinking
• Diverse ideas
• Rich Resources
BFS-US SSDU Research
5

Metrics of ET
Session-based test management (SBTM)
• Session-based test management (SBTM) is for managing ET.
• This technique introduces the idea of a session charters for measures and manages those test ideas. In
these Session Charters, some set of expectations like what kind of work will be completed and how it will
be reported.
• When that session ends, the tester prepares a session report, tagged with important information about
what they did. Each session is characteristically 45 minutes to 60 minutes occurs.
Anatomy of a Test Session
• These are the status of the ET process. Essentials of Session metrics are Number of sessions
completed, Number of problems found, Function areas covered, and Percentage of session time spent
setting up for testing, Percentage of session time spent testing, and Percentage of session time spent
investigating problems.
SBTM_Sample
Charter_sample1.html
BFS-US SSDU Research

Metrics of ET
Flow Chart
• It is an illustration which portrays a
procedure or the steps of an assignment
(project).
Add
Item
s
Update
Remove
• In ET, we can approximate scenario
activities with a flow chart
• Info seeking is a continuous procedure that
contains lot of steps; real interfaces will
provision changes among these steps with
minimum effort necessary of the user.
• The more unified the connections, the
more attention Testers can devote to the
scenario rather than to the search tool.
Items
added in
the cart.
At least
one item
must be
in cart.
Updating
on existing
items
The item
must be
in cart.
User deletes
the
particular
item
BFS-US SSDU Research

ET in Different Scenarios
• Illustrates about the application Testing of a shopping site say shopfree.com which details us how a defect can
be turned into a requirement.
Scenario Drawbacks of TCB ET
The site has a homepage and
a payment page which
performs the credential
verification and allows the
user to shop. Here the page
contains OK and CANCEL
buttons apart from the
credentials which helps us to
go ahead into the site and
cancel the application
respectively.
• As the LLD/HLD is followed by the test team,
some basic functionalities might get missed
which might turn into a serious defect.
• During the development of test scripts,
found difficulties like some ended obsolete,
deferred, because of changes in software or
hardware or firmware or incorrect
requirement or technical limitations or
schedule limitations.
• During Execution phase, as a round of 4-5
cycles is done on the same Test cases, Tester
might feel it boring performing the same
tests with occurrence of very less bugs
when compared to the initial cycles.
• Exploratory tester covers all the
aspects of the Homepage and the
payments page.
• The chances of missing any defects
or instances are less as the
Exploratory tester interacts and
explores the application.
• In this context, Tester while testing
the application checks for the
navigation from payments to
homepage.
• As there is no specific link provided
in the page to navigate to the
homepage, this is considered as a
defect which is reported to the
development team which can be a
requirement.
BFS-US SSDU Research

ET in Different Scenarios
TCB
Test Report
Test
cases
TC
Preparation(
In mins)
TC
Exection(
In mins)
Total
Duration(In
mins)
Login_1 5 10 20 30 2
Login_2 5 10 20 30 1
Homepage_1 6 12 24 36 3
Homepage_2 7 14 28 42 1
Bugs
ET
Test Report
On-
Chart
er
Off-
Charte
r
Total
Duration(In
mins)
Login_1 32 0 32 2
Login_2 28 0 28 4
Homepage_1 32 1 33 3
Homepage_2 38 2 40 2
Bugs
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Login_1 Login_2 Homepage_1 Homepage_2
Duration(i
n TCB)
(in hours)
Duration(i
n ET)
(in hours)
3.5
2.5 3
1.5 2
0.5 1
0
Bugs(in TCB)
Bugs(in ET)
BFS-US SSDU Research

ET in Different Scenarios
Scenario Drawbacks of TCB ET
Consider an Ad-hoc requirement
based project where the
requirements come dynamically
on the fly on call or through mail.
Requirements here refer to rules
which have to be followed while
loading data into the table. To
check if the table is loaded in
accordance to the rules, queries
are run against table to check for
data validation and in case if any
error records found; such records
have to be moved to Error table.
Resources available for the above
scenario are 1 Test Lead and 2 Test
Engineers.
• Agile projects with dynamically
changing requirements impose great
difficulty for TCB in planning and
strategizing approaches.
• Defect detection rate is low as the
tester is unaware of the inner
functionality of the product.
• Incorporation of requirements in the
correct place is difficult.
• Requirements are documented as a
Flowchart to get a clear picture of
all the tests to be made.
• ET maintains minimum
documentation and mainly focus
on detecting more defects within
fixed timelines.
• Diverse testing techniques are
followed to carry out testing.
• Skilled Exploratory Testers follow
their own approach to extract the
output with zero or minimal
planning.
• Exploratory Testers conduct
brainstorming sessions with a
group to generate various ideas to
perform testing.
BFS-US SSDU Research

ET in Different Scenarios
Rules
Tester
Bug Report
Fail
Appro
ach
New
Requirements
Pass
Log report
BFS-US SSDU Research

ET in Different Scenarios
Scenario Drawbacks of TCB ET
Localization Web based project for the
financial institution which needs to
test its application in 6 different
languages. This has multiple releases
which will be tested in two different
environments SIT and UAT. Timelines
are reduced from 4 months to 2
months due to reduced scope.
Available Resources at the customer
site is a team of 6 which includes a
Test lead other all are tester with 2-3
years of testing experience
• Possibility of Pending task at the
end of timeliness is high.
• High possibility that some
requirements are missed while
preparing the test strategy and the
test cases.
• Ambiguity in reporting the defect
as the no of languages tested is
high.
• With tester handling individual
language there is high possibility of
a critical defect getting missed
which may be also be a defect in
the other language.
• Test lead prepares some charter for
the testing.
• The team will split up in teams or
as individuals and start testing the
application covering all the
requirements.
• Approach is taken down on an
excel or a flowchart which might
help as future reference.
• Helps cover all the requirements
with proper defect report.
BFS-US SSDU Research

Advantages of ET
• ET provides complete coverage of the application and accountability.
• ET is structured approach while we’re using session based management testing.
• ET takes very less time compared to TCB which in turn saves clients revenue.
• Makes Regression testing easier as the Tester knows the inner functionality of the product which helps in
detecting the defects in less time.
• Skilled testers explore and interact with the product which covers every individual instance of the product.
• Chances of missing defects are quite low.
• Chances of requirements getting missed is very less as the tester here performs a complete test on the
application covering all the basic and critical requirements with his/her knowledge. Even if one tester misses
one it can be discussed with other team members/pair about the scenario.
• Tracking down the task by exploratory tester helps the team to run a regression run on it which will help the
team to complete the multiple releases and test the same application in multiple environments.
BFS-US SSDU Research

Future Scope
• ET Technique itself is new definition in the testing world. It’s a technique which gives tester a new platform or a
direction towards testing the product. Here the Tester applies his/her skills and performs the testing.
• In near future as the businesses are touching competing targets, this technique can be of great importance.
This is not a separate testing but a technique which can be applied in different type of testing. It can be used
Functional, Automation, Data warehouse, performance, etc type of testing.
BFS-US SSDU Research

Conclusion
• Exploratory Testing is described as simultaneous test planning and execution is a new technique which has
helped the Test managers or Business analyst to present a better and challenging approach to the testing
practice in software Industry.
• In our paper using the ET technique we are able to tackle the various different types of projects having different
type of hurdles with the ease if the testers are skilled and have a will to explore.
• With this technique we are in position to say that one can decrease the time, work and other resources used in
completing the project when compared with the other strategy. Also the other great achievement is that
quality of testing provided is much better and the defect detecting efficiency is also increased for which testers
are the in the scene of SDLC.
BFS-US SSDU Research

References
• Project Experience
• Exploratory-testing.pdf – By James Bach
• Misconceptions_About_Exploratory_Testing.pdf – By Rony Wolfinzon and Ayal Zylberman
• Tutorial on Exploratory Testing – By Cem Kaner
• http://www.satisfice.com/
• www.Google.com
• www.Youtube.com (Talk on Exploratory Testing-by John Bach)
BFS-US SSDU Research

Q&A

Thank You !