Analysis and Testing of Ajax-based Single-page Web Applications

Empirical evaluation (Wohlin et al., 2000) is an effective way to asses the
validity of any new concept or method. As far as the proposed Spiar architectural
style (Chapter 2) is concerned, empirical evaluation is very challenging,
if not impossible. Therefore, we have taken an analytical approach in
evaluating Spiar by discussing the rationale behind the style and its scope
regarding different web architectures. We have conducted a controlled experiment
(Wohlin et al., 2000) in Chapter 4 and have empirically evaluated Retjax
(Chapter 3), Crawljax (Chapter 5), and Atusa (Chapter 6) by conducting case
studies (Yin, 2003; Kitchenham et al., 1995) and reporting the results. One
challenge in conducting the case studies was the lack of existing comparable
methods and tools to form a comparison baseline (Kitchenham et al., 2002)
for the results.
Another issue we faced revolved around finding industrial real-world cases
for the evaluation part, which can be seen as an external threat to validity.
Due to the relatively new nature of the technology, we were not able to find
as many industrial Ajax-based applications to conduct experiments on, as we
had wished for. Therefore, most of our case study subjects are open source
applications. Although the size of the chosen case subjects is relatively small,
they are representative of the type of web applications our methods are focused
on, i.e., standards-based single-page web applications. With respect to
reliability and replication, the open source nature of both the case subjects
and the tools makes the experiment and case studies fully reproducible.
As far as the internal validity is concerned, we minimized the chance of
development errors by including rigorous unit testing for all the implemented
tools. We have used various external libraries and third party components,
and while we did encounter problems in some of them, they do not affect
the validity of our results. Sections 4.7 and 6.10 provide a more detailed
discussion of the threats to validity.
Our emphasis in this thesis has been on ideal ultimate standards-based
‘single-page’ web interfaces. As we have seen in this work, single-page web
interfaces are great if the surrounding web technologies, such as crawlers and
browsers, also support them. We believe that for the time being, a hybrid
approach which unifies the benefits of both dynamic fine-grained single-page
and static large-grain multi-page approaches, is the best option for developing
web applications without having to face the negative side effects of the two
ends, i.e., issues with amnesic pages without any history, linking, bookmarking,
indexing by general search engines on the one hand, and low latency,
responsiveness and interactivity on the other hand.
It is worth mentioning that for closed (authenticated) web applications,
single-page interfaces are ideal since many of the issues mentioned above,
such as discoverability by search engines, do not play a role.
Our Spiar architectural style is deduced from single-page Ajax applications.
Whether such a style can support hybrid architectures needs further
exploration. Our crawling and testing methods should have no difficulties
in dealing with such hybrid settings, although here again, more experiments
need to be carried out.
160 7.3. Evaluation