Quality Evaluation of Web-sites of Indian E-commerce Domain

Quality Evaluation of Web-sites of Indian
E-commerce Domain
Priyanka Tripathi, Namita Shrivastava and M Kumar
Department of Computer Applications
Maulana Azad National Institute of Technology , Bhopal, INDIA
Abstract-This paper describes the development
in the area of quality evaluation models for
web-applications. The authors have proposed a
design of quality evaluation method, it’s
implementation and partial results in order to
evaluate the Indian e-commerce web-sites. We
have considered four quality factors namely
usability, functionality, reliability and
efficiency to evaluate the web-site to measure
the quality preference outcome. Further, by
combining the outcome of each factor, we find
out the Global Quality Preference of the website.
We have used Logic Scoring Preferences
(LSP) grounded on Continuous Preference
Logic as mathematical background.
Key-words: Quality, Evaluation, E-commerce.
1 Introduction
E-commerce is an excellent alternative
for companies to get new customers fast in
todays commercial world. The success of these
e-stores depends heavily on their quality in
terms of quality of service given by them. To
evaluate the quality of e-commerce based websites
we need to identify the attributes which
contribute to the quality. Quality is an intrinsic
and multifaceted characteristics of a product[1].
Quality is not an absolute , it depends on the
appraiser’s perspective. Hence any quality
measure must be subjective , summarizing the
impression of a given class of individuals that
interact with the product[2]. The main purpose
of software quality evaluation is to supply
referential quantitative results to the software
products that are reliable, understandable and
acceptable to anyone’s interest [3].
Identification of the requirements of what you
should built is the hardest part of the design
process. This leads to the design of excellent
web shops from average ones[4]. Because of the
increasing size, complexity, quality needs and
market demands for web-applications, several
problems have frequently been reported [5].The
World Wide Web is a universal information
space overcoming barriers created by humans
towards people with different cultures or
physical limitations [6].Evaluating the quality
of a web site requires expensive methods such
as heuristic evaluations or empirical usability
tests. In the first case a group of specialists or
expert evaluators apply their experience to
conduct independent evaluations and usually it
does not permit to find problems related to
typical users of the site. In the second case a
group of users with different background, age,
and skills characteristics are called to browse
the web site in order to evaluate their
satisfaction in using it. Lack of measurement
criteria to evaluate aspects related to the quality
in use, such as usability and accessibility of a
web application is giving a significant rise in
number of poor quality web-application.
2 Literature Survey
Different approaches are defined for
web-site quality evaluation. An important
initiative towards web-site quality in cultural
environment is MINERVA[7] (Ministerial
NEtwoRk for Valorising Activities in
Digitisation). The quality criteria have therefore
a double objective, they represent the quality
factors for evaluating the quality of a cultural
site on the web; they direct and support the
process of design and development of a cultural
website. Mich et. al [8] presents very flexible
approach to evaluate a generic web site. The
2QCV3Q, also called 7-loci, is a conceptual
model to evaluate web site quality based on
seven dimensions: who-what-why-when-where-

how, and feasibility.A metamodel called
2QCV3Q (in Latin V stands for U),which takes
its name from the initials of the Ciceronian loci
of classical rhetoric that it is based on [9]. Also
called 7Loci, the meta-model takes into special
consideration the inherently communicative
nature of a Web site [8,10].This concept is used
to identify the fundamental dimensions of a
Web site, resulting in a framework that when
compared with existing models can be seen as a
meta-model for classification of diverse criteria
for quality (table 1). In other words, the seven
loci or dimensions constitute the general
framework of the “quality models”, which is
independent of the sites under analysis.
1. The first dimension, Identity, regards
the image that the organisation
projects and therefore all elements that
come together in defining the identity
of the owner of the site.
2. Content to the information
3. Services refer to the services available
for users.
4. Location regards the visibility of a
site; it also refers to the ability of the
site to offer a space where users can
communicate with each other and with
the organisation.
5. Maintenance comprises all activities
that guarantee proper functioning and
operability of the site.
6. Usability determines how efficiently
and effectively the site’s content and
services are made available to the user.
7. Feasibility includes all aspects related
to project management.
Table 1. Dimensions of the 2QCV3Q -
7Loci meta-model
QVIS(Who?) Identity
QVID(What?) Content
CVR(Why?) Services
VBI(Where?) Location
QVANDO(When?) Maintenance
QVOMODO(How?) Usability
QVIBUS AVXILIIS(With Feasibility
what means?)
ISO9126 , ISO9241 AND ISO13407 [3,11,12]
are very popular software quality
standards.ISO9126 , ISO9241 AND ISO13407
describe the standards for quality of software,
usability, user-centered product. Conformance
to standard is also the basis of W3C quality
assurance initiative [13]. The 7Loci meta-model
supports a systemic approach to evaluating web
site quality that takes into account these diverse
components coming together at a site and the
importance of satisfying the needs of all actors.
A usability-focused evaluation method for
hypermedia application is MiLE, based on a
combination of inspection from expert evaluator
and empirical testing through panels of end
users [14]. The evaluation model here is based
on two heuristic concepts: abstract and concrete
tasks. An analytic web site quality model is
proposed by Etnoteam [15].It is based on six
attributes :communication, content,
functionality, usability, management and
accessibility. The model can be
personalized.The sub-attributes are weighted
depending on the site category. An evolution of
this model is described in [16]. Another
analytical approach described in [17] proposes a
Web-site Quality Evaluation Method based on a
logic multi-attribute decision model and
procedures intended to be a useful tool to
evaluate artifact quality in the operational
phase.. Studies for quality of products and
processes for the web are very recent and still
there are no widely spread evaluation methods
and techniques for quality assessment.
Fleming 2001[18],Mendes et al. 2001[19],
Olsina and Rossi 2001[20], Pooley et
al.2002[21] have given the quality attributes
and also some of them have evaluated the
quality of different web-application domains.
The ISO/IEC 9126 series standard introduced a
hierarchical model with six major quality
characteristics, each very broad in nature .
They are subdivided into 27 sub-characteristics,
which contribute to external quality, and 21
sub-characteristics which contribute to internal
quality. Since the characteristics and subcharacteristics
are not properly defined,
ISO9126 does not provide a conceptual
framework within which comparable
measurements may be made by different parties
with different views of software quality, such as
users, developers and managers. The Authors
have given a Quality Evaluation Framework in
order to evaluate the quality of Indian ecommerce
based web-sites in Section 3. In our
previous research, we have validated factors and
sub-factors from user’s point of view [25] and

given quality metrics for e-commerce web-sites .
3 Designing the quality
evaluation frame work
A general model of the evaluation framework is
given in figure 2. It shows an initial phase
which includes the identification of the
evaluation requirements, a design phase in
which the evaluation plan and techniques
are defined, and a final ranking as per the
quality and suggestion phase.
I. Quality requirements specification:
What is the mission of the web-application.
Specify the functional Usability, Functionality
and non-functional quality requirements such
as Reliability, Efficiency of the webapplications.
What is the priority associated
with each of the factors. Identify the category of
users. web-application users needs vary
according to the category of users. Outcome of
Figure 1. ISO 9126 Model For Quality
this phase is evaluation goals. In designing a
web-application developer must see other webapplications
having similar objectives. The
reason for this is because the users always
compare a web application with the other
existing applications and uses the most
convenient or in other words web-application
which has a greater Usability.
II. Metrics selection: Select the metrics,
their corresponding weights the scale of
measurement, and for summation. Outcome
of this phase is evaluation plans.
III. Measurement: Measurement
implementation and result calculation.
IV. Result: Ranking of web-applications as
per the Global Quality Preference. Suggestions
for the evaluated web-application.

Figure 2: Design of the Quality Evaluation Framework
Figure 4. Homepage of http://www.ebay.in

URL of the
web-site
Figure 3. Global aggregation of preferences of Quality
Table2. Quality preference of factors of http://www.ebay.in
Usability
%
Functionality
%
Reliability
%
http://www.ebay.in 75.6 87.23 91 93.3
Global Quality
Preference
85.16%
4 Implementing the quality
evaluation framework
The evaluation process is done by using
small, controlled and well-planned experiments
[22] , used the students of MCA and also other
software professionals working with the
companies of international repute. They have
been given a list of on-line shopping Indian
web-sites and were provided the evaluation
scale for each metric. After evaluating the data
of selected web-sites of Indian e-commerce
domain, an evaluation model has been selected.
We have selected non-linear multi-scoring
criteria model. The evaluation process is based
on Logic Score of Preferences LSP[23,24] and
Continuous Preference Logic as mathematical
background. This can be achieved by means of a
preference aggregation function, called
generalized conjunction /disjunction or andor,
combining weighted power means to obtain the
Global Quality Preference. After evaluating the
factors of quality of the selected web-sites
Global Quality Preference is being calculated.
Efficiency
%
5 Results and discussion
In this work, we have presented the design
and implementation of a systematic and
quantitative engineering-based evaluation
method for web-applications. The block
diagram given in Figure 3 shows the quality
factors along with their weights in Global
Quality Preference of the web-site. It is expected
that organizations need to define their own
quality factors, criteria and metrics specific to
their system context. In future, we suggest
creation of a knowledge base of web-based
software’s quality factors, criteria, and metrics.
Software professionals can then make use of
already defined model that suits them or find
the closest model and modify it according to
their needs. We have evaluated the web-site of
ebay http://www.ebay.in and the results are
shown in Table 2. We further wish to evaluate
other selected Indian e-commerce based websites
to calculate their Global Quality
Preference, in order to rank these web-sites
according to their quality.

6 References
[1.] Tausworthe R. C., “Software quality
management through process and
product modeling” , Annals of
Software Engineering 1, 119-139,
1995.
[2.] Gentleman W. M., “Is software quality
a perception, how do we measure it?” ,
The 6th I.C. on Software Quality,
Ottawa, October 1996.
[3.] ISO/IEC FDIS 9126-1, Software
engineering - Product quality -Part 1:
Quality model,2000.
http://www.usabilitynet.org/tools/r_inte
rnational.htm - 9126-1
[4.] Summers K., “Identifying Web Site
Requirements”, 2001.
http://www.intercom.com/
[5.] Cutter Consortium. 2000. Poor Project
Management, Problem of E-Projects,
October2000.
http://www.cutter.com/consortium/pres
s/001019.html
[6.] Tim Berners-Lee,” Weaving the Web:
The Original Design and Ultimate
Destiny of the World Wide Web by Its
Inventor”, HarperSanFrancisco, ISBN
0-06-251587-X, 1999.
[7.] MInisterial NEtwoRk for Valorising
Activities in digitations, project web
site: http://www.minervaeurope.org/
[8.] Mich L., Franch M., Gaio L.,
“Evaluating and Designing the Quality
of Web Sites”, IEEE Multimedia,
pp.34-43, Jan-Mar, 2003.
[9.] Cicero M.T., “De Inventione”, 58 BC,
in De Inventione, De Optimo Genere
Oratorum, Topica, Vol. 2, Rhetorical
Treatises E.H. Warmington (ed.), H.M.
Hubbell (translator) Harvard University
Press.
[10.] Mich L., Franch M., “2QCV2Q: A
Model for Web Sites Analysis and
Evaluation”, Khosrowpour M. (ed),
Proc. IRMA, Anchorage, Alaska,
IDEA, Hershey, PA, pp. 586-589, May
21-24, 2000.
[11.] ISO 9241-11 Guidance on usability
1997.
http://www.usabilitynet.org/tools/r_inte
rnation
[12.] ISO 13407 Human centered design
processes for interactive systems 1999.
http://www.usabilitynet.org/tools//r_int
ernational.htm#13407
[13.] W3C Quality Assurance,
http://www.w3.org/QA/;
[14.] N. Di Blas, “Evaluating The Features
Of Museum Websites: The Bologna
Report”, Museums and the Web 2002.
[15.] Etnoteam S.p.A., "Un modello di
qualità per I siti web", Università degli
Studi di Milano, AA 1995-96.
http://www.etnoteam.it/webquality);
[16.] Polillo, Roberto, Il check-up dei siti
Web – Valutare la qualità per
migliorarla, APOGEO ISBN 88-503-
2282-8, 2004.
[17.] Olsina L. , Lafuente G.J. and Rossi G.,
“E-Commerce Site Evaluation: A Case
Study,” Lecture Notes in Computer
Science 1875,Proc.1 st International
Conf. Electronic Commerce and Web
Technologies, EC-Web 2000.
[18.] Fleming, J. Web Navigation:
Designing the User Experience,
O’Reilly & Associates,2001.
[19.] Mendes, E., Mosley, N. And Counsell
S.,”Web Metrics–Estimating Design
and Authoring Effort”,IEEE
Multimedia, pp 50-57, January-March
2001.
[20.] Olsina L., Rossi G., “Measuring Web
Application Quality with WebQEM”,
IEEE Multimedia, pp. 20-29, October
-December 2002.
[21.] Pooley R., Senior D., Christie D.
Collecting and Analyzing Web-Based
Project Metrics. IEEE Software 19 (1),
pp 52-58 January/February 2002.
[22.] Mendes E. , “A Systematic Review of
Web Engineering Research”, 4 th
International symposium on Empirical
Software Engineering ISESE, Noosa
Heads, Australia, In proc. of
IEEE,17-18 Nov 2005.
[23.] Dujmovic J.J.,”A Method for
Evaluation and Selection of Complex
Hardware and Software Systems”.
Procs. 22nd International Conference
for the Resource Management and

Performance Evaluation of Enterprise
Computer Systems, Turnersville, New
Jersey, 1996.
[24.] Dujmovic J.J., “Continuous Preference
Logic for System Evaluation”, In
Proceedings of Eurofuse 2005, edited
by B. De Baets, J. Fodor, and D.
Radojevic, ISBN 86-7172-022-5,
Institute “MihajloPupin”, Belgrade,
2005, pp. 56-80.
[25.] P. Tripathi, M. Kumar, “Some
Observations on Quality Model for
Web-Applications”, International
Conference on Web-applications,
ICWA06, In proc. of Advanced
Research Series by Macmillan,
Bhubaneswar, India, December 23-24,
2006 , pp 151-158 .
***