SEKE 2012 Proceedings - Knowledge Systems Institute

has shown that performance of the SAN model is very
close to that of the rejection rate reported by SoapUI.
III. RESULTS
Sixteen tests have been conducted on the banking web
service. Each test is run five times and the average of
extracted data is considered as experimental data. Table II
shows a sample data extracted from SoapUI. For each of
the 16 different loads, avg and tps values returned by
SoapUI are used in the SAN model (see Table I) and (5).
Table III shows the results for the 16 tests obtained from
SoapUI, the theoretical equation (5), and from running the
SAN model. As it is shown in the table, the worst
rejection rate happens when there are 380 users, which is
0.13 – 0.08 = 0.05, and 0.14 – 0.08 = 0.06 for the
theoretical and the SAN model, respectively.
Users
(threads
)
TABLE II.
200 10
350 10
352 10
354 10
360 10
370 10
380 10
TABLE III.
Number of
simultaneous
users
Runs
per
threa
d
SAMPLE DATA EXTRACTED FROM SOAPUI
Avg
Cnt (sec)
200
0 7.46
350 13.7
0 6
352 13.8
0 3
354 13.6
0 9
360 14.5
0 4
370 15.1
0 1
380 13.7
0 1
Numbe
r of
request
s
rejected
Total
time
Tps (sec)
24.6
8 82.55 0
22.9 153.2
2 5 0 0
22.2 158.2
5 1 0.5 0
22.6 156.5
1 8 5 0.001
22.1 162.6
5 7 24.2 0.006
21.7 170.4
8 6 185.4 0.05
24.1
3
Request
rejectio
n rate
157.6
4 332 0.08
REQUEST REJECTION RATE FOR ALL SIXTEEN TESTS
Request
rejection rate
(SoapUI)
Request
rejection rate
(rejection est /cnt)
Request
rejection rate
(rejection SRN
/cnt)
200 0 0 0.01
350 0 0 0.04
352 0 0 0.03
354 0.001 0 0.03
360 0.006 0.02 0.04
370 0.05 0.04 0.05
380 0.08 0.13 0.14
400 0.15 0.17 0.17
500 0.4 0.4 0.4
600 0.52 0.5 0.49
650 0.54 0.52 0.52
700 0.51 0.52 0.52
800 0.5 0.54 0.53
1000 0.54 0.51 0.51
1200 0.55 0.52 0.52
1500 0.51 0.5 0.49
Fig. 5 displays graphically the request rejection rate
for the three different models shown in Table III. The
figure shows that the rejection rate of the analytical and
the SAN models closely match the ones provided by
SoapUI. Another interesting result from the experimental
analysis is the throughput. Throughput is computed by:
(6)
Fig. 6 shows the throughput as a function of users. As
expected, the throughput decreases as the users are
increased. The tps range reported by SoapUI is from 21 to
45 for an average of about 33, with a standard deviation of
about 10. T he average throughput is about 22.5, which
means on average, 30% of the requests are rejected.
Rejection rate
Rate of web services not serviced
Throughput
0.6
0.5
0.4
0.3
0.2
0.1
0
30
25
20
15
10
5
0
Figure 6.
0.6
0.5
0.4
0.3
0.2
0.1
0
Figure 7.
200 352 360 380 500 650 800 1200
Number of virtual users
Figure 5. HTTP rejection rate.
Experimental model
Analytical model
200
350
352
354
360
370
380
400
500
600
650
700
800
1000
1200
1500
0
Number of simultanous users
Throughput based on virtual users (extracted
from soapUI).
Failure rate
Total unsuccessful rate (HTTP rejections + failures)
HTTP rejection rate
0.01
0.04
0.08
0.12
0.16
0.2
0.24
0.28
0.32
0.36
0.4
0.44
0.48
0.52
Web service failure probability
Web services rate not serviced using SAN model.
The close approximate behavior of the SAN model,
as shown in Fig. 5, allo ws for cases th at are more
difficult to rep licate in real world, e.g. inducing failures
in the application server (TJW activity in Fig. 4). As an
example, consider Fig. 7 that shows the rate o f failures
due to HT TP rej ections and web service failures. This
309