29.11.2012 Views

2nd USENIX Conference on Web Application Development ...

2nd USENIX Conference on Web Application Development ...

2nd USENIX Conference on Web Application Development ...

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

5.4.1. Local vs remote users<br />

We observed the resp<strong>on</strong>se time for a trace of 25 emulated<br />

users injected with our replay tool from a machine<br />

<strong>on</strong> the same LAN as the server and from a machine <strong>on</strong><br />

Amaz<strong>on</strong> EC2 East coast data center. As expected, the<br />

latencies are much higher <strong>on</strong> the WAN with 149ms<br />

average vs 44ms <strong>on</strong> the LAN. However, we observe<br />

that the latency standard deviati<strong>on</strong> more than doubles<br />

for the WAN compared to the LAN.<br />

The CPU usage for the WAN injecti<strong>on</strong> has already been<br />

presented in Figure 4 with an average CPU load of<br />

54.8%. The CPU usage for the LAN experiment shows<br />

a highly varying CPU load but at a much lower 38.3%<br />

average. We attribute most of these differences to the<br />

increased c<strong>on</strong>necti<strong>on</strong> times that require more processing<br />

to perform flow c<strong>on</strong>trol <strong>on</strong> the c<strong>on</strong>necti<strong>on</strong>s, more c<strong>on</strong>text<br />

switches between l<strong>on</strong>ger lived sessi<strong>on</strong>s and more<br />

memory pressure to maintain more sessi<strong>on</strong> states and<br />

descriptors simultaneously open. The exact root causes<br />

of these variati<strong>on</strong>s in server resource usage between<br />

LAN and WAN needs to be investigated further but we<br />

think that BenchLab is an ideal testbed for the research<br />

community to c<strong>on</strong>duct such experiments.<br />

5.4.2. Geographically dispersed load injecti<strong>on</strong><br />

We investigate the use of multiple data centers in the<br />

cloud to perform a geographically dispersed load injecti<strong>on</strong>.<br />

We re-use the same 25 user Cloudst<strong>on</strong>e workload<br />

and distribute <strong>Web</strong> browsers in different Amaz<strong>on</strong> data<br />

centers as follows: 7 US East coast (N. Virginia), 6 US<br />

West coast (California), 6 Europe (Ireland) and 6 Asia<br />

(Singapore). Such a setup (25 distributed instances) can<br />

be deployed for as little as $0.59/hour using Linux micro<br />

instances or $0.84/hour using Windows instances.<br />

More powerful small instances cost $2.30/hour for<br />

Linux and $3.00/hour for Windows. Figure 9 shows the<br />

latency reported by all <strong>Web</strong> browsers color coded per<br />

regi<strong>on</strong> (y axis is log scale).<br />

Table 4. Average latency and standard deviati<strong>on</strong><br />

observed in different geographic regi<strong>on</strong>s<br />

US East US West Europe Asia<br />

Avg latency 920ms 1573ms 1720ms 3425ms<br />

Std deviati<strong>on</strong> 526 776 906 1670<br />

As our <strong>Web</strong> applicati<strong>on</strong> server is located in the US East<br />

coast, the lowest latencies are c<strong>on</strong>sistently measured by<br />

browsers physically located in the East coast data center.<br />

Average latency almost doubles for requests originating<br />

from the West coast or Europe. Finally, as expected,<br />

the Asian data center experiences the l<strong>on</strong>gest<br />

latencies. It is interesting to notice that the latency<br />

standard deviati<strong>on</strong> also increases with the distance from<br />

the <strong>Web</strong> applicati<strong>on</strong> server as summarized in Table 4.<br />

The server average CPU usage is 74.3% which is slightly<br />

less than when all browsers were located in the East<br />

coast (77.7%).<br />

Figure 9. Browser (Firefox) perceived latency in ms<br />

(y axis is log scale) <strong>on</strong> a Cloudst<strong>on</strong>e workload with<br />

users distributed as follows: 7 US East coast, 6 US<br />

West coast, 6 Europe and 6 Asia. Server locati<strong>on</strong> is<br />

UMass Amherst (US East coast, Massachusetts).<br />

5.5. Summary<br />

We have shown that real modern <strong>Web</strong> applicati<strong>on</strong>s<br />

have complex interacti<strong>on</strong>s with <strong>Web</strong> browsers and that<br />

the state size of the applicati<strong>on</strong> can greatly affect the<br />

applicati<strong>on</strong> performance. Traditi<strong>on</strong>al trace replay tools<br />

cannot reproduce the rich interacti<strong>on</strong>s of browsers or<br />

match their optimized communicati<strong>on</strong> with <strong>Web</strong> applicati<strong>on</strong><br />

servers. Therefore the load observed <strong>on</strong> applicati<strong>on</strong><br />

servers varies greatly when the same workload is<br />

injected through an emulated browser or a real <strong>Web</strong><br />

browser. This is further accentuated when JavaScript<br />

code generates additi<strong>on</strong>al queries or performs error<br />

checking that prevents err<strong>on</strong>eous inputs to reach the<br />

server.<br />

Finally, we have shown the influence of LAN vs WAN<br />

load injecti<strong>on</strong> using real <strong>Web</strong> browsers deployed in a<br />

public cloud. Not <strong>on</strong>ly the latency and its standard deviati<strong>on</strong><br />

increase with the distance but the load <strong>on</strong> the<br />

server significantly differs between a LAN and a WAN<br />

experiment using the exact same workload.<br />

6. Related Work<br />

Benchmarks such as RUBiS [13] and TPC-W [16] have<br />

now become obsolete. BenchLab uses CloudSt<strong>on</strong>e [15]<br />

and Wikibooks [20] as realistic <strong>Web</strong> 2.0 applicati<strong>on</strong><br />

backends. The Rain [2] workload generati<strong>on</strong> toolkit<br />

separates the workload generati<strong>on</strong> from executi<strong>on</strong> to be<br />

able to leverage existing tools such as httperf. Bench-<br />

Lab uses the same c<strong>on</strong>cept to be able to replay real<br />

workload traces from real applicati<strong>on</strong>s such as Wikibooks<br />

or Wikipedia [17] in <strong>Web</strong> browsers.<br />

Browser automati<strong>on</strong> frameworks have been developed<br />

primarily for functi<strong>on</strong>al testing. BenchLab uses real<br />

<strong>Web</strong> browsers for <strong>Web</strong> applicati<strong>on</strong> benchmarking.<br />

Commercial technologies like HP TruClient [6] or<br />

Keynote <strong>Web</strong> performance testing tools [7] offer load<br />

injecti<strong>on</strong> from modified versi<strong>on</strong>s of Firefox or Internet<br />

Explorer. BrowserMob [4] provides a similar service<br />

<str<strong>on</strong>g>USENIX</str<strong>on</strong>g> Associati<strong>on</strong> <strong>Web</strong>Apps ’11: <str<strong>on</strong>g>2nd</str<strong>on</strong>g> <str<strong>on</strong>g>USENIX</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> <strong>Web</strong> Applicati<strong>on</strong> <strong>Development</strong> 47

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!