29.11.2012 Views

2nd USENIX Conference on Web Application Development ...

2nd USENIX Conference on Web Application Development ...

2nd USENIX Conference on Web Application Development ...

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

using Firefox and Selenium. However, these proprietary<br />

products can <strong>on</strong>ly be used in private clouds or dedicated<br />

test envir<strong>on</strong>ments. BenchLab is fully open and can be<br />

deployed <strong>on</strong> any device that has a <strong>Web</strong> browser. By<br />

deploying browsers <strong>on</strong> home desktops or cell ph<strong>on</strong>es,<br />

BenchLab can be used to analyze last mile latencies.<br />

Server-side m<strong>on</strong>itors and log analysis tools have been<br />

used previously to try to model the dependencies between<br />

file accesses and predict the full page load times<br />

observed by clients [10][18]. BenchLab’s use of real<br />

<strong>Web</strong> browsers allows it to accurately capture the behavior<br />

of loading <strong>Web</strong> pages composed of multiple files,<br />

and could be used by <strong>Web</strong> service providers to m<strong>on</strong>itor<br />

the performance of their applicati<strong>on</strong>s. When deployed<br />

<strong>on</strong> a large number of machines (home desktops, cell<br />

ph<strong>on</strong>es, cloud data centers…), BenchLab can be used to<br />

reproduce large scale flash crowds. When client browsers<br />

are geographically dispersed, BenchLab can be used<br />

to evaluate C<strong>on</strong>tent Delivery Network (CDN) performance<br />

or failover mechanisms for highly available <strong>Web</strong><br />

applicati<strong>on</strong>s.<br />

BenchLab is designed to be easily deployed across wide<br />

geographic areas by utilizing public clouds. The impact<br />

of wide area latency <strong>on</strong> <strong>Web</strong> applicati<strong>on</strong> performance<br />

has been studied in a number of scenarios [5]; Bench-<br />

Lab provides a standardized architecture to allow applicati<strong>on</strong><br />

developers and researchers to measure how their<br />

systems perform in real WAN envir<strong>on</strong>ments. We believe<br />

that BenchLab can be used to measure the effectiveness<br />

of WAN accelerators (CDNs or proxy caches)<br />

as well as validate distributi<strong>on</strong>s modeling WAN load<br />

patterns.<br />

7. C<strong>on</strong>clusi<strong>on</strong><br />

We have dem<strong>on</strong>strated the need to capture the behavior<br />

of real <strong>Web</strong> browsers to benchmark real <strong>Web</strong> 2.0 applicati<strong>on</strong>s.<br />

We have presented BenchLab, an open testbed<br />

for realistic benchmarking of modern <strong>Web</strong> applicati<strong>on</strong>s<br />

using real <strong>Web</strong> browsers. BenchLab employs a modular<br />

architecture that is designed to support different<br />

backend server applicati<strong>on</strong>s. Our evaluati<strong>on</strong> has illustrated<br />

the need for 1) updated <strong>Web</strong> applicati<strong>on</strong>s with<br />

realistically sized datasets to use as benchmarks, 2) real<br />

browser based load injecti<strong>on</strong> tools that authentically<br />

reproduce user interacti<strong>on</strong>s and 3) wide area benchmark<br />

client deployments to match the network behavior seen<br />

by real applicati<strong>on</strong>s. We believe that BenchLab meets<br />

these needs, and we hope that it will help the research<br />

community improve the realism and accuracy of <strong>Web</strong><br />

applicati<strong>on</strong> benchmarking. We are making all the<br />

BenchLab software (runtime, tools, <strong>Web</strong> applicati<strong>on</strong>s…)<br />

available to the community under an open<br />

source license <strong>on</strong> our <strong>Web</strong> site [3].<br />

8. Acknowledgement<br />

The authors would like to thank the an<strong>on</strong>ymous reviewers and<br />

our shepherd Geoffrey Voelker for their valuable feeback. We<br />

would also like to thank Fabien Mottet and Vivien Quema<br />

from INRIA Rhône-Alpes, Guillaume Pierre from Vrije University<br />

and Vimal Mathew from UMass, for their c<strong>on</strong>tributi<strong>on</strong>s<br />

to BenchLab. This research was supported in part by<br />

grants from Amaz<strong>on</strong> AWS, UMass President’s Science &<br />

Technology fund, and NSF grants CNS-0916972, CNS-<br />

083243, CNS-0720616 and CNS-0855128.<br />

9. References<br />

[1] Apache HttpComp<strong>on</strong>ents – http://hc.apache.org/<br />

[2] A. Beitch, B. Liu, T. Yung, R. Griffith, A. Fox and D.<br />

Patters<strong>on</strong> – Rain: A Workload Generati<strong>on</strong> Toolkit for<br />

Cloud Computing Applicati<strong>on</strong>s – Technical Report<br />

UCB/EECS-2010-14, February 10, 2010.<br />

[3] BenchLab - http://lass.cs.umass.edu/projects/benchlab/<br />

[4] BrowserMob - http://browsermob.com/performancetesting<br />

[5] S. Chen, K.R. Joshi, M.A. Hiltunen, W.H. Sanders and<br />

R.D. Schlichting – Link Gradients: Predicting the Impact<br />

of Network Latency <strong>on</strong> Multitier Applicati<strong>on</strong>s – IN-<br />

FOCOM 2009, pp.2258-2266, 19-25 April 2009.<br />

[6] HP - TruClient technology: Accelerating the path to<br />

testing modern applicati<strong>on</strong>s – Business white paper,<br />

4AA3-0172ENW, November 2010.<br />

[7] R. Hughes and K. Vodicka – Why Real Browsers Matter<br />

– Keynote white paper, http://www.keynote.com/docs/<br />

whitepapers/why_real_browers_matter.pdf.<br />

[8] D. Krishnamurthy, J. A. Rolia and Shikharesh Majumdar<br />

– A Synthetic Workload Generati<strong>on</strong> Technique for Stress<br />

Testing Sessi<strong>on</strong>-Based Systems – IEEE Transacti<strong>on</strong> <strong>on</strong><br />

Software Engineering. 32, 11 - November 2006.<br />

[9] HTTP Archive specificati<strong>on</strong> (HAR) v1.2 -<br />

http://www.softwareishard.com/blog/har-12-spec/.<br />

[10] Z. Li, M. Zhang, Z. Zhu, Y. Chen, A.G. Greenberg, and<br />

Y. Wang - <strong>Web</strong>Prophet: Automating Performance<br />

Predicti<strong>on</strong> for <strong>Web</strong> Services – NSDI, 2010, pp.143-158.<br />

[11] E. M. Nahum, M.C. Rosu, S. Seshan and J. Almeida –<br />

The effects of wide-area c<strong>on</strong>diti<strong>on</strong>s <strong>on</strong> WWW server<br />

performance – SIGMETRICS 2001.<br />

[12] Olio – http://incubator.apache.org/olio/<br />

[13] RUBiS <strong>Web</strong> site – http://rubis.ow2.org.<br />

[14] Selenium - http://seleniumhq.org/<br />

[15] W. Sobel, S. Subramanyam, A. Sucharitakul, J. Nguyen,<br />

H. W<strong>on</strong>g, A. Klepchukov, S. Patil, A. Fox and D. Patters<strong>on</strong><br />

– Cloudst<strong>on</strong>e: Multi-platform, multi-language<br />

benchmark and measurement tools for <strong>Web</strong> 2.0 – Cloud<br />

Computing and its Applicati<strong>on</strong>s CCA-08, 2008.<br />

[16] TPC-W Benchmark, Object<strong>Web</strong> implementati<strong>on</strong>,<br />

http://jmob.object<strong>Web</strong>.org/tpcw.html<br />

[17] G. Urdaneta, G. Pierre and M. van Steen – Wikipedia<br />

Workload Analysis for Decentralized Hosting – Elsevier<br />

Computer Networks, vol.53, July 2009.<br />

[18] J. Wei, C.Z. Xu - Measuring Client-Perceived Pageview<br />

Resp<strong>on</strong>se Time of Internet Services – IEEE Transacti<strong>on</strong>s<br />

<strong>on</strong> Parallel and Distributed Systems, 2010.<br />

[19] WikiBench - http://www.wikibench.eu/<br />

[20] Wikibooks – http://www.wikibooks.org<br />

[21] Wikipedia – http://www.wikipedia.org<br />

48 <strong>Web</strong>Apps ’11: <str<strong>on</strong>g>2nd</str<strong>on</strong>g> <str<strong>on</strong>g>USENIX</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> <strong>Web</strong> Applicati<strong>on</strong> <strong>Development</strong> <str<strong>on</strong>g>USENIX</str<strong>on</strong>g> Associati<strong>on</strong>

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!