High Performance Multi-Server Magento in der Cloud
High Performance Multi-Server Magento in der Cloud
High Performance Multi-Server Magento in der Cloud
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Scal<strong>in</strong>g <strong>Magento</strong><br />
for the Global Market<br />
Kian Gould<br />
Foun<strong>der</strong> & CEO
Angry Birds
Angry Birds
Angry Birds
Angry Birds
The Angry Birds <strong>Magento</strong> Universe<br />
DRM <strong>Server</strong><br />
3 Storefronts<br />
(US, EU, ASIA)<br />
Custom<br />
Report<strong>in</strong>g<br />
Banner<br />
Management<br />
Catalog<br />
Rules<br />
Warehouse<br />
Inventory<br />
Splitshipp<strong>in</strong>g<br />
Custom Tax,<br />
Custom Shipp<strong>in</strong>g<br />
…<br />
Mail Service<br />
Giftcards<br />
Fullfillment<br />
Warehouse /<br />
Inventory<br />
Management<br />
Bra<strong>in</strong>tree
<strong>High</strong> Level Requirements<br />
• Innovative, highly-scalable e-commerce systems<br />
• <strong>High</strong> availability<br />
• Unusually high peaks (20x and more)<br />
• <strong>High</strong> Speed<br />
• Backend server <strong>in</strong>dependent from frontend servers<br />
• No down-times, ever.
Solution components
<strong>Cloud</strong> Services<br />
EC2 S3 RDS <strong>Cloud</strong>- Elasti- ELB Route 53<br />
Front<br />
Cache<br />
Amazon AWS Console<br />
Management<br />
Deployments<br />
Auto-Scal<strong>in</strong>g<br />
RightScale Services<br />
Monitor<strong>in</strong>g<br />
Logg<strong>in</strong>g<br />
Script<strong>in</strong>g API
<strong>Magento</strong> <strong>in</strong> the <strong>Cloud</strong>
Share noth<strong>in</strong>g<br />
JS/CSS<br />
Bundles<br />
Code<br />
Assets<br />
Product images,<br />
CMS images,<br />
Scaled images<br />
<strong>Server</strong><br />
Database<br />
Session<br />
Storage<br />
Cache
shop.angrybirds.com<br />
<strong>Cloud</strong>Front<br />
CDN for Assets<br />
S3<br />
Assets Storage<br />
Route 53<br />
DNS-Service<br />
<strong>Cloud</strong>Front<br />
CDN for Sk<strong>in</strong><br />
ELB<br />
Load Balancer<br />
Monitor<strong>in</strong>g<br />
Logg<strong>in</strong>g<br />
EC2<br />
EC2<br />
Varnish Array<br />
EC2<br />
Backend Array<br />
EC2<br />
EC2<br />
Frontend Array<br />
EC2<br />
Worker Array<br />
S3<br />
Deployment<br />
ElastiCache<br />
Cache Backend<br />
RDS<br />
MySQL<br />
DRM<br />
Giftcards<br />
Mail<br />
Bra<strong>in</strong>tree
shop.angrybirds.com<br />
Route 53<br />
DNS-Service<br />
ELB<br />
Load Balancer<br />
Deployment X<br />
EC2<br />
EC2<br />
Varnish Array<br />
EC2<br />
Backend Array<br />
EC2<br />
EC2<br />
Frontend Array<br />
EC2<br />
Worker Array<br />
S3<br />
Deployment<br />
RDS<br />
MySQL
shop.angrybirds.com<br />
Route 53<br />
DNS-Service<br />
Direct Access<br />
through<br />
Local Host entry<br />
ELB<br />
Load Balancer<br />
Deployment X+1<br />
Release Y<br />
EC2<br />
EC2<br />
Varnish Array<br />
Cache<br />
warm<strong>in</strong>g<br />
EC2<br />
EC2<br />
Backend Array<br />
EC2<br />
EC2<br />
Frontend Array<br />
EC2<br />
Worker Array<br />
S3<br />
Deployment<br />
RDS<br />
MySQL
Deployment / Rollback<br />
shop.angrybirds.com<br />
Route 53<br />
DNS-Service<br />
ELB<br />
Load Balancer<br />
ELB<br />
Load Balancer<br />
Deployment X<br />
Deployment X+1<br />
Release Y<br />
Varnish Array<br />
EC2<br />
EC2<br />
Frontend Array<br />
EC2<br />
Worker Array<br />
EC2<br />
EC2<br />
Backend Array<br />
EC2<br />
EC2
Downtime dur<strong>in</strong>g the deployment:
All Basket Content,<br />
active Log<strong>in</strong>s and Sessions<br />
are carried through each<br />
Deployment / Rollback
<strong>Magento</strong>, faster!
Areas of optimization<br />
<strong>Magento</strong> Frontend Cach<strong>in</strong>g
Frontend-Optimization<br />
JavaScript / CSS<br />
• Bundl<strong>in</strong>g (<strong>Magento</strong>)<br />
• M<strong>in</strong>ify<strong>in</strong>g (with YUI Compressor)<br />
• Gzip-Compression (Apache)<br />
• Timestamp <strong>in</strong> filename<br />
• Send Cache-Hea<strong>der</strong><br />
Aoe_JsCssTstamp<br />
CDN (<strong>Cloud</strong>front)<br />
• Product images (+ scaled versions), CMS Images etc.,...<br />
• Sk<strong>in</strong>s<br />
• JS/CSS
<strong>Magento</strong>-Optimizations<br />
Reduce Logg<strong>in</strong>g as much as possible<br />
Deactivate non-cacheable features wherever possible<br />
Cach<strong>in</strong>g<br />
Aoe_AsyncCache<br />
• Asynchronous Cache Clear<strong>in</strong>g Queue<br />
• Delete old Cache-entries<br />
Aoe_CacheCleaner<br />
Cookie Storage for Username and Basket content<br />
Ajax-Basket<br />
and... Profil<strong>in</strong>g, Profil<strong>in</strong>g, Profil<strong>in</strong>g,…<br />
(xdebug + kcachegr<strong>in</strong>d, Newrelic, xhprof)
Cach<strong>in</strong>g<br />
95% of all hits<br />
are cacheable!<br />
5% dynamic<br />
content<br />
Homepage<br />
Categories<br />
Products<br />
CMS<br />
Basket<br />
Checkout<br />
Profile
Cach<strong>in</strong>g<br />
Varnish<br />
<strong>Magento</strong><br />
Reverse Proxy
Solutions<br />
Aoe_Static<br />
Magneto_Varnish<br />
Cach<strong>in</strong>g +<br />
Purg<strong>in</strong>g<br />
Handl<strong>in</strong>g of<br />
dynamic Content
Lessons learned<br />
• 404s are very expensive<br />
=> cache or redirect<br />
• <strong>Cloud</strong>front with S3 Backend<br />
doesn’t support gzip<br />
=> deploy time bundel<strong>in</strong>g/compression<br />
• Don’t run reports on live database!<br />
• You can’t plan for everyth<strong>in</strong>g<br />
=> but you can be ready to solve anyth<strong>in</strong>g quickly. How<br />
=> cont<strong>in</strong>uous deployment
Cont<strong>in</strong>uous Integration<br />
und Deployment
Don’t be afraid of<br />
deploy<strong>in</strong>g to Live!
<strong>Magento</strong> Installationpackage<br />
Files*<br />
‣htdocs [<strong>Magento</strong> Source]<br />
‣.modman [Custom Packages]<br />
SVN<br />
GIT<br />
Installer<br />
Files<br />
‣media<br />
Database<br />
tables<br />
System-<br />
storage<br />
ChangeLog<br />
Sett<strong>in</strong>gs<br />
‣URLs<br />
‣DB-Parameters<br />
‣…<br />
Sett<strong>in</strong>gs
Deployment Pipel<strong>in</strong>e<br />
Build<br />
Installation on<br />
„Latest“<br />
System<br />
Unit-Tests<br />
Acceptance<br />
Tests<br />
<strong>Performance</strong><br />
Tests<br />
PHPUnit
Deployment Pipel<strong>in</strong>e<br />
Installation on<br />
<strong>Cloud</strong><br />
Integration<br />
<strong>Cloud</strong><br />
Deployment<br />
Copy to S3<br />
Deployment<br />
Tests<br />
Deployment<br />
System<br />
Stag<strong>in</strong>g<br />
AOE + Rovio<br />
Production
Launch!<br />
USA wakes up<br />
Space game<br />
Launch<br />
Array scales up<br />
Deployment of<br />
new Release
In real life…
visitors/day
(real) concurrent users!
or<strong>der</strong>s per second at peak!
@aoemedia<br />
Thank You!<br />
AOE media Inc.<br />
251 Rhode Island Street<br />
San Francisco, CA 94103<br />
USA<br />
Tel.: +1 415 230-0690<br />
Fax: +1 415 230-0699<br />
www.aoemedia.com<br />
sales@aoemedia.com
Change language