Network Traffic Characteristics of Data Centers in the Wild - Sigcomm
Network Traffic Characteristics of Data Centers in the Wild - Sigcomm
Network Traffic Characteristics of Data Centers in the Wild - Sigcomm
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>Data</strong>Center <strong>Data</strong>Center Location Age(Years) SNMP Packet Topology Number Number Over<br />
Role Name (CurrVer/Total) Traces Devices Servers Subscription<br />
EDU1 US-Mid 10 22 500 2:1<br />
Universities<br />
EDU2<br />
EDU3<br />
US-Mid<br />
US-Mid<br />
(7/20)<br />
N/A<br />
<br />
<br />
<br />
<br />
<br />
<br />
36<br />
1<br />
1093<br />
147<br />
47:1<br />
147:1<br />
Private<br />
PRV1<br />
PRV2<br />
US-Mid<br />
US-West<br />
(5/5)<br />
> 5<br />
<br />
<br />
X<br />
<br />
<br />
<br />
96<br />
100<br />
1088<br />
2000<br />
8:3<br />
48:10<br />
CLD1 US-West > 5 X X 562 10K 20:1<br />
Commercial<br />
CLD2<br />
CLD3<br />
US-West<br />
US-East<br />
> 5<br />
> 5<br />
<br />
<br />
X<br />
X<br />
X<br />
X<br />
763<br />
612<br />
15K<br />
12K<br />
20:1<br />
20:1<br />
CLD4 S.America (3/3) X X 427 10K 20:1<br />
CLD5 S.America (3/3) X X 427 10K 20:1<br />
Table2:Summary<strong>of</strong><strong>the</strong>10datacentersstudied,<strong>in</strong>clud<strong>in</strong>gdevices,types<strong>of</strong><strong>in</strong>formationcollected,and<strong>the</strong>number<strong>of</strong>servers.<br />
tivesitesandwebportalsforstudentsandfaculty),andmulticast<br />
videostreams. Weprovide<strong>the</strong>exactapplicationmix<strong>in</strong><strong>the</strong>next<br />
section. Intalk<strong>in</strong>gto<strong>the</strong>networkoperators,wefoundthat<strong>the</strong>se<br />
datacenters“organically”evolvedovertime,mov<strong>in</strong>gfromacollection<strong>of</strong>devices<strong>in</strong>astorageclosettoadedicatedroomforserversandnetworkdevices.As<strong>the</strong>datacentersreachedcapacity,<strong>the</strong>operatorsre-evaluated<strong>the</strong>irdesignandarchitecture.Manyoperatorschosetomovetoamorestructured,two-layertopologyand<strong>in</strong>troducedservervirtualizationtoreduceheat<strong>in</strong>gandpowerrequirementswhilecontroll<strong>in</strong>gdatacentersize.<br />
Privateenterprises:TheprivateenterpriseITdatacentersserve<br />
corporateusers,developers,andasmallnumber<strong>of</strong>customers.Unlikeuniversitydatacenters,<strong>the</strong>privateenterprisedatacenterssupportasignificantnumber<strong>of</strong>customapplications,<strong>in</strong>additionto<br />
host<strong>in</strong>gtraditionalserviceslikeEmail,storage,andWebservices.<br />
They<strong>of</strong>tenactasdevelopmenttestbeds,aswell. Thesedatacentersaredeveloped<strong>in</strong>aground-upfashion,be<strong>in</strong>gdesignedspecificallytosupport<strong>the</strong>demands<strong>of</strong><strong>the</strong>enterprise.<br />
For<strong>in</strong>stance,to<br />
satisfy<strong>the</strong>needtosupportadm<strong>in</strong>istrativeservicesandbetatest<strong>in</strong>g<br />
<strong>of</strong>database-dependentproducts,PRV1commissioned<strong>the</strong>development<strong>of</strong>an<strong>in</strong>-housedatacenter5yearsago.PRV2wasdesignedover5yearsagomostlytosupportcustomL<strong>in</strong>e-<strong>of</strong>-Bus<strong>in</strong>essapplicationsandtoprovidelog<strong>in</strong>serversforremoteusers.<br />
Commercialclouddatacenters: Unlike<strong>the</strong>firsttwoclasses<br />
<strong>of</strong>datacenters,<strong>the</strong>commercialdatacenterscatertoexternalusers<br />
and<strong>of</strong>fersupportforawiderange<strong>of</strong>Internet-fac<strong>in</strong>gservices,<strong>in</strong>clud<strong>in</strong>g:InstantMessag<strong>in</strong>g,Webmail,search,<strong>in</strong>dex<strong>in</strong>g,andvideo.Additionally,<strong>the</strong>datacentershostlarge<strong>in</strong>ternalsystemsthatsupport<strong>the</strong>externallyvisibleservices,forexampledatam<strong>in</strong><strong>in</strong>g,storage,andrelationaldatabases(e.g.,forbuddylists).Thesedatacentersare<strong>of</strong>tenpurpose-builttosupportaspecificset<strong>of</strong>applications<br />
(e.g.,withaparticulartopologyorover-subscriptionratiotosome<br />
targetapplicationpatterns),but<strong>the</strong>reisalsoatensiontomake<strong>the</strong>m<br />
asgeneralaspossiblesothat<strong>the</strong>applicationmixcanchangeover<br />
timeas<strong>the</strong>usageevolves.CLD1,CLD2,CLD3hostavariety<strong>of</strong><br />
applications,rang<strong>in</strong>gfromInstantMessag<strong>in</strong>gandWebmailtoadvertisementsandwebportals.CLD4andCLD5areprimarilyused<br />
forrunn<strong>in</strong>gMapReducestyleapplications.<br />
3.3 TopologyandComposition<strong>of</strong><strong>the</strong><strong>Data</strong><strong>Centers</strong><br />
Inthissection,weexam<strong>in</strong>e<strong>the</strong>differencesandsimilarities<strong>in</strong><br />
<strong>the</strong>physicalconstruction<strong>of</strong><strong>the</strong>datacenters. Beforeproceed<strong>in</strong>g<br />
toexam<strong>in</strong>e<strong>the</strong>physicaltopology<strong>of</strong><strong>the</strong>datacentersstudied,we<br />
presentabriefoverview<strong>of</strong><strong>the</strong>topology<strong>of</strong>agenericdatacenter.In<br />
Figure1,wepresentacanonical3-Tiereddatacenter.The3tiers<strong>of</strong><br />
<strong>the</strong>datacenterare<strong>the</strong>edgetier,whichconsists<strong>of</strong><strong>the</strong>Top-<strong>of</strong>-Rack<br />
switchesthatconnect<strong>the</strong>serversto<strong>the</strong>datacenter’snetworkfabric;<br />
<strong>the</strong>aggregationtier,whichconsists<strong>of</strong>devicesthat<strong>in</strong>terconnect<strong>the</strong><br />
270<br />
Figure1:Canonical3-Tierdatacentertopology.<br />
ToRswitches<strong>in</strong><strong>the</strong>edgelayer;and<strong>the</strong>coretier,whichconsists<br />
<strong>of</strong>devicesthatconnect<strong>the</strong>datacenterto<strong>the</strong>WAN.Insmallerdata<br />
centers,<strong>the</strong>coretierand<strong>the</strong>aggregationtierarecollapsed<strong>in</strong>toone<br />
tier,result<strong>in</strong>g<strong>in</strong>a2-Tiereddatacentertopology.<br />
Now,wefocusontopologicalstructureand<strong>the</strong>keyphysical<br />
properties<strong>of</strong><strong>the</strong>constituentdevicesandl<strong>in</strong>ks. Wef<strong>in</strong>dthat<strong>the</strong><br />
topology<strong>of</strong><strong>the</strong>datacenteris<strong>of</strong>tenanaccident<strong>of</strong>history. Some<br />
haveregularpatternsthatcouldbeleveragedfortrafficeng<strong>in</strong>eer<strong>in</strong>gstrategieslikeValiantLoadBalanc<strong>in</strong>g[11],whilemostwould<br />
requireei<strong>the</strong>rasignificantupgradeormoregeneralstrategies.<br />
Topology. Of<strong>the</strong>threeuniversitydatacenters,wef<strong>in</strong>dthattwo<br />
(EDU1,EDU2)haveevolved<strong>in</strong>toastructured2-Tierarchitecture.<br />
Thethird(EDU3)usesastar-liketopologywithahigh-capacity<br />
centralswitch<strong>in</strong>terconnect<strong>in</strong>gacollection<strong>of</strong>serverracks–adesignthathasbeenuseds<strong>in</strong>ce<strong>the</strong><strong>in</strong>ception<strong>of</strong>thisdatacenter.As<br />
<strong>of</strong>thiswrit<strong>in</strong>g,<strong>the</strong>datacenterwasmigrat<strong>in</strong>gtoamorestructured<br />
set-upsimilarto<strong>the</strong>o<strong>the</strong>rtwo.<br />
EDU1usesatopologythatissimilartoacanonical2-Tierarchitecture,withonekeydifference:while<strong>the</strong>canonical2-Tierdata<br />
centersuseTop-<strong>of</strong>-Rackswitches,whereeachswitchconnectstoa<br />
rack<strong>of</strong>20-80serversorso,<strong>the</strong>setwodatacentersutilizeMiddle<strong>of</strong>-Rackswitchesthatconnectarow<strong>of</strong>5to6rackswith<strong>the</strong>potentialtoconnectfrom120to180servers.<br />
Wef<strong>in</strong>dthatsimilar<br />
conclusionsholdforEDU2(omittedforbrevity).<br />
Theenterprisedatacentersdonotdeviatemuchfromtextbookstyleconstructions.<br />
Inparticular,<strong>the</strong>PRV1enterprisedatacenter<br />
utilizesacanonical2-TierCiscoarchitecture.ThePRV2datacenterutilizesacanonical3-TierCiscoarchitecture.
Change language