Understanding Carrier Ethernet Throughput - MEF

Understanding Carrier Ethernet ThroughputAm I getting the throughput I should be getting?2010 July Version 2

Understanding Carrier Ethernet ThroughputTable of ContentsTable of ContentsIntroduction ............................................................................................................ 3Identifying Throughput Problems ............................................................................... 4Basic MEF Service Concepts ...................................................................................... 5Protocol Basics for Carrier Ethernet ......................................................................... 5The Basic Elements of Service ................................................................................ 7Factors Affecting Throughput .................................................................................... 9Customer Challenges in Conforming to the SLA ........................................................ 9The Impacts of Frame Size and Protocol on Throughput Efficiency ............................ 10Standard TCP Window Size and Effect on Throughput .............................................. 12Using Window Scaling to Increase TCP Throughput ................................................. 14Dropped/Errored Frames and the Retransmission Effect ........................................... 15Monitoring Throughput ........................................................................................... 16General Network Management Tools ..................................................................... 16Verifying Carrier Ethernet Performance .................................................................... 18Troubleshooting with PC Based Test Tools .............................................................. 18Software Based Test Tools ................................................................................... 18Hardware Based Test Tools .................................................................................. 19Summary and Recommendations............................................................................. 19Glossary Of Abbreviations ....................................................................................... 20References............................................................................................................ 21Acknowledgements ................................................................................................ 22Page 2 of 22 March 2010

Understanding Carrier Ethernet ThroughputIntroductionGlobalization, virtualization, and mobile computing drive a seemingly insatiable demandfor bandwidth, and only Carrier Ethernet efficiently scales up to meet this demand.Customers seeking high performance business Ethernet services can now easily purchasefaster Ethernet connections at 10 Mbit/s to 1 Gbit/s and beyond. But sometimes usersbelieve they are receiving lower throughput than they expected. This perception can bedue to poor application performance which is caused by factors un-related to Ethernetservice throughput. Many IP and application layer factors affect a user’s applicationexperience when utilizing an Ethernet service, most of which are under their own directcontrol.First and foremost, obtaining a good service requires selecting an Ethernet serviceprovider that is MEF certified to deliver a high quality Carrier Ethernet service. Secondly,Enterprise users must ensure that they are shaping the bandwidth offered to the networkto match the bandwidth profile of the service level agreement (SLA). For example, drivinga 50Mbit/s Ethernet service with 100 Mbit/s for a time period larger than the contractedCommitted Burst Size (CBS) will provide a poor user experience, with dropped traffic, retransmissions,and net throughputs which are much lower than expected. Other keyapplication issues include optimally setting Transport Control Protocol (TCP) window sizeon applications which require higher speed or are delivered over services with longerdelay. For instance, TCP window limitations can be seen on Ethernet services transmittingat information rates as low as 13 Mbit/s when combined with transmission delays in therange of 40 ms. In addition, the Ethernet frame size, the selection of higher layerprotocols and error rates can all affect both delay and throughput of an application beingdelivered over an Ethernet service.This MEF white paper presents an overview of the more common factors affecting CarrierEthernet throughput, provides some pointers for getting more performance from higherlayer protocols, and shows how to measure bandwidth throughput of a Carrier Ethernetservice. Note that to simplify the scope of this discussion; the focus of the whitepaper ison E-Line services that are point to point. More sophisticated E-LAN and E-Tree servicesare influenced by the same factors plus some additional factors such asmulticast/broadcast storms which are specific to their multi-point topologies.Page 3 of 22 March 2010

Understanding Carrier Ethernet ThroughputIdentifying Throughput ProblemsEthernet service providers report that a significant portion of customer trouble tickets areopened due to poor application performance. Typically, enterprise customers call theirservice provider when the transfer rate between the host and servers across the serviceprovider’s network appears to be slow (below the contracted throughput rate). Whenthey think there might be a problem, some end users “test” the rate by running a filetransfer between two sites (as measured by their operating system). By looking at thetransfer rate shown in Figure 1, an end user would presume that the maximum rate oftheir link is 339 KByte/s, or 2.78 Mbit/s. If the contracted service is supposed to be at 50Mbit/s committed information rate (CIR) as in Figure 2, the user will be frustrated.Figure 1. Windows XP file transfer dialog box showing transfer rateUNINID Operator 1(Service Provider)ENNIOperator 2(OOF operator)NIDUNI50 Mbit/s frame serviceover 1 GigE PhyFigure 2. Customer’s high performance 50 Mbit/s Ethernet Virtual ConnectionNote that in Figure 2, the customer has purchased a 50 Mb/s E-Line service, but has a 1Gbit/s Ethernet physical interface User Network Interface (UNI). A Network InterfacePage 4 of 22 March 2010

Understanding Carrier Ethernet ThroughputDevice, (NID), may be installed on his premises by the Service Provider to provide variousnetwork terminating functions. The Service Provider (Operator1) partners with anotherOOF (Out of Franchise) Operator 2, interconnecting their networks at the ExternalNetwork to Network Interface (ENNI). The two carriers have constructed an EthernetVirtual Connection (EVC) to implement the E-Line service.The end user with the throughput problem may react by blaming the service providers forthe lower achieved data rate because he or she thinks the network does not perform asdetailed in their service level agreements (SLAs). As these users complain to their ITdepartments, the IT personnel will often test the links with more advanced software-basedtools to validate these claims. However, because their tools are PC-based, they also comewith all of the limitations found in PC operating systems. Should a less seasoned ITprofessional use those tools, he might come to the same conclusion as the end user andopen a trouble ticket with the service provider.Basic MEF Service ConceptsProtocol Basics for Carrier EthernetBefore discussing the issues and behaviors of Ethernet Services and test tools in greaterdetail, it is important to understand the protocols found on Ethernet links.Figure 3. OSI Reference Model with examples of protocols for each layerThe Open Systems Interconnect (OSI) model is shown above and is used as a model fordeveloping data network protocols. Each layer works with the layers above and belowthem to enable communication with the same layer of another stack instance. CarrierPage 5 of 22 March 2010

Understanding Carrier Ethernet Throughputconnectionless and less reliable, but very fast with no upward limits on throughput. UDPmessages can be lost, duplicated or arrive out-of-sequence. With no flow control,messages can arrive faster than they can be processed. UDP relies on the applicationlayer for everything related to error validation, flow control and retransmission.UDP is stateless by nature (meaning that each message is seen as an independenttransaction). This behavior is used by real-time applications such as IPTV, VoIP, TrivialFile Transfer Protocol (TFTP) and network gaming, which benefit from the protocol’s lowlatency (total delay of getting data from one application to another through the network)and lack of speed limitations.TCP, on the other hand, is a connection-oriented protocol that requires handshaking to setup end-to-end communications. TCP provides the following functions and benefits:• Reliable, transparent transfer of data between networked end points• End-to-end error detection, recovery and data flow control• Sequential numbering of frames to keep track of them if they arrive out of order• Segmentation and reassembly of user data and higher-layer protocols• Adaptive transmission data rate control to utilize the available speed of the linkTCP is not perfect however. These details will be covered in more depth in the followingsections, but here is a short summary. TCP has more overhead bytes than UDP, with 20or more bytes in TCP’s overhead compared with only 8 bytes in UDP. In TCP, extra bytesare used for things like window scaling and selective acknowledgment. With TCP there issome delay in setting up a reliable link, because it goes through a significant handshakingprocedure to start the session. TCP’s continuous acknowledgments and need to bufferdata for potential retransmission can bog down processor performance. Finally, TCP’sperformance can be significantly compromised by transmission latency, or Frame Delay.This latency is increased by many factors:• Longer distance of transmission• Slower propagation velocity of transmission• A large quantity of network elements, and longer delay introduced by each of thenetwork elements• Use of large Ethernet framesHaving taken a look at some basic elements of the protocols, now let’s delve further intowhy we may not be getting the throughput we think we should.The Basic Elements of ServiceThe user should understand the basic elements of the Service Level Agreement (SLA).Users purchase bandwidth, called Committed Information Rate (CIR), from the ServiceProvider. Because the user’s traffic may be bursty in nature, a Committed Burst Size(CBS) may also be specified to guarantee SLA performance levels on bursts of frameswhich exceed the CIR by the number of bytes in the Committed Burst Size. Note that theCIR will typically be far less than the speed of the physical interface to which the customerPage 7 of 22 March 2010

Understanding Carrier Ethernet ThroughputStandard TCP Window Size and Effect on ThroughputIn TCP, each TCP segment (name of a data packet at Layer 4) is accounted for. Thismeans that for each block of information sent across a data path, an acknowledgementmust be received before sending an additional block of data. As it would be veryineffective to send only one segment at a time and then wait for each acknowledgment,TCP has a built-in capability to send multiple segments into the network at the same time;this capability also serves as a flow control mechanism. Should a receiving host havetrouble processing all of the received data; it will delay the acknowledgment to thesending host.A graphical view of TCP flow control is shown in Figure 10. The graph shows the totalamount of memory available to the issuance of sequence numbers. The frames that aretransmitted are assigned a limited set of numbers corresponding to the cumulativenumber of bytes transmitted since the start of the session. When the total number oftransmitted bytes exceeds 2 32 , the numbering goes back to the starting number andrepeats. It is important that all the frames that are currently in transmission are receivedwith unique numbers so that they can be reassembled in the right order. In this diagram,the blue arc shows these active frames and the range of numbers that have beenallocated to them. As long as the blue arc is much smaller than the circumference of thecircle, there is no problem. To assist with transmission rate optimization, the receiveradvertises (rwnd [receiver window] advertisement) how much window it has available forthe transmitter to fill up – the transmitter is free to continue boosting its transmitted rateif the advertisement shows buffer space available and if it hasn’t received a pause framefrom the receiver.Figure 10. TCP sliding window flow control protocol (source: Wikipedia)Page 12 of 22 March 2010

Understanding Carrier Ethernet ThroughputAnother notion that must be addressed before moving forward is the effect of highbandwidth and/or high latency on the TCP protocol. As the bit rate increases, the amountof data required to fill the pipe until an acknowledgement is received grows linearly. Italso grows linearly with latency (or Frame Delay). This behavior is defined as thebandwidth-delay product. Its formula is:Pipe Capacity (bits) = bandwidth delay product = bandwidth (bits/s) x round-trip time (s)Bandwidth is the transmission rate of the link. Round-trip time is the amount of time forbits to travel across the network and back. Pipe capacity refers to the number of bits “inflight.” In other words, capacity is the maximum number of bits in transit over the link ata given time. Since TCP transmission requires acknowledgement, a sender needs tobuffer at least enough bits to continue sending until an acknowledgement is received.This sender buffer for TCP is the same as the TCP receive window, and is a maximum of64 kB for standard TCP. So for TCP, the bandwidth delay product can be rewritten asfollows:CapacityBandwidth ≤Round-Trip TimeThus, the maximum bandwidth for a TCP circuit is limited by the size of the TCP receivewindow and as well as the round-trip time. The following table provides an example of thebandwidth-delay product for a link with 40 ms round-trip latency.Circuit RatePayload rate(Mbit/s)Capacity(Kbits)Capacity(KBytes)DS1 (1.5M) 1.536 61 7.5E1 (2M) 1.984 79 9.6864KB Windows TCP max 13.1 524 64DS3 (45M) 44.21 1,768 215100BASE-T 100 4,000 488OC-3/STM-1 (155M) 150 5,990 731OC-12/STM-4 (622M) 599 23,962 2,9251000BASE-T 1,000 40,000 4,882OC-48/STM-16 (2.5G) 2,396 95,846 11,700OC-192/STM-64 (10G) 9,585 383,386 46,80010GBASE-SW (WAN) 9,585 383,386 46,80010GBASE-SR (LAN) 10,000 400,000 48,828Table 1. Bandwidth-delay product for different circuit rates with 40ms round-trip timeThe column of interest is Capacity (in KBytes). This theoretical value provides themaximum number of bytes in the system at any time so that the link is filled to themaximum and that TCP can resend any dropped or errored segments. In a standard TCPimplementation, the maximum allowable TCP window is 65,535 bytes; this means that ata rate of 13.1 Mbit/s and more, with a round-trip time of 40 ms, a server running normalPage 13 of 22 March 2010

Understanding Carrier Ethernet ThroughputTCP cannot fill the circuit at 100%. This is a theoretical maximum; unfortunately, thenetwork might drop frames along the way, making a lower payload rate more likely.Companies who buy higher speed Carrier Ethernet circuits often have hundreds of usersor processes at a location that are sharing the circuit. In this case, the circuit will havemany TCP/IP streams sharing a single circuit. The effective bandwidth that any one of theusers is using will likely be not that high, and the overall Ethernet Virtual Connectionbandwidth may be used efficiently without any TCP/IP extensions as discussed in the nextparagraph.Using Window Scaling to Increase TCP ThroughputWindow scaling, RFC 1323, is a technique used to extend TCP’s throughput. The 16-bitcounter limitation for unacknowledged frames is expanded to 32 bits, which greatlyexpands the bandwidth delay product through which TCP can be transmitted by a factor ofroughly 65,000. Although developed many years ago, window scaling is not easilyavailable to most computer end users today. Techniques exist to manually modifyoperating systems like the Microsoft Windows system registry to invoke window scaling,but that is simply beyond the capability of most users. Nonetheless, users who need toget high bandwidth performance from a single TCP/IP stream on a long Ethernet circuitshould get some help to investigate this option. Figure 11 shows the potential speedboosts available for a single TCP session. The 13 Mbit/s limit of standard TCP expands towell over 10 Gbit/s with TCP Window Scaling.1000000010000001 GByte TCP Receive Window16 MByte TCP Receive Window64 KByte TCP Receive Window16 KByte TCP Receive WindowMaximum TCP Throughput [Mbps]100000100001000100Limit of Standard TCP10110 20 30 40 50 60 70 80 90 100Round Trip Time [ms]Figure 11. TCP Window Scaling Greatly Boosts Single Session ThroughputPage 14 of 22 March 2010

Understanding Carrier Ethernet ThroughputDropped/Errored Frames and the Retransmission EffectAll transport circuits have some underlying error rate. When the bit error rate is very lowon the order of 10 -6 or much less depending on the application, users will generally notsee much service degradation. Some protocols such as TCP require a retransmission anytime an error occurs anywhere in the frame, or if the frame is dropped duringtransmission. The frame size has a magnifying effect which multiplies the impact of a biterror rate. For instance if a frame is about 120 bytes long, that amounts to about 1,000bits. A bit error rate of 10 -12 is magnified into a frame error rate of 10 -9 . The effects offrame loss (whether by bit error or dropped traffic) on overall network throughput can bemodeled as follows (Mathis, et. al.).Maximum Segment SizeThroughput ≤Round-Trip Time * √ Probability of Frame ErrorFrom this formula, we can see that we can increase the throughput by reducing the roundtrip time, reducing the frame error rate, and/or increasing the segment size. MaximumSegment Size is the size of user data. The effects of frame loss are plotted as a functionof round-trip time in Figure 12 below. Note that for very high frame error rates like 10 -2 ,throughput can be limited to less than 2 Mbit/s by frame error rate alone (for delays over60ms). Note also that the range of round trip times correspond to Carrier Ethernetservices running over longer distances (with round trip times 2 ms and larger). For delaysmuch shorter than 2ms, the store and forward time of large packets can come into play.Recall also that traffic offered to the network above the CIR is carried only on a bestefforts basis. Excess traffic may be dropped all or in part. In the case of serious defect inthe implementation of service where offered traffic greatly exceeds the CIR, half or moreof the frames may be dropped (eg policed) and protocols like TCP may completely breakdown and carry virtually no data at all.Page 15 of 22 March 2010

Understanding Carrier Ethernet ThroughputMaximum TCP Throughput {Mbps]1000000100000100001000100101E-9 Frame Error Rate1E-7 Frame Error Rate1E-5 Frame Error Rate1E-2 Frame Error Rate110 20 30 40 50 60 70 80 90 100Round Trip Time [ms]Figure 12. Frame Error/Loss Ratio Limits Maximum ThroughputMonitoring ThroughputGeneral Network Management ToolsA wealth of literature and tools guide end users and network operators in the successfulmanagement of their Ethernet networks. One good tool is flow analysis applied in theservice provider network or in the end user network to identify top talkers and see whythey are utilizing so much bandwidth. It could be that malware has entered the computerof an end user, generating excessive spam email traffic which saps private and publicnetworks alike. Or, peer to peer file exchange may be occurring in violation of copyrightlaws at the same time as absorbing great network capacity. These sorts of problems canmake users think something is wrong with their Carrier Ethernet service, when in reality,the service is working fine and provides plenty of bandwidth for proper usages. ServiceProviders can gain access to special flow analysis software and systems available in theirrouter/switch management systems that provide excellent insight into the exact real-timesources of loads on the network. End users and service providers alike are advised toconsult the rich literature on these general network management subjects. Samplegraphics from these systems are shown in Figure 13 and Figure 14.Page 16 of 22 March 2010

Understanding Carrier Ethernet ThroughputFigure 13. Flow Analysis Diagramed Through the NetworkFigure 14. Single User Spam Malware Load on NetworkPage 17 of 22 March 2010

Understanding Carrier Ethernet ThroughputVerifying Carrier Ethernet PerformanceTroubleshooting with PC Based Test ToolsSo now let’s turn to the subject of how an end user can independently verify theperformance of and troubleshoot the Carrier Ethernet service. For our new CarrierEthernet User from Figure 1 and Figure 2, how relevant was a File Transfer Protocol (FTP)download rate when trying to validate the performance of an Ethernet service? We havejust seen issues that could have caused that problem, but there are more. One must firstunderstand the FTP environment. Like any other application, an FTP session relies on theunderlying hardware, software and communication protocols. The performance of a PC isvery much aligned with its hardware and the characteristics of the CPU (speed, its cachememory, RAM). The operating system and the different background programs loaded areadditional factors. Firewalls, anti-virus and spy-ware can further limit the performance ofa PC. From an operating system perspective, this is where the OSI stack resides. Thenetwork performance of a PC is directly related to the OS it is using. By default, theTcpWindowSize registry key value is set to 65,535 bytes, which affects the TCPperformance in high-bandwidth networks. Although there are utilities such as windowscaling to increase this value, some applications, like FTP, may possibly override theTcpWindowSize registry and use the 65,535 value, thereby reducing performance.Software Based Test ToolsFreeware software tools and online bandwidth test sites receive a lot of publicity fromdifferent sources as they can help test and benchmark networks. These tools use thesame PC architecture as an FTP download test. Although the TcpWindowSize registrycould be bypassed with these tools, their performance is also directly related to the PCperformance. A PC that does not have enough RAM memory or has too many backgroundprograms loaded will perform differently than another PC that is more recent and hasmore memory. Although the measurement can provide some insight on the problems ona network, the measurement will not be as repeatable and reliable as others will withdedicated hardware.Again, the bandwidth-delay product will influence performance. If one doesn't have thecapability to extend the TCP window size, the only way to prove that a link can support100% load of TCP traffic is to start multiple test sessions. Having multiple TCP streamswill fill the link under test, but multiple TCP streams will be “fighting” for the bandwidthand may degrade the PC performance they are running on. The peak rate of all teststreams might come close to the configured throughput of the link, but looking at theaverage may show that it is way off.Finally, when using these software-based tools, bear in mind that they are operatinggenerally at layer 4 and higher, and so they will generally not provide a directmeasurement of the conformance of the Layer 2 Carrier Ethernet service to the ServiceLevel Agreement.Page 18 of 22 March 2010

Understanding Carrier Ethernet ThroughputHardware Based Test ToolsHardware based test equipment for testing Ethernet services is also available and providesdefinitive confirmation of whether the Carrier Ethernet service is performing properly.This equipment may be portable/hand-held or integrated into other CPE or networkelements. These basic Ethernet test instruments have the capability to format test trafficup to wire speed (the maximum possible line rate) for the service being tested – even forGbE and 10 GbE services. In addition, they look at traffic at layer 2, 3 and even 4 orhigher in some cases. The test sets have a dedicated OSI stack which ensures that higherlevel protocol layers or applications can utilize all the measured bandwidth. With thisequipment end users or service provider field technicians can reliably verify that they aregetting the committed information rate, frame delay, and dropped frames per the ServiceLevel Agreement (SLA) on the layer 2 Ethernet service. Technicians can make long termtests to see if the network has certain times of the day where it underperforms. The testset’s layer-2 round-trip-time measurement is the value of circuit transmission delay usedfor calculating the bandwidth delay product. If a deeper analysis of circuit performance isneeded, the test set can be used to invoke portions of the RFC 2544 or MEF test suitesrelevant to the service under test. There are many whitepapers and articles availableonline that describe these test suites in much more detail. Dedicated Ethernet test setsare quite affordable for enterprise users who really want to understand the performanceof their Ethernet service. Figure 15 shows how you can plug in these test sets to make theneeded Carrier Ethernet measurements.NIDOperator 1ENNIOperator 2(Service Provider)(OOF operator)NIDUNI100 Mbit/s EPLUNIFigure 15. Testing an EVCSummary and RecommendationsCarrier Ethernet customers can optimize their service performance as follows. First, theyshould make sure their Ethernet carrier is MEF certified to ensure delivery of a high qualityEthernet service. Then, they should make sure that they are using routers that properlylimit or shape the traffic they send to the network so that problems with discardedoverflow traffic will be avoided. The customer should ensure that their router does notburst a greater number of frames than can be safely captured according to the service’scommitted information rate and committed burst size. If they have any question about theperformance of their layer 2 Ethernet service, they or their service provider should do asimple point-to-point test with Ethernet test sets to verify that the circuit they havePage 19 of 22 March 2010

Understanding Carrier Ethernet Throughputpurchased is living up to its Service Level Agreement. Users should be aware that thethroughput indicator on a PC file transfer or online bandwidth test site is likely showingthe limitation of the TCP/IP or FTP protocol rather than the high-speed layer 2 CarrierEthernet service.End users may also want to measure the Carrier Ethernet circuit’s round trip delay (orhave the service provider measure it), so that they can calculate the upper limitbandwidth achievable per standard TCP/IP stream from Figure11. If the delay supportsthe necessary throughput, no further change is required. If not, end users may want toexplore getting help to utilize RFC 1323 window scaling or other technique to get moreperformance out of their TCP/IP and FTP protocols on their high speed circuits, especiallythose with higher latency. If end users chose to use UDP to improve throughput, theyshould bear in mind that with UDP they will get some errors on the received data, andthey will either need to accept those errors or use a higher layer in the OSI stack toensure received data integrity. End users can further tune their Carrier Ethernet Circuitby using large frames to get the most efficient use of bandwidth, or by using short framesto get the lowest possible latency.Glossary Of AbbreviationsARPATMBCPBPDUBWACBSCIRCFMCLECCPEC-TagEFMEBSEIRE-LANE-LineENNIEPLE-TreeAddress Resolution ProtocolAsynchronous Transfer ModeBridging Control ProtocolBridge Protocol Data UnitBroadband Wireless AccessCommitted Burst SizeCommitted Information RateConnectivity Fault ManagementCompetitive Local Exchange CarrierCustomer Premise EquipmentCustomer Tag (VLAN Id)Ethernet in the First MileExcess Burst SizeExcess Information RateEthernet-LAN ServiceEthernet Point-to-PointExternal Network to NetworkInterfaceEthernet Private LineEthernet Tree service (1 to many)EVCEVPLFTPEthernet Virtual ConnectionEthernet Virtual Private LineFile Transfer ProtocolGbEGiga Bit EthernetIEEE Institute of Electrical & ElectronicsEngineersIETF Internet Engineering Task ForceILEC Incumbent Local Exchange CarrierIP Internet Protocol (Layer 3)IPTV Internet Protocol TelevisionITU-T International TelecommunicationUnion – TelecommunicationStandardization SectorLANLocal Area NetworkMAC Media Access Control (layer 2protocol)MEFNew name for entity formerly knownas Metro Ethernet ForumMSO Multiple Service Operator (Comcast,COX, Time Warner Cable, etc)Page 20 of 22 March 2010

Understanding Carrier Ethernet ThroughputNIDOAMOOFOSIPCQoSRFCSLASLOSOFS-TagNetwork Interface DeviceOperations, Administration andMaintenanceOut of FranchiseOpen Systems InterconnectPersonal ComputerQuality of serviceRequest For Comment (an IETF toolfor organizing/communicatingcomments)Service Level AgreementService Level ObjectivesStart Of FrameService Tag (VLAN Id)T1T3TCPTCP/IPTDMTFTPUDPUNIVoIPVLANT-Carrier 1 (1.544Mb/s)T-Carrier 3 (44.736 Mb/s)Transport Control Protocol(Layer 4)Transport Control Protocol/InternetProtocolTime Division MultiplexingTrivial File Transfer ProtocolUser Datagram ProtocolUser to Network InterfaceVoice over Internet ProtocolVirtual LANReferencesMatthew Mathis, Jeffrey Semke, Jamshid Mahdavi. The Macroscopic Behavior of the TCPCongestion Avoidance Algorithm. Computer Communication Review, 27(3), July 1997.(http://www.psc.edu/networking/papers/model_ccr97.ps)MEF Technical Specification, MEF 11. User Network Interface (UNI) Requirements andFramework. November 2004.MEF Technical Specification, MEF 10.2, “Ethernet Services Attributes – Phase 2”,October 2009.IETF RFC 2544, Benchmarking Methodology for Network Interconnect Devices, March1999.IETF RFC 1323, TCP Extensions for High Performance, May 1992Page 21 of 22 March 2010

Understanding Carrier Ethernet ThroughputAcknowledgementsThe MEF thanks the following member companies for their contribution to thisdocumentContributorMike BugenhagenFred EllefsonCraig FantiPhil FineBrooke FrischemeierBruno GiguereSteve HolmgrenRoman KrzanowskiAyal LiorPaul MarshallSteve OlenBrian RoseAbel TongCompanyCenturyLinkADVA Optical NetworkingCanoga PerkinsCalixCiscoEXFOattVerizon BusinessIndependent ConsultantSunrise TelecomOmnitron SystemsCox CableOmnitron SystemsMore information and updates onCarrier Ethernet Servicescan be found atwww.metroethernetforum.orgPage 22 of 22 March 2010