A Network Interface Card Architecture for I/O - IBM Haifa Labs

Technische Universität MünchenA Network Interface Card Architecture forI/O Virtualization in Embedded SystemsHolm RauchfussThomas WildAndreas HerkersdorfInstitute forIntegratedSystemsTheresienstr. 90D-8290 Munich,Germanywww.lis.ei.tum.de

Technische Universität MünchenOutline• Motivation• State of the Art for I/O virtualization• Specific requirements for embedded systems• Proposed architecture – ES-VNIC– Concept overview– Exemplary Rx packet processing– Preliminary performance estimation– Key components: Queue-Allocation and Management• Future work and summaryH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems2

Technische Universität MünchenMotivation• Virtualization is mainstream in HighPerformance Computing / Data Center– XEN, KVM, VMWare, Intel, AMD, …– I/O virtualization (IOV) is under research• Virtualization is emerging topic forembedded systems (ES)– Multiprocessor System-on-Chips (MPSoCs)– Consolidation of different, dynamicworkloads on shared platforme.g., automotive head unit Do current concepts for I/O virtualization fit for embedded systems?H. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems3

Technische Universität MünchenState of the Art for I/O virtualization – OverviewSW solutionsExtensions inNetwork InterfaceCard (NIC)VirtualMachineMonitorMulti-queuenetwork cardsDriver domainSelf-virtualizednetwork cards(VNICs)H. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems4

Technische Universität MünchenState of the Art – Virtual Machine Monitor and driver domain• Hypervisor itself (VMWare)– Increased complexity and trusted computing base– Needs own drivers• Driver domain (XEN)– Driver domain in critical data path– Latency and complex scheduling– Needs dedicatedresourcesManagement&Central I/OApps.Apps.Apps.OS (domo) OSOS (domU) OSOS (domU) OSOS (domU) OSBackendDevice DriverFrontendDevice driverFrontendDevice driverFrontendDevice driverControl-IFSecure HW-IFEvent-ChannelVirtual CPUXen Virtual Machine MonitorVirtual MMU[1]Hardware: CPUs, Memory, (IO-)DevicesH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems5

Technische Universität MünchenState of the Art – Multi-queue NICs and VNICs• Multi-queue NIC (VMDq):– Fixed number of queuepairs– Relying on driver domain– Rx scheduled by packetarrival (head of lineblocking), Tx as roundrobin[2]System Bus[3]• VNICs (RiceNIC, SV-VNIC)– Embedded systems on theirown (IXP2400, RiceNIC)– NIC-CPU/SW centric(RiceNIC)– More memory on NIC foreach additional interface[4]NICRx MAC TxDMANIC-CPUManagementDMA-Mgmt.SignalingHeader-ParsingQueueingSchedulingNIC Internal / Instruction MemoryDMASystem MemoryCPU CPUP/C ListsRx/Tx RingsPacketsH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems6

Technische Universität MünchenSpecific requirements for a NIC regarding ES and IOVExtend (only) goal of maximum throughput• Low latency• Real-time processing (for certain domains)• Differentiated service levels with signaling– Prioritization of packets and interfaces– Bandwidth guarantees• Limited HW extensions on NIC for I/O virtualization and reasonable sizecompared to actual embedded system• Offloaded I/O virtualization from VMM and domains i.e., spare CPUpower/processing New concepts/architectures are needed for I/O virtualization in embeddedsystemsH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems7

Technische Universität MünchenES-VNIC – Proposed ArchitectureSystem BusNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesCPUCPUManagementRx MAC TxHeader-ParsingFSMsSchedulingQueue-AllocSignalingDMASystem MemoryContextsP/C ListsRx/Tx RingsPacketsNIC Buffer• Pipelined, (re-)configurable and multithreaded FSMs for packet processing• System memory primary storage for configuration and data; cache on NIC Scalable and flexible resource sharing between interfacesH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems8

Technische Universität MünchenHeader parsing and bufferingNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesSystem BusCPUCPUManagement• Parsing of packet header to determinereceiving domain (minimum: MAC, VLAN)• Buffering packet on NIC as a whole• Arbitrarily access to any packet to allow outof order processing i.e., for high-prioritypacketsRx MAC TxHeader-ParsingFSMsSchedulingQueue-AllocNIC BufferSignalingDMASystem MemoryContextsP/C ListsRx/Tx RingsPacketsH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems9

Technische Universität München(Re-)Configuration via managmentNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesSystem BusCPUCPUManagement• Context define interface (handling, priority,base addresses, …)• Stored in system memory and cached foractive interfaces on the ES-VNIC• Pinning for critical interfaces• Parallel handling of packets to decreasestallingRx MAC TxHeader-ParsingFSMsSchedulingQueue-AllocNIC BufferSignalingDMASystem MemoryContextsP/C ListsRx/Tx RingsPacketsH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems10

Technische Universität MünchenQueue-Allocation and schedulingNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesSystem BusCPUCPUManagement• DMA descriptors for packets stored insystem memory and cached on ES-VNIC• Transfer of packet scheduled based onactive packets and priority (from context)Rx MAC TxHeader-ParsingFSMsSchedulingQueue-AllocSignalingDMASystem MemoryContextsP/C ListsRx/Tx RingsPacketsNIC BufferH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems11

Technische Universität MünchenDMA packet to system memoryNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesSystem BusCPUCPUManagement• Transfer of packet to system memory withDMARx MAC TxHeader-ParsingFSMsSchedulingQueue-AllocSignalingDMASystem MemoryContextsP/C ListsRx/Tx RingsPacketsNIC BufferH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems12

Technische Universität MünchenPreliminary performance estimationSystem BusSystem BusNICRxMAC TxDMANIC-CPUManagementDMA-Mgmt.SignalingHeader-ParsingQueueingSchedulingNIC Internal / Instruction MemoryDMACPU CPUSystem MemoryP/C ListsRx/Tx RingsPackets• Firmware on NIC-CPU withsequential trail of tasks Potential bottleneck• Data cache (re-)loading andinstruction fetching not optimalfor packet processing relatedtasks FSMs better suited• Pipelined architecture withserveral stages for ES-VNIC Same throughput withlower frequencyNICRx MAC TxLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesHeader-ParsingFSMsManagementSchedulingQueue-AllocNIC BufferSignalingDMASystem MemoryCPU CPUContextsP/C ListsRx/Tx RingsPacketsH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems13

Technische Universität MünchenPreliminary performance estimation (cont.)T NIC BufferTHeader−ParsingT ManagementT SchedulingTQueue−AllocT DMAT DelayRX= max(TNIC Buffer,THeader−Parsing) T max( max(T+T Management+ max(T+T DMAScheduling,TQueue−Alloc)Delta=RXT Managementmax(TT DMA)NIC Buffer,,SchedulingTHeader−Parsing, T ),Queue−Alloc),H. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems14

Technische Universität MünchenPreliminary performance estimation (cont.)TDelta=RXmax( max(TTmax(TT DMA)T ManagementNIC Buffer,,SchedulingTHeader−Parsing, T ),Queue−Alloc),• T Header-Parsing : Depend on header size• T NIC Buffer: Depend on packet size, dominantregarding T Header-Parsing• T Management : Depend on cache hit and systembus/memory (only few cycles if cached)Worst case: 64 Bytes packetsback-to-back for 1Gb Ethernet(64+20)∗8bit1Gbit/s672 nanoseconds= 84125MHz=672 nanosecondscycles!• T Queue-Alloc : Depend on cache hit and systembus/memory (only few cycles if cached)• T Scheduling : Only a few cycles• T DMA : Depends on packet size and systembus/memoryES-VNIC scales with system (bus/memory)Pinning/Prefetch for real-time interfaces T Queue-Alloc and T management are the most criticalelementsH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems15

Technische Universität MünchenSystem BusES-VNIC – Queue-AllocationNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesSystemMemoryNICFrom P/C ListsRx RingsTx RingsA B [m] C D[n]Rx MAC TxHeader-ParsingFSMsManagementSchedulingQueue-AllocSignalingDMASystem MemoryCPU CPUContextsP/C ListsRx/Tx RingsPacketsAssignableQueuesA[o]NIC BufferTo Scheduling• Rx/Tx Rings with DMA descriptors are held in system memory• Assignable queues provide cache for DMA descriptors of active interfaces• Sharing of queues between and reserving for real-time interfaces• Uneven number of Rx/Tx interfaces possible for broadcast and servicedifferentiationH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems16

Technische Universität MünchenSystem BusES-VNIC – ManagementNICLocal Cache for Contexts,P/C Lists, Rx/Tx QueuesSystemMemoryNICLocalCacheContexts…A B ZAXXX[v]A[w][m+n]To P/C ListsQueue-AllocMultithreadedFSMsTo Queue-AllocTo SchedulingRx MAC TxHeader-ParsingFSMsManagementSchedulingNIC BufferSignalingDMASystem MemoryCPU CPUContextsP/C ListsRx/Tx RingsPacketsFrom Header Parsing• Contexts are held in system memory• Fetched/cached on NIC for active interfaces• Pinning for real-time interfaces• Multithreaded FSM at the coreH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems17

Technische Universität MünchenMultithreaded, (re-)configurable FSMs• Fast switching for handling differentpackets/interfacesInput RegistersInput Input Registers Registers• Based on memory-based FSMs• Extended for changing by context– Default behavior– Adding/removing states andtransitions e.g., polling instead ofsignaling• Multiple sets of input/output registersfor holding information even whencontext has been removedContextsMemory-basedFSMInput RegistersInput RegistersOutput RegistersH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems18

Technische Universität MünchenFuture work and summary• ES-VNIC addresses I/O virtualization requirements for embedded systems– Work in progress• Validate architecture by simulation of key components– Scenarios with high dynamic (contexts, DMA descriptors, …)– Dimensioning of cache size, packet buffers, queues– Number of multithreaded FSMs and their functional verification• Implement as part of an MPSoC demonstrator in an FPGAH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems19

Technische Universität MünchenDiscussionH. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems20

Technische Universität MünchenReferences for pictures[1] M. Malhalingam, I/O Virtualization (IOV) for Dummies, VMWorld 2007[2] Intel Virtualization Technology for Connectivity, IDF 2008[3] H. Raj, K. Schwan (2007) High Performance and Scalable I/O Virtualizationvia Self-Virtualized Devices, HPDC 2007[4] J. Shafer and S. Rixner, "A Reconfigurable and Programmable GigabitEthernet Network Interface Card", Rice University Electrical and ComputerEngineering Technical Report TREE0611H. Rauchfuss – A Network Interface Card Architecture for I/O Virtualization in Embedded Systems21