Using FPGAs to Design Gigabit Serial Backplanes - Xilinx

Using FPGAs to
Design Gigabit
Serial Backplanes
April 17, 2002

Outline
•System Design Trends
•Serial Backplanes Architectures
•Building Serial Backplanes with FPGAs
A1 - 2

Key System Design Trends
•Need for ….
–Easy Scalability of Performance
–Extremely High Availability
–Flexible Architecture
–Use of Standards
• Rapid Time to Market
• Reduces Costs
A1 - 3

Merging Communications & Computers
• Communications Systems have serial backplanes
• Computers moving to serial backplanes
Database servers
Application servers
1U Web servers
Ethernet Switches
Internet
“Bladed Server”, PICMG 2.16
A1 - 4

Serial Standards
• Serial Standards
– Fiber Channel 1 Gbps
– Ethernet 1 Gbps
– InfiniBand 2.5 Gbps
• Parallel Standards Going Serial
– PCI => 3GIO
– RapidIO => Serial RapidIO
– ATA => Serial ATA
• Channel Bonded Serial Standards
– XAUI 10 Gigabit Ethernet (4 x 3.125 Gbps)
3GIO
A1 - 5

Serial Connectivity = Higher
Bandwidth & Fewer Pins
1 2 3 4 5
50
Example - PCI: 32-bit x 33MHz = 1 Gbps, Shared among 5 clients 250 total pins
1 2 3 4 5
4
Virtex-II Pro System: 2.5 Gbps x 4 x 5 = 50 Gbps 80 total pins
Each Client has 2.5 Gbps guaranteed to every other Client
50x higher bandwidth with less than 1/3 of the pins
RJG
A1 - 6

Link Rates (bits per sec.)
40G
10G
3.125G
1.25G
622M
155M
•OC-3
1998
Serial Link Rates
•OC-12
•FC
•OC-48
•GigE
•OC-192
•FC-2x
•10GE
•FC-10x
•SxI-5
•3GIO
•Infiniband
•Rapid IO
•OC-768
2000 2002 2004
Mainstream Deployment
2006
A1 - 7

Serial Topologies
Switched
PICMG 2.16
Full Mesh
PICMG 2.2
PICMG 3.x
A1 - 8

Building Serial Backplanes
Virtex-II
Gigabit Ethernet
MAC x2
with FPGAs
Gigabit Ethernet
Phy x2
Switched
PICMG 2.16
A1 - 9

Virtex-II Pro
Platform FPGA
A1 - 10

Virtex-II Pro Rocket I/O
32 / 16 / 8 bits
TXDATA
32 / 16 / 8 bits
RXDATA
Technology
Physical Coding Sublayer Physical Media Attachment
C
R
C
40 – 156.3 MHz
REFCLK
CRC
Elastic
Buffer
(PCS) (PMA)
8B / 10B
Encode
F
I
F
O
Channel Bonding
and
Clock Correction
8B / 10B
Decode
Serializer
TX Clock Generator
RX Clock Recovery
Deserializer
Comma Det.
Transmitter
Transmit
Buffer
20X Multiplier
Receiver
Receive
Buffer
RX+
RX-
Up to sixteen multi-gigabit serial transceivers
Support 622 Mbps to 3.125 Gbps full duplex operation
Flexible PCS/PMA feature set
Loop-back
TX+
TX-
A1 - 11

Xilinx® Rocket I/O Serial Backplane
Interface Technology
•Rocket I/O Serial Backplane Interface (RSBI)
Technology
– Simple solution for serial backplane applications
– Consists of two parts
• The RSBI/ “Aurora” link protocol
• An RSBI Reference Design (IP core) implementing
that protocol
A1 - 12

RSBI/ “Aurora” Protocol
•An Efficient, Lightweight Protocol to Move
Data Point to Point across Serial Link
•Used to transport higher-level protocols
–Standard protocols like Ethernet
–Proprietary, customer-defined, protocols
A1 - 13

RSBI Core
• Implements RSBI/ “Aurora” Protocol
• Provides a simple interface for transferring packets
across
– A single gigabit serial link
– Up to 16 bonded links
• Minimal Use of Resources (~350 slices)
• Frequency and Technology Independent
• Specifications and Source Code available from
Xilinx
A1 - 14

RSBI/ “Aurora” Features
• Uses 8B/10B and includes CRC protection
• Transmitter wraps user data in Rocket I/O “Aurora”
protocol
– Handles packet formation
– Sends error notification to remote core
• Receiver recovers frames and realigns data
– Strips packet/frame wrappers
– Aligns data on double word boundary
– Monitors error conditions
A1 - 15

USER
RSBI User Interface
32
SIMPLE
CONTROL
32
RSBI Rocket I/O Block
32
COMPLEX
CONTROL
32
Very
High
Speed
Data
User interface created so no knowledge of Rocket I/O block is necessary
A1 - 16

RSBI Channel Bonding
• All lanes must have passed comma detect before
channel bonding starts
• Uses same technique as 10 Gigabit Ethernet and
Serial RapidIO
– Sends semi-random IDLE pattern
Channel Bonding Logic
Virtual data
channel
Synchronization of individual channels
into a single large data channel
A1 - 17

• Allows maximum frame size of 8K Unlimited bytes
• 32-bit internal interface
RSBI/ “Aurora”
Implementation Parameters
• Other frame sizes and internal interface widths
planned in future releases
A1 - 18

Error Detection in Hardware
• Errors can be either in Rocket I/O block or link(s)
• Five sources of link failure detected:
– Link physical error threshold overflow
– Channel misalignment
– FIFO overflow
• Receiver or transmitter
– Transmitter sending an illegal K character
• Upon detection of link failure, core resets itself and forces
link partner to reset
A1 - 19

RSBI Transmission
• All data frames have a SOP/SOF and EOP/EOF to
indicate the start and end of packet
• User must send at least 2 DWORDs (8 bytes)
• Core will add pad data to allow CRC functionality
• User can not transmit data in increments less than input
width
– Data can now be transmitted in any increment
A1 - 20

RSBI Reception
•Strips off SOP/SOF* and EOP/EOF**
from frame
•Strips off pad data if necessary
•Signals any errors detected
• Checks CRC for correctness
*SOP/F = Starting of Packet/Frame
**EOP/F = End of Packet/Frame
A1 - 21

Serial Backplane: Full Mesh
X 15
2VP50
RSBI layer
Rocket I/O Transceivers
Design Approach Supporting
16 slot, 3.125 Gbps per link
Full Mesh Serial Backplane
A1 - 22

Rocket I/O
Transceiver
Implementing RSBI in
Virtex-II Pro Platform FPGA
FPGA Editor
View of 2VP7
Implementation in Virtex-II Pro FPGA
• Features
– Up to 40 Gbps
– 32-bit user interface
– Low overhead
• Utilization
– 350 Slices
RSBI
Link
Rocket I/O
Interface
Auto
Negotiation &
Synchronization
Auto
Negotiation &
Synchronization
RSBI Core
Framer
Framer
Framer
Framer
User
Interface
User
App
A1 - 23

Serial Backplane:Private Connection
X 15
2VP50
Parallel
Connection
on line card
X 4
2VP2
4 Transceivers
Channel Bonded
for “Private” board
to board connection
A1 - 24

Sample Serial Backplane
4
XAUI
Switch
2VP20
Approaches
4
XAUI
Switch
XAUI Core
Rocket I/O Transceivers
Backplane
Switched
A1 - 25

TXD, TXC
RXD, RXC
Implementing XAUI in
Virtex-II Pro FPGA
Transmitter
Receiver
Management Ports
XAUI Core
Deskew
State
Machine
Sync
Sync
Sync
Sync
XAUI_TX_L0
XAUI_RX_L0
XAUI_TX_L1
XAUI_RX_L1
XAUI_TX_L2
XAUI_RX_L2
XAUI_TX_L3
XAUI_RX_L3
XAUI Floorplan –
2VP7
Rocket I/O
Transceivers
• Implementation in V-II Pro
• Uses 4 Rocket I/O Transceivers
– Channel bonding
– 8B/10B encoding
• Utilization
– 800 Slices
A1 - 26

Signal Integrity Design Resources
http://www.xilinx.com/signalintegrity
A1 - 27

Power Filtering Network
A1 - 28

Power Filtering Network Components
Capacitors
C = 0.22uF
Package: 0603 SMT
Dielectric: X7R
Tolerance: 10%
WVDC: 10V
Spacing: Within 1cm of pin
Ferrite Beads
Murata BLM11A102S
Voltage Regulator
Linear Technology LT1963
All parts specified in PCB Design Requirements
Voltage Regulator must use circuit specified by manufacturer
A1 - 29

Top of board
VTTX, VTRX
Capacitors
Ferrite Bead
A1 - 30

AVCCAUXTX, AVCCAUXRX
Bottom of board
FPGA Bypass Caps
Ferrite Bead
Capacitor
A1 - 31

Reference Clock
• Must use EPSON EG-2121CA
– 2.5V LVPECL output oscillator
– Use according to manufacture specifications
• Resistor network for clock input:
R1 510 ohm
R2 130 ohm
R3 25 ohm
R4 100 ohm
A1 - 32

Implementation Resources
• Physical (PCB) Design Information
– Signal Integrity Central on Xilinx Web Site
(http://www.xilinx.com/signalintegrity)
– Rocket I/O User Guide
– Spice Suite
• RSBI Design Information (http://www.xilinx.com/rsbi)
– Specification, Implementation notes
– Source code (Verilog)
– Test bench
• XAUI information (http://www.xilinx.com/connectivity)
– In Networking/Datapath Products (10 Gigabit Ethernet)
A1 - 33

Thank You