Xcell Journal: The authoritative journal for programmable ... - Xilinx

We picked [9 Xilinx Virtex-II Pro devices], as it meant
a significant savings in device count (from 105 to 9).
One such experiment is called the
Compact Muon Solenoid (CMS), which is
based on a large superconducting magnet
system. The CMS will have a number of subdetectors,
including an Electromagnetic
Calorimeter (ECAL). The ECAL will use
about 80,000 crystals to capture the energy
of the photons and electrons. The data collected
from these crystals will be captured,
processed, and transmitted by the DCCs
(about 60 of them) for further analysis.
Design Overview
The DCC includes 70 high-speed optical
receiver channels (6 blocks of 12 channels
each) implemented on a 9U VME board
(36 cm x 40 cm) working at 800 Mbps
using a 2-byte 8b/10b protocol.
For the implementation of the transceivers,
we had two choices:
1. As many as 70 discreet deserializers,
along with 35 FPGAs for the required
control (this number was based on
cost considerations), for a total device
count of 105. This would have given
us more granularity and a lower cost,
but more components and hence
higher debug and testing times.
2. Only nine Xilinx ® Virtex-II Pro
devices with eight embedded
RocketIO transceivers on each (only
the XC2VP7-FG456 part was available
at the time). We would lose some granularity,
but the PCB would be much
less dense and easier to test.
We picked the second choice, as it meant
a significant savings in device count (from
105 to 9). And because the DCCs will be in
operation for four to five years, it will have a
huge impact on overall PCB design and the
final cost of production and maintenance
from a long-term perspective.
Also, after deserialization, we will need
to verify the integrity of received data and
reformat it for downstream processing and
analysis. We found that the remaining
resources in the selected device were enough
for most purposes. Of the 72 transceivers
available, we use 70 and leave the other two
unconnected. The use of 800 Mbps per
channel is a system choice, but the design
could work at 1.6 Gbps or higher.
PCB Design Issues
The DCC PCB is a 12-layer board with
four power planes and eight routing layers.
We have mostly followed the main
rules for high-speed design and analog
considerations from Chapter 4 of the
Xilinx RocketIO Transceiver User
Guide, such as:
• All high-speed traces
are impedance controlled
and routed
manually in
“microstrip-edge couple
differential pair,” with
impedance matched to
50 Ohms and as close as
possible to the source
(respecting the crosstalk
rules). No other lines were
designed in the same area
as the high-speed layout,
where the immediate layer
was the ground power plane.
• All high-speed differential
pair signals were AC coupled
with 100 nf capacitors and
internally terminated to 50 Ohms.
• All of the transceivers’ power supply
pins were filtered with an individual
LC filter and a separate power plane
for the “analog” supply, also with specific
filters. No transceiver power supply
was left unconnected, regardless of
whether it was used or not. We used
the same type of LC filters on the optical
receivers.
• Approximately 350 power supply decoupling
capacitors of three different values
(to match the main clock frequencies in
use on the board) were placed as close as
possible to the central power pins of the
Xilinx FPGAs. Other capacitors were
placed nearby each FPGA.
• Each FPGA received one high-quality
reference clock (low jitter – 100 ps
peak-to-peak) differential pair from an
individual buffer. We recommend using
two independent reference clock sources
to ease the internal usage of this clock
on the FPGA if using all of the
RocketIO transceivers.
Figure 1 – The DCC board fully assembled,
with the nine Virtex-II Pro FPGAs on the left.
RocketIO Implementation and Issues
Virtex-II Pro devices provide the first stage
of processing for the front-end data
(received from the on-detector electronics)
on the DCC board. Each device receives
800 Mbps of serial data on each of its eight
channels from the optical receivers, for a
total of 6.4 Gbps per device. In a nutshell,
the purpose of the Xilinx FPGAs is to
process this data and prepare it for readout.
RocketIO transceivers are used to deserialize
the received data and perform
8b/10b decoding. The 16-bit data is then
90 Xcell Journal Winter 2004