Xcell Journal: The authoritative journal for programmable ... - Xilinx

written in a programmable latency buffer
to match the trigger latency. A number of
data verification checks are carried out. The
data is finally formatted into 64-bit words
and written into FIFOs. From there, it is
read out by the event builder on the board.
Without going into the details of the
functionality, we will focus on the various
issues we faced (and solved) in making the
real hardware churn out correct data, with
a focus on the use of RocketIO transceivers.
Much of what we learned was on a
trial-and-error basis. The main issue was
related to the reference clock, which we’ll
describe in detail in the next section.
The other significant issue that we
faced was the alignment of the K character
within the 2-byte data path of the received
data. We were initially using
the Gigabit_Ethernet primitive
in half-rate mode for a 2-byte
data path. But we observed that
not all of the channels were
putting the K character in the
same place within the 2-byte
word and there was no way to
force this alignment in the
Gigabit_Ethernet primitive
(the ALIGN_COMMA_MSB
parameter of this primitive is
set to FALSE by default).
Because our protocol expected
the K to always appear on the
LSB of the word, we switched to
the GT_CUSTOM primitive,
where we could force the alignment and subsequently
swap the position of K to the LSB
of the data. The simulations showed perfect
alignment – but in real hardware, some of
the channels were getting misaligned.
A colleague of ours referred us to the
design note about 32-bit word comma
alignment in the RocketIO transceiver user
guide. Although this is usually needed only
for a 4-byte data path, we implemented a
similar scheme for our 2-byte data path and
this fixed our misalignment problem.
Clock, Programming, and JTAG
We cannot over-emphasize the need for a
high-quality reference clock. Besides satisfying
all of the criteria specified in the
RocketIO user manual, we made sure that
... this is a flexible approach, as the
FPGAs are reprogrammable and a more
economical solution in the long term.
our reference clock was as clean as we could
possibly get (see Figure 2).
We used a quartz-based phase-locked
loop (QPLL) circuit developed at CERN
for our system to provide the best jitter-free
clock source (100 ps peak-to-peak). We
found that a lot of problems in the performance
of the RocketIO devices could be
traced to a noisy/jittery reference clock. If
you are using RocketIO transceivers on
both halves of the chip, then it’s much bet-
Figure 2 – Clock jitter measurement
ter to have two reference clocks. We believe
this helps even if you are running the
RocketIO transceivers in half-rate mode
(which is our case).
Another aspect of the clocking scheme
that we used was to pass the reference clock
through a global clock buffer after an input
global differential clock buffer. We
observed improved stability and a more
uniform distribution of the reference clock
with the FPGA editor.
Also, though not directly related to the
high-speed transceivers, we found that an
independent post-configuration DCM
reset logic (usually recommended if you
have an external feedback clock) is useful
even when using internal feedback. This
solved a problem we were having with the
DCMs where they were sometimes not
locking after reconfiguration. Xilinx
Technical Support helped us find the solution
(Xilinx Answer Record 14425).
As for programming and JTAG, we
used the same group of EPROMs to configure
eight of the nine FPGAs. One of the
FPGAs is the master and provides the clock
for all the devices in the chain. The ninth
FPGA has a different pinout and a separate
EPROM for itself.
All circuits are connected in the same
JTAG chain, which improved reprogramming
time mainly during the “test” stages.
We found that a need exists for a pull-up
resistor on the TDO output of each Xilinx
device, something that we hope Xilinx will
add in future devices. The JTAG is used
also to check the board interconnections
after assembly.
Conclusion
In this article, we’ve shown the advantages
of using embedded deserializers instead of
discrete components on a large project. By
using nine 456-pin FPGAs to do the same
job as 105 TQFPs, we saved time, both in
the design and debugging phases. Plus, this
is a flexible approach, as the FPGAs are
reprogrammable and a more economical
solution in the long term.
We are currently considering migrating
to a bigger Xilinx device as our processing
requirements from the FPGAs increase.
Therefore, we are studying the new devices
available and how such a migration will
affect our PCB design in terms of the routing
of the high-speed lines.
We believe that by following the design
rules concerning high-speed design, like
clean clock distribution, power supply
filtering, and good routing of the internal
reference clocks, it is possible to obtain a
successful design in good time. For more
information, please write to us at
jc.silva@cern.ch or adarsh.jain@cern.ch.
Winter 2004 Xcell Journal 91