Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Algorithm Acceleration<br />
Logic Emulation<br />
ASIC Verification<br />
User <strong>Manual</strong><br />
DNV6F6PCIE
DNV6F6PCIE<br />
User <strong>Manual</strong><br />
DOCUMENT PATH<br />
C:\work\dncvs\Boards\DN0200_DNV6F6PCIE\Documents\<strong>Manual</strong>\DNV6F6PCIE_manual_rev02.docx<br />
LAST SAVED BY<br />
dpalmer on FULLSAIL<br />
LAST SAVE DATE<br />
8/2/2010 12:32:00 PM
Contents<br />
1 INTRODUCTION ............................................................................................................................................. 5<br />
1.1 AUDIENCE........................................................................................................................................................ 5<br />
1.2 CONVENTIONS .................................................................................................................................................. 5<br />
1.3 RESOURCES ...................................................................................................................................................... 5<br />
2 QUICK START GUIDE ...................................................................................................................................... 8<br />
2.1 STEPS TO FOLLOW ............................................................................................................................................. 8<br />
3 HARDWARE ................................................................................................................................................. 15<br />
3.1 OVERVIEW ..................................................................................................................................................... 15<br />
3.2 VIRTEX 6 FPGA .............................................................................................................................................. 16<br />
3.3 CLOCK RESOURCES .......................................................................................................................................... 18<br />
3.4 NMB BUS ..................................................................................................................................................... 24<br />
3.5 FPGA INTERCONNECT...................................................................................................................................... 26<br />
3.6 SODIMM (DDR3) CONNECTORS...................................................................................................................... 28<br />
3.7 DAUGHTER CARDS ...................................................................................................................................... 31<br />
3.8 FPGA CONFIGURATION .............................................................................................................................. 45<br />
3.9 MARVELL CPU ............................................................................................................................................. 45<br />
3.10 MARVEL TO NMB BRIDGE ................................................................................................................................ 51<br />
3.11 RS232 ......................................................................................................................................................... 53<br />
3.12 GPIO ACCESS HEADER................................................................................................................................. 54<br />
3.13 USER LEDS .................................................................................................................................................. 56<br />
3.14 FPGA-TO-FPGA ROCKETIO ........................................................................................................................... 56<br />
3.15 SPI FLASH .................................................................................................................................................... 58<br />
3.16 USER TEST POINTS ...................................................................................................................................... 59<br />
3.17 MICTOR CONNECTOR ....................................................................................................................................... 59<br />
3.18 USER SATA .................................................................................................................................................. 60<br />
3.19 SFP AND ETHERNET .................................................................................................................................... 61<br />
3.20 ROCKETIO HEADER ..................................................................................................................................... 63<br />
3.21 ENCRYPTION ............................................................................................................................................... 65<br />
3.22 JTAG ........................................................................................................................................................... 67<br />
3.23 MECHANICAL .............................................................................................................................................. 69<br />
3.24 POWER ....................................................................................................................................................... 70<br />
3.25 RESET .......................................................................................................................................................... 74<br />
3.26 SYSTEM MONITOR ...................................................................................................................................... 77<br />
3.27 LED REFERENCE LIST ................................................................................................................................... 77<br />
3.28 TEST POINT REFERENCE LIST....................................................................................................................... 78<br />
3.29 CONNECTOR REFERENCE LIST .................................................................................................................... 79<br />
3.30 CHARACTARIZATION REPORTS ................................................................................................................... 80<br />
3.31 UNUSABLE PINS .......................................................................................................................................... 81<br />
4 SOFTWARE .................................................................................................................................................. 82<br />
DNV6F6PCIE User <strong>Manual</strong> Page 3
4.1 EMU HOST SOFTWARE ...................................................................................................................................... 82<br />
4.2 WRITING YOUR OWN SOFTWARE. ....................................................................................................................... 87<br />
4.3 HOW EMULIB WORKS ...................................................................................................................................... 87<br />
4.4 MARVEL ENVIRONMENT ................................................................................................................................... 88<br />
5 REFERENCE DESIGN ..................................................................................................................................... 95<br />
5.1 DIAGRAM ...................................................................................................................................................... 95<br />
5.2 NMB SPACE MAP........................................................................................................................................... 95<br />
5.3 THINGS TESTED............................................................................................................................................... 95<br />
5.4 COMPILING .................................................................................................................................................... 95<br />
6 TROUBLESHOOTING .................................................................................................................................... 96<br />
6.1 BOARD IS DEAD ............................................................................................................................................... 96<br />
6.2 DOES NOT RESPOND OVER PCI EXPRESS ............................................................................................................... 96<br />
6.3 MY DESIGN ACTS WEIRD ................................................................................................................................... 96<br />
6.4 IT WORKS ON ONE FPGA AND NOT OTHERS .......................................................................................................... 96<br />
6.5 PACEMAKER STOPS WORKING ............................................................................................................................ 96<br />
6.6 SIGNALS LOOKS CRAZY ON SCOPE ........................................................................................................................ 96<br />
6.7 DCM DOES NOT LOCK ...................................................................................................................................... 96<br />
6.8 DESIGN DOESN'T RESPOND OVER NMB ............................................................................................................... 96<br />
6.9 FPGAS WILL NOT PROGRAM ............................................................................................................................... 96<br />
6.10 DOES NOT BOOT .............................................................................................................................................. 96<br />
7 ORDERING INFORMATION ........................................................................................................................... 97<br />
7.1 PART NUMBER ............................................................................................................................................... 97<br />
7.2 HOW TO ORDER ............................................................................................................................................. 97<br />
7.3 BOARD OPTIONS ............................................................................................................................................. 99<br />
7.4 COMPATIBLE PRODUCTS ................................................................................................................................... 99<br />
7.5 WARRANTY .................................................................................................................................................... 99<br />
7.6 COMPLIANCE INFORMATION.............................................................................................................................. 99<br />
8 INDEX ........................................................................................................................................................ 101<br />
9 GLOSSARY ................................................................................................................................................. 102<br />
10 REVISION HISTORY .................................................................................................................................... 105<br />
DNV6F6PCIE User <strong>Manual</strong> Page 4
1 INTRODUCTION<br />
text<br />
1.1 Audience<br />
This product is marketed and sold to engineers who are familiar with circuit board design, physically<br />
probing AC waveforms, programming FPGAs, wiring HDL code, reading device data sheets,<br />
reading C source code and writing software. The provided support material all assumes that the user<br />
already has these skills.<br />
1.2 Conventions<br />
text here<br />
1.3 Resources<br />
The following list includes the resources that you are expected to make use of.<br />
1.3.1 Website<br />
The product page for this product is on the internet, here:<br />
http://dinigroup.com/index.php?product=DNV6F6PCIe<br />
This page contains:<br />
- Block Diagram of the board<br />
- Marketing Product description<br />
- List supported features<br />
- Latest Errata<br />
- Latest software and firmware update package<br />
- Latest version of this document.<br />
1.3.2 Product Package<br />
The board comes with a USB memory stick with files on it. On the root directory, there is a file<br />
called "Support Package Contents.pdf" that describes the contents and the directory structure.<br />
This package contains the software installed on the board as well as the software that should be<br />
installed on your host computer.<br />
1.3.3 Reference Design<br />
The product package contains a set of FPGA designs written in Verilog HDL that produce working<br />
configurations files for the FPGAs on the DNV6F6PCIE. Project files and batch script files that use<br />
DNV6F6PCIE User <strong>Manual</strong> Page 5
ISE to build the designs are also provided. These example files can be use to quickly create working<br />
.bit files for the FPGS.<br />
The reference design implements every feature on the board, including DDR3 memory, RocketI/O,<br />
and others. You are free to adopt any of the device controllers used in this reference design.<br />
For most customers, the most interesting part of the reference design will probably be the UCF file,<br />
which contains a list of all the usable signals connected to the FPGA and the correct IOSTANDARD<br />
attribute to use with each.<br />
1.3.4 Schematics and Netlist<br />
This user manual fails to list specifications for all of the devices connected to the FPGAs, and so to<br />
correctly use them, you will have to refer to the device datasheet and the schematic. The schematic is<br />
provided in PDF format. If you need a machine-readable format, you can use the provided ASCII<br />
netlist of the board. The ASCII netlist contains only nets on the DNV6F6PCIE that are connected to<br />
usable I/O on the FPGA.<br />
1.3.5 Device Datasheet Library<br />
There is a PDF datasheet provided for every part used on the board. It is in the user support package.<br />
1.3.6 Xilinx<br />
Questions about the use of Virtex 6 FPGAs or ISE that aren't specific to the DNV6F6PCIE should be<br />
directed to Xilinx.<br />
1.3.7 Board Models<br />
1.3.8 EMAIL AND Telephone Technical support<br />
Phone support is available pacific standard time from 9AM to 5PM from Monday through Friday,<br />
excluding USA federal holidays. Support is available in English. Support for boards purchased<br />
through distributors can additionally be provided by the distributor. Distributors are listed in the<br />
ordering information section.<br />
Telephone (USA): 858-454-3419<br />
Formal technical support<br />
support@dinigroup.com<br />
Expertise Name Email Ext. WoW Class (Level)<br />
Schematic David Palmer dpalmer@dinigroup.com 30 Jester (30)<br />
Verilog Jack Fan jack@dinigroup.com 22 Rogue (14)<br />
Sales Mike Dini mdini@dinigroup.com 11 Barbarian (8)<br />
Production Dela "tats" Cruz dela@dinigroup.com 15 Sorceress (44)<br />
Quantum Physics Ivan Yulaev ivany@dinigroup.com 12 Wizard (19)<br />
Host Software Neal Harder nharder@dinigroup.com 28 Paladin (21)<br />
Marvell Software David Palmer dpalmer@dinigroup.com 30 Jester (30)<br />
DNV6F6PCIE User <strong>Manual</strong> Page 6
1.4 Errata<br />
The circuit board is currently in revision number: 02<br />
Errata exists for revision 01 of the circuit board.<br />
Issue: IDE connector doesn't physically fit on the "ACCESS A" header.<br />
Solution: You must cut the key off your cable connector.<br />
Issue: Real Time clock does not keep time between power-down cycles.<br />
Solution: None.<br />
Issue: Board may not power on immediately after powering off. When this condition occurs, the<br />
board will remain in a "reset" condition, and will be unusable.<br />
Solution: You may need to wait up to 5 seconds after powering down the board before it can be<br />
powered on again.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 7
2 Quick Start Guide<br />
This section will walk through an example session using the board.<br />
2.1 Steps to Follow<br />
Follow them.<br />
2.1.1 Examine Contents of Box<br />
The box containing the product should have come with the following units:<br />
DNV6F6PCIE board<br />
RS232 serial Cable<br />
DB9-to-IDC cable adapter<br />
Two PCI Express power cable adapters<br />
USB Stick containing user support package<br />
DNV6F6PCIE User <strong>Manual</strong> Page 8
USB Stick containing FPGA .bit files and a shell script.<br />
PSU "starter" device<br />
2.1.2 Before you power on<br />
Place the board on a clear desk with static control padding. You can use the silver-colored bag the<br />
board comes wrapped in as a static control surface. Make sure you neutralize the static in your<br />
fingers with the surface before every time you contact the board.<br />
2.1.3 Install the board in a PCI Express slot<br />
This step is optional. During the course of your project, if you intend to control the board using PCI<br />
Express, then you should complete this step.<br />
The board fits into any 4x, 8x or 16x PCI Express slot. Make sure the computer is powered off when<br />
you install the board into it. It is recommended that you have the computer laying down so that the<br />
DNV6F6 is oriented vertically to reduce physical stresses on the board.<br />
2.1.4 Connect Ethernet Cable<br />
This step is optional. During the course of your project, if you intend to use Ethernet to control the<br />
board, then you should complete this section.<br />
Have a computer network. In order to be able to access the board over the network, the network must<br />
support DHCP. Otherwise, the board will fail to have a usable IP address. Connect the RJ45<br />
connector on the DNV6F6 to your network.<br />
2.1.5 Connect USB Cable<br />
This step is optional. During the course of your project, if you intend to control the board directly<br />
from a computer over USB, then you should complete this step.<br />
Connect a USB cable from the host computer to the "USB type B" connector on the board. If the<br />
board is plugged into PCI Express, the computer that connects to the board with USB does not<br />
necessarily need to be the same computer.<br />
2.1.6 Connect power cables<br />
You need a computer power supply to supply power to the DNV6F6. If the DNV6F6 is installed<br />
inside a computer in a PCI Express slot, then you can use the power supply that powers the rest of<br />
the computer system. Alternately, the power supply can be sitting on a desktop.<br />
The board requires two "PCI Express" Power cables to be plugged in to operate. If you only use a<br />
single cable, the board will fail to power up properly. Most modern computer power supplies have at<br />
least two PCI Express power cables. If your power supply does not, you can use the provided adapter<br />
cables that plug into the "hard drive" power cables.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 9
When the board is plugged into a PCI Express slot, the two "PCI Express" power connectors are still<br />
required. If you power on the computer without the power connected to the board, then the board<br />
will not be accessible over PCI Express<br />
2.1.7 Power on the board<br />
Turn on the power supply. If the power supply is sitting on a desktop, then the power supply will not<br />
turn on without a "PSU starter" device. One has been provided for you.<br />
The board has a self-boot process that takes approximately one minute.<br />
2.1.8 Using a USB pen drive to brutally control the board<br />
The board is provided with a USB pen drive that has on it a Linux shell script. If you plug the USB<br />
stick into the board, then the board will automatically run the shell script. The shell script on the<br />
provided pen drive will cause the FPGAs to load with the reference design .bit files. You can tell that<br />
the .bit files are loading because after each FPGA configures, a blue LED will appear on the board.<br />
2.1.9 Host Software<br />
Whether the board is connected to a computer using USB, PCI Express or Ethernet, the board is<br />
controlled using a program that Dini Group has provided called "Emu". The Emu program is<br />
provided in source and as binaries on the user support package.<br />
The rest of this guide will assume you are using a Windows computer, however you can also use<br />
Linux. If you are using Linux the instructions may be slightly different.<br />
If you are using PCI Express then you need to install a PCI Express driver. This can be done in<br />
Windows using the "device manager". The driver files are provided in the support package<br />
D:\Host_controller_software\emu\drivers\pci_win32. Search the internet if you are unsure how to<br />
install a driver from device manager.<br />
If you are using USB then you will need to install a USB driver. This can be done in Windows using<br />
the "device manager". The driver files are provided in the support package<br />
D:\Host_controller_software\emu\drivers\usb_win32. Search the internet if you are unsure how to<br />
install a driver from the device manager.<br />
Ethernet does not require a driver.<br />
2.1.10 Selecting a board<br />
Run the provided "emu" program, located in the user support package here:<br />
D:/Host_software_applications/Emu/App/Bin/Emu_gui_win32.exe<br />
This window will appear.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 10
From the "Board" menu, chose "select board". If you are connected to the board over PCI Express,<br />
USB and Ethernet simultaneously, then there will be three options in the pull-down menu. Each<br />
interface is treated like a separate board. From the pull-down menu, you can see the serial number of<br />
each board. The serial number in this menu should match the serial number located on a sticker near<br />
DIMM D of your board.<br />
Once you have selected a board, your window should look like this.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 11
2.1.11 Configure an FPGA<br />
You can configure and FPGA by clicking on the image of it in EMU and selecting "configure" from<br />
the pop-up window. There are some example .bit files that you can use in the support package<br />
located at<br />
D:\FPGA_reference_designs\bitfiles\<br />
Be sure to choose bit files that are compiled for the correct type of FPGA that you have installed on<br />
the board, to avoid humiliation and ridicule.<br />
After the FPGA successfully configures, a blue dot will appear next to any configured FPGA.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 12
2.1.12 Setting board controls and options<br />
The primary board settings that you will need to modify are the clock settings. There are six clocks<br />
on the board that have modifiable options. Let's change the clock frequency of G0 for kicks.<br />
In the EMU window, click on the right side where the says "CLOCKS: G0". A pop-up menu will<br />
allow you to change the frequency of clock G0. You can also change the frequency using the<br />
"Clocks/Temps" menu in the menu bar. The clock frequency of the six main clocks is constantly<br />
measured and displayed on the screen for your intense pleasure.<br />
2.1.13 Hardware Verification Test<br />
To run the hardware test, from the "Test" menu, select "selected tests".<br />
The tests that you can run now are the temperature test, clock test, blockram test, intercon test, lvds<br />
test and flash test. Let's skip the DRAM test and the "factory tests" for now. After you hit the "OK"<br />
button, the program will ask you to locate the "bit file directory". This is where the test FPGA load<br />
files are stored. There is a "bit file directory" on the provided user support package in<br />
D:\FPGA_Reference_designs\bitfiles<br />
DNV6F6PCIE User <strong>Manual</strong> Page 13
Once the test runs, the Emu window will indecisively print "PASS" or "FAIL"<br />
DNV6F6PCIE User <strong>Manual</strong> Page 14
3 Hardware<br />
This section, more than any other, describes the board hardware.<br />
3.1 Overview<br />
Below is a block diagram of the board.<br />
Above is a block diagram of the board.<br />
The board contains six Xilinx Virtex-6 FPGAs in the "FF1759" package. There are 5 different Xilinx<br />
part numbers that come in this package. All 5 of these are compatible with this board. The board can<br />
come with any number from 1 to 6 of FPGAs installed, leaving the unused chip positions vacant.<br />
The Virtex 5 "Config FPGA" is not intended for your use, so you should think of it as more of a<br />
"NMB master controller/ bus switch"<br />
Interconnect between the FPGAs is fixed, and routed in a point-to-point fashion. The interconnect is<br />
represented in the block diagram as arrows between the FPGAs. All of the interconnect is userdefined.<br />
Notice that some of the interconnect on the board is colored gold instead of black. The<br />
yellow interconnect is only available for use if both of the two FPGAs to which it connects is a<br />
"large package" FPGA, namely LX550T or SX475T.<br />
To connect to other systems, or off-board I/O, there are three "daughtercard" connectors provided. In<br />
the block diagram these appear as yellow rectangles unless you are colorblind then they are grey.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 15
The user is expected to buy a daughter card that contains the I/O interface that is required, or to<br />
design their own.<br />
Four DDR3 sockets (red rectangles) are on the board to provide bulk memory for use in the FPGA.<br />
You can use standard, off-the-shelf laptop memory (SODIMM), or you can use one of the many<br />
memory technology DIMMs that the Dini Group provides.<br />
Getting data on and off the board is accomplished through the Marvell CPU. It provides USB,<br />
Ethernet, PCI Express, interfaces. It connected to the user FPGAs through the "NMB" interface,<br />
which is fast enough to sustain a full-speed x4 PCI Express connection to all FPGAs simultaneously.<br />
The interface inside the FPGA and on the host PC is very simple, because all of the software and<br />
hardware between has already been designed, proven, and optimized.<br />
3.2 Virtex 6 FPGA<br />
Virtex 6 is the most slightly better than Virtex 5 FPGA in the world. You will definitely want six of<br />
them.<br />
3.2.1 Stuffing Options<br />
Each of the six main FPGA locations can be installed with any compatible density of FPGA, or not<br />
installed at all, in any combination. In this way, you only will have to pay the (significant) FPGA<br />
cost for the FPGAs that you will actually use.<br />
Below there is a table that describes the major differences between the available FPGAs. Only<br />
FPGAs that Xilinx sells in the "FF1159" package are compatible with this board.<br />
FPGA Speed<br />
Grades<br />
Flip-flops Equivalent<br />
ASIC Gates<br />
All Board<br />
Features?<br />
25x18<br />
multipliers<br />
Memory<br />
(bits)<br />
LX550T 1L, 1, 2 687,360 4.0M Y 864 22.8M<br />
LX360T 1L, 1, 2, 3 455,040 2.6M N 576 15.0M<br />
LX240T 1L, 1, 2, 3 301,440 1.7M N 768 15.0M<br />
SX475T 1L, 1l, 2 595,200 3.4M Y 2016 38.3M<br />
SX315T 1L, 1, 2, 3 394,000 2.3M N 1344 25.3M<br />
Each LX550T FPGA can emulate approximately 4 million ASIC gates reasonably, however I just<br />
made this number up. It is strongly recommended that you synthesize your actual ASIC design,<br />
mapping to FPGA technology to get an accurate FPGA utilization estimate.<br />
3.2.2 Speed Grades<br />
Xilinx FPGAs usually come in three speed grades. There is no rule of thumb to estimate which speed<br />
grade you will need to run your design at your target frequency. You will only know this once you<br />
have run a synthesis, with FPGA place-and-route, targeting the actual FPGA device skew that you<br />
will be using.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 16
3.2.3 Upgrades<br />
If you would like to install only some FPGAs when you order the board, and later add FPGAs or<br />
upgrade FPGAs to larger parts, this is possible; however you should request this before ordering the<br />
board.<br />
3.2.4 Small FPGAs<br />
When one or more of the FPGA locations is populated with a LX360T, LX240T, or SX315T, the<br />
some features of the board become unavailable. This is because these three FPGAs have fewer I/O<br />
than physically exist on the 1759-pin BGA package. The on-board devices that the unused physical<br />
package pins are connected to cannot be used if the populated FPGA is one of these "small" FPGAs.<br />
On the product block diagram, signals that may be unusable due to "small" FPGAs are colored<br />
orange. Below there is a copy of the block diagram, with all orange signals removed. The block<br />
diagram below represents the features available on a board, even if "small" FPGAs are selected.<br />
3.2.5 Safe Handling of FPGAs<br />
There are three easy ways to break the FPGAs.<br />
1) Static electricity<br />
Make sure you keep the board on a static controlled surface, and that you neutralize your body with<br />
that surface before handling the board. Especially sensitive are the FPGA I/Os. These are exposed on<br />
the daughter card headers and also everywhere else.<br />
2) High Voltage<br />
The FPGA I/O can only withstand voltages below and up to the VCCO power supply. When<br />
interfacing the board to some external I/O, make sure your I/O signals are driven at levels that do not<br />
exceed VCCO. If you do not know what VCCO is, then you probably should not be interfacing the<br />
DNV6F6PCIE User <strong>Manual</strong> Page 17
oard to some external I/O. Note that the maximum allowable VCCO on Virtex 6 is 2.5V. If you are<br />
interfacing with a 3.3V device, then you automatically lose.<br />
3) Board warp<br />
If the board undergoes mechanical stress, the FPGA pins (balls) can separate from the PCB and<br />
result in non-connected signals. The only way I have seen people doing this is by installing and<br />
removing connectors. Make sure that when installing a connector, you are supporting the connector<br />
from the opposite side, so that board warp does not absorb the force of the insertion.<br />
3.3 Clock Resources<br />
The board provides clocks. Clocks are one of the features that board provides. There are clocks on<br />
the board. You can use the clocks that are on the DNV6F6 for clocking.<br />
3.3.1 Clock pins on the FPGA<br />
The Virtex 6 has many fewer "global clock" (GC) pins that previous generations. Instead there are<br />
"Clock Capable" (CC) pins that have restrictions on how they are used. Only some of the "global<br />
clock" networks on the DNV6F6 are connected to "GC" pins on the FPGA. The rest are connected<br />
on "clock capable" pins. A "clock capable" pin might be a MRCC ("multiple region clock capable")<br />
or SRCC ("single-region clock capable"). You will have to consult a datasheet to tell the difference.<br />
The use of each type of pin, MRCC, SRCC, GC result in different effects on timing.<br />
Also, banks 10-18 have different timing than banks 20-38. All "global clock" pins on the DNV6F6<br />
are connected to banks 20-38.<br />
It would be difficult and misleading if I tried to explain how clocks worked internally in a Virtex 6,<br />
so you will need to consult the data book. We tried to connect them to make them as useful and<br />
flexible as possible.<br />
3.3.2 Global Networks<br />
The "global clocks" are the clocks that are provided to all 6 FPGAs, with low skew between the<br />
arrival of the clock pulses at each FPGA. There are 6 such networks. USER_R, USER_L, G0, G1,<br />
G2, and CLK_25<br />
DNV6F6PCIE User <strong>Manual</strong> Page 18
Each of them is suitable for synchronous communication between FPGAs.<br />
3.3.3 Clocks G0, G1, G2<br />
The clocks G0, G1 and G2 are from a synthesizer that can produce any frequency from 2KHz to<br />
700MHz with a 50ppm tolerance or better.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 19
The synthesizer used is a high-performance, low jitter, high-precision clock generator chip, the<br />
Si5326. To change the clock frequency you can use the EMU software.<br />
The clocks G0, G1 and G2 can also be set to come from the config FPGA. The config FPGA in turn,<br />
can be set to source this clock from the FPGAs. In this way, FPGAs can drive the frequency onto the<br />
global clock networks. This can be useful for controlled clocks and step-clocks. It can also be useful<br />
when a local clock for an FPGA needs to be delivered to all 6 FPGAs with zero delay.<br />
G0 can be sourced from FPGA A or FPGA D. The signal that the FPGA should drive when this<br />
mode is used is called CLK_TO_SPARTAN_Ap and CLK_TO_SPARTAN_Dp respectively.<br />
G1 can be sources from FPGA B or FPGA E. The signal that the FPGA should drive when this mode<br />
is used is called CLK_TO_SPARTAN_Bp and CLK_TO_SPARTAN_Ep respectively.<br />
G2 can be sourced from FPGA C or FPGA F. The signal that the FPGA should drive when this mode<br />
is used is called CLK_TO_SPARTAN_Cp and CLK_TO_SPARTAN_Fp respectively.<br />
3.3.4 CLK_25<br />
This clock network is fixed at 25Mhz. You cannot change it, try as you might.<br />
68 CLK_25_SOURCEp<br />
68 CLK_25_SOURCEn<br />
From Spartan<br />
+3.3V<br />
C394<br />
0.1uF<br />
R1018<br />
4.7K<br />
C386<br />
0.1uF<br />
R995<br />
100R<br />
BUF_24_OE<br />
C385<br />
0.1uF<br />
Fixed<br />
Frequency<br />
16<br />
15<br />
22<br />
18<br />
21<br />
U35<br />
CLK<br />
nCLK<br />
OE<br />
17<br />
19<br />
VDD<br />
20<br />
VDD<br />
VDD<br />
GND<br />
GND<br />
14<br />
Q0<br />
13<br />
nQ0<br />
12<br />
Q1<br />
11<br />
nQ1<br />
10<br />
Q2<br />
9<br />
nQ2<br />
8<br />
Q3<br />
7<br />
nQ3<br />
6<br />
Q4<br />
5<br />
nQ4<br />
4<br />
Q5<br />
3<br />
nQ5<br />
2<br />
Q6<br />
1<br />
nQ6<br />
24<br />
Q7<br />
23<br />
nQ7<br />
ICS85408<br />
SOP65P640X120-24N<br />
CLK_25_TPp<br />
CLK_25_TPn<br />
TP57<br />
DNI<br />
R994<br />
100R<br />
CLK_25_Ap 60<br />
CLK_25_An 60<br />
CLK_25_Bp 60<br />
CLK_25_Bn 60<br />
CLK_25_Cp 60<br />
CLK_25_Cn 60<br />
CLK_25_Dp 60<br />
CLK_25_Dn 60<br />
CLK_25_Ep 60<br />
CLK_25_En 60<br />
CLK_25_Fp 60<br />
CLK_25_Fn 60<br />
CLK_25_CONFIGp 68<br />
CLK_25_CONFIGn 68<br />
LVDS<br />
To<br />
all<br />
FPGAs<br />
3.3.5 USER_L<br />
This clock has no frequency synthesizer, but can come from a variety of sources.<br />
TP15<br />
LVDS<br />
DNI<br />
LVDS<br />
U26<br />
U25<br />
2<br />
16<br />
CLK_USER_LEFTp<br />
R564 16<br />
14 CLK_USER_LEFT_TPp<br />
42 CLK_USER_LEFT_OUTAp<br />
3<br />
CLK0p Q0p<br />
15<br />
CLK_USER_LEFTn<br />
100R 15<br />
CLK Q0<br />
13 CLK_USER_LEFT_TPn<br />
4<br />
VT0<br />
Q0n<br />
nCLK nQ0<br />
12<br />
42 CLK_USER_LEFT_OUTAn<br />
CLK0n<br />
Q1<br />
CLK_USER_LEFT_Ap 9<br />
LVDS<br />
11<br />
nQ1<br />
CLK_USER_LEFT_An 9<br />
7<br />
LVDS<br />
+3.3V<br />
10<br />
60 CLK_USER_LEFT_OUTDp<br />
CLK_USER_LEFT_Bp 44<br />
8<br />
CLK1p<br />
Q2<br />
9<br />
nQ2<br />
CLK_USER_LEFT_Bn 44<br />
9<br />
VT1<br />
R548<br />
8<br />
60 CLK_USER_LEFT_OUTDn<br />
CLK1n<br />
Q3<br />
CLK_USER_LEFT_Cp 45<br />
4.7K<br />
7<br />
BUF_USER_LEFT_OE 22<br />
nQ3<br />
CLK_USER_LEFT_Cn 45<br />
14<br />
6<br />
44 CLK_USER_LEFT_OUTBp<br />
OE Q4<br />
CLK_USER_LEFT_Dp 42<br />
13<br />
CLK2p<br />
5<br />
+2.5V<br />
17<br />
nQ4<br />
CLK_USER_LEFT_Dn 42<br />
12<br />
VT2<br />
4<br />
44 CLK_USER_LEFT_OUTBn<br />
CLK2n<br />
19<br />
VDD Q5<br />
CLK_USER_LEFT_Ep 47<br />
3<br />
20<br />
VDD nQ5<br />
CLK_USER_LEFT_En 47<br />
19<br />
2<br />
3 CLK_USER_LEFT_SPARTANp<br />
VDD Q6<br />
CLK_USER_LEFT_Fp 45<br />
18<br />
CLK3p<br />
1<br />
CLK_USER_LEFT_Fn 45<br />
17<br />
VT3<br />
18<br />
nQ6<br />
24<br />
3 CLK_USER_LEFT_SPARTANn<br />
CLK3n<br />
CLK_USER_LEFT_Qp 68<br />
21<br />
GND Q7<br />
23<br />
6<br />
GND nQ7<br />
CLK_USER_LEFT_Qn 68<br />
3 CLK_USER_LEFT_MUX_SEL0<br />
5<br />
SEL0<br />
C127 C139 C128<br />
3 CLK_USER_LEFT_MUX_SEL1<br />
SEL1<br />
0.1uF 0.1uF 0.1uF<br />
ICS85408<br />
10<br />
1<br />
SOP65P640X120-24N<br />
11<br />
GND<br />
VDD<br />
20<br />
GND<br />
VDD<br />
C145<br />
ICS854057/TSSOP20<br />
0.1uF<br />
R570<br />
100R<br />
To<br />
all<br />
FPGAs<br />
DNV6F6PCIE User <strong>Manual</strong> Page 20
The following list are the available sources for the clock USER_L<br />
FPGA A SRC The FPGA A should drive the signal<br />
CLK_USER_LEFT_OUTAp/n differentially<br />
FPGA B SRC The FPGA B should drive the signal<br />
CLK_USER_LEFT_OUTBp/n differentially<br />
FPGA D SRC The FPGA D should drive the signal<br />
CLK_USER_LEFT_OUTDp/n differentially<br />
SMA The user should supply a clock single-ended into the SMA P36,<br />
located on the bottom right of the board. Voltages up to +2.5V<br />
are acceptable.<br />
MGT<br />
The clock will be the same frequency as the "MGT" clock. This<br />
has a dubious and unknown use.<br />
3.3.6 USER_R<br />
These come from a USER FPGA. They are used for generating a frequency from an FPGA and then<br />
using that new frequency across the board. This is just like USER_L, except there are different<br />
choices for the inputs.<br />
CONN_SMA<br />
LIGHTHORSE_SASF546-P26-X1<br />
J5<br />
3 4<br />
1<br />
2 5<br />
CONN_SMA<br />
LIGHTHORSE_SASF546-P26-X1<br />
J6<br />
3 4<br />
1<br />
2 5<br />
+2.5V<br />
C702<br />
R456<br />
4.7K<br />
R494<br />
4.7K<br />
1uF<br />
3<br />
3<br />
60<br />
60<br />
60<br />
60<br />
45<br />
45<br />
LVDS<br />
CLK_USER_RIGHT_OUTCp<br />
CLK_USER_RIGHT_OUTCn<br />
CLK_USER_RIGHT_OUTEp<br />
CLK_USER_RIGHT_OUTEn<br />
CLK_USER_RIGHT_OUTFp<br />
CLK_USER_RIGHT_OUTFn<br />
CLK_SMA_USER_LEFTp<br />
CLK_SMA_USER_LEFTmid<br />
CLK_SMA_USER_LEFTn<br />
CLK_USER_RIGHT_MUX_SEL0<br />
CLK_USER_RIGHT_MUX_SEL1<br />
Clock MUXes<br />
2<br />
3<br />
4<br />
7<br />
8<br />
9<br />
14<br />
13<br />
12<br />
19<br />
18<br />
17<br />
6<br />
5<br />
10<br />
11<br />
U23<br />
CLK0p<br />
VT0<br />
CLK0n<br />
CLK1p<br />
VT1<br />
CLK1n<br />
CLK2p<br />
VT2<br />
CLK2n<br />
CLK3p<br />
VT3<br />
CLK3n<br />
SEL0<br />
SEL1<br />
GND<br />
GND<br />
Q0p<br />
Q0n<br />
LVDS<br />
VDD<br />
VDD<br />
ICS854057/TSSOP20<br />
16<br />
15<br />
1<br />
20<br />
LVDS<br />
+2.5V<br />
CLK_USER_RIGHTp<br />
CLK_USER_RIGHTn<br />
C91<br />
0.1uF<br />
+3.3V<br />
C105<br />
0.1uF<br />
R533<br />
4.7K<br />
C99<br />
0.1uF<br />
BUF_USER_RIGHT_OE<br />
C103<br />
0.1uF<br />
R523<br />
100R<br />
16<br />
15<br />
22<br />
17<br />
19<br />
20<br />
18<br />
21<br />
U24<br />
CLK<br />
nCLK<br />
OE<br />
VDD<br />
VDD<br />
VDD<br />
GND<br />
GND<br />
Q0<br />
nQ0<br />
Q1<br />
nQ1<br />
Q2<br />
nQ2<br />
Q3<br />
nQ3<br />
Q4<br />
nQ4<br />
Q5<br />
nQ5<br />
Q6<br />
nQ6<br />
Q7<br />
nQ7<br />
14<br />
13<br />
12<br />
11<br />
10<br />
9<br />
8<br />
7<br />
6<br />
5<br />
4<br />
3<br />
2<br />
1<br />
24<br />
23<br />
ICS85408<br />
SOP65P640X120-24N<br />
LVDS<br />
TP12<br />
DNI<br />
CLK_USER_RIGHT_TPp<br />
CLK_USER_RIGHT_TPn<br />
CLK_USER_RIGHT_Ap 9<br />
CLK_USER_RIGHT_An 9<br />
CLK_USER_RIGHT_Bp 44<br />
CLK_USER_RIGHT_Bn 44<br />
CLK_USER_RIGHT_Cp 45<br />
CLK_USER_RIGHT_Cn 45<br />
CLK_USER_RIGHT_Dp 42<br />
CLK_USER_RIGHT_Dn 42<br />
CLK_USER_RIGHT_Ep 47<br />
CLK_USER_RIGHT_En 47<br />
CLK_USER_RIGHT_Fp 45<br />
CLK_USER_RIGHT_Fn 45<br />
CLK_USER_RIGHT_Qp 68<br />
CLK_USER_RIGHT_Qn 68<br />
R511<br />
100R<br />
To<br />
all<br />
FPGAs<br />
FPGA C SRC<br />
FPGA E SRC<br />
FPGA F SRC<br />
SMA<br />
The FPGA C should drive the signal CLK_USER_LEFT_OUTCp/n<br />
differentially<br />
The FPGA E should drive the signal CLK_USER_LEFT_OUTEp/n<br />
differentially<br />
The FPGA F should drive the signal CLK_USER_LEFT_OUTFp/n<br />
differentially<br />
The user should supply a clock single-ended or differential into the<br />
SMAs J5 and/or J6, located near the top left corner of the board.<br />
Voltage levels up to +2.5V are acceptable.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 21
3.3.7 Frequency-only networks<br />
Frequency only networks are networks that are provided to all six FPGAs, but do not guarantee lowskew<br />
between the inputs to the FPGAs. These networks should not be used for fully-synchronous<br />
communication between FPGAs, at least not without phase-adjustment.<br />
3.3.7.1 MGT<br />
The MGT clock network delivers a very low-jitter, high-precision frequency source to the MGT<br />
(GTP, GTX, HTX) tiles of all six FPGAs. This clock is intended to be used only for the RocketIO<br />
interconnect between FPGAs, however the clock is accessible for other types of logic inside the<br />
FPGA.<br />
The MGT network can be driven at one of four different frequencies. You can select the desired<br />
frequency from EMU. Additionally, you can run the MGT network at the same frequency as global<br />
clock G0. This will allow you to select any frequency that exists in the world.<br />
3.3.7.2 CCLK<br />
The "CCLK" pin on the FPGA, or the configuration clock, is used by the config FPGA to send<br />
configuration bitstreams to the FPGA over the selectmap bus. It is not a free-running clock, but has a<br />
minimum period of 20ns. It can be used in the FPGA in conjunction with the STARTUP_V6<br />
primitive. This clock is not configurable.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 22
3.3.8 Local Networks<br />
Local networks are networks that are only delivered to a single FPGA.<br />
3.3.9 CLK_TO_*<br />
Some FPGAs have signals that connect from on FPGA to another FPGA's global clock input pin.<br />
These signals are single ended and are called "clk_to_*" where * is either A, B, C, D, E or F. These<br />
can be used for forward a clock from one FPGA to another, without having to use "local routing"<br />
within the FPGA. The utility of this does not exceed 3 utils.<br />
3.3.10 Spartan/TP<br />
There is a test point connected to at least on clock input of each FPGA. There is no known use.<br />
3.3.11 Daughtercard Feedback<br />
Each FPGA that has a daughter card connector also has a signal that loops back from an FPGA<br />
output to a clock input of that same FPGA. The routing length of this signal is the same as the<br />
routing length of the I/O signals to the daughter card. The purpose of this is so that it is possible to<br />
have a clock in the FPGA that is phase-aligned to a clock arriving at the daughter card. The signal is<br />
called "DC*_FEEDBACK_P/N"<br />
3.3.12 DIMM Feedback<br />
Similarly, each FPGA that has a DIMM interface has a signal that is looped back from an output of<br />
the FPGA to a clock input of the FPGA. The routing length of this feedback is equal to the routing<br />
length of the signals to the DIMM connector. In this way, it is possible to have a clock inside the<br />
FPGA that is phase aligned with the arrival of the clock at the DIMM. The signal is called<br />
"DIMM*_FB_P/N"<br />
DNV6F6PCIE User <strong>Manual</strong> Page 23
3.3.13 From SEAM connector<br />
The SEAM daughter card provides four clock inputs delivered to the MGT (GTP, GTX, HTX) tiles<br />
of the connected FPGA. This clock is intended to be used for RocketIO communication with the<br />
SEAM interface, however it can be used in the FPGA for other types of logic.<br />
3.3.14 NMB<br />
The NMB interface includes one clock signal running at xxxxxx MHz. This clock is free-running,<br />
and is not configurable. It is received by each FPGA at the same frequency, however the phase<br />
relationship between the arrival of the clock at each FPGA is indeterminate.<br />
3.3.15 Generating clocks from FPGAs<br />
Notice how earlier I said that some of the clock networks can be driven from FPGAs? Well that<br />
means you can do all of your frequency generation in an FPGA.<br />
3.3.16 Clocking features not implemented<br />
Four of the networks can be driven from the configuration FPGA. If you need some special feature,<br />
then we could potentially add it. For example, single-step clocks, clocks from FPGA-to-FPGA.<br />
3.4 NMB Bus<br />
The NMB bus is the primary means you will use to get high quantities of data on and off the board.<br />
If you want to use the provided software (EMU) to push data to the board over USB, Ethernet and<br />
PCI Express, then you are required to interface to NMB in your FPGA design.<br />
3.4.1 Protocol<br />
You are expected to know nothing about the protocol of NMB and only interface to it using the<br />
provided HDL interface wrapper in your FPGA on one end, and in the EMU C++ code on the other<br />
end. However, the marketing material constantly makes reference to the inner workings of the<br />
underlying hardware and software, so I feel obligated to sort of describe it a little. This short section<br />
describes the implementation of the interface. I recommend you skip this "protocol" section. It is<br />
useless to know anything here.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 24
There is a block diagram of the NMB bus architecture above. It is physically point-to-point from<br />
each FPGA to the configuration FPGA. Each point-to-point connection consists of a 40-pin signal<br />
wire, which are used as 20 LVDS pairs. These pairs are further divided into 10 signals in each<br />
direction, with 1 clock signals, 1 control signal and 8 data signals. The clock frequency of the<br />
interface happens to be 1GHz (500 MHz clock with DDR capture). The theoretical throughput is<br />
therefore 1GB/sec in both directions simultaneously, to all 6 FPGAs simultaneously.<br />
The protocol supports four channels, demand mode, bursts, interrupts, link detection, some FIFO<br />
flags, and maybe some other stuff. The data to/from the FPGA is stored in buffers in the DRAM of<br />
the Marvell processor.<br />
3.4.2 User Interface<br />
An HDL module is provided in the support package around here:<br />
D:/FPGA_Rererence_designs/code/common/nmb/nmb_target_interface.v<br />
There is hopefully a PDF in that directory that gives a much better description of how to actually use<br />
the interface. But more or less the interface provides a simple Address/DataIn/DataOut type<br />
interface. You should think of the interface as a memory space.<br />
On the C++ side of the NMB there are simple functions like<br />
nmb_read(address, buffer, size)<br />
that can be used to view this memory space. The code for this is found in the support package here<br />
D:/Host_software_applications/Emu/EmuLib/dnapi.h<br />
DNV6F6PCIE User <strong>Manual</strong> Page 25
Hopefully there is also a PDF there that describes how to use it.<br />
3.4.3 Memory Spaces<br />
You should probably read the PDF describing dnapi.h instead of this document.<br />
D:\Host_Software\emu\Documents\Emu_<strong>Manual</strong>.pdf<br />
But here it goes: The memory space is 64-bit. Each address represents a single byte, however the<br />
data is required to be read and written in blocks of 32-bit. Addresses supplied to the interface must<br />
be divisible by 4. Therefore, the bottom 2 bits of the address space are stupid.<br />
Additionally, since there are no chip-selects on the NMB bus, it is necessary to pre-allocate address<br />
ranges for devices on the bus. On this board, there are six devices, and the NMB address ranges that<br />
they are able to respond to on the bus are given here:<br />
Target Starting Address End Address<br />
FPGA A 0x00000000_00000000 0x00FFFFFF_FFFFFFFF<br />
FPGA B 0x01000000_00000000 0x01FFFFFF_FFFFFFFF<br />
FPGA C 0x02000000_00000000 0x02FFFFFF_FFFFFFFF<br />
FPGA D 0x03000000_00000000 0x03FFFFFF_FFFFFFFF<br />
FPGA E 0x04000000_00000000 0x04FFFFFF_FFFFFFFF<br />
FPGA F 0x05000000_00000000 0x05FFFFFF_FFFFFFFF<br />
3.4.4 Error Conditions<br />
Exist.<br />
3.5 FPGA Interconnect<br />
Most of the I/O on the FPGA are used to connect each FPGA to other FPGAs. Most interconnect is<br />
routed point-to-point between FPGAs, in a nearest neighbor topology. The exact topology is shown<br />
in the diagram below.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 26
Note that the interconnect that is drawn in a gold color in the diagram is only available when both<br />
connected FPGAs are either the LX550T device, or the SX475T device. If either FPGA is a "small"<br />
FPGA device, then the signals are not usable. This is because the "small" FPGA devices do not allow<br />
the use of all of the pins of the FPGA as I/O.<br />
3.5.1 I/O protocol<br />
The protocol for the use of the I/O is user defined. The VCCO pin of the FPGA on each bank that is<br />
used for interconnect is +2.5V. This means that LVCMOS25 and LVDS are both reasonable choices<br />
for the IOSTANDARD attribute.<br />
The board features necessary to use terminated standards, such as LVDCI or SSTL are not provided<br />
on the board, and so DCI cannot be used for FPGA-to-FPGA communication. When using LVDS,<br />
you can still use the DIFF_TERM attribute to terminate signals.<br />
Since the "global clocks" on the board are delivered to each FPGA with low skew, any of the<br />
"global clocks" are suitable for use for FPGA interconnect.<br />
3.5.2 High-Speed interconnect<br />
The interconnect between FPGAs are divided into "banks". A single bank on one FPGA always<br />
connected to a single bank on one other FPGA. This pairing up of banks allows the use of some of<br />
the high-speed features of the Virtex 6, such as BUFR clocking, and ISERDES and OSERDES.<br />
Every "bank" of interconnect contains at least one LVDS pair in each direction that goes to a "clock<br />
capable" pin on the FPGA. This pair can be used as a clock to input all of the bits on that bank. The<br />
net name of this pair ends in "_CC" in the schematic.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 27
To achieve the highest switching rates on the interconnect banks, you must use the LVDS<br />
IOSTANARD.<br />
3.5.3 Signal net length report<br />
The shortest FPGA-to-FPGA interconnect signals is 40mm long. The longest is 290mm long. This<br />
corresponds to a skew of about 2ns.<br />
Within any single bus, the greatest skew is 75mm. This corresponds to 0.5 ns of skew.<br />
3.6 SODIMM (DDR3) Connectors<br />
For memory expansion, the DNV6F6PCIe has four socket connectors connected to four of the user<br />
FPGAs. The sockets accept standard off-the-shelf DDR3 laptop memory. The interface and reference<br />
design is compatible with any density or organization of memory.<br />
These sockets can also be used for types of memory other than DDR3 DRAM. Dini Group has a<br />
variety of modules that are compatible with the DNV6F6PCIe, including synchronous SRAM and<br />
others.<br />
The SODIMM interface is also potentially usable as an expansion interface for custom daughter<br />
cards.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 28
3.6.1 Memory Interface Generator<br />
The provided reference design uses a memory controller that is based on the memory controller that<br />
is produced by the "Memory Interface Generator" (MIG), part of the ISE software. However it has<br />
been modified. The modifications allow the use of dual-rank DIMMs, and also set some parameters<br />
automatically, such are RAS and CAS latencies, and total DIMM density.<br />
Some of the signals that are connected between the SODIMM connector and the FPGA are not used<br />
by MIG. These include the SODIMM "NC" (no connect) pins, the upper two address pins, the<br />
EVENTn pin.<br />
Additionally, the "feedback" clock is not used by MIG, or the DDR3 controller provided by Dini<br />
Group.<br />
If you want to use MIG you should use the HDL files produced by MIG, and use the UCF file<br />
provided by Dini Group, removing the unused signals.<br />
3.6.2 IO Standards<br />
The DIMM interface is voltage-selectable. When using DDR3 memory, it is suggested to use the<br />
1.5V IO voltage. When using this voltage, signals to the SODIMM should be of the IO Standard<br />
SSTL_II_18_DCI or SSTL_II_18_DIFF_DCI. The necessary board features to make SSTL work<br />
properly are provided.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 29
The EVENTn pin is also voltage selectable. It will be the same voltage as the rest of the DIMM.<br />
There are some LED signals on the FPGA that are connected to the DIMM bank. The<br />
IOSTANDARD attribute of these signals must be changes to match the voltage of the DIMM.<br />
The IIC signals (SDA and SCL) are fixed at +2.5V and should use a 2.5V standard.<br />
3.6.3 Voltage Selection<br />
Off-the-shelf DDR3 DRAM always uses 1.5V core power and IO signaling levels. If you are using<br />
DRAM, then you would never need to change the voltage output of the DRAM interfaces. However,<br />
when using alternate memory modules from Dini Group, or when designing your own daughter<br />
cards, you may need to supply a different voltage to the SODIMM and to the attached pins of the<br />
FPGA.<br />
+5.0V<br />
This<br />
resistor is<br />
duplicated<br />
on other<br />
pages<br />
R1193<br />
1K<br />
5,6,7,59 SEQ_DISABLE_VCCO#<br />
Silkscreen:<br />
R1192 1K<br />
1.35V<br />
1.5V<br />
1.8V<br />
Install all for 2.5V<br />
R1180<br />
DNI<br />
U91-1<br />
J32<br />
REG_DIMMA_TRACK E1<br />
G2 REG_DIMMA_FB<br />
1 2 REG_DIMMA_FB135 R1371 30K<br />
TRACK FB<br />
3 4 REG_DIMMA_FB15 R1370 13.7K<br />
REG_DIMMA_RUN D1<br />
5 6 REG_DIMMA_FB18 R1369 6.8K<br />
R1191 0R<br />
RUN/SS<br />
R1146 7 8 REG_DIMMA_FB25 R1368 19.1K<br />
REG_DIMMA_COMP G1<br />
DNI<br />
COMP<br />
B3 REG_DIMMA_SW0<br />
R1171<br />
F1 O.D.<br />
SW<br />
B4 REG_DIMMA_SW1<br />
TSM-136-01-T-DV<br />
DNI<br />
PGOOD SW<br />
LTM4604A/LGA66<br />
R1136<br />
DNI<br />
1.2V - 10K 1.35V - 7.25K<br />
2.7V - 2.1K 1.5V - 5.76K<br />
R1161<br />
1.8V - 4.02K<br />
10K_0.1%<br />
59 PWR_FAULTn_DIMMA<br />
2.5V - 2.37K<br />
Each SODIMM interface has an isolated regulator that allows you to independently select the<br />
voltage. There is a header next to each SODIMM where you can install jumpers to affect the output<br />
voltage of this regulator. To change the voltage, remove all jumpers currently installed on the header,<br />
and install a jumper next to the silkscreen text showing that voltage that you want. Be sure to probe<br />
the DIMM_VDD test point once you have completed the change.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 30
3.6.4 Daughter cards<br />
If you want to make your own custom daughter card for use in the SODIMM sockets, you are<br />
encouraged to download the complete schematic and layout files for one of our existing custom<br />
modules. These are available on our website with no restrictions on use.<br />
The signals on the DNV6F6PCIE circuit board are routes as 50-ohm impedance. Every signal<br />
connecting to the DIMM is length-matched, including the feedback clock.<br />
Check this webpage to see existing DIMMs and to access the schematic and layout files for existing<br />
DIMMs.<br />
http://dinigroup.com/index.php?page=DNSODM204<br />
3.6.5 Net Length Report<br />
The length of all signals from the FPGA to the SODIMM are length and delay-matched.<br />
3.7 Daughter Card Connectors<br />
The primary means of interfacing to the FPGA with external IO are through the 400-pin MEG-Array<br />
expansion connectors. There are three of these high-speed, high-density connectors on the board,<br />
DNV6F6PCIE User <strong>Manual</strong> Page 31
connected to FPGAs D,E and F. The FPGA connection to each of the three connectors is the same.<br />
The physical layout requirements of each of the three connectors is the same.<br />
They are located on the back side of the board in order to leave plenty of flexibility for the<br />
mechanical layout of the board.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 32
3.7.1 Electrical Spec<br />
60 DCD_CLK_DN_IN_P<br />
60 DCD_CLK_DN_IN_N<br />
42 DCD_CLK_UP_OUT_P<br />
42 DCD_CLK_UP_OUT_N<br />
E1<br />
F1<br />
E3<br />
F3<br />
P12-1<br />
CLK_DN_2.5_P<br />
CLK_DN_2.5_N<br />
CLK_UP_2.5_P<br />
CLK_UP_2.5_N<br />
PLUG<br />
+12V<br />
+12V<br />
RSVD_PWR<br />
RSVD_PWR<br />
+3.3V<br />
+3.3V<br />
+2.5V_LDO<br />
A1<br />
K1<br />
C1<br />
H1<br />
B2<br />
D2<br />
G2<br />
DCDVFUSED<br />
DCDSVD R5 DNI<br />
RESC1005N<br />
DCD3VFUSED<br />
DAUGHTERCARD_RESET_POWER 36,39<br />
F3<br />
5A<br />
FUSE_0429<br />
F6<br />
5A<br />
FUSE_0429<br />
+12V_R<br />
+3.3V<br />
DCDCO_CAP<br />
C59<br />
1uF<br />
K20<br />
VCCO_CAP<br />
RSTn_3.3_TOLERANT<br />
PLUG<br />
J2<br />
DAUGHTERCARD_RESETn 36,39<br />
MEG_400_Plug_Stratix3_30<br />
3.7.2 IO Electrical<br />
The part number of the connector part installed on the DNV6F6 is "84520102LF" from FCI. It is<br />
intended that the part number that will be used to connector to this board is the FCI 74390-101 part.<br />
The part 84520102LF is called the "plug" and the 74390-101 is called the "receptacle"<br />
The DNV6F6 is the "host" side of the interface. The mating card is called the "daughtercard"<br />
All signals from the FPGA to the connector are length matched to each other with a minimum<br />
tolerance of 50ps on all Dini Group host boards. All Dini host boards route the FPGA I/O signals as<br />
50-ohm transmission lines.<br />
It is recommended that daughter cards provide a bypass capacitor between the pins B0_VCC0,<br />
B1_VCC0, B2_VCC0, B3_VCC0, B4_VCC0 and ground close to the connector pin on the daughter<br />
card.<br />
Note that Virtex 6 is incompatible with +3.3V I/O signaling completely. +2.5V is the maximum<br />
supported I/O voltage. If you require +3.3V I/O, you must use voltage translation devices.<br />
3.7.2.1 _V pins<br />
Any pin name that ends in the string "_V" is a "VREF" pin.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 33
For some settings of the FPGA I/O attribute IOSTANDARD, you are required to supply a 1/2 VCCO<br />
voltage onto the "VREF" pins of the FPGA. If a daughtercard requires one of these IOSTANDARD<br />
settings, it must generate the VREF voltage, and drive it back on the _V pins of the daughtercard.<br />
There are no-stuff capacitor locations on the host board attached to all "VREF" pins that can be<br />
populated with capacitors, if this is required by your electrical analysis. Note that installing these<br />
capacitors will negatively impact the switching speed of these signals when used as regular I/O.<br />
3.7.2.2 _CC pins<br />
Any pin whose name ends in "_CC" is a "CC" pin.<br />
CC pins and _MRCC pins are connected to "CC" pins or "MRCC" pins on the host board's FPGA.<br />
You should see the Virtex 6 SelectIO user guide for the implications of this. But in general, these<br />
pins are suitable for I/O clocks driven from the daughtercard to the host FPGA.<br />
The "CC" pins are able clock I/O in other FPGA banks in some cases. For this capability, there are<br />
requirements that the banks have certain physical relationships to each other on the silicon die of the<br />
FPGA device. During the assignment of FPGA banks to the daughtercard, no provisions were made<br />
to restrict the bank selection to make cross-bank clocking consistent from one daughtercard header to<br />
the next. It is recommended that is CC pins are required, that a separate copy of the clock is driven to<br />
each FPGA bank that requires it, and do not rely on "multi-clock" regions.<br />
3.7.2.3 VRP and VRN Pins<br />
On all FPGA banks that are connected to the daughtercard, VRP and VRN pins are correctly<br />
connected to allow DCI to be used with the daughtercard. Note that there is still the Virtex 6 I/O rule<br />
that only a single type of DCI may be used per bank of the FPGA. This requirement may limit the<br />
use of DCI on the daughter card. The requirement is too complicated to describe here, so you may<br />
need to run a test place-and-route of your design to determine whether your desired pin out is<br />
acceptable.<br />
3.7.3 Reset Signal<br />
The signal RSTn_3.3V_TOLERANT is valid when +2.5V_LDO is above 0.7V. At all times, when<br />
the signal RSTn_3.3V_TOLERANT is valid, and has a voltage measuring below 0.7V, then all<br />
boards using this interface (host and daughtercard) must tri-state all I/O signals connected to the<br />
interface. Daughter cards that fail to tri-state signals until the de-assertion of<br />
RSTn_3.3V_TOLERANT may risk damaging the DNV6F6 board. The DNV6F6 will weakly pull up<br />
this signal to +2.5V. A daughter card is free to also pull up this signal weakly to any voltage between<br />
0.7V and 3.3V<br />
DNV6F6PCIE User <strong>Manual</strong> Page 34
+2.5V<br />
R4<br />
2.0K_0.1%<br />
R3<br />
1K<br />
+2.5V<br />
R6<br />
DNI<br />
R7<br />
1K<br />
R2<br />
MON_DC_ADJ1<br />
MON_DC_ADJ2<br />
DNI MON_DC_SEL<br />
U6<br />
1<br />
2<br />
ADJ1<br />
ADJ2<br />
3<br />
8<br />
REF<br />
SEL<br />
LTC2909<br />
VCC<br />
RST<br />
TMR<br />
GND<br />
6<br />
5<br />
7<br />
4<br />
MON_DC_TMR<br />
C2<br />
C4<br />
DNI<br />
TP2<br />
DNI<br />
TESTPOINT<br />
R347<br />
10K_0.1%<br />
DAUGHTERCARD_RESETn 36,39<br />
0.1uF<br />
+12V_R<br />
DC Linear Power Supply 2.5V @ 1mA<br />
R1<br />
1K<br />
RESC1005N<br />
Current<br />
Limits to<br />
20mA<br />
REG_DC_RESET_IN<br />
C614<br />
4.7uF_12V<br />
C6<br />
0.1uF<br />
U22<br />
8<br />
IN<br />
5<br />
LT1963AES8/SO8<br />
OUT<br />
SHDN SENSE/ADJ<br />
3<br />
6<br />
GND<br />
7<br />
GND<br />
GND<br />
NC<br />
1<br />
2<br />
4<br />
DAUGHTERCARD_RESET_POWERq<br />
REG_DC_RESET_ADJ<br />
R372<br />
6.04K<br />
R353<br />
6.8K<br />
C3<br />
4.7uF<br />
C5<br />
0.1uF<br />
D1<br />
1 2<br />
DIO_DFLS130L-7<br />
DAUGHTERCARD_RESET_POWER 36,39<br />
Requires a GND area fill for thermal<br />
performance, reference the<br />
datasheet.<br />
3.7.4 Power<br />
The DNV6F6 supplies power to the daughtercard at two voltages, 12V and 3.3V. Each pin of the<br />
Meg connector can supply no more than 1A of current, so the effective power limit of the<br />
daughtercard is 2A x 12V + 3A x 3.3V = 33.9W.<br />
It is strongly recommended that daughter cards provide a means of isolating (series resistor) their<br />
power net with the host board, and provide a means of bypassing the power input with an external<br />
power connector.<br />
On other Dini Group boards, the pins C1 and H1 may be power pins. On the DNV6F6, these pins are<br />
no-connects.<br />
The daughtercard should never be capable of supplying current back onto the host board on the<br />
+12V or +3.3V nets. This could potentially damage the host board.<br />
3.7.5 VCCO Power<br />
The FPGA I/O power pins are connected directly to the meg-array daughtercard interface. The<br />
intention of this design is for the daughtercard to drive the necessary I/O voltage back onto the host<br />
board. There are linear voltage regulators on the DNV6F6 that bias these power rails to 1.2V,<br />
however it is not recommended that you use these to power the daughtercard I/O. These regulators<br />
can supply up to 1A of current.<br />
If you build a daughter card that drives current back to the host board, it must be able to supply the<br />
current not only for the daughter card I/O, but also for the I/O current requirements of the host board.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 35
If you forget to include voltage regulators for VCCO on the daughtercard, you can rework the<br />
DNV6F6 bias regulators to output your required voltage, so only as the combined current<br />
requirements of the FPGA banks and the daughtercard do not exceed 1A per bank. Note that the 5<br />
voltage regulators on the host board are not set up to evenly share current loads, and that each must<br />
be individually load limited to 1A.<br />
DCD_B0_VCCO<br />
DCD_B1_VCCO<br />
DCD_B2_VCCO<br />
DCD_B3_VCCO<br />
DCD_B4_VCCO<br />
1<br />
2<br />
3<br />
4<br />
5<br />
6<br />
P3<br />
1<br />
2<br />
3<br />
4<br />
5<br />
6<br />
DNI<br />
HDR6<br />
There is a row of test points next to each daughtercard header allowing the easy probing of the 5<br />
voltage rails on the daughtercard interface.<br />
R367<br />
DNI<br />
+2.5V<br />
C18<br />
4.7uF<br />
U17<br />
8<br />
IN<br />
5<br />
OUT<br />
SHDN SENSE/ADJ<br />
3<br />
6<br />
GND<br />
7<br />
GND<br />
GND<br />
NC<br />
LT1963AES8/SO8<br />
SOIC127P600X175-8N<br />
1<br />
2<br />
4<br />
REG_DCD_ADJ_B0<br />
R397<br />
0R<br />
R398<br />
4.7K<br />
Arrogantly<br />
assumes more<br />
capacitance is<br />
on bank<br />
DCD_B0_VCCO 31<br />
36,39 DAUGHTERCARD_RESETn<br />
This is the bias regulator. There is one for each of the 5 banks on each daughtercard interface.<br />
3.7.6 Clock Pins<br />
The clock pins E1, F1, E3, F3 are always connected to 2.5V I/O on the FPGA. They are connected to<br />
clock input pins near the center of the FPGA, making them suitable for sending a clock from the<br />
daughter card to the host FPGA. They are not externally terminated on the board, but you can use<br />
DIFF_TERM in the FPGA if you are using differential signaling.<br />
In addition to these two clocks, there is a variable-voltage clock input to the host FPGA. This signal,<br />
B0_P1?_GCC_TERM is located on bank 0. The input levels will be determined by the bank 0<br />
voltage, and the IOSTANDARD settings of the I/O in the FPGA. There exists external endtermination<br />
resistors on the host board, terminating to a voltage of VCCO/2. This termination<br />
scheme will result in a high current on the signal.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 36
H42<br />
J39<br />
K36<br />
L33<br />
M30<br />
T28<br />
C846<br />
1uF<br />
U3-12<br />
VCCO_25<br />
VCCO_25<br />
VCCO_25<br />
VCCO_25<br />
VCCO_25<br />
VCCO_25<br />
FPGA D<br />
IO_L0P_25<br />
IO_L0N_25<br />
IO_L1P_25<br />
IO_L1N_25<br />
IO_L2P_25<br />
IO_L2N_25<br />
IO_L3P_25<br />
IO_L3N_25<br />
IO_L4P_25<br />
IO_L4N_VREF_25<br />
IO_L5P_25<br />
IO_L5N_25<br />
IO_L6P_25<br />
IO_L6N_25<br />
IO_L7P_25<br />
IO_L7N_25<br />
IO_L8P_SRCC_25<br />
IO_L8N_SRCC_25<br />
IO_L9P_MRCC_25<br />
IO_L9N_MRCC_25<br />
IO_L10P_MRCC_25<br />
IO_L10N_MRCC_25<br />
IO_L11P_SRCC_25<br />
IO_L11N_SRCC_25<br />
IO_L12P_25<br />
IO_L12N_25<br />
IO_L13P_25<br />
IO_L13N_25<br />
IO_L14P_25<br />
IO_L14N_VREF_25<br />
IO_L15P_25<br />
IO_L15N_25<br />
IO_L16P_VRN_25<br />
IO_L16N_VRP_25<br />
IO_L17P_25<br />
IO_L17N_25<br />
IO_L18P_GC_25<br />
IO_L18N_GC_25<br />
IO_L19P_GC_25<br />
IO_L19N_GC_25<br />
XC6VLX240T/550T_FF1759<br />
C730<br />
4.7uF<br />
K33<br />
K32<br />
N28<br />
P28<br />
K35<br />
K34<br />
L31<br />
L32<br />
J37<br />
J36<br />
N29<br />
N30<br />
H39<br />
H38<br />
L35<br />
L36<br />
K38<br />
J38<br />
P27<br />
R27<br />
R28<br />
R29<br />
K37<br />
L37<br />
H40<br />
H41<br />
L34<br />
M34<br />
J40<br />
J41<br />
M33<br />
M32<br />
K39<br />
K40<br />
M31<br />
N31<br />
J42<br />
K42<br />
P30<br />
P31<br />
DCD_B0_P17<br />
DCD_B0_N17<br />
DCD_B0_P9<br />
DCD_B0_N9<br />
DCD_B0_P7<br />
DCD_B0_N7<br />
DCD_B0_P3<br />
DCD_B0_N3<br />
DCD_B0_P2<br />
DCD_B0_N2_VREF<br />
DCD_B0_P11<br />
DCD_B0_N11<br />
DCD_B0_P16<br />
DCD_B0_N16<br />
DCD_B0_P14<br />
DCD_B0_N14<br />
DCD_B0_P15<br />
DCD_B0_N15<br />
DCD_B0_P4_CC<br />
DCD_B0_N4_CC<br />
DCD_B0_P13_CC<br />
DCD_B0_N13_CC<br />
DCD_B0_P6<br />
DCD_B0_N6<br />
DCD_B0_P5<br />
DCD_B0_N5<br />
DCD_B0_P1<br />
DCD_B0_N1_VREF C765<br />
DCD_B0_P10<br />
DCD_B0_N10<br />
DCD_B0_VRN<br />
R477<br />
DCD_B0_VRP<br />
R476<br />
DCD_B0_P12<br />
DCD_B0_N12<br />
DCD_B0_P8_GCC_BUS<br />
DCD_B0_N8_GCC_BUS<br />
DCD_B0_P0_GCC_DN<br />
DCD_B0_N0_GCC_DN<br />
C731<br />
4.7uF<br />
C832<br />
C684<br />
4.7uF<br />
DNI<br />
DCD_FEEDBACKp 42<br />
DCD_FEEDBACKn 42<br />
DNI<br />
50R<br />
50R<br />
C640<br />
4.7uF<br />
DCD_B0_VCCO 33<br />
C639<br />
4.7uF<br />
Meg-Array Connector<br />
C683<br />
4.7uF<br />
P12-2<br />
PLUG<br />
DCD_B0_P0_GCC_DN A3<br />
DCD_B0_N0_GCC_DN B4<br />
B0_P0_GCC_DN<br />
DCD_B0_P1<br />
A5<br />
B0_N0_GCC_DN<br />
DCD_B0_N1_VREF B6<br />
B0_P1<br />
DCD_B0_P2<br />
A7<br />
B0_N1_VREF<br />
DCD_B0_N2_VREF B8<br />
B0_P2<br />
DCD_B0_P3<br />
A9<br />
B0_N2_VREF<br />
DCD_B0_N3<br />
B10<br />
B0_P3<br />
DCD_B0_P4_CC A11<br />
B0_N3<br />
DCD_B0_N4_CC B12<br />
B0_P4_CC<br />
DCD_B0_P5<br />
A13<br />
B0_N4_CC<br />
DCD_B0_N5<br />
B14<br />
B0_P5<br />
DCD_B0_P6<br />
A15<br />
B0_N5<br />
DCD_B0_N6<br />
B16<br />
B0_P6<br />
DCD_B0_P7<br />
A17<br />
B0_N6<br />
DCD_B0_N7<br />
B18<br />
B0_P7<br />
DCD_B0_P8_GCC_BUS E5<br />
B0_N7<br />
DCD_B0_N8_GCC_BUS F5<br />
B0_P8_GCC_BUS<br />
DCD_B0_P9<br />
H3<br />
B0_N8_GCC_BUS<br />
DCD_B0_N9<br />
G4<br />
B0_P9<br />
DCD_B0_P10<br />
H5<br />
B0_N9<br />
DCD_B0_N10<br />
G6<br />
B0_P10<br />
DCD_B0_P11<br />
H7<br />
B0_N10<br />
DCD_B0_N11<br />
G8<br />
B0_P11<br />
DCD_B0_P12<br />
H9<br />
B0_N11<br />
DCD_B0_N12<br />
G10<br />
B0_P12<br />
DCD_B0_P13_CC H11<br />
B0_N12<br />
DCD_B0_N13_CC G12<br />
B0_P13_CC<br />
DCD_B0_P14<br />
H13<br />
B0_N13_CC<br />
DCD_B0_N14<br />
G14<br />
B0_P14<br />
DCD_B0_P15<br />
H15<br />
B0_N14<br />
DCD_B0_N15<br />
G16<br />
B0_P15<br />
DCD_B0_P16<br />
H17<br />
B0_N15<br />
DCD_B0_N16<br />
G18<br />
B0_P16<br />
DCD_B0_P17<br />
H19<br />
B0_N16<br />
DCD_B0_N17<br />
G20<br />
B0_P17<br />
B0_N17<br />
C46<br />
1uF<br />
A6<br />
B0_VCCO<br />
PLUG<br />
MEG_400_Plug_Stratix3_30<br />
DCD_B0_P8_GCC_BUS<br />
DCD_B0_N8_GCC_BUS<br />
DCD_B0_P0_GCC_DN<br />
DCD_B0_N0_GCC_DN<br />
R478<br />
R479<br />
R481<br />
R480<br />
R455<br />
R486<br />
R484<br />
R485<br />
100R<br />
100R<br />
100R<br />
100R<br />
100R<br />
100R<br />
100R<br />
100R<br />
3.7.7 Net Lengths<br />
All daughtercard signals are matched between 60mm and 80mm. This corresponds to a maximum<br />
signal skew of 180ps, in addition to the skew caused by the FPGA device.<br />
3.7.8 Physical Spec<br />
Daughter cards are mounted on the "back" or "solder" side of the PCB, opposite the FPGAs. This<br />
allows for maximum vertical mechanical flexibility. The vertical clearance of components on the<br />
reverse side of the DNV6F6 is 3mm. The board-to-board clearance between the daughtercard and the<br />
host board is 14mm. Therefore, the daughtercard may have 10mm components on its side facing the<br />
base board, leaving 1mm of clearance.<br />
The "front" or "component" side of the daughtercard is the face that points away from the host board.<br />
Therefore, the "back" or "solder" sides of the host and daughter cards face toward each other.<br />
The position and size of four mounting holes through the DNV6F6 PCB with respect to each<br />
daughtercard interface is fixed. Every Dini Group product with 400-pin MEG interfaces has at least<br />
four mounting holes with this offset.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 37
0.500"<br />
0.750"<br />
0.500"<br />
0.750"<br />
5.000"<br />
3.250"<br />
4.250"<br />
5.000"<br />
3.250"<br />
4.250"<br />
2.75"<br />
2.75"<br />
Type 2 Short<br />
Type 0/1/4 Short<br />
View: Top Side<br />
400-Pin Receptacle on Back<br />
P/N: 74390-101<br />
View: Top Side<br />
300-Pin Receptacle on Back<br />
P/N: 84553-101<br />
A1<br />
A1<br />
1.950"<br />
0.500"<br />
In the above diagram, a daughtercard is designed that has holes matching the position of the holes on<br />
the base board, so that a 14mm, M3 size standoff can be used to stabilize it onto the base board. The<br />
dimensions of the daughtercard above are guaranteed to physically fit onto any Dini Group host<br />
board while that board is installed inside a PC case. Boards larger than the above given dimensions<br />
may not physically fit onto your Dini Group host board. If you cannot fit your daughter card design<br />
in this form factor, then you must examine the physical dimensions of the target host board to<br />
determine the maximum dimensions available.<br />
1.950"<br />
0.500"<br />
For a larger vertical clearance between the host and daughter cards, a vertical extender board is<br />
available. The diagram below shows the vertical clearance when used with an extender.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 38
3.7.9 Insertion and removal<br />
Due to the small dimensions of the very high speed Meg Array connector system, the pins on the<br />
plug and receptacle of the Meg Array connectors are very delicate.<br />
When plugging in a daughter card, make sure to align the daughter card first before pressing on the<br />
connector. Be absolutely certain that both the small and the large keys at the narrow ends of the Meg<br />
Array line up BEFORE applying pressure to mate the connectors!<br />
Error! Objects cannot be created from editing field codes.<br />
Figure 1 - Daughter card installation step 1<br />
Place it down flat, then press down gently.<br />
Error! Objects cannot be created from editing field codes.<br />
Figure 2 - Install Daughter card step 2<br />
Mating can be started from either end. Locate and match the connector’s A1 position marking<br />
[triangle] for both the Plug and Receptacle. (Markings are located on the long side of the housing.)<br />
Rough alignment is required prior to connector mating as misalignment of >0.8mm could damage<br />
connector contacts. Rough alignment of the connector is achieved through matching the Small<br />
alignment slot of the plug housing with the Small alignment key of the receptacle housing and the<br />
large alignment slot with the large alignment key. Both connector housings have generous lead-in<br />
around the perimeter and will allow the user to blind mate assemble the connectors. Align the two<br />
connectors by feel and when the receptacle keys start into the plug slots, push down on one end and<br />
then move force forward until the receptacle cover flange bottoms on the front face of the plug.<br />
Like mating, a connector pair can be unmated by pulling them straight apart. However, it requires<br />
less effort to un-mate if the force is originated from one of the slot/key ends of the assembly.<br />
(Reverse procedure from mating) Mating or un-mating of the connector by rolling in a direction<br />
perpendicular to alignment slots/keys may cause damage to the terminal contacts and is not<br />
recommended.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 39
3.7.10 Clocking Methods<br />
A wide variety of clocking topologies were considered when designing the MEG interface. There<br />
should be at least one clocking method possible that will meet your needs. A list of reasonable<br />
clocking methods are listed and diagramed below.<br />
3.7.10.1 <strong>Manual</strong> phase alignment<br />
You can use a PLL inside the FPGA to manually align the phase of a clock that you send from the<br />
FPGA to a MEG card.<br />
3.7.10.2 Synchronous with daughter card<br />
You may need to have a clock with edges that arrive simultaneously to the FPGA and to the MEG<br />
daughter card. Each FPGA with a MEG interface has a feedback path that can be used to align a<br />
clock going to the MEG with one going to the FPGA.<br />
This allows a device on operate fully synchronously with the FPGA.<br />
3.7.10.3 Forwarding a global synchronous clock<br />
If you need a device on the MEG card to operate fully synchronously to all FPGAs on the DNV6F6,<br />
then you can use a PLL in the FPGA as shown in the diagram.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 40
This circuit will delay a copy of a "global clock" to the MEG card such that the phase will arrive at<br />
the MEG card at the same time that it arrives at all of the FPGAs.<br />
3.7.10.4 Source-Synchronous inputs<br />
Since the signals to the daughter card are length-matched, you can rely on a tight timing relationship<br />
between the clock and data that you send from the daughtercard to the FPGA. Since the FPGA has a<br />
zero-hold-time input, sending a clock from the MEG card whose rising edges are aligned with the<br />
data transitions on your data lines will result in reliable communication with the FPGA.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 41
3.7.10.5 Source-synchronous outputs<br />
You can repeat this setup in the opposite direction, as long as the hold time on the device on the<br />
MEG card is zero or negative.<br />
3.7.10.6 Skewed Clocks<br />
The I/O on the FPGA can be used on either rising or falling edges of a clock, so it is easy to invert<br />
the clock on the FPGA device, so that it operates 180 degrees out of phase with the daughter card. Of<br />
you can invert the clock as you output it to the daughter card. The maximum frequency of the<br />
interface when using this method is effectively reduced by half.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 42
3.7.11 Incorrect Clocking Methods<br />
The following are methods that don't work.<br />
3.7.11.1 Hold time violation<br />
The following diagram shows a method that potentially violates hold time on the device on the MEG<br />
card.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 43
3.7.11.2 PLL cascade<br />
When using PLLs in the FPGA, make sure that there is not another PLL anywhere in the feedback<br />
loop of the PLL or it will result in an unreliable phase output.<br />
3.7.12 Compatibility with older Dini Boards<br />
The "MEG" interface has been included on Dini Group board since 2005 with the Virtex 4 series of<br />
boards, however the features provided by the interface have changed since then. Any daughter card<br />
that meets the following criteria will be compatible with the DNV6F6PCIE:<br />
- Designed for use with a Dini Group Xilinx FPGA board (not Altera)<br />
- Uses only a single voltage for the whole daughter card<br />
- Does not use I/O signal levels above +2.5V<br />
- Uses the "reset" signals as an input only<br />
- Does not require the use of the +5.0V power pin<br />
- Uses no differential signals<br />
All Virtex 6 boards provide the same features on all MEG interfaces, so any daughtercard designed<br />
for use on a Virtex 6 board will work on any MEG connector on any Virtex 6 board that Dini Group<br />
sells.<br />
3.7.13 How to make a daughter card<br />
It is recommended that you start with one of the Dini Group daughter cards as a design template. The<br />
ORCAD schematic, and layout files are all provided on the Dini Group website for several daughter<br />
cards.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 44
3.7.13.1 Ensuring backward compatibility<br />
In order to build a daughter card that is compatible with all Dini Group host boards, you should use<br />
the following guidelines:<br />
1) Use the standard 2.75" x 4.25" form factor.<br />
2) Use the same I/O voltage for all I/O pins on the daughter card<br />
3) Supply I/O voltage from the daughter card on all 6 VCCO pins<br />
4) Do not use CC pins<br />
5) Only use differential signaling standards for clocks that drive to the base board<br />
6) Do not use the +5.0V power pin, if one is present.<br />
7) If VREF is needed, it must be driven onto all VREF pins of the entire MEG interface<br />
3.8 FPGA CONFIGURATION<br />
3.8.1 Select map<br />
3.8.2 JTAG<br />
3.9 Marvell CPU<br />
The clock networks, off-board I/O, NMB and configuration is controlled by a Marvell ARM CPU.<br />
This section describes the hardware attached to the Marvell processor. For information about<br />
programming the Marvell and the software running on the Marvell, see the software section of the<br />
manual.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 45
The primary purpose of the Marvell processor is to pump data from a host interface (PCI Express,<br />
Gigabit Ethernet, SATA, Flash, or USB) to and from the user's design in the FPGA.<br />
Additionally, these host interfaces can also be used to control the settings of the board's resources,<br />
like clock synthesizers, and configure FPGAs.<br />
The software that comes pre-loaded on the Marvel processor is running the Linux operating system.<br />
For this section, it is assumed that you have a working understanding of Linux.<br />
3.9.1 RS232<br />
The console output of Linux is directed to an RS232 port. In order to connect a terminal to this port,<br />
you will need to set the terminal settings to 19200 baud, 8bit, no parity, no flow control, 1 stop bit.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 46
The Linux console is connected to a shell, so you can interact with the system, however the main<br />
purpose of the console is to receive kernel debugging messages, so it is recommended that you<br />
instead use a telnet terminal, as this shell will be much less chatty.<br />
In order to interact with the boot loader, or see output from the boot process, you must use the RS232<br />
console, you cannot use telnet for these purposes.<br />
3.9.2 PCI Express<br />
The Marvell device natively contains a PCI Express device. The PCI express pins of the Marvell<br />
processor are connected directly to the edge connector of the DNV6F6 board.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 47
The device appears as three 1MB memory regions to the host machine. You are expected to use the<br />
Emu software to interface with PCI Express. There is no description available of the function of<br />
DNV6F6 as a PCI Express device.<br />
3.9.3 SATA<br />
The Marvell device contains natively two SATA host ports. In the pre-installed software these ports<br />
are managed by Linux, and you are not intended to directly interact with SATA hardware. Instead,<br />
when a device is installed, it is automatically mounted as a storage device with a filesystem.<br />
From that point, you should use Linux scripts and programs to interact with the mounted filesystem.<br />
3.9.4 DEV Bus: NAND<br />
The root file system of the Linux installation is contained on the NAND Flash device connected to<br />
the Marvell's "device bus". The NAND is 256MB in size, with the first 100MB already assigned to<br />
the Linux installation.<br />
3.9.5 DEV Bus: FPGA<br />
The Marvell "device bus" is also attached to the "config FPGA". This connection has no purpose.<br />
3.9.6 USB<br />
The Marvell device natively provides a USB host and peripheral device. The two "host" devices are<br />
managed by Linux. You are not expected to interact directly with the host ports. Instead, when a<br />
device is detected, Linux automatically mounts the devices as storage devices with filesystems. You<br />
should write linux scripts and programs to interact with the mounted filesystems.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 48
The single "device" interface is intended to be used as a connection to a host PC that will be used<br />
with the provided "EMU" software. You are expected to use the EMU controller library to interact<br />
with this device.<br />
3.9.7 IIC Bus: Temperature Sensors<br />
The temperature sensors for the user FPGAs, for the Marvell processor and for the config FPGA are<br />
connected to the Marvell's two-wire serial interface. The installed software will poll these IIC<br />
interfaces and measure the temperature of the FPGAs. If the FPGAs are not within a specified<br />
temperature range, the software will automatically clear the overheated FPGA both to alert the user<br />
to the problem, and to prevent damage to the FPGAs.<br />
3.9.8 ICE<br />
There is an interface for running a hardware debugger on the Marvell processor. It is not expected<br />
that anyone will use this interface, and so details are omitted here.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 49
3.9.9 SDRAM<br />
The Marvell environment has 1GB of DRAM managed by Linux.<br />
3.9.10 SPI Bus: Flash<br />
The linux kernel and uboot bootloader are contained on a SPI flash device. The marvell boots by<br />
running instructions from address 0x0 of the SPI flash device. There is a 4-pin SPI programming<br />
header attached to this SPI flash. In order to program the flash device, the Marvell must be<br />
continually held in reset to prevent interference with the SPI programming process.<br />
3.9.11 Ethernet<br />
The Marvell Processor natively contains three gigabit Ethernet port. Only one of these three ports are<br />
enabled. The ethernet port is managed by linux. You are expected to use standard Linux<br />
programming APIs to access these ports from the Marvell side of the link, and to write you own<br />
software for the host side of the link. Several ports are used by the provided Emu software, and you<br />
can use the communication framework provided to interact with your board over ethernet.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 50
3.9.12 Multi-CPU<br />
The Marvell Processor has two CPUs. The first, CPU0 is used by the Linux operating system. Since<br />
Linux does not support symmetric multi-processing, the second CPU, CPU1 must be operated in unhosted<br />
mode, in its own area of DRAM, without accessing devices. I/O must be accomplished<br />
through CPU0 under Linux.<br />
The use of PCI Express and the DMA engine in the configuration FPGA is possible. Interrupts may<br />
also be used by CPU1.<br />
3.10 Config FPGA<br />
The "seventh" FPGA on the board, the config FPGA (FPGA Q) is not really intended for the user. It<br />
is a "cleanup" FPGA that controls all the clock circuits on the board, configures the other FPGAs,<br />
and multiplexed the NMB bus from the Marvell CPU to the user FPGAs. You don't need to know<br />
anything about it or how it works. You are encouraged to skip this section.<br />
3.10.1 PCI Express<br />
The config FPGA is connected to the Marvell processor through a PCI Express interface. The config<br />
FPGA is a PCI Express endpoint, and the Marvell acts as a root port.<br />
3.10.2 Configuring the Config FPGA<br />
The configuration signals of the config FPGA are connected to the Marvell CPU. The Marvell CPU<br />
gets the .bit file for configuring the "config FPGA" off of the NAND flash. The "config FPGA" also<br />
DNV6F6PCIE User <strong>Manual</strong> Page 51
has a SPI flash attached to the configuration signals that can be used to store configuration data,<br />
however it is unused.<br />
3.10.3 RS232<br />
The "config" FPGA passes through the user's RS232 signals to the RS232 buffer.<br />
3.10.4 Count Clocks<br />
All of the boards "global clocks" are connected to the config FPGA. The config FPGA measures the<br />
frequency so that software can report it.<br />
3.10.5 JTAG<br />
The config FPGA has a dedicated JTAG chain and connector that can be used with the IMPACT<br />
program from Xilinx. It has no purpose.<br />
3.10.6 Device Bus<br />
The "config FPGA" has a "device bus" interface connected between it and the Marvell CPU. In this<br />
way the CPU can access the config FPGA as a memory-mapped device. This interface serves no<br />
purpose on this board.<br />
3.10.7 Blink LEDs<br />
The config FPGA is connected to 6 green LEDs. It blinks these LEDs incessantly. The LEDs have<br />
no purpose or meaning.<br />
3.10.8 Controlling Clocks<br />
The control signals for the "global clock" synthesizer chips, and the multiplexer chips are conncted<br />
to the config FPGA. Software is able to set these control siganls.<br />
3.10.9 Clock MUX<br />
The config FPGA serves as one stage of multiplexers for the clocking network. On the schematic<br />
you may see that clocks G0, G1 and G2 come from the config FPGA. The config FPGA drives out<br />
clock signals coming from the user FPGAs on the "TO_SPARTAN" wires, depending on software<br />
settings.<br />
3.10.10 Configuring the FPGAs<br />
The configuration FPGA is connected to the "selectmap" configuration bus of the six user FPGAs. It<br />
uses this bus to configure and read back the configuration data of the user FPGAs.<br />
3.10.11 Marvel to NMB Bridge<br />
The NMB interface connects each user FPGA to the config FPGA. The data that goes to and from<br />
the FPGA winds up in the Marvell CPU's DRAM. Since the config FPGA has a PCI Express link to<br />
the Marvell Processor, it is able to directly manipulate memory in the Marvell's DRAM. The config<br />
FPGA has a DMA controller inside of it that pushes data directly from the FPGAs to the Marvell<br />
DRAM.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 52
There is a bunch of detail about how to use this DMA controller that you don't need to know<br />
documented in the "PCIE DMA user manual" PDF document. You should not read this document.<br />
3.11 RS232<br />
There is a pair of signals (RX and TX) for RS232 "Serial" communication to the FPGAs.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 53
CPU<br />
J31<br />
1 2<br />
3 4<br />
5 6<br />
7 8<br />
9 10<br />
TSM-136-01-T-DV<br />
63 RS232_MCU_TX<br />
63 RS232_MCU_RX<br />
+1.8V_CPU<br />
R247 4.7K<br />
UART - CPU and CF FPGA (RS232)<br />
RS232_MCU_TX<br />
RS232_CFPGA_TX<br />
RS232_MCU_RX<br />
RS232_CFPGA_RX<br />
RS232_ON<br />
14<br />
13<br />
16<br />
15<br />
11<br />
8<br />
U101<br />
T1IN<br />
T2IN<br />
R1OUT<br />
R2OUT<br />
ON/OFF<br />
GND<br />
T1OUT<br />
T2OUT<br />
R1IN<br />
R2IN<br />
VCC<br />
VL<br />
SW<br />
CAP<br />
VDD<br />
VEE<br />
3<br />
4<br />
1<br />
2<br />
5<br />
12<br />
6<br />
9<br />
LTC2804-1/SSOP16<br />
SOP64P601X175-16N_REC<br />
7<br />
10<br />
RS232_TXD1<br />
RS232_TXD2<br />
RS232_RXD1<br />
RS232_RXD2<br />
RS232_SW<br />
RS232_CAP C2290<br />
C2298<br />
RS232_VDD<br />
RS232_VEE<br />
0.1uF<br />
0.1uF<br />
+5.0V<br />
L1<br />
10uH<br />
+12V_R<br />
R1391<br />
DNI<br />
+12V_LCD<br />
R1390<br />
DNI<br />
+5V_LCD<br />
+1.8V_CPU<br />
C2291<br />
1uF<br />
Configuration FPGA<br />
J37<br />
1 2<br />
3 4<br />
5 6<br />
7 8<br />
9 10<br />
TSM-136-01-T-DV<br />
C2304<br />
1uF<br />
C2299<br />
1uF<br />
RS232 is only available to FPGAs A, B and E. If you need access to it from other FPGAs, you will<br />
have to forward signals between FPGAs to accomplish this.<br />
3.12 GPIO ACCESS HEADER<br />
FPGA A has a few connections to a simple, tenth-inch header.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 54
This header is perfect for probing. Note that on revision 01 of the circuit board, it is located too close<br />
to the stiffener bar for a standard IDC cable with a key to plug into it. You can either flip the cable<br />
around backward, or cut the key off, or use a welding torch to burn away one millimeter of<br />
aluminum.<br />
ACCESS_PINS_D1<br />
ACCESS_PINS_D3<br />
ACCESS_PINS_D5<br />
ACCESS_PINS_D7<br />
ACCESS_PINS_D9<br />
ACCESS_PINS_D11<br />
ACCESS_PINS_D13<br />
ACCESS_PINS_D15<br />
ACCESS_PINS_D17<br />
ACCESS_PINS_D21<br />
ACCESS_PINS_D23<br />
ACCESS_PINS_D25<br />
ACCESS_PINS_D27<br />
ACCESS_PINS_D29<br />
ACCESS_PINS_D31<br />
ACCESS_PINS_D30<br />
ACCESS_PINS_D24<br />
ACCESS_PINS_D20<br />
ACCESS_PINS_D2<br />
1<br />
3<br />
5<br />
7<br />
9<br />
11<br />
13<br />
15<br />
17<br />
19<br />
21<br />
23<br />
25<br />
27<br />
29<br />
31<br />
33<br />
35<br />
37<br />
39<br />
J39<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
GND<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
Single GND<br />
IO<br />
IO<br />
PAIRS IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
IO<br />
KEY(GND)<br />
GND<br />
GND<br />
Single GND<br />
IO<br />
Single GND<br />
IO<br />
IO<br />
IO<br />
IO<br />
Single GND<br />
TENTH_INCHES<br />
HDR2X20<br />
2<br />
4<br />
6<br />
8<br />
10<br />
12<br />
14<br />
16<br />
18<br />
20<br />
22<br />
24<br />
26<br />
28<br />
30<br />
32<br />
34<br />
36<br />
38<br />
40<br />
ACCESS_PINS_D4<br />
ACCESS_PINS_D6<br />
ACCESS_PINS_D8<br />
ACCESS_PINS_D10<br />
ACCESS_PINS_D12<br />
ACCESS_PINS_D14<br />
ACCESS_PINS_D16<br />
ACCESS_PINS_D18<br />
ACCESS_PINS_D28<br />
ACCESS_PINS_D32<br />
ACCESS_PINS_D26<br />
ACCESS_PINS_D22<br />
ACCESS_PINS_D19<br />
The signals connect straight to the FPGA I/O pins. Note the signals are fixed at +2.5V. They are not<br />
+3.3V tolerant. If you connect a +3.3V device (like a hard drive) then FPGA A will likely be<br />
destroyed. The pin out is the same as a Ultra ATA cable. The Ultra ATA cable is capable of highspeed,<br />
relative to a standard IDC cable. This cable would be ideal to connect two boards to each<br />
other. Recall how I said that connecting a hard drive will damage the FPGA. This is because the<br />
DNV6F6PCIE User <strong>Manual</strong> Page 55
voltages driven by the hard drive will exceed +2.5V which is the maximum allowable voltage. The<br />
pin out was selected because the cable is easy to obtain, not because the FPGA is compatible with<br />
ATA.<br />
3.13 USER LEDS<br />
The FPGAs are all connected to LEDs which have the ON and OFF capability. Some of them go<br />
through FETs.<br />
There is not a lot to say about LEDs.<br />
+2.5V<br />
R971<br />
475R<br />
LED_Ar0<br />
DS46<br />
YELLOW<br />
LED_0603<br />
USER DEFINED LEDS<br />
1.8mA<br />
LED_Aq0<br />
Q16<br />
BSS138 3<br />
SOT-23N_GSD<br />
2<br />
1<br />
LED_A0<br />
R977<br />
475R<br />
LED_Ar1<br />
DS47<br />
YELLOW<br />
LED_0603<br />
LED_Aq1<br />
Q17<br />
BSS138 3<br />
SOT-23N_GSD<br />
2<br />
1<br />
LED_A1<br />
R981<br />
475R<br />
LED_Ar2<br />
DS48<br />
YELLOW<br />
LED_0603<br />
LED_Aq2<br />
Q18<br />
BSS138 3<br />
SOT-23N_GSD<br />
2<br />
1<br />
LED_A2<br />
R986<br />
475R<br />
LED_Ar3<br />
DS49<br />
YELLOW<br />
LED_0603<br />
LED_Aq3<br />
Q25<br />
BSS138 3<br />
SOT-23N_GSD<br />
2<br />
1<br />
LED_A3<br />
R990<br />
475R<br />
LED_Ar4<br />
DS50<br />
YELLOW<br />
LED_0603<br />
LED_Aq4<br />
Q32<br />
BSS138 3<br />
SOT-23N_GSD<br />
2<br />
1<br />
LED_A4<br />
If you pulse a signal to them, then they will be varying levels of brightness. They are sort of yellow<br />
in color. Other yellow items include sunflowers and fire.<br />
3.14 FPGA-to-FPGA ROCKETIO<br />
Most of the "Rocket IO" or "MGT" or "GTP" or "GTX" signals on the FPGA devices just connect to<br />
other FPGAs<br />
DNV6F6PCIE User <strong>Manual</strong> Page 56
Here is the connection topology shown in a diagram format. Note how some of the connections are<br />
drawn in a gold color. The signals that are represented in gold are only available when both FPGAs<br />
attached by them are "big" FPGAs, that is LX550T or SX475T.<br />
AH8<br />
AH7<br />
AK8<br />
AK7<br />
U0-33<br />
FPGA A<br />
MGTREFCLK1P_112<br />
MGTREFCLK1N_112<br />
MGTREFCLK0P_112<br />
MGTREFCLK0N_112<br />
XC6VLX240T/550T_FF1759<br />
MGTTXP0_112<br />
MGTTXN0_112<br />
MGTRXP0_112<br />
MGTRXN0_112<br />
MGTTXP1_112<br />
MGTTXN1_112<br />
MGTRXP1_112<br />
MGTRXN1_112<br />
MGTTXP2_112<br />
MGTTXN2_112<br />
MGTRXP2_112<br />
MGTRXN2_112<br />
MGTTXP3_112<br />
MGTTXN3_112<br />
MGTRXP3_112<br />
MGTRXN3_112<br />
AP3<br />
AP4<br />
AN5<br />
AN6<br />
AN1<br />
AN2<br />
AM7<br />
AM8<br />
AM3<br />
AM4<br />
AL5<br />
AL6<br />
AL1<br />
AL2<br />
AJ5<br />
AJ6<br />
MGT_ABn0<br />
MGT_ABp0<br />
MGT_BAn0<br />
MGT_BAp0<br />
MGT_ABn1<br />
MGT_ABp1<br />
MGT_BAn1<br />
MGT_BAp1<br />
MGT_ABn2<br />
MGT_ABp2<br />
MGT_BAn2<br />
MGT_BAp2<br />
MGT_ABn3<br />
MGT_ABp3<br />
MGT_BAn3<br />
MGT_BAp3<br />
U1-37<br />
E6<br />
E5<br />
MGTRXN3_117<br />
MGTRXP3_117<br />
F4<br />
F3<br />
MGTTXN3_117<br />
MGTTXP3_117<br />
F8<br />
F7<br />
MGTRXN2_117<br />
MGTRXP2_117<br />
G2<br />
G1<br />
MGTTXN2_117<br />
MGTTXP2_117<br />
G6<br />
G5<br />
MGTRXN1_117<br />
MGTRXP1_117<br />
H4<br />
H3<br />
MGTTXN1_117<br />
MGTTXP1_117<br />
H8<br />
H7<br />
MGTRXN0_117<br />
MGTRXP0_117<br />
J2<br />
J1<br />
MGTTXN0_117<br />
MGTTXP0_117<br />
FPGA B<br />
XC6VLX240T/550T_FF1759<br />
MGTREFCLK0N_117<br />
MGTREFCLK0P_117<br />
MGTREFCLK1N_117<br />
MGTREFCLK1P_117<br />
This is a schematic clipping making this section of the manual look much more complete.<br />
CLK_MGT_INTERCON_B117n 23<br />
CLK_MGT_INTERCON_B117p 23<br />
3.14.1 REFCLK Clocking<br />
There is a single clock network on the board, "CLK_MGT" that can be used with the FPGA-to-<br />
FPGA interconnect. This is the only clock network that can be used. You might be able to use a<br />
REFCLK that is sourced from the FPGA internally ("GREFCLK"), however you may not be able to<br />
achieve excessively high data rates using this method.<br />
In order to instantiate the MGT tile in your HDL, you must connect a "REFCLK" signal to the MGT<br />
tile. Only certain REFCLK inputs can be used with certain MGT inputs and outputs.<br />
G9<br />
G10<br />
E9<br />
E10<br />
DNV6F6PCIE User <strong>Manual</strong> Page 57
REFCLK<br />
125/250/312MHZ<br />
FPGA F<br />
FPGA E<br />
FPGA D<br />
SFP<br />
XXX<br />
XXX XXX<br />
118 117 116 115 114 113 112 111 110<br />
XXX<br />
XXX XXX XXX<br />
XXX XXX<br />
118 117 116 115 114 113 112 111 110 118 117 116 115 114 113 112 111 110<br />
SEAM<br />
SEAM<br />
SEAM<br />
110 111 112 113 114 115 116 117 118 110 111 112 113 114 115 116 117 118 110 111 112 113 114 115 116 117 118<br />
XXX XXX<br />
XXX XXX XXX<br />
XXX XXX XXX<br />
XXX<br />
FPGA C<br />
FPGA B<br />
FPGA A<br />
REFCLK<br />
125/250/312MHZ<br />
Therefore, the "CLK_MGT" network connects to multiple clock inputs on each FPGA. The above<br />
diagram shows how each REFCLK input on each FPGA is connected. For example, on FPGA E, if<br />
you want to use MGT tile number 114, you have to use the REFCLK input on tile 115. Tile 118,<br />
117, and 116 all share the single REFCLK input that is on tile 117. Does that make sense? Whatever.<br />
3.15 SPI FLASH<br />
Three of the user FPGAs, namely FPGA A, B, E and F have a serial flash device attached to them for<br />
the storage of warez.<br />
R983<br />
4.7K<br />
+2.5V +2.5V<br />
R1020<br />
4.7K<br />
U84<br />
FLASH_A_DOUT<br />
15<br />
8 FLASH_A_QINr R1021<br />
D<br />
Q<br />
R1022<br />
FLASH_A_CLK<br />
16<br />
3<br />
C DU<br />
4<br />
DU<br />
5<br />
DU<br />
6<br />
+2.5V<br />
DU<br />
11<br />
FLASH_A_CSN<br />
7<br />
DU<br />
12<br />
R1019 4.7K FLASH_A_WP<br />
9<br />
S DU<br />
13<br />
R984 4.7K FLASH_A_HOLD<br />
1<br />
W DU<br />
14<br />
+2.8V<br />
HOLD DU<br />
10<br />
GND<br />
SPI Flash for<br />
FPGA A<br />
VCC<br />
M25P128/SO16<br />
SOIC127P1030X265-16N<br />
2<br />
20mA<br />
C1810<br />
1uF<br />
100R FLASH_A_QIN<br />
1K<br />
SPI Serial<br />
FLASH<br />
(128Mbit)<br />
The interface is SPI. The SPI signals are at +2.5V levels.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 58
If this block diagram is correct, there are four SPI flash chips at your disposal. The part datasheet<br />
(provided) is a fascinating read.<br />
3.16 USER TEST POINTS<br />
Each FPGA has a single test point attached to it. They are located on the "SELECT_MAP" page of<br />
the schematic. It has no known function.<br />
3.17 Mictor Connector<br />
There is a Mictor connector on the back side of the board. It can be used for logic analyzers. The<br />
only signals that you can actually drive from an FPGA are SELECTMAP[8] - SELECTMAP[15].<br />
The rest of them cannot be used because they are driven from the config FPGA at all times.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 59
XC5VLX50T/85T/110T/SX50T/<br />
SX90T_FF1136 - Bank 18<br />
U51-13<br />
GROUND-> IO_L0P_18<br />
GROUND-> IO_L0N_18<br />
IO_L1P_18<br />
IO_L1N_18<br />
GROUND-> IO_L2P_18<br />
GROUND-> IO_L2N_18<br />
IO_L3P_18<br />
IO_L3N_18<br />
IO_L4P_18<br />
IO_L4N_VREF_18<br />
GROUND-> IO_L5P_18<br />
GROUND-> IO_L5N_18<br />
IO_L6P_18<br />
IO_L6N_18<br />
IO_L7P_18<br />
GROUND-> IO_L7N_18<br />
GROUND-> IO_L8P_CC_18<br />
IO_L8N_CC_18<br />
IO_L9P_CC_18<br />
IO_L9N_CC_18<br />
GROUND-> IO_L10P_CC_18<br />
GROUND-> IO_L10N_CC_18<br />
IO_L11P_CC_18<br />
IO_L11N_CC_18<br />
GROUND-> IO_L12P_VRN_18<br />
IO_L12N_VRP_18<br />
IO_L13P_18<br />
IO_L13N_18<br />
IO_L14P_18<br />
IO_L14N_VREF_18<br />
IO_L15P_18<br />
IO_L15N_18<br />
IO_L16P_18<br />
GROUND-> IO_L16N_18<br />
IO_L17P_18<br />
IO_L17N_18<br />
IO_L18P_18<br />
GROUND-> IO_L18N_18<br />
IO_L19P_18<br />
IO_L19N_18<br />
XC5VLX50T/85T/110T/SX50T/SX95T<br />
+2.5V<br />
AB30<br />
AE31<br />
AH32<br />
AJ29<br />
R31<br />
W29<br />
Config<br />
FPGA<br />
U0-11<br />
VCCO_24<br />
VCCO_24<br />
VCCO_24<br />
VCCO_24<br />
VCCO_24<br />
VCCO_24<br />
VCCO_18<br />
VCCO_18<br />
VCCO_18<br />
FPGA A<br />
IO_L0P_GC_24<br />
IO_L0N_GC_24<br />
IO_L1P_GC_24<br />
IO_L1N_GC_24<br />
IO_L2P_D15_24<br />
IO_L2N_D14_24<br />
IO_L3P_D13_24<br />
IO_L3N_D12_24<br />
IO_L4P_D11_24<br />
IO_L4N_VREF_D10_24<br />
IO_L5P_D9_24<br />
IO_L5N_D8_24<br />
IO_L6P_D7_24<br />
IO_L6N_D6_24<br />
IO_L7P_D5_24<br />
IO_L7N_D4_24<br />
IO_L8P_SRCC_24<br />
IO_L8N_SRCC_24<br />
IO_L9P_MRCC_24<br />
IO_L9N_MRCC_24<br />
IO_L10P_MRCC_24<br />
IO_L10N_MRCC_24<br />
IO_L11P_SRCC_24<br />
IO_L11N_SRCC_24<br />
IO_L12P_D3_24<br />
IO_L12N_D2_FS2_24<br />
IO_L13P_D1_FS1_24<br />
IO_L13N_D0_FS0_24<br />
IO_L14P_FCS_B_24<br />
IO_L14N_VREF_FOE_B_MOSI_24<br />
IO_L15P_FWE_B_24<br />
IO_L15N_RS1_24<br />
IO_L16P_RS0_24<br />
IO_L16N_CSO_B_24<br />
IO_L17P_VRN_24<br />
IO_L17N_VRP_24<br />
IO_L18P_24<br />
IO_L18N_24<br />
IO_L19P_24<br />
IO_L19N_24<br />
XC6VLX240T/550T_FF1759<br />
AC4<br />
AC5<br />
AB6<br />
AB7<br />
AA5<br />
AB5<br />
AC7<br />
AD7<br />
Y8<br />
Y9<br />
AD4<br />
AD5<br />
AA6<br />
Y7<br />
AD6<br />
AE6<br />
W6<br />
Y6<br />
AE7<br />
AF6<br />
AG5<br />
AF5<br />
W7<br />
V7<br />
AH5<br />
AG6<br />
Y11<br />
W11<br />
AH7<br />
AG7<br />
W10<br />
W9<br />
AJ7<br />
AJ6<br />
V8<br />
U8<br />
AK7<br />
AK6<br />
V10<br />
V9<br />
AC6<br />
W8<br />
AB9<br />
AE30<br />
AF30<br />
W30<br />
V30<br />
AG32<br />
AF31<br />
T30<br />
R30<br />
AH31<br />
AG31<br />
N33<br />
P33<br />
AJ33<br />
AH33<br />
P32<br />
R33<br />
AK33<br />
AJ32<br />
Y30<br />
AA30<br />
AA31<br />
AB31<br />
R32<br />
T32<br />
AK32<br />
AL32<br />
T31<br />
U31<br />
AH30<br />
AJ30<br />
V31<br />
W31<br />
AJ31<br />
AK30<br />
AC31<br />
AC30<br />
AH29<br />
AG29<br />
AD31<br />
AD30<br />
The mictor can also be used to configure a daughtercard, if the daughtercard has FPGAs on it. Note<br />
that the mictor connector has all of the required SELECTMAP signals on it for configuration. The<br />
EMU software supports configuring an FPGA using this method.<br />
3.18 USER SATA<br />
SELECTMAP_D15<br />
SELECTMAP_D14 +2.5V +2.5V<br />
SELECTMAP_D13<br />
SELECTMAP_D12<br />
SELECTMAP_D11<br />
SELECTMAP_D10<br />
R1335<br />
SELECTMAP_D9<br />
4.7K<br />
SELECTMAP_D8<br />
SELECTMAP_D7 R1325<br />
SELECTMAP_D6 4.7K<br />
SELECTMAP_D5<br />
SELECTMAP_D4<br />
SELECTMAP_D3<br />
SELECTMAP_D2<br />
SELECTMAP_D1<br />
SELECTMAP_D0<br />
RS232_RX<br />
RS232_TX<br />
FPGA_USER_RESETn_A<br />
FPGA_USER_RESETn_B<br />
FPGA_USER_RESETn_C<br />
FPGA_USER_RESETn_D<br />
FPGA_USER_RESETn_E<br />
FPGA_USER_RESETn_F<br />
FAN_TACH_A<br />
FAN_TACH_B<br />
FAN_TACH_D<br />
FAN_TACH_E<br />
FAN_TACH_C<br />
FAN_TACH_F<br />
FAN_TACH_CFPGA<br />
FAN_TACH_CPU<br />
MICTOR_CLK_LO<br />
MICTOR_CS15<br />
MICTOR_CS14<br />
MICTOR_CLK_HI<br />
MICTOR_RDWR<br />
MICTOR_DONE<br />
MICTOR_CCLK<br />
MICTOR_PROGn<br />
+2.5V<br />
SELECTMAP_D15<br />
SELECTMAP_D14<br />
SELECTMAP_D13<br />
SELECTMAP_D12<br />
SELECTMAP_D11<br />
SELECTMAP_D10<br />
SELECTMAP_D9<br />
SELECTMAP_D8<br />
SELECTMAP_D7<br />
SELECTMAP_D6<br />
SELECTMAP_D5<br />
SELECTMAP_D4<br />
SELECTMAP_D3<br />
SELECTMAP_D2<br />
SELECTMAP_D1<br />
SELECTMAP_D0<br />
RS232_RX<br />
RS232_TX<br />
FPGA_USER_RESETn_A<br />
Open drain<br />
please<br />
CLK_TO_SPARTAN_Ap 68<br />
MICTOR_CLK_LO<br />
SELECTMAP_D15<br />
SELECTMAP_D14<br />
SELECTMAP_D13<br />
SELECTMAP_D12<br />
SELECTMAP_D11<br />
SELECTMAP_D10<br />
SELECTMAP_D9<br />
SELECTMAP_D8<br />
SELECTMAP_D7<br />
SELECTMAP_D6<br />
SELECTMAP_D5<br />
SELECTMAP_D4<br />
SELECTMAP_D3<br />
SELECTMAP_D2<br />
SELECTMAP_D1<br />
SELECTMAP_D0<br />
CONFIG_EXTRA[11:0] 3,68<br />
Config Mictor<br />
Fixed 2.5V<br />
Do Not Connect<br />
MICTOR_CS15<br />
MICTOR_CS14<br />
MICTOR_CLK_HI<br />
MICTOR_RDWR<br />
MICTOR_DONE<br />
MICTOR_CCLK<br />
MICTOR_PROGn<br />
CONFIG_EXTRA11<br />
CONFIG_EXTRA10<br />
CONFIG_EXTRA9<br />
CONFIG_EXTRA8<br />
CONFIG_EXTRA7<br />
CONFIG_EXTRA6<br />
CONFIG_EXTRA5<br />
CONFIG_EXTRA4<br />
CONFIG_EXTRA3<br />
CONFIG_EXTRA2<br />
CONFIG_EXTRA1<br />
CONFIG_EXTRA0<br />
FPGA F has two SATA connectors attached to its MGTs. These can be used to make a SATA<br />
interface. One of the connectors can only be used when the FPGA is acting as a device. One of the<br />
1<br />
3<br />
5<br />
7<br />
9<br />
11<br />
13<br />
15<br />
17<br />
19<br />
21<br />
23<br />
25<br />
27<br />
29<br />
31<br />
33<br />
35<br />
37<br />
39<br />
40<br />
41<br />
J43<br />
1<br />
3 GND<br />
5 CLK<br />
7 D15<br />
9<br />
11<br />
13<br />
15<br />
17<br />
19<br />
21<br />
23<br />
25<br />
27<br />
29<br />
31<br />
33<br />
35<br />
37 D0<br />
GND<br />
GND<br />
GND<br />
2-767004-2<br />
CONN_MICTOR38<br />
CLK<br />
D15<br />
D0<br />
2<br />
4<br />
6<br />
8<br />
10<br />
12<br />
14<br />
16<br />
18<br />
20<br />
22<br />
24<br />
26<br />
28<br />
30<br />
32<br />
34<br />
36<br />
38<br />
LOC<br />
GND<br />
GND<br />
2<br />
4<br />
6<br />
8<br />
10<br />
12<br />
14<br />
16<br />
18<br />
20<br />
22<br />
24<br />
26<br />
28<br />
30<br />
32<br />
34<br />
36<br />
38<br />
44<br />
42<br />
43<br />
DNV6F6PCIE User <strong>Manual</strong> Page 60
SATA - DEVICE<br />
SATA - HOST<br />
connectors can only be used when the FPGA is acting as a controller. In this way, we prevent<br />
customers from emulating a raid controller.<br />
There is a dedicated oscillator connected to an appropriate REFCLK input. The frequency select pins<br />
of the oscillator are connected to the FPGA so that the user can select an appropriate REFCLK<br />
frequency for SATA I or SATA II.<br />
60<br />
60<br />
OSC_SFP_FS0<br />
OSC_SFP_FS1<br />
+2.5V_QUIET<br />
R573<br />
4.7K<br />
R587<br />
4.7K<br />
OSC_SFP_OE<br />
R567<br />
4.7K<br />
7<br />
8<br />
2<br />
3<br />
G1<br />
FS1<br />
FS0<br />
OE<br />
GND<br />
SI534/SMT-8<br />
Note: Frequency<br />
Select:<br />
FS: 00 - 125 MHz<br />
FS: 01 - 150 MHz<br />
FS: 10 - 250 MHz<br />
FS: 11 - 312.5<br />
MHz<br />
NC<br />
VDD<br />
4<br />
5<br />
1<br />
6<br />
OSC_SFP_NC<br />
OSC_SFP_VDD<br />
C148<br />
0.1uF<br />
C146<br />
4.7uF<br />
R588<br />
4.7K<br />
FB1<br />
FB1608<br />
CLK_SFP_SOURCEp<br />
CLK_SFP_SOURCEn<br />
+2.5V_QUIET<br />
20 Ohm @ 1 MHZ<br />
0.14 Ohm @ DC<br />
FBMH1608HM102-T<br />
C1040<br />
C1041<br />
0.1uFCLK_SFP_SOURCEpc<br />
0.1uFCLK_SFP_SOURCEnc<br />
M7<br />
M8<br />
K7<br />
K8<br />
U5-36<br />
FPGA F<br />
MGTREFCLK0N_116<br />
MGTREFCLK0P_116<br />
MGTREFCLK1N_116<br />
MGTREFCLK1P_116<br />
XC6VLX240T/550T_FF1759<br />
MGTRXN3_116<br />
MGTRXP3_116<br />
MGTTXN3_116<br />
MGTTXP3_116<br />
MGTRXN2_116<br />
MGTRXP2_116<br />
MGTTXN2_116<br />
MGTTXP2_116<br />
MGTRXN1_116<br />
MGTRXP1_116<br />
MGTTXN1_116<br />
MGTTXP1_116<br />
MGTRXN0_116<br />
MGTRXP0_116<br />
MGTTXN0_116<br />
MGTTXP0_116<br />
J6<br />
J5<br />
K4<br />
K3<br />
L6<br />
L5<br />
L2<br />
L1<br />
N6<br />
N5<br />
M4<br />
M3<br />
P8<br />
P7<br />
N2<br />
N1<br />
SATA_A_TxP1_c<br />
SATA_A_TxN1_c<br />
SATA_A_RxN1_c<br />
SATA_A_RxP1_c<br />
C991<br />
C1013<br />
C1025<br />
C1036<br />
0.1uF SATA_A_TxP1<br />
CAPC1005N<br />
0.1uF SATA_A_TxN1<br />
CAPC1005N<br />
0.1uF SATA_A_RxN1<br />
CAPC1005N<br />
0.1uF SATA_A_RxP1<br />
CAPC1005N<br />
1<br />
2<br />
3<br />
4<br />
5<br />
6<br />
7<br />
8<br />
9<br />
J14<br />
CLK+<br />
CLK-<br />
GND<br />
TX+<br />
TX-<br />
GND<br />
RX-<br />
RX+<br />
GND<br />
MTH<br />
MTH<br />
67800-5005<br />
MOLEX_67800-005-7_UPDATE<br />
SATA_A_TxP0_c<br />
SATA_A_TxN0_c<br />
SATA_A_RxN0_c<br />
SATA_A_RxP0_c<br />
C1222<br />
C1230<br />
C1254<br />
C1263<br />
0.1uF SATA_A_TxP0<br />
0.1uF CAPC1005N SATA_A_TxN0<br />
CAPC1005N<br />
0.1uF SATA_A_RxN0<br />
0.1uF CAPC1005N SATA_A_RxP0<br />
CAPC1005N<br />
J15<br />
1<br />
2<br />
GND<br />
3<br />
TX+<br />
4<br />
TX-<br />
5<br />
GND<br />
6<br />
RX-<br />
7<br />
RX+<br />
GND<br />
8<br />
9<br />
MTH<br />
MTH<br />
67800-5005<br />
The below photo shows the location of the SATA connectors.<br />
MOLEX_67800-005-7_UPDATE<br />
3.19 SFP AND ETHERNET<br />
FPGA F is also attached to an SFP connector. SFP connectors are useful for Fibrechannel or<br />
Ethernet. Since the FPGA is only capable of serial speeds up to 6Gbs, SFP+ modules probably can't<br />
be used.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 61
SFP JACK<br />
VIH: 2.1V<br />
SFP+ connector (one<br />
channel)<br />
+2.5V<br />
MOD-DEF0 is MOD_ABS (pulled to ground inside module)<br />
MOD-DEF1 is SCL<br />
MOD-DEF2 is SDA<br />
60<br />
60<br />
OSC_SFP_FS0<br />
OSC_SFP_FS1<br />
+2.5V_QUIET<br />
R573<br />
4.7K<br />
R587<br />
4.7K<br />
OSC_SFP_OE<br />
R567<br />
4.7K<br />
7<br />
8<br />
2<br />
3<br />
G1<br />
FS1<br />
FS0<br />
OE<br />
GND<br />
SI534/SMT-8<br />
Note: Frequency<br />
Select:<br />
FS: 00 - 125 MHz<br />
FS: 01 - 150 MHz<br />
FS: 10 - 250 MHz<br />
FS: 11 - 312.5<br />
MHz<br />
VEET<br />
TD-<br />
TD+<br />
VEET<br />
VCCT<br />
VCCR<br />
VEER<br />
RD+<br />
RD-<br />
VEER<br />
CLK+<br />
CLK-<br />
NC<br />
VDD<br />
4<br />
5<br />
1<br />
6<br />
OSC_SFP_NC<br />
OSC_SFP_VDD<br />
C148<br />
0.1uF<br />
C146<br />
4.7uF<br />
R588<br />
4.7K<br />
FB1<br />
FB1608<br />
CLK_SFP_SOURCEp<br />
CLK_SFP_SOURCEn<br />
+2.5V_QUIET<br />
20 Ohm @ 1 MHZ<br />
0.14 Ohm @ DC<br />
FBMH1608HM102-T<br />
C1040<br />
C1041<br />
0.1uFCLK_SFP_SOURCEpc<br />
0.1uFCLK_SFP_SOURCEnc<br />
M7<br />
M8<br />
K7<br />
K8<br />
U5-36<br />
FPGA F<br />
MGTREFCLK0N_116<br />
MGTREFCLK0P_116<br />
MGTREFCLK1N_116<br />
MGTREFCLK1P_116<br />
XC6VLX240T/550T_FF1759<br />
MGTRXN3_116<br />
MGTRXP3_116<br />
MGTTXN3_116<br />
MGTTXP3_116<br />
MGTRXN2_116<br />
MGTRXP2_116<br />
MGTTXN2_116<br />
MGTTXP2_116<br />
MGTRXN1_116<br />
MGTRXP1_116<br />
MGTTXN1_116<br />
MGTTXP1_116<br />
MGTRXN0_116<br />
MGTRXP0_116<br />
MGTTXN0_116<br />
MGTTXP0_116<br />
The SFP connector also has some low-speed sideband signals which also attach to the FPGA. These<br />
signals are fixed at +2.5V. A dedicated oscillator is provided that has a random variety pack of<br />
frequencies. You can select between these frequencies using the frequency select pins of the<br />
oscillator, which are attached to the FPGA. If the frequencies provided in the variety pack aren't<br />
appropriate you have one of the following options, which I've listed after this photo.<br />
J6<br />
J5<br />
K4<br />
K3<br />
L6<br />
L5<br />
L2<br />
L1<br />
N6<br />
N5<br />
M4<br />
M3<br />
P8<br />
P7<br />
N2<br />
N1<br />
+3.3_SFP_TX<br />
+3.3_SFP_RX<br />
MGT_SFP_TXn<br />
MGT_SFP_TXp<br />
MGT_SFP_RXp<br />
MGT_SFP_RXn<br />
20<br />
19<br />
18<br />
17<br />
16<br />
15<br />
14<br />
13<br />
12<br />
11<br />
40<br />
39<br />
38<br />
37<br />
36<br />
35<br />
34<br />
33<br />
32<br />
31<br />
J1<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
1367073<br />
TOP<br />
BOTTOM<br />
VEET<br />
O.D. TxFAULT<br />
3.3V TxDISABLE<br />
4.7K - 10K<br />
SDA<br />
VIH: 2.3V<br />
VIH: 2.3V SCL<br />
MOD_ABS<br />
VIH: 2.1V RS0<br />
O.D. RX_LOS<br />
RS1<br />
VEER<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
CAGE<br />
1<br />
2<br />
3<br />
4<br />
5<br />
6<br />
7<br />
8<br />
9<br />
10<br />
21<br />
22<br />
23<br />
24<br />
25<br />
26<br />
27<br />
28<br />
29<br />
30<br />
SFP_TXFAULT<br />
SFP_TXDISABLE<br />
SFP_MODABS<br />
R323<br />
4.7K<br />
R339<br />
DNI<br />
R346<br />
100R<br />
R345<br />
1K<br />
R344<br />
1K<br />
R349<br />
4.7K<br />
R348<br />
4.7K<br />
R350<br />
4.7K<br />
R354<br />
4.7K<br />
SFP_SDA 60<br />
SFP_SCL 60<br />
SFP_AS0 60<br />
SFP_LOS 60<br />
SFP_AS1 60<br />
1) Replace the oscillator with a better one. We will help you do this if you want.<br />
2) Use the CLK_MGT network. The CLK_MGT network can be driven from a frequency<br />
synthesizer that can hit any frequency that the world has ever known. The disadvantage is that then<br />
the entire CLK_MGT network must run at this frequency, possibly limiting your options for the<br />
FPGA-to-FPGA interconnect.<br />
3.19.1 IIC<br />
There are sideband signals on the SFP. They are attached to the FPGA. You are expected to figure<br />
out what to do with them.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 62
3.20 ROCKETIO HEADER<br />
Brand new in Virtex 6 just for you we invented a new type of header, the "GTX Expansion Header<br />
Interface" (abbreviated S.E.A.M.)<br />
There are three of these on the DNV6F6PCIE, attached to FPGAs A, C and D.<br />
We don't have any off-the-shelf SEAM daughter cards yet, so you are expected to design your own.<br />
Send a request to sales@dinigroup.com to see what we are willing to do for you.<br />
Each SEAM provides 8 channels of high-speed serial data to the FPGA. There are four REFCLK<br />
signals from the SEAM connector to the FPGA, making four independent interfaces on one SEAM<br />
connection feasible. Additionally, for the purpose of control, there are 16 "low speed" I/O that go to<br />
the FPGA "regular" I/O pins.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 63
Here is a photo of the SEAM connector, on the back side of the board.<br />
Here is a schematic clipping showing the connection between the FPGA and the SEAM card. The<br />
connection is electrically straight-through. The low speed I/O is fixed at +2.5V signaling levels.<br />
Three voltages are provided to the SEAM card, +3.3V, +12V and "VCCO". The VCCO supply is<br />
fixed at +2.5V, however the daughter card designer should keep in mind that future Dini Boards may<br />
chose to supply a different (probably lower) fixed voltage here, such as +1.8V.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 64
The pin out is designed for extreme high speed like you can't fathom.<br />
3.20.1 Mechanical<br />
The connector on the DNV6F6PCIE is a Samtec SEAM-20-03.5-S-08-2-A. The connector that you<br />
should use on a daughter card is Samtec SEAF-20-03.5-S-08-2-A. The connection scheme, like all<br />
sane connection schemes, is 1-1, 2-2, 3-3, 4-4, 5-5, 6-6, 7-7, 8-8, 9-9, 10-10, 11-11, 12-12, 13-13,<br />
14-14, 15-15, 16-16, 17-17, 18-18, 19-19, 20-20, 21-21, 22-22, 23-23, 24-24, 25-25, 26-26, 27-27,<br />
28-28, 29-29, 30-30, 31-31, 32-32, 33-33, 34-34, 35-35, 36-36, 37-37, etc.<br />
3.21 ENCRYPTION<br />
The Virtex 6 FPGA allows the use of encrypted bit files. You would want to encrypt a bit file if you<br />
want some person to pay you for the use of your IP per instance or something I don't know why.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 65
1<br />
2<br />
3<br />
In order to support encryption, we have provided the necessary battery on the board. This battery<br />
supplies voltage to the VBATT pin of the FPGA, even when the board is off. It also provides power<br />
to the VBAT pin of the real time clock on the board. Note that the real time clock does not work on<br />
revision 01 of the circuit board.<br />
The above photo shows where one might install a battery on the DNV6F6PCIE<br />
+3.3V<br />
Backup Battery Linear Regulator 2.5V @ 1uA<br />
64 +3.3VAUX_PEX0<br />
D13<br />
BAS40-05/SOT23<br />
SOT96P240X110-3N<br />
3<br />
VBATT_BAT 1 2<br />
D12<br />
BAS40-05/SOT23<br />
BT1 SOT96P240X110-3N<br />
3001<br />
KEYSTONE_3001<br />
VBATT_EXT<br />
VBATT_PRE<br />
TP41<br />
DNI<br />
TESTPOINT<br />
C1359<br />
4.7uF<br />
R891<br />
U78<br />
1<br />
IN<br />
Silkscreen: "VBATT_EXT"<br />
3<br />
2<br />
EN<br />
GND<br />
DNI<br />
OUT<br />
GND<br />
5<br />
4<br />
TPS78225/SOT23-5<br />
SOT95P280X110-5N<br />
TP42<br />
DNI<br />
TESTPOINT<br />
C1360<br />
1uF<br />
+VBATT<br />
Silkscreen: "CR1220"<br />
Accepts 3V Laser<br />
BR1220-1<br />
This circuit makes the section look more complete.<br />
Use a CR2012 type battery in the socket. RadioShack type 365. In order to change the battery<br />
without the loss of the encryption key you can attempt one of the following feats of daring.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 66
1) Switch the battery out while the board is powered on.<br />
2) Switch the battery out while the board is plugged into the PCI Express slot of a computer that is<br />
plugged in, but powered down.<br />
3) Attach some sort of external power supply to test point TP41.<br />
4) Switch the battery out in less than 10 seconds.<br />
5) Reprogram the encryption key after changing the battery.<br />
3.22 JTAG<br />
The FPGA JTAG chain of the Virtex 6 FPGAs is available for your use. It isn't used by any other<br />
circuit on the board. It just connects right up to a header that the Xilinx programmer cable connects<br />
to.<br />
The order of the FPGAs on the chain is A, B, C, D, E, F. The configuration FPGA is not attached to<br />
the same chain as the user FPGAs. Note that if you have ordered the board with fewer than six<br />
FPGAs, the JTAG chain will remain unbroken, however there will be a missing FPGA in the chain.<br />
So if you have only FPGA C, E and F on the board, then the chain will appear to only have three<br />
FPGAs on it.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 67
This photo gives an idea to you where the header of JTAG lies.<br />
+2.5V<br />
To<br />
FPGAs<br />
JTAG_TCK_A<br />
JTAG_TCK_B<br />
JTAG_TCK_C<br />
R976 33R JTAG_TCK_Ar<br />
R975 33R JTAG_TCK_Br<br />
R974 33R JTAG_TCK_Cr<br />
To<br />
FPGAs<br />
3.23 MECHANICAL<br />
If you wish to build a plate of aluminum with M3 holes pre-drilled to act as a base plate for the<br />
board, then the diagram below will be very helpful to you.<br />
The holes, except for the "SEAM mounting" holes are 3mm. They are all attached to "gound" on the<br />
board. The back side clearance is 14mm. The front side clearance is 1.75". The board thickness is<br />
0.063"<br />
DNV6F6PCIE User <strong>Manual</strong> Page 69
The above diagram is the same one as the one above it, except that this one is reversed.<br />
3.24 POWER<br />
text<br />
3.24.1 Power Headers<br />
There are two ingress points for power on the board. They are both compatible with the so-called<br />
"PCI Express graphics power" connector. This connector has 6-pins, and is increasingly common on<br />
power supplies. There are also some 8-pin version of "PCI Express graphic power" connectors on<br />
some power supplies. These can be plugged into the DNV6F6PCIE as well, with the extra two pins<br />
dangling uselessly off to the side. Both power connectors must be installed on the board for it to<br />
operate properly. The current pulled from each connector can exceed 15A (at 12V), which is already<br />
more than the connector is designed to supply.<br />
There are three jumper points on the board that can connect the two 12V "halves" of the board<br />
together, in case you really really want to run the board with one cable. There is also a jumper point<br />
that allows you to connect the 12V of the DNV6F6PCIE to the PCI Express edge connector 12V if<br />
you want to power the board without any cables at all. This is not recommended because if you draw<br />
more than, say 40W of power off the finger connector, it could burn the valuable gold plating off the<br />
connector.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 70
1 3<br />
2<br />
1 3<br />
2<br />
From 300W PCI Express<br />
Specification<br />
PCIe Power Cable (4A/pin)<br />
accepts 6-pin or 8-pin<br />
connector<br />
place LED near PCI<br />
Express power<br />
connector<br />
J7<br />
TOP ROW<br />
1<br />
5<br />
2<br />
12V GND<br />
6<br />
3<br />
12V Sense0<br />
7<br />
4<br />
12V GND<br />
8<br />
M2<br />
Sense1 GND<br />
M1<br />
NC NC<br />
45558-0002<br />
MOLEX_45558-0002_UPDATED<br />
R352<br />
10K_0.1%<br />
DS35<br />
RED<br />
LED_0603<br />
1.2mA<br />
12V_WARN_R<br />
12V_WARN_L<br />
8A<br />
8A<br />
TP1<br />
DNI<br />
TESTPOINT<br />
TP74<br />
DNI<br />
TESTPOINT<br />
1.2mA<br />
DS36<br />
RED<br />
LED_0603<br />
R351<br />
10K_0.1%<br />
place LED near PCI<br />
Express power<br />
connector<br />
1<br />
2<br />
3<br />
4<br />
M2<br />
J8<br />
TOP ROW<br />
12V GND<br />
12V Sense0<br />
12V GND<br />
Sense1 GND<br />
NC NC<br />
5<br />
6<br />
7<br />
8<br />
M1<br />
45558-0002<br />
MOLEX_45558-0002_UPDATED<br />
PCIe Power Cable (4A/pin)<br />
accepts 6-pin or 8-pin<br />
connector<br />
D2<br />
Note: Reverse polarity<br />
protection.<br />
8A<br />
TP10<br />
DNI<br />
TESTPOINT<br />
D3<br />
C628<br />
4.7uF_12V<br />
CAPC3225N<br />
+12V_L<br />
C53<br />
0.1uF<br />
Silkscreen: "+12V"<br />
TP4<br />
DNI<br />
TESTPOINT<br />
So you could use<br />
only one cable<br />
place kind of far apart<br />
from each other (1 inch or<br />
more)<br />
Note: Reverse polarity<br />
protection.<br />
+12V_R<br />
Silkscreen: "+12V"<br />
TP5<br />
DNI<br />
C629<br />
C57<br />
TESTPOINT<br />
4.7uF_12V 0.1uF<br />
Above is a schematic clipping to make this section seem more complete. From it you can see that the<br />
power connector connects the power.<br />
If you weren't able to locate the connectors on the board using the previously recommended "look on<br />
the board" method, I have drawn a diagram above with big red circles around them.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 71
+12V_PEX0<br />
PCI Express Edge Connector<br />
(X4)<br />
+3.3V_PEX0<br />
8 +3.3VAUX_PEX0<br />
To battery<br />
PEX0_R_p0<br />
PEX0_R_n0<br />
PEX_PRSNTn1<br />
PEX0_R_p1<br />
PEX0_R_n1<br />
PEX0_R_p2<br />
PEX0_R_n2<br />
PEX0_R_p3<br />
PEX0_R_n3<br />
PEX_PRSNTn4<br />
B1<br />
B2<br />
B3<br />
B4<br />
B5<br />
B6<br />
B7<br />
B8<br />
B9<br />
B10<br />
B11<br />
B12<br />
B13<br />
B14<br />
B15<br />
B16<br />
B17<br />
B18<br />
B19<br />
B20<br />
B21<br />
B22<br />
B23<br />
B24<br />
B25<br />
B26<br />
B27<br />
B28<br />
B29<br />
B30<br />
B31<br />
B32<br />
P4<br />
+12V<br />
+12V<br />
+12V<br />
GND<br />
SMCLK<br />
SMDAT<br />
GND<br />
+3.3V<br />
TRST<br />
+3.3VAUX<br />
WAKE<br />
RSVD<br />
GND<br />
PETp0<br />
PETn0<br />
GND<br />
PRSNT2<br />
GND<br />
PETp1<br />
PETn1<br />
GND<br />
GND<br />
PETp2<br />
PETn2<br />
GND<br />
GND<br />
PETp3<br />
PETn3<br />
GND<br />
RSVD<br />
PRSNT2<br />
GND<br />
KEY<br />
PRSNT1<br />
+12V<br />
+12V<br />
GND<br />
TCK<br />
TDI<br />
TDO<br />
TMS<br />
+3.3V<br />
+3.3V<br />
PERST<br />
GND<br />
REFCLK+<br />
REFCLK-<br />
GND<br />
PERp0<br />
PERn0<br />
GND<br />
RSVD<br />
GND<br />
PERp1<br />
PERn1<br />
GND<br />
GND<br />
PERp2<br />
PERn2<br />
GND<br />
GND<br />
PERp3<br />
PERn3<br />
GND<br />
RSVD<br />
PCI_EXPRESS X 4<br />
PCI_EXPRESS_X_4<br />
A1<br />
A2<br />
A3<br />
A4<br />
A5<br />
A6<br />
A7<br />
A8<br />
A9<br />
A10<br />
A11<br />
A12<br />
A13<br />
A14<br />
A15<br />
A16<br />
A17<br />
A18<br />
A19<br />
A20<br />
A21<br />
A22<br />
A23<br />
A24<br />
A25<br />
A26<br />
A27<br />
A28<br />
A29<br />
A30<br />
A31<br />
A32<br />
PEX0_TDIO<br />
+3.3V_PEX0<br />
PEX0_PERSTn R1435<br />
R1436<br />
CLK_PEX0_p0<br />
CLK_PEX0_n0<br />
PEX0_T_p0_c<br />
PEX0_T_n0_c<br />
PEX0_T_p1_c<br />
PEX0_T_n1_c<br />
Note: Use multiple power VIA's to connect<br />
+3.3V/+12V to the voltage planes on the PCB.<br />
RST_PEX0n_CPU<br />
RST_PEX0n_POR<br />
The power pins on the PCI Express edge connector are left unused. This is because they are superweak.<br />
3.24.2 Distribution Chart<br />
Power for the board is all derived from 12V. Lower voltages are generated either directly by<br />
converting 12V DC input, or by down converting another voltage that was down converted from<br />
12V.<br />
F24<br />
F16<br />
PEX_PRSNTn<br />
PEX0_T_p2_c<br />
PEX0_T_n2_c<br />
C515<br />
C516<br />
C517<br />
C518<br />
C519<br />
C520<br />
+12V_L<br />
DNI<br />
FUSE_0429<br />
DNI<br />
FUSE_0429<br />
R268<br />
R269<br />
0R<br />
DNI<br />
0.1uF<br />
0.1uF<br />
0.1uF<br />
0.1uF<br />
0.1uF<br />
0.1uF<br />
PEX0_T_p3_c C521 0.1uF<br />
PEX0_T_n3_c C522 0.1uF<br />
0R<br />
DNI<br />
R263<br />
475R<br />
PEX0_T_p0<br />
PEX0_T_n0<br />
PEX0_T_p1<br />
PEX0_T_n1<br />
PEX0_T_p2<br />
PEX0_T_n2<br />
PEX0_T_p3<br />
PEX0_T_n3<br />
PEX_PRSNTn4<br />
PEX_PRSNTn1<br />
PEX0_LED_A<br />
3.7mA<br />
DS76<br />
+3.3V<br />
RED<br />
LED_0603<br />
Silkscreen: "PWR FAIL"<br />
RST_PEX0n_CPU 67<br />
RST_PEX0n_POR 67<br />
HCSL<br />
DNV6F6PCIE User <strong>Manual</strong> Page 72
12V_L<br />
24.5A<br />
9A<br />
VCCO LDO<br />
3A<br />
3A<br />
3A<br />
12V_R<br />
22A<br />
Other<br />
5Ain<br />
3.3V<br />
5.5Ain<br />
DCF<br />
2A 2A<br />
DCE<br />
2A 2A<br />
DCD<br />
2A 2A<br />
4 FANS<br />
2Ain<br />
5.0V<br />
9Ain<br />
F<br />
1Vx30A<br />
3Ain<br />
E<br />
1Vx30A<br />
3Ain<br />
D<br />
1Vx30A<br />
3Ain<br />
4 FANS<br />
2Ain<br />
????<br />
AMPS<br />
3Ax2V<br />
1Ain<br />
3Ax2V<br />
1Ain<br />
1.1V_CPU<br />
1.8V_CPU<br />
1.0V_CPU<br />
1.0V_V5T<br />
Above I have provided a chart that shows where the current comes from for each power supply. I<br />
doubt there is any reason for you to use this chart.<br />
3.25 HEAT<br />
DIMMF<br />
DIMMC<br />
This board gets very hot.<br />
GTP<br />
1Vx13A<br />
3Ain<br />
C<br />
1Vx30A<br />
3Ain<br />
GTP<br />
1Vx13A<br />
3Ain<br />
3Ax2V<br />
1Ain<br />
3Ax2V<br />
1Ain<br />
3.25.1 Total thermal performance<br />
The heat sinks that were installed on the board before you removed them were each capable of<br />
dissipating one Watt for every degree (in Celsius) that the FPGA under it raises above the ambient<br />
temperature. For example, if you are using the board in a room at 25 degrees, and you configure one<br />
of the FPGAs with a design that uses 31 Watts, the core temperature of the FPGA will rise by 31<br />
degrees, for a total temperature of 56 degrees.<br />
You can use the Xpower tool in Xilinx to determine how much power your FPGA design uses. The<br />
amount of power that your FPGA uses may limit the maximum ambient temperature that your board<br />
can operate in. The FPGA is not guaranteed to function properly at core temperatures above 80<br />
degrees C.<br />
For example, if you put the board in a computer case with the lid closed, and the temperature inside<br />
the computer case is 65 degrees C, then the maximum power that your FPGA design can use and still<br />
operate reliably is 15 Watts. Note that ambient (air near the board) temperature measurements must<br />
be taken at full power.<br />
B<br />
1Vx30A<br />
3Ain<br />
GTP<br />
1Vx13A<br />
3Ain<br />
A<br />
1Vx30A<br />
3Ain<br />
DIMMD<br />
DIMMA<br />
2.5V<br />
4Ain<br />
Intercon<br />
2.5Vx18A<br />
DNV6F6PCIE User <strong>Manual</strong> Page 73
3.25.2 FANS<br />
The fans that are sitting on top of the heat sink are plugged into the board for power. They have a<br />
tachometer. The frequency of operation, in revolutions per minute, can be read from the EMU host<br />
software.<br />
FAN_TACH_A<br />
+2.5V<br />
R894<br />
R888<br />
1K<br />
1K<br />
+5.0V<br />
C1391<br />
0.1uF<br />
FAN_TACH_Ar<br />
Cooling Fan<br />
1<br />
2<br />
3<br />
J19<br />
GND<br />
VCC<br />
TACH<br />
22-27-2031<br />
22-23-2031-3<br />
If your fans start to make noise they need to be replaced. We will send you new ones.<br />
3.25.3 Temperature Sensors<br />
Each FPGA has a temperature sensor attached to it to measure the temperature of the FPGA code<br />
("die temperature"). Since correct operation of the FPGA is not guaranteed by Xilinx when the core<br />
temperature increases above 80 degrees, we helpfully reset the FPGA for you when the temperature<br />
hits 80. This behavior can be changed if you want, but you'll have to call and ask us how.<br />
3.26 RESET<br />
There is a board-wide reset circuit called "SYS_RESET" or "SYS_RSTn" or some flavor. It's<br />
purpose is two-and-a-half fold<br />
1) Cause power supplies to come up in a particular order<br />
2) Cause each device on the board to get a reset pulse after power is applied as required by the<br />
device datasheet with the minimum pulse width specified in the datasheet<br />
3) Prevent the user from using the board if any power supply is off by even a little bit<br />
4) Allow the user a way to reset the board to a repeatable state without having to power the board<br />
down and back up.<br />
3.26.1 Particular order<br />
The power supplies are allowed to supply voltage in a particular order.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 74
1.0V<br />
FPGAs<br />
MONITOR<br />
@ 3%<br />
3.3V<br />
1.8V<br />
2.5V<br />
MONITOR<br />
@ 4%<br />
MONITOR<br />
@ 10%<br />
MONITOR<br />
@ 4%<br />
Daughtercards<br />
DIMMS<br />
1.0V_CPU<br />
1.1V_CPU<br />
MONITOR<br />
@ ?%<br />
Button<br />
MPP13<br />
MPP21<br />
1/2<br />
Second<br />
JUMP<br />
Button<br />
CFPGA<br />
PROG<br />
IO IO<br />
Button<br />
PCIE<br />
Reset<br />
CPU<br />
SYSRST<br />
IO<br />
IO<br />
This diagram seems topical.<br />
3.26.2 Power supply failure detection<br />
There is a circuit on the board to detect when any of the voltages on the board falls below some<br />
minimum voltage.<br />
+3.3V_SEQ<br />
+3.3V<br />
R445 13.7K<br />
R441 19.1K<br />
C698 0.1uF<br />
TRIP_3.3<br />
U68<br />
3<br />
+INA<br />
405mV<br />
LT6700-3<br />
1<br />
OUTA<br />
FAIL_3.3V#<br />
FAIL_3.3V#q<br />
4<br />
+INB<br />
3.15V<br />
6<br />
383mV<br />
5<br />
OUTB<br />
SEQ_DISABLE_CPU_PWR# 58<br />
R450<br />
2<br />
VS<br />
GND<br />
4%<br />
1K<br />
C695<br />
LT6700-3<br />
0.1uF<br />
SOT95P280-6N<br />
DS27<br />
RED<br />
R315<br />
4.7K<br />
+12V_R<br />
If this happens, the result is identical to holding down the "SYS RST" button. A red LED comes on,<br />
and the board won't work at all even.<br />
3.26.3 Reset Button<br />
The "SYS_RSTn" button asserts the same signal that the power fail button does. So hopefully it<br />
returns the board to the same state that a power-on does.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 75
Power Supply Sequencer/RESET<br />
+3.3V_SEQ<br />
Reset Circuit<br />
RST_PORn_LED<br />
R1026<br />
59 PWR_FAULTn<br />
64 RST_PEX0n_POR<br />
62,63 RST_CPU_MPP13n<br />
63 JT_SRSTn<br />
+3.3V_SEQ<br />
+3.3V_SEQ<br />
+3.3V_SEQ<br />
C1902<br />
1uF<br />
R1028 R1032 R1031<br />
R1030<br />
4.7K 4.7K 4.7K U88<br />
4.7K<br />
8<br />
PWR_FAULTn<br />
1<br />
Vcc<br />
7 RST_EXTn<br />
RST_PEX0n_POR<br />
3<br />
1A 1Y<br />
5<br />
RST_CPU_MPP13n<br />
6<br />
2A<br />
O.D.<br />
2Y<br />
2<br />
C1897<br />
4<br />
3A 3Y<br />
1000pF<br />
GND<br />
74LVC3G07DCT<br />
U87<br />
TSOP65P400X130-8N<br />
5<br />
1<br />
SENSE RESET<br />
FORCE_RSTn<br />
3<br />
SUPERVISORY_CT 4<br />
MR<br />
R1038 DNI +3.3V_SEQ<br />
CT<br />
C1893<br />
2<br />
6<br />
0.1uF<br />
GND VDD<br />
TPS3808/SOT23-6<br />
R1027<br />
10K_0.1%<br />
+3.3V_SEQ<br />
R1037<br />
10K_0.1%<br />
RST_PORn<br />
+3.3V_SEQ<br />
C1907<br />
1uF<br />
DS54<br />
RED<br />
TP63<br />
DNI<br />
2mA<br />
475R<br />
Everything after<br />
this is not reliable<br />
before POR<br />
+3.3V_SEQ<br />
Recovery jumper<br />
C1892 +1.8V_CPU<br />
TP61<br />
1uF<br />
R1014<br />
U86<br />
4.7K<br />
8<br />
1<br />
Vcc<br />
7<br />
RST_CPUn<br />
3<br />
1A 1Y<br />
RST_CPUn 63<br />
5<br />
6<br />
2A<br />
O.D.<br />
2Y<br />
2<br />
4<br />
3A 3Y<br />
To CPU<br />
GND<br />
74LVC3G07DCT<br />
TSOP65P400X130-8N<br />
2 1<br />
4 3<br />
S2<br />
Reset Push Button<br />
RESET PULSE DURATION APROX. CT(nF)/175<br />
CT = 0.1uF (100nF) PRODUCES 572ms RESET PULSE<br />
Changed<br />
from<br />
Roy<br />
+3.3V_SEQ<br />
R1015<br />
4.7K<br />
RST_PORn 69<br />
To Spartan<br />
+2.5V<br />
R1029<br />
4.7K<br />
Used for<br />
CFPGA-First<br />
scheme<br />
2 1<br />
4 3<br />
S1<br />
CPU-Only reset<br />
button<br />
64<br />
RST_PEX0n_CPU<br />
RST_PEX0n_CPU<br />
R1025<br />
The SYS_RSTn button does not cause the power supplies to power down or the power-up sequence<br />
to be repeated.<br />
68<br />
RST_CFPGA_OUT<br />
DNI<br />
Here is a photo showing the location of the SYS_RSTn button.<br />
3.26.4 User Reset<br />
There is another reset button on the board called the "USER RESET" button. It is located as shown<br />
below.<br />
This button has nothing to do with the power monitors or power on. All it does is assert the<br />
USER_RESETn signal to the FPGAs. It can be used as a general-purpose pushbutton by the user in<br />
the FPGA.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 76
The USER_RESETn signal to the FPGA is also automatically asserted by the configuration FPGA<br />
while it is configuring FPGAs. After an FPGA is configured, the config FPGA will de-assert the<br />
USER_RESETn signal. For this reason it is useful for the purpose of resetting your logic.<br />
3.27 SYSTEM MONITOR<br />
Each Virtex 6 FPGA has an internal block called the "system monitor". The system monitor is<br />
enabled on the DNV6F6PCIE, however none of the recommended power supply filtering for the A-<br />
to-D converter are provided, so there may be unknown amounts of error caused in the A-to-D<br />
because of this. The VP and VN (analog) inputs of the FPGA are connected to a small, 0.1" test<br />
point. You will have to connect with wires to the board in order to use them.<br />
FPGA A<br />
ADC's enabled, but<br />
not high-precision<br />
U0-38<br />
FPGA_VP_A AA22<br />
ADC IN VBAT 1.0V - 2.5V<br />
R10<br />
R947 33R FPGA_VN_A AB21<br />
VP_0<br />
VFS - 2.5V<br />
VBATT_0<br />
AH10<br />
FPGA_VFS_A<br />
R80 DNI<br />
VN_0<br />
AVDD - 2.5V<br />
VFS_0<br />
Y22<br />
FPGA_AVDD_A<br />
R84 100R<br />
12mA<br />
AVDD_0<br />
R1002 0R<br />
+2.5V<br />
Recommended FSR<br />
1.25V<br />
AB22 FPGA_VREFP_A R1001 475R<br />
0V<br />
VREFP_0<br />
AA21<br />
R1010 475R<br />
AC22<br />
TEMPERATURE 100uA<br />
VREFN_0<br />
C1886<br />
C1887<br />
AC21<br />
DXP_0<br />
DIODE<br />
1uF<br />
1uF<br />
DXN_0<br />
+VBATT<br />
+2.5V<br />
You can also read back the voltages on these pins, the FPGA temperature, and the VCCINT and<br />
VCCAUX voltages using IMACT and the JTAG chain.<br />
3.28 LED REFERENCE LIST<br />
location, color, meaning<br />
XC6VLX240T/550T_FF1759<br />
AVSS_0<br />
37 YELLOW 1930 DS1,DS2,DS3,DS4,DS5,DS6,<br />
DS7,DS8,DS9,DS10,DS11,<br />
DS12,DS13,DS14,DS15,DS16,<br />
DS17,DS18,DS19,DS20,DS21,<br />
DS22,DS23,DS24,DS37,DS38,<br />
DS39,DS40,DS41,DS46,DS47,<br />
DS48,DS49,DS50,DS51,DS52,<br />
DS53<br />
24 RED 1927 DS25,DS26,DS27,DS28,DS29,<br />
DS30,DS31,DS32,DS33,DS34,<br />
DS35,DS36,DS42,DS43,DS44,<br />
DS45,DS54,DS67,DS68,DS69,<br />
DS70,DS71,DS74,DS76<br />
7 BLUE 1929 DS55,DS56,DS57,DS58,DS59,<br />
DS60,DS75<br />
8 GREEN 1928 DS61,DS62,DS63,DS64,DS65,<br />
DS66,DS72,DS73<br />
Y21<br />
DNV6F6PCIE User <strong>Manual</strong> Page 77
3.29 TEST POINT REFERENCE LIST<br />
The following is a list of all test points on the board in order of when I thought of them.<br />
Reference Connection<br />
Description<br />
Designator<br />
TP1, TP74, TP10 12V Used to connected 12V rails on board into single<br />
large super 12V rail. You should not use this for<br />
any reason.<br />
P1, P2, P3 Daughtercard VCCO Used to probe I/O voltage of daughtercards<br />
TP2 Daughter reset power Power. Should be 2.5V<br />
TP4 12V_L Power. Should be 12V<br />
TP5 12V_R Power. Should be 12V<br />
TP12 CLK_USER_RIGHT Used to measure frequency of global clock<br />
TP15 CLK_USER_LEFT Used to measure frequency of global clock<br />
TP57 CLK_25 Used to measure frequency of global clock<br />
TP31 CLK_MGT_INTERCON Used to measure frequency of global clock<br />
TP36 +5.0V Power. Should be 5V<br />
TP41 VBATT_EXT Used to insert external backup power for<br />
encryption and real time clock<br />
TP42 +VBATT Power. Should be 2.5V<br />
TP58 +1.0V_C Power. Should be 1.0V<br />
TP53 +1.0V_B Power. Should be 1.0V<br />
TP63 SYS_RSTn Main reset of the board. Low is active<br />
TP54 +1.0V_MGT_AD Do not use this for any reason.<br />
TP65 CLK_G2 Used to measure frequency of global clock<br />
TP66 CLK_G1 Used to measure frequency of global clock<br />
TP68 CLK_G1 Used to measure frequency of global clock<br />
TP69 CLK_G2 Used to measure frequency of global clock<br />
TP70 CLK_G0 Used to measure frequency of global clock<br />
TP71 +0.9V_VTT_M Power. Should be 0.9V<br />
TP76 +1.0_Q Power. Should be 1.0V<br />
TP83 +DIMMA_VDD For probing the I/O voltage of a SODIMM<br />
TP51 +DIMMC_VDD For probing the I/O voltage of a SODIMM<br />
TP6 +DIMMD_VDD For probing the I/O voltage of a SODIMM<br />
TP8 +DIMMF_VDD For probing the I/O voltage of a SODIMM<br />
TP60 +1.0V_A Power. Should be 1.0V<br />
TP13 +1.0V_F Power. Should be 1.0V<br />
TP14 +1.0V_E Power. Should be 1.0V<br />
TP17 +1.0V_D Power. Should be 1.0V<br />
TP16 +2.5V Power. Should be 2.5V<br />
TP75 +3.3V Power. Should be 3.3V<br />
TP77 +1.1V_CPU Power. Should be 1.1V<br />
TP79 +1.8V_CPU Power. Should be 1.8V<br />
TP80 +1.0V_CPU Power. Should be 1.0V<br />
TP61 RST_CPU For probing the CPU reset. Should be the same<br />
as SYS_RST<br />
DNV6F6PCIE User <strong>Manual</strong> Page 78
TP59 CLK_TP_B Connects directly to an FPGA I/O pin<br />
TP62 CLK_TP_C Connects directly to an FPGA I/O pin<br />
TP9 CLK_TP_E Connects directly to an FPGA I/O pin<br />
TP84 DEV_BURST Unknown purpose.<br />
TP87 DEV_READY Unknown purpose.<br />
TP81 ETH_TSTP Unknown purpose.<br />
TP85 SATA_USB_TP Unknown purpose.<br />
TP86 CPU_PEX_TP Unknown purpose.<br />
TP55, TP67, TP7, GND<br />
Used for physically securing SODIMM modules<br />
TP28<br />
TP32, TP33, TP35,<br />
TP64, TP72, TP73,<br />
TP82<br />
GND<br />
Used for physically securing the board.<br />
3.30 Connector Reference List<br />
The following is a list of the connectors on the board. It will allow you to find the datI got bored.<br />
GOMPF_9456-0285LC 2767 BRAK1<br />
MOUNTING_HOLES REMOVE_FROM_BOM M1,M2,M3<br />
STIFFENER_DN200_NORTH 2766<br />
MP1<br />
STIFFENER_DN200_SOUTH 2712 MP2<br />
1<br />
3<br />
1<br />
1<br />
12 TST PNT DNI TP3,TP7,TP28,TP32,TP33,<br />
TP35,TP55,TP64,TP67,TP72,<br />
TP73,TP82<br />
20 TESTPOINT 892 TP6,TP8,TP9,TP11,TP13,<br />
TP14,TP16,TP17,TP51,TP56,<br />
TP58,TP59,TP60,TP61,TP62,<br />
TP75,TP77,TP79,TP80,TP83<br />
25 FBA03HA450AB-00 2720 TP18,TP19,TP20,TP21,TP22,<br />
TP23,TP24,TP25,TP26,TP27,<br />
TP29,TP30,TP34,TP37,TP38,<br />
TP39,TP40,TP43,TP45,TP46,<br />
TP47,TP48,TP49,TP50,TP52<br />
1 SHUNT_JUMPER 2056 TP78<br />
5 TP_TH08 DNI TP81,TP84,TP85,TP86,TP87<br />
3 ACM2012 2602 T1,T2,T3<br />
DNV6F6PCIE User <strong>Manual</strong> Page 79
1 3001 1398 BT<br />
1 PCI_EXPRESS X 4 P4<br />
3 MEG_400_Plug_Stratix3_30 1157 P10,P11,P12<br />
1 Header_1x2 84 JP1<br />
1 1367073 579 J1<br />
5 CONN_SMA 1377 J2,J5,J6,J20,J36<br />
6 TSM-136-01-T-DV 83 J3,J4,J16,J31,J32,J37<br />
2 45558-0002 1827 J8,J7<br />
6 22-27-2031 431 J9,J10,J11,J17,J18,J19<br />
4 CONN_DDR3_SODIMM204 2516 J12,J13,J22,J24<br />
4 67800-5005 2589 J14,J15,J30,J35<br />
2 87832-1420 511 J38,J21<br />
2 USB Type-B 2628 J25,J23<br />
2 53047-0310-3 1801 J26,J29<br />
1 HEADER 5 84 J27<br />
1 USB Type-B 1156 J28<br />
1 10-88-1201/JTAG 20Pin_WITH_SHROUD 994 J33<br />
1 J0G-0059NL/RJ45 2517 J34<br />
1 TENTH_INCHES 2150 J39<br />
3 SEAM-20-03.5-S-08-2-A 2723 J40,J41,J42<br />
1 2-767004-2 1149 J43<br />
3.31 CHARACTARIZATION REPORTS<br />
3.31.1 PCI Express Speed<br />
3.31.2 Ethernet Speed<br />
3.31.3 Interconnect Speed<br />
3.31.4 Rocket IO Speed<br />
3.31.5 DDR3 Speed<br />
3.31.6 SATA Speed<br />
3.31.7 USB Speed<br />
3.31.8 Marvel CPU Speed<br />
Floating point: 1.3 GFLOPS<br />
DNV6F6PCIE User <strong>Manual</strong> Page 80
3.31.9 NMB Speed<br />
3.32 UNUSABLE PINS<br />
text<br />
3.32.1 configuration dedicated<br />
3.32.2 VREF<br />
3.32.3 no connect<br />
3.32.4 dci<br />
this is some text<br />
DNV6F6PCIE User <strong>Manual</strong> Page 81
4 Software<br />
Using the board requires configuring FPGAs, setting board controls such as clock frequency settings,<br />
and transferring data on and off board. For these purposes software has been provided. The best<br />
place to find details about the Emu software and programming API is in the user package here:<br />
D:/Host_software_programs/Emu/Documents/Emu_manual.pdf<br />
4.1 Emu host software<br />
In the user support package, here<br />
D:\\Host_Software\emu\App<br />
There is a program called "EMU". It can be used on Windows or Linux PCs. This program allow<br />
you to control the board.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 82
The window shown above the main window of EMU.<br />
There is also a command-line version of EMU.<br />
4.1.1 Selecting a board<br />
To connect to a board using the EMU program, make sure the board is connected to the computer<br />
either over Ethernet, USB, or PCI Express. If the board is connected to Ethernet, make sure the<br />
network supports DHCP, or else you may not be able to connect to the board.<br />
When using USB on Windows, a USB driver must be installed before the EMU program can detect<br />
the board. The Windows USB driver is located here:<br />
Host_Software\emu\Drivers\win32_usb<br />
You are expected to know how to install a windows driver using device manager.<br />
Linux does not require a USB driver.<br />
When using PCI Express on Windows, a PCI Express driver must be installed before the EMU<br />
program can detect the board. The Windows PCI Express driver is located here:<br />
Host_Software\emu\Drivers\win32_pci<br />
You are expected to know how to install a windows driver using device manager.<br />
When using PCI Express on Linux, a driver is required. The Linux PCI Express driver is located<br />
here:<br />
Host_Software\emu\Drivers\linux86_pci<br />
There is also provided a shell script that will load the kernel module, and create suitable device<br />
nodes on the file system. You must have root privileges to run this shell script.<br />
In Emu, from the Board menu, select Board->Select Board.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 83
A drop-down menu will appear allowing you to select which board you wish to control, and over<br />
which interface. If you have multiple boards connected to the system, you can only control one at a<br />
time using each instance of the Emu program. After you have selected the board, the main window<br />
will update to show a picture of the board you are using.<br />
Note that it may take about a minute from the time a board is powered on until when it becomes<br />
selectable from the EMU window. This is the time it takes the Marvell CPU to boot into Linux.<br />
4.1.2 Configuring FPGAs<br />
To configure and FPGA you can select the option FPGA->ConfigureFPGA from the menu bar. Or<br />
you can click on the photo of the board near one of the FPGA labels and select "configure FPGA"<br />
from the pop-up window. The program will ask for the path to the .bit file that you wish to use.<br />
When the FPGA is done being configured, a blue dot will appear next to any FPGA that has been<br />
configured. The dot will stay blue until you clear the FPGA. Also note that a blue LED will light on<br />
the board itself.<br />
You can clear an FPGA by clicking on it, and selecting "clear" from the pop-up window.<br />
4.1.3 Clocks<br />
There are 6 global clock networks on the board that have user-controllable settings. Each of those<br />
clocks has it's frequency continually monitored and displayed on the main EMU window on the right<br />
side of the board photo.<br />
To change the settings of the clocks, you can click on the text displaying that clock's frequency. A<br />
pop-up window will display options for the clock. Each clock may have different options, and all<br />
options may not be available for all clocks. For example, clocks G0, G1 and G2 can be set to a userspecified<br />
frequency, but USER_L and USER_R can not.<br />
4.1.4 Sending data to and from the FPGA<br />
The EMU program can also be used to transfer data to and from the FPGAs. The name of the<br />
interface on the FPGA that can be accessed from EMU is called "NMB". The "NMB" interface can<br />
be thought of as an address space. The EMU program can read and write to addresses on that space.<br />
Select either NMB read or NMB write from the "Data" menu.<br />
In order to transfer data to your FPGA, your design must implement an NMB endpoint. The code<br />
necessary to implement an NMB endpoint is provided on the user package at<br />
D:\\FPGA_Reference_designs\common\nmb<br />
DNV6F6PCIE User <strong>Manual</strong> Page 84
The "main ref" reference design provided here<br />
D:\\FPGA_Reference_designs\Fpga_programming_files\user_fpga\main_ref\<br />
Correctly implements an NMB endpoint. You can load these test files into one or more FPGAs and<br />
use the EMU's "nmb read" and "nmb write" functions to send data to the reference design.<br />
4.1.5 Hardware Tests<br />
To detect hardware failures, the EMU program is capable of testing the board. If you want to run a<br />
complete hardware test of the board, select the board in emu. From the Test Menu, select "Selected<br />
Tests". This window will appear.<br />
All of the check boxes are tests on the board that can be run independently. The items in the "Factory<br />
Tests" area all require test fixture hardware to pass, so they will be of limited use to users for testing<br />
the board. The tests in the "field tests" area can all pass without special test fixtures. Note that the<br />
DRAM Test requires that there is a DDR3 SODIMM (any density) installed in all 4 SODIMM slots<br />
on the board. If they are not installed, the test will fail.<br />
Before running, the test may ask for the path to the ".bit" files used to program the FPGAs. A<br />
directory with an appropriate structure is provided on the user package in this location:<br />
D:\\FPGA_Reference_Designs\Programming_Files<br />
DNV6F6PCIE User <strong>Manual</strong> Page 85
After the test(s) complete, the program will print out a message like this:<br />
ONESHOT: rocketio_field_test PASSED<br />
###################################<br />
ONESHOT: CUSTOM TEST COMPLETE.<br />
rocketio_field_test : PASS<br />
***********************************<br />
CUSTOM TEST (REP 1/1) PASSED<br />
DN0200_DNV6F6PCIE 1003019<br />
18:18:02 5/26/2010<br />
If the test stops or fails, you may need to hit the "q" key to regain control of the EMU program.<br />
4.1.6 Command Line version<br />
The EMU program compiles into two versions. The GUI version and the command-line version.<br />
Both versions can run in Windows or in Linux.<br />
The menu options in the command-line version and in the GUI version are identical. The commandline<br />
version lends itself well to scripting.<br />
4.1.7 Scripting with EMU<br />
The command-line version of EMU uses stdin and stdout for input and output, and so it is possible to<br />
write scripts that interact with it. In this way you potentially can use the board without ever having to<br />
write any software of your own.<br />
If you the program with the -c switch, then the menu system is collapsed into one single monolithic<br />
menu, and there is less output produced on stdout. With this switch, scripting is much easier.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 86
In the user package, there is an example script that interfaces well with the command-line version of<br />
EMU.<br />
4.1.8 Emu on the Marvell Linux environment<br />
The command-line version of EMU is also installed on the Linux system that is installed on the<br />
Marvell CPU. This allows EMU commands to be issued to the board using the RS232 terminal or<br />
over a telnet session to the board.<br />
4.2 Writing your own software.<br />
To write your own software it is recommended that you start by attempting to compile the existing<br />
gui or command-line version on Emu. There is a Emu software manual that describes the process in<br />
the support package.<br />
D:/Host_software_applications/emu/<br />
4.3 How EmuLib works<br />
The Emu program and EmuLib communicate to the board through a tall stack of software linking the<br />
host PC to the Marvell processor, the Marvell processor to the FPGA I/Os, and the FPGA I/Os to<br />
your HDL.<br />
However you are not expected, encouraged or allowed to understand any of the workings of this<br />
stack of software.<br />
On the host PC side, you are expected to understand and use the interface provided by the file<br />
diniapi.h, found in the user package. On the FPGA side, you are expected to understand and use the<br />
interface provided by the file nmb_target_interface.v<br />
The layers of software and hardware in between should operate transparently. For no particular<br />
reason details are given here:<br />
DNV6F6PCIE User <strong>Manual</strong> Page 87
1) Your C++ code calls nmb_write() function in the diniapi.h interface.<br />
2) The emulib library determines which of the three interfaces the board is connected on. Let's<br />
assume ethernet<br />
3) The emulib library creates a packet of data with a header and the data you supplied.<br />
4) Emulib sends the packet to the ip address of the board.<br />
5) On the board, a program called DiniCmos is listening to that very same IP address. It takes the<br />
data in the packet and drops it in the DRAM of the marvel.<br />
6) Over PCI Express the DiniCmos program sets some registers in the DMA controller in the<br />
configure FPGA.<br />
4.4 Marvel Environment<br />
The Marvell CPU is running a complete Linux operating system. Most standard Linux applications<br />
and utilities are already installed. You are able to program the Marvell processor with your own code<br />
so that the board can operate as a stand-alone device, without the need for a host computer in<br />
production environments.<br />
4.4.1 Linux Provided<br />
The Linux kernel on the board is version 2.6.22.18 as of this writing.<br />
There is no particular name for the "distribution" on the board, however many of the common Linux<br />
utilities are provided by busybox.<br />
4.4.2 Operating from the shell terminal<br />
You can get a linux shell terminal by using telnet to access the board. The board will register its host<br />
name with the dhcp server, if the dhcp server supports dns. The host name of the board will be in the<br />
form<br />
dnv6f6pcie-xxxxxxx<br />
where xxxxxxx is the 7-digit serial number of the board. The serial number can be found on the<br />
serial number sticker under the DIMM D memory socket.<br />
You can also access a terminal using the RS232 connector located near the lower right corner of the<br />
board, labeled "Marvell Serial". This connector is a standard computer "serial" port. 2 x 5 connector<br />
can be used with the provided IDC-to-DB9 adapter cable. The terminal settings are<br />
Baud: 19200<br />
Data Bits: 8<br />
Parity: None<br />
Stop Bits: 1<br />
Flow Control: OFF<br />
Emulation: VT200<br />
The RS232 output is also the system console, so you will also see system messages on your terminal<br />
in addition to the shell output. If this bothers you, you need to use telent.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 88
4.4.3 Running EMU in the terminal<br />
You can run EMU from the linux terminal. From here you can configure FPGAs, set clocks, and<br />
send data to and from FPGAs. The command is "emu_mv". Using the program is identical to using<br />
the command-line version on the EMU program on the host. You should connect to board by ip<br />
address at address 127.0.0.1<br />
If you really want, you can also control other boards from this board over Ethernet.<br />
4.4.4 Compiling code on the Marvell<br />
The compiler GCC and standard C headers and libraries are installed on the board. You can compile<br />
standard C and C++ programs that run in user space.<br />
4.4.5 Kernel Space<br />
If you want to run code on the Marvell Processor in kernel space, the complete kernel code is<br />
required. The kernel code is not installed on the board, so compiling kernel mode code for the board<br />
cannot be done on the board. For this, you will need a build environment. Likewise, to modify the<br />
kernel itself also requires a separate build environment. We can provide a VMWare virtual machine<br />
with a cross-compiler installed that is capable of building the kernel and kernel modules.<br />
When modifying the kernel, you should maintain your code as diff files, because if and when Dini<br />
Group modifies the kernel, we will not provide you with a change list, and you will have to re-port<br />
your changes to the new kernel source. Alternately, you can develop the kernel changes you require<br />
and provide the change list to Dini Group, and we will mainline your changes.<br />
Note that you may not need to make kernel modifications at all. We have provided a device located<br />
at<br />
/dev/mem<br />
That gives user-mode programs read/write access to the CPU0 memory space. If the only thing you<br />
need to do is access protected kernel memory, we recommend you just use this method.<br />
4.4.6 Boot Sequence<br />
When the board powers on, the CPU executes in place address 0xF60000 of the SPI device. In this<br />
case, the SPI device is an ATMEL data flash. The data flash contains code that initializes DRAM<br />
and loads the last 512KB of memory from the data flash into DRAM. The last 512KB of flash<br />
contains a u-boot bootloader image.<br />
When u-boot runs, it reads a set of environment variables off the flash. It runs the u-boot command<br />
defined by the "bootcmd" variable. That command will copy the entire contents of the data flash<br />
image from address 0x0 of the data flash to address 0x2000000 of DRAM. At address 0x0 in the data<br />
flash is a compressed linux binary image. u-boot uncompresses the image and jumps to it. U-boot<br />
can pass a single string of information to the linux image, called the "boot arguments". The value of<br />
these boot arguments can be modified using u-boot environment variables.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 89
Linux boots, setting up all the devices, including NAND flash. The NAND flash shows up as four<br />
MTD block devices, representing four different regions in the NAND flash. As the last step of<br />
booting, it mounts a JFFS2 file system from one device, specified in the boot arguments. Before<br />
starting a shell for the user, linux will run a shell script on the filesystem called startup.sh. This script<br />
loads the software that Dini Group wrote that controls the FPGAs and other function of the board.<br />
This program is called "DiniCmos" and runs in the background. You don't need to worry about it.<br />
4.4.7 Compiling U-boot<br />
Like the kernel, compiling u-boot requires the virtual machine. It is not recommended that you do<br />
this. Instead, submit your change request to Dini Group so that we can mainline your changes.<br />
4.4.8 Creating the Root File system<br />
There is no root file system build process. Instead we maintain a golden image in the form of a .tar<br />
file containing the contents of the root file system on the Marvell. Changes have to be made to the<br />
.tar file manually. Updating utilities are done manually.<br />
4.4.9 Update the software<br />
Updating the software on the Marvell board takes a long time.<br />
4.4.9.1 Installing a kernel update<br />
To get the your current version of the Linux kernel, you can check the boot messages for this line:<br />
## Booting image at 02000000 ...<br />
Image Name: Linux-2.6.22.18<br />
Created: 2010-04-02 1:27:51 UTC<br />
Image Type: ARM Linux Kernel Image (uncompressed)<br />
Data Size: 2840632 Bytes = 2.7 MB<br />
Load Address: 00008000<br />
Entry Point: 00008000<br />
Verifying Checksum ... OK<br />
Check the "Created" date.<br />
1) Connect the serial terminal to the board.<br />
2) Power on the board. You should see u-boot boot messages in the terminal. At some point u-boot<br />
should print<br />
Hit any key to stop autoboot: 3<br />
At this step, press any key. You will then recieve a u-boot prompt like this<br />
>><br />
3) Type this u-boot command<br />
DNV6F6PCIE User <strong>Manual</strong> Page 90
protect off 1:0-63<br />
U-boot will print this:<br />
Un-Protect Flash Sectors 0-63 in Bank # 1<br />
................................................................<br />
done<br />
4) boot into linux by typing this u-boot command<br />
boot<br />
5) Once linux is done booting you will receive a command prompt like this<br />
-sh#<br />
6) Type the following command<br />
mount -t tmpfs tmpfs /mnt/ram -o size=32M<br />
7) Type the following command<br />
cd /mnt/ram<br />
8) Get the update files from the dini group website. Put these files on the board. If you have the<br />
board connected to an internet-enabled network, you can use the wget command.<br />
wget http://dinigroup.com/~marvellfiles/uImage<br />
If you do not have access to the internet, then you will need to use some other method to transfer<br />
files to the board. You can use a USB key, a network mount, or any other linux trick you know.<br />
9) Type this command<br />
cat uImage > /dev/partition_spi<br />
This command updates the Linux kernel. If this command fails, the recovery procedure is still<br />
possible, but is more complicated.<br />
10) Type this command<br />
reboot<br />
It is important that you use the reboot command, and do not simply power-cycle the board. If you<br />
power cycle the board, then the SPI flash may not get written completely due to write buffering.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 91
4.4.9.2 Installing a RFS update<br />
This procedure will result in the loss of user data on the Linux file system. Back up your data. To<br />
check the version number of your root file system, you can type this Linux command:<br />
cat /root/image.date<br />
1) Connect the serial terminal to the board.<br />
2) Power on the board. You should see u-boot boot messages in the terminal. At some point u-boot<br />
should print<br />
Hit any key to stop autoboot: 3<br />
At this step, press any key. You will then receive a u-boot prompt like this<br />
>><br />
3) At the prompt, type this command<br />
run 'spi_boot_recoveryfs'<br />
5) Once Linux is done booting you will receive a command prompt like this<br />
-sh#<br />
6) At the command prompt type this Linux command<br />
sh /root/recover.sh<br />
7) The recovery process takes about 10 minutes, plus longer for the 180MB download from<br />
dinigroup.com. Wait patiently.<br />
8) When the recovery script is complete. Type this at the shell prompt<br />
reboot<br />
4.4.9.3 Installing a U-boot update<br />
A failure during this process will cause the board to be un-recoverable. Please consult Dini Group<br />
before starting this process. There is normally no reason for the user to use this procedure. To check<br />
your current version of u-boot, you can check the boot messages for this line<br />
The compile date of mv_main.c is May 14 2010<br />
1) Connect the serial terminal to the board.<br />
2) Power on the board. You should see u-boot boot messages in the terminal. At some point u-boot<br />
should print<br />
Hit any key to stop autoboot: 3<br />
DNV6F6PCIE User <strong>Manual</strong> Page 92
At this step, press any key. You will then receive a u-boot prompt like this<br />
>><br />
3) Type this u-boot command<br />
protect off 1:0-63<br />
U-boot will print this:<br />
Un-Protect Flash Sectors 0-63 in Bank # 1<br />
................................................................<br />
done<br />
4) boot into Linux by typing this u-boot command<br />
boot<br />
5) Once Linux is done booting you will receive a command prompt like this<br />
-sh#<br />
6) Type the following command<br />
mount -t tmpfs tmpfs /mnt/ram -o size=32M<br />
7) Type the following command<br />
cd /mnt/ram<br />
8) Get the update files from the Dini group website. Put these files on the board. If you have the<br />
board connected to an internet-enabled network, you can use the wget command.<br />
wget http://dinigroup.com/~marvellfiles/u-boot-db78200_MP.bin<br />
wget http://dinigroup.com/~marvellfiles/update_uboot<br />
wget http://dinigroup.com/~marvellfiles/uImage<br />
If you do not have access to the internet, then you will need to use some other method to transfer<br />
files to the board. You can use a USB key, a network mount, or any other Linux trick you know.<br />
9) Type this command<br />
chmod 500 update_uboot<br />
10) Type this command<br />
./update_uboot<br />
DNV6F6PCIE User <strong>Manual</strong> Page 93
11) Type this command<br />
diff new_contents.bin u-boot-db78200_MP.bin<br />
The file new_contents.bin contains the current contents of the SPI flash. This command is to make<br />
sure that the update was written successfully to the SPI flash. If this diff fails (shows the files are<br />
different) then something went wrong with the update, and you should not turn off you board. If you<br />
turn off the board at this time, the board will never boot again, and you will have to send it back to<br />
the factory for re-programming.<br />
12) Was the diff clean? If not then stop here!!<br />
13) Type this command<br />
cat uImage > /dev/partition_spi<br />
This command updates the Linux kernel. If this command fails, the recovery procedure is still<br />
possible, but is more complicated.<br />
12) Type this command<br />
reboot<br />
It is important that you use the reboot command, and do not simply power-cycle the board. If you<br />
power cycle the board, then the SPI flash may not get written completely due to write buffering.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 94
5 REFERENCE DESIGN<br />
text - what is the reference design, why does it exist?<br />
text<br />
5.1 Diagram<br />
5.2 NMB Space Map<br />
text<br />
5.3 Things Tested<br />
list of stuff?<br />
5.4 Compiling<br />
this is text<br />
DNV6F6PCIE User <strong>Manual</strong> Page 95
6 TROUBLESHOOTING<br />
Text<br />
6.1 board is dead<br />
text<br />
6.2 does not respond over PCI Express<br />
test<br />
6.3 My design acts weird<br />
6.4 It works on one FPGA and not others<br />
6.5 Pacemaker stops working<br />
6.6 Signals looks crazy on scope<br />
6.7 DCM does not lock<br />
text<br />
6.8 design doesn't respond over NMB<br />
text<br />
6.9 fpgas will not program<br />
text<br />
6.10 does not boot<br />
text<br />
DNV6F6PCIE User <strong>Manual</strong> Page 96
7 ORDERING<br />
text<br />
INFORMATION<br />
7.1 Part Number<br />
text<br />
7.2 How to Order<br />
text<br />
7.2.1 International Offices<br />
China<br />
Rm505, R&D. Complex, Qing Hua Xin Xi Gang Nan Shan S&T Industrial Park, Shenzhen, China<br />
TEL: +86-755-8618-6718<br />
FAX: +86-755-8618-6700<br />
Email : sales@e-elements.com<br />
TEL: +86-10-82757632<br />
FAX: +86-10-82756745<br />
Europe<br />
Name<br />
Address<br />
Phone<br />
Email<br />
India<br />
Intrinsic Solutions<br />
#2145, 17th Main, 2nd Cross<br />
HAL 2nd Stage, Indiranagar<br />
Bangalore 560008<br />
TEL:+91-80-4115-1400<br />
FAX:+91-80-4115-0797<br />
E-mail sales@intrinsic.in<br />
http://www.intrinsic.in<br />
Indonesia<br />
DNV6F6PCIE User <strong>Manual</strong> Page 97
Advinno Technologies<br />
http://www.advinno.com/<br />
Israel<br />
ISROTECH, The FPGA and SoC House<br />
Office: +972-(0_72-2342181<br />
Mobile: +972-(0)54-4-805-791<br />
Fax: +972-1538-9285271<br />
Email: info@isrotech.com<br />
Web: www.isrotech.com<br />
Japan<br />
Applistar Corporation<br />
3050 Okada, Atsugi AXT Main Tower 3F,<br />
Atsugi, 243-0021, Japan<br />
TEL:+81-46-227-3288<br />
FAX:+81-46-220-2901<br />
http://www.applistar.com<br />
Korea<br />
MDS Technology<br />
+82-2-2106-6000<br />
15Fl. Kolon Bilant 222-7 Guro3-dong, Seoul Korea 152-777<br />
http://www.mdstec.com<br />
Singapore<br />
E-Elements Technology Pte Ltd<br />
21 Science Park Road #03-16A, The Aquarius<br />
Singapore Science Park II Singapore 117628<br />
TEL: +(65)6777-2240<br />
FAX: +(65)6777-2249<br />
Email: sales@e-elements.com<br />
Svalbard<br />
Knytte et tau rundt en elg. Knytte den andre enden rundt din hals. Skremme elgen.<br />
Taiwan<br />
E-Elements Technology Co.<br />
6F-3, No.160, Sec. 6, MinChuan E. Rd., Taipei, Taiwan R.O.C.<br />
TEL: +886-2-2791-8139<br />
FAX: +886-2-2792-6942<br />
Email : sales@e-elements.com<br />
http://www.e-elements.com/<br />
DNV6F6PCIE User <strong>Manual</strong> Page 98
7.3 Board Options<br />
text<br />
7.4 Compatible Products<br />
text<br />
7.4.1 Dini Group Hardware<br />
7.4.2 Third Party Hardware<br />
SFP module<br />
DIMM<br />
7.5 Warranty<br />
7.6 Compliance Information<br />
text<br />
7.6.1 FCC<br />
We are willing to obtain an FCC certification for volume production.<br />
7.6.2 CE Mark<br />
7.6.3 Junior Miss Canada<br />
My daughter was 3rd place 2003.<br />
7.6.4 UL<br />
It might pass as a paperwieght.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 99
7.6.5 ROHS<br />
We can obtain a certification if necessary for volume production.<br />
7.6.6 Participation Awards<br />
I just got my 3 month chip.<br />
7.6.7 PCI-SIG<br />
The board passes our internal PCI Express compliance test. If you need the board added to the PCI-<br />
SIG integrator's list for some reason I can't fathom. This can be done.<br />
7.6.8 Export Control<br />
7.6.9 ISO 9001:2000<br />
DNV6F6PCIE User <strong>Manual</strong> Page 100
8 Index<br />
DNV6F6PCIE User <strong>Manual</strong> Page 101
9 Glossary<br />
You can expect this manual, marketing materials, diagrams, emails and source code from Dini Group<br />
to contain a lot of shorthand and imprecise phraseology. Instead of ensuring that we're consistent and<br />
precise, I've made a list of terms and words that we made up below:<br />
Word that exists<br />
Thing that word means<br />
.bit file<br />
Load file<br />
Configuration file<br />
Bitstream<br />
BUFG<br />
BUFH<br />
BUFIO<br />
BUFR<br />
DCM<br />
IDDR<br />
IDELAY<br />
ISERDES<br />
MMCM<br />
ODDR<br />
ODELAY<br />
OSERDES<br />
Core<br />
IP<br />
Verilog<br />
VHDL<br />
Design<br />
DCI<br />
DIFF_TERM<br />
DCM<br />
DLL<br />
PLL<br />
MMCM<br />
All of these words interchangeably refer to the file containing the data for<br />
the SRAM configuration bits for the FPGA. This file is generated by the<br />
Xilinx "bitgen" program.<br />
Each of these are primitive HDL modules available in the Virtex 6 FPGA.<br />
It is assumed that the reader is familiar with the behavior and use of any<br />
primitives mentioned in each section. Please see the Virtex 6 user guide<br />
for the behavior of these modules.<br />
These words refer interchangeably to HDL code.<br />
Digitally controlled impedance. It refers to the ability of the FPGA to<br />
output optimally-terminated signals.<br />
Attribute that can be set on differential inputs to differentially endterminate<br />
them.<br />
Digital clock manager<br />
Delay-locked loop<br />
Phase-locked loop<br />
something something clock manager<br />
DDR<br />
DRIVE attribute<br />
Drive strength<br />
FPGA Q<br />
Config FPGA<br />
NMB Bridge<br />
These words are (incorrectly) used interchangeably in this guide.<br />
Usually means "double data rate", meaning that a signal's value is<br />
significant on both rising and falling edges of a clock. Sometimes this<br />
guide also uses "DDR" to refer to "DRAM", just to add confusion.<br />
This refers to the ability of the FPGA to produce a variety of output<br />
impedaces on its I/O.<br />
There words are all used interchangeably to refer to the smaller, Virtex 5<br />
FPGA on the DNV6F6.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 102
U0<br />
Spartan<br />
V5<br />
GND<br />
Ground<br />
IOSTANDARD attribute<br />
ISE<br />
Bitgen<br />
iMpack<br />
XST<br />
CoreGen<br />
MIG<br />
LOC constraint<br />
MB<br />
KB<br />
Mbs<br />
Mb/s<br />
MTs<br />
MT/s<br />
MEG-Array<br />
Daughtercard Connector<br />
MHz<br />
Mux<br />
MV78200<br />
CPU<br />
MCU<br />
Processor<br />
uP<br />
Net<br />
Signal<br />
Rail<br />
Plane<br />
NMB<br />
Main Bus<br />
Bus<br />
PCIE<br />
RGMII<br />
SGMII<br />
RocketIO<br />
GTP<br />
GTX<br />
MGT<br />
RS232<br />
This refers to both a net on the DNV6F6, and to the absolute potential of<br />
that net at any given time. When an absolute voltage is given in this<br />
manual, it should be interpreted as a voltage relative to this net.<br />
This is an attribute of the I/O buffer primitive.<br />
These are each software programs provided by Xilinx<br />
This is a constraint type that controls which physical pin on the FPGA<br />
device an I/O will be mapped to.<br />
Megabyte. Usually 1,000,000 bytes, but sometimes 1,048,576 bytes.<br />
Kilobyte. Usually 1,000 bytes, but sometimes 1,024 bytes.<br />
Mega(b)its divided by seconds.<br />
Mega(T)ransfers per second.<br />
These are used interchangeably with MHz, to avoid any possible confusion<br />
when dealing with DDR interfaces<br />
There are several types of user-available connectors on this board. Usually,<br />
"the daughtercard connector" refers to one of the three "Meg-Array"<br />
connectors attached to FPGA D, E and F.<br />
1,000,000 divided by seconds.<br />
Multiplexer<br />
The on-board processor manufactured by Marvell.<br />
These interchangeably refer to a physical wire on the printed circuit board.<br />
A net on the circuit board is all points that are electrically shorted together.<br />
These interchangeably refer to the 40-signal bus between each FPGA and<br />
the configuration FPGA.<br />
This word can mean any one of the following:<br />
An interface that is more than one signal wide.<br />
An interface that has more than two endpoints.<br />
An interface.<br />
PCI Express<br />
This is an interface specification used for Ethernet physical interface<br />
devices.<br />
In this guide, these all refer interchangeably to the 6Gbs, high-speed serial<br />
transceivers available on Virtex 6 devices.<br />
This could refer to one of the following:<br />
DNV6F6PCIE User <strong>Manual</strong> Page 103
Serial Port<br />
SATA<br />
SEAM<br />
Serial Daughtercard<br />
SFP<br />
SMA<br />
SODIMM<br />
DDR3<br />
SDRAM<br />
DIMM<br />
Source Synchronous<br />
SPI<br />
SSTL<br />
LVDS<br />
LVCMOS<br />
HCSL<br />
HSTL<br />
CML<br />
LVTTL<br />
LVDS_EXT<br />
Countervail<br />
System Synchronous<br />
UCF<br />
CC<br />
GC<br />
MRCC<br />
SRCC<br />
VREF<br />
VRN<br />
VRP<br />
XAPP<br />
UG<br />
The serial port on a computer<br />
The connector on the DNV6F6 that is intended to connecting to a<br />
computer's serial port.<br />
Serial ATA<br />
These refer to three connectors attached to FPGAs A,C and D. These<br />
connectors carry high-speed serial signals from the Virtex-6 FPGAs.<br />
SEAM refers to the name of the Samtec brand connector series that were<br />
selected for this board.<br />
Small form-factor pluggable module. This refers to the socket attached to<br />
FGPA F that is intended for Gigabit Ethernet.<br />
These are little gold-colored screw-in coax connectors that are on the board<br />
in various places<br />
These all refer interchangeably to the four SODIMM connectors attached<br />
to FPGA A, C, D and F. These connectors are intended for memory<br />
modules to attach.<br />
This refers to a clocking strategy that involves introducing a controlled<br />
amount of clock skew to devices with respect to each other. Precisely what<br />
I mean each time I type it varies.<br />
I don't know what is stands for. Serial something interface maybe?? It is<br />
the protocol used for the serial Flash on this board.<br />
These are all signaling standards. They each require certain voltage levels<br />
and termination schemes.<br />
To counteract<br />
A clocking strategy where each device receives a low-skew clock. In this<br />
guide, I often use this word as shorthand for "using one of the provided<br />
global clock networks"<br />
I used this as shorthand for "the set of place-and-route constraints that you<br />
input into ISE".<br />
These are all special-purpose I/O pins on the Virtex 6 FPGA. For each<br />
section that mentions one of these pins, you are expected to know the<br />
purpose and usage of each.<br />
These refer to reference documents produced by Xilinx.<br />
DNV6F6PCIE User <strong>Manual</strong> Page 104
10REVISION HISTORY<br />
RCS file:<br />
/var/lib/cvs/dncvs/Boards/DN0200_DNV6F6PCIE/Documents/<strong>Manual</strong>/DNV6F6PCIE_ma<br />
nual_rev01.docx,v<br />
Working file: DNV6F6PCIE_manual_rev01.docx<br />
head: 1.2<br />
branch:<br />
locks: strict<br />
access list:<br />
symbolic names:<br />
keyword substitution: bx<br />
total revisions: 2; selected revisions: 2<br />
description:<br />
----------------------------<br />
revision 1.2<br />
date: 2010/05/28 23:44:59; author: dpalmer; state: Exp; lines: +100916<br />
-18154; kopt: bx; commitid: ab84c0055796eb8; filename:<br />
DNV6F6PCIE_manual_rev01.docx;<br />
took out the most offensive jokes, like the race jokes. I'm about half way<br />
through the "hardware" section.<br />
----------------------------<br />
revision 1.1<br />
date: 2010/02/19 03:20:26; author: dpalmer; state: Exp; kopt: bx;<br />
commitid: 5ab44b7e0379ab40; filename: DNV6F6PCIE_manual_rev01.docx;<br />
*** empty log message ***<br />
==========================================================================<br />
===<br />
DNV6F6PCIE User <strong>Manual</strong> Page 105