Retinal Prosthesis Dissertation - Student Home Pages

An Address Event Representation (AER)
Multicast Router for Colour Vision
A thesis submitted in partial fulfilment of the requirements
for the degree of Doctor of Philosophy
Alex Jameson
School of Electrical, Electronic and Computer Engineering,
Faculty of Science, Agriculture and Engineering,
University of Newcastle upon Tyne, NE1 7RU
July 2012

Acknowledgements
I am very grateful to my supervisors: Dr. Graeme Chester and Professor Alex
Yakovlev. Lacking the continual help and support of my primary supervisor; Dr
Chester, this report would not have been finished to such a high standard, whilst
Professor Yakovlev has also offered moral support to a high standard.
Dedication
Dedicated to my immediate family, past and present.
Statement of Copyright
The copyright of this report rests with the author and/or any one of his supervisory
team. Quotations from it may be published without the authors’ prior written consent
subject to the consent of any one of his supervisory team but information from it
should be acknowledged.
i of ix

Abstract\Foreword
This document proposes a prosthetic vision application utilizing an AER [1-4]
(Address Event Representation) Multicast router. It is novel in the sense of dealing
with `colour’ signal propagation from a ‘scene’ of pixels in the form suggested by
the trichromacy theory of: Red (R), Green (G) and Blue (B) signals flowing into the
optic nerve [5-15]. The context is within the transmission of the information between
a sender chip and a receiver chip. Power hungry processing will be done within the
sender chip and the multicast router at the receiver chip will route the AER
transmissions via driver circuitry to the relevant locations of the retinal implant.
The fundamental concept is to input a `scene’ into a sender chip and the visual
information from that scene will then be propagated by transmitting the events of
that scene using an Address Representation Format (AER) from the sender chip via
transmission lines to a receiver chip. The receiver chip will be capable of receiving
the event data and recreating the original scene via implant driver circuitry from the
information decoded from the received data. The test setup for this described
combination is shown in Figure 0.
AER Transmission
lines
INPUT
image
signal
SENDER
(chip)
RECEIVER
(chip)
Display
monitor
Figure 0 Test setup for AER propagation
ii of ix

The main advantage of AER in the context of aiding artificial vision for humans is
that events occurring in the scene input to the receiver chip are represented and
propagated as impulses or spikes which is the method used by the fibres of the optic
nerve. The maximum spike rate represents maximum hue intensity when referring to
the RGB true colour format. Note that propagation of signals in the optic nerve is
very energy efficient and it can be expected that the neuromorphic AER
communication protocol will be similar.
iii of ix

Contents
An Address Event Representation (AER) Multicast Router for Colour Vision .................................. i
Acknowledgements ................................................................................................................................ i
Dedication ............................................................................................................................................... i
Abstract\Foreword ................................................................................................................................. ii
Chapter 1 Introduction/research aim ................................................................................................. 10
1.1 Introductory Literature Review ............................................................................................... 10
1.1.1 Sub retinal ..................................................................................................................... 11
1.1.2 Epi retinal ............................................................................................................................. 14
1.2 Conceptual overview ............................................................................................................. 18
1.3 Sub retinal versus Epi-retinal ................................................................................................. 21
1.4 Introduction to stimulators ........................................................................................................... 21
1.5 Thesis structure ..................................................................................................................... 22
Chapter 2 Early directions .................................................................................................................. 24
2.1 Introduction to initially envisaged project ..................................................................................... 24
2.2 Neural network representation .................................................................................................... 25
2.2.1 The first artificial neuron ........................................................................................................... 26
2.2.2 Hebb’s rule ........................................................................................................................... 28
2.2.3 Adaline and the adaptive linear combiner: a bipolar case of above ..................................... 28
2.2.4 Neural Networks –perceptron training .................................................................................. 30
2.2.5 Linear separability ................................................................................................................ 32
2.2.6 The Delta Rule (used to determine separability) .................................................................. 35
2.3 Address Event Representation (AER) ......................................................................................... 37
2.3.1 The message packet ............................................................................................................ 37
2.4 Comparison to Frame Based Representation (FBR) ................................................................... 38
2.4.1 Comparison Criteria ............................................................................................................. 40
2.4.2 Address Event Representation ............................................................................................ 41
2.5 MATLAB sub-retinal simulation ................................................................................................... 42
2.5.1 MATLAB representation ....................................................................................................... 42
2.5.2 Program operation ............................................................................................................... 43
2.5.1 Files in Appendix A ................................................................................................................... 44
2.5.2 Use of the program ................................................................................................................... 44
2.6 Epi-retinal; method of choice ....................................................................................................... 45
2.6.1 MATLAB representation of epi_retinal approach.................................................................. 47
2.6.2 FPGA representation of epi_retinal approach .......................................................................... 48
2.7 Background to stimulators ........................................................................................................... 49
2.7.1 Charge considerations ......................................................................................................... 49
2.7.2 Achromatic current requirements ......................................................................................... 49
Chapter 3 Concepts ............................................................................................................................. 50
3.1 Neural concepts .......................................................................................................................... 52
3.2 Biological Neuron operation ........................................................................................................ 53
3.3 Natural limit for pulse duration ..................................................................................................... 54
3.4 Retinal colour perception ............................................................................................................. 55
3.5 Human visual processing ............................................................................................................ 57
3.6 Current Retinal Implants .............................................................................................................. 61
3.6.1 Retinal structure ................................................................................................................... 61
3.6.2 Retinal operations ................................................................................................................ 62
3.6.3 Commonality of sub retinal and epi retinal approaches ........................................................ 65
iv of ix

3.6 AER concept in context ............................................................................................................... 66
3.7 AER data format .......................................................................................................................... 67
3.8 Behavioural test image to be used .............................................................................................. 71
3.8.1 Other test images ................................................................................................................. 74
3.9 Biphasic pulse ............................................................................................................................. 74
3.10 Concluding this chapter ............................................................................................................. 76
Chapter 4 Sender chip ......................................................................................................................... 79
4.1 Sender concepts ......................................................................................................................... 80
4.2 Sender AER format ..................................................................................................................... 85
4.2.1 Alternative (discounted) sender AER format ........................................................................ 86
4.3 Test setup ................................................................................................................................... 87
4.4 Sender chip reports ..................................................................................................................... 87
4.4.1 Power Analysis .................................................................................................................... 88
4.5 Sender schematic ........................................................................................................................ 88
4.6 Chapter summary ........................................................................................................................ 91
Chapter 5 Receiver chip ...................................................................................................................... 94
5.1 Clocking calculations (sender) ..................................................................................................... 94
5.2 Power Analysis ....................................................................................................................... 96
5.3 Receiver schematics ................................................................................................................... 97
5.4 Clocking calculations (receiver) ................................................................................................... 99
5.5 Programming components descriptions ...................................................................................... 99
5.5.1 Incoming (short) AER stream ............................................................................................. 100
5.5.2 Production of colour data streams ..................................................................................... 100
5.5.3 Convert to long format........................................................................................................ 100
5.5.4 Production of output streams ............................................................................................. 100
5.6 Summary of FPGA resource utilisation...................................................................................... 101
5.7 Post processing for electrodes .................................................................................................. 101
5.8 Stimulator Retinal Interface ....................................................................................................... 103
5.8.1 Factors affecting current requirement for stimulation ......................................................... 103
5.8.2 Electrode size and positioning in current retinal approaches ............................................. 104
5.8.3 Recent developments ........................................................................................................ 104
Chapter 6 Concluding chapter.......................................................................................................... 105
6.1 Conclusions and results ............................................................................................................ 105
6.2 Implementation of design .......................................................................................................... 106
6.2.1 DAC operation ................................................................................................................... 107
6.2.2 Envisaged retinal array ...................................................................................................... 110
6.2.3 Connecting the detailed engineering to the overall concept ............................................... 111
6.2.4 Differences between proposed technique and current retinal implants .............................. 112
6.3 Lower power FPGA ................................................................................................................... 113
6.4 AER between sender and receiver chip .................................................................................... 115
6.5 Post Processing Interface ......................................................................................................... 116
6.6 Pre processing .......................................................................................................................... 117
6.7 Future work ............................................................................................................................... 117
6.7.1 Surgical implantation .......................................................................................................... 118
6.7.2 Inductive linking considerations ......................................................................................... 119
6.7.3 Initial configuration ............................................................................................................. 119
Bibliography ....................................................................................................................................... 120
%Appendix A MATLAB ..................................................................................................................... 133
--Appendix B FPGA Test setup......................................................................................................... 142
--Appendix C FPGA Sender chip ...................................................................................................... 150
--Appendix D FPGA Receiver chip ................................................................................................... 154
v of ix

--Appendix E FPGA Outputs ............................................................................................................. 171
--Appendix F Alternative literature review ....................................................................................... 189
vi of ix

List of figures
Figure 1 Layer structure ......................................................................................................................... 13
Figure 2 Eye diagram (image: Wikimedia Commons public domain) ..................................................... 14
Figure 3 Mapping of Image Pixels to Stimulus Electrode Signals .......................................................... 17
Figure 4 fouter = 50(fpacket) ........................................................................................................................ 19
Figure 5 fspike_clock = 5(fouterpulse) ............................................................................................................... 20
Figure 6 fpartial_spike_clock = 4(finnerpulse) ....................................................................................................... 20
Figure 7 replacing the retina .................................................................................................................. 25
Figure 8 pixel by pixel mapping ............................................................................................................. 26
Figure 9 the first neuron model .............................................................................................................. 27
Figure 10 Adaline detail showing the ALC feeding to a bipolar output function. .................................... 30
Figure 11 neuronic detail ....................................................................................................................... 31
Figure 12 (a) shows linear discrimination as opposed to (b), (c) and (d) ............................................... 33
Figure 13 A hyperplane separating two classes/clusters of data. .......................................................... 34
Figure 14 message packet .................................................................................................................... 37
Figure 15 I frame sequencing ................................................................................................................ 40
Figure 16 colour dichotomy ................................................................................................................... 42
Figure 17 Flowchart of MATLAB program of Appendix A ...................................................................... 43
Figure 18 program output ...................................................................................................................... 45
Figure 19 Comparing epi_retinal approaches ........................................................................................ 46
Figure 20 Simulation context diagram (level 0) ...................................................................................... 47
Figure 21 FPGA view of processing ...................................................................................................... 48
Figure 22 AER Delivery ......................................................................................................................... 52
Figure 23 Biological neuron ................................................................................................................... 53
Figure 24 Action Potential ...................................................................................................................... 54
Figure 25 Retinal colour processing (original) ....................................................................................... 56
Figure 26 Four Colour Opponent Theory ............................................................................................... 60
Figure 27 Retinal Structure .................................................................................................................... 63
Figure 28 Receptive field example......................................................................................................... 65
Figure 29 showing fifty pulses/s (representing maximum intensity) and 5 pulses/s
(representing minimum intensity) .................................................................................................. 67
Figure 30 50 pulses/s and 5 pulses/s for one plane of virtual wire of test scene ................................... 70
Figure 31 1024 pixel behavioural test image ......................................................................................... 72
Figure 32 Colour composition ................................................................................................................ 73
Figure 33 sixteen pixel image ................................................................................................................ 74
Figure 34 Biphasic pulse compared to `spike’ ....................................................................................... 76
Figure 35 Correspondence of pixels to colour ....................................................................................... 78
Figure 36; Envisaged System ................................................................................................................ 79
Figure 37 Tae versus fps ........................................................................................................................ 80
Figure 38 forming AER stream .............................................................................................................. 83
Figure 39 Forming AER stream ............................................................................................................. 83
Figure 40 sender program overview ...................................................................................................... 84
Figure 41 Display of AER test image ..................................................................................................... 87
Figure 42 sender chip power analyser screenprint ................................................................................ 88
Figure 43 sender_rtl_16 ......................................................................................................................... 90
Figure 44 upto 1024 image i.e. 32 by 32 ............................................................................................... 92
Figure 45 (1048576) image limit i.e. 1024 by 1024................................................................................ 93
Figure 46 Receiver block diagram ......................................................................................................... 94
Figure 47 receiver power analysis ......................................................................................................... 96
vii of ix

Figure 48 receiver rtl_16 ........................................................................................................................ 98
Figure 49 Outputting to electrodes....................................................................................................... 102
Figure 50 Shift (reference) register ...................................................................................................... 106
Figure 51 Enable DAC ......................................................................................................................... 107
Figure 52 An R-2R ladder network D/A converter................................................................................ 108
Figure 53 A bipolar D/A converter........................................................................................................ 109
Figure 54 Axon clearance .................................................................................................................... 110
Figure 55 Retinal implant ..................................................................................................................... 113
Figure 56 AP propagating through optic nerve .................................................................................... 117
Figure 57 Practical setup ..................................................................................................................... 118
Figure 58 sender design summary ...................................................................................................... 173
Figure 59 design summary .................................................................................................................. 183
viii of ix

Table 1 Correlation between fps and AER packet frequency ................................................................ 21
Table 2 `and’ and `or’ truth tables .......................................................................................................... 27
Table 3 `xor’ and `nxor’ truth tables ....................................................................................................... 28
Table 4 row and column addressing ...................................................................................................... 43
Table 5 Receptive ON/OFF operation .................................................................................................. 64
Table 6 Tspike related to fps .................................................................................................................... 81
Table 7 examples of clocking hierarchy ................................................................................................. 82
Table 8 short AER format ...................................................................................................................... 85
Table 9 example correspondences ........................................................................................................ 86
Table 10 electrode requirement ............................................................................................................. 92
Table 11 clocking calculations ............................................................................................................... 95
Table 12 simulation clocking .................................................................................................................. 95
Table 13 receiver calculations ............................................................................................................... 99
Table 14 Receiver chip FPGA resource data ...................................................................................... 101
Table 15 Control signals to enable DAC .............................................................................................. 107
Table 16 IGLOO product table ............................................................................................................. 115
Table 17 image size versus bit length .................................................................................................. 115
ix of ix

Chapter 1 Introduction/research aim
The approach to be taken for this work is to determine and then decide how to
implement a retinal prosthesis capable of restoring sufficient vision to a person
suffering visual impairment to improve their quality of life. My goal here is not to
implant an actual prosthesis but to evaluate ideas about the way to efficiently
communicate the signals to the prosthesis. The boundaries of my study are to
behaviourally programme in a Virtex 5 FPGA[16] a test image of 1024 pixels and
structurally implement; in terms of VHDL programming, using the Xilinx ISE
platform an image of 16 pixels as proof of concept. The limitation of the structural
implementation is that the output of the FPGA will further require a post processing
interface i.e. the microstimulator or driver portion [17-22] which typically resides on
the retinal implant itself and enables the outputted signals from the FPGA to control
the timing of the charge being delivered to the electrodes of the retinal implant array
[23-30]. Although ten percent of the population suffer from visual impairment; so in
the UK six million people, only ten percent of those suffer from AMD (age related
macular degeneration) [Aside young people can also suffer from this condition] and
RP (retinitis pigmentosa) a family of related diseases to which this study applies.
Although this represents less than one million people in the UK clearly the
European/worldwide market is much bigger. It also bears saying that the gift of sight
is priceless for each person affected to a greater or lesser degree.
1.1 Introductory Literature Review
The following subsections pick out the differing approaches from the relevant
literature and also the commonality of those approaches where this exists.
10 of 200

1.1.1 Sub retinal
The sub retinal approach is to replace the retina by an array of micro photodiodes
underneath the retina (the side adjacent to the choroid) to channel natural sunlight
[31-40], the concept is to replicate the processing of the retina. Essentially a
subretinal implant would be inserted into the space normally occupied by
photoreceptors in a healthy retina. The array of micro photodiodes then provides the
stimulation data to the retinal implant. Most, if not all, sub retinal approaches rely on
converting the colours of light to greyscale prior to stimulation [41]. This has an
inherent flaw as the biological retinal actually processes incoming colour [42] in
terms of four perceptions of colour namely red, green, blue and yellow and processes
those colours to target `red’ cones, `green’ cones and `blue’ cones. These cones in
turn propagate colour intensity of each of those planes of colour to specific retinal
ganglion cells arranged in a regular topographical arrangement. The seriousness of
this inherent flaw depends quite literally on your point of view inasmuch as to be
truly biomimetic i.e. to imitate nature, a retinal prosthesis should target cones; for
colour vision. However it seems to be perceived wisdom that achromatic targeting is
of adequate sufficiency. The technological limitations driving this approach have
been twofold: firstly electrode size; and secondly extra wiring required for the colour
approach. Until recently electrode size has not been small enough; in vivo, to
accommodate average parvocellular cell size of 10µm and in some cases average
magnocellular cells of 80µm [43] i.e. although in vitro electrode size of 10µm has
been utilised, typically electrode size in vivo has been typically 50µm or in some
case 100µm. As the foveola (fovea pit) extends to 200µm diameter in the centre of
the fovea centralis, where the predominance of cones governs visual acuity, it is
nonsensical to map those signals out to 50µm electrodes; for colour vision, [5-7, 9,
10, 12-14, 44-48] where the RGC axons are about 1µm in diameter [49]. In the
11 of 200

foveola (fovea pit) there is a one to one correspondence from cone to ganglion cell
via the bipolar cell [50] implying that within that limitation retinal processing would
concern only colour. Outside that limitation sub retinal processing would need to
take into account stimulation biologically initiated from rods, horizontal cells,
amacrine cells etc. and hence would be more complex than the epiretinal approach
described in the next subsection.
12 of 200

Figure 1 Layer structure
13 of 200

NB that the photoreceptors of the foveola (fovea pit) feed to the nerve fibres within
the blind spot or optic disc. The optic disc is an oval of 1.76mm by 1.92mm giving
an area of 2.66mm 2 (http://www.websters-onlinedictionary.org/definitions/Optic%20disc)
The current state of technology using a
high density electrode array gives ≈ 100 electrodes/mm 2 [29],implying over 200
electrodes capability for stimulating RGC’s with an epiretinal implant
Figure 2 Eye diagram (image: Wikimedia Commons public domain)
1.1.2 Epi retinal
The epiretinal refers to the side that faces the vitreous [51], essentially an
epiretinal implant would rest on the inner limiting membrane of the retina [52].
Whereas a sub retinal approach relies on stimulation data being supplied due to
14 of 200

light falling on an array of photocells the epiretinal approach relies on electrical
stimulation data being provided from an external camera. The most well-known
artificial retinal/retinal prosthetic is that developed by the “Artificial Retina
Project” [53] described in the following way: “A pair of glasses with a small
video camera is worn by the patient. The video captured by the camera is sent
wirelessly to a belt pack containing a microprocessor that processes the video
signal. This processed video signal is sent to an antenna in the eye. The antenna
is connected to an array of electrodes that have been implanted directly inside the
eye on top of the old retina. The array of electrodes transmits signals that directly
stimulate optic nerve cells that are responsible for sending images to the brain’s
vision centers. “The `sender’ chip of the system developed here will be that part
of the system that processes the camera video signal. Address Event
Representation (AER) will be the processed video signal which is then sent to the
`receiver’ chip housed in close proximity to the implanted retinal array. The
receiver chip will, after post processing of the stimulation data, deliver biphasic
current pulses to stimulate the retinal ganglion cells (RGC) via the electrodes of
the retinal implant. Due to image size, and hence event activity, increasing the
computational load quadratically then behaviorally only a 32 by 32 (1024 pixel)
image will be demonstrated. Also due to the FPGA resource being used this
prototype implementation will be restricted to a 4 by 4 (16 pixel) image implying
48 electrodes to be driven although with a different resource an 8 by 8 (64 pixel)
image implying 192 electrodes to be driven could easily be accommodated
within the present 256 wire limit. Should this limit reach circa 1000 then a 16 by
16 (256 pixel image) implying 768 electrodes could also easily be
accommodated. An implementation of 32 by 32 (1024 pixels) suggesting 3072
15 of 200

electrodes to be driven implies stimulating 3072 RGC which would still be in the
foveal pit which is the area receptive to cones and hence colour stimulation.
1.1.3 Commonality e.g. AER & related issues
Sub retinal and epiretinal approaches both eventually deliver biphasic pulses to the
retinal implant to drive the electrodes which in the epiretinal case are in contact with
the axons of the ganglion cells. [54], [49], [55], [56], [57], [58]. Medical experiments
have indicated that the equivalent impedance for the retina tissue of Retinitis
pigmentosa (RP) and Age Related Macular Degeneration (AMD) is about 10kΩ.
[53] The default loading for the Virtex 5 FPGA is 50Ω; although this can be altered
it is unlikely that a required current to support a charge of 100nC at the electrode tip
can be accommodated. On this basis a post processing interface is required as
additional driving circuitry for the electrodes of the retinal implant.
Address Event Representation (AER) [59], [60], [61], [62] has been chosen as the
communication protocol between the sender chip and the receiver chip for two
reasons: (1) Identification of each pixels worth of information; by this is meant each
individual pixel is identified by its own unique address determined by its positioning
on the image to be transmitted. This will enable each AER packet to be tracked and
hence routed to its destination electrode driver. (2) Neuromorphic capability in the
sense that it is widely perceived as the communication protocol of choice to
represent the spikes of biological neural networks and by implication artificial neural
networks (ANN) [63-89]. As biological spiking is being represented each AER[90]
packet must contain information for pulse length, amplitude, frequency and form. An
AER packet as defined in the “Extended Address Event Representation Draft
Standard” (http://www.stanford.edu/group/brainsinsilicon/documents/methAER.pdf)
has an address with a payload. In other words each specific event has an address
16 of 200

associated with it. In its most fundamental form this would entail sending an address
with a payload of one `spike’ where one spike would be the case where a spike either
occurs or not. In this implementation the AER packet will be sent with an address
having a payload of 50 potential spikes, specifically an address with a payload of a
pulse count. After transmission from the sender chip to the receiver i.e. at reception,
the AER packet will be translated from an address with pulse count payload to that
of an address with 50 bits each; for each colour plane, representing the occurrence or
non-occurrence of a spike.
The reasoning of this protocol is that each pixels worth of information (AER packet)
can be routed to the correct destination after transmission; diagram illustrates this
concept for a two by two image i.e. four pixels (synonymous with four AER packets)
and hence for the three planes of colour twelve electrodes must be driven.
Pixel_1 red signal
Pixel_1 green signal
Pixel_1 blue signal
Pixel_2 red signal
Pixel_2 green signal
Pixel_2 blue signal
Pixel_3 red signal
Pixel_3 green signal
Pixel_3 blue signal
Pixel_4 red signal
Pixel_4 green signal
Pixel_4 blue signal
Figure 3 Mapping of Image Pixels to Stimulus Electrode Signals
17 of 200

Typically AER is used as a multi sender, multi receiver asynchronous protocol
whereas however in this epiretinal approach to a retinal prosthesis application its
neuromorphic capability is as a timed synchronous protocol. In other words as there
is only a single image to be transmitted; although each image is composed of
multiple pixels, then this time multiplexed synchronous protocol will not have
contention between AER packets. So despite the image signal eventually being
multicast to many electrodes an asynchronous protocol enabling handshaking and
avoiding contention is not necessary or warranted and will eliminate overheads
associated with such extra processing.
1.2 Conceptual overview
To accommodate the initial test setup described in the abstract a frame rate of 25
frames per second (fps) was chosen, this corresponds to the UK TV camera standard.
The UK TV standard; as with the 25p video format, deals with persistence of vision
where an after image of a stimulus remains long enough for the phi phenomenon (an
optical illusion) to occur i.e. the perception of movement. At 1fps to transmit each
pixels worth of information would take 1s divided by the pixel size of the image. So
for a four by four image i.e. sixteen pixels, pixel time would equal 1/16 i.e. 0.0625s
meaning 16Hz for the frequency of AER packets i.e. synonymous with pixel
frequency. Each AER packet contains the pixel address and payload e.g. pulse count,
corresponding to the colour information, during the AER transmission. At 25 fps the
frequency of AER packets, for this example, would be 25*16=400Hz. The frequency
of AER packets (f packet ) is the frame rate multiplied by the image size in pixels. There
is a linear relationship between the frame rate for any particular image size and AER
packet frequency that needs to be set e.g. 50*16=800Hz and for a 1024 pixel image
@ 25fps, AER packet frequency would equal 25.6kHz.
18 of 200

For each colour plane of maximum intensity (i.e. 255 for 24 bit colour) the AER
packet time (T packet ) [e.g. at 400Hz this will be 0.0025s] can be conceived of as
having 50 time slots available to accommodate each spike and its `off time’ before
another spike makes sense. The concept of 50 time slots being derived from the real
time firing rate (simulation frequency) for maximum intensity of pulsing i.e. 50
spikes/second [54, 56, 91, 92].
This `outerpulse’ time will be T packet /50 giving an outerpulse frequency of 50* f packet
[e.g. at 400Hz this will be 20 kHz].
Real timing - outerpulse relationship
20ms
(T outer )
outer
pulse
pixel_clock
1000ms (T pixel )
Figure 4 f outer = 50(f packet )
Each outerpulse will translate into an `innerpulse’ i.e. the portion represents the
biological action potential (the spike - biphasic pulse) and the associated `off time’.
19 of 200

Real time - innerpulse relationship
4ms
One Two Three Four
Five
spike_clock
(innerpulse)
outerpulse
20ms
Figure 5 f spike_clock = 5(f outerpulse )
Note that each innerpulse will be represented by a four part biphasic pulse requiring
a partial_spike_clock to form each of the four component parts of it.
Real time - biphasic pulse relationship
1ms
One Two Three Four
partial_spike
_clock
spike_clock
(innerpulse)
4ms
biphasic
pulse of
values
+
0
-
1
4ms
1
Figure 6 f partial_spike_clock = 4(f innerpulse )
20 of 200

Num. of: Num. of: f pixel T pixel_allocation @1fps @25fps @50fps @100fps
rows columns pixel_numbers T (pixel) {secs} f aer_pkt f aer_pkt f aer_pkt f aer_pkt
1 2 2 5.00E-01 2 50 100 200
2 2 4 2.50E-01 4 100 200 400
4 4 16 6.25E-02 16 400 800 1600
8 8 64 1.56E-02 64 1600 3200 6400
16 16 256 3.91E-03 256 6400 12800 25600
32 32 1024 9.77E-04 1024 25600 51200 102400
64 64 4096 2.44E-04 4096 102400 204800 409600
128 128 16384 6.10E-05 16384 409600 819200 1638400
256 256 65536 1.53E-05 65536 1638400 3276800 6553600
512 512 262144 3.81E-06 262144 6553600 13107200 26214400
1024 1024 1048576 9.54E-07 1048576 26214400 52428800 104857600
Table 1 Correlation between fps and AER packet frequency
1.3 Sub retinal versus Epi-retinal
Typically the subretinal implant is embedded `underneath’ the retina and uses
photodiodes to replace damaged photoreceptors, relying on natural sunlight for
power. The retinal processing can be described using Ewald Herings theory[42] in
which in 1878 he wrote: "Yellow can have a red or green tinge, but not a blue one;
blue can have only either a red or a green tinge, and red only either a yellow or a
blue one. The four colours can with complete correctness therefore be described as
simple or basic colours, as Leonardo da Vinci has already done." Whereas at the
optic nerve ganglion cells act on the trichromacy theory[93] i.e. red, green and blue
this implies a conversion between one form to the other. As an epiretinal implant
effectively replaces the retinal function such a conversion can be avoided [34, 54, 88,
94-128].
1.4 Introduction to stimulators
The stimulator (microstimulator) circuitry [53, 54, 57, 129] forms the post
processing stage of current retinal implants (sub and epi) and will also be required as
21 of 200

the post processing stage of this work. The stimulator resides with the retinal implant
and produces a biphasic current with which to stimulate the electrodes of the retinal
implant. Equivalent retinal tissue impedance as determined by medical experiments
is circa 10kΩ (R) and a with a maximum current threshold value of 500µA (I) this
requires a potential difference of 5v. Average power will be determined by a number
of factors e.g. width of cathodic part of pulse, interphase delay, width of anodic part
of pulse, spike rate and number of driven electrodes. A maximum power per
electrode can be calculated given some presumptions e.g. with a 1000 clock period
per frame/pixel, a spike rate of 50 and a pulse width of four clock periods with
cathodic portion being one clock period, interpulse delay equating to two clock
periods and anodic portion of one clock period; where the clock period is 1ms. Then
over a second this is an average energy distribution of 0.5 x 0.2 x 500µA = 50µJ =>
50µW. So for a 100 electrodes maximum power dissipation would be 5mW. The
volume of the human eye is 6.5 ml (0.4 cu. in.) and the weight is 7.5 g (0.25oz). In
the USA the SAR limit is 1.6 W/kg, averaged over a volume of 1 gram of tissue; for
the head. In Europe, the European Union Council has adopted the recommendations
made by the International Commission on Non-Ionising Radiation Protection
(ICNIRP Guidelines 1998). These recommendations set a SAR limit of 2.0 W/kg in
10g of tissue.
1.5 Thesis structure
The thesis is divided into six chapters and five appendices, organised in the
following way:
Chapter one (Introduction/research aim); this one, containing research aim,
selected extracts from the literature review, Distinction between the two approaches
to an artificial retinal prosthesis within the literature and this thesis structure.
22 of 200

Chapter two (Early directions): A description of the first tentative approaches,
concerning the sub retinal approach, to this PhD project i.e. a neural network
approach utilising MATLAB software.
Chapter three (Concepts): Neuronic principles, homing in on their application to
human visual processing. Address Event Representation (AER); method of choice in
which the conventional measure of colour intensity for each plane of colour is
translated into a neuromorphic representation for transmission propagation from the
sender chip to receiver chip. Description of the biphasic pulse commonly used to
mimic the biological action potential.
Chapter four (Sender chip): Sender concepts and the chosen format. Description of
a test setup wherein the AER format derived from conventional colour representation
is displayed on a monitor to demonstrate efficacy of the format. Listing of the reports
capable of being produced by the software and to be shown in the relevant appendix
Chapter five (Receiver chip): Clocking calculation for the sender/receiver.
Components for the receiver chip. Discussion of post processing necessary between
the receiver chip output and that required for the implant electrodes.
Chapter six (Concluding chapter): Resulting conclusions and future work.
Appendices
Appendix A: MATLAB simulation
Appendix B: FPGA Test setup
23 of 200

Appendix C: FPGA Sender chip
Appendix D: FPGA Receiver chip
Appendix E: Outputs from FPGA
Chapter 2 Early directions
Initially investigations were commenced utilising the MATLAB application software
package as a cost effective precursor prior to a hardware oriented approach. Initial
investigations involved a sub-retinal approach utilising the neural network toolbox
supplied within the MATLAB software. Research into the application of Artificial
Neural Network (ANN) when dealing with `spikes’ [75, 130-141]; as a form of
neuromorphic communication, informed its suitability for this purpose.
2.1 Introduction to initially envisaged project
The approach would be to replace damaged parts of the retina, prior to the rods and
cones, with a neural network artificial retina to fulfil the function of the amacrine,
horizontal and bipolar cells[46].
24 of 200

Eye
Neural
Network –
Artificial
Retina
Lens
Optic nerve
Figure 7 replacing the retina
Within the visible spectrum extending from extreme red; 760.6nm, to extreme violet;
393.4 nm lie the four psychological primary colours of Ewald Hering’s theory[12].
So called because it does not appear to the human eye that they can be separated into
any more basic colours e.g. bluish-green or yellowish-green can be imagined but not
greenish-red or a bluish-yellow. The eye has three cone receptors conventionally
described as red, green and blue cones following Thomas Young’s trichromacy
theory[12]. The red (long γ) cone has pigments which absorb maximally at 565nm,
green cone (middle γ) absorbing maximally at 530nm and blue cone (short γ)
absorbing maximally at 420nm[12]. Therefore the retina converts the four primary
colours (red (620–750 nm), green (495–570 nm), blue (450–475 nm) and yellow
(570–590 nm)) to which it is sensitive into the three primary colours red (R), green
(G) and blue (B) to which the optic nerve is sensitive.
2.2 Neural network representation
25 of 200

Red
(660nm)
R / G Red
( 620750nm )
Green
(520nm)
Blue
(470nm)
Yellow
(575nm)
photodiodes
Artificial
Neural
Network
(ANN)
G/
R Green
( 495570nm)
B/
Y Blue
( 450475nm )
Output
signals
(5 – 50)
impulses/s
Figure 8 pixel by pixel mapping
Using Artificial Neural Network Theory (ANN) to mimic the processing of the
neural network of the retina has the advantage that given the correct inputs to the
network and the expected output the designed network is self-training in the sense
that once trained with a set of quality test inputs; of a predetermined quantity, it will
always produce the required results even with differing inputs applied. The following
subsections give a primer for the theory which needs to be understood for an ANN to
be designed as an approach to replace the retina using a photodiode approach as
opposed to bypassing it with a camera.
2.2.1 The first artificial neuron
Warren McCulloch and Walter Pitts in their 1943 paper, “A logical calculus of Ideas
Immanent in Nervous Activity” postulated that neurons with a binary threshold
function were analogous to first order logic sentences. This first neuron model is
shown here.
26 of 200

w2
w 1
Output
y
Neurode
x 2
Input
x 1
Input
Figure 9 the first neuron model
By setting a threshold relevant to the inputs to be expected e.g. 1 unit then for an
input of x 2 =0.5 units and x 1 = 0.5 units then threshold would be achieved and there
would be an output from the neurode/neuron. The Boolean logic of this model would
then be represented by the truth tables shown next.
AND
OR
Input x 1 Input x 2 Output Input x 1 Input x 2 Output
0 0 0 0 0 0
0 1 0 0 1 1
1 0 0 1 0 1
1 1 1 1 1 1
Table 2 `and’ and `or’ truth tables
This type of neuron could also be used to solve the NOT function (with only one
input) as well as the NOR and NAND functions. However the following truth tables
were not so easily implemented.
XOR
Input
Input
NXOR
Input
Input
x 1 x 2 Output x 1 x 2
0 0 0 0 0 1
0 1 1 0 1 0
Output
27 of 200

1 0 1 1 0 0
1 1 0 1 1 1
Table 3 `xor’ and `nxor’ truth tables
The problem with this simple model was that it only allowed for binary inputs and
outputs, it only used the threshold step (Heaviside) activation function and it did not
incorporate weighting the different inputs.
2.2.2 Hebb’s rule
“When an axon of cell A is near enough to excite a cell B and repeatedly or
persistently takes part in firing it, some growth process or metabolic change takes
place in one or both cells such that A’s efficiency, as one of the cells firing B, is
increased..”[142].To accommodate this idea into the McCulloch-Pitts neuron it
would be necessary to `weight’ the inputs.
2.2.3 Adaline and the adaptive linear combiner: a bipolar
case of above
“The Adaline is a device consisting of a single processing element; as such, it is not
technically a neural network…. The term Adaline is an acronym; however, its
meaning has changed somewhat over the years. Initially called the ADAptive Linear
Neuron, it became the ADAptive LINear Element, when neural networks fell out of
favour in the late 1960s….The dashed box; in the following figure, encloses a part of
the Adaline called the adaptive linear combiner (ALC). If the output of the ALC is
positive, the Adaline output is +1. If the ALC output is negative, the Adaline output
is -1. …..The ALC performs a sum-of-products calculation using the input and
weight vectors and applies an output function to get a single output value.
y w
o
n
j1
w
j
x
j
28 of 200

Where w
0
is the bias weight; if we make the identification, x 1 0
, we can rewrite the
preceding equation as
Or in vector notation,
n
y
j0
w j x j
y = w T x
The output function in this case is the identity function, as is the activation function.
The use of the identity function as both output and activation function means that the
output is the same as the activation, which is the same as the net input to the unit.
The Adaline (or the ALC) is ADAptive in the sense there exists a well-defined
procedure for modifying the weights in order to allow the device to give the correct
output value for the given input. What output value is correct depends on the
particular processing function being performed by the device. The Adaline (or the
ALC) is LINear because the output is a simple linear function of the input values.
…The Adaline could also be said to be a LINear Element…..”[74] Note that a
network of more than one Adaline is termed a Madaline (Many Adaline).
29 of 200

Adaptive linear combiner
w 0
1
Let x 0 = bias (b)
Bipolar output = sign y
Threshold
x 1
w 1
Body
+1
x 2
w 2
∑
v
y
x 3
w 3
x n
w n
-1
Activation
function
Figure 10 Adaline detail showing the ALC feeding to a bipolar output function.
2.2.4 Neural Networks –perceptron training
The model of the neuron described in the following figure was used by Frank
Rosenblatt in 1958, as the basis for the first trainable neural network. His perceptron
training algorithm provided the first procedure that could be used to allow a network
to learn a task. This classic task is that of separating several patterns into two
categories.
30 of 200

Axons
Synapses
x 1
w 1
Dendrites
x 2
w 2
Body
x 3
x n
w 3
w n
∑
Axon
ƒ y = Output
v
Non-
Linearity
w 0
Input
Weights
1
Bias (b)
y f
n
i1
( w x w0)
i
i
Figure 11 neuronic detail
From the model we find that the input (v) i.e. induced local field of the neuron to the
activation function; in this case a hard limiter, is:
n
v wi
x
i1
i
b
(2)
Where b refers to the bias, w the particular weight for an input, x a particular input
and v the hard limiter input. The aim of the perceptron is to correctly classify the set
of inputs into either one class or the other. That is to one class if the perceptron
output is > 0 (or 0) and to the other if it is

is correct (one class) and -1 (the remaining class) if the perceptron answer is wrong.
And y = binary output signal.
In its simplest form the two decision regions are separated by a hyperplane defined
by:
n
i wi xi
b 0
(3)
i1
Referring back to equation (2) it can be said that more generally (taking into account
one neuron connecting to another) that the induced local field of the neuron to the
activation function is:
net
j
w
n
0
xiwij
(4)
i1
Where w 0 is the bias and is considered to be connected to a unit that always has an
activation of 1; x i is the activation for neuron i and w ij is the weight connecting from
neuron i to neuron j.
2.2.5 Linear separability
The following figure illustrates the concept of linear separability – the pattern classes
can be separated into two classes by drawing a single line. The figure shows four
example pattern sets: the first is linearly separable and the other three are not. In this
example each pattern consists of a vector of two real numbers graphed as a single
point in the diagram. Patterns in the first class are graphed as points in the clear area
and patterns in the second class are graphed as points in the shaded area.
32 of 200

B
A
Linearly separable regions
(a)
B
A
A
B
A
B
B
A
(b)
(c)
(d)
Figure 12 (a) shows linear discrimination as opposed to (b), (c) and (d)
The linear separability concept can be demonstrated graphically by plotting the
inputs to a perceptron i.e. one single neuron (or unit) one against the other to
represent the input/pattern/feature space, this is shown in figure 13.
33 of 200

x2
Linear separation
1.00
0.90
0.80
0.70
0.60
0.50
0.40
x2
Class A
Class B
Linear (x2)
0.30
0.20
0.10
0.00
-0.60 -0.40 -0.20 0.00 0.20 0.40 0.60 0.80 1.00
x1
Figure 13 A hyperplane separating two classes/clusters of data.
Based on the equation for a straight line i.e. y = mx +c we can say that the equation
for a line separating the data clusters of class A and class B is x 2 = m x 1 +c. This can
be re-arranged to give x 2 – mx 1 – c = 0 and we can find the value for c by noting
where the line passes through the x 2 axis (the y of the fundamental straight line
equation), i.e. at x 2 = 0.235. Therefore: when x 1 = 0, x 2 = 0.235. Substituting this into
x 2 = m x 1 +c, this gives 2.35 – m*0 – c = 0 and thus c = 2.35.
Likewise m can be found by observing where the line crosses the x 1 axis, at x = -0.4,
so when x 2 = 0, then x 1 will equal -0.4. Substituting gets 0 – m (-0.4) – 2.35 = 0, thus
0 – m (-0.4) = 2.35 which is 0.4m = 2.35, therefore m = 2.35/0.4 = 5.875.
The equation for the straight line (hyperplane) of figure 10 is:
x 2 – 5.875x 1 – 2.35 = 0. (5)
In terms of neural networks this hyperplane is fixed by the weights and biases of the
neuron. The bias defines the position of the plane in terms of its perpendicular
34 of 200

distance from the origin. The weight vector of the neuron will determine the
orientation of the decision/hyperplane, i. e. it affects the slope (m) of the graph.
As the hyperplane is fixed by the weights and biases of the neuron if different values
of x 2 and x 1 are substituted into the hyperplane equation (5) and the result is greater
than zero then that co-ordinate point lies above the hyperplane and is class A.
Whereas if the result of the substitution < zero then that co-ordinate point lies below
the hyperplane and is class B.
Recall: our previous definition of the hyperplane (3) i.e. i wi
xi
b 0
Typically the bias term is replaced by w 0 x 0 where x 0 is always 1 therefore (3) can be
replaced by:
The following hyperplane definition
2
w i
i0
i
x w0
x0
w1
x1
w2
x2
0 (at thehyperplane )
(6)
Therefore: for a perceptron i.e. a single neuron.
The output of the unit, y, is 1 when the weighted sum is greater than 0, i.e.
y 1when
w x0
w1
x1
w2
x2
0
Similarly, the output is 0 when the weighted sum is less than o, i.e.
y 0 when w x0
w1
x1
w2
x2
0
The hyperplane between them corresponds to the weighted sum being equal to 0.
0
0
n
i1
2.2.6 The Delta Rule (used to determine separability)
“When an input vector is applied to an adaline neuron the computation of the net
input to the adaline is the standard weighted sum.
In a two dimensional example this means that the input will be: I = x 1 w 1 + x 2 w 2
35 of 200

This is also the Cartesian co-ordinate system method of computing the dot product of
the input vector and the weight vector.
Conclusion: the weighted sum most commonly used in the transfer functions of
neural networks is the same as computing the dot product of the input pattern vector
with the receiving neurons weight vector.
Alternatively x ● w = |x| x |w| Cos Ø where Ø is the angle between the input vector
and the weight vector. N.B: From trigonometry cos is +ve if the angle is < ± 90° and
–ve otherwise.”
w x w x w x 0
(7)
0 0 1 1 2 2
This can be rearranged to give:
w
2x2
w1
x1
w0
x0
(8)
w w
x
(9)
1 0
2
x1
x0
w2
w2
Comparing equation (9) with equation (5) i.e. x 2 – 5.875x 1 – 2.35 = 0 and putting
x 1gives
w1
w0
0
m , and c
w2
w2
There is no unique solution to these two equations. Just as an example, let us start by
assuming that w 1 0 then equation (5) is obtained from the values:
2
.
w
0
2.35 and w1
5.875
With these weights the unit can discriminate between the two classes of objects. A
single unit with these weights classifies objects A and B in the preceding figure 13.
36 of 200

2.3 Address Event Representation (AER)
Address-Event-Representation (AER) is a communication protocol that emulates the
nervous system’s neurons communication and that is typically used for transferring
images between chips. It was originally developed for bio-inspired and real-time
image processing systems.[4, 59, 61, 62, 64, 143-159]
2.3.1 The message packet
This consists of a source address segment; determined by the image size containing
the pixel data to be routed, and a payload segment containing the event or train of
events associated to each address. In the case of a train of events not all of the events
would necessarily be there.
Address segment
Payload
Figure 14 message packet
Although the payload can be empty when using the AER protocol (not definitive)
this would imply that the address itself is indicative of an event. The AER packet is
in fact used in this way; when processed after transmission reception, and also for
convenience in the test setup when demonstrating reception of the AER packet for
display to the Digital Video Interfaced (DVI) VGA monitor. However by using a
payload of a pulse count, rather than the way just detailed, bit rate can be reduced
dramatically in the sense that rather than 50 addresses being transmitted for 50
separate events; at maximum intensity, meaning 50 separate AER packets, this
reduces to one AER packet with one payload representing a train of events.
37 of 200

2.4 Comparison to Frame Based Representation
(FBR)
An objective of the retinal prosthesis as part of its aim to improve the quality of life
for those for whom it can be used is that it should operate in real time. Failure of real
time operation could be the cause of; for example, a road accident where traffic is
not seen at the time. The following sections describe and compare the use of FBR;
the methodology used for video and television with AER the methodology used for
neuromorphic communications to confirm the method closest to real time in the
present context.
Frame Based Representation (FBR) of images captures images up to 25 frames per
second (fps); H.264 is a new video compression scheme that is becoming the
worldwide digital video standard for consumer electronics and personal computers.
H.264 has been adopted by the Motion Picture Experts Group (MPEG) sometimes
referred to as MPEG-4 Part 10 or as AVC (MPEG-4’s Advanced Video Coding).
While MPEG-2 is a video-only format, MPEG-4 is a more generic media exchange
format, with H.264 as one of several video compression schemes offered by MPEG-
4. Because H.264 is up to 3 times more compressive than MPEG-2 due to extra
overhead e.g. entropy encoding, smaller block size and in-loop deblocking it comes
at an exceptionally high computational cost making it impossible to rely solely on
PC’s CPU, for example. As the AER task under discussion is not a generic media
exchange format, the MPEG-2 synchronous video-only format will now be discussed
in this context.
The MPEG-2 White paper states four levels of bitrate and the frame (image size
associated with those bitrates, the highest bitrate defined at the highest level is
80Mb/s. An MPEG bitrate for a 32 by 32 test scene with no interframe compression
38 of 200

would be 12.78Mbit and is defined for the Society of Motion Pictures and Television
Engineers (SMPTE). Essentially these levels refer to the source format and selfevidently
the “high” level will be used for this discussion. Along with these four
levels referring to source format being defined, five profiles are described; the inputs
to all profiles are YUV component video. The “High Profile’, to be used in our
discussion, includes tools for improving signal-to-noise-ration (SNR), or spatial
resolution, plus the ability to code line-simultaneous colour-difference signals.
Encoding of video information is accomplished by spatial compression occurring on
` (I) ntra’ frames and temporal compression occurring on both ` (P) redictive’ frames
and ` (B) i-directional’ frames. I, P and B frames are grouped into a specified
sequence known as a group of pictures (GOP). The `I-frame’ is a compressed version
of a single uncompressed (raw) frame. The MPEG - I frame is a variation of the
Joint Photographic Expert Group (JPEG) format which uses the discrete cosine
transform (DCT), quantising, run length codes and Huffman coding.{(JPEGs are not
suitable for graphs, charts and explanatory explanations because the text appears
fuzzy, especially at low resolutions). P frames require an `I frame’ or `P frame’ to
forward predict motion estimation, B frames are encoded using motion compensated
prediction on the previous and next pictures, which must be either a B or P frame. B
frames are not used in subsequent predictions. It is better to use a shorter group
length for fast moving scenes otherwise pixellated frames occur. For this reason of
quality affected by rapid scene changes and excessive motion the sequence of I to B
frames must often be altered, this is done by I frame “forcing” i.e. adding I frames.
In this way each new scene can start with a fresh I frame for B and P frame
calculations. GOPs are optional in an MPEG-2 bitstream but are compulsory in DVD
video.
39 of 200

Typical GOP structure (Typically an I frame would occur every 15 frames)
I B B P B B P B B P B B P B B
Figure 15 I frame sequencing
However AER is not constrained to the MPEG maximum bitrate and power
consumption is markedly lower than that of MPEG. The optic nerve expects a pulsed
format rather than a bit word. MPEG would require conversion to that format as it is
designed to output to a monitor. This AER system on the other hand will be designed
to output a bespoke pulsed format from the received transmission signal for the optic
nerve implants.[155, 160-167]
2.4.1 Comparison Criteria
Based on the biomimetics the visual prosthetic must be able to accommodate both
rapid and full scene changes. Using P and B frames MPEG-2 cannot reliably do this
to acceptable biological standards of visual acuity. Therefore in order to compare
synchronous MPEG-2 with AER for real time operation the GOP should be I frames
only, this unavoidably reduces compression from circa 83:1 to circa 5:1; same as M-
JPEG. This is fully justified as this AER system does not require motion estimation
and prediction to transmit scene changes which effectively show motion as they
occur in real time. Qualitative metrics: MPEG-2 is computationally intensive,
particularly at the encoding side, uses a global clock which will be power hungry,
does not give the required neuromorphic output at the decoding output, uses a
restricted bitrate and uses JPEG flavour spatial/intraframe compression that actually
discards some picture information. AER, described next (prior to quantitative
metrics) is not computationally intensive, gives a neuromorphic output, has low
power consumption, uses a bitrate restricted only by the technology; not the
standard, and gives a choice of picture format, thereby not necessarily discarding
40 of 200

picture information. In MATLAB terms *.bmp file format has been used for the
uncompressed (raw) frames upon which the I frames are based although in principle
*.jpg could be used. CCD/CMOS digital cameras give a choice of outputs: a “raw”
proprietary format commonly based on a bayer filter (though there are others)
YUV/RGB and with including software JPEG and TIFF (4-plane) format. The TIFF
format, in terms of its flexible usage of its fourth plane capacity could potentially
give the option of a yellow plane of colour! Retaining a fourth plane could, even as a
`yellow’ plane impart luminance data for use with RGC related to rods, for use with
hybrid retinal prosthesis as technological capability increases.
2.4.2 Address Event Representation
The Address Event Representation (AER) communication protocol {Thiago T.
Culurciello E. Andreou A.G., 2006 #56} uses a fast digital bus in conjunction with
time division multiplexing (TDM) to propagate spike information from one point to
another. This transmission has a twofold legitimacy to simplifying parallel
computation (1) reducing the number of dedicated wires which would be required in
a purely parallel computation, to that of a number of shared wires, and (2) that of
allowing decoding to be performed at the output end: in context; both in terms of
demultiplexing and converting the spike information from unit time into real time
spikes. The spikes; called events when discussing the protocol are inextricably linked
to an address, in effect counting the number of addresses per unit time gives the
number of events or spikes. Because time multiplexing is used an increased
bandwidth is expected e.g. compared to a standard using data compression such as
MPEG-2, however, as spikes are only produced when events occur a lot of the
bandwidth will use negligible power, less power consumption than say MPEG-2.
Events occur on a still image; the address is there, otherwise it would not be seen, the
41 of 200

point is an event occurs for spike frequency duration but at outerpulse frequency i.e.
one time in five and that at maximum intensity otherwise less often.
2.5 MATLAB sub-retinal simulation
Dealing with the visible light spectrum from a ‘scene’ of pixels in the form
suggested by the colour opponent theory of: Red (R), Green (G), Blue (B) and
Yellow (Y) primaries and converting those incoming retinal signals to Red (R),
Green (G), Blue (B) primaries suggested by the Young-Helmholtz trichromacy
theory required by the optic nerve.
Signals within the retina
Signals leaving the retina
Optic nerve
Figure 16 colour dichotomy
2.5.1 MATLAB representation
A MATLAB program has been constructed to illustrate the signals leaving the retina,
initially with a scene of 128 by 128 pixels later refined to an 8 by 8 scene for quicker
implementation [appendix A]. The program utilizing the Kronecker product[168] , as
a shortcut to compose some large matrices with repeated rows or columns, consists
of six files: mixedsource.m, ysend.m, pwork.m, scan.m, subi.m and ycut.m.
In terms of the Matlab simulation the control word for the addressing which will be
the heart of the AER router is fitted in between the R, G, B and Y planes of colour.
42 of 200

none
Thus for row addressing and then column addressing on planes five and four;
respectively, we have:
b8 b7 b6 b5 b4 b3 b2 b1
2 7 2 6 2 5 2 4 2 3 2 2 2 1 2 0
128 64 32 16 8 4 2 1
A7 A6 A5 A4 A3 A2 A1 A0
b8 b7 b6 b5 b4 b3 b2 b1
2 7 2 6 2 5 2 4 2 3 2 2 2 1 2 0
128 64 32 16 8 4 2 1
A7 A6 A5 A4 A3 A2 A1 A0
Table 4 row and column addressing
2.5.2 Program operation
The following flowchart outlines the main operation of the program.
start
Load
images
Initialise
variables
Read
previous
image
Read
current
image
more
Check for
more
images
Capture
changes
Display
revised
images
End
Figure 17 Flowchart of MATLAB program of Appendix A
43 of 200

2.5.1 Files in Appendix A
“ycut.m” is the main file which calls mixedsource.m to write the nine test frames
contained therein to the directory/folder to be read by the program. Those image files
are in `bmp’ format. The remaining files are function files. The first level function
file “pwork.m” applies to the group of event changes within each test frame. The
second level function files have the following functions: (1) “scan.m” adds the
control word for row and column addressing to the 5 th and 4 th planes respectively of
the RGB coloured frames and also adds the 6 th (`yellow’) plane. (2) “subi.m” breaks
down the image into separate pixel information which is then loaded into a cell array
for use in the processing. (3) “ysend.m” shows the neuromorphic representation of
the pulsing sent between the retina and the router/implant.
2.5.2 Use of the program
Run “ycut.m” accepting defaults with the return key. The figures show (from left to
right) the first reference scene; prior to a change (a collection of a group of event
changes). The second figure shows the changed scene. Thirdly, neuromorphic
pulsing is represented. The penultimate figure on the second row is a visual
representation of the changes sent from the retina to the router. The final image
represents the signals which would be presented to the implanted array at the optic
nerve. Presently, for ease of use in terms of implementation timing the test frames
are eight square, this can be altered to 128 square but will run more slowly.
44 of 200

Figure 18 program output
2.6 Epi-retinal; method of choice
The following diagram gives some idea of the differences involved between FBR
and an AER approach to producing the biphasic pulses required for propagation to
the ganglion cells comprising the optic nerve fibre. Essentially the AER approach is
pixel based rather than frame based.
45 of 200

MPEG-2 setup
AER setup
Camera
Camera
Encoder
Sender
Transmission lines
Decoder
Receiver
Router
Conversion
(to optic
nerve
format)
To optic nerve implants
Router
To optic nerve implants
Figure 19 Comparing epi_retinal approaches
In the case of the FBR approach it can be seen that an extra step is required for
conversion to optic nerve format that an AER approach inherently supplies. In effect
the sender chip receiving an RGB format from the camera generates an AER signal
for transmission to the receiver/router chip which drives the retinal implant array of
electrodes.
46 of 200

2.6.1 MATLAB representation of epi_retinal approach
AER protocol
Camera
stream
Sender
chip
Digital address
stream (AER)
Receiver
chip
Actual pulse
stream
Retinal
implant
32 x 32
image
Figure 20 Simulation context diagram (level 0)
In practice the camera would be affixed to specialised headgear or in the case of a
miniaturised camera to bespoke spectacle frames. The power supply for the camera
would be attached to a belt, also containing the `sender’ chip (responsible for
conversion to AER). After propagation of AER to the `receiver’ chip, wireless
telemetry could be adopted[169]. The receiver chip would be sited behind the ear
similar to the procedure presently adopted for cochlear implants [101]. Although
some early papers [56] suggest the possibility of incorporating the in vivo portion of
the receiver chip inside the eyeball itself this adds extra complication to present
telemetry communication already in use. Extra complication such as restriction of
eyeball movement (necessary for efficient inductive coupling), distance of telemetry
(unfeasible - from external to the skull to the eyeball), extra mass\size and extra
power dissipation within the eyeball being greater than achieving without breaching
safety guidelines. This document addresses the issue of prototype retinal prostheses
without considering extra telemetry issues such as those outlined which are best
relegated to future work. The wiring from the subcutaneous portion of the receiver
chip is presently (10/07/2012) limited to 256 wires into the eyeball due to the size
limitation (5mm) of the incision [54] which can safely be made without risk of
biological damage. Given present research into nanowire technology this limit may
potentially increase.
47 of 200

2.6.2 FPGA representation of epi_retinal approach
The following diagram (figure 21) gives a basic overview of the epi_retinal
prosthesis. The sender chip receives the camera output and converts it to an AER
protocol[170]. This serial communication of a stream of pulses is wired from the
sender chip to the receiver chip. In this diagram the separation of the receiver chip
into two distinct parts i.e. external to the body and internal to the body is not shown,
this is ably discussed in other papers [53]. The action of the receiver chip is to
convert the serial transmission into the parallel form required for transmission to the
retinal implant incorporating timing considerations, to form biphasic pulses. The
implant (Addressing the implant) is then addressed in the sense that all wires now
entering the eyeball carry a separate stream of information for each electrode to be
driven by the post processing electrode drivers attached to the electrode array.
AER protocol
Receiver.
Retinal implant.
PISO
Sender.
Digital address
stream (AER)
Serial to parallel conversion
Timing considerations
Addressing the implant
Form biphasic pulses
Electrode
array
Control/
addressing
Figure 21 FPGA view of processing
48 of 200

2.7 Background to stimulators
Retinal stimulation approaches up to 2007[21, 53, 54, 124, 171, 172] and in some
cases beyond [102, 173] have presumed for epiretinal prosthetic targets the use of
bioadhesives or retinal tacks [174, 175] to affix the retinal array in position.
Relatively large electrodes e.g. 500µm[102]; in relation to neuron size, are another
feature of current work. In 2008 [176] 3-D electrodes with diameter 100µm and
height of 25µm were used. In 2010 [97] penetrating electrodes were used. Note; the
latest penetrating array for epiretinal prosthesis [29] makes possible high density
arrays (≈100 electrodes/mm 2 ) and arrays of 2µm, 5µm, 10µm, 20µm and 30µm
diameter with heights of 60µm-100µm and pitches of 50µm-400µm. So closer
proximity to the target cell/neuron can be achieved. We already know [102] that a
smaller electrode size of 50–100µm in diameter can have a smaller current stimulus
of between 10-100µA; say rheobase of 50µA and 100µA at a chronaxie of 1ms and
closer proximity at smaller electrode diameters implies an improvement on this!
2.7.1 Charge considerations
The safe charge limit [103]for Platinum(Pt) electrodes is 0.35mC/cm 2 , for Iridium
oxide (IrO x ) 3mC/cm 2 and for titanium nitride(TiN) 23mC/cm 2 . So for the charge for
a 1ms pulse at 100µA; using Q=It we have a charge requirement for stimulation of
100nC (0.1µC). Given these figures implies the following minimum simulation site
diameters for the achromatic approach using disc electrodes ≈190µm (Pt), ≈65µm
(IrO x ) and ≈24µm.
2.7.2 Achromatic current requirements
In the early approach of “A Neuro-Stimulus Chip with Telemetry Unit for Retinal
Prosthetic Device” [53] the stimulator is based on a 4-bit binary-weighted digital-toanalog
converter (DAC), and thus consists of 15 parallel current sources. A switched
bridge circuit was designed to create a biphasic current pulse from a single current
source thus allowing a reduced voltage swing. The pulse width modulated (PWM)
signal used in this early effort allowed the recovery of a clock signal using the rising
edges of each pulse recovered from the amplitude shift keying (ASK) modulated
waveform. In other approaches [124, 171] there are outline diagrams showing the
voltages involved during biphasic stimulus.
49 of 200

Chapter 3 Concepts
An epiretinal implant has no light sensitive areas; unlike subretinal implants, but
typically receives electrical signals from a distant camera and processing unit outside
of the body. Electrodes in the epiretinal implant then directly stimulate the axons of
the inner layer ganglion cells that form the optic nerve. The epiretinal implant is
effectively a readout chip receiving electrical signals containing image information
from a camera and processing unit, and is electrically coupled to the ganglion cell
axons. This project will concentrate on the processing [177-182] unit outside of the
body but will be novel in the sense that it will use colour information of scenes
viewed as opposed to achromatic approaches taken in every other work to date.
Colour is an extra challenge as the RGC axon needs to be targeted in an individual
sense and as the RGC axon diameter is of the order of 1µm/2.5µm [49 {Schröder,
1988 #257]} use of 50µm/100µm electrodes is not an option as might be the case
with achromatic stimulation. Colour stimulation in that 10% of axons that are
devoted to the fovea is now viable due to the order of magnitude decrease in
electrode sizing [29]. A possible reason, among others, for the appearance of
coloured phosphenes in achromatic retinal prostheses may be this blanket coverage
of RGCs including those comparatively few expecting colour information as
opposed to the preponderance of nerve fibres stimulated by the rods. Another issue
with colour is three times the `wiring’ is needed for the same pixel size as is needed
for the achromatic approach. However targeting the high acuity region of the fovea
with its high concentration of RGCs; evolved to process colour from the cone
photoreceptors is the more biomimetic and natural approach. In other words we were
designed to see in colour not shades of grey.
50 of 200

The work has evolved from inception to using Address Event Representation (AER)
[12, 153, 155, 161-164, 183, 184]. as a way of addressing the issues involved with
transmission of visual signal processing information with coding of the signal to
avoid extensive wiring between system components, namely between the sender chip
and the receiver chip. Use of AER will also reduce bit rate and hence power
requirements. This is important if telemetry is used; to keep specific absorption rate
(SAR), to acceptable limits. In the USA the SAR limit is 1.6 W/kg, averaged over a
volume of 1 gram of tissue, for the head. In Europe, the European Union Council has
adopted the recommendations made by the International Commission on Non-
Ionising Radiation Protection (ICNIRP Guidelines 1998). These recommendations
set a SAR limit of 2.0 W/kg in 10g of tissue.[185] In terms of physical size of a
retinal implant chip it must fit within the eyeball: vertical diameter is 24 mm; the
transverse being larger. The weight of the human eye is 7.5g. The volume is 6.5 ml
(0.4 cu. in.) and the weight is 7.5 g (0.25oz).
51 of 200

AER Transmission
lines
INPUT
image
signal
SENDER
(chip)
RECEIVER
(chip)
256
electrode
implant
De_multiple
xer
Up to 256
256
electrode
implant
Figure 22 AER Delivery
3.1 Neural concepts
For many years algorithms have been sought to simulate the visual processing of the
human brain. The brain contains up to 100 billion processing units called neurons,
each having a switching delay of several milliseconds. The fast reaction of the brain
is achieved by massive parallel processing; due to the networking of these neurons,
i.e. massive interconnection. The basic component parts of a neuron are the axon –
signal output, the synapse – the interface between the axon of one neuron and the
input to the dendrites (thread like connections to the body of the neuron) of the next,
and of course the body (soma) itself. (See Figure: Biological neuron)
52 of 200

Figure 23 Biological neuron
3.2 Biological Neuron operation
The specific process performed by a neuron depends on the properties of its external
membrane. This fulfils five functions: it serves to propagate electrical impulses along
the length of the axon and its dendrites; it releases transmitter substances at the
extremity of the axon. It reacts with these transmitter substances in the dendrites at
the cell body. It reacts to the electrical impulses which are transmitted from the
dendrites and generates, or fails to generate a new electrical pulse [186-188]. During
development of the brain the external membrane enables the biological neuron, of
which it is part, to recognise other neurons to which it should be connected. The
53 of 200

electrical pulse generated is called in biological terminology, an action potential, and
is shown in Figure 21.
Membrane potential (mv)
+30
0
-70
0 1 2 3 4 5
Time(ms)
Figure 24 Action Potential
3.3 Natural limit for pulse duration
The biological process of producing an action potential delivered from the retina to
the optic nerve fibres creates the pulse. [10, 11, 48]As can be seen from Figure 21;
although the actual pulse occurs within 2ms there is a 2ms refractory period before
another pulse can be produced. The pulse rate varies between five to fifty pulses/s
where 5 is the eye `resting’ (if all fibres of the nerve receive the same signal) and 50
where the optic nerve fibre is responding to a full intensity signal. At this maximum
throughput a spike plus its interval to next spike will take 1000 50 = 20ms. At a
nominal minimum throughput of five spikes a spike plus its interval to next spike
would be 10005 = 200ms.
54 of 200

3.4 Retinal colour perception
The retina appears to be inside out, since the receptors that transduce visual signals
are located in the outer layer at the very back of the eye, with their photosensitive
outer segments aimed away from the incoming light [37, 189-191]. Over the whole
human retina, there are 120 x 10 6 rods and 6 x 10 6 cones but only 1 x 10 6 ganglion
cells.[46] The fovea comprises less than 1% of retinal size but takes up over 50% of
the visual cortex in the brain. The human fovea has a diameter of about 1.5 mm with
a high concentration of cone photoreceptors. The center of the fovea is the foveola
and is approximately 0.35 mm in diameter and lies in the center of the fovea and
contains only cone cells, and a cone-shaped zone of Müller cells [192]. The foveola
is located within a region called the macula, a yellowish, cone photo receptor filled
portion of the human retina. The following diagram illustrates visual processing
from a colour point of view. Within the diagram R, G and B signify the
photoreceptors for red, green and blue. RGC is the retinal ganglions cell the axon of
which forms part of the optic nerve, BC is the bipolar cell for the cone, HC the
relevant horizontal cell and AC the amacrine cell.
55 of 200

Figure 25 Retinal colour processing (original)
56 of 200

3.5 Human visual processing
Humans are trichromats i.e. the vast majority see in colour by the retina responses to
the colours red, green and blue. However upon leaving the retina via the optic nerve
(composed of 1.2 million nerve fibres) 80% of those fibres are tailored to accept
colour output from the retina in terms of three colour signal ranges historically
termed red, green and blue. The red cones respond maximally to 558nm but cover
the red wavelength (630-700) nm The green cones respond maximally to 531nm but
cover the green wavelength (520-570) nm and also the yellow wavelength of
580nm.The blue cones respond maximally to 420nm but cover the blue wavelength
(450-495) nm. [193] Rods respond maximally at 491nm. Figure_5 illustrates the way
the four colour opponent theory; operating inside the retina; is construed to operate
to achieve the three colour signals (RGB) leaving the retina at the ganglion cell
axons. [12]
Figure 4 depicts two channels of colour information: one channel carrying data
derived from the L (long wavelength (red)) and M (medium wavelength (green))
cones of the retina i.e. red and green signals and one channel derived from the L and
M and S (short wavelength (blue)) cones.
So the red (modulated by green) half channel propagates its signals down each of
the nerve fibres (ganglion cells) tailored to accept red signals and the green
(modulated by red) half channel propagates its signals down each of the nerve fibres
tailored to accept green signals The blue (modulated by `yellow’) half channel
propagates its signals down each of the nerve fibres tailored to accept blue signals.
The remaining half channel propagates a `yellow’ signal to koniocellular ganglion
57 of 200

cells to modulate this luminance pathway signal which is otherwise ON to the blue
cone. [193], [14]
Due to the retinotopic mapping involved in human vision massive (i.e. 120 million
rods, 6 million cones and about 1.2 million ganglion cells) parallel processing is
necessary. This retinotopic mapping means that each pixel by pixel operation is not
separable and must be completed in conjunction with neighbouring pixels in an
image within a time frame of notionally 20 milliseconds; the time between one spike
and another at maximum throughput. An issue arises with transferring information
from the sender chip; built to take in information from the outside world, and the
retinal receiver chip built to deliver a stimulus to the implant. Address Event
Representation (AER) reduces the amount of parallelism required for transfer of
information between the sender chip and the receiver chip. This is possible because
(1) the transfer of information propagated along the red, green, blue and yellow half
channels occurs at the relatively slow rate of between 5 to 50 impulses per second,
and (2) the AER protocol allows for time multiplexing thus reducing the amount of
parallelism needed for transmission of the `sender’ signal to the receiver chip. In
practice this will reduce the physical complexity of the proposed artificial visual
processing. The following work will focus on producing a workable AER protocol
for this purpose.
As the work proposed will concern a small (

microphotodiode pixels are 20 µm x 20 µm with 9 µm x 9 µm electrodes separated
from adjacent pixels by 10 µm channel stops [194]. The result of this
correspondence indicates that the half channel pathways of Figure 4 result in three
outgoing signals (signals leaving the retina) of red, green and blue to the axons of the
ganglion cells forming the optic nerve.
59 of 200

L+ M-
+ -
R-G G-R B-Y Y-B
Note: L(red), M(green) & S(blue).
Red-green pathway
L- and M-cone difference
Blue-yellow pathway
S-cone (difference) with SUM of L- & M-cone
responses
Luminance pathway
L- and M-cone sum
Figure 26 Four Colour Opponent Theory
60 of 200

3.6 Current Retinal Implants
Commonality exists between sub retinal [195, 196] and epi-retinal approaches in
terms of stimulus pulse requirements to be delivered by the implants. The difference
between these two approaches is that the sub retinal implant is meant to substitute
for retinal processing within the outer retina whereas the epi-retinal implant bypasses
both the inner and outer retina and uses signals derived from a camera to produce the
stimulus pulsing.
3.6.1 Retinal structure
There are five major classes of neuron within the retina which can be considered to
lie within the seven layers of the following representation (Figure 1) of a cross
section of the retinal architecture; namely cones and rods, horizontal, amacrine and
retinal ganglion cells (RGCs). The outer retina extends from the pigment epithelium
to the boundary of the inner nuclear layer whilst the inner retina encompasses the
inner nuclear layer, the inner plexiform layer and the ganglion cell layer. The
synaptic layers housing ion channels are the outer plexiform layer (OPL) and the
inner plexiform layer (IPL). A sub retinal implant (section 1.1.1) [197] would
occupy the space of photoreceptors in a healthy retina i.e. adjacent to the choroid
(the retinal pigment epithelium (RPE) lines the choroid), the photodiodes taking the
place of photoreceptors and the stimulatory electrodes impinging at the boundary of
the inner nuclear layer and typically but not exclusively stimulating bipolar cells. An
epi retinal implant (section 1.1.2) [111] is tacked to the inner limiting membrane of
the retina at the axon side of the ganglion layer reached from the inside of the
eyeball.
61 of 200

3.6.2 Retinal operations
Light reaching the photoreceptors (section 3.4) of which there are 126 x 10 6 over the
whole human retina (section 3.4), is transduced into electrical signals synapsing onto
bipolar and horizontal cells. The cones represent a twentieth of the photoreceptors
however their dendritic arbor is typically larger than that of the rods. Bearing in
mind that the optic nerve has only 1.2 million fibres [174] then it can be seen that the
retina is operating as a very efficient spatiotemporal filter. Photoreceptors threshold
and summate; with horizontal cells providing lateral processing, by interacting with
photoreceptor and bipolar cells. Amacrine cells also provide lateral processing in the
inner retina by interacting with bipolar cells and RGCs.
62 of 200

Pigment
epithelium
Outer
segments of
photoreceptors
Outer
nuclear
layer
Cone
Rod
Outer
plexiform
layer
Inner
nuclear
layer
Inner
plexiform
layer
Ganglion
cell layer
Horizontal
cell
Bipolar cell
Amarcrine
cell
RGCs
Parasol RGC Midget RGC Small bistratified RGC
Direction of
light from eye
lens
Figure 27 Retinal Structure
This convergence of the retina, in terms of intensity signals derived from the
photoreceptors is represented at the ganglionar level in terms of the receptive field
concept. In this abstraction the signals arriving at a ganglion cell are represented
either by an ON-OFF configuration when excitatory signals predominate [198] and
depolarisation causes an action potential to be produced and propagated along the
optic nerve fibre and an OFF-ON configuration when this is not the case i.e. where
inhibitory signal predominate (Table 1). The centre in a primate retina is half the size
of the periphery; characterised by spatial frequency, cycles per degree (c/d)
63 of 200

quantified by number of cycles present in the stimulus of a 1º visual angle. [198], the
size referred to here is in terms of the centre radius compared to the peripheral radius
[199]. So for a centre radius (r c ) the surround radius (r s ) will be 2r c giving an area of
centre of πr 2 c and an outer area of π(2r c ) 2 giving an annulus area of 4πr 2 c - πr 2 c i.e. 3
πr 2 c meaning the relationship between centre area and annulus is 3 (Figure 2).
Destination of signals On centre ganglion cell field Off centre ganglion cell field
Converging signals impinging
on surround only
Converging signals impinging
on centre only
No signals reaching centre or
surround
Signals reaching centre and
surround
ganglion cell axon does not fire
ganglion cell axon fires rapidly
ganglion cell axon does not fire
low frequency firing
contentious but an off-bipolar
cell field would fire
contentious but an off-bipolar
cell field would not fire
ganglion cell axon does not fire
low frequency firing
Table 5 Receptive ON/OFF operation
Diamete
r 3.36µm
When r surround equals 2r centre annulus area
equals 6.65µm 2 Receptive field
+
+
Diamete
r 1.68µm
function
64 of 200

Figure 28 Receptive field example
3.6.3 Commonality of sub retinal and epi retinal approaches
In Carver Meads 1990 invited paper “[200]” which refers to Mahowald’s “Silicon
Retina” [201] a description is given of each node in the retina (photoreceptor) and
the hexagonal resistive network between them. The effect of this network is to
compute a spatially weighted average of photoreceptor inputs meaning the output
from the circuit is the difference between the resistive network and that node. So this
sub retinal approach using photodiodes as receptors replicates the centre surround
processing of the vertebrae retina[202]. This centre surround processing is also used
in epi-retinal approaches by using the particular edge detection method described
below to process the received image from an external camera [40, 111, 200, 201,
203, 204]. The receptive field concept is described by Difference of Gaussians
(DOG) which for the centered two-dimensional case is,
( )
( ) ( ) ( ) ( )
(1)
Where x and y are the pixel co-ordinates, σ (standard deviation); pragmatically the
diameter of the centre, and β (space constant) is the ratio of outer diameter to inner
diameter of the receptive field function. To define the difference of Gaussian (DOG)
i.e. f(x,y,σ)from the above description in image processing terms, that part of the
equation relating to the inner circle i.e. the kernel is defined as g inner (x,y,σ) =
( ) ( )
( ) (2)
65 of 200

(Where * is the convolution operator and I refers to the intensity of the point)
Similarly g outer (x,y,βσ) =
( ) ( )
( )
(3)
So DOG = g inner (x,y,σ) - g outer (x,y,βσ)
(4)
Typically finite impulse response filters (FIRs) are used to implement these concepts
within FPGA fabric.
3.6 AER concept in context
AER is a time division multiplexing system, therefore signals are sent in one
complex transmission and the receiving end has to separate the individual signals.
This gives an opportunity to emulate the many wires that would otherwise be
required to transmit the values of pixel intensity of each pixel of a scene. So for a
1024 by 1024 scene: required at the fovea of the eye, 1,048,576 wires would need to
be emulated. In the case of a scene of 32 by 32 the connectivity of 1024 wires would
need to be emulated and for a scene of 4 by 4 the connectivity of 16 wires would
need to be emulated. At 25 fps the time for each frame would be 0.04s (40ms) such
timing would be appropriate for the initial test setup to display on a monitor. A spike
plus interval occurs in 20ms at maximum pulsing and in 200ms for minimal pulsing.
Pulses per second can be determined from interspike time, within this time frame;
see figure 27:
66 of 200

Spike Representation
Looking at the red plane at maximum pulse rate over 1 second
Tspike
(4ms)
Ispike
(16ms:
interspike
interval)
Tplane
Looking at the red plane at minimum pulse rate over 1 second
Tspike
(4ms)
Ispike
(196ms:
interspike
interval)
Tplane
Figure 29 showing fifty pulses/s (representing maximum intensity) and 5 pulses/s
(representing minimum intensity)
3.7 AER data format
The data transmitted through the AER ‘virtual’ wiring will be for the colour signals
of red (R), green (G) and blue (B). The major requirement for transmission of
information is that all information be transferred in 40ms or less; in conventional
terms (T frame ) this is 25 frames per second (fps). For all information to be transferred
in 40ms implies an address-event time (T ae ) for each virtual wire (pixel) to be related
to image size. So in the case of a 32 by 32 test image this time per pixel (T pixel )
would be 0.04s 1024 = 39062.5 ns. AER packet time (T packet ) is synonymous with
T pixel for this project and will encompass address bits with pixel data as payload.
During AER transmission for the three planes of each pixel the nominal time per
plane would be:
67 of 200

T
frame
C * R * M
p
Tpacket
i.e. Tplane
(1) (a)
M
p
Where T frame = MPEG frame time determined from fps of camera
T plane = time for one plane of one packet
T packet = time for one packet
C = number of columns in image
R = number of rows in image
M P = number of planes in message packet
So in this case T plane =
39062.5
3
≈ 13020.8 ns i.e.
0.
04
13020.
8ns
32 32
3
Before and after transmission T plane and T packet are concurrent and in this case the
following adaptation of (1) (a) is used:
T
frame
C* R
T (1); (b)
plane
This means to maintain maximum throughput for one plane of 50 spikes per unit
time implies spike + interval would be 39062.5 50 = 781.25 ns.
T
spike
Tplane
I
spike
(2)
P
p max
Where T spike = time for impulse to complete
I spike = the interspike interval
68 of 200

T plane = time for one plane
P p = number of pulses per plane
Equations (1) (b) and (2) can be combined in the following way:
T
spike
Tpacket
I
spike
(3)
P
p max
So in this case T spike + I spike =
39062.
5
781.
25ns
50
From observation of the preceding spike representation figure it can be seen that for
maximum pulsing:
T pulse = 5*T spike (where T pulse = (interval + T spike ) this gives a factor to be used with
(3) to calculate T spike directly:
T
spike
I
spike
Tpacket
M
p
* Pp
Tpacket
M
p
* Pp
max
max
1
5
4
5
. (4)
Using (3) and (4) would give a spike time of 781.25 ns 5 = 156.25 ns. This
minimum duration spike time, derived at maximum pulsing, will be a constant
spike constant
T for the test scene. In this case thenT spike
= 156.25 ns. Now that
constant
Tspike constant
has been determined for the system the constraint for maximum pulsing can
69 of 200

e relaxed and from (2) the temporal spacing between spikes at any pulsing can be
worked out.
So in this instance for a minimal pulsing of 5 and a maximal pulsing of fifty:
I spike
39062.5 156.
25
5
7812.
5 156.
25
7656.
25ns
I spike
39062.5 156.
25
50
781.
25 156.
25
625ns
Virtual Spike Representation
Looking at the red plane at maximum pulse rate over 39062.5ns
Tspike
(156.25ns)
Ispike
(625ns:
interspike
interval)
Tplane
Tspike
(156.25ns)
Ispike
(7656.25ns :
interspike
interval)
Tplane
Looking at the red plane at minimum pulse rate over 625000ns
Figure 30 50 pulses/s and 5 pulses/s for one plane of virtual wire of test scene
70 of 200

For the three colour planes; R, G, B at maximum throughput (North Sky White)
there will be 150 events or spikes per packet.
E
packet
f
event
(5)
T
packet
Where f event is event frequency
E packet is number of events per packet
T packet is cycle time for message packet
f event
150
0 . 04
3750 events per second
To accommodate the virtual wiring of our test scene this maximum event frequency
per packet must increase by the number of pixels within the image:
1024*
3750 3840000events/second
3.
84 Mevents/second
3.8 Behavioural test image to be used
The test image initial sizing is 32 * 32 pixels consisting of 64 blocks of colour:
71 of 200

one tw o three four five six seven eight
RED GREEN BLUE CYAN MAGENTA YELLOW ORANGE INDIGO
nine ten eleven tw elve thirteen fourteen fifteen sixteen
GREEN RED GREEN BLUE CYAN MAGENTA YELLOW ORANGE
seventeen eighteen nineteen tw enty tw enty_one tw enty_tw o tw enty_three tw enty_four
BLUE GREEN RED GREEN BLUE CYAN MAGENTA YELLOW
tw enty_five tw enty_six tw enty_seven tw enty_eight tw enty_nine thirty thirty_one thirty_tw o
CYAN BLUE GREEN RED GREEN BLUE CYAN MAGENTA
thirty_three thirty_four thirty_five thirty_six thirty_seven thirty_eight thirty_nine forty
MAGENTA CYAN BLUE GREEN RED GREEN BLUE CYAN
forty_one forty_tw o forty_three forty_four foprty_five forty_six forty_seven forty_eight
YELLOW MAGENTA CYAN BLUE GREEN RED GREEN BLUE
forty_nine fifty fifty_one fifty_tw o fifty_three fifty_four fifty_five fifty_six
ORANGE YELLOW MAGENTA CYAN BLUE GREEN RED GREEN
fifty_seven fifty_eight fifty_nine sixty sixty_one sixty_tw o sixty_three sixty_four
INDIGO ORANGE YELLOW MAGENTA CYAN BLUE GREEN RED
Figure 31 1024 pixel behavioural test image
The breakdown of the component primary colours of red (R), green (G) and blue (B)
forming each of these blocks is detailed completely in the figure following.
72 of 200

[1] R = 11111111 [2] R = 000 [3] R = 000 [4] R = 000 [5] R = 11111111 [6] R = 11111111 [7] R = 11111111 [8] R = 01111111
G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 01111111 G = 000
B = 000 B = 000 B = 11111111 B = 11111111 B = 11111111 B = 000 B = 000 B = 01111111
[9] R = 000 [10] R = 11111111 [11] R = 000 [12] R = 000 [13] R = 000 [14] R = 11111111 [15] R = 11111111 [16] R = 11111111
G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 01111111
B = 000 B = 000 B = 000 B = 11111111 B = 11111111 B = 11111111 B = 000 B = 000
[17] R = 000 [18] R = 000 [19] R = 11111111 [20] R = 000 [21] R = 000 [22] R = 000 [23] R = 11111111 [24] R =11111111
G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111
B = 11111111 B = 000 B = 000 B = 000 B = 11111111 B = 11111111 B = 11111111 B = 000
[25] R = 000 [26] R = 000 [27] R = 000 [28] R = 11111111 [29] R = 000 [30] R = 000 [31] R = 000 [32] R = 11111111
G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000
B = 11111111 B = 11111111 B = 000 B = 000 B = 000 B = 11111111 B = 11111111 B = 11111111
[33] R = 11111111 [34] R = 000 [35] R = 000 [36] R = 000 [37] R = 11111111 [38] R = 000 [39] R = 000 [40] R = 000
G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 111111111 G = 000 G = 11111111
B = 11111111 B = 11111111 B = 11111111 B = 000 B = 000 B = 000 B = 11111111 B = 11111111
[41] R = 11111111 [42] R = 11111111 [43] R = 000 [44] R = 000 [45] R = 000 [46] R = 11111111 [47] R = 000 [48] R = 000
G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000
B = 000 B = 11111111 B = 11111111 B = 11111111 B = 000 B = 000 B = 000 B = 11111111
[49] R = 11111111 [50] R = 11111111 [51] R = 11111111 [52] R = 000 [53] R = 000 [54] R = 000 [55] R = 11111111 [56] R = 000
G = 01111111 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111
B = 000 B = 000 B = 11111111 B = 11111111 B = 11111111 B = 000 B = 000 B = 000
[57] R = 01111111 [58] R = 11111111 [59] R = 11111111 [60] R = 11111111 [61] R = 000 [62] R = 000 [63] R = 000 [64] R = 11111111
G = 000 G = 01111111 G = 11111111 G = 000 G = 11111111 G = 000 G = 11111111 G = 000
B = 01111111 B = 000 B = 000 B = 11111111 B = 11111111 B = 11111111 B = 000 B = 000
Figure 32 Colour composition
73 of 200

3.8.1 Other test images
For other behavioural and structural work smaller test images based on subsets of the
32 by 32 test image may be used; in particular for receiver implementation the 4 by 4
test image i.e. 16 pixels will be used.
Red Green Blue Cyan
Magenta Yellow Orange Indigo
Green Red Green Blue
Cyan Magenta Yellow Orange
Figure 33 sixteen pixel image
3.9 Biphasic pulse
In order to ensure zero charge build up on nerve cells and allow chemical
interactions to occur; when the implant is driven by the electronics biphasic pulses
must be formed.[57] The shape of the proposed biphasic pulse for this work; in
comparison to spike representation maximum pulsing, is as shown in figure 32.
74 of 200

Pulse generation is not part of the receiver chip design per se rather it’s the driver
(circuitry driving the electrodes) post processing interface, typically mounted as part
of the retinal implant, typically atop the electrode array. The negative going phase of
the pulse lasts for one quarter of the biphasic pulse time in this case 1ms and the
positive going phase of the pulse again lasts for one quarter of the biphasic pulse
time i.e. 1ms. In this case the biphasic pulse time matches with the 4ms incurred by
the biological action potential shown earlier in this chapter.
Spike plus off time equals the time for the biological action potential; represented by
the biphasic pulse, plus the interspike interval which in the case of maximum pulsing
i.e. full hue intensity, will be 20ms. NB the spike time will never alter only the off
time will vary according to the shade of colour (hue intensity) between 16ms at full
pulsing and 196ms at minimum pulsing.
75 of 200

Tspike
(4ms)
Ispike
(16ms:
interspike
interval)
Tplane
-ve
0ve
Tspike
(4ms)
Spike plus off time
Interphase delay
Ispike
(16ms:
interspike
interval)
Tplane
Figure 34 Biphasic pulse compared to `spike’
3.10 Concluding this chapter
The spike time determined by this analysis of 52.08ns is specific to the behavioural
test scene to be used of 32 by 32 that is an image composed of 1024 pixels. As at
present (10/07/2012) retinal implants are unlikely to cope with much more than 1024
electrodes and achromatic approaches to retinal prostheses are now in clinical trials
[116] this at first sight seems a justifiable picture size to work with. Unfortunately
with three colours there would be an electrode requirement of three times this
amount i.e. 3072 electrodes. An electrode requirement of 3072 electrodes for the
retinal prosthesis; to deal solely with colour is difficult, given the RGC axon size of
76 of 200

the order of 1µm/2.5µm, and the foveal area (2.66mm 2 ) in which the preponderance
of colour sensitive fibres exist. Beyond the foveal area the positioning of electrodes
to interface with colour sensitive RGCs would be very onerous for little extra value
in terms of retinal prosthesis capability. In any case the camera approach in its
present form would also have a limit of a 1024 by 1024 image i.e. 1048576 pixels as
beyond that size there is no longer a one to one correspondence between cones and
ganglion axons via bipolar cells. What is being said here is that beyond that one-toone
correspondence horizontal cells and amacrine cells exert their influence to give
convergence of the receptive fields to the ganglion cells. [205]
77 of 200

INDIGO
ORANGE
YELLOW
MAGENTA
CYAN
BLUE
GREEN
RED
ORANGE
YELLOW
MAGENTA
CYAN
BLUE
GREEN
RED
GREEN
YELLOW
MAGENTA
CYAN
BLUE
GREEN
RED
GREEN
BLUE
MAGENTA
CYAN
BLUE
GREEN
RED
GREEN
BLUE
CYAN
CYAN
BLUE
GREEN
RED
GREEN
BLUE
CYAN
MAGENTA
BLUE
GREEN
RED
GREEN
BLUE
CYAN
MAGENTA
YELLOW
INDIGO
RED
GREEN
BLUE
CYAN
MAGENTA
YELLOW
ORANGE
RED
GREEN
BLUE
CYAN
MAGENTA
YELLOW
ORANGE
INDIGO
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128
129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192
193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224
225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256
257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288
289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320
321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352
353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384
385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416
417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448
449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480
481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512
513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544
545 546 547 548 549 550 551 552 553 554 545 546 547 548 549 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576
577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608
609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640
641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672
673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704
705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736
737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768
769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800
801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832
833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864
865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 890 891 892 893 894 895 896 897 898 899 890 891 892 893 894 895 896
897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928
929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960
961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992
993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024
Figure 35 Correspondence of pixels to colour
78 of 200

Chapter 4 Sender chip
The purpose of the sender chip is to convert an incoming signal from its
conventional representation to an AER format suitable for transmission to the
receiver chip. For the purposes of this work the incoming signal will be in the form
of the picture data held in a ROM. This picture data initially being in the form of a
test image of 32 by 32 i.e. 1024 pixel count, this being later revised to a smaller test
image for implementation. Eventually this source will be replaced by a direct camera
input as indicated by the following diagram, ideally the camera would output in
RGB format to alleviate pre-processing from other formats such as YUV.
Test scene
(from camera)
SENDER - Accept, and
process scene into AER
protocol
AER
Transmission
lines
RECEIVER
chip (routes
signals to
implants)
Figure 36; Envisaged System

4.1 Sender concepts
From chapter one we know that the maximum address event time for an AER packet
representing the pixel of a 1024 pixel count image can be 39062.5ns therefore the
full image would take 1024*39062.5 = 40000000ns i.e. 40ms. The figure calculated
for spike time (T spike ) on this basis was 156.25ns and for T pulse was 781.25ns i.e. 5*
T spike .
The AER packet time (T packet ) aka T pixel can be shortened at the expense of increased
fps the chart (taken from fps_calcs.xls) demonstrates this relationship.
45000
40000
35000
30000
25000
Tpacket
20000
fps
Tae (ns)
15000
10000
5000
0
25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100
fps
Figure 37 T ae versus fps

E.g. choosing a frame rate of 100 gives a spike time of 39.0625ns and a partial spike
time of 9.7656.25 ns from a subset of the table (fps_calcs.xls) used to construct the
above chart.
fps T f T ae (@1024 T ae (ns) T long_pulse T spike_pulse T partial_spike
f pixel
25 0.04 3.906E-05 39062.5 781.25 156.25 39.0625 25600
50 0.02 1.953E-05 19531.25 390.625 78.125 19.53125 51200
75 0.013333 1.302E-05 13020.83 260.4167 52.08333 13.02083 76800
100 0.01 9.766E-06 9765.625 195.3125 39.0625 9.765625 102400
Table 6 T spike related to fps
The partial spike time refers to the quarter component part of the biphasic pulse
which can be accommodated using the clock pulse (10ns) of a typical FPGA running
at 100MHz. For this chapter, dealing with AER stream production, we could put this
consideration on hold and typify the AER sender chip as operating at circa 25MHz
with an address event timing of T ae = 39062.5 ns and a spike time of 156.25 ns.
For one pixel of an image at maximum intensity there will be fifty `outerpulses’ i.e.
the actual spike plus the time before another one is allowed; or makes sense. As this
pixel information must be transmitted in one second; each outerpulse, in real time,
must take 0.02s. So in conventional terms for a frame rate of 50fps then each pixel,
during propagation, experiences one outerpulse per frame albeit at faster than `real
time’. As biologically the innerpulse (comprising `spike’ plus recovery time) takes
4ms (0.004s) this implies an `off’ time; at maximum intensity of (0.02-0.004) equals
0.016s, again in `real time’, as has been previously stated. Using an example of a
1024 pixel image at 50fps as each frame takes 0.02s then the serialised time for a

pixel must equate to 0.02/1024 i.e. 0.00001953125s = 19531.25ns as shown in table
5.
Example Hierarchy
T periodic (ns) T periodic (s) frequency (HZ) frequency (MHz)
[1] Circa 25MHz 40 0.00000004 25000000 25 partial_spike
[2] Circa 6.25MHz 160 0.00000016 6250000 6.25 spike
[3] Circa 1.25MHz 800 0.0000008 1250000 1.25 outer_ pulse
[4] Circa 25.6KHz 39062.5 3.9063E-05 25600 0.0256 pixel
2nd Example
Hierarchy
[1] Circa 25000MHz 0.04 4E-11 25000000000 25000 partial_spike
[2] Circa 6250MHz 0.16 1.6E-10 6250000000 6250 spike
[3] Circa 1250MHz 0.8 8E-10 1250000000 1250 outer_ pulse
[4] Circa 25MHz 40 0.00000004 25000000 25 pixel
3rd Example
Hierarchy
[1]Circa 100MHz 10 0.00000001 100000000 100 partial_spike
[2]Circa 25MHz 40 0.00000004 25000000 25 spike
[3]Circa 5MHz 200 0.0000002 5000000 5 outer_ pulse
[4]Circa 102.4KHz 9765.625 9.7656E-06 102400 0.1024 pixel
Table 7 examples of clocking hierarchy
The diagrams illustrate the component parts for construction of the sender chip. The
first following top level block diagram illustrates the component parts for
construction of the sender chip in high level programming terms and the second in
Xilinx FPGA conceptual terms and thirdly in
a VHDL programming
description[Appendix C].

Image (frame buffer)
Derive pulse
count from
hue intensity
AER stream
(incorporate
addresses)
Output AER
stream
Figure 38 forming AER stream
AER
Pkt.
Node
Pulses
Sender (FPGA)
MV-D640 SERIES
(Photonfocus)
Ctrl/Supervision
Camera ctrl
+
framegrabber
RGB
to
`spikes’ (with
address)
AER
Interface
AER
Packet =
Pixel
(spikes)
RAM
control
Into SRAM
RAM
Figure 39 Forming AER stream

Clocking hierarchy
(32 by 32 image)
FPGA
(incoming)
e.g.
100MHz
Data flow components
Stored
image
(frame)
Partial_spike
_clock e.g.
25MHz
Convert
intensity to
pulse_count
Spike_clock
(25÷4)MHz
Form AER
stream
Pixel_clock
(25÷1000)MHz
NB
partial_spike_clock
frequency is
determined by
image size
Output
AER
stream
R G B
Figure 40 sender program overview

4.2 Sender AER format
Each AER packet; representing a pixel, contains address bits followed by a pulse
count for each plane of colour within the pixel. An example of the first row of a 32
by 32 image is shown in the following table of values. The first ten bits are the
address bits and the remaining 18 bits represent the data containing colour
information in the form of a pulse count.
0000000001 110010000000000000
0000000010 110010000000000000
0000000011 110010000000000000
0000000100 110010000000000000
0000000101 000000110010000000
0000000110 000000110010000000
0000000111 000000110010000000
0000001000 000000110010000000
0000001001 000000000000110010
0000001010 000000000000110010
0000001011 000000000000110010
0000001100 000000000000110010
0000001101 000000110010110010
0000001110 000000110010110010
0000001111 000000110010110010
0000010000 000000110010110010
0000010001 110010000000110010
0000010010 110010000000110010
0000010011 110010000000110010
0000010100 110010000000110010
0000010101 110010110010000000
0000010110 110010110010000000
0000010111 110010110010000000
0000011000 110010110010000000
0000011001 110010011001000000
0000011010 110010011001000000
0000011011 110010011001000000
0000011100 110010011001000000
0000011101 011001000000011001
0000011110 011001000000011001
0000011111 011001000000011001
0000100000 011001000000011001
red
green
blue
cyan
magenta
yellow
orange
Indigo
Table 8 short AER format
In the example shown there are ten address bits and eighteen bits for the AER
`payload’ this payload will always be eighteen bits regardless of image size whereas
85 of 200

the number of address bits will alter with image size. The number of address bits
required for various image sizes can be calculated from the following formula; where
the answer obtained is rounded up to the nearest whole number.
Log (pixel_count)/log2
side of square
address bits
log
image
10 pixel_count
required
Total bit length
2 0.60206 4 2 20
4 1.20412 16 4 22
8 1.80618 64 6 24
16 2.40824 256 8 26
32 3.0103 1024 10 28
1024 6.0206 1048576 20 38
Table 9 example correspondences
4.2.1 Alternative (discounted) sender AER format
An alternative robust AER format can be constructed whereby the bitstream consists
of ten bits of address (32 by 32 image) followed by a payload of 150 bits of which
the first 50 bits encode red plane information; the second 50 bits encode green plane
information and finally 50 bits of blue plane information. Such a format would be
more error tolerant in terms of loss of bits within the payload e.g. a two bit error
would be negligible but could only be achieved if a longer bit stream could be
tolerated. As this format will be integral to the receiver chip design it can justifiably
be used as part of the test setup to demonstrate that such a format can be output to a
monitor to display the picture expected to be formed. Programming components (or
extracts thereof) will be included in appendix B of this thesis.
86 of 200

4.3 Test setup
The structural program for a test setup to demonstrate the display of an AER
formatted picture consists of the following components:
Image in
AER format
Convert AER
stream to
pulse count
VGA module
DVI interface
Convert
pulse count
to hue
intensity
Display
monitor
Figure 41 Display of AER test image
The purpose of the test setup is to display on a monitor the expected image to be
presented by the retinal implants and perceived by the patient.
4.4 Sender chip reports
The Xilinx software used for FPGA behavioural simulations and structural
implementation has the facility to produce detailed reports at various stages of the
design. The following detailed reports can be obtained: (1) Synthesis report, (2)
87 of 200

Translation report, (3) Map report, (4) Place and route report, (5) Post-PAR Static
Timing Report, (6) Power Report and (7) Bitgen report [Appendix E]
4.4.1 Power Analysis
Xilinx FPGAs have a power analyser facility example screenprints are shown here
for power analysis after the implementation of the sender chip within the FPGA
fabric. The quiescent power is 0.60008w, dynamic power is 0.03844w (38.4mw).
Figure 42 sender chip power analyser screenprint
4.5 Sender schematic
The Register Transfer Level (RTL) schematic (figure 43) illustrates the derivation of
the clocks required for the programming blocks from the Digital Clock Multiplier
(DCM) supplied as intellectual property (IP) within the FPGA fabric. Using the
DCM output frequency the partial_spike_clock frequency is derived the “out_clock”
signal and hence spikes (spike_clock) and pixel frequency (pixel_clock). At 25fps
88 of 200

for a 16 pixel image held in memory (mem_of_16_stream) then partial_spike_clock
frequency is 400kHz i.e.1000 times the required pixel frequency of 400Hz. Spike
frequency (spike_clock) is 250 times the required pixel frequency of 400Hz i.e.
100kHz. At 5fps for a 16 pixel image then using the same logic clock frequencies
would reduce to a fifth of these values.
89 of 200

90 of 200
CLKIN_IN
RST_IN
pixelclock_dcm
CLKDV_OUT
LOCKED_OUT
CLKO_OUT
CLKIN_IBUFG_
OUT
P
VCC
XST_VCC
PB_ENTER
USER_CLK
addra(3:0)
mem_of_16_stream
clka
Inst_mem_of_16_stream
douta(23:0)
Inst_pixelclock_dcm
value(7:0)
pc_clk
pulsar
Inst_pulsar_blue
ZZ(7:0)
B_count(7:0)
G_count(7:0)
R_count(7:0)
four_bit_addr
ess(3:0)
pc_clk
pc_stream(21:0)
pulse_count_to_pc_stream
Inst_pulse_count_to_pc_stream
IO_L14P_11(21:0)
get_pixel_clock_from_partial_spike
_clock
partial_spike_clock
pixel_clock
out_clock
received_clock
Inst_get_pixel_clock_from_partial
_spike_clock
get_partial_clock_from_incoming
_clock
Inst_get_partial_clock_from
_incoming_clock
Inst_get_spike_on_from_partial_spike_clock
partial_spike_clock
spike_clock
get_spike_on_from_partial_spike_clock
address1
port_data
Q(3:0)
COUNTER:1
COUNT
up
pulsar
pc_clk
ZZ(7:0)
Inst_pulsar_red
pulsar
Inst_pulsar_green
ZZ(7:0)
value(7:0)
pc_clk
Clk
value(7:0)
Figure 43 sender_rtl_16

In the Xilinx Virtex 5 the DCM has the following possible divisions from either
400MHz or 100MHz: 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 9, 10, 11, 12,
13, 14, 1`5 and 16. So one possible combination to obtain 400kHz as the `out_clock’
value would derive 40MHz as the output from the DCM and divide this by 100 in
the programming block labeled “get_partial_clock_from_incoming_clock”. At 5fps
for the same image to obtain 80kHz as the out_clock derive 20MHz as the output
from the DCM and divide this by 250.
In the block “get_spike_on_from_partial_spike_clock” divide by four to get the
spike_clock frequency for all fps e.g. @25fps this would be100kHz.
In the block get pixel_clock_from_partial_spike_clock divide by one thousand to get
the pixel frequency for all fps e.g. @25fps this would be 400Hz
The three instance of the “pulsar” block convert conventional 24 bit colour measure
to a pulse count (described earlier), over time, for each of the three colour planes:
red, green and blue.
The counter counts and outputs the address bits for each pixel as it occurs.
Finally “pulse_count_to_pc_stream” time multiplexes and outputs the AER stream
consisting of AER packets with a payload of 18 bits. The first six bits of the payload
represent the colour intensity for the red plane, followed by six bits for green and
blue planes respectively.
4.6 Chapter summary
The size of the image in pixel count (to be transmitted) will be limited by the number
of electrodes possible at the retinal implant and presently (2010) this is expected to
rise to over 1000 within several years. For the sender chip; programmes were
completed both behaviourally and structurally i.e. synthesised and implemented for
image sizes ranging from 16 pixels (4 by 4) up to 1024 pixels (32 by 32). To
91 of 200

incorporate colour planes for each pixel the electrode requirement, when received by
the receiver chip, will increase for each of those image sizes; shown in Table 9.
In essence the sender chip converts the colour image obtained from a commercial,
off-the-shelf (COTS) camera to an AER stream. AER packet bit length will increase
with image size as shown in the charts, following table 9 (source: aer_compare.xls).
pix_count x 3
4 12
8 24
16 48
32 96
64 192
128 384
256 768
512 1536
1024 3072
Table 10 electrode requirement
1200
1000
800
600
400
Total bit length
pixel count
200
0
20 22 24 26 28
Figure 44 upto 1024 image i.e. 32 by 32
92 of 200

1200000
1000000
800000
600000
400000
Total bit length
pixel count
200000
0
20 22 24 26 28 38
Figure 45 (1048576) image limit i.e. 1024 by 1024
93 of 200

Chapter 5 Receiver chip
The following block diagram indicates the components for the receiver chip
components; programming code extracts will be included in appendix D of this
thesis.
Incoming (`short’)
AER_stream
Produce colour
data streams
Convert to `long’
format
Stimulus
generator
Produce output
streams
Form biphasic
pulses and
stimulate
implant
Figure 46 Receiver block diagram
5.1 Clocking calculations (sender)
For one `frame’ at 25fps; of a 32 by 32 image, an entire AER stream of 1024 packets
must occur for delivery at the implant. As each `frame’ takes 0.04s each packet must
take at most 0.04/1024 = 0.0000390625s i.e. 39062.5ns so the frequency of each
packet (T packet ) is 25.6 kHz. Within each packet there are potentially 250 spikes
although only 50 are expected, this being the case the periodic time to be allowed for
each spike to occur will be 39062.5ns/250 = 156.25ns and hence f spike (frequency of
the spike clock) will be the reciprocal of 156.25ns i.e. 6.4MHz. Finally the periodic
94 of 200

time for T partial_spike ; nominally the `quarter component of a biphasic pulse, is
156.25ns/4 = 39.0625ns (a thousandth of T packet ) giving a frequency of 25.6MHz.
f packet T spike
f spike (f biphasic )
fps T frame row col pix_count T packet (ns) (kHz) (T biphasic )(ns) (MHz)
25 0.04 32 32 1024 39062.5 25600 156.25 6.4
25 0.04 16 16 256 156250 6400 625 1.6
25 0.04 8 8 64 625000 1600 2500 0.4
25 0.04 4 4 16 2500000 400 10000 0.1
25 0.04 2 2 4 10000000 100 40000 0.025
50 0.02 32 32 1024 19531.25 51200 78.125 12.8
50 0.02 16 16 256 78125 12800 312.5 3.2
50 0.02 8 8 64 312500 3200 1250 0.8
50 0.02 4 4 16 1250000 800 5000 0.2
50 0.02 2 2 4 5000000 200 20000 0.05
100 0.01 32 32 1024 9765.625 102400 39.0625 25.6
100 0.01 16 16 256 39062.5 25600 156.25 6.4
100 0.01 8 8 64 156250 6400 625 1.6
100 0.01 4 4 16 625000 1600 2500 0.4
100 0.01 2 2 4 2500000 400 10000 0.1
Table 11 clocking calculations
The following table (taken from “fps_calcs.xls”) illustrates these calculations:
Circa 25MHz f f(kHz) f(MHz) T periodic T periodic(ns) T periodic(µ)
T aer_stream 25 0.025 0.000025 0.04 40000000 40000
3.90625E-
T pixel 25600 25.6 0.0256 05 39062.5 39.0625
T spike 6400000 6400 6.4 1.5625E-07 156.25 0.15625
3.90625E-
T partial_spike circa 40ns 25600000 25600 25.6 08 39.0625 0.0390625
The following sub-table either reads each frame four times or allows for 100fps;
Circa 100MHz f f(kHz) f(MHz) T periodic T periodic(ns) T periodic(µ)
T aer_stream 100 0.1 0.0001 0.01 10000000 10000
T pixel 102400 102.4 0.1024
9.76563E-
06 9765.625 9.765625
T spike 25600000 25600 25.6
3.90625E-
08 39.0625 0.0390625
T partial_spike circa 10ns 102400000 102400 102.4
9.76563E-
09 9.765625 0.009765625
Using a 100MHz clock and showing the effect on T partial_spike !
Exactly 100MHz f f(kHz) f(MHz) T periodic T periodic(ns) T periodic(µ)
T aer_stream 97.65625 0.0976563 9.77E-05 0.01024 10240000 10240
T pixel 100000 100 0.1 0.00001 10000 10
T spike 25000000 25000 25 0.00000004 40 0.04
T partial_spike exactly
10ns 100000000 100000 100 0.00000001 10 0.01
Table 12 simulation clocking
95 of 200

5.2 Power Analysis
Xilinx FPGAs have a power analyser facility an example screenprint is shown here
for power analysis after the implementation of a 16 pixel (colour) receiver chip
within FPGA fabric. The quiescent power is 0.60006w, dynamic power is 0.03772w
(37.72mw).
Figure 47 receiver power analysis
96 of 200

5.3 Receiver schematics
In figure 47 the first block: “pulse_count_to_AER_stream receives from the sender
chip the AER stream formatted as address bits followed by a payload of 18 bits of
colour data. The 16 pixel image to be dealt with has 4 address bits associated with
the payload. This block converts the payload from a pulse count to a string of 50 bits
for each colour plane, retains the pixel address and outputs a 154 bit stream.
The second block “preform_for_electrodes” separates the planes of colour into three
separate streams with each plane of colour retaining the pixel address with which it
is associated, giving the potential for each plane of colour to be addressed separately.
The third block (produce_colour_data_streams) strips out the address bits and
continues the processing sequentially on each plane of colour data.
Block four (fully_extend_data_streams) packs the colour data into pixel length
streams to be fed into the routing stage. Block five, the routing stage
(produce_wire_outputs) has a twofold function (1) it routes the pixel streams to the
relevant inputs to the electrodes driving circuitry (2) it allows routing adjustments to
be made where necessary within the software environment.
97 of 200

98 of 200
produce_wire
_outputs
Inst_produce_
wire_outputs
blue(999:0)
green(999:0)
red(999:0)
extended_blue
(999:0)
extended_red
(999:0)
extended_green
(999:0)
blue_data
_incoming
(49:0)
fully_extend_
data_streams
Inst_fully_extend_data_
streams
green_data
_incoming(
49:0)
red_data_
incoming(
49:0)
pulse_count_to_
AER_stream
blue_data
_out(49:0)
green_data
_out(49:0)
red_data_
out(49:0)
produce_colour
_data_streams
B_pulse_
count(5:0)
R_pulse_
count(5:0)
single_address
(3::0)
pcs_clock
G_pulse_
count(5:0)
Inst_pulse_count_to
_AER_stream
XST_GND
g
gnd
Inst_produce_colour
_data_streams
preform_for_
electrodes
Inst_preform_
for_electrodes
blue
(57:0)
green
(57:0)
red
(57:0)
blue_with
_address
(57:0)
green_with
_address
(57:0)
red_with_
address
(57:0)
this_spike_
clock
spike_data
_clock
incoming
_stream(
153:0)
AER_RGB_pulse
_stream(153:0)
preform
_clock
blue_1(3:0)
blue_16(3:0)
blue_14(3:0)
blue_13(3:0)
blue_12(3:0)
blue_11(3:0)
blue_10(3:0)
blue_9(3:0)
blue_8(3:0)
blue_7(3:0)
blue_6(3:0)
blue_5(3:0)
blue_4(3:0)
blue_3(3:0)
blue_2(3:0)
blue_15(3:0)
green_16(3:0)
green_14(3:0)
green_13(3:0)
green_12(3:0)
green_11(3:0)
green_10(3:0)
green_9(3:0)
green_8(3:0)
green_7(3:0)
green_6(3:0)
green_5(3:0)
green_4(3:0)
green_3(3:0)
green_2(3:0)
green_15(3:0)
red_16(3:0)
red_14(3:0)
red_13(3:0)
red_12(3:0)
red_11(3:0)
red_10(3:0)
red_9(3:0)
red_8(3:0)
red_7(3:0)
red_6(3:0)
red_5(3:0)
red_4(3:0)
red_3(3:0)
red_2(3:0)
red_15(3:0)
green_1(3:0)
red_1(3:0)
IO_L15P_15(3:0)
IO_L15N_15(3:0)
IO_L16P_15(3:0)
IO_L16N_15(3:0)
IO_L17P_15(3:0)
IO_L17N_15(3:0)
IO_L18P_15(3:0)
IO_L18N_15(3:0)
IO_L19P_15(3:0)
IO_L19N_15(3:0)
IO_L13P_17(3:0)
IO_L13N_17(3:0)
IO_L14P_17(3:0)
IO_L15P_17(3:0)
IO_L15N_17(3:0)
IO_L16P_17(3:0)
IO_L13P_15(3:0)
IO_L19N_13(3:0)
IO_L19P_13(3:0)
IO_L18N_13(3:0)
IO_L18P_13(3:0)
IO_L17N_13(3:0)
IO_L17P_13(3:0)
IO_L16N_13(3:0)
IO_L16P_13(3:0)
IO_L15N_13(3:0)
IO_L15P_13(3:0)
IO_L14P_13(3:0)
IO_L13N_13(3:0)
IO_L13P_13(3:0)
IO_L13N_15(3:0)
IO_L18N_12(3:0)
IO_L18P_12(3:0)
IO_L17N_12(3:0)
IO_L17P_12(3:0)
IO_L16N_12(3:0)
IO_L16P_12(3:0)
IO_L15N_12(3:0)
IO_L15P_12(3:0)
IO_L14P_12(3:0)
IO_L13N_12(3:0)
IO_L13P_12(3:0)
IO_L14P_11(3:0)
IO_L13N_11(3:0)
IO_L13P_11(3:0)
IO_L14P_15(3:0)
IO_L19P_12(3:0)
IO_L19N_12(3:0)
CLKIN_IN
RST_IN
pixelclock_dcm
CLKDV_OUT
LOCKED_OUT
CLKO_OUT
CLKIN_IBUFG_
OUT
P
VCC
XST_VCC
USER_
CLK
Inst_pixelclock_dcm
get_spike_on_from_partial
_spike_clock
partial_spike_clock
spike_clock
biphasic
_clock
Inst_get_spike_on_from_partial_
spike_clock
get_pixel_clock_from_partial_spike
_clock
partial_spike_clock
pixel_clock
wire_
clock
out_clock
get_partial_clock_from
_incoming_clock
Inst_get_partial_clock_from
_incoming_clock
received_clock
Figure 48 receiver rtl_16

5.4 Clocking calculations (receiver)
The following table (Table 12) indicates the relationships between frequencies where
the assumption is made that the camera frame rate is 1fps. The frequency at which
the system would then be set to run is denoted by f part which will be each component
part of the biphasic pulse forming the representation of the biological spike. In real
time each spike lasts 4ms and therefore each component of the biphasic spike will
last 1ms. That is the 1ms of the biphasic pulse represents an `on’ pulse of negative
polarity, 2ms of zero polarity and 1ms representing an `on’ pulse of positive polarity.
Derived from f part is f spike i.e. spiking time and finally f packet which is the frequency of
the AER packet i.e. pixel frequency. For completeness the periodic time for these
frequencies is also shown in the table.
fps T frame pix_count T packet T part
T spike
(T biphasic )(ns)
f spike
(f biphasic )
(Hz)
f part {Hz)
f packet
(Hz)
1 1 1024 0.00097656 9.76563E-07 3.90625E-06 256000 1024000 1024
1 1 256 0.00390625 3.90625E-06 0.000015625 64000 256000 256
1 1 64 0.015625 0.000015625 0.0000625 16000 64000 64
1 1 16 0.0625 0.0000625 0.00025 4000 16000 16
1 1 4 0.25 0.00025 0.001 1000 4000 4
Table 13 receiver calculations
5.5 Programming components descriptions
These subsections describe the following component parts: `Incoming(short) AER
stream’, `Production of colour data streams’, `Convert to `long’ format’, `Production
of output streams’ and finally `Formation of biphasic pulses prior to delivery to the
retinal implant’.
99 of 200

5.5.1 Incoming (short) AER stream
The `short’ AER stream for a 16 pixel image consists of 8 address bits and 18 bits of
colour plane information. This initial component (preform_for_electrodes) simply
produces 8 address bits and 6 bits of colour information - this encoding of the integer
range 0..63 holds the pulse count of up to 50 pulses for maximum intensity; for each
plane of colour.
5.5.2 Production of colour data streams
The pulse count for each plane is now converted (produce_colour_data_streams) to a
50 bit representation whereby each bit of value one is an active pulse and of value
zero inactive i.e. no pulse. At this stage of the processing, within the FPGA fabric,
the value of one means that the next stage of the processing will recognise that an
innerpulse aka biphasic pulse representing a spike aka, an action potential in
biological terminology, occurs within an outerpulse.
5.5.3 Convert to long format
For the red, green and blue planes the colour data stream held in fifty bits is now
converted so that each outerpulse is represented by twenty bits where four bits
represent the `spike’ aka as the innerpulse and the remaining 16 bits represent the off
time of the outerpulse. Therefore the information for a full pixel for each plane is
held in 1000 bits.
5.5.4 Production of output streams
Using a case statement enables 250 blocks of four bits to be sent to each of the
outgoing wires (sixteen for each colour) equally spaced over a second where those
100 of 200

four bits represent a spike. The active spike aka biphasic pulse is represented by
1001 i.e. where the first bit will be a negative going pulse and the fourth a positive
going pulse. No pulse occurring is 0000. As there are only 50 active spikes allowed
to occur within a second due to an `off time’ being associated with each one then any
change in colour intensity will only be seen after this off time.
5.6 Summary of FPGA resource utilisation
Number of Slice Registers 102 28800 0%
Number of Slice LUTs 119 28800 0%
Number of fully used LUT-FF pairs 100 121 82%
Number of bonded IOBs 193 480 40%
Number of BUFG/BUFGCTRLs 3 32 9%
Number of DCM_ADVs 1 12 8%
Table 14 Receiver chip FPGA resource data
5.7 Post processing for electrodes
The biphasic pulse represented in the output stream by “1001” requires some post
processing enabling the first bit to represent a negative going current pulse to the
implant then after two biphasic clock pulses a positive going current pulse to be
delivered to the implant. Although this is a parallel bus topology at this point the
address bits have been expended within the receiver chip and hence each output wire
can deliver stimulation data directly to the digital to analogue converter circuitry
which in turn feeds each stimulation site of the implant. The following block
diagram illustrates this process.
101 of 200

Receiver
Red
plane
r1
DAC
(4-bit)
To
corresponding
electrodes
Where a
pulse exists,
zero
otherwise
r2
DAC
(4-bit)
r3
DAC
(4-bit)
r4
DAC
(4-bit)
r5
DAC
(4-bit)
r6
DAC
(4-bit)
r7
DAC
(4-bit)
r8
DAC
(4-bit)
r9
DAC
(4-bit)
r10
DAC
(4-bit)
r11
DAC
(4-bit)
r12
DAC
(4-bit)
r13
DAC
(4-bit)
r14
DAC
(4-bit)
r15
DAC
(4-bit)
r16
DAC
(4-bit)
Figure 49 Outputting to electrodes
102 of 200

5.8 Stimulator Retinal Interface
In the sub_retinal approach the electrode array is positioned, replacing damaged
photoreceptors, such that the electrodes impinge into the INL and stimulate bipolar
cells. In the epiretinal approach the electrode array is tacked onto the innermost layer
of the retina and the electrodes stimulate the axons or soma of the ganglion cells; the
stimulator circuitry is atop of the electrode array. The distinctions between these two
approaches were being formed in the 1990’s when several major projects in the
USA, Germany and elsewhere were on-going.
Early electrode array formulations [120, 206] were housed on thin flexible polyimide
strips; e.g. 4µm thick, on which at one end was housed the stimulator circuitry and at
the other the electrodes to be held to the retina (≈250µm thick). In 1999 [207]
Humayun et al, in their investigation of planar disc electrodes of > 125µm noted a
dramatic increase in current requirement, when the distance between a stimulating
electrode and the retina was more than 0.5mm (500µm); thus highlighting the
importance of this particular parameter.
5.8.1 Factors affecting current requirement for stimulation
Electrode positioning; in terms of distance [52] from electrode to point of
stimulation e.g. 30µm [49] , electrode geometry; in terms of size and shape,
electrode material in terms of biocompatibility and charge delivery capability.
Electrode arrays will also take into consideration number of electrodes, electrode
spacing and number of electrodes actually to be or being used. In the case of a
sub_retinal array [122, 174] typically the number of electrodes will match up to the
number of photodiodes as each photodiode will be connected to the electrode
designed ostensibly to stimulate bipolar cells. In the case of an epiretinal array it was
realised a decade ago (2002) [174] that to achieve safe current density with smaller
electrodes it would be necessary to use penetrating electrodes rather than planar
electrodes; as this geometry would allow a reduced distance to target cells that are
embedded in ganglion cell layers of 20-40µm within nerve fibre layers of 20-200µm.
Electrode materials used for stimulation are titanium nitride, platinum, and iridium
oxide [208], which have good charge carrying capacity.
103 of 200

5.8.2 Electrode size and positioning in current retinal
approaches
The diameter of implanted electrodes in current retinal approaches [88, 103, 209,
210] previous to 2007 has been of an order of magnitude of > 50µm. During 2007 it
was determined that a change in electrode-retina distance of 100µm can result in a
ten times change in threshold stimulation and also early results in the Second Sight
device (epiretinal) in Humans showed an inverse square relationship between
impedance and retina-electrode distance [210]. Clinical trials also began on the
Second Sight Second-Generation implant at this time (www.2-sight.com) an earlier
(2006) subretinal device [115] by Retina Implant GmbH and an a second clinical
trial of an epiretinal device by Intelligent Medical Implants AG. In this reference it
was pointed out that as an individual with 20/20 vision can resolve differences of
1/60 of one degree of visual angle that this would translate to 5µm on the retina
implying that 5µm electrodes would be needed for vision at this level. An estimate
was also that a 100µm electrode could provide vision equivalent to 20/400 acuity. In
2008 [176] the EPI-RET-3 prosthesis subsequently had 3D-electrodes with a
diameter of 100µm and a height of 25µm.
5.8.3 Recent developments
2010 to now (2012) has seen a resurgence of work in this area; subretinally [23] the
use of a parylene-based 3-D microelectrode array (MEA) demonstrated in vitro and
in vivo how distance between the stimulating electrode (Ti/Au/Pt) and targeted cells
could be decreased by the use of the tip shaped electrodes as opposed to nearly flat
planar electrodes. Epiretinally [102] an in vivo study indicates smaller current
thresholds; ≈100µA than previous thresholds; 500µA. Also [97] the possibility of
lower stimulus voltage for 3-D electrodes as opposed to flat planar electrodes. The
latest penetrating array for epiretinal prosthesis [29] makes possible high density
arrays (≈100 electrodes/mm 2 ) and arrays of 2µm, 5µm, 10µm, 20µm and 30µm
diameter with heights of 60µm-100µm and pitches of 50µm-400µm.
104 of 200

Chapter 6 Concluding chapter
This chapter presents the conclusions and results of this research and future
recommendations for work to be done both in this school and others.
6.1 Conclusions and results
The paper by Professor M. Humayun et al [171]describes the great promise of a
greyscale retinal implant of 16 pixels and 16 electrodes. The implementation
described in this work describes a structural colour version involving 16 pixels and
48 electrodes wherein 16 electrodes are used to represent each of the three colour
planes of red, green and blue. Also behaviourally programming has been done for
upto 1024 pixels i.e. a 32 by 32 image this would imply 3072 electrodes and for a 16
by 16 image i.e. 256 pixels implying 768 electrodes. Similarly for an 8 by 8 image
i.e. 64 pixels implying 192 electrodes. As the present (2012) limit for the wiring to
those electrodes is 256 wires through a 5mm incision made in the eyeball; use of an
8 by 8 image would leave (256-192) leaving 64 wires `spare’ for an 8 by 8 image.
The reasoning for the use of three electrodes for each pixel is down to retinotopic
mapping, [160]an established principle in neuroscience. In practice this implies that
the topography of mapping is directly related to geometric principles. In other words
there is typically a consistent grouping of ganglion nerve fibres such that each group
of three correctly spaced axons will expect to propagate red(R), green (G) and blue
(B) colour signals. So in the case of a fully functioning retina there is a one to one
correspondence between a red cone and a `red’ ganglion cell, a green cone and a
`green’ ganglion cell and a blue cone and a `blue’ ganglion cell. This being the case
by correctly forming the retinal implant to adhere to the retinotopic mapping then the
105 of 200

sequence of wiring from the receiver chip to the retinal implant should be red, green,
blue etc.
Prior to actual connection to an implant all behavioral programmes created can be
ran to produce files of written output to confirm signals are as to be expected. These
written files can be read with MATLAB software to display the expected produced
picture. The fully implemented 16 pixel image outputs its signals on the dedicated
input/output pins of the Virtex 5 FPGA 64 pin header.
6.2 Implementation of design
When the receiver discussed in the previous chapter has produced the pulse
frequency modulated (PFM) digital signal it goes to the digital to analogue converter
(DAC) within the electrode driver (stimulator) circuitry atop of the retinal array. In
this proposed design the incoming signal feeds into a shift register composed of four
flip flops. This circuit (see figure 50) allows data which appears on the data bus for
several milliseconds to be held at the output indefinitely,
output 1 output 2 output 3
output 4
Data
in
D
Q
D
Q
D
Q
D
Q
1
2
3
4
FPGA
clock
input
Figure 50 Shift (reference) register
106 of 200

So in the application of the D type flip flop as a shift register, this is used to store
number of bits which are entered into the register (or group of flip flops) in a serial
fashion. Data is presented a bit at a time to input D of flip flop 1. After a clock pulse
the data at D2 appears at output 2. Whilst the new data which had been at D1 is now
at Q 1 (or output 1). This process is repeated until this initial register is full. The data
is transferred out in parallel from outputs 1 to 4 for DAC operation. The forming of
the control signal; by masking, to enable operation of the DAC is shown in figure 51
(Enable DAC) and the Boolean table in Table15
b1
b2
b3
b4
Figure 51 Enable DAC
b1 b2 b3 b4 Output
0 0 0 0 0
1 0 0 1 1
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
Table 15 Control signals to enable DAC
6.2.1 DAC operation
To operate a unipolar converter; figure 52, as a bipolar converter, figure 53, it is
necessary to produce an offset voltage and the way the basis of the circuit to be used
107 of 200

does this is to inject an equivalent offset current, into the summing junction of the
op-amp as shown in figure 53.
I
R
R
R
Vref
2R 2R 2R
2R
2R
I/2
I/4 I/8
I/16
R F
-
+
V O
ENABLE
4-bit register
MSB
LSB
Figure 52 An R-2R ladder network D/A converter
For an R-2R ladder network the input resistance of the network is R, as we know that
the maximum current (I) delivered from the receiver (FPGA) circuit is to be 100µA
and retinal tissue resistance is at least 10kΩ, R is chosen to be 10kΩ. V ref can now be
calculated as 1v.
Typically, maximum output voltage is obtained for an input codeword consisting
entirely of logic 1’s so the current entering the feedback loop is
108 of 200

Hence the output voltage is
I
R
R
R
2R 2R 2R
2R
I/2
-V ref
I/16
2R
I/4 I/8
R F
-
+
V O
I OFF
Figure 53 A bipolar D/A converter
However the input codeword for this design is 1001, so in this case the total current
entering the summing junction of the amplifier will be
By making R f equal to 4R (in this case 40kΩ) then I off (offset current) will equal I/4
and therefore I s (summing junction current) will equal 5(I)/16 and
109 of 200

6.2.2 Envisaged retinal array
This is the nature of the connection between the stimulator and the retinal array. The
optic disc is an oval of 1.76mm by 1.92mm giving an area of 2.66mm 2
[211]. There
needs to be an assurance for a colour orientated prosthesis that only one axon will be
targeted. Given 50% of the ganglion cells within the optic disc are concerned with
colour signals originating from the fovea [212] then this implies 1.33mm 2 housing
600, 000 fibres will have fibre areas of ≈2.2µm 2 and diameters of 1.68µm. The
overlap between a 2µm diameter electrode and a 1.68µm fibre is 0.32 i.e. the
encroachment upon neighbouring fibres will be 0.16µm, however as supporting
tissue (0.34µm) is present before the axon we have a 0.17µm `safe’ zone before the
neighbouring axon is targeted, figure 54.
Figure 54 Axon clearance
The stimulation site of a 2µm electrode is therefore well able to stimulate the 1µm
axon of the RGC. As each electrode has an associated pitch of 50µm (0.05mm) the
obtainable electrode density can be estimated by producing a square to cover the
same area (1.33mm 2 ) meaning a side of ≈1.15mm. Then the number of pitches to
110 of 200

each side would be 1.15 ÷ 0.05 equals 23 giving potentially 23 by 23 equals 529
electrodes (in practice; due to pragmatic limitations 170 i.e (2 x 85)). However three
factors need to be born in mind; firstly, 256 wires through a 5mm incision is the
present (2012) limitation to electrode connections, secondly for structural
implementation of a four by four picture a third of this wiring capability (≈ 85) needs
to be available outside of this foveal specific area of the optic disc for the `blue’
signals to the small bistratified RGC’s [213, 214]. Recall the majority of cones fed
from the fovea are `red’ and `green’ whereas the `blue’ cone signals directed to the
parafoveal retina can use 30µm diameter electrodes to generate larger stimulation
sites to resonate [215] with the small bistratified RGC’s; responsible for handling
blue cone signals, outside of this foveal specific area of the optic disc. In
configuration each plane of the test image can be configured separately to ensure
correct placement of the phosphenes. Figure 55 shows the perimeter of 30µm
electrodes associated with blue cone signals. The expanded view of the internal
portion of the array, with 2µm electrodes; focused over the foveal pit, targets the
RGC’s carrying red and green cone signals. Effectively a one-to-one correspondence
between stimulation site and electrode
6.2.3 Connecting the detailed engineering to the overall
concept
Chapter four describes the design and implementation of the sender chip which
translates the camera image to an AER format for transmission to the receiver chip
described in chapter five. Chapter five describes the routing of the timed signals to
the driver circuitry housed atop the retinal implant. Future work could avoid
telemetry once a colour camera becomes available that can be housed in the eyeball;
111 of 200

a monochrome camera in this space is already possible, this would resolve issues of
variable inductive coupling which presently exist with current retinal implants.
6.2.4 Differences between proposed technique and current
retinal implants
Current retinal prostheses are asynchronous and achromatic requiring an arbiter in
the first case and amplitude variation to attain brightness variation of the greyscale
image; in the second case. Any amplitude variation in this approach would be purely
for reasons of electrode positioning and patient sensitivity. Current retinal prostheses
use comparatively large electrodes relying on pulse duration to resonate correctly
with RGC’s, this approach for the inner portion of the retinal implant is synchronous
whilst maintaining a frame rate for possible hybrid prostheses, although in principle
being pixel based rather than frame based this is not necessary.
112 of 200

500µm
880µm
1760µm
minor axis
Major
axis
1
9
2
0
µ
m
Pitch of 85 outer 30µm diameter electrodes: 68µm
Figure 55 Retinal implant
6.3 Lower power FPGA
A low power FPGA with a smaller footprint such as Actmel’s “IGLOO PLUS”
could be utilised to lower the power requirement for this initial prototype (circa
113 of 200

632mW) thus extending battery life by several orders of magnitude. Once this is
done any future work is likely to necessitate a post processing interface from this
FPGA to connect to the electrodes of the retinal implant and surgical expertise.
Power requirement for Virtex 5 used to implement 4 by 4 retinal prostheses as
described in this document.
-------------------------------------------------------
| Power Supply Summary |
-------------------------------------------------------
| | Total | Dynamic | Quiescent |
-------------------------------------------------------
| Supply Power (mW) | 631.29 | 36.59 | 594.70 |
-------------------------------------------------------
-------------------------------------------------------------------------------------
--------------------------
| Power Supply Currents
|
-------------------------------------------------------------------------------------
--------------------------
| Supply Source | Supply Voltage | Total Current (mA) | Dynamic Current (mA)
| Quiescent Current (mA) |
-------------------------------------------------------------------------------------
--------------------------
| Vccint | 1.00 | 417.80 | 16.59
| 401.20 |
| Vccaux | 2.50 | 83.40 | 8.00
| 75.40 |
| Vcco25 | 2.50 | 2.00 | 0.00
| 2.00 |
For upto 193 I/Os the AGL600 meets the replacement requirements, the product
table is shown here.
114 of 200

Table 16 IGLOO product table
6.4 AER between sender and receiver chip
Recall the relationship between image size and bit length of the AER transmission
packet as shown in the following table.
side of square image pixel count address bits required Total bit length
2 4 2 20
4 16 4 22
8 64 6 24
16 256 8 26
32 1024 10 28
1024 1048576 20 38
Table 17 image size versus bit length
115 of 200

For the implemented/structural case of the 32 by 32 image the bit length of the
transmitted AER packet is composed of ten address bits and eighteen stimulus bits.
This corresponds to a bit rate determined by the AER packet frequency. The AER
frequency is derived from the partial_spike clock frequency. In the case of the test
setup described earlier using an FPGA clock of 25Mhz as the partial_spike
frequency the spike frequency would be a quarter of this i.e. 6.25MHz. As an
outerpulse (at maximum intensity) has a periodic time of five times spike time then
outerpulse frequency is 1.25MHz. As there are fifty outerpulses in a pixel then pixel
(AER) frequency is 25Khz.
6.5 Post Processing Interface
In medical experiments equivalent impedance for the retinal tissue has been
estimated at ≈ 10KΩ [53]. The post processing required to convert the signals from
the FPGA to those suitable for driving the electrodes of 10kΩ impedance [54] must
deliver sufficient charge to simulate the biological action potential (AP) within the
RGC of the optic nerve fibre. As there is only one interconnect lead per stimulation
site (to reduce the wiring through the surgical incision) the interface will require two
supply voltages to produce the biphasic current pulse. For a minimum supply voltage
of 5v [51] i.e. +5v and -5v the maximum current per electrode would be 500µA. The
instantaneous power would therefore be 2.5mw resulting in 2.5w if all electrodes
were continuously active. However for 50 active biphasic pulses within a second the
electrode is only active for 2ms of each pulse (1ms for each phase – 10% duty cycle)
and that’s at maximum intensity then the stimulation power requirement will reduce
to 0.25w. Presuming a 0.3 activity factor i.e. a scene is rarely fully white therefore
less than 50 pulses per second are common then power requirement would fall to
250/3 i.e. circa 83mw.
116 of 200

Membrane potential (mv)
+30
0
-70
0 1 2 3 4 5
Time(ms)
Peak to peak 100
x 50 pulses equal 5v
Figure 56 AP propagating through optic nerve
6.6 Pre processing
Presently the incoming image for this prototype system is held in ROM, prior to full
implementation a camera would be interfaced to the FPGA, either a RGB camera or
a YUV camera utilizing an intellectual property (IP) core to convert to RGB signal
format. Future work could encompass the possibility of converting a camera with
universal signal bus (USB) protocol to RGB format. When using an RGB or YUV
camera the frame rate to be set will be determined by the partial_spike_clock
frequency derived from the FPGA clock frequency. The ratio is largely determined
by the image size to be transmitted e.g. for a circa 1000 pixel image a
partial_spike_clock frequency of 25Mhz would expect a frame rate of 25fps and
similarly a partial_spike_clock frequency of 50Mhz would expect a frame rate of
50fps for the same size of image.
6.7 Future work
117 of 200

The sender and receiver circuitry presently implemented in FPGA fabric needs to be
produced in the form of two separate application specific integrated circuits
(ASIC’s) to reduce the physical footprint. The power pack supplying power to the
system is worn at the waist of the implant patient. Camera and receiver chip is
housed on a pair of spectacles worn by the patient. The following diagram illustrates
this part of the setup.
Interconnection
to retinal implant
Receiver chip
Camera
power
Data
feed
Data
feed
Sender chip
Battery
pack
Belt
Figure 57 Practical setup
6.7.1 Surgical implantation
The microstimulator or driver circuitry (post processing interface) is implanted
alongside the epiretinal electrode array. A 5mm surgical incision allows up to 256
wires to pass through; at the time of writing (February 2012). So for the
implementation of a 4 by 4 image as has been implemented in this prototype 48
118 of 200

wires plus the return wire would pass through this incision when placing the retinal
implant. Can the commercially available Argus II retinal implant [110, 216] be
utilized for this colour implementation? The Argus II retinal implant is of an
achromatic design; therefore it will not be directly suitable for use in its present
configuration, although the driver circuitry may be applicable. As the cost of the
Argus II is presently $100000 then a bespoke colour vision design would be a good
idea if will retinal implantation is ever to be commercially viable. The issue to be
resolved is an alternative cheaper coating to the diamond coating presently used to
prevent bio degradation.
6.7.2 Inductive linking considerations
The interconnection to the retinal implant of this prototype system could be refined
from a physical connection through the skull wall, some 7mm, to a minimally
invasive inductive link [217-225]. Issues which would need to be addressed, among
others, are the extra power requirements, due to inefficiencies of power transfer
through an inductive link and also potential extra electromagnetic heating effects
[226-228].
6.7.3 Initial configuration
Within FPGA programming the control outputs feeding the post processing are
rerouted to differing electrodes of the implanted array as necessary. This enables
each plane of colour to be setup separately utilizing configuration test images to
allow for anomalies in electrode placement in terms of retinotopic diversity.
119 of 200

Bibliography
1. Woodburn R. Murray A.F., Pulse-stream techniques and circuits. Circuits
and Devices Magazine, IEEE, 1996. 12(4): p. 43-47.
2. Culurciello, E., R. Etienne-Cummings, and K. Boahen, Arbitrated address
event representation digital image sensor. Digest of Technical Papers.
ISSCC. 2001 IEEE International Solid-State Circuits Conference, 2001.,
2001: p. 92-93.
3. Teixeira, T., et al., Address-event imagers for sensor networks: evaluation
and modeling. IPSN 2006. The Fifth International Conference on Information
Processing in Sensor Networks, 2006., 2006: p. 458-466.
4. Serrano-Gotarredona, R., et al., A Neuromorphic Cortical-Layer Microchip
for Spike-Based Event Processing Vision Systems. IEEE Transactions on
Circuits and Systems I: Regular Papers,, 2006. 53(12): p. 2548-2566.
5. Saude T., Ocular Anatomy and Physiology. 1993.
6. Berne R. M. Levy M. N., Physiology (Fourth Edition). 1989.
7. Malvin G. M. Malvin R. L. Johnson M. D., Concepts of Human Physiology.
1997.
8. Snell R. S. Lemp M. A., Clinical Anatomy of the Eye. 1998.
9. Schmidt R. F. Thew G., Human Physiology. 2nd ed. 1983/1989.
10. DeWitt W., Human Biology Form, Function and Adaptation. 1989.
11. Widmair E. P. Raff H. Strang K. T., Vanders Human Physiology. 10th ed.
2006.
12. Tovec M. J., An Introduction to the Visual System. 1996.
13. Davison H., Physiology of the Eye. Fifth ed. 1990.
14. Forrester J. V. Dick A. B. McManamin P. G. Lee W. R., The Eye (basic
sciences in practice). 2nd ed. 2002.
15. Zeki S., A Vision of the Brain. 1993.
16. Helgemo D. R., Digital Signal Processing at 1GHz in a Field-Programmable
Object Array. MAPLD 2003 Conference, 2003.
17. Hu, N., et al. Displacement simulation analysis of Microelectrode Array in
Artificial Retina. in Complex Medical Engineering, 2007. CME 2007.
IEEE/ICME International Conference on
Complex Medical Engineering, 2007. CME 2007. IEEE/ICME International
Conference on VO -. 2007.
18. Weiland J.D. Anderson D.J. Humayun M.S., In vitro electrical properties for
iridium oxide versus titanium nitride stimulating electrodes. IEEE
Transactions on Biomedical Engineering, 2002. 49(12): p. 1574-1579.
19. Chu, A.P., et al., Stimulus induced pH changes in retinal implants.
Engineering in Medicine and Biology Society, 2004. IEMBS '04. 26th
Annual International Conference of the IEEE, 2004. 2: p. 4160-4162.
20. Gerhardt, M., J. Alderman, and A. Stett, Electric Field Stimulation of Bipolar
Cells in a Degenerated Retina&#x2014;A Theoretical Study. Neural Systems
and Rehabilitation Engineering, IEEE Transactions on. 18(1): p. 1-10.
21. Nadeau, P. and M. Sawan. A flexible high voltage biphasic currentcontrolled
stimulator. in Biomedical Circuits and Systems Conference, 2006.
BioCAS 2006. IEEE. 2006.
120 of 200

22. Vidal, J. and M. Ghovanloo. Towards a Switched-Capacitor based
Stimulator for efficient deep-brain stimulation. in Engineering in Medicine
and Biology Society (EMBC), 2010 Annual International Conference of the
IEEE.
23. Wang, R., et al., Fabrication and Characterization of a Parylene-Based
Three-Dimensional Microelectrode Array for Use in Retinal Prosthesis.
Microelectromechanical Systems, Journal of. 19(2): p. 367-374.
24. Lovell, N.H., et al., Biological&#x2013;Machine Systems Integration:
Engineering the Neural Interface. Proceedings of the IEEE. 98(3): p. 418-
431.
25. Xiaohong, S., et al. Encapsulation and Evaluation of a MEMS-Based
Flexible Microelectrode Array for Acute In-Vivo Experiment. in Biomedical
Engineering and Informatics, 2009. BMEI '09. 2nd International Conference
on. 2009.
26. Rodger, D.C. and T. Yu-Chong, Microelectronic packaging for retinal
prostheses. Engineering in Medicine and Biology Magazine, IEEE, 2005.
24(5): p. 52-57.
27. Sarje, A. and N. Thakor. Neural interfacing. in IEMBS '04. 26th Annual
International Conference of the IEEE Engineering in Medicine and Biology
Society, 2004. 2004.
28. Rodger, D.C., et al. Flexible Parylene-based Microelectrode Technology for
Intraocular Retinal Prostheses. in Nano/Micro Engineered and Molecular
Systems, 2006. NEMS '06. 1st IEEE International Conference on. 2006.
29. Ganesan, K., et al. Diamond penetrating electrode array for Epi-Retinal
Prosthesis. in Engineering in Medicine and Biology Society (EMBC), 2010
Annual International Conference of the IEEE.
30. Schröder, J.M., J. Bohl, and U. Bardeleben, Changes of the ratio between
myelin thickness and axon diameter in human developing sural, femoral,
ulnar, facial, and trochlear nerves. Acta Neuropathologica, 1988. 76(5): p.
471-483.
31. Asher, A., et al., Image Processing for a High-Resolution Optoelectronic
Retinal Prosthesis. Biomedical Engineering, IEEE Transactions on, 2007.
54(6): p. 993-1004.
32. Carlidge E., Vision on a chip. Physics World, 2007.
33. Degenaar P. LePioufle B. Griscom L. Tixier A. Akagi Y. Morita Y.
Murakami Y. Yokoyama K. Fujita H. Tamiya E., A Method for Micrometer
Resolution Patterning of Primary Culture Neurons for SPM Analysis. J
Biochem, 2001. 130: p. 367-376.
34. Banks, D., P. Degenaar, and C. Toumazou. A Bio-Inspired Adaptive Retinal
Processing Neuron with Multiplexed Spiking Outputs. in Circuits and
Systems, 2007. ISCAS 2007. IEEE International Symposium on. 2007.
35. Nikolic, K., et al. A Non-Invasive Retinal Prosthesis - Testing the Concept. in
Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th
Annual International Conference of the IEEE. 2007.
36. Banks, D.J., P. Degenaar, and C. Toumazou, Low-power pulse-widthmodulated
neuromorphic spiking circuit allowing signed double byte data
transfer along a single channel. Electronics Letters, 2007. 43(13): p. 704-
706.
37. Degenaar, P., T.G. Constandinou, and C. Toumazou, Adaptive ON-OFF
spiking photoreceptor. Electronics Letters, 2006. 42(4): p. 196-198.
121 of 200

38. Banks, D.J., P. Degenaar, and C. Toumazou, Distributed current-mode image
processing filters. Electronics Letters, 2005. 41(22): p. 1201-1202.
39. Constandinou, T.G., et al. An on/off spiking photoreceptor for adaptive
ultrafast/ultrawide dynamic range vision chips. in Biomedical Circuits and
Systems, 2004 IEEE International Workshop on. 2004.
40. Al-Atabany, W., et al., Designing and testing scene enhancement algorithms
for patients with retina degenerative disorders. BioMedical Engineering
OnLine. 9(1): p. 27.
41. Abu-Faraj, Z.O., et al. A Prototype Retinal Prosthesis for Visual Stimulation.
in Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th
Annual International Conference of the IEEE. 2007.
42. Hering, XI Hering's four-color theory Zone theories. Documenta
Ophthalmologica, 1999. 96(1): p. 165-174.
43. Rolando, C.A. and et al., Neuromorphic model of magnocellular and
parvocellular visual paths: spatial resolution. Journal of Physics: Conference
Series, 2007. 90(1): p. 012099.
44. Avraham, T. and Y. Schechner, Ultrawide Foveated Video Extrapolation.
Selected Topics in Signal Processing, IEEE Journal of. PP(99): p. 1-1.
45. Kandel E. R. Schwartz J. H. Jessel M., Principles of Neural Science. 2000:
McGraw Hill, ISBN 0-07-112000-9.
46. Patton H. D. Fuchs A. F. Hille B. Scher A. M. Steiner R., Textbook of
Physiology Excitable Cells and Neurophysiology. 21st ed. Vol. 1. 1989:
Harcourt.
47. Rolls E. T. Deco G., Computational Neuroscience of Vision. 2002: Oxford
University Press.
48. Sukkar M. Y. El-Munshid H. A. Ardaw M. S. M., Concise Human
Physiology. 2nd ed. 1993, 2000.
49. Greenberg, R.J., et al., A computational model of electrical stimulation of the
retinal ganglion cell. Biomedical Engineering, IEEE Transactions on, 1999.
46(5): p. 505-514.
50. Shapley R., Specificity of Cone Connections in the Retina and Colour Vision.
Focus on "Specificity of Cone Inputs to Macaque Retinal Ganglion Cells".
2005.
51. Theogarajan, L., et al. Visual prostheses: Current progress and challenges. in
VLSI Design, Automation and Test, 2009. VLSI-DAT '09. International
Symposium on. 2009.
52. Weiland, J.D. and M.S. Humayun, Intraocular retinal prosthesis.
Engineering in Medicine and Biology Magazine, IEEE, 2006. 25(5): p. 60-
66.
53. Liu W. Vichienchom K. Clements M. DeMarco S.C. Hughes C. McGucken
E. Humayun M.S. De Juan E. Weiland J.D. Greenberg R., A neuro-stimulus
chip with telemetry unit for retinal prosthetic device. IEEE Journal of Solid-
State Circuits, 2000. 35(10): p. 1487-1497.
54. Sivaprakasam, M., et al., Architecture tradeoffs in high-density
microstimulators for retinal prosthesis. Circuits and Systems I: Regular
Papers, IEEE Transactions on, 2005. 52(12): p. 2629-2641.
55. Jeng-Shyong S. Maia M. Weiland J.D. O'Hearn T. Shih-Jen C. Margalit E.
Suzuki S. Humayun, M.S., Electrical Stimulation in Isolated Rabbit Retina.
IEEE Transactions on Neural Systems and Rehabilitation Engineering,, 2006.
14(3): p. 290-298.
122 of 200

56. Sivaprakasam, M., et al., A variable range bi-phasic current stimulus driver
circuitry for an implantable retinal prosthetic device. Solid-State Circuits,
IEEE Journal of, 2005. 40(3): p. 763-771.
57. Sivaprakasam, M., et al. A Programmable Discharge Circuitry With Current
Limiting Capability for a Retinal Prosthesis. in 27th Annual International
Conference of the Engineering in Medicine and Biology Society. 2005.
58. Mahadevappa, M., et al., Perceptual thresholds and electrode impedance in
three retinal prosthesis subjects. Neural Systems and Rehabilitation
Engineering, IEEE Transactions on, 2005. 13(2): p. 201-206.
59. Linares-Barranco A. Jimenez-Moreno G. Linares-Barranco B. Civit-Balcells
A., On algorithmic rate-coded AER generation. IEEE Transactions on Neural
Networks, 2006. 17(3): p. 771-788.
60. Zaghloul K.A. Boahen K., Optic nerve signals in a neuromorphic chip II:
testing and results. IEEE Transactions on Biomedical Engineering, 2004.
51(4): p. 667-675.
61. Azadmehr M. Abrahamsen J. P. Hafliger P., A Foveated AER Imager Chip.
IEEE International Symposium on Circuits and Systems, 2005. 3: p. 2751 -
2754.
62. Linares-Barranco A. Jimenez-Moreno G. Civit-Ballcels A. Linares-Barranco
B., On synthetic AER Generation. Proceedings of the 2004 International
Symposium on Circuits and Systems, 2004. 5.
63. Abrahamsen J.P. Hafliger P. Lande T.S. A time domain winner-take-all
network of integrate-and-fire neurons. in ISCAS '04. Proceedings of the 2004
International Symposium on Circuits and Systems. 2004.
64. Vogelstein R.J. Mallik U. Vogelstein J.T. Cauwenberghs G., Dynamically
Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based
Synapses. Neural Networks, IEEE Transactions on, 2007. 18(1): p. 253-265.
65. Matolin D. Schreiter J. Schuffny R. Heittmann A. Ramacher U., Simulation
and implementation of an analog VLSI pulse-coupled neural network for
image segmentation. MWSCAS '04. The 2004 47th Midwest Symposium on
Circuits and Systems, 2004. 2: p. II-397-II-400 vol.2.
66. Chicca E. Indiveri G. Douglas R.J., An event-based VLSI network of
integrate-and-fire neurons. ISCAS '04. Proceedings of the 2004 International
Symposium on Circuits and Systems, 2004. 5: p. V-357-60 Vol.5.
67. Murray A.F. Del Corso D. Tarassenko L., Pulse-stream VLSI neural
networks mixing analog and digital techniques. IEEE Transactions on Neural
Networks, 1991. 2(2): p. 193-204.
68. Rowcliffe, P., J. Feng, and H. Buxton, Spiking perceptrons. Neural
Networks, IEEE Transactions on, 2006. 17(3): p. 803-807.
69. John L. Johnson and Mary Lou Padgett, M., IEEE, PCNN Models and
Applications. IEEE Transactions on Neural Networks, 1999. 10(3): p. 480-
498.
70. Morris, L.P., Dlay, S. S., DSFPN, a new neural network for optical character
recognition. IEEE Transactions on Neural Networks, 1999. 10(6): p. 1465-
1473.
71. M., C., Neural Networks - A Tutorial. 1993.
72. Haykin, S., Neural Networks - A Comprehensive Foundation. Second ed.
1999: Prentice Hall.
73. Duch, W., Uncertainty of data, fuzzy membership functions, and multilayer
perceptrons. Neural Networks, IEEE Transactions on, 2005. 16(1): p. 10-23.
123 of 200

74. Freeman, J.A., Skapura, D. M., , Neural Networks Algorithms, Applications
and Programming Techniques. 1991.1992: Addison-Wesley
75. Zamani, M., A. Sadeghian, and S. Chartier. A bidirectional associative
memory based on cortical spiking neurons using temporal coding. in Neural
Networks (IJCNN), The 2010 International Joint Conference on.
76. Zometzer S. F., D.J.L., Lau C., McKenna T., ed. An Introduction to Neural
and Electronic Networks (2nd Edition). Second ed. 1995, 1990, Academic
Press.
77. Jain L. C., V.V.R., Industrial Applications of Neural Networks. 1999: The
CRC Press.
78. Guoyin, W. and S. Hongbao, TMLNN: triple-valued or multiple-valued logic
neural network. Neural Networks, IEEE Transactions on, 1998. 9(6): p.
1099-1117.
79. Cotofana, S. and S. Vassiliadis, Periodic symmetric functions, serial
addition, and multiplication with neural networks. Neural Networks, IEEE
Transactions on, 1998. 9(6): p. 1118-1128.
80. Reinhard Eckhorn, A.M.G., Anreas Bruns, Andreas Gabriel, Basim Al-
Shaikhli and Mirko Saam, Different types of Signal Coupling in the Visual
Cortex Related to Neural Mechanisms of Associative Processing and
Perception. IEEE Transactions on Neural Networks, 2004. 15(5): p. 1039-
1052.
81. Garrett T. Kenyon, B.J.T., James Theiler, John S. George, Gregory J.
Stephens and David W. Marshak, Stimulus-Specific Oscillations in a Retinal
Model. IEEE Transactions on Neural Networks, 2004. 15(5): p. 1083-1091.
82. Eckhorn, R., Neural Mechanisms of Scene Segmentation: Recordings from
the Visual Cortex Suggest Basic Circuits for Linking Field Models. IEEE
Transactions on Neural Networks, 1999. 10(3): p. 464-478.
83. Kolman, E. and M. Margaliot. Are Artificial Networks White Boxes? IEEE
Transactions on Neural Networks [Fuzzy rule bases (FRBs)] 2005 July 2005
[cited 16 4]; 844-852].
84. Caudill M, B., C, Understanding neural networks. Vol. 1. 1992.
85. Galan, R.C., et al., A bio-inspired two-layer mixed-signal flexible
programmable chip for early vision. Neural Networks, IEEE Transactions
on, 2003. 14(5): p. 1313-1336.
86. Wohrer, A., P. Kornprobst, and T. Vieville. From Light to Spikes: a Large-
Scale Retina Simulator. in Neural Networks, 2006. IJCNN '06. International
Joint Conference on
Neural Networks, 2006. IJCNN '06. International Joint Conference on VO -. 2006.
87. Ohta, J., et al. Si-LSI Based Stimulators for Retinal Prosthesis. in Neural
Networks, 2007. IJCNN 2007. International Joint Conference on. 2007.
88. Ryu, S.B., et al. Optimal linear filter based light intensity decoding from
rabbit retinal ganglion cell spike trains. in Neural Engineering, 2007. CNE
'07. 3rd International IEEE/EMBS Conference on
Neural Engineering, 2007. CNE '07. 3rd International IEEE/EMBS Conference on
VO -. 2007.
89. Johnson J., P.P., Designing Intelligent Machines. Vol. Two. 1995:
Butterworth Heinmann.
90. Culurciello E. Andreou A.G., A comparative study of access topologies for
chip-level address-event communication channels. IEEE Transactions on
Neural Networks, 2003. 14(5): p. 1266-1277.
124 of 200

91. Mark, F.B., Barry, W. Connors, Michael, A. Paradiso Neuroscience
Exploring the Brain Third ed. 2006: Lippincott Williams and Wilkins
92. Kandel E. R. Schwartz J. H. Jessel T. M., Essentials of Neural Science and
Behaviour. 1995: Prentice Hall International.
93. Buchsbaum, G., The retina as a two-dimensional detector array in the
context of color vision theories and signal detection theory. Proceedings of
the IEEE, 1981. 69(7): p. 772-786.
94. Liu, W. Intraocular retinal prosthesis: microelectronics meets medicine. in
Microprocesses and Nanotechnology Conference, 2001 International. 2001.
95. Talukder, M.I., P. Siy, and G.W. Auner. Parallel Multiplexing - a Solution of
Large Scale Stimulation Needed by the Retinal Prostheses to Maintain the
Persistence of Vision. in Engineering in Medicine and Biology Society, 2006.
EMBS '06. 28th Annual International Conference of the IEEE. 2006.
96. Wentai, L. Retinal implant: bridging engineering and medicine. in Electron
Devices Meeting, 2002. IEDM '02. Digest. International. 2002.
97. Tran, N., et al. A Flexible Electrode Driver Using 65 nm CMOS Process for
1024-Electrode Epi-Retinal Prosthesis. in Future Information Technology
(FutureTech), 2010 5th International Conference on. 2010.
98. Benav, H., et al. Restoration of useful vision up to letter recognition
capabilities using subretinal microphotodiodes. in Engineering in Medicine
and Biology Society (EMBC), 2010 Annual International Conference of the
IEEE.
99. Wyatt, J. Steps toward the development of a chronic retinal implant. in
Wearable and Implantable Body Sensor Networks, 2006. BSN 2006.
International Workshop on. 2006.
100. Xu, Z., et al. Neuro-stimulus chip with photodiodes array for sub-retinal
implants. in Solid-State and Integrated-Circuit Technology, 2008. ICSICT
2008. 9th International Conference on. 2008.
101. Fan-Gang, Z., et al., Cochlear Implants: System Design, Integration, and
Evaluation. Biomedical Engineering, IEEE Reviews in, 2008. 1: p. 115-142.
102. Kuanfu, C., et al., An Integrated 256-Channel Epiretinal Prosthesis. Solid-
State Circuits, IEEE Journal of, 2010. 45(9): p. 1946-1956.
103. Weiland, J.D. and M.S. Humayun, A biomimetic retinal stimulating array.
Engineering in Medicine and Biology Magazine, IEEE, 2005. 24(5): p. 14-
21.
104. Yang, W.-C., et al. A low DC-level variation retinal chip based on the
neuromorphic model of on sluggish sustained ganglion cell set of rabbits. in
Cellular Nanoscale Networks and Their Applications (CNNA), 2010 12th
International Workshop on. 2010.
105. Greenberg, R.J., et al. Preliminary results from the argus II study: A 60
electrode epiretinal prosthesis. in Bionic Health: Next Generation Implants,
Prosthetics and Devices, 2009 IET. 2009.
106. Schwiebert, L., et al. A biomedical smart sensor for the visually impaired. in
Sensors, 2002. Proceedings of IEEE. 2002.
107. Watanabe, T., et al. Novel Retinal Prosthesis System with Three
Dimensionally Stacked LSI Chip. in Solid-State Device Research Conference,
2006. ESSDERC 2006. Proceeding of the 36th European. 2006.
108. Prives, L., An eye for detail. Women in Engineering Magazine, IEEE, 2009.
3(2): p. 19-20.
125 of 200

109. Weiland, J.D., et al. Progress Towards A High-Resolution Retinal Prosthesis.
in 27th Annual International Conference of the Engineering in Medicine and
Biology Society. 2005.
110. Humayun, M.S., et al. Preliminary 6 month results from the
argustm ii epiretinal prosthesis feasibility study. in
Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual
International Conference of the IEEE. 2009.
111. Vurro, M., et al. Simulation and Assessment of Bioinspired Visual Processing
System for Epi-retinal Prostheses. in Engineering in Medicine and Biology
Society, 2006. EMBS '06. 28th Annual International Conference of the IEEE.
2006.
112. Weiland, J.D., et al. Systems design of a high resolution retinal prosthesis. in
Electron Devices Meeting, 2008. IEDM 2008. IEEE International. 2008.
113. Chader, G.J., et al., Artificial vision: needs, functioning, and testing of a
retinal electronic prosthesis, in Progress in Brain Research. 2009, Elsevier.
p. 317-332.
114. Veraart C. Duret F. Brelen M. Delbeke J., Vision rehabilitation with the optic
nerve visual prosthesis. IEMBS '04. 26th Annual International Conference of
the IEEE Engineering in Medicine and Biology Society, 2004., 2004. 2: p.
4163-4164.
115. Weiland, J.D. and M.S. Humayun, Visual Prosthesis. Proceedings of the
IEEE, 2008. 96(7): p. 1076-1084.
116. Ahuja, A.K., et al., Blind subjects implanted with the Argus II retinal
prosthesis are able to improve performance in a spatial-motor task. British
Journal of Ophthalmology.
117. Liu, W., M.S. Humayun, and M.A. Liker, Implantable Biomimetic
Microelectronics Systems. Proceedings of the IEEE, 2008. 96(7): p. 1073-
1075.
118. Tanaka, T., et al. Fully Implantable Retinal Prosthesis Chip with
Photodetector and Stimulus Current Generator. in Electron Devices Meeting,
2007. IEDM 2007. IEEE International
Electron Devices Meeting, 2007. IEDM 2007. IEEE International VO -. 2007.
119. Kuanfu, C. and L. Wentai. Highly programmable digital controller for highdensity
epi-retinal prosthesis. in Engineering in Medicine and Biology
Society, 2009. EMBC 2009. Annual International Conference of the IEEE.
2009.
120. Wyatt J. Rizzo J., Ocular implants for the blind. Spectrum, IEEE, 1996.
33(5): p. 47-53.
121. Fornos, A.P., et al., Simulation of Artificial Vision, III: Do the Spatial or
Temporal Characteristics of Stimulus Pixelization Really Matter?
10.1167/iovs.04-1173. Invest. Ophthalmol. Vis. Sci., 2005. 46(10): p. 3906-3912.
122. Zrenner, E., Will Retinal Implants Restore Vision?
10.1126/science.1067996. Science, 2002. 295(5557): p. 1022-1025.
123. Theogarajan L. Wyatt J. Rizzo J. Drohan B. Markova M. Kelly S. Swider G.
Raj M.Shire D. Gingerich M. Lowenstein J. Yomtov B. Minimally Invasive
Retinal Prosthesis. in ISSCC 2006. Digest of Technical Papers. IEEE
International Solid-State Circuits Conference, 2006. 2006.
124. Sivaprakasam, M., et al., Challenges in System and Circuit Design for High
Density Retinal Prosthesis. Life Science Systems and Applications
Workshop, 2006. IEEE/NLM, 2006: p. 1-2.
126 of 200

125. Wei, H. and X. Guan. The Simulation of Early Vision in Biological Retina
and Analysis of Its Performance. in Image and Signal Processing, 2008.
CISP '08. Congress on
Image and Signal Processing, 2008. CISP '08. Congress on VO - 4. 2008.
126. Weiland, J.D. and M.S. Humayun, Old idea, new technology. Engineering in
Medicine and Biology Magazine, IEEE, 2005. 24(5): p. 12-13.
127. Pramassing, F., et al. Intraocular vision aid (IOS): optical signal
transmission and image generation. in Engineering in Medicine and Biology
Society, 2000. Proceedings of the 22nd Annual International Conference of
the IEEE. 2000.
128. Scribner, D., et al., A Retinal Prosthesis Technology Based on CMOS
Microelectronics and Microwire Glass Electrodes. Biomedical Circuits and
Systems, IEEE Transactions on, 2007. 1(1): p. 73-84.
129. Mokwa, W. Retinal implants to restore vision in blind people. in Solid-State
Sensors, Actuators and Microsystems Conference (TRANSDUCERS), 2011
16th International.
130. Stitt, J.P., et al. An artificial neural network for neural spike classification. in
Bioengineering Conference, 1997., Proceedings of the IEEE 1997 23rd
Northeast. 1997.
131. Kalayci, T. and O. Ozdamar, Wavelet preprocessing for automated neural
network detection of EEG spikes. Engineering in Medicine and Biology
Magazine, IEEE, 1995. 14(2): p. 160-166.
132. Safwan, S., W.E. Faller, and M.W. Luttges. Fuzzy analyses of biological
information processing. in Engineering in Medicine and Biology Society,
1994. Engineering Advances: New Opportunities for Biomedical Engineers.
Proceedings of the 16th Annual International Conference of the IEEE. 1994.
133. Kutlu, Y., Y. Isler, and D. Kuntalp. Detection of Spikes with Multiple Layer
Perceptron Network Structures. in Signal Processing and Communications
Applications, 2006 IEEE 14th. 2006.
134. Kasabov, N., L. Benuskova, and S.G. Wysoski. A computational
neurogenetic model of a spiking neuron. in Neural Networks, 2005. IJCNN
'05. Proceedings. 2005 IEEE International Joint Conference on. 2005.
135. Sovierzoski, M.A., L. Schwarz, and F. Azevedo. Binary Neural Classifier of
Raw EEG Data to Separate Spike and Sharp Wave of the Eye Blink Artifact.
in Natural Computation, 2009. ICNC '09. Fifth International Conference on.
2009.
136. Bidarte U. Ezquerra J. A. Zuloaga A. Martin J. L., VHDL Modeling of an
Adaptive Architecture
for Real-Time Image Enhancement. Proceedings of the Fall VIUF Workshop, 1999.
137. Serrano-Gotarredona T. Linares-Barranco B. Andreou A. G., Programmable
Kernel Analogue VLSI Convolution Chip for Real Time Vision Processing.
International Joint Conference on Neural Networks, 2000. 4.
138. Serrano-Gotarredona T. Andreou A. G. Linares-Barranco B., A 2D Filtering
Architecture for Real-Time Vision Processing Systems. IEEE Computer
Society, 1999: p. 415.
139. Indiveri G. Whatley A. M. Kramer J., A Reconfigurable Neuromorphic VLSI
Multi-Chip System Applied to Visual Motion Computation. Microelectronics
for Neural, Fuzzy and Bio-inspired Systems, 1999: p. 37 - 44.
140. Kameda S. Yagi T., An analog silicon retina with multichip configuration.
IEEE Transactions on Neural Networks, 2006. 17(1): p. 197-210.
127 of 200

141. Indiveri G. Chicca E. Douglas R., A VLSI Array of Low-Power Spiking
Neurons and Bistable Synapses with Spike Timing Dependent Plasticity.
IEEE transactions on neural networks, 2006. 17.
142. Hebb, D.O., The Organisation of Behaviour. 1949: Wiley.
143. Linares-Barranco A. Senhadj-Navarro R. Garcia-Vargas I. Gomez-Rodriguez
F. Jimenez G. Civit A., Synthetic Generation of Address-Events for Real-
Time Image Processing. Emerging Technologies and Factory Automation
Proceedings, 2003. 2: p. 462 - 467.
144. Linares-Barranco A. Paz-Vicente R. Jimenez G. Pedreno-Molina J. L.
Molina-Vilplana J., AER Neuro-Inspired Interface to Anthropomorphic
Robotic Hand. International Joint Conference on Neural Networks, 2006: p.
1497 - 1504.
145. Paz-Vicente R. Linares-Barranco A. Jimenez G. Civit A., PCI to AER
Hardware/Software Interface for Real-Time Vision Processing. World
Automation Congress, 2004, Proceedings, 2004. 18: p. 55 - 62.
146. Lazarro J. Wawrzynek J., A Multi-Sender Asynchronous Extension to the
AER Protocol. Sixteenth Conference on Advanced Research in VLSI, 1995:
p. 158 - 169.
147. Serrano-Gotarredona R. Serrano-Gotarredona T. Acosta-Jimenez A. J.
Linares-Barranco B., An Arbitrary Kernel Convolution AER-Transceiver
Chip for Real-Time Image Filtering. IEEE Symposium on Circuits and
Systems, 2006.
148. Serrano-Gotarredona T. Andreou A. G. Linares-Barranco B., AER Image
Filtering Architecture for Vision-Processing Systems. IEEE Transactions on
Circuits and Systems 1: Fundamental Theory and Applications, 1999. 46(9):
p. 1064 - 1071.
149. Gomez-Rodriguez F. Paz R. Linares-Barranco A. Rivas M. Miro L. Vicente
S. Jimenez G. Civit A. AER tools for communications and debugging. in
IEEE International Symposium on Circuits and Systems. 2006.
150. Choi T. Y. W. Shi B. E. Boahen K., An orientation selective 2D AER
transceiver. ISCAS '03. Proceedings of the 2003 International Symposium on
Circuits and Systems, 2003. 4: p. IV-800-IV-803 vol.4.
151. Thiago T. Culurciello E. Andreou A.G., An Address-Event Image Sensor
Network. 2006 IEEE International Symposium on Circuits and Systems,
2006: p. 4467-4470.
152. Linares-Barranco, A., et al., Poisson AER generator: inter-spike-intervals
analysis. ISCAS 2006. Proceedings. 2006 IEEE International Symposium on
Circuits and Systems, 2006: p. 4 pp.-3152.
153. Patel G.N. Reid M.S. Schimmel D.E. DeWeerth S.P., An asynchronous
architecture for modeling intersegmental neural communication. IEEE
Transactions on Very Large Scale Integration (VLSI) Systems, 2006. 14(2):
p. 97-110.
154. Lichtsteiner P. Delbruck T., A 64x64 aer logarithmic temporal derivative
silicon retina. 2005 PhD Research in Microelectronics and Electronics, 2005.
2: p. 202-205.
155. Boussaid F. Chen S. Bermak A., A scalable low power imager architecture
for compound-eye vision sensors. Proceedings. Fifth International Workshop
on System-on-Chip for Real-Time Applications, 2005: p. 203-206.
156. Linares-Barranco B. Serrano-Gotarredona T. Serrano-Gotarredona R. Costas-
Santos J., A new charge-packet driven mismatch-calibrated integrate-andfire
neuron for processing positive and negative signals in AER based
128 of 200

systems. ISCAS '04. Proceedings of the 2004 International Symposium on
Circuits and Systems, 2004. 5: p. V-744-V-747 Vol.5.
157. Paz-Vicente, R., et al. Synthetic retina for AER systems development. in
Computer Systems and Applications, 2009. AICCSA 2009. IEEE/ACS
International Conference on. 2009.
158. Serrano-Gotarredona, T., A.G. Andreou, and B. Linares-Barranco, AER
image filtering architecture for vision-processing systems. Circuits and
Systems I: Fundamental Theory and Applications, IEEE Transactions on,
1999. 46(9): p. 1064-1071.
159. Dominguez-Morales, M.J., et al. Performance study of synthetic AER
generation on CPUs for Real-Time Video based on Spikes. in Performance
Evaluation of Computer & Telecommunication Systems, 2009. SPECTS
2009. International Symposium on. 2009.
160. Choi T. Y. W. Merolla P. A. Arthur J. V. Boahen K. A. Shi B. E.,
Neuromorphic Implementation of Orientation Hypercolumns. IEE
Transaction on Circuits and Systems-1: Regular Papers, 2005. 52(6): p. pages
1049-1060.
161. Hafliger Ph., Asynchronous event redirecting in bio-inspired communication.
ICECS 2001. The 8th IEEE International Conference on Electronics, Circuits
and Systems, 2001. 1: p. 87-90 vol.1.
162. Boahen K.A., A retinomorphic vision system. Micro, IEEE, 1996. 16(5): p.
30-39.
163. Zhang L. Fang Z. Parker M. Mathew B. K. Schaelicke L. Carter J. B. Hsieh
W. C. McKee S. A., The Impulse memory controller. IEEE Transactions on
Computers,, 2001. 50(11): p. 1117-1132.
164. Lazzaro J. Wawrzynek J. Mahowald M. Sivilotti M. Gillespie D., Silicon
auditory processors as computer peripherals. IEEE Transactions on Neural
Networks, 1993. 4(3): p. 523-528.
165. Boussaid F. Shoushun C. Bermak A., A scalable low power imager for
compound-eye vision sensors. Proceedings of the International Database
Engineering and Application Symposium(IDEAS '05), 2005.
166. Choi T. Y. W. Shi B.E. Boahen K. A., An ON-OFF orientation selective
address event representation image transceiver chip. IEEE Transactions on
Circuits and Systems I: Regular Papers,, 2004. 51(2): p. 342-353.
167. Rivas M. Gomez-Rodriguez F. Paz R. Linares-Barranco A. Vicente S.
Cascado D., Tools for Address-Event Representation Communication
Systems and Debugging. Lecture notes in computer science, 2005.
168. Songnian, Z., et al., Neural computation of visual imaging based on
Kronecker product in the primary visual cortex. BMC Neuroscience. 11(1):
p. 43.
169. Gosalia, K., G. Lazzi, and M. Humayun, Investigation of a microwave data
telemetry link for a retinal prosthesis. Microwave Theory and Techniques,
IEEE Transactions on, 2004. 52(8): p. 1925-1933.
170. Trimberger S. Carberry D. Johnson A. Wong J., A time-multiplexed FPGA.
FPGAs for Custom Computing Machines, 1997. Proceedings., The 5th
Annual IEEE Symposium on, 1997: p. 22-28.
171. Sivaprakasam, M., et al., Towards a Modular 32 x 32 Pixel Stimulator for
Retinal Prosthesis. Life Science Systems and Applications Workshop, 2006.
IEEE/NLM, 2006: p. 1-2.
129 of 200

172. Singh, P.R., et al. A matched biphasic microstimulator for an implantable
retinal prosthetic device. in Circuits and Systems, 2004. ISCAS '04.
Proceedings of the 2004 International Symposium on. 2004.
173. Theogarajan, L.S., A Low-Power Fully Implantable 15-Channel Retinal
Stimulator Chip. Solid-State Circuits, IEEE Journal of, 2008. 43(10): p.
2322-2337.
174. Margalit, E., et al., Retinal Prosthesis for the Blind. Survey of
Ophthalmology, 2002. 47(4): p. 335-356.
175. Dagnelie G., Visual Prosthetics Physiology, Bioengineering, Rehabilitation.
2011.
176. Mokwa, W., et al. Intraocular epiretinal prosthesis to restore vision in blind
humans. in Engineering in Medicine and Biology Society, 2008. EMBS 2008.
30th Annual International Conference of the IEEE. 2008.
177. Keman Yu Jiang Li Jizheng Xu Shipeng Li, Very low bit rate watercolor
video. Proceedings of the 2003 International Symposium on Circuits and
Systems, 2003. 2: p. II-712-II-715 vol.2.
178. Min Hye, C., et al. A comparison of text reading speed using square and
rectangular arrays for visual prosthesis. in Information Technology and
Applications in Biomedicine, 2009. ITAB 2009. 9th International Conference
on. 2009.
179. Flexible, bus-powered USB high-speed storage is a reality - Texas
Instruments. IEE Review, 2005. 51(5): p. 5-5.
180. Ferwerda, J.A., Elements of early vision for computer graphics. Computer
Graphics and Applications, IEEE, 2001. 21(5): p. 22-33.
181. Todd, M. and R. Wilson. An anisotropic multi-resolution image data
compression algorithm. in Acoustics, Speech, and Signal Processing, 1989.
ICASSP-89., 1989 International Conference on. 1989.
182. Es, A. and V. Isler. GPU based real time stereoscopic ray tracing. in
Computer and information sciences, 2007. iscis 2007. 22nd international
symposium on. 2007.
183. Shi R. Z. Horiuchi T. K., A VLSI Model of the Bat Dorsal Nucleus of the
Lateral Lemniscus for Azimuthal Echolocation. IEEE Trans. Circuits Syst. II,
2005. 47(5).
184. Chicca E. Lichtsteiner P. Delbruck T. Indiveri G. Douglas R.J., Modeling
orientation selectivity using a neuromorphic multi-chip system. ISCAS 2006.
Proceedings. 2006 IEEE International Symposium on Circuits and Systems,
2006., 2006: p. 4 pp.
185. Cooper, J., et al., Determination of safety distance limits for a human near a
cellular base station antenna, adopting the IEEE standard or ICNIRP
guidelines. Bioelectromagnetics, 2002. 23(6): p. 429-443.
186. Gye-Hwan J. Jang Hee Y. Tae Soo L. Yong Sook G., Electrical Stimulation
of Isolated Rabbit Retina. 27th Annual International Conference of the
Engineering in Medicine and Biology Society, 2005: p. 5967-5970.
187. Finn W.E. LoPresti P.G., Wavelength and intensity dependence of retinal
evoked responses using in vivo optic nerve recording. IEEE Transactions on
Neural Systems and Rehabilitation Engineering, 2003. 11(4): p. 372-376.
188. Ohta J Tokuda T. Kagawa K. Furumiya T. Uehara A. Terasawa Y. Ozawa M.
Fujikado T. Tano Y., Silicon LSI-based smart stimulators for retinal
prosthesis. Engineering in Medicine and Biology Magazine, IEEE, 2006.
25(5): p. 47-59.
130 of 200

189. Ogmen, H. and M.H. Herzog, The Geometry of Visual Perception:
Retinotopic and Nonretinotopic Representations in the Human Visual System.
Proceedings of the IEEE. 98(3): p. 479-492.
190. Kyuel, T., W. Geisler, and J. Ghosh, Retinally reconstructed images: digital
images having a resolution match with the human eye. Systems, Man and
Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 1999.
29(2): p. 235-243.
191. Tianyi, Y., J. Fengzhe, and W. Jinglong. Central versus peripheral
retinotopic and temporal frequency sensitivities of human visual areas
measured using fMRI. in Mechatronics and Automation, 2009. ICMA 2009.
International Conference on. 2009.
192. Deutsch, S. and A. Deutsch, An Engineering Perspective. Understanding the
Nervous System. 1993.
193. Patton H. Fuchs A. F., Textbook of Physiology: Excitable Cells and
Neurophysiology. 21st ed. Vol. 1. 1989.
194. Chow, A. First trials and future technologies for artificial retinas. in Lasers
and Electro-Optics Society, 2001. LEOS 2001. The 14th Annual Meeting of
the IEEE. 2001.
195. Benav, H., et al. Restoration of useful vision up to letter recognition
capabilities using subretinal microphotodiodes. in Engineering in Medicine
and Biology Society (EMBC), 2010 Annual International Conference of the
IEEE.
196. Zrenner, E., et al., Subretinal electronic chips allow blind patients to read
letters and combine them to words. Proceedings of the Royal Society B:
Biological Sciences, 2012. 278(1711): p. 1489-1497.
197. Uehara, A., et al. System implementation of a CMOS vision chip for visual
recovery. in Sensors and Camera Systems for Scientific, Industrial, and
Digital Photography Applications IV. 2003. Santa Clara, CA, USA: SPIE.
198. Rolando, C.A., J.F. Carmelo, and M.C. Elisa, Neuromorphic model of
magnocellular and parvocellular visual paths: spatial resolution. Journal of
Physics: Conference Series, 2007. 90(1): p. 012099.
199. Croner, L.J. and E. Kaplan, Receptive fields of P and M ganglion cells across
the primate retina. Vision Research, 1995. 35(1): p. 7-24.
200. Mead, C., Neuromorphic electronic systems. Proceedings of the IEEE, 1990.
78(10): p. 1629-1636.
201. Mahowald, M., VLSI analogs of neuronal visual processing : a synthesis of
form and function. 1992, California Institute of Technology, Computer
Science Dept.: Pasadena, Calif.
202. Chen, J.S., A. Huertas, and G. Medioni, Fast Convolution with Laplacian-of-
Gaussian Masks. Pattern Analysis and Machine Intelligence, IEEE
Transactions on, 1987. PAMI-9(4): p. 584-590.
203. Young, R.A., The Gaussian derivative model for spatial vision: I. Retinal
mechanisms. Spatial Vision, 1987. 2(4): p. 273-293.
204. Zukal, M., P. Cika, and R. Burget. Evaluation of interest point detectors for
scenes with changing lightening conditions. in Telecommunications and
Signal Processing (TSP), 2011 34th International Conference on.
205. Rodriguez-Vazquez, A., et al., ACE16k: the third generation of mixed-signal
SIMD-CNN ACE chips toward VSoCs. IEEE Transactions on Circuits and
Systems I: Regular Papers,, 2004. 51(5): p. 851-863.
131 of 200

206. Stieglitz, T., et al. Development of flexible stimulation devices for a retina
implant system. in Engineering in Medicine and Biology Society, 1997.
Proceedings of the 19th Annual International Conference of the IEEE. 1997.
207. Humayun, M.S., et al., Pattern electrical stimulation of the human retina.
Vision Research, 1999. 39(15): p. 2569-2576.
208. Cogan, S.F., Neural Stimulation and Recording Electrodes. Annual Review
of Biomedical Engineering, 2008. 10(1): p. 275-309.
209. Humayun, M.S., et al., Visual perception in a blind subject with a chronic
microelectronic retinal prosthesis. Vision Research, 2003. 43(24): p. 2573-
2581.
210. Johnson, L., et al., Impedance-based retinal contact imaging as an aid for the
placement of high resolution epiretinal prostheses. Journal of Neural
Engineering, 2007. 4(1): p. S17-23.
211. Tasman, W. and E.A. Jaeger, Duane's Ophthalmology. 2009, Lippincott
Williams & Wilkins/Ovid.
212. http://www.britannica.com/EBchecked/topic/1688997/human-eye, humaneye.
2012.
213. Dacey, D.M. and O.S. Packer, Colour coding in the primate retina: diverse
cell types and cone-specific circuitry. Current opinion in neurobiology, 2003.
13(4): p. 421-7.
214. Szmajda, B.A., U. Grunert, and P.R. Martin, Retinal ganglion cell inputs to
the koniocellular pathway. The Journal of comparative neurology, 2008.
510(3): p. 251-68.
215. Paulo E. Stanga1, F.H., Jose A. Sahel3, Lyndon daCruz4, Francesco
Merlini5, Brian Coley5, Robert J. Greenberg6, Argus II Study Group.,
Patients Blinded By Outer Retinal Dystrophies Are Able To Perceive Color
Using The ArgusTm II Retinal Prosthesis System.
216. Kandagor, V., et al. Spatial characterization of electric potentials generated
by pulsed microelectrode arrays. in Engineering in Medicine and Biology
Society (EMBC), 2010 Annual International Conference of the IEEE.
217. Kendir, G.A., et al., An optimal design methodology for inductive power link
with class-E amplifier. Circuits and Systems I: Regular Papers, IEEE
Transactions on, 2005. 52(5): p. 857-866.
218. Chen, P.-J., et al. Implantable Flexible-Coiled Wireless Intraocular Pressure
Sensor. in Micro Electro Mechanical Systems, 2009. MEMS 2009. IEEE
22nd International Conference on. 2009.
219. Singh, V., et al. Bioelectromagnetics for a Retinal Prosthesis to Restore
Partial Vision to the Blind. in Electromagnetics in Advanced Applications,
2007. ICEAA 2007. International Conference on
Electromagnetics in Advanced Applications, 2007. ICEAA 2007. International
Conference on VO -. 2007.
220. Troyk, P.R. and A.D. Rush. Inductive link design for miniature implants. in
Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual
International Conference of the IEEE. 2009.
221. Mingui, S., et al., Passing data and supplying power to neural implants.
Engineering in Medicine and Biology Magazine, IEEE, 2006. 25(5): p. 39-
46.
222. Guoxing, W., et al. A Wireless Phase Shift Keying Transmitter with Q-
Independent Phase Transition Time. in 27th Annual International Conference
of the Engineering in Medicine and Biology Society. 2005.
132 of 200

223. Mingcui, Z., et al. A Transcutaneous Data Telemetry System Tolerant to
Power Telemetry Interference. in Engineering in Medicine and Biology
Society, 2006. EMBS '06. 28th Annual International Conference of the IEEE.
2006.
224. Guoxing, W., et al. A Dual Band Wireless Power and Data Telemetry for
Retinal Prosthesis. in Engineering in Medicine and Biology Society, 2006.
EMBS '06. 28th Annual International Conference of the IEEE. 2006.
225. Singh, V., et al. SAR in the human body by a wireless telemetry system for a
retinal prosthesis. in Antennas and Propagation Society International
Symposium, 2007 IEEE
Antennas and Propagation Society International Symposium, 2007 IEEE VO -.
2007.
226. Lazzi G. DeMarco S.C. Liu W. Weiland J.D. Humayun M.S., Computed SAR
and thermal elevation in a 0.25-mm 2-D model of the human eye and head in
response to an implanted retinal stimulator - part II: results. Antennas and
Propagation, IEEE Transactions on, 2003. 51(9): p. 2286-2295.
227. Singh, V., et al., On the Thermal Elevation of a 60-Electrode Epiretinal
Prosthesis for the Blind. Biomedical Circuits and Systems, IEEE
Transactions on, 2008. 2(4): p. 289-300.
228. Opie, N.L., et al. Thermal heating of a retinal prosthesis: Thermal model and
in-vitro study. in Engineering in Medicine and Biology Society (EMBC),
2010 Annual International Conference of the IEEE. 2010.
229. Jeng-Shyong, S., et al., Electrical Stimulation in Isolated Rabbit Retina.
Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 2006.
14(3): p. 290-298.
230. Kendir, G.A., et al., An optimal design methodology for inductive power link
with class-E amplifier. Circuits and Systems I: Regular Papers, IEEE
Transactions on, 2005. 52(5): p. 857-866.
231. Zaghloul, K.A. and K. Boahen, Optic nerve signals in a neuromorphic chip
II: testing and results. Biomedical Engineering, IEEE Transactions on, 2004.
51(4): p. 667-675.
232. Bratkova, M., S. Boulos, and P. Shirley, oRGB: A Practical Opponent Color
Space for Computer Graphics. Computer Graphics and Applications, IEEE,
2009. 29(1): p. 42-55.
%Appendix A MATLAB
%ycut.m is this main file
%this program to run frames and transmit events, and then show received
%events, uses the function file "pwork.m" at the heart of this operation
%
%required files: mixedsource.m (test frames) and the following function files
%pwork.m, scan.m, subi.m and, to show the neuromorphics - ysend.m.
%
%tidy;
set (0,'DefaultFigureWindowStyle','docked')
clear, close all, imtool close all
clc
%mixedsource;
133 of 200

xmixedsource;
numberOfFrames = 8;
for t = 1: numberOfFrames
%Load image
priortime = t-1;
gfr = imread (sprintf ('mixed%d.bmp', priortime));
%gfr = imresize (gfr, 16);
currentTime = t;
fhr = imread (sprintf ('mixed%d.bmp', currentTime));
%fhr = imresize (fhr, 16);
changes = pwork (gfr, fhr);
if t ~= numberOfFrames
delayUser = input ('Next frame? (enter): ');
end
end
set (0,'DefaultFigureWindowStyle','normal')
%Showing neuromorphics (ysend.m)
function [x] = send(x);
tic
rgb = double(x)/255;
[nx, ny, nz]=size (rgb);
r=reshape (rgb, nx*ny*nz, 1);
%Lets calculate the row value of this reshaped vector
rvalue = nx*ny*nz;
samplenum = 50;
fprintf ('You have chosen %4.f samples\n', samplenum);
% modulate the pixels as spike trains of 500 samples
spike= (diff (mod (kron(r/10, 0: samplenum)', 1))

%fprintf ('rgb2 is: %3.f rows, %3.f columns, colour planes: %0.f \n', t1, t2, t3);
%Next line gives the last showing of the scene
toc
timeinminutes = toc/60
%set (0,'DefaultFigureWindowStyle','normal')
function changes = pwork (grandfather, father);
square = 1; %Change this as required, 16 for 128 square from 8 square.
grandfatherscene = imresize (grandfather, square);
[d1, d2, d3] = size (grandfatherscene);
%The supporting function file called here concerns the control word
grandfatherscene = scan (grandfatherscene, d1, d2);
[a1, a2, a3] = size (grandfatherscene);
subplot (2, 3, 1)
imshow (grandfatherscene (: 1:3)), title ('Ref. scene (t-1)');
%figure, imshow (grandfatherscene (: 1:3)), title ('Ref. scene (t-1)');
%Load up the cell array with the pixels of the test scene
row = 1; %Setting dissection
col = 1; %Setting dissection
k = 0;
for rx = 1: row: d1
% b=num2str (rx);
for cy = 1: col: d2
% J=num2str (cy);
k = k + 1;
check {k} = subi (grandfatherscene, row, col, rx, cy);
%check {k}
%check {k} =imread (['D:\abc\subimage (' b ' th row) (' J ' th column).bmp']);
end
end
%CHANGE THE REFERENCE SCENE BY A CERTAIN AMOUNT
%chooseanimage
fatherscene = imresize (father, square);
[b1, b2, b3] = size (fatherscene);
subplot (2, 3, 2)
imshow (fatherscene (: 1:3)), title ('The scene changed');
%The supporting function file called here concerns the control word
fatherscene = scan (fatherscene, b1, b2);
[h1, h2, h3] = size (fatherscene);
%AT TIME - t i.e. Second reference
%figure, imshow (fatherscene (: 1:3)), title ('At time "t"');
rowf = 1; %Setting dissection
colf = 1; %Setting dissection
kf = 0;
for rxf = 1: rowf: h1
% b=num2str (rx);
for cyf = 1: colf: h2
% J=num2str (cy);
kf = kf + 1;
%Load up the cell array with the pixels of the test scene
checkf {kf} = subi (fatherscene, rowf, colf, rxf, cyf);
135 of 200

%check {k} =imread (['D:\abc\subimage (' b ' th row) (' J ' th column).bmp']);
end
end
fprintf ('Therefore number of subimages (kf) will be %4.f for the test scene\n', kf);
fprintf ('and the square root of kf will equal: %4.f\n', sqrt (kf));
%diary on
%diary worklog.txt
%COMPARE THIS CHANGED SCENE AT: t; TO THE REFERENCE SCENE AT
(t-1)
%fid = fopen ('exp.txt', 'w');
%I.E. fatherscene compared to grandfatherscene.
%imshow (check {1} (: 1:3))
rowf = 1; %Initialising rowf
colf = 1; %Initialising colf
k = 0; %Initialising k
kf = 0; %Initialising kf
ka = 0;
for rxf = 1: rowf: h1
for cyf = 1: colf: h2
k = k+1;
kf = kf+1;
if (check{k}(:,:,1))~= (checkf{kf}(:,:,1))| (check{k}(:,:,2))~= ...
(checkf {kf} (:,:, 2)) | (check{k}(:,:,3))~= (checkf{kf}(:,:,3))...
| (check{k}(:,:,4))~= (checkf{kf}(:,:,4))|(check{k}(:,:,5))~= ...
(checkf {kf} (: 5))|(check{k}(:,:,6))~= (checkf{kf}(:,:,6))
ka = ka + 1;
checka {ka} (: 1) = checkf {kf} (: 1);
checka {ka} (: 2) = checkf {kf} (: 2);
checka {ka} (: 3) = checkf {kf} (: 3);
checka {ka} (: 4) = checkf {kf} (: 4);
checka {ka} (: 5) = checkf {kf} (: 5);
checka {ka} (: 6) = checkf {kf} (: 6);
%sonscene(:,:,:) = [checka{ka}(:,:,1),checka{ka}(:,:,2),checka{ka}(:,:,3)]
%sonscene = checka {ka};
% address = [checka {ka} (: 5); checka {ka} (: 4)];
% fprintf (fid,'%2.f %2.f\n', address);
end
end
%dlmwrite ('test.txt', kf, 'delimiter', '\t', 'newline', 'pc');
end
%fclose (fid);
for count = 1: ka
changes (:,:,:) = cell2mat (checka);
end
%Transmit changes i.e from the send device to the receive device
%Note that the send function deals with the neuromorphic waveform
receive = ysend (changes);
%ASSIGN SCENE PRESENTLY AT RECEIVER AS THE RECEIVED
REFERENCE SCENE
%WHICH WILL INCOPORATE CHANGES TO BECOME THE NEW SCENE.
tprior = uint8 (grandfatherscene);
136 of 200

%Essentially tprior was the (t-1) scene
% [w1, w2, w3] = size (tprior);
%fprintf ('tprior is %3.f rows, %3.f columns, %0.f colour planes. \n', w1, w2, w3);
% [z1, z2, z3] = size (receive);
%fprintf ('receive is %3.f rows, %3.f columns, %0.f colour planes\n', z1, z2, z3);
%From "load up the cell array" to here for scene presently at receiver
%ALTER THIS SCENE RECEIVED AT (t-1) {tprior} BY THE CHANGES
NumOfChanges = ka;
for ka = 1: NumOfChanges %Where "NumOfChanges" refers to pixel count
%Setting variables "row" and "col" to their relevant planes
row = checka {ka} (: 5);
col = checka {ka} (: 4);
tprior (row, col, 1:6) = checka {ka};
%checks {ks} = checka {ka};
%sonscene(:,:,:) = [checka{ka}(:,:,1),checka{ka}(:,:,2),checka{ka}(:,:,3)]
%sonscene = checka {ka};
% address = [checka {ka} (: 5); checka {ka} (: 4)];
% fprintf (fid,'%2.f %2.f\n', address);
end
subplot (2, 3, 5)
imshow (tprior (: 1:3)), title ('Tprior with changes');
%figure, imshow (tprior (: 1:3)), title ('Tprior with changes');
%Name of this file is "scan.m", which adds two further planes to an image
%These planes will represent the "Control word" of addressing for a 128
%by 128 test scene for use by AER routing
function [f] = scan (f, m, n);
%SCAN adds further planes
pixelcount = m*n;
%Setting up the yellow plane
%fR = grandfatherscene (: 1);
%fG = grandfatherscene (: 2);
%c = 255 - fR;
%m = 255 - fG;
%grandfatherscene (: 4) = c;
%grandfatherscene (: 5) = m;
for count = 1 :( m*n)
fB = f (: 3);
y = (255-fB);
f (: 6) = y;
count = count+1;
end
%Getting the size of the inputted scene
% [d1, d2, d3] = size (f);
%fprintf ('\nThe rows of this reference scene are %4.f pixels\n', d1);
%fprintf ('The columns of this reference scene are %4.f pixels\n', d2);
%fprintf ('%0.f colour planes exist in this viewed scene\n', d3);
rx = 1;
cy = 1;
rowhigh = rx + m -1;
colhigh = cy + n -1;
137 of 200

%Two for loops the first (outer loop) assigning the row number for every
%column of the image
%The second (inner) loop assigns column numbers for every row of the outer
%loop. Note that the row numbers are stored on plane 5 and cols on plane 4.
for r = rx: rowhigh
for c = cy: colhigh
%Assigning column numbers
f(r, c, 4) = c;
end
%Name of this file is "subim.m", which accepts input(s)
function [s] = subi (f, m, n, rx, cy);
%SUBIM Extracts a subimage, s, from a given image, f.
%The subimage is of size m-by-n and the coordinates of
%its top, left corner are (rx, cy).
%maxsize = imread ('D:\testimages\maxsize.bmp');
%for count = 1 :( m*n)
%fB = f (: 3);
%y = (255-fB);
%f (: 6) = y;
%count = count+1;
%end
%rowsofinputimage = 1000;
%columnsofinputimage = 1000;
%f = zeros (rowsofinputimage, columnsofinputimage,'double');
%f (: 1) =0;
%f (: 2) =0;
%f (: 3) =0;
%s = zeros (m, n,'uint8');
%s (: 1) =0;
%s (: 2) =0;
%s (: 3) =0;
%showzero = imresize(s, 10);
%figure, imshow (showzero)
rowhigh = rx + m -1;
colhigh = cy + n -1;
%xcount = 0;
%for r = rx: rowhigh
% xcount = xcount + 1;
% ycount = 0;
% for c = cy: colhigh
% ycount = ycount + 1;
% s (xcount, ycount, 1) = f(r, c, 1);
% s (xcount, ycount, 2) = f(r, c, 2);
% s (xcount, ycount, 3) = f(r, c, 3);
% end
%end
r = rx: rowhigh;
c = cy: colhigh;
s (: 1) = f(r, c, 1);
s (: 2) = f(r, c, 2);
138 of 200

s (: 3) = f(r, c, 3);
s (: 4) = f(r, c, 4);
s (: 5) = f(r, c, 5);
s (: 6) = f(r, c, 6);
%Show the resultant image
%n = numel(s)
%showsubimage = imresize(s, 0.1);
%figure, imshow (showsubimage), title ('Final figure shown here at a tenth of actual
size');
%Resetting for this new image size of eight by eight
%This next test image is named as mixed9
%Resetting for this new image size of eight by eight
im=zeros (8, 8, 3,'uint8');
%This next test image is named as mixed8
%Resetting for this new image size of eight by eight
im=zeros (8, 8, 3,'uint8');
rr1c1=255; rr1c2=255; rr1c3=255; rr1c4=000; rr1c5=000; rr1c6=127; rr1c7=96;
rr1c8=127;
gr1c1=000; gr1c2=127; gr1c3=255; gr1c4=255; gr1c5=000; gr1c6=000; gr1c7=000;
gr1c8=127;
br1c1=000; br1c2=000; br1c3=000; br1c4=000; br1c5=255; br1c6=127; br1c7=96;
br1c8=127;
%Row break
rr2c1=127; rr2c2=255; rr2c3=255; rr2c4=255; rr2c5=000; rr2c6=000; rr2c7=127;
rr2c8=96;
gr2c1=127; gr2c2=000; gr2c3=127; gr2c4=255; gr2c5=255; gr2c6=000; gr2c7=000;
gr2c8=000;
br2c1=127; br2c2=000; br2c3=000; br2c4=000; br2c5=000; br2c6=255; br2c7=127;
br2c8=96;
%Row break
rr3c1=96; rr3c2=127; rr3c3=255; rr3c4=255; rr3c5=255; rr3c6=000; rr3c7=000;
rr3c8=127;
gr3c1=000; gr3c2=127; gr3c3=000; gr3c4=127; gr3c5=255; gr3c6=255; gr3c7=000;
gr3c8=000;
br3c1=96; br3c2=127; br3c3=000; br3c4=000; br3c5=000; br3c6=000; br3c7=255;
br3c8=127;
%Row break
rr4c1=127; rr4c2=96; rr4c3=127; rr4c4=255; rr4c5=255; rr4c6=255; rr4c7=000;
rr4c8=000;
gr4c1=000; gr4c2=000; gr4c3=127; gr4c4=000; gr4c5=127; gr4c6=255; gr4c7=255;
gr4c8=000;
br4c1=127; br4c2=96; br4c3=127; br4c4=000; br4c5=000; br4c6=000; br4c7=000;
br4c8=255;
%Row break
rr5c1=000; rr5c2=127; rr5c3=96; rr5c4=127; rr5c5=255; rr5c6=255; rr5c7=255;
rr5c8=000;
gr5c1=000; gr5c2=000; gr5c3=000; gr5c4=127; gr5c5=000; gr5c6=127; gr5c7=255;
gr5c8=255;
br5c1=255; br5c2=127; br5c3=96; br5c4=127; br5c5=000; br5c6=000; br5c7=000;
br5c8=000;
139 of 200

%Row break
rr6c1=000; rr6c2=000; rr6c3=127; rr6c4=96; rr6c5=127; rr6c6=255; rr6c7=255;
rr6c8=255;
gr6c1=255; gr6c2=000; gr6c3=000; gr6c4=000; gr6c5=127; gr6c6=000; gr6c7=127;
gr6c8=255;
br6c1=000; br6c2=255; br6c3=127; br6c4=96; br6c5=127; br6c6=000; br6c7=000;
br6c8=000;
%Row break
rr7c1=255; rr7c2=000; rr7c3=000; rr7c4=127; rr7c5=96; rr7c6=127; rr7c7=255;
rr7c8=255;
gr7c1=255; gr7c2=255; gr7c3=000; gr7c4=000; gr7c5=000; gr7c6=127; gr7c7=000;
gr7c8=127;
br7c1=000; br7c2=000; br7c3=255; br7c4=127; br7c5=96; br7c6=127; br7c7=000;
br7c8=000;
%Row break
rr8c1=255; rr8c2=255; rr8c3=255; rr8c4=255; rr8c5=255; rr8c6=255; rr8c7=255;
rr8c8=255;
gr8c1=255; gr8c2=255; gr8c3=255; gr8c4=255; gr8c5=255; gr8c6=255; gr8c7=255;
gr8c8=255;
br8c1=255; br8c2=255; br8c3=255; br8c4=255; br8c5=255; br8c6=255; br8c7=255;
br8c8=255;
%mixed PLANE COORDINATES
im(1,1,1) = rr1c1; im(1,2,1) = rr1c2; im(1,3,1) = rr1c3; im(1,4,1) = rr1c4;
im(1,5,1) = rr1c5; im(1,6,1) = rr1c6; im(1,7,1) = rr1c7; im(1,8,1) = rr1c8;
im(2,1,1) = rr2c1; im(2,2,1) = rr2c2; im(2,3,1) = rr2c3; im(2,4,1) = rr2c4;
im(2,5,1) = rr2c5; im(2,6,1) = rr2c6; im(2,7,1) = rr2c7; im(2,8,1) = rr2c8;
im(3,1,1) = rr3c1; im(3,2,1) = rr3c2; im(3,3,1) = rr3c3; im(3,4,1) = rr3c4;
im(3,5,1) = rr3c5; im(3,6,1) = rr3c6; im(3,7,1) = rr3c7; im(3,8,1) = rr3c8;
im(4,1,1) = rr4c1; im(4,2,1) = rr4c2; im(4,3,1) = rr4c3; im(4,4,1) = rr4c4;
im(4,5,1) = rr4c5; im(4,6,1) = rr4c6; im(4,7,1) = rr4c7; im(4,8,1) = rr4c8;
im(5,1,1) = rr5c1; im(5,2,1) = rr5c2; im(5,3,1) = rr5c3; im(5,4,1) = rr5c4;
im(5,5,1) = rr5c5; im(5,6,1) = rr5c6; im(5,7,1) = rr5c7; im(5,8,1) = rr5c8;
im(6,1,1) = rr6c1; im(6,2,1) = rr6c2; im(6,3,1) = rr6c3; im(6,4,1) = rr6c4;
im(6,5,1) = rr6c5; im(6,6,1) = rr6c6; im(6,7,1) = rr6c7; im(6,8,1) = rr6c8;
im(7,1,1) = rr7c1; im(7,2,1) = rr7c2; im(7,3,1) = rr7c3; im(7,4,1) = rr7c4;
im(7,5,1) = rr7c5; im(7,6,1) = rr7c6; im(7,7,1) = rr7c7; im(7,8,1) = rr7c8;
im(8,1,1) = rr8c1; im(8,2,1) = rr8c2; im(8,3,1) = rr8c3; im(8,4,1) = rr8c4;
im(8,5,1) = rr8c5; im(8,6,1) = rr8c6; im(8,7,1) = rr8c7; im(8,8,1) = rr8c8;
%mixed PLANE CORDINATES
im(1,1,2) = gr1c1; im(1,2,2) = gr1c2; im(1,3,2) = gr1c3; im(1,4,2) = gr1c4;
im(1,5,2) = gr1c5; im(1,6,2) = gr1c6; im(1,7,2) = gr1c7; im(1,8,2) = gr1c8;
im(2,1,2) = gr2c1; im(2,2,2) = gr2c2; im(2,3,2) = gr2c3; im(2,4,2) = gr2c4;
im(2,5,2) = gr2c5; im(2,6,2) = gr2c6; im(2,7,2) = gr2c7; im(2,8,2) = gr2c8;
im(3,1,2) = gr3c1; im(3,2,2) = gr3c2; im(3,3,2) = gr3c3; im(3,4,2) = gr3c4;
im(3,5,2) = gr3c5; im(3,6,2) = gr3c6; im(3,7,2) = gr3c7; im(3,8,2) = gr3c8;
im(4,1,2) = gr4c1; im(4,2,2) = gr4c2; im(4,3,2) = gr4c3; im(4,4,2) = gr4c4;
im(4,5,2) = gr4c5; im(4,6,2) = gr4c6; im(4,7,2) = gr4c7; im(4,8,2) = gr4c8;
im(5,1,2) = gr5c1; im(5,2,2) = gr5c2; im(5,3,2) = gr5c3; im(5,4,2) = gr5c4;
im(5,5,2) = gr5c5; im(5,6,2) = gr5c6; im(5,7,2) = gr5c7; im(5,8,2) = gr5c8;
im(6,1,2) = gr6c1; im(6,2,2) = gr6c2; im(6,3,2) = gr6c3; im(6,4,2) = gr6c4;
im(6,5,2) = gr6c5; im(6,6,2) = gr6c6; im(6,7,2) = gr6c7; im(6,8,2) = gr6c8;
140 of 200

im(7,1,2) = gr7c1; im(7,2,2) = gr7c2; im(7,3,2) = gr7c3; im(7,4,2) = gr7c4;
im(7,5,2) = gr7c5; im(7,6,2) = gr7c6; im(7,7,2) = gr7c7; im(7,8,2) = gr7c8;
im(8,1,2) = gr8c1; im(8,2,2) = gr8c2; im(8,3,2) = gr8c3; im(8,4,2) = gr8c4;
im(8,5,2) = gr8c5; im(8,6,2) = gr8c6; im(8,7,2) = gr8c7; im(8,8,2) = gr8c8;
%BLUE PLANE COORDINATES
im(1,1,3) = br1c1; im(1,2,3) = br1c2; im(1,3,3) = br1c3; im(1,4,3) = br1c4;
im(1,5,3) = br1c5; im(1,6,3) = br1c6; im(1,7,3) = br1c7; im(1,8,3) = br1c8;
im(2,1,3) = br2c1; im(2,2,3) = br2c2; im(2,3,3) = br2c3; im(2,4,3) = br2c4;
im(2,5,3) = br2c5; im(2,6,3) = br2c6; im(2,7,3) = br2c7; im(2,8,3) = br2c8;
im(3,1,3) = br3c1; im(3,2,3) = br3c2; im(3,3,3) = br3c3; im(3,4,3) = br3c4;
im(3,5,3) = br3c5; im(3,6,3) = br3c6; im(3,7,3) = br3c7; im(3,8,3) = br3c8;
im(4,1,3) = br4c1; im(4,2,3) = br4c2; im(4,3,3) = br4c3; im(4,4,3) = br4c4;
im(4,5,3) = br4c5; im(4,6,3) = br4c6; im(4,7,3) = br4c7; im(4,8,3) = br4c8;
im(5,1,3) = br5c1; im(5,2,3) = br5c2; im(5,3,3) = br5c3; im(5,4,3) = br5c4;
im(5,5,3) = br5c5; im(5,6,3) = br5c6; im(5,7,3) = br5c7; im(5,8,3) = br5c8;
im(6,1,3) = br6c1; im(6,2,3) = br6c2; im(6,3,3) = br6c3; im(6,4,3) = br6c4;
im(6,5,3) = br6c5; im(6,6,3) = br6c6; im(6,7,3) = br6c7; im(6,8,3) = br6c8;
im(7,1,3) = br7c1; im(7,2,3) = br7c2; im(7,3,3) = br7c3; im(7,4,3) = br7c4;
im(7,5,3) = br7c5; im(7,6,3) = br7c6; im(7,7,3) = br7c7; im(7,8,3) = br7c8;
im(8,1,3) = br8c1; im(8,2,3) = br8c2; im(8,3,3) = br8c3; im(8,4,3) = br8c4;
im(8,5,3) = br8c5; im(8,6,3) = br8c6; im(8,7,3) = br8c7; im(8,8,3) = br8c8;
mixed {8} = im;
%The preceding image planes represents an 8-by-8 image.
numberOfFrames = 9;
for t = 1: numberOfFrames
%Load image
priortime = t-1;
imwrite (mixed {t}, (sprintf ('mixed%d.bmp', priortime)));
end
141 of 200

--Appendix B FPGA Test setup
library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
use IEEE.STD_LOGIC_ARITH.ALL;
use IEEE.STD_LOGIC_UNSIGNED.ALL;
Library XilinxCoreLib;
Library UNISIM;
use UNISIM.vcomponents.all;
---- Uncomment the following library declaration if instantiating
---- any Xilinx primitives in this code.
--library UNISIM;
--use UNISIM.VComponents.all;
entity top is
Port (USER_CLK: in STD_LOGIC;
DVI_RESET_B: in STD_LOGIC;
--xps_tft_0_TFT_DVI_CLK_N_pin:
STD_LOGIC;
DVI_D0: out STD_LOGIC;
DVI_D1: out STD_LOGIC;
DVI_D2: out STD_LOGIC;
DVI_D3: out STD_LOGIC;
DVI_D4: out STD_LOGIC;
DVI_D5: out STD_LOGIC;
DVI_D6: out STD_LOGIC;
DVI_D7: out STD_LOGIC;
DVI_D8: out STD_LOGIC;
DVI_D9: out STD_LOGIC;
DVI_D10: out STD_LOGIC;
DVI_D11: out STD_LOGIC;
DVI_H: out STD_LOGIC;
DVI_V: out STD_LOGIC;
-- DVI_XCLK_N: out STD_LOGIC;
DVI_DE: out STD_LOGIC;
DVI_XCLK_P: out STD_LOGIC
-- VREF: IN std_logic
);
end top;
out
architecture Structural of top is
--COMPONENT mem_of_pix_1024_f1x is
----generic (width: integer; addr_width: integer);
-- Port (clka: in STD_LOGIC;
-- addra: in STD_LOGIC_VECTOR (9 downto 0);
-- douta: out STD_LOGIC_VECTOR (23 downto 0));
COMPONENT mem_of_aer_stream IS
port (
clka: IN std_logic;
142 of 200

addra: IN std_logic_VECTOR (9 downto 0);
douta: OUT std_logic_VECTOR (159 downto 0));
END COMPONENT; --Representation of "mem_of_pix_1024_f1x" in AER format
COMPONENT pixelclock_dcm
Port (
CLKIN_IN: IN std_logic;
RST_IN: IN std_logic;
CLKDV_OUT: OUT std_logic;
CLKIN_IBUFG_OUT: OUT std_logic;
CLK0_OUT: OUT std_logic;
LOCKED_OUT: OUT std_logic
);
END COMPONENT;
COMPONENT get_spike_on_from_clock_25MHz is
Port (clock_25MHz: in STD_LOGIC;
spike_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT get_pixel_from_spike_on_clock is
Port (spike_clock: in STD_LOGIC;
sim_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT avga
PORT (
clock_25MHz: IN std_logic;
red: IN std_logic_vector (7 downto 0);
green: IN std_logic_vector (7 downto 0);
blue: IN std_logic_vector (7 downto 0);
red_out: OUT std_logic_vector (7 downto 0);
green_out: OUT std_logic_vector (7 downto 0);
blue_out: OUT std_logic_vector (7 downto 0);
horiz_sync_out: OUT std_logic;
vert_sync_out: OUT std_logic;
address: out std_logic_vector (9 downto 0);
address for ROM
video_on: out std_logic
--confirm viewable data
);
END COMPONENT;
--Single
COMPONENT AER_test_stream_to_pulse_count is
Port ( ts_clock: in STD_LOGIC;
AER_RGB_incoming_stream:
in
STD_LOGIC_VECTOR (159 downto 0);
stream_red_pulse_count: out std_logic_vector (7
downto 0); --Originally nine
stream_green_pulse_count: out std_logic_vector (7
downto 0); --Originally nine
143 of 200

stream_blue_pulse_count: out std_logic_vector (7
downto 0) --Originally nine
);
END COMPONENT;
COMPONENT convert_hue_pulses_to_intensity is
Port ( ts_clock: in STD_LOGIC;
stream_red_pc: in std_logic_vector (7 downto 0); --
Originally nine
stream_green_pc: in std_logic_vector (7 downto 0); --
Originally nine
stream_blue_pc: in std_logic_vector (7 downto 0); --
Originally nine
large_red_counter: out std_logic_vector (7 downto 0);
large_green_counter: out std_logic_vector (7 downto
0);
large_blue_counter: out std_logic_vector (7 downto 0)
);
END COMPONENT;
COMPONENT dvi_clk_from_vga_clock is
Port (user_clock: in STD_LOGIC;
data_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT dvimux is
Port (dvi_clk: IN STD_LOGIC;
r_out: in STD_LOGIC_VECTOR (7 downto 0);
g_out: in STD_LOGIC_VECTOR (7 downto 0);
b_out: in STD_LOGIC_VECTOR (7 downto 0);
--TFT_DVI_DATA: out STD_LOGIC_VECTOR (11
downto 0)
TFT_DVI_DATA_0: out std_logic;
TFT_DVI_DATA_1: out std_logic;
TFT_DVI_DATA_2: out std_logic;
TFT_DVI_DATA_3: out std_logic;
TFT_DVI_DATA_4: out std_logic;
TFT_DVI_DATA_5: out std_logic;
TFT_DVI_DATA_6: out std_logic;
TFT_DVI_DATA_7: out std_logic;
TFT_DVI_DATA_8: out std_logic;
TFT_DVI_DATA_9: out std_logic;
TFT_DVI_DATA_10: out std_logic;
TFT_DVI_DATA_11: out std_logic
);
END COMPONENT;
signal clock_25MHz: std_logic;
signal red_out: std_logic_vector (7 downto 0);
signal green_out: std_logic_vector (7 downto 0);
signal blue_out: std_logic_vector (7 downto 0);
signal red: std_logic_vector (7 downto 0);
signal green: std_logic_vector (7 downto 0);
144 of 200

egin
signal blue: std_logic_vector (7 downto 0);
signal addra: std_logic_vector (9 downto 0); -- (11 downto 0)
signal address: std_logic_vector (9 downto 0);
signal douta: std_logic_vector (23 downto 0);
signal pixel_data: std_logic_vector (23 downto 0);
signal video_on: std_logic;
signal short_add: std_logic_vector (9 downto 0):= "0000000000";
signal spike_clock: std_logic;
signal sim_clock: std_logic;
signal AER_RGB_stream: std_logic_vector (159 downto 0);
signal red_hue_out: std_logic_vector (7 downto 0);
signal green_hue_out: std_logic_vector (7 downto 0);
signal blue_hue_out: std_logic_vector (7 downto 0);
signal red_intensity: std_logic_vector (7 downto 0);
signal green_intensity: std_logic_vector (7 downto 0);
signal blue_intensity: std_logic_vector (7 downto 0);
signal DVI_DATA: std_logic_vector (11 downto 0);
signal to_dvi_clk: std_logic;
--------------------------------------------------
constant COLUMNLENGTH: std_logic_vector (5 downto 0):= "100000";
--VGA standard: 640 displayable pixels (800 actual)
constant offset: std_logic_vector (5 downto 0):= "001000"; -- (11 downto 0)
constant cue_dot: std_logic_vector (5 downto 0):= "001000";
constant pixcount: std_logic_vector (9 downto 0):= "1111111111";
signal count: integer: = 0;
Inst_pixelclock_dcm: pixelclock_dcm
port map (
CLKIN_IN => USER_CLK,
RST_IN => not DVI_RESET_B,
CLKDV_OUT => clock_25MHz
);
Inst_avga: avga PORT MAP (
clock_25MHz => clock_25MHz,
red => red,
green => green,
blue => blue,
red_out => red_out,
green_out => green_out,
blue_out => blue_out,
horiz_sync_out => DVI_H,
vert_sync_out => DVI_V,
address => address,
video_on => video_on
);
-- DVI_XCLK_N

DVI_XCLK_P clock_25MHz,
spike_clock => spike_clock
);
Inst_get_pixel_from_spike_on_clock: get_pixel_from_spike_on_clock
Port map (spike_clock => spike_clock,
sim_clock => sim_clock
);
--Inst_mem_of_pix_1024_f1x: mem_of_pix_1024_f1x PORT MAP (
----generic (width: integer; addr_width: integer);
--clka => clock_25MHz,
--addra => addra,
--douta => pixel_data
--);
Inst_mem_of_aer_stream: mem_of_aer_stream PORT MAP (
clka => clock_25MHz,
addra => address,
douta => AER_RGB_stream
);
--architecture behave of my_block is
--begin
Mem_read: process (clock_25MHz) is
Begin
-- if (DVI_RESET_B = '0') then
-- addra blue_hue_out --
stream_blue_pc
);
146 of 200

Inst_convert_hue_pulses_to_intensity: convert_hue_pulses_to_intensity
Port map ( ts_clock => spike_clock,
--spike_clock
stream_red_pc => red_hue_out,
stream_green_pc => green_hue_out,
stream_blue_pc => blue_hue_out,
large_red_counter => red_intensity,
large_green_counter => green_intensity,
large_blue_counter => blue_intensity
);
-- red blue_intensity,
--TFT_DVI_DATA => DVI_DATA
TFT_DVI_DATA_0 => DVI_D0,
TFT_DVI_DATA_1 => DVI_D1,
TFT_DVI_DATA_2 => DVI_D2,
TFT_DVI_DATA_3 => DVI_D3,
TFT_DVI_DATA_4 => DVI_D4,
TFT_DVI_DATA_5 => DVI_D5,
TFT_DVI_DATA_6 => DVI_D6,
TFT_DVI_DATA_7 => DVI_D7,
TFT_DVI_DATA_8 => DVI_D8,
TFT_DVI_DATA_9 => DVI_D9,
TFT_DVI_DATA_10 => DVI_D10,
TFT_DVI_DATA_11 => DVI_D11
);
end Structural;
----------------------------------------------------------------------------------
-- Create Date: 15:04:17 11/25/2008
-- Design Name:
-- Module Name: dvimux - Behavioral
library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
use IEEE.STD_LOGIC_ARITH.ALL;
147 of 200

use IEEE.STD_LOGIC_UNSIGNED.ALL;
---- Uncomment the following library declaration if instantiating
---- any Xilinx primitives in this code.
--library UNISIM;
--use UNISIM.VComponents.all;
entity dvimux is
Port (dvi_clk: IN STD_LOGIC;
r_out: in STD_LOGIC_VECTOR (7 downto 0);
g_out: in STD_LOGIC_VECTOR (7 downto 0);
b_out: in STD_LOGIC_VECTOR (7 downto 0);
--TFT_DVI_DATA: out STD_LOGIC_VECTOR (11
downto 0)
TFT_DVI_DATA_0: out std_logic;
TFT_DVI_DATA_1: out std_logic;
TFT_DVI_DATA_2: out std_logic;
TFT_DVI_DATA_3: out std_logic;
TFT_DVI_DATA_4: out std_logic;
TFT_DVI_DATA_5: out std_logic;
TFT_DVI_DATA_6: out std_logic;
TFT_DVI_DATA_7: out std_logic;
TFT_DVI_DATA_8: out std_logic;
TFT_DVI_DATA_9: out std_logic;
TFT_DVI_DATA_10: out std_logic;
TFT_DVI_DATA_11: out std_logic
);
end dvimux;
architecture Behavioral of dvimux is
begin
produce_12_bit_word_twice: process (dvi_clk, r_out, g_out, b_out)
variable TFT_DVI_DATA: std_logic_vector (11 downto 0);
begin
if dvi_clk = '1' then
--if (dvi_clk'event) AND (dvi_clk = '1') then
--if rising_edge (dvi_clk) then
--TFT_DVI_DATA:= r_out (7 downto 0) & g_out (7 downto 4);
TFT_DVI_DATA:= r_out (7 downto 6) & g_out (7 downto 6) & b_out (7 downto 6)
& r_out (5 downto 4) & g_out (5 downto 4) & b_out (5 downto 4);
--elsif (dvi_clk'event) AND (dvi_clk = '0') then
elsif dvi_clk = '0' then
--elsif falling_edge (dvi_clk) then --not supported in the current software release!
--TFT_DVI_DATA:= g_out (3 downto 0) & b_out (7 downto 0);
TFT_DVI_DATA:= r_out (3 downto 2) & g_out (3 downto 2) & b_out (3 downto 2)
& r_out (1 downto 0) & g_out (1 downto 0) & b_out (1 downto 0);
-- TFT_DVI_DATA_0

-- TFT_DVI_DATA_4

--Appendix C FPGA Sender chip
library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
use IEEE.STD_LOGIC_ARITH.ALL;
use IEEE.STD_LOGIC_UNSIGNED.ALL;
---- Uncomment the following library declaration if instantiating
---- any Xilinx primitives in this code.
--library UNISIM;
--use UNISIM.VComponents.all;
entity top_wrapper is
Port (PB_ENTER: in STD_LOGIC;
-- SYSTEM_CLOCK: in STD_LOGIC;
USER_CLK: in STD_LOGIC;
-- IO_L13P_11: out STD_LOGIC_VECTOR (11 downto 0);--
"L33"(1) [1]
--IO_L13N_11: out STD_LOGIC_VECTOR (11 downto 0);--"M32"(2) [2]
--IO_L14P_11: out STD_LOGIC_VECTOR (11 downto 0)--"P34"(3) [3]
IO_L14P_11: out STD_LOGIC_VECTOR (21 downto 0)--"P34"(3) [3]
);
end top_wrapper;
architecture Behavioral of top_wrapper is
COMPONENT mem_of_16_stream is
Port (clka: in STD_LOGIC;
addra: in STD_LOGIC_VECTOR (3 downto 0);
douta: out STD_LOGIC_VECTOR (23 downto 0));
END COMPONENT;
COMPONENT pixelclock_dcm is
port (CLKIN_IN : in std_logic;
RST_IN : in std_logic;
CLKDV_OUT : out std_logic;
CLKIN_IBUFG_OUT: out std_logic;
CLK0_OUT : out std_logic;
LOCKED_OUT : out std_logic);
END COMPONENT;
COMPONENT get_partial_spike_clock_from_incoming_clock is
Port (received_clock: in STD_LOGIC;
out_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT get_spike_on_from_partial_spike_clock is
Port (partial_spike_clock: in STD_LOGIC;
spike_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT get_pixel_clock_from_partial_spike_clock is
Port (partial_spike_clock: in STD_LOGIC;
pixel_clock: out STD_LOGIC);
150 of 200

END COMPONENT;
COMPONENT pulsar --Intensity values to
pulse count.
Port (pc_clk: in std_logic; --pc_clk maps to "spike_clock"
value: in STD_LOGIC_VECTOR (7 downto 0); --8 bits
ZZ: out std_logic_vector (7 downto 0)); --8 bits
END COMPONENT;
--The following component: "pulse_count_to_pc_stream" is the OUTGOING signal
COMPONENT pulse_count_to_pc_stream is
Port (pc_clk: in STD_LOGIC;
four_bit_address: in STD_LOGIC_VECTOR (3 downto 0);
R_count: in STD_LOGIC_VECTOR (7 downto 0);
G_count: in STD_LOGIC_VECTOR (7 downto 0);
B_count: in STD_LOGIC_VECTOR (7 downto 0);
pc_stream: out STD_LOGIC_VECTOR (21 downto 0) --formerly (33/23
downto 0)
); --Use "file_support_conventional_stream.vhd" to write to
"colour_stream.csv"
END COMPONENT;
--dout: out std_logic_vector ((3*data_width) - 1 downto 0));
constant clock_cycle: time: = 39.0625 ns; --25MHz has the periodic time 40ns
--Simulation time will be: 2500000ns
--signal show_p_data: std_logic_vector (23 downto 0);
signal pixel_data: std_logic_vector (23 downto 0);
signal incoming_clock: std_logic;
signal incoming_clock_done: boolean: = not TRUE;
signal partial_spike_clock: std_logic;
signal spike_clock: std_logic;
signal pixel_clock: std_logic;
signal resits: std_logic;
signal partial_clock_done: boolean: = not TRUE;
signal address: std_logic_vector (3 downto 0):= "0000";
signal addra: std_logic_vector (3 downto 0):= "0000";
constant reset_time: time: = 1*clock_cycle;
--Declaring the raw colour signals
signal red: std_logic_vector (7 downto 0);
signal green: std_logic_vector (7 downto 0);
signal blue: std_logic_vector (7 downto 0);
--Declaring the colour signals in terms of their pulse count
signal red_pulse_count:std_logic_vector (7 downto 0);
signal green_pulse_count:std_logic_vector (7 downto 0);
signal blue_pulse_count:std_logic_vector (7 downto 0);
signal pulse_count_stream: std_logic_vector (21 downto 0); --formerly (33/23
downto 0)
begin
--tb_clk: process is
--begin
-- incoming_clock

-- else
-- resit '1',
CLKDV_OUT => incoming_clock
-- CLKIN_IBUFG_OUT =>,
-- CLK0_OUT =>,
-- LOCKED_OUT =>
);
Inst_get_partial_spike_clock_from_incoming_clock:
get_partial_spike_clock_from_incoming_clock
Port map (received_clock => incoming_clock,
out_clock => partial_spike_clock
);
Inst_get_spike_on_from_partial_spike_clock:
get_spike_on_from_partial_spike_clock
Port map (partial_spike_clock => partial_spike_clock,
spike_clock => spike_clock
);
Inst_get_pixel_clock_from_partial_spike_clock:
get_pixel_clock_from_partial_spike_clock
Port map (partial_spike_clock => partial_spike_clock,
pixel_clock => pixel_clock
);
Mem_read: process (pixel_clock, address) is
Begin
-- if (clock'event) AND (clock = '1') then
if rising_edge (pixel_clock) then --As each pixel relates to an
addressed `packet'.
address address,
douta => pixel_data
);
Inst_pulsar_red: pulsar PORT MAP (
pc_clk => spike_clock,
value => pixel_data (23 downto 16), --Red
152 of 200

ZZ => red_pulse_count
);
Inst_pulsar_green: pulsar PORT MAP (
pc_clk => spike_clock,
value => pixel_data (15 downto 8), --Green
ZZ => green_pulse_count
);
Inst_pulsar_blue: pulsar PORT MAP (
pc_clk => spike_clock,
value => pixel_data (7 downto 0), --Blue
ZZ => blue_pulse_count
);
--The following component: "pulse_count_to_pc_stream" is the OUTGOING signal
Inst_pulse_count_to_pc_stream: pulse_count_to_pc_stream
Port map (pc_clk => spike_clock,
four_bit_address => address,
R_count => red_pulse_count,
G_count => green_pulse_count,
B_count => blue_pulse_count,
pc_stream => pulse_count_stream -- Last eighteen bits are colour information
--That is after the address information
);
-- red

FPGA Receiver (structural) chip
--Appendix D FPGA Receiver chip
library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
--use IEEE.STD_LOGIC_ARITH.ALL;
--use IEEE.STD_LOGIC_UNSIGNED.ALL;
use IEEE.STD_LOGIC_SIGNED.ALL;
-- Uncomment the following library declaration if instantiating
---- any Xilinx primitives in this code.
--library UNISIM;
--use UNISIM.VComponents.all;
--Replacing 146 occurrences of (19 downto 0) with (3 downto 0)!
--"short_AER_RGB_stream"
use IEEE.numeric_std.ALL; --to_signed is used with this library
entity top_wrapper is
Port (USER_CLK: in STD_LOGIC;
IO_L13P_11: out STD_LOGIC_VECTOR (3 downto 0);--"L33"(1) [1]
IO_L13N_11: out STD_LOGIC_VECTOR (3 downto 0);--"M32"(2) [2]
IO_L14P_11: out STD_LOGIC_VECTOR (3 downto 0);--"P34"(3) [3]
IO_L13P_12: out STD_LOGIC_VECTOR (3 downto 0);--"H5"(1) [4]
IO_L13N_12: out STD_LOGIC_VECTOR (3 downto 0);--"G5"(2) [5]
IO_L14P_12: out STD_LOGIC_VECTOR (3 downto 0);--"R11"(3) [6]
IO_L15P_12: out STD_LOGIC_VECTOR (3 downto 0);--"F5"(4) [7]
IO_L15N_12: out STD_LOGIC_VECTOR (3 downto 0);--"F6"(5) [8]
IO_L16P_12: out STD_LOGIC_VECTOR (3 downto 0);--"T10"(6) [9]
IO_L16N_12: out STD_LOGIC_VECTOR (3 downto 0);--"T11"(7) [10]
IO_L17P_12: out STD_LOGIC_VECTOR (3 downto 0);--"G6"(8) [11]
IO_L17N_12: out STD_LOGIC_VECTOR (3 downto 0);--"G7"(9) [12]
IO_L18P_12: out STD_LOGIC_VECTOR (3 downto 0);--"T9"(10) [13]
IO_L18N_12: out STD_LOGIC_VECTOR (3 downto 0);--"U10"(11) [14]
IO_L19P_12: out STD_LOGIC_VECTOR (3 downto 0);--"E6"(12) [15]
IO_L19N_12: out STD_LOGIC_VECTOR (3 downto 0);--"E7"(13) [16]
IO_L13P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AK34” (1) [17]
IO_L13N_13: out STD_LOGIC_VECTOR (3 downto 0);--“AK33” (2) [18]
IO_L14P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AG32” (3) [19]
IO_L15P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AJ32” (4) [20]
IO_L15N_13: out STD_LOGIC_VECTOR (3 downto 0);--“AK32” (5) [21]
IO_L16P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AL34” (6) [22]
IO_L16N_13: out STD_LOGIC_VECTOR (3 downto 0);--“AL33” (7) [23]
IO_L17P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AM33” (8) [24]
IO_L17N_13: out STD_LOGIC_VECTOR (3 downto 0);--“AM32” (9) [25]
IO_L18P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AN34” (10) [26]
IO_L18N_13: out STD_LOGIC_VECTOR (3 downto 0);--“AN33” (11) [27]
IO_L19P_13: out STD_LOGIC_VECTOR (3 downto 0);--“AN32” (12) [28]
IO_L19N_13: out STD_LOGIC_VECTOR (3 downto 0);--“AP32” (13) [29]
IO_L13P_15: out STD_LOGIC_VECTOR (3 downto 0);--“T31” (1) [30]
IO_L13N_15: out STD_LOGIC_VECTOR (3 downto 0);--“R31” (2) [31]
IO_L14P_15: out STD_LOGIC_VECTOR (3 downto 0);--“U30” (3) [32]
IO_L15P_15: out STD_LOGIC_VECTOR (3 downto 0);--“T28” (4) [33]
IO_L15N_15: out STD_LOGIC_VECTOR (3 downto 0);--“T29” (5) [34]
IO_L16P_15: out STD_LOGIC_VECTOR (3 downto 0);--“U27” (6) [35]
154 of 200

FPGA Receiver (structural) chip
IO_L16N_15: out STD_LOGIC_VECTOR (3 downto 0);--“U28” (7) [36]
IO_L17P_15: out STD_LOGIC_VECTOR (3 downto 0);--“R26” (8) [37]
IO_L17N_15: out STD_LOGIC_VECTOR (3 downto 0);--“R27” (9) [38]
IO_L18P_15: out STD_LOGIC_VECTOR (3 downto 0);--“U26” (10) [39]
IO_L18N_15: out STD_LOGIC_VECTOR (3 downto 0);--“T26” (11) [40]
IO_L19P_15: out STD_LOGIC_VECTOR (3 downto 0);--“U25” (12) [41]
IO_L19N_15: out STD_LOGIC_VECTOR (3 downto 0);--“T25” (13) [42]
IO_L13P_17: out STD_LOGIC_VECTOR (3 downto 0);--“AD30” (1) [43]
IO_L13N_17: out STD_LOGIC_VECTOR (3 downto 0);--"AC29}"(2) [44]
IO_L14P_17: out STD_LOGIC_VECTOR (3 downto 0);--"AF31"(3) [45]
IO_L15P_17: out STD_LOGIC_VECTOR (3 downto 0);--"AE29"(4) [46]
IO_L15N_17: out STD_LOGIC_VECTOR (3 downto 0);--"AD29"(5) [47]
IO_L16P_17: out STD_LOGIC_VECTOR (3 downto 0)--"AJ31"(6) [48]
--IO_L13P_18: out STD_LOGIC_VECTOR (999 downto 0);--"Y11" {1}
--IO_L13P_19: out STD_LOGIC_VECTOR (999 downto 0);--"K28" [215]
--IO_L13N_19: out STD_LOGIC_VECTOR (999 downto 0);--"L28" {3}
--IO_L14P_19: out STD_LOGIC_VECTOR (999 downto 0);--"K27" [215]
--IO_L15P_19: out STD_LOGIC_VECTOR (999 downto 0);--"M28" [215]
--IO_L17P_20: out STD_LOGIC_VECTOR (999 downto 0);--"E12" {6}
--IO_L17N_20: out STD_LOGIC_VECTOR (999 downto 0);--"E13" {7}
--IO_L18P_20: out STD_LOGIC_VECTOR (999 downto 0);--"N10" {8}
--IO_L18N_20: out STD_LOGIC_VECTOR (999 downto 0); --"N9" {9}
--IO_L19P_20: out STD_LOGIC_VECTOR (999 downto 0);--"F13" {10}
--IO_L19N_20: out STD_LOGIC_VECTOR (999 downto 0);--"G13" {11}
--IO_L13P_21: out STD_LOGIC_VECTOR (999 downto 0);--"AF24" {12}
--IO_L13N_21: out STD_LOGIC_VECTOR (999 downto 0);--"AG25" {13}
--IO_L14P_21: out STD_LOGIC_VECTOR (999 downto 0);--"AG27" {14}
--IO_L15P_21: out STD_LOGIC_VECTOR (999 downto 0);--"AF25" {15}
--IO_L15N_21: out STD_LOGIC_VECTOR (999 downto 0);--"AF26" {16}
--IO_L16P_21: out STD_LOGIC_VECTOR (999 downto 0);--"AE27" {17}
--IO_L16N_21: out STD_LOGIC_VECTOR (999 downto 0);--"AE26" {18}
--IO_L17P_21: out STD_LOGIC_VECTOR (999 downto 0);--"AC25" {19}
--IO_L17N_21: out STD_LOGIC_VECTOR (999 downto 0);--"AC24" {20}
--IO_L18P_21: out STD_LOGIC_VECTOR (999 downto 0);--"AD26" {21}
);
end top_wrapper;
--1024 x 3 = 3072 i.e. three planes describing a 64 pixelcount picture
--As a pixel consists of three planes, 150 potential pulses are required per pixel
--That is 50 potential pulses per plane per pixel
architecture Behavioral of top_wrapper is
--type current integer range -70 uA to +70 uA
-- units
-- uA;
-- mA = 1000uA;
-- end units;
--COMPONENT mem_of_256_short_aer_stream is
----generic (width: integer; addr_width: integer);
-- Port (clka: in STD_LOGIC;
-- addra: in STD_LOGIC_VECTOR (7 downto 0);
-- douta: out STD_LOGIC_VECTOR (25 downto 0));
155 of 200

FPGA Receiver (structural) chip
--END COMPONENT;--Pulse count representation
COMPONENT mem_of_256_short_aer_stream is
--generic (width: integer; addr_width: integer);
Port (clka: in STD_LOGIC;
addra: in STD_LOGIC_VECTOR (1 downto 0);
douta: out STD_LOGIC_VECTOR (21 downto 0));
END COMPONENT;--Pulse count representation
COMPONENT pixelclock_dcm is
port (CLKIN_IN : in std_logic;
RST_IN : in std_logic;
CLKDV_OUT : out std_logic;
CLKIN_IBUFG_OUT: out std_logic;
CLK0_OUT : out std_logic;
LOCKED_OUT : out std_logic);
END COMPONENT;
COMPONENT get_partial_spike_clock_from_incoming_clock is
Port (received_clock: in STD_LOGIC;
out_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT get_pixel_clock_from_partial_spike_clock is
Port (partial_spike_clock: in STD_LOGIC;
pixel_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT get_spike_on_from_partial_spike_clock is
Port (partial_spike_clock: in STD_LOGIC;
spike_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT get_long_spike is
Port (partial_spike_clock: in STD_LOGIC;
long_spike_clock: out STD_LOGIC);
END COMPONENT;
COMPONENT preform_for_electrodes
port (preform_clock: std_logic;
incoming_stream: in std_logic_vector (153 downto
0);--Four by four!
--THE ADDRESSES FOLLOWING MUST REACH
4 ELECTRODES!
red_with_address: out std_logic_vector (57 downto 0);
green_with_address: out std_logic_vector (57 downto
0);
blue_with_address: out std_logic_vector (57 downto 0)
--58*3 equals 174 bits
-- out_with_rgb_address: out std_logic_vector
(173 downto 0)
);--Goes to "produce_colour_data_streams"
END COMPONENT;
COMPONENT produce_colour_data_streams --feeds "extend_data_streams"
port (spike_data_clock: std_logic;
red: in std_logic_vector (57 downto 0);
green: in std_logic_vector (57 downto 0);
156 of 200

FPGA Receiver (structural) chip
blue: in std_logic_vector (57 downto 0);
red_data_out: out std_logic_vector (49 downto 0);
green_data_out: out std_logic_vector (49 downto 0);
blue_data_out: out std_logic_vector (49 downto 0)
);--picking out separate data
END COMPONENT;
COMPONENT extend_data_streams
port (this_spike_clock: in std_logic;
red_data_incoming: in STD_LOGIC_VECTOR (49 downto
0);
green_data_incoming: in STD_LOGIC_VECTOR (49
downto 0);
blue_data_incoming: in STD_LOGIC_VECTOR (49 downto
0);
extended_red: out STD_LOGIC_VECTOR (199 downto 0);
extended_green: out STD_LOGIC_VECTOR (199 downto
0);
extended_blue: out STD_LOGIC_VECTOR (199 downto 0)
);
END COMPONENT; --Prior to forming output stream
COMPONENT fully_extend_data_streams is
Port (this_spike_clock: in STD_LOGIC;
red_data_incoming: in STD_LOGIC_VECTOR (49 downto 0);
green_data_incoming: in STD_LOGIC_VECTOR (49 downto 0);
blue_data_incoming: in STD_LOGIC_VECTOR (49 downto 0);
extended_red: out STD_LOGIC_VECTOR (999 downto 0);
extended_green: out STD_LOGIC_VECTOR (999 downto 0);
extended_blue: out STD_LOGIC_VECTOR (999 downto 0));
END COMPONENT;
COMPONENT file_support_fully_extended_data_streams is
Port (a_pixel_clock: in STD_LOGIC;--pixel_clock
red_pulse_stream_incoming: in STD_LOGIC_VECTOR (999 downto 0);
green_pulse_stream_incoming: in STD_LOGIC_VECTOR (999 downto 0);
blue_pulse_stream_incoming: in STD_LOGIC_VECTOR (999 downto 0));
END COMPONENT;
COMPONENT file_support_squeeze is
Port (a_pixel_clock: in STD_LOGIC;--pixel_clock
red_pulse_stream_incoming: in STD_LOGIC_VECTOR (7999 downto 0);
green_pulse_stream_incoming: in STD_LOGIC_VECTOR (7999 downto 0);
blue_pulse_stream_incoming: in STD_LOGIC_VECTOR (7999 downto 0));
END COMPONENT;
COMPONENT produce_wire_outputs is
Port (wire_clock: in STD_LOGIC;
biphasic_clock: in std_logic;
red: in STD_LOGIC_VECTOR (999 downto 0);--from
"outer_red_pulse_stream"
green: in STD_LOGIC_VECTOR (999 downto 0);--from
"outer_green_pulse_stream"
blue: in STD_LOGIC_VECTOR (999 downto 0);--from
"outer_blue_pulse_stream"
157 of 200

FPGA Receiver (structural) chip
--red_1: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "red_wire_1"
--red_2: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "red_wire_2"
--red_3: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "red_wire_3"
--red_4: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "red_wire_4"
--green_1: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "green_wire_1"
--green_2: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "green_wire_2"
--green_3: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "green_wire_3"
--green_4: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "green_wire_4"
--blue_1: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "blue_wire_1"
--blue_2: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "blue_wire_2"
--blue_3: out STD_LOGIC_VECTOR (999 downto 0);--feeds to "blue_wire_3"
--blue_4: out STD_LOGIC_VECTOR (999 downto 0)--feeds to "blue_wire_4"
red_1: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_1"
red_2: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_2"
red_3: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_3"
red_4: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_4"
red_5: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_4"
red_6: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_6"
red_7: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_7"
red_8: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_8"
red_9: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_9"
red_10: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_10"
red_11: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_11"
red_12: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_12"
red_13: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_13"
red_14: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_14"
red_15: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_15"
red_16: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "red_wire_16"
green_1: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_1"
green_2: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_2"
green_3: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_3"
green_4: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_4"
green_5: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_5"
green_6: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_6"
green_7: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_7"
green_8: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_8"
green_9: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_9"
green_10: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_10"
green_11: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_11"
green_12: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_12"
green_13: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_13"
green_14: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_14"
green_15: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_15"
green_16: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "green_wire_16"
blue_1: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_1"
blue_2: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_2"
blue_3: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_3"
blue_4: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_4"
blue_5: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_5"
blue_6: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_6"
blue_7: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_7"
158 of 200

FPGA Receiver (structural) chip
blue_8: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_8"
blue_9: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_9"
blue_10: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_10"
blue_11: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_11"
blue_12: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_12"
blue_13: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_13"
blue_14: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_14"
blue_15: out STD_LOGIC_VECTOR (3 downto 0);--feeds to "blue_wire_15"
blue_16: out STD_LOGIC_VECTOR (3 downto 0)--feeds to "blue_wire_16"
);
END COMPONENT; --produce_wire_outputs
COMPONENT produce_fast_wire_outputs is
Port (wire_clock: in STD_LOGIC;
red: in STD_LOGIC_VECTOR (199 downto 0);--from
"outer_red_pulse_stream"
green: in STD_LOGIC_VECTOR (199 downto 0);--from
"outer_green_pulse_stream"
blue: in STD_LOGIC_VECTOR (199 downto 0);--from
"outer_blue_pulse_stream"
red_1: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "red_wire_1"
red_2: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "red_wire_2"
red_3: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "red_wire_3"
red_4: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "red_wire_4"
green_1: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "green_wire_1"
green_2: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "green_wire_2"
green_3: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "green_wire_3"
green_4: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "green_wire_4"
blue_1: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "blue_wire_1"
blue_2: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "blue_wire_2"
blue_3: out STD_LOGIC_VECTOR (199 downto 0);--feeds to "blue_wire_3"
blue_4: out STD_LOGIC_VECTOR (199 downto 0)--feeds to "blue_wire_4"
);
END COMPONENT; --produce_fast_wire_outputs
COMPONENT produce_bit_wire_outputs is
Port (wire_clock: in STD_LOGIC;
red: in STD_LOGIC_VECTOR (49 downto 0);--from
"outer_red_pulse_stream"
green: in STD_LOGIC_VECTOR (49 downto 0);--from
"outer_green_pulse_stream"
blue: in STD_LOGIC_VECTOR (49 downto 0);--from
"outer_blue_pulse_stream"
red_1: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "red_wire_1"
red_2: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "red_wire_2"
red_3: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "red_wire_3"
red_4: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "red_wire_4"
green_1: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "green_wire_1"
green_2: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "green_wire_2"
green_3: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "green_wire_3"
green_4: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "green_wire_4"
blue_1: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "blue_wire_1"
blue_2: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "blue_wire_2"
159 of 200

FPGA Receiver (structural) chip
blue_3: out STD_LOGIC_VECTOR (49 downto 0);--feeds to "blue_wire_3"
blue_4: out STD_LOGIC_VECTOR (49 downto 0)--feeds to "blue_wire_4"
);
END COMPONENT; --produce_bit_wire_outputs
COMPONENT AER_test_stream_to_pulse_count is
--feeds "file_support_intense_pulses.vhd" & hence "intense_pulses.dat
generic (bit_width: integer: = 11); --Initially six, then altered to eleven.
Port ( ts_clock: in STD_LOGIC;
AER_RGB_incoming_stream:
in
STD_LOGIC_VECTOR (155 downto 0);
stream_red_pulse_count: out std_logic_vector (0
downto 0);
stream_green_pulse_count: out std_logic_vector (0
downto 0);
stream_blue_pulse_count: out std_logic_vector (0
downto 0);
out_red_counter: out integer;
out_green_counter: out integer;
out_blue_pulse_counter: out integer
);
END COMPONENT;
COMPONENT mapping_to_outgoing_addresses is
Port (implant_clk: in STD_LOGIC; --pixel clock
red_in: in STD_LOGIC_VECTOR (55 downto 0);
green_in: in STD_LOGIC_VECTOR (55 downto 0);
blue_in: in STD_LOGIC_VECTOR (55 downto 0);
red_outgoing_address_stream: in STD_LOGIC_VECTOR (5
downto 0);
green_outgoing_address_stream: in STD_LOGIC_VECTOR
(5 downto 0);
blue_outgoing_address_stream: in STD_LOGIC_VECTOR
(5 downto 0);
red_implant: out STD_LOGIC_VECTOR (55 downto 0);
green_implant: out STD_LOGIC_VECTOR (55 downto 0);
blue_implant: out STD_LOGIC_VECTOR (55 downto 0));
END COMPONENT;
COMPONENT generate_outgoing_address_streams is
--feeds "file_support_AER_outgoing_addresses.vhd" & hence
"outgoing_addresses.dat"
Port ( out_clock: in STD_LOGIC; --spike_clock
red_out_address: out std_logic_vector (5 downto 0);
green_out_address: out std_logic_vector (5 downto 0);
blue_out_address: out std_logic_vector (5 downto 0)
);
END COMPONENT;
COMPONENT file_support_AER_outgoing_addresses is
port (clk: in std_logic; --Sim_clock
160 of 200

FPGA Receiver (structural) chip
0);
downto 0);
0)
);
END COMPONENT;
red_outgoing_address: IN std_logic_vector (5 downto
green_outgoing_address: IN std_logic_vector (5
blue_outgoing_address: IN std_logic_vector (5 downto
COMPONENT get_rc_addresses_with_data is --Starting at the ZERO'TH
single_address.
Port (address_with_data: in STD_LOGIC_VECTOR (61 downto 0) ;
row_address: out STD_LOGIC_VECTOR (5 downto 0);
column_address: out STD_LOGIC_VECTOR (5 downto 0)
); --Fed from "AER_sep_streams_with_incoming_addresses"
END COMPONENT;
COMPONENT file_support_wire_outputs is
Port (an_pixel_clock: in STD_LOGIC;--pixel_clock
in_red_wire_1: in STD_LOGIC_VECTOR (49 downto 0);
in_red_wire_2: in STD_LOGIC_VECTOR (49 downto 0);
in_red_wire_3: in STD_LOGIC_VECTOR (49 downto 0);
in_red_wire_4: in STD_LOGIC_VECTOR (49 downto 0);
in_green_wire_1: in STD_LOGIC_VECTOR (49 downto 0);
in_green_wire_2: in STD_LOGIC_VECTOR (49 downto 0);
in_green_wire_3: in STD_LOGIC_VECTOR (49 downto 0);
in_green_wire_4: in STD_LOGIC_VECTOR (49 downto 0);
in_blue_wire_1: in STD_LOGIC_VECTOR (49 downto 0);
in_blue_wire_2: in STD_LOGIC_VECTOR (49 downto 0);
in_blue_wire_3: in STD_LOGIC_VECTOR (49 downto 0);
in_blue_wire_4: in STD_LOGIC_VECTOR (49 downto 0)
);
END COMPONENT; --end file_support_wire_outputs;
COMPONENT pulse_count_to_AER_stream is
Port (
pcs_clk: in STD_LOGIC;
single_address: in STD_LOGIC_VECTOR (3 downto
0); --short_AER_RGB_stream (19 downto 18)
R_pulse_count: in STD_LOGIC_VECTOR (5 downto
0); --short_AER_RGB_stream (17 downto 12)
G_pulse_count: in STD_LOGIC_VECTOR (5 downto
0);--short_AER_RGB_stream (11 downto 6)
B_pulse_count: in STD_LOGIC_VECTOR (5 downto
0); --short_AER_RGB_stream (5 downto 0)
AER_RGB_pulse_stream: out STD_LOGIC_VECTOR
(153 downto 0)
);
END COMPONENT;
subtype limited_current is integer range -70 to +70;
--signal num: limited_current;
--pixel time equals 39062.5 ns (@25fps), note that 25MHz has the periodic time of
40ns approx.
161 of 200

FPGA Receiver (structural) chip
constant clock_cycle: time: = 39.0625 ns; --formerly: =39062.5 ns; 25MHz has the
periodic time of 40ns.
--note that 20*Tclock = 781.25 ns (Tspike at 5fps for a 32 by 32 scene)
constant reset_time: time: = 1*clock_cycle;
signal pixel_total: integer: = 16;
signal height_in_rows: integer: = 2;
signal width_in_columns: integer: = 2;
signal row_length: integer: = 2;
signal clka: std_logic;
signal addra: std_logic_vector (1 downto 0):= "00";
signal address: std_logic_vector (1 downto 0):= "00";
signal resits: std_logic;
signal pixel_clock: std_logic;
signal pixel_done: boolean: = not TRUE;
signal resit_row_cluster_done: boolean: = not TRUE;
signal incoming_clock: std_logic;
signal incoming_clock_done: boolean: = not TRUE;
signal partial_spike_clock: std_logic;
--signal partial_clock_done: boolean: = not TRUE;
signal resit_part_spike: std_logic;
signal spike_clock: std_logic;
signal resit_spike: std_logic;
signal outerpulse_clock: std_logic;
signal row_cluster_clock: std_logic;
signal resit_outerpulse_spike: std_logic;
signal sel: std_logic:= '0';
----signal pixel_row_address: std_logic_vector (5 downto 0);
----signal pixel_column_address: std_logic_vector (5 downto 0);
----signal combined_pulse_stream: std_logic_vector (649 downto 0);
signal AER_RGB_stream: std_logic_vector (153 downto 0);
signal out_red_pc: std_logic_vector (9 downto 0);
signal out_green_pc: std_logic_vector (9 downto 0);
signal out_blue_pc: std_logic_vector (9 downto 0);
signal short_AER_RGB_stream: std_logic_vector (23 downto 0);
signal douta: STD_LOGIC_VECTOR (21 downto 0);
signal red_pulse_data: std_logic_vector (49 downto 0);
signal green_pulse_data: std_logic_vector (49 downto 0);
signal blue_pulse_data: std_logic_vector (49 downto 0);
signal stream_red_pc: std_logic_vector (9 downto 0);
signal stream_green_pc: std_logic_vector (9 downto 0);
signal stream_blue_pc: std_logic_vector (9 downto 0);
signal clk: std_logic;
signal out_red_counter: integer: = 0;
signal out_green_counter: integer: = 0;
signal out_blue_counter: integer: = 0;
signal hue_red_pc: integer;
signal hue_green_pc: integer;
signal hue_blue_pc: integer;
signal red_counter: std_logic_vector (9 downto 0);
signal green_counter: std_logic_vector (9 downto 0);
signal blue_counter: std_logic_vector (9 downto 0);
162 of 200

FPGA Receiver (structural) chip
signal stream_red_pulse_count: std_logic_vector (9 downto 0);
signal stream_green_pulse_count: std_logic_vector (9 downto 0);
signal stream_blue_pulse_count: std_logic_vector (9 downto 0);
--For a two by two image the address bits allowed are two bits
signal red_incoming_address: std_logic_vector (51 downto 0);
signal green_incoming_address: std_logic_vector (51 downto 0);
signal blue_incoming_address: std_logic_vector (51 downto 0);
signal red_feed: std_logic_vector (59 downto 0);
signal green_feed: std_logic_vector (59 downto 0);
signal blue_feed: std_logic_vector (59 downto 0);
signal rc_red_feed: std_logic_vector (61 downto 0);
signal rc_green_feed: std_logic_vector (61 downto 0);
signal rc_blue_feed: std_logic_vector (61 downto 0);
signal red_with_rc_address: std_logic_vector (61 downto 0);
signal green_with_rc_address: std_logic_vector (61 downto 0);
signal blue_with_rc_address: std_logic_vector (61 downto 0);
signal red_between_address: std_logic_vector (9 downto 0);
signal green_between_address: std_logic_vector (9 downto 0);
signal blue_between_address: std_logic_vector (9 downto 0);
--Signals to implant
signal red_data_preform: std_logic_vector (49 downto 0);
signal green_data_preform: std_logic_vector (49 downto 0);
signal blue_data_preform: std_logic_vector (49 downto 0);
--58*3 equals 174 {three in an rgb packet}
signal composite_addressed_stream: std_logic_vector (173 downto
0):="0000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000";
--THE ADDRESSES FOLLOWING MUST REACH 4 ELECTRODES!
signal red_1_4_addressed: std_logic_vector (57 downto 0);
signal green_2_5_addressed: std_logic_vector (57 downto 0);
signal blue_3_6_addressed: std_logic_vector (57 downto 0);
signal red_data: std_logic_vector (49 downto 0);
signal green_data: std_logic_vector (49 downto 0);
signal blue_data: std_logic_vector (49 downto 0);
signal red_pulse_stream: std_logic_vector (199 downto 0);
signal green_pulse_stream: std_logic_vector (199 downto 0);
signal blue_pulse_stream: std_logic_vector (199 downto 0);
signal pic_size: std_logic_vector (5 downto 0):= "000000";--Six bits;
signal qpulse_size: std_logic_vector (15 downto 0):= "0000000000000000";--
Sixteen bits
signal red_x_out: std_logic_vector (999 downto 0);
signal green_y_out: std_logic_vector (999 downto 0);
signal blue_z_out: std_logic_vector (999 downto 0);
signal number_out: std_logic_vector (0 downto 0):="0";
signal outer_red_pulse_stream: std_logic_vector (999 downto 0);
signal outer_green_pulse_stream: std_logic_vector (999 downto 0);
signal outer_blue_pulse_stream: std_logic_vector (999 downto 0);
signal send_num_out: std_logic_vector (0 downto 0);
signal send_num_unsigned_out: std_logic_vector (0 downto 0);
signal zero_out: std_logic_vector (0 downto 0);
163 of 200

FPGA Receiver (structural) chip
--signal red_wire_1: std_logic_vector (999 downto 0);
--signal red_wire_2: std_logic_vector (999 downto 0);
--signal red_wire_3: std_logic_vector (999 downto 0);
--signal red_wire_4: std_logic_vector (999 downto 0);
--signal green_wire_1: std_logic_vector (999 downto 0);
--signal green_wire_2: std_logic_vector (999 downto 0);
--signal green_wire_3: std_logic_vector (999 downto 0);
--signal green_wire_4: std_logic_vector (999 downto 0);
--signal blue_wire_1: std_logic_vector (999 downto 0);
--signal blue_wire_2: std_logic_vector (999 downto 0);
--signal blue_wire_3: std_logic_vector (999 downto 0);
--signal blue_wire_4: std_logic_vector (999 downto 0);
--signal red_wire_1: std_logic_vector (199 downto 0);
--signal red_wire_2: std_logic_vector (199 downto 0);
--signal red_wire_3: std_logic_vector (199 downto 0);
--signal red_wire_4: std_logic_vector (199 downto 0);
--signal green_wire_1: std_logic_vector (199 downto 0);
--signal green_wire_2: std_logic_vector (199 downto 0);
--signal green_wire_3: std_logic_vector (199 downto 0);
--signal green_wire_4: std_logic_vector (199 downto 0);
--signal blue_wire_1: std_logic_vector (199 downto 0);
--signal blue_wire_2: std_logic_vector (199 downto 0);
--signal blue_wire_3: std_logic_vector (199 downto 0);
--signal blue_wire_4: std_logic_vector (199 downto 0);
--signal red_wire_1: std_logic_vector (3 downto 0);
--signal red_wire_2: std_logic_vector (3 downto 0);
--signal red_wire_3: std_logic_vector (3 downto 0);
--signal red_wire_4: std_logic_vector (3 downto 0);
--signal green_wire_1: std_logic_vector (3 downto 0);
--signal green_wire_2: std_logic_vector (3 downto 0);
--signal green_wire_3: std_logic_vector (3 downto 0);
--signal green_wire_4: std_logic_vector (3 downto 0);
--signal blue_wire_1: std_logic_vector (3 downto 0);
--signal blue_wire_2: std_logic_vector (3 downto 0);
--signal blue_wire_3: std_logic_vector (3 downto 0);
--signal blue_wire_4: std_logic_vector (3 downto 0);
signal red_wire_1: std_logic_vector (3 downto 0);
signal red_wire_2: std_logic_vector (3 downto 0);
signal red_wire_3: std_logic_vector (3 downto 0);
signal red_wire_4: std_logic_vector (3 downto 0);
signal red_wire_5: std_logic_vector (3 downto 0);
signal red_wire_6: std_logic_vector (3 downto 0);
signal red_wire_7: std_logic_vector (3 downto 0);
signal red_wire_8: std_logic_vector (3 downto 0);
signal red_wire_9: std_logic_vector (3 downto 0);
signal red_wire_10: std_logic_vector (3 downto 0);
signal red_wire_11: std_logic_vector (3 downto 0);
signal red_wire_12: std_logic_vector (3 downto 0);
signal red_wire_13: std_logic_vector (3 downto 0);
signal red_wire_14: std_logic_vector (3 downto 0);
signal red_wire_15: std_logic_vector (3 downto 0);
164 of 200

FPGA Receiver (structural) chip
signal red_wire_16: std_logic_vector (3 downto 0);
signal green_wire_1: std_logic_vector (3 downto 0);
signal green_wire_2: std_logic_vector (3 downto 0);
signal green_wire_3: std_logic_vector (3 downto 0);
signal green_wire_4: std_logic_vector (3 downto 0);
signal green_wire_5: std_logic_vector (3 downto 0);
signal green_wire_6: std_logic_vector (3 downto 0);
signal green_wire_7: std_logic_vector (3 downto 0);
signal green_wire_8: std_logic_vector (3 downto 0);
signal green_wire_9: std_logic_vector (3 downto 0);
signal green_wire_10: std_logic_vector (3 downto 0);
signal green_wire_11: std_logic_vector (3 downto 0);
signal green_wire_12: std_logic_vector (3 downto 0);
signal green_wire_13: std_logic_vector (3 downto 0);
signal green_wire_14: std_logic_vector (3 downto 0);
signal green_wire_15: std_logic_vector (3 downto 0);
signal green_wire_16: std_logic_vector (3 downto 0);
signal blue_wire_1: std_logic_vector (3 downto 0);
signal blue_wire_2: std_logic_vector (3 downto 0);
signal blue_wire_3: std_logic_vector (3 downto 0);
signal blue_wire_4: std_logic_vector (3 downto 0);
signal blue_wire_5: std_logic_vector (3 downto 0);
signal blue_wire_6: std_logic_vector (3 downto 0);
signal blue_wire_7: std_logic_vector (3 downto 0);
signal blue_wire_8: std_logic_vector (3 downto 0);
signal blue_wire_9: std_logic_vector (3 downto 0);
signal blue_wire_10: std_logic_vector (3 downto 0);
signal blue_wire_11: std_logic_vector (3 downto 0);
signal blue_wire_12: std_logic_vector (3 downto 0);
signal blue_wire_13: std_logic_vector (3 downto 0);
signal blue_wire_14: std_logic_vector (3 downto 0);
signal blue_wire_15: std_logic_vector (3 downto 0);
signal blue_wire_16: std_logic_vector (3 downto 0);
begin
--tb_clk: process is
--begin
-- incoming_clock

FPGA Receiver (structural) chip
-- CLKIN_IBUFG_OUT =>,
-- CLK0_OUT =>,
-- LOCKED_OUT =>
);
Inst_get_partial_spike_clock_from_incoming_clock:
get_partial_spike_clock_from_incoming_clock
Port map (received_clock => incoming_clock,
out_clock => partial_spike_clock
);
Inst_get_spike_on_from_partial_spike_clock:
get_spike_on_from_partial_spike_clock
Port map (partial_spike_clock => partial_spike_clock,
spike_clock => spike_clock
);
Inst_get_pixel_clock_from_partial_spike_clock:
get_pixel_clock_from_partial_spike_clock
Port map (partial_spike_clock => partial_spike_clock,
pixel_clock => pixel_clock
);
Inst_get_long_spike: get_long_spike
Port map (partial_spike_clock => partial_spike_clock,
long_spike_clock => outerpulse_clock
);
--Inst_mem_of_256_short_aer_stream: mem_of_256_short_aer_stream PORT MAP
(
-- clka => pixel_clock,
-- addra => address,
-- douta => short_AER_RGB_stream --pulse count format
-- );
Mem_read: process (pixel_clock, address) is
Begin
-- if (pixel_clock'event) AND (pixel_clock = '1') then
if rising_edge (pixel_clock) then
address short_AER_RGB_stream (21
downto 18),--four bits
R_pulse_count => short_AER_RGB_stream (17
downto 12),--six bits
G_pulse_count => short_AER_RGB_stream (11
downto 6),--six bits
B_pulse_count => short_AER_RGB_stream (5
downto 0),--six bits
AER_RGB_pulse_stream => AER_RGB_stream --RHS
feeds out of component
166 of 200

FPGA Receiver (structural) chip
);--Instance (Instantiation) of pulse_count_to_AER_stream
Inst_preform_for_electrodes: preform_for_electrodes
--Eight bits will represent 192(24*8) addresses OK
--The spike_clock MUST be used here.
--fed from "pulse_count_to_AER_stream"
port map (preform_clock => spike_clock,
incoming_stream => AER_RGB_stream,
red_with_address => red_1_4_addressed,
green_with_address => green_2_5_addressed,
blue_with_address => blue_3_6_addressed
--out_with_rgb_address =>
composite_addressed_stream -- {174_bits}
);--Goes to "produce_colour_data_streams"
Inst_produce_colour_data_streams: produce_colour_data_streams
port map (spike_data_clock => spike_clock,
red => red_1_4_addressed,--Red with address bits
green => green_2_5_addressed,--Green with address bits
blue => blue_3_6_addressed,--Blue with address bits
red_data_out => red_data, --49 downto 0 {out on RHS}
green_data_out => green_data, --49 downto 0 i.e. colour
information only
blue_data_out => blue_data --49 downto 0 i.e. colour
information only
);--Colour information feeds to "extend_data_streams" AND
"fully_extend_data_streams"
--From "extend_dat_streams" then to nowhere!
--From "fully_extend_data_streams" to
"change_data_streams_to_pulsed_format"
Inst_fully_extend_data_streams: fully_extend_data_streams
--feeds "change_data_streams_to_pulsed_format"
port map (this_spike_clock => spike_clock,--NOT pixel_clock
red_data_incoming => red_data,
green_data_incoming => green_data,
blue_data_incoming => blue_data,
extended_red => outer_red_pulse_stream, --1000
bits.
extended_green => outer_green_pulse_stream, --1000
bits.
extended_blue => outer_blue_pulse_stream --1000
bits.
);--sends to `fully_extended.dat' via
"file_support_fully_extended_data_streams"
--Inst_extend_data_streams: extend_data_streams
--port map (this_spike_clock => partial_spike_clock,--NOT pixel_clock
-- red_data_incoming => red_data,
-- green_data_incoming => green_data,
167 of 200

FPGA Receiver (structural) chip
-- blue_data_incoming => blue_data,
-- extended_red => red_pulse_stream, --200 bits.
-- extended_green => green_pulse_stream, --200 bits.
-- extended_blue => blue_pulse_stream --200 bits.
-- );--sends to `extended.dat' via
"file_support_extend_data_streams"
--Inst_produce_wire_outputs: produce_wire_outputs
-- Port map (wire_clock => pixel_clock,--pixel_clock
-- biphasic_clock => spike_clock,
-- red => outer_red_pulse_stream,--1000 bits Originally "red_squeeze"
-- green => outer_green_pulse_stream,--1000 bits
-- blue => outer_blue_pulse_stream,--1000 bits
--red_1 => red_wire_1,
--red_2 => red_wire_2,
--red_3 => red_wire_3,
--red_4 => red_wire_4,
--green_1 => green_wire_1,
--green_2 => green_wire_2,
--green_3 => green_wire_3,
--green_4 => green_wire_4,
--blue_1 => blue_wire_1,
--blue_2 => blue_wire_2,
--blue_3 => blue_wire_3,
--blue_4 => blue_wire_4
--);--Instance (Instantiation) of produce_wire_outputs
Inst_produce_wire_outputs: produce_wire_outputs
Port map (wire_clock => pixel_clock,--pixel_clock
biphasic_clock => spike_clock,
red => outer_red_pulse_stream,--1000 bits
green => outer_green_pulse_stream,--1000 bits
blue => outer_blue_pulse_stream,--1000 bits
red_1 => red_wire_1,
red_2 => red_wire_2,
red_3 => red_wire_3,
red_4 => red_wire_4,
red_5 => red_wire_5,
red_6 => red_wire_6,
red_7 => red_wire_7,
red_8 => red_wire_8,
red_9 => red_wire_9,
red_10 => red_wire_10,
red_11 => red_wire_11,
red_12 => red_wire_12,
red_13 => red_wire_13,
red_14 => red_wire_14,
red_15 => red_wire_15,
red_16 => red_wire_16,
green_1 => green_wire_1,
green_2 => green_wire_2,
green_3 => green_wire_3,
green_4 => green_wire_4,
168 of 200

FPGA Receiver (structural) chip
green_5 => green_wire_5,
green_6 => green_wire_6,
green_7 => green_wire_7,
green_8 => green_wire_8,
green_9 => green_wire_9,
green_10 => green_wire_10,
green_11 => green_wire_11,
green_12 => green_wire_12,
green_13 => green_wire_13,
green_14 => green_wire_14,
green_15 => green_wire_15,
green_16 => green_wire_16,
blue_1 => blue_wire_1,
blue_2 => blue_wire_2,
blue_3 => blue_wire_3,
blue_4 => blue_wire_4,
blue_5 => blue_wire_5,
blue_6 => blue_wire_6,
blue_7 => blue_wire_7,
blue_8 => blue_wire_8,
blue_9 => blue_wire_9,
blue_10 => blue_wire_10,
blue_11 => blue_wire_11,
blue_12 => blue_wire_12,
blue_13 => blue_wire_13,
blue_14 => blue_wire_14,
blue_15 => blue_wire_15,
blue_16 => blue_wire_16
);--Instance (Instantiation) of produce_wire_outputs
--Signals to electrodes:
IO_L13P_11

FPGA Receiver (structural) chip
IO_L16P_13

--Appendix E FPGA Outputs
`Sender’ design summary
“Design Summary” facility; a recent (December 2010) screenprint is shown here:
imp_send_16 Project Status (12/31/2010 - 14:29:37)
Project
File:
Module
Name:
imp_send_16.ise
Implementation State:
Programming
Generated
top_wrapper Errors: No Errors
File
Target
Device:
xc5vlx50t-3ff1136 Warnings: 27 Warnings
Product
Version:
ISE 11.4
Routing
Results:
All Signals
Completely Routed
Design
Goal:
Balanced
Timing
Constraints:
All Constraints Met
Design
Strategy:
Xilinx
(unlocked)
Default
Final Timing
Score:
0 (Setup: 0, Hold:
0) (Timing Report)
Device Utilization Summary [-]
Slice Logic Utilization
Used Available Utilization Note(s)
Number of Slice Registers 139 28,800 1%
Number used as Flip Flops 136
Number used as Latches 3
Number of Slice LUTs 201 28,800 1%
Number used as logic 198 28,800 1%
Number using O6 output only 105
Number using O5 output only 90
Number using O5 and O6 3
Number used as exclusive routethru
Number of route-thrus 93
Number using O6 output only 93
Number of occupied Slices 67 7,200 1%
Number of LUT Flip Flop pairs used 219
Number with an unused Flip Flop 80 219 36%
3
171 of 200

Number with an unused LUT 18 219 8%
Number of fully used LUT-FF
pairs
Number of unique control sets 6
Number of slice register sites lost
to control set restrictions
121 219 55%
9 28,800 1%
Number of bonded IOBs 23 480 4%
IOB Flip Flops 4
Number of BlockRAM/FIFO 1 60 1%
Number using BlockRAM only 1
Number of 18k BlockRAM used 1
Total Memory used (KB) 18 2,160 1%
Number of BUFG/BUFGCTRLs 4 32 12%
Number used as BUFGs 4
Number of DCM_ADVs 1 12 8%
Average Fanout of Non-Clock Nets 2.65
Performance Summary [-]
Final
Score:
Timing
Routing Results:
Timing
Constraints:
0 (Setup: 0, Hold: 0)
All Signals Completely
Routed
All Constraints Met
Pinout
Data:
Pinout Report
Clock Data: Clock Report
Detailed Reports [-]
Report Name Status Generated Errors Warnings Infos
Synthesis Report
Translation Report
Map Report
Place and Route
Report
Power Report
Current
Current
Current
Current
Out
Date
Post-PAR Static
Current
Timing Report
of
Fri 31. Dec
14:24:49 2010
Fri 31. Dec
14:25:04 2010
Fri 31. Dec
14:25:48 2010
Fri 31. Dec
14:27:00 2010
Fri 8. Oct
14:06:21 2010
Fri 31. Dec
14:27:20 2010
0
23
Warnings
0 0 0
0
4
Warnings
1 Info
6 Infos
0 0 4 Infos
0
5
Warnings
1 Info
0 0 3 Infos
172 of 200

Bitgen Report
Current
Fri 31. Dec
14:29:37 2010
0
3
Warnings
1 Info
Secondary Reports [-]
Report Name Status Generated
Date Generated: 12/31/2010 - 14:29:37
Figure 58 sender design summary
Synthesis report extracts
Device utilization summary: (sending 1024 pixels worth of information)
---------------------------
Selected Device: 5vlx50tff1136-3
Slice Logic Utilization:
Number of Slice Registers: 145 out of 28800 0%
Number of Slice LUTs: 207 out of 28800 0%
Number used as Logic: 207 out of 28800 0%
Slice Logic Distribution:
Number of LUT Flip Flop pairs used: 219
Number with an unused Flip Flop: 74 out of 219 33%
Number with an unused LUT: 12 out of 219 5%
Number of fully used LUT-FF pairs: 133 out of 219 60%
Number of unique control sets: 8
IO Utilization:
Number of IOs: 30
Number of bonded IOBs: 29 out of 480 6%
IOB Flip Flops/Latches: 10
Specific Feature Utilization:
Number of Block RAM/FIFO: 1 out of 60 1%
Number using Block RAM only: 1
Number of BUFG/BUFGCTRLs: 4 out of 32 12%
Number of DCM_ADVs: 1 out of 12 8%
Device utilization summary: (sending 256 pixels worth of information)
---------------------------
Selected Device: 5vlx50tff1136-3
IO Utilization:
Number of IOs: 28
Number of bonded IOBs: 27 out of 480 5%
Specific Feature Utilization:
Number of Block RAM/FIFO: 1 out of 60 1%
Number using Block RAM only: 1
Number of BUFG/BUFGCTRLs: 2 out of 32 6%
Number of DCM_ADVs: 1 out of 12 8%
Device utilization summary: (sending 64 pixels worth of information)
---------------------------
Selected Device: 5vlx50tff1136-3
173 of 200

IO Utilization:
Number of IOs: 26
Number of bonded IOBs: 25 out of 480 5%
Specific Feature Utilization:
Number of BUFG/BUFGCTRLs: 2 out of 32 6%
Number of DCM_ADVs: 1 out of 12 8%
Device utilization summary: (sending 16 pixels worth of information)
---------------------------
Selected Device: 5vlx50tff1136-3
Slice Logic Utilization:
Number of Slice Registers: 139 out of 28800 0%
Number of Slice LUTs: 200 out of 28800 0%
Number used as Logic: 200 out of 28800 0%
Slice Logic Distribution:
Number of LUT Flip Flop pairs used: 213
Number with an unused Flip Flop: 74 out of 213 34%
Number with an unused LUT: 13 out of 213 6%
Number of fully used LUT-FF pairs: 126 out of 213 59%
Number of unique control sets: 9
IO Utilization:
Number of IOs: 24
Number of bonded IOBs: 23 out of 480 4%
IOB Flip Flops/Latches: 4
Specific Feature Utilization:
Number of Block RAM/FIFO: 1 out of 60 1%
Number using Block RAM only: 1
Number of BUFG/BUFGCTRLs: 4 out of 32 12%
Number of DCM_ADVs: 1 out of 12 8%
Translation report
Release 11.4 ngdbuild L.68 (NT)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
Command Line: C:\Xilinx\11.1\ISE\bin\nt\unwrapped\ngdbuild.exe -ise
imp_send_1024.ise -intstyle ise -dd _ngo -NT timestamp -i -p xc5vlx50t-ff1136-3
top_wrapper.ngc top_wrapper.ngd
Reading NGO files
"C:/MATLAB/R2009a/fpga_ver_11/imp_send_1024/top_wrapper.ngc"
...
Loading design module
"C:\MATLAB\R2009a\fpga_ver_11\imp_send_1024/mem_of_32_stream.ngc"...
Gathering constraint information from source properties...
Done.
Resolving constraint associations...
Checking Constraint Associations...
Done...
Checking Partitions...
Checking expanded design...
Partition Implementation Status
-------------------------------
No Partitions were found in this design.
-------------------------------
174 of 200

NGDBUILD Design Results Summary:
Number of errors: 0
Number of warnings: 0
Total memory usage is 88900 kilobytes
Writing NGD file "top_wrapper.ngd”...
Total REAL time to NGDBUILD completion: 7 sec
Total CPU time to NGDBUILD completion: 7 sec
Writing NGDBUILD log file "top_wrapper.bld"...
Map report extracts
Release 11.4 Map L.68 (nt)
Xilinx Mapping Report File for Design 'top_wrapper'
Design Information
------------------
Command Line : map -ise imp_send_1024.ise -intstyle ise -p xc5vlx50t-ff1136-3
-w -logic_opt off -ol high -t 1 -register_duplication off -global_opt off -mt
off -cm area -ir off -pr off -lc off -power off -o top_wrapper_map.ncd
top_wrapper.ngd top_wrapper.pcf
Target Device: xc5vlx50t
Target Package: ff1136
Target Speed : -3
Mapper Version: virtex5 -- $Revision: 1.51.18.1 $
Mapped Date : Thu May 20 14:35:25 2010
Design Summary
--------------
Number of errors: 0
Number of warnings: 4
Slice Logic Utilization:
Number of Slice Registers: 145 out of 28,800 1%
Number used as Flip Flops: 142
Number used as Latches: 3
Number of Slice LUTs: 208 out of 28,800 1%
Number used as logic: 204 out of 28,800 1%
Number using O6 output only: 102
Number using O5 output only: 98
Number using O5 and O6: 4
Number used as exclusive route-thru: 4
Number of route-thrus: 102
Number using O6 output only: 102
Slice Logic Distribution:
Number of occupied Slices: 70 out of 7,200 1%
Number of LUT Flip Flop pairs used: 217
Number with an unused Flip Flop: 72 out of 217 33%
Number with an unused LUT: 9 out of 217 4%
Number of fully used LUT-FF pairs: 136 out of 217 62%
Number of unique control sets: 5
Number of slice register sites lost
to control set restrictions: 3 out of 28,800 1%
A LUT Flip Flop pair for this architecture represents one LUT paired with
one Flip Flop within a slice. A control set is a unique combination of
clock, reset, set, and enable signals for a registered element.
175 of 200

The Slice Logic Distribution report is not meaningful if the design is
over-mapped for a non-slice resource or if Placement fails.
OVERMAPPING of BRAM resources should be ignored if the design is
over-mapped for a non-BRAM resource or if placement fails.
IO Utilization:
Number of bonded IOBs: 29 out of 480 6%
IOB Flip Flops: 10
Release 11.4 Map L.68 (nt)
Xilinx Mapping Report File for Design 'top_wrapper'
Design Information
------------------
Command Line : map -ise imp_send_16.ise -intstyle ise -p xc5vlx50t-ff1136-3 -w
-logic_opt off -ol high -t 1 -register_duplication off -global_opt off -mt off
-cm area -ir off -pr off -lc off -power off -o top_wrapper_map.ncd
top_wrapper.ngd top_wrapper.pcf
Target Device: xc5vlx50t
Target Package: ff1136
Target Speed : -3
Mapper Version: virtex5 -- $Revision: 1.51.18.1 $
Mapped Date : Thu May 20 14:30:05 2010
Design Summary
--------------
Number of errors: 0
Number of warnings: 4
Slice Logic Utilization:
Number of Slice Registers: 139 out of 28,800 1%
Number used as Flip Flops: 136
Number used as Latches: 3
Number of Slice LUTs: 201 out of 28,800 1%
Number used as logic: 198 out of 28,800 1%
Number using O6 output only: 105
Number using O5 output only: 90
Number using O5 and O6: 3
Number used as exclusive route-thru: 3
Number of route-thrus: 93
Number using O6 output only: 93
Slice Logic Distribution:
Number of occupied Slices: 67 out of 7,200 1%
Number of LUT Flip Flop pairs used: 219
Number with an unused Flip Flop: 80 out of 219 36%
Number with an unused LUT: 18 out of 219 8%
Number of fully used LUT-FF pairs: 121 out of 219 55%
Number of unique control sets: 6
Number of slice register sites lost
to control set restrictions: 9 out of 28,800 1%
A LUT Flip Flop pair for this architecture represents one LUT paired with
one Flip Flop within a slice. A control set is a unique combination of
clock, reset, set, and enable signals for a registered element.
The Slice Logic Distribution report is not meaningful if the design is
over-mapped for a non-slice resource or if Placement fails.
176 of 200

OVERMAPPING of BRAM resources should be ignored if the design is
over-mapped for a non-BRAM resource or if placement fails.
IO Utilization:
Number of bonded IOBs: 23 out of 480 4%
IOB Flip Flops: 4
Place and route report example
Release 11.4 par L.68 (nt)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
EECEAYPG11: Thu May 20 14:36:09 2010
par -ise imp_send_1024.ise -w -intstyle ise -ol std -t 1 top_wrapper_map.ncd
top_wrapper.ncd top_wrapper.pcf
Constraints file: top_wrapper.pcf.
"top_wrapper" is an NCD, version 3.2, device xc5vlx50t, package ff1136, speed -3
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
vv
INFO: Security: 54 - 'xc5vlx50t' is a WebPack part.
----------------------------------------------------------------------
INFO: Par: 465 - The PAR option, "-t" (Starting Placer Cost Table), will be disabled
in the next software release when
used in combination with MAP -timing (Perform Timing-Driven Packing and
Placement) or when run with V5 or newer
architectures. To explore cost tables, please use the MAP option, "-t" (Starting
Placer Cost Table), instead.
Initializing temperature to 85.000 Celsius. (default - Range: 0.000 to 85.000 Celsius)
Initializing voltage to 0.950 Volts. (default - Range: 0.950 to 1.050 Volts)
INFO: Par: 282 - No user timing constraints were detected or you have set the option
to ignore timing constraints ("par
-x"). Place and Route will run in "Performance Evaluation Mode" to automatically
improve the performance of all
internal clocks in this design. Because there are not defined timing requirements, a
timing score will not be
reported in the PAR report in this mode. The PAR timing summary will list the
performance achieved for each clock.
Note: For the fastest runtime, set the effort level to "std". For best performance,
set the effort level to "high".
Device speed data version: "PRODUCTION 1.66 2009-11-16".
Device Utilization Summary:
Number of BUFGs 4 out of 32 12%
Number of DCM_ADVs 1 out of 12 8%
Number of External IOBs 29 out of 480 6%
Number of LOCed IOBs 0 out of 29 0%
Number of OLOGICs 10 out of 560 1%
Number of RAMB36_EXPs 1 out of 60 1%
Number of Slice Registers 145 out of 28800 1%
Number used as Flip Flops 142
Number used as Latches 3
Number used as LatchThrus 0
Number of Slice LUTS 208 out of 28800 1%
Number of Slice LUT-Flip Flop pairs 217 out of 28800 1%
Overall effort level (-ol): Standard
177 of 200

Router effort level (-rl): Standard
Starting initial Timing Analysis. REAL time: 17 secs
Finished initial Timing Analysis. REAL time: 17 secs
Starting Router
Wirelength Stats for nets on all pins. NumPins: 812
Phase 1: 1080 unrouted; REAL time: 18 secs
Phase 2: 763 unrouted; REAL time: 19 secs
Phase 3: 172 unrouted; REAL time: 19 secs
Phase 4: 172 unrouted; (Par is working to improve performance) REAL time: 25
secs
Phase 5: 0 unrouted; (Par is working to improve performance) REAL time: 26
secs
Phase 6: 0 unrouted; (Par is working to improve performance) REAL time: 26
secs
Phase 7: 0 unrouted; (Par is working to improve performance) REAL time: 26
secs
Phase 8: 0 unrouted; (Par is working to improve performance) REAL time: 26
secs
Phase 9: 0 unrouted; (Par is working to improve performance) REAL time: 27
secs
Phase 10: 0 unrouted; (Par is working to improve performance) REAL time: 27
secs
Total REAL time to Router completion: 27 secs
Total CPU time to Router completion: 26 secs
Partition Implementation Status
-------------------------------
No Partitions were found in this design.
-------------------------------
Generating "PAR" statistics.
**************************
Generating Clock Report
**************************
+---------------------+--------------+------+------+------------+-------------+
| Clock Net | Resource |Locked|Fanout|Net Skew (ns) |Max Delay (ns)|
+---------------------+--------------+------+------+------------+-------------+
|Inst_get_spike_on_fr | | | | | |
|om_partial_spike_clo | | | | | |
| ck/clk_sig |BUFGCTRL_X0Y30| No | 24 | 0.077 | 1.479 |
+---------------------+--------------+------+------+------------+-------------+
|Inst_get_partial_spi | | | | | |
|ke_clock_from_incomi | | | | | |
|ng_clock/slow_clk_si | | | | | |
| g |BUFGCTRL_X0Y31| No | 16 | 0.034 | 1.367 |
+---------------------+--------------+------+------+------------+-------------+
| incoming_clock | BUFGCTRL_X0Y0| No | 8 | 0.028 | 1.355 |
+---------------------+--------------+------+------+------------+-------------+
|Inst_get_pixel_clock | | | | | |
|_from_partial_spike_ | | | | | |
| clock/slow_clk_sig | Local| | 6 | 0.411 | 0.789 |
+---------------------+--------------+------+------+------------+-------------+
|Inst_get_pixel_clock | | | | | |
178 of 200

|_from_partial_spike_ | | | | | |
|clock/cnt_cmp_eq0000 | | | | | |
| | Local| | 9 | 0.000 | 1.044 |
+---------------------+--------------+------+------+------------+-------------+
|Inst_get_spike_on_fr | | | | | |
|om_partial_spike_clo | | | | | |
| ck/cnt_cmp_eq0000 | Local| | 9 | 0.000 | 0.726 |
+---------------------+--------------+------+------+------------+-------------+
|Inst_get_partial_spi | | | | | |
|ke_clock_from_incomi | | | | | |
|ng_clock/cnt_cmp_eq0 | | | | | |
| 000 | Local| | 9 | 0.000 | 1.228 |
+---------------------+--------------+------+------+------------+-------------+
* Net Skew is the difference between the minimum and maximum routing
only delays for the net. Note this is different from Clock Skew which
is reported in TRCE timing report. Clock Skew is the difference between
the minimum and maximum path delays which includes logic delays.
Timing Score: 0 (Setup: 0 Hold: 0)
Asterisk (*) preceding a constraint indicates it was not met.
This may be due to a setup or hold violation.
------------------------------------------------------------------------------------------------------
----
Constraint | Check | Worst Case | Best Case | Timing |
Timing
| | Slack | Achievable | Errors | Score
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net Ins | SETUP | N/A| 1.958ns|
N/A| 0
t_get_spike_on_from_partial_spike_clock/c | HOLD | 0.416ns| | 0|
0
lk_sig | | | | |
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net Ins | SETUP | N/A| 1.242ns|
N/A| 0
t_get_pixel_clock_from_partial_spike_cloc | HOLD | 0.017ns| | 0|
0
k/slow_clk_sig | MINPERIOD | N/A| 1.818ns| N/A|
0
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net Ins | SETUP | N/A| 1.692ns|
N/A| 0
t_get_partial_spike_clock_from_incoming_c | HOLD | 0.504ns| |
0| 0
lock/slow_clk_sig | | | | |
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net inc | SETUP | N/A| 1.725ns|
N/A| 0
179 of 200

oming_clock | HOLD | 0.519ns| | 0| 0
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net Ins | SETUP | N/A| 0.832ns|
N/A| 0
t_get_pixel_clock_from_partial_spike_cloc | HOLD | 0.604ns| | 0|
0
k/cnt_cmp_eq0000 | MINLOWPULSE | N/A| 1.047ns|
N/A| 0
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net Ins | SETUP | N/A| 0.814ns|
N/A| 0
t_get_spike_on_from_partial_spike_clock/c | HOLD | 0.588ns| | 0|
0
nt_cmp_eq0000 | MINLOWPULSE | N/A| 1.055ns|
N/A| 0
------------------------------------------------------------------------------------------------------
----
Autotimespec constraint for clock net Ins | SETUP | N/A| 0.802ns|
N/A| 0
t_get_partial_spike_clock_from_incoming_c | HOLD | 0.577ns| |
0| 0
lock/cnt_cmp_eq0000 | MINLOWPULSE | N/A| 1.046ns|
N/A| 0
------------------------------------------------------------------------------------------------------
----
All constraints were met.
INFO: Timing: 2761 - N/A entries in the Constraints list may indicate that the
constraint does not cover any paths or that it has no requested value.
Generating Pad Report.
All signals are completely routed.
Total REAL time to PAR completion: 35 secs
Total CPU time to PAR completion: 35 secs
Peak Memory Usage: 269 MB
Placer: Placement generated during map.
Routing: Completed - No errors found.
Number of error messages: 0
Number of warning messages: 0
Number of info messages: 2
Writing design to file top_wrapper.ncd
PAR done!
Post-PAR Static Timing Report example
Release 11.4 Trace (nt)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
C:\Xilinx\11.1\ISE\bin\nt\unwrapped\trce.exe -ise
C:/MATLAB/R2009a/fpga_ver_11/imp_send_1024/imp_send_1024.ise -intstyle ise
-e 3
-s 3 -xml top_wrapper.twx top_wrapper.ncd -o top_wrapper.twr top_wrapper.pcf
180 of 200

Design file: top_wrapper.ncd
Physical constraint file: top_wrapper.pcf
Device, package, speed: xc5vlx50t, ff1136,-3 (PRODUCTION 1.66 2009-11-16,
STEPPING level 0)
Report level: error report
Environment Variable Effect
-------------------- ------
NONE
No environment variables were set
--------------------------------------------------------------------------------
INFO: Timing: 2698 - No timing constraints found, doing default enumeration.
INFO: Timing: 2752 - To get complete path coverage, use the unconstrained paths
option. All paths that are not constrained will be reported in the
unconstrained paths section(s) of the report.
INFO: Timing: 3339 - The clock-to-out numbers in this timing report are based on
a 50 Ohm transmission line loading model. For the details of this model,
and for more information on accounting for different loading conditions,
please see the device datasheet.
Data Sheet report:
-----------------
All values displayed in nanoseconds (ns)
Clock to Setup on destination clock USER_CLK
---------------+---------+---------+---------+---------+
| Src: Rise| Src: Fall| Src: Rise| Src: Fall|
Source Clock |Dest:Rise|Dest:Rise|Dest:Fall|Dest:Fall|
---------------+---------+---------+---------+---------+
USER_CLK | | | | 1.725|
---------------+---------+---------+---------+---------+
Analysis completed Thu May 20 14:37:07 2010
--------------------------------------------------------------------------------
Trace Settings:
-------------------------
Trace Settings
Peak Memory Usage: 220 MB
Total REAL time to Trace completion: 18 secs
Total CPU time to Trace completion: 18 secs
Power Report example
Release 11.4 - XPower Analyzer L.68 (nt)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
# NOTE: This file is designed to facilitate import into a
spreadsheet program
# such as Microsoft Excel for viewing, printing and sorting. The
"|" character
# should be selected as the data field separator.
C:\Xilinx\11.1\ISE\bin\nt\unwrapped\xpwr.exe
-ise
C:/MATLAB/R2009a/fpga_ver_11/imp_send_1024/imp_send_1024.ise -
intstyle ise
top_wrapper.ncd top_wrapper.pcf -o top_wrapper.pwr
Design | top_wrapper.ncd
|
Preferences | top_wrapper.pcf
|
181 of 200

Part | xc5vlx50tff1136-3
|
Process | Typical
|
Data version | PRODUCTION, v1.63, 12-10-08
|
Default settings | Value |
---------------------------------------------------
FF Toggle Rate (%) | 12.5 |
I/O Toggle Rate (%) | 12.5 |
Output Load (pF) | 5.0 |
I/O Enable Rate (%) | 100.0 |
BRAM Write Rate (%) | 50.0 |
BRAM Enable Rate (%) | 25.0 |
DSP Toggle Rate (%) | 12.5 |
Power summary | I (mA) | P (mW) |
----------------------------------------------------------------
Total estimated power consumption | | 638.51 |
---
Total Vccint 1.00V | 425.02 | 425.02 |
Total Vccaux 2.50V | 83.40 | 208.50 |
Total Vcco25 2.50V | 2.00 | 5.00 |
---
BRAM | | 0.00 |
Clocks | | 3.78 |
DCM | | 68.00 |
IO | | 0.15 |
Logic | | 0.01 |
Signals | | 0.00 |
---
Quiescent Vccint 1.00V | 406.58 | 406.58 |
Quiescent Vccaux 2.50V | 75.40 | 188.50 |
Quiescent Vcco25 2.50V | 2.00 | 5.00 |
Thermal summary
----------------------------------------------------------------
Estimated junction temperature | 52C |
Ambient temp | 50C |
Case temp | 52C |
Theta J-A | 3C/W |
Analysis completed: Fri May 21 11:55:48 2010
----------------------------------------------------------------
Receiver chip design summary
hyper_receive_16_pix Project Status (05/25/2010 - 13:41:40)
Project
File:
Module
Name:
hyper_receive_16_pix.ise Implementation State:
top_wrapper Errors:
Placed
Routed
and
Target
Device:
xc5vlx50t-3ff1136 Warnings:
Product
Version:
ISE 11.4
Routing
Results:
All Signals
Completely
Routed
182 of 200

Design
Goal:
Balanced
Timing
Constraints:
All Constraints
Met
Design
Strategy:
Xilinx
(unlocked)
Default
Final
Timing Score:
0 (Setup: 0,
Hold:
0) (Timing
Report)
hyper_receive_16_pix Partition Summary [-]
No partition information was found.
Device Utilization Summary [+]
Performance Summary [-]
Final
Score:
Timing
Routing Results:
Timing
Constraints:
0 (Setup: 0, Hold: 0)
All Signals Completely
Routed
All Constraints Met
Pinout
Data:
Pinout Report
Clock Data: Clock Report
Detailed Reports [-]
Report Name Status Generated Errors Warnings Infos
Synthesis Report
Translation Report
Map Report
Place and Route
Report
Power Report
Current
Current
Current
Current
Current
Post-PAR Static Out
Timing Report Date
Bitgen Report
Figure 59 design summary
of
Mon 24. May
14:23:37 2010
Tue 25. May
13:35:05 2010
Tue 25. May
13:35:57 2010
Tue 25. May
13:36:55 2010
Tue 25. May
13:41:39 2010
Tue 25. May
13:37:16 2010
0
2347
Warnings
0 0 0
2880 Infos
0 3 Warnings 6 Infos
0 0 3 Infos
0 0 3 Infos
Synthesis report extract
183 of 200

Selected Device: 5vlx50tff1136-3
Slice Logic Utilization:
Number of Slice Registers: 102 out of 28800 0%
Number of Slice LUTs: 119 out of 28800 0%
Number used as Logic: 119 out of 28800 0%
Slice Logic Distribution:
Number of LUT Flip Flop pairs used: 121
Number with an unused Flip Flop: 19 out of 121 15%
Number with an unused LUT: 2 out of 121 1%
Number of fully used LUT-FF pairs: 100 out of 121 82%
Number of unique control sets: 9
IO Utilization:
Number of IOs: 193
Number of bonded IOBs: 193 out of 480 40%
IOB Flip Flops/Latches: 90
Specific Feature Utilization:
Number of BUFG/BUFGCTRLs: 3 out of 32 9%
Number of DCM_ADVs: 1 out of 12 8%
Translation report
Release 11.4 ngdbuild L.68 (NT)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
Command Line: C:\Xilinx\11.1\ISE\bin\nt\unwrapped\ngdbuild.exe -ise
hyper_receive_16_pix.ise -intstyle ise -dd _ngo -NT timestamp -i -p
Xc5vlx50t-ff1136-3 top_wrapper.ngc top_wrapper.ngd
Reading NGO file
"C:/MATLAB/R2009a/fpga_ver_11/hyper_receive_16_pix/top_wrapper.ngc”...
Gathering constraint information from source properties...
Done.
Resolving constraint associations...
Checking Constraint Associations...
Done...
Checking Partitions...
Checking expanded design...
Partition Implementation Status
-------------------------------
No Partitions were found in this design.
-------------------------------
NGDBUILD Design Results Summary:
Number of errors: 0
Number of warnings: 0
Total memory usage is 88900 kilobytes
Writing NGD file "top_wrapper.ngd”...
Total REAL time to NGDBUILD completion: 8 sec
Total CPU time to NGDBUILD completion: 5 sec
Writing NGDBUILD log file "top_wrapper.bld"...
Map report extract
184 of 200

Release 11.4 Map L.68 (nt)
Xilinx Mapping Report File for Design 'top_wrapper'
Design Information
------------------
Command Line : map -ise hyper_receive_16_pix.ise -intstyle ise -p
xc5vlx50t-ff1136-3 -w -logic_opt off -ol high -t 1 -register_duplication off
-global_opt off -mt off -cm area -ir off -pr off -lc off -power off -o
top_wrapper_map.ncd top_wrapper.ngd top_wrapper.pcf
Target Device: xc5vlx50t
Target Package: ff1136
Target Speed : -3
Mapper Version: virtex5 -- $Revision: 1.51.18.1 $
Mapped Date : Tue May 25 13:35:12 2010
Design Summary
--------------
Number of errors: 0
Number of warnings: 3
Slice Logic Utilization:
Number of Slice Registers: 102 out of 28,800 1%
Number used as Flip Flops: 99
Number used as Latches: 3
Number of Slice LUTs: 120 out of 28,800 1%
Number used as logic: 117 out of 28,800 1%
Number using O6 output only: 24
Number using O5 output only: 90
Number using O5 and O6: 3
Number used as exclusive route-thru: 3
Number of route-thrus: 93
Number using O6 output only: 93
Slice Logic Distribution:
Number of occupied Slices: 42 out of 7,200 1%
Number of LUT Flip Flop pairs used: 123
Number with an unused Flip Flop: 21 out of 123 17%
Number with an unused LUT: 3 out of 123 2%
Number of fully used LUT-FF pairs: 99 out of 123 80%
Number of unique control sets: 6
Number of slice register sites lost
to control set restrictions: 6 out of 28,800 1%
Place and route report
Release 11.4 par L.68 (nt)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
EECEAYPG11:: Tue May 25 13:36:10 2010
par -ise hyper_receive_16_pix.ise -w -intstyle ise -ol std -t 1
top_wrapper_map.ncd top_wrapper.ncd top_wrapper.pcf
Constraints file: top_wrapper.pcf.
"top_wrapper" is an NCD, version 3.2, device xc5vlx50t, package ff1136, speed -3
INFO:Par:465 - The PAR option, "-t" (Starting Placer Cost Table), will be disabled
in the next software release when
185 of 200

used in combination with MAP -timing(Perform Timing-Driven Packing and
Placement) or when run with V5 or newer
architectures. To explore cost tables, please use the MAP option, "-t" (Starting
Placer Cost Table), instead.
Initializing temperature to 85.000 Celsius. (default - Range: 0.000 to 85.000 Celsius)
Initializing voltage to 0.950 Volts. (default - Range: 0.950 to 1.050 Volts)
INFO:Par:282 - No user timing constraints were detected or you have set the option
to ignore timing constraints ("par
-x"). Place and Route will run in "Performance Evaluation Mode" to automatically
improve the performance of all
internal clocks in this design. Because there are not defined timing requirements, a
timing score will not be
reported in the PAR report in this mode. The PAR timing summary will list the
performance achieved for each clock.
Note: For the fastest runtime, set the effort level to "std". For best performance,
set the effort level to "high".
Device speed data version: "PRODUCTION 1.66 2009-11-16".
Device Utilization Summary:
Number of BUFGs 3 out of 32 9%
Number of DCM_ADVs 1 out of 12 8%
Number of External IOBs 193 out of 480 40%
Number of LOCed IOBs 0 out of 193 0%
Number of OLOGICs 90 out of 560 16%
Number of Slice Registers 102 out of 28800 1%
Number used as Flip Flops 99
Number used as Latches 3
Number used as LatchThrus 0
Number of Slice LUTS 120 out of 28800 1%
Number of Slice LUT-Flip Flop pairs 123 out of 28800 1%
Overall effort level (-ol): Standard
Router effort level (-rl): Standard
Starting initial Timing Analysis. REAL time: 17 secs
Finished initial Timing Analysis. REAL time: 17 secs
Starting Router
Power report extract
Release 11.4 - XPower Analyzer L.68 (nt)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
# NOTE: This file is designed to facilitate import into a spreadsheet program
# such as Microsoft Excel for viewing, printing and sorting. The "|" character
# should be selected as the data field separator.
C:\Xilinx\11.1\ISE\bin\nt\unwrapped\xpwr.exe
-ise
C:/MATLAB/R2009a/fpga_ver_11/hyper_receive_16_pix/hyper_receive_16_pix.ise
-intstyle ise top_wrapper.ncd top_wrapper.pcf -v -l 1000 -o top_wrapper.pwr
Design | top_wrapper.ncd |
Preferences | top_wrapper.pcf |
Part | xc5vlx50tff1136-3 |
Process | Typical |
Data version | PRODUCTION,v1.63,12-10-08 |
Default settings | Value |
186 of 200

---------------------------------------------------
FF Toggle Rate (%) | 12.5 |
I/O Toggle Rate (%) | 12.5 |
Output Load (pF) | 5.0 |
I/O Enable Rate (%) | 100.0 |
BRAM Write Rate (%) | 50.0 |
BRAM Enable Rate (%) | 25.0 |
DSP Toggle Rate (%) | 12.5 |
Power summary | I(mA) | P(mW) |
----------------------------------------------------------------
Total estimated power consumption | | 637.79 |
---
Total Vccint 1.00V | 424.29 | 424.29 |
Total Vccaux 2.50V | 83.40 | 208.50 |
Total Vcco25 2.50V | 2.00 | 5.00 |
---
Clocks | | 3.06 |
DCM | | 68.00 |
IO | | 0.15 |
Logic | | 0.01 |
Signals | | 0.00 |
---
Quiescent Vccint 1.00V | 406.57 | 406.57 |
Quiescent Vccaux 2.50V | 75.40 | 188.50 |
Quiescent Vcco25 2.50V | 2.00 | 5.00 |
Post-PAR Static Timing Report example
Release 11.4 Trace (nt)
Copyright (c) 1995-2009 Xilinx, Inc. All rights reserved.
C:\Xilinx\11.1\ISE\bin\nt\unwrapped\trce.exe -ise
C:/MATLAB/R2009a/fpga_ver_11/hyper_receive_16_pix/hyper_receive_16_pix.ise
-intstyle ise -e 3 -s 3 -xml top_wrapper.twx top_wrapper.ncd -o top_wrapper.twr
top_wrapper.pcf
Design file: top_wrapper.ncd
Physical constraint file: top_wrapper.pcf
Device,package,speed: xc5vlx50t,ff1136,-3 (PRODUCTION 1.66 2009-11-16,
STEPPING level 0)
Report level: error report
Environment Variable Effect
-------------------- ------
NONE
No environment variables were set
--------------------------------------------------------------------------------
INFO:Timing:2698 - No timing constraints found, doing default enumeration.
INFO:Timing:2752 - To get complete path coverage, use the unconstrained paths
option. All paths that are not constrained will be reported in the
unconstrained paths section(s) of the report.
INFO:Timing:3339 - The clock-to-out numbers in this timing report are based on
a 50 Ohm transmission line loading model. For the details of this model,
and for more information on accounting for different loading conditions,
please see the device datasheet.
187 of 200

Data Sheet report:
-----------------
All values displayed in nanoseconds (ns)
Clock to Setup on destination clock USER_CLK
---------------+---------+---------+---------+---------+
| Src:Rise| Src:Fall| Src:Rise| Src:Fall|
Source Clock |Dest:Rise|Dest:Rise|Dest:Fall|Dest:Fall|
---------------+---------+---------+---------+---------+
USER_CLK | | | | 1.715|
---------------+---------+---------+---------+---------+
Analysis completed Tue May 25 13:37:16 2010
--------------------------------------------------------------------------------
Trace Settings:
-------------------------
Trace Settings
Peak Memory Usage: 220 MB
Total REAL time to Trace completion: 18 secs
Total CPU time to Trace completion: 18 secs
188 of 200

--Appendix F Alternative literature review
The approach to be taken for this work is to determine and then decide how to
implement a retinal prosthesis capable of restoring sufficient vision to a person
suffering visual impairment to improve their quality of life. Although ten percent of the
population suffer from visual impairment; so in the UK six million people, only ten
percent of those suffer from AMD (age related macular degeneration) [Aside young
people can also suffer from this condition] and RP (retinitis pigmentosa) a family of
related diseases to which this study applies. Although this represents less than one
million people in the UK clearly the European/worldwide market is much bigger. It also
bears saying that the gift of sight is priceless for each person affected to a greater or
lesser degree.
Literature Review
The following subsections pick out key points from relevant papers within the literature.
These papers adopt varying approaches to the idea of retina replacement and how this
may best be accomplished.
Visual Prostheses: Current Progress and
Challenges
“The epi-retinal referring to the side that faces the vitreous and the sub-retinal to the
side that is adjacent to the choroid.”…” This in turn implies that we need 500mW for a
1000 electrode array,”
[51]
Old Idea, New Technology

“For these implantable devices to be most effective, they must be biomimetic; that is,
they must restore a lost function that mimics the biology”, “When treating a medical
problem, focus on the needs of the patient.”
[126]
Intraocular Retinal Prosthesis
“Recent clinical trials have shown that a prototype epiretinal implant, despite having
few electrodes contacting the retina, still allows test subjects to perform simple visual
tasks. Ongoing engineering research is focusing on the fabrication of a high-resolution
implant.”, “The retina lines the back of the vitreous cavity, a 6-cm 3 space in the back of
the eye that is normally filled with vitreous gel but is routinely replaced with saline
(after removing the vitreous during retinal surgery)(Figure 1). This fluid-filled cavity
will permit a device of significant size to be placed near the retina without disrupting
other tissue.”; “An epiretinal implant will rest on the inner limiting membrane of the
retina, while a subretinal implant would be inserted in the space occupied by
photoreceptors in a healthy retina.”
[52]
A Neuro-Stimulus Chip with Telemetry Unit
for Retinal Prosthetic Device
“It is fabricated by MOSIS with 1.2-mm CMOS technology and was demonstrated to
provide the desired biphasic current stimulus for an array of 100 retinal electrodes at
video frame rates.”, “Over 10,000,000 people worldwide are blind because of
photoreceptor loss due to degenerative retinal diseases such as age-related macular
degeneration (AMD) and retinitis pigmentosa (RP).”, “Medical experiments have
estimated that the equivalent impedance for the retina tissue of RP and AMD patients is
about 10 k and, in the worst case, the current threshold value is about 600 A . Thus,

a voltage drop up to 6 V is expected across the retinal tissue.”. “The total power
consumption of the device is contributed from two sources, namely the power
consumption of the stimulus chip itself and the power dissipated into the load (retinal
tissue).”, “When added to the previous 3 mW associated with the stimulator circuits,
this accounts for intraocular power dissipation on the order of 5 mW.”, “This chip uses
a simple synchronous protocol and timing control.”
[53]
A Computational Model of Electrical
Stimulation of the Retinal Ganglion Cell
“Since the dendritic arbor of a single ganglion cell may spread up to 500 µm in diameter
and overlap the dendritic field of other ganglion cells, stimulation of dendrites might
lead to larger perceived spots than if the soma was preferentially stimulated.”, “Still
another possibility is that the soma is excited directly, which would yield a potential
resolution of around 10 m in humans.” [49]
Architecture Tradeoffs in High-Density
Microstimulators for Retinal Prosthesis
” In both methods, retinal neurons are electrically stimulated which bypasses the
damaged photoreceptors thereby creating visual excitation. The discussions in this paper
will refer to the epi-retinal approach.” … “Fig. 1 shows the block diagram of the epiretinal
prosthesis system. It comprises of two units – implant and external. While the
external unit is powered by battery, power for the implant unit is transmitted wirelessly
through a high efficiency power amplifier , rectified and regulated into DC voltages
required by the electronics .”, “ For 1024 stimulation sites (32 x 32 array), the number
of interconnect leads between the microstimulator and tissue are 1025 and 2048 for
configurations in Fig. 8(a) and (b) respectively.”, “For example, a 256-output

microstimulator with a 1:4 demultiplexer built on the electrode array results in 1024-
pixel prosthesis.”, “For 1024 sites, 10 address bits are required. If the stimulus data for
each driver is around 20 bits, this results in 50% increase in the data rate.”, “The chip
consumes around 20 mW when delivering 600 A bi-phasic current pulses of 1 ms
phase widths (anodic, cathodic and interphase delay) at a stimulation rate of 60 Hz.”
[49]
Electrical Stimulation in Isolated Rabbit Retina
“Analysis of the data from this study predicts that a 50- m electrode may have enough
safe charge capacity to evoke a retinal response. “, “With current electrode technology,
more than one thousand 50- m-diameter electrodes could be placed in the macula (5-
mm-diameter area around fovea) . Simulations of prosthetic vision, suggest that face
recognition and reading will be possible with 1000 macular electrodes.”, “Overall,
epiretinal electrodes and subretinal electrodes appear to have similar stimulus pulse
requirements.” [229]
A Biomimetic Retinal Stimulating Array
“.. and in severe cases will lose most of their vision in the central 20º of the visual field.
The macula is the part of the retina that anatomically corresponds to the central 20º of
the visual field”… “When 32 × 32 electrodes were used, the recognition scores
improved to over 80%.” “The electrodes should support 100 nC (100 μA for 1 ms)
pulses without damaging either the electrodes or tissue.”…”Standard visual acuity
testing demonstrated that 20/30 vision could be achieved using 625 pixels in the central
1.7º of the visual field.”… “To accommodate a square arrangement, the electrode must
be reduced to 93µm in order to achieve 1,000 macular electrodes.”
[103]

An Optimal Design Methodology for Inductive
Power Link
“We have used the design methodology to implement a power link for a retinal
prosthesis using the proposed closed-loop control technique. The entire power link has
been experimentally demonstrated through a hybrid system that is capable of delivering
250 mW at 16 V using a supply voltage of less than 5 V from a distance of 7 mm with
an overall efficiency of 67%, which includes the power dissipated in the control
circuitry.” [230]
Biological–Machine Systems Integration:
Engineering the Neural Interface
“The World Health Organization (2002 figures) estimates that globally there are more
than 160 million visually impaired, of which some 40 million are blind (loss of
walkabout vision). While many causes of blindness are either preventable or treatable,
there are limited treatment options available for conditions such as glaucoma, agerelated
macular degeneration (AMD) or retinitis pigmentosa (RP). These three diseases
provide the bulk of the motivation to develop a visual prosthesis.” [24]
Specificity of Cone Connections in the Retina
and Colour Vision. Focus on "Specificity of
Cone Inputs to Macaque Retinal Ganglion
Cells"
“L- (long wavelength; peak absorption, 565nm), M- (medium wavelength; peak
absorption, 535nm), and S- (short wavelength; peak absorption, 440nm) cones onto a
retinal ganglion cell?” [50]
A neuro-stimulus chip with telemetry unit for
retinal prosthetic device

“Medical experiments have estimated that the equivalent impedance for the retina tissue
of RP and AMD patients is about 10kΩ and, in the worst case, the current threshold
value is about 600µA. Thus a voltage drop up to 6v is expected across the retinal
tissue”… “The power consumption in the stimulus chip depends on the image frame
rate. In our application the frame rate is greater than 60 frames/s. The power dissipated
at 100 frames/s, corresponding to a data rate of 40kb/s would be approximately 3mw.”
[53]
On the Thermal Elevation of a 60-Electrode
Epiretinal Prosthesis for the Blind
“In this paper, the thermal elevation in the human body due to the operation of dual-unit
epiretinal prosthesis to restore partial vision to the blind affected by irreversible retinal
degeneration is presented. An accurate computational model of a 60-electrode device
dissipating 97 mW power, currently under clinical trials is developed and positioned in
a 0.25 mm resolution, heterogeneous model of the human head to resemble actual
conditions of operation of the prosthesis.”… “This indicates that it is possible to
position and operate the implant so that the induced temperature increase in the eye over
the body core temperature is less than 2 degree Centigrade.”
[227]
A variable range bi-phasic current stimulus
driver circuitry for an implantable retinal
prosthetic device
“The duration of anodic and cathodic pulses is required to be a maximum of 1ms each.
Effectively any output is on for only 2ms in one stimulation cycle.”
[56]

A Programmable Discharge Circuitry With
Current Limiting Capability for a Retinal
Prosthesis
“Biphasic stimulation is the most commonly used electrical pattern in FES (Functional
Electrical Stimulation). In such cases, a charge balanced waveform is essential to
prevent any charge accumulation in the biological tissue.”
[57]
Vision on a chip
“A “subretinal” implant uses photodiodes to replace the function of damaged
photoreceptors. The photodiodes are connected to electrodes that stimulate the
remaining neural cells in contrast; an “epiretinal” implant does not detect light but
instead uses the electrical signal from an external camera to directly stimulate the
ganglion cells’ axons, which form the optic nerve.” [32]
Specificity of Cone Connections in the Retina
and Color Vision
“… parvocellular neurons that mainly receive L-M, or M-L inputs, and in the
magnocellular neurons that receive L+M inputs? … each cone only connecting with a
one sign of input either excitatory or inhibitory” [50]
Perceptual Thresholds and Electrode Impedance
in Three Retina Prosthesis Subjects
“suggesting a complex interaction that is not completely predictable by electric field
theory…Proximity to the retina plays a role in determining the threshold and impedance
but only for electrodes that are greater than 0.5mm from the retina. Within this distance,
perception thresholds and impedances do not seem to depend on the diameter of the
electrode.” [58]

A Retinomorphic Vision System
“Instead, the communications channel includes an arbiter to deal with contention and a
queue where unsuccessful contenders wait. This architecture was proposed by Sivilotti
and Mahowald.” [162]
On Algorithmic Rate-Coded AER Generation
“This paper addresses the problem of converting a sequence of frames into the spike
event-based representation known as the address-event-representation (AER)...AERbased
interchip communication was originally proposed by Mahowald and Sivilotti…A
growing community of researchers is using this scheme for bio inspired vision...AER
has been an important mainstream line at the annual National Science Foundation
(NSF) funded Telluride Neuromorphic Workshop series. Transforming the video stream
of a conventional frame-based representation (FBR) sequence...Such a transformation is
the purpose of the present paper….Events can be time shifted to a certain degree
because AER receiver reconstruct signals by integrating (averaging) over time…”
[59]
Optic Nerve Signals in a Neuromorphic Chip II:
Testing and Results
“Our retinomorphic chip produces spike trains for 3600 ganglion cells (GCs) and
consumes 62.7 Mw at 45 spikes/s/GC… by capturing the neural code of the mammalian
retina our chip can provide researchers with realistic retinal input with which they can
design and test subsequent neuromorphic circuits.” [231]
A Prototype Retinal Prosthesis for Visual
Stimulation
“Live webcam images are converted to an 8 x 8 mosaic of 256 greyscale shades.
Subsequently, electrical impulses are generated by the excitatory circuit in real-time to

topographically stimulate the corresponding epiretinal cells. Following their conversion
to greyscale, recorded data from the central pixel of the mosaic yielded 36.24nc for
black, 48.84nc for red, 55.68 for green, 67.68nc for blue and 91.92 for white. These
results correlate well with data reported in the literature.”
[41]
A Foveated AER Imager Chip
“Sensory cells in the retina at the back of the eyeball do not merely record an image but
they already process it before it is conveyed to the brain. The photoreceptors perform
spatial and temporal filtering. Since the nineties this has also been introduced into a
number of `intelligent’ silicon imagers such as the silicon retina…. The form of pulse
communication by dedicated point to point connections is the general method of
communication between nerve cells. A method to emulate this …is address event
representation (AER): In short the superior speed of electronic systems is traded in for
the inferior cable density [61]
An Address-Event Image Sensor Network
“CMOS imager (the ALOHA imager)… The system uses commercial, off-the-shelf
(COTS)… At this point it should be stated that although AER is just a data
representation format, it imposes a degree of imager-level information filtering as well.
This is because an “event” is, in itself, the presence of a relevant feature”
[157]
Dynamically Reconfigurable Silicon Array of
Spiking Neurons With Conductance-Based
Synapses

“The common language of neuromorphic chips is the address-event representation
(AER) communication protocol, which uses time-multiplexing to emulate extensive
connectivity between neurons.
[64]
On Synthetic AER Generation
“The number of time slots depends on the time assigned to a frame (for example T frame =
40ms) and the time required to transmit a single event (for example T pulse = 10ns)….The
system has been implemented using VHDL …It can read or write an AER event every
T pulse = 40ns. If T frame = 40ms, then this implies N x M x K ≤ T frame /T pulse = 10 6 .”
[62]
The retina as a two-dimensional detector array
in the context of color vision theories and signal
detection theory
“The Young-Helmholtz hypothesis of three color mechanisms has served as the basis
for literally all color theories and is essentially the foundation for all geometrical color
theories. Another important contribution that enhanced the development of color
research were the Grassman Laws, which stated the commutative and associative
properties of color mixing in simple mathematical terms. The geometrical color theories
developed around the idea that if there exist three different color mechanisms in the eye,
and if color mixtures obey certain linear mathematical rules, then color space can be
described by a three-dimensional simple geometrical space.”[93]
oRGB: A Practical Opponent Color Space for
Computer Graphics
“Ewald Hering first advocated the opponent process theory of color in the 1870s. It
departed from the prevalent theory of the time, the trichromatic theory of Thomas

Young and Hermann von Helmholtz, by proposing four hue primaries: red, green,
yellow, and blue, instead of the traditionally accepted
three: red, green, and blue.”[232]
A biomedical smart sensor for the visually
impaired
“The main advantage of the epi-retinal implant is the greater ability to dissipate heat
because it is not embedded under tissue. This is a significant consideration in the retina.
The normal temperature inside the eye is less than the normal body temperature of 98.6º
Fahrenheit. Besides the possibility that heat build-up from the sensor electronics could
jeopardise the chronic implantation of the sensor, there is also the concern that the
elevated temperature produced by the sensor could lead to infection, especially since the
implanted device could become a haven for bacteria” [106]
Subretinal versus Epi-retinal
Typically the subretinal implant is embedded `underneath’ the retina and uses
photodiodes to replace damaged photoreceptors, relying on natural sunlight for power.
However as the retina uses Ewald Herings theory[42] in which in 1878 he wrote:
"Yellow can have a red or green tinge, but not a blue one; blue can have only either a
red or a green tinge, and red only either a yellow or a blue one. The four colours can
with complete correctness therefore be described as simple or basic colours, as
Leonardo da Vinci has already done." Whereas at the optic nerve ganglion cells act on
the trichromacy theory[93] i.e. red, green and blue this implies a conversion between
one form to the other. As an epiretinal implant effectively replaces the retinal function
such a conversion can be avoided.