Lecture slides

The laminar connectivity basis of hierarchical relationships between cortical areasV2descending connectionV5

The laminar connectivity basis of hierarchical relationships between cortical areasV6‘intermediate’ connectionV5

Felleman & Van Essen 1991[ref. 4]A systematic hierarchy can only beconstructed if the ‘rules’ of laminarconnectivity are universal:1. Distinct patterns of termination:ascending pathways terminate in layer 4descending pathways terminate in layers 1 & 6.2. Reciprocity:if area ‘A’ sends an ascending output to area ‘B’, then‘B’ sends a descending output to ‘A’.- area B is termed a ‘higher’ area3. Transitivity:if area ‘B’ is higher than area ‘A’,and area ‘C’ is higher than ‘B’-- then area ‘C’ will also be found to be higher than‘A’. .

Retino-geniculo-cortical channelsRETINALATERAL GENICULATE NUCLEUSLGNKPM+ colour+ directionorientationnon-orientedPRIMARY VISUAL CORTEX V1

LGN – receptive field properties of 3 different channelsPPPPMMKSW +LW – MW –SW –LW + MW+wavelengthopponent;co-extensiveLW +MW –LW -MW +MW-LW+MW+LW -wavelengthopponent;concentricLW+ MW+LW - MW -LW – MW -LW + MW +broadband;concentric(spatialopponent)equivalent class retinal ganglion neuron ‘bistratified’ ‘midget’ ‘parasol’Koniocellular Parvocellular Magnocellular

RECEPTIVE FIELD PROPERTIES OF LGN CELLSPROPERTYMAGNOCELLULARPARVOCELLULARKONIOCELLULARspectralbroadbandLW v. MWopponentbroadband &SW v. LW & MW oppnt.spatiallarge, opponent RF’ssmall, opponent RF’slarge non-opponentRF’scontrasthigh sensitivitylow sensitivitymediumtemporal responsetransientsustainedvariousflickerhigh fusion ratelow fusion ratevariousmorphologylarge cell bodysmall cell bodytiny cell bodyaxonal signalvelocityfastslowvery slowproportion1101

Retino-geniculo-cortical channelsRETINALATERAL GENICULATE NUCLEUSLGNKPM+ colour+ directionorientationnon-orientedFeedback to LGNPRIMARY VISUAL CORTEX V1

The ventral visual pathway... a chain of areas achieving object recognitionAreaV1AreaV4C o l o u rV i s i o n

‘R E F L E C T A N C E’reflectance = reflected light intensityincident light intensityThe goal of colourperception is to ‘see’ thespectral reflectance functionof the surfaces in view.

Mondrian display under chromatically varying illuminationUse of the multicoloured ‘Mondrian’ display, under illumination of varied spectralcomposition, to test wavelength and colour selectivity of cortical neurons.

LW illumination‘White light’ illuminationMW LWMW LWMW LWsame ratio;differing intensityMW LWdiffering ratio;differing intensityCONES:MW LWCONES:MW LWV1 ‘red’ neuron responds well to red areaV1 ‘red’ neuron responds mildly to red areaV1 ‘red’ neuron: responds whenever LW light is dominant within RF- i.e. when LW:MW cone ratio > 1

LW illumination‘Greenish-White light’ illuminationMW LWMW LWMW LWsame ratio;differing intensityMW LWdiffering ratio;differing intensityCONES:MW LWCONES:MW LWV1 neuron responds well to ‘red’ areaV1 neuron unresponsive to ‘red’ areaV1 ‘red’ neuron: responds when LW light is dominant within RF;- Fails to respond when LW:MW cone ratio < 1

LW illumination‘White light’ illuminationMW LWMW LWMW LWCONES:MW LWsame ratio;differing intensityMW LWCONES:MW LWdiffering ratio;differing intensityV4 neuron unresponsive to ‘red’ area V4 neuron responds to ‘red’ area –that has maximal ratio of LW:MW cone activityV4 ‘red’ neuron with hypothetical centre-surround RF:- responds when centre LW:MW cone ratio is greater than average surround LW:MW cone ratio

LW illumination‘Greenish-White light’ illuminationMW LWLW:MW = 0.5MW LWMW LWCONES:MW LWsame ratio;differing intensityLW:MW = 0.9MW LWLW:MW = 0.7CONES:MW LWdiffering ratio;differing intensityV4 neuron unresponsive to ‘red’ area V4 neuron responds to ‘red’ area –that has maximal ratio of LW:MW cone activityV4 ‘red’ neuron with hypothetical centre-surround RF:- responds when centre LW:MW cone ratio is greater than average surround LW:MW cone ratio

Colour constancy depends on ‘discounting the illuminant’!In this example, the percept behaves as if some yellowness hasbeen subtracted from the grey surface, such that it appears blue.

ON centreOFF surroundBlue ON centreBlue OFF surroundYellow OFF centreYellow ON surroundbaseline

perceivedB/Y (V4 activity profile)B/Y double opponent output(V1 / V2 )Profile of spectral composition in imagelessmoreyellowyellowretinal imageThe brain (i.e. area V4 ?) reconstructs a Y/B profile from chromatic edge signals, and adjusts for overallexcess of yellow (or blue) light.

DOUBLE OPPONENT CELLS (V1, V2)theoretical construction of receptive fieldsColour cell receptive fieldin visual cortex (V1/V2)Receptive fields as found in retina & LGNLW +MW ---SW - ( LW + MW)LW -MW +LW+ MW+LW- MW-+MW-LW +--MW+LW -(LW + MW) - SWLW- MW-LW+ MW+centre: [ SW – (MW+LW)] +surround: [SW – (MW+ LW)] -­‐ wavelengthopponent;co-extensivewavelengthopponent;concentricbroadbandconcentric(spatial opponent)

DOUBLE OPPONENT CELLS (V1, V2)theoretical construction of receptive fieldsReceptive fields as found in retina & LGNColour cell receptive fieldin visual cortex (V1/V2)+-SW - ( LW + MW)LW +MW -LW -MW +MW-LW +LW+ MW+LW- MW-MW+LW -(LW + MW) - SWLW- MW-LW+ MW+centre: [ LW – MW] +surround: [LW – MW)] -­‐centre: LW +surround: LW -­‐MW -­‐ MW+wavelengthopponent;co-extensivewavelengthopponent;concentricbroadbandconcentric(spatial opponent)

DOUBLE OPPONENT CELLS IN V1 or V2--+--centre: [ SW – (MW+LW)] +surround: [SW – (MW+ LW)] -­‐centre: [(LW – MW)] +surround: [(LW – MW)] -­‐

Colour constancy depends on ‘discounting the illuminant’!Can a blue-detecting cell respond to agrey surface under conditions of excessyellow illumination ?(i.e. when that grey surface has anillusory blue appearance).

Kusunoki, Moutoussis & Zeki (2006)[ref 6]Colourinduction in V4 neuronstest (‘blue’) neuron responsemonkey’s perceptual judgment

The ventral visual pathway... a chain of areas achieving object recognitionAreaV1F o r mV i s i o n

Hubel & Wiesel: hypothesis for construction of ‘simple’ and ‘complex’ receptive fieldsconcentric(e.g. layer 4)‘simple’ RF‘simple’ cell‘complex’ RF‘simple’ cells‘complex’ cell

Von der Heydt &Peterhans 1989[ref. 5]responses to illusory contoursrecordings in area V2

[ref 7]Gallant et al. 1996V4 neurons tested with Cartesian, polar and hyperbolic gratingsExample responses of individual neuronsSome cells werehighly selective withinthis stimulus space...Cell ‘2H5E0’ wasselectively responsiveto hyperbolic gratingsonly;The response of theneuron is coded by theblue-red ‘heat’ scale.Cell ‘2H2I0’ wasselective for cartesiangratings.

[ref 7]Gallant et al. 1996V4 neurons tested with Cartesian, polar and hyperbolic gratingsExample responses of individual neurons...other neuronswere less selective.

Model of object recognition withinfull ventral visual pathwayCadieu, Poggio et al (2007)[ref 12]

Hubel & Wiesel: hypothesis for construction of ‘simple’ and ‘complex’ receptive fieldsconcentric(e.g. layer 4)‘simple’ RFT. Poggio: ‘Selectivity’ operation creates specificity of response= Logical ‘AND’‘simple’ cell‘complex’ RF‘simple’ cells‘complex’ cellT. Poggio: ‘Max’ operation creates invariance of response= Logical ‘OR’

Model of object recognition withinfull ventral visual pathwayCadieu et al (2007) [12]Actual response of V4 neuronC2S2S2S2S2S2S2S2S2S2Model of ‘S2’ neuronPasupathy & Connor (2001) [13]Shape representation in V4: positionspecifictuning for boundary conformation.The intensity of the response is givenby the background greyscale:darkest = most activePooled ‘C1’ inputs

The ventral visual pathway...F o r mV i s i o nAreas TEO & TE- known together as ‘IT cortex’ (inferior temporal)

Tanaka et al 1991/2003[ref. 8]complexobject sensitivity of IT cellsThe ‘image reduction’ method

Tanaka et al 1991 [ref.8]complex object sensitivity of IT cells

Tanaka et al 1991/2003 [8,10]complex object sensitivity of IT cells‘size invariance’

Tanaka et al 1991/2003 [8,10]complex object sensitivity of IT cells

Tanaka et al 1991/2003 [8,10]complex object sensitivity of IT cells

Tanaka et al 1991/2003 [8,10]complex object sensitivity of IT cells

Tanaka et al 1991/2003 [8,10]complex object sensitivity of IT cellscombination of colourand form selectivity

Tanaka / Rolls distributed code for object representation(as opposed to ‘Grandmother’ cells)broad categorisation mammalcomparisondogbeaglenarrowcomparison identification ‘Snoopy’Tanaka’s model of IT cortex- nearby columns have similar properties

Desimone et al 1984 [ref. 9] STS cells sensitive to face stimuli

Desimone et al 1984[ref. 9]selectivityto profile ( = ‘view selectivity’)

Perret et al (1992)[ref 14]Hierarchical Modelview invariantcell5%Hebbian wiringlatency = 130 msecP o o l e dI n p u t sview selectivecells95%latency = 120msec

HEBB’S LAW (1949)If presynaptic activity frequently coincides with postsynaptic depolarisation, and spiking……the potency of the synapse will increase.‘Cells that fire together wire together’ Donald Hebb 1904-1985

Perret et al (1992)[ref 14]- distribution of face-sensitivecells in anterior temporal lobeSuperiorTemporalSulcusInferotemporal cortex

Perret et al (1992)[ref 14]- distribution of face-sensitivecells in anterior temporal lobeTsao et al (2006)Face processing centres in STSobserved by fMRISuperiorTemporalSulcusInferotemporal cortex

Tsao et al 2008 [15]Comparing face patch systems in macaques & humansmacaquehumanPosterior lateralMiddle lateralAnterior lateralAnterior medialMiddle fundalAnterior fundal

Friewald & Tsao 2010 [16]Face neurons: (a) view selectivity & view invariance;(b) individual face selectivity.1 cmStimuli8 face view anglesNeurophysiology12.......25 different subject identitiesRecordings in 4 patches:ML & MFALAM

Friewald & Tsao 2010 [16]Face neurons: (a) view selectivity & view invariance;(b) individual face selectivity.1 cmCell 1 Cell 2 Cell 3C e l l p o p u l a t i o nImage no.ML/MFCell 4 Cell 5 Cell 6C e l l p o p u l a t i o nImage no.ALCell 7 Cell 8 Cell 9C e l l p o p u l a t i o nImage no.AMTime msec

Friewald & Tsao 2010 [16]Hierarchical modelAM patchView invariantlatency = 124 msecAL patchView symmetricallatency = 104 msecML/MF patchesView selectivelatency = 88 msec

Friewald & Tsao 2010 [16]Hierarchical modelRecursive inhibitionenhances subject specificityAM patchView invariantlatency = 124 msecLATERALINHIBITIONAL patchView symmetricallatency = 104 msecML/MF patchesView selectivelatency = 88 msec

Freiwald et al 2009 [ref 17] ‘middle face patch’ neuron properties:Cartoon faces with 7 elements:face outline (aspect ratio), hair, eyes, irises, eyebrows, nose & mouthHair widthEyebrow slantEyebrow heightInter-ocular distanceIris sizeShape of top of mouth

Freiwald et al 2009 [17] ‘middle face patch’ neuron properties:1. Selectivity for multiple face features;

Freiwald et al 2009 [17] ‘middle face patch’ neuron properties:1. Selectivity for multiple face features;

Freiwald et al 2009 [17] ‘middle face patch’ neuron properties:1. Selectivity for multiple face features;2. Less selectivity for inverted facesSelective response to aninverted feature is less frequent*Selective response to mouth became more frequentSelective response to eyebrows was lost

Freiwald et al 2009 [17] ‘middle face patch’ neuron properties:1. Selectivity for multiple face features;2. Less selectivity for inverted faces;3. ‘Ramp’ tuning curves– i.e. maximally driven by extreme feature settings

Visual evidence seeking an explanation…Visual hypothesis seeking corroborative evidence

Perception as an active process of ‘hypothesis testing’...What we perceive is not invariably determined by what we see (i.e., by the imageformed on the retina):Ambiguous / bistable percepts;Illusions of 3D depth;*Prior assumptions about the sceneNecker CubeSwiss CrystallographerLouis Necker (1832)Rubin’s VaseDanish psychologistEdgar Rubin (1915)Face mask illusion‘illumination fromabove’ prior

Predictive Coding – an information processing theory requiring iterative hierarchical computationprediction error = sensory data - prediction‘error’units‘state’units(or ‘expectation’ unit)The free-energy principle: a rough guide to the brain? Friston KJ, Trends in Cognitive Sciences 13: 293-301 (2009).

Schematic model of intrinsic & extrinsic wiring subserving predictive coding:reciprocal exchange of prediction & precision (backward) & error (forward)- e.g. as modelled by serial hierarchical connections between sensory cortical areasLEVEL (i-1) LEVEL (i) LEVEL (i+1)predictionpredictionExpectation unit Error unit Precision unitexcitatory connectioninhibitory connectionReflections on agranular architecture: predictive coding in the motor cortex. Shipp S, Trends in Neuroscience 36: 706-716 (2013)

Note on neural implementation of predictive coding (previous slide)1. This is all rather conjectural, an exercise in fitting the theoretical computational procedures of predictive coding to the known patterns of intrinsicand extrinsic connectivity of the cortex.2. The fundamental idea underlying predictive coding is that all perception is a guess (or hypothesis) – it has to be, because the retinal image neverprovides information adequate to specify exactly what is present, where, in the visual field.3. The basic computational mechanism is that backward connections convey predictions (blue), and forward connections convey prediction error(green); a second class of backward connection conveys ‘precision’ (red).4. Correspondingly, there should be three discernible classes of pyramidal neurons: expectation (EXP) and precision units that are the source of backprojections, and error (ERR) units that are the source of forward projections.5. The signals carried by the backward projections of EXP units can be regarded as predictions of the activity of EXP units at the level below.6. The ERR units within a level compute the difference between the EXP units in the same level, and the prediction arriving from the level above. TheERR signal is conveyed to the level above, where it modifies the activity of the EXP units at that level.7. A simpler way of thinking about this is that the ERR units compute the difference between the sensory data reaching an area and the prediction ofthat data provided by higher areas. Note that the difference is computed as a simple subtraction (data – prediction), so the prediction shouldreach the ERR units via an inhibitory intrinsic relay.8. The combined activity of EXP units across areas determines our percept of the image on the retina; the activity of ERR units has the effect ofmodifying that percept.9. Precision is a term used to describe the gain of the ERR signal (i.e. that signal is boosted by high precision, and attenuated by low precision).Precision therefore regulates the balance between expectation and sensory evidence (data) in determining what we see.10. The various examples given of a prior hypothesis governing what we see, to resolve ambiguity or even to produce an illusion (e.g. that light comesfrom above, that staircases do not have gaps, and that vertebrate heads are convex, not concave structures) would be rationalised ascircumstances where the prior hypothesis is very strong, and the precision applied to the ERR signal is not strong enough to allow contrarysensory evidence to modify the representation of the hypothesis by EXP units in high level visual areas.GlossaryIntrinsic connections are those within an area, extrinsic connections are those between areasPyramidal neurons are excitatory (i.e. use an excitatory neurotransmitter) as opposed to inhibitory interneurons