FIAS Scientific Report 2011 - Frankfurt Institute for Advanced Studies ...

Self-organization in recurrent neural networks with multiple forms of plasticity
Collaborators: P. Zheng 1 , C. Dimitrakakis 1,2 , A. Lazar 1,3 , G. Pipa 1,4 , D. Krieg 1 , P. Sterne 1 , L. Aitchison 5 ,
J. Triesch 1
1 Frankfurt Institute for Advanced Studies, 2 EPFL, Switzerland, 3 Max-Planck Institute for Brain Research Frankfurt,
4 University of Osnabrück, 5 Gatsby Computational Neuroscience Unit, UK
Understanding the network structure of cortical circuits is of fundamental importance for understanding cortex
function. The distribution of synaptic strengths of local excitatory connections in the cortex is long-tailed
(approximately lognormal), but individual synapses can undergo strong fluctuations from day to day. This raises
the question how the brain can maintain stable long-term memories at all. Recent evidence has shown that very
strong synapses are relatively more stable than weak ones and could thus be the physiological basis of longlasting
memories. At present it is unclear, however, how the measured distribution of synaptic strengths and the
pattern of synaptic fluctuations arise. Our explanation is based on concepts from self-organization. We show
that the distribution of synaptic strengths and their pattern of fluctuations are explained by self-organization
in a recurrent spiking network model combining spike-timing-dependent plasticity, synaptic scaling, structural
plasticity, and intrinsic plasticity. In this network, STDP induces a rich-get-richer mechanism for excitatory
synapses, while synaptic scaling induces competition between them. The resulting self-organization produces a
network with irregular activity patterns consistent with cortical recordings, lognormal-like weight distributions,
and patterns of synapse fluctuations closely matching experimental data (see Figure). This self-organization
is very robust to parameter changes in the network but critically depends on the presence of the different
homeostatic plasticity mechanisms. Our findings have far reaching implications for the development, structure,
and function of cortical circuits. First, it reveals the fundamental importance of different homeostatic plasticity
mechanisms for the organization of cortex. Second, it sheds light on the structure of spontaneous activity in
the cortex, which has traditionally been treated as noise. Our work suggests that this activity is in fact highly
organized and essential for shaping network structure and function.
In a second line of research we study these self-organizing networks from the viewpoint of information theory
and try to understand their learning mechanisms as approximations to the maximization of mutual information.
In a similar vein, we have developed a theory of spike-timing dependent plasticity and other forms of synaptic
long-term plasticity that tries to bridge biophysical and computational levels of description.
Figure: Distribution of synaptic
weights in our model matches lognormal
distribution of EPSPs in
cortex. A: distribution of EPSP
sizes from Song et al. (2005) and
lognormal fit. B: distribution of
weight strength in our model and
lognormal fit.
Related publications:
1) P. Zheng, C. Dimitrakakis, J. Triesch, Network Self-organization Explains the Statistics and Dynamics of
Synaptic Connection Strengths in Cortex, in preparation.
2) A. Lazar, G. Pipa, J. Triesch, Emerging Bayesian Priors in a Self-Organizing Recurrent Network, Proc. of
the International Conference on Artificial Neural Networks (ICANN), 2011.
3) D. Krieg, J. Triesch, A unifying theory of synaptic long-term plasticity based on a sparse synaptic strength,
submitted.
4) P. Sterne, L. Aitchison, J. Triesch, SORN network dynamics approximately maximise mutual information, in
preparation.
81