Attractor networks in systems with underlying random connectivity

P.E. Latham

Most treatments of Hopfield networks [1] use a deterministic weight matrix. We consider the more realistic case in which the weight matrix has an underlying random component associated with sparse connectivity and heterogeneous coupling. Randomly connected networks of excitatory and inhibitory neurons (i.e., networks with no structure) exhibit, over a broad range of parameters, a single stable state at low firing rate. We investigate the conditions for the formation of additional fixed points -- new memories -- as connectivity among subpopulations of neurons increases. Using both mean field analysis and simulations with spiking model neurons, we find that there is a threshold connectivity for memories to form, that memories form at high firing rate, and that memories can form only if the neuronal transfer function -- the firing rate of a post-synaptic neuron as a function of the mean firing rates of the pre-synaptic neurons coupled to it -- has a particular shape.

1. J.J. Hopfield, PNAS, 1982.