Sequence learning in attractor networks with the BCPNN learning rule

There is a long-standing interest in the relationship between sequences of brain activity and behavior and cognition. Hebb theorized that sequential activation of cell assemblies (“phase-sequences”) would form the basis of “the thought process” [1]. Sequential neural activity that is time-locked to behavior has been observed in the neural circuits of the hippocampus [2], cortex [3] and in the “high vocal center” (HVC ) in birds [4]. Revealing the underlying neural mechanisms of these sequential neural activity patterns and explaining how these correlates participate in information processing remains an open task in computational neuroscience.
Recent computational modelling has provided us with frameworks for understanding sequential activity in both the cortex [5] and the striatum [6]. However, we find that the question of the sequence storage capacity of such models has not been to date addressed systematically. That is, there is no systematic account of how many sequences an attractor neural network can reliably learn without interference. Here we present advances in this direction. Building on the previous modelling efforts in our lab [5, 7] we show 1) an instance of a firing rate model with the BCPNN learning rule that reliably stores sequences, 2) a parameterization of the sequence space that allows us to systematically address the question of storage capacity, and 3) a comparative analysis of the functionality of our network model with capabilities of the state-of-the-art models.

References
[1] D. O. Hebb, The Organization of Behavior. New York: John Wiley Inc., 1949., 10.1016/S0361-9230(99)00182-3
[2] E. Pastalkova, V. Itskov, A. Amarasingham, and G. Buzsáki, “Internally Generated Cell Assembly Sequences in the Rat Hippocampus,” Science (80-. )., vol. 321, no. 5894, 2008., 10.1126/science.1159775
[3] A. Luczak, P. Barthó, S. L. Marguet, G. Buzsáki, and K. D. Harris, “Sequential structure of neocortical spontaneous activity in vivo.,” Proc. Natl. Acad. Sci. U. S. A., vol. 104, no. 1, pp. 347–52, Jan. 2007., 10.1073/pnas.0605643104
[4] R. H. R. Hahnloser, A. A. Kozhevnikov, and M. S. Fee, “An ultra-sparse code underliesthe generation of neural sequences in a songbird,” Nature, vol. 419, no. 6902, pp. 65–70, Sep. 2002., 10.1038/nature00974
[5] P. J. Tully, H. Lindén, M. H. Hennig, A. Lansner, and M. Lundqvist, “Spike-Based Bayesian-Hebbian Learning of Temporal Sequences,” PLOS Comput. Biol., vol. 12, no. 5, p. e1004954, May 2016., 10.1371/journal.pcbi.1004954
[6] J. M. Murray and G. S. Escola, “Learning multiple variable-speed sequences in striatum via cortical tutoring,” Elife, vol. 6, May 2017., 10.7554/eLife.26084
[7] Sandberg, Anders, et al. "A Bayesian attractor network with incremental learning." Network: Computation in neural systems13.2 (2002): 179-194.