New paper on learning in spiking neural networks

Chris Kim and I recently published a paper in eLife:

Learning recurrent dynamics in spiking networks.

Abstract

Spiking activity of neurons engaged in learning and performing a task show complex spatiotemporal dynamics. While the output of recurrent network models can learn to perform various tasks, the possible range of recurrent dynamics that emerge after learning remains unknown. Here we show that modifying the recurrent connectivity with a recursive least squares algorithm provides sufficient flexibility for synaptic and spiking rate dynamics of spiking networks to produce a wide range of spatiotemporal activity. We apply the training method to learn arbitrary firing patterns, stabilize irregular spiking activity in a network of excitatory and inhibitory neurons respecting Dale’s law, and reproduce the heterogeneous spiking rate patterns of cortical neurons engaged in motor planning and movement. We identify sufficient conditions for successful learning, characterize two types of learning errors, and assess the network capacity. Our findings show that synaptically-coupled recurrent spiking networks possess a vast computational capability that can support the diverse activity patterns in the brain.

 

The ideas that eventually led to this paper were seeded by two events. The first was about five years ago when I heard Dean Buonomano talk about his work with Rodrigo Laje on how to tame chaos in a network of rate neurons. Dean and Rodrigo expanded on the work by Larry Abbott and David Sussillo. The guiding idea from these two influential works stems from the “the echo state machine” or “reservoir computing”. Basically, this idea exploits the inherent chaotic dynamics of a recurrent neural network to project inputs onto diverse trajectories from which a simple learning rule can be deployed to extract a desired output.

To explain the details of this idea and our work, I need to go back to Minsky and Papert and their iconic 1969 book on feedforward neural networks (called perceptrons), who divided learning problems into two types.  The first type is linearly separable, which means that if you want to learn a classifier on some inputs, then a single linear plane can be drawn to separate the two input classes on the space of all inputs. The classic example is the OR function.  When given inputs (x_1,x_2) = (0,1), (1,0), (1,1), it outputs 1 and when given (0,0) it outputs 0. If we consider the inputs on the x-y plane then we can easily draw a line separating point (0,0) from the rest. The classic linearly non-separable problem is exclusive OR or XOR, where (0,0) and (1,1) map to 0, while (0,1) and (1,0) map to 1. In this case, no single straight line can separate the points. Minsky and Papert showed that a single layer perceptron, where the only thing you learn is the connection strengths from input to output, can never learn a linearly inseparable problem. Most interesting and nontrivial problems are inseparable.

Mathematically, we can write a perceptron as x_i^{\alpha+1} = \sum w_{ij}^{\alpha}f^{\alpha}(x_j^{\alpha}), where x_i^{\alpha} is the value of neuron i in layer \alpha and f is a connection or gain function. The inputs are x_i^{0} and the output are x_i^{L}. The perceptron problem is to find a set of w‘s such that the output layer gives you the right value to a task posed in the input layer, e.g. perform XOR. A single layer perceptron is then simply x_i^{1} = \sum w_{ij}^{\alpha}f^{\alpha}(x_j^{0}). Now of course we could always design f to do what you ask but since we are not training f, it needs to be general enough for all problems and is usually chosen to be a monotonic increasing function with a threshold. Minsky and Papert showed that the single layer problem is equivalent to the matrix equation w = M v and this can never solve a linearly inseparable problem, since it defines a plane. If f is a linear function then a multiple layer problem reduces to a single layer problem so what makes perceptron learning and deep learning possible is that there are multiple layers and f is a nonlinear function. Minsky and Paper also claimed that there was no efficient way to train a multi-layered network and this killed the perceptron for more than a decade until backpropagation was discovered and rediscovered in the 1980’s. Backprop rekindled a flurry of neural network activity and then died down because other machine learning methods proved better at that time. The recent ascendancy of deep learning is the third wave of perceptron interest and was spurred by the confluence of 1) more computing power via the GPU, 2) more data, and 3) finding the right parameter tweaks to make perceptron learning work much much better. Perceptrons still can’t solve everything, e.g. NP complete problems are still NP complete, they are still far from being optimal, and they do not discount a resurgence or invention of another method.

The idea of reservoir computing is to make a linearly inseparable problem separable by processing the inputs. The antecedent is the support vector machine or kernel method, which projects the data to a higher dimension such that an inseparable problem is separable. In the XOR example, if we can add a dimension and map (0,0) and (1,1) to (0,0,0) and (1,1,0) and map (1,0) and (0,1) to (1,0,1) and (0,1,1) then the problem is separable. The hard part is finding the mapping or kernels to do this. Reservoir computing uses the orbit of a chaotic system as a kernel. Chaos, by definition, causes initial conditions to diverge exponentially and by following a trajectory for as long as you want you can make as high dimensional a space as you want; in high enough dimensions all points are linearly separable if they are far enough apart. However, the defining feature of chaos is also a bug because any slight error in your input will also diverge exponentially and thus the kernel is inherently unstable. The Sussillo and Abbott breakthrough was that they showed you could have your cake and eat it too. They stabilized the chaos using feedback and/or learning while still preserving the separating property. This then allowed training of the output layer to work extremely efficiently. Laje and Bunomano took this one step further by showing that you could directly train the recurrent network to stabilize chaos. My thought at that time was why are chaotic patterns so special? Why can’t you learn any pattern?

The second pivotal event came in a conversation with the ever so insightful Kechen Zhang when I gave a talk at Hopkins. In that conversation, we discussed how perhaps it was possible that any internal neuron mechanism, such as nonlinear dendrites, could be reproduced by adding more neurons to the network and thus from an operational point of view it didn’t matter if you had the biology correct. There would always exist a recurrent network that could do your job. The problem was to find the properties that make a network “universal” in that it could reproduce the dynamics of any other network or any dynamical system. After this conversation, I was certain that this was true and began spouting this idea to anyone who would listen.

One of the people I mentioned this to was Chris Kim when he contacted me for a job in my lab in 2015. Later Chris told me that he thought my idea was crazy or impossible to prove but he took the job anyway because he wanted to be in Maryland where his family lived. So, upon his arrival in the fall of 2016, I tasked him with training a recurrent neural network to follow arbitrary patterns. I also told him that we should do it on a network of spiking neurons. I thought that doing this on a set of rate neurons would be too easy or already done so we should move to spiking neurons. Michael Buice and I had just recently published our paper on computing finite size corrections to a spiking network of coupled theta neurons with linear synapses. Since we had good control of the dynamics of this network, I thought it would be the ideal system. The network has the form

\dot\theta_i = f(\theta_i, I_i, u_i)

\tau_s \dot u_i= - u_i+ 2 \sum_j w_{ij}\delta(\theta_j-\pi)

Whenever neuron j crosses the angle \pi it gives an impulse to neuron i with weight scaled by w_{ij}, which can be positive or negative. The idea is to train the synaptic drive u_i(t) or the firing rate of neuron i to follow an arbitrary temporal pattern. Despite his initial skepticism, Chris actually got this to work in less than six months. It took us another year or so to understand how and why.

In our paper, we show that if the synapses are fast enough, i.e. \tau_s is small enough, and the patterns are diverse enough, then any set of patterns can be learned. The reason, which is explained in mathematical detail in the paper, is that if the synapses are fast enough, then the synaptic drive acts like a quasi-static function of the inputs and thus the spiking problem reduces to the rate problem

\tau_s \dot u_i= - u_i+ \sum_j w_{ij}g(u_j)

where g is the frequency-input curve of a theta neuron. Then the problem is about satisfying the synaptic drive equation, which given the linearity in the weights, boils down to whether \tau_s \dot u_i + u_i is in the space spanned by \sum w_{ij} g(u_j),  which we show is always possible as long as the desired patterns imposed on u_i(t) are uncorrelated or linearly independent enough. However, there is a limit to how long the patterns can be, which is governed by the number of entries in w_{ij}, which is limited by the number of neurons. The diversity of patterns limitation can also be circumvented by adding auxiliary neurons. If you wanted some number of neurons to do the same thing, you just need to include a lot of other neurons that do different things. A similar argument can be applied to the time averaged firing rate (on any time window) of a given neuron. I now think we have shown that a recurrent network of really simple spiking neurons is dynamically universal. All you need are lots of fast neurons.

 

Addendum: The dates of events may not all be correct. I think my conversation with Kechen came before Dean’s paper but my recollection is foggy. Memories are not reliable.

 

Advertisements

Mosquito experiment concluded

It’s hard to see from the photo but when I checked my bucket after a week away, there were definitely a few mosquito larvae swimming around. There was also an impressive biofilm on the bottom of the bucket. It took less than a month for mosquitoes to breed in a newly formed pool of stagnant water. My son also noticed that a nearby flower pot with water only a few centimeters deep also had larvae. So the claims that mosquitos will breed in tiny amounts of stagnant water is true.IMG_3158

Insider trading

I think one of the main things that has fueled a backlash against the global elites is the (correct) perception that they play by different rules. When they make financial mistakes, they get bailed out with taxpayer dollars with no consequences. Gains are privatized and losses are socialized. Another example is insider trading where people profit from securities transactions using nonpublic information. While there have been several high profile cases in recent years (e.g. here is a Baltimore example), my guess is that insider trading is rampant since it is so easy to do and so hard to detect. The conventional wisdom for combating insider trading is stronger enforcement and penalties. However, my take is that this will just lead to a situation where small time insider traders get squeezed out while the sophisticated ones who have more resources will continue. This is an example where a regulation creates a monopoly or economic rent opportunity.

Aside from the collapse of morality that may come with extreme wealth and power (e.g. listen here), I also think that insider traders rationalize their activities because they don’t think that it hurts anyone even though there is an obvious victim. For example, if someone gets inside information that a highly touted drug has failed to win approval from the FDA then they can short the stock (or buy put options), which is an agreement or opportunity to sell the stock at the current price in the future. When the stock decreases in value after the announcement, they just buy the stock at the lower price, resell at the higher price, and reap the profits. The victim is the counter party to the trade who could be a rich trader but could also be someone’s pension fund or employees of the company.

Now the losing party or a regulatory agency could suspect a case of insider trading but to prove it would require someone confessing or finding an email or phone recording of the information passed. They could also try to set up a sting operation to try to catch serial violators. All of these things are difficult and costly. The alternative may seem ridiculous but I think the best solution may be to make insider trading legal. If it were legal then several things would happen. More people would do it which would drive down the prices for the trades, the information would more likely be leaked to the public since people would not be afraid of sharing it, and people would be more careful in making trades prior to big decisions because the other party may have more information than they do. Companies would be responsible for policing people in their firms that leak information. By making insider information legal, the rent created by regulations would be reduced.

Selection of the week

Sorry for the long radio silence. However, I was listening to the radio yesterday and this version of the Double Violin Concerto in D minor, BWV 1043 by JS Bach came on and I sat in my car in a hot parking lot listening to it. It’s from a forty year old EMI recording with violinists Itzhak Perlman and Pinchas Zukerman with Daniel Barenboim conducting the English Chamber Orchestra. I’ve been limiting my posts to videos of live performances but sometimes classic recordings should be given their due and this is certainly a classic. Even though I posted a version with Oistrakh and Menuhin before, I just had to share this.

What Uber doesn’t get

You may have heard that ride hailing services Uber and Lyft have pulled out of Austin, TX because they refuse to be regulated. You can read about the details here. The city wanted to fingerprint drivers, as they do for taxis, but Uber and Lyft forced a referendum on the city to make them exempt or else they would leave. The city voted against them. I personally use Uber and really like it but what I like about Uber has nothing to do with Uber per se or regulation. What I like is 1) no money needs to be exchanged especially the tip and 2) the price is essentially fixed so it is in the driver’s interest to get me to my destination as fast as possible. I have been taken on joy rides far too many times by taxi drivers trying to maximize the fare and I never know how much to tip. However, these are things that regulated taxis could implement and should implement. I do think it is extremely unfair that Uber can waltz into a city like New York and compete against highly regulated taxis, who have paid as much as a million dollars for the right to operate. Uber and Lyft should collaborate with existing taxi companies rather than trying to put them out of business. There was a reason to regulate taxis (e.g. safety, traffic control, fraud protection), and that should apply whether I hail a cab on the street or I use a smartphone app.

New review paper on GWAS

Comput Struct Biotechnol J. 2015 Nov 23;14:28-34
Uncovering the Genetic Architectures of Quantitative Traits.
Lee JJ, Vattikuti S, Chow CC.

Abstract
The aim of a genome-wide association study (GWAS) is to identify loci in the human genome affecting a phenotype of interest. This review summarizes some recent work on conceptual and methodological aspects of GWAS. The average effect of gene substitution at a given causal site in the genome is the key estimand in GWAS, and we argue for its fundamental importance. Implicit in the definition of average effect is a linear model relating genotype to phenotype. The fraction of the phenotypic variance ascribable to polymorphic sites with nonzero average effects in this linear model is called the heritability, and we describe methods for estimating this quantity from GWAS data. Finally, we show that the theory of compressed sensing can be used to provide a sharp estimate of the sample size required to identify essentially all sites contributing to the heritability of a given phenotype.
KEYWORDS:
Average effect of gene substitution; Compressed sensing; GWAS; Heritability; Population genetics; Quantitative genetics; Review; Statistical genetics