Persistent activity

Whenever I think about the recession and the stimulus package, I’m reminded of the phenomenon of working memory and persistent activity in the brain, which I’ve worked on in the past.  I’ll explain the connection at the end of the post. Working memory is the short term memory we use when we remember a phone number just long enough to call someone and then forget shortly afterward.  Neuroscientists have found neurons in the pre-frontal cortex of monkeys that are correlated with the memory.  So when you present the monkey a transient stimulus that it must remember, these neurons start firing  and remain activated until the memory is no longer needed.  This is a neural correlate of working memory.  This implies that there must be bistability (or multistability) in the firing state of a neuron.

Hence, either a single neuron is bistable or a network of neurons are bistable.  Except for some special circumstances, bistability in neurons have not been observed.  Therefore, the network model is more likely in my opinion.  So, how do you get network bistability?  Well that can occur through recurrent excitation.  For example, if a group of neurons are connected to each other with excitatory connections then it is possible that when they fire they excite each other and maintain firing.  We can see this with a simple model.  Let r be the firing rate of a neural unit (this could be one neuron or a pool of neurons). The firing rate is determined by the inputs u to the neuron through what is called the gain function f(u), which is usually some sigmoidal function.  The firing rate obeys the equation \displaystyle r=f(u) .  Now recurrent activity means that the inputs to the network are due to the firing of other neurons in the network so we can set u=w r and obtain the condition for a self-consistent activity state to be r=f(w r).    Graphically, this looks like this:

curiefigure

In the figure, there are three possible solutions given by the intersections of the red and blue lines. Thus far there are no dynamics.    We can add some dynamics by considering first order kinetics via \tau \dot{r} = f(w r)-r.  For these dynamics, it is easy to show that the middle solution is unstable while the other two are stable so we have a simple model of a two state  working memory.  A transient stimulus can kick the state from one stable state to the other.

Now this is for a single variable but we can generalize to a network.  For example, suppose that there is a network of neural units labeled by i, so that the firing rate of each unit is given by r_i = f(u_i).  Now if we assume that the input the unit u_i is due to a sum over the firing rates of the other neurons in the network through a weight function then we can write u_i=\sum_j^N w_{ij} r_j, and we get the equation r_i=f(\sum_j w_{ij} r_j).  There is a nice symmetry between firing rate and input because this also implies that u_i=\sum_j w_{ij} f(u_i), which is the form I like to use.  So, now depending on the choice of the weight function w_{ij} you can again get persistent actiivty of various shapes.  If \displaystyle w_{ij} is a “Mexican hat” function (e.g.  w_{ij}= A\exp(-a(i-j)^2) - B\exp(-b(i-j)^2)), then you can get localized “bumps” of persistent activity.   Adding dynamics and taking a continuum limit gives  a “neural field” equation of the form

\partial_t u(x,t) = -u(x,t) + \int w(x-y) f[u(y)] dy

which has been analyzed by many people, including myself.  Examples of equilibrium bump solutions of these equations are shown here (Guo and Chow, 2005):

Amaribigsmall

Generally, you get a stable big bump and an unstable small bump.  The no activity or u=0 state is also a solution.

Now, thus far I’ve only described what are called population rate or mean field equations.  These equations are interpreted as some sort of average over the spiking dynamics of actual neurons.  I’ve been careful not to call the neural units described by r_i neurons because they need not be.  The rigorous transformation of a network of spiking neurons to a network of rate neurons that includes dynamics has yet to be done and is something that Michael Buice and myself are actively pursuing.  For example, there is nothing that says the dynamics should obey first order kinetics.  However, one can demonstrate that the stationary solutions of a network of spiking neurons can be represented by a rate equation where the gain function corresponds to the firing rate function (FI curve)  of a single neuron as long as the neurons fire asynchronously.  Simulations also show that a network of interconnected spiking neurons can exhibit persistent activity.    Below is an example of a raster plot of neural activity.  Each point represents the time a neuron fires.  The firing rate profile is below the raster.

bump15
pro1

One big difference between the mean field approximation and the spiking model is that the active state can shut off if the neurons become too synchronized as seen in this example (Laing and Chow, 2001; Gutkin et al.  2001)

offnoise

An excitatory stimulus starts the bump that is then shut off by a second excitatory synchronizing pulse at time 70.  Synchrony kills persistent activity because the input to a neuron arrives at once and then there isn’t any left to cause the neuron to fire again after its refractory period is over.   Asynchronous or uncorrelated activity is essential for persistent activity.

So now I can make the connection to the economy.  I see the economy as a recurrently connected network.  Instead of passing synaptic pulses to each other we pass money.  The circulation of money keeps the economy going.  The economy also depends on positive feedback.  If everyone suddenly decides to stop shopping then stores and manufacturers will lay off workers  leading to more people  with less money to shop.  On the other hand when people start consuming more then stores an manufacturers will  start to hire again leading to more shoppers.  In the analogy to neural activity, the firing rate r could represent the amount of money or income a person has, which is a function of the cash input to the person. The input in turn is given by a sum over the money of a subset of other people.

Thus, we can envision that there could be multiple possible stable states for  economic activity depending on the economic gain and weight (connectivity) functions and synchronous or correlated activity, like bubbles, can kill activity. We can also see how the a stimulus package could knock the economy out of a recession and into a higher stable state.  However, it is also clear that the timing of a stimulus is important.  For a neural network, the same amount of input spread over a long time is not the same as all of it administered at once.  A very brief stimulus is bad because it synchronizes all the neurons, which then cannot maintain persistent activity.  However, a very slow stimulus never gives enough input to kick the network to the higher state.   This may also be true for the economy.  In the neural system, what determines what is fast or slow is the time constant of the decay of activity.  Knowing the time constant of the economy would then seem to be important for timing a stimulus.  Obviously, the economy is much, much more complicated than this toy model but I think it captures some aspects of it.

References:

C.R. Laing and C.C. Chow, `Stationary bumps in networks of spiking neurons’, Neural Comp. 13, 1473-1494 (2001).

B.S. Gutkin, C.R. Laing, C.L. Colby, C.C. Chow, and G.B. Ermentrout. `Turning on and off with excitation: the role of spike-timing asynchrony and synchrony in sustained neural activity’, J. Comp. Neurosci. 11, 121-134 (2001).

Y. Guo and C.C. Chow, `Existence and stability of standing pulses in neural networks: I. Existence’, SIAM Journal on Applied Dynamical Systems 4, 217-248 (2005) 

Y. Guo and C.C. Chow, `Existence and stability of standing pulses in neural networks: II. Stability’, SIAM Journal on Applied Dynamical Systems 4, 249-281 (2005) 

C.C. Chow and S. Coombes, ‘Existence and Wandering of bumps in a spiking neural network model’. SIAM Journal on Applied Dynamical Systems 5, 552-574 (2006)

All of these papers can be downloaded from here.

Advertisements

5 thoughts on “Persistent activity

  1. Hi,
    interesting post.
    I guess that when looking at the time constant, there is a big difference between neurons and individuals. Whereas a neurons that receive regularly input spikes every 20 ms may never fire (if the membrane potential resets within e.g. 10 ms), an individual who receives 100$ each month will eventually spend them someday.
    While there is a conservation of money, there is no such thing as a conservation of the spikes or of the depolarisation intensity.
    Another way to say this is that the money time constant is infinite. If you are not a frenetic buyer, whether you get 1200$ dollars each year or 100$ each month is (almost) the same.
    Do you agree?
    Romain

    Like

  2. Hi Romain,

    I would agree that disposable income has no time constant but living expenses have time constants from days to months. Everyone must buy food every day or week, pay the rent every month, buy insurance every year and so forth. So in some sense, they need an injection of money every two weeks or month to stay out of bankruptcy. However, surplus money can be saved or invested and that is different from a simple neuron model but maybe there is an analogy to neuroscience if we include plasticity. Perhaps the surplus could be stored in some other mode of memory, like synaptic strength.

    best,
    Carson

    Like

  3. Hi again,

    when I wrote my first comment, I was more concerned about the relevance of the idea of synchrony. If the time constant is infinite, synchrony is pretty much useless.

    However, I thought about what the french government did to lower the effect of the crisis on the car industry. In 2008, factories started to decrease their production below their maximal level because of the sales drop. In the first semester of 2009, the government gave a 1000 € bonus if you bought a new car while getting rid of your old one.
    What happened? more cars were bought than in 2008. People that wanted to buy a car at the end of the year bought it earlier. They somehow synchronized their expenses (reminiscent of Mirollo-Strogatz).
    But now the bonus is canceled, what will happen? We may fear a even worse drop in the car sales, since everyone already has a new car. The spendings in the car industry may also spread to other industries and thereby boost economy.
    Here, I guess what matters is really the time constant of the diffusion (spread) of the money invested in cars. So I agree with you.

    However, there may be a caveat. Not everyone bought a new car. People that had already bought a car between 2006 and 2008 did not buy a new one. (There may be other reasons for not buying a new one)
    It would be interesting to see in your model the percentage of neurons that need to synchronize to kill the persistent activity, some kind of critical mass.

    Romain.

    PS: as I said in my first comment, this is a very very interesting post!

    Like

  4. Hi Romain,

    The question of critical mass is a very interesting one. Steve Coombes and I sort of did an estimate in our 2005 paper. As you correctly state, the time constant and refractory times are critically important. So suppose there are n neurons firing and m fire synchronously while the others do not. Then you can take one of the m synchronized neurons and ask if it can fire again given that particular input. The critical number will depend on the parameters of the model. For a simple enough neuron model you can calculate it in closed form.

    best,
    Carson

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s