New paper on neural networks

Michael Buice and I have just published a review paper of our work on how to go beyond mean field theory for systems of coupled neurons. The paper can be obtained here. Michael and I actually pursued two lines of thought on how to go beyond mean field theory and we show how the two are related in this review. The first line started in trying to understand how to create a dynamic statistical theory of a high dimensional fully deterministic system. We first applied the method to the Kuramoto system of coupled oscillators but the formalism could apply to any system. Our recent paper in PLoS Computational Biology was an application for a network of synaptically coupled spiking neurons. I’ve written about this work multiple times (e.g. here,  here, and here). In this series of papers, we looked at how you can compute fluctuations around the infinite system size limit, which defines mean field theory for the system, when you have a finite number of neurons. We used the inverse number of neurons as a perturbative expansion parameter but the formalism could be generalized to expand in any small parameter, such as the inverse of a slow time scale.

The second line of thought was with regards to the question of how to generalize the Wilson-Cowan equation, which is a phenomenological population activity equation for a set of neurons, which I summarized here. That paper built upon the work that Michael had started in his PhD thesis with Jack Cowan. The Wilson-Cowan equation is a mean field theory of some system but it does not specify what that system is. Michael considered the variable in the Wilson-Cowan equation to be the rate (stochastic intensity) of a Poisson process and prescribed a microscopic stochastic system, dubbed the spike model, that was consistent with the Wilson-Cowan equation. He then considered deviations away from pure Poisson statistics. The expansion parameter in this case was more obscure. Away from a bifurcation (i.e. critical point) the statistics of firing would be pure Poisson but they would deviate near the critical point, so the small parameter was the inverse distance to criticality. Michael, Jack and I then derived a set of self-consistent set of equations for the mean rate and rate correlations that generalized the Wilson-Cowan equation.

The unifying theme of both approaches is that these systems can be described by either a hierarchy of moment equations or equivalently as a functional or path integral. This all boils down to the fact that any stochastic system is equivalently described by a distribution function or the moments of the distribution. Generally, it is impossible to explicitly calculate or compute these quantities but one can apply perturbation theory to extract meaningful quantities. For a path integral, this involves using Laplace’s method or the method of steepest descents to approximate an integral and in the moment hierarchy method it involves finding ways to truncate or close the system. These methods are also directly related to WKB expansion, but I’ll leave that connection to another post.

Failure at all scales

The premise of most political systems since the enlightenment is that the individual is a rational actor. The classical liberal (now called libertarian) tradition believes that social and economic ills are due to excessive government regulation and intervention. If the individuals are left to participate unfettered in a free market then these problems will disappear.  Conversely, the traditional Marxist/Leninist left posits that the capitalistic system is inherently unfair and can only be cured by replacing it with a centrally planned economy. However, the lesson of the twentieth century is that there is irrationality, incompetence, and corruption at all levels, from individuals to societies. We thus need regulations, laws and a government that take into account of the fact that we are fallible at all scales, including the regulations, laws and the government.

Markets are not perfect and often fail but they are clearly superior to central planning for the distribution of most resources (particularly consumer goods). However, they need to be monitored and regulated. When markets fail, government should intervene. Even the staunchest libertarian would support laws that prevent the elimination of your competitors by violence. Organized crime and drug cartels are an example of how businesses would run in the absence of laws. However, regulations and laws should have built-in sunset clauses that force them to be reviewed after a finite length of time. In some cases, a freer market makes sense. I believe that the government is bad in picking winners so if we want to promote alternative energy, we shouldn’t be helping nascent green industries but rather tax fossil fuel use and let the market decide what is best. Making cars more fuel-efficient may not lead to less energy use but just encourage people to drive more. If we want to save energy, we should make energy more expensive. We should also make regulations as universal and simple as possible to minimize  regulatory capture. I think means testing for social services like medicare is a bad idea because it will just encourage people to find clever ways to circumvent it. The same probably goes for need-based welfare. We should just give everyone a minimum income and let everyone keep any income above it. This would then provide a safety net but not a disincentive to work. Some people will choose to live on this minimum income but as I argued here, I think they should be allowed to. If we want to address wealth inequality then we should probably tax wealth directly rather than income. We want to encourage people to make as much money as possible but then spend it to keep the wealth circulating. By the same reasoning, I don’t like a consumption tax. Our economy is based on consumer spending so we don’t want to discourage that (unless it is for other reasons than economic).

People do not suddenly become selfless and rational when the political system changes but systems can mitigate the effects of their irrational and selfish tendencies. As the work of Kahneman, Tversky, Ariely, and others have shown, rational and scientific thinking does not come naturally to people. Having the market decide what is the most effective medical treatment is not a good idea. A perfect example is in a recent Econtalk podcast with libertarian leaning economist John Cochrane on healthcare. Cochrane suggested that instead of seeing a doctor first, he should just be allowed to buy antibiotics for his children whenever they had an earache. The most laughable part was his idea that we have rules against self-administering of drugs to protect uneducated people. Actually, the rules are to protect highly educated people like him who think that expertise in one area transfers to another. The last thing we want is for even more antibiotic use and more antibiotic resistant bacterial strains. I definitely do not want to live in a society where I have to wait for the market to penalize companies that provide unsafe food or build unsafe buildings. It doesn’t help me if my house collapses in an earthquake because the builder used inferior materials. Sure they may go out of business but I’m already dead.

There is no single perfect system or set of rules that one should always follow. We should design laws, regulations, and governments that are adaptable and adjust according to need. The US Constitution has been amended 27 times. The last time was in 1992, which just changed the rules on salaries for elected officials. The 26th amendment in 1971 made 18 the universal threshold age for voting. We are thus due for another amendment and I think the 2nd amendment, which guarantees the right to bear arms, is a place to start. We could make it more explicit what types of arms are protected and what types can be regulated by local laws. If we want to reduce gun violence then gun regulation makes sense. People will do things they later regret. If one is in the heat of an argument and there is a gun available then it could be used inadvertently. It takes a lot of training and skill to use a gun effectively. Accidents will happen. In the case of guns, failure often leads to death. I would prefer to live in a society where guns are scarce rather than one where everyone carries a weapon like the old wild west.