I’m on my way back from the 2011 Joint Mathematics Meeting. I gave a talk yesterday on finite size effects in neural networks. I gave a pedagogical talk on the strategy that Michael Buice and I have employed to analyze finite size networks in networks of coupled spiking neurons. My slides are here. We’ve adapted the formalism we used to analyze the finite size effects of the Kuramoto system (see here for summary) to a system of synaptically coupled phase oscillators.
Archive for the ‘Physics’ Category
I was asked a while ago what I thought of the Large Hadron Collider at CERN. Although I’ve been critical of high energy physics in the past (see for example here), I strongly support the LHC and think it is a worthwhile endeavor. My reason is because I think it will be important for future technology. By this I don’t just mean spin offs, like the World Wide Web, which was invented at CERN by Tim Berners Lee. What I mean is that knowledge gained at the high energy scale could be useful for saving the human race one day.
Let me elaborate. My criticism of high energy or particle physics in the past was mostly because of the claim that it was more “fundamental” than other areas of science like condensed matter physics or psychology. Following noble laureate Philip Anderson’s famous article “More is Different” (Science 177:393-396, 1971), what is fundamental to me is a matter of perspective. For example, the fact that I can’t find a parking spot at the mall a week before Christmas is not because of particle physics but because of the pigeonhole principle, (i.e. if you have more things than boxes, then if you try to put the things into the boxes at least one box must contain more than one thing). This is as fundamental to me as any high energy theory. The fact that you can predict an election using polling data from a small sample of the electorate is because of the central limit theorem, (i.e. the sum of a bunch of random events tends to obey a normal distribution), and is also independent of what particles that comprise the electorate. Ironically, the main raison d’etre of the LHC is to look for the Higgs boson, which is thought to give masses to some subatomic particles. The Higgs mechanism is based on the idea of spontaneous symmetry breaking, which came from none other than Phil Anderson who was studying properties of magnets.
So how could high energy physics be pertinent to our existence some day? Well, some day in the very distant future the sun will expand into a red giant and swallow the earth. If humans, or whatever our descendants will be called, are to survive they are going to need to move. This will take space faring technology that could rely on some yet unknown principle of high energy physics that could be discovered by the LHC. And in the very, very distant future the universe will end either in a big crunch or by expanding so much that matter won’t be able to persist. If and when that time comes and life forms still exist, then to survive they’ll have to figure out how to “tunnel” into a new universe or new existence. This will take real science fiction-like stuff that will likely depend on knowledge of high energy physics. So although high energy physics does not hold a monopoly on fundamental concepts, it may still be absolutely necessary for life saving future technology.
This year’s Nobel prize in physics went to Andre Geim and Konstantin Novoselov for making single layer graphite or graphene using scotch tape. However, Geim is also famous for having won an Ignoble prize for demonstrating diamagnetic levitation using a frog. You can see a video of a flying frog and tomato here. Sir Michael Berry of Berry’s phase fame and Geim wrote a paper demonstrating that diamagnetic but not paramagnetic objects can be levitated stably in a solenoidal magnetic field. This was somewhat surprising because there is a theorem (Earnshaw’s theorem) that says you cannot suspend an object with fixed charges and magnets in any combination of static, magnetic, and gravitational fields. The reason diamagnetic levitation works is because Earnshaw’s theorem does not apply to induced magnetism.
I visited the University of Pittsburgh today to give a colloquium. I was supposed to have come in February but my plane was cancelled because of a snow storm. This was not the really big snow storm that closed Washington, DC and Baltimore for a week but a smaller one that hit New England and not the DC area. My flight was on Southwest and I presume that they have such a tightly correlated flight system, where planes circulate around the country in a “just in time” fashion, that a disturbance in one part of the country affects the rest of the country. So while other airlines just had cancellations in New England, Southwest flights were cancelled for the day all across the US. It seems that there is a trade off between business efficiency and robustness. I drove this time. My talk was on the finite size effects in the Kuramoto model, which I’ve given several times already. However, I have revised the slides on pedagogical grounds and they can be found here.
Here are the slides for my SIAM talk on generalizing the Wilson-Cowan equations to include correlations. This talk was mostly on the paper with Michael Buice and Jack Cowan that I summarized here. However, I also contrasted our work with the recent work of Paul Bressloff who uses a system size expansion of the Markov process that Michael and Jack proposed as a microscopic model for Wilson-Cowan in their 2007 paper. The difference between the two approaches stems from the interpretation of what the Wilson-Cowan equation describes. In our interpretation, the Wilson-Cowan equation describes the firing rate or stochastic intensity of a Poisson process. A Poisson distribution is notable because all cumulants are equal to the mean. Our expansion is in terms of factorial cumulants (we called them normal ordered cumulants in the paper because we didn’t know there was a name for them), which are deviations from Poisson statistics. Bressloff, on the other hand, considers the Wilson -Cowan equation to be the average population firing rate of a large population of neurons. In the infinite size limit, there are no fluctuations. His expansion is in terms of regular cumulants and the inverse system size is the small parameter. In our formulation, the expansion parameter is related to the distance to a critical point where the expansion would break down. In essence, we use a Bogoliubov hierarchy of time scales expansion where the higher order factorial cumulants decay to steady state much faster than the lower order ones.
The Deepwater Horizon well is situated 1500 m below the surface of the Gulf of Mexico. The hydrostatic pressure is approximately given by the simple formula of where is the pressure of the atmosphere, is the density of water, and is the gravitational acceleration. Putting the numbers together gives , which is or about 150 times atmospheric pressure. Hence, the oil and natural gas must be under tremendous pressure to be able to leak out of the well at all. It’s no wonder the Top Kill operation, where mud was pumped in at high pressure, did not work.
Currently, it is estimated that the leak rate is somewhere between 10,000 and 100,000 barrels of oil per day. A barrel of oil is 159 litres or 0.159 cubic metres. So basically 1600 to 16000 cubic metres of oil is leaking each day. This amounts to a cube with sides of about 11 metres for the lower value and 25 metres for the upper one, which is about the length of a basketball court. However, assuming that the oil forms a layer on the surface of the ocean that is 0.001 mm thick, this then corresponds to a slick with an area between 1,600 to 16,000 square kilometres. Given that the leak has been going on for almost two months and the Gulf of Mexico is 160,000 square kilometres, this implies that the slick is either very thick, oil has started to wash up on shore, or a lot of the oil is still under the surface.
The most recent episode of WNYC’s Radiolab was about human limits. The first two stories were on the limits of the human body and mind and had people telling stories of surviving extreme endurance events and participating in memory competitions. The last story was about the limits of science. I was expecting the usual take on Godel’s Incompleteness Theorems but they touched on something different. Instead they talked about how this algorithm called Eureqa written by two Cornell computer scientists could deduce the dynamics of unknown systems. They used it to deduce the equations of a double pendulum based on the time series of the angles and angular velocities. They then applied it to a biological system and produced some dynamical equations. However, they then claimed that they had trouble publishing the results because they couldn’t explain what those equations described or meant. Steve Strogatz then came on and started to lament on the fact that as we begin to explore more and more complex systems, our brains may not be able to ever understand it. He basically said that once we reached that limit we may need to hand over science to computers.
I think that Steve is confounding the limits of a single human being with the limits of humans in general. To me, understanding is all about data compression. One says they understand something when they can give a simpler description of it or relate it to something they know already. Understanding is not a binary process. I understand some things better than other things and I attain a greater and sometimes lesser understanding of things the more I think about them. However, I do agree that there may be limits to the number of different things I personally can understand. This applies to things that other humans already understand like for example Turkish. Now, perhaps if I studied hard enough I could learn to speak Turkish but the time it took for me to do that would preclude me from learning something else like say Category Theory.
What Steve was specifically referring to I believe was that it may be difficult or impossible to understand certain complex systems by trying to relate them to what we know now. I think this is probably true but that doesn’t mean we won’t have intuitive understanding of such systems in the future. For example, it would be very difficult for an adult 500 years ago, who should be genetically indistinguishable to a person alive today, to understand Andrew Wile’s proof of Fermat’s last theorem. Most of them would first have to learn how to read and then learn 500 years of mathematics. Wiles basically used everything humans know about math up to this point to prove the theorem. The average mathematician alive today that doesn’t specialize in arithmetic algebraic geometry has trouble understanding the proof. They simply don’t have the background to follow all the arguments.
Now, I do believe that there are things that we can never understand because we are bound by the rules of computation. Turing showed us that there are undecidable problems that cannot be solved in general like if a computation will halt. However, I do think that humans have the capability to understand individual things that arise out of computations and that includes physical objects like biological systems. We may not know whether a given computation will halt but we could understand what has already been computed. For complex systems that Steve was alluding to, we just don’t yet know what the form of that understanding will be. Consider Brownian motion, which is the modern paradigm of an unpredictable process. Until Einstein pointed out that the process should be understood probabilistically and calculated the time dependence of the mean square deviation of a Brownian particle people didn’t even know how to think about the phenomenon. I think most physicists would claim that they have a good understanding of Brownian motion even though they have no idea what a single trajectory of a Brownian particle will do. From a neuroscience perspective, Brownian motion has become a primitive concept and we can understand more complex things in terms of it. I think this will hold true for even more complex phenomenon. We can always reform what we consider to be intuitive and build from that.
Addendum: I forgot to relate everything back to the title of the post. I called this the scale invariant life because I think everyone will go through similar stages where they learn what is known, make some new discoveries and then reach a crisis where they can’t understand something new in terms of what they already know. Thus there are no absolute thresholds of discovery or knowledge. We just make excursions from where we start and then the next generation takes over.
One of the recent results of string theory is the revitalization of an old idea for the origin of the universe first proposed by Boltzmann. This was nicely summarized in an article by Dennis Overbye in the New York Times. Cosmologist Sean Carroll has also blogged about this multiple times (e.g. see here and here). Boltzmann suggested that the universe, which is not in thermal equilibrium, could have arisen as a fluctuation from a bigger universe in a state of thermal equilibrium. (This involves issues of the second law of thermodynamics and the arrow of time, which I’ll post on at some later point.) A paper by Dyson, Kleban and Susskind in 2002, set off a round of debates in the cosmology community because this idea leads to what is now called the Boltzmann’s brain paradox. The details are nicely summarized in Carroll’s posts. Basically, the idea is that if a universe could arise out of a quantum fluctuation then a disembodied brain should also be able to pop into existence and since a brain is much smaller than the entire universe then it should be more probable. So, why is it that we are not disembodied brains?
I had two thoughts when I first heard about this paradox. The first was – how do you know you’re not a disembodied brain? and the second was - it is not necessarily true that the brain is simpler than the whole universe. What the cosmologists seem to be ignoring or discounting is nonlinear dynamics and computation. The fact that the brain is contained in the universe doesn’t mean it must be simpler. They don’t take into account the possibility that the Kolmogorov complexity, which is the smallest description of an entity, of the universe is smaller than that of the brain. So although the universe is much bigger than the brain and contains many brains among other things, it may in fact be less complex. Personally, I happen to like the spontaneous fluctuation idea for the origin of our universe.
I’m currently at the University of Toronto to give two talks in a series that is jointly hosted by the Physics department and the Fields Institute. The Fields Institute is like the Canadian version of the Mathematical Sciences Research Institute in the US and is named in honour of Canadian mathematician J.C. Fields, who started the Fields Medal (considered to be the most prestigious prize for mathematics). The abstracts for my talks are here.
The talk today was a variation on my kinetic theory of coupled oscillators talk. The slides are here. I tried to be more pedagogical in this version and because it was to be only 45 minutes long, I also shortened it quite a bit. However, in many ways I felt that this talk was much less successful than the previous versions. In simplifying the story, I left out much of the history behind the topic and thus the results probably seemed somewhat disembodied. I didn’t really get across why a kinetic theory of coupled oscillators is interesting and useful. Here is the post giving more of the backstory on the topic, which has a link to an older version of the talk as well. Tomorrow, I’ll talk about my obesity work.
From my years as both a math professor and observer of people, I’ve come up with a list of hurdles for mathematical thinking. These are what I believe to be the essential set of skills a person must have if they want to understand and do mathematics. They don’t need to have all these skills to use mathematics but would need most of them if they want to progress far in mathematics. Identifying what sorts of conceptual barriers people may have could help in improving mathematics education.
I’ll first give the list and then explain what I mean by them.
1. Context dependent rules
2. Equivalence classes
3. Limits and infinitesimals
4. Formal logic
I’ve always been intrigued by how long we live compared to the age of the universe. At 14 billion years, the universe is only a factor of older than a long-lived human. In contrast, it is immensely bigger than us. The nearest star is 4 light years away, which is a factor of larger than a human, and the observable universe is about 25 billion times bigger than that. The size scale of the universe is partly dictated by the speed of light which at m/s is coincidentally (or not) the same order of magnitude faster than we can move as the universe is older than we live.
Although we are small compared to the universe, we are also exceedingly big compared to our constituents. We are comprised of about cells, each of which are about m in diameter. If we assume that the density of the cell is about that of water () then that roughly amounts to molecules. So a human is comprised of something like molecules, most of it being water which has an atomic weight of 18. Given that proteins and organic molecules can be much larger than that a lower bound on the number of atoms in the body is .
The speed at which we can move is governed by the reaction rates of metabolism. Neurons fire at an average of approximately 10 Hz, so that is why awareness operates on a time scale of a few hundred milliseconds. You could think of a human moment as being one tenth of a second. There are 86,400 seconds in a day so we have close to a million moments in a day although we are a sleep for about a third of them. That leads to about 20 billion moments in a lifetime. Neural activity also sets the scale for how fast we can move our muscles, which is a few metres per second. If we consider a movement every second then that implies about a billion twitches per lifetime. Our hearts beat about once a second so that is also the number of heart beats in a lifetime.
The average thermal energy at body temperature is about Joules, which is not too far below the binding energies of protein-DNA and protein-protein interactions required for life. Each of our cells can translate about 5 amino acids per second, which is a lot of proteins in our lifetime. I find it completely amazing that a bag of or more things, incessantly buffeted by noise, can stay coherent for a hundred years. There is no question that evolution is the world’s greatest engineer. However, for those that are interested in artificial life this huge expanse of scale does pose a question - What is the minimal computational requirement to simulate life and in particular something as complex as a mammal? Even if you could do a simulation with say or more objects, how would you even know that there was something living in it?
I’ve noticed that my last few posts have been veering towards the metaphysical so I thought today I would talk about some kitchen science, literally. The question is what is the most efficient way to boil water. Should one turn the heat on the stove to the maximum or is there some mid-level that should be used? I didn’t know what the answer was so I tried to calculate it. The answer turned out to be more subtle than I anticipated.
Richard Feynman’s famous 1964 lecture series “The character of physical law“, is now available on the web curtesy of Bill Gates, who bought the rights. For any of you who are uamiliar with Feynman, I would recommend watching the brilliant physicist and expositor at work. There are seven lectures in total. I watched the first, describing the law of grativation, and the fifth on the distinction of past and future, where he gives a beautiful and clear explanation of entropy and the arrow of time.
Feynman also anticipates complexity theory in the fifth lecture. He says that knowing the fundamental laws don’t help much in understanding complex phenomena like entropy. There is a lot of analysis that must be done to get there. He also talks about the hierachy of descriptions from the laws of elementary forces and particles all the way up to human concepts like beauty and hope. He then says that he doesn’t believe either end of this spectrum or any of the steps in between are any more “closer to God”, i.e. more fundamental than any other. He jokes that the people working on these very different fields should not have any animosity towards each other and that they’re all doing essentially the same thing, which is to try to relate the various levels of the hierarchy to each other.
I was at the FACM ’09 conference held at the New Jersey Institute of Technology the past two days. I gave a talk on “Effective theories for neural networks”. The slides are here. This was an unsatisfying talk on two accounts. The first was that I didn’t internalize how soon this talk came after the Snowbird conference and so I didn’t have enough time to properly prepare. I thus ended up giving a talk that provided enough information to be confusing and hopefully thought provoking but not enough to be understood. The second problem was that there is a flaw in what I presented.
I’ll give a brief backdrop to the talk for those unfamiliar with neuroscience. The brain is composed of interconnected neurons and as a proxy for understanding the brain, computational neuroscientists try to understand what a collection of coupled neurons will do. The state of a neuron is characterized by the voltage across its membrane and the state of its membrane ion channels. When a neuron is given enough input, there can be a massive change of voltage and flow of ions called an action potential. One of the ions that flows into the cell is calcium, which can trigger the release of neurotransmitter to influence other neurons. Thus, neuroscientists are highly focused on how and when action potentials or spikes occur.
We can thus model a neural network at many levels. At the bottom level, there is what I will call a microscopic description where we write down equations for the dynamics of the voltage and ion channels for each neuron. These neuron models are sometimes called conductance-based neurons and the Hodgkin-Huxley neuron is the first and most famous of them. They usually consist of two to four differential equations and can easily be a lot more. On the other hand, if one is more interested in just the spiking rate, then there is a reduced description for that. In fact, much of the early progress in mathematically understanding neural networks used rate equations, examples being Wison and Cowan, Grossberg, Hopfield and Amari. The question that I have always had was what is the precise connection between a microscopic description and a spike rate or activity description. If I start with a network of conductance-based neurons can I derive the appropriate activity based description?
I’m currently at the SIAM Dynamical Systems meeting in Snowbird, Utah. I gave a short version of my talk on calculating finite size effects of the Kuramotor coupled oscillator model using kinetic theory and path integral approaches. Here is the longer and more informative version of the talk. I summarized the papers on this talk here.
We’ve just uploaded a revised version of our paper: Systematic fluctuation expansion for neural network activity equations, by Buice, Cowan and Chow to the arXiv. Hopefully, this is more readable (especially the path integral section) than the previous version.
I think it’s fair to say that many physicists have little knowledge of statistics. As I posted previously, this mostly arises because there is little need for statistics in physics (see here and here). As a result, whenever they see someone attempting to explain data or extract information from data by fitting a statistical model, they’ll simply dismiss it as “merely curve fitting”. I’m going to argue here that this perceived dichotomy between a mechanistic model and a statistical model is artificial and in fact both camps could profit immensely by learning from each other. To a statistician, a model generally means capturing the data in terms of a smaller number of degrees of freedom. The model chosen will then depend on the data and any prior information. They then spend most of their effort testing the validity of their model and assessing the significance of the fit. To a physicist, a model implies some mechanistic description, often in the form of a function or differential equations that are based on prior knowledge that is independent of the particular data set.
I’m currently in Edinburgh for a Mathematical Neuroscience workshop. I gave a tutorial today on using field theoretic methods to solve stochastic differential equations (SDE’s). The slides are here. The methods I presented have been around for decades but as far as I know they haven’t been collated together into a pedagogical review for nonexperts. Also, there is an entire community of theorists and mathematicians that are unaware of path integral methods. In particular, I apply the response function formalism stemming from the work of Martin Siggia and Rose. Field theory and diagrammatic methods are a nice way to organize perturbation expansions for nonlinear SDE’s. I plan to write a review paper on this topic in the next few months and will post it here.
Addendum: Jan 20, 2011. The review paper can be found here.
In recent months, physicists have been lamenting that economics needs to be completely revamped to be more like physics (for example see here). Meanwhile, some nonphysicists are blaming physicists for our current mess (for example see here). I will argue in this post that economics is not more like physics mostly because it is not like physics. I also think that everyone on Wall Street from the CEO to the quants must share in the blame for the credit crisis. However, I don’t blame the models they have used. All models are based on a set of assumptions and it is up to the modeler to decide when they hold.
The first reason why economics and especially macroeconomics is not like physics is that it is almost impossible to verify theories. The problem is even worse than in biology because in biology you can at least try to average over cells or organisms. But in economics you only have one sample. The analogy to physics would be to ask when a specific spin in a magnet would flip. The theories can only predict what the distribution will be. For example, we can never really know if the stimulus package recently passed in the US will actually work because we can’t do the controlled experiment where we see what happens if we didn’t have the stimulus. Even economists agree on this point (for example see what Harvard economist Greg Mankiw says). So you can have prominent economists vehemently disagreeing on basic points like whether or not more government spending will help and they can always point to reasons for why they are correct. That is one of the reasons why economics is so mathematical. Without any strong empirical evidence, the best you can do is to prove theorems.