Talk at Pitt

I visited the University of Pittsburgh today to give a colloquium.  I was supposed to have come in February but my plane was cancelled because of a snow storm.  This was not the really big snow storm that closed Washington, DC and Baltimore for a week but a smaller one that hit New England and not the DC area.  My flight was on Southwest and I presume that they have such a tightly correlated flight system, where planes circulate around the country in a “just in time” fashion, that a disturbance in one part of the country affects the rest of the country.  So while other airlines just had cancellations in New England, Southwest flights were cancelled for the day all across the US.  It seems that there is a trade off between business efficiency and robustness.  I drove this time. My talk was on the finite size effects in the Kuramoto model, which I’ve given several times already.  However, I have revised the slides on pedagogical grounds and they can be found  here.

Slides for second SIAM talk

Here are the slides for my SIAM talk on generalizing the Wilson-Cowan equations to include correlations.  This talk was mostly on the paper with Michael Buice and Jack Cowan that I summarized  here.  However, I also contrasted our work with the recent work of Paul Bressloff who uses a system size expansion of the Markov process that Michael and Jack proposed as a microscopic model for Wilson-Cowan in their 2007 paper.  The difference between the two approaches stems from  the interpretation of what the Wilson-Cowan equation describes.  In our interpretation, the Wilson-Cowan equation describes the firing rate or stochastic intensity of a Poisson process.  A Poisson distribution is notable because all cumulants are equal to the mean.  Our expansion is  in terms of factorial cumulants (we called them normal ordered cumulants in the paper because we didn’t know there was a name for them), which are deviations from Poisson statistics.  Bressloff, on the other hand, considers the Wilson -Cowan equation to be the average population firing rate of  a large population of neurons.  In the infinite size limit, there are no fluctuations.  His expansion is in terms of regular cumulants and the inverse system size is the small parameter.  In our formulation, the expansion parameter  is related to the distance to a critical point where the expansion would break down.   In essence, we use a Bogoliubov  hierarchy of time scales expansion where the higher order  factorial cumulants decay to steady state much faster than the lower order ones.

Some numbers for the BP leak

The Deepwater Horizon well is situated 1500 m below the surface of the Gulf of Mexico.  The hydrostatic pressure is approximately given by  the simple formula of where is the pressure of the atmosphere,    is the density of water, and is the gravitational acceleration.  Putting the numbers together gives , which is or about 150 times atmospheric pressure.  Hence, the oil and natural gas must be under tremendous pressure to be able to leak out of the well at all.  It’s no wonder the Top Kill operation, where mud was pumped in at high pressure, did not work.

Currently, it is estimated that the leak rate is somewhere between 10,000 and 100,000 barrels of oil per day.  A barrel of oil is 159 litres or 0.159 cubic metres.  So basically 1600 to 16000 cubic metres of oil is leaking each day.  This amounts to a cube with sides of about 11 metres for the lower value and 25 metres for the upper one, which is about the length of a basketball court.  However, assuming that the oil forms a layer on the surface of the ocean that is 0.001 mm thick, this then corresponds to a slick with an area between 1,600 to 16,000 square kilometres.  Given that the leak has been going on for almost two months and the Gulf of Mexico is 160,000 square kilometres, this implies that the slick is either very thick, oil has started to wash up on shore, or a lot of the oil is still under the surface.

The scale invariant life

The most recent episode of WNYC’s Radiolab was about human limits.  The first two stories were on the limits of the human body and mind and had people telling stories of surviving extreme endurance events and participating in memory competitions.  The last story was about the limits of science.  I was expecting the usual take on Godel’s Incompleteness Theorems but they touched on something different.  Instead they talked about how this algorithm called Eureqa written by two Cornell computer scientists could deduce the dynamics of unknown systems.  They used it to deduce the equations of a double pendulum based on the time series of the angles and angular velocities.  They then applied it to a biological system and produced some dynamical equations.  However, they then claimed that they had trouble publishing the results because they couldn’t explain what those equations described or meant.  Steve Strogatz then came on and started to lament on the fact that as we begin to explore more and more complex systems, our brains may not be able to ever understand it.  He basically said that once we reached that limit we may need to hand over science to computers.

I think that Steve is confounding the limits of a single human being with the limits of humans in general.  To me, understanding is all about data compression.  One says they understand something when they can give a simpler description of it or relate it to something they know already.  Understanding is not a binary process.   I understand some things better than other things and I attain a greater and sometimes lesser understanding of things the more I think about them.  However, I do agree that there may be limits to the number of different things I personally can understand.  This applies to things that other humans already understand like for example Turkish.  Now, perhaps if I studied hard enough I could learn to speak Turkish but the time it took for me to do that would preclude me from learning something else like say Category Theory.

What Steve was specifically referring to I believe was that it may be difficult or impossible to understand certain complex systems by trying to relate them to what we know now. I think this is probably true but that doesn’t mean we won’t have intuitive understanding of such systems in the future.  For example,  it would be very difficult for an adult 500 years ago, who should be genetically indistinguishable to a person alive today, to understand Andrew Wile’s proof of Fermat’s last theorem.  Most of them would first have to learn how to read and then learn 500 years of mathematics.  Wiles basically used everything humans know about math up to this point to prove the theorem.  The average mathematician alive today that doesn’t specialize in arithmetic algebraic geometry has trouble understanding the proof.  They simply don’t have the background to follow all the arguments.

Now, I do believe that there are things that we can never understand  because we are bound by the rules of computation.  Turing showed us that there are undecidable problems that cannot be solved in general like if a computation will halt.  However, I do think that humans have the capability to understand individual things that arise out of computations and that includes physical objects like biological systems. We may not know whether a given computation will halt but we could understand what has already been computed.   For complex systems that Steve was alluding to, we just don’t yet know what the form of that understanding will be.  Consider Brownian motion, which is the modern paradigm of an unpredictable process.  Until  Einstein pointed out that the process should be understood probabilistically and calculated  the time dependence of the mean square deviation of a Brownian particle people didn’t even know how to think about the phenomenon.  I think most physicists would claim that they have a good understanding of Brownian motion even though they have no idea what a single trajectory of a Brownian particle will do.  From a neuroscience perspective, Brownian motion has become a primitive concept and we can understand more complex things in terms of it.  I think this will hold true for even more complex phenomenon.  We can always reform what we consider to be intuitive and build from that.

Addendum:  I forgot to relate everything back to the title of the post.  I called this the scale invariant life because I think everyone will go through similar stages where they learn what is known, make some new discoveries and then reach a crisis where they can’t understand something new in terms of what they already  know.  Thus there are no absolute thresholds of discovery or knowledge.  We just make excursions from where we start and then the next generation takes over.

Boltzmann’s Brain and Universe

One of the recent results of string theory is the revitalization of an old idea for the origin of the universe first proposed by Boltzmann.  This was nicely summarized in an article by Dennis Overbye in the New York Times. Cosmologist Sean Carroll has also blogged about this multiple times  (e.g. see here and here). Boltzmann suggested that the universe, which is not in thermal equilibrium, could have arisen as a fluctuation from a bigger universe in a state of thermal equilibrium.  (This involves issues of the second law of thermodynamics and the arrow of time, which I’ll post on at some later point.)  A paper by Dyson, Kleban and Susskind in 2002, set off a round of debates in the cosmology community because this idea leads to what is now called the Boltzmann’s brain paradox.  The details are nicely summarized in Carroll’s posts. Basically, the idea is that if a universe could arise out of a quantum fluctuation then a disembodied brain should also be able to pop into existence and since a brain is much smaller than the entire universe then it should be more probable.  So, why is it that we are not disembodied brains?

I had two thoughts when I first heard about this paradox.  The first was – how do you know you’re not a disembodied brain? and the second was -  it is not necessarily true that the brain is simpler than the whole universe.  What the cosmologists seem to be ignoring or discounting is nonlinear dynamics and computation.  The fact that the brain is contained in the universe  doesn’t mean it must be simpler.  They don’t take into account the possibility that the Kolmogorov complexity, which is the smallest description of an entity, of the universe is smaller than that of the brain.  So although the universe is much bigger than the brain and contains many brains among other things, it may in fact be less complex.  Personally, I happen to like the spontaneous fluctuation idea for the origin of our universe.

Two talks at University of Toronto

I’m currently at the University of Toronto to give two talks in a series that is jointly hosted by the Physics department and the Fields Institute.  The Fields Institute is like the Canadian version of the Mathematical Sciences Research Institute in the US and is named in honour of  Canadian mathematician J.C. Fields, who started the Fields Medal (considered to be the most prestigious prize for mathematics).  The abstracts for my talks are here.

The talk today was a variation on my kinetic theory of coupled oscillators talk.  The slides are here.  I tried to be more pedagogical in this version and because it was to be only 45 minutes long, I also shortened it quite a bit.   However, in many ways I felt that this talk was much less successful than the previous versions.  In simplifying the story, I left out much of the history behind the topic and thus the results probably seemed somewhat disembodied.  I didn’t really get across why a kinetic theory of coupled oscillators is interesting and useful.   Here is the post giving more of the backstory on the topic, which has a link to an older version of the talk as well.  Tomorrow, I’ll talk about my obesity work.

Hurdles for mathematical thinking

From my years as both a math professor and observer of people, I’ve come up with a list of  hurdles for mathematical thinking.  These are what I believe to be the essential set of skills  a  person must have if they want to understand and do mathematics.  They don’t need to have all these skills to use mathematics but would need most of them if they want to progress far in mathematics.  Identifying what sorts of conceptual barriers people may have could help in improving mathematics education.

I’ll first give the list and then explain what I mean by them.

1. Context dependent rules

2. Equivalence classes

3. Limits and infinitesimals

4. Formal  logic

5. Abstraction

Human scale

I’ve always been intrigued by how long we live compared to the age of the universe.  At 14 billion years, the universe is only a factor of older than a long-lived human.  In contrast, it is immensely bigger than us.  The nearest star is 4 light years away, which is a factor of larger than a human, and the observable universe is about 25 billion times bigger than that.    The size scale of the universe is partly dictated by the speed of light which at m/s is coincidentally (or not) the same order of magnitude faster than we can move as the universe is older than we live.

Although we are small compared to the universe, we are also exceedingly big compared to our constituents. We are comprised of about cells, each of which are about m in diameter.  If we assume that the density of the cell is about that of water () then that roughly amounts to molecules.  So a human is comprised of something like molecules, most of it being water which has an atomic weight of 18.  Given that proteins and organic molecules can be much larger than that a lower bound on the number of atoms in the body is .

The speed at which we can move is governed by the reaction rates of metabolism.  Neurons fire at an average of approximately 10 Hz, so that is why awareness operates on a time scale of a few hundred milliseconds.  You could think of a human moment as being one tenth of a second.  There are 86,400 seconds in a day so we have close to a million moments in a day although we are a sleep for about a third of them.  That leads to about 20 billion moments in a lifetime. Neural activity also sets the scale for how fast we can move our muscles, which is a few metres per second.  If we consider a movement every second then that implies about a billion twitches per lifetime.  Our hearts beat about once a second so that is also the number of heart beats in a lifetime.

The average thermal energy at body temperature is about Joules, which is not too far below the binding energies of protein-DNA and protein-protein interactions required for life.   Each of our  cells can translate about 5 amino acids per second, which is a lot of proteins in our lifetime.  I find it completely amazing that  a bag of or more things, incessantly buffeted by noise, can stay coherent for a hundred years.  There is no question that evolution is the world’s greatest engineer.  However, for those that are interested in artificial life this huge expanse of scale does pose a question -  What is the minimal computational requirement to simulate life and in particular something as complex as a mammal?  Even if you could do a simulation with say or more objects,  how would you even know that there was something living in it?

The numbers came from Wolfram Alpha and Bionumbers.

Energy efficiency and boiling water

I’ve noticed that my last few posts have been veering towards the metaphysical so I thought today I would talk about some kitchen science, literally. The question is what is the most efficient way to boil water.  Should one turn the heat on the stove to the maximum or is there some mid-level that should be used?  I didn’t know what the answer was so I tried to calculate it.  The answer turned out to be more subtle than I anticipated.

Feynman’s “The Character of Physical Law”

Richard Feynman’s famous 1964 lecture series “The character of physical law“, is now available on the web curtesy of Bill Gates, who bought the rights.  For any of you who are uamiliar with Feynman, I would recommend watching the brilliant physicist and expositor at work.  There are seven lectures in total.  I watched the first, describing the law of grativation,  and the fifth on the distinction of past and future, where he gives a beautiful and clear explanation of entropy and the arrow of time.

Feynman also anticipates complexity theory in the fifth lecture.  He says that knowing the fundamental laws don’t help much in understanding complex phenomena like entropy.  There is a lot of analysis that must be done to get there.  He also talks about the hierachy of descriptions from the laws of elementary forces and particles all the way up to human concepts like beauty and hope.  He then says that he doesn’t believe either end of this spectrum or any of the steps in between are any more “closer to God”, i.e. more fundamental than any other.  He jokes that the people working on these very different fields should not have any animosity towards each other and that they’re all doing essentially the same thing, which is to try to relate the various levels of the hierarchy to each other.

Talk at NJIT

I was at the FACM ’09 conference held at the New Jersey Institute of Technology the past two days.  I gave a talk on “Effective theories for neural networks”.  The slides are here.  This was an unsatisfying talk on two accounts.  The first was that I didn’t internalize how soon this talk came after the Snowbird conference and so I didn’t have enough time to properly prepare.   I thus ended up giving a talk that provided enough information to be confusing and hopefully thought provoking but not enough to be understood.   The second problem was that there is a flaw in what I presented.

I’ll give a brief backdrop to the talk for those unfamiliar with neuroscience.  The brain is composed of interconnected neurons and as a proxy for understanding the brain, computational neuroscientists try to understand what a collection of coupled neurons will do.   The state of a neuron is characterized by the voltage across its membrane and the state of its membrane ion channels.  When a neuron is given enough input,  there can be a  massive change of voltage and flow of ions called an action potential.  One of the ions that flows into the cell is calcium, which can trigger the release of neurotransmitter to influence other neurons.  Thus, neuroscientists are highly focused on how and when action potentials or spikes occur.

We can thus model a neural network at many levels.  At the bottom level, there is what I will call a microscopic description where we write down equations for the dynamics of the voltage and ion channels for each neuron.  These neuron models are sometimes called conductance-based neurons and the Hodgkin-Huxley neuron is the first and most famous of them.  They usually consist of two to four differential equations and can easily be a lot more.  On the other hand, if one is more interested in just the spiking rate,  then there is a reduced description for that.  In fact, much of the early progress in mathematically understanding neural networks used rate equations, examples being Wison and Cowan, Grossberg, Hopfield and Amari.  The question that I have always had was what is the precise connection between a microscopic description and a spike rate or activity description.  If I start with a network of conductance-based neurons can I derive the appropriate activity based description?

Snowbird conference

I’m currently at the SIAM Dynamical Systems meeting in Snowbird, Utah.  I gave a short version of my talk on calculating finite size effects of the Kuramotor coupled oscillator model using kinetic theory and path integral approaches.  Here is the longer and more informative version of the talk.  I summarized the papers on this talk here.

Revised version of paper

We’ve just uploaded a revised version of our paper: Systematic fluctuation expansion for neural network activity equations, by Buice, Cowan and Chow to the arXiv. Hopefully, this is more readable (especially the path integral section) than the previous version.

Curve fitting

I think it’s fair to say that many physicists have little knowledge of statistics. As I posted previously, this mostly arises because there is little need for statistics in physics (see here and here). As a result, whenever they see someone attempting to explain data or extract information from data by fitting a statistical model, they’ll simply dismiss it as “merely curve fitting”. I’m going to argue here that this perceived dichotomy between a mechanistic model and a statistical model is artificial and in fact both camps could profit immensely by learning from each other. To a statistician, a model generally means capturing the data in terms of a smaller number of degrees of freedom. The model chosen will then depend on the data and any prior information. They then spend most of their effort testing the validity of their model and assessing the significance of the fit. To a physicist, a model implies some mechanistic description, often in the form of a function or differential equations that are based on prior knowledge that is independent of the particular data set.

Systematic fluctuation expansion for neural networks slides

I gave a talk today at the Mathematical Neuroscience workshop on my recent work with Michael Buice and Jack Cowan on deriving generalized activity equations for neural networks.  My slides are here.  The talk is based on the paper we uploaded to the arXiv recently and I summarized here.

Path integral methods for stochastic equations

I’m currently in Edinburgh for a Mathematical Neuroscience workshop. I gave a tutorial today on using field theoretic methods to solve stochastic differential equations (SDE’s). The slides are here.  The methods I presented have been around for decades but as far as I know they haven’t been collated together into a pedagogical review for nonexperts.  Also, there is an entire community of theorists and mathematicians that are unaware of path integral methods.  In particular, I apply the response function formalism stemming from the work of Martin Siggia and Rose.  Field theory and diagrammatic methods are a nice way to organize perturbation expansions for nonlinear SDE’s.  I plan to write a review paper on this topic in the next few months and will post it here.

Addendum: Jan 20, 2011.  The review paper can be found here.

Why can’t economics be more like physics?

In recent months, physicists have been lamenting that economics needs to be completely revamped to be more like physics (for example see here).  Meanwhile, some nonphysicists are blaming physicists for our current mess (for example see here).  I will argue in this post that economics is not more like physics mostly because it is not like physics.  I also think that everyone on Wall Street from the CEO to the quants must share in the blame for the credit crisis.  However, I don’t blame the models they have used.  All models are based on a set of assumptions and it is up to the modeler to decide when they hold.

The first reason why economics and especially macroeconomics is not like physics is that it is almost impossible to verify theories.  The problem is even worse than in biology because in biology you can at least try to average over cells or organisms.  But in economics you only have one sample.  The analogy to physics would be to ask when a specific spin in a magnet would flip. The theories can only predict what the distribution will be.  For example, we can never really know if the stimulus package recently passed in the US will actually work because we can’t do the controlled experiment where we see what happens if we didn’t have the stimulus.  Even economists agree on this point (for example see what Harvard economist Greg Mankiw says).   So you can have prominent economists vehemently disagreeing on basic points like whether or not more government spending will help and they can always point to reasons for why they are correct.  That is one of the reasons why economics is so mathematical.  Without any strong empirical evidence, the best you can do is to prove theorems.

Systematic fluctuation expansion for neural networks

A new paper  “Systematic fluctuation expansion for neural network activity equations“, by Michael Buice, Jack Cowan and myself has just been uploaded to the q-bio arXiv.  The paper arose from a confluence of my desire to adapt moment hierarchy approaches from kinetic theory to studying fluctuations in neural networks and Michael and Jack’s field theory formulation of stochastic neural dynamics (see here).  In this paper, we show that the two approaches are identical and give a systematic scheme to derive the equations.  We give an example for self-consistent equations for the first two moments.

Classically, neural networks have been described either by rate equations, such as the Wilson-Cowan equation of the form (and the continuum version) or networks of (more biophysical)  spiking neurons.  Although rate equations average over neural spikes,  they have been extremely successful in describing many neural phenomena.  Wilson and Cowan, Grossberg, Amari, Hopfield, Ermentrout, and many others, have used these types of equations to describe phenomena as diverse as associative memory, working memory, persistent activity, hallucinations, orientation tuning, and neural activity waves.  In fact, the term neural network, has essentially been co-opted to imply a network of rate equations (i.e., multi-layer perceptron) with a back propagation learning rule for the weights to perform supervised learning.

Why can’t biology or economics be more like physics?

February 12 was the 200th anniversay of Charles Darwin and Abraham Lincoln. I’m not going to comment on either of the two directly in this post (given the large amount of press devoted to them recently) even though their impact on our lives cannot be overstated.  What I do want to talk about is whether or not biology and economics can be more like physics.  I will do this in two parts, with this post focusing on  biology. By “not like physics”, I mean that there is not a more quantitative and unifying approach to biology.  I think many physicists feel that biologists miss the big picture and that much more could be gleaned if they only started to think like physicists.    This attitude is perfectly represented in biophysicist Bob Austin’s letter to Physics Today a decade ago, which can be found here.  I think this view has evolved recently as more physicists work on biology but I still see it.

Although I am a former physicist,  I’m going to take the side of the biologists.  I’m not saying that biology couldn’t be more quantitative and better understood.  I’m also not saying that ideas from physics couldn’t be useful.  These are all probably true.  What I am saying is that the reason biology is not more like physics isn’t because biologists are misguided (or as Bob Austin puts it “can’t reason their way out of a paper bag”) but because biology is different from physics.

Kinetic theory of coupled oscillators

Last week, I gave a physics colloquium at the Catholic University of America about recent work on using kinetic theory and field theory approaches to analyze finite-size corrections to networks of coupled oscillators.  My slides are here although they are converted from Keynote so the movies don’t work.   Coupled oscillators arise in  contexts as diverse as the brain, synchronized flashing of fireflies, coupled Josephson junctions, or unstable modes of the Millennium bridge in London.  Steve Strogatz’s book Sync gives a popular account of the field.  My talk considers the Kuramoto model

   (1)

where the frequencies are drawn from a fixed distribution . The model describes the dynamics of the phases of an all-to-all connected network of oscillators.  It can be considered to be the weak coupling limit of a set of nonlinear oscillators with different natural frequencies and a synchronizing phase response curve.