Gottingen talk

Here are my slides for the talk I gave today at The 4th workshop on Advanced Methods in Theoretical Neuroscience, Structure and disorder: From random connections to functional circuits, July 10-12 2019, Göttingen, Germany.

Audio of SIAM talk

Here is an audio recording synchronized to slides of my talk a week and a half ago in Pittsburgh. I noticed some places where I said the wrong thing such as conflating neuron with synapse.  I also did not explain the learning part very well. I should point out that we are not applying a control to the network.  We train a set of weights so that given some initial condition, the neuron firing rates follow a specified target pattern. I also made a joke that implied that the Recursive Least Squares algorithm dates to 1972. That is not correct. It goes back much further back than that. I also take a pot shot at physicists. It was meant as a joke of course and describes many of my own papers.

Talk at SIAM Annual Meeting 2017

I gave an invited plenary talk at the 2017 Society of Applied and Industrial Mathematics Annual Meeting in Pittsburgh yesterday. My slides are here. I talked about some very new work on chaos and learning in spiking neural networks. My fellow Chris Kim and I were producing graphs up to a half hour before my talk! I’m quite excited about this work and I hope to get it published soon.

During my talk, I made an offhand threat that my current Mac would be the last one I buy. I made the joke because it was the first time that I could not connect to a projector with my laptop since I started using Mac almost 20 years ago. I switched to Mac from Linux back then because it was a Unix environment where I didn’t need to be a systems administrator to print and project. However, Linux has made major headway in the past two decades while Mac is backsliding. So, I’m seriously thinking of following through. I’ve been slowly getting disenchanted with Apple products over the past three years but I am especially disappointed with my new MacBook Pro. I have the one with the silly touch screen bar. The first thing the peeves me is that the activate Siri key is right next to the delete key so I accidentally hit and then have to reject Siri every five minutes. What mostly ties me to Mac right now is the Keynote presentation software, which I like(d) because it is easy to embed formulas and PDF files into. It is much harder to do the same in PowerPoint and I haven’t found an open source version that is as easy to use. However, Keynote keeps hanging on my new machine. I also get this situation where my embedded equations will randomly disappear and then reappear. Luckily I did a quick run through just before my talk and noticed that the vanished equations reappeared and I could delete them. Thus, the Keynote appeal has definitely diminished. Now, if someone would like to start an open source Keynote project with me… Finally, the new Mac does not seem any faster than my old Mac (it still takes forever to boot up) and Bard Ermentrout told me that his dynamical systems software tool XPP runs five times slower. So, any suggestions for a new machine?

Talk at Jackfest

I’m currently in Banff, Alberta for a Festschrift for Jack Cowan (webpage here). Jack is one of the founders of theoretical neuroscience and has infused many important ideas into the field. The Wilson-Cowan equations that he and Hugh Wilson developed in the early seventies form a foundation for both modeling neural systems and machine learning. My talk will summarize my work on deriving “generalized Wilson-Cowan equations” that include both neural activity and correlations. The slides can be found here. References and a summary of the work can be found here. All videos of the talks can be found here.

 

Addendum: 17:44. Some typos in the talk were fixed.

Addendum: 18:25. I just realized I said something silly in my talk.  The Legendre transform is an involution because the transform of the transform is the inverse. I said something completely inane instead.

Talk at GRC

I’m currently in Mt. Snow, Vermont to give a talk at the Gordon Research Conference on Computer Aided Drug Design. Yes, I know nothing about drug design. I am here because the organizer, Anthony Nicholls, asked me to give a pedagogical talk on Bayesian Inference. My slides are here. I only arrived yesterday but the few talks I’ve seen have been quite interesting. One interesting aspect of this conference is that many of the participants are from industry. The evening sessions are meant to be of more general interest. Last night were two talks about how to make science more reproducible. As I’ve posted before, many published results are simply wrong. The very enterprising Elizabeth Iorns has started something called the Reproducibility Initiative. I am not completely clear about how it works but it is part of another entity she started called Science Exchange, which helps to facilitate collaborations with a fee-for-service model. The Reproducibility Initiative piggy backs on Science Exchange by providing a service (for a fee) to validate any particular result. Papers that pass approval get a stamp of approval. It is expected that pharma would be interested in using this service so they can inexpensively check if possible drug targets actually hold up. Many drugs fail at phase three of clinical trials because they’ve been shown to be ineffective and this may be due to the target being wrong to start with.

On a final note, I flew to Albany and drove here. Unlike in the past when I would have printed out a map, I simply assumed that I could use Google Maps on my smart phone to get here. However, Google Maps doesn’t really know where Mt. Snow is. It tried to take me up a dirt road to the back of the ski resort. Also, just after I turned up the road, the phone signal disappeared so I was blind and had no paper backup. I was suspicious that this was the right way to go so I turned back to the main highway in hopes of finding a signal or a gas station to ask for directions. A few miles down Route 9, I finally did get a signal and also found a sign that led me the way. Google Maps still tried to take me the wrong way. I should have followed what I always tell my daughter – Always have a backup plan.

SIAM Life Sciences Meeting

I’m currently in San Diego for the SIAM Life Sciences conference.  I gave a talk reviewing my work on binocular rivalry.  My slides are here.

Artiefest

Our lab is hosting a symposium at NIH in honor of our chief Arthur Sherman. The meeting will be June 7 and 8 and is open to everyone. The talks will cover a wide range of topics. Information is here.

Criticality

I attended a conference on Criticality in Neural Systems at NIH this week.  I thought I would write a pedagogical post on the history of critical phenomena and phase transitions since it is a long and somewhat convoluted line of thought to link criticality as it was originally defined in physics to neuroscience.  Some of this is a recapitulation of a previous post.

Criticality is about phase transitions, which is a change in the state of matter, such as between gas and liquid. The classic paradigm of phase transitions and critical phenomena is the Ising model of magnetization. In this model, a bunch of spins that can be either up or down (north or south) sit on lattice points. The lattice is said to be magnetized if all the spins are aligned and unmagnetized or disordered if they are randomly oriented. This is a simplification of a magnet where each atom has a magnetic moment which is aligned with a spin degree of freedom of the atom. Bulk magnetism arises when the spins are all aligned.  The lowest energy state of the Ising model is for all the spins to be aligned and hence magnetized. If the only thing that spins had to deal with was the interaction energy then we would be done.  What makes the Ising model interesting and for that matter all of statistical mechanics is that the spins are also coupled to a heat bath. This means that the spins are subjected to random noise and the size of this noise is given by the temperature. The noise wants to randomize the spins. The presence of randomness is why there is the word “statistical” in statistical mechanics. What this means is that we can never say for certain what the configuration of a system is but only assign probabilities and compute moments of the probability distribution. Statistical mechanics really should have been called probabilistic mechanics.

Aspen summer program

The Aspen Center is a famous place where physicists congregate during the summer.  This summer they will be hosting a program on the physics of behavior (see the invitation letter from the organizers below).  This will combine neuroscience and animal behavior and should be very interesting.  I plan to be attending.  I encourage everyone to apply.

The Aspen Center for Physics will be hosting a 3-week long workshop on Physics of Behavior between May 27 and June 16, 2012, with an application deadline of January 31, 2012. The idea of the workshop stems from the understanding that the role of physics in biology is broad, as physical constraints define the strategies and the biological machinery that living systems use to shape their behavior in the dynamic, noisy, and resource-limited physical world. To date, such holistic, physics-driven picture of behavior has been achieved, arguably, only for bacterial chemotaxis. Can a similar understanding emerge for other, more complex living systems?To begin answering this, we would like to use the Aspen Center workshop to bring together a diverse group of scientists, from field biologists to theoretical physicists, broadly interested in animal behavior. We would like to broaden the horizons of physicists by inviting experts who quantify behavior of a wide range of model organisms, from molecular circuits to mammals. We would like to explore behavior as possibly optimal responses given the physical and the statistical structure of environment. Our topics will include, in particular, navigation and foraging, active sensing, locomotion and rhythmic behavior, and learning, memory, and adaptive behaviors.

As workshop organizers, we encourage you to apply.  We would also like you to encourage other people who are active in this field to apply.  We do need to be clear, however, that we cannot guarantee admission to the workshop.  Admission to the workshop is granted not by the workshop organizers, but by the Admissions Committee of the Center (with some input from the workshop organizers).  The Admissions Committee will endeavor to accommodate as many applicants to the Workshop as possible, but because of the constraints imposed by the rest of the AspenCenter for Physics program, they may not be able to admit everyone who applies.

We encourage you to visit the web site of the workshop here , and of the Center, http://www.aspenphys.org/, for more information and for application instructions. For those of you unfamiliar with the Center, it is located in lively and beautiful Aspen, CO. It’s a great place to work, to enjoy the mountains, and to bring  family. The Center partially subsidizes lodging for admitted participants. The Center requires that theorists commit for a minimum stay of two weeks, and a three week stay is preferred. Shorter durations are possible for experimentalists.

We hope you will choose to apply. Please don’t hesitate to contact us if you have questions.

Sincerely,

The Organizers
Ila Fiete, UT Austin
Ilya Nemenman, Emory U
Leslie Osborne, U Chicago
William Ryu, U Toronto
Greg Stephens, Princeton U

Talk in Marseille

I just returned from an excellent meeting in Marseille. I was quite impressed by the quality of talks, both in content and exposition. My talk may have been the least effective in that it provoked no questions. Although I don’t think it was a bad talk per se, I did fail to connect with the audience. I kind of made the classic mistake of not knowing my audience. My talk was about how to extend a previous formalism that much of the audience was unfamiliar with. Hence, they had no idea why it was interesting or useful. The workshop was on mean field methods in neuroscience and my talk was on how to make finite size corrections to classical mean field results. The problem is that many of the participants of the workshop don’t use or know these methods. The field has basically moved on.

In the classical view, the mean field limit is one where the discreteness of the system has been averaged away and thus there are no fluctuations or correlations. I have been struggling over the past decade trying to figure out how to estimate finite system size corrections to mean field. This led to my work on the Kuramoto model with Eric Hildebrand and particularly Michael Buice. Michael and I have now extended the method to synaptically coupled neuron models. However, to this audience, mean field pertains more to what is known as the “balanced state”. This is the idea put forth by Carl van Vreeswijk and Haim Sompolinsky to explain why the brain seems so noisy. In classical mean field theory, the interactions are scaled by the number of neurons N so in the limit of N going to infinity the effect of any single neuron on the population is zero. Thus, there are no fluctuations or correlations. However in the balanced state the interactions are scaled by the square root of the number of neurons so in the mean field limit the fluctuations do not disappear. The brilliant stroke of insight by Carl and Haim was that a self consistent solution to such a situation is where the excitatory and inhibitory neurons balance exactly so the net mean activity in the network is zero but the fluctuations are not. In some sense, this is the inverse of the classical notion. Maybe it should have been called “variance field theory”. The nice thing about the balanced state is that it is a stable fixed point and no further tuning of parameters is required. Of course the scaling choice is still a form of tuning but it is not detailed tuning.

Hence, to the younger generation of theorists in the audience, mean field theory already has fluctuations. Finite size corrections don’t seem that important. It may actually indicate the success of the field because in the past most computational neuroscientists were trained in either physics or mathematics and mean field theory would have the meaning it has in statistical mechanics. The current generation has been completely trained in computational neuroscience with it’s own canon of common knowledge. I should say that my talk wasn’t a complete failure. It did seem to stir up interest in learning the field theory methods we have developed as people did recognize it provides a very useful tool to solve the problems they are interested in.

Addendum 2011-11-11

Here are some links to previous posts that pertain to the comments above.

https://sciencehouse.wordpress.com/2009/06/03/talk-at-njit/

https://sciencehouse.wordpress.com/2009/03/22/path-integral-methods-for-stochastic-equations/

https://sciencehouse.wordpress.com/2009/01/17/kinetic-theory-of-coupled-oscillators/

https://sciencehouse.wordpress.com/2010/09/30/path-integral-methods-for-sdes/

https://sciencehouse.wordpress.com/2010/02/03/paper-now-in-print/

https://sciencehouse.wordpress.com/2009/02/27/systematic-fluctuation-expansion-for-neural-networks/

Snowbird summary

The Snowbird meeting finishes today.  I think it has been highly successful with over 800 participants.  Last night, I was on the “forward looking panel” moderated by Alan Champneys and one of the questions asked was what defines nonlinear dynamics and this meeting.  I gave a rather flip answer about how we are now in the age of machine learning and statistics and this meeting is everything in applied math that is not that.  Of course that is not true given that data assimilation was a major part of this meeting and Sara Solla gave a brilliant talk on applying the generalized linear model to neural data to estimate functional connectivity in the underlying cortical circuit.

Given some time to reflect on that question, I think the common theme of Snowbird is the concept of taking a complicated system and reducing it to something simpler that can be analyzed.   What we do is to create nontrivial models that can be accessed mathematically. This is distinctly different from other branches of applied math like optimization and numerical methods.  However, one difference between previous meetings and now is that before the main tools to analyze these reduced systems were methods of dynamical systems such as geometric singular perturbation theory (e.g. see here) and bifurcation theory.  Today, a much wider range of methods are being utilized.

Another question posed was whether there was too much biology  at this meeting.   I said yes because I thought there were too many parallel sessions.  Although, I said it partially with tongue in cheek, I think there are both good and bad things about biology being overly represented.   It is good that biology is doing well and attracting lots of people but it would be a bad thing if the meeting becomes so large that it devolves into multiple concurrent meetings where people only go to the talks that are directly related to what they already know. In a meeting with fewer parallel sessions one has more chance to learn something new and see something unexpected.  I really have no idea what should be done about this if anything at all.

Finally, a question about how data will be relevant to our field was posed from the audience. My answer was that the big trend right now was in massive data mining but I thought that it had overpromised and would eventually fail to deliver.  Eventually, dynamical systems methods will be required to help reduce and analyze the data.  However, I do want to add that data will play a bigger and bigger role in dynamical systems research.  In the past, we mostly strived to just qualitatively match experiments but now the data has improved to the point that we can try to quantitatively match it.   This will require using statistics.  Readers of this blog will know that I have been an advocate of using Bayesian methods.  I really believe that the marriage of dynamical systems and statistics will have great impact. Statistics is about fitting models to data but the models used are rather simple and generally not  motivated by mechanisms.  Our field is about producing models based on the underlying mechanisms.  It will be a perfect fit.

Snowbird meeting

I’m currently at the biannual SIAM Dynamical Systems Meeting in Snowbird Utah.  If a massive avalanche were to roll down the mountain and bury the hotel at the bottom, much of applied dynamical systems research in the world would cease to exist.  The meeting has been growing steadily for the past thirty years and has now maxed out the capacity of Snowbird.   The meeting will either eventually have to move to a new venue or restrict the number of speakers. My inclination is to move but I don’t think that is the most popular sentiment.  Thus far, I have found the invited talks to be very interesting.  Climate change seems to be the big theme this year.  Chris Jones and Raymond Pierrehumbert  both gave talks on that topic.  I chaired the session by noted endocrinologist and neuroscientist Stafford Lightman who gave a very well received talk on the dynamics of hormone secretion. Chiara Daraio gave a very impressive talk on manipulating sound propagation with chains of ball bearings.  She’s basically creating the equivalent of nonlinear optics and electronics in acoustics.  My talk this afternoon is on finite size effects in spiking neural networks.  It is similar but not identical to the one I gave in New Orleans in January (see here).  The slides are here.

The Philosopher’s Zone

Last Thursday I had to drive from Baltimore to State College, PA for the 16th Congress of the US National Congress on Theoretical and Applied Mechanics to give a talk in one of the sessions.  I gave a condensed version of the kinetic theory of coupled oscillators talk I gave in Warwick last month.  The theme of the session was on recent advances in nonlinear dynamics so the topics were quite diverse.  I’m not sure my talk resonated with the audience.  The only question I received was how was this related to the NIH!

During the six hours of driving I did going back and forth, I listened to podcasts of the Australian radio show The Philosopher’s Zone.  This is a wonderful program hosted by Alan Saunders, who has a PhD in philosophy and is also a food expert.  Every show consists of Saunders talking to a guest, who is usually a philosopher but not always, about either a book she has recently written or some other philosophical topic.  The topics can range from the philosophy of Buffy the Vampire Slayer to Stoicism and everything in between.  Saunders has a knack for making complex philosophical ideas accessible and interesting.  In addition to The Philosopher’s Zone, I still regularly listen to Quirks and Quarks, The Science Show, Radio Lab, and The Naked Scientists.  I’ll also sneak in All in the Mind from time to time.