Nonlinearity

I think people generally view nonlinear effects in one of two ways.  They either 1) do not think of it at all and basically view everything through a linear lens or 2) they think of it in terms of a lack of predictability as in chaos and the butterfly effect. I think both are somewhat dangerous viewpoints.  Given that the twenty first century is shaping up to be one where complex systems, such as the economy and climate, directly influence our lives, it is important that the general public and especially scientists have a more precise understanding of what nonlinearity can and cannot do.  Although Stan Ulam once remarked something to the effect that the term “nonlinear science” was about as meaningful as calling the bulk of zoology, the “study of non-elephant animals”, I actually think there are some concrete notions to the term that can give some valuable insight. In particular, nonlineariy does not only imply a lack of predictability, in some cases it can make things more predictable.

The first thing is to note that for most applications, there are basically two important effects of nonlinearity, namely “threshold” and “saturation”.  In fact, saturation can be thought of as negative feedback with a threshold, so threshold is really the only effect to keep in mind although separating the two is useful conceptually sometimes. By threshold, I mean that some variable does nothing until it crosses a threshold and by saturation, I mean that the effect of some variable does not change much beyond some point.  Here, I’ll give some examples of how saturation and thresholds can go a long way in understanding complex phenomena.

So for example, let’s consider a chaotic system.  There are some technical mathematical definitions for chaos, but for our purposes let’s simply say that any system with extreme sensitivity to initial conditions is chaotic.  We can explain how chaos could arise rather simply by using the concept of threshold.  Consider any system (with three or more dimensions) where there is some approximate periodicity, by that I only mean that there is a means to send a variable back after it runs away.  Remember that oscillatory behavior does not require nonlinearity, you just need two quantities that interact with some delay.  Now, let’s suppose there is a threshold somewhere along an orbit that sends you off in two different directions depending on what side of the threshold you are on.  So, during an oscillation two orbits can be arbitrarily close together but if they arrive on different sides of the threshold they will be separated.  That in a nutshell is chaos.  Although it is hard to prove a system is chaotic, it is not hard to understand conceptually how it can arise.

The opposite of chaos is integrability.  Technically, an integrable system is a Hamiltonian system that can be reduced to action-angle variables (i.e. motion is confined to invariant tori).  What that implies is that a nonlinear system can be very coherent and predictable.  A nice example of an integrable system is a soliton, which is a solution of a nonlinear wave equation that can propagate without changing shape and even pass through each other unperturbed.  (My PhD thesis was actually about solitons and chaos).  Solitons can actually be generated quite easily in a long and narrow water tank and have applications in fiber optics communication.  The big red spot on Jupiter is thought to be a coherent nonlinear structure although probably not a soliton per se.  Integrability is quite deep mathematically so it’s hard to give a simple picture in terms of thresholds and saturation.  It’s more of a balancing act between nonlinear and linear effects.  In the nonlinear three wave system that I studied in my thesis, solitons arose from a nonlinear saturation of a linear instability.  However, in the Korteweg-de Vries equation, solitons arise from a balance between linear dispersion and nonlinear wave steepening.  However, even though precise integrability requires a fine tuning of parameters, the effects of integrability are more widespread because by the KAM theorem, some invariant tori will survive even when you perturb away from integrability.

Another example is computation. Any Boolean logical computation or Boolean truth table (and hence computer) can be constructed out of a combination of NAND (not AND) gates.  A logic gate takes two binary inputs and produces a binary output.  For the NAND gate, two one inputs produces a zero output, and the other three combinations of inputs produces a one output.  So basically if you had a function with a threshold between 1 and 2, then they can be combined to construct a NAND gate.   No combination of linear functions can ever produce a NAND gate and in fact Minsky and Papert showed that even a two layer perceptron  cannot produce NAND logic, so you need more than one threshold function.

Now, let’s take an example that is closer to home, like nutritional supplements, which is a multi-billion dollar unregulated industry.  I think if you wanted to make money in this economy, come up with some new nutritional supplement and a great marketing campaign.  There are many supplements that have no discernible effects and no plausible mechanism.  In these cases, people just use them because they are duped and understanding nonlinearity probably won’t help here.   However, there are supplements that are motivated by the fact that a deficiency in them can lead to bad effects.  The rationale is then that if a shortage is bad then a surplus must be good.  Obviously this may not be true if there are thresholds and saturation.  It could be that if you are below some threshold then it is bad but after you’ve crossed it, it is saturated so more is not better.  The dangerous situation is that more of it could actually be bad by crossing another threshold.  I suggest that the dose dependency of drugs and supplements is something that is not universally understood. I could go on and on about how nonlinear effects can be observed and somewhat understood.