Irish crisis

Michael Lewis has a must-read article on the Irish economic crisis in Vanity Fair this month. The Irish situation is much, much worse then the United States.  The country is in debt to the tune of a hundred billion Euros for a population that is one hundredth that of the US.  This would be equivalent to the US being ten trillion dollars in debt, which is three times the US budget.  Ireland had lagged behind the rest of Europe economically for most of its history and then astonishingly became one of the richest countries in the world right before the crash.  It is now back to being a developing nation.  The crisis was a result of an out of control real estate bubble fueled by completely irresponsible lending.  Right after Lehman Brothers went under in September of 2008, the Irish banks came under extreme stress.  Then in possibly one of the stupidest acts in modern history, the Irish government decided to guarantee all of the banks losses.  This included both depositors and bond holders, the latter which included foreign countries and Goldman-Sachs.  I think this is a sad example of how a decision based on either incomplete or fraudulent information can lead to such dire consequences.  One bit changed the life of an entire nation.  Here is an excerpt

Vanity Fair:  Back in September 2008, however, there was evidence that he wasn’t. On September 17 the financial markets were in turmoil. Lehman Brothers had failed two days earlier, shares of Irish banks were plummeting, and big corporations were withdrawing their deposits from them. Late that evening Lenihan phoned David McWilliams, a former senior European economist with UBS in Zurich and London, who had moved back home to Dublin and turned himself into a writer and media personality. McWilliams had been loudly skeptical about the Irish real-estate boom. Two weeks earlier he had appeared on a radio show with Lenihan, and Lenihan appeared to him entirely untroubled by the turmoil in the financial markets. Now he wanted to drive out to McWilliams’s house and ask his advice on what to do about the Irish banks.

One bit can change your life

Each second you’re taking in perhaps a million bits of information.  See here for the estimate.  Most of those bits don’t affect your brain or your life at all.  The information you take in just walking down the street is immense yet you probably ignore most of it.  However, a single bit of information can change your entire life.  A blood test that indicates that your LDL concentration is too high can cause you to change what you eat and how much you exercise.  A single yes or no answer can change your mood for a day or the rest of your life.  You could imagine how such sensitivity would arise if the bit were anticipated as when you wait for an answer.   The really interesting problem is how we are able to react to unanticipated bits of great importance such as when you feel your chair suddenly shaking.  Our brains are shaped to find those needles in the haystack.

Moore’s law and science

It’s been almost twenty years since I finished graduate school. According to Moore’s law, (which I’ll take to be a doubling every two years),  computer power should have increased by a factor of a thousand  over that time.  I remember in the early-to-mid nineties when I had a 95 MHz Pentium processor.  I now  have a two year old Mac with 8 processors running at 3 GHz. Hence, depending on how you count, Moore’s law seems to have held over the last two decades.  My question then is what progress in science has resulted because of this vast increase in computer power.   Even though we all can carry in our briefcase  the equivalent of a Cray supercomputer from twenty years ago, it is not at all obvious to me what developments have been achieved as a result. This increase in power can be made particularly concrete if we consider that chess programs running on laptops today (e.g. here) can beat the IBM computer Deep Blue that defeated Gary Kasporov in 1997.

One thing that comes to my mind is that the human genome project may not have been completed as quickly without fast computing power to piece together all the overlapping short reads required in the shotgun sequencing method.  However, I think it would have only delayed the project by months if we were stuck with 1990 computing power.  Perhaps, weather modeling has improved greatly.  I don’t hear people complaining as much about forecasts these days.  Computing power will probably play a major role if we start to fully sequence large numbers of people.  In computational neuroscience, it has allowed projects such as Blue Brain to proceed although it is not clear what will be achieved by it.  Otherwise, I can’t really think of a major breakthrough that occurred just because of an increase in computing power.  We haven’t cured cancer.  We still can’t predict earthquakes.  We still don’t understand the brain.

In my own work, I’ve only fully exploited Moore’s law twice.  The first was in doing large-scale simulations of the Kuramoto-Sivashinsky equation (see here and here) and the second is right now in a GWAS data analysis project where I need to manipulate very large matrices. It is true that we can now do Bayesian inference and model comparison on larger models.  However, the curse of dimensionality strongly works against us here.  If you wanted to sample a parameter at just ten values (which is extremely conservative), then you would add a factor of ten in computer time for each new parameter.  With a current desktop computer, I feel confident that we can test a model of say fifteen parameters with poor prior information about where the parameters should be.  Going to twenty four parameters would be an increase of or a billion and going to a hundred parameters would require Moore’s law to hold for another 500 or so years.  This is why I’m skeptical about realistic modeling.  It can only work if we have a very good idea of what the parameter values should be and getting all that data does not follow directly from Moore’s law.   In many ways, the main impact of faster computers is to allow me to be lazier.  Instead of working hard to optimize programs written in C or C++, I can now cobble together hastily written code in Matlab.  Instead of thinking more clearly about what a model should do, I can just run multiple simulations and try it out.  Hence, I think using computing power wisely is a great arbitrage opportunity.

Watson and Jeopardy

Any doubts that computers can do natural language processing ended dramatically yesterday as IBM’s Watson computer defeated the world’s two best players in the TV quiz show Jeopardy.  Although the task was constrained, it clearly shows that it won’t be too long before we’ll have computers that can understand most of what we say.  This Nova episode gives a nice summary of the project.  A description of the strategy and algorithms used by the program’s designers can be found here.

I think there are two lessons to be learned from Watson.  The first is that machine learning will lead  the way towards strong AI. (Sorry Robin Hanson, it won’t be brain emulation). Although they incorporated “hard coded” algorithms, the engine behind Watson was supervised learning from examples.  The second lesson is that we may already have all the algorithms to get there.  The Watson team didn’t have to invent any dramatically new algorithms.  What was novel is the way they integrated many existing algorithms.   This  is analogous to what I called the Hopfield Hypothesis in that we may already know enough biology to understand how the brain works.  What we don’t understand yet is how these elements combine.

 

Addendum:  Here is a YouTube link for the show last night.

Is the singularity really singular?

Time magazine has an article this month that summarizes the ideas of Ray Kurzweil and the Singularity, which he defines as the point in time  when machine intelligence surpasses that of humans.  I’ve posted on this topic  here and here.  What strikes me in this article and others  is the lack of precision in the arguments used for and against the occurrence of the Singularity.  Here are four examples:

1) Exponentials are not singular. Kurzweil and his colleagues argue that technology grows exponentially so by a generalized Moore’s law, computers and algorithms should improve enough  in forty years to achieve the Singularity.  The irony of this statement is that an exponential function is mathematically nonsingular.  In fact, it is the epitome of a well-defined (entire) function.    If they wanted a real singularity, they could have chosen a rational function like where there is a singularity at .  What they should say is that there will be a kink in the growth rate when machines exceed humans because we will then go into hyper-Moore’s law growth, which is the definition that Robin Hanson uses.  The singularity would then be in the derivative of the growth function.

Distribution cheat sheet

I can never remember the form of distributions so here is a cheat sheet for the density functions of some commonly used ones.

Binomial:

Multinomial:

Negative binomial:

Poisson:

Normal:

Multivariate normal:

Gamma:

Inverse gamma:

Beta:

Student t:

Cauchy:

Chi-square:

Scientific arbitrage

In many of my research projects, I spend a nontrivial amount of my time wondering if I am reinventing the wheel.   I try  to make sure that what I’m trying to do hasn’t already been done but this is not always simple because  a solution from another field may be hidden in another form using unfamiliar jargon and concepts.  Hence, I think that there is a huge opportunity out there for scientific arbitrage, where people can look for open problems that can be easily solved by adapting solutions from other areas.  One could argue that my own research program is a form of arbitrage since I use methods of applied mathematics and theoretical physics to tackle problems in biology. However, generally in my work, the problem comes first and then I look for the best tool to use rather than specifically work on problems that are open to arbitrage.

I’m certain that some fields will be more amenable to arbitrage than others. My guess is that fields that are very vertical like pure mathematics and theoretical physics will be less susceptible  because many people have thought about the same problem and have tried all of the available techniques. Breakthroughs in these fields will generally require novel ideas that build upon previous ones, such as in the recent proofs of the Poincare Conjecture and the Kepler sphere packing problem.   Using economics language, these fields are efficient. Ironically, economics itself may be a field  that is not as efficient and be open to arbitrage since many of the standard models, such as for supply and demand, are based on reaching an equilibrium.  It seems like a dose of dynamical systems may be in order.

 

ESP, Bayes and being wrong

The most recent dust-up over statistical significance and being wrong regards the publication of a paper on ESP in a top psychology journal.  Here’s a link to a paper criticizing the result using Bayesian analysis.  Below is an excerpt from the journal Science.

Science: The decision by a top psychology journal to publish a paper on extrasensory perception (ESP) has sparked a lively discussion on blogs and in the mainstream media. The paper’s author, Daryl Bem, a respected social psychologist and professor emeritus at Cornell University, argues that the results of nine experiments he conducted with more than 1000 college students provide statistically significant evidence of an ability to predict future events. Not surprisingly, word that the paper will appear in an upcoming issue of the Journal of Personality and Social Psychology (JPSP) has provoked outrage from pseudoscience debunkers and counteraccusations of closed-mindedness from those willing to consider the possibility of psychic powers.

It has also rekindled a long-running debate about whether the statistical tools commonly used in psychology—and most other areas of science—too often lead researchers astray. “The real lesson to be learned from this is not that ESP exists, it’s that the methods we’re using aren’t protecting us against spurious results,” says David Krantz, a statistician at Columbia University…

…But statisticians have long argued that there are problems lurking in the weeds when it comes to standard statistical methods like the t test. What scientists generally want to know is the probability that a given hypothesis is true given the data they’ve observed. But that’s not what a p-value tells them, says Adrian Raftery, a statistician at the University of Washington, Seattle. In Bem’s erotic photo experiment, the p-value of less than .01 means, by definition, that there’s less than a 1% chance he would have observed these data—or data pointing to an even stronger ESP effect—if ESP does not exist. “Some people would turn that around and say there’s a 99% chance there’s something going on, but that’s wrong,” Raftery says.

Not only does this type of thinking reflect a misunderstanding of what a p-value is, but it also overestimates the probability that an effect is real, Raftery and other statisticians say. Work by Raftery, for example, suggests that p-values in the .001 to .01 range reflect a true effect only 86% to 92% of the time. The problem is more acute for larger samples, which can give rise to a small p-value even when the effect is negligible for practical purposes, Raftery says.

He and others champion a different approach based on so-called Bayesian statistics. Based on a theory developed by Thomas Bayes, an 18th century English minister, these methods are designed to determine the probability that a hypothesis is true given the data a researcher has observed. It’s a more intuitive approach that’s conceptually more in line with the goals of scientists, say its advocates. Also, unlike the standard approach, which assumes that each new experiment takes place in a vacuum, Bayesian statistics takes prior knowledge into consideration.