Bayesian model comparison Part 2

In a previous post, I summarized the Bayesian approach to model comparison, which requires the calculation of the Bayes factor between two models. Here I will show one computational approach that I use called thermodynamic integration borrowed from molecular dynamics. Recall, that we need to compute the model likelihood function

    (1)

for each model where is just the parameter dependent likelihood function we used to find the posterior probabilities for the parameters of the model.

The integration over the parameters can be accomplished using the Markov Chain Monte Carlo, which I summarized previously here. We will start by defining the partition function

   (2)

where is an inverse temperature. The derivative of the log of the partition function gives

   (3)

which is equal to the ensemble average of . However, if we assume that the MCMC has reached stationarity then we can replace the ensemble average with a time average .  Integrating (3) over from 0 to 1 gives

From (1) and (2), we see that  , which is what we want to compute  and .

Hence, to perform Bayesian model comparison, we simply run the MCMC for each model at different temperatures (i.e. use as the likelihood in the standard MCMC) and then integrate the log likelihoods over at the end. For a Gaussian likelihood function, changing temperature is equivalent to changing the data “error”. The higher the temperature the larger the presumed error. In practice, I usually run at seven to ten different values of and use a simple trapezoidal rule to integrate over .  I can even do parameter inference and model comparison in the same MCMC run.

Erratum, 2013-5-2013,  I just fixed an error in the final formula

Discounting the obvious

The main events in the history of science have involved new ideas overthrowing conventional wisdom. The notion that the earth was the center of the universe was upended by Copernicus. Species were thought to be permanent and fixed until Darwin. Physics was thought to be completely understood at the end of the nineteenth century and then came relativity theory and quantum mechanics to mess everything up. Godel overthrew the notion that mathematics was infallible. This story has been repeated so many times that people now seem to instinctively look for the counterintuitive answer to every problem. There are countless books on thinking outside of the box.  However, I think that the supplanting of “linear” thinking with “nonlinear” thinking is not always a good idea and sometimes it can have dire consequences.

A salient example is the current idea that fiscal austerity will lead to greater economic growth. GDP is defined as the sum of  consumption, investment, government spending and exports minus imports. If consumption or investment were to decline in an economic contraction, as in the Great Recession, then the simple linear idea would be that GDP and growth can be bolstered by increased government spending. This was the standard government response immediately after the financial crisis of 2008. However, starting in about 2010 when the recovery wasn’t deemed fast enough instead of considering the simple idea that the stimulus wasn’t big enough, the idea that policy makers, especially in Europe, adopted was that government spending was crowding out private spending so that a decrease in government spending would lead to a net increase in GDP and growth. This is very nonlinear thinking because it requires a decrease in GDP to induce an increase in GDP. Thus far this idea is not working and austerity has led to lower GDP growth in all countries that have tried it.  This idea was reinforced by a famous, now infamous, paper by Reinhart and Rogoff, which claimed that when government debt reaches 90% of GDP, growth is severely curtailed. This result has been taken as undisputed truth by governments and the press even though there were many economists who questioned it.  However, it turns out that the paper has major errors (including an Excel coding error). See here for a summary.  This is case where the nonlinear idea (as well as conflating correlation with causation) is probably wrong and has inflicted immense hardship on a large number of people.

 

New paper on fat

Sex-Associated Differences in Free Fatty Acid Flux of Obese Adolescents.

Diane C Adler-Wailes, Vipul Periwal, Asem H Ali, Sheila M Brady, Jennifer R McDuffie, Gabriel I Uwaifo, Marian Tanofsky-Kraff, Christine G Salaita, Van S Hubbard, James C Reynolds, Carson C Chow, Anne E Sumner, Jack A Yanovski

Section on Growth and Obesity (D.C.A.-W., A.H.A., S.J.R.M., G.I.U., M.T.-K., J.A.Y.), Program in Developmental Endocrinology and Genetics, Eunice Kennedy Shriver National Institute of Child Health and Human Development; Mathematical Cell Modeling Section (V.P., C.C.C.), Division of Extramural Activities (C.G.S.), Division of Nutrition Research Coordination (V.S.H.), and Laboratory of Endocrinology and Receptor Biology (A.E.S.), National Institute of Diabetes and Digestive and Kidney Diseases; and Nuclear Medicine Department (J.C.R.), Hatfield Clinical Research Center, National Institutes of Health, U.S. Department of Health and Human Services, Bethesda, Maryland 20892.

The Journal of clinical endocrinology and metabolism (impact factor: 6.5). 02/2013; DOI:10.1210/jc.2012-3817

ABSTRACT Context: In obesity, increases in free fatty acid (FFA) flux can predict development of insulin resistance. Adult women release more FFA relative to resting energy expenditure (REE) and have greater FFA clearance rates than men. In adolescents, it is unknown whether sex differences in FFA flux occur. Objective: Our objective was to determine the associations of sex, REE, and body composition with FFA kinetics in obese adolescents. Participants: Participants were from a convenience sample of 112 non-Hispanic white and black adolescents (31% male; age range, 12-18 years; body mass index SD score range, 1.6-3.1) studied before initiating obesity treatment. Main Outcome Measures: Glucose, insulin, and FFA were measured during insulin-modified frequently sampled iv glucose tolerance tests. Minimal models for glucose and FFA calculated insulin sensitivity index (SI) and FFA kinetics, including maximum (l0 + l2) and insulin-suppressed (l2) lipolysis rates, clearance rate constant (cf), and insulin concentration for 50% lipolysis suppression (ED50). Relationships of FFA measures to sex, REE, fat mass (FM), lean body mass (LBM) and visceral adipose tissue (VAT) were examined. Results: In models accounting for age, race, pubertal status, height, FM, and LBM, we found sex, pubertal status, age, and REE independently contributed to the prediction of l2 and l0 + l2 (P < .05). Sex and REE independently predicted ED50 (P < .05). Sex, FM/VAT, and LBM were independent predictors of cf. Girls had greater l2, l0 + l2 and ED50 (P < .05, adjusted for REE) and greater cf (P < .05, adjusted for FM or VAT) than boys. Conclusion: Independent of the effects of REE and FM, FFA kinetics differ significantly in obese adolescent girls and boys, suggesting greater FFA flux among girls.

Slides for Hopkins talk

I gave the Bodian Seminar at the Zanvyl Krieger Mind/Brain Institute of Johns Hopkins today.  I talked about cortical dynamics in the presence of conflicting stimuli. My slides are here. A summary of part of my talk can be found here.  Other pertinent papers can be found here and here.

Slides for ACP talk

I just gave a talk on obesity at a diabetes course at the American College of Physicians meeting in San Francisco.  My slides are here.

Hepatitis C and the folly of prizes

The scientific world was set slightly aflutter when Michael Houghton turned down the prestigious Gairdner Award for the the discovery of Hepatitis C. Harvey Alter and Daniel Bradley were the two other recipients. Houghton, who had previously received the Lasker Award with Alter, felt he could not accept one more award because two colleagues Qui-Lim Choo and George Kuo did not receive either of these awards, even though their contributions were equally important.

Hepatitis, which literally means inflammation of the liver, was characterized by Hippocrates and known to be infectious since the 8th century. The disease had been postulated to be viral at the beginning of the 20th century and by the 1960′s two viruses termed Hepatitis A and Hepatitis B had been established. However, there still seemed to be another unidentified infectious agent which was termed Non-A Non-B Hepatitis NANBH.

Michael Hougton, George Kuo and Qui-Lim Choo were all working at the Chiron corporation in the early 1980′s.   Houghton started a project to discover the cause of NANBH in 1982 with Choo joining a short time later. They made significant process in generating mouse monoclonal antibodies with some specificity to NANBH infected materials from chimpanzee samples received from Daniel Bradley at the CDC. They used the antibodies to screen cDNA libraries from infected materials but they had not isolated an agent. George Kuo had his own lab at Chiron working on other projects but would interact with Houghton and Choo. Kuo suggested that they try blind cDNA immunoscreening on serum derived from actual NANBH patients. This approach was felt to be too risky but Kuo made a quantitative assessment that showed it was viable. After two years of intensive and heroic screening by the three of them, they identified one clone that was clearly derived from the NANBH genome and not from human or chimp DNA. This was definitive proof that NANBH was a virus, which is now called Hepatitis C. Kuo then developed a prototype of a clinical Hepatitis C antibody detection kit and used it to screen a panel of NANBH blood provided by Harvey Alter of the NIH. Kuo’s test was a resounding success and the blood test that came out of that work has probably saved 300 million or more people from Hepititis C infection.

The question then is who deserves the prizes. Is it Bradley and Alter, who did careful and diligent work obtaining samples or is it Houghton, Choo, and Kuo, who did the heroic experiments that isolated the virus? For completely unknown reasons, the Lasker was awarded to just Houghton and Alter, which primed the pump for more prizes to these two. Now that the Lasker and Gairdner prizes have been cleared, that leaves just the Nobel Prize. The scientific community could get it right this time and award it to Kuo, Choo, and Houghton.

 

Addendum added 2013-5-2:  I should add that many labs from around the world were also trying to isolate the infective agent of NANBH and all failed to identify the correct samples from Alter’s panel.  It is not clear how long it would have been and how many more people would have been infected if Kuo, Choo, and Houghton had not succeeded when they did.

New paper on neural networks

Michael Buice and I have just published a review paper of our work on how to go beyond mean field theory for systems of coupled neurons. The paper can be obtained here. Michael and I actually pursued two lines of thought on how to go beyond mean field theory and we show how the two are related in this review. The first line started in trying to understand how to create a dynamic statistical theory of a high dimensional fully deterministic system. We first applied the method to the Kuramoto system of coupled oscillators but the formalism could apply to any system. Our recent paper in PLoS Computational Biology was an application for a network of synaptically coupled spiking neurons. I’ve written about this work multiple times (e.g. here,  here, and here). In this series of papers, we looked at how you can compute fluctuations around the infinite system size limit, which defines mean field theory for the system, when you have a finite number of neurons. We used the inverse number of neurons as a perturbative expansion parameter but the formalism could be generalized to expand in any small parameter, such as the inverse of a slow time scale.

The second line of thought was with regards to the question of how to generalize the Wilson-Cowan equation, which is a phenomenological population activity equation for a set of neurons, which I summarized here. That paper built upon the work that Michael had started in his PhD thesis with Jack Cowan. The Wilson-Cowan equation is a mean field theory of some system but it does not specify what that system is. Michael considered the variable in the Wilson-Cowan equation to be the rate (stochastic intensity) of a Poisson process and prescribed a microscopic stochastic system, dubbed the spike model, that was consistent with the Wilson-Cowan equation. He then considered deviations away from pure Poisson statistics. The expansion parameter in this case was more obscure. Away from a bifurcation (i.e. critical point) the statistics of firing would be pure Poisson but they would deviate near the critical point, so the small parameter was the inverse distance to criticality. Michael, Jack and I then derived a set of self-consistent set of equations for the mean rate and rate correlations that generalized the Wilson-Cowan equation.

The unifying theme of both approaches is that these systems can be described by either a hierarchy of moment equations or equivalently as a functional or path integral. This all boils down to the fact that any stochastic system is equivalently described by a distribution function or the moments of the distribution. Generally, it is impossible to explicitly calculate or compute these quantities but one can apply perturbation theory to extract meaningful quantities. For a path integral, this involves using Laplace’s method or the method of steepest descents to approximate an integral and in the moment hierarchy method it involves finding ways to truncate or close the system. These methods are also directly related to WKB expansion, but I’ll leave that connection to another post.

Failure at all scales

The premise of most political systems since the enlightenment is that the individual is a rational actor. The classical liberal (now called libertarian) tradition believes that social and economic ills are due to excessive government regulation and intervention. If the individuals are left to participate unfettered in a free market then these problems will disappear.  Conversely, the traditional Marxist/Leninist left posits that the capitalistic system is inherently unfair and can only be cured by replacing it with a centrally planned economy. However, the lesson of the twentieth century is that there is irrationality, incompetence, and corruption at all levels, from individuals to societies. We thus need regulations, laws and a government that take into account of the fact that we are fallible at all scales, including the regulations, laws and the government.

Markets are not perfect and often fail but they are clearly superior to central planning for the distribution of most resources (particularly consumer goods). However, they need to be monitored and regulated. When markets fail, government should intervene. Even the staunchest libertarian would support laws that prevent the elimination of your competitors by violence. Organized crime and drug cartels are an example of how businesses would run in the absence of laws. However, regulations and laws should have built-in sunset clauses that force them to be reviewed after a finite length of time. In some cases, a freer market makes sense. I believe that the government is bad in picking winners so if we want to promote alternative energy, we shouldn’t be helping nascent green industries but rather tax fossil fuel use and let the market decide what is best. Making cars more fuel-efficient may not lead to less energy use but just encourage people to drive more. If we want to save energy, we should make energy more expensive. We should also make regulations as universal and simple as possible to minimize  regulatory capture. I think means testing for social services like medicare is a bad idea because it will just encourage people to find clever ways to circumvent it. The same probably goes for need-based welfare. We should just give everyone a minimum income and let everyone keep any income above it. This would then provide a safety net but not a disincentive to work. Some people will choose to live on this minimum income but as I argued here, I think they should be allowed to. If we want to address wealth inequality then we should probably tax wealth directly rather than income. We want to encourage people to make as much money as possible but then spend it to keep the wealth circulating. By the same reasoning, I don’t like a consumption tax. Our economy is based on consumer spending so we don’t want to discourage that (unless it is for other reasons than economic).

People do not suddenly become selfless and rational when the political system changes but systems can mitigate the effects of their irrational and selfish tendencies. As the work of Kahneman, Tversky, Ariely, and others have shown, rational and scientific thinking does not come naturally to people. Having the market decide what is the most effective medical treatment is not a good idea. A perfect example is in a recent Econtalk podcast with libertarian leaning economist John Cochrane on healthcare. Cochrane suggested that instead of seeing a doctor first, he should just be allowed to buy antibiotics for his children whenever they had an earache. The most laughable part was his idea that we have rules against self-administering of drugs to protect uneducated people. Actually, the rules are to protect highly educated people like him who think that expertise in one area transfers to another. The last thing we want is for even more antibiotic use and more antibiotic resistant bacterial strains. I definitely do not want to live in a society where I have to wait for the market to penalize companies that provide unsafe food or build unsafe buildings. It doesn’t help me if my house collapses in an earthquake because the builder used inferior materials. Sure they may go out of business but I’m already dead.

There is no single perfect system or set of rules that one should always follow. We should design laws, regulations, and governments that are adaptable and adjust according to need. The US Constitution has been amended 27 times. The last time was in 1992, which just changed the rules on salaries for elected officials. The 26th amendment in 1971 made 18 the universal threshold age for voting. We are thus due for another amendment and I think the 2nd amendment, which guarantees the right to bear arms, is a place to start. We could make it more explicit what types of arms are protected and what types can be regulated by local laws. If we want to reduce gun violence then gun regulation makes sense. People will do things they later regret. If one is in the heat of an argument and there is a gun available then it could be used inadvertently. It takes a lot of training and skill to use a gun effectively. Accidents will happen. In the case of guns, failure often leads to death. I would prefer to live in a society where guns are scarce rather than one where everyone carries a weapon like the old wild west.

The monopoly of finance

Given the recent post by Noah Smith on the profitability of finance, I thought I would put up a link to a previous post of mine that asked the same question.

Brain activity map

The biggest news for neuroscientists in President Obama’s State of the Union Address was the announcement of the Brain Activity Map (BAM) project (e.g. see here and here). The goal of this project as outlined in this Neuron paper is to develop the technological capability to measure the spiking activity of every single neuron in the brain simultaneously. I used to fantasize about such a project a decade ago but now I’m more ambivalent. Although the details of the project have not been announced, people involved are hoping for 300 million dollars per year for ten years. I do believe that a lot will be learned in pursuing such a project but it may also divert resources for neuroscience towards this one goal. Given that the project is mostly technological, it may also mostly bring in new engineers and physicists to neuroscience rather than fund current labs. It could be a huge boon for computational neuroscience because the amount of data that will be recorded will be enormous. It will take a lot of effort just to curate this data much less try to analyze and makes sense of it. Finally, on a cautionary note, it could be that much of the data will be superfluous. After all, we understand how gases behave (at least enough to design refrigerators and airplanes, etc.) without measuring the positions and velocities of every molecule in a room. I’m not sure we would have figured out the ideal gas law, the Carnot cycle, or the three laws of thermodynamics if we just relied on an “Air Activity Map Project” a century ago. There is probably a lot of compression going on in the brain. If we knew how this compression worked, we could then just measure the nonredundant information. That would certainly make the BAM project a whole lot easier.

Aboriginals and Canada

This lecture by John Ralston Saul captures the essence of Canada better than anything else I’ve ever heard or read.  Every Canadian should listen and non-Canadians could learn something too!

Minimum wage and the efficient market

The current debate about the effect of raising the minimum wage on employment poses an interesting question about how efficient the labour market is. According to the classical view of economics (and the Freshwater school) raising the minimum wage should decrease the number of people working because it will induce employers to shed workers to minimize costs. However, multiple studies of the effects of minimum wages seem to show that small changes do not decrease employment and sometimes even increases it (e.g. see here). If people were completely rational, equally competent and the market were efficient then raising wages should decrease employment. If it turns out that this does not happen then one or more of these assumptions is false. My take is that they are all suspect.

According to basic economics theory, the wages employers offer is set by the marginal cost of adding an additional worker. This means that they will select a wage such that the productivity gains for the last worker they hire is balanced by the cost of that worker. However, this assumes that there is a readily available pool of workers willing to take that wage and that the productivity of a worker is independent of the wage offered. Neither of these assumptions may be true. The argument of the Freshwater school is that during times of high unemployment there should be lots of people willing to take jobs at any wage. However,  there is likely to be a distribution for the lowest acceptable wage. In fact, Keynesian theory is predicated on the fact that wages and prices are ‘sticky’ so in an economic downturn, neither fall fast enough for the market to clear. A person is probably unwilling to take a lower paying job because it may affect her prospects of securing a higher paying one in the future when the economy rebounds. Hence, employers attempt to set a wage at some point on the lower tail of the acceptable wage distribution so that the cost of waiting to find someone willing to take that wage plus the cost of hiring is balanced by the expected increase in sales. Increasing the minimum wage could then actually increase employment by forcing employers to hire at a wage point where the wait time is much shorter. The savings from the shortened wait time and possibly lower turnover balances the cost of the higher wage.

Employment may also increase if productivity scales with the lowest acceptable wage.  By forcing employers to pay a higher wage, they may actually hire much more productive workers that increases sales enough to hire another worker. In fact, the employers may not even realize that increasing the offered wage would actually increase sales because they never sampled that part of the distribution. They could be stuck in a local minimum where they offer low wages to low productivity workers when they could have offered slightly higher wages to much higher productivity workers. In that scenario, the minimum wage would actually benefit the employers more than the employees.

Both of these situations are plausible indications of a market failure in the labour market. The market may be locally efficient in that the wages are optimal for small changes but not globally efficient in that there could be another more efficient fixed point somewhere far away on the wage curve. Government intervention could actually push the market to a more efficient point and this is not even accounting for the fact that the increased wages would be spent, which could cause a Keynesian multiplier boost to the economy. This is not to suggest that the government knows any better than the employers but just that unintended consequences can go both ways.

Mass

Since the putative discovery of the Higgs boson this past summer, I have heard and read multiple attempts at explaining what exactly this discovery means. They usually go along the lines of “The Higgs mechanism gives mass to particles by acting like molasses in which particles move around …” More sophisticated accounts will then attempt to explain that the Higgs boson is an excitation in the Higgs field. However, most of the explanations I have encountered assume that most people already know what mass actually is and why particles need to be endowed with it. Given that my seventh grade science teacher didn’t really understand what mass was, I have a feeling that most nonphysicists don’t really have a full appreciation of mass.

To start out, there are actually two kinds of mass. There is inertial mass, which is the resistance to acceleration and is mass that goes into Newton’s second law of   and then there is gravitational mass which is like the “charge” of gravity. The more gravitational mass you have the stronger the gravitational force. Although they didn’t need to be, these two masses happen to be the same.  The equivalence of inertial and gravitational mass is one of the deepest facts of the universe and is the reason that all objects fall at the same rate. Galileo’s apocryphal Leaning Tower of Pisa experiment was a proof that the two masses are the same. You can see this by noting that the gravitational force is given by

Epipheo video

The narration comes from an interview with me.

 

New paper on finite size effects in spiking neural networks

Michael Buice and I have finally published our paper entitled “Dynamic finite size effects in spiking neural networks” in PLoS Computational Biology (link here). Finishing this paper seemed like a Sisyphean ordeal and it is only the first of a series of papers that we hope to eventually publish. This paper outlines a systematic perturbative formalism to compute fluctuations and correlations in a coupled network of a finite but large number of spiking neurons. The formalism borrows heavily from the kinetic theory of plasmas and statistical field theory and is similar to what we used in our previous work on the Kuramoto model (see here and  here) and the “Spike model” (see here).  Our heuristic paper on path integral methods is  here.  Some recent talks and summaries can be found here and here.

Creating vs treating a brain

The NAND (Not AND) gate is all you need to build a universal computer. In other words, any computation that can be done by your desktop computer, can be accomplished by some combination of NAND gates. If you believe the brain is computable (i.e. can be simulated by a computer) then in principle, this is all you need to construct a brain. There are multiple ways to build a NAND gate out of neuro-wetware. A simple example takes just two neurons. A single neuron can act as an AND gate by having a spiking threshold high enough such that two simultaneous synaptic events are required for it to fire. This neuron then inhibits the second neuron that is always active except when the first neuron receives two simultaneous inputs and fires. A network of these NAND circuits can do any computation a brain can do.  In this sense, we already have all the elementary components necessary to construct a brain. What we do not know is how to put these circuits together. We do not know how to do this by hand nor with a learning rule so that a network of neurons could wire itself. However, it could be that the currently known neural plasticity mechanisms like spike-timing dependent plasticity are sufficient to create a functioning brain. Such a brain may be very different from our brains but it would be a brain nonetheless.

The fact that there are an infinite number of ways to creating a NAND gate out of neuro-wetware implies that there are an infinite number of ways of creating a brain. You could take two neural networks with the same set of neurons and learning rules, expose them to the same set of stimuli and end up with completely different brains. They could have the same capabilities but be wired differently. The brain could be highly sensitive to initial conditions and noise so any minor perturbation would lead to an exponential divergence in outcomes. There might be some regularities (like scaling laws) in the connections that could be deduced but the exact connections would be different. If this were true then the connections would be everything and nothing. They would be so intricately correlated that only if taken together would they make sense. Knowing some of the connections would be useless. The real brain is probably not this extreme since we can sustain severe injuries to the brain and still function. However, the total number of hard-wired conserved connections cannot exceed the number of bits in the genome. The other connections (which is almost all of them) are either learned or are random. We do not know which is which.

To clarify my position on the Hopfield Hypothesis, I think we may already know enough to create a brain but we do not know enough to understand our brain. This distinction is crucial.  What my lab has been interested in lately is to understand and discover new treatments for cognitive disorders like Autism (e.g. see here). This implies that we need to know how perturbations at the cellular and molecular levels affect the behavioural level.  This is an obviously daunting task. Our hypothesis is that the bridge between these two extremes is the canonical cortical circuit consisting of recurrent excitation and lateral inhibition. We and others have shown that such a simple circuit can explain the neural firing dynamics in diverse tasks such as working memory and binocular rivalry (e.g. see here). The hope is that we can connect the genetic and molecular perturbations to the circuit dynamics and then connect the circuit dynamics to behavior. In this sense, we can circumvent the really hard problem of how the canonical circuits are connected to each other. This may not lead to a complete understanding of the brain or the ability to treat all disorders but it may give insights into how genes and medication act on cognitive function.

A meal for a day

The Center for Science in the Public Interest has some examples of meals in restaurants that contain the caloric requirements for a whole day.  And you doubted the push hypothesis for the obesity epidemic.

The spawn of SPAUN

SPAUN (Semantic Pointer Architecture Unified Network) is a model of a functioning brain out of Chris Eliasmith’s group at the University of Waterloo. I first met Chris almost 15 years ago when I visited Charlie Anderson at Washington University, where Chris was a graduate student. He was actually in the philosophy department (and still is) with a decidedly mathematical inclination. SPAUN is described in Chris’s paper in Science (obtain here) and in a forthcoming book. SPAUN can perform 8 fairly diverse and challenging cognitive tasks using 2.5 million neurons with an architecture inspired by the brain. It takes input through visual images and responds by “drawing” with a simulated arm. It decodes images, extracts features and compresses them, stores them in memory, computes with them, and then translates the output into a motor action.  It can count, copy, memorize, and do a Raven’s Progressive Matrices task. While it can’t learn novel tasks, it is pretty impressive.

However, what is most impressive to me about SPAUN is not how well it works but that it mostly implements known concepts from neuroscience and machine learning. The main newness was putting it all together. This harkens back to what I called the Hopfield Hypothesis, which is that we already know all the elementary pieces for neural functioning. What we don’t know is how they fit and work together. I think one of the problems in computational neuroscience is that we’re too timid. I first realized this many years ago when I saw a talk by roboticist Rodney Brooks. He showed us robots with very impressive capabilities (this was when he was still at MIT)  that were just implementing well-known machine learning rules like back-propagation. I recall thinking that robotics was way ahead of us and that reverse engineering may be harder than engineering. I also think that we will likely construct a fully functioning brain before we understand it. It could be that if you connect enough neurons together that incorporate a set of  necessary mechanisms and then expose it to the world, it would start to develop and learn cognitive capabilities. However, it would be as difficult to reverse engineer exactly what this constructed brain was doing as it is to reverse engineer a real brain. It may also be computationally undecidable or intractable to a priori determine the essential set of necessary mechanisms or the number of neurons you need. You  might just have to cobble something together and try it out. A saving grace may be that these elements may not be unique. There could be a large family of mechanisms that you could draw from to create a thinking brain.

 

 

Saving large animals

One  story in the news lately is the dramatic increase in the poaching of African elephants (e.g. New York Times). Elephant numbers have plunged dramatically in the past few years and their outlook is not good. This is basically true of most large animals like whales, pandas, rhinos, bluefin tuna, whooping cranes, manatees, sturgeon, etc. However, one large animal has done extremely well while the others have languished. In the US it had a population of zero 500 years ago and now it’s probably around 100 million.That animal as you have probably guessed is the cow. While wild animals are being hunted to extinction or dying due to habitat loss and climate change, domestic animals are thriving. We have no shortage of cows, pigs, horses, dogs, and cats.

Given that current conservation efforts are struggling to save the animals we love, we may need to try a new strategy. A complete ban on ivory has not stopped the ivory trade just as a ban on illicit drugs has not stopped drug use. Prohibition does not seem to be a sure way to curb demand. It may just be that starting some type of elephant farming may be the only way to save the elephants. It could raise revenue to help protect wild elephants and could drop the price in ivory sufficiently to make poaching less profitable. It could also backfire and increase the demand for ivory.

Another counter intuitive strategy may be to sanction limited hunting of some animals. The introduction of wolves into Yellowstone park has been a resounding ecological success but it has also angered some people like ranchers and deer hunters. The backlash against the wolf has already begun. One ironic way to save wolves could be to legalize the hunting of them. This would give hunters an incentive to save and conserve wolves. Given that the set of hunters and ranchers often have a significant intersection, this could dampen the backlash. There is a big difference in attitudes towards conservation when people hunt to live versus hunting for sport. When it’s a job, we tend to hunt to extinction like  buffalo, cod, elephants, and bluefin tuna. However, when it’s for sport, people want to ensure the species thrives. While I realize that this is controversial and many people have a great disdain for hunting, I would suggest that hunting is no less humane and perhaps more than factory abattoirs.

Two New Papers

I have two new papers in the Journal of Biological Chemistry:

Z Zhang, Y Sun, YW Cho, CC Chow, SS Simons. PA1 Protein, a New Competitive Decelerator Acting at More than One Step to Impede Glucocorticoid Receptor-mediated Transactivation. J Biol Chem:42-58 (2012). [PDF]

JA Blackford, C Guo, R Zhu, EJ Dourgherty, CC Chow and SS Simons. Identification of Location and Kinetically Defined Mechanism of Cofactors and Reporter Genes in the Cascade of Steroid-regulated Transactivation. J Biol Chem:40982-95 (2012). [PDF]

Both are applications of our theory for steroid-mediated gene induction.  The theory is applicable for any biochemical system where the dose response curve strictly follows a Michaelis-Menten curve.  A summary of the theory can be found here and here.  Slides for talks on the topic can be found here.  In Zhang et al, we use the theory to predict a mechanism for a protein called PA1.  In Blackford et al, we show that DNA is the effective rate limiting step for gene transcription in steady state, which we dub concentration limiting step, since there are really no rates at steady state.