New paper on neural networks

Michael Buice and I have a new paper in Frontiers in Computational Neuroscience as well as on the arXiv (the arXiv version has fewer typos at this point). This paper partially completes the series of papers Michael and I have written about developing generalized activity equations that include the effects of correlations for spiking neural networks. It combines two separate formalisms we have pursued over the past several years. The first was a way to compute finite size effects in a network of coupled deterministic oscillators (e.g. see here, here, here and here).  The second was to derive a set of generalized Wilson-Cowan equations that includes correlation dynamics (e.g. see here, here, and here ). Although both formalisms utilize path integrals, they are actually conceptually quite different. The first formalism adapted kinetic theory of plasmas to coupled dynamical systems. The second used ideas from field theory (i.e. a two-particle irreducible effective action) to compute self-consistent moment hierarchies for a stochastic system. This paper merges the two ideas to generate generalized activity equations for a set of deterministic spiking neurons.

Paper on compressed sensing and genomics

New paper on the arXiv. The next step after the completion of the Human Genome Project, was the search for genes associated with diseases such as autism or diabetes. However, after spending hundreds of millions of dollars, we find that there are very few common variants of genes with large effects. This doesn’t mean that there aren’t genes with large effect. The growth hormone gene definitely has a large effect on height. It just means that variations of genes that are common among people have small effects on the phenotype. Given the results of Fisher, Wright, Haldane and colleagues, this was probably expected as the most likely scenario and recent results measuring narrow-sense heritability directly from genetic markers (e.g. see this) confirms this view.

Current GWAS microarrays consider about a million or two markers and this is increasing rapidly. Narrow-sense heritability refers to the additive or linear genetic variance, which means the phenotype is given by the linear model , where is the phenotype vector, is the genotype matrix, are all the genetic effects we want to recover, and are all the nonadditive components including environmental effects. This is a classic linear regression problem. The problem comes when the number of coefficients far exceeds the number of people in your sample, which is the case for genomics. Compressed sensing is a field of high dimensional statistics that addresses this specific problem. People such as David Donoho, Emmanuel Candes and Terence Tao have proven under fairly general conditions that if the number of nonzero coefficients are sparse compared to the number samples, then the effects can be completely recovered using L1 penalized optimization algorithms such as the lasso or approximate message passing. In this paper, we show that these ideas can be applied to genomics.

Here is Steve Hsu’s summary of the paper

Application of compressed sensing to genome wide association studies and genomic selection

Shashaank Vattikuti, James J. Lee, Stephen D. H. Hsu, Carson C. Chow

(Submitted on 8 Oct 2013)

We show that the signal-processing paradigm known as compressed sensing (CS) is applicable to genome-wide association studies (GWAS) and genomic selection (GS). The aim of GWAS is to isolate trait-associated loci, whereas GS attempts to predict the phenotypic values of new individuals on the basis of training data. CS addresses a problem common to both endeavors, namely that the number of genotyped markers often greatly exceeds the sample size. We show using CS methods and theory that all loci of nonzero effect can be identified (selected) using an efficient algorithm, provided that they are sufficiently few in number (sparse) relative to sample size. For heritability h2 = 1, there is a sharp phase transition to complete selection as the sample size is increased. For heritability values less than one, complete selection can still occur although the transition is smoothed. The transition boundary is only weakly dependent on the total number of genotyped markers. The crossing of a transition boundary provides an objective means to determine when true effects are being recovered. For h2 = 0.5, we find that a sample size that is thirty times the number of nonzero loci is sufficient for good recovery.

Comments:	Main paper (27 pages, 4 figures) and Supplement (5 figures) combined
Subjects:	Genomics (q-bio.GN); Applications (stat.AP)
Cite as:	arXiv:1310.2264 [q-bio.GN]
	(or arXiv:1310.2264v1 [q-bio.GN] for this version)

Happiness and divisive inhibition

The Wait But Why blog has an amusing post on why Generation Y yuppies (GYPSYS) are unhappy, which I found through the blog of Michigan economist  Miles Kimball. In short, it is because their expectations exceed reality and they are entitled. What caught my eye was that they defined happiness as “Reality-Expectations”. The key point being that this is a subtractive expression. My college friend Peter Lee, now Professor and Director of the University Manchester X-Ray imaging facility, used to define happiness as “desires fulfilled beyond expectations”. I always interpreted this as a divisive quantity, meaning “Reality/Expectations”.

Now, the definition does have implications if we actually try to use it as a model for how happiness would change with some quantity like money. For example, consider the model where reality and expectations are both proportional to money. Then happiness = a*money – b*money. As long as b is less than a, then money always buys happiness, but if a is less than b then more money brings more unhappiness. However, if we consider the divisive model of happiness then happiness = a*money/ b*money = a/b and happiness doesn’t depend on money at all.

However, the main reason I bring this up is because it is analogous to the two possible ways to model inhibition (or adaptation) in neuroscience. The neurons in the brain generally interact with each other through two types of synapses – excitatory and inhibitory. Excitatory synapses generally depolarize a neuron and make its potential get closer to threshold whereas inhibitory neurons hyperpolarize the neuron and make it farther from threshold (although there are ways this can be violated). For neurons receiving stationary asynchronous inputs, we can consider the firing rate to be some function of the excitatory E and inhibitory I inputs. In subtractive inhibition, the firing rate would have the abstract form f(E-I) whereas for divisive inhibition it would have the form f(E)/(I+C), where f is some thresholded gain function (i.e. zero below threshold, positive above threshold) and C is a constant to prevent the firing rate from reaching infinity. There are some critical differences between subtractive and divisive inhibition. Divisive inhibition works by reducing the gain of the neuron, i.e. it makes the slope of the gain function shallower while subtractive inhibition makes the threshold effectively higher. These properties have great computational significance, which I will get into in a future post.

The cost of the shutdown and sequester

People may be wondering how the US government shutdown is affecting the NIH. I can’t speak for the rest of the institutes but I was instructed to not come to work and to not use my NIH email account or NIH resources. Two new fellows, who were supposed to begin on Oct 1, now have to wait and they will not be compensated for the missed time even if Congress does decides to give back pay to the furloughed employees. I really was hoping for them to start in August or September but that was pushed back because of the Sequester (have people forgotten about that?), which cut our budgets severely. In fact, because of the Sequester, I wasn’t able to hire one fellow because the salary requirements for their seniority exceeded my budget. We were just starting to get some really interesting psychophysics results on ambiguous stimuli but that had to be put on hold because we couldn’t immediately replace fellow Phyllis Thangaraj, who was running the experiments and left this summer to start her MD/PhD degree at Columbia. Now it will be delayed even further. I have several papers in the revision process that have also been delayed by the shutdown. All travel has been cancelled and I heard that people at conferences were ordered to return immediately, including those who were on planes on Oct 1. My quadrennial external review this week has now been postponed. All the flights for the committee and ad hoc members have to be cancelled and we now have to find another date where 20 or more people can agree on. All NIH seminars and the yearly NIH research festival has been cancelled. I was supposed to review an external NIH research proposal this week and that has been postponed indefinitely along with all other submitted proposals awaiting review. Academic labs, students and postdocs depending on their NIH grants this fiscal year will be without funding until the government is reopened. Personally, I will probably come out of this reasonably intact. However, I do worry how this will affect young people, who are the future.

Heritability and additive genetic variance

Most people have an intuitive notion of heritability being the genetic component of why close relatives tend to resemble each other more than strangers. More technically, heritability is the fraction of the variance of a trait within a population that is due to genetic factors. This is the pedagogical post on heritability that I promised in a previous post on estimating heritability from genome wide association studies (GWAS).

One of the most important facts about uncertainty and something that everyone should know but often doesn’t is that when you add two imprecise quantities together, while the average of the sum is the sum of the averages of the individual quantities, the total error (i.e. standard deviation) is not the sum of the standard deviations but the square root of the sum of the square of the standard deviations or variances. In other words, when you add two uncorrelated noisy variables, the variance of the sum is the sum of the variances. Hence, the error grows as the square root of the number of quantities you add and not linearly as it had been assumed for centuries. There is a great article in the American Scientist from 2007 called The Most Dangerous Equation giving a history of some calamities that resulted from not knowing about how variances sum. The variance of a trait can thus be expressed as the sum of the genetic variance and environmental variance, where environment just means everything that is not correlated to genetics. The heritability is the ratio of the genetic variance to the trait variance.

Government shutdown

As of today, I am officially furloughed without pay since the NIH is officially closed and nonessential employees like myself are barred from working without pay by the Antideficiency Act of 1884. However, given that blogging is not considered an official duty, I can continue to post to Scientific Clearing House. Those who are not up on American politics may be wondering why the US government has shutdown. The reason is that the US fiscal year begins on Oct 1 and according to the the US Constitution, only Congress can appropriate funds for the functioning of government and they did not pass a budget for the new fiscal year by midnight of September 30. Actually, Congress has not passed a budget on time in recent years but has passed Continuing Resolutions that to keep the government going. So why have they not passed a budget or a CR this year? Well, currently the US government is divided with the Democratic party controlling the Senate and Presidency and the Republican party controlling the House of Representatives. All three entities must agree for a law to pass. Three years ago, the Democrats controlled the Congress, which includes both the House and Senate, and passed the Affordable Care Act, also known as Obamacare, which the President signed into law.   The Republicans took control of the House in 2011 and have been trying to repeal the ACA ever since but have been stopped by the Senate. This year they decided to try a new tactic, which was to pass a budget that withholds funding for the ACA. The Senate did not agree, passed a budget with the ACA and sent it back to the House, which then took out funding for the ACA again with some modifications and sent it back. This went on back and forth without converging to an agreement and thus we are closed today.