Why it is so hard to forecast COVID-19

I’ve been actively engaged in trying to model the COVID-19 pandemic since April and after 5 months I am pretty confident that models can estimate what is happening at this moment such as the number of people who are currently infected but not counted as a case. Back at the end of April our model predicted that the case ascertainment ratio ( total cases/total infected) was on the order of 1 in 10 that varied drastically between regions and that number has gone up with the advent of more testing so that it may now be on the order of 1 in 4 or possibly higher in some regions. These numbers more or less the anti-body test data.

However, I do not really trust my model to forecast what will happen a month from now much less six months. There are several reasons. One is that while the pandemic is global the dynamics are local and it is difficult if not impossible to get enough data for a detailed fine grained model that captures all the interactions between people. Another is that the data we do have is not completely reliable. Different regions define cases and deaths differently. There is no universally accepted definition for what constitutes a case or a death and the definition can change over time even for the same region. Thus, differences in death rates between regions or months could be due to differences in the biology of the virus, medical care, or how deaths are defined and when they are recorded. Depending on the region or time, a person with a SARS-CoV-2 infection who dies of a cardiac arrest may or may not be counted as a COVID-19 death. Deaths are sometimes not officially recorded for a week or two, particularly if the physician is overwhelmed with cases.

However, the most important reason models have difficulty forecasting the future is that modeling COVID-19 is as much if not more about modeling the behavior of people and government policy than modeling the biology of disease transmission and we are just not very good at predicting what people will do. This was pointed out by economist John Cochrane months ago, which I blogged about (see here). You can see why getting behavior correct is crucial to modeling a pandemic from the classic SIR model

\frac{dS}{dt} = -\beta SI

\frac{dI}{dt} = \beta SI - \sigma I

where I and S are the infected and susceptible fractions of the initial population, respectively. Behavior greatly affects the rate of infection \beta and small errors in \beta amplify exponentially. Suppression and mitigation measures such as social distancing, mask wearing, and vaccines reduce \beta, while super-spreading events increase \beta. The amplification of error is readily apparent near the onset of the pandemic where I grows like e^{\beta t}. If you change \beta by \delta \beta, then the I will grow like e^{\beta t+\delta \beta t} and thus the ratio is growing (or decaying) exponentially like e^{\delta \beta t}. The infection rate also appears in the initial reproduction number R_0 = \sigma/\beta. From a previous post, I derived approximate expressions for how long a pandemic would last and show that it scales as 1/(R_0-1) and thus errors in \beta will produce errors R_0, which could result in errors in how long the pandemic will last, which could be very large if R_0 is near one.

The infection rate is different everywhere and constantly changing and while it may be possible to get an estimate of it from the existing data there is no guarantee that previous trends can be extrapolated into the future. So while some of the COVID-19 models do a pretty good job at forecasting out a month or even 6 weeks (e.g. see here), I doubt any will be able to give us a good sense of what things will be like in January.

One thought on “Why it is so hard to forecast COVID-19

  1. Out of curiosity I skimmed Cochrane’s model because i remebered his name though forgot who it was—he’s at Hoover/U Chicago so now i remember. (birds of a feather flock together –a sort of informal and generalized quarantining or lockdown, or lockout—‘competetive exclusion’–human groups or tribes tend to share memes or ‘viruses of the mind’).

    His model looks reasonable, intuitive —just makes the ‘constants’ beta, R_0 time dependent. I sort of call this ‘adding more nonlinearity’.

    In a sense (or in some ranges) his slightly more complex model (he cites someone at stanford which i guess is related) doesn’t change anything. A few more wiggles, but basically same limit for most cases–and he points out to an extent one can choose the differences either through informal rules (persoal choice, education, nudges) or govt policy.

    (Same issue about laws versus behavioral change arrived informally arises in discussions of climate change, economic growth, and population—-some people prefer controlling population via ‘fertility taxes’ (a mild form of china’s one child policy) while others promote using educattion instead (ie educated people can plan their own family sizes and likely this will have same effect as a law on sizes) .
    Some people favor lots of carbon, pollution, value added, Pigouvian or sin taxes, to cut consumption of carbon and problem producing goods, and forms of wealth or progressive income taxes for basic income, health, and education services, to make the economy stable—less prone to unrest –which itself can affect population and climate . Others favor using ‘shaming’, persuasion by creating alternative models of consumption and distribution, etc. rather than laws.)

    My ‘guess’ is that COVID-19 will go as ‘business as usual’ like other viruses and diseases, so its not the black plague (which i think killed off 1/3rd of europe). So January will be like an average of the last 6-10 months-which means the same, or else the virus will be gradually be wiped out with possibly periodic recurrences, or possibly it will just become a chronic condition at least for some (including me—something you live with–even tho i tested negative).

    Actually, my interpretations of the idea that globally the epidemic is ‘flat’ is that is an application of the law of large numbers and/or CLT (i forget exactly how these differ–sometimes they may not, sometimes they may) —just take a random sample off the earth and one will find some ‘modes’ increasing while others decrease –a la the Fermi-Pasta-Ulam experiments. Eventually it will approach the stationary or equilibrium distribution (basically like an ‘old growth forest’–it doesn’t mean its totally static –trees fall, occasional small fires, new growth, etc. . ).

    Whatever pattern of governmental regulation took—extreme lockdowns, or minor ones— it willl all sort of even out—possibly at different rates. (Actually the pprevious post on ‘why there is no herd immunity’ suggests that if different regions have very different rules and there is migration between them, that could mean reaching the equiblibirum or stationary distibution could take a very long time–and maybe never will.)

    There are so parameters in these models in theory probably everything is possible. Cochran emphasizes the difference between epidimiology as done by people who study animals for whom R and B are constants, versus economists who study human animals which may be able to change behavior (there is some debate about this–eg how hard wired are people, if they even have any free will–they may think so, just as they may believe god doesn’t exist, but may be wrong. Of course if they do have free will then they should be able to create god. ).

    So its quite possible ‘this doesn’t change everything’ .


    As an aside I looked at Cochran’s first blog which is on finance and mentions MMT modern monetary theory’ and S kelton who promotes it). I agree with his assessment of MMT–its not a theory (or at best warmed over keynsianism with a twist called ‘the tax break for the rich’—MMT says ‘let the the poor have their cake and eat it2’ so long as they work at their govt paid for guaranteed job (basically voluntary wage slavery —if you want to eat, work)) . ; and also dont take cake from the rich to pay for it. The poor can make and eat their own cake or whatever they are able to affrod to make –eg stone soup.

    MMT people pass this off as an alternative to a UBI to help the poor; but i view it as a possible trojan horse —people are just going to turned into a proletarian caste like hamsters who make their own wheels. ) In a sense we already have this—its called the youtube/podcast culture.

    Actually MMT and UBI are equivalent in some limits–just as are capitalism, marxism-communism , and anarcho-communism. This is my tendency to see all models as limits of the same model—same for COVID, gun regulations, voting versus non-electoral politics (ie personal change oor protesting.)


    He also writes on inequality—says for example in his view its misplaced for people Like Krugman or Pietty and others to care about inequality. He says a poor laborer in California doesn’t care about the jet owned by a billionaire flying above the clouds above the farm or factory s/he’s working at–s/he’s worried about paying the rent. S pinker has the same view—says ‘envy’ or worry about income and wealth inequality are myths by ‘liberals’ and ‘leftists’ trying to make the poor angry.

    my view is they aren’t myths—poor people do worry about those billionaires and jets because those are the people who own and run where they work, and determine whether they can pay rent.

    if they dont it may also be a behavioral strategy —dont worry about things you can’t change because its unhealthy (R Aapolsky advises this from Stanford).

    and as far as being grateful that the rich do provide some jobs, they may not be happy that it comes with all sorts of health hazards—some of which would be preventible by making fewer profits, etc.


    If Cochran used the same kind of behavioral logic for COVID that he uses in his analyses of economics of inequality he probably would say a billionaire or poor person in Wuhang (sic?) China who has COVID has no effect on someone in the USA and hence should be ignored.

    —— Most of the people sort of promoting less stringent controls and more ‘herd imunity’ emphasize economic issues —i wonder how these interact. (I discussed this a bit with people who volunteer on the ‘endcovid’ list maintained by NECSI but it didn’t go far—no modelers were in that group. Putting that in a model would add more complexity.)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s