A new paper on the evolution of overconfidence (arXiv:0909.4043v2) will appear shortly in Nature. (Hat tip to J.L. Delatre). It is well known in psychology that people generally overvalue themselves and it has always been a puzzle as to why. This paper argues that under certain plausible conditions, it may have been evolutionarily advantageous to be overconfident. One of the authors is James Fowler who has garnered recent fame for claiming with Nicholas Christakis that medically noninfectious phenomena such as obesity and divorce are socially contagious. I have always been skeptical of these social network results and it seems like there has been some recent push back. Statistician and blogger Andrew Gelman has a summary of the critiques here. The problem with these papers fall in line with the same problems of many other clinical papers that I have posted on before (e.g. see here and here). The evolution of overconfidence paper does not rely on statistics but on a simple evolutionary model.

The model considers competition between two parties for some scarce resource. Each party possess some heritable attribute and the one with the higher value of that attribute will win a contest and obtain the resource. The model allows for three outcomes in any interaction: 1) winning a competition and obtaining the resource with value W-C (where C is the cost of competing), 2) claiming the resource without a fight with value W, and 3) losing a competition with a value -C. The parties assess their own and their opponents attributes before deciding to compete. If both parties had perfect information, participating in a contest would be unnecessary. Both parties would realize who would win and the stronger of the two would claim the prize. However, because of the error and biases in assessing attributes, resources will be contested. Overconfidence is represented as a positive bias in assessing oneself. The authors chose a model that was simple enough to explicitly evaluate the outcomes of all possible situations and show that when the reward for winning is sufficiently large compared to the cost, then overconfidence is evolutionarily stable.

Here I will present a simpler toy model of why the result is plausible. Let P be the probability that a given party will win a competition on average and let Q be the probability that they will engage in a competition. Hence, Q is a measure of overconfidence. Using these values, we can then compute the expectation value of an interaction:

(i.e. the probability of a competition and winning is , the probability of winning and not having to fight is , the probability of losing a competition is , and it doesn’t cost anything to not compete.) The derivative of E with respect to Q is

Hence, we see that if , i.e. the reward of winning sufficiently exceeds the cost of competing, then the expectation value is guaranteed to increase with increasing confidence. Of course this simple demonstration doesn’t prove that overconfidence is a stable strategy but it does affirm Woody Allen’s observation that “95% of life is just showing up.”