Group selection versus kin selection
Let's start with some background. Unselfish, cooperative behavior has long been a puzzle in evolutionary biology, because natural selection should never favor individuals who make significant sacrifices for the benefit of others. Sure, an unselfish individual might expect those she helps to reciprocate later; but a population of the unselfish would be easily overrun by those who don't reciprocate.
There have historically been two answers to the problem of the selfish out-competing the unselfish. The first case is basically an extension of logic we all learned in kindergarten: cooperative groups can do things that uncooperative groups can't. Like, for instance, start a neighborhood garden.
Under this model, neighborhoods of cooperative, garden-making people are nicer places to live, and their inhabitants can collectively out-compete other neighborhoods that can't get it together to start a community garden. In evolutionary terms, this is group selection—even if individuals sacrifice to build the garden, the group as a whole benefits. Unfortunately, this breaks down if the new garden attracts selfish people to move to the neighborhood, buy up all the cheap real estate, and open Urban Outfitters franchises.
There's another possibility, though. What if unselfish behavior isn't always truly unselfish? For instance, if you help your relatives, you're actually helping some of your own genes. You share half your genes with your siblings, a quarter of your genes with half-siblings, an eighth of your genes with first cousins, and so on. This means that Michael Bluth might be on to something.
Evolutionarily speaking, it doesn't matter if Michael spends all his time helping his feckless family, as long those efforts help someone in the family (G.O.B., most likely) reproduce and perpetuate some of the genes that Michael shares with him or her. This idea was advanced by W.D. Hamilton in two 1964 papers, one mathematical [PDF], and one more focused on real-world examples [PDF]; we now know it as kin selection. It doesn't hold up so well for maintaining the kind of complex society humans have today, where we interact with lots of completely unrelated people—but it might have got the ball rolling toward the wheel, war, New York and so forth by selecting for cooperative behaviors within small tribes back at the dawn of history.
The group selection versus kin selection debate has gone back and forth for decades, and the new paper is a shot across the bow of kin selection. The authors, Martin Nowak, Corina Tarnita, and E.O. Wilson, aim to do two things: first, prove that kin selection is wrong; and second, describe an alternative explanation. For the first, they argue that kin selection only applies in narrow circumstances, that those circumstances never show up in nature, and that empirical studies just don't support the model. Johnny Humphreys makes some reasonable objections to these arguments, and so do several folks interviewed by Carl Zimmer, and I'll refer you there rather than try to improve on them.* I'm more interested in the second part: the alternative explanation.
Enslaved by Mom
No individual fitness for you—you're cogs in the Superorganism. Photo by jby.Nowak et al. propose to explain the evolution of unselfishness as it applies to eusociality—organisms like ants or bees or naked mole rats, in which colonies of (closely related) individuals defer most or all of their opportunities to reproduce, in order to support one or a few individuals that reproduce a lot. As Johnny points out in his critique, it's not clear that eusociality is the same thing as unselfishness at all, even though it's historically cited as an example of unselfishness [$a]. The new model that Nowak et al. develop actually makes the difference between eusociality and unselfishness even clearer. Under their model, it's not that worker ants give up reproductive opportunities to help their mother, the Queen, reproduce—it's that the Queen takes away their reproductive opportunities.
The key insight of the new model is that, in evolving from a non-social insect to a eusocial one, the natural selection that matters affects not the individuals evolving into workers, but the individual who would be Queen. Consider an insect similar to the probable ancestor of ants: females build nests, provision them with food, and lay eggs inside. Nowak et al. propose that a female who evolved the ability to lay "worker" eggs—females that grow up not to found their own nest, but to help in their mother's—would have greater fitness than females without such helpful offspring.
Aside from the probability of evolving "worker" eggs (which is not a small issue, I think), this shift in perspective from the fitness of the worker to the fitness of the Queen makes all sorts of sense to me. I've often wondered why myrmecologists don't treat ant colonies as single organisms, rather than collections of cooperating individuals.
But this approach also seems to sidestep the key question biologists hope to answer with kin selection and group selection models—these models aim to explain how individuals can come together to cooperate, but Nowak et al. have built a model that looks more like enslavement. I can't learn anything about how unselfish behavior can spontaneously evolve in a population by looking at a population that has had unselfishness imposed upon it. To indulge in one last especially geeky pop culture reference, it'd be like trying to learn about market economics by studying The Borg.
Nowak, Tarnita, and Wilson might have come up with a very good model for the evolution of eusociality; but if so, it means that eusociality is a bad model for the evolution of cooperation as we usually conceive it.
* I will, however, note that Nowak et al. do something I've never seen in a scholarly paper before—in dismissing empirical studies of kin selection, they defer substantive discussion to the Supplementary Information. There are, in fact, 43 pages of SI for this 6-page paper, including two major mathematical models and the discussion of empirical kin selection studies. This is a problem, but one that is beyond the scope of this already-long post.
Axelrod, R., & Hamilton, W. (1981). The evolution of cooperation. Science, 211 (4489), 1390-1396 DOI: 10.1126/science.7466396
Hamilton, W.D. (1964). The genetical evolution of social behaviour. I. Journal of Theoretical Biology, 7 (1), 1-16 DOI: 10.1016/0022-5193(64)90038-4
Hamilton, W.D. (1964). The genetical evolution of social behaviour. II. Journal of Theoretical Biology, 7 (1), 17-52 DOI: 10.1016/0022-5193(64)90039-6
Nowak, M., Tarnita, C., & Wilson, E. (2010). The evolution of eusociality. Nature, 466 (7310), 1057-62 DOI: 10.1038/nature09205