New species form when separate populations of related organisms are no longer able to interbreed. Reproductive isolation can arise if two populations evolve different mating behaviors, or lifestyles so different that individuals from different populations don't even encounter each other—but it need not mean that matings between the two populations never occur. In fact, speciation can arise in the face of quite a lot of interbreeding, so long as the hybrids produced by interbreeding are less fit than "purebred" individuals.
Edith's checkerspot, a small butterfly native to Western North America. Checkerspot populations in California use a wide variety of different host plants, and a recent study has shown that the offspring of parents from different host plants are maladapted in the wild.
In the Sierra Nevada mountains, logging has created a new kind of habitat for Edith's checkerspot [PDF]—patches of cleared forest where the butterfly's locally preferred host plant, Pedicularis semibarbata, is rare or nonexistent, but an alternative host plant, Collinsia torreyi, is plentiful. In the transition between clearings and less-disturbed forest, the two plants may often grow side by side.
If these differences have a genetic basis, then hybrid checkerspots might exhibit intermediate behaviors, which might not work so well on either host plant. To test for this "hybrid inviability," the new study's authors crossed checkerspots from populations encountering only one host plant or the other, and then tested the hybrids' performance in the field—and what they found confirms those predictions.
The Goldilocks principle—intermediate is better–doesn't apply to hybrid checkerspots. Hybrid caterpillars foraged on leaves at an intermediate height on both host plants, and grew more slowly than purebred caterpillars. Hybrid females laid an intermediate number of eggs on both host plants, and laid them at an intermediate height. This left their offspring in a poor position for foraging after they hatched—and indeed, they grew more slowly than larvae hatched from eggs that were laid at the "traditional" heights on the host plants.
So it looks as though natural selection for better performance on Collinsia has led to the evolution of checkerspots that are at a disadvantage using Pedicularis (and vice versa). This even to the point that hybrids, which feed and oviposit in ways that are only somewhat different from the optimum, pay performance costs.
What's interesting, though, is that this hasn't led to greater genetic differentiation of checkerspot populations using different host plants; as assessed using randomly-selected genetic markers, there is an isolation-by-distance effect, but no effect of host plant use. (The authors cite a previous study using about 400 AFLP loci [PDF].) That suggests that only a few genes are responsible for the observed adaptive differences, and that natural hybridization between checkerspot populations using different hosts may be mixing together the rest of the genome.
McBride, C., & Singer, M. (2010). Field studies reveal strong postmating isolation between ecologically divergent butterfly populations. PLoS Biology, 8 (10) DOI: 10.1371/journal.pbio.1000529
Singer, M.C., & Wee, B. (2005). Spatial pattern in checkerspot butterfly-host plant association at local, metapopulation and regional scales. Annales Zoologici Fennici, 42, 347-61