Neandertal-"modern" mixing

Neandertals and moderns mixed, and it matters:

Twenty-five years ago, the Middle-to-Upper Paleolithic transition in Europe could be represented as a straightforward process subsuming both the emergence of symbolic behavior and the replacement of Neandertals by modern humans...Over the last decade, however, taphonomic critiques of the archeology of the transition have made it clear that, in Europe, fully symbolic sapiens behavior predates both the Aurignacian and moderns. And, in line with evidence from the nuclear genome rejecting strict replacement models based on mtDNA alone, the small number of early modern specimens that passed the test of direct dating present archaic features unknown in the African lineage, suggesting admixture at the time of contact.

In the realm of culture, the archeological evidence also supports a Neandertal contribution to Europe's earliest modern human societies, which feature personal ornaments completely unknown before immigration and are characteristic of such Neandertal-associated archeological entities as the Châtelperronian and the Uluzzian. The chronometric data suggest that, north of the Ebro divide, the entire interaction process may have been resolved within the millennium centered around 42,000 calendar years ago. Such a rapid absorption of the Neandertals is consistent with the size imbalance between the two gene reservoirs and further supports significant levels of admixture.

I can't follow the paleontology & archeology with any critical eye, but the argument about different population sizes does interest me. The basic contention of the review paper is that the genetic replacement of Neandertals by H. sapiens sapiens from Africa was a matter of assimilation of a smaller group by a larger one. Migration of individuals between populations has a strong equilibrating influence on allele frequency differences, and small populations can quickly be genetically swamped by large populations. Assume for example that you have a small populations, A, next to a very large population B. Assume that at time t = 0 A is fixed for allele 1 and B for allele 2. Assume that there is a 1% decrement to the frequency of allele 1 per generation due to admixture via a small number of cross-population marriages (e.g., what if they were separated by a water barrier and lowered sea levels resulted in a narrow isthmus). Within 20 generations, 1000 years, population A is only 60% frequency for allele 1, the "ancestral" allele on the locus. In this model we are assuming for simplicity that B is large enough in size that A has little effect upon it. There is a real world analog to this: Jews in Europe. It looks like that low levels of admixture (e.g., 1-2% outmarriage per generation) over a few thousand years have resulted in a non-trivial amount of "non-Semitic" alleles being common within the genomes of most Ashkenazi Jews (you can see similar dynamics in Jewish populations around the world). On the other hand, the Jewish contribution to the gentile populations is minimal because their base population was small. When a population imbalance exists genetic exchange is not a two way street.

In the paper above the author makes the case that cultural modernity was sparking across western Eurasia ~50 K BP, amongst both African and European sapiens (our putative ancestors and Neandertals). Why did the Neandertals get replaced? The author seems to make the case that the cultural revolution resulted in a greater expansion of African populations because of the greater natural carrying capacity. In contrast, Ice Age Europe was instrinsically constrained because of the climatic conditions so the Neandertals were simply swamped by newcomers as the demes became interconnected by migration out of Africa.

I think this is too pat. The effective population size arguments are interesting, and I think there is something to this (John Hawks has argued that large effective population in Africa over millions of years is one reason why phylogenies tend to coalesce in that continent rather than elsewhere), but, it seems likely that modern "Europeans" overall are more Ice Age than farming people. That is, the settlers before 10 BP made the predominant genetic contribution to modern European genomes (this varies by region), not the waves of farmers from Anatolia on the cusp of the Neolithic Revolution. It seems that the Middle East population base expanded greatly in population ~10 K BP, and yet cultural diffusion was fast enough that the northwest fringes of Europe were almost untouched by the waves of demographic advance pulsing out of Anatolia (in contrast to say Greece or Italy, which bear strong "Neolithic" genetic stamps).¹ Why couldn't the Neandertals simply expand their populations too? If Europe was instrinsically incapable of sustaining large populations during the Ice Age why is it that most modern Europeans seem to descend from populations which settled between 30 - 10 K BP and not after the Neolithich Revolution? Certainly the invention of agriculture was a major cultural innovation which sparked a demographic tsunami which we are still dealing with today.

The current dominant narrative about "First Contact" was that biologically distinct Neandertals were totally replaced by culturally and superior H sapiens sapiens (whose cognitive processes were biologically novel and allowed for more flexible thought and abstraction). I suspect this is wrong in the stark and extreme simplicity, the data seems to be that some admixture did occur from the morphology, and soon genetics. Nevertheless, by the nature of the genetic and anatomical evidence it does seem probable that the preponderance of our ancestry, or specifically of Europeans, is "African," and not archaic European (Neandertal). So there was a population size imbalance as the two demes fused. Nevertheless, that imbalance was likely always in existance, and the Neandertals had survived and endured for hundreds of the thousands of years as a persistent ecotype. Something changed between 50 K - 30 K in the African-Neandertal dynamic. The Neandertals were assimilated as Africans exploded out of their home continent and swept all before them in a wave of genetic advance. So I suspect that the Africans were special in some way, but the Neandertals were no primitive brutes with no skills or knowledge to hand off to the new players on the scene, hundreds of thousands of years in Europe would no doubt have resulted in specialized local adaptations and cultural traditions ripe for the picking.

1 - A clarifying point, a "wave of advance" by its nature becomes dilute and distinct from its source character as it sweeps across demes. Nevertheless, the Bantu "wave of advance" in East and South Africa (from West-Central Africa) or the Yayoi advance in Japan overlain over the Jomon substrate (so that the Yayoi ancestral quantum outweights the Jomon 3:1 in the modern Japanese) show that a powerful pulse can be transformative and outpace cultural assimilation and reaction.