Peer Review Is an Awful System, but It's Better Than All of the Other Awful Systems: The McKinsey ACA Report Edition

Peer review, the process in which scientific publication has to pass muster with critical reviewers, has a lot of problems. But a widely-cited report by McKinsey & Company consulting which claimed that Obamacare will result in one-third of companies dropping healthcare coverage for their employees is worth examining (italics mine):

The predictable fallout led Democrats, and several reporters, to press McKinsey for the survey itself -- a request McKinsey has declined on the grounds that the material is proprietary.

But multiple sources both within and outside the firm tell TPM the survey was not conducted using McKinsey's typical, meticulous methodology. Indeed, the article the firm published was not intended to give the subject matter the same authoritative treatment as more thorough studies on the same topic -- particularly those conducted by numerous think tanks, and the Congressional Budget Office, which came to the opposite conclusion. And that's created a clamor within the firm at high levels to set the record straight.

"This particular survey wasn't designed in away that would allow it to be peer review published or cited academically," said one source familiar with the controversy....

What was one of the problems? Well:

Lastly, this tidbit was included in the McKinsey Quarterly article about the survey:

"...our survey educated respondents about [employer sponsored insurance] implications for their companies and employees before they were asked about post-2014 strategies."

In other words, those conducting the survey may have primed respondents to say they would keep or drop coverage.

And there's no potential conflict of interest either (italics mine):

A third source speculates that the firm may have reached its outlying conclusion by basing its questions on the firm's own advice to clients on how best to arbitrage the new reforms. Specifically, under the law, employers could devise their benefits packages in ways that makes them unappealing to lower-income employees, who would then have to enter the exchanges. Though TPM could not confirm this, the conclusion is supported by a disclosure within the McKinsey study itself.

There are all sorts of problems with federally-funded research and peer review. But this scandal--and it is a scandal, although no penises were photographed--should put things in proper perspective.