Data Dredging at VaxGen

By David Michaels

It came as no surprise to some observers that VaxGen (a biotech company in Brisbane, California) failed to meet the specifications of its contract to provide the US government with 75 million doses of a new anthrax vaccine. VaxGen has been playing fast and loose for quite some time – most notably with a famous instance of data dredging in the analysis of the clinical trials for AIDSVAX, its failed AIDS vaccine. I’ll come back to that below.

On Tuesday, the Department of Health and Human Services announced was ending its sole-supplier contract with VaxGen, which would have been worth up to $877.5 had the company been able to produce vaccines that worked. HHS officials would not discuss the reasons of the cancellation, but, according to Renae Merle’s piece in today’s Washington Post, “HHS evidently canceled the contract after VaxGen missed its deadline Monday to begin human testing because of concerns at the Food and Drug Administration about the product’s reliability.”


Ken Silverstein at Harper.org has more of the details:

VaxGen received the anthrax-vaccine contract two years ago, in spite of the fact that the decade-old firm had produced no marketable products, had demonstrated disturbing ethical failures in the area of human-subject drug testing, and had been delisted by Nasdaq because of accounting irregularities. An earlier, similarly hyped-up effort to develop an AIDS vaccine—for which the company received millions from the National Institutes for Health—ended in total failure and led to a number of lawsuits by investors who charged that VaxGen had fraudulently hyped the vaccine’s prospects for success.

VaxGen’s competitors suspected that the firm, in winning the anthrax contract, may have received a helping hand from Dr. Phil Russell, formerly a top contracting official at HHS who was involved in the BioShield project. Forbes describes Russell as a “long time acquaintance” of VaxGen CEO Lance Gordon; Russell, for his part, has denied that he helped out VaxGen.

Whatever the reasons for picking VaxGen, the choice has proved a disaster. VaxGen pledged to produce 75 million doses of the vaccine, with deliveries slated to begin early in 2006. But late last year, the company announced that unforseen “product refinements” would force it to postpone delivery of its vaccine until late 2006. Further problems ensued, and now the earliest delivery date would be late 2008 or 2009.

What about the data dredging? Epidemiologists know that with a sufficiently large data base, a good researcher can often come up with something deemed statistically significant. Dredge deep enough, wide enough, long enough, and then shock the world with your new finding. VaxGen did some data dredging in its clinical trial for AIDSVAX, which resulted in lots of false hopes, and longer trials of a vaccine that simply didn’t work.

As reported in JAMA, when the results of the first important trial were analysed, the vaccine was found not to reduce infection rates in the entire group studied, but there was a statistically significant protective effect among the African-Americans in the study. The vaccine did not work for Asians, but there were so few Asians in the trial that they could be added to blacks and yield a “Blacks and Asians” category that was still technically significant. In fact, “others” could be added without ruining the count, so “blacks and Asians and others” was still statistically significant. Ideally, Hispanics could have been added to the group and the vaccine might then have been touted for “non-whites,” but the study showed no benefit for Hispanics, of whom there were many in the trial, so the “non-white” category was no good. Hence the post hoc category “Blacks and Asians”

Of course, it’s possible that the test had turned up something of interest, but there was only one way to know for sure: conduct a new clinical trial to test if this group really did benefit from the vaccine. But since “Blacks, Asians, and others” was the best VaxGen company could do, that’s what it went with. When the vaccine received its next real test, a large clinical trial in Thailand, it failed miserably.

David Michaels heads the Project on Scientific Knowledge and Public Policy (SKAPP) and is Professor and Associate Chairman in the Department of Environmental and Occupational Health, the George Washington University School of Public Health and Health Services.

    The site is undergoing maintenance presently. Commenting has been disabled. Please check back later!