As several commenters had speculated, the cause of the problem appears to have been the flipping of a 96-well plate (a solid block of plastic containing 96 separate samples in individual wells) by 180 degrees, resulting in all of the samples being in the incorrect locations for downstream analysis.
I'm frankly astonished that this was possible at an accredited genotyping facility - plate-flipping is an age-old problem, but trivial to prevent with good plate/machine design that only allows plates to be loaded in a single orientation. And indeed, in the post 23andMe announces that LabCorp "has adjusted the mounting process for these 96-well plates and this new adjustment physically prevents any incorrect manual placement of the plates used at this step of processing". It boggles the mind that LabCorp didn't see fit to do this earlier.
In the comments thread on my post yesterday there was discussion about additional steps that could be taken by 23andMe to spot such mix-ups in future. Neil Walker noted that matching subject sex with that inferred from genotype data is a routine component of quality control in large-scale academic genomic studies, and I suggested a slightly expanded approach:
There's no reason not to have an "opt-in" questionnaire when people register on the website (after mailing off their sample) for the purposes of sample tracking: sex, eye colour and hair colour alone would be a start. If even 50% of customers filled in these details that would be enough to spot a dodgy plate with high confidence, even allowing wiggle room for incorrect answers given by some customers.It looks as though the company will be adopting at least a basic version of this approach: the 23andMe post says that "23andMe will collect data regarding sex for all new customers prior to laboratory processing so an additional quality check can be conducted prior to uploading data".
More elaborate plate-tracking measures - such as the incorporation of known control samples at different positions in each plate being processed, serving as a kind of molecular barcode - are not discussed, but these might be a worthwhile investment if the company is to avoid a repeat of the horrendous negative publicity aroused by this incident.
The bulk of the blame for this incident must be laid squarely on LabCorp, who failed to adequately protect against an extremely well-known source of sample mix-ups. However, it's worth noting that 23andMe routinely suggests that customers fill in basic surveys (which - from memory - include sex and other useful physical traits) after mailing in their kit but before their results are returned. If even a fraction of customers had filled in these surveys with some degree of accuracy - which I suspect they had - and if 23andMe had used this information to compare against the results from each plate, this mix-up would almost certainly have been spotted well before any customer received potentially distressing and erroneous results.
Still, kudos to 23andMe for responding to this problem, rapidly implementing a solution, and discussing the situation openly - and for other personal genomics companies out there, hopefully this has been an important lesson in the negative PR impact of quality control problems.
Life Science
Medicine & Health
Comments
That really is boggling, so simple to avoid by plate design, this is one mistake that would have been avoided by cutting corners.
It is amazing that it is not a regulatory requirement - isn't this CLIA domain, to establish procedures to minimise wherever possible incorrect results? Both LabCorp and 23andMe have some tidying up to do, embarrassing for both but I agree that the 23andMe response has been rapid and apparently honest. Compared to other corporate mistakes this is not up there at the top and probably makes 23andMe/LabCorp a safe bet for the future.
They are also lucky to have so many experts weighing in with excellent free advice via the blogs and good for them that they have not ignored it.
Continuing with transparency and honesty - I would like the various companies to publish their ongoing routine rate of accuracy. A quick survey gives figures from 99% up to 99.99% for 23andMe, I would hope that it is nearer the latter. Not so critical for these services but even at 1 miscall in 10,000 means that in a 20 SNP pharmacogenetic panel 1 person in 500 would get a wrong call.
Posted by: Keith Grimaldi | June 9, 2010 1:35 AM
Hi Daniel, FYI here is an interesting opinion piece on personal genomics at UC-Berkeley:
http://www.bionews.org.uk/page_62055.asp
Posted by: nicole | June 9, 2010 2:28 AM
Hi Keith,
They are also lucky to have so many experts weighing in with excellent free advice via the blogs and good for them that they have not ignored it.
Just to be clear - I don't think 23andMe came up with their plan after reading about it here or on other blogs; it's a basic sanity check that's widely used in other places doing large-scale genotyping. Presumably the only reason it hasn't been implemented before this is concern about privacy, but this has now been outweighed by the clearly disastrous consequences of further sample mix-ups.
Posted by: Daniel MacArthur | June 9, 2010 8:16 AM
Perhaps the excellent series on the origin of Murphy's law should be required reading in any application for a labortatory permit. The loss of data from a dangerous rocket sled test ride was caused by misconnection of sensor wires. The apparatus was brought to the test site in California by one Murphy, an engineer from Wright-Patterson Air Force Base if memory serves. The local technicians were unfamiliar with the new device and connected the sensor wires wrong. This led Murphy to develop a polarized plug that could only be attached in the correct way. The articles trace various ideas as to how the term "Murphy's Law" came to be applied. The basic thrust is that it is not a statement of universal pessimism as is usually inferred. Rather it conveys the idea that if something can go wrong sooner or later it will and that smart design makes that impossible -- at least with regard to foreseeable problems.
The link is: http://improbable.com/airchives/paperair/volume9/v9i5/murphy/murphy0.html
Posted by: Murphy | June 9, 2010 3:55 PM
Mistakes in clinical testing probably occur with far greater frequency than most realize; checks and backup systems catch the vast majority before they get out the door, and most of the rest are probably switches or contaminations that affect only 1 or 2 individuals at a time. What is stunning in this case, from a PR standpoint, is 96 people being affected by a single simple error. Glad I'm not the one having to explain this to the press or involved families!
Posted by: Shecky Riemann | June 10, 2010 5:36 PM
"It is amazing that it is not a regulatory requirement - isn't this CLIA domain, to establish procedures to minimise wherever possible incorrect results?"
The requirements for the testing done on behalf of companies like 23andMe are probably less strict than those for testing on behalf of hospitals, allowing corners to be cut in order to reduce cost.
They're after all not a medical facility, but a commercial enterprise selling information services rather than medical treatment or advice.
If like Prescan (a company that performs full body MRI scans and uses the results to diagnose your potential future medical problems, or claims to, they seem to have extremely lucrative contracts with doctors who use those diagnoses to sell you all kinds of treatment or stuff) they do offer medical services based on the results, rules would (or at least should) be stricter.
Posted by: J.T. Wenting | June 11, 2010 2:01 AM
Cut molded corners and in-line QC standards are well sorted out techniques to reduce sample error. At one of my genetic analysis companies, we found samples that contained very rare ...and even impossible genotypes (non-human) to screen against as a positive and negative control at set plate position wells.
so...there should never be a result in A5 ect.....
Posted by: Damon Hostin | June 15, 2010 11:33 AM