Four scientists - including the omnipresent J. Craig Venter (left) - have penned an opinion piece in the latest issue of Nature based results from five individuals genotyped by two separate personal genomics companies. The article highlights some deficiencies in the way that genetic data are currently used by direct-to-consumer companies to generate risk predictions and to present them to customers.
[We] found that only two-thirds of relative risk predictions qualitatively agree between 23andMe and Navigenics when averaged across our five individuals ... For four diseases, the predictions between the two companies completely agree for all individuals. In contrast, for seven diseases, 50% or less of the predictions agree between the two companies across the individuals.
Companies should report the genetic contribution for the markers tested: personal genomics companies generally do a good job of explaining what proportion of disease risk is due to genetic vs environmental factors, but they typically don't spell out what proportion of the genetic disease risk is explained by the markers they test for. I agree with the authors that companies need to do a much better job of making this clear to customers; however, it's also worth acknowledging that this calculation is often non-trivial given the information currently provided in the published literature. This is as much an issue for the broader genetics community as for personal genomics companies.
Hi Daniel, Awesome review of the paper. Thanks. You mentioned CLIA standards compliance would cause more accurate genotyping accuracy but that seemed to be good news in the study (that the two were accurate). Would CLIA compliance have any impact on improving disease risk prediction from the genotype data?
CLIA compliance may have had a positive effect on genotyping accuracy, but I haven't seen anything to suggest that these companies had any worse accuracy prior to meeting the CLIA standards.
And no, as far as I know CLIA compliance would have no effect whatsoever on risk prediction algorithms; it purely relates to quality control of the genotyping procedure itself.
thanks for the review! This is indeed a great demonstration of how scientists can constructively engage in DTC genomics. I agree that we need to build prospective studies to evuate the long term benefits of having priviledge risk information. But there is no need to limit that to genomic information, we are in a good position to make personalized risk-profiles based on clinical measures. Has that had a great impact? NO. How long before we reach public health significance....?
Amusing that Craig Venter - he who sparked the race for preventative patenting of the human genome by the publicly-funded genome consortium - should be calling for DTC testing of the "real" risk markers!
examining whether receiving genetic risk information actually changes behavioural outcomes,
The first few studies are in:
Disclosure of APOE genotype for risk of Alzheimer's disease.
Green RC, Roberts JS, Cupples LA, Relkin NR, Whitehouse PJ, Brown T, Eckert SL, Butson M, Sadovnick AD, Quaid KA, Chen C, Cook-Deegan R, Farrer LA; REVEAL Study Group.
N Engl J Med. 2009 Jul 16;361(3):245-54.PMID: 19605829
And companion opinion piece:
Effect of genetic testing for risk of Alzheimer's disease.
Kane RA, Kane RL.
N Engl J Med. 2009 Jul 16;361(3):298-9. PMID: 19605835
Just for completeness' sake, I should point out that 23andMe offers three well-validated pharmacogenomics reports: warfarin sensitivity, clopidogrel efficacy, and 5-fluorouracil sensitivity (with more to come). (Full disclosure: I work for 23andMe.)
Another important behavioral paper came out recently on the NIH Multiplex Initiative.
Characteristics of users of online personalized genomic risk assessments: implications for physician-patient interactions [PDF].
McBride CM, Alford SH, Reid RJ, Larson EB, Baxevanis AD, Brody LC.
Genet Med. 2009 Aug;11(8):582-7
I don't see how it's constructive to engage with such work at all. It's plainly obvious that we're no where close to understanding gene/organism/environment interactions as a whole (except for particular simply "broken" genes) as long as the phrase "genetic contribution" still exists.
How much does one transistor "contribute" to the function of a cpu? Put that in a simple proportion and you'll have a patent that will make you the richest person in the world -- right next to the guy who invents a perpetual motion machine.
Until then, this work mostly sits right next to 19th century snake-oil salesmanship -- hey, sometimes even those potions turned out to have an effect on something.
To the extent the science allows it, most of the recommendations are already being used by the freely downloadable Promethease program, which compares DNA data from any consumer platform against the SNPedia wiki of genetic variation.
And since SNPedia is a wiki, consensus among geneticists, genetic counselors and other scientists can be readily seen and applied for the benefit of everyone.
Is this really true? I'm not an expert so I'm not sure, but I find even limited utility hardly comparable to snake oil. Can you back your assertion with evidence?
In related news, TrelawneyDivination Incorporated has announced it is launching a DTC Personal Genomics platform to bolster its current lineup of classic Divination Technologies, including Tea Leaf and Astrological assays.
That many of the claims for DTC are over-stated, does not mean that all of them are wrong.
For example, some of the pharmacogenomics ambitions are beginning to be realised:
Well Known Pairs of Gene-Drug PGx Relationships
If you look at the last example, on warfarin dosage, and follow the link you'll see why this information may be of some benefit in averting Adverse Drug Events, as it is used for:
the treatment of retinal vascular occlusion, pulmonary embolism, cardiomyopathy, atrial fibrillation and flutter, cerebral embolism, transient cerebral ischaemia, arterial embolism and thrombosis.
i.e. lots of life-threatening illnesses, but:
Warfarin is also used as a rodenticide.
So maybe a quick way of narrowing down a non-toxic dose would be useful?
Getting your entire genome sequenced will eventually be cheaper than giving birth at a hospital.
So all this is rearguard stuff is moot.
The medical genomics revolution is here to stay.
"Companies should test pharmacogenomic markers: the authors argue that genetic variants predicting response to drugs will prove particularly useful. I agree, and so do personal genomics companies - given their utility and their interest for customers I have little doubt that these variants will generally be added to the companies' chips as soon as they become available."
While they should be freely available, I think tests for pharmacogenomic markers will for the most part be patented, and Pharma companies may or may not license third parties to do those tests on whoever wants them, and even if they do the collection of license fees might be unaffordable. (the three drugs that 23andMe runs tests for are off patent).
1. there's a genetic test to predict whether a drug will cause severe side effects. Your doctor (and the drug company) won't want to rely on a DTC test before prescribing it, they'll want to run their own test.
2. There's a genetic test to predict effectiveness of a drug without severe side effects. Question: would the drug company prefer that you try their drug for a few months before deciding it doesn't work and switching to a competitor's product, or that you just take a DTC test and skip buying their product altogether? Answer: Make the license fee for the test more expensive than several months worth of the drug, if you license the test to third parties at all.
Here's another fun scenario. Imagine you're a drug company that has an approvable drug finishing up Phase III. A competitor already has a drug on the market for the same condition. You have the option of spending $$$ to find a genetic marker for the rare but severe side effects to your drug, and then submitting that to the FDA as well. If you do, the FDA may well approve your drug with the proviso that everyone who is prescribed the drug must be tested first ... whereas your competitor's drug is under no such restrictions. Given the ethics shown by some companies when it comes to conducting and publishing their clinical trials and the marketing of their drugs, my guess is the test's introduction might well be delayed for a few years.
If the "personal" in "personal genomics" means individual rather than the opposite of societally-sponsored, we should also throw this story into the mix:
which reports on:
Schroth, W. et al (2009)
Association Between CYP2D6 Polymorphisms and Outcomes Among Women With Early Stage Breast Cancer Treated With Tamoxifen
with a tantalising link back to an earlier story from 2006:
FDA Panel Leans Toward Including CYP2D6 Dx in Tamoxifen Label, But is Split on Language
November 15, 2006
By Turna Ray
Science Clinical Pharmacology Subcommittee was split over whether the label should ârecommendâ the genetic test or make it an âoptionâ for health care providers and patients to discuss.
i.e. if the drug is expensive enough, and the consequences of the wrong treatment dramatic enough, then a genetic test becomes just another routine pre-treatment assessment.
Agreed, you would probably want this done in the sort of lab you could sue if they did it wrong, which rules out the DTC caveat emptor types ...
Technical CLIA doesn't apply to for 23andMe. It has no internal lab and as such CMS has decided not to regulate it as a laboratory under CLIA. But LabCorp, which provides lab services to 23andMe is CLIA compliant. The company began working with LabCorp last year. Either way the validity of the algorithms has nothing to do with CLIA compliance.
It's important to put the reported accuracy in context. While on the face of it a rate of 1 in 3000 (or 25000) seems quite good, if one is making 1 million calls (I rounded up to make the math easier) this means that there would be between 40 and over 300 incorrect calls. If these incorrect calls occurred in one of the Ashkenazi BRCA mutations or CYP2C9/VKORC1 for Warfarin dose(23andME) the impact could be significant. Also some alleles are much harder to correctly to genotype than others, so the errors may not be evenly distributed. It suggests to me that for highly penetrant alleles testing outside the chip might be preferable.
The other issue is that proficiency testing and publication of the error rates should be publicly available for review. CLIA certification doesn't require publication of results, and it does not mandate proficiency testing (allows 'alternative methods' which are not tightly defined).
Context is very important with these accuracy numbers. You missed an important factor, however: the distribution of error. If you assume an even error rate across the entire genotyping chip, then yes, 40 errors out of a million could be significant if they landed in important regions like BRCA or CYP2C9.
But that's not the case. Errors are not random. Some probes are better than others. Most of the discordance lies in SNPs that aren't associated with anything, at the moment; probes that have not been scrutinized because they aren't reported on. For things like BRCA, companies like 23andMe take a lot of effort to ensure that their calls are of high accuracy and that people with ambiguity in their data are informed as much.
It was pointed out to me that you actually made my observation in your own post, so my apologies for missing that and erecting a straw man to burn.
The SNPs found in these scans are associative and have no proven bearing on disease outcomes. In other words they are statistical associations garnered from people with certain conditions and certain SNPs. It is better to look at the results as a possible indicator of disease outcomes if bad lifestyle choices have done made. Personal choice is important. Telling someone who has brown eyes they have a high probability of having brown eyes is not much use. Knowing the disease and longevity outcomes of one's extended family is probably more useful than finding out you have SNPs which are associated with Type2 Diabetes.
As a geneticist I don't look at these scans as snake oil. I think the more we know about ourselves, the SNPs included, is all the better. The only problem is the cost, and the lack of cheap Genomic scans.