As an addendum to my previous post on the controversial "longevity genes" study, you should go and check this out. It's a post on the blog of personal genomics company 23andMe, and it's a pretty impressive piece of scientific dissection of the longevity GWAS paper - in addition to detailing a variety of methodological problems with the study, the authors actually used the 23andMe database to look at the predictive value of the longevity GWAS algorithm on their own customers:
We took a preliminary look in our customer data to see if the proposed SNP-based model described in Sebastiani et al. is predictive of exceptional longevity. A commonly used measure of test discrimination is to calculate how often, for a randomly selected case and control, a test correctly assigns a higher score to the case. This is known as the "c statistic" or "area under the curve". The authors of the new study say their model scored a 0.93 for this statistic. But when we compared 134 23andMe customers with age â¥ 95 to more than 50,000 controls, we obtained a test statistic of 0.532, with a 95% confidence interval from 0.485 to 0.579. Using 27 customers with age â¥ 100, we get a value of 0.540, with a 95% confidence interval from 0.434 to 0.645. A random predictor of longevity would give a 0.5 on this scale, so based on our data, performance of this model is not significantly better than random. Even with our small sample size, we can also clearly exclude values as high as the published result of 0.93.
Small numbers, but certainly not good news for Sebastiani et al.
However, people who have already had their genomes analyzed, through services such as 23andMe, will soon be able to predict their risk score through a free website that Perls' collaborator is developing. But Perls hopes to head off commercial efforts to market this kind of test. "We are concerned that the marketing [for such a test] will not mention the shortcomings of the test," says Perls.
No need to worry - 23andMe has now done a massively better job of conveying "the shortcomings of the test" than the study's authors have, and I think we can now safely say that the authors won't need to head off a stampede to commercialise their algorithm.
Yes I thought the 23andme post was really good as well, they DO know their stuff. The only other caveat about their longevity analysis apart from the small numbers is the accuracy of their customer data, I mean would they be able to check for sure that the ages given are the true ages? Probably not given their confidentiality.
It is hilarious though about the authors (pompous) attempt to give away a free test and stymie those nasty capitalist unscrupulous DTC companies. So they were (or still are?) going to give away the interpretation based on just one study with no independent replication and with what we now know to be dodgy data (actually I think that we knew the very next day that it was dodgy didn't we, via Guardian and Jeff Barrett?).
Dammit these academics/clinicians need regulating, I've been saying so for years, told ya!
I agree with the comment about wondering how valid the ages of the 95+ set are.... if accurate, then it certainly is an interesting group of people to live that long yet still be early adopters in purchasing a genetic risk prediction product!
where's dr. s murphy's comment on this blog post? oh - it correctly lauds a DTC company? i see. doubt we will see him here.
BRAVO. 23andSerge has a team of competent scientists who tried to play doctor.
They are obviously better at vetting the data than trying to trick people into using it for medicine.
I say, this is the perfect role for 23andMe. Don't sell medical tests, sell your analysis of studies......
p.s. what the hell is up with all these Dan's?
My grandfather is 95 years old and a customer of 23andme. I can't verify the other 133 individuals though.
It makes sense that people interested in geneology would get their centenarian and near centenarian relatives to spit before they pay for their younger relatives to do the same, so it is not hard to believe that there are quite a few in this age group.
I'm not meaning to doubt the likely veracity of customer supplied data, it's just one of those things that needs to be worked out in order to use the customer base for research. However I'm confident about the ability of 23andme to do so.
Pity about the longevity study - I heard that Prince Charles was about to do the test on his mother...
PS - OK on the 23andme plate mixup reply but I really cannot understand how such sophisticated genotyping equipment could have been designed to allow plate inversion, it's such a simple thing to avoid