In 2007, I wrote a series of posts about what I found to be a fascinating yet at the same time disturbing phenomenon, specifically self-experimentation by cancer patients using an as yet unapproved drug called dichloroacetate. If you'll recall, DCA is a small molecule drug that was used to treat congenital lactic acidosis in children through its inhibition of the enzyme pyruvate dehydrogenase kinase. This inhibition shifts the metabolism of glucose towards oxidative metabolism in the mitochondria and away from glycolysis, the product of which is lactic acid. In January 2007, Dr. Evangelos Michelakis of the University of Alberta had the clever idea of using DCA to target the Warburg effect in tumors and published an article in Cancer Cell describing preclinical experiments in rats in which DCA resulted in marked shrinkage of multiple tumor types. The Warburg effect was first described in 1928 by Otto Warburg and refers to the tendency of many tumors to rely on glycolysis for their energy supply rather than oxidative phosphorylation, even in the presence of oxygen, which in the case of normal cells favors oxidative metabolism.
Because DCA is a small molecule and relatively easy to synthesize, however, a pesticide salesman named Jim Tassano decided that he would sell "pet DCA" to treat cancer in pets. Tassano fooled no one, of course; the real purpose was to take advantage of desperate cancer patients, some of whom flocked to his BuyDCA.com website to purchase what he claimed to be pharmaceutical grade DCA. It also led to a most disturbing phenomenon, namely self-experimentation by desperate cancer patients with DCA and discussion among patients on online forums, which was even likened to "clinical trials," complete with reports of success based on magical thinking more than anything else. This observation led me to take interest in the more general question of the "right" of patient access to experimental drugs versus the negative effect that such unrestricted access would have on clinical trials. Last year, the full D.C. Circuit Court of Appeals reversed an earlier decision by its own three-judge panel and ruled 8-2 against a patient's right to have access to any experimental therapy. I thought things were fairly settled then, but apparently not, as this recent story points out:
Until last year, Alan Felzer was an energetic engineering professor who took the stairs to his classes two steps at a time. Now the 64-year-old grandfather sits strapped to a wheelchair, able to move little but his left hand, his voice a near-whisper.Felzer suffers from ALS, also known as Lou Gehrig's disease. The fatal neurological disorder steals the body's ability to move, speak and ultimately to breathe. But rather than succumb to despair along with his illness, Felzer turned to the Web to become his own medical researcher -- and his own guinea pig.
Dozens of ALS patients are testing treatments on their own without waiting on the slow pace of medical research. They are part of an emerging group of patients willing to share intimate health details on the Web in hopes of making their own medical discoveries.
ALS is a horrible disease. Indeed, it's arguably more horrible than most forms of cancer, robbing, as it does, people of the ability to control their bodies. It's fatal, and before it kills it renders its victims helplessly dependent on others before it takes away their ability to breathe. Although a minority of ALS victims can live a long time with the disease (Steven Hawking, for instance), the majority succumb within five years of the onset of symptoms. If ever there was a disease that understandably spurs people to take desperate measures to slow or arrest its course, ALS is it. This group testing lithium for ALS is no different than the desperate cancer patients trying out DCA, and it's subject to the same shortcomings and opportunities for bias and self-deception that were so apparent on Jim Tassano's DCA discussion boards.
Just as Tassano sparked "wild experimentation" of patients using DCA to treat cancer with little or no medical supervision based on Michelakis' single study using rat tumor models, patient experimentation with lithium to treat ALS appears to have been sparked by an Italian study that reported a significant slowing of the progression of ALS. This study looked at both a mouse model and reported the results of a small randomized trial of lithium plus riluzole (riluzole is the only drug approved by the FDA to treat ALS) versus riluzole alone. As a result of this study, ALS patients are taking lithium off-label:
Felzer began taking lithium in January, and his scientifically minded family reached out to other ALS patients. "All those people are taking it anyway," said Alan Felzer, whose smile remains bright and his gaze sharp even as the rest of his body fails him. "So it only made sense to keep track of what was happening."
The task of leading the ALS-lithium project fell to Felzer's daughter, Karen, a U.S. Geological Survey earthquake researcher. Her partner in the effort was Humberto Macedo, a 42-year-old computer systems analyst, father of six and ALS patient in Brasilia, Brazil.
The study grew naturally out of the strong reliance of ALS patients on one another for information, Macedo said.
Working online, Karen Felzer and Macedo recruited nearly 200 patients worldwide to take a specific lithium dosage and answer standard surveys to gauge their symptoms. They began running their study through a Web site called PatientsLikeMe.com, using it to attract volunteers and track their progress.
Sound familiar? This is simply a somewhat more organized method of doing the same thing that patients did on Jim Tassano's discussion boards. Heck, Tassano even tried to set up a "clinical research" protocol that was a travesty. The discussion boards on PatientsLikeMe.com are not quite so bad (it would be hard for anyone's studies to be more poorly designed or self-deluding), but I can't help but think that the designers of this project are fooling themselves if they think an unblinded, patient-driven clinical trial will accomplish what they think it will, although they do exhibit appropriate caution:
History also teaches us that patients sharing stories with each other will not answer the question alone. Chinese stem cells, herbal supplements, nutraceuticals -- all have been discussed extensively on the internet with some claiming cures and some describing great harm; yet we have no definitive answer.
Except that in many cases we do have definitive answers, and even where we don't scientific plausibility (specifically the lack of scientific plausibility) argues strongly against the utility many of these therapies. However well-intentioned Humberto Macedo and Karen Felzer, the organizers of this effort, may be, they clearly don't understand why unrandomized, uncontrolled trials are so prone to bias and false positives. After all, what they are doing is nothing more than the collection of anecdotes, and, as I like to say, the plural of "anecdote" is not data. Granted, that's a bit of an overstatement in some cases, but not by much. The reason is that this survey doesn't even rise to the level of a case series, given that it relies only on Internet reporting rather than the actual direct assessment of the patient's condition by a physician or other health care professional:
Together, with all the patients involved, we will run the first real-time, real-world, open and non-blinded, patient-driven trial. We believe we will have the power, within months, to begin answering the question of how much lithium modifies the progression of ALS. Unlike a blind placebo control trial, we are watching the use of this drug in the real world, and because of the number of patients and our system's sophisticated data modeling, we can determine the significance of each reported change in each patient as he/she deviates from his/her predicted course. There are many risks to our approach, patient optimism, the placebo effect, uncertain quality, and many other variables will compromise our data. Despite these, and many other challenges, we remain committed to solving this problem.
The problem, unfortunately, is that this "problem," as it is described, probably cannot be solved because it is inherent in the very design of this study, which is not randomized, not controlled, and not blinded. There are no controls, only in essence historical controls (i.e., the "predicted course"). What's even more puzzling is that the organizers of this trial even seem to recognize these problems, acknowledging that the placebo effect, patient optimism, and many other variables may compromise the data. Yet they nonetheless express optimism that their "sophisticated data modeling" will overcome these problems. How? They don't answer. They don't seem to realize, either, that what they are doing is not new. There's a reason why randomized, double-blind clinical trials using either placebo or the standard of care as the control are viewed as the strongest form of clinical evidence for the efficacy of a treatment, and it's because it became appreciated that non-blinded and non-randomized trials are prone to all sorts of biases. They just don't give answers reliably unless the superiority of one therapy is so much greater than that of the control that it's undeniable and detectable after just a few patients treated.
So what's the harm? This is, of course, not an unreasonable question for patients faced with a fatal disease like metastatic cancer or ALS. As hard as it is to believe, even for patients with ALS it is possible to make things worse. Lithium, for instance, is not a benign drug. It can cause neurological complications and diabetes insipidus. It also requires monitoring to achieve therapeutic serum lithium levels. In the case of DCA for cancer, the drug could cause neuropathy and other complications. Overall, when taken as a whole, it is far more likely that patients will be harmed by taking experimental or off-label drugs than significantly helped. Worse, in the "real world" case of most drugs, effects that are not as dramatic as an antibiotic curing a urinary tract infection, it's difficult, if not impossible, using methodology like that of PatientsLikeMe.com to detect an actual therapeutic effect:
Heywood too hoped that lithium was the breakthrough he and others had been seeking.
But after six months, none of the 87 people who stuck with the study showed any letup in the disease's progress, said Karen Felzer. She now doubts the Italian study's results.
"It's obvious to everyone it's not the miracle drug we thought at first," she said. She also thinks other tests of lithium for ALS should be halted to spare patients the drug's possible side effects, such as tremors, weakness and difficulty breathing. Her father stopped taking the drug, though Macedo is continuing.
However, other reseachers say professional lithium studies should go forward. Dr. Merit Cudkowicz, a Harvard Medical School professor, is set to begin one in December with 84 patients. Her study will stick to the so-called gold standard of research, in which each patient will be randomly chosen to take the drug in question or a placebo. Neither patients or researchers will know who got the drug to avoid introducing bias.
Because the patient-led lithium study lacked those tight controls, it is unreliable as a measure of safety and effectiveness, Cudkowicz said. With an incurable disease, she said, "You don't want to be throwing something away that works because of a bad study."
Exactly right. Just because a badly designed study failed to find a result does not mean that lithium is not an effective treatment for ALS. It might well be. Unfortunately, the PatientsLikeMe.com study was (and is) a bad study, which by its very design was highly unlikely to find any but the most dramatic treatment effect from lithium. The same was true of the sharing of anecdotes about DCA for cancer. The only way Jim Tassano would have found a treatment effect for DCA is if it were truly the "miracle" drug that patients hoped it to be. Anything less, and unrandomized, unblinded patient-driven studies don't have the reliability or sensitivity to detect differences between the experimental and control groups--or worse, they are prone to bias that may lead to a false positive. As understandable as it may be that patients want to do something, these efforts are more likely to mislead than enlighten.
There will always be a conflict between wanting to do something now for suffering patients, damn the consequences, and following the scientific method to demonstrate efficacy and safety. It's perfectly understandable and the need of dying patients is especially compelling. We have seen both extremes. Indeed, until 1906, drug sellers could make essentially any claim and sell essentially anything to the public as a drug without regulation. We all know how well that worked out. Early in the history of the FDA, companies often tested new drugs by sending them to doctors to offer to their patients, asked for little information regarding side effects and complications, and had no standard criteria for efficacy. Efforts of groups like PatientsLikeMe.com may be an improvement over a century ago in that they do go to great efforts to ask patients about side effects and changes they note in how they feel, but, boiled down to its essence, this patient-driven trial is the same thing as the unscientific experimentation of that era. It benefits neither science nor patients.
- Log in to post comments
I understand their desperation, but they aren't really helping themselves or others. Labeling this collection of anecdotes a study is, in my opinion, far more likely to cloud the issue than provide any useful knowledge.
Having seen a relative succumb to ALS within 2 years - I'd try that too, and science be damned. If you got zero prognosis, and knowing there is nothing in the pipeline to change it, all you have is miracle drugs. Sure, it's a 1 in a million shot, but if your scientific shot is zero, what do you have to lose? The pure prospect of ending ones life as a diapered corpse that's still breathing is terrible enough to grasp for anything.
I too understand their desperation, and the issue surrounding the flaws inherent in the approach. However, I wonder is there a way to turn such conviction, and resources into the "gold standard", or if not, at least into something worthy of calling "data"? It would be a shame to throw away the conviction these online support groups have, especially when they seem eager to progress science, even if it isn't with a worthwhile method. I don't think science has to be done in a university lab*, and it really is all about the method. Any thoughts?
* - Nor do I think Orac, or anyone else here believes this.
I'm not arguing against the scientific approach here - actually, I might. My question is - can I ethically do a classic trial here - in which I know that the half that lands on the placebo side WILL die horribly in two years? We argue against the Gonzales's of this world, but would still do a DBRCT when we know that the control group is SOOL? If the new treatment is so marginal that the RCT is needed to show effectiveness - the treatment isn't doing much anyway in the first place.
With something as terrible as ALS I might well be tempted to participate in a "trial" like this. I recognize the drawbacks all too well (including the potential for discouraging real research and the enrichment of quacks who take advantage of other people's suffering).
Another example of support group "empowerment"/encouragement of quackery - the use of crude oleander extracts ("soup") for treatment of cancer, AIDS, MS and various other ailments:
http://www.disabled-world.com/artman/publish/oleander-hiv.shtml
"I don't think science has to be done in a university lab*, and it really is all about the method. Any thoughts?
* - Nor do I think Orac, or anyone else here believes this."
Yes, science is heavily about the method, which means that real science is done in a "lab" (i.e. by qualified persons under controlled conditions). Observations and anecdotes may eventually lead to significant discoveries, but by themselves are only starting points, and generally misleading ones at that.
"Yes, science is heavily about the method, which means that real science is done in a "lab" (i.e. by qualified persons under controlled conditions). Observations and anecdotes may eventually lead to significant discoveries, but by themselves are only starting points, and generally misleading ones at that."
I understand the need for controls, and lab work, and was not suggesting we rid ourselves of this. I was asking if there was anyway of turning the current study, which is probably doomed to fail, into something worthwhile for science. I disagree that "real science", whatever that might be, requires a lab. For example, making the current study into a collection of clinical series, as opposed to a collection of anecdotes. Clinical series are of value to the scientific community, and don't require a laboratory. They do require physician evaluation, and medical records.
Another example is the science of archaeology, which does much of it's data collection in the field using careful procedures, and sophisticated tools. To claim that "real science" is done in the lab is patently absurd.
I think his point is that the lab is the venue where experimental research takes place, regardless of whether that place is a cleanroom in a university, a wing of a research hospital, or even the great wide world.
Archeology isn't an experimental science, so it's not directly comparable, but it does have an analogy: the tight controls of a dig site. You don't want to contaminate the data, so you don't just let anybody in to trample all over the dig, now do you?
Mu, Your choice is a false dichotomy. The range is not simply a horrible death in 2 years or perhaps relief, but also includes a faster death, additional severe side effects, drug interactions, etc.
Calli,
That may be his point, but that misses my point, or rather question. Is there a way that internet communities that are desperate for a treatment, or cure, can be turned into sources of data, as opposed to anecdotes? I take it your answer is "no".
Anecdotal evidence has potential for highlighting a treatment as worth looking at in detail. But since lithium is already being studied in proper trials, even that is not possible here.
So no, there is no realistic chance of any worthwhile data coming out of this. What anecdotes can do, has already been done.
Robster, if you can think of side effects worse than advancing ALS ...
BTW, the case I knew would have probably taken "faster death" as a positive outcome of the medication (and yes, as a reader of the blog I'm completely aware of the value of anecdotal evidence).
I find it ironic that in many states you can now legally get lethal doses of drugs to commit suicide, but at the same time experimental treatments at the same stage are still off-limits.
I'm left wondering if there isn't a best-of-all-worlds solution here. Imagine a centralized distribution system in which
1) Volunteers suffering from contion C submit vast pools of personal data
2) They are initially (round 1) sent un-marked medication which initially contains their current prescription dosage D1 of some medicine Y (the current best-known treatment for C) and 0 mg of X (a freshly approved-for-human-trial alternative to X).
3) Using sophisticated data-mining techniques, a computer sifts through the personal data of #1 and determines a round-2 batch a mix of X+Y for each individual (where either X or Y may b absent). Nobody but the computer knows what is in each round received by each participant.
4) At the end of a given round (say, on a weekly basis), all participants get check-ups (or just surveys), the results of which are fed back into the computer.
5) The computer combines personal info + all meds given so far + all results so far to compute a round 3 for each individual, with the intention to maximize the probability of positive results and minimize side-effects for the entire pool.
My experience with data modeling leads me to believe that such an approach would have a genetic-algorithm ability to develop a optimal function that maps [vast-pool-of-personal-info] -> [optimal combination of X+Y to treat condition C]. If X is non-effective (or negatively impacts Y), it will quickly be weeded out of all dosages sent to the volunteer population. However, perhaps it is effective in combination with Y in women, and best when used alone in hypertensive people, or works best at 1 2:1 ratio with Y in general, etc. Any of the above pieces of information would likely never be discovered with basic DBRCT's (or would require massive pools of volunteers and meta-analyses from massive numbers of trials).
Is there any work on developing such a Genetic Algorith approach to testing drugs/dosages - or at least testing something like it to see if it could rapidly optimize dosages in mouse trials?
"So no, there is no realistic chance of any worthwhile data coming out of this. What anecdotes can do, has already been done."
Out of this alleged "study"? I agree, and have said twice already. What about out of the internet community that has decided to collect these anecdotes? Is there a way to put them on track, so to speak, get them to churn out meaningful data? Is this already done elsewhere, or is it not worth a researcher's time?
Some people seem to be conflating two (or more) very different issues here.
On the one hand, there is the question of whether people should be allowed to try a drug like lithium (which is FDA-approved) for something like ALS. This is currently allowed (as "off-label" use of lithium) and is a question that patients and their doctors can wrestle with in private.
The other issue is whether there is any benefit in doing (or participating in) poorly-designed, "patient led" studies.
The problem is that the people in the first case (patient and doctor wrestling with the question of whether to use lithium to treat ALS) could really use some legitimate data to inform their decision. And the information that comes out of poorly-done studies of any stripe is not "data" - it is pooled anecdote.
I've seen this sort of thing come and go through many different medical issues. Orac mentions DCA for cancer, but I can also remember when massive doses of steroids for spinal cord trauma were thought to be helpful, based on a few case reports and some animal studies. That particular treatment came very close to becoming "standard of care", before a large multi-centre study found that it was no better than placebo (if I remember correctly).
The information from any sort of uncontrolled and unblinded clinical study - even a good one - is likely to overestimate the benefits of a treatment. This has been seen on numerous occasions, but one that I know best is the use of secretin for autism. Early, less rigorous studies found a benefit, but more careful study revealed it was no better than placebo.
I have no problem with people using lithium to try to treat their ALS, but I think it would be folly to try to use that "data" to evaluate the treatment's effectiveness. This is especially true given the unpredictable speed of ALS prgression.
It's important to keep the two issues - "compassionate" (or "desperation") use of a drug "off-label" and research into the treatment's efficacy - separate. As Orac pointed out, a poorly-done study of 87 patients using lithium for ALS that showed no effect has neither shown that lithium "works" or that it "doesn't work" - it was a complete waste of effort.
And if a poorly-done study shows that lithium "works" and a better study shows that it "doesn't work", which study will ALS sufferers cling to?
There is no medical intervention that has zero potential for adverse effect. Even homeopathy - which is the very definition of a placebo - costs money and may delay a patient seeking effective treatment - or even effective palliation.
Lithium has long been known to be a difficult and dangerous drug to work with, so the potential to make ALS patients worse should not be discounted so readily. ALS is, without a doubt, a dreadful disease, but it is always possible to make a patient worse through ill-conceived intervention.
Prometheus
Other than them volunteering for the real trial, no.
Is there a way that internet communities that are desperate for a treatment, or cure, can be turned into sources of data, as opposed to anecdotes
Probably. But just asking them to submit their personal experiences isn't going to do it. To go back to your archeology analogy, that would be like inviting armchair archeologists and Indiana Jones fans in to take a whack at the dig site without any training or controls or anything. "Hey look! I found this potsherd!" But without knowing exactly where, how it was lying, and without carefully transferring it instead of handling it like crazy, a lot of important data will be lost. Compare artifacts uncovered by an archeological team to artifacts uncovered from the local flea market, where looters have been selling them. The ones in the flea market are nearly useless from an archeological standpoint. Not completely useless, like collections of anecdotes from ALS patients on lithium, but you won't get anywhere near as much information as if the artifact had been removed with proper controls and documentation.
The Internet might be a good source for recruiting people to studies. And certainly it could be used for communication. But the study has to be done with proper controls from the start, no matter what the medium used to share information among patients and investigators. And the patients should DEFINITELY not be the investigators themselves. Far too much potential for bias.
ooh ooh i think i disagree with orac a bit, this is so exciting.
patientslikeme is a really interesting project, collecting a mass of data that is by its very nature flawed, but open to all kinds of really interesting eg free text analysis to spot side effects, and so on. some of the data is slightly systematised, too, in a really interesting way. by its nature it's also messy. that's observational epidemiology for you.
what sets patientslikeme apart from yer average poor quality quack trial, from what i've seen of their self-presentation, is that they are absolutely clear about the limitations of their data, which is crucial. and there are occasions where it might be reasonable (or at least the only practical option on the road at the time) to do an unrandomised open label trial. the crucial thing is, you present it as such, and speculative, and are clear about the flaws. i reckon they've been pretty clear about that. but i might be wrong.
for hypothesis generating i reckon it's an interesting and i'm almost tempted to say exciting project. by v weird coincidence i was reading about it again earlier today, having met http://en.wikipedia.org/wiki/James_Heywood at scifoo earlier this year, and was wondering if they might need some cheery help analysing their mass of data, when i discovered that a pretty solid scientist from the Institute of Psychiatry in London has gone to work for them (guy called wicks, we've chatted on email but i don't think we've met). i don't know if that's a declaration of interest or a recommendation - i guess i'm defending them because because i think the key is that this is clearly flagged as a new and odd way of collecting data.
unless they're overstating their case somewhere? in which case, heh, ATTACK etc x
Mu, from the known side effects of lithium, I'd say risk of convulsions is pretty bad with an overdose. Very common side effects include nausea, which would compound the symptoms of hyperactive gag reflex and difficulty swallowing. Lithium's associated muscle twitches could also confuse the signs of disease progression.
One of the signs of lithium overdose is slurred speech. Another is loss of appetite. Would a patient be able to tell a sign of lithium overdose and an ALS symptom, even with proper briefing and physician observation that a clinical trial could offer?
Information about the Italian studies will have been spreading among ALS sufferers and their carers. My naive reading of the study is that it shows a 100% reduction of death rate after 9 and 15 months, which will have sounded really exciting. With conditions like ALS, there is a lot of social and internal urge to try to "do something". If this collection of an anecdote set has contradicted the lithium miracle story, it has taken pressure off people with early ALS, which is a good thing. I'd call that a success. We need to wait for the high quality trials to show whether and how lithium can be used as a meaningful treatment, but we now know it's no miracle, and off-label use is not terribly helpful.
If they promote Lithium use, they're doing the wrong thing, but the above sounds like they're doing the right thing - collecting and analysing information from people who have decided to try it.
Disclosure: My beloved father in law died of ALS in 2000. I helped a little in nursing him.
It's okay for the chemical companies to keep dumping chemicals into our bodies via air, water, and food (see yesterdays USA today on air pollutants around schools), but its not okay for a patient to try to save his life using chemicals, even when he has no hope of living. Such a ridiculous thread this Orac keeps reviving. Lets see how he reacts if/when his doctor delivers His death sentence via disease. If it happens, thank your local chemical company, oil company, coal utility, farmer, banker, and all the folks who made it happen, including your useless EPA.
I saw two things recently the bugged me in a similar manner. One was that lithium story the Boston Globe:
Through website, patients creating own drug studies
http://www.boston.com/news/local/articles/2008/11/16/through_website_pa…
Soon after I was arguing with an anti-vax crank on another board. She was telling me how it was _democratic_ for people to be involved in this process. It struck me as another goal-post-mover like the "green our vaccine" crap. She was flogging "A place at the table" and "citizen science" stuff.
There are valid places for community input and citizen observations and such. But this could get dangerously twisted by the woo-sters.