In which Orac gets even more "shrill and brutish" about chelation therapy and TACT

If there's one thing that a certain subset of people who view themselves as reasonable and science-based don't like, it's harshness: Harshness in criticism, harshness in discussion, or—horror of horrors!—anything they view as "incivility." That's all well and good as far as it goes, but the problem is that sometimes there are things that demand a harsh response because they are just that bad. For instance, when the government spends $30 million on a clinical trial to test a wildly implausible treatment that is not without risks for no good scientific reason and no real reason other than that quacks are using it now, then I find that to be a good reason to be harsh, particularly when the therapy used is not without risk.

Apparently, Orac is "shrill and brutish" for saying so in no uncertain terms. At least, so sayeth eminent Yale cardiologist Dr. Harlan Krumholz in a post on his Forbes blog yesterday. Apparently, he was quite offended by my criticism of his unfortunate comments about the results of the Trial To Assess Chelation Therapy (TACT), which were reported in JAMA a couple of weeks ago. Sadly, Dr. Krumholz is quite misguided in his assessment, as I pointed out before. However, his characterization of my original post is not my main concern, although I will address it more towards the end of the post. First, let's deal with what's most important: TACT itself.

What caught my attention more than Dr. Krumholz's concern trolling and castigation of the critics of TACT (and me in particular) for supposedly being so very, very mean and nasty is his discussion of an interview on Cardioexchange in which Gervasio Lamas and the other TACT investigators respond to the copious and, in my not-so-humble opinion, well-deserved criticism of TACT. It is laden with straw men and other logical fallacies, plus unconvincing defenses of the numerous problems with TACT that were so well documented by Atwood et al in 2008. In fact, I find it far more useful to deconstruct the disingenuous and self-serving defense of TACT than to worry overmuch about the concern trolling being flung about by someone like Dr. Krumholz, who from his article (in which he points out that he knows Dr. Lamas and assures us that all the main investigators for TACT are awesome clinical trialists) appears more than anything else to be really ticked off that his good buddies are being criticized. How dare we peons say such mean and nasty things about his good buddy, who, he assures us, is really and truly a total dude? I'll explain why, and I can't think of a better starting point than Dr. Lamas' answers to TACT critics, which is basically a fawning puff piece in which—you guessed it!—Dr. Krumholz himself lobs softball interview questions at Dr. Lamas and company. Sadly, given that the questions Dr. Krumholz served up were more akin to batting practice pitches than anything challenging, you'd have expected that Dr. Lamas could have hit every one of them out of the ballpark. Instead, what we get are at best inconclusive foul balls devoid of illumination and mostly resting on the very same shortcomings of evidence-based medicine as opposed to science-based medicine that I've been discussing for years now.

It turns out that some of the questions are rehashes of issues that I've already discussed in detail before; so I will in essence "cherry pick" the answers that interest me now and refer you to my previous posts about TACT for criticisms of other responses by Dr. Lamas. For instance, the very first question is about statistical significance and how the results of TACT just barely reached statistical significance, the criteria for which were changed based on the enrollment. I discussed that issue in my original post on the presentation of the first results of TACT back in November, and, quite frankly, it wasn't much of an issue for me other than that the ultimate result of the trial was statistically significant only by the barest whisker, with p=0.035 and the predetermined level of statistical significance being 0.036. Indeed, I don't really even care if there was diddling with the statistical significance threshold; I'll even accept Lamas' response at face value. Think of it as a freebie. It changes nothing; it's not even the worst thing about the trial, although I can't resist mentioning again that this barely statistically significant result appears to have been driven entirely by a seemingly beneficial result in two subgroups: diabetics and those with anterior MIs. There were no statistically significant differences between any of the other subgroups. Nada. Zip. To repeat: A proper discussion of the trial would emphasize that. But let's move on, as I've discussed that issue in detail before.

One of the main complaints about TACT was that the endpoint of the study was a composite endpoint that included "soft" outcomes. In other words, instead of looking at the effect of chelation therapy on death, MI, stroke, coronary revascularization, or hospitalization for angine individually, the investigators created an aggregate outcome for "major adverse cardiovascular events" which included all of these outcomes lumped together. Now here's Dr. Lamas' explanation:

The primary endpoint consisted of time to the first occurrence of the following major adverse cardiovascular events: death, MI, stroke, coronary revascularization, or hospitalization for angina. Combined endpoints are used when the most important endpoint to provide proof of efficacy, all-cause mortality, is likely to occur so seldom that an impractical and unaffordable number of patients and patient-years of follow-up would be needed to have enough statistical power to detect a difference between groups. In a stable coronary disease population, the incidence of mortality is low, so a combined primary endpoint is necessary for most studies. As we were planning the study, we selected a population very similar to that of the WIZARD study, which used a primary endpoint similar to ours. Other studies with similar combined endpoints, particularly including revascularization, are JUPITER and ACCELERATE.

Once again, composite endpoints are not in general a good thing. There's but they can be made less bad, so to speak, by not including subjective endpoints like coronary revascularization and hospitalization, both of which are subject to a great deal of clinical judgment in deciding who requires them. More problematic is the variation in usage of such interventions that has nothing to do with whether the treatment works or not. For instance, in the state of Michigan, there is up to a 2.4-fold variation in rates of percutaneous coronary interventions (PCI; i.e., angioplasty) between the highest use and lowest use areas. That's just one state. Similar studies have shown wide variability in coronary revascularization rates just on geography alone. Adding such a variable to a composite endpoint is thus a bad idea on a scientific basis alone. At best, it could add unnecessary variability to the composite outcome measure for no useful benefit; at worse it could add significant bias, particularly if subjects being treated in academic centers, where patients are more likely to be referred for appropriate coronary revascularization interventions were in areas with significantly different PCI usage rates than areas where subjects being treated in "complementary and alternative medicine" (CAM) centers, where one might reasonably expect that referral for revascularization might be a bit less—shall we say?—expeditious. I could see a composite endpoint in which the components were all "harder" endpoints, such as the triple endpoint of death, nonfatal MI, and nonfatal stroke, but even such endpoints are not without problems.

Let's just put it this way. The excuses made for lumping the "hard endpoints" with the more subjective endpoints are just not that convincing. For example, if, as Dr. Sanjay Kaul argues, "the inclusion of subjective endpoints like CV hospitalization and revascularization increases the potential for ascertainment error and misclassification that typically biases the results towards the null," then it was stupid (oh, dear, there I go being "shrill and brutish" again!) of Dr. Lamas to include such subjective endpoints in his composite outcome measure because by doing so he would have decreased the chances of a $30 million trial under heavy fire of producing a positive result and provided opponents of the trial a convenient weapon to use against it, to boot. Whatever I think of Dr. Lamas' decision to take on TACT, I don't think he's stupid. I think he probably thought that using a composite endpoint would decrease the sample size necessary to achieve the desired statistical power, which is the reason some cardiovascular investigators use such composite endpoints.

Oh wait. There's no "probably" about it. That's exactly what Dr. Lamas said in his justification for using the composite endpoint!

Unfortunately, such endpoints are fraught with problems. A sampling of articles describing such problems in detail can be found here, here and here. In fact, if you start typing "problems with composite endpoints" into Google, it will autocomplete with "in cardiovascular studies." To preempt the likely criticism, I am only mentioning this as an indication that there are lots of problems with composite endpoints in cardiovascular trials, to the point where people search for the term a lot. Besides, the articles I referenced more than indicate that such composite endpoints have major problems. Let's just put it this way, a common retort to criticisms of such endpoints is that there have been lots of major cardiovascular trials that used such endpoints. If that's true, then the state of cardiovascular research is not as good from a scientific standpoint as I had previously believed. Indeed, the responses to criticisms of the TACT trial have given me an education in this area, and I don't mean that in a good way.

Another aspect of TACT that we skeptics have always found problematic is the inclusion of heparin and other compounds in the chelation solution but not in the control, which led to this exchange between Dr. Krumholz and the TACT crew:

A major comment concerns the components of the chelation therapy, specifically about the inclusion of procaine and heparin and their possible effects on cardiovascular outcomes. Why were they included, and do you think they had an effect?

The NIH RFA to which we responded had gone through NHLBI and NCCAM Councils and called for a definitive trial of EDTA chelation therapy, as it was currently being implemented in clinical practice. When we looked into the infusions, we found that they included many compounds, not just EDTA. Therefore, to achieve a result that would be consistent with how chelation therapy is actually used in practice, we chose to mimic precisely the most prevalent infusion in use. We have no reason to believe that a small amount of procaine or 2500 U of unfractionated heparin once weekly would affect outcomes to the extent that we found in TACT.

Another comment is that the placebo solution contained 1.2% glucose in order to match the osmolarities of the control and experimental solutions. Some people think that might have contributed to worse outcomes in the control group. What is your view of that possibility? What options did you consider for the placebo infusion?

We wanted to keep the placebo solution as simple as possible and not introduce an unexpected risk or benefit. Normal saline fit the bill, as we excluded patients who had active heart failure. The amount of glucose in 500 mL of 1.2% is 1.2 grams X 5 = 6.0 grams of glucose weekly. It is not plausible that this would lead to a greater coronary risk in diabetic patients. We also considered other iso-osmolar solutions such as D5W but felt these would have introduced a greater sugar load. Another option would have been a solution that encompassed all the ingredients except EDTA. However, as stated above, our intent was to investigate a treatment currently in use and determine whether it was safe and effective, rather than deconstruct the solution. The present design allows us to draw those conclusions.

I'm particularly amused by the statement that they have "no reason to believe" that the heparin or procaine would affect the outcomes. The reason is that Lamas and colleagues really had "no reason to believe" that chelation therapy would have any beneficial effect, but that didn't stop them from spending $30 million of taxpayer money to do this trial and expose hundreds of patients to potential danger for damned near no chance of benefit. There are also other differences, too, that Dr. Lamas doesn't mention, such as magnesium chloride (2 g) and vitamin C (7 g). The placebo solution consisted of 1.2% dextrose in normal saline. Now, the idea is that it's the chelation that matters; yet the study design was incapable of testing whether that was true. Yes, I know that the high dose vitamins and minerals were also tested, but there was really no reason to think that they would be beneficial either, and lots of reason to think that in some patients they might be harmful. In any case, if you're going to test whether chelation therapy does any good, then test whether chelation therapy does any good. TACT was designed in such a way that it can't answer that question.

As for the bit about wanting to test chelation therapy "as it's done in the community," that's one of the lamest excuses I've ever heard. it really is. Think about it. Don't you want to test the idea behind the therapy? Even if I accepted the results of this trial as likely to be valid, I'd be infuriated that the trial was done in such a way that it can't answer the question of whether chelation therapy has any biological validity as a treatment for atherosclerotic cardiovascular disease.

I suppose I'm just being "shrill and brutish" again.

Come to think of it, so is Dr. Lamas:

Much of the criticism and controversy surrounding TACT was originated by several small, self-appointed groups with a history of aggressive opposition to CAM practices and to TACT in particular, as they feel that 1) CAM practices should not be studied; 2) if studied and the results are negative, the trial should be condemned as a waste of money; and 3) if studied and the results are positive, the trial should be condemned as incompetently and unethically carried out, and the results discounted. Obviously, TACT has received the 3rd response. We ask our cardiology colleagues to look at the study critically, without emotion, and ask themselves if they would feel the same way about our results if the words EDTA chelation never appeared, and instead “stem cell” or “new statin” appeared. We believe that the debate should focus on the unexpected biological activity of chelation therapy rather than on spurious allegations that TACT investigators were involved in willful wrongdoing that affected the results of the trial.

To which TACT defender (or at least someone who argues that TACT isn't as bad as all we "shrill" critics say) Dr. Sanjay Kaul adds:

I agree with the notion that there is clearly a double standard when it comes to accepting the results of trials evaluating so-called dubious quack cures such as chelation versus trials assessing promising de rigueur cures such as gene transfer or stem cell therapy.

Actually, to Dr. Lamas, Dr. Krumholz, and Dr. Kaul, I would say: Yes. You might not believe me, but, yes I would feel the same way about the results. In fact, let me throw the question back at Dr. Lamas and Dr. Kaul in a manner relevant to TACT: Would you feel so satisfied with the results of a study of, say, stem cells that produced the same results as TACT? Now, imagine: Would you take such results seriously enough to think that there might be something there and to advocate more study, if you knew that many of the centers where the trial was carried out make their living selling stem cell quack treatments like this one or this one, along with all manner of other quackery, from homeopathy to antiaging quackery, to just about anything you can imagine that's pseudoscientific medicine? You think that's unfair? No, it's not, because that's exactly what many of the "CAM" treatments administered at these centers are like. I refer you back to my description of the quack centers recruited to participate in the clinical trial, as summarized by Dr. R. W. Donnell and his Magical Mystery Tour of NCCAM Chelation Study Sites (part 1, part 2, part 3, part 4, part 5, part 6, part 7). Many of the study sites are highly dubious clinics touting highly dubious therapies, including heavy metal analysis for chronic fatigue, intravenous infusions of vitamins and minerals, antiaging therapies, assessment of hormone status by saliva testing, and many, many more forms of quackery.

If I'm ever feeling particularly shrill, I might go through the entire list of TACT sites and see what other quack therapies many of them offer. Alas, I'm too tired tonight as I write this, and this post is already too long. It would probably take several hours to do.

Recall that Dr. Krumholz characterized my post on the TACT results as "shrill and brutish." Dr. Krumholz proclaims himself to be so very, very offended at the "shrill and brutish" tone of the criticism of TACT, wanting to "steer the public dialogue back to the science," and be a champion of "respectful and civil" dialogue. Excellent! I applaud him on his resolve and hope that he actually goes through with this, but—whoops!—Dr. Krumholz basically said the same thing as Dr. Lamas but in a less "shrill" fashion in his original post on TACT; so I suppose he agrees with Dr. Lamas. He also links approvingly to an interview he did with his good buddy Dr. Lamas in which he accuses TACT critics of, in essence, hypocrisy. Apparently it's just fine to be—dare I say?—shrill and brutish in attacking one's critics, as long as one is a friend. Tribalism? Oh, yes. That's definitely what I'm detecting here.

But let's get back to what Dr. Lamas is saying. Do you detect any straw men here? I detect at least two. Number three is certainly a straw man in that we are not just reflexively dismissing the results of TACT because we think chelation therapy is quackery (although it is). Yet that is exactly what Dr. Lamas is more than implying. Indeed, when his criticism is boiled down to its essence, Dr. Lamas is basically calling TACT critics dishonest and hypocritical. (How "shrill"! Funny how Dr. Krumholz seems not at all "offended" by that.) We are criticizing TACT because we honestly believe that it was bad science and unethical, just as, I expect, Dr. Krumholz honestly believes TACT isn't so bad to the point where he thinks a followup study is mandated.

Number one is also a straw man, indeed a straw man that Dr. Kaul probably doesn't realize is beloved of cranks and quacks of all stripes. None of us say that CAM shouldn't be studied. What we say is that practices that are so implausible as to be incredibly unlikely to yield informative results are not worth doing and potentially unethical. These include treatments whose proposed mechanism violates scientific principles that rest on far more solid evidence than equivocal, bias-and-error-prone clinical trials. These include energy medicine, claims based on nonexistent anatomic structures (such as reflexology, iridology, and chiropractic subluxations), claims that violate multiple laws of physics (for instance, homeopathy) and and claims based on nonexistent physiological functions. Now, it's actually a fairly interesting question (to me, at least) where chelation therapy stands in this order of pseudoscience. Even defenders of Dr. Lamas, such as Dr. Sanjay Kaul, are quick to say that they "do not endorse" chelation therapy for cardiovascular disease. Then they go right ahead and defend the trial.

In fact, Dr. Kaul exhibits one of the key problems with TACT defenders, and that's a misunderstanding of what it is that we are actually saying. For instance, he argues:

First, let me state there are several examples in medicine that have shone the spotlight on the slippery slope argument of plausibility (antioxidant vitamins, hormone replacement therapy, magnesium for MI, anti-reperfusion injury therapy come to mind). Thomas Huxley said it best, “the tragedy of science, the slaying of a beautiful (plausible) hypothesis by ugly facts!”

Ah, yes, it's a variant of the "science was wrong before!" gambit. I know that Dr. Kaul doesn't realize that he's using a favorite argument of cranks and quacks, but he is, unfortunately. As a breast cancer surgeon, I happen to know a bit about hormone replacement therapy, and the problem with HRT was far more that it was adopted prematurely without adequate study. (Rather like chelation therapy by quacks, actually.) It's also a lot more complicated than "hormone replacement therapy was considered good but is really bad." In any case, Dr. Kaul clearly misunderstands plausibility in the way we argue it, because he then goes on to say:

Second, the reason for the low prior probability of benefit assertion is not clear to me. The available evidence for treatment effects of chelation therapy in patients with coronary heart disease or peripheral arterial disease is mixed, and limited to case series and three small trials evaluating surrogate endpoints. No outcome data derived from properly designed clinical trials existed to support or refute treatment effect! Despite the lack of supportive evidence, the number of patients undergoing chelation therapy increased by >50% between 2002 and 2007. Thus, it was important to clarify the benefits and harms of chelation therapy in a controlled outcome trial. Accordingly, the argument that the trial was unethical, as has been suggested by some, doesn’t hold water!

Unfortuantely, Dr. Kaul apparently unknowingly uses the logical fallacy known as argumentum ad populum to argue that the trial is not unethical because, hey, chelationists are out there doing it; so it's ethical to adopt their methods and to study it. The only proper response to that argument, I'm afraid, is not, "If you can't beat 'em, join 'em," but rather to state that chelation therapy should not be given outside the context of a properly constituted randomized clinical trial, and no randomized clinical trial is justified until there is a lot more preclinical evidence to support one. Imagine if a drug company tried to justify a clinical trial for a drug based on the thin gruel of preclinical and clinical studies of chelation that existed before TACT. I'd bet that Dr. Kaul and Dr. Krumholz would be outraged. Yet they don't mind such reasons when they are put forth to justify TACT. In any case, the trial is every bit as unethical as Dr. Atwood and colleagues have argued and Dr. Atwood has reiterated.

It's also more than just the lack of preclinical and clinical evidence, although Kimball Atwood and colleagues laid it all out in excruciating detail in an extensive analysis published in Medscape in 2008 entitled Why the NIH Trial to Assess Chelation Therapy (TACT) Should Be Abandoned. I strongly suspect that Dr. Kaul has not read the whole thing. It is not that there was no evidence to support or refute chelation. The question was: Did those trials show any convincingly positive signal worth spending $30 million for a huge multicenter clinical trial of a couple of thousand patients to followup on? The answer is no, particularly given the extreme implausibility on a physiological basis, an implausibility in pharmacology that doesn't make sense either in mechanism or, even if one accepts the mechanism might have validity, stoichiometry. (I hope Drs. Krumholz and Kaul, will click on the links and actually read the mechanistic criticisms this time. It's so obvious that they didn't bother last time.)

Dr. Kaul's otherwise well-considered Bayesian argument ultimately fails because he, like so many, almost certainly vastly overestimates the prior plausibility of chelation therapy, estimating a 5% prior probability of an extreme treatment effect (i.e.,>25% risk reduction or >33% risk increase. Many would argue that this is a reasonably skeptical prior distribution, going on to say that "many would argue that this is a reasonably skeptical prior distribution." Perhaps "many" would, but "many" also ignore the lack of even suggestive evidence in human or animal studies and the extreme mechanistic implausibility. 5% is way too high. Maybe 0.05% (or 0.5% if I'm feeling particularly generous). Some would call my estimated prior plausibility far too generous, but I will concede that we're not talking homeopathy-level implausibility (about as close to zero as it's possible to imagine) when it comes to chelation.

Dr. Kaul, like so many good-hearted academic physicians unfamiliar with quacks, seems to imply that there is sort of “societal value” to test interventions that are widely used in society even when those interventions whose proposed mechanisms are highly implausible. I might agree with him up to a point, except for two considerations. First, no amount of studies will convince, for example, homeopaths that homeopathy doesn’t work. Similarly, chelationists are still giving chelation therapy to nondiabetics even though TACT itself is as negative as a negative trial can be on that question: Even by the most optimistic interpretation of TACT, chelation does not work for nondiabetics and those without an anterior MI. Second, research funds are scarce and likely to become even more so over the next few years. From a societal perspective, it’s very hard to justify allocating scarce research dollars to the study of incredibly implausible therapies like chelation therapy for atherosclerotic cardiovascular disease. Given that, for the foreseeable future, research funding will be a zero sum game, it would be incredibly irresponsible to allocate funds to studies like TACT knowing that those are funds that won’t be going to treatment modalities that have a far better chance of actually working. In other words, it's not often a good idea to test highly implausible health claims without more preclinical data that make them less implausible. Such a failure to consider prior probability appropriately has led to truly unethical clinical trials of, for example, homeopathy for infectious diarrhea in children in third world countries.

Finally, there's the issue of wrongdoing among TACT site investigators, something that really, really gets Dr. Lamas' dander up, not to mention Dr. Kaul's as well. This is probably worth a post of its very own, given how complex the issue is, but let me just point to post by Kimball Atwood and a PDF of a newsletter to TACT investigators as a little wafer to cleanse the palate.

Kimball Atwood points out examples of problems. For instance, Dr. Rajiv Chandra of Melbourne, FL, is frequently cited as the “top enroller” for the trial. Chandra, as Atwood et al explained, promised this to prospective subjects:

If you participate in this study, you will receive 28 months of treatment, and be asked to participate in up to 32 months of follow-up. You will not be charged for participating in this exciting study and will receive the study drug and vitamin and mineral supplements.

Ooops! Did he just promise in an ad to provide the study drug? OK, OK, maybe it's a newbie site investigator making a rookie mistake. Maybe. Interestingly, however, there is this newsletter, in which Dr. Lamas apparently feels obligated to warn site investigators against certain actions:

• During follow-up phase, after the study infusions, patients must not start open-label chelation therapy at the TACT site or at any other site.
• Offering free infusions after the end of the study as an inducement to recruitment is not recommended.
• Patients must not receive open-label chelation during their participation in TACT. (5 years after enrollment or in July 2009).

It sure sounds to me as though something was going on at some sites that Dr. Lamas was concerned enough about to warn his investigators not to do. Otherwise, why would he have bothered? One wonders, one does. But that's enough for now. This post is long even by Orac standards, and it's late. Much of the deconstruction would involve delving deeply into various reports and Dr. Lamas' responses. So I'll finish with an observation that comes from what is definitely the most risible statement in an article full of risible statements, Dr. Lamas' response a question about the documented fact that some of the TACT site investigators had been involved in insurance fraud and others were convicted felons:

Were there investigators who had violated the law – and how might unethical behavior by individuals who strongly believed in CAM have altered the trial? Is this any different from other trials where investigators are invested in the success of the intervention – or did this trial have specific issues that must be considered?

All investigators had an unrestricted license to practice medicine in their states, and they received human-subjects training, protocol training in person and online, research training, IRB approval, in-person site visits, and electronic data monitoring by the Data Coordinating Center at Duke.

Yes, that's right. Dr. Lamas completely dodged the question, and Dr. Krumholz let him! In any case, apparently Dr. Lamas' standards are that all site investigators had to have was a medical license and a pulse, as long as they did the training mandated. Never mind those disciplinary actions by state medical boards and insurance fraud! Yes, that's a very low bar indeed for clinical investigators.

I understand that Dr. Krumholz and Dr. Kaul have probably never dealt with quacks who sell chelation therapy and aren't aware of the ins and outs of the history of TACT. (That's why they should read the deconstruction by Kimball and colleagues.) I also realize that Drs. Krumholz and Kaul are buddies with Dr. Lamas. So automatically TACT critics are the outsiders, and I'm "shrill and brutish," and so very, very mean and nasty, for writing all the things that I've written about TACT, which is run by such a good buddy and fine clinical investigator who couldn't possibly have ever gotten himself involved in something so questionable as TACT. So obviously, to them, TACT can't possibly be so dubious and questionable! It just can't be! I understand that. Now understand this: I could be a bit more civil if I wanted to be, but, in reality, seeing $30 million wasted on a pointless and unethical trial of a highly implausible form of quackery that endangers patients with almost no likelihood that they will be helped makes me just a wee bit angry. And when I get angry I guess I get a bit "shrill and brutish." Sometimes, however, it's justified. This is one of those cases.

Obviously, Dr. Krumholz must agree on some level. Certainly he didn't criticize Dr. Lamas for his not-so-subtle labeling of TACT critics as dishonest and hypocrites. I'd take his protestations of wanting a "civil discourse" more seriously if his calls for "civility" weren't so one-sided. He's just too nice to take the next step and express a bit of anger himself when he deems it justified.