Surgical research: "Tinkering" versus "innovation" versus "research"

Well, here I am in sunny Phoenix, having spent pretty much all of yesterday at the conference, sneaking in alterations to and practicing of my talk in between sessions. All in all not a bad day, although I spent the entire day indoors and didn't get to partake of the bright and cheery warmth, which is sad, because it's particularly welcome given the weather at my present abode. The conference produced one other thing for me as well: A good blogging topic. Not only that, but it's a good blogging topic that fits in with the whole "Just Science" theme of this week. Don't worry, though, no steeenkin' theme' gonna keep me from doing Your Friday Dose of Woo, although maybe I'll try to make it more "science-y" than usual--assuming I manage to find time to do it.)

Yesterday afternoon, I attended a rather fascinating session sponsored by the Society of University Surgeons on surgical research issues with regard to bioethics and institutional review boards. I realize that there are those out there, particularly on the medical side of things, who with a sneer will refer to the term "surgical research" as an oxymoron, but, trust me on this, there are a lot of surgeons out there trying to do scientifically rigorous studies within the bounds of their discipline to advance the art and science of surgery, all for the good of their patients. (OK, there's also glory to be had, but the primary motivation still usually boils down to wanting to do good for our patients, with the glory being a secondary motivation.) The difference is that surgery is just as much a craft as it is a science. How a surgeon ties knots, where he places the incision and how large he makes it, how he puts together two ends of bowel so that they don't leak, all of these things are examples where the skill of the individual surgeon matters as much as the scientific validity of what operation the individual surgeon is performing. Surgical research differs from research carried out in medical specialties in that it is much more difficult to standardize operations; placebo controls (i.e., sham operations) are rarely, if ever, ethically acceptable; and double blind studies are in essence impossible, because the operator will always know what he did and blinding the patient to what was done is similarly impossible without doing sham operations. Moreover, new surgical procedures have a learning curve, such that they may seem less desirable early on in their development but their benefits become apparent the more practice surgeons have at them and as more surgeons learn how to do them well. All of these factors tend to make surgical research, I would argue, in many ways more difficult than medical research, particularly research comparing drug treatments, where double blinding is possible and placebo controls can often be used.

One commonly accepted definition of research is found in federal regulations about human subject research: "Research means a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalized knowledge." Innovative surgery is often not systematic. Rather, it is designed to benefit individual patients; there is often no intent to publish the surgical series at a later time, or such intent is secondary. In reality, many, if not most, advances in surgery don't come from careful controlled trials (at least not at first), and the topic of the conference was how to decide when alterations in an established surgical procedure constitute what was referred to as "tinkering" and when they constitute an "innovation," which is more substantive (more on this later). And when does a surgeon cross the line from tinkering into innovation into research, anyway? These are actually pretty tough questions, with a lot of complexities involved, and the main impetus for the SUS to undertake a project to try to write guidelines about these issues, the preliminary draft of which were presented at this session, was as a pre-emptive strike: To do it first before the government steps in and does it in its typically ham-fisted manner. Also, a lot of this tinkering and innovation go on outside the confines of academic medical centers, out in the "real" world of private practice surgeons.

Sid Schwab described a perfect example of tinkering by a private surgeon when he told how, over time, he developed a "mini-cholecystectomy." (This was before the days of laparoscopic cholecystectomies.) There was no clinical trial to test his hypothesis that he could do a cholecystectomy through a smaller incision. He just did it, confident that he could always make the incision bigger if he was struggling too much. (It's an axiom in surgery that you can always make the incision bigger, but you can't make it smaller.) Clearly this qualifies as "tinkering"; i.e., the minor modification of a surgical procedure that does not produce a reasonable expectation of increased risk to the patient. Of course, if Dr. Schwab were too much of a "cowboy" to admit when he couldn't do the operation using a the smaller incision and started getting into trouble because he couldn't see or get his instruments in there adequately, you could imagine that this tinkering might have the potential to cause harm. But Dr. Schwab is a prudent and skilled surgeon; it's reasonable to trust that his judgment is good and that he'll do the right thing and make the incision bigger if he finds himself struggling too much. Lots of procedures have been improved by such tinkering, and, in fact, often such tinkering is simply a reflection on the skill of the surgeon; for example, one of my partners can do a right colectomy through what to me is an amazingly small incision, while other surgeons use much bigger incisions to do the same operation.

Basically, it was widely agreed that there is generally no need for IRB involvement when tinkering with surgical procedures, as long as informed consent is given or, if the operation was done emergently, the patient is informed afterwards what was done. In practice, though, I can see this as a bit problematic. Consider Sid in the example above. Suppose he went up to his patient and said, "I'm going to do your cholecystectomy, but I'm going to do it a little different. I'm going to use a small incision. It means I might struggle a bit and the operation may take a little longer, but I can always open wider if I have to." Most patients wouldn't object, but many would ask: "How much smaller?" A reasonable question. If the incision is only, say, 5% smaller than the usual incision, would it be necessary to tell the patient? After all, only Sid would know for sure that it was smaller than his usual incision. Other surgeons might make that size incision just from their own training, and even Sid, unless he measures all of his incisions to the millimeter, probably varies his incision size by as much as 5% from patient to patient just in the normal scope of his practice.

So, what is innovation? Innovation differs from research in that it is a change in therapy designed to benefit an individual, whereas research is a study according to protocol in which the goal is to gain knowledge, not necessarily to benefit the individual being treated. As the Belmont Report, the basis of all human subjects research regulations in the U.S. states:

By contrast, the term "research' designates an activity designed to test an hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge (expressed, for example, in theories, principles, and statements of relationships). Research is usually described in a formal protocol that sets forth an objective and a set of procedures designed to reach that objective.

When a clinician departs in a significant way from standard or accepted practice, the innovation does not, in and of itself, constitute research. The fact that a procedure is "experimental," in the sense of new, untested or different, does not automatically place it in the category of research. Radically new procedures of this description should, however, be made the object of formal research at an early stage in order to determine whether they are safe and effective. Thus, it is the responsibility of medical practice committees, for example, to insist that a major innovation be incorporated into a formal research project.

The draft recommendations suggest that such innovations must comply with FDA and NIH regulations and that informed consent must be obtained, except in the case of emergency surgery, in which case the patient must be informed afterwards, in the case of "unanticipated innovation." (Of course, one surgeon's "unanticipated innovation" is another surgeon's "surgical misadventure.") It was also recommended that, if all possible, such innovations should be tried out first in an animal before being performed on humans. A further recommendation was the creation of an "innovations database," to which surgeons could report their attempted innovations and how they worked. This is particularly important because most innovations do not work, and some even cause harm. If a surgeon is thinking of trying a new innovation, it would be useful if that surgeon could consult a database to see if someone had tried it before. After all, what's the first thing we scientists do when considering embarking on a new research project? We hit the scientific literature, of course, to see if anyone has done anything like what we're thinking of doing before! Scientists have PubMed; many, if not most, innovations, particularly the ones that don't work, are never published in the peer-reviewed literature. The final recommendation was that "innovation review committtees" be set up to review attempts at surgical innovations. These would tend to be ad hoc committees based on the specific case and far more informal than an institutional review board.

Of course, the distinction between tinkering and innovation (and, for that matter, research) is artificial, and definitely somewhat arbitrary. It all really exists as a continuum. Consider the example of Dr. Schwab's "minicholecystectomy" again. It's really a remarkable story of how a surgical procedure, in the hands of a skilled surgeon, evolved over time. Simply by trying to decrease the size of the incision in a systematic fashion, Dr. Schwab ultimately ended up changing his practice. He started using a headlight, special instruments, and a different surgical technique to get the gallbladder out. Ultimately, his consistent tinkering led to innovation, and, I daresay, had it not been for the sudden advent of laparoscopic cholecystectomy in the late 1980's and early 1990's, the mini-cholecystectomy that he and others developed would likely have propagated far and wide among surgeons and become the preferred technique for removing gallbladders. In terms of the scientific method, tinkering could be looked at as the stage of hypothesis generation, while innovation could be considered preliminary testing and refining of the hypothesis. The actual research is then the stage where the refined hypothesis is subjected to rigorous testing in a highly controlled, protocolized manner.

Oddly enough, laparosocopic cholecystectomy (LC) is a great example of an innovation that rather got ahead of itself. The potential benefits of the procedure in terms of decreased hospital stay, decreased postoperative pain, and faster return to work were obvious and quickly recognized. However, this procedure, perhaps more than any other, reveals some of the pitfalls in surgical innovation. For one thing, it revealed the effects of marketing and patient demand. Surgeons who learned how to do LCs in those heady early days advertised it heavily, and patients came to see the new procedure as obviously superior to the old before there was much comparative data on complication rates. They started to demand it, and more and more surgeons felt obligated to learn it even before the data from research (formal definition) was all in, lest their patients go elsewhere and their referring physicians abandon them. Eventually, as the research was finally done, it became apparent that there was a considerably higher rate of common bile duct injuries in LC compared to the old method. Granted, it was still a very small number, but the number was nearly ten times smaller still using the boring, old-fashioned open approach to taking gallbladders out, and common bile duct injuries can be devastating, requiring large operations to repair. In the end, it was fairly clear that the benefits outweighed the larger (albeit still very small) risk of common bile duct injuries with LC, but by the time that information was known the procedure had almost entirely supplanted the older procedure. There was no chance to weigh the risk-benefit ratio of LC in anything approaching a calm, measured fashion before introducing the procedure; the analysis was all done after the "cat was out of the bag," so to speak. Ironically, Sid's mini-cholecystectomy produced similar good results in his hand to LC; maybe it would have been a better, albeit less sexy, alternative, although it's always possible that there were complications that would only become apparent if thousands of patients were studied. That's why innovation needs to be accompanied by research.

There are many problems in considering innovations and research, not the least of which are issues of liability. As one surgeon at the session asked: "When does 'innovation' plus bad outcome equal departure from the standard of care?" The answer was not that encouraging. Innovation by definition is departure from the standard of care, and in the case of a bad outcome it can be a big problem. Innovation review committees can ameliorate this problem somewhat by giving cover to the surgeon, as can rigorous informed consent, but clearly in this litigious age the threat of being sued will have a chilling effect on innovation that even these measures may not forestall. Another issue is the composition of these committees. In academic centers, finding the expertise is usually not that big of a problem, but what about in community settings, where either the expertise is lacking or the potential members of such a committee would often be riddled with conflicts of interest, given that potential members would tend to be either partners of or competitors with the surgeon involved. Finally, there is the issue of intellectual property, particularly in the case of devices, even simple ones. Surgeons will be reluctant to share their ideas in a database if they are thinking of patenting them, although it was pointed out that it's fairly easy to protect one's idea beforehand by filing a preliminary patent application. And, of course, we must remember that, as much as we hate to admit it, surgeons are human too. Not all of them can always be counted on to act based primarily on what they perceive to be the best interests of their patients.

The bottom line, regardless, is that academic surgeons are going to have to get used to more oversight. Vigorous self-regulation would be the best option if we can pull it off. Gone are the days of the "Wild West," where "cowboy" surgeons could try whatever innovative idea that caught their fancy. Although many bemoan this new oversight and worry that it will quash surgical innovation, that need not be so if surgeons, rather than the government, act to codify how we distinguish between "tinkering," innovation, and research and require the level of oversight appropriate to each: IRBs for research and something formalized, but less rigorous, for innovation.