Lancet study is wrong

The New York Times reported:

The World Bank failed to follow through on its pledges to spend up to $500 million to combat malaria, let its staff working on the disease shrink to zero, used false statistical data to claim success and wasted money on ineffective medicines, according to a group of public health experts writing in the British medical journal The Lancet.

The experts, in an article to be published online today, argue that the bank should relinquish the money it has to fight malaria, which kills an African child every 30 seconds, and instead let the Global Fund to Fight AIDS, Tuberculosis and Malaria distribute the bank's malaria funds.

The World Bank conceded in a written reply to the article that its malaria programs were understaffed and underfinanced, but denied using false statistics or paying for obsolete medicines. It said that in the past year it had revitalized its malaria program.

Bank officials said in an interview on Monday that the number of staff members working on malaria had grown from none to more than 40 in the past year, while $62 million in new spending had recently been approved, an amount expected to rise to $190 million by June. The Global Fund does not have staff on the ground in Africa to monitor how the money is spent, while the World Bank does, bank officials said in their written reply.

"The story captures a lot of the bank's shortcomings from a year ago," Suprotik Basu, a public health specialist in the bank's malaria program, said yesterday. "But now we've had a year of progress."

You can read the criticism by Attaran, Barnes, Bate, Binka, d'Alessandro, Fanello, Garrett, Mutabingwa, Roberts, Sibley, Talisuna, Van geertruyden and Watkins here and the reply from the World Bank here. (Registration required.)

It is interesting to note that one of the authors of the criticism of the World Bank is Roger Bate, who last year wrote how the main failing of the World Bank's efforts against malaria was that it didn't support DDT use. But this criticism doesn't mention DDT at all. It would seem that the other authors did not agree with Bate on this point.

So what about the charge that the World Bank is using false statistics? Attaran et al write:

The Booster Program claims that in Brazil, the Bank's $73 million malaria-control project was a success because "reported malaria cases dropped by 60%, from 557,787 in 1989, to 221,600 in 1996". That claim seems to be based on malaria incidence statistics that Bank employees and others published in 1999, but on closer inspection, the statistics show a peculiar trend: no reduction in malaria cases for 6 years of the Bank's work (1989-95), and suddenly, a 60% drop in the final year (1996).19

We compared the Bank's statistics (figure 1, pink bars) with those of the Brazilian Government (parasite-positive slides, green line) and the Roll Back Malaria Partnership (reported cases, red line).20,21 The Brazilian Government and Roll Back Malaria statistics roughly agree with each other. But neither is consistent with the Bank's claim that malaria cases dropped 60% during its project, and according to the first-hand statistics of the Brazilian Government, the decline was just 23%.

i-019ec3b0b8ca339e810b541e0462ab33-Brazilmalaria.png
Numbers of cases according to World Bank's data are in pink.19 Green line=number of cases according to Brazilian Government;20 red line=number of cases according to Roll Back Malaria Partnership.21 Note that in final year of the Bank's project (1996), when pressure would have built to show success, the Bank claims sudden drop in malaria cases, which is not corroborated by other data sources.

Overall, neither the Brazilian Government nor the Roll Back Malaria Partnership statistics support the Bank's interpretation that its project achieved a deep reduction in malaria in Brazil. Yet the Bank has long maintained that Brazil is one of its success stories, writing in the Global Strategy & Booster Program that it "prevented nearly 2 million cases of malaria and 231,000 deaths" in Brazil.1 That claim, assessed in light of Brazil's own data, is certainly wrong.

Reference 19 is a 1999 paper by Akhavan et al. Figure 1, which I've reproduced below, shows the number of cases of malaria each year. I added the red line to make it perfectly clear that it shows that the number of cases in 1996 was about 440,000, not 220,000 as shown in Attaran's figure 1. Akhavan, Roll Back Malaria and the Brazilian Government all give about the same number for malaria cases in 1996. Indeed, on page 92 the World Bank's Booster Program also gives the same number -- it's only given incorrectly in the passage Attaran quotes.

i-a41d9489189a4b29efc09c1d19badffb-brazilmalariafigure1.png

So why did Attaran and the World Bank's Booster Program write that Akhavan et al found that there was a drop to 221,600 cases in 1996? The answer is that table 2 of Akhavan, which shows how they calculated that their program had "prevented nearly 2 million cases of malaria and 231,000 deaths", only shows half of the cases from 1996. They write:

For 1996, only half the cases treated or estimated to have been prevented are counted, because the project ended in June of that year and the calculation of costs runs through only half the year.

That means that the number of cases of malaria shown in Table 2 for is half the number for the whole year.

How did they calculate the number of cases prevented? Before the program started, the number of cases was steadily increasing (see their figure 1 above). They argue that the number had not reached saturation, so, in the absence of the World Bank's intervention, the number of cases would have continued to rise. The 2 million cases prevented is the difference, up to the middle of 1996, between the expected number of cases and the actual number of cases. Note that only including half the cases for 1996 reduces this number.

The Booster Program made an error. Here is a corrected version of their Brazil summary:

The outcome of this program was that reported malaria cases dropped by
60 20 percent, from 557,787 in 1989 to 221,600 443,200 in 1996. The percentage of
cases caused by Plasmodium falciparum (the most lethal form of malaria)
decreased from 47 percent to 29 percent during the same period. It has been
projected that these efforts prevented nearly 2 million cases of malaria and
231,000 deaths.

Attaran et al have made a more serious error. The authors of the Booster Program just misunderstood a table. Attaran et al knew that something was wrong, but failed to properly investigate and find out what had happened. They (and remember, there were thirteen authors) missed what was shown in figure 1, missed the discussion of figure 1, misunderstood table 2, and missed the description of table 2 I quoted above. Instead they insinuated that Akhavan et al had deliberately falsified their data. They assert that the Booster Program's claim that the program "prevented nearly 2 million cases of malaria and 231,000 deaths" in Brazil "assessed in light of Brazil's own data, is certainly wrong." But if they had really assessed that claim they would have discovered that it did not depend on their being 221,600 cases in 1996, but on the calculations in table 2. And even if you believe that Akhavan had understated cases by 220,000, that's only 10% of the 2 million cases that Akhavan say were prevented, so you still could not conclude that this claim was "certainly wrong".

Attaran et al must have really wanted to find the World Bank guilty of falsifying data to have missed all of this.

Update: Musgrove replies to Attaran here.

Tags

More like this

Tim Lambert: well done, that's good work, and serves to confirm that The Lancet is a very shonky journal, as previously shown by their Iraq death counts. Minimal due diligence would have shown that both papers were without merit. But are you in the clear? Julian Simon was an exceptionally clear sighted economist, and his "deceit" about malaria bans was in fact largely correct, since to this day there are widespread bans on other than indoor spraying, even when the truth is that no country has ever eradicated malaria without both indoor and outdoor spraying (see the new book on malaria eradication in Italy, only achieved as late as 1962). The Carson mantras against outdoor spraying still rule on the basis of dubious data on thinning eggs and residual DDT in breast milk.

I'm a co-author (with Abrantes and Gusmao)of the study by Akhavan that Attaran et al. cited and so thoroughly misinterpreted. We greatly appreciate your correct interpretaion of our work, and your conclusion that Attaran and his careless colleagues were simply looking for an excuse to attack the World Bank, and uninterested in the true story.

I want to clarify just one detail in your note. You say, "even if you believe that Akhavan had understated cases by 220,000..." There is no understatement. The estimates of cases prevented and deaths averted (1.73 mln, 231,000) are the right numbers for the 7-1/2 years covered by our analysis, as our Table 2 shows. As you correctly pointed out (with the red line) our Figure 1 shows the total cases of malaria for each year, starting with 1975, so all cases for 1996 are counted there. Our cost-effectiveness analysis, and the estimates of savings in cases and lives, are based only on the interval corresponding to the 1989 to mid-1996: the effectiveness estimates had to end at the same point as the cost data.

Thanks again.

--Philip Musgrove

By Philip Musgrove (not verified) on 18 May 2006 #permalink

I assume mr cutrin doesnt know about DDT resistance, or any of the myriad posts that Mr Lambert has made on the topic?

"since to this day there are widespread bans on other than indoor spraying"
de facto I presume? since there is no such language anywhere in the treaty.

Not to threadjack, at least not for very long, but Mr. Curtin seems unaware that Iraq Body Count in its criticism of the Les Roberts paper ended up tacitly endorsing numbers that were only moderately lower than the 100,000 figure. They wanted to chop 39,000 war-related deaths down to 28,000 (based on the UN survey) and if that relative correction were applied across the board, that would be 70,000 rather than 100,000 deaths.

End of threadjack.

By Donald Johnson (not verified) on 18 May 2006 #permalink

Tim, you are quite deeply confused here. There is an apples-and-oranges difference between what the Akhavan and Attaran papers say.

The paper from Akhavan reads that in 1996 it reported only "half the cases treated or estimated to have been prevented". But the point Attaran et al made has nothing to do with malaria cases treated, or malaria cases estimated to have been prevented. Instead the sketchy World Bank data that they point to are for ALL reported malaria cases, full stop. Akhavan's paper does not say that all malaria cases were discounted by half, but only the "treated" or the "prevented" malaria cases in their model were. Anyway, Akhavan worked for the World Bank, and the World Bank turned to his numbers for their Booster Program. As you point out, later the Bank had to correct those numbers, which rather suggests they had it wrong, and not Attaran et al surely.

Also, what is your incredible blind spot with DDT? There has been a very dramatic study in PLoS Medicine that for South Africa shows malaria rates soared when DDT was temporarily stopped. It took restarting DDT to bring malaria back down again. See the first figure here by Barnes et al.

http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10%…

Maybe stick to the computer science, Tim.

By Kurt Boggs (not verified) on 18 May 2006 #permalink

Kurt, Table 2 is showing their calculations of the number of cases prevented. The numbers for 1996 are halved for the reason they give. The number of cases of malaria is shown in their figure 1. Did you see it in my post? Did you see the red line that shows how many cases there were? Both the World Bank and Attaran misunderstood Akhavan et al. One of the authors of Akhavan et al has already posted a comment agreeing with my interpretation.

And maybe you should consider actually reading what I have written about DDT? I have [commented on the South African experience](http://timlambert.org/2005/09/ddt-use-in-south-africa/). Heck, I don't think you have even read the article you linked. Was DDT the only factor involved? What was the title of the paper?

Tim Lambert: the paper describes a combination of PREVENTION (by vector control using DDT spraying in traditional housing,in part because of growing resistance to pyrethroids) and CURE (by artemisin treatment).

Tim Lambert, I accept that it is possible Table 2 of Akhavan's paper contains only half of 1996's "reported" malaria cases. But this fact remains: Akhavan et al goofed in never saying so. They said they counted half the "treated" or "estimated to have been prevented" malaria cases. To be accurate, they should have said they counted half the "reported" cases too. For whatever reason they didn't say this, and duped even their own colleagues writing the Booster Program, necessitating a correction to the numbers later.

And what about the fact that malaria cases in Brazil stayed about the same throughout the Bank's project, 1989-1996? What about the fact that malaria cases in Brazil stayed about the same after the Bank's project, from 1997 onward? Where malaria control works, as in the South Africa example, the malaria cases drop by something like 90%, maybe more. The fact that malaria cases hovered around the same number both before and after the World Bank's project in Brazil -- flatlined, basically -- suggests that the Brazil-World Bank project was a horrible waste.

Now, you seem proud that Mr. Musgrove, who was an author of the Akhavan paper, agrees with your interpretation. Well, Mr. Musgrove is an economist who WORKED for the World Bank in Brazil: http://www.fic.nih.gov/dcpp/musgrove.html

So you expect Mr. Musgrove to disagree? Get real.

When you start being a flak for the World Bank, better you stick with the computer science, Tim.

By Kurt Boggs (not verified) on 19 May 2006 #permalink

It's obvious Bate isn't quoted so Lambert can misrepresent what he wrote. Bate didn't criticise the World Bank for not supporting DDT use, he wrote:

These programs entail spraying tiny amounts of insecticide, such as DDT, on the inside walls of houses to repel or kill (or both) the malaria-carrying mosquitoes. This method of control is safe and highly effective: Malaria rates have plummeted in the very poor northern parts of Zambia where this approach is currently employed. Yet RBM and the World Bank, always politically correct, have eschewed this method of control. The World Bank even went as far as to require that its of funding malaria control in Eritrea be conditional on non-use of DDT.

The World Bank's stinginess, lack of transparency, and overall malaria strategy need serious overhauls.

Nothing posted by Lambert can be accepted as accurate.

Kurt, table 2 only has columns for "case prevented" and "case reported" there is no separate column for "cases treated". If you had bothered to read the paper you would have discovered that part of their plan was immediate treatment of any cases detected, so "cases reported" is the same as "cases treated". That's why in table 2 "gain from treatment" applies to all of the cases reported. Both you and the Booster Program have misunderstood the table. Attaran can charge the Booster Program with getting the numbers from Akhavan wrong, but he got them wrong too, and his mistake was worse.

As for your second paragraph, you have once again ignored their paper. If you want to take issue with it I think you might take the time to undestand their arguments. As for the situation after 1996, are you saying that efforts to control malaria were abandoned after 1996?

As for Musgrove, he posted under his real name, so we know he worked for the World Bank. Who do you work for?

And yes, I'm a computer scientist. And I've done a better job of understanding the Akhavan paper than you have. Do you have any relevant qualifications?

Tim, why don't you read Sebastian Mallaby, or Jeffrey Sachs, or Bill Easterley, or Amartya Sen, or Stephen Lewis, or Joe Stiglitz on the World Bank, because one thing they agree on is the World Bank's structural adjustments and debt destroyed health care around the world. You know, maybe Attaran et al have a point about the World Bank, because if they were serious, there wouldn't still be malaria in Brazil. Or sorry, maybe YOU, Tim Lambert, are a forgotten genius who is the only right person on the whole planet. No need to win a Nobel prize or write a book or take a stand like these other brave people did, if you are Tim Lambert. Amazing.

Besos, did you read my post at all? I didn't write anything about the World Bank's structural adjustments. Attaran claimed that Akhavan et al presented falsified statistics. They didn't. Attaran et al made a serious error in their paper. Don't you think they should correct it?

Yes, they still have malaria in Brazil. Can you name one tropical country where malaria has been eradicated? Are you claiming that no-one at all is serious about the problem? If you have the secret to eliminating malaria in the tropics, tell us about it, because if you can come up with the way to do it, you will get the Nobel prize.

"And what about the fact that malaria cases in Brazil stayed about the same throughout the Bank's project, 1989-1996? What about the fact that malaria cases in Brazil stayed about the same after the Bank's project, from 1997 onward? Where malaria control works, as in the South Africa example, the malaria cases drop by something like 90%, maybe more. The fact that malaria cases hovered around the same number both before and after the World Bank's project in Brazil -- flatlined, basically -- suggests that the Brazil-World Bank project was a horrible waste."

1. What was the population growth in Brazil over those seven years?

2. What are the relative infection rates for malaria in Brazil and South Africa? If the rate of infection was lower in brazil to begin with a dramatic 90% redcution woudl be harder to achieve.

By Ian Gould (not verified) on 20 May 2006 #permalink

T Besos: "Tim, why don't you read Sebastian Mallaby, or Jeffrey Sachs, or Bill Easterley, or Amartya Sen, or Stephen Lewis, or Joe Stiglitz on the World Bank, because one thing they agree on is the World Bank's structural adjustments and debt destroyed health care around the world."

So is life expectancy in the developing world going up or down?

If its going up, how do you reconcile that with health care being "destroyed"?

The World Bank has made mistakes,is making mistakes and will almost definitely continue to make mistakes in the future.

It has also done a lot of good.

By Ian Gould (not verified) on 20 May 2006 #permalink

Tim Lambert said: "If you have the secret to eliminating malaria in the tropics, tell us about it, because if you can come up with the way to do it, you will get the Nobel prize".

I say: let's propose Gerrit J.T. Schuurkamp for that Nobel. His Ph.D thesis (published as The Epidemiology of Malaria and Filiaris in the Ok Tedi region of the Western Province of Papua New Guinea, Tabubil, 1992)reached the following conclusions (pp.105, 135).

"The impact of DDT residual spray activity at the village level around the developing Ok Tedi mining infrastructure between 1983 and 1986 saw a major reduction in parasite and spleen rates, especially in children under 10 years of age... When the 1983 baseline results were compared with the 1985 and 1986 surveys...the results were dramatic: [from Table 5.5]: the infant parasite rate was reduced from 55% to 4%, the 67% PR for all children under 10 years was reduced to 20%, and the overall PR reduced from 37% to <15%."

"Spraying with residual insecticides, if properly carried out, still plays a major role in the control of vector borne diseases. ...DDT, despite its environmental track record, is still extremely effective in Papua New Guinea".

Tim Lambert, the question is are you for or against the use of DDT? Your highly misleading account of the Eritrea study, which you presented as if it was a laboratory test of DDT versus treated bednets, in the sense of DDT in one area and bednets in another, to the detriment of DDT of course, when it was nothing of the kind, suggests that you like the World Bank remain opposed to DDT; your attacks on Roger Bate for his DDT advocacy suggest the same. I invite you to join me on a trip to Ok Tedi so that you can explain to the villagers why they were wrong to accept the mining company's DDT spraying.

Tim Curtin, you might want to look up the word "eliminate" in the dictionary. For the umpteenth time: I do not oppose the use of DDT and neither does the World Bank. I think that the most cost effective means should be used. That means that DDT should be used in some circumstances but not in others. Why do you find this so hard to understand?

Attaran and Bate are the heroes who turned the tide against DDT being banned, and they saved who knows how many lives. The Lancet is a really amazing science journal. Sen, Stiglitz, Easterly, Sachs, these guys are giants and Nobel winners who call it like it is with the World Bank. And Laurie Garrett, who wrote with Attaran and Bate, is a journalist and Pulitzer winner, who worked on the Bank's malaria plan and blew the whistle on them.

And against all these courageous and super bright people is Tim Lambert who is....well, Tim Lambert. Whoa, impressive!

Oh Tim, you asked to "name one tropical country where malaria has been eradicated". Ever hear of Singapore? Others like Cuba, or Costa Rica, or Sri Lanka, or South Africa and Zimbabwe once upon a time, got really close.

Tim Lambert said "you might want to look up the word "eliminate" in the dictionary. For the umpteenth time: I do not oppose the use of DDT and neither does the World Bank. I think that the most cost effective means should be used. That means that DDT should be used in some circumstances but not in others."

Bravo, but as ever you are a perfectionist, so reducing rates to 4% is not good enough for you, even in villages with growing populations over the period with an influx from untreated areas. How did Australia eliminate malaria 100%? was it bednets? how many birds' eggs became thin?

From the same site:

"Although malaria is no longer endemic in Australia, approx. 700-800 cases occur here each year in travellers infected elsewhere, and the region of northern Australia above 19oS latitude is the receptive zone for malaria transmission. Occasional cases of local transmission occur in the Torres Strait islands and rarely in northern Queensland, and vigilance is required to prevent reestablishment of the infection in some northern localities."

In other words, australia has successfully prevented the re-emergence of malaria without the use of DDT.

DDT is a useful tool for malaria control in some circumstances but it is neither necessary not sufficient for eradication.

By Ian Gould (not verified) on 21 May 2006 #permalink

Ian Gould and Tim Lambert:

do check "Impregnated nets or DDT residual spraying?:, Mead Over et al., Am. J. Trop. Med. Hyg. 71 (2 suppl), 2004, 214-223.

For example, "DDT spraying is substantially more effective than impregnated bednets in these field conditions [in Solomon Islands]....increasing DDT application by 50% [to 60 kg per zone per month]... would reduce incidence to zero, while a sixfold increase in [ITN} would be required to achieve the same result... [and] at 60 kg DDT per month per zone no [ITN] would be required to achieve zero incidence...[if ITN were exclusively relied on]...cases would triple....The evidence from the current study suggests that impregnated bed nets cannot easily replace DDT spraying without substantial increases in malaria incidence.."

So Tim tell us, hod DID Australia eliminate malaria?

By Ian Gould (not verified) on 22 May 2006 #permalink

While you're at it Tim C, feel free to show me where either Tim L or myself ever argued against indoor residual spraying.

However, in the real world, as opposed to controlled experiments (even field experiments)there are going to be some circumstances where bednets are preferable. For starters, IRS requires teams of government workers on the ground, good luck getting that to fly in eastern Congo or the Myanmar/Thailand border.

Ideally, of course, everyone at risk would be protected by both bednets and IRS (with DDT or with another insecticide where DDT resistance is a problem)and have access to effective treatment as well.

Unfortunately we don;t live in an ideal world.

If we did we wouldn't be subjected to the unedifying spectacle of the right's gradual and grudging retreat from "Rachel Carson is worse than Hitler" to "there's a theoretical possibility that maybe, somewhere, sombody died because cabramate was used instead of DDT for IRS."

By Ian Gould (not verified) on 22 May 2006 #permalink