In late 2006, a statistical study of deaths that occurred after the invasion of Iraq ignited a storm of controversy. This Lancet study estimated that more than 650,000 additional Iraqis died during the invasion than would have at pre-invasion death rates, a vastly higher estimate than any previous. But in January, a World Health Organization study placed the number at about 150,000.
The conflicting findings highlight just how difficult it is to gather reliable information in a war zone. But they also show the increasing involvement of statisticians in informing responses to humanitarian crises. In addition to the work in Iraq, statisticians have gathered evidence that has aided in the prosecution of Slobodan Milosevic, guided reparations for the civil war in Sierra Leone, and helped determine the needs of Katrina survivors, among many other projects.
“You can go to a congressional hearing or an international war crimes tribunal and you can hear the stories,” says Lynn Lawry of the International Medical Corps. “But how many are we talking about? How many people are at risk? How many people are affected?”
Statisticians are well-suited to answer these questions because they have the tools to put together partial information into a global picture. For example, even if complete records can’t be gathered, a statistician can survey a small number of randomly chosen people affected by a crisis and infer from their experiences the likely impact on the population as a whole. For example, Jana Asher of Carnegie Mellon University in Pittsburgh, Pa., developed an estimate of the rates of rape across Sierra Leone by determining how many women from a national sample had been raped.
But humanitarian crises pose huge challenges. Little information may be available—even from before a crisis—about how many people live where. Even if a previous census was taken, the high birth and death rates in developing countries tend to quickly make censuses outdated. Areas within continuing war zones can be unsafe for survey workers.
“When you have a displaced population that has been forced to flee their homes, all the traditional census methods really break down very badly,” says David Banks, a statistician at Duke University in Durham, N.C. “The refugees don’t have addresses. They’re wandering from one camp to another. Communication is poor.”
These challenges have to be met with very carefully designed protocols. For example, the Lancet study of Iraq, with the shockingly high mortality rates, was initially criticized for not surveying people who lived in back alleys because the areas were too dangerous for surveyors. Les Roberts, who was at Johns Hopkins University in Baltimore at the time but is now at Columbia University, and his collaborators on the study argued that the critics had misunderstood their randomization technique.
Random surveys are not the only useful statistical method. To tally the number of deaths related to the conflict in Timor-Leste, Romesh Silva and Patrick Ball of the Human Rights Data Analysis Group combined incomplete datasets to generate a broader picture of events. The Indonesian military claimed that its occupation of Timor-Leste had caused no deaths. Many stories had been told of killings and famine, but Silva and Ball wanted solid evidence.
Along with gathering about 8,000 personal accounts conveyed to the Commission for Reception, Truth and Reconciliation, Silva and Ball conducted a census of public graveyards including 319,000 gravestones and a survey of a random sample of 1,400 households about displacements and deaths. The researchers found that the different lines of evidence corroborated one another strongly, adding to the strength of each approach. In addition, Silva and Ball could observe how often names recurred across the different databases and get a much better estimate of the total number of deaths across the country.
They found that Indonesian occupation of Timor-Leste from 1974 to 1999 led to more than 100,000 deaths beyond what would have been expected in peacetime, through a combination of direct killings, famine, and illness.
The conflicting studies in Iraq show just how tricky it is to apply these methods in messy real-life situations. About the Lancet study, Asher says, “I don’t think there was anything obvious in what they did that someone can point to and say this method is flawed. But the WHO study used appropriate methodology too.”
The most suspect part of the Lancet study, Asher says, is that the researchers didn’t supervise the survey workers closely. On the other hand, the World Health Organization relied on government workers to administer the questionnaires. People can be intimidated by government workers and be less inclined to say much, a phenomenon that is particularly common in unstable countries. The only way to resolve the conflict, Asher says, is to do yet another study, with an even more careful design.
If you would like to comment on this article, please see the blog version.
