J'accuse: Iraq the Model responds to the Lancet Lies

I was hoping for something a little more insightful than just an ad hominem attack. Writing that the authors had to "resort to mathematics" to arrive at their total is not a little like Bush's infamous remark that the bloodshed in that country was "just a comma."

Oct 11, 2006 03:03 PM

Surely, Steve, you are referring to the Lancet study?

Oct 11, 2006 03:18 PM

According to the AP news article on the study, "Their conclusion, based on interviews of households and not a body count, is that about 600,000 died from violence, mostly gunfire. They also found a small increase in deaths from other causes like heart disease and cancer."

So it sounds like they did knock on doors and ask.

Could someone please write a *real* response to the study that actually critiques their methodology instead of their politics?

Oct 11, 2006 03:22 PM

A real response? I see, a man who actually lives in Iraq can not be trusted but this nonsensical report can.

According to this report about 770 people have died every day. Every day. Where are they? They do not say. According to this report 14% of the 655,000 people died as a result of suicide bombers, that would be about 91,700 people. When and where were these people killed? They don't have a clue.

A bad month in Iraq is about 2600 casualties, but according to this socalled survey, 22,800 people have died every single month.

A third, or more than 200,000 would have died in air raids. That is more than Hiroshima and Nagasaki.

According to this report 57% were killed by gunfire, that would be about 373,000 people.

Where are the bodies? When and where they buried? Did the BBC and AP, UPI, Reuters, AlJazeera all conspire to help hide the truth? Are the US and the Iraqi government together with all the doctors and mayors part of the plot ot hide these hundreds of thousands of dead people?

They took the deaths of a few hundred people and exptrapolated them into more than 600,000 and they compared numbers to the preinvasion Iraq saying there had been virtually no violent deaths in Iraq the 14 months before the invasion. Think about how absurd that is.

Oh yes, Iraq was just a paradise of kite flying children.

This study is absurd on its face and it is not even close the figures that everyone from the Iraq Body Count, to the UN, to the Iraqis themselves have come up with.

I have an idea, if it is true, let the people who did the report prove it. They are the ones who made the claim.

Oct 11, 2006 04:01 PM

From the link:

"For Burnham's study, researchers gathered data from a sample of 1,849 Iraqi households with a total of 12,801 residents from late May to early July. That sample was used to extrapolate the total figure."

So they did some house-to-house, then did an extrapolation. The devil is in the details of that extrapolation. The last time this same researcher did this kind of extrapolation, he was laughed out of town for using some really dodgy statistical fudge factors that inflated the number of deaths by over an order of magnitude. The researcher also openly stated that the last study, released just before the 2004 presidential election, was explicitly politicial in its intent.

Bottom line: real scientists don't need press releases.

Oct 11, 2006 04:52 PM

GH said, "Could someone please write a *real* response to the study that actually critiques their methodology instead of their politics?"

Omar was assuming any serious person could do the math to quickly see how ludicrous these numbers are. I hope terrye's explanation was dumbed down enough so you could understand it but if not, below is a link that might help to explain why you are having trouble understanding this issue.

http://piv.pivpiv.dk

Oct 11, 2006 05:30 PM

Real scientists publish results which can be peer reviewed. We don't see that here do we? The methods employed by the author are suspect, as is the timing of the release of the report. We used to call this level of analysis a WAG - it doesn't even rate being called a SWAG.

Oct 11, 2006 05:58 PM

PG,

Perhaps you might stay a fortnight and walk in Omar's shoes or those of his fellow Iraqi's. Better yet, perhaps you could have walked with him circa 2001?

I think it only fair to give the man a bit of editorial flexibility at feeling a bit used by outsiders.

If you would like a bit of a *real* response to the study that actually critiques their methodology, perhaps just begin with the fact that the study found 547 actual deaths in their surveyed group and extrapolated a mind boggling 655,000 from such a miniscule sample.

That'd be a fair start.

Oct 11, 2006 06:56 PM

Well, any time the author comes right out and admits he's committing agenda science one should be very skeptical to begin with.

Keep in mind, the violence is almost entirely in three provinces. This isn't 600K out of 25M, it's 600K out of maybe 10M.

We would expect to find wounded at about five times that rate. That would mean about 3M wounded, or nearly 1 in 3 Iraqis in those areas-- man, women and child. Does sound reasonable to anyone?

This study is so bad, it actually further discredits its flawed predecessor.

Oct 11, 2006 07:01 PM

You are going to have to do better than a speech playing to the gallery Omar.

This is a scientific study published in an august journal. Of course that does not mean its right. But if you you can't debunk it scientifically without flights of rhetoric, don't bother.

Oct 11, 2006 07:04 PM

"Validation of cluster sampling methodologies as an appropriate alternative to simple random sampling is difficult in conflict situations."

From .pdf page 3 of the Appendix to the report.

If the methodology is not validated for conflict situations, why should anyone consider it reliable.

Omar is right. It is a lot of talk about something that admits it has not been validated.

Oct 11, 2006 07:05 PM

This will be a big attention getter in my next sermon.

Oct 11, 2006 07:44 PM

There's a sort of historical innumeracy at work here. 650,000 violent deaths is about 150,000 more than the number of soldiers who died (violently and by disease) during the American Civil War, a conflict which involved a population about 50% larger than Iraq's, which lasted a year than the current conflict has been going on, and was fought over continental distances. There is nothing in Iraq that looks like Shiloh, Antietam, Gettysburg, Cold Harbor, etc. As terrye notes, this figure is absurd.

Oct 11, 2006 07:47 PM

I wonder if global warming experts, would mind if the temperature data was collected by skeptics, in the same fashion.

Now that is hypocrisy.

Oct 11, 2006 07:50 PM

the study's conclusions are obviously false/inflated.

these false/inflated numbers may not have been premeditaed, but due to a faulty model yielding a result which the biased researchers liked.

here's how that might have happened:

the study is NOT based on death certficates or a survey of actual burials. the study was based on INTERVIEWS.

the interview questions might have been ambiguous.

perhaps those interviewed related the number of deaths per EXTENDED family (and not "hiousehold").

IF the researchers interviewed members of the same extended family (but who were not in the same household), then they might have double-dipped, so-to-speak.

EXAMPLE: i lost someone in my family on 9/11, but not in my household. if a researcher asked me if i lost a family member on 9/11 i would answer yes - as would members of several households. all for the same solitary death.

we all know that ambiguous poll questions yield bad results. if these bad results jibe with the biases of the pollsters, then they are often not subject to further review. to critical analysis.

in this sense, these false poll results are like the news-story that's too good to fact check.

so, the culprit is two-fold: (1) a bad method for elucidating the facts (polls are not as good as death certificates); and (2) the bias of the researchers - (which has been thoroughly documented: the editor of lancet is a george galloway BDS afflicted left-wing wacko.)

and there's always the possibility that the biased researchers/pollsters deliberately asked ambiguous questions which would allow them to "stretch the truth"?

Oct 11, 2006 08:04 PM

Let's see...655,000/1500 days = 436.67 deaths per day, every day, since the statue fell. That's over 13,000 deaths per month. Yet the newspapers have been shreaking the headlines that recent months have seen the highest death tolls ever, with over 3,000 to 4,000 people dying per month.

Seems there's a difference of about 400% (at a minimum) in the study's figures from reported numbers.

Iraq Casualies - http://icasualties.org/oif/IraqiDeaths.aspx - puts the death toll for Iraqi police, military and civilians from Jan 2005 to the present at 14,328, based upon news reports.

I guess that means that the other 273,872 deaths weren't reported. Wonder what they did with all those bodies?

Oct 11, 2006 08:05 PM

Ya see, Steve, Omar is, in a word, pissed.

Pissed that the sacrifices of the Iraqi people are being diluted for political purposes. Pissed that his national struggle is being blatantly lied about. Pissed that his cousin, or sister, or father-in-law, or best friend, has died and is now used as a cog in the oh-so-western political bullshit machine.

Those Iraqi dead are people, and to use them, and exagerate them, in this manner is grotesque. Ya see, Stevo, he's pissed, and he has a reason to be. Let the man vent a little before you fall back on your "rationalism". Which it isn't, because you can't even see that he's pissed.

Oct 11, 2006 08:08 PM

In order for this figure to be correct, there would have to have been 512 deaths per day since March 2003. Even the most fierce war critic couldn't come up with numbers like that.

Oct 11, 2006 08:09 PM

The criticism is completely valid. If the Lancet wanted to count the actual bodies, they could do so in a now-free Iraq. By contrast, nobody knows how many people have died in Kim Jong Il's deathcamps, and those who should care don't care at all. They worry about the U.S. possibly doing something to upset Kim Jong Il's regime.

Let's say, arguendo, the numbers are accurate. Are less people going to die if the U.S. pulls out precipitiously leaving a power vacuum to be filled by pan-Arab Baathist Sunnis, Iranian-backed Shia, and Takfir Wahhabi jihadis all hoping to dominate a chaotic Iraq? No.

Why not consider the possibility that those who murder people in cold blood and dump their bodies are the ones responsible? Yes, they aren't Americans and you can't pin everything on George Bush. But let's try it anyway. Consider it a thought experiment.

Oct 11, 2006 09:06 PM

I'll shoot a hole in the Lancet Study.

The mortality rate in the EU is 10.10/1000.

The Mortality rate in the US is 8.5/1000.

The mortality rate in Hungary is 13/1000

The world average mortality rate is 8.5/1000 per year.

The Lancet study uses a "baseline" mortality rate of 5.5. Half the mortality rate of Europe.

After all their fancy interviews...the Lancet Study comes up with a mortality rate for Iraq that is statistically the same as Hungary.

I must have missed it, but Hungary hasn't been at war for decades. The EU has been at peace for 60 years. They have the worlds best healthcare.

But their baseline mortality rates are more than double Saddams Iraq.

It is really easy to come up with a huge "excess deaths" number if one believes that no one ever died of anything in Iraq prior to 2003.

Iraq was an idyllic socialist paradise...unequaled since the Garden of Eden during Saddams reign.

Just ask Saddam...and not those pesky Kurds or Shiites that were mass murdered.

But then...so was the Soviet Union except for those 10's of millions that just magically "disappeared".

There is no lie too great that the hardcore leftist communists wouldn't tell.

Oct 11, 2006 09:19 PM

So they did interviews in 47 clusters, 12 of them were in Baghdad, 3 were in Basra, and 3 were in Anbar. Approx 25% of their estimate comes from Baghdad (only 21% of the population is in Baghdad), and all of their interviews came from populated areas where they could interview people (so word-of-mouth of their purpose would help keep the interviewers safe) but also where the violence is concentrated.

No wonder they think 650,000 people died.

Further, they say 31% of violent deaths were caused by Americans with a 95% CI of 26-37%. 24% of deaths were caused by "other". Guessing that the CI for the "other" violent deaths is 19-30% means that there is a good chance that "others" killed more Iraqis than the Americans, assuming the interview responses assigning blame were accurate. (45% were unknown)

Seems to me that the simple solution is to find and count the death certificates:

"At the conclusion of household interviews
where deaths were reported, surveyors requested to see a
copy of any death certificate and its presence was
recorded. Where differences between the household
account and the cause mentioned on the certificate
existed, further discussions were sometimes needed to
establish the primary cause of death." (page 2 - right above the part where they say that MS Excel was one of the programs they used for data entry and analysis - sorry to admit my bias against Excel for stats)

Additionally, this line:
"Thus, the data presented here validates our 2004
study, which conservatively estimated an excess mortality
of nearly 100 000 as of September, 2004." is wrong. This study does not validate their earlier estimate. This study verifies that they used the same procedure in 2006 that they did in 2004.

Finally, "Across Iraq, deaths and injuries from violent causes
were concentrated in adolescent to middle age men." (page 6) kinda rules out collateral damage as the cause of death.

I could go on, but really, what is the point?

Oct 11, 2006 11:46 PM

You know, just because someone does some polling doesn't mean their conclusions are correct.

First, it's a 95% confidence interval...that still leaves the 5% of the time where its' not so accurate.

Secondly, the way the question is asked and the extrapolation could all be problematic.

Obviously, this study is not passing the smell test, but we'll have to see where it went wrong or where it went right.

If they checked death certificates as they claimed why doesn't the Iraqi government confirm, especially when the health ministry in question is in Sadr's hands?

Oct 12, 2006 12:15 AM

I read on the internet that this whole "War in Iraq" thing is just a hoax George Bush is using so he can increase government spending.

Oct 12, 2006 01:27 AM

Aaron:

If I understand it correctly they checked out the death certificates of the 547 people they used to base their politcally motivated claims on.

Oct 12, 2006 03:08 AM

Unfortunately, Mr. Fadil's comments are a mile wide and an inch deep. I too was skeptical, but after reading the Lancet article and the study methodology, I'm on board. My own take his here: http://search.blogger.com/?as_q=numbers&ie=UTF-8&ui=blg&b l_url=kikoshouse.blogspot.com&x=11&y=3

Oct 12, 2006 04:06 AM

If death certificates prove anything, why don't they just find out how many death certificates have been issued over the last three-and-a-half years?

The numbers are absurd, because, as has been pointed out, there aren't 650,000 bodies (or any number remotely approaching that), and because for that many deaths you'd expect a couple of million wounded, which is also not the case.

If the statistical predicion does not match the facts, it's not the facts that are wrong.

Oct 12, 2006 04:13 AM

To quote Aaron Levenstein,
"Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital."

Oct 12, 2006 07:24 AM

I address the study here:

http://burkeophilia.blogspot.com/2006/10/new-report-on-de aths-in-iraq-new-study.html

While the mathematics and survey sampling design are not problematic, in my opinion, the research design certainly is worthy of scrutiny. I'm not convinced that their model isn't sensisitive to outlier clusters, I'm not convinced their choice of a baseline interval was appropriate, and I'm not convinced their extrapolation provides us with a meaningful result.

Other than that, it's fine. ;)

Oct 12, 2006 07:35 AM

The mortality rates in the developed world are higher because we have much older populations.

However, the Iraq mortality rate used is certainly on the low end. That's important because the model is sensitive to changes in that baseline mortality rate.

That their study has a substantial Baghdad component is certainly disconcerting. I noted in my critique that I think its possible they over-sampled hot-spot areas...thus resluting in their inflated number.

Given two numbers...one which is an extrapolation from a survey conducted in Iraq where respondents were asked about deaths as much as 4 years in the past...the other of which are actual body counts from hospitals and morgues...I'd go with the latter and not the former.

Oct 12, 2006 07:42 AM

The sad thing about all this is that I bet we all *still* can't agree that, 200,000 or 650,000, a horrific number of people have died because of the dishonesty and incompetence of this administration.

Oct 12, 2006 07:55 AM

1) The Lancet points out that the passive counting method, which has given us about 50,000 deaths, misses many deaths in a conflict area, because not all deaths are reported.

2) To confirm that people weren't making things up, the Lancet study asked for and received Death Certificates in over 80% of cases.

3) It apparently has not occurred to the Lancet that if Death Certificates are available in 80+% of cases, then passive methods (i.e., counting up Death Certificates) should in fact be quite accurate.

Oct 12, 2006 08:10 AM

Why would we characterize it as 'incompetent' if the U.S. military is particularly effective at killing insurgents? Again, the Lancet study makes no effort to distinguish between collateral damage and enemy casulties.

Oct 12, 2006 08:15 AM

The simple fact is however you count it at least 100,000 have been killed since this war started.

Now who am I going to believe Johns Hopkins and Harvard researchers or George Bush.

Even the Deputy Health Minister Adel Mohsin who thinks things are going swimmingly admitted that that more than 2,660 civilians were killed in Baghdad alone in September, despite a significant US-Iraqi military operation to curb the violence. That is one month one city and he also admitted that 'it was possible that as many as 50,000 people had died because of insufficient healthcare facilities'

I see Iraq Body Count quoted here, they only use publically quoted figures in the media which clearly does not report on every death in Iraq and undercounts by a huge margin.

So it is probably not 45,000 and probably not 600,000 but somewhere towards the middle. A HUGE number by anyone's count and to think we were going in to make their lives better.

Oct 12, 2006 08:16 AM

Jeff writes that the sad thing about all this is that I bet we all *still* can't agree that, 200,000 or 650,000, a horrific number of people have died because of the dishonesty and incompetence of this administration.

Everyone should understand the initial deaths belong at the doorstep of Saddam, who defied 17 UN resolutions. THIS administration merely enforced the UN resolutions. Had Saddam complied with the resolutions, there would be no deaths due to this war. Since the end of Saddam's regime, the coalition has provided security so Iraqis can govern themselves freely and democratically. The blame for this bloodshed belongs on the terrorists and so-called insurgents and religious fanatics who don't get that there is a legitimate government in place and it is unlawful to kill people as a means to show their displeasure with that. Any blood the legitimate government and its coalition allies must shed to defend itself and the Iraqi people is the fault of the attackers. These deaths are inflated, but whatever their numbers, they are not the result of any democratically elected administration's alleged "dishonesty" or "incompetence." They are the result of criminal blood-thirsty acts by evil people who would put all but their cronies in chains, if they held power. To miss or seek to dismiss this obvious fact is infuriating. This is why Omar is so angry with this pseudo-scientific report. And he is right to be angry.

Oct 12, 2006 09:00 AM

"A HUGE number by anyone's count and to think we were going in to make their lives better."

Well, as cold as sounds, we weren't going in to make their lives better, we went in to make our lives better.

I think we succeeded in that, i.e., thousands of jihadists from all over the world have been killed in Iraq, and one pillar of the "axis of evil" is down.

Whether the total loss of lives will ultimately benefit "moderate" Iraqis is yet to be seen.

Oct 12, 2006 10:05 AM

1) The Lancet claims that their active survey is more accurate than a “passive” system of counting media reports, morgue reports or other lists of the dead (and generally totalling about 50,000 dead in Iraq), which are often grossly incomplete in a war zone. This seems reasonable.

2) To make sure people weren’t making things up, The Lancet asked for and received Death Certificates 80% of the time. Also reasonable.

3) But The Lancet fails to see that if 80% of the deaths were recorded by Death Certificate, then a passive accounting of Iraqi Death Certificates should give an accurate accounting.

4) The 95% confidence intervals are pretty tight for this study (meaning no more or less than that the random sampling error is small). So could there be any explanation other than stupidity or bad faith to explain how their death figures could be an order of magnitude higher than the passive counts?

Oct 12, 2006 10:18 AM

This report reminds me of the bogus data about suicides among viet nam vets. In that case, the number of suicides outnumbered the actual number of soldiers,sailors, airmen, marines who served there. As Omar says Big surprise this comes out 4 weeks before the mid term. I guess the Foley flap isnt paying big enough dividends

Oct 12, 2006 10:36 AM

Manual trackback

Oct 12, 2006 11:33 AM

I'm not too familiar with the study, but I am a scientists, for what it's worth. But it doesn't take a scientist to realize that a survey of 1000 people and extrapolation to an entire country of ~25 million isn't statistically sound. And it was reported in lancet, which is a fine peer-reviewed journal. I think they should increase their sample size alot. There are alot of variables that make this sort of thing troublesome, and certainly not extrapolatable to the entire country. It would be like going to Harlem and getting the body count then saying the US murder rate is such and such. Such statistics are very much region dependant, population dependant, etc. But at least this points out that further epidemiological study might clarify the issue (but won't do a damn thing to stop the killing).

Oct 12, 2006 12:54 PM

I don't have a problem with Omar's response. Already, plenty of number jockeys online are doing analyses and debunking the methods and conclusions. However, not many can say what it feels like to have people you've lost (as I know his family has lost) baldly exploited for an agenda and their numbers bloated as if the reality of their actual deaths didn't mean enough. That's his rarer story to tell, and among all the necessary factual debates, I was glad to read it.

Oct 12, 2006 02:32 PM

"J'accuse"?

yeah, you're just like emile zola, only he didn't do anything as naff as criticise people for "resort[ing] to mathematics".

Oct 12, 2006 03:29 PM

I am a student taking an advanced stats class. I brought this to my professor's attention last night and today he assigned the study, giving me credit for the extra work, to the entire class. He saw no problems with the framework of the study. The set-up was sound and used existing methods. I don't understand Omar Fadil's comments. He attacks the messengers, not those that are actually killing and above all, those that are dying. Iraqi Body Count admits that its figure of 50,000 Iraqi deaths is way to short, because all they count are the bodies that make it to the morgue. None of the civilians bodies arrive at the morgue. Muslims bury their dead very quickly, sometimes within hours which is way to fast for the authorities to count them as dead. Anthony Cordesman, who is quoted as the expert questioning the 600,000 is not a statistican but a quasi-politican who, among other titles is Weapons of Mass Destruction expert at the Center for Strategic & International Studies. I wonder if he has ever been wrong before? I'll let you know next week what my professor thinks about the methology of the study and HIS expert take on the possibilities of right and wrong.

Oct 12, 2006 04:07 PM

Stop blaming the Lancet for this study. The Lancet is a journal.
It just publishes the article, it doesn't write it, or endorse it. It's like blaming the media for all the Bush Administrations lies and distortions. Scientific journal always check the facts and verify the science they publish. Just as the media always refutes any clearly ridiculous statements made by GW.

Oh. Wait...

Scratch that.

Death to the Lancet!

Oct 12, 2006 04:34 PM

I question why anyone would take seriously a study where the margin of error is +/- 200,000. Back when I was majoring in economics any of my professors would have laughed me out of the classroom if I came up with something like that. "Give or take a few hundred thousand" didn't cut it with them.

To illustrate how absurd the numbers are: The population of Iraq according to this site is 26,783,383. If you look at the margin of error of the study, the margin of error alone is .7% of the total population of the country.

Not only that, according to the UN Development Programme Iraq Living Conditions Survey 2004 : "The Living Conditions Survey data indicates 24,000 [war related] deaths, with a 95 percent confidence interval from 18,000 to 29,000 deaths"
A compilation of deaths reported by news agencies shows the number at 14,328 for the period of Jan 2005/Oct 2006.

Oct 12, 2006 04:53 PM

Stop blaming the Lancet for this study. The Lancet is a journal.
It just publishes the article, it doesn't write it, or endorse it.

I got some nice swampland in Florida for sale? Anyone interested?

Can I interest you in a classy vintage bridge in Brooklyn?

Oct 12, 2006 06:06 PM

Stik looked up a brief history of Lancet "scientifically neutral" articles. My favorite one was on "near death experiences".....it came to the conclusion that NDErs had become much more empathic and accepting of others since their NDE.

Sheesh....Groundbreaking...

Oct 12, 2006 06:46 PM

I am a sociologist who has been looking closely at the Lancet study and wanted to say that I find many of the comments useful here, as I craft a critique of the Lancet study. So thanks, I think there are many astute critical observations. A few points, some general, some particular, as I think through this.

1. I supported the war on human rights grounds (have written about this extensively so won't go into it here), and at the time, I was aware of much research that indicated that the majority of the people of Iraq welcomed the invasion to be liberated from Saddam. Subsequent events illustrate high support for democracy, elections, etc. And to be honest this has waned to an almost reverse situation where now most Iraqis support attacks on US troops. Anyway, I literally begged my colleagues to look at this data and they completely ignored it, and in so doing denied the desire of Iraqis to be free. They hated Bush so much that it didn't matter to them that oppressed people (whom they would have supported if Bush were oppressing them) wanted our help and assistance. Now, what is ironic is that they are frothing at the mouth to go in and document death and destruction which legitimates their anti-war sentiments. Why didn't they go do research two years after the war, when times were relatively good, and collect data? Because they didn't want to find that the Iraqis mostly supported the war.

2. From what I know about this sampling, the gravest error was that they should have seperated Iraq into three regions and then sampled the same way within these regions: Kurdistan, Central Iraq, an Southern Iraq. They would have found virtually no excess death in Kurdistan (in fact, maybe even an overall improvement), in Central Iraq, probably something of the order of magnitude they actually did discover, and in Southern Iraq, much less than in Central Iraq. To have 25% of the sample be from Baghdad and extrapolate to, say, Kurdistan, is like taking the crime rate from Washington DC and extrapolating to Montana. This is very bad methodology and indicates to me that the researchers, who are partisan without question (see some of their interviews in defense of their study on line, they are loaded with political positions).

I am writing a critique of this for our new movement website: www.newamericanliberalism.org, so if anyone had further ideas or resources, please send them to me, I'd be grateful, and once again, thanks,

Tom

Oct 12, 2006 07:07 PM

I think Omar's anger is not only excusable but appropriate.

For the numbers folk: 600,000 people x 60" tall x 18" wide = 94 football fields (or 103 acers or 65 city blocks) packed with dead people. Surprising no one noticed until now.

For anyone following the Korean situation, this is good news. If we're trying to HELP in Iraq and end up killing 600,000 people, North Korea's 1.5 million man army doesn't look so tough - we'd be trying to kill them, not help them.

Oct 12, 2006 07:26 PM

"Omar was assuming any serious person could do the math to quickly see how ludicrous these numbers are. "

Well, I have a degree in maths and I read the study and I couldn't see any major errors in the methodology. Want to point them out if they're so FUCKING OBVIOUS?

Oct 12, 2006 08:00 PM

So it sounds like they did knock on doors and ask.

Could someone please write a *real* response to the study that actually critiques their methodology instead of their politics?
***********************************
They used a methodology known as "cluster sampling," which can be valid if using real data and not anecdotal reporting. Most of the original Lancet clusters reported no deaths at all, with the journal admitting, "two-thirds of all violent deaths were reported in one cluster in the city of Fallujah." Fallujah? Hello?

Fallujah at the time just happened to be a major concentration of pro-Saddam and anti-American sentiment, the home base for the homicide bombers and terrorist "resistance" before the U.S. Army and Marines cleared out that nest of thugs.
***********************************
That do it?

Oct 12, 2006 08:28 PM

I haven't looked at this study much, but the previous study by the same people had serious problems that would not be shown simply by examining just the statistical methods, although the error ranges then and now are absurdly high.

They oversampled regions where people had an incentive to make the body count high. This time they took a lot of their sample in Baghdad, and another cluster in Anbar (which is notorious for its Sunni insurgency and Al Qaeda activity) - again oversampling the highest violence areas. Many folks in these regions have incentives to lie - to create higher death rates. Many are anti-freedom Sunni, and in Baghdad's Sadr City, there are many Iranian-fomented Shia fanatics. In other words, the study appears to be designed to sample the areas of highest violence density and most unfriendly population.

The previous study used unreliable surveyors - including many with reasons to lie (did they *really* see a death certificate?). Do we know that they used trustworthy ones this time?

Furthermore, a disproportionate number of reported deaths were of those of fighting age. What a surprise? They may be measuring our ability to kill enemies.

The first study, also just before the election, which was widely reported as saying 100,000 had died, had error bars from 5000-180000 (approx)! The researchers just picked the average and used it in their summaries.

The timing of the study, the strong indications of misbehavior of the same researchers the first time, the obviuos sampling flaws, and the known bias of Lancet - all of these indicate that this study should be considered a production by the Michael Moore's of the sociology trade.

Oct 12, 2006 10:34 PM

Very curious. In 1938, Lewis S. C. Smythe, an American in Nanking, performed a similar sort of door-to-door survey and got a much smaller "death toll" figure than is accepted today by people with a motivation to inflate the Nanking atrocities death toll as much as possible (i.e., Chinese communists and others who have done no primary research of their own).

Today, few "researchers" know of or directly cite this research or its findings and methods precisely for this reason (body count is "too low").

Although saying "this research is bogus" because the authors seem to have an interest in a specific outcome is easy, these kinds of responses won't do. The methodology must be rigorously examined and if it's bunk, it will be very obvious.

To date, nobody has debunked Smythe's work, and it stands as the only useful and quantifiable piece of research done on the death toll in that city. Yet it is largely unknown. Iraq must not become another "Nanking", only in reverse.

Oct 13, 2006 12:00 AM

Some of the objections here are sound. Some are just emotional blowing off steam. "How could this be?!?!" And some are just wrong. Two very wrong objections are: where are the bodies? And why don't they count death certificates?

Well, there are two easy answers to those questions. In Muslim culture, it is required that you bury the body before the end of the day. It's the same reason why major US attacks against insurgent forces end up with virtually no dead bodies, despite definitive claims from US soldiers that they killed dozens of insurgents. The bodies are buried immediately. They aren't embalmed. And they are rarely sent to a morgue. So it's not hard to answer where the bodies are: they were buried right away.

The objection about death certificates assumes that there is a neutral, functioning health ministry that can sign death certificates for every murder. Well, the Health Ministry is controlled by the Sadrists, and most Sunnis want nothing to do with them. Again, death certificates only represent a fraction of overall deaths in modern Iraq.

A better criticism is that the report over-samples Anbar. But Basra has not exactly been a peaceful place either. In fact, nowhere outside of Kurdistan has been peaceful, so it isn't hard to imagine rampant killings elsewhere.

Another sound criticism is that it assumes more peace in Saddam's time than is warranted. But I'm not sure how much of a difference it would make if we adjusted that upward.

But let's see if this passes the believability test. We know there are 110 reported deaths in Baghdad every day. Considering the suspicions of health ministry officials, it's safe to assume that the actual death toll is closer to 200 a day in Baghdad. Then there are other war-torn cities and towns like Ramadi, Samarra, Baquouba, Kut, Diwaniyah, Mosul, Kirkuk, Fallujah, Basra, Tal Afar, Tikrit, etc. Some of these places are more violent than Baghdad. Some are less violent. Taken together, is it hard to imagine 400 killed a day in all of these other cities combined? I don't think so.

655,000 deaths is a shocking statistic. But it isn't shocking because some researcher wants to exploit Iraq's misery. It's shocking because Iraq has fallen into utter lawlessness. You don't a Battle of the Shiloh to produce deaths on this magnitude. You just need hundreds of ruthless militias given free reign to carry out revenge.

Oct 13, 2006 07:32 AM

Tom: Have sent you a mail. If you don't receive it, could you let me know? Cheers.

Oct 13, 2006 09:10 AM

I was lookng for a critique of the Lancet. Not here. This post said less than nothing.

Oct 13, 2006 04:22 PM

Some excellent comments here which point out some of the flaws in the study. The methodology is truly laughable, but what troubles me is that they took the highest possible number their methods could produce and have released it to the media which is now trumpeting it as if it were legitimate.

My debunking of the methodology can be found on blogcritics.org.

Dave

Oct 13, 2006 04:42 PM

The three levels: Lies, Big Lies a
Statistics...

Oct 13, 2006 05:03 PM

I hope Omar understands that this is a scientific journal and not a newspaper.

As an Iraqi I think this number is an approximate of the number of deaths and martyrs.

I have a lot of families and friends in Iraq and there are hardly any Iraqi families who havent lost at least 3 or 4 relatives since 2003 in violent activities, with the exception to people in the 3 Northern provinces.

The UN confirms that 100 people a day are reported dead. What about those who haven't been reported ? If only 20% get reported, you do the maths.....

and Omar you don't have to be a puppet and defend the Americans, unless you are an agent or is being paid stocks of cash to keep you praising the Americans and their invasion of Iraq.

Oct 14, 2006 03:27 AM

This idiot Omar, never read the report, before he write such crap he should know that:

1- Lancet is part of “John Hopkins University”, which ranked #2 in the world.

2- The team did their investigation on the ground in Iraq, this is something not even the Iraqis did.

3- UNHCR said that at least 1.8 Iraqi refugees around the world (virtually dead), the number confirmed by the Iraq Immigration Ministry (reasons for fleeing Iraq?)

Oct 14, 2006 06:58 AM

This is just another example of the uneducated graduates colleges around the world have turned out. They aren't capable of doing anything constructive so they lie their way through life.

The United States currently has more lying and unqualified for the job professors in the education system than at any time in history. They knew they couldn't make it in industry where you have to prove what you say by actually 'producing' a result, so they stayed as 'teachers' so they could brainwash your children and lie about what they don't know.

At what time in history could you spend $46,000 + per year for an education, spend four years listening to a left wing liberal idiot, er professor and graduate with a deficit in knowledge. You can in the United States today at 75% + of the major colleges.

Oct 14, 2006 10:03 AM

1) The U.S. government has lied consistently about the war, WMDs, likely reaction of Iraqis to invasion, cost, current state of affairs, who we are killing. Should they have any credibility.
2) The society is in a state of anarchy. Assuming someone dies violently, do you bring information to authority that may be responsible, or quietly bury. Remember U.S. authorities don't want this bad news out.
3. U.S. press is in green zone, Iraqi press being killed regularly, where is the correct information coming from.
4. Is it unrealistic to expect a 3.5 year war/civil war to have only 40,000 casualties. Especially when civilians are target.

Oct 14, 2006 01:27 PM

600,000 dead egh?

I'd thought you guys would have that up to around 1 million by now...LOL

Oct 14, 2006 01:44 PM

Well said Karim

Oct 14, 2006 06:50 PM

Read the report for yourself. The data collection methodology -- a sampling of 47 randomly selected sites involving all but two Iraq provinces -- is about as good as it gets in this situation. Yes, they interviewed people, but also verified answers by examining death certificates provided by approximately 80% of sample respondents.

The estimate has a fairly wide range. However, the possibility of the true number of deaths falling in that range is 95%. And, the probablility is highest that the true number is around 650,000.

Mr. Fadil's anger is understandable. But, facts are facts.

Criticize the timing of the report all you want. I assume most conservatives would prefer the data not come out at all -- or be released after the elections? And how would either of these possibilities provide for an informed electorate?

Oct 15, 2006 04:57 PM

"Yes, they interviewed people, but also verified answers by examining death certificates provided by approximately 80% of sample respondents."

OK, so they say their numbers are legit becuase they got 80%+ death certificates...

Assuming this data actually is representative (and it has to be, or the whole study is worthless), that means that less than 20% of the deaths don't get a death certificate.

That means that the "passive" method (counting death certificates) represents about 80% (or more) of the "real" death number.

Simple enough for you, Mr. "I have a degree in maths"?

Not to mention that the people LIVING in Iraq question this because of "where are the bodies"?!?

Yeah, you westerners can just tell them all about their own culture now. Burying bodies quickly, and all that. Arrogance, redefined.

600K is about 2.5% of the ENTIRE population of Iraq. That's one person in 40.

THOSE are the OBVIOUS NUMBERS that he's talking about. This story, just like the previous one this guy put out, is lies for the sake of politics, and he practically ADMITS it. Go read his comments.

There's a word for people who believe this stuff.

Oct 16, 2006 09:06 AM

While it may be reasonable to assume that there have been twice as many deaths as reported, is it reasonable to assume that the actual number is more than an order of magnitude greater than reported? What about Omar’s observation concerning the ratio of injured people to those killed? Certainly the ratio must be greater than two to one. If so, where are the 1.3 million wounded people? No doubt the math in this report is good, but the conclusions just don’t seem logical.

Oct 16, 2006 11:02 AM

The Burnham paper is incredibly flawed. While its methodology is about as good as you can expect for a war region, and the maths/statistics are fine as far as I can tell, it really doesn't matter if the authors get it wrong in the end...which they do. And you don't have to look at the controversial numbers (violent deaths) to see how wrong they are.

Several comments on here have asked for a critique of the authors' methodology. Some have responded by pointing out, among other things, the difficulty of extrapolating incredibly small sample sizes to an Iraq-sized population. While this is a valid concern, we have to remember that this is epidemiology, not "real" hard science. The METHODS in this paper are flawed, but no more so than those used in most epidemiological studies, or, for that matter, in political polling. We can find fault with the regions in Iraq the authors' survey teams selected (leaving out, for example, two of the least violent Governates in the country; oversampling urban regions; oversampling regions afflicted by high crime and sectarian violence); but the methodology at this level is no more flawed than in most epidemiological studies.

Similarly, I can find nothing wrong with the authors' statistics. Given their small sample size, the 95% confidence interval is wide--shockingly wide to those unfamiliar with basic statistics--but it is really no surprise for a seasoned statistician. I don't see much that's wrong with the paper's figures and math (except for a little insignificant gaff where they refer to three 13-month periods as 14-month intervals).

So what is wrong with the paper? Two major things. First, the conclusions. Second, the bias introduced by the authors.

1) Conclusions. Forget about the violent death numbers for a second. According to the UN Population Division, deaths in Iraq probably total around 250,000 per year. This has been the case since the mid-90s, and apparently hasn't changed. So the author's death numbers--if you look at them as total deaths (not "excess deaths")--are not implausible.

What about specific categories of death that the authors do record? Say, a category that's not linked to much controversy? The UN Population Division and UNICEF place infant mortality in Iraq at right around 100 deaths per 1000 live births. This has been the case for every reported year since 1998 and remains unchanged post-invasion. The CIA factbook gives the lowest mortality estimate I've seen, at about 50/1000 births. The Hopkins authors find only 40 infant deaths out of 1474 births during their study period, or only 27/1000, which would put Iraq right on par with Iran, Syria, and Brazil, among others. But the figure of 27/1000 is only half of the heretofore lowest mortality estimate, and only a quarter of the UN figure, which is fairly solid and based on censuses and surveys. The authors's oversampling of urban populations, which suffer lower infant mortality but higher crime-, politics-, and religion-based violence than rural areas, has led to a massive underreporting of infant mortality. The same is true for child mortality. In addition, the birth rate given by the authors is a fraction of birth rate estimates from any other source.

These outcome problems are important, because they show how wrong the authors' conclusions are.

But if a figure of even 750,000 deaths post-invasion is plausible (but due mainly to non-violent deaths), why did the authors find violent death numbers an order of magnitude higher than those given by other (even other anti-Iraq war) sources?

The answer is simple:

2) BIAS.

Bias is the overwhelming source of the paper's problems. The authors, their survey teams, and the respondents all have so many different potential types of bias that this survey is hopelessly conflicted.

As I have written elsewhere, whenever a Western researcher with little or no local linguistic or cultural fluency uses a local research team to investigate a controversial topic in an underdeveloped country, the data are unverifiable and amount to hearsay. The subjects often tell the survey team what they think they want to hear, and the survey team consciously or unconsciously directs the survey and redacts/compiles the results in such a manner as to obtain what they think the Western supervisors want to hear. This is how epidemiology works. Most of the time, the resulting inaccuracies are benign. If I go to an underdeveloped African country and hire local teams to survey rates of AIDS, I will probably end up with rates much higher than actually obtain in the population. This is because it is prohibitively expensive to verify infection by multiple molecular diagnostic tests. So instead, "surrogate markers" are used, such as tuberculosis, wasting, diarrhea, etc., etc. Surveyors ask, "how many of these conditions have you (or someone in your family, perhaps a now-deceased person) had in the past year?" (or whatever). If enough of these conditions are present, the sick person is categorized as having HIV/AIDS. The educated people who administer the survey also realize that the higher the rate of infection, the more dire the health emergency, the more money will flow into the country through NGOs and foreign government aid programs. In fact, they will probably benefit too. The poor, uneducated people who are recruited for the survey will similarly realize that some answers are better than others. So I get a gross overestimate of the number of people who are actually infected with HIV that has progressed to AIDS. But who cares? No one gets hurt, rich people give more money to poor people, and I and my research teams reap the $$ and also know we're helping people, whether or not their sickness is AIDS.

The difference between the AIDS example and the Burnham paper is that few AIDS researchers actually WANT to find higher rates of AIDS than actually exist. The bias is introduced mainly at the local level. In the Burnham paper, at least several of the authors are outspoken anti-war activists who have actually STATED that they did this work precisely because they KNEW in advance what they would find. Here's Gil Burnham, interviewed on the 2004 paper: "we wouldn't go to the effort of doing something like this if we didn't feel that here was a situation that was egregious and, you know, there really needs to be some attention to what we can do to better protect the civilians." Hardly the stance of a quasi-objective scientist.
Nor does Les Roberts, the investigator who spent some time in Iraq, have any apparent capacity for concealing his political views. He actually ran for Congress on an anti-Iraq war platform, raising $160,000 before dropping out in the spring, some of it from his epidemiologist colleagues--you know, the people who comment on his article in newspapers and on TV, and the people who "peer review" his work.

The lead investigators make their bias known to the actual researchers (who are paid by the investigators). The survey teams are then free to exert their own biases in the field. And respondents, most of them in areas characterized by violent opposition to the coalition, understand the bias in the survey and give their own answers accordingly. One can see how nonviolent deaths would suddenly turn into violent deaths due to coalition actions, and how pre-invasion, non-violent deaths would be shunted into the post-invasion period by angry family members helping the survey teams get the results they want.

As for the death certificates mentioned in the paper, I am skeptical. The authors misleadingly state that "92%" of the deaths were verified by certificate. Actually, in only 80% of cases were certificates "present" (547 out of 629 deaths) and the authors do not describe whether or how these certificates were analyzed for authenticity. They note that "discrepancies" were sometimes present, but don't go into any detail. It is not clear to me how many of the "present" certificates were examined, and it is possible that the survey teams only queried the presence of the certs in a subset of the cases.

In any case, absolutely no "uniqe identifiers" were collected. This means that ABSOLUTELY NONE of the data are verifiable. They amount to hearsay. And we see the accuracy of the results.

At the very least, The Lancet should have asked the authors to acknowledge their bias in their conflict of interest statement. Unfortunately, the Editor of the journal, one Richard Horton, is a more outspoken critic of the Iraq War and what he calls the "axis of Anglo-American Imperialism" than even the authors of this political paper. And so he let slide the journal's policy of full disclosure of potential conflicts of interest.

Oct 16, 2006 11:51 AM

I am familiar with the sampling and statistical methods used by the Johns Hopkins researchers. They are well tested methods that have proved reliable in estimating mortality rates in many previous studies. The accuracy of these figures can be debated but not be dismissed out of hand. I thought Omar was supposed to be a dentist. If so, I am surprised your training did not instil in you any appreciation of basic scientific methods. Your article is not in any way a serious critique. I will look elsewhere to find some serious and intelligent analysis of this questionable data.

Oct 17, 2006 07:35 AM

Most useful is the very thorough analysis at Iraqi Body Count. See the IBC report for the details. Just click here. (http://www.iraqbodycount.org/press/pr14.php).

According to the IBC, for the Lancet study to be correct, the Iraqi Ministry of Health would have had to have issued about 500,000 death certificates more than they recorded. That's half a million missing death certificates.

Also, for the Lancet report to be true, over half a million people were injured, but not killed, by blast effects yet did not go to the hospital. There must be at least twice as many injured than killed, which would require over a million additional Iraqis with severe burns or severed limbs who are not only failing to complain but are in hiding. That's too many to believe.

Oct 17, 2006 09:52 AM