Nafeez Ahmed’s smear piece on IBC – part 1

June 2, 2015 – I’ve read many ugly smear pieces, from both right and left – but few worse than Nafeez Ahmed’s recent article on Iraq Body Count (IBC). This was circulated as “investigative journalism”, but most of the material presented has been available on the web for years, and the shocking, conspiratorially-framed claims that Ahmed makes about IBC rely more on misleading rhetoric than on facts.

One of Ahmed’s sources, a statistics professor who writes for the Washington Post, has since complained that he was selectively quoted and misrepresented by Ahmed’s article. Another source has complained of exaggerations and errors, suggesting that Ahmed was engaged in “personal attacks” on IBC and its researchers. Ahmed has now written a second piece which responds to these criticisms. I quote from both of his pieces in what follows.

Before I go through Ahmed’s false and misleading assertions, I’ll list some of the rhetorical phrases that he uses against IBC and “IBC-affiliated scholars”. You should read these in context, of course (if only to convince yourself that Ahmed is serious) – these snippets give some idea of the type of framing we’re dealing with:-

“IBC’s metric to whitewash war-crimes”, “fraudulent attacks on standard scientific procedures”, “statistical manipulation to whitewash US complicity”, “dubious ideological alliances”, “subordination of academic conflict research to the interests of the Pentagon”, “misleading pseudoscience”, “the IBC’s directors are selling casualty recording as a way to legitimize military operations, and increase the effectiveness of counter-insurgency responses to armed resistance.”

Ahmed’s misleading assertions & falsehoods

Ahmed repeatedly conflates IBC and its output with “IBC-affiliated” persons and projects, leading to false claims about IBC’s funding, misleading assertions about “fraud” and nonsensical inferences which read more like conspiracy theory than logic. Remarkably, he asserts that “all” of IBC’s publications breach ethical standards by not disclosing certain funding:

“The breach is committed with such systematic impunity throughout their academic publication record that it does, indeed, raise serious and fundamental and perfectly legitimate questions about the integrity of their research.”

The funding (from sources “connected to US and European government foreign policy agencies and departments”) which Ahmed believes should have been “disclosed” by IBC and/or its publications was not, in fact, received by IBC or any of its publications. IBC’s funding is listed on its website. The “IBC”(-related) funding which Ahmed refers to has been openly disclosed by the publications/organisations which it funded – all non-IBC. There are no “egregious ethical breaches” by IBC here, but there is a lot of seriously misleading rhetoric from Ahmed. (Occasionally, Ahmed slips from specious bombast into clear falsehood, as when he writes of “USIP’s selection of IBC for funding”. USIP has never “selected” IBC for funding – more on this below).

Ahmed is also selective in the extreme, setting up a sort of “heroes vs villains” narrative, but failing to apply his reasoning consistently to both the “villains” of his piece (IBC) and the “heroes” (eg the authors of the Lancet 2006 Iraq survey). For example, he claims – falsely, as it turns out – that IBC’s work is influenced by funding from the United States Institute of Peace (USIP), and builds a picture of USIP as a “neocon front agency” whose “entire purpose is to function as a research arm of the executive branch and intelligence community”. He concludes, as a result of such claims, that IBC “is deeply embedded in the Western foreign policy establishment”.

But Ahmed doesn’t mention that USIP has funded and repeatedly collaborated with Gilbert Burnham, lead author of the 2006 Lancet Iraq study. Presumably his investigative journalism didn’t stretch that far. Most of the shocking inferences which Ahmed draws from “connections” with agencies such as USIP would also apply to Burnham, as we’ll see. (USIP has also funded other researchers used by Ahmed as authoritative sources – eg Professor Amelia Hoover Green).

Another example of Ahmed’s selective treatment of the “evidence” is provided when he cites the 2013 UCIMS epidemiological study (published in PLOS Medicine) as “corroboration” of the 2006 Lancet Iraq survey, but fails to mention that its estimate for violent deaths is far closer to (ie approximately matches) IBC’s figure. This fact alone tends to undermine the central premise of Ahmed’s article, since if IBC’s role is to “whitewash war-crimes” (Ahmed’s words), why would its violent-causes body count be in approximate agreement with the PLOS study (which was co-authored by Gilbert Burnham and vaunted as a methodological improvement on Burnham’s 2006 Lancet study)?

“Tapestry of connections” (aka conspiracy)

Ahmed attempts to implicate IBC and “IBC affiliated researchers” in what he calls a “tapestry of connections”. And he certainly weaves a remarkable conspiracy – his cast of characters includes “Senator Hugh Burns’ Fact Finding Committee on Un-American Activities”, “The Pentagon’s counterinsurgency geographer”, “Colombian paramilitary groups involved in drug-trafficking”, “Colombia’s state-run Central Bank”, “the Nazis” and “Nazi scientists” (among others). I struggled to see how there’s any connection between these and IBC, and, to be fair to Ahmed, he doesn’t imply there’s any real link with the Nazis. He simply mentions how the Nazis achieved success in the “abuse of science to legitimize war and sanitize death” – no doubt to give us some “precedent” for IBC’s work.

Most of this stuff is spun by Ahmed from “connections” to research by Professor Michael Spagat which has nothing to do with IBC (other than that Spagat has been an advisor to IBC). Ahmed attempts to (falsely) implicate IBC in these “connections”. For example:

Spagat’s early career connections to IREX and NCEEER, both conduits for US State Department propaganda operations, as well as to Radiance Technology, USAID, and USIP, raise serious ethical questions, as well as questions about the reliability and impartiality of his work, and that of IBC.

The grants to Spagat from IREX and NCEEER, which Ahmed refers to here, are from 1994-1996 (years before IBC existed) and relate to research on the “Transition to a Market Economy in Russia”. This research funding is listed on Spagat’s CV, which takes about five seconds to find on Google. Despite the obvious irrelevance of this to IBC, Ahmed asserts that it raises “questions about the reliability and impartiality” of IBC’s work.

Ahmed goes into some detail on IREX and NCEEER, presumably to convince us that they are “intimately related to US government propaganda operations”. For example, NCEEER’s Board of Directors includes one Richard Combs, and, “from 1950 Combs was chief investigator, counsel and senior analyst for Senator Hugh Burns’ Fact Finding Committee on Un-American Activities”. Ahmed continues:

Combs’ anti-Communist witch-hunt, supported by US intelligence agencies, led him to target black people and Muslims in America. As documented by Indiana University historian Claude Andrew Clegg, in the Burns Committee’s eleventh report to the California legislature, Combs’ report found the “Negro Muslims” to be “un-American” purveyors of anti-white sentiment in schools for African American children.

Combs played a major role in inaccurate and unwarranted persecution of the black civil rights movement and the Nation of Islam, which was mischaracterized as a conduit for a “Communist conspiracy” fostering “progressive disillusionment, dissatisfaction, disaffection and disloyalty.”

Recall that this is part of the material provided by Ahmed to convince us of something about NCEEER, which once gave a grant to Spagat (years before IBC existed) for research relating to the Russian economy. At the end of a section which is mostly about such non-IBC “connections” to Spagat’s research funding – eg from the 1990s, non-Iraq (except for a digression on the National Iraqi News Agency, which is one of IBC’s media sources) – Ahmed concludes:

This tapestry of connections between IBC affiliated researchers and the militaries of key NATO members, demonstrates that IBC’s characterization of itself as an independent anti-war group doing reliable and credible research on civilian casualties is false.

This type of tenuous guilt-by-association “logic” runs throughout Ahmed’s article. It’s the “logic” of smear. With similar rhetoric one could just as easily “implicate” the “heroes” of Ahmed’s piece. Gilbert Burnham, for example, has received research funding from the World Bank, the Afghanistan government, USAID (another agency which Ahmed depicts as nefarious), Procter & Gamble, etc – in addition to his collaborations with “neocon front agency” USIP, mentioned above (including on Iraq and Afghanistan). Imagine the “connections” here. What does all this say about “the reliability and impartiality” of Burnham’s work, including the 2006 Lancet Iraq survey? What does it say about the people he’s “affiliated” with?

Of course, it says very little by itself. It says even less when seen against Burnham’s work in its entirety. But that wouldn’t stop someone writing a rhetorical smear piece on Burnham by expanding on all the “suggestive”, “suspicious” connections in a misleading way – as Ahmed has done with IBC. In some ways – as we’ll see – there’s even more of this type of material available to throw at Burnham than there is to hurl at IBC.

One of the “connections” that Ahmed’s website detective work reveals regarding “IBC-affiliated” Professor Michael Spagat is the following:

In the acknowledgements to his 2010 critique of the Lancet study in Defence and Peace Economics, Spagat thanks Colin Kahl, indicating that a senior member of Obama’s Pentagon reviewed his manuscript before submission and publication.

It’s a pity that Ahmed didn’t apply the same “investigative” technique to the UCIMS/PLOS Iraq study (the one that was co-authored by Gilbert Burnham and vaunted as an improved update to the 2006 Lancet study). He would have found that a certain Skip Burkle “reviewed the manuscript” of that study “before submission and publication”. Burkle was appointed by the Bush administration as Interim Minister of Health in Iraq in 2003. (Of course, one could probably also find things in Burkle’s favour, but why bother if the main purpose is to list a series of “connections” to warmongers).

Continuing with the “investigation” into a “tapestry of connections”, what could we say about Gilbert Burnham’s Afghanistan study, whose “assessments were funded by the Ministry of Public Health through grants from the World Bank”. A ZNet article from 2009 made an interesting observation about this study:

A case in point is Afghanistan, where the war dead are measured “only” in the thousands, and where the “excess deaths” calculation can be interpreted as favouring the NATO invasion, if numbers are taken to be the sole criterion. For example, a Johns Hopkins University study (run by Gilbert Burnham, co-author of the 2006 Lancet Iraq survey) found lower infant and child mortality rates, due to improved medical care, following the invasion. The implication here is that the number of lives saved exceeds both tallied and estimated death tolls from the fighting.

So, if we used a distorting, exaggerating, overheated rhetorical style, we would say that Burnham’s study (funded by the World Bank and a US puppet-government) “whitewashed” the crimes of US/NATO occupiers in Afghanistan by hawking the “life-saving” (in “excess deaths” terms) and “health-improving” benefits and “improvements” of bloody military intervention imposed by imperialist Western foreign policy.

This is doubly ironic considering Burnham’s – and Ahmed’s – critical comments on so-called “passive surveillance” (a common misnomer for the journalistic “surveillance” used by IBC-type projects). This approach to casualty recording (eg as utilised by Professor Marc Herold) more or less showed the bloody mass slaughter in Afghanistan for what it was, in contrast to the epidemiological “excess deaths” whitewashing calculation, which “scientifically demonstrated” – with statistical “number-crunching” – some spurious “net benefit” of war crime.

“The delivery of public health service is improving steadily in Afghanistan as the Ministry of Public Health makes progress towards meeting its goals” – Gilbert Burnham (quoted in 2007 press release: ‘Substantial Improvements Achieved in Afghanistan’s Health Sector’)

“The percentage of women in rural Afghanistan receiving antenatal care during pregnancy from a skilled provider increased from an estimated 4.6 in 2003 to 32.2 in 2006. […] More children are receiving vital childhood immunizations, according to the assessments.”

“The assessments were funded by the Ministry of Public Health through grants from the World Bank.” (From same 2007 press release quoted above)  

Continuing with the Ahmed-style rhetoric, we could add the fact that Skip Burkle (named Interim Minister of Health in Iraq in 2003 by Donald Rumsfeld and Dick Cheney, as mentioned above) is a longtime associate of Burnham (whose Afghanistan research has been repeatedly funded by the Afghanistan Ministry of Public Health) and that Burnham has collaborated with USIP (the “neocon front agency”) on its 2007 “Rebuilding a Nation’s Health in Afghanistan” symposium, and also that the Johns Hopkins Bloomberg School of Public Health (Burnham’s school) has continually collaborated with USIP on “conflict research” (including Afghanistan and Iraq) – and you have a “tapestry of connections” (or something) which certainly looks “deeply embedded in the Western foreign policy establishment”.

Or, rather, you don’t. What you have is a relatively innocuous bunch of facts which can probably be woven into something by someone who cherry-picks and embroiders with rhetoric – with the purpose of discrediting the target of that rhetoric.

False claims about IBC’s funding

Careless readers of Ahmed’s rhetoric might come to the conclusion, as someone posting to Twitter did, that IBC has “dodgy imperialist military funding sources”. In fact, Ahmed doesn’t reveal any “imperialist” or “military” funding for IBC, although that doesn’t stop him making assertions such as these:

This investigation confirms that Spagat and the IBC are part of a pseudoscientific campaign financed by the US and Western governments, that is undermining confidence in epidemiological surveys, and discrediting higher death toll estimates of US-led wars in Iraq, Afghanistan, Colombia, and beyond. […]

Rather than being the product of genuine, independent academic inquiry, this investigation confirms that the IBC’s output, often with support from leading academic institutions, has largely been performed under the financial and organizational influence of the very same powerful vested interests that have fostered armed conflict in Iraq, Afghanistan, Colombia and beyond.

These claims are false. Ahmed does not demonstrate that IBC (or “IBC’s output”) is “financed by” or “performed under the financial and organizational influence of” these interests. Instead, he documents some funding of projects undertaken by the Oxford Research Group (ORG) and others “affiliated” to IBC. Here’s what Ahmed provides regarding this funding:

(On ORG’s Every Casualty program) “The two-year initiative that ran from 2012 to 2014, ‘Documenting Existing Casualty Recording Practice Worldwide,’ was funded by a US government-backed agency [USIP] which played a key role in the 2003 Iraq War.” […]

“ORG’s Annual Report filed with the Charity Commission for the year ending 2010 confirms IBC as a “partner” of ORG’s USIP-funded ‘Recording the Casualties of Armed Conflict.’ ” […]

“Every Casualty now lists its core funders as USIP, the Swiss Federal Department of Foreign Affairs, the German Federal Foreign Office, and the Norwegian Ministry of Foreign Affairs.”

These examples are as close as Ahmed gets to supporting his claims of “financial influence” on IBC – and they are not very close. Let’s look at each instance, starting with the most recent. Every Casualty was initially an ORG project (initiated in 2007) and then became an independent charitable organisation from October 2014, with co-directors Dardagan and Sloboda. Its work is separate – and separately-funded – from IBC, although, as Ahmed notes, IBC is listed as a member of its ‘International Practitioner Network’ (or ‘Casualty Recorders Network’ as it’s now named).

Ahmed doesn’t mention the other members of Every Casualty’s peer network (apart from CERAC), so I’ll put this particular “connection” to IBC in perspective by listing all of them:

Action on Armed Violence (AOAV)
Afghanistan Independent Human Rights Commission
Afghanistan Rights Monitor (ARM)
Amnesty International, Pakistan Team
B’Tselem
The Bureau of Investigative Journalism
Centre for Study of Political Violence, Jindal School of International Affairs
Colombian Campaign Against Landmines
Conflict Analysis Resource Centre (CERAC)
Conflict Monitoring Center
Center for Statistics and Research – Syria
Darfur Peace and Development Organisation
Documenta
Elman Peace
Guatemala Forensic Anthropolgy Foundation (FAFG)
Hakikat Adalet Hafiza Merkezi
Handicap International
The Human Rights Center
Humanitarian Law Centre, Kosovo
Humanitarian Law Centre, Serbia
Iniskoy for Peace and Democracy Organisation
INSAN
INSEC
The Institute for Conflict Management
International Commission on Missing Persons (ICMP)
Iraq Body Count
Kaah Foundation for Community Concern
Liberia Armed Violence Observatory
LRA Crisis Tracker
NAMRIGHTS
Nigeria Watch
Nuestra Aparente Rendición
The National Violence Monitoring System
Observatory for Conflict and Violence Prevention
Organisation for Human Rights Activists
Organisation for Somalis’ Protection and Development
Pak Institute for Peace Studies
Pakistan Body Count
Palestinian Centre for Human Rights (PCHR)
Peruvian Forensic Anthropology Team (EPAF)
Research and Documentation Center of Sarajevo
Somali Human Rights Association
South Sudan Action Network on Small Arms
Syria Justice and Accountability Centre
Syria Tracker
Syrian Network for Human Rights
Syrian Shuhada
Tamil Information Centre
Violation Documentation Center (VDC)

Ahmed fails to produce any evidence that IBC’s output (which, of course, started in 2003) has “been performed under the financial and organizational influence” of “powerful vested interests” (eg as a result of IBC being one of the many members of the above peer network).

Ahmed doesn’t provide any links when he mentions the funding by the “US government-backed agency” (ie USIP) of ORG’s “two-year initiative that ran from 2012 to 2014” – perhaps because of the obviously innocuous nature of this work and the fact that not a single penny of the USIP funding went to IBC.

As for IBC being a “partner” of “ORG’s USIP-funded ‘Recording the Casualties of Armed Conflict’”, it’s worth citing the entire section which Ahmed refers to (from ORG’s Annual Report, year-ending December 2010). Ahmed doesn’t provide a link to this, but there is a scan available of the document. The paragraph on IBC is actually about IBC’s work with Wikileaks.

Recall that Wikileaks’ Julian Assange and IBC’s John Sloboda spoke together at the press conference covering the Iraq War Logs. Assange commented: “Working with Iraq Body Count, we have seen there are approximately 15,000 never previously documented cases of civilians who have been killed…”. What can we infer from this? Perhaps Julian Assange, in this instance, “performed under the financial and organizational influence” of USIP? There’s certainly a “link”, so it’s possible. And if we follow the conspiriological mode of inference, then it’s surely our moral obligation to conclude that this remote possibility is a definite certainty which should be “exposed”. All in the name of “investigative journalism”.

Parody aside, we can see that the “supporting” material provided here by Ahmed doesn’t add up to the shocking rhetorical “conclusions” he infers. These examples (ORG and Every Casualty) are the best that Ahmed can come up with to “support” his claims about “financial influence” on IBC’s output. The rest (largely describing funding for projects by Professor Michael Spagat, eg on Colombia) seem irrelevant to IBC’s work.

Conspiriologically transferable “conflicts of interest”

As we’ve seen from his (IBC-accusing) inference about 1990s grants to Spagat from NCEEER and IREX, etc, and from his (IBC-accusing) inferences about USIP funding for ORG/’Every Casualty’, Ahmed’s reasoning is based on a premise that the perceived taint, or “compromise”, from a given piece of funding is transferable to research with completely different funding – even to research from different decades, on different countries and topics, or by different people. As long as some association or “connection” can be asserted, then a perceived “conflict of interest” can be (and should be) transferred.

It’s curious, then, that in Ahmed’s follow-up piece, in which he replies to his critics (including statistics professor Andrew Gelman), he writes the following:

In this case, the fact that Gelman at some time in his career received some NSA funding for some specific research is neither here nor there – it reveals little if nothing about the general validity of his research on statistics, and certainly the same applies to his views on the Iraq question.

Firstly, one might ask how Ahmed can know for certain – without a thorough “investigation” – that Gelman’s NSA funding wasn’t as relevant to the “general validity” of his work as Spagat’s NCEEER funding (for example) was to the “reliability and impartiality” of IBC’s work? In a moment of temporary reasonableness followed by spectacular falsehood, Ahmed then writes:

The point here is that Gelman would be under obligation to have disclosed his funding to publications with respect to the specific research being funded and published, in order for readers and evaluators of the researchers to be aware of the relevant context.

This is precisely what didn’t happen in relation to all of the peer-reviewed publications put out by the IBC team and those associated with the IBC, Spagat included. In not a single one of those publications did they disclose that a significant portion of funding for their conflict research, including specifically on Iraq, came from US and European government agencies which happen to be closely linked with foreign policy.

Given the seriousness of the allegations that Ahmed makes here, it’s worth scrutinising what he says closely. When he asserts, “This is precisely what didn’t happen” with “the IBC team”, he is of course referring to the obligation to disclose funding with respect to “the specific research being funded and published” (his words). To repeat, the specific research being funded and published. Not some other, non-specified “conflict research” which has been funded separately.

In other words, Ahmed clearly appears to be claiming that for “all of the peer-reviewed publications put out by the IBC team”, funding received “from US and European government agencies” was not disclosed “with respect to the specific research being funded and published” in “a single one of those publications”.

This, of course, would be a huge and serious falsehood. To clarify, I asked Ahmed to “please list the IBC publications which you think were funded by these agencies, so that I can check directly with IBC to see if your assertion is true”. His response was defensive and failed to answer my question:

This may be challenging for you to understand, but by “their conflict research” I did not mean the singular publication in question, but “their conflict research.”

His phrase, “their conflict research” (eg on Iraq), was as “specific” as he would get. It could, of course, refer to the ORG or ‘Every Casualty’ research, whose funding was openly disclosed by the publications of those groups. Ahmed continued:

Do you dispute that throughout the period in which these IBC-linked publications emerged, IBC researchers who authored them have received funding for their conflict research from USIP and other government agencies?

This should indeed be disputed, as it’s misleading – aside from the fact that it’s irrelevant to whether funding for “the specific research being funded and published” (Ahmed’s wording) was disclosed (it was). The main period in which “IBC researchers” (eg Sloboda, Dardagan, Hicks) received such funding (ie for Every Casualty, 2012-2014) doesn’t coincide with their published “IBC-linked” journal output (2009-2011, mostly in 2009). Of course, Ahmed’s wording is so vague (“their conflict research”) that he might argue there’s a bit of overlap somewhere (eg with some ORG funding in 2010). But his statement, “throughout the period in which these IBC-linked publications emerged”, is misleading. Note, also, the fairly obvious fact that IBC’s ongoing work (not their journal publications) began in 2003, a decade or so before this non-IBC funding is supposed to have “influenced” IBC’s output.

Ahmed repeatedly slams “IBC researchers” with accusations of “undisclosed research funding”, “conflicts of interest” and “breach of ethics” regarding their journal publications. But his accusations in this regard are typically couched in vague and non-specific rhetoric. The journal-published material by “IBC researchers” is easy enough to list and access, as is the openly-disclosed funding for ORG and Every Casualty, etc. If the journal publications (mostly from 2009) were guilty of “egregious ethical breaches”, as Ahmed asserts, then it would be easy enough to check and confirm (or, rather, refute) – given specific claims which he doesn’t provide.

Note that the earliest USIP funding of “IBC-affiliated researchers” which Ahmed documents is for a 2010 ORG project. If this funding somehow “influenced” IBC’s 2009 journal publications, then Ahmed should probably add “clairvoyance” to the list of things that he accuses IBC of. The more one looks at specifics and bare facts on this matter, the more one gets an idea of the extent of Ahmed’s rhetorical distortions. Here’s a good example of the latter:

In the case of Spagat and IBC executives receiving funding, and having a close institutional relationship with, the US Institute for Peace (USIP), the matter is even more alarming, given USIP’s deep involvement in Iraq War policymaking under the Bush administration. Not only is the lack of such disclosure unethical, it tends to confirm legitimate suspicions about deep-seated conflicts of interest behind the IBC’s and Spagat’s work, which in turn does raise legitimate questions about the integrity of the research methodologies.

Given that the Johns Hopkins Bloomberg School of Public Health has a “close institutional relationship” with USIP (arguably “closer” than “IBC executives” have – as mentioned above), including collaborations on conflict research (Iraq, Afghanistan, etc), perhaps all Johns Hopkins “linked” publications should declare a conflict of interest? Indeed, to paraphrase Ahmed’s rhetoric, “lack of such disclosure is unethical and it tends to confirm legitimate suspicions about deep-seated conflicts of interest behind Johns Hopkins’ work, which in turn does raise legitimate questions about the integrity of conflict research carried out at, or by, the Johns Hopkins school”.

Dirty War Index (DWI)

Ahmed makes a series of misleading claims in a section which even has a deeply misleading (not to say hysterical) title: “IBC’s new metric to whitewash war-crimes”.

The “Dirty War Index” (DWI) which he discusses in this section is, in fact, not “IBC’s” in any sense. It’s a completely separate piece of work, by Madelyn Hsiao-Rei Hicks and Michael Spagat (published in PLOS Medicine in December 2008). I can’t see how anybody who reads the PLOS paper would seriously entertain the notion that it’s an extension of IBC’s casualty recording work – although, obviously, data from conflicts (whether from IBC’s database or from other sources, including epidemiological surveys) can be used as input to the DWI tool.

Ahmed begins by quoting the first sentence of a paragraph from PLOS Medicine Senior Editor, Amy Ross. Here’s how the full quote from Ross should read:

While the reliability of the DWI is tied to the reliability of the data sources available, which tend to be poor from areas of armed conflicts (read more about limitations of its uses here and here), it’s a flexible tool that has shown to be acceptable to multiple types of audiences concerned with war and civilian protection. As summed up nicely by the lead author of the paper, Madelyn Hsiao-Rei Hicks: “The DWI approach that connects public health, civilian protection, and international humanitarian law seems able to enter discussions and the ongoing, active formulation of norms, standards and practice in warfare.”

Of course, it doesn’t require Ahmed’s “investigative journalism” to point out the obvious fact that the usefulness of DWI depends on the quality and reliability of data used with it. Ahmed cites a study which claims that DWI relies on assumptions about the data utilised which are “seldom, if ever, met in a conflict situation”. Professor Nathan Taback (also cited by Ahmed) comments on a use for DWI originally proposed by Hicks and Spagat – namely, drawing attention to war-related rapes:

Selection bias impedes the generalizability of a DWI, but the DWI could nevertheless be sufficient for policy or legal purposes. For instance, if a biased sample of rape victims has a rape DWI of 10%, then this information might, for example, be useful in planning a prevention program or gathering evidence for a criminal investigation, even if the magnitude of the bias is not readily quantifiable.

In a separate PLOS Medicine piece, Egbert Sondorp concludes as follows:

[A] whole range of DWIs can be constructed, from rape to the use of prohibited weapons to the use of child soldiers, as long as acts counter to humanitarian law can be counted. The authors hope that the ease of use and understanding of DWIs will facilitate communication on the effects of war, with the ultimate goal being to moderate these effects, a similar aim to that of humanitarian law.

But here’s how Nafeez Ahmed characterises DWI:

It is not surprising that IBC’s DWI is seen as such a useful tool by NATO. The inherent inaccuracy built into the DWI means that it systematically conceals and obscures violence in direct proportion to the intensification of violence. In the hands of the Pentagon, DWI provides a useful pseudo-scientific tool to mask violence against civilians.

This is misleading in nearly every respect. As pointed out above, DWI is not “IBC’s” in any sense. There’s no “inherent inaccuracy built into the DWI” – any inaccuracy present derives from the data used with it. It’s obviously false – and absurd – to assert that DWI (which is a simple ratio, in itself neutral) “systematically conceals and obscures violence in direct proportion to the intensification of violence”. DWI could, of course, be used by the Pentagon for all kinds of nefarious purposes – but that’s not saying much (pretty much anything could be used by the Pentagon for nefarious purposes, including, as we’ve seen, the epidemiological method to provide “excess death” figures for Western imperialist wars).

Nafeez Ahmed’s characterisation of DWI as “IBC’s new metric to whitewash war-crimes” says more about Ahmed’s purpose in writing his article than it does about IBC or DWI.

Double standards regarding “valid” criticism

It’s worth noting that the two main ‘Lancet’ studies on Iraq mortality received criticisms relating to the political motivations of the studies’ authors and publishers. For example, questions were asked about the politics of the Lancet journal’s editor, Richard Horton, and about the timing of the studies – proximity to US elections, etc.

“Les and Gil [lead authors, respectively, of 2004 and 2006 Lancet studies] put themselves in position to be criticized on the basis of their views,” Garfield concedes, before adding, “But you can have an opinion and still do good science.” Perhaps, but the Lancet editor who agreed to rush their study into print, with an expedited peer-review process and without seeing the surveyors’ original data, also makes no secret of his leftist politics. At a September 2006 rally in Manchester, England, Horton declared, “This axis of Anglo-American imperialism extends its influence through war and conflict, gathering power and wealth as it goes, so millions of people are left to die in poverty and disease.”

Advocates of the Lancet studies were dismissive of this type of criticism (which is from a January 2008 National Journal article cited by John Tirman), pointing out that the work had been judged on its own merits as part of the peer-review process. It seems ironic, then, that advocates of the Lancet studies now use the same type of criticism to rubbish peer-reviewed work which they dislike.

Nafeez Ahmed, for example, writes that the journal (Defence and Peace Economics) which published Spagat’s 2008 paper is “ideologically slanted toward promoting and defending US global hegemony”, and that it is a journal “whose editors and peer-review network would lean ideologically toward publishing a fraudulent paper” (specifically one which was critical of the 2006 Lancet Iraq study).

You wouldn’t think the rhetoric and double-standards regarding peer-reviewed journals could be ratcheted up more than that, but in the next paragraph, Ahmed writes that there should be “a formal investigation” into IBC’s “capacity to garner legitimacy by publishing in scientific journals”. Perhaps such an investigation would show that the editors of journals which publish material by “IBC-affiliated researchers” (including The Lancet, PLOS Medicine, New England Journal of Medicine, Nature, etc) have “ideological leanings” which Ahmed doesn’t approve of.

(Incidentally, some conflict research which Professor Michael Spagat co-authored made the cover of the prestigious Nature journal. Recall that Ahmed gratuitously refers to Spagat as a “pseudoscientist”. Perhaps the editorial board of Nature should be subject to a “formal investigation” by Ahmed).

In order to appreciate the degree of hypocrisy in Ahmed’s allegations about “IBC-affiliated” funding and “propaganda”, consider a purely hypothetical situation in which IBC receives a $46,000 grant from USIP for “a public education effort to promote discussion of the mortality issue”. Consider, further, that this grant relates to a specific IBC paper which “fails” to disclose this funding – and that IBC’s reason for non-disclosure is that the funding was for “public education” only, and didn’t influence the research itself. One can imagine the colourful rhetoric about “propaganda” and “conflict of interest” that such a scenario would give rise to from those who wished to discredit IBC.

However, if you substitute George Soros’s Open Society Institute (OSI) for USIP, then this scenario is precisely what occurred with the 2006 Lancet Iraq study, according to comments from John Tirman (who commissioned the study). Tirman has said that a $46,000 OSI grant was received on 4 May 2006, and that the Lancet survey itself was started in “late spring [2006]”. He comments that the funding was for “public education” on the “mortality issue” addressed by the Lancet study, but had “no influence over the conduct or outcome” of the research itself. The published Lancet paper (October 2006) didn’t disclose this OSI funding.

There was much outrage from advocates of the 2006 Lancet study over the fact that its critics had used the non-disclosure of the OSI funding as a way of attacking the overall integrity of the study and its findings. Ahmed alludes to this, and seems fairly outraged himself – particularly when he accuses the “IBC-affiliated” Michael Spagat of repeating a “falsehood” about the OSI funding. (Spagat had merely mentioned that “Munro and Canon [2008] revealed that the Open Society Institute of George Soros was an important funder of L2, a fact that was not disclosed in the L2 paper”).

Of course, critics of IBC (such as Nafeez Ahmed) would never use claims about non-disclosure of funding (spurious or not) to attack the overall integrity of research. Of course not:

The context of this sort of institutional funding bias provides important context in understanding the serious and egregious statistical misinformation that is replete throughout IBC’s and Spagat’s conflict work. (Nafeez Ahmed, follow-up piece) 

“Exclusive investigation” or old, recycled claims

For some reason which I can’t fathom, Ahmed’s article includes a very long section which recycles an old one-sided “debate” about a peer-reviewed paper by “IBC-affiliated researcher” Michael Spagat – complete with selective quoting and false claims lifted straight from Ahmed’s sources.

The paper in question is Ethical and Data-Integrity Problems in the Second Lancet Survey of Mortality in Iraq (which I’ll abbreviate to EDIP-L2). It has little – if anything – to do with IBC, except that it references content from various Iraq mortality studies including IBC. One of Ahmed’s main sources for criticising this paper is a section of a “science blog” called Deltoid, written by blogger Tim Lambert and dating mostly from 2006-2008.

Lambert was a strong advocate of the Lancet Iraq studies, and had already made a series of attacks on both Spagat and IBC by the time EDIP-L2 was published. For example, he’d claimed that a vast majority of press coverage misreported IBC’s figures and that IBC “don’t even seem to be trying” to correct it. But it turned out that Lambert’s “analysis” of these press reports contained numerous errors – and, remarkably, he admitted that he hadn’t even read to the end of the reports he was supposedly analysing.

In order to attack Spagat, Ahmed lifts claims straight from Lambert’s blog, without realising that Lambert was later shown to be seriously in error. For example, Ahmed claims that Lambert “demolished” a paper on “main street bias”, or “MSB” (which was critical of the Lancet 2006 study, and co-authored by Spagat). Ahmed writes:

Lambert pointed out, citing Dr. Jon Pedersen — head of research at the Fafo Institute for Applied International Science — that “if there was a bias, it might be away from main streets [by picking streets which intersect with main streets].” Pedersen, Lambert noted, “thought such a ‘bias,’ if it had existed, would affect results only 10% or so.”

This is horribly wrong in several ways. First, it was Stephen Soldz – not Dr Pedersen – who was being cited here. Soldz was commenting on what he “thought” Pedersen said to him – wrongly, as it turns out. Pedersen later confirmed that Soldz (and by implication Lambert, and Ahmed, in turn) misrepresented his views:

“Yes, probably Stephen Soldz confused the issue somewhat here. There are actual several issues:
1) I very much agree with the MSB-team that there is some main stream bias, and that this is certainly an important problem for many surveys – not only the Iraq Lancet one.
2) I am unsure about how large that problem is in the Iraq case – I find it difficult to separate that problem from a number of other problems in the study. A main street bias of the scale that we are talking about here, is very, very large, and I do not think that it can be the sole culprit.
3) The MSB people have come up with some intriguing analysis of these issues.”
(Jon Pedersen, email to Robert Shone, 4/12/06)

With Pedersen’s permission, this email was publicly posted, back in December 2006. Lambert was made aware of it at the time, but has never corrected his blog to indicate that he’d misrepresented Pedersen (by regurgitating Stephen Soldz). Hopefully, Nafeez Ahmed will correct his own article so that misrepresentations like these aren’t “churnalised” any further.

Incidentally, Dr Pedersen (who is a respected authority in this field, and on Iraq) has elsewhere commented that the Lancet 2006 mortality estimates were:

“high, and probably way too high. I would accept something in the vicinity of 100,000 but 600,000 is too much.” (Source: Washington Post, 19 Oct 2006)

(It should be noted that Dr Pedersen said this independently of any “influence” from “neocon” funding sources or “counterinsurgency” conspiracies involving IBC.)

There’s much more in this section which Ahmed seems to lift from the “debates” which raged between 2006 and 2008, including further (inaccurate) comments on “main street bias”, “conflict of interest”, etc. Certainly nothing new. And yet Ahmed claims, in his follow-up piece, that “My investigation showed, for the first time, that Spagat’s work itself is false, fraudulent, and unreliable, and that its publication in scientific journals appears to have been enabled through unethical and undeclared conflicts of interests”. (My Bold)

Allegations of “fraud”

Ahmed repeats his assertion that Spagat’s work is “fraudulent” several times, presumably to ensure that his readers get it: “fraud”, “fraudulent attacks”, “fraudulent distortion”, “fraudulent paper”, “statistically-fraudulent claims”. Perhaps if one can throw the word “fraud” at Spagat enough, the mud will stick? And perhaps it will also stick, by association, to IBC. Note, also, that in his introductory paragraphs, Ahmed writes:

IBC has not only systematically underrepresented the Iraqi death toll, it has done so on the basis of demonstrably fraudulent attacks on standard scientific procedures.

But the only references one can find in Ahmed’s article to “fraud” or “fraudulent” attacks, etc, are those directed at Spagat’s EDIP-L2 paper (plus a few referring to Spagat’s own charges within that paper). It’s clearly nonsense to assert that the “Iraqi death toll” is represented by IBC “on the basis of” the material in Spagat’s paper (which is about problems with the 2006 Lancet study).

So, what is the basis (if any) for Ahmed’s claim that his “exclusive investigation” has demonstrated “fraud”? The first reference to fraud (after the introduction) is in a quote from blogger Tim Lambert, which refers to a graph in Spagat’s paper: “Since Spagat didn’t produce the deceitful graphic, he isn’t guilty of fraud, just of incompetence”. It’s on this point that Ahmed quotes statistics professor Andrew Gelman (who later complained that Ahmed had selectively quoted him and misrepresented his views). Here’s Gelman’s quote, in full, regarding Lambert’s point:

Spagat has clearly done a lot of work here and I haven’t read his paper in detail, nor have I carefully studied the original articles by Burnham et al. Also, some of Spagat’s criticisms seem less convincing than others. When I saw the graph on page 16 (in which three points fall suspiciously close to a straight line, suggesting at the very least some Mendel’s-assistant-style anticipatory data adjustment), I wondered whether these were just three of the possible points that could be considered. Investigative blogger Tim Lambert made this point last year, and having seen Lambert’s post, I don’t see Spagat’s page 16 graph as being so convincing.

So, Gelman doesn’t find the graph “so convincing” – that’s all. Moving on, Ahmed’s next reference to “fraud” is this:

Spagat’s modus operandi is to begin from highly questionable (and usually quite ignorant) assumptions about what ‘ought’ to happen in a conflict zone, and then to generate speculative statistical artifacts of improbability to prove high chances of falsification or fabrication.

Throughout, these arguments demonstrate a degree of willful dishonesty, and worse, fraudulent distortion and misrepresentation. There can be no doubt, as the doctors group PSR has conceded, that there are legitimate criticisms of the 2006 Lancet survey, and that scrutiny of the survey’s design and methodology is a welcome path to improving knowledge.

There’s a lot of general assertion here, but no specific claims. Moving back to the paragraph immediately preceding this, to see if it sheds any light, we get:

Spagat’s paper, like much of his previous conflict analysis work, is not just fundamentally unethical and politically compromised, but repeatedly rigs, manufactures and manipulates data to reach his desired objective of dismissing the Lancet survey with finality. His verdict that the 2006 Lancet survey makes no reliable or valid contribution to knowledge about the Iraqi War death toll is not sustained.

This is just more general assertion, with nothing specific that’s capable of being checked. In fact, it’s impossible to know which specific claims (if any) Ahmed is referring to when he asserts “fraudulent distortion”. In his next reference to “fraud”, Ahmed gives up any pretense of providing specific, refutable claims to back up his charge, and instead suggests that the paper itself is “fraudulent”:

If ever there was a journal whose editors and peer-review network would lean ideologically toward publishing a fraudulent paper critical of The Lancet’s 2006 estimate of 655,000 excess Iraqi deaths due to the war, it was Defence and Peace Economics.

The next, and final, reference to “fraud” is in Ahmed’s conclusion:

In particular, questions must be asked as to how and why elements of the scientific community have irresponsibly allowed statistically-fraudulent claims about conflict trends derived from convenience samples to be published in serious journals.

Again, tracing back through his article, it’s impossible to know which “claims”, specifically, Ahmed is referring to here. Of course, he does make various specific claims about Spagat’s paper, but they are mostly recycled opinions and assertions from a one-sided debate which occurred between 2006 and 2008 (as mentioned above), and they don’t come remotely close to making a case about “fraud”. Ahmed never asserts “fraud” where he’s writing about these specifics – presumably because such assertions could easily be checked and refuted.

The authors of the 2006 Lancet study didn’t respond to Spagat’s paper when offered the chance to do so, even though they have responded to most other published criticisms of their study. In the header of Spagat’s published paper, there’s an “editor’s note” from the journal, which reads:

“Editor’s Note: The authors of the Lancet II Study were given the opportunity to reply to this article. No reply has been forthcoming.”

As Professor Andrew Gelman commented, “Ouch.”

(Part 2 of this article is now available here.)

“Our data suggests that the (March 2003) shock-and-awe campaign was very careful, that a lot of the targets were genuine military targets. So, I think it is correct that in 2006, probably in almost any month, there were more civilians dying than during shock-and-awe.”
– Les Roberts, co-author of the 2006 Lancet Iraq study [quoted in 2008]

“Shock and awe invasions using massive air power and overwhelming force caused a far higher concentration of deaths, injuries and child fatalities than even the intense insurgency we are experiencing now”
– John Sloboda, IBC co-founder [quoted in 2005]