What's Wrong with the GAO Report
Measuring failure--or the failures of measuring.
3:37 PM, Sep 4, 2007 • By FREDERICK W. KAGAN
At first glance--as those who leaked it last week saw--the Government Accountability Office's report on Iraq, released today, paints a dark view of progress and prospects in Iraq. Its subtitle offers the most attractive thesis to opponents of the current strategy: "Iraqi Government Has Not Met Most Legislative, Security, and Economic Benchmarks." Its opening paragraph dourly states that "the Iraqi government met 3, partially met 4, and did not meet 11 of its 18 benchmarks." Surely its release marks a grim moment for the Bush administration's efforts to sustain their approach in the war. Or perhaps not.
The GAO report reflects everything that has been wrong with the discussion about Iraq since the end of 2006. Through no fault of the GAO's, the organization was sent on a fool's errand by Congress. Its mandate was not to evaluate progress in Iraq, but to determine whether or not the Iraqi government had met the 18 benchmarks. As a result, as the report repeatedly notes, the GAO was forced to fit an extraordinarily complicated reality into a black-and-white, yes-or-no simplicity. In addition, the GAO's remit extended only to evaluating progress on the Congressionally-sanctioned 18 benchmarks, 14 of which were established between eight and 11 months ago in a very different context. As a result, the report ignores completely a number of crucial positive developments that were not foreseen when the benchmarks were established and that, in fact, offer the prospect of a way forward that is much more likely to succeed than the year-old, top-down concept the GAO was told to measure. As the situation in Iraq has been changing dynamically over the past eight months, as American strategy and operations, both military and political, have been adjusting on the ground to new realities, the debate in Washington has remained mired in the preconceptions and approaches of 2006. The GAO report epitomizes this fact.
A number of commentators have already pointed out the absurdity of measuring whether or not the Iraqis had accomplished benchmarks rather than considering their progress toward doing so. Even the GAO found that task ridiculous, which is why, after criticism from the Departments of State and Defense, it invented the category of "partially met" as a third option, a category not foreseen in the legislation mandating the report.
One of the most striking things about the GAO Report is its failure to take adequate notice of the Anbar Awakening and the general movement within the Sunni Arab community against Al Qaeda In Iraq and toward the Coalition. "Anbar" appears twice in the document, both times in a comment noting that violence has fallen in that province, but without reference to the turn of the Sunni population against the terrorists. That omission is unfathomable considering the significance of the movement among Sunnis over precisely the time in which the GAO was researching and producing this report. During the same period in which the report's authors note that they were in Iraq, I was also in Iraq, and received detailed briefings on the Sunni movement not only in Anbar, but also in Diyala, Baghdad, and Babil provinces. It is difficult to imagine that the GAO authors did not receive similar briefings, but even harder to understand why, if they did, they made no mention of the phenomenon. Of course, the Congressionally-mandated benchmarks take no account of the grassroots Sunni movement, and so made it difficult for the GAO to bring them into the picture.
A more serious assessment would have included the effects of that movement in at least two key categories:
Of course, some of the problems of the GAO report are the fault of its authors, not those who chartered it. For example, it assesses the goal of providing "three trained and ready Iraqi brigades to support Baghdad operations" as "partially met." It starts by offering the background: "During the summer of 2006, a large number of Iraqi security forces refused to deploy to Baghdad to conduct operations in support of the previous Baghdad Security Plans. In January 2007, the President said that the Iraqi government had agreed to resolve this problem under the current plan and had committed three additional Iraqi brigades to support the new plan." It then reports: "Since February 2007, the Iraqi government deployed nine Iraqi army battalions equaling three brigades for 90-day rotations to support the Baghdad Security Plan. In the July 2007 report, the administration stated that the Iraqi government had difficulty deploying three additional army brigades to Baghdad at sufficient strength. In commenting on our draft report, DOD stated that current present for duty rates for deployed units is 75 percent of authorized strength. However, the July 2007 administration report stated that the government has deployed battalions from multiple Iraqi Army divisions to provide the required three brigade-equivalent forces to support the Baghdad security plan. After the initial deployment of the required brigades, the Iraqi government began the rotation plan. Nineteen units have currently deployed in support of the Baghdad security plan. Several of these units voluntarily extended, and others were rotated every 90 days in accordance with the plan. In addition, all of the Iraqi units had pre-deployment training to support operations in Baghdad. The administration's July 2007 report states that progress toward this benchmark has been satisfactory, and the overall effect has been satisfactory in that three brigades are operating in Baghdad."
This sounds like success by any reasonable measure, considering that the benchmark was adopted to address the problem that in 2006 we couldn't get the requested number of Iraqi Army units to deploy to Baghdad at all. But the GAO notes that the performance of the units that did get to Baghdad has varied. It assesses that "of the 19 Iraqi units that had supported operations in Baghdad, 5 units had performed well while the remaining had proven to be problematic for several reasons: lack of personnel, lack of individual fighting equipment and lack of vehicles to conduct their assigned missions." Indeed--the Iraqi Army is an immature force with limited logistics capabilities. Simply moving 19 units from their home bases to a training center, from there to Baghdad, fighting in Baghdad for 90 days, then moving them back to their home bases, was a daunting challenge to such a force--which it met. The units were not perfectly equipped, and did not perform perfectly. But if the standard for "meeting" such a benchmark is perfection, then few armies in the world would pass.
The evaluation of the benchmark requiring that "the Baghdad security plan will not provide a safe haven for any outlaws, regardless of [their] sectarian or political affiliation" is likewise an undeservedly harsh "partially met." The intention of this benchmark is clearly stated in the GAO report: "previous plans to secure Baghdad have failed, in part, because political and sectarian interference and rules of engagement in place for those plans prevented Iraqi and coalition forces from entering neighborhoods that are safe havens to those fueling the sectarian violence." But for some reason the GAO report defined success in this realm as follows: "We defined this benchmark as 'met' if Iraqi government policy did not allow safe havens and none existed; defined this benchmark as 'partially met' if Iraqi government policy prohibited safe havens yet some existed; and defined this benchmark as 'not met' if the Iraqi government had no stated policy on safe havens."
But these criteria are inappropriate. The issue at hand was whether or not the Maliki government would allow Coalition forces to operate at will throughout the battlespace or whether it would continue to cordon off certain Shia areas as "no-go" zones for U.S. Troops, as it had done in 2006. As the report notes, Coalition forces routinely operate in all neighborhoods of Baghdad, including Sadr City, and, we might add, throughout Iraq. The Maliki government has not established areas of the city in which Coalition forces cannot travel--in marked distinction to last year's debacle--with the result that Coalition and Iraqi forces have been able to inflict significant damage on the Jaysh al-Mahdi even within its Baghdad strongholds. Again, the standard of "policy did not allow safe havens and none existed" is a measure of perfection--when there are no areas in which insurgents or militias can operate freely, then the war is over. Until that point, there will be some temporary "safe havens" according to the GAO report, but this has nothing to do with the intention of the benchmark, which the Iraqi Government has clearly met.
Perhaps the most controversial assertion of the GAO report is likely to be that the goal of "Reducing the level of sectarian violence in Iraq and eliminating militia control of local security" was "not met" based on the following criterion: "there was no clear and reliable evidence that the level of sectarian violence was reduced and that militia control of local security was eliminated." The report notes that it would have assessed this goal as "partially met" if either condition held.
The assertion that there is no "clear and reliable evidence that the level of sectarian violence was reduced" will surprise those who have been listening to American and Iraqi officers alike brief that the levels have been falling for months--as well as those who have walked the streets of formerly war-torn neighborhoods in Baghdad. The GAO notes against these briefs that overall attacks on civilians have remained constant (although its data appear to end in July), but its explanation for its unwillingness to accept the figures of the U.S. command is nothing short of bizarre: "GAO cannot determine whether sectarian violence in Iraq has been reduced because measuring such violence requires understanding the perpetrator's intent, which may not be known." As a statement of epistemology, this sentence is correct and worth meditating on. As the basis for denying that there has been a reduction in sectarian attacks it is ludicrous. When Shia gunmen seize Sunnis, bind them, torture them, execute them, and dump their bodies on the street, it is probably reasonable to assume--as we have been assuming for years--that the motivation was sectarian in nature. When a Sunni al Qaeda fighter (and there are virtually no Shia in Al Qaeda In Iraq) drives a dump truck full of home-made explosive into a crowded market in a heavily Shia neighborhood, it is also fair to say that his intention probably had something to do with sectarianism (considering Al Qaeda In Iraq's repeated denunciation of Shia as "dogs" and "apostates" worthy of death, and considering former AQI leader Abu Musaab al Zarqawi's stated intention of killing Shia to provoke a sectarian war). The U.S. military command has been monitoring both Shia execution-style killings and mass-casualty attacks on Shia targets as "sectarian killings" for many months, and now reports that the trends of such attacks are at one-fourth to one-third of their pre-surge levels. To set against this clear statement the philosophical quandary of looking into the soul of the killer before categorizing the crime is ridiculous.
One could go on cataloguing the failings of the GAO report, both in its mandate and in its execution, but the exercise would quickly become tedious. The GAO was given a fool's errand by a Congress determined to generate at least one report this September that it could reliably cite showing failure in Iraq. Well, Congress accomplished its goal. For those of us who are interested in what is really happening in Iraq, the reports of General Petraeus and Ambassador Crocker will be far more useful.
Frederick W. Kagan is a contributing editor to THE WEEKLY STANDARD and a resident scholar at the American Enterprise Institute. He is grateful for assistance from Daniel Barnard and Joel Rayburn in the preparation of this article.