Morning Jay: Special “Hulk Angry! Hulk Smash!” Edition
6:30 AM, Nov 1, 2010 • By JAY COST
My internal conflict between “Bruce Banner,” who predicts a 1994-style scenario, and “The Incredible Hulk,” who thinks 2010 will be as Republican as anything since the 1920s, has been resolved.
Hulk wins. Here’s why.
The generic ballot has become very important this cycle, with more pollsters than ever offering up their version of the question, which essentially asks if you plan to support a generic Republican or Democrat for Congress.
A common inferential error with this metric comes in reading it like presidential polls. The latter tend to be very accurate, but the generic ballot polls have exhibited a systematic tilt or skew to the Democrats, at least in midterm elections.
This is most notably true for generic ballot polls that query registered voters, as opposed to likely voters. You can see this in particular with Gallup’s track of registered voters, here at the bottom of this page.
But the problem goes beyond that. In fact, the final pre-election generic ballot numbers of likely voters tend to exhibit a Democratic skew as well.
To demonstrate this, I’ve collected final likely voter generic ballot numbers going back to 1994. My condition for including polls is twofold. First, is the pollster conducting generic ballot polls today that are included in the RealClearPolitics generic ballot average? Second, was the poll concluded no more than a week before Election Day?
What I am interested in is the “skew” of the poll, or the extent to which the pollster under- or over-estimates the Democratic margin. For instance, in 1994 the final margin in the popular vote was R 52.4% to D 46.0%. That’s a 6.4% Republican victory. The final ABC News/Washington Post poll found the Democrats up by 2%. So, that makes for a final “skew” of D +8.4.
I’ve calculated that for every final likely voter generic ballot number that I could find that fits my two criteria (currently used by RealClearPolitics and was in the field no more than one week before the midterm), I get this:
[Two technical notes. (i) I've calculated skew a little bit differently here than in last week's post, as I think this works better for polls with more undecided voters such as these; (ii) NBC News/WSJ did a typical likely voter generic ballot question in 1994, showing the Republicans with an 11-point lead; but it hasn't done one since, and it is not in the RealClearPolitics average because of important variations in its question wording, so it is excluded from the chart above.]
What we see here is an average skew to the Democrats of about three points. That holds true for good Republican years (1994 and 2002), good Democratic years (2006), and split decision years (1998).
In fact, there has really only been one pollster in 15 years that has not exhibited a systematic partiality, and that is the Gallup poll. This was my point last week, that the Gallup final likely voter generic ballot number is extremely accurate. And as you can see it is more accurate, on average, than any other poll. In fact, it’s even substantially more accurate than an average of all the other final likely voter polls. In three out of four years, tossing out all the other polls and following only Gallup would have gotten you closer to the actual result.
This suggests, in turn, that our best approach to minimizing error when it comes to predicting the final popular vote spread is to favor heavily Gallup’s final likely vote projection.
Indeed, a regression analysis conducted by Emory University’s Alan Abramowitz suggests an extremely close fit between what Gallup predicts and what the final result will be. Abramowitz has lately taken to criticizing Gallup in a sharply worded post, but nevertheless the data is the data.