I have noticed for some time now, the disconnect between polling done by Gallup on the national level and that done by organizations doing polling in the states. Gallup seems to represent polling results that are 2-3 points lower than you would expect judging by the state data. At worst, Gallup often agrees with Rasmussen – which isn’t really in the business of polling, and operates as an arm of the Republican Party. Pew, and some of the other polling organizations seem to come up with numbers consistently higher for Obama that Gallup.
Unlike Rasmussen – there is no reason to believe that Gallup is tweaking the poll numbers. Gallup is the most established and highly respected pollster out there. So why the difference?
The difference appears to be race. And no – Gallup isn’t racist. Nor is there any evidence that they intentionally skew their numbers. That is not what is being said here. It has to do with how they assemble their samples. With 90% of black voters supporting Obama, and under-participation of black folks in the polling has almost a 1-1 correlation with the results. That is, if the statistical sample doesn’t match the racial makeup of the population, then the result skews 1 point for each point of over, or under – representation of black, and Hispanic voters. Gallup’s current polling methodology under-counts Minority voters.
The following is a really good article on how Gallup does its polling, and how their choices of how to do sampling impacts their data.
With the race for president between Mitt Romney and Barack Obama now shifting into high gear, politicians, journalists and the general public are scrutinizing each new poll, with every small swing in one direction or another elevated to outsized importance.
Among the many polls released every day, one always stands out. The Gallup Poll is arguably the most trusted survey brand in the world, a name virtually synonymouswith public opinion polling. It has measured presidential job approval and vote preference without interruption since the 1940s and now conducts a daily tracking poll that reaches more than 3,600 adults every week — a volume of data that dwarfs that produced by other firms. As a result, Gallup’s numbers enjoy unique influence and public prominence.
Over the past few years, however, polling junkies have noticed something curious: Gallup’s polls have produced results that appear slightly but consistently more negative to President Obama than those produced by other firms.
The Huffington Post has conducted an independent analysis that confirms the phenomenon and points to a likely explanation. The problem lies in the way that Gallup handles the racial composition of its samples, and the findings highlight significant issues with how polls are developed and conducted today.
The dirty little secret of telephone surveys now conducted by most media outlets is that their unweighted samples alone cannot provide reliable estimates of population demographics like race and Hispanic ancestry. A dramatic fall in response rates has led to what pollsters call “non-response bias” in their raw data. Partly because survey response rates are typically lowest in urban areas, unweighted samples routinely under-represent black and Hispanic Americans.
As a Pew Research Center study recently demonstrated, random-sample surveys continue to provide accurate data on most measures — but only when their samples of telephone numbers include both landline and mobile phones, and only when the completed interviews are weighted to match the demographic composition of the population. That means the weighting procedures that pollsters use are critical to producing accurate results.
The need to weight accurately by race and ancestry is particularly significant when it comes to evaluating the contest between Obama and Romney. As Gallup itself reported in early May, Romney led Obama among non-Hispanic white voters by 54 to 37 percent, while the president had the support of more than three-quarters of non-white registered voters (77 percent). Obama’s support among African Americans on Gallup’s tracking poll stood at 90 percent.
That gap makes the way pollsters account for race hugely important. When pollsters weight their samples to match population demographics, every percentage point increase in black representation translates into a nearly one-point improvement in Obama’s margin against Romney. The difference of just a few percentage points in the non-white composition of a poll can produce a significant skew in its horse race results… (Read the rest of this article here)