The New Jim Crow — Why Some Polls Under-Report Obama’s Approval Numbers

I have noticed for some time now, the disconnect between polling done by Gallup on the national level and that done by organizations doing polling in the states. Gallup seems to represent polling results that are 2-3 points lower than you would expect judging by the state data. At worst, Gallup often agrees with Rasmussen – which isn’t really in the business of polling, and operates as an arm of the Republican Party. Pew, and some of the other polling organizations seem to come up with numbers consistently higher for Obama that Gallup.

Unlike Rasmussen – there is no reason to believe that Gallup is tweaking the poll numbers. Gallup is the most established and highly respected pollster out there. So why the difference?

The difference appears to be race. And no – Gallup isn’t racist. Nor is there any evidence that they intentionally skew their numbers. That is not what is being said here.  It has to do with how they assemble their samples. With 90% of black voters supporting Obama, and under-participation of black folks in the polling has almost a 1-1 correlation with the results. That is, if the statistical sample doesn’t match the racial makeup of the population, then the result skews 1 point for each point of over, or under – representation of black, and Hispanic voters.  Gallup’s current polling methodology under-counts Minority voters.

The following is a really good article on how Gallup does its polling, and how their choices of how to do sampling impacts their data.

2012-06-11-Blumenthal-ApprovalChart.png

Obama’s Approval Rating has been consistently lower by a few points on Gallup (in red) versus other polling organizations.

Race Matters: Why Gallup Poll Finds Less Support For President Obama

With the race for president between Mitt Romney and Barack Obama now shifting into high gear, politicians, journalists and the general public are scrutinizing each new poll, with every small swing in one direction or another elevated to outsized importance.

Among the many polls released every day, one always stands out. The Gallup Poll is arguably the most trusted survey brand in the world, a name virtually synonymouswith public opinion polling. It has measured presidential job approval and vote preference without interruption since the 1940s and now conducts a daily tracking poll that reaches more than 3,600 adults every week — a volume of data that dwarfs that produced by other firms. As a result, Gallup’s numbers enjoy unique influence and public prominence.

Over the past few years, however, polling junkies have noticed something curious: Gallup’s polls have produced results that appear slightly but consistently more negative to President Obama than those produced by other firms.

2012-06-11-Blumenthal-PresidentVote.png

Romney’s projected percentage of vote has been consistently higher on Gallup (in red) than in other polls.

The Huffington Post has conducted an independent analysis that confirms the phenomenon and points to a likely explanation. The problem lies in the way that Gallup handles the racial composition of its samples, and the findings highlight significant issues with how polls are developed and conducted today.

The dirty little secret of telephone surveys now conducted by most media outlets is that their unweighted samples alone cannot provide reliable estimates of population demographics like race and Hispanic ancestry. A dramatic fall in response rates has led to what pollsters call “non-response bias” in their raw data. Partly because survey response rates are typically lowest in urban areas, unweighted samples routinely under-represent black and Hispanic Americans.

As a Pew Research Center study recently demonstrated, random-sample surveys continue to provide accurate data on most measures — but only when their samples of telephone numbers include both landline and mobile phones, and only when the completed interviews are weighted to match the demographic composition of the population. That means the weighting procedures that pollsters use are critical to producing accurate results.

The need to weight accurately by race and ancestry is particularly significant when it comes to evaluating the contest between Obama and Romney. As Gallup itself reported in early May, Romney led Obama among non-Hispanic white voters by 54 to 37 percent, while the president had the support of more than three-quarters of non-white registered voters (77 percent). Obama’s support among African Americans on Gallup’s tracking poll stood at 90 percent.

That gap makes the way pollsters account for race hugely important. When pollsters weight their samples to match population demographics, every percentage point increase in black representation translates into a nearly one-point improvement in Obama’s margin against Romney. The difference of just a few percentage points in the non-white composition of a poll can produce a significant skew in its horse race results… (Read the rest of this article here)

Republican Bias in “Polling”

Been saying for a long time that Rasmussen is in the bag for Republicans in their polling. This analysis of post election results versus the polls conducted by the various polling outfits shows just how badly Rasmussen consistently gives extra points to Republican candidates – and how the majority of the polls lean Republican.

Also interesting is how badly CNN polls performed.

Rasmussen Polls Were Biased and Inaccurate; Quinnipiac, SurveyUSA Performed Strongly

The majority of Polls showed a consistent Republican bias, feeding the impression of an inevitable Republican victory in the media. If releasing actual vote counts before the Polls close can influence voter turnout... What role do biased and inaccurate polls play?

 

On Tuesday, polls conducted by the firm Rasmussen Reports — which released more than 100 surveys in the final three weeks of the campaign, including some commissioned under a subsidiary on behalf of Fox News — badly missed the margin in many states, and also exhibited a considerable bias toward Republican candidates.

Other polling firms, like SurveyUSA and Quinnipiac University, produced more reliable results in Senate and gubernatorial races. A firm that conducts surveys by Internet, YouGov, also performed relatively well.

What follows is a preliminary analysis of polls released to the public in the final 21 days of the campaign. Our process here is quite simple: we’ve taken all such polls in our database, and assessed how accurate they were, on average, in predicting the margin separating the two leading candidates in each race. For instance, a poll that had the Democrat winning by 2 percentage points in a race where the Republican actually won by 4 would have an error of 6 points.

We’ve also assessed whether a company’s polls consistently missed in either a Democratic or Republican direction — that is, whether they were biased. The hypothetical poll I just described would have had a 6 point Democratic bias, for instance.

The analysis covers all polls issued by firms in the final three weeks of the campaign, even if a company surveyed a particular state multiple times. In our view, this provides for a more comprehensive analysis than focusing solely on a firm’s final poll in each state, since polling has a tendency to converge in the final days of the campaign, perhaps because some firms fear that their results are an outlier and adjust them accordingly…

The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight’s database, which includes all polls conducted since 1998.

Moreover, Rasmussen’s polls were quite biased, overestimating the standing of the Republican candidate by almost 4 points on average. In just 12 cases, Rasmussen’s polls overestimated the margin for the Democrat by 3 or more points. But it did so for the Republican candidate in 55 cases — that is, in more than half of the polls that it issued.

If one focused solely on the final poll issued by Rasmussen Reports or Pulse Opinion Research in each state — rather than including all polls within the three-week interval — it would not have made much difference. Their average error would be 5.7 points rather than 5.8, and their average bias 3.8 points rather than 3.9.

Nor did it make much difference whether the polls were branded as Rasmussen Reports surveys, or instead, were commissioned for Fox News by its subsidiary Pulse Opinion Research. (Both sets of surveys used an essentially identical methodology.) Polls branded as Rasmussen Reports missed by an average of 5.9 points and had a 3.9 point bias. The polls it commissioned on behalf of Fox News had a 5.1 point error, and a 3.6 point bias. (more)

Even more interesting is how this bias plays into President Obama’s “Approval ratings” -

The discrepancies between Rasmussen Reports polls and those issued by other companies were apparent from virtually the first day that Barack Obama took office. Rasmussen showed Barack Obama’s disapproval rating at 36 percent, for instance, just a week after his inauguration, at a point when no other pollster had that figure higher than 20 percent.

SurveyUSA and Quinnipac were the best this election cycle. But the question really needs to be asked if some of the “polls” aren’t more political than statistical…

And whether that should be legal in view that election results are held until after polling stations close to prevent influencing voters.

Follow

Get every new post delivered to your Inbox.

Join 133 other followers