2020 Election Polling Had A Democratic Bias

UPDATED November 6, 2020.

Overview

The polling in the 2020 election was badly biased against Republicans and Trump, and in favor of Democrats and Biden, in particular. 

The pro-Democrat bias in the U.S. Senate polling was even worse (an average of 6.99 percentage points) than in the Presidential race (an average of 4.42 percentage points).

Of 51 state polls in the Presidential race, the national Presidential rate polling and polling in 35 U.S. Senate races, a total of 87 polling averages, 79 (91%) erred in favor of Democrats. 

Of the 87 polling averages, 54 (63%) were off by more than 4 percentage points (easily more than one standard deviation), with 98% of these large errors in favor of Democrats and the one large error in favor of a Republican coming in a race without a Democrat running. 

If the polls were not biased, you would expect a mean error of close to zero for 87 polling averages, about equal numbers of polls biased in favor of Democrats and of Republicans both in terms of any errors, and in terms of large errors (about 43-44 errors for each party), and 28 or fewer large errors (about 14 errors for each party).

It isn't obvious why this bias occurred in both 2016 and 2020 polling. 

The most plausible theory is that Trump supporters and Republicans disproportionately refuse to answer political polls, thereby omitting their support even after adjustments for demographic balance. This would imply a reduced response rate of about ten to twenty percent by Republicans relative to non-Republicans in the same demographic categories.

The Presidential Race

The final 538 blog 2020 election forecast was primarily a weighted average of lots and lots of good quality polls. These predictions were way off.

In the 50 U.S. state and District of Columbia predicting the outcome of the Presidential race, there was an average bias in favor of Biden relative to the actual results of the election was 4.42 percentage points. There were 45 errors in favor of Biden and 6 in favor of Trump.

Twenty-seven of the fifty-one polls were off by 4 or more percentage points, outside the one standard deviation margin of error. Of those outliers, all twenty-seven were in Biden's direction.

The popular vote prediction was an 8.0 percentage point lead for Biden. The actual outcome was a 2.8 percentage point lead for Biden. This may improve from a 5.2 percentage point error to as little as a 3 percentage point error as more votes are added to tallies from slow counting safe states like California.

All of the errors in favor of Trump were in states that were either as safely Republican as Utah, or as safely Democratic as Colorado. Every single battleground state had an error in favor of Biden. There were swing state errors in favor of Biden of 4 percent or more in Illinois (5.6), Michigan (5.2), Nevada (4.5), Wisconsin (7.7), Pennsylvania (4.5), Florida (4.5), Texas (4.4), Ohio (7.5), Iowa (6.7) and South Carolina (4.5). Pennsylvania may improve somewhat as vote counting continues there, but it will almost surely be off by more than 4 percentage points in favor of Biden when all of the votes are counted. The pro-Biden polling error was smaller in Arizona (0.2), Georgia (0.9), and North Carolina (3.1).

The U.S. Senate race

The 538 blog also made 35 polling average driven predictions for the 35 U.S. Senate races. The average Democratic bias in the polling was 6.99 percentage points. There were 33 errors in favor of Democrats and 2 in favor of Republicans.

Twenty-seven of the thirty-five polls were off by 4 or more percentage points, outside the one standard deviation margin of error. Of those outliers, one was in a Republican's favor (in a safe race with no Democrat against a third party candidate), and twenty-six were in the Democratic candidate's direction.

There were less than safe state polling errors in favor of Democrats of 4 percent or more in U.S. Senate races in the Georgia Special Election (6.0), New Mexico (7.0), Minnesota (7.0), Michigan (5.4), North Carolina (5.0), Iowa (5.2), Maine (9.5), Montana (6.8), Kansas (5.6) and South Carolina (8.3). 

The only U.S. Senate races reasonably in play which the polling got close to right on were Colorado (1.0 bias in favor of GOP), Arizona (1.0 bias in favor of the Democrat),  Georgia regular election (1.7 bias in favor of the Democrat), Texas (2.5 bias in favor of the Democrat), 

Analysis

This is surprising. The factors usually associated with major polling errors weren't present in this case. 

The poll numbers were robust to errors limited to particular polling operations. They were weighted by methodology quality past partisan bias, by sample size, and by recency. In many cases there were eight or more recent, high quality polls that went into the averages.

There was not a strong third-party candidate or a lot of undecided voters except for one U.S. Senate race (Arkansas) which was off by a larger margin than any other Presidential or U.S. Senate race this cycle except the U.S. Senate races in Alaska and West Virginia.  None of those races were remotely close.

Polling was stable leading up to the election, not volatile as it had been in 2016, and there were trend lines in favor of Democrats and Biden in particular leading up to the election (except for Cunningham, the Democratic Senate candidate in North Carolina whose affair was revealed in an October surprise).

Turnout was higher than predicted (the highest since the year 1900 in any national general election), which should have favored a pro-Democratic adjustment in polling models.

Pollsters adjusted their methodologies after the major errors of 2016 in an effort to remedy the problems then, but those problems returned this year in many of the same places.

The explanation in bold below seems to be the most likely one.

The 2020 polling error “matches the pattern of the 2016 error really well, so there really does seem to be something wrong here,” explained G. Elliott Morris, a data journalist who runs the Economist’s election forecast, during a Wednesday postmortem on the “Science of Politics” podcast. “It’s not just two random polling errors.” . . . 

Researchers have largely ruled out the idea of “shy Trump voters” who lie to pollsters and say they’re undecided or that they favor someone else when they really favor Trump. But it’s possible, Grossman and Morris speculated, that pro-Trump, non-college-educated whites are simply less inclined to pick up the phone or participate in polls than pro-Biden, non-college-educated whites.

Why? 

Because the pro-Trump cohort also tends to have less “social trust” — i.e., less “trust in other people or institutions,” as Morris put it. Spurred by Trump’s “fake news” mantra, participating in polls may have itself become politicized. When overall response rates are as low as 4 percent, this could skew the results against Trump in places like the Rust Belt or Texas. 

A similar dynamic may have also made it seem like more Republicans were flipping from Trump to Biden than ultimately did — again, because pro-Trump Republicans may be less inclined than pro-Biden Republicans to answer a pollster’s call or participate in an online survey. 

Other potential reasons for 2020’s big miss may have been beyond anyone’s control. It’s unlikely that late-breaking voters who decided within the last week made the difference, even though they told exit pollsters they favored Trump over Biden by 14 percentage points. There simply weren’t enough of them this year — just 4 to 5 percent of the overall electorate vs. about 14 percent in 2016 — to explain Trump’s overperformance on Election Day. 

A more plausible scenario, Morris said, is that a significant number of pandemic-induced mail ballots are either arriving late, or being rejected, or not being returned at all. If tons of people tell pollsters they’ve voted by mail and then, for whatever reason, some those “likely votes” don’t actually count on Election Day, it could widen the gap between the polls and the results. 

Political scientists and pollsters will debate these problems for years to come, and they’ll probably devise new approaches to deal with them. But after being told that Joe Biden could win in a landslide — and then watching as Trump beat his polls by even more than 2016 in state after state — the broader public might be more inclined to dismiss political surveys in the future. 

There are “systematic problems that they haven’t solved since 2016, and in fact seem to be worse this time,” Morris said. “That’s pretty troubling if you’re a pollster — especially if you’ve spent the last four years trying to reckon with the fact that polls were missing Trump supporters. So they have a big reckoning ahead of them.”

From Yahoo News. 

It may well be days, if not weeks, before the winner of the 2020 presidential race is decided, but one clear lesson from Tuesday night’s election results is that pollsters were wrong again. It’s a “giant mystery” why, says Nick Beauchamp, assistant professor of political science at Northeastern.

There are a number of possible explanations, Beauchamp says. One is that specific polls undercounted the extent to which certain demographics—such as Hispanic voters in specific states—shifted toward President Trump.

Another is that, just as in 2016, polls undercounted hard-to-reach voters who tend to be less educated and more conservative. Beauchamp is less convinced that “shy” Trump voters deliberately misrepresented their intentions to pollsters.

“Whatever the cause, it has huge implications not just for polling, but for the outcome of the presidential and Senate elections,” Beauchamp says. “If the polls have been this wrong for months, since they have been fairly stable for months, that means that campaign resources may have also been misallocated.” . . . 

This year’s polling errors, Beauchamp said, were “enormous” even compared to 2016, when polls failed to predict Trump’s defeat of Democratic nominee Hillary Clinton.

Indeed, just days before the presidential contest, FiveThirtyEight founder Nate Silver predicted that Biden was slightly favored to win Florida and its 29 Electoral College votes. The race was called on election night with Trump comfortably ahead. . . .

“I think Nate Silver and the other pollsters are saying ‘Well that’s just within the bounds of systematic polling,’ but it seems awfully large to me for just that,” Beauchamp says.

Now, election watchers and media leaders are questioning the value of polling overall. Washington Post media columnist Margaret Sullivan wrote that “we should never again put as much stock in public opinion polls, and those who interpret them, as we’ve grown accustomed to doing.”

“Polling,” she wrote, “seems to be irrevocably broken, or at least our understanding of how seriously to take it is.”

Polling misses aren’t unique to the United States, Beauchamp points out. Polls also failed to predict the 2015 elections in the United Kingdom, as well as the UK’s 2016 “Brexit” vote to exit the European Union.

In those cases, as with the 2016 US election, Beauchamp said, pollsters “made these mistakes, which are relatively small, but in the same direction and with fairly significant effects.”

To avoid a repeat of 2016 and 2020 in the United States, Beauchamp says, pollsters should shift their tactics—and perhaps attach different weights to factors they’re trying to measure, such as social distrust or propensity to be a non-voter.

“Hopefully they’re going to start modeling all of that information as a way to better capture these issues in voters,” Beauchamp says.

From here.

The bottom line is that distrust of the media has translated into distrust of pollsters, and that has made the polling work product  highly unreliable.

Pollsters take lots of classes in statistics and psychology (driven disproportionately by studies based on "WEIRD" samples), and not nearly so many in anthropology. 

But one of the core lessons in advanced anthropology fieldwork is that you need to win the trust of your informants from another subculture or culture to get accurate information from them. 

Pollsters haven't even begun to scratch the surface to figure out how to do that and until then, polls pertinent to our national culture war are going to be seriously flawed.