Polling industry the night’s big loser as 2016 debacle repeats itself

Sign up for the Guardian’s First Thing newsletter

Whoever ultimately wins the White House, one loser of the presidential election is already clear. The political opinion polling industry that promised to get its act together after a spectacular failure to detect Donald Trump’s 2016 triumph is under new scrutiny after another shambolic performance that left its relevance as a useful predictive tool in serious question.

Trump overperformed in several key states up to eight percentage points better than pre-election predictions had indicated, according to independent analysis, and what was supposed to be an easy victory for Joe Biden nationwide has come right down to the wire.

Winner of 2020 US election may depend on a few thousand votes as race remains tight

Read more

Poll aggregators Real Clear Politics and FiveThirtyEight, two of the nation’s most popular go-to sources for election polling, drew criticism for wildly disparate predictions in certain battleground states, including Iowa and Ohio, both of which Trump won handily after being shown to be in statistically tied races.

Those aggregators collate polls from a number of sources to produce an overall prediction of how a candidate is standing in a particular race. For example, the Real Clear Politics’ final call for Ohio showed Trump leading Joe Biden by a single point (he won by 8.1), and in Iowa by two points (winning margin 8.2).

In Florida, FiveThirtyEight had Biden winning by 2.5 points, and he lost by 3.4.

Probably the most inaccurate poll of election season, however, was the eye-opening ABC/Washington Post survey in October that gave Biden a 17-point lead over Trump in Wisconsin. FiveThirtyEight tagged it “an outlier” but also said it was not an easy poll to disregard. As of Wednesday afternoon, Wisconsin is still counting, but Biden’s lead is a far more modest 0.6%.

“This is a disaster for the polling industry and for media outlets and analysts that package and interpret the polls for public consumption,” political writer David Graham said in an article for the Atlantic that characterized the crisis as “a catastrophe for American democracy”.

“The real catastrophe is that the failure of the polls leaves Americans with no reliable way to understand what we as a people think outside of elections, which in turn threatens our ability to make choices, or to cohere as a nation.”

It wasn’t only in the presidential election that predictions were off. Susan Collins, the Republican senator for Maine, was expected to lose her re-election bid to her Democratic challenger, Sara Gideon, by a wide margin, with one Quinnipiac poll awarding the seat to Gideon by 12 points. Instead, Collins trounced her rival by 6.2 percentage points and more than 40,000 votes.

“The polling industry is a wreck and should be blown up,” Politico writers Jake Sherman and Anna Palmer said in their post-election analysis.

The 2020 election was a chance for the polling industry to prove that it had recovered from the debacle of four years ago. Pollsters, including Nate Silver, editor in chief of FiveThirtyEight, acknowledged that credibility with the public had taken a severe hit, and many made modifications to try to produce more accurate polling this time around.

Measures included widening the range of interviewees by education to include more non-college-educated voters, finding a better balance of suburban versus rural subjects, and finding more expedient ways to reach them, such as by text and email instead of relying on landline telephones.

“So, perhaps, after four years of hand-wringing, the polls will show they were all right after all,” FiveThirtyEight analysts Geoffrey Skelley and Nathaniel Rakich said in an article last month explaining the improvements, but still claiming their 2016 predictions, at least in terms of the popular vote, were “within the normal range of accuracy”.

To Graham, however, a second successive polling disaster has deep implications. “Much of American democracy depends on being able to understand what our fellow citizens think,” he said. “We no longer spend much time around people who disagree with us [and] public-opinion polling was one of the last ways we had to understand what other Americans actually believe.”

Silver, the FiveThirtyEight founder, was defensive in a tweet sent on Wednesday, at least over the prediction that Biden would capture the White House.

“Don’t want to stir things up too much but it seems like if a forecast says that Biden is favored because he could survive a 2016-style (~3 point) polling error when Clinton couldn’t, and you get that polling error and he indeed (probably) survives, it was fairly informative?” he wrote.