The surprises of the Brexit vote and the Electoral College win for Donald Trump have put the focus back on pollsters who failed – spectacularly -- to predict these two global shockers.
Polls in June had shown the "Remain" side would win and the United Kingdom would stay European Union. Virtually every major media outlet published polls showing Hillary Clinton beating Donald Trump – in popular vote and the Electoral College.
Instead, the "Leave" side won with 51.9 per cent of the vote and Trump claimed a surprising victory thanks mainly to four traditional Democratic states opting to support the Republican.
The head of YouGov – a polling company which predicted a victory for the Remain side in the British referendum – admits the industry made some mistakes, but says the polls were essentially correct – or within the margin of error.
Writing in a blog post, Anthony Wells says some polls were weighted towards the college educated demographic, which skewed the results.
One major mistake of pollsters, he says, was a failure to follow up on telephone calls: Those who answered their phones and participated in polls had their views recorded, but if certain demographic groups were hard to reach, many pollsters simply ignored them in favour of more easily accessible volunteers.
"Phone polls appeared to face substantially greater problems in obtaining a representative sample than online polls," Wells wrote.
But Wells points out that the polls predicting a Remain victory were actually within the margin of error.
A 'long line of failures'
The result of the Brexit referendum may have caught many off guard, but it was the election of Donald Trump that had many slamming the polling industry.
"This is really the latest in a long line of failures," said Quito Maggi, the CEO of Mainstreet Research, in an interview on CTV's Power Play. Among the errors he notes:
- In Ohio, many prospective voters were labelled as "unlikely voters" due to not having voted for 10 or 20 years.
- Those same people were registering to vote again and despite their intention to vote, were excluded from polls as they were seen as unreliable.
- In Ohio, Trump mirrored increased his early primary performance by eight percentage points. In Pennsylvania, Trump saw a 9.8 point boost between the poll and the ballot box. In comparison, Clinton underperformed by a total of 26 points across the 15 critical swing states.
"Really, the industry has gotten a black eye again," Maggi said.
But others aren't so sure.
Nik Nanos, CTV News' pollster, says people may have found the result surprising, but the general public has been too quick to blame pollsters.
He points out that Clinton won the popular vote and pollsters weren’t far off in predicting that in most of the country. Nanos says part of the problem pollsters in the U.S. faced was a desire to predict how people would vote far ahead of election day, discounting people who would change their minds.
A better method, Nanos says, would have been to look at Trump's polling numbers trending upward over the course of the campaign. Looking at a trend lines might have given a better view into voters who change their minds in the final days of the campaign.
"In a close race, a small swing in voters can have a disproportionate impact on the outcome," he said. "I call them last-minute political shoppers who wait until the very end to make a decision."
Nanos says one issue in the U.S. is the conventional method of relying on data from the past two or three days for results. A simple solution to help eliminate confusion would be to target voters one day before a poll is published, similar to the method he employs.
"Last week's prediction isn't the prediction for the election," he said.
Differences in polling methods could help limit Canada from the scenes being seen in U.S. and the United Kingdom.
Pollsters in Canada are allowed to use automated techniques – such as robo-calling or robo-polling – Maggi says, which makes it easier to reach those same voters which were untapped in U.S. polls.
In the U.S., the practice is restricted or banned in several state and federal laws. For example, calling cellphones using an automated dialing technique is banned.
A research note written by the American Association for Public Opinion Research says that banning those methods makes it harder to reach citizens who only use their cellphones in place of a traditional landline.
Lack of access to such methods can also result in a lower response rate or an incomplete set of answers, and has been blamed for undersampling of younger voters.
Not all Canadian pollsters use the robo-polling methods (Nanos doesn’t), but Maggi says the technique allows for greater accuracy and fewer surprises for pollsters and voters when the final results roll in.
But the best advice Nanos has, is if the race is too close to call, don’t try to make it seem otherwise.
"Pollsters have to be circumspect," he said. "If it's too close to call, just say that."
