Calgary election polling failures caused by 'perfect storm,' Mainstreet research firm says
Mainstreet Research repeatedly said Bill Smith would be elected, but Calgary Mayor Naheed Nenshi won 3rd term
Mainstreet Research says its internal investigation into the polling failures that led the firm to wrongly predict Calgary Mayor Naheed Nenshi would lose the October municipal election concludes they were caused by a "perfect storm" of factors.
This includes a failure to poll in non-official languages, a tendency among Nenshi voters not to respond to the firm's polls, and — most crucially — a misgauging of youth voter turnout and their voting intentions, Mainstreet said in a release Monday morning.
Before the election, Mainstreet Research released three surveys on behalf of Postmedia, which owns the Calgary Sun and Calgary Herald.
All suggested a relatively unknown challenger, Bill Smith, was leading Nenshi by between nine and 17 percentage points, with Mainstreet Research president Quito Maggi predicting on Oct. 7 the "near-certain election of Bill Smith" and adding he likely wouldn't be the only one to lose his seat in "this change election."
Instead, Nenshi won re-election to a third term as mayor in the Oct. 16 election by eight points. All 10 incumbent councillors were also re-elected.
- Read the key findings of Mainstreet's internal investigation report here.
Many long-time political watchers and polling experts questioned Mainstreet's results during the campaign, prompting Mainstreet to conduct an internal review led by Mainstreet Research VP of analytics Dr. Joseph Angolano and vetted by a panel of independent polling experts.
It found "absolutely no evidence that the polling errors were intentional or malicious," Mainstreet said in its release.
The report concludes that several factors led to "a near 25 per cent deviation" between Mainstreet's last pre-election poll and the actual outcome.
"The ratio of young people in Calgary, and a few other western urban centres is unlike anything we see in other parts of the country," said Maggi in the release.
"We missed a lot of these young voters, because they are harder to reach. Even those we did reach had significant response bias as revealed by the report."
'Humbling' experience
Maggi said the experience has been a humbling one that will ultimately make the firm better.
The review was also critical of the tone taken by Maggi in his response to critics during the campaign. Maggi accepted the criticism.
"In the heat of an election race, myself and some of my staff were not as respectful as we should have been," he said.
"We received more than a healthy dose of skepticism and critique. We also faced allegations — on social media and elsewhere — that our intent was malicious, or that our polls were rigged. We took it personally and responded emotionally."
Maggi vowed to change the way he and his firm communicate during election campaigns, and let the firm's research speak for itself.
Mainstreet conducted Interactive Voice Response (IVR) polling, which is also known as robo-call or push button polling.
Critics of the method say IVR has a low response rate and tends to over-represent older, more conservative-leaning voters.
Angolano's report found that growing youth demographics and declining response rates puts the future utility of IVR in doubt.
"This review has cast a harsh light on some of the shortcomings of IVR (Interactive Voice Response) in polling," the release said.
Next-generation polling
While Mainstreet remains convinced the technique has merit, with some adjustments, the firm is also working on next-generation polling technology, it said.
Maggi apologized to the Calgary Herald, the Calgary Sun and media in Calgary generally.
"You deserve better and I regret any embarrassment it caused the media and Postmedia specifically," he said, vowing that the experience will make him a better, more vigilant and cautious pollster.
To Calgary's mayoral candidates, including the victor, Nenshi, Maggi also offered apologies.
He said he especially regrets any effect his firm's polls had on Andre Chabot's chances, by prompting strategic polarization of support to either Naheed Nenshi or Bill Smith.
The Mainstreet polling data, predicting big lead for Smith, might also have led to complacency among his supporters and negatively affected voter turnout, for which Maggi also said he was sorry.
"By setting the expectation of your victory, the true significance of your achievement was lost, the third largest vote count in Calgary mayoral election history, and getting 44 per cent of the vote against Naheed Nenshi was unimaginable just a few short weeks before the election," he said.