Justin Trudeau's behind in Papineau? How to read this NDP-commissioned poll

According to the NDP, the New Democrats are poised to win Justin Trudeau's Montreal riding of Papineau.

It is a little like my mother thinking I'd make a good prime minister. Well, of course she would think that.

This assertion, though, comes from a CROP poll of Papineau voters commissioned by the NDP, which was leaked to the media on Thursday.

The poll gave the NDP's Anne Lagacé Dowson an 11-point lead over Trudeau in the riding he has held since 2008.

Bad news for the Liberals, the leaked results landed coincidentally just before the Globe and Mail debate on the economy Thursday night, and the story was picked up by most Montreal newspapers, including the Montreal Gazette, La Presse, and Le Devoir.

But the added information gathered by Le Devoir reveals a few issues with the poll that put its results into question.

• VOTE COMPASS: Check where you fit on the political spectrum
• CANADA VOTES: More federal election coverage
• POLL TRACKER: See how the parties are doing

The sample size of 375 is not atypical for a riding poll. Though it is still a rather small sample, even if it was conducted in just one riding.

The margin of error is just over five points, but that is for the entire sample. For the 281 decided voters in the CROP poll, the margin of error increases to plus or minus 5.8 per cent, which is almost enough to erase the gap between Lagacé Dowson and Trudeau.

Calculations related to the margin of error, however, are based on the assumption that the sample is random and representative. And there are reasons to believe that this might not be the case with this poll.

According to the poll report posted by Le Devoir the sample included 117 male respondents and 258 female respondents. According to the census, 50.1 per cent of residents in Papineau are women and 49.9 per cent are men.

But this poll had the number of women respondents at 69 per cent. Though CROP weighted the results to match the census, there can be a problem if a poll comes out of the field with such unrepresentative results.

The same goes with how respondents in this poll recalled their vote in 2011. According to Le Devoir, just 14 per cent of respondents said they voted for the Liberals. But 38 per cent of people living within the current boundaries of Papineau voted for the Liberals in 2011.

Now, memories of how someone voted four years ago can be fuzzy. Two provincial elections have also been held in Quebec since then, which can further confuse things. But the provincial Liberals won the ridings that make up Papineau handily in 2014, suggesting that errors in memory should boost Liberal numbers, not decrease them.

People do move around a great deal in a working-class riding like Papineau, so the electorate can change quite a bit. But, still, these are very large discrepancies.

This is not to question the authenticity of the numbers gathered by CROP, a respected pollster in Quebec, or to argue that it is impossible that Trudeau is trailing in his own riding. But this does raise some doubts about the reliability of this particular poll.

(Update: A new Mainstreet Research poll commissioned by Postmedia and published Friday by the Montreal Gazette, surveying 783 people in the riding, gives Trudeau a five-point edge over Lagacé Dowson. The poll was published shortly after this article was written.)

The perils of reporting party polls

Considering that the New Democrats commissioned this poll, let alone the sampling oddities, why was this poll reported in the first place?

Internal party polls should be treated with a large amount of caution. This is not because the numbers themselves should be doubted, but the motivation behind the leaking of party polls is highly suspect. 

Internal party polls are usually only leaked when they are positive for that party, meaning the public does not see the polls that put the party in a bad position.

And if the results are counterintuitive, the only polls that leave party offices may in fact be outliers and not representative of the real picture. Not having any other party polls with which to compare these leaked polls, we cannot know for certain.

Also, there is a great danger in being fed a narrative that may not endure. If the NDP commissions more polls in Papineau that put them behind, these polls are unlikely to ever see the light of day. But the idea of Trudeau trailing in his home riding will have been planted in voters' minds.

This is the issue with reporting internal party polls. While the polls themselves may or may not be deliberately biased towards the party commissioning them (and with a reputable pollster like CROP, which does not comment on its polls commissioned by parties, that is very unlikely in this case), on the whole the ones that the public sees carry that bias.

If a party conducts five polls and the public only sees the two that were good for that party, the public is being fed a skewed and inaccurate view of an election campaign.

Internal party polls are not to be avoided at all costs by the media, as they can be used to help understand a party's strategy in a given riding and can provide background for more in-depth analysis. But they should not be reported like any other poll.