How trustworthy are online reviews?

As the popularity of consumer review websites continues to climb, so too do the complaints about their reliability and fairness.

While experts differ on how "grassroots" many of these reviews are, there's no denying the influence they are wielding on the way consumers choose products and services.

"Increasingly, they are having a huge impact, especially now in the mobile era because it’s just so easy to access those reviews at the last second," says Elias Makos, the Montreal-based editor of The Post PC, a website that examines the latest trends in technology.

"I really don't remember the last time that I was out with a group of friends, and we don't know where to eat or we don't know where to have a drink, and we just decide to walk around. Someone will invariably whip out a smartphone and go to Urbanspoon and go to Yelp to see what's being said."

According to a survey conducted by the website Search Engine Land, a site devoted to news and information about search engine marketing and optimization, 72 per cent of consumers surveyed said that they trust online reviews as much as personal recommendations, while 52 per cent said that positive online reviews make them more likely to use a local business.

There is no question that these online reviews are "democratizing the way we think about  buying a product and democratizing information," said Michael Luca, an assistant professor of business administration at Harvard Business School who conducted a number of studies on the impact of consumer review websites, focusing on Yelp and Amazon.com.

For Yelp, Luca worked with the Washington State Department of Revenue and looked at all the restaurants that had operated in Seattle over a six-year span, covering before and after Yelp was live. He matched up all the restaurants with all the Yelp reviews, which are displayed in half stars.

Luca found that for independent restaurants, every star lead to at least a five per cent increase in revenue.

The Yelp star ratings for chain restaurants, like McDonald’s, did not correlate to an increase in revenue. But Luca said there is an impact on the chains as the ratings make it easier for people to choose local establishments over the big name restaurants.

One of his charts shows that 70 per cent of restaurants in Seattle are on Yelp whereas only five per cent of restaurants have been reviewed by all other media sources.

"It’s such a tremendous amount of information because you have so many people who are leaving these reviews," Luca said.

Fake reviews

Luca also studied data from the book reviews on Metacritic, which aggregates professional reviewers, and compared it with consumer reviews on Amazon.com.

Looking at the 100 most popular books over a two-year span, Luca said he found a correlation in the reviews overall but also found areas where they deviate.

For example, professional reviewers tend to be more generous to authors who have received other media attention, he said. While consumer reviewers are more favourable to first-time authors.

But the accuracy of the reviews on these review sites has increasingly come under scrutiny. The owner of a Vancouver electrolysis chain recently accused Yelp of filtering out positive reviews about her company.

Yelp denied the charge but does admit that its filters wean certain reviews from a site that they think are spam or fakes.

"As we see [these sites] becoming more important, we see this increase in gaming the system with fake reviews," Makos said. "It's a symptom of their success."

Most consumer review systems include a hierarchy of reviewers, Luca says. The so-called elite reviewers usually have a higher status on a website, their reviews are displayed more prominently and given more weight.

He said these elite reviews act somewhat like a screening mechanism. On Yelp, Luca found that the elite reviewers have a lower variance in the reviews they leave, meaning  the non-elites were more likely to rate something either a one or a five, while elite reviewers tend not to be at the extremes.

"In some sense, [the elite reviewers] are screened in that they are people who we know are real, they have profiles, you could see what else they reviewed. So you get a lot of information about these power users that you don't have about other anonymous reviewers."

But Luca acknowledged that spam reviews remain a problem in the industry and that filtering fake reviews is a difficult problem.

Yelp filters out 15 per cent of all its reviews, Luca said, adding that their current filter is "not very good."

However, Makos said things are being done to combat the fake reviews. For example, Google said it has created a new algorithm to detect fake reviews. Based on how quickly certain reviews pop up, they say they can discover which reviews have been posted by spam or whether someone was paid to do it.

"I think there's going to be a bit of that cat and mouse game where more and more companies are going to have to use computer programs to sort of screen for faulty reviews.

Luca conceded there are a "tonne of problems' with these types of websites,

"There’s the filtering, there's the status, how do you aggregate it, how do you know if you have similar tastes to this person," he said.

"But on the other hand, these types of problems already existed in other types of reviews, but now we have a lot more information from a lot more people."