Face-reading tech could make shopping more convenient — and creepier

As you cruise through a shopping mall, you know you're being filmed by security cameras keeping an eye out for shoplifters. What you may not know is that some of those cameras may be capable of recording far more than a temporary copy of your image.

Software like Eyeris's Emovu, NEC's Neoface or Eyesee let retailers determine your gender, race and approximate age, where and how long you looked at different displays and whether you were smiling or frowning at the time.

In some cases, they will recognize exactly who you are. That information can potentially be cross-referenced with other data about you, such as how often you shopped at that store, when you last visited, what you bought at that time (and whether you've ever stolen anything.)

Such technologies have the potential to give brick-and-mortar retail stores "the same data accuracy and ad targeting as any modern website," according to the "Retail Collective Lab" exhibit showcasing examples at this year's DX3 technology, digital marketing and retail conference in Toronto earlier this month.

Online marketers have long argued the technology would lead to a more personalized and convenient shopping experience.

But privacy and surveillance experts say there are a lot of potential risks when such highly personal information is being recorded, possibly without any clear indication that it's happening.

Among the technologies showcased at the retail collective lab was Eyeris's Emovu, which has just started testing its gender, age, emotion and gaze detecting software in the retail space, said Alaoui Modar, the company's founder and CEO. That could help stores optimize their layout and displays, potentially improving the experience for customers.

So far, the software is looking at things like how much time customers spend in front of a given shelf before choosing, say, Coke versus Pepsi, or the last impression they had before leaving the store.

"You can have a camera right at the exit. Were they smiling? Happy? In a rush? Or just neutral?" Modar says. "And you can correlate all this data."

Works with any camera

Most stores already have security cameras to deter theft, so including systems like Eyeris is easy.

You're getting into this area of creepiness, really, where you may not know that this is happening and you may not know how much this is actually happening.- Geoff White, PIAC

"We work with any camera, so many deployments don't necessarily have to be a major investment per se," he said.

Consumer privacy advocates like Geoff White say the trouble is that such software systems are generally invisible.

"You're getting into this area of creepiness, really, where you may not know that this is happening and you may not know how much this is actually happening," said White, a lawyer with the Ottawa-based Public Interest Advocacy Centre.

Some retail surveillance devices don't even look like cameras. In a 2013 report on facial recognition, Canada's privacy commissioner flagged EyeSee, a mannequin that uses a camera disguised as an eye to collect data about shoppers' age, gender and race.

At DX3, another product featuring cameras, facial recognition and emotion detection was Pepper the Robot, sold by Paris-based Aldebaran. Pepper can identify customers' age, gender and facial expressions and is able to recognize and remember individual faces. She can show you products you're looking for using voice recognition and a built-in tablet. If you smile, she may show you more features. If you frown, she may switch to another product, explained Alia Pyros, international communications manager for Aldebaran.

Jack Grannan, sales engineer for Aldebaran Softbank Group, downplayed potential privacy concerns.

"It's really not any different than having a security camera that points at the storefront," he said, adding, "You're not required to stop and talk to the robot."

'Way more intrusive than video'

But Randy Lippert, an associate professor of criminology at the University of Windsor who conducts research on camera surveillance, disagrees.

"This is way more intrusive than video," he said. "This is really, really intimate information that may seem harmless on the surface. But who knows in particular contexts what it could be used for."

That could include manipulative marketing or even discrimination by age, gender or race. And customers won't necessarily know how the information is stored or shared with third parties.

Canada's privacy commissioner notes that "facial image data is particularly sensitive because it is unique and can be linked to many other individual level data."

While Canada's Personal Information Protection and Electronic Documents Act requires businesses to obtain consent from customers to obtain their personal information for particular purposes, a 2012 study found that most stores failed to comply with privacy laws even when using ordinary surveillance cameras.

Can be carried out surreptitiously

The Office of the Privacy Commissioner acknowledged that "since facial recognition can be carried out surreptitiously without individuals' knowledge and consent, it may sometimes prove tempting for organizations to simply not inform individuals, and individuals would have no way of knowing how their personal information is being used."

While a 2015 survey by IT consulting firm Computer Science Corp. found 27 per cent of U.K. retailers use facial recognition, Lippert said he doesn't think anyone knows the extent of its use in Canada.

Some recent reports suggested that high-end fashion retailer Saks Fifth Avenue, which opened its first Canadian stores this month, uses facial recognition to identify potential shoplifters. However, the company told CBC News that those reports are "erroneous."

The Office of the Privacy Commissioner of Canada told CBC News Friday it has not received any complaints about retail use of facial recognition technology.

As for gaze and emotion detecting technology, Lippert doubts most Canadians are even aware it exists.

PIAC's White says customers need to start thinking and asking questions about what's being collected by cameras in stores and how it's being used, stored and shared.

"There's a great potential to use a lot of information for positive purposes," he said. "There's just a great deal of risk, especially when it comes to the security of that information."