Going beyond Pokemon Go: preparing for an augmented reality future

Augmented reality is a pretty impressive technology. It lets us integrate our real, physical worlds with our digital lives. But the ways the average person uses it now are mostly pretty silly.

You can use Snapchat to switch faces with your dog. The Sephora app lets you see how different colours of eye shadow will look. Ikea has an app that lets you place virtual furniture around your house to see how it will look before you buy. And, of course, there's Pokemon Go.

Rather than some revolutionary new technology, augmented reality, or AR, has come into our lives piecemeal. Some version of the technology has been around for awhile now. Jet fighters have used heads-up displays in their helmets for decades, and Boeing tried using displays that overlaid wiring diagrams to help technicians in the early 90's.

Franzi Roesner is an assistant professor at the Paul G. Allen School of Computer Science and Engineering at the University of Washington. She says that there are, "two spaces in which augmented reality is being used right now. One is in ways that are already commercially available for people. And that is mostly through phone apps." 
 
But then there's the world that was imagined when Google released their AR glasses Google Glass in 2013, where the experience is more immersive, rather than something that being limited to a phone screen. Microsoft has developed their HoloLens AR headset, and other companies are developing augmented reality car windshields to display things like driving directions directly on the road in front of you.

Franzi argues that AR is essentially different than other kinds of technology we interact with because of its immersive potential. "It is directly sitting between your perception of reality and your actions in the physical world," she says.

"Unlike your laptop or your phone screen that you can step out of reality to use, the idea with augmented reality is that you're still interacting with and perceiving the physical world… That's both really exciting for the potential it has, but can also be potentially scary."

Part of what makes this technology potentially scary is that it brings with it new and unique privacy and security concerns. "If you're wearing an augmented reality headset," Franzi says, "in order for that to work well it needs to be continuously sensing everything about your physical world... sensor data, audio data, video data, about your physical surroundings."

While that continuous surveillance raises privacy concerns, the biggest danger according to Franzi is on the output side, especially if an AR user installs a malicious app.

"What can those apps do once they have the ability to change your view of the physical world? ...If you're driving, an application might block your view of pedestrians crossing the street," Franzi says, "or they might startle you with scurrying spiders."

With these threats in mind, Franzi says that the job of protecting users will go to the designers of the operating systems that run augmented reality technology.

"If you install an application, even if that application tries to do things like block your view of oncoming cars when you're crossing the street, it shouldn't be able to do that.  So I think a lot of the responsibility and opportunity is for the designers of the operating systems to figure out how to constrain applications."

So far we are only getting a small glimpse of what will be possible with AR, but Franzi says this is exactly the right time to be preparing for these concerns.

"Now is kind of the critical time where we still have the opportunity to make some of these design decisions that can take into account the potential future risks before the technologies are already widely deployed."