Reality is becoming a lot less real. We know about virtual reality but now there’s also augmented and mixed reality.
AR and MR might not have the name recognition of VR, but they’re huge tech leaps that are blending digital worlds and graphics with our physical surroundings — if you’ve ever used a Snapchat filter or played Pokémon GO, you’ve already experienced AR. In the simplest terms, it puts virtual graphics into real-life environments — think Pikachu popping up in your kitchen on screen.
Microsoft’s first Windows MR headsets are out next week, while an AR machine has probably found its way into your pocket already: your smartphone. Apple’s ARKit and Google’s ARCore development platforms mean augmented reality is about to be with you absolutely everywhere. As Apple CEO Tim Cook recently said in London, the results will be “dramatic”.
Meanwhile, VR is in the news. Mark Zuckerberg sent an avatar of himself to hurricane-damaged Puerto Rico this week in a livestream that has caused controversy — he was promoting his new tech — but was this a sensitive thing to do in a disaster zone?
Whether you are into AR, VR or MR, or just don’t know yet, here’s what the new reality means for you.
Windows into other worlds
Windows Mixed Reality is a new platform bringing AR and VR to PC. For now, the first wave of Windows MR headsets veer more to VR (key apps include zombie-shooting game Arizona Sunshine and social network vTime), turning the PC operating system into a digital room with web browsers and documents adorning the walls.
The key difference to the likes of Oculus Rift and HTC Vive is the in-built room-tracking tech in Windows MR headsets. Plug it into your laptop, and the headset scans your boundaries and works out your location, rather than relying on external cameras.
Microsoft’s HoloLens is the indicator of where Windows MR is heading — a still-in-development augmented reality headset with its own built-in computer, meaning you can leave your laptop behind and see computer software around your living room using AR.
“We’re building a platform for the future,” says James Guion, Microsoft Product Manager. “The aim is that VR, AR and MR all merge. One day you’ll be able to jump from a fully-immersive VR experience to an augmented experience and cross the spectrum.”
We’re already seeing these technologies cross over, blending the real and unreal in immersive experiences.
Earlier this week, Zuckerberg took Facebook users on a video trip to Puerto Rico via Facebook’s “social VR” tool Spaces — somewhere been a live video conference call and virtual reality tour (pictured bottom).
Motion-tracked avatars represented Zuckerberg and head of social virtual reality Rachel Franklin, who in real life were in different offices wearing headsets looking at the area. Expect to see more of these experiments as the technology evolves.
The Royal Academy’s Virtually Real project in pictures
1/4 The Royal Academy’s Virtually Real exhibition in pictures
2/4 The Royal Academy’s Virtually Real exhibition in pictures
3/4 The Royal Academy’s Virtually Real exhibition in pictures
4/4 The Royal Academy’s Virtually Real exhibition in pictures
Try before you buy
Ikea is beloved for its cheap meatballs, but less so as ground zero for blazing arguments. Now its AR is solving domestic debates about what colour sofa to get or whether that shelf will fit in your kitchen.
Coming soon to the iOS App Store, Ikea Place uses ARKit’s depth-sensing abilities to digitally place IKEA products around your home on your phone so you can see how they look. More than 2,000 products have been rendered to scale in the app with detailed textures. It’s a bit like The Sims but practical.
Ikea isn’t the only retailer already using AR — Specsavers’ app takes a 3D map of your face and lets you try glasses on digitallys.
Augmented reality will show us more about the world around us. Popular app Night Sky uses AR to point out planets, constellations and satellites when users point their iPhones skywards, while the new ARKit-powered Grand Orrery feature conjures a detailed solar system in mid-air.
“You’re able to walk through the asteroid belt, see planetary moons, and manipulate time and space,” says Andy Weekes, CEO of iCandi Apps. “You can even track Nasa missions within your living room.”
It’s the perfect example of AR enhancing the world around us with relevant information.
“Using AR, Night Sky educates users without detracting from the natural beauty of the stars,” says Weekes. “Apps could augment navigation around the city, showing directions on the real street in front of you, or names of nearby mountains in a national park. The possibilities are endless.”
An app already moving towards this is Trarvel, created by King’s College graduate Clarence Ji. Hold your mobile up to the London skyline and the app points out landmarks, with info and food tips for each area.
“We want Trarvel to become the ultimate all-in-one tourist information app, using AR to make exploration an interactive experience,” explains Ji, adding that New York and Hong Kong are the next cities for the app..
As with VR, AR and MR are changing the gaming world. ARise conjures a floating puzzle in the middle of a room, tasking players to solve forced-perspective brainteasers by walking around levels to find a solution.
“The strength of ARKit is that we know where the iPhone or iPad is in space to a fine degree,” says Simon Gardner, CEO of Climax Studios. “Augmented reality will allow players to experience gameplay anywhere and it’s more immersive.”
Lenovo is working on Mirage, a smartphone-powered AR headset which brings games into the room. The first is Star Wars: Jedi Challenges, with a lightsaber controller.
Windows Mixed Reality headsets will be available from October 17