In a flashy presentation at its Worldwide Developers Conference this June, Apple discussed the potential augmented reality holds for the iPhone. “When you bring these things together, the results can actually be quite profound,” Apple’s vice president of software engineering Craig Federighi said in reference to the iPhone’s touchscreen and camera — the two elements that are most crucial for providing AR experiences.
Federighi was unveiling ARKit, a set of tools Apple released for developers with iOS 11 to make creating augmented reality apps for iPhones and iPads much easier. Augmented reality is a technology that allows your phone to mix real-world surroundings with what’s being shown on screen using the device’s camera. It’s how, for example, a game like Pokémon Go is able to make it look like monster-pal Squirtle is standing on your nightstand or on the sidewalk, not just floating in space disengaged.
The first wave of ARKit-powered apps has been available for three weeks since iOS 11 debuted on September 19. As with early websites and mobile apps, AR experiences at this point aren’t as profound or polished as Apple would have you believe. Some still take too long to detect a flat surface, while others feel more like “me too” trend-following, as developers struggle to figure out how best to avail themselves of the new technology.
Should you delve into any of these at this point? I’ve explored several, and found a handful that while imperfect, offer compelling glimpses of how AR could eventually rewrite the ways in which we’ll be able to transform reality through our smart devices.
Video tutorials and photos can be helpful for learning about specific subjects, but as the viewer, you have little control over the angles at which you can observe the scene. If you’re watching a video to learn about NASA’s Mars Curiosity rover, for example, you can only catch glimpses of the vehicle, and it can be difficult to fully visualize how it operates. It’s a problem JigSpace hopes to solve.
The app includes a small library of animated 3D graphics with accompanying text that you can explore to learn how certain objects function. After selecting a tutorial, such as “How a Piano Works,” the app will scan your surroundings for a flat surface that it can virtually place the object on. Once it’s done this, you can pinch to zoom-in on the graphic, then rotate it to view it from any angle.
The app’s selection is currently limited, with just four sections and 26 tutorials or “jigs.” But it illustrates how AR can be employed to offer immediately useful experiences that don’t feel gimmicky.
Smartphones have supplanted tools like calculators, wristwatches, flashlights and numerous other utilitarian gadgets. But one they haven’t is the tape measure. That’s about to change with AR, and ARKit apps like TapMeasure exemplify why smartphones are about to become essential tools for depth-sensing and other forms of digital measurement.
TapMeasure is incredibly simple to use: just tap the button that appears on screen to set your starting point, move your phone along the surface you’d like to measure, then press the button again to set your end point. The app uses your iPhone’s camera to help with other forms of measurements, too, offering tools for leveling picture frames and generating 3D scans of rooms.
Holographic maps like those shown in The Hunger Games are still futuristic fantasies, but AR is unlocking new ways of viewing locational data.
Take Fitness AR, an app that integrates with fitness social network Strava, so that you can visualize runs and bike routes in 3D. Fitness AR creates a 3D map of paths you’ve logged in the app that’s viewable from multiple vantage points. You can zoom in to get a closer look and scout potential courses using the Fitness AR’s route building option.
As AR technology improves, I imagine a tool like this helping athletes who want to enhance their performance by studying the terrain on previous routes, focusing on areas during which they struggled.
If you’re the go-to IT expert in your family, PTC Inc.’s Vuforia Chalk app is both a blessing and a curse. It makes explaining instructions to someone remotely easier, which in turn increases the likelihood you’ll be getting calls from exasperated family members.
With Vuforia Chalk, you can share your device’s view of something with someone else to discuss a problem, say which button to push on a remote, or how to operate a homemade cappuccino maker. The two of you can then annotate the screen to pass along or comment on directions in real time. The app uses augmented reality to make sure those markups stay in place as you move your phone’s camera around the space.
News outlets are constantly evolving their storytelling formats to keep up with new technologies, and Quartz’s mobile app is a great example of this. When receiving conversational news tidbits from Quartz’s app, certain stories include an option to view the content in augmented reality. A story about Tesla’s Model 3, for example, might be accompanied by a 3D graphic of the car, which readers can then examine from multiple perspectives. It’s going to take time for news publishers and app developers to find the best uses for augmented reality, but Quartz’s approach is a promising start.