AR stands for Augmented Reality. Augmented Reality is an interactive experience of a real-world environment “augmented” by computer generated images or holograms. The primary value of AR, is its ability to enrich the real world with context specific information such as directions or instructions.
While Augmented Reality can be experienced on a screen, such as a phone or tablet with AR capabilities, only AR headsets can provide stereo 3D imagery with correct environmental lighting and accurate mapping. There are a number of key components that make this happen, which I will discuss below. I should also note that heads-up displays like Google Glass, are not considered to be Augmented Reality headsets even though they do provide a degree of augmentation, as these devices do not take into account the environment the user is in, and cannot display 3D information in the real world.
So how does it work?
1. SLAM
SLAM (Simultaneous Localization and Mapping) is a technique used to map a surrounding environment using a variety of sensors and cameras while simultaneously keeping track of the agent’s position (the sensors doing are mapping). AR makes full use of this to place holograms in your environment, as well as know where exactly you are placed in the world and where you are looking at.
On mobile phones or tablets, SLAM is done using only the built in camera and computer vision algorithms like ARCore and ARKit. Headsets like the HoloLones have built in depth sensors as well as cameras, allowing for a much higher degree of accuracy.
2. OPTICS
Unlike VR headsets, which produce visuals via a standard display situated right in front of your eyes, AR headsets like HoloLens and MagicLeap are pass through devices. That means that, you see the real world through the device’s clear lenses, and the images (holograms) are projected onto a separate set of layered lenses, producing the illusion that the images exist within the real world.
On mobile, the holograms are overlaid onto the device camera feed, with the most well known example Pokemon Go.
3. INTERACTION
Interaction can vary from device to device. HoloLens uses hand gesture and gaze input as the primary mode of interaction. This mode can become strenuous, as hand tracking is far from perfect and having to turn your head around to click on something isn’t something you’ll want to do for long periods.
MagicLeap opted for a to a hand controller with 3DOF (degrees of freedom) which works similarly in a sense to a computer mouse. This is also the current go-to standard for Mobile VR devices. While input methods still miss on some key features, like haptics and eye tracking (so you can just look at something instead of having to turn your head), we can soon expect major improvements as adoption increases.
