So as we all now know augmented reality (AR) is about placing virtual objects (graphics or data) on top of real world vision. There are two main approaches to achieving this.

The first involves the more hyped smart glasses, whereby images are projected onto the lenses of the glasses (e.g. Microsoft Hololens 2 and Magic Leap 1), or directly onto the eye.

The second uses external cameras. These cameras “pass through” the real world view, and add the virtual elements on the way. Think of VR headsets that are fully enclosed, but you can still see the real world through its external cameras. These cameras are common on VR headsets and mostly used for safety reasons.

At Headfelt XR we believe that it is the second, VR headset passthrough, that will offer the more immediate and functional delivery of consumer accessible Augmented Reality. There are a number of reasons for this.

Firstly, the failings of Augmented Reality Smart Glasses/Headset:

• AR Smart glasses, well the cool looking ones, are not yet a reality. Word on the street is saying 2024 – 2025 before a worthy consumer product is available to the masses.
• The current AR Hololens and Magic Leap headsets are enterprise (business) units. They are expensive and not overly robust. Not for the consumer!
• Smart Glasses/Headset image quality and Field of View (FOV) still require significant improvement

In favour of the Virtual Reality headset:

• There is a large, existing VR consumer base
• Devices are cost effective; $500 – $1500
• Full colour VR passthrough will soon be available in most consumers VR devices. Well, much sooner than smart glasses
• VR hardware is already starting to provide the required sensors and processing needed to deliver high quality AR experiences  
• AR features and apps will be a natural extension to the existing VR  experience and offerings.

The reason Virtual Reality  passthrough provides a better quality Augmented Reality experience is that it is not trying to map virtual assets onto real world elements that you are seeing directly with your eye. Passthrough ‘pre-renders’ the virtual assets onto the relevant real world elements in the video stream, before it is displayed to you. This is far more efficient, more robust, and produces a much more immersive result.

Now consider Oculus’ recent release of the Quest 2 API to developers. In short, they are starting to open up access to the camera for developers to start creating this augmented content and applications. Although the current camera is B&W, it is expected that the new Quest (Project Cambria), due for release in early 2022, will have a colour feed.

Once this happens we expect a massive growth in the creation and use of functional consumer focused AR applications. Just imagine the VR headset as more than only an entertainment device, such that it is specifically worn at times when functional AR needs arise e.g. While cooking dinner and following a recipe, or when repairing the lawnmower, learning how to knit, or redesigning the backyard.

Below are some recent, ‘early’ videos that show the foundations of what we will be building upon.

• Cubism Passthrough
• Quest2 Passthrough testing

When you are interested in discussing the potential of this emerging tech, let us know.