VR vs. AR vs. MR: What Is Each One Good for?

Skype June 1, 2019 Technology Leave a comment 397 Views

Stanford is using Microsoft HoloLens in breast cancer surgery

Hardly a day goes by without the promise of some new, world-changing, product in the alternative reality space (currently a mash-up of VR, AR, MR, and now sometimes called XR). It can be hard to keep the technologies straight, so we’ve put together a short guide to the major aspects of each of them, and their relative strengths and weaknesses.

Virtual Reality (VR)

The first of the “Rs” to become “the next big thing” Virtual Reality (VR) solutions are defined by their replacement of the reality around you with a virtual substitute. That currently means that VR devices are a helmet or goggles with their own display(s) that completely replace your view of the outside world. There is typically also some type of tracking solution, so that you can move in your new virtual reality. Low-end solutions like Google’s Cardboard that rely on a smartphone IMU only allow you to look around, and don’t track other head or body motions. Higher-end versions like Oculus and Vive devices rely on some form of tracking that allows moving around in the virtual environment. Initial tracking systems required external tracking beacons, but now “inside-out” tracking using cameras along with an IMU has become a less-expensive alternative. Windows 10 “MR” headsets, along with the Oculus Quest, are good examples of VR devices that rely on their own sensors to track both head and body movement.

Due to the large amount of processing power required, initial VR headsets were all tethered — often with multiple cables for video and tracking. HTC broke through that requirement with a wireless add-on adapter, but now models like Quest from Oculus have combined inside-out tracking with on-board processing to provide a completely un-tethered solution. Its low-power processor can’t provide the same gaming performance as its larger sibling the Rift attached to a VR-ready Windows computer, but it’s certainly a lot easier to set up and use.

Augmented Reality (AR)

Augmented Reality (AR) comes in two very different flavors. Early efforts that gained traction were based on projecting objects onto an otherwise clear pair of glasses or visor, overlaying them on the real world. Like early VR solutions, those were expensive and cumbersome, although a few like Lenovo’s Mirage AR headset have been created that re-purpose your smartphone display as the projection source. Perhaps the most popular has been the Epson Moverio line of AR glasses. But as smartphones improved their depth-sensing and processing capabilities, AR on mobile devices emerged as a very important category.

Annotating or projecting objects as overlays on the live view from a smartphone or tablet camera has proven very useful for applications such as industrial maintenance. As the cost of the needed sensors has come down and they’ve been included in consumer phones, consumer applications like visualizing furniture layout and AR-centric games have become increasingly popular. Without a doubt Pokemon Go quickly became the most-popular AR application ever, while its success still hasn’t been matched by anything since, despite the large amount of hype around AR.

Apple and Google have both officially gotten on board the AR bandwagon, with Apple’s ARKit and Google’s ARCore providing the raw material for AR app developers. There are some impressive apps for each environment, but aside from special-purpose applications like the Loew’s Furniture layout app, and a few games like Pokemon Go, applications based on them still haven’t become mainstream.

Mixed Reality (MR)

It didn’t take too long before the idea of having the best of both AR and VR in a single device started gaining ground, and Mixed Reality (MR) was born. The most famous MR device is certainly Microsoft’s Hololens. There are multiple ways to create MR devices. The most common is to use a natively-AR visor/goggle like Hololens does, but with sufficient projected display intensity to completely obscure portions of the actual scene, mimicking VR for a portion of the visual field. But native VR devices can also be used, by adding one or more cameras to the front, so that a “real world” image can be projected by the VR displays with additional virtual elements added as needed.

Unfortunately, Microsoft itself has seriously-muddied the waters by introducing what it calls “Windows Mixed Reality” devices. There really isn’t anything “mixed” about them. They are VR headsets that feature cameras to help with inside-out tracking — meaning they don’t require external beacons — but they do not allow mixing virtual and augmented experiences the way true MR devices do. Their use of the term MR really doesn’t make any sense, except to help distance their new generation of devices from the already-shipping “Windows VR” devices like the Oculus Rift and HTC Vive.

So, What’s XR?

As the acronym list got longer and the definitions got murkier, the industry djinned up an umbrella term that covers all of the possibilities. The X in XR can be said to stand for “Cross” as in “cross reality” or simply as a placeholder variable to cover all the possibilities. Since it is relatively new, using it can make you sound like you’re on top of the very latest, and you won’t have to figure out whether the app or device you’re describing is AR, VR, or MR at least.

One device I might put in the XR category is Google Glass. Glass is frequently described as an AR device, but unlike true AR devices, the Glass display is small and deliberately out of the way. It isn’t designed to overlay anything onto your view of the real world.

VR: The Best Way to Travel to Other Worlds

VR’s biggest strength is immersion. With a proper setup including full motion tracking, high-quality video, and spatial audio, you can start to believe in the reality of entirely artificial constructs. There is no more immediate demonstration of this than the beam-balancing demo. It takes some real mental effort to step onto the virtual balance beam, even though you know you are in a room with a perfectly flat floor. And panicking if you start to step off is a common reaction.

For creativity and gaming, the addition of touch controllers also provides a whole new dimension. Wielding a paintbrush or swinging a sword becomes a natural action. Well-implemented applications let you stay immersed using only the touch controllers, because needing to pull up your headset to find and use your keyboard mid-game is really annoying.

Birdly is by far the most immersive interactive digital experience I’ve ever gotten to try. You can just about convince yourself you are a bird, as you flap and turn through the sky above a city. It is not for the easily motion sick, though, or for those with a fear of heights. If you get a chance when visiting San Jose, it is absolutely worth the additional few dollars The Tech Museum charges for the opportunity. Like the other VR experiences in the museum’s exhibit, it is only for those aged 13 and older.

AR: The Best Way to Bring Things to You

Whether you want to populate your yard with pink flamingoes, or your living room with potential furniture purchases, AR is the preferred solution. It is also the best way to annotate real world scenes with additional data. That can be anything from additional information about a painting in a museum to the service instructions for an elevator needing repair.

Current AR solutions struggle to place objects realistically in the 3D real world, with heavily-funded Magic Leap being one of the leaders in using more-natural lightfield-based projection displays to try and solve the shortcoming.

Magic Leap has a well-deserved reputation for hyping its tech to the max, and this marketing text for its developer headset is no exception.

MR: Pushing Past AR and VR

Current MR devices are best thought of as either VR devices that add cameras so that you can baseline your virtual world with the actual world around you or AR devices that have sufficient display intensity to create a believable virtual reality in some portion of your visual field. No device currently exists that can do an excellent job of both VR and AR, so they all involve tradeoffs that favor one or the other.

While Hololens is certainly the most well known MR device, its limited Field of View means that virtual experiences are not nearly as immersive as when using a VR headset. Likewise, putting a pair of cameras in front of a VR headset and showing you the world in front of you on the headset’s displays is no substitute for looking out at the real thing. One of the most intriguing MR devices I’ve used was from Meta (now reborn as Meta View). It allowed you to create virtual desktops and TV screens at will, and by the final iteration was actually easy enough to use (initially grasping and moving was difficult) that I could imagine myself eventually using something like it.

Future Of XR

AR, VR, and MR technology are all advancing quickly, with new generations of devices rolling out each year. However, the “Holy Grail” continues to be finding applications that will drive them to mass market acceptance. By definition, VR is awkward and isolating, so the experiences need to be both compelling and more than just a novelty. And AR has to get accurate and powerful enough that it becomes the best way for people to accomplish everyday tasks. In the meantime, both industries are flourishing in a variety of commercial and niche applications, so expect continued progress on all fronts.

Now Read: