Extended reality: what is XR?

A Pokémon that seems to run across the street in front of us on our screen? A 360° viewing of a property available for sale in another country? A sketch book with char­ac­ters that appear in 3D after coloring them in? These are examples of extended reality. The umbrella term covers all immersive tech­nolo­gies that expand our real world and combine it with virtual elements. We explain what exactly extended reality is and what dif­fer­ences and sim­i­lar­i­ties exist between the various tech­nolo­gies: augmented reality, virtual reality, and mixed reality.

The de­f­i­n­i­tion of extended reality – also known as cross reality (XR) – is any tech­nol­o­gy that creates computer-generated en­vi­ron­ments or objects. The term en­com­pass­es both developed and future forms of XR. The various tech­nolo­gies are dif­fer­en­ti­at­ed primarily by their re­la­tion­ship between the real and virtual world. While augmented reality users perceive virtual objects as an extension of the real world, users of virtual reality immerse them­selves in a purely virtual world. Another key aspect of the extended reality de­f­i­n­i­tion is that it always concerns an immersive tech­nol­o­gy.

Three forms of extended reality currently exist. We introduce these tech­nolo­gies as well as their dif­fer­ences and sim­i­lar­i­ties below:

Augmented reality (or AR) expands on the real world by inserting virtual objects and in­for­ma­tion into the real world. Virtual elements include images, text, and an­i­ma­tions. In the case of augmented reality, per­cep­tion still centers around the real en­vi­ron­ment. Users are only able to interact with virtual objects to a limited extent – if at all.

Augmented reality is so far the most commonly used form of extended reality. One main reason for the success of this tech­nol­o­gy is the low technical re­quire­ment. Augmented reality can be used with just a smart­phone, tablet, or PC, making it easily ac­ces­si­ble to end users. What’s more, special AR glasses are also available that project virtual objects in front of the wearer’s eyes. Here, the external world remains visible.

A par­tic­u­lar­ly well-known and suc­cess­ful example of augmented reality is the game Pokémon Go – available as an app for iOS and Apple. It is among the most suc­cess­ful games for mobile end devices and, having generated more than a billion dollars of profit, it is also the most suc­cess­ful AR game to date. With Pokémon Go, the mobile device displays digital char­ac­ters (Pokémons) in the real world and thereby builds on reality. Extended reality has also arrived on social media channels: Filters allow users to add virtual objects like hats, sun­glass­es, or make-up to their photos and videos, for example.

Virtual reality (ab­bre­vi­at­ed to VR) creates a fully computer-generated 360° en­vi­ron­ment. Users com­plete­ly immerse them­selves into the virtual world and interact with it, while the real world is hidden from view.

In contrast to augmented reality, special devices are required for this XR or extended reality tech­nol­o­gy. Wearing a head-mounted display, users visually perceive the virtual world; the real world is fully concealed and invisible during use. The degree of immersion with this closed system is es­pe­cial­ly high and allows users to dive right into a virtual world. These devices usually require a computer con­nec­tion. The best-known example of a VR headset is Oculus Rift by Facebook.

Mixed reality (or MR) works by combining elements of AR and VR. It’s the latest and most complex immersive tech­nol­o­gy. This form of extended reality combines our real world with virtual en­vi­ron­ments, thereby creating new sur­round­ings. The user si­mul­ta­ne­ous­ly interacts with the real and virtual en­vi­ron­ment. Physical objects in the real world have an effect on digital elements. The use of this tech­nol­o­gy requires an MR headset and a lot of pro­cess­ing power.

An example of a mixed reality headset is the Microsoft HoloLens presented in 2015. The headset trans­par­ent­ly displays 3D objects in the user’s direct en­vi­ron­ment. The head-mounted display is equipped with in­te­grat­ed sensors, speakers, and its own computing unit. No ad­di­tion­al equipment is required.

The following table shows the three current extended reality tech­nolo­gies in a com­par­i­son:

Find out more about the different types of XR – augmented reality and virtual reality – in this video:

Some augmented reality ap­pli­ca­tions are already part of our daily life, such as the apps mentioned above like Pokémon Go or popular filters on social media. The use of virtual reality is advancing par­tic­u­lar­ly in the gaming sector, but there are still technical re­quire­ments that stand in the way of mass adoption. By contrast, mixed reality is pre­dom­i­nant­ly limited to companies. A number of chal­lenges still need to be overcome for these latter two forms of extended reality to become common in everyday life. We outline the major hurdles below:

AR tech­nolo­gies like Pokémon Go or social media filters are ac­ces­si­ble for any smart­phone user. However, the use of virtual and mixed reality requires rel­a­tive­ly expensive equipment. These immersive tech­nolo­gies are therefore primarily appealing to companies or specific target groups like gamers or very tech-savvy in­di­vid­u­als.

Lots of detailed as well as personal data is often collected with the use of XR tech­nol­o­gy, including the user’s personal en­vi­ron­ment. For this reason, com­pli­ance with all data pro­tec­tion re­quire­ments is a challenge that de­vel­op­ers need to face up to if they want to make their extended reality products ac­ces­si­ble to a broad range of users.

During a Pokémon Go gaming event with over 20,000 par­tic­i­pants, serious network problems arose. The event or­ga­niz­ers were forced to refund all admission proceeds and accepted losses in the millions. This is a striking example of the fact that wide­spread use of extended reality also calls for IT in­fra­struc­ture upgrades.

There is still de­vel­op­ment potential in terms of the technical devices them­selves: For instance, the Microsoft HoloLens can be con­trolled with gestures and voice commands, but gesture controls are limited to just two hand movements. Moreover, head-mounted displays are often too heavy and un­com­fort­able for prolonged use. For end customers, the design is also an important purchase factor which man­u­fac­tur­ers fre­quent­ly give in­suf­fi­cient con­sid­er­a­tion to.

The extended reality market is ex­pe­ri­enc­ing constant growth. So far, it has typically been the en­ter­tain­ment industry at the forefront, where most in­vest­ment is made in de­vel­op­ing and dis­trib­ut­ing the tech­nol­o­gy. But XR and its versatile ap­pli­ca­tions are becoming in­creas­ing­ly relevant in other sectors – such as in the medicine, military, in­dus­tri­al, and tourism in­dus­tries.

In the en­ter­tain­ment industry, the best-known ap­pli­ca­tion of extended reality is video gaming. This is also due to the fact that the gaming industry adopted VR tech­nol­o­gy rel­a­tive­ly early on, making it available to end customers. But the new tech­nolo­gies are promising many new op­por­tu­ni­ties for events like concerts and sports com­pe­ti­tions. Par­tic­i­pants could visit events from the comfort of their home via an internet con­nec­tion and still ex­pe­ri­ence a real concert or theater per­for­mance thanks to extended reality and a virtual en­vi­ron­ment. Event or­ga­niz­ers would then have the pos­si­bil­i­ty to increase their ticket sales con­sid­er­ably.

Existing and prospec­tive customers can discover and try out products in a virtual en­vi­ron­ment with the use of XR. In addition, this tech­nol­o­gy offers companies new ap­proach­es for in­ter­act­ing directly with customers remotely.

Extended reality offers sig­nif­i­cant ad­van­tages for companies and customers in this sector: Prospec­tive buyers or tenants of an apartment or house can con­ve­nient­ly view prop­er­ties from their own home. This reduces the workload of real estate agents and owners. When planning a new property, the immersive tech­nol­o­gy lets ar­chi­tects and designers visualize their ideas.

Extended reality enables companies to let potential customers try out their products. Here, immersive tech­nolo­gies represent a sig­nif­i­cant op­por­tu­ni­ty for online stores. When online shopping, customers are normally unable to see and touch products before they buy. At least the former is now made possible with virtual coun­ter­parts thanks to the use of XR. Learn more about the use of augmented reality in e-commerce in the article “Augmented Reality in E-Commerce”.

Extended reality also offers added value for customers in brick-and-mortar retail. The furniture store IKEA is already using this tech­nol­o­gy – in the form of augmented reality. Using the cor­re­spond­ing smart­phone app, customers can project items of furniture into their own home. This shows what a selected piece of furniture would look like in their home, easing the purchase decision.

From the military and medical sectors to aerospace: With many pro­fes­sions, it is difficult or even im­pos­si­ble to train for risky sit­u­a­tions without exposing people to danger. Thanks to XR tech­nol­o­gy, trainees are able to train their behavior and actions in such sit­u­a­tions within a safe en­vi­ron­ment, allowing them to prepare ac­cord­ing­ly.