When Mark Zuckerberg restructured Facebook as Meta in October 2021, he predicted:
We believe the metaverse will be the successor to the mobile internet, we’ll be able to feel present – like we’re right there with people no matter how far apart we actually are.
After investing US$36 billion, Meta has reportedly shut down its metaverse project.
A recent report by the UK’s Ada Lovelace Institute (Rincon and Perez) examines why predictions of large-scale adoption of virtual reality technologies have stalled, while identifying the niche applications where immersive technologies are succeeding. The study also points to a new phase of innovation where the virtual and real worlds increasingly overlap.
A gradient of immersion
The Lovelace report describes the differences between virtual and non-virtual technologies (including general purpose AI) as follows:
Traditional digital interfaces, such as digital screens, function as maps of digital information that can be searched. In contrast, immersive technologies seek to transform a map into territory that users can explore and manipulate.
Immersive technologies fall on a spectrum:
Virtual Reality (VR): VR is fully immersive. Users are transported to entirely digital environments, which can mimic reality or create fantastical scenarios. Users’ senses are tricked into thinking they’re in a different environment.
Augmented Reality (AR): AR overlays digital information over real-world elements while keeping the real world central. For example, IKEA’s Kreative app allows you to substitute IKEA products for your existing furniture in your living spaces.
Mixed Reality (MR): MR combines real-world and digital elements. While combining both real and virtual worlds like AR, MR lets users bring virtual objects into the real world and manipulate them as if they were real. MR creates a two-way interaction: it can place holograms in the physical world and when a user is in virtual reality, physical obstacles such as walls or furniture are digitally represented so that the user can avoid collisions. For example, Microsoft’s HoloLens allows you to tether a virtual book to a bedside table so that when you go to bed, it’s there for you to pick up where you left off.
Evolution of immersive technologies
Immersive technologies date back to the 1960s with Morton Heilig’s patent of the Sensorama Simulator, a device with a head-mounted binocular display, stereophonic speakers and mechanisms to release odours to ‘stimulate the senses of an individual to simulate an actual experience realistically’.
Technical advances, particularly in video quality and hand controls (haptic devices) have dramatically increased the capability to ‘trick’ the senses into believing the virtual is real. For example, Nintendo’s Virtual Boy, released in 1995, had a display resolution of 384 x 224 pixels while Apple’s Vision Pro, released in February 2024, has a resolution of 3,800 x 3,000 pixels.
Yet after a COVID era spike, consumer and investor interest in immersive technologies is declining:
Sales of Meta Quest headsets were down by 16% in November 2024 compared with 2023.
Total start-up funding for immersive technologies fell by more than half between 2022 and 2023.
What went wrong?
The Lovelace report identifies the following bottlenecks to large-scale adoption of immersive technologies as a general-purpose technology:
Hardware: headsets can be uncomfortable because of their weight, ‘bulkiness’ and unattractive design. There are also challenges with usability; extended use can cause eyestrain and nausea (cyber sickness) in some users.
Challenges integrating into daily life: even if we spend more time on screens, we sometimes want to share on-screen content with those around us and gauge their reactions through facial expressions. Wearing a VR headset isolates you from those interactions.
Technical issues: immersive technology products can be glitchy, require significant computing power and ample physical space to function effectively.
Preference for incumbent technologies: limited consumer applications exist, and few offer enough added value to compel users to switch.
Cost: the Apple Vision Pro retails from GBP3,499.
What has worked?
The Lovelace report highlights the growing success of niche applications of immersive technologies, including:
Process design: prior to the commencement of construction on BMW's new factory in Debrecen, Hungary, a digital twin of the factory was created, allowing designers to virtually explore, observe assembly lines and modify designs – boosting efficiency by 30%.
Product quality inspection: Siemens uses digital twin and VR technologies to help manufacturers design and optimise automated quality inspection stations, reducing the need for costly physical prototypes. For example, engineers can virtually develop and test a robotic gripper to handle smoke detectors on an assembly line safely and efficiently, ensuring precision and preventing damage.
Remote meetings: video conferencing poorly conveys non-verbal communication which in face-to-face meetings can help to regulate conduct, such as eye contact from the meeting manager facilitating turn-taking between meeting participants. In a study using VR to conduct meetings, many participants found facial expressions and body language easier to discern, improving engagement. Participants also reported being less distracted by emails and texts.
Surgery: surgeons typically use video feeds during operations, but MR adds a 3D view that can be superimposed on real-time video. VR also supports surgical planning, student training and patient education.
Public services: the Seoul City Council has built a hyper-realistic community space called Metaverse Seoul where citizens can visit the virtual mayor’s office, walk around the city’s top tourist attractions, file complaints, obtain documents and pay local taxes.
Neurodivergence: an immersive app called EMooly supports social emotional learning for autistic children by generating personalised content featuring interactive activities shared with a caregiver and AR avatar.
Mental health: the UK Government’s research and innovation agency has allocated GBP20 million for investment in projects that deliver digital mental health therapeutics.
Risks
Immersive technologies amplify the risks associated with social media, AI and other digital technologies because they collect far more information about the user. As the Lovelace study notes:
Through their wearable hardware and sensors, many immersive technology products are able to create and collect data at a significantly greater scale than non-wearable products based on mobile phones and laptops. The use of wearable devices also entails the collection of more sensitive data, such as biometric data, which often consists of unconscious and involuntary physiological responses… The volume and sensitivity of data collected by immersive products is further amplified when user interactions are managed by LLMs, which require large volumes of data to function. The integration of LLMs may also prompt the user to disclose more sensitive data through natural language interactions.
This biometric data includes eye or head movements and changes in posture. The Lovelace paper warns that some of this biometric data may not be protected by recent biometric-specific amendments to EU and UK privacy laws.
There have been suggestions that such data could be used to target advertising, monitor student engagement or measure employee productivity and safety (for example, detecting when an employee ‘nods off’). A recent survey found that around 79% of companies in OECD’s surveyed European countries report adopting workplace monitoring.
However, the Lovelace paper sounds a caution:
Use cases drawing on immersive evaluation functions in workplace contexts may use employees’ physiological signals to infer behavioural traits like levels of productivity. These inferences can inform employment-related decisions, despite often being based on contested or emerging scientific assumptions.
Other commentators argue that the use of biometric data in immersive technologies can cause broader ‘socio-technical risks’, including alienation, stereotyping and overwork.
The Lovelace paper also raises concerns about Big Tech dominance in immersive technologies: noting, Meta’s 70% share of the AR/VR headset market.
The future
The Lovelace paper forecasts a potentially brighter future for immersive technologies, including the emergence of consumer-focused products, driven by three key trends:
More convenient, cheaper headsets for everyday use. The Ray-Ban Meta smart glasses, priced at about US$800, allow users to take photos and videos to make calls and play music while moving through their day. The next version will reportedly include a video overlay, while not yet achieving full holographic display capability, this represents a step towards it through the use of machine vision for object recognition.
Integration of AI and immersive technologies, which allows for user generated synthetic content creation and user interaction management – rather than just generating that image of a dog with a hat eating ice cream, you will now be able to shake its paw.
More sophisticated MR technology seamlessly merges virtual and physical worlds, projecting holograms into real spaces visible to others wearing MR glasses.
Peter Waters
Consultant