The already formidable Quest 3 mixed-reality headset is set to reach all-new heights. Yesterday, Meta announced plans to bring its 'supercharged' Meta AI assistant to the device in August — racing ahead of Bloomberg reports that Apple has similar plans to bring Apple Intelligence to Vision Pro headsets in the near future.
First unveiled in Sept. 2023, Meta AI has been a core pillar in the company's push to bring AI into its services and hardware over the last ten months. The multimodal large language model (LLM) is similar in scope to OpenAI's ChatGPT, capable of natural language interactions with users across text, voice, and more recently visual inputs.
In a nutshell, Meta AI can understand and reply to anything it reads, sees, or hears, and has already begun to feature in services like WhatsApp, Instagram, Messenger, Facebook, the Meta.AI web portal, and within the company's Ray-Ban Meta Smart Glasses.
The world's first AI assistant for AR/VR headsets
In a post to the Meta Quest blog, Meta revealed that its AI assistant's next destination would be Quest 3 headsets — making use of the platform's mixed-reality abilities to showcase its multimodal capabilities, especially that of translating and understanding real-time video and visuals.
Meta AI will begin rolling out to Quest 3 headsets next month, with the assistant available to users in the US and Canada first. To access Meta AI, you'll need to enable it in the Horizon OS Experimental panel, found within the main Settings panel.
Initially, users will be able to interact with Meta AI through voice prompts, where you can quiz the AI on any topic you're interested in, or even ask for more contextual information about places to eat or visit nearby, as well as using your visual surroundings as part of prompts.
However, Meta state that it will improve the experience over time, which could lead to an explosion in use cases for its AI assistant.
Meta AI on Quest headsets: What we'd love to see
While not everyone is keen to see Meta's AI pop-up in their messaging and social media platforms, there's a boatload of untapped potential in seeing the assistant interact with the world around you.
In one video shared by Meta, the Quest 3's upcoming AI assistant recommends nearby spots to play basketball, another shows it passing out wardrobe advice based on an item of clothing it can see.
However, the potential applications of Meta AI within the Quest headset could stretch far beyond these more advanced Siri/Alexa-like interactions — with the AI gaining an entirely new arsenal of mixed-reality tools and features.
Let's look at some of the ways we'd love to see Meta AI use these tools to transform the Quest experience.
- In-game assistance: We saw the potential for AI to collide with gaming in a unique way during Microsoft's 2024 Build event, with the company's Copilot AI getting involved in a game of Minecraft to instruct users how to play or what actions they should take next.
Meta AI's ability to "see" what the wearer sees could extend beyond its passthrough lenses and reach into the virtual domain, where it could, in theory, recognize certain games and provide similar assistance.
With a lot of Quest titles following an escape room-like format (like the impossibly fun I Expect You to Die 3), having a little mental assistance from time to time could be a great way to make use of the headset's new AI wingman.
- Room setup: The Quest 3 has the capability of scanning its surroundings and identifying key pieces of furniture within your play space already. However, combine that with Meta AI and you could automate things even further.
Imagine taking your Quest 3 headset with you on a trip, ready to break it out in your hotel room to enjoy a cinema-like, large-screen movie night in AR. Now imagine asking Meta AI to set up everything for you while you go about unpacking.
Meta AI could load up your favorite streaming service, prime the film you're waiting to watch, and even position the virtual display in the perfect place for you — meaning all that's left for you to do is crash out and hit play.
- Automatic passthrough: Meta AI's ability to recognize items, landmarks, locations, animals, and people could be used to give you a heads-up as people or pets enter the room, preventing them from catching those hands as you swing your way to victory in Creed: Rise to Glory.
Better still, Meta's assistant could enable passthrough if it notices someone enter the room, pausing your play session and letting you interact with them without ever having to fiddle with controls or remove the headset.
- Hands-free navigation: The Quest 3 has a solid pairing of control methods. The Quest Touch Plus controllers offer incredible accuracy in fast-paced games, while hand tracking gives you a more natural interaction with AR and VR apps.
However, Meta's assistant could unlock a hands-free way of navigating the headset's many menus and interfaces — letting Meta AI do the work for you as you prompt it with voice commands.
A vision of of the future
After seeing the videos accompanying yesterday's announcement, it's hard not to see a vertical slice of Meta's vision of the future when it comes to the merging of wearable tech and AR/AI.
The company is dead-set on its VR/AR ambitions, and a recent sighting of a strange pair of smart glasses in a months old Mark Zuckerberg post to Threads has ignited rumors that the company is preparing to showcase its long-rumored 'true' AR glasses later this year.
Could this new Quest experience be an early example of what Meta hope to achieve in the smart glasses market over the coming years? It's highly likely. However, while CEO Mark Zuckerberg has confirmed that there'll be news to share about future frames this year, Meta's AR smartglasses are still said to be in the prototype stage.
More from Laptop Mag
- I caved to the Apple Vision Pro hype and bought a Meta Quest 3 — did I make the right call?
- 'Google has no real vision or ambition in driving the XR space': Former department lead pours cold water on Google Glass 2 rumors
- Meta to unveil new smart glasses this year after Threads post outs its AR frames
