Researchers have integrated cameras into VR controllers to allow for full-body tracking on VR headsets, which could be the next step for upcoming Meta Quest headsets so you can finally make use of your virtual legs.
Future Interfaces Group, a research lab in the Human-Computer Interaction Institute at Carnegie Mellon University, have tested using cameras in VR controllers with a Meta Quest 2. While not without its limitations, it allowed users to stomp, lean, squat, lunge, balance and perform many leg-driven interactions in VR.
- Project Cambria (Quest 2 Pro): Everything we know about Meta's VR headset
- The evolution of VR fitness: How FitXR is creating the next virtual fitness gym
- Best VR-ready laptops in 2022
Known as ControllerPose, the inside-out body capture method could be used in several leg-centric VR activities made for testing. This included a human Tetris game, a hockey goal demo so users could use their feet to block pucks, balancing exercises for health and fitness, along with Feet Saber, so players could use their legs to cut blocks.
Check out the demo below:
The research was inspired by the lack of lower body capture in most VR experiences. "Some systems are able to capture other joints, such as the legs, by using accessory sensors, either worn or placed in the environment. The latter systems are comparably rare as the additional hardware cost and extra inconvenience of setup has been a major deterrent for consumer adoption," the report states.
While using cameras in a headset are noted to be a more "practical approach," especially with cameras coming up in the rumored Project Cambria, it's clear that line-of-sight to the lower body can be blocked by a user’s clothing or body. The research lab's testing showed that "hands operate in front of the body a majority of the time (68.3%)." In these instances, VR headsets could track motions such as kicking a soccer ball.
However, there were also issues that made it hard for full-body tracking to take place: "In other cases, the hands are either too close or resting beside the body – in these cases, pose tracking will fail or would have to rely on e.g., inverse kinematics using available IMU data."
These aren't the only instances of cameras being found in VR controllers. Leaked images on Reddit last year saw cameras in what seemed to be a Meta Quest controller, while Magic Leap's AR headset saw camera sensors in the Magic Leap 2 controller (via Road to VR). ControllerPose will be shown at the ACM CHI 2022 conference on May 4.
Cameras in VR controllers may be coming soon, and judging from this research, they will finally let players take control of their virtual legs. This would be a big step for VR fitness, along with upcoming games such as NFL Pro Era and Among Us VR —especially without using extra accessories. For more things VR, check out everything announced at the Meta Quest Gaming Showcase.
