Vision Pro Foveated Streaming In 26.4 Beta Hints At Big Things For Apple Headsets
Open source contributors working on the ALVR project for streaming PC games to headsets over Wi-Fi are digging into a new Foveated Streaming framework from Apple that may lead to improved streaming quality for PC VR games.
What’s most intriguing is the idea that the framework might lead to a new kind of experience that uses a combination of local and remote rendering.
“Foveated Streaming enables visionOS apps to display high-resolution, low-latency immersive content from streaming endpoints,” Apple’s documentation explains. “On Apple Vision Pro, Foveated Streaming allows you to display visionOS spatial content alongside streaming content. For example, a flight simulator app can render a cockpit using RealityKit, and stream a processor-intensive landscape from a remote computer to the device.”
That sounds great. I want more.
Taking Off A Gravity Glove To Grab iPhone
What if your iPhone in hand was rendered as “spatial content” in RealityKit instead of seen with low resolution passthrough?
In a hypothetical scenario playing Half-Life: Alyx or Elite Dangerous streaming from a nearby PC over Wi-Fi, a call on the phone would vibrate in my pocket just like anywhere else. Instead of popping up in my vision and killing immersion, I would have the choice to ignore the phone or reach into my pocket.
PlayStation VR2 controllers would function like gloves in this scenario. Take off your Gravity Glove by just dropping one of the controllers to hang from its wrist strap, and then pull out iPhone instead. Alyx’s hand would be replaced with your actual hand holding an iPhone in City 17, with clarity for the screen far higher than passthrough.
Finish interacting with reality by putting the iPhone away and grabbing your controller again.
If Valve would consider selling Steam Frame controllers in Apple stores, separate to its headset, Vision Pro buyers would have a more portable option for haptic hands than Sony’s ringed solution.