iRacing Joins X-Plane 12 In Streaming High-End Simulation To Apple Vision Pro: What About Mac Support?

Dual announcements from iRacing and X-Plane point to a big moment for fans of high-end simulation coming to Apple headsets.

From the iRacing website:

iRacing is pleased to share that through a collaboration with Apple and NVIDIA, the world’s premier motorsport simulation will be available on Apple Vision Pro later this spring with the release of visionOS 26.4 and NVIDIA’s CloudXR 6.0 streaming platform…

Apple Vision Pro delivers a uniquely immersive sim racing experience by seamlessly blending your racing rig with the virtual cockpit, aligning your physical steering wheel with the one in-game. iRacers see their physical hands on the wheel, producing an unparalleled experience that ensures racers remain visually connected to their racing rig while using Apple Vision Pro.

The announcement parallels an earlier one for X-Plane, a high-end flight simulator, with similar language in their announcement:

By leveraging image detection through ARKit, the sim can reliably track cockpit positions, allowing you to seamlessly merge the physical and digital worlds.

The announcements praise the involvement of both Apple and NVIDIA, even though Apple’s role is framed as secondary to NVIDIA’s. In that X-Plane announcement, for example, it indicates streaming from a “PC equipped with NVIDIA’s RTX GPU” and I had to confirm over email they meant a local device rather than a “cloud” one.

X-Plane is sold for Mac as well as PC, but it is not sold through the Mac App Store, and iRacing isn’t available for Mac at all. In either case, Apple isn’t profiting from the sale of some of the most expensive and well-loved simulation software about to be used with its spatial computing platform. That doesn’t sound very Apple-like, does it?

My original report about foveated streaming coming in visionOS 26.4 noted the split-rendering example Apple gave for this functionality:

“Foveated Streaming enables visionOS apps to display high-resolution, low-latency immersive content from streaming endpoints,” Apple’s documentation explains. “On Apple Vision Pro, Foveated Streaming allows you to display visionOS spatial content alongside streaming content. For example, a flight simulator app can render a cockpit using RealityKit, and stream a processor-intensive landscape from a remote computer to the device.”

This level of alignment between physical and virtual content, plus the technical complexity of split rendering, is practically unheard of in VR. Alignment typically requires extraordinary expense and time to deliver and most of the solutions fall short of what’s promised here without robust end-to-end platform support. To my knowledge, only Varjo has even attempted anything like this with the same degree of quality and immersion, and their headsets can be far more expensive than the Vision Pro.

Even close watchers of the VR industry don’t quite understand the implications of what Apple might do here for high-end simulation.

High Frame Rates Can Raise The Bar For Comfort In Simulation

Since 2016, with both the Rift and Vive headsets, 90 frames per second has been set as the bare minimum for comfort in VR.

Valve, with its open PC culture, didn’t restrict developers, instead allowing them to set their own minimum and recommend specifications for what the developer deems is a good experience. That’s changing with the coming Steam Frame, with Valve now requiring 90 frames per second for verification of comfort in standalone.

To enable Meta’s mad rush into standalone VR on cell phone-class chips from about 2019 — and to try to fork some developers away from PCs and Valve — the entire industry has been warped by Mark Zuckerberg’s spending into using a variety of tricks to produce comfortable frame rates for VR. Meta declares 72 FPS as the minimum rate on Quest standalone and also allows developers to deploy an array of solutions to essentially drop below that minimum, quite a bit in some instances, and then warp the scene or simulate missing frames to meet that lower rate for delivering a sense of comfort.

I call bullshit on the idea that the headaches people complain about from Quest headsets are entirely due to the weight and balance of the headset, or to the mismatch between simulated locomotion and physical movement. I think a substantial amount of fall-off in Quest usage is due to these tricks being deployed invisibly and frequently, whether in standalone or streaming from a PC, with the effect manifesting as a VR player simply feeling like shit after too much use.

Skip this paragraph if you’re squeamish, but it would be like being blindfolded and using a cheese grater to slice off bits of your favorite cheese. Sometimes, without warning and through no fault of your own that you cannot avoid, you cut off some of your hand too. You find yourself blinded to what you’ve done and taking a bite of bloody cheese, only to blame the cheese for tasting weird without knowing why any of this has happened. If that sounds like madness, it is, but it’s kind of like what happens when experiencing VR games that perform poorly, and I’m convinced it’s one of the reasons Meta noped out of its current position in VR so hard after cutting all these corners.

When I used the Galaxy XR headset running Android XR last year I was astounded to find that the entire operating system seemed to run, by default, at 72 FPS. I only experienced 90 for the first time when I installed Guy Godin’s Virtual Desktop and the difference was dramatic — I didn’t want to leave the app. As Valve’s Jeremy Selan noted when I visited Valve last year to try Steam Frame, “spatial resolution in VR is both a function of the spatial resolution of your displays at 2160, but also a function of frame rate. Because as you move your head with very small amounts of angle, the number of samples per second, essentially, combines spatially and temporally to give you that information.”

"That’s one of the reasons, in addition to gaming, that we’re so excited about those higher refresh rates,” Selan said of the device, which has an experimental mode running up to 144 Hz.

When I asked VR engineer Mark Schramm about his Recreation Room, he mentioned he drives and flies in VR with 96 gigabytes of RAM in a NVIDIA 5090 system with a DOF Reality Motion Rig and a quick-change system he built specifically to swap quickly between the two activities. He’s usually able to supersample with that rig, but noted that Microsoft Flight Simulator 2024 still struggles a bit with that. That’s insane too.

It should not cost the equivalent of a downpayment on a house to enjoy solid frame rates at super high resolutions in VR. The structural limits on the VR market are real due to bad decisions made by GPU makers and AI bros. It’s all somewhat invisible to practically everyone except the people who genuinely want to practice their favorite hobbies at home for many hours without developing a headache.

The simple environments of Beat Saber make it one of the few games that can often run at much higher frame rates with extra comfort. Most assume gameplay is the reason for that title’s success, but I’d argue it’s also the high frame rates.

Apple’s Opportunity

Apple can bring both iRacing and X-Plane into the Mac App Store and carve off their standard cut of sales by solving these problems with Apple Silicon and split rendering.

That paragraph above quoted from Apple notes local headset rendering of a cockpit but you can apply the same idea to the interior of a car. This can do two key things for people experiencing a high-end simulation in VR. RealityKit may be able to render that cockpit or car at the headset’s base frame rate, potentially up to 120 FPS on the M5 Vision Pro. That in and of itself would be a dramatic win for VR comfort, while the content itself provides a continuous framing to the user helping to ground them. Vignetting, for example, is an option many games deploy to reduce discomfort in stick-based movement systems, artificially reducing what you see to a small circle. This reduces how much of your field of vision receives a mismatch from what your body feels. The car’s cabin or the plane’s cockpit can do the same thing automatically in every moment.

I noted in my podcast with Cubism and Laser Dance developer Thomas Van Bouwel that his work in this medium is at two ends of the spectrum when it comes to the idea of not using artificial locomotion. Laser Dance sees you physically moving through a room to avoid a laser maze cut to the size of your environment while Cubism is a spatial toy kind of like a Rubik’s Cube that keeps you seated the entire time. Cubism would be right at home in visionOS while Laser Dance on a Vision Pro might not work so well because the system simply isn’t optimized for low latency walk-around mixed reality.

Mark Zuckerberg called mixed reality like he deployed in Quest 3 “the better product period” in 2024 while the number of developers who embraced the idea, like Van Bouwel has, are vanishingly small. I propose that’s because Zuckerberg was likely reading from a script when he said that rather than from his personal desire to experience something like what Schramm’s rig provides in supreme comfort at more reasonable prices.

“It’s of course common sense that the higher the title and the power, the more pressure there is for immediate returns as their guide, focused more on financials and less so on the actual research hours into understanding this medium for themselves,” long-time VR and AR developer Steve Lukas wrote.

Now imagine the new M5 MacBook Pro carrying as much RAM as Schramm packed into his PC, or a new model Mac Studio, carrying that same chipset and sucking down power to turn into rendering the view you see when you lean over in your comfortable jet’s cockpit to look out the window at supersonic speeds.

The opportunity is so large here Apple could single-handedly crash the AI economy with demand for comfortable and more affordable local simulation like this, especially if it could combine such a robust offering with a mainline Vision Air headset that drops weight and cost. Of course, you can always turn on DLSS 5 and let your senses swallow NVIDIA’s hallucinations instead. Is that what most people want?

Me? Now’s a good time to put my subscribe button down there so you can maybe help me start sequestering funds to build a local sim rig that’s genuinely high frame rate, high resolution, and comfortable.