OpenXR - Mixed Reality (2024)

  • Article

OpenXR - Mixed Reality (1)

OpenXR is an open royalty-free API standard from Khronos, providing engines with native access to a range of devices across the mixed reality spectrum.

You can develop using OpenXR on a HoloLens 2 or Windows Mixed Reality immersive VR headset on the desktop. If you don't have access to a headset, you can use the HoloLens 2 Emulator or the Windows Mixed Reality Simulator instead.

Why OpenXR?

With OpenXR, you can build engines that target both holographic devices, like HoloLens 2, and immersive VR devices, like Windows Mixed Reality headsets for desktop PCs. OpenXR lets you write code once that's then portable across a wide range of hardware platforms.

The OpenXR API uses a loader to connect your application directly to your headset's native platform support. End users get maximum performance and minimum latency, whether they're using a Windows Mixed Reality or any other headset.

What is OpenXR?

The OpenXR API provides the core pose prediction, frame timing, and spatial input functionality you need to build an engine that can target both holographic and immersive devices.

To learn about the OpenXR API, check out the OpenXR 1.0 specification, API reference, and quick reference guide. For more information, see the Khronos OpenXR page.

To target the full feature set of HoloLens 2, you also use cross-vendor and vendor-specific OpenXR extensions.These enable features such as articulated hand tracking, eye tracking, spatial mapping, and spatial anchors. For more information, see the Roadmap section on the extensions coming later this year.

OpenXR isn't itself a mixed reality engine. Instead, OpenXR enables engines like Unity and Unreal to write portable code once that can then access the native platform features of the user's holographic or immersive device, whatever vendor built that platform.


The OpenXR specification defines an extension mechanism that enables runtime implementers to expose other functionality beyond the core features defined in the base OpenXR 1.0 specification.

There are three kinds of OpenXR extensions:

  • Vendor extensions (for example, XR_MSFT_...): Enables per-vendor innovation in hardware or software features. Any runtime vendor can introduce and ship a vendor extension at any time.
  • Experimental vendor extensions (for example, XR_MSFTX_...): Experimental vendor extensions being previewed to gather feedback. XR_MSFTX_... extensions are for developer devices only and are removed when the real extension ships. To experiment with them, you can enable preview extensions on your developer device.
  • Cross-vendor EXT extensions: Cross-vendor extensions that multiple companies define and implement. Groups of interested companies can introduce EXT extensions at any time.
  • Official KHR extensions: Official Khronos extensions ratified as part of a core spec release. KHR extensions are covered by the same license as the core spec itself.

The Windows Mixed Reality OpenXR Runtime supports a set of MSFT and EXT extensions that brings the full set of HoloLens 2 features to OpenXR applications:

Feature areaExtension availability
Systems + sessionsOpenXR 1.0 core spec:
XrInstance, XrSystemId, XrSession
Reference spaces (view, local, stage)OpenXR 1.0 core spec:
View configurations (mono, stereo)OpenXR 1.0 core spec:
Swapchains + frame timingOpenXR 1.0 core spec:
XrSwapchain... + xrWaitFrame
Composition layers
(projection, quad)
OpenXR 1.0 core spec:
XrCompositionLayer... + xrEndFrame
Input and hapticsOpenXR 1.0 core spec:
Direct3D 11/12 integrationOfficial KHR extensions released:
Unbounded reference space(world-scale experiences)MSFT extension released:
Spatial anchors

MSFT extension released:

MSFT extension released in runtime 108: (HoloLens 2 only)

Hand interaction(grip/aim pose, air-tap, grasp)

HoloLens 2 only

MSFT extension released:
Hand articulation + hand mesh

HoloLens 2 only

EXT extension released:

MSFT extension released:

Eye gaze

HoloLens 2 only

EXT extension released:
Mixed Reality Capture(third render from PV camera)

HoloLens 2 only

MSFT extensions released:
Motion controller interaction profiles
(Samsung Odyssey and HP Reverb G2)
MSFT extensions released in runtime 103:
Motion controller render modelsMSFT extension released in runtime 104:
Scene understanding (planes, meshes)

HoloLens 2 only

MSFT extension released in runtime 106:
Scene marker (QR Codes)

HoloLens 2 only

MSFT extension released in runtime 113:
Composition layer reprojection modes(auto-planar or orientation-only reprojection)MSFT extension released in runtime 106:
Interop with other Mixed Reality SDKs
(for example, QR)
MSFT extension released:
Interop with UWP CoreWindow API
(for example, for keyboard/mouse)
MSFT extension released in runtime 103:
Interop with legacy Mixed Reality WinRT APIs
(for example, SpatialAnchorTransferManager))

MSFT extension released in runtime 105:

Other cross-vendor extensions

Official KHR extensions released:

EXT extensions released:

While some of these extensions may start out as vendor-specific MSFT extensions, Microsoft and other OpenXR runtime vendors are working together to design cross-vendor EXT or KHR extensions for many of these feature areas. Cross-vendor extensions make the code you write for those features portable across runtime vendors, as with the core specification.

Where can you use OpenXR?

OpenXR - Mixed Reality (2)

Minecraft's new RenderDragon engine has built its desktop VR support using OpenXR!

Microsoft has been working with Unity and Epic Games to ensure the future of mixed reality is open, not just for HoloLens 2, but across the full breadth of PC VR, including HP’s new Reverb G2 headset. OpenXR powers the cross-vendor VR support for major titles shipping today, such as Minecraft and Microsoft Flight Simulator! For more information on developing for HoloLens (1st gen), see the release notes.

To learn how you can get started with OpenXR in Unity, Unreal Engine or your own engine, read on!

OpenXR in Unity

Microsoft’s current recommended Unity configuration for HoloLens 2 and Windows Mixed Reality development is Unity 2020.3 LTS with the Mixed Reality OpenXR plugin. This plugin includes support for the OpenXR extensions that light up the full capabilities of HoloLens 2 and Windows Mixed Reality headsets, including hand/eye tracking, spatial anchors, and HP Reverb G2 controllers. MRTK-Unity supports OpenXR as of MRTK 2.7. For more information about getting started with Unity 2020 and OpenXR, see Choosing a Unity version and XR plugin.

If you're developing for HoloLens (1st gen), you need to continue using Unity 2019.4 LTS with the legacy WinRT API backend. If you’re targeting the new HP Reverb G2 controller in a Unity 2019 app, see our HP Reverb G2 input docs.

Starting in Unity 2021.3 LTS, OpenXR is the only supported Unity backend for targeting HoloLens 2 and Windows Mixed Reality headsets.

OpenXR in Unreal Engine

Unreal Engine 4.23 was the first major game engine release to ship preview support for OpenXR 1.0! Now in Unreal Engine 4.26, support for HoloLens 2, Windows Mixed Reality, and other desktop VR headsets is available through Unreal Engine's built-in OpenXR support. Unreal Engine 4.26 also supports Microsoft's OpenXR extension plugin. This enables hand interaction and HP Reverb G2 controller support, lighting up the full feature set of HoloLens 2 and Windows Mixed Reality headsets. Unreal Engine 4.26 is available today on the Epic Games Launcher, with MRTK-Unreal 0.12 supporting OpenXR projects.

OpenXR for native development

You can develop using OpenXR on a HoloLens 2 or Windows Mixed Reality immersive VR headset on the desktop. If you don't have access to a headset, you can use the HoloLens 2 Emulator or the Windows Mixed Reality Simulator instead.

To start developing OpenXR applications for HoloLens 2 or Windows Mixed Reality VR headsets, see how to get started with OpenXR development.

For a tour through all the major components of the OpenXR API, along with examples of the real-world applications using OpenXR today, check out this 60-minute walkthrough video:

See also

OpenXR - Mixed Reality (2024)


Top Articles
Latest Posts
Article information

Author: Greg Kuvalis

Last Updated:

Views: 6385

Rating: 4.4 / 5 (75 voted)

Reviews: 82% of readers found this page helpful

Author information

Name: Greg Kuvalis

Birthday: 1996-12-20

Address: 53157 Trantow Inlet, Townemouth, FL 92564-0267

Phone: +68218650356656

Job: IT Representative

Hobby: Knitting, Amateur radio, Skiing, Running, Mountain biking, Slacklining, Electronics

Introduction: My name is Greg Kuvalis, I am a witty, spotless, beautiful, charming, delightful, thankful, beautiful person who loves writing and wants to share my knowledge and understanding with you.