meta-quest/immersive-web-emulator

Browser extension that emulates Meta Quest devices for WebXR development. Lead: Felix Zhang ([email protected])

title	description	tags
Immersive Web Emulator	Describes how to use the Immersive Web Emulator to run WebXR apps on a desktop browser without an XR device..	WEB_PLATFORM WEBXR

Immersive Web Emulator is a browser extension that assists WebXR content creation. It enables developers to responsively run WebXR apps on a desktop browser without the need of an XR device.

Inspired by the official WebXR Emulator Extension by Mozilla Reality and our previous efforts of extending it for better functionality, Immersive Web Emulator is designed and rebuilt from the ground up with an emphasis on full input emulation (including touch and analog input), better usability with a re-engineered UI, and more features, such as keyboard input mirroring and input session recording/playback, specifically for Meta Quest headsets.

Immersive Web Emulator Features

• WebXR API polyfill
• 6DOF transform control for the headset and left and right controllers, powered by Three.js
• Full input emulation support for Meta Quest Touch controllers
• External input mirroring support for both keyboard and gamepad
• Cross browsers support with WebExtensions API

Installation

This extension is built on WebExtensions API and implements Manifest V3. It works on Chrome, Microsoft Edge, and other browsers that support the API. The immersive Web Emulator can be installed from the following browser extension stores:

• Google Chrome Web Store Page
• Microsoft Edge Add-ons Store Page

Manual Installation

For other Chromium-based browsers, you can download our latest release, and try to manually install the extension by:

1. Download the latest release here, unpack it and note the unpacked directory.
2. Open the Extensions page of the browser, it can typically be found in the browser menu, or you can try navigating to "chrome://extensions" in the URL bar if the browser is Chromium-based.
3. Locate and Enable developer mode on the Extensions page.
4. Locate and click on the Load unpacked button and select the unpacked directory from step 1.

The process may vary for different browsers.

Immersive Web Emulator Usage

After successfully installing the emulator, do the following to use Immersive Web Emulator:

1. Go to a WebXR app page (for example, see the WebXR Examples). Notice that the app detects an XR device (emulated) and allows you to enter the immersive VR mode.
2. Open the Immersive Web Emulator tab by going to the Chrome settings button and selcting More tools > Developer tools and looking for it on the tab bar at the top of the screen. You might need to click >> if there are many tabs. From there, you can control the emulated devices. You can move the headset and controllers, and trigger the controller buttons. You can see their transforms reflected in the WebXR application. Note that the Immersive Web Emulator tab is only available on WebXR app pages.

Device Nodes Transform Controls

By clicking a device node in the emulator's 3D viewport, you can select gizmo mode of the device. By dragging a gizmo, you can rotate or translate the device. Alternatively, you can use the following keyboard shortcut to cycle through gizmo modes of the different device nodes:

Device Node	Keyboard Binding
Headset	Number Key 1
Left Controller	Number Key 2
Right Controller	Number Key 3

Meta Touch Controller Emulated Controls

Button Action	Description
Touch	Toggle 'GamepadButton.touched'.
Press	Set 'GamepadButton.pressed' to true and revert after 0.25 seconds.
Lock	Toggle 'GamepadButton.pressed'.
Joystick	Set values on 'Gamepad.axes'.
Joystick-Sticky	Toggle the auto-return feature of the emulated joystick.
Joystick-Reset	Recenter the emulated joystick.
Slider	Emulate analog input for trigger and grip.

Stereo Effect

You can enable/disable Stereo Effect which renders two views.

Pose Controls

You can save an arbitrary combination of device nodes transform data as the default starting pose of the emulator.

Keyboard Control & Events Pass-Through

There are keyboard mapping built in for some important controller emulated controls:

Button Action	Keyboard Binding
Left Joystick	W/A/S/D
Left Joystick Click	C
Button X	X
Button Y	Z
Left Trigger	E
Left Grip	Q
Right Joystick	Arrow Keys
Right Joystick Click	.
Button A	'
Button B	/
Right Trigger	Enter
Right Grip	Shift

Keyboard events other than those reserved for controller emulation are passed through to the main WebXR experience, you may wish to utilize this feature to build in some shortcuts for debugging purposes.

Note

• Even if native WebXR API is available the extension overrides it with WebXR polyfill

WebXR Examples

• WebXR Samples
• Three.js WebXR VR examples
• Babylon.js WebXR examples
• A-Frame

Contributing

See the CONTRIBUTING file for how to help out.

License

Immersive Web Emulator is MIT licensed, as found in the LICENSE file.

immersive-web-emulator's Issues

Head is always at 1.75cm even when not using local floor

When using local floor, it's nice to have the head at a default normal height, but when local floor is not enabled, the head should be at height 0, because in that case if u enter VR normally that would be the case.

Having it set at 1.75 make it so I have to make work-around when the device is emulated to adjust the height of the head so not to have it super high.

Reference Space Type is "viewer" in the frame the session start, instead of the proper one

If I set the space type to "local floor", the emulator does actually set it as local floor, but only the frame after the session started (or so).

U can check this by simply logging the space type on session start.

On a real headset, the space type will be the proper on session start already.

This make it so I have to add special cases for the emulator.

Input button retains state at page load before entering immersive mode

Notice that if I press the buttons below (to switch from hands to controller or viceversa) before entering immersive mode it has no effect. It retains the option / state that had at page load. It works as expected once I enter immersive mode (I can switch from hands to controllers or viceversa)

Connection Status is missed (gamepad.connected == undefined)

EmulatedXRDevice.gamepads objects have no "connected" property.
After entering in VR mode, gamepad has property "connected" = undefined.
Modules that depend on this status may not work correctly.

gamepad axes are undefined

xrFrame.session.inputSources[n].gamepad.axes[0] can be undefined even though the gamepad object exists and is not null.

This apparently happens for the Quest Pro emulator, perhaps as a result of using a polyfill.

The Mozilla extension reports a number as required by the gamepad spec.

Emulator room/cube appears in-scene while emulator is on

The hex room/cube appears in the scene

Developing in AFRAME

Simulator does remove canvas from custom web components and after session the canvas is not in the document anymore

Hello, currently when ending the XRSession the renderer seems to be broken.

It works as expected on device returning to the original rendering state

20240104-134440_Test_-_Google_Chrome.mp4

The reference space type is not updated until the frame after the session started

In the actual quest, the new reference type is already local floor when the session start, but with the emulator is not which is annoying and requires me to have special cases to handle the emulator if I want to test with it.

As u can see in the video, if I ask for it after the session started is properly updated to local-floor in this case.

2689.mp4

This is with Wonderland Engine

Add an option to specify the emulator current reference space type

I think that currently the emulator picks "local-floor" if enabled, and switch to "local" if not or something like that.

Would be nice to be able to test what happens on different headset that might not have local-floor by disabling it manually in the emulator

XRSession missing enabledFeatures property

Details: https://immersive-web.github.io/webxr/#dom-xrsession-enabledfeatures

WebXR Anchors module always enabled.

The emulator doesn't require 'anchors' to be passed as optional feature to navigator.xr.requestSession

To match spec behavior. The optional feature should be required and if not available an error should be thrown when calling frame.createAnchor ...not available for this kind of session...

Reset default pose

Hi, is there a way to reset the pose to the original default pose?

Feature Request: sample hand animations and hand tracking support

For testing custom hand models, it would be great if this extension had a few hand recordings packed with it to choose from.

Screen-based AR support

I understand this may not be a priority for Meta, but I wanted to capture my thoughts here nonetheless.
We're using the simulator for AR cases as well, including phones.

A couple of behaviours differ right now that could be improved:

• DOMOverlay element isn't styled as it should, should be using styles like this to ensure it fills the screen
• Clicks / touches should simulate controllers in targetRayMode screen

And some more of lower importance:

• leaving AR sessions ends up in a bad state (this is just the case with the simulator right now but could be better I think)
• ability to choose the environment blend mode and simulation for those modes

Cycling gizmo modes via numpad keybindings not working.

Trying to cycle the gizmo modes is not working. Tested on multiple websites.
Sorry for non-descriptive issue. I am willing to provide more info if needed.

Also is there a way to group move headset and controllers together?

Thanks for your response.

Ability to turn WebXR shim off without uninstalling the extension

Maybe I'm missing something, but seems that currently when the extension is installed it always polyfills - since we also want to test codepaths where XR is not available that's not very helpful, so I had to remove the extension again.

Could maybe be as simple as having "None" in the device dropdown?

This also extends to a note written in the docs:

Even if native WebXR API is available the extension overrides it with WebXR polyfill

which is also not really ideal... I find the extension useful but constantly uninstalling / installing it while testing something is a bit.

Record controller and hands events for later playback

To be able to integration test WebXR applications in commodity virtualized Linux containers, it would be useful if the emulator could record/save and load/playback a series of controller, hands, and plane detection events.

Being able to get this quality feedback before going to expensive on-device testing is key for scaling engineering efforts in an economically defensible way.

FWIW, we took this same approach on a game console project I worked on where we used react-native-web and injected some extra gamepad extensions, so we could get app startup profiling data and basic integration smoke test feedback before running a fuller test suite on rare prototype hardware.

The WebXR simulation window cannot be found in the debug window that appears after pressing F12 in the Edge browser.

The WebXR simulation window cannot be found in the debug window that appears after pressing F12 in the Edge browser.

Error in the chrome extension manager

This is the error I get when I tried to run a WebXR sample from this link-
https://immersive-web.github.io/webxr-samples/inline-session.html

Can someone help me understand what is causing this?

Add/Remove Controller / Hand

Hi, thanks for working on this

I have two questions regarding input:

1. I can't find an option for adding or removing controllers or hands at runtime. I'd like to be able to test the removal of one or both controllers (or one or two hands)
2. Is it possible to mix and match hands + controllers (e.g. one hand and one controller) - It looks like this is not yet exposed to WebXR but in discussion?

Removed meshes remain in frame.detectedMeshes

I enter immersive-ar session and in the meshes tab I can add new ones (desk, lamp, ceiling). The new ones are added in frame.detectedMeshes but those I remove remain until I reload the page and enter immersive-ar again

Extension Store release?

Hey - is it planned to release this on the Chrome Extension store to make it more widely available?
Thanks!

Change mesh size

I can add / remove meshes but cannot change the size of an existing one. My understanding is that the geometry of detected meshes can change during an immersive session (lastChangedTime parameter). It would be useful if that can be also emulated.

Unexpected `press` or `selectstart` behaviour

Hi, see attached video

expected:

• down and holding "press" button in simulator presses the button (value increases to 1) - triggering "selectstart"
• releasing "press" button releases the button (values decreases to 0) - triggering "selectend"

actual:

• down and holding "press" button does have no imediate effect
• releasing "press" button increases and decreases value and triggers "selectstart" and "selectend" at once

Type in controllers / headset position and rotation

Sometimes for testing one wanna try precise values.

Bug: On smaller screen only half of the scene is shown

What the title says, when using a bigger external monitor the scene looks as it should. But on a smaller screen (macbook pro 14') only half of the scene is shown, like if only seeing from one eye.

Pro controller has grip and trigger but is missing third top plate pressure sensitive trigger

The pro controllers has a third pressure sensitive button on the top plate adjacent to the X/Y buttons this is missing in the extension

THREE version used in emulator leaks into content.

Get the following warning in A-Frame:

WARNING: Multiple instances of Three.js being imported.

By the time A-Frame loads theres a THREE r147 revision defined

requestHitTestSource doesn't check for feature request

The XRSession functions requestHitTestSource and requestHitTestSourceForTransientInput are not checking if the hit-test feature was requested.
As described in the specs
https://www.w3.org/TR/webxr-hit-test-1/#requesting-hit-test
"If hit-test feature descriptor is not contained in the session’s list of enabled features, reject promise with NotSupportedError and abort these steps."

Error: Manifest version 2

When I installed this on Chrome, I got this error:

Manifest version 2 is deprecated, and support will be removed in 2023. See https://developer.chrome.com/blog/mv2-transition/ for more details.

Add a button to trigger a View Recenter event

With this I mean what happens if u long press the oculus button.
In my case, working on locomotion features, I want to recenter to actually keep the same view so it would be nice to be able to test it.

Include WebXR Layers API polyfill

It appears these layers samples don't work unless this polyfill is used.

Panel no longer appears

For some reason the devtools panel no longer appears.

https://threejs.org/examples/webxr_xr_dragging

Sometimes it does appear but it's all white.

And when it appears and it shows the controls, changing then doesn't affect the app anymore.

Seems like something got pretty unstable... I fear it may be a collision of multiple instances of three.js in memory...?

Expose the emulator to make it usable as an API, straight from the browser page

Clicking on button is nice but for testing not ideal. As mentioned in MozillaReality/WebXR-emulator-extension#296 being able to make calls directly from the browser console allows for testing programmatically, that means tests could be provided with the WebXR page itself and make the development cycle more efficient.

Bug: Hands are enabled on sessions with no hand_tracking

On this demo https://de-panther.github.io/WebXR-Unity-Samples/VR/WebXR-Quick-VR/Build/
The hand_tracking feature is not requested, but they are displayed when using the emulator in hands mode.

Conflicts with older WebXR extension in Chrome

In Chrome (at least Chrome on Windows latest version) you need to deactivate the older extension or it will consume all the events and the app will not respond to headset movements in the new extension.

Feature Request: AR Mode

Hello!

This tool looks awesome! When I'm playing around with it, I see that it supports the WebXR "VR" features, but it doesn't have support for AR. Would it be possible to enable this? Maybe by providing a default "pass through" image of the user's surroundings? (As I say this, I'm thinking that there would probably need to be spatial anchors, "plane detectors", etc. which would make this significantly more complicated, especially to be as customizable as the rest of the browser extension appears to be.)

Would be nice to not have "AR NOT FOUND"

I can add a mesh with no width, height, depth.

And those can't be selected / removed unless I clear all the settings. The object label renders in the emulator view but it's not selectable

XR Planes for user defined meshes

Meta browser returns a plane for the user defined meshes (e.g a table). The emulator only returns XRMeshes.

Firefox Support

I've been a long time user of Firefox, and for a while I was able to load this extension as a temporary add-on (I assume this was before aligning it with Manifest V3). As someone with no experience in WebExtensions development, what exactly is the primary issue preventing this from working on Firefox now? From a quick skim of resources Mozilla seems to declare that they're supporting V3, but maybe you can shed some more light on the subject.

add ability to toggle XRMeshes visibility

I noticed that the emulator always renders the XRMeshes in the app viewport. It would be nice to have a way to hide them so they don't interfere with the application scene.

Bug: Errors in WebXR Samples - Controller State

When using the Controller State sample
https://immersive-web.github.io/webxr-samples/controller-state.html
First there's an error

motion-controllers.module.js:120 Uncaught (in promise) Error: No matching handedness, , in profile generic-hand
    at fetchProfile (motion-controllers.module.js:120:13)
fetchProfile @ motion-controllers.module.js:120
Promise.then (async)
(anonymous) @ input-renderer.js:248
dispatchEvent @ webxr-polyfill.js:2958
_checkInputSourcesChange @ webxr-polyfill.js:3391
XRSession$1.<computed>.onDeviceFrame @ webxr-polyfill.js:3161
requestAnimationFrame (async)
requestAnimationFrame @ webxr-polyfill.js:27152
XRSession$1.<computed>.startDeviceFrameLoop @ webxr-polyfill.js:3178
XRSession$1 @ webxr-polyfill.js:3252
requestSession @ webxr-polyfill.js:4461
await in requestSession (async)
CustomWebXRPolyfill.XRSystem.requestSession @ webxr-polyfill.js:32464
initXR @ controller-state.html:254
(anonymous) @ controller-state.html:364

And when entering the XR session, the hands poses are tracked, but not the joints or pinch input.

Capture mouse/keyboard input from website viewport to control emulated headset and controllers.

Would be very useful for testing to be able to control the application directly with mouse and keyboard. Both the headset and controllers. Probably disabled by default so it doesn't conflict with the application inadvertently surprising the dev.

It's hard to perform multiple quick press on the emulator

Since the press on the emulator requires the "value" bar to reach the end and back, is hard to perform quick clicks, which I use in my games and are easy to perform with the actual controller.

3673.mp4

support for bHaptics tact gloves

Hi @RaananW,

Can I add support to the Oculus web browser to enable it to detect my haptic gloves? I am using tact gloves to get haptic feedback upon touching any object in the VR scene.

Not Enough Vertical Space

I go to my VR website, click on open vr and it gives this message not enough vertical space.
Tried to change the window size but didnt work.
No clue how to make this work
Chrome version 111.0.5563.148

Support for transient-pointer

https://webkit.org/blog/15162/introducing-natural-input-for-webxr-in-apple-vision-pro/

Emulator not working with three.js and A-Frame samples

Hi, unfortunately the emulator is not working for me, with either three.js or A-Frame samples (pls see attached screenshots, tested in Edge). It does work with the babylon.js samples and the immersive-web samples. Have I done smth wrong, with the the setup, or any ideas what might be the problem here?

best regards :)

Add Quest3 controller support