godotvr/godot_openxr_vendors

Godot 4 wrapper for OpenXR vendors loaders and extensions

godot_openxr_vendors's Issues

Khronos Library on Vive Focus 3

Hi. Good news is that I've been able to confirm that the new Khronos library does work on the Vive Focus 3, but only after adding the following line to the android manifest and only with the renderer set to gl_compatibility.

<category android:name="com.htc.intent.category.VRAPP" />

Without the manifest line, the headset starts the application in 2D and upside down.
I added the line under the current one for Oculus in the template and it works.

I've noticed a few issues with the current XR Tools Demo such as the skybox not rendering in 3D and the gravity demo going mental, but I'll raise those issues under that project.

Great work getting it going though!
Thanks

Merge GDExtension into a single implementation

This has been discussed before but I'm raising it here so we can document the discussion and come to a consensus.

Right now we're creating a GDExtension per vendor. The problem with this approach is that any API that is adopted by multiple vendors, needs to be duplicated. From a deployment POV this is not a big issue, but from an IDE POV, where we need to register each extension class in order to actually develop against these extension classes, we'll end up registering the same class multiple times.

Combining them into a single GDExtension library that is deployed to all platforms also ensures that vendors that adopt other vendor APIs in the future automatically gain support from Godot.

Finally this also opens the door to including the vendor GDExtension for non Android platforms as also PCVR runtimes may implement various vendor extensions, including those of other vendors.

Unsupported extensions are already filtered out by OpenXR checking for available extensions, and while each Android platform registers a new export plugin, these are uniquely named.

This does mean rejigging our folder structure so the GDExtension source is separately compiled, but the same Android binaries can then be included in the AAR for each Android platform.

Only first permission in `GodotOpenXRMeta.onMainCreate()` is actually requested

I'm experiencing an issue where the com.oculus.permission.USE_SCENE is never actually requested, because only the first permission request actually goes through.

So, right now we have this in GodotOpenXRMeta.onMainCreate():

        // Request the eye tracking permission if it's included in the manifest
        if (PermissionsUtil.hasManifestPermission(activity, EYE_TRACKING_PERMISSION)) {
            Log.d(TAG, "Requesting permission '${EYE_TRACKING_PERMISSION}'")
            PermissionsUtil.requestPermission(EYE_TRACKING_PERMISSION, activity)
        }
        // Request the face tracking permission if it's included in the manifest
        if (PermissionsUtil.hasManifestPermission(activity, FACE_TRACKING_PERMISSION)) {
            Log.d(TAG, "Requesting permission '${FACE_TRACKING_PERMISSION}'")
            PermissionsUtil.requestPermission(FACE_TRACKING_PERMISSION, activity)
        }
        // Request the scene API permission if it's included in the manifest
        if (PermissionsUtil.hasManifestPermission(activity, SCENE_PERMISSION)) {
            Log.d(TAG, "Requesting permission '${SCENE_PERMISSION}'")
            PermissionsUtil.requestPermission(SCENE_PERMISSION, activity)
        }

But what I see in adb logcat is this message:

03-07 11:17:50.127 18997 18997 D GodotOpenXRMeta: Requesting permission 'com.oculus.permission.USE_SCENE'
03-07 11:17:50.128 18997 18997 W Activity: Can request only one set of permissions at a time

However, if I remove all the permission requests above, except for the SCENE_PERMISSION that I care about, then the permission actually gets requested, and in the headset the window pops up giving me the option to grant permission.

@m4gr3d How best to solve this? Is there something we can do here? Or, is this best resolved in Godot, where we can update PermissionsUtil?

It would be kind of nice if PermissionsUtil had a function that accepted a list of permissions, and then tried each one in order, checking if they are in the manifest first, and then waiting until after we get a result for each one before moving on to the next.

PICO 4 Hand tracking and Passthrough

Hi. How to get passthrough and hand tracking working on pico 4? I only see meta support in the settings

There is no option to enable the plugin

I am missing the option to enable the plugin, so the Action Map is unable to open

Add PICO loader

We need to package the PICO loader in order to support PICO devices.

Combine all loaders in a single repo?

One of the suggestions made was to combine all the loaders in one repo? That would mean renaming this repo to godot_openxr_loaders and altering the scripts so we build multiple AARs, one for meta, one for khronos, potentially one for PICO?

Then this can be uploaded as a loader pack to the asset library and users can just enable/disable them.

godotlib...

Right now I've included a copy of the godotlib aar from the Godot 4 beta. I'm wondering if it makes more sense to just wget this file in the CI script from the official Godot site (i.e. https://downloads.tuxfamily.org/godotengine/4.0/beta1/)

When eye gaze is not enabled, we get errors when action map contains eye gaze interaction

The action map logic in Godot is build in such a way that all interactions can be added to the action map provided the entries appear in the meta data. The action map logic will then filter out unsupported bindings on load.

The problem with our eye gaze extension is that when it's not enabled, we don't instantiate the class at all, leading to errors when we're trying to load action maps that contain the eye gaze interaction:

modules/openxr/editor/openxr_interaction_profile_editor.cpp:260 - Parameter "profile_def" is null.

In situations where extensions introduce meta data for the action map system, we should probably make an exception and instance the class, but skip trying to activate the extension.

Manifest merger failed building for Quest 2 with 1.0.0 Alpha 2

I'm trying to export the godot-xr-tools demo project (4.0-dev) for Quest native. If I use 1.0.0 Alpha 1 the Gradle build works fine.

If instead I use the 1.0.0 Alpha 2 build, it fails with the following error:

Execution failed for task ':processReleaseMainManifest'.
> Manifest merger failed : Attribute meta-data#com.oculus.supportedDevices@value value=(all) from AndroidManifest.xml:55:13-32
  	is also present at [godotopenxrmeta-debug.aar] AndroidManifest.xml:20:13-50 value=(quest|quest2|questpro).
  	Suggestion: add 'tools:replace="android:value"' to <meta-data> element at AndroidManifest.xml:53:9-55:35 to override.

Add documentation for the Eye tracking feature

Add documentation for how to use the Eye tracking feature introduced in Godot 4.2.

See godotengine/godot#82614 and #48

Warning if multiple loaders are selected

Currently the interface allows multiple loaders being selected, we should add a check for this and warn the user this will give undefined results:

openxr_fb_spatial_anchor_tracked custom_data bug

var created_spatial_entity: OpenXRFbSpatialEntity = null

# Called manually on newly created scenes
#    var anchor_visualizer: Node3D = anchor_visualization.instantiate()
#    get_node("/root").add_child(anchor_visualizer)
func create_anchor(node: Node3D):
	spatial_anchor_manager.create_anchor(node.transform)

# openxr_fb_spatial_anchor_tracked but named differently
func _on_spatial_anchor_tracked(anchor_node: XRAnchor3D, spatial_entity: OpenXRFbSpatialEntity) -> void:
	if not spatial_entity.custom_data.has("is_saved"):
		spatial_entity.custom_data.get_or_add("is_saved", false)

	created_spatial_entity = spatial_entity
	print("Anchor tracked ", spatial_entity.uuid)

# Called on button press
func save_last_created_anchor(location: OpenXRFbSpatialEntity.StorageLocation):
	if created_spatial_entity.custom_data.get("is_saved") == false:
		created_spatial_entity.openxr_fb_spatial_entity_saved.connect(self._on_saved_anchor.bind(created_spatial_entity))
		created_spatial_entity.save_to_storage(location)
	else:
		print("Already saved anchor")

When I create 1 anchor and press the save button it works as expected:

Local anchor: 33528d6b-3ee4-ca2b-90af-206778fa3796 is success: true
Already saved anchor

But when I create more anchors and press save after every anchor spawn I get this

Already saved anchor
Already saved anchor
Already saved anchor

First placed anchor that got tracked has no custom_data, which is as expected. So we add the data and set it as saved

Now when I spawn a new anchor (On a newly instantiated node) It somehow has the custom_data already

var anchor_visualizer: Node3D = anchor_visualization.instantiate()
get_node("/root").add_child(anchor_visualizer)

Does anyone know what's going on here? I must admit I am new to Godot and GDScript. I first though that maybe my created_spatial_entity was somehow the same, but the ObjectID's are completely different.

Add Khronos loader

We need to package the official Khronos loader in order to support devices using the Khronos loader

Document the process to create a Godot XR plugin

Depth API

Hi,

Are there any plans on implementing a depth API for meta devices?

I would love to contribute and help but I have never worked on open source before.
(I do have a very solid knowledge of C++)

I expected to find the Meta Native SDK somewhere in the repository but I couldn't find it.

Regards,

Editor crash when closed

After upgrading from 2.0.1 to 2.0.2 on godot 4.2, the editor crashes when it is closed.

Godot(48212,0x7ff860710640) malloc: *** error for object 0x7feaf700d588: pointer being freed was not allocated
Godot(48212,0x7ff860710640) malloc: *** set a breakpoint in malloc_error_break to debug

I'm using macOS 13.4 on an intel macbook pro

Creating mesh crashes with MeshDataTool

Hello,

I'm trying to use this plugin to slice meshes in VR: https://github.com/PiCode9560/Godot-4-Concave-Mesh-Slicer
This works fine on 2d screen, but in VR it breaks.

In logcat I found this:

12-07 21:21:49.449  9978 16260 E godot   : USER ERROR: Index r[i] = 48896 is out of bounds (vcount = 24).
12-07 21:21:49.449  9978 16260 E godot   :    at: create_from_surface (scene/resources/mesh_data_tool.cpp:73)

To reproduce with minimal code:

var mdt = MeshDataTool.new() 
var mesh: Mesh = rigid_body.mesh
var new_array_mesh = ArrayMesh.new()
new_array_mesh.add_surface_from_arrays(Mesh.PRIMITIVE_TRIANGLES, mesh.surface_get_arrays(0))
mdt.create_from_surface(new_array_mesh , 0)

The rigid body is just a node with a meshinstace 3d with a box shape.

I'm using Godot 4.2 and a Pico 4 VR headset.

Any help would be appricated.

Regards

grip_touch controller signal not happening on Quest2 and Pico4

The XRPositionalTracker.button_pressed signal never emits a "grip_touch" touch event on the Quest2 or Pico4.

It's detecting all the other button touches in clicks in my benchmark example.

See the picture where I'm holding the trigger and grip buttons fully in (hence the input_float_changed is registering 1 on both of them).

My working demo is at:
https://github.com/goatchurchprime/godot-demo-projects/tree/openxr_seeallfeatures/xr/OpenXRHandTrackingDemo

Custom build -> Gradle build

Just a heads up as this changes the instructions somewhat and I haven't had time to update the docs yet.

This PR was merged into Godot 4 RC1: godotengine/godot#72552

This renames "Android custom build" to "Android gradle build".

Project export has no vendor checkboxes

According to the documentation, you're supposed to have checkboxes to pick which OpenXR vendor to build for. However, on both Windows and macOS, these checkboxes are not present after installing the plugin.

I've tried restarting and reinstalling the editor, using a different project, making various export presets, and toggling between "ignore asset root" during plugin installation.

However, none of these work. Could you please help me figure out why these checkboxes aren't present?

Successful Builds

Building to the Meta Quest Pro anyways results in a normal Android app that complains about not having an OpenXR environment, then crashes...

Replace the use of `get_next_frame_time` with `get_predicted_display_time`

See godotengine/godot#89734 (comment)

GDExtension library not found: `meta/plugin.gdextension`

Using Godot 4.2 stable downloaded today.
Using the release copy of this plugin here: https://github.com/GodotVR/godot_openxr_vendors/releases/tag/2.0.1-stable
I've also tried with 2.0.0.

I get this error on startup with the plugin enabled:

  core/extension/gdextension.cpp:688 - GDExtension library not found: 
  Failed loading resource: res://addons/godotopenxrvendors/meta/plugin.gdextension. Make sure resources have been imported by opening the project in the editor at least once.

(the project has been opened at least once)

Inspecting the file, I see this:

[libraries]

android.debug.arm64 = "res://addons/godotopenxrvendors/meta/.bin/debug/arm64-v8a/libgodotopenxrmeta.so"
android.release.arm64 = "res://addons/godotopenxrvendors/meta/.bin/release/arm64-v8a/libgodotopenxrmeta.so"
macos.debug = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.macos.template_debug.framework"
macos.release = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.macos.template_release.framework"
windows.debug.x86_64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.windows.template_debug.x86_64.dll"
windows.release.x86_64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.windows.template_release.x86_64.dll"
linux.debug.x86_64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.linux.template_debug.x86_64.so"
linux.release.x86_64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.linux.template_release.x86_64.so"
linux.debug.arm64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.linux.template_debug.arm64.so"
linux.release.arm64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.linux.template_release.arm64.so"
linux.debug.rv64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.linux.template_debug.rv64.so"
linux.release.rv64 = "res://addons/godotopenxrvendors/meta/.bin/libgodotopenxrmeta.linux.template_release.rv64.so"

Inspecting the res://addons/godotopenxrvendors/meta/.bin/ path however in the release ZIP file, I see:

• debug
  • godotopenxrmeta-debug.aar
• release
  • godotopenxrmeta-release.aar

More of a traceback:

Godot Engine v4.2.stable.official.46dc27791 - https://godotengine.org
OpenGL API 3.3.0 NVIDIA 546.17 - Compatibility - Using Device: NVIDIA - NVIDIA GeForce GTX 1070 Ti

ERROR: Parameter "profile_def" is null.
   at: _update_interaction_profile (modules/openxr/editor/openxr_interaction_profile_editor.cpp:260)
ERROR: GDExtension library not found:
   at: open_library (core/extension/gdextension.cpp:688)
ERROR: Failed loading resource: res://addons/godotopenxrvendors/meta/plugin.gdextension. Make sure resources have been imported by opening the project in the editor at least once.
   at: (core/io/resource_loader.cpp:274)
ERROR: GDExtension library not found:
   at: open_library (core/extension/gdextension.cpp:688)
ERROR: Failed loading resource: res://addons/godotopenxrvendors/meta/plugin.gdextension. Make sure resources have been imported by opening the project in the editor at least once.
   at: (core/io/resource_loader.cpp:274)
ERROR: GDExtension library not found:
   at: open_library (core/extension/gdextension.cpp:688)
ERROR: Failed loading resource: res://addons/godotopenxrvendors/meta/plugin.gdextension. Make sure resources have been imported by opening the project in the editor at least once.
   at: (core/io/resource_loader.cpp:274)

Support for Lynx R1

The Lynx is beginning to ship to kickstarter backers and developers.

The sdk is currently only a 3 month old unity project but there was statements that godot was one they wanted to support.

https://portal.lynx-r.com/support/category/openxr

Currently these OpenXR extensions are listed -

XR_EXT_hand_tracking: Version=4
XR_KHR_android_create_instance: Version=3
XR_KHR_opengl_es_enable: Version=8
XR_ULTRALEAP_hand_tracking_forearm: Version=1

with support for

XR_KHR_vulkan_enable2: Version=2

still buggy in their Unity sdk.

Add descriptions to tickboxes

Godot has the ability to add descriptions in the popup for tickboxes, we should make use of this to clarify what each settings is for. May require upstream changes.

Class registration issues

Probably pretty harmless, and at least one should be resolved by recent pending changes from Malcolm, but we get some error spam at start up:

Cannot get class 'OpenXRFbSceneCaptureExtensionWrapper'.
Cannot get class 'OpenXRFbSpatialEntityExtensionWrapper'.
Cannot get class 'OpenXRFbSpatialEntityQueryExtensionWrapper'.
Cannot get class 'OpenXRFbSpatialEntityContainerExtensionWrapper'.
Cannot get class 'OpenXRFbSceneExtensionWrapper'.
Cannot get class 'OpenXRFbFaceTrackingExtensionWrapper'.

On a broader spectrum we should discuss about whether extensions should be exposed 1:1 as singletons, or whether we should always add nodes/resources that users can use that expose the functionality. I'm a fan of the later option and do not believe the wrappers should be exposed in this way.

Licenses

While ultimately the responsibility of those deploying games with Godot, we need to look into including the correct license files for the bits that we include in this repo.

godot 4 beta 8 and pico 4 not working anymore

hi,
when i copy GodotOpenXRPico.gdap into my projects android/plugins/ folder it returns

modules/gdscript/gdscript_analyzer.cpp:670 - Parser bug (please report): tried to assign unset node without an identifier.

I have no option to enable "godot openxr pico" anymore in Project->Export view ?

when i build the apk and run it on device , it shows a black screen in "not VR-mode" .

Add keyboard models to XR_FB_render_model extension wrapper

#76 Adds in an XR_FB_render_model extension wrapper and corresponding OpenXRFBRenderModel node. There are two controller models and three keyboard models, but only the controller models are supported at this time (as tested on Quest 2 and 3).

To support at least the virtual keyboard model, the XR_META_virtual_keybaord extension will need a wrapper implemented.

Once keyboard models are supported, be sure to add their paths as suggestions to the get_suggested_render_model_path_names() function in openxr_fb_render_model.cpp.

Update PICO Loader to 2.2.0

There is a new version of the loader for PICO,

The APK in PICO not working

I follow the instructions in the video:
https://www.youtube.com/watch?v=wDXnsy2IH1A
with Godot 4.0 Beta v16 and PICO 4 VR glasses.

It works with SteamVR and PICO Link (Streaming assistant), but when I export the project to Android (always following the instructions of the video it doesn't work.

After a few seconds loading the PICO glasses show a image with ground and "sky" and a second before closes.

If I do a Remote Debug Godot shows the message ("platform/windows/display_server_windows.cpp:1008 - Condition "!windows.has(p_window)" is true. Returning: Point2i()") 44 times or more.

Statically link libstdc++ on Linux for portability

This is a portability issue identified first in the webrtc libraries (since fixed) godotengine/webrtc-native#110

I think all it needs are some link flags in the SConstruct file
godotengine/webrtc-native@1de32aa

Contents of the ZIP file

So there is an important difference between the zip file created for a PR as an artifact and the zip file created as part of a release.

The zip file on the artifact will contain the following entries:

• android/plugins/GodotOpenXRLoaders_CHANGES.md
• android/plugins/godotopenxrmeta.gdap
• android/plugins/godotopenxrmeta/license.txt
• android/plugins/godotopenxrmeta/godotopenxrmeta-release.aar
• android/plugins/godotopenxrmeta/godotopenxrmeta-debug.aar

This is what you would expect to see.

The zip file on the release however will contain the following entries:

• asset/android/plugins/GodotOpenXRLoaders_CHANGES.md
• asset/android/plugins/godotopenxrmeta.gdap
• asset/android/plugins/godotopenxrmeta/license.txt
• asset/android/plugins/godotopenxrmeta/godotopenxrmeta-release.aar
• asset/android/plugins/godotopenxrmeta/godotopenxrmeta-debug.aar

A root folder is required by the Godot Asset Library, the contents of this root folder will be extracted into the Godot project resulting in the correct file structure.

Metalic materials don't reflect the skybox

Here's a section from the Quest2 where I have a panorama sky set from an image with metalic=0.96 roughness=0 (running mobile renderer).

This is the same exact code running as deployed on my Android phone

So it must be a mobile-VR problem, not a mobile Android problem

(Same problem of no skybox reflection on the Pico4 as well.)

Hand tracking on the Pico 4 seems to be broken

I believe that the hand tracking on pico 4 has a bug or two, or possibly I'm using OpenXR the wrong way
I'm using:

• Godot 4.3 beta 1
• Godot XR tools for Godot 4 (4.3.1) (for the hands and there skeletons)
• Godot OpenXR Vendors plugin v3.0.0 beta 1

I have tied both the depreciated OpenXRHand and XRNode3D + XRHandModifier3D

1. With both when I hold my hands horizontal the hands point up inside VR
2. Only the OpenXRHand modifies the bones, the XRHandModifier3D does nothing
3. The OpenXRHand with bone update = "rotation only" bends my fingers in the direction of my pinky finger
4. The Pico 4 insists that the program doesn't support running with hand tracking

In the export settings I have tired bot optional and required for hand tracking

In the project settings i have enabled hand tracking and tried both with and without "Hand Interaction Profile" enabled

Some of the rotation and bending issues might be solvable by applying a rotation matrix to the data from the Pico 4 or perhaps even just swapping an axis or two.
It may of cause simply be a bug in the Pico firmware, but it would be nice if it could be worked around until they fix it (if ever).

As to why the XRHandModifier3D doesn't modify the bones I wouldn't rule out a misconfiguration on my part

Oh and last but not least: Thanks for the plugin quite amazing how relatively easy it is to code VR applications using it

XR_LOADER_DEBUG=all support on godot android

Exposing XR_LOADER_DEBUG might be useful for android XR debugging but android has restrictions on env use (each process has its own freestanding env).

https://registry.khronos.org/OpenXR/specs/1.0/loader.html#loader-debugging

I think the cleanest way to do that would be to modify the gradle build target something like

buildConfigField "String", "XR_LOADER_DEBUG", "\"${System.env.GODOT_ANDROID_XR_LOADER_DEBUG}\""

Then add GODOT_ANDROID_XR_LOADER_DEBUG to the build environment.

This could be checked in godotscript by

https://docs.godotengine.org/en/stable/classes/class_os.html#class-os-method-get-environment

*Warning this is untested as I don't have an android XR device.

Change package name for Meta loader

I'd prefer to keep loaders in a shared package namespaces associated with the Godot game engine. As far as I can see, godot dot com does not belong to the project. If that is correct, could we change it to something within godotengine.org?

In #13 I proposed org.godotengine.openxrloaders.pico for Pico, which would lead to org.godotengine.openxrloaders.meta. That said, I'm open to other naming conventions.

Crashes on Quest 2 Godot 4.2 Mono

In Godot 4.2 release candidate 1 mono and Godot 4.2 beta 5 mono

TLDR: the game initially launches like a 2d app, then the game goes into vr mode for 1-2 frames. finally the takes the user back to the quest's home with a bugged ui

The app crashes with the plugin's Quest option enabled in the export menu
When launching the app a window appears like the game is a 2d application, and then after a second the window disappears. Sometimes you can notice one frame flash on the headset and it looks like the game actually started in vr, but then the headset goes back to the home environment, but the ui is bugged. The menu isnt there and the menu buttons dont work.

Without the quest option checked
the app launched with a window like its trying to run a normal 2d application, and theres an error about openxr failing to start.

The reason i am using godot 4.2 is because the previous versions of godot 4 do not support exporting to android with godot mono and I am planning on using c# for my code