Inside Out Positional Head Tracking (standalone 6DoF) for GearVR/Cardboard/Daydream using ARCore v1.6.0
ARCore v1.6.0 enabled Inside Out Positional Tracking (six degrees of freedom) for all ARCore capable devices.
-
WARNING YOU MIGHT GET SICK: The current versions use interpolation and smoothing functions to cover up imprecise tracking. This leads to relative high latency in response to head motion. This is bad for people who get sick from motion sickness quickly. If you know that you are susceptible to motion sickness, these apps might not be for you jet.
-
Fast movements, featureless, extrem sunny and poorly lit areas can affect the quality of tracking severely. When the app starts up it takes 3-5 seconds for ARCore to detect a plain at this stage it is best to move just a little.
-
Your phone might get hot very quickly. Make sure to end the game after using. The Cardboard version seems to stay on even if the phone gets very hot while the GearVR version turns itself off automatically. Therefore, cardboard users should make sure to manually end the app after use.
-
The goal of this project is to bring high quality (6DoF) positional tracking to GearVR, Daydream and Cardboard using ARCore/ARKit software. The HTC Vive Focus and the Daydream powered Lenovo Mirage Solo offer this functionality already but are very pricy and not available everywhere yet. Because they use dedicated hardware, they can offer better performance and quality tracking. The Oculus Santa Cruz/Quest currently offers the best 6DoF head and controller tracking which makes it a highly desirable device.
-
Before installing one of the apps (.apk files) make sure you have installed Google ARCore from the Play Store on your device. Check if your device supports ARCore on this page.
-
LATEST NEWS: The new ARCore version (v1.11.0) now called "Google Play Services for AR" promises to support 60fps tracking which would be great for a smoother head tracking. The previous ARCore versions which are still used in this project use the 30fps tracking which in reality are not even real 30fps since there are a lot of tracking errors even with perfect lighting and environment condition. My first test showed that older devices like the S7 can not yet make use of the new 60fps feature and Google lists only the Pixel 2 and Pixel 3 as 60fps compatible on the supported device page. Also, the new ARCore supports build in depth cameras which might be interesting for the S10 5G and should improve tracking even more.
Projects and Apps (.apk) | Description | Recommended Devices | VR Headset |
---|---|---|---|
WhiteIsland.apk | Simple environment. Using nativeARCore for tracking. Noticeable latency due to the used smoothing function. Use the controller touchpad to fly forward in viewing direction and the trigger button to enable/disable visualization of real-world boundaries. | All ARcore + GearVR compatible devices | GearVR |
BoxyRoomCardboard.apk | App should run on every smartphone with ARCore support. Daydream users need to uncover the camera e.g. as explained here and might need to turn of NFC to avoid Daydream from starting on top of the Cardboard app. | All ARCore compatible devices | Cardboard |
VikingVillageGearVR.apk | Interesting because of the high-quality environment which was captured with the Seurat tool. This tool allows for the capture of a (small) area of a high-quality environment which allows for a limited free movement only in this captured area. | All ARCore + GearVR compatible devices | GearVR |
VikingVillageCardboard.apk | Interesting because of the high-quality environment which was captured with the Seurat tool. This tool allows for the capture of a (small) area of a high-quality environments which allows for a limited free movement only in this captured area. | All ARCore compatible devices | Cardboard |
VikingVillageForOculusGO.apk | The Oculus GO has no camera so there is no head tracking possible, but because the high-quality environment scene captured with the Seurat tool could be interesting for Go users as well, I added controller support so that flying through the scene by pressing the touchpad is possible. | OculusGo | OculusGo |
VuforiaTracking(Experimental) | This app uses Vuforia which has an ARCore integration for tracking. But the performance of the ARCore integration is very poor compared to native ARCore. Therfore this project will be discontinued. | All GearVR capable devices (low tracking quality) | GearVR |
NativeARCoreTracking(Experimental) | Is not up to date and will be removed soon. Please use WhiteIsland instead. | All ARCore + GearVR capable devices | GearVR |
MovingMarkerTracking.apk | Uses Vuforia Vusion which combines ARCore and Vuforias marker tracking. This combination allows for 6DOF marker tracking in a limited area in front of the camera. The marker can be found here. Simply open the marker on your pc screen or print it on paper and look at it through the camera when the app is running. Since ARCore 1.9 now supports moving marker tracking this project will likely be replaced soon since Vuforia Marker tracking has many issues. | All GearVR capable devices (low tracking quality) | GearVR |
Works with all phones who support ARCore. Just download the BoxyRoomCardboard.apk or the VikingVillageCardboard.apk (Seurat) and install.
Video showing BoxyRoomCardboard.apk in action.
You need to restart the app when misalignment between the headset and the head movement is observed. The misaligning is a bug which was solved for the GearVR version but is still present in the Cardboard/Daydream versions.
Cardboard VR apps lag the time warp functionalities that we can use in GearVR and Daydream apps. The Cardboard headset is not very good either if you can use a GearVR or Daydream headset to get a noticeably improved experience.
Installing on GearVR requires the app to be signed first. But since the current GearVR version is better than the Daydream/Cardboard version the few extra steps to sign the apk file are worth it for you.
Signing the .apk file:
First. Download the WhiteIsland.apk. or the VikingVillage.apk.
Second. Sign the apk using the free Injector for GearVR app. Just safe the .apk file on your phone, use the Injector app to sign it and install. Make sure the app you want to sign and install is not already installed on your phone because in my case this prevents the app from installing again.
If the Injector app doesn't work for you try this alternative: Find out your device ID here. With the .apk file and the device ID you can sign the apk yourself. How you can sign the apk is explained in this Youtube video. I recommend the option where you download the addosig.bat program to sign the apk. You can either follow the link under the Youtube page or download the Add OSIG.zip file containing the addosig.bat program directly from this repository.
Video showing WhiteIsland.apk 6DoF GearVR version.
Video showing VikingVillage (Seurat) 6DoF GearVR version.
Click the touchpad on the side of the headset once to recenter the view when you notice misalignment.
Daydream users: You can either use one of the Cardboard apks or even better switch the Cardboard project over to Daydream inside the Unity project yourself. Use for example the "BoxyRoomChardboard" project and under build settings, XR settings replace Cardboard with Daydream. I will upload a version specifically build for Daydream users as soon as I get access to a Daydream device and headset to test it.
iPhone user: You should look at this project first for a quick solution. Unfortunately only Cardboard VR, which lags some critical features like reprojection runs on iphones making iphones not well suited for mobile VR. However most iPhones support ARCore so you can try to download the cardboard project, switch it to IOS and install it on your device if you interested in this project. If I find the time, I will test and upload an iphone cardboard version as well.
The current tracking version you find in this repository suffers from lag, jumps in position, drift from tracking being lost and other inconveniences. Fixing these issues seems very doable, and I am currently working on that most of the time. The most promising solution seems to use a filter which detects tracking errors and removes them. For filtering errors the kalman filter (video lecture here) and block post here looks like a good start and an Unity implementation can also be found here. I don't think we reach the same quality as dual camera setups in extreme situations but I predict a future version could offer almost perfect head tracking when just performing normal head movements. Since ARCore and phones get improved all the time, the tracking should constantly get better.
Hand tracking using software from Manomotion and uSenseAR (video) is available and works with VR and ARCore. Unfortunately, the performance requirements are very high. First tests with Manomotion where not so successful but I will follow the development closely. One disadvantage is that the Manomotion software is not free of charge. Moving markers attached to the hand looks like a worthy alternative since it is included for free in the ARCore software and also works more reliable and with better performance compared to the hand detection.
Video showing Manomotion hand tracking in combination with ARCore.
Google recently released an apparently extremely fast and robust monocamera hand tracking solution that is open source and can be found here.
Robust and quick mono camera hand tracking developed by Google.
Another open-source project for hand gesture recognition on ARKit can be found here.
ARCore version 1.9 now supports moving marker tracking. Read more about it here. The MovingMarkerTracking app you find in this project is currently using an older version based on Vuforia marker tracking. I am now looking into ARCore based moving marker tracking which should be a much better solution. The interesting questions moving marker tracking opens up are:
-
Can we use it to further improve head tracking?
-
Can we add a marker to your hands/elbows/feets/controller to get some sort of hand/body/controller tracking?
But even if it works the limited field of view of the camera will probably make it less useful for many applications.
The company ZapBox already sells kits based on this idea as you can see in this video. Notice that they use a clip-on-lens to achive a greater field of view for the see-through image and the area where the moving markers can be tracked also gets bigger. I tested multiple wide angle clip-on-lenses but non of them worked with ARCore either stoping tracking all together or drifting heavily. The same seems to be true for ARKit as you can read here. Since some of the latest smartphones like the S10 have a dedicated wide field of view camera we can only hope that the ARCore team adds support for these cameras as well.
Google clearly thought about solutions to 6DOF controller tracking as you can read in this roadToVR article. But since thay use stereo cameras the results may not be directly transferable to mobile phones with only a single camera.
Images from a paper published by Google describing markerless controller tracking.
Two 3D Scanner projects using Google Tango Open Constructor and the master thesis Tango-Poc are available as open source projects. I also recommend this informative video about Tango 3D scanning technology.
An ARCore version is also available (not open source) and a plugin for Unity can be bought here. It could be used to further improve the boundary system and automatically building a virtual environment which fits perfectly over your real environment. This is incredible, and I will try to implement something similar when the tracking works well. I recommend reading the related paper here and here (use scihub.tw to get free access). Also download the free 3D scanning app to check it out directly.
Another idea for the boundary system could be a kind of edge detection and visualisation algorithm as shown in this video.
Object detection/classification using the camera feet could allow for some unique game and application mechanics. Two projects using Google tensor flow as the underling machine learning software are available here and here.
360° and 180° photos and videos are nice but with additional depth information they become really interesting. Positional tracking on GearVR would be a natural fit since no hand controllers are needed, mobile phones can display high resolution images and since the head movements are rather slow we have no problems with camera motion blur and loss of tracking. There are three distinct ways of approaching this idea. 1.) Image overlayed on a depth map either by capturing the depth map separately or by using software to recover a point cloud from a stereoscopic video (seen here) and further discussed here. There is already an app available on the Oculus store for GearVR which displays images and videos with additional 3D depth information. 2.) Photogrametry allows for higly realistic but static 3D models. 3.) Using a light field capture of the environment seems to be the holy grail of 3D photography. A demo project (video) was published by Google for free but is only running on desktop pc VR headsets. Using the seurat pipeline and this github light field project it might be possible to bring the same experience to mobile VR. The same is true for bringing very detailed photogrametry models on the phone in VR. I am currently exploring those ideas for my first app for the Oculus store.
Forcing 6DoF onto old 3DoF apps would be an amazing feature if we could bring it to GearVR. This feature is available on the Lenovo Mirage Solo see here and now also on the Oculus Quest see here. I have absolutely no idea how this works since the apps where never intended to be used in 6DoF but I would love to get this working as well for ARCore enabled positional tracking. If you have an idea how this works please open an issue and let me know.
To further increase the reach, it makes sense to build apps that run on Android and iOS devices. The AR Foundation github project brings this functionality to Unity. Unfortunately the combination of VR and AR is not working yet I hope this changes in the future so we only need to build an app once.
Using VR outside or in a large play space is a lot of fun and feels very different from games experienced inside where the ability to move freely is limited. Here accessing 3D maps might be interesting and result in exiting apps consisting of AR and VR elements.
This iconic shooter game can now be played using the GearVR with the option of using positional tracking. A good introduction video can be found here. The game and source code can be downloaded here. After signing the .apk you can use the GearVR controller to activate the positional tracking mode in the game settings. I am very impressed with the performance and tracking quality even on old phones like the S7. The developer didn't use a game engine like Unity to build the game but instead used the Android Studio which seemed to allow him to get a higher performance out of the phone by putting some tasks on different cores. I like the way he managed to give the user a window to the outside by tapping the headset but I think it is better to show the surrounding automatically when you come closer to an obstacle since you might forget to check the surrounding when you immersed in the game (I will start working on this soon since I have an idea already). I think this is excellent work and you should check it out.
Such a project can be found here and here. A paper comparing ARCore and ARKit can be found here (use sci-hub.tw if you don't have access to the full text). Most important findings of the paper for this project are that ARKit tracks with 60fps, can deal with faster movements and plain detection is more accurate indoors. ARCore seems to be better outdoors. Also noticeable is the finding that the performance drops on some older Android phones like Nexus 6P and Nexus 5X. According to @lvonasek ARKit is more accurate because of a better camera calibration process by Apple.
Positional tracking on iPhone using ARKit + Cardboard.
They also experimented with ARCore and GearVR. They put some serious thought in solving many of the little issues but the code they use seems not to be public. Learn more here.
Fast Travel Games showing of simplefied Apex Construct running on GearVR.
Tango is only available on the Asus Zenfone AR and has better room tracking capabilities than ARCore thanks to the time of flight sensor build in. Useful links here, here and an excellent talk about the underlying technology here. It was also shown in this video that some form of hand tracking using the depth sensor is possible. The main issue with the now rather old devices is that the chip can't handle all the performance requirements and overheats rather quickly as was mentioned in this article.
ZapBox, HoloKit and Aryzon headsets.
Some projects (e.g. ZapBox) use the camera seethrough mode for AR projects but I don't think it is a good approach to use a mono camera and add 3D objects on top of the 2D camera image. This is guaranteed to give you a bad experience. The HoloKit project builds on a much better idea but would profit from a better headset design and some improvements we already see in the Daydream and GearVR headsets like low persistency screen mode and smooth head tracking. It would also profit from smoother positional head tracking which is the mission of this project.
This introductory video, the github project site and the HoloKit website can give you a good overview over the HoloKit project. You can also buy the headset from thair website but more on that down below.
There is also a company called Aryzon which is selling almost the same product with thair own SDK which is also open source. A good review of this headset can be found here and here. I only found an older video showing a view through the headset with the tracking looking terrible but tracking should be much better with ARCore.
There are many possibilities that such a headset would allow for some examples here (markers), here (hand tracking) and here.
Be careful when buying a no name headset they are very likely not compatible with the HoloKit and Aryzon SDK's since distance between the lenses and phone as well as the mirror and lense quality will make a big difference. The lenses on the original HoloKit (video) and Aryzon video headset look much more advanced compared to the ones on the cheaper Chinese versions (video 1 and video 2).
Despite the interesting use cases, I can not give an unconditional recommendation to buy the HoloKit or Aryzon headset right know mainly because the software is not as advanced as we used to from the GearVR/Daydream tracking which in my opinion is the minimal standard we need to have an enjoyable experience.
Improvements to the ergonomic would also be required. A different headset design which was already planed and promised for the end of 2018 by the HoloKit team is still not available and I would suggests the software flaws need to be fixed first. Improving the HoloKit idea further seems to be an interesting project and I am currently testing the Aryzon headset to see if it can be made better with the right software tweaks.
ZapBox shows of some interesting ideas like a lens to increase the field of view and markers for the controller tracking.
Marker based 6DoF hand controller developed by ZapBox.
However, the ZapBox software needs a lot of improvement (video showing funny tracking errors here) and in my opinion lags far behind Googles ARCore tracking. The underling idea of creating a mixed reality headset with only one camera seems somewhat silly, and I can not recommend doing it when not also tackling the stereoscopic issue. Zapbox tries to improve the image by using depth informations from the pointcloud and bluring parts of the image as they state in this reddit post. The Phantom AR project also worked on the issue of generating a stereoscopic view from a monoscopic image see here for more info. You can try out the 2D to 3D conversion results yourself by watching this video and putting your phone inside a headset starting at timestamp 2:31.
To get a 3D view from a 2D image it is a good approach to first capture a depth map and use it together with the mono camera image to generate a 3D stereoscopic view. For this approach the upcoming depth sensors in the latest phones could be leveraged. There is also a lot of machine learning research in the area of inferring depth information from 2D video or directly generating 3D video output from 2D video input. Having a high quality, low latency, wide field of view 3D passthrough image on the GearVR would open the possibility for some interesting mixed reality applications on such a cheap headset and would also be beneficial to the boundary system so something similar to the solution found in the Oculus Quest where the real world is displayed to the user when he comes to close to objects could be implemented.
This combination seems be suited for bringing full body movement to the GearVR: Check out the related reddit post
This sweet open source software project connects the GearVR to the pc and uses head tracking for simulating expensive headsets. Check out the project if you haven't heard about it here. Might be interesting for the Oculus Quest too. For Oculus Quest users this should be interesting as well, see this video to learn more.
Using the computing power of the cloud to process data seems to be the path of the future. The Phantom AR project explores the possibilities of such a future. Learn more about it in this video
The Just a Line experimental project explored the possibilities of multiuser experiences between iPhone and Android phones. This project makes use of the cloud anchor capabilities of ARCore.
Owlchemy labs showed of the research project mobile spectator which allow a third person to use a mobile phone running ARCore to see and participate in a VR experience. This allows for interesting new experiences/games or could make VR more social.
A multiplayer VR game based just on the same starting point (seen here) would also be possible not requiring a connection to the cloud but some kind of marker or phone to phone communication to know the location of the other player. However in my opinion using cloud anchors is the right way to go since it relies on finding the same feature points in a scene to match the location.
Is an interesting idea and in this paper (use scihub.tw to get free access) a working version was developed.
Are you aware of further interesting projects, or you work on an interesting project yourself? Please don't hesitate to open a new issue and spread the word.
The Daydream and GearVR platforms are both loosing momentum with Daydream not beeing supported by the latest Pixel phones and GearVR no longer beeing supported by the upcomming Note 10. All this is understandable since new GearVR and Daydream headsets might be the cheapest way to get into VR when you already own a compatible phone but with ever more affordable standalone headsets like the Oculus GO and Quest which offer a simpler setup and often a better experience customer have a great alternative. Nonetheless I believe there are still many people who would like to see these platforms keep developing and there are so many VR capable phones and headsets out there that these platforms will continue to be used for quite some time.
Qualcomm already offers chips that have positional tracking in mind. Starting with the Snapdragon 835 in the Samsung S8 better stereo camera based tracking should be possible regarding to Qualcomm and you can read more about it here. Qualcomm mentions 4 key advantages of stereo over monocular 6-DoF:
• Instant accurate scene depth
• Faster initialization
• Better performance with quick and rotational motions
• Improved tolerance to camera occlusion
On top of the list but still somehow speculative since it is not clear if the featured hQVGA 3d depth/time of flight sensor will work in combination with the GearVR. When we get access to the depth sensing data low latency positional tracking similar to the Asus Zenfone AR could be possible. The ultrawide field of view of the camera as well as the depth sensor could be used for better hand/moving marker tracking. Compatibility with the GearVR should result in a great VR experience and the 5G feature might allow for streaming high quality data directly from the cloud allowing for experiences outperforming even desktop VR. Interestingly it seems that support for the Daydream platform was dropped for these phones. Also be aware that most of the underling software is still not even developed so don't expect these features to be available anytime soon.
Supporting Daydream which delivers a good enough VR experience and in combination with its highly accurate depth camera and the Google Tango software it allows for excellent positional tracking. The disadvantage is that Google stopped developing Tango, that almost no one owns the hardware and that no performance improvements are expected in the future making the development for this device and the Tango platform not very attractive. If you decide to purchase this device be aware of the overheating issue due to the rather old chipset.
They deliver better performance than the S7 which is important for running hand tracking and better looking games. In terms of image quality, field of view and tracking latency, they are identical to the S7. The ultrawide field of view camera of the S10 and S10+ could be interesting for improving hand and marker tracking in the future but this would depend on the tracking software (Manomotion or uSense) and ARCore if they support the wide field of view camera. Be aware that the S10 dose not support Daydream so if you want to use Daydream and GearVR on the same device you should get an S8 or S9 instead.
Runs ARCore and works with many GearVR headsets without immediately overheating. For hand-tracking the performance might be not enough but this is still an open research question. In combination with a cheap second hand GearVR for roughly 30$ it might be the best value device for experiencing VR right now. This software project was tested on the S7.
They are powerful, often also support ARCore and the VR quality with the second generation Daydream View is almost as good as GearVR. Daydream is a much more open platform compared to GearVR and runs on multiple phones which only need to meet the hardware requirements and the ok from Google or the device manufacturer to be supported. That said the S10 didn't seem to get Daydream support which is strange since it clearly has the capable hardware. It should be possible to root and force Daydream on the S10 or other ARCore ready phones but that trick comes which some risks and might not work for you in my case testing it on the S7 once it kind of worked but I had many issues making it unusable. If you are interested in how to force Daydream on your device have a look at this and this XDA developer post. Currently, this project is working better on GearVR since resetting the view on Daydream is somewhat more complicated. This might change in the future if the issue can be solved. Since the Pixel 3 is one of the few phones which now supports 60fps tracking it might become one of the best phones for mobile VR with positional tracking and this should be further investigated.
Most phones out there do not support GearVR or Daydream and if you are not willing to mess with the system, your only option is Google Cardboard which is noticeably worse because of two factors. The lense quality of Cardboard lenses and the inferior Cardboard software. This project offers a Cardboard version but I recommend when you have the option to use a GearVR or Daydream project instead. On some phones which support Daydream the Cardboard apps now also offer low persistency mode and great tracking performance which where introduces in a new Cardboard SDK (read more about that here) but older phones do not offer a good experience using the Cardboard software. If you are willing to mess around, you can try to force Daydream on the phone like explained in this post. The problem is that the Daydream requirements often have their justification (the S10 might be an exaction from the rule) and the experience might not be as good as on supported devices.
ARCore runs on many more devices compared to Daydream/GearVR and your phone might support it even when VR is not supported. When your phone is not an ARCore ready phone you can force it either with the help of this ARCore for everyone github project or by rooting your phone and using a special ROM which contains the ARCore feature. But the chances of getting good tracking when forcing ARCore on an unsupported device are not very good since the ARCore software seems to be specifically adjusted to the phone camera to work correctly. But you can give it a try to know for sure.
The OnePlus 7 Pro should also be mentioned here since it will offer a 90Hz refresh rate display which should make a noticeable difference in the visual quality. Currently, it is not clear if it will get ARCore support and if forcing Daydream onto it will work. So no recommendation yet but keep an eye on this interesting device.
An overview over all the commercially available mobile VR headsets can be found here.
If you are looking for a new phone which is well suited for VR and works in combination with this project I can recommend these 3 devices:
Amazon Store: USA | UK | CA | DE | FR | ___ Store: USA | UK | CA | DE | FR |
Amazon Store: USA | UK | CA | DE | FR |
If you own a Samsung Smartphone S7 or above it will support ARCore and I recommend the GearVR with a controller. Make sure to find the right GearVR headset for your phone the connector and size requirements might change depending on your phone. If you own a smartphone which only has Daydream support, I can also recommend the second generation of the Daydream View headsets which offers improved lenses compared to the first Daydream View generation. Booth GearVR and Daydream View 2 offer great 3DOF VR experiences and interesting experimental 6DOF and computer vision capabilities. I cannot recommend the Cardboard headset the quality is just not good enough.
Amazon Store: USA | UK | CA | DE | FR | __ Store: USA | UK | CA (not available)| DE | FR |
The Oculus Quest is an all in one device with better resolution, better head/controller tracking. But it has some disadvantages compared to the phone based solutions for example the tracking will not work outside when the sun is to bright (at dawn it seems to work quite well see this video), the tracking area is limited to 60m^2 ~ (7mx7m), many features that ARCore provides like marker tracking, cloud anchors for multiplayer (some experiments kind of work already see this video) and hand tracking will not be available but it will be a great device, nonetheless.
Amazon Store: USA | UK | CA | DE | FR |
The Amazon store links are affiliated links and they are a great way to support my work directly if you use Amazon anyway. Using the link will not change the price for you but Amazon rewards me after each purchase with 2.5% of the product price. Nevertheless, I would recommend to buy second hand items if possible, which will protect the environment and your wallet.
What is Seurat? Read the introduction here In short it can capture a high-quality scene and make it run on a mobile device. The catch only a predefined area looks sharp and correct. In this experience it is a 2x3 meter box between the wooden gate and the stone bridge. This means if you try to cross the bridge you will see holes in the textures and distorted meshes. This is the limitation of this technique, but the area can be made bigger. I am not sure how big the area can get I will try that later for now enjoy the limited area between bridge and wooden gate. ;) Here and here you can learn how to capture a high-quality scene.
Video describes how a Searat scene is captured for later use with a mobile VR headset.
You will need to download this project for the pipeline to transform the captured data. See the reddit post for more details. Here you can learn how to bring the captured scene in your unity project.
If you lose tracking and the virtual and real-world movement doesn't line up correctly press the touchpad on your GearVR controller or on the side of your GearVR to realign the view. If using the cardboard version there is no convenient solution yet and you need to restart the whole app. Moving slowly also prevents the cameras from becoming misaligned on Daydream devices.
If the positional tracking does not work, please make sure you have installed the ARCore App from the Appstore.
If you experience very low frames and bad jittering please restart the phone and try again sometimes the phone does things in the background that hurt performance. The app should run smoothly at 60fps.
Everything here was tested on the S7 if you have problems getting it to work please open a new issue and let me know.
I want to give credit to the following developers who published useful information I used in building this project:
- FusedVR did some very early work in this area.
- Roberto Lopez Mendez with his very first positional tracking project. Paper (you can use sci-hub.tw to get free access)
- Dimitri Diakopoulos (ddiakopoulos) provided a simplified Seurat pipeline
- Reddit user st6315 (Jung Yi Hung) provided me with a mesh for the vikingVillage app and posted useful information here.
This project uses software from Google (ARCore, Daydream, Cardboard) and Facebook (Oculus) which is not fully accessible so we don't know exactly what data is collected. When using their software, we also have to agree to their privacy terms and conditions. In principle gyro/accelerometer sensor data like head rotation and heat maps of what you are looking at could be logged in the background and passwords could be infer from this read related paper to learn more about the risks. Data about your movement could be used to make predictions about your physical health and provide a digital fingerprint of you since the movement is unique to every user. Information about your surrounding using the camera might be even more problematic and should not be sent to googles servers if possible. This project doesn't store any of your data but please refer to the privacy policy of Google and Oculus to learn what data they might collect on you.
"This application runs on ARCore, which is provided by Google LLC and governed by the Google Privacy Policy"
An excellent vice article about the challenges we face in VR/AR privacy can be found here and I can highly recommend this read if the topic is new to you. If this topic is close to your heart you might be interested in the openarcloud.org oragnisation as well.
If you like this work and you want to support the development of this free software project, please consider a donation via Bitcoin and PayPal. Also recommend this resource to friends who might be interested in it. I am also looking forward to structuring this project better so that it becomes easier for you to collaborate if you are interested.
Paypal: paypal.me/ChristophGeske.