hobovr-labs/issue_tracker

General issues to fix or features to be implemented

issue_tracker's Issues

[BUG]: lazy-sockets receiver shutdown behaviour

Current the threaded packet receiver we have can block the client until the server exits, at least in our driver, which is a big no no.

Poser implementations

We have a working driver (with a new version almost done 💯 ), but at the same time we only have a few test posers which are either old or down right terrible (as in some are usable, but not much more that, not maintainable thats for sure).

We need some new simple posers, weather its a complicated poser that implements some new tracking system, or a small poser used to make something useful out of old cardboards, or something else completely.
In either case it deserves a repo and it's own page on our doc website.

Oh and the old stuff thats currently living in the same repo as the driver, it should be gradually reduced to only test posers.

Fix typos in docs

I noticed that we have a bunch of typos in our docs, and some on the main page of our website.

[BUG]: Fix receiver buffer realloc issue

Our lazy-socket receiver class has an issue with it's internal receive buffer where it can smash heap after a realloc, because right now it's done with an almost raw buffer by calling realloc().
Switching to one of the STL containers for managing memory will be ideal.

[FEATURE REQ]: The ability to emulate a Wii controller using a VR controller

Emulating Nintendo Wii games in a more user friendly way
I think it would be super cool if Nintendo Wii game emulation could utilise a VR controller for interaction with the game.

There are plenty of ways to setup a genuine Wii remote with the sensor, DualShock controllers, Switch controllers and many more.
The problem with these solutions; they require you to have Wii hardware which is getting harder to come by in a reliable state. If not Wii hardware, you looking at controllers that just don't match the experience of playing on the Wii. With the uptake of VR among enthusiasts (often many of which overlap with the emulation community) I think having a reliable, easy to setup system for VR motion controls in Wii emulation would be amazing.

The ability to play Wii games and use VR controllers as a Nunchuck and Wii controller seamlessly. Not requiring the use of a HMD especially would feel like a very genuine Wii experience. So far the closest thing I know if is: https://github.com/MichaelJW/DorsalVR

There are many current "solutions" but they all require an extensive setup process and are extremely irritating to work with. They vary from platform to platform, methods used and the controllers/hardware they require. I think it should be possible to make a consistent experience using VR controllers a better experience for Wii emulation. While the idea of playing Wii games in a virtual environment in VR is also cool, it needs to be easy to use, otherwise no one will use it. Plus, having the ability to choose the way you experience the game is key.

All the methods available are awesome, but they are difficult to use, unreliable and confusing. Until a unified, easy downloadable solution is available, I feel that we don't have a way to preserve the Wii experience once the real hardware becomes rare, old and collectable.

Just an idea lol

The current methods so far: https://wiki.dolphin-emu.org/index.php?title=DSU_Client

Ukrainian Support Charity Page

It would be nice if we could have a 4 across and down page of icons for the charities page. If not easy to explain I can make a mock-up design to explain it.

[BUGS]: Fix all driver UB

Fix undefined behavior in our driver.
Make a proper event loop, handle events properly, etc. etc.

Test gaze master inputs

We need gaze master tested, imo VRChat is the most popular choice now (they added eye tracking to avatars through their new API), for us to use gaze master with VRChat we'll need a new plugin for that, shouldn't be too complicated... hopefully...

[FEATURE REQ]: Make gaze master icons

[FEATURE REQ]: Add support for device plugins

Is your feature request related to a problem? Please describe.
Currently all supported devices are hard coded into the driver, this complicates to process of adding support for new devices for both users and developers.

Describe the solution you'd like
Add support for user defined device plugins.
To be more precise add a mechanism to change some or all devices available to posers using a factory in a dll approach.

Describe alternatives you've considered
I couldn't come up with any good alternatives for the factory pattern to add support for device plugins, suggestions are more than welcomed in that regard!

Additional context
More info about the factory pattern:

• The Factory Pattern - Mike Shah - CppCon 2021 - YouTube
• Factory Method - Refactoring.Guru
• Factory method pattern - Wikipedia
• How to implement the factory method pattern in C++ correctly - StackOverflow

Add Eye Tracking Functionality and API

Is your feature request related to a problem? Please describe.
VRChat needs to see my emotions.

Describe the solution you'd like
This is a bit more than just tracking pupils. In order to get emotions passing into games you need:

• Eye tracking for both eyes including pupil position and size
• Infrared Camera for tracking both eyes
• A simple API that can be expanded on

Then, later on:

• Track facial shape keys or landmarks around eyes
  • eyebrows
  • upper eyelid closed
  • squinting
  • etc. there are a lot of muscles around the eyes
• Track the mouth using facial landmarks or shape keys
• Have an API that allows a few or all tracking features to be implemented.

Additional context
Already got pupil tracking working. Need to work on the hardware and some API now:

2021-06-24_16-48-49.mp4

Add More Complex Controller Tracking

Describe the solution you'd like
We currently have blob tracking with IMU only. There are other tracking methods like constellation, and several OpenCV tracking methods like POSEIT. We should be able to open up some program, upload a 3D model of the LEDs or controller, and get a file needed for tracking the controller with one of those methods out of it.

Describe alternatives you've considered

• Visual Deep learning Tracking. Can be done with IMU estimation and camera input only. Requires a lot of training and may require special GPUs.
• Constellation Tracking
• OpenCV POSEIT + ORB
• Other Generic OpenCV Tracking Algorithms

Out of all of those, POSEIT + ORB seems the simplest to implement and most general.

Additional context
Currently working on the program to get the data needed for POSEIT:

The right image is the keypoints from ORB, and the left is the projected points onto the 3D ring in 3D space as a point cloud. However, the point cloud is wrong for some reason, so there needs to be another way to get the 3D points.

Update the garbage driver device protocol

Is your feature request related to a problem? Please describe.
The protocol our driver uses to communicate device tracking and inputs is garbage. It feels like two different unfinished protocols thrown together, because it is. It's awful and i hate it.

The main issues with the current protocol is:

• inefficient packet structure
• lack of two way communication in the tracking protocol
• its inconsistent af
• socket code in the driver can be improved, like a lot

Describe the solution you'd like
We need to go over all the protocols used in the driver currently, throw them in the garbage, and come up with replacements that are:

• consistent
• efficient, i.e. make packets as small as possible
• doesn't make me want to kms

Additional context
This has been an issue for a while, posers can't tell if the driver ignores them unless they try to actually sync their device lists, and a lot of other terribleness. Time to finally fix it!

[FEATURE REQ]: Better dependency handling

Is your feature request related to a problem? Please describe.
Our driver starts to grow in the number of external libraries it depends on, right now all that is handled in it's CMakeLists file and each library needs to be added by hand, each of them has different paths, weird source trees (looking at you openvr), etc.

Describe the solution you'd like
Since we have forks of all of them external libs we're using, we should format them all to follow a single convention for source trees and build instructions, so that our driver's CMakeLists doesn't grow larger then the driver itself...

Describe alternatives you've considered
I considered other build systems like bazel and a few python based ones, but the amount of effort needed to transition to those is too much imo.

Additional context
If we don't come up with some standard to follow early on, it will really bite us in the ass later on.

Create multi fresnel lens mechanical design

Is your feature request related to a problem? Please describe.
There are some recent super thin headsets, and you'd need a pretty good magnification to do that, which is likely only possible with multiple Fresnel lenses.

Describe the solution you'd like

• Create a mechanical design for a lens array where both the lens closer to the screen and the lens closer to the eye can be adjusted
  • This can be in OnShape, Solidworks, OpenSCAD, etc., as long as the shape can easily be changed to fit later headset designs.
• Build a prototype and check that it has enough light transmission to be usable.
  • If not, design a backlight using the same tech for the soft light for the controller tracking.
• Make sure there's space for an eye tracking camera and IR LED somewhere
• Incorporate the design into a larger headset

[FEATURE REQ]: gaze master app plugins

Allow plugins for gaze master to change the API it uses to comm with apps.

[BUG]: Cmake boost builds borked on Windows

Describe the bug
Boost can't build for Windows in GitHub CI, due to boost/system/details/sprintf.hpp no picking up visual studio properly and trying to use the wrong vnprintf definition

To Reproduce
Steps to reproduce the behavior:

1. Recursively clone: git clone --recursive https://github.com/HoboVR-Labs/boost
2. Build it like a normal cmake project: mkdir build && cd build && cmake .. && cmake --build .
3. Observe compile error

Expected behavior
A successful build

Desktop (please complete the following information):

• OS: Windows
• hobo_vr Version: 0.7.x

[BUG]: Fix installers

Our online installers are broken, cuz the repo structure changed, we need to fix that.

[LICENSE]: We need to update our copyright headers

Some license annoyance for ya, we need to update the dates in our license headers.

Not urgent.

[FEATURE REQ]: Better socket solution

Is your feature request related to a problem? Please describe.
Our driver uses sockets to communicate with posers, but right now its done using our own wrapper which is so fragmented its killing me, some parts barely work together or work differently on different platforms for seemingly no reason.

Describe the solution you'd like
Better raw socket wrapper solution which adds 0 performance overhead. It needs too be low level and fast, but also comfortable enough to be useful.

Something like our own cross platform library that does just that would be perfect.

Describe alternatives you've considered
ZMQ, but i had to ditch the idea for our tracking loop because it added too much overhead :/

Additional context
We use sockets a lot and i don't want to reinvent the wheel every time another component needs to use them.

[BUG]: Fix device status issues

Right now the new driver version suffers from this weird bug on windows when the devices won't show up as active unless something sends a haptic event to one of the controllers.

Weird thing is that this is only a windows issue and that it is not a problem on older versions of our driver.

[FEATURE REQ]: Finally add support for update checks

Is your feature request related to a problem? Please describe.
So our driver had a plug for an update checker for almost a year now, right now it does nothing :/

It's a function that gets called by each device when it activates with that device's serial.

The aforementioned function is in hobovr_device_base.h: line 45.

Describe the solution you'd like
Ideally it should check weather or not there is a new driver version released,
something like comparing the latest release tag to the current driver version using the github web api is a good start.

Describe alternatives you've considered
Setting up some web based trigger on our website for updates, for example setting up another subdomain which will signal the latest version.

To be fair any other web based trigger will work, if we implement it properly in that function.

Additional context
Add any other context or screenshots about the feature request here.

Free hosting yes

Yes

https://www.netlify.com/

[Feature request] WII BALANCE BOARD for VR LOCOMOTION

Hello, with this software https://github.com/lshachar/WiiBalanceWalker/releases WiiBalanceWalker I managed to use one WII Balance board to map on to keyboard to control myself by leaning on my feet to strafe right and left and backwards and march/ jog on the spot to move forwards, this worked well. However there was limitations to this software since this doesn't support VR inputs only VJoy and I don't think there was any way to combine multiple WII balance boards to create one great big controller.

In the image I uploaded here is the idea I am going for. Since WII FIT boards are only £10 each and are not used anymore they are readily available to purchase. With one WII FIT balance board you can move with the idea above while facing one direction only for those people who only face forwards, 1 or 2 wii balance board is a solid easy idea to use. To move you can jog, march for those who want to burn calories, or for those people who want comfort and minimal effort there should be a lean by putting your body weight on your feet to move around similar to the PSVR rudder.

However with multiple WII balance boards this will enable people to jog/ march/lean 360 degrees, this is possible with 4 or 5 wii balance boards, to move diagonally you would use your whole body and turn and jog on the spot for example, When strafing one foot would be on the wii balance board while the other foot is off, or even a better idea would be to have two foot on the board while jogging/marching and leaning right or left to strafe at the same time. This should be highly customisable for the user to choose the best most easy way to navigate. To turn 180 degrees the user will have to turn their physical body 180 degrees to face the opposite WII balance board and walk in real life which will naturally translate in game as room scale locomotion.

I firmly believe this idea will work very well, as the WII Balance boards are cheap as chips being £10 in the UK, they connect via Bluetooth and have a battery life of 20 hours, depending on the type of rechargeable battery the user would use. Thats enougth for 3 hours of VR gameplay everyday for a week. They are well built as they are made of plastic and steel, so they have weight to them, and they have 4 rubber grips below for stability so it is sturdy enough to march or jog aggressively on the spot if one would choose to do so. It would also be perfect if the software can reconise when the player is aggressively jogging to run ingame or marching lightly to walk at a steady pace ingame for example. Of course there should be plenty of sliders to customise one's locomotive needs. But to also have plenty of presets options ranging for inactive, comfortable and athletic. Even better is for presets to support popular games to make this easy and accessible for the player, since old games and newer games will have different or similar movements.

I hope you can consider this idea as a natural easy form of locomotion, I would pay a fair price to use this for my VR gaming needs if you do release it on Steam. Now imagine combining this with a xbox 1 kinect, I dream too much sometimes haha. Please let me know if you will consider this, thank you regards Wick.

[FEATURE REQ]: Set an expiration for CI artifacts

Is your feature request related to a problem? Please describe.
Yeah so our free storage space is not infinite and if we keep all artifacts from dev builds, it'll run out pretty soon.

Describe the solution you'd like
Github CI supports artifact expiration, either by date or a time period, setting it to like a few days or a week would be long enough for us, we don't need to keep them all forever anyway.

Release builds and master builds can have artifacts that don't expire though.

Describe alternatives you've considered
The alternative is paying for more storage xd