kariaro / vrigunity Goto Github PK
View Code? Open in Web Editor NEWVRM hand tracking using mediapipe
Home Page: https://discord.com/invite/Enaup9TJPd
License: MIT License
VRM hand tracking using mediapipe
Home Page: https://discord.com/invite/Enaup9TJPd
License: MIT License
Currently even when approved external PRs can't run any unity tests because they do not have access to any secrets
.
Find a way to make this work while still being secure
Compile this project for OSX/iOS and make sure the basic features work.
Virtual webcams are not required to work for this issue.
Update project readme and make it easier to use.
Download
button)Check that MediaPipe works during a pull request to develop
Add a recording button to export/save mocap data that should be able to be imported into tools like blender
Make a script that can automatically update the project version.
This could be updated from an editor script in Unity e.g the ApplicationVersion
script
These files needs to be automatically updated
installer.iss
ProjectSettings/ProjectSettings.asset
How does your motion capture project support importing fbx models? My Fbx model is based on APose, and VRM is based on TPose.
Adding a virtual camera mode for linux systems
Steps:
Are you interested in supporting an Android build?
Add an issue template to make it easier to fix bugs or issues
I found several bugs.
Hi, how can I free up the movement of the hips in 3D space? What is the part that is blocking the hips?
thanks
Check if lowering the AI model complexity will lower the CPU usage
Thanks for the amazing example on mapping skeleton data to vrm. I'm using the following code to map the legs.
{
Vector3 rKnee = ConvertPoint(eventArgs.value, 25);
Vector3 rAnkle = ConvertPoint(eventArgs.value, 27);
Vector3 vRigA = Vector3.right;
Vector3 vRigB = rKnee - rhip;
Quaternion rot = Quaternion.FromToRotation(vRigA, vRigB);
rUpperLeg = rot;
Vector3 vRigC = rAnkle - rKnee;
rot = Quaternion.FromToRotation(vRigA, vRigC);
rLowerLeg = rot;
}
But the mapping appears to be wrong. The leg is rotated and translated wrong. Anything wrong with the way that I compute the rotation?
Add the version number to the window title
Currently, I can only track upper body, will you be able to track lower body?
Will the project support whole-body detection or even motion posture detection?
Add some workflow code that automatically changes target branches to develop if the branch is not develop
Add an experimental option to allow leg movements. This would be useful for both vr and mmd programs.
Instead of making large issues lets make them small.
First part of the bodytracking components is making sure the wrist and arm are rotated together
Find a way to implement v4l2loopback
to allow virtual camera devices to be used on Linux.
Try find solutions that require as little maintenance as possible.
Hey Kariaro, I see that MediaPipeUnityPlugin has been updated MediaPipeUnityPlugin v0.12.0 , then ARKit BlendeShape support has been added, is it possible to add this support in the next version, thanks.
To reduce CPU usage check if other projects than mediapipe gives better performance
This is in response to #66
Add a baseline for language in the app
Great repo. Could I run it in mobile device?
Add a box selection tool that allows a user to select were the hands should be tracked.
After receiving some more views on this project I’m going to see if I could spend some time improving the model rigging :D
Windows
v0.4.1
Windows 11, CPU i7 7700k , webcam Logitech c925e
The software using the camera and sending VMC is using 65% of my CPU.
Even when webcam and VMC sender is off , the CPU usage this is 18 %.
It does not make sense to me. I have done the handtracking in mediapepi in python and my usage was lower than 10%.
Maybe an option to change de model complexity, mine was 0 and working on the hands.
extras issues:
No response
Add a low poly default model as an option
Update to version 2021.3.14f1
to fix this issue
Add a way to use this tool for VRChat
For large models > 100 MB add a warning message that pops up that it can take a long time to load
Find a way to remove .meta
fiels from the StreamingAssets
folder.
Tasks:
Assets
.gitignore
is introduced removing .meta
files from the folderWhy do I use the Apose character model to replace the Vrm model in the project, and there are ghosts in the activity?
Hello author! Your holistic motion capture is much smoother than what I wrote, thank you for making it open source. But I noticed a problem, I can't move the position of the VRM model in the scene. I have a VR treadmill. In my code, as long as I import the SDK of the VR treadmill, I can move the VRM model while motion capture. But in your code, I can't change the position of VRM. Can you tell me what to change in the code to move the VRM model?
Take inspiration from other UI’s and make a small sketch of how it could look.
Talk with the community on what should be prioritized.
Add an option to select the output resolution of the virtual camera
This should not be dependant on the size of the window
Try update how the tracking is applied to the 3d model
Branch feature/optimize-app
Windows
0.5.0
There's an issue with tracking of fingers being constantly lost or not picked up straight away, while tracking of wrists (and everything else) is working fine. Happens when using this program in conjunction with vSeeFace. Changing settings doesn't seem to fix the problem.
VOD Showing the issue: https://www.youtube.com/live/1pHrDu1hecI?si=I3io7epS-T0PYhCN
No response
Hi author, I copied the code into a new project (only the model and Solution nodes remain in the scenario). When running the project, the arms of the model will not move, and the upper body, head, and hands can move normally. Do you know what causes this? I turn on all the bone nodes when I can turn on the camera.This question has been bothering me for a long time...
`public void onBtnClick()
{
SetCamera(!isCameraShowing);
}
private void SetCamera(bool enable)
{
isCameraShowing = enable;
for (int i = 0; i < 14; i++)
{
BoneSettings.Set(i, enable);
SolutionUtils.GetSolution().Model.OnBoneUpdate(i, enable);
}
if (enable)
{
SolutionUtils.GetSolution().Play((_, _) => {
// Error handling
SetCamera(false);
});
}
else
{
SolutionUtils.GetSolution().Model.ResetVRMAnimator();
SolutionUtils.GetSolution().Stop();
}
}`
Update the MediaPipeUnityPlugin to v0.10.1 and add a build readme for how to download the packages for local builds.
Update the workflow scripts to download and unpack the libraries for mediapipe.
An issue has been posted on the MediaPipeUnityProject:
homuler/MediaPipeUnityPlugin#779
Add a -resetSettings
command to make it possible to fix configuration errors
Research and find a way to denoise the output data using temporal cohesion
Add a button that shows debug information:
This could maybe also contain more information such as:
Update the project UI and make use of better design patterns
The project is starting to contain very large files for different systems.
To fix all these issues the project will migrate to git lfs
This will break all current forks
Check how easy it would be to add code linting to the project.
This would make sure all code if formatted correctly
I cannot get this to work at all, I downloaded DSSBodyTracker which was a clone of an older version of this project and that clone version actually worked, other than not sending anything through VMC. So I found the original (this project) and whenever I choose start camera the model's eyes close. It doesn't matter which camera I use. This results in this program being completely useless in it's current state.
For version v0.3.0
read the camera resolution and update the camera resolution list to get the maximum size for the camera.
Also look if it's possible to add a custom field to edit the camera resolution manually
Mac
lastest version
After installing the app on my laptop, a MacBook Pro with the cpu M2, and the Mac OS 13.2.1, it didn't work. It shows the only hint "The application 'VRigUnity' cannot be opened."
No response
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.