danial-kord/digihuman

Automatic 3D Character animation using Pose Estimation and Landmark Generation techniques

digihuman's Issues

The mixamo character doesn't work

After doing the procedure to add a new mixamo character, when I upload only a face video, it doesn't work.

[book-keeping] Local filepaths were left in the code

Searching for your home-directory C:/Danial results in 136 results. In some cases, it might be nice to un-hardcode those and turn them into parameters:

The only ones that might cause an issue are in pose_estimator.py

hand rotation deformation

When the hand trap and movable trap are all turned on, the wrist will be deformed when the hand is turned over. Do you have this problem?

Hi, does is possible to mgirate this project to UE based?

Hi, does is possible to mgirate this project to UE based?

Why don't you use MediaPipeUnityPlugin

Hello,
I'm working an a similar project and your work here is most valuable. I'm wondering if you considered using the MediaPipeUnityPlugin? It seems it would greatly simply the code architecture so it would not be necessary to use the python backend. Is there a reason you chose not to use it?

Support the latest 52 blendshapes in your Face Capture

Feature Description

The following commit provides the latest mediapipe 52 blendshapes
google/mediapipe@ba10ae8

The geometry and coefficients are listed here

With this new addition, this provides facial mocap for avatars that use the ARKit 52 blendshapes.

Current Behaviour/State

Currently, Users who are using industry standards e.g. Character Creator 3 limit themselves only facial mocap apps from Apple App Store

The Mixamo character not shown

I choose mixamo charactor but not shown in game,

Support of Apple Chipset?

Hi there,

I am wondering if this project can support the Apple CPU?
Here are the problems I get:

1. mediapipe package is not officially support which may not be your problem
2. warnings package is not supported: I removed it from the code and requirements.txt, which is also fine
3. python server.py caused errors, this maybe cause by the opencv

objc[1834]: Class CaptureDelegate is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb25a0) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_videoio.3.4.16.dylib (0x122d30860). One of the two will be used. Which one is undefined.
objc[1834]: Class CVWindow is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb25f0) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_highgui.3.4.16.dylib (0x116620a68). One of the two will be used. Which one is undefined.
objc[1834]: Class CVView is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb2618) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_highgui.3.4.16.dylib (0x116620a90). One of the two will be used. Which one is undefined.
objc[1834]: Class CVSlider is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb2640) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_highgui.3.4.16.dylib (0x116620ab8). One of the two will be used. Which one is undefined.
Traceback (most recent call last):
  File "/Users/nobody/Documents/Projects/playground/DigiHuman/Backend/server.py", line 122, in <module>
    app = Flask(__name__)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 558, in __init__
    self.add_url_rule(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 66, in wrapper_func
    return f(self, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 1216, in add_url_rule
    self.url_map.add(rule)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 1562, in add
    rule.bind(self)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 711, in bind
    self.compile()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 767, in compile
    self._build = self._compile_builder(False)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 1128, in _compile_builder
    return self.BuilderCompiler(self).compile(append_unknown)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 1119, in compile
    co = types.CodeType(*code_args)
TypeError: code() takes at least 14 arguments (13 given)

where is the logic driven charactor in unity?

the server just response 3d world coordinates, how to using this control unity character?

Webcam input

Thank you for the amazing project.
Is it possibile to use it with a webcam input?

face animation

Thank you for sharing your great work!
I'm interested in face animation.
I cloned this repository and run on unity, but Unity-chan's face was not animated.
I have questions.

1. where can I get the charactor using demo?
2. how to specify each blendshape value to the BlendShapeController component?

Facial animation demo missing

Hello Danial,

This is a great work. I tried running it, but it seems some prefabs are missing and I also could not find the sample for facial animation.
Could you please let me know how to get those?
Thank you in advance