moygcc/hpcwild

Human Performance Capture from Monocular Video in the Wild (3DV2021)

hpcwild's Issues

torch-mesh-isect is not compatible with your environment.yml

Thank you for publishing this code. It is quite intriguing! Unfortunately, torch-mesh-isect does not work with the environment.yml you have published. I presume it because of an incompatibility with CUDA versions greater than 9. I am unable to compile and install it with CUDA 10.2, and it would appear to be an issue for all v10 installations.

Is it possible to eliminate or workaround the torch-mesh-isect dependency? Perhaps use a similar (but more up-to-date) library?

Multiple changes to dependencies and code required

I have been able to get your project working on my end, but I ran into several obstacles along the way and wanted to share the modifications required to make installation more generic. Here are the steps I took:

1. Create a test_images directory at project ROOT (it was missing).

2. Include additional PIP dependencies: pyhocon, ipykernel, pycocotools, chumpy, smplpytorch

3. Install PyMCubes with pip install git+https://github.com/pmneila/PyMCubes.git

4. Git clone NuScenes-Devkit from https://github.com/nutonomy/nuscenes-devkit.git and install from setup directory.

5. Manually install PyTorch3d with pip install git+https://github.com/facebookresearch/[email protected]

6. Further, there are several places the code needs to change, because it is pointing to your personal home directory (/home/chen). Here are the locations:

registration/lib
th_SMPL.py

smpl_right_hand_idx = np.load('assets/smpl_right_hand_idx.npy')
smpl_left_hand_idx = np.load('assets/smpl_left_hand_idx.npy')

smpl_layer.py

self.model_path = 'smpl_rendering/smpl_model/SMPL_FEMALE.pkl'
self.model_path = 'smpl_rendering/smpl_model/SMPL_MALE.pkl'	

smpl_paths.py
fname = 'registration/faces.npy'

registration/smpl_registration
fit_SMPLD_final.py
part_labels = pkl.load(open('assets/smpl_parts_dense.pkl', 'rb'))

smpl_rendering
smpl_renderer.py
_, faces, aux = load_obj('smpl_rendering/text_uv_coor_smpl.obj', load_textures=True)

7. Next, the smpl_parts_dense.pkl file is missing from IPNet, so those assets needed to be copied over as well.

With these changes, the entire pipeline works. The only question I have for you is how you have generated the sequence in your teaser with the normals overlay. This is specifically what I would like to capture (not the OBJ sequence in .pkl). The resulting front-view output is the mesh overlay. Please be so kind as to indicate where in the code I can enable normals output.

Also, one more suggestion for you (if you will have desire + time): I find the OpenPose library to be very inaccurate for many complex movements. For example, the fencing sequence --- it breaks down half-way into the movement by flipping the character's direction:

If you will replace OpenPose with HuMoR, this project would be in a league of its own. Please consider it. I would help, but I have very little time to spare.