zengwang430521 / decomr Goto Github PK
View Code? Open in Web Editor NEWRepository for the paper " 3D Human Mesh Regression with Dense Correspondence "
License: Apache License 2.0
Repository for the paper " 3D Human Mesh Regression with Dense Correspondence "
License: Apache License 2.0
May I ask what is the physical meaning behind line 45-46, is that orthographic projection? or transformation to UV space?
https://github.com/zengwang430521/DecoMR/blob/master/datasets/preprocess/generate_gt_iuv.py#L45
and
https://github.com/zengwang430521/DecoMR/blob/master/datasets/preprocess/generate_gt_iuv.py#L52
why do you have '1 - uv_tmp[:, :, 0]'
Additionally, I agree it is continuous among different semantic body parts in your UV map, which makes the learning easier.
Is the frontal view and back view of the same finger continuous in your defined UV map? as in Fig 3 in original paper.
Traceback (most recent call last):
File "demo.py", line 140, in <module>
img=img, use_bg=True, body_color='pink')
File "/data/DecoMR/utils/renderer.py", line 351, in render
return self.renderer.r
File "/home/xumingkai/.local/lib/python3.6/site-packages/chumpy/ch.py", line 555, in r
self._call_on_changed()
File "/home/xumingkai/.local/lib/python3.6/site-packages/chumpy/ch.py", line 550, in _call_on_changed
self.on_changed(self._dirty_vars)
File "/home/xumingkai/.local/lib/python3.6/site-packages/opendr/renderer.py", line 1080, in on_changed
self.vbo_verts_face.set_array(np.array(self.verts_by_face).astype(np.float32))
AttributeError: 'ColoredRenderer' object has no attribute 'vbo_verts_face'
Hi,
Thanks the great work! You propose the Boundary Free IUV Map to replace the DensePose IUV Map. I agree you said the BF IUV Map is more continuous than the DensePose IUV. So I want to use BF IUV Map. But you only provide the 56x56 pred_IUV Map, It is not accurate for me. I want to tranining the generating BF IUV Map model to satisfy my needed. Could you provide the BF IUV Map Datasets?
Thanks very much!
Will the code of constructing new UV map / new obj file be released?
@zengwang430521 I have seen the smpl boundry free image in the paper and I want to generate it. But I did not find the part that generates smpl boundry free image in the code. Any help would be greatly appreciated.
1、if i use the SMPL parameter generated by SMPLify-X. Should i use the 'trans' matrix genereted by SMPLify-X of SMPL to get the 'sampled_vertices' , or could i use SMPL(pose, betas, trans) to get the 'sampled_vertices' ?
2、And how do we get the camera coordinates from joints calaculated from 'sampled_vertices' generated by SMPL(). use 'self.smpl.get_train_joints(sampled_vertices)' ? but it never use 'J_regressor_h36m.npy'
I've just been in touch with this research direction, and I'm looking forward to your reply. Thank you
when i try to running the demo ,then it happened,i don't know about this question ,but i install all the environments, please help me,thank you,i really need to running it .
the file is missing
Missing file 'up_3d_trainval.npz' when I run
'python preprocess_datasets.py --gt_iuv'
I do not see the 'Bboxes' in original H36M website, can you also provide the 'Bboxes'
Hi Wang,
I am trying to understand the bary-weight calculation process.
Basically you are trying to mapping the uv space faces to xyz space faces and want to solve the weight between them.
However, I start to get lost from https://github.com/zengwang430521/DecoMR/blob/master/models/uv_generator.py#L179
and fail to understand why and what is the theory/intuition behind that about
the calculation of w, u, v?
I know from https://github.com/zengwang430521/DecoMR/blob/master/models/uv_generator.py#L205 you are trying to construct a mask in UV space, but the lines before that seems confusing to me.
Hi Zeng Wang,
Thanks for this great work! What is the license for this repository? If you could add it, that would be greatly appreciated. Thanks in advance.
FYI - here's what happens if a repository has no license:
"You're under no obligation to choose a license. However, without a license, the default copyright laws apply, meaning that you retain all rights to your source code and no one may reproduce, distribute, or create derivative works from your work."
Best,
Saafke
Hi @zengwang430521 ,
I noticed that in your implementation of the warp_feature, a single element in the output transferred F_{uv} could potentially correspond to multiple elements in the local feature F_{im}.
For example, the u,v coordinate of a element in F_{uv} is [16,16]. While in the IUV image, the u,v coordinates are of float type. Then, rounding is required to create the correspondence, and thus multiple locations in the IUV image (e.g., an element with [u,v] = [16.1, 16.6] and another element with [u,v] = [16.4, 16.7]). Then two locations in F_{im} correspond to the single location in the F_{uv}.
Therefore, I am wondering which feature vectors will be transferred?
Best.
hello,i find we predict the camera parameter[z,tx,ty],and then we get the wold coordinate[tx,ty,2×focal_length/img_size×z],and we get this picture just use the world coordinate in render,
now,i want to use pytorch3d to replace the original renderer,but in pytorch3d,the setting in camera should know the R and T,but we only konw the world coordinate [tx,ty,2×focal_length/img_size×z],can you help me,how can i use pytorch3d to get the same img(There can be no background image)
I search around and only find resnet is initialized, may I ask how is other part like 'ConvBottleNeck' initialized ?
Line 42 in 69b5986
when i run the eval.py on the lsp_test dataset.
`/home/ni/anaconda3/bin/python /home/ni/DecoMR/eval.py --checkpoint=data/model/h36m_up3d/checkpoints/h36m_up3d.pt --dataset=lsp --log_freq=20
Checkpoint loaded
data loader finish
Traceback (most recent call last):
File "/home/ni/DecoMR/eval.py", line 318, in
run_evaluation(model, args, options, args.dataset, args.log_freq)
File "/home/ni/DecoMR/eval.py", line 122, in run_evaluation
from utils.part_utils import PartRenderer
File "/home/ni/DecoMR/utils/part_utils.py", line 7, in
import neural_renderer as nr
File "/home/ni/anaconda3/lib/python3.7/site-packages/neural_renderer/init.py", line 3, in
from .load_obj import load_obj
File "/home/ni/anaconda3/lib/python3.7/site-packages/neural_renderer/load_obj.py", line 8, in
import neural_renderer.cuda.load_textures as load_textures_cuda
ImportError: /home/ni/anaconda3/lib/python3.7/site-packages/neural_renderer/cuda/load_textures.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail37_typeMetaDataInstance_preallocated_32E
Process finished with exit code 1`
Do you know the cause of this problem
May I ask is the pred_vertices in camera coordinate?
Line 134 in 726c7a1
and will the following operations make it in root-relative coordinate and in mm unit? so that we can use the pred_keypoints_3d to calculate MPJPE with the ground truth in root-relative coordinate?
pred_keypoints_3d = smpl.get_train_joints(pred_vertices)[:, joint_mapper]
pred_pelvis = (pred_keypoints_3d[:, [2]] + pred_keypoints_3d[:, [3]]) / 2
pred_keypoints_3d = pred_keypoints_3d - pred_pelvis
May I ask how do you reconstruct SMPL parameters given the vertex location (location map) from your work?
And what is the shape of the location map? (is that a 3 channel xyz denoting the camera coordinate in mm?)
Hi,
Thanks for the great work. Do you have the (u,v) coordinates corresponding to SMPL mesh vertices in your continuous UV map? Or a script to obtain the same?
Thanks
definition of tv_loss:
Line 246 in 69b5986
Will this loss only apply only at uv_res 128 or less? If I have a uv_res at 256, does that mean we can get rid of the loss?
@zengwang430521 Hi, how to generate uv map preoposed in this paper on my own datasets? Any help would be greatly appreciated.
I run your evaluation code on SURREAL test set (493 imgs), and found out that the dataloader code only provides female genders.
Why don't you use the groundtruth genders which are provided in the dataset for the correct evaluation?
When I test your pre-trained model on SURREAL test set with groundtruth genders, I get
MPJPE: 67.82
MPVPE (Surface error): 64.59
These numbers are far different from the numbers reported in the paper (MPVPE: 56.5 , Tab.3) and not much good compared to the accuracy of BodyNet (MPVPE (Surface error): 65.8).
When I test your pre-trained model on SURREAL test set with intact code, I get
MPJPE: 54.29
MPVPE (Surface error): 51.65
Do you have any idea to get the numbers you reported in the paper?
And I think assuming all target meshes as female is inappropriate and unfair to compare to BodyNet results.
Hi, thanks for your great work!
I have a question about the location map. Specifically, I want to know which is the coordinate system w.r.t. the 3 channel(xyz) of the pixel in the location map. It seems be defined in the camera coordinate rather than the world coordinate, am I right? If so, can you tell me why not use the world coordinate?
Thanks!
Hi,
Thank you for such a great job. I am very interested in the new UV map you have proposed. I want to know how SMPL UV map can be converted to this BF UV map.
Thanks.
May I ask what is the meaning of the weight in line 276-280
in https://github.com/zengwang430521/DecoMR/blob/master/train/base_trainer.py#L276?
Is this weighting on whether the u channel should penalize more or v channel should penalize more?
or
it is each pixel and channel weight for activation larger than 0?
Will this weight make overall loss smaller?
and why the UV head needs Sigmoid? only to normalize the activation to 0~1? will this stabilize training and make it converge faster?
Line 128 in f96526b
Thanks!
Hi @zengwang430521 , nice work! But there is no script that preprocesses SURREAL. Could you share it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.