kuboshizuma/uvtextureconverter

To convert atlas texuture (defined in Densepose) to normal texture (defined in SMPL), and vice versa.

About Me

• ACES,Inc Co-Founder
  • Currently, we are developing SHARON, which is an API/SDK service for image recognition algorithms, and working on collaboration projects with companies in various fields.
  • Now hiring algorithm engineers and software engineers! (recruit page)
• Aspects of an algorithm developer
  • My research area is image generation by a deep generative model. Especially engaged in computer vision in fasion field.
    • [Kubo+ ICCVW'19] UVTON: UV Mapping to Consider the 3D Structure of a Human in Image-Based Virtual Try-On Network
    • [Kubo+ IPSJ Journal'19] SwapGAN: Cloth-Region Aware Generative Adversarial Networks toward Virtual Try-On System
  • Currently engaged in general development of computer vision algorithms and service development. In paticular I am focus on face analysis, object recognition, and pose estimation algorithms.
• Aspects of a software developer
  • I have been developing multiple web services since I was a university student. The most exprerienced web framework is Ruby on Rails and I like Ruby.
  • Recently, Python is often used to match an algorithm development language.
  • Java/Kotlin and swift has started to be used for native application development.
• Links (Japanese only)
  • Homepage | seishin55.com
  • Blog | ほろ酔い開発日誌
  • Qiita | shizuma

Qiita Works

Paper Reading at Deep Learning JP

• 2020/10/23 [Tu+ ECCV'20] VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment
• 2020/07/31 [Jin+ ECCV'20] Whole-Body Human Pose Estimation in the Wild
• 2020/04/10 [Zhou+ ECCV'20] Tracking Objects as Points
• 2019/12/06 [Feichtenhofer+ ICCV'19] SlowFast Networks for Video Recognition
• 2019/09/13 [Shysheya+ CVPR'19] Textured Neural Avatars
• 2019/06/14 [Zhou+ arXiv'19] Objects as Points
• 2019/03/08 [Zhao+ AAAI'19] M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network
• 2018/11/30 [Saito+ SIGGRAPH Asia'18] 3D Hair Synthesis Using Volumetric Variational Autoencoders
• 2018/08/17 [Sabokrou+ ACCV'18] AVID: Adversarial Visual Irregularity Detection
• 2018/06/01 [Zhang+ arXiv'18] Self-Attention Generative Adversarial Networks
• 2018/03/16 [Karras+ ICLR'18] Progressive Growing of GANs for Improved Quality, Stability, and Variation

GitHub Summary

uvtextureconverter's Issues

How to get IUV ?

Hello, the results are impressive and I would like to try create_uv_texture_from_image_by_using_densepose.ipynb with other images, but how to get the corresponding IUV.png?
Thanks in advance for your answer

Is it possible to extract textures directly from the iuv array rather than using IUV.png?

About Atlas2Normal conversion

Thanks for your code!

I have a few questions about the function to convert atlas texture to normal texture.

1. There are three extra data files that need to be loaded in advance, including config/UV_Processed.mat, config/normal_faces.pickle, and config/normal.pickle. What are these files, and what kinds of data these files contain, respectively?

2. If the input size of the atlas texture and mask is [24, part_size, part_size, 3] and [24, part_size, part_size], respectively, and the output normal texture size is, say 512x512, thus, the Atlas2Normal function is actually trying to convert the part-based UV coordinates (defined in SMPL model) into a full-body version of UV coordinates. Is that correct? If so, how to get such correspondence between part-based UV coordinates and the full-body version of UV coordinates? Is there any mapping functions or index to find where the location should be in the outputted normal textures?

3. I have noticed that, in the initialization function of Atlas2Norm, the function tries to load another data file, namely, mapping_relations/atlas2normal_{atlas_size}_{normal_size}.pickle, which has been calculated when performing the first time of atlas2normal texture conversion. Does it mean the mapping function from atlas texture with atlas_size to normal texture with normal_size is actually fixed?

P.S. I have got some hints about the UV_Processed.mat data file from facebookresearch/DensePose#146 (comment). But still do not know the data stored in the other extra files? Would you please help me to understand the whole process, and how to use these extra data to perform Atlas2Normal conversion?

Thanks a lot! :)

Error creating UV texture from image by using densepose

The code uses the provided code. I don't know whether it is my IUV picture or the code

There's something wrong about the hands part and feet part.

Found some bad texture in feet part and hands part. Is there any plan to optimize them?

How can it work without the 'I' information?

Hello, thanks for a such a great repository!

I have one question, how can the create_uv_texture_from_image_by_using_densepose.py script work without the part information? If I understand correctly, the original IUV formulation generates outputs two type of data: the UV coordinates of each body part (of range 0,1), and the body part (of range 1-24) that specifies a what body part (or island of the UV map) each pixel corresponds.

My question is, the script create_uv_texture_from_image_by_using_densepose.py only requires a .jpg with the dense correspondences but in that file there is not information about the body part (what is originally refers to I in the DensePose formulation). I don't see how it can manage to produce the final texture. Could you share some details? I looked at the source code but couldn't find the underlaying logic.

Thanks!

two vital errors when using this project.

When using this project, i met two vital errors when trying to visualize the human mesh. First, the type check code. The additional command type make it impossible to input the PIL.Image.
elif isinstance(im, type(Image)) and isinstance(iuv, type(Image)): im = im iuv = iuv
Second, why the iuv should be multiply 255? This behaviour cause huge confusion.

How to reduce unmapped pixels?

The generated texture contains many unmapped pixels, is there any way to deal with these pixels?
I'm not sure it's dense pose problem or resolution problem.

Is there a way to generate texture from image to smpl 10 parts style UV map?

こんにちは, kuboshizuma-sama,
Thanks for your nice work!
It's really wonderful and cool!
I noticed that your developed API can convert image to densepose 24 parts texture UV map, and then do atlas2normal translate to 10 parts UV map style.
I would like to ask if there is a function that can convert image to 10 part UV texture directly since I found the 2-step conversion consumes too much time( atlas2normal runs about 20 sec in my case).

Misalignment for the hands and feet part.

The texture is misaligned when doing Normal2Atlas conversion.
Same issue as the one closed before. issue 16
Could you please help solve this issue? @kuboshizuma
Thanks a lot!