havenfeng / photometric_optimization Goto Github PK
View Code? Open in Web Editor NEWPhotometric optimization code for creating the FLAME texture space and other applications
License: MIT License
Photometric optimization code for creating the FLAME texture space and other applications
License: MIT License
Hey,
first of all thanks for your great work!
I was reading how you built the texture space from the FFHQ dataset in the Readme-file.
Is it possible that you release the code you used for building the texture space? I would be especially interested in seeing how you formulated the photometric loss.
hi, I need to replace the customized landmark for fitting. How to make the new ‘landmark_embedding.npy' file?
Nice work!
I wonder how can I apply this for a custom image?
I guess I need both
for the custom image, right? Do I need more than this? How can I obtain these files for a custom image?
Thanks!
Hi, I could not find texture_mean_image_completed.npy and texture_basis_image_completed.npy in FLAME project. Could you please share them or add them into FLAME project? Thx! @HavenFeng
Hi, I found it needs a inpainting model to complete the unvisible part of the face, could you share your pretrained inpainting model?
I was wondering if it is possible to see C++ implementation of Pytorch3dRasterizer.
How did you fit BFM model to FLAME? I have tried fitting a Flame mean shape to BFM mean shape, but couldn't find good shape parameters for FLAME model.
If you can give me some info on this it would be helpful.
hi, many thanks for your great work! I wonder why there mul -1
here
Are there any specific reasons for you using orthogonal projection instead of perspective? I have tried reconstruction of the head from the photo with your code, and it doesn't align well with the ground truth point cloud obtained through photogrammetry. I suspect the problem is with the orthogonal projection. So, my question is, can I expect it to work after replacing an ortho projection with the perspective one.
Thanks for the great work!
Hi!
From the code, it is not clear what is the difference between 3d lmks and 2d lmks? Both have 68 points and both have an effective dimension of 3. As I can judge from the code, 3d lmks are only used for visualization. But what is the actual difference between them, and why 2dlmks have the prefix "2d" even if they are actually 3d?
Thanks
thanks for your work!
I want to know why cam[:,0]=5
?
Thanks for your great work!
It can be seen from the log that Adam is very stable, but the convergence time is longer. I tried to torch.optim.lbfgs
to speed up the convergence, but I don’t understand why nan
always appears in training. Can you provide some solutions? Have been troubled for a long time, looking forward to your reply!
Hi,
I looked through the code and tried to comprehend how the dynamic landmarks are calculated. Additionally to the indices and the barycentric coordinates of the dynamic and static landmarks I found full_lmk_faces_idx
and full_lmk_bary_coords
in the code.
What are these landmarks and why are they needed in addition to the dynamic and static landmarks?
Best,
Dominik
Hello, I am a beginner, I want better head reconstruction results, how should I do it, can you give me some detailed suggestions?
Thank you.
The Readme.md has the following text:
"The code is available for non-commercial scientific research purposes."
And the LICENSE file has a MIT license that allows commercial use.
When I am running the demo python photometric_fitting.py 00000 cuda
First problem:
FileNotFoundError: [Errno 2] No such file or directory: './data/generic_model.pkl'
I downloaded flame2023.pkl
and rename to it. I am not sure this is the right thing to do.
Further another problem appeared.
/home/guest/anaconda3/envs/smart_class/lib/python3.10/site-packages/pytorch3d/io/obj_io.py:548: UserWarning: Mtl file does not exist: ./data/template.mtl
warnings.warn(f"Mtl file does not exist: {f}")
making ./test_results//00000
----iter: 0, time: 2023-06-20-20:31:41
landmark: 0.12438790500164032, all_loss: 0.12438790500164032,
Traceback (most recent call last):
File "/data/share/FLAME/Photometric_fitting/photometric_optimization/photometric_fitting.py", line 265, in <module>
fitting.run(imagepath, landmarkpath)
File "/data/share/FLAME/Photometric_fitting/photometric_optimization/photometric_fitting.py", line 218, in run
single_params = self.optimize(images, landmarks, image_masks, savefolder)
File "/data/share/FLAME/Photometric_fitting/photometric_optimization/photometric_fitting.py", line 91, in optimize
util.tensor_vis_landmarks(images[visind], landmarks[visind]))
File "/data/share/FLAME/Photometric_fitting/photometric_optimization/util.py", line 209, in tensor_vis_landmarks
image_landmarks = plot_kpts(image, predicted_landmark, color)
File "/data/share/FLAME/Photometric_fitting/photometric_optimization/util.py", line 250, in plot_kpts
image = cv2.circle(image,(st[0], st[1]), 1, c, 2)
cv2.error: OpenCV(4.7.0) :-1: error: (-5:Bad argument) in function 'circle'
> Overload resolution failed:
> - Can't parse 'center'. Sequence item with index 0 has a wrong type
> - Can't parse 'center'. Sequence item with index 0 has a wrong type
In addition, I have to install different versions of libraries to fit the codes?
Can you please update an library list so we can install the correct versions?
Hi, thanks for sharing this great work!
I have a little question regarding the main fitting code in the following line:
Why hard code cam[:, 0] to 5. ? Is this a default setting in FLAME so that it may benefit the convergence of the fitting process?
Thank you!
Hi! How can I calculate the Head Pose in Euler angles from FLAME pose parameters? Thank you.
Hi
Could you tell me how to get the file "landmark_embedding.npy", especially, 'dynamic_lmk_faces_idx', THX
Nice work! The textures look so good.
Do you have any paper for this work?
Or where can I get more details regarding the pipeline and some evaluation result of the final PCA texture model, like compactness, etc.?
Thanks a lot!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.