Comments (9)
Hi, for 1-2, the rtk_path
are commented out on purpose. If no rtk_path
is found in the config file, banmo will use the camera files in database/DAVIS/Cameras/Full-Resolution/$seqname/%05d.txt
, which should be the path to the auto-generated cam files.
For 3, from my observation, if camera viewpoints are initialized as all identity rotations, the final camera viewpoint will only cover 90-180 over 360 degrees on a circle even after optimization. I don't have numbers but these will look very bad.
For 4, I'm trying to release all of those in the future (possibly after this cvpr deadline). Please send me an email if you need it at an early date.
from banmo.
Thanks for the answer!
I used the latest main to run an optimization on the Eagle dataset without any modification in the code base. However, Eagle's head is missing in the reconstruction. Do you know how to get the same reconstruction results here? I followed the same steps in your instructions for data processing, optimization, and evaluation... Is there anything that I need to change in the code to achieve the same quality as your results?
My results are shown below:
a-eagle-.0.-all.mp4
# Quantitative Results
ave chamfer dis: 17.7 cm
max chamfer dis: 20.8 cm
ave f-score at d=1%: 15.0%
min f-score at d=1%: 8.2%
ave f-score at d=2%: 34.3%
min f-score at d=2%: 20.2%
ave f-score at d=5%: 68.1%
min f-score at d=5%: 51.6%
from banmo.
Hi, the results look really strange. Both the camera pose and deformation seem to be off a lot.
Before I get resource to reproduce it, it would help if you can verify a couple of things. Are you able to get reasonable results for the cat videos? Could you post the results of drawing root pose trajectory here?
from banmo.
I can get reasonable results for the cat videos. This is the result I got after running the command python scripts/visualize/render_root.py --testdir logdir/known-cam-a-eagle-e120-b256-init/ --first_idx 0 --last_idx 120
. Does it look reasonable?
mesh-cam.mp4
from banmo.
I reverted back to this version of the repository and ran Eagle optimization again (Basically without your latest changes on eikonal loss). The results improved but the head of the eagle is still missing and there is still a big gap between this and the result on the paper and website.
a-eagle-.0.-all1.mp4
mesh-cam1.mp4
ave chamfer dis: 10.0 cm
max chamfer dis: 15.9 cm
ave f-score at d=1%: 19.6%
min f-score at d=1%: 9.2%
ave f-score at d=2%: 51.5%
min f-score at d=2%: 28.0%
ave f-score at d=5%: 87.2%
min f-score at d=5%: 71.2%
from banmo.
Interesting that eikonal loss made it worse for eagle.
I also noticed an error in the doc
bash scripts/render_mgpu.sh 0 $seqname logdir/known-cam-$seqname-e120-b256/params_latest.pth \
"0" 256
which should be querying the model after all the training stages (...-ft2)
bash scripts/render_mgpu.sh 0 $seqname logdir/known-cam-$seqname-e120-b256-ft2/params_latest.pth \
"0" 256
But assuming you've already querying the model from known-cam-$seqname-e120-b256-ft2
, the only reason I can think of is about camera pose. Maybe you could try freezing the camera pose in the 1st stage by adding --freeze_root \
before this line?
from banmo.
I guess freezing the camera pose improved the result and now the head is correctly reconstructed. Now the results are close to what you had. By the way, is it possible to sometimes see this kind of artifact happening on canonical shape (for example, the hole on the main body of the eagle)?
from banmo.
It happens in the original paper as well when surface is not well estimated. The motivation of adding eikonal loss is to reduce such artifacts. Perhaps freezing the root in the first stage + eikonal loss will further improve it.
from banmo.
I see. Thanks so much for your help!
Besides the minor typo you mentioned earlier, I think the evaluation command in your doc is missing a -
and ft2
as well. It should be bash scripts/eval/run_eval.sh 0 logdir/known-cam-$seqname-e120-b256-ft2/
.
from banmo.
Related Issues (20)
- Training BANMo using my own videos HOT 6
- SOS, the cat is so cute~~~~~ I cannot focused on codes! o(=•ェ•=)m
- bones_dfm or bones_rst when using gauss_mlp_skinning? HOT 2
- Canonical embeddings matching HOT 6
- Questions on the evaluation on root pose prediction HOT 2
- Question for Nerfies experiment HOT 6
- Question about mesh of Viser in Banmo HOT 2
- Generation of synthetic datasets HOT 3
- No "from pytorch3d import _C"
- Visualize of articulated shape HOT 1
- Optimized models on dropbox HOT 2
- update_delta_rts
- Bone reinitialization HOT 1
- Volume rendering HOT 2
- Question about result : a difference between the results in the paper and my results HOT 6
- Questions by using pre-optimized models HOT 7
- issue of "tmp/nvs-5-0-traj-all.mp4: No such file or directory" when trying pre-optimized models HOT 6
- some problem with colab demo HOT 1
- can banmo be used for 3D reconstruction of birds? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from banmo.