Comments (8)
Hi, I have encountered NaN during training in the following situations, and can be avoided by using corresponding methods:
- When using pytorch 1.7. or 1.8, there may be NaN bugs. Using 1.9.0 or 1.6.0 always solves the problem.
- Change this line to: std = torch.clamp_min(var, 1e-8).sqrt()
ENeRF/lib/networks/enerf/utils.py
Line 495 in 942ecdf
from enerf.
Solved! Thank you for your advice!
from enerf.
Hi, I met another problem during training.
- It seems like that the network can not converge.
At the beginning of training, the PSNR of training set is only about 10 and ends up at about 11 after epoch 0. When it comes to the zjumocap data, the PSNR is more than 20 at first and increases fast to about 25 after epoch 0. - The PSNR of test set is much higher than training set.
After several epochs, the PSNR of training set reamins 11-12, but the PSNR of test set can increases to about 22.
I wonder if I should adjust the config or there is something wrong with my dataset (I follow the tutorial of Easymocap). Thank you for your help!
from enerf.
Hi, what kind of data did you use? Is it similiar to ZJU-Mocap (Multi-view dynamic dataset, including mask information)?
Could you provide more information so I can locate the problem?
from enerf.
I use 4 cameras to record and extract images (1280x720, padding to 1280x1280 and resize to 1024x1024).
I also use a segmentation model to get the mask information.
I set the train input view in the yaml and the length of source view in enerf.py to avoid out of range.
At first I think the calibration from Easymocap is not correct. But I use Matlab to double check it and the two results are close.
from enerf.
Try to use more source views, (one input view is not enough).
train_input_views: [3,4] train_input_views_prob: [0.5, 0.5] test_input_views: 3(or 4)
I have some other suggestions:
- Try rendering a test image from the remaining 3 images instead of using the GUI to render a novel view. This helps avoid extrapolation problems, which image-based rendering is not good at.
- Use 4 views as training views: Change these two lines to [0,-1,1] https://github.com/zju3dv/ENeRF/blob/master/configs/enerf/zjumocap/zjumocap_train.yaml#L28
from enerf.
I followed your advice and tried, but the result seems the same. Both the PSNR and the visualization are still bad.
-
Actually, when I use the same yaml for training, with zjumocap (only use 4 cameras data) and my dataset. The result of zjumocap seems good. Only after epoch 0, PSNR of zjumocap (4 cameras) can raise from 20 to 25. For my dataset, it only begins with 10 and ends up with 13. The initial network can get such results with different data and same training strategy. So it has to be my data or my calibration that cause the difference, right?
-
I also checked the depth_range calculated by the 3D bbox. It usually ranges from 2 to 4.5 in CoreView313. But for my dataset it ranges from 0.1 to 0.5, and sometimes the value is negative. Is that a problem?
Thank you so much for your help these days!
from enerf.
As far as I know, it is the normalization of the extrinsics that cause the problem.
After the modification, everything seems good.
And I deleted the images above for privacy.
Thank you for your help and your great work!
from enerf.
Related Issues (20)
- custom outdoor dataset
- Issues on evaluation HOT 4
- Doubt with homo_warp
- How to make bounding box data? HOT 1
- Strange error while funetunning zjumocap HOT 1
- Super low GPU/CPU usage while training HOT 1
- Error in DTU Eval HOT 1
- Composition of the enerf-outdoor dataset HOT 1
- Run Enerf on a self-made zju-mocap HOT 4
- run gui_human.py on my own dataset. Problem with visualization
- About the camera? HOT 2
- Camera Color Calibration
- Properly formatted annots.npy file HOT 1
- 关于zjumocap_train.yaml文件下某些项的作用 HOT 1
- Question about config file HOT 3
- weird result after visualizing on the ENeRF-Outdoor dataset
- Problems with visualizing outdoor video HOT 1
- Two questions regarding the video rendering accuracy and the number of inference cameras for Enerf Outdoor. HOT 1
- Is camera radial distortion taken into account? HOT 1
- The Link of pretrained model from dtu_pretrain has expired HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from enerf.