baegwangbin/dsine

[CVPR 2024 Oral] Rethinking Inductive Biases for Surface Normal Estimation

dsine's Issues

training data release

Hi, thanks for your great work. I wondered if you plan to open source the training data. Sharing it could benefit the community.

The evaluation accuracy problem on Hypersim

Hi, thanks for releasing such great work! I am testing the provided model on the Hypersim.

However, I found the accuracy is not very good. Therefore, I want to have a verification with you: if the bad results are reasonable or there is something wrong in my processing.

I test on the ai_001_010/cam_00 of the Hypersim, and use the frame.xxxx.normal_cam.png as ground truth.
The accuracy is

total_iter (# 78198048): ai_001_010/cam_00:
mean median rmse 5 7.5 11.25 22.5 30
52.066 41.052 69.400 8.312 14.636 26.022 40.487 43.882

Besides, I don't know which camera coordinate system (opencv/opengl) you use and camera in the Hypersim is the opengl coordinate system ( x-axis points right, the positive y-axis points up, and the positive z-axis points away from where the camera is looking.) Therefore, I convert the GT normal to the opencv-camera coordinate system, where I negate the y-axis and z-axis , and evaluate again. The accuracy becomes worse:

total_iter (# 78198048): ai_001_010/cam_00
mean median rmse 5 7.5 11.25 22.5 30
170.977 177.555 171.943 0.000 0.000 0.000 0.001 0.002

Here is a example contains the ground truth, input color image and the results of DSINE.

P.S. I have provide DSINE with the intrinsic of Hypersim.

Meet error when using torch hub

Meet error when using torch hub:

I think replacing utils.pad_input with utils.get_padding can solve this error.

Training code and larger model

Hi, @baegwangbin

Thanks for your awesome work!
I would like to know will the training code be released and if will you train the model on a larger dataset, e.g. Omnidata v2.

Gradio Demo

Congrats on the release!!

I wanted to see how this performed so I went ahead and quickly built a huggingface space for it. I wanted to make this was okay before I promoted it. I made sure I added a modal so folks can agree to the license. I also made sure to download the model after the fact such that it doesn't show up in the huggingface space files. Let me know if this is okay!

https://huggingface.co/spaces/pablovela5620/DSINE-space

training code release

Hello, thanks for your great work. I'm a student looking into this problem and this repo saved me a month. I wonder if you plan to open source the training code.