Visualization of training results

[Ying-yin11]

Visualization of training results

[Ying-yin11]

According to your latest instructions, the scene has been successfully relighted. I am very happy and can't wait to attach the reconstruction results after lighting. Regarding the other matters you mentioned, I haven't made any attempts yet.Next, I will try one by one. Thank you for your help!You are such a nice person！

[tyz1030]

Your feedback is always appreciated!

[tyz1030]

Hi Ying-yin11, SIBR_remoteGaussian_app should work perfectly during training, if it doesn't work let me know. But once finished training, for visualizing a trained model, SIBR_gaussianViewer_app will never work. I have some my own python script hacking through will clean it and release this week.

[tyz1030]

updated. Please see the latest readme.

[Ying-yin11]

Thank you very much for your patient answer! After installing SIBR_VIEWER on the server, I was able to successfully view the results in real-time during the training process. But I have new questions. Why did the result I saw not illuminate the scene under global illumination? Did I miss any steps?How to change lighting to Lambertian to illuminate the entire scene？

Thank you again!

[tyz1030]

Hi, thanks for the feedbacks. You are not missing anything. the dirty trick I did is in darkgs/scene/light.py, uncomment line 129. Maybe in Line 141 you need to multiply the returned value by 1.4 to make it brighter. Also you may want to define a white balance matrix after rendering to correct the color to make it look better. (because I'm using machine vision camera with RAW format so it looks dark and green). Let me know if things works!

But overall these are pretty arbitrary selections, also the example datas I used have very limited FoV coverage, so I'm not releasing it very soon. The difficulties in scaling it up to larger scene is actually the possible future directions.

[Ying-yin11]

Thank you again for your enthusiastic answer.
Unfortunately, according to your prompt, I am unable to brighten up the dark scene. The first image is the visualized training result after I modified lines 129 and 141 in darkgs/scene/light.py. Is there anything else that needs to be modified? I noticed that the 29th line of darkgs/scene/shading. py mentions extended finetuning. When I uncomment the following code, errors related to sigma and mlp parameters will occur. I'm not quite sure if any modifications are needed here?

[tyz1030]

Hi, I think you are almost there!

Please train the model with the code untouched, and when you run viz_chkpt.py, do that trick to line 129 in light.py. That simply works.

The rest optional todos are 1) brighten the image -> 2) white balance -> 3) Gamma correction will give you good visual looking of rendering.

If you train with line 129 uncommented that will totally stop working.

If anything else just let me know!

[tyz1030]

Line 29 stuff in shading.py is very experimental though. If you just want to reproduce my existing results, don't touch it. That's another research direction. Can you optimize all the camera-light parameters together with buidling 3dgs, without pre-calibration with a apriltag target?