toughstonex/u-mvs

Official code for ICCV paper "Digging into Uncertainty in Self-supervised Multi-view Stereo"

Hi there 👋

u-mvs's Issues

each "final3d_model.ply" is only 4k

Hi, when reproducing u_mvs_mvsnet, it is very confusing for me that:
whether I use pretrained model / self_pretrained model,
after run test_pretrain.sh, the final3d_model.ply for every scan are both only 4k

some log is as followings:
`Processing camera 44
Found 0.00 million points
Processing camera 45
Found 0.00 million points
Processing camera 46
Found 0.00 million points
Processing camera 47
Found 0.00 million points
Processing camera 48
Found 0.00 million points
ELAPSED 2.416907 seconds
Error: no kernel image is available for execution on the device
Writing ply file ./outputs/scan118/points_mvsnet//consistencyCheck-20220215-174227//final3d_model.ply
store 3D points to ply file
/home/softwares/anaconda3_gy/lib/python3.8/site-packages/torch/nn/functional.py:3981: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(

Convert mvsnet output to gipuma input
Run depth map fusion & filter
./fusion/fusibile/build/fusibile -input_folder ./outputs/scan11/points_mvsnet/ -p_folder ./outputs/scan11/points_mvsnet/cams/ -images_folder ./outputs/scan11/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.3 --num_consistent=3.0`

the training script of U-MVS (CasMVSNet backbone)

Hello @ToughStoneX , thanks for your amazing work!

When the training script of u_mvs_cascade will be released?

In addition, if you are convenient, could you please give me the model weight file which is only supervised by photometric consistency in CasMVSNet backbone?
The photometric consistency signal without uncertainty estimation:

Upload point cloud results

Hi,

Thanks for your nice work. Could you please upload your point cloud results (DTU & TnT, or DTU only)? It seems the reproduced results are not so good.

Thanks a lot!

About the GPU memory cost in DTU training.

Hi, thanks for the excellent work for unsupervised learning based MVS. I have one simple question since the code is not published.

How is the GPU memory cost in every stage if the two stages are separately trained on DTU_train datasets?

Thanks.

Question about the performance with photometric consistency only

Hi @ToughStoneX
Thanks for your amazing work on unsupervised MVS.
I have one doubt about the performance when the model is trained with photometric consistency (denoted as Lpc in your paper) only.
In JDACS, MVSNet with Lpc achieved 0.7215/0.6339/0.6777 on Acc./Comp./Overall, respectively.
In U-MVS, MVSNet with Lpc achieved 0.5527/0.5345/0.5436 on Acc./Comp./Overall, respectively.

Is there any difference in experimental settings?
Or, the modified photometric consistency Lpc' is adopted in U-MVS instead of the original Lpc ?

Looking forward to your reply.
Many thanks!

Open source time

Hello, I am very interested in such an effective paper that you have done. I saw that you just created this repo. I would like to ask what is the time plan for the open-source code, thank you very much!

Dataset about Forward Flow and Backward Flow

Will the author provide the dataset about 'Forward Flow' and 'Backward Flow'?

I want to try Cross-view Depth-Flow Consistency, but I cannot get the flow map as good as the optical flow map in the paper.

I hope you could respond as soon as possible😊.

Will you release the code of U-MVS-CasMVSNet?

Thanks for your amazing work!
As more memory-friendly and better results are given by CasMVSNet than MVSNet, would you mind releasing the code of U-MVS-CasMVSNet?

acc/com problem

Hi, have you validate the precision of this code/model that supposed to be align with the paper?
I try to reproduce with the setting of mvsnet backbone, using the pretrained.ckpt model + fusible + matlab evaluation code,
but the results is relatively low by a large margin:

| | acc | com | Overall|
|paper|0.4695|0.4308|0.4501|
|reproduce| 0.5728 |0.4978 |0.5353|