ewrfcas/mvsformer

Codes of MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth (TMLR2023)

MVSFormer

Codes of MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth (TMLR2023)

Arxiv Paper | Project Page

• Releasing codes of training and testing
• Adding dynamic pointcloud fusion for T&T
• Releasing pre-trained models

Installation

git clone https://github.com/ewrfcas/MVSFormer.git
cd MVSFormer
pip install -r requirements.txt

We also highly recommend to install fusibile from (https://github.com/YoYo000/fusibile) for the depth fusion.

git clone https://github.com/YoYo000/fusibile.git
cd fusibile
cmake .
make

Tips: You should revise CUDA_NVCC_FLAGS in CMakeLists.txt according the gpu device you used. We set -gencode arch=compute_70,code=sm_70 instead of -gencode arch=compute_60,code=sm_60 with V100 GPUs. For other GPU types, you can follow

# 1080Ti
-gencode arch=compute_60,code=sm_60

# 2080Ti
-gencode arch=compute_75,code=sm_75

# 3090Ti
-gencode arch=compute_86,code=sm_86

# V100
-gencode arch=compute_70,code=sm_70

More compile relations could be found in here.

Datasets

DTU

1. Download preprocessed poses from DTU training data, and depth from Depths_raw.
2. We also need original rectified images from the official website.
3. DTU testing set can be downloaded from MVSNet.

dtu_training
 ├── Cameras
 ├── Depths
 ├── Depths_raw
 └── DTU_origin/Rectified (downloaded from the official website with origin image size)

BlendedMVS

Download high-resolution images from BlendedMVS

BlendedMVS_raw
 ├── 57f8d9bbe73f6760f10e916a
 .   └── 57f8d9bbe73f6760f10e916a
 .       └── 57f8d9bbe73f6760f10e916a
 .           ├── blended_images
             ├── cams
             └── rendered_depth_maps

Tank-and-Temples (T&T)

Download preprocessed T&T pre-processed by MVSNet. Note that users should use the short depth range of cameras, run the evaluation script to produce the point clouds. Remember to replace the cameras by those in short_range_caemeras_for_mvsnet.zip in the intermediate folder, which is available at short_range_caemeras_for_mvsnet.zip

tankandtemples
 ├── advanced
 │  ├── Auditorium
 │  ├── Ballroom
 │  ├── ...
 │  └── Temple
 └── intermediate
        ├── Family
        ├── Francis
        ├── ...
        ├── Train
        └── short_range_cameras

Training

Pretrained weights

DINO-small (https://github.com/facebookresearch/dino): Weight Link

Twins-small (https://github.com/Meituan-AutoML/Twins): Weight Link

Training MVSFormer (Twins-based) on DTU with 2 32GB V100 GPUs cost 2 days. We set the max epoch=15 in DTU, but it could achieve the best one in epoch=10 in our implementation. You are free to adjust the max epoch, but the learning rate decay may be influenced.

CUDA_VISIBLE_DEVICES=0,1 python train.py --config configs/config_mvsformer.json \
                                         --exp_name MVSFormer \
                                         --data_path ${YOUR_DTU_PATH} \
                                         --DDP

MVSFormer-P (frozen DINO-based).

                                         
CUDA_VISIBLE_DEVICES=0,1 python train.py --config configs/config_mvsformer-p.json \
                                         --exp_name MVSFormer-p \
                                         --data_path ${YOUR_DTU_PATH} \
                                         --DDP

We should finetune our model based on BlendedMVS before the testing on T&T.

CUDA_VISIBLE_DEVICES=0,1 python train.py --config configs/config_mvsformer_blendmvs.json \
                                         --exp_name MVSFormer-blendedmvs \
                                         --data_path ${YOUR_BLENDEMVS_PATH} \
                                         --dtu_model_path ${YOUR_DTU_MODEL_PATH} \
                                         --DDP

Test

Pretrained models: OneDrive

For testing on DTU:

CUDA_VISIBLE_DEVICES=0 python test.py --dataset dtu --batch_size 1 \
                                       --testpath ${dtu_test_path} \
                                       --testlist ./lists/dtu/test.txt \
                                       --resume ${MODEL_WEIGHT_PATH} \
                                       --outdir ${OUTPUT_DIR} \
                                       --fusibile_exe_path ./fusibile/fusibile \
                                       --interval_scale 1.06 --num_view 5 \
                                       --numdepth 192 --max_h 1152 --max_w 1536 --filter_method gipuma \
                                       --disp_threshold 0.1 --num_consistent 2 \
                                       --prob_threshold 0.5,0.5,0.5,0.5 \
                                       --combine_conf --tmps 5.0,5.0,5.0,1.0

For testing on T&T, T&T uses dpcd, whose confidence is controled by conf rather than prob_threshold. Sorry for the confused parameter names, which is the black history of this project. Note that we recommend to use num_view=20 here, but you should build a new pair.txt with 20 views as MVSNet.

CUDA_VISIBLE_DEVICES=0 python test.py --dataset tt --batch_size 1 \
                                      --testpath ${tt_test_path}/intermediate(or advanced) \
                                      --testlist ./lists/tanksandtemples/intermediate.txt(or advanced.txt)
                                      --resume ${MODEL_WEIGHT_PATH} \
                                      --outdir ${OUTPUT_DIR} \ 
                                      --interval_scale 1.0 --num_view 10 --numdepth 256 \
                                      --max_h 1088 --max_w 1920 --filter_method dpcd \
                                      --prob_threshold 0.5,0.5,0.5,0.5 \
                                      --use_short_range --combine_conf --tmps 5.0,5.0,5.0,1.0

Cite

If you found our project helpful, please consider citing:

@article{caomvsformer,
  title={MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth},
  author={Cao, Chenjie and Ren, Xinlin and Fu, Yanwei},
  journal={Transactions of Machine Learning Research},
  year={2023}
}

Our codes are partially based on CDS-MVSNet, DINO, and Twins.

mvsformer's Issues

KeyError: 'optimizer'

when training，is_finetune = True, it appears KeyError: 'optimizer'

does someone know what "DPCD" and "PCD" filters stand for ?

Testing on DTU scan1 & 4

I tested MSVformer on DTU scan 1 and 4 using weights provided on one drive, however it seems that the ply file generated as output is empty (does not contains 3D points)

---

filtering: 100% 49/49 [00:00<00:00, 59.34it/s]
Convert mvsnet output to gipuma input
convert depth maps: 0% 0/49 [00:00<?, ?it/s]1769472

convert depth maps: 2% 1/49 [00:00<00:16, 2.93it/s]1769472

convert depth maps: 4% 2/49 [00:00<00:17, 2.66it/s]1769472

convert depth maps: 6% 3/49 [00:01<00:17, 2.60it/s]1769472

convert depth maps: 8% 4/49 [00:01<00:17, 2.58it/s]1769472

convert depth maps: 10% 5/49 [00:01<00:17, 2.49it/s]1769472

convert depth maps: 12% 6/49 [00:02<00:17, 2.48it/s]1769472

convert depth maps: 14% 7/49 [00:02<00:17, 2.44it/s]1769472

convert depth maps: 16% 8/49 [00:03<00:17, 2.37it/s]1769472

convert depth maps: 18% 9/49 [00:03<00:16, 2.40it/s]1769472

convert depth maps: 20% 10/49 [00:04<00:17, 2.18it/s]1769472

convert depth maps: 22% 11/49 [00:04<00:17, 2.16it/s]1769472

convert depth maps: 24% 12/49 [00:05<00:16, 2.21it/s]1769472

convert depth maps: 27% 13/49 [00:05<00:15, 2.29it/s]1769472

convert depth maps: 29% 14/49 [00:05<00:14, 2.37it/s]1769472

convert depth maps: 31% 15/49 [00:06<00:14, 2.30it/s]1769472

convert depth maps: 33% 16/49 [00:06<00:14, 2.32it/s]1769472

convert depth maps: 35% 17/49 [00:07<00:13, 2.33it/s]1769472

convert depth maps: 37% 18/49 [00:07<00:12, 2.48it/s]1769472

convert depth maps: 39% 19/49 [00:07<00:11, 2.62it/s]1769472

convert depth maps: 41% 20/49 [00:08<00:10, 2.74it/s]1769472

convert depth maps: 43% 21/49 [00:08<00:09, 2.87it/s]1769472

convert depth maps: 45% 22/49 [00:08<00:09, 2.96it/s]1769472

convert depth maps: 47% 23/49 [00:09<00:08, 3.02it/s]1769472

convert depth maps: 49% 24/49 [00:09<00:08, 3.03it/s]1769472

convert depth maps: 51% 25/49 [00:09<00:07, 3.13it/s]1769472

convert depth maps: 53% 26/49 [00:10<00:07, 3.10it/s]1769472

convert depth maps: 55% 27/49 [00:10<00:07, 3.10it/s]1769472

convert depth maps: 57% 28/49 [00:10<00:06, 3.14it/s]1769472

convert depth maps: 59% 29/49 [00:11<00:06, 3.18it/s]1769472

convert depth maps: 61% 30/49 [00:11<00:06, 3.10it/s]1769472

convert depth maps: 63% 31/49 [00:11<00:05, 3.14it/s]1769472

convert depth maps: 65% 32/49 [00:11<00:05, 3.20it/s]1769472

convert depth maps: 67% 33/49 [00:12<00:05, 3.11it/s]1769472

convert depth maps: 69% 34/49 [00:12<00:04, 3.14it/s]1769472

convert depth maps: 71% 35/49 [00:12<00:04, 3.14it/s]1769472

convert depth maps: 73% 36/49 [00:13<00:04, 3.11it/s]1769472

convert depth maps: 76% 37/49 [00:13<00:04, 2.64it/s]1769472

convert depth maps: 78% 38/49 [00:14<00:03, 2.76it/s]1769472

convert depth maps: 80% 39/49 [00:14<00:03, 2.89it/s]1769472

convert depth maps: 82% 40/49 [00:14<00:02, 3.02it/s]1769472

convert depth maps: 84% 41/49 [00:15<00:02, 3.13it/s]1769472

convert depth maps: 86% 42/49 [00:15<00:02, 3.07it/s]1769472

convert depth maps: 88% 43/49 [00:15<00:01, 3.11it/s]1769472

convert depth maps: 90% 44/49 [00:15<00:01, 3.18it/s]1769472

convert depth maps: 92% 45/49 [00:16<00:01, 3.15it/s]1769472

convert depth maps: 94% 46/49 [00:16<00:00, 3.16it/s]1769472

convert depth maps: 96% 47/49 [00:16<00:00, 3.17it/s]1769472

convert depth maps: 98% 48/49 [00:17<00:00, 3.17it/s]1769472

convert depth maps: 100% 49/49 [00:18<00:00, 2.62it/s]
Run depth map fusion & filter
./fusibile/fusibile -input_folder /content/MVSFormer/output/scan1/points_mvsnet/ -p_folder /content/MVSFormer/output/scan1/points_mvsnet/cams/ -images_folder /content/MVSFormer/output/scan1/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.1 --num_consistent=2.0 -color_processing
Command-line parameter error: unknown option -input_folder
input folder is /content/MVSFormer/output/scan1/points_mvsnet/
image folder is /content/MVSFormer/output/scan1/points_mvsnet/images/
p folder is /content/MVSFormer/output/scan1/points_mvsnet/cams/
pmvs folder is
numImages is 49
img_filenames is 49
Device memory used: 2312.110107MB
Device memory used: 2312.110107MB
P folder is /content/MVSFormer/output/scan1/points_mvsnet/cams/
numCameras is 49
Camera size is 49
Accepted intersection angle of central rays is 10.000000 to 30.000000 degrees
Selected views: 49
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48,
Reading normals and depth from disk
Size consideredIds is 49
Reading normal 0
Reading disp 0
Reading normal 1
Reading disp 1
Reading normal 2
Reading disp 2
Reading normal 3
Reading disp 3
Reading normal 4
Reading disp 4
Reading normal 5
Reading disp 5
Reading normal 6
Reading disp 6
Reading normal 7
Reading disp 7
Reading normal 8
Reading disp 8
Reading normal 9
Reading disp 9
Reading normal 10
Reading disp 10
Reading normal 11
Reading disp 11
Reading normal 12
Reading disp 12
Reading normal 13
Reading disp 13
Reading normal 14
Reading disp 14
Reading normal 15
Reading disp 15
Reading normal 16
Reading disp 16
Reading normal 17
Reading disp 17
Reading normal 18
Reading disp 18
Reading normal 19
Reading disp 19
Reading normal 20
Reading disp 20
Reading normal 21
Reading disp 21
Reading normal 22
Reading disp 22
Reading normal 23
Reading disp 23
Reading normal 24
Reading disp 24
Reading normal 25
Reading disp 25
Reading normal 26
Reading disp 26
Reading normal 27
Reading disp 27
Reading normal 28
Reading disp 28
Reading normal 29
Reading disp 29
Reading normal 30
Reading disp 30
Reading normal 31
Reading disp 31
Reading normal 32
Reading disp 32
Reading normal 33
Reading disp 33
Reading normal 34
Reading disp 34
Reading normal 35
Reading disp 35
Reading normal 36
Reading disp 36
Reading normal 37
Reading disp 37
Reading normal 38
Reading disp 38
Reading normal 39
Reading disp 39
Reading normal 40
Reading disp 40
Reading normal 41
Reading disp 41
Reading normal 42
Reading disp 42
Reading normal 43
Reading disp 43
Reading normal 44
Reading disp 44
Reading normal 45
Reading disp 45
Reading normal 46
Reading disp 46
Reading normal 47
Reading disp 47
Reading normal 48
Reading disp 48
Resizing globalstate to 49
Run cuda
Run gipuma
Grid size initrand is grid: 48-36 block: 32-32
Device memory used: 5189.402832MB
Number of iterations is 8
Blocksize is 15x15
Disparity threshold is 0.100000
Normal threshold is 6.283185
Number of consistent points is 2
Cam scale is 1.000000
Fusing points
Processing camera 0
Found 0.00 million points
Processing camera 1
Found 0.00 million points
Processing camera 2
Found 0.00 million points
Processing camera 3
Found 0.00 million points
Processing camera 4
Found 0.00 million points
Processing camera 5
Found 0.00 million points
Processing camera 6
Found 0.00 million points
Processing camera 7
Found 0.00 million points
Processing camera 8
Found 0.00 million points
Processing camera 9
Found 0.00 million points
Processing camera 10
Found 0.00 million points
Processing camera 11
Found 0.00 million points
Processing camera 12
Found 0.00 million points
Processing camera 13
Found 0.00 million points
Processing camera 14
Found 0.00 million points
Processing camera 15
Found 0.00 million points
Processing camera 16
Found 0.00 million points
Processing camera 17
Found 0.00 million points
Processing camera 18
Found 0.00 million points
Processing camera 19
Found 0.00 million points
Processing camera 20
Found 0.00 million points
Processing camera 21
Found 0.00 million points
Processing camera 22
Found 0.00 million points
Processing camera 23
Found 0.00 million points
Processing camera 24
Found 0.00 million points
Processing camera 25
Found 0.00 million points
Processing camera 26
Found 0.00 million points
Processing camera 27
Found 0.00 million points
Processing camera 28
Found 0.00 million points
Processing camera 29
Found 0.00 million points
Processing camera 30
Found 0.00 million points
Processing camera 31
Found 0.00 million points
Processing camera 32
Found 0.00 million points
Processing camera 33
Found 0.00 million points
Processing camera 34
Found 0.00 million points
Processing camera 35
Found 0.00 million points
Processing camera 36
Found 0.00 million points
Processing camera 37
Found 0.00 million points
Processing camera 38
Found 0.00 million points
Processing camera 39
Found 0.00 million points
Processing camera 40
Found 0.00 million points
Processing camera 41
Found 0.00 million points
Processing camera 42
Found 0.00 million points
Processing camera 43
Found 0.00 million points
Processing camera 44
Found 0.00 million points
Processing camera 45
Found 0.00 million points
Processing camera 46
Found 0.00 million points
Processing camera 47
Found 0.00 million points
Processing camera 48
Found 0.00 million points
ELAPSED 1.522794 seconds
Error: no kernel image is available for execution on the device
Writing ply file /content/MVSFormer/output/scan1/points_mvsnet//consistencyCheck-20230529-221442//final3d_model.ply
store 3D points to ply file

Testing with custom data

When testing MVSFormer, I used camera parameters and pair.txt file generated from Colmap on DTU scan_1 images, and the results were good with a full .ply file as shown below:

However, when I generated the camera parameters and pair.txt file on a custom dataset, which consisted of 49 images of an object I took from different angles, the resulting .ply file was empty (0.00 million points are found).

the question is what's the difference between using dtu scan images and a custom data images ?
Can anyone suggest why this might have happened and What steps can I take to troubleshoot the issue? thanks

Tanks&Temples场景调参

您好，方便问下TAT数据集里面inter和advanced的每个场景的阈值参数您这边是如何调整的呢?(主要是光度一致性和几何一致性参数)

about the GPU using

Thanks for your excellent work!
I tried to retrain your model on 2 Rtx8000 without changing any hyper-parameters(batchsize=8), and found it used about 80G GPU memory. Science you trained on 2 V100 GPU with 64G memory, I wonder how much memory cost on your device and why it had more than 10G difference?

it will be appreciated if you could give me some explanation, thank you so much!

blendmvs high_res

你好，我想请问下blendmvs high_res数据集链接好像无法下载，有没有其他方法下载

missing "short_range_cameras" in tt dataset

Hi, great work！Where is "short_range_cameras"? could you share it?

dtu数据集

您好，请问为什么选择用原尺寸（DTU_origin/Rectified）图片呢？（而不用预处理过的图片）
我想在UniMVSNet上用一下Twins来处理FPN提取到的特征，该咋弄呢。谢谢大佬

我想知道测试t&t数据集时出现的深度图异常状况原因，感谢作者

Error when running the DTU Test

*Interval_Scale
1.06
dataset test metas: 1078 interval_scale:{'scan1': 1.06, 'scan4': 1.06, 'scan9': 1.06, 'scan10': 1.06, 'scan11': 1.06, 'scan12': 1.06, 'scan13': 1.06, 'scan15': 1.06, 'scan23': 1.06, 'scan24': 1.06, 'scan29': 1.06, 'scan32': 1.06, 'scan33': 1.06, 'scan34': 1.06, 'scan48': 1.06, 'scan49': 1.06, 'scan62': 1.06, 'scan75': 1.06, 'scan77': 1.06, 'scan110': 1.06, 'scan114': 1.06, 'scan118': 1.06}
drop_path_rate: --- 0.2
Traceback (most recent call last):
File "/mnt/d/s225241/MVSFormer/test.py", line 585, in
save_depth(testlist, config)
File "/mnt/d/s225241/MVSFormer/test.py", line 198, in save_depth
model = TwinMVSNet(config['arch']['args'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/d/s225241/MVSFormer/models/mvsformer_model.py", line 329, in init
self.vit = gvts.alt_gvt_small()
^^^^^^^^^^^^^^^^^^^^
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 552, in init
super(alt_gvt_small, self).init(
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 469, in init
super(ALTGVT, self).init(img_size, patch_size, in_chans, num_classes, embed_dims, num_heads,
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 458, in init
super(PCPVT, self).init(img_size, patch_size, in_chans, num_classes, embed_dims, num_heads,
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 386, in init
super(CPVTV2, self).init(img_size, patch_size, in_chans, num_classes, embed_dims, num_heads, mlp_ratios,
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 276, in init
_block = nn.ModuleList([block_cls(
^^^^^^^^^^^
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 276, in
_block = nn.ModuleList([block_cls(
^^^^^^^^^^
File "/mnt/d/s225241/MVSFormer/models/gvt.py", line 205, in init
super(GroupBlock, self).init(dim, num_heads, mlp_ratio, qkv_bias, qk_scale, drop, attn_drop,
TypeError: Block.init() takes from 3 to 10 positional arguments but 11 were given

When the code will be released

Hello~Thanks for your excellent work！And I wonder when the code will be released. I really want to learn more details from the code.Looking forward to your reply.

DTU_aug_masks

请问这个
File "/root/MVSFormer/datasets/dtu_dataset_ms.py", line 310, in getitem
img_, depth_hr_, intrinsics_, depth_mask_hr_, offset_y, offset_x = self.final_crop(img, depth_hr, intrinsics, depth_mask_hr,
File "/root/MVSFormer/datasets/dtu_dataset_ms.py", line 226, in final_crop
offset_y = random.randint(0, h - crop_h)
File "/root/miniconda3/lib/python3.8/random.py", line 248, in randint
return self.randrange(a, b+1)
File "/root/miniconda3/lib/python3.8/random.py", line 226, in randrange
raise ValueError("empty range for randrange() (%d, %d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0, -558, -558)
这里报错该怎么更改设置呢？

Custom Dataset

hello,

I'm really interested in running your code, but I would like to use my custom dataset. My dataset consists of images from a small room, captured using an iPad. I've already converted the video frames into individual images.

I'm quite new to the field of MVS and find some aspects challenging to grasp. Could you please provide detailed steps on how to run your code with my custom dataset? Any guidance would be greatly appreciated.

thank you!

missing keys when load 'alt_gvt_small.pth'

hi, thanks for your work.

When I load the pretrained Twins weights, I meet the problem missing keys:['norm_list.0.weight', 'norm_list.0.bias', 'norm_list.1.weight', 'norm_list.1.bias', 'norm_list.2.weight', 'norm_list.2.bias', 'norm_list.3.weight', 'norm_list.3.bias']. Is this message normal? And will this error affect the performance?

Looking forward to your reply.

关于T&T advanced 数据集测试的问题

首先感谢作者出色的工作！
我采用了如下命令来测试：
CUDA_VISIBLE_DEVICES=0 python test.py --dataset tt --batch_size 1
--testpath /CFSDATA/advanced
--testlist ./lists/tanksandtemples/advanced.txt
--resume /MVSFormer_try0/weight/best.pth
--outdir /CFSDATA/result_advanced/ \
--interval_scale 1.0 --num_view 10 --numdepth 256
--max_h 512 --max_w 640 --filter_method dpcd
--prob_threshold 0.5,0.5,0.5,0.5
--combine_conf --tmps 5.0,5.0,5.0,1.0
去掉了--use_short_range选项来测试advanced数据集效果，发现advanced数据集生成的.pfm文件转换成图像后的深度效果（有一大部分的深度图都是这样的单值图像）很差：

与此同时，当我生成Temple时无论分辨率是什么都会有以下报错：

感谢任何帮助

自建数据集如何获取depths_raw

我如果是普通相机拍摄的照片，那我该如何获取depths_raw呢，我看dtu数据集本身也未带depths_raw，您是如何操作获取的呢，感谢您的回复

T&T数据集pair.txt视图数量

您好，感谢您开源了代码。我看您将T&T中参考图像提升至20，并在最终结果中也是使用的20参考图像。如果可以的话，能否麻烦您提供一下20视图的pair.txt。十分感谢！

想知道你是如何获取论文中dtu的gt的RGB图像的

作者你好，单卡4090无法训练问题

我将config文件参数设置为如图所示才能运行

然而train到下面这个位置报错

报错信息如下：Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/ch/anaconda3/envs/pt/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
self.run()
File "/home/ch/anaconda3/envs/pt/lib/python3.10/site-packages/tensorboardX/event_file_writer.py", line 202, in run
data = self._queue.get(True, queue_wait_duration)
File "/home/ch/anaconda3/envs/pt/lib/python3.10/multiprocessing/queues.py", line 117, in get
res = self._recv_bytes()
File "/home/ch/anaconda3/envs/pt/lib/python3.10/multiprocessing/connection.py", line 221, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/home/ch/anaconda3/envs/pt/lib/python3.10/multiprocessing/connection.py", line 419, in _recv_bytes
buf = self._recv(4)
File "/home/ch/anaconda3/envs/pt/lib/python3.10/multiprocessing/connection.py", line 388, in _recv
raise EOFError
EOFError
Traceback (most recent call last):
File "/home/ch/sn_d/code/MVS/MVSFormer-main/train.py", line 191, in
mp.spawn(main, nprocs=args.world_size, args=(args, config))
File "/home/ch/anaconda3/envs/pt/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 246, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method="spawn")
File "/home/ch/anaconda3/envs/pt/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 202, in start_processes
while not context.join():
File "/home/ch/anaconda3/envs/pt/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 163, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/home/ch/anaconda3/envs/pt/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 74, in _wrap
fn(i, *args)
File "/home/ch/sn_d/code/MVS/MVSFormer-main/train.py", line 146, in main
trainer.train()
File "/home/ch/sn_d/code/MVS/MVSFormer-main/base/base_trainer.py", line 78, in train
result = self._train_epoch(epoch)
File "/home/ch/sn_d/code/MVS/MVSFormer-main/trainer/mvsformer_trainer.py", line 164, in _train_epoch
self.scaler.step(self.optimizer)
File "/home/ch/anaconda3/envs/pt/lib/python3.10/site-packages/torch/cuda/amp/grad_scaler.py", line 412, in step
assert (
AssertionError: No inf checks were recorded for this optimizer.
请问作者用24G显存的显卡跑过有这样的问题吗？该如何解决
最后 十分感谢您的工作！

What are the thresholds for depth confidence filter?

Thanks for the nice work!

As mentioned in the paper, the depth maps are filtered by certain thresholds.

Specifically, how should I set the prob_threshold parameters in depthfusion.py for the tnt intermediate and advanced dataset to reproduce results in the paper?

Moreover, could you please provide the post-processing steps that prepare the submission to TNT official evaluation?

占用显存问题

您好，我在复现mvsformer中TwinMVSNet模型时，用4张4090运行，在config_mvsformer.json文件中，将batchsize设置为4，也就是每张gpu分得的batch为1。即使这样，每张gpu显存占用也达到了23G【一共24G】。
请问，我如果在mvsformer工作基础上进行改进的话，就会因为显存不足而无法进行训练。请问有什么好的解决办法吗？

T&T advanced测试集问题

您好，非常感谢您出色的工作。您在readme文件中提及到T&T intermediate 需要下载short_range_cameras文件，那么有关于advanced测试集呢，我观察到在general_eval.py中，凡是tt测试集，需要寻找short_range_cameras文件，因此我在test advanced时也遇到了问题：FileNotFoundError: [Errno 2] No such file or directory: 'xxx/tankandtemples/advanced/short_range_cameras/cams_palace/00000000_cam.txt'

关于TT数据集内参问题

作者你好，拜读你的代码发现TT数据集的内参做了四倍缩小处理，而图像未做任何改动，这样做的原因是什么

Problem in depth map fusion & filter step

Hello everyone, hope you can help me!
During the fusion & filter step of testing MVSFormer on the DTU testset, the script tries to load images saved as .png while they are .jpg and then loads 0 points per camera.

This is the output:

filter depth map with probability map
filtering: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 49/49 [00:02<00:00, 20.97it/s]
Convert mvsnet output to gipuma input
convert depth maps: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 49/49 [00:56<00:00, 1.16s/it]
Run depth map fusion & filter
./fusibile/fusibile -input_folder exps/dtu_test/scan1/points_mvsnet/ -p_folder exps/dtu_test/scan1/points_mvsnet/cams/ -images_folder exps/dtu_test/scan1/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.1 --num_consistent=2.0 -color_processing
Command-line parameter error: unknown option -input_folder
input folder is exps/dtu_test/scan1/points_mvsnet/
image folder is exps/dtu_test/scan1/points_mvsnet/images/
p folder is exps/dtu_test/scan1/points_mvsnet/cams/
pmvs folder is
numImages is 49
img_filenames is 49
Device memory used: 954.728455MB
Device memory used: 954.728455MB
P folder is exps/dtu_test/scan1/points_mvsnet/cams/
numCameras is 49
Camera size is 49
Accepted intersection angle of central rays is 10.000000 to 30.000000 degrees
Selected views: 49
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48,
Reading normals and depth from disk
Size consideredIds is 49
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000000.png'): can't open/read file: check file path/integrity
Reading normal 0
Reading disp 0
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000001.png'): can't open/read file: check file path/integrity
Reading normal 1
Reading disp 1
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000002.png'): can't open/read file: check file path/integrity
Reading normal 2
Reading disp 2
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000003.png'): can't open/read file: check file path/integrity
Reading normal 3
Reading disp 3
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000004.png'): can't open/read file: check file path/integrity
Reading normal 4
Reading disp 4
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000005.png'): can't open/read file: check file path/integrity
Reading normal 5
Reading disp 5
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000006.png'): can't open/read file: check file path/integrity
Reading normal 6
Reading disp 6
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000007.png'): can't open/read file: check file path/integrity
Reading normal 7
Reading disp 7
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000008.png'): can't open/read file: check file path/integrity
Reading normal 8
Reading disp 8
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000009.png'): can't open/read file: check file path/integrity
Reading normal 9
Reading disp 9
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000010.png'): can't open/read file: check file path/integrity
Reading normal 10
Reading disp 10
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000011.png'): can't open/read file: check file path/integrity
Reading normal 11
Reading disp 11
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000012.png'): can't open/read file: check file path/integrity
Reading normal 12
Reading disp 12
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000013.png'): can't open/read file: check file path/integrity
Reading normal 13
Reading disp 13
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000014.png'): can't open/read file: check file path/integrity
Reading normal 14
Reading disp 14
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000015.png'): can't open/read file: check file path/integrity
Reading normal 15
Reading disp 15
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000016.png'): can't open/read file: check file path/integrity
Reading normal 16
Reading disp 16
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000017.png'): can't open/read file: check file path/integrity
Reading normal 17
Reading disp 17
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000018.png'): can't open/read file: check file path/integrity
Reading normal 18
Reading disp 18
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000019.png'): can't open/read file: check file path/integrity
Reading normal 19
Reading disp 19
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000020.png'): can't open/read file: check file path/integrity
Reading normal 20
Reading disp 20
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000021.png'): can't open/read file: check file path/integrity
Reading normal 21
Reading disp 21
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000022.png'): can't open/read file: check file path/integrity
Reading normal 22
Reading disp 22
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000023.png'): can't open/read file: check file path/integrity
Reading normal 23
Reading disp 23
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000024.png'): can't open/read file: check file path/integrity
Reading normal 24
Reading disp 24
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000025.png'): can't open/read file: check file path/integrity
Reading normal 25
Reading disp 25
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000026.png'): can't open/read file: check file path/integrity
Reading normal 26
Reading disp 26
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000027.png'): can't open/read file: check file path/integrity
Reading normal 27
Reading disp 27
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000028.png'): can't open/read file: check file path/integrity
Reading normal 28
Reading disp 28
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000029.png'): can't open/read file: check file path/integrity
Reading normal 29
Reading disp 29
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000030.png'): can't open/read file: check file path/integrity
Reading normal 30
Reading disp 30
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000031.png'): can't open/read file: check file path/integrity
Reading normal 31
Reading disp 31
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000032.png'): can't open/read file: check file path/integrity
Reading normal 32
Reading disp 32
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000033.png'): can't open/read file: check file path/integrity
Reading normal 33
Reading disp 33
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000034.png'): can't open/read file: check file path/integrity
Reading normal 34
Reading disp 34
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000035.png'): can't open/read file: check file path/integrity
Reading normal 35
Reading disp 35
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000036.png'): can't open/read file: check file path/integrity
Reading normal 36
Reading disp 36
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000037.png'): can't open/read file: check file path/integrity
Reading normal 37
Reading disp 37
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000038.png'): can't open/read file: check file path/integrity
Reading normal 38
Reading disp 38
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000039.png'): can't open/read file: check file path/integrity
Reading normal 39
Reading disp 39
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000040.png'): can't open/read file: check file path/integrity
Reading normal 40
Reading disp 40
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000041.png'): can't open/read file: check file path/integrity
Reading normal 41
Reading disp 41
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000042.png'): can't open/read file: check file path/integrity
Reading normal 42
Reading disp 42
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000043.png'): can't open/read file: check file path/integrity
Reading normal 43
Reading disp 43
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000044.png'): can't open/read file: check file path/integrity
Reading normal 44
Reading disp 44
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000045.png'): can't open/read file: check file path/integrity
Reading normal 45
Reading disp 45
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000046.png'): can't open/read file: check file path/integrity
Reading normal 46
Reading disp 46
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000047.png'): can't open/read file: check file path/integrity
Reading normal 47
Reading disp 47
[ WARN:[email protected]] global /home/conda/feedstock_root/build_artifacts/libopencv_1658893721168/work/modules/imgcodecs/src/loadsave.cpp (239) findDecoder imread_('exps/dtu_test/scan1/points_mvsnet/images/00000048.png'): can't open/read file: check file path/integrity
Reading normal 48
Reading disp 48
Resizing globalstate to 49
Run cuda
Run gipuma
Grid size initrand is grid: 48-36 block: 32-32
Device memory used: 3729.260498MB
Number of iterations is 8
Blocksize is 15x15
Disparity threshold is 0.100000
Normal threshold is 6.283185
Number of consistent points is 2
Cam scale is 1.000000
Fusing points
Processing camera 0
Found 0.00 million points
Processing camera 1
Found 0.00 million points
Processing camera 2
Found 0.00 million points
Processing camera 3
Found 0.00 million points
Processing camera 4
Found 0.00 million points
Processing camera 5
Found 0.00 million points
Processing camera 6
Found 0.00 million points
Processing camera 7
Found 0.00 million points
Processing camera 8
Found 0.00 million points
Processing camera 9
Found 0.00 million points
Processing camera 10
Found 0.00 million points
Processing camera 11
Found 0.00 million points
Processing camera 12
Found 0.00 million points
Processing camera 13
Found 0.00 million points
Processing camera 14
Found 0.00 million points
Processing camera 15
Found 0.00 million points
Processing camera 16
Found 0.00 million points
Processing camera 17
Found 0.00 million points
Processing camera 18
Found 0.00 million points
Processing camera 19
Found 0.00 million points
Processing camera 20
Found 0.00 million points
Processing camera 21
Found 0.00 million points
Processing camera 22
Found 0.00 million points
Processing camera 23
Found 0.00 million points
Processing camera 24
Found 0.00 million points
Processing camera 25
Found 0.00 million points
Processing camera 26
Found 0.00 million points
Processing camera 27
Found 0.00 million points
Processing camera 28
Found 0.00 million points
Processing camera 29
Found 0.00 million points
Processing camera 30
Found 0.00 million points
Processing camera 31
Found 0.00 million points
Processing camera 32
Found 0.00 million points
Processing camera 33
Found 0.00 million points
Processing camera 34
Found 0.00 million points
Processing camera 35
Found 0.00 million points
Processing camera 36
Found 0.00 million points
Processing camera 37
Found 0.00 million points
Processing camera 38
Found 0.00 million points
Processing camera 39
Found 0.00 million points
Processing camera 40
Found 0.00 million points
Processing camera 41
Found 0.00 million points
Processing camera 42
Found 0.00 million points
Processing camera 43
Found 0.00 million points
Processing camera 44
Found 0.00 million points
Processing camera 45
Found 0.00 million points
Processing camera 46
Found 0.00 million points
Processing camera 47
Found 0.00 million points
Processing camera 48
Found 0.00 million points
ELAPSED 1.082505 seconds
Error: no kernel image is available for execution on the device
Writing ply file exps/dtu_test/scan1/points_mvsnet//consistencyCheck-20240228-171216//final3d_model.ply
store 3D points to ply file

Do gipuma only work in Windows system?

There is an argument called args.fusibile_exe_path, but .exe file only work in Windows system, do gipuma only work in Windows system or both win & linux?

关于DTU测试分辨率问题

您在论文中指出DTU测试时使用的分别率为1152x1536，定量对比的结果为
0.327 0.251 0.289

但使用您的预训练模型测试时，我发现使用896x1152和1152x1536的测试结果分别如下：

请问您是在论文定量对比部分，用的数据是896x1152而非1152x1536分辨率图像的测试结果吗？

您好，我运行了您的demo，但有一个问题想咨询您

您的depths_raw是怎么生成的，如果我使用自建数据集，我如何获取depths_raw

The OneDrive link of pretrained models can not be accessed.

Is the link (https://1drv.ms/u/s!Ah2VkULmkiqPryH_Tl2PUS6Is831?e=BgCuOY) outdated? I can not access it and I tried many proxies.

Could someone help to upload pretrained models to Google drive?

Testing on DTU

I got this error when testing on DTU:

it seems that this issue is resolved in an updated version of timm (0.9.2), but when upgrading it, another error occurs:

Any suggestion on how to fix this?
Thanks

Issue running pre-trained weights

Hi,

Great work and impressive results! I'm trying to run your pretrained weights, and I'm getting the following errors for both the MVSFormer and MVSFormer-Blended weights.

model = TwinMVSNet(config['arch']['args']) File "/source/models/mvsformer_model.py", line 329, in __init__ self.vit = gvts.alt_gvt_small() File "/source/models/gvt.py", line 552, in __init__ super(alt_gvt_small, self).__init__( File "/source/models/gvt.py", line 469, in __init__ super(ALTGVT, self).__init__(img_size, patch_size, in_chans, num_classes, embed_dims, num_heads, File "/source/models/gvt.py", line 458, in __init__ super(PCPVT, self).__init__(img_size, patch_size, in_chans, num_classes, embed_dims, num_heads, File "/source/models/gvt.py", line 386, in __init__ super(CPVTV2, self).__init__(img_size, patch_size, in_chans, num_classes, embed_dims, num_heads, mlp_ratios, File "/source/models/gvt.py", line 276, in __init__ _block = nn.ModuleList([block_cls( File "/source/models/gvt.py", line 276, in <listcomp> _block = nn.ModuleList([block_cls( File "/source/models/gvt.py", line 205, in __init__ super(GroupBlock, self).__init__(dim, num_heads, mlp_ratio, qkv_bias, qk_scale, drop, attn_drop, File "/opt/conda/lib/python3.8/site-packages/timm/models/vision_transformer.py", line 257, in __init__ self.attn = Attention(dim, num_heads=num_heads, qkv_bias=qkv_bias, attn_drop=attn_drop, proj_drop=drop) File "/opt/conda/lib/python3.8/site-packages/timm/models/vision_transformer.py", line 213, in __init__ self.proj_drop = nn.Dropout(proj_drop) File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/dropout.py", line 14, in __init__ if p < 0 or p > 1:

Have you seen this issue? I'm able to use the -P weights, but are they slower than the normal pertained weights?

Thanks,

error code about gvt.py

恭喜，这篇工作给予了我很大的启发，我在尝试复现的Twins的过程中遇到了一些麻烦，作者大大能否给予我一些提示？
具体错误出现在gvt.py文件中 line 204行 groupblock这一段，传参只有十个但是报错说给了11个，对此bug毫无头绪，请作者大大有时间帮我瞅一眼，多谢！是因为self的原因嘛
报错显示为“init() takes from 3 to 10 positional arguments but 11 were given
File "/home/ywang/my_projects/others/CFNet-main/models/gvt.py", line 207, in init
super(GroupBlock, self).init(dim, num_heads, mlp_ratio, qkv_bias, drop, attn_drop, drop_path, act_layer, norm_layer”

Problem with current version of required libraries while execution of files

Hello My dear fellows. I am trying to execute the MVSFormer test file in this project but the problem is that some libraries required versions are not currently available like torch,torchvision. Because of this version not found issue, timm library is producing errors because it is using features of older torch and torchvisio libraries. Here are my enviroment versions: -

pip: - 23.1.2
python: - 3.10.12
cuda: - 11.8
timm: - 0.3.2
torch: - 1.11.0
torchvision: - 0.12.0

Kindly help me with this issue Thanks in advance

License file missing

Hi, thank you for creating great new addition to Deep MVS world and providing pretrained weights.
I would like to test your network, but I need to know what a license is for using or modifying your code.
Thank you for answer!

关于Tanks & Temples测试过程的视图数量

您好，我看到论文在Tanks & Temples测试过程中将视图数量提升到20，并且取得了不错的效果。但是，在pair.txt文件中只有10个候选视图，请问您是如何选择20个视图的呢？谢谢！

MVSFormer-H 测试

您好，我这边在测试MVSFormer-H 这块时，发现如下问题，请问可以帮忙看一下吗？

Lots of noise in output ply in tnt dataset

Thanks for the author's great work.
Following the tnt test instruction, Lots of noise in the output ply

bash:
CUDA_VISIBLE_DEVICES=0 python test.py --dataset tt --batch_size 1 --testpath /data/tankandtemples/intermediate/
--testlist ./lists/tanksandtemples/intermediate.txt
--resume "/home/methods/MVSFormer/pretrained_weights/MVSFormer/best.pth"
--outdir /data4/trained_models/MVSFormer_tnt_nview10/
--interval_scale 1.0 --num_view 10 --numdepth 256 --max_h 1088 --max_w 1920
--filter_method dpcd --prob_threshold 0.5,0.5,0.5,0.5
--use_short_range --combine_conf --tmps 5.0,5.0,5.0,1.0
Do the author has any idea to solve the problem?