jiarenchang/psmnet

Pyramid Stereo Matching Network (CVPR2018)

psmnet's Issues

About the difference of map size between groud truth and the estimated

Hi, The size of input image and the ground truth disparity are both 1242x375, but the size of estimated disparity is 1248x384. So should I do the padding for ground truth or drop some elements of the estimation in order to calculate the error map, what's the specific rule for this ?

Unable to extract from tar file of the trained model.

I am trying to get the final model finetuned on KITTI 2015. I downloaded the tar file (21MB) from the Google drive link but am unable to extract it. It just says "An error has occured. The files cannot be extracted." Did anyone face this problem?

AttributeError: 'module' object has no attribute 'no_grad'

error info:"File "submission.py", line 79, in test with torch.no_grad(): AttributeError: 'module' object has no attribute 'no_grad'" I am a newer for PyTorch, and i have no idea about the error. The versions of PyTorch i use are gpu-0.1.12 and cpu-0.3.0 @JiaRenChang

I encounter some question when I delete " with no_grad"

I try to finetune the PSMNet on KITTI 2015,but I find it will be "out of memory". So I decrease the "batch_size" from 12 to 2. And as you say, I delete the sentence " with torch.no_grad" and set the variables imgL and imgR volatile = True. Then I encounter a new problem " TypeError: transpose received an invalid combination of arguments - got (NoneType), but expected (int dim0, int dim1) ". How can I solve this problem? My GPU is GTX1070 8G ,why do low memory problems occur?

How much memory is required to train the network?

I used two 8GB GTX1080 to train, but still got out of memory error. How much memory is at least required to train the network?
Thanks.

Question about the result

Hello,
I executed 'submission.py' with 'pretrained_model_KITTI2015.tar'. but I got result below that isn't much like the result you posted. Did I something wrong?

the pre-train model is corrupted

Bad generalization ability on testing dataset?

Hi guys,

I am getting really good results on KITTI 2015(images are all in gray) testing set which looks like below:

later I decided to test on other stereo datasets and got results like what follows below:

Is there any way to improve the result?
Looking forward to any suggestion.
Thanks

3px error evaluation

In your finetune.py, when you calculate 3px error you try to find correct pixels and not outliers. I didn't get how thats an error

the dilation parameters

hi, thanks for your awesome job. I have a question: the parameters in the feature_extraction (layer3 and layer4), the dilation is 1 and 2, while in the paper is 2 and 4. So which one is right? Thank you !

Can this work on Windows platform?

I use torch0.4.0+torchvision0.2.0+python3.6 to run this code, but it doesn't work. And using torch0.4.0 , the some function is wrong such as torch.randperm and torch.cat @JiaRenChang

Can you give more details about the training dataset ?

Hello,
Can you give more details about the training dataset ?

FlyingThings3D test set

Hello,

In the FlyingThing3D dataset there are several stereo pairs with a very large disparities (10^3) and negative disparities. Do you delete such images during training and testing? Thank you in advance!

Unable to download pretrained model

When extracting the downloaded model, it seems the .tar file is broken.

tar xvf pretrained_model_KITTI2015.tar
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

About finetune

I think, in finetune.py
Line 150： max_acc=0 ---> min_err=100
Line 178/179: if total_test_loss/len(TestImgLoader)*100 > max_acc:
max_acc = total_test_loss/len(TestImgLoader)*100 --->
if total_test_loss/len(TestImgLoader)*100 < min_err:
min_err = total_test_loss/len(TestImgLoader)*100
Line 194: print(max_acc) ---> print(min_err)
I'm not sure if it is right ? Thank you !

Parameters number

Hi JiaRen,
Could you possibly provide parameters number for your network?

the pretrained model

hi,the pretrained model i download from goole is only 20MB, is that right ?
but i can unrar it correctly!
looking forward your answer!

regarding image normalization

First of all, congrats for this excellent work! Coming to my question, I think I never saw other papers explicitly mentioning image normalization for disparity estimation, do you have any numbers about how much image normalization improved your results? Secondly, why not use mean / stddev values from Scene Flow dataset instead of ImageNet? Thanks!

Question about the pretrained model

@JiaRenChang Hello, thanks for your wonderful work!
I am doing some experiments based on your work. But I am confused about the pretrained model you provide. Which part of the KITTI 2015 dataset is used for training the PSMNet model you provide? The whole 200 training images or 160 training images you split?
If the pretrained model using the whole 200 training images, would you please provide the model training on the 160 images? Or could you provide the PSMNet model pretrained on Scene Flow dataset?
Any help would be appreciated.

Would you please provides the PSMNet model pertrained on Scene Flow

Hello, Jia-Ren Chang.

I want to conduct some experiments based on your work.
However, I have trouble in reproducing your experiment result on KITTI Stereo Evaluation 2015 due to the lack of PSMNet model pertrained on Scene Flow dataset.
Would you please provides the PSMNet model pertrained on Scene Flow for me?

Thank you very much!

Softmax or Softmin for disparity regression

Thanks for sharing your code!!

I noticed that in your paper you mentioned that your softmin (i.e., softmax(-cost)) whereas in your code you use softmax (i.e., softmax(cost)). Am I missing anything in your code? Shouldn't there be a "minus" before cost?

How did you split the training/validation sets?

Can you share which images are used for validation? Thanks

About the crop size 544*960

Hello. Thank you for the paper and code. I an confused about the dataloader for sceneflow dataset. As the image size of the dataset is 540940, however, when in test mode, the code crop the image to 544960..

else:
           w, h = left_img.size
           left_img = left_img.crop((w-960, h-544, w, h))
           right_img = right_img.crop((w-960, h-544, w, h))
           processed = preprocess.get_transform(augment=False)  
           left_img       = processed(left_img)
           right_img      = processed(right_img)

           return left_img, right_img, dataL

Could you explain why it is doing for ? Thanks!

Some question about scene flow dataset

Hello!Thank you for sharing! I have some question about the scene flow dataset.
Do I need to download a complete dataset? I find the size of dataset is more than 100G.
I choose to download "Sample Pack" to test your code, but I find the path is error.
my path is './SceneFlowData/monkaa_frames_cleanpass(or disparity)/left(or right)'.
And I cannot find the folder named "TRAIN" in the "SceneFlow/frames_cleanpass".Maybe my dataset is wrong.Can you tell me the list of your folders? Or can you give me a link to download the right dataset?

Some problems while training on CPU

Thanks for sharing your code on github. I ran into some problems while running your code on my laptop, am i able to run finetune.py or submission.py without CUDA installed? While running submission.py it shows AssertionError: Torch not compiled with CUDA enabled. Besides, does finetune.py use the 'loadmodel' from the google drive download from your link? I am not able to type the right path of the model, anyone else face the same problem?

The pretrained model from google drive is not working?

Can this work with python 3.6?

3 pixels error on Scene Flow

When reading you paper I noticed that you don't mention 3 pixels error on Scene Flow dataset, only end point error. Is it comparable to error of other methods like CRL and GC?

Question About Result

@JiaRenChang Hi, The model "pretrained_model_KITTI2015.tar" I download from your link. It is trained on SenceFlowData with a constant learning rate of 0.001 for 10 epochs firstly , and then finetune it on KITTI 2015 training dataset for 300 epochs . And the learning rate of this fine-tuning began at 0.001 for the first 200 epochs and 0.0001 for the remaining 100 epochs. right? Because I trained the model as your paper said with your code, I cannot get the same result as "pretrained_model_KITTI2015.tar". It is worse than yours, Thank you very much!!!

problem about the reslut

hello,

1. do you test the result of model only trained the scene flow on the kitti2015-all-traing-image? the result is?
   2.when you finetune on the kitti2015-traing, the result of model on the kitti2015-all-traing-image is?

my problom is when I finetune on the kitti-2015-traing, then test the result on the kitti2015-all-traing-image, the relslut has little change after 30epoch， now the result of err_rate is about 3.4%? do you have other special trainging method? I am chinese, we can chat in weixin if you want, thank you!
Please! I am troubled in there, the performance is pool.

About use GPU id

Hello,when I training ,I meet two problems:
1.Whether I train the model or test , must use gpu_id 0,“ model= nn.DataParallel（model,device_ids=[0,1]）”Otherwise cannot runing .
2. I training using two gups, but my batchsize only set 2,cannot set 4 or 8,if so ,always reminding me out of memory .Does someone meet this?

How many GPU needed when test on KITTI2015 and train on SceneFlowData?

Hi, thanks for share. I wanted to test the model using submission.py on KITTI2015, but encountered out of memory. I want to know how many GPU I need when training on SceneFlowData and testing on KITTI2015.

Thanks!

about the model and result on the scene flow dataset

您好，我发现你release的模型结构不是你论文中最好的模型结构？请问是这样吗？具体有什么不同啊？

我按照你的说明，下载了数据集，然后在scene folw上面直接跑（从头开始训练），我训练了10个epoch,loss（我理解的就是EPE）在测试集是3左右。这个EPE是我直接通过跑main.py那个代码电脑上显示的结果，没有用其他代码测试，请问你看这个EPE也是通过这种方式把？
此时我的环境是只用了一块GPU卡，bathc size是3，然后测试的时候是2。
请问是我哪里搞错了吗？麻烦您回复

can I change the batchsize?

pretrained model on scene flow dataset

Hi @JiaRenChang ,
Thanks for your excellent work. Could you please release your pretrained model on scene flow dataset?

Evaluation in finetune.py

Hi JiaRen,

I have read your code, but I am confused about the total_test_loss in finetune.py. When you get the mean 3-pixel error, you use total_test_loss / len(TestImgLoader) * 100. But I think your total_test_loss have divided batch_size. So I think your final mean error is total test loss / (batch size * len(TestImgLoader)).

regarding smooth l1 loss function

In the paper, you mentioned that you use smooth L1 loss instead of L2, did you experiment with the standard L1 loss? Did it result in blurry disparity image? Thanks!

/SenceFlowData/

Hi, I want to ask the list of SenceFlowData.
First, "SenceFlowData";
Second, "Driving" and "FlyingThings3D" and "Monkaa";
Third, for example, the list are "frames_cleanpass" and "disparity" in Driving?
Thanks a lot.

Test the KITTI 2015 dataset with low GPU or CPU

Thanks for your working. Sorry to bother you. I just want to test one image pairs of KITTI 2015 with your pre-trained model.
python submission.py --datapath /media/jennifer/Papers/1_StereoData/KITTI2015_data_scene_flow/testing/ --loadmodel pretrained_model_KITTI2015.tar
While the process is out of memory. I just change the image size in submission.py, I also get some errors.
I choose no-cuda, there are also some errors.
Can I test kitti 2015 dataset with low GPU or CPU?
Thanks.

Augmentation

Hi, thank you for very impressive work!
Do you use augmentation (color, geometry) when training on KITTI? I see some augmentation code, but you did not mentioned that you use any augmentation in the paper.

Weight Decay

Just a question if really weight decay is put to zero in optimization?

About the pretrained_model_KITTI2015.tar

Hi JiaRen,

Did you use the pretrained_model_KITTI2015.tar (you uploaded) for your paper?
Can I get reliable results with the pre-trained model?
I get unreliable results, so I just want to check that it is my fault on the running code or not.

Thank you

AttributeError: 'module' object has no attribute 'no_grad'

Hi, I installed PyTorch according to the official website, PyTorch's version is 0.3.1，but I encountered this problem:
"
with torch.no_grad():
AttributeError: 'module' object has no attribute 'no_grad'
“
Can you tell me how to solve it? Please ,Thank you very much！

Will a small batchsize reduce the accuracy?

I only have one GPU, so batchsize is set to 1. Does this have a serious impact on the result?

dataset query

Hello, I'm new to this area, so I'm not too familiar with the data. I would like to ask which data I should download (can you give me a specific link?), I downloaded a data set (http://www.cvlibs.net/download.php?file=data_scene_flow.zip), however, it is different from the file directory required in your code. thank you very much.

How can I test my own dataset?

I tried test some of my own stereo images.
I got error like ValueError: cannot reshape array of size 1863000 into shape (1,3,375,1242).

What should i do?
Is that a easy way to test a pair of stereo images?

question about evaluate code

Hi , Could you please provide the evaluate code? I use the "development kit" code (matlab version) from KITTI to evaluate the disparity result that is based on your model("pretrained_model_KITTI2015.tar"). I can not get the same error rate as the Table 2 (KITTI 2015 Val Err is 1.83%). Instead of the 1.83%, I get about 0.70% , I can not find the reason. so can you help me or provide the evaluate code? Thanks a lot!
And my val images are:[10,11,14,20,24,38,41,48,50,54,59,62,68,69,74,77,82,89,92,94,99,107,108,109,120,125,132,145,146,154,161,166,173,174,175,182,185,188,189,191], but it should have nothing to do with the error rate.

question about size

hello，is this a cvpr2018 poster paper or arxiv paper?
Besides, when you test, your input is not crop(256512), is 3841248. If it will be out of memory? if my code will be out of memory when 384*1248, how can I test?
thank you very much.

Some puzzles on the cost volume

for i in range(self.maxdisp/4):
            if i > 0 :
             cost[:, :refimg_fea.size()[1], i, :,i:]   = refimg_fea[:,:,:,i:]
             cost[:, refimg_fea.size()[1]:, i, :,i:] = targetimg_fea[:,:,:,:-i]
            else:
             cost[:, :refimg_fea.size()[1], i, :,:]   = refimg_fea
             cost[:, refimg_fea.size()[1]:, i, :,:]   = targetimg_fea

This piece of codes has puzzled me. I am wondering why there would be negtive numbers, if using range() ,should all the iteration be positive?
Besides, I am also wondering the way to concat two feature maps. How to understand the last dimension where in code "if i > 0", how to understand the index i: and :-i？

cuda problem when training the model

Hi, JiaRenChange!
When I'm going to train the model, the problem go like this:
File "/home/work/hejinying/local/lib/python2.7/site-packages/torch/nn/parallel/parallel_apply.py", line 67, in parallel_apply
raise output
RuntimeError: cuda runtime error (77) : an illegal memory access was encountered at /pytorch/torch/lib/THC/generic/THCTensorCopy.c:21

I have 8GPUs, Tesla K80.
I use batch_size the same as you train.
Do you know the problem?