ywz978020607/hesic

Official Code of "Deep Homography for Efficient Stereo Image Compression"[cvpr21oral]

CompressAI

paper link

HESIC Project is inherited from https://github.com/InterDigitalInc/CompressAI

Installation：

pip install -e . 
pip install opencv-contrib-python==3.4.2.17 
pip install kornia 
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --set show_channel_urls yes
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
conda install pytorch==1.6.0 torchvision cudatoolkit=10.1

test scripts:

cd /ywz/mywork/

python test3real.py -d "/home/ywz/database/aftercut512" --seed 0 --patch-size 512 512 --batch-size 1 --test-batch-size 1

or

python test3_savereal.py -d "/home/ywz/database/aftercut512" --seed 0 --patch-size 512 512 --batch-size 1 --test-batch-size 1

Errata notes

Recently, when we pushed forward new work, we discovered that there was an error in the code, which led to the wrong results of the conference paper, and we made sufficient improvements and fine-tunings, and attached the final correct results. The reason for the error: We are based on the training and test scripts of the example/train.py internship of the CompressAI framework, but in the 2020.7 version, CompressAI incorrectly used the 'val' variable in the package class of 'AverageMeter'. It was not checked due to the time of submission and our negligence.

At the same time, we have completed the serialization part of the code, and used the left-eye decoded image to re-enter the left-eye code to guide the right-eye entropy model.

（If using the old models and change 'avg' to 'val', the wrong results in the paper could be achieved.）

Result.

datasets:

Pan Baidu :

link：https://pan.baidu.com/s/1sSbMCl-6LXPal_asBt5Giw code：k8rb

Google Drive: link: https://drive.google.com/drive/folders/1tTMs8vf7Z4FAjwCg2aQVGA_pc9O_VpS1?usp=sharing

pretrained_models:

old models:

Pan Baidu :

link：https://pan.baidu.com/s/1q0_2NZ46fYOCeDDg40nUaw code：qrfu

Google Drive: link: https://drive.google.com/drive/folders/1tTMs8vf7Z4FAjwCg2aQVGA_pc9O_VpS1?usp=sharing

new models: we have put the new models now. link: https://bhpan.buaa.edu.cn:443/link/2DFC695B03950A85EF137D8D0FEB62CD 有效期限：2023-04-01 23:59

Serialize

cd ywz/mywork

newnet1.py : HESIC

newnet1_joint.py : HESIC+

test2_codec.py : test script for codec-compress & decompress

​ -- import newnet1 or import newnet1_joint

cd ywz/DSIC

mynet6_plus.py: DSIC with codec

mytrain2_test_codec.py: test script for codec in DSIC

Migration on Mindspore

https://github.com/ywz978020607/2021Summer-Image-Compression

Future and better:

MASIC-link

hesic's Issues

MS-SSIM数据

您好，关注到您更新了实验结果，请问您可以提供下MS-SSIM数据嘛？

alternative mirror

Hey, thanks a lot for the interesting work!

Is it possible to provide an alternative to pan baidu for datasets and pretrained_models?
Without a Chinese mobile phone number, it is impossible for me to create an account and download the data.

train and test?

Hi, thanks for amazing work,

Some questions regarding your work:

1. Would you mind provide an example script for train and test?

2. I browse your "mywork" folder but only find test files and your command mentioned that there is a "newtrain.py" file, so I reckon that you mind missing uploading some files.

3. In the paper, it mentioned that you randomly select the samples from stereo2k and KITTI for training and testing, so would you mind providing your train and test list?

training

Hi,

I am trying to reproduce the results by training my own model. In your codebase, which is the correct loss to use? Is "out_criterion['loss']" in line 196 of test3real.py the complete loss used for training? If you could push your training code that would be even better of course.

Thanks in advance!

using pre-trained models

Hi!
I am trying to use your scripts to reproduce the results in the paper but I am not sure I am doing it right.
In particular, when running the following command (as suggested in the README.md):

python test3real.py -d "/home/ywz/database/aftercut512" --seed 0 --patch-size 512 512 --batch-size 1 --test-batch-size 1

I get an import error due to the missing newnet9 module.

By using the script test3_real.py, instead I am able to load all the models and complete the evaluation.

I wrote a small Colab Notebook so you can check what I have done (I think it may also be useful for others).
https://colab.research.google.com/drive/1kWtOCHsQRd8Ae_tCgVn5XfPyhM-OxppB?usp=sharing

I have some questions:

1. Is the one described above the right procedure for reproducing your best results?
2. The numbers on InStereo2K seems similar to the ones reported in the paper. However I am observing a drop when testing on KITTI. Is this expected?
3. On which resolution should I test KITTI? I can't find it on the paper

I apologize for these multiple questions. Hope you can help!