bupt-ai-cz/llvip

LLVIP: A Visible-infrared Paired Dataset for Low-light Vision

LLVIP: A Visible-infrared Paired Dataset for Low-light Vision

Project | Arxiv | Kaggle | |

News

• ⚡(2024-1-8): The pre-trained pix2pixGAN on LLVIP is released Here.
• ⚡(2023-2-21): The annotations of a small part of images have been corrected and updated, including the annotation of some missing pedestrians, and the optimization of some imprecise annotations. The updated dataset is now available from the homepage or here. If you need the previous version of the annotations, please refer to here.
• ⚡(2022-5-24): We provide a toolbox for various format conversions (xml to yolov5, xml to yolov3, xml to coco)
• ⚡(2022-3-27): We released some raw data (unregistered image pairs and videos) for further research including image registration. Please visit homepage to get the update. (2022-3-28 We have updated the link of Baidu Yun of LLVIP raw data, the data downloaded from the new link supports decompression under windows and macos. The original link only support windows.)
• ⚡(2021-12-25): We released a Kaggle Community Competition "Find Person in the Dark!" based on part of LLVIP dataset. Welcome playing and having fun! Attention: only the visible-image data we uploaded in Kaggle platform is allowed to use (the infrared images in LLVIP or other external data are forbidden)
• ⚡(2021-11-24): Pedestrian detection models were released
• ⚡(2021-09-01): We have released the dataset, please visit homepage or here to get the dataset. (Note that we removed some low-quality images from the original dataset, and for this version there are 30976 images.)

---

---

Dataset Downloading:

Homepage or DatasetDownload

---

Citation

If you use this data for your research, please cite our paper LLVIP: A Visible-infrared Paired Dataset for Low-light Vision:

@inproceedings{jia2021llvip,
  title={LLVIP: A visible-infrared paired dataset for low-light vision},
  author={Jia, Xinyu and Zhu, Chuang and Li, Minzhen and Tang, Wenqi and Zhou, Wenli},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={3496--3504},
  year={2021}
}

or

@misc{https://doi.org/10.48550/arxiv.2108.10831,
  doi = {10.48550/ARXIV.2108.10831}, 
  url = {https://arxiv.org/abs/2108.10831},
  author = {Jia, Xinyu and Zhu, Chuang and Li, Minzhen and Tang, Wenqi and Liu, Shengjie and Zhou, Wenli}, 
  keywords = {Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {LLVIP: A Visible-infrared Paired Dataset for Low-light Vision},
  publisher = {arXiv},
  year = {2021},
  copyright = {arXiv.org perpetual, non-exclusive license}
}

Image Fusion

Baselines

• GTF
• FusionGAN
• Densefuse
• IFCNN

FusionGAN

Preparation

• Install requirements

  git clone https://github.com/bupt-ai-cz/LLVIP.git
  cd LLVIP/FusionGAN
  # Create your virtual environment using anaconda
  conda create -n FusionGAN python=3.7
  conda activate FusionGAN
  
  conda install matplotlib scipy==1.2.1 tensorflow-gpu==1.14.0 
  pip install opencv-python
  sudo apt install libgl1-mesa-glx

• File structure

  FusionGAN
  ├── ...
  ├── Test_LLVIP_ir
  |   ├── 190001.jpg
  |   ├── 190002.jpg
  |   └── ...
  ├── Test_LLVIP_vi
  |   ├── 190001.jpg
  |   ├── 190002.jpg
  |   └── ...
  ├── Train_LLVIP_ir
  |   ├── 010001.jpg
  |   ├── 010002.jpg
  |   └── ...
  └── Train_LLVIP_vi
      ├── 010001.jpg
      ├── 010002.jpg
      └── ...
  

Train

python main.py --epoch 10 --batch_size 32

See more training options in main.py.

Test

python test_one_image.py

Remember to put pretrained model in your checkpoint folder and change corresponding model name in test_one_image.py. To acquire complete LLVIP dataset, please visit https://bupt-ai-cz.github.io/LLVIP/.

Densefuse

Preparation

• Install requirements

  git clone https://github.com/bupt-ai-cz/LLVIP
  cd LLVIP/imagefusion_densefuse
  
  # Create your virtual environment using anaconda
  conda create -n Densefuse python=3.7
  conda activate Densefuse
  
  conda install scikit-image scipy==1.2.1 tensorflow-gpu==1.14.0

• File structure

  imagefusion_densefuse
  ├── ...
  ├──datasets
  |  ├──010001_ir.jpg
  |  ├──010001_vi.jpg
  |  └── ...
  ├──test
  |  ├──190001_ir.jpg
  |  ├──190001_vi.jpg
  |  └── ...
  └──LLVIP
     ├── infrared
     |   ├──train
     |   |  ├── 010001.jpg
     |   |  ├── 010002.jpg
     |   |  └── ...
     |   └──test
     |      ├── 190001.jpg
     |      ├── 190002.jpg
     |      └── ...
     └── visible
         ├──train
         |   ├── 010001.jpg
         |   ├── 010002.jpg
         |   └── ...
         └── test
             ├── 190001.jpg
             ├── 190002.jpg
             └── ...
  

Train & Test

python main.py 

Check and modify training/testing options in main.py. Before training/testing, you need to rename the images in LLVIP dataset and put them in the designated folder. We have provided a script named rename.py to rename the images and save them in the datasets or test folder. Checkpoints are saved in ./models/densefuse_gray/. To acquire complete LLVIP dataset, please visit https://bupt-ai-cz.github.io/LLVIP/.

IFCNN

Please visit https://github.com/uzeful/IFCNN.

Pedestrian Detection

Baselines

• Yolov5
• Yolov3

Yolov5

Preparation

Linux and Python>=3.6.0

• Install requirements

  git clone https://github.com/bupt-ai-cz/LLVIP.git
  cd LLVIP/yolov5
  pip install -r requirements.txt

• File structure

  The training set of LLVIP is used for training the yolov5 model and the testing set of LLVIP is used for the validation of the yolov5 model.

  yolov5
  ├── ...
  └──LLVIP
     ├── labels
     |   ├──train
     |   |  ├── 010001.txt
     |   |  ├── 010002.txt
     |   |  └── ...
     |   └──val
     |      ├── 190001.txt
     |      ├── 190002.txt
     |      └── ...
     └── images
         ├──train
         |   ├── 010001.jpg
         |   ├── 010002.jpg
         |   └── ...
         └── val
             ├── 190001.jpg
             ├── 190002.jpg
             └── ...
  

  We provide a toolbox for converting annotation files to txt files in yolov5 format.

Train

python train.py --img 1280 --batch 8 --epochs 200 --data LLVIP.yaml --weights yolov5l.pt --name LLVIP_export

See more training options in train.py. The pretrained model yolov5l.pt can be downloaded from here. The trained model will be saved in ./runs/train/LLVIP_export/weights folder.

Test

python val.py --data --img 1280 --weights last.pt --data LLVIP.yaml

Remember to put the trained model in the same folder as val.py.

Our trained model can be downloaded from here: Google-Drive-Yolov5-model or BaiduYun-Yolov5-model (code: qepr)

• Click Here for the tutorial of Yolov3 （Our trained Yolov3 model can be downloaded from here: Google-Drive-Yolov3-model or BaiduYun-Yolov3-model (code: ine5)）.

Results

We retrained and tested Yolov5l and Yolov3 on the updated dataset (30976 images).

Where AP means the average of AP at IoU threshold of 0.5 to 0.95, with an interval of 0.05.

The figure above shows the change of AP under different IoU thresholds. When the IoU threshold is higher than 0.7, the AP value drops rapidly. Besides, the infrared image highlights pedestrains and achieves a better effect than the visible image in the detection task, which not only proves the necessity of infrared images but also indicates that the performance of visible-image pedestrian detection algorithm is not good enough under low-light conditions.

We also calculated log average miss rate based on the test results and drew the miss rate-FPPI curve.

Image-to-Image Translation

Baseline

• pix2pixGAN

pix2pixGAN

Preparation

• Install requirements

  cd pix2pixGAN
  pip install -r requirements.txt

• Prepare dataset
• File structure

  pix2pixGAN
  ├── ...
  └──datasets
     ├── ...
     └──LLVIP
        ├── train
        |   ├── 010001.jpg
        |   ├── 010002.jpg
        |   ├── 010003.jpg
        |   └── ...
        └── test
            ├── 190001.jpg
            ├── 190002.jpg
            ├── 190003.jpg
            └── ...
  

Train

python train.py --dataroot ./datasets/LLVIP --name LLVIP --model pix2pix --direction AtoB --batch_size 8 --preprocess scale_width_and_crop --load_size 320 --crop_size 256 --gpu_ids 0 --n_epochs 100 --n_epochs_decay 100

Test

python test.py --dataroot ./datasets/LLVIP --name LLVIP --model pix2pix --direction AtoB --gpu_ids 0 --preprocess scale_width_and_crop --load_size 320 --crop_size 256

See ./pix2pixGAN/options for more train and test options.

Results

We retrained and tested pix2pixGAN on the updated dataset(30976 images). The structure of generator is unet256, and the structure of discriminator is the basic PatchGAN as default.

License

This LLVIP Dataset is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications, or personal experimentation. Permission is granted to use the data given that you agree to our license terms.

Call For Contributions

Welcome to point out errors in data annotation. If you want to modify the label, please refer to the annotation tutorial, and email us the corrected label file.

More annotation forms are also welcome (such as segmentation), please contact us.

Acknowledgments

Thanks XueZ-phd for his contribution to LLVIP dataset. He corrected the imperfect annotations in the dataset.

Contact

email: [email protected], [email protected], [email protected], [email protected]

llvip's Issues

关于yolov5测试集

您好，请问项目中yolov5_infrared.pt和yolov5_visible.pt都是用train数据集训练，test数据集验证的么？测试阶段仍然是使用test数据集做测试么？

测试数据集的问题

你好 我通过你们官网下载的数据集 test数据集里面是没有，而我看你们yolov5实验里面是由test的xml文件的

Number of image pairs for CVC-14 dataset in paper

Hi maintainer:

Thanks for providing this awesome dataset. It is amazing.
I try to summarise popular datasets in this domain, and I found that the number of image pairs in CVC-14 dataset should be 8490, instead of 849 in Table 1.

I download the CVC-14 dataset from (http://adas.cvc.uab.es/elektra/enigma-portfolio/cvc-14-visible-fir-day-night-pedestrian-sequence-dataset/)

Raw datasets : There are too many missing files.

Thank you for your great work!

When I use the raw dataset with original data(aligned dataset), there are too many missing images..

Also, I don't know why train/test images are different between aligned(original) and raw datasets.

Could you elaborate a bit?

missing train data list

'010062.jpg', '030098.jpg', '030414.jpg', '070190.jpg', '070203.jpg', '070295.jpg', '070368.jpg', '080073.jpg', '080259.jpg', '080437.jpg', '080526.jpg', '081140.jpg', '081178.jpg', '081479.jpg', '090012.jpg', '090082.jpg', '090115.jpg', '090173.jpg', '090188.jpg', '090211.jpg', '090470.jpg', '090471.jpg', '090558.jpg', '090600.jpg', '090650.jpg', '090663.jpg', '090665.jpg', '090753.jpg', '090821.jpg', '090975.jpg', '090994.jpg', '091023.jpg', '091032.jpg', '091067.jpg', '091154.jpg', '091213.jpg', '091254.jpg', '100244.jpg', '100486.jpg', '100899.jpg', '110123.jpg', '110132.jpg', '110159.jpg', '110339.jpg', '110342.jpg', '110355.jpg', '110400.jpg', '110450.jpg', '120116.jpg', '120128.jpg', '120140.jpg', '120189.jpg', '120289.jpg', '120379.jpg', '120382.jpg', '130001.jpg', '130002.jpg', '130003.jpg', '130004.jpg', '130005.jpg', '130006.jpg', '130007.jpg', '130008.jpg', '130009.jpg', '130010.jpg', '130011.jpg', '130012.jpg', '130013.jpg', '130014.jpg', '130015.jpg', '130016.jpg', '130017.jpg', '130018.jpg', '130019.jpg', '130020.jpg', '130021.jpg', '130022.jpg', '130023.jpg', '130024.jpg', '130025.jpg', '130026.jpg', '130027.jpg', '130028.jpg', '130029.jpg', '130030.jpg', '130031.jpg', '130032.jpg', '130033.jpg', '130034.jpg', '130035.jpg', '130036.jpg', '130037.jpg', '130038.jpg', '130039.jpg', '130040.jpg', '130041.jpg', '130042.jpg', '130043.jpg', '130044.jpg', '130045.jpg', '130046.jpg', '130047.jpg', '130048.jpg', '130049.jpg', '130050.jpg', '130051.jpg', '130052.jpg', '130053.jpg', '130054.jpg', '130055.jpg', '130056.jpg', '130057.jpg', '130058.jpg', '130059.jpg', '130060.jpg', '130061.jpg', '130062.jpg', '130063.jpg', '130064.jpg', '130065.jpg', '130066.jpg', '130067.jpg', '130068.jpg', '130069.jpg', '130070.jpg', '130071.jpg', '130072.jpg', '130073.jpg', '130074.jpg', '130075.jpg', '130076.jpg', '130077.jpg', '130078.jpg', '130079.jpg', '130080.jpg', '130081.jpg', '130082.jpg', '130083.jpg', '130084.jpg', '130085.jpg', '130086.jpg', '130087.jpg', '130088.jpg', '130089.jpg', '130090.jpg', '130091.jpg', '130092.jpg', '130093.jpg', '130094.jpg', '130095.jpg', '130096.jpg', '130097.jpg', '130098.jpg', '130099.jpg', '130100.jpg', '130101.jpg', '130102.jpg', '130103.jpg', '130104.jpg', '130105.jpg', '130106.jpg', '130107.jpg', '130108.jpg', '130109.jpg', '130110.jpg', '130111.jpg', '130112.jpg', '130113.jpg', '130114.jpg', '130115.jpg', '130116.jpg', '130117.jpg', '130118.jpg', '130119.jpg', '130120.jpg', '130121.jpg', '130122.jpg', '130123.jpg', '130124.jpg', '130125.jpg', '130126.jpg', '130127.jpg', '130128.jpg', '130129.jpg', '130130.jpg', '130131.jpg', '130132.jpg', '130133.jpg', '130134.jpg', '130135.jpg', '130136.jpg', '130137.jpg', '130138.jpg', '130139.jpg', '130140.jpg', '130141.jpg', '130142.jpg', '130143.jpg', '130144.jpg', '130145.jpg', '130146.jpg', '130147.jpg', '130148.jpg', '130149.jpg', '130150.jpg', '130151.jpg', '130152.jpg', '130153.jpg', '130154.jpg', '130155.jpg', '130156.jpg', '130157.jpg', '130158.jpg', '130159.jpg', '130160.jpg', '130161.jpg', '130162.jpg', '130163.jpg', '130164.jpg', '130165.jpg', '130166.jpg', '130167.jpg', '130168.jpg', '130169.jpg', '130170.jpg', '130171.jpg', '130172.jpg', '130173.jpg', '130174.jpg', '130175.jpg', '130176.jpg', '130177.jpg', '130178.jpg', '130179.jpg', '130180.jpg', '130181.jpg', '130182.jpg', '130183.jpg', '130184.jpg', '130185.jpg', '130186.jpg', '130187.jpg', '130188.jpg', '130189.jpg', '130190.jpg', '130191.jpg', '130192.jpg', '130193.jpg', '130194.jpg', '130195.jpg', '130196.jpg', '130197.jpg', '130198.jpg', '130199.jpg', '130200.jpg', '130201.jpg', '130202.jpg', '130203.jpg', '130204.jpg', '130205.jpg', '130206.jpg', '130207.jpg', '130208.jpg', '130209.jpg', '130210.jpg', '130211.jpg', '130212.jpg', '130213.jpg', '130214.jpg', '130215.jpg', '130216.jpg', '130217.jpg', '130218.jpg', '130219.jpg', '130220.jpg', '130221.jpg', '130222.jpg', '130223.jpg', '130224.jpg', '130225.jpg', '130226.jpg', '130227.jpg', '130228.jpg', '130229.jpg', '130230.jpg', '130231.jpg', '130232.jpg', '130233.jpg', '130234.jpg', '130235.jpg', '130236.jpg', '130237.jpg', '130238.jpg', '130239.jpg', '130240.jpg', '130241.jpg', '130242.jpg', '130243.jpg', '130244.jpg', '130245.jpg', '130246.jpg', '130247.jpg', '130248.jpg', '130249.jpg', '130250.jpg', '130251.jpg', '130252.jpg', '130253.jpg', '130254.jpg', '130255.jpg', '130256.jpg', '130257.jpg', '130258.jpg', '130259.jpg', '130260.jpg', '130261.jpg', '130262.jpg', '130263.jpg', '130264.jpg', '130265.jpg', '130266.jpg', '130267.jpg', '130268.jpg', '130269.jpg', '130270.jpg', '130271.jpg', '130272.jpg', '130273.jpg', '130274.jpg', '130275.jpg', '130276.jpg', '130277.jpg', '130278.jpg', '130279.jpg', '130280.jpg', '130281.jpg', '130282.jpg', '130283.jpg', '130284.jpg', '130285.jpg', '130286.jpg', '130287.jpg', '130288.jpg', '130289.jpg', '130290.jpg', '130291.jpg', '130292.jpg', '130293.jpg', '130294.jpg', '130295.jpg', '130296.jpg', '130297.jpg', '130298.jpg', '130299.jpg', '130300.jpg', '130301.jpg', '130302.jpg', '130303.jpg', '130304.jpg', '130305.jpg', '130306.jpg', '130307.jpg', '130308.jpg', '130309.jpg', '130310.jpg', '130311.jpg', '130312.jpg', '130313.jpg', '130314.jpg', '130315.jpg', '130316.jpg', '130317.jpg', '130318.jpg', '130319.jpg', '130320.jpg', '130321.jpg', '130322.jpg', '130323.jpg', '130324.jpg', '130325.jpg', '130326.jpg', '130327.jpg', '130328.jpg', '130329.jpg', '130330.jpg', '130331.jpg', '130332.jpg', '130333.jpg', '130334.jpg', '130335.jpg', '130336.jpg', '130337.jpg', '130338.jpg', '130339.jpg', '130340.jpg', '130341.jpg', '130342.jpg', '130343.jpg', '130344.jpg', '130345.jpg', '130346.jpg', '130347.jpg', '130348.jpg', '130349.jpg', '130350.jpg', '130351.jpg', '130352.jpg', '130353.jpg', '130354.jpg', '130355.jpg', '130356.jpg', '130357.jpg', '130358.jpg', '130359.jpg', '130360.jpg', '130361.jpg', '130362.jpg', '130363.jpg', '130364.jpg', '130365.jpg', '130366.jpg', '130367.jpg', '130368.jpg', '130369.jpg', '130370.jpg', '130371.jpg', '130372.jpg', '130373.jpg', '130374.jpg', '130375.jpg', '130376.jpg', '130377.jpg', '130378.jpg', '130379.jpg', '130380.jpg', '130381.jpg', '130382.jpg', '130383.jpg', '130384.jpg', '130385.jpg', '130386.jpg', '130387.jpg', '130388.jpg', '130389.jpg', '130390.jpg', '130391.jpg', '130392.jpg', '130393.jpg', '130394.jpg', '130395.jpg', '130396.jpg', '130397.jpg', '130398.jpg', '130399.jpg', '130400.jpg', '130401.jpg', '130402.jpg', '130403.jpg', '130404.jpg', '130405.jpg', '130406.jpg', '130407.jpg', '130408.jpg', '130409.jpg', '130410.jpg', '130411.jpg', '130412.jpg', '130413.jpg', '130414.jpg', '130415.jpg', '130416.jpg', '130417.jpg', '130418.jpg', '130419.jpg', '130420.jpg', '130421.jpg', '130422.jpg', '130423.jpg', '130424.jpg', '130425.jpg', '130426.jpg', '130427.jpg', '130428.jpg', '130429.jpg', '130430.jpg', '130431.jpg', '130432.jpg', '130433.jpg', '130434.jpg', '130435.jpg', '130436.jpg', '130437.jpg', '130438.jpg', '130439.jpg', '130440.jpg', '130441.jpg', '130442.jpg', '130443.jpg', '130444.jpg', '130445.jpg', '130446.jpg', '130447.jpg', '130448.jpg', '130449.jpg', '130450.jpg', '130451.jpg', '130452.jpg', '130453.jpg', '130454.jpg', '130455.jpg', '130456.jpg', '130457.jpg', '130458.jpg', '130459.jpg', '130460.jpg', '130461.jpg', '130462.jpg', '130463.jpg', '130464.jpg', '130465.jpg', '130466.jpg', '130467.jpg', '130468.jpg', '130469.jpg', '130470.jpg', '130471.jpg', '130472.jpg', '130473.jpg', '130474.jpg', '130475.jpg', '130476.jpg', '130477.jpg', '130478.jpg', '130479.jpg', '130480.jpg', '130481.jpg', '130482.jpg', '130483.jpg', '130484.jpg', '130485.jpg', '130486.jpg', '130487.jpg', '130488.jpg', '130489.jpg', '130490.jpg', '130491.jpg', '130492.jpg', '130493.jpg', '130494.jpg', '130495.jpg', '130496.jpg', '130497.jpg', '130498.jpg', '130499.jpg', '130500.jpg', '250001.jpg', '250002.jpg', '250003.jpg', '250004.jpg', '250005.jpg', '250006.jpg', '250007.jpg', '250008.jpg', '250009.jpg', '250010.jpg', '250011.jpg', '250012.jpg', '250013.jpg', '250014.jpg', '250015.jpg', '250016.jpg', '250017.jpg', '250018.jpg', '250019.jpg', '250020.jpg', '250021.jpg', '250022.jpg', '250023.jpg', '250024.jpg', '250025.jpg', '250026.jpg', '250027.jpg', '250028.jpg', '250029.jpg', '250030.jpg', '250031.jpg', '250032.jpg', '250033.jpg', '250034.jpg', '250035.jpg', '250036.jpg', '250037.jpg', '250038.jpg', '250039.jpg', '250040.jpg', '250041.jpg', '250042.jpg', '250043.jpg', '250044.jpg', '250045.jpg', '250046.jpg', '250047.jpg', '250048.jpg', '250049.jpg', '250050.jpg', '250051.jpg', '250052.jpg', '250053.jpg', '250054.jpg', '250055.jpg', '250056.jpg', '250057.jpg', '250058.jpg', '250059.jpg', '250060.jpg', '250061.jpg', '250062.jpg', '250063.jpg', '250064.jpg', '250065.jpg', '250066.jpg', '250067.jpg', '250068.jpg', '250069.jpg', '250070.jpg', '250071.jpg', '250072.jpg', '250073.jpg', '250074.jpg', '250075.jpg', '250076.jpg', '250077.jpg', '250078.jpg', '250079.jpg', '250080.jpg', '250081.jpg', '250082.jpg', '250083.jpg', '250084.jpg', '250085.jpg', '250086.jpg', '250087.jpg', '250088.jpg', '250089.jpg', '250090.jpg', '250091.jpg', '250092.jpg', '250093.jpg', '250094.jpg', '250095.jpg', '250096.jpg', '250097.jpg', '250098.jpg', '250099.jpg', '250100.jpg', '250101.jpg', '250102.jpg', '250103.jpg', '250104.jpg', '250105.jpg', '250106.jpg', '250107.jpg', '250108.jpg', '250109.jpg', '250110.jpg', '250111.jpg', '250112.jpg', '250113.jpg', '250114.jpg', '250115.jpg', '250116.jpg', '250117.jpg', '250118.jpg', '250119.jpg', '250120.jpg', '250121.jpg', '250122.jpg', '250123.jpg', '250124.jpg', '250125.jpg', '250126.jpg', '250127.jpg', '250128.jpg', '250129.jpg', '250130.jpg', '250131.jpg', '250132.jpg', '250133.jpg', '250134.jpg', '250135.jpg', '250136.jpg', '250137.jpg', '250138.jpg', '250139.jpg', '250140.jpg', '250141.jpg', '250142.jpg', '250143.jpg', '250144.jpg', '250145.jpg', '250146.jpg', '250147.jpg', '250148.jpg', '250149.jpg', '250150.jpg', '250151.jpg', '250152.jpg', '250153.jpg', '250154.jpg', '250155.jpg', '250156.jpg', '250157.jpg', '250158.jpg', '250159.jpg', '250160.jpg', '250161.jpg', '250162.jpg', '250163.jpg', '250164.jpg', '250165.jpg', '250166.jpg', '250167.jpg', '250168.jpg', '250169.jpg', '250170.jpg', '250171.jpg', '250172.jpg', '250173.jpg', '250174.jpg', '250175.jpg', '250176.jpg', '250177.jpg', '250178.jpg', '250179.jpg', '250180.jpg', '250181.jpg', '250182.jpg', '250183.jpg', '250184.jpg', '250185.jpg', '250186.jpg', '250187.jpg', '250188.jpg', '250189.jpg', '250190.jpg', '250191.jpg', '250192.jpg', '250193.jpg', '250194.jpg', '250195.jpg', '250196.jpg', '250197.jpg', '250198.jpg', '250199.jpg', '250200.jpg', '250201.jpg', '250202.jpg', '250203.jpg', '250204.jpg', '250205.jpg', '250206.jpg', '250207.jpg', '250208.jpg', '250209.jpg', '250210.jpg', '250211.jpg', '250212.jpg', '250213.jpg', '250214.jpg', '250215.jpg', '250216.jpg', '250217.jpg', '250218.jpg', '250219.jpg', '250220.jpg', '250221.jpg', '250222.jpg', '250223.jpg', '250224.jpg', '250225.jpg', '250226.jpg', '250227.jpg', '250228.jpg', '250229.jpg', '250230.jpg', '250231.jpg', '250232.jpg', '250233.jpg', '250234.jpg', '250235.jpg', '250236.jpg', '250237.jpg', '250238.jpg', '250239.jpg', '250240.jpg', '250241.jpg', '250242.jpg', '250243.jpg', '250244.jpg', '250245.jpg', '250246.jpg', '250247.jpg', '250248.jpg', '250249.jpg', '250250.jpg', '250251.jpg', '250252.jpg', '250253.jpg', '250254.jpg', '250255.jpg', '250256.jpg', '250257.jpg', '250258.jpg', '250259.jpg', '250260.jpg', '250261.jpg', '250262.jpg', '250263.jpg', '250264.jpg', '250265.jpg', '250266.jpg', '250267.jpg', '250268.jpg', '250269.jpg', '250270.jpg', '250271.jpg', '250272.jpg', '250273.jpg', '250274.jpg', '250275.jpg', '250276.jpg', '250277.jpg', '250278.jpg', '250279.jpg', '250280.jpg', '250281.jpg', '250282.jpg', '250283.jpg', '250284.jpg', '250285.jpg', '250286.jpg', '250287.jpg', '250288.jpg', '250289.jpg', '250290.jpg', '250291.jpg', '250292.jpg', '250293.jpg', '250294.jpg', '250295.jpg', '250296.jpg', '250297.jpg', '250298.jpg', '250299.jpg', '250300.jpg', '250301.jpg', '250302.jpg', '250303.jpg', '250304.jpg', '250305.jpg', '250306.jpg', '250307.jpg', '250308.jpg', '250309.jpg', '250310.jpg', '250311.jpg', '250312.jpg', '250313.jpg', '250314.jpg', '250315.jpg', '250316.jpg', '250317.jpg', '250318.jpg', '250319.jpg', '250320.jpg', '250321.jpg', '250322.jpg', '250323.jpg', '250324.jpg', '250325.jpg', '250326.jpg', '250327.jpg', '250328.jpg', '250329.jpg', '250330.jpg', '250331.jpg', '250332.jpg', '250333.jpg', '250334.jpg', '250335.jpg', '250336.jpg', '250337.jpg', '250338.jpg', '250339.jpg', '250340.jpg', '250341.jpg', '250342.jpg', '250343.jpg', '250344.jpg', '250345.jpg', '250346.jpg', '250347.jpg', '250348.jpg', '250349.jpg', '250350.jpg', '250351.jpg', '250352.jpg', '250353.jpg', '250354.jpg', '250355.jpg', '250356.jpg', '250357.jpg', '250358.jpg', '250359.jpg', '250360.jpg', '250361.jpg', '250362.jpg', '250363.jpg', '250364.jpg', '250365.jpg', '250366.jpg', '250367.jpg', '250368.jpg', '250369.jpg', '250370.jpg', '250371.jpg', '250372.jpg', '250373.jpg', '250374.jpg', '250375.jpg', '250376.jpg', '250377.jpg', '250378.jpg', '250379.jpg', '250380.jpg', '250381.jpg', '250382.jpg', '250383.jpg', '250384.jpg', '250385.jpg', '250386.jpg', '250387.jpg', '250388.jpg', '250389.jpg', '250390.jpg', '250391.jpg', '250392.jpg', '250393.jpg', '250394.jpg', '250395.jpg', '250396.jpg', '250397.jpg', '250398.jpg', '250399.jpg', '250400.jpg', '250401.jpg', '250402.jpg', '250403.jpg', '250404.jpg', '250405.jpg', '250406.jpg', '250407.jpg', '250408.jpg', '250409.jpg', '250410.jpg', '250411.jpg', '250412.jpg', '250413.jpg', '250414.jpg', '250415.jpg', '250416.jpg', '250417.jpg', '250418.jpg', '250419.jpg', '250420.jpg', '250421.jpg', '250422.jpg', '250423.jpg'

missing test data list

'230239.jpg', '230451.jpg', '260001.jpg', '260002.jpg', '260003.jpg', '260004.jpg', '260005.jpg', '260006.jpg', '260007.jpg', '260008.jpg', '260009.jpg', '260010.jpg', '260011.jpg', '260012.jpg', '260013.jpg', '260014.jpg', '260015.jpg', '260016.jpg', '260017.jpg', '260018.jpg', '260019.jpg', '260020.jpg', '260021.jpg', '260022.jpg', '260023.jpg', '260024.jpg', '260025.jpg', '260026.jpg', '260027.jpg', '260028.jpg', '260029.jpg', '260030.jpg', '260031.jpg', '260032.jpg', '260033.jpg', '260034.jpg', '260035.jpg', '260036.jpg', '260037.jpg', '260038.jpg', '260039.jpg', '260040.jpg', '260041.jpg', '260042.jpg', '260043.jpg', '260044.jpg', '260045.jpg', '260046.jpg', '260047.jpg', '260048.jpg', '260049.jpg', '260050.jpg', '260051.jpg', '260052.jpg', '260053.jpg', '260054.jpg', '260055.jpg', '260056.jpg', '260057.jpg', '260058.jpg', '260059.jpg', '260060.jpg', '260061.jpg', '260062.jpg', '260063.jpg', '260064.jpg', '260065.jpg', '260066.jpg', '260067.jpg', '260068.jpg', '260069.jpg', '260070.jpg', '260071.jpg', '260072.jpg', '260073.jpg', '260074.jpg', '260075.jpg', '260076.jpg', '260077.jpg', '260078.jpg', '260079.jpg', '260080.jpg', '260081.jpg', '260082.jpg', '260083.jpg', '260084.jpg', '260085.jpg', '260086.jpg', '260087.jpg', '260088.jpg', '260089.jpg', '260090.jpg', '260091.jpg', '260092.jpg', '260093.jpg', '260094.jpg', '260095.jpg', '260096.jpg', '260097.jpg', '260098.jpg', '260099.jpg', '260100.jpg', '260101.jpg', '260102.jpg', '260103.jpg', '260104.jpg', '260105.jpg', '260106.jpg', '260107.jpg', '260108.jpg', '260109.jpg', '260110.jpg', '260111.jpg', '260112.jpg', '260113.jpg', '260114.jpg', '260115.jpg', '260116.jpg', '260117.jpg', '260118.jpg', '260119.jpg', '260120.jpg', '260121.jpg', '260122.jpg', '260123.jpg', '260124.jpg', '260125.jpg', '260126.jpg', '260127.jpg', '260128.jpg', '260129.jpg', '260130.jpg', '260131.jpg', '260132.jpg', '260133.jpg', '260134.jpg', '260135.jpg', '260136.jpg', '260137.jpg', '260138.jpg', '260139.jpg', '260140.jpg', '260141.jpg', '260142.jpg', '260143.jpg', '260144.jpg', '260145.jpg', '260146.jpg', '260147.jpg', '260148.jpg', '260149.jpg', '260150.jpg', '260151.jpg', '260152.jpg', '260153.jpg', '260154.jpg', '260155.jpg', '260156.jpg', '260157.jpg', '260158.jpg', '260159.jpg', '260160.jpg', '260161.jpg', '260162.jpg', '260163.jpg', '260164.jpg', '260165.jpg', '260166.jpg', '260167.jpg', '260168.jpg', '260169.jpg', '260170.jpg', '260171.jpg', '260172.jpg', '260173.jpg', '260174.jpg', '260175.jpg', '260176.jpg', '260177.jpg', '260178.jpg', '260179.jpg', '260180.jpg', '260181.jpg', '260182.jpg', '260183.jpg', '260184.jpg', '260185.jpg', '260186.jpg', '260187.jpg', '260188.jpg', '260189.jpg', '260190.jpg', '260191.jpg', '260192.jpg', '260193.jpg', '260194.jpg', '260195.jpg', '260196.jpg', '260197.jpg', '260198.jpg', '260199.jpg', '260200.jpg', '260201.jpg', '260202.jpg', '260203.jpg', '260204.jpg', '260206.jpg', '260207.jpg', '260208.jpg', '260209.jpg', '260210.jpg', '260211.jpg', '260212.jpg', '260213.jpg', '260214.jpg', '260215.jpg', '260216.jpg', '260217.jpg', '260218.jpg', '260219.jpg', '260220.jpg', '260221.jpg', '260222.jpg', '260224.jpg', '260225.jpg', '260226.jpg', '260227.jpg', '260228.jpg', '260229.jpg', '260230.jpg', '260231.jpg', '260232.jpg', '260233.jpg', '260234.jpg', '260235.jpg', '260236.jpg', '260237.jpg', '260238.jpg', '260239.jpg', '260241.jpg', '260242.jpg', '260243.jpg', '260244.jpg', '260245.jpg', '260246.jpg', '260247.jpg', '260248.jpg', '260249.jpg', '260250.jpg', '260251.jpg', '260252.jpg', '260253.jpg', '260254.jpg', '260255.jpg', '260256.jpg', '260257.jpg', '260258.jpg', '260259.jpg', '260260.jpg', '260261.jpg', '260262.jpg', '260263.jpg', '260264.jpg', '260265.jpg', '260266.jpg', '260267.jpg', '260268.jpg', '260269.jpg', '260270.jpg', '260271.jpg', '260272.jpg', '260273.jpg', '260274.jpg', '260275.jpg', '260276.jpg', '260277.jpg', '260278.jpg', '260279.jpg', '260280.jpg', '260281.jpg', '260282.jpg', '260283.jpg', '260284.jpg', '260285.jpg', '260286.jpg', '260287.jpg', '260288.jpg', '260289.jpg', '260290.jpg', '260291.jpg', '260292.jpg', '260293.jpg', '260294.jpg', '260295.jpg', '260296.jpg', '260297.jpg', '260298.jpg', '260299.jpg', '260300.jpg', '260301.jpg', '260302.jpg', '260303.jpg', '260304.jpg', '260305.jpg', '260306.jpg', '260307.jpg', '260308.jpg', '260309.jpg', '260310.jpg', '260311.jpg', '260312.jpg', '260313.jpg', '260314.jpg', '260315.jpg', '260316.jpg', '260317.jpg', '260318.jpg', '260319.jpg', '260320.jpg', '260321.jpg', '260322.jpg', '260323.jpg', '260324.jpg', '260325.jpg', '260326.jpg', '260327.jpg', '260328.jpg', '260329.jpg', '260330.jpg', '260331.jpg', '260332.jpg', '260333.jpg', '260334.jpg', '260335.jpg', '260336.jpg', '260337.jpg', '260338.jpg', '260339.jpg', '260340.jpg', '260341.jpg', '260342.jpg', '260343.jpg', '260344.jpg', '260345.jpg', '260346.jpg', '260347.jpg', '260348.jpg', '260349.jpg', '260350.jpg', '260351.jpg', '260352.jpg', '260353.jpg', '260354.jpg', '260355.jpg', '260356.jpg', '260357.jpg', '260358.jpg', '260359.jpg', '260360.jpg', '260361.jpg', '260362.jpg', '260363.jpg', '260364.jpg', '260365.jpg', '260366.jpg', '260367.jpg', '260368.jpg', '260369.jpg', '260370.jpg', '260371.jpg', '260372.jpg', '260373.jpg', '260374.jpg', '260375.jpg', '260376.jpg', '260377.jpg', '260378.jpg', '260379.jpg', '260380.jpg', '260381.jpg', '260382.jpg', '260383.jpg', '260384.jpg', '260385.jpg', '260386.jpg', '260387.jpg', '260388.jpg', '260389.jpg', '260390.jpg', '260391.jpg', '260392.jpg', '260393.jpg', '260394.jpg', '260395.jpg', '260396.jpg', '260397.jpg', '260398.jpg', '260399.jpg', '260400.jpg', '260401.jpg', '260402.jpg', '260403.jpg', '260404.jpg', '260405.jpg', '260406.jpg', '260407.jpg', '260408.jpg', '260409.jpg', '260410.jpg', '260411.jpg', '260412.jpg', '260413.jpg', '260414.jpg', '260415.jpg', '260416.jpg', '260417.jpg', '260418.jpg', '260419.jpg', '260420.jpg', '260421.jpg', '260422.jpg', '260423.jpg', '260424.jpg', '260425.jpg', '260426.jpg', '260427.jpg', '260428.jpg', '260429.jpg', '260430.jpg', '260431.jpg', '260432.jpg', '260433.jpg', '260434.jpg', '260435.jpg', '260436.jpg', '260437.jpg', '260438.jpg', '260439.jpg', '260440.jpg', '260441.jpg', '260442.jpg', '260443.jpg', '260444.jpg', '260445.jpg', '260446.jpg', '260447.jpg', '260448.jpg', '260449.jpg', '260450.jpg', '260451.jpg', '260452.jpg', '260453.jpg', '260454.jpg', '260455.jpg', '260456.jpg', '260457.jpg', '260458.jpg', '260459.jpg', '260460.jpg', '260461.jpg', '260462.jpg', '260463.jpg', '260464.jpg', '260465.jpg', '260466.jpg', '260467.jpg', '260468.jpg', '260469.jpg', '260470.jpg', '260471.jpg', '260472.jpg', '260473.jpg', '260474.jpg', '260475.jpg', '260476.jpg', '260477.jpg', '260478.jpg', '260479.jpg', '260480.jpg', '260481.jpg', '260482.jpg', '260483.jpg', '260484.jpg', '260485.jpg', '260486.jpg', '260487.jpg', '260488.jpg', '260489.jpg', '260490.jpg', '260491.jpg', '260492.jpg', '260493.jpg', '260494.jpg', '260495.jpg', '260496.jpg', '260497.jpg', '260498.jpg', '260499.jpg', '260500.jpg', '260501.jpg', '260502.jpg', '260503.jpg', '260504.jpg', '260505.jpg', '260506.jpg', '260507.jpg', '260508.jpg', '260509.jpg', '260510.jpg', '260511.jpg', '260512.jpg', '260513.jpg', '260514.jpg', '260515.jpg', '260516.jpg', '260517.jpg', '260518.jpg', '260519.jpg', '260520.jpg', '260521.jpg', '260522.jpg', '260523.jpg', '260524.jpg', '260525.jpg', '260526.jpg', '260527.jpg', '260528.jpg', '260529.jpg', '260530.jpg', '260531.jpg', '260532.jpg', '260533.jpg', '260534.jpg', '260535.jpg', '260536.jpg'

红外-可将光配准是怎么做的？

你好，图像对齐时，每张图片都手动选取对应点计算Homography矩阵吗？

Inference code for image to image translation

Thank for your work, it is very interesting!

But I don't see the inference code for image-to-image translation task. Can you release it??

Camera matrices

Could you provide camera matrices for both of the cameras? (fx, fy, cx, cy)

UnetSkipConnectionBlock Attribute Error

Hello,

I am trying to run pix2pix model and getting this error:

python test.py --dataroot ./datasets/ --model pix2pix --direction AtoB --preprocess scale_width_and_crop --load_size 320 --crop_size 256

dataset [AlignedDataset] was created
initialize network with normal
model [Pix2PixModel] was created
loading the model from ./checkpoints\experiment_name\latest_net_G.pth
Traceback (most recent call last):
File "D:\LLVIP-main\LLVIP-main\pix2pixGAN\test.py", line 47, in
model.setup(opt) # regular setup: load and print networks; create schedulers
File "D:\LLVIP-main\LLVIP-main\pix2pixGAN\models\base_model.py", line 88, in setup
self.load_networks(load_suffix)
File "D:\LLVIP-main\LLVIP-main\pix2pixGAN\models\base_model.py", line 198, in load_networks
self.__patch_instance_norm_state_dict(state_dict, net, key.split('.'))
File "D:\LLVIP-main\LLVIP-main\pix2pixGAN\models\base_model.py", line 174, in __patch_instance_norm_state_dict
self.__patch_instance_norm_state_dict(state_dict, getattr(module, key), keys, i + 1)
File "D:\LLVIP-main\LLVIP-main\pix2pixGAN\models\base_model.py", line 174, in __patch_instance_norm_state_dict
self.__patch_instance_norm_state_dict(state_dict, getattr(module, key), keys, i + 1)
File "D:\LLVIP-main\LLVIP-main\pix2pixGAN\pix2pix_env\lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'UnetSkipConnectionBlock' object has no attribute '1'

conversion to COCO format

Thermal Images

Hello,
We are working on thermal-visible fusion and I have some questions.

1. What are values stored in thermal image pixels?
2. Is there any way to reverse-engineer thermal images and get temperature values?

Thanks

would you like share your code about how to generate test file TXT that [person .98 1000 300 120 260] format by yolov5？

code for fusion experiments

Nice work. But I found no code for fusion experiments in this repo. Would you release the adapted codes recently?

测试集数据标签问题

您好，非常感谢您收集的数据集，最近在这个数据集上做了一些工作，发现存在如下的问题：处于图像边缘的行人，有时候有标签框，有时候也没有标，还存在少部分标错的图像，我可以对测试数据进行更新标记吗？

请问作者有试过使用image fusion后的图像，进行目标检测吗？有的话精度怎么样呢？

The yolov5 and yolov3 model weight format may not be matched

Hi,

Thanks for your amazing work!

When I use the weight of the yolov5 model which was downloaded from https://drive.google.com/file/d/1SPbr0PDiItape602-g-bstkX0P7NZo0q/view, I find that the format you provide is 'yolov5_ trained_ model.rar', but the code only supports .pt, .onnx, .tflite, .pb formats.

Looking forward to your reply.

Key conv52/batch_normalization/beta/ExponentialMovingAverage not found in checkpoint

Hello.
I get the following error when running evaluate.py in yolov3.
Unfortunately, I have not found a solution to my problem.
Help me please

Error :
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key conv52/batch_normalization/beta/ExponentialMovingAverage not found in checkpoint
[[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[{{node save/RestoreV2/_723}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_728_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

更新后的数据集

你好，感谢您发布的未配准的数据集，我现在进行的工作就是研究未配准下的多光谱行人检测，请问您这个未配准的数据集不对齐的情况严重吗，是否可以用于未配准下的多光谱行人检测研究？

关于红外低照度图像

感谢您发布的数据集，想请教下，我看大多数方法都是通过低照度图像和红外图像进行融合的方法改善效果，那是否可以直接在红外低照度图像（不使用可见光的低照度图像）上进行增强改善红外图像的效果？

Annotations in COCO format

Hi,

Do you have these annotations in COCO format? Or a script to convert them?

Thanks

关于红外

作者你好，我想问这个红外图像为什么是黑白的，在我的印象里红外图像好像是红色的（如第二张图），请问您是将红外图像调整过吗？还是您所使用的摄像机拍摄的红外图像本身就是黑白的？

How to draw the miss rate-FPPI curve?

Can you provide the pretrained model for yolov5 and yolov3 on visible images

Hi, thank you for your awesome work. Can you provide your retrained model for yolov5 and yolov3 on visible images? Thank you in advance.

About registration methods.

Thank you for your great work!

I have some question about your paper.

1. In your paper, you said "We first manually select several pairs of points that need to be aligned between the two images, then calculate the projection transformation to deform the infrared image". Is it means that you calculated the Homography matrix between visible and infrared images..? (if it is corrected, could you provide the value of homography matrix..?)

2. When i saw the some of image pairs, i think that infrared images are only resized. Do you crop the infrared images when you cut out the images to get the registered image pairs..?

Looking forward to your reply.

what about coco format? could you give us a script for the conversion

would you like share your code about cliping and registering image pairs so that they have exactly the same field of vision and the same image size?

缺少测试集

您好，当前的数据集仅有训练和验证集，请问可否提供测试集用于测试？

数据集链接无法打开

百度网盘链接密码错误。

数据集下载

百度网盘提取码不对

如何画MR-FPPI曲线

AP or MR

Thanks for your contribution！

1. I found this benchmark when reading paper CFT (as mentioned in this issue #11)，now I see that your current baseline of YOLOV5 with infrared data maintains 67.0 AP, which is much better than CFT (63.6). However, for the Log Average Miss Rate, results for your baseline and CFT are 10.66 VS 5.40. I wonder which metric is more reasonable to demonstrate the power of model?
2. I also noticed that some followers asked about the poor performance of image fusion method for pedestrian detection (like CFT), which is even worse than detectors using single modal. Have you extended your baseline YOLOV5 models with both RGB and thermal inputs? How's the results comparing with the original baseline?
   Thanks.

pix2pixGAN pretrianed weight

请问可以分享一下使用pix2pixGAN在该数据集上预训练好的模型权重吗，我想在另一个数据集上测试一下

VOC如何下载

您好，我下载数据集后没有找到检测使用的labels
Our LLVIP dataset contains 30976 images (15488 pairs), 12025 pairs for train and 3463 for test.
The same pair of visible and infrared images share the same annotation, and they have the same name.
The labels are in VOC format.

红外图像噪声

您好，关于红外成像，向您请教一下，在红外图像上为什么在墙壁上会出现人像的倒映，如下图所示

questions about image fusion part experiments

Thanks for your great work ~ I have some questions about the experiments.
I want to know how many images you used in the image fusion part experiment. Did you use all the images for training the FusionGAN and Densefuse algorithm? Or just the images in the directory? And what are the parameters you used?
Looking forward to your reply.

。

Whether the object detection task uses early stopping strategy or not?

Hello ~
Just like the title, I want to know if the task trained completely for 200 epochs or just stopped in the epochs without improvement?

Some Questions about Your Datasets LLVIP

Thank you for your great work!

Recently, I'm studying on the image motion deblurring in the image fusion task and when I fed the images into the deblurring
network, I found that it rarely has any effect. So, I have some confusions about your dataset.

 1. Before you released the dataset from the video to registered images，have you made some pre-processing methods? Like deblurring or denoising.

 2. The images you released were selected randomly or deliberately. 

Thank you for your time and look forward to your reply.