chenzhaowei13/light-condition-style-transfer

Lane Detection in Low-light Conditions Using an Efficient Data Enhancement : Light Conditions Style Transfer (IV 2020)

Light Conditions Style Transfer

Paper

Lane Detection in Low-light Conditions Using an Efficient Data Enhancement : Light Conditions Style Transfer

Accepted by 2020 IEEE Intelligent Vehicles Symposium (IV 2020).

The main framework is as follows:

Empirically, lane detection model trained using our method demonstrated adaptability in low-light conditions and robustness in complex scenarios. (It can achieve 73.9 F1-measure in CULane testing set)

Datasets

CULane

The whole dataset is available at CULane.

CULane
├── driver_23_30frame       # training&validation
├── driver_161_90frame      # training&validation
├── driver_182_30frame      # training&validation
├── driver_193_90frame      # testing
├── driver_100_30frame      # testing
├── driver_37_30frame       # testing
├── laneseg_label_w16       # labels
└── list                    # list

Generated Images

The images in low-light conditions are generated by the proposed SIM-CycleGAN.

Requirements

• PyTorch 1.3.0.

• Matlab (for tools/prob2lines), version R2017a or later.

• Opencv (for tools/lane_evaluation).

Before start

conda create -n  your_env_name python=3.6
conda activate your_env_name
conda install pytorch==1.3.0 torchvision==0.4.1 cudatoolkit=10.0 -c pytorch
pip install -r requirements.txt 

SIM-CycleGAN

The source code for SIM-CycleGAN has been released. (11/03)

train

Train your own SIM-CycleGAN model as follow.

python train.py  --name repo_name \
                 --dataset_loadtxt_A /path/to/domain_A_txt \
                 --dataset_loadtxt_B /path/to/domain_B_txt \
                 --gpu_ids 6 \

test

Use your trained model to generate images.

python test.py   --name repo_name \
                 --model simcycle_gan \
                 --dataset_loadtxt_A /path/to/domain_A_txt \
                 --dataset_loadtxt_B /path/to/domain_B_txt \
                 --gpu_ids 6 \

Lane Detetcion

The source code used for the lane detction is made publicly available by HOU Yuenan.

Test for Demo

We provide demo for testing a single image or a video.

sh ./demo.sh

You can get the results as follow.

Result for probability map

Result for points

If you want to test the model for video, you can set mode=0 in demo.sh.

Evaluate the Model

The trained model used in this paper is available in ./trained.

1. Run test script

sh ./test_erfnet.sh

2. Get lines from probability maps

cd tools/prob2lines
matlab -nodisplay -r "main;exit"

Please check the file path in Matlab code before.

3. Evaluation

cd /tools/lane_evaluation
make
# You may also use cmake instead of make, via:
# mkdir build && cd build && cmake ..
sh eval_all.sh    # evaluate the whole test set
sh eval_split.sh  # evaluate each scenario separately

The evaluation results are saved in /tools/lane_evaluation/output.

Performance

Light Conditions Stlye Transfer

Some examples of real images in normal light conditions and their corresponding translations images in low-light conditions.

Lane Detetcion

Performance ( (F₁-measure) ) of different methods on CULane testing set. For crossroad, only FP is shown.

Category	ERFNet	CycleGAN+ERFNet	SIM-CycleGAN + ERFNet(ours)	SCNN	ENet-SAD	ResNet-101-SAD
Normal	91.5	91.7	91.8	90.6	90.1	90.7
Crowded	71.6	71.5	71.8	69.7	68.8	70.0
Night	67.1	68.9	69.4	66.1	66.0	66.3
No Line	45.1	45.2	46.1	43.4	41.6	43.5
Shadow	71.3	73.1	76.2	66.9	65.9	67.0
Arrow	87.2	87.2	87.8	66.9	65.9	67.0
Dazzle Light	66.0	67.5	66.4	58.5	60.2	59.9
Curve	66.3	69.0	67.1	64.4	65.7	65.7
Crossroad	2199	2402	2346	1990	1998	2052
Total	73.1	73.6	73.9	71.6	70.8	71.8

The probability maps output by the three methods above are shown as following

To do

• Add attenction on ERFNet

• Open the source code for SIM-CycleGAN

• Upgade pytorch (from 0.3.0 to 1.3.0)

• Upload demo for test

Citation

Please cite this in your publication if our work helps your research.

@inproceedings{Liu2020Lane,
  title={Lane Detection in Low-light Conditions Using an Efficient Data Enhancement : Light Conditions Style Transfer},
  author={Liu, Tong and Chen, Zhaowei and Yang, Yi and Wu, Zehao and Li, Haowei},
  booktitle={2020 IEEE intelligent vehicles symposium (IV)},
  year={2020},
}

Acknowledgement

This project refers to the following projects:

• Codes-for-Lane-Detection
• SCNN
• CycleGAN

light-condition-style-transfer's Issues

how to pick out a 3200 low light images

I would like to ask you how to pick out a 3200 low light images from CULane for training, thanks.

SIM-CycleGAN

您好，很高兴你能开源您的代码，我对 SIM-CycleGAN这部分比较感兴趣，请问大概什么适合能开源这部分的代码 @Chenzhaowei13

What are the computational resources in your code?

What are the computational resources in your code?

result of SIM-CycleGAN

您好，很高兴你能开源您的代码，我对 SIM-CycleGAN这部分比较感兴趣，请问可以把SIM-CycleGAN生成的夜间图像数据集开源吗？

有测试的权重吗

F1 measure of Better-CycleGAN + ERFNet

For Curve category, the F1 measure of ERFNet should be 66.3 instead of 71.6
https://github.com/cardwing/Codes-for-Lane-Detection

I ran the evaluation code on your trained model, and obtained exactly the same metrics except F1-measure corresponding to the Curve category. On my side, its 67.1 different from 72.4 reported in your table. Please check that.

Btw, I was able to achieve even better results following your paper, although my generated fake night images do not look as good as yours. I guess it also depends on the data selection for image translation. Nevertheless, I look forward to your releasing the source code for Better-CycleGAN.

Thanks!

ModuleNotFoundError: No module named 'models.cycle_gan_model'

请问以下error如何修改？感谢！
cd /content/Light-Condition-Style-Transfer/SIM-CycleGAN/
python train.py --dataroot ./datasets/maps --/content/Light-Condition-Style-Transfer/Culane/driver_37_30frame --name maps_cyclegan --model cycle_gan

请问有哪些 models. 有lowlight的作用？

File "train.py", line 28, in
opt = TrainOptions().parse() # get training options
File "/content/Light-Condition-Style-Transfer/SIM-CycleGAN/options/base_options.py", line 118, in parse
opt = self.gather_options()
File "/content/Light-Condition-Style-Transfer/SIM-CycleGAN/options/base_options.py", line 78, in gather_options
model_option_setter = models.get_option_setter(model_name)
File "/content/Light-Condition-Style-Transfer/SIM-CycleGAN/models/init.py", line 50, in get_option_setter
model_class = find_model_using_name(model_name)
File "/content/Light-Condition-Style-Transfer/SIM-CycleGAN/models/init.py", line 33, in find_model_using_name
modellib = importlib.import_module(model_filename)
File "/usr/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'models.cycle_gan_model'

How to generate images

Hi, I would like to know how to generate images in low light condition?
Thank you!

demo.py argumenst

Hello sir !
when i try to run the demo.py i get the following error

there was no mention as to specifying these arguments !! could you please guide me on this thank you!

How to test?

What is the purpose of the test_img.txt used in the test?
Why does every picture in the file correspond to / laneseg_ label_ w16/driver_ 23_ 30frame/05170950_ 0742.MP4/04280. png 1 1 1 1。
When testing, the label used is xxx.lines.txt?

when will you release the source code of SIM-CycleGAN?

_{Sent from PPHub}

when will you open the source code for Better-CycleGAN?

How to select low light images in CULane train set?

CULane test set has split list for nine classes, while train set not. In your paper, you select 3200 low light images to train Better-CycleGAN.
Can you provide the list of low light images in CULane train set?
Thanks a lot!

the trained model

hello， thanks for you contribution， Would you like to share the trained model of SIM-CycleGAN