avbuffer/msffn

An MultiSpectral Feature Fusion Network (MSFFN) for object detection or pedestrian detection.

MSFFN

An MultiSpectral Feature Fusion Network (MSFFN) for object detection or pedestrian detection based on KAIST Multispectral Pedestrian Detection Benchmark.

Download KAIST dataset

Download KAIST Multispectral Pedestrian Detection Benchmark [KAIST]

Extract all of these tars into one directory and rename them, which should have the following basic structure.

Kaist datasets path

1. data/dataset/kaist/annos

2. data/dataset/kaist/images

2.1) data/dataset/kaist/images/lwir

2.2) data/dataset/kaist/images/visible

3. data/dataset/kaist/imgsets

3.1) data/dataset/kaist/imgsets/train.txt

3.2) data/dataset/kaist/imgsets/val.txt

Make kaist train and val annotation

$ python scripts/annotation.py

Then edit your `core/config.py` to make some necessary configurations

__C.YOLO.CLASSES = "data/classes/pedestrian.names"

__C.TRAIN.ANNOT_PATH = "data/dataset/pedestrian_train.txt"

__C.TEST.ANNOT_PATH = "data/dataset/pedestrian_val.txt"

Train KAIST dataset

Two files are required as follows:

• data/classes/pedestrian.names

   person
  

• data/dataset/pedestrian_train.txt

   data/dataset/kaist/images/visible/set03_V001_I00909.jpg data/dataset/kaist/images/lwir/set03_V001_I00909.jpg 323,319,345,273,0 287,215,301,249,0 279,222,288,244,0 1,240,36,441,0
  

• data/dataset/pedestrian_val.txt

   data/dataset/kaist/images/visible/set00_V008_I00627.jpg data/dataset/kaist/images/lwir/set00_V008_I00627.jpg 385,228,408,285,0
  

Train method

$ python train.py
$ tensorboard --logdir data/log/train

Evaluate method

$ python evaluate.py

mAP

$ python evaluate.py
$ cd mAP
$ python main.py

Demo

Two steps:

Step1: freeze graph from ckpt file into pb file in order to speed up

Step2: config pb_file and images or videos file, or num_classes / input_size / score_thresh / iou_thresh in demo.py

$ python scripts/freeze_graph_ckpt2pb.py
$ python demo.py