qiufengsly/fs-map-project

Few-Shot Multi-Agent Perception

(FS-MAP)

This is the homepage of Few-Shot Multi-Agent Perception (FS-MAP).

• Chenyou Fan, Junjie Hu, Jianwei Huang. "Few-Shot Multi-Agent Perception." In 29th ACM International Conference on Multimedia (ACM MM'21).

Dataset

We use AirSim dataset to perform few-shot segmentation task. We modify this dataset to form few-shot learning tasks. We provide direct download link below.

Experiment on AirSim Dataset

• The code is modified upon DeepEMD. Please properly cite their excellent work if you use this code in research.
• We provide self-contained implementation in the following section.

Download dataset and our split

• Download dataset from Google Drive link, extract airsim-mrmps-data into ./dataset/ folder.
• Download our trained models from Google Drive link, extract to ./results folder, check results/seg/meta folders.
• Check our split in configs/split_save_files.pkl

pre-train segmentation

python train.py --ph=0 --is_seg=1 --pretrain_dir=results/seg/pre_train

Train

python train.py --ph=1 --is_seg=1 --pretrain_dir=results/seg/pre_train

Evaluation

• check results/seg/meta and find the latest checkpoint dir, to replace XXX
• set "--shot=5" to test 5-shot case

python test.py --is_seg=1 --model_dir=XXXX  --loop=0
python test.py --is_seg=1 --model_dir=XXXX  --loop=0 --shot=5

to use our trained models, download as above mentioned, and execute

python test.py --is_seg=1 --model_dir=results/seg/meta/loop0
python test.py --is_seg=1 --model_dir=results/seg/meta/loop0_st5 --shot=5

Reference

Please cite our work if you use this code.

@inproceedings{fan2021fsmap,
  title={Few-Shot Multi-Agent Perception},
  author={Fan, Chenyou and Hu, Junjie and Huang, Jianwei},
  booktitle={ACM MultiMedia},
  year={2021}
}

Other references

Please also properly cite the following excellent work in research.

• DeepEMD.
• Multi-Agent Perception.

Requirements

Python = 3.8 PyTorch = 1.7+ [here]

GPU training with 4G+ memory, testing with 2G+ memory.

pip install scikit-learn pretrainedmodels