shirly-han/pmms

Prototype Mixture Models for Few-shot Semantic Segmentation

Prototype Mixture Models

This code is for the paper "Prototype Mixture Models for Few-shot Semantic Segmentation" in European Conference on Computer Vision(ECCV 2020).

PMMs architecture: RPMMS architecture:

Overview

This code contains two methods called PMMs and RPMMs.You can train or test them on Pascal voc or COCO dataset.

The experiments are divided into into 4 independent groups for cross validation.

• config/ contains the config setting file for data and network;
• data/ contains the dataloader and dataset;
• data_list/ contains the data list for training and inference;
• models/ contains the backbone & PMMs module;
• networks/ contains the implementation of the PMMs(FPMMs.py) & RPMMs(FRPMMs.py);
• scripts/ contains the shell file to use this code succinctly;
• utils/ contains other dependent code;

Dependencies

python == 3.7, pytorch1.0,

torchvision, pillow, opencv-python, pandas, matplotlib, scikit-image

Usage

This code is very simple to use. You can train and test it just follow the steps below.

Preparation

After downloading the code, installing dependencies. You should modify the data path and model path in config/settings.py.

Note that you may need to check the hierarchy of the dataset in data/voc_train.py, data/voc_val.py, data/coco_val.py, data/coco_val.py

Training

cd scripts
sh train_group0.sh

Inference

If you want to test all of the saved models, you can use:

python test_all_frame.py

If you want to test our pretrained model, you can download them from https://github.com/ECCV20/PMMs/tree/master/snapshots/FRPMMs. And test them using:

python test_frame.py

Cross-validation classes for Pascal-5ⁱ

Dataset	Test class
Pascal-50	aeroplane, bicycle, bird, boat, bottle
Pascal-51	bus, car, cat, chair, cow
Pascal-52	diningtable, dog, horse, motorbike, person
Pascal-53	potted plant, sheep, sofa, train, tv/monitor

Cross-validation classes for COCO-20ⁱ

Dataset	Test class
COCO-200	person, airplane, boat, parking meter, dog, elephant, backpack, suitcase, sports ball, skateboard, wine glass, spoon, sandwich, hot dog, chair, dining table, mouse, microwave, scissors
COCO-201	bicycle, bus, traffic light, bench, horse, bear, umbrella, frisbee, kite, surfboard, cup, bowl, orange, pizza, couch, toilet, remote, oven, book, teddy bear
COCO-202	car, train, fire hydrant, bird, sheep, zebra, handbag, skis, baseball bat, tennis racket, fork, banana, broccoli, donut, potted plant, tv, keyboard, sink, toaster, clock, hair drier
COCO-203	motorcycle, truck, stop sign, cat, cow, giraffe, tie, snowboard, baseball glove, bottle, knife, apple, carrot, cake, bed, laptop, cell phone, refrigerator, vase, toothbrush

Performance

Setting	Backbone	Method	Pascal-50	Pascal-51	Pascal-52	Pascal-53	Mean
1-shot	VGG16	RPMMs	47.14	65.82	50.57	48.54	53.02
	Resnet50	PMMs	51.98	67.54	51.54	49.81	55.22
		RPMMs	55.15	66.91	52.61	50.68	56.34
5-shot	VGG16	RPMMs	50.00	66.46	51.94	47.64	54.01
	Resnet50	PMMs	55.03	68.22	52.89	51.11	56.81
		RPMMs	56.28	67.34	54.52	51.00	57.30

Citations

Please consider citing our paper in your publications if the project helps your research.

@inproceedings{PMMs2020,
  title   =  {Prototype Mixture Models for Few-shot Semantic Segmentation},
  author  =  {Boyu Yang and Chang Liu and Bohao Li and Jianbin Jiao, and Ye, Qixiang},
  booktitle =  {ECCV},
  year    =  {2020}
}

References

Some of our Code is based on the following code:

EMANet:https://github.com/XiaLiPKU/EMANet

CANet:https://github.com/icoz69/CaNet

SG-One:https://github.com/xiaomengyc/SG-One