shuangw98/mfdc

Multi-Faceted Distillation of Base-Novel Commonality for Few-shot Object Detection, ECCV 2022

Multi-Faceted Distillation of Base-Novel Commonality for Few-shot Object Detection, ECCV 2022

This repo is built upon DeFRCN, where you can download the datasets and the pre-trained weights.

Requirements

Python == 3.7.10

Pytorch == 1.6.0

Torchvision == 0.7.0

Detectron2 == 0.3

CUDA == 10.1

File Structure

    ├── weight/                   
    |   ├── R-101.pkl              
    |   └── resnet101-5d3b4d8f.pth   
    └── datasets/
        ├── coco/           
        │   ├── annotations/
        │   ├── train2014/
        │   └── val2014/
        ├── cocosplit/
        ├── VOC2007/            
        │   ├── Annotations/
        │   ├── ImageSets/
        │   └── JPEGImages/
        ├── VOC2012/            
        │   ├── Annotations/
        │   ├── ImageSets/
        │   └── JPEGImages/
        └── vocsplit/

Training and Evaluation

• For VOC

sh voc_train.sh mfdc SPLIT_ID

• For COCO

sh coco_train.sh mfdc

Citation

If you find our code helpful in your research, please cite the following publication:

@inproceedings{wu2022multi,
  title={Multi-faceted Distillation of Base-Novel Commonality for Few-Shot Object Detection},
  author={Wu, Shuang and Pei, Wenjie and Mei, Dianwen and Chen, Fanglin and Tian, Jiandong and Lu, Guangming},
  booktitle={European Conference on Computer Vision},
  pages={578--594},
  year={2022},
  organization={Springer}
}

Contact

Please feel free to contact me (Email: [email protected]) if you have any questions.

mfdc's Issues

Checkpoints & logs for COCO & VOC split 2,3

Hello, I saw that you posted the Google drive link for the VOC split 1 training logs and checkpoints. I wonder if you can provide the links for the other settings especially COCO. Thank you very much!

使用单个gpu训练，应该怎么修改defrcn/modeling/roi_heads/roi_heads.py中的代码？

有一些使用到分布式的代码，例如这段代码：
def concat_all_gather(tensor): tensors_gather = [ torch.ones_like(tensor) for _ in range(torch.distributed.get_world_size()) ] torch.distributed.all_gather(tensors_gather, tensor, async_op=False) output = torch.cat(tensors_gather, dim=0) return output
应该如何修改？

a question about lr

I want to ask a question about lr. I run the code with 1 gpu and batch_size is 2, Why should my learning rate drop to 10^(-7) to run the code successfully?

A question about "loss_cls_score_aug" in roi_heads.py

Hi, thank you for this work. I run the code with 1 gpu and batch_size is 1, Why is the "loss_cls_score_aug" of each iteration always 0? I get the following:

About seed ?

Thanks for the great work, I wonder whether the result in your repo is all based on the seed0 experiment?

单个GPU训练参数如何设置

作者，您好，请问我是用户单个GPU对于您的代码进行训练时，超参数该如何设置呢？期待您的解答，谢谢

Question about queue update

I am running the code during novel fine-tuning on VOC split 1, 1-shot seed 0. During the memory bank update, it seems like the memory bank is updating more samples than there are in the novel set. See below at the debugging output

As you can see, the uniq_c indices are [0, 9, 10, 17, 18]. The novel indices correspond to [15, 16, 17, 18, 19]. For index 17, features_c_s is of shape [26, 2048]. This seems to violate the 1-shot scenario.

I may be mis-understanding something. What is happening here?

FSOD vs GFSOD

Hi, thank you for this work. In the repository, you have provided the configuration files for the gfsod evaluation scenario. If I'm not mistaken, the results in table 1 of your paper provides results for the fsod scenario, correct? I have tried to reproduce your results running the code exactly as in the README, but I can not get the results in table 1.

Do you provide the configuration files for the fsod scenario?

question about the config files

hello, can you please tell me what is the meaning of this?
thank you very much!!!

Question about base-training checkpoint

Can you provide the base-training checkpoint of the voc and coco datasets?

Thank You!

argument

Dear author：
--opts TEST.PCB_MODELPATH ${IMAGENET_PRETRAIN_TORCH}，其中TEST.PCB_MODELPATH是什么意思呢

File not found ! ! !

Hello, first of all, thank you for your work. I have downloaded the pre training model，I want to know how to obtain ./model_final.pth files? Looking forward to your reply!

modeling/roi_heads.py

Dear author, please can you share the design concept of this module with me. I would like to ask you for your humble advice

modeling/roi_heads.py

@torch.no_grad()
def _dequeue_and_enqueue(self, keys_s, keys_l, gt_class):
    keys_s = keys_s[:self.queue_len]
    keys_l = keys_l[:self.queue_len]
    batch_size = keys_s.shape[0]
    ptr = int(self.queue_ptr[gt_class])
    if ptr + batch_size <= self.queue_len:
        self.queue_s[gt_class, ptr:ptr + batch_size] = keys_s
        self.queue_l[gt_class, ptr:ptr + batch_size] = keys_l
    else:
        self.queue_s[gt_class, ptr:] = keys_s[:self.queue_len - ptr]
        self.queue_s[gt_class, :(ptr + batch_size) % self.queue_len] = keys_s[self.queue_len - ptr:]
        self.queue_l[gt_class, ptr:] = keys_l[:self.queue_len - ptr]
        self.queue_l[gt_class, :(ptr + batch_size) % self.queue_len] = keys_l[self.queue_len - ptr:]
        
    if ptr + batch_size >= self.queue_len:
        self.queue_full[gt_class] = 1
    ptr = (ptr + batch_size) % self.queue_len
    self.queue_ptr[gt_class] = ptr

Why the different number of images on inference between MDFC and DeFRCN?

MDFC inference log: defrcn.evaluation.evaluator INFO: Start inference on 4952 images
But, DeFRCN inference log (author uploaded): defrcn.evaluation.evaluator INFO: Start inference on 619 images
Thanks and forward to your reply.

Have a question about the config.

Thanks for nice paper and source code.
Could you tell me what the 'removevoc' stands for in the fine-tuning config file?
removevoc_2007_trainval_allx_2shot_seedx

I have well understood FSOD benchmark. Does it have any difference with data setting of {TFA, DeFRCN}?.