er-muyue/defrcn

Introduction

This repo contains the official PyTorch implementation of our ICCV paper DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection.

Updates!!

• 【2021/10/10】 We release the official PyTorch implementation of DeFRCN.
• 【2021/08/20】 We have uploaded our paper (long version with supplementary material) on arxiv, review it for more details.

Quick Start

1. Check Requirements

• Linux with Python >= 3.6
• PyTorch >= 1.6 & torchvision that matches the PyTorch version.
• CUDA 10.1, 10.2
• GCC >= 4.9

2. Build DeFRCN

• Clone Code

  git clone https://github.com/er-muyue/DeFRCN.git
  cd DeFRCN
  

• Create a virtual environment (optional)

  virtualenv defrcn
  cd /path/to/venv/defrcn
  source ./bin/activate
  

• Install PyTorch 1.6.0 with CUDA 10.1

  pip3 install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html

• Install Detectron2

  python3 -m pip install detectron2==0.3 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.6/index.html
  

  • If you use other version of PyTorch/CUDA, check the latest version of Detectron2 in this page: Detectron2.
  • Sorry for that I don’t have enough time to test on more versions, if you run into problems with other versions, please let me know.
• Install other requirements.

  python3 -m pip install -r requirements.txt
  

3. Prepare Data and Weights

• Data Preparation

  • We evaluate our models on two datasets for both FSOD and G-FSOD settings:

    Dataset	Size	GoogleDrive	BaiduYun	Note
    VOC2007	0.8G	download	download	-
    VOC2012	3.5G	download	download	-
    vocsplit	<1M	download	download	refer from TFA
    COCO	~19G	-	-	download from offical
    cocosplit	174M	download	download	refer from TFA

  • Unzip the downloaded data-source to datasets and put it into your project directory:

      ...
      datasets
        | -- coco (trainval2014/*.jpg, val2014/*.jpg, annotations/*.json)
        | -- cocosplit
        | -- VOC2007
        | -- VOC2012
        | -- vocsplit
      defrcn
      tools
      ...
    

• Weights Preparation
  • We use the imagenet pretrain weights to initialize our model. Download the same models from here: GoogleDrive BaiduYun
  • The extract code for all BaiduYun link is 0000

4. Training and Evaluation

For ease of training and evaluation over multiple runs, we integrate the whole pipeline of few-shot object detection into one script run_*.sh, including base pre-training and novel-finetuning (both FSOD and G-FSOD).

• To reproduce the results on VOC, EXP_NAME can be any string (e.g defrcn, or something) and SPLIT_ID must be 1 or 2 or 3 (we consider 3 random splits like other papers).

  bash run_voc.sh EXP_NAME SPLIT_ID (1, 2 or 3)
  

• To reproduce the results on COCO, EXP_NAME can be any string (e.g defrcn, or something)

  bash run_coco.sh EXP_NAME
  

• Please read the details of few-shot object detection pipeline in run_*.sh, you need change IMAGENET_PRETRAIN* to your path.

Results on COCO Benchmark

• Few-shot Object Detection

  Method			mAPnovel
  Shot	1	2	3	5	10	30
  FRCN-ft	1.0*	1.8*	2.8*	4.0*	6.5	11.1
  FSRW	-	-	-	-	5.6	9.1
  MetaDet	-	-	-	-	7.1	11.3
  MetaR-CNN	-	-	-	-	8.7	12.4
  TFA	4.4*	5.4*	6.0*	7.7*	10.0	13.7
  MPSR	5.1*	6.7*	7.4*	8.7*	9.8	14.1
  FSDetView	4.5	6.6	7.2	10.7	12.5	14.7
  DeFRCN (Our Paper)	9.3	12.9	14.8	16.1	18.5	22.6
  DeFRCN (This Repo)	9.7	13.1	14.5	15.6	18.4	22.6

• Generalized Few-shot Object Detection

  Method			mAPnovel
  Shot	1	2	3	5	10	30
  FRCN-ft	1.7	3.1	3.7	4.6	5.5	7.4
  TFA	1.9	3.9	5.1	7	9.1	12.1
  FSDetView	3.2	4.9	6.7	8.1	10.7	15.9
  DeFRCN (Our Paper)	4.8	8.5	10.7	13.6	16.8	21.2
  DeFRCN (This Repo)	4.8	8.5	10.7	13.5	16.7	21.0

• * indicates that the results are reproduced by us with their source code.
• It's normal to observe -0.3~+0.3AP noise between your results and this repo.
• The results of mAP^base and mAP^all for G-FSOD are list here GoogleDrive, BaiduYun.
• If you have any problem of above results in this repo, you can download configs and train logs from GoogleDrive, BaiduYun.

Results on VOC Benchmark

• Few-shot Object Detection

  Method			Split-1					Split-2					Split-3
  Shot	1	2	3	5	10	1	2	3	5	10	1	2	3	5	10
  YOLO-ft	6.6	10.7	12.5	24.8	38.6	12.5	4.2	11.6	16.1	33.9	13.0	15.9	15.0	32.2	38.4
  FRCN-ft	13.8	19.6	32.8	41.5	45.6	7.9	15.3	26.2	31.6	39.1	9.8	11.3	19.1	35.0	45.1
  FSRW	14.8	15.5	26.7	33.9	47.2	15.7	15.2	22.7	30.1	40.5	21.3	25.6	28.4	42.8	45.9
  MetaDet	18.9	20.6	30.2	36.8	49.6	21.8	23.1	27.8	31.7	43.0	20.6	23.9	29.4	43.9	44.1
  MetaR-CNN	19.9	25.5	35.0	45.7	51.5	10.4	19.4	29.6	34.8	45.4	14.3	18.2	27.5	41.2	48.1
  TFA	39.8	36.1	44.7	55.7	56.0	23.5	26.9	34.1	35.1	39.1	30.8	34.8	42.8	49.5	49.8
  MPSR	41.7	-	51.4	55.2	61.8	24.4	-	39.2	39.9	47.8	35.6	-	42.3	48.0	49.7
  DeFRCN (Our Paper)	53.6	57.5	61.5	64.1	60.8	30.1	38.1	47.0	53.3	47.9	48.4	50.9	52.3	54.9	57.4
  DeFRCN (This Repo)	55.1	57.4	61.1	64.6	61.5	32.1	40.5	47.9	52.9	47.5	48.9	51.9	52.3	55.7	59.0

• Generalized Few-shot Object Detection

  Method			Split-1					Split-2					Split-3
  Shot	1	2	3	5	10	1	2	3	5	10	1	2	3	5	10
  FRCN-ft	9.9	15.6	21.6	28.0	52.0	9.4	13.8	17.4	21.9	39.7	8.1	13.9	19	23.9	44.6
  FSRW	14.2	23.6	29.8	36.5	35.6	12.3	19.6	25.1	31.4	29.8	12.5	21.3	26.8	33.8	31.0
  TFA	25.3	36.4	42.1	47.9	52.8	18.3	27.5	30.9	34.1	39.5	17.9	27.2	34.3	40.8	45.6
  FSDetView	24.2	35.3	42.2	49.1	57.4	21.6	24.6	31.9	37.0	45.7	21.2	30.0	37.2	43.8	49.6
  DeFRCN (Our Paper)	40.2	53.6	58.2	63.6	66.5	29.5	39.7	43.4	48.1	52.8	35.0	38.3	52.9	57.7	60.8
  DeFRCN (This Repo)	43.8	57.5	61.4	65.3	67.0	31.5	40.9	45.6	50.1	52.9	38.2	50.9	54.1	59.2	61.9

• Note that we change the λ^GDL-RCNN for VOC to 0.001 (0.01 in paper) and get better performance, check the configs for more details.

• The results of mAP^base and mAP^all for G-FSOD are list here GoogleDrive, BaiduYun.

• If you have any problem of above results in this repo, you can download configs and logs from GoogleDrive, BaiduYun.

Acknowledgement

This repo is developed based on TFA and Detectron2. Please check them for more details and features.

Citing

If you use this work in your research or wish to refer to the baseline results published here, please use the following BibTeX entries:

@inproceedings{qiao2021defrcn,
  title={DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection},
  author={Qiao, Limeng and Zhao, Yuxuan and Li, Zhiyuan and Qiu, Xi and Wu, Jianan and Zhang, Chi},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={8681--8690},
  year={2021}
}

defrcn's Issues

R-101 not work on detectron2 v0.5

I'm sorry I encountered the following problem: I tried to train the network using my custom dataset, however ResNet101 doesn't work, an error is as follows:
ERROR [11/06 15:55:17 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "d:\python_project\detectron2\detectron2\engine\train_loop.py", line 149, in train
self.run_step()
File "d:\python_project\detectron2\detectron2\engine\train_loop.py", line 273, in run_step
loss_dict = self.model(data)
File "D:\CodingTools\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "D:\python_project\condaproject\20211105_fs\DeFRCN-main\defrcn\modeling\meta_arch\rcnn.py", line 53, in forward
proposal_losses, detector_losses, _, _ = self.forward_once(batched_inputs, gt_instances)
File "D:\python_project\condaproject\20211105_fs\DeFRCN-main\defrcn\modeling\meta_arch\rcnn.py", line 79, in forward_once
proposals, proposal_losses = self.proposal_generator(images, features_de_rpn, gt_instances)
File "D:\CodingTools\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "d:\python_project\detectron2\detectron2\modeling\proposal_generator\rpn.py", line 477, in forward
proposals = self.predict_proposals(
File "d:\python_project\detectron2\detectron2\modeling\proposal_generator\rpn.py", line 503, in predict_proposals
return find_top_rpn_proposals(
File "d:\python_project\detectron2\detectron2\modeling\proposal_generator\proposal_utils.py", line 103, in find_top_rpn_proposals
raise FloatingPointError(
FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.
[11/06 15:55:17 d2.engine.hooks]: Total training time: 0:00:01 (0:00:00 on hooks)
[11/06 15:55:17 d2.utils.events]: iter: 2 total_loss: 2.317e+23 loss_cls: 8.796e+22 loss_box_reg: 1.438e+23 loss_rpn_cls: 3.45e+15 loss_rpn_loc: 3.017e+16 data_time: 0.9142 lr: 3.998e-05 max_mem: 2848M

Obviously the value of loss function is abnormally large.
However, when I replaced the backbone with R-50, everything worked fine
May I ask what might have caused this,thanks.
By the way, I have tried using R-101 to extract the characteristics of my custom dataset on detectron2 V0.5 and it works.

A simple question abount the coco dataset images configuration

Thanks for your great work! It's a simple yet effective few-shot object detection method!
I have a simple question when I'm building the project dataset:
coco dataset has three sub-directories, respectively called trainval2014, val2014, and annotations. Does the sub-directory trainval2014 mean that this sub-directory contains all training and evaluating images in it? If so, this sub-directory (trainval2014) is used for training, which means the model can see evaluating images information during training. Is this OK?
Hope for your reply sincerely!

ablation experiment 消融实验

How to do ablation experiment? Turn enable_ Can GDL be closed if decouple is changed to false? PCB_ Can the PCB be closed if enable is changed to false?
RPN: ENABLE_DECOUPLE: True BACKWARD_SCALE: 0.0 FREEZE: False ROI_HEADS: ENABLE_DECOUPLE: True BACKWARD_SCALE: 0.001 NUM_CLASSES: 5 FREEZE_FEAT: True CLS_DROPOUT: True

question about backbone

I have great respect for your excellent work. and I have some questions.
You provide two models contains a ".pkl" and a ".pth". Noticing that these models are pre-trained on ImageNet, I'd like to know whether the ".pkl" model is converted from ".pth". I use your model get the proposed performance. However, I try to convert "pth" to "pkl" and train the base classes, but can not get the same performance.
Could you please give some suggestions?

Difference Between ’Remove‘ and ’Randinit‘

1. I am experimenting on my custom dataset. When I initialize with the ’Remove‘ setting, I can reach 40 nAP, but when I initialize with the 'Randinit' setting, the nAP is close to 0. Why is this?
2. I noticed that your code is different from TFA. The combine operation is not used in 'Remove', which is equivalent to randomly initializing the weights of the novel and base classes. Why do it？ Looking forward to your reply.

loss explosion

Thanks for sharing the source codes.

I met loss explosion during base_training phase, as FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.
I used 2 gpu with batch_size 4, lr=0.01. When i set extremely low value, training goes well.
I think lr=0.01 is not that high value to train the object detector. Is there any bug?

could you share the checkpoint files?

Thanks for nice paper, nice codes.
I just want to reproduce the experimental result and fine-tuning process.
Could you share the checkpoint files? (both base-training and Fine-tuning)

Question about Figure 4 in the Paper

Hello, Thanks again for your contribution in FSOD work.

In the paper, there is a study on the degree of contribution of RPN GDL and RCNN GDL towards the performance, which is summarized in Figure 4.

I would like to know what is the metric for these numbers? Are they AP50 for VOC or mAP for COCO? (are these numbers from VOC or COCO)? Thanks.

purpose of the 'class dropout'

Nice paper and thanks for the source code!
I noticed that you use 'class dropout' which is not mentioned in your paper.
What's the purpose of the dropout function with RoI-feature?

Thx,

Importance of w,w/o FPN in DeFRCN

Hey there! Thanks a lot for the quality codes.

I'm wondering why you guys have taken out FPN from FRCN.
Have you guys done the same experiments with FPN? (wondering if GDL or PCB were afftected by it or all that jazz...)

I couldn't find any details about this in the paper.

The weight of Affine Matrix

Thanks for your great contribution!

I am very curious about the Affine Matrix. I found that the weights of Affine Matrix are not saved in the trained model. Whenever I re-load the trained model, the value of the affine matrix is 1 (initial value).

Am I doing something wrong? Or the Affine Matrix is only used for weighted back-propagation?

visualization

Does your code include the visualization program for external input of a single image for detection? If not, can you provide it? Thank you very much.

Unable to reproduce the results

Hi, I see that someone has reproduced your experimental results, but I haven't been able to reproduce them. I would like to ask if the version of python, cuda, pytorch has a big impact on the results?
Because I use a 3090 graphics card, the version of cuda and pytorch cannot be consistent with the author.
cuda==11.0 torch==1.7 detectron2==0.3
Only the number of GPUs is modified in the code, and the others are not modified. The base model provided by the author is used in the reproduction process, and the experimental results of the reproduction are quite different from the author. And it fluctuates a lot.

FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.

iter: 1 total_loss: 4.02e+06 loss_cls: 3.218e+06 loss_box_reg: 4.646e+05 loss_rpn_cls: 1.197e+05 loss_rpn_loc: 2.168e+05 data_time: 9.5348 lr: 2e-05 max_mem: 5626M
FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.

I experimented on COCO dataset with default settings，and the above error occurred.

Predicted boxes or scores contain Inf/NaN. Training has diverged

When trying to run the training code, I encountered the following error:

In the previous process, my steps were:

• Refer to the readme, install torch, detectron and other environment dependencies based on conda

• Data preparation: vocdata in readme is used
  

• Modify operating errors：
  1、in the run_*.sh ,change into
  
  2、as the picture shows（Refer to：https://giters.com/fanq15/FewX/issues/40?amp=1）
  

• Intercept part of the training code and try to start training
  

then i meet the error：Predicted boxes or scores contain Inf/NaN. Training has diverged。I got the same error when I tried on two different servers (cuda10.2, cuda11.1 both torch1.8)

I tried to modify the learning rate-related settings（to different value , vary from 0.00005 to 0.95） in the configuration file, but no effect

Unable to reproduce the results on Pascal VOC datasets

Hi, I tried to reproduce the G-FSOD results on Pascal VOC using the default config, but I am not able to reproduce the 2/3/5/10 shot results. Only the 1-shot result can be reproduced. The only change I made is the GPU number, which is changed from 8 to 2. Batch size and other config are kept unchanged.

G-FSOD, split 1, average results of seed 0-9

shot	reported AP (github repo)	reproduced AP
1-shot	43.8	41.95
2-shot	57.5	44.24
3-shot	61.4	45.09
5-shot	65.3	46.70
10-shot	67.0	47.51

training_script.txt
results.txt

Is there anything wrong with my experiment? Also, could you please provide a 10-shot trained model? Thank you very much.

Whether the PCB is used ？

Hello, author. I found "_CC.TEST.PCB_ENABLE = False" in "defrcn/config/defaults.py", does it mean that PCB is not used in the inference process ?
When it is False, "outputs = pcb.execute_calibration(inputs, outputs)" is not executed in "evaluator.py".
Hope to get a reply soon. thanks.

Train and test on my own dataset

Thanks for sharing this great model !

I would like to train your model on my own annotated data. I still there are a few tweaks I need to make here and there, including in DeFRCN/defrcn/data/builtin.py to take into account my data and also in run_coco.sh. Could you please provide a few guidelines on how I could train your model only once using all my training data and test it on unseen images.

That would help me a lot (and I guess other people as well).
Any help would be greatly appreciated !

How do I find your core code of GDL and PCB?

Great job, how do I find your core code, or where are the major contributions of GDL and PCB code? Thank you so much!

What does 'if final, 10 -> 1 2 3 5 10' in run_voc.sh mean?

Hi, thank you for your good work.
When i run FSRW-like fine-tuning in run_voc.sh （only line24~line41）, I noticed that after repeat_id 9 and shot 10, my machine went back to repeat_id 0 and shot 1 and started another round of fine-tuning. It doesn't make sense. Does it have anything to with the note 'if final, 10 -> 1 2 3 5 10' in run_voc.sh? Would you help me please?

detectron2 0.5

using cuda11.1 torch 1.8 or 1.9 and detectron2 0.4 or 0.5 .
When I run the main , all setting have this problem :
segfault occurs and the core has been dumped.

About PCB module

I was wondering where is pcb module. Am I missing something?

how can I get detection results?

I want visualize the predicted image, can you share the demo for visualization? thank you very much.

How long will the code be released?

Thanks for your great work! The 10-shot COCO nAP can achieve 18 :), by the way, I think DeFRCN is the combination of Transfer-Learning based FSOD and Meta-Learning based FSOD.

the code to find the similarity

Hello, first of all, thank you for your work! Where is the code to find the similarity between the images in the support set and the query set? thank you!

cocosplit/datasplit/trainvalno5k

If I use my own datasets, how to generate the split data?

GFSOD

I would like to ask why the AP50 of my run out of GFSOD are 60~70?

k-shot采样策略

k-shot采样策略分为采样k个包含某一类别的图片和采样k个包含某一类别的标记框，请问这篇文章用的是哪一类方法？

Experimental setting is different from TFA and FSRW

In TFA/FSRW, FSOD fine-tune on both base and novel classes, using the same training examples provided by FSRW. And G-FSOD train for multiple runs on different random samples to obtain averages.

puzzled about learning rate of RPN

Dear Authors,

Thanks for your great work. I have puzzled the learning rate of RPN.

As stated in the paper, "Moreover, the λ in GDL of RPN is set to 0 for stopping gradient", does it mean that the RPN is never learnt? If the RPN is never learnt, which way is used to initialize it ?

Best!

GPU、

problem in Running novels

Thank you for sharing your code

I used of DeFRCN code (python3.7 and detectron 0.3, cuda 10.2) but I had AP base 12 on my own data, I downloaded pretrain model and replace path into run_coco.sh but I had AP base 12 on my own data and in novels result.txt is empty and i have this Error:
can you help me?

Traceback (most recent call last): File "tools/extract_results.py", line 59, in main() File "tools/extract_results.py", line 34, in main results.append([fid] + [float(x) for x in res_info.split(':')[-1].split(',')]) File "tools/extract_results.py", line 34, in results.append([fid] + [float(x) for x in res_info.split(':')[-1].split(',')]) ValueError: could not convert string to float: " ResNet.make_stage(first_stride=) is deprecated! Use 'stride_per_block' or 'stride' instead."

How to used in the customer dataset?

Hello,i want use the model in my customer dataset. But the result is not so good just used the provided model in this hub.
Any advice to fit my own dataset?
Thank you.

What if 'RCNN Head' is trainable as well?

Nice paper, thanks for releasing your source code.
I got impressed that the cnn backbone is trainable in your theme and make effect.
But what if 'RCNN Head' is trainable as well?
Have you ever done about it? or is there any reason that you freeze this module?

Why TFA links point to cvpods？

In
"Acknowledgement
This repo is developed based on TFA and Detectron2."

Why TFA links point to cvpods？The TFA is the TFA model(Frustratingly Simple Few-Shot Object Detection) or cvpods？

If i use my own datasets, the structure of my datasets？

` ...
datasets
| -- coco (trainval2014/.jpg, val2014/.jpg, annotations/*.json)

...
`
The structure is ok?

Change batch size

Dear Author,
I changed batch size from 16 to 2 and LR from 0.02 to 0.0025 because i want to run your code on one GPU, so How i should change steps and iteration to have same result? please help me for that.

Thanks

When using other pretrained ResNet backbone to train DeFRCN, the performance of novel classes few-shot detection drops a lot

Thanks for your great work!
I notice that you used MSRA's pretrained ResNet-101 as your initial weights. When I changed it to MSRA's pretrained ResNet-50 to train DeFRCN on base classes and novel classes, I got normal mAP (I also changed the PCB model to pretrained ResNet-50 of torchvision). The following two pictures are my recovering results using MSRA's ResNet-50 as backbone, first one is PASCAL VOC split 1 base classes evaluation results, second one is PASCAL VOC split 1 shot 1 seed 0 repeat 0 fsod fsrw-like results.


Then I changed the pretrained backbone to ResNet-50 of torchvision. I also noticed that the input channel order of ResNet-50 is RGB, so I changed this item and corresponding pixel normalized mean and std value in Detectron2's config file. Following the instruction of Detectron2, I also chage the item MODEL.RESNETS.STRIDE_IN_1X1 to False (because I used torch model pretrained weights). After above operations, I used Detectron2's detectron2/tools/convert-torchvision-to-d2.py to convert torchvision's pretrained .pth file to .pkl file, the used the converted .pkl file as pretrained backbone weights. The following two pictures are my results using torchvision's ResNet-50 as backbone, first one is PASCAL VOC split1 base classes evaluation results, second one is PASCAL VOC split1 shot 1 seed 0 repeat0 fsod fsrw-like results.


It is very curious that the mAP of base traing phase is normal when using both MSRA and torchvision's ResNet-50 pretrained model, but the novel classes few-shot performance using torchvision one is much lower than the MSRA one. Ohter shots/seeds/repeats are same. What caused this phenomenon? Is there anything I ignored?

Otherwise, If I don't use Detectron2's detectron2/tools/convert-torchvision-to-d2.py to convert torchvision's pretrained .pth file to .pkl file (this means the model directly loads weights from torchvision's ResNet-50 .pth file), the mAP of base training dropped a lot, What are the reasons for this problem?

Sincerely waiting for your response!

Results of not using w/G in Table 2

作者您好！在表2 的结果中，我想问一下对于coco 不使用G-FSOD的结果（9.3，12.9，14.8，16.1，18.5，22.6），是采用FSOD（FSRW-like、TFA-like）的哪一种结果？第一种吗？

Json file

Hello, thank you very much for your work. I noticed that the base class and new class in the file are the same JSON file. How to distinguish the base class and new class during training and testing? In addition, feed shot only has the name and address of the database. Where can I download the corresponding JSON file address?

ValueError: could not convert string to float

i tried to bash run_voc.sh:

Traceback (most recent call last):
  File "tools/extract_results.py", line 59, in <module>
    main()
  File "tools/extract_results.py", line 34, in main
    results.append([fid] + [float(x) for x in res_info.split(':')[-1].split(',')])
  File "tools/extract_results.py", line 34, in <listcomp>
    results.append([fid] + [float(x) for x in res_info.split(':')[-1].split(',')])
ValueError: could not convert string to float: ' [Checkpointer] Loading from checkpoints/voc/defrcn/defrcn_det_r101_base1/model_reset_remove.pth ...'

how to solve the problem, thx

How to update the affine layer transformation A?

I saw your pseudo-code on backward function of GDL. However it only return the input's gradient (backbone gradient) for learning in the backbone, meanwhile, the gradient of parameter layer A is None, so how the autograd can update the parameters of channel-wise weights without gradient?

mAP

Hi, Thanks for sharing your Code.
The evaluation metric in your method is mAP. We know that the AP must be between 0 and 1.
Considering that the result of executing your code on COCO in 30 shots mode is equal to 22.6. Does this mean that your model has a detection mAP about 0.22 in real?

Thank you

python tools/extract_results.py Error

Dear Authors,

Thanks for your great work. When I run python tools/extract_results.py, there is an error:

Traceback (most recent call last):
File "tools/extract_results.py", line 64, in
main()
File "tools/extract_results.py", line 39, in main
results.append([fid] + [float(x) for x in res_info.split(':')[-1].split(',')])
File "tools/extract_results.py", line 39, in
results.append([fid] + [float(x) for x in res_info.split(':')[-1].split(',')])
ValueError: could not convert string to float: ' Using a generated random seed 17953176'

Could you give me some advice? Thank you.

GPU CPU

I run the code on 1 GPU(3090),but it is running on cpu.why?

About Res5ROIHeads

Hi! Thanks for your great work!
I have noticed that in the config you used Res5ROIHeads(and C4 features) instead of StandardROIHeads(and FPN features) used by TFA. Is there some special consideration behind this design decision? Or have you done some experiments on whether the model works (or even works better) with FPN?

where is the `PCB` module？

Hi, I am sorry that I can not find the code where the PCB module for re-weighting the box-predictor' cls scores refers to.
Hope someone who knows could help me :), thx.

RuntimeError: The server socket has failed to listen on any local network address. The server socket has failed to bind to [::]:50153 (errno: 98 - Address already in use). The server socket has failed to bind to ?UNKNOWN? (errno: 98 - Address already in use).

I am getting this error while running this command
python3 main.py --num-gpus 8 --config-file configs/voc/defrcn_det_r101_base${SPLIT_ID}.yaml
--opts MODEL.WEIGHTS ${IMAGENET_PRETRAIN}
OUTPUT_DIR ${SAVE_DIR}/defrcn_det_r101_base${SPLIT_ID}

Results of the VOC dataset

Is the result of the base class of the VOC dataset different for each run? Does the difference in the results of the base class have a big impact on the results of the novel class?

AssertionError: Checkpoint checkpoints/voc/defrcn/defrcn_det_r101_base3/model_reset_surgery.pth not found

i used bash run_voc.sh

File "/.../conda_envs/tpami/lib/python3.7/site-packages/fvcore/common/checkpoint.py", line 153, in load
assert os.path.isfile(path), "Checkpoint {} not found!".format(path)
AssertionError: Checkpoint checkpoints/voc/defrcn/defrcn_det_r101_base3/model_reset_surgery.pth not found!

how to solve it? Thx.

Traceback (most recent call last): File "/.../conda_envs/tpami/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap fn(i, *args) File "/.../conda_envs/tpami/lib/python3.7/site-packages/detectron2/engine/launch.py", line 126, in _distributed_worker main_func(*args) File "/.../fsod/DeFRCN-main/main.py", line 61, in main trainer.resume_or_load(resume=args.resume) File "/.../fsod/DeFRCN-main/defrcn/engine/defaults.py", line 290, in resume_or_load self.cfg.MODEL.WEIGHTS, resume=resume File "/.../conda_envs/tpami/lib/python3.7/site-packages/fvcore/common/checkpoint.py", line 227, in resume_or_load return self.load(path, checkpointables=[]) File "/.../conda_envs/tpami/lib/python3.7/site-packages/fvcore/common/checkpoint.py", line 153, in load assert os.path.isfile(path), "Checkpoint {} not found!".format(path) AssertionError: Checkpoint checkpoints/voc/defrcn/defrcn_det_r101_base3/model_reset_remove.pth not found!