gidariss/fewshotwithoutforgetting

fewshotwithoutforgetting's Issues

Test the model on own dataset.

Ok lets assume that ia have created the pickles files with the data for the training phase. What parameters should i change in order to match my number of base and novel class?

Reminder: installation tips

Lots of you may be using pytorch 1.0++. But using the required version can save you a lot of time. Using pytorch 1.2, but you need to spend a lot of time on fixing bugs.
To sucsefully run this code, plz download cu75/torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
and

pip install torch*whl
pip install torchvision=0.2.0

The above versions have been tested on RTX2080Ti without modifying any lines of the code.
Also pytorch0.4 also works.

Error when training

When executing the command below:
CUDA_VISIBLE_DEVICES=0 python train.py --config=miniImageNet_Conv128CosineClassifier

It prompts:

Exception KeyError: KeyError(<weakref at 0x7f619db132b8; to 'tqdm' at 0x7f619db23090>,) in <bound method tqdm.__del__ of
  0%|                                                                 | 0/2000 [00:00<?, ?it/s]> ignored
Traceback (most recent call last):
  File "train.py", line 110, in <module>
    algorithm.solve(dloader_train, dloader_test)
  File "/teamscratch/msravcshare/v-weijxu/code/few-shot/DynamicFewShot/algorithms/Algorithm.py", line 286, in solve
    eval_stats = self.evaluate(data_loader_test)
  File "/teamscratch/msravcshare/v-weijxu/code/few-shot/DynamicFewShot/algorithms/Algorithm.py", line 330, in evaluate
    eval_stats_this = self.evaluation_step(batch)
  File "/teamscratch/msravcshare/v-weijxu/code/few-shot/DynamicFewShot/algorithms/FewShot.py", line 84, in evaluation_ste
p
    return self.process_batch(batch, do_train=False)
  File "/teamscratch/msravcshare/v-weijxu/code/few-shot/DynamicFewShot/algorithms/FewShot.py", line 87, in process_batch
    process_type = self.set_tensors(batch)
  File "/teamscratch/msravcshare/v-weijxu/code/few-shot/DynamicFewShot/algorithms/FewShot.py", line 60, in set_tensors
    nKnovel = 1 + labels_train.max() - self.nKbase
RuntimeError: Expected object of type torch.cuda.LongTensor but found type torch.LongTensor for argument #3 'other'

Environment:
Python 2.7
PyTorch 0.4 @ CUDA 9.1

Training issue with Pytorch1.1, python2.7

Hi
Thanks for sharing the code. I have some problems when i run the second stage:
CUDA_VISIBLE_DEVICES=0 python train.py config=miniImageNet_Conv128CosineClassifierGenWeightAttN1.
The erros is : File "/xulan/code/few_shot_withoutforgetting /FewShotWithoutForgetting-master/algorithms/FewShot.py", line 65, in set_tensors
self.tensors['labels_train_1hot'].resize_(labels_train_1hot_size).fill_(0).scatter_(
I check the labels_train_1hot_size, it is [8, 5, tensor([5, 5, 5, 5, 5, 5, 5, 5])]. I think there might be some issue tensor([5, 5, 5, 5, 5, 5, 5, 5]. Do you know what's the problem here. My envirionment is python2.7+pytorch1.1

Obtained test accuracy lower than reported for miniImageNet

Hello , thank you for the implementation of your work. For the first stage, I used the Conv128CosineClassifier configuration. Later with Attention based 5way 1-shot training(untouched configuration), I get 55.9 on novel class and 70.3 on base class for the test set. It is lower than your report. I wonder whether the result I got is reasonable or not?

question of data_train_opt

1. in training stage one, although base category setting is nTestBase =32, batch_size=8 , actual batchsize is 32 *8 as like general training procedure. so why you split the one arg 'batch_size' to two arg 'nTestBase' and 'batch_size'.
2. in training stage two, settings is data_train_opt['nTestNovel'] = nKnovel * 3
   data_train_opt['nTestBase'] = nKnovel * 3, why we need * 3 and i can not find any info in original paper.

it is so kind if you can help me to understand the code. thanks a lot

the code problem in your paper : how to get weight_base

Dear author

We would like to quote your article. After seeing your code in architectures/ClassifierWithFewShotGenerationModule.

I am a little confused about how the weight_base is obtained in class: Classifier in line 112 and185.

In def s init__(self, opt), only initializations are made.

In def get_classification_weights, I don't know how the weight_base is obtained without passing parameters.

FewShotWithoutForgetting/architectures/ClassifierWithFewShotGenerationModule.py

Line 112 in 0efdc78

weight_base = torch.FloatTensor(nKall, nFeat).normal_(

FewShotWithoutForgetting/architectures/ClassifierWithFewShotGenerationModule.py

Line 185 in 0efdc78

weight_base = self.weight_base[Kbase_ids.view(-1)]

Question on whether the process of feature extraction has used the pretrained model on ImageNetFS

Dear author,

Right now I am trying to reproduce the whole process for ImageNet Few-Shot. However, when I go through the second stage, e.g. using pre-trained model in stage1 to extract both train and val features. It seems that I have not found the options for using pretrained model in imagenet_Resnet10CosineClassifier. I wonder if there is anything wrong with my finding?

Hard-coded parameters in evaluate.py leads to errors

In evaluate.py:

dloader_test = FewShotDataloader(
     dataset=MiniImageNet(phase=test_split),
     nKnovel=5, # number of novel categories on each training episode.
     nKbase=64, # number of base categories.
     nExemplars=nExemplars, # num training examples per novel category
     nTestNovel=15 * 5, # num test examples for all the novel categories
     nTestBase=15 * 5, # num test examples for all the base categories
     batch_size=1,
     num_workers=0,
     epoch_size=epoch_size, # num of batches per epoch
)

When evaluating the Proto Nets/Matching Nets, the hard-coded parameters above leads to some error related to missing Kbase_ids in FewShot.py because nKbase is set as non-zero (64). A possible fix is:

data_test_opt  = config['data_test_opt']
dloader_test = FewShotDataloader(
    dataset=MiniImageNet(phase=test_split),
    nKnovel=data_test_opt['nKnovel'], # number of novel categories on each training episode.
    nKbase=data_test_opt['nKbase'], # number of base categories.
    nExemplars=data_test_opt['nExemplars'], # num training examples per novel category
    nTestNovel=data_test_opt['nTestNovel'], # num test examples for all the novel categories
    nTestBase=data_test_opt['nTestBase'], # num test examples for all the base categories
    batch_size=data_test_opt['batch_size'],
    num_workers=args_opt.num_workers, #0
    epoch_size=epoch_size # num of batches per epoch # data_test_opt['epoch_size']
)

Dynamic Few-Shot Visual Learning without Forgetting

It is a really nice job! We tried to use resnet as a backbone just like what you did in our own experiment. However, it didn't improve performance as we expected and even performed worse than 4-conv. I wonder if there are some tricks when you trained your model with a resnet. Thank you very much.

Could you please give me some advices for improving acc_both?

Hi, gidariss:
Thanks for your shared code.
I tested this mechanism for my own feature model and dataset. And got high acc_base and acc_novel except acc_both, could you please give me some advices?

I trained a model on my own dataset(trainset samples > 110k) and got acc_base 91.22%, acc_novel 90.48% and acc_both 74.97% in stage1. And the model is training in stage2 now.
But I found the acc_both is still very low.
And should I use the feature net with best acc_both instead of best acc_novel from stage1 in stage2?

Thanks.

Can you share some details on how to generate the miniImageNet pickle files, I want to test on my own datasets, many thanks.

implementation differs with the reported

Hi, I am studying your code. I found there are some places in the code that are different from what were reported in the paper.

1. There are overlaps between base class and novel classes. In the implementation, some classes are used both as novel class and base class, which is not the case as described in the paper.
2. The weights obtained in the first training stage are not used in the second stage. The weights in the second stage are randomly sampled from a normal distribution, rather than using what were obtained in the first stage.

Could you please give some explanation for your purpose of doing so? Thanks.

Another place that I am unclear is that it seems the novel classes are always the last five classes (labeled with 59~63), which however should be the case in practice.

I will keep reading your code -- maybe I misunderstand something. But your explanations could enlighten me for fully understanding your algorithm. Thanks,

ask for environment details

Thanks for the code! I am wondering if I can have the details of the environment, such as the list of packages the their versions, cuda9 or cuda8 etc.
I am trying to run your code for my school project but I've got too many errors about that...
Thank you.

UnicodeDecodeError when training

Could upload miniImagennet's pickle files for train/val/test

Details about the implementation

Dear authors, I tried to re-implement the paper but failed to get the performance in the paper. I think there may be some details I missed. I trained a cosine-based classifier on base classes (64 classes) and the accuracy is about 57% on validation set. When I extract the feature before classification, which is 3200-d for C128, the 5-way, 1-shot accuracy is only about 46% on test classes and 43% on val classes. Do you have some suggestions on the implementation? Thanks.

link to the dataset is invalid

Hi, the link to the mini-imagenet dataset is invalid now. Could you hep check it and update the link. thanks.

Installation Requirements

Hi, @gidariss could you please list the installation requirements/dependencies to run your code? i.e., python 2.X/3.X, pytorch 0.X, other packages, etc?

A question about the training process

Hello brother, it's really a good job. But what confused me is that in training step 2, when we need to train a weight generator, you keep on training the weight_base, it seems weight_base has already trained well in step 1(pretrian step) , so is there any special reason for this operators? how can we ensure the compatibility between weight_base and generated parameters , and the compatibility between generated parameters?
Also, can such method be used in situations when N is very large(N-way K-shot)? In extreme cases，maybe N is larger than the number of weight_base, If possible, I hope you can give me some suggestions.
Thank you~

Training issue und

The dataset

Who can share the pickle of miniimagenet with me,I can’t download it!

Bias in the Classifier Class

First off, thanks for sharing your code!
I was looking at the Classifier class in ClassifierWithFewShotGenerationModule.py and I realized that self.bias is a scalar rather than a vector. In the simple dot product classifier (or linear classifier), the bias should be a vector with size equal to the number of classes. Am I missing something or is this a bug in the code?

Can you tell me how to get accuracyboth and accuracybase? After running the code, I only get accuracynovel_ CNF

Can you tell me how to get accuracyboth and accuracybase? After running the code, I only get accuracynovel_ CNF

Code release time

It's really an excellent work! Do you have concrete time for releasing the code? Thanks!

ValueError: The provided metric AccuracyNovel for keeping the best model is not computed by the evaluation routine.

Can you give me some advice about this Error?

2019-07-31 08:51:00,403 - algorithms.Algorithm - INFO   - Training: miniImageNet_Conv32CosineClassifier
 20%|█▉        | 197/1000 [00:05<00:21, 37.46it/s]2019-07-31 08:51:06,434 - algorithms.Algorithm 
100%|██████████| 1000/1000 [00:27<00:00, 35.99it/s]
2019-07-31 08:51:28,193 - algorithms.Algorithm - INFO   - ==> Training stats: {'loss': 2.469}
2019-07-31 08:51:28,200 - algorithms.Algorithm - INFO   - Evaluating: miniImageNet_Conv32CosineClassifier
2019-07-31 08:51:28,200 - algorithms.Algorithm - INFO   - ==> Dataset: MiniImageNet_val [2000 batches]
  0%|          | 0/2000 [00:00<?, ?it/s]/data1/zjj/meta-code/withoutForgetting/algorithms/FewShot.py:185: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  images_test_var = Variable(images_test, volatile=is_volatile)
/data1/withoutForgetting/algorithms/FewShot.py:190: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  images_train_var = Variable(images_train, volatile=is_volatile)
100%|██████████| 2000/2000 [01:14<00:00, 26.36it/s]
2019-07-31 08:52:43,083 - algorithms.Algorithm - INFO   - ==> Results: {'loss': 3.1131, 'AccuracyNovel_cnf': 0.4092}
2019-07-31 08:52:43,084 - algorithms.Algorithm - INFO   - ==> Evaluation stats: {'loss': 3.1131, 'AccuracyNovel_cnf': 0.4092}
Traceback (most recent call last):
  File "train.py", line 110, in <module>
    algorithm.solve(dloader_train, dloader_test)
  File "/data1/withoutForgetting/algorithms/Algorithm.py", line 288, in solve
    self.keep_record_of_best_model(eval_stats, self.curr_epoch)
  File "/data1/withoutForgetting/algorithms/Algorithm.py", line 359, in keep_record_of_best_model
    .format(metric_name))
ValueError: The provided metric AccuracyNovel for keeping the best model is not computed by the evaluation routine.

The error occurred in：

    def keep_record_of_best_model(self, eval_stats, current_epoch):
        if self.keep_best_model_metric_name is not None:
            metric_name = self.keep_best_model_metric_name
            if (metric_name not in eval_stats):
                raise ValueError('The provided metric {0} for keeping the best '
                                 'model is not computed by the evaluation routine.'
                                 .format(metric_name))
            metric_val = eval_stats[metric_name]

Thanks a lot！

Obtained test accuracy higher than reported for miniImageNet

Hello , thank you for the implementation of your work. For the first stage, I used the ResNetLikeCosineClassifier configuration and changed the batch size to 6 (due to memory limitation of my gpu). Later with Attention based 5way 1-shot training(untouched configuration), I get 56.89+/-0.83 on novel class and 78.70 on base class for the test set. What I'm wondering is, if there is any additional step involved ?

The first stage training report an error

I have reported the following mistakes in training, if you know how to solve them, please tell me, thank you！

Traceback (most recent call last):
File "I:/scienceresearch/lowshotlearn/05FewShotWithoutForgetting2/FewShotWithoutForgetting/train.py", line 100, in
algorithm = alg.FewShot(config)
File "I:\scienceresearch\lowshotlearn\05FewShotWithoutForgetting2\FewShotWithoutForgetting\algorithms\FewShot.py", line 32, in init
Algorithm.init(self, opt)
File "I:\scienceresearch\lowshotlearn\05FewShotWithoutForgetting2\FewShotWithoutForgetting\algorithms\Algorithm.py", line 24, in init
self.set_log_file_handler()
File "I:\scienceresearch\lowshotlearn\05FewShotWithoutForgetting2\FewShotWithoutForgetting\algorithms\Algorithm.py", line 67, in set_log_file_handler
self.log_fileHandler = logging.FileHandler(self.log_file)
File "E:\anacoda3\lib\logging_init_.py", line 1030, in init
StreamHandler.init(self, self.open())
File "E:\anacoda3\lib\logging_init.py", line 1059, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
OSError: [Errno 22] Invalid argument: 'I:\scienceresearch\lowshotlearn\05FewShotWithoutForgetting2\FewShotWithoutForgetting\experiments\miniImageNet_Conv128CosineClassifier\logs\LOG_INFO_2021-07-21_20:47:21.087383.txt'

fixed branch