hzzone/propos

Self-Supervised Learning for Deep Clustering (TPAMI 2022)

propos's Issues

how to reproduce pcl

Hello, how do you implement pcl on the cifar-10 data set, and how to set the parameters (such as the size of the queue, etc.)

Could u please share the config file for STL?

Hi, I have spent a really tough time trying to reimplement BYOL on STL, would u mind sharing the config file for STL, my reproduced performance below only reaches around 60%, which gets stuck at 200 epochs...

About Performance on Imagenetdogs

Hi, I want to ask a small question, the performance I implemented on Imagenetdogs using BYOL/resnet34 only reaches 56% ACC, far lower than the reported 69% ACC, is there anything I miss :)

How to change parameters when num_device =2

First of all, thank you very much for your code, since there are only two Gpus, I would like to ask you if there are any parameters that need to be changed？

How to obtain the clustering result after training?

Dear author,

Thanks for sharing the spherical kmeans implementation in #1.

What confuses me is how to get the final clustering result after training? Maybe send the features or projections from the target encoder to spherical kmeans clustering again?

Besides, which one of the features or projections from the target encoder should be sent to the spherical kmeans algorithm when generating the pseudo labels?

Thanks for answering!

How to train Propos on STL-10 dataset

Propos is a charming work, and thanks to your code. I want to ask how to train Propos on STL-10 cos I wonder if the unlabeled images are used for training. Hope that you could show the code. Thanks.

what does the memory data loader do in the basic_template.py?

I am trying to use ProPos on my custom unlabeled data. I believe I will need to rewrite basic_template.py. Can you explain a bit what deos the memory loader do?

About spherical k-means implementation

Dear author,

Thanks for your outstanding work! The class collision issue caused by negative samples in contrastive clustering has plagued me in my applications, and this article gives me hope to solve the problem.

Since the paper is still in the double-blind review stage, I wanna reproduce it on my own first. There are some points that I do not understand.

• Is there an off-the-shelf implementation of spherical k-means, especially for PyTorch?
• According to the pseudocode, can I implement the positive sampling strategy by simply adding the random noise to the query's feature, and matching it against the key's feature, Instead of finding an existing embedding within the key's neighborhood?

Thanks for answering!

How to train ProPos on the STL-10 dataset?

Thanks for your code!
And could you show me the code about how to train ProPos on the STL-10 datasets?
Thank you so much!

About cifar10 performance on SimSiam

Hi, I reran the experiments for SimSiam, but I can only achieve around 60% ACC, I am not sure where I was wrong, could u please share the config file for training SimSiam? Your response is really appreciated!

Can't Reproduce Result in CIFAR-20

I use config/cifar10_r18_propose.yml and only modify 'dataset' to cifar20. But I can't achieve the result in your paper. Is there anything need to change?

How to generate pseudo label for unlabeled data in STL-10 and use them in PSL term?

Results on CIFAR10 with ResNet34

Hi, thanks for your inspiring work! I have managed to reproduce cifar10 on ResNet18, but I encountered some issues reproducing the results on cifar10 using ResNet34 since my reproduced result with ResNet34 is only 90.4% ACC. Could u please provide the config file for training cifar10 on resnet34? Many thanks!

Training speed of ImageNet-10

Thanks for your great code bases and fast responses! i just have a minor question regarding the training speed of imagenet-10/imagenet-dogs, which i find is very slow in dataloader processing, causing low gpu utilization. Sorry I am not very familiar with training ImageNet, is this a issue that can be improved or a common phenonmenon? I really appreciate your help!

Could you provide the pre-trained models for ProPos on ImageNet-1k, ImageNet-50/100/200

Thanks

I can't get the results in the paper using the pretrained models you provided

I got the ACC 0.92 on cifar10 and 0.57 on cifar20 using the pre-trained model, while the paper had 0.94 and 0.61.
I use the features after projection head from the target encoder (ema-updated one). My transforms are resize(32) and normalize.The kmeans settings are the same as yours.

Finally, could you provide the training config on the stl10, imagenet10, imagenet-dogs, and tiny-imagenet?
Thank you

Got 65% ACC for BYOL on ImageNetdogs

Hi, I am really sorry to bother u, but I only get 65% ACC for BYOL on ImageNetdogs, which is 4% lower than reported in paper, and I have made hard trials to figure out the reason, but still have no idea. Is there anything I miss? Thanks so much!
Here is my config file:

batch_size: 64
num_devices: 4
momentum_base: 0.996
momentum_max: 1.0
momentum_increase: true
dataset: imagenetdogs
eval_metric:
  - nmi
  - acc
  - ari
whole_dataset: true
encoder_name: resnet34
epochs: 1000
feat_dim: 256
hidden_size: 4096
#img_size: 96
img_size: 96
lambda_predictor_lr: 10
learning_rate: 0.05
learning_eta_min: 0.
reassign: 1
save_freq: 100
save_checkpoints: true
shuffling_bn: true
symmetric: true
temperature: 0.5
#use_gaussian_blur: false
use_gaussian_blur: true
warmup_epochs: 50
weight_decay: 0.0005
dist: true
data_resample: true
v2: true
#byol_transform: true
test_resized_crop: true

model_name: propos
cluster_loss_weight: 0.
latent_std: 0.

Here is my performance curve: