richard-peng-xia/lmpt

LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition

lmpt's Issues

Why do I need to input class weight as class count into hinge loss?

class_weights = sfm(torch.from_numpy(np.asarray(mmcv.load(freq_file)['neg_class_freq'])).to(torch.float32).cuda())
hinge_loss = SoftMarginHingeEmbeddingLoss(margin=0.2, class_counts=class_weights)

Why do I need to input class weight as class count into hinge loss? By the way, the class weights is two-dimension which is not suitable for the hinge_loss.

What is this label?

No such file or directory: '../data/voc/voc_labels.txt'.

What is this label?

Loss bug in lmpt training

Hi everyone, could you please help to figure out this error? Thanks!

Traceback (most recent call last):
File "train.py", line 281, in
main(args)
File "train.py", line 223, in main
loss_2 = hinge_loss(x, y)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/host/home/yong/Documents/LMPT/lmpt/csel.py", line 25, in forward
class_weights = (1 / self.class_counts) ** self.gamma / torch.sum((1 / self.class_counts) ** self.gamma, dim=2)
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

Problem about the implementation of lmpt/csel.py.

Some of the code is different from the paper.

1. Computing $n_i$
   In the paper $n_i$ denotes the number of training examples that contain class i. In the code, neg_class_ferq is loaded (the number of negative samples).

2. Computing μ
   In the paper，
   
   In the code
   There is an addition softmax operation.

Can not execute.

because self.class_counts has value 0.

I wondering if there are any issues with the implementation of the CSE loss function? thanks!

Question about the calculation of prediction probability

Hi,

Thanks for your excellent work for long-tailed multi-label visual recognition.

I am little confused about the calculation of prediction probability as shown below (formula 2). For example, let us assume that there exist three labels (i=a,b or c) in an image. We hope the prediction probability of the three labels are high enough. However, the prediction probability is normalized by the sum of all label classes like softmax, which can decline the prediction probability significantly when several labels exist. It seems really strange. Could you please give me some instructions?

Best regards

About data preparation

Thank you for your excellent work, could you please provide the class_freq.pkl file for voc and coco? Looking forward to your reply!