Why use memory feature for positive samples?

[hobbitlong]

Why use memory feature for positive samples?

[HobbitLong]

Hi,

Good catch! I opted to use positive from the memory bank due to the following two reasons:

(1) consistency: it's consistent with how negative features are obtained;
(2) stability: the features in the memory bank is more stable than those from the current batch. Memory bank is smoothly updated with momentum, while in current batch, the random crop makes the feature less stable.

But I haven't quantitatively measured the performance difference yet.

[LLLLLLI]

Thanks!

I agree that it has advantages of consistency and stability. But it also makes the training procedure convergence slowly cause the model may change a lot in one epoch. I think it's meaningful to find a better way.

Thanks again!

[HobbitLong]

Thanks for inputting your thoughts. I agree with you it definitely deserves to further explore other sampling strategies.

I will close it now. Feel free to reopen if you would like to discuss more!