About the paper

[EdenGabriel]

About the paper

[wjun0830]

Hello Eden!
First of all, thank you for your interest in our work.

Yes, you are right that positives and negatives are distinguished with the ground-truth labels.

However, as the higher-level aligned space is only needed to infer the fine-grained relationship (clip-word) that is utilized to train the attention weights, we do not use the clip-word correlation learner during the inference.

You can find this detail in our main figure (Fig. 2) that the lines linked to the clip-word correlation learner are dotted and also the contour of the box which denotes only used in training.

If have more questions, please feel free.
Thank you.

[EdenGabriel]

Thank you for replying so quickly.

Ah, I was careless, I get it. By the way, can I understand Clip-Word Correlation Learner as an auxiliary strategy to better learn dummy tokens?

Thank you.

[wjun0830]

Dummy tokens are mostly supervised with the saliency scores.
Although the clip-word correlation learner may provide further training guidance to learn the dummy tokens, its main objective is to calibrate the attention between each clip and word.
Thank you

[EdenGabriel]

OK. I see. Thank you.
Best wishes to you.