enze5088/wentiansearch

A baseline for WenTianSearch

Hi there 👋

I'm Enze, a research engineer in Natural language processing.

My research interests are in Natural language processing, recommender systems, computational advertising, and information retrieval, and broader interests include comparative learning, multi-modal pre-training model, transfer learning, and their applications in advertising, search, and recommendation.

Coding is probably my favorite thing. If you have anything else want to know, please drop me an email, or comment a message on my Website.

wentiansearch's Issues

对z1z2做normalize的目的

您好，

请教一下，代码中对z1和z2向量做normalize有什么作用？

WenTianSearch/simcse/models.py

Line 202 in b722f54

z1 = F.normalize(z1, p=2, dim=-1)

负样本构建

感谢baseline提供。想请教负样本pair是如何构建的呢？

dropout 2次是哪里进行的？

请教一下，dropout 2次通过一个batch里构造重复样本，这个逻辑在哪里呀？

greater_is_better为何设为True？

看到greater_is_better设置为True，会导致存储的checkpoint是metric最大的的
而metric默认是infoNCE loss，那么是不是会导致存下来的模型不是最优的呢？

    greater_is_better: Optional[bool] = field(
        default=True, metadata={"help": "Whether the `metric_for_best_model` should be maximized or not."}
    )

WenTianSearch/2.train.py

Line 229 in b722f54

greater_is_better: Optional[bool] = field(