What are the best query strategies to use as a baseline approach?

[webis-de]

What are the best query strategies to use as a baseline approach?

[chschroeder]

Hi @renebidart,
I am not aware of any comprehensive benchmark of this kind. An exhaustive benchmark is unlikely to exist (unless from one of the larger well-known organization) because active learning experiments can very quickly become computationally expensive since there are a lot of combinations for such a large benchmark.

I would advise to try uncertainty-based methods first (such as BreakingTies). They are computationally cheap and usually provide a strong baseline:
https://aclanthology.org/2022.findings-acl.172.pdf

[renebidart]

Thanks for the quick reply @chschroeder!
And that's a great paper, I'll try out that method.