script to request and download model NDCG

[github]

script to request and download model NDCG

[hamelsmu]

Is this a duplicate of #66 if so please clarify how this issue is different?

Not sure what you mean by a model statistics file. You can see model evaluation metrics in Weights and Biases. MRR is something you are expected to calculate yourself (this is only a proxy metric for training), however you can feel free to inspect our code for a guideline on how to do this. Of course, I could be misunderstanding your question, so please do let us know if there is something we can clarify 🙇

[celsofranssa]

NDCG is the commonly used metrics in information retrieval systems. However, because relevance judgments are not public, the only way to calculate them is by submitting model_predictions.csv to WB. It was not clear to me how I can check NDCG in W&B if I have only model_predictions.csv.

For this reason, I suggested a request script, which just sends the model_predictions.csv file to W&B and returns the NDCG for each language.

[hamelsmu]

You can submit your model_predictions.csv to W&B without logging a full run on W&B.

wandb.init()
wandb.save()

Read these docs for more details. We won't be able to make this type of script at the moment, but this should provide you with what you are looking for

[celsofranssa]

Great.