zhihu/serank

An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow implementation of SERank model. The code is developed based on TF-Ranking.

serank's Issues

problem about Recurrence experiment in Web30k

Hi, thx for your great job. but I have problem about recurrence experiment in web30k. I download the code, and change data to web30k, ndcg@5 of the last ckpt is only 0.409, which is 0.456 in paper.
I do not change serank.py. and the run_script is

DATA=/path-to-web30k/Fold1

output_dir=outputs
rm -r $output_dir
mkdir $output_dir

python serank.py \
  --train_path=$DATA/train.txt \
  --vali_path=$DATA/vali.txt \
  --test_path=$DATA/test.txt \
  --output_dir=$output_dir \
  --num_features=136 \
  --serank=True \
  --query_label_weight=True

tail of the train log is

I0322 18:19:51.462585 140648353728320 evaluation.py:276] Finished evaluation at 2021-03-22-18:19:51
INFO:tensorflow:Saving dict for global step 100000: global_step = 100000, labels_mean = 0.66818184, logits_mean = -0.27165264, loss = 258.99927, metric/arp = 38.71134, metric/ndcg@1 = 0.38938853, metric/ndcg@10 = 0.43623617, metric/ndcg@3 = 0.39571184, metric/ndcg@5 = 0.40858307, metric/ordered_pair_accuracy = 0.6484862
I0322 18:19:51.462839 140648353728320 estimator.py:2066] Saving dict for global step 100000: global_step = 100000, labels_mean = 0.66818184, logits_mean = -0.27165264, loss = 258.99927, metric/arp = 38.71134, metric/ndcg@1 = 0.38938853, metric/ndcg@10 = 0.43623617, metric/ndcg@3 = 0.39571184, metric/ndcg@5 = 0.40858307, metric/ordered_pair_accuracy = 0.6484862
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 100000: outputs/model.ckpt-100000
I0322 18:19:51.463721 140648353728320 estimator.py:2127] Saving 'checkpoint_path' summary for global step 100000: outputs/model.ckpt-100000

thx for your help.