Optimizing CA model toward different metrics

[jjfiv]

Optimizing CA model toward different metrics

[jjfiv]

I added this cell to the bottom of the notebook, but the TrainRequest object has a measure field as a string.

train_request = TrainRequest.coordinate_ascent()
train_request.measure = 'map'
params = train_request.params
params.init_random = True
params.normalize = True
params.seed = 1234567
coordinate_ascent = dataset.train_model(train_request)

# simple python-accessible models (serialization is up to you)
coordinate_ascent.to_dict()

The supported measures are: AP, RR, and NDCG for now. I know ERR is pretty popular in LTR literature, but I've never really used it, myself, so that's why it's been on the backburner.

fastrank/src/evaluators.rs

Lines 148 to 153 in b05cf8a

	evaluator: match name.as_str() {
	"ap" | "map" => Arc::new(AveragePrecision::new(dataset, judgments.clone())),
	"rr" | "mrr" => Arc::new(ReciprocalRank),
	"ndcg" => Arc::new(NDCG::new(depth, dataset, judgments.clone())),
	_ => Err(format!("Invalid training measure: \"{}\"", orig_name))?,
	},

I don't have plans for user-defined functions as metrics, because most of the 'speed' of coordinate ascent comes from how fast you can predict new model scores, sort, and compute your metric, but it shouldn't be hard to add some additional measures.

[mfayoub]

Ok, thanks for your answer.