liyinxiao/lambdaranknn

LambdaRank Neural Network model using Keras.

LambdaRankNN

Python library for training pairwise Learning-To-Rank Neural Network models (RankNet NN, LambdaRank NN).

Supported model structure

It supports pairwise Learning-To-Rank (LTR) algorithms such as Ranknet and LambdaRank, where the underlying model (hidden layers) is a neural network (NN) model.

Installation

pip install LambdaRankNN

Example

Example on a LambdaRank NN model, with the training data below.

import numpy as np
from LambdaRankNN import LambdaRankNN

# generate query data
X = np.array([[0.2, 0.3, 0.4],
              [0.1, 0.7, 0.4],
              [0.3, 0.4, 0.1],
              [0.8, 0.4, 0.3],
              [0.9, 0.35, 0.25]])
y = np.array([0, 1, 0, 0, 2])
qid = np.array([1, 1, 1, 2, 2])

# train model
ranker = LambdaRankNN(input_size=X.shape[1], hidden_layer_sizes=(16,8,), activation=('relu', 'relu',), solver='adam')
ranker.fit(X, y, qid, epochs=5)
y_pred = ranker.predict(X)
ranker.evaluate(X, y, qid, eval_at=2)

Converting model to pmml

The trained model can be conveniently converted to pmml, with Python library rankerNN2pmml.

from rankerNN2pmml import rankerNN2pmml
params = {
    'feature_names': ['Feature1', 'Feature2', 'Feature3'],
    'target_name': 'score'
}

rankerNN2pmml(estimator=ranker.model, file='Model_example.xml', **params)

lambdaranknn's Issues

AttributeError: module 'tensorflow' has no attribute 'placeholder'

I was trying to run the toy example given in the repo. It is showing below error

Traceback (most recent call last):
File "lamdarank_test.py", line 14, in
ranker = LambdaRankNN(input_size=X.shape[1], hidden_layer_sizes=(16,8,), activation=('relu', 'relu',), solver='adam')
File "/home/user/stellar_env/lib/python3.6/site-packages/LambdaRankNN/init.py", line 243, in init
super(LambdaRankNN, self).init(input_size, hidden_layer_sizes, activation, solver)
File "/home/user/stellar_env/lib/python3.6/site-packages/LambdaRankNN/init.py", line 31, in init
self.model = self._build_model(input_size, hidden_layer_sizes, activation)
File "/home/user/stellar_env/lib/python3.6/site-packages/LambdaRankNN/init.py", line 44, in _build_model
input1 = Input(shape=(input_shape,), name='Input_layer1')
File "/usr/local/lib/python3.6/dist-packages/keras/engine/input_layer.py", line 178, in Input
input_tensor=tensor)
File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/input_layer.py", line 87, in init
name=self.name)
File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 517, in placeholder
x = tf.placeholder(dtype, shape=shape, name=name)
AttributeError: module 'tensorflow' has no attribute 'placeholder'

Evaluation Plots for LambdaRankNN

Hi @liyinxiao,

Firstly great work, made my life easy to use the Learning2Rank models using Keras.
Few questions I got while using the library:

1. what is the loss function that is being used during training process - is it binary_cross_entropy?
2. If yes, to above question, How do I plot the evaluation graphs for training & evaluation sets in terms of ndcg
3. how do I extend the eval metrics to MAP ?

Any guidance or support is appreciated?

Thanks in advance

left shift error

I get this error, please check, does qid need to be particular type?

python3.7

bst7 = LambdaRankNN(input_size=X.shape[1], hidden_layer_sizes=(8,4,), activation=('relu', 'relu',), solver='adam')
qid = np.ones(len(train))
bst7.fit(X[train, :], y[train], qid)

Traceback (most recent call last):
  File "/home/dnachbar/python/lightgbm/zip.py", line 66, in <module>
    bst7.fit(X[train, :], y[train], qid)
  File "/home/dnachbar/.local/lib/python3.7/site-packages/LambdaRankNN/__init__.py", line 121, in fit
    X1_trans, X2_trans, y_trans, weight = self._transform_pairwise(X, y, qid)
  File "/home/dnachbar/.local/lib/python3.7/site-packages/LambdaRankNN/__init__.py", line 287, in _transform_pairwise
    original = ((1 << pos_label) - 1) * pos_loginv + ((1 << neg_label) - 1) * neg_loginv
TypeError: ufunc 'left_shift' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Meaning of the labels

Hello, thanks for sharing the algorithm! I'm just wondering, what do the label numbers actually mean in the example.

Does 0 mean rank position 0; 1 means rank position 1 and so on?

Or 0 actually means irrelevant?

Thanks

Query_id: user_id + item_id

Hi! Not sure how to construct the query_id dataset for a user_id+item_id (2 ids) recommendation problem. Is it possible to do with LambdaRankNN? Thanks!

Not able to save model as pmml

Traceback (most recent call last):
File "/home/administrator/ilabs/GIT/learning2rank/rank/LR_testing.py", line 22, in
rankerNN2pmml(estimator=ranker.model, file='Model_example.xml', **params)
File "/home/administrator/ilabs/GIT/learning2rank/.venv/lib/python3.6/site-packages/rankerNN2pmml/init.py", line 207, in rankerNN2pmml
feature_names = _validate_inputs(estimator, transformer, feature_names)
File "/home/administrator/ilabs/GIT/learning2rank/.venv/lib/python3.6/site-packages/rankerNN2pmml/init.py", line 29, in _validate_inputs
raise TypeError("Provided model is not supported.")
TypeError: Provided model is not supported.