adamingas/ordinalgbt

A package to build Gradient boosted trees for ordinal labels

OrdinalGBT

Introduction

OrdinalGBT, which stands for Ordinal gradient boosted trees, is a Python package that implements an ordinal regression loss function using the lightGBM framework. Ordinal regression is a type of regression analysis used for predicting an ordinal variable, i.e. a variable that can be sorted in some order. LightGBM is a gradient boosting framework that uses tree-based learning algorithms and is designed to be distributed and efficient.

Installation

You can install OrdinalGBT using pip:

pip install ordinalgbt

Usage

Here are a few examples on how to use the LGBMOrdinal class:

1. Fitting the model

from ordinalgbt.lgb import LGBMOrdinal
import numpy as np

# Create the model
model = LGBMOrdinal()

# Generate some data
X = np.random.rand(100, 10)
y = np.random.randint(0, 3, 100)

# Fit the model
model.fit(X, y)

2. Predicting with the model

After fitting the model, you can use it to make predictions:

# Generate some new data
X_new = np.random.rand(10, 10)

# Use the model to make predictions
# the .predict method returns the class prediction rather than raw score or
# probabilities
y_pred = model.predict(X_new)

print(y_pred)

3. Predicting probabilities with the model

The predict_proba method can be used to get the probabilities of each class:

# Use the model to predict probabilities
y_proba = model.predict_proba(X_new)

print(y_proba)

TODOs

• Create XGBoost and Catboost implementations
• Bring test coverage to 100%
• Implement the all-thresholds loss function
• Implement the ordistic loss function
• Create more stable sigmoid calculation
• Experiment with bounded and unbounded optimisation for the thresholds
• Identify way to reduce jumps due to large gradient

ordinalgbt's Issues

Support for Loss Matrix (Cost Sensitive Classification)

Have you thought on the following case, Cost Sensitive Classification:

$$ \boldsymbol{L} = \begin{bmatrix} l \left( y = 1, \hat{y} = 1 \right) & l \left( y = 1, \hat{y} = 2 \right) & \dots & l \left( y = 1, \hat{y} = K \right) \\ l \left( y = 2, \hat{y} = 1 \right) & l \left( y = 2, \hat{y} = 2 \right) & \dots & l \left( y = 2, \hat{y} = K \right) \\ \vdots & \vdots & \vdots & \vdots \\ l \left( y = K, \hat{y} = 1 \right) & l \left( y = K, \hat{y} = 2 \right) & \dots & l \left( y = K, \hat{y} = K \right) \end{bmatrix} $$

Is there a way to implement is where the Gradient and Hessian are easy to calculate for LightGBM?