liu3xing3long/thundergbm

ThunderGBM: Fast GBDTs and Random Forests on GPUs

Documentations | Parameters | Python (scikit-learn) interface

Overview

The mission of ThunderGBM is to help users easily and efficiently apply GBDTs and Random Forests to solve problems. ThunderGBM exploits GPUs to achieve high efficiency. Key features of ThunderGBM are as follows.

• Often by 10x times over other libraries.
• Support Python (scikit-learn) interfaces.
• Supported Operating System(s): Linux.
• Support classification, regression and ranking.

Why accelerate GBDT and Random Forests: A survey conducted by Kaggle in 2017 shows that 50%, 46% and 24% of the data mining and machine learning practitioners are users of Decision Trees, Random Forests and GBMs, respectively.

GBDTs and Random Forests are often used for creating state-of-the-art data science solutions. We've listed three winning solutions using GBDTs below. Please check out the XGBoost website for more winning solutions and use cases. Here are some example successes of GDBTs and Random Forests:

• Halla Yang, 2nd place, Recruit Coupon Purchase Prediction Challenge, Kaggle interview.
• Owen Zhang, 1st place, Avito Context Ad Clicks competition, Kaggle interview.
• Keiichi Kuroyanagi, 2nd place, Airbnb New User Bookings, Kaggle interview.

Getting Started

Prerequisites

• cmake 2.8 or above | gcc 4.8 or above for Linux | CUDA 8 or above

Download

git clone https://github.com/zeyiwen/thundergbm.git
git submodule init cub && git submodule update

Build on Linux

cd thundergbm
mkdir build && cd build && cmake .. && make -j

Build the test cases

git submodule update --init src/test/googletest

Quick Start

./bin/thundergbm-train ../dataset/machine.conf
./bin/thundergbm-predict ../dataset/machine.conf

You will see RMSE = 0.489562 after successful running.

Installation

• Add the require binaries to $PATH (where path_to_cuda is the home directory of cuda, e.g., /usr/local/cuda-9.0/bin/; path_to_mpi is the home directory of MPI, e.g., /opt/openmpi-gcc/bin/)

export PATH="path_to_cuda:$PATH"
export PATH="path_to_mpi:$PATH"

• Build ThunderGBM

cd thundergbm
mkdir build && cd build && cmake .. && make -j

• Run ThunderGBM with MPI

make runtest-mpi

How to cite ThunderGBM

If you use ThunderGBM in your paper, please cite our work (preprint).

@article{wenthundergbm19,
 author = {Wen, Zeyi and Shi, Jiashuai and He, Bingsheng and Chen, Jian and Li, Qinbin},
 title = {{ThunderGBM}: Fast {GBDTs} and Random Forests on {GPUs}},
 journal = {To appear in arXiv},
 year = {2019}
}

Other related paper

• Zeyi Wen, Bingsheng He, Kotagiri Ramamohanarao, Shengliang Lu, and Jiashuai Shi. Efficient Gradient Boosted Decision Tree Training on GPUs. The 32nd IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 234-243, 2018. pdf

Key members of ThunderGBM

• Zeyi Wen, NUS
• Jiashuai Shi, SCUT (a visiting student at NUS)
• Qinbin Li, NUS
• Advisor: Bingsheng He, NUS
• Collaborators: Jian Chen (SCUT), Kotagiri Ramamohanarao (The University of Melbourne)

Other information

• This work is supported by a MoE AcRF Tier 2 grant (MOE2017-T2-1-122) and an NUS startup grant in Singapore.

Related libraries

• ThunderSVM, which is another Thunder serier software tool developed by our group.
• XGBoost | LightGBM | CatBoost | cuML