h2oai/xgboost

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Flink and DataFlow

eXtreme Gradient Boosting

Documentation | Resources | Installation | Release Notes | RoadMap

XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.

What's New

• XGBoost GPU support with fast histogram algorithm
• XGBoost4J: Portable Distributed XGboost in Spark, Flink and Dataflow, see JVM-Package
• Story and Lessons Behind the Evolution of XGBoost
• Tutorial: Distributed XGBoost on AWS with YARN
• XGBoost brick Release

Ask a Question

• For reporting bugs please use the xgboost/issues page.
• For generic questions or to share your experience using XGBoost please use the XGBoost User Group

Help to Make XGBoost Better

XGBoost has been developed and used by a group of active community members. Your help is very valuable to make the package better for everyone.

• Check out call for contributions and Roadmap to see what can be improved, or open an issue if you want something.
• Contribute to the documents and examples to share your experience with other users.
• Add your stories and experience to Awesome XGBoost.
• Please add your name to CONTRIBUTORS.md and after your patch has been merged.
  • Please also update NEWS.md on changes and improvements in API and docs.

License

© Contributors, 2016. Licensed under an Apache-2 license.

Reference

• Tianqi Chen and Carlos Guestrin. XGBoost: A Scalable Tree Boosting System. In 22nd SIGKDD Conference on Knowledge Discovery and Data Mining, 2016
• XGBoost originates from research project at University of Washington, see also the Project Page at UW.

xgboost's Issues

Error : No package 'arrow' found

Hello, I am trying to build xgboost from source but it fails with "No package 'arrow' found" error.

Can somebody pls help ?

Environment info

Operating System: Redhat 7.5 (Maipo) ppc64le (POWER9)
Compiler: gcc 4.8.5
Package used (python/R/jvm/C++): Python 2.7.15, R 3.4.1, OpenJDK 1.8.0_171-b10,

xgboost version used: v0.60 (v0.60-552-g35c6b5a )

If installing from source, please provide

1. The commit hash (git rev-parse HEAD) : 35c6b5a
2. Logs will be helpful (If logs are large, please upload as attachment).

If you are using python package, please provide

1. The python version and distribution : Anaconda2-5.2.0-Linux-ppc64le (Python 2.7.15)
2. The command to install xgboost if you are not installing from source : building from source

If you are using R package, please provide

1. The R sessionInfo() R version 3.4.1 (2017-06-30), Platform: powerpc64le-unknown-linux-gnu (64-bit), Running under: OpenStack
2. The command to install xgboost if you are not installing from source : building from source

Steps to reproduce

1. $ git clone -b h2oai https://github.com/h2oai/xgboost.git
2. $ cd xgboost/
3. $ git submodule init
4. $ git submodule update --recursive
5. $ ./build.sh
6. $ make -f Makefile2 libxgboost
   ...
   -- Found PythonLibs: /opt/anaconda2/lib/libpython2.7.so
   -- Found NumPy: version "1.14.5" /opt/anaconda2/lib/python2.7/site-packages/numpy/core/include
   -- Searching for Python libs in /opt/anaconda2/lib64;/opt/anaconda2/lib;/opt/anaconda2/lib/python2.7/config
   -- Looking for python2.7
   -- Found Python lib /opt/anaconda2/lib/libpython2.7.so
   -- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
   -- Checking for module 'arrow'
   -- No package 'arrow' found
   CMake Error at cmake_modules/FindArrow.cmake:130 (message):
   Could not find the Arrow library. Looked for headers in , and for libs in
   Call Stack (most recent call first):
   CMakeLists.txt:197 (find_package)

• Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-Vv3SDe/pyarrow/build/temp.linux-ppc64le-2.7/CMakeFiles/CMakeOutput.log".
  See also "/tmp/pip-install-Vv3SDe/pyarrow/build/temp.linux-ppc64le-2.7/CMakeFiles/CMakeError.log".
  error: command 'cmake' failed with exit status 1

---

Failed building wheel for pyarrow
Running setup.py clean for pyarrow
Failed to build pyarrow

--> I already have pip 0.12.1 as below.

$ pip list | grep arrow
arrow 0.12.1

$ rpm -qa | grep arrow
python2-arrow-0.8.0-3.el7.noarch

Trying to install pyarrow results in the same error message.

$ pip install pyarrow
...
-- Looking for python2.7
-- Found Python lib /opt/anaconda2/lib/libpython2.7.so
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
-- Checking for module 'arrow'
-- No package 'arrow' found
CMake Error at cmake_modules/FindArrow.cmake:130 (message):
Could not find the Arrow library. Looked for headers in , and for libs in
Call Stack (most recent call first):
CMakeLists.txt:197 (find_package)

- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-Fmq07V/pyarrow/build/temp.linux-ppc64le-2.7/CMakeFiles/CMakeOutput.log".
See also "/tmp/pip-install-Fmq07V/pyarrow/build/temp.linux-ppc64le-2.7/CMakeFiles/CMakeError.log".
error: command 'cmake' failed with exit status 1

Add how much GPU memory was being requested instead of just fail

H2OXGBoostEstimator with cv cannot report validation score correctly

Since I upgraded my h2o python package to 3.18.0.8, the H2OXGBoostEstimator could not give me the score on validation set correctly. The details are following:

Steps to reproduce

# python version: 3.6.4 |Anaconda custom (x86_64)| (default, Dec 21 2017, 15:39:08) 
# [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
import h2o
h2o.init()

# load iris data
data = h2o.load_dataset('iris')

# train the xgboost model with 10-fold cross-validation
from h2o.estimators import H2OXGBoostEstimator
model = H2OXGBoostEstimator(nfolds=10, keep_cross_validation_predictions=True)
model.train(y = 'Species', training_frame = data)

# get model for each fold, and print the score
cv_models = model.cross_validation_models()
for i in range(10):
    model_cv = cv_models[i]
    print (model_cv.r2(train=True, valid=True, xval=True))

The results of code above will be:

Parse progress: |█████████████████████████████████████████████████████████| 100%
xgboost Model Build progress: |███████████████████████████████████████████| 100%
{'train': 0.988033979503409, 'valid': 0.988033979503409, 'xval': None}
{'train': 0.980701555221906, 'valid': 0.980701555221906, 'xval': None}
{'train': 0.9884792314320985, 'valid': 0.9884792314320985, 'xval': None}
{'train': 0.9937521418957451, 'valid': 0.9937521418957451, 'xval': None}
{'train': 0.998061238277609, 'valid': 0.998061238277609, 'xval': None}
{'train': 0.9789531276501665, 'valid': 0.9789531276501665, 'xval': None}
{'train': 0.9909617503865926, 'valid': 0.9909617503865926, 'xval': None}
{'train': 0.9979398347526219, 'valid': 0.9979398347526219, 'xval': None}
{'train': 0.9980672446348753, 'valid': 0.9980672446348753, 'xval': None}
{'train': 0.9973039440210139, 'valid': 0.9973039440210139, 'xval': None}

As you can see, the score on train set and validation set are exactly the same. This does not happen for other estimators. If I am doing this wrong, please correct me. Thank you!

undefined reference to `xgboost::tree::__dmlc_registry_file_tag_updater_gpu_hist2__()'

Hello, I am trying to build xgboost from source with GPU support, but it fails with this error of "undefined reference to `xgboost::tree::dmlc_registry_file_tag_updater_gpu_hist2()'"

Can somebody pls help ?

Environment info

Operating System: Redhat 7.5 (Maipo) ppc64le (POWER9)
Compiler: gcc 7.2
Package used (python/R/jvm/C++): Python 3.6.5, R 3.4.1, OpenJDK 1.8.0_171-b10,

xgboost version used: v0.60 (v0.60-552-g35c6b5a )

If installing from source, please provide

1. The commit hash (git rev-parse HEAD) : 6489d8d
2. Logs will be helpful (If logs are large, please upload as attachment).

If you are using python package, please provide

1. The python version and distribution : Anaconda3-5.2.0-Linux-ppc64le (Python 3.6.5)
2. The command to install xgboost if you are not installing from source : building from source

If you are using R package, please provide

1. The R sessionInfo() R version 3.4.1 (2017-06-30), Platform: powerpc64le-unknown-linux-gnu (64-bit), Running under: OpenStack
2. The command to install xgboost if you are not installing from source : building from source

Steps to reproduce

1. $ git clone -b h2oai https://github.com/h2oai/xgboost.git
2. $ cd xgboost/
3. $ git submodule init
4. $ git submodule update --recursive
5. $ mkdir build && cd build
6. $ cmake .. -DUSE_CUDA=ON -DUSE_NCCL=ON
7. $ make -j 16
   ...
   [ 95%] Built target gpuxgboost
   Scanning dependencies of target runxgboost
   [ 97%] Building CXX object CMakeFiles/runxgboost.dir/src/cli_main.cc.o
   [ 98%] Linking CXX executable ../xgboost
   **CMakeFiles/objxgboost.dir/src/tree/tree_updater.cc.o: In function _GLOBAL__sub_I_tree_updater.cc': tree_updater.cc:(.text.startup+0xd8): undefined reference to** xgboost::tree::dmlc_registry_file_tag_updater_gpu_hist2()'
   collect2: error: ld returned 1 exit status
   make[2]: *** [../xgboost] Error 1
   make[1]: *** [CMakeFiles/runxgboost.dir/all] Error 2
   make: *** [all] Error 2
   ...

Automatically select gpu_hist1 if gpu_hist2 cannot be handled.

Not running on Win 7, 12 core

Hello,
I get "java.lang.AssertionError: Unregistered algorithm xgboost" in R-Studio while trying to run it on win 7, 12 core. Tested with H2o 3.16.02 and nightly build 3.17. Are there any hints available?

nccl2 needs libnccl_static.a but not found for ppc build

http://mr-0xc1:8080/blue/organizations/jenkins/h2o4gpu-ppc64le-cuda9/detail/PR-665/3/pipeline

Scanning dependencies of target gpuxgboost

[ 95%] Linking CXX static library libgpuxgboost.a

[ 95%] Built target gpuxgboost

Scanning dependencies of target runxgboost

[ 97%] Building CXX object CMakeFiles/runxgboost.dir/src/cli_main.cc.o

[ 98%] Linking CXX executable ../xgboost

/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/../../../libnccl_static.a when searching for -lnccl_static

/usr/bin/ld: skipping incompatible /lib/libnccl_static.a when searching for -lnccl_static

/usr/bin/ld: skipping incompatible /usr/lib/libnccl_static.a when searching for -lnccl_static

/usr/bin/ld: cannot find -lnccl_static

collect2: error: ld returned 1 exit status

make[4]: *** [../xgboost] Error 1

make[3]: *** [CMakeFiles/runxgboost.dir/all] Error 2

make[2]: *** [all] Error 2

make[1]: *** [libxgboostp2nccl] Error 2

make: *** [xgboost] Error 2

script returned exit code 2

Add varimp stuff from h2oai back into xgboost as utility

The calc_varimp stuff is very good and general and should be in xgboost. See below how I kinda changed things so no need for h2oai specific stuff. It can be cleaned even more.

This is just clean-up to avoid putting xgboost specific stuff into h2oai and hoping we get it into dmlc.

https://github.com/h2oai/h2o4gpu/blob/master/examples/py/demos/H2O4GPU_GBM.ipynb

xgboost is old version 0.60 or new one 0.71

may you help understand where it is written what versions of software from other company you using, for example, xgboost is old version 0.60 or new one 0.71

For bugs or installation issues, please provide the following information.
The more information you provide, the more easily we will be able to offer
help and advice.

Environment info

Operating System:

Compiler:

Package used (python/R/jvm/C++):

xgboost version used:

If installing from source, please provide

1. The commit hash (git rev-parse HEAD)
2. Logs will be helpful (If logs are large, please upload as attachment).

If you are using python package, please provide

1. The python version and distribution
2. The command to install xgboost if you are not installing from source

If you are using R package, please provide

1. The R sessionInfo()
2. The command to install xgboost if you are not installing from source

Steps to reproduce

What have you tried?

h2o.xgboost doesn't support more than 920+ variables while modelling.

I am getting below error if my predictors are exceeding 920 in h2o.xgboost() version 3.18.05. It was working perfectly fine till 3.17.x version of h2o. Please find error below. Any suggestion or help will be appreciated.

DistributedException from localhost/127.0.0.1:54321: 'null', caused by java.lang.NullPointerException at water.MRTask.getResult(MRTask.java:478) at water.MRTask.getResult(MRTask.java:486) at water.MRTask.doAll(MRTask.java:390) at water.MRTask.doAll(MRTask.java:386) at ml.dmlc.xgboost4j.java.XGBoostScoreTask.runScoreTask(XGBoostScoreTask.java:45) at hex.tree.xgboost.XGBoostModel.makePreds(XGBoostModel.java:367) at hex.tree.xgboost.XGBoostModel.makeMetrics(XGBoostModel.java:343) at hex.tree.xgboost.XGBoostModel.makeMetrics(XGBoostModel.java:337) at hex.tree.xgboost.XGBoostModel.doScoring(XGBoostModel.java:387) at hex.tree.xgboost.XGBoost$XGBoostDriver.doScoring(XGBoost.java:454) at hex.tree.xgboost.XGBoost$XGBoostDriver.scoreAndBuildTrees(XGBoost.java:357) at hex.tree.xgboost.XGBoost$XGBoostDriver.buildModelImpl(XGBoost.java:330) at hex.tree.xgboost.XGBoost$XGBoostDriver.buildModel(XGBoost.java:260) at hex.tree.xgboost.XGBoost$XGBoostDriver.computeImpl(XGBoost.java:250) at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:206) at water.H2O$H2OCountedCompleter.compute(H2O.java:1263) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.NullPointerException at hex.tree.xgboost.XGBoostUtils.dense(XGBoostUtils.java:313) at hex.tree.xgboost.XGBoostUtils.convertChunksToDMatrix(XGBoostUtils.java:281) at ml.dmlc.xgboost4j.java.XGBoostScoreTask.map(XGBoostScoreTask.java:137) at water.MRTask.compute2(MRTask.java:657) at water.MRTask.compute2(MRTask.java:591) at water.H2O$H2OCountedCompleter.compute1(H2O.java:1266) at ml.dmlc.xgboost4j.java.XGBoostScoreTask$Icer.compute1(XGBoostScoreTask$Icer.java) at water.H2O$H2OCountedCompleter.compute(H2O.java:1262) ... 5 more

Error: DistributedException from localhost/127.0.0.1:54321: 'null', caused by java.lang.NullPointerException

git clone --recursive https://github.com/h2oai/xgboost fails

@mdymczyk git clone --recursive https://github.com/h2oai/xgboost fails on

Fetched in submodule path 'cub', but it did not contain 89de7ab20167909bc2c4f8acd397671c47cf3c0d. Direct fetching of that commit failed.

leading to incomplete repo content