andriymulyar/sklearn-oblique-tree

a python interface to OC1 and other oblique decision tree implementations

🌳 Oblique Decision Trees in Python

A python interface to oblique decision tree implementations:

• OC1 (Murthy, Kasif, Salzberg 1994)
• CART-Linear Combinations (Breiman et al, 1984, Chapter 5)

Installation (Python 3)

First install numpy with:

pip install numpy

then run:

pip install git+https://github.com/AndriyMulyar/sklearn-oblique-tree

Use

Trees can be induced with the normal scikit-learn classifier api. For instance:

from sklearn.datasets import load_iris, load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn_oblique_tree.oblique import ObliqueTree

random_state = 2

#see Murthy, et all for details.
#For oblique with consideration of axis parallel
#tree = ObliqueTree(splitter="oc1, axis_parallel", number_of_restarts=20, max_perturbations=5, random_state=random_state)
#
#For multivariate CART select 'cart' splitter
#tree = ObliqueTree(splitter="cart", number_of_restarts=20, max_perturbations=5, random_state=random_state)

#consider only oblique splits
tree = ObliqueTree(splitter="oc1", number_of_restarts=20, max_perturbations=5, random_state=random_state)

X_train, X_test, y_train, y_test = train_test_split(*load_iris(return_X_y=True), test_size=.4, random_state=random_state)

tree.fit(X_train, y_train)

predictions = tree.predict(X_test)


print("Iris Accuracy:",accuracy_score(y_test, predictions))

Acknowledgements

VCU Imbalanced Learning and Data Stream Mining Laboratory

Original (unmodified) OC1 Source Code

https://github.com/AndriyMulyar/sklearn-oblique-tree/tree/412d502c04d66046388e469a329d8bcf195bf34b/oc1_implementation

sklearn-oblique-tree's Issues

Add interface for splitting criteria

The ability to switch between impurity metrics is implemented in the underlying implementation. Interface it with python.

Add visualization.

The original implementation contains an excellent visualization tool. Extend it into the python interface.

Can't access the number of tree nodes

Thank you for providing this useful package of the oblique trees. It's really useful to some academic users like me who currently try to develop a new multivariate tree model. Although you provide the classification results of the oblique tree, it is a little pity that I can't access the number of tree nodes. The number of tree nodes is a critical feature for the comparative studies of tree models. So it will be wonderful for academic users like me if you can provide the information on the number of tree nodes in future versions.

Install Error

Windows10, 64bit, Python2.7 (anaconda)

pip install git+https://github.com/AndriyMulyar/sklearn-oblique-tree --no-deps

C:\Users\bscully1\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\ProgramData\Anaconda2\lib\site-packages\numpy\core\include -I. -IC:\ProgramData\Anaconda2\include -IC:\ProgramData\Anaconda2\PC /Tcsklearn_oblique_tree/oblique/_oblique.c /Fobuild\temp.win-amd64-2.7\Release\sklearn_oblique_tree/oblique/_oblique.obj -w
cl : Command line warning D9025 : overriding '/W3' with '/w'
_oblique.c
c:\programdata\anaconda2\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
c:\users\bscully1\appdata\local\temp\1\pip-hf3jgq-build\sklearn_oblique_tree\oblique\../../oc1_source/mktree.c(1432) : error C2059: syntax error : ')'
c:\users\bscully1\appdata\local\temp\1\pip-hf3jgq-build\sklearn_oblique_tree\oblique\../../oc1_source/mktree.c(1439) : error C2440: '=' : cannot convert from 'double *' to 'double'
error: command 'C:\\Users\\bscully1\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2

Then trying python setup.py install

running install
running bdist_egg
running egg_info
writing requirements to sklearn_oblique_tree.egg-info\requires.txt
writing sklearn_oblique_tree.egg-info\PKG-INFO
writing top-level names to sklearn_oblique_tree.egg-info\top_level.txt
writing dependency_links to sklearn_oblique_tree.egg-info\dependency_links.txt
reading manifest file 'sklearn_oblique_tree.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'sklearn_oblique_tree.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
running build_ext
building 'sklearn_oblique_tree.oblique._oblique' extension
C:\Users\bscully1\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\ProgramData\Anaconda2\lib\site-packages\numpy\core\include -I. -IC:\ProgramData\Anaconda2\include -IC:\ProgramData\Anaconda2\PC /Tcsklearn_oblique_tree/oblique/_oblique.c /Fobuild\temp.win-amd64-2.7\Release\sklearn_oblique_tree/oblique/_oblique.obj -w
cl : Command line warning D9025 : overriding '/W3' with '/w'
_oblique.c
c:\programdata\anaconda2\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
c:\users\bscully1\downloads\sklearn-oblique-tree-master\sklearn-oblique-tree\sklearn_oblique_tree\oblique../../oc1_source/mktree.c(1432) : error C2059: syntax error : ')'
c:\users\bscully1\downloads\sklearn-oblique-tree-master\sklearn-oblique-tree\sklearn_oblique_tree\oblique../../oc1_source/mktree.c(1439) : error C2440: '=' : cannot convert from 'double *' to 'double'
error: command 'C:\Users\bscully1\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe' failed with exit status 2

Add predict_prob

Implement predict_prob with something like laplace smoothing. This is trivial but will support the use of the classifier directly in scikit-learns random forest which averages class predictions over-top of class probability not discrete counts.