"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and say 'fourteen' very loudly. Everyone does it." - Geoff Hinton
HyperTools is designed to facilitate dimensionality reduction-based visual explorations of high-dimensional data. The basic pipeline is to feed in a high-dimensional dataset (or a series of high-dimensional datasets) and, in a single function call, reduce the dimensionality of the dataset(s) and create a plot. The package is built atop many familiar friends, including matplotlib, scikit-learn and seaborn. Our package was recently featured on Kaggle's No Free Hunch blog. For a general overview, you may find this talk useful (given as part of the MIND Summer School at Dartmouth).
Click the badge to launch a binder instance with example uses:
or
Check the repo of Jupyter notebooks from the HyperTools paper.
To install the latest stable version run:
pip install hypertools
To install the latest unstable version directly from GitHub, run:
pip install -U git+https://github.com/ContextLab/hypertools.git
Or alternatively, clone the repository to your local machine:
git clone https://github.com/ContextLab/hypertools.git
Then, navigate to the folder and type:
pip install -e .
(These instructions assume that you have pip installed on your system)
NOTE: If you have been using the development version of 0.5.0, please clear your data cache (/Users/yourusername/hypertools_data).
- python 2.7, 3.5+
- PPCA>=0.0.2
- scikit-learn>=0.18.1
- pandas>=0.18.0
- seaborn>=0.8.1
- matplotlib>=1.5.1
- scipy>=0.17.1
- numpy>=1.10.4
- future
- requests
- deepdish
- pytest (for development)
- ffmpeg (for saving animations)
If installing from github (instead of pip), you must also install the requirements:
pip install -r requirements.txt
If you encounter an error related to installing deepdish (hdf5) on a MacOS system, try installing hdf5 directly using homebrew:
$ brew tap homebrew/science
$ brew install hdf5
and then re-start the installation.
Check out our readthedocs page for further documentation, complete API details, and additional examples.
We wrote a short JMLR paper about HyperTools, which you can read here, or you can check out a (longer) preprint here. We also have a repository with example notebooks from the paper here.
Please cite as:
Heusser AC, Ziman K, Owen LLW, Manning JR (2018) HyperTools: A Python toolbox for gaining geometric insights into high-dimensional data. Journal of Machine Learning Research, 18(152): 1--6.
Here is a bibtex formatted reference:
@ARTICLE {,
author = {Andrew C. Heusser and Kirsten Ziman and Lucy L. W. Owen and Jeremy R. Manning},
title = {HyperTools: a Python Toolbox for Gaining Geometric Insights into High-Dimensional Data},
journal = {Journal of Machine Learning Research},
year = {2018},
volume = {18},
number = {152},
pages = {1-6},
url = {http://jmlr.org/papers/v18/17-434.html}
}
## Contributing
[![Join the chat at https://gitter.im/hypertools/Lobby](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/hypertools/Lobby)
If you'd like to contribute, please first read our [Code of Conduct](https://www.mozilla.org/en-US/about/governance/policies/participation/).
For specific information on how to contribute to the project, please see our [Contributing](https://github.com/ContextLab/hypertools/blob/master/CONTRIBUTING.md) page.
## Testing
[![Build Status](https://travis-ci.org/ContextLab/hypertools.svg?branch=master)](https://travis-ci.org/ContextLab/hypertools)
To test HyperTools, install pytest (`pip install pytest`) and run `pytest` in the HyperTools folder
## Examples
See [here](http://hypertools.readthedocs.io/en/latest/auto_examples/index.html) for more examples.
## Plot
import hypertools as hyp hyp.plot(list_of_arrays, '.', group=list_of_labels)
![Plot example](images/plot.gif)
## Align
import hypertools as hyp hyp.plot(list_of_arrays, align='hyper')
### BEFORE
![Align before example](images/align_before.gif)
### AFTER</center>
![Align after example](images/align_after.gif)
## Cluster
import hypertools as hyp hyp.plot(array, '.', n_clusters=10)
![Cluster Example](images/cluster_example.png)
## Describe
import hypertools as hyp hyp.tools.describe(list_of_arrays, reduce='PCA', max_dims=14)
![Describe Example](images/describe_example.png)