RNNLM Toolkit

This is a C++ implementation of RNNLM toolkit that supports three algorithms: standard RNNLM, NCE and BlackOut as described in "Blackout: Speeding up recurrent neural network language models with very large vocabularies, ICLR 2016".

License

All source code files in the package are under Apache License 2.0.

The code is developed and tested on UNIX-based systems with the following software dependencies:

Intel Compiler (The code is optimized on Intel CPUs)
OpenMP (No separated installation is needed once Intel compiler is installed)
MKL (The latest version "16.0.0 or higher" is preferred as it has much better support for tall-skinny sgemms)
Boost library (at least 1.49)
Numactl package (for multi-socket NUMA systems)

Install Intel C++ development environment (i.e., Intel compiler, OpenMP, MKL "16.0.0 or higher". free copies are available for some users)
Enable Intel C++ development environment

source /opt/intel/compilers_and_libraries/linux/bin/compilervars.sh intel64 (pointing to the path of your installation)

sudo yum install boost-devel (on RedHat/Centos)
sudo apt-get install libboost-all-dev (on Ubuntu)

sudo yum install numactl (on RedHat/Centos)
sudo apt-get install numactl (on Ubuntu)

Download the code: git clone https://github.com/IntelLabs/rnnlm
Compile the code: make clean all
Download the data: cd data; .\getsmall.sh or .\get1billion.sh
Run the demo script: cd sandbox; ./example_blackout.sh
Run the code on the 1-billion-word-benchmark: cd billion; ./run_64k.sh or ./run_800k.sh or ./run_1m.sh (please set the ncores=number of physical cores of your machine)

Shihao Ji, S. V. N. Vishwanathan, Nadathur Satish, Michael J. Anderson, Pradeep Dubey, Blackout: Speeding up recurrent neural network language models with very large vocabularies, in International Conference on Learning Representations (ICLR'16), 2016.

For questions and bug reports, please email [email protected]