microsoft/distributed_skipgram_mixture

Distributed skipgram mixture model for multisense word embedding

Distributed Multisense Word Embedding

The Distributed Multisense Word Embedding(DMWE) tool is a parallelization of the Skip-Gram Mixture [1] algorithm on top of the DMTK parameter server. It provides an efficient "scaling to industry size" solution for multi sense word embedding.

For more details, please view our website http://www.dmtk.io

Download

$ git clone https://github.com/Microsoft/distributed_skipgram_mixture

Build

Prerequisite

DMWE is built on top of the DMTK parameter sever, therefore please download and build DMTK first (https://github.com/Microsoft/multiverso).

For Windows

Open windows\distributed_skipgram_mixture\distributed_skipgram_mixture.sln using Visual Studio 2013. Add the necessary include path (for example, the path for DMTK multiverso) and lib path. Then build the solution.

For Ubuntu (Tested on Ubuntu 12.04)

Download and build by running $ sh build.sh. Modify the include and lib path in Makefile. Then run $ make all -j4.

Run

For parameter settings, see scripts/parameters_settings.txt. For running it, see the example script scripts/run.py.

Reference

[1] Tian, F., Dai, H., Bian, J., Gao, B., Zhang, R., Chen, E., & Liu, T. Y. (2014). A probabilistic model for learning multi-prototype word embeddings. In Proceedings of COLING (pp. 151-160).

Microsoft Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

distributed_skipgram_mixture's Issues

CMake Error

cmake .. && make && sudo make install
OpenMP found
/usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so
/usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so
CMake Warning at /usr/share/cmake-3.5/Modules/FindBoost.cmake:725 (message):
Imported targets not available for Boost version
Call Stack (most recent call first):
/usr/share/cmake-3.5/Modules/FindBoost.cmake:763 (_Boost_COMPONENT_DEPENDENCIES)
/usr/share/cmake-3.5/Modules/FindBoost.cmake:1332 (_Boost_MISSING_DEPENDENCIES)
Test/unittests/CMakeLists.txt:3 (find_package)

CMake Error at /usr/share/cmake-3.5/Modules/FindBoost.cmake:1677 (message):
Unable to find the requested Boost libraries.

Unable to find the Boost header files. Please set BOOST_ROOT to the root
directory containing Boost or BOOST_INCLUDEDIR to the directory containing
Boost's headers.
Call Stack (most recent call first):
Test/unittests/CMakeLists.txt:3 (find_package)

CMake Error at Test/unittests/CMakeLists.txt:9 (MESSAGE):
message called with incorrect number of arguments

Boost_INCLUDE_DIR-NOTFOUND
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
/clouddisk/shaolin/DMTK/multiverso/Test/unittests/Boost_INCLUDE_DIR
used as include directory in directory /clouddisk/shaolin/DMTK/multiverso/Test/unittests

-- Configuring incomplete, errors occurred!
See also "/clouddisk/shaolin/DMTK/multiverso/build/CMakeFiles/CMakeOutput.log".

Addition of a build system generator

I suggest to reuse a higher level build system than your current small make file so that powerful checks for software features will become easier.

• CMake
• Autotools

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

• fopen ⇒ MultiversoSkipGramMixture::InitSenseCntInfo
• fprintf ⇒ ParameterLoader<T>::ParseAndRequest

are there pre-trainned models?

can you post your pretrainned models? or are there any available?

it takes a while to train them so it would be nice. if storage is the issue, torrent is fine.

How can I integrate models if training was conducted on multi-machine?

Hello, I trained 3GB text file on 3 machines.(I divided the text file into three 1GB files as instruction in run.py)
After training, output files(models) were made on each 3 machines separately.
How can I integrate those models?

Thank you.