The community - driven code for [Natural ** Language ** Processing ** in ** Action](https: // bit.ly / nlpiabook).
A community - developed book about building Natural Language Processing pipelines for prosocial chatbots that contribute to communities.
- Install Python
[Anaconda](https: // docs.anaconda.com / anaconda / install /) or miniconda are easy ways to get python3 installed without messing up your system python on any machine (including Windows and Mac OSX).
Another advantage of the conda
package manager is that it can install some packages that fail with pip, due to binary dependencies like QtPython
for matplotlib
or pyaudio
for SpeechRecognition
.
export OS=Linux # or OS=MacOSX
wget -c "https://repo.continuum.io/miniconda/Miniconda3-latest-${OS}-x86_64.sh" -o $HOME/install_miniconda3.sh
chmod +x $HOME/install_miniconda3.sh
cd $HOME
./install_miniconda.sh -b -p $HOME/miniconda
export PATH=$HOME/miniconda/bin:$PATH
echo 'export PATH=$HOME/miniconda/bin:$PATH' >> $HOME/.bash_profile
bash $HOME/.bash_profile # or just restart your terminal
conda install pip
- Clone this repository
git clone https://github.com/totalgood/nlpia.git
cd nlpia
- Use
conda - env
ORpip
to install dependencies
Depending on your OS you may have better luck using conda to install the dependencies
The environment.yml file creates a conda environment called nlpiaenv
conda env create -n nlpiaenv -f conda/environment.yml
conda install pip # to get the latest version of pip
source activate nlpiaenv
pip install --upgrade pip
mkvirtualenv nlpiaenv
source nlpiaenv/bin/activate
pip install -r requirements-test.txt
pip install -e .
pip install -r requirements-deep.txt
The chatbots(including TTS and STT audio drivers) that come with nlpia
may not be compatible with Windows due to problems install pycrypto
.
If you are on a Linux or Darwin(Mac OSX) system or want to try to help us debug the pycrypto problem feel free to install the chatbot requirements:
# pip install -r requirements-chat.txt
# pip install -r requirements-voice.txt
- Activate this new environment
# source activate nlpia
- Install an "editable"
nlpia
package in this conda environment(also called nlpia)
# pip install -e .
- Check out the code examples from the book in
nlpia / nlpia / book / examples
# cd nlpia/book/examples
# ls
Help your fellow readers by contributing to your shared code and knowledge. Here are some ideas for a few features others might find handy.
Skeleton code and APIs that could be added to the https://github.com/totalgood/nlpia/blob/master/src/nlpia/transcoders.py:`transcoders.py` module.
def find_acronym(text):
"""Find parenthetical noun phrases in a sentence and return the acronym/abbreviation/term as a pair of strings.
>>> find_acronym('Support Vector Machine (SVM) are a great tool.')
('SVM', 'Support Vector Machine')
"""
return (abbreviation, noun_phrase)
def glossary_from_dict(dict, format='asciidoc'):
""" Given a dict of word/acronym: definition compose a Glossary string in ASCIIDOC format """
return text
def glossary_from_file(path, format='asciidoc'):
""" Given an asciidoc file path compose a Glossary string in ASCIIDOC format """
return text
def glossary_from_dir(path, format='asciidoc'):
""" Given an path to a directory of asciidoc files compose a Glossary string in ASCIIDOC format """
return text
Use a parser to extract only natural language sentences and headings/titles from a list of lines/sentences from an asciidoc book like "Natural Language Processing in Action". Use a sentence segmenter in https://github.com/totalgood/nlpia/blob/master/src/nlpia/transcoders.py:[nlpia.transcoders] to split a book, like NLPIA, into a list sentences.