Pynini is a Python extension module which allows the user to compile, optimize, and apply grammar rules. Rules can be compiled into weighted finite state transducers, pushdown transducers, or multi-pushdown transducers. For general information and a detailed tutorial, see pynini.opengrm.org.
Pynini is primarily developed by Kyle Gorman with the help of contributors. If you use Pynini in your research, we would appreciate if you cite the following paper:
K. Gorman. 2016. Pynini: A Python library for weighted finite-state grammar compilation. In *Proc. ACL Workshop on Statistical NLP and Weighted Automata*, 75-80.
Note that the GitHub respository is a (primarily read-only) mirror to enable bug reports and contributions from outside collaborators. Feel free to submit bugs here.
Pynini depends on:
- A standards-compliant C++ 11 compiler (GCC >= 4.8 or Clang >= 700)
- The most recent version of OpenFst (at
the time of writing, 1.6.7) built with the
far
,pdt
,mpdt
, andscript
extensions (i.e., built with./configure --enable-grm
) and headers - A recent version of re2 (at
the time of writing, tag
2018-03-01
; issuegit checkout 2018-03-01; git pull
from there2
directory to sync to this tag) and headers - Python 2.7 and headers
It is tested with: Ubuntu Linux 14.04 LTS on x86_64, GCC 4.8, Python 2.7.6, Cython 0.27.3.
Execute python setup.py install
. Depending on your environment, you
may need to be superuser while running this command for installation to
complete.
To confirm successful installation, execute python setup.py test
.
Pynini is not regularly tested using Python 3 but it should work with
some modifications, assuming you have Cython (a Python-to-C transpiler).
Minimally, you will want to regenerate pywrapfst.cc
and
pynini.cc
(in the src
directory) like so:
cython -3 --cplus -o pywrapfst.cc pywrapfst.pyx cython -3 --cplus -o pynini.cc pynini.pyx
and then (re)compile as described above. There are still some warts related to the switch from byte to Unicode strings.
Pynini is released under the Apache license. See LICENSE for more information.
See CONTRIBUTING.md for more information. We would greatly appreciate contributions focusing on Python 3 compatibility, but we can’t accept pull request which break Python 2.7 compatibility.
Please contact the author before preparing a pull request.
This is not an official Google product.