Comments (2)
As far as I know KenLM is fundamentally optimized for streaming processing of datasets bigger than RAM, in the order of hundreds of gigabytes, rather than compute parallelism. You would need to reimplement LM estimation from scratch, and that's if Kneser-Ney smoothing even allows for this from an algorithmic perspective (in which case you'd need to replace KenLM with something fundamentally different like a Transformer).
from stt.
Since solving this issue requires a change that is much larger in scope, I'm closing this issue.
from stt.
Related Issues (20)
- Bug: lm_optimize fails due to check failing. HOT 2
- Bug: Segmentation Fault HOT 1
- Bug: Scorer.fill_dictiomary() Python function throws SWIG exception
- Feature request: Multiple Parallel/Concatenatable Models
- Bug: Android couldn`t find libstt-jni.so
- Feature request: Cancel previous workflow actions
- Feature request: Tensorflow 2.0 compatibility HOT 11
- Feature request: add Typescript @types for the WASM bindings
- Bug: --alphabet required with --force_bytes_output_mode off but not accepted as a CLI option HOT 4
- Update `genrate_scorer_package` error message when not given any `checkpoint` HOT 1
- Bug: Update `Python` inside `Dockerfile.build` HOT 1
- Bug: Illegal Hard Instruction on generate_scorer_package
- Improvment: `NotFoundError`: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for `best_dev_checkpoint` HOT 2
- Bug: stt complains libbz2.so.1.0 not found HOT 6
- Bug: "import stt" works in notebook but not in bash command HOT 1
- Feature request: Replace Scorer.KenLM with Scorer.Transform HOT 18
- Bug: Importer `import_librivox.py` can't render absolute path of WAV files in CSV HOT 2
- Bug: Update `set-output` calls for ci pipeline v3 HOT 1
- Upload missing aarch64 and arm32 wheels to PyPi
- Bug: Model zoo seems to be gone HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stt.