State of the Art Language models and Classifier for Malayalam, which is spoken by the Malayali people in the Indian state of Kerala and the union territories of Lakshadweep and Puducherry
License: MIT License
Jupyter Notebook 99.98%Python 0.02%
nlp-for-malyalam's Introduction
NLP for Malayalam
This repository contains State of the Art Language models
and Classifier for Malayalam, which is spoken by the Malayali people
in the Indian state of Kerala and the union territories of
Lakshadweep and Puducherry.
Hi, I have worked on extracting common crawl data for Malayalam and has made it publicly available. Please check it out and see if it can be used to improve your results.
It has around 8 GB of Raw Malayalam text from the 2020-05 crawl.
@goru001 While cloning the repo, getting the following error message:
Downloading language-model/embeddings.tsv (45 MB)
Error downloading object: language-model/embeddings.tsv (0509f1b): Smudge error: Error downloading language-model/embeddings.tsv (0509f1b101ebcace773f9eccdd9be921002e751a4288804e64d49b410a8a1755): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
Errors logged to /media/athul/ee4ac28b-dca1-4dd5-b1ef-1e0be6e5e867/home/athul/nlp/nlp-for-malyalam/.git/lfs/logs/20200211T154509.772925999.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: language-model/embeddings.tsv: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'``
`