domerin0/opensubtitles-parser

download, extract, parse and tokenize the opensubtitles dataset with this script

opensubtitles-parser's Issues

tags

A few tags like < /i> seem to have survived but that is all I found.

a bytes-like object is required, not 'str'

Hey,

Thanks a lot for making this parser. However, when I ran python opensubtitleparser.py, I got the following errors :-

Processing OpenSubtitles/en/Western/1957/4374_89054_119873_310_to_yuma.xml...
a bytes-like object is required, not 'str'
Error in OpenSubtitles/en/Western/1957/4374_89054_119873_310_to_yuma.xml
Processing OpenSubtitles/en/Western/1970/16091_192694_255736_cannon_for_cordoba.xml...
a bytes-like object is required, not 'str'
Error in OpenSubtitles/en/Western/1970/16091_192694_255736_cannon_for_cordoba.xml
Processing OpenSubtitles/en/Western/1970/4870_82230_109417_topo_el.xml...
a bytes-like object is required, not 'str'
Error in OpenSubtitles/en/Western/1970/4870_82230_109417_topo_el.xml
Processing OpenSubtitles/en/Western/1972/13750_206687_272145_jeremiah_johnson.xml...
a bytes-like object is required, not 'str'
Error in OpenSubtitles/en/Western/1972/13750_206687_272145_jeremiah_johnson.xml
Processing OpenSubtitles/en/Western/1972/4748_207956_273772_buck_and_the_preacher.xml...
a bytes-like object is required, not 'str'
Error in OpenSubtitles/en/Western/1972/4748_207956_273772_buck_and_the_preacher.xml

No spaces between words

Have you figured out this issue yet?
Thanks, Lotem

i and l confusion

It seems that something is happening in your conversion, maybe encoding? where i and l are switched.

cia --> cla
tell -> teii
i'll -> i'il

spacing with apostrophe

there are a lot of spacing issues with apostrophe

love's -> love 's
can't -> can 't
we've -> we 've

Bugs under Python 3

The code does not run with Python 3. I got the following error:

Error in OpenSubtitles/en/Action/1982/3182_58119_74046_tron.xml
Processing OpenSubtitles/en/Action/1982/905_41278_50921_conan_the_barbarian.xml...
a bytes-like object is required, not 'str'

Maybe we can this in README file, that the code only runs on Python 2.7.

Error when using `pip install chatterbot`

Command "C:\Users\krist\PycharmProjects\machine_learning_backend\venv\Scripts\python.exe C:\Users\krist\Pych armProjects\machine_learning_backend\venv\lib\site-packages\pip-19.0.3-py3.7.egg\pip install --ignore-instal led --no-user --prefix C:\Users\krist\AppData\Local\Temp\pip-build-env-j8jmcagt\overlay --no-warn-script-loc ation --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel>0.32.0.<0.33.0 Cython cymem>=2.0.2,<2.1.0 preshed>=2.0.1,<2.1.0 murmurhash>=0.28.0,<1.1.0 thinc==7.0.0.dev6" failed with er ror code 1 in None

i get this error, whenever i try to pip install the package

i just ran pip install chattebot

The format has changed

It looks like the format of the files to be downloaded has changed. Now after extracting it is OpenSubtitle/xml/en which is different than what is expected. but even changing that gives error in parsing. Am I right or doing something wrong along the way?