diaoul/subliminal

Subtitles, faster than your thoughts

Subliminal

Subtitles, faster than your thoughts.

‼️ Subliminal is looking for maintainers! See here

Project page:	https://github.com/Diaoul/subliminal
Documentation:	https://subliminal.readthedocs.org/

Usage

CLI

Download English subtitles:

$ subliminal download -l en The.Big.Bang.Theory.S05E18.HDTV.x264-LOL.mp4
Collecting videos  [####################################]  100%
1 video collected / 0 video ignored / 0 error
Downloading subtitles  [####################################]  100%
Downloaded 1 subtitle

Library

Download best subtitles in French and English for videos less than two weeks old in a video folder:

#!/usr/bin/env python

from datetime import timedelta

from babelfish import Language
from subliminal import download_best_subtitles, region, save_subtitles, scan_videos

# configure the cache
region.configure('dogpile.cache.dbm', arguments={'filename': 'cachefile.dbm'})

# scan for videos newer than 2 weeks and their existing subtitles in a folder
videos = scan_videos('/video/folder', age=timedelta(weeks=2))

# download best subtitles
subtitles = download_best_subtitles(videos, {Language('eng'), Language('fra')})

# save them to disk, next to the video
for v in videos:
    save_subtitles(v, subtitles[v])

Docker

Run subliminal in a docker container:

$ docker run --rm --name subliminal -v subliminal_cache:/usr/src/cache -v /tvshows:/tvshows -it diaoulael/subliminal download -l en /tvshows/The.Big.Bang.Theory.S05E18.HDTV.x264-LOL.mp4

Installation

Subliminal can be installed as a regular python module by running:

$ [sudo] pip install subliminal

For a better isolation with your system you should use a dedicated virtualenv or install for your user only using the --user flag.

Nautilus/Nemo integration

See the dedicated project page for more information.

subliminal's Issues

Activating plugins

How do I use the plugin SubScene and Podnapisi ?

My config.ini :

[DEFAULT]
languages = en,fr
plugins = BierDopje,OpenSubtitles,Podnapisi,Addic7ed,SubScene

[SubtitleSource]
key =

What subliminal tells me :

ausset@silmeria:~$ /usr/local/bin/subliminal -m -l en -l fr "series/Scrubs/Season 7"
ERROR:subliminal:Plugin Podnapisi does not exist
ERROR:subliminal:Plugin SubScene does not exist

---

Release 0.5

Create an API

For basic use of subliminal:

• list
• download

Better unicode support

Find a better way to fix encoding issues

KeyError: 'episodeNumber'

Is it possible to return the filename that has generated this error?
So a rename could fix the problem.

Remove try block from downloadFile

Because it hides errors that should be raised up to the worker so it can try to download the next file

Implement with statement

That would automatically manage workers inside the with statement.

Error when no languages set

When no languages are submitted, we use all available languages which can cause trouble for some plugins.

A possible fix could be to check for valid languages in list and download or even before creating the task itself.

Switch to argparse

As optparse is deprecated since 2.7

Add http://www.sous-titres.eu/

Problem installing last version

Subliminal is not really happy with the very last version.

ausset@silmeria:~/subliminal$ sudo python setup.py install
Traceback (most recent call last):
File "setup.py", line 24, in
import subliminal
File "/home/ausset/subliminal/subliminal/init.py", line 34, in
from core import *
File "/home/ausset/subliminal/subliminal/core.py", line 35, in
import plugins
File "/home/ausset/subliminal/subliminal/plugins.py", line 410
def list(self, video, languages):
^
IndentationError: expected an indented block

Thanks

More unittests

• Plugins
  • query
  • list
  • download
• Subliminal
  • list
  • download

Error with kaa.base

After an upgrade Subliminal no longer works, because it is missing kaa.base:
Error:
Traceback (most recent call last):
File "/opt/local/bin/subliminal", line 4, in
import pkg_resources
File "/opt/lib/python2.6/site-packages/pkg_resources.py", line 2603, in
working_set.require(requires)
File "/opt/lib/python2.6/site-packages/pkg_resources.py", line 666, in require
needed = self.resolve(parse_requirements(requirements))
File "/opt/lib/python2.6/site-packages/pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: kaa-metadata>=0.7.7

After some search the update had an failed installation:
Code:
Installed /opt/local/lib/python2.6/site-packages/subliminal-0.5-py2.6.egg
Processing dependencies for subliminal==0.5
Searching for kaa-metadata>=0.7.7
Reading http://pypi.python.org/simple/kaa-metadata/
Reading http://freevo.org/kaa/
Best match: kaa-metadata 0.7.8dev-r4569-20111003
Downloading http://pypi.python.org/packages/source/k/kaa-metadata/kaa-metadata-0.7.8dev-r4569-20111003.tar.gz#md5=8f75f349423c323f872ca1bc20d2461e
Processing kaa-metadata-0.7.8dev-r4569-20111003.tar.gz
Running kaa-metadata-0.7.8dev-r4569-20111003/setup.py -q bdist_egg --dist-dir /tmp/easy_install-EBelGO/kaa-metadata-0.7.8dev-r4569-20111003/egg-dist-tmp-OX3RUB
kaa.base not installed
error: Setup script exited with 1

Looks like it hase newer version of kaa-metadata.

Or is this still under construction???

Two result queues

Split resultQueue in two :

• listResultQueue
• downloadResultQueue

This will give more flexibility when dealing with ListTask and DownloadTask at the same time.
High priority download won't polute your resultQueue with paths

Issues with CSI NY on Bierdopje

When I use subliminal to download srt from bierdopje I always get the following issue on CSI NY:

DEBUG 2011-10-09 14:38:31 subliminal Setting plugins to ['BierDopje']
DEBUG 2011-10-09 14:38:31 subliminal Setting languages to ['nl']
DEBUG 2011-10-09 14:38:31 subliminal Worker Thread-1 added to the pool
DEBUG 2011-10-09 14:38:31 subliminal Worker Thread-2 added to the pool
DEBUG 2011-10-09 14:38:31 subliminal Worker Thread-3 added to the pool
DEBUG 2011-10-09 14:38:31 subliminal Worker Thread-4 added to the pool
DEBUG 2011-10-09 14:38:31 subliminal Listing subtitles set(['nl']) for u'/nfs/volume1/films/Series/CSI NY/Season 08/CSI_NY.S08E02.720p.WEB-DL.DD5.1.H264-NFHD.mkv' with ['BierDopje']
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 1.00: {'container': 'mkv'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 1.00: {'type': 'episode'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 0.80: {'language': 'Chichewa'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 1.00: {'season': 8}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 1.00: {'season': 8, 'episodeNumber': 2}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 1.00: {'videoCodec': 'h264'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 1.00: {'screenSize': '720p'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 0.80: {'language': 'Chichewa'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 0.40: {'series': 'DL DD5 1'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found with confidence 0.40: {'title': 'NFHD'}
DEBUG 2011-10-09 14:38:31 guessit.matcher Found match tree:
0 111 2222222 33333 444444 555555 666666666 77777777777777777777777777777777777777777 888
0 000 0000000 00000 000000 000000 000000000 00000000000000000000000000000000000000000 000
0 000 0000000 00000 000000 000011 000000000 00001123333334555566667777777777888899999 000
//nfs/volume1/films/Series/CSI /_______/CSI___...WEB_________________.___
ll SSSSSSSSS ll EEEEEE ssss TTTTTTTTTTvvvvttttt ccc
/nfs/volume1/films/Series/CSI NY/Season 08/CSI_NY.S08E02.720p.WEB-DL.DD5.1.H264-NFHD.mkv
DEBUG 2011-10-09 14:38:31 guessit.guess Updating matching property 'season' with confidence 1.00
DEBUG 2011-10-09 14:38:31 guessit.matcher Final result: {
[1.00] "episodeNumber": 2,
[1.00] "videoCodec": "h264",
[1.00] "container": "mkv",
[0.80] "language": [
"Chichewa"
],
[0.40] "title": "NFHD",
[1.00] "season": 8,
[1.00] "screenSize": "720p",
[0.40] "series": "DL DD5 1",
[1.00] "type": "episode"
}
DEBUG 2011-10-09 14:38:31 subliminal.BierDopje Reading showids from cache: {u'fringe': u'3350', u'terra nova': u'14227'}
DEBUG 2011-10-09 14:38:31 subliminal.BierDopje Showname: dl dd5 1
DEBUG 2011-10-09 14:38:31 subliminal.BierDopje Retrieving show id from web at http://api.bierdopje.com/A2B638AC5D804C2E/GetShowByName/dl%20dd5%201
DEBUG 2011-10-09 14:38:31 subliminal.worker Poison pill received, terminating thread Thread-2
DEBUG 2011-10-09 14:38:31 subliminal.worker Thread Thread-2 terminated
DEBUG 2011-10-09 14:38:31 subliminal.worker Poison pill received, terminating thread Thread-3
DEBUG 2011-10-09 14:38:31 subliminal.worker Thread Thread-3 terminated
DEBUG 2011-10-09 14:38:31 subliminal.worker Poison pill received, terminating thread Thread-4
DEBUG 2011-10-09 14:38:31 subliminal.worker Thread Thread-4 terminated
DEBUG 2011-10-09 14:38:31 subliminal.worker Poison pill received, terminating thread Thread-1
DEBUG 2011-10-09 14:38:31 subliminal.worker Thread Thread-1 terminated
No subtitles found

As you see the Guessit part is causing the issue

Difference with Periscope ?

Hi, this is just a question. This project seems to be a fork of Periscope, right ?
What is the difference between Periscope and Subliminal ?

Add http://www.u-sub.net/

add language at end of .srt file

Example If you download an english subtitle for Thor.mkv, the subtitlefile will be named Thor.en.srt

Add http://www.seriessub.com/

Use classes for task and subtitle

Task should be an object so it's way more readable.
Same goes for subtitle

Currently, dicts are used. yikes

pip install subliminal doesn't work

pip install subliminal won't work because the setup.py import all the thing (just to retrieve version)

Solve that by creating a version.py with the version information and just import that in the setup.py

Handle timeout localy

This will require the use of another lib. Maybe requests?

Compare using lower (or upper) case only for teams

Or subtitles won't match (FQM != fqm)

Handle .sub subtitles extension

When a .sub subtitle is found on OpenSubtitles (for example) it is not handled correctly

Add http://www.addic7ed.com

Add betaseries.com provider

It's a good one for french srt and it has a API as far as I know

https://www.betaseries.com/

Priority enhancements

• Create a addTask method to add tasks with default priority in subliminal
• (?) Create a self.list_priority and self.download_priority with some default values to set up default priorities for tasks
• Document subliminal priority so it can be used at its full potential

Add getsubtitle.com provider

WebService: http://www.getsubtitle.com/webService/
Suds: https://fedorahosted.org/suds/

Put 1.x versions in 0.x

Because we're far from a steady module right now.

• 1.0 => 0.4
• 1.1 => 0.5

Then I'll start to maintain 0.5.x and switch it to 1.0 as soon as it is steady/stable without adding any feature

This should not impact anyone as no version of subliminal were actually released except some betas for my Sick-Beard.
However, that'll seem strange for OpenSubtitles and TheSubDB that already received 1.0 and 1.1 API requests :D

Configuration file ?

That makes the code heavy, and for what ?
Only useful when used with UI integration which is not the case.

If keeping the configuration file, switch to ConfigObj for better encoding/unicode support but it's better to remove it

Improve performances

• urllib3 or requests (soon) allows multiple requests on the same socket, it can boost performances but may not be easy to implement
• connect to API just one time as we don't need to authenticate each time we want to search for a subtitle. Is that possible in a multithreaded environment?

Refactor recursiveSearch

• Have recursiveSearch actually search for subtitles than for missing subtitles
• Move it to module level (not class) so it can be used by other users
• Have it detect multiple extensions (.sub, .srt) even if subliminal only downloads .srt it is not necessary to download a .srt if a .sub is present

Scan for embedded subtitles

http://pypi.python.org/pypi/hachoir-parser

Permissions on created files

Allow manual setting of permissions for newly created files

Add more unittests

Because it is important.

Support for Brazilian

No language code exists for brazilian in ISO-639-1

Common workaround is pt-br. OpenSubtitles uses pb/pob.

Priority queue

So the workers can be paused (with high priority poison pills)

kaa-base, kaa-metadata not available for win32

Subliminal builds fine on Linux, no problem. It also used to run fine on Windows, but not anymore since you started using kaa-metadata. It seems it is quite difficult to get the kaa libraries working on win32. (cf. http://www.mail-archive.com/[email protected]/msg18713.html)

Accept bytestring, not just unicode

Because if the user wants to pass crap, we shall allow him to do so. Log a warning if non unicode is used though.

Strange error in Subliminal 1.1

DEBUG 2011-11-08 18:07:45 guessit.guess Updating matching property 'season' with confidence 1.00
DEBUG 2011-11-08 18:07:45 guessit.guess Updating matching property 'series' with confidence 0.85
DEBUG 2011-11-08 18:07:45 guessit.matcher Final result: {
[1.00] "episodeNumber": 5,
[1.00] "container": "mkv",
[1.00] "screenSize": "720p",
[0.50] "title": "DL DD5 1 h 264",
[0.85] "series": "Fringe",
[1.00] "season": 4,
[1.00] "type": "episode"
}
DEBUG 2011-11-08 18:07:45 subliminal.BierDopje Initializing
DEBUG 2011-11-08 18:07:45 subliminal.BierDopje Initializing cache...
ERROR 2011-11-08 18:07:45 subliminal.worker Exception raised in worker Thread-1
Traceback (most recent call last):
File "/home/selo/subliminal/subliminal/core.py", line 305, in run
subtitles = plugin.list(task.video, task.languages)
File "/home/selo/subliminal/subliminal/plugins.py", line 374, in list
results = self.query(video.season, video.episode, languages, video.path or video.release, video.tvdbid, video.series)
File "/home/selo/subliminal/subliminal/plugins.py", line 321, in query
self.initCache()
File "/home/selo/subliminal/subliminal/plugins.py", line 303, in initCache
raise PluginError('Cache directory is required')
PluginError: Cache directory is required
DEBUG 2011-11-08 18:07:45 subliminal.worker Poison pill received in thread Thread-1
DEBUG 2011-11-08 18:07:45 subliminal.BierDopje Terminating
DEBUG 2011-11-08 18:07:45 subliminal.worker Poison pill received in thread Thread-3
DEBUG 2011-11-08 18:07:45 subliminal.worker Thread Thread-3 terminated
DEBUG 2011-11-08 18:07:45 subliminal.worker Poison pill received in thread Thread-2
DEBUG 2011-11-08 18:07:45 subliminal.worker Thread Thread-2 terminated
DEBUG 2011-11-08 18:07:45 subliminal.worker Thread Thread-1 terminated
DEBUG 2011-11-08 18:07:45 subliminal.worker Poison pill received in thread Thread-4
DEBUG 2011-11-08 18:07:45 subliminal.worker Thread Thread-4 terminated
No subtitles found

Logger raises UnicodeDecodeError

When parameters are utf-8 encoded bytestrings and sys.getdefaultencoding() is 'ascii', Python try to decode the bytestring parameter from 'ascii' to convert it into unicode to put it in the unicode string of logging message.

The best solution would be to :

• Put a warning when non-unicode is used
• Use %r instead of %s which is the representation, not the string itself, no encoding issues using repr

Ability to upload subtitles

Hang in command line

Every single time I invoke subliminal ( Command line ) I have this result :

File "/usr/local/bin/subliminal", line 5, in
pkg_resources.run_script('subliminal==master', 'subliminal')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 467, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1200, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/subliminal-master-py2.7.egg/EGG-INFO/scripts/subliminal", line 100, in
main()
File "/usr/local/lib/python2.7/dist-packages/subliminal-master-py2.7.egg/EGG-INFO/scripts/subliminal", line 88, in main
subtitles = subliminal_client.downloadSubtitles(args)
File "/usr/local/lib/python2.7/dist-packages/subliminal-master-py2.7.egg/subliminal/subliminal.py", line 347, in downloadSubtitles
paths.append(self.resultQueue.get(timeout=10))
File "/usr/lib/python2.7/Queue.py", line 176, in get
raise Empty
Queue.Empty

But it does download some subtitles

BierDopje isn't working for download

It requires a cache_dir (unused when downloading)

Better error handling

It happens that some errors aren't catched properly. A better try/except positioning should do the trick.

Use a Movie and Episode class

Be OO

Use classes for Movie and Episode. Derived from a parent Video with useful hash methods, etc.
PluginBase should really just be just like an interface.
Guessit has a major role in determining here as it'll be used to build those objects (in video constructor or a factory)

Video

• Path
• Hash (OpenSubtitles & TheSubDB)
• Keywords
• Associated subtitles
• Scan() to refresh associated subtitles

Episode (from Video)

• Show Name
• Episode number
• Season number
• Episode Title

Movie (from Video)

• Name
• Year

Add your own API key

In case some site will block the current API-key if this program is misused by some users.
For instance you can use your personal API-key for Bierdopje.com if you create an acoount.

This is more a feature request

Add http://www.subsfactory.it/

Add http://www.italiansubs.net/

Add http://www.tvsubtitles.net/