Windows 10: Response Code 400

[xonshiz]

Windows 10: Response Code 400

@csisoap Thank you so much, I managed to download the subtitles of the TV series "Fear the Walking Dead" by renameing the link to

https://www.hulu.com/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

and used the new link and it worked (seems that the name works only if it has two words, "rick-and-morty" to work you need to rename it to "and-morty").

If the TV series has one word only like "futurama" the link will not work, adding "-" before the name will make the link works like "-futurama".

https://www.hulu.com/series/futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

https://www.hulu.com/series/-futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

[csisoap]

Try https://www.hulu.com/series/great-north-374cdbc2-3b08-4b81-a68f-a45a70665849

@Xonshiz I deleted the comment by myself for the same reason.

@Xonshiz This might help you in figuring out a solution for the problem of the non working links for the TV shows like

https://www.hulu.com/series/fear-the-walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

and

https://www.hulu.com/series/futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

when i compared between the "Hulusubs_dl_Error_Log" for the non working link for the TV show "Fear the walking dead"

https://www.hulu.com/series/fear-the-walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

DEBUG: eab_id_matches: ['the', 'walking', 'dead', '6b075a0b', '3d97', '42a8', 'b2f1', 'c1d698d64d53']
DEBUG: initial eab_id: walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 400 277
DEBUG:
initial series_metadata:
None

and the working one for the TV show "Fear the walking dead"

https://www.hulu.com/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

DEBUG: eab_id_matches: ['dead', '6b075a0b', '3d97', '42a8', 'b2f1', 'c1d698d64d53']
DEBUG: initial eab_id: 6b075a0b-3d97-42a8-b2f1-c1d698d64d53
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 200 38669
DEBUG:
initial series_metadata: {'_type': 'hub', 'id': '6b075a0b-3d97-42a8-b2f1-c1d698d64d53', 'href': 'https://discover.hulu.com/content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1', 'name': 'Fear the Walking Dead', 'theme': 'hub_theme_entity_detail', 'artwork': {'title.treatment.horizontal': {'path':

I found that with the non working link for the TV show "Fear the walking dead" we get the wrong link (It has the words "walking-dead-" which they should not be in the link)

https://discover.hulu.com/content/v5/hubs/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

and in the working link for the TV show "Fear the walking dead" we get the correct link

https://discover.hulu.com/content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

and when i compared between the "Hulusubs_dl_Error_Log" for the non working link for the TV show "Futurama"

https://www.hulu.com/series/futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

DEBUG: eab_id_matches: ['85bf4cc1', 'cd8b', '4469', 'ad87', '7289217a0b74']
DEBUG: initial eab_id: cd8b-4469-ad87-7289217a0b74
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 400 259
DEBUG:
initial series_metadata:
None

and the working one for the TV show "Futurama"

https://www.hulu.com/series/-futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

DEBUG: eab_id_matches: ['futurama', '85bf4cc1', 'cd8b', '4469', 'ad87', '7289217a0b74']
DEBUG: initial eab_id: 85bf4cc1-cd8b-4469-ad87-7289217a0b74
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 200 16034
DEBUG:
initial series_metadata: {'_type': 'hub', 'id': '85bf4cc1-cd8b-4469-ad87-7289217a0b74', 'href': 'https://discover.hulu.com/content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1', 'name': 'Futurama', 'theme': 'hub_theme_entity_detail', 'artwork': {'title.treatment.horizontal': {'path':

I found that with the non working link for the TV show "Futurama" we get the wrong link (the links is missing "85bf4cc1-")

https://discover.hulu.com/content/v5/hubs/series/cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

and in the working link for the TV show "Futurama" we get the correct link

https://discover.hulu.com/content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

[Xonshiz]

Woah, this is a weird thing that's happening. I'll add a bug tag to this and see if it's happening with all the series or just some handful of them.

@Xonshiz The script does not work with the links of the movies like

https://www.hulu.com/movie/terminator-dark-fate-3be9a47d-4a50-447a-980a-e5cae615f522

it works only when i replace "/movie/" and the name of the movie with "/watch/".

https://www.hulu.com/watch/3be9a47d-4a50-447a-980a-e5cae615f522

and the name of the subtitle file of the movie become "No Name Found - S01E01 [en Sub].srt"

[Xonshiz]

Yes, that "/watch" thing is a known "issue" or "thing" however you term it. It was me being lazy to not add one more entry of check after I had checked in a general code that worked for both. It's just an extra step haha. I'll patch it up in next fix. But, what I'm more curious about is that the script is working with just "2 words"... like that walking dead link. That needs to be looked into. I'll see if I can muster up some time to look into this.

Ah No Name found thing.... Didn't know about it. Seems like Hulu has changed some things again. I'll need to update this.

@Xonshiz It seems to be about the "-" more than the title has two words, like the TV series "Malcolm in the Middle" could work with only "-middle".

https://www.hulu.com/series/malcolm-in-the-middle-ca1ac46e-9883-4125-a6e8-97efce9a2bf5

https://www.hulu.com/series/-middle-ca1ac46e-9883-4125-a6e8-97efce9a2bf5

[csisoap]

@Xonshiz i think instead of grabbing the first two, try to grab five words from the last.

[Xonshiz]

See.. that's the thing... I don't remember putting any code that deletes/grabs any specific set of words. It should take all and send to HULU APIs. That's why I'll need to see what is happening, where and why.

[clayton-grey]

Thanks for the response. Your link was recognized, but I ran into a new issue:

Downloading Subtitle For The Great North - S1E1 [en Sub].vtt
Couldn't find vtt in Hulu

Which also doesn't produce a log, but it seems like it should work?

@clayton-grey You could download the subtitles of the TV show "The Great North" in "srt" format.

[Xonshiz]

That subtitle thing is another issue. Could you open a separate bug for that. Helps in checking what all things I need to fix.

Also, @Essam3111 good that you were helping, but, I'd suggest that you do not share an actual subtitle file over here. I think your comment was removed and I really don't want this repo getting nuked on some BS DMCA notice.

[Xonshiz]

Hhahah I see... good, thanks :D

@Xonshiz Just a simple observation, when I try to download the subtitles in "ttml" format or "smi" format for any TV show, the script download only the first episode, but when I go to the folder of the downloaded subtitles no subtitles are actually downloaded and the folder is empty, Hulu subtitles in "ttml" do not work correctly with Subtitle Edit, Subtitle Edit displays the time wrongly (nothing wrong with the subtitles themselves), and Hulu subtitles in "smi" format are encrypted.

[Xonshiz]

Ah, I see. Could you please open a new bug report for that and also mention some URLs where you can see this issue.

[clayton-grey]

@clayton-grey You could download the subtitles of the TV show "The Great North" in "srt" format.

I could, but I use VTT, so it saves me conversion steps, and it should be possible to get from their servers.

I'll open a new ticket for this.

@clayton-grey This is just a workaround, until the script got fixed you could use it for downloading the subtitles in "webvtt" format, in config file add 'webvtt'.

Hulusubs_dl.webvtt.zip

In utils.py and hulu_subs_dl.py I replaced 'vtt' with 'webvtt' (this solves the problem of the error message "Couldn't find vtt in Hulu")

In hulu.py I replaced

if extension == 'srt':

with

if not extension == 'srt':

In subtitle_processing.py I replaced

replacement = re.sub(r'(\d\d:\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
                     r'\1,\2 --> \3,\4\n', file_contents)
replacement = re.sub(r'(\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
                     r'\1,\2 --> \3,\4\n', replacement)
replacement = re.sub(r'(\d\d).(\d\d\d) --> (\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n', r'\1,\2 --> \3,\4\n',
                     replacement)
replacement = re.sub(r'WEBVTT\n', '', replacement)
replacement = re.sub(r'Kind:[ \-\w]+\n', '', replacement)
replacement = re.sub(r'Language:[ \-\w]+\n', '', replacement)
replacement = re.sub(r'<c[.\w\d]*>', '', replacement)
replacement = re.sub(r'</c>', '', replacement)
replacement = re.sub(r'<\d\d:\d\d:\d\d.\d\d\d>', '', replacement)
replacement = re.sub(r'::[\-\w]+\([\-.\w\d]+\)[ ]*{[.,:;\(\) \-\w\d]+\n }\n', '', replacement)
replacement = re.sub(r'Style:\n##\n', '', replacement)

with

replacement = re.sub(r'(\d\d:\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
                     r'\1.\2 --> \3.\4\n', file_contents)
#replacement = re.sub(r'(\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
#                     r'\1,\2 --> \3,\4\n', replacement)
#replacement = re.sub(r'(\d\d).(\d\d\d) --> (\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n', r'\1,\2 --> \3,\4\n',
#                     replacement)
#replacement = re.sub(r'WEBVTT\n', '', replacement)
#replacement = re.sub(r'Kind:[ \-\w]+\n', '', replacement)
#replacement = re.sub(r'Language:[ \-\w]+\n', '', replacement)
#replacement = re.sub(r'<c[.\w\d]*>', '', replacement)
#replacement = re.sub(r'</c>', '', replacement)
#replacement = re.sub(r'<\d\d:\d\d:\d\d.\d\d\d>', '', replacement)
#replacement = re.sub(r'::[\-\w]+\([\-.\w\d]+\)[ ]*{[.,:;\(\) \-\w\d]+\n }\n', '', replacement)
#replacement = re.sub(r'Style:\n##\n', '', replacement)

@Xonshiz I found out why the link works only if it has two words, this regular expression removes one word

https://www.hulu.com/series/castle-rock-b11816c9-9e35-44f3-bf04-220b1d12f770

https://github.com/Xonshiz/Hulusubs_dl/blob/master/hulusubs_dl/hulu.py#L86

https://regex101.com/r/48hdPs/1

and this line removes another word

https://github.com/Xonshiz/Hulusubs_dl/blob/master/hulusubs_dl/hulu.py#L89

and we get the correct link

https://discover.hulu.com/content/v5/hubs/series/b11816c9-9e35-44f3-bf04-220b1d12f770?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

this is the correct regular expression for selecting all the words

([0-9A-Za-z]+

https://regex101.com/r/ZogdBy/1

[clayton-grey]

Nice sleuthing.

[Xonshiz]

Holy crap... You've literally fixed it all. hahahaha Thank you so much @Essam3111 .

EDIT: Oh I think I just remembered why I used that regex. Didn't see if failing this badly :(

@Xonshiz Thank you so much, I tested the dev branch and everything is working great until now, just one more problem still needs to be fixed, the movie subtitle file name is "No Name Found - S01E01 [en Sub]", here is the Hulusubs_dl_Error_Log for the movie

https://www.hulu.com/movie/terminator-dark-fate-3be9a47d-4a50-447a-980a-e5cae615f522

Hulusubs_dl_Error_Log.zip

[Xonshiz]

Yep, the thing is, I don't have the JSON that's returned. So, what I'm guessing is that for "movie", they have a different JSON getting back and for a show video, they have different JSON getting back. Thanks for sharing the verbose dump, this should have the JSON, maybe I can work my way out of that one.

@Xonshiz Just a very minor problem with the name of the downloaded subtitle file for the movie, the number of the season added to the name of the subtitle file and a season folder created like.

C:\Hulusubs_dl\Terminator - Dark Fate\01\Terminator - Dark Fate - S01E01 [en Sub].srt

https://www.hulu.com/movie/terminator-dark-fate-3be9a47d-4a50-447a-980a-e5cae615f522

[Xonshiz]

Ah, got it. Could you open a separate bug for that? It's cased via different functionality and this issue serves purpose for 400 error.