changil/avspeech-downloader

AVSpeech downloader

avspeech-downloader's Issues

Getting lots of repeative error, not sure how to fix these. I am using google's train.csv to download the data

(standard_in) 2: syntax error ERROR: Unable to download webpage: (caused by URLError(u'unknown url type: opacity',)) [2019/08/21 21:05:52] <style>_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: u'<style>' is not a valid URL. Set --default-search "ytsearch" (or run youtube-dl "ytsearch:<style>" ) to search YouTube [2019/08/21 21:05:52] }_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: u'}' is not a valid URL. Set --default-search "ytsearch" (or run youtube-dl "ytsearch:}" ) to search YouTube [2019/08/21 21:05:52] html_000.000000-000.000000: ERROR: failed to get video url /bin/bash: line 0: printf: body {: invalid number (standard_in) 1: syntax error ERROR: u'html' is not a valid URL. Set --default-search "ytsearch" (or run youtube-dl "ytsearch:html" ) to search YouTube [2019/08/21 21:05:52] background: #fff;_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: Unable to download webpage: (caused by URLError(u'unknown url type: background',)) [2019/08/21 21:05:52] margin: 0;_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: Unable to download webpage: (caused by URLError(u'unknown url type: margin',)) [2019/08/21 21:05:52] padding: 0;_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: Unable to download webpage: (caused by URLError(u'unknown url type: padding',)) [2019/08/21 21:05:52] font-family: Arial_000.000000-000.000000: ERROR: failed to get video url /bin/bash: line 0: printf: sans-serif;: invalid number ERROR: Unable to download webpage: (caused by URLError(u'unknown url type: font-family',)) [2019/08/21 21:05:52] position: absolute;_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: Unable to download webpage: (caused by URLError(u'unknown url type: position',)) [2019/08/21 21:05:52] </style>_000.000000-000.000000: ERROR: failed to get video url (standard_in) 2: syntax error ERROR: Unable to download webpage: <urlopen error [Errno -2] Name or service not known> (caused by URLError(gaierro

ERROR: failed to rename temporary video

Can you please tell what might be causing this error and how to download the dataset? I'm running Ubuntu 18.04.1 LTS. Let me know if any more information is needed.

Terminal command:

njobs=4 ./download.sh avspeech_train.csv train

Output:

[2019/03/12 21:42:28] *** download starts ***
[2019/03/12 21:43:45] CJoOwXcjhds_233.266000-239.367000: ERROR: failed to rename temporary video
mv: cannot stat 'train/CJoOwXcjhds_233.266000-239.367000.mp4.inprogress.1.mp4': No such file or directory
[2019/03/12 21:44:35] Y8HMIm8mdns_171.607767-174.607767: ERROR: failed to rename temporary video
mv: cannot stat 'train/Y8HMIm8mdns_171.607767-174.607767.mp4.inprogress.1.mp4': No such file or directory

The train directory looks like this:

ffmpeg keyframe seeking issue

ffmpeg uses keyframe seeking when stream copying. Consequently, when a cut boundary doesn't fall at a keyframe (which happens most of the time) it cuts from the previous keyframe and sets a negative start time to compensate it. So any subsequent tools that take the cut video clips should take the start time into account. Most video players do, but if you programatically process video clips, you should do it yourself and discard first part of both audio and video accordingly. A fix would be to transcode video when cutting, at the cost of extended processing time. I will push a patch soon.

Affected lines of code:

avspeech-downloader/download.sh

Line 94 in ab3bb49

$ffmpeg -ss "$start" -i "$outdir/$filename.inprogress.1" -t "$duration" -c:v copy -c:a copy -f mp4 -threads 1 "$outdir/$filename.inprogress.2" < /dev/null || { show_msg "$id" "ERROR: failed to cut video"; return; }

avspeech-downloader/download.sh

Line 107 in ab3bb49

$ffmpeg -ss "$start" -i "$url" -t "$duration" -c:v copy -c:a copy -f mp4 -threads 1 "$outdir/$filename.inprogress" < /dev/null || { show_msg "$id" "ERROR: failed to download video"; return; }