antinucleon/edx-dl

edx-dl's Issues

'executable_path has been deprecated'

C:\Users\edx-dl-main\edx-dl.py:25: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
self.driver = webdriver.Chrome(
C:\Users\edx-dl-main\edx-dl.py:25: DeprecationWarning: use options instead of chrome_options
self.driver = webdriver.Chrome(

seems like 'executable_path' is currently belong to 'Service' object... However I've never learned about it, could someone fix it?

网上查的说是版本升级，‘executable_path’被整合到Service模块了，但我不会爬虫。。。有没有哪位大神帮忙改改？

GL is disabled

Could you please help me?

What is mean "Passthrough is not supported. GL is disabled"?

What can I do to fix it?

Chromedriver is in the path, yet reports not in the path!

Hi,

This fails to run with "chromedriver not in the path"!

~/Downloads/gitrepos/antincleon-edx-dl ]$ which chromedriver
./bin/chromedriver

$ echo $PATH
PATH=./bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:./bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/zenny/gitrepos/antincleon-edx-dl/bin

$ ls -la ./bin/chromedriver
-rwxr-xr-x 1 zenny zenny 12M Jul 27 03:27 ./bin/chromedriver

All requirements (binary and path and executablity) are satisfied, yer running the script errors out as if chromedriver is not in the path:

Course home URL: https://learning.edx.org/course/course-v1:HarvardX+HDS3221.3x+3T2020/home
/home/zenny/Downloads/gitrepos/antincleon-edx-dl/edx-dl.py:25: DeprecationWarning: use options instead of chrome_options
  self.driver = webdriver.Chrome(
Traceback (most recent call last):
  File "/home/zenny/Downloads/gitrepos/antincleon-edx-dl/edx-dl.py", line 199, in <module>
    run()
  File "/usr/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/zenny/Downloads/gitrepos/antincleon-edx-dl/edx-dl.py", line 195, in run
    course = EdxCourse(user, pwd, url)
  File "/home/zenny/Downloads/gitrepos/antincleon-edx-dl/edx-dl.py", line 25, in __init__
    self.driver = webdriver.Chrome(
  File "/home/zenny/.local/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
    RemoteWebDriver.__init__(
  File "/home/zenny/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/zenny/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/zenny/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/zenny/.local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary

Even install chromium package to satisfy Chrome binary, it opens up the chromium and does just create a folder without any content!

Hmmmm???

cant handle the '/' character in name

Traceback (most recent call last):
File "edx-dl.py", line 199, in
run()
File "/home/fe/.local/lib/python3.7/site-packages/click/core.py", line 1137, in call
return self.main(*args, **kwargs)
File "/home/fe/.local/lib/python3.7/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "/home/fe/.local/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/fe/.local/lib/python3.7/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)[
File "edx-dl.py", line 196, in run
course()
File "edx-dl.py", line 46, in call
self.mkdir(sub_title)
File "edx-dl.py", line 89, in mkdir
os.mkdir(path)
FileNotFoundError: [Errno 2] No such file or directory: '3_Lesson_3:_Path_coverage_and_MC/DC'

We cannot escape the '/' character
I would suggest to replace the '/' character by '|'

Not Working if you can please find the time to fix

Hi
running on windows
downloaded chromcast into ./bin
aria2
put them both in windows path.
when trying to run, the chrome browser opens with edx page I enter user and password,
and then nothing.
this is what shows on the terminal:

python edx-dl --user me --pwd hi --url https://learning.edx.org/course/course-v1:MichiganX+ANATOMY403.1x+2T2021/home
D:\dev\edx-dl\edx-dl.py:25: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  self.driver = webdriver.Chrome(
D:\dev\edx-dl\edx-dl.py:25: DeprecationWarning: use options instead of chrome_options
  self.driver = webdriver.Chrome(

DevTools listening on ws://127.0.0.1:53200/devtools/browser/70043eee-7c9e-4af6-9629-6721d2167ec2
[24488:10456:0814/090030.519:ERROR:device_event_log_impl.cc(214)] [09:00:30.519] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[24488:10456:0814/090030.520:ERROR:device_event_log_impl.cc(214)] [09:00:30.519] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[24488:10456:0814/090030.527:ERROR:device_event_log_impl.cc(214)] [09:00:30.527] Bluetooth: bluetooth_adapter_winrt.cc:1074 Getting Default Adapter failed.
Traceback (most recent call last):
  File "D:\dev\edx-dl\edx-dl.py", line 199, in <module>
    run()
  File "C:\ProgramData\Miniconda3\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\ProgramData\Miniconda3\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\ProgramData\Miniconda3\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\ProgramData\Miniconda3\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:\dev\edx-dl\edx-dl.py", line 195, in run
    course = EdxCourse(user, pwd, url)
  File "D:\dev\edx-dl\edx-dl.py", line 28, in __init__
    self._login(username, pwd)
  File "D:\dev\edx-dl\edx-dl.py", line 101, in _login
    uname_input = self.driver.find_element_by_name("emailOrUsername")
AttributeError: 'WebDriver' object has no attribute 'find_element_by_name'

If you can kindly fix the project, or give some direction.
Thanks

[Kindly need your help!]: Chromium unexpectedly close after enter into course's home

Hello @antinucleon,

I can see the busy status in your profile. But I am in need of some assistance of yours very badly. Hope that you kindly help me.

I follow the instruction to run edx-dl.py properly. But after enter into course's home page, my chromium unexpectedly close and found this error message:

Traceback (most recent call last):
  File "/home/arju/rafid/programming/projects/edxdl/edx-dl/edx-dl.py", line 199, in <module>
    run()
  File "/home/arju/.local/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/arju/.local/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/arju/.local/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/arju/.local/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/arju/rafid/programming/projects/edxdl/edx-dl/edx-dl.py", line 196, in run
    course()
  File "/home/arju/rafid/programming/projects/edxdl/edx-dl/edx-dl.py", line 38, in __call__
    units = self._parse_course()
  File "/home/arju/rafid/programming/projects/edxdl/edx-dl/edx-dl.py", line 117, in _parse_course
    title.click()
  File "/home/arju/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 81, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/arju/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 710, in _execute
    return self._parent.execute(command, params)
  File "/home/arju/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "/home/arju/.local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <div class="collapsible-trigger" role="button" tabindex="0" aria-expanded="false">...</div> is not clickable at point (455, 571). Other element would receive the click: <div class="col-10 p-0 ml-3 text-break">...</div>
  (Session info: chrome=96.0.4664.93)
Stacktrace:
#0 0x5563d69fcee3 <unknown>
#1 0x5563d64ca608 <unknown>
#2 0x5563d6508044 <unknown>
#3 0x5563d6505af4 <unknown>
#4 0x5563d65032da <unknown>
#5 0x5563d6501ce2 <unknown>
#6 0x5563d64f57b8 <unknown>
#7 0x5563d651e272 <unknown>
#8 0x5563d64f5063 <unknown>
#9 0x5563d651e37e <unknown>
#10 0x5563d65313bc <unknown>
#11 0x5563d651e163 <unknown>
#12 0x5563d64f3bfc <unknown>
#13 0x5563d64f4c05 <unknown>
#14 0x5563d6a2ebaa <unknown>
#15 0x5563d6a44651 <unknown>
#16 0x5563d6a2fb05 <unknown>
#17 0x5563d6a45a68 <unknown>
#18 0x5563d6a2405f <unknown>
#19 0x5563d6a60818 <unknown>
#20 0x5563d6a60998 <unknown>
#21 0x5563d6a7beed <unknown>
#22 0x7f3147106259 <unknown>

System info:

OS: Manjaro Linux x86_64
Kernel: Linux 5.15.7-1-MANJARO
Shell: bash 5.1.12
DE: Plasma 5.23.4

Chromium Version: Version 96.0.4664.93 (Official Build) Arch Linux (64-bit)

I hope you will be kindly help me in this situation.

amazing

thanks for your efforts it works really well, but there is just a simple error with naming the folders

'chromedriver' executable needs to be in PATH.

When I try to download a course, I get this error. It's something that has to do with PATH and environment variables I think but I don't know what it is.

Course home URL: https://learning.edx.org/course/course-v1:PennX+SD4x+3T2020/home
C:\Users\MarwanShehata\Downloads\edx-dl-main\edx-dl-main\edx-dl.py:25: DeprecationWarning: use options instead of chrome_options
self.driver = webdriver.Chrome(
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "C:\Python39\lib\subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Python39\lib\subprocess.py", line 1420, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\MarwanShehata\Downloads\edx-dl-main\edx-dl-main\edx-dl.py", line 199, in
run()
File "C:\Python39\lib\site-packages\click\core.py", line 1137, in call
return self.main(*args, **kwargs)
File "C:\Python39\lib\site-packages\click\core.py", line 1062, in main
rv = self.invoke(ctx)
File "C:\Python39\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Python39\lib\site-packages\click\core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "C:\Users\MarwanShehata\Downloads\edx-dl-main\edx-dl-main\edx-dl.py", line 195, in run
course = EdxCourse(user, pwd, url)
File "C:\Users\MarwanShehata\Downloads\edx-dl-main\edx-dl-main\edx-dl.py", line 25, in init
self.driver = webdriver.Chrome(
File "C:\Python39\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in init
self.service.start()
File "C:\Python39\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

element not interactable

Download is stop. And I see:

selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable

How to fix it?

Note that this happens when two videos on a page or page look like this