Course downloader for teachable platform written in python3 using selenium and yt-dlp
License: GNU Lesser General Public License v3.0
ERROR: Could not find login: Message:
Stacktrace:
GetHandleVerifier [0x00007FF70BB782B2+55298]
(No symbol) [0x00007FF70BAE5E02]
(No symbol) [0x00007FF70B9A05AB]
(No symbol) [0x00007FF70B9E175C]
(No symbol) [0x00007FF70B9E18DC]
(No symbol) [0x00007FF70BA1CBC7]
(No symbol) [0x00007FF70BA020EF]
(No symbol) [0x00007FF70BA1AAA4]
(No symbol) [0x00007FF70BA01E83]
(No symbol) [0x00007FF70B9D670A]
(No symbol) [0x00007FF70B9D7964]
GetHandleVerifier [0x00007FF70BEF0AAB+3694587]
GetHandleVerifier [0x00007FF70BF4728E+4048862]
GetHandleVerifier [0x00007FF70BF3F173+4015811]
GetHandleVerifier [0x00007FF70BC147D6+695590]
(No symbol) [0x00007FF70BAF0CE8]
(No symbol) [0x00007FF70BAECF34]
(No symbol) [0x00007FF70BAED062]
(No symbol) [0x00007FF70BADD3A3]
BaseThreadInitThunk [0x00007FF8BD9E257D+29]
RtlUserThreadStart [0x00007FF8BF2AAA78+40]
Platform: Windows.
Also, to activate an environment on Windows, the command is: env\Scripts\activate
Describe the bug
I've tried with both Ubuntu and macOS and encountered a similar problem:
Error:
ERROR: Could not download course: https://svit-i-s-school.teachable.com/courses/enrolled/2150650 cause: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="__next"]/div/div/div[2]/div/div[1]/img"}
(Session info: chrome=120.0.6099.62); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
0 uc_driver 0x00000001030b6c08 uc_driver + 4852744
1 uc_driver 0x00000001030ae173 uc_driver + 4817267
2 uc_driver 0x0000000102c7a7bd uc_driver + 411581
3 uc_driver 0x0000000102ccae30 uc_driver + 740912
4 uc_driver 0x0000000102ccafe1 uc_driver + 741345
5 uc_driver 0x0000000102d16864 uc_driver + 1050724
6 uc_driver 0x0000000102cf4c2d uc_driver + 912429
7 uc_driver 0x0000000102d13ae2 uc_driver + 1039074
8 uc_driver 0x0000000102cf49a3 uc_driver + 911779
9 uc_driver 0x0000000102cbc103 uc_driver + 680195
10 uc_driver 0x0000000102cbd71e uc_driver + 685854
11 uc_driver 0x0000000103076752 uc_driver + 4589394
12 uc_driver 0x000000010307b95c uc_driver + 4610396
13 uc_driver 0x000000010305bc71 uc_driver + 4480113
14 uc_driver 0x000000010307c6d6 uc_driver + 4613846
15 uc_driver 0x000000010304d1fc uc_driver + 4420092
16 uc_driver 0x000000010309c758 uc_driver + 4745048
17 uc_driver 0x000000010309c90e uc_driver + 4745486
18 uc_driver 0x00000001030addb3 uc_driver + 4816307
19 libsystem_pthread.dylib 0x00007ff80729d202 _pthread_start + 99
20 libsystem_pthread.dylib 0x00007ff807298bab thread_start + 15
Expected behavior
I expect the app to download the course)
Screenshots
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
I would like to add support for non-latin languages in the title of the pages(Cyrillic/Chinese/Japanese)
Describe alternatives you've considered
None
Additional context
This is what currently I see as I download a course:
I have came accross some lessons that contain more than one video. Currently, the program just downloads the first one, so I was wondering if it was possible to download all of them.
Describe the bug
When accessing a course, the HTML pages are created for the course sections, but no video is downloaded, and warning/error messages pop up.
I can successfully bypass Cloudflare and automatically log in to the course URL. Once the folders are created and navigated to the first section of the course, the HTML page is successfully created, and I assume the video is downloaded afterward. At this point, there are warnings as follows.
WARNING: No attachments found for video: Welcome!-(6-55-)
[generic] Extracting URL: https://vod-akm.play.hotmart.com/video/rRAWPmGkL1/hls/master-t-1649091442001.m3u8?hdnts=st%3D1698...e8-9725-e0efc7b5d29c
[generic] master-t-1649091442001: Downloading webpage
ERROR: [generic] Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
WARNING: Could not download subtitle: Welcome!-(6-55-) cause: ERROR: [generic] Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
WARNING: Could not download subtitle: Welcome!-(6-55-) cause: local variable 'info_json' referenced before assignment
[generic] Extracting URL: https://vod-akm.play.hotmart.com/video/rRAWPmGkL1/hls/master-t-1649091442001.m3u8?hdnts=st%3D1698...e8-9725-e0efc7b5d29c
[generic] master-t-1649091442001: Downloading webpage
Additional context
I have followed all the installation instructions and requirements and for sure have the latest version of yt-dlp. When looking into
that package github there's a issue that I THINK may be related but I'm not sure.
Regardless of course URL, no videos are downloading but HTML pages are being collected properly.
I was having lots of trouble to have uc working on apple m1 with chrome_driver 116. Then I tried seleniumbase and it worked really well.
You should consider replacing uc with seleniumbase in your code.
ultrafunkamsterdam/undetected-chromedriver#1477 (comment)
Describe the bug
Video downloaded in mp4 format can not be opened by mp4 player
To Reproduce
Steps to reproduce the behavior:
Use this to download video from teachable.
Expected behavior
The video file is able to be played by mp4 player as file is in mp4 format.
Desktop (please complete the following information):
I login with --manual-url option, it logs in, no problem with cloudflare, script finds course, script says downloading course content, but then this happens:
INFO: Downloading lecture: Special-Thanks-&-Credits
INFO: Disabling autoplay
ERROR: Could not download course: [courselinkhere] cause: Message: javascript error: Cannot read properties of null (reading 'checked')
(Session info: chrome=119.0.6045.159)
Stacktrace:
#0 0x564078918723
#1 0x5640785e7047
#2 0x5640785ed205
#3 0x5640785ef901
#4 0x56407867741f
#5 0x56407865aeb2
#6 0x564078676a40
#7 0x56407865ac83
#8 0x564078626533
#9 0x5640786274de
#10 0x5640788e1eea
#11 0x5640788e68a4
#12 0x5640788d0d22
#13 0x5640788e72c0
#14 0x5640788b70be
#15 0x564078907848
#16 0x564078907a3a
#17 0x564078917849
#18 0x7fe83f1ddac3
INFO: Cleaning up
so i guess javascript error here while disabling autoplay or what? any idea how can i fix this please
In chrome 115, it gets stuck in Cloudflare Captcha
Could you please have a look and let me know what I could be doing wrong? Thanks!
ERROR: Could not download course: https://school.internetlifestylehub.com/courses/enrolled/831151 cause: Message: stale element reference: element is not attached to the page document
(Session info: chrome=112.0.5615.55)
Stacktrace:
0 undetected_chromedriver 0x000000010ef12ae8 undetected_chromedriver + 4958952
1 undetected_chromedriver 0x000000010ef0a7b3 undetected_chromedriver + 4925363
2 undetected_chromedriver 0x000000010eacc1d6 undetected_chromedriver + 475606
3 undetected_chromedriver 0x000000010ead0274 undetected_chromedriver + 492148
4 undetected_chromedriver 0x000000010eacfe56 undetected_chromedriver + 491094
5 undetected_chromedriver 0x000000010ead02fc undetected_chromedriver + 492284
6 undetected_chromedriver 0x000000010eb0f63c undetected_chromedriver + 751164
7 undetected_chromedriver 0x000000010eb0fc81 undetected_chromedriver + 752769
8 undetected_chromedriver 0x000000010eb020f3 undetected_chromedriver + 696563
9 undetected_chromedriver 0x000000010eb35c3d undetected_chromedriver + 908349
10 undetected_chromedriver 0x000000010eb018f1 undetected_chromedriver + 694513
11 undetected_chromedriver 0x000000010eb35dce undetected_chromedriver + 908750
12 undetected_chromedriver 0x000000010eb5125b undetected_chromedriver + 1020507
13 undetected_chromedriver 0x000000010eb359e3 undetected_chromedriver + 907747
14 undetected_chromedriver 0x000000010eaffaef undetected_chromedriver + 686831
15 undetected_chromedriver 0x000000010eb0100e undetected_chromedriver + 692238
16 undetected_chromedriver 0x000000010eece9ad undetected_chromedriver + 4680109
17 undetected_chromedriver 0x000000010eed3c12 undetected_chromedriver + 4701202
18 undetected_chromedriver 0x000000010eeda79f undetected_chromedriver + 4728735
19 undetected_chromedriver 0x000000010eed4b7a undetected_chromedriver + 4705146
20 undetected_chromedriver 0x000000010eea727e undetected_chromedriver + 4518526
21 undetected_chromedriver 0x000000010eef4908 undetected_chromedriver + 4835592
22 undetected_chromedriver 0x000000010eef4a87 undetected_chromedriver + 4835975
23 undetected_chromedriver 0x000000010ef0400f undetected_chromedriver + 4898831
24 libsystem_pthread.dylib 0x00007ff8139ea1d3 _pthread_start + 125
25 libsystem_pthread.dylib 0x00007ff8139e5bd3 thread_start + 15
Describe the bug
So there is a list of 5 videos, 3 of them downloads as suspected but the other 2 gets skipped.
During the downloading process at 2 certain videos it gets this error :
ERROR: [generic] Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>)
And then the video doesn't download, it skips over it?
I suspect it's the video set to private or some sort of permissions blocking it from been access and then been downloaded.
To Reproduce
Steps to reproduce the behavior:
python3 main.py --url https://maryvance.teachable.com/courses/1289331 --email <some-email> --password <some-passowrd>Expected behavior
Well, I suspect it should download the video as all the other videos been downloaded?
Screenshots
Desktop (please complete the following information):
Additional context
None.
getting "bash: syntax error near unexpected token `newline'" when running source env/bin/activate python3 main.py --url <course-url> --email <email> --password <password> (with all fields filled out)
Describe the bug
On some systems (e.g. macOS) there is a limit to how long a file name can be.
In case a lecture name is very long it can exceed this limit and the download of that lecture will fail.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The download should trim the length of the lecture title in order to fit within the file name length limit.
Desktop (please complete the following information):
Additional context
On macOS the file name limit is 255 characters. There is no limit for the lecture title on Teachable platform. Thus it can happen that the lecture is not downloaded due to the lecture title just being too long.
Describe the bug
After starting the script it says that I should click the captcha checkbox. I cannot do that, however, as there is no visible window. I can see the Chromium process but it does not seem to have a GUI.
Expected behavior
A Chromium window is displayed in which I can check the captcha checkbox.
Desktop (please complete the following information):
is there a way to get the script to start where it left off in case of an interruption?
When downloading a course, the browser repeatedly steals focus.
The browser needs to not steal focus while the script is running to allow the user to complete other tasks while a download is running in the background.
Is your feature request related to a problem? Please describe.
When viewed in an offline environment, images do not load.
Describe the solution you'd like
Save images locally and use those in place of server images.
Describe alternatives you've considered
I can connect to the internet (which goes against the focus of this project) or manually save all images when I'm online (which should be scriptable).
Additional context
Seems to only save videos to local on my end, images are ignored? Images are still very important to learning curriculum.
Describe the bug
I might be doing something wrong but when trying to download I receive the following error:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:55045
from session not created: This version of ChromeDriver only supports Chrome version 112
Current browser version is 114.0.5702.0
To Reproduce
Steps to reproduce the behavior:
Desktop (please complete the following information):
Is it possible to run it headless in a docker container? Because I am on windows and I did not want to dual boot ubuntu right now.
After a bit of waiting I am getting this error:
Traceback (most recent call last):
File "/data/Teachable-Dl/main.py", line 652, in <module>
downloader = TeachableDownloader(verbose_arg=verbose, complete_lecture_arg=args.complete_lecture)
File "/data/Teachable-Dl/main.py", line 73, in __init__
self.driver = uc.Chrome(options=self.chrome_options)
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/undetected_chromedriver/__init__.py", line 443, in __init__
super(Chrome, self).__init__(
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in __init__
super().__init__(
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
super().__init__(
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
self.start_session(capabilities, browser_profile)
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/undetected_chromedriver/__init__.py", line 674, in start_session
super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "/data/Teachable-Dl/env/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:39317
from chrome not reachable
Stacktrace:
#0 0x564cb171a4e3 <unknown>
#1 0x564cb1449b00 <unknown>
#2 0x564cb1437436 <unknown>
#3 0x564cb14769be <unknown>
#4 0x564cb146e884 <unknown>
#5 0x564cb14adccc <unknown>
#6 0x564cb14ad47f <unknown>
#7 0x564cb14a4de3 <unknown>
#8 0x564cb147a2dd <unknown>
#9 0x564cb147b34e <unknown>
#10 0x564cb16da3e4 <unknown>
#11 0x564cb16de3d7 <unknown>
#12 0x564cb16e8b20 <unknown>
#13 0x564cb16df023 <unknown>
#14 0x564cb16ad1aa <unknown>
#15 0x564cb17036b8 <unknown>
#16 0x564cb1703847 <unknown>
#17 0x564cb1713243 <unknown>
#18 0x7ff1285efb43 <unknown>
anything I can do?
Describe the bug
I followed all the steps and all the modules installed in Ubuntu terminal. After I put in the line below, I saw it open up a Chrome browser, go to the website, login, and then it stayed on the main page which shows all the courses available. Then it shut down Chrome and gave me the below Max retries exceeded error.
(env) danny@danny-VirtualBox:~$ python3 main.py --url https://school.patbailouni.com/courses/challenge21311111 --email [email protected] --password *****
Could not download course: https://school.patbailouni.com/courses/challenge21311111 cause: HTTPConnectionPool(host='localhost', port=43325): Max retries exceeded with url: /session/25c596d8858b4817335e16d7adc01f15/url (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eba86a560>: Failed to establish a new connection: [Errno 111] Connection refused'))
Desktop (please complete the following information):
getting this error when running source env/bin/activate
env/bin/activate (line 41): Unsupported use of '='. In fish, please use 'set VIRTUAL_ENV "/home/user/Teachable-Dl/env"'.
VIRTUAL_ENV="/home/user/Teachable-Dl/env"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
from sourcing file env/bin/activate
garuda linux
Is your feature request related to a problem? Please describe.
Noticed some courses also have quizzes inside
Describe the solution you'd like
Save the quizzes in html or possibly some other form
Describe alternatives you've considered
if all else fails save question and answers in pdf
Describe the bug
After I have manually authenticated myself for the course, I get the message "Downloader does not support this course template. Please open an issue on github".
To Reproduce
Steps to reproduce the behavior:
Additional context
I have replaced the course ID in the URL for this issue with a placeholder.
when run command python3 main.py --url --email --password
Get error->
ERROR: Could not download course: https://crypto-erke.teachable.com/courses/enrolled/2326744 cause: find_elements() got an unexpected keyword argument 'timeout'
Device: MacOS
Is your feature request related to a problem? Please describe.
If for some reason you have to execute a redownload, it is nice if the previously successful content is not redownloaded.
Describe the solution you'd like
Check if some content is already downloaded (e.g. the files already exist).
Describe alternatives you've considered
An option would be to also group lesson files into separate directories and check if the directory exists.
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
As a user I would like the app to be able to detect PDF and MP3 files and download them.
Describe alternatives you've considered
None
Additional context
None
Describe the bug
The bug is happening in Windows because the program want to name a file like this:
'C:\\Users\\user\\Teachable-Dl\\courses\\Grenzwerte-von-Funktionen\\04-Fall-3--0*inf'
To Reproduce
Steps to reproduce the behavior:
Expected behavior
remove *
We should also rename or remove these symbols:
? / \ | : < > " *
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Currently, the course pages are saved as an HTML file, but I think saving as an image or as a PDF would be much better when a user wish to access the course offline from a table/mobile.
We need something like this - https://pythonbasics.org/selenium-screenshot/
Some kind
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
None
Describe the solution you'd like
Need to add the progress bar to the script using rich framework
Describe the bug
Apologies for an issue that I think may be trivial, but every time I run the command given in the README, google chrome is launched, and it asks me to log in. Which upon doing, it just opens the video, and chrome then closes, and I get the message that it couldnt' log in. I also had to try it on google chrome and not anything else, coz of selenium's requirements of having a version over 112.
To Reproduce
Steps to reproduce the behavior:
python main.py --url https://visualpath.teachable.com/courses/ax-technical-d365-september-2022-batch-by-maitra-8-30-pm/lectures/42829022 --email [email] --password [pass]Expected behavior
The video was supposed to downlaod.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.