Powerful tool for downloading content posted by creators on patreon.com. Supports content hosted on patreon itself as well as external sites (additional plugins might be required).
License: MIT License
Hey, thanks for this program, I tested it and it works really nice, the only thing which is weird is that it doesn't download the full size images. So if there is an post with an image which is bigger if I click on it, it just saves 2 times the small thumbnail.
In the json there is the correct url for the bigger one saved as "original", so it seems like a bug for me that this isn't downloaded.
First of all, thanks for this great project. It's working perfectly for downloading a lot of attachments (without having to go through each attachment on every post manually).
I'm not sure how feasibly it is, so it's mainly just an idea/suggestion, but some of the sites listed under "Known not implemented or not tested features" are supported by a tool called youtube-dl.
While I don't really have a clue on the most ideal way to implement youtube-dl, it might be worth looking into.
Thanks again for the great tool. Looking forward to further improvements :)
If downloader encounters that there are no more free disk space it should abort execution instead of creating empty files.
Hello, It appears any custom user tags generate errors and abort any image downloads.
With verbose logging on this is what appears for a post
2020-11-23 17:00:00.2282 DEBUG [83874473] Verification: Started
2020-11-23 17:00:00.2282 DEBUG [83874473] Is a media
2020-11-23 17:00:00.2282 DEBUG [83874473] Verification: OK
2020-11-23 17:00:00.2282 DEBUG [user_defined;3d] Verification: Started
2020-11-23 17:00:00.2282 ERROR Verification for user_defined;3d: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;3d: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 DEBUG [user_defined;group] Verification: Started
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;group: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;group: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 DEBUG [user_defined;stereo] Verification: Started
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;stereo: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;stereo: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 DEBUG [user_defined;touch] Verification: Started
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;touch: Unknown type for "included": post_tag
2020-11-23 17:00:00.2441 ERROR Verification for user_defined;touch: Unknown type for "included": post_tag
As the creator in question is exceedingly diligent in adding tags this means all posts downloads fail.
I left it running for about 40 mins to be sure that the attachments wouldn't download later, but ended up with a 5GB log file instead, available if needed.
i have no coding experience and next to no knowledge with any of this so it could just be my own ignorance but i cannot get this to work at all. i am constantly getting fatal errors every time i try to run it.
i try this in command line
"dotnet run --creator https://www.patreon.com/*creator*/posts --download-directory D:\pat down\downloads"
and get this error
2020-05-10 10:59:04.7814 FATAL Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.PatreonDownloaderException: Unable to retrieve campaign id
at PatreonDownloader.Engine.PatreonDownloader.Download(String creatorName, PatreonDownloaderSettings settings) in D:\pat down\PatreonDownloader-0.9.1.1\PatreonDownloader.Engine\PatreonDownloader.cs:line 162
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, Boolean headlessBrowser, PatreonDownloaderSettings settings) in D:\pat down\PatreonDownloader-0.9.1.1\PatreonDownloader.App\Program.cs:line 136
at PatreonDownloader.App.Program.Main(String[] args) in D:\pat down\PatreonDownloader-0.9.1.1\PatreonDownloader.App\Program.cs:line 58
i have no clue what im doing wrong please help my noobish ass.
and i've tried running it as a .exe but the window closes as soon as it opens.
Hello there!
Before anything thank you for making such a utility!
Sadly when attempting to run it throws a fatal error when retrieving the cookies
[daniel@daniel-pc ~]$ /home/daniel/net3.1-linux-x64-release/PatreonDownloader.App --creator LeeandLie --descriptions --embeds --campaign-images --json --verbose
2020-07-04 00:22:47.9929 INFO Retrieving cookies...
2020-07-04 00:22:48.0217 DEBUG Calling login check
2020-07-04 00:22:48.0298 DEBUG Retrieving browser
2020-07-04 00:22:48.0298 DEBUG Downloading browser
2020-07-04 00:22:48.0377 DEBUG Launching browser
2020-07-04 00:22:48.3404 DEBUG Opening new page
2020-07-04 00:22:48.4574 DEBUG Creating IWebBrowser
2020-07-04 00:22:48.4809 DEBUG Checking login status
2020-07-04 00:22:49.3793 DEBUG We are NOT logged in, opening login page
2020-07-04 00:22:49.3793 DEBUG Puppeteer is in headless mode, restarting in full mode
2020-07-04 00:22:50.4189 DEBUG Downloading browser
2020-07-04 00:22:50.4189 DEBUG Launching browser
2020-07-04 00:22:50.6886 DEBUG Opening new page
2020-07-04 00:22:50.7399 DEBUG Creating IWebBrowser
2020-07-04 00:22:56.3797 FATAL Fatal error, application will be closed: PuppeteerSharp.TargetClosedException: Target closed (Session closed)
at PuppeteerSharp.Page.WaitForRequestAsync(Func2 predicate, WaitForOptions options) at PatreonDownloader.PuppeteerEngine.Wrappers.Browser.WebPage.WaitForRequestAsync(Func2 predicate, WaitForOptions options) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\Wrappers\Browser\WebPage.cs:line 56
at PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever.Login() in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCookieRetriever.cs:line 66
at PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever.RetrieveCookies() in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCookieRetriever.cs:line 93
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, Boolean headlessBrowser, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 117
at PatreonDownloader.App.Program.Main(String[] args) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 58
I am not sure if I was to provide the cookie in a file somewhere, the browser opens, loads patreon then closes a second afterwards.
On Manjaro, the browser was not opening at all with firejail enabled, disabled firejail but sadly didn't get much further.
42574421_description.txt
42252813_description.txt
41382370_description.txt
attached are some examples of post descriptions where external links aren't downloaded when embedded. i'm not sure if this only affects MEGA links or not.
the post(s) look like this on patreon: https://i.imgur.com/0VEJBEE.png
Instead of just skipping the link when unavailable, the program stops after attempting to download an unavailable file. Because of this, it is impossible for it to ever finish.
2021-04-03 09:21:57.5200 ERROR [PatreonDownloader.GoogleDriveDownloader.Plugin] GOOGLE DRIVE ERROR: The service drive has thrown an exception: Google.GoogleApiException: Google.Apis.Requests.RequestError
File not found: 1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI. [404]
Errors [
Message[File not found: 1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI.] Location[fileId - parameter] Reason[notFound] Domain[global]
]
at Google.Apis.Requests.ClientServiceRequest1.ParseResponse(HttpResponseMessage response) at Google.Apis.Requests.ClientServiceRequest1.Execute()
at PatreonDownloader.GoogleDriveDownloader.GoogleDriveEngine.Download(String id, String path, Boolean overwrite)
at PatreonDownloader.GoogleDriveDownloader.Plugin.Download(CrawledUrl crawledUrl, String downloadDirectory)
2021-04-03 09:21:57.5260 FATAL [PatreonDownloader.App.Program] Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.PatreonDownloaderException: Error while downloading https://drive.google.com/open?id=1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI: Unable to download https://drive.google.com/open?id=1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI
---> PatreonDownloader.Common.Exceptions.DownloadException: Unable to download https://drive.google.com/open?id=1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI
---> The service drive has thrown an exception: Google.GoogleApiException: Google.Apis.Requests.RequestError
File not found: 1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI. [404]
Errors [
Message[File not found: 1HvIsC6MDN6kTXm1Ca9q_CTS8TvHseMyI.] Location[fileId - parameter] Reason[notFound] Domain[global]
]
I don't know if this happens with dropbox or mega either, but it would be appreciated if it just skips them if they don't dl.
Is there an ability, didn't see it in the "--help" that will allow for the downloading of a specific post or a range of dates. Also, is there a method to test the code to a smaller patreon that has a video link to youtube? (want to see about implementing the ability to call python ala youtube-dl)
Hi. I ran some tests. I compared the file downloaded by the app with the "save as" of the browser.
The dimensions of the photos are ok. I noticed a sudden difference in the file size.
Example.
Save as File: 600kb
App file: 200kb.
Is that so?
Any way to fix broken download links from GFYCAT site?
2021-05-01 14:01:48.3073 ERROR Failed to download https://gfycat.com/LimpWhoppingBasil: Error while downloading https://gfycat.com/LimpWhoppingBasil: [25759172] Unable to retrieve name for external entry of type ExternalUrl: https://gfycat.com/LimpWhoppingBasil
When I manually go to this URL eg https://gfycat.com/LimpWhoppingBasil it seems to redirect to https://www.redgifs.com/watch/limpwhoppingbasil
I was able to get this running on Linux with Dotnet Core really easily. Steps were:
dotnet restoredotnet msbuild -property:Configuration=ReleasePatreonDownloader.App/bin/Release/netcoreapp3.1/PatreonDownloader.AppJust FYI in case you're interested or want to add instructions.
Is it possible to implement the ability to ignore duplicates? This would provide the ability to pause/resume, or to continue in case something disconnects or goes out.
I don't get it how to use this tool.
PatreonDownloader.exe --help leads to "Command not found"
PatreonDownloader.App.exe --help leads to "Access denied"
It's my first time using the app. I've logged in the downloaded browser and now the app exits and I'm getting this error:
C:\Users\user\Downloads\net3.1-win-x64-release>PatreonDownloader.App.exe --url "https://www.patreon.com/m/xxx/posts" --download-directory c:\xxx --descriptions --embeds --campaign-images --json --verbose
2021-01-16 16:15:37.9460 DEBUG Initializing PuppeteerEngine with parameters , True
2021-01-16 16:15:38.0084 DEBUG Found 15 chrome processes (not sure which one yet)
2021-01-16 16:15:38.0240 INFO Retrieving cookies...
2021-01-16 16:15:38.0396 DEBUG Calling login check
2021-01-16 16:15:38.0552 DEBUG Retrieving browser
2021-01-16 16:15:38.0552 DEBUG Downloading browser
2021-01-16 16:15:38.0708 DEBUG Launching browser
2021-01-16 16:15:38.8330 DEBUG Opening new page
2021-01-16 16:15:39.1542 DEBUG Creating IWebBrowser
2021-01-16 16:15:39.2834 DEBUG Checking login status
2021-01-16 16:15:44.1108 DEBUG We are logged in
2021-01-16 16:15:44.1368 DEBUG Retrieving browser
2021-01-16 16:15:44.1368 DEBUG Retrieving cookies
2021-01-16 16:16:17.5231 FATAL Fatal error, application will be closed: PuppeteerSharp.NavigationException: Timeout of 30000 ms exceeded
---> System.TimeoutException: Timeout of 30000 ms exceeded
at PuppeteerSharp.Helpers.TaskHelper.<>c__DisplayClass2_0.<WithTimeout>b__0()
at PuppeteerSharp.Helpers.TaskHelper.WithTimeout(Task task, Func`1 timeoutAction, TimeSpan timeout, CancellationToken cancellationToken)
at PuppeteerSharp.FrameManager.NavigateFrameAsync(Frame frame, String url, NavigationOptions options)
--- End of inner exception stack trace ---
at PuppeteerSharp.FrameManager.NavigateFrameAsync(Frame frame, String url, NavigationOptions options)
at PatreonDownloader.PuppeteerEngine.Wrappers.Browser.WebPage.GoToAsync(String url, Nullable`1 timeout, WaitUntilNavigation[] waitUntil) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\Wrappers\Browser\WebPage.cs:line 30
at PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever.RetrieveCookies() in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCookieRetriever.cs:line 124
at PatreonDownloader.App.Program.RunPatreonDownloader(String url, Boolean headlessBrowser, Uri remoteBrowserAddress, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 135
at PatreonDownloader.App.Program.Main(String[] args) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 74
2021-05-11 20:02:27.8874 FATAL [PatreonDownloader.Engine.PatreonDownloader] Cookie validation failed: PatreonDownloader.Engine.Exceptions.CookieValidationException: __cfduid cookie not found
at PatreonDownloader.Engine.Stages.Initialization.CookieValidator.ValidateCookies(CookieContainer cookieContainer) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\Stages\Initialization\CookieValidator.cs:line 33
at PatreonDownloader.Engine.PatreonDownloader.Download(String url, PatreonDownloaderSettings settings) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\PatreonDownloader.cs:line 194
command: PatreonDownloader.App.exe --url https://www.patreon.com/xxx/posts --download-directory C:\Users\This-PC\Pictures --descriptions --embeds --campaign-images --json
Hi, @AlexCSDev
question if it would be possible to create folder by post name and content.
So the Pictures would get kinda sortet and not just all dumped into one folder.
I am running dotnet run and dotnet publish -c Release -r win-x64 --self-contained -f netcoreapp3.1 -o bin\publish\net3.1-win-x64-release in PatreonDownloader.App directory in powershell. None of the commands create the bin directory.
There are some http errors (like 503 and 525) which may indicate that there are temporary server or connectivity issues. Downloader should perform serveral download retries before abandoning attempts to download file.
I have recently got access to creator with a lot of mega.nz links, so I plan to make mega.nz plugin available starting with 0.9.4
There's this open source tool that's really useful to download instagram posts https://github.com/instaloader/instaloader
Hello, I think I'm doing everything as per the instructions but I'm getting this error over and over with two different creators. I don't know anything about programming - I'm just trying to use the downloader. please help.
here's the log
2020-11-03 - Copy.log
Currently all classes are being manually injected into each other. This is starting to become a mess so automatic DI solution is needed.
For example:
(NSFW) https://www.patreon.com/m/605785/posts
I tried to use m/605785 as the creator name.
Well, it started to work but stalled at the downloading progress.
The download folder it created was download\m\605785.
I am not sure whether this is correct or not.
Thanks.
I'm probably just a little slow, but the instructions aren't entirely clear as to where the URL should be placed. Whenever I run the program command prompt opens then immediately closes yet nothing happens. I've tried all methods of usage except the Google Drive method, so I'm trying to figure out what I missed.
Hello
Where should i input my account login and password please ?
I am attempting to use the PatreonDownloader application via WSL2 which should function as a typical linux machine would (in theory). I've configured my WSL2 distribution to support gui applications via an X-server (however in this case PatreonDownloader attempts to use the native Windows chrome application). This behavior would be ok for my use-case however it generates the error message below. However, I'm not convinced that it would function even if it did use WSL2's chrome.
Here are the instructions I followed to setup dotnet a few months ago:
https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#install-the-sdk
Here are my dotnet sdks + runtimes:
dotnet --list-sdks
3.1.404 [/usr/share/dotnet/sdk]
dotnet --list-runtimes
Microsoft.AspNetCore.App 3.1.10 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 3.1.10 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Here is the command I am running: dotnet run --verbose --url https://www.patreon.com/tacticalmap/posts
Here are the logs generated:
2021-01-14 12:02:00.5076 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , True 2021-01-14 12:02:00.5280 INFO [PatreonDownloader.App.Program] Retrieving cookies... 2021-01-14 12:02:00.5280 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Calling login check 2021-01-14 12:02:00.5280 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Retrieving browser 2021-01-14 12:02:00.5280 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser 2021-01-14 12:02:00.5382 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser 2021-01-14 12:02:00.7632 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page 2021-01-14 12:02:00.8546 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser 2021-01-14 12:02:00.8785 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Checking login status 2021-01-14 12:02:01.2029 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] We are NOT logged in, opening login page 2021-01-14 12:02:01.2029 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Puppeteer is in headless mode, restarting in full mode 2021-01-14 12:02:02.2309 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , False 2021-01-14 12:02:02.2309 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser 2021-01-14 12:02:02.2309 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser 2021-01-14 12:02:02.4143 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page 2021-01-14 12:02:02.4562 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser 2021-01-14 12:02:07.1358 FATAL [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Login error: Target closed (Session closed) 2021-01-14 12:02:07.1377 FATAL [PatreonDownloader.App.Program] Fatal error, application will be closed: System.Exception: Unable to retrieve cookies at PatreonDownloader.App.Program.RunPatreonDownloader(String url, Boolean headlessBrowser, Uri remoteBrowserAddress, PatreonDownloaderSettings settings) in /home/ivln/workspace/PatreonDownloader/PatreonDownloader.App/Program.cs:line 141 at PatreonDownloader.App.Program.Main(String[] args) in /home/ivln/workspace/PatreonDownloader/PatreonDownloader.App/Program.cs:line 74
I get this error when trying to use this tool. When using no-headless parameter I can see that images aren't loading correctly in the browser. I don't know what can be causing this issue.
Command line and messages received after:
PatreonDownloader.App.exe --url https://www.patreon.com/someone/posts --descriptions true --no-headless
2021-02-21 13:29:12.8628 INFO Retrieving cookies...
2021-02-21 13:29:20.3901 INFO Preparing to download...
2021-02-21 13:29:21.4686 INFO Retrieving campaign information...
2021-02-21 13:29:26.9974 INFO Crawling...
2021-02-21 13:29:27.0432 FATAL Fatal error, application will be closed: System.ArgumentException: Campaign cover url cannot be null or empty
at PatreonDownloader.Engine.Stages.Crawling.PageCrawler.Crawl(CampaignInfo campaignInfo, PatreonDownloaderSettings settings, String downloadDirectory) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\Stages\Crawling\PageCrawler.cs:line 42
at PatreonDownloader.Engine.PatreonDownloader.Download(String url, PatreonDownloaderSettings settings) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\PatreonDownloader.cs:line 240
at PatreonDownloader.App.Program.RunPatreonDownloader(String url, Boolean headlessBrowser, Uri remoteBrowserAddress, PatreonDownloaderSettings settings) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 161
at PatreonDownloader.App.Program.Main(String[] args) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 74
I succeeded in downloading the file through the first released file. However, from the second attempt, this error message appears. Plz help!
PatreonDownloader.app --creator #vinectschwenk --download-directory c:\downloadss --descriptions --embeds
2020-05-13 01:04:32.7909 INFO Retrieving cookies...
2020-05-13 01:05:16.4248 INFO Preparing to download...
2020-05-13 01:05:17.7809 INFO Retrieving campaign information...
2020-05-13 01:05:20.3790 FATAL Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.PatreonDownloaderException: Unable to retrieve campaign id
at PatreonDownloader.Engine.PatreonDownloader.Download(String creatorName, PatreonDownloaderSettings settings) in C:\Users\jeong\Downloads\PatreonDownloader-dev\PatreonDownloader-dev\PatreonDownloader.Engine\PatreonDownloader.cs:line 162
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, Boolean headlessBrowser, PatreonDownloaderSettings settings) in C:\Users\jeong\Downloads\PatreonDownloader-dev\PatreonDownloader-dev\PatreonDownloader.App\Program.cs:line 136
at PatreonDownloader.App.Program.Main(String[] args) in C:\Users\jeong\Downloads\PatreonDownloader-dev\PatreonDownloader-dev\PatreonDownloader.App\Program.cs:line 58
Pretty much what it says on the tin. Either by just setting the url to patreon.com/home or as a - setting that replaces -url. Downloaded files should be split into folders for each creator.
Unable to log in with google account
Getting a bunch of these instead of the zip files hoped for, Normal download via browser works.
Error while downloading https://www.patreon.com/file?h=37188912&i=7097946: [45879210] Unable to retrieve name for external entry of type ExternalUrl: https://www.patreon.com/file?h=37188912&i=7097946
How can I solve this problem? plz help..
2020-04-24 20:43:32.9199 FATAL Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.PatreonDownloaderException: Unable to retrieve campaign id
at PatreonDownloader.Engine.PatreonDownloader.Download(String creatorName, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\PatreonDownloader.cs:line 162
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, Boolean headlessBrowser, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 136
at PatreonDownloader.App.Program.Main(String[] args) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 58
Hello.
First of all, thanks for the amazing software!
I found this bug on Windows using this command:
PatreonDownloader.App.exe --url https://www.patreon.com/user/posts?u=9011118
Than application tries to write content to the default folder download\<user_name>. But in this case the <user_name> contains a trailing space and looks like this "ducky 3d ". Looks like Windows doesn't like trailing spaces in a path. So it couldn't write files to the default folder.
Yes, I fixed that using "--download-directory"
Every time I open the exe, it instantly closes
Using all variations of PatreonDownloader.App.exe --creator CreatorName alone or to dl everything we get
2020-02-15 23:12:54.8364 FATAL [PatreonDownloader.App.Program] Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.PatreonDownloaderException: Unable to retrieve campaign id: Unable to download from https://www.patreon.com/xxxx/posts: Response status code does not indicate success: 403 (Forbidden). ---> PatreonDownloader.Engine.Exceptions.DownloadException: Unable to download from https://www.patreon.com/xxxx/posts: Response status code does not indicate success: 403 (Forbidden). ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(Task`1 getTask)
at PatreonDownloader.Engine.WebDownloader.DownloadString(String url) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.Engine\WebDownloader.cs:line 75
--- End of inner exception stack trace ---
at PatreonDownloader.Engine.WebDownloader.DownloadString(String url) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.Engine\WebDownloader.cs:line 79
at PatreonDownloader.Engine.Stages.Crawling.CampaignIdRetriever.RetrieveCampaignId(String url) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.Engine\Stages\Crawling\CampaignIdRetriever.cs:line 33
--- End of inner exception stack trace ---
at PatreonDownloader.Engine.Stages.Crawling.CampaignIdRetriever.RetrieveCampaignId(String url) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.Engine\Stages\Crawling\CampaignIdRetriever.cs:line 46
at PatreonDownloader.Engine.PatreonDownloader.Download(String creatorName, PatreonDownloaderSettings settings) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.Engine\PatreonDownloader.cs:line 149
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, PatreonDownloaderSettings settings) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.App\Program.cs:line 131
at PatreonDownloader.App.Program.Main(String[] args) in D:\Desktop\PatreonDownloader-dev\PatreonDownloader.App\Program.cs:line 55
I get the same error as before this new version when trying to download from a patreon. Please can you fix a timeout for this issue?
I get the trouble after going through a poll and when it dies on the same place everytime.
2021-01-08 20:30:02.4218 ERROR Verification for 149135: Unknown type for "included": poll
2021-01-08 20:30:34.5517 FATAL Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.DownloadException: Unable to download from https://www.patreon.com/api/posts?include=user%2Cattachments%2Ccampaign%2Cpoll.choices%2Cpoll.current_user_responses.user%2Cpoll.current_user_responses.choice%2Cpoll.current_user_responses.poll%2Caccess_rules.tier.null%2Cimages.null%2Caudio.null&fields%5Bpost%5D=change_visibility_at%2Ccomment_count%2Ccontent%2Ccurrent_user_can_delete%2Ccurrent_user_can_view%2Ccurrent_user_has_liked%2Cembed%2Cimage%2Cis_paid%2Clike_count%2Cmin_cents_pledged_to_view%2Cpost_file%2Cpost_metadata%2Cpublished_at%2Cpatron_count%2Cpatreon_url%2Cpost_type%2Cpledge_url%2Cthumbnail_url%2Cteaser_text%2Ctitle%2Cupgrade_url%2Curl%2Cwas_posted_by_campaign_owner&fields%5Buser%5D=image_url%2Cfull_name%2Curl&fields%5Bcampaign%5D=show_audio_post_download_links%2Cavatar_photo_url%2Cearnings_visibility%2Cis_nsfw%2Cis_monthly%2Cname%2Curl&fields%5Baccess_rule%5D=access_rule_type%2Camount_cents&fields%5Bmedia%5D=id%2Cimage_urls%2Cdownload_url%2Cmetadata%2Cfile_name&sort=-published_at&filter%5Bcampaign_id%5D=650363&filter%5Bis_draft%5D=false&filter%5Bcontains_exclusive_posts%5D=true&json-api-use-default-includes=false&json-api-version=1.0&page%5Bcursor%5D=C_gAIHZ3jTWm9Y-gNxkf_ubiQ4c: Timeout of 30000 ms exceeded
---> PuppeteerSharp.NavigationException: Timeout of 30000 ms exceeded
---> System.TimeoutException: Timeout of 30000 ms exceeded
at PuppeteerSharp.Helpers.TaskHelper.<>c__DisplayClass2_0.b__0()
at PuppeteerSharp.Helpers.TaskHelper.WithTimeout(Task task, Func1 timeoutAction, TimeSpan timeout, CancellationToken cancellationToken) at PuppeteerSharp.FrameManager.NavigateFrameAsync(Frame frame, String url, NavigationOptions options) --- End of inner exception stack trace --- at PuppeteerSharp.FrameManager.NavigateFrameAsync(Frame frame, String url, NavigationOptions options) at PatreonDownloader.PuppeteerEngine.Wrappers.Browser.WebPage.GoToAsync(String url, Nullable1 timeout, WaitUntilNavigation[] waitUntil) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\Wrappers\Browser\WebPage.cs:line 33
at PatreonDownloader.Engine.WebDownloader.DownloadString(String url) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\WebDownloader.cs:line 104
--- End of inner exception stack trace ---
at PatreonDownloader.Engine.WebDownloader.DownloadString(String url) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\WebDownloader.cs:line 104
at PatreonDownloader.Engine.Stages.Crawling.PageCrawler.Crawl(CampaignInfo campaignInfo, PatreonDownloaderSettings settings, String downloadDirectory) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\Stages\Crawling\PageCrawler.cs:line 71
at PatreonDownloader.Engine.PatreonDownloader.Download(String url, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Engine\PatreonDownloader.cs:line 240
at PatreonDownloader.App.Program.RunPatreonDownloader(String url, Boolean headlessBrowser, Uri remoteBrowserAddress, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 161
at PatreonDownloader.App.Program.Main(String[] args) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 74
Pastebin link of stacktrace here: https://pastebin.com/ZDC6XaqJ
Username was changed to <REDACTED> just so it's not in the issue tracker forever, but it is just their username and doesn't have any symbols or anything.
Using the latest available pre-compiled windows version of the tool. Downloading other creators seems to work fine but it's just this specific creator that seems to be getting a timeout error. Will send the name of the creator privately if it's needed to diagnose this.
I'd like to use this but I'm entirely unfamiliar with building .NET code. I've tried loading it up in Visual Studio with the .net core 2.2 runtime installed and it builds/publishes dll files, but never a .exe.
I've tried building the .sln file and each individual project within and just can't figure out how to get an .exe out of this code.
I've also tried dotnet build and dotnet publish in the repo root and in each project and get the same results (only dlls, no executable)
Thank you!
Hi,
First, thanks a lot for this application, it's very useful :)
Unfortunately, I noticed some issue with medias with no name or the same name. I have several creators I'm supporting who post several images in the same post and all of them have the same name "1.jpg". PatreonDownloader only keeps one version of them and doesn't rename it, so if there are 10 images with the same name in the same post, there will be only one 1.jpg :(
Please could you add a fix for it ? Perhaps just checking if the file is already present and adding a suffix.
Thanks :)
If I have a little time, I'll check your code to see if I can fix the issue myself :)
Downloading big files results in huge amount of memory being wasted due to the fact that WebDownloader asks _httpClient to buffer the file completely in memory.
This should be fixed by using calling _httpClient.SendAsync with HttpCompletionOption.ResponseHeadersRead parameter.
Unfortunately using browser's user agent and its cookies is not enough. For some reason httpclient triggers cloudflare's cookie protection.
It is possible to fix this by using browser for api calls but this is last resort solution because I want to use browser as less as possible. So right now the task is to research possible bypasses for that protection.
Receiving the following error:
PS C:\Users\Grygon\Documents\PatreonDownloader\PatreonDownloader.App> dotnet run --creator XXXXX --download-directory c:\patron-downloads --descriptions --embeds --campaign-images --json
2020-02-15 21:14:09.2670 INFO Retrieving cookies...
2020-02-15 21:14:12.3122 INFO Preparing to download...
2020-02-15 21:14:12.3411 FATAL Fatal error, application will be closed: PatreonDownloader.Engine.Exceptions.CookieValidationException: __ssid cookie not found
at PatreonDownloader.Engine.Stages.Initialization.CookieValidator.ValidateCookies(CookieContainer cookieContainer) in C:\Users\Grygon\Documents\PatreonDownloader\PatreonDownloader.Engine\Stages\Initialization\CookieValidator.cs:line 34
at PatreonDownloader.Engine.PatreonDownloader.Download(String creatorName, PatreonDownloaderSettings settings) in C:\Users\Grygon\Documents\PatreonDownloader\PatreonDownloader.Engine\PatreonDownloader.cs:line 114
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, PatreonDownloaderSettings settings) in C:\Users\Grygon\Documents\PatreonDownloader\PatreonDownloader.App\Program.cs:line 131
at PatreonDownloader.App.Program.Main(String[] args) in C:\Users\Grygon\Documents\PatreonDownloader\PatreonDownloader.App\Program.cs:line 55
Tried to run app, then log in. I see the window loading (with the logged-in view) each time I run it, then it crashes.
Hello , I met an error when I used it.
COMMAND: PatreonDownloader.App.exe --creator https://www.patreon.com/u_u_ASMR
OUTPUT:
2020-10-24 16:31:07.4316 INFO Retrieving cookies...
2020-10-24 16:31:52.3954 FATAL Fatal error, application will be closed: System.NullReferenceException: Object reference not set to an instance of an object.
at PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever.Login() in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCookieRetriever.cs:line 49
at PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever.RetrieveCookies() in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCookieRetriever.cs:line 93
at PatreonDownloader.App.Program.RunPatreonDownloader(String creatorName, Boolean headlessBrowser, PatreonDownloaderSettings settings) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 118
at PatreonDownloader.App.Program.Main(String[] args) in E:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 59
Thank you in advance
I would like to run this completely headlessly on a server. Unfortunately, it doesn't look like that would work, since the current auth flow logs in via a Chrome browser window.
Would it be possible to accept the login cookies via CLI arg, or otherwise allow fully headless usage?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.