cebtenzzre/tumblr-utils

A fork of tumblr-utils with Python 3 support, bug fixes, and lots of features I found useful.

tumblr-utils

About this fork

This fork is focused on tumblr_backup.py. It adds Python 3 compatibility, various bug fixes, a few enhancements to normal operation, support for dashboard-only blogs, and several other features - see the output of tumblr_backup.py --help for the full list of options. Check out the "experimental" branch if you want to try out some less stable extra functionality.

---

This is a collection of utilities dealing with Tumblr blogs.

• tumble.py creates new posts from RSS or Atom feeds
• tumblr_backup.py makes a local backup of posts and images
• mail_export.py mails tagged links to a recipient list

These scripts are or have been useful to me over the years.

More documentation can be found in each script's docstring or in tumblr_backup.md.

The utilities run under Python 2.7, except for tumblr_backup.py and tumblr_login.py, which run on Python 3.6 and later.

Notice

On 2015-06-04, I made the v2 API the default on the master branch. The former master branch using the v1 API is still available on Github as api-v1, but will no longer be updated. The one feature that's only available with the old API is the option to backup password-protected blogs. There's no way to pass a password in Tumblr's v2 API.

License

GPL3.

tumblr-utils's Issues

Download hidden tumblr

Hi,
sorry to disturb you. I've managed to use the tumbrl-utils to download normal blogs, and it works without any problems. Your backup.py works very well as well.
But I just can't understand how you can use login.py and the other files to download hidden blogs. I've tried to copy the files in the same folder of backup.py, but it doesn't work (the folder with the blog name is created, but there isn't anything inside).
May you explain me how it can be done? Sorry, i'm just a newbie.
Thank you for your attention.

index.html contains a title but is missing the actual index

Have recently been trying out fresh installs of Tumblr-Utils (the latest version: June 2023.) with just the API key added, while trying to check for any issues to help others out that wanted to backup their Tumblr.

In the latest version, can confirm that "no module named 'fcntl'" error appears.

(I use Windows)

With Python 3.10, the error doesn't appear when just commenting out "import fcntl" in util.py. So at least there's that.

While fcntl was intended as replacement for deprecated imghdr, as your comment here for the May 16 2023 commit says:
f89c663

For Python 3.11, since imghdr is deprecated as a fallback, and fcntl not appear, it would cause some issues.

It ends up skipping image downloads as-is while backing up, and just links to tumblr hosted images, similar to when using the -k command... except not actually using the command. (This ends up happening to someone I was trying to help, who uses Mac OS and first tried with Python 3.11)

The error in question (not mine):
without commenting out "import fcntl"

with commenting out "import fcntl"

Still, even with commenting out "import fcntl" so that bug doesn't appear, the index page will be blank besides the tumblr title and description.

... Though this also happens on fresh installs (with just API key added) of an older tumblr-utils I had downloaded before. So it doesn't seem related as initially thought.

The same thing of blank index page happens for those am trying to help with their fresh installs, including on MacOS. At least 2 other people.

Meanwhile, my old install (not fresh install) on a different hard drive, using January 2022 tumblr-utils, does not have either issue and can make a proper index page.
(the same blog, on same day)

And have been trying figure out why the blank index page thing happens, but have no idea.
When the script runs, these fresh installs (of January 2022, and latest of June 2023) don't flash the message "building index page" before finishing, unlike the old one I use.

Am reporting both because when tried to "fix" one issue, the other popped up right after.

SSLError: certificate verify failed

Hi @cebtenzzre,

Thanks for your response on my issue on the other repo. I've moved everything across to your fork but a different problem has cropped up. I get the following error for all the sites I try:

[wget] URL is https://66.media.tumblr.com/tumblr_DETAIL.jpg [wget] Error retrieving resource: urllib3 reached a retry limit. Caused by: WGHTTPSConnectionPool(host='66.media.tumblr.com', port=443): Max retries exceeded with url: /tumblr_DETAIL.jpg (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

I'm guessing this might not be related to your script at all. I would appreciate it if you could point me in the right direction to try and resolve this. I'm on OSX 10.14.6. I have tried running the wget command by itself and that seems to work fine.

Please let me know if you need more information.

Program stalls/can't download entire blog

Downloaded 3 days ago, have been trying since then and don't know what I'm doing wrong. I know pretty much nothing about Python or any coding language so this is all pretty new to me.

I've tried the all of these variations of the command:
tumblr_backup.py -i --save-video --save-audio --tag-index blog-name
tumblr_backup.py --save-video --save-audio --tag-index blog-name
tumblr_backup.py -i --save-video --save-audio --tag-index -p year blog-name
tumblr_backup.py -i --save-video --save-audio --tag-index -p year-month blog-name
tumblr_backup.py --save-video --save-audio --tag-index -p year-month blog-name

The first two would work at first, but eventually result in a stall, with a message like "downloading 7000 to 7050" that never moved again. I saw people saying this would be fixed with the -p command, so I tried that. It worked for most of my blog (2016 to 2020), but I got the same stall once I tried 2021. So then, I tried adding the month to the command. After some frustration with the program telling me "Stopping backup: Incremental backup complete, 0 posts backed up", I took out the -i command and that seemed to work. But now I am stuck again, this time on the message "Waiting for worker threads to finish." I don't know what's causing these stalls or how to fix them. I had seen some people saying it could be caused by the fancy/colored text offered in more recent Tumblr updates, but the post that seemed to stall one of my "year-month" attempts didn't have any of that, it was just an image.

_retrieve_loop fails with FileNotFoundError on Windows

Am I missing something or are you supposed to be able to run this exactly like the original one made by bbolli? I have both 2.7 and 3.9 for python and I can run the original just fine but when I try and run this version all I get is this long mess in command prompt with empty folders. It creates the folder for the blog, and then a blank folder for both media and posts.

Any help would be appreciated. I created the video for the original script and have had people coming to me for a while trying to solve some of these kinds of errors.

Error When Running.txt

os.fdatasync not supported on macos

I'm running the latest commit from the cebtenzzre branch: 3582d8d

I get the following exception since os.fdatasync is not supported on macos: https://docs.python.org/3/library/os.html#os.fdatasync

Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "tumblr_backup.py", line 1600, in handler
    work()
  File "tumblr_backup.py", line 1092, in save_content
    append_try('body')
  File "tumblr_backup.py", line 1076, in append_try
    elt = re.sub(r'''(?i)(<img\s(?:[^>]*\s)?src\s*=\s*["'])(.*?)(["'][^>]*>)''',
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "tumblr_backup.py", line 1255, in get_inline_image
    saved_name = self.download_media(image_url, image_filename)
  File "tumblr_backup.py", line 1336, in download_media
    wget_retrieve(url, open_file(lambda f: f, path_parts), post_timestamp=self.post['timestamp'])
  File "/Users/dustinlane/tumblr-utils-fork/tumblr-utils/wget.py", line 736, in __call__
    _retrieve_loop(hstat, url, file, post_timestamp, adjust_basename, self.options, self.log)
  File "/Users/dustinlane/tumblr-utils-fork/tumblr-utils/wget.py", line 690, in _retrieve_loop
    os.fdatasync(hstat.dest_dir)
AttributeError: module 'os' has no attribute 'fdatasync'

ImportError: No module named typing

When trying to run the latest version I get the following error:

Traceback (most recent call last):
  File "tumblr_backup.py", line 23, in <module>
    from typing import TYPE_CHECKING
ImportError: No module named typing

I have tried importing the module but to no success.

Only backups up from 2017 til now

Hello, I hope I'm posting this in the right place. If not, please let me know.

I have the problem that my blog only gets backed from 2017 to now, whether I run it with the incremental flag or not. When I try to backup a specific year before 2017 it doesn't work either.

The version: bc43e85
I downloaded this version on the 11th of December.

The first time, which seemed to have got stuck at the time:
tumblr_backup.py --tag-index --save-audio --save-video blogname

The second time, to make sure everything had been got, but which concluded pretty quick, then I noticed it only went up to 2017:
tumblr_backup.py -i --tag-index --save-audio --save-video blogname

The attempt to get something before 2017:
tumblr_backup.py -p 2015 --tag-index --save-audio --save-video blogname

It rolls back media but still counts up from the whole 59000 posts or so, and then when it gets to the relevant posts (i figure) it stops. Then suddenly it goes BEYOND 59000 to 99999 but it still hasn't got anything before 2017. I've tried this several times.

See:

Nothing follows. I just press control C to break things up.

I'd be very grateful if you could take a look at this. My questions basically boil down to: what is causing this andnd how do I fix it?

Thank you very much.

Not all likes are downloaded

Using the latests version I was unable to download all my blog's likes. I have something like 1410 liked posts and the script returned the following:

blogname: 1086 liked posts backed up