andyjsmith/smugmug-downloader

Download all the images from a SmugMug user

smugmug-downloader's Issues

CPU bound when files are already download

First of, thank you for this tool. It's amazing that with ~100 lines of code you put together a tool that gets the job done, with no faff, and a nice UI to boot!

The only thing I find odd is that if you restart it against a partially downloaded collection, it uses 100% CPU for a long time until it gets back to the point where it left off. At first I thought it was checking hashes but reading the code I can see that's not the case.

I'm thinking maybe get_json is slow? Since that's pretty much the only thing that happens in the inner loop.

Downloaded files empty

Hello there,
first of all thanks a lot for sharing this downloader!

I've just started experimenting with it and from the terminal output it would seem I successfully got it to work; album list is downloaded, output directories are created and I can see how the file download progresses, too, and the process closes with a "Completed" message.
When I check the actual files, however, they don't have any content - the correct number of files is created, with the correct file names, but the size is 0 for each one of them, they are just plain empty.

Do you maybe have an idea where the problem might be?

Thank you!
mark

Album titles containing quotation

Hi there.
How do I download albums that contains quote or double quotations in em.
My current solution is to escape the quote with a backslash (). Found this by googeling. But this is not very self-explanatory.
Would be nice if I could just paste in the URL. Feature request btw.
Thanks.

user may not exist or password protected

Hi, I have tried this several times. I have logged into my smugmug account and grabbed my ssms cookie but still get the issue. i have even turned the site lock off so there is only one main password to get the account. any thoughts or suggestions?

Thank you

Receiving "Failed to download image:" on Every Image

I tried to download some specific albums and every album came back with the same error message for every picture. Alongside a [!]. I cannot see what error is occurring, just that it failed. Is there a way to see why it failed?

Syntax Error in line 78 and line 97

Good morning.

py -3 gets me this error:

File "smdl.py", line 78
desc=f"{fg('yellow')}{attr('bold')}{format_label('All Albums')}{attr('reset')}"):

the error in question is the last " mark. I imagine it's a tqdm version mismatch, but it could be something else.

Thanks you.

-

Hi.

Sub-Album

How do I designate a sub-album? Target SM user has a large album folder and I want to download an album / album tree inside the folder.
I understand that the --album argument exists, but not sure how to designate the sub-album.
Thanks,

Doesn't download hidden folders anymore

Here's an example

tony-harris.smugmug.com/

tried it and can't get

tony-harris.smugmug.com/Renee/n-4bkZFK/

what could it be?

Frequent errors and exit with very large collection

Firstly, thank you so much for a one-of-a-kind tool. This is a lifesaver!

I'm running into an issue with a very large collection (9786 albums, 487886 photos). It was easy to set up the job and run it, but I noticed after about 5-10 minutes I would randomly receive an error along the lines of the following... If I simply run the same command again the job restarts, scans all completed albums to find its spot again, and the error does not occur in the same spot, so it seems to not be a problem with the actual album or file from the first time around.

Traceback (most recent call last):
  File "smdl.py", line 108, in <module>
    image_req = get_json(image["Uris"][largest_media]["Uri"])
  File "smdl.py", line 45, in get_json
    return json.loads(pres[-1].text)
IndexError: list index out of range

At the moment I'm using a tool called "immortal" to run this python script supervised by another program which simply restarts it automatically if the job exits, but as I'm sure you can imagine the fact that it has to start again by scanning every single album means that this will exponentially grow in time to complete, and I have a feeling it might be impossible to complete for that reason with so many albums.

I have run apt update/upgrade. Have tried Windows and Ubuntu (seems to run longer on Ubuntu so that's what I'm using now).

If the tool simply continued through to the end, it would take a while but I know it would work perfectly. It's this constant restarting and rescanning that has ended up being a bit of a brick wall for me. Any suggestions?

No albums were found

The last few days I am getting this for every user I try:

"No albums were found for the user (username). The user may not exist or may be password protected."

Tried for multiple users and they were definitely working a week ago

Update instead of full download

Hi there. First - thanks for the script, useful.
Idea - would it be possible to add a flag / function to only check changes (as in rsync) instead of redownloading all the stuff? At first I though that's its default behavior but now I see it goes around again.
Thanks!

Certain album names causing issues

The instructions for downloading specific albums by name says to "split by $". I attempted to get some albums with the following string:

--albums "02. Bride$03. Groom$04. Bride + Groom$05. First Look$06. Bridesmaids$07. Groomsmen$08. Bridal Party$09. Ceremony$10. Family$11. Reception$12. Details"

The output in the terminal kept telling me

Downloading album list...done.
Creating output directories...done.
All Albums              : 100%|███████████████████████████████████████████████████████████████████████████████████████████|  34/34 
Completed.

and wouldn't actually download anything at all. I put in some basic print statements (I'm not a Python dev) so I could see what specificAlbums was coming out as and it was the following:

['02. Bride. Groom. Bride + Groom. First Look. Bridesmaids. Groomsmen. Bridal Party. Ceremony. Family. Reception. Details']

I'm assuming something with the leading 0s was causing an issue perhaps? Anyhow, I was able to solve it by using 02. Bride $ 03. Groom... (spaces around the $) as the delimiter rather than just $ and that seems to have worked in my case, giving the following output for specificAlbums:

['02. Bride', '03. Groom', '04. Bride + Groom', '05. First Look', '06. Bridesmaids', '07. Groomsmen', '08. Bridal Party', '09. Ceremony', '10. Family', '11. Reception', '12. Details']

Just wanted to log this for others who might have the same issue.

Doesn't seem to handle absent or duplicate filenames

Edit: I originally described this issue as "improper handling of duplicate filenames" but it also doesn't handle images with no filename. Somehow I have several galleries containing numerous images with no filename. When I download those galleries via SmugMug's web interface, the resulting zipfile contains images named .jpg, -1.jpg, -2.jpg etc.

I did an Extremely Crude Hack to add an incrementing number to every filename and it does allow the script to get past this problem. However, this also means that there's no file extension on images with a blank filename.

2nd edit: A separate but possibly related issue is that it's possible to have more than one gallery with the same name. Trying to use smdl to retrieve such a gallery leads to a crash. I thought there might be an issue with multiple identically named galleries, but it's really just the empty filename issue happening. However, there's no way to download just one of those multiple galleries - perhaps it would be possible to specify the "URL name" for the gallery, just like what's used for creating the image directories on disk?

Images downloaded does not have the GPS Data

I found downloaded images does not have the GPS Data that the original pic have. Looking thru your code I notice you use the LargestImage, is there a reason why you didn't use ImageDownload. ImageDownload seems to download the original pic with the GPS data. I am thinking I am missing something.

KeyError: 'bar'

Hi, I'm getting this error, using python3.7

Downloading album list...done.
Creating output directories...done.
Traceback (most recent call last):
File "smdl.py", line 78, in
desc=f"{fg('yellow')}{attr('bold')}{format_label('All Albums')}{attr('reset')}"):
File "/Users/mathias/.pyenv/versions/3.7.1/envs/gpt/lib/python3.7/site-packages/tqdm/_tqdm.py", line 945, in init
self.display()
File "/Users/mathias/.pyenv/versions/3.7.1/envs/gpt/lib/python3.7/site-packages/tqdm/_tqdm.py", line 1315, in display
self.sp(self.repr() if msg is None else msg)
File "/Users/mathias/.pyenv/versions/3.7.1/envs/gpt/lib/python3.7/site-packages/tqdm/_tqdm.py", line 984, in repr
return self.format_meter(**self.format_dict)
File "/Users/mathias/.pyenv/versions/3.7.1/envs/gpt/lib/python3.7/site-packages/tqdm/_tqdm.py", line 419, in format_meter
return bar_format.format(**format_dict)
KeyError: 'bar'