captn3m0/muse-dl

Download and stitch books from Project MUSE

Hi there 📚 🎲 🐘

• I did security, infrastructure, and open-source things at Razorpay.
• Ask me about boardgames, infosec, self-hosting, and Kubernetes or my time at @recurse.
• How to reach me: captnemo.in/contact.
• Pronouns: He/him.
• Here's my current setup, my security setup, and my homeserver setup.
• Here are my various projects, and talks, and ideas. I like working on ebooks.
• Fun fact: I don't have an Aadhaar.

Here's some of my recent writing that you might like reading:

• Writing on books - Fantastic Beasts and Where to Find Them
• Analysing the Indian government cyberspace
• A survey of OneCardToRuleThemAll companies
• The game breaking Deal Breaker card

A few twitter threads:

• A response to Square Crypto's whitepaper titled "Bitcoin is Key to an Abundant, Clean Energy Future".
• Hotstar's differential pricing strategy and why I think its bad.
• Cool dev tools
• Help revive RSS (Really Simple Syndication)

You can find a mirror of most of my code on git.captnemo.in. I have a backup GitHub account @captnemo.

muse-dl's Issues

Actually cleanup

The --dont-cleanup flag doesn't make a difference, since we aren't really cleaning up properly.

Fix this on priority.

Support to download journals

This shouldn't be that hard, except that we'll have to add "Journal Discovery" first.

EPUB file generation for non-PDF books

Might be a tad bit difficult, and require a large refactor, since there are lots of PDF-assumptions in the code

Crash on dowloading

Hi,

downloading the book:

https://muse.jhu.edu/book/475

crash the software:

Downloaded 1865151
Downloaded 1865152
Downloaded 2183800
Downloaded 2183798
Downloaded 8830
Downloaded 8831
Downloaded 8832
Downloaded 8833
Downloaded 8834
Downloaded 8835
Downloaded 8836
Downloaded 8837
Downloaded 8838
Downloaded 8839
Downloaded 8840
Downloaded 8841
Downloaded 8842
Downloaded 8843
Unhandled exception: (no message) (Muse::Dl::Errors::CorruptFile)
  from /Users/fox/Downloads/muse-dl/muse-dl/src/pdftk.cr:103:9 in 'stitch'
  from /Users/fox/Downloads/muse-dl/muse-dl/src/muse-dl.cr:37:11 in 'dl'
  from /Users/fox/Downloads/muse-dl/muse-dl/src/muse-dl.cr:59:11 in 'run'
  from /Users/fox/Downloads/muse-dl/muse-dl/src/muse-dl.cr:68:1 in '__crystal_main'
  from /usr/local/Cellar/crystal/0.33.0/src/crystal/main.cr:106:5 in 'main_user_code'
  from /usr/local/Cellar/crystal/0.33.0/src/crystal/main.cr:92:7 in 'main'
  from /usr/local/Cellar/crystal/0.33.0/src/crystal/main.cr:115:3 in 'main'

is it fixable?

Regards,
S.

--help shows help twice

This is with the newest static build on the releases page.

$ ./muse-dl --help
Usage: muse-dl [--flags] [URL|INPUT_FILE]

URL: A link to a book on the Project MUSE website, eg https://muse.jhu.edu/book/875
INPUT_FILE: Path to a file containing a list of links

    --no-cleanup                     Don't cleanup temporary files
    --tmp-dir PATH                   Temporary Directory to use
    --output FILE                    Output Filename
    --no-bookmarks                   Don't add bookmarks in the PDF
    --input-pdf INPUT                Input Stitched PDF. Will not download anything
    --clobber                        Overwrite the output file, if it already exists. Not compatible with input-pdf
    --cookie COOKIE                  Cookie-header
    -h, --help                       Show this help
Usage: muse-dl [--flags] [URL|INPUT_FILE]

URL: A link to a book on the Project MUSE website, eg https://muse.jhu.edu/book/875
INPUT_FILE: Path to a file containing a list of links

    --no-cleanup                     Don't cleanup temporary files
    --tmp-dir PATH                   Temporary Directory to use
    --output FILE                    Output Filename
    --no-bookmarks                   Don't add bookmarks in the PDF
    --input-pdf INPUT                Input Stitched PDF. Will not download anything
    --clobber                        Overwrite the output file, if it already exists. Not compatible with input-pdf
    --cookie COOKIE                  Cookie-header
    -h, --help                       Show this help````

Easier captcha bypass

Maybe automatically open a browser window against a local running webserver with the captcha and submit the captcha on the backend?

Switch to GHCR

DockerHub is no longer hosting old images, and GHCR is much better.

Set output directory

Fairly easy todo. Right now we drop everything in the current working directory. It will be nice to have a --output-directory flag so it can be run more easily.

instruction to use in catalina 10.15.6 or high Sierra

Hello: would you be able to provide instructions on how to install on either catalina or high Sierra OSX? I am not very knowledgable but can follow instructions. I am very bored of dl chapter by chapter then stitching -- so would hugely appreciate your script working in my environment.Thankyou for your time.