Giter Site home page Giter Site logo

Comments (27)

neckothy avatar neckothy commented on September 16, 2024 1

Is this still broken?

Yes. A "fix" would essentially be an entirely new project since the API this relies on no longer exists.

If you're looking for a public tool to back up your ComiXology and Kindle comic purchases, DeDRM_tools or unkindle.py (included in this repo) are likely your best bets for the foreseeable future. Methods for accessing the quality previously offered through the ComiXology API (and by extension, this tool) are not currently public knowledge, so you would need to figure it out yourself or hope someone shares in the future.

from comix.

noaione avatar noaione commented on September 16, 2024 1

Edit: New one: KFX Input Mod.zip since I messed up the ternary order

Quick modification based on v2.6.0, use the calibre-debug CLI:
calibre-debug -r "KFX Input" -- -z -l Book.kfx (either use --lossless or -l, only support CBZ/PDF)

As a side note, this would make the output bigger and slower to process (PNG takes longer to save)

The new Kindle HD didn't looks that much worse with pretty similar JPEG artifacts to CMX HD as far as I can see. Although, if you have a god eyes or something this is 100% not similar quality to CMX HD.

from comix.

rando2345 avatar rando2345 commented on September 16, 2024

The old cmx api is dead atm. No idea what's going to happen next.

from comix.

Pocokk avatar Pocokk commented on September 16, 2024

Is this still broken?

from comix.

Pocokk avatar Pocokk commented on September 16, 2024

Methods for accessing the quality previously offered through the ComiXology API (and by extension, this tool) are not currently public knowledge, so you would need to figure it out yourself or hope someone shares in the future.

Thank you.

from comix.

neckothy avatar neckothy commented on September 16, 2024

I made a mistake before, as I had not researched the topic well enough at the time. Access to "HD" comics from Kindle is currently semi-easily achievable with publicly available tools.

The latest version of DeDRM_tools along with the KFX Input plugin in calibre should support the recent KFX format(s).

While I'm not sure which specific Kindle platforms are eligible for downloading HD content, it seems a recent version of Android and a high DPI device is enough. This discussion walks through obtaining the keyfile from your (rooted) Android device and importing it into DeDRM_tools.

The process is obviously not as streamlined as this project was, and recently released comics (since the ComiXology API being closed) are likely not offered in the same quality as older ComiXology releases.

I would recommend you create your backups ASAP, and avoid buying digital comics from Amazon as much as possible in the future.

from comix.

noaione avatar noaione commented on September 16, 2024

While I'm not sure which specific Kindle platforms are eligible for downloading HD content, it seems a recent version of Android and a high DPI device is enough.

Anything around 2K screen with 320+ DPI I think should be enough, you can always check if you get double download notification for your older stuff (and an obvious KFX file format)

likely not offered in the same quality as older ComiXology releases.

I can see a more obvious JPEG artifact on the image (reprocessed JPEG...?) but the "quality" param used if imagemagick can be believed seems to be higher.

  • danke-Empire rip: x3713 q92
    • Optimized: ~376mb
  • Post-app merge (or whatever you want to call it): x3713 q96
    • Still in KFX format (DeDRM'd): ~628mb (Oh hey! It's close to the number listed in Amazon)
    • Run through KFX Input to CBZ/ZIP conversion: ~527mb (non-optimized), ~494mb (optimized)

Might edit this comment to show some actual diff.

EDIT: I misread your comment on that lmao, but yeah recent stuff got a downgraded quality (although even before the API got discontinued they already started doing it)

Reposted because reason

from comix.

neckothy avatar neckothy commented on September 16, 2024

I can see a more obvious JPEG artifact on the image (reprocessed JPEG...?) but the "quality" param used if imagemagick can be believed seems to be higher.

Recompressed seems likely if you look at the resources extracted with KFX Input's --unpack. danke also mentioned in a Win-O' post a while back that obtaining the untouched JPEG was no longer possible, which is why they first denoised before resaving as JPEG for a full color comic (paraphrasing).

However, I am too lazy (and likely too dumb) to bother looking through the DeDRM / KFX Input code responsible for creating the full images. It may or may not need to be modified to avoid lossy operations.

from comix.

noaione avatar noaione commented on September 16, 2024

I think changing some line to force saving as PNG would work to at least not get a recompressed JPEG. I look at the code and most of the resaving occurs in kfxlib/resources.py.

Edit: Looked it more, --unpack should not recompressed the image, unpacking as CBZ would.

from comix.

neckothy avatar neckothy commented on September 16, 2024

It might be worth bringing up the suspected recompression with the developer of the KFX Input plugin. They may be interested in fixing it or offering a lossless toggle, if it isn't something complex. I don't have a mobileread account.

from comix.

noaione avatar noaione commented on September 16, 2024

Alright, after more testing this should fix some problems that I ran into:

KFX Input Mod.zip

The only problem is that PNG is saved with JPEG extension but I'm too lazy to fix it

from comix.

neckothy avatar neckothy commented on September 16, 2024

It might be worth bringing up the suspected recompression with the developer of the KFX Input plugin. They may be interested in fixing it or offering a lossless toggle, if it isn't something complex. I don't have a mobileread account.

@j-howell I believe this is you. Apologies for the random mention if this isn't something you're interested in discussing or modifying for your plugin's main release. I couldn't find an easy way to search your thread for previous mentions of the topic, and am not interested in creating a mobileread account for a one-off post.

from comix.

salami-ch avatar salami-ch commented on September 16, 2024

Very interesting! I tried to follow the linked guides, unfortunately I'm still getting the following error:
calibre_plugins.kfx_input.kfxlib.utilities.KFXDRMError: Book container xxx.kfx has DRM and cannot be converted

I have successfully extracted, converted and imported my keyfile and the book shows up in the Calibre library correctly. Does anyone have a clue what I might be missing? Thanks!

from comix.

noaione avatar noaione commented on September 16, 2024

@salami-ch

You would also need to replace one line of code in DeDRM plugin, the author in discussion thread recently provide an edited version if you haven't replaced your version yet.

If you properly setup everything, this is how the imported KFX books looks like:

image

Note: That it said Formats: KFX and not Formats: KFX-ZIP (KFX-ZIP -> Fails to DeDRM)

If you're backing up something that you know is a HD comics (above x2600 resolutions), make sure that the size of the folder is similar to what Amazon advertises in their website (or check if the folder in Android/data/com.amazon.kindle/files/ASIN if the folder size keep growing after you got a finished download notification)

image

To make it easier, you can just ZIP all the files of the KFX(s) folder, name it to BOOK_ASIN.kfx-zip and import it to Calibre to make it more consistent.

image

from comix.

salami-ch avatar salami-ch commented on September 16, 2024

Thanks for the detailed hints @noaione

Looks like I have already correctly performed these steps. My mistake was to specify the source files rather than the resulting kfx in the Calibre library after import. I was assuming that the calibre CLI would imvoke DeDRM on the fly, but apparently this process is only performed during the library import.

I can now successfully convert a recently published comic in its full x3056 resolution. You guys are awsome, thanks a lot!

from comix.

2023comicjtr avatar 2023comicjtr commented on September 16, 2024

It is fantastic to have this back, but does anyone know of any way to get the cover in full 1988x3056 resolution? So far I'm only getting the old Kindle cover of 1249x1920. Even if you unpack the KFX file instead of converting to CBZ, the cover is still the lower resolution version. Apparently the scene groups have figured it out (help?), but they would have no reason to talk to me, so I figured I'd ask here. Thanks!

from comix.

noaione avatar noaione commented on September 16, 2024

I don't think so, the easiest way is to just get the cover from Amazon website and denoise it. The cover is usually around x2560 so it's good enough for most people.

You can use this bookmarklet to easily get the "high quality" cover: https://github.com/rRoler/bookmarklets#amazon

  1. Click Bookmarklet code
  2. Click Raw
  3. Copy the javascript:xxxxxxxxxxxxxxxxxxx line
  4. Create new browser bookmark and paste it to the URL box

You can just then go to the book page in Amazon and click the bookmark

from comix.

2023comicjtr avatar 2023comicjtr commented on September 16, 2024

Thanks! That's super helpful. I do generally grab those covers, but had been going to this site to get them: https://www.ereader-palace.com/get-high-resolution-cover-image-amazon-kindle-ebooks/

The bookmarklet is much more convenient. :-)

from comix.

neckothy avatar neckothy commented on September 16, 2024

If you have a method of finding the "ComiXology ID" of a book, it may have a full resolution cover available at a url like:
https://m.media-amazon.com/images/S/cmx-images-prod/Item/id/id.jpg
or
https://images-(cn|eu|fe|na).ssl-images-amazon.com/images/S/cmx-images-prod/Item/id/id.jpg
e.g. one from a random comic

I haven't looked into methods of finding CmX IDs for recent releases, so I don't know if these are available for books released post API closure. For any old releases you own, I would check for the ID on the Wayback Machine.

edit: Wrote a quick script for testing URLs sequentially and it seems they do exist for books released after the API closure. You'd probably still want to find a more reliable method for getting IDs.

edit again: and they exist for books released after the app merge as well.

edit3: took a few to poke around the Kindle app and Amazon store pages, but didn't see anything obvious that would lead to getting ComiXology book IDs for a specific book. Someone smarter or more interested may be able to find something. I would assume the IDs are listed somewhere (or there's a different method to access the same images using ASIN) if scene releases are using them. At least, I would hope they aren't doing something like downloading ranges of covers and manually matching them. Though, AFAIK, none of them other than danke shared proper Kindle HD books until after tools and information to do so were all shared publicly, so who knows what they're up to.

from comix.

2023comicjtr avatar 2023comicjtr commented on September 16, 2024

If you have a method of finding the "ComiXology ID" of a book...

Sadly, I'm not well-versed enough in Python to come up with a script, and even though brute-forcing a range of values isn't totally off-putting, I don't understand wget/curl enough to come up with a command that allows for both the folder name and the file name increasing incrementally. Thanks for doing so much checking though; much appreciated!

from comix.

neckothy avatar neckothy commented on September 16, 2024

Writing the script shouldn't be difficult if you figure out a method of getting the CmX id for a given book from its store page, ASIN, etc.

Saving sequential covers is easy enough, but it isn't really viable if you don't have a way of matching the ids to the books you're wanting. Too much guesswork and/or manual effort involved, especially if you're looking for covers for older books.

I'm not sure I understand what you're saying about folder names. Here is a very simple Python script that should count up from a starting id and save any valid cover (including placeholder) to the folder you run it in. I wouldn't recommend using it for mass downloading.

from comix.

mslmn avatar mslmn commented on September 16, 2024

Just noticed a book I grabbed came with two folders, ASIN1 and ASIN2_update. Both have .kfx files in them. Any idea how to handle this?

from comix.

2023comicjtr avatar 2023comicjtr commented on September 16, 2024

I had that happen a couple of times. Once I just dumped everything from the update folder into the main one (replacing everything in the main folder with the files from the update folder if they had the same name), and I could then process the main folder. The other time I could just process the update folder on its own (since it had the metadata file and drm-voucher). I think one time nothing worked and I had to redownload.

from comix.

mslmn avatar mslmn commented on September 16, 2024

The update folder has no drm-voucher or metadata file.

Dumping everything from the update folder into the main one and importing into Calibre results in a kfx-zip file which gives the error calibre_plugins.kfx_input.kfxlib.utilities.KFXDRMError: Book container CR!01-long-number.kfx has DRM and cannot be converted when I try to convert it. With other books, I was getting just a kfx file when importing Calibre.

Funnily enough, importing just the main folder works fine.

from comix.

neckothy avatar neckothy commented on September 16, 2024

It might be worth bringing up the suspected recompression with the developer of the KFX Input plugin. They may be interested in fixing it or offering a lossless toggle, if it isn't something complex. I don't have a mobileread account.

The next release of KFX Input should losslessly join tiled resources instead of adding an additional JPEG compression step. Expected around the end of the year. Thanks to whoever took the time to create an account to mention it in the mobileread thread, if you happened to come from here.

from comix.

athrowaway2021 avatar athrowaway2021 commented on September 16, 2024

@neckothy @mslmn @2023comicjtr @noaione

The new unkindle tool should now work to get highest possible quality for both old ComiXology titles and post-merge titles. I've tested it with a few and I'm getting x3056 outputs for Ultimate Spider-Man (2024) for example.

The only major issue now is that the covers are all still x1920 and I haven't seen any way to reliably get ComiXology IDs for books to get the HD covers like mentioned here previously.

from comix.

neckothy avatar neckothy commented on September 16, 2024

Great, thanks for sharing. I made a few attempts to modify it myself but didn't find much success.

The best method I came up with for covers is to create and compare pHash for them, which @noaione made into something a bit more proper. This isn't helpful for developing your tool, but could be a useful additional tool for anyone searching for HD covers until something more streamlined and reliable is found (if ever).

from comix.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.