Comments (27)
Is this still broken?
Yes. A "fix" would essentially be an entirely new project since the API this relies on no longer exists.
If you're looking for a public tool to back up your ComiXology and Kindle comic purchases, DeDRM_tools or unkindle.py (included in this repo) are likely your best bets for the foreseeable future. Methods for accessing the quality previously offered through the ComiXology API (and by extension, this tool) are not currently public knowledge, so you would need to figure it out yourself or hope someone shares in the future.
from comix.
Edit: New one: KFX Input Mod.zip since I messed up the ternary order
Quick modification based on v2.6.0, use the calibre-debug CLI:
calibre-debug -r "KFX Input" -- -z -l Book.kfx
(either use --lossless
or -l
, only support CBZ/PDF)
As a side note, this would make the output bigger and slower to process (PNG takes longer to save)
The new Kindle HD didn't looks that much worse with pretty similar JPEG artifacts to CMX HD as far as I can see. Although, if you have a god eyes or something this is 100% not similar quality to CMX HD.
from comix.
The old cmx api is dead atm. No idea what's going to happen next.
from comix.
Is this still broken?
from comix.
Methods for accessing the quality previously offered through the ComiXology API (and by extension, this tool) are not currently public knowledge, so you would need to figure it out yourself or hope someone shares in the future.
Thank you.
from comix.
I made a mistake before, as I had not researched the topic well enough at the time. Access to "HD" comics from Kindle is currently semi-easily achievable with publicly available tools.
The latest version of DeDRM_tools along with the KFX Input plugin in calibre should support the recent KFX format(s).
While I'm not sure which specific Kindle platforms are eligible for downloading HD content, it seems a recent version of Android and a high DPI device is enough. This discussion walks through obtaining the keyfile from your (rooted) Android device and importing it into DeDRM_tools.
The process is obviously not as streamlined as this project was, and recently released comics (since the ComiXology API being closed) are likely not offered in the same quality as older ComiXology releases.
I would recommend you create your backups ASAP, and avoid buying digital comics from Amazon as much as possible in the future.
from comix.
While I'm not sure which specific Kindle platforms are eligible for downloading HD content, it seems a recent version of Android and a high DPI device is enough.
Anything around 2K screen with 320+ DPI I think should be enough, you can always check if you get double download notification for your older stuff (and an obvious KFX file format)
likely not offered in the same quality as older ComiXology releases.
I can see a more obvious JPEG artifact on the image (reprocessed JPEG...?) but the "quality" param used if imagemagick
can be believed seems to be higher.
danke-Empire
rip: x3713 q92- Optimized: ~376mb
- Post-app merge (or whatever you want to call it): x3713 q96
- Still in KFX format (DeDRM'd): ~628mb (Oh hey! It's close to the number listed in Amazon)
- Run through KFX Input to CBZ/ZIP conversion: ~527mb (non-optimized), ~494mb (optimized)
Might edit this comment to show some actual diff.
EDIT: I misread your comment on that lmao, but yeah recent stuff got a downgraded quality (although even before the API got discontinued they already started doing it)
Reposted because reason
from comix.
I can see a more obvious JPEG artifact on the image (reprocessed JPEG...?) but the "quality" param used if
imagemagick
can be believed seems to be higher.
Recompressed seems likely if you look at the resources extracted with KFX Input's --unpack
. danke also mentioned in a Win-O' post a while back that obtaining the untouched JPEG was no longer possible, which is why they first denoised before resaving as JPEG for a full color comic (paraphrasing).
However, I am too lazy (and likely too dumb) to bother looking through the DeDRM / KFX Input code responsible for creating the full images. It may or may not need to be modified to avoid lossy operations.
from comix.
I think changing some line to force saving as PNG would work to at least not get a recompressed JPEG. I look at the code and most of the resaving occurs in kfxlib/resources.py
.
Edit: Looked it more, --unpack
should not recompressed the image, unpacking as CBZ would.
from comix.
It might be worth bringing up the suspected recompression with the developer of the KFX Input plugin. They may be interested in fixing it or offering a lossless toggle, if it isn't something complex. I don't have a mobileread account.
from comix.
Alright, after more testing this should fix some problems that I ran into:
The only problem is that PNG is saved with JPEG extension but I'm too lazy to fix it
from comix.
It might be worth bringing up the suspected recompression with the developer of the KFX Input plugin. They may be interested in fixing it or offering a lossless toggle, if it isn't something complex. I don't have a mobileread account.
@j-howell I believe this is you. Apologies for the random mention if this isn't something you're interested in discussing or modifying for your plugin's main release. I couldn't find an easy way to search your thread for previous mentions of the topic, and am not interested in creating a mobileread account for a one-off post.
from comix.
Very interesting! I tried to follow the linked guides, unfortunately I'm still getting the following error:
calibre_plugins.kfx_input.kfxlib.utilities.KFXDRMError: Book container xxx.kfx has DRM and cannot be converted
I have successfully extracted, converted and imported my keyfile and the book shows up in the Calibre library correctly. Does anyone have a clue what I might be missing? Thanks!
from comix.
You would also need to replace one line of code in DeDRM plugin, the author in discussion thread recently provide an edited version if you haven't replaced your version yet.
If you properly setup everything, this is how the imported KFX books looks like:
Note: That it said Formats: KFX
and not Formats: KFX-ZIP
(KFX-ZIP -> Fails to DeDRM)
If you're backing up something that you know is a HD comics (above x2600 resolutions), make sure that the size of the folder is similar to what Amazon advertises in their website (or check if the folder in Android/data/com.amazon.kindle/files/ASIN
if the folder size keep growing after you got a finished download notification)
To make it easier, you can just ZIP all the files of the KFX(s) folder, name it to BOOK_ASIN.kfx-zip
and import it to Calibre to make it more consistent.
from comix.
Thanks for the detailed hints @noaione
Looks like I have already correctly performed these steps. My mistake was to specify the source files rather than the resulting kfx in the Calibre library after import. I was assuming that the calibre CLI would imvoke DeDRM on the fly, but apparently this process is only performed during the library import.
I can now successfully convert a recently published comic in its full x3056 resolution. You guys are awsome, thanks a lot!
from comix.
It is fantastic to have this back, but does anyone know of any way to get the cover in full 1988x3056 resolution? So far I'm only getting the old Kindle cover of 1249x1920. Even if you unpack the KFX file instead of converting to CBZ, the cover is still the lower resolution version. Apparently the scene groups have figured it out (help?), but they would have no reason to talk to me, so I figured I'd ask here. Thanks!
from comix.
I don't think so, the easiest way is to just get the cover from Amazon website and denoise it. The cover is usually around x2560 so it's good enough for most people.
You can use this bookmarklet to easily get the "high quality" cover: https://github.com/rRoler/bookmarklets#amazon
- Click Bookmarklet code
- Click
Raw
- Copy the
javascript:xxxxxxxxxxxxxxxxxxx
line - Create new browser bookmark and paste it to the URL box
You can just then go to the book page in Amazon and click the bookmark
from comix.
Thanks! That's super helpful. I do generally grab those covers, but had been going to this site to get them: https://www.ereader-palace.com/get-high-resolution-cover-image-amazon-kindle-ebooks/
The bookmarklet is much more convenient. :-)
from comix.
If you have a method of finding the "ComiXology ID" of a book, it may have a full resolution cover available at a url like:
https://m.media-amazon.com/images/S/cmx-images-prod/Item/id/id.jpg
or
https://images-(cn|eu|fe|na).ssl-images-amazon.com/images/S/cmx-images-prod/Item/id/id.jpg
e.g. one from a random comic
I haven't looked into methods of finding CmX IDs for recent releases, so I don't know if these are available for books released post API closure. For any old releases you own, I would check for the ID on the Wayback Machine.
edit: Wrote a quick script for testing URLs sequentially and it seems they do exist for books released after the API closure. You'd probably still want to find a more reliable method for getting IDs.
edit again: and they exist for books released after the app merge as well.
edit3: took a few to poke around the Kindle app and Amazon store pages, but didn't see anything obvious that would lead to getting ComiXology book IDs for a specific book. Someone smarter or more interested may be able to find something. I would assume the IDs are listed somewhere (or there's a different method to access the same images using ASIN) if scene releases are using them. At least, I would hope they aren't doing something like downloading ranges of covers and manually matching them. Though, AFAIK, none of them other than danke shared proper Kindle HD books until after tools and information to do so were all shared publicly, so who knows what they're up to.
from comix.
If you have a method of finding the "ComiXology ID" of a book...
Sadly, I'm not well-versed enough in Python to come up with a script, and even though brute-forcing a range of values isn't totally off-putting, I don't understand wget/curl enough to come up with a command that allows for both the folder name and the file name increasing incrementally. Thanks for doing so much checking though; much appreciated!
from comix.
Writing the script shouldn't be difficult if you figure out a method of getting the CmX id for a given book from its store page, ASIN, etc.
Saving sequential covers is easy enough, but it isn't really viable if you don't have a way of matching the ids to the books you're wanting. Too much guesswork and/or manual effort involved, especially if you're looking for covers for older books.
I'm not sure I understand what you're saying about folder names. Here is a very simple Python script that should count up from a starting id and save any valid cover (including placeholder) to the folder you run it in. I wouldn't recommend using it for mass downloading.
from comix.
Just noticed a book I grabbed came with two folders, ASIN1
and ASIN2_update
. Both have .kfx
files in them. Any idea how to handle this?
from comix.
I had that happen a couple of times. Once I just dumped everything from the update folder into the main one (replacing everything in the main folder with the files from the update folder if they had the same name), and I could then process the main folder. The other time I could just process the update folder on its own (since it had the metadata file and drm-voucher). I think one time nothing worked and I had to redownload.
from comix.
The update folder has no drm-voucher or metadata file.
Dumping everything from the update folder into the main one and importing into Calibre results in a kfx-zip file which gives the error calibre_plugins.kfx_input.kfxlib.utilities.KFXDRMError: Book container CR!01-long-number.kfx has DRM and cannot be converted
when I try to convert it. With other books, I was getting just a kfx file when importing Calibre.
Funnily enough, importing just the main folder works fine.
from comix.
It might be worth bringing up the suspected recompression with the developer of the KFX Input plugin. They may be interested in fixing it or offering a lossless toggle, if it isn't something complex. I don't have a mobileread account.
The next release of KFX Input should losslessly join tiled resources instead of adding an additional JPEG compression step. Expected around the end of the year. Thanks to whoever took the time to create an account to mention it in the mobileread thread, if you happened to come from here.
from comix.
@neckothy @mslmn @2023comicjtr @noaione
The new unkindle tool should now work to get highest possible quality for both old ComiXology titles and post-merge titles. I've tested it with a few and I'm getting x3056 outputs for Ultimate Spider-Man (2024) for example.
The only major issue now is that the covers are all still x1920 and I haven't seen any way to reliably get ComiXology IDs for books to get the HD covers like mentioned here previously.
from comix.
Great, thanks for sharing. I made a few attempts to modify it myself but didn't find much success.
The best method I came up with for covers is to create and compare pHash for them, which @noaione made into something a bit more proper. This isn't helpful for developing your tool, but could be a useful additional tool for anyone searching for HD covers until something more streamlined and reliable is found (if ever).
from comix.
Related Issues (20)
- Not listing new title HOT 3
- Protobuf patched HOT 4
- comix not working for unlimited titles, unkindle seems out of date
- Error when downloading: Bad password for file 'COMIXOLOGY_BOOK_ENCRYPTED_HD_0001.jpg HOT 4
- Doesn't work with Viz Media titles. HOT 2
- .
- PNG output
- Still Getting 2FA error HOT 1
- Bad password for file 'COMIXOLOGY_BOOK_ENCRYPTED_HD_0001.jpg'
- Not listing Portuguese Editions bought from Amazon/Kindle HOT 1
- Does not seem to list new comics
- BadZipFile("File is not a zip file") when trying to download a book HOT 1
- Unknown com.amazon.yjconversion feature: yj_publisher_panels-3
- Script does not handle MFA well HOT 2
- Error when running on Mac HOT 2
- Did they retroactively increase the image quality on all titles? HOT 3
- Cover page always lower quality jpg HOT 1
- Can not authenticate (2FA) HOT 3
- KeyError: 'YJFragmentList get has multiple matches HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comix.