Comments (2)
Hi there,
I can't speak for the team at this point, but I can say it would be a fairly large piece of work to add support for >2GB-size stuff. The ZIP format itself was not initially designed to handle file sizes larger than 4GB, and with parts of the code in Java we opted for a max size of 2GB (the largest value of an integer in Java). Since an individual entry in the zip file can't be bigger than the zip file itself, no effort was put into making the suffix sort code handle anything bigger, either, if I recall correctly (it's been a few years). It's a sanity limit, but it makes sense with the bigger picture of the ZIP specification that was implemented.
The ZIP specifications does have support for large archives (ZIP64) and there is a path forward there - but it's nontrivial (requires quite a lot more work) and it would need to get done before it was really useful to adjust the suffix sort algorithm that is used to generate the diffs. An alternative would be to add proper support for just files up to 4GB in size, sticking with the original ZIP... that might be less problematic than fully supporting the later ZIP specification for large archives, but again you'd have to change a fair amount of code, and if you were going to do that you may as well just go for full ZIP64 support. Here's a great summary from Wikipedia:
The original .ZIP format had a 4 GiB (232 bytes) limit on various things (uncompressed size of a file, compressed size of a file, and total size of the archive), as well as a limit of 65,535 (216) entries in a ZIP archive. In version 4.5 of the specification (which is not the same as v4.5 of any particular tool), PKWARE introduced the "ZIP64" format extensions to get around these limitations, increasing the limits to 16 EiB (2^64 bytes). In essence, it uses a "normal" central directory entry for a file, followed by an optional "zip64" directory entry, which has the larger fields.
For the original use case that Archive Patcher was designed for - reducing application patch size on Android - this just wasn't a problem in any real sense; APK files just aren't that big, as a rule. The extra complexity just wasn't justified.
from archive-patcher.
Thanks for the quick response @andrewhayden! We're building patches for mobile VR games, which tend to have large APKs/OBBs. We ended up finding a solution to our large file problem with https://github.com/sisong/HDiffPatch.
from archive-patcher.
Related Issues (20)
- Compile instructions for bsdiff_jni.cc HOT 8
- Massive performance regression in v2 non-native generator HOT 3
- Truth incompatible with android HOT 4
- Run the demo, it has a NPE, msg is "Inflater has been closed" HOT 19
- Support For iOS HOT 2
- Update README to explain that bsdiff is used without compression
- java.lang.NullPointerException: Inflater has been closed HOT 3
- Is this lib production ready or deprecated? HOT 9
- How to run the sample code? HOT 4
- Very slow to apply patch on Android device HOT 3
- Failed to generate patch from APK: java.util.zip.ZipException: EOCD record not found in last 32k of archive, giving up HOT 4
- Android: java.lang.IllegalStateException: setLevel cannot be called after setInput HOT 3
- How to build code on branch v2 HOT 1
- Please open a security advisory HOT 8
- Android 11 (R) [API 30] : Zlib not compatible on this system HOT 3
- Some zlib versions fail the DefaultDeflateCompatibilityWindow compatability test HOT 2
- Android12 aplier File failed HOT 3
- On Android 12 devices, The file generated with the patch was different from the origin file; HOT 16
- Security Policy violation Binary Artifacts HOT 23
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from archive-patcher.