Comments (2)
The main problem with that file is that libdeflate creates too many blocks and wastes too much space sending Huffman codes. Case in point: if you increase OPTIM_BLOCK_LENGTH in deflate_compress.c from 16384 to 65536 and recompile, then libdeflate gzip -12 compresses enwiki8 to 35,056,842 bytes, beating 7-Zip -mx9.
I am aware that there is work needed to improve the block splitting, especially at high compression modes. Currently I have spent the most effort optimizing the decompressor and the compressor at low to medium compression modes, which is where libdeflate is primarily intended to compete. It should also be understood that if you want the "optimum" gzip compression ratio you should probably use Zopfli, which was specifically designed for that purpose.
from libdeflate.
libdeflate v0.2 gzip -12 compresses enwiki8 to 35,100,553 bytes.
from libdeflate.
Related Issues (20)
- Build type does not default to Release with CMake 3.10 and earlier HOT 5
- Version `1.8` is able to decompress data that version `1.17` cannot, but only on Windows HOT 9
- Comparison to zlib and its forks HOT 2
- POSITION_INDEPENDENT_CODE ON for static compilations HOT 2
- v1.14 - v1.18 deflate compatibility HOT 5
- Decompressing Microsoft DOCX Files in Unreal Engine 5 with Libdeflate HOT 4
- Windows Explorer cannot unpack some files generated with libdeflate (reward offered) HOT 9
- Tests fail on Windows in Release mode with VS platform toolset v141 HOT 1
- Please also generate a static source code archive for release HOT 2
- Libdeflate compressed stream in kunzip fail to decompress HOT 12
- Slow level 0 "compression" HOT 2
- I added stream & multi-thread support for libdeflate HOT 16
- ESA matchfinder HOT 4
- crc32 is twice as fast in zlib-ng HOT 19
- how to change sliding window size at runtime? HOT 1
- arm_acle.h compiler errors when building for iOS with Xcode 15.3 HOT 4
- analyzer warning regression: The left operand of '+' is a garbage value due to array index out of bounds HOT 1
- Unaligned access on PowerPC: forgotten macros in `common_defs.h` HOT 11
- 32-bit build error when using cctools 949.0.1 with assembler that doesn't support AVX instructions HOT 7
- VNNI support seems to require something later than GCC 11.1 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libdeflate.