Comments (21)
I havent really announced this officially yet, but I have been working on a fork of zlib. With the intentions being to get rid of legacy code (Require C99 and drop workarounds for ancient compilers/os), enable new optimizations (such as compiler intrinsics) and make it easier for contributers to contribute code. Currently I have merged in changes from both Intel and Cloudflare, and a couple more. Also lots of legacy code has been removed and some changes have been done to improve readability of the code, and so on.. Read the commit log for (a lot) more details.
Among the main challenges currently are copyright related, as I do not really know what more/less will be required. I have attempted contacting Mark Adler, but he has not answered my email (probably 6 months ago) and the mailing list archive was also down at that time and every time I have checked since.
Anyhow, if anybody would like to review, test and/or contribute to the fork then I would be glad to receive pull requests and/or comments. I have not had much time to work on zlib the last few months due to a lot going on at work, but I hope to get back to doing some more improvements soon.
https://github.com/Dead2/zlib-ng
I do not remember the numbers right now, but compression on x86-64 and armv6 (raspberry pi) was seeing major speed improvements compared to stock zlib (20-50% or so). More testing needs to be done, but it should be pretty stable at this point and it should be a win on all supported platforms in theory.
Sorry for highjacking the thread, but it seems a good idea to allow others to check out my fork.
PS: I do not really have any intentions of replacing zlib, but I hope to provide a good performance-oriented fork with a more community-driven approach. zlib itself will hopefully remain the extremely portable reference implementation.
from zlib.
Ah, ok. I think I know how to fix the deflate bug. It's on my list.
from zlib.
We've added code coverage measurement & reporting on Chrome's testing infrastructure. You can see an example here:
https://chromium-coverage.appspot.com/reports/558266/linux/chromium/src/third_party/zlib/report.html
We likely won't be getting the coverage up to 100% though. There are parts of zlib that Chrome doesn't currently intend to use.
We have also added some fuzz tests. Over time we'll add more, as well as additional unit tests. Zlib already has some unit tests but those aren't currently running on Chrome's testing infrastructure.
from zlib.
There are a handful of zlib alternatives available. Each have their pros and cons.
But so many things rely on zlib itself. I think in an ideal world we'll keep zlib active and healthy.
from zlib.
I think http://www.zlib.net/ contains some pointers on how zlib is maintained and contact information to reach the maintainers.
from zlib.
@Dead2 , this is great. I think this is something we might be interested in and I'll forward the thread along to some of my colleagues who are looking into making the changes.
from zlib.
I support the idea of a group of dedicated people to maintain zlib.
from zlib.
I hope the requirement for your fork would not be 'very strict C99' as that would block usage on most Windows systems. The Visual C++ compiler supports most C99 constructs, but not all. Most C99 based projects that I know require that it also stays compatible with that toolchain.
I would very much welcome new activity in this project... There are many small and larger improvements that could really use some central review. (And there are some bugs in current 'contrib' code that make me recommend against using those optimizations).
The current license of zlib is very liberal, I would hope that it stays that way. Switching to something viral/GPL based would require many current zlib users to stick with the current code, as those license are not compatible with the rest of our code. (In my case the entire code base is Apache 2 licensed, which is incompatible with plain GPL)
from zlib.
rhuijben: I have gone the way of requiring C89 and preferring the use of C99 features where possible (the MS compiler luckily often includes non-standard ways of doing the same or nearly the same thing, requiring only minor workarounds).
Copyright wise my questions are more along the way of whether the header of the files can be modified to remove the copyright with name, since that header really becomes a bit of misinformation when the file has had major modifications. Having a central file of contributor names derived from the git commit log would be my preferred method.
License can likely not be changed from zlib (unless Mark Adler does this himself), but I really do not see any reason to do that anyway. Sure there is a lot of good GPL code that could be merged in, but I see no significant reasons to do so.
madler: That is good news :)
I would volunteer to become part of that group. Possibly you could head such a group and gradually distribute responsibility to the group or something along those lines?
I do not think zlib-ng should be merged into zlib though (just to make my opinion clear), since zlib-ng does cut out lots of legacy support and has some major changes. But that could be done in the future if that is what the community wants.
from zlib.
So what is the state of the zlib project by now?
- There were 33 commits in development branch since April 2013 when v1.2.8 was released - so one commit by month on average.
- There are 50 open pull requests - some with overlapping content. Especially #120 shows how serious the zlib development is considered.
I am just curious if we ever can exepct a new official release?
from zlib.
Most of the fixes in development branch have been feature additions or bug fixes for minor operating systems... I'm assuming we need some major issue to be fixed before it warrants release of v1.2.9.
from zlib.
@Dead2 I'm working on a header-only port of ZLib to modern C++ (no macros, object oriented). It supports raw deflate, and its for my HTTP/WebSocket library:
https://github.com/vinniefalco/Beast/tree/zlib/include/beast/zlib
from zlib.
I think having a group of dedicated people to maintain zlib would be wise.
Zlib is a critically important project. A vast amount of the modern world uses it. But since it isn't a funded project, it cannot have a full-time developer. Rather, it needs to be faithfully maintained in free time, which is limited. More people would mean more total free time.
For what it is worth, @Adenilson and I have been trying to make sure any contributions coming in from Chromium are as easy-to-digest & accept as possible. We know the resources for review are limited and we want to pre-review strictly. I'm also extending this to the rest of Google where possible.
I suggest others who are contributing to zlib do the same.
Try to keep changes within a subdirectory of contrib/ if possible. This might mean copying files into contrib/, making changes to your copy, and providing a CMake or similar build file that uses your copied file instead. Make small, isolated pull requests.
As a community, our limiting resource is review time. The easier it is to understand and accept the pull request, the less of our limited review time it will use. Please treat our limited resource as precious. :)
from zlib.
I second what @ProgramMax said, we should try to help the mainstream project as much as we can.
from zlib.
Integrate codecov and Travis CI, add tests, get coverage up to 100%.
from zlib.
Maybe I am biased, but I think that your collective efforts would be better spent on improvements made to Beast's header-only port of Zlib, since it is now part of Boost (and I intend to write a paper proposing it for the C++ standard). My port is modern and uses C++11 idioms (for example, by replacing macros with lambdas).
from zlib.
The decompressor is broken for windowBits==8 maybe start there
from zlib.
How so?
from zlib.
grr... I always get deflate and decompress confused:
https://github.com/madler/zlib/blob/master/deflate.c#L303
from zlib.
@vinniefalco does it using lambdas somehow improve the generated assembly (or the performance) than using macros?
While I personally like C++11/14, it is important to remember that there are quite a few cases (e.g. linux kernel, ARM embedded devices, etc) where we don't have access to a recent C++ compiler or can't use modern C++ features.
The use of plain C in zlib is a nice feature that improves portability.
from zlib.
it is important to remember that there are quite a few cases...where
Yeah in those cases the "plain C" version might be better suited.
from zlib.
Related Issues (20)
- unzOpen does not open chinese or japanese filenames HOT 1
- Both zlib and minizip are producing the library named "libz.a" and I am confused. HOT 1
- Is it correct that minizip allows duplicate files in a zip? HOT 6
- deflateBound() returns incorrect result for gzip wrapped stream when called after deflate() HOT 2
- Report a potential bug in inflateSync() HOT 1
- compiling zlib-1.2.11 for SunOS 5.11-SPARC 64 bit. HOT 3
- linux - unable to install libzlib=1.3.1 HOT 2
- [BUG] bazel cross-compile protobuf with aarch64-linux-toolchain generating zlib errors. HOT 3
- Files bigger than 4 gb? HOT 1
- Code improvement for iowin32.c HOT 1
- Visual Studio .def file error HOT 1
- Shared library is disabled when undefined versions produce errors HOT 1
- Can we clean clang compiler warning ”reserve-id-macro"?
- zlib-1.3.1, deflate.c, line 497 HOT 2
- What's the error reason about this? HOT 4
- never mind I'll make a post later delete this
- zlib.error: Error -3 while decompressing data: invalid code -- missing end-of-block HOT 2
- Problem with CMake`s check_type_size HOT 1
- Inconsistency with new LIT_MEM (likely bug) HOT 1
- Unable to open a 10gb zip file HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zlib.