Comments (10)
Thank you for reporting. This is expected and I have following language in README.md:
At this time the package cannot compete with the xz tool regarding compression speed and size.
I haven't found the time so far to work on code optimization. On the plus side there is a lot of potential on improving the situation. Unfortunately I cannot promise when I will work on it.
from xz.
There is work ahead. I left the issue open.
from xz.
I just ran into slow decompression and the (partial) solution is to wrap your reader in bufio.NewReader()
. It turns out this library uses ReadByte()
a great deal and on unbuffered input this is incredibly slow.
I say "partial" as unfortunately this fails on some inputs with
writeMatch: distance out of range
Very weird that it fails when buffered but works when unbuffered..
from xz.
Yes, the library doesn't implement its own buffering and because it uses ReadByte it benefits from buffered readers. I should have documented it.
Rationale at the time has been that I wanted to use a buffered reader only if there is a need for it. For instance I didn't want to use a buffered reader for a bytes.Buffer.
A buffered reader shouldn't make a difference for the reading process. The gxz tool is using a buffered reader and I have run extensive tests for it.
Can you provide the file that you want to decompress?
from xz.
Sure, I was decompressing the Zig tarballs from here.
from xz.
Fixed!
from xz.
I have now downloaded all 0.8.0 files and decompressed it with the gxz tool, which uses bufio.Reader and there were no problems to decompress all of them.
Please provide:
- name of the actual file generating issues
- version of the xz module
- the code you are using to decompress the file
- output of go.env
from xz.
Oh you're asking for the failing one, sorry, that wasn't clear - I thought you were asking for one of the slow ones.
from xz.
This is the one that fails. Interestingly it also fails with github.com/xi2/xz
from xz.
Hi, this a deb file, which is an ar file. You must do the following:
$ ar xv bzip2_1.0.6-9.2_deb10u1_amd64.deb
x - debian-binary
x - control.tar.xz
x - data.tar.xz
The two xz files can easily be uncompressed and generate no issues for me. The debian-binary is a plain-text file. Infos about the deb format can be found by the manual page for deb.
from xz.
Related Issues (20)
- use of internal package github.com/ulikunitz/xz/internal/xlog not allowed HOT 2
- Not clear how to trade most resources for most compression HOT 3
- Checksum None is valid HOT 4
- expose blockreader HOT 4
- Current maturity of project (and other semantics) HOT 3
- Achieving maximum xz compression HOT 2
- memory leak HOT 2
- [SECURITY] Implementation of readUvarint vulnerable to CVE-2020-16845 HOT 17
- How to use multi CPU work for compression? HOT 3
- Panic with invalid input HOT 2
- missing match limit, was "lzip" HOT 11
- How to write compress & decompress data in file? HOT 2
- Don't worry about this lssue HOT 1
- Missing common APIs like Reader:Close() Writer:Flush() HOT 3
- Out of Memory bug when using a large reader HOT 3
- high allocation ratio HOT 3
- Expose `processFile` function HOT 2
- Plan for rewrite branch HOT 4
- How i can compressed folder? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xz.