boljen/snappy-go

Automatically exported from code.google.com/p/snappy-go

snappy-go's Issues

test uses data file that's not in the repo

Can you please include the file needed for testing (snapp_test.go):

        words, err := ioutil.ReadFile("/usr/share/dict/words")

Original issue reported on code.google.com by [email protected] on 14 Nov 2011 at 8:05

Project front page installation instructions are outdated

s/goinstall/go get/

Original issue reported on code.google.com by [email protected] on 17 Jan 2013 at 2:48

Decode() may return nil when output length is 0 (patch attached)

If Decode() is called with a nil destination buffer and the uncompressed length 
is zero, then a nil slice will be returned. This contradicts the docs, which 
say that "a newly allocated slice will be returned" in this case.

Repro: http://play.golang.org/p/NiNH49J4M2 . This doesn't actually run because 
the playground can't import snappy, but you can copy/paste this and run it on 
your machine.

There are two alternatives for fixing this problem:

1. If the current behavior is "as designed", update the docs to say that a nil 
slice will be returned of the decompressed output has length 0 and the dst 
input is nil.
2. Add a nil check and allocate a zero-length byte slice to handle this case.

I've attached patches for each of these alternatives (but don't apply them both 
since the two solutions are mutually exclusive).

Original issue reported on code.google.com by [email protected] on 28 Aug 2014 at 5:25

Attachments:

• alloc_zero_length_return.diff
• doc_as_designed.diff

Decode panics when passed malformed input

What steps will reproduce the problem?

Call snappy.Decode(nil, x) where x has the form 0xff 0xff 0xff 0xff ... 0xff 
0x00. The function call issues a runtime panic.

What is the expected output? What do you see instead?

No panic and a non-nil error returned, "snappy: corrupt input" or something 
reasonable. This is the behavior if the input has fewer than five 0xff bytes.

What version of the product are you using? On what operating system?

Revision 12e4b4183793 (go1.3.1-linux-amd64)

Please provide any additional information below.

There seem to be panics in three different locations, depending on the length 
of the input. I put together a gist showing the lengths at which each different 
panic occurs.

https://gist.github.com/bmatsuo/2cad6e15c1e3a04d9879

I'm not actually positive these inputs are invalid by the spec. I just stumbled 
into them testing behavior of "github.com/mreiferson/go-snappystream" when 
decoding junk input.

Original issue reported on code.google.com by bryan.matsuo on 8 Sep 2014 at 2:11

support API for official snappy framing

I would like to see some functions to facilitate building a framed snappy 
stream. Some functions that would be helpful:

EncodeFrame(dst, src []byte)
StartFrames(dst)
FinishFrames(dst)

Please provide any additional information below.

The official snappy repository has a document describing the framing format for 
snappy (http://code.google.com/p/snappy/source/browse/trunk/framing_format.txt) 

Original issue reported on code.google.com by jehiah on 19 Jul 2012 at 3:58

Why no io.Reader and io.Writer interface for snappy?

Interested to find out why there's no streaming interface for snappy?  Wouldn't 
the protocol support it?  Would be nice to know that data streams could use 
snappy too like the json stuff: `json.NewEncoder(w).Encode(obj)` 

Original issue reported on code.google.com by [email protected] on 25 Jul 2014 at 5:20

Why does Encode return an error?

The signature is:

func Encode(dst, src []byte) ([]byte, error)

The current implementation only ever returns nil for the errors.

Furthermore, how could this function ever return an error? Aren't any possible 
values of dst and src valid?

For comparison, encoding/hex has

func Encode(dst, src []byte) int
func Decode(dst, src []byte) (int, error)

Original issue reported on code.google.com by [email protected] on 1 Oct 2014 at 11:22

Outdated contributing instructions

I believe the contributing instructions in the README are no longer accurate. 
They suggest that users follow Go's practice for contributing to snappy-go, 
citing the following link:

  http://golang.org/doc/contribute.html

But the contributing instructions for Go have changed since the switch to git. 
And, it's no longer clear to me how one would submit a patch to this repo for 
code review. Further, while Go now uses the Gerrit system for code review it 
looks like the older Rietveld system may still be in use here? But I just don't 
know for sure.

Original issue reported on code.google.com by bryan.matsuo on 23 Feb 2015 at 7:29

Encode appears to allocate ~10K more than it needs on a 64K block

I'm sorry if this is flat-out wrong or just too unimportant to fix; I'm new 
here.

Encode allocates MaxEncodedLen(len(src)) bytes for dst. However, the (very 
well-explained) scenario motivating the formula in MaxEncodedLen(srcLen) 
involves an encoder emitting 5-byte-long copy elements each copying four bytes. 
But snappy-go can't encounter that scenario because it emits at most 
3-byte-long copy elements, each copying at least four bytes, regardless of 
block length. 

Given the restrictions on what snappy-go emits, the worst case I can see is a 
3-byte-long copy element that copies four bytes, followed by a 2-byte-long 
literal element with length 61, followed by the literal bytes, repeated for the 
whole block: then it takes 66 bytes to encode every 65 bytes. Add three bytes 
for the length, and 64K encodes to 65536*(66/65)+3=66547 bytes, 10393 fewer 
than the 76940 bytes actually allocated. Maybe someone can find a worse case.

If snappy-go did, in the future, emit 5-byte copy elements, presumably it still 
wouldn't do so in a 64K block, because offsets within a 64K block always fit in 
two bytes. 

(If MaxEncodedLen is meant to represent the worst possible blowup with the 
Snappy encoding, rather than the worst possible blowup with snappy-go or other 
currently-existing encoders, note that the encoding appears to allow 
5-byte-long copy elements to only copy one byte, so the absolute most 
pessimistic upper bound may be len(src)*5 plus space for the preamble.)

Original issue reported on code.google.com by [email protected] on 6 Aug 2014 at 6:18