floydpink/lzwcompress.js

Lossless LZW compression/decompression implemented in JavaScript for strings/JSON/JS objects.

lzwCompress.js

Lossless LZW compression/decompression implemented in JavaScript for strings/JSON/JS objects.

Usage:

Install lzwCompress from npm:

npm install lzwcompress

And then to use it in your node.js applications:

import lzwCompress from 'lzwcompress';

const json = {
  name: 'Mr. JavaScript Kumar',
  age: 42,
  start_date: new Date(),
  address: {
    street: '123 MG Road',
    city: 'Mumbai',
    state: 'Maharashtra',
    country: 'India'
  }
};

// to compress objects
const compressed = lzwCompress.pack(json);

// to uncomress
const original = lzwCompress.unpack(compressed);

console.log(original);

Applies LZW compression and JSON key optimization and makes JSON (or any javascript object) smaller for local storage, to ship up to the server etc.

License

MIT License

Other Libraries

For use in Angular 1.X projects, check out the angular-lzwcompress module

lzwcompress.js's Issues

not working with nodejs Buffer

const obj = { name: "abc" }
const res = lzwcompress.pack(obj)
const buffer = Buffer.from(res)

The value of res is [
123, 34, 95, 95, 107, 34, 58, 91, 34, 110, 97, 109, 101, 34, 93, 44, 257, 95, 118, 261, 256,
48, 261, 34, 97, 98, 99, 34, 125, 125
] while the value of buffer is {
"type": "Buffer",
"data": [
123, 34, 95, 95, 107, 34, 58, 91, 34, 110, 97, 109, 101, 34, 93, 44, 1, 95, 118, 5, 0,
48, 5, 34, 97, 98, 99, 34, 125, 125
]
}.

TypeScript support

Having TypeScript definition file would make it easier and less error-prone to use this library. Please consider for a future version.

Unpacking javascript results in `null` value

This one is a crazy edge case.

Attempting to compress the following results in a null value here for .toString that then throws here

Offending string:

const text = 'foo'.toString();
console.log(text, "const text = 'foo'.toString();");

Is there a size limit to the input?

I just tried calling .pack() on a 5 MB JSON object and the result was a string instead of the array of numbers that I was expecting.

lzwCompress.unpack expected input type

First, I would like to thank you about this library.
I am trying to save some disk space storing a big array of JSON (items here). The problem I am facing probably comes from my understanding of Node.js or the library please bare with me for a moment.

I put comments on code I am trying, first I compress items then store results as string, all works well but the size is not shrinked which is normal, so I tried encoding the array of numbers (result of lzwCompress.pack) to a Uint8Array array before saving it to disk as a binary file.
Now, When I read the file back, I feed it directly to lzwCompress.unpack, which does not work. So my problem probably comes from
encoding before saving
encoding after reading
Or the expected type of array from lzwCompress.unpack

const fs = require('fs');
const lzwCompress = require('lzwcompress');

// to compress objects
const compressed = lzwCompress.pack(items);
// to uncomress
const original = lzwCompress.unpack(compressed);

// This way there is no gain in memory but the contrary
fs.writeFileSync('original.bin', JSON.stringify(items), 'binary');
// because I am saving the array of numbers as string
fs.writeFileSync('compressed1.bin', compressed, 'binary');

// Perhapse I need encoding then,
const _16384 = compressed.length;
const buffer = new Uint8Array(_16384); // create a buffer of some length
for (let index = 0; index < _16384; index++) {
  buffer[index] = compressed[index];
}
// Oh very nice, now I see the file is very very compressed!!!
fs.writeFileSync('compressed2.bin', buffer, 'binary');

// but now I want to get my compressed data back
// from what I read, readFileSync returns a Uint8Array already
const encoded = fs.readFileSync('compressed2.bin', 'binary');
const original2 = lzwCompress.unpack(encoded);                // <-------------- HERE IS MY PROBLEM, NORMALLY  unpack ACCEPTS encoded WITHOUT ANY FURTHER TRANSFORMATIONS, RIGHT ???
console.log(original2);               // <-------------- THIS SHOWS GIBBERISH

Thanks a lot

Bug under at least IE 8 and a potential fix

The problem is in _decode:

            for (var prop in obj) {
              if (!Array.isArray(obj)) {
                if (obj.hasOwnProperty(prop)) {
                  obj[_keys[prop]] = _decode(obj[prop]);
                  delete obj[prop];
                }
              } else {
                obj[prop] = _decode(obj[prop]);
              }
            }

The code iterates over the properties of obj, which it expects to generally have keys which are strings representing a numeric index into _keys, and values which need to be decoded. It deletes these and adds properties with the name from _keys and the decoded value from a recursive call to _decode. In Chrome these newly added properties are not iterated over, but in Internet Explorer 8 (and perhaps others) they are. The proposed fix is to not do anything if the property is not in _keys. This seems to work for at least the case that was breaking me.

            for (var prop in obj) { // IE 8 seems like it might do things in funny order when stuff is deleted?  http://stackoverflow.com/questions/3122548/iterating-javascript-object-properties-and-arrays-with-for-in-when-ordering-is
              if (!Array.isArray(obj)) {
                if (obj.hasOwnProperty(prop)) { // In case iteration follows inheritance chain?  In case it has been deleted?
                  if (_keys[prop]) { // For IE 8 and perhaps other browsers which iterate over the props we've been adding as well as the initial ones
                    obj[_keys[prop]] = _decode(obj[prop]);
                    delete obj[prop];
                  }
                }
              } else {
                obj[prop] = _decode(obj[prop]);
              }
            }

Is there a recommended way to convert the output into a string?

I've noticed that this module returns an array of numbers. Is there a recommended way to convert the output into a string?

I could use .join(',') for example, but I feel that such approach is contrary to the idea of compression. Surely a lot more information could fit in a string if I use all printable ASCII characters instead of only numbers and a comma.

Thanks!!

What is the form of the output?

Hello! I noticed that this module returns an array of numbers.

• What can I expect of the sizes of each number? The largest I got was 303 on a small example, and the smallest was 34. Is there a guaranteed lower bound? And upper bound?

• What can I expect of the length of this array?

Can't handle strings

Numerous test cases fail, either by a hard error, or by returning something that isn't the original string. Simplest failing test case:

var compressed = lzwCompress.pack('""');
var original = lzwCompress.unpack(compressed);

Node Js - lzwCompress.unpack() is taking more time to decompress the json

lzwCompress.unpack() is taking more time to decompress the json

Compress for an URL?

A feature request.

I'd like to compress my JS code state to be stored in an URL.

I would like lzwCompress to use only characters that are allowed in an URL in the result, so it would not need to be encoded.

Is it possible?

Docs: Brotli Doesn't Compress lzw Output Effectively

Was running a quick test of different compression libraries to compress JSON objects. Due to the way that lzwCompress uses numbers to represent the JSON, Brotli can't compress it.

In our case we had a ~500kb JSON object we needed to write to HTML. lzw reduced the size by ~42% but when comparing the lzw page vs the regular page, the lzw page was 20% larger with Brotli compression.

Killer library (really quiet impressed) but was bummed to see it didn't play well with Brotli.

May be worth mentioning in the docs.

Does this package work for tiff files?

I am looking for a tool to compress tiff files and i'm wondering if this works. Sorry if this is a useless issue.