JNI Problem

[lz4]

JNI Problem

[SaberParker]

Ok Build Succeed...^_^
Just change ${java.home}/../include under the build.xml to your JNI include file directory

[jpountz]

I'm glad it worked, I am not even sure someone else than me ever tried to build this project. :-) Does it work as expected?

[SaberParker]

Works Perfectly Well Done.
I Just Try'n to write some IndexInput And StreamInput for Lucene.
I 'll give you a touch once i done it.

[jpountz]

Oh I'm very curious now! Are you trying to compress stored fields? For ElasticSearch, Solr or maybe a custom Lucene app?

[SaberParker]

Yeah For ElasticSearch can you help me out?
We can share and do it together if you would like ;)

[jpountz]

I have taken a look at Elasticsearch compression a few weeks ago, it should be very easy to add LZ4 compression!

I can't help you until next week, feel free to ask me questions if you need help!

[SaberParker]

Ok i just wonder about the context and IndexInput class of LZ4 compress

[SaberParker]

Thank you + i'll keep in touch if i get to something ;)

[SaberParker]

Well Done.

[jpountz]

Any news?

[SaberParker]

Hi, actually there a problem among JNI Compressor couple tests has been failed. i ve check out the safe-compressor and the results are not quite equal for python. hope you can share me the real compress instructions of LZ4 among you r library or share me some ideas about Compressor for elastic.

[jpountz]

The problem with JNI is that is has a lot of overhead (bad for small inputs) and you need to build the shared library for every existing operating system and architecture. I think it would make more sense to use the Java Unsafe impl.

What do you mean by "the results are not quite equal for python"?

[SaberParker]

Ok, seems that's what was the problem i'm using Mac OSX maybe it's need to be debugged more on OSX i ve tried with safe java to compress some data and check uncompress on python wasn't match...i'll try to use the Unsafe impl. and give you feedback...

[jpountz]

Maybe your python impl encodes the length at the beginning of the compressed stream. Are the last bytes more similar?

[SaberParker]

Yes...it's working i really appreciate that notice the library is quite well also i would to enhance the building of library instructions well paths, JNI and some fix...hope that you can share me a simple implementation of Compressor Component for Elastic Search...Great Job...Well done...

[jpountz]

hope that you can share me a simple implementation of Compressor Component for Elastic Search...

There are already compressor components for other compression algorithms in ElasticSearch, have a look at the org.elasticsearch.common.compress package.