provide a small core distribution without type-specialization

[leventov]

provide a small core distribution without type-specialization

[peter-lawrey]

Can you give an indication of how much difference this would make to your
business? E.g. It would reduce the download times.
On 18/01/2015 3:12 PM, "Viktor Szathmáry" [email protected] wrote:

Many projects could benefit from drop-in replacement Set/Map
implementations (Object -> Object). Please provide a small distribution of
koloboke (in Maven central), that only contains this core, instead of the
20Mb version with all the primitive specializations

—
Reply to this email directly or view it on GitHub
#33.

[phraktle]

Hi Peter,

I imagine it would make more of a difference for Koloboke as more projects would use it ;) It simply hurts my sense of aesthetics that one would depend on a collections library an order of magnitude larger than the application itself, on the same order as the JDK runtime jar.

Kidding aside, in the cases where a memory-efficient primitive collection is a must have (i.e. the working set does not fit otherwise), one can justify pretty much any size of the distributable. In our use-case it's not a must-have but would provide around a 5% performance improvement, at the cost of tripling the distributable size and increasing deployment/restart times. A minimal edition would make the choice clear and would most likely expand the scope of applicability for Koloboke (e.g. even to mobile apps, though I understand that's not the original design context).

(Hopefully some day built-in specialization will make all this bloat unnecessary: http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html)

[peter-lawrey]

You can configure a build which has exactly the combinations you need.

From our point of view we would have to have a good reason to distribute
all the combinations people might want.
On 18/01/2015 6:00 PM, "Viktor Szathmáry" [email protected] wrote:

Hi Peter,

I imagine it would make more of a difference for Koloboke as more projects
would use it ;) It simply hurts my sense of aesthetics that one would
depend on a collections library an order of magnitude larger than the
application itself, on the same order as the JDK runtime jar.

Kidding aside, in the cases where a memory-efficient primitive collection
is a must have (i.e. the working set does not fit otherwise), one can
justify pretty much any size of the distributable. In our use-case it's not
a must-have but would provide around a 5% performance improvement, at the
cost of tripling the distributable size and increasing deployment/restart
times. A minimal edition would make the choice clear and would most likely
expand the scope of applicability for Koloboke (e.g. even to mobile apps,
though I understand that's not the original design context).

(Hopefully some day built-in specialization will make all this bloat
unnecessary:
http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html)

—
Reply to this email directly or view it on GitHub
#33 (comment).

[fvalente]

Hello

I agree with Viktor; I used the library during testing for my application
and was about to include it as it brought about 10% improvement.
In the end, I decided adding close to 20 MB to my 4MB app was not worth the
performance improvement.
Personally, I only need drop in replacement for HashObjObjMaps.
I "might" consider creating a build later but i would rather have something
that does not require effort in figuring out and maintaining;

Thanks,
Fred

On Sun, Jan 18, 2015 at 7:00 PM, Viktor Szathmáry [email protected]
wrote:

Hi Peter,

I imagine it would make more of a difference for Koloboke as more projects
would use it ;) It simply hurts my sense of aesthetics that one would
depend on a collections library an order of magnitude larger than the
application itself, on the same order as the JDK runtime jar.

Kidding aside, in the cases where a memory-efficient primitive collection
is a must have (i.e. the working set does not fit otherwise), one can
justify pretty much any size of the distributable. In our use-case it's not
a must-have but would provide around a 5% performance improvement, at the
cost of tripling the distributable size and increasing deployment/restart
times. A minimal edition would make the choice clear and would most likely
expand the scope of applicability for Koloboke (e.g. even to mobile apps,
though I understand that's not the original design context).

(Hopefully some day built-in specialization will make all this bloat
unnecessary:
http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html)

—
Reply to this email directly or view it on GitHub
#33 (comment).

[phraktle]

Hi Peter,

The combination I had in mind is fairly universal: it's basically the equivalent of what's in the JDK already: HashObjObjMap and HashObjSet (and supporting classes), but none of the primitive specialization interfaces/implementations. Since people get by reasonably well with using the basic JDK collections too, I'm pretty sure that a minimal package would provide a good entry point for a lot of people to start using Koloboke collections. The upside for you is that the project might get more visibility/traction.

Regards,
Viktor

[leventov]

I really don't know how to better resolve this problem. Is it Ok to ship identical classes in different packages? We can end up shipping 64 separate packages for all combinations, that is too much.

I know 20 Mb is unacceptably too much. Why you don't use ProGuard?

Also there was an idea to move code gen inside the lib and generate needed collections on the fly, ~ the same way as Java-Lang data model classes are generated & cached now.

[phraktle]

Hi @leventov,

I don't propose distributing all permutations either, just a basic JDK-equivalent core (objects only, no primitives). I agree, you do have to draw the line somewhere. I will let other users weigh in on whether this is as universally applicable as I think...

Thanks for the ProGuard tip, that might be applicable (though certainly not as convenient as a simple dependency). Runtime specialization might also be a good approach (in essence, that's what the Valhalla project is also considering).

It would also be helpful if you could expand on how to create such a basic distribution. I found https://github.com/OpenHFT/Koloboke/wiki/Koloboke:-roll-the-collection-implementation-with-features-you-need and https://github.com/OpenHFT/Koloboke/blob/master/lib/impl/build.gradle but it's not quite clear what the appropriate solution is.

Regards,
Viktor

[leventov]

@phraktle to generate subset of impls refer to #26 (comment). For you it should be enough to insert never 'byte|char|short|int|long|float|double' line.

[sirinath]

On the fly generation would also be a since solution.

[leventov]

This is effectively covered by Koloboke Compile.