jankotek / elsa Goto Github PK
View Code? Open in Web Editor NEWJava serialization, faster and space efficient version of ObjectOutputStream
License: Apache License 2.0
Java serialization, faster and space efficient version of ObjectOutputStream
License: Apache License 2.0
From email at Kryo mailing list:
My use case would be a large-ish (several GB) in-memory state that needs
to be saved periodically (every five minutes or so). No in-place
updates, new state is generated from scratch (but a large amount of
substructures are supposed to be shared with previous state).
So the object graph has new root on each update and data are immutable. Now it makes sense.
In theory it could be handled the same way as circular references.
Serializer would save its state (IdentityHashMap) between updates.
On next update serializer would just continue with original Map and
replace objects from first version, with their pointers.
There are two problems:
Investigate Objenesis for easy object instantiation https://github.com/easymock/objenesis
Check serialization of Scala collections (maps, lists) is efficient. It should use Elsa graph traversal for entries, rather than fall-back into ObjectOutputStream
for entire collection.
Some well known JVM classes are not handled by Elsa. There are some problems with their deserializaiton, some fields are private, object stack can not be handled... etc
That includes:
Following classes are private, and there is problem with object stack on deserialization
Following singletons are registered in MapDB, but not in plain Elsa
Elsa should also handle Externalizable
from other frameworks, namely:
Elsa uses recursion to dive into object graph. That could generate long call stack and cause StackOverflow exception. Compared to Java serialization Elsa uses less stack frames and behaves better.
It should be possible to eliminate recursion and dive into object graph with simple loop. Some data structure needs to replace call stack (trie?). It should store path traversed on graph dive. Special care needs to be taken to preserve order of objects
Also check if Serializable interface is required right now
MapDB should have some level of compatibility with Kryo 3.0. It should be possible to:
plug kryo serializer into Elsa serialization
Use Elsa as Kryo serializer, so frameworks which already use Kryo could also use Elsa.
Either this will be added into separate project. Or distributed in Elsa Jar and Kryo would become compile time dependency.
Getting java.lang.OutOfMemoryError when initilize on android:
ElsaSerializerBase.Deserializer<Bitmap> deser = new ElsaSerializerBase.Deserializer<Bitmap>() {
@Override
public Bitmap deserialize(DataInput in, ElsaStack objectStack) throws IOException {
int bufferLength = in.readInt();
byte[] byteArray = new byte[bufferLength];
in.readFully(byteArray, 0, bufferLength);
return BitmapFactory.decodeByteArray(byteArray, 0, bufferLength);
}
};
ElsaSerializerBase.Serializer<Bitmap> ser = new ElsaSerializerBase.Serializer<Bitmap>() {
@Override
public void serialize(DataOutput out, Bitmap bitmap, ElsaStack objectStack) throws IOException {
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 80, stream);
byte[] byteArray = stream.toByteArray();
out.writeInt(byteArray.length);
out.write(byteArray);
}
};
serializer = new ElsaMaker()
.registerSerializer(BITMAP_HEADER, Bitmap.class, ser)
.registerDeserializer(BITMAP_HEADER, deser)
.make();
java.lang.OutOfMemoryError: Failed to allocate a 953893208 byte allocation with 6291456 free bytes and 507MB until OOM, max allowed footprint 10774144, growth limit 536870912
I'd like to be able to store an Instant
.
Good developers don't use java.util.Date
anymore.
There may be other types in the package which other people would find useful to serialise as well.
In the Elsa README the Maven dependency XML is as follows:
<dependency>
<groupId>org.mapdb</groupId>
<artifactId>mapdb</artifactId>
<version>VERSION</version>
</dependency>
should the artifactId be 'elsa' instead of 'mapdb'? The paragraph just above the XML says Elsa is no longer included in MapDB.
Support class rename in SerializerPojo
. Unregistered class names are stored in binary data.
ElsaMaker.registerClassRename()
to register alternative class nameSupport class field rename in SerializerPojo
.
ElsaMaker.registerClassFieldRename()
to register aliasgetXXX()
getter in deserialization as alias for non-existing renamed fieldsetXXX()
setter in serialization as alias for non-existing renamed fieldA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.