vigoo/desert

Binary serialization library for Scala

desert - a Scala binary serialization library

See the project's site for documentation and examples.

---

YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler, and YourKit YouMonitor.

desert's Issues

PersistedThrowable gets wrapped again upon reserialization

The Throwable codec keeps wrapping throwables, even the PersistedThrowable to PersistedThrowable. This causes troubles in serialization comparisation when the deserialize and serialize is getting called several times.

Example code:

import io.github.vigoo.desert.zioschema.DerivedBinaryCodec
import zio.schema.{DeriveSchema, Schema}

import scala.util.{Failure, Try}

object Main {
  case class TestClass(testException: Try[Int])

  object TestClass {
    import io.github.vigoo.desert.zioschema.schemas._
    import io.github.vigoo.desert._

    implicit val testClassSchema: Schema[TestClass] = DeriveSchema.gen
    implicit val testClassCodec: BinaryCodec[TestClass] = DerivedBinaryCodec.derive
  }

  def main(args: Array[String]): Unit = {
    import io.github.vigoo.desert._
    val serialized = serializeToArray(TestClass(Failure(new RuntimeException("Test Exception"))))
    val deserialized = deserializeFromArray[TestClass](serialized.toOption.get)
    println("First: " + deserialized)
    val serialized2 = serializeToArray[TestClass](deserialized.toOption.get)
    val deserialized2 = deserializeFromArray[TestClass](serialized2.toOption.get)
    println("Second: " + deserialized2)
  }
}

Result:

First: Right(TestClass(Failure(io.github.vigoo.desert.PersistedThrowable: <java.lang.RuntimeException> Test Exception)))
Second: Right(TestClass(Failure(io.github.vigoo.desert.PersistedThrowable: <io.github.vigoo.desert.PersistedThrowable> <java.lang.RuntimeException> Test Exception)))

UUID codec

String deduplication can break compatibility with data structure evolution

Fix: #28

Switch the core to ZIO Prelude for better performance

Implemented in #34

Waiting for stable zio-prelude release

TransientConstructor

Ability to mark constructors of a sealed trait as transient, indicating that we never intend to serialize it. In this case derive can still derive the sealed trait serializer for the rest of the constructors, and it gives a runtime error when trying to serialize the marked one.

Handle @transient

Handle @transient fields in the GenericBinaryCodec.

• In derivation time collect the field annotations via Annotations in shapeless, zip result with the HList from LabelledGeneric
• Simply skip fields marked as transient in serialization
• Require a Default type class for types to get a default value during deserialization
• Provide Default[Option[T]] by default

ClassCastException at serialization when container type occours recursively in the type hierarchy

Maybe the title a bit misleading, but I can give a concrete example:
Given

  case class Media(url: String, size: Int)
  case class Slide(media: List[Media])
  case class TestClass(uri: String, slides: List[Slide])

When I try to derive a schema and a decoder and use it:

  object TestClass {
    implicit val testClassSchema: Schema[TestClass] = DeriveSchema.gen
    implicit val testClassCodec: BinaryCodec[TestClass] = DerivedBinaryCodec.derive
  }
    val serialized = serializeToArray[TestClass](
      TestClass(
        "test", 
        List(
          Slide(
            List(Media("media1", 500), Media("media2", 700))
          )
        )
      )
    )

It throws the following exception:

Exception in thread "main" java.lang.ClassCastException: class Messages$Media cannot be cast to class Messages$Slide (Messages$Media and Messages$Slide are in unnamed module of loader 'app')
	at io.github.vigoo.desert.zioschema.DerivedBinaryCodec$BinaryCodecDeriver.$anonfun$deriveRecord$5(DerivedBinaryCodec.scala:58)
	at scala.collection.immutable.List.map(List.scala:246)
	at io.github.vigoo.desert.zioschema.DerivedBinaryCodec$BinaryCodecDeriver.$anonfun$deriveRecord$4(DerivedBinaryCodec.scala:55)
	at io.github.vigoo.desert.internal.AdtCodec.serialize(AdtCodec.scala:67)
	at io.github.vigoo.desert.custom.BinarySerializerOps.write(BinarySerializerOps.scala:52)
	at io.github.vigoo.desert.custom.BinarySerializerOps.write$(BinarySerializerOps.scala:51)
	at io.github.vigoo.desert.custom.package$.write(package.scala:3)
	at io.github.vigoo.desert.Codecs$$anon$27.$anonfun$serializeWithKnownSize$1(Codecs.scala:504)
	at io.github.vigoo.desert.Codecs$$anon$27.$anonfun$serializeWithKnownSize$1$adapted(Codecs.scala:503)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
	at zio.Chunk$Arr.foreach(Chunk.scala:1685)
	at io.github.vigoo.desert.Codecs$$anon$27.serializeWithKnownSize(Codecs.scala:503)
	at io.github.vigoo.desert.Codecs$$anon$27.serialize(Codecs.scala:490)
	at io.github.vigoo.desert.Codecs$$anon$27.serialize(Codecs.scala:454)
	at io.github.vigoo.desert.BinarySerializer$$anon$1.serialize(BinaryCodec.scala:13)
	at io.github.vigoo.desert.BinaryCodec$$anon$5.serialize(BinaryCodec.scala:61)
	at io.github.vigoo.desert.internal.AdtCodec.serialize(AdtCodec.scala:302)
	at io.github.vigoo.desert.internal.AdtCodec.$anonfun$serialize$1(AdtCodec.scala:69)
	at io.github.vigoo.desert.internal.AdtCodec.$anonfun$serialize$1$adapted(AdtCodec.scala:68)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at io.github.vigoo.desert.internal.AdtCodec.serialize(AdtCodec.scala:68)
	at io.github.vigoo.desert.custom.BinarySerializerOps.write(BinarySerializerOps.scala:52)
	at io.github.vigoo.desert.custom.BinarySerializerOps.write$(BinarySerializerOps.scala:51)
	at io.github.vigoo.desert.custom.package$.write(package.scala:3)
	at io.github.vigoo.desert.Codecs$$anon$27.$anonfun$serializeWithKnownSize$1(Codecs.scala:504)
	at io.github.vigoo.desert.Codecs$$anon$27.$anonfun$serializeWithKnownSize$1$adapted(Codecs.scala:503)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
	at zio.Chunk$Arr.foreach(Chunk.scala:1685)
	at io.github.vigoo.desert.Codecs$$anon$27.serializeWithKnownSize(Codecs.scala:503)
	at io.github.vigoo.desert.Codecs$$anon$27.serialize(Codecs.scala:490)
	at io.github.vigoo.desert.Codecs$$anon$27.serialize(Codecs.scala:454)
	at io.github.vigoo.desert.BinarySerializer$$anon$1.serialize(BinaryCodec.scala:13)
	at io.github.vigoo.desert.BinaryCodec$$anon$5.serialize(BinaryCodec.scala:61)
	at io.github.vigoo.desert.internal.AdtCodec.serialize(AdtCodec.scala:302)
	at io.github.vigoo.desert.internal.AdtCodec.$anonfun$serialize$1(AdtCodec.scala:69)
	at io.github.vigoo.desert.internal.AdtCodec.$anonfun$serialize$1$adapted(AdtCodec.scala:68)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at io.github.vigoo.desert.internal.AdtCodec.serialize(AdtCodec.scala:68)
	at io.github.vigoo.desert.custom.BinarySerializerOps.write(BinarySerializerOps.scala:52)
	at io.github.vigoo.desert.custom.BinarySerializerOps.write$(BinarySerializerOps.scala:51)
	at io.github.vigoo.desert.custom.package$.write(package.scala:3)
	at io.github.vigoo.desert.BinarySerialization.serialize(BinarySerialization.scala:22)
	at io.github.vigoo.desert.BinarySerialization.serialize$(BinarySerialization.scala:14)
	at io.github.vigoo.desert.package$.serialize(package.scala:3)
	at io.github.vigoo.desert.BinarySerialization.serializeToStream(BinarySerialization.scala:74)
	at io.github.vigoo.desert.BinarySerialization.serializeToStream$(BinarySerialization.scala:68)
	at io.github.vigoo.desert.package$.serializeToStream(package.scala:3)
	at io.github.vigoo.desert.BinarySerialization.serializeToArray(BinarySerialization.scala:111)
	at io.github.vigoo.desert.BinarySerialization.serializeToArray$(BinarySerialization.scala:106)
	at io.github.vigoo.desert.package$.serializeToArray(package.scala:3)
	at Main$.main(Main.scala:24)
	at Main.main(Main.scala)

Helper functions to lift Try to Ser/Deser

It would be useful when wrapping external code in custom serializers.

Try to replace shapeless based derivation with magnolia

This may not be possible, research needed

Lower level codec API and optimizations

There should be a lower level unsafe codec API below the ZPure one, and the library's built-in codecs especially the adt codec should be rewritten to use that to increase performance.

Helper functions to read/write compressed byte array

Wrapping the Java Inflater and Deflater in convenient Ser and Deser functions

Derive must not require implicit codecs for transient field types

Fix website build

Generate nicer byte array visualizations into the documentation

It is probably possible with https://scalameta.org/mdoc/docs/modifiers.html#postmodifier

SerializationFailure for custom serializer errors

SerializationFailure should be similar to DeserializationFailure, a way to report arbitrary errors from custom serializers.

Offset and length support for writeBytes