Failure to deserialise scala Map when default typing is activated

[alexwei]

Failure to deserialise scala Map when default typing is activated

[pjfanning]

@alexwei jackson-module-scala is an ugly set of hacks around the Java-centric library 'jackson-databind'

With the endless variety of configuration that jackson allows but with Java-centric implementations in jackson-databind, jackson-module-scala simply can't keep up.

Fixing this could take days of fiddly debugging and require changes in both jackson-databind and jackson-module-scala.

My main aim is to get jackson-module-scala to handle the vanilla configuration of ObjectMapper. Once people start tweaking advanced Java-centric configs of jackson-databind - I lose interest. The cost-benefit is heavily skewed against spending time on this.

[pjfanning]

The issue seems to be a serialization issue:

vanilla ObjectMapper writes:

{"a":"abc","b":1,"c":"zyx","d":[0,1,2],"e":{"a":"1","b":"2","c":"3"}}

.activateDefaultTyping(mapper.getPolymorphicTypeValidator, DefaultTyping.NON_FINAL, JsonTypeInfo.As.WRAPPER_ARRAY) causes this output with terrible class choices

["com.fasterxml.jackson.module.scala.ser.DefaultTypingSerializationTest$JacksonTestData",{"a":"abc","b":1,"c":"zyx","d":["scala.collection.immutable.$colon$colon",[0,1,2]],"e":["scala.collection.convert.JavaCollectionWrappers$MapWrapper",{"a":"1","b":"2","c":"3"}]}]

[pjfanning]

I, personally, would never use a Map when dealing when serializing/deserializing data. I would try to use collections that contain strongly typed data - Seq[MyCaseClass] or Seq[(A, B)] or something like that.

Putting class names in JSON output doesn't feel great.

If I had to get something working with JacksonTestData, I would change the Map[String, String] to Seq[String, String] or java.util.Map[String, String]

Or just use a vanilla mapper instance (ie without the DefaultTyping set).

[retronym]

Noting that this used to work in Scala 2.12 through a bit of good fortune.

AFAICT

• jackon-module-scala's MapConverter calls map.asJava to convert (well, wrap) the Scala Map as a Java Map.
• jackson-databind-s MapSerializer.serializeWithType decides what class name to emit in the JSON by calling ClassNameIdResolver.idFromType in a resolver with this._baseType = MapLikeType("scala.collection.immutable.Map", ...)
• In Scala 2.12, the .asJava call gives an inner class scala.collection.convert.Wrappers$MapWrapper. In 2.13 we get an non-inner class (as it enclosed in an scala object, not class) scala.collection.convert.JavaCollectionWrappers$MapWrapper.
• idFromType falls back to using the base type (which is what we're after, I think), in 2.12. In 2.13 it emits the type id of JavaCollectionWrappers$MapWrapper, but this leads to a failure in deserializing.

A workaround would be to replace the .asJava call with an explicit wrapping of the value in a custom class that a) implements java.util.Map, and b) is an inner class. 🤮!

[cowtowncoder]

The big problem really is that Jackson's databind has pretty decent support for Map and Collection implementations, but Scala's counterparts do not implement them. As such there's much less support for "Map-like" and "Collection-like" types so implementation is by definition more involved. Although if existence of these was known at the time original Map/Collection (de)serializers were built they could probably have been implemented in a way to improve reuse.

Now doing 2-phase processing using Java Map/Collection intermediates works ok for most other aspects but NOT for polymorphic typing (esp. via Default Typing) because it by definition is very tightly coupled to implementation types -- and in this case information for Wrapper is used.

I guess I am just repeating much of what @retronym just said. :)

[pjfanning]

I spent some time trying to hack a solution but I just kept hitting issues. I have many higher priority things to work on so I'm not planning to spend much time on this.

• It is very easy to workaround - don't enable default typing when deserializing such JSON
• this is not a new issue and so far only 1 person has raised an issue
• probably the best solution is #470 but that involves forking a 1000+ line Java serializer and adding tonnes of test coverage for that 1000 lines of code to cover all the various code paths through that 1000 lines

[retronym]

Thanks for the notes.

I'm doing maintenance work across a large codebase and luckily there only seems to be one small spot using Default Typing within fairly extensive usage of jackson-scala. It looks like this was done to serialise a Scala sealed hierarchy.

I'm going to use an alternative mechanism for that instead. I believe @ JsonTypeInfo + @ JsonSubtypes should work as per https://stackoverflow.com/questions/18027233/jackson-scala-json-deserialization-to-case-classes .

[pjfanning]

Thanks @retronym - JsonTypeInfo/JsonSubtypes annotations are the best way to do this.

DefaultTyping should be ok if you avoid Scala Maps.

[pjfanning]

@retronym I'm not sure if it's a great idea but I have reflection based libs that extend jackson-module-scala and that auto-discover subtypes for sealed traits. You still need the JsonTypeInfo annotation (but don't need to list the sub types). I was hoping to create a jackson-databind enhancement that would allow you to avoid the JsonTypeInfo annotation too but I haven't gotten around to it.

https://github.com/FasterXML/jackson-module-scala#related-projects (separate projects for Scala 2 and Scala 3 - because the code differs quite a bit depending on which Scala version you are using).

[retronym]

@pjfanning I finally got back to trying to work around this bug. It proved awkward so I took another look at finding a workaround in the library:. Here's my PR: #658