tarmil/fsharp.systemtextjson

System.Text.Json extensions for F# types

FSharp.SystemTextJson

This library provides support for F# types to System.Text.Json.

It adds support for the following types:

• F# records (including struct records and anonymous records);

• F# discriminated unions (including struct unions), with a variety of representations;

• F# collections: list<'T>, Map<'T>, Set<'T>.

It provides a number of customization options, allowing a wide range of JSON serialization formats.

Documentation

• How to use FSharp.SystemTextJson

• Serialization format

• Customizing the format

FAQ

• Does FSharp.SystemTextJson support alternative formats for unions?

Yes!

• Does FSharp.SystemTextJson support representing 'T option as either just 'T or null (or an absent field)?

Yes! Starting with v0.6, this is the default behavior. To supersede it, use an explicit JsonUnionEncoding that does not include UnwrapOption.

• Does FSharp.SystemTextJson support JsonPropertyNameAttribute and JsonIgnoreAttribute on record fields?

Yes! It also provides a more powerful JsonNameAttribute that supports non-string union tags.

• Does FSharp.SystemTextJson support options such as PropertyNamingPolicy and IgnoreNullValues?

Yes! It also supports naming policy for union tags.

• Can I customize the format for a specific type?

Yes!

• Does FSharp.SystemTextJson allocate memory?

As little as possible, but unfortunately the FSharp.Reflection API requires some allocations. In particular, an array is allocated for as many items as the record fields or union arguments, and structs are boxed. There is work in progress to improve this.

• Are there any benchmarks, e.g. against Newtonsoft.Json?

Yes!

fsharp.systemtextjson's Issues

Add benchmarks

In particular against Newtonsoft.Json.

Omitting option fields in records throws

There is actually a test case for this in FSharp.SystemTextJson/tests/FSharp.SystemTextJson.Tests/Test.Record.fs on row 73 where the issue can be reproduced: let allow omitting fields that are optional ()

The test is apparently not being run at the moment since it is missing the Fact attribute.

> let options = JsonSerializerOptions();;                                               
val options : JsonSerializerOptions

> options.Converters.Add(JsonFSharpConverter());;
val it : unit = ()

>     type SomeSearchApi =
-         {
-             filter: string option
-             limit: int option
-             offset: int option
-         };;
type SomeSearchApi =
  { filter: string option
    limit: int option
    offset: int option }

> let result  = JsonSerializer.Deserialize<SomeSearchApi>("""{}""", options);;
System.Text.Json.JsonException: Missing field for record type FSI_0042+SomeSearchApi
   at System.Text.Json.Serialization.JsonRecordConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in C:\projects\fsharp-systemtextjson\src\FSharp.SystemTextJson\Record.fs:line 105
   at System.Text.Json.JsonPropertyInfoNotNullable`4.OnRead(ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.JsonPropertyInfo.Read(JsonTokenType tokenType, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
   at System.Text.Json.JsonSerializer.ReadCore(Type returnType, JsonSerializerOptions options, Utf8JsonReader& reader)
   at System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at <StartupCode$FSI_0043>.$FSI_0043.main@()```

How is this library versioned in v0.x?

My JSON:API framework Felicity has a dependency on FSharp.SystemTextJson.

Currently I am pinning the exact version of FSharp.SystemTextJson, including the patch version. But if you know that during 0.x you will always bump the minor version (not just the patch version) on binary breaking changes, I can relax this constraint to support any patch version and just lock the minor version.

Of course, it has only rarely happened that FSharp.SystemTextJson has had multiple patch releases for a given minor release, so the issue may be a bit theoretical, but I'm asking anyway. :)

Unwrapping single-field single-case discriminated union.

Hello, thansk for the awsome library.

Is it possible to just serialzie the value of the field in case of single case du?

type Foo = Foo of value:int

So {| Prop1 = Foo 1; Prop2 = Foo 2 |} is serialized as { prop1: 1, prop2: 2 }

Prevent nulls in non-option fields?

Is there a way to prevent System.Text.Json from inserting nulls at every opportunity?

It seems like whenever a value is missing in the incoming json (either the field is missing, or explicititly a null from the client), it will just fill in a null and be happy. This leads to disastrous NullReferenceException: Object reference not set to an instance of an object. in our F# code since F# doesn't expect nulls at every corner.

The only time I want to allow a null or a missing field is if the field is of the (F#) option type.

Any advice?

Add support for DictionaryKeyPolicy for Map<string, 'T>

JsonUnionEncoding.FSharpLuLike mismatch with fsharplu for single field union cases

There's a mismatch on the encoding for union cases with a single field when using JsonUnionEncoding.FSharpLuLike. In cases where field count = 1 FSharpLu will just serialize the object instead an array with the one object.

Example for type SingleFieldCase = Single of string

Current FSharpLuLike encoding: {"Single":["hi"]}

Actual FSharpLu encoding: { "Single": "hi" }

I was not sure what the best way to approach this would be wrt affecting other encoding options. Happy to help implement any acceptable changes as needed.

Add support for different encodings for unions

The current encoding for a value such as:

type Foo =
    | Foo of x: int * y: string
    | Bar

is what we can call Newtonsoft-like:

{ "Case": "Foo",
  "Fields": [ 123, "Hello world!" ] }

{ "Case: "Bar" }

I propose we add encoding formats like so:

type UnionEncoding =
  | AdjacentTag = 0x00_01 // 2-valued object: tag and fields (Newtonsoft-like)
  | ExternalTag = 0x00_02 // 1-valued object: tag is the field name
  | InternalTag = 0x00_04 // (n+1)-valued object or array: tag is inlined with the fields
  | Untagged    = 0x01_08 // n-valued object: tag is not stored 
  | NamedFields = 0x01_00 // whether fields are unnamed in an array or named in an object

This would give the following formats:

• AdjacentTag: Newtonsoft-like.

  { "Case": "Foo",
    "Fields": [123, "Hello world!"] }
  
  { "Case": "Bar" }

• AdjacentTag ||| NamedFields:

  { "Case": "Foo",
    "Fields": { "x": 123, "y": "hello" } }
  
  { "Case": "Bar" }

• ExternalTag:

  { "Foo": [ 123, "hello" ] }
  
  { "Bar": [] }

• ExternalTag ||| NamedFields:

  { "Foo": { "x": 123, "y": "hello" } }
  
  { "Bar": {} }

• InternalTag: Thoth-like.

  [ "Foo", 123, "hello" ]
  
  [ "Bar" ]

• InternalTag ||| NamedFields: WebSharper-like.

  { "Case": "Foo",
    "x": 123,
    "y": "hello" }
  
  { "Case": "Bar" }

• Untagged: includes the NamedFields bit, since Untagged with unnamed fields doesn't really make sense. Deserialization is only possible if all cases have differently named fields and there's only one nullary case.

  { "x": 123, "y": "hello" }
  
  {}

The naming is shamelessly stolen from Rust's serde.

Note that when a field doesn't have an explicit name, F# automatically assigns the name "Item" if there is only one field, or "Item1", "Item2", etc if there are several fields.

The question of customizing the names of the fields "Case" and "Fields" should be resolved separately.

Spelling of JsonUnionEncoding.SuccintOption

Is there a typo in the JsonUnionEncoding.SuccintOption flag, should it be JsonUnionEncoding.SuccinctOption ?

Unwrapping single-field record cases

I'm looking to 'unwrap' a record so that it's properties are at the same level as the case field.

Is there some kinda combination I could set for myOptions which would give me the behaviour I'm testing for here?

type D =
  | Da
  | Db of int
  | Dc of {| x: string; y: bool |}
  | Dd of Dd
and Dd = { x: string; y: bool }

let myOptions = JsonSerializerOptions()
myOptions.Converters.Add(JsonFSharpConverter(JsonUnionEncoding.InternalTag ||| JsonUnionEncoding.NamedFields ||| JsonUnionEncoding.UnwrapSingleFieldCases))

[<Fact>]
let ``deserialize unwrapped single-field record cases`` () =
    Assert.Equal(Da, JsonSerializer.Deserialize("""{"Case":"Da"}""", myOptions))
    Assert.Equal(Db 32, JsonSerializer.Deserialize("""{"Case":"Db","Item":32}""", myOptions))
    Assert.Equal(Dc {| x = "test"; y = true |}, JsonSerializer.Deserialize("""{"Case":"Dc","x":"test","y":true}""", myOptions))
    Assert.Equal(Dd { x = "test"; y = true }, JsonSerializer.Deserialize("""{"Case":"Dd","x":"test","y":true}""", myOptions))

[<Fact>]
let ``serialize unwrapped single-field record cases`` () =
    Assert.Equal("""{"Case":"Da"}""", JsonSerializer.Serialize(Da, myOptions))
    Assert.Equal("""{"Case":"Db","Item":32}""", JsonSerializer.Serialize(Db 32, myOptions))
    Assert.Equal("""{"Case":"Dc","x":"test","y":true}""", JsonSerializer.Serialize(Dc {| x = "test"; y = true |}, myOptions))
    Assert.Equal("""{"Case":"Dd","x":"test","y":true}""", JsonSerializer.Serialize(Dd { x = "test"; y = true }, myOptions))

Using FSharp.SystemTextJson in F# Interactive + Visual Studio 2019: error FS1133: No constructors are available for the type 'JsonFSharpConverter'

Hi,

I'm having trouble trying to get FSharp.SystemTextJson working in FSI scripts in VS2019. I don't seem to have the same issue in compiled code. I'm doing a lot of exploratory coding, hence the need to use FSI.

I'd appreciate any suggestions for getting FSharp.SystemTextJson working in FSI - e.g. configuring paket, what libraries+paths to #r at the top of the script, etc.

When I try to run the first lot of example code in FSI, e.g.:

    let options = JsonSerializerOptions()
    options.Converters.Add(JsonFSharpConverter())
    let jsonString = JsonSerializer.Serialize({| x = "Hello"; y = "world!" |}, options)

I get the error message:

error FS1133: No constructors are available for the type 'JsonFSharpConverter'

The example code compiles and works OK if I put it in a .fs source file, build and run the project.

My setup:

VS Tools Options, F# Interactive:

• 64-bit F# Interactive: True
• F# Interactive options: --optimize- --debug+ --targetprofile:netstandard -d:DEBUG -d:TRACE
• Shadow copy assemblies: True

I'm using Paket:

paket.dependencies:

    source https://api.nuget.org/v3/index.json
    storage: none
    framework: auto-detect

    nuget Newtonsoft.Json
    nuget System.Text.Json
    nuget FSharp.SystemTextJson
    nuget FsToolkit.ErrorHandling
    nuget FSharp.Data
    nuget FSharpPlus 

(I need Newtonsoft.Json for some other libraries)

paket.references:

    System.Text.Json restriction: == netstandard2.0
    FSharp.SystemTextJson
    FsToolkit.ErrorHandling
    FSharp.Data
    FSharpPlus

The full FSI script:

    #r "netstandard.dll"
    #load "../../.paket/load/netstandard2.1/main.group.fsx"
    open System.Text.Json
    open System.Text.Json.Serialization

    let test =
        let options = JsonSerializerOptions()
        options.Converters.Add(JsonFSharpConverter())
        let jsonString = JsonSerializer.Serialize({| x = "Hello"; y = "world!" |}, options)
        printfn "%s" jsonString
       0 
    test

Any suggestions?

Thanks,

Tim.

Allow omitting fields that are option(al)

It should not be mandatory to include a (record) field that is optional. If an optional field is missing, it should just be treated as None imo.

SuccinctOption does not support ValueOption type

Succinct option does not support ValueOption while I would expect it to be treated just like Option

Ex: ValueSome "hi" came out as {"Case":"ValueSome","Fields":["hi"]} with default settings.

I was able to make a quick commit following the work done for option. Will send a PR shortly in case it's good enough. Thanks.

Treat single-case union as the value

type PersonId = PersonId of int
let person = {| Name = "Tarmil"; PersonId = PersonId 123 |}

I would expect person to be formatted as

{name: "Tarmil", personId: 123} in the example above, not as:
{name: "Tarmil", personId: {Item: 123}} which is the closest thing I've found using options:JsonUnionEncoding.BareFieldlessTags ||| JsonUnionEncoding.Untagged.

Is it possible to add a super simple formatter specifically for Single-Case DUs? They're a really common pattern in F#, and we're using them to wrap basically all our primitives to add semantic meaning to them.

E.g. Map<PersonId, ProfileInfo> makes a lot more sense than Map<string, ProfileInfo> in which case you have to remember that the string refers to a person id. Single-Case DUs makes everything very easy to comprehend 😄

Maybe there already is support for it?

Change map behavior for a particular key types

If I have a map with a single-case DU as a key that wraps a GUID I would also want it to be serialized as an object.
It would be nice to be able to configure that.

Consider caching the result of `FSharpType.IsRecord` and `FSharpType.IsUnion`

Per the implementation here and here, these functions have to use reflection to get type custom attributes, deconstruct them, and do some additional processing. Would it make sense to build a dict of hash(type.FullName) -> bool and check that before delegating to the underlying function for the IsRecord/IsUnion checks that are done as part of CanConvert for the serializers?

Prevent nulls inside array/lists

You should not accept ["hello", null, "world"] unless the type is string option list. If the type is a string list with nulls inside, it should fail.

This has led to nasty NullReferenceExceptions in our backend because the code downstream expected a list of 'a, but the list turned out to contain some nulls.

I'm currently trying to fix it, but I'm unable to use the isNullableFieldType (from Helpers.fs) because the fsOptions: JsonFSharpOptions isn't avaiable in the type JsonListConverter<'T>() etc classes.

Treat single-case unions like the underlying type in Map keys

Currently Map<string, V> is serialized as a plain JSON object. However, Map<K, V> where K is a single-case union containing a string is still treated like an arbitrary key type, and serialized as an array of [key, value] pairs. It should be serialized as an object instead.

type UserId = UserId of string

JsonSerializer.Serialize(Map [(UserId "Jane", 123); (UserId "John", 456)], options)
// Currently: [["Jane",123],["John",456]]
// Expected: {"Jane":123,"John":456}

Support OpenApi Schema generation

It would be nice if we could add OpenApi 3.0 schema generation. So F# types could directly be exposed in REST models. Its possible to override schema using SchemaFilter in Swashbuckle.

JsonException path, position, error message etc. incorrect for records

FSharp.SystemTextJson completely messes up the path and error messages when deserialization fails.

This is causing me some consternation, because I am creating an F# framework for JSON:API (using an immutable record-based document model) and want to return helpful deserialization messages to API clients, but that's not currently possible.

Repro:

DeserializationErrorTest.zip

open System.Text.Json
open System.Text.Json.Serialization


type ARecord = {
  X: int
}

type BRecord = {
  A: ARecord
}

type CRecord = {
  B: BRecord
}


type A () =
  member val X: int = 0 with get, set

type B () =
  member val A: A = Unchecked.defaultof<A> with get, set

type C () =
  member val B: B = Unchecked.defaultof<B> with get, set


[<EntryPoint>]
let main argv =

  let opts = JsonSerializerOptions()
  opts.Converters.Add(JsonFSharpConverter())
  let json = """{"B":{"A":{"X":"invalid"}}}"""

  // Correct path/message
  printfn "Deserializing normal classes\n"
  try JsonSerializer.Deserialize<C>(json, opts) |> ignore
  with ex -> printfn "%A" ex

  // Incorrect path/message
  printfn "\n\n\nDeserializing records\n"
  try JsonSerializer.Deserialize<CRecord>(json, opts) |> ignore
  with ex -> printfn "%A" ex

  0

Expected outcome:

Deserializing normal classes

System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $.B.A.X | LineNumber: 0 | BytePositionInLine: 24.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'String' as a number.
   <stack trace omitted>



Deserializing records

System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $.B.A.X | LineNumber: 0 | BytePositionInLine: 24.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'String' as a number.
   <stack trace omitted>

Actual outcome:

Expected outcome:

Deserializing normal classes

System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $.B.A.X | LineNumber: 0 | BytePositionInLine: 24.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'String' as a number.
   <stack trace omitted>



Deserializing records

System.Text.Json.JsonException: The JSON value could not be converted to System.Int32. Path: $ | LineNumber: 0 | BytePositionInLine: 9. Path: $ | LineNumber: 0 | BytePositionInLine: 5. Path: $ | LineNumber: 0 | BytePositionInLine: 5. Path: $ | LineNumber: 0 | BytePositionInLine: 5.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'String' as a number.
   <stack trace omitted>

Add support for JsonPropertyNameAttribute

On record fields, obviously.

Maybe also on union cases to change the name used in "Case": "Xyz"?

Update ReadMe with the new DU serialization options like UnwrapRecordCases

Option to use lowercase in union case tag name

Would be nice if:

type Color = Red | Blue | Green
serialize {| color = Red |}

Would serialize as {"color": "red"} instead of {"color": "Red"}.

.NET 5 nullref exception

Upon upgrading to .NET 5 I suddenly start getting a nullref exception that I didn't get before. Turns out one of my records was expecting an int for which the deserializing json didn't contain the corresponding member. After changing from int to int option, the nullref went away.

The exception thrown by this scenario needs to be refined so it's meaningful to the caller, otherwise the current behavior seems ok.

Union deserialization doesn't support reordering "Case" and "Fields"

Given the following type definition:

type Foo = Foo of int

the following JSON fails to deserialize:

{ "Fields": [123], "Case": "Foo" }

because the deserializer expects "Case" first. We currently only use the forward parsing API, so we need to read the case first to know how to deserialize the fields. Accepting the fields first requires using the AST-based parsing API, which is potentially less efficient; but we can do it without changing the way we handle the case first.

Add support for PropertyNameCaseInsensitive

Add support for PropertyNamingPolicy

Don't use case/fields with option

It seems that option-wrapped types are serialized just like any other union, e.g. serializing Some 2 gives {"Case":"Some","Fields":[2]}. It would be better if option was simply unwrapped entirely, i.e. Some 2 serializes to 2. See e.g. FSharpLu.Json.

Encode F# collections

We should encode F# collections in a way that makes sense.

• list<'T> as a JSON array.

• Set<'T> as a JSON array.

• Map<string, 'V> as a JSON object.

• Map<'K, 'V> when 'K <> string as an array of 2-valued arrays (key-value pairs).

Allow distinguishing JsonUnionEncoding.Untagged cases with some (but not total) overlap in field names

Use case: I want to be able to serialize and deserialize the following union case:

[<JsonUnionEncoding.Untagged>]
type ListOperation =
    | AddOp of list: string * add: string list
    | RemoveOp of list: string * remove: string list

into JSON that looks like this:

{ list: "numbers", add: ["one", "two"] }
-or-
{ list: "numbers", remove: ["one"] }

Serialization looks like I'd expect it to look, but deserializing produces the error "Union ListOperation can't be deserialized as Untagged because it has duplicate field names across unions". The list name is shared between union cases, but the names add and remove are distinct, and should (I thought) be enough to distinguish between those two cases.

Is this a known limitation, or is what I'm trying to do something that should be possible?

P.S. That's a simplified representation of what I'm trying to do; my actual data structure is more complex, involving several other fields, but this is the minimal reproducible example.

Add support for JsonIgnoreAttribute

Allow specializing options in the JsonFSharpConverter

Allow passing an IDictionary<Type, JsonFSharpOptions> to customize how specific types are serialized. This would be equivalent to putting JsonFSharpConverterAttribute on the given type, except that:

• It doesn't require modifying the type.
• It can apply customizations that can't be passed to the attribute, such as unionTagNamingPolicy.

Options must be overridable by JsonFSharpConverterAttribute

Currently, if JsonFSharpConverter is added to the converters, it will always take precedence over a type's JsonFSharpConverterAttribute. It should be possible to invert this, so that a specific type can override options such as its unionTagName and unionFieldsName.

When DU cases are external, their naming should conform to the policy specified in the main options.

First, this is a great library! For me, it was the thing, that finally gave me the opportunity to leave newtonsoft.json and switch to system.text.json.
And then, the title basically says it all: when DU cases are external, their naming should conform to the policy specified in the main options. As they are represented as property names, all property names should conform to the policy that comes from the main options (not from the converter one). Also, when there are no fields, the case name is just a string, so it should not be left unmodified, because it's not a property, it's just a string.

For example, say you have a discriminated union like this:

type State =
| Pending
| Available of int

let state1 = Pending
let state2 = Available 10

and you use a camel-case naming strategy in the main options, also external tags, and unwrap fieldless tags in the converter options. Then the correct output for state1 and state2 should be: "Pending" and { "available": 10 } respectively.

How would I serialize read-only property of a record?

I have a type like this

type Mesago =
    { Dialogo : DialogoID
      From : UzantoID
      Content : MesagoContent
      SentAt : DateTime }
    with
        [<JsonPropertyName("id")>]
        member mesago.ID =
            { SentAt = mesago.SentAt
              From = mesago.From }
        [<JsonIgnore>]
        member mesago.IDValue =
            mesago.ID |> MesagoID.value

How can I serialize ID property along with others?

UnwrapOption and nullable-DateTimes

Thanks for the library! It's been very useful so far.

I think I'm just having a misunderstanding of how to configure the options properly. So if there's a better place to ask support-like questions, let me know.

I receive a response from another service which I then deserialize, examine it, and then serialize it back as a record. It looks like this:

{
    "deletedOn": "2020-06-27T16:47:18.660832Z"
}

This value can also be null, so I represent this as a DateTime option in the object I deserialze it to. When I serialize the record, I end up with a result like this:

{
    "deletedOn": {
        "value": "2020-06-27T16:47:18.660832Z"
    }
}

Is there a special setup I need to apply to make it so the result is "deletedOn": "2020-06-27T16:47:18.660832Z" instead? I thought that perhaps because DateTime is a value type it may need to be a voption<DateTime> but that did not seem to resolve it. So far I've tried to explicitly tell the converter to UnwrapOption types, as well as adding the JsonUnwrapValueOptionConverter explicitly to the options. Neither of those have worked.

Target netstandard2.0

There's no reason to only support netcoreapp.

Changelog for v0.14?

I don't see 0.14 in the changelog; could it be added?

Add support for JsonPropertyName on union cases

For example:

type Foo =
    | [<JsonPropertyName "myfoo">] Foo of int

Foo(1) // --> { "Case": "myfoo", "Fields": [1] }

Add support for IgnoreNullValues

For record types add support of default values for missing fileds

Hi. Thank you for your great library! It proved to be a neat thing for union types for me :)
Still, unfortunately, I've learned that it doesn't suit me in one of my scenarios.
Long story short I have the same model for both of my POST and GET endpoints.

[<JsonFSharpConverter>]
type SomeModel = {
    id: string
    name: string
    //more fields
}

When I perform POST (read: create my model) obviously I don't have id on client-side and I generate it on the server. Naturally, for GET request I already have id present. Also, I want to have the same model for both GET and POST as this is how things are in REST endpoints.
However, as you might have already guessed, when I try to deserialize my model with no id provided it fails. Also, I've studied the code a little bit and I can see that it was intended to be this way.
So my question is why did you decide to come with such a decision and is it possible to revert it?

I hope you're having a good day :)
Regards,
Bohdan

More consistent naming of options

Right now we have union encoding options called BareFieldlessTags, SuccintOption, EraseSingleCaseUnions. Even ignoring the spelling mistake in Succinct (#34), these are 3 different verbs/adjectives for options that do something similar: make the JSON less verbose by removing a layer of wrapping. So I propose that we make this more consistent by using the same word in all these options, as well as the new one introduced by #32. I suggest the verb Unwrap: UnwrapFieldlessTags, UnwrapOption and UnwrapSingleCaseUnions.

We can keep the existing names alongside the new ones and put Obsolete attributes on them. Unfortunately this attribute is currently ignored by the F# compiler (see dotnet/fsharp#6628), but I think it's still fine.

"read too much or not enough" when parsing list of DU-elements

System.Text.Json.JsonException: The converter 'System.Text.Json.Serialization.JsonUnionConverter`1[KT.Takst.Domain+ForholdVedBygg]' read too much or not enough. Path: $[1] | LineNumber: 0 | BytePositionInLine: 24.

using these options:

    let serializerOptions =
        let rules =
            JsonUnionEncoding.InternalTag ||| JsonUnionEncoding.EraseSingleCaseUnions ||| JsonUnionEncoding.SuccintOption
            ||| JsonUnionEncoding.BareFieldlessTags

        let serializerOptions =
            JsonSerializerOptions(PropertyNamingPolicy = JsonNamingPolicy.CamelCase, DictionaryKeyPolicy = JsonNamingPolicy.CamelCase)
        serializerOptions.Converters.Add(JsonFSharpConverter(rules))

Given example:

type Color = Red | Blue | Green

JsonSerializer.Deserialize<Color list>("""[ "Red", "Blue"] """, serializerOptions)

Add NamingPolicy for union case tags

FSharp.SystemTextJson & trimming

Hello Loïc,

I'm trying to trim some projects on .net 5 with those parameters:

dotnet publish -c $(config) -r linux-x64 -p:PublishTrimmed=true -p:TrimMode=Link -p:PublishSingleFile=true -p:DebugType=embedded <proj>

and I'm getting this error on publish:

  Mono.Linker.LinkerFatalErrorException: /home/runner/work/FSharp.SystemTextJson/FSharp.SystemTextJson/src/FSharp.SystemTextJson/Collection.fs(15,9): error IL1005: System.Text.Json.Serialization.JsonListConverter<T>.Read(Utf8JsonReader&,Type,JsonSerializerOptions): Error processing method 'System.Text.Json.Serialization.JsonListConverter<T>.Read(Utf8JsonReader&,Type,JsonSerializerOptions)' in assembly 'FSharp.SystemTextJson.dll'
   ---> System.InvalidOperationException: Operation is not valid due to the current state of the object.
     at Mono.Linker.Dataflow.ReflectionMethodBodyScanner.GetValueNodeFromGenericArgument(TypeReference genericArgument)
     at Mono.Linker.Dataflow.ReflectionMethodBodyScanner.ProcessGenericArgumentDataFlow(GenericParameter genericParameter, TypeReference genericArgument, IMemberDefinition source)
     at Mono.Linker.Steps.MarkStep.MarkGenericArgumentConstructors(IGenericInstance instance, IMemberDefinition sourceLocationMember)
     at Mono.Linker.Steps.MarkStep.MarkGenericArguments(IGenericInstance instance, IMemberDefinition sourceLocationMember)
     at Mono.Linker.Steps.MarkStep.GetOriginalMethod(MethodReference method, DependencyInfo reason, IMemberDefinition sourceLocationMember)
     at Mono.Linker.Steps.MarkStep.MarkMethod(MethodReference reference, DependencyInfo reason, IMemberDefinition sourceLocationMember)
     at Mono.Linker.Steps.MarkStep.MarkInstruction(Instruction instruction, MethodDefinition method, Boolean& requiresReflectionMethodBodyScanner)
     at Mono.Linker.Steps.MarkStep.MarkMethodBody(MethodBody body)
     at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method, DependencyInfo& reason)
     at Mono.Linker.Steps.MarkStep.ProcessQueue()
     --- End of inner exception stack trace ---
     at Mono.Linker.Steps.MarkStep.ProcessQueue()
     at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue()
     at Mono.Linker.Steps.MarkStep.Process()
     at Mono.Linker.Steps.MarkStep.Process(LinkContext context)
     at Mono.Linker.Pipeline.Process(LinkContext context)
     at Mono.Linker.Driver.Run(ILogger customLogger)
  Optimizing assemblies for size, which may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
/usr/local/share/dotnet/sdk/5.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.ILLink.targets(41,5): error NETSDK1144: Optimizing assemblies for size failed. Optimization can be disabled by setting the PublishTrimmed property to false.

The trigger is the PublishTrimmed=true. This is no a bug per se of FSharp.SystemTextJson but a bug of the linker.

Just for information, it looks like there is a workaround meanwhile:
dotnet/runtime#43222 (comment)

Maybe this coud be ok to implement this ? It will try this on my side and let you know.

Map<string, _> where the string-key has a unit of measure doesn't (de)serialize correctly

We use FSharp.UMX to add some type-safety/specificity to our primitive types – like string.

[<Measure>] type userId

let usersAge: Map<string<userId>, int> = Map [%"user1", 30; %"user2", 40]

However it doesn't serialize as I'd hope: {"user1": 30, "user2": 40}, but instead as [["user1", 30], "user2", 40]

I've looked at the code here:

    static member internal CreateConverter(typeToConvert: Type) =
        let genArgs = typeToConvert.GetGenericArguments()
        let ty =
            if genArgs.[0] = typeof<string> then
                typedefof<JsonStringMapConverter<_>>
                    .MakeGenericType([|genArgs.[1]|])

Is there anything we can do to get the un-tagged version of the type so that it deserializes correctly?

Something like:

if genArgs.[0].UnderlyingSystemType = typeof<string> then ?

Add skippable type

As requested by @cmeeren: #24 (comment)

It is sometimes necessary to distinguish between a null field and the absence of a field. This can be represented as follows:

type Skippable<'T> =
  | Skip
  | Include of 'T

type Example =
  { a: int option Skippable }

"{}" <=> { a = Skip }
"""{"a":null}""" <=> { a = Include None }
"""{"a":42}""" <=> { a = Include (Some 42) }

Technically it's isomorphic to 'T option option, but a separate type makes the intention clear.

Converting array of array or list of lists

This package contains F# lists converter. But it does not work with included arrays:

type MyType () = 
    member val Items: int array array = [||] with get, set

for such JSON

"{ \"items\": [[1, 2, 3], [4, 5, 6]] }"

This will fail with

System.NotSupportedException: Collection was of a fixed size

String-keyed Map output like as Dictionary

I wonder why they haven't the same output.
Just curious what's the reason behind this?

Is there any possibility to have the same, Map like Dictionary?

Release new version

...So we can use all the recent changes 😄

---

In particular I need to depend on the code after the merge of this one: #36