deniszykov/msgpack-unity3d

MessagePack and JSON serializer for Unity3D

msgpack-unity3d's Issues

Json deserialization of null primitive types

Sorry if newbie question and maybe a non-issue...
Anyway I'd like to know the correct way to handle deserialization of null primitive types, using the plug-in.

Thanks in advance.
Angelo

Json to deserialize

{
   "elementList":[
      {
         "okInt":10,
         "koInt":null
      }
   ]
}

Code used

using System.Collections.Generic;
using System.Runtime.Serialization;
using UnityEngine;
using GameDevWare.Serialization;


[DataContract]
public class ElementList
{
    [DataMember(Name = "elementList")]
    public List<Element> elementList;
}
public class Element
{
    [DataMember(Name = "okInt")]
    public string okInt;

    [DataMember(Name = "koInt")]
    public int koInt;
}

public class TestJson : MonoBehaviour {

    private string testJson = "{\"elementList\":[{\"okInt\":10, \"koInt\":null}]}";

    // Use this for initialization
    void Start () {
        ElementList e = Json.Deserialize<ElementList>(testJson);
	}
}

Resulting exception

JsonSerializationException: Unexpected token readed 'Null' while any of 'Boolean, DateTime, Null, Number, StringLiteral' are expected.
GameDevWare.Serialization.Serializers.PrimitiveSerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/PrimitiveTypeSerializer.cs:52)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:191)
Rethrow as SerializationException: Failed to read value for member 'koInt' of 'Element' type.
More detailed information in inner exception.
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:195)
GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:74)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Serializers.ArraySerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ArraySerializer.cs:63)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:191)
Rethrow as SerializationException: Failed to read value for member 'elementList' of 'ElementList' type.
More detailed information in inner exception.
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:195)
GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:74)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Json.Deserialize (System.Type objectType, System.String jsonString, GameDevWare.Serialization.SerializationContext context) (at Assets/Plugins/GameDevWare.Serialization/Json.cs:283)
GameDevWare.Serialization.Json.Deserialize[ElementList] (System.String jsonString) (at Assets/Plugins/GameDevWare.Serialization/Json.cs:331)
TestJson.Start () (at Assets/TestJson.cs:28)

.NET Standard 2.0

Hi. How about make a project with .NET Standard 2.0? To use this package to create another nuget package.

Ignore Getters

Hi, is possible to add flag which would ignore all getters in the class?

I have class:

    /// <summary>
    /// DTO class which is used to login a player via HTTP request to OAuth server (Lonely Backend).
    /// </summary>
    public class PlayerLoginRequestDTO : AbstractRequestDTO
    {
        /// <summary>
        /// Grant type for the OAuth server
        /// </summary>
        public readonly string grant_type;

        /// <summary>
        /// Login name of the user / player
        /// </summary>
        public readonly string username;

        /// <summary>
        /// Password in plain text of the user / player
        /// </summary>
        public readonly string password;

        /// <summary>
        /// Constructor creates a player login DTO which is serialized to JSON before the HTTP request is send.
        /// </summary>
        /// <param name="username">Login name of the user / player</param>
        /// <param name="password">Password in plain text of the user / player</param>
        /// <param name="grantType">The type of the credentials used. Currently supported only "password"</param>
        public PlayerLoginRequestDTO(string username, string password, string grantType)
        {
            this.username = username;
            this.grant_type = grantType;
            this.password = password;
        }

        /// <summary>
        /// Grant type for the OAuth server
        /// </summary>
        public string GrantType
        {
            get { return grant_type; }
        }

        /// <summary>
        /// Login name of the user / player
        /// </summary>
        public string Username
        {
            get { return username; }
        }

        /// <summary>
        /// Password in plain text of the user / player
        /// </summary>
        public string Password
        {
            get { return password; }
        }
    }

Which resutl in json:

{
  "grant_type": "passwod",
  "username": "uname",
  "password": "pass",
  "GrantType": "password",
  "Username": "uname",
  "Password": "pass"
}

Vector Crash

Define a class include Vector3, and Serialize a Dictionary of the class, the Unity Crash directly.

Can we deserialize directly to already created object?

Hi, in Unity you can image simple case when deserialized is class which inherits from MonoBehaviour. Then I have to create it by using AddComponent method to be able to attach it.

Failed to read value for member.

Hi, I have some problem with deserialization on ios.

System.Runtime.Serialization.SerializationException: Failed to read value for member 'Decorations' of 'Info' type.
More detailed information in inner exception. ---> System.TypeLoadException: A type load exception has occurred.
at System.Type.GetType (System.String typeName, Boolean throwOnError, Boolean ignoreCase) [0x00000] in :0
at GameDevWare.Serialization.SerializationContext.GetType (System.String name, Boolean throwOnError, Boolean ignoreCase) [0x00000] in :0
at GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.Serializers.ArraySerializer.Deserialize (IJsonReader reader) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) [0x00000] in :0
at GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) [0x00000] in :0
at GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) [0x00000] in :0
at GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) [0x00000] in :0
at GameDevWare.Serialization.MsgPack.Deserialize[T] (System.IO.Stream msgPackInput, GameDevWare.Serialization.SerializationContext context) [0x00000] in :0

[Serializable]
public class Info
{
    public Decoration[] Decorations;
}

[Serializable]
public class Decoration
{
    public Point<byte> Position;
}

[Serializable]
public class Point<TCoord> where TCoord : struct
{
    public TCoord X;
    public TCoord Y;
}

For fixed it the problem I added custom serializer and registered it in Json.DefaultSerializers.

public sealed class PointByteSerializer : TypeSerializer
{
    public override Type SerializedType { get { return typeof(Point<byte>); } }

    public override object Deserialize(IJsonReader reader)
    {
        if (reader == null) throw new ArgumentNullException("reader");

        if (reader.Token == JsonToken.Null)
            return null;

        var value = new Point<byte>();

        reader.ReadObjectBegin();
        while (reader.Token != JsonToken.EndOfObject)
        {
            var memberName = reader.ReadMember();
            switch (memberName)
            {
                case "X": value.X = reader.ReadByte(); break;
                case "Y": value.Y = reader.ReadByte(); break;
                default: reader.ReadValue(typeof(object)); break;
            }
        }
        reader.ReadObjectEnd(nextToken: false);
        return value;
    }

    public override void Serialize(IJsonWriter writer, object value)
    {
        if (writer == null) throw new ArgumentNullException("writer");
        if (value == null) throw new ArgumentNullException("value");

        var point = (Point<byte>)value;
        writer.WriteObjectBegin(2);
        writer.WriteMember("X");
        writer.Write(point.X);
        writer.WriteMember("Y");
        writer.Write(point.Y);
        writer.WriteObjectEnd();
    }
}

but it didn't solve. Can you help, please?

Ignore Field Names In Serialization

Can we ignore field names in serialization ? I use SerializationOptions.SuppressTypeInformation but i have my class members names in serialization.

Cross Platform Usage Problem

I am using msgPack in unity with .Net for online game and serialize objects to send over the net to back-end server application which developed in java, and I pack this class for example :

public class MsgPackTestClass  {

    public int stringManipulationForProgrammers;
    public int stringManipulationForProgrammers2;
    public int stringManipulationForProgrammers3;

    public MsgPackTestClass()
    {}
}

with this values :

var m = new MsgPackTestClass();
    m.stringManipulationForProgrammers = 15; 
    m.stringManipulationForProgrammers2 = 255; 
    m.stringManipulationForProgrammers3 = 65537; 

  var s = new MemoryStream();                
  MsgPack.Serialize<MsgPackTestClass>(m, s , SerializationOptions.SuppressTypeInformation );

and finally i got this byte array ( hex converted ) :

83d920737472696e674d616e6970756c6174696f6e466f7250726f6772616d6d6572730fd921737472696e6 74d616e6970756c6174696f6e466f7250726f6772616d6d65727332d1ff00d921737472696e674d616e6970756c6174696f6e466f7250726f6772616d6d65727333d201000100

and java server using - https://github.com/msgpack/msgpack-java/tree/develop/msgpack-jackson - packing same class with same values got this byte array :

  MsgPackTestClass msgPackTestClass = new MsgPackTestClass();

  msgPackTestClass.setStringManipulationForProgrammers(15);
  msgPackTestClass.setStringManipulationForProgrammers2(255);
  msgPackTestClass.setStringManipulationForProgrammers3(65537);

  ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());

  byte[] bytes = objectMapper.writeValueAsBytes(msgPackTestClass);

83D920737472696E674D616E6970756C6174696F6E466F7250726F6772616D6D6572730FD921737472696E674D616E6970756C6174696F6E466F7250726F6772616D6D65727332CCFFD921737472696E674D616E6970756C6174696F6E466F7250726F6772616D6D65727333CE00010001

which is different with mine and we have problems in decode class objects.
why these two libraries pack the same class ( and values ) in different way ?
also please guide me how to use msgPack in cross-platform way between java and C# ?
thank you in advance...

Unexpected token readed 'BeginArray' while 'BeginObject' is expected.

Hi. I am using masgpack along with Best Http/2 in Unity project to deserialize SignalR messages. I used this guide to set it.
I can't fix the following exception:

{"tid":1,"div":"WebSocketTransport","msg":"OnMessage(byte[])
","ex": [{"msg": "Unexpected token readed 'BeginArray' while 'BeginObject' is expected.", 
"stack": "  at GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize 
(GameDevWare.Serialization.IJsonReader reader) [0x00026] in ...

Here is my code for SignalR hub connection:

_connection = new HubConnection(url, new MessagePackProtocol(), options);
_connection.On<StepResponseMessage>("Step", (step) =>
{
    Debug.Log("Step.result: " + step.Result);
});

Here is response message model:

[DataContract]
public class StepResponseMessage
{
    [DataMember(Order = 0)]
    public int Turn { get; set; }
    [DataMember(Order = 1)]
    public long TimeMicroseconds { get; set; }
    [DataMember(Order = 2)]
    public IReadOnlyList<MatchCollection> Changes { get; set; }
    [DataMember(Order = 3)]
    public IReadOnlyList<AttackInfo> Player { get; set; }
    [DataMember(Order = 4)]
    public IReadOnlyList<RecoveryInfo> PlayerRecovery { get; set; }
    [DataMember(Order = 5)]
    public IReadOnlyList<AttackInfo> Enemy { get; set; }
    [DataMember(Order = 6)]
    public IReadOnlyList<RecoveryInfo> EnemyRecovery { get; set; }
    [DataMember(Order = 7)]
    public string? Result { get; set; }
    [DataMember(Order = 8)]
    public IReadOnlyList<AbilityInfo> PlayerAbilities { get; set; }
    [DataMember(Order = 9)]
    public IReadOnlyList<AbilityInfo> EnemyAbilities { get; set; }
    [DataMember(Order = 10)]
    public IReadOnlyList<GenericAbilityActionInfo> AbilityActions { get; set; }
}

Without _connection.On<StepResponseMessage> call I get the following message to message _connection.OnMessage callback:

I am sure that StepResponseMessage corresponds to the messagepack object from arguments.
I am using v2.4.3 from asset store.

Could you help me with it? Am I do something wrong? Thanks.

Issue with other Msgpack libraries

Hey Denis,

first off: thanks a lot for your work on this library.

However, I ran into an issue which is propably just bad thinking on my end. Here is the deal: I try to encode data in Unity, writing it with a FileStream to Application.persistentDataPath on a 64bit Windows machine. Afterwards I try to read the file with u-msgpack-python and while the string work out fine, the float and int data ist totally wrong. Do I need to take special care of something when I do this, because it seems to me that is an encoding problem? Something about the Endian? If I do it write / read the other way around (Python -> Unity) I have the same issue. The data modified in Python is not correctly read.

Int (Unity, write): 'targetHairCount': 150, 'shaveHairCount': 270
Int (Python, read): 'targetHairCount': -27136, 'shaveHairCount': 3585

Float (Unity, write): Vector3(1,2,3); Vector3(4,5,6)
Float (Python, read): '{'y': 8.96831017167883e-44, 'x': 4.600602988224807e-41, 'z': 2.304855714121459e-41} {'y': 5.74868682004613e-41, 'x': 4.600743118071239e-41, 'z': 6.89663052202102e-41}

C# code:

    public void WriteData<T>(T data) {
        MemoryStream stream = new MemoryStream();
        MsgPack.Serialize<T>(data, stream);
        stream.Position = 0;

        Debug.Log("Data serialized");

        using (FileStream file = new FileStream(path, FileMode.Create, System.IO.FileAccess.Write)) {
            byte[] bytes = new byte[stream.Length];
            stream.Read(bytes, 0, (int)stream.Length);
            file.Write(bytes, 0, bytes.Length);
            stream.Close();
        }
    }

    public T ReadData<T>() {
        MemoryStream stream = new MemoryStream();
        using (FileStream file = new FileStream(path, FileMode.Open, FileAccess.Read)) {
            byte[] bytes = new byte[file.Length];
            file.Read(bytes, 0, (int)file.Length);
            stream.Write(bytes, 0, (int)file.Length);
        }

        stream.Position = 0;

        return MsgPack.Deserialize<T>(stream);
    }

Python code:

import umsgpack

with open('pathtofile', 'rb') as f:
    unpacked = umsgpack.unpackb(f.read())

Can not JSON serialize an IndexedDictionary

Hello,

Using your provided IndexedDictionary type, Json serialize does not work.
This little code snippet fails :

	var dict = new IndexedDictionary<string, int> ();
	dict ["foo"] = 42;

	var stream = new MemoryStream();
	Json.Serialize(dict, stream);

The error is :

InvalidCastException: Cannot cast from source type to destination type.

It happens in DictionarySerializer.cs, line 176 when casting the value to an IDictionary.

It would be convenient to support serializing for this type. In my real project I have an IndexedDictionary object from deserializing an msgpack message I want to convert to Json.

"Invalid cast"?

Hey there,

I'm curious why I can't cast a value directly to int? See my REPL output:

> message[4]
53
> (int) message[4]
Invalid cast.
> (uint) message[4]
Invalid cast.
> (byte) message[4]
53

Isn't 53 castable to int?

Cheers!

'Unexpected token readed 'EndOfStream' while 'BeginObject' is expected.' ?

Hi
I am use your library first time
but I get this error.
Can you help me ?
Thank you.
'Unexpected token readed 'EndOfStream' while 'BeginObject' is expected.'

private T ByteConverter<T>(byte[] value) { var stream = new MemoryStream(); stream.Write(value, 0, value.Length); T result = (T)MsgPack.Deserialize(typeof(T), stream); return result; }

Got an exception in WebGL build on MsgPack.Deserialize<IndexedDictionary<string, object>>(s);

Hi, Denis!
Thanks for your great library! I'm trying to use it in our Unity WebGL project. When I make build with exception handling setting other then "None", I get no exception what so ever, all works as expected.
But when I set exception handling setting to "None" I got exception, log below

Exception at: Error at jsStackTrace (blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:888:12) at stackTrace (blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:902:11) at blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:81:36 at __ZN6il2cpp2vm9Exception5RaiseEP15Il2CppException [il2cpp::vm::Exception::Raise(Il2CppException*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1979621:2) at __ZN6il2cpp6icalls8mscorlib6System10Reflection4Emit13DynamicMethod21create_dynamic_methodEP29Il2CppReflectionDynamicMethodS7_ [il2cpp::icalls::mscorlib::System::Reflection::Emit::DynamicMethod::create_dynamic_method(Il2CppReflectionDynamicMethod*, Il2CppReflectionDynamicMethod*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2274584:2) at _DynamicMethod_CreateDynMethod_m3215158047 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1330456:5) at _DynamicMethod_CreateDelegate_m4040554444 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2058181:3) at _EmitContext_CreateDelegate_m2145581271 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2321076:9) at _CompilationContext_CreateDelegate_m2596329754 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2199828:9) at _CompilationContext_CreateDelegate_m2324599677 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2195885:9) at _LambdaExpression_Compile_m2019146358 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2165284:9) at _Expression_1_Compile_m1160731105_gshared (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2246223:7) at Array._GettersAndSetters__cctor_m857770292 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1706401:2) at Array.__Z31RuntimeInvoker_Void_t1841601450PK10MethodInfoPvPS2_ [RuntimeInvoker_Void_t1841601450(MethodInfo const*, void*, void**)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2362417:44) at __ZN6il2cpp2vm7Runtime6InvokeEPK10MethodInfoPvPS5_PP15Il2CppException [il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2236771:56) at __ZN6il2cpp2vm7Runtime9ClassInitEP11Il2CppClass [il2cpp::vm::Runtime::ClassInit(Il2CppClass*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1226771:6) at _TypeDescription__ctor_m2210842921 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:741450:93) at _TypeDescription_Get_m327958899 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1504759:3) at _ObjectSerializer__ctor_m217371151 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1266361:25) at _SerializationContext_CreateObjectSerializer_m4214698421 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2124258:3) at _SerializationContext_GetSerializerForType_m2482441769 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:773663:35) at _JsonReaderExtentions_ReadValue_m4065696277 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1299635:68) at Array._DictionarySerializer_Deserialize_m3731092432 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:502617:47) at __ZN16VirtFuncInvoker1IP12Il2CppObjectS1_E6InvokeEjPvS1_ [VirtFuncInvoker1<Il2CppObject*, Il2CppObject*>::Invoke(unsigned int, void*, Il2CppObject*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2180082:50) at _JsonReaderExtentions_ReadValue_m4065696277 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1299635:8) at _MsgPack_Deserialize_m3395848870 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1702849:10) at Array._MsgPack_Deserialize_TisIl2CppObject_m3158509327_gshared (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1575717:7) at _MsgPack_Deserialize_TisIl2CppObject_m4015109748_gshared (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2269837:53) at _FrontListener_OnMessageHandler_m2730297209 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1262710:7) at _FrontListener_OnByteMessage_m3250805368 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1403336:2) at Array._ReversePInvokeWrapper_FrontListener_OnByteMessage_m3250805368 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2189979:2) at _call_cb_vb (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2419574:43) at asm._call_cb_vb (blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:18761:26) at SendMessage (https://socket.testserver.com/bundles/leaderweb/js/game_webgl.js?v=v1502358720:61:8) at window.sendMessageToUnity (https://socket.testserver.com/bundles/leaderweb/js/game_webgl.js?v=v1502358720:77:9) at e.ws.onmessage (https://socket.testserver.com/bundles/leaderweb/js/game_webgl.js?v=v1502358720:100:13) at HTMLDivElement.<anonymous> (https://socket.testserver.com/bundles/leaderweb/js/reconnecting-websocket.js?v=v1502358720:1:877) at WebSocket.s.onmessage (https://socket.testserver.com/bundles/leaderweb/js/reconnecting-websocket.js?v=v1502358720:1:2502)

My code line that causes this error is
var str = MsgPack.Deserialize<IndexedDictionary<string, object>>(s); s is Stream. Also I tried var str = MsgPack.Deserialize<Dictionary<string, object>>(s); with Dictionary class but with no luck.

Message I'm receiving is following
'eventName': 'authenticate', 'data': { 'access_token': accessToken } ,
just packed.

It all works well in Editor btw.
As I understand problem is with DynamicMethod function, but what it is exactly and how to work it around I don't know.

Looking forward to your reply!

Exposing JsonSerializationException.ErrorCode

While I'm currently relying on JsonSerializationException.ErrorCode.UnexpectedEndOfStream (integer value: 10) to cope with asynchronous deserialization with an intermediate MemoryStream, I found JSonSerializationException.Code is a cast from JsonSerializationException.ErrorCode. I have to use a hardcoded integer value (10) in my application code to check for this special condition (not a hard failure, but waiting for more data, carry on later).

It would be great to change Code type to ErrorCode, while make ErrorCode public.

MsgPackReader.cs ReadBytes(long bytesRequired, bool forceNewBuffer = false) Problem

Hi, I have a serialized class from java msgpack library that have byte[] field, and I get array out of bound exception in :
MsgPack.cs line 536 :
Buffer.BlockCopy(this.buffer, this.bufferOffset, bytes, 0, this.bufferAvailable);

because required bytes are 589 and destination byte array is byte[589] but bufferAvailable ( passed as count ) is 1308 !

would you help me to fix this issue ?

Pair of serialization/deserialization of IDictionary doesn't work

Make property of type Dictionary<Key, Value>, everything works great. Change it type to generic interface IDictionary<Key, Value>, works on serialization but throw exception when trying to deserialize.

[question] problem encoding types

Hey there,

I'm afraid there are some differences between the serialization of this library and Node's msgpack-lite

Still trying to debug to find where the difference is, I'd appreciate if you can give me any hints!

Here's the differences when trying to encode this object:

{ players: {}, messages: [] }

JavaScript:

[130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]

C#

// second and third bytes differs from JavaScript
[130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144]

Still, in JavaScript, I have the same output when trying to decode both byte arrays:

> msgpack.decode([130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }

> msgpack.decode([130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }

My guess is that C# is serializing empty objects as a different type.

This issue looks harmless on the surface but unfortunately, it's preventing me from applying byte patches using FossilDelta due to this byte difference.

Looking forward to hearing your thoughts on this!
Cheers

Decode bytes starting from offset

Hi @deniszykov, I hope you are well!

Is it currently possible to decode a message starting from an offset? I have a byte array with data mixed up with MessagePack objects, and it would be great if I could decode a message starting from an offset.

As a workaround, I'm creating a "slice" of the bytes in order to decode them: https://github.com/colyseus/colyseus-unity3d/blob/b6c23dded7e1541c01252bf1970af85eb48e74c5/Assets/Plugins/Colyseus/Room.cs#L306-L309

I'd appreciate it if you can give me any hints! Cheers!

BeginDeserialize/EndDeserialize

Currently MsgPack.Deserialize eventually invokes Stream.Read which is blocking. It would be great if there is an asynchronous version BeginDeserialize<T>/EndDeserialize<T> that takes advantage of Stream.BeginRead / Stream.EndRead, so to make the operation asynchronous.

This is good enough for Unity3D (.NET 3.5). For higher version of the runtime (e.g. the experimental .NET 4.6), of course the TAP APIs can be added over the foundational Begin/End async API.

Currently to avoid blocking, I have to create a MemoryStream, asynchronously read (BeginRead/EndRead) from the NetworkStream, and copy the data into the MemoryStream object, and invoke MsgPack.Deserialize on the MemoryStream to make the operation fully asynchronous.

Unexpected value when deserializing (int)

Hi @deniszykov, hope you are well!

Look, I have another discrepancy deserializing a byte array. My byte array is:

[130, 167, 112, 108, 97, 121, 101, 114, 115, 129, 169, 72, 74, 114, 111, 108, 67, 118, 57, 71, 131, 161, 120, 0, 161, 121, 226, 161, 122, 208, 146, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]

Result in JavaScript (notepack.io)

msgpack.decode(new Buffer([130, 167, 112, 108, 97, 121, 101, 114, 115, 129, 169, 72, 74, 114, 111, 108, 67, 118, 57, 71, 131, 161, 120, 0, 161, 121, 226, 161, 122, 208, 146, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]))

{ players: { HJrolCv9G: { x: 0, y: -30, z: -110 } },
  messages: [] }

Result in C#

In C#, though, the "y" value is decoded as "-2", do you have any idea why this happens? Does it have something to do with the data structure I'm de-serializing into? (IndexedDictionary<string, object>)

var memoryStream = new MemoryStream(new byte[] { 130, 167, 112, 108, 97, 121, 101, 114, 115, 129, 169, 72, 74, 114, 111, 108, 67, 118, 57, 71, 131, 161, 120, 0, 161, 121, 226, 161, 122, 208, 146, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144 });

var deserialized = MsgPack.Deserialize<IndexedDictionary<string, object>> (memoryStream);
Debug.Log(deserialized["players"]["HJrolCv9G"]["y"])
// -2

Here follows a screenshot, if that helps:

Any help would be appreciated, thanks a lot, and have a great week! :)

Guid de-serialization

Hi, I found a bug with Guid de-serialization, v.2.3.0.

        Guid before = Guid.NewGuid();
        Console.WriteLine($"before: {before}");

        MemoryStream stream = new MemoryStream();
        MsgPack.Serialize(before, stream);

        stream.Position = 0;

        Guid after = (Guid)MsgPack.Deserialize(typeof(Guid), stream);
        Console.WriteLine($"after: {after}");

before: 0672a5be-20bd-4f54-8b86-c2e391b0cc99
after: bea57206-bd20-544f-8b86-c2e391b0cc99

ReflectionExtensions.GetDataMemberName doesn't work under WebGL in Unity

[DataContract] public class Token { [DataMember(Name = "expires_in")] private long _expiresIn; }

To resolve this I had to cast to DataMemberAttribute directly and access "Name" property.

Compatibility with notepack.io's "undefined" value

Hey there,

Would it be possible to support notepack.io's undefined type? (https://github.com/darrachequesne/notepack#notes)

I understand there's no undefined in C#, so we could use null instead, what do you think?

I managed to parse the value by adding this check below:

diff --git a/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs b/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs
index 383e580..d618e2e 100644
--- a/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs
+++ b/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs
@@ -571,6 +571,9 @@ namespace GameDevWare.Serialization.MessagePack
                {
                        switch ((MsgPackExtType)extType)
                        {
+                               case MsgPackExtType.None:
+                                       this.Value.SetValue (null, JsonToken.Null, pos);
+                                       break;
                                case MsgPackExtType.DateTimeOld:
                                        this.Value.SetValue(new DateTime(this.bitConverter.ToInt64(data.Array, data.Offset), DateTimeKind.Utc), JsonToken.DateTime, pos);
                                        break;

After a few milliseconds I've got this error though:

DecoderFallbackException: Exception of type 'System.Text.DecoderFallbackException' was thrown.
System.Text.DecoderExceptionFallbackBuffer.Fallback (System.Byte[] bytesUnknown, Int32 index) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/DecoderExceptionFallbackBuffer.cs:48)
System.Text.UTF8Encoding.Fallback (System.Object provider, System.Text.DecoderFallbackBuffer& buffer, System.Byte[]& bufferArg, System.Byte* bytes, Int64 index, UInt32 size) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:651)
System.Text.UTF8Encoding.InternalGetCharCount (System.Byte* bytes, Int32 count, UInt32 leftOverBits, UInt32 leftOverCount, System.Object provider, System.Text.DecoderFallbackBuffer& fallbackBuffer, System.Byte[]& bufferArg, Boolean flush) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:543)
System.Text.UTF8Encoding.InternalGetCharCount (System.Byte[] bytes, Int32 index, Int32 count, UInt32 leftOverBits, UInt32 leftOverCount, System.Object provider, System.Text.DecoderFallbackBuffer& fallbackBuffer, System.Byte[]& bufferArg, Boolean flush) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:469)
System.Text.UTF8Encoding.GetCharCount (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:687)
System.Text.Encoding.GetChars (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/Encoding.cs:328)
System.Text.Encoding.GetString (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/Encoding.cs:751)
System.Text.UTF8Encoding.GetString (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:1052)
GameDevWare.Serialization.MessagePack.MsgPackReader.NextToken () (at Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs:246)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:730)
GameDevWare.Serialization.Serializers.DictionarySerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/DictionarySerializer.cs:139)
Rethrow as SerializationException: Failed to read 'Object' value for key 'players' in dictionary: Exception of type 'System.Text.DecoderFallbackException' was thrown.
More detailed information in inner exception.
GameDevWare.Serialization.Serializers.DictionarySerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/DictionarySerializer.cs:140)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.MsgPack.Deserialize (System.Type objectType, System.IO.Stream msgPackInput, GameDevWare.Serialization.SerializationContext context) (at Assets/Plugins/GameDevWare.Serialization/MsgPack.cs:66)
GameDevWare.Serialization.MsgPack.Deserialize[IndexedDictionary`2] (System.IO.Stream msgPackInput, GameDevWare.Serialization.SerializationContext context) (at Assets/Plugins/GameDevWare.Serialization/MsgPack.cs:83)
GameDevWare.Serialization.MsgPack.Deserialize[IndexedDictionary`2] (System.IO.Stream msgPackInput) (at Assets/Plugins/GameDevWare.Serialization/MsgPack.cs:71)
Colyseus.Room.Patch (System.Byte[] delta) (at Assets/Plugins/Colyseus/Room.cs:176)
Colyseus.Room.ParseMessage (System.Byte[] recv) (at Assets/Plugins/Colyseus/Room.cs:164)
Colyseus.Room.Recv () (at Assets/Plugins/Colyseus/Room.cs:73)
Colyseus.Client.Recv () (at Assets/Plugins/Colyseus/Client.cs:90)
ColyseusClient+<Start>c__Iterator0.MoveNext () (at Assets/ColyseusClient.cs:48)
UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)

[Request] Support for Unity 2018.3.0f2 (Script Runtime Version .NET 4.x)

Hi,
When using this in Unity 2018.3.0f2 (Default of Script Runtime Version is .NET 4.x) it throw error:
DynamicMethod not found in ReflectionUtils.cs. Script Runtime Version .NET 3.5.x will be remove in the future of Unity. So please fix this !
Thanks in advanced.