deniszykov / msgpack-unity3d Goto Github PK
View Code? Open in Web Editor NEWMessagePack and JSON serializer for Unity3D
License: MIT License
MessagePack and JSON serializer for Unity3D
License: MIT License
Hey Denis,
first off: thanks a lot for your work on this library.
However, I ran into an issue which is propably just bad thinking on my end. Here is the deal: I try to encode data in Unity, writing it with a FileStream to Application.persistentDataPath
on a 64bit Windows machine. Afterwards I try to read the file with u-msgpack-python and while the string work out fine, the float
and int
data ist totally wrong. Do I need to take special care of something when I do this, because it seems to me that is an encoding problem? Something about the Endian? If I do it write / read the other way around (Python -> Unity) I have the same issue. The data modified in Python is not correctly read.
Int (Unity, write): 'targetHairCount': 150, 'shaveHairCount': 270
Int (Python, read): 'targetHairCount': -27136, 'shaveHairCount': 3585
Float (Unity, write): Vector3(1,2,3); Vector3(4,5,6)
Float (Python, read): '{'y': 8.96831017167883e-44, 'x': 4.600602988224807e-41, 'z': 2.304855714121459e-41} {'y': 5.74868682004613e-41, 'x': 4.600743118071239e-41, 'z': 6.89663052202102e-41}
C# code:
public void WriteData<T>(T data) {
MemoryStream stream = new MemoryStream();
MsgPack.Serialize<T>(data, stream);
stream.Position = 0;
Debug.Log("Data serialized");
using (FileStream file = new FileStream(path, FileMode.Create, System.IO.FileAccess.Write)) {
byte[] bytes = new byte[stream.Length];
stream.Read(bytes, 0, (int)stream.Length);
file.Write(bytes, 0, bytes.Length);
stream.Close();
}
}
public T ReadData<T>() {
MemoryStream stream = new MemoryStream();
using (FileStream file = new FileStream(path, FileMode.Open, FileAccess.Read)) {
byte[] bytes = new byte[file.Length];
file.Read(bytes, 0, (int)file.Length);
stream.Write(bytes, 0, (int)file.Length);
}
stream.Position = 0;
return MsgPack.Deserialize<T>(stream);
}
Python code:
import umsgpack
with open('pathtofile', 'rb') as f:
unpacked = umsgpack.unpackb(f.read())
Hi @deniszykov, I hope you are well!
Is it currently possible to decode a message starting from an offset? I have a byte array with data mixed up with MessagePack objects, and it would be great if I could decode a message starting from an offset.
As a workaround, I'm creating a "slice" of the bytes in order to decode them: https://github.com/colyseus/colyseus-unity3d/blob/b6c23dded7e1541c01252bf1970af85eb48e74c5/Assets/Plugins/Colyseus/Room.cs#L306-L309
I'd appreciate it if you can give me any hints! Cheers!
Hi, I have a serialized class from java msgpack library that have byte[] field, and I get array out of bound exception in :
MsgPack.cs line 536 :
Buffer.BlockCopy(this.buffer, this.bufferOffset, bytes, 0, this.bufferAvailable);
because required bytes are 589 and destination byte array is byte[589] but bufferAvailable ( passed as count ) is 1308 !
would you help me to fix this issue ?
Hey there,
Would it be possible to support notepack.io
's undefined
type? (https://github.com/darrachequesne/notepack#notes)
I understand there's no undefined
in C#, so we could use null
instead, what do you think?
I managed to parse the value by adding this check below:
diff --git a/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs b/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs
index 383e580..d618e2e 100644
--- a/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs
+++ b/Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs
@@ -571,6 +571,9 @@ namespace GameDevWare.Serialization.MessagePack
{
switch ((MsgPackExtType)extType)
{
+ case MsgPackExtType.None:
+ this.Value.SetValue (null, JsonToken.Null, pos);
+ break;
case MsgPackExtType.DateTimeOld:
this.Value.SetValue(new DateTime(this.bitConverter.ToInt64(data.Array, data.Offset), DateTimeKind.Utc), JsonToken.DateTime, pos);
break;
After a few milliseconds I've got this error though:
DecoderFallbackException: Exception of type 'System.Text.DecoderFallbackException' was thrown.
System.Text.DecoderExceptionFallbackBuffer.Fallback (System.Byte[] bytesUnknown, Int32 index) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/DecoderExceptionFallbackBuffer.cs:48)
System.Text.UTF8Encoding.Fallback (System.Object provider, System.Text.DecoderFallbackBuffer& buffer, System.Byte[]& bufferArg, System.Byte* bytes, Int64 index, UInt32 size) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:651)
System.Text.UTF8Encoding.InternalGetCharCount (System.Byte* bytes, Int32 count, UInt32 leftOverBits, UInt32 leftOverCount, System.Object provider, System.Text.DecoderFallbackBuffer& fallbackBuffer, System.Byte[]& bufferArg, Boolean flush) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:543)
System.Text.UTF8Encoding.InternalGetCharCount (System.Byte[] bytes, Int32 index, Int32 count, UInt32 leftOverBits, UInt32 leftOverCount, System.Object provider, System.Text.DecoderFallbackBuffer& fallbackBuffer, System.Byte[]& bufferArg, Boolean flush) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:469)
System.Text.UTF8Encoding.GetCharCount (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:687)
System.Text.Encoding.GetChars (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/Encoding.cs:328)
System.Text.Encoding.GetString (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/Encoding.cs:751)
System.Text.UTF8Encoding.GetString (System.Byte[] bytes, Int32 index, Int32 count) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Text/UTF8Encoding.cs:1052)
GameDevWare.Serialization.MessagePack.MsgPackReader.NextToken () (at Assets/Plugins/GameDevWare.Serialization/MessagePack/MsgPackReader.cs:246)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:730)
GameDevWare.Serialization.Serializers.DictionarySerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/DictionarySerializer.cs:139)
Rethrow as SerializationException: Failed to read 'Object' value for key 'players' in dictionary: Exception of type 'System.Text.DecoderFallbackException' was thrown.
More detailed information in inner exception.
GameDevWare.Serialization.Serializers.DictionarySerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/DictionarySerializer.cs:140)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.MsgPack.Deserialize (System.Type objectType, System.IO.Stream msgPackInput, GameDevWare.Serialization.SerializationContext context) (at Assets/Plugins/GameDevWare.Serialization/MsgPack.cs:66)
GameDevWare.Serialization.MsgPack.Deserialize[IndexedDictionary`2] (System.IO.Stream msgPackInput, GameDevWare.Serialization.SerializationContext context) (at Assets/Plugins/GameDevWare.Serialization/MsgPack.cs:83)
GameDevWare.Serialization.MsgPack.Deserialize[IndexedDictionary`2] (System.IO.Stream msgPackInput) (at Assets/Plugins/GameDevWare.Serialization/MsgPack.cs:71)
Colyseus.Room.Patch (System.Byte[] delta) (at Assets/Plugins/Colyseus/Room.cs:176)
Colyseus.Room.ParseMessage (System.Byte[] recv) (at Assets/Plugins/Colyseus/Room.cs:164)
Colyseus.Room.Recv () (at Assets/Plugins/Colyseus/Room.cs:73)
Colyseus.Client.Recv () (at Assets/Plugins/Colyseus/Client.cs:90)
ColyseusClient+<Start>c__Iterator0.MoveNext () (at Assets/ColyseusClient.cs:48)
UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)
Hey there,
I'm curious why I can't cast a value directly to int
? See my REPL output:
> message[4]
53
> (int) message[4]
Invalid cast.
> (uint) message[4]
Invalid cast.
> (byte) message[4]
53
Isn't 53
castable to int
?
Cheers!
Hi
I am use your library first time
but I get this error.
Can you help me ?
Thank you.
'Unexpected token readed 'EndOfStream' while 'BeginObject' is expected.'
private T ByteConverter<T>(byte[] value) { var stream = new MemoryStream(); stream.Write(value, 0, value.Length); T result = (T)MsgPack.Deserialize(typeof(T), stream); return result; }
Hi,
When using this in Unity 2018.3.0f2 (Default of Script Runtime Version is .NET 4.x) it throw error:
DynamicMethod not found in ReflectionUtils.cs. Script Runtime Version .NET 3.5.x will be remove in the future of Unity. So please fix this !
Thanks in advanced.
While I'm currently relying on JsonSerializationException.ErrorCode.UnexpectedEndOfStream
(integer value: 10) to cope with asynchronous deserialization with an intermediate MemoryStream
, I found JSonSerializationException.Code
is a cast from JsonSerializationException.ErrorCode
. I have to use a hardcoded integer value (10) in my application code to check for this special condition (not a hard failure, but waiting for more data, carry on later).
It would be great to change Code
type to ErrorCode
, while make ErrorCode
public
.
Hi @deniszykov, hope you are well!
Look, I have another discrepancy deserializing a byte array. My byte array is:
[130, 167, 112, 108, 97, 121, 101, 114, 115, 129, 169, 72, 74, 114, 111, 108, 67, 118, 57, 71, 131, 161, 120, 0, 161, 121, 226, 161, 122, 208, 146, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]
Result in JavaScript (notepack.io)
msgpack.decode(new Buffer([130, 167, 112, 108, 97, 121, 101, 114, 115, 129, 169, 72, 74, 114, 111, 108, 67, 118, 57, 71, 131, 161, 120, 0, 161, 121, 226, 161, 122, 208, 146, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]))
{ players: { HJrolCv9G: { x: 0, y: -30, z: -110 } },
messages: [] }
Result in C#
In C#, though, the "y" value is decoded as "-2", do you have any idea why this happens? Does it have something to do with the data structure I'm de-serializing into? (IndexedDictionary<string, object>)
var memoryStream = new MemoryStream(new byte[] { 130, 167, 112, 108, 97, 121, 101, 114, 115, 129, 169, 72, 74, 114, 111, 108, 67, 118, 57, 71, 131, 161, 120, 0, 161, 121, 226, 161, 122, 208, 146, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144 });
var deserialized = MsgPack.Deserialize<IndexedDictionary<string, object>> (memoryStream);
Debug.Log(deserialized["players"]["HJrolCv9G"]["y"])
// -2
Here follows a screenshot, if that helps:
Any help would be appreciated, thanks a lot, and have a great week! :)
[DataContract] public class Token { [DataMember(Name = "expires_in")] private long _expiresIn; }
To resolve this I had to cast to DataMemberAttribute directly and access "Name" property.
Hi, Denis!
Thanks for your great library! I'm trying to use it in our Unity WebGL project. When I make build with exception handling setting other then "None", I get no exception what so ever, all works as expected.
But when I set exception handling setting to "None" I got exception, log below
Exception at: Error at jsStackTrace (blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:888:12) at stackTrace (blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:902:11) at blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:81:36 at __ZN6il2cpp2vm9Exception5RaiseEP15Il2CppException [il2cpp::vm::Exception::Raise(Il2CppException*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1979621:2) at __ZN6il2cpp6icalls8mscorlib6System10Reflection4Emit13DynamicMethod21create_dynamic_methodEP29Il2CppReflectionDynamicMethodS7_ [il2cpp::icalls::mscorlib::System::Reflection::Emit::DynamicMethod::create_dynamic_method(Il2CppReflectionDynamicMethod*, Il2CppReflectionDynamicMethod*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2274584:2) at _DynamicMethod_CreateDynMethod_m3215158047 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1330456:5) at _DynamicMethod_CreateDelegate_m4040554444 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2058181:3) at _EmitContext_CreateDelegate_m2145581271 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2321076:9) at _CompilationContext_CreateDelegate_m2596329754 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2199828:9) at _CompilationContext_CreateDelegate_m2324599677 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2195885:9) at _LambdaExpression_Compile_m2019146358 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2165284:9) at _Expression_1_Compile_m1160731105_gshared (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2246223:7) at Array._GettersAndSetters__cctor_m857770292 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1706401:2) at Array.__Z31RuntimeInvoker_Void_t1841601450PK10MethodInfoPvPS2_ [RuntimeInvoker_Void_t1841601450(MethodInfo const*, void*, void**)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2362417:44) at __ZN6il2cpp2vm7Runtime6InvokeEPK10MethodInfoPvPS5_PP15Il2CppException [il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2236771:56) at __ZN6il2cpp2vm7Runtime9ClassInitEP11Il2CppClass [il2cpp::vm::Runtime::ClassInit(Il2CppClass*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1226771:6) at _TypeDescription__ctor_m2210842921 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:741450:93) at _TypeDescription_Get_m327958899 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1504759:3) at _ObjectSerializer__ctor_m217371151 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1266361:25) at _SerializationContext_CreateObjectSerializer_m4214698421 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2124258:3) at _SerializationContext_GetSerializerForType_m2482441769 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:773663:35) at _JsonReaderExtentions_ReadValue_m4065696277 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1299635:68) at Array._DictionarySerializer_Deserialize_m3731092432 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:502617:47) at __ZN16VirtFuncInvoker1IP12Il2CppObjectS1_E6InvokeEjPvS1_ [VirtFuncInvoker1<Il2CppObject*, Il2CppObject*>::Invoke(unsigned int, void*, Il2CppObject*)] (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2180082:50) at _JsonReaderExtentions_ReadValue_m4065696277 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1299635:8) at _MsgPack_Deserialize_m3395848870 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1702849:10) at Array._MsgPack_Deserialize_TisIl2CppObject_m3158509327_gshared (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1575717:7) at _MsgPack_Deserialize_TisIl2CppObject_m4015109748_gshared (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2269837:53) at _FrontListener_OnMessageHandler_m2730297209 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1262710:7) at _FrontListener_OnByteMessage_m3250805368 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:1403336:2) at Array._ReversePInvokeWrapper_FrontListener_OnByteMessage_m3250805368 (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2189979:2) at _call_cb_vb (blob:https://socket.testserver.com/f1f418f5-2198-43aa-82da-386542f59d35:2419574:43) at asm._call_cb_vb (blob:https://socket.testserver.com/8641e04a-1e75-4181-8350-065785072045:18761:26) at SendMessage (https://socket.testserver.com/bundles/leaderweb/js/game_webgl.js?v=v1502358720:61:8) at window.sendMessageToUnity (https://socket.testserver.com/bundles/leaderweb/js/game_webgl.js?v=v1502358720:77:9) at e.ws.onmessage (https://socket.testserver.com/bundles/leaderweb/js/game_webgl.js?v=v1502358720:100:13) at HTMLDivElement.<anonymous> (https://socket.testserver.com/bundles/leaderweb/js/reconnecting-websocket.js?v=v1502358720:1:877) at WebSocket.s.onmessage (https://socket.testserver.com/bundles/leaderweb/js/reconnecting-websocket.js?v=v1502358720:1:2502)
My code line that causes this error is
var str = MsgPack.Deserialize<IndexedDictionary<string, object>>(s);
s is Stream. Also I tried var str = MsgPack.Deserialize<Dictionary<string, object>>(s);
with Dictionary class but with no luck.
Message I'm receiving is following
'eventName': 'authenticate', 'data': { 'access_token': accessToken }
,
just packed.
It all works well in Editor btw.
As I understand problem is with DynamicMethod function, but what it is exactly and how to work it around I don't know.
Looking forward to your reply!
I am using msgPack in unity with .Net for online game and serialize objects to send over the net to back-end server application which developed in java, and I pack this class for example :
public class MsgPackTestClass {
public int stringManipulationForProgrammers;
public int stringManipulationForProgrammers2;
public int stringManipulationForProgrammers3;
public MsgPackTestClass()
{}
}
with this values :
var m = new MsgPackTestClass();
m.stringManipulationForProgrammers = 15;
m.stringManipulationForProgrammers2 = 255;
m.stringManipulationForProgrammers3 = 65537;
var s = new MemoryStream();
MsgPack.Serialize<MsgPackTestClass>(m, s , SerializationOptions.SuppressTypeInformation );
and finally i got this byte array ( hex converted ) :
83d920737472696e674d616e6970756c6174696f6e466f7250726f6772616d6d6572730fd921737472696e6 74d616e6970756c6174696f6e466f7250726f6772616d6d65727332d1ff00d921737472696e674d616e6970756c6174696f6e466f7250726f6772616d6d65727333d201000100
and java server using - https://github.com/msgpack/msgpack-java/tree/develop/msgpack-jackson - packing same class with same values got this byte array :
MsgPackTestClass msgPackTestClass = new MsgPackTestClass();
msgPackTestClass.setStringManipulationForProgrammers(15);
msgPackTestClass.setStringManipulationForProgrammers2(255);
msgPackTestClass.setStringManipulationForProgrammers3(65537);
ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
byte[] bytes = objectMapper.writeValueAsBytes(msgPackTestClass);
83D920737472696E674D616E6970756C6174696F6E466F7250726F6772616D6D6572730FD921737472696E674D616E6970756C6174696F6E466F7250726F6772616D6D65727332CCFFD921737472696E674D616E6970756C6174696F6E466F7250726F6772616D6D65727333CE00010001
which is different with mine and we have problems in decode class objects.
why these two libraries pack the same class ( and values ) in different way ?
also please guide me how to use msgPack in cross-platform way between java and C# ?
thank you in advance...
Can we ignore field names in serialization ? I use SerializationOptions.SuppressTypeInformation but i have my class members names in serialization.
Hi. I am using masgpack along with Best Http/2 in Unity project to deserialize SignalR messages. I used this guide to set it.
I can't fix the following exception:
{"tid":1,"div":"WebSocketTransport","msg":"OnMessage(byte[])
","ex": [{"msg": "Unexpected token readed 'BeginArray' while 'BeginObject' is expected.",
"stack": " at GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize
(GameDevWare.Serialization.IJsonReader reader) [0x00026] in ...
Here is my code for SignalR hub connection:
_connection = new HubConnection(url, new MessagePackProtocol(), options);
_connection.On<StepResponseMessage>("Step", (step) =>
{
Debug.Log("Step.result: " + step.Result);
});
Here is response message model:
[DataContract]
public class StepResponseMessage
{
[DataMember(Order = 0)]
public int Turn { get; set; }
[DataMember(Order = 1)]
public long TimeMicroseconds { get; set; }
[DataMember(Order = 2)]
public IReadOnlyList<MatchCollection> Changes { get; set; }
[DataMember(Order = 3)]
public IReadOnlyList<AttackInfo> Player { get; set; }
[DataMember(Order = 4)]
public IReadOnlyList<RecoveryInfo> PlayerRecovery { get; set; }
[DataMember(Order = 5)]
public IReadOnlyList<AttackInfo> Enemy { get; set; }
[DataMember(Order = 6)]
public IReadOnlyList<RecoveryInfo> EnemyRecovery { get; set; }
[DataMember(Order = 7)]
public string? Result { get; set; }
[DataMember(Order = 8)]
public IReadOnlyList<AbilityInfo> PlayerAbilities { get; set; }
[DataMember(Order = 9)]
public IReadOnlyList<AbilityInfo> EnemyAbilities { get; set; }
[DataMember(Order = 10)]
public IReadOnlyList<GenericAbilityActionInfo> AbilityActions { get; set; }
}
Without _connection.On<StepResponseMessage>
call I get the following message to message _connection.OnMessage
callback:
I am sure that StepResponseMessage corresponds to the messagepack object from arguments.
I am using v2.4.3 from asset store.
Could you help me with it? Am I do something wrong? Thanks.
Hi. How about make a project with .NET Standard 2.0? To use this package to create another nuget package.
Hi, in Unity you can image simple case when deserialized is class which inherits from MonoBehaviour. Then I have to create it by using AddComponent method to be able to attach it.
Sorry if newbie question and maybe a non-issue...
Anyway I'd like to know the correct way to handle deserialization of null primitive types, using the plug-in.
Thanks in advance.
Angelo
Json to deserialize
{
"elementList":[
{
"okInt":10,
"koInt":null
}
]
}
Code used
using System.Collections.Generic;
using System.Runtime.Serialization;
using UnityEngine;
using GameDevWare.Serialization;
[DataContract]
public class ElementList
{
[DataMember(Name = "elementList")]
public List<Element> elementList;
}
public class Element
{
[DataMember(Name = "okInt")]
public string okInt;
[DataMember(Name = "koInt")]
public int koInt;
}
public class TestJson : MonoBehaviour {
private string testJson = "{\"elementList\":[{\"okInt\":10, \"koInt\":null}]}";
// Use this for initialization
void Start () {
ElementList e = Json.Deserialize<ElementList>(testJson);
}
}
Resulting exception
JsonSerializationException: Unexpected token readed 'Null' while any of 'Boolean, DateTime, Null, Number, StringLiteral' are expected.
GameDevWare.Serialization.Serializers.PrimitiveSerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/PrimitiveTypeSerializer.cs:52)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:191)
Rethrow as SerializationException: Failed to read value for member 'koInt' of 'Element' type.
More detailed information in inner exception.
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:195)
GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:74)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Serializers.ArraySerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ArraySerializer.cs:63)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:191)
Rethrow as SerializationException: Failed to read value for member 'elementList' of 'ElementList' type.
More detailed information in inner exception.
GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:195)
GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) (at Assets/Plugins/GameDevWare.Serialization/Serializers/ObjectSerializer.cs:74)
GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) (at Assets/Plugins/GameDevWare.Serialization/JsonReaderExtentions.cs:726)
GameDevWare.Serialization.Json.Deserialize (System.Type objectType, System.String jsonString, GameDevWare.Serialization.SerializationContext context) (at Assets/Plugins/GameDevWare.Serialization/Json.cs:283)
GameDevWare.Serialization.Json.Deserialize[ElementList] (System.String jsonString) (at Assets/Plugins/GameDevWare.Serialization/Json.cs:331)
TestJson.Start () (at Assets/TestJson.cs:28)
Hello,
Using your provided IndexedDictionary type, Json serialize does not work.
This little code snippet fails :
var dict = new IndexedDictionary<string, int> ();
dict ["foo"] = 42;
var stream = new MemoryStream();
Json.Serialize(dict, stream);
The error is :
InvalidCastException: Cannot cast from source type to destination type.
It happens in DictionarySerializer.cs, line 176 when casting the value to an IDictionary.
It would be convenient to support serializing for this type. In my real project I have an IndexedDictionary object from deserializing an msgpack message I want to convert to Json.
Currently MsgPack.Deserialize
eventually invokes Stream.Read
which is blocking. It would be great if there is an asynchronous version BeginDeserialize<T>/EndDeserialize<T>
that takes advantage of Stream.BeginRead
/ Stream.EndRead
, so to make the operation asynchronous.
This is good enough for Unity3D (.NET 3.5). For higher version of the runtime (e.g. the experimental .NET 4.6), of course the TAP APIs can be added over the foundational Begin/End async API.
Currently to avoid blocking, I have to create a MemoryStream, asynchronously read (BeginRead
/EndRead
) from the NetworkStream
, and copy the data into the MemoryStream
object, and invoke MsgPack.Deserialize
on the MemoryStream
to make the operation fully asynchronous.
Define a class include Vector3, and Serialize a Dictionary of the class, the Unity Crash directly.
Hi, I have some problem with deserialization on ios.
System.Runtime.Serialization.SerializationException: Failed to read value for member 'Decorations' of 'Info' type.
More detailed information in inner exception. ---> System.TypeLoadException: A type load exception has occurred.
at System.Type.GetType (System.String typeName, Boolean throwOnError, Boolean ignoreCase) [0x00000] in :0
at GameDevWare.Serialization.SerializationContext.GetType (System.String name, Boolean throwOnError, Boolean ignoreCase) [0x00000] in :0
at GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.Serializers.ArraySerializer.Deserialize (IJsonReader reader) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) [0x00000] in <filename unknown>:0 at GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary
2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) [0x00000] in :0
at GameDevWare.Serialization.Serializers.ObjectSerializer.DeserializeMembers (IJsonReader reader, GameDevWare.Serialization.IndexedDictionary`2 container, GameDevWare.Serialization.Serializers.ObjectSerializer& serializerOverride) [0x00000] in :0
at GameDevWare.Serialization.Serializers.ObjectSerializer.Deserialize (IJsonReader reader) [0x00000] in :0
at GameDevWare.Serialization.JsonReaderExtentions.ReadValue (IJsonReader reader, System.Type valueType, Boolean nextToken) [0x00000] in :0
at GameDevWare.Serialization.MsgPack.Deserialize[T] (System.IO.Stream msgPackInput, GameDevWare.Serialization.SerializationContext context) [0x00000] in :0
[Serializable]
public class Info
{
public Decoration[] Decorations;
}
[Serializable]
public class Decoration
{
public Point<byte> Position;
}
[Serializable]
public class Point<TCoord> where TCoord : struct
{
public TCoord X;
public TCoord Y;
}
For fixed it the problem I added custom serializer and registered it in Json.DefaultSerializers.
public sealed class PointByteSerializer : TypeSerializer
{
public override Type SerializedType { get { return typeof(Point<byte>); } }
public override object Deserialize(IJsonReader reader)
{
if (reader == null) throw new ArgumentNullException("reader");
if (reader.Token == JsonToken.Null)
return null;
var value = new Point<byte>();
reader.ReadObjectBegin();
while (reader.Token != JsonToken.EndOfObject)
{
var memberName = reader.ReadMember();
switch (memberName)
{
case "X": value.X = reader.ReadByte(); break;
case "Y": value.Y = reader.ReadByte(); break;
default: reader.ReadValue(typeof(object)); break;
}
}
reader.ReadObjectEnd(nextToken: false);
return value;
}
public override void Serialize(IJsonWriter writer, object value)
{
if (writer == null) throw new ArgumentNullException("writer");
if (value == null) throw new ArgumentNullException("value");
var point = (Point<byte>)value;
writer.WriteObjectBegin(2);
writer.WriteMember("X");
writer.Write(point.X);
writer.WriteMember("Y");
writer.Write(point.Y);
writer.WriteObjectEnd();
}
}
but it didn't solve. Can you help, please?
Hey there,
I'm afraid there are some differences between the serialization of this library and Node's msgpack-lite
Still trying to debug to find where the difference is, I'd appreciate if you can give me any hints!
Here's the differences when trying to encode this object:
{ players: {}, messages: [] }
JavaScript:
[130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]
C#
// second and third bytes differs from JavaScript
[130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144]
Still, in JavaScript, I have the same output when trying to decode both byte arrays:
> msgpack.decode([130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }
> msgpack.decode([130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }
My guess is that C# is serializing empty objects as a different type.
This issue looks harmless on the surface but unfortunately, it's preventing me from applying byte patches using FossilDelta due to this byte difference.
Looking forward to hearing your thoughts on this!
Cheers
Make property of type Dictionary<Key, Value>, everything works great. Change it type to generic interface IDictionary<Key, Value>, works on serialization but throw exception when trying to deserialize.
Hi, I found a bug with Guid de-serialization, v.2.3.0.
Guid before = Guid.NewGuid();
Console.WriteLine($"before: {before}");
MemoryStream stream = new MemoryStream();
MsgPack.Serialize(before, stream);
stream.Position = 0;
Guid after = (Guid)MsgPack.Deserialize(typeof(Guid), stream);
Console.WriteLine($"after: {after}");
before: 0672a5be-20bd-4f54-8b86-c2e391b0cc99
after: bea57206-bd20-544f-8b86-c2e391b0cc99
Hi, is possible to add flag which would ignore all getters in the class?
I have class:
/// <summary>
/// DTO class which is used to login a player via HTTP request to OAuth server (Lonely Backend).
/// </summary>
public class PlayerLoginRequestDTO : AbstractRequestDTO
{
/// <summary>
/// Grant type for the OAuth server
/// </summary>
public readonly string grant_type;
/// <summary>
/// Login name of the user / player
/// </summary>
public readonly string username;
/// <summary>
/// Password in plain text of the user / player
/// </summary>
public readonly string password;
/// <summary>
/// Constructor creates a player login DTO which is serialized to JSON before the HTTP request is send.
/// </summary>
/// <param name="username">Login name of the user / player</param>
/// <param name="password">Password in plain text of the user / player</param>
/// <param name="grantType">The type of the credentials used. Currently supported only "password"</param>
public PlayerLoginRequestDTO(string username, string password, string grantType)
{
this.username = username;
this.grant_type = grantType;
this.password = password;
}
/// <summary>
/// Grant type for the OAuth server
/// </summary>
public string GrantType
{
get { return grant_type; }
}
/// <summary>
/// Login name of the user / player
/// </summary>
public string Username
{
get { return username; }
}
/// <summary>
/// Password in plain text of the user / player
/// </summary>
public string Password
{
get { return password; }
}
}
Which resutl in json:
{
"grant_type": "passwod",
"username": "uname",
"password": "pass",
"GrantType": "password",
"Username": "uname",
"Password": "pass"
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.