Comments (12)
Probably looks a lot like this:
sealed class ProtobufNetSerializer : IFusionCacheSerializer
{
private readonly TypeModel _model;
public ProtobufNetSerializer(TypeModel model = null)
=> _model = model ?? RuntimeTypeModel.Default;
public byte[] Serialize<T>(T obj)
{
using var ms = new MemoryStream();
_model.Serialize<T>(ms, obj);
return ms.ToArray();
}
public T Deserialize<T>(byte[] data)
=> _model.Deserialize<T>((ReadOnlyMemory<byte>)data);
ValueTask<T> IFusionCacheSerializer.DeserializeAsync<T>(byte[] data)
=> new(Deserialize<T>(data));
ValueTask<byte[]> IFusionCacheSerializer.SerializeAsync<T>(T obj)
=> new(Serialize<T>(obj));
}
There is an API that can avoid having to use the MemoryStream
/ unknown-length thing, but right now: I'd go with ^^^
from fusioncache.
Thanks @mgravell !
I was about to push my first impl and ask for your opinion, but you've beaten me to it 😄
From what I can see they are quite similar, at least in principle.
Here it is:
public class FusionCacheProtoBufNetSerializer
: IFusionCacheSerializer
{
public FusionCacheProtoBufNetSerializer(RuntimeTypeModel? model = null)
{
_model = model ?? RuntimeTypeModel.Default;
// ENSURE MODEL REGISTRATION FOR FusionCacheEntryMetadata
if (_model.IsDefined(typeof(FusionCacheEntryMetadata)) == false)
{
_model.Add(typeof(FusionCacheEntryMetadata), false)
.SetSurrogate(typeof(FusionCacheEntryMetadataSurrogate))
;
}
}
private readonly RuntimeTypeModel _model;
private readonly object _modelLock = new object();
private void EnsureDistributedEntryModelIsRegistered<T>()
{
// TODO: OPTIMIZE THIS
var _t = typeof(T);
if (_t.IsGenericType == false || _t.GetGenericTypeDefinition() != typeof(FusionCacheDistributedEntry<>))
return;
if (_model.IsDefined(_t))
return;
lock (_modelLock)
{
if (_model.IsDefined(_t))
return;
// ENSURE MODEL REGISTRATION FOR FusionCacheDistributedEntry<T>
_model.Add(typeof(T), false)
.Add(1, nameof(FusionCacheDistributedEntry<T>.Value))
.Add(2, nameof(FusionCacheDistributedEntry<T>.Metadata))
;
}
}
public byte[] Serialize<T>(T? obj)
{
EnsureDistributedEntryModelIsRegistered<T>();
using (var stream = new MemoryStream())
{
_model.Serialize<T?>(stream, obj);
return stream.ToArray();
}
}
public T? Deserialize<T>(byte[] data)
{
if (data.Length == 0)
return default(T);
EnsureDistributedEntryModelIsRegistered<T>();
using (var stream = new MemoryStream(data))
{
return _model.Deserialize<T?>(stream);
}
}
public ValueTask<byte[]> SerializeAsync<T>(T? obj)
{
return new ValueTask<byte[]>(Serialize(obj));
}
public ValueTask<T?> DeserializeAsync<T>(byte[] data)
{
return new ValueTask<T?>(Deserialize<T>(data));
}
}
A couple of notes:
- since models must be defined for Protobuf to work best, and since I don't want users of FusionCache to have to define the needed boilerplate types themselves but, at most, their own types, I decided to automatically ensure the needed ones are there. In the end they are the
FusionCacheEntryMetadata
type (added in the ctor) and each specializedFusionCacheDistributedEntry<T>
type (added before each use) for each T that is used, because generics etc. Would you say this is the best approach? - in the auto-registration code, after checking if the type is a
FusionCacheDistributedEntry<T>
, I'm currently usingIsDefined(Type)
. Is this the best approach? - I see you're using
TypeModel
instead ofRuntimeTypeModel
: would I be correct in assuming that, since I'm auto-adding models, usingRuntimeTypeModel
instead is the right thing? - in the
Deserialize()
impl I'm checking first if the data has a length of zero and in that case I'm returningdefault(T)
. I'm doing this because, strangely enough, I noticed that if I serialize anull
string and then I deserialize it, it seems to return an empty string instead. I've already searched online for a behaviour like this, but I only found this SO answer which does not seem to be my case
For the last point this is a minimal repro:
string? value1 = null;
string? value2 = null;
byte[] data;
using (var ms = new MemoryStream())
{
ProtoBuf.Serializer.Serialize(ms, value1);
data = ms.ToArray();
}
using (var ms = new MemoryStream(data))
{
value2 = ProtoBuf.Serializer.Deserialize<string>(ms);
}
Console.WriteLine($"VALUE 1 IS NULL: {value1 is null}");
Console.WriteLine($"VALUE 2 IS NULL: {value2 is null}");
And its output is:
VALUE 1 IS NULL: True
VALUE 2 IS NULL: False
Am I missing something?
Thanks!
from fusioncache.
Mmmh, upon further investigation it seems that the call to IsDefined()
is auto-adding the type to be checked, breaking the logic. I assume it may be related to one of the AutoAdd/AutoCreate options in the ModelType.
I'm trying to understand more, maybe using the CanSerialize()
method, and will update later.
from fusioncache.
from fusioncache.
from fusioncache.
Better option might be to hook the events on the model that are invoked when discovering a new type; I'm not at a PC to provide an example, but: Before/After something something!
Thanks for the suggestion, will definitely look into it!
from fusioncache.
Also, if it won't upset the other serializers: the config can be expressed via DataContract/DataMember(Order=...)
Will definitely check this, too: If I remember correctly the 2 classes are already using those attributes (maybe with a Name, and not yet with an Order, but I can add that) but as the code suggests I'm also using a surrogate class for the metadata class, since that in turn is using a DateTimeOffset
prop (and that is how I ended up on that other issue on the protobuf-net repo).
Anyway will play with the DataContract/DataMember approach too, thanks!
from fusioncache.
Hi @mgravell , FYI I've been able to create a minimal repro about the null
string issue I was observing, so I opened an issue in the protobuf-net repo.
Hope this helps.
from fusioncache.
Better option might be to hook the events on the model that are invoked when discovering a new type; I'm not at a PC to provide an example, but: Before/After something something!
Hi @mgravell , thanks for your help.
As per your suggestion I've looked into the before/after events to hook into registering a type just before serializing, but I've only found BeforeApplyDefaultBehaviour
and AfterApplyDefaultBehaviour
which don't seem to fire when trying to serialize an unregistered type.
Also as said, calls to both IsDefined()
and CanSerialize()
to check if a type is already registered seem to auto-add them, defeating the purpose of the check, and trying to add the same type multiple times without having checked throws an exception (which in and on itself makes sense).
The only way I've found is one of these 2:
- keep an external list of already registered types, to keep track of the ones that still needs to be registered and avoid the exceptions
- simply call
Add()
every time, but inside a try/catch block just to suppress the exception
Both of these are clearly not good solutions imho, so I'm still trying to come up with something better.
Initially I also played with the idea of using a small internal static class like internal static MyTypeCache<T>
where I tried to use the static ctor (guaranteed to only be called once etc) and use that to be sure to register a type only once: the problem with such approach is that it would be in practice a singleton, and since the specific TypeModel
to use is passed in my serializer ctor for better flexibility, I cannot do that, too.
Any idea?
from fusioncache.
Update: I've pushed the branch, the serializer code can be found here.
from fusioncache.
Update: I went with a mix of both options.
I added an internal cache (static/singleton) to keep track of each model's registered types, of course only the ones related to FusionCache, to auto-register them. I also added some locks (with the usual double check) to ease hypothetical concurrency issues that may arise and finally, when registering the types, I also included a try/catch block to avoid any issue with double registration (it should not happen, but I don't have exclusive control over what happens to a model that may be passed in the ctor.
All seems to be fine now: there are no more simply try/catch blocks that would throw every time, the locking seems fine (to me at least) and the little overhead with the internal models/types cache should be negligible, so I'm feeling good overall about the approach.
Will release it soon, and will notify it here too, just to keep track.
Thanks for your support, and if you happen to have any suggestion it would still be more than welcome.
from fusioncache.
I've released v0.16.0 which includes support for Protobuf 🎉
Thanks again @mgravell for your support!
from fusioncache.
Related Issues (20)
- [FEATURE] 🪞 Remove all usage of reflection HOT 6
- [FEATURE] Better detection of incoherent `CacheName`s options HOT 4
- [QUESTION] Can I use FusionCache only for getting data from a 2nd level (distributed) without InMemory part? HOT 5
- [FEATURE] 📢 Add `IgnoreIncomingBackplaneNotifications` options HOT 2
- [FEATURE] Improve nullable for GetOrSet with non-null factory HOT 8
- Explicitly tell the Factory that it failed HOT 9
- [FEATURE] Combination of eager refresh with distributed cache duration. HOT 5
- [FEATURE] Add a method for list all caches in IFusionCacheProvider (Ex: for dispose purposes) HOT 4
- Possibility of creation of a count variable in distributed cache HOT 2
- [FEATURE] Post factory executed event with possibility to change the cache settings before it be inserted in cache HOT 3
- Syncing different cloud instances HOT 4
- [BUG] Value is stored for longer than FailSafeMaxDuration HOT 7
- [BUG] AllowBackgroundDistributedCacheOperations serialization and cache operations seem to be happening in the foreground HOT 3
- best way to store FusionCacheEntryOptions in iConfiguration HOT 3
- Nullable `Size` option HOT 3
- [BUG] Deadlock caused by cancellation of eager refresh HOT 8
- [BUG] Slow Performance of FusionCache compared to Microsoft.Extensions.Caching.Memory.MemoryCache HOT 4
- [BUG] FailSafe max duration is somehow ignored HOT 7
- [FEATURE] Optimise dependencies by adding TFM HOT 9
- 💡 Add this project to awesome-italia-opensource HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fusioncache.