ziggycreatures / fusioncache Goto Github PK
View Code? Open in Web Editor NEWFusionCache is an easy to use, fast and robust cache with advanced resiliency features and an optional distributed 2nd level.
License: MIT License
FusionCache is an easy to use, fast and robust cache with advanced resiliency features and an optional distributed 2nd level.
License: MIT License
We have two sources of data, primary and secondary.
❓ How can we configure distributed cache to a different value than the local cache?
Also, setting Duration = TimeSpan.Zero
results in an exception for distributed cache, which would be solved by setting distributed cache duration differently:
System.ArgumentOutOfRangeException: The absolute expiration value must be in the future. (Parameter 'AbsoluteExpiration')
Actual value was 04.07.2022 12:17:49 +00:00.
at Microsoft.Extensions.Caching.StackExchangeRedis.RedisCache.GetAbsoluteExpiration(DateTimeOffset creationTime, DistributedCacheEntryOptions options)
at Microsoft.Extensions.Caching.StackExchangeRedis.RedisCache.SetAsync(String key, Byte[] value, DistributedCacheEntryOptions options, CancellationToken token)
at ZiggyCreatures.Caching.Fusion.Internals.Distributed.DistributedCacheAccessor.<>c__DisplayClass15_0`1.<<SetEntryAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at ZiggyCreatures.Caching.Fusion.Internals.FusionCacheExecutionUtils.RunAsyncActionAdvancedAsync(Func`2 asyncAction, TimeSpan timeout, Boolean cancelIfTimeout, Boolean awaitCompletion, Action`1 exceptionProcessor, Boolean reThrow, CancellationToken token)
Hey @jodydonetti,
I don't think is a major issue, but it did trip up a unit test I was working on. In some scenarios it seems like IsFailsSafeEnabled = false
is ignored. Here's an example.
[Fact]
public void IsFailSafeEnabledIgnoredHere()
{
using var cache = new FusionCache(new FusionCacheOptions()
{
DefaultEntryOptions = new FusionCacheEntryOptions()
{
IsFailSafeEnabled = true,
Duration = TimeSpan.FromSeconds(1),
FailSafeMaxDuration = TimeSpan.FromSeconds(2)
}
});
const string cacheKey = "cacheKey";
cache.Set(cacheKey, "someValue");
var cacheItem = cache.TryGet<string>(cacheKey);
Assert.True(cacheItem.HasValue);
Thread.Sleep(1500);
cacheItem = cache.TryGet<string>(cacheKey);
Assert.True(cacheItem.HasValue);
cacheItem = cache.TryGet<string>(cacheKey, new FusionCacheEntryOptions() { IsFailSafeEnabled = false });
Assert.False(cacheItem.HasValue); //Fails here
Thread.Sleep(2001);
cacheItem = cache.TryGet<string>(cacheKey);
Assert.False(cacheItem.HasValue);
}
[Fact]
public void IsFailSafeEnabledWorksAsExpected()
{
using var cache = new FusionCache(new FusionCacheOptions()
{
DefaultEntryOptions = new FusionCacheEntryOptions()
{
IsFailSafeEnabled = true,
Duration = TimeSpan.FromSeconds(1),
FailSafeMaxDuration = TimeSpan.FromSeconds(2)
}
});
const string cacheKey = "cacheKey";
cache.Set(cacheKey, "someValue");
var cacheItem = cache.TryGet<string>(cacheKey);
Assert.True(cacheItem.HasValue);
Thread.Sleep(1500);
cacheItem = cache.TryGet<string>(cacheKey, new FusionCacheEntryOptions() { IsFailSafeEnabled = false });
Assert.False(cacheItem.HasValue); //Is false here
Thread.Sleep(2001);
cacheItem = cache.TryGet<string>(cacheKey);
Assert.False(cacheItem.HasValue);
}
[Fact]
public void IsFailSafeEnabledWorksAsExpected2()
{
using var cache = new FusionCache(new FusionCacheOptions()
{
DefaultEntryOptions = new FusionCacheEntryOptions()
{
IsFailSafeEnabled = true,
Duration = TimeSpan.FromSeconds(1),
FailSafeMaxDuration = TimeSpan.FromSeconds(2)
}
});
const string cacheKey = "cacheKey";
cache.Set(cacheKey, "someValue");
var cacheItem = cache.TryGet<string>(cacheKey);
Assert.True(cacheItem.HasValue);
Thread.Sleep(1500);
cacheItem = cache.TryGet<string>(cacheKey, new FusionCacheEntryOptions());
Assert.False(cacheItem.HasValue);
cacheItem = cache.TryGet<string>(cacheKey, new FusionCacheEntryOptions() { IsFailSafeEnabled = false });
Assert.False(cacheItem.HasValue); //Is false here
Thread.Sleep(2001);
cacheItem = cache.TryGet<string>(cacheKey);
Assert.False(cacheItem.HasValue);
}
Are my assumptions incorrect on how to use FusionCacheEntryOptions
? This was only an issue if a null FusionCacheEntryOptions
is provided on the first TryGet
Describe the bug
I have FusionCache setup with Redis and backplane, all seems to be working ok on the surface, can see the keys being added to Redis and used (on occasion) however on closer inspection (while investigating cold start behaviour I noticed unexpected behaviour.
After cold start some keys are looked up in the distributed cache and collected while others are not.
To Reproduce
var cacheEntry = await _cache.GetOrSetAsync("User:UserViewModel:1_40181", async _ => await BuildUserModelFromContext(), options => options.SetDuration(TimeSpan.FromMinutes(5)));
var search = await _cache.GetOrSetAsync("User:Search:MainSearchResult:1_40181", async _ => {... code... return data;}, options => options.SetDuration(TimeSpan.FromMinutes(5)));
There are 2 uses of the cache using same options called shortly after one another but they behave differently
2022-08-17 15:57:38.8322|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=aca34cf0429f4dd4a9bedfbbf5755552 K=User:UserViewModel:1_40181): calling GetOrSetAsync<T> FEO[LKTO=/ DUR=5m DDUR=/ JIT=2s PR=L FS=Y FSMAX=02:00:00 FSTHR=30s FSTO=100ms FHTO=2s TOFC=Y DSTO=1s D
HTO=2s ABDO=Y BN=Y BBO=Y]
2022-08-17 15:57:38.8322|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=aca34cf0429f4dd4a9bedfbbf5755552 K=User:UserViewModel:1_40181): memory entry not found
2022-08-17 15:57:38.8322|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=aca34cf0429f4dd4a9bedfbbf5755552 K=User:UserViewModel:1_40181): memory entry not found
2022-08-17 15:57:38.8667|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=aca34cf0429f4dd4a9bedfbbf5755552 K=User:UserViewModel:1_40181): distributed entry found FE[FFS=N LEXP=33s]
2022-08-17 15:57:38.8667|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=aca34cf0429f4dd4a9bedfbbf5755552 K=User:UserViewModel:1_40181): saving entry in memory MEO[CEXP=2022-08-17T16:57:39.7442166+00:00 PR=L S=1] FE[FFS=N LEXP=34s]
2022-08-17 15:57:38.8667|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=aca34cf0429f4dd4a9bedfbbf5755552 K=User:UserViewModel:1_40181): return FE[FFS=N LEXP=34s]
2022-08-17 15:57:38.8667|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=0d112accd53c4b0ea88eb15c08e59dc2 K=User:Search:MainSearchResult:1_40181): calling GetOrSetAsync<T> FEO[LKTO=/ DUR=5m DDUR=/ JIT=2s PR=L FS=Y FSMAX=02:00:00 FSTHR=30s FSTO=100ms FHTO=2s TOFC=
Y DSTO=1s DHTO=2s ABDO=Y BN=Y BBO=Y]
2022-08-17 15:57:38.8667|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=0d112accd53c4b0ea88eb15c08e59dc2 K=User:Search:MainSearchResult:1_40181): memory entry not found
2022-08-17 15:57:38.8667|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=0d112accd53c4b0ea88eb15c08e59dc2 K=User:Search:MainSearchResult:1_40181): memory entry not found
2022-08-17 15:57:38.9551|DEBUG|ZiggyCreatures.Caching.Fusion.FusionCache|FUSION (O=0d112accd53c4b0ea88eb15c08e59dc2 K=User:Search:MainSearchResult:1_40181): calling the factory (timeout=2s)
2022-08-17 15:57:38.9551|DEBUG|Microsoft.EntityFrameworkCore.Query|Compiling query expression:
'DbSet<usp_GetCatalogueSummaryListResult>().FromSql(EXEC [Products].[usp_GetCatalogueSummaryList] @FormatId, __p_0)
...
Why does User:UserViewModel:1_40181
look in distributed cache (and find it and use it) but User:Search:MainSearchResult:1_40181
does not, I can see it is in the Redis database and has not expired.
Expected behavior
For User:Search:MainSearchResult:1_40181
to be checked in distributed cache, found and used.
Versions
I've encountered this issue on:
Add any other context about the problem here.
Hello, thanks for this library!
I'm getting an error when trying to use the redis layer.
Type Error: NOPERM this user has no permissions to access one of the keys used as arguments.
https://redis.io/topics/acl
I think I've tracked down the problem to this line of code:
https://github.com/jodydonetti/ZiggyCreatures.FusionCache/blob/1cb0c169d56898746f6823405a3fe3f54bfa8389/src/ZiggyCreatures.FusionCache/Internals/Distributed/DistributedCacheAccessor.cs#L16
My connection is limited to only read and write from keys that have a prefix in the form foo:bar:*
so, having interactions like v1:foo:bar
break. Is there a way around that? I'm thinking of implementing my own IDistributedCache
that uses the normal implementation but overrides that key prefix, it looks hacky but should work.
Hello, I'm interested in adding FusionCache into one of my libraries as a replacement for LazyCache. However it's missing a feature I use from the base Microsoft cache abstractions, namely the ability to set the cache expiry dynamically at the time of entry renewal via the ICacheEntry
interface like so:
await _cache.GetOrAddAsync("key", async e =>
{
var response = await _client.GetRemoteItemAsync();
e.SetAbsoluteExpiration(response.Expiry);
return response.Item;
});
This is super useful if you don't know the expiry of the item upfront. Are there any plans to support something like this?
Hi there FusionCache users, I need some help from you all.
Recently, while designing some new features and working on the backplane (#11) I realized that, in terms of design, the FusionCacheOptions.CacheKeyPrefix
feature is probably not that good to have inside on FusionCache itself.
Let me explain.
The idea to introduce the CacheKeyPrefix
option came up because sometimes we want to use the same distributed cache instance (eg. Redis) for multiple different sets of data at the same time.
For example in one of my projects I'm setting a prefix based on the environment type, like "d:" for development, "s:" for staging and "p:" for production, so I can use the same Redis instance and save some bucks.
With this little "trick" I can have the same "logical" piece of data in the cache like the product with id 123 (which would normally have a cache key of "product/123") but for 2 different environments, without one inadvertently overwrite the other (because with the CacheKeyPrefix we will have 2 different cache keys: "d:product/123" and "p:product/123").
Because of this I initially thought that having such a small feature would be a nice addition, so I went on and implemented it.
From "the outside" you would be able to specify your cache keys, and work with every method specifying your "logical" cache keys (eg: "product/123") knowing that on "the inside" the little magic will happen (eg: turn it to "p:product/123").
The thing is, cache key generation is NOT FusionCache's concern, but your own: it is outside the scope and responsibility of FusionCache, and it should be only yours.
The moment FusionCache modify a cache key it receives, it starts messing around with a critical piece of data (the cache key itself) and this may have unintended consequences.
For example right now each single method in FusionCache receives a cache key and, before doing anything else, modifies it by applying the specified CacheKeyPrefix
(if any) and then moving on, forgetting about the original cache key altogether and working only on the modified one.
This means that everything that happens inside of FusionCache (and, in turn, in the memory and distributed cache) will be tied to the modified cache keys, including logging, events and whatnot.
Right now I'm working on the Backplane (#11): suppose we have 3 nodes (N1, N2 and N3), suppose we use the "logical" cache key of "product/123" and a CacheKeyPrefix
of "foo:".
Then this would happen:
Set("product/123", product)
on N1"foo:product/123"
"foo:product/123"
"foo:product/123"
set
operation for the key "foo:product/123"
"foo:product/123"
"foo:product/123"
, and try to evict the entries from their local cache for that keySet
, Remove
, etc) and that, in turn, would re-apply the prefix, obtaining a cache key of "foo:foo:product/123"
(see the double "foo:" there?)This is a nice case of Leaky Abstraction we got, amirite?
Now, of course I may "simply" un-apply the prefix to the cache key before sending the notification (point 6) by doing some substring magic or something, or I may "keep" the original cache key + the modified one, but both of those would mean extra work at runtime (cpu/memory), it would smell really bad and, as said, it's not really a good thing design-wise.
So, what are the possible alternatives?
The fact is that every distributed cache out there (and every impl of IDistributedCache
) is already natively able to specify different "sets" to be able to split the data in the same service instance, but in a more idiomatic and native way:
DefaultDatabase
SchemaName
and TableName
DatabaseName
and ContainerName
DatabaseName
and CollectionName
Each of them is surely the best way to handle such a scenario in each specific service.
I'd like to remove support for the feature altogether, since:
Now, to be more precise, I'd like to do it NOT by removing the CacheKeyPrefix
prop itself, otherwise dependent code would not compile anymore and people wouldn't know why, but by decorating the prop as [Obsolete]
with the error
param set to true
: this would make the dependent code not compile, BUT with the ability to specify a descriptive message to explain things and suggest how to handle the removal, maybe even with a link to an online page with a thorough explanation and some examples.
If you are actually using this option (let me know!) and you really like your cache keys with the prefix they currently have, here's an easy way to keep them as they are.
I can imagine that, probably in a Startup.cs
file you will have something like this:
services.AddFusionCache(options =>
{
[...]
if (Env.IsProduction())
options.CacheKeyPrefix = "prod:";
});
Later on you are using FusionCache like this:
cache.GetOrSet<Product>($"product/{id}", ...);
What you would have to do is declare some public static prop like this:
public static class MyCache {
public static string? Prefix { get; set; }
}
change the startup code to something like this:
if (Env.IsProduction())
MyCache.Prefix = "prod:";
services.AddFusionCache(options =>
{
[...]
});
and finally when you later use FusionCache like this:
cache.GetOrSet<Product>($"{MyCache.Prefix}product/{id}", ...);
If instead you just like to keep data apart without using the prefix, simply set a Database, DatabaseName, CollectionName or ContainerName based on the technology you are using (Redis, MongoDB, SqlServer, etc...) like mentioned in the Alternatives chapter above.
CacheKeyPrefix
option?Right now I'm the only one I know of that is using this obscure little feature.
I've talked with about a dozen people who are using FusionCache, and none of them are using it.
But since the downloads for the main package alone just crossed 17K (thanks all 🎉) I'm here asking you all how you feel about it.
If you agree with this change, simply vote with a 👍.
If you disagree, please comment with the reasoning behind it so I can understand more about your use case.
Thanks to anyone who will participate!
Say I have a web application running on 4 servers that uses FusionCache with a Redis secondary cache and I remove a cache entry from one of the web applications. It looks like the local FusionCache will be removed and the Redis secondary cache will have it's entry removed but how will the other 3 servers know to clear their FusionCache entries?
Is your feature request related to a problem? Please describe.
Sometimes it may be useful to be able to directly catch serialization/deserialization exceptions, instead of just logging them: this may be useful to better dignose serialization issues.
Describe the solution you'd like
Add the ability to re-throw exceptions happened during serialization/deserialization instead of just logging them, via a new ReThrowSerializationExceptions
option.
The very same feature is already available for the distributed cache exceptions, via the ReThrowDistributedCacheExceptions
option.
I'm using the library to store values in Redis. When saving data with the SetAsync
method I've found that it doesn't fail the task when an exception occurs when saving the data. I would like to be able to recover from this scenario as I need to ensure that the data was saved successfully. Whats the best way of handling this scenario?, digging into the code I see that the RunAsyncActionAdvancedAsync
in FusionCacheExecutionUtils
has a reThrow
attribute that would solve the issue, but it is always being passed as false with no way to override it. I believe I could simply fetch the value afterwards with TryGet
but think it would be easier if there was a way to know if the Set method failed.
Thanks for your help. I'm really liking the library so far.
Hi! I'd like to have a different configuration for the duration of the in memory values than the ttl in redis. Is there a way around that?
The idea is that the memory cache refreshes more often than the redis cache, so if other node updates the value in the distributed cache, the memory one will only be stalled by the duration of the in memory configuration.
Thanks! :)
A good way to reduce the amount of allocations would be to switch from Task<T>
to ValueTask<T>
.
In theory (and in practice, at least on a couple of projects where I tested it) there should be no change in code already using FusionCache, at least if that code is not doing something esoteric.
When calling async methods by using the standard await
keyword the changes required are effectively none.
More info here https://blog.marcgravell.com/2019/08/prefer-valuetask-to-task-always-and.html , with the great Marc Gravell explaining everything way better than me.
Opinions?
Is your feature request related to a problem? Please describe.
Since the backplane is implemented on top of a distributed component (in general some sort of message bus, like the Redis Pub/Sub feature) sometimes things can go bad: the message bus can restart or become temporarily unavailable, transient network errors may occur or anything else. In those situations each local nodes' memory caches will become out of sync, since they missed some notifications.
FusionCache should help in handling these situations, ideally automatically.
Describe the solution you'd like
The idea is to add an auto-recovery feature, that will detect notifications that failed to be sent, put them in a local temporary queue and later on, as soon as the backplane will become available again, will try to send them to all the other nodes to re-sync them correctly.
It should also handle specific scenarios like:
Hi,
We are testing your cache implementation, well done for a great work. During Load testing on One API call we started to get the Unable to activate FAIL-SAFE (no entries in memory or distributed)
warning.
It managed to set data on high volume of requests, but after many many misses
Here is my code:
public async Task<Data> GetDataAsync(string identifier, IdentifyBy identifyBy)
{
var cacheKey = $"Info:{identifier}:{identifyBy}";
var data = await _cache.GetAsync<Data>(cacheKey);
if(data == null)
{
data = new Data { Id = 1 };
await _cache.SetAsync(cacheKey , data, TimeSpan.FromMinutes(10));
} else
{
_logger.LogInformation($"Successfully got Data Information from Cache. identifier: {identifier} by { identifyBy }");
}
}
Thanks for sharing your amazing library!
I have FusionCache configured with a secondary cache. I am running multiple instances/nodes of my ASP.NET Core web application. I have a value I am caching for 10 minutes.
In Node A this value is updated from X to Z and I call FusionCache's SetAsync()
to update the value in the primary and secondary cache. In Node B the old value, X, was read and cached 1 minute before it was updated in Node A.
If GetOrSetAsync()
is called 2 minutes later in Node B, will it still return the old value, X, from it's primary cache? Or, is there some magical mechanism to push primary cache invalidations across all nodes?
"Don't do that" is a perfectly valid response. Open to any insights on how to handle this corner case or how to avoid getting into this case. Thanks!
While playing with the implementation of the backplane (see #11) and talking about adding metrics (see #9) the need emerged to be able to react to core events happening inside FusionCache.
The plan is to add a set of core events to FusionCache (like hit, miss, etc) so it will be possible to subscribe to them.
An example of these events (still a draft):
These events will NOT be used for the core flow, that is FusionCache will call subscribers to them but it will NOT need them to function properly or wait for them to return some results.
Also, the order in which subscribers will be called will not be guaranteed (for potential perf optimizations).
There are different ways to model this flow:
StackExchange.Redis
package for example uses a model where both the Subscribe
and Unsubscribe
methods accept the handler
simply as a lambda param (see here). This is very simple and lightweight (no extra allocations), but when using it with anonymous methods (created on the fly) it requires storing them in a variable to be able to later unbubscribeIDisposable
that can be used to unsubscribe later on, by simply calling Dispose()
on it. This is very straightforward an it allows to simply collect the results of each Subscribe()
call, maybe in a list, and later on just call Dispose()
on all of them to unsubscrive from them all, but it incurs in an additional allocation (the disposable itself). Maybe a struct implementing IDisposable may alleaviate the allocation cost? Does it feel right?Any suggestion is more than welcome.
Is your feature request related to a problem? Please describe.
FusionCache provides a way to add custom serialization format support, by implementing the IFusionCacheSerializer
interface.
It already provides 2 implementations, both for the JSON format:
It recently added (but not yet released) a third one: MessagePack.
It would be great to add support for the Protobuf serialization format, too.
Describe the solution you'd like
A new package that add supports for the Protobuf format, probably based on protobuf-net by @mgravell which is probably the most used implementation on .NET and the most performant.
Describe alternatives you've considered
Everyone that needs it should otherwise implement their own, which is meh 😐
I had tried out ZiggyCreatures.FusionCache.Backplane.Memory for local development with a single instance. Maybe I didn't set it up right, but if I set a breakpoint in events for cache.Events.Set and either cache.Events.Remove or cache.Events.Memory.Eviction (sorry, I don't remember which it was hitting), then each time a cache value was set, it was immediately removed. Effectively, the cache was never used and the factory was always called.
After I removed this line, then everything worked fine and dandy:
services.AddFusionCacheMemoryBackplane();
I didn't really have any good reason for using it for local dev and am fine without it, so I'm good with closing this issue immediately if you like.
hi,
I would like to understand the difference between using the 2nd level option (of using a distributed cache) and Backplane with distributed cache? Somehow both of them looks same to me. How are they different?
Also, for distributed cache, you mention that any backend that supports IDistributedCache can be used. However for backplane, only Redis is supported, not the others which support IDistributedCache. Why is that?
Thanks!
Is your feature request related to a problem? Please describe.
FusionCache provides a way to add custom serialization format support, by implementing the IFusionCacheSerializer
interface.
It already provides 4 implementations:
It would be great to add support for the new super optimized binary format by @neuecc , MemoryPack !
Describe the solution you'd like
A new package that add supports for the MemoryPack format, based on MemoryPack.
Describe alternatives you've considered
Everyone that needs it should otherwise implement their own, which is meh 😐
I've read through most documentation I can find and a good amount of discussions on here in the issues. Perhaps I'm not thinking through my scenario clearly, but I feel like GetOrSet should have a option of some kind to ignore null values and just return null.
In a previous issue, someone had mentioned something along the lines of GetOrSetOrDefault ... which may be what I'm thinking.
The issue I see is this:
return _cache.GetOrSet(
"key",
ctx => {
_repository.Get(...);
},
opt => { ... }
);
If _repository.Get(..)
returns null
, I don't want to cache this. My only option is to throw an exception from within the factory func. Is this the intended design?
I could, of course, break this down into separate TryGetAsync
, followed by a SetAsync
.
Add some docs about DI integration, usage, example, etc including advanced customization.
I like all of the functionality currently provided within FusionCache, but I would like the ability to turn off the memorycache option.
When using Azure Functions I would prefer to just go straight to Redis and not have the overhead of memorycache due to their stateless nature. I still want to have the fallback value capability in the event Redis is unavailable, but I just don't need the extra step of dealing with memorycache.
This would also help with longer-lived data which could change very infrequently but is accessed often. If I have a long TTL on a value I could have an instance where a load-balanced environment could be out of sync. Let's say I have a TTL of 90 minutes on data but due to a production problem I need to update the current value in Redis. I have no way of evicting the current memorycache values even though Redis has a more current (and correct) value.
This does touch a bit on your backplane idea, but in that case, I would still like to have a preferred cache type for a value. I think this could go in the FusionCacheEntryOptions as a PreferredCache property. This value could be an enum of Local or Distributed. This way I could determine which cache to prefer for a value.
So, this could be seen as 1 of 2 proposals to the FusionCacheEntryOptions:
public CacheLocation CacheLocation { get; set; } = CacheLocation.All;
public enum CacheLocation
{
All,
LocalOnly,
RemoteOnly
}
public PreferredCache PreferredCache { get; set; } = PreferredCache.Local;
public enum PreferredCache
{
Local,
Remote
}
Looking at the GetOrSetEntryInteral I am not sure if one will be easier than the other. If I had to pick only one option I would say option 1 would do what I really want at the moment. Since I know I would be losing performance by going to Redis first then memorycache would have a smaller performance benefit since I can still use the fallback option.
What are your thoughts?
Is your feature request related to a problem? Please describe.
FusionCache provides a way to add custom serialization format support, by implementing the IFusionCacheSerializer
interface.
It already provides 2 implementations, both for the JSON format:
It would be great to add support for the MessagePack serialization format, too.
Describe the solution you'd like
A new package that add supports for the MessagePack format, probably based on the most used implementation on .NET, the one by Neuecc.
Describe alternatives you've considered
Everyone that needs it should implement their own, which is meh 😐
The following code will trigger the SET event and I thing that is good and correct. But it does not trigger the Miss event. I think it should trigger the Miss event if it did not previously exist in the cache at anytime. Think about it this way. If you have a cache item in a cache it had to get there because once upon a time it as not in the cache, thus the rational that it should have historicaly been a cache miss. I discovered this while building my OpenTelemetry plugin examples. I am using the GetOrSet now that I have the Adaptive Caching feature in my hands. That is when I realized couldn't find a Miss events in my prometheus (time series) database.
await cache.GetOrSetAsync<int>(
"foo",
async (_) =>
{
await Task.Delay(1);
return 123;
});
I was hoping you could add documentation on how to unit test when using this.
I am unable to moq the task being passed to the cache or to moq the cache method that would return the data. Any advice would be greatly appreciated.
Hi @jodydonetti,
have you ever considered extracting interfaces and common classes to separate library like ZiggyCreatures.FusionCache.Abstractions
?
This may allow plugins and higher app layers not to depend on main package.
A discussion has started around the possibility of introducing the concept of "cache regions" (see #33 ): there it is possible to see the potential problems involved, and some possible ways around those.
Thanks to @jasenf to for bringing that up and providing his experience on the subject.
This issue will be used to track the hypothetical design, limitations and feature set around regions.
(This is my first time using GitHub, so if I'm doing something wrong, please don't hesitate to tell me)
Hello,
I am considering using FusionCache for a project and I am currently testing it out.
I have a scenario that I don't know how to solve with my current knowledge about FusionCache, hence this question.
The scenario is that I want to prefill the cache with data, but then let the GetOrSet factory stuff handle cache misses, timeouts, cache refreshes and all that good stuff after the cache have been filled.
I want to prefill the cache from a database and the important detail about this scenario is that i only want to call the database once, when prefilling.
Example:
I have 10.000.000 key value pairs in a db, and i know the ids upfront.
I have a method that takes a single id called GetValue(int id), which accesses the db.
I could prefill the cache by doing GetOrSet({id}, _ => GetValue(id)) for each of the ids.
But to my understanding this would go to the db 10M times, which is the reason I want to go to the db once.
So conceptually I would like to do something like
But that just seems... a bit hacky to me as i neither want to get nor set the value, i just want to set the factory up
I hope it makes sense, else I am happy to provide further information or input :)
Is your feature request related to a problem? Please describe.
Despite configuring all minimum log levels to Error
, there are still lots of logs spamming my Serilog sinks with Debug level like these:
[12:03:15 DBG] FUSION (O=9ff57fa21f0b4df5968ceda1f0289538 K=MyKey): memory entry found FE[]
[12:03:15 DBG] FUSION (O=9ff57fa21f0b4df5968ceda1f0289538 K=MyKey): return FE[]
[12:03:15 DBG] FUSION (O=e4c2667cd74d49198de1f16e981e0484 K=MyKey): calling GetOrSetAsync<T> FEO[LKTO=/ DUR=06:00:00 JIT=0 PR=N FS=N FSMA
X=1.00:00:00 FSTHR=30s FSTO=0 FHTO=/ TOFC=Y DSTO=/ DHTO=/ ABDO=N BN=Y BBO=Y]
I would like to configure a minimum log level for FusionCache, in addition to the existing log level options for individual groups of logs.
Describe the solution you'd like
Add FusionCacheOptions.MinimumLogLevel
, which controls the minimum log level. Defaults to Verbose.
Describe alternatives you've considered
Followed instructions in https://github.com/jodydonetti/ZiggyCreatures.FusionCache/blob/main/docs/StepByStep.md#9-logging.
Explored all configurable options, could not find any.
Workaround is to configure Serilog to set a minimun log level override for these loggers.
.MinimumLevel.Override("ZiggyCreatures.Caching.Fusion.FusionCache", LogEventLevel.Information)
Hi,
The more we use the library, the more we love it, but there is odd issue happens from time to time:
Newtonsoft.Json.JsonSerializationException: Type specified in JSON 'ZiggyCreatures.Caching.Fusion.Internals.Distributed.FusionCacheDistributedEntry`1[[System.Object, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ZiggyCreatures.FusionCache, Version=0.1.9.0, Culture=neutral, PublicKeyToken=null' is not compatible with 'ZiggyCreatures.Caching.Fusion.Internals.Distributed.FusionCacheDistributedEntry`1[[System.Collections.Generic.IEnumerable`1[[infra.Models.Notification, infra.Models, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ZiggyCreatures.FusionCache, Version=0.1.9.0, Culture=neutral, PublicKeyToken=null'. Path '$type', line 1, position 171.
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolveTypeName(JsonReader reader, Type& objectType, JsonContract& contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, String qualifiedTypeName)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadMetadataProperties(JsonReader reader, Type& objectType, JsonContract& contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue, Object& newValue, String& id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
at ZiggyCreatures.Caching.Fusion.Serialization.NewtonsoftJson.FusionCacheNewtonsoftJsonSerializer.Deserialize[T](Byte[] data)
at ZiggyCreatures.Caching.Fusion.Serialization.NewtonsoftJson.FusionCacheNewtonsoftJsonSerializer.DeserializeAsync[T](Byte[] data)
at ZiggyCreatures.Caching.Fusion.Internals.Distributed.DistributedCacheAccessor.TryGetEntryAsync[TValue](String operationId, String key, FusionCacheEntryOptions options, Boolean hasFallbackValue, CancellationToken token)
ZiggyCreatures.Caching.Fusion.FusionCache: Warning: FUSION (O=0005a0db3f0742339f84e64d84f56196 K=f35741d8-a1aa-425e-89d9-eec64e028789:NotificationRepository.All): unable to activate FAIL-SAFE (no entries in memory or distributed)
ZiggyCreatures.Caching.Fusion.FusionCache: Warning: FUSION (O=4a99e5456567499a859d49af206e8082 K=f35741d8-a1aa-425e-89d9-eec64e028789:SettingsDBRepository.All): an error occurred while deserializing an entry
This is the entry:
{
"$id": "1",
"$type": "ZiggyCreatures.Caching.Fusion.Internals.Distributed.FusionCacheDistributedEntry`1[[System.Object, System.Private.CoreLib]], ZiggyCreatures.FusionCache",
"Value": {
"$type": "System.Collections.Generic.List`1[[infra.Models.Notification, infra.Models]], System.Private.CoreLib",
"$values": [
{
"$id": "2",
"$type": "infra.Models.Notification, infra.Models",
"Id": 2,
"Title": "Test",
"Url": "https://www.youtube.com/watch?v=UBUArSZWZeg",
"EventAlertConfigId": null,
"ExternalClickUrl": null,
"CreatedBy": null
}
]
}
}
This is the configuration:
services.AddFusionCacheNewtonsoftJsonSerializer(new JsonSerializerSettings
{
ContractResolver = new JsonIgnoreAttributeIgnorerContractResolver(),
ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
PreserveReferencesHandling = PreserveReferencesHandling.Objects,
TypeNameHandling = TypeNameHandling.All
});
We use In Memory and Redis with .net6
In the Cache Levels docs add a specific mention about how to achieve a 2nd level that is based on a local file (via Sqlite).
Hi,
I am curious whether you have explicitly decided not to support batch methods (if so what are the reasons - implementation burden / other patterns to use / no need etc.) or is it planned for the future?
BTW. Neat project ;)
Hello,
I'd like to know if there is the possibility to set a sliding expiration so that if the cached object is updated or read, the expiration counter is restarted.
Is this possible? If not, are there other ways to acheive it?
Thanks in advance and keep up the good work!
Currently the AddFusionCache
has the following parameters:
Action<FusionCacheOptions>? setupOptionsAction = null,
bool useDistributedCacheIfAvailable = true,
bool ignoreMemoryDistributedCache = true,
Action<IServiceProvider, IFusionCache>? setupCacheAction = null
Imagine some wants to control caching behavior/setup (memory or memory+distributed or memory+distributed+backplane).
useDistributedCacheIfAvailable
allows to skip distributed cache setup during AddFusionCache
call, so that it can be setup later using setupCacheAction
. Thus, defered setup through setupCacheAction
allows to e.g. resolve a IOptions<CacheSettings>
that may store whether to use distributed caching or not. Unfortunately, backplane "auto" setup cannot be disabled through a parameter.
Write custom AddFusionCache extension method in application codebase.
Change Action<IServiceProvider, IFusionCache>? setupCacheAction
to Action<FusionCacheFactoryContext>? setupCacheAction
public class FusionCacheFactoryContext
{
public IServiceProvider ServiceProvider { get; }
public IFusionCache Cache { get; }
public FusionCacheContext(IServiceProvider serviceProvider, IFusionCache cache)
{
ServiceProvider = serviceProvider;
Cache = cache;
}
}
Then, having a FusionCacheFactoryContext
that wraps both IServiceProvider
and IFusionCache
allows writing extensions to setup e.g.:
public static FusionCacheFactoryContext SetupBackplaneFromServices(this FusionCacheFactoryContext context)
{
var backplane = context.ServiceProvider.GetService<IFusionCacheBackplane>();
if (backplane is not null)
{
context.Cache.SetupBackplane(backplane);
}
}
Finally there can be two AddFusionCache
overloads:
(this IServiceCollection services, Action<FusionCacheOptions>? setupOptionsAction = null)
that setups FusionCache using default/current "auto" behavior(this IServiceCollection services, Action<FusionCacheOptions>? setupOptionsAction = null, Action<FusionCacheFactoryContext> setupCacheAction, bool setupDefaults = true)
that allows to freely customize FusionCache setupI have a use case where the returned data from a function passed to FusionCache contains the TTL (time to live), kind of like a DNS query answer. I would like to use that result to set the duration from calls like GetOrSetAsync
.
I see MemoryOptionsModifier
was something that might have been the way to go but the property is marked obsolete and not wired in.
Of course I can call TryGetAsync and then SetAsync in the meantime. @jodydonetti curious about your thoughts on this?
Thank you so much for the library. I'm trying to use it on my project and faced an issue when using default .net 3.1 json serializer and IDistributedCache
backed by Redis.
Steps to reproduce:
services.AddFusionCacheSystemTextJsonSerializer
class LastModified
{
public DateTime Date { get; set; }
}
await _cache.GetOrSetAsync<LastModified>(_cacheKey, ReadFromDb, token: token);
You'll notice a warning written to the console:
Here is what is read from the Redis
Notice the Metadata
object. It doesn't have a parameterless constructor and that's what exception is about.
Hi! First, thank you for making such a great library! I am trying to implement FusionCache into an API running on EF Core 5, Mysql and Redis. The issue is that when the FactorySoftTimeout is triggered and the factory is supposed to continue in the background and update the cache when finished, we are instead presented by an System.ObjectDisposedException error:
It looks like the DbContext gets disposed before the factory running in the background is finished working with it. How can I keep the DbContext open for the factory to finish its task?
This is my EF Core database configuration:
This is the FusionCache configuration:
And this is how FusionCache is used inside a Controller:
In the Service, DbContext object is accessed via a dependency injection.
If you need any additional information or code snippets I will be happy to provide.
I would be very thankful for any help or advice you can offer.
Kind regards,
Tomaž
First off, the library looks great, especially with the introduction of the backplane!
This may be an extension to the ongoing cache region discussion, but I think deems it's own topic.
I am working on a PoC utilising this library and came across a gap compared to our current implementation on top of LazyCache.
It is the equivalent of a Clear mechanism affecting all or part of the cache.
MemoryCacheEntryOptions of IMemoryCache supports entries with attached ExpiryTokens. Each entry can have multiple tokens assigned to it and when the token is cancelled, all associated cache values are expired.
Currently we track the issued tokens in a ConcurrentDictionary keyed on a string.
This allows for us to do 2 things
The primary reason for this approach was to solve cache invalidation across services.
ServiceA caches
ServiceB caches
Now if I update data in ServiceB, I raise an event (Rest API, or Message Queue) that ServiceA listens to and thus calls .Clear("ServiceB"). (note only one listener responds on ServiceA due to load balancing).
The second part of this implementation is segmentation/relations inside of ServiceA
Where CacheKey2 & CacheKey3 data is built on top of CacheKey1 data. (I understand this may not be the 'right' way but it addresses our situation, where cacheKey1 is computationally intensive as is CacheKey2 & CaheKey3)
cacheKey1 = "baseCacheData" eg "template"
cacheKey2 = "UserID:baseCacheData+UserSpecificEnhancements" eg "123:template:nhanced"
cacheKey3 = "UserID:baseCacheData+UserSpecificEnhancements" eg "234:templateEnhanced"
Now if I change cacheKey1 it should invalidate cacheKey2 and cacheKey3.
Currently I would use a cacheGroup token to relate these cache entries together, and expire cacheKey1 before setting a new value for it.
This approach also supports nested relations like
"{GroupID}:{UserID}:Data"
Each entry has 2 change tokens, 1 for the group and 1 for the user.
With this I can invalidate a variety of data subsets, either on user change to invalidate across all groups, or on group change to invalidate across all users.
I am not sure of the side effects of this in FusionCache, especially with Background and Backplane support.
I would happily keep the the .Clear() functionality and tracking of ExpiryTokens outside of FuctionCache, but would be great if I could easily add the token when creating an entry.
Would be great to discuss this possibility.
The current implementation does not take into account the AbsolutExpiration
set during the addition of the entry to the IDistributeCache
This may lead to the hidden extension of the lifetime of the entry (especially for the TryGet[Async]
method).
As IDistributedCache
does not return back the expiration time, it's possible to extend the Metadata
payload with the calculated absolute expiration and use it during the creation of the entry in the in-memory cache.
Also, it is worth reflecting in the documentation that the current implementation uses FusionCacheEntryOptions
for the restored entires.
While playing with the implementation of the backplane (see #11) and talking about adding metrics (see #9) the need emerged to be able to add functionalities around FusionCache via external code.
In some cases a specific interface is needed because it is a core part of the system (like the already existing IFusionCacheSerializer
), but in a lot of other cases a more generic one would probably suffice.
The objective is to:
The idea is to create a plugin subsystem where multiple plugins can be implemented using a common interface that would allow the coordination with a FusionCache instance.
As a first draft I'm thinking about something like this:
public interface IFusionCachePlugin
{
void Start(IFusionCache cache);
void Stop(IFusionCache cache);
}
In the Start
method a plugin implementer will receive a cache instance to then, for example, subscribe to the events they'd like (see #14) or do something else, while in the Stop
they can remove the events subscriptions they've created before, to keep a system clean.
A plugin will be added to a cache with a method like IFusioCache.AddPlugin(IFusionCachePlugin plugin)
, similarly to the one for the distributed cache which would add the plugin instance to an internal list of plugins, and call its Start
method. In the same vein, a method IFusioCache.RemovePlugin(IFusionCachePlugin plugin)
can be called to remove a plugin (which in turn will call the Stop
method on the plugin) for housekeeping purposes.
In a DI scenario the method will be called automatically for all the registered services implementing the IFusionCachePlugin type, like what is already happening for the IDistributedCache type here, but with potentially multiple implementations.
The code may be something like this:
[...]
var plugins = serviceProvider.GetServices<IFusionCachePlugin>();
foreach (var plugin in plugins)
{
cache.AddPlugin(plugin);
}
[...]
In case specific functionalities may be needed by FusionCache, a more specialized plugin type may be created simply inheriting from the base IFusionCachePlugin
type and adding the specific apis needed.
If, for example, the metrics plugins need a special metrics-related method to be called for metrics-related things, we would have something like this:
public interface IFusionCacheMetricsPlugin:
: IFusionCachePlugin
{
Task<bool> DoMetricsStuffAsync(int param1, string param2, [etc...]);
}
NOTE: I'm still playing with the backplane impl, and it could very well fit into this as a "normal" plugin.
Should there be a way to identify a plugin, apart from its clr type? Something like a string Id { get; set; }
? It may be useful in some contexts (eg: logging).
Any suggestion is more than welcome.
In the docs there's a FactoryOptimization.md
which says
Special care is put into calling just one factory per key, concurrently
Can you please provide more details on how this works? I'm curious what happens within FusionCache when:
Does FusionCache implement some sort of distributed lock on a given storage to synchronize nodes? For instance, some atomic operation is needed in SQL to create a "locking" row for a key, call factory and insert value, or remove lock if factory failed.
In a multi-node scenario the typical multi-level cache configuration uses a different number of local memory caches and one distributed cache, used to share entries between the different nodes.
When an entry is set on a local memory cache it is also set in the distributed cache, so that other nodes will get the entry from there when they see it's not in their local memory cache.
A problem may arise when an entry is already in one or more nodes' memory cache and the entry is overwritten on another node: in this situation the memory cache for which the Set
method has been called will be updated and the same can be said for the distributed cache, but the other nodes with the old entries would still use those old entries until they expire.
There are 2 ways to alleviate this situation:
use a very low cache duration, but that in turn may increase the load of the data source (eg: a database)
use a lower duration for the memory cache and a higher one for the distributed cache so that the shared (updated) entries are frequently read by the nodes, but that is not (currently) possible in FusionCache and may also lead to a potentially higher load on the distributed cache (instead of on the datasource), on top of still using stale data even if for shorter amount of time
Both of these solutions may be good in some use cases, and thanks to FusionCache combo of fail-safe and advanced timeouts with background factory completion the result for your end users would be good, but it's not a real solution to the problem.
The idea is to introduce the concept of a backplane which would allow a communication between all the nodes involved about update/removal of entries, so that they can stay up to date about the state of the system.
A new IFusionCacheBackplane
interface to model a generic backplane, which could then be implemented in various ways on top of different systems.
It should contain a couple of core methods to notify the change or removal of an entry with 2 different semantics for them because:
an explicit remove on a node (eg: a call to Remove(key, ...)
) should actually remove the entries on the other nodes to avoid finding a value that should not be there anymore
an update (eg: a call to Set(key, ...)
) should not remove the entries on the other nodes but just mark those entries - if there - as "logically expired" (eg: change their FusionCacheEntryMetadata.LogicalExpiration
) so that at the next access the factory would be executed to get the new value, while still keeping the ability to use the stale value in case of problems or timeouts during the factory execution, which is an added bonus of using FusionCache
It should be noted that directly sending the updated values with the notifications themselves is not considered for various reasons:
Additionally a small circuit-breaker like the one already present in FusionCache when talking to the distributed cache would be a nice addition, since the same problems of intermittent conection can potentially happen with the backplane.
Ideally I would also explore a form of batching to allow sending an invalidation notification for multiple keys at once, to save some bandwidth (but that may introduce a higher complexity in the codebase which I would like to keep as readable as possible).
The first implementation would be on Redis, because:
One thing to know about the pub/sub mechanism in Redis is that any message sent will be received by all the nodes connected, including the sender itself. To avoid the eviction of the entry in the same node that originated the notification a form of sender identifier (like a UUID/ULID or similar) should be included in the message payload.
Also the design should be evolvable, to avoid a situation in the future where a new protocol design would break the system when introduced into a live system where nodes are communicating with the v1 and v2 is being introduced.
Of course other implementations may be done with different tecnologies.
Say we want to "cache something for 10min
".
Easy peasy, we can do something like this:
var id = 42;
cache.GetOrSet<Product>(
$"product:{id}",
_ => GetProductFromDb(id),
options => options.SetDuration(TimeSpan.FromMinutes(10))
);
Sometimes though we may want to do something like "cache something for 10min
, but start refreshing it some time before expiration so that at the 10min
mark there would not be a slowdown because of the refresh operation".
With FusionCache this has always been possible, thanks to fail-safe + soft/hard timeouts: we just have to change the way to pose the requirement to something like "cache something for 10min
, but in case the refresh will take more than (say) 10ms
just temporarily reuse the stale value so there would not be a slowdown because of the refresh operation".
The code needed would be:
var id = 42;
cache.GetOrSet<Product>(
$"product:{id}",
_ => GetProductFromDb(id),
options => options
.SetDuration(TimeSpan.FromMinutes(10))
.SetFailSafe(true)
.SetFactoryTimeouts(TimeSpan.FromMilliseconds(10))
);
The end result is basically the same (no delays when refreshing), but the thing needed is a mental shift in how to think about what to do.
Now, here's the deal: it would be nice to be able to just specify "eagerly refresh some time before the expiration" or something like that, instead of having to change the mental model.
This approach is also not completely new: in the caching field there are things like the StaleAfter
option in the CacheTower library, or the "Cache Prefreshing" option in the Akamai CDN.
It seems reasonable to provide a way to obtain the same result but with a direct and more clear approach, even just to lower the mental gymnastics needed and to lower the entry barrier.
Finally, this approach may be used in conjunction with the aforementioned existing features (fail-safe and timeouts), so that we may be able to either use eager refresh without fail-safe (if so desired) or to use all of them together.
A new addition in the FusionCacheEntryOptions
class, to be able to specify how eagerly to start the refresh, even if the cache entry is not yet expired.
There are 2 possible ways to specify "how eagerly".
As a TimeSpan
this would be a direct value, like TimeSpan.FromSeconds(10)
.
DefaultEntryOptions
, to automatically adapt to each call's specific Duration
Duration
in the future (error prone)As a percentage, in the usual floating point notation: an example may be 0.9
, meaning 90%
(of the Duration
).
Duration
used. For example by saying 0.9
you will know that it will be "90%
of the Duration
", without having to do mental calculations (most probably the mental approach is not tied to a specific TimeSpan
value, but more something like "I would like it to happen at 90% of the Duration
")TimeSpan
) to keep the 2 aligned (less error prone)DefaultEntryOptions
and automatically applied to every call, dynamically adapting to each call's Duration
Duration
the data will be refreshed" would most probably be more than enough. Also, if we think about debugging/logging, it's really easy to log the eager duration as both a percentage AND as the resulting (calculated) TimeSpan
, for ease of useBecause of the reasons above, it seems clear that the percentage approach would be better, so this will be explored in an impl and see how it goes.
Also, although this does not imply anything in particular, it gives some confidence knowing that the Akamai CDN actually uses the percentage approach: this is, at least, a point in favor of such approach, since it has been widely used in a battle tested production environment with success.
One additional idea may be to have support for both: this solution though would mean worse performance (more memory consumed to store both of the values).
Also, it would probably create some confusion about what approach to use, and what may happen when setting both values (which one should win? should setting one value reset the other? etc).
Finally, for the reasons explained above, it may possibly be more error prone: for example by specifying a Duration of 10min
and an eager refresh of 9min
, only to later change the Duration
to 20min
and forgetting to update the eager refresh to 18min
(or whatever would be the related new value).
As described at the beginning, the current approach of fail-safe + timeouts may get you the same approach, but it seems to require more mental gymnastics.
Finally, there may be a use-case for using the 3 features together: eager refresh + fail-safe + timeouts, which may be nice.
Of course in a highly concurrent scenario, only one request would start an eager refresh: this is the same Cache Stampede prevention that happens when normally running a factory to refresh the data after expiration, so the same mechanism should also be used here for the same reasons.
Additionally, during an eager refresh the underlying cache entry is not yet expired, so only one call should obtain the mutex and start the background refresh, while all the others should simply skip it: this can be done by trying to acquire the mutex with a timeout of zero. This would allow only the first request arrived after the passing of the eager refresh to get the mutex and start the background refresh, while all the other requests would simply see that the mutex is already "taken" and move on by using the current value.
Some benchmarks should be made to ensure that the performance does not degrade (or anyway, at least in a reasonable way) between a series of calls with and without eager refresh enabled, in each phase (before the "eager threshold" is hit, and after that).
Finally it should be safe to hit the actual expiration even when an eager refresh is still running, and maybe decide what should happen in such an edge case.
The next version of FusionCache will have an important new component: a backplane.
In the design phase and while discussing it with the community (@jasenf and @sanllanta in particular) a question arose: would it be possible to use just a memory cache + a backplane, without having a distributed cache?
We'll use this issue to discuss the problems with this approach, tentative ideas around it and potential solutions.
So the question is: is this possible?
No.
Well, maybe. With some limitations, but maybe. See this thread.
This idea in fact seems like a nice one!
In a multi-node scenario we would like to use only the memory cache on each node + the backplane for cache synchronization, without having to use a shared distributed cache.
Technically you can in fact setup a FusionCache instance without a distributed cache but with a backplane.
But don't do it ⛔.
You see, the problem with this approach is that it will continually evict cache entries, all the time, on all nodes basically overloading your datasource (eg: the database).
This is because every time a cache is set or removed, it will automatically send eviction notifications on the backplane, which in turn will evict local caches on the other nodes, which in turn - the next time someone asks for that same cache entry on those other nodes - will set the cache, sending notifications and so on, going on like that forever.
To better illustrate this scenario imagine a multi-node setup with 3 nodes (N1, N2, N3), each with a memory cache, initially empty:
N1
calls GetOrSet
for "product/123"
N1
) for 5 min
, and notify everybody about the changeN2
and N3
receive the notification, and evict their local cache for "product/123"
N2
calls GetOrSet
for "product/123"
N2
) for 5 min
, and notify everybody about the changeN1
and N3
receive the notification, and evict their local cache for "product/123"
As you can see this basically means that every time somebody directly SET a cache entry (eg: when calling the Set
method) or call a GetOrSet
(logically a GET + a SET) the entry will be evicted, rendering the entire thing useless.
One idea we may think about is to send notifications only after a Remove
call or a Set
call, and not when calling GetOrSet
: the problem now is that, apart from being not logical (a GetOrSet
is a GET + SET and the SET part is logically the same as the one in a Set
method call), it would also end up NOT keeping all the caches synchronized.
Why? Let us follow this scenario:
N1
calls GetOrSet
for "product/123"
N1
) for 5 min
, without notifying everybody about the change (since it is not a Set
method call)N2
calls GetOrSet
for "product/123"
N2
) for 5 min
, without notifying everybody about the change (since it is not a Set
method call)5 min
, N1
and N3
will see different versions of "product/123"
As you can see there's no way to escape this, at least that I'm aware of.
Finally, in theory we may say that if we establish that ALL changes to the data are done via a piece of code that uses FusionCache, and ALL of those changes to the database are ALWAYS followed by a direct Set
call and we ONLY consider the direct Set
calls (+ the Remove
ones) to send notifications then yeah, maybe it should work.
Well... yes, maybe, in theory that would be the case, but it would also be a very a brittle system, IMHO.
Anyway, I'm absolutely open to new ideas or point of views.
If you have a brilliant proposition that works and is not brittle please let me know so we may be able to work something out!
Hi @jodydonetti
firstly thanks for your contribution to the community, we have just started to use FusionCache on production and has been working flawesly since then. We are constantly trying to use it in different case scenarios, like the one I'm exposing to you right now. In our case, we are using the backplane with several nodes synchronized with redis.
We are considering using fusion cache as a deduplicator for streaming messages coming from different sources. Right now the best solution we have came was using the two core methods TryGet and Set in a way that it mimics the sliding expiration, but if the same request happens in different nodes with the same key, the set method will be called twice and the deduplicator won't be able to handle correctly that scenario.
public async Task<bool> IsDuplicated(string deviceName, DateTime timestamp)
{
var key = LocationCacheKey(deviceName, timestamp);
var isProcessed = await _cache.TryGetAsync<int>(key);
if (_logger.IsEnabled(LogLevel.Information))
{
if (isProcessed.HasValue)
{
_logger.LogInformation("Device {Device} location with timestamp {Timestamp} duplicated, refreshing duration", deviceName, timestamp.ToString("O"));
}
else
{
_logger.LogInformation("Device {Device} location with timestamp {Timestamp} not duplicated, processing location and setting the cache", deviceName, timestamp.ToString("O"));
}
}
await _cache.SetAsync<int>(key, 1, EntryOptions);
return isProcessed.HasValue;
}
The GetOrSet method won't be a fit to this case because as per the documentation, this 'problem' also exists. And we also need the "sliding expiration" we mimic using the Set method.
We have considered using just the StackExchange redis client, but we would lost the in memory cache.
The solution that we would like to have is the one I expose below, using the TrySet method. In that case the value won't be set twice because the TrySet would return false/null and the other nodes will know that is a duplicate.
public async Task<bool> IsDuplicated(string deviceName, DateTime timestamp)
{
var key = LocationCacheKey(deviceName, timestamp);
var isProcessed = await _cache.TryGetAsync<int>(key);
if (isProcessed.HasValue)
{
// Refresh duration
_logger.LogInformation("Device {Device} location with timestamp {Timestamp} duplicated, refreshing duration", deviceName, timestamp.ToString("O"));
await _cache.SetAsync<int>(key, 1, EntryOptions);
return true;
}
// Try set the key
var isSet = await _cache.TrySetAsync<int>(key, 1, EntryOptions);
if (isSet)
{
_logger.LogInformation("Device {Device} location with timestamp {Timestamp} not duplicated, processing location and setting the cache", deviceName, timestamp.ToString("O"));
}
else
{
_logger.LogInformation("Device {Device} location with timestamp {Timestamp} duplicated when trying to set the key", deviceName, timestamp.ToString("O"));
}
return !isSet;
}
I know using just the Redis client would be possible because of the SET command which has the NX option
NX -- Only set the key if it does not already exist.
but, since fusion cache is using IDistributedCache, I believe that won't be easy to fit, but I'm making this proposal so you can consider it.
Please let me know if you need more context or I can help you.
Hi.
I've noticed that all my keys are getting prefixed with v1. Why is this happening? Is it possible to remove the v1 prefix?
Hi, if the cache is initally empty, even a fail-safe GetOrSet
still throws an exception, since there is no expired value for the key which can be retrieved. There is a GetOrDefault
, but from the otherwise excellent documentation and the example code alone it is not clear, how to combine these features. Is that really some kind of missing method (like a GetOrSetOrDefault
feature or an overload with a default value) or is this by design? Either way, it would be awesome if this case is mentioned. Thank you very much!
Almost every FusionCache method that act on cache entries - like Set
, Remove
, GetOrSet
, etc - can receive various options via a param of type FusionCacheEntryOptions
which contains things like Duration
, Priority
, FailSafeMaxDuration
and so on.
This is all well and good but in the case of the GetOrSet
method, there could be one more opportunity: the abilty to adapt the caching options to the value about to be cached when returned by the factory, if and when that is called.
For example if our factory retrieves a Product
from a database, the cache duration may be small in case the product has just been created/updated (since it may receive additional updates soon) or very large in case a product is old and no more available to sell (since it most probably won't change anymore).
In this case the information to base the adaptive logic would be inside the product instance itself, making it self-contained.
A different example can be a resource, let's say a News
piece in an online newspaper, gathered not via a database call but via a remote webservice call: in the http response we may have a Cache-Control
header which can drive the cache duration inside FusionCache itself.
In this case the information to base the adaptive logic would NOT be inside the news instance itself, making it not self-contained.
I want to add a way to change the FusionCacheEntryOptions
param passed to a GetOrSet
method call based on the value produced by a factory (or adjacent data, see the previous News example).
There are different ways to achieve this, and I'd like to highlight them, with pros and cons.
💡 NOTE |
---|
Regardless of which proposal will be choosen, new overloads will be added to match the currently available methods, so that method calls in already existing code will not need to be fixed. |
The idea here is to have an additional param of type Action<TValue, FusionCacheEntryOptions>
so to be able to have a way to receive the value produced by the factory + the options param currently being used and change something there.
An example can be something like this:
var product = cache.GetOrSet<Product>(
$"product:{id}",
_ => GetProductFromDb(id),
options => options.SetDuration(TimeSpan.FromMinutes(1)),
(product, options) => {
// IF THE PRODUCT IS EXPIRED -> CHANGE THE DURATION TO 1 HOUR
if (product.ExpirationDate < DateTimeOffset.UtcNow) {
options.SetDuration(TimeSpan.FromHours(1));
}
}
);
👍 PROS:
public static ProductUtils.AdaptCachingOptions(Product product, FusionCacheEntryOptions options)
)👎 CONS:
Cache-Control
header of the http response to drive the cache duration: in the factory we would have access to the http response, before deserializing the response body to a Product instance, but not anymore in the separate lambdaℹ NOTES:
null
which would indicate not to change the options, basically disabling adaptivity (and the extra allocation)The idea is to change the common signature for the factory param, from:
Func<CancellationToken, Task<TValue>>
to:
Func<CancellationToken, FusionCacheEntryOptions, Task<TValue>>
Again, new overloads to map the currently existing factory signature would be created, so existing code would continue working as usual.
👍 PROS:
👎 CONS:
(token, options) => factory(token)
) and there it is another invisible allocationℹ NOTES:
[Obsolete]
with an explanation message to use the new signature, and just move on. But I'm not that sure this is the right way to handle this situation, I'm still thinking about thisIObjectWithFusionCacheAdaptiveOptions
interfaceThis is not a substitute for any of the 2 previous proposals, but may be a nice addition.
The idea is to have a new IObjectWithFusionCacheAdaptiveOptions
interface modeled like this:
public interface IObjectWithFusionCacheAdaptiveOptions
{
void AdaptFusionCacheEntryOptions(FusionCacheEntryOptions options);
}
and it would be used, following the Product example above, like this:
public class Product
: IObjectWithFusionCacheAdaptiveOptions
{
[...]
public DateTimeOffset ExpirationDate { get; set; }
[...]
public void AdaptFusionCacheEntryOptions(FusionCacheEntryOptions options)
{
// IF THE PRODUCT IS EXPIRED -> CHANGE THE DURATION TO 1 HOUR
if (ExpirationDate < DateTimeOffset.UtcNow) {
options.SetDuration(TimeSpan.FromHours(1));
}
}
}
👍 PROS:
👎 CONS:
ℹ NOTES:
Special care must be put into NOT changing a FusionCacheEntryOptions
instance when that is not the intention.
What do I mean?
A good way to save some allocations is to define and reuse the same options object over and over again, passing it to the same method call all the time.
Example:
public static class CachingDefaults
{
public static FusionCacheEntryOptions ProductOptions = new FusionCacheEntryOptions()
{
Duration = TimeSpan.FromMinutes(1),
IsFailSafeEnabled = true,
FailSafeMaxDuration = TimeSpan.FromHours(24),
FailSafeThrottleDuration = TimeSpan.FromSeconds(30),
FactorySoftTimeout = TimeSpan.FromMilliseconds(100)
};
public static FusionCacheEntryOptions CategoryOptions = new FusionCacheEntryOptions()
{
Duration = TimeSpan.FromMinutes(10),
IsFailSafeEnabled = true,
FailSafeMaxDuration = TimeSpan.FromHours(12)
};
}
[...]
var product = cache.GetOrSet<Product>(
$"product:{42}",
_ => GetProductFromDb(42),
CachingDefaults.ProductOptions // THIS SAME INSTANCE WILL BE PASSED EVERY TIME
);
var category = cache.GetOrSet<Category>(
$"category:{42}",
_ => GetCategoryFromDb(42),
CachingDefaults.CategoryOptions // THIS SAME INSTANCE WILL BE PASSED EVERY TIME
);
In this case the productOptions
or the categoryOptions
instances will be reused, greatly reducing the number of allocations.
But if we later decides to let the options be adaptive (in any of the proposed ways) and forget to create a new FusionCacheEntryOptions
object every time, the options instance passed in would be changed, even for the following requests. A practical example is that the ProductOptions is set with a Duration of 1 min, but as soon as an old product will get requested with adaptive caching, the Duration would become 1 hour for all subsequent requests.
To avoid this an idea may be to just duplicate the options object, only in case adaptive caching needs to be applied (which would be possible btw only with Proposal 01, since the adaptive logic is explicitly in a separate param). This would solve the problem, but it would needlessly duplicate those options even in case the instance passed in is already a "throwaway" one. Ideally I should find a way to correctly identify which are reusable and which are not, but I need to experiment a little bit because I can already see some troubling edge cases.
Any suggestion or observation would be really, really appreciated 🙏.
I'm thinking about using the new type MaybeValue<T>
- introduced for the new failSafeDefaultValue
in the GetOrSet
method - also as the return value of the TryGetResult<T>
method.
The basic rationale behind this is unification of the "maybe a value, maybe not" concept.
The main difference from now would be that TryGetResult<T>
implicitly converts to a bool
(the idea was to use that in an if
statement) whereas MaybeValue<T>
implicitly converts to/from a T
value, much like a Nullable<T>
. But, just like a Nullable<T>
, it does have an HasValue
property for easy checks, so it should not be that problematic.
In practice, before we should have done one of these:
// ASSIGNMENT + IMPLICIT CONVERSION TO bool, KINDA WEIRD
TryGetResult<int> foo;
if (foo = cache.TryGet<int>("foo"))
{
var value = foo.Value;
}
// ASSIGNMENT DONE BEFORE
var foo = cache.TryGet<int>("foo");
if (foo)
{
var value = foo.Value;
}
whereas now we can do one of these:
var foo = cache.TryGet<int>("foo");
if (foo.HasValue) // EXPLICIT .HasValue CHECK
{
// EXPLICIT .Value USAGE
var value = foo.Value;
// OR IMPLICIT CONVERSION, LIKE WITH A Nullable<T>
var value = foo;
}
// OR .GetValueOrDefault(123) USAGE, LIKE A Nullable<T>
var foo = cache.TryGet<int>("foo");
var value = foo.GetValueOrDefault(123);
Even though FusionCache is still in the 0.X
version phase (so can have breaking changes), just in case of existing usage of the TryGet[Async]
method in someone else's code I can easily add the old Success
prop marked as Obsolete
with an explanation on how to use the new thing. Maybe some pieces of code with a direct check in an if
statement would stop compiling, but that would be resolved by simply adding .HasValue
.
Opinions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.