How to use persistent store such as Redis

[graph-gophers]

How to use persistent store such as Redis

[nicksrandall]

As of v5, we only support in memory caches because we cache the Thunk (function), and not the value it holds. The reason for this limitation is because we uses the cache for both caching AND batching items. Eventually, I would like to decouple the batching cache from the value cache because it would allow more flexible caching strategies like the one you are suggesting.

That said, I do have a potential solution that I can offer you. I would suggest that you move your Redis logic into the batchFn so that you can retrieve items in bulk and fall back to more expensive calls only when you have to.

For example you could do something like:

func batchFunc(_ context.Context, keys dataloader.Keys) []*dataloader.Result {
  results := make([]*dataloader.Result, len(keys)) 

  // get what you can from redis
  values, _ := redisClient.MGet(...keys.Keys()).Result()

  // make a list of everything that was not found in redis
  var cacheMisses map[int]string
  for i := range keys {
    if values[i] == redis.Nil {
      cacheMisses[i] = keys[i].String()
    } else {
      results[i] = &dataloader.Result{values[i], nil}
    }
  }

  // get the missing items from more expensive location (like DB)
  for idx, key := range cacheMisses {
    value, err := db.GetValues(key) // Pseudo code!
    redisClient.Set(key, value)
    results[idx] = &dataloader.Result{value, err}
  }


  return results
}

This solution is nice because you can potentially batch both expensive DB calls and Redis calls which, depending on use case, COULD increase performance from one-off get calls.

[simonrycroft]

Just to be clear, I need to store a text representation of the value I'm caching, so I'm trying to extract the value so I can marshal it. The go-redis.Set() func will only accept a string as the value.

[simonrycroft]

That's awesome. Thanks so much for your help! I'll give this a shot.

[nicksrandall]

I'm going to close this ticket. Feel free to re-open if you need more help.