Rethinking a way to handle API response

[vuex-orm]

Rethinking a way to handle API response

[kiaking]

Here are some more organized thoughts on this topic.

Background

Currently, many API requesting libraries offer a smooth caching experience—for example, swrv, vue-query, etc. So when you want to query server-side data, you don't have to worry about calling it too many times.

However, the response data from the API is cached based on the "api key". For example, if we were to query user data, in swrv we do it like this.

const { data, error } = useSWRV('/api/user/1', fetcher)

Here, the '/api/user/1' is the "key". If we use useSWRV composable somewhere else with the same '/api/user/1' key, it wouldn't make a request but retrieves data from the cache.

While this is wonderful, the problem is that the API may return the exact same "data", on a different "key." For example, if we're logged in to the system as a user of id 1, api call to /api/users/1 and /api/me would return the same user data. But because "data" is not the key to the cache, swrv wouldn't know the "relation" of /api/users/1 and /api/me.

It's OK to call api twice. We probably should because even though the returned data is the same user, the user's content may differ. For example, /api/me would only return the user's name and age, where /api/users/1 might return more detailed data like profile, phone number, etc.

The problem is when "mutating" the user data.

Let's say we like to display the logged-in user's name on the global header by fetching data from /api/me, and the user's profile on the profile page using data from /api/users/1. And say on the profile page, we can edit the user's name. If both users on the global header and the profile page are the same, we would like to update the user's name on the global header as well as the profile page.

But it's not easy. Because the API key is different, we must re-execute the api on the global header, but only when the mutated user is the logged-in user. How can we handle this situation?

Maybe, Vuex ORM can nicely handle it.

What is Vuex ORM's responsibility?

Vuex ORM provides fluent querying and mutation to the local Vuex store. This enables us to be able to update all components that referencing the store.

Because Vuex ORM is not about API "key", it handles the "data". It doesn't care where the data came from or what "key" users used to fetch them. It only cares about the actual data.

For Vuex ORM, user data from /api/me and /api/users/1 is the exact same user if they share the same primary key. So in the previous case, if we update users like this:

userRepo.update({ id: 1, name: 'New Name' })

Both user's name on the profile page and global header get updated. Because even though the original data might come from different endpoints ("key"), the actual "data" is the same.

Isn't this sound amazing? If we can nicely combine api caching feature provided by common api libraries and Vuex ORM's "object-relational mapping" feature, we might be able to do pretty cool stuff. For example, in the ideal scenario, every component could only care about itself, call whatever api they want, and never have to worry about re-executing the api on other components when they might share the same data in different "key".

API: Querying data

How about if we provide useData composable. Say we want to fetch a user. The simplest form, using the "fetch" api.

const { data } = useData(User, () => fetch('/api/users/1'))

The first argument is either Model or Repository. The second is any function that returns a promise.

The returned data is ref value will that first be undefined. Then once the api call is done, it will persist the data in Vuex ORM via save method and return computed value using the revive method (both introduced in this PR).

const { data } = useData(User, () => fetch('/api/users/1'))

data.value // <- undefined

watch(data, (d) => {
  d // User
})

So this is how we think. When using this composable in component, we think as:

// I would like to have data of
const { data } = useData(
  User, // User model
  () => fetch('/api/users/1') // with data of this endpoint
)

We may also add an additional query on the 3rd argument.

const { data } = useData(
  User,
  () => fetch('/api/users/1')
  (repo, result) => { // `result` is return value from the 2nd arg
    repo.with('post').where('id', 1).first()
  }
)

With this, we can retrieve user data with posts, even if /api/users/1 would not contain posts. This is very handy if we have a different api for fetching users and posts.

Integrating with other libraries

The API caching should be handled by api libs. If it's axios with a cache module installed, we can simply do this.

const { data } = useData(
  User,
  () => axios.get('/api/users/1') // cached
)

When we use this composable in multiple places, the api request should only happen once and return the same data.

The second argument could also support ref value. This enables us to integrate it with composable libs like swrv.

const { data: response } = useSWRV('/api/users/1', fetch)

const { data: user } = useData(User, response)

In this case, user returned from useData gets filled whenever response.value changes (we'll setup reactivity).

Now we may create a wrapper function on userland to have a more smooth experience.

function useUser(id: number) {
  const { data: response } = useSWRV(`/api/users/${id}`, fetch)
  const { data: user } = useData(User, response)

  return { user }
}

const { user } = useUser(1)

It's up to developers to create integration between api lib and Vuex ORM, but it should be straight forward as you can see.

API: Mutating data

To mutate data, we'll provide useChange composable. By the way, I'm using useData and useChange because other api libs often use ' useQueryanduseMutation`.

const { change } = useChange(User, (id, data) => fetch(`/api/users/${id}`, {
  method: 'POST',
  body: JSON.stringiy(data)
}))

change(1, { name: 'John Doe' }) // Request change

As same as useData, it takes an Promise as the second argument. It acts similarly to useData where it will persist the return value, but the execution will only happen when we call change method.

What does this all mean?

We're essentially treating Vuex ORM as a smarter cache feature for the api response with this approach. With this approach, we can integrate Vuex ORM with almost any kind of api libs. And the benefits are that we don't have to worry about re-executing the api call on mutation. All we need to do is to update Vuex ORM along side with the api call.

[kiaking]

Is save basically a mirror of the previous insertOrUpdate implementation?

Yes, pretty much!

Also, the PK's in the posts result are strings but passed in as integers, is that a typo?

No, it will be strings because it's actually index IDs rather than primary key values 👍

[kiaking]

Additional thoughts. I thought we could return ids of inserted records from existing methods such as insert or update, but I find it's not possible. Because update will not return the correct ids.

For example, when we normalize a following data, where User has many Post.

{
  id: 1,
  name: 'John Doe',
  posts: [
    { id: 1, userId: 1, title: 'Title 1' },
    { id: 2, userId: 1, title: 'Title 2' }
  ]
}

We get this normalized state.

{
  result: '1' // id of user.
  entities: {
    users: {
      1: {
        id: 1,
        name: 'John Doe',
        posts: ['1', '2']
      }
    },
    posts: {
      1: { id: 1, userId: 1, title: 'Title 1' },
      2: { id: 2, userId: 1, title: 'Title 2' }
    }
  }
}

Now when we use update, and let's say if the Post with id of 1 doesn't exists in the store, it will not persist the data. So, when we use the revive method with the above normalized state, it fails because it can't fetch back post id of 1.

Well, we could update the above normalized state based on the result of the function such as update, but, I think it would be better to have dedicated method that always saves data to the store. Yes, insertOrUpdate method.

How about we just name it save method. And let users use this save method almost all of the time.

const result = store.$repo(User).save({
  id: 1,
  name: 'John Doe',
  posts: [
    { id: 1, userId: 1, title: 'Title 1' },
    { id: 2, userId: 1, title: 'Title 2' }
  ]
})

/*
  {
    id: 1,
    posts: ['1', '2']
  }
*/

[cuebit]

Is save basically a mirror of the previous insertOrUpdate implementation?

Also, the PK's in the posts result are strings but passed in as integers, is that a typo?

[kiaking]

Closing the thread for now to reduce the confusion. I think the overall idea is good, but it might need more thinking and if I ever do, I would probably open a new issue.