Sagas should rather be totally autonomous

[redux-saga]

Sagas should rather be totally autonomous

[slorber]

@timdorr it is not because it's written in the doc in a simple way to make it easy to understand for event-sourcing new-comers that is it an absolute truth :)

Browsers and backend systems are not so different: they manage state. The main difference is that the frontend receives the user intent synchronously so it generally handles that intent based on an up-to-date state. I'm pretty sure frontend and backend will be more and more similar in the future, and don't forget than @gaearon has also been influcend by the Turning the database inside out talk which is about backend primarily :)

Your UI is simply a function on state, i.e. React(state) = view. Replaying an event log to compute that view doesn't make any sense. You should let your state container (Redux) handle that computation of final state so that React can render it.

Absolutely not. It does make a lot of sense and it permits to implement features like time-travel. You know what, backend guys are doing time-travel for decades :) The saga concept itself is from the backend / eventsourcing world.

Instead of thinking React(state) = view, you should consider React(Redux(eventlog)) = view

If Redux is claimed to be the source of truth it is probably to be simpler to understand, but Redux treats itself the event-log as the source of truth. The beauty of this is that you can use this event log for many other usages:

• You can sync 2 Redux stores that are on 2 different browser (for example imagine someone taking remote control of your local redux app for assistance...)
• You can project that event log in other systems
• You can send that event log to the backend and compute bigdata statistics based in UI usage
• so many possibilities...

Absolutely! You may have non-visible state that needs to be managed. Take analytics data for instance. You might collect that into your state to occasionally ship back to your server.

Please tell me any drawback of storing these statistics outside of the Redux tree if they are not displayed in the UI?

If you ship the event log to the server directly instead of computing the analytics on the client, you are still able to implement reducers in the backend to compute these analytics (in the language of your choice btw!). You never loose any data and can replay that event log 1 year later, on another browser or a backend if you want to. (Shipping the event log still has a network cost however...)

If you have an app in production for 1 year, and you want to introduce a new analytics that count the TodoCreated actions for a given user. If you compute the analytics on the frontend, then you will start with a counter value = 0. If you ship the event log to the backend, and want to introduce that statistic, you have 1 year of historical event-log to compute a counter value: you don't start at 0 but you have your new stat instantatenously!

Redux is just a system to project an event-log (source of truth) into an easy-to-comsume state (projection of source of truth) for view applications like React. Nothing forces you to use a single projection at a time of your event log.

[slorber]

@youknowriad

@slorber May be It is because I have not the necessary backend knowledge you have to consider the event log as the source of truth for frontend application. I think I need to see an implementation of this to have a precise idea about this.

Just look at this and it will click: http://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/

Redux suggest the state of the store is this source of truth and It works quite well for any frontend application.

The source of truth for React is the Redux store.
You can put the Redux state into React and it computes the same view.

The source of truth for Redux is the event log.
You can put the event log into Redux and it will computes the same state.

The source of truth for the event log is the dom events happening on a given UI.
You can trigger the dom events on the same UI and it will produce the same event log.

---

The thing is some source of truth seems to actually be derived from a former source of truth.

For a long time on the backend we considered the database (ie MySQL / MongoDB) as the source of truth (most of us still do actually). While even internally these databases are using event-logs as the source of truth for technical reasons like replication: isn't that funny?

---

You have to consider the source of truth according to what you will want to record / replay and how the derived source of truth should behave after code change.
The history of things you record should be immutable: you should rather not change the past, but you can eventually change your interpretation of the past: this is hot reloading.

state sourcing

If you consider state as a source of truth, then you can record state and replay them in the same React app. Here's a video i've done some time ago. If you record only state, you don't have the event log and then if you change a reducer the state history will remain the same: you can only hot-reload React views

event sourcing

If you record events (or actions) of what has happened, then you can replay these events into redux reducers to recompute the whole history of states, and replay this state history into React to show something. If you change a reducer, then you can compute a new history of state: this is how Redux hot reload works. However you can not modify the event log.

command sourcing

If you choose to record the commands (ie the user intent) then you can recompute an event log from the intent log, and then a state log from the event log. The intent is generally translated to events in actionCreators and jsx views where we transform low-level dom-events to Redux actions.

For example imagine a video game in React. When the user press left arrow, an event "WentLeft" is fired. If you hot-reload the JSX or actionCreator so that when left arrow is pressed it actually fires a "Jump", and you time-travel with Redux, you will see that in your history you still have "WentLeft" because Redux hot reload does not affect the past.

Command sourcing would permit to hot-reload the interpretation layer too and would replace the "WentLeft" by a"Jump" in the event log before computing the state log and before injection states in React. In practice it has not much interest and may be more complicated to do (not sure but maybe ELM is doing this no?)

See also
http://stackoverflow.com/questions/9448215/tools-to-support-live-coding-as-in-bret-victors-inventing-on-principle-talk/31388262#31388262

[gaearon]

It's storing a log of events (actions) that happened from the bootstrap of the application (or from the backend for isomorphism first loading), and generate the state (redux state and sagas state) by "playing" those events.

Tangentially this is exactly how Redux DevTools works. It uses Redux to store the event log itself. Inception.

[youknowriad]

Hello,

There are some problems I see here :

• This breaks the redux principle of single source of truth
• It could make devTools harder to implement (time travel)
• Personally, I dont' see the Redux State as an UI state but more as the Application state, The UI state is extracted from the application state using selectors. I've found your approach of using the getState in the root Saga only quite nice, somehow equivalent to the "smart/dump React components" approach of using the redux State.
• What about isomorphic support ?

Thanks

[slorber]

@youknowriad

Actually I don't know what is the claim of Redux, but for me Redux has never been the source of truth. The source of truth is the event log. Redux reducers permit to project that event log into a JS object usable by views but still the event log is the source of truth. You can see it because it's the event log used during devtools time travel, and not store state snapshots. If an event is fired but no reducer use it, then you still have the event, even if you don't see any change in your Redux store: the events are your data, not the store.

During time-travel, the Saga should not emit new events because it can't modify the history. This means that if you follow this reco then time-travel will continue to work like before. For sagas hot reloading, the saga can be update, and recompute its state from the event log with new logic, but should rather fire new events only in the future.

For App state vs UI state. As I said, the real source of truth is the event log. It is both your app and UI state. It does not matter how much time you project this event log into different shapes. In the backend we very often do that to construct very business-specific indexes. The event log is in some global Kafka log and multiple computers listen to that event log, every one computing a specific view. Some projects into Cassandra for timelines, some for Oracle for business analysis, some to ElasticSearch for full-text search... You can project the event log to as many places as you want to according to your needs. You can project this event log to 2 or * redux store instances. They don't even have to use the same root reducer. Your current redux store is just ONE possible projection of your app state, but there are an infinite number of other possibilities.
See http://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/
You shape the projection according to your query needs. We are using immutable data in frontend now mostly because it permits to leverage good performances easily with shouldComponentUpdate. However for state that is not passed to any React view (like saga state), using immutable data structures does not benefit from this performance advantage. I'm not saying we should necessarily use mutable data structures but still it's something to consider: it is really worth projecting everything in a Redux store and immutable data structures if they are not even rendered? or should we do like the backend, and choose the best storage system that solves our problem?

About isomorphism / universal: I'm not sure to understand what your concerns are.

[timdorr]

Actually I don't know what is the claim of Redux, but for me Redux has never been the source of truth.

Well, then you're using it wrong 😄 It's the first of Redux's three core principles.

You can project this event log to 2 or * redux store instances.

There is only one view in your application. You don't need multiple stores. That just needlessly complicates your application. I think you might be influenced a bit too much by backend systems. Browsers and Javascript are a very different paradigm.

Your UI is simply a function on state, i.e. React(state) = view. Replaying an event log to compute that view doesn't make any sense. You should let your state container (Redux) handle that computation of final state so that React can render it.

It is really worth projecting everything in a Redux store and immutable data structures if they are not even rendered?

Absolutely! You may have non-visible state that needs to be managed. Take analytics data for instance. You might collect that into your state to occasionally ship back to your server.

[youknowriad]

@slorber May be It is because I have not the necessary backend knowledge you have to consider the event log as the source of truth for frontend application. I think I need to see an implementation of this to have a precise idea about this.

But what I'm certain of is that we need to have only one single source of truth for the entire frontend application. Redux suggest the state of the store is this source of truth and It works quite well for any frontend application.

If I understand what you suggest, It's storing a log of events (actions) that happened from the bootstrap of the application (or from the backend for isomorphism first loading), and generate the state (redux state and sagas state) by "playing" those events. While I understand that storing those events is helpfull when implementing TimeTravel (debug features), I think that It may overcomplicates things compared to juste using getState on root Components and root Sagas to achieve quite the same thing.

[youknowriad]

@slorber you were right, I took a look at the talk, and I got your point now.

What I think now is that your approach is nice, but It can't fit in Redux (at least for now) because Redux does not store the event log (It does in the dev tools), It stores the current state. Even If it has all the necessary logic to do the job (dispatch, subscribe and state that could be equal to array of actions). The main dispatcher (which dispatch all actions) needs to be separated from the projection using reducers of those actions. Somehting like that

// just a handy way to create a dispatcher
const createDispatcher = () => {
  let listeners = [];

  return {
    subscribe: (listener) => {
      listeners.push(listener);
      return () => {
        listeners = listeners.filter(l => l !== listener);
      }
    },
    emit: (state) => {
      listeners.forEach(listener => listener(state));
    }
  }
};

// create the log store 
const createActionStore = (initialActions) => {
  let actions = initialActions;
  const actionDispatcher = createDispatcher();

  return {
    subscribe: (listener) => actionDispatcher(listener),
    dispatch: (action) => {
      actions.push(action);
      actionDispatcher.emit(action);
    }
  }
};

// Redux Store ?
const createUIStore = (actionStore, reducer) => {
  const uiDispatcher = createDispatcher();
  let state = reducer({ type: 'INIT' });
  actionStore.subscribe(action => {
    state = reducer(state, action);
    uiDispatcher.dispatch(state);
  });

  return {
    subscribe: (listener) => actionDispatcher(listener)
  }
}

// Sagas
const initSagas = (actionStore, sagas) => {
  let sagasEngine = {
    handle: () => {
      // use sagas
      // What's currently done in the redux-sagas middleware comes here
    }
  };
  actionStore.subscribe(action => {
    sagasEngine.handle(action);
  });
}

// Boostraping
const intialActions = [];
const reducer = (state, action) => state;
const sagas = [];
const actionStore = createActionStore(intialActions);
const uiStore = createUIStore(actionStore, reducer);
initSagas(actionStore, sagas);

Well, I dont know really what to think of this. I clearly see your point about decoupling the sagas logic from any store, but in the same time I find it really easy to reason about an application where all my state leaves in one place as in Redux.

Interesting discussion btw 👍

[slorber]

@youknowriad I'm not really sure to understand what we are discussing here and what you try to do with this implementation :) Redux already provide the devtools to record and replay events so it's not really worth it to record them another time one step ahead (unless you want to be able to replay them in another system than Redux but you could easily write a store enhancer that record dispatched actions)

Initially I just wanted to be sure that the Saga would be able to manage its own state without having to query Redux's getState().

[slorber]

@yelouafi after thinking about it a bit it seems to be a non issue because in your examples you have shawn that a saga could be living the whole app lifetime with a while (true), and that it could use local variables outside of the loop as state so basically it seems to me that getState is not a requirement to implement stateful sagas

Like the authenticate example: you have to know when the user is connected or disconnected to perform the appropriate effects, however it did not require any use of getState at all.

However, the caching system in the "real world" example would probably be harder to write without getState: https://github.com/yelouafi/redux-saga/blob/master/examples/real-world/sagas/index.js
I would like to see how you could handle that without getState :)

[youknowriad]

@slorber What I was trying to say in my implementation is That if Redux is just a projection of the event log for UI, then It should not be able to record and play the events but instead subscribe to those events and update the UI store, and Sagas as well (I mean Redux and Sagas are totally decoupled).

I know we can achieve the same using Redux Store Enhancer, It is just not clear enough. It is not so important btw.

[slorber]

Imo Redux is a framework that handles already the publishing, record/replay and projection of events.

It could be splitted into 3 different decoupled parts but it would make it harder to understand for newcomers that already have to understand functional programming. If you have 3 libs that 99% of people already always use together it's not a big deal to couple them in a single framework if you still allow the 1% the freedom to replace what they want.

A more complete and opiniated framework is easier to understand, and Redux can still allow you to add an event log on top of it or eventually plug another kind of devtool, but yes you have to understand the inner-working for that :)

[dts]

A canonical example of a JS application where multiple replicas might be kept in sync with streams of actions would be a browser extension with a background page and content scripts, or a web page with web workers. In these cases, the way for these contexts to communicate is through actions, and sending "diffs" or copies of the state is less advantageous. These all also have the additional benefit of only caring about certain subsets of actions - a content script, for example, only subscribes and and processes actions relating directly to it, whereas the redux context for the background page evaluates and manages the ones that it needs to (which in most cases is probably all of them).

[yelouafi]

However, the caching system in the "real world" example would probably be harder to write without getState: https://github.com/yelouafi/redux-saga/blob/master/examples/real-world/sagas/index.js
I would like to see how you could handle that without getState :)

Yes, that's because the loadUser is forked and not called.

A possible solution is to declare loadUser inside watchLoadUserPage so it can access its local state (same principle can be applied to loadStarred)

function* watchLoadUserPage() {
  let userCache = {}
  while(true) {
    const {login, requiredFields = []} = yield take(actions.LOAD_USER_PAGE)
    yield fork(loadUser, login, requiredFields)
  }

  // load user unless it is cached
  function* loadUser(login, requiredFields) {
    const user = userCache[login]
    if (!user || requiredFields.some(key => !user.hasOwnProperty(key))) {
      userCache[login] = yield call(fetchUser, login) // update the cache
    }
  }
}