Giter Site home page Giter Site logo

esm-hmr's Introduction

ESM Hot Module Replacement (ESM-HMR) Spec

Authors: Fred K. Schott (Snowpack), Jovi De Croock (Preact), Evan You (Vue)
Status: Archived, no longer under development.

Hot Module Replacement (HMR) lets your browser live-update individual JavaScript modules in your application during development without triggering a full browser reload or losing the current web application state. This speeds up your development speed with faster updates on every change.

Web bundlers like Webpack, Rollup, and Parcel all implemented different, bundler-specific HMR interfaces. This makes it hard to share HMR integrations across dev environments. As a result, many framework integrations like React Fast Refresh and Preact's Prefresh need to be rewritten for every bundler that they'd like to support. See:

ESM-HMR is a standard HMR API for ESM-based dev environments. The rise of bundle-free development creates the opportunity for a common, standard HMR API built on top of the browser's native module system. ESM-HMR is built for the browser's native module system, so it can be used in any ESM-based dev environment.

Who's Using ESM-HMR?

What's in This Repo?

  1. esm-hmr/client.js - A client-side ESM-HMR runtime.
  2. esm-hmr/server.js - A server-side ESM-HMR engine to manage connected clients.
  3. An ESM-HMR spec to help your write your own client/server pieces. (coming soon)

Usage Example

export let foo = 1;

if (import.meta.hot) {
  // Receive any updates from the dev server, and update accordingly.
  import.meta.hot.accept(({ module }) => {
    try {
      foo = module.foo;
    } catch (err) {
      // If you have trouble accepting an update, mark it as invalid (reload the page).
      import.meta.hot.invalidate();
    }
  });
  // Optionally, clean up any side-effects in the module before loading a new copy.
  import.meta.hot.dispose(() => {
    /* ... */
  });
}

ESM-HMR API Overview

All ESM-HMR implementations will follow this API the behavior outlined below. If you have any questions (or would like clarity on some undefined behavior) file an issue and we'll take a look!

import.meta.hot

if (import.meta.hot) {
  // Your HMR code here...
}
  • If HMR is enabled, import.meta.hot will be defined.
  • If HMR is disabled (ex: you are building for production), import.meta.hot should be undefined.
  • You can expect your production build to strip out if (import.meta.hot) { ... } as dead code.
  • Important: You must use the fully expanded import.meta.hot statement somewhere in the file so that the server can statically check and enable HMR usage.

Note: import.meta is the new location for module metadata in ES Modules.

import.meta.hot.accept

accept()

import.meta.hot.accept();
  • Accept HMR updates for this module.
  • When this module is updated, it will be automatically re-imported by the browser.
  • Important: Re-importing an updated module instance doesn't automatically replace the current module instance in your application. If you need to update your current module's exports, you'll need a callback handler.

USE CASE: Your module has no exports, and runs just by being imported (ex: adds a <style> element to the page).

accept(handler: ({module: any}) => void)

export let foo = 1;
import.meta.hot.accept(
  ({
    module, // An imported instance of the new module
  }) => {
    foo = module.foo;
  }
);
  • Accept HMR updates for this module.
  • Runs the accept handler with the updated module instance.
  • Use this to apply the new module exports to the current application's module instance. This is what accepts your update into the the running application.

This is an important distinction! ESM-HMR never replaces the accepting module for you. Instead, the current module is given an instance of the updated module in the accept() callback. It's up to the accept() callback to apply that update to the current module in the current application.

USE CASE: Your module has exports that need to be updated.

accept(deps: string[], handler: ({deps: any[]; module: any;}) => void)

import moduleA from "./modules/a.js";
import moduleB from "./modules/b.js";

export let store = createStore({ a: moduleA, b: moduleB });

import.meta.hot.accept(
  ["./modules/a.js", "./modules/b.js"],
  ({ module, deps }) => {
    // Get the new
    store.replaceModules({
      a: deps[0].default,
      b: deps[1].default,
    });
  }
);

Sometimes, it's not possible to update an existing module without a reference to its dependencies. If you pass an array of dependency import specifiers to your accept handler, those modules will be available to the callback via the deps property. Otherwise, the deps property will be empty.

USE CASE: You need a way to reference your dependencies to update the current module.

dispose(callback: () => void)

document.head.appendChild(styleEl);
import.meta.hot.dispose(() => {
  document.head.removeChild(styleEl);
});

The dispose() callback executes before a new module is loaded and before accept() is called. Use this to remove any side-effects and perform any cleanup before loading a second (or third, or forth, or...) copy of your module.

USE CASE: Your module has side-effects that need to be cleaned up.

decline()

import.meta.hot.decline();
  • This module is not HMR-compatible.
  • Decline any updates, forcing a full page reload.

USE CASE: Your module cannot accept HMR updates, for example due to permenant side-effects.

invalidate()

import.meta.hot.accept(({ module }) => {
  if (!module.foo) {
    import.meta.hot.invalidate();
  }
});
  • Conditionally invalidate the current module when called.
  • This will reject an in-progress update and force a page reload.

USE CASE: Conditionally reject an update if some condition is met.

import.meta.hot.data

export let foo = 1;

if (import.meta.hot) {
  // Recieve data from the dispose() handler
  import.meta.hot.accept(({ module }) => {
    foo = import.meta.hot.data.foo || module.foo;
  });
  // Pass data to the next accept handler call
  import.meta.hot.dispose(() => {
    import.meta.hot.data = { foo };
  });
}
  • You can use import.meta.hot.data to pass data from the dispose() handler(s) to the accept() handler(s).
  • Defaults to an empty object ({}) every time an update starts.

Prior Art

This spec wouldn't exist without the prior work of the following projects:

esm-hmr's People

Contributors

bouzuya avatar fredkschott avatar moonball avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

esm-hmr's Issues

Support: React Fast Refresh

/cc @gaearon
Context: facebook/react#16604 (comment)

I'm expecting this to be the hardest use case to support, given React's CJS reliance. But, Snowpack can now install React as ESM, so maybe we're not as far away from supporting Fast Refresh as I'd thought.

@gaearon would love your feedback on this one. Based on your code snippets in that thread, it looks like your HMR API needs are actually pretty simple. I only see use of module.hot.accept() without any arguments. It appears that most of the complexity lives in properly connecting react-refresh/runtime into your app, which is outside the scope of this spec.

Any specific HMR API requirements needed other than import.meta.hot.accept()? Any more detailed implementations that you'd recommend checking out?

The main different between ESM-HMR and bundler-specific HMR APIs is that the first loaded module is the only copy of that module ever connected to the app. Instead of having new modules replace old ones, the import.meta.hot.accept(({module}) => ...) method is always responsible for updating the original module using the exports of the updated module.

provide access to the module being replaced

Currently with snowpack there is no way to get runtime access to the module where the js reside.

So if the import.meta.hot.accept could also provide the previousModule as argument to the callback, this would allow the reload to perform logic based on the previous module

like for example

 if (import.meta.hot) {
        import.meta.hot.accept(({module, previousModule}) => {
            for (const field of Object.keys(module)) {
                const newPrototype = Reflect.getPrototypeOf(module[field]);
                Reflect.setPrototypeOf(previousModule[field], newPrototype);
            }
       });
}

`bubbled` info in accept handlers

I've learned empirically that it can be useful to have a way to tell, from an accept handler, whether it is called because of a direct update hitting the auto accepting module, or if has bubbled from a dependency.

In rollup-plugin-hot, it's implemented as a bubbled argument that is passed to the handler.

The practical use case is CSS injection in svelte-hmr. When a Svelte component is modified, it checks whether only the CSS part has actually changed and, in this case, it does a softer CSS injection -- which is some orders of magnitude safer than recreating stateful JS components and tinkering with DOM elements.

However, it can only do that safely if it is sure that the update came from the component's file itself, and not one of its dependency. If the update has bubbled from a dependency, then the module still need to be reloaded to reflect the change.

Feature: Dependency Accept Handling

Yeah, so here's a use case we've had quite often for the whitelist usage (pseudo code based on Vuex usage, but applies to Redux as well for store.replaceReducer):

// store.js
import moduleA from './modules/a'
import moduleB from './modules/b'

export const store = createStore({
  modules: {
    a: moduleA,
    b: moduleB
  }
})

import.meta.hot.accept(['./modules/a', './modules/b'], ([newA, newB]) => {
  store.replaceModules({
    a: newA,
    b: newB
  })
})

The key here is the store is exported and used by the application. Preserving the current store.js instance is important - otherwise, you would create a new store but the app will still be using the old one. And with your current implementation, it seems a bit awkward to force rerender the app with a new store provider from the hot callback of store.js.

Originally posted by @yyx990803 in #7 (comment)

Trouble staying on the same route

Hi! Thanks for the hmr module, super cool.

I'm having trouble staying on the same route. What I mean is if I'm on /product and edit some code & save. I end up back at the root /.

I'm using plain JS with pushState + CustomEvent for location changes and onpopstate for forward/back buttons.

My app looks like

const App = async () => {

  const pages = [
    { path: /^[/]$/, load: async () => (await import('./home')).default },
    { path: /^[/]product$/, load: async () => (await import('./product')).default }
  ];

  onLocationChanged(async () => {
    const location = matchLocation(pages);
    const content = await location.page.load();

    render(content(), window.document.body);
  });

  replaceLocation('/');
};

export default App;

if (import.meta.hot) {
  import.meta.hot.accept();
}

How does browser swap in the new module?

Hey guys, does anyone understand how ES Module's HMR works under-the-hood (for example in Vite or React Fast Refresh)? i.e. How is it able to swap out a modified module at runtime?

Usually, every time you change a module, browser will refetch that module via HTTP.

But after that, how does the browser replace the old module, and have the new code take into effect?

The only thing I found is https://itnext.io/hot-reloading-native-es2015-modules-dc54cd8cca01, is this the solution we're taking?

Support: Servor

/cc @lukejacksonn

I'm not sure if you have any interest in implementing HMR on Servor, but I'd still love to hear your thoughts on this.

Check out the reference implementation in the repo. Feel free to grab that file, it should be all you need on the client-side to add HMR support to Servor. Server-side, you'd only need to add to your /livereload the ability to send down updates of individual files/urls that have changed instead of full "reload" events. The client-side would then decide if a full-page reload was required or if an an individual file update could be handled via HMR.

Further work & notes

I know this project is archived but I wanted to point out some additional work I've done in this space for hot reloading nodejs services: https://github.com/braidnetworks/dynohot

In particular I've explored and implemented support for top-level await, dynamic imports, promise-returning handlers, relinking of imported bindings, and refined the order in which handlers should run.

  • Top-level await doesn't add too much complexity and is generally handled by the host environment.
  • Handling hot reloadable dynamic imports without risk of deadlocking is, I believe, a fancy version of the halting problem. What I've found in practice is that dynamic imports tend to fall into two categories: lazy imports, and conditional imports. In both cases you can more or less treat them as static imports while reloading and have them participate in the ECMAScript cyclic module resolution algorithm for reloading purposes.
  • To that end, modules which participate in a module cycle as described in the specification cannot be accepted unless all members are also accepted. In my implementation I just warn you when you've accepted a cycle member and that it cannot be accepted.
  • All handlers should be able to return promises which will be awaited before continuing the reload or invoking other handlers in the same scope. This one is a no brainer and ensures resources are collected cleanly and exceptions are propagated correctly.
  • Live relinking of imported bindings feels like the way to go. Getting the implementation right is difficult but the developer experience is a lot better. One condition it does create is that any consumers of a self-accepted modules will suddenly have their imported bindings swapped out from under them without any notification channel.
  • accept handlers should be invoked in the order they are defined, but dispose (and probably prune) handlers should be invoked in reverse order. When you get into the nitty gritty of cleaning up complex middleware this shakes out naturally in the same way C++ figured out RAII.
  • Oh yeah prune is definitely good to include. Vite got this one correct, and it has applications on the server side too.

Anyway that's a very abbreviated brain dump of the thoughts I had while building dynohot. Thanks for your work on this repo!

Feature: Event Bubbling

  • Based on behavior outlined here & found in most other HMR implementations.
  • Needed by Prefresh, others.
  • /cc @JoviDeCroock who wanted to implement in Snowpack

Requirements

  • If a changed file has no accept() handler, the client needs a way to "bubble up" the change event to be handled by it's parent(s)
  • If a file has multiple parents, the event must bubble up each parent's chain of imports.
  • If an event bubbles up to a dependency tree entrypoint, trigger a full page refresh.
  • No import.meta.hot API changes, if possible
  • The server cannot scan a file that hasn't been requested by the client (implication: no "full application" scanning step allowed)

The "Let the Server Do All The Work" Proposal

  • No change to the Server->Client interface. Keep existing update message format:

    • {type: 'update': url}.
  • The server stores a representation of your website dependency tree in memory:

    DependencyTree {
      [url: string]: {
        dependents: url[];
        dependencies: url[];
        isHmrEnabled: boolean;
      }
    }
    
  • At server startup, DependencyTree is empty.

  • Every time we serve a file, we scan the JS response and then update the DependencyTree with information about that one file. This never scans more than that one response.

  • Every time a file changes, we call updateOrBubble(file):

    function updateOrBubble(url) {
      node = DependencyTree[url];
      if (node.isHmrEnabled) {
        send to the client: `{type: 'update': url: file}`
      } else if (node.dependents.length > 0) {
        node.dependents.map(updateOrBubble);
      } else {
        // We've reached the top, trigger a full page refresh
        send to the client: `{type: 'reload'}`
      }
    }
    

Open Questions

  • Is it okay to send "update" events as we traverse the DependencyTree, or should we send them all at once in a batch? Webpack seems to send them in batches, but I believe this is more a limitation of bundling and not actually a requirement for good HMR.
  • How do we track our progress through the DependencyTree so that we never call updateOrBubble() twice on the same node (or, would we ever need to?)
  • Make sure that we can handle added/moved/removed imports in the DependencyTree.
  • Make sure that we can handle added/moved/removed files in the DependencyTree.

Reference: Vite

Maybe you are not aware of it, but Vite has a working implementation of ESM-based HMR that covers most of the proposed usage. I think at the very least it would serve as a useful reference for this spec. Being included in the discussion also helps alignment of the API design to avoid potential fragmentation.

Allow Client to Handle Reload

The current client implementation handles reloading by calling window.reload.

https://github.com/pikapkg/esm-hmr/blob/07bf62c2ff783a4ea3319c0b2baa3351b65c073c/src/client.ts#L16-L18

This doesn't make sense for WebExtensions where reloading is done by calling runtime.reload. I'm sure there are other contexts where you'd want to customize the reload behavior also (electron?).

It would be great if there was an API to customize the implementation of this function.

Maybe something along the lines of

import.meta.hot.registerReloadHandler(() => {
  // do your reloading here
  browser.runtime.reload()
});

The default registered reload handler could be the location one to preserve existing behavior.

Feature: Error Reporting

We should support an "error" event type so that the server can notify the client whenever an error occurred at the build stage.

Current Behavior

  • A file fails to build
  • The developer gets a blank screen with a "couldn't load module"
  • The developer has to flip back to the dev server terminal window to see what broke

Expected Behavior

  • A file fails to build
  • The developer is notified directly in the browser, either via a console.error() message or an overlay (up to the implementer).

HMR bubbles up all the way to the entrypoint on simple changes

We're seeing a lot of page reloads and we don't know what exactly is causing that:

[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/components/foo/foo.js", bubbled: false}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/asd.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/foo.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/routes.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/meow.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/index.js", bubbled: true}
[ESM-HMR] message: reload
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/a.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/b.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/c.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/d.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/e.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/f.js", bubbled: true}
[ESM-HMR] message: update {type: "update", url: "/_dist_/qaz/bar/containers/views/x.js", bubbled: true}

We're using snowpack 2.14.3 extending @snowpack/app-scripts-react.

These changes look relatively simple (like changing the text in an h1 element) and they do not cause full page reloads using CRA.

Unfortunately, I cannot put the codebase online, but do you have any pointers for us what might be causing this?

Proposal: Call dispose() before accept()

From @rixo in FredKSchott/snowpack#331

I think the dispose handler should be called before the accept handler.

Since your accept handler is static, I think it's all the more important to implement a way to pass data from the last version of the module to the next. I strongly recommend that you implement Webpack's dispose data pattern for that.

// preserve existing value or create it if first run
const value = import.meta.hot && import.meta.hot.data.value || []

export const push = (...args) => value.push(...args)

// dispose handler is passed the data object to be mutated before running the new module
import.meta.hot.dispose(data => {
  data.value = value
})

// I'm hot update able!
import.meta.hot.accept()

mobx

Wrapping components to mobx's 'observer' (that wrap them in React.memo) prevents hmr to work

    "@snowpack/plugin-react-refresh": "^2.0.3",
    "@snowpack/app-scripts-react": "^1.9.0",
    "mobx": "^5.15.4",
    "mobx-react": "^6.2.3",
    "mobx-react-lite": "^2.0.7",
    "snowpack": "^2.8.0",

decline() vs not accept()

Calling import.meta.hot.decline() clearly indicates that a change to the module should result in a full page reload.

What about a file that doesn't call any APIs on import.meta.hot?

Because HMR usually requires cooperation of the module to work, it seems like a module should by default behave as though it called import.meta.hot.decline(), so that changes to it result in a full page refresh.

HMR bubbling unspecified

Currently the expected bubbling behavior is unspecified.

For example in this scenario:

a.js

import './b.js';
import './c.js';
import.meta.hot.accept(); 

b.js

import.meta.hot.accept();

c.js

// nothing

As far as I know in the current implementations of vite and snowpack an edit for c.js will trigger a reload of a.js but an edit of b.js does not trigger a reload ofa.js.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.