Giter Site home page Giter Site logo

alesgenova / post-me Goto Github PK

View Code? Open in Web Editor NEW
482.0 11.0 13.0 820 KB

๐Ÿ“ฉ Use web Workers and other Windows through a simple Promise API

Home Page: https://www.npmjs.com/package/post-me

License: MIT License

JavaScript 12.25% TypeScript 85.48% HTML 2.27%
communication postmessage worker webworker iframe postmate typescript web-worker javascript concurrency parallel-computing promise

post-me's Introduction

workflow status npm package codecov

post-me

Communicate with web Workers and other Windows using a simple Promise based API

diagram

With post-me it is easy for a parent (for example the main app) and a child (for example a worker or an iframe) to expose methods and custom events to each other.

Features

  • ๐Ÿ” Parent and child can both expose methods and/or events.
  • ๐Ÿ”Ž Strong typing of method names, arguments, return values, as well as event names and payloads.
  • ๐Ÿค™ Seamlessly pass callbacks to the other context to get progress or partial results.
  • ๐Ÿ“จ Transfer arguments/return values/payloads when needed instead of cloning.
  • ๐Ÿ”— Establish multiple concurrent connections.
  • ๐ŸŒฑ No dependencies: 2kb gzip bundle.
  • ๐Ÿงช Excellent test coverage.
  • ๐Ÿ‘ Open source (MIT)

Demo

In this live demo the main window communicates with a web worker and an iframe (source).

Content:

  1. Install
  2. Basic Usage
  3. Typescript Support
  4. Other Uses
  5. Callbacks as parameters
  6. Transfer vs Clone
  7. Debugging
  8. Parallel Programming
  9. API Documentation
  10. References

Install

Import post-me as a module:

npm install post-me
import { ParentHandshake } from 'post-me';

Import post-me as a script:

<script src="https://unpkg.com/post-me/dist/index.js"></script>

<script>
  const ParentHandshake = PostMe.ParentHandshake;
</script>

Usage

In the example below, the parent application calls methods exposed by the worker and listens to events emitted by it.

For the sake of simiplicity, only the worker is exposing methods and events, however the parent could do it as well.

Parent code:

import { ParentHandshake, WorkerMessenger } from 'post-me';

const worker = new Worker('./worker.js');

const messenger = new WorkerMessenger({ worker });

ParentHandshake(messenger).then((connection) => {
  const remoteHandle = connection.remoteHandle();

  // Call methods on the worker and get the result as a promise
  remoteHandle.call('sum', 3, 4).then((result) => {
    console.log(result); // 7
  });

  // Listen for a specific custom event from the worker
  remoteHandle.addEventListener('ping', (payload) => {
    console.log(payload) // 'Oh, hi!'
  });
});

Worker code:

import { ChildHandshake, WorkerMessenger } from 'post-me';

// Methods exposed by the worker: each function can either return a value or a Promise.
const methods = {
  sum: (x, y) => x + y,
  mul: (x, y) => x * y
}

const messenger = WorkerMessenger({worker: self});
ChildHandshake(messenger, methods).then((connection) => {
  const localHandle = connection.localHandle();

  // Emit custom events to the app
  localHandle.emit('ping',  'Oh, hi!');
});

Typescript

Using typescript you can ensure that the parent and the child are using each other's methods and events correctly. Most coding mistakes will be caught during development by the typescript compiler.

Thanks to post-me extensive typescript support, the correctness of the following items can be statically checked during development:

  • Method names
  • Argument number and types
  • Return values type
  • Event names
  • Event payload type

Below a modified version of the previous example using typescript.

Types code:

// types.ts

export type WorkerMethods = {
  sum: (x: number, y: number) => number;
  mul: (x: number, y: number) => number;
}

export type WorkerEvents = {
  'ping': string;
}

Parent Code:

import {
 ParentHandshake, WorkerMessenger, RemoteHandle
} from 'post-me';

import { WorkerMethods, WorkerEvents } from './types';

const worker = new Worker('./worker.js');

const messenger = new WorkerMessenger({ worker });

ParentHandshake(messenger).then((connection) => {
  const remoteHandle: RemoteHandle<WorkerMethods, WorkerEvents>
    = connection.remoteHandle();

  // Call methods on the worker and get the result as a Promise
  remoteHandle.call('sum', 3, 4).then((result) => {
    console.log(result); // 7
  });

  // Listen for a specific custom event from the app
  remoteHandle.addEventListener('ping', (payload) => {
    console.log(payload) // 'Oh, hi!'
  });

  // The following lines have various mistakes that will be caught by the compiler
  remoteHandle.call('mul', 3, 'four'); // Wrong argument type
  remoteHandle.call('foo'); // 'foo' doesn't exist on WorkerMethods type
});

Worker code:

import { ChildHandshake, WorkerMessenger, LocalHandle } from 'post-me';

import { WorkerMethods, WorkerEvents } from './types';

const methods: WorkerMethods = {
  sum: (x: number, y: number) => x + y,
  mul: (x: number, y: number) => x * y,
}

const messenger = WorkerMessenger({worker: self});
ChildHandshake(messenger, methods).then((connection) => {
  const localHandle: LocalHandle<WorkerMethods, WorkerEvents>
    = connection.localHandle();

  // Emit custom events to the worker
  localHandle.emit('ping',  'Oh, hi!');
});

Other Uses

post-me can establish the same level of bidirectional communications not only with workers but with other windows too (e.g. iframes) and message channels.

Internally, the low level differences between communicating with a Worker, a Window, or a MessageChannel have been abstracted, and the Handshake will accept any object that implements the Messenger interface defined by post-me.

This approach makes it easy for post-me to be extended by its users.

A Messenger implementation for communicating between Windows and MessagePorts is already provided in the library (WindowMessenger and PortMessenger).

Windows

Here is an example of using post-me to communicate with an iframe.

Parent code:

import { ParentHandshake, WindowMessenger } from 'post-me';

// Create the child window any way you like (iframe here, but could be popup or tab too)
const childFrame = document.createElement('iframe');
const childWindow = childFrame.contentWindow;

// For safety it is strongly adviced to pass the explicit child origin instead of '*'
const messenger = new WindowMessenger({
  localWindow: window,
  remoteWindow: childWindow,
  remoteOrigin: '*'
});

ParentHandshake(messenger).then((connection) => {/* ... */});

Child code:

import { ChildHandshake, WindowMessenger } from 'post-me';

// For safety it is strongly adviced to pass the explicit child origin instead of '*'
const messenger = new WindowMessenger({
  localWindow: window,
  remoteWindow: window.parent,
  remoteOrigin: '*'
});

ChildHandshake(messenger).then((connection) => {/* ... */});

MessageChannels

Here is an example of using post-me to communicate over a MessageChannel.

import { ParentHandshake, ChildHandshake, PortMessenger } from 'post-me';

// Create a MessageChannel
const channel = new MessageChannel();
const port1 = channel.port1;
const port2 = channel.port2;

// In the real world  port1 and port2 would be transferred to other workers/windows
{
  const messenger = new PortMessenger({port: port1});
  ParentHandshake(messenger).then(connection => {/* ... */});
}
{
  const messenger = new PortMessenger({port: port2});
  ChildHandshake(messenger).then(connection => {/* ... */});
}

Callbacks as call parameters

Even though functions cannot actually be shared across contexts, with a little magic under the hood post-me let's you pass callback functions as arguments when calling a method on the other worker/window.

Passing callbacks can be useful to obtain progress or partial results from a long running task.

Parent code:

//...
ParentHandshake(messenger).then(connection => {
  const remoteHandle = connection.remoteHandle();

  const onProgress = (progress) => {
    console.log(progress); // 0.25, 0.5, 0.75
  }

  remoteHandle.call("slowSum", 2, 3, onProgress).then(result => {
    console.log(result); // 5
  });
});

Worker code:

const methods = {
  slowSum: (x, y, onProgress) => {
    onProgress(0.25);
    onProgress(0.5);
    onProgress(0.75);

    return x + y;
}
// ...
ChildHandshake(messenger, methods).then(connection => {/* */})

Transfer vs Clone

By default any call parameter, return value, and event payload is cloned when passed to the other context.

While in most cases this doesn't have a significant impact on performance, sometimes you might need to transfer an object instead of cloning it. NOTE: only Transferable objects can be transfered (ArrayBuffer, MessagePort, ImageBitmap, OffscreenCanvas).

post-me provides a way to optionally transfer objects that are part of a method call, return value, or event payload.

In the example below, the parent passes a very large array to a worker, the worker modifies the array in place, and returns it to the parent. Transfering the array instead of cloning it twice can save significant amounts of time.

Parent code:

// ...

ParentHandshake(messenger).then((connection) => {
  const remoteHandle = connection.remoteHandle();

  // Transfer the buffer of the array parameter of every call that will be made to 'fillArray'
  remoteHandle.setCallTransfer('fillArray', (array, value) => [array.buffer]);
  {
    const array = new Float64Array(100000000);
    remoteHandle.call('fillArray', array, 5);
  }

  // Transfer the buffer of the array parameter only for this one call made to 'scaleArray'
  {
    const array = new Float64Array(100000000);
    const args = [array, 2];
    const callOptions = { transfer: [array.buffer] };
    remoteHandle.customCall('scaleArray', args, callOptions);
  }
});

Worker code:

// ...

const methods = {
  fillArray: (array, value) => {
    array.forEach((_, i) => {array[i] = value});
    return array;
  },
  scaleArray: (buffer, type value) => {
    array.forEach((a, i) => {array[i] = a * value});
    return array;
  }
}

ChildHandshake(messenger, model).then((connection) => {
  const localHandle = connection.localHandle();

  // For each method, declare which parts of the return value should be transferred instead of cloned.
  localHandle.setReturnTransfer('fillArray', (result) => [result.buffer]);
  localHandle.setReturnTransfer('scaleArray', (result) => [result.buffer]);
});

Debugging

You can optionally output the internal low-level messages exchanged between the two ends.

To enable debugging, simply decorate any Messenger instance with the provided DebugMessenger decorator.

You can optionally pass to the decorator your own logging function (a glorified console.log by default), which can be useful to make the output more readable, or to inspect messages in automated tests.

import { ParentHandshake, WindowMessenger, DebugMessenger } from 'post-me';

import debug from 'debug';          // Use the full feature logger from the debug library
// import { debug } from 'post-me'; // Or the lightweight implementation provided

let messenger = new WindowMessenger({
  localWindow: window,
  remoteWindow: childWindow,
  remoteOrigin: '*'
});

// To enable debugging of each message exchange, decorate the messenger with DebugMessenger
const log = debug('post-me:parent'); // optional
messenger = DebugMessenger(messenger, log);

ParentHandshake(messenger).then((connection) => {
  // ...
});

Output: debug output

Parallel Programming

@post-me/mpi is an experimental library to write parallel algorithms that run on a pool of workers using a MPI-like syntax. See the dedicated README for more information.

API Documentation

The full API reference can be found here.

References

The post-me API is loosely inspired by postmate, with several major improvements and fixes to outstanding issues:

  • Native typescript support
  • Method calls can have both arguments and a return value: (#94)
  • Parent and child can both expose methods and/or events (instead of child only): #118
  • Exceptions that occur in a method call can be caught by the caller.
  • Better control over handshake origin and attempts: (#150, #195)
  • Multiple listeners for each event: (#58)

post-me's People

Contributors

alesgenova avatar birtles avatar cmawhorter avatar dependabot[bot] avatar hostxx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

post-me's Issues

handshake code expects global.window to exist

Edit: that change prevents the error about an unsupported worker, but it looks like isWindow is used in many places. i'll just inject a global.

I based my tests on the ones here (but with mocha), and it fails because this code checks thisWindow against a global Window that doesn't exist for me:

post-me/src/handshake.ts

Lines 106 to 115 in 5763644

if (isWindow(thisWindow) && isWindow(otherWindow)) {
addMessageListener = makeWindowAddMessageListener(
thisWindow,
acceptedOrigin
);
}
if (isWindow(thisWindow) && !isWindow(otherWindow)) {
addMessageListener = makeWorkerAddMessageListener(otherWindow);
}

I'm guessing jest is doing something to inject the global. Otherwise I don't understand how the tests are passing.

It works if I change it to (like postMessage above). I have no idea if it's correct though:

    if (isWindow(thisWindow) && isWindow(otherWindow)) {
      addMessageListener = makeWindowAddMessageListener(
        thisWindow,
        acceptedOrigin
      );
    }
    else {
      addMessageListener = makeWorkerAddMessageListener(thisWindow); // maybe otherWindow?
    }

WorkerMessenger `worker` parameter should accept an argument of type DedicatedWorkerGlobalScope

Describe the bug
The self object in a dedicated worker has type DedicatedWorkerGlobalScope. However, the WorkerMessenger constructor expects the worker parameter to be of type Worker.

It seems like that argument could be simply a union type allowing Worker or DediatedWorkerGlobalScope.

I'm happy to make up a PR for this, but I want to be sure you agree it's a bug first.

To Reproduce
Steps to reproduce the behavior:

  1. Use a worker file declare const self: DedicatedWorkerGlobalScope.
  2. Try to add new WorkerMessenger({ worker: self })

Expected behavior
No type errors.

Actual behavior
Typescript complains that self doesn't have the terminate method which is required for it to satisfy the definition of Worker.

Calls should reject promise if target window is unresponsive

Is your feature request related to a problem? Please describe.
If I call A() on a child window, the target window may become unresponsive while running A() and the caller is stuck waiting on the result of A().

Describe the solution you'd like
I'd like for the call to fail after a set amount of unresponsive time (e.g. 2 seconds).

Describe alternatives you've considered
I can manually implement a simple ping() method on the target and have the caller ping it periodically to check if the target window is still responsive. This is not ideal because we have a complex architecture with many target windows and we'd have to implement ping() on all of them. I think changes in post-me are required for this, I can do it myself if you don't have time but any guidance is appreciated.

We have the target windows catching errors and onunload events and those are handled properly, but there are situations in which those aren't triggered.

Additional context
Related but different is a timeout parameter for calls. We don't want calls to timeout for our specific use case because we expect some calls to take an indefinite amount of time. We just want to check that the window continues to be responsive.

old ibridge and post-me community update and migration guides

Make sure that we

  • deprecate old repos in favor of this one
  • deprecate post-me npm package in favor of this one
  • provide links from those old places into this one
  • perhaps migrate issues from the old repos to this one
  • create migration guides from old ibridge and post-me into new ibridge

Extend the library to more use cases

The utilities provided by post-me can be useful for any use case in which there is only one low level channel of communication (similar to postMessage).

By adding more classes implementing the Messenger interface (and possibly extending the Messenger interface if needed) we can directly support more scenarios.

Just a couple of example that come to mind:

  • Websockets
  • MessageChannels [#47 ]

Add TypeScript support for "Import post-me as a script"

Is your feature request related to a problem? Please describe.
I'm using "Import post-me as a script" way, on TypeScript. But can't find a built in way to code strong type in TypeScript.

Describe the solution you'd like
I add a new export_post-me.d.ts as:

export * from '../lib/post-me/dist/index';
export as namespace PostMe;

Then I can new PostMe.WindowMessenger() object, or get PostMe.RemoteHandle type.
I suggest post-me to include export as namespace PostMe, and add description in README.md. Let new comer like me spent less time try to find the right way.

Describe alternatives you've considered
I have no other way to make post-me strong type, except any. If you have a better way to make TypeScript happy, I'm glad to know. And still hope to add description in README.md too.

Additional context
No.

jsdoc on every public method

jsdoc is a great tool for providing inline documentation and combining it with typescript information, additionally
with a tool such as typedoc (which is far far from perfect and far far from i.e. rust doc) we can generate automatic API documentation and even host it in github pages.

Additionally most modern IDE understand both typescript, type declarations and jsdoc comments so they will provide useful autocomplete information off of it.

  • document all public methods and interfaces
  • do not use jsdoc type annotations since we already have typescript
  • run some tests with typedoc

high level apis

Lib consumers (we have some feedback in old ibridge repo) value that we provide high level apis that abstract
iframe creation / hidding etc. Make sure we also include that high level functionality on this new ibridge,
it should be pretty simple to implement on top of the core that we have been designing, i.e.

import ibridge from 'ibridge'

const remoteOrigin = SOME_URL
// resolves when the iframe triggers onLoad
const remoteWindow = await ibridge.spawnIframe(remoteOrigin)

const iparent = new ibrdige.Parent(WindowMessenger(window, remoteWindow, remoteOrigin), model)

we could even allow for more abstraction

const remoteOrigin = SOME_URL
const windowMessenger = await ibridge.Messenger.spawnIframe(window, remoteOrigin)

const iparent = new ibridge.Parent(windowMessenger, model)

Starting child too long after parent handshake causes handshake message to be missed

Describe the bug
Initiating parent handshake before child causes bot handshakes to wait indefinitely.

To Reproduce

  const parentOrigin = 'https://parent.example.com';
  const childOrigin = 'https://child.example.com';
  const [parentWindow, childWindow] = makeWindows(parentOrigin, childOrigin);
  let parentConnection: Promise<Connection>,
    childConnection: Promise<Connection>;
  // kick off parent handshake
  parentConnection = ParentHandshake(
    {},
    childWindow,
    childWindow.origin,
    parentWindow
  );
  // force a delay so problem always happens
  await (new Promise(resolve => setTimeout(resolve, 100));
  childConnection = ChildHandshake({}, parentWindow.origin, childWindow);
  // never succeeds because both connections miss their message
  await Promise.all([parentConnection, childConnection]);

Expected behavior
Both parent and child should continuously retry their handshake message until it succeeds or there is a hard error.

Getting 'DataCloneError' if function returns Promise

Describe the bug
Host exposed a function which returns "Promise". On calling that function from iframe url code, we get following error "Response object could not be cloned." If the Host function returns objects/array/string it works fine.
What is the workaround.

Allow using classes to implement local methods

Is your feature request related to a problem? Please describe.
I would like to provide a class instance to implement local methods, e.g.

class MyThing {
  private localState = 'abc';

  doStuff() {
    return this.localState;
  }
}

const thing = new MyThing();
ClientHandshake(messenger, thing);

However, due to the way local methods are invoked, attempting to do the above will fail when invoking doStuff from the other end because this.localState will be undefined.

Describe the solution you'd like
If we could make the following line from handles.ts:

const method = this._methods[methodName];

do something like:

const method = this._methods[methodName].bind(this._methods);
// (Presumably this would need to go after the part where we check if
// `method` is a function or not)

then I think this would allow using class instances.

Describe alternatives you've considered
The alternative is the client has to wrap their class instance in such a way that all methods get bound to the class instance. post-me could provide such a utility, I suppose.

I'm happy to work on a PR for this if you agree with the problem and proposed solution.

Using post-me with child website

Hello,

Very nice project, i am wondering if can use post-me for this use-case :

  • Say (for example) main page needs a css selector from another web site, opened in another tab
  • According to the readme, post-me is able to communicate between tabs
  • Is it possible to make some overlay above child website, prompting user to click on something on HTML, then the parent's click handler would return the event (and its target!)
  • From the event parent's page would display what user clicked

Is this possible with post-me ? (browsers' security block me...)
If not, any idea ?

Thanks

Add support for functions

I need to encrypt file in a worker, and get back a progress, my function take a progress callback, and of course this callback can't be cloned. But I know that this is possible to transfer into workers and it will help me a lot to have this feature in post-me

Run tests in a real browser environment instead of JSDOM

It is hard to realistically test some of the features provided by post-me in a mocked JSDOM environment.
It would be nice to run our tests in a real browser environment (puppeteer?), especially for:

  • Cross origin iframes
  • Workers
  • Transfer instead of Clone

Cherry on top, would be to have the code coverage reports from the browser tests

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.