Giter Site home page Giter Site logo

webext-messenger's People

Contributors

dependabot[bot] avatar fregante avatar github-actions[bot] avatar twschiller avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

webext-messenger's Issues

`chrome.runtime.sendMessage` does not behave as previously expected

I found a peculiar behavior of the native sendMessage API. You'd think that browser.runtime.sendMessage() sent a message to the background page, but you'd be wrong. Try:

  1. Open the options page
  2. Run this in its console: browser.runtime.onMessage.addListener(console.warn)
  3. Open another tab where PB has permissions

You'll see a bunch of messages on the options page: It's receiving every message that's meant for the background page.

Now this hasn't generally been a problem because messages that don't have a local handler have been ignored, but the messenger throws errors if there's no handler registered:

throw new Error("No handler registered for " + message.type);

So this happens:

  1. A context sends a message "to the background page" via chrome.runtime.sendMessage()
  2. The browser sends 2 messages, in order:
    • To the options page
    • To the background page
  3. The options page responds first, with an error
  4. The sender immediately receives the errors

The issues I've been having all week with #43 and related PRs is that the sidebar (which is an "extension page") has been receiving the messages meant for the background page.

How to proceed now:

  • Drop error throwing
  • Re-evaluate life messaging decisions

This discovery also has interesting results, such as the ability to message the sidebar or modalForm directly, from any context, without knowing anything about it.

Add temporary listeners

Messenger is set up around permanent listeners and that makes sense in the vast majority of cases, but sometimes we want to specifically start listening for a message.

Currently this can be done on the userside, by setting up their own event handler.

I wonder if this could be implemented locally somehow. I have a few ideas:

addListener/removeListener

The user can register a dummy method that just ensures we never throw "handler not registered"

registerMethods({MY_METHOD: () => {}})

and then somewhere else they can attach or remove listeners via:

function temporaryListener() {}
messenger.addListener('MY_METHOD', temporaryListener)
messenger.removeListener('MY_METHOD', temporaryListener)

Pros:

  • multiple listeners can be attached without changing how registerMethods works

Cons:

  • the listeners can't respond
  • the sender can't know if it was handled by these listeners

unregisterMethods

In addition to the static registerMethods call in registration.ts, the user can register and unregister methods as necessary.

Pros:

  • the temporary listener can respond
  • the sender knows whether a listener was registered

Cons if we allow more than one listener:

  • the listener still can't respond, the call will return Promise<void>

Cons if we allow only one listener:

  • Runtime errors if we're not careful about registering only one listener

expectCall

Perhaps in addition to those generic listeners, there could be a simple one-time listener like:

openTab('page.html');
const target = await expectCall('REGISTRATION');

This specifically would help removing tabReady:

Either support non-async handlers or throw error when they aren't

Problem:

  • A sync method can currently be registered, but the code expects a sync function internally (.then)
TypeError: Cannot read properties of undefined (reading 'then')

const response = await handler.call(sender, ...message.args).then(

Solutions:

Add local testing page with better explanation

I'm currently loading random lightweight websites to test the various parts of the package. I think it's best to publish some flexible "server"/page that can be configured via URL (e.g. in order to add some explanation to each link as well)

Handle repeated injection gracefully

Due to this error, mistakenly running the messenger twice (e.g. via executeScript) will now cause the second handler to respond with an error: The "registered methods" array is empty for every execution because it's not shared.

Possible solutions

  • stop replying with "no handlers registered yet" error
    • Easiest, but losing DX. It could be replaced by a local error log
  • detect duplicate injection and completely disable repeated runs (including registerMethods, which would have to throw an error)
    • Medium effort, high chance of future issues (feels like digging a deeper hole)
  • make the "registered methods" really global, like window[MESSENGER_SYMBOLS] ??= []
    • Medium effort, medium chance of future issues, we'll have to ensure that all such data is stored globally

cc @twschiller

Provide way to see unified log of message events

Context

  • Browser messaging is a distributed system with some unintuitive message routing behavior
  • During development/debugging, it would be nice to see the exact sequence of message events, possibly using a tool for distributed systems like https://bestchai.bitbucket.io/shiviz/
    -Depending on how it's implemented, for testing, we could even write assertions like "happened before", etc.

Implementation Sketch

  • Create a "trace" flag to control whether event tracing is enabled
  • Send event metadata to a common location, e.g., a text file, IDB database, or to the background page
    • For each message received, log timestamp, who it was received from, and type
    • For each message sent, log timestamp, the target it was sent to, etc.
    • For each message responded to, log timestamp, who it was received from, success/error, etc.

Provide way to correlate messages/responses in logs

Currently there's not way to determine which responses belong to which messages
Messenger: ENSURE_CONTEXT_MENU [{…}] from {sender: {…}}
Messenger: ENSURE_CONTEXT_MENU responds {value: undefined}
Messenger: ENSURE_CONTEXT_MENU responds {value: undefined}

It would be nice to include a nonce or global sequence number

Related Issues

Preserve error stack traces across contexts

  1. content.js calls myGet()
  2. myGet’s handler returns an error in the background page
  3. content.js throws the error

Since the error was generated in the background page, its stack trace won't have any reference to myGet() nor its origin. Only the console will point to the throw Error inside the response handler, but nothing before that.

Maybe something like this will append the current stack trace to the error that was received:

  const error = deserializeError(response.error);
+ error.stack += Error.captureStackTrace();
  throw error;

How can a content script target a frame?

Content scripts should be able to send messages to "the sidebar".

At the moment, the background has some logic to track the sidebar, which I assume is:

https://github.com/pixiebrix/pixiebrix-extension/blob/b37306c112a8d495e226be0b0445d1da33d0c060/src/background/browserAction.ts#L226

This allows the content script to message the sidebar without knowing its ID. This is only possible because there's a smart "middle man" that tracks the connection between the two.

This specific situation could be fixed by having the sidebar register itself directly with the content script instead of the background page. Then the content script will be able to contact the sidebar with the usual target information:

const registeredSidebar: Target = {tabId: 123 /* current tab */, frameId: 3};
await renderPanelMessage(registeredSidebar, panels);

Messaging architecture

I like the current setup for a few reasons:

  • the caller can call a lifted function whether it's local or external
  • the functions appear as regular async functions and they're typed correctly
  • it's somewhat guaranteed that the internal message is being listened to (compared to "is there a handler for this message on the other side or did we forget to register it?")

However there are some issues:

  • dependencies are bundled whether they're used locally or not
  • because they're opaque, any async function may or may not throw messaging errors, which may or may not need to be handled in the calling point
  • double-hop functions need to be explicitly lifted twice (a lot of duplicate code)

We can discuss this further to see whether these issues exist. I can also look into improving messaging without losing the good parts.

I mentioned I had already thought about a possible (explicit) messaging architecture, but this did not carry over the function types.

webext-messenger example
// unsafe.js
import {sendMessage} from './messenger';
const key = await sendMessage('get-key', otherArg);
// contentscript.js
import './messenger'; // Proxies automatically
// background.js
import {onMessage} from './messenger';

onMessage({
  name: 'get-key',
  allowUnsafe: true,
  async handler(event) {
    if (event.source === 'unsafe') {
      return getKey(...event.args);
    }
  }
});

Also I find the code sharing with PB.com problematic because that introduces another invisible step that does not guarantee at all that the shared functions have the same signature. I found that Chrome occasionally does not update the extension until you completely remove it and reinstall it. Refined GitHub users are occasionally a few months behind for no apparent reason. We can discuss why this was implemented and if there's any alternative to it.

Target current tab from any iframe

We have tabId: "this" for this purpose:

however it only works between extension pages because:

  • only extension pages can use chrome.tab without intermediary steps

  • the current logic only handles "this" by comparing the sender to the current context, but this could be trivially changed:

     on send:
     	if isContentScript && tabId === "this"
     		send to background
    
     on background receive:
     	if tabId === "this"
     		forward to this.trace[0].tab.id

    // Set "this" tab to the current tabId
    if (to.tabId === "this" && thisTarget.tabId === from.tab?.id) {
    to.tabId = thisTarget.tabId;
    }

Messages from the dev tools have no reference about the tab

Since the editor is just a page running in a custom app (the dev tools), they don't have a "tab" and don't report which tab they're operating on. This is the sender information for a message coming from the dev tools (to the background page):

{
  id: "mpjjildhmpddojocokjkgmlkkkfjnepo",
  origin: "chrome-extension://mpjjildhmpddojocokjkgmlkkkfjnepo",
  url: "chrome-extension://mpjjildhmpddojocokjkgmlkkkfjnepo/devtoolsPanel.html"
}

This means that any context known to lack a sender.tab information will need to also provide a tabId. In this case, the browser.runtime.onMessage handler in background would be called with these arguments:

[
  { // Message, custom
    type: 'REPORT_ERROR',
    args: [],
    tab: 123
  },
  { // Sender, browser-provided
    id: "mpjjildhmpddojocokjkgmlkkkfjnepo",
    origin: "chrome-extension://mpjjildhmpddojocokjkgmlkkkfjnepo",
    url: "chrome-extension://mpjjildhmpddojocokjkgmlkkkfjnepo/devtoolsPanel.html"
  }
]

and then it would build the usual MessengerMeta object like this one before calling the handler:

{
	trace: [{
		tabId: 123,
		context: "devtools"
	}]
}

Related

Allow local calls

This feels like a smell, but I'll open an issue to possibly discuss it.

Previously

Lifters could be called in either context because the messenger could also call the locally-available function if it was in the same context.

Currently

If you call browser.runtime.sendMessage() in the background page, it will fail even if runtime.onMessage.addListener was called.

Possibly

If a public API finds the type in the local handlers Map it could immediately call it.

Advantages

  • It would make it easier to share a "public method" regardless of the context

Drawbacks

  • Additional code: especially around response handling
  • Differing API: the receiver could not get a sender because there wouldn't be a call. This is easy in a background script, but a content script doesn't know its own data (it would require extra code/messaging anyway)
  • It blurs the line between contexts again, allowing code to be loaded and run in any context

The latter might still be useful in some cases, but perhaps it's best to make it explicit (i.e. don't allow local calls) and avoid the rest of the drawbacks.

Sequentially number messages automatically

The number is per-context, so it might make sense to store it in the trace:

let lastMessage = 0;
export function renderPanelsMessage() {
	if (lastMessage > this.trace[0].sequence) {
		// Ignore message
	}

	// Handle it

	lastMessage = this.trace[0].sequence
}

A possible alternative/companion to:

Prior art

Automatic numbering at the origin:

https://github.com/pixiebrix/pixiebrix-extension/blob/1857acd5b9fe9cf2d2a0fb58fc042d155ac081f2/src/actionPanel/native.tsx#L167-L174

Skipping stale messages in the handler

https://github.com/pixiebrix/pixiebrix-extension/blob/1857acd5b9fe9cf2d2a0fb58fc042d155ac081f2/src/actionPanel/protocol.tsx#L81:L91

Notifications are not retried

Notifications were explicitly implemented as a way to ignore failures and not attempt retries:

if (!options.isNotification) {
return manageMessage(type, sendMessage);
}
void sendMessage().catch((error: unknown) => {
debug(type, "notification failed", { error });
});

This may be unexpected in some cases, especially if attempting to message an iframe that is still loading.

This is actually specified in the code and the solution would be to just use regular messages in this case.

* "Notifications" won't await the response, return values, attempt retries, nor throw errors

I'll open and close this issue just as a way to track this reasoning. I'll add what a "Notification" implies to the glossary.

Preserve sender trace information while forwarding

The current MessengerMeta (this) is the raw information that the browser provides as a sender of the message.

In forwarded messages, the sender always points to the latest step at the moment.

Ideally

Preserve whole stack of senders in the MessengerMeta:

// Message sent from CS to background page
[
	{tabId, frameId}
]
// Message sent from CS to other tab, via background page
[
	{tabId, frameId},
	{context: 'background'}
]
// Message sent from unsafe context to background page
[
	{tabId, frameId, unsafe: true},
	{tabId, frameId}, // Only if `externally_connectable` is missing
	{context: background}
]

Related

Queue messages

Follows:

Regarding queues, which have not been implemented, they'd block every message until a previous message has responded, which in practice would slow down the application at a minimum and cause pileups in worst case.

The simplest of response (ACK) takes about 100ms (I've seen this before), so we could only ever send 10 messages per second:

|---RETRY---ACK . . . . . . Response|
               |---ACK . . . . . . Response|
                      |---ACK . . . . . . Response|
                             |---ACK . . . . . . Response|

while a non-queued version (what we have) is able to just queue the messages in the browser, which can then respond as necessary:

// Retries are slightly randomized with https://github.com/tim-kos/node-retry
|---RETRY. . . . . . Response|
|----RETRY . . . . . . . Response|
|--RETRY . . . . . . . Response|
|---RETRY . . . . . . . . . Response|
|----RETRY . . . . . . . . Response|

If the application’s responses benefit from or require a certain order of unrelated messages, that's a code smell to me. I think ordering is best handled where necessary.


Something I could improve though is to stop the retries if a message is currently being retried:

|---RETRY. . . . . . Response|

// Then other calls are attempted while waiting
// but they all start once the first retry is done
  |------RETRY . . . . . . . Response|
   |-----RETRY . . . . . . . . Response|
    |----RETRY . . . . . . . . . Response|
    |----RETRY . . . . . . . . Response|

But I don't know whether this is worth it either.


I'll try porting this file to Messenger since I saw this explanation:
https://github.com/pixiebrix/pixiebrix-extension/blob/0fbacfedb1e0eee40d76a2168e6e4cb0b67711c5/src/background/browserAction.ts#L149-L157

Messages are forwarded twice if 2 runtime pages are open

Due to this:

if (!to.page) {
return "forward";
}

Any runtime page that receives a forwardable message, will forward it. This means that a tab could receive the same message multiple times.

Likely repro

  1. Open /options.html
  2. Open website A
  3. Open website B
  4. Send a message from the content script of A to B with target {tabId}, which will trigger a runtime.sendMessage call
  5. Both the options and background page will receive the message
  6. Both the options and background page will "forward it", i.e. making two tabs.sendMessage call

Likely solution

Only forward in the background page

"No handler registered for # in the receiving end" is not quite right for named targets

  1. Double-click the browser action button in PixieBrix
  2. See error in background page

The error is caused by:

await pingSidebar({ tabId: "this", page: "/sidebar.html" });

The target doesn't existing long enough to respond, or rather "it never existed," so the message should be along the lines of "target not found" rather than "no handlers registered."

Quick refresher

This message is sent once via browser.runtime.sendMessage and is received by every open chrome:// page and background, which decides whether to answer or ignore.

Resolution

  • Case: target doesn't exist or didn't call chrome.runtime.onMessage.addListener
    • Browser throws: Could not establish connection. Receiving end does not exist.
      • Sender should throw: Target could not be found
  • Case: target didn't call registerMethods
  • Case: target didn't register requested method
    • Receiver should throw: No handler registered for # in [target]

Notes

The current retrier does not remember errors. If the first call returns "No handlers registered" and the second error is "Target could not be found". This is particularly troublesome because only the last error ever surfaces.

In this case, it should probably stop retrying because the target was found but is now gone.

Related

This issue is somewhat related to a recent one, but it has a completely different cause:

Implement notifications

"Notifications" could be easily implemented as:

void handleNavigate(target).catch(noop)

or at the API point as:

export const handleNavigate = ignoreErrors(getContentScriptMethod("HANDLE_NAVIGATE"));

But by looking at the liftContentScript, it seems that notifications are meant to be completely lost even if they can't be delivered, not just as a way to indicate that we're not listening to its return value:

https://github.com/pixiebrix/pixiebrix-extension/blob/c89b34b539f1b894a2663fa6b2f740979bb47927/src/contentScript/backgroundProtocol.ts#L124-L130

This becomes more obvious now that I implemented retries (#18), which specifically expect the listener to eventually respond, which is the opposite of the above implementation:

Screen Shot

The log above will end up reading:

Messenger: HANDLE_NAVIGATE notification failed, will not retry

More context:

Documentation: How sendMessage responds

sendMessage is called without callback

No error will be logged, even if the tab doesn't exist, so for these errors to appear you must pass a callback like chrome.tabs.sendMessage(2, 'hi', console.log)

Tab doesn't exist

Could not establish connection. Receiving end does not exist.

Tab didn't call chrome.runtime.onMessage.addListener

Could not establish connection. Receiving end does not exist.

onMessage listener did not call sendResponse

If the listener returns true

The error is thrown when the page is navigates away or the tab is closed.

Before Chrome 102:

The message port closed before a response was received.

In Chrome 102:

A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received

If the listener returns anything else

This error is thrown immediately:

The message port closed before a response was received.

Move meta from `this` to external/contextual getter

Information about the sender/trace and options is currently stored on this. This is because:

  • Most methods don't need access to this information
  • It can be easily be left out of the method definition when not used
  • It can be easily used regardless of parameter length
  • It allows the the registration of any function, not just functions with specific parameters (e.g. arguments[0] is the meta")

this is fine for me, but maybe there alternative solutions. This is what I came up with:

import {getCurrentMeta} from "webext-messenger";

function whoAmI() {
	const meta = getCurrentMeta();
	return meta.trace[0];
}

This pattern:

  • Preserves the previous wins
  • Actually simplifies our types further (due to the lack of this)
  • Somewhat matches other patterns like:
     import.meta.url; // Only available in the current file, in ES Modules
    
     function onMessageHandler() {
     	if (chrome.runtime.lastError) { // Available only in this call
     		
     	}
     }

The implementation would look like:

- await handler.apply(meta, args)
+ currentMeta = meta;
+ await handler(...args)
+ currentMeta = undefined;

and

export function getCurrentMeta() {
	return currentMeta;
}

Throw error if `registerMethods` was never called

  1. A notify method calls browser.runtime.sendMessage
  2. The background page exists, has unrelated message handler that ignores the message
  3. sendMessage returns Promise<void>

There's no way to determine whether void is the return value of notify or if the message was never handled.

Solution

  1. The listener should wrap each response in a custom object to ensure that an undefined response is not confused with a non-response, like {__webext_messenger_response__: undefined}

return handler.call(sender, ...message.args).catch((error: unknown) => ({

  1. getMethod should then unwrap this response before returning it

return response;

Silencing logs

The messenger logs a lot of calls to the console. Currently they can be hidden:

  • by hiding debug calls in the console filter
  • by typing -messenger in the text filter

@twschiller mentioned that we should probably provide a way to silence the messenger further, but there are many ways we can achieve this, so we need to pick what makes the most sense to us. Some examples, can be combined:

  • disable all by default
  • toggle via localStorage (similarly to how dark mode was implemented)
  • toggle via ENV
  • filter via regex
  • filter by message ID (array)

Other options? The easiest and most helpful solution is probably:

  • off by default
  • enabled via ENV in local .env file

Remove branch protection from repo

Opening an issue to keep track of the conversation.

Re: 24192c0#commitcomment-82700570

For our upcoming SOC-2 (security audit) we need to enable branch protections on any code that we publish

We don't publish webext-messenger directly to consumers, it's an npm dependency that is covered by the branch protections included in the extension itself. Adding a branch protection here is akin to requiring that every one of our npm dependencies must also be protected by us.

For that reason I don’t think we need to have branch protection here (also for your own ease of update)

Where are notification errors handled?

Notifications never throw errors, they are not meant to be awaited and therefore there's no way to async catch errors either.

I think that currently every notification errors are just logged to the console, but they don't bubble up and reportError won't catch them.

webext-messenger/index.ts

Lines 147 to 149 in 2e13ca0

void sendMessage().catch((error: unknown) => {
console.debug("Messenger:", type, "notification failed", { error });
});

Question

  • Should such error be re-thrown in that handler? That would let a global unhandledrejection handler catch it.

Alternatively, a user interested in errors should just not send notifications so that they can use the usual:

myApiCall().catch(reportError)

Prior art

A background forwarder catches an error:

https://github.com/pixiebrix/pixiebrix-extension/blob/ea335a13e28a8c63ab1c73a6e4c2c2ba27a65da9/src/background/browserAction.ts#L250-L252

But the original sender just ignores the message:

https://github.com/pixiebrix/pixiebrix-extension/blob/ea335a13e28a8c63ab1c73a6e4c2c2ba27a65da9/src/actionPanel/native.tsx#L167-L168

With integrated message forwarding (#23, #31) this user-defined middleman/handler won't exist.

Provide way to explicitly mark receiver as ready to start handling messages

Context

  • Currently the messenger receiver starts listening on import
  • An application may have application-specific setup/initialization to do before it's ready to receive messages
  • Currently the application has to be careful not to import the messenger receiver until the rest of the setup has occurred. This is tricky if there are transitive dependencies to the messenger

Related Discussion

Related Issues

Allow discarding messages

In some rare cases, only the latest version of a message is useful, so it's not important to preserve messages that are currently being retried.

Concerns

  • This might be a micro-optimization, depending on the amount of expected messages that will be sent while a target isn't ready
  • The retry logic is handled by p-retry and it might not be easy to check whether a message is currently in queue. Maybe an AbortSignal could be used to automatically cancel any messages in queue

Prior art

Discarding logic:

https://github.com/pixiebrix/pixiebrix-extension/blob/main/src/actionPanel/native.tsx#L167:L167

Used as a notification only:

https://github.com/pixiebrix/pixiebrix-extension/blob/ea335a13e28a8c63ab1c73a6e4c2c2ba27a65da9/src/background/browserAction.ts#L150-L158

Add automated testing

Context

The tests run on a few pages and they output their results in the console, which the tester has to manually open to verify:

  • background page (to test the CS api)
  • a website (to test the background API)
  • a website (to test the CS api via the background page) in #22
  • options page
  • … more in the future

Ideally

  1. Browser launches via CLI
  2. Retrieves the log from each context in series (or separate runs)
  3. Outputs it to the CLI
  4. Exits with 0 if the tests fail

Prior art

https://github.com/juliangruber/tape-run

I used this before and it's fantastic, but it runs the code autonomously, as a regular page script (we'd need the code to run in extension contexts)

Alternatives

Minimal, complementary, Node-only testing with mocks could be written, but it would not replace the existing tests. This would verify the internals (logging, retries, exact messages)

Generics are stripped

Expected repro

function proxyService<T extends string>(value: T): T {
	return value;
}

declare global {
  interface MessengerMethods {
    PROXY: typeof proxyService;
  }
}

registerMethods({
  PROXY: proxyService,
});
export const messengerProxyService = getMethod("PROXY");

The type of messengerProxyService won't have the generic:

function proxyService(value: unknown): unknown;

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.