Giter Site home page Giter Site logo

proposal-import-attributes's Introduction

Import Attributes

Champions: Sven Sauleau (@xtuc), Daniel Ehrenberg (@littledan), Myles Borins (@MylesBorins), Dan Clark (@dandclark), and Nicolò Ribaudo (@nicolo-ribaudo).

Status: Stage 3

⚠️ The specification in this proposal might be out of date. tc39/ecma262#3057 is the latest version.

Some of the changes present in the current specification have not been presented to committee yet: #142

Please leave any feedback you have in the issues!

Synopsis

The Import Attributes proposal, formerly known as Import Assertions, adds an inline syntax for module import statements to pass on more information alongside the module specifier. The initial application for such attributes will be to support additional types of modules in a common way across JavaScript environments, starting with JSON modules.

The syntax will be as follows (shown here is the proposed method for importing a JSON module):

import json from "./foo.json" with { type: "json" };
import("foo.json", { with: { type: "json" } });

The specification of JSON modules was originally part of this proposal, but it was resolved during the July 2020 meeting to split JSON modules out into a separate Stage 3 proposal.

Motivation

Standards-track JSON ES modules were proposed to allow JavaScript modules to easily import JSON data files, similarly to how they are supported in many nonstandard JavaScript module systems. This idea quickly got broad support from web developers and browsers, and was merged into HTML, with an implementation for V8/Chromium created by Microsoft.

However, in an issue, Ryosuke Niwa (Apple) and Anne van Kesteren (Mozilla) proposed that security would be improved if some syntactic marker were required when importing JSON modules and similar module types which cannot execute code, to prevent a scenario where the responding server unexpectedly provides a different MIME type, causing code to be unexpectedly executed. The solution was to somehow indicate that a module was JSON, or in general, not to be executed, somewhere in addition to the MIME type.

Some developers have the intuition that the file extension could be used to determine the module type, as it is in many existing non-standard module systems. However, it's a deep web architectural principle that the suffix of the URL (which you might think of as the "file extension" outside of the web) does not lead to semantics of how the page is interpreted. In practice, on the web, there is a widespread mismatch between file extension and the HTTP Content Type header. All of this sums up to it being infeasible to depend on file extensions/suffixes included in the module specifier to be the basis for this checking.

There are other possible pieces of metadata which could be associated with modules, see #8 and tc39/proposal-import-reflection#18 for further discussion.

Proposed ES module types that are blocked by this security concern, in addition to JSON modules, include CSS modules and potentially HTML modules if the HTML module proposal is restricted to not allow script.

Rationale

There are three places where this data could be provided:

  • As part of the module specifier (e.g., as a pseudo-scheme)
    • Challenges: Adds complexity to URLs or other module specifier syntaxes, and risks being confusing to developers (further discussion: #11)
    • webpack supports this sort of construct (docs).
      • Demand from users for similar behavior in Parcel, with pushback from some maintainers (#3477)
  • Separately, out of band (e.g., a separate resource file)
    • Challenges: How to load that resource file; what should the format be; unergonomic to have to jump between files during development (further discussion: #13)
  • In the JavaScript source text
    • Challenges: Requires a change at the JavaScript language level (this proposal)

This proposal pursues the third option, as we expect it to lead to the best developer experience, and are hopeful that language design/standardization issues can be resolved.

Proposed syntax

Import attributes have to be made available in several different contexts. They use a key-value syntax is used preceded by the with keyword, with the key type used as an example indicating the module type. Such key-value syntax can be used in various different contexts.

import statements

The ImportDeclaration would allow any arbitrary attributes after the with keyword.

For example, the type attribute could be used to indicate a module type, for example importing a JSON module with the following syntax.

import json from "./foo.json" with { type: "json" };

The with syntax in the ImportDeclaration statement uses curly braces, for the following reasons (as discussed in #5):

  • JavaScript developers are already used to the Object literal syntax and since it allows a trailing comma copy/pasting attributes will be easy.
  • it clearly indicates the end of the attributes list when splitting them across multiple lines.

re-export statements

Similar to import statements, the ExportDeclaration, when re-exporting from another module, would allow any arbitrary attributes after the with keyword.

export { val } from './foo.js' with { type: "javascript" };

dynamic import()

The import() pseudo-function would allow import attributes to be indicated in an options bag in the second argument.

import("foo.json", { with: { type: "json" } })

The second parameter to import() is an options bag, with the only option currently defined to be with: the value here is an object containing the import attributes. There are other proposals for entries to put in the options bag: for example, the Module Source Imports proposal introduces a phase property.

Integration of modules into environments

Host environments (e.g., the Web platform, Node.js) often provide various different ways of loading modules. The analogous string could be passed through these ways of loading other kinds of modules.

Worker instantiation

new Worker("foo.wasm", { type: "module", with: { type: "webassembly" } });

Sidebar about WebAssembly module types and the web: it's still uncertain whether importing WebAssembly modules would need to be marked specially, or would be imported just like JavaScript. Further discussion in #19.

HTML

Although changes to HTML won't be specified by TC39, an idea here would be that each import attribute, preceded by with, becomes an HTML attribute which could be used in script tags.

<script src="foo.wasm" type="module" withtype="webassembly"></script>

(See the caveat about WebAssembly above.)

WebAssembly

In the context of the WebAssembly/ESM integration proposal: for imports of other module types from within a WebAssembly module, this proposal would introduce a new custom section (named importattributes) that will annotate with attributes each imported module (which is listed in the import section).

Proposed semantics and interoperability

This proposal does not specify behavior for any particular attribute key or value. The JSON modules proposal will specify that type: "json" must be interpreted as a JSON module, and will specify common semantics for doing so. It is expected the type attribute will be leveraged to support additional module types in future TC39 proposals as well as by hosts. HTML and CSS modules are under consideration, and these may use similar explicit type syntax when imported.

Attributes in addition than type may also be introduced for purposes not yet foreseen.

JavaScript implementations are encouraged to reject attributes and type values which are not implemented in their environment (rather than ignoring them). This is to allow for maximal flexibility in the design space in the future--in particular, it enables new import attributes to be defined which change the interpretation of a module, without breaking backwards-compatibility.

FAQ

Why not out of band?

Why not both? The champions of this proposal think that exploring both an in- and out of band solutions to various kinds of metadata. While we prefer in-band metadata for module types, we are happy to see the development of various out-of-band manifests of modules being proposed and implemented in certain JS environments:

This proposal does not exclude out-of-band metadata being used for module types. And it definitely doesn't argue that all metadata should be in-band. For example, integrity hashes simply don't work in-band, both because module circularities make them impossible to calculate, and because of the need for a "cascading" update when a deep dependency changes.

Out-of-band solutions face certain downsides; these are not necessarily fatal, but are interesting to take into account when considering the solution space and making tradeoffs:

  • By-hand authoring experience: While an in-band solution is somewhat verbose, it is also more straightforward for developers to adopt when writing code without much tooling. For smaller projects developers do not need to create an extra file by hand.
  • Tooling complexity for large projects: For large project with many dependencies, developers will not have to worry about creating a large manifest by compiling the metadata of all of their dependencies. Module authors will also not have to worry about shipping a manifest in order for consumers to be able to run their modules.
  • Performance tradeoffs: The experience in Node.js's experimental, out-of-band policy files is that they can carry significant startup cost, due to certain aspects of loading and parsing.

How is common behavior ensured across JavaScript environments?

A central goal of this proposal is to share as much syntax and behavior across JavaScript environments as possible. To the same end, we also propose a standardization of JSON modules to the extent that this is possible (omitting just the contents of the redundant type check, which necessarily differs between environments, in addition to the pre-existing host-defined parts such as interpreting the module specifier and fetching the module).

However, at the same time, behavior of modules in general, and the set of module types specifically, is expected to differ across JavaScript environments. For example, WebAssembly, HTML and CSS modules may not make sense in certain minimal embedded JavaScript environments. We hope that environments can experiment and collaborate where it makes sense for them.

We see the management of compatibility issues across environments as similar, independent of whether metadata is held in-band or out-of-band. An out of band solution would also suffer from the risk of inconsistent implementation or support across host environments if some kind of coordination does not occur.

The topic of attribute divergence is further discussed in #34.

How would this proposal work with caching?

Attributes are part of the module cache key and can affect how a module is loaded: the cache key is extended from (referrer, specifier) to (referrer, specifier, attributes).

Why not use more terse syntax to indicate module types, like import json from "./foo.json" as "json"?

Another option considered and not selected has been to use a single string as the attribute, indicating the type. This option is not selected due to its implication that any particular attribute is special; even though this proposal only specifies the type attribute, the intention is to be open to more attributes in the future. (discussion in #12).

Should more than just strings be supported as attribute values?

We could permit import attributes to have more complex values than simply strings, for example:

import value from "module" with { attr: { key1: "value1", key2: [1, 2, 3] } };

This would allow import attributes to scale to support a larger variety of metadata.

We propose to omit this generalization in the initial proposal, as a key/value list of strings already affords significant flexibility to start, but we're open to a follow-on proposal providing this kind of generalization.

What are you open to changing? When do we have to settle down on the details?

We are planning to make descisions and reach consensus during specific stages of this proposal. Here's our plan.

Original plan before Stage 2 and Stage 3

Before stage 2

We have achieved consensus on the following core decisions as part of Stage 2, including:

  • The attribute form; key-value or single string (#12)
// Not selected
import value from "module" as "json";

// Not selected
import value from "module" with type: "json";

// Proposal that was approved from Stage 2 to Stage 3 the first time
import value from "module" assert { type: "json" };

Before stage 3

After Stage 2 and before Stage 3, we're open to settling on some less core details, such as:

  • Considering alternatives for the with/if/assert keywords (#3)
import value from "module" when { type: 'json' };
import value from "module" given { type: 'json' };
  • How dynamic import would accept import attributes:
import("foo.wasm", { with: { type: "webassembly" } });

For consistency the assert key is used for both dynamic and static imports.

An alternative would be to remove the assert nesting in the object:

import("foo.wasm", { type: "webassembly" });

However, that's not possible with the Worker API since it already uses an object with a type key as the second parameter. Which would make the APIs inconsistent.

Before Stage 4

  • The integration of import attributes into various host environments.
    • For example, in the Web Platform, how import attributes would be enabled when launching a worker (if that is supported in the initial version to be shipped on the Web) or included in a <script> tag.
new Worker("foo.wasm", { type: "module", with: { type: "webassembly" } });

Standardization here would consist of building consensus not just in TC39 but also in WHATWG HTML as well as the Node.js ESM effort and a general audit of semantic requirements across various host environments (#10, #24 and #25).

History

  • 2019-12: The proposal, named module attributes is approved for Stage 1 (notes part 1, notes part 2, slides) to explore metadata for module imports, and to explore guarantees about modules with no code execution.
  • 2020-06: Module attributes advances to Stage 2 (notes part 1, notes part 2, slides), with consensus based on the restriction that import attributes cannot be part of the cache key in the modules map. The proposed syntax is import { x } from "./mod" with type: "json", something: "else";.
  • 2020-09: The proposal, renamed to import assertions, advances to Stage 3 (notes, slides). The rename better describes the agreed assert-only semantics, and the keyword changes from with to assert. However, the proposal relaxes the caching restriction so that HTML can still include the module type as part of the cache key, while still respecting the "spirit" of the proposal.
  • 2021-052022-02: The proposal, with the import { x } from "./mod" assert { type: "json" }; syntax, is implemented and shipped in Chrome, Node.js and Deno. They all support the JSON modules proposal.
  • 2023-01: Due to incommpatibility with the semantics needed by HTML for non-JavaScript modules, specifically regarding HTTP fetching and CSPs, the proposal is demoted back to Stage 2 (notes part 1, notes part 2, slides) to investigate a solution to the web platform's needs.
  • 2023-03: The proposal is renamed to Import attributes and moves back to Stage 3 (TODO: notes, slides). The restriction on the cache key is completely removed, and the keyword changes back from assert to with: import { x } from "./mod" with { type: "json" };. For compatibility with existing implementations, the assert keyword will still be supported until it's safe to remove it, if it will ever be.

Specification

proposal-import-attributes's People

Contributors

bakkot avatar bfarias-godaddy avatar dandclark avatar devsnek avatar ghoullier avatar jlhwung avatar lahmatiy avatar littledan avatar mylesborins avatar nicolo-ribaudo avatar xtuc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

proposal-import-attributes's Issues

Semantics of this feature in HTML/the Web

Note: The semantics of this proposal on the Web would be standardized in WHATWG/HTML, not in this repository. This issue is purely for requirements-gathering/brainstorming.

The README has examples of using the type: module attribute, but the semantics of this are not really defined anywhere. I think there's been some confusion in the discussion in issues as different people make assumptions about different semantics. Here's how I am currently picturing that this would work on the Web/in HTML:

  • When JavaScript loads a module, either through an import statement or dynamic import(), it passes an optional additional parameter to the embedder, of the options bag of module attributes.
  • When HTML gets these module attributes, it ignores unrecognized attributes (for reasons discussed in #21) and just looks at type:. If the type is present and unrecognized, module fetching and parsing is aborted with an error.
  • Otherwise, the module is fetched, and the MIME type is compared with the type, and if no type is provided, then JavaScript is implicitly used. Each known type has a set of MIME types that it permits. If the MIME type is not contained in the type's set, then module fetching and parsing also errors out. If the module had already been fetched previously and lived in the cache, then the MIME type is cached along with it and used for this comparison.
  • The module is then parsed and processed according to its MIME type (which may provide more detailed information than simply the type:). The type: is not used at this point--it simply provided an additional check before the parser was invoked.

Any thoughts on this logic?

Define what a type is

Currently the proposal seems to imply that the type strictly matches with a mimetype: json will check for the application/json mimetype for intance.

However, there are cases where it might be too specific:

  1. BinAST has the mimetype application/javascript-binast while JavaScript has application/javascript. Some browser can support both depending what the server sends.
    For instance:

    import u from a with type: "javascript";

    Should allow both a BinAST encoded and a JavaScript to be imported.

  2. Similarly images; it's common for a web server to send different image format or quality depending on the client's request (based Accept header or similar).
    For instance:

    import u from a with type: "image";

    Should allow image/jpeg or image/webp in the response mimetype.

The interpretation of type should be something like:

  • If type is image, then
    1. assert image/jpeg, image/webp
  • If type is javascript, then
    1. assert application/javascript, application/javascript-binast

"the type attribute is used simply for a check"

From the readme:

For example, on the Web and similar environments, when providing a type module attribute, the module type would not form part of the cache key, where modules are always cached by the resolved URL. Instead, the type attribute is used simply for a check, which would cause the module graph to fail to load in the event of a mismatch

This suggests you couldn't do something like:

import jsText from './module.js' as 'text';
import { foo, bar } from './module.js';

I agree these should coalesce at the fetch & HTTP cache level, but they should be different modules.

Should WebAssembly modules be explicitly marked?

Various opinions are given in WICG/webcomponents#839. On one hand, Wasm is somehow equivalently powerful to JS, suggesting that the security needs are less stark. On the other hand, they have different parsers, so ambiguity may still be damaging. I don't think we need a final resolution within this proposal on this question, but this blocks WebAssembly/ESM integration. Cc @rniwa @annevk

How are the module attributes interpreted across host environments?

Note: The module attributes proposal would not require or recommend that hosts use any particular interpretation of the attributes. Some people in the JS community have a goal of non-Web environments working towards partial Web-compatibility, but such efforts lie outside of the scope of this proposal. This issue exists for brainstorming and requirements gathering that may feed back into the module attributes proposal, but not to make any sort of requirement or recommendation for JS environments.

The ECMAScript specification leaves resolution of module specifiers up to the host environment, e.g., HTML or Node.js. Probably we'd do the same for this proposal. At the same time, there's a broad effort to define similar semantics across many embedding environments. For one example, JSON modules make just as much sense in HTML and Node.js, even if they would have some host-specific semantics (e.g., checking the MIME type on the web). How should we coordinate to build compatibility here?

I want to suggest that, specifically for module types, we could have the logic to dispatch on certain type strings (initially, just JSON) and give them semantics in ECMA-262, or say "no interpretation" and fall back to the host. This would be invoked after the MIME type is checked. There's more details to work out when writing the spec to verify that it's a viable layering, though.

(Edit: Added the following paragraph to replace the previous one)

I want to suggest that the host be responsible for handling module attributes, and may use facilities provided by ECMA-262 to handle certain module types, for example JSON modules. It's still to be determined whether non-Web hosts would want to require a type: declaration the way it seems like the Web would.

Would non-web hosts still require with type: "json", or would they be comfortable relying on the file extension? Would tooling end up adding with type: "json"? My personal preference at this point is to require the author to include this in all cases, for maximum consistency/compatibility across environments, but this is a tradeoff for ergonomics and incremental upgrade. I'd be interested in hearing your thoughts.

How should unrecognized attributes or types be handled?

One option is to ignore them. However, that risks forward compatibility issues if they are later given semantics. I would suggest aborting the whole module graph when these are found, just like when an invalid module specifier or syntax error is there.

Should `undefined` allowed in second assignment of import call?

Currently the following use case

const assets = [
  ["ad-banner.mjs"],
  ["i18n/zh.json", { with: { type: "json" } }]
];

for (const [path, attrs] of assets) {
  await import(path, attrs);
}

will throw runtime type error as attrs can be undefined.

Should we allow import(moduleName, undefined) delegated to import(moduleName)?

Use object-literal-like notation for attributes?

To me, reading the key-value structure of attributes is a little difficult without the common structure of braces. An option is to use object-liter-like notation:

import {foo} from 'module' with {type: 'json'};

Since braces are already used with imports, and they aren't actual expressions, I don't think this would be any more confusing than what we have now. Both uses may look a bit like expressions (destructuring and object-literals respectively) and that's nice to intuitively understand some of the syntax, but they aren't and this is ok because import is special.

Request ContentType

May I ask to consider use of ContentType?

The keyword "if"

import value from "module" if { type: "json" };

implies that request would be issued with ContentType:*/* and response would be discarded based on ContentType. Which is wasteful, we already know what we accept.

Mechanism already defined in HTML <script language=vbscript type=text/vbscript>. Unfortunately used by type="module" but

<script src="foo.css" type="text/css"></script>
<script src="foo.json" type="application/json"></script>
<script src="foo.wasm" type="application/wasm"></script>

makes no requests. So it can be defined as "import WASM module".

It covers the need for Content Negotiation:

<script src="foo" type="application/json,application/wasm;q=0.9">

may be not pretty, but functional, tested in transition to PNG.

Currently there is a need to define "new types", searching for "type that is not MIME type" may be hard. While we are half there with image/*, audio/*, video/*.

Please reconsider use of alias (iftype) as core dependency. It may be built on top:

import value from "module" with { typeAlias: "json" };
contentTypeAlias.define("javascript", "application/javascript,application/javascript-binast")
contentTypeAlias.define("json|wasm", "application/json,application/wasm;q=0.9")

Thank you.

Use general key-value syntax, or a single string?

In the original thread at WICG/webcomponents#839, there are several suggestions of syntax which includes a single string, rather than key-value syntax. Let's discuss the advantages and disadvantages of this option in this thread.

For example, the syntax for an import statement could be:

import json from "./foo.json" as "json";

Advantages of a single string

  • Less to type: save 6 characters of "type: ". Including the type will be annoying enough as is!
  • Less overall complexity for the syntax and semantics
  • Less to coordinate across environments (c.f., #8, h/t to @MylesBorins for raising this issue)

Advantages of key-value syntax

  • Generalizes to other use cases (c.f., #8). Additional module parameters was an idea tossed around from even before this issue, so even if we don't find one of those use cases persuading now, we have some circumstantial evidence that it may come up in the future.
  • If we don't provide a general k/v syntax, there's a chance that people will see the need to do so inside the string (as was already proposed for the module specifier, see #11)
  • The as keyword is likely to be pretty confusing, as it's also a keyword with completely different meaning in other parts of an import statement

My opinion is that the advantages of a general key-value syntax outweigh the advantages for a single string. It's circumstantial evidence (since this proposal doesn't suggest acting on any of them yet), but the relatively high number of other possible use cases points to a k/v syntax to me. This would be analogous to the decision to make import.meta an object, rather than just adding a syntax for import.meta.url.

README should put our proposed syntax front and center

For someone skimming the document, the first code sample they encounter is not from this proposal, but a potential extension that we're not proposing here. I think we should put a concrete, easy-to-read example of how to use JSON modules front and center in the explainer, so it's easy to understand for people new to the proposal.

What syntax could be used to pass attributes to other subresources on the web?

Although module types only make sense in module contexts, other attributes (e.g., fetch options) may make sense for all sorts of subresources (e.g., <link> elements and @import in CSS). It would be ideal if we had a uniform way to pass these same flags to those contexts as well.

Otherwise, we're sort of back to shoving something into URLs, for those cases. Although this is really sort of a web platform issue, it affects the viability of using these options out of the URL, so I want to suggest that we start discussion here.

Let's try to write down the different subresources in the web platform, and see if we can find syntax for them to accept an options bag for further parameters, analogous to what this proposal does for JavaScript.

h/t to @slightlyoff for raising this issue offline

Should type: "js" work?

Not sure if this has been discussed yet, but it occurred to me that we should consider allowing "js" (or maybe the full "javascript") as a valid value for the type key, e.g.:

import {whatever} from "./foo.js" with type: "js";

Which would be exactly equivalent to just:

import {whatever} from "./foo.js";

Pro: Consistency. It could be surprising to developers that specifying the type for "js" is is an error while other module types require it to be specified.
Con: Now there would be two ways to do the same thing since JS is the default.

Thoughts? I'm leaning towards saying that it should be supported but this is not a strongly held opinion.

Should a syntax within module specifiers be used for this data instead?

In comments like WICG/webcomponents#839 (comment) and #3 (comment), various people have proposed that the existing module specifier be used to communicate the module type, perhaps with a scheme-like syntax at the beginning of the specifier.

I wouldn't really be in favor of this approach due to the following downsides:

  1. Various existing interpretations of module specifiers--e.g., URLs and paths--already have very complicated grammars, and it would add additional technical complexity to incorporate
  2. It will take time for existing URL parsers and processers to learn about the new syntax, and given all the manipulation they do, this seems bound to lead to security issues (h/t @slightlyoff for pointing this out)
  3. More importantly, it would be difficult for a human to parse out the module type and to write it
  4. A pseudo-scheme syntax wouldn't cleanly scale to multiple parameters, and a fragment/query parameter syntax would not work as it'd change semantics

What are other people's thoughts here?

Module type and content-type sniffing

I understand that this is up to the host to decide but wanted to make sure we are aware of it.

Browser are able to indentify the mimetype of a resource when not specified by the server using https://mimesniff.spec.whatwg.org.

A server can disable this behavior by sending the following header X-Content-Type-Options: nosniff.

Do we want the type attribute to rely on sniffing? or should we require the mimetype to be correctly send to the client?

Module cache keying semantics

Currently the module cache is typically keyed uniquely by URL.

With this proposal the module identity now seems to include both the URL and the attributes.

Would the expectation be that the module cache keying would be extended to support the attribute identity?

Or alternatively will the URL remain the primary keying with some reconciliation approaches in place? How would those approaches avoid causing indeterminism based on load ordering?

Privilege of a type

I've been wondering if it's worth documenting clearly why we have this type mechanism to ensure it does not get circumvented by types going forward.

E.g., say an "html" module type were added and those modules could themselves not execute script or import other modules. Then if a script execution ability was added, that should warrant a new type as existing consumers might not anticipate that happening.

Make the operator overloading proposal dependent on the import attributes proposal?

The operator overloading proposal has a similar syntax to ideas from this import attributes proposal:

import Decimal from "./decimal.mjs";
with operators from Decimal;

Seems to me like that proposal could align with this attributes proposal, so it might instead look like

import Decimal from "./decimal.mjs" with overloads: Decimal;

or something.

Or would we want with overloads from Decimal; to be separate from imports so that one does not have to actually import overloads to use overloads?

Should we allow string literal as key in condition entries?

import "./foo.json" if { "type": "json" }

As @littledan mentioned in #77 (review)

[It is] for further consistency with object literals

Since JavaScript implementations are encouraged to reject conditions which are not implemented, as of now "type" is the only valid use case. However, type is always preferred over "type" for code size reasons. I lean to disallow it and revisit in the future if we must.

Besides, the literal property name also includes numeric literals. So even if string literal is supported, the literal property name is still a superset of the condition entry key.

Dynamic import should accept multiple arguments

Current syntax:

ImportCall[Yield, Await]:
    import(AssignmentExpression[+In, ?Yield, ?Await])

Proposed syntax:

ImportCall[Yield, Await]:
    import Arguments[+In, ?Yield, ?Await]

Currently import("a", { }) is SyntaxError and it's harder for developers to do "feature detecting" in future. The module attribute proposal is going to add 2nd parameter for ImportCall but developers can't write code like this:

try {
    return await import("./a.js", { attribute: something })
} catch(e) {
    return import("./a.js")
}

Because it is a SyntaxError. IMO it's better to allow more than 1 parameter and throw a runtime error so it will be easier for the developers in the future.

Since the module attribute is just stage 1, make this change in that proposal will be too late. Too many versions of the browser will only accept the exact 1 argument and it's impossible to write the feature detect above.

It's better to extend this syntax asap for the future (and throw at runtime for now) and it won't break anything (because it just makes invalid syntax valid).

Inline vs out of line module attributes

This proposal is all about putting module attributes inline in the JavaScript source text. I believe this is better because it would be annoying to have to switch to a separate resource file to write the module type, and keep these in correspondence. Probably it would only be practical to generate from tools, not write by hand. I'd prefer if we can reduce the number of administrative steps we need tools for.

What do you think? Does anyone think that we put the attributes out of line, in some sort of resource file? If so, where should we put them? (unclear whether import maps would be suitable)

Consider strengthening host invariants

In the current draft, hosts have lots of flexibility:

  • Hosts may ignore or reject unknown attributes
  • Hosts may reject imports of the same specifier with mismatching attributes, or make a separate copy interpreted differently
  • Hosts may expose attributes in import.meta, but not guaranteed

Based on the discussion in openjs-foundation/standards#91 , I think it's important that we consider making more specific requirements in hosts. This investigation would be based on details of concrete hosts and attributes.

Consider more name change fallout from 'if' --> 'assert' rename

#80 changed the keyword from if to assert. There are two other name changes to consider as follow-ups:

  • Rename the proposal from "Import Conditions" to "Import Assertions"
  • Rename ModuleRecord's [[Conditions]] fields to [[Assertions]]

Thoughts/objections/any other changes we're missing?

Avoid adding a new keyword for static imports (`with`)

Wouldn't it be possible to avoid the e.g with keyword for static imports by e.g

// Module
import module from 'url';
// Module with Attributes 
import module from ('url', attrs);

e.g

import module from './file.js';
// equal to
import module from ('./file.js', { type: 'js' });

with the benefit of being kind of similiar to the syntax of a dynamic import?

import module from ('./file.js', { type: 'js' });

import('file.js', { type: 'js' }).then((module) => module)

My apologies if this was already being discussed/proposed before

Type of builtin modules

It seems builtin modules would also want a type associated with them? IDK if this has been thought about but they would be ensured to not go through user-land (provide by language or host?).

import 'builtin:thing' with type:'???';

Attributes divergence

One of the concerns with arbitrary attributes it's a possible divergence in an already divergent ecosystem (tooling configuration are not shared for instance).

For the Babel implementation the plan would be to make it available, via the Babel AST, to plugins. Plugin authors would be free to define their own attributes.

Use cases for module attributes besides module type

The initial use case for module attributes is types, but module attributes had been previously discussed over several years. Some other possible use cases:

  • Fetch parameters -- e.g., various policies, authentication options, etc. Unclear how this would interact with caching modules, or how this would be represented
  • SRI or other integrity markers -- this interacts poorly with caching, and it may be best to have this out of line in some other yet-to-be-designed resource file
  • Marking CJS modules in Node -- this may be a good use of the type field
  • Redirection to find the module within a WebPackage -- I don't know the current state of all this; maybe @nyaxt could clarify if this makes any sense
  • Built-in module polyfilling -- now that import maps are not proposed to serve this purpose (initially, a complex module specifier was proposed, so this would be analogous)

For each of these, there would be a significantly greater amount of investigative work to make a real proposal. This repository does not aim to do that work, but I'd like to just understand a bit more about the broader space. Some questions to discuss in this issue:

  • Does anyone have any other use cases besides type, or comments on the above?
  • Do strings make sense as values for the additional possible module attributes?
  • Do we really think it's likely that we'll eventually want any of them, or will type alone probably suffice?

Unmixing type from other attributes

Having read through the other issues, I note that this part of the proposal isn't exactly accurate:

Another option considered and not selected has been to use a single string as the attribute, indicating the type. This option is not selected due to its implication that any particular attribute is special; even though this proposal only specifies the type attribute, the intention is to be open to more attributes in the future.

As currently envisioned, the type option is indeed special. It defines the loader to use to transform the resource into its imported form. Any other additional attributes are really arguments for that loader.

I mean, the reason there's a need for type in the first place is that we can't trust that a network request for a .json file actually resolves to be a JSON file; type is fulfilling the role that file extensions have traditionally taken.

Making a keyword-based type explicitly special would also provide a way for those attributes not to need to be defined inline and separately for each import, by providing a key to use in a central registry for other attribute defaults.

Finally, the simplicity of a string as value and the power of a with key:value set of attributes are not mutually exhaustive. Why not have both?

import.meta.registerType('custom', loadFunction, { some: 'attributes' })

import foo from './foo' as 'json'
import foo from './foo' as 'json' with foo: 'bar'
import foo from './bar' as 'custom'
import foo from './bar' as 'custom' with some: 'override'

This sort of syntax would provide both simplicity and flexibility, allowing for often-used attribute sets to be controlled centrally, while maintaining the liberty of defining specific values for a single instance.

Overall, the role of module attributes seems rather similar to that fulfilled e.g. by Webpack loaders. This is what the Webpack docs say on this now, based on a few years of experience:

Use module.rules whenever possible, as this will reduce boilerplate in your source code and allow you to debug or locate a loader faster if something goes south.

<script> type attribute and WorkerOptions dict type are a little overloaded now

<script>'s type attribute is a little odd with this change.
<script type="module"> means a JavaScript module, while <script type="webassembly"> also means a module, but a different type of module. This is not super intuitive, but I don't really see a backwards-compatible workaround. I guess this can be justified by saying that there is no such thing as a 'classic' WebAssembly script (is that right?) so the distinction is clear in context.

Same thing with Workers; https://github.com/littledan/proposal-module-attributes#worker-instantiation conflicts a bit with the current way that the worker options dict is used to decide whether a Worker should be classic or module: https://html.spec.whatwg.org/multipage/workers.html#worker-processing-model

const worker = new Worker('resource.json', {type:'module'});

or

const worker = new Worker('resource.json', {type:'classic'});

Now we would also have

const worker = new Worker('resource.json', {type:'webassembly'});

Which is a little confusing because it is also a module, just not a JS one.

I don't see a good way to address this as changing <script type="module"> is obviously a non-starter. I thought I'd make a note of the issue though, as this is likely a criticism that we'll have to speak to.

Change keyword «with»

Keyword with already exists. Maybe better define MIME type like this:
import module from 'module' as 'json'?

Verbosity/synchronization challenges

Both URL band and source text band solutions require potentially large duplication of these attributes. This is a challenge as it increases shipping size for web, which is the primary target of this proposal, and means keeping all location of the attributes in sync. Additionally, if the attributes are in multiple locations due to using a solution that allows/requires such, it should clarify how mismatches are resolved.

Example given a URL band solution:

// x.mjs
import a from 'import+json://foo.json';
import b from 'import+javascript://foo.json';
import c from 'import://foo.json';

// y.mjs
import d from 'import+json://foo.json';

Could all 3 in x.mjs succeed? They have separate request URLs in web spec terms, so it seems like they would resolve independently.

I think solving for the verbosity impacts synchronization of these in non-trivial ways. E.g. SRI being defined per source text would mean invalidating the whole dep graph when an SRI changes. Out of band solutions/single location solutions could preserve parts of the dep graph that were unchanged still.

I think having more attributes and their use cases discussed would increase the visibility of this concern.

Should we allow more than just strings as module attributes?

We've talked about generalizing to static-looking object literals in general. A more scoped generalization, proposed by @Jack-Works , would be just allowing Numbers, BigInts, null, booleans, and (maybe?) undefined.

I want to propose that we could go to Stage 2 with just strings, and consider generalizations between Stage 2 and 3. In particular, I believe the generalizations @Jack-Works proposes would not be core to the data model or other fundamental decisions that we need to assess for the viability of the proposal overall.

Multi-typed modules

Do we have concerns about module upgrade paths where you are intending to transition from 1 type to another and/or are not concerned with the target type?

import '//test.local/config' with type="json" || type="wasm"

etc.

dynamic import should accept an object

Hello. As a developer who has been working with modules, I'm very confused seeing the current import("file.json", "json") dynamic import syntax, because I am used to existing patterns used extensively on the web, where a more complex "options" object with explicit flags is passed in. What I would expect is something more like:

import("file.json", {as: "json"})

This is much more explicit & clear to me. It also:

  1. Matches the static import and worker syntax more clearly, re-using the as terminology.
  2. Allows for further potential extension of the dynamic import syntax if the need arises.

The immediate example I can see on the web for passing in a string argument is EventTarget#addEventListeners's old third useCapture argument. That worked fine for years. But that proved to be insufficiently flexible in the end, so now addEventListener accepts an options argument in it's place.

I can't think of a lot of other places on the web where naked string options are passed in. I am significantly afraid that some day we may need to pass in other options to dynamic import, and that, in the current form, this proposal would roadblock the language's further development. For consistency sake, for clarity sake, & for extensibility sake, I would like dynamic import to accept an options second argument such as {as: "json"}.

Potential use case: mocking internals

A thought I had, perhaps module attributes could be used to mock unexposed parts of modules. This is a serious current gap for testing and something we are trying to figure out how to best solve in node core... this would not have to be standardized in ecma262 imho, but would be a good one to explore

import feature from './feature.mjs' with {
  internal: {
    express: './path/to/mocked/express.mjs'
  }
}

edit:

dynamic example for completeness

const feature = import('./feature.mjs', {
  internal: {
    express: './path/to/mocked/express.mjs'
  }
});

Using attributes to support different types of runtime ES Module implementations.

Currently, there are two types of ES Module implementations (as far as I'm aware):

  • browsers' ESM
  • Node ESM

Tools like NW.js currently have an issue, whereby it is possible to use ESM in the browser context (because it wraps Chromium, so it comes for free with browser-based ESM).

However, to import a Node modules in NW.js, one is required to require (hehe) any Node module that they want to import.

For example:

const fs = require("fs") // Node-based import using require
import value from "./file.js"; // regular browser-based ESM import here

The NW.js user can not use import to import Node modules.

Probably a similar issues exists with Electron.

So what I'm thinking is maybe there can be a way to disambiguate between module implementations using import attributes. Then it would be possible to patch a browser context engine (f.e. Chromium), to allow things like this using an official syntax:

import fs from "fs" with { type: "node-esm" }; // Node-based ESM import
import value from "./file.js"; // regular browser-based ESM import.

Of course, the naming (with, type, node-esm) is from a shed full of bikes to choose from.

The alternative for a project like NW.js would be that it would need to patch the browser engine to make it so if it encounters an identifier like fs, it will delegate to Node ESM for that, otherwise follow the code path of normal browser-based ESM.

PROPOSAL to cache imported modules

currently on Deno, it's done like this:

// main.ts
import "https://deno.land/std/examples/welcome.ts";  
// output: Welcome to Deno 🦕

the file is cached in .cache/deno/gen/https/deno.land/std/examples/welcome.ts.js

my ideia is, the main.ts above wouldn't be cached that way

to cache would be like this:


to cache fragmented on .cache/deno/deps you could do like this:

// main.ts
import "https://deno.land/std/examples/welcome.ts" cachefragmentedon ".cache/deno/deps/";
// output: Welcome to Deno 🦕

this would be cached on .cache/deno/deps/https/deno.land/std/examples/welcome.ts.js


to cache on a folder inside .cache/deno, you could do like this:

// main.ts
import "https://deno.land/std/examples/welcome.ts" cacheon ".cache/deno/deps/module/version/";
// output: Welcome to Deno 🦕

this would be cached on .cache/deno/deps/module/version/welcome.ts.js


to cache fragmented in other folder out of .cache/deno/deps, would be:

// main.ts
import "https://deno.land/std/examples/welcome.ts" cachefragmentedon "/home/user/folder/";
// output: Welcome to Deno 🦕

this would be cached on /home/user/folder/https/deno.land/std/examples/welcome.ts.js


to cache in other folder out of .cache/deno, would be:

// main.ts
import "https://deno.land/std/examples/welcome.ts" cacheon "/home/user/folder/module/version/";
// output: Welcome to Deno 🦕

this would be cached on /home/user/folder/module/version/welcome.ts.js

Semantics of this feature in Node.js

Note: The semantics of this proposal in Node.js would be determined by Node collaborators and the Node Modules Team, not in this repository. This issue is purely for requirements-gathering/brainstorming.

A lot of the discussion in the issues seems to relate to how this feature would integrate into non-Web environments. For concreteness, I'm going to talk about how this might integrate into Node.js, since I know slightly more about it than other non-Web environments (but I am no Node expert). I see two general paths for the interpretation. (Note, these are not the only possible paths, they're just two that I thought of to initiate discussion)

Option A: Maximizing for similarity with the web

If we want to be maximally compatible with the web -- that is, code written for Node.js would work on the Web without a build step to handle this aspect of semantics -- then we could simply use the logic in #24, but with s/MIME type/file extension (and possibly more information in package.json)/. The file extension used here would be after some resolution is done, as described in #4, so it may not be what's textually included in source. This would mean that importing JSON modules would require type: "json" to be textually in JS source that's used in Node.js, which differs from traditional ES6 module -> CJS transforms (which of course didn't have this syntax, but JSON modules were present anyway).

Option B: Maximizing for terseness/non-redundancy

On the other hand, we could make type: optional for non-JS all module types in Node.js. The semantics would be based on the template of Option A, but with the following change: if no type: is provided, then there is no check done, and processing is based just on the file extension (and possibly more information in package.json). This would mean that a build step is expected to add the with type: "json" or whatever else when building for the web.

How this would interact with other alternatives for where to put the attributes

If the attributes are put in an out-of-band file, or in the module specifier, or in a single string, Options A and B are still both available. They make different tradeoffs about web compatibility vs lack of redundancy. This proposal would permit Node.js and other environments to make this decision themselves, regardless of the choice of format for where to put the metadata.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.