Comments (18)
@WebReflection @twop closing - v0.25.0+ now ship ESM builds and work fine via esm.sh CDN. All the included build variants use a separate WASM file, although single file variants are available from NPM separately. See the changelog entry here: https://github.com/justjake/quickjs-emscripten/blob/main/CHANGELOG.md#esmodules--browser-support
from quickjs-emscripten.
I’m happy to accept a PR addressing this issue, but don’t have time to work on it myself.
from quickjs-emscripten.
Yeah, I tried web-ifying about a year ago but got discouraged wading through all the issues you cited. My takeaway is that if I want the library to “just work” everywhere I need to start bundling out separate ESM and CJS bundles with Rollup or something. At the time the Emscripten module build mode didn’t work in Node either. Maybe these days it’d be possible to keep platform set to everything but turn on module mode and it would be happier, I think Emscripten made some progress on that side. Still it’s a shlep.
Anyways glad you got a fork working without too much trouble. I’ll link to it from my README.
from quickjs-emscripten.
Thank you for the issue.
It’s helpful to know that the FS issue messes with people. For what it’s worth, I solve this in webpack following this stack overflow question: https://stackoverflow.com/questions/59487224/webpack-throws-error-with-emscripten-cant-resolve-fs
You are encouraging ESM output in the Node module. I would like to do so, but have little experience configuring ESM tools. It would help to set up an ESM example project in ./examples to demonstrate a typical ESM web app solution.
I have some other questions as well:
I published the wasm file to npm, but the required step to make it available by ./quickjs-emscripten-module.wasm is manual and tool dependent (I just copied the wasm file to my dist folder). To my knoweledge emscripten doesn't allow to specify wasm file location (wasm-pack does).
Do you mean you copied the .wasm file into quickjs-emacripten’s dist folder as part of the build process of the npm package, or that the consumer who installs the npm package has to copy the .wasm file as part of their build process? I don’t want consumers to need to copy files or change build tool config to use this package. Is there a work around there?
I'm thinking that expanding the api surface area for wasm file itself might get tricky overtime. I usually use a different approach with rust/wasm.
Can you explain more what advantages you see of building a command struct, vs the function calls that the project currently uses? What is the specific advantage does this post with regard to resource use? Would you expect the library consumer to change significantly? When you say bookkeeping is expensive, are you referring to the Lifetime
class?
from quickjs-emscripten.
Goals:
• esmodule output ready to be consumed on the web
I understand why this is wanted
• shipped as the latest ecmascript target.
I understand why this is wanted a bit less. I assume, cruft-free and your build tools can handle it
• ship actual .wasm file
This one I understand the least. I would think that avoiding an extra .wasm file actually makes the users life easier. Can you explain this?
from quickjs-emscripten.
Let me try to comment on your latest questions
Goals: • esmodule output ready to be consumed on the web
I understand why this is wanted
It is possible to set up export map and configure the output to be consumed as an es module and cjs. https://docs.skypack.dev/package-authors/package-checks#export-map
Has some good info on this.
• shipped as the latest ecmascript target.
I understand why this is wanted a bit less. I assume, cruft-free and your build tools can handle it
According to some studies es5 js target is one of the major perf issues on the web (https://www.youtube.com/watch?v=lFd0tfYWGJ8). Shipping a library as es5 you are making a target choice for the developers, because they can't transpile it back es5 -> es2018 (yet). Also, WASM is a relatively new thing, browsers that support WASM at least support es6 (things like classes)
• ship actual .wasm file
This one I understand the least. I would think that avoiding an extra .wasm file actually makes the users life easier. Can you explain this?
There are several benefits to this:
- size is smaller than base64 version (or, so I assume?)
- browsers can cache compiled version of wasm files. Meaning that if the browser already compiled wasm file to native machine code it can just reuse the result with instant startup in the next visit. I believe Firefox is already doing it.
The downside of having a separate WASM file is that you somehow need to integrate that into your tooling. I don't know what is the right approach there (I don't have wasm libraries published) but wasm-pack has several modes of generating glue code but all of them are leveraging a separate .wasm file: https://rustwasm.github.io/wasm-pack/book/tutorials/npm-browser-packages/building-your-project.html
from quickjs-emscripten.
I'm thinking that expanding the api surface area for wasm file itself might get tricky overtime. I usually use a different approach with rust/wasm.
Can you explain more what advantages you see of building a command struct, vs the function calls that the project currently uses? What is the specific advantage does this post with regard to resource use? Would you expect the library consumer to change significantly? When you say bookkeeping is expensive, are you referring to the
Lifetime
class?
I was mostly referring to the conversion between js and wasm values. This article goes in depth about this issue: https://hacks.mozilla.org/2018/10/calls-between-javascript-and-webassembly-are-finally-fast-%f0%9f%8e%89/
I think the main advantage that I see is that in the model of commands you don't need to store a "handle", at least in some cases.
// before
const world = vm.createString('world')
vm.setProp(vm.global, 'NAME', world)
world.dispose()
//after
vm.setProp(vm.global, 'NAME', vm.newString('world'))
// vm.newString('world') just produces a desc of a string that will be GCollected as usual
If the value world
is consumed immediately (which is often the case) then there is no need to manage object lifecycles between two runtimes. The only case (in my opinion at least) when you do need to keep the handle alive when it is important to pass the same reference to different places (like an object or function references).
Like so
// pseudocode
const objHandle = vm.newObj({...})
const a = vm.newFunction('a', () => objHandle));
const b = vm.newFunction('b', () => objHandle)); // <-- the same reference
// inside vm
if (a() === b()) { ...}
In my experience working with WASM it never came up. Thus, I think transferring object by values is simpler and less error prone.
This is essentially the same difference between Deno and Node. Deno uses a single binary method doStuff(arrayBuffer)
to talk between rust and V8. At least that is my understanding based on one of the talks they gave.
Hope that helps.
from quickjs-emscripten.
Are there any updates on this? I'd like to use a .wasm/latest-ecmascript version of the library too, and @twop's is a few commits behind this repo (which gets updated regularly, which is great, but which also means that @twop's fork will usually lag behind).
from quickjs-emscripten.
@71 I'd welcome any contributions on these topics. Here's the situation:
.wasm file
I discussed this with @carlosdp a few days ago, he may be working on it already.
I think the model we should follow is how sql.js does this, by providing a locateFile
option when initializing the library. See an example here: https://github.com/sql-js/sql.js/#usage which uses the underlying Emscripten feature described here: https://emscripten.org/docs/api_reference/module.html#Module.locateFile
Ecmascript Modules syntax
I like the idea of producing a ESM build as a secondary output included in the package. I think it's possible to provide this using a field in package.json, or even just a separate entrypoint file eg import { getQuickJS } from 'quickjs-emscripten/esm'
. I don't know much about this subject; I haven't used ESM yet.
Compatibility
- This library should easy to use with NodeJS 12+ without the consumer needing to figure out
locateFile
or how to transpile-down ESM in order to importquickjs-emscripten
. - It would be nice to have the same
const QuickJS = await getQuickJS()
work in both browser and node, and makelocateFile
optional to opt-in to loading a .wasm file; but that may not be possible while playing well with bundlers. I think it's okay to regress in this area for performance and/or compatibility improvements.
from quickjs-emscripten.
I haven't started on this yet, been prioritizing other things, but want to get to it soon. That said, I think maintaining backward-compatibility with current functionality should be fine, at least for my use-case. It won't help with bundle size, sure, but the issue I'm having is with loaded memory, so that would be solved even if the non-locateFile
option with the binary embedded still exists.
from quickjs-emscripten.
Same error about fs
here ... emscripten ships its own FS so it'd be great to avoid needing node:fs
entirely and use what other WASM based interpreters use. If you'd like to grab that at runtime here there's just that: https://github.com/WebReflection/fs#readme
from quickjs-emscripten.
@justjake I might just use this module and re-package it in a way that fs
is never used or it's just ignored but beside that I wonder if there's any live demo in-browser that actually works and doesn't have issues with the FS import ... any pointer would do, thank you!
from quickjs-emscripten.
I use nextjs, you can use these webpack settings to fix the issue; most bundlers should have a similar setting: https://stackoverflow.com/questions/59487224/webpack-throws-error-with-emscripten-cant-resolve-fs
There should be a webpack demo in this repo, is that broken?
from quickjs-emscripten.
My website has an online demo, but isn’t open source https://jake.tl/projects/quickjs-emscripten
from quickjs-emscripten.
I'd like to have emscripten FS instead, so that files can be pre-fetched, as example, and the code can use import stuff from './path.js'
within the evaluated code. If passing those flags to EMSCRIPTEN solves, what does it take to have a build in here that produces also Web ports of QuickJS? A fork could be an answer but it would lack inevitably behind for a reason or another ... would that PR be considered?
from quickjs-emscripten.
If you’re brave enough to get into the Makefile and add it as another build variant I’d be happy to accept. Note that you can already (without Emscripten filesystem) shim import and implement whatever module loading system you need my examples use node FS but you can use a Map etc to implement the module loading code.
from quickjs-emscripten.
to provide little extra context, the polyscript (fairly new) project works lovely with Pyodide, MicroPython, but also Wasmoon (Lua) and ruby-wasm-wasi ... and I am pretty sure with a type="quickjs-emscripten"
things will be even more interesting for its users, as that unlocks tons of "sandboxed" use cases with extreme ease out of the box. Pyodide and microPython both ship with Emscripten FS and we can pre-fetch files and let them use from module import ...
and it works great so that I'd love to see that happening with QuickJS too. End of the fairy-tale :-)
from quickjs-emscripten.
@justjake the module is up and running https://www.npmjs.com/package/@webreflection/quickjs-emscripten
something I've observed and the reason I won't file a PR:
- to make it work on Web I should copy all WASM flags and drop just some ... but ...
- the whole TS + CJS story here is very hostile even for bun bundle + stuff breaks in prod if I enforce ESM in emscripten ... this means the PR would be massive:
- it would use real standard modules resolution with
.js
at the end of any import - it can't use
require
anywhere ... orcreateRequire
... - it would drop anything that is based on (deprecated) TS or (awkward on Web) CJS modules systems
- it should change the
package.json
type
field asmodule
breaking lot more stuff than it fixes - there's
yarn
all over the place and I think these daysyarn
is really not needed ... FWIWI I've usedbun
asnpm
was complaining about broken dependencies - there are broken or outdated dependencies
- the version of Emscripten is also pretty outdated
- it would use real standard modules resolution with
Similar to you, I don't have time to provide such PR so I'll just try to keep my fork updated and, on that regard, it looks like your version of uickjs is also outdated so I suggest you run update-quickjs
in the future or I'd be merging older version and need to do that myself after rebase.
edit P.S. my fork and branch that builds the module is here: https://github.com/WebReflection/quickjs-emscripten/tree/wasm-web and the module is published from here https://github.com/WebReflection/quickjs-emscripten/tree/wasm-web/esm
from quickjs-emscripten.
Related Issues (20)
- Evaluating a module (any code that uses import) always returns undefined HOT 6
- Module "fs", "path", and "crypto" have been externalized for browser compatibility HOT 2
- Tests fail on s390x machines HOT 5
- Proxy log receiver blocking error HOT 1
- Async Regression "TypeError: Object not disposable" in v0.26.0 HOT 4
- wish to add a new api (QuickJSHandle.keepAlive) HOT 2
- `getQuickJS()` callback never resolves? HOT 3
- [vite - MIME type Error]It's cannot work in vite HOT 2
- How to access wasm module's Memory? HOT 3
- Feature request: Expose GetModuleExport APIs HOT 4
- Timeout not working HOT 6
- [feature request] Upgrade to v2024-01-13 HOT 3
- TypeError: Failed to construct 'URL': Invalid URL HOT 7
- Loading WasmMemory into module HOT 11
- Determinism of the library HOT 2
- Distributed packages have no license HOT 2
- `evalCode` won't return error in ES module mode HOT 3
- Can't run cloudflare example HOT 1
- `Out of bounds memory access` Error only on ARM Safari HOT 10
- Having troubles with ArrayBuffer HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quickjs-emscripten.