dop251/goja_nodejs

Nodejs compatibility library for Goja

goja_nodejs's Issues

Roadmap

goja#76 is related to this issue. I'd like to start contributing some important node libraries just as functionality from fs, path, etc. I noticed you haven't answered the Windows filepath issue PR, so before I made a PR for a roadmap / priority list, I wanted to first make sure you are still interested in maintaining this library at least as far as accepting PRs goes. I am interested in doing some work but lack the experience you have.

On a somewhat related note, I would like to start a goja_browser project which has dom functionality. If you are interested in accepting PRs, I think you are more qualified to own the repo.

Prioritise native modules

Hi and thanks for this wonderful project!! :)

I've been using it in our framework for a while and I saw that you downprioritized the resolving of native modules. Is it possible to do this configurable?

I've done a ugly hack to achieve this.

Cheers,
Mario :)

eventloop: Aux jobs sometimes run on stopped loop

I believe there is a race condition within the eventloop package which in certain conditions causes aux jobs queued via loop.addAuxJob to be run after the loop is stopped via Stop.

I can fairly reliably reproduce the race with the following test added to eventloop/eventloop_test.go:

func TestRunOnStoppedLoop(t *testing.T) {
	t.Parallel()
	loop := NewEventLoop()
	wg := &sync.WaitGroup{}
	wg.Add(1)
	go func() {
		defer wg.Done()
		for !t.Failed() {
			loop.Start()
			<-time.After(10 * time.Millisecond)
			loop.Stop()
		}
	}()
	wg.Add(1)
	go func() {
		defer wg.Done()
		for !t.Failed() {
			loop.RunOnLoop(func(*goja.Runtime) {
				if !loop.running {
					t.Fatal("running job on stopped loop")
					return
				}
			})
			<-time.After(10 * time.Millisecond)
		}
	}()
	done := make(chan struct{})
	go func() {
		wg.Wait()
		close(done)
	}()
	select {
	case <-done:
	case <-time.After(2 * time.Second):
	}
}

With example output looking like:

$ go test -run TestRunOnStoppedLoop
--- FAIL: TestRunOnStoppedLoop (0.76s)
    eventloop_test.go:297: running job on stopped loop
FAIL
exit status 1
FAIL	github.com/dop251/goja_nodejs/eventloop	0.766s

I believe this happens when the following interleaving of events occurs within a single loop iteration of loop.run:

1. The loop.run goroutine runs the job queued by the loop's Stop method here, thus setting loop.running to false.
2. Another goroutine queues the aux job which checks if loop.running here.
3. The loop.run goroutine runs the aux job which checks if loop.running here. Since loop.running has been set to false in step 1, the test fails.

I'm running the latest master for both goja and goja_nodejs.

join contribution

Would love to contribute and join the development. Beginner with golang but can work fairly with nodejs, python. can i?

EventLoop: runtime management

Issue

I am creating a function fetch for creating http requests. But I need access to the runtime to create a Promise object.

Any ideas on how I can deal with this? I tried using the RunOnLoop function inside of the fetch function, but it was never scheduled.

Which makes since since the fetch function needs to finish executing before the runtime can schedule anything else.

Potential Fix

Add GetRuntime() *Runtime, which panics when the event loop is not running?

goja.Parse

github.com/dop251/goja_nodejs/require

/Users/zfd/go/pkg/mod/github.com/dop251/[email protected]/require/module.go:149:18: undefined: goja.Parse
/Users/zfd/go/pkg/mod/github.com/dop251/[email protected]/require/module.go:149:38: undefined: parser.WithSourceMapLoader

Misleading error message when registry is not enabled

Discussed in dop251/goja#404

^{Originally posted by pgundlach July 12, 2022}
I came across dop251/goja#116 to enable console.log(), but when I use the command, I get an error

package main

import (
	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/console"
)

func main() {
	runtime := goja.New()
	console.Enable(runtime)
}

returns (go run main.go)

go run main.go 
panic: TypeError: Please enable require for this runtime using new(require.Require).Enable(runtime)

goroutine 1 [running]:
github.com/dop251/goja_nodejs/require.Require(0x140001ba960?, {0x102737917, 0x7})
        /Users/patrick/go/pkg/mod/github.com/dop251/[email protected]/require/module.go:198 +0x144
github.com/dop251/goja_nodejs/console.Enable(0x14000198000?)
        /Users/patrick/go/pkg/mod/github.com/dop251/[email protected]/console/module.go:75 +0x30
main.main()
        /Users/patrick/prog/go/goja/main.go:10 +0x38
exit status 2

but

new(require.Require).Enable(runtime)

does not seem to be valid Go (require.Require (value of type func(runtime *goja.Runtime, name string) goja.Value) is not a type).

What can I do to load the console module?

Support for `process.env`?

I have a JavaScript Library, namely immer.js that I would love to run in goja. Judging by the looks of it immer prefers to work with Proxy objects, but can explicitly made ES5 compatible. It's default index.js however expects to be run in a node.js compatible environment:

'use strict'
if (process.env.NODE_ENV === 'production') {
  module.exports = require('./immer.cjs.production.min.js')
} else {
  module.exports = require('./immer.cjs.development.js')
}

This causes a ReferenceError: process is not defined at main.js:5135:7(1) when I try to load a file that has been put together by Rollup.

I'm explicitly not asking to implement all of the node Process API, only process.env. From my (very limited) understanding this would require an object named process with a env property to be made available in the global namespace. The values could probably be retrieved from os.Environ().

Would such a PR be accepted and a good "first" kind of issue to explore the codebase?

Fatal Error: Concurrent Map Writes

When registering a module concurrently, it panics.

github.com/dop251/goja_nodejs/require.RegisterCoreModule(...)

goja_nodejs/require/module.go

Lines 240 to 246 in 27eeffc

	func RegisterCoreModule(name string, loader ModuleLoader) {
	if builtin == nil {
	builtin = make(map[string]ModuleLoader)
	}
	name = filepathClean(name)
	builtin[name] = loader
	}

Is goja-node still being maintained

Hello, I am the author of airoot-uisys, referring to your project. Goja is a great project.

I want to use "goja-nodejs" to run "uglifyJS",uglifyJS is a nodejs project.

I would like to know whether the goja-nodejs project is mature now and whether it can be supported in the long time。

Docs Request

It would be better if there is documentation explaining how to use theses modules. Or the documentation actually exists but I didn't find it? (I have viewed the docs in pkg.go.)

I enjoy the clear and intelligible examples in the README of goja, and look forward to similar examples in these modules.

I would like to allow users to run 'setInterval' and 'setTimeout'.

And many thanks to your great work about goja!

No way to interrupt a script running EventLoop

It doesn't seem like the Go code can interrupt a running job. The example in https://github.com/dop251/goja#interrupting cannot be replicated when a script is in an infinite loop inside of EventLoop.

GoError: Invalid module at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)

I have a very basic program, trying out this library before actually incorporating it into my project.

`script, err := os.ReadFile("algos/sample_plugin/sample_plugin.js")
if err != nil {
panic(err)
}

// Read the program into memory
program, err := goja.Compile("sample_plugin", string(script), true)
if err != nil {
	fmt.Println("Error compiling JavaScript program:", err)
	return
}

vm := goja.New()
vm.SetFieldNameMapper(goja.TagFieldNameMapper("json", true))
registry := new(require.Registry) // this can be shared by multiple runtimes
registry.Enable(vm)

// now run the program
_, err = vm.RunProgram(program)
if err != nil {
	panic(err)
}

predict, ok := goja.AssertFunction(vm.Get("predict"))
if !ok {
	fmt.Println("could not get predict")
	return
}`

When I try to run this, I get the following errors:
GoError: Invalid module at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)

Things I have tried:

1. Remove and Re-install both goja and goja_nodejs
2. Update using go mod tidy.

Can some one please help me?

Is there any plan to implement http

NodeJS URL Support

We've been using the goja runtime for a bit now. We've required the need of the URL NodeJS implementation. We figured we might as well take a stab at it. We managed to implement most of the API in the docs. I've created a PR with my implementation hoping we could give back to the repo. Let me know what you guys think!

This isn't much of an issue, but I wasn't able to assign reviewers to the PR, so I was hoping to gain visibility here. Hope this helps anyone in need.

PR: #40

setInterval setImmediate

setImmediate if of the nodejs cheak，It has a higher priority，but goja use select，weakup or jobchan No priority， it is bug？

Event loop gets stuck unexpectedly

Consider the following case, which I temporarily added to eventloop_test.go to ensure it could be reproduced outside of my project:

func TestEventLoop_BadCase(t *testing.T) {
	t.Parallel()
	const SCRIPT = `
	let aTimer;
	function a() {
		if(aTimer) {
			clearTimeout(aTimer);
		}
		console.log("ok");
		aTimer = setTimeout(a, 1000);
	}
	a();`

	prg, err := goja.Compile("main.js", SCRIPT, false)
	if err != nil {
		t.Fatal(err)
	}

	loop := NewEventLoop()
	loop.Start()

	loop.RunOnLoop(func(vm *goja.Runtime) {
		vm.RunProgram(prg)
	})

	time.Sleep(20 * time.Second)
	loop.Stop()
}

What I expected to see: "ok" being printed to the console roughly 20 times.
What actually happens: "ok" only gets printed three times and the event loop gets unexpectedly stuck after that.

If the call to clearTimeout is removed, the code works as expected.

Better example code

The provided example is not very concise or easy to understand (it actually doesn't even provide output, nor log an error in case the module can't be found).

Would be great if we could come up with some better examples.
I'm reworking the original example provided in the README RN.

Alas, I've yet to figure out how to actually invoke console.log.
The comments say it should be activated by default but this code fails

package main

import (
	"fmt"

	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/eventloop"
)

func main() {
	runtime := goja.New()
	loop := eventloop.NewEventLoop()
	// req := registry.Enable(runtime)

	loop.Run(func(vm *goja.Runtime) {
		fmt.Println(runtime.RunString(`
        console.log("Hello World")
        `))
    })
}

Error logged @ stdout: <nil> ReferenceError: console is not defined at <eval>:2:9(0)

Need community channels

I'd like to propose that a community be built around this for all contributors to communicate, as well as for others to asked question.

Was thinking discord.

`require('./util')` will return the "native module" that register by `goja_nodejs/util/module.go`, expect is my `util.js` file.

This code will show the problem

import (
	"github.com/dop251/goja"
	"github.com/dop251/goja_nodejs/console"
	"github.com/dop251/goja_nodejs/require"
	"testing"
)

func TestRegistryUtil(t *testing.T) {
	vm := goja.New()

	require.NewRegistry().Enable(vm)
	console.Enable(vm)

	v, err := vm.RunScript("root.js", "require('./util')")
	if err != nil {
		panic(err)
	}

	t.Logf("%+v", v.Export())
}

util.js

exports.default = "1";

It will print map[format:0x13b84c0] instead of map[default: 1].

require: loading of directory modules is broken in windows

Loading a directory module (i.e. require('dir')) currently fails with The handle is invalid. in windows. I believe it's a problem with go, raised an issue: golang/go#43322

require need add current runtime's parser options

when require a javascript file with source map lint, but source map is not exists, the require failed.

maybe add a option to disable source map

[Feature]: Support Path module

Background

Hey. I notice this project. I'm interested in some of these modules. Are you have any plan to support
the path module?

require directory not working

require directory not working as in:
https://stackoverflow.com/questions/5364928/node-js-require-all-files-in-a-folder

I need await/sync syntax, can give some suggestions?

Can native crypto packages be supported?

problem background

i have install a node package [uuid](https://www.npmjs.com/package/uuid), the uuid package using native package crypto, when i run script bellow, a error has occur:
GoError: Invalid module
at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)
at node_modules/uuid/dist/rng.js:8:45(22)
at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)
at node_modules/uuid/dist/v1.js:8:42(22)
at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)
at node_modules/uuid/dist/index.js:61:40(116)
at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)
at :18:19(13)

const uuid = require('uuid');
console.log(uuid.v4());

EventLoop : Wait for all jobs to finish

Is it possible to add a function to the eventloop to allow for waiting for all jobs to finish.

Need contribution guidelines

[Feature Request] Add a option for disable the `require` method when using EventLoop

I want to use eventloop, but I don't want the method require because I don't want the script can access my filesystem.

I can make a source loader that will always return an error, but the built-in modules such as node:console could still be required.
It's not a such big issue, but I already implement my own console object, so I don't want the script use the other.

How can i get details error message when a package is not found?

question background

when i run the script below, i got a error message:
GoError: Invalid module at github.com/dop251/goja_nodejs/require.(*RequireModule).require-fm (native)
but i want detailed error information, such as which package does not exist.
thanks!

const xx = require('xx')
const notfound = require('notfound package')
console.log(notfound)

Performance issue due to native modules being lower priority

I have a custom source loader which loads modules from a remote source and upon updating this package spotted an issue when loading native modules.

Due to #36 it now tries to download the module from the remote source before attempting to load the native module and this log should highlight the issue with that:

JS script:

console.log('hello world')

Log output:

time="2023-01-21T13:00:26Z" level=info msg="downloading node:console..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:console.js..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:console.json..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:console/package.json..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:console/index.js..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:console/index.json..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:util..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:util.js..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:util.json..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:util/package.json..."
time="2023-01-21T13:00:26Z" level=info msg="downloading node:util/index.js..."
time="2023-01-21T13:00:27Z" level=info msg="downloading node:util/index.json..."
2023/01/21 13:00:27 hello world

It's making 12 HTTP requests before attempting to load the native module!

Possible workarounds could be either making the priority configurable through an option on require.NewRegistry, or introducing a sentinel error that tells the resolver to stop looking and move on to attempting to load the module from the native modules. My preference would be implementing both! Happy to submit a PR if this is something you'd merge.

evenloop.Run hang with setInterval

There is a setInterval that has not been cleared, it will be hang at evenloop.Run, see the code below:

func main() {
	loop := eventloop.NewEventLoop()

	var test goja.Callable
	loop.Run(func(vm *goja.Runtime) {
		vm.RunString(`
// Simulated users forget to clear setInterval
function leak(){
	setInterval(() => {
	},500)
}

async function test(){
	leak()
	return await new Promise((resolve, reject) => {
		setTimeout(() => {
			console.log('done')
			resolve('done')
		}, 1000)	
	})
}
`)
		test, _ = goja.AssertFunction(vm.Get("test"))
	})

	var (
		result goja.Value
		err    error
	)
        // hanged this line
	loop.Run(func(vm *goja.Runtime) {
		result, err = test(nil)
	})
	if err != nil {
		panic(err)
	}
	if vp, ok := result.Export().(*goja.Promise); ok {
		fmt.Println("result", vp.State(), vp.Result())
	}
}

How do I deal with this?

could you help me understand the example on README

Hi, I copied the example provided in README and ran it. But I got two error "Invalid module".
Not sure where got wrong, Could you help me make it work？

Selfwrited Mudule Loader.

I want to download module from database when it is needed. Is it possible?

setImmediate support

I think one simple and useful nodejs compatibility "win" would be to implement setImmediate/clearImmediate. Looking at the code, the "aux jobs" mechanism already seems to implement exactly the semantics of setImmediate (source):

When multiple calls to setImmediate() are made, the callback functions are queued for execution in the order in which they are created. The entire callback queue is processed every event loop iteration. If an immediate timer is queued from inside an executing callback, that timer will not be triggered until the next event loop iteration.

and it is easy enough to make setImmediate available to the goja environment without changing the eventloop implementation, using RunOnLoop. And clearImmediate can probably also be implemented by checking some condition relative to each Immediate on the RunOnLoop callback before calling the JS callback, but this feels like it could be slightly more optimized if we could remove entries from loop.auxJobs instead. Do you think this warrants being part of the eventloop core, or is it best to just implement it outside the package?

how to return valuein eventloop？

it ia can return the value use vm.tovaule