fengari-lua/fengari-interop

Fengari <=> JS Interop

JS library for Fengari

Fengari is a lua VM written in Javascript. Its implementation makes use of the JS garbage collector, which means it is fully capable of cross language interop.

Features

• Call any JS function from Lua
• Give Lua tables/functions/userdata to Javascript

js library

js = require "js"

null

A userdata representing JavaScript null

global

A reference to the JavaScript global context. In the browser, this is usually equivalent to the window object. In node.js it's equal to global.

new(constructor, ...)

Invokes the JavaScript new operator on constructor passing the arguments specified.

Returns the created object.

of(iterable)

Returns a iterating function and an iterator state that behave like a JavaScript for...of loop. Suitable for use as a lua iterator. e.g.

for f in js.of(js.global:Array(10,20,30)) do
	print(f)
end

Note: this function only exists if the JavaScript runtime supports Symbols

createproxy(x[, type])

Note: Only available if your JS environment has the Proxy constructor

Creates a JavaScript Proxy object. The proxy supports configuring traps by setting them as metamethods on your object.

type may be "function" (the default) "arrow_function" or "object":

• "function":
  • typeof p === "function"
  • Can be used as a constructor
• "arrow_function":
  • typeof p === "function"
  • Can not be used as a constructor
• "object":
  • typeof p === "object"
  • Can not be used as a constructor

Note that JavaScript coerces all types except Symbols to strings before using them as a key in an indexing operation.

tonumber(x)

Coerces the value x to a number using JavaScript coercion rules.

tostring(x)

Coerces the value x to a string using JavaScript coercion rules.

instanceof(x, y)

Returns if the value x is an instance of the class y via use of the JavaScript instanceof operator

typeof(x)

Returns what JavaScript sees as the type of x. Uses the JavaScript typeof operator

JavaScript API

push(L, value)

Pushes an arbitrary JavaScript object value as the most suitable lua type onto the lua stack L. Performs deduplication so that the same JavaScript objects are pushed as the same lua objects.

pushjs(L, value)

Pushes an arbitrary JavaScript object value as a userdata onto the lua stack L. Rarely used; see push(L, value) instead.

checkjs(L, idx)

If the value on the lua stack L at index idx is a JavaScript userdata object (as pushed by push or pushjs) then return it. Otherwise throw an error.

testjs(L, idx)

If the value on the lua stack L at index idx is a JavaScript userdata object (as pushed by push or pushjs) then return it. Otherwise returns undefined.

tojs(L, idx)

Returns the object on the lua stack L at index idx as the most suitable javascript type.

• nil is returned as undefined
• booleans are returned as booleans
• numbers are returned as numbers
• strings are returned as JavaScript strings (Note: this can throw an error if the lua string is not represenable as a JavaScript string)
• JavaScript userdata object (as pushed by push or pushjs) returns the pushed JavaScript object
• Other objects are returned wrapped in a JavaScript function object with methods:
  • apply(this, [args...]): calls the lua object. Returns only the first return value
  • invoke(this, [args...]): calls the lua object. Returns results as an array
  • get(key): indexes the lua object
  • has(key): checks if indexing the lua object results in nil
  • set(key, value)
  • delete(key): sets the key to nil
  • toString() JavaScript arguments to these methods are passed in via push() and results are returned via tojs(). Calling the function is equivalent to calling the lua function wrapped.

luaopen_js

The entrypoint for loading the js library into a fengari lua_State. Usually passed to luaL_requiref.

Symbols

If the JavaScript environment supports Symbols, then some runtime-wide symbols can be used to customise behaviour:

__pairs

The __pairs Symbol can be used to describe how to iterate over a JavaScript object. Use Symbol.for("__pairs") to get the symbol. It should be used as a key on your objects, where the value is a function returning an object with three properties: "iter", "state" and "first".

"iter" should be a function that follows the standard Lua generic for protocol, that is, it gets called with your state (as this) and the previous value produced; it should return an array of values or undefined if done.

e.g. to make pairs on a Map return entries in the map via the iterator symbol:

Map.prototype[Symbol.for("__pairs")] = function() {
	return {
		iter: function(last) {
			var v = this.next();
			if (v.done) return;
			return v.value;
		},
		state: this[Symbol.iterator]()
	};
};

If there is no __pairs Symbol attached to an object, an iterator over Object.keys is returned.

__len

The __len Symbol can be used to describe how to get the length (used by the lua # operator) of a JavaScript object. Use Symbol.for("__len") to get the symbol. It should be used as a key on your objects, where the value is a function returning the length of your objects (passed as this).

e.g. to have the lua # operator applied to a Map return the size field:

Map.prototype[Symbol.for("__len")] = function() {
	return this.size;
};

If there is no __len Symbol attached to an object, the value of the .length property is returned.

fengari-interop's Issues

Passing Lua tables to JS functions using JSON

I have an use case where I want to be able to pass lua tables to javascript functions using fengari-interop. Soon enough I noticed that they are passed as weakmaps making impossible to work with them in javascript without knowing in advance the keys of the object passed.

I found a workaround this by converting the lua table to a JSON string using (https://github.com/rxi/json.lua) in the Lua code before calling the javascript function inside Lua and then converting the JSON string to JS object in javascript side.
This works fine but I need to create "proxy" methods to allow the users to use this functions transparently.

Ex.

function Context.set (key, value)
    if type(value) == "table" then value = JSON.encode(value) end
    Context._set(key, value)
end

The users can pass a Lua table to the 'public' API is context.set which then internally calls another context._set with the JSON string as argument. The issue here is that everytime we have a new agument or a new function we need to add this 'proxy' functions.

Is there a way to modify the fengari-interop default conversion for LUA_TTABLE arguments to become regular strings (JSON)?

const tojs = function(L, idx) {
	switch(lua_type(L, idx)) {
		case LUA_TNONE:
		case LUA_TNIL:
			return void 0;
		case LUA_TBOOLEAN:
			return lua_toboolean(L, idx);
		case LUA_TLIGHTUSERDATA:
			return lua_touserdata(L, idx);
		case LUA_TNUMBER:
			return lua_tonumber(L, idx);
		case LUA_TSTRING:
			return lua_tojsstring(L, idx);
		case LUA_TUSERDATA: {
			let u = testjs(L, idx);
			if (u !== void 0)
				return u;
		}
		/* fall through */
		case LUA_TTABLE:                      <------- Convert the Lua Table to JSON string
                        **something here**            <------ in Javascript and pass the value as if it
                        return lua_tojsstring(L, idx) <------ was a regular string?
		case LUA_TFUNCTION:
		case LUA_TTHREAD:
		/* fall through */
		default:
			return wrap(L, lua_toproxy(L, idx));
}

I appreciate any pointer for this issue, and thanks again for this library :)

JS class helper?

Add a helper to extend a JS class?

Strategy for creating async extensions

Hi. I'm trying to create some extensions that call javascript code that needs to run asynchronously. In effect this returns a Promise to Lua, instead of the resolved value.

I see an open (#2) issue from 2017 that seems related but hasn't been resolved. This problem is proving a bit of a blocker to my project at the moment, so I'd be happy to contribute a PR to move this forward, but I'm not sure exactly what the strategy should be. Did you have any ideas how to resolve #2?

Or is current best strategy just to avoid async code in extensions?

Add __len for more types

• https://developer.mozilla.org/en-US/docs/Web/API/CSSValueList
• https://developer.mozilla.org/en-US/docs/Web/API/CSSRuleList
• https://developer.mozilla.org/en-US/docs/Web/API/SVGLengthList
• https://developer.mozilla.org/en-US/docs/Web/API/SVGNumberList
• https://developer.mozilla.org/en-US/docs/Web/API/SVGStringList return .numberOfItems instead?
• https://developer.mozilla.org/en-US/docs/Web/API/SVGTransformList
• https://developer.mozilla.org/en-US/docs/Web/API/PerformanceObserverEntryList
• https://developer.mozilla.org/en-US/docs/Web/API/StyleSheetList
• https://developer.mozilla.org/en-US/docs/Web/API/TouchList
• https://developer.mozilla.org/en-US/docs/Web/API/DOMStringList
• https://developer.mozilla.org/en-US/docs/Web/API/DataTransferItemList
• https://developer.mozilla.org/en-US/docs/Web/API/FileList
• https://developer.mozilla.org/en-US/docs/Web/API/SourceBufferList
• https://developer.mozilla.org/en-US/docs/Web/API/TrackDefaultList

?

• https://developer.mozilla.org/en-US/docs/Web/API/SpeechGrammarList
• https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognitionResultList
• https://developer.mozilla.org/en-US/docs/Web/API/PresentationConnectionList
• https://developer.mozilla.org/en-US/docs/Web/API/SVGAnimatedLengthList
• https://developer.mozilla.org/en-US/docs/Web/API/SVGAnimatedNumberList

Passing arguments from JS to Lua functions

So if I have this code in lua:

js = require "js"
function foo(x)
    print(x)
  end
js.global.foo = foo

And I call global.foo in my js script with an argument:

global.foo(88)

print(x) outputs [object global], which is the same object as js.global (printing the key/value pairs in x confirms that).

Is there a way to pass args from js to Lua with interop?

Integers retrieved from JS objects become floats

It seems fengari-interop retrieves any number on a JS object as a float, internally using lua_pushnumber, rather than lua_pushinteger for integers.

Example using https://fengari.io:

→ js.global.document.documentElement.clientWidth
1628.0

Is this intentional?

ownKeys metamethod should default to using lua_next

ownKeys metamethod should default to using lua_next.

fengari-interop/src/js.js

Line 652 in fa936f5

throw Error("ownKeys unknown for fengari object");

Open question: is this an API breaking change?

How to prevent DOM object access by lua, but keep variable transfer between js & lua ?

example to add script from lua :

<script type="application/lua">

local js = require "js"
local g = js.global;
local window = js.global;
local document = window.document

s = document:createElement('script');
s.src = "https://www.domain.com/js/grab_info.js";
document:getElementsByTagName("head")[0]:appendChild(s);

</script>

Add js.instanceof function

Should have js.instanceof function

Use arrow function for function proxy base objects

Reflect.ownKeys((()=>void 0))
// Array [ "length", "name" ]
Reflect.ownKeys((function(){}))
// Array [ "prototype", "length", "name" ]

(tested in firefox 57)

This matters for the ownKeys trap.

However I'm not sure how we can get this past babel? Obviously it only works in browsers that support arrow functions

Show full stack trace on errors instead of just the error?

Moving the issue from fengari-lua/fengari#124 over here instead. Near the end of the thread for that issue, I suggested possibly adding some kind of option for enabling stack traces on errors to aid in debugging. Any way we can get this feature?

Waiting for an async js function result inside Lua code

I'm registering a function to be used inside Lua, the problem is that function is a Javascript async operation.
I'm noticing that the Lua code is not waiting for that to resolve so it's returning an empty object.
Is there a way to make this work?

Context.get is the Javascript async function that is not being awaited.

function registerStdlibFunction(L, module, funcName, funcPointer, thisPointer) {
    fengari.lua.lua_getglobal(L, module)
    if (!fengari.lua.lua_istable(L, -1)) {
        fengari.lua.lua_newtable(L)
        fengari.lua.lua_setglobal(L, module)
        fengari.lua.lua_pop(L, 1)
        fengari.lua.lua_getglobal(L, module)
    }

    fengari.lua.lua_pushstring(L, funcName)
    interop.push(L, function () {
        const arg = JSON.parse(JSON.stringify(this))

        return funcPointer.bind(thisPointer, arg).apply(void 0, arguments)
    })

    fengari.lua.lua_settable(L, -3)
    fengari.lua.lua_pop(L, 1)
}


const L = luaL_newstate()

luaL_openlibs(L)
luaL_requiref(L, to_luastring(''), luaopen_js, 0)
lua_pop(L, 1)

const luaCode = `
return function()
    ${luaBlock}
end
`

registerStdlibFunction(L, 'Context', 'get', this.contextLibrary.get, this.contextLibrary)

const result = luaL_dostring(L, to_luastring(luaCode))

Can't call JS functions in browser

I'm just trying to figure out how to call basic JS functionality from Lua (or specifically moonscript). This is my current code and the output it generates:

HTML: <p id="foobar">This is a test.</p>

js = require "js"

-- works
print "Hello!"

jq = js.global.jQuery

-- no error
el = jq '#foobar'

-- error: Invalid invocation
ela = js.global.document.getElementById 'foobar'

-- nil
print el\html!

Here's the generated Lua code, too:

local js = require("js")
print("Hello!")
local jq = js.global.jQuery
local el = jq('#foobar')
local ela = js.global.document.getElementById('foobar')
print(el:html())

Am I misunderstanding the way js is supposed to work?

Passing objects / arrays to Lua

Hi, I tried to ask this on Freenet but I had to exit the client - hope it's not a problem if I ask here asynchronously. Maybe it can be useful to others, too.

Is it possible to pass a Javascript array / object to lua with fengari-interop and have it converted into an equivalent Lua table? I'm trying something like this gist.
When I run it, the Lua interpreter says that the type of the passed array is userdata, but I can call pairs on it with no problems. If I try to get the length of it though (#), it excepts because it's not a table.

If it's not possible to do array/object => table automatically, what's my best option? Serializing by hand and pass via string?

Document JS => Lua interop

The README describes how to call JS functions from Lua (extending), but what about the other direction of interop (embedding)?
Would it be possible to easily call a Lua function from JS using this library, instead of resorting to the raw fengari API , like in this issue?

jQuery and Fengari - not playing nice

I'm having some mental fengari -> js // js -> fengari corroboration issues:
in my browser console:
fengari.load('js=require("js"); local jq=js.global.jQuery; print(jq:find(\'body\'):html())')();

Gives me:
fengari-web.js:8 Uncaught [string "?"]:1: attempt to call a nil value (method 'html')

I'm not sure why?

Get keys from Lua table

I'm finding that when I convert a Lua table to JS, I can't examine its keys. Is there a way of doing this?

{ foo = 1, bar = 2}

In Javascript, I would like to type:

# ... convert above Lua to js using tojs() as object o
Object.keys(o)

how to get the jQuery object?

I try this code, but not works:

local js = require "js"
local window = js.global
local jq = js.global.jQuery
function ready()
    local body = jq('body')
    -- can not hide the body
    body:hide()
end
jq(document):ready(ready)

how can I get the jq object?

webpack fails while building

Dies here because webpack is not defined.
Just adding const webpack = require('webpack') at the top should do the trick.

js.typeof is broken

Minimal example, e.g. in the https://fengari.io REPL:

js.typeof(js.global)

Uncaught exception TypeError: Cannot read property 'funcOff' of undefined
    at rt (javascript:8)
    at Et (javascript:8)
    at It (javascript:8)
    at t (javascript:8)
    at t (javascript:8)
    at Object.t [as luaD_precall] (javascript:8)
    at Object.t.exports.luaV_execute (javascript:8)
    at ut (javascript:8)
    at Object.ht [as luaD_callnoyield] (javascript:8)
    at Object.b [as luaT_callTM] (javascript:8)

Automatically yield when calling an AsyncFunction

• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncFunction

Fully sandboxed execution & interaction

We'd really like to migrate from Moonshine to Fengari, but we're having trouble figuring out if we can really execute Lua in a fully sandboxed environment, and how to integrate with it in the same way.

Ideally we'd like to do the following:

• Create a new Lua state, giving a set of JS variables and functions to be placed into the global Lua namespace.
• Execute Lua in this state, with it only having access to the supplied functions/variables.
• Have Lua call the JS functions that have been supplied to it.
• Have JS call Lua functions that have been passed to it.

Is this possible with the current state of Fengari? Would it be possible to get working examples of how to achieve this?

Have metamethods for *all* operations for foreign proxy

Allow objects from foreign fengari global states to have full lua semantics.

Add __len symbol

Similar to how __pairs has a symbol.

Should have default implementation(s) for:

• Array
• TypedArray(s)
  Get reference via Object.getPrototypeOf(Int8Array). As mentioned in https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray#Description

As mentioned in #10 (comment)

Passing null values to lua functions

I'm having a problem passing Javascript null values into Lua scripts. I would expect the null value to be converted to a Lua nil value. When the Lua script is evaluated, the value in the variable that I think should have been converted to nil is actually some sort of object. Am I misunderstanding something?

Here is an example:

var func = fengari.load(`return function(var1)
    if var1 == nil then
        return "Passed value is nil"
    else
        return "Passed value is: " .. var1
    end
end`)();
// I would expect this to log "Passed value is nil"
console.log(func.call(null));

I would expect the above function to log "Passed value is nil", but instead I see:

attempt to concatenate a js object value (local 'var1')

Am I doing something wrong?

Build process that emits function body

Related to webpack/webpack#6874

JS values passed improperly to Lua callbacks

This works fine:

local canvas = document:createElement('canvas')
canvas.width = 400
canvas.height = 400
function canvas:onclick(event)
  print(event.offsetX, event.offsetY)
end

document.body:appendChild(canvas)

but this prints nil for the values in event:

local canvas = document:createElement('canvas')
canvas.width = 400
canvas.height = 400
canvas:addEventListener('click', function (event)
  print(event.offsetX, event.offsetY)
end)

document.body:appendChild(canvas)

the event is passed correctly, as it's printed as [object HTMLCanvasEvent].

The same restriction prevents me from accessing for example the timestamp passed into a requestAnimationFrame callback.

Given that I know the type of the JS object there, isn't there a way I can at least extract the properties?

Dealing with unrepresentable strings

Instead of throwing an error, should we do what e.g. node.js does and replace with the unicode replacement character?

"Error: ownKeys unknown for fengari object" when attempting to use fengari with BabylonJS

Good day!

After upgrading the BJS library I can no longer use fengari-web, as it fails with the following error:

uncaught exception Error: ownKeys unknown for fengari object
at Object.ownKeys (fengari-web.js:5218)
at assign (<anonymous>)
at __assign (tslib.es6.js:38)
at new Scene (scene.ts:1375)
at new (fengari-web.js:5104)
at Object.t [as luaD_precall] (fengari-web.js:2152)
at Object.t.exports.luaV_execute (fengari-web.js:4240)
at ut (fengari-web.js:2278)
at Object.ht [as luaD_callnoyield] (fengari-web.js:2426)
at Ot (fengari-web.js:5603)``

I've already posted on their forums, providing more details that might help: https://forum.babylonjs.com/t/bjs-seemingly-causes-an-error-in-another-library-im-using-by-assigning-removing-object-properties/11879

I don't really understand how to work around this problem, but effectively this means I either can't upgrade BJS or use fengari, putting a hard stop to my project as I need the latest features of the rendering framework and I use Lua scripting for the logic and APIs.

I sure don't want to rewrite everything, plus the whole point of the project is to use Lua for the UI and general logic after all. I'd be delighted if you could either fix this or provide me with more information so I can try to do it myself. Thank you! :)

Improve way we get reference to global environment

https://mathiasbynens.be/notes/globalthis

Found via https://news.ycombinator.com/item?id=19891190

Non-object bound JS functions 'lose' first argument when invoked from Lua

This may be misusing how the library was meant to be used, but here goes.

Given a Lua script that cannot be changed (so no js.global.myFunc):

myFunc(1, 2);

Rather than pushing a C/JS function and handling Lua state, I naively tried the following using push:

const myFunc = function () { console.log(this, arguments) };
push(L, myFunc);
lua_setglobal(L, to_luastring('myFunc'));

Executing the Lua script works fine, except that only 2 is given as an argument to myFunc with 1 being the this / context of the call on the JS side of things.

Currently, the workaround is something like this:

push(L, function (...args) {
  // TODO: Work around fengari-interop inability to handle first argument
  // on non-object bound global methods
  return myFunc(this, ...args);
});
lua_setglobal(L, to_luastring('myFunc'));

Any thoughts on how to best get this to work? Or is managing Lua state manually using lua_pushjsfunction the way to go?

Why Lua tables aren't accepted as JS dictionaries?

I want to translate this JS one-liner to Lua:

Split(['#Left', '#Right'], {gutterSize: 15, minSize: 270, sizes: [25, 75], snapOffset: 0, expandToMin: true});

This works, but is too bulky:

local window = js.global
local options = js.new(window.Object)
options.gutterSize = 15
options.minSize = 270
options.sizes = window:Array(25, 75)
options.snapOffset = 0
options.expandToMin = true
window:Split(window:Array('#Left', '#Right'), options)

This looks nice, but doesn't work:

local window = js.global
window:Split(window:Array('#Left', '#Right'), {gutterSize = 15, minSize = 270, sizes = window:Array(25, 75), snapOffset = 0, expandToMin = true})

Why Fengari doesn't convert Lua tables to JS dictionaries on-the-fly?
It's too boring to write this conversion manually.

If there are some reasons why such conversion could not be done automatically,
please add a function js.dict(lua_table) which will convert Lua-curly-braces into JS-curly-braces.
This function would make creating a JS dictionary on Lua side a lot easier:

window:Split(window:Array('#Left', '#Right'), js.dict{gutterSize = 15, minSize = 270, sizes = window:Array(25, 75), snapOffset = 0, expandToMin = true})

Yes, I know that I can write such function myself on Lua side, but this function is a "must-have" for every developer.

Failed to minify the code

This is similar to fengari-lua/fengari#89, but complicated by the fact that you're outputting an ES6 bundled file in dist, making it non-consumable by any toolchain based on create-react-app.

remove .prototype property from function proxys

.bind will remove it. Thanks @kovensky

Reflect.ownKeys((function(){}).bind(null))

Content-security-policy and use of Function

I'm adding fengari-web into an application that uses a Content-Security-Policy. The application has a simple policy of Content-Security-Policy: script-src 'self';. However, because of this, the browser refuses to load fengari-web.js because of the use of Function to evaluate a string at this line:

fengari-interop/src/js.js

Line 701 in 1626687

const make_arrow_function = Function("return ()=>void 0;");

The error is EvalError: call to Function() blocked by CSP

I don't understand the Fangari code well enough to suggest a solution. Is there any way that the same could be achieved without evaluating code with the use of Function?

As an experiment I tried replacing the above code with var make_arrow_function = function () {return ()=>void 0;} and everything ran as expected (although I realise this could lead to other problems as explained in the comments in the code).

Grateful for any assistance/comments. Many thanks.

Rename file so that filename matches expected loader pattern for 'js'

Import keyword in JS Lua module

Hello, I have noticed some JS package has been used the ESModule instead CommonJS. Its annoying since there's no built-in JS function in lua like js.import.

Does the team have a plan to add import keyword onto JS Lua module?

Thrown lua errors should look nice when caught by JS

If JS has a try/catch around an exposed lua function then it currently gets an ugly error object:

function js.global:lua_error() error("AN ERROR") end
js.global:eval("try { lua_error() } catch(e) { console.error(e) }")

Object { status: 2, previous: null }

Default __len for NodeList Objects

Default __len on NodeListPrototype

Add all proxy traps

• .apply()
• .construct()
• .defineProperty()
• .deleteProperty()
• .enumerate() (deprecated)
• .get()
• .getOwnPropertyDescriptor()
• .getPrototypeOf()
• .has()
• .isExtensible()
• .ownKeys()
• .preventExtensions()
• .set()
• .setPrototypeOf()

e.g. Object.getPrototypeOf(myproxy) could return getmetatable(object).prototype.

string "" in bytecode throw TypeError

use empty string "" or '' in bytecode, throw "TypeError: Cannot read property 'tsslen' of null"
case:

local str = ""

TypeError: Illegal Invocation

For some reason when using the getElementById function on an HTML object I get the error shown in the title
I have attached a .zip file containing an example of the error in action.
errexample.zip