Persistent data structures

[ShalokShalom]

Persistent data structures

[jchassoul]

So, we could have a better documentation! by using Erlang you are already using a language where data is immutable, Luerl as the Lua implementation that it is, gives you a standard Lua 5.3 where data in the Lua side of things is mutable like in any other implementation of Lua its on our hands to decide what data structures we pass though the eye of the needle.

[rvirding]

Luerl started out as implementation of a language with shared, global and mutable data as an experiment to see how this could be done. Lua was the language I chose and it kept growing till it finally became an implementation of Lua, at least as close as I could reasonably make it. Hence the handling of data is by design very Lua.

Also seeing you are running inside Erlang with and have a very close coupling I feel that if you wish to have that type of data handling you do it in Erlang. You can always implement a module in Erlang with a Lua interface which gives you that type of data.

[rvirding]

Depends how you define "have this capacity". It is the system builder who defines what users have access to. To be able to add modules which can extend how I can access Erlang data and the Erlang system I need to be able to access the system though the Erlang interface. So if I can ONLY write Lua code which is executed by the system then I cannot do this.

This is what makes sandboxing relatively easy. When setting up the system you can quite simply remove certain tables, for example io and os, or functions. The when executing Lua these are just not available and can not be added. Check out the luerl_sandbox.erl module.

[jchassoul]

No, there are no generic global things in the Erlang environment.

We don't share memory, there is no global state or central thread of execution, in Luerl you could have global things that only apply to the execution of that process runing the Luerl VM but in practice is common to have more than one Lua process running and the way those interact between each other and the environment is not by reading a global state but by passing messages.

That's for example why you don't hear of Luerl coroutines since an Erlang process with it's own Lua VM is already lightweight and much more powerful than that.

[jchassoul]

Hi (;

Luerl is an implementation of Lua 5.3 where instead of a C API we have Erlang at the lowest level, this means you can have the full power of the Lua language at the same time that you benefit from Erlang and all it features like immutable data and the powerful OTP and BEAM VM.

Is common in Luerl projects to use and deal with both Erlang and Lua, the same "eye of the needle approach" and "mechanisms not policies" from Lua apply, the big difference is that Luerl processes live inside the Erlang environment and not in the OS and interact with an Erlang API instead of C.

Is there any way to use Lua as an immutable language (thanks to luerl)? No, Luerl give you Lua a mutable scripting language implemented on top of an immutable language, with Luerl normally we use and access both, it depends on you to decide when and how base on your use case.

Have fun with Luerl (=

[ShalokShalom]

You write you benefit from Erlang and all it features like immutable data and with Luerl normally we use and access both

This sounds like a yes.

So, how would look that like? Since I don't see any examples in your documentation...

[jchassoul]

Here are some examples

I don't see any examples in your documentation... Yes, I agree documentation needs more love, hopefully we can improve on that thanks to questions, issues and feedback like yours. =D

[ShalokShalom]

So, we could implement persistent data structures of which all could benefit?

[ShalokShalom]

Yeah, a friend told me yesterday we can implement persistent data structures in luerl.

We could implement them in Erlang and then export it in Lua as a function.

I think this is worth investigating.

[ShalokShalom]

Could this work?

https://github.com/PaulBernier/castl
https://www.npmjs.com/package/persistent

[rvirding]

Couldn't guarantee that Luerl could efficiently handle everything generated by castl. There are a few things Luerl doesn't handle yet: _ENV and goto and labels. Well you can define a label but you can't goto it.

So far no one has complained about the lack of these so they are not really in the pipeline, not the goto anyway.

Anyway as @jchassoul has pointed out from within Luerl all its data behaves in the same share/global/mutable way so you can't really get immutable data from the Lua inside Luerl. There is no support for that in Lua so it would be really strange.

Luerl has userdata in a similar way to Lua which allows you to pass Erlang data around inside Luerl but there is no way you can directly access it from Lua. Again the same as in standard Lua. So for example you could write a module in Erlang which you could call from Lua to send/receive messages and keep pids in the userdata. Or you could write a module which internally uses Erlang maps and access it from Lua. Note that these modules must be written in Erlang to be able to access Erlang.

Note that Lua can only directly handle Erlang data if it is in the forms it recognises.

[ShalokShalom]

Thanks a lot for these comments, I really appreciate it. ❤️ 🤗

So far as I understand is the core reasoning that Luerl has no immutability simply because Lua has not?

I can see your point, while you could say also that Lua has none of the properties that Erlang has so Luerl as a whole could be considered so strange as you find that idea.

I think there is a solid idea behind this project and I am fully aware of the implications so that this addition would compile fine for Luerl and not for Lua or any other implementation, for that matter.

I just think this is worth it.

[ShalokShalom]

Yes, this is exactly as I thought. Do you see it as beneficial for every user, to have this capacity?

[ShalokShalom]

There seems to be a misunderstanding.
My understanding is that this is something that can be implemented globally on a generic way. Is this correct?

[rvirding]

Getting back to this really late!

What you could do is write a module which you call from Luerl which accesses data in an ETS table. This could then be shared with other Luerl processes and other things written in Erlang. While this is not shared, global or mutable it is about as close as you can get to that in the Erlang system.