gleam-lang/otp

📫 Fault tolerant multicore programs with actors

Gleam OTP

A Gleam library for building fault tolerant multi-core programs using the actor model. It is compatible with Erlang's OTP framework.

Gleam’s actor system is built with a few primary goals:

• Full type safety of actors and messages.
• Be compatible with Erlang’s OTP actor framework.
• Provide fault tolerance and self-healing through supervisors.
• Have equivalent performance to Erlang’s OTP.

This library documents its abstractions and functionality, but you may also wish to read the documentation or other material on Erlang’s OTP framework to get a fuller understanding of OTP, the problems it solves, and and the motivations for its design.

Not all Erlang/OTP functionality is included in this library. Some is not possible to represent in a type safe way, so it is not included. Other features are still in development, such as further process supervision strategies.

Usage

Add this library to your Gleam project.

gleam add gleam_otp

Actor hierarchy

This library provides several different types of actor that can be used in Gleam programs.

Process

The process is the lowest level building block of OTP, all other actors are built on top of processes either directly or indirectly. Typically this abstraction would not be used very often in Gleam applications, favour other actor types that provide more functionality.

Gleam's process module is defined in the gleam_erlang library.

[Documentation]

Actor

The actor is the most commonly used process type in Gleam and serves as a good building block for other abstractions. Like Erlang's gen_server it handles OTP's system messages automatically to enable OTP's debugging and tracing functionality.

[Documentation]

Task

A task is a kind of process that performs a single task and then shuts down. Commonly tasks are used to convert sequential code into concurrent code by performing computation in another process.

[Documentation]

Supervisor

Supervisors is a process that starts and then supervises a group of processes, restarting them if they crash. Supervisors can start other supervisors, resulting in a hierarchical process structure called a supervision tree, providing fault tolerance to a Gleam application.

[Documentation]

Limitations and known issues

This library does not currently replicate all of the Erlang/OTP functionality. Some limitations include:

• There is no support for named processes. They are untyped global mutable variables which may be uninitialized, more research is needed to find a suitable type safe alternative.
• There are relatively few actor abstractions provided by this library. More will be added in the future.
• Actors do not yet support all OTP system messages. Unsupported messages are dropped.
• Supervisors do not yet support different shutdown periods per child. In practice this means that children that are supervisors do not get an unlimited amount of time to shut down, as is expected in Erlang or Elixir.
• This library has not seen much testing compared to the Erlang OTP libraries, both in terms of unit tests and real world testing in applications.

otp's Issues

Proper error logging when actors crash

Currently debugging is challenging. We should be as good as gen_server.

Why

Considering the Erlang OTP module can be used, could you add to the readme, why this module gets build?

Just so it is in Rust?
So for performance increasements?
New, clearified code base?

Current status of gleam/otp/actor

Playing more with Gleam I had to play with actors and how they will become typed, my biggest gripe with dialyzer and Erlang/Elixir. I assume the typing is tied to the Channel that's used to pass messages around, right?
If so: I read through the actor source and didn't find a way to get a channel to the actor after starting it with a spec as is done in the tests. Is this just rough state because of active development or did I miss a piece?

Also regarding this concept I was wondering how typing would fit into things like dynamic process registries.

Documentation outdated

Hello,

The readme refer to a blog post for gleam 0.12, explaining processes and channels.

Could we update the blog post (and i'll open an issue on gleam/website to mark this blog post as outdated) ?
Or can we update the current readme to show basic examples ?

I'll be happy to send a PR after few guidances of what solution is the best from your opinion

[Feature] DynamicSupervisor please

Hello.

I think Gleam's typed OTP and its concept of subject is great.

But, It would be better to have some types of DynamicSupervisor that allow me to spawn workers as I need.

[Discussion] Allow `actor` to change its selector after creation

This came up during the process of supporting user-defined messages in mist WebSockets.

The way I am looking at structuring it currently does not allow for a user to pass a selector. This is because the actor init is called when the socket opens. This will be well before any user handler hits the path to upgrade the connection.

We talked about this on Discord, and I think landed on "lets talk about it" before actually moving forward with anything.

Right now the Subject passed around from the actor is all related to the consumer's messages. I'm not sure what exactly a good API would look like to update the selector. Maybe an additional type for actor.Next?

Feature request: Function equivalent to elixir's `Task.async_stream`

As previously discussed, this would be a nice & useful function to have.

Naming of abstractions

What should we name our abstractions?

Here are the current names:

Process

The lowest level abstraction. Generally this is not to be used by programmers, instead one of the other abstractions is to be used.

This matches Erlang's naming.

Actor

A process that has a loop function that handles typed messages, can trap exits, and can monitor other process or ports.

It automatically handles system messages for the programmer.

This can be thought of being similar to a gen_server.

Supervisor

A process that handles the starting and restarting of other processes.

The programmer cannot add additional functionality to the supervisor process.

Agent

A specialised actor process that holds some state and receives messages containing functions that transform the held state.

The programmer cannot define a custom accepted message type for these.

It cannot monitor other process or ports, and cannot trap exits.

It automatically handles system messages for the programmer.

Task

A process that performs an item of work asynchronously, sending back the response.

The programmer cannot add additional functionality to the supervisor process.

One shot

A process that performs an item of work asynchronously, discarding any response.

[Bug] Cannot not be compile after add `gleam_otp`

After I run gleam add gleam_top under my terminal, the gleam.tomk add one more dependency
for gleam_top (gleam_otp = "~> 0.7")

But when I ran gleam run, it just show me the following error:

The gleam version is 0.30.5, and I ran it under MacOS, installed by homebrew

Possible Bug: `try_await_forever` letting panic through

My understanding of task.try_await_forever is that it should catch when a process panics and return an error. However it seems to cause the calling process to panic

pub fn main() {
  io.println("Try await a panicking task")
  let _ = task.try_await_forever(task.async(fn() { panic }))

  // This line is never reached
  io.println("Try await caught the error")
}

Is this a bug or am I misunderstanding the function?

New Feature: Registry

I was wondering about local (and clustered, for that matter) registries so one can have the following features:

• Local discovery of one or more processes
• Global discovery of processes
• Maybe: Management of an agent-pool

Especially with clustered discovery I feel like the compile-time type checking might be problematic because one can't guarantee that the remote side runs exactly the same version and therefore has the same type-details.

Bug: actor not unmonitoring finished task

I have an actor that processes a task via task.async and task.try_await.
I always seem to get Actor discarding unexpected message: DOWN after the task has finished.

As I understand it, the actor that creates the task monitors it.
The task is completed and goes 'down' and the actor reports this as a warning.

As suggested by @lpil on Discord, changes seem to be needed in these places, i.e. adding a call to process.demonitor_process:

otp/src/gleam/otp/task.gleam

Lines 89 to 91 in e4478fe

	case process.select(task.selector, timeout) {
	// The task process has sent back a value
	Ok(FromSubject(x)) -> Ok(x)

otp/src/gleam/otp/task.gleam

Lines 119 to 123 in e4478fe

	pub fn try_await_forever(task: Task(value)) -> Result(value, AwaitError) {
	assert_owner(task)
	case process.select_forever(task.selector) {
	// The task process has sent back a value
	FromSubject(x) -> Ok(x)

To reproduce:

pub fn main() {
  let loop = fn(msg, state) {
    task.async(fn() { state })
    |> task.try_await(500)

    Continue(state)
  }

  assert Ok(subject) = actor.start(0, loop)

  actor.send(subject, 1)
  process.sleep(1000)
}

Variants to test:

• try_await
• try_await_forever

Unable to compile gleam_otp in a Phoenix app

When incorporating gleam_otp into a Phoenix application, the library fails to compile. This isn't a big deal for me, as I have a workaround for now, and it's for a personal side project that currently only runs on my dev machine.

Compile output

➜ mix compile
===> Rebar3 detected a lock file from a newer version. It will be loaded in compatibility mode, but important information may be missing or lost. It is recommended to upgrade Rebar3.
===> Compiling gleam_otp
===> Compiling src/gleam_otp_external.erl failed
src/gleam_otp_external.erl:14: can't find include file "gen/src/gleam@otp@process_Sender.hrl"
src/gleam_otp_external.erl:15: can't find include file "gen/src/gleam@otp@process_Exit.hrl"
src/gleam_otp_external.erl:16: can't find include file "gen/src/gleam@otp@process_PortDown.hrl"
src/gleam_otp_external.erl:17: can't find include file "gen/src/gleam@otp@process_ProcessDown.hrl"
src/gleam_otp_external.erl:18: can't find include file "gen/src/gleam@otp@process_StatusInfo.hrl"

src/gleam_otp_external.erl:119: record process_down undefined
src/gleam_otp_external.erl:122: record port_down undefined
src/gleam_otp_external.erl:125: record exit undefined
src/gleam_otp_external.erl:194: record sender undefined
src/gleam_otp_external.erl:207: record status_info undefined
src/gleam_otp_external.erl:210: variable 'Debug' is unbound
src/gleam_otp_external.erl:210: variable 'Mode' is unbound
src/gleam_otp_external.erl:210: variable 'Parent' is unbound
src/gleam_otp_external.erl:212: variable 'Mode' is unbound
src/gleam_otp_external.erl:212: variable 'Parent' is unbound
src/gleam_otp_external.erl:212: variable 'State' is unbound
src/gleam_otp_external.erl:214: variable 'Mod' is unbound

src/gleam_otp_external.erl:118: Warning: variable 'Pid' is unused
src/gleam_otp_external.erl:118: Warning: variable 'Reason' is unused
src/gleam_otp_external.erl:121: Warning: variable 'Port' is unused
src/gleam_otp_external.erl:121: Warning: variable 'Reason' is unused
src/gleam_otp_external.erl:124: Warning: variable 'Pid' is unused
src/gleam_otp_external.erl:124: Warning: variable 'Reason' is unused
src/gleam_otp_external.erl:192: Warning: variable 'Pid' is unused
src/gleam_otp_external.erl:193: Warning: variable 'Prepare' is unused

** (Mix) Could not compile dependency :gleam_otp, "/home/matt/.mix/rebar3 bare compile --paths /home/matt/projects/beam_ci/_build/dev/lib/*/ebin" command failed. You can recompile this dependency with "mix deps.compile gleam_otp", update it with "mix deps.update gleam_otp" or clean it with "mix deps.clean gleam_otp"

Other Notes

This happens when building with both gleam_compile and mix_gleam

Workaround

I've worked around this by copying gleam.toml from my local checkout of gleam_otp into the deps/gleam_otp folder and running rebar3 compile there. Once it's compile once, everything works.

Seemingly outdated documentation for `untyped_send`

Why?

The process.gleam module is seemingly the root of actor model in Gleam.
Documentation for untyped_send is the first non-trivial thing newcomers see, and currently it is unclear.

What?

• Clear up (remove "not not", long sentences),
• Address the missing Channel type conundrum,
• If possible, add moduledoc.

How?

• Figure out what "Channel type" was (I suggest using git-temporal plugin for VSCode! It's awesome!)
• Learn how it evolved into #(Sender(t), Receiver(t))
• Describe this approach, preferrably in the moduledoc section of the module, where the notion of channels gets introduced (we think it's process.gleam)
• Mention this approach briefly in place of outdated "see the Channel type" reference.

Current main breaks builds of gleam-elli

Sadly the currently pushed commit on the main branch breaks gleam-experiments/elli - This can be fixed by pinning this repo to an older commit without this issue but still might repel some people :)

Implement supervisor.supervisor function for starting supervisor type children

If you wish to prepare a new supervisor type child see the supervisor function.

I could not find a supervisor function anywhere.

Max restart intensity defaults are reversed from Erlang

When starting a supervisor using supervisor.start, the max restart intensity defaults to 5 restarts per 1 second. This is the reverse of the default values in Erlang, which allow 1 restart per 5 seconds. Since the defaults when using start are not documented, I wasn't sure whether this was an intentional divergence from Erlang's defaults or just an oversight. I would be happy to PR a doc update and/or a fix, depending on which of those it is.

Syntax error at top-level `if` statements (e.g. `if erlang { ... }`).

I can't compile, open a shell for, or test this code. (I'd like to tweak the tests to do my own experiments.) Whenever I try I get an error like this:

jeff@jbb-dell:otp-clone$ rebar3 shell
===> Verifying dependencies...

error: Syntax error
   ┌─ /home/jeff/code/concurrent-langs/gleam/by-others/otp-clone/_build/default/lib/gleam_stdlib/src/gleam/int.gleam:34:1
   │
34 │ if erlang {
   │ ^^ I was not expecting this.

Expected one of: An import, const, type, or function.

===> Unable to compile Gleam project
jeff@jbb-dell:otp-clone$

I grepped for it and there are a lot of "if erlang" and "if javascript" clauses, but almost all of them are in the project's _build/ subfolder, which is not under version control -- that is, my own system appears to have built it. The src/ and test/ subfolders don't even mention the word javascript.

[feat] an equivalent for `gen_statem`

Hi,

The gen_statem behaviour provides a powerful way to manage stateful processes, especially for development of complex state machines and stateful systems.

It would be great to see something similar for Gleam.