golemcloud/golem

Golem: Transparent durable execution for any programming language

Golem

This repository contains Golem - a set of services enable you to run WebAssembly components in a distributed cloud environment.

See Golem Cloud for more information.

Getting Started

It is possible to start using Golem locally by using our published Docker containers. Please refer to the document link below on how to get golem OSS running using docker. https://learn.golem.cloud/docs/quickstart#setting-up-golem

Once you have Golem running locally, you can use golem-cli to interact with Golem services.

cargo install golem-cli

# component is your compiled code written in Rust, C, etc
# https://learn.golem.cloud/docs/building-templates helps you write some code and create a component - as an example
golem-cli component add <location-to-component-file> 

# Now we need a worker corresponding from component, that can execute one of the functions in component
# If worker doesn't exist, it is created on the fly whey you invoke a function in component
golem-cli worker invoke-and-await  --component-id <component-id> --worker-name my-worker --function golem:it/api/add-item --parameters '[{"product-id" : "foo", "name" : "foo" , "price" : 10, "quantity" : 1}]'

Internally, it is as simple as golem-cli using golem-client sending requests to Golem Services hosted in Docker container. Therefore, you can see what's going on and troubleshoot things by inspecting docker containers.

+-----------------------+         +-----------------------+
|                       |         |                       |
|  Use golem-cli        |  --->   |  Golem Services       |
|                       |         |  hosted in            |
|  commands             |         |  Docker container     |
|  (Send Requests)      |         |                       |
+-----------------------+         +-----------------------+

Compiling Golem locally

Find details in the contribution guide about how to compile the Golem services locally.

golem's Issues

Investigate Issues Accessing Environment Variables In C++

A user reported that they were unable to access environment variables that were set for the worker using C++. It is unclear whether this is an issue in the C++ support or user error but we should add a test verifying that this functionality works correctly and fix it or raise an issue if not.

Make sure we can serve arbitrary content from workers

• Need to make sure we can use string and vec responses in API Gateway to directly set the body (not wrapping in JSON etc)
• and that we can set the content type header

Make a Pollable version of the wasm-rpc host interface

Now that there is a working implementation of the rpc host functions, it can be easily extended to have a variant that returns a Pollable , similar to how the WASI Http and socket interfaces work. In WASI Preview3 it can become a future.

Test Worker-Service with latest changes in Expr in docker environment

Add TypeScript templates and docs

Maximise the reuse of Worker Service's API Gateway Implementation between Cloud and OSS

Implement begin/end markers

Revisit Golem OSS Docker builds

Durable WASI UDP sockets

Implement Remaining Cases of Expr in API Gateway Spec

The spec for the API gateway has the following subtypes of Expr:

enum Expr {
    NewUUID,
	  Literal(String),
    PathVar(String),
    ApiVar,
    AuthVar,
    Concat(Vec<Expr>),
    RequestVar,
    SelectField(Box<Expr>, String),
    SelectElement(Box<Expr>, u32),
    Equals(Box<Expr>, Box<Expr>),
    Cond(Box<Expr>, Box<Expr>, Box<Expr>),
}

Currently only PathVar, Concat, Cond, and Literal` are implemented. We need to implement the remaining ones.

Design a Rust version of the high level transaction API

Add more round trip tests for api-gateway Expr::to_json_value

Plan the hub feature

Roundtrip api-gateway expr prop tests

Write property based test generating Expr and test round trip: generate Expr, evaluate the Expr using some variables, convert to the original to_json and parse it again, and re-evaluate to form the same result

Change the guest language tiers

Currently we reserve "tier 1" for a language that natively supports the component model, but have nothing in that. We should drop this empty group and just call tier 1 everything that can use all of Golem's features

Document (move) the api-gateway-examples in OSS in website.

Add validation to OSS worker service for management API.

Implement get/set oplog index

Observable event log

Worker executors (and any other component of the system) could publish account-level events that the user can query. This could be events like running out of gas, or even instance creation/termination, etc.

In the open-source edition this would be one global event log, in golem-cloud it should be per account.

Durable WASI TCP sockets

Currently we are not persisting anything for tcp connections. There are several things we should do, not detailed in the table below yet:

• we should probably only allow outgoing connections, and maybe even limit the ip/port range
• each operation that communicates with an external host should be persisted
• similarly to the http durability, if we “ran out of replay” and still have a previously open connection, every further operation on that socket should fail

Review logging in all services

CLI self update

We should have a way to:

• notify the users if golem-cli needs to be updated
• possibly automate the update process

The check should not just look for the latest version, it should check the version of the Golem servers it is connecting to, and ensure we have the same version of the CLI tooling.

Implement worker snapshotting

Only for components marked as safe for snapshotting

• Using the core instance enumeration in our forked wasmtime
• Decide to save snapshot when a worker goes to suspended mode, and use it during recovery.
• Make the decision logic configurable
• The Table also has to be serailized in the snapshots (including open handles, key-value elements, etc)
• Also the filesystem of the worker

Create a Version of a C++ HTTP Library for Golem

Resource information into worker metadata

We support creating resource instances now but there is no way gather any information about the number of held resources. This ticket should:

• Add some basic numeric information into the worker metadata - number of instances per exported resource type. We only expose the count not the actual handles, so there is no way for “outside” manipulation of a created resource.
• Add some new metrics so we see resource creation/dropping/average count per worker

Document Expr language spec

Do not inline all types in template metadata

Right now the types in Worker metadata do not include names. Meaning that recursive definitions are not supported.

To be able to display the wit definitions in full, the names should be included in the following type.

com.golem.component.Model.Type

  enum Type {
    // other types...
    case Record(fields: Chunk[(String, Type)])
    case Flags(names: NonEmptyChunk[String])
    case Enum(names: NonEmptyChunk[String])
    case Variant(cases: NonEmptyChunk[(String, scala.Option[Type])])
  }

maybe another case is necessary like "reference", so that definitions could refer to themselves or other types that are embedded.

case Reference(name: String)

Implement explicit commit

Verify the current promise API implementation

Promise operations are not written to the oplog - verify if that's correct and fix if not.

Make sure to support all of the terms in ConstructorPattern in Expr

Persistent invocation queue

Currently we don’t have a persistent invocation queue / mailbox for workers - the incoming invocations are implicitly enqueued in the worker executor’s memory but in case it gets restarted, they get lost.

This is not a critical problem if the clients are using the “invoke-and-await” with a timeout, because they can retry the same invocation with the same invocation key to eventually get a result.

However, when using the “invoke” API, the caller has no (easy) way to do a retry in such scenarios. The expectation is that once we trigger an invocation, Golem will perform that no matter what.

docker examples

The quickstart page uses this broken links: https://github.com/golemcloud/golem-services/blob/main/docker-examples/docker-compose.yaml (docker-compose.yaml does not exist)

The actual docker-compose files in the docker-examples do not run per the given instructions reproduced below, given the required link correction:

curl -sSL https://raw.githubusercontent.com/golemcloud/golem-services/main/docker-examples/docker-compose-sqlite.yaml> docker-compose.yaml
docker-compose up
WARN[0000] The "REDIS_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "REDIS_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SHARD_MANAGER_HTTP_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SHARD_MANAGER_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SHARD_MANAGER_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_HTTP_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_HTTP_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_HTTP_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "REDIS_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GOLEM_SERVICE_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_EXECUTOR_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_EXECUTOR_HTTP_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "SHARD_MANAGER_GRPC_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_EXECUTOR_HTTP_PORT" variable is not set. Defaulting to a blank string. 
WARN[0000] The "WORKER_EXECUTOR_HTTP_PORT" variable is not set. Defaulting to a blank string. 
validating /Users/johncoleman/golemcloud/docker-compose.yaml: services.golem-service.ports array items[0,1] must be unique

It would be useful to provide script to configure the ports which presumably would fix these issues, or just provide some sensible defaults such that the docker-compose is readily executable.

Add GRPC API for worker-service api-gateway side.

Create plan of the Golem Transaction API

Review CI caches

Figure out how exactly and what do we cache in each CI pipeline

• make sure different platforms are not overwriting the same cache
• make sure we are not caching more and more old dependencies until the disk becomes full
• verify that it speeds up the build and not slows it down
• if it does not make sense, remove cache

Optimize worker to worker communication

For this task we should set up some worker-to-worker communication benchmark based on the worker-executor integration test framework to measure the differences in the various improvements.

All implementations should always provide a protocol version that allows seamless upgrade of the worker-to-worker wire format betwen worker executors, including co-existing multiple formats during deployments.

• A simple step can be to just use a cached pool of gRPC connections instead of always creating a new one
• A possible improvement (to be measured) is to use a bidirectional gRPC stream instead
• Based on the results we can try encoding the wit-value payloads with a different serializer and only use grpc / grpc-streams as envelopes.
• A future step, not to be done as part of this ticket can be to try to reduce the number of conversions in the process, possibly by directly encoding in the stub to the wire format and accessing it from the host function as a linear byte array.

Revisit configuration setup

It's quite confusing, which configuration is coming from where, and cannot reason about how things work

Write a blog post about the new transaction API and related features

Document the new transaction API host functions on learn.golem.cloud

Check for dynamic table manipulations in the WASM analyzer and store it in metadata

This would allow snapshotting even if we don't have a solution for persisting the tables, as if there is no dynamic table manipulation instructions in the WASM, we don't have to persist the tables just the memories.

Write worker executor tests for the high level Rust transaction API

support is_empty and is_non_empty in Expr

useful when the worker response is a collection (and need fall back logic when collection is empty)

Showing test reports on CI

There is an experiment in #42

Let's decide if we want to have this and merge or delete.

Add core instance enumeration to wasmtime for memory snapshotting

See bytecodealliance/wasmtime@2a0b4aa

Implement retry policy override

Make sure Expr::from(expr_string).to_string() produce exactly expr_string (user experience)

Workers Templates serde and schema evolution support

• fix TemplateMetadata serde - we have TODO in case of FunctionResult.typ and FunctionParameter.typ - where in relation to naming fixes, original tpe was renamed to typ ; as there was issue to read stored data, we added #[serde(rename = "tpe")] for those attributes, (should not we consider to add custom structure-s, with custom serde, to decouple internal storage implementation and API implementation?)
• add support for versioning of metadata in TemplateRepo

Implement idempotence mode

Move Zig to Tier 2

• Add/modify the integration tests showing how to use wit-bindgen c together with Zig
• Update the documentation and the template projects

Split Worker-Service to Worker-service-base and worker-service