Giter Site home page Giter Site logo

cloudstateio / go-support Goto Github PK

View Code? Open in Web Editor NEW
9.0 6.0 10.0 582 KB

User Language Support for Go

Home Page: https://cloudstate.io/docs/go/gettingstarted.html

License: Apache License 2.0

Dockerfile 0.96% Shell 3.32% Go 95.72%
serverless cloudstate golang

go-support's Introduction

Cloudstate was an open source protocol and reference implementation exploring ideas for stateful serverless, and was originally developed by Lightbend.

The project is no longer active, since 2021. An open source alternative is Eigr.

A continuation of the ideas can be found in Lightbend's platform-as-a-service Akka Serverless.

go-support's People

Contributors

marcellanz avatar pvlugter avatar viktorklang avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar

Forkers

marcellanz sleipnir chongkai hungaikev mrcllnz marcoderama eigr-labs pavansokkenagaraj mabwacharles

go-support's Issues

Implement cloudstate.crdt.Crdt Service

Implement the cloudstate.crdt.Crdt Service from protobuf/protocol/cloudstate/crdt.proto

Tasks

  • CRDT Service
    • State Handling
    • Command Handling
      • Non-Streamed
      • Streamed
    • Cancellation
    • "Callbacks"
    • Context API
  • GCounter
  • PNCounter
  • GSet
  • ORSet
  • Flag
  • LWWRegister
  • ORMap
  • Vote
  • apply new context based API to the evensourced API
  • verify implementation (TCK, CrdtSpecs(?))
    • Currently blocked by cloudstateio/cloudstate#316, but marking progress on the issue to have a
      TCK capable to verify the CRDT state model.
    • TCK verification is replaced by functional tests.
  • presence example
  • paradox documentation => see #36
  • Optional: provide CRDT protocol spec feedback (mainly for clarification without reading the implementation)

WIP-branch
https://github.com/cloudstateio/go-support/tree/feature/crdt_support

Implement EntityDiscoveryServer.ReportError according to new spec recommendations

As discussed on todays constributors call, ReportError should:

  • Should be used for informational purposes only—the message MUST be reported to the developer. Not logged as DEBUG. Typically that would mean writing it to STDOUT but it may go somewhere else depending on the language used.
  • If stream is closed afterwards, then it should passivate the entity (just as normal)

therefore, if the proxy does not close the stream, the entity will receive further commands/events for the entity. The user (developer) should receive the error and then decides what to do. It is advised to make the error discoverable, most probably through writing it to stderr/stdout.

Go Support v0.2.0

An issue collecting features and changes for the "Go Support 0.2.0" release:

  • CRDT Support
  • Eventsoured re-implementation
  • Integration Tests for the CRDT implementation, to be used for the TCK
  • Integration Tests for the Eventsourced implementation
  • Adoption of new Cloudstate Model based TCK tests
  • Protobuf cleanup for go packages
  • Eventsourced TCK Support
  • Eventsourced effects and forward support
  • Significant test coverage increase
  • Dapr contrib model support with new CRDT support
  • Chat example
  • CRDT Shopping Cart example
  • devcontainer.json Support
  • Updated documentation
  • CRDT and Effects & Forwards documentation

CRDT support has been added as the second state model supported. Together with the Eventsoured implementation, stream runners where introduced which enables multiple entities of the same id to be run at the same time. The first (kind of naive) implementation v0.1.x had a bug where solely one entity of the same id where able to run at the same time.

The Eventsourced support was re-implemented using the CRDT stream runner and its way to run a messages stream. Similar, error handing was adopted and also the new atomic command handling error behaviour with entity restart support has been added.

The Integration Tests for the CRDT implementation follow the new model-based approach of the Cloudstate TCK. They run with an in-memory gRCP Client and Server and are written in Go. These tests are yet not available in the Cloudstate TCK and complement them having local functional tests and also tests not done by the TCK itself.

The Protobuf cleanup for go packages where necessary because the go_package used so far, where incompatible and incorrect. This is fixed in this milestone and also fixed in the main Cloudstate repository.

With Eventsoured TCK Support also Eventsourced effects and forward support was correctly implemented and validated with the TCK.

A Significant test coverage increase was achieved with about 80 integration- and 150 unit-tests in this milestone.

Updated documentation and CRDT documentation is done with the new Antora based template.

The new API has been used in different contexts of available examples and public use of cloudstate. One is the Dapr contrib model support which uses a CRDT state model. This branch https://github.com/mrcllnz/components-contrib/tree/feature/cloudstate_crdt_support implements the Dapr Cloudstate component with the new Go API.

non-canonical import path "github.com/cloudstateio/go-support/cloudstate/" (should be "github.com/cloudstateio/go-support/cloudstate")

Compiling the shopping cart example results in an error when compiling shoppingcart.pb.go
The import:
_ "github.com/cloudstateio/go-support/cloudstate/"
results in:
shopping/shoppingcart.pb.go:16:2: non-canonical import path "github.com/cloudstateio/go-support/cloudstate/" (should be "github.com/cloudstateio/go-support/cloudstate")
expected: no error

This can be addressed by removing the trailing / from
entity_key.proto as in:
option go_package = "github.com/cloudstateio/go-support/cloudstate;cloudstate";

TCK with CRDT Support

As with issue cloudstateio/cloudstate#316 started, this issue is about to have TCK support for CRDT state models.

The story so far is to have synthetic TCK test cases and possibly a human consumable use case like a shopping cart or the like. The linked issue should discuss that. The primary focus with this issue is on complete coverage through synthetic TCK tests.

Sub-Tasks

  • CRDT Tests for Commands
    • GCounter
    • PNCounter
    • GSet
    • ORSet
    • Flag
    • LWWRegister
    • ORMap
    • Vote
  • CRDT Tests for Streamed Commands
  • CRDT TCK Service (synthetic)
  • verify entity passivation => done by main TCK
  • provide an implementation for JS- or Java-Support => done by main TCK

Improve type safety for command, forward and effect handlers

The Go gRPC protobuf compiler plugin doesn't expose service descriptors. As a consequence, command, forward and effect handlers require the user to provide unchecked plain strings to reference a commands gRPC service name or services method names. The same applies for service names and command names used with effects and forwards.

Provide an idiomatic way to improve type safety in this regard.

Entity throws error after 30 seconds of inactivity

Hi,

I'm playing around with the shopping-cart example. After I send a command with a new user-id, the entity is created. If I keep sending commands to it, everything works fine. But if I stop for 30 seconds and send another command, I see a 500 Internal server error returned from sidecar. In the sidecar logs, I see the following:

2020-06-10 06:02:04.370 ERROR io.cloudstate.proxy.Serve$ - User Function responded with a failure: Incorrect command id in reply, expecting 1 but got 0
2020-06-10 06:02:04.374 ERROR akka.actor.ActorSystemImpl - Internal server error, sending 500 response
java.util.NoSuchElementException: head of empty stream
at akka.stream.scaladsl.Sink$.$anonfun$head$3(Sink.scala:198)
at scala.Option.getOrElse(Option.scala:189)
at akka.stream.scaladsl.Sink$.$anonfun$head$2(Sink.scala:198)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:56)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:74)
at akka.dispatch.internal.SameThreadExecutionContext$$anon$1.unbatchedExecute(SameThreadExecutionContext.scala:21)
at akka.dispatch.BatchingExecutor.execute(BatchingExecutor.scala:123)
at akka.dispatch.BatchingExecutor.execute$(BatchingExecutor.scala:117)
at akka.dispatch.internal.SameThreadExecutionContext$$anon$1.execute(SameThreadExecutionContext.scala:20)
at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:72)
at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:288)
at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:288)
at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:288)
at scala.concurrent.Promise.trySuccess(Promise.scala:94)
at scala.concurrent.Promise.trySuccess$(Promise.scala:94)
at scala.concurrent.impl.Promise$DefaultPromise.trySuccess(Promise.scala:187)
at akka.stream.impl.HeadOptionStage$$anon$3.onUpstreamFinish(Sinks.scala:236)
at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:523)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:390)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:625)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:502)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:600)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:769)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$shortCircuitBatch(ActorGraphInterpreter.scala:759)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:785)
at akka.actor.Actor.aroundReceive(Actor.scala:535)
at akka.actor.Actor.aroundReceive$(Actor.scala:533)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:691)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:577)
at akka.actor.ActorCell.invoke(ActorCell.scala:547)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

Provide a Logging story and implementation

… so that the user support library can output debug logs as well as users function can log in a way they have full control how to do it

see also discussion in #5 (comment)

Ideas

  1. We could provide an interface the user chooses to log with, where we decide on the logging library to use
  2. In contrast to 1. we could provide an interface that can be implemented by the user where the user decides how to log and if the user support is instructed to log debug messages, the support lib uses this implementation. This way we would not force the user to a dependency he does not like to use. If no implementation is choosen by the user, the support lib could just log with the std libs implementation.

Context.Fail() is not an idiomatic Go API

As follows, I think ctx.Fail to be accessibe by the user function is not an idiomatic Go API. The JavaScript and Java support use ctx.Fail to indicate a failed context and the user function would return a failure reply to the proxy.

In case of a context to be failed, the error value for the context failure is set by ctx.Fail() but it also could be idiomatcally returned as an error value by a command handled function. This would remove the unusual sequence of

ctx.Fail(error) and then returning (nil,nil) for such a function. Instead, we woul return the error as a "nomal" error value and this error value gets handled. The ctx.Fail() method could be still available as a private methode, so that internal processing can invalidate a context.

// AddItem implements the AddItem command handling of the shopping cart service.
func (sc *ShoppingCart) AddItem(ctx *eventsourced.Context, li *shoppingcart.AddLineItem) (*empty.Empty, error) {
	if li.GetQuantity() <= 0 {
		ctx.Fail(fmt.Errorf("cannot add negative quantity of to item %s", li.GetProductId()))
		return nil, nil
	}
	ctx.Emit(&domain.ItemAdded{
		Item: &domain.LineItem{
			ProductId: li.ProductId,
			Name:      li.Name,
			Quantity:  li.Quantity,
		}})
	return &empty.Empty{}, nil
}

Add devcontainer support

Add devcontainersupport.
Proposed docker image name would be: cloudstateio/go-support-devcontainer:latest

Passivation configuration by the discovery protocol

With cloudstateio/cloudstate#486 entity passivation can be configured through the discovery protocol.

//
// The semantics is to provide a flexible way for entity user functions to configure the passivation strategy.
// This strategy is sent to the proxy at discovery time allowing the proxy to configure the corresponding entities.
// The only passivation strategy supported is the timeout strategy and configuring this is optional for the entity.
// If an entity user function does not configure the passivation strategy the proxy used its fallback default value.
//
// The passivation strategy for the entity user function.
message EntityPassivationStrategy {
    oneof strategy {
        // the timeout passivation strategy.
        TimeoutPassivationStrategy timeout = 1;
    }
}

(crdt|eventsourced).Context vs. context.Context

We should have a story to enable user functions to get the context.Context from the gRPC server handler function in a clean way. context.Context shall be propagated (and not stored in struct types) by chained func calls, but having functions like

func handleCommand(c context.Context, ctx crdt.Context, ....)

is one context too much to bear for an API.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.