cloudstateio / go-support Goto Github PK
View Code? Open in Web Editor NEWUser Language Support for Go
Home Page: https://cloudstate.io/docs/go/gettingstarted.html
License: Apache License 2.0
User Language Support for Go
Home Page: https://cloudstate.io/docs/go/gettingstarted.html
License: Apache License 2.0
Currently, the main cloudstate tck will run the go tck binary off of an image on an external gcr.io.
Implement the cloudstate.crdt.Crdt Service from protobuf/protocol/cloudstate/crdt.proto
Tasks
WIP-branch
https://github.com/cloudstateio/go-support/tree/feature/crdt_support
The go-support library has to get the CloudState protocol prortobuf files from the main repository. A few options how to do that where discussed in cloudstateio/cloudstate#118.
Compiling the shopping cart example results in an error when compiling shoppingcart.pb.go
The import:
_ "github.com/cloudstateio/go-support/cloudstate/"
results in:
shopping/shoppingcart.pb.go:16:2: non-canonical import path "github.com/cloudstateio/go-support/cloudstate/" (should be "github.com/cloudstateio/go-support/cloudstate")
expected: no error
This can be addressed by removing the trailing /
from
entity_key.proto as in:
option go_package = "github.com/cloudstateio/go-support/cloudstate;cloudstate";
With polyglot user language documentation coming to the user support language repositories that brings dependencies like sbt and paradox, we better might have a separate go module inside the repository so that users don't go get
these artefacts. This could be also applicable for the TCK, which is probably not of interest for the user.
As implemented by cloudstateio/cloudstate#494
Spec:
https://github.com/cloudstateio/cloudstate/blob/master/protocols/protocol/cloudstate/action.proto
WIP:
https://github.com/cloudstateio/go-support/tree/feature/valueentity_state_model
issued by: #17
Implement the Cloudstate Actions Service from /protobuf/protocol/cloudstate/action.proto
WIP:
https://github.com/cloudstateio/go-support/tree/feature/action_state_model
According to cloudstateio/cloudstate#103 (comment)
As discussed on todays constributors call, ReportError should:
therefore, if the proxy does not close the stream, the entity will receive further commands/events for the entity. The user (developer) should receive the error and then decides what to do. It is advised to make the error discoverable, most probably through writing it to stderr/stdout.
Implement Eventing Support as introduced with cloudstateio/cloudstate#465
In general, streaming out services have to handle context cancellation properly, especially when no .Send() calls are involved to get errors like 'io.EOF' while the streaming user handler is running.
Backport issues found with #67
The new TCK uses a model based approach to test state models:
https://github.com/cloudstateio/cloudstate/blob/master/tck/src/main/scala/io/cloudstate/tck/CloudStateTCK.scala#L143
Implement this model similar to the Java implementation.
… so that the user support library can output debug logs as well as users function can log in a way they have full control how to do it
see also discussion in #5 (comment)
Ideas
With cloudstateio/cloudstate#486 entity passivation can be configured through the discovery protocol.
//
// The semantics is to provide a flexible way for entity user functions to configure the passivation strategy.
// This strategy is sent to the proxy at discovery time allowing the proxy to configure the corresponding entities.
// The only passivation strategy supported is the timeout strategy and configuring this is optional for the entity.
// If an entity user function does not configure the passivation strategy the proxy used its fallback default value.
//
// The passivation strategy for the entity user function.
message EntityPassivationStrategy {
oneof strategy {
// the timeout passivation strategy.
TimeoutPassivationStrategy timeout = 1;
}
}
Port the dapr contrib cloudstate state component as an exemplary use of the new CRDT support.
Tasks
I'm working on this. WIP branch:
https://github.com/mrcllnz/components-contrib/tree/feature/cloudstate_crdt_support
A Cloudstate user function instance should shutdown properly and gracefully.
Non-API code should be moved into internal packages.
As introduced by cloudstateio/cloudstate#500 the Shopping Cart is not anymore part of the TCK. We have to remove it.
We should have a story to enable user functions to get the context.Context from the gRPC server handler function in a clean way. context.Context shall be propagated (and not stored in struct types) by chained func calls, but having functions like
func handleCommand(c context.Context, ctx crdt.Context, ....)
is one context too much to bear for an API.
The Go gRPC protobuf compiler plugin doesn't expose service descriptors. As a consequence, command, forward and effect handlers require the user to provide unchecked plain strings to reference a commands gRPC service name or services method names. The same applies for service names and command names used with effects and forwards.
Provide an idiomatic way to improve type safety in this regard.
The protobuf apiv2 has been released a few weeks ago. The protbuf project supports the apiv1 indefinitely but we don't have any obligations to do so explicitly.
refs:
tasks:
There are dead links in new antora documentation:
For cloudstateio/cloudstate#479 cloudstateio/cloudstate@a0ae7b0 removed the state setting in the CRDT protocol.
Hi,
I'm playing around with the shopping-cart example. After I send a command with a new user-id, the entity is created. If I keep sending commands to it, everything works fine. But if I stop for 30 seconds and send another command, I see a 500 Internal server error returned from sidecar. In the sidecar logs, I see the following:
2020-06-10 06:02:04.370 ERROR io.cloudstate.proxy.Serve$ - User Function responded with a failure: Incorrect command id in reply, expecting 1 but got 0
2020-06-10 06:02:04.374 ERROR akka.actor.ActorSystemImpl - Internal server error, sending 500 response
java.util.NoSuchElementException: head of empty stream
at akka.stream.scaladsl.Sink$.$anonfun$head$3(Sink.scala:198)
at scala.Option.getOrElse(Option.scala:189)
at akka.stream.scaladsl.Sink$.$anonfun$head$2(Sink.scala:198)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:56)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:74)
at akka.dispatch.internal.SameThreadExecutionContext$$anon$1.unbatchedExecute(SameThreadExecutionContext.scala:21)
at akka.dispatch.BatchingExecutor.execute(BatchingExecutor.scala:123)
at akka.dispatch.BatchingExecutor.execute$(BatchingExecutor.scala:117)
at akka.dispatch.internal.SameThreadExecutionContext$$anon$1.execute(SameThreadExecutionContext.scala:20)
at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:72)
at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:288)
at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:288)
at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:288)
at scala.concurrent.Promise.trySuccess(Promise.scala:94)
at scala.concurrent.Promise.trySuccess$(Promise.scala:94)
at scala.concurrent.impl.Promise$DefaultPromise.trySuccess(Promise.scala:187)
at akka.stream.impl.HeadOptionStage$$anon$3.onUpstreamFinish(Sinks.scala:236)
at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:523)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:390)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:625)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:502)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:600)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:769)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$shortCircuitBatch(ActorGraphInterpreter.scala:759)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:785)
at akka.actor.Actor.aroundReceive(Actor.scala:535)
at akka.actor.Actor.aroundReceive$(Actor.scala:533)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:691)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:577)
at akka.actor.ActorCell.invoke(ActorCell.scala:547)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
PR cloudstateio/python-support#35 shows how to setup user support library based documentation.
The godoc and travis-ci ref in README.md points to the wrong repo.
As the go-support library is separated from the main CloudState repository the TCK within CloudState has to be adapted to include a build of the TCK Shopping Cart Example implemented in Go that can be used for TCK runs.
as described here, remove handling by reflection, introduce handleCommand instead
#5 (comment)
As follows, I think ctx.Fail to be accessibe by the user function is not an idiomatic Go API. The JavaScript and Java support use ctx.Fail to indicate a failed context and the user function would return a failure reply to the proxy.
In case of a context to be failed, the error value for the context failure is set by ctx.Fail() but it also could be idiomatcally returned as an error value by a command handled function. This would remove the unusual sequence of
ctx.Fail(error)
and then returning (nil,nil)
for such a function. Instead, we woul return the error as a "nomal" error value and this error value gets handled. The ctx.Fail() method could be still available as a private methode, so that internal processing can invalidate a context.
// AddItem implements the AddItem command handling of the shopping cart service.
func (sc *ShoppingCart) AddItem(ctx *eventsourced.Context, li *shoppingcart.AddLineItem) (*empty.Empty, error) {
if li.GetQuantity() <= 0 {
ctx.Fail(fmt.Errorf("cannot add negative quantity of to item %s", li.GetProductId()))
return nil, nil
}
ctx.Emit(&domain.ItemAdded{
Item: &domain.LineItem{
ProductId: li.ProductId,
Name: li.Name,
Quantity: li.Quantity,
}})
return &empty.Empty{}, nil
}
Reduce source code included in the paradox documentation, but what is there should be automatically verified. The documentation lives for now in the main repository:
https://github.com/marcellanz/cloudstate/tree/feature/go-support_additions/docs/src/test/go/docs/user/eventsourced
The main task here is, how to verify these files as they are not in the go-support repository.
As with issue cloudstateio/cloudstate#316 started, this issue is about to have TCK support for CRDT state models.
The story so far is to have synthetic TCK test cases and possibly a human consumable use case like a shopping cart or the like. The linked issue should discuss that. The primary focus with this issue is on complete coverage through synthetic TCK tests.
Sub-Tasks
… and how it's used. As mentioned in PR reviews, the event emitter could be passed through a context object that is passed explicitly to handlers.
Tasks
An issue collecting features and changes for the "Go Support 0.2.0" release:
devcontainer.json
SupportCRDT support has been added as the second state model supported. Together with the Eventsoured implementation, stream runners where introduced which enables multiple entities of the same id to be run at the same time. The first (kind of naive) implementation v0.1.x had a bug where solely one entity of the same id where able to run at the same time.
The Eventsourced support was re-implemented using the CRDT stream runner and its way to run a messages stream. Similar, error handing was adopted and also the new atomic command handling error behaviour with entity restart support has been added.
The Integration Tests for the CRDT implementation follow the new model-based approach of the Cloudstate TCK. They run with an in-memory gRCP Client and Server and are written in Go. These tests are yet not available in the Cloudstate TCK and complement them having local functional tests and also tests not done by the TCK itself.
The Protobuf cleanup for go packages where necessary because the go_package
used so far, where incompatible and incorrect. This is fixed in this milestone and also fixed in the main Cloudstate repository.
With Eventsoured TCK Support also Eventsourced effects and forward support was correctly implemented and validated with the TCK.
A Significant test coverage increase was achieved with about 80 integration- and 150 unit-tests in this milestone.
Updated documentation and CRDT documentation is done with the new Antora based template.
The new API has been used in different contexts of available examples and public use of cloudstate. One is the Dapr contrib model support which uses a CRDT state model. This branch https://github.com/mrcllnz/components-contrib/tree/feature/cloudstate_crdt_support implements the Dapr Cloudstate component with the new Go API.
The travis-ci configuration has some tasks that are valuable also for local development, make them available by a Makefile.
Tasks
Right now we start the server using PORT
and HOST
environment variables.
Context: https://github.com/dapr/components-contrib/blob/v0.1.0/state/cloudstate/cloudstate_crdt.go#L266
We might start the server by parameters given through the cloudstate.CloudState
API.
Based on cloudstateio/cloudstate#491 add the TCK model for the CRDT state model.
Add devcontainer
support.
Proposed docker image name would be: cloudstateio/go-support-devcontainer:latest
Hi @marcellanz,
In file, https://github.com/cloudstateio/go-support/blob/master/protobuf/frontend/cloudstate/entity_key.proto, the go_package option has a tailing slash,
option go_package = "github.com/cloudstateio/go-support/cloudstate/;cloudstate";
which causes the following import in the generated go code,
_ "github.com/cloudstateio/go-support/cloudstate/"
go compile reports error "non-canonical import path" on such code.
TestEventsourcingShoppingCart fails occasionally on travis:
-- FAIL: TestEventsourcingShoppingCart/the_entity_should_have_consistent_state_after_a_context_failure (0.00s)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.