serviceweaver / weaver Goto Github PK
View Code? Open in Web Editor NEWProgramming framework for writing and deploying cloud applications.
Home Page: https://serviceweaver.dev
License: Apache License 2.0
Programming framework for writing and deploying cloud applications.
Home Page: https://serviceweaver.dev
License: Apache License 2.0
Hi everyone, thank you for introducing the new way of service implementation.
I'm trying to implement a POC for the service weaver and I stumbled upon this error
internal/domain/reverser/reverser.go:16:6: weaver.Implements argument contract.Reverser is a type outside the current package.
The thing is, implementing the contract in a separate package could be a better solution, well at least for the implementation I'm tryna made.
I'm just wondering, is there any specific case of this rule that prevented the generator to use the interface contract outside of the implementation package?
// Implementation of the Reverser component.
type reverser struct {
weaver.Implements[contract.Reverser]
}
step1: weaver multi deploy weaver.toml
╭───────────────────────────────────────────────────╮
│ app : hello │
│ deployment : cb92a3e5-a956-46ff-86ac-269d7b09b0c9 │
╰───────────────────────────────────────────────────╯
S0321 15:16:50.243458 stdout 567250c1 ] hello listener available on 127.0.0.1:12345
S0321 15:16:50.243765 stdout 020d2067 ] hello listener available on 127.0.0.1:12345
step2: curl "localhost:12345/hello?name=Weaver"
keep blocking
step3: weaver multi status
╭──────────────────────────────────────────────────────╮
│ DEPLOYMENTS │
├───────┬──────────────────────────────────────┬───────┤
│ APP │ DEPLOYMENT │ AGE │
├───────┼──────────────────────────────────────┼───────┤
│ hello │ cb92a3e5-a956-46ff-86ac-269d7b09b0c9 │ 3m56s │
╰───────┴──────────────────────────────────────┴───────╯
╭───────────────────────────────────────────────╮
│ COMPONENTS │
├───────┬────────────┬───────────┬──────────────┤
│ APP │ DEPLOYMENT │ COMPONENT │ REPLICA PIDS │
├───────┼────────────┼───────────┼──────────────┤
│ hello │ cb92a3e5 │ │ │
│ hello │ cb92a3e5 │ main │ 15208, 15209 │
╰───────┴────────────┴───────────┴──────────────╯
╭─────────────────────────────────────────────────╮
│ LISTENERS │
├───────┬────────────┬──────────┬─────────────────┤
│ APP │ DEPLOYMENT │ LISTENER │ ADDRESS │
├───────┼────────────┼──────────┼─────────────────┤
│ hello │ cb92a3e5 │ hello │ 127.0.0.1:12345 │
╰───────┴────────────┴──────────┴─────────────────╯
component only one , expect two.
the doc has $ signs in the beginning. While its visually appealing it makes copying the command difficult.
Either we remove the dollar sign all the commands or make the copy button no copy the $ sign.
$ go run . # Run locally.
$ weaver gke deploy weaver.toml # Run in the cloud.
Hi folks! Thanks a lot for your efforts with ServiceWeaver!
I'm learning how to work with it and specifically to deploy it to GKE.
Here is an example of my weaver.toml file:
[serviceweaver]
name = "project"
binary = "./project"
[gke]
project = "project"
account = "[email protected]"
regions = ["europe-west2"]
public_listener = [
{name = "project", hostname = "project.com"},
]
["/.../project/components/A"]
A = "test"
B = "test"
["/.../project/components/B"]
A = "test"
B = "test"
When I run the GKE deployment I see the following line:
Creating cluster serviceweaver-config in location us-central1... Done
After all, in GCP console I see that serviceweaver-config cluster is deployed in us-central1 region.
My project's zone:
➜ $ gcloud config get compute/zone
europe-west2-a
Is it expected that serviceweaver-config is always deployed in us-central1 and ignores weaver.toml configuration?
If yes - why it's needed?
Thanks in advance!
Currently weaver is using a custom logger; https://github.com/ServiceWeaver/weaver/blob/main/logger.go, https://github.com/ServiceWeaver/weaver/tree/main/runtime/logging
There's a proposal to bring structured logging to the Go standard library. That proposal looks likely to be accepted; It was marked as;
Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group
2days ago: golang/go#56345 (comment)
An implementation of that proposal already exists at; https://pkg.go.dev/golang.org/x/exp/slog
Since weaver is just starting, I think it should switch to golang.org/x/exp/slog
early on. It does not have to do so right away, we can wait a few more days to see if the exp/slog proposal is accepted upstream
The docs do not describe a ssh deployment using weaver. Is it not possible or should it be written?
#79 reported an issue building on MacOS, which was resolved by explicitly documenting installation instructions. However, during the discussion, the idea of removing the sqlite CGO depencency was raised. I would like to revive this discussion as I believe there are many reasons to remove any CGO dependencies in a Go project including:
Please replace the CGO sqlite driver with the CGO free one from https://pkg.go.dev/modernc.org/sqlite.
when i click stop
button in golang idea debug
mode, the file in share\serviceweaver\single_registry
cannot be deleted.
In run
mode or ctrl + c
is ok
╭───────────────────────────────────────────────────╮
│ app : collatz │
│ deployment : bb245bfa-1a97-4e0e-92b7-4e9d33d53621 │
╰───────────────────────────────────────────────────╯
S0310 15:22:06.728927 stderr ea38724c ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
S0310 15:22:06.729157 stderr 303508ff ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
GO111MODULE="" GOARCH="arm64" GOBIN="" GOCACHE="/Users/devops/Library/Caches/go-build" GOENV="/Users/devops/Library/Application Support/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GOMODCACHE="/Users/devops/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/devops/go" GOPRIVATE="" GOPROXY="https://goproxy.cn,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GOVCS="" GOVERSION="go1.19.5" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/devops/go/src/serviceweaver/go.mod" GOWORK="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lk/k280wks50hv_1zt5y1lmrlhw0000gn/T/go-build3603867559=/tmp/go-build -gno-record-gcc-switches -fno-common"
go install github.com/ServiceWeaver/weaver/cmd/weaver@latest
in .bashrc and .bash_profile
export PATH="$PATH:$GOPATH/bin"
source ~/(.bashrc | .bash_profile)
weaver --help
zsh: command not found: weaver
what more information do I need to provide?
Assuming I am like this now,
package main
import (
"context"
"fmt"
"github.com/ServiceWeaver/weaver"
)
type Adder interface {
Add(context.Context, int, int) (int, error)
}
type adder struct {
weaver.Implements[Adder]
}
func (a adder) Add(_ context.Context, x, y int) (int, error) {
return x + y, nil
}
type adder2 struct {
weaver.Implements[Adder]
}
func (a adder2) Add(_ context.Context, x, y int) (int, error) {
return x + 2*y, nil
}
func main() {
ctx := context.Background()
root := weaver.Init(ctx)
addrer, err := weaver.Get[Adder](root)
if err != nil {
panic(err)
}
sum, err := addrer.Add(ctx, 1, 2)
if err != nil {
panic(err)
}
fmt.Println(sum)
}
Both adder and adder2 have implemented weaver.Implements[Adder],
run
weaver generate.
it will generate a duplicate adder_local_stub struct.
package main
// Code generated by "weaver generate". DO NOT EDIT.
import (
"context"
"github.com/ServiceWeaver/weaver/runtime/codegen"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/trace"
"reflect"
"time"
)
func init() {
codegen.Register(codegen.Registration{
Name: "weaver-test/Adder",
Iface: reflect.TypeOf((*Adder)(nil)).Elem(),
New: func() any { return &adder{} },
LocalStubFn: func(impl any, tracer trace.Tracer) any { return adder_local_stub{impl: impl.(Adder), tracer: tracer} },
ClientStubFn: func(stub codegen.Stub, caller string) any {
return adder_client_stub{stub: stub, addMetrics: codegen.MethodMetricsFor(codegen.MethodLabels{Caller: caller, Component: "weaver-test/Adder", Method: "Add"})}
},
ServerStubFn: func(impl any, addLoad func(uint64, float64)) codegen.Server {
return adder_server_stub{impl: impl.(Adder), addLoad: addLoad}
},
})
codegen.Register(codegen.Registration{
Name: "weaver-test/Adder",
Iface: reflect.TypeOf((*Adder)(nil)).Elem(),
New: func() any { return &adder2{} },
LocalStubFn: func(impl any, tracer trace.Tracer) any { return adder_local_stub{impl: impl.(Adder), tracer: tracer} },
ClientStubFn: func(stub codegen.Stub, caller string) any {
return adder_client_stub{stub: stub, addMetrics: codegen.MethodMetricsFor(codegen.MethodLabels{Caller: caller, Component: "weaver-test/Adder", Method: "Add"})}
},
ServerStubFn: func(impl any, addLoad func(uint64, float64)) codegen.Server {
return adder_server_stub{impl: impl.(Adder), addLoad: addLoad}
},
})
}
// Local stub implementations.
type adder_local_stub struct {
impl Adder
tracer trace.Tracer
}
func (s adder_local_stub) Add(ctx context.Context, a0 int, a1 int) (r0 int, err error) {
span := trace.SpanFromContext(ctx)
if span.SpanContext().IsValid() {
// Create a child span for this method.
ctx, span = s.tracer.Start(ctx, "main.Adder.Add", trace.WithSpanKind(trace.SpanKindInternal))
defer func() {
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
}
span.End()
}()
}
return s.impl.Add(ctx, a0, a1)
}
type adder_local_stub struct {
impl Adder
tracer trace.Tracer
}
func (s adder_local_stub) Add(ctx context.Context, a0 int, a1 int) (r0 int, err error) {
span := trace.SpanFromContext(ctx)
if span.SpanContext().IsValid() {
// Create a child span for this method.
ctx, span = s.tracer.Start(ctx, "main.Adder.Add", trace.WithSpanKind(trace.SpanKindInternal))
defer func() {
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
}
span.End()
}()
}
return s.impl.Add(ctx, a0, a1)
}
// Client stub implementations.
type adder_client_stub struct {
stub codegen.Stub
addMetrics *codegen.MethodMetrics
}
func (s adder_client_stub) Add(ctx context.Context, a0 int, a1 int) (r0 int, err error) {
// Update metrics.
start := time.Now()
s.addMetrics.Count.Add(1)
span := trace.SpanFromContext(ctx)
if span.SpanContext().IsValid() {
// Create a child span for this method.
ctx, span = s.stub.Tracer().Start(ctx, "main.Adder.Add", trace.WithSpanKind(trace.SpanKindClient))
}
defer func() {
// Catch and return any panics detected during encoding/decoding/rpc.
if err == nil {
err = codegen.CatchPanics(recover())
}
err = s.stub.WrapError(err)
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
s.addMetrics.ErrorCount.Add(1)
}
span.End()
s.addMetrics.Latency.Put(float64(time.Since(start).Microseconds()))
}()
// Preallocate a buffer of the right size.
size := 0
size += 8
size += 8
enc := codegen.NewEncoder()
enc.Reset(size)
// Encode arguments.
enc.Int(a0)
enc.Int(a1)
var shardKey uint64
// Call the remote method.
s.addMetrics.BytesRequest.Put(float64(len(enc.Data())))
var results []byte
results, err = s.stub.Run(ctx, 0, enc.Data(), shardKey)
if err != nil {
return
}
s.addMetrics.BytesReply.Put(float64(len(results)))
// Decode the results.
dec := codegen.NewDecoder(results)
r0 = dec.Int()
err = dec.Error()
return
}
type adder_client_stub struct {
stub codegen.Stub
addMetrics *codegen.MethodMetrics
}
func (s adder_client_stub) Add(ctx context.Context, a0 int, a1 int) (r0 int, err error) {
// Update metrics.
start := time.Now()
s.addMetrics.Count.Add(1)
span := trace.SpanFromContext(ctx)
if span.SpanContext().IsValid() {
// Create a child span for this method.
ctx, span = s.stub.Tracer().Start(ctx, "main.Adder.Add", trace.WithSpanKind(trace.SpanKindClient))
}
defer func() {
// Catch and return any panics detected during encoding/decoding/rpc.
if err == nil {
err = codegen.CatchPanics(recover())
}
err = s.stub.WrapError(err)
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
s.addMetrics.ErrorCount.Add(1)
}
span.End()
s.addMetrics.Latency.Put(float64(time.Since(start).Microseconds()))
}()
// Preallocate a buffer of the right size.
size := 0
size += 8
size += 8
enc := codegen.NewEncoder()
enc.Reset(size)
// Encode arguments.
enc.Int(a0)
enc.Int(a1)
var shardKey uint64
// Call the remote method.
s.addMetrics.BytesRequest.Put(float64(len(enc.Data())))
var results []byte
results, err = s.stub.Run(ctx, 0, enc.Data(), shardKey)
if err != nil {
return
}
s.addMetrics.BytesReply.Put(float64(len(results)))
// Decode the results.
dec := codegen.NewDecoder(results)
r0 = dec.Int()
err = dec.Error()
return
}
// Server stub implementations.
type adder_server_stub struct {
impl Adder
addLoad func(key uint64, load float64)
}
// GetStubFn implements the stub.Server interface.
func (s adder_server_stub) GetStubFn(method string) func(ctx context.Context, args []byte) ([]byte, error) {
switch method {
case "Add":
return s.add
default:
return nil
}
}
func (s adder_server_stub) add(ctx context.Context, args []byte) (res []byte, err error) {
// Catch and return any panics detected during encoding/decoding/rpc.
defer func() {
if err == nil {
err = codegen.CatchPanics(recover())
}
}()
// Decode arguments.
dec := codegen.NewDecoder(args)
var a0 int
a0 = dec.Int()
var a1 int
a1 = dec.Int()
// TODO(rgrandl): The deferred function above will recover from panics in the
// user code: fix this.
// Call the local method.
r0, appErr := s.impl.Add(ctx, a0, a1)
// Encode the results.
enc := codegen.NewEncoder()
enc.Int(r0)
enc.Error(appErr)
return enc.Data(), nil
}
type adder_server_stub struct {
impl Adder
addLoad func(key uint64, load float64)
}
// GetStubFn implements the stub.Server interface.
func (s adder_server_stub) GetStubFn(method string) func(ctx context.Context, args []byte) ([]byte, error) {
switch method {
case "Add":
return s.add
default:
return nil
}
}
func (s adder_server_stub) add(ctx context.Context, args []byte) (res []byte, err error) {
// Catch and return any panics detected during encoding/decoding/rpc.
defer func() {
if err == nil {
err = codegen.CatchPanics(recover())
}
}()
// Decode arguments.
dec := codegen.NewDecoder(args)
var a0 int
a0 = dec.Int()
var a1 int
a1 = dec.Int()
// TODO(rgrandl): The deferred function above will recover from panics in the
// user code: fix this.
// Call the local method.
r0, appErr := s.impl.Add(ctx, a0, a1)
// Encode the results.
enc := codegen.NewEncoder()
enc.Int(r0)
enc.Error(appErr)
return enc.Data(), nil
}
Perhaps currently only the following features are available:
weaver gke deploy
for the first timeI would like to see the following features added:
Cleanup of all GCP components
-> For those who first encounter weaver through the tutorial, most will test it for a while and then remove the test environment.
For now, it looks like I'll probably have to clear it myself.
Wouldn't it be nice if we could explicitly do gke init/cleanup via weaver-cli?
Inquiry/change of GCP infrastructure resources controlled by weaver
-> Like terraform, if we can check and discern which resources are controlled by weaver, it will be helpful for ongoing infrastructure management.
What do you think about this?
when I manually killed components' processes and then used command weaver single/multi status
, I found the killed pids are still presented in column REPLICA PIDS
. The corresponding codes are in file internal/status/status.go
:
// formatDeployments pretty-prints the set of components.
func formatComponents(w io.Writer, statuses []*Status) {
...
for _, status := range statuses {
...
for _, component := range status.Components {
prefix, _ := formatId(status.DeploymentId)
c := logging.ShortenComponent(component.Name)
sort.Slice(component.Pids, func(i, j int) bool {
return component.Pids[i] < component.Pids[j]
})
pids := make([]string, len(component.Pids))
for i, pid := range component.Pids {
pids[i] = fmt.Sprint(pid)
}
t.Row(status.App, prefix, c, strings.Join(pids, ", "))
}
}
}
Maybe check for process liveness is needed before assembling pids info. Glad to help if you think this optimization is needed.
os: macOS Catalina Version 10.15.7
go version go1.20.1 darwin/amd64
commit: 1bd05fa
deploy: examples/hello weaver multi deploy weaver.toml
benchmark command: ab -n 100000 -c 8 -k http://10.79.161.90:12345/hello?name=world
around finishing 40000 requests, it would throw errors like "http: proxy error: dial tcp 127.0.0.1:54178: connect: can't assign requested address"
found the dial is from http proxy that launched by weaver, and default transport does not limit MaxConnsPerHost, connections were massively created when under pressure;
tried to set transport args MaxConnsPerHost and MaxIdleConnsPerHost to a high number, tried (100, 100), (150, 100), (100, 150), no longer seeing the same error above, but a new issue come out "panic: net/http: internal error: connCount underflow", this one seems to come up intermittently, see golang/go#38172
tried the same on a Debian 5.4.143, however the issue didn't reproduce, it seemed to be macOS related.
This framework is really amazing, is there any plan for developing a C++ version?
Assume that the Reverser
component will call another Adder
component and I want the method call is exexuted as RPC
instead of single process when running weaver multi deploy
command.
I just making a few changes to the code in tutorial :
func (r *reverser) Reverse(ctx context.Context, s string) (string, error) {
runes := []rune(s)
n := len(runes)
for i := 0; i < n/2; i++ {
runes[i], runes[n-i-1] = runes[n-i-1], runes[i]
}
a, _ := weaver.Get[Adder](common.GlobalInstance(ctx))
add, err := a.Add(ctx, rand.Int63(), rand.Int63())
if err != nil {
log.Fatal(err)
}
return fmt.Sprintf("%s_%d", string(runes), add), nil
}
common.GlobalInstance
:
var instance weaver.Instance
var once sync.Once
func GlobalInstance(ctx context.Context) weaver.Instance {
if instance == nil {
once.Do(func() {
instance = weaver.Init(ctx)
})
}
return instance
}
The application can startup without error log, however, there is no output when I curl 127.0.0.1:12345/hello?name=weaver
.
> weaver multi deploy weaver.toml
╭───────────────────────────────────────────────────╮
│ app : hello │
│ deployment : ac742841-b8c5-44d0-bd98-807a798050d8 │
╰───────────────────────────────────────────────────╯
S0305 10:16:59.219106 stderr 0bd24c35 ] panic: listener "hello": localhost:8080 already in use
S0305 10:16:59.219231 stderr 0bd24c35 ]
S0305 10:16:59.219239 stderr 0bd24c35 ] goroutine 1 [running]:
S0305 10:16:59.219247 stderr 0bd24c35 ] main.main()
S0305 10:16:59.219254 stderr 0bd24c35 ] hello/main.go:18 +0xc5
S0305 10:16:59.230226 stderr 9fbaada9 ] panic: listener "hello": localhost:8080 already in use
S0305 10:16:59.230359 stderr 9fbaada9 ]
S0305 10:16:59.230368 stderr 9fbaada9 ] goroutine 1 [running]:
S0305 10:16:59.230375 stderr 9fbaada9 ] main.main()
S0305 10:16:59.230382 stderr 9fbaada9 ] hello/main.go:18 +0xc5
Rolling update doesn't work well in gke.
In my opinion, if I redeploy with the same toml configuration, weaver should basically do a rolling update with no downtime.
If not, please correct me.
So, currently I have implemented the health check API in the form of "GET /".
And actually, the old version of the deployment is going down based on the health check.
The problem is that switching between versions of the server is not done non-disruptively.
A load balancing error occurs for a while as above.
What could be the problem?
# weaver.toml
[serviceweaver]
name = "weaver-start"
binary = "./weaver-start"
[gke]
regions = ["asia-northeast3"]
public_listener = [
{name = "weaver-start", hostname = "myyrakle.com"},
]
First of all thank you for the framework. I'm wondering how to use the custom mux with listeners like:
ctx := context.Background()
root := weaver.Init(ctx)
opts := weaver.ListenerOptions{ LocalAddress: ":4001"}
listner, err := root.Listener("greeter", opts)
if err != nil {
log.Fatal(err)
}
// custom mux
mux := chi.NewRouter()
mux..Get("/", func(w http.ResponseWriter, _ *http.Request) {
response.Write([]bye("Hello"))
})
http.Serve(listner, mux)
When I run the app, my custom routes made with mux are not working. Please let me know if we can do this and where I'm doing wrong.
Is it possible to integrate weaver with cloud-native tools ( like argocd , external DNS, sealed secrets ..... etc ) ?
for example:
Weaver-docs has a guide for local machine multiprocess deployment, but I don't see a guide for deployment to multiple machines。
Hi,
../../go/pkg/mod/github.com/!service!weaver/[email protected]/runtime/logging/logger.go:85:9: cannot use h (variable of type *LogHandler) as slog.Handler value in return statement: *LogHandler does not implement slog.Handler (wrong type for method Handle)
have Handle(slog.Record) error
want Handle(context.Context, slog.Record) error
The handler used in the package is correct with the corresponding interface signature but the newer version of slog is using another Handle method signature. This leads to the error above. I'm doing something wrong while installing weaver on my local machine?
After to run the hello
example with go run .
the CPU consumption was at 100%
pprof output
(pprof) top
Showing nodes accounting for 29.98s, 100% of 29.99s total
Dropped 7 nodes (cum <= 0.15s)
flat flat% sum% cum cum%
15.66s 52.22% 52.22% 29.98s 100% github.com/ServiceWeaver/weaver.(*weavelet).watchComponentsToStart
9.71s 32.38% 84.59% 11.14s 37.15% github.com/ServiceWeaver/weaver/runtime/retry.(*Retry).Continue
3.18s 10.60% 95.20% 3.18s 10.60% github.com/ServiceWeaver/weaver.(*singleprocessEnv).GetComponentsToStart
1.43s 4.77% 100% 1.43s 4.77% context.(*emptyCtx).Err
0 0% 100% 29.98s 100% github.com/ServiceWeaver/weaver.startWork.func1
Failed to install weaver: Missing stdlib.h
Which I managed to get around by using the following: export CGO_ENABLED=1; export CC=gcc; go install github.com/ServiceWeaver/weaver/cmd/weaver@latest
. Not sure if it's worth mentioning this somewhere in the docs... Cheers!
go install github.com/ServiceWeaver/weaver/cmd/weaver@latest
go: downloading github.com/ServiceWeaver/weaver v0.1.1
go: downloading github.com/google/cel-go v0.12.5
go: downloading github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8
go: downloading google.golang.org/genproto v0.0.0-20221109142239-94d6d90a7d66
go: downloading go.opentelemetry.io/otel/trace v1.13.0
go: downloading golang.org/x/exp v0.0.0-20230206171751-46f607a40771
go: downloading go.opentelemetry.io/otel v1.13.0
go: downloading github.com/hashicorp/golang-lru/v2 v2.0.1
go: downloading github.com/mattn/go-sqlite3 v1.14.16
go: downloading go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.7.0
# github.com/mattn/go-sqlite3
../../../go/pkg/mod/github.com/mattn/[email protected]/backup.go:14:10: fatal error: 'stdlib.h' file not found
#include <stdlib.h>
^~~~~~~~~~
1 error generated.
Go env:
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/[HIDDEN]"
GOENV="/Users/[HIDDEN]"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/[HIDDEN]"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/[HIDDEN]"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.19.3/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.19.3/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.19.3"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="[HIDDEN]"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f7/pzdxngcx2cl4lm0xk722f3dr0000gr/T/go-build96214606=/tmp/go-build -gno-record-gcc-switches -fno-common"
I'm using it according to the official documentation. I have a question because I couldn't find a part where I could set up a replica.
P.S. It would be nice to have a Discussion for Q&A...
Hi there!
I'm learning how to work with ServiceWeaver and obviously have some questions that aren't covered in https://serviceweaver.dev.
Is there any communication channel that can be used for expertise sharing?
Many thanks!
Hi all, thank you for making this dream framework come true. I have been looking forward to this kind of tool in Go for a long time.
But sadly I noticed that, in the Serializable Types section of FAQ, chan t
type is not supported.
I am wondering if there are any current plans or ongoing discussions around supporting it. As it is something that I need, I would be also interested in getting involved and contributing to its development.
Hi,
quick question about the development of deployers. Is the weaver-gke deployer developed as described in this blog or is there something else to take in concern if you someone wants to develop an "official" deployer to contribute to the project?
Cheers.
Congratulations on the first release! ServiceWeaver looks very interesting and I'm looking for ways to contribute.
I noticed that errcheck
is disabled with a comment about too many errors. I've found this linter to be useful because unhandled errors can often cause bugs.
Would you be interested in a few small PRs to fix or ignore the current warnings so that the errcheck
linter can be enabled?
I'll leave some more detail in a comment below about how I'd approach that change.
I want to start this issue to track Nutanix private cloud environment as supported deployment target for ServiceWeaver.
I work at Nutanix and I am willing to drum-up support for ServiceWeaver within Nutanix and get this implemented.
weaver config:
[serviceweaver]
binary = "./hello"
colocate = [
["main"],
["com/bar/ReverserBar", "com/bar/ReverserFoo"],
]
code dir:
.
├── cmd
├── com
│ ├── bar
│ │ ├── bar.go
│ │ └── weaver_gen.go
│ └── foo
│ ├── foo.go
│ └── weaver_gen.go
├── go.mod
├── go.sum
├── hello
├── main.go
└── weaver.toml
multi delopy error:
error initializing Service Weaver: internal error creating weavelet: component "com.bar.ReverserBar" not registered
why? how to fixed it?
$ go build ./cmd/weaver/
go: downloading github.com/google/uuid v1.3.0
go: downloading golang.org/x/tools v0.2.0
go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
go: downloading google.golang.org/protobuf v1.28.1
go: downloading github.com/google/pprof v0.0.0-20221010195024-131d412537ea
go: downloading go.opentelemetry.io/otel/sdk v1.11.1
go: downloading github.com/fsnotify/fsnotify v1.5.4
go: downloading github.com/google/cel-go v0.12.5
go: downloading google.golang.org/genproto v0.0.0-20221109142239-94d6d90a7d66
go: downloading github.com/BurntSushi/toml v1.2.0
go: downloading go.opentelemetry.io/otel/trace v1.14.0
go: downloading github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8
go: downloading golang.org/x/exp v0.0.0-20230206171751-46f607a40771
go: downloading go.opentelemetry.io/otel v1.14.0
go: downloading go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.7.0
go: downloading github.com/hashicorp/golang-lru/v2 v2.0.1
go: downloading github.com/mattn/go-sqlite3 v1.14.16
go: downloading golang.org/x/sys v0.1.0
go: downloading golang.org/x/text v0.7.0
go: downloading github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20220418222510-f25a4f6275ed
go: downloading github.com/stoewer/go-strcase v1.2.0
go: downloading github.com/go-logr/logr v1.2.3
go: downloading github.com/go-logr/stdr v1.2.2
go: downloading golang.org/x/mod v0.6.0
github.com/ServiceWeaver/weaver/runtime/perfetto
runtime\perfetto\db.go:539:23: undefined: sqlite3.Error
runtime\perfetto\db.go:541:41: undefined: sqlite3.ErrBusy
runtime\perfetto\db.go:541:77: undefined: sqlite3.ErrLocked
go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\xxx\AppData\Local\go-build
set GOENV=C:\Users\xxx\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\xxx\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\xxx\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=D:\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=D:\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.1
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=0
set GOMOD=D:\Projects\weaver\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\xxx\AppData\Local\Temp\go-build2543708436=/tmp/go-build -gno-record-gcc-switches
same with go install
command
When i use go install github.com/ServiceWeaver/weaver-gke/cmd/weaver-gke-local@latest and go install github.com/ServiceWeaver/weaver/cmd/weaver@latest , to use this tutorial .
Version "eb99fc75-e9d8-4288-bc73-65523bdb3d5a" of app "hello" started successfully.
Note that stopping this binary will not affect the app in any way.
Tailing the logs...
S0314 18:01:20.394463 stderr 71addc96 ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
S0314 18:01:20.406475 stderr 3e6318a8 ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
S0314 18:01:20.432982 stderr 71addc96 ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
S0314 18:01:20.447786 stderr 3e6318a8 ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
S0314 18:01:20.475621 stderr 71addc96 ] error initializing Service Weaver: internal error creating weavelet: new weavelet conn: string field contains invalid UTF-8
when i use the weaver gke-local status
╭──────────────────────────────────────────────────────────────────╮
│ Deployments │
├───────┬──────────────────────────────────────┬────────┬──────────┤
│ APP │ DEPLOYMENT │ AGE │ STATUS │
├───────┼──────────────────────────────────────┼────────┼──────────┤
│ hello │ 8d0b6e69-c2fa-49c6-8f07-01d2ecc9ca7d │ 53m52s │ DELETING │
│ hello │ 60991bdc-02d8-4fb1-a6d7-124c837317cc │ 48m51s │ DELETING │
│ hello │ 6f5ce217-39a8-436c-8524-4c3865af7158 │ 47m26s │ DELETING │
│ hello │ 8942443f-2e0c-4a19-898e-15828e0ef55b │ 45m37s │ DELETING │
│ hello │ 3a46ab2a-5a2e-4486-8982-1e5d6776be98 │ 41m45s │ DELETING │
│ hello │ 45a552ba-d610-453f-930e-0f623555e1c5 │ 40m13s │ DELETING │
│ hello │ 1bac82e7-9dd3-4fa2-92a3-c31e83d5c4ea │ 33m49s │ DELETING │
│ hello │ 06d3532d-325e-4239-9417-7c464b1178f1 │ 28m36s │ DELETING │
│ hello │ a9176f47-587c-4f03-81a6-ec3a9d4155f4 │ 28m21s │ DELETING │
│ hello │ abce5cb9-d349-455c-befc-11b5e592f466 │ 27m47s │ DELETING │
│ hello │ 77088351-d631-4f49-b967-6e570fad91c4 │ 18m32s │ DELETING │
│ hello │ a48bcd83-c105-40f5-ae8d-e2e09ceafde6 │ 16m21s │ DELETING │
│ hello │ a9188e41-6a4f-4452-9511-e0ac2ac36a5e │ 14m33s │ DELETING │
│ hello │ f1353f27-52e0-4bd6-b7e4-d4742f70ce4d │ 11m21s │ DELETING │
│ hello │ e54ad469-b832-4e30-86e9-ebc4b46ecf7a │ 10m17s │ DELETING │
│ hello │ 306ba873-6bef-4110-ac46-a82a607b4c15 │ 8m15s │ DELETING │
│ hello │ 54f956ea-effa-4584-a593-d13b2ccf7b53 │ 8m6s │ DELETING │
│ hello │ 1c23e162-a3a5-484b-803d-1a5f713e831c │ 6m8s │ DELETING │
│ hello │ 72972e52-6398-4140-a47c-bfffe3ec0759 │ 5m3s │ DELETING │
│ hello │ 5193b0f1-ba8e-456e-8d7f-213e0948c226 │ 2m18s │ DELETING │
│ hello │ eb99fc75-e9d8-4288-bc73-65523bdb3d5a │ 26s │ ACTIVE │
╰───────┴──────────────────────────────────────┴────────┴──────────╯
╭───────────────────────────────────────────────────╮
│ COMPONENTS │
├─────┬────────────┬──────────┬───────────┬─────────┤
│ APP │ DEPLOYMENT │ LOCATION │ COMPONENT │ HEALTHY │
├─────┼────────────┼──────────┼───────────┼─────────┤
╰─────┴────────────┴──────────┴───────────┴─────────╯
╭──────────────────────────────────────────────────────────────────────────────╮
│ TRAFFIC │
├──────┬────────────┬─────┬────────────┬──────────┬─────────┬──────────────────┤
│ HOST │ VISIBILITY │ APP │ DEPLOYMENT │ LOCATION │ ADDRESS │ TRAFFIC FRACTION │
├──────┼────────────┼─────┼────────────┼──────────┼─────────┼──────────────────┤
╰──────┴────────────┴─────┴────────────┴──────────┴─────────┴──────────────────╯
╭────────────────────────────╮
│ ROLLOUT OF hello │
├─────────────────┬──────────┤
│ │ us-west1 │
├─────────────────┼──────────┤
│ TIME │ eb99fc75 │
│ Mar 14 10:01:46 │ 1.00 │
╰─────────────────┴──────────╯
curl --header 'Host: hello.com' "localhost:8000/hello?name=Weaver"
host "hello.com" not found
this feels like an env variable problem, but its not clear which var or how to override.
when following the tutorial, i get to the multi deploy
phase and i get this output when trying to hit the endpoint for the first time:
S0313 11:24:35.338221 stderr c17304c8 ] error initializing Service Weaver: error creating internal listener: listen tcp: lookup HOSTNAME.COMPANY.com: no such host
(hostname PII redacted)
it seems like it is trying to resolve my FQDN, but should really just be using localhost or the loopback IP.
FWIW, single deploy is working fine:
hello listener available on 127.0.0.1:12345
╭───────────────────────────────────────────────────╮
│ app : hello │
│ deployment : ddde8609-00e3-4074-a41c-60f6fa3e4275 │
╰───────────────────────────────────────────────────╯
curl "localhost:12345/hello?name=Weaver"
Hello, revaeW!
when I executed weaver multi deploy weaver.toml
, it doesn't work.
config
[serviceweaver]
name = "weaver"
binary = "./weaver.exe"
temp/serviceweaver/logs/weaver-multiweaver..f1de7a14-6cfb-49fb-9f63-1fec84875a3b.error.log
,�
�weaver�
babysitter"$f1de7a14-6cfb-49fb-9f63-1fec84875a3b)��#9N? 2�error:dC:/Users/lemon/go/pkg/mod/github.com/!service!weaver/[email protected]/internal/babysitter/babysitter.go@?J�e.RunR�serviceweaver/systemR R�errRSfork/exec C:\Users\lemon\GolandProjects\weaver\weaver.exe: not supported by windows,�
�weaver�
babysitter"$f1de7a14-6cfb-49fb-9f63-1fec84875a3b)��#9N? 2�error:dC:/Users/lemon/go/pkg/mod/github.com/!service!weaver/[email protected]/internal/babysitter/babysitter.go@?J�e.RunR�serviceweaver/systemR R�errRSfork/exec C:\Users\lemon\GolandProjects\weaver\weaver.exe: not supported by windows
is weaver multiprocess does not support windows?
This issue was also partly discussed on #100
When deploying from macOS the operating system defaults to a /var/folders/*
temporary directory and this will not work with unix systems
Right now the ssh deployer pulls the temp directory via os.TempDir()
weaver/internal/tool/ssh/deploy.go
Line 143 in 35727ae
Determining the correct temp directory is not the easiest thing for example the "canonical environment variable in Unix and POSIX" TMPDIR environment variable is not always set
aran@###:~$ uname -a
Linux ### 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
aran@####:~$ echo $TMPDIR
aran@####:~$
Comparing this to macOS I get the following output
❯ echo $TMPDIR
/var/folders/sq/3sdsh9n14p790p7xwhdlvdzc0000gn/T/
~
❯
To fix this I propose introducing an additional configuration option to the ssh deployer allowing for this variable to be override from the default /tmp
which should work both on macOS and Unix systems
This new configuration option can be added to the sshConfigSchema
type as a string and defaults to /tmp
weaver/internal/tool/ssh/deploy.go
Lines 181 to 183 in 35727ae
The new tmp directory variable can then be passed into the Deployment
weaver/runtime/protos/runtime.proto
Lines 104 to 106 in 35727ae
That is used by the copyBinaries
func
weaver/internal/tool/ssh/deploy.go
Line 136 in 35727ae
The new tmp directory will be used on the following line instead of os.TempDir()
weaver/internal/tool/ssh/deploy.go
Line 143 in 35727ae
Configuration would look as follows
[serviceweaver]
name = "hello"
binary = "./hello"
[ssh]
locations_file = "./ssh_locations.txt"
tmp_dir = "/something/custom"
Of course if the tmp_dir
is omitted then it would default to /tmp
which works for both unix and posix systems
Hopefully that explains the issue and a possible solution to the issue. Let me know your thoughts on this and whether you feel like should do something differently?
Hi,
I'm trying to deploy the hello world sample and getting this error.
My environment is a clean gitpod container with gcloud freshly setup.
Here more logs:
Creating cluster serviceweaver-config in location us-central1... Done
internal error: couldn't create clientset for cluster "serviceweaver-config" in region "us-central1": The gcp auth plugin has been removed.
Please use the "gke-gcloud-auth-plugin" kubectl/client-go credential plugin instead.
See https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke for further details
exit status 1
I've tried both setting up my own test weaver application and using the collatz example and both times when running
❯ weaver ssh deploy weaver.toml
I get the following error message
unable to parse ssh config: section "ssh" has unknown keys [locations_file]
I've tried looking at the code but nothing seems to stand out to me at a quick glance as to why this is erroring. I don't fully understand how all the code works just yet but thought I'd raise the issue and continue debugging
Hi, I am pretty new to this project but found it very interesting!
I am working on building a hexagonal architecture structure in Go, and one of the thing I am doing is separating the ports (interface) from their corresponding concrete implementations adapters.
I was looking to compose my adapter with the weaver.Implements[T], where T is the port (interface, abstraction) which reside on another package.
There is this limitation now, of requiring the interface in the same package as the concrete implementation (when using the weaver CLI). For me that goes against the coupling and segregation concerns I want to enforce with this type of architecture.
I was wondering if that limitation be removed in the future.
Thanks
// numProcsIn returns the number of processes assigned to the provided group,
// using the placement specified in the provided set of components.
func numProcsIn(group *protos.ColocationGroup, components map[string]*component) int {
procs := map[string]bool{}
for _, c := range components {
if c.colocGroupName == group.Name {
procs[c.processName] = true
}
}
return len(procs)
}
In the Place function, components are divided into group,and components in the same group get there processName set to the first component's component name。So in func numProcsIn, len(procs) can never be > 1
Hi everybody,
Having an interface
makes sense when you have multiple implementations
.
I wondered, how can I have multiple implementations
for an interface/component
and choose among them?
Hi, I wanted to ask if there are plans to support Kubernetes as a deployer, I see the gke
deployer has kubernetes in the mix, but I was wondering why pure kubernetes deploy was not part of the initial release
I was hoping to see some possibility for injecting component references through the Init
method or via embedding a generic interface. At the moment, the only way for component A
to talk to component B
is through a piece of code that contains references to the both (e.g. an HTTP handler). First, A
gets called, then it's result gets passed to B
. His effectively earns that the entire gluing layer can only leave in a single process at best.
Very interesting approach, thanks for the amazing work.
I have a use case where the application is sometimes deployed:
A deployer for could run would be very welcome, any plans for this?
This issue was automatically created by Allstar.
Security Policy Violation
Project is out of compliance with Binary Artifacts policy: binaries present in source code
Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see the Security Scorecards Documentation for Binary Artifacts.
Remediation Steps
To remediate, remove the generated executable artifacts from the repository.
Artifacts Found
Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.
Allstar has been installed on all Google managed GitHub orgs. Policies are gradually being rolled out and enforced by the GOSST and OSPO teams. Learn more at http://go/allstar
This issue will auto resolve when the policy is in compliance.
Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.