WARNING: Project is still in alpha, backwards incompatible changes will be made.
A full-stack implementation of secure-scuttlebutt using the Go programming language.
If you encounter a bug, please refer to our public issue tracker.
See the FAQ for more. See this project for current focus.
Want to contribute patches to go-ssb? Read the developer documentation hosted at dev.scuttlebutt.nz. If you have a large change you want to make, reach out first and we'll together make sure that the resulting PR will be accepted 🖤
- Follow-graph implementation (based on gonum) to authorize incoming connections
- Blobs store and replication
- Publishing new messages to the log
- Legacy feed replication
- Epidemic Broadcast Trees (EBT) feed replication in beta (use
go-sbot -enable-ebt) - Invite mechanics (peer-invites partially done, too. See Issue 45 for more.)
You can install the project using Golang's install command which will place the commands into the directory pointed to by the GOBIN environment variable.
git clone https://github.com/ssbc/go-ssb
cd go-ssb
go install ./cmd/go-sbot
go install ./cmd/sbotcliRequirements:
- Golang version 1.17 or higher
The tool in cmd/go-sbot is similar to ssb-server (previously called scuttlebot or sbot for short).
See the quick start document for a walkthrough and getting started tour. You may also be interested in the ways you can configure a running go-sbot, see the configuration guide.
If you have an existing feed with published contact messages, you can just resync it from another go or js server. To get this going you copy the key-pair ($HOME/.ssb/secret by default) to $HOME/.ssb-go/secret, start the program and connect to the server (using the multiserver address format).
mkdir $HOME/.ssb-go
cp $HOME/.ssb/secret $HOME/.ssb-go
go-sbot &
sbotcli connect "net:some.ho.st:8008~shs:SomeActuallyValidPubKey="This currently constructs legacy SSB messages, that still have the signature inside the signed value:
{
"key": "%EMr6LTquV6Y8qkSaQ96ncL6oymbx4IddLdQKVGqYgGI=.sha256",
"value": {
"previous": "%rkJMoEspdU75c1RpGbwjEH7eZxM/PJPFubpZTtynhsg=.sha256",
"author": "@iL6NzQoOLFP18pCpprkbY80DMtiG4JFFtVSVUaoGsOQ=.ed25519",
"sequence": 793,
"timestamp": 1457694632215,
"hash": "sha256",
"content": {
"type": "post",
"text": "@dust \n> this feels like the cultural opposite of self-dogfooding, and naturally, leaves a bad taste in my mouth \n \n\"This\" meaning this thread? Or something in particular in this thread? And if this thread or something in it, how so? I don't want to leave a bad taste in your mouth.",
"root": "%I3yWHMF2kqC7fLZrC8FB+Kuu/6MQZIKzJGIjR3fVv9g=.sha256",
"branch": "%cNJgO+1R4ci/jgTup4LLACoaKZRtYtsO7BzRCDJh6Gg=.sha256",
"mentions": [
{
"link": "@/02iw6SFEPIHl8nMkYSwcCgRWxiG6VP547Wcp1NW8Bo=.ed25519",
"name": "dust"
}
],
"channel": "patchwork-dev"
},
"signature": "bbjj+zyNubLNEV+hhUf6Of4KYOlQBavQnvdW9rF2nKqTHQTBiFBnRehfveCft3OGSIIr4VgD4ePICCTlBuTdAg==.sig.ed25519"
},
"timestamp": 1550074432723.0059
}The problem with this (for Go and others) is removing the signature field from value without changing any of the values or field ordering of the object, which is required to compute the exact same bytes that were used for creating the signature. Signing JSON was a bad idea. There is also other problems around this (like producing the same byte/string encoding for floats that v8 produces) and a new, canonical format is badly needed.
What you are free to input is the content object, the rest is filled in for you. The author is determined by the keypair used by go-sbot. Multiple identities are supported through the API.
go-sbot also exposes the same async publish method that ssb-server has. So you can also use it with ssb-client!
To do this programatically in go, you construct a margaret.Log using multilogs.OpenPublishLog (godoc) that publishes the content portion you Append() to it the feed of the keypair.
Example:
package main
import (
"log"
"fmt"
"github.com/ssbc/go-ssb"
"github.com/ssbc/go-ssb/multilogs"
"github.com/ssbc/go-ssb/sbot"
)
func main() {
sbot, err := sbot.New()
check(err)
publish, err := multilogs.OpenPublishLog(sbot.ReceiveLog, sbot.UserFeeds, *sbot.KeyPair)
check(err)
alice, err := refs.ParseFeedRef("@alicesKeyInActualBase64Bytes.ed25519")
check(err)
var someMsgs = []interface{}{
map[string]interface{}{
"type": "about",
"about": sbot.KeyPair.ID().Ref(),
"name": "my user",
},
map[string]interface{}{
"type": "contact",
"contact": alice.Ref(),
"following": true,
},
map[string]interface{}{
"type": "post",
"text": `# hello world!`,
},
map[string]interface{}{
"type": "about",
"about": alice.Ref(),
"name": "test alice",
},
}
for i, msg := range someMsgs {
newSeq, err := publish.Append(msg)
check(fmt.Errorf("failed to publish test message %d: %w", i, err))
log.Println("new message:", newSeq)
}
err = sbot.Close()
check(err)
}
func check(err error) {
if err != nil {
log.Fatal(err)
}
}Has some commands to publish frequently used messages like post, vote and contact:
sbotcli publish contact --following '@p13zSAiOpguI9nsawkGijsnMfWmFd5rlUNpzekEE+vI=.ed25519'
sbotcli publish contact --blocking '@p13zSAiOpguI9nsawkGijsnMfWmFd5rlUNpzekEE+vI=.ed25519'
sbotcli publish about --name "cryptix" '@p13zSAiOpguI9nsawkGijsnMfWmFd5rlUNpzekEE+vI=.ed25519'They all support passing multiple --recps flags to publish private messages as well:
sbotcli publish post --recps "@key1" --recps "@key2" "what's up?"For more dynamic use, you can also just pipe JSON into stdin:
cat some.json | sbotcli publish rawThere are two binary executable in this project that are useful right now, both located in the cmd folder. go-sbot is the database server, handling incoming connections and supplying replication to other peers. sbotcli is a command line interface to query feeds and instruct actions like connect to X. This also works against the JS implementation.
If you just want to build the server and play without contributing to the code (and are using a recent go version > 1.17), you can do this:
# clone the repo
git clone https://github.com/ssbc/go-ssb
# go into the servers folder
cd go-ssb/cmd/go-sbot
# build the binary (also fetches pinned dependencies)
go build -v -i
# test the executable works by printing it's help listing
./go-sbot -h
# (optional) install it somwhere on your $PATH
sudo cp go-sbot /usr/local/binIf you want to hack on the other dependencies of the stack, you can use the replace statement.
E.g. for hacking on a locally cloned copy of go-muxrpc, you'd do:
diff --git a/go.mod b/go.mod
index c4d475f..51c2757 100644
--- a/go.mod
+++ b/go.mod
@@ -6,6 +6,8 @@ module github.com/ssbc/go-ssb
+replace github.com/ssbc/go-muxrpc/v2 => ./../go-muxrpc
+Which points the new build of go-sbot/sbotcli to use your local copy of go-muxrpc.
Once you have configured your environment set up to build the binaries, you can also run the tests. We have unit tests for most of the modules, most importantly message, blobstore and the replication plugins (gossip and blobs). There are also interoperability tests with the nodejs implementation (this requires recent versions of node and npm).
$ go test -v ./message
2019/01/08 12:21:55 loaded 236 messages from testdata.zip
=== RUN TestPreserveOrder
--- PASS: TestPreserveOrder (0.00s)
=== RUN TestComparePreserve
--- PASS: TestComparePreserve (0.02s)
=== RUN TestExtractSignature
--- PASS: TestExtractSignature (0.00s)
=== RUN TestStripSignature
--- PASS: TestStripSignature (0.00s)
=== RUN TestUnicodeFind
--- PASS: TestUnicodeFind (0.00s)
=== RUN TestInternalV8String
--- PASS: TestInternalV8String (0.00s)
=== RUN TestSignatureVerify
--- PASS: TestSignatureVerify (0.06s)
=== RUN TestVerify
--- PASS: TestVerify (0.06s)
=== RUN TestVerifyBugs
--- PASS: TestVerifyBugs (0.00s)
PASS
ok github.com/ssbc/go-ssb/message 0.180sIf you encounter a feed that can't be validated with our code, there is a encode_test.js script to create the testdata.zip from a local sbot. Call it like this cd message && node encode_test.js @feedPubKey.ed25519 and re-run go test.
$ go test ./plugins/...
ok github.com/ssbc/go-ssb/plugins/blobs 0.021s
? github.com/ssbc/go-ssb/plugins/control [no test files]
ok github.com/ssbc/go-ssb/plugins/gossip 0.667s
? github.com/ssbc/go-ssb/plugins/test [no test files]
? github.com/ssbc/go-ssb/plugins/whoami [no test files](Sometimes the gossip test blocks indefinitely. This is a bug in go-muxrpcs closing behavior. See the FAQ for more information.)
To run the interop tests you need to install the dependencies first and then run the tests. Diagnosing a failure might require adding the -v flag to get the stderr output from the nodejs process.
$ cd message/legacy && npm ci
$ go test -v
$ cd -
$ cd tests && npm ci
$ go test -vThe tests run under this Github Actions configuration.
We cache both the global ~/.npm and **/node_modules to reduce build times.
This seems like a reasonable approach because we're testing against a single
Node.js version and a locked package set. Go dependencies are also cached.
- secret-handshake key exchange using secretstream
- JS interoparability by using go-muxprc
- Embedded datastore, no external database required (margaret abstraction with BadgerDB backend, similar to flumedb)
- pull-stream-like abstraction (called luigi) to pipe between rpc and database.
- Post to the
#go-ssb/#go-ssb-devchannels on ssb - Raise an issue
- Or mention us individually on ssb
- cryptix:
@p13zSAiOpguI9nsawkGijsnMfWmFd5rlUNpzekEE+vI=.ed25519 - keks:
@YXkE3TikkY4GFMX3lzXUllRkNTbj5E+604AkaO1xbz8=.ed25519 - decentral1se
@i8OXtTYaK0PrF002pd4vpXmrlg98As7ZMaHGKoXixdM=.ed25519
- cryptix:
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent