ldsec/drynx

Decentralized, Secure, Verifiable System for Statistical Queries and Machine Learning on Distributed Datasets

Drynx

Drynx is a library for simulating a privacy-preserving and verifiable data sharing/querying tool. It offers a series of independent protocols that when combined offer a verifiably-secure and safe way to compute statistics and train basic machine learning models on distributed sensitive data (e.g., medical data).

The paper has all the details: Drynx: Decentralized, Secure, Verifiable System for Statistical Queries and Machine Learning on Distributed Datasets

Drynx is developed by LDS (Laboratory for Data Security in EPFL) in collaboration with DeDiS (Laboratory for Decentralized and Distributed Systems). It is build on the UnLynx library and does an intensive use of Overlay-network (ONet) library and of the Advanced Crypto (kyber) library.

Documentation

• For more information regarding the underlying architecture please refer to the stable version of ONet go.dedis.ch/onet/v3
• To check the code organisation, have a look at Layout
• For more information on how to run our protocols, services, simulations and apps, go to Running Drynx

Getting Started

To use the code of this repository you need to:

• Install Golang
• [Recommended] Install IntelliJ IDEA and the GO plugin
• Set $GOPATH to point to your workspace directory
• Add $GOPATH/bin to $PATH
• Git clone this repository to $GOPATH/src git clone https://github.com/ldsec/drynx.git or...
• go get repository: go get github.com/ldsec/drynx

Version

We only have a development version. The master-branch in github.com/ldsec/drynx is the development version that works but can have incompatible changes.

License

Drynx is licensed under a End User Software License Agreement ('EULA') for non-commercial use. If you need more information, please contact us.

Contact

You can contact any of the developers for more information or any other member of lds:

• David Froelicher (PHD student) - [email protected]
• Joao Andre Sa (Software Engineer) - [email protected]

drynx's Issues

survey with `QueryDPDataGen{GroupByValues: a}` panic for ∀a where ∃y∈a,y <= 0

Running a survey with QueryDPDataGen containing a GroupByValues field with an array of a least one element <= 0 panics with

E : websocket.go:333 (v3.wsHandler.ServeHTTP)@9515                                               - Got an error while executing drynx/SurveyQuery: panic with runtime error: index out of range [0] with length 0:
    go.dedis.ch/onet/v3.callInterfaceFunc.func1
        /home/tharvik/.go/pkg/mod/go.dedis.ch/onet/[email protected]/processor.go:413

Use go.modules and port to cothority.v3

Port Drynx and Unlynx to cothority.v3

Easy Deployment

Discuss how to do the deployment and what the security-model is for the different nodes - how they onboard and who trusts whom.

lib/SurveyQuery.ServerToDP can be malformed

as protobuf disallow map with nil values, we can easily create an unserialisable struct, by giving a nil in lib/SurveyQuery.ServerToDP. From what I saw in the code, there is no need to differentiate between nil and empty list.

Missing tests

Some test files are empty. The corresponding operations are all checked in order test but they should still have their own test. Here is the list of empty tests:

• crypto_test

• key_switching_proof_test

• service_data_provider_test

• service_skipchain

Moreover, in service_test, the output is checked by printing the results and assertions should be added instead.

Error while closing node: Can't shutdown empty ProtocolInstance

Spurious error found while running the tests on travis.

error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
error while processing msg type PTID(platform.simulInit:0c0b5b067d115c86ac2441341f51b506) from tcp://127.0.0.1:33563: sending: <nil>
Error while closing node: Can't shutdown empty ProtocolInstance:
    go.dedis.ch/onet/v3.(*TreeNodeInstance).closeDispatch
        /home/travis/gopath/pkg/mod/go.dedis.ch/onet/[email protected]/treenode.go:372
TransmitMsg failed: creating protocol:
    go.dedis.ch/onet/v3.(*Overlay).TransmitMsg
        /home/travis/gopath/pkg/mod/go.dedis.ch/onet/[email protected]/overlay.go:172
  - could not create new protocol:
    go.dedis.ch/onet/v3.(*serviceManager).newProtocol.func2
        /home/travis/gopath/pkg/mod/go.dedis.ch/onet/[email protected]/service.go:523
  - runtime error: invalid memory address or nil pointer dereference
Error starting the 'waiting' threads: sending: connection read: read tcp 127.0.0.1:54508->127.0.0.1:46554: i/o timeout
FAIL	github.com/ldsec/drynx/simul	303.075s
FAIL

Replace License with dual-licensed AGPL

The current license used is not compatible with the license from cothority/onet. It should be replaced with a dual-licensed AGPL license.

survey with `QueryDPDataGen{Min: $x, Max: $x}` panic for ∀x

Running a survey with the same value of min and max for QueryDPDataGen panic a goroutine with

panic: invalid argument to Intn

goroutine 49 [running]:
math/rand.(*Rand).Intn(0xc00009a1b0, 0x0, 0x133cee0)
	/usr/lib/go/src/math/rand/rand.go:169 +0x9c
math/rand.Intn(...)
	/usr/lib/go/src/math/rand/rand.go:329
github.com/ldsec/unlynx/data.random(0x1, 0x1, 0x1)
	/home/tharvik/.go/pkg/mod/github.com/ldsec/[email protected]/data/handle_data.go:29 +0xf9
github.com/ldsec/unlynx/data.CreateInt64Slice(0x1, 0x1, 0x1, 0x0, 0x0, 0x0)
	/home/tharvik/.go/pkg/mod/github.com/ldsec/[email protected]/data/handle_data.go:168 +0x84
github.com/ldsec/drynx/protocols.createFakeDataForOperation.func1(0xc0000223e4, 0x1, 0x1, 0x1, 0xc0003f6760, 0x1, 0x1, 0x0)
	/home/tharvik/c4dt/drynx/protocols/data_collection_protocol.go:380 +0x80
created by github.com/ldsec/drynx/protocols.createFakeDataForOperation
	/home/tharvik/c4dt/drynx/protocols/data_collection_protocol.go:378 +0x12b

client.GenerateSurveyQuery with empty (but not nil) PublishSignatureBytes panic

client.GenerateSurveyQuery(
    [...]
    []*[]drynx_lib.PublishSignatureBytes{},
    [...]

panic with

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
github.com/ldsec/drynx/services.(*API).GenerateSurveyQuery(0xc000098ec0, 0xc000098e80, 0xc000098e80, 0xc000a21170, 0xc000a211a0, 0xc00023ed10, 0xf, 0xc00023ed78, 0x8, 0x1, ...)
	/home/tharvik/c4dt/drynx/services/api.go:63 +0x401
main.surveyRun(0xc00024e2c0, 0x0, 0xc0001c7220)
	/home/tharvik/c4dt/drynx/cmd/client/survey.go:95 +0x776
github.com/urfave/cli.HandleAction(0xbb2740, 0xcebb70, 0xc00024e2c0, 0xc00024e2c0, 0x0)
	/home/tharvik/.go/pkg/mod/github.com/urfave/[email protected]/app.go:523 +0xbe
github.com/urfave/cli.Command.Run(0xcb943c, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/tharvik/.go/pkg/mod/github.com/urfave/[email protected]/command.go:174 +0x51c
github.com/urfave/cli.(*App).RunAsSubcommand(0xc00024a1c0, 0xc00024e000, 0x0, 0x0)
	/home/tharvik/.go/pkg/mod/github.com/urfave/[email protected]/app.go:404 +0x869
github.com/urfave/cli.Command.startApp(0xcbb7cd, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0xcc46d7, 0x12, 0x0, ...)
	/home/tharvik/.go/pkg/mod/github.com/urfave/[email protected]/command.go:329 +0x845
github.com/urfave/cli.Command.Run(0xcbb7cd, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0xcc46d7, 0x12, 0x0, ...)
	/home/tharvik/.go/pkg/mod/github.com/urfave/[email protected]/command.go:102 +0x9ca
github.com/urfave/cli.(*App).Run(0xc00024a000, 0xc0000b0150, 0x3, 0x3, 0x0, 0x0)
	/home/tharvik/.go/pkg/mod/github.com/urfave/[email protected]/app.go:276 +0x718
main.main()
	/home/tharvik/c4dt/drynx/cmd/client/main.go:59 +0x2ab

survey with `ServerToDP` mapping a CN to the same `ServerIdentity` as DP hang

client.GenerateSurveyQuery(
    [...]
    map[string]*[]onet_network.ServerIdentity{ // map CN to DPs
        roster.List[0].String(): &[]onet_network.ServerIdentity{*roster.List[0]}}
    [...]

make the system wait for two (because by default CN is added as DP, so two the same) messages from roster.List[0] but there is only one answer, thus hanging.

trash

survey without connnected client in `ServerToDP` panic

client := drynx_services.NewDrynxClient(roster.List[0], os.Args[0])
client.GenerateSurveyQuery(
    [...]
    map[string]*[]onet_network.ServerIdentity{ // map CN to DPs
        roster.List[1].String(): &[]onet_network.ServerIdentity{*roster.List[0]}},
    [...]

gives

E : websocket.go:333 (v3.wsHandler.ServeHTTP)@9082                                               - Got an error while executing drynx/SurveyQuery: panic with runtime error: index out of range [0] with length 0:
    go.dedis.ch/onet/v3.callInterfaceFunc.func1
        /home/tharvik/.go/pkg/mod/go.dedis.ch/onet/[email protected]/processor.go:413

coveralls badge links to 404

27455b9 sed'ed lca1 to ldsec, travis and goreport seems to not have issues with it, coveralls does: it can't find the repo.

Use ByzCoin for logging

Replace the skipchain with a byzcoin, so that the logging can be done on a more used basis.