e7dal/pygrid

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science

PyGrid is a peer-to-peer network of data owners and data scientists who can collectively train AI models using PySyft.

Overview

• Overview
• How to install
  • PyGrid library
  • Common Installation Issues:
• Getting started
  • Build Grid Platform Locally
  • Build images
  • Let's put all together
• Try out the Tutorials
• Start Contributing
• High-level Architecture
• Disclaimer
• License

How to install

PyGrid library

Make PyGrid library importable in python:

$ pip install .

Common Installation Issues:

• On macOS, you might get ld: library not found for -lssl. It happens when openssl is missing. Kindly install openSSL, add it to you path and try again. You can also fix the issue by installing the appropriate Xcode Command line tools from here.

Getting started

To boot the entire PyGrid platform locally, we will use docker containers. To install docker the dependencies, just follow docker documentation.

Start Grid platform locally

It will download the latest openmined's docker images and start a grid platform with 1 gateway and 4 grid nodes. PS: Feel free to increase/decrease the number of initial PyGrid nodes (you can do this by changing the docker-compose.yml file).

$ docker-compose up

On MacOS, you have to work-around the lack of support for network_mode: host in Docker for Mac by adding following to your /etc/hosts

127.0.0.1 host.docker.internal

and running:

$ docker-compose -f docker-compose-mac.yml up

Kubernetes deployment.

You can now deploy the grid-gateway and grid-node docker containers on kubernetes. This can be either to a local (minikube) cluster or a remote cluster (GKE, EKS, AKS etc). The steps to setup the cluster can be found in ./k8s/Readme.md

Build your own images

$ docker build -t openmined/grid-node ./app/websocket/  # Build PyGrid node image
$ docker build -t openmined/grid-gateway ./gateway/  # Build gateway image

Try out the Tutorials

A comprehensive list of tutorials can be found here. A list with latest experimental tutorials can be found in the dev branch here.

These tutorials cover how to create a PyGrid node and what operations you can perform.

Start Contributing

The guide for contributors can be found here. It covers all that you need to know to start contributing code to PySyft in an easy way.

Also join the rapidly growing community of 2500+ on Slack. The slack community is very friendly and great about quickly answering questions about the use and development of PyGrid/PySyft!

We also have a Github Project page for PySyft and PyGrid here.

High-level Architecture

Disclaimer

Do NOT use this code to protect data (private or otherwise) - at present it is very insecure.

License

Apache License 2.0