hyperledger/fabric-private-chaincode

FPC enables Confidential Chaincode Execution for Hyperledger Fabric using Intel SGX.

Hyperledger Fabric Private Chaincode

Hyperledger Fabric Private Chaincode (FPC) enables the execution of chaincodes using Intel SGX for Hyperledger Fabric.

The transparency and resilience gained from blockchain protocols ensure the integrity of blockchain applications and yet contradicts the goal to keep application state confidential and to maintain privacy for its users.

To remedy this problem, this project uses Trusted Execution Environments (TEEs), in particular Intel Software Guard Extensions (SGX), to protect the privacy of chaincode data and computation from potentially untrusted peers.

Intel SGX is the most prominent TEE today and available with commodity CPUs. It establishes trusted execution contexts called enclaves on a CPU, which isolate data and programs from the host operating system in hardware and ensure that outputs are correct.

This project provides a framework to develop and execute Fabric chaincode within an enclave. It allows to write chaincode applications where the data is encrypted on the ledger and can only be accessed in clear by authorized parties. Furthermore, Fabric extensions for chaincode enclave registration and transaction verification are provided.

Fabric Private Chaicode is based on the work in the paper:

• Marcus Brandenburger, Christian Cachin, Rüdiger Kapitza, Alessandro Sorniotti: Blockchain and Trusted Computing: Problems, Pitfalls, and a Solution for Hyperledger Fabric. https://arxiv.org/abs/1805.08541

This project was accepted via a Hyperledger Fabric RFC and is now under development. We provide an initial proof-of-concept implementation of the proposed architecture. Note that the code provided in this repository is still prototype code and not yet meant for production use!

For up to date information about our community meeting schedule, past presentations, and info on how to contact us please refer to our wiki page.

Architecture and components

Overview

This project extends a Fabric peer with the following components: A chaincode enclave that executes a particular chaincode, running inside SGX. In the untrusted part of the peer, an enclave registry maintains the identities of all chaincode enclaves and an enclave transaction validator that is responsible for validating transactions executed by a chaincode enclave before committing them to the ledger.

The following diagram shows the architecture:

The system consists of the following components:

1. Chaincode enclave: The chaincode enclave executes one particular chaincode, and thereby isolates it from the peer and from other chaincodes. A chaincode library acts as intermediary between the chaincode in the enclave and the peer. The chaincode enclave exposes the Fabric chaincode interface and extends it with additional support for state encryption, attestation, and secure blockchain state access. This code is executed inside an Intel SGX enclave.

2. Enclave Endorsement validation: The enclave endorsement validation complements the peer’s validation system and is responsible for validating transactions produced by a chaincode enclave. In particular, the validator checks that a transaction contains a valid signature issued by a registered chaincode enclave. Iff the validation is successful, it causes the state-updates of the transaction to be committed to the ledger. This code is a normal Fabric transaction, i.e., executed and endorsed on multiple peers as required by the organization trust.

3. FPC Chaincode Pkg: This component bundles together the chaincode enclave and the enclave endorsement validation logic into a fabric chaincode. It also includes a shim component which (a) proxies the chaincode enclave shim functionality, e.g., access to ledger, to the fabric peer, and (b) dispatches FPC flows to either the chaincode enclave (via __invoke queries) or to the enclave endorsement validation logic (via __endorse transactions).

4. Enclave registry: The enclave registry (ercc) is a chaincode that runs outside SGX and maintains a list of all existing chaincode enclaves in the network. It performs attestation with the chaincode enclave and stores the attestation result on the blockchain. The attestation demonstrates that a specific chaincode executes in an actual enclave. This enables the peers and the clients to inspect the attestation of a chaincode enclave before invoking chaincode operations or committing state changes.

Design

More detailed architectural information and overview of the protocols can be found in the Fabric Private Chaincode RFC.

The full detailed operation of FPC is documented in a series of UML Sequence Diagrams. Note that FPC version 1.x corresponds to FPC Lite in documents and code.

Specifically:

• The fpc-lifecycle-v2(puml) diagram describes the normal lifecycle of a chaincode in FPC, focusing in particular on those elements that change in FPC vs. regular Fabric.
• The fpc-registration(puml) diagram describes how an FPC Chaincode Enclave is created on a Peer and registered in the FPC Registry, including the Remote Attestation process.
• The fpc-key-dist(puml) diagram describes the process by which chaincode-unique cryptographic keys are created and distributed among enclaves running identical chaincodes. Note that in the current version of FPC, key generation is performed, but the key distribution protocol has not yet been implemented.
• The fpc-cc-invocation(puml) diagram illustrates the invocation process at the beginning of the chaincode lifecycle in detail, focusing on the cryptographic operations between the Client and Peer leading up to submission of a transaction for Ordering.
• The fpc-cc-execution(puml) diagram provides further detail of the execution phase of an FPC chaincode, focusing in particular on the getState and putState interactions with the Ledger.
• The fpc-validation(puml) diagram describes the FPC-specific process of validation.
• The fpc-components(puml) diagram shows the important data structures of FPC components and messages exchanged between components.
• The detailed message definitions can be found as protobufs.
• The interfaces document defines the interfaces exposed by the FPC components and their internal state.

Additional Google documents provide details on FPC 1.0:

• The FPC for Health use case describes how FPC 1.0 enables a health care use case. The document also gives more details on the FPC 1.0-enabled application domains and related constraints. Lastly, it provides a security analysis why these constraints are sufficient for security.
• The FPC externalized endorsement validation describes the FPC 1.0 enclave endorsement validation mechanism.

Source organization

• client_sdk: The FPC Go Client SDK
• cmake: CMake build rules shared across the project
• common: Shared C/C++ code
• config: SGX configuration
• docs: Documentation and design documents
• ecc_enclave: C/C++ code for chaincode enclave (including the trusted code running inside an enclave)
• ecc: Go code for FPC chaincode package, including dispatcher and (high-level code for) enclave endorsement validation.
• ecc_go: Go code for FPC Go Chaincode Support
• ercc: Go code for Enclave Registry Chaincode
• samples: FPC Samples
• fabric: FPC wrapper for Fabric peer and utilities to start and stop a simple Fabric test network with FPC enabled, used by integration tests.
• integration: FPC integration tests.
• internal: Shared Go code
• protos: Protobuf definitions
• scripts: Scripts used in build process.
• utils/docker: Docker images and their build process.
• utils/fabric: Various Fabric helpers.

Releases

For all releases go to the Github Release Page.

WARNING: This project is in continous development and the main branch will not always be stable. Unless you want to actively contribute to the project itself, we advise you to use the latest release.

Getting started

The following steps guide you through the build phase and configuration, for deploying and running an example private chaincode.

We assume that you are familiar with Hyperledger Fabric; otherwise we recommend the Fabric documentation as your starting point. Moreover, we assume that you are familiar with the Intel SGX SDK.

This README is structure as follows. We start by cloning the FPC repository and explain how to prepare your development environment for FPC in Setup your FPC Development Environment. In Build Fabric Private Chaincode we guide you through the building process and elaborate on common issues. Finally, we give you a starting point for Developing with Fabric Private Chaincode by introducing the FPC Hello World Tutorial.

Clone Fabric Private Chaincode

Clone the code and make sure it is on your $GOPATH. (Important: we assume in this documentation and default configuration that your $GOPATH has a single root-directoy!) We use $FPC_PATH to refer to the Fabric Private Chaincode repository in your filesystem.

export FPC_PATH=$GOPATH/src/github.com/hyperledger/fabric-private-chaincode
git clone --recursive https://github.com/hyperledger/fabric-private-chaincode.git $FPC_PATH

Setup your Development Environment

There are two different ways to develop Fabric Private Chaincode. Using our preconfigured Docker container development environment or setting up your local system with all required software dependencies to build and develop chaincode locally.

Option 1: Using the Docker-based FPC Development Environment

In this section we explain how to set up a Docker-based development environment that allows you to develop and test FPC chaincode. The docker images come with all necessary software dependencies and allow you a quick start. We recommend to set privileges to manage docker as a non-root user. See the official docker documentation for more details.

First make sure your host has

• Docker v23.0 (or higher). It also should use /var/run/docker.sock as socket to interact with the daemon (or you will have to override in $FPC_PATH/config.override.mk the default definition in make of DOCKER_DAEMON_SOCKET)
• GNU make

Once you have cloned the repository, you can either use the pre-built images or you can manually build them. After that you will start the development container.

Pull docker images

To pull the docker image execute the following:

make -C $FPC_PATH/utils/docker pull pull-dev 

Manually build docker images

In order to build the development image manually you can use the following commands. Note that this process may take some time.

make -C $FPC_PATH/utils/docker build build-dev 

Start the dev container

Next we will open a shell inside the FPC development container, with environment variables like $FPC_PATH appropriately defined and all dependencies like the Intel SGX SDK, ready to build and run FPC. Continue with the following command:

make -C $FPC_PATH/utils/docker run-dev

Note that by default the dev container mounts your local cloned FPC project as a volume to /project/src/github.com/hyperledger/fabric-private-chaincode within the docker container. This allows you to edit the content of the repository using your favorite editor in your system and the changes inside the docker container. Additionally, you are also not loosing changes inside the container when you reboot or the container gets stopped for other reasons.

A few more notes:

• We use Ubuntu 20.04 by default. To build also docker images based on Ubuntu 18.04, add the following to $FPC_PATH/config.override.mk.

  DOCKER_BUILD_OPTS=--build-arg UBUNTU_VERSION=18.04 --build-arg UBUNTU_NAME=bionic

• If you run behind a proxy, you will have to configure the proxy, e.g., for docker (~/.docker/config.json) and load the configuration inside the dev container by setting DOCKER_DEV_RUN_OPTS += -v "$HOME/.docker":"/root/.docker" in $FPC_PATH/config.override.mk. See Working from behind a proxy below for more information. Also note that with newer docker versions (i.e., docker desktop), the docker socket is located on the host in ~/.docker/. This may cause issues when using docker inside the FPC dev container as the docker client is not able to access the docker socket at the path of the host system. You may try to switch the docker context to use /var/run/docker.sock. We do not recommend this approach and happy for suggestions.
• If your local host is SGX enabled, i.e., there is a device /dev/sgx/enclave or /dev/isgx and your PSW daemon listens to /var/run/aesmd, then the docker image will be sgx-enabled and your settings from ./config/ias will be used. You will have to manually set SGX_MODE=HW before building anything to use HW mode.
• If you want additional apt packages to be automatically added to your container images, you can do so by modifying $FPC_PATH/config.override.mk file in the fabric-private-chaincode directory. In that file, define DOCKER_BASE_RT_IMAGE_APT_ADD_PKGS, DOCKER_BASE_DEV_IMAGE_APT_ADD_PKGS'and/or DOCKER_DEV_IMAGE_APT_ADD_PKGS with a list of packages you want to be added to you all images, all images where fabric/fpc is built from source and the dev(eloper) container, respectively. They will then be automatically added to the docker image.
• Due to the way the peer's port for chaincode connection is managed, you will be able to run only a single FPC development container on a particular host.
• For support for Apple Mac (M1 or newer) see the Troubleshooting section.

Now you are ready to start development within the container. Continue with building FPC as described in the Build Fabric Private Chaincode Section and then write your first Private Chaincode.

Option 2: Setting up your system to do local development

As an alternative to the Docker-based FPC development environment you can install and manage all necessary software dependencies which are required to compile and run FPC.

Requirements

Make sure that you have the following required dependencies installed:

• Linux (OS) (we recommend Ubuntu 20.04, see list supported OS)

• CMake v3.5.1 or higher

• Go 1.21.x or higher

• Docker 18.09 (or higher) and docker-compose 1.25.x (or higher) Note that version from Ubuntu 18.04 is not recent enough! To upgrade, install a recent version following the instructions from docker.com, e.g., for version 1.25.4 execute

  sudo curl -L "https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose	
  sudo chmod +x /usr/local/bin/docker-compose

  To install docker-componse 1.25.4 from docker.com, execute

  sudo curl -L "https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
  sudo chmod +x /usr/local/bin/docker-compose

• yq v3.x (newer versions, v4.x and higher, are currently not supported!) You can install yq v3 via go get.

    GO111MODULE=on go get github.com/mikefarah/yq/v4

• Protocol Buffers

  • Protocol Buffers 3.0.x needed for the Intel SGX SDK
  • Protocol Buffers 3.11.x or higher and Nanopb 0.4.7

• SGX PSW & SDK v2.12 for Linux (alternatively, you could also install it from the source

• Credentials for Intel Attestation Service, read here (for hardware-mode SGX)

• Intel Software Guard Extensions SSL (we recommend using branch lin_2.10_1.1.1g OpenSSL 1.1.1g)

• Hyperledger Fabric v2.5.4

• Clang-format 6.x or higher

• jq

• hex (for Ubuntu, found in package basez)

• A recent version of PlantUML, including Graphviz, for building documentation. See Documentation for our recommendations on installing. The version available in common package repositories may be out of date.

Intel SGX SDK and SSL

Fabric Private Chaincode requires the Intel SGX SDK and SGX SSL to build the main components of our framework and to develop and build your first private chaincode.

Install the Intel SGX software stack for Linux by following the official documentation. Please make sure that you use the SDK version as denoted above in the list of requirements.

For SGX SSL, just follow the instructions on the corresponding github page. In case you are building for simulation mode only and do not have HW support, you might also want to make sure that simulation mode is set when building and installing it.

Once you have installed the SGX SDK and SSL for SGX SDK please double check that SGX_SDK and SGX_SSL variables are set correctly in your environment.

Protocol Buffers

We use nanopb, a lightweight implementation of Protocol Buffers, inside the enclaves to parse blocks of transactions. Install nanopb by following the instruction below. For this you need a working Google Protocol Buffers compiler with python bindings (e.g. via apt-get install protobuf-compiler python-protobuf libprotobuf-dev). For more detailed information consult the official nanopb documentation http://github.com/nanopb/nanopb.

export NANOPB_PATH=/path-to/install/nanopb/
git clone https://github.com/nanopb/nanopb.git $NANOPB_PATH
cd $NANOPB_PATH
git checkout nanopb-0.4.7
cd generator/proto && make

Make sure that you set $NANOPB_PATH as it is needed to build Fabric Private Chaincode.

Moreover, in order to build Fabric protobufs we also require a newer Protobuf compiler than what is provided as standard Ubuntu package and is used to build the Intel SGX SDK. For this reason you will have to download and install another version and use it together with Nanopb. Do not install the new protobuf, though, such that it is not found in your standard PATH but instead define the PROTOC_CMD, either as environment variable or via config.override.mk to point to the new protoc binary

wget https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip
unzip protoc-3.11.4-linux-x86_64.zip -d /usr/local/proto3
export PROTOC_CMD=/usr/local/proto3/bin/protoc

Hyperledger Fabric

Our project fetches the latest supported Fabric binaries during the build process automatically. However, if you want to use your own Fabric binaries, please checkout Fabric 2.5.4 release using the following commands:

export FABRIC_PATH=$GOPATH/src/github.com/hyperledger/fabric
git clone https://github.com/hyperledger/fabric.git $FABRIC_PATH
cd $FABRIC_PATH; git checkout tags/v2.5.4

Note that Fabric Private Chaincode may not work with the Fabric main branch. Therefore, make sure you use the Fabric v2.5.4 tag. Make sure the source of Fabric is in your $GOPATH.

Build Fabric Private Chaincode

Once you have your development environment up and running (i.e., using our docker-based setup or install all dependencies on your machine) you can build FPC and start developing your own FPC application. Note by default we build FPC with SGX simulation mode. For SGX hardware-mode support please also read the Intel SGX Attestation Support Section below.

To build all required FPC components and run the integration tests run the following:

cd $FPC_PATH
make docker
make

Besides the default target, there are also following make targets:

• build: build all FPC build artifacts
• docker: build docker images
• test: run unit and integration tests
• clean: remove most build artifacts (but no docker images)
• clobber: remove all build artifacts including built docker images
• checks: do license and linting checks on source

Also note that the file config.mk contains various defaults which can all be redefined in an optional file config.override.mk.

See also below on how to build the documentation.

Intel SGX Attestation Support

To run Fabric Private Chaincode in hardware mode (secure mode), you need an SGX-enabled hardware as well corresponding OS support. However, even if you don't have SGX hardware available, you still can run FPC in simulation mode by setting SGX_MODE=SIM in your environment.

Note that the simulation mode is for developing purpose only and does not provide any security guarantees.

As mentioned before, by default the project builds in SGX simulation mode, SGX_MODE=SIM as defined in $FPC_PATH/config.mk and you can explicitly opt for building in hardware-mode SGX, SGX_MODE=HW. In order to set non-default values for install location, or for building in hardware-mode SGX, you can create the file $FPC_PATH/config.override.mk and override the default values by defining the corresponding environment variable.

Note that you can always come back here when you want a setup with SGX hardware-mode later after having tested with simulation mode.

Register with Intel Attestation Service (IAS)

If you run SGX in simulation mode only, you can skip this section. We currently support EPID-based attestation and use the Intel's Attestation Service to perform attestation with chaincode enclaves.

What you need:

• a Service Provider ID (SPID)
• the (primary) api-key associated with your SPID

In order to use Intel's Attestation Service (IAS), you need to register with Intel. On the IAS EPID registration page you can find more details on how to register and obtain your SPID plus corresponding api-key. We currently support both linkable and unlinkable signatures for the attestation.

Place your ias api key and your SPID in the ias folder as follows:

echo 'YOUR_API_KEY' > $FPC_PATH/config/ias/api_key.txt
echo 'YOUR_SPID_TYPE' > $FPC_PATH/config/ias/spid_type.txt
echo 'YOUR_SPID' > $FPC_PATH/config/ias/spid.txt

where YOUR_SPID_TYPE must be epid-linkable or epid-unlinkable, depending on the type of your subscription.

FPC Playground for non-SGX environments

FPC leverages Intel SGX as the Confidential Computing technology to guard Fabric chaincodes. Even though the Intel SGX SDK supports a simulation mode, where you can run applications in a simulated enclave, it still requires an x86-based platform to run and compile the enclave code. Another limitation comes from the fact that the Intel SGX SDK is only available for Linux and Windows.

To overcome these limitations and allow developers to toy around with the FPC API, we provide two ways to getting started with FPC.

1. Using the Docker-based FPC Development Environment (works well on x86-based platforms on Linux and Mac).
2. FPC builds without SGX SDK dependencies (targets x86/arm-based platforms on Linux and Mac).

We now elaborate on how to build the FPC components without the SGX SDK. Note that this is indented for developing purpose only and does not provide any protection at all.

In your config.override.mk set the following to variables:

FPC_CCENV_IMAGE=ubuntu:20.04
ERCC_GOTAGS=

This configuration sets a standard Ubuntu image as alternative to our fabric-private-chaincode-ccenv image and overrides the default build tags we use to build ercc.

Next you can build ercc using the following command:

GOOS=linux make -C $FPC_PATH/ercc build docker

For building a chaincode, for instance $FPC_PATH/samples/chaincode/kv-test-go, just run:

GOOS=linux make -C $FPC_PATH/samples/chaincode/kv-test-go with_go docker

You can test your FPC chaincode easily with one of the sample deployments tutorials. We recommend to start with the-simple-testing-network.

Notes:

• On Mac use a recent version of bash (brew install bash).
• TODO more to come

Troubleshooting

This section elaborate on common issues with building Fabric Private Chaincode.

Docker

Building the project requires docker. We do not recommend to run sudo make to resolve issues with mis-configured docker environments as this also changes your $GOPATH. Please see hints on docker installation above.

The makefiles do not ensure that docker files are always rebuild to match the latest version of the code in the repo. If you suspect you have an issue with outdated docker images, you can run make clobber build which forces a rebuild. It also ensures that all other download, build or test artifacts are scrubbed from your repo and might help overcoming other problems. Be advised that that the rebuild can take a fair amount of time.

Working from behind a proxy

The current code should work behind a proxy assuming

• you have defined the corresponding environment variables (i.e., http_proxy, https_proxy and, potentially, no_proxy) properly, and
• docker (daemon & client) is properly set up for proxies as outlined in the Docker documentation for clients and the daemon.
• the docker version is correct. Otherwise you may run into problems with DNS resolution inside the container.
• the docker-compose version is correct. For example, the docker-compose from Ubuntu 18.04 (docker-compose 1.17) is not recent enough to understand ~/.docker/config.json and related proxy options.

Furthermore, for docker-compose networks to work properly with proxies, the noProxy variable in your ~/.docker/config.json should at least contain 127.0.0.1,127.0.1.1,localhost,.org1.example.com,.example.com.

Another problem you might encounter when running the integration tests insofar that some '0.0.0.0' in integration/config/core.yaml used by clients -- e.g., the peer CLI using the address: 0.0.0.0:7051 config as part of the peer section -- result in the client being unable to find the server. The likely error you will see is err: rpc error: code = Unavailable desc = transport is closing. In that case, you will have to replace the '0.0.0.0' with a concrete ip address such as '127.0.0.1'.

Environment settings

Our build system requires a few variables to be set in your environment. Missing variables may cause make to fail. Below you find a summary of all variables which you should carefully check and add to your environment.

# Path to your SGX SDK and SGX SSL
export SGX_SDK=/opt/intel/sgxsdk
export SGX_SSL=/opt/intel/sgxssl

# Path to nanopb
export NANOPB_PATH=$HOME/nanopb

# SGX simulation mode
export SGX_MODE=SIM

# SGX simulation mode
export SGX_MODE=HW

The file config.mk contains various defaults for some of these, but all can be (re)defined also in an optional file config.override.mk.

Clang-format

Some users may experience problems with clang-format. In particular, the error command not found: clang-format appears even after installing it via apt-get install clang-format. See here for how to fix this.

ERCC setup failures

If, e.g., running the integration tests executed when you run make, you get errors of following form:

Error: endorsement failure during invoke. response: status:500 message:"Setup failed: Can not register enclave at ercc: Error while retrieving attestation report: IAS returned error: Code 401 Access Denied"

In case you run in SGX HW mode, check that your files in config/ias are set properly as explained in Section Intel Attestation Service (IAS). Note that if you run initially in simulation mode and these files do not exist, the build will create dummy files. In case you switch later to HW mode without configuring these files correctly for HW mode, this will result in above error.

no Raft leader

The following error message sometimes appears when running the integration tests in the $FPC_PATH/integration folder. The output contains the following:

got unexpected status: SERVICE_UNAVAILABLE -- no Raft leader

Rerunning the tests usually works. If this error appers during the make step of building FPC than uncommenting some integration tests fixes the issue.

Working with the FPC dev container

To make starting and stopping the dev container more reliable it is advised to use the following commands:

• Start the container and get a shell: make -C $FPC_PATH/utils/docker run-dev
• Get another shell inside the dev container: docker exec -it fpc-development-main /bin/bash
• Stop the container: docker stop fpc-development-main

Development on Apple Mac (M1 or newer)

For developers using Apple Mac (M1 or newer) we suggest to use the prebuilt FPC dev container. Add the following configuration to your config.override.mk, pull the docker images and start the FPC dev container as described above in Option 1: Using the Docker-based FPC Development Environment. Note that SGX is not supported on Apple platforms, and hence, FPC chaincode can only be used in simulation mode. Alternatively, a cloud-based development environment can be used with SGX HW support, see our tutorial How to use FPC with Azure Confidential Computing.

DOCKER_BUILD_CMD=buildx build
DOCKER_BUILD_OPTS=--platform linux/amd64
DOCKER_DEV_RUN_OPTS=--platform linux/amd64

Building Documentation

To build documentation (e.g., images from the PlantUML .puml files), you will have to install java and download plantuml.jar. Either put plantuml.jar into in your CLASSPATH environment variable or override PLANTUML_JAR or PLANTUML_CMD in config.override.mk (see config.mk for default definition of the two variables). Additionally, you will need the dot program from the graphviz package (e.g., via apt-get install graphviz on Ubuntu).

By running the following command you can generate the documentation.

cd docs
make

Developing with Fabric Private Chaincode

In the samples folder you find a few examples how to develop applications using FPC and run them on a Fabric network. In particular, samples/application contains examples of the FPC Client SDK for Go. In samples/chaincode we give illustrate the use of the FPC Chaincode API; and in samples/deployment we show how to deploy and run FPC chaincode on the Fabric-samples test network and with K8s (minikube).

More details about FPC APIs in the Reference Guides Section.

Your first private chaincode

Create, build and test your first private chaincode with the Hello World Tutorial.

Developing and deploying on Azure Confidential Computing

We provide a brief FPC on Azure Tutorial with the required steps to set up a confidential computing instance on Azure to develop and test FPC with SGX hardware mode enabled.

Reference Guides

Management API

While the management API for Fabric is mostly unchanged, some modifications are needed for FPC to work. In particular, FPC extends the Fabric's lifecycle API with additional commands to create an FPC enclave and handle the key provisioning. These are detailed separately in the FPC Management API document

FPC Shim

The FPC Shim follows the programming model used in the standard Fabric Go shim and offers a C++ based FPC Shim to FPC chaincode developers. It currently comprises only a subset of the standard Fabric Shim and is complemented in the future. These details are documented separately in the Shim header file itself: ecc_enclave/enclave/shim.h

Important: The initial version of FPC, FPC 1.0 (aka FPC Lite), has a few constraints in applicability and programming model. Hence, study carefully the section discussing this in the FPC RFC and the comments at the top of shim.h before designing, implementing and deploying an FPC-based solution.

FPC Client SDK

In order to interact with a FPC chaincode you can use the FPC Client SDK for Go or use the Peer CLI tool provided with FPC. Both make FPC related client-side encryption and decryption transparent to the user, i.e., client-side programming is mostly standard Fabric and agnostic to FPC.

The FPC Client SDK for Go is located in client_sdk/go. See also Godocs.

For the command-line invocations, use the $FPC_PATH/fabric/bin/peer.sh wrapper script. We refer to our integration tests for usage examples.

Getting Help

Found a bug? Need help to fix an issue? You have a great idea for a new feature? Talk to us! You can reach us on Discord in #fabric-private-chaincode.

We also have a weekly meeting every Tuesday at 3 pm GMT on Zoom. Please see the Hyperledger community calendar for details.

Contributions Welcome

For more information on how to contribute to Fabric Private Chaincode please see our contribution section.

References

• Marcus Brandenburger, Christian Cachin, Rüdiger Kapitza, Alessandro Sorniotti: Blockchain and Trusted Computing: Problems, Pitfalls, and a Solution for Hyperledger Fabric. https://arxiv.org/abs/1805.08541

• Fabric Private Chaincode RFC

• Presentation at the Hyperledger Fabric contributor meeting August 21, 2019. Motivation, background and the inital architecture. Slides

• Presentation of at the Hyperledger Fabric contributor meeting November 11, 2020. The design and rationale for FPC Lite (FPC 1.0). Slides

Project Status

Hyperledger Fabric Private Chaincode was accepted via a Hyperledger Fabric RFC and is now under development. Before, the project operated as a Hyperledger Labs project. This code is provided solely to demonstrate basic Fabric Private Chaincode mechanisms and to facilitate collaboration to refine the project architecture and define minimum viable product requirements. The code provided in this repository is prototype code and not intended for production use.

Initial Committers

• Marcus Brandenburger ([email protected])
• Christian Cachin ([email protected])
• Rüdiger Kapitza ([email protected])
• Alessandro Sorniotti ([email protected])

Core Team FPC 1.0

• Mic Bowman ([email protected])
• Marcus Brandenburger ([email protected])
• Jeb Linton ([email protected])
• Michael Steiner ([email protected])
• Bruno Vavala ([email protected])

Sponsor

Gari Singh ([email protected])

License

Hyperledger Fabric Private Chaincode source code files are made available under the Apache License, Version 2.0 (Apache-2.0), located in the LICENSE file.

fabric-private-chaincode's Issues

isgx device name for docker is hardcoded, but the name changes in DCAP

(raised by Michael @g2flyer )
DCAP's sgx device is named 'sgx', so the sgx-enable patch does not provide support for that.

CI support

This project may benefit from a CI for automated testing.

• this depends on #38 (creating global make)
• Travis CI is a good candidate

auction demo example with no submitting transactions fails

If one creates an auction, closes it and evaluates it (so without having submitted any bits), an error is thrown and the peer stops.
error.txt

path of ias credentials is hardcoded

possible solution:
create template of core.yaml for fabric with placemarks for ias credentials path
expand the template with environment variable set by user

fabric-ccenv-sgx image creating fails

make prep fails when applying patches.

patching file /tmp/linux-sgx/linux/installer/common/psw/install.sh
patching file /tmp/linux-sgx/linux/installer/bin/install-sgx-psw.bin.tmpl
Hunk #1 FAILED at 30.

Quick fix would be to pin commit 08c2457.

store spid as text

Remove the requirement to do the xxd hack when creating spid.txt (or at least rename file to spid.bin iff we really insist on this binary format ...)

SGX Simulator support

The current code base uses SGX HW mode and requires IAS access for remote attestation. This makes developing and getting started with this project hard.

The build process should provide a build switch to build ecc_enclave and tlcc_enclave for SGX simulation mode.

Refactoring of ecc (SDKization)

Renaming of component names from trusted to private, e.g., tlcc -> plcc

ecc only includes SDK (with a shim library )

auction (fpc-) chaincode goes to examples/auction

fabric/sgxconfig might also move to examples?

we could also replace all /path/to/fabric by $(FABRIC_SRC_ROOT)/.. with FABRIC_SRC_ROOT as $(SCC_ROOT)/../../hyperledger/fabric/ (ie., assume a single root in GOPATH and then all paths to fabric can be done in relative paths ..

TOP variable in makefiles depends on caller's working directory

Need to:

• set TOP to the path of the top level makefile
• export it for the other makefiles
• perhaps call it FPC_TOP_DIRECTORY

Update names to newly agreed "Fabric Private Chaincode" (or "FPC" for short)

Enclave/peer binding

Enclave registry should maintain some data to bind an enclave to a specific peer. That way we can map an enclave to an organization in Fabric.

Note that, when an enclave is registered, the endorsing peer is usually the enclave host. The attestation should also reflect this.

refactor shim and chaincode interface

the methods provided by shim.h should be revisited. Error handling is missing, for instance, getState does not return an error when a integrity violation is detected.

should parson be exposed to the chaincode app?

pro:

• it is only included when the chaincode needs that

cons:

• it's a dependency external to the shim

other question:

• does every chaincode need it? If so, better put it in the shim

create dockerfile for fabric-ccenv-sgx

Currently, our sgx-enabled docker image fabric-ccenv-sgx depends on https://github.com/tozd/docker-sgx. Thanks again for providing this! However, a recent updated led to errors when building the docker image as reported in #29.

We should provide and maintain or own docker file.

@g2flyer has suggested:

For more long-term i propose we eventually dump docker-sgx and go with our own. We could already use the one from PDO but probably better would be to do the switch once i have a docker image for DCAP (which still supports epid) which i plan to do for PDO and try to do it also in a way that it would be easy shareable/copyable for TCC ...

New IAS authentication

IAS has a new api-key based authentication (described in revision 5.0 of IAS spec)

ecc test failes caused by concurrent map access

return value of ocalls (e.g., ocall_get_state) are not checked

Add missing license identifier

some source files are missing license identifiers.

I propose to follow the Fabric way and use SPDX licence identifiers. They are SHORT, easy to use, machine-readable and language neutral.

Here an example

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0

Moreover, we can add fabric's license checking mechanism to our checks target.

Chaincode enclave / tlcc binding.

When a new chaincode enclave spawns it binds itself to a given tlcc enclave using local attestation. Typically, this includes a key exchange to protect the communication between the chaincode enclave and tlcc later.

However, the current code base has hardcoded shared key. Attestation and key exchange needs a revision.

Steps

• Design of API and high-level architecture
• Library implementation
  • quick'n'dirty mock version
  • secure version
• Library test-cases
• Library integration
  • Session setup
    • from a staging perspective, we might "leak" the key here and stuff it into the place of the currently hard-coded (global) key [note this will prevent concurrent chaincodes with the session lifetime as defined in the UMLs ...)
  • Session request handling

See #410 and #420 for related bigger context.

Update SGX SDK to version 2.5

New SGX SDK v2.5 for Linux has been released recently. We should upgrade and test.

Inappropriate return statement in auction demo example

In my opinion, some return statements in the fabric-secure-chaincode/ecc_enclave/enclave/auction/auction_cc.cpp are not correct. For example in the following section of the auction_submit method (line 115-118):

 if (auction_bytes_len == 0) {
        LOG_DEBUG("AuctionCC: Auction does not exist");
        return AUCTION_ALREADY_EXISTS; 
}

Shouldn't it return a AUCTION_DOES_NOT_EXIST? Which would imply the definition of an appropriate #define alias.

Upgrade Fabric 1.4.1

We should upgrade to the current Fabric version 1.4.1.

Also, it might be a good idea to also pin a release tag instead of using the 1.4 release branch.

Add code check target to build system

formatting check, referring to golinter from fabric; potentially separate comment which auto-indents potentially also clangformat related target for C code

update READMEs to match above changes ..

This is needed for CI as well.

This links back to #38

• code check for GO

• code check for c/cpp #76

• spellchecker (optional)

• license #73

bad return values in sgxcc

Integration test

Refactor run-action into a 'quick-fix' integration test

Define & implement endorsement policy support

In MVP, we support only designated endorser (#273) which requires exactly a single, fixed endorser. So not endorsement policies, in particular not the default MAJORITY policy will work in that case. Following are the necessary steps:

• define the sub-set of valid fabric endorsement policy expression which we support and what the semantics is
• document above in docs/design/fabric-v2+/fpc-management.md.
• (ideally) check in fabric/bin/peer.sh
  • in approveformyorg that policy is provided explicitly and specified policy as satisfiable (note realistically with designated peer, we can never realistically rely on a default policy as single peer policy would be insecure in the general fabric case)
    • actually, one "cheapo" way to check is to call the createenclave with "invoke .... --waitForEvent". This will actually run the specified endorsement policy and would bail iff that enclave is not from the correct org :-)
  • in createenclave that the peer is consistent with the specified policy.
• update FPC examples (& docu) to use the endorsement policy as specified in this work item.

Make logging consumable ...

• more dev-friendly logging in sgxconfig/core.yaml
  - core.yaml vm->docker->attachStdout = true to get output in peer
  - core.yaml chaincode->logging->level = debug
  - core.yaml chaincode->logging->shim = info

• peer add default definition (if not yet defined) of FABRIC_LOGGING_SPEC to a meaningful default, e.g., vscc,ltcc,tl-encalve=DEBUG:comm.grpc=ERROR:INFO

• some reference to logging config in README

• more long term:

  • replace printf in edl with log-function which has levels so we so C levels also in Go
  • separate in enclave shim log the namespace for user-chaincode log statements from log-statements by the FPC runtime (shim)

code checks for c/cpp

We also need automated code checks, in particular for linting, as part of our checks target.
This is part of #55 and works torwards CI support #48.
PDO, for instance, uses clang to check code style.

Project top-level make

We want a build system with a top-level make and a make file in each module

• build
• test
• check
• integration
• clean

This is part of #38

Refactor NanoPB dependencies

make NANOPB include directly from external source (via $NANOPB_PATH so we don't need any cp (or related, currently missing, clean target)?

update READMEs to match above changes ..

this issue was part of #38

run auction fails on first try

This is a known and documented issue.
Yet, it should be automated (at least for now) for users and new developers.

logging should be in the shim, not in the auction

SGX-enable patch may pass a non-existent sgx device to docker container

(raised by Michael @g2flyer )
The isgx device may not exist, for instance in non-sgx-capable machines that simply want to use simulation mode.

Fabric 1.4 Support

Currently fabric-secure-chaincode works only with Fabric 1.2.

Update to Fabric 1.4 and prepare for Fabric 2.xx

Make generate_enclave.sh more robust

Currently, ecc_enclave and tlcc_enclave use a script to generate mrenclave.go. However, this script is a) duplicated and b) not robust.

As suggested by @g2flyer ...

... bigger problem seems that we do no catch any errors and so it was happily running so far (just producing only an empty mrenclave). Easiest fix for that seems to be o just add a 'set -e' in the beginning?

Also, given that the generate_mrenclave.sh is the same in ecc_enclave and tlcc_enclave, maybe we should move that to a common bin directory?

Thinking of it, the method here also seems rather brittle, the way we do it in pdo is more robust:

add to sgx_sign sign the '-dumpfile "${SIGNED_ENCLAVE_METADATA}"' option (see https://github.com/hyperledger-labs/private-data-objects/blob/master/common/CMakeLists.txt) and then
extract it with a more understandable like the ' VAR_MRENCLAVE=$$(perl -0777 -ne 'if (/metadata->enclave_css.body.enclave_hash.m:([a-fx0-9 \n]+)/) { $$eh = $$1; $$eh=~s/0x| |\n//g; $$eh=~tr/a-z/A-Z/; print "$${eh}\n"; }' $(CONTRACT_ENCLAVE_MRENCLAVE_META_FILE))' in target '$(CONTRACT_ENCLAVE_MRENCLAVE_C_FILE)' in pservice/Makefile

Issue with Fabric Secure Chaincode - Chaincode (ercc) Instantiation

I was trying to experiment with Secure Chaincode Execution using Intel SGX for Hyperledger Fabric. I followed all the prerequisites and steps given in the github link - https://github.com/hyperledger-labs/fabric-secure-chaincode .
After executing all the steps, when I tried to instantiate chaincode (ercc) using the command - $peer chaincode instantiate -n ercc -v 0 -c '{"args":["init"]}' -C $chanid -V ercc-vscc, I am getting the below error –
Error: could not assemble transaction, err Proposal response was not successful, error code 500, msg failed to execute transaction 76c60f8238b886375987a7e2a2ff4a366b5fa2e17da32ab2ab9ec251dad8860a: error starting container: error starting container: API error (500): linux runtime spec devices: error gathering device information while adding custom device "/dev/isgx": no such file or directory

Peer log is as mentioned below –

2018-12-20 12:31:34.498 UTC [golang-platform] GenerateDockerBuild -> INFO 030 building chaincode with tags:
2018-12-20 12:32:07.209 UTC [dockercontroller] func2 -> INFO 031 Container dev-jdoe-ercc-1 has closed its IO channel
2018-12-20 12:32:07.362 UTC [dockercontroller] Start -> ERRO 032 start-could not start container: API error (500): linux runtime spec devices: error gathering device information while adding custom device "/dev/isgx": no such file or directory
2018-12-20 12:32:07.467 UTC [chaincode] Launch -> ERRO 033 start failed: API error (500): linux runtime spec devices: error gathering device information while adding custom device "/dev/isgx": no such file or directory
error starting container
error starting container
2018-12-20 12:32:07.467 UTC [endorser] SimulateProposal -> ERRO 034 [mychannel][76c60f82] failed to invoke chaincode name:"lscc" , error: API error (500): linux runtime spec devices: error gathering device information while adding custom device "/dev/isgx": no such file or directory
error starting container
error starting container

I would be really grateful if anyone of you can provide any guidance to resolve this issue.

Thanks in Advance,
With Best Regards,
Chandrika Basak

Generalize attestation support for linkable/unlinkable signatures

part of #15
and necessary for the "one make" milestone, since users that require linkable attestations have to make changes to the code

Unit test cleanup

Update to IAS v3

Currently we use IAS v2 protocol. Seems this is EOL already and needs to be changed, otherwise ercc can not contact IAS to verify quotes.

Generalize attestation support

Generalize attestation support to cover linkable and unlinkable EPID as well as DCAP/3rd-party attestation

Update SGX SDK to version 2.4

Also update to most recent sgx ssl version (2.4.1)

TLCC: persistant channels

currently, there is no state which enables tlcc to automatically re-join channels joined on a previous "incarnation" of peer.

PS: It should work, though, by manually calling 'JOIN_CHANNEL' using the peer cli after each peer restart. It just might take a while before tlcc is fully operational if the ledger is large (and not sure whether we ever tested that?)

TLCC: multi-channel support

should be relatively straightforward

• TrustedLedgerCC needs a map channelname -> enclave rather than only enclave
• getMetadata has to do stub.GetChannelID() and then pick correct enclave

Note: a peer though should still be able to have additional channels as long as they do not run or interact with FPC chaincode.

allow FPC CC names other than "ecc"

It actually does "work" (in a sense of that it does not fail) when run with different names but lots of code in tlcc and ercc seems to rely on the ns to be "ecc" for checks which probably are simply skipped if the name is different?

PS: Further qualification to "work". It seems it works in that case somewhat randomly as if there is a race-condition somewhere. Which probably also relates to all the sleeps in our tests which we might have to replace with something less brittle (i.e., checks that async action on ledger really happened correctly)

Proxy support

ercc does not run behind a proxy as it cannot connect to IAS

ias credentials not checked

IAS credentials should be checked (for existence) upfront

• in in the integration folder (once PR #79 is merged)
• in the fabric/sgxconfig/ias folder

Issue with Instantiation of ecc chaincode

I installed the example chaincode using command –
$peer chaincode install -n $ccid -v 0 -p github.com/hyperledger/fabric/examples/chaincode/go/example02/cmd

But I didn’t see any docker container or image getting created here at this point of time (checked it by running commands – “docker ps -a” and “docker images -a”).

Then I went to “ecc” folder and ran the command –
make docker DOCKER_IMAGE=dev-jdoe-ecc-0
This created a new docker image but I am not sure how should I link this image with the installed chaincode (example chaincode).

Then I instantiated the chaincode using command
$peer chaincode instantiate -o $orderer -C $chanid -n $ccid -v 0 -c '{"args":["init"]}' -V ecc-vscc
This throw an error but at the same time, created a separate docker image and docker container.
The error says –
Error: could not assemble transaction, err Proposal response was not successful, error code 500, msg transaction returned with failure: Incorrect number of arguments. Expecting 4

Question remains -
• How do I rebuild the docker image for the ecc chaincode manually in such a case?
• How will I recreate the ecc docker container?
as there is no image/container getting created post installation of example chaincode.

Please suggest!
Thanks in Advance,
Chandrika

Global Makefile

• build system with a top-level make and a make file in each module. More details in #56

  • build
  • test
  • check
  • integration (end-to-end-test (just calling run-auction initially?))
  • clean

• config.mk file with global variables like

  • GO
  • GOFLAGS

• code checking target. More details in #55

• refactoring of ecc (SDKization)

  • renaming of component names from trusted to secure, e.g., tlcc -> slcc
  • ecc only includes SDK (with a library)
  • auction goes to examples/auction
    • fabric/sgxconfig might also move to examples?
      • we could also replace all /path/to/fabric by $(FABRIC_SRC_ROOT)/.. with FABRIC_SRC_ROOT as $(SCC_ROOT)/../../hyperledger/fabric/ (ie., assume a single root in GOPATH and then all paths to fabric can be done in relative paths ..
        => delegated to issue #69

• Nanopb refactoring. More details in #55

• update READMEs to match above changes ..

cleanup of 'make clean'

-ercc force docker remove
-force rm in ecc_enclave
-force rm sgxcclib
-ecc docker stop check container exists
-docker rmi fails (requires 1 argument)
-tlcc rm enclave/ ... force it
-tlcc rm trustedledger ... force it
-check docker_image in ecc (patch below)
diff --git a/ecc/Makefile b/ecc/Makefile
index 7faef50..d2c0700 100644
--- a/ecc/Makefile
+++ b/ecc/Makefile
@@ -52,6 +52,7 @@ clean: docker-clean

rm enclave/mrenclave.go

docker:

•   @if [ -z "${DOCKER_IMAGE}" ]; then echo "ERROR: you have to run_action.sh first before invoking this target"; exit 1; fi
    $(DOCKER) build -t $(DOCKER_IMAGE) -f Dockerfile ..
  

docker-run:

Rename fabric-secure-chaincode to fabric-private-chaincode

As we renamed the project and repo we need to change all readme files