180protocol/codaptor

Instantly add Corda to any tech stack and improve resiliency of your architecture

Introduction

180Protocol is an open-source toolkit for data sharing. 180Protocol represents the next evolution of data sharing infrastructure, an infrastructure that ensures a secure, rewarding, and efficient data sharing experience.

180Protocol targets enterprise use cases and is uniquely positioned to improve the value and mobility of sensitive business data. The software introduces a distributed network design to the problem of enterprise data sharing, solving for the legacy barriers that have limited data mobility and value. 180Protocol makes data available where it needs to be, moving it and transforming it along they way, all the while preserving privacy.

Design and Components

Developers can utilize 180Protocol to create data unions - permissioned networks on R3 Corda to share their private structured data. Unions (also referred to as Coalitions) are composed of corda nodes that can act as Data Providers and Data Consumers, and a union host.

• Data Providers - nodes that have pre-approved structured private data assets that they want to share and be rewarded for
• Data Consumer - nodes that have a need to consume unique and commercially valuable data outputs
• Union Host - node that runs the trusted enclave (via R3 Conclave) and arbitrates communication between providers and consumers

180Protocol comprises the following components:

1. Aggregator SDK - allows defining a data transformation service that can take flexible data input, data output, and provenance data definitions. A data transformation algorithm can also be configured to map inputs to outputs. The Aggregator includes a Rewards Engine that can be configured to reward data providers in a union for sharing sensitive data. Since the aggregator runs inside a trusted enclave, the sensitive data is safe from exploitation. Additionally, the Rewards Engine, providers data inputs are rewarded unbiasedly. Data Consumers get data outputs based on a known transformation algorithm.

2. 180Dashboard - a React based front end application that allows Data Providers and Data Consumers to keep track of shared data, view data aggregation history and keep track of rewards for each data aggregation.

3. Codaptor - a middleware that connects to the Corda RPC and generates OpenAPI bindings automatically for any CordApp

The above components can be used by application developers to create decentralized applications that enable private structured data to be shared, transformed and rewarded for in a secure way.

This repository contains example applications built on the 180Protocol framework and these be utilized by developers as a blueprint to start building more complex applications. We encourage application developers and businesses to build on 180Protocol and provide feedback, so we can introduce better features over time.

Please read more detailed design and API specifications on our Wiki

Use 180Protocol

Please follow the documentation on our Wiki to get started with 180Protocol -

• Set up and run 180Protocol
• Managing a 180Protocol union network
• Writing your 180Protocol CoApp
• Using Filecoin as Storage
• Examples
• Aggregator SDK API's

Features and design

Read about our features and design details -

• Features
• Network Architecture
• Data Flows

Storage

180Protocol received a grant from Filecoin to integrate with the Filecoin ecosystem. More details can be found here. As part of the grant, we have utilized the Estuary API to store input data from Providers and output data for Consumers on Filecoin. We have created a separate module called estuaryStorage that can be optionally deployed as a CordaApp. This module overrides base AggregatorSDK workflows and gives Providers and Consumers the option to store data on Filecoin.

Read more about the architecture and design on our wiki - Storage

Roadmap

This is the alpha release of 180Protocol and is experimental in nature. There are improvements we plan to release in the coming weeks including -

1. Introducing a variety of example use cases across industry verticals - ✅
2. Introducing Filecoin as a storage layer for data inputs and outputs - 🟡
3. Supporting multiple enclave communication frameworks beyond R3 Conclave. We are currently testing compatibility with Anjuna.io - 🟡
4. Introducing a regression based rewards engine that can be utilized within the Aggregator SDK - ⚪
5. We have tested the enclave based computations in the Enclave 'Mock' mode. We are testing deployment using an MS Azure cloud enclave - ⚪
6. The Aggregator SDK workflows are consumer driven only. All providers in the network are required to share data when requested by the consumer. We intend to introduce further variations of the Aggregation Flows that account for varied commercial use cases - ⚪
7. Integrating the Aggregator SDK with a public blockchain to enable data providers to monetize their rewards using a token economy - ⚪

Legend

• ✅ - completed
• 🟡 - in progress
• ⚪ - planned

Contact

• Commercial queries: [email protected]
• Developer Discord: 180Protocol Discord
• Community channels: www.180protocol.com

Licenses

Please take note of the license obligations you are bound under by downloading the source code. The repository contains two folders, under different licenses -

1. 180Dashboard - released under the Apache2.0 License
2. protocolAggregator - released under the GNU AGPL3.0 license

Additionally, protocolAggregator has a dependency on R3 Conclave, which requires developers to download the Conclave API. Please take a look at the R3 Conclave license considerations for further information.

We have tried to emulate MongoDB's philosophy in this regard. Please read further

codaptor's Issues

Non composable type flows are not serialized

Steps to reproduce:

1. Run cordaptor in standalone mode with obligation cordapp (both corda version 4.5)
2. Error is thrown preventing Swagger UI from coming up
   {"cause":{"class":"tech.b180.cordaptor.rest.SerializationException","message":"Flow type identified as Parameterised(CordaFlowInstruction<CashExitFlow>) was not introspected as composable.\nIntrospection details: CashExitFlow)","cause":"null"},"errorType":"GENERIC_ERROR","message":"Unknown server error"}

Reference Cordapp can't use simple flow

Steps to reproduce:

1. Build cordaptor on Corda version 4.4
2. Error results in
   net.corda.core.transactions.MissingContractAttachments: Cannot find contract attachments for tech.b180.ref_cordapp.TrivialContractnull.

expose deployment-agnostic Corda vault query API to Cordaptor extensions

As a developer creating CorDapp-specific extension for Cordaptor I want to use Corda vault query API, so that my code is more expressive.

Rationale: Corda offers nearly identical API for querying the node's vault both over Corda RPC and internally via VaultService. These APIs could be surfaced to extensions in a uniform way. This will simplify development of CorDapp-specific extensions that work across all deployment models.

Corda Internal API can't access cordapp list

Steps to reproduce:

1. Build cordaptor on Corda version 4.4
2. Error results in
   java.lang.AssertionError: Internal API is accessible via service hub instance expected:<1> but was:<0>

Swagger UI request body don't show for Map data type

Describe the bug
Swagger UI request body don't show for Map data type

To Reproduce
Steps to reproduce the behavior:

Startup the application up using docker compose
Open the host node swagger UI at localhost:9500/swagger
The request configuration does not show up for map data type request

Expected behavior
The sample request configuration should show up

java.time.Duration serialisation is not supported

Steps to reproduce:

1. Run cordaptor v 0.1.0 with obligation cordapp (Advanced examples from corda samples repo) in embedded mode
2. Open the api.json page

While trying to serialize an object with java.time.Duration class as a property, there is a serialisation exception thrown.

{ "cause": { "class": "tech.b180.cordaptor.rest.SerializationException", "message": "Cannot create a serializer for type Erased(net.corda.finance.contracts.asset.Obligation$State) introspected as non-composable for the following reason: Has properties [template] of types that are not serializable:\ntemplate [net.corda.finance.contracts.asset.Obligation$Terms<*>]: Has properties [timeTolerance] of types that are not serializable:\n timeTolerance [class java.time.Duration]: Mandatory constructor parameters [arg0, arg1] are missing from the readable properties []", "cause": "null" }, "errorType": "GENERIC_ERROR", "message": "Unknown server error" }

Improve CordApp Introspection for flows and states

Current CordApp introspection logic involves scanning Cordapp object (from Corda Core library) for all underlying cordapps loaded at the given node.
States
For states this logic, involves looking at all cordapp classes and filtering out all package names that are of type ContractState. However, this utilizes the Java Class forName API that instantiates an instance of the said package, causing issues. To improve this, package names only containing 'state' in their qualified name are filtered and inspected.
Flows
Additionally, for flow inspection, the allFlows parameter on the Cordapp object was utilized. This caused issues inspecting flows that inherit a given parent flow. Therefore, serviceFlows or rpcFlows parameters are used respectively for embedded and standalone modes.

Scripts Have Windows Line Endings

The scripts don't start on linux because they use Windows file endings:

./cordaptor.sh
bash: ./cordaptor.sh: /bin/sh^M: bad interpreter: No such file or directory

file cordaptor.sh
cordaptor.sh: POSIX shell script, ASCII text executable, with CRLF line terminators

Support serialisation for Corda LinearPointers and StaticPointers

Steps to reproduce:

1. Start Auction Cordapp from Corda repository Advanced samples
2. An error is thrown when serialisaing the AuctionState - auctionItem object which is of LinearPointer type
3. The following error message is displayed -
   {​​​​​"cause":{​​​​​"class":"tech.b180.cordaptor.rest.SerializationException","message":"Cannot create a serializer for type Unparameterised(net.corda.samples.auction.states.AuctionState) introspected as non-composable for the following reason: Has properties [auctionItem] of types that are not serializable:\nauctionItem [net.corda.core.contracts.LinearPointer<net.corda.core.contracts.LinearState>]: Has properties [type] of types that are not serializable:\n type [java.lang.Class<net.corda.core.contracts.LinearState>]: No unique deserialization constructor can be identified","cause":"null"}​​​​​,"errorType":"GENERIC_ERROR","message":"Unknown server error"}​​​​​

Allow to override the URL prefix used for CorDapps

User story:
As a CorDapp developer who wants to expose its API via Cordaptor, I want to be able to configure URL prefix to use for the API,
so that I can make it look reasonable without changing anything else in the CorDapp.

Rationale:
As of v0.1.0, Cordaptor takes shortName from net.corda.core.cordapp.Cordapp.Info, which is derived from attributes of MANIFEST.MF of the CorDapp JAR in Corda 4. This value is used as part of an URL prefix for API endpoints.

There is no common convention for this value, which means some CorDapps will have characters that are not compatible with URLs, have very long prefixes, or both, e.g. https://github.com/corda/bn-extension/blob/release/1.1/build.gradle

There has to be a way to override this value at the level of the CorDapp.

Other consideration is that it's likely in future there will be other settings to configure at the CorDapp level, so a solution with more broad potential is likely to be useful.

add support for custom fields in queryable states in vault query API

Build error in reference-cordapp and rest-endpoint modules

Steps to reproduce:

1. Cordaptor 4.5
2. Attempt to compile cordaptor
3. Error in test cases for rest-endpoint & reference-cordapp modules

tech.b180.ref_cordapp.CordaInternalApiTest > can access cordapps list FAILED
    org.junit.ComparisonFailure at CordaInternalApiTest.kt:74

Cordapp introspection fails for contract states

Summary:

Current Cordapp introspection logic queries and filters all classes on the cordapp for contract state classes. This includes dependencies such as those on apache and other libraries. Such libraries are sometimes sealed and thus introspection can fail -

Caused by: java.lang.SecurityException: sealing violation: package org.apache.commons.lang3.time is sealed
cordaptor_1    |        at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:400)
cordaptor_1    |        at java.net.URLClassLoader.definePackageInternal(URLClassLoader.java:420)
cordaptor_1    |        at java.net.URLClassLoader.defineClass(URLClassLoader.java:452)
cordaptor_1    |        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
cordaptor_1    |        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
cordaptor_1    |        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
cordaptor_1    |        at java.security.AccessController.doPrivileged(Native Method)
cordaptor_1    |        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
cordaptor_1    |        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
cordaptor_1    |        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
cordaptor_1    |        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
cordaptor_1    |        at java.lang.Class.forName0(Native Method)
cordaptor_1    |        at java.lang.Class.forName(Class.java:264)
cordaptor_1    |        at tech.b180.cordaptor.corda.CordappInfoBuilder.build(Introspection.kt:96)
cordaptor_1    |        at tech.b180.cordaptor.rpc.ClientNodeCatalogImpl.<init>(ClientNodeCatalog.kt:30)  

create serializers for anonymous and concrete parties

CorDapp developers tend to use AbstractParty when they want to support concrete parties, as well as anonymous identities. Currently Cordaptor only supports concrete parties. We need to change serialization logic to support both anonymous and concrete parties.

Cordaptor is coupled with the underlying node, so it will follow the principle that if the node knowns about anonymous party identity, it will provide its X.500 name, otherwise only a hash of the owning key.

Hashing mechanism for the keys will obtain SHA256 hash of the key bytes, and then represent it as a string encoded as Base58, in line with Corda and general crypto community practice

WebSockets-based real-time flow tracking

Add support for flow initiation and progress tracking via WebSockets.

In some cases API clients may require tighter feedback about execution of a flow instance. Add an ability to use websockets for receiving real-time updates from flow progress tracker and flow outcomes.

Serialisation of NonEmptySet causes an error

Steps to reproduce:

1. Run cordaptor v 0.1.0 with obligation cordapp (Advanced examples from corda samples repo) in embedded mode
2. Open the api.json page

Serialisation of corda core type NonEmptySet is not supported and throws an error -

{
  "cause": {
    "class": "tech.b180.cordaptor.rest.SerializationException",
    "message": "Error finding serializer for property acceptableContracts of object Parameterised(Terms<*>)",
    "cause": "java.lang.AssertionError: Don't know how to make instances of class net.corda.core.utilities.NonEmptySet"
  },
  "errorType": "GENERIC_ERROR",
  "message": "Unknown server error"
}

NonEmptySet is introspected as a Collection type and the solution would handle all such non supported collection types that do not need to be de-serialised.

Readme links to Sonatype OSS Snapshots broken

Hello,
I was trying to get the latest version of Codaptor, with the Feature 28: CORS Request support

However, it seems that the links to the Sonatype OSS Snapshots in the Readme Section Versions and Artifacts are broken. They redirect to the following url https://oss.sonatype.org/content/repositories/snapshots/
with the message Directory listing forbidden.

Would you be able to fix that please?

Also, do you have any plans as to when a new version will be released? Useful features such as CORS support were implemented since the last release.

Thank you very much for your work. Codaptor truly makes creating REST APIs for CorDapps very easy!

Corda 5 support

Upgrade Codaptor to Corda 5 and test out serialisation framework

Flow classes not extending FlowLogic directly are not serialised

Steps to reproduce:

1. Run cordaptor v 0.1.0 with obligation cordapp (Advanced examples from corda samples repo) in standalone mode
2. Open the api.json page
3. CashFlowExit flow is not serialised and causes an exception

cordaptor_1          | Exception in thread "main" java.lang.AssertionError: Flow class class net.corda.finance.flows.CashExitFlow does not seem to extend FlowLogic
cordaptor_1          |  at tech.b180.cordaptor.corda.ReflectionUtilsKt.determineFlowResultClass(ReflectionUtils.kt:16)

The fix will involve introspecting all supertypes rather than direct supertype when introspecting flow classes

We need a custom serializer for Corda Amount<Currency>

Introspection of the Amount class generates a weird-looking JSON, and we need to make it easier to work with.

Further, most commonly used token for Amount is a java.util.Currency, and it would be better if we represented it as a JSON string containing ISO code.

Support CORS requests

Certain web app will require cross origin request. Config to enable CORS by controlling the AllowedMethods and AllowedOrigins for requests coming to the Undertow server.

For detailed CORS config the following plugin can be used https://github.com/Download/undertow-cors-filter

Support for different versions of Corda 4

Cordaptor 0.1.0 is built with platform version 7, expecting Corda 4.5+. We don’t have critical dependency on any APIs, and retrofitting support for older versions should be straightforward.

Please comment on what versions you’d like to see supported

Support attachment uploads for node

• New AttachmentEndpoint that can allow a node to upload attachment in the form of zip or jars
• Support for both standalone and embedded

Support Multi Part Form Requests for Structured Objects like Corda Flow Instructions

Currently, serialising Files is only supported as part of the CordaNodeAttachmentSerialiser. However, File streams can be inputs to flows. Hence a InputStreamSerialiser needs to be written to handle File binary streams to be serialised as InputStream

Multi Part Form Serialisation breaks for Swagger UI based requests

While testing flow triggering with multipart/form-data Content Type with Swagger UI, an options parameter is added by default. This caused the serialisation to break as options parameter is a nested structured object, that requires to be treated as such.

Cannot create CordaFlowResult for type List

Steps to reproduce:

1. Open codaptor with snakesandladders-cordapp on the corda github Advanced examples
2. After startup an error is displayed with the following message -

{​​​​​"cause":{​​​​​"class":"tech.b180.cordaptor.rest.SerializationException","message":"Error finding serializer for property value of object Parameterised(CordaFlowResult<List (erased)>)","cause":"java.lang.AssertionError: Don't know how to make instances of interface java.util.List"}​​​​​,"errorType":"GENERIC_ERROR","message":"Unknown server error"}​​​​​
Edit:

It seems the issue is because the Flow return type for flow AccountInfoByName under Accounts SDK of Corda has nested parametrisation - e.x., FlowLogic<List<StateAndRef>>

The SerialiserKey in this case is not being created correctly

Upgrade to corda version 4.7

Support all operators for custom queries

1. Currently only the equals operator is supported for cordaptor expression vault queries
2. We need to support other operators as well

kotlin Unit class is not serialised correctly

Steps to reproduce:

1. Cordaptor 4.5, Negotiation cordapp 4.5
2. Attempt to start cordaptor in standalone mode
3. Error in serialising ModificationFlow

{ "cause": { "class": "tech.b180.cordaptor.rest.SerializationException", "message": "Error finding serializer for property value of object Parameterised(CordaFlowResult<Unit>)", "cause": "tech.b180.cordaptor.rest.SerializationException: Don't know how to create a serializer for type Unparameterised(kotlin.Unit) (introspected as net.corda.serialization.internal.model.LocalTypeInformation.Singleton)" }, "errorType": "GENERIC_ERROR", "message": "Unknown server error" }