verida / verida-js Goto Github PK

View Code? Open in Web Editor NEW

935.0 14.0 33.0 3.87 MB

The Verida SDK provides several SDKs to interact with the Verida Network

Home Page: https://developers.verida.network

License: ISC License

TypeScript 99.98% Shell 0.02%

blockchain decentralized identity personal-data privacy web3

verida-js's Introduction

Verida Javascript Library

This monorepo contains Verida Client Library and a variety of utility packages that support that library.

There is a React Native repository that maintains a slightly modified version of the Verida Client Library that is used to generate the @verida/client-rn package.

Packages

These are the main packages you are likely to use:

client-ts: Verida client library. Install this library in your project to add support for the Verida protocol.
account-web-vault: Create a Verida network connection that uses a QR code and the Verida Vault to sign authentication requests. Use this for your web application.
account-node: Create an instance of a Verida account using a private key or seed phrase. Automatically signs all authentication requests. Use this for NodeJS server side applications or React Native applications.

These are helper packages that typically aren't used directly:

account: Common code shared amongst the various account-xxx implementations
encryption-utils: Encryption utilities to make using tweetnacl a bit easier
keyring: Keyring for managing asym, sym and signing keys for a given account context
storage-link: Utilities to help linking storage endpoint information to a Verida account

Developer Notes

Node Version

This requires node v14.17.1 to build. We use nvm to manage this:

nvm install v14.17.1
nvm use v14.17.1

Linking dependencies

It's not possible to add dependencies between monorepo packages using yarn (ie: yarn add @verida/encryption-utils) if that package hasn't been published to npm.

Unpublished dependencies between monorepo packages can be linked by:

Manually adding the expected dependency to package.json (ie: @verida/encryption-utils)
Run npx lerna bootstrap in the root directory of this project

Building

Build everything:

In the root directory:

npx lerna bootstrap
npx lerna run build

To build a specific package:

cd package/package-name
yarn
yarn build

Creating a release

Update all the CHANGELOG.md files in each package to include entries for all the changes made since the last release.

$ npx lerna run build
$ npx lerna publish --dist-tag next

Use tag next for an upcoming release or latest for the latest version.

verida-js's People

Contributors

Stargazers

Watchers

verida-js's Issues

[account-web-vault] login screen should make QR code visible or scrollable when embedded

The tech capability demo needs to embed the login screen in a iframe.

Currently the QR Code is cut off and the iframe cannot be scrolled. It appears we are doing something to stop this, since other embedded codesandbox frames scroll fine (including our own code).

We need to be able to scroll to see the QR code

Example showing scrolling works

Update vault common to use client-ts

Ceramic issue happened randomly

This error is thrown randomly when creating/connecting to an account.

HTTP request to 'https://ceramic-clay.3boxlabs.com/api/v0/streams' failed with status '': <html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
</body>
</html>...

[account-web-vault] Support logging out per context

disconnect() currently logs out of all sessions and ignores the contextName parameter.

Verida protocol schema research

Verida schemas

The Verida protocol supports defining a schema for data stored in Verida databases. These schema'd databases are called datastores. See Verida schema documentation.

Anyone can define a schema and reference it by URL. The Verida protocol defines a collection of base schemas in the schema repo.

All Verida schemas should match the JSON schema specification.

We need a Verida meta-schema

Verida schemas need to provide additional metadata, beyond just "validation rules".

This includes:

Information about the database storing this schema (database name, indexes etc.)
Display information used when displaying this schema in the Verida Vault (or other applications)

You can see an example of this additional metadata in the draft Veirda social/contact schema.

Tasks:

Define a JSON structure for storing the above metadata
Define and publish a Verida JSON metaschema
Update our existing schemas to reference the new Verida JSON metaschema

We need a plan for schema versioning

Schemas will change. We need to have a clearly defined strategy and process to help developers update schemas.

I'm not sure we need to solve this problem now, but at least need a plan.

Things to consider:

What prior art / existing versioning strategies that exist? (ask community?)
How to migrate data from one schema to an upgraded version?
How to ensure all client applications are using the latest schema?
How strict should client implementations be to ensure the latest schema is being used?
What role can the Verida Vault play to ensure valid schemas are being used?
Need to support versioning of the meta schema
When data is signed, it includes signing the schema URI. This means "updating" the schema to a new URI will invalidate the signature unless the data is re-signed by the data originator (unlikely).

My Initial thoughts:

In phase 1, we support versioning by having a convention of building it into the URL, ie: .../social/contact/schema/v1.json. Since multiple schemas can use the same database, it's possible to have records stored in the same database but using different schema versions.

In phase 2, we support a "data migration" process, whereby a new schema version can define a data migration schema. The Verida client can support applying this data transform based on the data migration schema to convert data from an older schema to the new schema.

Tasks:

Research current versioning strategies
Document a set of staged recommendations to build into the Verida protocol
Sign off
Implement first phase of recommendations

Schema security

There's a security risk where a schema is specified by URL and then the schema is modified (or the hosting provider hacked) to generate a different URL. For example, modifying the schema to remove the list of required fields, allowing data to be saved across the network with invalid data.

I don't think we need to solve this right now, but need to consider the implications and have a strategy to improve this in the future.

It's possible to use IPFS to store a schema and then reference the content addressable URI within data saved using the Verida protocol.

In a future phase, we could support on chain "schema hashes" via the Trust Framework. This allows schemas to be referenced by an on-chain hash instead of a https URL. Ceramic network also provides similar capabilities.

Tasks:

Document an initial assessment of the security risks
Research appropriate mitigation strategies and document recommendations

Community resources

The following community resources exist and seem active:

JSON schema slack -- I've had success here in the past
Google Groups
Github Discussions

Add support for external datastores

Update existing applications to use new repo

Update @verida/vault-mobile
Update all demos

Refactor profiles

Public profile is currently key/value.

Refactor to support multiple public profiles in the same database with any number of schemas.

Add support for a basicProfile schema based on Ceramic basicProfile.

Requires:

Refactoring client-ts
Update vault-common
Updating vault-mobile

User / Context Mapping

Each user (DID) has an array of data storage contexts (applications) where their data is stored. This allows a user to have data stored across three different applications and stored on three different infrastructure providers.

Additionally, each of those storage contexts have a set of public keys (asymmetric and signing) that can be used to communicate / verify data relating to that user in the particular storage context.

This data is stored in Ceramic's IDX and is public. See this example entry for the Verida Vault for a given DID (3ID):

{
  id: 'Verida Vault',
  publicKeys: {
      asymKey: {
          type: 'Curve25519EncryptionPublicKey',
          base58: 'kjzly24aefcy24aefcy24aefc8b5d'
      },
      signKey: {
          type: 'ED25519SignatureVerification',
          base58: 'zjzly24aefcy24aefcy24aefc8b5d'
      }
  },
  services: {
		storageServer: {
			type: 'VeridaStorage',
			endpointUri: 'https://localhost:5000/'
		},
		messageServer: {
			type: 'VeridaStorage',
			endpointUri: 'https://localhost:5000/'
		}
	}
}

Question: Should the id property (Verida Vault) be hashed along with the user's DID?

ie: instead of id=Verida Vault we would set id=hash(did + context.id).

This prevents a third party looking up a user's DID and easily seeing all the applications they have connected to. A third party could manually build a list of all applications in the Verida network and hash them with the user's DID to produce a list — but that's a significant amount of work and doesn't scale to easily index every Verida user.

This doesn't prevent a third party checking if a DID is using a particular application context (which we obviously don't want, as this information is public and needs to be looked up).

Add support for creating a DIDJWT for an account

This is required for: verida/vault-auth-server#7

Enforcing tombstones instead of deleting documents

From the confidential storage working group discussion and spec

"Delete Resource (Document or Stream): Deletes the encrypted resource (note that a tombstone object should remain behind for replication purposes)."

Verify how tombstones is currently enforced.
Check that the database access permissions don't allow users / applications to delete documents

Support account-web-vault disconnect

Add support for a disconnect() method in account-web-vault/vault-account.ts (https://github.com/verida/verida-js/blob/main/packages/account-web-vault/src/vault-account.ts).

Requires adding a disconnect() method to the AccountInterface (#37)

[account-web-vault] Login when already on phone doesn't work

The Login with Verida Vault link does nothing when clicked.

This needs to redirect to the same URL that is embedded in the QR code.

Also, is there any way we can detect you're on a mobile device and auto redirect if you are?

Check indexes are created if added to a schema

There is an issue where a new index added to a schema doesn't get created for a user if they have previously opened that schema.

[account-web-vault] Emit cancelled event when QR modal is closed

The account-web-vault library should emit an event when the QR modal is is closed and login request is not made , so that the frontend applications can listen to this cancelled event and update its loading state .

Create Web Account implementation

Create an account implementation that handles logging in via Ceramic 3ID in the browser and then handles signing consent messages.

See the AutoAccount implementation that automatically signs a consent message given a private key on a NodeJs server: https://github.com/verida/verida-js/blob/feature/datastore-support/packages/account/src/auto.ts

Migrate messaging

Migrate the inbox and outbox functionality from @verida/datastore into client-ts.

Investigate ceramic deterministic stream error

Unit tests are now throwing this error:

Error: Cannot change 'deterministic' or 'unique' properties on existing Streams

Support NEAR DID's

Update 3id-utils to support NEAR accounts. Add unit tests.

Update code documentation

Update existing documentation to match the changes made in this new package.

Requires:

client-ts: API documentation of all public methods
client-ts: Updated README.md
account-node: Updated README.md
account-web-vault: Updated README.md

Create apidocs.verida.io

Create autobuild from old data store library, in package.json
deploy

Return unsubscribe function from each event handler

Each event handler should return an unsubscribe function so that client can handle it correctly, especially in React.
For example:

const unsubscribe = messaging.onMessage(function () {
        fetchInboxCount()
})

// Unsubscribe when that screen is not rendered anymore:
unsubscribe()

Support account disconnect

Tasks:

Add to account interface
Call disconnect() when client.disconnect() is called.

[account-web-vault] Closing the connect modal shows "Connecting..."

Closing the connect modal shows Connecting... screen.

However the user has clicked the [x] button or clicked away from the modal without scanning the QR code. In that case, the modal should disappear and the original webpage be shown.

Refactor DID support to use 3ID's

Verida currently has its own DID generation approach. This has served it well for a PoC, but it's better to migrate to an alternative DID solution. In this issue I will outline a migration plan to use Ceramic network's 3ID DID provider.

Current architecture

Verida currently generates a DID on a per application basis:

A blockchain wallet (Ethereum, VeChain, NEAR) is used to sign a consent message (ie: Do you want to login to the Verida Vault application?)
The signature from this consent message is used as entropy to generate a new hierarchical deterministic wallet
This HD wallet is then used to generate an encryption keypair and signing keypair
A new Verida DID document is created that lists the public keys of those keypairs, which is then stored in a centralized API
This DID document also stores information such as the database address for the user's encrypted data (using DID service definition)
The encryption key is used to encrypt all data within that application context
The signing key is used to sign all generated data within that application context

This offers the following benefits:

An existing blockchain wallet can be used to unlock the data for any application
This unlocking is deterministic. No private key needs to be stored, it can be generated on demand from the master blockchain wallet private key
Each application has it's own secure enclave of data, allowing a web application to only gain access to data relevant for its use case

Ceramic architecture

Ceramic provides an alternative approach, whereby multiple blockchain wallets can be used to control a 3ID DID via their 3ID DID Provider. Utilizing the Ceramic network and IDX protocol, it's possible to store public or encrypted data for each 3ID.

Under this model a user will need a 3ID in order to use the Verida network. A user will authenticate (or sign up) as follows:

A user of a web application will login via 3ID connect (a web based interface for authenticating with a blockchain wallet to unlock an existing 3ID)
Alternatively, a user of a web application will login via the Verida QR code login system connected to the Verida Vault

In the future, it may be possible for the Verida Vault QR code auth to be integrated into 3ID Connect.

Behind the scenes, the following needs to occur:

Unlock or create the user's 3ID
Once authenticated, the user can write data into their stream
The 3ID provider instance private key can be used to deterministically sign a consent message to authenticate an application
The 3ID provider instance can be used to generate encryption and signing keys (as per current process)
The 3ID provider instance can be used to store the public key on a per application basis within the 3ID's IDX profile (instead of storing a Verida DID in a central database)

Why Ceramic?

It's built to provide this core DID infrastructure
Utilizes the latest decentralized standards and plays nice with DIF projects
It has many features (key rotation, multiple blockchains supported, well supported project)
It unlocks an ecosystem of existing apps that can tap into the power of the Verida network

Risks to eliminate

Create a web based PoC that validates the above approach to using 3ID to store and unlock per application databases will work as expected
Ceramic + IDX won't work within React Native environment
Ceramic + IDX introduces latency issues in web or mobile environments
Ceramic + IDX adds a significant file size overhead to the Verida Datastore library

Datastore validation of emails doesn't work

This was identified in the web-sandbox.

Manually test in web browser environment

Manually test the code running in a web browser environment.

Depends on #6.

In the future create a full web test suite.

Add verida connect button from [account-web-vault] package

Task

The account-web-vault packages should support adding the verida connect button image to third parties application,
this can be done by adding custom classnames and using the window.onload to inject the img tag into the DOM

Question: Context nomenclature

In Verida, the user stores data in database collections. These collections are siloed within different contexts.

For example, a user may have three contexts:

Verida: Vault (all data stored in their vault)
Secure Markdown Editor (a markdown editor application)
Social platform (a generic social media application)

Each of these contexts have their own private key to unlock access to the data. Each private key can be unlocked by the master private key controlled by the user's blockchain account.

Questions: What should this context be called?

In a traditional application you would think of a context as the name of an application (ie: Secure Markdown Editor). However in the new world of de-centralized applications there is a separation between the user interface and the storage of data. This means you could have multiple user interfaces (or applications) that interact with the same siloed collection of databases.

For example, three different applications could be developed that all share the same Social platform dataset (and schemas). Your data would be the same regardless of which application you logged into.

A developer intuitively would likely think of each context as an application name, however as mentioned above that's not really accurate and undermines the capabilities of new world of de-centralized applications.

3box called these spaces which makes sense, however the have recently dropped this due to perceived confusion from developers about what a space actually was.

In the new refactor I have called these secure contexts as it seemed the most fitting, but I'd like feedback from the team.

It's important to get a meaningful name as this is a new concept to most developers entering this world and we need to try and help them understand the concepts as quickly as possible.

@Dickson-Mwendia An explanation of this new concept should be included in our documentation.

Create react native release

Create a separate @verida/client-rn package and make necessary React Native changes

Support database helper methods to deal with losing internet connectivity

Add a resync() and isconnected() methods to database and datastore in client-ts.

Investigate auto-reconnection in the underlying pouchdb connections.

[account-web-vault] Handle default servers supplied by application

At the moment the default servers for a context are supplied by the Vault mobile application.

Need to update the auth flow so the defaults can be provided by the application itself.

Datastore change events need to respect schema

Only raise change events for rows that change for the current schema.

Update all demos to use client-ts

Update all demos to use the new Verida client-ts package:

Depends on #1

Update sandbox to use client-ts

Migrate profile manager

Migrate the profile manager from @verida/datatore into client-ts.

Give some thought into supporting IDX profiles for public profiles.

Support new datastore server changes

(GNaP https://github.com/interop-alliance/gnap-client-js, ZCaP, JWT auth)

See https://github.com/verida/datastore-server/issues/3

Support native 3ID account type

Support creating an account-3id natively, without linking any existing blockchain account.

Update to npm v14

Simplify context initialisation

Support a factory method that instantiates a context from a single configuration object.

This will be the primary way to use the library as it's the most common use case.

Inbox messaging between different applications appears broken

Test code in web-sandbox doesn't work.

Requires investigation.

FYI @pkhien95

Migrate to pbkdf2

Need to update dependencies to use pbkdf2 password hashing algorithm as it works across nodeJs, browser and react-native environments.

After a lot of investgation, there is no easy cross-environment solution exists for argon2id or scrypt.

Update crypto-pouch library

Update from using https://github.com/tahpot/crypto-pouch.git to the latest version of crypto-pouch (v4.0.1) this will remove some vulnerabilities in the elliptic package calvinmetcalf/crypto-pouch#76 that is currently being used.

Repositories requiring the change

package.json

yarn.lock

verida/verida-js

Support authenticating multiple application contexts within the same app (for connecting to remote third party app inboxes)
#2
Support multiple, injectable authentication strategies (server side, qr code, metamask, key in a cookie)
Migrate existing @verida/datastore library into new Typescript refactor library (@verida/network-ts)
Support swappable storage engines
Rename @verida/network-ts to @verida/client-ts

verida / verida-js Goto Github PK

verida-js's Introduction

Verida Javascript Library

Packages

Developer Notes

Node Version

Linking dependencies

Building

Creating a release

verida-js's People

Contributors

Stargazers

Watchers

Forkers

verida-js's Issues

Verida schemas

We need a Verida meta-schema

We need a plan for schema versioning

Schema security

Community resources

Current architecture

Ceramic architecture

Why Ceramic?

Risks to eliminate

Task

Recommend Projects

Recommend Topics

Recommend Org