peggyjv/ocular

Ocular

Ocular seeks to be the preferred cosmos client library UX for Rust projects. It is strongly based on Lens, a go client library for blockchains built with the Cosmos SDK.

Features

Existing

• Automatic client configuration based on the cosmos chain registry:

let client = ChainClient::new("cosmoshub");

In Progress

• Core module querys
• Tendermint querys
• Key management
• TX signing with the familiar cosmos Accounts paradigm
• Automatic IBC relayer path retrieval

Future

• Arbitrary gRPC message sending (for custom modules)

ocular's Issues

Ocular_CLI (Add Chain)

Create a command to add a new chain to ocular

Ocular_CLI (List Chain)

List all local chains, with the list command.

Ocular_CLI (Registry_list Chain)

List all chains in the registry with the registry_list command. The purpose of this, is to view all possible chain names

Create TX design doc

Test coverage

We need unit test coverage on methods that do not require network calls and more comprehensive integration test flows for various code paths.

1.0.0 Release candidate criterea discussion

Before release?:

• Reliable gRPC endpoint selection when using automatic client construction
• Easy Account reference construction from key
• Auth, Authz and Bank module txs and queries
• Automatic and Manual client construction paths
• Airdrop functionality
  • Rigorous testing
• Unit and integration test coverage

Finish chain config

Sketch of library requirements for Hermes operator user-story

This sketches part of the requirements on the ocular library.

The main persona here is the relayer operator. I'll assume the perspective of Hermes relayer, since ocular is a Rust library and Hermes will most likely be a representative user of ocular.

There are two stories that come to mind and & would be useful to have.

1. Generate the relayer's configuration file starting from a tag

This story starts with a CLI that handles initialization of the relayer's config.toml starting from one TAG. The operator would invoke a CLI as follows:

hermes config gen TAG

This would generate a config.toml for Hermes with all the chains and all the paths from the chain registry that match the given TAG. We can define "match" to mean either a substr/regex or exact string equality, not sure yet, depending on the tag type definition & content from the chain-registry.

Ocular requirement A

This use-case translates into an ocular requirement for an API roughly as follows:

fn get_paths_filtered(tag_filter: String) -> Vec<Path>

I imagine calling this method from the relayer (while executing hermes config x) to result into a vector of Paths which can be JsonValue or similar types, filtered by their tags.

Of course, transforming these Paths into their appropriate representation and generating the config.toml will be done on the Hermes side. To do so, we need one more API from ocular, documented below.

Ocular requirement B

To be able to fully generate the relayer's config.toml, we need additional pieces of information for each chain from the chain registry. The full set of details are as follows:

• chain_id: this is already part of each Path object, we have it already
• rpc_addr, grpc_addr websocket_addr: needed, should be part of the chain registry (or added there)
• account_prefix: this is present in the chain registry already
  • example: https://github.com/cosmos/chain-registry/blob/256f368f119605a833f77511de3dc186d9579af1/akash/chain.json#L8
• key_name: this could be handled by ocular directly (if ocular has a keystore)
• store_prefix is typically 'ibc': we can keep this hardcoded (not needed)
• denom: present in chain registry already
  • https://github.com/cosmos/chain-registry/blob/256f368f119605a833f77511de3dc186d9579af1/akash/chain.json#L9
• consensus parameter max_block_size, we need this to compute the max_tx_size that Hermes is allowed to submit to the chain
  • this can be fetched from a full node's RPC via /consensus_params endpoint, the interesting field is called consensus_params.block.max_bytes.
• clock_drift: needed, does not exist in chain registry, should be added
• max_block_time: needed, does not exist in chain registry, should be added

To summarize, in an ideal world ocular would provide an API as follows:

fn chain_info(chain_id: String) -> ChainInfo

Where ChainInfo would comprise the corresponding attributes to describe the necessary fields. To recap: rpc_addr, grpc_addr websocket_addr, account_prefix, denom, max_block_size, clock_drift, max_block_time, and optionally key_name if ocular supports key management. None of these are a hard requirement. However, the fewer of these fields are present, the less useful the API will be.

2. Append IBC paths to the relayer's config.toml

In this story, the config.toml is already pre-populated with most of the chain information (i.e., we don't need to call into chain_info). The relayer operator is only interested in appending path information to each section of the configuration file. Path information in Hermes consists of a simple list (example, example) of channel identifiers, but is subject to change. In the future it will comprise connection and client also. Suppose this information is missing from config.toml and we'd want to add it automatically from chain registry.

The top-level relayer's CLI would look as follows:

hermes config paths

Then Hermes would call into ocular to fetch path information for every pair of chains present in the config.toml. Suppose config.toml comprises three chains A, B, C, then Hermes would call into ocular to fetch three paths: A <> B, B <> C, A <> C. To do so, would be great if ocular exposed the following API:

Ocular requirement C

fn get_paths_pairwise(source_chain: ChainId, dest_chain: ChainId) -> Vec<Path>

The meaning is pretty straightforward. This should return a Path such as this one, which may be a vector, but not sure.

Manual client configuration path

Provide a means of creating a client with configurable settings, bypassing the chain registry.

Simplify UX of creating a client

Users should be able to simply pass a name or enum to the ChainClient constructor. The ChainInfo and ChainClientConfig creation necessary should be abstracted away.

Ex:

let client = ChainClient::new("osmosis").unwrap();

I should have already implemented this but forgot 😮‍💨

Retrieve relayer paths pairwise

Covers the final API requirement of #8

Something like

fn get_paths_pairwise(source_chain: ChainId, dest_chain: ChainId) -> Vec<Path>

Ocular_CLI (Show Chain)

Show details of a chain in ocular with the show command

Batch tx support for toml input

Something in the format of

[sender]
source_account = “xyz”
...

[transaction]
  [transaction.details]
  name = “xyz”
  destination_account = “xyz”
  amount = 1
  ...

This issue should include toml parsing, tx sending support, unit testing, and integration testing.

REST API for cached chain registry

We need either a feature or separate ocular crate in the workspace for hosting a cache of the chain registry with a RESTful API for ocular to query chain and IBC path information. This should be optional as a user may opt to manually configure their chain client and/or read in IBC path data some other way.

Ocular_CLI (Delete Chain)

Create a command to delete ocular chain

Add in grpc caching mechanisms

1.) Should support ability to read and write from file based cache
2.) Option for in ephemeral in-memory caching
3.) Option to have no caching whatsoever

This PR should be followed by #23 to actually utilize the cache. This is split into 2 issues to separate actual cache implementation and utilization bc the latter will require a bit of a refractor.

Doc comments

Use improved grpc endpoint retrieval in client construction

Add in release tests for each chain in the registry

Try every chain and asset chain json in cosmos/chain-registry so that each version of ocular that is deployed is guaranteed to function correctly when released.

Ocular_CLI (Acceptance Test)

There needs to be an acceptance test for all commands in the ocular CLI. This is to test that all commands work as expected. An example can be found in the Steward repository:

https://github.com/PeggyJV/steward/blob/main/steward/tests/acceptance.rs

Complete list of tx's from parsed toml using authz tx delegation

Msg type URL public constants

Provide Msg type URLs as public constants for the core SDK modules in the ocular::tx::<module> modules respectively

Airdrop Testing

• Integration tests for full airdropping flow, delegated and undelegated
• Integration tests for individual methods where relevant
• Unit tests of relevant methods
• Mainnet test with real funds

AuthZ support (to delegate and perform tx's)

• create & sign tx's
  • Cosmrs should have this support (cosmos/cosmos-rust#177) as of v0.5
• delegate and undelegate tokens
• Should include some integration testing (e.g. https://github.com/cosmos/cosmos-rust/blob/main/cosmrs/tests/integration.rs#L35-L101)

Include additional chain info for relayers

Covers 1B of #8

Fields not included in the chain registry chain.json:

• websocket_addr
• account_prefix
• key_name (covered by key management PR)
• max_block_size
• clock_drift
• max_block_time

Append these to the current ChainInfo struct or create a superstruct?

Ocular_CLI (Set_default Chain)

Users can change the default chain that comes with Ocular. To do this, they'll use the set_default command. This way, they can change the default to the chain of their choice.

Basic module query clients and queries

Implement the query logic that exists in lens and also expose the different modules' query clients in case users want more fine-tuned queries (with pagination for example)

• Bank query client and queries
• Other module query clients
• Test coverage

Create deep_space feature parity checklist

Ocular_CLI (Edit Chain)

Create a command to edit ocular chain

Update batch tx's to use MultiSend instead of single send

Add a more flexible grpc call wrapper

Per #22 (comment), we should add in a wrapper around gRPC connections/calls that:

1.) Adds in an ability to do optional backoff retries
2.) Keeps track of healthy gRPC endpoints on the fly and update cache

This PR should be done after #31. This is split into 2 issues to separate actual cache implementation and utilization bc the latter will require a bit of a refractor.

This PR should also extend the cache hashset into a hashmap with values indicating how many times in a row connection attempts have failed and remove endpoints that have failed more than some threshold value

Re-export/wrap types to prevent dependency "leaks"

We don't want users to have to include cosmrs or other crates that ocular depends on in order for it to work.

Account management

There should be some sensible harmony of key/account management. Accounts will be used for transactions and the lower-level key management for just that.

Accounts are just abstractions on top of keys; therefore "creating" an account is just an abstraction of generating and saving a new private key, or deriving a new address from an existing public key.
Account should be able to yield addresses with any prefix provided, with a go-to method defaulting to "cosmos".

Need to look into supporting module accounts as well.

Add Tendermint RPC client and necessary query convenience methods

The equivalent in lens would be this field and this method.

This will also include inclusion in the ChainClient struct and instantiation.

Think through caching config

It's been unofficially decided ocular doesn't really need a cli because it's redundant with lens. However, one really useful thing about the lens cli is that it caches chain registry config data locally. Ocular should probably have this capability in case the registry becomes inaccessible such as during a Github outage.

Ocular could cache config in a local file when a chain client is first constructed so that future construction of clients for that chain wouldn't need to make calls to the registry. We may need to think about some kind of policy around making sure the config doesn't get stale like if endpoints are updated in the registry. Alternatively we could leave this to the user. Perhaps some kind of way to configure the behavior at client construction time (a method like ChainClient::new_with_options()?). This issue for starting a discussion.

Ocular_CLI (Chain Editor)

This command opens ocular configuration in an editor.

Ocular_CLI (Show_default Chain)

Command to show details of the default chain in ocular. Ocular comes with some default chains, you can check the default chain with the show-default command

Batch tx support from toml input

Process delegated tx's from a toml file contingent upon a valid grant existing for a grantee account from a specified granter.

Toml should look something like

[sender]
grantee_account = “xyz”...
granter_account = “xyz”...
...

[transaction]
  [transaction.details]
  name = “xyz”
  destination_account = “xyz”
  amount = 1
  ...

This issue should include toml parsing, tx sending support, unit testing, and integration testing.

Retrieve relayer paths info given a tag

Covers 1A of #8

Implement abstract key manager/key ring equivalent with Filesystem key store

This feature should implement an abstract keyring as well as an implementation for the Filesystem key store

Control cosmos/chain-registry version pulls

While running cargo tests I saw results such as:

---- chain::client::query::tests::gets_bank_client stdout ----
thread 'chain::client::query::tests::gets_bank_client' panicked at 'Assay test failed', ocular/src/chain/client/query.rs:158:5

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ChainRegistry(InvalidChainInfo(Error("expected `,` or `}`", line: 196, column: 17)))', ocular/src/chain/client/query.rs:160:57
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

On a deeper dive we can see this is caused by a missing comma at cosmos/chain-registry@4d03592

We should consider pinning commit versions in our chain registry pull at https://github.com/PeggyJV/ocular/blob/main/ocular/src/chain/registry.rs#L68-L74 to prevent main branch regressions in cosmos/chain-registry from affecting ocular.

Per conversing with Collin, we should additionally add in tests that try every chain and asset chain json in cosmos/chain-registry so that each version of ocular that is deployed is guaranteed to function correctly when released.