Contract Spec: Add macros that generates a cross-contract calling impl

[leighmcculloch]

Contract Spec: Add macros that generates a cross-contract calling impl

[leighmcculloch]

To add more details to this because conversation is starting about it (stellar/soroban-examples#29):

The intent is to have a macro that generates all the code for cross contract calls that includes the ability to test the cross contract calls locally. Ideally the macro would accept as inputs:

• A contract ID, which is used at runtime to invoke the tests.
• Optionally, a WASM file path, that the macro reads the contract spec out of for generating the appropriate trait and implementation that can be used to invoke.
• Optionally, a Horizon or similar URL for where the WASM file could be retrieved instead of being a local path. (This is likely a follow up feature when we have Horizon support.)

Ideally it should be as simple as:

contractuse!(TheirContract, id = "0909234890482304", wasm = "path/to/file.wasm");

#[test]
fn test_cross_contract_call() {
    let env = Env::default();
    env.register_contract(..., TheirContract);
    env.register_contract(..., MyContract);
    // ...
}

The id field is optional and overridable, so that it could be provided at runtime.

[leighmcculloch]

Oh I forgot, there was also a second macro @paulbellamy and I had talked about similar to the above one, where instead of being from a WASM it was from a trait, so we need both:

#[contractuse(id = "0909234890482304")]
pub trait TheirContract {
    // ...
}

The id field is optional and overridable, so that it could be provided at runtime.

[leighmcculloch]

In the examples I posted above we should probably omit the contract ID, and instead accept it as a parameter in every invocation, so that it is easier to programmatically use a single contract definition with multiple deployments, or in the case of a standard interface, multiple contracts.

[graydon]

yeah something I'm unclear on here is the higher-order part -- how the user is supposed to declare a function signature that takes a contract ID that has to conform to some interface. Is it just like the recipient says "you give me a contract ID and I will cast it to interface Foo and use it as such, and it will fail if some part of the interface isn't supported when we try to call it"?

I expect people will want to accept, "X: some token contract" as an input, and it'd be cool if the provided contract ID was actually checked at some point before calling it that it implemented the token contract interface.

[leighmcculloch]

I imagine contract IDs to never be validated. There's the contract definition, and the contract ID, and they can be combined in any way, and if you call a contract with the wrong definition it will probably, or might at least, error.

I think checking a contract ID adheres to a specific interface is something the host could provide as a host function, where the host function goes and gets the contract, gets its spec, and checks if the spec matches the expected spec. But we probably don't need this in the near term, and it can probably be bolted on as additional verification step that's available if a developer needs it.

[leighmcculloch]

This is changing shape a little. The will take a contract spec (either as xdr, or as a wasm file) and generate:

• the user-defined types in the spec
• a trait for the functions in the spec
• a client cross contract caller implementation for the trait that calls env.invoke_contract with the parameters
• and, if the wasm is provided, it will provide a way (undefined right now) to register that wasm using the existing register contract test capabilities so that it can be easily registered in tests

I hope, but not sure if I can deliver it, that this ☝🏻 would be easier to use that hunting down the code for a contract and importing it as a Rust lib and fiddling with export features, etc.

At the moment the goal is for the following macro call, with the add test contract, to generate something like:

contractuse!(
    name = "add",
    spec = "AAAAAAAAAANhZGQAAAAAAgAAAAIAAAACAAAAAQAAAAI=",
    // or wasm = "add.wasm",
    client = true,
);

mod add {
    // Contents of wasm file if provided.
    pub FILE: &[u8] = [...];

    // Any user-defined types would be generated here.
    
    #[contractclient(name = "ContractClient")]
    pub trait Contract {
        fn add(env: Env, a: u32, b: u32) -> u32;
    }
}

mod add {
    pub struct Client;
    impl Contract for Client {
        fn add(env: Env, contract_id: FixedBinary<32>, a: u32, b: u32) -> Result<u32, ConversionError> {
            const FN: Symbol = Symbol::from_str("add");
            let args = vec![&env, a.into_env_val(&env), b.into_env_val(&env)];
            env.invoke_contract(&contract_id, &FN, args).try_into()
        }
    }
}

The client would be optional, so you could use this to generate a trait for a contract interface you want to implement.

I’m a little concerned that while a macro is convenient, it will suffer from a developer not being able to see 👀 the trait right in front of them, which is why the contract client is generated from another attribute macro. The idea being we have the option to have the soroban-cli code gen the trait from a contract spec which someone could copy and paste into their code.

I'm thinking of generating everything inside a mod because it is the easiest way to prevent name collisions for user-defined types.

Without doing anything else the contract could be tested by doing the following.

#[test]
fn test_cross_contract_call() {
    let env = Env::default();
    env.register_contract_file(..., add::FILE);
    env.register_contract(..., MyContract);
    // ...
}

[leighmcculloch]

Current state of #411 and stellar/stellar-cli#74 is below.

Some things of note:

• Significant oversight that I left function parameter names out of the spec, oops! Will address in #434.
• I'm leaning into generating clients using another attribute. The CLI will generate the below, or, the below will be generatable via a contractuse!(...). Tbh I'm not sure which will create the best ergonomics, so I'm going to provide both, and we can measure.
• I'm not sure what to do with the contract name, as it isn't stored in the spec, and I don't think it should be... at least I'm not convinced yet. I'm just using the name Contract, and the generated client will be Client.
• Unclear what the contractwasm macro will do yet, but I think it will embed the spec into a const and the WASM file into const which the user can use to register the contract in tests, or deploy it on chain. So I think it would also be useful add some functions for deploying it, and registering it for tests.

❯ soroban-cli codegen --wasm ../rs-soroban-sdk/target/wasm32-unknown-unknown/release/example_udt.wasm
// File: ../rs-soroban-sdk/target/wasm32-unknown-unknown/release/example_udt.wasm
// Contract Spec: AAAAAgAAAAdVZHRFbnVtAAAAAAIAAAAEVWR0QQAAAAAAAAAEVWR0QgAAAAEAAAfQAAAACVVkdFN0cnVjdAAAAAAAAAEAAAAJVWR0U3RydWN0AAAAAAAAAwAAAAFhAAAAAAAABAAAAAFiAAAAAAAABAAAAAFjAAAAAAAD6gAAAAQAAAAAAAAAA2FkZAAAAAACAAAH0AAAAAdVZHRFbnVtAAAAB9AAAAAHVWR0RW51bQAAAAABAAAABA==
#[::soroban_sdk::contractclient]
pub trait Contract {
    fn add(env: ::soroban_sdk::Env, a0: UdtEnum, a1: UdtEnum) -> (i64);
}
#[::soroban_sdk::contracttype]
pub struct UdtStruct {
    pub a: i64,
    pub b: i64,
    pub c: ::soroban_sdk::Vec<i64>,
}
#[::soroban_sdk::contracttype]
pub enum UdtEnum {
    UdtA,
    UdtB(UdtStruct),
}

❯ soroban-cli codegen --wasm ../soroban-token-contract/target/wasm32-unknown-unknown/release/soroban_token_contract.wasm
// File: ../soroban-token-contract/target/wasm32-unknown-unknown/release/soroban_token_contract.wasm
// Contract Spec: AAAAAAAAAAppbml0aWFsaXplAAAAAAAEAAAH0AAAAApJZGVudGlmaWVyAAAAAAABAAAACQAAAAkAAAAAAAAAAAAAAAVub25jZQAAAAAAAAEAAAfQAAAACklkZW50aWZpZXIAAAAAAAEAAAAKAAAAAAAAAAlhbGxvd2FuY2UAAAAAAAACAAAH0AAAAApJZGVudGlmaWVyAAAAAAfQAAAACklkZW50aWZpZXIAAAAAAAEAAAAKAAAAAAAAAAdhcHByb3ZlAAAAAAMAAAfQAAAAEktleWVkQXV0aG9yaXphdGlvbgAAAAAH0AAAAApJZGVudGlmaWVyAAAAAAAKAAAAAAAAAAAAAAAHYmFsYW5jZQAAAAABAAAH0AAAAApJZGVudGlmaWVyAAAAAAABAAAACgAAAAAAAAAJaXNfZnJvemVuAAAAAAAAAQAAB9AAAAAKSWRlbnRpZmllcgAAAAAAAQAAAAUAAAAAAAAABHhmZXIAAAADAAAH0AAAABJLZXllZEF1dGhvcml6YXRpb24AAAAAB9AAAAAKSWRlbnRpZmllcgAAAAAACgAAAAAAAAAAAAAACXhmZXJfZnJvbQAAAAAAAAQAAAfQAAAAEktleWVkQXV0aG9yaXphdGlvbgAAAAAH0AAAAApJZGVudGlmaWVyAAAAAAfQAAAACklkZW50aWZpZXIAAAAAAAoAAAAAAAAAAAAAAARidXJuAAAAAwAAB9AAAAANQXV0aG9yaXphdGlvbgAAAAAAB9AAAAAKSWRlbnRpZmllcgAAAAAACgAAAAAAAAAAAAAABmZyZWV6ZQAAAAAAAgAAB9AAAAANQXV0aG9yaXphdGlvbgAAAAAAB9AAAAAKSWRlbnRpZmllcgAAAAAAAAAAAAAAAAAEbWludAAAAAMAAAfQAAAADUF1dGhvcml6YXRpb24AAAAAAAfQAAAACklkZW50aWZpZXIAAAAAAAoAAAAAAAAAAAAAAAlzZXRfYWRtaW4AAAAAAAACAAAH0AAAAA1BdXRob3JpemF0aW9uAAAAAAAH0AAAAApJZGVudGlmaWVyAAAAAAAAAAAAAAAAAAh1bmZyZWV6ZQAAAAIAAAfQAAAADUF1dGhvcml6YXRpb24AAAAAAAfQAAAACklkZW50aWZpZXIAAAAAAAAAAAAAAAAACGRlY2ltYWxzAAAAAAAAAAEAAAABAAAAAAAAAARuYW1lAAAAAAAAAAEAAAAJAAAAAAAAAAZzeW1ib2wAAAAAAAAAAAABAAAACQAAAAEAAAAVS2V5ZWRFZDI1NTE5U2lnbmF0dXJlAAAAAAAAAgAAAApwdWJsaWNfa2V5AAAAAAfQAAAABFUyNTYAAAAJc2lnbmF0dXJlAAAAAAAH0AAAAARVNTEyAAAAAQAAABlLZXllZEFjY291bnRBdXRob3JpemF0aW9uAAAAAAAAAgAAAApwdWJsaWNfa2V5AAAAAAfQAAAABFUyNTYAAAAKc2lnbmF0dXJlcwAAAAAH0AAAABRBY2NvdW50QXV0aG9yaXphdGlvbgAAAAIAAAANQXV0aG9yaXphdGlvbgAAAAAAAAMAAAAIQ29udHJhY3QAAAAAAAAAB0VkMjU1MTkAAAAAAQAAB9AAAAAEVTUxMgAAAAdBY2NvdW50AAAAAAEAAAfQAAAAFEFjY291bnRBdXRob3JpemF0aW9uAAAAAgAAABJLZXllZEF1dGhvcml6YXRpb24AAAAAAAMAAAAIQ29udHJhY3QAAAAAAAAAB0VkMjU1MTkAAAAAAQAAB9AAAAAVS2V5ZWRFZDI1NTE5U2lnbmF0dXJlAAAAAAAAB0FjY291bnQAAAAAAQAAB9AAAAAZS2V5ZWRBY2NvdW50QXV0aG9yaXphdGlvbgAAAAAAAAIAAAAKSWRlbnRpZmllcgAAAAAAAwAAAAhDb250cmFjdAAAAAEAAAfQAAAABFUyNTYAAAAHRWQyNTUxOQAAAAABAAAH0AAAAARVMjU2AAAAB0FjY291bnQAAAAAAQAAB9AAAAAEVTI1NgAAAAEAAAAJTWVzc2FnZVYwAAAAAAAAAwAAAAVub25jZQAAAAAAAAoAAAAGZG9tYWluAAAAAAABAAAACnBhcmFtZXRlcnMAAAAAA+oAAAfQAAAABkVudlZhbAAAAAAAAgAAAAdNZXNzYWdlAAAAAAEAAAACVjAAAAAAAAEAAAfQAAAACU1lc3NhZ2VWMAAAAAAAAAEAAAAQQWxsb3dhbmNlRGF0YUtleQAAAAIAAAAEZnJvbQAAB9AAAAAKSWRlbnRpZmllcgAAAAAAB3NwZW5kZXIAAAAH0AAAAApJZGVudGlmaWVyAAAAAAACAAAAB0RhdGFLZXkAAAAACAAAAAlBbGxvd2FuY2UAAAAAAAABAAAH0AAAABBBbGxvd2FuY2VEYXRhS2V5AAAAB0JhbGFuY2UAAAAAAQAAB9AAAAAKSWRlbnRpZmllcgAAAAAABU5vbmNlAAAAAAAAAQAAB9AAAAAKSWRlbnRpZmllcgAAAAAABVN0YXRlAAAAAAAAAQAAB9AAAAAKSWRlbnRpZmllcgAAAAAABUFkbWluAAAAAAAAAAAAAAhEZWNpbWFscwAAAAAAAAAETmFtZQAAAAAAAAAGU3ltYm9sAAAAAAAA
#[::soroban_sdk::contractclient]
pub trait Contract {
    fn initialize(
        env: ::soroban_sdk::Env,
        a0: Identifier,
        a1: u32,
        a2: ::soroban_sdk::Bytes,
        a3: ::soroban_sdk::Bytes,
    ) -> ();
    fn nonce(env: ::soroban_sdk::Env, a0: Identifier) -> (::soroban_sdk::BigInt);
    fn allowance(
        env: ::soroban_sdk::Env,
        a0: Identifier,
        a1: Identifier,
    ) -> (::soroban_sdk::BigInt);
    fn approve(
        env: ::soroban_sdk::Env,
        a0: KeyedAuthorization,
        a1: Identifier,
        a2: ::soroban_sdk::BigInt,
    ) -> ();
    fn balance(env: ::soroban_sdk::Env, a0: Identifier) -> (::soroban_sdk::BigInt);
    fn is_frozen(env: ::soroban_sdk::Env, a0: Identifier) -> (bool);
    fn xfer(
        env: ::soroban_sdk::Env,
        a0: KeyedAuthorization,
        a1: Identifier,
        a2: ::soroban_sdk::BigInt,
    ) -> ();
    fn xfer_from(
        env: ::soroban_sdk::Env,
        a0: KeyedAuthorization,
        a1: Identifier,
        a2: Identifier,
        a3: ::soroban_sdk::BigInt,
    ) -> ();
    fn burn(
        env: ::soroban_sdk::Env,
        a0: Authorization,
        a1: Identifier,
        a2: ::soroban_sdk::BigInt,
    ) -> ();
    fn freeze(env: ::soroban_sdk::Env, a0: Authorization, a1: Identifier) -> ();
    fn mint(
        env: ::soroban_sdk::Env,
        a0: Authorization,
        a1: Identifier,
        a2: ::soroban_sdk::BigInt,
    ) -> ();
    fn set_admin(env: ::soroban_sdk::Env, a0: Authorization, a1: Identifier) -> ();
    fn unfreeze(env: ::soroban_sdk::Env, a0: Authorization, a1: Identifier) -> ();
    fn decimals(env: ::soroban_sdk::Env) -> (u32);
    fn name(env: ::soroban_sdk::Env) -> (::soroban_sdk::Bytes);
    fn symbol(env: ::soroban_sdk::Env) -> (::soroban_sdk::Bytes);
}
#[::soroban_sdk::contracttype]
pub struct KeyedEd25519Signature {
    pub public_key: U256,
    pub signature: U512,
}
#[::soroban_sdk::contracttype]
pub struct KeyedAccountAuthorization {
    pub public_key: U256,
    pub signatures: AccountAuthorization,
}
#[::soroban_sdk::contracttype]
pub struct MessageV0 {
    pub nonce: ::soroban_sdk::BigInt,
    pub domain: u32,
    pub parameters: ::soroban_sdk::Vec<EnvVal>,
}
#[::soroban_sdk::contracttype]
pub struct AllowanceDataKey {
    pub from: Identifier,
    pub spender: Identifier,
}
#[::soroban_sdk::contracttype]
pub enum Authorization {
    Contract,
    Ed25519(U512),
    Account(AccountAuthorization),
}
#[::soroban_sdk::contracttype]
pub enum KeyedAuthorization {
    Contract,
    Ed25519(KeyedEd25519Signature),
    Account(KeyedAccountAuthorization),
}
#[::soroban_sdk::contracttype]
pub enum Identifier {
    Contract(U256),
    Ed25519(U256),
    Account(U256),
}
#[::soroban_sdk::contracttype]
pub enum Message {
    V0(MessageV0),
}
#[::soroban_sdk::contracttype]
pub enum DataKey {
    Allowance(AllowanceDataKey),
    Balance(Identifier),
    Nonce(Identifier),
    State(Identifier),
    Admin,
    Decimals,
    Name,
    Symbol,
}

[leighmcculloch]

I've got the contractimport! macro working now. It's use looks like this:

rs-soroban-sdk/tests/import_contract/src/lib.rs

Lines 5 to 9 in 32c6f0a

	mod addcontract {
	soroban_sdk::contractimport!(
	wasm = "target/wasm32-unknown-unknown/release/example_add_i32.wasm"
	);
	}

The import generates a client that can be used to invoke a contract, like so:

rs-soroban-sdk/tests/import_contract/src/lib.rs

Lines 13 to 18 in 32c6f0a

	#[contractimpl]
	impl Contract {
	pub fn add_with(env: Env, x: i32, y: i32) -> i32 {
	addcontract::Client::add(&env, &BytesN::from_array(&env, ADD_CONTRACT_ID), x, y)
	}
	}

Registration of the contract in tests is still simple:

rs-soroban-sdk/tests/import_contract/src/lib.rs

Lines 28 to 39 in 32c6f0a

	let e = Env::default();
	let add_contract_id = BytesN::from_array(&e, ADD_CONTRACT_ID);
	e.register_contract_wasm(&add_contract_id, addcontract::WASM);
	let contract_id = BytesN::from_array(&e, [1; 32]);
	e.register_contract(&contract_id, Contract);
	let x = 10i32;
	let y = 12i32;
	let z = add_with::invoke(&e, &contract_id, &x, &y);
	assert!(z == 22);

The Client generated is rather simple:

pub struct Client;
impl Client {
    pub fn add(
        env: &::soroban_sdk::Env,
        contract_id: &::soroban_sdk::BytesN<32>,
        arg_0: i32,
        arg_1: i32,
    ) -> (i32) {
        use ::soroban_sdk::IntoVal;
        const FN_SYMBOL: ::soroban_sdk::Symbol = ::soroban_sdk::Symbol::from_str("add");
        env.invoke_contract(
            contract_id,
            &FN_SYMBOL,
            ::soroban_sdk::Vec::from_array(
                env,
                [arg_0.into_env_val(&env), arg_1.into_env_val(&env)],
            ),
        )
    }
}

The soroban-cli generates the following that interacts with the above:

❯ soroban-cli codegen --wasm ../rs-soroban-sdk/target/wasm32-unknown-unknown/release/example_add_i32.wasm
#[::soroban_sdk::contractfile(
    file = "../rs-soroban-sdk/target/wasm32-unknown-unknown/release/example_add_i32.wasm",
    sha256 = "58f98f171987ffff001c4c7f15357bb2c2a4870ec9294c28a401d6c0143ed0e5"
)]
#[::soroban_sdk::contractclient(name = "Client")]
pub trait Contract {
    fn add(env: ::soroban_sdk::Env, a0: i32, a1: i32) -> (i32);
}

cc @jonjove @paulbellamy

---

Much like the soroban-cli codegen subcommand the trait is still by the contractimport! macro:

pub trait Contract {
    fn add(env: ::soroban_sdk::Env, a0: i32, a1: i32) -> (i32);
}

The trait can be implemented by someone who wants to import a contract just to reimplement the interface, but this presents a problem for anyone who is not using an IDE that supports macro expansion, or for anyone whose IDE frequently errors on macro expansion. The latter problem happens to everyone afaik from time to time.

So I suspect there will be two workflows:

1. Dev imports .WASM file with the intent to invoke the Client, and uses contractimport!.

2. Dev wishes to implement some common standard, and either copy-pastes the trait from the standard document, or copy-pastes the spec from the standard document and generates the trait using the CLI.

I had hoped the second case could also be addressed by contractimport! but the macro makes it all a bit too opaque.

[leighmcculloch]

@jonjove @paulbellamy I'd be interested in hearing your thoughts on the above, and if it would be better for us to:

1. Encourage use of contractimport! as is. i.e. You import a .wasm file with a single line, and everything is at hand. e.g.: This is what will be in peoples code:

   soroban_sdk::contractimport!(file = "target/wasm32-unknown-unknown/release/example_add_i32.wasm"); 

2. Or, encourage use of soroban-cli codegen to generate a trait, types, and then users will copy and paste, or pipe it to a file. e.g. This is what will be in peoples code:

   #[::soroban_sdk::contractfile(
       file = "target/wasm32-unknown-unknown/release/example_add_i32.wasm",
       sha256 = "58f98f171987ffff001c4c7f15357bb2c2a4870ec9294c28a401d6c0143ed0e5"
   )]
   #[::soroban_sdk::contractclient(name = "Client")]
   pub trait Contract {
       fn add(env: ::soroban_sdk::Env, a0: i32, a1: i32) -> (i32);
   }
   #[::soroban_sdk::contracttype]
   pub struct MyUdt {
      ...

(1) is more magical, and relies on no external tools to build and compile the code. But it is so magical you don't really see any hint of what's been generated. (2) gives you an in-your-face view of what's been generated, at least a little more, but is much more verbose. You technically still can't see the Client code, so you still need to rely on your IDE auto-completing the macro created type.

[leighmcculloch]

One way we could address the opaqueness of (1) is to include in our documentation an example of how to view the generated code using cargo-expand. The instructions would involve installing cargo-expand, and then demonstrating how to run a command to expand the macro. For the addcontract example, it'd be: cargo +nightly expand addcontract.

[leighmcculloch]

Another way we could address the opaqueness of (1) is by demonstrating how to generate docs for your code, which produces an outline of what functions and types have been generated. For example, running cargo doc --document-private-items --open will produce:

[jonjove]

My guess is that people will probably prefer the first "magical" option because it's extremely easy to get set up. Providing tooling and instructions to help people get the underlying implementation is probably the important thing to make that work.

In the short run, supporting both like we did with contractfn and contractimpl might be a reasonable approach to see which approach feels better for us and other early users. If it's not too much work for you, of course.

[leighmcculloch]

(Reminder to ignore for the moment that the args are named arg_0, arg_1, etc. This will be addressed in #434.)

[leighmcculloch]

It's not too much work to support both. Both are backed by the same code. The code generation is separated from the macro library, so we can import the code generation into other things like the CLI. I think we will always want the codegen in the CLI even if we keep contractimport!, because I think if someone is writing a SEP for a particular contract interface / standard, they'll want a way to output a text version of the contract interface for including in the doc. In that situation people may end up copying and pasting a thing rather than using a .wasm file. There's probably some wonky things about that specific flow still.