Add methods to check and generate timestamp tetraplets

It could be useful to have methods in SDK to generate correct tetraplets (or call parameters) to use in tests like this and to check in services like this

Investigate ways to enable IDE autocomplete process for generated code

Problem

People complain that they don't have code completion when writing tests with marine_test macro. Can we do something to enable auto-completion in IDE? The answer is yes.

Possible solution

Right now code is generated by procedural macros. I didn't find methods to enable auto-completion from them, but we can use a different approach: generate definitions in cargo build scripts. CLion and VS Code support code completion for names from files generated by cargo build scripts.

Cargo build scripts

This is typically a build.rs file on the same level as Config.toml and the src folder. It executes before build and it can write files in the directory specified in the OUT_DIR environment variable, read files anywhere. If the build script generated a file, say, some_generated_file.rs in OUT_DIR, it can be included in the project by writing in some file the following line:

include!(concat!(env!("OUT_DIR"), "/some_generated_file.rs"));

Then code in this file will have access to any names defined in some_generated_file.rs. But there is some more to do to enable auto-completion for names from some_generated_file.rs:

CLion

in the Help -> Actions -> Experimental Futures enable org.rust.cargo.evaluate.build.scripts
refresh cargo project in order to update generated code: change Cargo.toml and build from IDE or press Refresh Cargo Project in Cargo tab.

VS Code

install rust-analyzer plugin
change Cargo.toml to let plugin update code from generated files

What to do

find a good way to use build scripts for generating definitions
discuss it
implement it

marine test : allow to control service lifecycle

Description

Declare 2 services form same wasm hash, so different file paths (my-service-1.wasm and my-service-2.wasm):

[[module]]
    name = "my-service-1"
    mem_pages_count = 1
    logger_enabled = true    
[[module]]
    name = "my-service-2"
    mem_pages_count = 1
    logger_enabled = true

Will get in code:

tests::flow::my_service_1_structs::__m_generated_my_service_1::MGeneratedStructmy_service_1
tests::flow::my_service_2_structs::__m_generated_my_service_2::MGeneratedStructmy_service_2

Actual

Same wasm, 2 different Rust structs.

Expected

Can instance same module API 2 times which share same Rust types.

Solution:

[[module]]
    name = "my-service"
[[service]]
    name = "my-service_1"
    module = "my-service"
    mem_pages_count = 1
    logger_enabled = true
[[service]]
    name = "my-service_2"
    module = "my-service"
    mem_pages_count = 1
    logger_enabled = true

In code

let mut s =my_service_1::new();
s =my_service_2::new();

Why?

I need test which checks replication logic.

marine_test: report path in 'Wasm file is corrupted'

Currently, if I make a mistake with file paths somewhere, I get the following error:

error: Can't load Wasm modules into Marine: provided Wasm file is corrupted: No such file or directory (os error 2)
   --> src/tests.rs:257:19
    |
257 |     #[marine_test(config_path = "../Config.toml", modules_dir = "../artifacts/")]
    |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

However, it's hard to tell what file is missing, and what path is assumed.

It would be great if the error included the path that is missing/corrupted.

marine_test: implement service integration testing

Problem

Testing services integration is hard. Right now it is necessary to write Aqua code, deploy services to some network and somehow interpret results. There are also user's requests: fluencelabs/marine-rs-sdk#55, #3.

Solution

We plan to add new functionality to the sdk-test. We determined some points:

reuse marine_test instead of adding new macros: change behaviour by using it with different arguments and on mod or fn. It makes coding and supporting easier.
use the simplest interface to the facade module, and module field to access the internal interface
deprecate current marine_test behaviour if new versions will cover its functionality with the same ease of use
link interface structs between services to avoid unnecessary transformation code
decide later if it is crucial to not ban names ServiceInterface and ModuleInterface for structs in modules

Interface

We came up with two new use cases for marine_test macro in addition to one existing:
ver1 — current behaviour.

#[marine_test(config_path = "../Config.toml", modules_dir = "../artifacts")]
fn empty_string(module1: marine_test_env::module1::ModuleInterface) {
    let data = marine_test_env::module1::Data {/*...*/}
    let res = module1.function(data);
    assert!(res);
}

ver2 — take several services, apply to a fn . Allows creating services by hand.

#[marine_test(
  service1(config_path = "../service1/Config.toml", modules_dir = "../service1/artifacts"),
  service2(config_path = "../service2/Config.toml", modules_dir = "../service2/artifacts"),
)]
fn my_test() {
    let service1 = marine_test_env::service1::ServiceInterface::new();        // access to a service constructor
    let service2 = marine_test_env::service2::ServiceInterface::new();
    let start_data = marine_test_env::service1::modules::db_module::Entry {}; // access to an internal module structure
    service1.modules.db_module().put_data(start_data);                        // access to an internal module function

    let data = marine_test_env::service1::ProcessDataInput {};                // access to a facade module structure
    let res = service1.process_data(data);                                    // access to a facade module function
    let res = service2.process_service1_res(res);                             // types are linked between services (is it that clear?)
    assert!(res);
}

ver3 — take several services, apply to a mod. It allows writing utility functions and sharing state between tests.

#[marine_test(
  service1(config_path = "../service1/Config.toml", modules_dir = "../service1/artifacts"),
  service2(config_path = "../service2/Config.toml", modules_dir = "../service2/artifacts"),
)]
mod s1_s2_integration_tests {
    let service1 = marine_test_env::service1::ServiceInterface::new(); // needs to be thread-safe 
    let service2 = marine_test_env::service2::ServiceInterface::new();

    fn util_foo(service1: &mut marine_test_env::service1::ServiceInterface) {
        service1.foo()
    }

    #[test]
    fn test_bar() {
        let res = util_foo(&service1)
        assert!(service2.bar(res));
    }

    #[test]
    fn clear_test_bar() {
        let service1 = marine_test_env::service1::ServiceInterface::new();
        let service2 = marine_test_env::service2::ServiceInterface::new();
        let res = util_foo(&service1)
        assert!(service2.bar(res));
    }
}

It is not discussed as we like ver2 and ver3 more than ver1, but it is possible to create ver1 applied to a mod.

Roadmap

This feature will be delivered incrementally, with the following parts:

multi-service marine_test applied to a function (ver2)
multi-service marine_test applied to a module (ver3)
interface for generating multi-service marine_test_env in build.rs, as proposed in #2

First is the first, order of others is to be discussed.

marine_test: change directory mapping between tests

Currently, if a service writes some state to the disk by a static file name, this state is shared between different tests. This leads to 2 things:

Tests can't run in parallel because of an implicit shared state
Each test has to clean state on the disk (either in the beginning, or at the end, or even both)

This makes tests very fragile and cluttered: forget to clean state and you have a failing test that is really hard to debug.

I think the situation can be improved by generating directory mapping randomly. For example, /tmp can be mapped to /tmp/$UUID on the host.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

cargo

Cargo.toml

fluence-app-service 0.35.1

serde 1.0.162

serde_json 1.0.96

uuid 1.3.2

trybuild 1.0

crates/marine-build-rs-generator/Cargo.toml

crates/marine-test-macro-impl/Cargo.toml

fluence-app-service 0.35.1

marine-it-parser 0.15.0

itertools 0.10.5

darling 0.20.1

quote 1.0.26

proc-macro2 1.0.69

proc-macro-error 1.0.4

syn 2.0.15

thiserror 1.0.40

static_assertions 1.1.0

marine-macro-testing-utils 0.1.0

crates/marine-test-macro/Cargo.toml

quote 1.0.26

proc-macro2 1.0.69

proc-macro-error 1.0.4

syn 2.0.15

github-actions

.github/workflows/lint.yml

amannn/action-semantic-pull-request v5

actions/checkout v3

reviewdog/action-actionlint v1

.github/workflows/release.yml

google-github-actions/release-please-action v3

actions/checkout v3

actions-rust-lang/setup-rust-toolchain v1

baptiste0928/cargo-install v2.0.0

stefanzweifel/git-auto-commit-action v4

actions/checkout v3

hashicorp/vault-action v2.5.0

actions-rust-lang/setup-rust-toolchain v1

baptiste0928/cargo-install v2.0.0

lwhiteley/dependent-jobs-result-check v1

hashicorp/vault-action v2.5.0

ravsamhq/notify-slack-action v2

.github/workflows/tests.yml

actions/checkout v3

hashicorp/vault-action v2.5.0

actions-rust-lang/setup-rust-toolchain v1

actions/download-artifact v3

fluencelabs/setup-marine v1

actions-rust-lang/rustfmt v1

Check this box to trigger a request for Renovate to run again on this repository

marine_test: how to test service restart?

On the network, nodes can restart. On the restart, they recreate services with the same service ids and the same disk state. Also, main is called again.

Is there a way to test for that with marine_test?

allow to set cp via context, instead of passing via _cp in test

works to pass cp into marine service
works to pass cp if i run service not as marine, but as native code

fluencelabs / marine-rs-sdk-test Goto Github PK

marine-rs-sdk-test's Issues

Problem

Possible solution

Cargo build scripts

CLion

VS Code

What to do

Description

Actual

Expected

Solution:

Why?

Problem

Solution

Interface

Roadmap

Rate-Limited

Open

Detected dependencies

Recommend Projects

Recommend Topics

Recommend Org