stjude-rust-labs/wdl

Rust crates for working with Workflow Description Language (WDL) documents.

wdl

Rust crates for working with Workflow Description Language (WDL) documents.
Explore the docs »

Request Feature · Report Bug · ⭐ Consider starring the repo! ⭐

📚 Getting Started

The wdl family of crates consists of (a) a number of component crates (any crate that is not explicitly wdl) that are developed and versioned independently, and (b) a convenience crate (the wdl crate) that exists to ease syncing compatible component crates versions. Component crates can be enabled using features and are generally re-exported crates without the wdl- (or wdl_) prefix.

This repository contains crates that can be used to work with WDL within your own Rust projects—if you're looking for a command-line tool built on top of these crates instead, you should check out sprocket.

Convenience Crate

Most users should prefer selecting a version of the convenience crate and enabling features as they wish. For example,

cargo add wdl --features grammar

and then

use wdl::grammar;

Component Crate(s)

You are free to include component crates directly. For example,

cargo add wdl_grammar

and then

use wdl_grammar;

Be aware, however, that versions between component crates are explicitly not compatible. In other words, if you choose not to use the convenience crate, it is not simple to derive which crate versions are compatible, and you'll need to manually sync those. We highly recommend using the convenience crate if you intend to use more than one component crate in conjunction.

🖥️ Development

To bootstrap a development environment, please use the following commands.

# Clone the repository
git clone [email protected]:stjude-rust-labs/wdl.git
cd wdl

# Build the crate in release mode
cargo build --release

# List out the examples
cargo run --release --example

🚧️ Tests

Before submitting any pull requests, please make sure the code passes the following checks (from the root directory).

# Run the project's tests.
cargo test --all-features

# Run the tests for the examples.
cargo test --examples --all-features

# Ensure the project doesn't have any linting warnings.
cargo clippy --all-features

# Ensure the project passes `cargo fmt`.
cargo fmt --check

# Ensure the docs build.
cargo doc

🤝 Contributing

Contributions, issues and feature requests are welcome! Feel free to check issues page.

📝 License

This project is licensed as either Apache 2.0 or MIT at your discretion.

Copyright © 2023-Present St. Jude Children's Research Hospital.

wdl's Issues

[BUG] Error parsing comment in multi-line boolean expression

Here's another bug in parsing, though I admit it's a bit of a silly edge case. If you have a multi-line boolean expression with a comment in the middle for some reason, the parser will not interpret it correctly. Here are the steps to reproduce:

1. Make a file called MinimalParse.wdl with the contents:

version 1.0

workflow MinimalParse {
    input {
        String? name
    }

    if (defined(name) &&
       # comment here
       defined(name)) {

    }
}

2. Try to parse with the wdl crate. Here is the snippet of code:

// Inside main
    let src = std::env::args().nth(1).expect("missing src");
    let contents = std::fs::read_to_string(src).expect("could not read file contents");

    // Generate the parse tree from the document source.
    let parse_tree = grammar::v1::parse(&contents)
        .expect("Cannot parse")
        .into_tree()
        .expect("Cannot convert to tree");

    // Generate the abstract syntax tree from the parse tree.
    let syntax_result = ast::v1::parse(parse_tree);

3. The program panics with output:

internal error: entered unreachable code: unknown primary in expression: COMMENT

4. Removing the comment line (keeping the multi-line boolean expression) parses just fine. Also everything in the Rust snippet works until the last line where the AST is parsed.

Both womtool validate and cromwell are happy with the original file. I agree it's probably not a good way to write comments in WDL in general, but the error produced here at the very least is not helpful. I suspect this edge case was not considered since the error mentions "unreachable code" but do think this might be legal WDL that should be parsed correctly.

`cargo build --release` failing due to dependency using `#![feature]`

When building from main I am getting -

   Compiling to_snake_case v0.1.1
error[E0554]: `#![feature]` may not be used on the stable release channel
  --> /Users/mrschre/.cargo/registry/src/index.crates.io-6f17d22bba15001f/to_snake_case-0.1.1/src/lib.rs:31:1
   |
31 | #![feature(let_chains)]
   | ^^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0554`.
error: could not compile `to_snake_case` (lib) due to 1 previous error

Rust version is rustc 1.77.2 (25ef9e3d8 2024-04-09)

Is it intended that the project be built with the rust nightly build?

[BUG] Error parsing ternary operator assignment

Hi folks, I think I found a subtle bug in parsing a particular type of ternary operator assignment. Here are the steps to reproduce:

1. Make the following file called MinimalParse.wdl:

version 1.0

workflow MinimalParse {
    input {
        Int num = 8
    }

    Int new_num=if(num>5) then 1 else 2  # Add space after `if` to parse with wdl crate
}

2. Try to parse using the wdl crate. Here is a Rust snippet:

// Inside main
    let src = std::env::args().nth(1).expect("missing src");
    let contents = std::fs::read_to_string(src).expect("could not read file contents");

    // Generate the parse tree from the document source.
    let parse_result = grammar::v1::parse(&contents);
    let parse_tree = match parse_result {
        Ok(_) => {
            let inner = parse_result.unwrap();
            println!("Inner is: {:?}", &inner);
            let tree = inner.into_tree().expect("Cannot make tree");
            tree
        }
        Err(e) => {
            println!("{}", e);
            panic!()
        }
    };

3. The printed output is:

Inner is: Result { concerns: Some(Concerns(NonEmpty { head: ParseError(Error { message: "The following tokens are required: singular_identifier.", location: Position(Position { line_no: 8, col_no: 32, byte_no: 107 }) }), tail: [] })), tree: None }

4. Add a space after if in the original WDL, i.e. Int new_num=if (num>5) then 1 else 2 and rerun. It now parses successfully.

I'm not defending the gross-looking syntax where there's no space after if, but womtool validate passes and cromwell does in fact run the original WDL, so this seems to be a parsing bug.

Curiously enough, the bug doesn't seem to care about the other shifty looking lack of spaces. Adding a space before or after the = does nothing to fix this. It also doesn't seem to care if you use a space or not after if in normal blocks outside of a ternary assignment.