Giter Site home page Giter Site logo

knuffel's Introduction

A KDL file format parser with great error reporting and convenient derive macros.

About KDL

To give you some background on the KDL format. Here is a small example:

foo 1 key="val" "three" {
    bar
    (role)baz 1 2
}

Here is what are annotations for all the datum as described by the specification and this guide:

foo 1 "three" key="val" {                           ╮
─┬─ ┬ ───┬─── ────┬────                             │
 │  │    │        ╰───── property (can be multiple) │
 │  │    │                                          │
 │  ╰────┴────────────── arguments                  │
 │                                                  │
 └── node name                                      ├─ node "foo", with
                                                    │  "bar" and "baz"
    bar                                             │  being children
    (role)baz 1 2                                   │
     ──┬─                                           │
       └────── type name for node named "baz"       │
}                                                   ╯

(note, the order of properties doesn't matter as well as the order of properties with respect to arguments, so I've moved arguments to have less intersections for the arrows)

Usage

Most common usage of this library is using derive and [parse] function:

#[derive(knuffel::Decode)]
enum TopLevelNode {
    Route(Route),
    Plugin(Plugin),
}

#[derive(knuffel::Decode)]
struct Route {
    #[knuffel(argument)]
    path: String,
    #[knuffel(children(name="route"))]
    subroutes: Vec<Route>,
}

#[derive(knuffel::Decode)]
struct Plugin {
    #[knuffel(argument)]
    name: String,
    #[knuffel(property)]
    url: String,
}

# fn main() -> miette::Result<()> {
let config = knuffel::parse::<Vec<TopLevelNode>>("example.kdl", r#"
    route "/api" {
        route "/api/v1"
    }
    plugin "http" url="https://example.org/http"
"#)?;
# Ok(())
# }

This parses into a vector of nodes as enums TopLevelNode, but you also use some node as a root of the document if there is no properties and arguments declared:

#[derive(knuffel::Decode)]
struct Document {
    #[knuffel(child, unwrap(argument))]
    version: Option<String>,
    #[knuffel(children(name="route"))]
    routes: Vec<Route>,
    #[knuffel(children(name="plugin"))]
    plugins: Vec<Plugin>,
}

let config = parse::<Document>("example.kdl", r#"
    version "2.0"
    route "/api" {
        route "/api/v1"
    }
    plugin "http" url="https://example.org/http"
"#)?;

See description of Decode and DecodeScalar for the full reference on allowed attributes and parse modes.

Errors

This crate publishes nice errors, like this:


Screenshot of error. Here is how narratable printer would print the error:
Error: single char expected after `Alt+`
Diagnostic severity: error
\
Begin snippet for test.kdl starting at line 17, column 1
\
snippet line 17:     }
snippet line 18:     key "Alt+" mode="normal" {
label starting at line 18, column 10: invalid value
snippet line 19:         move-focus "left"

To make them working, miette's "fancy" feature must be enabled in the final application's Cargo.toml:

[dependencies]
miette = { version="4.3.0", features=["fancy"] }

And the error returned from parser should be converted to [miette::Report] and printed with debugging handler. The most manual way to do that is:

# #[derive(knuffel::Decode, Debug)]
# struct Config {}
# let file_name = "1.kdl";
# let text = "";
let config = match knuffel::parse::<Config>(file_name, text) {
    Ok(config) => config,
    Err(e) => {
         println!("{:?}", miette::Report::new(e));
         std::process::exit(1);
    }
};

But usually function that returns miette::Result is good enough:

# use std::fs;
# #[derive(knuffel::Decode)]
# struct Config {}
use miette::{IntoDiagnostic, Context};

fn parse_config(path: &str) -> miette::Result<Config> {
    let text = fs::read_to_string(path).into_diagnostic()
        .wrap_err_with(|| format!("cannot read {:?}", path))?;
    Ok(knuffel::parse(path, &text)?)
}
fn main() -> miette::Result<()> {
    let config = parse_config("my.kdl")?;
    # Ok(())
}

See miette guide for other ways of configuring error output.

The Name

KDL is pronounced as cuddle. "Knuffel" means the same as cuddle in Dutch.

License

Licensed under either of

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

knuffel's People

Contributors

boringcactus avatar irevoire avatar rinarakaki avatar tailhook avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

knuffel's Issues

RTL property names don’t work without explicit name?

I’m not sure why this is happening, but this does not work (resulting in "unexpected property"):

#[derive(knuffel::Decode)]
pub struct الطاب {
    #[knuffel(property)]
    pub الطاب: i64,
}

… but this, with an explicit name, does:

#[derive(knuffel::Decode)]
pub struct الطاب {
    #[knuffel(property(name = "الطاب"))]
    pub الطاب: i64,
}

Child or Property decoding

Hello! I would like to parse the following document:

image "nginx" {
    reference "docker.io/library/nginx:1.23.4"
}

image "nginx" reference="docker.io/library/nginx:1.23.4"

Imagine that image could have 100 files. Or, it could have 1. Opening an entire block for just one argument seems excessive, but specifying 100 options using properties is similar. It would be nice if there was a way to decode either #knuffel(child, unwrap(argument)) or #knuffel(property) with a single derive argument. Here is my current, though admittedly fairly digusting, solution:

enum Image {
    Explicit(ExplicitImage),
    Inline(InlineImage),
}

impl<S: ErrorSpan> Decode<S> for Image {
    fn decode_node(
        node: &knuffel::ast::SpannedNode<S>,
        ctx: &mut knuffel::decode::Context<S>,
    ) -> Result<Self, knuffel::errors::DecodeError<S>> {
        if node.properties.is_empty() {
            ::knuffel::Decode::decode_node(node, ctx).map(Self::Explicit)
        } else {
            ::knuffel::Decode::decode_node(node, ctx).map(Self::Inline)
        }
    }
}

#[derive(knuffel::Decode)]
struct ExplicitImage {
    #[knuffel(argument)]
    name: String,
    #[knuffel(child, unwrap(argument))]
    reference: String,
}

#[derive(knuffel::Decode)]
struct InlineImage {
    #[knuffel(argument)]
    name: String,
    #[knuffel(property)]
    reference: String,
}

At the end of the day, we end up with two identical structs and a manual Decode implementation. That's a lot of boilerplate!

KDL 2.0 compliance

KDL 2.0.0-draft.3 is now generally available (kdl-org/kdl#286), and with it, we've decided to encourage existing implementations to start working towards 2.0 support and send feedback and suggestions for anything you might run into, so I'm making this tracking issue. :)

Interest in a (hopefully temporary) community fork?

Hi all!

It's been a while since @tailhook has been on GitHub, and whilst I don't particularly love the idea of creating a fork, with KDL 2.0 on the horizon (#35), it might be something thinking about?

I'm happy to host a fork in the meantime, I'm already needing to use a patched git-version of knuffel, but it's not certain that I'll have the time to implement bigger changes like the move to KDL 2.0, so that might take @tailhook 's help, or some help from the broader community!

If anyone else is keen on having a crates.io version of knuffel with some patches merged in, let me know and I can sort something out!

Encoding

Are there any plans to support writing to the KDL format?

Expects `DecodeScalar` for `f32`

Given the following:

#[derive(Clone, Debug, knuffel::Decode)]
pub struct ManifestNode {
    #[knuffel(child, unwrap(argument))]
    pub version: f32,
}

cargo check

error[E0277]: the trait bound `f32: DecodeScalar<S>` is not satisfied
 --> backend/src/manifest.rs:9:24
  |
9 | #[derive(Clone, Debug, knuffel::Decode)]
  |                        ^^^^^^^^^^^^^^^ the trait `DecodeScalar<S>` is not implemented for `f32`

Why is this?

Inconsistency of syntactical sites to become node names between nodes and their child nodes

As we can see below,

  • whether nodes have arguments, properties or doesn't have one at all,
  • whether they are modelled by tuple structs, struct structs or unit structs,

knuffel's mantra in mapping KDL to Rust should be simple: a node's name is a struct's name or an enum's variant name:

node0
node1 1 "hoge"
node2 a=1 b="hoge"
structs
#[derive(Decode)]
struct Node0;
#[derive(Decode)]
struct Node1(
    #[knuffel(argument)] i32,
    #[knuffel(argument)] String
);
#[derive(Decode)]
struct Node2(
    #[knuffel(property(name = "a"))] i32,
    #[knuffel(property(name = "b"))] String
);
#[derive(Decode)]
struct Node1 {
    #[knuffel(argument)] a: i32,
    #[knuffel(argument)] b: String
}
#[derive(Decode)]
struct Node2 {
    #[knuffel(property)] a: i32,
    #[knuffel(property)] b: String
}
enums
                          #[derive(Decode)]
                            enum Node {
                                Node0,
                                Node1(
                                    #[knuffel(argument)] i32,
                                    #[knuffel(argument)] String
                                ),
                                Node2(
                                    #[knuffel(property(name = "a"))] i32,
                                    #[knuffel(property(name = "b"))] String
                                )
                            }
                          #[derive(Decode)]
                            enum Node {
                                Node0,
                                Node1 {
                                    #[knuffel(argument)] a: i32,
                                    #[knuffel(argument)] b: String
                                },
                                Node2 {
                                    #[knuffel(property)] a: i32,
                                    #[knuffel(property)] b: String
                                }
                            }

But when it comes to modelling children, knuffel supposes a different syntactical site to be those children's names:

node {
  child 1
}

is modelled not by

#[derive(Decode)]
struct Node {
    #[knuffel(child)]
    a: Child
}

#[derive(Decode)]
struct Child(#[knuffel(argument)] i32);

but

#[derive(Decode)]
struct Node {
    #[knuffel(child)]
    child: A
}

#[derive(Decode)]
struct A(#[knuffel(argument)] i32);

And this is the cause of every problem in knuffel ––

  • Impossibility

Since the child's name is anchored by the field's name, enums with Decode cannot be used for the single child. #7

  • Inconsistency

This especially comes up when combined with serde's Serialize derive macro and knuffel::parse::<Vec<Node>> pattern, where the Node's aren't camel-cased (node) but outputted raw (Node).

  • Redundancy

    • children directive takes name argument.
    • node_name attribute.
    • type_name attribute.
  • Partiality

Tuple structs cannot have children, only struct structs can.

  • Incompleteness

  • Structural typing

Instead of nominal typing which must be the culmination in writing KDL and the shared spirit with Rust.

Feat: allow extracting multiple children names into one field

e.g. I have

#[derive(knuffel::Decode)]
pub struct Config {
    #[knuffel(children(name = "replace"))]
    pub replace: Vec<ReplaceRegex>,
    #[knuffel(children(name = "job"))]
    pub jobs: Vec<Job>,
}

It turns out that a lot of replace rules end up being used in practice, and I'd like to allow the use of r as a shorthand. With current knuffel, I'd have to do this as a separate field. I would instead like to do something like the following:

#[derive(knuffel::Decode)]
pub struct Config {
    #[knuffel(children(name = "replace", name = "r"))]
    pub replace: Vec<ReplaceRegex>,
    #[knuffel(children(name = "job"))]
    pub jobs: Vec<Job>,
}

and both replace and r children will go into self.replace.

Q: Best representation for a structure

Hi, awesome library but I'm struggling with representing a rather freestyle structure like this:

genres {
    rock
    metal
    pop {
         albums-by-year {
             y2000 "a" "b" "c"
             y2001 "d" "e"    
         }
    }
}

Here, for example I expect there to be a top-level genres node but I don't restrict which and how many child nodes it has. Then each node again is expected to have smth like albums-by-year which itself is again pretty freestyle key-value table. How can I do this with knuffel? I expect to be able to do smth like this:

#[derive(Debug, knuffel::Decode)]
struct Genre {
    #[knuffel(node_name)]
    node_name: String // name of the genre
    #[knuffel(children)]
    info: Option<Info>
}
#[derive(Debug, knuffel::Decode)]
struct Genres {
    #[knuffel(children)]
    genres: Vec<Genre>,
}

decode arguments into enums

I have a KDL document like:

pad {
    left "Left"
    right "Right"
    up "Up"
    down "Down"
}

And I want to decode it into the following structure.

#[derive(knuffel::Decode)]
pub struct Config {
    #[knuffel(child)]
    pub pad: Pad,
}

#[derive(knuffel::Decode)]
pub struct Pad {
    #[knuffel(child, str, unwrap(argument))]
    pub left: KeyMap,
    #[knuffel(child, str, unwrap(argument))]
    pub right: KeyMap,
    #[knuffel(child, str, unwrap(argument))]
    pub up: KeyMap,
    #[knuffel(child, str, unwrap(argument))]
    pub down: KeyMap,
}

#[derive(enum_utils::FromStr)]
pub enum KeyMap {
    Left,
    Right,
    Up,
    Down
}

This is a very common pattern when working with serde but it seems knuffel doesnt support enums as arguments/properties very well?
I tried the knuffel derive attributes that I included in this example as well as a few others but all of them either failed to compile or failed to parse the kdl.

Can't use enum as single child

Given

#[derive(Debug, knuffel::Decode)]
struct SomeDocument {
    #[knuffel(child)]
    some_node: SomeEnum,
}

#[derive(Debug, knuffel::Decode)]
enum SomeEnum {
    SomeVariant(SomeVariant),
    AnotherVariant(AnotherVariant),
}

#[derive(Debug, knuffel::Decode)]
struct SomeVariant {}

#[derive(Debug, knuffel::Decode)]
struct AnotherVariant {}

if I try to parse

some-variant

I get errors: "unexpected node some-variant" and "child node some-node is required"

but if I try to parse

some-node

I get "expected one of some-variant, another-variant"

I tried many variations of this in hope that I'm just misunderstanding KDL or knuffel, but in the end I checked the Decode implementations with cargo expand and it appears that the node is expected to have name of the parent struct field and one of the enum variants at the same time.

full reproduction here: https://github.com/silen-z/knuffel-enum-bug-repro/blob/main/src/lib.rs

[Question] How to parse arguments of different types?

I am not able to figure out how to parse a node which takes arguments of different types i.e, int, float or string. For example.

/* Only Floats */
margin 0.5 0.5 0.5 0.5

/* Only Int */
margin 1 2 3 4

/* Mixed */
margin 0.5 1 1.5 "$.top - 3"

And I want that in, somewhat, similar structure as below.

#![allow(dead_code)]

#[derive(Debug, Default, knuffel::Decode)]
struct Document {
    #[knuffel(child)]
    margin: Option<Direction>,
}

#[derive(Debug, Default, knuffel::Decode)]
struct Direction {
    top: Val,
    right: Val,
    bottom: Val,
    left: Val,
}

#[derive(Debug, knuffel::Decode)]
enum Val {
    Int(#[knuffel(argument)] u32),
    Float(#[knuffel(argument)] u32),
    Str(#[knuffel(argument)] String),
}

impl Default for Val {
    fn default() -> Self {
        Self::Int(0)
    }
}

fn main() {
    let doc: Document = knuffel::parse("keeb.kdl", r#"margin 0.5 1 1.5 "$.top - 3""#).unwrap();
    dbg!(&doc);
}

I am getting unexpected argument after the margin. This code doesn't seem right to me either, but I also couldn't figure out how to fix this.

image

`#[knuffel(child)] field: Option<T>`

Just like #[knuffel(child)] field: bool maps the presence/absence of field to true/false,
I expect #[knuffel(child)] field: Option<T> for T withDecode (without Decode) to map the presence/absence of field { ..t } (field t) to Some(t)/None where t : T.

Currently we get this error in some cases:

error[E0277]: the trait bound std::option::Option<T>: Decode<S> is not satisfied.

field modes of T Option<T>
all fields are property DOESN’T WORK
all fields are children WORKS

Can't decode enum tuple variants?

Maybe I'm doing something wrong, but trying to do this:

use knuffel;

#[derive(knuffel::Decode)]
pub enum MyEnum {
    MyFirstVariant,
    VariantWithTuple(u8),
}

fn main() {
    println!("Hello, world!");
}

I get:

$ cargo check

error[E0277]: the trait bound `u8: Decode<S>` is not satisfied
 --> src/main.rs:3:10
  |
3 | #[derive(knuffel::Decode)]
  |          ^^^^^^^^^^^^^^^ the trait `Decode<S>` is not implemented for `u8`
  |
  = help: the following other types implement trait `Decode<S>`:
            Arc<T>
            Box<T>
            MyEnum
            Node<T>
            Rc<T>
            Spanned<Node<T>, T>
  = note: this error originates in the derive macro `knuffel::Decode` (in Nightly builds, run with -Z macro-backtrace for more info)

Any clues?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.