[Enhancement] Associate high level syntax elements with their respective tokens

[skaupper]

[Enhancement] Associate high level syntax elements with their respective tokens

[kraigher]

First of all I think it is exciting thay you want to use vhdl_lang for generating documentation. I created the library with the hope that other people would build tools such as document generators upon the library. I will happily support with feedback on ideas and reviewing code. Since I work on this on my spare time the response might sometimes be delayed when other things in life take priority.

Just so you know @Schottkyc137 has also been interested in the topic of adding more token information to the AST.

Regarding adding token references I very much prefer option 1 since the WithTokenRef<T> just complicates the type hierarchy too much. It is better to add the fields to the current AST-nodes that need them directly without wrapping the type. If you want to process the different nodes that can have start and end token with unified code I would instead add a trait:

trait HasTokenSpan {
   fn start_token(&self) -> TokenId;
   fn end_token(&self) -> TokenId;
}

[skaupper]

Since I work on this on my spare time the response might sometimes be delayed when other things in life take priority.

Yeah, no problem there.

Just so you know @Schottkyc137 has also been interested in the topic of adding more token information to the AST.

I see. Seems like his work is the sort of thing I have in mind for other elements too.

If you want to process the different nodes that can have start and end token with unified code I would instead add a trait

I will play around with it as soon as I have time and see what I can do. Ideally I would want generalize as much as possible and maybe even combine my changes with what @Schottkyc137 has already done for UseClause etc. When I have a working proof of concept I will create a PR.

[skaupper]

I implemented the trait approach with added start/end token fields in #211. The implementation is done using macros to avoid code duplication as much as possible.

[kraigher]

Just so you know I have not focused on creating a nice public API for vhdl_lang. I think if you start creating a document generator as a third party crate on top of vhdl_lang it could serve as a nice pioneer in helping us establish I nice public API. I think we have the embryo of a nice api in the ASTNode and Visitor trait created by @Schottkyc137 but I would like to tweak it a bit to not require heap-allocation to traverse the AST.

[skaupper]

Since the relevant PR is already merged, I will close this issue.