vhdl-ls/rust_hdl

Overview

This repository contains a fast VHDL language server and analysis library written in Rust.

The speed makes the tool very pleasant to use since it loads projects really fast and does not consume a lot of ram. A 200.000 line VHDL project is analyzed in 160 ms on my Desktop using 8 cores and only consumes 180 MByte of RAM when loaded.

I very much appreciate help from other people especially regarding semantic analysis of VHDL. You do not need to be a programmer to help, it is even more helpful to interpret and clarify the VHDL standard and provide minimal examples and describe how they should work according to the standard. Further information about contributing can be found by reading the Contributors Guide

Contributors

• Maintainer: Lukas Scheller
• Founder: Olof Kraigher

Projects

VHDL Language Server

Goals

• A complete VHDL language server protocol implementation with diagnostics, navigate to symbol, find all references etc.

Features

• Live syntax and type checking
• Checks for missing and duplicate declarations
• Supports goto-definition/declaration (also in presence of overloading)
• Supports find-references (also in presence of overloading)
• Supports goto-implementation
  • From component declaration to matching entity by default binding
  • From entity to matching component declaration by default binding
• Supports hovering symbols
• Rename symbol
• Find workspace symbols
• View/find document symbols

When Installing it from Crate

When installing the VHDL_LS from crates.io the required
vhdl_libraries directory will not be installed automatically and
will need to be copied into the parent directory of the VHDL_LS binary manually.

Trying it out

A language server is never used directly by the end user and it is integrated into different editor plugins. The ones I know about are listed here.

Use in VSCode

https://github.com/Bochlin/rust_hdl_vscode

Use in emacs

VHDL LS has built-in support by emacs lsp-mode since 2020-01-04.

It can be set up automatically by installing the package vhdl-ext and adding the following snippet to your config:

(require 'vhdl-ext)
(vhdl-ext-mode-setup)
(vhdl-ext-eglot-set-server 've-rust-hdl) ;`eglot' config
(vhdl-ext-lsp-set-server 've-rust-hdl)   ; `lsp' config

Installation for Neovim

Automatic Installation

You can install rust_hdl automatically in Neovim using :Mason. Within Mason, the package is called rust_hdl. If you don't have :Mason, you can simply install the binary as previously described.

Automatic Configuration using nvim-lspconfig

nvim-lspconfig has a built in configuration for vhdl_ls

In order to configure it, simply add

lspconfig = require('lspconfig')
lspconfig['vhdl_ls'].setup({
  on_attach = on_attach,
  capabilities = capabilities
})

Manual Configuration using Neovim's built in client

Neovim provides an LSP client to the VHDL_LS language server. Download the
VHDL_LS release. The binary must be on the path and executable (if you can run
"vhdl_ls -h" in the terminal then you're good).

In your Neovim config.lua add the following:

function STARTVHDLLS()
  vim.lsp.start({
    name = 'vhdl_ls',
    cmd = {'vhdl_ls'},
  })
end
vim.api.nvim_set_keymap('n', '<F5>', ':lua STARTVHDLLS()<CR>', { noremap = true, silent = true })

Using the example above, pressing F5 while inside Neovim starts the language
server. There are also other options, like automatically starting it when
opening a certain file type, see the Neovim LSP documentation for more.

Configuration

The language server needs to know your library mapping to perform full analysis of the code. For this it uses a configuration file in the TOML format named vhdl_ls.toml.

vhdl_ls will load configuration files in the following order of priority (first to last):

1. A file named .vhdl_ls.toml in the user home folder.
2. A file name from the VHDL_LS_CONFIG environment variable.
3. A file named vhdl_ls.toml in the workspace root.

Settings in a later files overwrites those from previously loaded files.

Define the VHDL revision to use for parsing and analysis with the standard key. The expected value is the year associated the VHDL standard. Supported standards are 1993, 2008 and 2019 where both the long version ("2008") and the short version ("08") can be used. If nothing is specified, 2008 is used.

Note

Defining the standard feature is a relatively new feature (since april 2024). Anything but the 2008 standard will not change much at the moment.

Example vhdl_ls.toml

# What standard to use. This is optional and defaults to VHDL2008.
standard = "2008"
# File names are either absolute or relative to the parent folder of the vhdl_ls.toml file
[libraries]
lib2.files = [
    'pkg2.vhd',
]
lib1.files = [
    'pkg1.vhd',
    'tb_ent.vhd'
]

# Wildcards are supported
lib3.files = [
    'test/*.vhd',
    'src/*.vhd',
    'src/*/*.vhd',
]

# Libraries can be marked as third-party to disable some analysis warnings, such as unused declarations
UNISIM.files = [
    'C:\Xilinx\Vivado\2023.1\data\vhdl\src\unisims\unisim_VCOMP.vhd',
]
UNISIM.is_third_party = true

[lint]
unused = 'error' # Upgrade the 'unused' diagnostic to the 'error' severity
unnecessary_work_library = false # Disable linting for the 'library work;' statement

Using the lint table, you can configure the severity of diagnostics or turn of diagnostics altogether.

Warning

You can overwrite every diagnostic error code including syntax or analysis errors using the lint table. However, the intended use-case is for lints only. Overwriting syntax or analysis errors (e.g., error codes unused or syntax) can cause unwanted side effects

Paths in the vhdl_ls.toml can contain glob patterns (i.e., .../*/). On Unix machines, they can contain environment variables using the $NAME or ${NAME} syntax. On Windows machines, use the %NAME% syntax to substitute environment variables.

As an LSP-client developer how should I integrate VHDL-LS?

I recommend that the lsp-client polls GitHub and downloads the latest VHDL-LS release from GitHub.

VHDL-LS has frequent releases and the automatic update ensures minimal maintenance for the lsp-client developer as well as ensuring the users are not running and outdated version.

VHDL Language Frontend

Goals

• This project aims to provide a fully featured open source VHDL frontend that is easy to integrate into other tools.
• A design goal of the frontend is to be able to recover from syntax errors such that it is useful for building a language server.
• Analysis order must be automatically computed such that the user does not have to maintain a compile order.
• Comments will be part of the AST to support document generation.
• Separate parsing from semantic analysis to allow code formatting on non-semantically correct code.

rust_hdl's Issues

Add comments to the AST

Background

Tool such as document generators and code formatters also need to care about comments.

• Document generators need to associate documenting comments with right syntactic elements.
• Code formatters need to preserve comments and keep their association with the original syntactic element.

To achieve this the parser shall also extract comments into the AST.

Terminology

There are two types of comments leading and trailing.
An element may have several leading comments and at most one trailing comment.

-- A leading comment associated with foo
-- Another leading comment associated with foo
signal foo : natural; -- A trailing comment associated with foo
-- A leading comment associated with bar
signal bar : natural;

Unnatural comments

There are plenty of opportunities for placing comments in unnatural places:

if foo 
-- Unnatural comment but legal
then
end 
-- Unnatural comment but legal
if
-- Unnatural comment but legal
;

I think the acceptable solution for such comments are:

1. Moving them with warning
2. Ignoring them with warning

Not acceptable solutions would be:

1. Moving them without warning
2. Ignoring them without warning

Implementation concerns

There are several possible solutions:

1. It would be possible to add a token kind for comments, leading and trailing comments must be distinguishable. These token need to be handled differently than other tokens otherwise the cost will huge as all parsing functions must be able to handle comments everywhere. For example if a parsing function today expects a SemiColon it must be able to ignored a comment. Thus the TokenStream must have dedicated methods to optionally expect a leading or a trailing comment.

2. A better approach might be to add a method to the TokenStream to extract the leading and trailing comments for the last token. The TokenStream could keep track of comments which had not been extracted and thus were ignored. For each of these ignored comments a warning could be emitted after parsing is completed. Thus support for comments can be added incrementally and comments not handled will be explicitly flagged as ignored.

3. A variant of 2. is to attach the comments as metadata to the Token structs. This is very clean as the comments are directly associated with the token. There would still need to be a mechanism to detect comments which where not handled. I think this can be handled by storing a reference in the Token struct to an external storage where the comment remains if it was not extracted. Thus ignored comments will remain in the external structure which can be traversed after parsing to print warnings for ignored comments.

Recovery in declarative parts

Terminology

A declarative part contains declarations. For the purpose of recovery we divide declarations into two categories.

• Compound - may contain nested declarative parts, example subprograms, protected type
• Simple - does not contain nested declarative parts, example object declaration, alias

Recovering from error in a simple declaration

Simply scanning for a synchronization token is enough.

Examples

Recover by synchronizing with second constant keyword:

constant bad :
variable good : natural;

Recover by synchronizing with the end of the declarative region

constant bad :
begin

constant bad :
end

Recovering from error in a compound declaration

A compound declaration may contain nested declarations and thus recovery cannot simply search for a synchronization token unless we risk synchronizing with a nested declaration.

Examples

Error in nested declarative part

-- parent region

procedure proc is
  procedure nested is
  begin
     if -- Syntax error
  end;

  -- We cannot synchronize with this if the above syntax error 
  -- is propagated all the way back to the parent region
  variable good : natural;
begin
end;

"pure" before function declaration/definition is flagged as unexpected

With the VHDL LS plugin (0.3.1) of VisualStudioCode (1.47.3) the VHDL keyword pure is flagged as unexpected before the function keyword. A list of expected keywords is shown. The keyword impure is among the ones suggest, which I consider mildly ironic.
The keyword pure is optional but correct in this position (the only position where pure can actually appear at all I guess) and should not be flagged.

According to the plugin's description "As VHDL LS uses the Rust HDL language server, issues related to syntax and semantic checks should be reported directly to Rust HDL".

error: Edition 2018 is unstable

I cannot build project on my machine:

[slawek@slawek-desktop rust_hdl]$ uname -a
Linux 4.18.19-100.fc27.x86_64 #1 SMP Wed Nov 14 22:04:34 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[slawek@slawek-desktop rust_hdl]$ rustc -V
rustc 1.30.1

Output:

[slawek@slawek-desktop rust_hdl]$ cargo run --bin vhdl_parser --verbose
       Fresh unicode-width v0.1.5                                                                                                                                        
       Fresh ansi_term v0.11.0                                                                                                                                           
       Fresh bitflags v1.0.4                                                                                                                                             
       Fresh vec_map v0.8.1                                                                                                                                              
       Fresh strsim v0.7.0                                                                                                                                               
       Fresh fnv v1.0.6                                                                                                                                                  
       Fresh textwrap v0.10.0                                                                                                                                            
       Fresh pad v0.1.5                                                                                                                                                  
       Fresh libc v0.2.45                                                                                                                                                
       Fresh serde v1.0.82                                                                                                                                               
       Fresh atty v0.2.11                                                                                                                                                
       Fresh toml v0.4.10                                                                                                                                                
       Fresh clap v2.32.0                                                                                                                                                
   Compiling vhdl_parser v0.4.0 (/home/slawek/git/github/rust_hdl/vhdl_parser)                                                                                           
     Running `rustc --edition=2018 --crate-name vhdl_parser vhdl_parser/src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=3b2de8b47672ecc5 -C extra-filename=-3b2de8b47672ecc5 --out-dir /home/slawek/git/github/rust_hdl/target/debug/deps -C incremental=/home/slawek/git/github/rust_hdl/target/debug/incremental -L dependency=/home/slawek/git/github/rust_hdl/target/debug/deps --extern clap=/home/slawek/git/github/rust_hdl/target/debug/deps/libclap-663bd2b0902d3fff.rlib --extern fnv=/home/slawek/git/github/rust_hdl/target/debug/deps/libfnv-f76d38542e21cab9.rlib --extern pad=/home/slawek/git/github/rust_hdl/target/debug/deps/libpad-1b67d0c55374ab28.rlib --extern toml=/home/slawek/git/github/rust_hdl/target/debug/deps/libtoml-6542184d7b6585e4.rlib`
error: Edition 2018 is unstable and only available for nightly builds of rustc.                                                                                          
                                                                                                                                                                         
error: Could not compile `vhdl_parser`.                                                                                                                                  

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name vhdl_parser vhdl_parser/src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=3b2de8b47672ecc5 -C extra-filename=-3b2de8b47672ecc5 --out-dir /home/slawek/git/github/rust_hdl/target/debug/deps -C incremental=/home/slawek/git/github/rust_hdl/target/debug/incremental -L dependency=/home/slawek/git/github/rust_hdl/target/debug/deps --extern clap=/home/slawek/git/github/rust_hdl/target/debug/deps/libclap-663bd2b0902d3fff.rlib --extern fnv=/home/slawek/git/github/rust_hdl/target/debug/deps/libfnv-f76d38542e21cab9.rlib --extern pad=/home/slawek/git/github/rust_hdl/target/debug/deps/libpad-1b67d0c55374ab28.rlib --extern toml=/home/slawek/git/github/rust_hdl/target/debug/deps/libtoml-6542184d7b6585e4.rlib` (exit code: 1)

According to kbknapp/cargo-outdated#141 it is somehow related to edition switch.

Case statement in a process should auto-insert all states if the select variable is an enumerate

When using the case syntax, a nice feature would be to automatically insert all case names. To accomplish this, the parser would need to see that the case variable as shown below is an enumerate, and if so, it would automatically insert all options:

type fsm_t is (IDLE, STEP_1, DONE);
signal state : fsm_t := IDLE;

...
process(all)
begin
    case state is -- after user presses tab when entering case select variable, it will auto populate options like below only if parser recognizes signals type as an enumerate
        when IDLE =>

        when STEP_1 =>

        when DONE =>

        when others => null; -- could have option for the user to set state to first option in enumerate for the others case
    end case;
end process;

Question about syntax tree

@kraigher I was loosely thinking about adding support for sources formatting.
Right now it does not look like an easy task, because the syntax tree looks more like an Abstract Syntax Tree, not Concrete Syntax Tree and there is no easy access to separators (spaces, tabs, newlines). Such information of course could be retrieved based on the positions, however it wouldn't look like a clean solution (at least for me).
I am not a compilers expert, but do you think this is pure AST? It holds information about positions, and AST usually does not contain such information.

Have you ever been thinking about reformatting functionality? Have you had any idea how you would like to achieve this functionality?

Atom client requires vhdl_ls.toml

I use rust_hdl from docker and after some update language server stopped working in Atom. I was using it without any vhdl_ls.tom for simple syntax checking. Now it seems toml project file is necessary?

export LSIF file

I'd like to use this program to export the LSIF data to a file so that it can be consumed by GitLab for its "Code Intelligence" feature. This enables hover features, etc. while browsing the code in the web IDE.

Is generating this LSIF dump currently possible? The statement in the README, "...has no intended usability at this point", makes me think it may not be possible, but I was hoping you could tell me clearly whether it is or not. If not, is that an intended feature in the future? Thanks!

p.s. If this is something that would be a "good first issue", I'd be willing to look into it. I'm interested in working in Rust and have read the documentation for the language, but haven't had an opportunity to work on a real project in that language. It would probably be several months before I have the free time to work on this though.

Analysis vs syntax directories.

@kraigher would you be so kind to explain, what was the intention of analysis directory and how it differs from syntax directory? What should be added/looked for in any of them?

Add VHDL release packaging

In summary this still needs to be done;

• Build vhdl_parser as packaged
• Publish github release on v.*.*.* tag
• Use release notes from release_notes/v*.*.*.md file

Create Atom integration

Create an Atom integration of VHDL Language Server and publish instructions.

Create NeoVim integration

Create NeoVim integration of VHDL language server and publish instructions on how to use it.

Precompiled libraries

I would like to implement support for precompiled libraries. Precompiled libraries would speed up the initial analysis of a project, because std and ieee would not have to be parsed and analyzed every time.
My thoughts on the subject:

• vhdl_lang should not prescribe a file format, nor should it be involved in locating precompiled libraries on disk.
• vhdl_lang should probably not include a reference implementation of the (de)serialization step. vhdl_ls is a different matter though, and should probably provide such an implementation. Not sure on that point...
• Since vhdl_lang isn't able to locate, load, and deserialize a precompiled library, and programs using vhdl_lang should, performance reasons, not be forced to load and deserialize every available library beforehand, the following procedure would make sense:
  1. Create a Project and parse the source files.
  2. Call a method Project::get_missing_libs() which returns the set of library names which are referenced in the library clauses of any parsed design unit, minus the set of library names which are already known to the Project instance.
  3. Load and deserialize the missing libraries. The newly constructed ASTs and Library instances are added to the Project.
  4. Call Project::analyse(). No changes should be needed anywhere in the analysis code.

Regarding the concrete (de)serialization implementation:

• I can see two major use cases for precompiled libraries:
  1. Black box libraries: A "closed" library without references to the original source files; provided by, e.g., a simulator vendor. SrcPos instances of declarations would probably be stored as some form of "named entity #17259 in library 'xyz'".
  2. Work in progress libraries: A library which retains full SrcPos data. Typical scenario: when working with command line tools like ModelSim's vcom and vsim, which incrementally add source files to a working library.
• Do you have any plans regarding the Reference type? Currently it uses SrcPos as a way to identify declarations, which is okay for a language server, but suboptimal for a simulator or synthesis tool. It's also a bit problematic when serializing a black box library.

Project structure

@kraigher I wanted to ask one question, as you have probably already considered this issue. Have you thought about structuring project in the same way the standard is structured.

For example each section would be a module, and subsection would be represented by structs, enums, traits, etc.
At first it looks like a great idea, but maybe there is something that I have overlooked.
I would really appreciate you experience as you have probably tackled a lot of problems during implementation.

Code documentation

In an effort to make myself familiar with the code base, I've started adding documentation to types, traits, and the occasional method. While I'm at it, I also fixed some typos I noticed. Example.

@kraigher Are you interested in these kinds of changes? If not, that's not a problem at all, I'll simply keep them in my local repo. If yes, I'd gather some more before I create a pull request.

You also cautioned me against documenting the analysis module because it will change frequently. Would it be okay if I add brief, high-level comments there anyway? Like I said, I'm doing this mostly for myself to aid understanding, and I wouldn't mind at all if you delete them later in case they become out of date.

"end postponed process" marked as error

When a postponed process is used, vhdl_ls doesn't like it when the end tag is end postponed process rather than just end process or end. It marks the end postponed as an error and says it was expecting "process".

I'm not sure what the standard says, but questasim, synopsys vcs, and aldec riviera all accept the end postponed process syntax.

proc_label: postponed process
  variable whatever: integer;
begin
  whatever := 0;
end postponed process;
    ^^^^^^^^^
    "expected process"

Add support for ignoring/blackboxing libraries or DU names

When running projects with pre-compiled/vendor libraries or mixed language projects, not all files will be in vhdl libraries added to the vhdl_ls.toml file.
It would be very useful to be able to add primary units with "missing declarations" and unknown libraries, to an ignore list;

for unknown libraries (lib_x) it would be an option to add them to the vhdl_ls.toml file without any files, in which case all:
"use lib_x.pkg" statements would be skipped by the semantics checker

Improve documentation

The current documentation is almost non-existing. @kraigher could you please document at least vhdl_lang crate public items?
I am trying to use vhdl_lang as dependency, but I stuck analyzing the internals. I guess the point of sharing code via crate is a bit different.

Rationale for Message and MessagePrinter structs.

@kraigher would you be so kind to explain what is the rationale for Message and MessagePrinter structs? Do I understand correctly, that Message has been introduced to abstract the message exchanged with LSP client, but later you wanted to also print some of the messages in the vhdl_lang binary, so you have introduced MessagePrinter?

AST to VHDL

Implement conversion of the AST back into a VHDL text file.

There are a number of possible uses for having the ability to go back from the AST to VHDL.

• Debugging of parsing. Do VHDL -> AST -> VHDL and check it doesn't effect simulation results.
• Implement an obfuscator by changing the symbol names, and then writing the VHDL out.
• Implement a code formatter like rustfmt for VHDL.
• VHDL2008 to well-supported VHDL transformations. Do transformations on AST, and then write out VHDL.

But mostly I thought it was a reasonably independent project that I could work on and not get in your way too much :).

Requirements on library configuration

There is a need for a configuration file format to declare which files belong to which libraries.
For the language server the configuration file will have to be placed in the workspace by the user.
I think TOML will be a suitable format for the configuration file format using this rust library https://docs.rs/toml/0.4.8/toml/.
In the absense of a configuration file the user would be warned and semantic analysis would be turned off giving the user just syntax errors.

The command line parser demonstrator could also read the configuration file. Is it still usable for the parser demonstrator to be able to read files as a list in argv without library information to just perform syntax checking? Maybe the parser-only without semantic analysis should be a separate binary if we need it. You have started to use the parser binary @mbrobbel, what do you think?

The data structure for the library would have to support clearing all design units from a certain file and updating them again to work with the language server when a file is edited and design units are removed or added.

vhdl_ls.toml for Language Server client

I've started to implement support for standard libraries, vendor libraries and some of the more prominent open source libraries (e.g. VUnit) in VHDL LS. I would like to do this by letting the extension maintain its own vhdl_ls.toml file. The user would be able to select which libraries to enable (including install dirs in some cases) and the client then generates a vhdl_ls.toml pointing out the standard/vendor libraries. The generated file would be local to the project (stored somewhere in .vscode). To do this I need a way to pass this file to vhdl_ls. I believe that the easiest way is as a command line parameter to vhdl_ls but I'm open for any suggestions.

Suggestion:
Add a command line parameter to vhdl_ls to point out a specific vhdl_ls.toml, priority according to list below:

1. A file named .vhdl_ls.toml in the user home folder.
2. A file name from the VHDL_LS_CONFIG environment variable.
3. Command line parameter ...
4. A file named vhdl_ls.toml in the workspace root.

Add GitHub CI for Window and Linux replacing Travis

• Replace Travis with GitHub CI.
• Run unit tests on Linux & Windows.

Create VSCode integration

Create VSCode integration of VHDL language server and publish instructions on how to use it.

Tests fail on Windows

Tests for both language server and parser fail on Windows. I don't think any of them look serious, it's mostly just directory separator conventions.

vhdl_ls

rust_hdl>cargo test --package vhdl_ls
    Finished dev [unoptimized + debuginfo] target(s) in 1.68s
     Running target\debug\deps\vhdl_ls-b008e55f06f9c466.exe

running 7 tests
test vhdl_server::tests::initialize ... ok
test vhdl_server::tests::utf8_to_latin1_conversion_gives_error_diagnostic ... ok
test vhdl_server::tests::did_open_with_diagnostics_and_change_without ... ok    
test vhdl_server::tests::did_open_no_diagnostics ... ok
test vhdl_server::tests::initialize_with_config_missing_files ... FAILED        
test vhdl_server::tests::initialize_with_bad_config ... ok
test vhdl_server::tests::initialize_with_config ... FAILED

failures:

---- vhdl_server::tests::initialize_with_config_missing_files stdout ----
thread 'vhdl_server::tests::initialize_with_config_missing_files' panicked at 'Object({"message": String("Loaded workspace root configuration file: C:\\Users\\mark_\\AppData\\Local\\Temp\\.tmpDOjfq4\\vhdl_ls.toml"), "type": Number(4)}) does not contain string "Loaded workspace root configuration file: C:\\Users\\mark_\\AppData\\Local\\Temp\\.tmpDOjfq4\\vhdl_ls.toml"', vhdl_ls\src\rpc_channel.rs:158:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

---- vhdl_server::tests::initialize_with_config stdout ----
thread 'vhdl_server::tests::initialize_with_config' panicked at 'Object({"message": String("Loaded workspace root configuration file: C:\\Users\\mark_\\AppData\\Local\\Temp\\.tmpa9KZcR\\vhdl_ls.toml"), "type": Number(4)}) does not contain string "Loaded workspace root configuration file: C:\\Users\\mark_\\AppData\\Local\\Temp\\.tmpa9KZcR\\vhdl_ls.toml"', vhdl_ls\src\rpc_channel.rs:158:25


failures:
    vhdl_server::tests::initialize_with_config
    vhdl_server::tests::initialize_with_config_missing_files

test result: FAILED. 5 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

vhdl_parser

rust_hdl>cargo test --package vhdl_parser
    Finished dev [unoptimized + debuginfo] target(s) in 0.10s  
     Running target\debug\deps\vhdl_parser-2639f44ec7d3e3aa.exe

running 525 tests

ommited test list

failures:

---- config::tests::test_warning_on_emtpy_glob_pattern stdout ----
thread 'config::tests::test_warning_on_emtpy_glob_pattern' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 [
     Message {
         message_type: Warning,
<        message: "Pattern \'parent_folder\\missing*.vhd\' did not match any file",
>        message: "Pattern \'parent_folder/missing*.vhd\' did not match any file",
     },
 ]

', vhdl_parser\src\config.rs:420:9

---- config::tests::test_append_config stdout ----
thread 'config::tests::test_append_config' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 Config {
     libraries: {
         "lib1": LibraryConfig {
             name: "lib1",
             files: [
<                "parent_folder0\\pkg1.vhd",
>                "parent_folder0/pkg1.vhd",
             ],
         },
         "lib3": LibraryConfig {
             name: "lib3",
             files: [
<                "parent_folder1\\pkg3.vhd",
>                "parent_folder1/pkg3.vhd",
             ],
         },
         "lib2": LibraryConfig {
             name: "lib2",
             files: [
<                "parent_folder0\\pkg2.vhd",
<                "parent_folder1\\ent.vhd",
>                "parent_folder0/pkg2.vhd",
>                "parent_folder1/ent.vhd",
             ],
         },
     },
 }

', vhdl_parser\src\config.rs:323:9

---- config::tests::test_warning_on_missing_file stdout ----
thread 'config::tests::test_warning_on_missing_file' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 [
     Message {
         message_type: Warning,
<        message: "File parent_folder\\missing.vhd does not exist",
>        message: "File parent_folder/missing.vhd does not exist",
     },
 ]

', vhdl_parser\src\config.rs:344:9

---- config::tests::config_from_str stdout ----
thread 'config::tests::config_from_str' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 [
     "C:\\Users\\mark_\\AppData\\Local\\Temp\\.tmphdYH7e\\pkg2.vhd",
>    "\\\\?\\C:\\Users\\mark_\\AppData\\Local\\Temp\\.tmpfc9mko\\absolute.vhd",
 ]

', vhdl_parser\src\config.rs:266:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.


failures:
    config::tests::config_from_str
    config::tests::test_append_config
    config::tests::test_warning_on_emtpy_glob_pattern
    config::tests::test_warning_on_missing_file

test result: FAILED. 521 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out

VHDL configuration

following configuration seems correct (accepted by ghdl), but syntax error is raised:

configuration dut_sim_config of top is
for structural
	for inst : comp
		use entity ent
		port map (
            a => a
        );
	end for;
end for;
end dut_sim_config;

Support adding files to built-in IEEE library

In order to add e.g. VITAL packages to the IEEE library, the entire ieee library must be redefined as any vhdl_ls.toml defining an ieee library with the VITAL files will override the built-in definition. Same problem occurs for the Synopsys libraries,

• VITAL could probably be supported out of the box using VITAL 2000, I don't think that there are any critical differences for an editor. The license is unclear as the header does not state any copyright or license (GHDL distributes it though).
• Synopsys libraries should be supported out of the box by vhdl_ls in order to better support legacy designs. The license states that they can be freely distributed.

There would still be some corner cases with e.g. Mentor's packages but those could be solved by re-defining ieee.

postponed process not fully supported

myproc: postponed process(i_clk, i_rst) is
begin
 -- blablabla
end postponed process;

The parser will break at the end postponed process line because it isn't expecting the postponed keyword

Add graceful handling of non-latin-1 characters

Currently the parser/LSP panics when it encounters a non latin-1 character:

thread 'main' panicked at 'Source was not legal latin-1: Custom { kind: Other, error: "Invalid latin-1 character � in -- ...

VSCode will attempt to restart the server leading to a new crash and after 5 crashes within 3 minutes VCode will no longer try to start the server. To start the server again, VSCode must be restarted.

It would be better if an error was emitted for the location of the non valid character. The parser can then either stop parsing the file or try to recover.

Analysis order

A goal of the analyser is to automatically compute the analysis order of design units such that the user does not have to care about it.

Automatically computing the analysis order requires traversing the entire AST and finding all use clauses, packages instances as well as instantiations. Circular dependencies might occur which should trigger an error.

Simple case

use lib.unit;

Simple case, introduces depenency on lib.unit

Harder case

use lib.all;

Trickier case especially if lib is work. All names within lib cannot be considered a dependency automatically since it might create circular dependencies especially for work but also with co-dependent libraries.
We will have to check which names are actually used and only then create the dependency.

use lib.all; -- contains pkg
use pkg.foo; -- Only here we create the depenency on lib.pkg

Corner case

This also unfortunately interacts with normal declarative part analysis, for example.

use lib.all; -- contains pkg
package pkg is new gpkg;
use pkg.foo; -- foo might be a nested package instance within pkg
-- Should we consider lib.pkg a dependency here, what if pkg was a local constant?

Maybe it is ok to falsely introduce a dependency here to simplify the logic without any practical problem.

Potential solutions

I see two solutions to making automatic analysis order work.

1. Separate pass to compute the analysis order

Have a separate pass to explicitly only compute the analysis order.

pros

• Clean with an explicit pass to extract dependencies.
• Can be used directly to emit a compile order for use by other tools
• Can be used to determine when things do not need to be re-analysed in case of incremental changes.
• Might be used to compute opportunities for parallel analysis.
  cons
• A separate pass costs performance
• Potentially duplicate code with normal declarative part analysis

2. Compute analysis order with recursion

Compute the analysis order concurrently with other analysis implicitly. If the use of an unanalysed unit is encountered we re-curse and analyse the unit. Units will have to be flagged to detect circular dependencies.

pros

• Single pass
• Can share code with normal scope analysis

cons

• Maybe will run into stack overflow for large project if unlucky
• Analysis order is not explicitly computed and extra structure must be created to extract it
• Analysis order can not be had without running full analysis
• Analysis order is not known before starting full analysis which might inhibit parallelism

vhdl_ls and vhdl_parser are not able to parse standard package generated by ghdl --disp-standard

I tried to set up vhdl_ls for my VHDL project, for which I use ghdl for simulation. To fully profit of vhdl_ls, I also created a vhdl_ls.toml file referencing my project sources as well as the sources of the used packages from libraries std and ieee. I found the sources of the standard packages in my GHDL installation below $PREFIX/lib/ghdl/src/. I only missed the source of std.standard. Using the standard.vhd file from your example_project, I was able to parse my project sources and use vhdl_ls in my Emacs. However, your standard.vhdis for VHDL'08 and my project is still restricted to VHDL'93. So, I looked wether ghdl could provide the source of std.standard (cf. to ghdl issue #802). @tgingold pointed me to use ghdl --disp-standard --std=93 to generate the VHDL source of std.standard.

Unfortunately, vhdl_parser and vhdl_ls fail to parse the generated VHDL source. vhdl_parser generates parsing errors on at least the definition of type UNIVERSAL_INTEGER (Integer too large for 64-bits signed), real(invalid float literal due to use of + in exponent), and time (same as for UNIVERSAL_INTEGER). It would be great if rust_hdl's VHDL parser could be enhanced to parse the std.standard package VHDL source generated by ghdl --disp-standard for all supported language standards by ghdl (VHDL'87, VHDL'93, VHDL'08).

To reproduce the issue, build ghdl and rust_hdl from their respective master branches. Then, generate the std.standard package VHDL source and try to parse it using vhdl_parser of rust_hdl:

$ ghdl --disp-standard --std=87 > standard.v87
$ ghdl --disp-standard --std=87 > standard.v93
$ ghdl --disp-standard --std=87 > standard.v08
$ vhdl_parser standard.v87
$ vhdl_parser standard.v93
$ vhdl_parser standard.v08

For example, this is the output for standard.v93:

$ vhdl_parser standard.v93
Error when parsing standard.v93
error: Integer too large for 64-bits signed
    --> standard.v93:103
     |
101  |    -- function "<=" (<anonymous>: severity_level;
102  |    --                <anonymous>: severity_level) return boolean;
103 -->   type UNIVERSAL_INTEGER is range -9223372036854775808 to 9223372036854775807;
     |                                     ~~~~~~~~~~~~~~~~~~~
104  |    -- function "=" (<anonymous>: <UNIVERSAL_INTEGER>;
105  |    --               <anonymous>: <UNIVERSAL_INTEGER>) return boolean;

error: Expected 'library', 'use', 'context', 'entity', 'architecture', 'configuration' or 'package'
    --> standard.v93:103
     |
101  |    -- function "<=" (<anonymous>: severity_level;
102  |    --                <anonymous>: severity_level) return boolean;
103 -->   type UNIVERSAL_INTEGER is range -9223372036854775808 to 9223372036854775807;
     |                                                         ~~
104  |    -- function "=" (<anonymous>: <UNIVERSAL_INTEGER>;
105  |    --               <anonymous>: <UNIVERSAL_INTEGER>) return boolean;


Summary:
Found no warnings
BAD: Found errors in 1 files

LSP should handle config file changed and/or created after startup

Currently Rust HDL looks for the config file when the LSP is started, so any changes to it (for example adding files, fixing libraries, etc) require the server/editor to be restarted.

Ideally, I think restarting or refreshing the server should happen transparently to the user, so that the process of changing and fixing the config file is faster.

Add support for directories or file patterns in vhdl_ls.toml

Being able to configure the language server to recursively include a directory would simplify project setup. A common practice is to have all RTL sources in a single directory and it should be possible to point to this directory in vhdl_ls.toml. Implementing support for directory references would remove the need to update vhdl_ls.toml when new files are added to a project.

To account for some variability, it could also be possible to configure include and exclude patterns for the files within the directory.

Suggestion to expand vhdl_ls.toml to:

1. Support glob-style patterns in library.files (glob::glob?)
2. Allow entry in files array to be a directory (checked during config parse) in which case the directory is searched recursively for .vhd files (**/*.vhd)
3. Add excludes field to library to filter out unwanted files. Supports the same pattern styles as library.files.

Example:

[libraries]
lib1.files = [
    'src',
    'tests/*/tb_*.vhd',
]
lib1.exclude = [
    'test/*/*_mock.vhd',
]

Labeled "end if" not supported

specifying a label for an end if clause is not supported.

mylabel: if condition then
   call_function(a,b,c);
end if mylabel;
       ^^^^^
       Expected ';'

Note that labelled if statement using a regular non-labelled end if is supported

Visibility rules

The next step is to work on visibility rules for use clauses and nested scopes.

Keywords: potentially visible, directly visible, implicit, explicit, declarative region, immediate scope.

I have identified the following rules.

Immediate scope always takes precedence for non-overloaded

status: Done

constant ident : natural := 0;
use work.pkg.ident;
-- local ident trumps use
constant ident2 : natural := ident;

status: Pending analysis of formal part of association

 use work.pkg;
 type rec_t is record
    const : natural;
 end record;
  constant pkg : rec_t := (const => 0);
--  constant pkg : natural := 0; Error if pkg was not record in selected name below

  -- Uses record
  constant const3 : natural := pkg.const;

status: Pending resolving overloaded names by signature

package pkg is
  procedure foo;
end package;

package body pkg is
  procedure foo is
  begin
    report "pkg foo";
  end procedure;
end package body;

entity ent is
end entity;

architecture a of ent is
  procedure foo is
  begin
    report "arch foo";
  end procedure;
begin
  main : process
    use work.pkg.foo;
  begin
    foo; -- arch foo
  end process;
end architecture;

Multiple non overloaded names require disambiguation

status: Done

use work.pkg1.ident;
use work.pkg2.ident;
-- Error requires disambiguation event if pkg1.ident and pkg2.ident are different types
-- Also true if .all was used
constant ident2 : natural := ident;

-- The following would work without ambiguity though
use work.pkg1.ident;
constant ident2 : natural := ident;
use work.pkg2.ident;

Multiple overloaded names do not require disambiguation

status: Pending resolving overloaded names by signature

package pkg is
 function fun return natural;
 function same return natural;
end package;

package pkg2 is
  function fun return boolean;
  function same return natural
end package;

use work.pkg.all;
use work.pkg2.all;
-- Works
constant ident : natural := fun;
-- Requires disambiguation
constant ident2 : natural := same;

Mixed overloaded names require disambiguation

status: Done

package pkg is
 function fun return natural;
 function same return natural;
end package;

package pkg2 is
  constant fun : boolean := false;
  constant same : natural := 0;
end package;

use work.pkg.all;
use work.pkg2.all;
-- Requires disambiguation
constant ident : natural := fun;
-- Requires disambiguation
constant ident2 : natural := same;

Enum literals are automatically visible when using type

status: Done

package pkg is
  type enum_t is (v1, v2);
end package;

use work.pkg.enum_t;
constant ident: enum_t := v1;

Enum literals are overloaded

status: Done

package pkg is
  type enum_t is (v1, v2);
end package;

package pkg is
  type enum2_t is (v1, v2);
end package;

use work.pkg.enum_t;
use work.pkg2.enum2_t;

-- works because of oveloading
constant ident: enum_t := v1;

status: Done

package pkg is
  type enum_t is (v1, v2);
end package;

package pkg is
  constant v1 := 0;
end package;

use work.pkg.enum_t;
use work.pkg2.v1;

-- Ambiguous
constant ident : enum_t := v1;

May use nested packages

status: Done

package gpkg is
  generic (constant init : natural);
  constant const : natural := init;
end package;

package pkg is
  use work.gpkg;
  package ipkg is new gpkg generic map (init => 1);
end package;

package pkg2 is
  use work.pkg.ipkg.all;
  constant ident : natural := const;
end package;

Library names are nothing special

status: Done

library lib;

package pkg is
  type rec_t is record
    pkg : natural;
  end record;

  constant work : rec_t := (pkg => 0);
  constant foo : natural := work.pkg;

  constant lib : rec_t := (pkg => 0);
  constant foo : natural := lib.pkg;

  -- Should not work
  use lib.pkg.all;
end package;

Enum literals must be handled specially when they are implicitly visible

status: Requires type inferences to make names unambigous. Currently no error

package pkg is
  type enum_t is (v1, v2);
  type enum2_t is (v2, v1);
end package;

use work.pkg.enum_t;
-- Ambiguous if not commented out
-- use work.pkg.enum2_t;

package pkg2 is
  -- A use of an enum type should not bring make all v1 names visible only the one from the specific typ. 
  constant const : string := v1'instance_name;
end package;

Directly visible declarations always take precedence over use clause even when nested

status: Done

package pkg is
  constant const : natural := 0;
end package;

entity tb_ent is
end entity;

architecture a of tb_ent is
  constant const : natural := 1;
begin
  main : process
    use work.pkg.const;
  begin
    assert const = 1; 
  end process;
end architecture;

Nested use clauses still conflict

status: Done

package pkg is
  constant const : natural := 0;
end package;

package pkg2 is
  constant const : natural := 2;
end package;

entity tb_ent is
end entity;

architecture a of tb_ent is
  use work.pkg.const;
begin
  main : process
    use work.pkg2.const;
  begin
    report integer'image(const); -- Should be an error as const is ambiguous
  end process;
end architecture;

Implicitly defined items never conflict with explicit ones

status: Pending

package pkg is
  function MINIMUM (L, R: REAL) return REAL;
end package;

package body pkg is
  function MINIMUM (L, R: REAL) return REAL is
  begin
    return L + R;
  end;
end package body;

entity ent is
end entity;

use std.standard.MINIMUM;
use work.pkg.MINIMUM;

architecture a of ent is
begin
    main : process
    begin
      report real'image(MINIMUM(1.0, 2.0)); -- Not ambiguous, prints 3.0
    end process;
end architecture;

Crates on https://crates.io/

On https://crates.io/ there are two crates vhdl_parser and vhdl_lang. Both point to this repository. However there is no directory named vhdl_parser, even when you grep for vhdl_parser you get nothing. Can you explain how vhdl_parser is created and what is the difference between vhdl_parser and vhdl_lang?

Goto definition/declaration does not work for functions and procedures

While it works fine on most of the definitions, running "Goto definition" on function or procedure calls seems not to be implemented yet.

(Tested on VS Code running current master build 189c98a)

Update crates.io API token

v0.16.0 crates.io release action failed due to API key revocation.

error: api errors (status 401 Unauthorized): The given API token does not match the format used by crates.io. Tokens generated before 2020-07-14 were generated with an insecure random number generator, and have been revoked. You can generate a new token at https://crates.io/me. For more information please see https://blog.rust-lang.org/2020/07/14/crates-io-security-advisory.html. We apologize for any inconvenience.

Logic behind declaration search.

@kraigher I struggle to analyze how the declaration position is determined when LSP client asks for Go to Declaration. Everything is clear until search_reference method is called on DesignRoot

    pub fn search_reference(&self, source: &Source, cursor: Position) -> Option<SrcPos> {
        ItemAtCursor::search(self, source, cursor)
    }

If I understand correctly, first the item under the cursor is found, what is done with ItemAtCursor. By the way, I have temporarily modified is_inside function to print some debug info, and it looks like this function returns true twice for the same cursor and pos values during the single Go to Declaration procedure.

    fn is_inside(&self, pos: &SrcPos) -> bool {
        // cursor is the gap between character cursor and cursor + 1
        // Thus cursor will match character cursor and cursor + 1
        if pos.start() <= self.cursor && self.cursor <= pos.end() {
            error!("{:#?}", self.cursor);
            error!("{:#?}", pos);
            return true
        } else {
            return false
        }
    }

Output:

[2020-08-03T13:35:53Z ERROR vhdl_lang::ast::search] Position {
    line: 91,
    character: 18,
}
[2020-08-03T13:35:53Z ERROR vhdl_lang::ast::search] SrcPos {
    source: Source {
        source: Source {file_name: "/home/mkru/workspace/smx/rcv_frames_shuffler/rtl/rcv_frames_shuffler.vhd"},
    },
    range: Range {
        start: Position {
            line: 91,
            character: 12,
        },
        end: Position {
            line: 91,
            character: 20,
        },
    },
}
[2020-08-03T13:35:53Z ERROR vhdl_lang::ast::search] Position {
    line: 91,
    character: 18,
}
[2020-08-03T13:35:53Z ERROR vhdl_lang::ast::search] SrcPos {
    source: Source {
        source: Source {file_name: "/home/mkru/workspace/smx/rcv_frames_shuffler/rtl/rcv_frames_shuffler.vhd"},
    },
    range: Range {
        start: Position {
            line: 91,
            character: 12,
        },
        end: Position {
            line: 91,
            character: 20,
        },
    },
}

Nevertheless, what I do not understand is what happens when the item under the cursor is already found. How is the actual declaration found? The whole search logic looks a bit distributed to me, and I find it hard to get the overall view.

Problems reported for standard libraries can severely clutter important issues

If adding a vhdl_ls.toml without all standard libraries used, the workspace can get disgustingly cluttered (i.e. 15k problems reported in VSC). I don't have all the libraries for some things as I jump around a lot and would rather not point to them in multiple workspaces. Can a check for standard libraries be added along with a flag to possibly ignore these? The part I care about is the wiring between design files and syntax, not that I don't have my environment configured to point to all standard libraries and vendor libraries.

Replace from_utf8_unchecked with from_utf8

Conversion from slice of bytes to utf-8 in latin_1.rs is done with from_utf8_unchecked.
From documentation:

Converts a slice of bytes to a string slice without checking that the string contains valid UTF-8.

I think it has been chosen to achieve better performance, however in case of invalid UTF-8, the later execution may result with undefined behavior.

Getting root of syntax tree for particular file.

I would like to add functionality for generating .dot files with syntax tree graphs. It requires the root of the syntax tree for particular file.

Now, correct my if I am wrong anywhere.

It is impossible right now, because design_unit is a root for single syntax tree, not design_file. However, it looks like it is possible to get design units from particular file. First I need to get DesignRoot from Project. Then single Library from DesignRoot. Next, from units_by_source hash map I can get UnitId for units coming from the file I am interested in. From UnitId I can fetch UnitKey. Then I have to come back to the Library and from units I can fetch LockedUnit. LockedUnit contains unit, which contains AnyDesignUnit, which is the root of syntax tree for particular design unit.

I have mixed attribute names with type names, but I hope you understand.

If you had any implementation guidelines I'd really appreciate.

user-defined attributes for some entity classes are not supported

Similar to #31 label, subtype, literal, units,group,file,property and sequence is also not supported for user-defined attributes.

See IEEE Std 1076-2008 7.2 (p. 95):

attribute_specification ::=
    attribute attribute_designator of entity_specification is expression ;

entity_specification ::=
    entity_name_list : entity_class

entity_class ::=
    entity
    | architecture
    | configuration
    | procedure
    | function
    | package
    | type
    | subtype
    | constant
    | signal
    | variable
    | component
    | label
    | literal
    | units
    | group
    | file
    | property
    | sequence

Syntax Highlighting of pure/impure function

EDIT: Ignore this Issue. This issue belongs in the vscode client repo. sry

"pure function ..." and "impure function ..." don't seem to Syntax highlight properly.
The Problem seems to be that the match for "function" only takes whitespace before "function".
Removing "^\s*" in the file vhdl.tmLanguage fixes the Problem for me.

       <key>function_prototype_pattern</key>
        <dict>
            <key>patterns</key>
            <array>
                <dict>
                    <key>begin</key>
                    <string>(?x)
                        # From the beginning of the line
                        ^\s* <-- remove this line
						
                        # The word function $1
                        ((?i:function))\s+
                        ...

        <key>function_definition_pattern</key>
        <dict>
            <key>patterns</key>
            <array>
                <dict>
                    <key>begin</key>
                    <string>(?x)
                        # From the beginning of the line
                        ^\s* <-- remove this line

                        # The word function $1
                        ((?i:function))\s+
                        ...

Sublime Plugin

I use Sublime as editor and would like to try this out. Does anyone know if you can/should add sublime package to the main repo, or a separate one?

GHDL has a linter plugin implemented as a separate repository.

Externally visible generics

There is uncertainty in the standard regarding externally visible generics:

Se GHDL issue:
ghdl/ghdl#440

package gpkg is
  generic (const : natural);
end package;

package ipkg is new work.gpkg generic map (const => 1);

entity ent is
end entity

architecture a of ent is
begin
  main : process
     -- Case 1
     assert work.ipkg.const = 1; -- Should this result in a 'no declaration of const' error?
     -- case 2     
     assert << constant @work.ipkg.const : natural>> = 1; -- Should this be visible?
  end process;
end architecture;

Block comments are not supported

Block comments (/* */) are not supported.

See IEEE Std 1076-2008 15.4 (p. 234):

A comment is either a single-line comment or a delimited comment. A single-line comment starts with two adjacent hyphens and extends up to the end of the line. A delimited comment starts with a solidus (slash) character immediately followed by an asterisk character and extends up to the first subsequent occurrence of an asterisk character immediately followed by a solidus character.

Status of the project.

@kraigher Is there anywhere information about what you are working on right now, and what are the plans for near future?

lsp-define-stdio-client deprecated

Hi
It seems that lsp-define-stdio-client is deprecated and unless I'm missing something has been completely removed. I was trying to follow your example for lsp-mode in Emacs but got an error that lsp-define-stdio-client cannot be found.
I'm using Emacs 26.2 and latest version of the elp-mode package.

User-defined attributes not supported

I've been trying rust_hdl with my projects which are mostly targeting Xilinx FPGAs with the Vivado tools. It's looking very, very useful (thanks for the hard work!), but I've got a bit of a problem with the parsing.

The tool throws a lot of errors with the message

Expected 'entity', 'architecture', 'configuration', 'package', 'signal', 'variable', 'procedure' or 'function'

wherever user-defined attributes are used. Typically there are only a few of these in my own VHDL files, but when I include the Xilinx component libraries I have thousands of errors.

The problem also prevents other errors being thrown (related to #25 I guess). In the sample below there is an extra comma on the instantiation which is not flagged.

Here's a sample which shows the problem. It happens whether or not there's a vhdl_ls.toml included.

sample.vhd:

entity sample1 is
	port (
		CLK : in std_logic;
		nCLK : out std_logic
	);
end entity;

architecture a of sample1 is

	component sample2 is
		port (
			CLK : in std_logic;
			nCLK : out std_logic
		);
	end component sample2;

	attribute BOX_TYPE : string;
	attribute BOX_TYPE of sample2 : component is "user_black_box";

begin
	Inst_sample2 : sample2
		port map	(
			CLK <= CLK,
			nCLK <= nCLK,
		);
end a;

entity sample2 is
	port (
		CLK : in std_logic;
		nCLK : out std_logic
	);
end entity;

architecture a of sample2 is
	signal s_nclk : std_logic;
begin
	s_nclk <= not CLK;
	nCLK <= s_nclk;
end a;