mikepopoloski/slang

SystemVerilog compiler and language services

slang - SystemVerilog Language Services

slang is a software library that provides various components for lexing, parsing, type checking, and elaborating SystemVerilog code. It comes with an executable tool that can compile and lint any SystemVerilog project, but it is also intended to be usable as a front end for synthesis tools, simulators, linters, code editors, and refactoring tools.

slang is the fastest and most compliant SystemVerilog frontend (according to the open source chipsalliance test suite).

Full documentation is available on the website: https://sv-lang.com

Features

• Fully parse, analyze, and elaborate all SystemVerilog features - see this page for current status.
• Be robust about compilation, no matter how broken the source text. This makes the compiler usable in editor highlighting and completion scenarios, where the code is likely to be broken because the user is still writing it.
• The parse tree should round trip back to the original source, making it easy to write refactoring and code generation tools.
• Provide great error messages, ala clang.
• Be fast and robust in the face of production-scale projects.

Use Cases

Some examples of things you can use slang for:

• Very fast syntax checking and linting tool
• Dumping the AST of your project to JSON
• Source code introspection via included Python bindings
• SystemVerilog code generation and refactoring
• As the engine for an editor language server
• As a fast and robust preprocessor that sits in front of downstream tools
• As a frontend for a synthesis or simulation tool, by including slang as a library

Getting Started

Instructions on building slang from source are here.

The slang binary can be run on your code right out of the box; check out the user manual for more information about how it works.

If you're looking to use slang as a library, please read through the developer guide.

Try It Out

Experiment with parsing, type checking, and error detection live on the web (inspired by Matt Godbolt's excellent Compiler Explorer).

Contact & Support

If you encounter a bug, have questions, or want to contribute, please get in touch by opening a GitHub issue or discussion thread.

Contributions are welcome, whether they be in the form of bug reports, comments, suggestions, documentation improvements, or full fledged new features via pull requests.

License

slang is licensed under the MIT license:

Copyright (c) 2015-2024 Michael Popoloski

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

slang's Issues

Update Parser code to use new SyntaxFactory methods

Simple cleanup to make the code more readable.

Make token concatenation more robust

The SystemVerilog preprocessor supports token pasting (like the C preprocessor). This is implemented in Lexer::concatenateTokens.

There are a bunch of TODOs in the code that cover things like testing all corner cases, better error handling, accounting for trivia in either token, and correctly setting the resulting token's location for proper expansion reporting in diagnostics.

Make timescale directives more robust

The initial implementation of timescale directives needs error checking for robustness.

Support macro-replaced include file names

Include file names can actually be specified as a macro that should be expanded; this is fairly straightforward but pretty annoying since a bracketed include name will show up as a whole bunch of tokens in the stream. C++ supports this as well, so we can take inspiration from GCC or Clang here.

Finish and test hierarchy symbol binding

Module symbol binding has been started, and good progress has been made on binding all parameters. Any useful thing we might want to do with the compiler requires at least having the hierarchy bound, so make sure that all works correctly.

Add the concept of "controlling macros" to include files

Compilers like GCC and Clang notice the common ifndef define pattern at the top of include files and use it as a marker to skip lexing the entire file if it's already been seen. As an optimization we should do the same.

This is low priority.

Make SyntaxNode as<> casting do a sanity check

Add some safety to the as() method on SyntaxNodes, which currently just does a blind cast. We know the Kind of the node at runtime, we should be able to check that against the static type given by the template parameter and assert.

Handle parsing properties and sequences

See title.

Handle instance arrays

Rework Module instance symbols

Rather than have a ParameterizedModuleSymbol, move that into the instance itself.

Fix BumpAllocator to call destructors where necessary

BumpAllocator was never designed for types with non-trivial destructors, but we've started using it that way. It should support detecting types that require this and maintain a separate list so that we can call them.

Design an interface for backend code generation

So far the project has focused on parsing and semantic analysis; the "front end" of the compiler. We need to think about how to take symbols and bound expression trees and generate output from that.

We'll eventually want several backends; LLVM for simulation, netlists for hardware synthesis. For now though, I think we want to go to simplified Verilog for quicker bringup.

Make module instantiation and parameter evaluation robust again

Once refactoring is finished, re-enable all of the various error checking tests.

Make sure escaped identifiers are not turned into keywords

Identifier tokens that match a keyword name are turned into the appropriate keyword; make sure this isn't happening for escaped identifiers and write a test to verify it.

Account for tabs when positioning caret and ranges in diagnostics

DiagnosticWriter::formatDiag should take into account the tab stops of the output terminal when positioning the source location caret and any source ranges.

Set correct token locations in all corner cases during macro expansion.

Make sure even in weird scenarios that all expanded tokens get correct locations assigned.

Fix const issues in SyntaxNode

There are a few places where we const_cast in SyntaxNode out of laziness; add another overload to getChild() on each derived type so that we don't need to do this.

Stop lexing if we encounter too many errors

The lexer tries to handle non-ASCII characters embedded in the source gracefully. If there are too many of them however, maybe the user accidentally pointed us at a binary file or something. Keep a counter and if we hit some threshold just swallow the rest of the text instead of producing a billion errors.

Reorganize source directory

After things have settled down a bit, go through and move everything into folders, like the original filter setup in the vsproj.

Implement __FILE__ and __LINE__ macros

The intrinsic FILE and LINE macros are currently recognized but generate empty tokens ("" and 0 respectively). The LINE macro in particular will require making use of the source manager infrastructure for figuring out which line we're looking at.

Note that this plays into the `line directive as well, which changes the "apparent" line number (like #line in C++).

Handle interface declarations and instantiations

Make SVInt allocator aware

Temporaries produced during constant evaluation should all live in a temporary heap that we can just throw away.

Make macro stringification more robust

Implemented in Lexer::stringify. It's not always clear what kind of spacing should be in the resulting string; the standard doesn't have much to say about it. Probably we should take a look at what existing Verilog compilers do and try to match them.

Make sure all loops in the parser correctly handle skipping tokens

There are cases now where running into a syntax error in a loop will cause it to loop forever fabricating new tokens via expect().

Finish type symbol binding

Type symbol binding has only a very bare bones implementation; there are many cases left to handle.

Finish full robust error handling for macro expansion, concatenation, and stringification.

There are a bunch of TODOs to handle corner cases and report errors appropriately. These should be worked out.

Finish adding features to the parser

The parser can currently handle 75-80% of the language, and handles the most common parts. There is definitely code out there using the parts we don't yet handle though.

This involves adding new syntax nodes to syntax.txt, adding a new type to SyntaxKind, and then making the parser produce the new nodes. It's pretty straightforward one you do one or two of them.

Create a preprocess utility program.

Useful as a standalone tool and for checking the preprocessor.

Expand LValue support in ConstantEvaluator

Currently we only support simple variable lvalues, but there are others like concatenation.

Add checks to prevent stack overflow in the parser

There's a test that's currently commented out that has ridiculously nested code to test overflow cases in the compiler. Since there's no handling right now, it predictably crashes. Whenever we have parser calls that have the potential to recurse on themselves, we should add some guards to check for nesting that's too deep and issue an error if we hit the threshold.

Throw errors when encountering special preprocessor tokens outside of a macro define

For tokens like " and ", they must occur inside a macro definition.

Perfect hashing for keywords

This would be a nice-to-have: the StringTable used to lookup whether an identifier is a keyword could use an offline-computed perfect hash to make it much faster.

Finish and test expression binding

This could probably be broken down into a bunch of issues. The ExpressionBinder has basic implementation for arithmetic operators and simple variable lookup, but a bunch of features aren't done yet. Examples of things that make this hard are supporting function calls at compile time (for constant expressions).

We should take a look at what Clang does for constant expression evaluation; in contexts where we know we need a constant expression, the expression binder will keep track of reasons why it was unable to propagate a constant so that we can give useful error messages.

Handle remaining preprocessor directives

There are a few preprocessor directives that aren't yet handled. Implement them in Preprocessor::next().

Make symbol creation lazier

Rather than try to greedily create all symbols, the semantic model should only create each symbol and its children when it becomes necessary. This should solve ordering issues between having symbols created and evaluating things like initializers.

Add an options system

Add some way to pass options around so we don't hardcode various things like recursion limits.

Handle truncation when binding in assignment-like contexts

See Binder.cpp:147

Handle missing closing paren in macro definition

If the closing parenthesis is missing, the macro parser might loop forever. There is a TODO to handle skipping invalid tokens.

Support non-integral types in constant evaluation

Pretty much everything right now is hardcoded for integrals.

Make SourceManager thread-safe

SourceManager should be one of the few thread-safe components in the system. Someone just needs to do the work to make it that way.

Switch to a less ad-hoc build system

Use something like premake to generate project files instead of maintaining them by hand.

Cut down on the number of parser diagnostics

There are a bunch of "expected" diagnostics in the parser that could probably be collapsed down.

Check whether keywords are actually eligible given current PP state

SystemVerilog has a preprocessor feature that enables and disables various sets of keywords. We should take into account that state and downgrade keyword tokens back into simple identifier tokens.

Handle parsing cover points.

There is initial support for this but it's not complete.

Report an error when trying to redefine a built-in directive

The preprocessor should report an error if you try to redefine a built in directive.

Investigate making syntax nodes use reference types where possible

This should alleviate some of the null checking we currently do to be safe.

Support Macro Arguments which have keyword names

Like

`define FOO(type, var) type var;

type, var are keywords, but they need to be parsed as macro arguments and replaceable as macro arguments

Report a warning when redefining a macro

If the user redefines a macro, a warning should be reported unless the two macros have exactly the same bodies.

Include full relative path in error messages

Instead of just the file name, use the relative path to the file when printing error messages.

Start of line check for directives

Most preprocessor directives require that they start on their own line. Currently this isn't checked or enforced.