degory/ghul-test

ghūl compiler integration test runner

repositories for the ghūl programming language, its compiler and supporting tools

ghul

compiler for the ghūl programming language

ghul-vsce

a Visual Studio Code extension providing ghūl language support

ghul-runtime

low level dependencies required by all ghūl applications

ghul-templates

templates for the .NET new command which create new ghūl console application and class library projects

ghul-repository-template

a template for creating a new ghūl project repository on GitHub

ghul-test

a snapshot based integration testing framework used by the ghūl compiler project

ghul-examples

ghūl programming language examples

ghul-web-api

ghūl ASP.NET web API example

ghul-dev

ghūl programming language website

ghul-test's Issues

Tests can fail if ghul.runtime version differs from runtime used by the compiler under test

Because the compiler being tested is not being run by MSBuild it doesn't get passed reference assemblies based on NuGet package and SDK dependencies. In particular the reference to the ghūl runtime assembly is bodged by symbolically linking to the runtime the ghul-test application was built with. This works provided the compiler has also been built with the same runtime version, but if not that it can cause some tests to wrongly fail due to being unable to load types from the runtime.

Workaround is to update the runtime for ghul-test immediately before updating the runtime to the same version for the compiler

Lazy static initialize of _collate_environment is not thread safe

ghul-test/src/runner.ghul

Line 48 in e64e4c6

if !_collate_environment? then

This lazy static initialization is not thread safe and could be responsible for spurious test failures (if the environment was used by another worker thread before being fully initialized then diff and sort child processes could return incorrect results).

It probably could be made safe with a combination of volatile intrinsics and System.Threading.Monitor but there's no benefit really - it can simply be initialized ahead of time before any worker threads are started.