cluelang/clue

C/Rust like programming language that compiles into Lua code

The Clue programming language

Clue is a programming language that compiles blazingly fast into Lua code with a syntax similar to languages like C or Rust.

Clue tries to be almost as simple as Lua (with only a slightly more advanced syntax) but adds many optional features that can make code look better or make some things (like metatables) easier to code.

Clue does not compile to a specific version of Lua: flags can be toggled to alter the output to allow most if not all versions or modifications of Lua to be compiled to with Clue.

General syntax differences

• Code blocks are now inside {} instead of then/do/repeat and end/until
• Comments are made with // ... or /* ... */

If you want a complete documentation of every change and addition in Clue check the wiki.

Example code

@ifos linux {
	@define GREETING "Hello, Linux user "
} @else_ifos macos {
	@define GREETING "Hello, MacOS user "
} @else {
	@define GREETING "Hello, Windows user "
}
  
@macro GREET(target) { $GREETING .. $target .. "!" }
  
print($GREET!("Maiori"))

local fn add(x = 0, y = 0) {
    return x + y
}

global n = 1

while n < 10 {
    n += add($, $)
    match n {
        3 => {
            continue
        }
        4 if x => {
            break
        }
        default => {
            print(n < 3 ? n : -n)
        }
    }
}

More examples can be found in:

• the wiki
• the Examples directory
• a game made with LÖVE
• Clue's MessagePack library
• a simple graph visualizer made with LÖVE

How to install

Using Cargo

1. Paste and run this command in the console: cargo install clue
2. Type clue in the console to run the compiler, it will explain the rest

Clue supports extra features that can be toggled when installing:

• interpreter: adds the --execute flag to let Clue run the generated output using mlua
• rpmalloc: uses rpmalloc to improve performance, not available on all platforms

By default Clue enables both features.

Using Linux packages

These can be downloaded in the latest release.

• .deb

sudo dpkg -i clue_<version>_<arch>.deb

• .rpm

sudo rpm -i clue-<version>.<arch>.rpm

Using the AUR

• With paru

paru -S clue

• With yay

yay -S clue

• With makepkg

git clone https://aur.archlinux.org/clue.git
cd clue
makepkg -si

Manual installation

1. Download the latest release and save it somewhere
2. Open your system environment variables
3. Add the path to the directory that contains clue.exe in the PATH variable
4. Type clue in your cmd/PowerShell to run the compiler, it will explain the rest

More coming soon!

There are still features that I'm considering adding and others that will be added soon. The most likely ones to be added in the future are:

• type system (coming in 4.0)
• better error messages (comming in 4.0)
• proper language server support (coming in 4.0)

For any suggestion or bug you can make a github issue. If you need help with the language itself, you can check out the Discord server.

I hope Clue will be useful to you :)

Why is Clue named Clue?

I have no clue.

clue's Issues

String match

All other string functions seem to work. I have tried string.sub, string.gsub, string.find, etc.
They all work. However, string.match does not seem to work.

Allow to choose target

Adding a --target flag for cross compiling
Since 3.0 is feature locked, this will be added in 3.1

Add Preprocessor

Here will be listed all the preprocessor directives to add into Clue 3.0:

• ifos
• ifdef
• ifcmp
• if
• else
• else_ifos
• else_ifdef
• else_ifcmp
• else_if
• define
• macro
• error
• print

Suggest any other directive in the comments

Unsafe code

As of now, the code is littered of unsafe. The proposal is to change the code with the ultimate goal of removing all unsafe blocks from the codebase.
First of all, the finaloutput variable should not be global: there's no reason for it because there are other possible ways to achieve the same thing, some of those include better readability.
About the ENV variables, the same thing as above applies.

Possible solution: create a struct where everything could be loaded inside.

2.5.1's new dependency likely os-dependant

mlua with the vendored feature seems to only work out of the box for linux, which would mean we should use another dependency (rlua?)

I am not 100% sure if this is the issue with 2.5.1, more testing required

Optimize outputted Lua code

As per title, optimize the outputted Lua code.
There are always room for optimizations here and there in the code, so adding a step inside the compiler between the parser and the compiler.
This feature should be added in 4.0

Expand macro functionality

If macros could change behaviour based on their args they could get much more useful

Update installation method

To build and install the compiler on Linux simply do git clone https://github.com/ClueLang/Clue.git && cd clue && cargo install --path .
For other OS, it should be similar to this command

[Idea] Compiler from Lua to clue

If for example you had a big Lua codebase that you wanted to convert to clue it would be a bit of a pain. The job of this compiler would be to convert Lua code blocks(then/do/repeat, end/until) into clue code blocks (curly braces). The difficult part on this is parsing Lua.

Edit: I might try to implement it myself

Debug comments

Debug comments are useless in the compiled file unless used for debugging.
Change the default compiler's behavior to enable the --nodebugcomments flag and add a new flag to do the reverse: --debugcomments

[Bug] nil type isn't inferred in multiple assignments

Problem

Given this example

local a,b,c = 1

the compiler produces the following AST

[
    VARIABLE {
        local: true,
        names: [
            "a",
            "b",
            "c",
        ],
        values: [
            [
                SYMBOL(
                    "1",
                ),
            ],
        ],
        line: 1,
    },
]

While not invalid the compilation step fails because the compiler can't handle b, c having no assigned value so the actual compilation steps fails

Backtrace

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/main.rs:191:94
stack backtrace:
   0:        0x108823824 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha91aabaedad056ac
   1:        0x10883d14b - core::fmt::write::h7f97e3fd364350dc
   2:        0x108820fd8 - std::io::Write::write_fmt::h0e066bfd479ec021
   3:        0x108824dbd - std::panicking::default_hook::{{closure}}::h32539b3840f9f60c
   4:        0x108824b14 - std::panicking::default_hook::hc1eaf31bd9d94572
   5:        0x1088252e8 - std::panicking::rust_panic_with_hook::h2173a17e4bf36be5
   6:        0x1088251ea - std::panicking::begin_panic_handler::{{closure}}::hc31bf00fee812eea
   7:        0x108823ca7 - std::sys_common::backtrace::__rust_end_short_backtrace::h91456626e7de558e
   8:        0x108824efa - _rust_begin_unwind
   9:        0x108848bd3 - core::panicking::panic_fmt::hf9bbc3974c4088ec
  10:        0x108848ab7 - core::panicking::panic::h2554e7c9fbba2a12
  11:        0x1087a6ed7 - clue::main::h1bb9af1451bca2f9
  12:        0x1087aad6b - std::sys_common::backtrace::__rust_begin_short_backtrace::hbaac911544a2b697
  13:        0x1087ae5a8 - std::rt::lang_start::{{closure}}::he990c930998175a4
  14:        0x10881cabe - std::rt::lang_start_internal::h1a04d881dc83f51a
  15:        0x1087a7079 - _main

Add safe calling `?(`

Would work the same way as safe indexing, but for function calls

Obfuscated code blocks and table after '||' operator

testcase:

clue code:

local a = nil
local b = a || {"123"}
print(b[1])

expected result：

local a = nil
local b = a or {"123"}
print(b[1])

clue v2.5.1 with run error:
Error in file "test.clue" at line 3!
Error: "Operator '||' has invalid right hand token"

Of course this error can be easily avoided：

local a = nil
local c = {"123"}
local b = a || c
print(b[1])

I'm not sure if the syntax of clue doesn't support || followed with table {}

Improve where internal variables are located

a = b
    ? c
    : d
        ? e
        : f

this is compiled into

local _internal0;
if d then
	_internal0 = e;
else
	_internal0 = f;
end
local _internal1;
if b then
	_internal1 = c;
else
	_internal1 = _internal0;
end
a = _internal1;

Which means the second ternary is always checked even if it might not be needed.
To solve this we could implement some sort of stack that points to where things like ternary should go.

local _internal1;
if b then
	_internal1 = c;
else
	local _internal0;
	if d then
		_internal0 = e;
	else
		_internal0 = f;
	end
	_internal1 = _internal0;
end
a = _internal1;

while parsing a ternary the code blocks used internally could be pushed to the stack, thus moving the internal variables there.
No idea how actually feasible this is, but I will try to add it in 3.3 once school stops being awful as usual.

Errors

Create and use error types inside the code

Improve debugger

clue --debug currently doesn't do much nor does it do it very well
Clue could perhaps embed lovebird (or something similar) into the output when compiling with the debug flag

Rename `arg!`

It seems to be about "arguments" although it's about "environment data".
I propose to rename it to env!

Distribution

Add Clue in some distro repos (like Debian, Arch, Solus, etc...)
Prob for 4.0

Compiler feature request

When I use Clue to generate lua, I found that the original comments are discarded, can the compiler add an option to keep the comments.

Table parsing needs a refactor

The function that parses table is badly written and buggy, it should be remade before 3.0, I'll fix it myself sometime soon

Multithreaded Buffered Reader

Implement, in a way or the other, a file reader that loads the content in a buffer and does so in another thread
The goal is to parallelize the code analyzer and the file reading to gain something with the performance (this change should be visible with larger files)
Prob this should be for 3.1 but can later be changed to 3.X or even 4.0 in case it misses the feature window

[Feature Request] conditional compilation support

As we all know that lua has no conditional compilation support, if we develop cross-platform code, lua will compile all the code into bytecode.

like:

if os.name() == 'windows' then
    -- some code
else
    -- some code
end

That will introduces unnecessary code on specific platforms.
So it would be great if the Clue language had conditional compilation support.

like C:

#ifdef xxx
    // code
#else
    // code
#endif

or Rust:

#[cfg(windows)]
//code

Use Lua's packages instead of `_modules`

There might be a way of putting Clue's modules directly into Lua's packages, that way require(...) could be used to load both...
This should analyzed more before proceeding and must definitively go to 4.0

Cannot set output file name when compiling a single file

Somehow this was never discovered

Tests

After what happened yesterday, the need of testing is rising. Every pr should firstly pass all the test and then can be reviewed.
I'll volunteer on this work.
What tests do you suggest to add?

Safe indexing to function call broken

The way 3.0 compiles function calls with safe indexing turns out to be broken and should be fixed asap

Multi-threaded compilation

At the current state, when called to compile an entire folder of Clue files, the compiler will compile the files one by one. That by itself isn't something bad, but for bigger projects it may take quite a lot of time to compile everything.
The proposal is as follows: change the behavior of compile_folder inside main.rs to add files in a queue to be processed by some compilation thread instead of compiling everything inside there.
This may even set a stepping stone for future work on multi-threading the compiler with fast compilations in mind (although as of now this is not really a necessity but with the planned features things may change)

Test to see if webhook works

Check for errors

At the current state, the compiler doesn't check if a function called in a file from another file is either local or global thus making errors go through the Lua version.
Example:
main.clue

local test = import("test")

local fn print_var(var) {
    print("Info " .. var)
}

print_var("Hello")
print_var(double(2))
print_var(multiply(2, 2))

test.clue

local fn multiply(a, b) {
    return a * b
}

global fn double(a) {
    return multiply(a, 2)
}

lua main.lua

Info Hello
Info 4
lua: main.lua:40: attempt to call a nil value (global 'multiply')
stack traceback:
        main.lua:40: in local 'cached'
        main.lua:11: in local 'import'
        main.lua:44: in main chunk
        [C]: in ?

Pseudo variables can't read `=>` properly

a = match x {1 => $}
$ becomes 1 instead of a

VecDeque

As to try ChatGPT, I asked it various questions about LinkedList vs VecDeque and it seems that for the current use case it'd be better using VecDeque instead of LinkedList.
Prob for 4.0

Library interface for clue

Clue needs a good library interface to be properly embeddable.

Ternary not reading pseudo variables

The code foo = $ ? $ : $ will incorrectly compile the first $ to nil and the 2nd and 3rd $s to _t0 while all 3 of them should probably be compiled to foo instead.

Add table destructuring in other places

currently table destructuring is hard-coded to be used in locals only (local {x, y, ...} = table), but it might be simple to add table destructuring in other places.

Block expressions

Currently, Clue does not support (first-class) block expressions.

The main challenge with introducing block expressions to Clue is how to distinguish them from table expressions.

Language features

There's the lack of a spec sheet for the language. Well, it is based on Lua and there's the wiki that explains the main differences but what about the new features you want to add like types? How do (or should) things work under the hood? What are the other planned features (3.0 and beyond)? How do you think these features should be implemented?

Documentation mistakes

• MATCH_BLOCK's name has no text
• FOR_LOOP is generally incorrect
• CONTINUE_LOOP and BREAK_LOOP rappresent the keyword only, not the entire loop
• Many ///TODO lying around (not really a mistake, but..)
• Check everything else

There are likely more I missed, so let's not close this until we are sure.

Casing

As it is now, the casing is all a mess + some of the functions and variables (and other things) don't follow the standard casing used in Rust.
What should be done about it?

what is the scanner based on ?

I have learned compiler knowledge , As far as I know , the scanner is based on transiton graph or nfa or dfa or regular expression . When I read the source code of scanner , few information about these terms present.
How do you start the scanner under the hood?

Feature Request: Official Linux support.

I'm aware that I can just compile this myself, but having an official build available either for download or in repositories would be nice.

VSCode Language Server Extension

I just found out about the project and wanted to fiddle with it a bit, but when I searched for a Clue extension on VSCode it wasn't there.
I think that's something really useful for the userbase and should 100% be implemented.
Here's the official guide on how to make one and this one seems to be a bootstrapper tool to make a syntax highlighter
Typescript example

Merge Clue-wasm and Clue

Since Clue-wasm is very outdated and Clue 3.0 will already use Cargo's workspace feature we could use it to merge the 2 repositories together.

New language features

• nil coalescing and nil coalescing assignment
• 0 indexed arrays
• destructuring
• logical assignment operators

Clippy errors

Running the current next branch with cargo clippy -- -D clippy::correctness -D clippy::complexity -D clippy::pedantic -D clippy::nursery -D clippy::perf -D clippy::cargo -D clippy::restriction gives 1425 errors and 4 warnings.
I'm pretty sure some stuff is useful to resolve but not everything
Prob for 3.1 but can later be changed to 3.X or 4.0

Docs

About docs, I'm not talking of the wiki (although that needs some rework) but about the codebase.
If one wants to start contributing to the project they need first to understand the code they'll work on, and the best way to learn about the code is to read what the code does.
Reading the code as-is may be something really difficult, here's why there's the need of documenting the code: to make the life easier to people that want to contribute on the project.

cursed

The code is cursed, we should call a chaman to get rid of that #![allow(non_camel_case_types)]

Add `if local`

Possible implementation:

if local x = expr {
    //...
}

which becomes

local x = expr
if x {
    --...
}

turn `enum` into `@enum`

when converting old 2.5 code to 3.0 I noticed I changed all enums to a bunch of @define, which made me realise enum would work better as a preprocessor directive that sets many preprocessor variables

`--pathiscode` no longer shows the output

--pathiscode used to print the output without needing --output, but that behaviour was lost in 3.0 somehow.

Condition indexing breaks on loop conditions

The following code:

local a = {}
while !(a?.b) {
	a = {b = true}
}

Will compile to:

local a = {};
local _internal0 = a;
while not (_internal0 and _internal0.b) do
        a = {
                b = true
        };
end

But since a is modified instead of _internal0, the condition will never be true even if it should.