jhauberg/lext

LEXT is Lexical Templates

LEXT

LEXT, or Lexical Templates, is a format specification and zero-dependency library for generating pseudo-random sequences of text through recursive substitution.

# lxt for magical weaponry

type (Axe, Sword)
element (Earth, Wind, Water, Fire)
prefix (Frozen, Fiery)

common <@type of @element>
magic <@prefix @common>

Fiery Sword of Water
Axe of Earth

Disclaimer:

Both library and specification has been built and established as an exercise more so than an attempt at making an actually useful thing.

Usage

Integrating and using the LEXT library is relatively straightforward.

You can build it as a static library that you link into your program, or just drop its few sources directly into your existing project. See Building for instructions on building the library from source.

Example

Here's a small program that generates and prints Hello World:

#include <lext/lext.h>

int32_t
main(void)
{
    char const * const format = "word (World, Hello) sequence <@word @word>";
    char buffer[64];

    uint32_t seed = 12345;

    lxt_gen(buffer, sizeof(buffer), format, (struct lxt_opts) {
        .generator = NULL,
        .seed = &seed
    });
    
    printf("%s\n", buffer);
    
    return 0;
}

Note that given the same seed, LEXT will generate an identical result on any platform.

Take a look in examples for more samples of usage.

CLI

The project provides a basic CLI for using LXT-patterns from a terminal. You just have to build it.

Building

LEXT can be easily built as a static library by using the included CMake scripts.

To generate build files for your system, make your way to the root of the LEXT repository and run CMake:

$ cd lext
$ cmake .

Format Specification

The LEXT format is simple and consist of only two basic concepts; containers and generators.

Containers

A container holds all the pieces of text that can be sequenced by a generator.

A LEXT can have any number of containers.

The following example defines a container named letter that holds the strings/characters a, b, c and d:

letter (a, b, c, d)

Generators

A generator defines the format and sequence of a generated output.

A LEXT can have any number of generators.

In this example, a generator scramble is defined. This generator then defines a sequence of 3 variables (indicated by a word starting with @), each pointing to a container named letter:

scramble <@letter, @letter, @letter>

When this generator is invoked and a result is to be sequenced, each variable will be replaced by a randomly picked item from the letter container. For instance, a result could be c, a, b, a, b, c or even a, a, a.

Sequences

A sequence can hold a variable that points to another generator. This makes sequencing fully recursive, allowing for more complex patterns.

For example, expanding on the previous scramble generator:

example <a few letters: @scramble>

When invoked, the example generator will sequence the initial text a few letters: before finally resolving and sequencing the scramble generator. This results in an output like a few letters: b, c, a.

License

LEXT is a Free Open-Source Software project released under the MIT License.

lext's Issues

Prefer returning bool/enum values over integers

In the most basic cases, where a function is indeed only either successful or not, bool should always be used.

In cases where a non-successful result carries more information, an enum should be used. For example, lxt_parse, which can hit several different failures, should return which exact failure it hit (e.g. max containers, entries etc.).

It might even be worthwhile to go further than just simple enums and instead return stateful information so that the library can report exactly where and why something went wrong; e.g. parsing errors at line and column numbers. But that is probably better fit as a separate issue.

Support escaping keyword symbols as literals

For example, currently, you specifically can't use either @ or > literally in a generator sequence, because these, respectively, indicates either start of variable or end of sequence. It would be nice being able to escape these so they can be used as literals.

Similarly, this is also an issue in container entries, where you e.g. can't use commas (,) because they would indicate end of entry.

You can get around the issue by limiting commas to sequences, and limiting variable/end of sequence symbols to only occur in entries, but generally, you'd probably expect just being able to escape these as literals.

Sequence variables only resolve if followed by whitespace

Some patterns won't resolve as expected because variables are not parsed properly in certain cases.

For example, the following pattern starts out with a variable @letter; however, this particular variable happens to be followed by a comma, making the parser think it is part of the variable name (e.g. variable,). In this case, it shouldn't be.

letter (a, b, c) seq <@letter, @letter, @letter>

I suppose a simple fix is to break on both isspace(..) and any non-alphabet character (such as ,, . etc.).

Alternatively, require an escape for any character not part of the name- something like <@letter\, @letter\, @letter>.