pest-parser/pest

The Elegant Parser

pest's Issues

Add changelog

Would be nice to see what changes between each version, can be just a footnote on a README

process will reach out of bounds instead of failing a pattern.

Check bounds before slicing in patterns.

Parser end() should be replaced by eoi().

Add Packrat parser.

Add input() reference to Parser.

Improve documentation & host it.

Add macro pattern to define process! with &mut self.

Rename ws to whitespace.

List parsing

I have the following rule in my grammar:

function = { symbol ~ ["("] ~ expression ~ ([","] ~ expression) ~ [")"] }

and I try to parse it like this:

(_: function, fun: main(), arguments: _arguments()) => {
    Function(fun, arguments)
},

_arguments(&self) -> Vec<Expression> {
    (head: main(), tail: _arguments()) => {
        let mut tail = tail;
        tail.push(head);
        tail
    },
    () => {
        Vec::new()
    }
}

This however panics! with the message that no rules matched.
What am I doing wrong?

Add Lua example.

Add interrogation rules.

Add repetition rules.

Add optional rule.

try! fails inside of process! matchers.

try! tries to return a Result<A, B>, but all matchers are defined as (Result<A, B>, usize) behind the scenes.

One possible solution to this problems is to catch every try! call in the macro and replace it with a modified version that knows about the extra usize.

Referenced from #51.

Rething empty string match []

Rename exp to expression in all examples.

Rethink empty string match [""] in examples.

Recursion limit causes problems with larger character classes

I am developing a parser for email headers. One of the rules is designed to match a single character out of a large list:

atext           =       ALPHA / DIGIT / ; Any character except controls,
                        "!" / "#" /     ;  SP, and specials.
                        "$" / "%" /     ;  Used for atoms
                        "&" / "'" /
                        "*" / "+" /
                        "-" / "/" /
                        "=" / "?" /
                        "^" / "_" /
                        "`" / "{" /
                        "|" / "}" /
                        "~"

Trying to code this directly fails to compile with the "recursion limit reached" error. Reducing the list of characters using ['X'..'Y'] ranges still fails to compile as there are still too many ranges.

Ideally, this would be supported by a single test like [["abcdefg..!#$%&...{|}"]].

Add capturing.

AST or closure.

The parser should remember rule, start & length in a queue after parsing.

Only leaf errors should be reported.

Add atomic rule.

Add calculator example.

Make sure all the terminology is explained in the crate's main documentation.

Remove Box from Rdp's Input.

Restyle grammar.

• replace lists, heaps, stacks from () to []
• space everything accordingly

Add benchmarks to README.md.

Comparison with other parsers

As mentioned on Reddit, some comparisons between nom / oak / lalrpop etc in the README would be neat

Mention Rule in impl_rdp! and Token docs.

ws should not skip_ws.

Add debugging tips.

Move Token outside of macro.

Add buffered file input.

Mention recursion limit in the docs.

Add a prelude.

Change process to accept lifetimes.

Right-associative operator

Is there any way to tell the generator that an operator is right-associative?

process! support mutability-specifier for bindings

What do you think about extending the process!-macro to support a syntax like:

(&a: token, mut b: _parserXYZ()) => { ... }

This would essentially allow passing e.g. a vec/linked list around and modifying it in-place.

The current workaround would be forcing a move:

(&a: token, b: _parserXYZ()) => { let mut b_mut = b; ... }

Add error reporting.

Rename Parser::expected.

expected is not very clear, but it's kind of hard to find a better name for it.

Consider inlining rules.

Parallelize parsing.

This can be an interesting idea to keep in mind for the future. I have already experimented with single-consumer-single-producer queues in order to have a token-producing and a token-consuming queue, but the performance is lack-luster, even when using state-of-the-art data structures and algorithms.

A possible approach here would be to have a 2-layered parsing approach: one would divide the input with a really simple grammar, and the other would parallelly parse these divisions.

Add examples for process!

The calculator does the math but I assume most users will want to build an AST so an example for that would be nice, the one in the docs is a bit too light imo.

The Lua language one would be nice but might be quite complex to do, right now the docs seems a bit small for the process! part. We could actually put Tera there as an example once it's done.

Add CI.

Silent atomic rule?

It seems @_{ ... } or _@{ ... } is not allowed, but I guess it's necessary for some scenario.

Add precedence climbing.

Error handling during process!-matchers

Are there any plans to introduce some syntax-sugar for error handling
during process!-matchers, to make error handling easier?

The only way I currently see is if for example every matcher returns a Result<A, B>,
that you'd always have to do something like:

_xyz(&self) -> Result<A, B> {
    (&x: y, z_result: _matcher_z()) => {
        let z = try!(z_result);
        Ok(...)
   }
}

Some possible syntax sugar (without checking if it can be implemented):

_xyz(&self) -> Result<A, B> {
    (&x: y, z?: _matcher_z()) => {
        Ok(...)
   }
}

Is there any better method to do it and what are your plans in this area as well as
for error handling generally?

Grammar multi-tool.

An external tool that can analyze pest grammars (maybe process too?) and convert between different types of grammars (if/where applicable).

• check grammar syntax
• check left-recursion
• check name collisions
• check arbitrary file with grammar

Add calculator example.

Readme: Speedup should be slowdown?

In the readme where the benchmark results for pest are published should speedup be slowdown? Is ANTRL 4 48x faster than pest?

Add rule to Input for position to (line, column) conversion.

Implement macro to handle output.