pest-parser/pest

The Elegant Parser

pest's Issues

Add calculator example.

Make sure all the terminology is explained in the crate's main documentation.

try! fails inside of process! matchers.

try! tries to return a Result<A, B>, but all matchers are defined as (Result<A, B>, usize) behind the scenes.

One possible solution to this problems is to catch every try! call in the macro and replace it with a modified version that knows about the extra usize.

Referenced from #51.

Add benchmarks to README.md.

Improve documentation & host it.

Error handling during process!-matchers

Are there any plans to introduce some syntax-sugar for error handling
during process!-matchers, to make error handling easier?

The only way I currently see is if for example every matcher returns a Result<A, B>,
that you'd always have to do something like:

_xyz(&self) -> Result<A, B> {
    (&x: y, z_result: _matcher_z()) => {
        let z = try!(z_result);
        Ok(...)
   }
}

Some possible syntax sugar (without checking if it can be implemented):

_xyz(&self) -> Result<A, B> {
    (&x: y, z?: _matcher_z()) => {
        Ok(...)
   }
}

Is there any better method to do it and what are your plans in this area as well as
for error handling generally?

Add error reporting.

Move Token outside of macro.

Add Packrat parser.

Implement macro to handle output.

Add input() reference to Parser.

Add CI.

Consider inlining rules.

Add debugging tips.

process! support mutability-specifier for bindings

What do you think about extending the process!-macro to support a syntax like:

(&a: token, mut b: _parserXYZ()) => { ... }

This would essentially allow passing e.g. a vec/linked list around and modifying it in-place.

The current workaround would be forcing a move:

(&a: token, b: _parserXYZ()) => { let mut b_mut = b; ... }

Rething empty string match []

Grammar multi-tool.

An external tool that can analyze pest grammars (maybe process too?) and convert between different types of grammars (if/where applicable).

• check grammar syntax
• check left-recursion
• check name collisions
• check arbitrary file with grammar

Add examples for process!

The calculator does the math but I assume most users will want to build an AST so an example for that would be nice, the one in the docs is a bit too light imo.

The Lua language one would be nice but might be quite complex to do, right now the docs seems a bit small for the process! part. We could actually put Tera there as an example once it's done.

Change process to accept lifetimes.

process will reach out of bounds instead of failing a pattern.

Check bounds before slicing in patterns.

Mention Rule in impl_rdp! and Token docs.

Rename exp to expression in all examples.

Add rule to Input for position to (line, column) conversion.

Add changelog

Would be nice to see what changes between each version, can be just a footnote on a README

Right-associative operator

Is there any way to tell the generator that an operator is right-associative?

Add macro pattern to define process! with &mut self.

Add buffered file input.

Add repetition rules.

Add capturing.

AST or closure.

The parser should remember rule, start & length in a queue after parsing.

Parallelize parsing.

This can be an interesting idea to keep in mind for the future. I have already experimented with single-consumer-single-producer queues in order to have a token-producing and a token-consuming queue, but the performance is lack-luster, even when using state-of-the-art data structures and algorithms.

A possible approach here would be to have a 2-layered parsing approach: one would divide the input with a really simple grammar, and the other would parallelly parse these divisions.

List parsing

I have the following rule in my grammar:

function = { symbol ~ ["("] ~ expression ~ ([","] ~ expression) ~ [")"] }

and I try to parse it like this:

(_: function, fun: main(), arguments: _arguments()) => {
    Function(fun, arguments)
},

_arguments(&self) -> Vec<Expression> {
    (head: main(), tail: _arguments()) => {
        let mut tail = tail;
        tail.push(head);
        tail
    },
    () => {
        Vec::new()
    }
}

This however panics! with the message that no rules matched.
What am I doing wrong?

Add Lua example.

Rethink empty string match [""] in examples.

Recursion limit causes problems with larger character classes

I am developing a parser for email headers. One of the rules is designed to match a single character out of a large list:

atext           =       ALPHA / DIGIT / ; Any character except controls,
                        "!" / "#" /     ;  SP, and specials.
                        "$" / "%" /     ;  Used for atoms
                        "&" / "'" /
                        "*" / "+" /
                        "-" / "/" /
                        "=" / "?" /
                        "^" / "_" /
                        "`" / "{" /
                        "|" / "}" /
                        "~"

Trying to code this directly fails to compile with the "recursion limit reached" error. Reducing the list of characters using ['X'..'Y'] ranges still fails to compile as there are still too many ranges.

Ideally, this would be supported by a single test like [["abcdefg..!#$%&...{|}"]].

Mention recursion limit in the docs.

Add a prelude.

ws should not skip_ws.

Silent atomic rule?

It seems @_{ ... } or _@{ ... } is not allowed, but I guess it's necessary for some scenario.

Add calculator example.

Add precedence climbing.

Rename ws to whitespace.

Only leaf errors should be reported.

Add atomic rule.

Remove Box from Rdp's Input.

Add optional rule.

Readme: Speedup should be slowdown?

In the readme where the benchmark results for pest are published should speedup be slowdown? Is ANTRL 4 48x faster than pest?

Add interrogation rules.

Comparison with other parsers

As mentioned on Reddit, some comparisons between nom / oak / lalrpop etc in the README would be neat

Rename Parser::expected.

expected is not very clear, but it's kind of hard to find a better name for it.

Parser end() should be replaced by eoi().

Restyle grammar.

• replace lists, heaps, stacks from () to []
• space everything accordingly