Wishlist: debug output for grammar parsing

[philippesigaud]

Wishlist: debug output for grammar parsing

[yoe]

apparently part of the input I gave had some special meaning to github, syntax-wise. Please just look at the source.

[PhilippeSigaud]

The problem is here:

Statement <- ( AutoStatement / HotplugStatement / EmptyLine / Comment )

EmptyLine is tested before Comment and will always check. So
Statement on a #... line will be parsed as an EmptyLine and
then, as there is no EOL, Grammar switch to parsing EOI.

Use this:

Statement <- AutoStatement / HotplugStatement / Comment / EmptyLine

It works for me.

Tell me if you need help in defining comments and such in your grammar.

[yoe]

Yes, I eventually found that myself, through heavy debugging of the generated code, and now have a working parser for what I need.

This was made much easier after I changed the code (with a Q&D perl script) so it would write a "matched foo" or "failed to match foo", followed by the complete parse tree, to stdout every time a match or fail would occur. This generates a huge amount of data, but allows you to trace the parser as it's running.

Perhaps this might be an interesting feature to have in Pegged itself?

[yoe]

Thought it might make sense to add a pointer to that Q&D perl script:

https://github.com/yoe/ipcfg/blob/06/src/adddebug.pl

This would probably be better implemented in pegged/development/grammarfunctions.d, but I couldn't immediately figure out why it didn't work for me, and the perl script worked...

[PhilippeSigaud]

I'm not that well-versed in Perl. It scans the module itself, looking
for some patterns?

[yoe]

Basically, the script reads the output of the asModule() function one line at a time. When the regex /Output parse/ matches the input line, the variable "$last" is set to "parse", and the variable "counter" is set to 0. When the variable "counter" reaches 2, we output "writeln("attempting " ~ ruleName);\n".

Example class after it ran through this perl script would look like this:

class S : Drop!(Or!(Lit!("\t"),Lit!(" ")))
{
enum grammarName = RULES;
enum ruleName = S;

static Output parse(ParseLevel pl = ParseLevel.parsing)(Input input)
{

writeln("attempting " ~ ruleName);
mixin(okfailMixin());

    auto p = typeof(super).parse!(pl)(input);
    static if (pl == ParseLevel.validating)
        p.capture = null;
    static if (pl <= ParseLevel.matching)
        p.children = null;
    static if (pl >= ParseLevel.parsing)
    {
        if (p.success)
        {                                

writeln(" MATCHED " ~ ruleName ~ "
", p);
static if (pl == ParseLevel.parsing)
p.parseTree = decimateTree(p.parseTree);

            if (p.grammarName == grammarName || pl >= ParseLevel.noDecimation)
            {
                p.children = [p];
            }

            p.grammarName = grammarName;
            p.ruleName = ruleName;
        }
        else

{
writeln("failed " ~ ruleName ~ "
", p);
return fail(p.parseTree.end,
(grammarName~.~ruleName ~ failure at posd ~ to!dstring(p.parseTree.end) ~ (p.capture.length > 0 ? p.capture[1..$] : p.capture)));
}
}

    return p;
}    
mixin(stringToInputMixin());

}

like I said, it's quick and dirty :-)

[PhilippeSigaud]

Ah OK, I didn't understand it modified the file itself (as I said, my
Perl knowledge is quite limited).

Why don't you modify one of the wiki pages to link to your script and
explain what it does? Aren't they access-free?

[veelo]

This was made much easier after I changed the code (with a Q&D perl script) so it would write a "matched foo" or "failed to match foo", followed by the complete parse tree, to stdout every time a match or fail would occur. This generates a huge amount of data, but allows you to trace the parser as it's running.

Perhaps this might be an interesting feature to have in Pegged itself?

This is now implemented, see https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Debugging.