Performance improvements

[philippesigaud]

Performance improvements

[PhilippeSigaud]

Pegged now use indices internally, instead of always returning the entire input in the output. Can you tell me if you see any difference in speed?

[alexrp]

I need to catch some sleep now, but I'll give it a test run tomorrow!

[alexrp]

I'm not sure why, but my grammar broke after updating Pegged. Is there any way to get some sort of info on what went wrong?

The grammar in question: https://github.com/lycus/cursive/blob/master/src/cursive/parsing/grammar.d

[PhilippeSigaud]

My mistake for not informing you (I sent the message to other Pegged users but forgot to tell you): I dropped the non-standard space-consuming behavior of sequences. To get a space-munching sequence, use the 'space arrow': <. What you did with your grammar (Spacing everywhere) is correct, just drop the '>' operator in your RealSuffix and IntegerSuffix, it's not needed anymore.

The docs are updated.

Nice grammar btw. For such a big grammar I now prefer to write it in a file and import the file:

enum gram = "my grammar";

void main()
{
    asModule("program", gram);
}

and then

import gram;

// use it

It compiles for me, but I don't know what program to give it to parse.
I'm not sure I understand your comment code:

LineComment <: Slash Slash (&(EOL / EOI) (EOL / EOI) / .*)

Shouldn't it be:

LineComment <: "//" (!(EOL/EOI) .)* (EOL/EOI)

?

Then define locally (in your grammar) Spaces to be:

Spaces <: Spacing / LineComment / BlockComment

And replace Spacing by Spaces in your grammar. Not perfect, but at least comments won't get parsed and can be inserted anywhere a blank is called for.

[alexrp]

Thanks for the explanation! I've fixed the grammar. Now I'm having another problem, though: Generating the grammar is pretty much impossible on my desktop system. DMD instantly reaches 100% CPU and consumes close to all memory in the system (16 GB!). Do you have any idea why this is happening? It's odd, because I can build the grammar just fine on a server machine.

I'm not sure I understand your comment code:

I remember I was having some problems with comments, but I forget the details. I'll look into that later.

[PhilippeSigaud]

I guess CTFE doesn't free memory as the runtime does. Did you try to write it as a module?

asModule("myGrammar", string peg);

then use it as a standard module:

import myGramar;

[alexrp]

I can try that tomorrow, but I don't think that it will make a difference; the problem is the grammar() call, which asModule() does anyway.

[PhilippeSigaud]

On Thu, Mar 22, 2012 at 19:42, Alex Rønne Petersen
[email protected]
wrote:

I can try that tomorrow, but I don't think that it will make a difference; the problem is the grammar() call, which asModule() does anyway.

I thought you were doing it at compile-time previously.

[alexrp]

Ah, of course, that one does the computation at runtime... proof right there that I needed sleep. ;)

I tried it out, and it works very well (grammar generated in less than two seconds, 3317 lines). I just can't help but wonder how the CTFE generation can eat so many resources... In any case, feature request: I'd like to be able to specify the exact file name in addition to the module name to asModule(). Doable? (I need this in order to integrate this grammar generation approach properly in my build process.)

[alexrp]

And performance:

$ time ./test2 test3.crs
real    6m12.649s
user    6m11.279s
sys 0m0.076s

It seems to have gotten worse. :( FWIW, during the parsing process, it uses virtually no memory but sits at 100% CPU the entire process.

[PhilippeSigaud]

asModule(string moduleName, string fileName, string grammarString): OK done.

For the speed, can you send me your grammar and test file? I'm doing a new branch on internal optimizations and I'm curious to see if I see any effet.

([email protected])

[PhilippeSigaud]

I merged a speed-up branch a few weeks ago and obtained 1-2 orders of magnitude improvement in speed. Then using memoization can add up to 40% speed increase and some other thingies also added 20-40% increase.

I'll continue to find new ways to improve parsing speed.