zaach/lex-parser

A parser for lexical grammars used by jison.

lex-parser

A parser for lexical grammars used by jison and jison-lex.

install

npm install lex-parser

build

To build the parser yourself, clone the git repo then run:

make

This will generate lex-parser.js.

usage

var lexParser = require("lex-parser");

// parse a lexical grammar and return JSON
lexParser.parse("%% ... ");

example

The parser can parse its own lexical grammar, shown below:

NAME              [a-zA-Z_][a-zA-Z0-9_-]*

%s indented trail rules
%x code start_condition options conditions action

%%

<action>[^{}]+          return 'ACTION_BODY'
<action>"{"             yy.depth++; return '{'
<action>"}"             yy.depth == 0 ? this.begin('trail') : yy.depth--; return '}'

<conditions>{NAME}      return 'NAME'
<conditions>">"         this.popState(); return '>'
<conditions>","         return ','
<conditions>"*"         return '*'

<rules>\n+              /* */
<rules>\s+              this.begin('indented')
<rules>"%%"             this.begin('code'); return '%%'
<rules>[a-zA-Z0-9_]+    return 'CHARACTER_LIT'

<options>{NAME}         yy.options[yytext] = true
<options>\n+            this.begin('INITIAL')
<options>\s+\n+         this.begin('INITIAL')
<options>\s+            /* empty */

<start_condition>{NAME}         return 'START_COND'
<start_condition>\n+            this.begin('INITIAL')
<start_condition>\s+\n+         this.begin('INITIAL')
<start_condition>\s+            /* empty */

<trail>.*\n+                    this.begin('rules')

<indented>"{"                   yy.depth = 0; this.begin('action'); return '{'
<indented>"%{"(.|\n)*?"%}"      this.begin('trail'); yytext = yytext.substr(2, yytext.length-4);return 'ACTION'
"%{"(.|\n)*?"%}"                yytext = yytext.substr(2, yytext.length-4); return 'ACTION'
<indented>.+                    this.begin('rules'); return 'ACTION'

"/*"(.|\n|\r)*?"*/"             /* ignore */
"//".*                          /* ignore */

\n+                             /* */
\s+                             /* */
{NAME}                          return 'NAME'
\"("\\\\"|'\"'|[^"])*\"         yytext = yytext.replace(/\\"/g,'"');return 'STRING_LIT'
"'"("\\\\"|"\'"|[^'])*"'"       yytext = yytext.replace(/\\'/g,"'");return 'STRING_LIT'
"|"                             return '|'
"["("\\\\"|"\]"|[^\]])*"]"      return 'ANY_GROUP_REGEX'
"(?:"                           return 'SPECIAL_GROUP'
"(?="                           return 'SPECIAL_GROUP'
"(?!"                           return 'SPECIAL_GROUP'
"("                             return '('
")"                             return ')'
"+"                             return '+'
"*"                             return '*'
"?"                             return '?'
"^"                             return '^'
","                             return ','
"<<EOF>>"                       return '$'
"<"                             this.begin('conditions'); return '<'
"/!"                            return '/!'
"/"                             return '/'
"\\"([0-7]{1,3}|[rfntvsSbBwWdD\\*+()${}|[\]\/.^?]|"c"[A-Z]|"x"[0-9A-F]{2}|"u"[a-fA-F0-9]{4}) return 'ESCAPE_CHAR'
"\\".                           yytext = yytext.replace(/^\\/g,''); return 'ESCAPE_CHAR'
"$"                             return '$'
"."                             return '.'
"%options"                      yy.options = {}; this.begin('options')
"%s"                            this.begin('start_condition');return 'START_INC'
"%x"                            this.begin('start_condition');return 'START_EXC'
"%%"                            this.begin('rules'); return '%%'
"{"\d+(","\s?\d+|",")?"}"       return 'RANGE_REGEX'
"{"{NAME}"}"                    return 'NAME_BRACE'
"{"                             return '{'
"}"                             return '}'
.                               /* ignore bad characters */
<*><<EOF>>                      return 'EOF'

<code>(.|\n)+                   return 'CODE'

%%

license

MIT

lex-parser's Issues

repository format in package.json is not correct

Currently it is,

"repository": {
    "type": "git",
    "repository": "https://github.com/zaach/lex-parser.git"

Correct configuration,from another module,

"repository": {
    "type": "git",
    "url": "git://github.com/jonschlinkert/glob-base.git"

So second 'repository ' field should be changed to 'url'.

This caused npm2deb to crash (it is being fixed though) https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840169 it may cause problems with other tools.

ranges with char numbers doesn't seem to work

letter     [a-zA-Z] 

works fine,

however

asciichar [\32-\126] 

doesn't seems to work.

[GerHobbelt/lex-parser] `reject()` API et al: it's silly to have lexer option `backtrack_lexer` as a run-time option.

It doesn't make a lick of sense to flip that switch at lexer run-time.
It'd better be a compile-time setting.

please tag releases

We use github tarballs for debian package and without tags, we have to manually find the commits for each release and get a snapshot. But still dbeian's tools like uscan cannot do this automatically. It would be good if you can tag your releases.

end of line ($) doesn't work as expected

just tried to use jison, and this got me lost for hours.

$ doesn't seem to work.

jison documentation points to bison, where $ is widely used.

in the samples here the \n+ is been used instead.

Create LICENSE file

Would you please create a LICENSE or LICENSE.md file with your copyright information and the text of the MIT license? The MIT license specifically states that the license text must accompany the source code.

^ (start of line) doesn't work as expected

^ is working, but it seems to be recognized in more situations than it should.

start of line is very handy when writing parsers where end-of-line is an entry delimiter (such as when parsing HTTP)

^{header}\={value}$