cannot mix strings and tokens

[GoogleCodeExporter]

cannot mix strings and tokens

[GoogleCodeExporter]

hi.  the main issue here isn't a bug - it's how lepl works.  you can either 
work in tokens, or not, but not both.  that's what the error message says.

trying to tokenize String fails because String is too complex for lepl to 
tokenize automatically.  it might be possible to write String so that it can be 
converted automatically, and i will add that to the list of things to do, but 
meantime you can simply define your own regular expression:

  myString = Regexp("'[^']*'")

or similar.

andrew

Original comment by [email protected] on 1 Jan 2012 at 10:55

• Changed state: WontFix

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Well, I have defined my own string indeed

    string = Token('"[^"]+"') | Token("'[^']+'")

However I have no idea how to allow quoting of apostrophe or quotation
characters with backslash, i.e. "test\"string" or 'test\'string'.

Using Python regular expressions that would be

     r""""(([^"]|\")+)"|'(([^']|\')+)'"""

w

Original comment by [email protected] on 1 Jan 2012 at 11:48

[GoogleCodeExporter]

the syntax for regexps should be the same as python, except that capturing 
groups are not supported.  so you need to replace each (...) with (?:...)

andrew

Original comment by [email protected] on 2 Jan 2012 at 12:11

[GoogleCodeExporter]

Thanks for the tip.

The definition is as follows

    string = Token(r'"(?:[^"]|\\")+"') | Token(r"'(?:[^']|\\')+'")

But I wonder how above is different from SingleLineString?

If it is not different, then I would like to report that the following code 
fails

"""
from lepl import *

v = Token('[a-z]+') & Token(' +') & Token(SingleLineString())
v.parse('aaa "aaa"')
"""

w

Original comment by [email protected] on 5 Jan 2012 at 6:53