lionelauroux/pyrser

A PEG Parsing Tool

pyrser's Issues

Hooks without parameters but with () like #my_hook() not working.

Salut,

@LionelAuroux : Tu m'as dit que pour faire un hook sans paramètres il faut enlever les parenthèses, mais je trouve que cela prête beaucoup à confusion vu que dans la plupart des langages une fonction sans paramètres s'écrit avec des parenthèses vides: fct().

Ce serait cool de pouvoir supporter les deux syntaxes, non ? (Je veux dire avec et sans les parenthèses).

Du coup je t'envoie l'issue que tu m'avais demandé.

meta.hook doesn't check function's return type

A meta.hook as said in lessons has to return a boolean value which tells if the function succeeded or not. But when forgetting the return value it sometimes complains sometimes not, and it's really annoying.

I know for sure, it's possible to check the return type and if it isn't a boolean an Exception or a Warning should be raised to warn the user. Obviously, it will help debugging projects when the return statement was forgotten in the meta.hook'd function.

Cordially, payet_s.

conflits de hooks

Il y a conflit avec des fonctions internes de pyrser lorsque l'on crée un hook du même nom.
Par exemple avec "end_tag". Cela a pour cause de faire bug les Node et les rendre vide.

[Typesystem ]Funcion and Variable erasing each other

Fun() and Var() erases themself when having the same tret and same name.

t1 = Type('int')
var = Var('a', 'int')
f3 = Fun('a', 'int', [])
scope = Scope(sig=[t1, var, f3])
print(str(scope))

same problem when adding

t1 = Type('int')
var = Var('a', 'int')
f3 = Fun('a', 'int', [])
scope = Scope()
scope.add(t1)
scope.add(var)
scope.add(f3)
print(str(scope))

built-in hooks like #set(dst, src) not working

Salut !

@LionelAuroux : Avec la version de Pyrser qui m'est venue avec pip3 install pyrser mardi 15 septembre 2015 j'ai eu un bug avec #set(dst, src) que je t'avais montré et que tu n'avais pas compris sur le moment. Du coup voici l'issue que tu m'avais demandé de rédiger.

A noter que je n'ai testé le bug qu'avec #set(dst, src), je n'ai essayé avec aucun autre built-in.

(Et on avait aussi essayé avec d'autres manières de spécifier l'import que tu m'avais dit d'essayer sur le moment mais qui n'avaient pas marché non plus.)

from pyrser import grammar, meta
from pyrser.hooks import *

class parserExample(grammar.Grammar):
    entry = "Example"
    grammar = """ 
    Example = [ id:i eof #set(_, i) ]

    """

Ex = parserExample()
res = Ex.parse("someExample")

outputs:

Traceback (most recent call last):
  File "parserCSV.py", line 12, in <module>
    res = Ex.parse("someExample")
  File "/usr/local/lib/python3.4/dist-packages/pyrser/grammar.py", line 144, in parse
    return self._do_parse(entry)
  File "/usr/local/lib/python3.4/dist-packages/pyrser/grammar.py", line 124, in _do_parse
    raise self.diagnostic
  File "/usr/local/lib/python3.4/dist-packages/pyrser/grammar.py", line 107, in _do_parse
    res = self.eval_rule(entry)
  File "/usr/local/lib/python3.4/dist-packages/pyrser/parsing/base.py", line 215, in eval_rule
    res = rule_to_eval(self)
  File "/usr/local/lib/python3.4/dist-packages/pyrser/parsing/functors.py", line 29, in __call__
    res = self.do_call(parser)
  File "/usr/local/lib/python3.4/dist-packages/pyrser/parsing/functors.py", line 165, in do_call
    if not pt(parser):
  File "/usr/local/lib/python3.4/dist-packages/pyrser/parsing/functors.py", line 29, in __call__
    res = self.do_call(parser)
  File "/usr/local/lib/python3.4/dist-packages/pyrser/parsing/functors.py", line 502, in do_call
    return parser.eval_hook(self.name, valueparam)
  File "/usr/local/lib/python3.4/dist-packages/pyrser/parsing/base.py", line 229, in eval_hook
    raise self.diagnostic
pyrser.error.Diagnostic: ===============================================================================
error : Unknown hook : set
from /tmp/tmpt4kdv4hy at line:1 col:12 :
someExample
           ^
-------------------------------------------------------------------------------
error : Exception during the evaluation of 'eof'
from /tmp/tmpt4kdv4hy at line:1 col:12 :
someExample
           ^
-------------------------------------------------------------------------------
error : Parse error in 'eof'
from /tmp/tmpt4kdv4hy at line:1 col:12 :
someExample
           ^
-------------------------------------------------------------------------------

Diagnostic get_content bug

In pyrser.error.Diagnostic, when the file end with an empty line and an error occurs before that line
an _IndexError: list index out of range_ is thrown

get_content

#!/usr/bin/env python3.5

from pyrser import meta, error
from pyrser.grammar import Grammar
from pyrser.hooks.predicate import pred_false

class Exemple(Grammar):
    entry = 'exemple'
    grammar = """
        exemple = [ id #false ]
    """

ex = Exemple()

try:
    ex.parse_file('test_file')
except error.Diagnostic as e:
    print(e.logs[0].get_content(with_locinfos=True))

file tested (note many empty lines below "stuff")

 stuff

Users not warn of invalid file path in parse_file

Parse a file using parse_file method won't raise any exception if the file doesn't exist, which is quite annoying. Indeed, the path existence is checked using os.path.exists but others standard IOError exceptions are raised normally which doesn't make so more sense.

The path checking can be omitted and a standard exception will be raised. Then, catch it (with others standard IOError as well) and raise custom ones if wanted. Anyway, the with statement should be used to be safer.

Otherwise, if you really want to keep this behavior you could provide at least a method's default parameter set to raise those exceptions and then with a boolean silently catch them. Because it shouldn't be the default behavior and it is very disrupting while debugging or for newbies.

Cordially, payet_s.

read_until inhibitor does not work

In class parserBase when calling read_until, it seems the inhibitor is not working properly when there are several on the same line

i provided code sample to test

read_until_error_exemple

#!/usr/bin/env python3.5

from pyrser import meta
from pyrser.grammar import Grammar

class Exemple(Grammar):
    entry = 'exemple'
    grammar = """
        exemple = [ broken_read_until:res #dump(res) ]
    """

@meta.rule(Exemple)
def broken_read_until(self):
    self.read_until('\n', '\\')
    return True

@meta.hook(Exemple)
def dump(self, stuff):
    print(self.value(stuff))
    return True

ex = Exemple()

ex.parse_file('test_file')

test_file

stuff on \
multiple \\\
line

exception str() failed on eol

Hi,
On a first try implementing a CSV parser, I get a exception str() failed with no other infos, when using the default rule eol

Input :

A;B;C;D
E;F;G

Parser :

class parserCSV(grammar.Grammar):
    entry = "Csv"
    grammar = """
        Csv = [
            @ignore("null")
            Lines
            eof
        ]

        Lines = [
            Line* LastLine
        ]

        Line = [ Fields eol ]
        LastLine = [ Fields eol? ]

        Fields = [ id [ ';' id ]* ]
    """

Full error :

Traceback (most recent call last):
  File "./test.py", line 8, in <module>
    res = csv.parse_file(sys.argv[1])
  File "/usr/lib/python3.5/site-packages/pyrser/grammar.py", line 165, in parse_file
    return self._do_parse(entry)
  File "/usr/lib/python3.5/site-packages/pyrser/grammar.py", line 130, in _do_parse
    raise self.diagnostic
pyrser.error.Diagnostic: <exception str() failed>

When I change Line & LastLine to :

        Line = [ Fields end ]
        LastLine = [ Fields end? ]

        end = [ "::" ]

With input :

A;B;C;D::E;F;G

It works correctly, so the error should be somewhere aroud eol

And finally, when I change end to eol or to '\r'? '\n' it still raise an exception...

Good luck !

String literals concatenation

According to the C standards, successive string literals are interpreted as one string, composed of all the strings concatenated (i.e. printf("Hello " "world !"); is the same as printf("Hello world !").
Pyrser raises an exception with this syntax, even though it is valid.