mwallerb/fsource

Fortran static analysis tool written in pure Python

fsource's Issues

Parser fails at INCLUDE statements

Hi,
I tried to run the parser on (part of) this package, and got the following error:

> fsource parse ref-calc.v1.6.f --fixed-form


ref-calc.v1.6.f:42:7: parser error: malformed statement inside program
|
|             INCLUDE "PARAM"
|             ^~~~~~~

The ref-calc.v1.6.f file begins:

      program leed

C  include parameter statements for dimensions etc

      INCLUDE "PARAM"

C  set unchangeable constants

      INCLUDE "GLOBAL"

Am I missing some option to run the parser on the entire project in a way that would make the INCLUDE statements work, or is this simply not implemented yet?

wrapper fails for simple testprogram

the log is actually the f90-program. sebastian@debian:~/ps/nms2018$ ~/.local/bin/fsource wrap u06a.f90
/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py:156: UserWarning: CANNOT IMBUE <fsource.analyzer.CompilationUnit object at 0x7f9404b7bf90> WITH program_decl
warnings.warn("CANNOT IMBUE %s WITH %s" % (parent, self.ast[0]))
/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py:159: UserWarning: CANNOT RESOLVE program_decl
warnings.warn("CANNOT RESOLVE %s" % self.ast[0])
Traceback (most recent call last):
File "/home/sebastian/.local/bin/fsource", line 10, in
sys.exit(main())
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/_cli.py", line 234, in main
cmd_wrap(args)
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/_cli.py", line 214, in cmd_wrap
print (asr.cdecl().get())
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py", line 189, in cdecl
return CWrapper.union(self.children)
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py", line 67, in union
wraps = tuple(elem.cdecl() for elem in elems)
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py", line 67, in
wraps = tuple(elem.cdecl() for elem in elems)
AttributeError: 'Ignored' object has no attribute 'cdecl'
u06a.log

Question about handling Hollerith format fields

fsource lex does not recognize the token 1H1 in the line FORMAT (1H1)

This should be expected behavior since fsource is advertised as handling F77 and later, and Hollerith fields were considered obsolete by F77.

I am working on a project to detect and evaluate problematic constructs in legacy code such as Hollerith fields, alternate return points, ENTRY statements, character data stored in non-character variables, etc. My question is where I should focus my efforts to extend lexer.py to recognize these difficult and obsolete constructs? After a quick scan of lexer.py, it looks like I should modify formattok in get_lexer_regex() - does this seem reasonable?

Does not parse program without program statement

For

print*,"hi"
end

I get

c:\python3\code>fsource parse try.f90
Traceback (most recent call last):
  File "c:\intelpython3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\intelpython3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\intelpython3\Scripts\fsource.exe\__main__.py", line 9, in <module>
  File "c:\intelpython3\lib\site-packages\fsource\__main__.py", line 187, in main
    cmd_parse(args)
  File "c:\intelpython3\lib\site-packages\fsource\__main__.py", line 167, in cmd_parse
    ast = parser.compilation_unit(tokens, fname)
  File "c:\intelpython3\lib\site-packages\fsource\parser.py", line 222, in rule_setup
    value = fn(tokens, *args)
  File "c:\intelpython3\lib\site-packages\fsource\parser.py", line 1851, in compilation_unit
    expect_cat(tokens, lexer.CAT_DOLLAR)
  File "c:\intelpython3\lib\site-packages\fsource\parser.py", line 122, in expect_cat
    raise NoMatch()
fsource.parser.NoMatch

fsource does work for

program main
print*,"hi"
end program main

Scale factor without comma separator not recognized

The scale factor edit descriptor P may or may not be separated from other edit descriptors with a comma. The rules on this are not terribly clear (see Steve Lionel's comment on https://stackoverflow.com/questions/54945112/error-comma-required-after-p-descriptor-in-format-string). The case I typically see resembles 1PE12.4 where the comma is omitted.

I had some success replacing:

    formattok = r"""\d*(?: [IBOZ] \d+ (?: \.\d+)?
                         | [FD]   \d+     \.\d+
                         | E[NS]? \d+     \.\d+  (?: E\d+)?
                         | G      \d+ (?: \.\d+  (?: E\d+)?)?
                         | L      \d+
                         | A      \d*
                         | [XP]
                         )(?=\s*[:/,)])"""

with

    formattok = r"""\d*(?: [IBOZ] \d+ (?: \.\d+)?
                         | [FD]   \d+     \.\d+
                         | E[NS]? \d+     \.\d+  (?: E\d+)?
                         | G      \d+ (?: \.\d+  (?: E\d+)?)?
                         | L      \d+
                         | A      \d*
                         | X
                         )(?=\s*[:/,)])
                    |(?:-?\d+P)(?=\s*,?[\d:DEFG])"""

in get_lexer_regex() in lexer.py but I'm not convinced the lookahead assertion (?=\s*,?[\d:DEFG]) is necessary or correct.