mwallerb/fsource

Fortran static analysis tool written in pure Python

fsource - Fortran static analysis tool

fsource is a collection of tools allowing you to parse Fortran 77 through Fortran 2008 programs. It is written in pure Python and has no external dependencies.

You install fsource via pip (you may want to append --user flag to install it just for you or --prefix=/install/path to choose the installation location):

$ pip install fsource

You can also simply download the source, since there are no external dependencies.

$ git clone github.com/mwallerb/fsource
$ cd fsource

In this case you should run bin/fsource instead of fsource, which augments the python path with the downloaded source files.

Command line interface

fsource currently features a command line interface:

• a parser, which takes a Fortran file and outputs an abstract syntax tree (for the definitions) allowing you to extract modules, subprograms, derived types, parameters, etc.:

  $ fsource parse FILE.f90
  

• a wrapper, which builds on the Fortran parser to extract module variables, types and subroutines which can be interfaced with C and generates header files for them:

  $ fsource wrap FILE.f90
  

• a line splicer and a lexer, low-level tools which split a Fortran file into a set of logical lines and tokens, respectively. This allows you to set up your parsing infrastructure on top of fsource:

  $ fsource splice FILE.f90
  $ fsource lex FILE.f90
  

fsource's Issues

Question about handling Hollerith format fields

fsource lex does not recognize the token 1H1 in the line FORMAT (1H1)

This should be expected behavior since fsource is advertised as handling F77 and later, and Hollerith fields were considered obsolete by F77.

I am working on a project to detect and evaluate problematic constructs in legacy code such as Hollerith fields, alternate return points, ENTRY statements, character data stored in non-character variables, etc. My question is where I should focus my efforts to extend lexer.py to recognize these difficult and obsolete constructs? After a quick scan of lexer.py, it looks like I should modify formattok in get_lexer_regex() - does this seem reasonable?

Parser fails at INCLUDE statements

Hi,
I tried to run the parser on (part of) this package, and got the following error:

> fsource parse ref-calc.v1.6.f --fixed-form


ref-calc.v1.6.f:42:7: parser error: malformed statement inside program
|
|             INCLUDE "PARAM"
|             ^~~~~~~

The ref-calc.v1.6.f file begins:

      program leed

C  include parameter statements for dimensions etc

      INCLUDE "PARAM"

C  set unchangeable constants

      INCLUDE "GLOBAL"

Am I missing some option to run the parser on the entire project in a way that would make the INCLUDE statements work, or is this simply not implemented yet?

Does not parse program without program statement

For

print*,"hi"
end

I get

c:\python3\code>fsource parse try.f90
Traceback (most recent call last):
  File "c:\intelpython3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\intelpython3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\intelpython3\Scripts\fsource.exe\__main__.py", line 9, in <module>
  File "c:\intelpython3\lib\site-packages\fsource\__main__.py", line 187, in main
    cmd_parse(args)
  File "c:\intelpython3\lib\site-packages\fsource\__main__.py", line 167, in cmd_parse
    ast = parser.compilation_unit(tokens, fname)
  File "c:\intelpython3\lib\site-packages\fsource\parser.py", line 222, in rule_setup
    value = fn(tokens, *args)
  File "c:\intelpython3\lib\site-packages\fsource\parser.py", line 1851, in compilation_unit
    expect_cat(tokens, lexer.CAT_DOLLAR)
  File "c:\intelpython3\lib\site-packages\fsource\parser.py", line 122, in expect_cat
    raise NoMatch()
fsource.parser.NoMatch

fsource does work for

program main
print*,"hi"
end program main

Scale factor without comma separator not recognized

The scale factor edit descriptor P may or may not be separated from other edit descriptors with a comma. The rules on this are not terribly clear (see Steve Lionel's comment on https://stackoverflow.com/questions/54945112/error-comma-required-after-p-descriptor-in-format-string). The case I typically see resembles 1PE12.4 where the comma is omitted.

I had some success replacing:

    formattok = r"""\d*(?: [IBOZ] \d+ (?: \.\d+)?
                         | [FD]   \d+     \.\d+
                         | E[NS]? \d+     \.\d+  (?: E\d+)?
                         | G      \d+ (?: \.\d+  (?: E\d+)?)?
                         | L      \d+
                         | A      \d*
                         | [XP]
                         )(?=\s*[:/,)])"""

with

    formattok = r"""\d*(?: [IBOZ] \d+ (?: \.\d+)?
                         | [FD]   \d+     \.\d+
                         | E[NS]? \d+     \.\d+  (?: E\d+)?
                         | G      \d+ (?: \.\d+  (?: E\d+)?)?
                         | L      \d+
                         | A      \d*
                         | X
                         )(?=\s*[:/,)])
                    |(?:-?\d+P)(?=\s*,?[\d:DEFG])"""

in get_lexer_regex() in lexer.py but I'm not convinced the lookahead assertion (?=\s*,?[\d:DEFG]) is necessary or correct.

wrapper fails for simple testprogram

the log is actually the f90-program. sebastian@debian:~/ps/nms2018$ ~/.local/bin/fsource wrap u06a.f90
/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py:156: UserWarning: CANNOT IMBUE <fsource.analyzer.CompilationUnit object at 0x7f9404b7bf90> WITH program_decl
warnings.warn("CANNOT IMBUE %s WITH %s" % (parent, self.ast[0]))
/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py:159: UserWarning: CANNOT RESOLVE program_decl
warnings.warn("CANNOT RESOLVE %s" % self.ast[0])
Traceback (most recent call last):
File "/home/sebastian/.local/bin/fsource", line 10, in
sys.exit(main())
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/_cli.py", line 234, in main
cmd_wrap(args)
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/_cli.py", line 214, in cmd_wrap
print (asr.cdecl().get())
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py", line 189, in cdecl
return CWrapper.union(self.children)
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py", line 67, in union
wraps = tuple(elem.cdecl() for elem in elems)
File "/home/sebastian/.local/lib/python2.7/site-packages/fsource/analyzer.py", line 67, in
wraps = tuple(elem.cdecl() for elem in elems)
AttributeError: 'Ignored' object has no attribute 'cdecl'
u06a.log