benjamincorcoran/sasdocs

A small parsing library for SAS code

sasdocs's Issues

Suggestion for simple example in the tests/samples folder

Hello,

I was very happy to find this library, thank you for making it open source!

When I tried to follow the simple example I got these error/warning messages:

Unable to directly resolve path: [WinError 3] The system cannot find the path specified: 'a\\bad\\path'
Unable to directly resolve path: [WinError 3] The system cannot find the path specified: 'a\\bad\\path'
Unable to directly resolve path: [WinError 3] The system cannot find the path specified: 'path\\to\\output'
ERROR:sasdocs.program: "a\bad\path": Unable to resolve path: [WinError 3] The system cannot find the path specified: 'a\\bad\\path'

It took me a little while to understand that it was due to the %include "a/bad/path"; and libname output "path/to/output"; statements in the samples.

Maybe you could consider to use an initial example that does result in error/warnings messages? Or mention something about the paths in the instructions? Just a suggestion.

Excited to start using your library! Thanks again. Cheers

/ Jonas

Handle macro variables when required usage is string

Parse calls to macros

Issue

Call to macros are not parsed and so drop the over all parsed percentage for the sasProgram. Must be careful not to capture macro language as macro calls

/*This*/
%runMacro;

/*Not this*/
%put;

Object level logging

Issue

Make object level logging for better debugging.

Unit tests for sasProgram class

Need to write unit tests for sasProgram class

Resolve macroVariables where possible

Issue

The following should be resolved so that the library data has a valid path. Repeat process for %include statements.

%let mVar = C:\SASData;
libname data "&mVar.";

Directive for SAS files outside of docs folder

Macro arguments don't resolve paths

/ options for datasteps and procedures

Issue

Parser currently fails when presented with a datastep or procedure with explicitly defined options.

data test / view=testvew;
    set test;
run;

Add word boundaries to set

Merge Sphinx integration and Jinja branches into single documentation generation branch

Parser for `proc sql` statements

Issue

SAS proc sql statements are not currently parsed. These should be parsed into a object similar to a datastep or procedure, with multiple potential inputs and outputs. Column creation and join methods can be largely ignored at this point.

proc sql;
    create table work.foo from 
    select * from work.bar;
quit;

Considerations

• Proc sql statements can have multiple SQL statements between the proc sql and quit tags.
• * text ; is a valid inline comment delimiter in other contexts
• join statements can be repeated multiple times
• select statements can be nested indefinitely
• select into :mvar is a valid method for creating macro variables

Sphinx SAS directive handles folder or files

Option network and codemirror switches in ..sasinclude

Give SAS objects parent attribute pointing to SASProgram

Issue

Need to be able to get to parent program from individual SAS object. Add pointer to parent program where object was parsed.

Parser fail in presence of only MacroEnd

Issue

rebuild_macros fails when only a macro end is parsed from the SAS program.

Handle broken paths in jinja2 markdown

Sphinx Warning

Unknown target name. Do no write out to markdown link if no available link.

Unit tests for sasProject class

Need to write unit tests for sasProject class

Macro options

Issue

Parser fails when macro defined with options i.e.

%macro test /des='Hello' secure; 
%mend;

Not parsing spaces immediately after brackets in dataline options

Compatibility with python 3.6 & 3.7

Multiple outputs in procedures

Issue

Current parser does not capture multiple outputs in procedures.

proc summary data=test;
    class a;
    var b;
    output out=sum sum=;
    output out=cnt n=;
run;

Unit tests for jinja & sphinx

Parse non-data producing procedures

Issue

Currently no data producing procedure statements are not parsed.

E.g.

proc datasets lib=work kill; run;
proc print data=test; run

get posistion of object in raw code from parser

Issue

In order to reference back to the raw code, need to grab the point in the raw code string that the parser extracts the object.

Space between dataobject and ; generate blank data item

Issue

The following code

data test ;
    set a;
run;

is parsed into two outputs work.data and work.

PROC SQL fails to parse with mark=True

Issue

proc sql statements fail to parse when force_partial_parse has mark argument set to True. Issue is attempting to run set_found_posistion on parsed list of SQL statements.

Libname clear

Clearing libname should be parsed correctly.

Remove D3.js and Codemirror.js and use CDNs

Add a macro index page to documentation

Optional space when both macro argument default and documentation

Exploit networkx and d3.js to produce live network diagram

Produce network diagrams showing flow of data between steps and procedures in a single piece of SAS code.

Extension:
Display SAS code diagrams in context of large project.

Identify null datasets as such

Macro variables cause network JSON dump to fail

Maco variables cause network JSON dump to fail as cannot serialise python object

Fix:

Use _lib and _ds attributes, which have already encoded macrovariables into strings.

Give Macro object an about attribute

Issue

Macro object needs an about attribute capturing the documentation of the macro variable. This should be the first set of comments inside the macro. The comments need to be collapsed into a single string object.

Update documentation for macroinclude directive

Update object documentation

Change ..sasinclude directive action based on export format

In the case of a html export from sphinx, it is possible to exploit additional javascript packages for code markup and d3.js see #49

Documentation for Sphinx directive

Write a tutorial for using the Sphinx directive.

Exploit jinja2 to generate markdown documentation

Use jinja2 templating library to output parsed SAS objects into markdown documents natural documentation

Parsed with None for inputs/outputs needs to bet set to empty list

Test cases for sasProgram class

Requires

Test cases for the sasProgram class.

• Correctly loads and fails to load valid and invalid paths respectively
• Parses code as expected
• get_objects yields objects as expected

Reference to path fails for libname functions

Libname functions like below, fail when libname.path is called as no path attribute is generated.

libname test clear

datasteps without a set or merge statement

Issue

Currently only datasteps with a set or merge statement are parsed. However this does not account for datalines and cards statements, as well as infile statements

data a;
    input a;
    datalines;
1
;
run;

Should produce dataStep object with single output of dataObject: a

proc import parsing should capture imported file

Issue

Proc import parsing should capture the datafile path as well as producing a dataObject from the out statement.

proc import datafile="path/to/file.csv" dbms=csv out=a; run;