How can I use the JCLLexer file ?

[cschneid-the-elder]

How can I use the JCLLexer file ?

[cschneid-the-elder]

I was using ANTLR 4.7.2, just that little bit later than 4.7.1, so maybe that's it?

[Heidernlee]

@cschneid-the-elder

Thanks for the response.

I tried 4.7.2, still Errors...

Could you kindly please check the following questions ?

1. Is this g4 file newest ? the error says [missing RULE_COLON at '{ COMMENTS }'] about the [channels] statement
2. Is the [JCLParser & JCLLexer] only needed ? your repo contains other g4 Files ,like [JCLPParser ,JCLDDAMPParser].
3. Call JCLParser one time is only for one step, right ? for parse full jcl file , we must call JCLparser many times ?

It's so hard to find a good Grammar-Parser for JCL in 2020, Thanks again.

[cschneid-the-elder]

1. I copied the contents of the JCLLexer.g4 file here on github and compared it to the copy I have locally. diff reports them as the same. I ran java -jar antlr-4.7.2-complete.jar JCLLexer.g4 and it generated the .java and .tokens files with no errors reported.

2. In the demonstration application (Demo01.java) I use JCLPPLexer and JCLPPParser in PreProcessing the JCL - this is where line numbers in columns 73 - 80 get removed, an edge case with continued comment gets handled, symbolic parameters get replaced with their values., and INCLUDEs and PROCs are resolved. I use the other .g4 files to parse special cases like the AMP parameter, and the contents of SYSTSIN when PGM=IKJEFT01 (or IKJEFT1A or IKJEFT1B) is executed.

3. In the demonstration application I call the preprocessor lexer once per JCL file, get rid of the line numbers in columns 73 - 80, fix up continued comments, write the fixed up file to a temp file, then run the preprocessor lexer and parser on that temp file, then iteratively resolve symbolic parameters, INCLUDEs, and PROCs, creating temp files of each iteration. When all the INCLUDEs are resolved, I run the JCLLexer and JCLParser on that final iteration. I use ANTLR Listeners to gather the information needed to do all this.

There are couple of examples of why this is so complicated in the readme.md file.

[Heidernlee]

@cschneid-the-elder

Thanks for your speedy reply

1. I can also get the lexer.java file by running lexer.g4 .
   But I don't know whether these Errors on the Eclipse made wrong result of JCL Parsing.
   I just compared your Lexer.java and mine , It's all the same so I decided to Ignore these Errors.

2. Thanks, I'll run the Demo01 and check the result.

3. I've tried make a Grammar-Tree on Eclipse with JCLParser directly,
   but only one [step] statement created successfully ,like this :
   

   I think the problem is here :
   jclStep : execStatement (cntlStatementAmalgamation |
   ddStatementAmalgamation | outputStatement |
   includeStatement | commentStatement)* ;
   between [execStatement] and [ddStatemrnt], Is anything should be processed while Preprocessing ?

[cschneid-the-elder]

I don't use Eclipse, so I'm not sure what's happening there.

There are often many steps in one JCL job or procedure, the lexer processes them all, no need to call it once for each step.

When I create an instance of the JCLLexer, it's from a CharStream which is created from the entire file of JCL steps, that file is like a member in a mainframe PDS. The lexer emits a CommonTokenStream, which is used to instantiate a parser, which assembles meaningful JCL constructs from the tokens. Then I create an instance of an ANTLR listener I wrote to pick up the JCL constructs of interest.

I don't use either the lexer or the parser on their own, except with the ANTLR test rig - see the testrig and testrigpp targets in the Makefile.

[Heidernlee]

@cschneid-the-elder

Sorry, I'm so blind....I just noticed your comment from [mapa/jcl/],
it shows how to run the JCLParser.jar.....

Now It seems working, but if a JCL File contains [/*] like this:

//AAAA
/*
//BBBB

many Errors occurred, and I think It's correct for JCL

[cschneid-the-elder]

//AAAA
/*
//BBBB

Interesting. I shall look into it.

[cschneid-the-elder]

If you switch to the dlmstmt01 branch, there is a JCLParser.jar that you can try.

[Heidernlee]

@cschneid-the-elder

I'll replace [SLASH STAR] => [SLASH SLASH STAR] from [rewriteWithoutCol72to80] method simply,
I don't know why it didn't make a Visible Full-Grammar-Tree on Eclipse by using Parser.g4 directly,
But now your Antlr-Grammar can parse all my Example JCL Programs.

It's very, very helpful, Thanks again.

My next step is a Antlr-Grammar-Parser for RPGII Language,
Do you happen to have one?

[cschneid-the-elder]

I'm glad the JCL grammar is useful to you. Sorry, I don't have a grammar for RPG II.