ghaffarian/progex

PROGEX (Program Graph Extractor); a cross platform tool for extracting graphical program representations from software source code

progex's Issues

Error when generating AST of large files

Hi,

When I try to generate AST for large files, PROGEX give me the following error:

Exception in thread "main" java.lang.NullPointerException
at java.base/java.lang.StringBuilder.(StringBuilder.java:124)
I have increased the memory. But still getting the error.

Thanks.

Error when generating ICFG

Before all, thank you for your excellent work ,it is very helpful for me.
But when I use command line for generating ICFG, some errors was reported as below:

java.io.FileNotFoundException: lib\src.zip 
java.io.FileNotFoundException: lib\src.zip 
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(Unknown Source)
        at java.util.zip.ZipFile.<init>(Unknown Source)
        at java.util.zip.ZipFile.<init>(Unknown Source)
        at ghaffarian.progex.java.JavaClassExtractor.extractJavaLangInfo(JavaClassExtractor.java:52)
        at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:72)
        at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
        at ghaffarian.progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
        at ghaffarian.progex.Execution.execute(Execution.java:219)
        at ghaffarian.progex.Main.main(Main.java:36)

So how to deal with it?
Thank you in advance for your help.
Lin

A question about the data dependency graph

Hi,

Thank you for your work.
May I ask why the generated data dependence graph contains control flow edges?

Thanks for your time.

FileNotFoundException: lib/src.zip

Hello,

When I try to execute ICFG I have this problem:

java.io.FileNotFoundException: lib/src.zip (Aucun fichier ou dossier de ce type)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:225)
at java.util.zip.ZipFile.(ZipFile.java:155)
at java.util.zip.ZipFile.(ZipFile.java:126)
at ghaffarian.progex.java.JavaClassExtractor.extractJavaLangInfo(JavaClassExtractor.java:52)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:72)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
at ghaffarian.progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
at ghaffarian.progex.Execution.execute(Execution.java:226)
at ghaffarian.progex.Main.main(Main.java:36)

What does that mean ?

I use this command line:
java -jar "/home/progex-3.2.0/progex.jar" -outdir /home/progex-3.2.0/output/synapse_1.0 -icfg -lang java -format dot /home/progex-3.2.0/projets_source/synapse_1.0

Error when generating ICFG

Hi, I am getting the following error while generating ICFG.
I tried the methods in other issues and still failed. How can I fix this problem?

FileNotFoundException lib\src.zip

The directory which is being scanned by progex does not have a reference to "src.zip" so I'm not sure how to chase the bug from a user perspective. Attached is a progex.log file showing the bug occurring. Aside from -info, it also happens with -cfg and -icfg. This code is great work! Thank you for posting it!
progex.log

support for Java 8 lambda expressions

Hi ghaffarian,
thx for your briliiant tool progex, but it seems not supoort parsing for lambda expression and method reference yet.
is there any plan for adding support for these new features?

Export to JSON Doesn't Properly Handle New-Line Characters

Hi, I find another error when export json.
for instance,

     {
      "id": 6,
      "line": 14,
      "label": "if (base.substring(i, i + remove.length()).toLowerCase().equals(
            remove.toLowerCase()))"
    },

and

{
      "id": 4,
      "line": 10,
      "label": "if (str.length() >= 2 &&
        (str.charAt(0) == 'g' && str.charAt(1) != 'g' ||
        str.charAt(str.length()-1) == 'g' && 
        str.charAt(str.length()-2) != 'g'))"
    },

All conditions in same IF STATEMENT are successfully in same node, however, it is not recognized as one string. Therefore, when I try to read that json, it is failed with JSONDecodeError. Can you fix it?

Issue in Export JSON

I export a json not a dot, and it generates a json file successfully.
However, when I try to read the json file using below codes, it shows an error JSONDecodeError: Expecting value

with open("Test1-CFG.json", 'r') as f:
          data = json.load(f)

I try to fix it. When I remove each comma in the list of nodes and edges, I can read the json file correctly.

From

  "nodes": [
    {
      "id": "n1",
      "line": "4",
      "code": "void main(String[] args)"
    },
    {
      "id": "n10",
      "line": "14",
      "code": "System.out.println(\"End\")"
    },
  ],
 "edges": [
    {
      "id": "e1",
      "source": "n1",
      "target": "n2",
      "label": ""
    },
    {
      "id": "e2",
      "source": "n2",
      "target": "n3",

to

  "nodes": [
    {
      "id": "n1",
      "line": "4",
      "code": "void main(String[] args)"
    },
    {
      "id": "n10",
      "line": "14",
      "code": "System.out.println(\"End\")"
    }
  ],
 "edges": [
    {
      "id": "e1",
      "source": "n1",
      "target": "n2",
      "label": ""
    },
    {
      "id": "e2",
      "source": "n2",
      "target": "n3",

Can you fix this in the code?
I think it is maybe in nodes and edges

If it is a last node or a last edge, it does not have to write ,(comma)

Problem Following the Usage-Guide and Building from Source

I'm trying to use progex, I'm having trouble following your usage guide, would you mind explaining how to setup everything in more details?

how do i get the progex.jar for java -jar PROGEX.jar /path/to/program/src

NullPointerException @ JavaICFGBuilder.java:1079

When running PROGEX's ICFG analysis on its own source code, the following exception is thrown:

Exception in thread "main" java.lang.NullPointerException                                                                                                                                                         
        at progex.java.JavaICFGBuilder$ICFGVisitor.classInstantiate(JavaICFGBuilder.java:1079)                                                                                                                    
        at progex.java.JavaICFGBuilder$ICFGVisitor.findClassbyName(JavaICFGBuilder.java:993)                                                                                                                      
        at progex.java.JavaICFGBuilder$ICFGVisitor.findVariableType(JavaICFGBuilder.java:969)                                                                                                                     
        at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprPrimary(JavaICFGBuilder.java:627) 
...

More precisely, the exception is raised while analysing line 32 of progex.java.JavaClass.java, which is:

    fields = new ArrayList<>();

The source of the issue is at line 1079 of JavaICFGGuilder.java inside the classInstantiate(...) method declaration, which is:

    String[] genericParam = params.split(",");

At this point a NullPointerException is raised, because params is null. This is most probably due to not handling the case where the generic type parameter is empty (e.g. new ArrayList<>();).

ICFG crashes!

ICFG crashes with a NullPointerException given the following test case:
Test3.txt

Note that other analyses (such as CFG and PDG) run successfully on the same test case.

Here is the stack trace:

Exception in thread "main" java.lang.NullPointerException
	at progex.java.JavaICFGBuilder$ICFGVisitor.findMethodbyName(JavaICFGBuilder.java:858)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprPrimary(JavaICFGBuilder.java:587)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprPrimary(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$ExprPrimaryContext.accept(JavaParser.java:7443)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprDotID(JavaICFGBuilder.java:600)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprDotID(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$ExprDotIDContext.accept(JavaParser.java:7617)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprMethodInvocation(JavaICFGBuilder.java:693)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprMethodInvocation(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$ExprMethodInvocationContext.accept(JavaParser.java:7257)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitVariableInitializer(JavaBaseVisitor.java:265)
	at progex.java.parser.JavaParser$VariableInitializerContext.accept(JavaParser.java:2948)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitVariableDeclarator(JavaBaseVisitor.java:251)
	at progex.java.parser.JavaParser$VariableDeclaratorContext.accept(JavaParser.java:2829)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitVariableDeclarators(JavaBaseVisitor.java:244)
	at progex.java.parser.JavaParser$VariableDeclaratorsContext.accept(JavaParser.java:2765)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitLocalVariableDeclaration(JavaICFGBuilder.java:468)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitLocalVariableDeclaration(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$LocalVariableDeclarationContext.accept(JavaParser.java:5322)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitLocalVariableDeclarationStatement(JavaBaseVisitor.java:496)
	at progex.java.parser.JavaParser$LocalVariableDeclarationStatementContext.accept(JavaParser.java:5267)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitBlockStatement(JavaBaseVisitor.java:489)
	at progex.java.parser.JavaParser$BlockStatementContext.accept(JavaParser.java:5203)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:251)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$BlockContext.accept(JavaParser.java:5136)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitTryStatement(JavaICFGBuilder.java:413)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitTryStatement(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$TryStatementContext.accept(JavaParser.java:5438)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitBlockStatement(JavaBaseVisitor.java:489)
	at progex.java.parser.JavaParser$BlockStatementContext.accept(JavaParser.java:5203)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:251)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$BlockContext.accept(JavaParser.java:5136)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitMethodBody(JavaBaseVisitor.java:356)
	at progex.java.parser.JavaParser$MethodBodyContext.accept(JavaParser.java:3924)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitMethodDeclaration(JavaICFGBuilder.java:505)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitMethodDeclaration(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$MethodDeclarationContext.accept(JavaParser.java:1913)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitMemberDeclaration(JavaBaseVisitor.java:160)
	at progex.java.parser.JavaParser$MemberDeclarationContext.accept(JavaParser.java:1797)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitClassBodyDeclaration(JavaBaseVisitor.java:153)
	at progex.java.parser.JavaParser$ClassBodyDeclarationContext.accept(JavaParser.java:1681)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitClassBody(JavaBaseVisitor.java:139)
	at progex.java.parser.JavaParser$ClassBodyContext.accept(JavaParser.java:1547)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitClassDeclaration(JavaICFGBuilder.java:231)
	at progex.java.JavaICFGBuilder$ICFGVisitor.visitClassDeclaration(JavaICFGBuilder.java:144)
	at progex.java.parser.JavaParser$ClassDeclarationContext.accept(JavaParser.java:811)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitTypeDeclaration(JavaBaseVisitor.java:41)
	at progex.java.parser.JavaParser$TypeDeclarationContext.accept(JavaParser.java:452)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at progex.java.parser.JavaBaseVisitor.visitCompilationUnit(JavaBaseVisitor.java:20)
	at progex.java.parser.JavaParser$CompilationUnitContext.accept(JavaParser.java:214)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
	at progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:82)
	at progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
	at progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
	at progex.Execution.execute(Execution.java:211)
	at progex.Main.main(Main.java:32)

mvn compilation problem

Hello,

Does the "mvn install" works well to build the source code ?
Personaly, I have problem with th none-logger and graphs dependencies ...

Adding Support for More Languages

Hi,

I wonder what other input languages are supported by this tool, apart from java, and how one may call them when using the application.

Regards
Hooman

Linking AST, CFG, and PDG

Hello,

Before all, congratulations for your source code and your presentation, it is very very clean and structured. I executed and generated the DOT files. That's excellent !!
I saw that it's possible to generate separately files AST / CFG / PDG... but how to make a merge of the 3? It's possible for me to complete your source code in this goal ? I would like to reproduce a single global graph merged like the Code property Graph defined in this document http://user.informatik.uni-goettingen.de/~krieck/docs/2014-ieeesp.pdf

Thank you in advance for your help

IllegalArgumentException: no such vertex in graph: 0: ELSE

While processing progex.java.JavaDDGBuilder.java from PROGEX's own source code, the following exception is thrown during control-dependency analysis (JavaCDGBuilder), after processing line 620: debug("METHOD DEF INFO NOT FOUND!"); of the JavaDDGBuilder.java file:

Exception in thread "main" java.lang.IllegalArgumentException: no such vertex in graph: 0: ELSE
	at org.jgrapht.graph.AbstractGraph.assertVertexExist(AbstractGraph.java:132)
	at org.jgrapht.graph.AbstractBaseGraph.addEdge(AbstractBaseGraph.java:178)
	at progex.java.JavaCDGBuilder$ControlDependencyVisitor.checkBuildFollowRegion(JavaCDGBuilder.java:670)
	at progex.java.JavaCDGBuilder$ControlDependencyVisitor.addNodeEdge(JavaCDGBuilder.java:642)
	at progex.java.JavaCDGBuilder$ControlDependencyVisitor.visitStatementExpression(JavaCDGBuilder.java:175)
	at progex.java.JavaCDGBuilder$ControlDependencyVisitor.visitStatementExpression(JavaCDGBuilder.java:51)
...

The issue is most likely related to the else if (...) statement at line 585; which may not be processed correctly by JavaCDGBuilder.

The input Java file is attached below:
JavaDDGBuilder.java.txt

Error while creating ICFG

The code in the file JavaClassExtractor looks for path "\lib\src.zip" which is not available.

Interprocedural Control-Flow Analysis

java.nio.file.NoSuchFileException: lib\src.zip
java.nio.file.NoSuchFileException: lib\src.zip
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:198)
at java.base/java.nio.file.Files.readAttributes(Files.java:1842)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1160)
at java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:719)
at java.base/java.util.zip.ZipFile.(ZipFile.java:239)
at java.base/java.util.zip.ZipFile.(ZipFile.java:169)
at java.base/java.util.zip.ZipFile.(ZipFile.java:140)
at ghaffarian.progex.java.JavaClassExtractor.extractJavaLangInfo(JavaClassExtractor.java:52)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:72)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
at ghaffarian.progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
at ghaffarian.progex.Execution.execute(Execution.java:219)
at ghaffarian.progex.Main.main(Main.java:36)

Process finished with exit code 0