m-zakeri/codart

Source Code Automated Refactoring Toolkit

I enjoy domain-independent problem-solving from a Computer Engineering (CE) perspective. I like devising new thinking paradigms, problem-solving methods, and systems for complexity engineering. Complex real-life problems could be simplified and solved through various thinking paradigms: systematic thinking, abstract thinking, computational thinking, data thinking, network thinking, and eventually, what I call, reverse computational thinking.

⚡READ MORE | Explore My RESEARCH | Explore My TEACHING | My PhD in a nutshell | For CE students

codart's Issues

Extract method

Extract Method

The extract method is a kind of refactoring that avoids duplications.
For example, consider this class:

package ir.mag.test;

public class Main {

    public static void main(String[] args) {
        Student student = new Student("97524698", "smith", 20);
        student.actionNumberOne();
        student.actionNumberTwo();
    }

}


class Student {
    private String studentId;
    private String name;
    private int age;

    public Student(String studentId, String name, int age) {
        this.studentId = studentId;
        this.name = name;
        this.age = age;
    }

    public void actionNumberOne() {
        this.name = "IUST_" + this.name;
        System.out.print("Student { ");
        System.out.print("name: " + this.name + ", ");
        System.out.print("age: " + this.age + ", ");
        System.out.print("studentId: " + this.studentId + " }\n");
    }

    public void actionNumberTwo() {
        this.age = 32;
        System.out.print("Student { ");
        System.out.print("name: " + this.name + ", ");
        System.out.print("age: " + this.age + ", ");
        System.out.print("studentId: " + this.studentId + " }\n");
    }

    public String getStudentId() {
        return studentId;
    }

    public void setStudentId(String studentId) {
        this.studentId = studentId;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
}

To avoid duplications, it is possible to create a new method. Just like this:

package ir.mag.test;

public class Main {

    public static void main(String[] args) {
        Student student = new Student("97524698", "smith", 20);
        student.actionNumberOne();
        student.actionNumberTwo();
    }

}


class Student {
    private String studentId;
    private String name;
    private int age;

    public Student(String studentId, String name, int age) {
        this.studentId = studentId;
        this.name = name;
        this.age = age;
    }

    public void actionNumberOne() {
        this.name = "IUST_" + this.name;
        printStudent();
    }

    public void actionNumberTwo() {
        this.age = 32;
        printStudent();
    }

    public String getStudentId() {
        return studentId;
    }

    public void setStudentId(String studentId) {
        this.studentId = studentId;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public void printStudent() {
        System.out.print("Student { ");
	System.out.print("name: "+this.name+", ");
	System.out.print("age: "+this.age+", ");
	System.out.print("studentId: "+this.studentId+" }\n");
    }

}

Make class final refactoring is not working

Describe the bug
The make class final refactoring existing in refactorings/make_class_final.py doesn't work at all!

To Reproduce
Steps to reproduce the behavior:

1. Go to refactorings/make_class_final.py'
2. Run the script on your test project.
3. See the problem

Expected behavior
Make a class final and check its post-conditions.

Extract subclass should create the new class in a new file

Describe the bug
extract subclass refactoring appends new class after source class.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'refactorings/extract_subclass.py'
2. Run script on your test project
3. See the problem

Expected behavior
Create a new file and put the new class there!

Move Field: Issue with package name

Describe the bug
Cannot move fields in classes that don't belong to a package! As you know in Java classes in "src" folder don't need to have a package name! The problem is that I couldn't move field in those classes!
Consider this example:

src/SourceClass.java:

public class SourceClass {
    private int field_for_move = 33;

    public int getFieldForMove(){
        return this.field_for_move;
    }

    public void method_for_move(){
        System.out.println("SourceClass.method_for_move()");
    }
}

src/TargetClass.java:

public class TargetClass {
    
}

Error Print

Traceback (most recent call last):
  File "/home/ali/Documents/dev/CodART/refactorings/move_field.py", line 256, in __get_usage
    source_class, target_class, field = self.get_metadata(program)
  File "/home/ali/Documents/dev/CodART/refactorings/move_field.py", line 36, in get_metadata
    target_class = program.packages[self.target_package_name].classes[self.target_class_name]
KeyError: 'TargetClass'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ali/Documents/dev/CodART/refactorings/move_field.py", line 450, in <module>
    refac = refactoring.move()
  File "/home/ali/Documents/dev/CodART/refactorings/move_field.py", line 330, in move
    usages, program = self.__get_usage()
  File "/home/ali/Documents/dev/CodART/refactorings/move_field.py", line 258, in __get_usage
    raise UnResolvedMetaError("Source or destination not found!")
__main__.UnResolvedMetaError: Source or destination not found!

File Path
refactorings/move_field.py

Inline Class: Major Problems!

1. Wrong input structure.
2. Errors related to listener and parser
3. Wrong directory structure.

Inline Class is not working at all!

Removing dependency to Understand

The problem

CodART uses Understand Python API for the following task:

• Finding the usages (ref by) of a given entity (variable, method, class, interface, and enum) in the program for some refactoring operations.
• Computing source code metrics required for quality attribute measurements and code smell detection.

The usage of Understand must be replaced with a free implementation of Understand API in CodART.

Describe the solution you'd like

Find the required dependencies (def-use chain) between different source code entities, for refactoring operations, must be performed by developing a static analyzer with ANTLR listeners.
To this aim, the first step is to implement analysis passes to create a program Symbole Table like the Understand database.
The second step is to implement the required query on the created Symbol Table.
Some best practices and class diagrams for building the required Symbol table are discussed in [1, 2].

Describe alternatives you've considered

To find an open-source implementation of Understand Python API.

References
[1] N. Tsantalis and A. Chatzigeorgiou, “Identification of move method refactoring opportunities,” IEEE Trans. Softw. Eng., vol. 35, no. 3, pp. 347–367, May 2009, doi: 10.1109/TSE.2009.1.
[2] V. E. Zafeiris, S. H. Poulias, N. A. Diamantidis, and E. A. Giakoumakis, “Automated refactoring of super-class method invocations to the Template Method design pattern,” Inf. Softw. Technol., vol. 82, pp. 19–35, Feb. 2017, doi: 10.1016/j.infsof.2016.09.008.

Command-line interface (CLI) for CodART

The problem
There is no unified interface for the CodART project.
As the CodART project is developed and the number of features is increased, it is vital to have an interface to interact with this framework especially for ad-hoc use and testing.
Currently, the project contains several main files which are confusing. None of them implement a standard interface.

Describe the solution you'd like
A primary solution is to implement a command-line interface (CLI).
The CLI can be added to the main module in the projects' root. For example, to apply a specific refactoring to a given Java project the following command would be nice:
codart --refactoring "encapsulate_field" --project "./P1" --long_class_name "A" --feild_name "f" --refactored_project "./P2"

Alternatives solution
There are other alternative solutions that are more suitable for future development:

• A graphical user interface (GUI)
• Web user interface or rich internet application (RIA)
• Python APIs

Additional context
The important problem to consider is that the CLI must be as standard and comprehensive as possible. It could be changeable and understandable. For example, have comprehensive help (-h switch, etc.) and documentation.
Table 1 of the project documentation defines all required arguments for each refactoring which should be considered when developing the CLI.

Make Method/Field Non-Static

Describe the bug
When we are going to convert a static field/method to a non-static one, we must change all static usages. For instance, ClassName.field should be converted to new ClassName().field

Extract subclass propagation

Describe the bug
There is no propagation after extract subclass

Expected behavior
I think (I'm not sure) it's better to change the code properly due to the refactoring. For example method calls to previous class should refer to the new extracted class.

Additional context
@m-zakeri
Without propagation the source code is still compile-able! Is it necessary to propagate changes? If not please close this issue!

Push Down Method: Not Working!

Describe the bug
I defined a new package with three classes: Unit, Tank, Soldier:
push_down_method/Unit.java:

package push_down_method;

public class Unit {
    public int getFuel(){
        System.out.println("push_down_method.Unit.getFuel()");
        return 25;
    }
}

push_down_method/Tank.java:

package push_down_method;

public class Tank extends Unit {
    
}

Soldier is also an empty class.
I moved want to move getFuel() to Tank.java but I couldn't. The code just removed it from Unit.java but did not move it to Tank.java,
Here is my main config:

my_list = get_filenames_in_dir('/home/ali/Desktop/JavaTestProject/src/')
    print("Testing pushdown_method...")
    if PushDownMethodRefactoring(my_list, "push_down_method", "Unit", "getFuel", ["Tank", "Soldier", ]).do_refactor():
        print("Success!")
    else:
        print("Cannot refactor.")

File Path
refactorings/pushdown_method2.py

Collapse hierarchy does not check pre-conditions

Describe the bug
For collapse hierarchy it's required to make sure about input classes. You can look at here for more details about this refactoring

To Reproduce
Steps to reproduce the behavior:

1. Go to 'refactorings/collapse_hierarchy.py'
2. Run the script on your test project
3. See error

Expected behavior
As an example if A does no extends B we should check that before beginning the process!

Extract Method: Does not create a new method!

Describe the bug
I created a class called ExtractMethod:
src/ExtractMethod.java:

public class ExtractMethod {

    String name = "Ali";

    void printBanner() {
        System.out.println("ExtractMethod.printBanner()");
    }

    int getOutstanding(){
        return 334;
    }

    void printOwing() {
        printBanner();

        // Print details.
        System.out.println("name: " + this.name);
        System.out.println("amount: " + getOutstanding());
    }
}

I want to extract lines 17 and 18 (including println calls) into a new method called printDetails, so I set the configuration like this:

_conf = {
        'target_package': '',
        'target_file': "/home/ali/Desktop/JavaTestProject/src/ExtractMethod.java",
        'output_file': "/home/ali/Desktop/JavaTestProject/src/ExtractMethod.java",
        'target_class': 'ExtractMethod',
        'target_method': 'printOwing',
        # which lines of source code to extract
        # todo: after smell detection in phase 2 , we'll get a list of lines automatically
        'lines': [17, 18, ],
        'new_method_name': 'printDetails',
    }

But After running the script I got this:

public class ExtractMethod {

    String name = "Ali";

    void printBanner() {
        System.out.println("ExtractMethod.printBanner()");
    }

    int getOutstanding(){
        return 334;
    }

    void printOwing() {
        printBanner();

        // Print details.
		printDetails();
    }
}

As you can see it successfully detected lines 17 and 18 and replaced them with a new method call but there is no printDetails!

File Path
refactorings/extract_method.py

Group
@sinaziaee

No Move Field Refactoring!

Handle static and private for move method

Describe the bug
When we move a method to a new class we should take care of the variable and methods used in that method specially if they are private or static!

To Reproduce
Steps to reproduce the behavior try to move "getNumber2" in a target class:

Unclear Output Systems/Files

Increase Field Visibility

Parameters:
udb_path = "/home/ali/Documents/compiler/Research/xerces2-j/xerces2-j3.udb"
class_name = "ListNode"
field_name = "uri"

Problem:
It detects field and does the refactoring but doesn't propagate the changes! Look the following screenshots:

Move Method Doesn't Function Properly.

You can find this refactoring at refactorings/move_method.py
This codes moves a method to its destination but there are some problems described below:

1. Refactoring does not support formatting. It's better to re-write the source code in java pretty format.
2. The changes does not propagate correctly.
3. For propagating changes we should use the provided API.

Extract method candidate generation

The problem
Currently, CodART uses JDeodorant to generate extract method candidates.
The Python implementation of JDeodorant slicing algorithms is required for CodART.

Describe the solution you'd like
JDeodorant slicing algorithm [1] can be implemented using the ANTRL listeners. The following article should be extracted from source code during static analysis:

• Control flow graph (CFG)
• Call graph
• Program dependence graph (PDG)
• Control dependency graph
• Data dependency graph
• Backward slicing
• Block-based slicing

Describe alternatives you've considered
Other extract method candidate generation algorithms can be used [2, 3].

Prerequisite
Issue #114: Removing dependency to Understand

References
[1] N. Tsantalis and A. Chatzigeorgiou, “Identification of extract method refactoring opportunities for the decomposition of methods,” J. Syst. Softw., vol. 84, no. 10, pp. 1757–1782, Oct. 2011, doi: 10.1016/j.jss.2011.05.016.
[2] S. Charalampidou, A. Ampatzoglou, A. Chatzigeorgiou, A. Gkortzis, and P. Avgeriou, “Identifying extract method refactoring opportunities based on functional relevance,” IEEE Trans. Softw. Eng., vol. 43, no. 10, pp. 954–974, Oct. 2017, doi: 10.1109/TSE.2016.2645572.
[3] J. Hubert, “Implementation of an automatic extract method refactoring,” University of Stuttgart, 2019.

Pull Up Field Pre-Conditions

Actually this is not an issue but we can make this refactoring better by adding various pre-conditions. For example the defined pre-conditions don't support multi-level inheritance.

Functional interface for some refactoring

Describe the bug
Some refactoring does not have a class interface.
It makes them hard to use.

To Reproduce
Look at refactoring such as pullup_field in the refactoring package.

Expected behavior
It can be used as an object instance responsible for refactoring.

Screenshots
...

Additional context
The CodART is a fully object-oriented project. All refactoring must have a wrapper class with a clear interface.

Multiple Mains

Handle static method for push down method

File:
refactorings/pushdown_method.py

Description:
This files moves non-static methods correctly but fails in moving static methods.

Make abstract class refactoring is not working

Describe the bug
The make_abstract_class refactoring existing in refactorings/make_abstract_class.py is not working at all!

To Reproduce
Steps to reproduce the behavior:

1. Go to 'refactorings/make_abstract_class.py'
2. Run the script on your test project
3. See the problem

Expected behavior
Makes a class abstract, and propagates changes

Push Down Field: No propagation or invalid preconditions

Describe the bug
If we have a public field in a class, then push it down to its child, all the usages of that field should change otherwise the project will not compile!

File Path
refactorings/pushdown_field.py

Lacking post conditions for "make field final"

Java rules about final fields are:

• The blank final field length may not have been initialized
• When a variable is declared with final keyword, its value can’t be modified

Move Method

File: refactorings/move_method.py

1. Duplicated Imports for propagation.

Pull Up Constructor: Should check for constructor

Describe the bug
If we have the same constructor with same arguments in super class, this script adds a second one to the superclass which is incorrect and generates compile error.

Solution
Check if constructor exists.

File Path
refactorings/pullup_constructor.py

Increase Field Visibility: Propagation

Waiting for Mr Shakeri

MoveClassRefactoringListener, ReplaceDependentObjectsListener import error

ImportError: cannot import name 'MoveClassRefactoringListener' from 'refactorings.move_class' (E:\git\CodART\refactorings\move_class\__init__.py)

file affected: move_class /move_class.py (main.py)

Extract Class Doesn't Accept New File Path

File: refactorings/extract_class.py
It creates the new class in the same file that sourse_class exists.
We should be able to declare a new file path and the new generated class should write in that path.

Move method candidate generation

The problem
Currently, CodART uses JDeodorant to generate move method candidates.
The Python implementation of JDeodorant similarity algorithms is required for CodART.

Describe the solution you'd like
JDeodorant similarity algorithm [1] can be implemented using the ANTRL listeners.

Describe alternatives you've considered
Move method candidate can generate when detecting feature envy code smells as move method refactoring opportunities [2].

Prerequisite
Issue #114: Removing dependency to Understand

References
[1] N. Tsantalis and A. Chatzigeorgiou, “Identification of move method refactoring opportunities,” IEEE Trans. Softw. Eng., vol. 35, no. 3, pp. 347–367, May 2009, doi: 10.1109/TSE.2009.1.
[2] M. K. B. Fowler, Refactoring: improving the design of existing code, Second Edi. Addison-Wesley, 2018.

Move Method: No Propagation.

Describe the bug
When move a method from one class to another class we should change references in project!
File Path
refactorings/move_method.py

RemoveField: RemoveField does not detect fields

Tested on \CodART\testproject\input\src\main\java\org\json\JSONObject.java:

my_listener = RemoveFieldRefactoringListener(common_token_stream=token_stream, class_identifier='JSONObject', fieldname='NUMBER_PATTERN', filename=args.file)

Error: Field "NUMBER_PATTERN" NOT FOUND (tested on a test field called N too)
File : remove_field/remove_field.py (main.py)

EncapsulateFiledRefactoringListener not working correctly for private fields

EncapsulateFiledRefactoringListener defines getter and setter for private fields and also adds additional getter and setter for a field that has getter and setter already.
input:

class A_New extends A
{
    public string h;

    void printH(){
        print(this.h);
    }
}
class A
{
    private int f; /* printF , printF, */
    public int g; /* printF, printG */


    // Method 1
    public void printF(int i)
    {
        this.f = i * this.f;
    }

    // Method 2
    public void printF(float i){
        this.f = (int) (i * this.f);
        this.g = (int) (i * this.g);
    }

    // Method 3
    void printG(){
        print(this.g);
    }
}

Expected behavior:

output:

class A_New extends A
{
    public string h;

    void printH(){
        print(this.h);
    }
}
class A
{
    private int f; /* printF , printF, */
    public int g; /* printF, printG */

    public int getF(){
        return this.f;
    }

    public void setF(int f){
        this.f=f;
    }

    // Method 1
    public void printF(int i)
    {
        this.f = i * this.f;
    }

    // Method 2
    public void printF(float i){
        this.f = (int) (i * this.f);
        this.g = (int) (i * this.g);
    }

    // Method 3
    void printG(){
        print(this.g);
    }
}

Actual behavior:

/*After refactoring (Refactored version)*/
class A_New extends A
{
    public string h;

    void printH(){
        print(this.h);
    }
}
class A
{
    private int f;
	public int getF() { 
		 return this.f;
	}
	public void setF(int f) { 
		this.f = f;
	}
	/*End of accessor and mutator methods!*/

	public int g; /* printF, printG */

    // Method 1
    public void printF(int i)
    {
        this.setF(i * this.getF());
    }

    // Method 2
    public void printF(float i){
        this.setF((int) (i * this.getF()));
        this.g = (int) (i * this.g);
    }

    // Method 3
    void printG(){
        print(this.g);
    }
}

Information about the issue:

the original code adds an extra getter and setter for the field that already has a getter and setter. We fixed it.

Step to reproduce the behavior:

1. run EncapsulateFiledRefactoringListener for f variable with given above input.
2. run main and see the output.

Make Field Static and Non Static is not in correct format

File: main_omidvar.py

• There is a make field static implemented by Mr Omidvar, but it's not in our project format.
• There should be a file called make_field_static.py and all related listeners and functions plus a main function should be implemented there.
• It creates a new folder inside the root project and copies all files into it. While I think it should apply changes on the source codes on the same files.

Move Class: Invalid propagation!

Describe the bug
After moving class from one package to another, all imports should fix and refer to the target package, but this script deletes all the imports from source package while it's required to delete or replace some of them. For example, consider this scenario:

In the User class we used MoveClass (from source_package):

import source_package.MoveClass;
import source_package.SourceClass;

public class User {
    SourceClass sourceClass;
    MoveClass moveClass;
    
    public User(){
        this.sourceClass = new SourceClass();
        this.moveClass = new MoveClass();
        sourceClass.method_for_move();
        System.out.println(SourceClass.field_for_move);
        System.out.println(moveClass.getName());
    }
}

After moving MoveClass from source_package to target_package the User class changes as follow:

import target_package.MoveClass;
public class User {
    SourceClass sourceClass;
    MoveClass moveClass;
    
    public User(){
        this.sourceClass = new SourceClass();
        this.moveClass = new MoveClass();
        sourceClass.method_for_move();
        System.out.println(SourceClass.field_for_move);
        System.out.println(moveClass.getName());
    }
}

It's seen that it removed all the imports from source_package while it should remove just imports related to MoveClass, so our project is not runnable any more!

Exception in thread "main" java.lang.Error: Unresolved compilation problem: 
        The field SourceClass.field_for_move is not visible

        at User.<init>(User.java:10)
        at App.main(App.java:4)

File Path
refactorings/move_class.py

Pull Up Field: Multi-Level Inheritance

Describe the bug
Can not detect and refactor multi-level inheritance in pullup field refactoring.

Pull Up Constructor

This refactor helps to make the code cleaner and moves the identical codes in the constructors to the parent.
For example:

class Manager extends Employee {
  public Manager(String name, String id, int grade) {
    this.name = name;
    this.id = id;
    this.grade = grade;
  }
  // ...
}

and

class Admin extends Employee {
  public Admin(String name, String id, int grade) {
    this.name = name;
    this.id = id;
    this.grade = grade;
  }
  // ...

are the children of Employee and their constructors are similar. So:

class Manager extends Employee {
  public Manager(String name, String id, int grade) {
    super(name, id, grade);
  }
  // ...
}

and the parent class:

class Employee {
  public Employee(String name, String id, int grade) {
    this.name = name;
    this.id = id;
    this.grade = grade;
  }
  // ...
}

No Push Down Method Refactoring!

Move Field Refactoring

We have a kind of move field that moves only static field; we need a more general version of this refactoring to move any kind of field from one class to another!

Add C++ support for new Java grammar

The current parser which is based on JavaParserLabeled.g4 ANTLR grammar works with pure Python. Adding a C++ back-end for parsing Java source code files increases the paring efficiency.

Extract Class Doesn't Propagate Changes!

File: refactorings/extract_class.py
After doing the refactoring it should traverse all project java files and change the occurrences.

• If the occurrences refer to a connected components it should change the reference name properly due to source class and new class.
• If the occurrences refer to different connected components it should created to change reference name properly due to source class and new class.

MoveMethod doesn't work on real life examples

Move Field: No Propagation!

Describe the bug
When there is a static field in a class there are two cases to consider for propagation or pre-conditions:

1. That static field can be used in internal methods of its class
2. The static field can be used in external methods and classes

In both cases we need to propagate changes after refactoring otherwise the project will not compile at all!

To Reproduce
File Path: refactorings/move_field.py
Test the refactoring with a static field that used in several methods (internals and externals)

Extract Class: Can Not Refactor (tested on JSON)

Tested on E:\git\CodART\testproject\input\src\main\java\org\json\JSONPointer.java (ExtractClass main.py)

source_class = 'JSONPointer' new_class = 'JSONPointerNew' moved_fields = ['ENCODING'] moved_methods = ['unescape']
File Affected : extract_class/extract_class.py (main.py)

Make class concrete post conditions not working

Describe the bug
This refactoring does not check post conditions and after running this on my test project, I got this error when compiling:
The abstract method sides in type "Shape" can only be defined by an abstract classJava(67109227)

To Reproduce
Steps to reproduce the behavior:

1. Go to 'refactorings/make_class_concrete.py'
2. Run the script on your test project
3. Compile the test project
4. See error

Expected behavior
Check post conditions properly. Source code must be compile-able after refactoring!

Collapse hierarchy reads source code twice

Describe the bug
This refactoring reads source code twice, once for refactoring and once for propagating changes.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'refactorings/collapse_hierarchy.py'
2. Run the script on your test project

Expected behavior
It's better to read project once using understand API.

No Move Class Refactoring!

file placement

I think main.py and input.java files should come to the root folder due to the imports in main.py

from refactorings.encapsulate_field import EncapsulateFiledRefactoringListener
from refactorings.extract_class import ExtractClassRefactoringListener
from refactorings.gen.Java9_v2Lexer import Java9_v2Lexer
from refactorings.gen.Java9_v2Parser import Java9_v2Parser

The code above is the import section of main.py and each line imports from the refactorings folder which should be in the same hierarchy level as main.py . So if we move main.py and input.java to their parent folder this issue gets fixed. Otherwise, we should add root folder to system Path which is an unnecessary overdo.