Code similarity via graph-based features.
This folder contains a modified version on GraphGrepSX. The changes concern the creation of an additional XML output of input graphs index (as prefix tree).
This folder contains the BNF grammars in G4 format of LLVM-IR 7.0.0 language. The original grammar description can be found on LLVM mailing list at this link.
This folder contains all the source files regarging the execution pipeline.
This folder contain ccpt
python package.
The source code is located in /src
directory and the unit tests in
/tests
directory.
This folder contains some workflow examples of the clustering and classification
problems. They are both contained inside a jupyter notebook.
It also contains a /data
folder in which are a sample of the Codenet dataset source files.
This folder contains a set of .csv
files which represents
some predefinite maps that can be used for A-CFG creation.
In order to make the preprocessing steps (.c
.dot
CFG)
of ccpt
works, clang-7
and opt-7
must be installed: v 7.0.0 required.
-
Build specific version of llvm project from source:
mkdir <llvm-7-dir> cd <llvm-7-dir> git clone https://github.com/llvm/llvm-project.git cd llvm-project git checkout tags/llvmorg-7.0.0 cmake -S llvm -B build -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DLLVM_ENABLE_PROJECTS=clang cmake --build build -j 8
-
Then, create a symlink:
ln -s /full/path/to/<llvm-7-dir>/llvm-project/build/bin/clang /usr/bin/clang-7 ln -s /full/path/to/<llvm-7-dir>/llvm-project/build/bin/clang++ /usr/bin/clang++-7 ln -s /full/path/to/<llvm-7-dir>/llvm-project/build/bin/opt /usr/bin/opt-7 ln -s /full/path/to/<llvm-7-dir>/llvm-project/build/bin/llvm-cxxfilt /usr/bin/llvm-cxxfilt-7
It is only required that Clang/Opt 7.0.0 could be called from CLI with clang-7
and opt-7
commands. To check if they work:
clang-7 --version
clang++-7 --version
opt-7 --version
llvm-cxxfilt-7 --version
To install graphviz on Ubuntu:
sudo apt install graphviz
To compile GraphGrepSX on your own machine:
cd /GraphGrepSX
make clean
make -B
cp ggsx /py/src/ccpt/cfg_to_trie
To install Python requirements it is raccomanded to create a new virtual environment:
python3 -m venv venv
To activate the virtual environment venv
:
-
Linux/MacOS
source venv/bin/activate
-
Windows
venv\Scripts\activate
To install requirements for the project in the virtual environment
venv
:
(venv)$ pip install -r py/requirements.txt
To install also additional requirements for development:
(venv)$ pip install -r py/requirements_dev.txt
(To deactivate venv):
(venv)$ deactivate
From virtual environment:
(venv)$ pip install -e py/
Test if ccpt
is installed:
pip show ccpt
For description and usage of ccpt
:
Launch jupyter notebook server:
python -m jupyter notebook
The analysis phase is performed using jupyter notebook. The analysis in these notebooks are done on a series of precomputed features extracted from the A-CFGs generated from the source code.
Execute these commands to generate the features starting from the source code.
Create a folder for the generated files inside /analysis
cd analysis
mkdir featuresData
Choose a file from the data folder and generate the features
python3 gen-features-callgraph.py data/p00000/[somefile] -od featuresData
In order to run the examples inside the jupyter notebooks some packages must be installed, which are:
- sklearn
- yellowbrick
After activating the virtual environment created above, execute the following commands:
skelearn
pip3 install -U scikit-learn
yellowbrick
pip install yellowbrick
After activating the virtual environment, start the jupyter server from the 'analysis' folder:
jupyter notebook
This command will start a new tab in the default browser showing a list of all the files and folder located into the folder in which the command has been executed, in this case analysis.
Select the desired notebook and execute the internal boxes.