vovak/codestyle

codestyle's Issues

Compute programmers coding style representation

Compute programmers coding style for top-10 and top-50 contributors in idea.

Compute statistics and clear extracted data

Compute stats on (extracted data)https://seafile.ifi.uzh.ch/d/808e216262174188b653/). We are interested in distribution of changes across programmers, distribution of paths across changes, some insights on additions/deletions/changes of methods.
Then we should clear the data based on insights we get. It includes removing of low-populated classes from dataset, de-noising etc.

Train a model for classification

Training includes several sub-problems:

1. Choose split of dataset
2. Choose a metric for evaluation
3. Choose hyperparameters

Review of code2vec implementations

Since we want to use code2vec, we need an implementation of it. There are quite many published implementations at the moment and we need a review with pros and cons for each one.

Mine data from other projects

Currently we're only working with the IDEA Community repo.

We need to mine the data from several other projects to be able to validate the results.

Add method packs to the model

1. Create new dataset with packs of several methods instead of single methods.
2. Change data loading pipeline.
3. Update model, give each method in a pack attention weight.

Research Questions

1. Get a distributed representation for programmers coding style
   a. Stylometry papers
   b. To analyze team dynamics based on code
2. Representation should be interpretable
3. Analyze characteristics of the representation:
   a. Clusterization, which programmers are considered similar
   b. How it changes through time
   c. Is it robust
4. Future work:
   a. Code review, other SE practices
   b. Make representation context-independent

Clean up the mining code

Currently our mining tool is far from nice: the implementation is obscure, undocumented, and hardcoded for the IntelliJ repo in places.

Required to make #10 and #11 possible:

• Remove hardcoded fragments

Also (better be done by the FSE deadline as we are going to share the code for reproducibility):

• Refactor
• Add comments
• Switch to jgit and get rid of the Python backend

Write the introduction and research questions

Begin writing introduction and formulate research questions.

Implement a standalone AST paths mining tool

With the recent success of the code2vec paper, we can expect a certain demand for a standalone tool capable of mining the AST-path-based representations of code.

It looks feasible to convert a part of our mining pipeline into such a tool and submit a short tool paper to MSR 2019.

Evaluation of the research

A possible approach to evaluation is predicting/projecting information about programmers from the representations but we need to know what to predict. To do so, we can look at PAN competitions and literature domain.

Explore code2seq

There is another paper by authors of code2vec called code2seq. It is based on the same idea as code2vec but has some differences in network architecture and problem statement.
We need to explore those differences and discuss them at the next talk.

Commit analysis

• Analyze several large github projects and their commit histories
• Collect information and statistics on the commits
• Select one project
• Formalize the way to represent commit diffs

AST parsing

• Try out parsing ASTs with Babelfish for C++, Java
  • Pay attention to parsing of incomplete code snippets
• Compare to possible alternatives (javaparser, intellij psi trees)