thiscakeisalie/memo

MeMo

MeMo is an approach to automatically identify metamorphic relations inside natural language, unstructured code comments. The identified MRs are automatically translated into executable test oracles that can be weaved into existing test suites. As of now the prototype works on Javadoc method summaries, and thus produces Java assertions.

Releases

To produce the executable (MeMo-1.0-all.jar), after cloning the repo launch ./gradlew shadowJar.

Prerequisites

Please notice that both the Gradle wrapper and the Major tool needed for the analysis explained in the next section may show inconsistent behaviour with Java 9+. Until a workaround for such issues is not found, you are advised to get a Java 8 release before running any experiment with MeMo (e.g., here)

Translations

Under this folder you find the ground truth. You may check the files for each project class manually. The format looks like:

expected-equivalences/<project-identifier>/<fully-qualified-class-name>_goal.json

Each first-level object is the representation of a method of the class. Inside each method, look for "equivalence" objects: if attribute "condition" is non-empty, that is the translation it is expected by MeMo for such method and comment.

Example. To check class org.apache.commons.math3.geometry.euclidean.threed.FieldRotation of Apache Commons Math, reach the corresponding file.

You will find an expected translation for method applyInverseTo(org.apache.commons.math3.geometry.euclidean.threed.FieldRotation r).

To verify MeMo performances on such dataset, clone this repo and launch ./gradlew accuracyEQComplete, which will show the translations produced on the present ground truth. For each method of each class, MeMo may produce exactly the translation written in the file (Correct), a non-empty translation that differs from the expected one (Wrong), an empty translation when condition is not empty (Missing), or a non-empty translation when no condition was expected (Spurious).

Mutation analysis experiments

Please go to this repository for complete instructions on how to reproduce the Mutation analysis results. These experiments practically show how MeMo assertions can detect bugs.

Troubleshooting

If you encounter any problem, please open an issue or contact the main developer at [email protected]