rendon/test_program

Automation tool for programming contest problems.

TestProgram

TestProgram is a specific purpose testing tool for competitive programming programs, i.e. Codeforces, lightOJ, OmegaUp, etc.

When we solve a problem we have to very careful and aim to tackle every possible case. The more test cases our solution solves correctly before summiting the better.

You can always do something like:

#ifndef ONLINE_JUDGE
   freopen("input.txt", "rt", stdin);
   freopen("output.txt", "wt", stdout);
#endif

Or (my former strategy):

$ ./mysolution < input.txt > output.txt

However, if you have more than one test case, this soon becomes a hassle, error prone, time consuming and other problems related to manual testing.

I've developed TestProgram for help automate test case management and the execution of them. For now the supported languages are C++, Java, Python and Ruby, however it's easy to add support for other languages.

NOTE: For each language, TestProgram assumes that you have the corresponding compilers or interpreters installed.

TestProgram runs in GNU/Linux but it's also possible to run it on windows using Cygwin. I haven't tested it on Mac OS but I think it should run well without major complications.

#Usage For the impatients here you have two demos in video, the first one describes the general use and the second shows how to use it with Codeforces.

TestProgram work by mean of commands and each command has actions. These are the available commands:

• init Setups a contest environment.
• test Create, show, edit and delete test cases.
• gen Generate solution template.
• clean Remove any test related file.

###tp init Setups a programming contest environment in the current working directory.

usage: tp init <contest> [template]

Where contest is a key of a supported programming contest, e.g. codeforces. With [template] you can also indicate which programming language you want for the solution templates passing the file extension of a supported language. See the following list:

cpp     =>  C++
java    =>  Java
rb      =>  Ruby
py      =>  Python

###tp test Manage the test cases for a particular problem.

Available actions:

• add Creates a new test case manually.
• rm Deletes a specific test case.
• edit Edit a specific test case.
• show Shows the contents of a specific test case.
• run Executes a particular test case. If no test case is specified all test cases will be executed.

Each test case is identified by an integer, all actions, with exception of add, need this identifier to proceed.

###tp gen Generates a solution template for a supported language.

usage: tp gen <file.lang>

Where lang corresponds to the programming language extension, these are the extensions for the supported languages:

cpp     =>  C++
java    =>  Java
rb      =>  Ruby
py      =>  Python

###tp clean Remove all file generated by this tool.

Available options:

-r  Find and deletes generated files recursively.

#Know bugs

• When downloading test cases from codeforces tp creates testing files for five problems (A to E).

#Limitations

• Only literal comparisons of the output, floating point comparisons and solution checkers are coming soon.

#Requirements TestProgram depends on the following programs:

• Bash
• Diff
• Vim (not optional by the moment)
• timeout
• GCC, Java, Python and/or Ruby depending on what programming language do you use.

#Bug reports and contributions If you find a bug please create an issue, or even better, fix it and create a Pull Request.

All contributions are welcome!

#License This project is under GPLv3, with exception of cf-parser which is an adaptation of Codeforces Parser, which I think is also a free software license: http://codeforces.com/blog/entry/10416#comment-175092.