mfornet/acmx

Competitive programming made simple. VSCode extension.

ACMX

acmX is a tool that empower contestants to solve competitive programming problems easily.

Features

• Contest/Problem parsing. (Via Competitive-Companion extension).
• Running solution against testcases.
• Automatic verdict results (OK, WA, RTE, TLE, CE).
• Manage testcases easily.
• Stress solution against brute solution using a generator (Useful to find corner cases).
• Support for multiple languages.
• Smart copy.

Documentation

Check the wiki for details about all features and help.

Getting started

acmX have been designed to run automatically boilerplate actions repeated often in competitive programming. Next is the expected pipeline to interact with it.

1. Install the extension from the marketplace.

2. Install competitive companion.

3. Open online contest/problem you want to solve and parse with competitive-companion extension. All problems along with the testcases are downloaded and you are ready to code.

4. Work on your solution on sol.cpp.

5. After you finish call Run and automatically your program will be compiled and run against every testcases. If the solutions is correct, it will be reported as Ok otherwise you will see failing test case. You can always go back to original layout calling View: Code.

Join the conversation

We have a group to discuss about this tool in Telegram.

Contributing

The easiest was to contribute is giving a star to the github repository and to the extension in the marketplace if you found the extension useful.

If you found a bug using the extension or have some suggestion open an issue to discuss about it.

If you want to implement some feature but you are not sure where to start check good first issues. You can join dev channel in Telegram too.

acmx's Issues

Close open tabs on View Code

When View Code command is called the following tabs should be closed:

• Tabs about test cases
• Tabs about files that are not related about the current project.

Nice to have: They layout will change to single and the main solution is opened. Instead of opening the main solution open the current active document if it is not a test case.

Edge case: What about tabs that are not saved and should be closed. Options:

1. Save + Close
2. Ask to save + Close
3. Ignore
4. Close (without saving) (this should be avoided)

Test case summary | Improved UI

Print a summary of the test execution.

• Number of tests passed
• Number of tests failed
• Time execution per test.
• Input / Output of failing tests

Allow tests without output. This tests should not be reported as failing (unless it crashes or is time limit).

Allow configuring tests:

• Disable/Enable test (most relevant for debugging)
• Mark test as output not known.
• Disable All / Enable All

New configuration proposal.

Short circuit run mode: Run test cases until the last test case that failed.

It would be very cool to add the interface from https://github.com/agrawal-d/competitive-programming-helper

This will allow to show all testcase in a better way and interact better with testcase.

It also needs:

• A way to open summary from any other view (to see and modify test cases).
• Button to set a test to debug.
• Button to save a generated test.

When run:

1. if something failed:

List with all tests and metadata:

• Test input
• Test expected
• Test output | Veredict
• Time / Memory used

2. Show small summary in a pop-up that said max-time, and everything went ok. Put a button that open full summary.

New command: Save testcase

Allow test cases generated automatically using a generator to be saved in the list of tests.

Test cases generated are overwritten, this command will allow to do a right click on top of a generated test (.in / .out) and convert the test into a fixed test case with sub-extension .hand

Create wiki / Update Readme

Create Wiki with tutorial:

• How to use/setup acmX
• Configure it for several working environments/languages
• Explain each feature (with gifs, or examples preferably)
• Document each settings

Add on README link to wiki page and leave only gif showing acmX without going into details.

Don't report ok if no testcases available

Got control of vscode after calling openFolder

After calling openFolder another instance of vscode is open, so next commands are not called.

Avoid out folder

Right now out folder is in the gitignore, yet it is used at some points incorrectly (I think for checkers, etc...)

This affect developing locally this tool.

Debug information

For every possible command / action / feature of acmx try to create a separate log environment (like a terminal) where we log all steps that are being done. If something fails we should look first to this logs to see at which point it failed.

This will be useful for bugs reports. Maybe create a file under ~/.acmx/acmx.log that can be uploaded. This is relevant for commands that close current environment and opens them again. If this file idea is created be sure the file doesn't exceed a top capacity (like 10MB).

Have a flag to enable/disable logging (by default it should be enabled).

I install plugin but when write code and click acmx:Run code appear error WA in test 0?

please tell if you can fix this error ....

An unknown error occurred (on compilation error)

There are not available logs.

Put time limit on all type of programs

Right now checker / generators / brute have no timelimit.

Idea: Create a wrapper for executables that allow running them, piping input/output, and setting limits on memory and time, as well as being able to measure running time (and ideally memory consumed).

Fix tcgen

• Make a global command for tcgen mfornet/tcgen#1 and use it instead of global python path
• Add documentation
• Add useful message if tcgen is not installed (like Do you want to install tcgen?)

Setting debugging testcases for generic tools (language)

Right now to modify the current testcase to debug we are modifying the task.json using regular expression which is not really good idea.

From this repository https://github.com/agrawal-d/competitive-programming-helper use a configuration to store the path to the testcase that should be debugged.

Use variable substitution in the task to use that test.

Add config.json to old problems

Add config.json by default when it is missing. Autodetect main solution and set it. This will allow using new changes on old problems.

Save on Compile/Run

When running compile or run command, it would be nice if the source file is auto saved

Archive/Unarchive

Allow archive/unarchive contest/problems.
Proposal is to have two sub-directories

solutions/
   archive/
   active/

Recently downloaded problems should be in active and there should be an option to archive them (move from active to archive).
Also there should be an option to unarchive, move from archive to
active.

Important be careful of contest/problems that are downloaded for second time and are already archived. Define what will be the behavior here.

Java configuaration

how to configure it with java 13

Give ability to run the file in terminal.

This would enable the user to use this extension for interactive problems and also to give custom input in normal problems.

Show error if compiling is unsuccessful

This extension does not show error message or warning message that g++ outputs if compilation is unsuccessful. Also cerr output is not shown.

Upgrade is not working (missing template)

ENOENT: no such file or directory, copyfile '/Users/marcelo/.vscode/extensions/marx24.acmx-0.2.7/out/static/template.cpp' -> '/Users/marcelo/Documents/acm/solutions/AtCoder Grand Contest 013/C - Ants on a Circle/brute.cpp'

Show error is compiling is unsuccessful

This feature works in ubuntu but does not work in windows

Submit from acmx

New command to submit solutions directly from acmx.

The idea is using a web browser extension that spawns a server and is listening for new solutions to be submitted. This way, if the extension is already logged in in the website then should be easy to make the submission.

Maybe use the competitive-companion tool or create a brand new extension.

Relevant link: https://developer.chrome.com/apps/sockets_tcpServer

Adaptive checker (to handle float precision error)

Default checker used atm is wcmp which is great to compare tokens and ignore trailing whitespaces and EOL (also similar checkers are commonly used on several platforms, primarily Codeforces).

But another common checker is using decimal numbers and allowing some margin for mistake (1e-7).

Proposoal: Use an hybrid of wcmp where if token comparation fails, try to read both token as doubles and check whether precision is below some threshold. This idea rarely will give some false positive, and will automatically work for test cases that requires a decimal number where some error is allowed.

Use parsed time limit

When used competitive companion we are provided with the time limit for each problem. Use this time limit instead of a global time limit.

To implement it, store the time limit on attic/config.json as a separate entry.
Try to load the time limit from the config, if not found, use general time limit.

One clear advantage of this approach is that we can have a different time limit per problem.

Adopt coding guidelines for the codebase

Example: https://github.com/microsoft/TypeScript/wiki/Coding-guidelines

"command 'acmx.addProblem' not found"

On Visual Studio Code, I installed this extension, but when I click on Competitive Companion nothing happens (the port is set) and when I manually click on "Add Problem" I get command 'acmx.addProblem' not found.

Configuration and metadata per problem

Each problem has particular information associated with it.

Add a config.json with the following data:

• Time Limit
• Memory Limit
• Is interactive
• Hash sum of compiled files (this can live in another config)
• Url (and extra info) from the platform (useful for reverse queries while submitting, etc...)

This config should be editable in a similar UI to the one offered by vscode to edit global configuration.

Run programs on integrated terminal.

Right now compile errors and standard error output is displayed in the terminal calling echo.
While this is a useful feature to have the way it is implemented is far from ideal since:

• it is not cross-platform.
• echoed commands become part of the terminal history.

The new proposal is capturing such errors, and show them in separate editor file instead of using terminal for that.

Support for multiple languages

Improve tool design in order to allow support for multiple languages other than c++. This should allow using multiple languages for:

• Solution
• Brute force
• Generator
• Checker

Target languages:

• c++
• java
• python3
• rust (nice to have)

Make a separate repository with examples of different configurations, or add a page in the wiki. It should contain exact configuration (.json file) and steps about extra actions.

Follow up from: #12

The idea is implement several tasks (vscode tasks), one for language that can be modified and are triggered at different moments.

Task Name: |

• OnCreation | cpp
• BeforeRun | cpp
• Run | cpp

Interactive problem support

• Design pipeline to work with interactive problems. Probably they will require special checker and interactor.
• Allow debugging interactive problems.

It can be used a script similar to the one used in google codejame.

Testing framework

• Currently testing framework is broken. mac and windows system were removed from workflow, and they are not evaluated anymore, but PR still says that 2 more evaluations are pending.

• Address issue with mac and window so the platform is tested on three platforms.

• Add tests for each feature.

Templates for checker / generators / brutes

When upgrade function is called a checker / generator / brute program is created in the problem workspace. Templates should be changed easily.

I think the best way is create all the templates in $HOME/.acmx/templates/. Modifying templates is as easy as modifying files there.

Allow to match files using prefix of the name (this allow to use different extensions for different languages).

Also create a configuration to fully change the path to template for each type of file.

First use of acmX not working

When used for the first time it can't create a new folder with global attic. In that case it should specify the path to store all problems. Use by default some standard location like ~/.acmx/solutions so it always work, even after the first use.

Compile/Run on save

When the main code is saved run the code Run the code and report status.

This should be optional, as some users might save too frequently, or might not want this issue at all.

Possible solution for frequent saving

Only run 1 time every X seconds. If saving was invoked at least once after last run and X seconds have passed run the program again. With this heuristic the program will be executed at most once every X but also will be run after last save.

Cancel operation

Right now all operations that involves running commands on the shell (compiling files, generating test cases, running solutions) other than running main solution have a friendly time limit. However a high time limit might be bad, because if there is a bug (infinite cycle, or really slow program) , extension will stop working until that command finish.

Proposal to fix. On executing a command, pop up an info box with the command that is being executed and context (running checker, running generator, etc...) And add a cancel button that allow cancelling that operation.

This way, user will have feedback about what is happening, and at the same time, the ability to stop it at any time.

Extension .real is misleading and wrong for testcases

Use convention:
.out for user output
.ans for expected output, real answer.

Ignore static folder on language statistic

This repository is marked by github as a c++ first project, however I haven't write c++ for this. Ignore static/ folder using overrides as explained here: https://github.com/github/linguist

Any possibility of adding support to WSL

Allow multiple solution for the same problem

Allow to have several solutions for the same problem.

Right click to set-up a solution as main solution. Right click to set-up a solution as brute solution.

By default program with prefix sol is the main solution. By default program with prefix brute is the brute solution.

New option to disable brute solution (using right click on brute solution).

Persistent generated tests

All input tests created using generated should be stored after each run.

Generator should create a new test deterministically given a seed from the standard input.

The files created will be of the form:

testcases/
  test-$seed.gen.in
  test-$seed.gen.out

Test cases should be erased before creating a new batch of tests.

Missing 2nd Problem while parsing Contests

I have tried with codechef and codeforces.. it doesn't create any folders for 2nd problem in particular. I tried checking if it was issue with companion extension but not - In debug, all problems were sent by the extension.

Automatic status detection

Try to detect automatically the current status of the problem regarding: Accepted | Failed | Not tried

Try to download more information and stats about live contests. This will be probably done through an integration with competitive companion or similar extension.

Add button to stress and upgrade solution

Integration tests

• Mock online judge
• Competitive companion integration test

https://code.visualstudio.com/api/working-with-extensions/continuous-integration

Document all common commands and make a test around them. Let all test run on all platforms (until this is not done, acmX can't claim to be cross platform)

Don't like new "diff" method for comparing test cases

In VScode, the diff editor normally displays diffs like this:

In competitive programming, this is extremely unhelpful, because it is normally the case that there are differences on certain lines, as opposed to there being inserted / deleted lines in the input and output.

I would rather just have the expected output / normal output side by side.

HOME is not an environment variable in windows

Use USERPROFILE instead for windows.

Integration with competitive companion extension

This tool helps me as i prefer VS Code for C++ than any other but it only supports Cofeforces and not anything.

While Competitive Companion can greatly parse many sites contests and problems. The list of sites is mentioned in the above link as well.

If this tool can be integrated with Competitive Companion, It would be great for us all. @mfornet

Save md5 hash after successful compilation

Right now the md5 hash is stored always, even if the compilation fails, and it can be misleading, since some times you run your program using old code.

Remove md5 hash file

Move the md5 hash file to the config file, or at least move them to attic folder so they are not visible.

Feedback on stressing

While stressing a solution give feedback to the user.

• Show changing tests in real time. Change layout to display tests. Change layout back when finish.
• Show test number in the status bar.