API for running RARS from the command line

[thethirdone]

API for running RARS from the command line

[TheThirdOne]

Making a minimal api per your suggestion would be quite easy (I think).

However, I would like to go a little bit beyond the minimum. But I haven't come up with a good idea to have a nice, flexible api that doesn't require understanding the internals of RARS.

Here are some requirements that I would like to hit:

• Configurable like the CLI
  • setting things like the maximum step limit should be easy
• Be able to have multiple files assembled together
• RISC-V programs can take input from argument list and STDIN
  • Both provided as a string to the api
• RISC-V programs can output to STDOUT
  • Which is captured and returned as a string and not outputted to the screen
• Error information is preserved
  • It should be clear if an error is an assembly issue or runtime issue
• If a program terminates early (like by a breakpoint), it should be possible to continue it
• Initial memory and register state should be possible to preset
• Final memory and register state should be possible to inspect

I think all of the above requirements can be met technically without much work, but I don't know how to make that into a clear interface.

Its possible the STDIN and STDOUT redirecting may take a little bit of extra work, but I think thats probably worth it.

Additionally, while thinking about this I had some requirements that would be nice to have (and would help the project as a whole), but are not easy technically.

• Have state entirely contained in objects
  • This requirements getting rid of all static objects in the code
  • The idea of this is that it makes the interface like a pure function
  • This would allow multiple programs to be run concurrently
  • Having this done would also open up multithreading as a concept for RARS as a whole
• Make some of the internal classes api ready
  • The RISCVProgram class would be a useful datatype, but it isn't well documented or fit to be part of an api
  • Cleaning up/documenting some of the internal classes and exposing them as a stable api would probably help anyone trying to get into working on the internals

[martinberger]

Everything you propose sounds eminently doable. Without knowing much about RARS' internals, I can imagine that exposing such an API would also be a good chance to refactor and clean up the code.

Naturally, refactoring can introduce new errors, so I recommend thinking about how the old code can be used as a testing oracle for the refactored code.

[TheThirdOne]

I have come up with a design I am pretty happy with.

The bulk of the interface would be in a Program class as follows:

public class Program {
    public Program(Options set);
    public ErrorList assemble(ArrayList<String> files, String main) throws AssemblyException;
    public void setup(ArrayList<String> args, String STDIN);
    public Simulator.Reason simulate() throws SimulationException;
    public String getSTDOUT();
    public RegisterBlock getRegisters();
    // Some memory access not quite sure yet
}

ErrorList, AssemblyException, SimulationException, Simulator.Reason and RegisterBlock are all internal classes which I think are clear and clean enough to be used externally. Though I could probably clean them up and give them better documentation.

An example usage might be:

Options op = new Options();
op.startAtMain = true;
op.maxsteps = 1000; // prevent student code from running indefinitely 
ArrayList<String> files = new ArrayList<>();
files.add("placeholderstudentcode.s")
files.add("somelIbrarycode.s")
for(String studentFile: studentFiles){
    Program p = new Program(op);
    files.set(0,studentFile);
    ErrorList warnings, errors;
    try{
        warnings = p.assemble(files,studentFile);
    }catch(AssemblyException ae){
         errors = ae.errors();
    }
    if (errors != null){
           // student failed to assemble
        continue;
    }
    for(TestCase t: tests){
        p.setup(t.args,t.stdin);
        // Potentially set memory or registers here
        try {
            Simulator.Reason r = p.simulate();
            if(r != Simulator.Reason.NORMAL_TERMINATION) {
                // test case
                continue;
            }
        } catch (SimulationException se){
               // test case
               continue;
        }
        // check any final register or memory state
        if(!t.stdout.equals(p.getSTDOUT())){
           // test fails
        }
        // test passed
    }
}

This api requires modifying the SystemIO to allow STDIN and STDOUT to be set and a small modification to Memory allowing it to be reset to a given state easily. Additionally, a small modification to how assembling works would allow a raw string to serve as source code.

I can also imagine that there would be some streamlined methods on top of this that makes it easy to make a simple pass/fail per testcase. But I think this interface makes it pretty easy to drive the simulator however you want and wraps up the internals nicely.

[martinberger]

Looks good.

Is it possible to get register content after termination?

[martinberger]

PS Would it be difficult to return the number of steps taken upon termination? What about other profiling data, like memory used, maximal height of stack (I known that RISC-V doesn't have an explicit stack pointer, so this really amounts to returning the range of values (min, max) a register attains during execution, or at least register X2)? It's not vital at all for my use case, but if such facilities existed, I could give students more interesting feedback.

(OTOH, there is always the danger of over-engineering ...)

[TheThirdOne]

Is it possible to get register content after termination?

With the proposed design thats pretty easy. It would be something like:

p.getRegisters().getValue("t0") 

Instead of the

// check any final register or memory state

Would it be difficult to return the number of steps taken upon termination?

Yes, it would be more difficult. However, I looked into it a bit while working on the api thus far and I think I have an idea for adding a way for a bunch of information to come out of the simulator.

For the time being, I don't want to commit to adding that feature which is about the same difficulty as finishing the rest of the planned interface. I believe it would be easy to add in to the interface in the future.

[martinberger]

Fair enough. Keeping things minimal is a good software engineering technique.

As to getValue("t0") is there any particular reason why register access is via string lookup, rather than something that is type-checked (e.g. getters, public members for each register, enums etc)?

[TheThirdOne]

One of the reasons is that string lookup already has to exist for the assembler.

With that in mind, adding enums to represent each would make it more typesafe, but doesn't provide much safety as there isn't any way in a standard compiler to ensure all members are used once in a switch or similar construct. So when adding access by enum the possibility there is the possibility a bug is introduced.

Access by explicit method has a similar possibility of messing up when writing as you would need to assign each register to both an array for string searching and whatever mechanism you are using to make the methods work. Additionally, that would result in a ton of boiler plate code.

The current system allows for easy code reuse (RegisterBlock is used to back the normal, floating point and control and status registers). And the only place the access by string is used is in system calls. If you are testing the system calls, there is no possibility of something happening that the type system could detect.

TLDR; mechanisms to make it more type checked cost lines of code with the only benefit being that issues can be caught at compile time rather than runtime (in which case with any testing, they will be caught).

[TheThirdOne]

522ebbd introduced the cycle and instret CSRs which provide an easy way to get the "number of steps taken upon termination" as requested.

I forgot to mention it then, but I was reminded of the request when working on 0703e79

[amoe]

For anyone on this thread who's looking for a currently-working example usage, here's a fully-worked API usage example that compiles against RARS 345c17b.

import rars.AssemblyException;
import rars.ErrorList;
import rars.SimulationException;
import rars.api.Options;
import rars.api.Program;
import rars.simulator.Simulator;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class RarsUsageDemo {
    static class TestCase {
        public ArrayList<String> args;
        public String stdin;
        public String stdout;

        public TestCase(ArrayList<String> args, String stdin, String stdout) { this.args = args; this.stdin = stdin; this.stdout = stdout; }
    }

    public static void main(String[] args) {
        Options op = new Options();
        op.startAtMain = true;
        op.maxSteps = 1000; // prevent student code from running indefinitely
        ArrayList<String> files = new ArrayList<>();

        ArrayList<String> args2 = new ArrayList<>();
        String stdin = "";
        String stdout = "";

        TestCase t1 = new TestCase(args2, stdin, stdout);
        List<TestCase> tests = Arrays.asList(t1);

        files.add("foo.asm");
        System.out.println("Working Directory = " + System.getProperty("user.dir"));

        for(String studentFile: files){
            Program p = new Program(op);
            files.set(0,studentFile);
            ErrorList warnings;
            ErrorList errors = null;

            try{
                warnings = p.assemble(files,studentFile);
            }catch(AssemblyException ae){
                ae.printStackTrace();
                errors = ae.errors();
            }
            if (errors != null) {
                // student failed to assemble
                continue;
            }
            for(TestCase t: tests){
                p.setup(t.args,t.stdin);
                // Potentially set memory or registers here
                try {
                    Simulator.Reason r = p.simulate();
                    System.out.println("Termination reason: " + r);
                } catch (SimulationException se){
                    se.printStackTrace();
                    // test case
                    continue;
                }
                // check any final register or memory state
                int registerValue = p.getRegisterValue("a0");
                System.out.println("register was " + registerValue);

                if(!t.stdout.equals(p.getSTDOUT())){
                    // test fails
                }
                // test passed
            }
        }
    }
}

Usage looks as such with input file foo.asm with contents li a0 10:

$ javac -cp rars.jar RarsUsageDemo.java                                                                                                                                                     $ java -cp rars.jar RarsUsageDemo                                                                                                                                                           Working Directory = /home/amoe/vcs/rars
Termination reason: CLIFF_TERMINATION
register was 10

[TheThirdOne]

@amoe, thanks for adding an example. Having a readable example should definitely be helpful to other people trying to use the API.

It should also be noted though that RARS uses the API internally for testing and cli functionality. Both of those are a little complex, but demonstrate how most of the features of it can be used.

Because there is very little documentation on how to effectively use the API, I am also happy to answer questions.