kipr/simulator

Simulator of KIPR Demobots and code workspace.

Simulator

A Robotics Simulator built in TypeScript. Simulates a botball/JBC style demobot with a built in IDE.

Development

Requirements

• Git Large File Storage
• Node.js v16 or higher
• yarn
• CMake
• SWIG 4+
• Python 3.7 or newer

Debian/Ubuntu

# to get newer versions of Node.js through apt-get, you likely need to add the correct NodeSource repositories
# for details, see https://github.com/nodesource/distributions
sudo apt-get update
sudo apt-get install -y wget git cmake build-essential python3.8 swig zlib1g-dev doxygen nodejs
sudo npm install --global npm
sudo npm install --global yarn
yarn --version

#(if yarn does not work, reboot or use "sudo yarn" on the rest of the instructions)

macOS

You will need to install Command Line Tools and Homebrew before continuing.

# If you need Command Line Tools
xcode-select --install

brew install node cmake swig

npm install --global yarn

Windows

It is recommended to use WSL2 with Ubuntu on Windows.

Clone repository and submodules

Clone this repository and its submodules:

git clone --recurse-submodules https://github.com/kipr/Simulator

Or, if you've already cloned the repository without --recurse-submodules, you can initialize the submodules separately:

git submodule update --init

Optional: Pull large files

git lfs pull

Build Dependencies

# Python 3.7+ is required for the build process
python3 dependencies/build.py

Tip: if you are experiencing issues with this step, you may try deleting the repository and follow the steps listed above again.

Install JavaScript Dependencies

Navigate to the root directory of this repository, then run:

yarn install

Build Translations

yarn run build-i18n

Running

In one terminal, build in watch mode:

yarn watch
# If you get an error that says "digital envelope routines::unsupported", use:
NODE_OPTIONS=--openssl-legacy-provider yarn watch

In another terminal, run the server:

node express.js

Configuration

The server can be configured using environment variables. Variables without default values must be provided.

Variable	Description	Default value
SERVER_PORT	The port on which to listen for requests	3000
CACHING_STATIC_MAX_AGE	The max duration (in ms) to allow static assets to be cached	3600000 (1 hr)
FEEDBACK_WEBHOOK_URL	The url for the discord webhook to send feedback to
API_URL	The url for the KIPR database server	https://db-prerelease.botballacademy.org

Using a local database service

Clone the KIPR database service and follow the instructions. Start the simulator server with:

API_URL=http://localhost:4000 node express.js

Linting

The project is set up with ESLint for JavaScript/TypeScript linting. You can run ESLint manually by running yarn lint at the root.

To ease development, we highly recommend enabling ESLint within your editor so you can see issues in real time. If you're using Visual Studio Code, you can use the VS Code ESLint extension. For other editors, see available ESLint integrations.

Internationalization (i18n)

Simulator leverages gettext PO files to create a i18n.json file located in /i18n/i18n.json. The source files are scanned for imports of @i18n, and uses of the default exported function (tr by convention) are detected and inserted into PO files located in /i18n/po/. These PO files are "built" into the JSON file, which is then injected into the frontend via webpack's DefinePlugin (see configs/webpack/common.js for details). The tr function reads this object at runtime to make translations available.

To update the PO files, run yarn run generate-i18n. While the generation script should preserve your work-in-progress, it is recommended to commit, stash, or otherwise backup the PO files prior to running this script.

To build the PO files into a JSON object suitable for consumption by the frontend, run yarn run build-i18n.

There are many editors available for PO files. We recommend Poedit.

The format of the i18n.json is as follows:

{
  "context1": {
    "See `src/util/LocalizedString.ts` for a list of available language identifiers": {
      "en-US": "See `src/util/LocalizedString.ts` for a list of available language identifiers",
      "ja-JP": "利用可能な言語識別子のリストについては、「src/util/LocalizedString.ts」を参照してください"
    }
  },
  "..."
}

Building image

The repo includes a Dockerfile for building a Docker image of the simulator:

# If you don't have jq, you can just use export VERSION=latest
export VERSION=$(jq -r .version simulator/package.json)
docker build -t kipr/simulator:$VERSION .
docker run -ti -p 3000:3000 kipr/simulator:$VERSION

simulator's Issues

Add more friction to wheels or mat for turning

Robot spins out of control when just turning one wheel. Probably need to adjust friction between wheels and mat to prevent this.
Code for testing below:

#include <stdio.h>
#include <kipr/wombat.h>

int main()
{
  printf("Hello, World!\n");
  motor(0, 100);
  msleep(2500);
  return 0;
}

Clicking "compile" also runs the user program

Clicking "compile" runs the user program if it successfully compiles. It is hard to tell because it runs it without hooking up any of the registers (i.e. motors won't move), but it is running in the background. At best, it's running without needing to. At worst, there could be unintended side effects.

Adding objects anywhere

Adding a dialog box to add predefined objects anywhere on the game board could be useful to those wanting to create their own challenges.

Add UI for toggling sensor visualization

There should be a checkbox in the UI to toggle sensor visualization. This could be a single checkbox for all sensors, or a checkbox per sensor.

Add UI for displaying sensor values

Similar to the motor and servo interface, there should be UI elements to display sensor values in real time.

Sensor values do not update multiple times throughout program. Test with code below.

#include <stdio.h>
#include <kipr/wombat.h>

int main()
{
printf("Hello, Sensors! %d\n", analog(0));
motor(0,50);
motor(3,50);
msleep(3000);
ao();
printf("Goodbye, Sensors! %d\n", analog(0));
return 0;
}

Cans aren't positioned properly on the board

The cans don't line up with the circles on the board anymore. All of them seem to be offset on the z axis.

Warnings about deprecated usage of `morgan`

Running the server gives some warnings about deprecated usage of morgan:

morgan deprecated undefined format: specify a format express.js:21:9
morgan deprecated default format: use combined format express.js:21:9

Reduce webpack bundle size

A production build of the bundle is currently 5.6 MB. It's not the end of the world, but especially if we expect users with slower internet connections, it might be worth using a bundle analyzer to see why it's so large.

Fix and test motor functions

• clear_motor_position counter
• ao
• move_to_position

"clear motor position" has a short delay before taking effect

(originated from PR #84)

Calling cmpc() and then immediately calling gmpc() usually returns the old motor position. After a short time, it will return 0. This happens because the simulator is rapidly sending motor position updates to the worker, so even after the worker has sent 0 to the registers, the simulator will keep updating from the previous motor position until the registers sync up.

Program to repro:

#include <stdio.h>
#include <kipr/wombat.h>

int main()
{
  // Move motor 0 forward a bit, to a non-zero position
  motor(0, 50);
  msleep(3000L);
  off(0);
  msleep(1000L);
  
  // Clear the motor position and immediately print the position
  cmpc(0);
  for(int i = 0; i < 100; ++i)
  {
    printf("%d\n", gmpc(0));
    msleep(10L);
  }
  return 0;
}

Expected output:

0
0
0
...

Actual output

2265
2265
2265
0
0
0
...

Workaround

A short msleep() after cmpc() will allow the registers to sync up.

Robot moves very slightly when motors are off

When the motors are off, the left wheel turns forward very slowly, causing the robot to turn right over time. You can see this if you leave the simulator open for a couple minutes without running a program.

Registers are no longer reset after user program ends

Register values used to be reset after a user program was done running. We had to remove that logic due to _main() now being asynchronous. We need a new way of detecting the end of user programs.

Production build fails and is slow

Building for production using yarn build gives an error:

ERROR in ./ammo.js
Module not found: Error: Can't resolve 'fs' in '/home/nazam/Simulator/src'
 @ ./ammo.js 2:1426-1439
 @ ./Sim.tsx
 @ ./components/SimulatorArea.tsx
 @ ./components/Root.tsx
 @ ./index.tsx

The build is also really slow (1 hr 20 mins on my machine).

Implement touch sensors

The demobot has at least 3 touch sensors (1 on the front, 2 on the back) that need to function in the simulator, including reading their state from the user program via libwallaby.

Use actual wheel rotation for motor position

Currently the motor position is purely based on the motor speed and elapsed time. It'd be better to calculate the motor position based on the actual rotation of the wheel, since things like collisions and friction could prevent the wheel from turning as fast as it's trying to turn.

Allow running server on other ports via env vars

The port are 3000, but if change to
const portNumber = process.env.PORT || 3000;
I think it will be better

Second print statement not returning

#include <stdio.h>
#include <kipr/wombat.h>

int main() {
printf("Hello, World! %lf\n", seconds());
motor(0,50);
motor(3,50);
msleep(3000);
ao();
printf("stopping");
msleep(2000);
return 0;
}

Ran this code and only the first print statement prints out. I tried it without the ao() as well and it still did not print.

Convert backend API to TypeScript

Most of the codebase is TypeScript, but the few files related to the backend API (like express.js) are still plain JavaScript. Switching these to TS will make the whole codebase more consistent, and it may be important as the backend grows.

mtp does not stop at position

move_to_position does not stop at specified position and continues when msleep is longer than period needed to get to position

Robot Movement Inaccurate

Robot does not go the same distance when going backwards and forwards. Test at long distances to ensure accuracy.

requestAnimationFrame should use high res timestamp

Currently, WorkerInstance.tick() calculates the elapsed time using Date.now(), which is much less precise than the timestamps provided by the Performance API. A high res timestamp is passed into the tick() callback anyway, so we should use that timestamp instead of Date.now(). This will make any time-dependent calculations (like motor position) more accurate.

Show indication that code is compiling

When a user clicks compile, it'd be nice to show some kind of indication that the code is compiling. Some ideas:

• Show text like "Compiling..."
• Disable the "compile" and "run" buttons
• Disable the code editor area
• A loading spinner

Consider compiling user programs using -O3

The Emscripten docs highly recommend compiling with -O3 optimization when using Asyncify (see https://emscripten.org/docs/porting/asyncify.html).

We now use Asyncify, so it's worth looking into whether compilation would be too slow with -O3. I ran into some issues when I tried compiling with -O3 so there may be additional changes required to get it working.

UI should indicate how the motors, servos, and sensors are mapped

The simulator UI doesn't tell you which ports are mapped to which motors, servos, and sensors. A quick fix could be to include this info as comments in the starting code. But it'd be better to indicate it in the UI somewhere, maybe in a help section/dialog.

Motor speeds not capped at 100%

Motor speeds aren't capped at 100 which also causes unintentional sliding when they move too fast. Test at speeds of 500 or 1000 to replicate.

NaN appears when current number is deleted in position input boxes

In the position input boxes, if you delete the current number, it changes to NaN and is tricky to get it back to a number. I'm guessing this was an issue before.

Originally posted by @navzam in #82 (comment)

Warnings about invalid license expression in `package.json`

Running any yarn commands gives a warning about the license expression in package.json:

warning package.json: License should be a valid SPDX license expression

"Compile succeeded" message isn't always displayed

You can repro this by compiling a program that doesn't have any printf statements in it.

The flow for showing "compile succeeded" is very strange. It relies on running the program after compilation and displaying any standard output as "compile succeeded":

// SimulatorSidebar.tsx

private onStdCompOutput_ = () => {
  this.setState({
    console: `Compile Succeeded\n`
  });
};

...

const compiled = await compile(this.state.code);
WorkerInstance.onStdOutput = this.onStdCompOutput_;

which won't work for programs without standard output. And we shouldn't run user programs on compile anyway (#52).

Implement camera

In addition to analog/digital sensors, it would be nice if the simulator supported camera functions. The demobot doesn't seem to have a camera has part of the model, so this might require a model update for it to really make sense.

Push model for sensor updates can clog the event queue

Currently sensors send frequent register updates via postMessage() to the worker. When the user program needs a sensor value (for example, calls analog()), it yields to the worker, which processes all updates that have been queued. But if the user program rarely reads sensor values, this can needlessly clog the event queue with 1000s of messages over the running time of a typical Botball program.

We should consider changing to a pull model (worker requests register values when needed), as long as it wouldn't hurt performance too much.

Add a linter

To improve code consistency and reduce bugs, we should add a linter (probably ESLint) and fix any existing issues that the linter reports.

A GitHub Action that runs on every PR would be great too.

Physics colliders and bot meshes are sometimes misaligned

When the simulator initially loads, the physics colliders sometimes are not aligned with the loaded demobot meshes. The bot still behaves properly since the colliders are correct, but the visible meshes look wrong.

stderr output isn't displayed

Output to stderr doesn't get displayed in the simulator's output box.

Looks like there's some code that's supposed to do this, like SimulatorSidebar.onStdError_() and WorkerInstance.onStdError(), but nothing from stderr shows up.

Implement IR sensors

The demobot has at least 1 IR sensor (on the front pointing down) that needs to function in the simulator, including reading its state from the user program via libwallaby.

Issues loading on iPad

Some iPad users have issues with the simulator not loading/displaying. @tcorbly brought this up and may have more details.

Update scripts / README to use libwallaby master

Now that libwallaby's master branch has emscripten support, we should update the simulator install scripts/instructions to use master instead of the emscripten-specific branches.

Build project in CI workflow

The current CI workflow runs the linter, but it doesn't actually build the project. Now that #64 is resolved, the production build should be fast enough to not slow the workflow significantly.

No way to stop a running program

There's no way to stop a program while it's running. This is especially important for programs with long-running (potentially infinite) loops. Currently you have to refresh the simulator.

We could send a "stop" signal to the worker, but this would rely on the worker being available to listen for messages, which isn't always the case. We may have to terminate() the worker instead.

Implement arm/claw servos

The arm and claw should have the appropriate joints required for them to turn. They should also be hooked up such that they react to servo commands in the user program.

Some of this work may already be done. Currently if you run the simulator, you'll see the claw physics imposter spinning independently of the claw model.

Automatically position/align meshes and physics imposters

Currently the robot's meshes are manually created, positioned, and rotates to align properly. For example:

private buildWheels () {
    const wheelMaterial = new Babylon.StandardMaterial("wheel_mat", this.scene);
    this.wheel1 = Babylon.MeshBuilder.CreateCylinder("wheel1",{height:0.7, diameter:6.8}, this.scene);
    this.wheel1.material = wheelMaterial;
    this.wheel1.position.x = 7.9;
    this.wheel1.position.y = 4.2;
    this.wheel1.position.z = -7.2;
    this.wheel1.rotation.z = Math.PI/2;
    this.wheel1.isVisible = this.collidersVisible;

    this.wheel2 = Babylon.MeshBuilder.CreateCylinder("wheel1",{height:0.7, diameter:6.8}, this.scene);
    this.wheel2.material = wheelMaterial;
    this.wheel2.position.x = -7.9;
    this.wheel2.position.y = 4.2;
    this.wheel2.position.z = -7.2;
    this.wheel2.rotation.z = -Math.PI/2;
    this.wheel2.isVisible = this.collidersVisible;
}

Ideally we can do this alignment more automatically by using the individual parts of the demobot model.

Zooming in/out in the canvas causes the whole page to scroll up/down

Scrolling in the 3D area (to make the camera zoom in and out) also causes the whole page to scroll. Scroll events in the 3D area shouldn't affect the rest of the page.

Reset button doesn't reset robot's position

Clicking "reset" doesn't reset the position of the robot. This was likely left over from the 2D simulator and needs to be rewritten for the 3D version.

Robot Movement Inaccurate

Robot movement is not the same going backwards and forwards. Needs to be tested at longer distances to ensure accuracy.

Switch to react-codemirror2

Currently we use react-codemirror, which isn't being maintained anymore (last commit and release in May 2017, see issue #126). Might be better to switch to react-codemirror2 so we continue getting updates.

I ran into this because we're getting warnings about using deprecated React methods, which are being used by react-codemirror since it's on an older version of React.

Ability to load different mats / game environments

The code to load the physical environment should be genericized so that it's easy to support different mats / game boards. As a start, we should let the user choose between "mat A" (the currently loaded mat) and "mat B."

On latest emscripten, robot does not auto stop motors when the program ends

Robot does not stop motors when not locally run. The program ending or an ao() does not appear to stop it either. This issue only appears when connecting to the server on AWS, not on locally run instances.

Update to stable version of Babylon.js

We're currently on Babylon.js version 4.2.0-alpha.20. The stable version of 4.2.0 has since been released, so we should switch to it.

Update to newer version of TypeScript

The project is using TypeScript 3.8.3. Updating to 4.2 or higher would enable additional TS features and improve build performance.

Support Python user programs

The simulator should support Python programs. Currently it only supports C.

This will take some experimentation to find a good approach. Some ideas that have been tossed around:

• Use Transcrypt to compile Python to JS
• Compile CPython with emscripten to bring the interpreter to the browser
• Write a custom transpiler that's sufficient for Botball needs