systemetric/shepherd-2

A rewrite of shepherd

shepherd-2's Issues

USB sticks

USB sticks behave as they do in shepherd-1

DONE IS:

1. All of here is implemented: https://robocon.uk/docs/running-code-from-a-usb.html
2. Zone detection from USB sticks works
3. Tested on a brain

NOTES
See here for shepherd-1 implementation: https://github.com/systemetric/shepherd/blob/28473503130cddd2c40702240f3deaad3a21e52b/shepherd/__init__.py#L56

Add favicon.ico

Add a favicon.ico

Could use old shepered logo or use the RC logo.

4 states

Currently shepherd has the states:

• READY
• RUNNING
• STOPPED

Really there is a period of time between when the usercode is started and when it is ready to run, this is the "INIT" state. For now READY has been renamed to INIT but a new READY state needs to be re-introduced. In this state code has reached the break point in robot.Robot() but has not yet been sent the start signal.

Hopefully this does not block #3

Setup CI

AS A shepherd developer
I WANT to not be allowed to commit broken code to master
SO THAT the user expierence is bug free
DONE IS:

• CI is setup running all of the tests in /test
• It is not possible to commit to master with failing tests
• Linting etc should also be run, nothing too aggressive

NOTES:

• @shardros's personal vote for Github Actions + PyTest + Pipenv

Serve sheep

DONE IS:

1. Sheep is served from an end point
2. There is an endpoint where sheep is abel to save files to seperate to \upload
3. Blockly works

NOTES:
Ideally use the current endpoints in shepherd-1

Run parity with `shepherd-1`

AS A shepherd dev
I WANT the run to be sane and bug free
SO THAT I can be confident about altering it that it is bug free
DONE IS:

1. Sheep can upload and run code
2. Shepherd can kill user code when the time is up or the stop button is pressed in sheep
3. The Robot.Reset is called (need a mock made for testing so that shepherd can run on x86.
4. Tests are written

Notes:
The shepherd-2 run is heavily refactored from shepherd-1. We need to make sure that we support any API end points which are used by sheep.

Add blockly tests

We continously forget that blockly is a thing. We should have tests to make sure that the code which ship works and is upto date.

DONE IS:

• Blockly tests make sure:
  • The blocks match the shipped version of the robotlib
  • The blocks work (as much as it is possible to test this
  • The blockly info delivered in files/ is correct.

Serve Docs

DONE IS:

1. The robocon docs are serverd from sheep.
2. There is a mechanism for upading the docs, probally the same as before, just document it in the README.md

NOTES:
Might need to worry about FASTAPI's own docs at /docs might have moved that
Should be a copy paste job from shepherd-1 otherwise

Commits to master should have the latest version of sheep/docs

AS A robocon release manager
I WANT to know that the compiled version of the docs and sheep is the correct ones
SO THAT I don't get compatability bugs with them falling out of sync
DONE IS:

1. CI automatically builds docs and sheep only master
2. CI either:
   a. Commits the compiled version to master if someone commits an out of sync version (preferable for docs)
   b. Blocks a commit to master if the freshly built version does not match the one in the repo (preferable for sheep)

NOTES:

• Docs should just be whatever the latest on master in the robocon-website repo is

Landing page

Currently the landing page for shepherd-2 is JSON a webpage.

I am currenlty wondering if we should serve sheep as the default page with a help/getting started button.

Either way we should have some kind of website thing, the current homepage will suffice if the implementer believes so.

Run linter on usercode

DONE IS:

• shepherd runs a linter on the usercode and passes a message to the user if their code does not pass the test.

Make robot interface thin

Weak coupling between usercode and the robot code is desirable as it reduces the security risk on thre brain and makes the understanding of the state of the brain more sensible. The Robot doesn't blow up when the usercode exits.

AS A person running robocon
I WANT a thin interface for the usercode to the robot libarry and the actual code avaiable over a network socket
SO THAT
DONE IS

1. Shepherd exposes the public methods of the robot object over a network socket
2. There is no tight coupling between shepherd and the robot object
3. Performance of R.see() is evaluated to ensure that it returning large matracies does not take significantly more time
4. If R.see() performance is anything more than 250ms worse an investigation into how to reduce the return payload is implemented.

NOTES:

• This is something which could get very complicated and it would be good to keep this as simple as possible.
• https://rpyc.readthedocs.io/en/latest/ this is an easy ready made solution
• The R.see() investigation must test:
  • all resolutions
  • images with and without april tags

Alternatives prebuilt RPC is:

• https://docs.python.org/3/library/xmlrpc.html
• https://zeromq.org/

@shardros 's Personal opion is taht RPPy should be used.

Upload parity with `shepherd-1`

AS A robocon developer
I WANT to not break sheep
SO THAT we only change one thing at a time
DONE IS:

• Shepherd-2 has a /upload which matches shepherd-1

Note: If there is a more elegant way to implement this, do that and we will try and depricate the old way

PYLS

AS A Robocon Competitor
I WANT to have accurate intelisense
SO THAT I don't have to read docs
DONE IS:

1. A Python Language Server end point is running on both Arm and x86 shepherd
2. Autocomplete for the robot lib works
3. The version of monaco in sheep is running the newest possible (currently the version of the pyls prevents us updating this and monaco itself, and we want to be running the latest version of pyls to get the best possible completions.
4. Tests are written, the current version fails for unknown reasons, I want the new version to be much more resilient, I think the current failure might be around reconnecting to the socket but I am not really sure.

Serve http image and logs

The image where there is a rest preview which needs to be re-fetched everytime is not great but I think it should still be supported just incase an arena rewrite is not done. It would also be a good fail safe and useful for debugging.

DONE IS:

• static/image.jpg is the last image that the camera saw
• On startup is a custom team image see the code in config.py for this.
• Logs are served and appear in sheep
• Tests written.

Explore async

Shepherd is usally used by a single user at a time yet there are probally still benifits to be gained by sprinkling async everywhere.

Not sure if this requires us to do anything fancy with FastAPI.

DONE IS:

• Evaluate weather its worth it to do this
• Maybe do it

Simplify `start_settings`

Currently we pass in

             {   "mode": "comp",
                "zone": int(config.zone),
                "arena": "A",
            }

To the robot lib do we need all of this?

Remove `_fix_bad_spools`

Due to a python bug not all of the interfaces are compatible.

The fastapi.UploadFile type inherits from SpooledTemporaryFile a builtin
which does not specify the abstract for IOBase and does not support the
seekable method which is required for writing to the file sys. There is
some work to fix this:

https://bugs.python.org/issue26175

A fix for this has been merged in into the 3.11 rc so hopefully this will be
fixed soon:

python/cpython@78e70be

For now we reach in to the object to try and patch it ourselves.
This will not work for files larger than UploadFile._max_size as the
attributes change:

https://stackoverflow.com/a/47169185/5006710

We make sure that the object has a large enough size in
increase_max_file_size

When 3.11 rolls out (and we are able to use it) _fix_bad_spools and
increase_max_file_size can be deleted.

Publish/Subscribe image and logs

AS A consumer of logs
I WANT to be notified when a new event happens
SO THAT I don't have to poll
DONE IS

1. Logs/images are avaliable over a publish/subscribe interface
2. The old REST end points should still exist to support legacy code (the arena)
3. Sheep is updated to use this new socket based interface

NOTES:
This interface should also allow for a seperate
MQTT is one such publish subscribe interface which is already used in the arena

Start button

When robot.Robot() is caleld the user code waits for the start signal to be sent. We need to send the start signal in the transition to running state function.

DONE IS:

1. An interupt based method allows for the start button to move the robot into a RUNNING state
2. Shepherd-2 still runs on x86 just without this functionality enabled
3. When moving into RUNNING state the block at robot.Robot is unblocked.

Notes:
Shepherd-1 implementation: https://github.com/systemetric/shepherd/blob/28473503130cddd2c40702240f3deaad3a21e52b/shepherd/__init__.py#L101

Remember to add pull ups

Autopep8 formatting

DONE IS:

1. Either remove the bit about autopep8 in the readme
   OR
2. have it done in CI