The roboscapesimulator from gsteinltu

Encrypt/reset button lockouts

Some scenarios represent activities that currently rely a bit on the honor system for students to not cheat by resetting or encrypting the robots mid-course. For cybersecurity lessons especially this is not desirable as they encrypt button would be too powerful and is meant to represent having the robot physically.

Would require a client-side addition to display the buttons as disabled, although it could potentially be implemented as the server "claiming" the robots itself. #116 is likely a prereq for this to create a state where it makes more sense to disable reset, but activities targeting cybersecurity and using the race timer could potentially just disable encrypt while the timer is running.

Sub-rooms/sub-environments

This will have #5 as a prereq.

We will need to test if this is more efficient than having multiple separate rooms. The idea is that the room's environment would be cloned and translated, so students can have separate "sub rooms" for their own robots, which can be created/merged/destroyed by the instructor.

A cool visualization (for the instructor/classroom view) would show all of the sub-environments in a grid so what all the students are working on is visible at the same time. A camera able to zoom/pan between them smoothly would be nice on top of that.

Debugging help features for students

May need some brainstorming of exactly what would be helpful, but being able to get a log of commands or speeds/positions could be useful.

Continuous Deployment

It would be nice to have a GitHub action set up to make a container image so the main server can update automatically, and other instances (for #21) can be spun up quicker.

Obviously, requires #59 as a CI step eventually to be more useful.

Scaling, server balancing

A long term goal will see the simulator running on multiple server instances to both distribute load and allow for scaling. The main instance will need to coordinate with other servers to set up a web socket connection on the client to where the room is hosted, and, if necessary/possible, start additional servers to meet demand.

It may be necessary to set up a "budget" for each server to ensure quality of service and reduce costs. Then room creation could allow for more limited rooms (lower robot count, lower tick-rate, etc) if necessary.

"Competition" timers

Timer that instead of starting on room start, starts when robot starts moving or gets first command.

Eventually it would be nice to have a countdown that locks out robots, but this is sufficient for now.

Additional client interactions defined in environments

So far, environments have only used the reset button for everything, but it would be nice to have options defined by each environment's needs. For example, a "start" button for a race environment that toggles to a "stop" button, with some features changed during the "race" period (e.g. locking out reset/encrypt) so an "official" run can have separate rules from non-official runs. This will also need work on the client-side.

Suspension issue

After the physics model was scaled more accurately, the "suspension" became looser:

IoTScape service stops working sometimes

A student had PositionSensor fail to work, but the Waypoint service was still working.

Triggers in environments

Environments would be more useful if they could keep score for the games played inside them. For example, a robot soccer environment where trigger areas on each goal add a point to their respective teams and move the ball back to the middle.

Possible actions on trigger brainstorming:

increase/decrease global counter
increase/decrease per robot counter (may require robot activating the trigger)
reset triggering object
remove triggering object
reset environment
move triggering object outside of trigger bounds (e.g. for walls that only apply to the robot, not to pushable objects)

There will need to be some more general filters available for robots/pushables as a class.

This may require a client-side UI to display some environment-specific information and may eventually evolve into a full scripting interface for environments, which comes with its own set of challenges.

Modify events to be compatible with Unity plugin

Current events are not compatible with how the Unity plugin sends messages. This will need to be changed.

Move to geckos for UDP

UDP makes more sense for this application, switching to Geckos.io would provide this.

Removing Entities

Need the ability to remove entities.

Carlos' requested environment

His request:

50mx50mx50m area with ground and walls
Box and "robot" inside it
Box and "robot" can be moved through an IoTScape service that allows for get/set position commands
No physics outside of collisions with the boundaries of the space

This sounds simple but implementing it properly will need a few changes to how rooms work to make them more flexible. It would probably be a good refactor to give more flexibility around physics engines in the future if Bepu needs replacing or gets a major breaking update.

Camera controls not working in Chrome/Brave

Room administrative features

Room creators should be able to change settings for their rooms, reset the environment, kick users/robots, etc.

The solution used for #4 will likely change details of how this is implemented, as an account system would be the more straightforward way to decide who the owner of a room is.

Event-based sensors

Current sensors are all polling-based, but to enable some educational possibilities (mostly to make them vulnerable to sensor spoofing), it would be good to have an option to subscribe to updates from a sensor, and to have sensors be able to have one of the modes disabled based on scenario.

Box pushing "race" environment

New boxes appear as pushed off, with timer and count.

User assigned robot colors/IDs

For IDs this could be a bit trickier to avoid collisions, especially with physical robots (querying the NetsBlox server may be the solution there) and may be unnecessary with the "robots in room" block, but users should be able to at least choose the robot "model" among compatible meshes, possibly with a fully customizable color that would be consistent between rooms. Color "collisions" could also be a problem, but it's not a big issue.

More sensor types

Current ideas:

Disable external networking while running tests

It's minor, but tests cause robots to show up on netsblox.org by default and it would be nicer if they didn't.

Long-term server connection issue

After running for some amount of time (a week?), the server becomes semi-unresponsive regardless of usage level.

Clients appear to connect (the server logs them connecting/disconnecting), but will not receive any responses from the server.

Expand testing

Current tests do not provide much coverage. Need to look into the best mocking options for socket.io in order to create a larger test suite. Client-side tests may also be feasible to add.

Move to .NET 7

Now that it is available, we should take advantage of its improvements. Native AOT could make for much better improvement if added to the build process. System.Text.Json has some improvements which could be useful.

Recreate traffic light environment

Four-way intersection, traffic light at center with each direction controlled by an IoTScape service. Autonomous cars spawn and attempt to cross. A traffic sensor accessible through IoTScape detects vehicles waiting on each direction. When the light in a direction is green/yellow, cars will cross in that direction. When it is red, they will stop.

Could be a use for null physics engine if adding triggers to it is not too difficult (depending on implementation it could be very difficult or not at all) and handling the car-car collisions isn't too hard.

Also requires the ability to update visualinfo (#112) after start.

Bepu crash issues

Previously had:

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. malloc(): corrupted top size

Today had:

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 15401 times:
--------------------------------
   at BepuPhysics.Trees.Tree.SplitSubtreesIntoChildrenBinned(BepuPhysics.Trees.BinnedResources ByRef, Int32, Int32, Int32, Int32 ByRef, Single ByRef)
   at BepuPhysics.Trees.Tree.CreateStagingNodeBinned(BepuPhysics.Trees.BinnedResources ByRef, Int32, Int32, Int32 ByRef, Single ByRef)
--------------------------------
   at BepuPhysics.Trees.Tree.BinnedRefine(Int32, BepuUtilities.Collections.QuickList`1<Int32> ByRef, Int32, BepuUtilities.Collections.QuickList`1<Int32> ByRef, BepuPhysics.Trees.BinnedResources ByRef, BepuUtilities.Memory.BufferPool)
   at BepuPhysics.Trees.Tree.RefitAndRefine(BepuUtilities.Memory.BufferPool, Int32, Single)
   at BepuPhysics.CollisionDetection.BroadPhase.Update(BepuUtilities.IThreadDispatcher)
   at BepuPhysics.Simulation.CollisionDetection(Single, BepuUtilities.IThreadDispatcher)
   at BepuPhysics.DefaultTimestepper.Timestep(BepuPhysics.Simulation, Single, BepuUtilities.IThreadDispatcher)
   at BepuPhysics.Simulation.Timestep(Single, BepuUtilities.IThreadDispatcher)
   at RoboScapeSimulator.SimulationInstance.Update(Single)
   at RoboScapeSimulator.Room.Update(Single)
   at Program+<>c__DisplayClass0_1.<<Main>$>b__1(System.Object)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.TimerQueueTimer.Fire(Boolean)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()

May need a better check for this sort of failure to auto-restart, should also talk to bepu people about this.

Help/Documentation content

Need to have help content (possibly available through the wiki/pages here) and a way to access relevant info in use. Tooltips would be helpful.

Access control limitation

There needs to be a system to control who is allowed to create rooms, eventually even who can see/join rooms. At some point teachers would have access codes to create rooms, but how this whole system will work will probably require input from Akos.

Testing

Need to figure out how best to test things, the physics engine might be light enough to run tests inside of it accelerated.

Update visualinfo after room start/initial join

Drone test

Quadcopter style drone simulation

Goals:

Allow use in same environments as ground-based robots (e.g. a drone and a ground robot work as a team)
Easy interface for students to interact with, but also allow exploration into control theory
Should be possible to crash if code tries to do something invalid, would be nice to be able to do "tricks" as well, but that might be harder depending on how controlling them works
Combine with some sort of sensor (at least "GPS", but LIDAR would be very cool). Maybe a sensor that can take advantage of the drone's altitude (e.g. a thermal camera for a firefighting robot to team with)?

Big question is how to control it:

Direct motor speed control is too complex to model well and too complex for novice students to work with, may not even be possible with NetsBlox latency
PID(?) controller in simulator, NetsBlox RPCs allow tuning it?
Very simplified commands for "go to x,z", "go to height y"? Maybe set target x/y/z speeds and require student code to handle getting to correct position with control running in NetsBlox?

Robot reset test sometimes failing

Variations on four color environment for creative uses

LIDAR and PositionSensor
Varying numbers
With/without walls

Rename TransformSensor to PositionSensor

Also add getPosition RPC that returns x y z tuple.

Better default environment

Current default environment doesn't show many features (although it's pretty lightweight, which is desirable), and the random boxes/robots sometimes have issues. It would be better to have a default environment that's more reliable and a little more interesting.

Quick room connect link

Allow for directly connecting to a room with a link, with a share room button in the sidebar.

Tournament management system

This is a longer term goal, especially since #5 is a prerequisite for it.

One of the big challenges with using the robots IRL is assessment. Competitions are "fun" and mean students do more than just wait their turn to demonstrate, but tournaments are hard to manage while running a class, and not all tournament systems are good for all groups. Some may want a quick elimination bracket, some may have time for a Swiss tournament which has more "fairness", some may want a full round-robin.

With the robots reset when the round begins, it also eliminates the usual real-life difficulty of making sure the robots have been reset and were turned on at the same time.

Ideally, the system would allow for students to be given a "practice" section of the environment to work with (see #7), and the students in the current competition round would be alerted and have their robots moved to the main area so they can start.

Bepu 2.4.0 Update

This new release provides many fixes and performance improvements, but may require some changes to switch to.

Cybersecurity Activity Environments

Customizable environments created in NetsBlox

It would be nice if some portion of the environment could be created from the NetsBlox side, even if it's just specifying where some blocks/robots would go. Things like triggers could even be included, although they would likely need to talk to NetsBlox code instead of C# code. The resource limits of this would need to be figured out (e.g. how many robots are allowed?), but if it can be combined with a "create room" block, custom environments could be entirely contained in a NetsBlox project. Some of this functionality could also allow for things like #111 depending on how this functionality is exposed.

Robot hitbox not exact

Robots occasionally get stuck on corners they should be fine going around, the hitboxes may be slightly too wide.

"Claimable" robots for cybersec scenarios

Scenarios can specify that robots are "claimable", presenting a button in the client UI, or for a claimed robot, the user claiming it
Some scenarios may require allowing multiple claims per user
A claimed robot can only be individually reset or have its button pressed by the claimant, and LED/numeric display messages will only be sent to that user
User can unclaim robot
Once claiming user disconnects, the robot should be unclaimed again

"Realism" options for sensors

Reusable code to create noise/error/drift/latency/non-linearity/bucketing/etc for sensors to make them more realistic could be useful for more advanced scenarios.

Simulated network latency/loss

Ben requested this.

Latency (and possibly other network conditions) for robot communications should be able to be set by the room's owner. This allows for simulating distance between robot and user, which would be useful both for teaching and for testing performance of other code under poor network conditions.

Ultrasound "area" sensor

Copy button should be hidden in non-secure context

Clipboard API won't work in all contexts, it should be hidden to prevent confusion from a non-functional button.

New networking system?

It would be nice to be able to use something like geckos.io for UDP-based networking, or even just newer versions of socket.io, but the C# library for socket.io being used only supports an older version. My current plan for this is to have a Node program handle the networking, with messages passed back and forth with some form of IPC, but this will require a lot of testing to make sure it is feasible/reliable. This may solve #68 if that is due to the library used.

Warm up on server start

There seems to be some sort of issue with JIT where the first rooms launched by the server misbehave and need a reset. Having the server startup include opening a room and running it for a bit should fix this.

gsteinltu / roboscapesimulator Goto Github PK

roboscapesimulator's People

Contributors

Watchers

Forkers

roboscapesimulator's Issues

Recommend Projects

Recommend Topics

Recommend Org