robotpy/mostrobotpy

Official Repository of python implementation of WPILib components

RobotPy: Python for FRC cRIO Robot Controller

Author:	Peter Johnson, FRC Team 294
Author:	Dustin Spicuzza, FRC Team 2423/1418
Copyright:	Copyright © 2010, Peter Johnson, Ross Light, Dustin Spicuzza

About RobotPy

RobotPy is a distribution of Python intended to be used for the FIRST Robotics Competition. Teams can use this to write their robot code in Python, a powerful dynamic programming language.

Features

• Python is simple to learn and easy to maintain.
• RobotPy lets you reload code without restarting.
• RobotPy provides access to the WPILib class library.
• You don't need to use WindRiver (unless you're rebuilding RobotPy itself).

Installation for most people

Download the binary releases from our FIRSTForge site, unpack them, and run install.py. Download from: http://firstforge.wpi.edu/sf/projects/robotpy

Installation from Source using our build VM

Using the provided build virtual machine is the easiest way to build RobotPy from source. See build-vm/README.txt for instructions.

Installation from Source (Advanced Users Only)

Step 1: Install Python

Python 2.7.5 is known to work. Ensure that you add your python installation directory to your PATH variable.

Note: This step is required for SIP installation

Step 2: Install SIP v4.15.3:

The cmake build process currently checks for exact versions of SIP, as the generated code changes from release to release.

http://www.riverbankcomputing.com/static/Docs/sip4/installation.html

Note: Installation of SIP on Windows requires Visual Studio or MinGW installed, see the SIP build instructions for more details.

Step 3: Build the source tree

TODO: Update these instructions for Windows too?

The RobotPy build process uses cmake to build the RobotPy distribution. frcmake is a wrapper around cmake that sets the build environment up to build for the cRio.

RobotPy currently uses the GCC 4.8 compiler distributed at the following URL to build the binary distribution. Wind River may be used, but we haven't built it with Wind River, so you may run into trouble using it with frcmake.

http://firstforge.wpi.edu/sf/projects/c--11_toochain

Create a directory (the 'binary directory') outside of the RobotPy source tree, and run this:

$ frcmake /path/to/robotpy

Once you've done this, you can build the code:

$ make

To copy the build output to the 'dist' directory, run this command from your binary directory:

$ make install

To build a binary release, just run the following command and a zip file will be created in your build directory.

$ make package

Step 4: Robot Installation

From the 'dist' directory inside of your binary directory, run install.py

Alternatively, you can FTP the contents of the 'robot' directory to your cRio directly. However, install.py is easier.

Development tools

See the utilities directory for useful development tools that may make your RobotPy development experience easier.

The pyfrc python package is a recommended development package that provides unit testing and other capabilities for your robot code.

https://github.com/robotpy/pyfrc

Technical Overview

RobotPy is a packaging of a patched Python 3.2 interpreter (found in the RobotPy/Python subdirectory of the source code). All access to the WPILib is generated by a SIP interface, which is found in Packages/wpilib/sip/. When the robot is started, it initializes the Python interpreter and runs the file py/boot.py. From there, all responsibility is given to the boot.py script, which is referred to as the bootloader.

If boot.py ever exits (due to an exception, for example), the C++ code exits. The default boot.py simply exits on any user exception. If this happens, you can reboot easily via NetConsole by simply typing "reboot" followed by hitting the enter key. This is how code reloads are performed. As boot.py is written in Python, this behavior can be customized as desired.

Major Differences from standard Python

• Several Python modules with large and/or incompatible dependencies removed, namely: curses, dbm, gdbm, tkinter, nis, ossaudiodev, resource, spwd, syslog, termios, audioop, bz2, crypt, grp, ssl, pwd, and mmap.

Licensing

A brief overview of licensing terms:

• Python: PSF License
• RobotPy (all other support code): MIT License

If you redistribute RobotPy and add other libraries, please include their licensing information here.

RobotPy

Copyright © 2010 Peter Johnson

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

mostrobotpy's Issues

infinite value not propagated to NT4 topic properties

MRE:

import wpilib
import ntcore

class MyRobot(wpilib.TimedRobot):
    def robotInit(self): 
        table = ntcore.NetworkTableInstance.getDefault().getTable("TestTable")
        
        topic = table.getDoubleTopic("testVal")
        self.pub = topic.publish(ntcore.PubSubOptions(
                                    sendAll=False, keepDuplicates=False))
        self.pub.setDefault(0.0)

        propValDes = float('-inf')
        topic.setProperty("doesntWork", propValDes)

        propValAct = topic.getProperty("doesntWork")
        print(propValDes)
        print(propValAct)

    def robotPeriodic(self):
        self.pub.set(wpilib.Timer.getFPGATimestamp())

if __name__ == '__main__':
    wpilib.run(MyRobot)

In Java, it was supported to pass inf and -inf as a value for the properties. In Python, a value of inf is returned from a paired get/set properties call. However, in NT4 clients (both the javascript one I've got and outlineviewer) the value comes back as null

cscore on roborio wpi::Now error

Need to call WPI_Impl_SetupNowUseDefaultOnRio if on rio and no HAL is going to be loaded

Support new structured bindings for ntcore

I think the best way to make it dynamic is to accept any type that has a _wpistruct member (or something) that contains a serialization object of a known type. Then that thing does the conversion, no central registry needed. We would implement a C++ class that implements the correct concept and interacts with the serialization object dynamically, and any function that takes a structured binding concept would be bound with this base type.

Just have to remember to add the little factories for each type that supports it, but that should be easy enough and avoids any dependency issues, and if we're careful you should be able to easily support your own structured binding objects from python.

Remove pkg_resources usage

python3.12/site-packages/wpilib/_impl/main.py:9
  python3.12/site-packages/wpilib/_impl/main.py:9: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import iter_entry_points

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

For non-releases change artifact dependencies

They should depend on == some version otherwise could accidentally install something that isn't part of the PR or whatever build.

Add protobuf serialization support for ntcore

Not really sure if anyone is going to care about this? Seems likely a bit complicated, and we'd need to build the protobuf wheel in roborio-wheels.

Cleanup old repository content

Old workflows, RTD, unused development tools, whatever.

Suppress 'Camera Server Library Not Found' message

It isn't relevant for us. Might need a wpilib patch.

Bring in robotpy-romi

rdev: yoink progressbar from allrobotpy

wpimath: Rotation2d.fromRotations missing

[BUG]: ADIS16470_IMU crashes in test

Problem description

When we run python -m robotpy test with an ADIS16470_IMU, it crashes.

C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\python.exe -m robotpy test 
11:09:19:364 WARNING : pyfrc.physics       : Cannot enable physics support, C:\Users\etienne.beaulac\Desktop\test\physics.py not found
============================= test session starts =============================
platform win32 -- Python 3.12.1, pytest-7.4.4, pluggy-1.3.0
rootdir: C:\Users\etienne.beaulac\Desktop\test\tests
plugins: reraise-2.1.2
collected 4 items

pyfrc_test.py .Windows fatal exception: access violation

Thread 0x00003ca8 (most recent call first):
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\ntcore\_logutil.py", line 94 in _logging_thread
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\threading.py", line 1010 in run
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\threading.py", line 1073 in _bootstrap_inner
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\threading.py", line 1030 in _bootstrap

Current thread 0x00001a38 (most recent call first):
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pyfrc\test_support\pytest_plugin.py", line 114 in robot
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\fixtures.py", line 911 in _teardown_yield_fixture
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\fixtures.py", line 1024 in finish
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\fixtures.py", line 701 in <lambda>
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 526 in teardown_exact
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 182 in pytest_runtest_teardown
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_callers.py", line 77 in _multicall
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_hooks.py", line 493 in __call__
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 262 in <lambda>
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 341 in from_call
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 261 in call_runtest_hook
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 222 in call_and_report
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 134 in runtestprotocol
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\runner.py", line 114 in pytest_runtest_protocol
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_callers.py", line 77 in _multicall
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_hooks.py", line 493 in __call__
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\main.py", line 350 in pytest_runtestloop
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_callers.py", line 77 in _multicall
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_hooks.py", line 493 in __call__
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\main.py", line 325 in _main
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\main.py", line 271 in wrap_session
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\main.py", line 318 in pytest_cmdline_main
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_callers.py", line 77 in _multicall
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pluggy\_hooks.py", line 493 in __call__
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\_pytest\config\__init__.py", line 169 in main
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pyfrc\mains\cli_test.py", line 115 in _run_test
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\pyfrc\mains\cli_test.py", line 62 in run
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\robotpy\main.py", line 314 in main
  File "C:\Users\etienne.beaulac\AppData\Local\miniconda3\envs\frc2024\Lib\site-packages\robotpy\__main__.py", line 4 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

Process finished with exit code -1073741819 (0xC0000005)

Operating System

Windows

Installed Python Packages

bcrypt                   4.1.2     
cffi                     1.16.0    
colorama                 0.4.6     
cryptography             41.0.7    
iniconfig                2.0.0     
numpy                    1.26.3    
packaging                23.2      
paramiko                 3.4.0     
Pint                     0.23      
pip                      23.3.2
pluggy                   1.3.0
pycparser                2.21
pyfrc                    2024.0.1
PyNaCl                   1.5.0
pynetconsole             2.0.4
pyntcore                 2024.1.1.1
pytest                   7.4.4
pytest-reraise           2.1.2
robotpy                  2024.1.1.3
robotpy-apriltag         2024.1.1.1
robotpy-cli              2024.0.0
robotpy-commands-v2      2024.0.0b4
robotpy-cscore           2024.1.1.1
robotpy-hal              2024.1.1.1
robotpy-halsim-ds-socket 2024.1.1.1
robotpy-halsim-gui       2024.1.1.1
robotpy-halsim-ws        2024.1.1.1
robotpy-installer        2024.1.1
robotpy-rev              2024.2.0
robotpy-wpilib-utilities 2024.0.0
robotpy-wpimath          2024.1.1.1
robotpy-wpinet           2024.1.1.1
robotpy-wpiutil          2024.1.1.1
setuptools               69.0.3
tomli                    2.0.1
typing_extensions        4.9.0
wheel                    0.42.0
wpilib                   2024.1.1.1

Reproducible example code

import wpilib

class Robot(wpilib.TimedRobot):
    def robotInit(self):
        self.gyro = wpilib.ADIS16470_IMU()

Add __repr__ for TrapezoidProfile.constraints/state et al

Add pyi to linuxarm64 wheels

Unlike the rio wheels, editing code directly on a Raspberry Pi with intellisense is possible (Vscode Remote).

It would be good to run stubgen on wheels targeting the Pi.

RobotPy startup on a RoboRIO is slow

On my RoboRIO with a 2016 image, I get the following (mostly consistent timing between runs, no DS connected):

roboRIO-2423-FRC:~$ time python3 -c 'import hal'

real	0m4.954s
user	0m1.280s
sys	0m0.170s
roboRIO-2423-FRC:~$ time python3 -c 'import wpilib'

real	0m6.984s
user	0m3.270s
sys	0m0.260s

HAL library loading may be a big chunk of this:

roboRIO-2423-FRC:~$ time python3 -c 'import ctypes; ctypes.CDLL("/usr/local/lib/python3.5/site-packages/hal_impl/libHALAthena.so")'

real	0m3.093s
user	0m0.470s
sys	0m0.070s

As a baseline:

roboRIO-2423-FRC:~$ time python3 -c 'pass'

real	0m0.210s
user	0m0.180s
sys	0m0.020s
roboRIO-2423-FRC:~$ time python3 -c 'import pip'

real	0m3.002s
user	0m2.760s
sys	0m0.220s

It would be good to reduce the startup time somehow.

RFC: remove `python robot.py xxx`

In pursuit of making installation simpler, I'm contemplating changing the way that users launch their code (which has been the same since the 2015 season). It is currently a week before kickoff, but most of this is fairly simple.

For example, to run simulation:

---

Currently

py robot.py sim          # Windows
python robot.py sim      # Linux/macOS
./robot.py sim           # Linux/macOS w/executable bit set

And users must also add the following magic to their file:

if __name__ == "__main__":
    wpilib.run(MyRobot)

---

Proposed

Users no longer need to add the magic to their file. The run command will assume that the robot is in robot.py, and that it is a subclass of wpilib.RobotBase and it is defined in robot.py. If there are multiple robot classes, the user will need to add a setting to pyproject.toml.

robotpy sim              # Installed python script in path (windows or linux)
py -m robotpy sim        # Windows
python -m robotpy sim    # Linux/macOS

As an alternative (could do the above and this), a user could also create a file __main__.py

import robotpy.launcher

And then you could do:

py . sim          # Windows
python . sim      # Linux/macOS
./__main__.py sim # Linux/macOS w/executable bit set

---

As part of this, I propose removing the robotpy-installer command, and move all of its installation actions into the robotpy command via the extension mechanism that we currently use to register new subcommands of robot.py.

robotpy download
robotpy install
.. etc

In particular, this solves the chicken/egg problem I ran into when contemplating how to implement a sync command that a user could use to ensure their local robotpy installation is up to date. If a user has an import that isn't resolvable, then robot.py sync isn't possible to implement, so users would need to learn two commands.

---

Existing users have a lot of muscle memory that will need to be changed, but I think it would be better to do this change before the first official season instead of afterwards.

Not requiring robot.py to be ran directly will allow us to create better error messages for users with missing imports, make some aspects of unit tests simpler to change in the future, and provide a unified entrypoint for users to interact with robotpy.

---

Questions

• Is this too risky to do a week before kickoff?
  • Doing this isn't technically challenging, just have to shuffle a lot of pieces around
  • ... haven't written any installation docs for frc-docs, so no risk that it doesn't get changed :-D
• Do the advantages of a unified launcher outweigh destroying existing user's muscle memory?
  • Existing documentation on the internet says do python robot.py XXX

[BUG]: add typealias for units to typestubs

Problem description

As mentioned in robotpy/robotpy-wpimath#29 by @auscompgeek :

Ideally we'd have meters: TypeAlias = float somewhere. I don't think we have a way of injecting things into the type stub output yet.

This would be the right way to fix it... but sounds like work. IMO it's better to have the names in there (and have the type hints be invalid) than to not have the names, since the units aren't always obvious.

Would likely require a change in pybind11-stubgen and/or robotpy-build.

Operating System

Windows, MacOS, Linux

Installed Python Packages

No response

Reproducible example code

No response

TrapezoidProfile signature includes default object repr for State

despite having a __repr__ implementation in the template_inline_code.

[BUG]: PWMMotorController.addFollower causes segfaults on motor.set

Problem description

When adding a follower to a PWMMotorController without keeping a reference to the follower, a subsequent call to .set() will segfault when trying to set the follower.

Operating System

Windows, MacOS, Linux, RoboRIO, Raspbian

Installed Python Packages

No response

Reproducible example code

import wpilib

class Robot(wpilib.TimedRobot):
    def robotInit(self):
        self.motor = wpilib.PWMSparkMax(1)
        self.motor.addFollower(wpilib.PWMSparkMax(2))

    def autonomousPeriodic(self):
        self.motor.set(0.5)

`import wpilib` in repl kills interpreter

Importing wpilib in a repl on the roborio causes the python interpreter to be killed.

[BUG]: CommandGenericHID button trigger should use raw state

Problem description

The current button trigger implementation in commands2 uses getRawButtonPressed, but it should use getRawButton, as done here :
https://github.com/wpilibsuite/allwpilib/blob/84ef71ace0648a71bd6e60e16c38be75914bb59d/wpilibNewCommands/src/main/java/edu/wpi/first/wpilibj2/command/button/CommandGenericHID.java#L57

The consequence is that commands scheduled with whileTrue end instantly because getRawButtonPressed returns False right after a True state.

Operating System

Windows

Installed Python Packages

bcrypt                   4.1.2
cffi                     1.16.0
colorama                 0.4.6
cryptography             41.0.7
iniconfig                2.0.0
numpy                    1.26.3
packaging                23.2
paramiko                 3.4.0
Pint                     0.23
pip                      23.3.2
pluggy                   1.3.0
pycparser                2.21
pyfrc                    2024.0.1
PyNaCl                   1.5.0
pynetconsole             2.0.4
pyntcore                 2024.2.1.2
pytest                   7.4.4
pytest-reraise           2.1.2
robotpy                  2024.2.1.1
robotpy-apriltag         2024.2.1.2
robotpy-cli              2024.0.0
robotpy-commands-v2      2024.2.1
robotpy-cscore           2024.2.1.2
robotpy-hal              2024.2.1.2
robotpy-halsim-ds-socket 2024.2.1.2
robotpy-halsim-gui       2024.2.1.2
robotpy-halsim-ws        2024.2.1.2
robotpy-installer        2024.1.3
robotpy-rev              2024.2.0
robotpy-wpilib-utilities 2024.0.0
robotpy-wpimath          2024.2.1.2
robotpy-wpinet           2024.2.1.2
robotpy-wpiutil          2024.2.1.2
setuptools               69.0.3
tomli                    2.0.1
typing_extensions        4.9.0
wheel                    0.42.0
wpilib                   2024.2.1.2

Reproducible example code

import commands2
import wpilib
from commands2 import CommandScheduler
from commands2.button import CommandJoystick, Trigger


class TestCommand(commands2.Command):
    def initialize(self):
        print("initialized")

    def execute(self):
        print("execute")

    def isFinished(self) -> bool:
        return False

    def end(self, interrupted: bool):
        print("end", f"{interrupted=}")


class Robot(wpilib.TimedRobot):
    def robotInit(self):
        self.joystick = CommandJoystick(0)
        self.joystick.button(1).whileTrue(TestCommand())
        # Trigger(lambda: self.joystick._hid.getRawButton(1)).whileTrue(TestCommand())

    def robotPeriodic(self):
        CommandScheduler.getInstance().run()

Add XRP/ROMI support to pyproject.toml

Referring to:

• https://docs.wpilib.org/en/stable/docs/xrp-robot/programming-xrp.html
• https://github.com/robotpy/examples/blob/main/RomiReference/robot.py#L28

probably need to add something that enables the ws-client by default if the config is there, and also sets the HALSIMWS_HOST and HALSIMWS_PORT environment variables.

Document operator overloads for geometry classes

rdev: add option to turn off stubgen

fork_exec: cannot allocate memory

  File "robot.py", line 59, in createObjects
    wpilib.CameraServer.launch()
  File "/usr/local/lib/python3.8/site-packages/wpilib/cameraserver.py", line 83, in launch
    proc = subprocess.Popen(
  File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1637, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
OSError: [Errno 12] Cannot allocate memory

This user is importing rev, ctre, numpy, and lots of things, and my belief is that when we do a subprocess.run it's copying the entire process memory space (because that's what fork does on linux? -- though I thought it was supposed to be copy on write?), and failing to do so.

A workaround is to put the cameraserver launch at the beginning of the robot.py before anything else is imported. A bit of a hack, but it works. Need to think about a better way to address this.

New Feature: Single Step Debugger Support

Problem description

Basic debug support through debugpy works well today with vsCode.

Documenting this here for potential future integration.

Overall design goal: single-click debugger activation that lets you walk code line by line on a development computer, while execution is happening on the RIO.

1. Standard vsCode python extensions
2. install debugpy on both the RIO and local desktop
3. Add a vsCode task to start the robot in debug mode - for us, this involves dropping a blank file with a magic name in the RIO's filesystem, then restarting the robot code
4. a small snippet of code in the robot main code to detect the "wait for debugger" file trigger and wait for a debug connection.

tasks.json:

        {
            "label": "PyFRC: _Activate Debug",
            "type": "shell",
            "windows": {
                "command": "ssh [email protected] -t 'chmod +x ./robotCommand; touch /home/lvuser/py/enableDebug; ./robotCommand || true; echo Waiting for robot program to start...; sleep 5' "
            },
            "linux": {
                "command": "ssh [email protected] -t 'chmod +x ./robotCommand; touch /home/lvuser/py/enableDebug; ./robotCommand || true; echo Waiting for robot program to start...; sleep 5' "
            },
            "group": {
                "kind": "build",
            },
            "presentation": {
                "reveal": "always",
                "panel": "dedicated",
                "clear": true,
                "focus": true,
                "showReuseMessage": false
            },
            "problemMatcher": [],
            "icon": {
                "id": "cloud-upload"
            }
        },

launch.json:

...
        {
            "name": "Debug RoboRIO",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "roboRIO-1736-frc.local",
                "port": 5678
            },
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "/home/lvuser/py"
                }
            ],
            "justMyCode": true,
            "preLaunchTask": "PyFRC: _Activate Debug"
        },
...

Robot Code:

if __name__ == '__main__':

    enableDebug = os.path.isfile("/home/lvuser/py/enableDebug")
    if(enableDebug):
        print("Starting Debug Support....")
        import debugpy 
        debugpy.listen(('0.0.0.0', 5678))
        debugpy.wait_for_client()

    wpilib.run(MyRobot)

The real trick is just to make sure the robot code gets restarted with the correct value for enableDebug - it does slow down execution noticeably, and isn't something I think you'd want all the time.

I've got no idea if there's a better way than a file (environment variables?), but that just seemed to be the most reliable way.

One nice thingabout using a file: if the code is simply redeployed, the file is wiped away, so the code goes back to "normal/non-debug" mode.

Hopefully helps? Other than the funkiness of the file itself, I'd highly recommend the rest.

Operating System

Windows

Installed Python Packages

N/A

Reproducible example code

N/A

rdev: enable ccache by default

rdev: add support for running rdev on Windows

Either

1. Launch from python on all platforms
2. Add a batch file for windows

[BUG]: python 3.8/3.9 broken

Problem description

pkg_resources migration wasn't tested.

Operating System

Windows, MacOS, Linux, RoboRIO, Raspbian

Installed Python Packages

No response

Reproducible example code

No response

[BUG]: NT struct array topic doesn't publish schema

Problem description

x = ntcore.NetworkTableInstance.getDefault().getStructArrayTopic("foo", Translation2d).publish()
x.set([Translation2d(0,0)])

is rendered as opaque in glass, and there is only one publisher, "foo."

adding

y = ntcore.NetworkTableInstance.getDefault().getStructTopic("foo", Translation2d).publish()
y.set(Translation2d(0,0)

adds the "/.schema" publisher and fixes the array rendering.

Operating System

Linux

Installed Python Packages

No response

Reproducible example code

No response

rdev: wpimath build depends on numpy

Wpimath build depends on numpy due to stubgen.
Either

1. Add numpy as a dependency
2. #16

[BUG]: SimpleMotorFeedforwardRadians missing from __init__.py

Problem description

SimpleMotorFeedforwardRadians is not listed in https://github.com/robotpy/mostrobotpy/edit/main/subprojects/robotpy-wpimath/wpimath/controller/__init__.py

It appears to be present and functional in controller.py in the same directory.

Operating System

Windows

Installed Python Packages

alabaster                     0.7.16
asttokens                     2.4.1
attrs                         23.2.0
Babel                         2.14.0
bcrypt                        4.1.2
certifi                       2023.11.17
cffi                          1.16.0
charset-normalizer            3.3.2
colorama                      0.4.6
comm                          0.2.1
cryptography                  41.0.7
debugpy                       1.8.0
decorator                     5.1.1
doc8                          1.1.1
docutils                      0.20.1
executing                     2.0.1
hypothesis                    6.96.1
idna                          3.6
imagesize                     1.4.1
iniconfig                     2.0.0
ipykernel                     6.29.0
ipython                       8.20.0
jedi                          0.19.1
Jinja2                        3.1.3
jupyter_client                8.6.0
jupyter_core                  5.7.1
MarkupSafe                    2.1.4
matplotlib-inline             0.1.6
nest-asyncio                  1.5.9
numpy                         1.26.3
packaging                     23.2
paramiko                      3.4.0
parso                         0.8.3
pbr                           6.0.0
phoenix6                      24.1.0
photonlibpy                   2024.2.4
Pint                          0.23
pip                           24.0
platformdirs                  4.1.0
pluggy                        1.3.0
prompt-toolkit                3.0.43
psutil                        5.9.8
pure-eval                     0.2.2
pycparser                     2.21
pyfrc                         2024.0.1
Pygments                      2.17.2
PyNaCl                        1.5.0
pynetconsole                  2.0.4
pyntcore                      2024.2.1.2
pytest                        7.4.4
pytest-reraise                2.1.2
python-dateutil               2.8.2
pywin32                       306
pyzmq                         25.1.2
requests                      2.31.0
restructuredtext_lint         1.4.0
robotpy                       2024.2.1.1
robotpy-apriltag              2024.2.1.2
robotpy-cli                   2024.0.0
robotpy-commands-v2           2024.2.1
robotpy-cscore                2024.2.1.2
robotpy-ctre                  2024.1.1
robotpy-hal                   2024.2.1.2
robotpy-halsim-ds-socket      2024.2.1.2
robotpy-halsim-gui            2024.2.1.2
robotpy-halsim-ws             2024.2.1.2
robotpy-installer             2024.1.3
robotpy-navx                  2024.1.0
robotpy-pathplannerlib        2024.1.1
robotpy-playingwithfusion     2024.1.0
robotpy-rev                   2024.2.0
robotpy-romi                  2024.2.1.2
robotpy-wpilib-utilities      2024.0.0
robotpy-wpimath               2024.2.1.2
robotpy-wpinet                2024.2.1.2
robotpy-wpiutil               2024.2.1.2
robotpy-xrp                   2024.2.1.2
setuptools                    69.0.3
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
Sphinx                        7.2.6
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
stack-data                    0.6.3
stevedore                     5.1.0
tomli                         2.0.1
tornado                       6.4
traitlets                     5.14.1
typing_extensions             4.9.0
urllib3                       2.1.0
wcwidth                       0.2.13
wpilib                        2024.2.1.2

Reproducible example code

import wpimath.controller

ffr = wpimath.controller.SimpleMotorFeedforwardRadians

Automatically put deploy.json information into NetworkTables?

Seems like it might be useful to just do this automatically.

Create unified API docs

If threads crash with unhandled exceptions, the driver station should be notified

I suspect there's a python mechanism to do this. The DriverStation error reporting mechanism could be used to do this.

Automatically release robotpy-meta when this repo pushes releases

We did it before, we should do it again. Need to:

• Bump the various requirements*.txt in robotpy-meta
• Push a new tag for robotpy-meta

One thing we need to be smart about for that is to check the tag and ensure that it matches the tag MAJOR.MINOR for this repository (so if the current version of robotpy-meta is 2024.1.2, and a 2024.2.3.0 release is pushed for mostrobotpy, then the next robotpy-meta version should be 2024.2.1. Also need to handle beta versions too.

cscore cv::Mat converter segfault

Original CD post: https://www.chiefdelphi.com/t/segmentation-fault-from-cvsink/453849/6

The core problem is that the original numpy array is being destroyed, and the new numpy array is referring to the old data. To prevent the crash from happening, you just need to keep a reference to the original data.

Here's a reproducer:

def test_nonempty_img():
    img = np.zeros(shape=(480, 640, 3), dtype=np.uint8)
    sink = cs.CvSink("something")
    _, img = sink.grabFrame(img)
    _, img = sink.grabFrame(img) # crashes here because img is a new numpy object, not the original one

LinearFilter *FiniteDifference not implemented

Should take template arguments (1-2, 2-5). Implement as a function that takes the template args and calls the right thing based on that.

robotpy-build won't let you set your own args, so it'll break at the moment unless you do it via inline_cpp, but then you have to duplicate the docs.

Create a pytest helper to aid users in running tests against directories of images

Ideally, if your image processing code were structured correctly, such a helper would allow you to compute scores for each image and compare them to known values, so that as you improve your image processing code you can be assured that you haven't broken other cases you're tuning for.

There's a lot of boilerplate for this sort of thing, would be useful.

QuickVision example fails with SIGABRT

It's hard to duplicate this, but...

(gdb) bt
#0  0xb69031f4 in raise () from target:/lib/libc.so.6
#1  0xb68ee1d8 in abort () from target:/lib/libc.so.6
#2  0xb5d57470 in ?? () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#3  0xb6ac506c in ?? () from target:/lib/libpthread.so.0
#4  0xb5d6199c in uv_once(int*, void (*)()) () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#5  0xb5d57624 in uv__work_submit(uv_loop_s*, uv__work*, uv__work_kind, void (*)(uv__work*), void (*)(uv__work*, int)) ()
   from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#6  0xb5d6dd9c in uv_getaddrinfo(uv_loop_s*, uv_getaddrinfo_s*, void (*)(uv_getaddrinfo_s*, int, addrinfo*), char const*, char const*, addrinfo const*) () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#7  0xb5d20128 in wpi::uv::GetAddrInfo(wpi::uv::Loop&, std::shared_ptr<wpi::uv::GetAddrInfoReq> const&, std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, addrinfo const*) ()
   from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#8  0xb5d3aa50 in wpi::ParallelTcpConnector::Connect() ()
   from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#9  0xb5d3fff8 in std::_Function_handler<void (wpi::promise<void>, std::function<void (wpi::uv::Loop&)>), wpi::EventLoopRunner::Thread::Thread()::{lambda(auto:1, std::function<void (wpi::uv::Loop&)>)#1}>::_M_invoke(std::_Any_data const&, wpi::promise<void>&&, std::function<void (wpi::uv::Loop&)>&&) () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#10 0xb5d40130 in wpi::uv::AsyncFunction<void (std::function<void (wpi::uv::Loop&)>)>::Create(std::shared_ptr<wpi::uv::Loop> const&, std::function<void (wpi::promise<void>, std::function<void (wpi::uv::Loop&)>)>)::{lambda(uv_async_s*)#1}::_FUN(uv_async_s*) () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#11 0xb5d63b60 in ?? () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#12 0xb5d5e9f4 in uv__io_poll(uv_loop_s*, int) () from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#13 0xb5d6b8fc in uv_run(uv_loop_s*, uv_run_mode) ()
   from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#14 0xb5d3f7d4 in wpi::EventLoopRunner::Thread::Main() ()
   from target:/usr/local/lib/python3.12/site-packages/wpinet/lib/libwpinet.so
#15 0xb62c8d28 in ?? () from target:/usr/local/lib/python3.12/site-packages/wpiutil/lib/libwpiutil.so
#16 0xb60a32b8 in ?? () from target:/usr/lib/libstdc++.so.6
#17 0xb6abcda0 in ?? () from target:/lib/libpthread.so.0

I killed the webserver process and it seems to work, so it might be that it's running out of memory.

ntcore: TimestampedRaw should provide bytes, not list[int]

Mentioned in passing by Photonvision folks, but I don't know any of the details.

Inherited from how NT4 returns the raw bytes topic value

Originally posted by @gerth2 in PhotonVision/photonvision#1040 (comment)

SendableRegistry Add/AddLW/AddChild are broken

They are trying to use keep alives to keep the pointers alive but it's not the right solution.

Probably need to do the same thing we do for SmartDashboard with a separate hash map to hold references.

import wpilib
import wpilib.drive
import wpiutil

leftLeader = wpilib.PWMSparkMax(1)
rightLeader = wpilib.PWMSparkMax(2)
drive = wpilib.drive.DifferentialDrive(
            leftLeader,
            rightLeader,
        )

wpiutil.SendableRegistry.addChild(drive, leftLeader)

$ python ../../t.py 
Traceback (most recent call last):
  File "../../t.py", in <module>
    wpiutil.SendableRegistry.addChild(drive, leftLeader)
RuntimeError: Could not activate keep_alive!

Pin robotpy-build version

rdev: turn on parallel compilation by default

wpimath: TrajectoryConstraint.MinMax needs convenience constructor

Can't just construct and return at this time.

[BUG]: getDeployDirectory returns

Problem description

wpilib.getDeployDirectory() is returning tests\deploy for the deploy directory, rather than just deploy. It looks like the underlying wpilib implementation doesn't account for the fact that the working directory for tests is not the same folder as robot.py?

Currently, we're using pathplanner to store .path files in the deploy directory. However, when running the test suite, the code that works on the robot fails due to the extra test in the path.

I think that, at least, during tests, the python implementation might need to be unique to make sure it points back at the correct folder, regardless of where the code is running from.

Operating System

Windows

Installed Python Packages

Package                   Version
------------------------- -----------
2to3                      1.0
anyio                     3.6.2
argon2-cffi               21.3.0
argon2-cffi-bindings      21.2.0
arrow                     1.2.3
astroid                   2.15.2
asttokens                 2.2.1
attrs                     23.1.0
backcall                  0.2.0
bcrypt                    4.0.1
beautifulsoup4            4.12.2
bleach                    6.0.0
cffi                      1.15.1
click                     8.1.3
colorama                  0.4.6
comm                      0.1.3
coverage                  7.2.3
cryptography              40.0.2
debugpy                   1.6.7
decorator                 5.1.1
defusedxml                0.7.1
dill                      0.3.6
executing                 1.2.0
fastjsonschema            2.16.3
fqdn                      1.5.1
idna                      3.4
importlib-metadata        6.8.0
importlib-resources       6.1.0
iniconfig                 2.0.0
ipykernel                 6.22.0
ipython                   8.12.0
ipython-genutils          0.2.0
ipywidgets                8.0.6
isoduration               20.11.0
isort                     5.12.0
jedi                      0.18.2
Jinja2                    3.1.2
jsonpointer               2.3
jsonschema                4.17.3
jupyter                   1.0.0
jupyter_client            8.2.0
jupyter-console           6.6.3
jupyter_core              5.3.0
jupyter-events            0.6.3
jupyter_server            2.5.0
jupyter_server_terminals  0.4.4
jupyterlab-pygments       0.2.2
jupyterlab-widgets        3.0.7
lazy-object-proxy         1.9.0
libusb                    1.0.26b5
libusb-package            1.0.26.2
markdown-it-py            3.0.0
MarkupSafe                2.1.2
matplotlib-inline         0.1.6
mccabe                    0.7.0
mdurl                     0.1.2
mistune                   2.0.5
nbclassic                 0.5.5
nbclient                  0.7.3
nbconvert                 7.3.1
nbformat                  5.8.0
nest-asyncio              1.5.6
notebook                  6.5.4
notebook_shim             0.2.3
packaging                 23.1
pandocfilters             1.5.0
paramiko                  3.1.0
parso                     0.8.3
pickleshare               0.7.5
Pint                      0.20.1
pip                       23.1.2
pkg-about                 1.0.8
platformdirs              3.2.0
pluggy                    1.0.0
prometheus-client         0.16.0
prompt-toolkit            3.0.38
psutil                    5.9.5
pure-eval                 0.2.2
pycparser                 2.21
pyfrc                     2023.0.1
pyfu-usb                  1.0.2
Pygments                  2.15.1
pylint                    2.17.2
PyNaCl                    1.5.0
pynetconsole              2.0.4
pyntcore                  2023.4.3.0
pyrsistent                0.19.3
pytest                    7.3.1
pytest-reraise            2.1.2
python-dateutil           2.8.2
python-json-logger        2.0.7
pyusb                     1.2.1
pywin32                   306
pywinpty                  2.0.10
PyYAML                    6.0
pyzmq                     25.0.2
qtconsole                 5.4.2
QtPy                      2.3.1
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rich                      13.6.0
robotpy                   2023.4.3.0
robotpy-apriltag          2023.4.3.0
robotpy-commands-v2       2023.4.3.0
robotpy-cscore            2023.4.3.0
robotpy-ctre              2023.1.0
robotpy-hal               2023.4.3.0
robotpy-halsim-ds-socket  2023.4.3.0
robotpy-halsim-gui        2023.4.3.0
robotpy-halsim-ws         2023.4.3.0
robotpy-installer         2023.0.3
robotpy-navx              2023.0.3
robotpy-pathplannerlib    2023.3.4.1
robotpy-photonvision      2023.4.2
robotpy-playingwithfusion 2023.1.0
robotpy-rev               2023.1.3.2
robotpy-wpilib-utilities  2023.1.0
robotpy-wpimath           2023.4.3.0
robotpy-wpinet            2023.4.3.0
robotpy-wpiutil           2023.4.3.0
Send2Trash                1.8.0
setuptools                65.5.0
six                       1.16.0
sniffio                   1.3.0
soupsieve                 2.4.1
stack-data                0.6.2
terminado                 0.17.1
tinycss2                  1.2.1
tomli                     2.0.1
tomlkit                   0.11.7
tornado                   6.3.1
traitlets                 5.9.0
uri-template              1.2.0
usb                       0.0.83.dev0
wcwidth                   0.2.6
webcolors                 1.13
webencodings              0.5.1
websocket-client          1.5.1
widgetsnbextension        4.0.7
wpilib                    2023.4.3.0
wrapt                     1.15.0
zipp                      3.17.0

Reproducible example code

in robot_init:

wpilib.getDeployDirectory()

Move examples to root directory

The only subprojects that have examples are cscore and ntcore, I think.

Adding Auto-Wi-fi Switching When Deploying Robot Code

Hi guys, I realize that this is not at all a bug or an issue with RobotPy, but would it be possible to automatically switch to the robot network when running the deploy command? I found it to be a mild annoyance to have to switch all the time and I think it would be cool to have the program automatically take care of that for you. I would love to add this feature myself, could I get some guidance on where to start?

Missing struct DataLogEntries

We have NetworkTables entries that spit out structs, but not DataLog entries like C++/Java has.

https://robotpy.readthedocs.io/projects/robotpy/en/stable/wpiutil.log.html

https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/util/datalog/package-summary.html

[BUG]: getDeployDirectory incompatible with simulation

Problem description

Hello,

When testing simulation on our base code, a teammate and I had an issue in which pathplanner went to the wrong directory for paths due to the way it is traced (sorry I haven't written these before). This issue is pretty similar to #31 , and at the end, y'all responded saying you're open to suggestions. My humble idea is to instead use the getOperatingDirectory function (https://robotpy.readthedocs.io/projects/robotpy/en/stable/wpilib/functions.html#wpilib.getOperatingDirectory), which returns a usable output on both a regular computer and the roboRIO as depicted in the docs. Unless I'm missing something, this should bridge the gap between the simulator and real robot so that there's no confusion.

Our temporary solution was to move the auto paths into .venv/bin so that the path is found, but obviously that isn't very good.

Link to our code: https://github.com/1757WestwoodRobotics/RobotBase

The specific section (robotcontainer line ~75):

pathsPath = os.path.join(wpilib.getDeployDirectory(), "pathplanner", "autos") #as of right now, it is necessary to move the deploy folder into .venv/scripts or .venv/bin
        for file in os.listdir(pathsPath):
            relevantName = file.split(".")[0]
            auton = PathPlannerAuto(relevantName)
            wpilib.SmartDashboard.putData(f"autos/{relevantName}", auton)
            self.chooser.addOption(relevantName, auton)

Error message:

FileNotFoundError: [Errno 2] No such file or directory: '/home/landon/code/frc/robotcode/RobotBase/.venv/bin/deploy/pathplanner/autos/New Auto.auto'
#real path should be ..../RobotBase/deploy/pathplanner/autos/New Auto.auto

According to the getDeployDirectory docs, when simulating it will instead run dirname(robot.py)/deplay, however according to this behavior it either isn't doing that or isn't detecting simulation in some way (maybe due to the new way it runs with python -m robotpy sim instead of python robot.py sim?)

Thanks for your help, and feel free to ask for any questions or greater clarification

Operating System

Linux

Installed Python Packages

astroid                  3.0.2
bcrypt                   4.1.2
black                    23.12.1
cffi                     1.16.0
click                    8.1.7
cryptography             41.0.7
dill                     0.3.7
exceptiongroup           1.2.0
iniconfig                2.0.0
isort                    5.13.2
mccabe                   0.7.0
mypy-extensions          1.0.0
packaging                23.2
paramiko                 3.4.0
pathspec                 0.12.1
phoenix6                 24.0.0b5
photonlibpy              2024.1.1b42
Pint                     0.23
pip                      22.0.2
platformdirs             4.1.0
pluggy                   1.3.0
pycparser                2.21
pyfrc                    2024.0.0b3
pylint                   3.0.3
PyNaCl                   1.5.0
pynetconsole             2.0.4
pyntcore                 2024.0.0b4
pytest                   7.4.4
pytest-reraise           2.1.2
robotpy                  2024.0.0b4
robotpy-apriltag         2024.0.0b4
robotpy-cli              2024.0.0
robotpy-commands-v2      2024.0.0b3
robotpy-cscore           2024.0.0b4
robotpy-ctre             2024.0.0b4
robotpy-hal              2024.0.0b4
robotpy-halsim-ds-socket 2024.0.0b4
robotpy-halsim-gui       2024.0.0b4
robotpy-halsim-ws        2024.0.0b4
robotpy-installer        2024.0.2
robotpy-navx             2024.0.0b1
robotpy-pathplannerlib   2024.0.0b6.post3
robotpy-rev              2024.0.0b1.post1
robotpy-wpilib-utilities 2024.0.0b1
robotpy-wpimath          2024.0.0b4
robotpy-wpinet           2024.0.0b4
robotpy-wpiutil          2024.0.0b4
setuptools               69.0.3
tomli                    2.0.1
tomlkit                  0.12.3
typing_extensions        4.9.0
wpilib                   2024.0.0b4

Reproducible example code

Sorry it's not minimum but using https://github.com/1757WestwoodRobotics/RobotBase will work