dmklee/nuro-arm

Low-cost platform to explore concepts in robotic manipulation

nuro-arm's Issues

create simpler installation procedure

There is probably a way to create an .exe file or a single bash script that will install everything. Even if it does not work everytime, it will make things much easier to get students up and running without having to enter lots of commands or switch to different terminals.

Long term, it might be good to look at docker..

improving gripper urdf

The xArm gripper is a four bar linkage. Currently it is approximated by a prismatic joint, since this accurately models collisions for the two extreme positions: fully open and fully closed. It is possible to use constraints in pybullet to model the actual behavior which should improve the modeling accuracy.

Add support for continuous motor mode

The Hiwonder motors are able to run in continuous or servo mode. Currently they are in servo mode allowing for positional control in a range of 4 radians.
The benefits of continuous mode are:

• larger range of joint positions can be reached
• ability to fine-tune the feedback to potentially get better accuracy
• as an educational tool to explore feedback
• more easily allow for complex movements in joint space

improving error statements

Currently there are a lot of situations where an error occurs and it is quite confusing to a new user. Even if I have a custom error function with an interpretable message, the addition of the traceback causes it to get lost.

• Add more custom exceptions
• When errors occur, catch them and print only a short, interpretable explanation to the terminal, perhaps with a pointer to where in the installation guide they can go to address to issue
• Additional details that may be useful for an experienced user or developper should be sent to a log file

Known errors:

• Cable is not connected between some servos ( this currently causes an indexerror on xarm_controller._read_servo_offset())
• Device is not connected. This means either hidapi is not installed or something isnt plugged in correctly
• ServoOffsetError, this is already a custom exception but needs to be explained better
• No config file
• Divide by zero error occurs later with bad calibration of grippers. This should be caught during calibration and the user prompted to try calibrating again
• ModuleNotFoundError: we cannot really fix this but it might be good to add to FAQ, this means that they need to run pip install
• [Windows] hid.HIDException('unable to open device') => it is not plugged in
• [Windows] in XArmController.connect ImportErrror on (device=Device()) this means that hidapi.dll was not placed in proper location
• Camera connection: capture object has no thread

If we can do a log file, then ask for students to submit it. That would be a great way to identify potential bugs that occur and improve the error statement interpretability

improve the imports

I think it would be better to structure the repo such that the user can import like import neu_ro_arm as neuro; neuro.RobotArm.

This can be done with init files although i need to work out the details

Providing more low level control to xarm

• Ability to power on/off individual servos
• Ability to send movement speed as float
• Support for movements non-linear in joint space

improving simulator

Feature requests:

• headless simulator for fast RL stuff
• motion planning can be run with simulator client (with pybullet>3.0.9, collision detection doesnt require moving robot) to reduce number of threads
• better stability with non realtime mode
• realistic collisions/interactions with cubes in the scene

Improve accuracy of Aruco Tag pose detection

The pose detection for aruco tags currently suffers from two problems.

1. There is ambiguity about the normal direction of the tag since we are estimating 3D pose from coplanar points. This a problem that is inherent to using aruco tags and there are some potential solutions such as filtering or adding more tags to an object.
2. The depth of the tag in the scene is not accurate. I believe this is becausne the tag is so small and the camera so low resolution. The depth is based on the estimated dimensions of the tag in the image. This could be improved by using larger tags, by upgrading the camera, improving the camera intrinsics/extrinsics accuracy, or some heuristic for binding the cubes to above ground

random fixes that i notices while making documentation

• put gripper opened and gripper closed in RobotArm class
• fix RobotArm.move_hand_to. need to improve the handling of IK errors or collision errors. and have better control over the rotation, i think the pitch-roll stuff should be in another function
• robotarm init should have headless as optional and assume based on mode. the user can force override this by providing boolean
• add print statement to robot arm init if not calibrated that user should run calibration command; what is the clean way to run calibration scripts
• remove opencv dependencies unless camera is used

Gripper State bug

The robot arm handles gripper state inconsistently. Calling RobotArm.get_gripper_state returns a linear interpolation from 0 to 1 but does not force it to be from 0 to 1. However, calling RobotArm.set_gripper_state raises an assertion error for a gripper state outside of [0,1].

The appropriate solution is to clip values in both cases so that gripper state is always in [0,1] but no errors occur if it is outside the range.

Support for multiple xarms

It would be cool to support sending commands to multiple xArms at once. I think the only change required would be to provide an argument or dialog box to select among possible hid devices found. I am not sure if collision detection, as it is currently formulated, would be possible with two robots; they would have to both exist within a single simulator and the motions would have to be aligned in time

Add text entry to scripts/record_movements.py

This was suggested by a student and I think would add a lot of functionality. Might need to add button for this (like Custom Row), or it can be a click option somewhat like you would change an Excel sheet

Another nice addition would be the ability to customize movement speed and add delays

Properly storing configs files

improve move arm gui

The caution statement should be removed as it seems too scary.

The joint limits should be further restricted (specifically the base joint)

The speed limits should be tuned so it doesn't move too quickly even if scales are spammed

There should be the ability to toggle between passive to active mode. In active mode, the sliders move the joints, in passive mode the sliders reflect the observed positions

Maybe add xyz end effector position printout

Servo Offsets are reset to 0 when robot is turned off

I noticed that the servo offsets are reset when power is lost to the system. The xarm controller is currently set up under the assumption that the offsets are stored. This is obviously inconvenient because it forces the user to recalibrate every time it is powered on. Moreover, this results in a potentially unsafe movement during power up since the arm may exist in a position outside of the servo range (because position is in integer, it causes wrapping so the arm moves rapidly to the opposite end of the joint space).

Fixes to make:

• Load servo offsets from config file within XArmController.__init__
• Ensure XArmController.read_command clips the positional reading to [0,1000]

Generate Docs with Sphinx

Calibration scripts should use pop ups as input

Currently, setup_xarm and setup_camera use terminal input. I think this is a bit confusing to users who are not familiar with Python. It seems better to switch to using a small pop up window with tkinter where we can optionally show photos.

Along with this change, we should improve the error messages produced and the ability to handle closing the window prematurely

improve motion command accuracy

The monitor function is currently not tuned for the robot, resulting in the monitor function terminated early or taking a long time.

HID issues on Windows/Mac with serial number

In commit b2f495f, there was code introduced to specify serial number while connecting to the xArm (to allow the user to control multiple arms at once). However, the functionality for Mac and Windows was not tested and appears to fail.

Either remove support for multiple arms or test that it works on these operating systems