A MuJoCo/Gym environment for robot control using Reinforcement Learning. The task of agents in this environment is pixel-wise prediction of grasp success chances.
License: MIT License
I'm a Data Scientist / ML Engineer looking to effect positive change.
๐จ Skills:
๐ My GitHub Stats:
โ๏ธ Connect with me:
Encountered the following error while running the example_agent.py file, How to provide the observation space while running the script?
Hi Thank you for sharing your grateful project!
I want to validate performance of the weight through reinforcement learning on simulator with real UR Robot.
test.py with weight earned by simulator?test.py, Is it right that the work connecting ros and moveit should proceed ahead?I want apply MujocoController with kinova jaco2, then got trouble in open_griper(). robot run wrong.In the function move_group_to_joint_target() in MujocoController, there is temp = self.sim.data.body_xpos[self.model.body_name2id('ee_link')] - [0, -0.005, 0.16]. so i want to know what does [0, -0.005, 0.16] mean to UR5?
Hello, I run the command: python Grasping_Agent.py, in my conda virtual env(python3.6), facing an issue as below:
File "/home/wu/github_resource/MuJoCo_RL_UR5/gym_grasper/envs/GraspingEnv.py", line 19, in
from decorators import debug
ImportError: cannot import name 'debug'
my environment should be ok, ubuntu18.04, mujoco_py2.0, gym, and other ones required, like your requirements.txt
and the decorators module has been installed.
I search some messages by google, but useful ones are few.
so, how can I do to figure out this issue.
could you give me some suggestions, thank you~~
While running "example.py" , it throws error as in title
Hi all, i have two errors, when i used codes. How can i solve them. Thx for informations.
python3 example_agent.py
.
.
.
EPISODE 1 STEP 1
#################################################################
Traceback (most recent call last):
File "example_agent.py", line 27, in
observation, reward, done, _ = env.step(action, record_grasps=True)
TypeError: step() got an unexpected keyword argument 'record_grasps'
Then i tried grasping agent code and ..
python3 Grasping_Agent_multidiscrete.py
Creating window glfw
Created new control group 'Arm'.
Created new control group 'Gripper'.
CURRENT EPSILON: 1.0
#################################################################
EPISODE 1 STEP 1
#################################################################
Traceback (most recent call last):
File "Grasping_Agent_multidiscrete.py", line 583, in
main()
File "Grasping_Agent_multidiscrete.py", line 545, in main
next_state, reward, done, _ = agent.env.step(
TypeError: step() got an unexpected keyword argument 'action_info'
Hi! Thanks for your grear work!
It seems that the gripper can not grasp anything during training. Can you tell me what's wrong with it?
Thanks!
Hi, In MuJoCo_RL_UR5, inappropriate dependency versioning constraints can cause risks.
Below are the dependencies and version constraints that the project is using
mujoco_py
numpy
simple_pid
termcolor
ikpy
pyquaternion
opencv-python
matplotlib
gym
The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict.
The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.
After further analysis, in this project,
The version constraint of dependency numpy can be changed to >=1.8.0,<=1.23.0rc3.
The above modification suggestions can reduce the dependency conflicts as much as possible,
and introduce the latest version as much as possible without calling Error in the projects.
The invocation of the current project includes all the following methods.
numpy.linalg.inv
numpy.round agent.optimizer.state_dict numpy.min self.viewer.render self.controller.pixel_2_world self.policy_net.load_state_dict self.standardize_rgb max_value.item collections.defaultdict numpy.concatenate depth_list.append depth_tensor.detach.clone self.optimizer.zero_grad torch.optim.Adam abs self.controller.depth_2_meters device.state.to.self.policy_net.view torchvision.transforms.Normalize Grasp_Agent.learn fig.add_subplot.set_ylabel str parameter.numel pixel.np.round.astype AverageMeter.update Modules.MULTIDISCRETE_RESNET.to.eval self.ee_chain.inverse_kinematics model.named_parameters torch.optim.Adam.zero_grad self.bn3 collections.defaultdict.append targets.positive_predection.int data.unsqueeze collections.deque numpy.sqrt self.standardize_depth matplotlib.pyplot.subplots_adjust numpy.array.flatten self.ee_chain.forward_kinematics gym_grasper.controller.MujocoController.MJ_Controller.move_ee self.add_marker self.UP2 targets.sum.item gym.spaces.MultiDiscrete agent.env.step Grasping_Agent_multidiscrete.Grasp_Agent.update_tensorboard mujoco_py.load_model_from_path agent.writer.close kwargs.items open max_idx.view.view self.standardize_depth.detach conv2d_size_out Grasping_Module_multidiscrete grad.view.cpu pickle.loads self.policy_net.state_dict self.render torch.load.keys self.last_1000_rewards.append collections.defaultdict.items self.model.actuator_id2name idx_list.append self.bn2 gym_grasper.controller.MujocoController.MJ_Controller.toss_it_from_the_ellbow gym.make.reset self.plot_list.append torch.nn.Sequential controller self.writer.add_scalars sys.path.insert math.radians pyquaternion.Quaternion.random self.target_net.eval Grasping_Agent_multidiscrete.Grasp_Agent abs.dot torch.optim.Adam.step gym.utils.EzPickle.__init__ rgb_list.append Grasping_Agent_multidiscrete.Grasp_Agent.transform_action format time.strftime test torchvision.transforms.ToTensor torch.nn.functional.binary_cross_entropy.backward end_idx.start_idx.batch.action.torch.cat.to self.model.actuator_id2name.append Perception_Module torch.device self.Grasping_Module_multidiscrete.super.__init__ self.C1 self.RB1 ValueError torch.optim.Adam.load_state_dict torchvision.transforms.ColorJitter simple_pid.PID self.get_mean_std self.viewer.add_marker Modules.ReplayBuffer gym.envs.mujoco.mujoco_env.MujocoEnv.close torch.randn_like self.rotate_wrist_3_joint_to_value self.env.controller.pixel_2_world fmtstr.format prettytable.PrettyTable numpy.flipud torch.utils.tensorboard.SummaryWriter.add_scalar rewards.append torch.zeros numpy.fliplr Transition pickle.dump fig.add_subplot.plot self.controller.set_group_joint_target self.controller.stay self.writer.add_scalar cv2.imwrite torch.optim.SGD copy.deepcopy policy_net self.BasicBlock.super.__init__ torch.nn.functional.relu inspect.signature axis.yaxis.set_label_coords torch.nn.Linear matplotlib.pyplot.clf zip numpy.linalg.inv self.C1.size self.fc1 self.controller.move_group_to_joint_target self.model.camera_name2id func norm_layer termcolor.colored self.normal_rgb_no_jitter_no_noise self.env.model.camera_name2id matplotlib.pyplot.savefig max_idx.view.unsqueeze_ conv3x3 agent.policy_net.state_dict random.randrange i.self.actuators.self.plot_list.append func.keys targets.sum int self.ee_chain.inverse_kinematics_frame range self.CONV3_FC1.super.__init__ axis.xaxis.set_label_coords torch.nn.MaxPool2d numpy.ones gym.envs.registration.register data.len.state_batch.policy_net.view.gather self.close_gripper self.n_actions_2.MULTIDISCRETE_RESNET.to self.HEIGHT.self.WIDTH.torch.zeros.to self.create_group resnet isinstance numpy.max numpy.tan state_batch.policy_net.view self.controller_list.append self.model.body_name2id Grasping_Agent_multidiscrete.Grasp_Agent.epsilon_greedy numpy.std self.controller.move_ee agent.transform_observation.to self.UP1 self.optimizer.step end_idx.start_idx.batch.state.torch.cat.to self.C1.view torch.utils.data.DataLoader states.append torch.cuda.is_available inspect.signature.bind_partial self.model.body_id2name self.memory.sample numpy.reshape cv2.imshow simple_Transition self.model.joint_id2name print self.normal_rgb max_idx.unsqueeze_.cpu sys.path.append self.controller.ik self.controller.open_gripper targets.size matplotlib.pyplot.figure self.controller.grasp self.writer.add_histogram positive_threshold.predictions.int random.seed tqdm.tqdm self.bn1 bound_values.arguments.items AverageMeter.reset random.sample.append time.localtime NotImplementedError gym.spaces.Discrete numpy.where rgb.self.standardize_rgb.float self.controller.close_gripper Modules.MULTIDISCRETE_RESNET.to.state_dict self.ik self.last_100_loss.append pathlib.Path self.conv1 cv2.waitKey self.conv3 self.create_camera_data grad.view.cpu.numpy self.reset torch.save observation.astype torch.tensor next_state_batch.self.target_net.max self.relu setuptools.setup end_idx.start_idx.batch.reward.torch.cat.to.float ljust gym_grasper.controller.MujocoController.MJ_Controller.show_model_info self.writer.add_graph data.unsqueeze.to typeassert env.action_space.sample time.time self.C1.squeeze_ self.policy_net.parameters file.read main enumerate Modules.Transition torch.utils.tensorboard.SummaryWriter prettytable.PrettyTable.add_row collections.namedtuple repr self.conv2 ikpy.chain.Chain.from_urdf_file torch.nn.Conv2d targets.int torch.cuda.manual_seed math.exp numpy.random.seed negative_threshold.predictions.int negative_positions.correct_prediction_2.sum.item self.move_and_grasp Grasping_Agent_multidiscrete.Grasp_Agent.transform_observation fig.add_subplot.set_xlabel agent.env.reset Modules.MULTIDISCRETE_RESNET.to.load_state_dict Modules.MULTIDISCRETE_RESNET Grasp_Agent.update_tensorboard gym.make self.sim.step ee_position.copy depth.torch.tensor.float self.create_joint_angle_plot self.sim.data.get_joint_qpos pyquaternion.Quaternion gym_grasper.controller.MujocoController.MJ_Controller numpy.array cv2.cvtColor self.policy_net MAX_POSSIBLE_SAMPLES.state_batch.self.policy_net.view.gather torchvision.transforms.Lambda next_state_batch.self.target_net.max.unsqueeze self.transform_observation torch.manual_seed gym.make.step self.fill_plot_list agent.env.close torch.nn.BatchNorm2d self.memory.append random.sample numpy.mean torch.utils.data.random_split matplotlib.pyplot.figure.add_subplot self.target_net.load_state_dict max actions.append agent.optimizer.zero_grad func.items torch.nn.ReLU fig.add_subplot.axhline gym.make.print_info rgb.self.normal_rgb.float grasping_dataset.Grasping_Dataset data.to targets.negative_predection.int torch.nn.functional.binary_cross_entropy self.last_1000_actions.append Modules.simple_Transition Grasp_Agent targets.correct_prediction.sum.item gym_grasper.controller.MujocoController.MJ_Controller.stay self.MP2 end_idx.start_idx.batch.next_state.torch.cat.to Modules.MULTIDISCRETE_RESNET.to.parameters self.actuators.append self.sigmoid self.create_lists self.Grasping_Module.super.__init__ type numpy.argmax Grasp_Agent.transform_action self.optimizer.load_state_dict binary_accuracy self.RB2 list torch.cat os.path.realpath rgb.self.normal_rgb_no_jitter_no_noise.float count_parameters len torchvision.transforms.Compose self.MP1 time.perf_counter mujoco_py.MjViewer self.get_observation data.unsqueeze.to.float self.sim.render random.random numpy.zeros eval_on_test_set self.Perception_Module.super.__init__ mujoco_py.MjSim numpy.expand_dims coordinates_2.np.round.np.ndarray.coordinates_2.isinstance.ljust torch.nn.Sigmoid torch.nn.UpsamplingBilinear2d inspect.signature.bind self.set_state self.plot_list.keys next_state_batch.self.target_net.max.unsqueeze.detach targets.correct_prediction.sum Grasping_Module AverageMeter agent.epsilon_greedy.item gym.make.close torch.nn.functional.binary_cross_entropy.item MULTIDISCRETE_RESNET Grasp_Agent.epsilon_greedy negative_positions.correct_prediction_2.sum gym_grasper.controller.MujocoController.MJ_Controller.grasp agent.memory.push self.target_net torch.optim.Adam.state_dict j.self.actuators torch.Tensor join cv2.destroyAllWindows fig.add_subplot.set_title self.RB3 Grasping_Agent_multidiscrete.Grasp_Agent.learn gym.envs.mujoco.mujoco_env.MujocoEnv.__init__ device.state.to.self.policy_net.view.max torch.load numpy.random.normal TypeError super state_batch.self.policy_net.view Modules.MULTIDISCRETE_RESNET.to self.controller.get_image_data depth_tensor.detach.clone.unsqueeze_ grad.view torchvision.transforms.ToPILImage self.move_group_to_joint_target self.groups.keys self.model.get_joint_qpos_addr Grasp_Agent.transform_observation end_idx.start_idx.batch.reward.torch.cat.to self.model.camera_id2name torch.no_grad functools.wraps numpy.random.uniform BasicBlock
@developer
Could please help me check this issue?
May I pull a request to fix it?
Thank you very much.
When I run the example.py file, I always prompt 'x0' is feasible. Is it a version issue with ikpy
Hi, thanks for sharing this amazing project!
I have successfully run example.py and example_agent.py. Here are two questions I currently have:
Much appreciate.
how can I deal with the problem of "could not execute all movements successfully, and did not grasp anything, and could not reach target location " when training, I am trained with the file of Grasping_Agent_multidiscrete.py,
appreciate very much.
Hey Paul,
I am working on a project where I need to convert the end-effector position (normalized from -1 to 1) from Mujoco environment to coordinate in the image rendered in Mujoco.
I am not sure how to do it. Could you help me ?
Hey Paul,
I ran into a problem when I downloaded this code for the second time, saying ' x0 is infeasible. Could not find an inverse kinematics solution. '
Could you please help me?
Thank you for sharing your project.
I want to run UR3 Reinforcement learning with Mujoco simulator.
But when I run UR3 in mujoco, It cannot solve its kinematics at all.
It's similar to controller (that ur5 controller)
How did you make ur5 controller?
Hi! When I run python example.py , I got this
x0 is infeasible.
Could not find an inverse kinematics solution.
And I try to change the position's value in controller.move_ee function, but it didn't work.
I use python3.6.9, mujoco2.1.0 , mujoco-py2.1.2.14 and ikpy3.3.
How to make the robotic arm move, only the clip is moving after training. Help!
Hi, when I test your code, a segmentation fault happended.
Could you help me?
Hi,
I just found an error message "AttributeError: 'GraspEnv' object has no attribute 'sim'" when I ran the command "python3 Grasping_Agent_multidiscrete.py".
Traceback is shown as follows:
Traceback (most recent call last):
File "Grasping_Agent_multidiscrete.py", line 583, in
main()
File "Grasping_Agent_multidiscrete.py", line 521, in main
agent = Grasp_Agent(seed=rand_seed, load_path=None, learning_rate=lr, depth_only=False)
File "Grasping_Agent_multidiscrete.py", line 85, in init
self.env = gym.make(
File "/home/steve/anaconda3/envs/mujoco_py/lib/python3.8/site-packages/gym/envs/registration.py", line 592, in make
env = env_creator(**_kwargs)
File "/home/steve/MuJoCo_RL_UR5/gym_grasper/envs/GraspingEnv.py", line 51, in init
self.controller = MJ_Controller(self.model, self.sim, self.viewer)
AttributeError: 'GraspEnv' object has no attribute 'sim'
Appreciate!
Hi, I have configured your repo, and when i run example_agent.py, the init function of gym.make() reported: the 'observation_space' must be the required argument. Could you please tell me how can i define the space? thanx~
Does anyone know how to solve this problem?
(mujoco_py) xxxx@XXXX:~/.mujoco/mujoco-py/examples/MuJoCo_RL_UR5$ python3 example_agent.py
Creating window glfw
Created new control group 'Arm'.
Created new control group 'Gripper'.
Model timestep: 0.002
Set number of frames skipped: 1
dt = timestep * frame_skip: 0.002
Frames per second = 1/dt: 500
Actionspace: MultiDiscrete([40000 6])
Observation space: Dict(rgb:Box(200, 200, 3), depth:Box(200, 200))
Found 1 GPUs for rendering. Using device 0.
Could not make EGL context current
Traceback (most recent call last):
File "example_agent.py", line 16, in
obs = env.reset()
File "/home/xxxx/anaconda3/envs/mujoco_py/lib/python3.8/site-packages/gym/envs/mujoco/mujoco_env.py", line 108, in reset
ob = self.reset_model()
File "/home/xxxx/.mujoco/mujoco-py/examples/MuJoCo_RL_UR5/gym_grasper/envs/GraspingEnv.py", line 477, in reset_model
return self.get_observation(show=self.show_observations)
File "/home/xxxx/.mujoco/mujoco-py/examples/MuJoCo_RL_UR5/gym_grasper/envs/GraspingEnv.py", line 398, in get_observation
rgb, depth = self.controller.get_image_data(
File "/home/xxxx/.mujoco/mujoco-py/examples/MuJoCo_RL_UR5/gym_grasper/controller/MujocoController.py", line 719, in get_image_data
self.sim.render(width=width, height=height, camera_name=camera, depth=True)
File "mjsim.pyx", line 156, in mujoco_py.cymj.MjSim.render
File "mjsim.pyx", line 158, in mujoco_py.cymj.MjSim.render
File "mjrendercontext.pyx", line 46, in mujoco_py.cymj.MjRenderContext.init
File "mjrendercontext.pyx", line 114, in mujoco_py.cymj.MjRenderContext._setup_opengl_context
File "opengl_context.pyx", line 130, in mujoco_py.cymj.OffscreenOpenGLContext.init
RuntimeError: Failed to initialize OpenGL
Hello
Running the example code throws out an error saying inverse kinematics is not possible for the coordinates
When I run example.py, the following error is generated.I tried to lower the ikpy version to 3.1 and still got the error.
Traceback (most recent call last):
File "C:/Users/eEthan1/PycharmProjects/RL_UR5/example.py", line 4, in
controller = MJ_Controller()
File "C:\Users\eEthan1\PycharmProjects\RL_UR5\gym_grasper\controller\MujocoController.py", line 47, in init
self.ee_chain = ikpy.chain.Chain.from_urdf_file(path + "/UR5+gripper/ur5_gripper.urdf")
AttributeError: module 'ikpy' has no attribute 'chain'
Why your joint controllers have to run repetitively until reaching their setpoints? Usually, these controllers constantly run a much higher frequency in the inner loop while their setpoints are set from an outer loop which often runs at a slower rate. Therefore, the controllers do not have to reach their setpoints before the setpoints are changed. But it seems you are not following this structure.
Hi @PaulDanielML ,
I am trying to use your simulator in a project related to imitation learning. I installed the simulator according to instructions. Then I am trying to run example_agent.py but its giving me the error mentioned below.
File "/home/himanshu/MuJoCo_RL_UR5/gym_grasper/controller/MujocoController.py", line 719, in get_image_data
self.sim.render(width=width, height=height, camera_name=camera, depth=False)
File "mjsim.pyx", line 156, in mujoco_py.cymj.MjSim.render
File "mjsim.pyx", line 158, in mujoco_py.cymj.MjSim.render
File "mjrendercontext.pyx", line 46, in mujoco_py.cymj.MjRenderContext.__init__
File "mjrendercontext.pyx", line 114, in mujoco_py.cymj.MjRenderContext._setup_opengl_context
File "opengl_context.pyx", line 130, in mujoco_py.cymj.OffscreenOpenGLContext.__init__
RuntimeError: Failed to initialize OpenGL
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google โค๏ธ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.