alescontrela / amp_for_hardware Goto Github PK
View Code? Open in Web Editor NEWCode for "Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions"
License: Other
Code for "Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions"
License: Other
Describe the bug
There is something wrong when I run the train.py.
To Reproduce
Steps to reproduce the behavior:
./legged_gym/scripts/train.py task=a1_amp
Expected behavior
I want to run the demo of train.py.
System (please complete the following information):
Additional context
Add any other context about the problem here.
Hello !
I'm trying to use AMP_for_hardware to learn a nice walk on a small bipedal robot. I have produced motion example data in the same format as you used here.
I tried running trainings with pretty much the same hyperparameters as the ones in your example a1_amp_config.py
. I had to tweak some things to match the size and weight of my own robot, but not a lot.
To validate that there is no bug in my adaptations, I first train with only a walking forward example motion and only positive x linear velocity command and no noise/randomization. The robot progressively learns to move forward a little bit, but by shaking its feet and making small jumps, not imitating the walking example motion. Here is an example (after 1000 steps of 8000 envs)
And here is what the reference motion looks like (using your replay_amp_data.py
script)
The training curves look like this :
I am able to run your a1_amp
environment with the provided checkpoint and it runs great, it's very fun ton control it with a joystick :)
My questions are :
a1_amp_config.py
to train the provided policy ? Meaning only the velocity tracking rewards and amp_reward_coef = 2.0
?Thank you very much !
Hi Alescontrela,
Thanks for your great job on amp_for_hardware!
I can train it on a1, it is awesome. I have a question that I find that your mocap data is txt file and issacgym offical data, ase/amp mocap data is npy file. I want to try humanoid motion data on your repo. could you tell me how to translate that data?
I noticed that your code AMP_LOADER has 'reorder_from_pybullet_to_isaac', is that translated by pybullet?
Thanks!
Describe the bug
There is no logs file.
To Reproduce
Steps to reproduce the behavior:
python legged_gym/scripts/play.py --task=a1_amp
Hi @Alescontrela I'm trying to reproduce the hopturn result using the provided hopturn dataset, I'm wondering what's your training setting for the hopturn task, if possible can you share the config used for the hopturn task?
I want to use amp training to walk on complex terrain. Have you tried it before? What parameters do you think have a significant impact on terrain adaptability
Hi,
Thank you for sharing this amazing work.
I trained a policy with the go2 robot using the retarget_scripts branch, but when I tried to load the policy with play_real.py in pybullet, I received the following message:
"except robot_config.SafetyError as e:
AttributeError: module 'legged_gym.envs.go2_robot.robot_config' has no attribute 'SafetyError'
What could I be missing?
Thank you
Describe the bug
It seems like when I replace the robot with 'go1', the limbs become detached from the body. Additionally, in the final learned behavior of 'go1', the front legs move, but the hind legs only spread apart to the sides without any movement. Have you encountered such an issue before, and how should it be resolved?
Expected behavior
A clear and concise description of what you expected to happen.
System (please complete the following information):
Additional context
Add any other context about the problem here.
Describe the bug
Hi, the problem I meet now is the low speed for training. I remember you presented a result that the training was finished in 20 minutes, while executing the program by myself costed about 9-10 days. Do you know what happened with it? And what were commands you used in the terminal when you operated the program?
To Reproduce
Steps to reproduce the behavior:
Expected behavior
I hope I can reproduce your result that spending about 20minutes finishes the train. Do you know what happened with it that the training speed is too slow? And what were commands you input to the terminal when you operated the program?
System (please complete the following information):
Describe the bug
I check the website https://sites.google.com/berkeley.edu/amp-in-real/home experiment result, from the result "Informal race between Fu et al.* (left) and AMP Policy (right)", we could see A1 could run at a very high speed, but I check the trot0 and trot1 data from datasets file, it seems these data are not related to race, if yes, could you provide motion data related to race experiment? Thanks!
To Reproduce
Steps to reproduce the behavior:
./path/script arg1 arg2
Expected behavior
A clear and concise description of what you expected to happen.
System (please complete the following information):
Additional context
Add any other context about the problem here.
Awesome work!
I am curious as to how you format/convert the mocap data to the specific dataset format in your dataset files. If I want to train A1 with other mocap data, how to get the motion into matching/similar to your format?
(Edit: didn't mean to add "bug" label, just a question. I apologize.)
Describe the bug
It worked fine with CPU but when I switched to GPU, the following error shown many times
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA
to enable device-side assertions.
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
System (please complete the following information):
Additional Notice
The following showing is the whole output:
python3 legged_gym/scripts/train.py --task=a1_amp
Importing module 'gym_38' (/home/tianhu/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/tianhu/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 2.0.0+cu118
Device count 1
/home/tianhu/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/tianhu/.cache/torch_extensions/py38_cu118 as PyTorch extensions root...
Emitting ninja build file /home/tianhu/.cache/torch_extensions/py38_cu118/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Setting seed: 1
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
/home/tianhu/anaconda3/envs/amp_hw/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Loaded 2.499s. motion from datasets/mocap_motions/rightturn0.txt.
Loaded 10.311s. motion from datasets/mocap_motions/pace1.txt.
Loaded 0.798s. motion from datasets/mocap_motions/pace0.txt.
Loaded 0.672s. motion from datasets/mocap_motions/trot1.txt.
Loaded 0.672s. motion from datasets/mocap_motions/trot0.txt.
Loaded 0.9450000000000001s. motion from datasets/mocap_motions/leftturn0.txt.
AMPOnPolicyRunner
Actor MLP: Sequential(
(0): Linear(in_features=42, out_features=512, bias=True)
(1): ELU(alpha=1.0)
(2): Linear(in_features=512, out_features=256, bias=True)
(3): ELU(alpha=1.0)
(4): Linear(in_features=256, out_features=128, bias=True)
(5): ELU(alpha=1.0)
(6): Linear(in_features=128, out_features=12, bias=True)
)
Critic MLP: Sequential(
(0): Linear(in_features=48, out_features=512, bias=True)
(1): ELU(alpha=1.0)
(2): Linear(in_features=512, out_features=256, bias=True)
(3): ELU(alpha=1.0)
(4): Linear(in_features=256, out_features=128, bias=True)
(5): ELU(alpha=1.0)
(6): Linear(in_features=128, out_features=1, bias=True)
)
Loaded 2.499s. motion from datasets/mocap_motions/rightturn0.txt.
Loaded 10.311s. motion from datasets/mocap_motions/pace1.txt.
Loaded 0.798s. motion from datasets/mocap_motions/pace0.txt.
Loaded 0.672s. motion from datasets/mocap_motions/trot1.txt.
Loaded 0.672s. motion from datasets/mocap_motions/trot0.txt.
Loaded 0.9450000000000001s. motion from datasets/mocap_motions/leftturn0.txt.
Preloading 2000000 transitions
Finished preloading
PxgCudaDeviceMemoryAllocator fail to allocate memory 339738624 bytes!! Result = 2
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 4210
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 3480
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 3535
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 6137
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 991
Traceback (most recent call last):
File "legged_gym/scripts/train.py", line 47, in
train(args)
File "legged_gym/scripts/train.py", line 42, in train
ppo_runner, train_cfg = task_registry.make_alg_runner(env=env, name=args.task, args=args)
File "/home/tianhu/AMP_for_hardware/legged_gym/utils/task_registry.py", line 149, in make_alg_runner
runner = runner_class(env, train_cfg_dict, log_dir, device=args.rl_device)
File "/home/tianhu/AMP_for_hardware/rsl_rl/rsl_rl/runners/amp_on_policy_runner.py", line 104, in init
_, _ = self.env.reset()
File "/home/tianhu/AMP_for_hardware/legged_gym/envs/base/legged_robot.py", line 99, in reset
obs, privileged_obs, _, _, _, _, _ = self.step(torch.zeros(self.num_envs, self.num_actions, device=self.device, requires_grad=False))
File "/home/tianhu/AMP_for_hardware/legged_gym/envs/base/legged_robot.py", line 113, in step
self.torques = self._compute_torques(self.actions).view(self.torques.shape)
File "/home/tianhu/AMP_for_hardware/legged_gym/envs/base/legged_robot.py", line 431, in _compute_torques
actions_scaled = actions * self.cfg.control.action_scale
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA
to enable device-side assertions.
Hi,
Thank you very much for sharing your work! it is awesome :)
I want to work with my quadruped robot. However, after many attempts and changes in the configurations, I still don’t know what's going wrong.
Please can you give me an idea of what's wrong with my settings to make my quadruped robot behave so strangely from the start, as you can see in the video below?
I generated the reference motions using this repository for my robot configuration, and it was looking fine.
I’ve created a robot class for this robot and modified some parameters to my robot config:
class QUADAMPCfg( LeggedRobotCfg ):
class env( LeggedRobotCfg.env ):
num_envs = 5480
include_history_steps = None # Number of steps of history to include.
num_observations = 42
num_privileged_obs = 48
reference_state_initialization = True
reference_state_initialization_prob = 0.85
amp_motion_files = MOTION_FILES
get_commands_from_joystick = False
class init_state( LeggedRobotCfg.init_state ):
pos = [0.0, 0.0, 0.3] # x,y,z [m]
default_joint_angles = { # = target angles [rad] when action = 0.0
'fl_abad': 0.126, # [rad]
'fl_shoulder': 0.61, # [rad]
'fl_knee': -1.22 , # [rad]
'fr_abad': -0.126, # [rad]
'fr_shoulder': 0.61, # [rad]
'fr_knee': -1.22, # [rad]
'bl_abad': 0.126, # [rad]
'bl_shoulder': 0.689,
'bl_knee': -1.22,
'br_abad': -0.126, # [rad]
'br_shoulder': 0.61, # [rad]
'br_knee': -1.22, # [rad]
}
class control( LeggedRobotCfg.control ):
# PD Drive parameters:
control_type = 'P'
stiffness = {'': 16.} # [N*m/rad]
damping = {'': 0.35} # [N*m*s/rad]
# action scale: target angle = actionScale * action + defaultAngle
action_scale = 0.25
# decimation: Number of control action updates @ sim DT per policy DT
decimation = 6
class terrain( LeggedRobotCfg.terrain ):
mesh_type = 'plane'
measure_heights = False
class asset( LeggedRobotCfg.asset ):
file = '{LEGGED_GYM_ROOT_DIR}/resources/robots/QuadB12/quad.urdf'
foot_name = "contact"
penalize_contacts_on = ["HFE", "KFE"]
terminate_after_contacts_on = [
"base_link", "HFE", "KFE"]
self_collisions = 0 # 1 to disable, 0 to enable...bitwise filter
class domain_rand:
randomize_friction = True
friction_range = [0.25, 1.75]
randomize_base_mass = True
added_mass_range = [-1., 1.]
push_robots = True
push_interval_s = 15
max_push_vel_xy = 1.0
randomize_gains = True
stiffness_multiplier_range = [0.9, 1.1]
damping_multiplier_range = [0.9, 1.1]
class noise:
add_noise = True
noise_level = 1.0 # scales other values
class noise_scales:
dof_pos = 0.03
dof_vel = 1.5
lin_vel = 0.1
ang_vel = 0.3
gravity = 0.05
height_measurements = 0.1
class sim:
dt = 0.005
substeps = 1
gravity = [0., 0. ,-9.81] # [m/s^2]
up_axis = 1 # 0 is y, 1 is z
class physx:
num_threads = 10
solver_type = 1 # 0: pgs, 1: tgs
num_position_iterations = 4
num_velocity_iterations = 0
contact_offset = 0.01 # [m]
rest_offset = 0.0 # [m]
bounce_threshold_velocity = 0.5 #0.5 [m/s]
max_depenetration_velocity = 1.0
max_gpu_contact_pairs = 2**23 #2**24 -> needed for 8000 envs and more
default_buffer_size_multiplier = 5
contact_collection = 2 # 0: never, 1: last sub-step, 2: all sub-steps (default=2)
class rewards( LeggedRobotCfg.rewards ):
soft_dof_pos_limit = 0.9
base_height_target = 0.20
class scales( LeggedRobotCfg.rewards.scales ):
termination = 0.0
tracking_lin_vel = 1.5 * 1. / (.005 * 6)
tracking_ang_vel = 0.5 * 1. / (.005 * 6)
lin_vel_z = 0.0
ang_vel_xy = 0.0
orientation = 0.0
torques = 0.0
dof_vel = 0.0
dof_acc = 0.0
base_height = 0.0
feet_air_time = 0.0
collision = 0.0
feet_stumble = 0.0
action_rate = 0.0
stand_still = 0.0
dof_pos_limits = 0.0
class commands:
curriculum = True
max_curriculum = 1.
num_commands = 4 # default: lin_vel_x, lin_vel_y, ang_vel_yaw, heading (in heading mode ang_vel_yaw is recomputed from heading error)
resampling_time = 10. # time before command are changed[s]
heading_command = False # if true: compute ang vel command from heading error
class ranges:
lin_vel_x = [-1.0, 2.0] # min max [m/s]
lin_vel_y = [-0.3, 0.3] # min max [m/s]
ang_vel_yaw = [-1.57, 1.57] # min max [rad/s]
heading = [-3.14, 3.14]
class QUADAMPCfgPPO( LeggedRobotCfgPPO ):
runner_class_name = 'AMPOnPolicyRunner'
class algorithm( LeggedRobotCfgPPO.algorithm ):
entropy_coef = 0.01
amp_replay_buffer_size = 1000000
num_learning_epochs = 5
num_mini_batches = 4
class runner( LeggedRobotCfgPPO.runner ):
run_name = ''
experiment_name = 'QuadB12_amp_'
algorithm_class_name = 'AMPPPO'
policy_class_name = 'ActorCritic'
max_iterations = 500000 # number of policy updates
amp_reward_coef = 2.0
amp_motion_files = MOTION_FILES
amp_num_preload_transitions = 2000000
amp_task_reward_lerp = 0.3
amp_discr_hidden_dims = [1024, 512]
min_normalized_std = [0.05, 0.02, 0.05] * 4
I also checked if the body names were read correctly:
body_names ['base_link', 'bl_HAA_link', 'bl_HFE_link', 'bl_KFE_link', 'bl_contact', 'br_HAA_link', 'br_HFE_link', 'br_KFE_link', 'br_contact', 'fl_HAA_link', 'fl_HFE_link', 'fl_KFE_link', 'fl_contact', 'fr_HAA_link', 'fr_HFE_link', 'fr_KFE_link', 'fr_contact']
Termination contact names: ['base_link', 'bl_HFE_link', 'br_HFE_link', 'fl_HFE_link', 'fr_HFE_link', 'bl_KFE_link', 'br_KFE_link', 'fl_KFE_link', 'fr_KFE_link']
In addition, I made sure that my robot is with the right orientations, and I have simplified mash as you can see in the picture:
Here is the picture after more than 1k interactions, looks like it is learning not to move.
Thank you in advance!
i just followed readme.md
when running train.py or play.py:
Traceback (most recent call last):
File "../AMP_for_hardware-main/legged_gym/scripts/play.py", line 126, in
play(args)
File "../AMP_for_hardware-main/legged_gym/scripts/play.py", line 58, in play
env, _ = task_registry.make_env(name=args.task, args=args, env_cfg=env_cfg)
File "../AMP_for_hardware-main/legged_gym/utils/task_registry.py", line 97, in make_env
env = task_class( cfg=env_cfg,
File "../AMP_for_hardware-main/legged_gym/envs/base/legged_robot.py", line 90, in init
self.amp_loader = AMPLoader(motion_files=self.cfg.env.amp_motion_files, device=self.device, time_between_frames=self.dt)
File "../AMP_for_hardware-main/rsl_rl/rsl_rl/datasets/motion_loader.py", line 131, in init
self.all_trajectories_full = torch.vstack(self.trajectories_full)
RuntimeError: vstack expects a non-empty TensorList
hi, everyone. I use this framework to train my robot. I got a nice policy in running and verity in mujoco then validate in real robot. For walking works in issac and mujoco but perform terrible in real robot
But I found some questions.
I try walk, run and jump, they all works in mojuco, buy their gait frequency are much faster than the mocap data I give, maybe double or triple.
even though the lin_vel and angular reward are much bigger than others, but I can just control the speed a little or just doesn't work at all, same for the direction.
You can go through my video, it shows the robot executable its skill in a very fast frequency
Describe the bug
'ValueError: Expected parameter loc (Tensor of shape (32880, 12)) of distribution Normal(loc: torch.Size([32880, 12]), scale: torch.Size([32880, 12])) to satisfy the constraint Real(), but found invalid values:
tensor([[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]], device='cuda:0',
grad_fn=)`
To Reproduce
Execute python legged_gym/scripts/train.py --task=a1_amp
This error occurs.
Expected behavior
The normal record and no bugs.
System (please complete the following information):
Commit: https://github.com/Alescontrela/AMP_for_hardware/commit/8f3b9caa6376b81b36d5242c45a297f79ee5348c
OS: Ubuntu 18.04
GPU: GeForce GTX 1080 Ti
CUDA: 11.1
GPU Driver: 510.47.03
This looks great. Would you mind sharing the deployment code. This will help me deploy the policy on a Unitree A1 robot. Many thanks
Hello:
Your work is interesting!
How to obtain more motions data (A1 robot)?
The file "dog_clips_info.txt" included many task names, but the file "mocap_motions_a1/" only included several tasks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.