gemcollector / rl-vigen Goto Github PK
View Code? Open in Web Editor NEWThis is the repo of "RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalization"
License: MIT License
This is the repo of "RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalization"
License: MIT License
Thanks a lot for your wonderful work! I'm curious about the scale of image inputs in the robosuite. Do we directly input the image within [0, 255]? Or is it necessary to rescale it into [0, 1]? Thanks a lot!
Hello @gemcollector
I have been trying to reproduce SGQN and PIEG results for DMC. For SGQN, for walker walk environment in hard settings, I get the results about 380. For train settings, I get the results about 750. If we look at the paper https://arxiv.org/pdf/2209.09203.pdf in Table 2 (page 9), the results must be much much higher.
I know those papers work on DMC-GB from Hansen (you also mentioned in the paper), but I don't see the major difference in both the environments from the eval videos (especially for DMC case). Can you please try to replicate and see if you get the same results, or can you explain what creates the difference.
If you need log files or any further explanation, please let me know.
Thanks a lot for the wonderful work! When I run the code with the task in the robosuite, sometimes the code will be killed with the Error information like
RuntimeError: DataLoader worker (pid 22358) is killed by signal: Killed.
It seems that the problem is because of the workers in Dataloader. Unfortunately, I still have this problem when I set replay_buffer_num_workers=1. Have you met this problem? Thanks a lot again!
I'm trying to run carla_env, I ran bash scripts/carlatrain.sh directly and encountered the problem:
pygame 2.1.2 (SDL 2.0.16, Python 3.8.18) Hello from the pygame community. https://www.pygame.org/contribute.html workspace: /raid/fangyu_data/RL-ViGen/exp_local/2024.01.29/164236_action_repeat=2,feature_dim=50,lr=1e-4,nstep=1,num_train_frames=1001000,save_snapshot=True,save_video=False,seed=3,task=carla,use_tb=False,use_wandb=False,wandb_group= Error executing job with overrides: ['task=carla', 'seed=3', 'action_repeat=2', 'use_wandb=False', 'use_tb=False', 'num_train_frames=1001000', 'save_snapshot=True', 'save_video=False', 'lr=1e-4', 'feature_dim=50', 'nstep=1', 'wandb_group='] Traceback (most recent call last): File "carlatrain.py", line 262, in main workspace = W(cfg) File "/raid/fangyu_data/RL-ViGen/carlatrain.py", line 51, in __init__ self.setup() File "/raid/fangyu_data/RL-ViGen/carlatrain.py", line 71, in setup self.train_env = carla_make(action_repeat=self.cfg.action_repeat) File "/raid/fangyu_data/RL-ViGen/wrappers/carla_wrapper.py", line 109, in carla_make env = make_env_10(action_repeat) File "/raid/fangyu_data/RL-ViGen/./wrappers/carlaenv/utils.py", line 34, in make_env_10 env = CarlaEnv10(cfg_dict) File "/raid/fangyu_data/RL-ViGen/./wrappers/carlaenv/carla_env_10.py", line 263, in __init__ self.world = self.client.load_world(cfg_dict['map']) # change map here RuntimeError: time-out of 500000ms while waiting for the simulator, make sure the simulator is ready and connected to localhost:2018
I followed carla_README.md and wrote the code:
from carlaenv.utils import make_env_10
action_repeat = 2
env = make_env_10(action_repeat)
I have encountered the same problem. Can you please help me to solve it? Thank you
Hi, thanks for your great work. However, I have trouble finding the Dexterous Hand Manipulation
mentioned in Sec. 4.1.3. of the paper. I searched the whole repo for Adroit / color_hard, but I could not find any code related to the experiment results.
I noticed there's a separate branch called ViGen-adroit, do I have to checkout to that branch to use the generalization environment?
When training svea or pieg on the walker_walk task of the dmc environment, even if I set the seed, the results are still different each time. why is that? Looking forward to your reply.
Line 52 in 0158cec
This throws error as there is no agent variable defined.
Instead, this should be:
agent = torch.load('%s/snapshot.pt' % (work_dir), map_location='cuda:0')
self.agent = agent['agent']
self._global_step = agent['_global_step']
Btw, great work!
Hello, authors,
Thanks for your excellent work, it is really helpful for the community.
But I am confused about how can we achieve parallel training (it is actually not an issue of this repo). For example, the image size of dm_control suite is about 84*84, and when I train one exp, the used GPU memory is quite small but the GPU-Util is high. If I manually train several seeds at the same time, every training process will be slow.
So my question is that how can I achieve parallel training to accelerate the process (multiprocessing?).
I changed the locodmc_eval.sh and added save_video=True in the script. It shows error while running, specifically:
File "locodmc_eval.py", line 166, in main
workspace.eval()
File "/home/vedant/RL-ViGen/locodmc_eval.py", line 130, in eval
self.video_recorder.save(f'{self.global_frame}.mp4')
File "/home/vedant/RL-ViGen/video.py", line 40, in save
imageio.mimsave(str(path), self.frames, fps=self.fps)
File "/home/vedant/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/imageio/core/functions.py", line 408, in mimwrite
raise ValueError("Image is not numeric, but {}.".format(imt.__name__))
ValueError: Image is not numeric, but NoneType.
This arises due to the fact that all the frames during rendering from video i.e returns None.
Line 34 in 0158cec
FYI, I have done this additionally in locodmc_eval.py to enable video capture:
Line 34 in 0158cec
self.video_recorder =VideoRecorder(self.work_dir if self.cfg.save_video else None)
Am I missing here something?
I successfully trained the vrl3 model, but I have some questions during testing.
I discovered that during testing with mode video-hard
, the agent's actual input is the observation obtained from observation = time_step.observation
, as shown below:
However, neither of the two video recording functions returns something corresponding to the above image:
1: In the record_gif
function, what's actually recorded is the image of size (224, 224, 3)
rendered by self.eval_env._env._env.env.sim.render
. The image and final video are as follows:
2: In self.video_recorder.record(self.eval_env)
, what's recorded is the image of size (224, 224, 3)
returned by env.render()
. The image and final video are as follows (these are also the images provided in the paper and on the website):
I would like to know why the images presented in the paper and on the website are not those of the agent's actual testing, or what do these two videos mean separately?
Thank you for adding new environments to the visual reinforcement learning generalization. I look forward to your response.
Hello, @gemcollector,
I have noticed that you have loaded the evaluation scores of most experiments. However, the detailed evaluation scores on Robosuite with cross embodiments settings are not provded, specifically in Figure 9. Could you upload the missing results, please?
Thank you for this valuable project. I encountered an issue while trying to reproduce the Adroit environment and would appreciate your help.
I installed the environment using:
conda create -n vigen-adroit python=3.8
bash install_adroit.sh
However, when executing:
cd src/
bash train.sh
I encountered an error:
ModuleNotFoundError: No module named 'mj_envs.hand_manipulation_suite.door_shadowhand_v0'
After executing the commented-out commands in bash install_adroit.sh
:
git clone https://github.com/watchernyu/rrl-dependencies.git \
&& cd rrl-dependencies \
&& pip install -e mj_envs/. \
&& pip install -e mjrl/.
I encountered a new error:
Traceback (most recent call last):
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
return func()
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 368, in <lambda>
lambda: hydra.run(
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 110, in run
_ = ret.return_value
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/core/utils.py", line 233, in return_value
raise self._return_value
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/core/utils.py", line 160, in run_job
ret.return_value = task_function(task_cfg)
File "train_adroit_dhm.py", line 535, in main
workspace = W(cfg)
File "train_adroit_dhm.py", line 73, in __init__
self.setup()
File "train_adroit_dhm.py", line 143, in setup
self.train_env = dhm.make_env_RRL(env_name, test_image=False, num_repeats=self.cfg.action_repeat,
File "/datadrive/xxx/code/RL-ViGen/DHM/dhm/make_env_RRL.py", line 894, in make_env_RRL
env = DHMEnv(cfg_dict, env_name, test_image, cam_list, num_repeats, num_frames, env_feature_type, device,
File "/datadrive/xxx/code/RL-ViGen/DHM/dhm/make_env_RRL.py", line 703, in __init__
env = GymEnv(self.env_name)
File "/datadrive/xxx/code/RL-ViGen/DHM/dhm/utils.py", line 27, in __init__
env = gym.make(env)
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/gym/envs/registration.py", line 235, in make
return registry.make(id, **kwargs)
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/gym/envs/registration.py", line 128, in make
spec = self.spec(path)
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/gym/envs/registration.py", line 203, in spec
raise error.UnregisteredEnv("No registered env with id: {}".format(id))
gym.error.UnregisteredEnv: No registered env with id: hammer-adroit-v0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_adroit_dhm.py", line 545, in <module>
main()
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
_run_hydra(
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 367, in _run_hydra
run_and_report(
File "/home/xxx/miniconda3/envs/vigen-adroit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 251, in run_and_report
assert mdl is not None
AssertionError
I suspect it might be a version issue. Do you know how to resolve this? Looking forward to your reply, and thank you in advance.
Hi!
I got the following error when training the robosuite
env:
Traceback (most recent call last):
File "train.py", line 304, in main
workspace.train()
File "/home/xzc/Documents/RL-ViGen/train.py", line 260, in train
self.eval()
File "/home/xzc/Documents/RL-ViGen/train.py", line 168, in eval
self.video_recorder.init(self.eval_env, enabled=(episode == 0))
File "/home/xzc/Documents/RL-ViGen/video.py", line 25, in init
self.record(env)
File "/home/xzc/Documents/RL-ViGen/video.py", line 39, in record
frame = env.render()
File "/home/xzc/Documents/RL-ViGen/wrappers/dmc.py", line 183, in __getattr__
return getattr(self._env, name)
File "/home/xzc/Documents/RL-ViGen/wrappers/robo_wrapper.py", line 106, in __getattr__
return getattr(self._env, name)
File "/home/xzc/mambaforge/envs/pieg/lib/python3.8/site-packages/dm_control/suite/wrappers/action_scale.py", line 103, in __getattr__
return getattr(self._env, name)
File "/home/xzc/Documents/RL-ViGen/wrappers/dmc.py", line 67, in __getattr__
return getattr(self._env, name)
File "/home/xzc/Documents/RL-ViGen/wrappers/dmc.py", line 149, in __getattr__
return getattr(self._env, name)
AttributeError: 'Gym2DMC' object has no attribute 'render'
It seems that this error was triggered by missing __getattr_
in Gym2DMC
wrapper. Is that right?
In appendix d.1.2,
"The Easy level includes soft_noisy_low_light and soft_high_light, while the Medium level is defined by the HardRainSunset and SoftRainSunset conditions. The Hard level contains hard_low_light and hard_noisy_low_light."
Q1. For the easy level, the weather is 'soft_noisy_low_light'. However, when setting the weather in carlaenv10_eval_config.yaml to "soft_noisy_low_light', the following error occurs.
I think I might need to change the weather: soft_noisy_low_light in cfgs/setting/setting.yaml to custom_soft_noisy_low_light. Is this correct? That is,
Q2. There are two weathers each for Easy, Medium, and Hard levels. So, for something like the Score in Figure 4, do you evaluate each difficulty with num_eval_episode for both environments (e.g., HardRainSunset and SoftRainSunset for Medium) and then display the average score of the two weathers?
Hello, @gemcollector,
I'm trying to reproduce the locomotion results illustrated in Figure 18. The model is obtained by running train.py and the task is unitree stand. When I'm running the locoeval.py, the following problem is occurred:
Traceback (most recent call last):
File "locoeval.py", line 70, in evaluate
obs = env.reset()
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/suite/wrappers/action_scale.py", line 94, in reset
return self._env.reset()
File "/home/zhqin/RL_ViGen/envs/DMCVGB/dmcvgb/wrappers.py", line 90, in reset
return self.env.reset()
File "/home/zhqin/RL_ViGen/envs/DMCVGB/dmcvgb/wrappers.py", line 213, in reset
obs = self.env.reset()
File "/home/zhqin/RL_ViGen/envs/DMCVGB/dmcvgb/wrappers.py", line 329, in reset
return self._greenscreen(self.env.reset())
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/gym/wrappers/time_limit.py", line 26, in reset
return self.env.reset(**kwargs)
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/gym/wrappers/order_enforcing.py", line 18, in reset
return self.env.reset(**kwargs)
File "/home/zhqin/RL_ViGen/envs/DMCVGB/dmc2gym/dmc2gym/wrappers.py", line 199, in reset
obs = self._get_obs(time_step)
File "/home/zhqin/RL_ViGen/envs/DMCVGB/dmc2gym/dmc2gym/wrappers.py", line 140, in _get_obs
obs = self.render(
File "/home/zhqin/RL_ViGen/envs/DMCVGB/dmc2gym/dmc2gym/wrappers.py", line 215, in render
return self._env.physics.render(
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 216, in render
camera = Camera(
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 709, in init
if self._physics.contexts.mujoco is not None:
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 540, in contexts
self._make_rendering_contexts()
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/mujoco/engine.py", line 523, in _make_rendering_contexts
render_context = _render.Renderer(
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/_render/glfw_renderer.py", line 40, in init
super().init(max_width, max_height, executor.PassthroughRenderExecutor)
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/_render/base.py", line 63, in init
ctx.call(self._platform_init, max_width, max_height)
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/_render/executor/render_executor.py", line 138, in call
return func(*args, **kwargs)
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/dm_control/_render/glfw_renderer.py", line 51, in _platform_init
self._context = glfw.create_window(width=max_width, height=max_height,
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/glfw/init.py", line 1168, in create_window
return _glfw.glfwCreateWindow(width, height, _to_char_p(title),
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/glfw/init.py", line 628, in errcheck
_reraise(exc[1], exc[2])
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/glfw/init.py", line 52, in _reraise
raise exception.with_traceback(traceback)
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/glfw/init.py", line 607, in callback_wrapper
return func(*args, **kwargs)
File "/home/zhqin/anaconda3/envs/rl-vigen/lib/python3.8/site-packages/glfw/init.py", line 820, in _handle_glfw_errors
raise GLFWError(message)
glfw.GLFWError: (65543) b'GLX: Failed to create context: BadLength (poly request too large or internal Xlib length error)'
Have you ever encountered this problem? How can I solve it?
Is SVEA in this repo based on Drq-v2?
So, does that mean, Drq(v1) is the only SAC_based algorithm among the baselines?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.