frt03/mxt_bench

A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation (ICLR2023)

mxt_bench's Issues

Requirements

It took me a while to get this code base up and running on a MacBook Pro, as it seems to have a few relatively old dependencies (and some dependencies not in the requirements file). I thought I'd post this here in case it's of use to anybody else.

When I ran the code from the requirements file to install a past commit of brax, I found that some of the brax files were missing (e.g. the brax/experimental folder), which produced error messages, and so I ended up performing the following (admittedly convoluted) steps:

• I forked brax to my own github repo
• I then git cloned the forked brax repo to my local computer
• I then reset the cloned local repo to the relevant past commit: git reset --hard 0ebb332
• I then added the local brax repo to PYTHONPATH so it could be found by mxt: export PYTHONPATH=‘/path/to/brax’

I needed to downgrade from python 3.11 to python 3.9, so I could access jaxlib 0.1.71.

Instead of running the requirements file, I manually installed the following packages:

• pip install jaxlib==0.1.71
• pip install flax==0.3.6
• pip install optax==0.0.9
• pip install tensorflow-probability==0.14.0
• pip install jax==0.2.21 (I installed this last because if I did it first, flax and optax would uninstall it and re-install a more recent version)

I then installed the following packages to circumvent errors:

• pip install google
• pip install protobuf==3.20.0
• pip install trimesh
• pip install gym
• pip install chex==0.1.0

AssertionError: losses/entropy_loss 0 NaN

The bugs:

I1014 21:54:03.384178 140228452884864 ppo_mlp.py:499] starting iteration 0 14.847318649291992
I1014 21:54:40.566177 140228452884864 ppo_mlp.py:541] {'eval/episode_distance': DeviceArray(8957.285, dtype=float32), 'eval/episode_reward': DeviceArray(-8957.285, dtype=float32), 'eval/episode_reward_agent1___distance': DeviceArray(-8957.285, dtype=float32), 'eval/episode_score': DeviceArray(8957.285, dtype=float32), 'eval/episode_score_agent1___distance': DeviceArray(8957.285, dtype=float32), 'eval/completed_episodes': DeviceArray(128., dtype=float32), 'eval/avg_episode_length': DeviceArray(1000., dtype=float32), 'speed/sps': 0, 'speed/eval_sps': 3443.126173938599, 'speed/training_walltime': 0, 'speed/eval_walltime': 37.17551374435425, 'speed/timestamp': 0, 'num_timesteps': 0, 'eval/avg_final_distance': DeviceArray(9.663254, dtype=float32), 'eval/success_rate': DeviceArray(0., dtype=float32)}
I1014 21:54:44.653600 140228452884864 ppo_mlp.py:499] starting iteration 1 56.11674904823303
I1014 21:54:48.082901 140228452884864 ppo_mlp.py:541] {'eval/episode_distance': DeviceArray(8651.73, dtype=float32), 'eval/episode_reward': DeviceArray(-8651.73, dtype=float32), 'eval/episode_reward_agent1___distance': DeviceArray(-8651.73, dtype=float32), 'eval/episode_score': DeviceArray(8651.73, dtype=float32), 'eval/episode_score_agent1___distance': DeviceArray(8651.73, dtype=float32), 'losses/entropy_loss': DeviceArray(nan, dtype=float32), 'losses/policy_loss': DeviceArray(nan, dtype=float32), 'losses/total_loss': DeviceArray(nan, dtype=float32), 'losses/v_loss': DeviceArray(nan, dtype=float32), 'eval/completed_episodes': DeviceArray(128., dtype=float32), 'eval/avg_episode_length': DeviceArray(1000., dtype=float32), 'speed/sps': DeviceArray(0., dtype=float32), 'speed/eval_sps': 37404.16184777793, 'speed/training_walltime': 3.1930909156799316, 'speed/eval_walltime': 40.59758901596069, 'speed/timestamp': 3.1930909156799316, 'num_timesteps': 0, 'eval/avg_final_distance': DeviceArray(9.345238, dtype=float32), 'eval/success_rate': DeviceArray(0., dtype=float32)}
Traceback (most recent call last):
File "/home/caiyishuai/workTable/mxt_bench/mxt_bench/train_ppo_mlp.py", line 172, in
app.run(main)
File "/home/caiyishuai/systems/anaconda3/envs/brax_env/lib/python3.8/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/home/caiyishuai/systems/anaconda3/envs/brax_env/lib/python3.8/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "/home/caiyishuai/workTable/mxt_bench/mxt_bench/train_ppo_mlp.py", line 136, in main
inference_fn, params, _ = ppo_mlp.train(
File "/home/caiyishuai/workTable/mxt_bench/mxt_bench/algo/ppo_mlp.py", line 543, in train
progress_fn(num_timesteps, metrics, None)
File "/home/caiyishuai/workTable/mxt_bench/mxt_bench/brax/brax/experimental/braxlines/experiments/init.py", line 270, in progress
assert not np.isnan(v), f'{key} {num_steps} NaN'
AssertionError: losses/entropy_loss 0 NaN

I don't konw how to deal with it? Could you please help me?

Which version of python did the author install?

I use an NVIDIA graphics card, which version of python did the author install?