ffeng1996/factored_nonstationary_rl

Factored Adaptation for Non-Stationary Reinforcement Learning

Requirements

The main requirements can be found in requirements.txt.

To install the requirements, you can follow the instructions below:

pip install -r requirements.txt

Additionally, Mujoco should be installed.

Overview

The main training loop is in learner.py, the VAE set-up and losses are in vae.py, the model design is in models/, the RL algorithms are in algorithms/, and the hyperparameters are in config/. You need to modify the hyperparameters for the specific environment you want to run.

Running experiments

To evaluate, run

python main.py --env-type <env_name>

which will use hyperparameters from config/args_<env_name>.py.

factored_nonstationary_rl's Issues

bug

doesn't the state_decoder only return self.fc_out(h)?

Factored_Nonstationary_RL/vae.py

Line 159 in 41dfff1

state_pred, state_pred_next, latent_mean_pred, latent_logvar_pred = self.state_decoder(latent, prev_latent, latent_mean_prev, latent_logvar_prev, prev_obs, action)

Missing code

Hi @ffeng1996
Would you please complete the code for the environment creation part?
from environments.parallel_envs import make_vec_envs

Code not complete

Hi
Can you please update the code base to accomodate the changes on Mujoco for Non-stationarity in terms of how you apply and vary the wind force, vary acceleration due to gravity and vary the target velocity.
The code seems to be incomplete.

Thanks.

Any plan to release the full codebase in the near future?

This paper is excellent and insightful. I am currently working on a related work and hope to make some reasonable comparisons. However, I am not clear about the specific training process, parameter settings, environment settings and other implementation details from the current code. I noticed that you are preparing to release the full code. May I ask if it can be released soon?