moripiri/diverserl

(Yet another) Reinforcement Learning repository that aims to implement and benchmark various RL algorithms

diverserl's Issues

configuration management + training flow change

expected outcome

from diverserl.algos import PPO



if __name__ == '__main__':
    args = get_args()

    algo = PPO(**args)
    algo.train()

use hydra

Neural network improvement

• Network layer-by-layer initialization
• d2rl network

Console printing

Use rich(or other library) for better console printing

Wide Env support

gymansium Atari env, Pettingzoo, etc

• deepmind control suite

model-free rl

Model-free RL 기본

• DQN
• DDPG
• TD3
• SAC
• PPO

Python environment in local

• Mac (Ventura 13.4.1, M1 Max)

  • Python 3.10
  • poetry
  • gymnasium
  • pytorch, jax

• Windows (Windows 10)

  • Python 3.10
  • poetry
  • gymnasium
  • pytorch, jax

Pixel-based RL

• RL algorithms that specializes in learning RL envs that have images as observation
• ex. RAD, SAC-AE, DrQ, CURL, DrQ-v2, etc
• need dm_control suite env support (use Shimmy)

Multiprocessing

Use multiprocessing in deep RL

rl run hyperparameter improvement

• yaml file로 parse.args 관리, 저장, load
• env, atari_env, trainer의 common hyperparameter 관리

benchmark

• configurations update
• wandb model upload
• wandb gif upload
• wandb model load + evaluation
• ReadMe update

ppo improvement

• Change environment to vectorized ones
• followed by trainer and algorithm fix
• then improve ppo -> first paper-precise, then add features

add pre-commit, typing

add pre-commit code formatter(black, isort, etc)
typing

Tensorflow2, Jax support

• Tensorflow2, jax support

Buffer fix + enhancement

Fix the current buffer more efficiently
PER suppor, etc..

logging & saving

local logging & saving

• tensorboard, wandb
• pytorch model saving

Add these features in Trainer

Docstring

Write docstring