Two similar custom environment, PPO learns on both but SAC only on one

[tfederico]

Two similar custom environment, PPO learns on both but SAC only on one

[tfederico]

To add more info, I am currently running a hyperparameters search over 5M steps on the following parameters

parameters:
  learning_rate:
    values: [0.003, 0.001, 0.0003, 0.0001, 0.00003]
  buffer_size:
    values: [1000000, 2000000]
  learning_starts:
    values: [100, 5000, 10000]
  batch_size:
    values: [256, 512]
  tau:
    values: [0.0, 0.1, 0.25, 0.5]
  ent_coef:
    values: ["auto", "auto_0.1", "0.25", "0.5", "1.0"]
  target_entropy:
    values: ["auto", 0.1, 0.2]
  log_std_init:
    values: [-1.0, -3.0, -5.0]