(Solved) No env.reset() at the end of each training epoch.

[slDeng1003]

(Solved) No env.reset() at the end of each training epoch.

[ZheruiHuang]

Hello! I think the training code is logically the same as OpenAI's.

Maybe you are misled by these two similar loops: https://github.com/openai/spinningup/blob/038665d62d569055401d91856abb287263096178/spinup/algos/pytorch/ppo/ppo.py#L299 and

PPO-PyTorch/train.py

Line 173 in 728cce8

for t in range(1, max_ep_len+1):

In the former (OpenAI's) implementation, this loop will perform more than one episode, and it calls reset when an episode is done (but not jump out the loop). In the latter (this repo's) implementation, the loop performs only one episode. When an episode is done, it breaks the loop and resets the env (before the next episode begins).

Hope it makes scene to you!

[slDeng1003]

Dear Huang,
I appreciate your reply. I have checked the code and find out that you are right.
Thank you again for your help!👍
@ZheruiHuang