bbrighttaer/irelease

Inverse Reinforcement Learning-based Structural Evolution of Small Molecules

irelease's Issues

About replay buffer

Hi,
I have some questions about the replay buffer. I found that in Link, we sample the trajectories from the replay buffer and then concate the experiment trajectories and the sampled trajectories, and finally push the concated trajectories back to the replay buffer. However, based on the code, once the size of the replay buffer is not larger than its batch_size, it will sample all trajectories in the replay buffer and do the following steps, then the trajectories will be added repetitively, i.e., in the first iteration of the fit function, len(trajectories) will be 12+0=12, the size of the replay buffer will be 0+12 = 12; in the second iteration, len(trajectories) will be 12+12=24, the size of the replay buffer will be 12+24 = 36. Then the trajectories of the experiment trajectories from the first iteration will be duplicated. Right? It seems a little bit weird for me.