about the loss of generator

[lantaoyu]

about the loss of generator

[LantaoYu]

Hi, this part is a little tricky. In REINFORCE, we first sample a trajectory according to the policy here and then send the sequence to discriminator to get reward here. After this, I need to feed back the reward signal to the generator and to reproduce the probability distribution for each step of generating that sequence (since the received reward is only for that specific sequence) I use a supervised style to feed in that sequence. For example, suppose I get a reward Q(e|{a,b,c,d}) and I have to reproduce G(e|{a,b,c,d}), then at each time step I should feed in {a,b,c,d}.

[yooceii]

Hi,
I have a follow-up question.
When we sampling the trajectory. Didn't we obtain the state-action transition probability at the same time?
Why do you have to reproduce the probability later?