does overestimate occurs in advantage actor-critic？

[sfujim]

does overestimate occurs in advantage actor-critic？

[sfujim]

Yes it should, but as it's not studied in this paper, I can't say if the overestimation is significant or if it impacts the performance of the algorithm, especially since it uses n-step estimates.

[ToTheMonn]

I am confused about why the value of underestimated actions will not be explicitly propagated
through the policy update?

[sfujim]

Informally, the policy update pushes the policy to avoid underestimated actions.