ValueError: Expected parameter logits

[voidful]

ValueError: Expected parameter logits

[voidful]

I also find that repetition_penalty can cause that issue, so I removed that parameter in the latest version.

[voidful]

It is the latest version of textrl?
The issue could very likely be due to a significant update in the optimizer. Try adjusting the optimizer to see what effect it has.

[tsaoni]

it seems like the function top_k_top_p_filtering in the forward function of SoftmaxCategoricalHead class results in this error..?
it filters the value by replacing -inf, which causes loss_policy runs into nan

[tsaoni]

I'm sorry to bother but still want to ask,
is it reasonable to put top k top p filter inside the forward function? cause it would change the log probability when training
I suppose we only have to do it before sampling action?

[voidful]

I'm sorry to bother but still want to ask,
is it reasonable to put top k top p filter inside the forward function? cause it would change the log probability when training
I suppose we only have to do it before sampling action?

When we use model, we normally use top k top p sampling, my opinion is to align rl training to the actual usage.