Reward policy agent environment is not training with Finetuned model

[voidful]

Reward policy agent environment is not training with Finetuned model

[voidful]

It appears that your problem is coming from the fact that the RL model does not recognize your fine-tuned models as supported architectures.

This exception is raised when the code checks for the model's architecture and doesn't find it in the pre-defined set of supported models.

TextRL/textrl/actor.py

Line 59 in 3412399

if 'transformer' in parents: # gpt2/bloom:

In order to make it work, you would have to extend the TextRLActor class to support the specific architectures of your models.