`stop_strings` Argument in `model.generate()` Results in Exception if Generation Completes Without `stop_string` Being Generated

[lapp0]

`stop_strings` Argument in `model.generate()` Results in Exception if Generation Completes Without `stop_string` Being Generated

[Rocketknight1]

Fix is open here

[zucchini-nlp]

Seems that the bug appears only when the stop string is a single letter, because in that case it's impossible to get token_valid_positions not empty. cc @Rocketknight1 here also

And the tokenizer-related issue is a bug, will be fixed soon!

[lapp0]

Might be a duplicate of #31435

[lapp0]

It looks like this line sets the tokenizer to None automatically, creates a related but not identical issue.

https://github.com/huggingface/transformers/blob/eed9ed67987/src/transformers/generation/utils.py#L1643

@ahmed-moubtahij could you please take a look? Popping tokenizer from kwargs twice guarantees it will be None even if passed.

[Rocketknight1]

On it!

[ahmed-moubtahij]

It looks like this line sets the tokenizer to None automatically, creates a related but not identical issue.

https://github.com/huggingface/transformers/blob/eed9ed67987/src/transformers/generation/utils.py#L1643

@ahmed-moubtahij could you please take a look? Popping tokenizer from kwargs twice guarantees it will be None even if passed.

@lapp0 Thanks for the headsup! I just forked to remove it but @gante outsped me :)

Not sure how critical, but the "we only use it for stopping criteria" comment doesn't stand anymore