The model requires `num_beams`, although it is not needed in the example

[LEv145]

The model requires `num_beams`, although it is not needed in the example

[sh0tcall3r]

I have the same problem while generating text with the model.
Firstly it requires num_beams and after it's set, AttributeError: 'NoneType' object has no attribute 'tolist' appears like in the post above.
Please fix or provide comments on how to resolve it

[TatianaShavrina]

Hey @LEv145 , thank you for bringing that up!

The num_beams parameter refers to the beam search decoding strategy for the model: see HuggingFace explanation
Try to pass in to the generation function as an argument, or stick to sampling or greedy generation

The parameters can be found in the generate function in xl_wrapper script

[LEv145]

Hey @LEv145 , thank you for bringing that up!

The num_beams parameter refers to the beam search decoding strategy for the model: see HuggingFace explanation Try to pass in to the generation function as an argument, or stick to sampling or greedy generation

The parameters can be found in the generate function in xl_wrapper script

Thanks it works!
But there is a problem when processing the result:

Load checkpoint from /mnt/store/models/rugpt3xl/mp_rank_00_model_states.pt
Model Loaded
Traceback (most recent call last):
  File "/mnt/store/tests/test_rugpt3xl.py", line 29, in <module>
    main()
  File "/mnt/store/tests/test_rugpt3xl.py", line 19, in main
    result = gpt.generate(
  File "/opt/ru-gpts/src/xl_wrapper.py", line 244, in generate
    return list(map(self.tokenizer.decode, res.tolist()))
AttributeError: 'NoneType' object has no attribute 'tolist'

Code

import os
import sys

sys.path.append("/opt/ru-gpts/")
os.environ["USE_DEEPSPEED"] = "1"
os.environ["MASTER_ADDR"] = "127.0.0.1"
os.environ["MASTER_PORT"] = "5000"
from src.xl_wrapper import RuGPT3XL


def main():
    gpt = RuGPT3XL.from_pretrained(
        "sberbank-ai/rugpt3xl",
        weights_path="/mnt/store/models/rugpt3xl/mp_rank_00_model_states.pt",
        seq_len=512,
    )
    result = gpt.generate(
        "Кто был президентом США в 2020? ",
        max_length=50,
        num_beams=5,
        early_stopping=True,
    )
    print(result)


if __name__ == "__main__":
    main()