glm4 9b 1m 启动报错

[brightzhu2020]

glm4 9b 1m 启动报错

[bridgearchway]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).
there is no clear basic environment recommendations. too bad

I just successfully fixed it. The version of transformer seems to be 4.40.0, which can be found in requirements.txt in basic_demo. Once I renew the version, it works. Hopefully this can help you as well.

[huaizhe2012]

情况一样,tokenizer加载报错
transformer 升级为4.40.0确实好了

[zRzRzRzRzRzRzR]

请严格按照req安装依赖哦，如果是windows系统，不能装vLLM，使用transformers后端

[JiaweiMorris]

使用huggingface model页的多模态demo代码出现同样报错
return self.mergeable_ranks[token]
KeyError: '<|endoftext|>'

[bridgearchway]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).

[brightzhu2020]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).
there is no clear basic environment recommendations. too bad

[brightzhu2020]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).
there is no clear basic environment recommendations. too bad

I just successfully fixed it. The version of transformer seems to be 4.40.0, which can be found in requirements.txt in basic_demo. Once I renew the version, it works. Hopefully this can help you as well.

would you please shall your full enviroment details?
i had other issues....
CUDA
Pytorch
etc..?

Thanks!

[bridgearchway]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).
there is no clear basic environment recommendations. too bad

I just successfully fixed it. The version of transformer seems to be 4.40.0, which can be found in requirements.txt in basic_demo. Once I renew the version, it works. Hopefully this can help you as well.

would you please shall your full enviroment details? i had other issues.... CUDA Pytorch etc..?

Thanks!

Sure, my envs are:

win 10,
cuda 12.1,
python 3.10,
gpu 3090*2,
transformer==4.40.0,
torch==2.1.0

That's all for inference. Btw, I noticed that the README in basic_demo suggests "GPUs above A100, V100, 20 and older GPU architectures are not supported". I hope this may help.

[brightzhu2020]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).
there is no clear basic environment recommendations. too bad

I just successfully fixed it. The version of transformer seems to be 4.40.0, which can be found in requirements.txt in basic_demo. Once I renew the version, it works. Hopefully this can help you as well.

would you please shall your full enviroment details? i had other issues.... CUDA Pytorch etc..?
Thanks!

Sure, my envs are:

win 10, cuda 12.1, python 3.10, gpu 3090*2, transformer==4.40.0, torch==2.1.0

That's all for inference. Btw, I noticed that the README in basic_demo suggests "GPUs above A100, V100, 20 and older GPU architectures are not supported". I hope this may help.

Thanks!
it shall be a amper GPU, not a tuning GPUT

[brightzhu2020]

请严格按照req安装依赖哦，如果是windows系统，不能装vLLM，使用transformers后端
非常感谢
我是从GLM3的环境继承的，全局的CUDA版本也不一样
还是希望官方出一般最低要求或者兼容性比较好的要求。一台本地机器上能跑GLM3 6b，也能跑GLM4-9b
期待越来越好，越来越完善。
等待你们的更新

[zRzRzRzRzRzRzR]

或许可以使用我们trans_cli_demo，环境还是要重新装的，现在的依赖默认不会装vLLM，但是如果用trans后段能推理的长度非常非常短，8K差不多就到消费卡24G显存的上限

[brightzhu2020]

Me win 10, cuda 12.1, python 3.10, gpu 3090*2, also encountered the same problem with KeyError: '<|endoftext|>'. It seems that the problem comes from AutoTokenizer.from_pretrained(...).
there is no clear basic environment recommendations. too bad

I just successfully fixed it. The version of transformer seems to be 4.40.0, which can be found in requirements.txt in basic_demo. Once I renew the version, it works. Hopefully this can help you as well.

would you please shall your full enviroment details? i had other issues.... CUDA Pytorch etc..?
Thanks!

Sure, my envs are:

win 10, cuda 12.1, python 3.10, gpu 3090*2, transformer==4.40.0, torch==2.1.0

That's all for inference. Btw, I noticed that the README in basic_demo suggests "GPUs above A100, V100, 20 and older GPU architectures are not supported". I hope this may help.

May I ask what is the motherboard and CPU? can two GPU run maxium performance, as the PCIE channel limts of the CPU vs. two 3090. Thanks!