Error running inference on CogVLM2 when distributing it on multiple GPUs: Expected all tensors to be on the same device, but found at least two devices

[ghazalsaheb]

Error running inference on CogVLM2 when distributing it on multiple GPUs: Expected all tensors to be on the same device, but found at least two devices

[zucchini-nlp]

Hey!

CogVLM uses custom code from the hub when you set trust_remote_code=True and the model is not yet added to transformers. There is an open PR here to port the model to transformers, which is in progress afaik, cc @NielsRogge

For the device mismatch issue, please open an issue in the THUDM/cogvlm2-llama3-chat-19B hub repo.

[ghazalsaheb]

Hey!

CogVLM uses custom code from the hub when you set trust_remote_code=True and the model is not yet added to transformers. There is an open PR here to port the model to transformers, which is in progress afaik, cc @NielsRogge

For the device mismatch issue, please open an issue in the THUDM/cogvlm2-llama3-chat-19B hub repo.

@ zucchini-nlp I see, thanks. By THUDM/cogvlm2-llama3-chat-19B you mean here?