Would it be possible to run the model on single A100 (40GB) or 2xV100 (32GB) ?

[yandex]

Would it be possible to run the model on single A100 (40GB) or 2xV100 (32GB) ?

[alkavan]

Update

So I was unable get to anything working by utilizing 2xV100, tried different configurations of HPARAM_ARGS variable.

If anyone could provide some more details about the following parameters, and what exactly their influence on the model or resource utilization:

--embedding-size 2048
--hidden-size 10240
--intermediate-size 27308
--activation-type geglu
--num-attention-heads 128
--max-position-embeddings 1024

[nzinov]

You will need Offload to use less GPU memory. See this issue #7