Why not calculate prompt on the server side?

[hdzhang-code]

Why not calculate prompt on the server side?

[txsun1997]

Yeah, it is okay to perform SGD to obtain an optimized prompt for the user. But the optimization can be expensive for large models. The computation and memory cost of SGD/Adam is proportional to the size of the model. If we consider hyper-parameter tuning (in fact, this is necessary for prompt tuning), the cost is more expensive. In comparison, black-box tuning only requires the model forward process, which is cheaper and easier to be accelerated with the support of ONNX/TensorRT.

For another solution, if the server returns the gradient w.r.t. the prompt to the user, the deployed model will be vulnerable (See LAMP: Extracting Text from Gradients with Language Model Priors).

So the answer is yes, the solution you mentioned will translate to additional burden of the server.