Check whether the input request is too long

[vllm-project]

Check whether the input request is too long

[WoosukKwon]

Without the fix, cacheflow can suddenly hang without any notice. Let's fix this.

[zhuohan123]

There are two limits for the sequence length, and we should take whichever is smaller:

1. Model limits: For example, for OPT-13B, the learned position embedding can support only config.max_position_embeddings positions.
2. vLLM limit: Specifically, an input cannot be longer than max_num_batched_tokens.

The real limit should be limit = min(model limit, vllm limit).

Also, for a request:

1. If len(prompt) > limit, we should directly reject the request or return nothing.
2. If len(prompt) + len(generated) > limit, we should stop the generation and return the partial results with the finish reason to be SequenceStatus.FINISHED_LENGTH_CAPPED.

In addition, we should not directly let the server fail when there is one request that is too long. We should return a proper error message or partial result.

[zhuohan123]

@LiuXiaoxuanPKU

[hmellor]

Closing as this should now be fixed.