vLLM is an inference and serving engine for large language models (LLMs). From version 0.5.5 to before 0.11.1, the /v1/chat/completions and /tokenize endpoints allow a chat_template_kwargs request parameter that is used in the code before it is properly validated against the chat template. With the right chat_template_kwargs parameters, it is possible to block processing of the API server for long periods of time, delaying all other requests. This issue has been patched in version 0.11.1.
| Software | From | Fixed in |
|---|---|---|
vllm
|
0.5.5 | 0.11.1 |
| vllm / vllm | 0.5.5 | 0.11.1 |
| vllm / vllm | 0.11.1-rc0 | 0.11.1-rc0.x |
| vllm / vllm | 0.11.1-rc1 | 0.11.1-rc1.x |