6.5 MEDIUM
- CVSS version (CVSS): 3.1
- Attack Vector (AV): Network (N)
- Attack Complexity (AC): Low (L)
- Privileges Required (PR): Low (L)
- User Interaction (UI): None (N)
- Scope (S): Unchanged (U)
- Confidentiality (C): None (N)
- Integrity (I): None (N)
- Availability (A): High (H)
- Modified Attack Vector (MAV): Network (N)
- Modified Attack Complexity (MAC): Low (L)
- Modified Privileges Required (MPR): Low (L)
- Modified User Interaction (MUI): None (N)
- Modified Confidentiality (MC): None (N)
- Modified Scope (MS): Unchanged (U)
- Modified Integrity (MI): None (N)
- Modified Availability (MA): High (H)
Activity log
- Created suggestion
vLLM: Completion prompt lists fan out into unbounded engine requests
vLLM is an inference and serving engine for large language models. From 0.19.0 until 0.26.0, the /v1/completions CompletionRequest.prompt field in vllm/entrypoints/openai/completion/protocol.py accepts an unbounded list[str] or list[list[int]], prompt_to_seq() in vllm/renderers/inputs/preprocess.py and OnlineRenderer.preprocess_completion() in vllm/renderers/online_renderer.py expand every element, and vllm/entrypoints/openai/completion/serving.py creates one engine generator and response slot per prompt, allowing an authenticated API client to exhaust CPU, memory, async scheduling capacity, engine request slots, and response buffering with one request. This issue is fixed in version 0.26.0.
References
-
https://github.com/vllm-project/vllm/security/advisories/GHSA-87x5-vmc3-756j x_refsource_CONFIRM
-
https://github.com/vllm-project/vllm/pull/47845 x_refsource_MISC
-
https://github.com/vllm-project/vllm/releases/tag/v0.26.0 x_refsource_MISC
Affected products
- ==>= 0.19.0, < 0.26.0
Matching in nixpkgs
pkgs.vllm
High-throughput and memory-efficient inference and serving engine for LLMs
pkgs.pkgsRocm.vllm
High-throughput and memory-efficient inference and serving engine for LLMs
pkgs.python313Packages.vllm
High-throughput and memory-efficient inference and serving engine for LLMs
Package maintainers
-
@happysalada Raphael Megzari <raphael@megzari.com>
-
@daniel-fahey Daniel Fahey <daniel.fahey+nixpkgs@pm.me>
-
@LunNova Luna Nova <nixpkgs-maintainer@lunnova.dev>
-
@CertainLach Yaroslav Bolyukin <iam@lach.pw>