owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

add as VLLM_RUNPOD prefix and update readme

Browse Source
This commit is contained in:
velaraptor-runpod
2026-02-24 17:37:57 -06:00
parent 42443f735e
commit efb093e198
4 changed files with 41 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/configuration.md
+22
View File
@@ -156,6 +156,28 @@ The way this works is that the first request will have a batch size of `DEFAULT_
| `DISABLE_LOGGING_REQUEST` | False | `bool` | Disable logging requests. |
| `MAX_LOG_LEN` | None | `int` | Max number of prompt characters or prompt ID numbers being printed in log. |
## VLLM_RUNPOD_ Prefix: Pass Any Engine Arg
Any vLLM `AsyncEngineArgs` field can be set via an environment variable using the `VLLM_RUNPOD_` prefix. The suffix maps directly to the field name (case-insensitive, underscores preserved).
**Format:** `VLLM_RUNPOD_<ARG_NAME>=<value>`
**Examples:**
| Environment Variable | vLLM Engine Arg | Value Example |
| ------------------------------------- | -------------------------- | ------------- |
| `VLLM_RUNPOD_MAX_MODEL_LEN` | `max_model_len` | `4096` |
| `VLLM_RUNPOD_ENFORCE_EAGER` | `enforce_eager` | `true` |
| `VLLM_RUNPOD_ENABLE_CHUNKED_PREFILL` | `enable_chunked_prefill` | `true` |
| `VLLM_RUNPOD_NUM_SCHEDULER_STEPS` | `num_scheduler_steps` | `8` |
| `VLLM_RUNPOD_TOKENIZER_POOL_SIZE` | `tokenizer_pool_size` | `4` |
**Notes:**
- Only valid `AsyncEngineArgs` fields are applied. Unknown keys are silently ignored.
- Values are automatically cast to the correct type (`int`, `float`, `bool`, `str`, or JSON for `dict`/`list`).
- `VLLM_RUNPOD_` overrides are applied **after** all other worker env vars, so they take precedence.
- For a full list of available engine args, see the [vLLM AsyncEngineArgs documentation](https://docs.vllm.ai/en/latest/serving/engine_args.html).
## Docker Build Arguments
These variables are used when building custom Docker images with models baked in: