Merge pull request #265 from runpod-workers/fix/zero-max-model-num_batches

fix: check for zero param and set to None
2026-02-13 15:26:06 -06:00 · 2026-02-13 15:23:54 -06:00
1 changed files with 7 additions and 0 deletions
@@ -288,6 +288,13 @@ def get_engine_args():
    
    # Set max_num_batched_tokens to max_model_len for unlimited batching.
    # vLLM defaults max_num_batched_tokens to 2048 when None, which is too low.
+
+    if args.get("max_model_len") == 0:
+        args["max_model_len"] = None
+
+    if args.get("max_num_batched_tokens") == 0:
+        args["max_num_batched_tokens"] = None
+
    if args.get("max_num_batched_tokens") is None:
        max_model_len = args.get("max_model_len")
        if max_model_len is None:
Author	SHA1	Message	Date
chrisvelaandGitHub	1606cff557	Merge pull request #265 from runpod-workers/fix/zero-max-model-num_batches Release / release (push) Waiting to run fix: check for zero param and set to None	2026-02-13 15:26:06 -06:00
velaraptor-runpod	e705c9494b	fix: check for zero param and set to None	2026-02-13 15:23:54 -06:00