Files
worker-vllm/src/constants.py
T
Tim PietruskyandGitHub 66e1b1605b
Release / release (push) Waiting to run
Merge pull request #226 from runpod-workers/fix/cse-839-max-concurrency
fix: max concurrency = 30 instead of 300
2025-10-22 22:54:21 +02:00

4 lines
116 B
Python

DEFAULT_BATCH_SIZE = 50
DEFAULT_MAX_CONCURRENCY = 30
DEFAULT_BATCH_SIZE_GROWTH_FACTOR = 3
DEFAULT_MIN_BATCH_SIZE = 1