owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #207 from JhennerTigreros/main

Browse Source 
Update requirements and engine creation to support new 0.10.0 vLLM version
This commit is contained in:
Marut Pandya
2025-08-09 09:08:19 -07:00
committed by GitHub
parent 15f569f970 fb0c030797
commit 72a643dd0c
3 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Dockerfile
+1 -1
View File
@@ -12,7 +12,7 @@ RUN --mount=type=cache,target=/root/.cache/pip \
python3 -m pip install --upgrade -r /requirements.txt python3 -m pip install --upgrade -r /requirements.txt
# Install vLLM (switching back to pip installs since issues that required building fork are fixed and space optimization is not as important since caching) and FlashInfer # Install vLLM (switching back to pip installs since issues that required building fork are fixed and space optimization is not as important since caching) and FlashInfer
RUN python3 -m pip install vllm==0.9.1 && \ RUN python3 -m pip install vllm==0.10.0 && \
python3 -m pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.3 python3 -m pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.3
# Setup for Option 2: Building the Image with the Model included # Setup for Option 2: Building the Image with the Model included
builder/requirements.txt
+3 -1
View File
@@ -8,5 +8,7 @@ typing-extensions>=4.8.0
pydantic pydantic
pydantic-settings pydantic-settings
hf-transfer hf-transfer
transformers transformers>=4.55.0
bitsandbytes>=0.45.0 bitsandbytes>=0.45.0
kernels
torch==2.6.0
src/engine.py
-1
View File
@@ -207,7 +207,6 @@ class OpenAIvLLMEngine(vLLMEngine):
model_config=self.model_config, model_config=self.model_config,
base_model_paths=self.base_model_paths, base_model_paths=self.base_model_paths,
lora_modules=self.lora_adapters, lora_modules=self.lora_adapters,
prompt_adapters=None,
) )
await self.serving_models.init_static_loras() await self.serving_models.init_static_loras()