owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
405 Commits 1 Branch 69 Tags
11f96a09d7639dd2a2b4b9f094009ad387c8219e
Commit Graph
405 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Marut PandyaandGitHub 6fc770415d Merge pull request #157 from runpod-workers/m-c 
Enabling model caching.
 2025-02-06 11:52:19 -08:00
Mohamed NagyandGitHub 04288240f6 updating the . in ['awq', 'squeezellm', 'gptq'. 'bitsandbytes'] for the QUNATIZATION row 2025-02-02 14:58:48 +02:00
Marut PandyaandGitHub 9e8d9196b0 Merge pull request #154 from runpod-workers/fx-eng 
update engine.py
v1.9.0 		2025-01-28 21:40:26 -08:00
pandyamarut 30dd7c1eb5 update engine.py 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-01-28 21:40:00 -08:00
Marut PandyaandGitHub aa7c37ffb0 Merge pull request #152 from runpod-workers/fix0.7.0-1 
update oai serving classes
 2025-01-28 20:58:40 -08:00
pandyamarut dc8c88027a update serving classes 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-01-28 20:58:07 -08:00
Marut PandyaandGitHub a948e90caa Merge pull request #151 from runpod-workers/fix0.7.0 
dynamic lora loading
 2025-01-28 19:02:59 -08:00
pandyamarut c703254f71 dynamic lora loading 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-01-28 19:00:57 -08:00
Marut PandyaandGitHub 2747106403 Merge pull request #150 from runpod-workers/up-0.7.0 
update vllm 0.7.0
 2025-01-28 14:27:37 -08:00
pandyamarut 91ed30e9a2 update readme 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-01-28 14:21:33 -08:00
pandyamarut fcbfe84f63 update vllm 0.7.0 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-01-28 14:18:10 -08:00
Marut PandyaandGitHub 610429df23 Merge pull request #143 from sihamouda/main 
Add limit_mm_per_prompt engine argument for multimodels input
 2025-01-27 16:41:47 -08:00
Hailong Yang a578c6df23 update docker file 2025-01-26 23:28:27 -05:00
Mohamed NagyandGitHub 9299f43b8b solving "typing_extensions" compatibility with "bitsandbytes" 
```
2025-01-22 18:04:01 [INFO] > [stage-0 5/8] RUN --mount=type=cache,target=/root/.cache/pip python3 -m pip install --upgrade pip && python3 -m pip install --upgrade -r /requirements.txt:
2025-01-22 18:04:01 [INFO] #13 8.904
2025-01-22 18:04:01 [INFO] #13 8.904 The conflict is caused by:
2025-01-22 18:04:01 [INFO] #13 8.904 The user requested typing-extensions==4.7.1
2025-01-22 18:04:01 [INFO] #13 8.904 bitsandbytes 0.45.0 depends on typing_extensions>=4.8.0
```
 2025-01-22 18:06:46 +02:00
mohamednaji7 131c17569f correct access to "args" dictionary 2025-01-21 22:33:28 +02:00
mohamednaji7 8882f6d50b adding 'bitsandbytes' to QUANTIZATION 2025-01-21 14:46:40 +02:00
mohamednaji7 331bc30101 adding 'bitsandbytes' option 2025-01-21 14:08:37 +02:00
mohamednaji7 a27f72a33a inforce args.quantization for bnb load_froamt 2025-01-21 14:01:32 +02:00
mohamednaji7 7167985f23 including bitsandbytes "src:https://docs.vllm.ai/en/stable/quantization/bnb.html" 2025-01-21 13:49:36 +02:00
sihamouda 0e3359a70e add engine argument for multimodels input 2025-01-19 02:44:39 +01:00
Marut PandyaandGitHub 3f0a20d28e Merge pull request #141 from runpod-workers/main 
Rebase
 2025-01-02 20:49:48 -08:00
pandyamarut 8e3c26be14 update worker-config 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
v1.8.0 		2024-12-30 17:41:10 -08:00
pandyamarut 66ea8b1110 update engine 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-12-30 17:13:03 -08:00
Marut PandyaandGitHub a3d432afdf Merge pull request #140 from runpod-workers/nw-updte 
upgrade vllm version
 2024-12-30 15:23:44 -08:00
pandyamarut 06c2bb1715 upgrade vllm version 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-12-30 15:20:11 -08:00
Marut PandyaandGitHub d2e355eae9 Merge pull request #137 from kldzj/fix-tool-calling 
fix: openai tool calling
 2024-12-15 20:50:47 -08:00
kldzj b7787d8ca8 fix: remove unused import 2024-12-06 16:29:06 +01:00
kldzj b1fca5d257 fix: move tool flags away from engine args 2024-12-06 16:07:23 +01:00
kldzj 3e86d16892 fix: openai tool calling 2024-12-06 13:08:48 +01:00
Marut PandyaandGitHub d9f54ce76a Update README.md 2024-11-29 16:30:45 -08:00
Marut PandyaandGitHub 149da95cd0 Merge pull request #133 from kldzj/tool-calling 
feat: add tool calling flags
 2024-11-29 16:29:56 -08:00
Marut PandyaandGitHub 0a89394f1d Update worker-config.json 2024-11-29 16:29:33 -08:00
Nikolai Kolodziej 8df7f41f1d fix: set empty tool_call_parser to None 2024-11-24 06:55:43 +01:00
Nikolai Kolodziej 4d7b8c03c0 feat: tool calling flags 2024-11-24 06:51:41 +01:00
pandyamarut 6c6bf50379 update env 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-11-22 15:14:57 -08:00
pandyamarut 27a2ee5754 add model cache 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-11-22 13:11:12 -08:00
Marut PandyaandGitHub 2df915a145 Merge pull request #132 from runpod-workers/wc-uo 
update worker-config
v1.7.0 		2024-11-20 14:54:45 -08:00
pandyamarut aadc025849 update worker-config 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-11-20 14:42:56 -08:00
Marut PandyaandGitHub 8b4a49073d Merge pull request #131 from runpod-workers/up-0.6.4 
[Core]Update vllm-0.6.4
 2024-11-20 10:37:46 -08:00
pandyamarut 4e10641d69 update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-11-19 12:14:04 -08:00
Marut PandyaandGitHub 6e8696c12a Merge pull request #121 from sven-knoblauch/lora-modules 
add changes for lora adapter support and /v1/models endpoint
 2024-10-31 15:07:02 -04:00
Marut PandyaandGitHub b49e81a75a Merge pull request #126 from runpod-workers/up-wc-1 
update worker-config
 2024-10-15 17:52:13 -07:00
pandyamarut 65932f85e1 update worker-config 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-10-15 17:47:58 -07:00
Marut PandyaandGitHub 94840cfbbb Update README.md 2024-10-15 17:38:31 -07:00
Marut PandyaandGitHub ce47c41f4a Merge pull request #125 from runpod-workers/up-0.6.3 
update vllm
v1.6.0 		2024-10-15 17:04:20 -07:00
pandyamarut c03ecc42fe update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-10-15 16:49:41 -07:00
Dean QuiñanolaandGitHub ae56b9f43d Merge pull request #123 from runpod-workers/dependency-update-refactored 2024-10-13 20:23:33 -07:00
Dean QuiñanolaandGitHub 891699be1e Merge pull request #124 from runpod-workers/readme-correction 2024-10-13 20:23:05 -07:00
Dean QuiñanolaandGitHub c28aa02576 Update README.md 
v1.5.0 not v15
 2024-10-12 17:11:59 -07:00
Dean QuiñanolaandGitHub eba20c0704 Update CI-runpod_dep.yml 
Dependency has changed to `runpod~=<version>` and not `runpod==<version>`
 2024-10-12 16:47:31 -07:00