Soren Dreano
11f96a09d7
remove requirements for MODEL_NAME in local_model_args.json
...
We want to use the same local_model_args.json for multiple models
which have different names. It would be very convenient to only
have a single local_args file and not have to create it every time
A warning should be enough for users
2025-05-23 17:59:19 +02:00
Marut Pandya and GitHub
a9786a2481
Revert "fix: added back limit_mm_per_prompt to engine args"
2025-05-07 12:29:33 -07:00
Marut Pandya and GitHub
4e474c41c8
Merge pull request #177 from aleksandar-babic/main
...
fix: added back limit_mm_per_prompt to engine args
2025-05-03 21:01:58 -07:00
Marut Pandya and GitHub
6b64bb93bd
Merge pull request #147 from mohamednaji7/BitsAndBytes
...
completing the "bitsandbytes" option - based on https://docs.vllm.ai/en/stable/quantization/bnb.html
2025-04-21 11:39:53 -07:00
Aleksandar Babic
cfc258674b
chore: added trailing comma to the final arg
2025-04-21 10:06:08 -04:00
Aleksandar Babic
8beafed06b
fix: added back limit_mm_per_prompt to engine args
2025-04-21 10:02:46 -04:00
pandyamarut
c703254f71
dynamic lora loading
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2025-01-28 19:00:57 -08:00
mohamednaji7
131c17569f
correct access to "args" dictionary
2025-01-21 22:33:28 +02:00
mohamednaji7
a27f72a33a
inforce args.quantization for bnb load_froamt
2025-01-21 14:01:32 +02:00
sihamouda
0e3359a70e
add engine argument for multimodels input
2025-01-19 02:44:39 +01:00
kldzj
b1fca5d257
fix: move tool flags away from engine args
2024-12-06 16:07:23 +01:00
Nikolai Kolodziej
8df7f41f1d
fix: set empty tool_call_parser to None
2024-11-24 06:55:43 +01:00
Nikolai Kolodziej
4d7b8c03c0
feat: tool calling flags
2024-11-24 06:51:41 +01:00
pandyamarut
c03ecc42fe
update vllm
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-10-15 16:49:41 -07:00
pandyamarut
967eaba573
change to float
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-09 14:41:07 -07:00
pandyamarut
9cb9336cf5
update vllm version 0.5.4
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-09 12:01:42 -07:00
pandyamarut
0f8657e58d
update env default args
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-07-26 16:52:51 -07:00
alpayariyak
5bd6f3a75e
0.5.3, any vllm arg as env var, refactor and fixes, moving away from building separate image from vLLM fork
2024-07-25 12:41:48 -07:00
alpayariyak
a08d83f600
Allow any vLLM engine args as env vars, refactor
2024-07-02 19:44:01 +00:00
Soren Dreano