owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
407 Commits 1 Branch 69 Tags
70cd1c8113fbd75cc42cdd377ef8014a980619c5
Commit Graph
88 Commits
Author SHA1 Message Date
pandyamarut 70cd1c8113 update vllm version 0.9.0 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-06-05 11:59:15 -07:00
Marut PandyaandGitHub 6db2c44d3b Update Dockerfile 2025-05-09 09:27:54 -07:00
pandyamarut f33e8d2bcd update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-04-21 13:14:32 -07:00
pandyamarut 3d067cd472 update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-04-07 11:54:57 -07:00
pandyamarut acbdf63de8 update vllm 0.8.2 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-03-26 15:55:30 -07:00
pandyamarut 56dc4ad075 update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-02-24 15:01:45 -08:00
pandyamarut c9791f1163 update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-02-11 13:27:57 -08:00
pandyamarut fcbfe84f63 update vllm 0.7.0 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2025-01-28 14:18:10 -08:00
Hailong Yang a578c6df23 update docker file 2025-01-26 23:28:27 -05:00
pandyamarut 06c2bb1715 upgrade vllm version 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-12-30 15:20:11 -08:00
pandyamarut 4e10641d69 update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-11-19 12:14:04 -08:00
pandyamarut c03ecc42fe update vllm 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-10-15 16:49:41 -07:00
pandyamarut 1420091588 update vll 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-09-27 19:23:50 -07:00
pandyamarut 2ac6a0108f update vllm v0.6.0 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-09-16 14:55:40 -07:00
pandyamarut 5e1c8c8128 update vllm version 0.5.5 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-08-28 22:46:45 -07:00
pandyamarut 9cb9336cf5 update vllm version 0.5.4 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-08-09 12:01:42 -07:00
pandyamarut bd96b5e0de update v0.5.3.post1 
Signed-off-by: pandyamarut <pandyamarut@gmail.com>
 2024-07-25 21:29:45 -07:00
alpayariyak 5bd6f3a75e 0.5.3, any vllm arg as env var, refactor and fixes, moving away from building separate image from vLLM fork 2024-07-25 12:41:48 -07:00
alpayariyak c8458fef2b preparation for 1.0.0 release 2024-06-12 12:28:03 -07:00
alpayariyak ec7ea0b760 Update default base image version to fix github actions build 2024-05-09 20:18:34 -04:00
Alpay Ariyakandalpayariyak 874379a0c5 1.0.0preview update for Llama 3 support and more (vLLM 0.3.3 -> 0.4.2) (#62) 2024-05-09 00:42:08 -04:00
alpayariyak db7167d57f 0.3.3 2024-03-05 19:14:35 +00:00
Alpay Ariyakandalpayariyak d91ccb866f 0.3.1: bug fixes 2024-02-29 02:55:44 -05:00
alpayariyak 6bcd9d7c67 Preparing for 0.3.0 2024-02-22 18:04:34 -05:00
alpayariyak 8de10468dd Working tokenizer and model download fix 
Fix handler startup
 2024-02-02 19:55:35 -05:00
alpayariyak b1720a154d Added download of model extras into weights folder, separate download of tokenizer, making engine.py utilize downloaded tokenizer, model and tokenizer revision 2024-01-31 22:57:32 -05:00
alpayariyak e7b340d73b Update vLLM base image 2024-01-31 03:37:55 +00:00
alpayariyak 12d6f0778e Fixed Model bake-in, added Custom Chat Templates, Custom Tokenizer 2024-01-31 03:13:21 +00:00
Alpay AriyakandGitHub fa5556434c Bug fix 2024-01-29 11:26:22 -05:00
alpayariyak 4cebe66b36 0.2.0 Release 
- You no longer need a linux-based machine or NVIDIA GPUs to build the worker.
- Over 3x lighter Docker image size.
- OpenAI Chat Completion output format (optional to use).
- Extremely fast image build time.
- Docker Secrets-protected Hugging Face token support for building the image with a model baked in without exposing your token.
- Support for `n` and `best_of` sampling parameters, which allow you to generate multiple responses from a single prompt.
- New environment variables for various configuration.
- vLLM Version: 0.2.7
 2024-01-25 20:49:15 -05:00
Alpay Ariyakandalpayariyak 368c5f87fb Temporary Dockerfile Fix 
Temporary Dockerfile fix
 2024-01-25 20:48:26 -05:00
alpayariyak 584852f0f6 Docker-Protected HF Token, Refactor, Better Documentation 2024-01-18 18:20:51 -05:00
alpayariyak c59038e902 Fixed CUDA 11.8 workers 2023-12-20 07:50:14 +00:00
Alpay AriyakandGitHub c321d828ab Update Dockerfile 2023-12-19 00:58:45 -05:00
alpayariyak f324bef8a0 Bump to vLLM 0.2.6, fixes and improvements 2023-12-19 01:52:55 +00:00
alpayariyak 22de0b322d Complete refactor, improved overall functionality 2023-12-13 10:33:03 +00:00
alpayariyak ef89868570 Small changes 2023-12-12 20:42:27 +00:00
alpayariyak b51827388c bug fix 2023-12-09 19:06:41 +00:00
alpayariyak 8578581c6a Small changes 2023-12-09 18:50:00 +00:00
alpayariyak f1afbcbe4f CPU Limit 2023-12-09 18:47:45 +00:00
alpayariyak 5e38a1a2fa Multi-GPU Fix 2023-12-09 10:36:26 +00:00
Justin Merrell bcf37e7bd2 Update Dockerfile 2023-12-08 20:42:15 -05:00
Justin Merrell deb6f67a25 Update Dockerfile 2023-12-08 20:42:05 -05:00
Justin Merrell 11e6495304 Update Dockerfile 2023-12-08 20:28:44 -05:00
Justin Merrell 35a74475f2 Update Dockerfile 2023-12-08 20:23:23 -05:00
alpayariyak 8dbd8f15dd Token counting, cuda-related changes 2023-12-06 19:02:12 +00:00
alpayariyak b705544bd4 model packing, cuda version selection on build 2023-12-06 18:52:21 +00:00
alpayariyak 48c520b240 Version Bump + small changes 2023-12-06 16:48:12 +00:00
alpayariyak 66c23d45b0 Model Downloading, no GPU 2023-12-06 01:43:09 +00:00
alpayariyak b4c61e61c4 Packing model into worker for internal use 2023-12-05 20:26:56 -05:00