owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
188 Commits 1 Branch 69 Tags
8de10468dd418afadf161995a053efe92277cac7
Commit Graph
46 Commits
Author SHA1 Message Date
alpayariyak 8de10468dd Working tokenizer and model download fix 
Fix handler startup
 2024-02-02 19:55:35 -05:00
alpayariyak b1720a154d Added download of model extras into weights folder, separate download of tokenizer, making engine.py utilize downloaded tokenizer, model and tokenizer revision 2024-01-31 22:57:32 -05:00
alpayariyak 370698442c Update RunPod SDK version and Docker Tag 2024-01-31 00:54:58 -05:00
alpayariyak 3e2cd080a2 Fix Tensor Parallel 2024-01-31 05:12:58 +00:00
alpayariyak 46eee12819 Simplify Tensor Parallel 2024-01-31 03:28:02 +00:00
alpayariyak 12d6f0778e Fixed Model bake-in, added Custom Chat Templates, Custom Tokenizer 2024-01-31 03:13:21 +00:00
alpayariyak 97726372c0 Update release tag in README.md 2024-01-25 23:37:07 -05:00
alpayariyak 9fc8e1e54c Non-streaming OpenAI Chat Completions 2024-01-25 23:24:18 -05:00
alpayariyak 4cebe66b36 0.2.0 Release 
- You no longer need a linux-based machine or NVIDIA GPUs to build the worker.
- Over 3x lighter Docker image size.
- OpenAI Chat Completion output format (optional to use).
- Extremely fast image build time.
- Docker Secrets-protected Hugging Face token support for building the image with a model baked in without exposing your token.
- Support for `n` and `best_of` sampling parameters, which allow you to generate multiple responses from a single prompt.
- New environment variables for various configuration.
- vLLM Version: 0.2.7
 2024-01-25 20:49:15 -05:00
alpayariyak 15cc7cd36f Updated Documentation 2024-01-19 10:48:12 -05:00
alpayariyak 584852f0f6 Docker-Protected HF Token, Refactor, Better Documentation 2024-01-18 18:20:51 -05:00
Alpay AriyakandGitHub 65454c024a Update README.md (temporary) 2024-01-17 00:16:56 -05:00
Justin Merrell f023e2e097 Update README.md 2024-01-16 19:55:20 -05:00
alpayariyak a5dc8b53b7 Fix Concurrency, Add Max Model Length 2024-01-16 17:58:46 -05:00
Alpay AriyakandGitHub 23ff046586 Update README.md 2024-01-11 12:48:54 +07:00
alpayariyak 914caaa374 Merge branch 'refactor' 2024-01-05 23:44:59 +07:00
alpayariyak 618dd8e0c4 Temporarily remove examples 2024-01-05 16:44:12 +00:00
alpayariyak 3e08a69291 Documentation 2024-01-05 16:40:52 +00:00
alpayariyak 2475cd7a66 Local development, disable log requests 2023-12-29 02:21:24 +00:00
Alpay AriyakandGitHub a247a3afe1 Update README.md 2023-12-22 03:16:04 -05:00
alpayariyak 1fc64fef2b Additional Examples and Documentation for Chat Template and Messages list 2023-12-21 00:38:14 +00:00
alpayariyak 3f413c1038 Documentation for Chat Templates, Messages format, Refactor Streaming, etc 2023-12-21 00:22:23 +00:00
alpayariyak 8a11a7e4ed Add version 2023-12-19 02:57:06 +00:00
alpayariyak f324bef8a0 Bump to vLLM 0.2.6, fixes and improvements 2023-12-19 01:52:55 +00:00
Alpay AriyakandGitHub d6c145bc1e Update README.md 2023-12-18 20:08:35 -05:00
Alpay AriyakandGitHub 840d5ff0d2 Update README.md 2023-12-16 01:30:49 -05:00
Justin Merrell 06660fb8b9 fix: update badge 2023-12-14 16:26:52 -05:00
alpayariyak 22de0b322d Complete refactor, improved overall functionality 2023-12-13 10:33:03 +00:00
Alpay AriyakandGitHub 2dbab5c254 Update README.md 2023-12-12 15:34:47 -05:00
Alpay AriyakandGitHub 50f06a5761 Update README.md 2023-12-12 14:22:25 -05:00
Alpay AriyakandGitHub 7ca8ee7595 Update README.md 2023-12-12 14:09:33 -05:00
alpayariyak 5e38a1a2fa Multi-GPU Fix 2023-12-09 10:36:26 +00:00
alpayariyak 820a21f32c Download logic + other changes 2023-11-28 19:15:08 -05:00
alpayariyak 97480ff0ae handler changes 2023-11-28 23:39:48 +00:00
alpayariyak e7c8de4678 Changes to worker 2023-11-15 20:53:04 -05:00
Alpay AriyakandGitHub 24feadd903 Update README.md 2023-11-14 19:25:41 -05:00
Samuel Willandalpayariyak 4f792062aa vllm 0.2.1.post1 - speed boost, quantization, mistral support 2023-11-14 19:12:58 -05:00
Jorg DokuandGitHub 9e836cc6d1 Update README.md 2023-08-03 10:48:40 -05:00
Jorg DokuandGitHub 04baa8903d Update README.md 2023-08-03 10:45:57 -05:00
Jorg DokuandGitHub 53f9967ea4 Update README.md 2023-08-03 10:45:32 -05:00
Jorg DokuandGitHub 569b3059f0 Update README.md 2023-08-03 10:33:52 -05:00
Jorg DokuandGitHub da05702372 Update README.md 2023-08-03 10:31:56 -05:00
Jorg DokuandGitHub 7b1e0c29d7 Update README.md 2023-08-03 10:30:27 -05:00
Jorg DokuandGitHub 00477f2131 Update README.md 2023-08-03 10:29:14 -05:00
Jorg DokuandGitHub c4bebc030d Update README.md 2023-07-03 13:43:09 -05:00
Jorg DokuandGitHub 38829c2720 Initial commit 2023-07-03 13:42:43 -05:00