owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
187 Commits 1 Branch 69 Tags
b1720a154d0de02afb319a720222b9f3f3091852
Commit Graph
45 Commits
Author SHA1 Message Date
alpayariyak b1720a154d Added download of model extras into weights folder, separate download of tokenizer, making engine.py utilize downloaded tokenizer, model and tokenizer revision 2024-01-31 22:57:32 -05:00
alpayariyak 370698442c Update RunPod SDK version and Docker Tag 2024-01-31 00:54:58 -05:00
alpayariyak 3e2cd080a2 Fix Tensor Parallel 2024-01-31 05:12:58 +00:00
alpayariyak 46eee12819 Simplify Tensor Parallel 2024-01-31 03:28:02 +00:00
alpayariyak 12d6f0778e Fixed Model bake-in, added Custom Chat Templates, Custom Tokenizer 2024-01-31 03:13:21 +00:00
alpayariyak 97726372c0 Update release tag in README.md 2024-01-25 23:37:07 -05:00
alpayariyak 9fc8e1e54c Non-streaming OpenAI Chat Completions 2024-01-25 23:24:18 -05:00
alpayariyak 4cebe66b36 0.2.0 Release 
- You no longer need a linux-based machine or NVIDIA GPUs to build the worker.
- Over 3x lighter Docker image size.
- OpenAI Chat Completion output format (optional to use).
- Extremely fast image build time.
- Docker Secrets-protected Hugging Face token support for building the image with a model baked in without exposing your token.
- Support for `n` and `best_of` sampling parameters, which allow you to generate multiple responses from a single prompt.
- New environment variables for various configuration.
- vLLM Version: 0.2.7
 2024-01-25 20:49:15 -05:00
alpayariyak 15cc7cd36f Updated Documentation 2024-01-19 10:48:12 -05:00
alpayariyak 584852f0f6 Docker-Protected HF Token, Refactor, Better Documentation 2024-01-18 18:20:51 -05:00
Alpay AriyakandGitHub 65454c024a Update README.md (temporary) 2024-01-17 00:16:56 -05:00
Justin Merrell f023e2e097 Update README.md 2024-01-16 19:55:20 -05:00
alpayariyak a5dc8b53b7 Fix Concurrency, Add Max Model Length 2024-01-16 17:58:46 -05:00
Alpay AriyakandGitHub 23ff046586 Update README.md 2024-01-11 12:48:54 +07:00
alpayariyak 914caaa374 Merge branch 'refactor' 2024-01-05 23:44:59 +07:00
alpayariyak 618dd8e0c4 Temporarily remove examples 2024-01-05 16:44:12 +00:00
alpayariyak 3e08a69291 Documentation 2024-01-05 16:40:52 +00:00
alpayariyak 2475cd7a66 Local development, disable log requests 2023-12-29 02:21:24 +00:00
Alpay AriyakandGitHub a247a3afe1 Update README.md 2023-12-22 03:16:04 -05:00
alpayariyak 1fc64fef2b Additional Examples and Documentation for Chat Template and Messages list 2023-12-21 00:38:14 +00:00
alpayariyak 3f413c1038 Documentation for Chat Templates, Messages format, Refactor Streaming, etc 2023-12-21 00:22:23 +00:00
alpayariyak 8a11a7e4ed Add version 2023-12-19 02:57:06 +00:00
alpayariyak f324bef8a0 Bump to vLLM 0.2.6, fixes and improvements 2023-12-19 01:52:55 +00:00
Alpay AriyakandGitHub d6c145bc1e Update README.md 2023-12-18 20:08:35 -05:00
Alpay AriyakandGitHub 840d5ff0d2 Update README.md 2023-12-16 01:30:49 -05:00
Justin Merrell 06660fb8b9 fix: update badge 2023-12-14 16:26:52 -05:00
alpayariyak 22de0b322d Complete refactor, improved overall functionality 2023-12-13 10:33:03 +00:00
Alpay AriyakandGitHub 2dbab5c254 Update README.md 2023-12-12 15:34:47 -05:00
Alpay AriyakandGitHub 50f06a5761 Update README.md 2023-12-12 14:22:25 -05:00
Alpay AriyakandGitHub 7ca8ee7595 Update README.md 2023-12-12 14:09:33 -05:00
alpayariyak 5e38a1a2fa Multi-GPU Fix 2023-12-09 10:36:26 +00:00
alpayariyak 820a21f32c Download logic + other changes 2023-11-28 19:15:08 -05:00
alpayariyak 97480ff0ae handler changes 2023-11-28 23:39:48 +00:00
alpayariyak e7c8de4678 Changes to worker 2023-11-15 20:53:04 -05:00
Alpay AriyakandGitHub 24feadd903 Update README.md 2023-11-14 19:25:41 -05:00
Samuel Willandalpayariyak 4f792062aa vllm 0.2.1.post1 - speed boost, quantization, mistral support 2023-11-14 19:12:58 -05:00
Jorg DokuandGitHub 9e836cc6d1 Update README.md 2023-08-03 10:48:40 -05:00
Jorg DokuandGitHub 04baa8903d Update README.md 2023-08-03 10:45:57 -05:00
Jorg DokuandGitHub 53f9967ea4 Update README.md 2023-08-03 10:45:32 -05:00
Jorg DokuandGitHub 569b3059f0 Update README.md 2023-08-03 10:33:52 -05:00
Jorg DokuandGitHub da05702372 Update README.md 2023-08-03 10:31:56 -05:00
Jorg DokuandGitHub 7b1e0c29d7 Update README.md 2023-08-03 10:30:27 -05:00
Jorg DokuandGitHub 00477f2131 Update README.md 2023-08-03 10:29:14 -05:00
Jorg DokuandGitHub c4bebc030d Update README.md 2023-07-03 13:43:09 -05:00
Jorg DokuandGitHub 38829c2720 Initial commit 2023-07-03 13:42:43 -05:00