pandyamarut
3293245c81
update tags
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-28 23:28:39 -07:00
Marut Pandya and GitHub
286d6ba702
Merge pull request #102 from carlson-svg/main
...
Add human readable worker-config.json
2024-08-26 20:22:47 -07:00
carlson-svg
4fa4a8e0e6
added to worker config docs to table of contents + added side note
2024-08-26 01:20:18 -07:00
Marut Pandya and GitHub
2111c9e7a5
Update README.md
2024-08-12 21:02:43 -07:00
Marut Pandya and GitHub
571ef2b805
Update README.md
2024-08-09 14:58:46 -07:00
pandyamarut
967eaba573
change to float
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-09 14:41:07 -07:00
Marut Pandya and GitHub
f023f57217
Update README.md
2024-08-07 15:38:26 -07:00
Marut Pandya and GitHub
673597fd46
Update README.md
2024-08-07 15:32:23 -07:00
Marut Pandya and GitHub
c50543ebd9
Update README.md
2024-08-07 15:30:33 -07:00
pandyamarut
3498e99b2f
update
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-05 14:33:56 -07:00
pandyamarut
5da96ce9a6
update
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-05 14:26:56 -07:00
Marut Pandya and GitHub
e32626ca9d
Update README.md
2024-08-05 14:19:38 -07:00
pandyamarut
e846ecae9d
update readme
...
Signed-off-by: pandyamarut <pandyamarut@gmail.com >
2024-08-02 14:13:55 -07:00
Marut Pandya and GitHub
9f066be620
Update README.md
2024-08-02 13:29:18 -07:00
Marut Pandya and GitHub
e1b41795f7
Update README.md
2024-07-31 12:46:34 -07:00
Marut Pandya and GitHub
0814d76654
Update README.md
2024-07-30 17:09:01 -07:00
alpayariyak
0e1e38326a
Fix deprecated max_context_len_to_capture engine argument
2024-06-13 17:48:05 +00:00
alpayariyak
c8458fef2b
preparation for 1.0.0 release
2024-06-12 12:28:03 -07:00
30cb56a3df
Fix MODEL_REVISION env var (Merge pull request #50 from joennlae/rename-revision and #67 from mikljohansson/main)
...
Fixed MODEL_REVISION environment variable
Co-Authored-By: Jannis Schönleber <1493766+joennlae@users.noreply.github.com >
Co-Authored-By: Mikael Johansson <mikl.johansson@gmail.com >
2024-05-09 20:45:07 -04:00
alpayariyak
9f2cb7b1d0
Update documentation to include rename of fp8_e5m2 to fp8
2024-05-09 23:59:11 +00:00
Alpay Ariyak and GitHub
4f61b04afe
Update README.md
2024-05-09 01:05:04 -04:00
Alpay Ariyak and alpayariyak
874379a0c5
1.0.0preview update for Llama 3 support and more (vLLM 0.3.3 -> 0.4.2) ( #62 )
2024-05-09 00:42:08 -04:00
alpayariyak
0a5b5bc095
Add badges
2024-03-15 21:23:52 -04:00
alpayariyak
2936e4d95d
Update automatic builds, documentation
2024-03-15 11:57:38 -04:00
Alpay Ariyak and GitHub
cee4e484d5
Update README.md for 0.3.2
2024-03-12 19:07:37 -04:00
alpayariyak
db7167d57f
0.3.3
2024-03-05 19:14:35 +00:00
Alpay Ariyak and alpayariyak
d91ccb866f
0.3.1: bug fixes
2024-02-29 02:55:44 -05:00
Alpay Ariyak and GitHub
36e9b670ee
Add notice on what to do when HuggingFace is down
2024-02-28 17:10:39 -05:00
Alpay Ariyak and alpayariyak
91167b873a
v0.3.0: OpenAI Compatibility, Dynamic Stream Batching, Refactor, Error Catching
2024-02-23 22:34:00 -05:00
Alpay Ariyak and GitHub
6f5718f191
Update Docker image tag in README.md
2024-02-23 02:15:50 -05:00
alpayariyak
708f68d7f8
Final Bug fixes, configurable oai response role, served model name override, documentation
2024-02-23 04:14:03 +00:00
alpayariyak
a2d9535652
Update documentation further
2024-02-23 01:34:38 +00:00
alpayariyak
9129d0a252
New ENV Vars
2024-02-22 23:58:20 +00:00
alpayariyak
6bcd9d7c67
Preparing for 0.3.0
2024-02-22 18:04:34 -05:00
Patrick Rachford and GitHub
7993818f5f
Update README.md
...
Remove formatting of tables
2024-02-21 19:28:03 -08:00
Patrick Rachford and GitHub
e97917cc14
Fixes import statement
...
Fixes import statements, formats tables, run black on code blocks
2024-02-21 19:17:23 -08:00
Alpay Ariyak and GitHub
3549cf24d5
Merge branch 'main' into openai-sse-output
2024-02-21 19:56:55 -05:00
alpayariyak
aed0408f19
Documentation for 0.3.0, small fixes and changes
2024-02-21 19:28:59 -05:00
Alpay Ariyak and GitHub
2941db0fb8
Update worker-vllm version
2024-02-09 23:12:29 -05:00
alpayariyak
8de10468dd
Working tokenizer and model download fix
...
Fix handler startup
2024-02-02 19:55:35 -05:00
alpayariyak
b1720a154d
Added download of model extras into weights folder, separate download of tokenizer, making engine.py utilize downloaded tokenizer, model and tokenizer revision
2024-01-31 22:57:32 -05:00
alpayariyak
370698442c
Update RunPod SDK version and Docker Tag
2024-01-31 00:54:58 -05:00
alpayariyak
3e2cd080a2
Fix Tensor Parallel
2024-01-31 05:12:58 +00:00
alpayariyak
46eee12819
Simplify Tensor Parallel
2024-01-31 03:28:02 +00:00
alpayariyak
12d6f0778e
Fixed Model bake-in, added Custom Chat Templates, Custom Tokenizer
2024-01-31 03:13:21 +00:00
alpayariyak
97726372c0
Update release tag in README.md
2024-01-25 23:37:07 -05:00
alpayariyak
9fc8e1e54c
Non-streaming OpenAI Chat Completions
2024-01-25 23:24:18 -05:00
alpayariyak
4cebe66b36
0.2.0 Release
...
- You no longer need a linux-based machine or NVIDIA GPUs to build the worker.
- Over 3x lighter Docker image size.
- OpenAI Chat Completion output format (optional to use).
- Extremely fast image build time.
- Docker Secrets-protected Hugging Face token support for building the image with a model baked in without exposing your token.
- Support for `n` and `best_of` sampling parameters, which allow you to generate multiple responses from a single prompt.
- New environment variables for various configuration.
- vLLM Version: 0.2.7
2024-01-25 20:49:15 -05:00
alpayariyak
15cc7cd36f
Updated Documentation
2024-01-19 10:48:12 -05:00
alpayariyak
584852f0f6
Docker-Protected HF Token, Refactor, Better Documentation
2024-01-18 18:20:51 -05:00
pandyamarut