Commit Graph

Select branches

Hide Pull Requests

main

0.1.0

0.2.0

0.2.1

0.2.2

0.2.3

0.3.0

0.3.1

0.3.2

1.0.0

1.0.1

v.2.15.0

v.2.16.0

v.2.2.1

v1.1

v1.2.0

v1.3.0

v1.3.1

v1.4.0

v1.5.0

v1.6.0

v1.7.0

v1.8.0

v1.9.0

v2.1.0

v2.10.0

v2.11.0

v2.11.1

v2.11.2

v2.11.3

v2.12.0

v2.12.1

v2.12.2

v2.12.3

v2.13.0

v2.13.1

v2.14.0

v2.17.0

v2.18.0

v2.18.1

v2.19.0

v2.19.1

v2.19.2

v2.2.0

v2.2.2

v2.20.0

v2.20.1

v2.21.1

v2.22.0

v2.22.1

v2.22.2

v2.22.3

v2.22.4

v2.22.5

v2.23.0

v2.3.0

v2.3.1

v2.3.2

v2.4.0

v2.5.0

v2.5.1

v2.6.0

v2.8.0

v2.9.0

v2.9.1

v2.9.2

v2.9.3

v2.9.4

v2.9.5

v2.9.6

Mono Color

• 8dbd8f15dd Token counting, cuda-related changes alpayariyak 2023-12-06 19:02:12 +00:00
• b705544bd4 model packing, cuda version selection on build alpayariyak 2023-12-06 18:52:21 +00:00
• 48c520b240 Version Bump + small changes alpayariyak 2023-12-06 16:48:12 +00:00
• 66c23d45b0 Model Downloading, no GPU alpayariyak 2023-12-06 01:43:09 +00:00
• b4c61e61c4 Packing model into worker for internal use alpayariyak 2023-12-05 20:26:56 -05:00
• 3a9393bafd Batched Tokens, cleanup alpayariyak 2023-12-05 17:46:28 +00:00
• 8d27261645 Logging change alpayariyak 2023-12-01 10:18:48 +00:00
• b242b0cedd Refactor handler + add non-streaming, add utils.py alpayariyak 2023-12-01 10:12:56 +00:00
• daf2f4d07b Make handler stream aggregate_text alpayariyak 2023-12-01 04:21:03 +00:00
• 8dbf197362 Update handler.py Justin Merrell 2023-11-30 11:00:16 -05:00
• 86916bf628 Update handler.py alpayariyak 2023-11-29 23:37:17 -05:00
• f958014d10 Update handler.py Justin Merrell 2023-11-29 22:41:59 -05:00
• bd3ede60f3 Update Dockerfile Justin Merrell 2023-11-29 22:32:30 -05:00
• 4131b7a16d Update Dockerfile Justin Merrell 2023-11-29 22:32:13 -05:00
• c455508931 Update handler.py Justin Merrell 2023-11-29 22:01:17 -05:00
• 9b9c1cf503 cleanup Justin Merrell 2023-11-29 21:32:40 -05:00
• 820a21f32c Download logic + other changes alpayariyak 2023-11-28 19:15:08 -05:00
• fdddeacf3c Update handler.py alpayariyak 2023-11-28 18:49:15 -05:00
• 7cadff29f5 edge case alpayariyak 2023-11-28 23:46:59 +00:00
• 97480ff0ae handler changes alpayariyak 2023-11-28 23:39:48 +00:00
• 7f73f41c9f Merge remote-tracking branch 'origin/new-concurrency' into revamp2 alpayariyak 2023-11-26 18:23:50 -05:00
• 98af834124 fix: added hf_transfer requirement Justin Merrell 2023-11-21 20:50:12 -05:00
• d06fb9ea01 Update Dockerfile Justin Merrell 2023-11-21 20:20:15 -05:00
• ff3ed55c9f Update requirements.txt Justin Merrell 2023-11-21 20:14:51 -05:00
• cbde2cbf9c Update download_model.py Justin Merrell 2023-11-21 20:10:37 -05:00
• 12a151d184 Merge branch 'revamp2' of https://github.com/runpod-workers/worker-vllm into revamp2 Ubuntu 2023-11-22 00:41:48 +00:00
• 96612413ce worker changes Alpay AriyakandUbuntu 2023-11-22 00:38:29 +00:00
• c1b9ff1bc9 worker changes Ubuntu 2023-11-22 00:38:29 +00:00
• 5ca8e56bbe fix: clean up extra Justin Merrell 2023-11-21 19:18:08 -05:00
• c34d79ea49 Entrypoint for testing alpayariyak 2023-11-20 18:56:00 -05:00
• 8f17764f8a Update Dockerfile alpayariyak 2023-11-15 20:55:17 -05:00
• e7c8de4678 Changes to worker alpayariyak 2023-11-15 20:53:04 -05:00
• 24feadd903 Update README.md Alpay AriyakandGitHub 2023-11-14 19:25:41 -05:00
• 4f792062aa vllm 0.2.1.post1 - speed boost, quantization, mistral support Samuel Willandalpayariyak 2023-11-03 10:19:17 +02:00
• 5bccd40952 using vllm 0.1.7 Jorg Doku 2023-09-21 09:41:59 -05:00
• ba4e6e6df0 fix aggre Jorg Doku 2023-09-05 23:51:35 -05:00
• 871196dafb update Jorg Doku 2023-09-05 22:06:46 -05:00
• 5e04e5ebb0 print out llm metrics, make metrics more verbose Jorg Doku 2023-08-31 19:09:04 -05:00
• 06183813cf take advantage of layering Jorg Doku 2023-08-31 18:30:59 -05:00
• 54d229cb16 upate metrics Jorg Doku 2023-08-31 18:26:37 -05:00
• 84d6047cbf update Jorg Doku 2023-08-31 17:41:39 -05:00
• 96c4d70d2e metrics now included Jorg Doku 2023-08-31 17:03:59 -05:00
• c5662663ce update response Jorg Doku 2023-08-31 16:00:28 -05:00
• 9f1dd5a9b1 finished Jorg Doku 2023-08-31 15:56:13 -05:00
• 9eb099ded2 fix bug Jorg Doku 2023-08-31 14:42:18 -05:00
• 460f3dd348 update the handler Jorg Doku 2023-08-31 13:48:57 -05:00
• 50689a2caa test Jorg Doku 2023-08-30 17:57:46 -05:00
• e73e108193 reduce logs Jorg Doku 2023-08-30 12:13:20 -05:00
• be4421e8e2 finalize Jorg Doku 2023-08-30 02:33:51 -05:00
• 66056e694c finalize Jorg Doku 2023-08-30 01:58:27 -05:00
• 5e4865e708 complete Jorg Doku 2023-08-30 00:08:11 -05:00
• 476498b9bc fix metrics Jorg Doku 2023-08-29 22:17:50 -05:00
• 5fd458381d win Jorg Doku 2023-08-29 21:29:51 -05:00
• 5b112b30c2 update handler Jorg Doku 2023-08-29 21:22:01 -05:00
• 244ce53679 fix Jorg Doku 2023-08-29 20:41:50 -05:00
• f79c834268 fix Jorg Doku 2023-08-29 18:04:19 -05:00
• 73c55b87e4 fix Jorg Doku 2023-08-29 17:18:21 -05:00
• 62961ce773 update vllm version Jorg Doku 2023-08-29 16:37:24 -05:00
• acb120bf54 improving metrics in worker Jorg Doku 2023-08-29 11:07:25 -05:00
• 51d231c37a done Jorg Doku 2023-08-25 18:26:28 -05:00
• e3cdd46a99 update Jorg Doku 2023-08-25 12:03:43 -05:00
• d6cce5d5b6 update Jorg Doku 2023-08-24 15:46:30 -05:00
• 33ab5600bf update Jorg Doku 2023-08-24 13:58:38 -05:00
• f4e8c84695 update Jorg Doku 2023-08-22 15:44:28 -05:00
• 5dd9555306 update Jorg Doku 2023-08-22 15:44:02 -05:00
• 6498c67340 update Jorg Doku 2023-08-22 15:40:09 -05:00
• 177c8c57c0 update Jorg Doku 2023-08-22 15:36:43 -05:00
• eb1bcdd049 incorporating metrics into the vLLM worker Jorg Doku 2023-08-21 17:34:51 -05:00
• 9bf9039151 updating the llm worker Jorg Doku 2023-08-18 09:40:02 -05:00
• 3838e54a12 update Jorg Doku 2023-08-09 10:42:44 -05:00
• 68a4e3c874 add Jorg Doku 2023-08-09 10:39:22 -05:00
• 1976bf19bf Merge remote-tracking branch 'refs/remotes/origin/main' Jorg Doku 2023-08-09 10:37:53 -05:00
• 3ab0770b1b including env variable for tensor parallelism / multi-gpu Jorg Doku 2023-08-09 10:37:43 -05:00
• f8fba87ce6 Merge pull request #4 from vladmihaisima/main Jorg DokuandGitHub 2023-08-08 10:53:20 -05:00
• 58324fed8f Fixed STREAMING environment variable not being interpreted as boolean, when set in environment to 'False'. Vlad-Mihai Sima 2023-08-08 15:01:03 +02:00
• 72be7ff31d update Jorg Doku 2023-08-07 11:28:36 -05:00
• fdfe69c717 Merge remote-tracking branch 'refs/remotes/origin/main' Jorg Doku 2023-08-06 14:20:00 -05:00
• 4e4d3a27fc fixed Jorg Doku 2023-08-06 14:17:53 -05:00
• 9e836cc6d1 Update README.md Jorg DokuandGitHub 2023-08-03 10:48:40 -05:00
• 04baa8903d Update README.md Jorg DokuandGitHub 2023-08-03 10:45:57 -05:00
• 53f9967ea4 Update README.md Jorg DokuandGitHub 2023-08-03 10:45:32 -05:00
• 569b3059f0 Update README.md Jorg DokuandGitHub 2023-08-03 10:33:52 -05:00
• da05702372 Update README.md Jorg DokuandGitHub 2023-08-03 10:31:56 -05:00
• 7b1e0c29d7 Update README.md Jorg DokuandGitHub 2023-08-03 10:30:27 -05:00
• 00477f2131 Update README.md Jorg DokuandGitHub 2023-08-03 10:29:14 -05:00
• 401a4637fa latest version Jorg Doku 2023-08-03 10:05:51 -05:00
• 728e4f4f2e updating the handler Jorg Doku 2023-07-31 14:04:34 -05:00
• b78a3d4429 update Jorg Doku 2023-07-31 14:01:38 -05:00
• c862d043dc latest Jorg Doku 2023-07-28 23:14:03 -05:00
• 67b6cb8b33 Merge remote-tracking branch 'refs/remotes/origin/main' Jorg Doku 2023-07-20 18:45:26 -05:00
• 9fe114c049 llama 2 Jorg Doku 2023-07-20 18:45:08 -05:00
• 9c84810977 Update dev Jorg DokuandGitHub 2023-07-20 08:46:57 -05:00
• ff59fb6b8f this version works! Jorg Doku 2023-07-19 09:21:42 -05:00
• 9046fc811b win Jorg Doku 2023-07-18 22:04:35 -05:00
• 2c186072d2 updating serverless worker to include http streaming Jorg Doku 2023-07-17 16:28:55 -05:00
• 57d7d7b589 update worker Jorg Doku 2023-07-17 03:36:07 -05:00
• 56e91ea42a update handler logic Jorg Doku 2023-07-14 20:56:58 -05:00
• ab3f0e1c58 update worker Jorg Doku 2023-07-14 11:17:25 -05:00
• be65b7a99c update the worker Jorg Doku 2023-07-14 00:46:14 -05:00
• 488a9a9451 temp Jorg Doku 2023-07-05 22:34:26 -05:00