owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
257 Commits 1 Branch 69 Tags
37d140aba645ba4419bc308c549d79bfd3b63afa
Commit Graph
257 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
alpayariyak b705544bd4 model packing, cuda version selection on build 2023-12-06 18:52:21 +00:00
alpayariyak 48c520b240 Version Bump + small changes 2023-12-06 16:48:12 +00:00
alpayariyak 66c23d45b0 Model Downloading, no GPU 2023-12-06 01:43:09 +00:00
alpayariyak b4c61e61c4 Packing model into worker for internal use 2023-12-05 20:26:56 -05:00
alpayariyak 3a9393bafd Batched Tokens, cleanup 2023-12-05 17:46:28 +00:00
alpayariyak 8d27261645 Logging change 2023-12-01 10:18:48 +00:00
alpayariyak b242b0cedd Refactor handler + add non-streaming, add utils.py 2023-12-01 10:12:56 +00:00
alpayariyak daf2f4d07b Make handler stream aggregate_text 2023-12-01 04:21:03 +00:00
Justin Merrell 8dbf197362 Update handler.py 2023-11-30 11:00:16 -05:00
alpayariyak 86916bf628 Update handler.py 2023-11-29 23:37:17 -05:00
Justin Merrell f958014d10 Update handler.py 2023-11-29 22:41:59 -05:00
Justin Merrell bd3ede60f3 Update Dockerfile 2023-11-29 22:32:30 -05:00
Justin Merrell 4131b7a16d Update Dockerfile 2023-11-29 22:32:13 -05:00
Justin Merrell c455508931 Update handler.py 2023-11-29 22:01:17 -05:00
Justin Merrell 9b9c1cf503 cleanup 2023-11-29 21:32:40 -05:00
alpayariyak 820a21f32c Download logic + other changes 2023-11-28 19:15:08 -05:00
alpayariyak fdddeacf3c Update handler.py 2023-11-28 18:49:15 -05:00
alpayariyak 7cadff29f5 edge case 2023-11-28 18:49:04 -05:00
alpayariyak 97480ff0ae handler changes 2023-11-28 23:39:48 +00:00
alpayariyak 7f73f41c9f Merge remote-tracking branch 'origin/new-concurrency' into revamp2 2023-11-26 18:23:50 -05:00
Justin Merrell 98af834124 fix: added hf_transfer requirement 2023-11-21 20:50:12 -05:00
Justin Merrell d06fb9ea01 Update Dockerfile 2023-11-21 20:20:15 -05:00
Justin Merrell ff3ed55c9f Update requirements.txt 2023-11-21 20:14:51 -05:00
Justin Merrell cbde2cbf9c Update download_model.py 2023-11-21 20:10:37 -05:00
Ubuntu 12a151d184 Merge branch 'revamp2' of https://github.com/runpod-workers/worker-vllm into revamp2 2023-11-22 00:41:48 +00:00
Alpay AriyakandUbuntu 96612413ce worker changes 2023-11-22 00:39:52 +00:00
Ubuntu c1b9ff1bc9 worker changes 2023-11-22 00:38:29 +00:00
Justin Merrell 5ca8e56bbe fix: clean up extra 2023-11-21 19:18:08 -05:00
alpayariyak c34d79ea49 Entrypoint for testing 2023-11-20 18:56:00 -05:00
alpayariyak 8f17764f8a Update Dockerfile 2023-11-15 20:55:17 -05:00
alpayariyak e7c8de4678 Changes to worker 2023-11-15 20:53:04 -05:00
Alpay AriyakandGitHub 24feadd903 Update README.md 2023-11-14 19:25:41 -05:00
Samuel Willandalpayariyak 4f792062aa vllm 0.2.1.post1 - speed boost, quantization, mistral support 2023-11-14 19:12:58 -05:00
Jorg Doku 5bccd40952 using vllm 0.1.7 2023-09-21 09:41:59 -05:00
Jorg Doku ba4e6e6df0 fix aggre 2023-09-05 23:51:35 -05:00
Jorg Doku 871196dafb update 2023-09-05 22:06:46 -05:00
Jorg Doku 5e04e5ebb0 print out llm metrics, make metrics more verbose 2023-08-31 19:09:04 -05:00
Jorg Doku 06183813cf take advantage of layering 2023-08-31 18:30:59 -05:00
Jorg Doku 54d229cb16 upate metrics 2023-08-31 18:26:37 -05:00
Jorg Doku 84d6047cbf update 2023-08-31 17:41:39 -05:00
Jorg Doku 96c4d70d2e metrics now included 2023-08-31 17:03:59 -05:00
Jorg Doku c5662663ce update response 2023-08-31 16:00:28 -05:00
Jorg Doku 9f1dd5a9b1 finished 2023-08-31 15:56:13 -05:00
Jorg Doku 9eb099ded2 fix bug 2023-08-31 14:42:18 -05:00
Jorg Doku 460f3dd348 update the handler 2023-08-31 13:48:57 -05:00
Jorg Doku 50689a2caa test 2023-08-30 17:57:46 -05:00
Jorg Doku e73e108193 reduce logs 2023-08-30 12:13:20 -05:00
Jorg Doku be4421e8e2 finalize 2023-08-30 02:33:51 -05:00
Jorg Doku 66056e694c finalize 2023-08-30 01:58:27 -05:00
Jorg Doku 5e4865e708 complete 2023-08-30 00:08:11 -05:00