7 Commits

Include renames

Author	SHA1	Message	Date
alpayariyak	f324bef8a0	Bump to vLLM 0.2.6, fixes and improvements	2023-12-19 01:52:55 +00:00
alpayariyak	bfa3703d54	Concurrency modifier	2023-12-13 12:26:01 +00:00
alpayariyak	22de0b322d	Complete refactor, improved overall functionality	2023-12-13 10:33:03 +00:00
alpayariyak	5acbbf5528	Cleanup	2023-12-06 20:57:48 +00:00
alpayariyak	3a9393bafd	Batched Tokens, cleanup	2023-12-05 17:46:28 +00:00
alpayariyak	8d27261645	Logging change	2023-12-01 10:18:48 +00:00
alpayariyak	b242b0cedd	Refactor handler + add non-streaming, add utils.py	2023-12-01 10:12:56 +00:00