5 Commits

Author	SHA1	Message	Date
Cyberes	e5fbc9545d	add ratelimiting to websocket streaming endpoint, fix queue not decrementing IP requests, add console printer	2023-09-27 21:15:54 -06:00
Cyberes	1d9f40765e	remove text-generation-inference backend	2023-09-12 13:09:47 -06:00
Cyberes	4c9d543eab	implement vllm backend	2023-09-11 20:47:19 -06:00
Cyberes	0aa52863bc	forgot to start workers	2023-08-23 20:33:49 -06:00
Cyberes	6f8b70df54	add a queue system	2023-08-23 20:12:38 -06:00