6 Commits

Author	SHA1	Message	Date
Cyberes	76a1428ba0	implement streaming for vllm	2023-09-23 17:57:23 -06:00
Cyberes	03e3ec5490	port to mysql, use vllm tokenizer endpoint	2023-09-20 20:30:31 -06:00
Cyberes	9740df07c7	add openai-compatible backend	2023-09-12 16:40:09 -06:00
Cyberes	ba063f7f1b	caching	2023-08-23 12:40:13 -06:00
Cyberes	e04d6a8a13	minor adjustments	2023-08-21 22:49:44 -06:00
Cyberes	8cbf643fd3	MVP	2023-08-21 21:28:52 -06:00