Cyberes
|
4c9d543eab
|
implement vllm backend
|
2023-09-11 20:47:19 -06:00 |
Cyberes
|
ba0bc87434
|
add HF text-generation-inference backend
|
2023-08-29 13:46:41 -06:00 |
Cyberes
|
6c0e60135d
|
exclude tokens with priority 0 from simultaneous requests ratelimit
|
2023-08-28 00:03:25 -06:00 |
Cyberes
|
c16d70a24d
|
limit amount of simultaneous requests an IP can make
|
2023-08-27 23:48:10 -06:00 |
Cyberes
|
11a0b6541f
|
fix some stuff related to gunicorn workers
|
2023-08-23 22:01:06 -06:00 |
Cyberes
|
de19af900f
|
add estimated wait time and other time tracking stats
|
2023-08-23 21:33:52 -06:00 |
Cyberes
|
0aa52863bc
|
forgot to start workers
|
2023-08-23 20:33:49 -06:00 |
Cyberes
|
6f8b70df54
|
add a queue system
|
2023-08-23 20:12:38 -06:00 |