Commit Graph

38 Commits

Author SHA1 Message Date
Cyberes 0059e7956c Merge cluster to master (#3)
Co-authored-by: Cyberes <cyberes@evulid.cc>
Reviewed-on: #3
2023-10-27 19:19:22 -06:00
Cyberes d9bbcc42e6 more work on openai endpoint 2023-09-26 22:09:11 -06:00
Cyberes e0af2ea9c5 convert to gunicorn 2023-09-26 13:32:33 -06:00
Cyberes b44dda7a3a option to show SYSTEM tokens in stats 2023-09-25 23:39:50 -06:00
Cyberes 2d299dbae5 openai_force_no_hashes 2023-09-25 22:01:57 -06:00
Cyberes 135bd743bb fix homepage slowness, fix incorrect 24 hr prompters, fix redis wrapper, 2023-09-25 17:20:21 -06:00
Cyberes bbe5d5a8fe improve openai endpoint, exclude system tokens more places 2023-09-25 09:32:23 -06:00
Cyberes 8d6b2ce49c minor changes, add admin token auth system, add route to get backend info 2023-09-24 15:54:35 -06:00
Cyberes 84a1fcfdd8 don't store host if it's an IP 2023-09-23 23:14:22 -06:00
Cyberes 03e3ec5490 port to mysql, use vllm tokenizer endpoint 2023-09-20 20:30:31 -06:00
Cyberes 3c1254d3bf cache stats in background 2023-09-17 18:55:36 -06:00
Cyberes 3100b0a924 set up queue to work with gunicorn processes, other improvements 2023-09-14 17:38:20 -06:00
Cyberes a89295193f add moderation endpoint to openai api, update config 2023-09-14 15:07:17 -06:00
Cyberes 79b1e01b61 option to disable streaming, improve timeout on requests to backend, fix error handling. reduce duplicate code, misc other cleanup 2023-09-14 14:05:50 -06:00
Cyberes 12e894032e show the openai system prompt 2023-09-13 20:25:56 -06:00
Cyberes e053f48fdc change gpt4 prompt 2023-09-12 16:47:08 -06:00
Cyberes 9740df07c7 add openai-compatible backend 2023-09-12 16:40:09 -06:00
Cyberes 6152b1bb66 fix invalid param error, add manual model name 2023-09-12 10:30:45 -06:00
Cyberes 4c9d543eab implement vllm backend 2023-09-11 20:47:19 -06:00
Cyberes 2816c01902 refactor generation route 2023-08-30 18:53:26 -06:00
Cyberes bf648f605f implement streaming for hf-textgen 2023-08-29 17:56:12 -06:00
Cyberes ba0bc87434 add HF text-generation-inference backend 2023-08-29 13:46:41 -06:00
Cyberes c16d70a24d limit amount of simultaneous requests an IP can make 2023-08-27 23:48:10 -06:00
Cyberes d64152587c reorganize nvidia stats 2023-08-25 15:02:40 -06:00
Cyberes 0230ddda17 dynamically fetch GPUs for netdata 2023-08-24 21:56:15 -06:00
Cyberes 16b986c206 track nvidia power states through netdata 2023-08-24 21:36:00 -06:00
Cyberes ec3fe2c2ac show total output tokens on stats 2023-08-24 20:43:11 -06:00
Cyberes 763dd832cc update home, update readme, calculate estimated wait based on database stats 2023-08-24 16:47:14 -06:00
Cyberes f3fe514c11 add home template 2023-08-23 23:11:12 -06:00
Cyberes 3317bd5f1a allow hiding of more variables 2023-08-23 22:08:10 -06:00
Cyberes 9fc674878d allow disabling ssl verification 2023-08-23 16:11:32 -06:00
Cyberes 06ae8adf0d add backend name to error messages 2023-08-22 21:14:12 -06:00
Cyberes a525093c75 rename, more stats 2023-08-22 20:42:38 -06:00
Cyberes a9b7a7a2c7 display error messages in sillytavern 2023-08-22 20:28:41 -06:00
Cyberes a59dcea2da more proxy stats 2023-08-22 16:50:49 -06:00
Cyberes ad9a91f1b5 concurrent gens setting, online status 2023-08-22 00:26:46 -06:00
Cyberes e04d6a8a13 minor adjustments 2023-08-21 22:49:44 -06:00
Cyberes 8cbf643fd3 MVP 2023-08-21 21:28:52 -06:00