 Cyberes
|
44e692c9cf
|
remove debug print
|
2023-09-25 12:35:36 -06:00 |
|
Cyberes
|
1646a00987
|
implement streaming on openai, improve streaming, run DB logging in background thread
|
2023-09-25 12:30:40 -06:00 |
|
Cyberes
|
bbe5d5a8fe
|
improve openai endpoint, exclude system tokens more places
|
2023-09-25 09:32:23 -06:00 |
|
Cyberes
|
6459a1c91b
|
allow setting simultaneous IP limit per-token, fix token use tracker, fix tokens on streaming
|
2023-09-25 00:55:20 -06:00 |
|
Cyberes
|
d2651756df
|
update requirements.txt
|
2023-09-24 21:46:48 -06:00 |
|
Cyberes
|
320f51e01c
|
further align openai endpoint with expected responses
|
2023-09-24 21:45:30 -06:00 |
|
Cyberes
|
84ea2f8891
|
handle when auth token is not enabled
|
2023-09-24 15:57:39 -06:00 |
|
Cyberes
|
8d6b2ce49c
|
minor changes, add admin token auth system, add route to get backend info
|
2023-09-24 15:54:35 -06:00 |
|
Cyberes
|
2678102153
|
handle error while streaming
|
2023-09-24 13:27:27 -06:00 |
|
Cyberes
|
cb99c3490e
|
rewrite tokenizer, restructure validation
|
2023-09-24 13:02:30 -06:00 |
|
Cyberes
|
62412f4873
|
add config setting for hostname
|
2023-09-23 23:24:08 -06:00 |
|
Cyberes
|
84a1fcfdd8
|
don't store host if it's an IP
|
2023-09-23 23:14:22 -06:00 |
|
Cyberes
|
0015e653b2
|
adjust a few final things
|
2023-09-23 22:30:59 -06:00 |
|
Cyberes
|
fab7b7ccdd
|
active gen workers wait
|
2023-09-23 21:17:13 -06:00 |
|
Cyberes
|
7ee2311183
|
whats going on
|
2023-09-23 21:10:14 -06:00 |
|
Cyberes
|
94e845cd1a
|
if there's less than num concurrent wait time is 0
|
2023-09-23 21:09:21 -06:00 |
|
Cyberes
|
41e622d19c
|
fix two exceptions
|
2023-09-23 20:55:49 -06:00 |
|
Cyberes
|
0530fa9870
|
getting sql error
|
2023-09-23 19:08:30 -06:00 |
|
Cyberes
|
b0beb7efeb
|
fix exception
|
2023-09-23 18:55:52 -06:00 |
|
Cyberes
|
f67ac8175b
|
fix wrong approach for streaming
|
2023-09-23 18:44:07 -06:00 |
|
Cyberes
|
8a4de7df44
|
oops
|
2023-09-23 18:01:12 -06:00 |
|
Cyberes
|
76a1428ba0
|
implement streaming for vllm
|
2023-09-23 17:57:23 -06:00 |
|
Cyberes
|
81452ec643
|
adjust vllm info
|
2023-09-21 20:13:29 -06:00 |
|
Cyberes
|
f9a80f3028
|
change proompters 1 min to 5 min
|
2023-09-20 21:21:22 -06:00 |
|
Cyberes
|
8593198216
|
close mysql cursor
|
2023-09-20 21:19:26 -06:00 |
|
Cyberes
|
9341492a8a
|
fix json error
|
2023-09-20 21:14:12 -06:00 |
|
Cyberes
|
03e3ec5490
|
port to mysql, use vllm tokenizer endpoint
|
2023-09-20 20:30:31 -06:00 |
|
Cyberes
|
2d390e6268
|
*blushes* oopsie daisy
|
2023-09-17 20:22:17 -06:00 |
|
Cyberes
|
eb3179cfff
|
fix recent proompters to work with gunicorn
|
2023-09-17 19:06:53 -06:00 |
|
Cyberes
|
3c1254d3bf
|
cache stats in background
|
2023-09-17 18:55:36 -06:00 |
|
Cyberes
|
edf13db324
|
calculate estimateed wate time better
|
2023-09-17 18:33:57 -06:00 |
|
Cyberes
|
7434ae1b5b
|
openai: improve moderation checking
|
2023-09-17 17:40:05 -06:00 |
|
Cyberes
|
354ad8192d
|
fix division by 0, prettify /stats json, add js var to home
|
2023-09-16 17:37:43 -06:00 |
|
Cyberes
|
77edbe779c
|
actually validate prompt length lol
|
2023-09-14 18:31:13 -06:00 |
|
Cyberes
|
3100b0a924
|
set up queue to work with gunicorn processes, other improvements
|
2023-09-14 17:38:20 -06:00 |
|
Cyberes
|
5d03f875cb
|
adjust prompt
|
2023-09-14 15:43:04 -06:00 |
|
Cyberes
|
1cf4c95ba2
|
ah, oops
|
2023-09-14 15:14:59 -06:00 |
|
Cyberes
|
a89295193f
|
add moderation endpoint to openai api, update config
|
2023-09-14 15:07:17 -06:00 |
|
Cyberes
|
8f4f17166e
|
adjust
|
2023-09-14 14:36:22 -06:00 |
|
Cyberes
|
507327db49
|
lower caching of home page
|
2023-09-14 14:30:01 -06:00 |
|
Cyberes
|
93a344f4c5
|
check if the backend crapped out, print some more stuff
|
2023-09-14 14:26:25 -06:00 |
|
Cyberes
|
79b1e01b61
|
option to disable streaming, improve timeout on requests to backend, fix error handling. reduce duplicate code, misc other cleanup
|
2023-09-14 14:05:50 -06:00 |
|
Cyberes
|
e79b206e1a
|
rename average_tps to estimated_avg_tps
|
2023-09-14 01:35:25 -06:00 |
|
Cyberes
|
c45e68a8c8
|
adjust requests timeout, add service file
|
2023-09-14 01:32:49 -06:00 |
|
Cyberes
|
035c17c48b
|
reformat info page info_html field
|
2023-09-13 20:40:55 -06:00 |
|
Cyberes
|
15a0390875
|
typo
|
2023-09-13 20:27:20 -06:00 |
|
Cyberes
|
12e894032e
|
show the openai system prompt
|
2023-09-13 20:25:56 -06:00 |
|
Cyberes
|
320c3fc710
|
calculate time stats based on backend url
|
2023-09-13 12:34:14 -06:00 |
|
Cyberes
|
3d40ed4cfb
|
shit code
|
2023-09-13 11:58:38 -06:00 |
|
Cyberes
|
1582625e09
|
how did this get broken
|
2023-09-13 11:56:30 -06:00 |