 Cyberes
|
ee44371fdf
|
Merge branch 'master' into cluster
|
2023-10-27 19:05:27 -06:00 |
|
Cyberes
|
28c250385d
|
add todo
|
2023-10-27 19:00:49 -06:00 |
|
Cyberes
|
563630547a
|
add robots.txt
|
2023-10-23 17:32:33 -06:00 |
|
Cyberes
|
b4e01e129d
|
fix when all offline
|
2023-10-23 17:28:59 -06:00 |
|
Cyberes
|
96ba48affc
|
make sure to regen stats on startup
|
2023-10-23 17:26:15 -06:00 |
|
Cyberes
|
177dabd209
|
Give some time for the background threads to get themselves ready to go
|
2023-10-23 17:25:38 -06:00 |
|
Cyberes
|
0771c2325c
|
fix inference workers quitting when a backend is offline, start adding logging, improve tokenizer error handling
|
2023-10-23 17:24:20 -06:00 |
|
Cyberes
|
3cf73fec9b
|
fix a few exceptions when all backends go offline
|
2023-10-23 15:22:57 -06:00 |
|
Cyberes
|
d43f110a14
|
fix redis cycle and add no reset to daemon
|
2023-10-22 12:19:20 -06:00 |
|
Cyberes
|
e236e93a79
|
clean up a bit
|
2023-10-20 18:26:41 -06:00 |
|
Cyberes
|
f39e976b34
|
dameon printer: Calculate the queue size the same way it's done on the stats
|
2023-10-20 17:41:47 -06:00 |
|
Cyberes
|
1a15232400
|
tests: make sure all prompts are the same
|
2023-10-20 17:35:21 -06:00 |
|
Cyberes
|
763139c949
|
fix keyerror
|
2023-10-20 14:02:30 -06:00 |
|
Cyberes
|
e838f591aa
|
fix keyerror?
|
2023-10-20 14:00:24 -06:00 |
|
Cyberes
|
0abd4b94fb
|
track down keyerror
|
2023-10-20 13:57:20 -06:00 |
|
Cyberes
|
4f5b2dbecb
|
add tests
|
2023-10-20 12:14:34 -06:00 |
|
Cyberes
|
6e74ce7c28
|
fix old code in completions
|
2023-10-19 17:59:27 -06:00 |
|
Cyberes
|
b9566e9db7
|
docs and stuff
|
2023-10-18 09:23:54 -06:00 |
|
Cyberes
|
56a2ca464b
|
change print
|
2023-10-18 09:12:30 -06:00 |
|
Cyberes
|
50377eca22
|
track lag on get_ip_request_count()
|
2023-10-18 09:09:22 -06:00 |
|
Cyberes
|
92e4ecd8a1
|
refer to queue for tracking IP count rather than seperate value
|
2023-10-18 09:03:10 -06:00 |
|
Cyberes
|
be03569165
|
use backend handler to build parameters when sending test prompt
|
2023-10-17 12:42:48 -06:00 |
|
Cyberes
|
90adffaec8
|
test
|
2023-10-17 12:32:41 -06:00 |
|
Cyberes
|
4c2c164ce1
|
test
|
2023-10-17 12:29:12 -06:00 |
|
Cyberes
|
2fed87d340
|
remove timed-out items from queue
|
2023-10-17 11:46:39 -06:00 |
|
Cyberes
|
7998cfca87
|
cleanup
|
2023-10-16 23:47:34 -06:00 |
|
Cyberes
|
2ed0e01db6
|
background thread
|
2023-10-16 23:44:11 -06:00 |
|
Cyberes
|
6f65791795
|
adjust
|
2023-10-16 23:40:07 -06:00 |
|
Cyberes
|
9e3cbc9d2e
|
fix streaming slowdown?
|
2023-10-16 23:36:25 -06:00 |
|
Cyberes
|
c3c053e071
|
test
|
2023-10-16 23:29:17 -06:00 |
|
Cyberes
|
70cf6843e5
|
update requiorements
|
2023-10-16 23:12:41 -06:00 |
|
Cyberes
|
806e522d16
|
don't pickle streaming
|
2023-10-16 18:35:10 -06:00 |
|
Cyberes
|
81baf9616f
|
revert
|
2023-10-16 18:16:19 -06:00 |
|
Cyberes
|
21755450a3
|
test
|
2023-10-16 18:10:21 -06:00 |
|
Cyberes
|
1e68e10b62
|
fix GeneratorExit
|
2023-10-16 18:04:49 -06:00 |
|
Cyberes
|
20047fa0e4
|
2000 chunk size
|
2023-10-16 18:01:17 -06:00 |
|
Cyberes
|
19a193b792
|
increase tokenization chunk size
|
2023-10-16 17:59:21 -06:00 |
|
Cyberes
|
f421436048
|
add nginx config
|
2023-10-16 17:43:37 -06:00 |
|
Cyberes
|
2c7773cc4f
|
get streaming working again
|
2023-10-16 16:22:52 -06:00 |
|
Cyberes
|
151b3e4769
|
begin streaming rewrite
|
2023-10-16 00:18:05 -06:00 |
|
Cyberes
|
24aab3cd93
|
fix streaming disabled
|
2023-10-15 20:59:11 -06:00 |
|
Cyberes
|
381bdb950f
|
remove debug print
|
2023-10-15 20:46:32 -06:00 |
|
Cyberes
|
31ab4188f1
|
fix issues with queue and streaming
|
2023-10-15 20:45:01 -06:00 |
|
Cyberes
|
3ec9b2347f
|
fix wrong datatype
|
2023-10-15 17:24:18 -06:00 |
|
Cyberes
|
b3f0c4b28f
|
remove debug print
|
2023-10-15 15:14:32 -06:00 |
|
Cyberes
|
83f3ba8919
|
trying to fix workers still processing after backend goes offline
|
2023-10-15 15:11:37 -06:00 |
|
Cyberes
|
ca7044bc90
|
update gradio chat
|
2023-10-11 18:49:31 -06:00 |
|
Cyberes
|
4e3985e156
|
fix wrong status code on openai streaming
|
2023-10-11 18:17:02 -06:00 |
|
Cyberes
|
74cf8f309b
|
clean up
|
2023-10-11 18:04:15 -06:00 |
|
Cyberes
|
169e216a38
|
add background thread to gradio
|
2023-10-11 12:56:04 -06:00 |
|
Cyberes
|
69b8c1e35c
|
fix openai confusion
|
2023-10-11 12:50:20 -06:00 |
|
Cyberes
|
1d1c45dc1a
|
add length penalty param to vllm
|
2023-10-11 12:22:50 -06:00 |
|
Cyberes
|
78114771b0
|
fix oai exception
|
2023-10-11 09:20:00 -06:00 |
|
Cyberes
|
7286e38cb0
|
t
|
2023-10-11 09:10:32 -06:00 |
|
Cyberes
|
f4e5b5275d
|
test
|
2023-10-11 09:09:41 -06:00 |
|
Cyberes
|
18e37a72ae
|
add model selection to openai endpoint
|
2023-10-09 23:51:26 -06:00 |
|
Cyberes
|
5f7bf4faca
|
misc changes
|
2023-10-09 18:12:12 -06:00 |
|
Cyberes
|
ae4d4e5ca9
|
fix exception
|
2023-10-09 10:31:35 -06:00 |
|
Cyberes
|
467e1893ea
|
fix issue with null data on openai
|
2023-10-08 19:36:12 -06:00 |
|
Cyberes
|
3e5feb9c97
|
fix stat
|
2023-10-05 21:43:49 -06:00 |
|
Cyberes
|
e8964fcfd2
|
fix the queue??
|
2023-10-05 21:37:18 -06:00 |
|
Cyberes
|
ea61766838
|
fix
|
2023-10-05 20:19:57 -06:00 |
|
Cyberes
|
da20d1807b
|
actually wait again
|
2023-10-05 20:17:42 -06:00 |
|
Cyberes
|
e9f6fdf65e
|
fix streaming?
|
2023-10-05 20:14:28 -06:00 |
|
Cyberes
|
67173f30dd
|
t
|
2023-10-05 19:35:12 -06:00 |
|
Cyberes
|
8df667bc0a
|
t
|
2023-10-05 19:25:08 -06:00 |
|
Cyberes
|
c4cc7bbaa0
|
f
|
2023-10-05 19:13:43 -06:00 |
|
Cyberes
|
0bef14ea55
|
t
|
2023-10-05 19:11:59 -06:00 |
|
Cyberes
|
5540112607
|
t
|
2023-10-05 19:09:25 -06:00 |
|
Cyberes
|
9befda5acb
|
c
|
2023-10-05 19:05:41 -06:00 |
|
Cyberes
|
50992116f5
|
fix
|
2023-10-05 18:59:19 -06:00 |
|
Cyberes
|
96dd62478f
|
fix
|
2023-10-05 18:53:19 -06:00 |
|
Cyberes
|
a37b12a221
|
t
|
2023-10-05 18:51:56 -06:00 |
|
Cyberes
|
46d44f95ac
|
t
|
2023-10-05 18:50:10 -06:00 |
|
Cyberes
|
817c454c89
|
t
|
2023-10-05 18:48:07 -06:00 |
|
Cyberes
|
9b819573e8
|
fix import error
|
2023-10-05 18:39:31 -06:00 |
|
Cyberes
|
e07e31df0a
|
fix
|
2023-10-05 18:37:50 -06:00 |
|
Cyberes
|
0718f10eb9
|
t
|
2023-10-05 18:36:02 -06:00 |
|
Cyberes
|
fb8bc05b4c
|
t
|
2023-10-05 18:30:54 -06:00 |
|
Cyberes
|
6be1e9acd3
|
t
|
2023-10-05 18:16:17 -06:00 |
|
Cyberes
|
10eb6269b7
|
t
|
2023-10-05 18:15:02 -06:00 |
|
Cyberes
|
64d7a9edbb
|
fix
|
2023-10-05 18:09:24 -06:00 |
|
Cyberes
|
5a61bdccd4
|
f
|
2023-10-05 18:07:59 -06:00 |
|
Cyberes
|
3d0a5cf0a2
|
t
|
2023-10-05 18:06:36 -06:00 |
|
Cyberes
|
01fb619b9b
|
f
|
2023-10-05 18:05:31 -06:00 |
|
Cyberes
|
a229b4d6c5
|
c
|
2023-10-05 18:04:29 -06:00 |
|
Cyberes
|
a53790ee37
|
fix???
|
2023-10-05 18:01:08 -06:00 |
|
Cyberes
|
84c1ed8737
|
t
|
2023-10-05 17:58:23 -06:00 |
|
Cyberes
|
979a945466
|
t
|
2023-10-05 17:55:44 -06:00 |
|
Cyberes
|
19e62be3e8
|
t
|
2023-10-05 17:41:01 -06:00 |
|
Cyberes
|
27e461c76b
|
test
|
2023-10-05 17:00:35 -06:00 |
|
Cyberes
|
08df52a4fd
|
fix exception when not valid model
|
2023-10-05 12:28:00 -06:00 |
|
Cyberes
|
acf409abfc
|
fix background logger, add gradio chat example
|
2023-10-04 19:24:47 -06:00 |
|
Cyberes
|
1670594908
|
fix import error
|
2023-10-04 16:29:19 -06:00 |
|
Cyberes
|
6723dd79dc
|
fix exceptoin
|
2023-10-04 16:04:03 -06:00 |
|
Cyberes
|
09fa69e031
|
fix
|
2023-10-04 13:37:39 -06:00 |
|
Cyberes
|
62d5d43da4
|
handle backend offline in tokenizer
|
2023-10-04 13:34:59 -06:00 |
|
Cyberes
|
7acaa3c885
|
g
|
2023-10-04 13:32:54 -06:00 |
|
Cyberes
|
d78ef652fc
|
c
|
2023-10-04 13:21:43 -06:00 |
|
Cyberes
|
5e90fa54d4
|
handle model offline
|
2023-10-04 13:18:47 -06:00 |
|
Cyberes
|
754a4cbdf3
|
r
|
2023-10-04 13:11:43 -06:00 |
|
Cyberes
|
d0eec88dbd
|
f
|
2023-10-04 13:03:58 -06:00 |
|
Cyberes
|
6bad5b3fa0
|
t
|
2023-10-04 13:02:53 -06:00 |
|
Cyberes
|
77db34a6a7
|
g
|
2023-10-04 12:59:19 -06:00 |
|
Cyberes
|
364b795268
|
fix
|
2023-10-04 12:57:11 -06:00 |
|
Cyberes
|
7cb624c5f5
|
f
|
2023-10-04 12:47:59 -06:00 |
|
Cyberes
|
6af5365015
|
c
|
2023-10-04 12:45:20 -06:00 |
|
Cyberes
|
f3a13fcda8
|
c
|
2023-10-04 12:44:33 -06:00 |
|
Cyberes
|
a15b5465df
|
c
|
2023-10-04 12:44:09 -06:00 |
|
Cyberes
|
95d781725e
|
t
|
2023-10-04 12:42:18 -06:00 |
|
Cyberes
|
1b21cb69c1
|
test
|
2023-10-04 12:40:29 -06:00 |
|
Cyberes
|
4deb32bf1c
|
test
|
2023-10-04 10:32:11 -06:00 |
|
Cyberes
|
7e3af3599d
|
test
|
2023-10-04 10:29:58 -06:00 |
|
Cyberes
|
4634e36eeb
|
text
|
2023-10-04 10:26:39 -06:00 |
|
Cyberes
|
b76e77a66a
|
fix exception
|
2023-10-04 10:24:28 -06:00 |
|
Cyberes
|
5f4e4710c1
|
option to prioritize by parameter count
|
2023-10-04 10:19:44 -06:00 |
|
Cyberes
|
6dc3529190
|
show online status on stats page
|
2023-10-03 23:39:25 -06:00 |
|
Cyberes
|
1a7f22ec55
|
adjust again
|
2023-10-03 20:47:37 -06:00 |
|
Cyberes
|
67f5df9bb9
|
fix stats page
|
2023-10-03 20:42:53 -06:00 |
|
Cyberes
|
f88e2362c5
|
remove some debug prints
|
2023-10-03 20:01:28 -06:00 |
|
Cyberes
|
33b4b8404b
|
clean up streaming
|
2023-10-03 14:10:50 -06:00 |
|
Cyberes
|
e16f415749
|
fix
|
2023-10-03 13:49:00 -06:00 |
|
Cyberes
|
581a0fec99
|
fix exception
|
2023-10-03 13:47:18 -06:00 |
|
Cyberes
|
32ad97e57c
|
do default model rather than default backend, adjust moderation endpoint logic and add timeout, exclude system tokens from recent proompters, calculate number of moderators from endpoint concurrent gens, adjust homepage
|
2023-10-03 13:40:08 -06:00 |
|
Cyberes
|
63c12ea830
|
fix
|
2023-10-03 01:25:43 -06:00 |
|
Cyberes
|
ca1baa4870
|
test
|
2023-10-03 00:15:16 -06:00 |
|
Cyberes
|
62eb0196cc
|
t
|
2023-10-03 00:13:55 -06:00 |
|
Cyberes
|
0f5e22191c
|
test
|
2023-10-03 00:12:37 -06:00 |
|
Cyberes
|
70126acdf2
|
test
|
2023-10-03 00:12:13 -06:00 |
|
Cyberes
|
f6acd67738
|
t
|
2023-10-03 00:05:32 -06:00 |
|
Cyberes
|
07d6f6d8e9
|
test
|
2023-10-03 00:03:39 -06:00 |
|
Cyberes
|
cd325216e2
|
test
|
2023-10-02 22:45:07 -06:00 |
|
Cyberes
|
aed5db4968
|
trying to narrow down error
|
2023-10-02 21:43:36 -06:00 |
|
Cyberes
|
94141b8ecf
|
fix processing not being decremented on streaming, fix confusion over queue, adjust stop sequences
|
2023-10-02 20:53:08 -06:00 |
|
Cyberes
|
4f226ae38e
|
handle requests to offline backends
|
2023-10-02 11:11:48 -06:00 |
|
Cyberes
|
b0089859d7
|
fix ratelimiting
|
2023-10-02 02:05:15 -06:00 |
|
Cyberes
|
d1c4e68f8b
|
fix openai models response
|
2023-10-01 23:07:49 -06:00 |
|
Cyberes
|
21da2f6373
|
fix openai error message
|
2023-10-01 22:58:08 -06:00 |
|
Cyberes
|
a594729d00
|
fix keyerror
|
2023-10-01 22:37:13 -06:00 |
|
Cyberes
|
51881ae39d
|
fix tokenizer
|
2023-10-01 17:19:34 -06:00 |
|
Cyberes
|
f7e9687527
|
finish openai endpoints
|
2023-10-01 16:04:53 -06:00 |
|
Cyberes
|
2a3ff7e21e
|
update openai endpoints
|
2023-10-01 14:15:01 -06:00 |
|
Cyberes
|
93d19fb95b
|
fix exception
|
2023-10-01 10:25:32 -06:00 |
|
Cyberes
|
d203973e80
|
fix routes
|
2023-10-01 01:13:13 -06:00 |
|
Cyberes
|
25ec56a5ef
|
get streaming working, remove /v2/
|
2023-10-01 00:20:00 -06:00 |
|
Cyberes
|
b10d22ca0d
|
cache the home page in the background
|
2023-09-30 23:03:42 -06:00 |
|
Cyberes
|
3ecb7bcf88
|
adjust jinja template
|
2023-09-30 22:14:50 -06:00 |
|
Cyberes
|
c5b30d985c
|
adjust jinja template
|
2023-09-30 22:11:51 -06:00 |
|
Cyberes
|
bc25d92c95
|
reduce tokens for backend tester
|
2023-09-30 21:48:16 -06:00 |
|
Cyberes
|
9235725bdd
|
adjust message
|
2023-09-30 21:35:55 -06:00 |
|
Cyberes
|
61856b4383
|
adjust message
|
2023-09-30 21:34:32 -06:00 |
|
Cyberes
|
7af3dbd76b
|
add message about settings
|
2023-09-30 21:31:25 -06:00 |
|
Cyberes
|
592eb08cb1
|
add message for /v1/
|
2023-09-30 21:07:12 -06:00 |
|
Cyberes
|
166b2316e8
|
depricate v1
|
2023-09-30 20:59:24 -06:00 |
|
Cyberes
|
1151bb5475
|
adjust stats
|
2023-09-30 20:42:48 -06:00 |
|
Cyberes
|
91ba2fad1b
|
add proompter stats back in
|
2023-09-30 20:11:14 -06:00 |
|
Cyberes
|
e553fa6e9f
|
adjust home page fontsize
|
2023-09-30 20:01:36 -06:00 |
|
Cyberes
|
11a10f85c1
|
adjust home page
|
2023-09-30 19:59:30 -06:00 |
|
Cyberes
|
e6267a7d46
|
remove vllm from requirements.txt
|
2023-09-30 19:45:04 -06:00 |
|
Cyberes
|
e0f86d053a
|
reorganize to api v2
|
2023-09-30 19:42:41 -06:00 |
|
Cyberes
|
114f36e709
|
functional
|
2023-09-30 19:41:50 -06:00 |
|
Cyberes
|
624ca74ce5
|
mvp
|
2023-09-29 00:09:44 -06:00 |
|
Cyberes
|
e7b57cad7b
|
set up cluster config and basic background workers
|
2023-09-28 18:40:24 -06:00 |