Commit Graph

428 Commits

Author SHA1 Message Date
nai-degen 785b1f69f3 implements new local risu validation (via @kwaroran) 2023-08-28 05:28:58 -05:00
nai-degen c05bfefba4 fixes incorrectly applied doubleCsrf to REST routes 2023-08-10 15:54:01 -05:00
nai-degen 9b184ab245 removes QUOTA_DISPLAY_MODE config as OpenAI no longer supports it 2023-08-09 18:29:38 -05:00
nai-degen 6bb67281d9 removes QUEUE_MODE config (now always enabled) 2023-08-09 18:29:34 -05:00
nai-degen 5d3fb6af3a removes IP redaction from pino 2023-08-09 18:29:29 -05:00
khanon 268165e2be Add CSRF protection to server-rendered views (khanon/oai-reverse-proxy!34) 2023-08-09 23:11:26 +00:00
nai-degen 6f4e581bf2 fixes forgotten http=true on admin cookie 2023-08-09 11:01:30 -05:00
nai-degen 358339d48b fixes issue with Claude <EOT> token disallowed 2023-08-08 17:43:12 -05:00
nai-degen c8d8e2e58f avoids instantiating new Claude tiktoken on every call 2023-08-08 17:38:03 -05:00
nai-degen d1d83b41fa uses accurate Claude tokenization 2023-08-08 17:29:36 -05:00
nai-degen 81ceee7897 adds pagination info below user table 2023-08-05 23:49:14 -05:00
nai-degen 21ee00f057 fixes issue with IP count sorting 2023-08-05 23:26:15 -05:00
nai-degen 61d90f3f3a fixes admin ui sort during pagination 2023-08-05 21:16:44 -05:00
khanon bb230469b2 Admin user management UI (khanon/oai-reverse-proxy!32) 2023-08-06 00:58:33 +00:00
nai-degen 125bbe6441 fixes issue with writeErrorResponse 2023-08-04 13:49:11 -05:00
nai-degen d29c304d5a increases tokenizer failsafe to 500000 characters 2023-07-27 15:21:06 -05:00
nai-degen addfa7c57b restores trial key detection via workaround 2023-07-24 14:07:02 -05:00
nai-degen e5b4c7bc9e removes key limit/trial status from infopage 2023-07-24 13:14:44 -05:00
nai-degen 51503dec14 disables key checker, mostly 2023-07-24 13:11:45 -05:00
nai-degen 00346360af fixes turbo-16k incompatibility 2023-07-23 20:13:38 -05:00
nai-degen e2bd8a6b86 extracts Risu auth into new middleware so queue can use it too 2023-07-22 13:48:02 -05:00
nai-degen b8534dafae reduces default MAX_OUTPUT_TOKENS_ANTHROPIC 2023-07-21 19:18:21 -05:00
khanon 56a4902599 Add tokenizers and configurable context size limits (khanon/oai-reverse-proxy!28) 2023-07-22 00:11:32 +00:00
khanon 7634afeea4 Implement rate limit for risuai.xyz (khanon/oai-reverse-proxy!31) 2023-07-21 21:48:07 +00:00
nai-degen 77c2309b52 correctly flags trial keys during startup even if over quota 2023-07-20 23:06:37 -05:00
khanon aa5380d2ef Rework OpenAIKeyChecker to remove usage tracking and test all keys for liveness (khanon/oai-reverse-proxy!29) 2023-07-21 04:00:12 +00:00
nai-degen cbf9f16108 removes clamp on quota display to better show glitched keys 2023-07-19 23:34:09 -05:00
breathingmanually 576423d1f8 Fix JSON parse exception when Claude finishes streaming (khanon/oai-reverse-proxy!25) 2023-07-20 01:57:50 +00:00
Xrystallized 120b7da340 Include non /v1 url in check (khanon/oai-reverse-proxy!27) 2023-07-19 16:00:29 +00:00
nai-degen d7a4829d13 handles keys which have been banned (but not revoked) by openai 2023-07-19 10:28:38 -05:00
nai-degen c749e2d57d adjusts claude rate limit handling to retry more aggressively 2023-07-19 01:58:44 -05:00
nai-degen efa1b03570 uses claude-v1 by default as anthropic seems to be turning off v1.2 2023-07-19 01:48:57 -05:00
goanon016 f6f13f7955 Fix cell size error in sheets (khanon/oai-reverse-proxy!26) 2023-07-16 08:35:42 +00:00
khanon 7478112077 fixes embarrassing auth oversight 2023-07-16 07:31:44 +00:00
nai-degen aee382c84e adds claude-2 to supported models 2023-07-11 09:14:46 -05:00
nai-degen 32605fff53 fixes infopage regression when CHECK_KEYS=false 2023-07-08 15:29:43 -05:00
nai-degen 71882b18ae adds feature to prevent GPT-4 model selection (default off) 2023-07-06 16:09:30 -05:00
nai-degen 561c063d90 assumes keys are GPT-4 by default since it's now GA 2023-07-06 15:07:01 -05:00
nai-degen 327e860967 fixes wrong/misleading error msg when no Turbo keys available 2023-07-04 11:49:12 -05:00
nai-degen 6598b4df0d requests old version of Anthropic API due to breaking SSE changes 2023-06-24 14:50:48 -05:00
nai-degen 6a7f64b037 adds missed change from origin header adjustment 2023-06-24 14:25:30 -05:00
nai-degen c8b3238398 reorganizes origin header middleware 2023-06-24 14:25:01 -05:00
nai-degen 602931bf7f removes origin/referer headers from proxied request 2023-06-23 00:08:09 -05:00
nai-degen db034a51b3 prevents crash on startup when git is not installed 2023-06-21 01:24:41 -05:00
khanon 43359779e7 Implements more robust anti-zoomer functionality (khanon/oai-reverse-proxy!24) 2023-06-14 04:05:51 +00:00
nai-degen c0ac69df27 adjusts default origin block 2023-06-13 21:18:31 -05:00
nai-degen 3a2a6e96fd adds new OpenAI June 2023 models 2023-06-13 16:24:34 -05:00
khanon 96cf4a0e2d Automatically add Anthropic "\n\nHuman:" preamble when necessary (khanon/oai-reverse-proxy!23) 2023-06-09 07:16:40 +00:00
khanon 4f2a12ef14 Show per-model queues and keys on info page (khanon/oai-reverse-proxy!22) 2023-06-08 18:50:04 +00:00
khanon 120040c028 Separate `MAX_OUTPUT_TOKENS` config for OpenAI/Claude (khanon/oai-reverse-proxy!21) 2023-06-08 00:58:57 +00:00
nai-degen a6a0c21f7d makes gatekeeper compatible with sillytavern claude impl 2023-06-06 20:53:50 -05:00
nai-degen ace32c7e6e fix for huggingface not sending x-forwarded-host anymore 2023-06-06 20:47:45 -05:00
khanon 7b5e612921 Workaround for Anthropic requiring some prompts to start with `Human:` (khanon/oai-reverse-proxy!20) 2023-06-05 21:55:59 +00:00
khanon cd754bd601 Allow overriding default Claude model selection (khanon/oai-reverse-proxy!19) 2023-06-05 03:12:34 +00:00
nai-degen e6a8c548c2 reduces max log batch size to account for claude 100k prompts 2023-06-04 12:42:12 -05:00
nai-degen 176eef34db makes log queue backoff on errors rather than stop 2023-06-04 12:40:57 -05:00
nai-degen 38ff0e3f3b fixes prompt logging for streamed Claude requests 2023-06-04 12:16:29 -05:00
khanon dae1262f7a Refactor request middleware (khanon/oai-reverse-proxy!18) 2023-06-02 04:03:16 +00:00
nai-degen a26979f7bc returns static model list instead of proxying 2023-06-01 20:17:09 -05:00
nai-degen a6c6e4c694 allows KeyPool.available to be scoped per-service 2023-06-01 08:31:41 -05:00
nai-degen b7fa65d9f9 fixes dumb /v1/models bug affecting sillytavern again 2023-05-31 13:53:24 -05:00
nai-degen f621031ec1 fixes circluar dependency that broke tsc but not esbuild 2023-05-30 04:01:03 -05:00
khanon 6723cbf662 Anthropic endpoint improvements (khanon/oai-reverse-proxy!16) 2023-05-30 03:13:17 +00:00
nai-degen 2c8c81e6dd notes kobold endpoint as being deprecated 2023-05-29 19:43:43 -05:00
nai-degen 30bdc7cd5e shitty quick fix for tavern+openai not working properly 2023-05-29 13:23:38 -05:00
nai-degen f102f02a65 adds stop sequences for better UJB/group chat behavior on claude 2023-05-29 13:07:15 -05:00
nai-degen 304893e13d uses claude v1.2 instead of v1.1 2023-05-29 12:39:45 -05:00
nai-degen dc43183353 adds anthropic endpoint to info page 2023-05-29 12:21:39 -05:00
nai-degen d7ddbeea54 shortens backoff on claude rate limit 2023-05-29 12:16:46 -05:00
khanon 2d93463247 Implement support for Anthropic keys and Claude API (khanon/oai-reverse-proxy!15) 2023-05-29 17:08:08 +00:00
nai-degen 03aaa6daad wraps SSE error responses in code block backticks 2023-05-23 12:28:06 -05:00
nai-degen 13b6a3d7b8 adds header to improve nginx compatibility 2023-05-23 11:57:14 -05:00
nai-degen 03616f4bbc increases wait time estimation window to 5min 2023-05-23 11:26:40 -05:00
nai-degen 01eee61e8b checks the request content-type instead... 2023-05-22 21:14:58 -05:00
nai-degen e3603a60e9 follow-up to Accept header adjustment 2023-05-22 21:02:11 -05:00
nai-degen b24894d98f fixes origin block response to account for clients sending bad headers 2023-05-22 20:29:49 -05:00
nai-degen 26a6e4cadb increases wait time calculation window 2023-05-22 19:33:03 -05:00
nai-degen a61fa04fbf Implements origin/referrer block (khanon/oai-reverse-proxy!13) 2023-05-22 21:08:20 +00:00
yukianon a3620db591 Implement Test Generation for Trial Keys (khanon/oai-reverse-proxy!12) 2023-05-20 19:10:21 +00:00
nai-degen fe305bee98 shows more detailed build info 2023-05-19 20:53:13 -05:00
nai-degen 5be848d0d1 removes Render queue note as Tavern bug is now fixed 2023-05-19 09:34:56 -05:00
nai-degen 2bad644772 Prefer user tokens as rate-limit/queue keys when available (khanon/oai-reverse-proxy!10) 2023-05-19 04:33:20 +00:00
nai-degen dfd8f0cc97 fixes missing closing bracket in infopage html 2023-05-16 15:42:49 -05:00
nai-degen 27caf1ff06 adds temporary disclaimer about streaming issues on Render 2023-05-16 01:19:13 -05:00
nai-degen b4d8ebca4e adds noindex tag to infopage 2023-05-15 17:05:04 -05:00
nai-degen ec4e7a3671 Add docs and support for Render.com deployments (khanon/oai-reverse-proxy!9) 2023-05-15 21:47:30 +00:00
nai-degen 7f202dc9ef cleans up configs shown on info page 2023-05-14 15:45:30 -05:00
nai-degen 3f46d132f7 disables per-user IP limit by default 2023-05-14 14:46:14 -05:00
nai-degen 9ff7381f31 exempts 'special' user role from auto-ban 2023-05-14 14:37:50 -05:00
nai-degen 6eb121ac66 disables keys shared between too many IP addresses 2023-05-14 14:30:32 -05:00
nai-degen 285186d70e fixes keys being lost if not used at least once 2023-05-14 13:32:41 -05:00
nai-degen f1ac64fa12 Implement user persistence via Firebase (khanon/oai-reverse-proxy!8) 2023-05-14 04:26:08 +00:00
nai-degen 7126fb6c6c adds unhandled promise rejection listener just in case 2023-05-13 13:37:42 -05:00
nai-degen a7514e594d displays wait times more prominently on infopage 2023-05-13 13:33:05 -05:00
nai-degen 546b28cca6 increases wait time calc window to 90sec 2023-05-13 13:32:37 -05:00
nai-degen 977247d7a2 User authentication (khanon/oai-reverse-proxy!7) 2023-05-13 00:58:15 +00:00
nai-degen 5238aff378 reverts default quotaDisplay to partial; removes rejectSampleRate 2023-05-11 20:35:28 -05:00
nai-degen 7382bdc92e cleans up ratelimit table more frequently to smooth proompters display 2023-05-11 11:50:44 -05:00
nai-degen 09184079af reduces wait time window from 2min to 1min 2023-05-11 11:45:15 -05:00
nai-degen 8ff43aef1a disables kobold endpoint when queueing is enabled 2023-05-10 11:44:03 -05:00
nai-degen e03f3d48dd Implements request queueing (khanon/oai-reverse-proxy!6) 2023-05-09 23:11:57 +00:00
nai-degen e9e9f1f8b6 corrects openai incorrectly referred to as 'downstream' 2023-05-06 10:07:09 -05:00
nai-degen e541a7b892 forgot string.replace doesn't actually mutate the string 2023-05-06 09:55:59 -05:00
nai-degen 119c2f827a fixes org-ID redaction on errors which don't include one 2023-05-06 09:51:18 -05:00
gronon e0529197cd Prevent leaking OpenAI organization-ids in error responses (khanon/oai-reverse-proxy!5) 2023-05-06 12:22:05 +00:00
nai-degen 432023f875 cycles out keys which have been disabled 2023-05-05 09:50:43 -05:00
nai-degen 77bcbc7954 fixes bad pino redaction syntax 2023-05-04 16:43:26 -05:00
nai-degen 4dc9c649d8 fixes server blowing up when a streaming request drops out 2023-05-04 16:37:16 -05:00
nai-degen be1e672e68 prevents revealing x-forwarded-for ip addresses in HTTP logs 2023-05-04 16:31:40 -05:00
nai-degen 1fc0138e65 prevents revealing api keys in logs during startup crash 2023-05-04 16:29:36 -05:00
nai-degen 94720b51e7 fixes post-startup checks 2023-05-01 18:35:35 -05:00
nai-degen 40eac590af checks multiple keys at once during startup 2023-05-01 18:31:04 -05:00
nai-degen b1a331bde0 reduces keychecker delay, really not necessary to wait so long in between each key 2023-05-01 18:16:12 -05:00
nai-degen 176a37928d Implement support for streamed OpenAI responses (khanon/oai-reverse-proxy!4) 2023-05-01 22:01:47 +00:00
nai-degen 2b783e0f2b improves model_not_found error message 2023-04-30 22:01:45 -07:00
nai-degen 16b5ccb977 disables key downgrade feature to prevent abuse 2023-04-30 21:30:42 -07:00
nai-degen 1d69078514 displays huggingface metadata on infopage if present 2023-04-17 14:19:58 -07:00
nai-degen 9afa4570ec adds repo link to prompt logging notice 2023-04-15 01:11:00 -07:00
nai-degen e4495798ee omits undefined values from listConfig 2023-04-15 00:44:52 -07:00
nai-degen 5c885eaa1a removes duplicate keys which cause keychecker to get stuck 2023-04-14 21:13:24 -07:00
nai-degen 673f6f6185 actually fixes prompt logger path detection 2023-04-14 20:54:55 -07:00
nai-degen 717788b9b0 fixes key checker getting stuck in infinte loop 2023-04-14 20:51:05 -07:00
nai-degen 5e046b9060 fixes bug introduced in previous commit which prevented logging 2023-04-14 20:19:26 -07:00
nai-degen 34ed165a6f fixes issue with attempting to log non-completion endpoints 2023-04-14 20:10:14 -07:00
nai-degen fc3043dad0 Implements prompt logging via Google Sheets (khanon/oai-reverse-proxy!1) 2023-04-15 01:21:04 +00:00
nai-degen a767044850 Refactor onProxyRes handlers to use middleware stack (khanon/oai-reverse-proxy!2) 2023-04-14 15:29:11 +00:00
nai-degen a98665f013 reverts disabling keys which have gone above hardlimit, just let the API do that 2023-04-11 12:10:58 -07:00
nai-degen bbcc76266f improves downstream API error handling 2023-04-11 05:01:35 -07:00
nai-degen a8340b0dd4 redacts bearer tokens from logs 2023-04-11 03:39:24 -07:00
nai-degen 65c5cfeef2 fixes quota checking not counting the current day 2023-04-11 03:28:50 -07:00
nai-degen 920612f1ce rewrites urls to deal with client idiosyncrasies 2023-04-11 03:20:19 -07:00
nai-degen 44f667976a simplifies infopage and updates README 2023-04-11 03:19:05 -07:00
nai-degen 15ed8e7c25 adds TODOs so I don't forget 2023-04-10 20:02:48 -07:00
nai-degen f3a13129f9 fixes dead keys being 'revived' by a queued check 2023-04-10 19:57:32 -07:00
nai-degen 885487a61d fixes kobold key selection, I think 2023-04-10 19:50:42 -07:00
nai-degen f37a2fa7fa temporarily disable gpt-4 for kobold as it is broken 2023-04-10 04:56:04 -07:00
nai-degen f0afc8c420 fixes kobold endpoint not actually being rate-limited 2023-04-10 06:42:12 -05:00
nai-degen 6b47c1cdc5 improves quota display 2023-04-10 04:16:14 -07:00
nai-degen 4200d8203d redacts cookies in logs; fix sha on infopage 2023-04-10 03:39:13 -07:00
nai-degen 2a4dfe2501 Merge branch 'main' of https://github.com/nai-degen/oai-reverse-proxy 2023-04-10 03:11:36 -07:00
nai-degen 1238ee77c9 adds SHA to infopage 2023-04-10 03:11:26 -07:00
nai-degen 75e004d2b1 tries to handle exhausted trials a bit better 2023-04-10 03:11:14 -07:00
nai-degen 111fb1a7c5 improves service info page 2023-04-10 04:44:38 -05:00
nai-degen de5833e1e5 unfucks quota checking for real this time 2023-04-10 02:42:56 -07:00
nai-degen ca7f57f493 improves key checker 2023-04-10 01:49:52 -07:00
nai-degen 91cba67650 fixes kobold content-encoding 2023-04-10 00:53:27 -07:00
nai-degen 14c890f268 refactors keypool and implements key health checks 2023-04-10 00:27:45 -07:00
nai-degen 39cdb2e1e8 moves keys.ts into keys/key-pool.ts 2023-04-09 21:26:58 -07:00
nai-degen 7bb70ddfdc correctly decompresses gzip/deflate payloads 2023-04-09 02:30:59 -05:00
nai-degen 326def06a3 adjusts logging to troubleshoot errors from openai 2023-04-09 01:13:41 -05:00
nai-degen c4f8679bda exempts hosted agnai from rate limit 2023-04-09 00:49:14 -05:00
nai-degen 41637aca1b updates info page with new kobold url 2023-04-09 00:30:20 -05:00
nai-degen fd75f5c0c6 implements kobold endpoint 2023-04-09 00:28:51 -05:00
nai-degen b889374cef adds variable filter sampling rate 2023-04-08 21:57:51 -05:00
nai-degen d0ce8a4632 implements language filter 2023-04-08 21:49:06 -05:00
nai-degen 51becc36b9 fixes rate limiter to work with HF load balancing 2023-04-08 21:00:21 -05:00
nai-degen 5c585d1777 adds rate limiter 2023-04-08 20:40:45 -05:00
nai-degen e9b275d488 refactors request rewriters 2023-04-08 19:10:15 -05:00
nai-degen c51a0ef94d improves config handling 2023-04-08 18:32:49 -05:00
nai-degen 11308b2baa adds prompt counter 2023-04-08 11:13:55 -05:00
nai-degen c74c527f46 fixes http headers being set in wrong order 2023-04-08 10:13:00 -05:00
nai-degen 45451c4486 fixes incorrect Service Info URLs 2023-04-08 09:31:04 -05:00
nai-degen e5e6f61ddf removes unused proxy.ts 2023-04-08 09:01:46 -05:00
nai-degen f4716bead5 updates documentation 2023-04-08 09:00:19 -05:00
nai-degen 8143795d5e updates docs 2023-04-08 08:50:03 -05:00
nai-degen 51e07b82f5 removes unused files from failed rename 2023-04-08 08:34:46 -05:00
nai-degen 0c133b0a2d fixes openai proxy 2023-04-08 08:24:07 -05:00
nai-degen 66b8b6a5d0 loads keys on startup 2023-04-08 06:02:59 -05:00
nai-degen ba4aca7608 improves startup logging 2023-04-08 05:45:09 -05:00
nai-degen e2509d087c adds info page 2023-04-08 05:43:28 -05:00
nai-degen 6ba5670f29 clones retrieved keys to messy object mutation shit 2023-04-08 05:35:19 -05:00
nai-degen a4840e0fe6 implements basic key rotation 2023-04-08 05:32:24 -05:00
nai-degen 5ed37bf035 implements preliminary openai proxy 2023-04-08 04:42:36 -05:00
nai-degen ce09e4c0be adds router 2023-04-08 04:13:36 -05:00
nai-degen bbfb922dab adds auth 2023-04-08 04:12:58 -05:00
nai-degen e1606e3f66 adds basic key management 2023-04-08 04:12:15 -05:00
nai-degen 5d0184310c sets up server 2023-04-08 02:59:43 -05:00
nai-degen d98ae67c91 project scaffolding 2023-04-08 02:34:21 -05:00