Commit Graph

Select branches

Hide Pull Requests

cluster

master

#3

v0.9

vllm-gptq-b633191

Mono Color

• c5b30d985c adjust jinja template Cyberes 2023-09-30 22:11:51 -0600
• bc25d92c95 reduce tokens for backend tester Cyberes 2023-09-30 21:48:16 -0600
• 9235725bdd adjust message Cyberes 2023-09-30 21:35:55 -0600
• 61856b4383 adjust message Cyberes 2023-09-30 21:34:32 -0600
• 7af3dbd76b add message about settings Cyberes 2023-09-30 21:31:25 -0600
• 592eb08cb1 add message for /v1/ Cyberes 2023-09-30 21:07:12 -0600
• 166b2316e8 depricate v1 Cyberes 2023-09-30 20:59:24 -0600
• 1151bb5475 adjust stats Cyberes 2023-09-30 20:42:48 -0600
• 91ba2fad1b add proompter stats back in Cyberes 2023-09-30 20:11:14 -0600
• e553fa6e9f adjust home page fontsize Cyberes 2023-09-30 20:01:36 -0600
• 11a10f85c1 adjust home page Cyberes 2023-09-30 19:59:30 -0600
• e6267a7d46 remove vllm from requirements.txt Cyberes 2023-09-30 19:45:04 -0600
• e0f86d053a reorganize to api v2 Cyberes 2023-09-30 19:42:41 -0600
• 114f36e709 functional Cyberes 2023-09-30 19:41:50 -0600
• 3e651e64d2 Update 'other/vllm/Docker/start-container.sh' Cyberes 2023-09-30 19:24:20 -0600
• 428123e9f1 Update 'other/vllm/Docker/start-container.sh' Cyberes 2023-09-30 19:11:26 -0600
• ee83608a52 Update 'other/vllm/Docker/start-container.sh' Cyberes 2023-09-30 19:10:10 -0600
• 79fef7bc5a Update 'other/vllm/Docker/Dockerfile' Cyberes 2023-09-30 19:04:31 -0600
• c65b722211 Update 'other/vllm/Docker/DOCKER.md' Cyberes 2023-09-30 13:32:46 -0600
• d3f529ca8b Upload files to 'other/vllm/Docker' Cyberes 2023-09-30 13:25:10 -0600
• c047df0dc0 Update 'other/vllm/Docker/start-container.sh' Cyberes 2023-09-30 13:24:47 -0600
• d8ac9dc042 Update 'other/vllm/Docker/supervisord.conf' Cyberes 2023-09-30 13:00:01 -0600
• 2e998344d6 Update 'other/vllm/vllm_api_server.py' Cyberes 2023-09-29 22:36:03 -0600
• c888f5c789 update docker Cyberes 2023-09-29 22:28:38 -0600
• 624ca74ce5 mvp Cyberes 2023-09-29 00:09:44 -0600
• e7b57cad7b set up cluster config and basic background workers Cyberes 2023-09-28 18:40:24 -0600
• 89e9f42663 remove secrets from dockerfile, use /storage instead Cyberes 2023-09-28 17:02:45 -0600
• e1d3fca6d3 try to cancel inference if disconnected from client Cyberes 2023-09-28 09:55:31 -0600
• e42f2b6819 fix negative queue on stats Cyberes 2023-09-28 08:47:39 -0600
• 347a82b7e1 avoid sending to backend to tokenize if it's greater than our specified context size Cyberes 2023-09-28 03:54:20 -0600
• 467b804ad7 raise printer interval Cyberes 2023-09-28 03:47:27 -0600
• 315d42bbc5 divide by 0??? Cyberes 2023-09-28 03:46:01 -0600
• 59f2aac8ad rewrite redis usage Cyberes 2023-09-28 03:44:30 -0600
• a4a1d6cce6 fix double logging Cyberes 2023-09-28 01:34:15 -0600
• ecdf819088 fix try/finally with continue, fix wrong subclass signature Cyberes 2023-09-28 00:11:34 -0600
• 3a538d649a fix docker typo lol Cyberes 2023-09-28 00:02:41 -0600
• e86a5182eb redo background processes, reorganize server.py Cyberes 2023-09-27 23:36:44 -0600
• 097d614a35 fix duplicate logging from console printer thread Cyberes 2023-09-27 21:28:25 -0600
• adc0905c6f fix imports Cyberes 2023-09-27 21:20:08 -0600
• e5fbc9545d add ratelimiting to websocket streaming endpoint, fix queue not decrementing IP requests, add console printer Cyberes 2023-09-27 21:15:54 -0600
• 43299b32ad clean up background threads Cyberes 2023-09-27 19:39:04 -0600
• 35e9847b27 set inference workers to daemon, add finally to inference worker, hide estimated avg tps Cyberes 2023-09-27 18:36:51 -0600
• abef9eba7d adjust dockerfile paths Cyberes 2023-09-27 17:50:55 -0600
• 1874e6f7c4 fix docker logging Cyberes 2023-09-27 17:00:46 -0600
• ffb7af8f3c rename docker service Cyberes 2023-09-27 16:58:49 -0600
• 3b0ec723a5 fix docker Cyberes 2023-09-27 16:57:14 -0600
• 74f16afa67 update dockerfile Cyberes 2023-09-27 16:12:36 -0600
• eade509947 improve dockerfile Cyberes 2023-09-27 14:59:33 -0600
• 105b66d5e2 unify error message handling Cyberes 2023-09-27 14:48:47 -0600
• 957a6cd092 fix error handling Cyberes 2023-09-27 14:36:49 -0600
• 90bb68115f asdjust docker Cyberes 2023-09-27 00:04:37 -0600
• aba2e5b9c0 don't use db pooling, add LLM-ST-Errors header to disable formatted errors Cyberes 2023-09-26 23:59:22 -0600
• 7456bbe085 adjust docker Cyberes 2023-09-26 23:23:54 -0600
• 048e5a8060 fix API key handling Cyberes 2023-09-26 22:49:53 -0600
• d9bbcc42e6 more work on openai endpoint Cyberes 2023-09-26 22:09:11 -0600
• 9e6624e779 modify dockerfile for paperspace Cyberes 2023-09-26 21:45:13 -0600
• e3c57d874a add vllm dockerfile Cyberes 2023-09-26 14:48:34 -0600
• e0af2ea9c5 convert to gunicorn Cyberes 2023-09-26 13:32:33 -0600
• 0eb901cb52 don't log entire request on failure Cyberes 2023-09-26 12:32:19 -0600
• b44dda7a3a option to show SYSTEM tokens in stats Cyberes 2023-09-25 23:39:50 -0600
• e37cde5d48 exclude system token more places Cyberes 2023-09-25 23:22:16 -0600
• bbdb9c9d55 try to prevent "### XXX" responses on openai Cyberes 2023-09-25 23:14:35 -0600
• 11e84db59c update database, tokenizer handle null prompt, convert top_p to vllm on openai, actually validate prompt on streaming, Cyberes 2023-09-25 22:32:48 -0600
• 2d299dbae5 openai_force_no_hashes Cyberes 2023-09-25 22:01:57 -0600
• 8240a1ebbb fix background log not doing anything Cyberes 2023-09-25 18:18:29 -0600
• 8184e24bff fix sending error messages when streaming Cyberes 2023-09-25 17:37:58 -0600
• 7ce60079d7 fix typo Cyberes 2023-09-25 17:24:51 -0600
• 289b40181c forgot to test all config possibilities Cyberes 2023-09-25 17:23:43 -0600
• 30282479a0 fix flask exception Cyberes 2023-09-25 17:22:28 -0600
• 135bd743bb fix homepage slowness, fix incorrect 24 hr prompters, fix redis wrapper, Cyberes 2023-09-25 17:20:21 -0600
• 52e6965b5e don't count SYSTEM tokens for recent prompters, fix sql exclude for SYSTEM tokens Cyberes 2023-09-25 13:00:39 -0600
• 3eaabc8c35 fix copied code Cyberes 2023-09-25 12:38:02 -0600
• 44e692c9cf remove debug print Cyberes 2023-09-25 12:35:36 -0600
• 1646a00987 implement streaming on openai, improve streaming, run DB logging in background thread Cyberes 2023-09-25 12:30:40 -0600
• bbe5d5a8fe improve openai endpoint, exclude system tokens more places Cyberes 2023-09-25 09:32:23 -0600
• 6459a1c91b allow setting simultaneous IP limit per-token, fix token use tracker, fix tokens on streaming Cyberes 2023-09-25 00:55:20 -0600
• d2651756df update requirements.txt Cyberes 2023-09-24 21:46:48 -0600
• 320f51e01c further align openai endpoint with expected responses Cyberes 2023-09-24 21:45:30 -0600
• 84ea2f8891 handle when auth token is not enabled Cyberes 2023-09-24 15:57:39 -0600
• 8d6b2ce49c minor changes, add admin token auth system, add route to get backend info Cyberes 2023-09-24 15:54:35 -0600
• 2678102153 handle error while streaming Cyberes 2023-09-24 13:27:27 -0600
• cb99c3490e rewrite tokenizer, restructure validation Cyberes 2023-09-24 13:02:30 -0600
• 62412f4873 add config setting for hostname Cyberes 2023-09-23 23:24:08 -0600
• 84a1fcfdd8 don't store host if it's an IP Cyberes 2023-09-23 23:14:22 -0600
• 0015e653b2 adjust a few final things Cyberes 2023-09-23 22:30:59 -0600
• fab7b7ccdd active gen workers wait Cyberes 2023-09-23 21:17:13 -0600
• 7ee2311183 whats going on Cyberes 2023-09-23 21:10:14 -0600
• 94e845cd1a if there's less than num concurrent wait time is 0 Cyberes 2023-09-23 21:09:21 -0600
• 41e622d19c fix two exceptions Cyberes 2023-09-23 20:55:49 -0600
• 0530fa9870 getting sql error Cyberes 2023-09-23 19:08:30 -0600
• b0beb7efeb fix exception Cyberes 2023-09-23 18:55:52 -0600
• f67ac8175b fix wrong approach for streaming Cyberes 2023-09-23 18:44:07 -0600
• 8a4de7df44 oops Cyberes 2023-09-23 18:01:12 -0600
• 76a1428ba0 implement streaming for vllm Cyberes 2023-09-23 17:57:23 -0600
• 81452ec643 adjust vllm info Cyberes 2023-09-21 20:13:29 -0600
• f9a80f3028 change proompters 1 min to 5 min Cyberes 2023-09-20 21:21:22 -0600
• 8593198216 close mysql cursor Cyberes 2023-09-20 21:19:26 -0600
• 9341492a8a fix json error Cyberes 2023-09-20 21:14:12 -0600
• 03e3ec5490 port to mysql, use vllm tokenizer endpoint Cyberes 2023-09-20 20:30:31 -0600
• 2d390e6268 *blushes* oopsie daisy Cyberes 2023-09-17 20:22:17 -0600