Commit Graph

257 Commits

Author SHA1 Message Date
Cyberes 92ae8312af docker: update build script 2023-10-15 16:06:36 -06:00
Cyberes d68dd7127f docker: revert 2023-10-15 16:05:14 -06:00
Cyberes e650c4e891 docker: install jupiter notebook 2023-10-15 16:00:26 -06:00
Cyberes cae8d8356b docker: use jupiter notebook 2023-10-15 15:53:32 -06:00
Cyberes 949ddda649 docker: update 2023-10-15 15:45:25 -06:00
Cyberes f58fa1a6f1 docker: git clone 2023-10-15 15:43:37 -06:00
Cyberes d9d3e03f59 docker: split into base and runtime 2023-10-15 15:38:48 -06:00
Cyberes ac5cf18784 docker: revert 2023-10-15 15:24:58 -06:00
Cyberes 521db377b4 docker: update readme 2023-10-15 14:57:14 -06:00
Cyberes 2d2d776fd7 docker: remove updater script 2023-10-15 14:55:32 -06:00
Cyberes c56ad3c66a docker: no need to install requirements 2023-10-15 14:53:05 -06:00
Cyberes de37ab3771 docker: actualy do things 2023-10-15 14:51:41 -06:00
Cyberes c222f96f24 docker: clean up 2023-10-15 14:50:04 -06:00
Cyberes 21ff182ec1 docker: ??? 2023-10-15 12:37:45 -06:00
Cyberes f849e17944 adjust docker caching 2023-10-15 12:16:12 -06:00
Cyberes aba79bdcf7 docker: adjust expose 2023-10-15 12:15:18 -06:00
Cyberes f2ab2a0e23 docker: adjust caching 2023-10-15 12:13:53 -06:00
Cyberes 5df955dae0 docker: fix?? 2023-10-15 12:12:01 -06:00
Cyberes d4b6d7ef82 docker: reverd 2023-10-15 12:03:16 -06:00
Cyberes b1d73abfd8 docker: fix launch? 2023-10-15 10:43:35 -06:00
Cyberes 448ebcbe8d docker: install pytorch 11.8 2023-10-14 20:41:40 -06:00
Cyberes 076501a784 adjust vllm docker 2023-10-09 18:18:29 -06:00
Cyberes 782678de5f Update 'other/vllm/Docker/Dockerfile' 2023-10-01 02:09:37 -06:00
Cyberes 23cb4df7db Update 'other/vllm/Docker/Dockerfile' 2023-10-01 02:08:16 -06:00
Cyberes ac43cb0f5a Update 'other/vllm/Docker/README.md' 2023-09-30 22:41:11 -06:00
Cyberes fdbc3feae4 Update 'other/vllm/Docker/start-container.sh' 2023-09-30 22:19:04 -06:00
Cyberes 3e651e64d2 Update 'other/vllm/Docker/start-container.sh' 2023-09-30 19:24:20 -06:00
Cyberes 428123e9f1 Update 'other/vllm/Docker/start-container.sh' 2023-09-30 19:11:26 -06:00
Cyberes ee83608a52 Update 'other/vllm/Docker/start-container.sh' 2023-09-30 19:10:10 -06:00
Cyberes 79fef7bc5a Update 'other/vllm/Docker/Dockerfile' 2023-09-30 19:04:31 -06:00
Cyberes c65b722211 Update 'other/vllm/Docker/DOCKER.md' 2023-09-30 13:32:46 -06:00
Cyberes d3f529ca8b Upload files to 'other/vllm/Docker' 2023-09-30 13:25:10 -06:00
Cyberes c047df0dc0 Update 'other/vllm/Docker/start-container.sh' 2023-09-30 13:24:47 -06:00
Cyberes d8ac9dc042 Update 'other/vllm/Docker/supervisord.conf' 2023-09-30 13:00:01 -06:00
Cyberes 2e998344d6 Update 'other/vllm/vllm_api_server.py' 2023-09-29 22:36:03 -06:00
Cyberes c888f5c789 update docker 2023-09-29 22:28:38 -06:00
Cyberes 89e9f42663 remove secrets from dockerfile, use /storage instead 2023-09-28 17:02:45 -06:00
Cyberes e1d3fca6d3 try to cancel inference if disconnected from client 2023-09-28 09:55:31 -06:00
Cyberes e42f2b6819 fix negative queue on stats 2023-09-28 08:47:39 -06:00
Cyberes 347a82b7e1 avoid sending to backend to tokenize if it's greater than our specified context size 2023-09-28 03:54:20 -06:00
Cyberes 467b804ad7 raise printer interval 2023-09-28 03:47:27 -06:00
Cyberes 315d42bbc5 divide by 0??? 2023-09-28 03:46:01 -06:00
Cyberes 59f2aac8ad rewrite redis usage 2023-09-28 03:44:30 -06:00
Cyberes a4a1d6cce6 fix double logging 2023-09-28 01:34:15 -06:00
Cyberes ecdf819088 fix try/finally with continue, fix wrong subclass signature 2023-09-28 00:11:34 -06:00
Cyberes 3a538d649a fix docker typo lol 2023-09-28 00:02:41 -06:00
Cyberes e86a5182eb redo background processes, reorganize server.py 2023-09-27 23:36:44 -06:00
Cyberes 097d614a35 fix duplicate logging from console printer thread 2023-09-27 21:28:25 -06:00
Cyberes adc0905c6f fix imports 2023-09-27 21:20:08 -06:00
Cyberes e5fbc9545d add ratelimiting to websocket streaming endpoint, fix queue not decrementing IP requests, add console printer 2023-09-27 21:15:54 -06:00