cyberes
/
local-llm-server
Archived
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

Merge cluster to master #3

Merged
cyberes merged 163 commits from cluster into master 2023-10-27 19:19:22 -06:00
Conversation 0 Commits 163 Files Changed 91 +4 -3
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 6e74ce7c28 - Show all commits

7
llm_server/routes/openai/completions.py
View File

@ -84,9 +84,10 @@ def openai_completions(model_name=None):
} }
}) })
stats = redis.get('proxy_stats', dtype=dict) # TODO:
if stats: # stats = redis.get('proxy_stats', dtype=dict)
response.headers['x-ratelimit-reset-requests'] = stats['queue']['estimated_wait_sec'] # if stats:
# response.headers['x-ratelimit-reset-requests'] = stats['queue']['estimated_wait_sec']
return response, 200 return response, 200
else: else:
if not opts.enable_streaming: if not opts.enable_streaming: