cyberes
/
local-llm-server
Archived
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

Merge cluster to master #3

Merged
cyberes merged 163 commits from cluster into master 2023-10-27 19:19:22 -06:00
Conversation 0 Commits 163 Files Changed 91 +2 -2
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 20047fa0e4 - Show all commits

4
llm_server/llm/vllm/tokenize.py
View File

@ -22,8 +22,8 @@ def tokenize(prompt: str, backend_url: str) -> int:
tokenizer = tiktoken.get_encoding("cl100k_base") tokenizer = tiktoken.get_encoding("cl100k_base")
# Split the prompt into 1000 character chunks # Split the prompt into 2000 character chunks
chunk_size = 1000 chunk_size = 2000
chunks = [prompt[i:i + chunk_size] for i in range(0, len(prompt), chunk_size)] chunks = [prompt[i:i + chunk_size] for i in range(0, len(prompt), chunk_size)]
# Define a function to send a chunk to the server # Define a function to send a chunk to the server