local-llm-server/other/vllm/README.md

9 lines
239 B
Markdown

### Nginx
1. Make sure your proxies all have a long timeout:
```
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_send_timeout 300;
```
The LLM middleware has a request timeout of 120 so this longer timeout is to avoid any issues.