|
…
|
||
|---|---|---|
| .. | ||
| custom_kernels | ||
| exllama_kernels | ||
| tests | ||
| text_generation_server | ||
| .gitignore | ||
| Makefile | ||
| Makefile-flash-att | ||
| Makefile-flash-att-v2 | ||
| Makefile-vllm | ||
| README.md | ||
| poetry.lock | ||
| pyproject.toml | ||
| requirements.txt | ||
README.md
Text Generation Inference Python gRPC Server
A Python gRPC server for Text Generation Inference
Install
make install
Run
make run-dev