History

…
..
custom_kernels	…
exllama_kernels	…
exllamav2_kernels	…
tests	…
text_generation_server	…
.gitignore	…
Makefile	…
Makefile-awq	…
Makefile-eetq	…
Makefile-flash-att	…
Makefile-flash-att-v2	…
Makefile-vllm	…
README.md	…
poetry.lock	…
pyproject.toml	…
requirements_common.txt	…
requirements_cuda.txt	…
requirements_rocm.txt	…

README.md

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

Run

make run-dev