History

Daniël de Kok 9ed0c85fe1 nix: add black and isort to the closure (#2619 ) To make sure that everything is formatted with the same black version as CI. I sometimes use isort for new files to get nicely ordered imports, so add it as well. Also set the isort configuration to format in a way that is compatible with black.		2024-10-09 11:08:02 +02:00
..
custom_kernels	…
exllama_kernels	Update ROCM libs and improvements (#2579 )	2024-09-30 10:54:32 +02:00
exllamav2_kernels	Update ROCM libs and improvements (#2579 )	2024-09-30 10:54:32 +02:00
tests	…
text_generation_server	Add support for fused MoE Marlin for AWQ (#2616 )	2024-10-08 11:56:41 +02:00
.gitignore	…
Makefile	…
Makefile-awq	…
Makefile-eetq	…
Makefile-exllamav2	…
Makefile-fbgemm	…
Makefile-flash-att	…
Makefile-flash-att-v2	Update ROCM libs and improvements (#2579 )	2024-09-30 10:54:32 +02:00
Makefile-flashinfer	…
Makefile-lorax-punica	…
Makefile-selective-scan	…
Makefile-vllm	Update ROCM libs and improvements (#2579 )	2024-09-30 10:54:32 +02:00
README.md	…
poetry.lock	Add support for fused MoE Marlin for AWQ (#2616 )	2024-10-08 11:56:41 +02:00
pyproject.toml	nix: add black and isort to the closure (#2619 )	2024-10-09 11:08:02 +02:00
requirements_cuda.txt	Mllama flash version (#2585 )	2024-10-02 11:22:13 +02:00
requirements_intel.txt	Mllama flash version (#2585 )	2024-10-02 11:22:13 +02:00
requirements_rocm.txt	Mllama flash version (#2585 )	2024-10-02 11:22:13 +02:00

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

make run-dev