History

erikkaum 060b2db0df add 'mamba' as model config		2024-08-01 18:16:32 +02:00
..
custom_kernels	…
exllama_kernels	…
exllamav2_kernels	…
tests	feat: add ruff and resolve issue (#2262 )	2024-07-26 10:29:09 -04:00
text_generation_server	add 'mamba' as model config	2024-08-01 18:16:32 +02:00
.gitignore	…
Makefile	hotfix: update nccl	2024-07-23 23:31:28 +02:00
Makefile-awq	…
Makefile-eetq	…
Makefile-fbgemm	chore: update to torch 2.4 (#2259 )	2024-07-23 20:39:43 +00:00
Makefile-flash-att	…
Makefile-flash-att-v2	Softcapping for gemma2. (#2273 )	2024-07-22 18:27:10 +02:00
Makefile-lorax-punica	Enable multiple LoRa adapters (#2010 )	2024-06-25 14:46:27 -04:00
Makefile-selective-scan	…
Makefile-vllm	Add support for Deepseek V2 (#2224 )	2024-07-19 17:23:20 +02:00
README.md	…
poetry.lock	Install Marlin from standalone package (#2320 )	2024-07-29 15:37:10 +02:00
pyproject.toml	Install Marlin from standalone package (#2320 )	2024-07-29 15:37:10 +02:00
requirements_cuda.txt	hotfix: pin numpy (#2289 )	2024-07-23 17:53:19 +02:00
requirements_intel.txt	hotfix: pin numpy (#2289 )	2024-07-23 17:53:19 +02:00
requirements_rocm.txt	hotfix: pin numpy (#2289 )	2024-07-23 17:53:19 +02:00

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

make run-dev