History

drbh f852190060 fix: prefer hidden_activation over hidden_act in gemma2 (#2381 )		2024-08-08 14:08:56 -04:00
..
custom_kernels	…
exllama_kernels	MI300 compatibility (#1764 )	2024-05-17 15:30:47 +02:00
exllamav2_kernels	…
tests	feat: add ruff and resolve issue (#2262 )	2024-07-26 10:29:09 -04:00
text_generation_server	fix: prefer hidden_activation over hidden_act in gemma2 (#2381 )	2024-08-08 14:08:56 -04:00
.gitignore	…
Makefile	hotfix: update nccl	2024-07-23 23:31:28 +02:00
Makefile-awq	…
Makefile-eetq	…
Makefile-fbgemm	chore: update to torch 2.4 (#2259 )	2024-07-23 20:39:43 +00:00
Makefile-flash-att	Hotfixing `make install`. (#2008 )	2024-06-04 23:34:03 +02:00
Makefile-flash-att-v2	Softcapping for gemma2. (#2273 )	2024-07-22 18:27:10 +02:00
Makefile-lorax-punica	Enable multiple LoRa adapters (#2010 )	2024-06-25 14:46:27 -04:00
Makefile-selective-scan	…
Makefile-vllm	Add support for Deepseek V2 (#2224 )	2024-07-19 17:23:20 +02:00
README.md	…
poetry.lock	Install Marlin from standalone package (#2320 )	2024-07-29 15:37:10 +02:00
pyproject.toml	Install Marlin from standalone package (#2320 )	2024-07-29 15:37:10 +02:00
requirements_cuda.txt	hotfix: pin numpy (#2289 )	2024-07-23 17:53:19 +02:00
requirements_intel.txt	hotfix: pin numpy (#2289 )	2024-07-23 17:53:19 +02:00
requirements_rocm.txt	hotfix: pin numpy (#2289 )	2024-07-23 17:53:19 +02:00

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

make run-dev