History

Alvaro Bartolome 0b7df77178 Add LoRA adapters support for Gemma2 (#2567 ) * Add LoRA adapters support for Gemma2 * Make `black` formatting happy		2024-09-26 10:54:08 +02:00
..
custom_kernels	All integration tests back everywhere (too many failed CI). (#2428 )	2024-08-16 21:19:46 +02:00
exllama_kernels	…
exllamav2_kernels	…
tests	Fix tokenization yi (#2507 )	2024-09-11 22:41:56 +02:00
text_generation_server	Add LoRA adapters support for Gemma2 (#2567 )	2024-09-26 10:54:08 +02:00
.gitignore	…
Makefile	Lots of improvements (Still 2 allocators) (#2449 )	2024-08-29 16:29:01 +02:00
Makefile-awq	…
Makefile-eetq	…
Makefile-exllamav2	Upgrading exl2. (#2415 )	2024-08-14 11:58:08 +02:00
Makefile-fbgemm	Add Directory Check to Prevent Redundant Cloning in Build Process (#2486 )	2024-09-07 13:19:43 +02:00
Makefile-flash-att	…
Makefile-flash-att-v2	Softcapping for gemma2. (#2273 )	2024-07-22 18:27:10 +02:00
Makefile-flashinfer	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490 )	2024-09-11 18:10:40 +02:00
Makefile-lorax-punica	…
Makefile-selective-scan	…
Makefile-vllm	Add support for Deepseek V2 (#2224 )	2024-07-19 17:23:20 +02:00
README.md	…
poetry.lock	Update to moe-kenels 0.3.1 (#2535 )	2024-09-19 22:16:32 +02:00
pyproject.toml	Update to moe-kenels 0.3.1 (#2535 )	2024-09-19 22:16:32 +02:00
requirements_cuda.txt	hotfix: add syrupy to the right subproject (#2499 )	2024-09-06 12:47:06 +02:00
requirements_intel.txt	hotfix: add syrupy to the right subproject (#2499 )	2024-09-06 12:47:06 +02:00
requirements_rocm.txt	hotfix: add syrupy to the right subproject (#2499 )	2024-09-06 12:47:06 +02:00

README.md

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

Run

make run-dev