History

Leo Tronchon b8fefa6b55 raise exception on invalid images (#999 ) # What does this PR do? This PR is meant to handle cases in which the images provided are invalid. ## Who can review? Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR. @Narsil --------- Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>		2023-10-03 10:26:10 +02:00
..
custom_kernels	feat(server): Rework model loading (#344 )	2023-06-08 14:51:52 +02:00
exllama_kernels	feat: add cuda memory fraction (#659 )	2023-07-24 11:43:58 +02:00
tests	feat: format code (#1070 )	2023-09-27 12:22:09 +02:00
text_generation_server	raise exception on invalid images (#999 )	2023-10-03 10:26:10 +02:00
.gitignore	Support eetq weight only quantization (#1068 )	2023-09-27 11:42:57 +02:00
Makefile	Support eetq weight only quantization (#1068 )	2023-09-27 11:42:57 +02:00
Makefile-awq	Add AWQ quantization inference support (#1019 ) (#1054 )	2023-09-25 15:31:27 +02:00
Makefile-eetq	Support eetq weight only quantization (#1068 )	2023-09-27 11:42:57 +02:00
Makefile-flash-att	feat(server): use latest flash attention commit (#543 )	2023-07-04 20:23:55 +02:00
Makefile-flash-att-v2	feat: add mistral model (#1071 )	2023-09-28 09:55:47 +02:00
Makefile-vllm	feat: add mistral model (#1071 )	2023-09-28 09:55:47 +02:00
README.md	feat(router): refactor API and add openAPI schemas (#53 )	2023-02-03 12:43:37 +01:00
poetry.lock	Add AWQ quantization inference support (#1019 ) (#1054 )	2023-09-25 15:31:27 +02:00
pyproject.toml	Preping 1.1.0 (#1066 )	2023-09-27 10:40:18 +02:00
requirements.txt	Preping 1.1.0 (#1066 )	2023-09-27 10:40:18 +02:00

README.md

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

Run

make run-dev