4cce84301b
As more and more people begin to use Baichuan's open-source models, the influence of Baichuan models is growing, especially in China. Many community members are interested in adding support for Baichuan models to TGI. Meanwhile, Baichuan is a very open company, and in the future, it plans to open-source more and more models, taking all this into consideration, we would like to add support for the Baichuan model to TGI. To do this, we need to make some changes, which we hope can be merged into the main branch of TGI. In the future, we would be happy to help maintain support for Baichuan models in TGI. We sincerely hope that our pull request can be accepted. Thank you. By the way, the changes of this time mainly for supporting Baichuan-7B. --------- Co-authored-by: xiaoyuze <xiaoyuze@baichuan.com> Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com> |
||
|---|---|---|
| .. | ||
| custom_kernels | ||
| exllama_kernels | ||
| tests | ||
| text_generation_server | ||
| .gitignore | ||
| Makefile | ||
| Makefile-flash-att | ||
| Makefile-flash-att-v2 | ||
| Makefile-vllm | ||
| README.md | ||
| poetry.lock | ||
| pyproject.toml | ||
| requirements.txt | ||
README.md
Text Generation Inference Python gRPC Server
A Python gRPC server for Text Generation Inference
Install
make install
Run
make run-dev