31d76e238d
- Avoid theoretical hang in batcher loop - Avoid a couple of clones in the router generate method - Keep attention mask tensors as integers - Remove num_heads attribute Co-authored-by: OlivierDehaene <Olivier.dehaene@gmail.com> |
||
---|---|---|
.. | ||
text_generation | ||
.gitignore | ||
Makefile | ||
README.md | ||
poetry.lock | ||
pyproject.toml |
README.md
BLOOM Inference Python gRPC Server
A Python gRPC server for BLOOM Inference
Install
make install
Run
make run-dev