fix(readme): Typo

This commit is contained in:
OlivierDehaene 2022-11-14 16:22:10 +01:00
parent 91f5f86280
commit feb7806ca4
1 changed files with 1 additions and 1 deletions

View File

@ -11,7 +11,7 @@ to power Bloom, BloomZ and MT0-XXL api-inference widgets.
## Features
- [Dynamic bathing of incoming requests](https://github.com/huggingface/text-generation-inference/blob/main/router/src/batcher.rs#L88) for increased total throughput
- [Dynamic batching of incoming requests](https://github.com/huggingface/text-generation-inference/blob/main/router/src/batcher.rs#L88) for increased total throughput
- Quantization with [bitsandbytes](https://github.com/TimDettmers/bitsandbytes)
- [Safetensors](https://github.com/huggingface/safetensors) weight loading
- 45ms per token generation for BLOOM with 8xA100 80GB