fix(readme): Typo
This commit is contained in:
parent
91f5f86280
commit
feb7806ca4
|
@ -11,7 +11,7 @@ to power Bloom, BloomZ and MT0-XXL api-inference widgets.
|
|||
|
||||
## Features
|
||||
|
||||
- [Dynamic bathing of incoming requests](https://github.com/huggingface/text-generation-inference/blob/main/router/src/batcher.rs#L88) for increased total throughput
|
||||
- [Dynamic batching of incoming requests](https://github.com/huggingface/text-generation-inference/blob/main/router/src/batcher.rs#L88) for increased total throughput
|
||||
- Quantization with [bitsandbytes](https://github.com/TimDettmers/bitsandbytes)
|
||||
- [Safetensors](https://github.com/huggingface/safetensors) weight loading
|
||||
- 45ms per token generation for BLOOM with 8xA100 80GB
|
||||
|
|
Loading…
Reference in New Issue