6ded76a4ae | ||
---|---|---|
.. | ||
src | ||
.gitignore | ||
Cargo.lock | ||
Cargo.toml | ||
README.md | ||
rust-toolchain.toml |
README.md
A lightweight benchmarking tool based inspired by oha and powered by tui.
Install
make install-benchmark
Run
First, start text-generation-inference
:
text-generation-launcher --model-id bigscience/bloom-560m
Then run the benchmarking tool:
text-generation-benchmark --tokenizer-name bigscience/bloom-560m