Mirrors
/
preemo_text-generation-inference
mirror of https://github.com/Preemo-Inc/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
preemo_text-generation-infe.../benchmark
History
Jason Sun 1e646fb41d
Compilation fix: Correct method argument types in generation.rs and validation.rs (#10) 
* fix: Correct method argument types in generation and validation

In the `generation.rs` and `validation.rs` files, corrected the argument types
passed to the `decode` method. Replaced `Vec<u32>` with `&[u32]` using the
`as_ref()` method to match the expected argument types. This resolves the
mismatched types compilation error during the Rust build process.

Closes
[#9](https://github.com/Preemo-Inc/text-generation-inference/issues/9)

* Update benchmark/src/generation.rs

Co-authored-by: Yang, Bo <pop.atry@gmail.com>

* Update router/src/validation.rs

Co-authored-by: Yang, Bo <pop.atry@gmail.com>

---------

Co-authored-by: Yang, Bo <pop.atry@gmail.com>
 		2023-08-23 13:52:49 -07:00
..
src Compilation fix: Correct method argument types in generation.rs and validation.rs (#10) 2023-08-23 13:52:49 -07:00
Cargo.toml feat(benchmarker): add summary tables (#368) 2023-05-25 13:38:36 +02:00
README.md feat(benchmark): tui based benchmarking tool (#149) 2023-03-30 15:26:27 +02:00

README.md

Text Generation Inference benchmarking tool

benchmark

A lightweight benchmarking tool based inspired by oha and powered by tui.

Install

make install-benchmark

Run

First, start text-generation-inference:

text-generation-launcher --model-id bigscience/bloom-560m

Then run the benchmarking tool:

text-generation-benchmark --tokenizer-name bigscience/bloom-560m