hf_text-generation-inference/launcher
OlivierDehaene ebc74d5666
feat(router): use number of tokens in batch as input for dynamic batching (#226)
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
2023-04-24 17:59:00 +02:00
..
src feat(router): use number of tokens in batch as input for dynamic batching (#226) 2023-04-24 17:59:00 +02:00
tests feat(server): add flash attention llama (#144) 2023-04-11 16:38:22 +02:00
Cargo.toml v0.6.0 (#222) 2023-04-21 21:00:57 +02:00