hf_text-generation-inference/router/src
Nicolas Patry f512021e77
Stream options. (#2533)
* Stream options.

* Fetch stuff from nix integration test for easier testing.

* Adding the assert.

* Only send the usage when asked for.

* Update the docs.

* Impure test because we need network.

* develop.

* Optional usage.

* Fixes.

* Workflow
2024-09-19 20:50:37 +02:00
..
infer Prefix test - Different kind of load test to trigger prefix test bugs. (#2490) 2024-09-11 18:10:40 +02:00
config.rs add gptj modeling in TGI #2366 (CI RUN) (#2372) 2024-08-07 21:32:37 -04:00
kserve.rs fix: simplify kserve endpoint and fix imports (#2119) 2024-06-25 19:30:10 -04:00
lib.rs Stream options. (#2533) 2024-09-19 20:50:37 +02:00
logging.rs Rebase TRT-llm (#2331) 2024-07-31 10:33:10 +02:00
main.rs.back Rebase TRT-llm (#2331) 2024-07-31 10:33:10 +02:00
server.rs Stream options. (#2533) 2024-09-19 20:50:37 +02:00
usage_stats.rs refactor usage stats (#2339) 2024-07-31 16:29:07 +02:00
validation.rs Lots of improvements (Still 2 allocators) (#2449) 2024-08-29 16:29:01 +02:00