hf_text-generation-inference/backends
Funtowicz Morgan 3f385991b0
More fixes trtllm (#2342)
* (backend) use parking_lot crate for RwLock fairness

* (docker) let's put rust in the TRTLLM folder when building

* (docker) build ompi with SLURM support

* (launcher) default new server::run parameters to false for now

* (chore) fmt ... why?
2024-08-14 12:02:05 +02:00
..
client Add support for prefix caching to the v3 router (#2392) 2024-08-12 14:59:17 +02:00
grpc-metadata Rebase TRT-llm (#2331) 2024-07-31 10:33:10 +02:00
trtllm More fixes trtllm (#2342) 2024-08-14 12:02:05 +02:00
v3 Keeping the benchmark somewhere (#2401) 2024-08-12 15:22:02 +02:00