hf_text-generation-inference/proto
OlivierDehaene 757223b352
feat: add SchedulerV3 (#1996)
- Refactor code to allow supporting multiple versions of the
generate.proto at the same time
- Add v3/generate.proto (ISO to generate.proto for now but allow for
future changes without impacting v2 backends)
- Add Schedule trait to abstract queuing and batching mechanisms that
will be different in the future
- Add SchedulerV2/V3 impl
2024-06-04 15:56:56 +02:00
..
v3 feat: add SchedulerV3 (#1996) 2024-06-04 15:56:56 +02:00
generate.proto feat: add SchedulerV3 (#1996) 2024-06-04 15:56:56 +02:00