hf_text-generation-inference

Commit Graph

Author	SHA1	Message	Date
OlivierDehaene	8aece3bd68	feat: move allocation logic to rust (#1835 ) Close #2007	2024-06-05 12:18:38 +02:00
OlivierDehaene	757223b352	feat: add SchedulerV3 (#1996 ) - Refactor code to allow supporting multiple versions of the generate.proto at the same time - Add v3/generate.proto (ISO to generate.proto for now but allow for future changes without impacting v2 backends) - Add Schedule trait to abstract queuing and batching mechanisms that will be different in the future - Add SchedulerV2/V3 impl	2024-06-04 15:56:56 +02:00

Author

SHA1

Message

Date

OlivierDehaene

8aece3bd68

feat: move allocation logic to rust (#1835 )

Close #2007

2024-06-05 12:18:38 +02:00

OlivierDehaene

757223b352

feat: add SchedulerV3 (#1996 )

- Refactor code to allow supporting multiple versions of the
generate.proto at the same time
- Add v3/generate.proto (ISO to generate.proto for now but allow for
future changes without impacting v2 backends)
- Add Schedule trait to abstract queuing and batching mechanisms that
will be different in the future
- Add SchedulerV2/V3 impl

2024-06-04 15:56:56 +02:00

2 Commits