deec30f893
The minimum batch size logic could cause prefix blocks to be deallocated without prefill. The next allocation of the same prefix would then use garbage blocks. |
||
---|---|---|
.. | ||
client | ||
grpc-metadata | ||
trtllm | ||
v3 |