hf_text-generation-inference/server/text_generation_server
Daniël de Kok e903770897
Support different image sizes in prefill in VLMs (#2065)
When a batch contained images if different sizes during prefill, the
server would fail (see e.g. #2056). Images were processed separately and
then concatenated. However, this can fail for images with different sizes.

Fix this by preprocessing all images in the batch together, so that the
image processor can ensure that all image tensors have compatible sizes.
2024-06-17 10:49:41 +02:00
..
layers Add support for GPTQ Marlin (#2052) 2024-06-14 09:45:42 +02:00
models Support different image sizes in prefill in VLMs (#2065) 2024-06-17 10:49:41 +02:00
pb chore: add pre-commit (#1569) 2024-02-16 11:58:58 +01:00
utils Add support for GPTQ Marlin (#2052) 2024-06-14 09:45:42 +02:00
__init__.py feat(clients): Python client (#103) 2023-03-07 18:52:22 +01:00
cache.py fix(server): decrease memory fragmentation (#557) 2023-07-06 14:28:33 +02:00
cli.py ROCm and sliding windows fixes (#2033) 2024-06-10 15:09:50 +08:00
interceptor.py v2.0.0 (#1736) 2024-04-12 18:38:34 +02:00
server.py ROCm and sliding windows fixes (#2033) 2024-06-10 15:09:50 +08:00
tracing.py feat(clients): Python client (#103) 2023-03-07 18:52:22 +01:00