6 Commits

Author	SHA1	Message	Date
fxmarty	227f78f3fe	Merge branch 'main' into ci_amd3	2024-06-26 12:08:42 +02:00
Daniël de Kok	fc9c3153e5	Add pytest release marker (#2114) ... * Add pytest release marker Annotate a test with `@pytest.mark.release` and it only gets run with `pytest integration-tests --release`. * Mark many models as `release` to speed up CI	2024-06-25 16:53:20 +02:00
Felix Marty	284894303a	remove require_backend decorators on handles, for some reasons fails in github actions	2024-06-24 18:51:32 +02:00
fxmarty	3de8f3647b	fix decorators	2024-06-24 18:51:08 +02:00
fxmarty	406885638b	skip exl2 tests on rocm	2024-06-24 18:49:45 +02:00
Daniël de Kok	36dd16017c	Add support for exl2 quantization ... Mostly straightforward, changes to existing code: * Wrap quantizer parameters in a small wrapper to avoid passing around untyped tuples and needing to repack them as a dict. * Move scratch space computation to warmup, because we need the maximum input sequence length to avoid allocating huge scratch buffers that OOM.	2024-05-30 11:28:05 +02:00