Commit Graph

7 Commits

Author SHA1 Message Date
Felix Marty 9fd395fae4 fix tests 2024-07-01 12:12:26 +00:00
fxmarty 227f78f3fe Merge branch 'main' into ci_amd3 2024-06-26 12:08:42 +02:00
Daniël de Kok fc9c3153e5
Add pytest release marker (#2114)
* Add pytest release marker

Annotate a test with `@pytest.mark.release` and it only gets run
with `pytest integration-tests --release`.

* Mark many models as `release` to speed up CI
2024-06-25 16:53:20 +02:00
Felix Marty 284894303a
remove require_backend decorators on handles, for some reasons fails in github actions 2024-06-24 18:51:32 +02:00
fxmarty 3de8f3647b
fix decorators 2024-06-24 18:51:08 +02:00
fxmarty 406885638b
skip exl2 tests on rocm 2024-06-24 18:49:45 +02:00
Daniël de Kok 36dd16017c Add support for exl2 quantization
Mostly straightforward, changes to existing code:

* Wrap quantizer parameters in a small wrapper to avoid passing
  around untyped tuples and needing to repack them as a dict.
* Move scratch space computation to warmup, because we need the
  maximum input sequence length to avoid allocating huge
  scratch buffers that OOM.
2024-05-30 11:28:05 +02:00