hf_text-generation-inference/integration-tests
Daniël de Kok 2ce8019480
Use GPTQ-Marlin for supported GPTQ configurations (#2111)
GPTQ-Marlin is currently the best-performing kernel for GPTQ models. So
let's use it by default if the kernels are installed, the GPU supports
it, and the kernels support the configuration.

For models generated by `text-generation-server quantize`, use
`sym=False`. This subcommand symmetric quantization since the beginning
and incorrectly reporting the model to be symmetric will use
GPTQ-Marlin (which does not support asymmetric quantization).
2024-07-01 12:59:12 +02:00
..
images Pali gemma modeling (#1895) 2024-05-16 06:58:47 +02:00
models Use GPTQ-Marlin for supported GPTQ configurations (#2111) 2024-07-01 12:59:12 +02:00
conftest.py Add pytest release marker (#2114) 2024-06-25 16:53:20 +02:00
poetry.lock fix: improve tool type, bump pydantic and outlines (#1650) 2024-03-21 12:45:56 -04:00
pyproject.toml v2.0.1 2024-04-18 17:20:36 +02:00
pytest.ini chore: add pre-commit (#1569) 2024-02-16 11:58:58 +01:00
requirements.txt fix: improve tool type, bump pydantic and outlines (#1650) 2024-03-21 12:45:56 -04:00