Daniël de Kok 90a1d04a2f Add support for GPTQ-quantized MoE models using MoE Marlin (#2557) ... This change add support for MoE models that use GPTQ quantization. Currently only models with the following properties are supported: - No `desc_act` with tensor parallelism, unless `group_size=-1`. - No asymmetric quantization. - No AWQ.		2024-09-30 11:14:32 +02:00
..
images	Pali gemma modeling (#1895)	2024-05-16 06:58:47 +02:00
models	Add support for GPTQ-quantized MoE models using MoE Marlin (#2557)	2024-09-30 11:14:32 +02:00
conftest.py	Adding a test for FD. (#2516)	2024-09-16 17:00:54 +02:00
poetry.lock	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00
pyproject.toml	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00
pytest.ini	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
requirements.txt	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00