Daniël de Kok 52e48739a5 Remove vLLM dependency for CUDA (#2751) ... * Remove vLLM dependency for CUDA This change adds `attention-kernels` as a dependency for paged attention and cache reshaping. With that, we don't use vLLM anywhere for CUDA. Tested run (since we don't have paged attention in CI): ``` ❯ ATTENTION=paged python -m pytest integration-tests -k "llama and awq" --release [...] 5 snapshots passed. ``` * Fix clippy warning		2024-11-17 17:34:50 +01:00
..
attention	Remove vLLM dependency for CUDA (#2751)	2024-11-17 17:34:50 +01:00
awq	fix incorrect output of Qwen2-7B-Instruct-GPTQ-Int4 and Qwen2-7B-Inst… (#2717)	2024-11-04 16:07:51 +01:00
compressed_tensors	Add initial support for compressed-tensors checkpoints (#2732)	2024-11-10 13:54:07 +01:00
gptq	fix incorrect output of Qwen2-7B-Instruct-GPTQ-Int4 and Qwen2-7B-Inst… (#2717)	2024-11-04 16:07:51 +01:00
marlin	Add initial support for compressed-tensors checkpoints (#2732)	2024-11-10 13:54:07 +01:00
moe	Add support for fused MoE Marlin for AWQ (#2616)	2024-10-08 11:56:41 +02:00
__init__.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
bnb.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
conv.py	Refactor layers. (#1866)	2024-05-13 12:44:30 +02:00
eetq.py	feat(fp8): use fbgemm kernels and load fp8 weights directly (#2248)	2024-07-20 19:02:04 +02:00
exl2.py	Add support for Deepseek V2 (#2224)	2024-07-19 17:23:20 +02:00
fp8.py	Add initial support for compressed-tensors checkpoints (#2732)	2024-11-10 13:54:07 +01:00
layernorm.py	Removing IPEX_AVAIL. (#2115)	2024-06-25 13:20:57 +02:00
linear.py	Update ROCM libs and improvements (#2579)	2024-09-30 10:54:32 +02:00
lora.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
medusa.py	Prefix caching (#2402)	2024-08-20 11:15:30 +02:00
mlp.py	Tied embeddings in MLP speculator. (#2473)	2024-08-29 17:44:54 +02:00
rotary.py	Support qwen2 vl (#2689)	2024-10-30 12:40:51 -04:00
speculative.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
tensor_parallel.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00