Daniël de Kok 72ab60fdd5 Use FP8 KV cache when specified by compressed-tensors (#2761) ... The compressed-tensors configuration can specify the configuration of the KV cache as well. Use an FP8 KV cache when the configuration tells us to do so (all other options and types are ignored for now).		2024-11-26 08:27:41 +01:00
..
custom_modeling	fix: adjust llama MLP name from dense to mlp to correctly apply lora (#2760)	2024-11-19 15:10:22 -05:00
__init__.py	Use FP8 KV cache when specified by compressed-tensors (#2761)	2024-11-26 08:27:41 +01:00
bloom.py	Refactor dead code - Removing all `flash_xxx.py` files. (#2166)	2024-07-05 10:29:56 +02:00
causal_lm.py	feat: prefill chunking (#2600)	2024-10-16 12:49:33 +02:00
flash_causal_lm.py	feat: add payload limit (#2726)	2024-11-21 18:20:15 +00:00
galactica.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
globals.py	feat: prefill chunking (#2600)	2024-10-16 12:49:33 +02:00
idefics_causal_lm.py	feat: prefill chunking (#2600)	2024-10-16 12:49:33 +02:00
mamba.py	Choosing input/total tokens automatically based on available VRAM? (#2673)	2024-10-28 04:59:49 +01:00
metadata_kernels.py	feat: add payload limit (#2726)	2024-11-21 18:20:15 +00:00
mllama_causal_lm.py	feat: add triton kernels to decrease latency of large batches (#2687)	2024-10-25 21:10:00 +00:00
model.py	Choosing input/total tokens automatically based on available VRAM? (#2673)	2024-10-28 04:59:49 +01:00
pali_gemma.py	feat: add ruff and resolve issue (#2262)	2024-07-26 10:29:09 -04:00
seq2seq_lm.py	feat: prefill chunking (#2600)	2024-10-16 12:49:33 +02:00
types.py	feat: prefill chunking (#2600)	2024-10-16 12:49:33 +02:00
vlm_causal_lm.py	fix cuda graphs for qwen2-vl (#2708)	2024-11-01 03:05:34 +01:00