drbh 6f15ac60b2 feat: support force downcast after FastRMSNorm multiply for Gemma (#1658) ... This PR adds `force_downcast_after` to `FastRMSNorm.forward` which is used in the Gemma model. References https://github.com/huggingface/transformers/pull/29402 and https://github.com/huggingface/transformers/pull/29729 Setting `force_downcast_after=True` will perform the `hidden_states * weight` multiplication in f32 and then downcast to half. This differs slightly from the current implementation which first casts the `hidden_states` to a half and then multiples.		2024-03-21 10:25:11 +01:00
..
__init__.py	feat(server): flash santacoder (#153)	2023-04-03 19:06:42 +02:00
bloom_modeling.py	feat: Qwen2 (#1608)	2024-02-28 15:50:31 +01:00
flash_gemma_modeling.py	feat: support force downcast after FastRMSNorm multiply for Gemma (#1658)	2024-03-21 10:25:11 +01:00
flash_llama_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_mistral_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_mixtral_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_neox_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_phi_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_qwen2_modeling.py	feat: Qwen2 (#1608)	2024-02-28 15:50:31 +01:00
flash_rw_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_santacoder_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
flash_starcoder2_modeling.py	feat: starcoder2 (#1605)	2024-02-28 12:07:08 +01:00
idefics_config.py	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
idefics_image_processing.py	chore: formatting	2023-12-11 14:49:52 +01:00
idefics_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
idefics_perceiver.py	feat: format code (#1070)	2023-09-27 12:22:09 +02:00
idefics_processing.py	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
idefics_vision.py	feat: format code (#1070)	2023-09-27 12:22:09 +02:00
mamba_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
mpt_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
neox_modeling.py	feat: Qwen2 (#1608)	2024-02-28 15:50:31 +01:00
opt_modeling.py	feat: Qwen2 (#1608)	2024-02-28 15:50:31 +01:00
phi_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
t5_modeling.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00