drbh 6f15ac60b2 feat: support force downcast after FastRMSNorm multiply for Gemma (#1658) ... This PR adds `force_downcast_after` to `FastRMSNorm.forward` which is used in the Gemma model. References https://github.com/huggingface/transformers/pull/29402 and https://github.com/huggingface/transformers/pull/29729 Setting `force_downcast_after=True` will perform the `hidden_states * weight` multiplication in f32 and then downcast to half. This differs slightly from the current implementation which first casts the `hidden_states` to a half and then multiples.		2024-03-21 10:25:11 +01:00
..
models	feat: support force downcast after FastRMSNorm multiply for Gemma (#1658)	2024-03-21 10:25:11 +01:00
pb	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
utils	fix: correctly index into mask when applying grammar (#1618)	2024-03-01 18:22:01 +01:00
__init__.py	feat(clients): Python client (#103)	2023-03-07 18:52:22 +01:00
cache.py	fix(server): decrease memory fragmentation (#557)	2023-07-06 14:28:33 +02:00
cli.py	Revamp medusa implementation so that every model can benefit. (#1588)	2024-02-26 19:49:28 +01:00
interceptor.py	feat(server): empty cache on errors	2023-07-12 17:06:19 +02:00
server.py	fix: fix gpt-q with groupsize = -1 (#1358)	2023-12-18 16:07:05 +01:00
tracing.py	feat(clients): Python client (#103)	2023-03-07 18:52:22 +01:00