Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
hf_text-generation-inference/server/text_generation_server/models
History
drbh 30be188400
Fix: don't apply post layernorm in SiglipVisionTransformer (#2459) 
* Fix: don't apply post layernorm in SiglipVisionTransformer

This fixes a bug with LLaVA Next when using Siglip as the vision model. LLaVA Next expects the output of the vision model to be the encoder outputs before layernorm (see original transformers implementation here: https://github.com/huggingface/transformers/blob/main/src/transformers/models/llava_next/modeling_llava_next.py#L813).

This also makes Siglip consistent with the existing Clip implementation:

https://github.com/huggingface/text-generation-inference/blob/main/server/text_generation_server/models/custom_modeling/clip.py#L613

* fix: adjust pali gemma for post layer norm and small refactors

---------

Co-authored-by: Travis Addair <tgaddair@gmail.com>
 		2024-08-26 17:04:46 -04:00
..
custom_modeling Fix: don't apply post layernorm in SiglipVisionTransformer (#2459) 2024-08-26 17:04:46 -04:00
__init__.py feat: validate template variables before apply and improve sliding wi… (#2403) 2024-08-12 10:58:40 -04:00
bloom.py Refactor dead code - Removing all `flash_xxx.py` files. (#2166) 2024-07-05 10:29:56 +02:00
causal_lm.py Fixing exl2 and other quanize tests again. (#2419) 2024-08-15 11:12:51 +02:00
flash_causal_lm.py Prefix caching (#2402) 2024-08-20 11:15:30 +02:00
galactica.py feat: add ruff and resolve issue (#2262) 2024-07-26 10:29:09 -04:00
globals.py Prefix caching (#2402) 2024-08-20 11:15:30 +02:00
idefics.py Upgrading exl2. (#2415) 2024-08-14 11:58:08 +02:00
idefics_causal_lm.py Upgrading exl2. (#2415) 2024-08-14 11:58:08 +02:00
mamba.py Fixing exl2 and other quanize tests again. (#2419) 2024-08-15 11:12:51 +02:00
model.py feat: add ruff and resolve issue (#2262) 2024-07-26 10:29:09 -04:00
pali_gemma.py feat: add ruff and resolve issue (#2262) 2024-07-26 10:29:09 -04:00
seq2seq_lm.py Fixing exl2 and other quanize tests again. (#2419) 2024-08-15 11:12:51 +02:00
types.py feat: add ruff and resolve issue (#2262) 2024-07-26 10:29:09 -04:00
vlm_causal_lm.py Prefix caching (#2402) 2024-08-20 11:15:30 +02:00