drbh da5ab46705 Improve vlm support (add idefics3 support) (#2437) ... * feat: expand vlm support and add image token logic and tests * fix: avoid unused perceiver config * feat: integrate image tokens into inputs embeds * feat: add simple idefics3 test * feat: update docs, image token logic and weight names * fix: improve image processing * feat: improve prefix for idefics3 * fix: bump idefics3 tests and snapshots * fix: improve text model loading * feat: consolidate changes with existing vlms and add support and test for smolvlm * fix: create new idefic3 file, simplify logic and adjust llama weight loading * fix: lint with ruff * fix: clean up idefics 3 and improve prefix handling * fix: improve typing * fix: improve prompt_split_image with ref to original impl * fix: adjust ruff lints and small refactors * fix: adjust FlashLlamaModel prefix logic		2025-01-09 10:35:32 -05:00
..
images	Pali gemma modeling (#1895)	2024-05-16 06:58:47 +02:00
models	Improve vlm support (add idefics3 support) (#2437)	2025-01-09 10:35:32 -05:00
conftest.py	Improve vlm support (add idefics3 support) (#2437)	2025-01-09 10:35:32 -05:00
poetry.lock	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00
pyproject.toml	nix: add black and isort to the closure (#2619)	2024-10-09 11:08:02 +02:00
pytest.ini	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
requirements.txt	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00