Nicolas Patry 90b226db29 We can have a tokenizer anywhere. (#2527) ... * We can have a tokenizer anywhere. * Handling potential lack of offsets (python tokenizer) * Remove redundancy. * Fixing the tests. * Flake.lock update ? * Fixing the GIL locking. * Fixing mamba by using the transformers version. * Adding the legacy handle. * Ellide lifetime. * Lint. * Deprecation message. * Fixing bad rebase.		2024-10-28 05:00:24 +01:00
..
images	Pali gemma modeling (#1895)	2024-05-16 06:58:47 +02:00
models	We can have a tokenizer anywhere. (#2527)	2024-10-28 05:00:24 +01:00
conftest.py	feat: prefill chunking (#2600)	2024-10-16 12:49:33 +02:00
poetry.lock	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00
pyproject.toml	nix: add black and isort to the closure (#2619)	2024-10-09 11:08:02 +02:00
pytest.ini	chore: add pre-commit (#1569)	2024-02-16 11:58:58 +01:00
requirements.txt	Prefix test - Different kind of load test to trigger prefix test bugs. (#2490)	2024-09-11 18:10:40 +02:00