Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,026 Commits 179 Branches 50 Tags 605 MiB
Commit Graph

1026 Commits

Author SHA1 Message Date
Nicolas Patry 5838f2139f
Tied embeddings in MLP speculator. 2024-08-29 12:30:26 +02:00
Nicolas Patry 5e2932552c
Revert the Cohere tokenizer change (for now using a revision instead). 2024-08-29 11:35:18 +02:00
Nicolas Patry fc7ea202c2
Fix disabling prefix caching - Fix windowing checks. 2024-08-29 11:34:50 +02:00
Nicolas Patry bef2f6bdaa
Fixing the free algorithm to handle times where the common prefix is 
smaller.
 2024-08-29 09:17:00 +02:00
Nicolas Patry 9c839ca5df
Adding error message when assert is violated. 2024-08-28 21:22:36 +02:00
Nicolas Patry e7e036389e
Revert the integrationt tests change (seem linked to head_size 
modification).
 2024-08-28 19:38:51 +02:00
Nicolas Patry 8a4df6e181
Only n_heads / process_group.size() are necessary. 2024-08-28 16:34:58 +02:00
Nicolas Patry 8d01848370
Update server tests 
- Default to throughput test in k6
- Use TGI_WIGGLE_ROOM to adjust wiggle room
 2024-08-28 15:42:05 +02:00
Nicolas Patry 12325564dc
Put back default pure shell. 2024-08-28 14:54:05 +02:00
Nicolas Patry f886747949
Oops this doesn't belong here. 2024-08-28 14:49:00 +02:00
Nicolas Patry e6ee67f301
Truncating left for radix purposes. 2024-08-28 10:53:22 +02:00
Nicolas Patry 0a60973166
Fixing the batching tokenization in flash causal lm. 2024-08-28 10:34:10 +02:00
Nicolas Patry c6f1a61267
Update the chat test. 2024-08-27 23:02:12 +02:00
Nicolas Patry 8ac1ffa087
Removing encoder_decoder (seq2seq). 2024-08-27 21:11:49 +02:00
Nicolas Patry ccaf1d0030
Fixing the test. 2024-08-27 20:06:12 +02:00
Nicolas Patry 2cf1f5c00e
Fixing the issue with `add_special_tokens` not being passed around. 2024-08-27 20:06:12 +02:00
Nicolas Patry e0069a3a26
Fixing seqlen with the new vlms. 2024-08-27 20:06:12 +02:00
Nicolas Patry 9dacac3b15
add_special_tokens is internal only 2024-08-27 20:06:12 +02:00
Nicolas Patry 55d984d730
Fixed flashinfer version. 2024-08-27 20:06:12 +02:00
Nicolas Patry bb9769ed42
Update all models. 2024-08-27 20:06:11 +02:00
Nicolas Patry 65b94a69bd
Fixing prefix caching for flashdecoding. 2024-08-27 20:06:11 +02:00
Nicolas Patry 7f1816a4e1
Change `add_special_tokens` in order to have the correct tokens for chat 
input and not (since it's super important with the prefixing now)
 2024-08-27 20:06:11 +02:00
Nicolas Patry f1c0735453
Don't enable prefix caching on VLM just yet. 2024-08-27 20:06:11 +02:00
Nicolas Patry e30fb25444
Fixing the default for vlm. 2024-08-27 20:06:11 +02:00
Nicolas Patry 27b566baa8
Downgrade some logs. 2024-08-27 20:06:11 +02:00
Nicolas Patry 26e5037de4
This seems to be working. 2024-08-27 20:06:10 +02:00
Nicolas Patry f5182c188c
Is this enough to make it work ? 2024-08-27 20:06:10 +02:00
Nicolas Patry 1568e82548
OVerride the env in server tests. 2024-08-27 20:06:10 +02:00
Nicolas Patry 682db34b6a
Handling debugger. 2024-08-27 20:06:10 +02:00
Nicolas Patry c53968dc45
Remove lambda for cleaner function. 2024-08-27 20:06:10 +02:00
Nicolas Patry 32f6416358
Upgrade resolution system for less errors in resolution. 2024-08-27 20:06:10 +02:00
Nicolas Patry 5eb6ea0063
Tmp 2024-08-27 20:06:09 +02:00
Nicolas Patry 0bf4eb9683
Updated flake lock 2024-08-27 20:06:09 +02:00
Nicolas Patry b80593bfa3
Apply suggestions from code review 
Co-authored-by: drbh <david.richard.holtz@gmail.com>
 2024-08-27 20:06:09 +02:00
Nicolas Patry 8d0220a695
Forgot last default place. 2024-08-27 20:06:09 +02:00
Nicolas Patry 860b550cdf
Everywhere 1.80 2024-08-27 20:06:09 +02:00
Nicolas Patry 344fee0d44
Upgrade to 1.80 because of bitstream... 2024-08-27 20:06:09 +02:00
Nicolas Patry 17c8a5e574
Update cargo lock ? 2024-08-27 20:06:06 +02:00
Nicolas Patry ba1ce20ce8
Updating integration tests with new values with FI/FD. 
Remove paged as a default too, and using FD everywhere.
 2024-08-27 20:05:29 +02:00
Nicolas Patry ffb6841121
Update lock 2024-08-27 20:05:29 +02:00
Nicolas Patry f0b35f94b8
More specific codes. 2024-08-27 20:05:29 +02:00
Nicolas Patry a6cd5fef23
Disable prefix caching for lora. 2024-08-27 20:05:29 +02:00
Nicolas Patry cba59aca03
Disabling flashinfer/prefix caching on odd head_dim 2024-08-27 20:05:29 +02:00
Nicolas Patry f55278de2d
Allowing window_left_size (dummy version). 2024-08-27 20:05:29 +02:00
Nicolas Patry f2bdc65098
Using prebuilt. 2024-08-27 20:05:28 +02:00
Nicolas Patry 9d4c5d39fe
Include flashinfer in the docker. 2024-08-27 20:05:28 +02:00
Nicolas Patry 60719babf6
Making prefix/flashinfer the default and testing the full release tests. 2024-08-27 20:05:28 +02:00
drbh 21187c27c9
fix: bump minijinja version and add test for llama 3.1 tools (#2463) 
* fix: support tojson and avoid message indexing issue in template

* fix: prefer minijinja native methods and prefer workspace level dependency

* fix: adjust comment typo
 2024-08-27 13:31:08 -04:00
Nicolas Patry 2788d41a76
Fixing CI. (#2462) 2024-08-27 15:33:02 +02:00
drbh cfa73b5c99
Pr 2451 ci branch (#2454) 
* fix[router]: Fix tools not passed in chat template

Signed-off-by: GitHub <noreply@github.com>

* feat: improve default tool serialization and lints

* feat: refactor tool logic to include notify_error in prompt and adjust typing

* fix: adjust non tool template apply

* fix: simplify tool grammar logic and improve schema

* feat: avoid skip tool test and avoid empty tool prompts

* fix: increase test client timeout for grammar compilation tests

---------

Signed-off-by: GitHub <noreply@github.com>
Co-authored-by: Simone Rossi <simone.rossi.93@gmail.com>
 2024-08-26 20:19:38 -04:00