Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,053 Commits 179 Branches 50 Tags 605 MiB
Commit Graph

1053 Commits

Author SHA1 Message Date
Nicolas Patry 2f1f082abe
Tmate. 2024-09-16 17:03:28 +02:00
Nicolas Patry 1a6b9926f6
missing lib. 2024-09-16 17:03:27 +02:00
Nicolas Patry 332e42f59a
Attempt. 2024-09-16 17:03:27 +02:00
Nicolas Patry ec6fe324c6
Link to nix owned lib 2024-09-16 17:03:27 +02:00
Nicolas Patry 83ee55a617
Trye somethign. 2024-09-16 17:03:27 +02:00
Nicolas Patry 047530216c
No idea where the shared disk is. 2024-09-16 17:03:27 +02:00
Nicolas Patry 9f548fa82a
Change the home location ? 2024-09-16 17:03:27 +02:00
Nicolas Patry 3ff12084b7
Revert "No tmate." 
This reverts commit 6b9b6d951897127ae1ce09c8f61f86a64b301fec.
 2024-09-16 17:03:26 +02:00
Nicolas Patry 26634f9697
No tmate. 2024-09-16 17:03:26 +02:00
Nicolas Patry a533d086f0
Tmate to find cache. 2024-09-16 17:03:26 +02:00
Nicolas Patry a5b81ab457
Home. 2024-09-16 17:03:26 +02:00
Nicolas Patry 98f2241a88
Put back libnvidia-ml 2024-09-16 17:03:26 +02:00
Nicolas Patry 72a805d50d
Remove tmate. 2024-09-16 17:03:26 +02:00
Nicolas Patry 45c0129976
Attempting something. 2024-09-16 17:03:25 +02:00
Nicolas Patry 2b18537f85
More tmate. 2024-09-16 17:03:25 +02:00
Nicolas Patry 12b88204b0
Putting the cuda package in the flake. 2024-09-16 17:03:25 +02:00
Nicolas Patry d7333830b5
Tmate. 2024-09-16 17:03:25 +02:00
Nicolas Patry f8a7df41c4
More timeout to the healthcheck. 2024-09-16 17:03:05 +02:00
Nicolas Patry 68f7d75f23
Fixing the update to outlines. 2024-09-16 17:02:45 +02:00
Nicolas Patry c4bbe06bf1
Simpler command 2024-09-16 17:02:45 +02:00
Nicolas Patry d0ae24a167
Release tests. 2024-09-16 17:02:25 +02:00
Nicolas Patry 5c4b2eaa30
Seeing the damage on the release tests. 2024-09-16 17:01:51 +02:00
Nicolas Patry 2bd0117c27
Remove unused code. 2024-09-16 17:01:51 +02:00
Nicolas Patry 70f910bba6
Remove tmate. 2024-09-16 17:01:51 +02:00
Nicolas Patry b4654a36dc
Fixing up the tests ? 2024-09-16 17:01:51 +02:00
Nicolas Patry 5adece6313
This doesn't seem needed. 2024-09-16 17:01:51 +02:00
Nicolas Patry dd4b774e0d
New cargo lock 2024-09-16 17:01:44 +02:00
Nicolas Patry b7cb8d5145
Let's figure out the issue... 2024-09-16 17:01:30 +02:00
Nicolas Patry 3d7b81535a
Only link cuda driver librairies. 2024-09-16 17:01:30 +02:00
Nicolas Patry e898483db6
Updating outlines to 0.0.46 2024-09-16 17:01:30 +02:00
Nicolas Patry ce3efc83ed
Remove tmate. 2024-09-16 17:01:30 +02:00
Nicolas Patry 7f58f7dc61
Symlink all the things. 2024-09-16 17:01:29 +02:00
Nicolas Patry 42107de71f
Let's try to find libnvidia-ml 2024-09-16 17:01:29 +02:00
Nicolas Patry edaa7f847d
Does this work ? 2024-09-16 17:01:29 +02:00
Nicolas Patry d1e79ddae0
Fix override. 2024-09-16 17:01:29 +02:00
Nicolas Patry db054b95df
Check the paths. 2024-09-16 17:01:29 +02:00
Nicolas Patry afcd047a58
Yaml yaml. 2024-09-16 17:01:29 +02:00
Nicolas Patry 60db294f9a
Link cuda to nix ? 2024-09-16 17:01:28 +02:00
Nicolas Patry 8e7c7c61f1
Let's see what the issue is ? 2024-09-16 17:01:28 +02:00
Nicolas Patry 815449da74
Removing unused code. 2024-09-16 17:01:28 +02:00
Nicolas Patry c227345878
Run on actual GPUs. 2024-09-16 17:01:28 +02:00
Nicolas Patry 3d73c99ebe
Attempt at integration tests. 2024-09-16 17:01:28 +02:00
Nicolas Patry f47cdc1fe1
Attempting rapidly the integration tests. 2024-09-16 17:01:26 +02:00
Nicolas Patry 38fcafcf96
Adding a test for FD. (#2516) 
* Adding a test for FD.

* Fixing flashdecoding (empty batch doesn't work).

* Fixing the invalid popping.

* Fixing radix with block_size > 1

* Last reference.

* Use an actual hash.

* Update hash for slice.len() == 1

* Update the locks.

* Increasing docker timeout.
 2024-09-16 17:00:54 +02:00
Daniël de Kok 7774655297
Add tests for Mixtral (#2520) 
Disable by default because CI runners do not have enough GPUs.
 2024-09-16 12:39:18 +02:00
Alex Strick van Linschoten 9cca3e0b03
Use `ratatui` not (deprecated) `tui` (#2521) 
* use ratatui not archived tui

* bump ratatui all the way with options
 2024-09-13 18:45:28 +02:00
Wang, Yi 3ac7df2b6d
hotfix : enable intel ipex cpu and xpu in python3.11 (#2517) 
enable intel ipex cpu and xpu in python3.11

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2024-09-12 17:23:49 +02:00
drbh 628334d336
fix: pass missing revision arg for lora adapter when loading multiple… (#2510) 
fix: pass missing revision arg for lora adapter when loading multiple adapters
 2024-09-12 17:04:52 +02:00
Nicolas Patry d95c670ada
Add nix test. (#2513) 
* Add nix test.

* Modifying yourself means you need to rerun.

* Fixing the test + adding click (needed for pre-commit hooks).

* Try thuis.

* Our runner + pure test (not written)

* Reemove server.

* Root user.

* Different user ?

* Add the actual test target.

* Forgot this modification.

* Add a formatter.

* Add the secrets.

* Fixed the auth token ?

* Adding the other tests.

* Missing pre-commit.

* Test requires cargo for cargo fmt.

* Update it a bit.

* Up.

* Attempting to use a cache location for the models.

* Ignore the cache for now.
 2024-09-12 14:54:56 +02:00
Daniël de Kok 94304649f1
nix: support Python tokenizer conversion in the router (#2515) 
Ideally we wouldn't have the router wrapper that this change adds,
but when I give PyO3 a Python interpreter with packages, it ends
up linking libpython from the Python interpreter rather than the
constructed environment and cannot pick up the Python modules as
a result.
 2024-09-12 10:44:01 +02:00