Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,063 Commits 187 Branches 52 Tags 606 MiB
Commit Graph

5 Commits

Author SHA1 Message Date
Mohit Sharma 47c81d2924 Merge remote-tracking branch 'upstream/main' into fix_rocm_fa 2024-09-27 10:34:16 +00:00
Mohit Sharma 829144d15a addressed review comments 2024-09-27 10:28:37 +00:00
Daniël de Kok 3f14cd1420
Add `DenseMoELayer` and wire it up in Mixtral/Deepseek V2 (#2537) 
This replaces the custom layers in both models.
 2024-09-24 14:27:06 +02:00
Mohit Sharma 4fb947d2aa fixed style 2024-09-19 14:28:21 +00:00
Daniël de Kok ce85efa968
Move to moe-kernels package and switch to common MoE layer (#2511) 
* Move to moe-kernels package and switch to common MoE layer

This change introduces the new `moe-kernels` package:

- Add `moe-kernels` as a dependency.
- Introduce a `SparseMoELayer` module that can be used by MoE
  models.
- Port over Mixtral and Deepseek.

* Make `cargo check` pass

* Update runner
 2024-09-17 18:08:58 +02:00