Commit Graph

165 Commits

Author SHA1 Message Date
OlivierDehaene f9047562d0
fix(docker): fix image (#185) 2023-04-14 18:58:38 +02:00
OlivierDehaene 1bb394631d
fix(docker): fix docker image (#184) 2023-04-14 17:31:13 +02:00
OlivierDehaene 01c0e368e5
fix(ci): fix cosign error (#183) 2023-04-14 12:35:26 +02:00
OlivierDehaene 53ee09c0b0
fea(dockerfile): better layer caching (#159) 2023-04-14 10:12:21 +02:00
OlivierDehaene 12e5633c4d
fix(ci): fix ci permissions (#181) 2023-04-13 16:32:37 +02:00
OlivierDehaene c1e2ea3b78
feat(ci): faster scanning (#180) 2023-04-13 16:23:47 +02:00
OlivierDehaene 13f1cd024b
feat(ci): use large runners (#179) 2023-04-13 16:11:48 +02:00
OlivierDehaene 9683c37bd3
feat(ci): add Trivy and scan docker image (#178) 2023-04-13 15:43:17 +02:00
OlivierDehaene 643a39d556
feat(ci): add image signing with cosign (#175) 2023-04-13 15:26:34 +02:00
OlivierDehaene 64347b05ff
fix(ci): fix CVE in github-slug-action (#174) 2023-04-13 12:43:05 +02:00
OlivierDehaene e3a63b6fbc
fix(launcher): revert change on shard errors (#173) 2023-04-13 11:07:11 +02:00
OlivierDehaene 880a76eed5
feat(server): support sharded santacoder (#167) 2023-04-12 17:18:08 +02:00
OlivierDehaene 5fa8ae041c
feat(server): optimize decode for sane tokenizers (#170) 2023-04-12 12:03:10 +02:00
OlivierDehaene 6f0f1d70f6
v0.5.0 (#168) 2023-04-11 20:32:18 +02:00
OlivierDehaene f26dfd0dc1
feat(server): support OPT models (#55)
OPT models do not all have a `tokenizer.json` file on the hub at the
moment. Can't merge for now.
2023-04-11 19:16:41 +02:00
OlivierDehaene 299217c95c
feat(server): add flash attention llama (#144) 2023-04-11 16:38:22 +02:00
OlivierDehaene 9987960062
feat(router): make router input validation optional (#164) 2023-04-09 20:22:27 +02:00
OlivierDehaene 7dec65a244
fix(router): use buckets for metrics histograms (#163) 2023-04-09 20:13:28 +02:00
OlivierDehaene 5cddc055e6
fix(rust-client): use join_all instead of select_all to hopefully fix nccl issues (#162) 2023-04-09 20:07:02 +02:00
OlivierDehaene e63a21eb4d
feat(launcher): allow disabling hf_transfer (#161) 2023-04-09 20:00:05 +02:00
OlivierDehaene 1883d8ecde
feat(docker): improve flash_attention caching (#160) 2023-04-09 19:59:16 +02:00
OlivierDehaene 3f2542bb6a
fix(server): fix escape characters in stop sequence (#155) 2023-04-05 19:37:41 +02:00
Guspan Tanadi 9122e7bd9c
docs(readme): provide link Logits Warper README (#154) 2023-04-04 13:27:46 +02:00
OlivierDehaene c0aeb32583
feat(server): flash santacoder (#153) 2023-04-03 19:06:42 +02:00
OlivierDehaene fef1a1c381
v0.4.3 (#152) 2023-03-30 17:28:14 +02:00
OlivierDehaene 84722f3e33
v0.4.2 (#151) 2023-03-30 17:10:01 +02:00
OlivierDehaene 08b7e4a282
fix(server): fix flash neox rotary embeddings (#150) 2023-03-30 16:12:23 +02:00
OlivierDehaene 610bb1f978
feat(benchmark): tui based benchmarking tool (#149) 2023-03-30 15:26:27 +02:00
OlivierDehaene 55106ec476
fix(ci): fix sagemaker action (#148) 2023-03-29 22:27:01 +02:00
OlivierDehaene d503e8f09d
feat: aws sagemaker compatible image (#147)
The only difference is that now it pushes to
registry.internal.huggingface.tech/api-inference/community/text-generation-inference/sagemaker:...
instead of
registry.internal.huggingface.tech/api-inference/community/text-generation-inference:sagemaker-...

---------

Co-authored-by: Philipp Schmid <32632186+philschmid@users.noreply.github.com>
2023-03-29 21:38:30 +02:00
OlivierDehaene c9bdaa8b73
feat(server): reduce mlp and attn in one op for flash neox (#145) 2023-03-28 16:51:41 +02:00
OlivierDehaene f000068944
feat(server): clear cache on error (#143) 2023-03-28 11:29:35 +02:00
Nick Hill 8e8dd984d8
feat(server): Add mypy-protobuf (#141)
Generates .pyi files for protobuf stubs which provide strong typing
information. Very helpful for IDE auto-completion, etc.
2023-03-27 09:25:15 +02:00
Nick Hill 462530c2b0
fix(server): Avoid using try/except to determine kind of AutoModel (#142) 2023-03-27 09:23:22 +02:00
OlivierDehaene ab5fd8cf93
v0.4.1 (#140) 2023-03-26 16:37:51 +02:00
OlivierDehaene 678b2f3900
feat(server): cleanup flash neox loading (#139) 2023-03-26 16:37:21 +02:00
OlivierDehaene d6a93fe992
fix(server): fix flash-neox scores warping (#137) 2023-03-24 18:21:41 +01:00
OlivierDehaene 05e9a796cc
feat(server): flash neoX (#133) 2023-03-24 14:02:14 +01:00
OlivierDehaene 23e1028822
feat(python-client): add CI (#136) 2023-03-23 18:13:04 +01:00
OlivierDehaene 5d04525cb9
feat(python-client): release v0.4.0 (#135) 2023-03-23 18:07:20 +01:00
lewtun 5e5e9d4bbd
feat: Add note about NVIDIA drivers (#64)
Co-authored-by: OlivierDehaene <olivier@huggingface.co>
2023-03-23 18:03:45 +01:00
OlivierDehaene 603e20b5f7
feat(ci): add ci paths (#134) 2023-03-23 18:01:30 +01:00
dconathan 7850119055
feat(python-client): add cookies to Client constructors and requests (#132)
I have a use case where we need to pass cookies (for auth reasons) to an
internally hosted server.

Note: I couldn't get the client tests to pass - do you need to have an
HF token?

```python
FAILED tests/test_client.py::test_generate - text_generation.errors.BadRequestError: Authorization header is correct, but the token seems invalid
```
2023-03-23 18:01:01 +01:00
OlivierDehaene a3b7db932f
fix(python-client): relax dependencies (#129) 2023-03-16 12:57:07 +01:00
OlivierDehaene b49dbf2d88
fix(server): use server tokenizer as gt (#128) 2023-03-16 12:12:26 +01:00
OlivierDehaene 8ad60b752f
fix(server): add position ids to neox (#126) 2023-03-15 13:12:49 +01:00
OlivierDehaene cbd36aa4d1
fix(server): revert gpt-neox optims (#123) 2023-03-13 22:57:08 +01:00
OlivierDehaene 6860ce9c67
feat: add OpenAssistant/oasst-sft-1-pythia-12b to the list of supported models (#122)
…ed models
2023-03-13 20:42:10 +01:00
OlivierDehaene 411d6247f4
v0.4.0 (#119) 2023-03-09 16:07:01 +01:00
OlivierDehaene d8dc8f1b0c
feat(python-client): add new parameters (#118) 2023-03-09 16:05:33 +01:00