Added note on weight-cache-override (#994)

Added note on serving supported models from a different folder without
re-downloading them.

---------

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
This commit is contained in:
Merve Noyan 2023-09-27 11:06:07 +02:00 committed by GitHub
parent a049864270
commit 36c2868853
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 6 additions and 0 deletions

View File

@ -28,6 +28,12 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")`
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto") AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
``` ```
If you wish to serve a supported model that already exists on a local folder, just point to the local folder.
```bash
text-generation-launcher --model-id <PATH-TO-LOCAL-BLOOM>
``````
## Supported Hardware ## Supported Hardware