Added note on weight-cache-override (#994)
Added note on serving supported models from a different folder without re-downloading them. --------- Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
This commit is contained in:
parent
a049864270
commit
36c2868853
|
@ -28,6 +28,12 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")`
|
||||||
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
|
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you wish to serve a supported model that already exists on a local folder, just point to the local folder.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
text-generation-launcher --model-id <PATH-TO-LOCAL-BLOOM>
|
||||||
|
``````
|
||||||
|
|
||||||
|
|
||||||
## Supported Hardware
|
## Supported Hardware
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue