update readme
This commit is contained in:
parent
3b56d7669b
commit
7a6fad6aac
|
@ -155,7 +155,7 @@ model=meta-llama/Llama-2-7b-chat-hf
|
|||
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
|
||||
token=<your cli READ token>
|
||||
|
||||
docker run --gpus all --shm-size 1g -e HUGGING_FACE_HUB_TOKEN=$token -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.0.3 --model-id $model
|
||||
docker run --gpus all --shm-size 1g -e HUGGING_FACE_HUB_TOKEN=$token -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.1.0 --model-id $model
|
||||
```
|
||||
|
||||
### A note on Shared Memory (shm)
|
||||
|
|
|
@ -85,7 +85,7 @@ curl 127.0.0.1:8080/generate \
|
|||
To see all possible deploy flags and options, you can use the `--help` flag. It's possible to configure the number of shards, quantization, generation parameters, and more.
|
||||
|
||||
```bash
|
||||
docker run ghcr.io/huggingface/text-generation-inference:1.0.3 --help
|
||||
docker run ghcr.io/huggingface/text-generation-inference:1.1.0 --help
|
||||
```
|
||||
|
||||
</Tip>
|
||||
|
|
Loading…
Reference in New Issue