cyberes
/
local-llm-server
Archived
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

docker: clean up

This commit is contained in:
Cyberes 2023-10-15 17:14:14 -06:00
parent bc74a3dc50
commit a5f31ca831
4 changed files with 3 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
other/vllm/Docker/Dockerfile
View File

@ -17,6 +17,7 @@ RUN apt-get update && \
apt-get install -y wget aria2 git-lfs git openssh-server openssh-client nano tmux file && \ apt-get install -y wget aria2 git-lfs git openssh-server openssh-client nano tmux file && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
# Update the git repo
RUN cd /local-llm-server && git reset --hard && git pull RUN cd /local-llm-server && git reset --hard && git pull
# Enable root SSH login # Enable root SSH login

2
other/vllm/Docker/README.md
View File

@ -30,7 +30,7 @@
You **must** have a GPU attached to your system when building the container (required for building VLLM). You **must** have a GPU attached to your system when building the container (required for building VLLM).
1. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) and CUDA 11.8. 1. Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) and CUDA 11.8.
2. `sudo docker build .` If you want to build the latest VLLM, add `--no-cache` 2. `bash build-docker.sh`
To run the container on your local machine: To run the container on your local machine:

1
other/vllm/Docker/build-docker.sh
View File

@ -2,7 +2,6 @@
# Build and push the container. # Build and push the container.
git pull
git pull git pull
sudo docker build . -f Dockerfile.base -t cyberes/vllm-paperspace-base --no-cache && sudo docker push cyberes/vllm-paperspace-base:latest sudo docker build . -f Dockerfile.base -t cyberes/vllm-paperspace-base --no-cache && sudo docker push cyberes/vllm-paperspace-base:latest
sudo docker build . -t cyberes/vllm-paperspace && sudo docker push cyberes/vllm-paperspace:latest sudo docker build . -t cyberes/vllm-paperspace && sudo docker push cyberes/vllm-paperspace:latest

5
other/vllm/Docker/init-container.sh
View File

@ -18,8 +18,5 @@ if [ ! -f /storage/vllm/cmd.txt ]; then
echo "--max-num-batched-tokens 4098 --quantization awq --model /storage/vllm/models/model-path" >/storage/vllm/cmd.txt echo "--max-num-batched-tokens 4098 --quantization awq --model /storage/vllm/models/model-path" >/storage/vllm/cmd.txt
fi fi
# Copy the idling notebook to storage. This will create a blank notebook every # Copy the idling notebook to storage. This will create a blank notebook every time the container is started.
# time the container is started.
cp /local-llm-server/other/vllm/Docker/idle.ipynb /notebooks/idle.ipynb cp /local-llm-server/other/vllm/Docker/idle.ipynb /notebooks/idle.ipynb
# cp /local-llm-server/other/vllm/Docker/update-container.sh /notebooks/update-container.sh