Add simple script for running gitian-build
This shell script creates a docker image for running the gitian-build.py script, to avoid version incompatibilities. The builds may be signed automatically after manually reviewing the build hashes. Only docker.io and apt-cacher-ng are required to be installed on the host. The MacOSX SDK should also be present in this directory if Mac builds are desired. A container named "gitrun" is used to launch the builds, and all output will reside in this container as well.
This commit is contained in:
Howard Chu
parent
2243318000
commit
fc4e5741ae
|
|
@ -0,0 +1,85 @@
|
||||||
|
Quick Gitian building with docker
|
||||||
|
=================================
|
||||||
|
|
||||||
|
*Setup instructions for a Gitian build of Monero using Docker.*
|
||||||
|
|
||||||
|
Gitian supports other container mechanisms too but if you have a Debian or
|
||||||
|
Ubuntu-based host the steps can be greatly simplified.
|
||||||
|
|
||||||
|
Preparing the Gitian builder host
|
||||||
|
---------------------------------
|
||||||
|
|
||||||
|
The procedure here will create a docker container for build preparation, as well as
|
||||||
|
for actually running the builds. The only items you must install on your own host
|
||||||
|
are docker and apt-cacher-ng. With docker installed, you should also give yourself
|
||||||
|
permission to use docker by adding yourself to the docker group.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt-get install docker.io apt-cacher-ng
|
||||||
|
sudo usermod -aG docker $USER
|
||||||
|
su $USER
|
||||||
|
```
|
||||||
|
|
||||||
|
The final `su` command is needed to start a new shell with your new group membership,
|
||||||
|
since the `usermod` command doesn't affect any existing sessions.
|
||||||
|
|
||||||
|
If you want Mac binaries included in your build, you need to obtain the MacOS SDK:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -O https://bitcoincore.org/depends-sources/sdks/MacOSX10.11.sdk.tar.gz
|
||||||
|
```
|
||||||
|
|
||||||
|
Other User Preparation
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
The final step will be to `gpg` sign the results of your build and upload them to GitHub.
|
||||||
|
Before you can do that, you'll need
|
||||||
|
* a GitHub account.
|
||||||
|
If your GitHub account name is different from your local account name, you must
|
||||||
|
set your GitHub account name for the script to use:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export GH_USER=<github account name>
|
||||||
|
```
|
||||||
|
|
||||||
|
* PGP keys - if you don't have one already, you can use `gpg --quick-gen-key` to generate it.
|
||||||
|
* a fork of the [gitian.sigs](https://github.com/monero-project/gitian.sigs/) repo on your GitHub account.
|
||||||
|
Please follow the directions there for uploading your key first.
|
||||||
|
|
||||||
|
**Note:** Please ensure your gpg public key is available to check signatures by adding it to the [gitian.sigs/gitian-pubkeys/](https://github.com/monero-project/gitian.sigs/tree/master/gitian-pubkeys) directory in a pull request.
|
||||||
|
|
||||||
|
|
||||||
|
Building the Binaries
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
The dockrun.sh script will do everything to build the binaries. Just specify the
|
||||||
|
version to build as its only argument, e.g.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./dockrun.sh v0.17.2.3
|
||||||
|
```
|
||||||
|
|
||||||
|
The build should run to completion with no errors, and will display the SHA256 checksums
|
||||||
|
of the resulting binaries. You'll be prompted to check if the sums look good, and if so
|
||||||
|
then the results will be signed, and the signatures will be pushed to GitHub.
|
||||||
|
|
||||||
|
You can also look in the [gitian.sigs](https://github.com/monero-project/gitian.sigs/) repo and / or [getmonero.org release checksums](https://web.getmonero.org/downloads/hashes.txt) to see if others got the same checksum for the same version tag. If there is ever a mismatch -- **STOP! Something is wrong**. Contact others on IRC / GitHub to figure out what is going on.
|
||||||
|
|
||||||
|
|
||||||
|
Other Options
|
||||||
|
-------------
|
||||||
|
|
||||||
|
This script just runs the [gitian-build.py](gitian-build.py) inside a container named `gitrun`.
|
||||||
|
You can set other options for that script by setting the OPT variable when running `dockrun.sh`
|
||||||
|
e.g.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
OPT="-j 8" ./dockrun.sh v0.17.2.3
|
||||||
|
```
|
||||||
|
|
||||||
|
You can also examine the build and install logs by running a shell in the container, e.g.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec -it gitrun /bin/bash
|
||||||
|
more builder/var/install-linux.log
|
||||||
|
```
|
||||||
|
|
@ -31,6 +31,8 @@ This guide explains how to set up the environment, and how to start the builds.
|
||||||
|
|
||||||
* Gitian gives you the option of using any of 3 different virtualization tools: `kvm`, `docker` or `lxc`. This documentation will only show how to build with `lxc` and `docker` (documentation for `kvm` is welcome). Building with `lxc` is the default, but is more complicated, so we recommend docker your first time.
|
* Gitian gives you the option of using any of 3 different virtualization tools: `kvm`, `docker` or `lxc`. This documentation will only show how to build with `lxc` and `docker` (documentation for `kvm` is welcome). Building with `lxc` is the default, but is more complicated, so we recommend docker your first time.
|
||||||
|
|
||||||
|
* For a shortcut using `docker` follow the instructions in [DOCKRUN.md](DOCKRUN.md) instead
|
||||||
|
of following the rest of this document..
|
||||||
|
|
||||||
## Create the gitianuser account
|
## Create the gitianuser account
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,133 @@
|
||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
if [ $# -ne 1 ]; then
|
||||||
|
echo "usage: $0 <version>"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
VERSION=$1
|
||||||
|
|
||||||
|
DOCKER=`command -v docker`
|
||||||
|
CACHER=`command -v apt-cacher-ng`
|
||||||
|
|
||||||
|
if [ -z "$DOCKER" -o -z "$CACHER" ]; then
|
||||||
|
echo "$0: you must first install docker.io and apt-cacher-ng"
|
||||||
|
echo " e.g. sudo apt-get install docker.io apt-cacher-ng"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
GH_USER=${GH_USER-$USER}
|
||||||
|
|
||||||
|
TAG=gitrun-bionic
|
||||||
|
TAG2=base-bionic-amd64
|
||||||
|
IMAGE=`docker images | grep $TAG`
|
||||||
|
|
||||||
|
WORKDIR=/home/ubuntu
|
||||||
|
|
||||||
|
if [ -z "$IMAGE" ]; then
|
||||||
|
GID=`getent group docker`
|
||||||
|
mkdir -p docker
|
||||||
|
cd docker
|
||||||
|
|
||||||
|
# container for running gitian-build.py
|
||||||
|
cat <<EOF > ${TAG}.Dockerfile
|
||||||
|
FROM ubuntu:bionic
|
||||||
|
|
||||||
|
ENV DEBIAN_FRONTEND=noninteractive
|
||||||
|
RUN echo 'Acquire::http { Proxy "http://172.17.0.1:3142"; };' > /etc/apt/apt.conf.d/50cacher
|
||||||
|
RUN echo "$GID" >> /etc/group
|
||||||
|
RUN apt-get update && apt-get --no-install-recommends -y install lsb-release ruby git make wget docker.io python3 curl
|
||||||
|
|
||||||
|
RUN useradd -ms /bin/bash -U ubuntu -G docker
|
||||||
|
USER ubuntu:docker
|
||||||
|
WORKDIR $WORKDIR
|
||||||
|
|
||||||
|
RUN git clone https://github.com/monero-project/gitian.sigs.git sigs; \
|
||||||
|
git clone https://github.com/devrandom/gitian-builder.git builder; \
|
||||||
|
cd builder; git checkout c0f77ca018cb5332bfd595e0aff0468f77542c23; mkdir -p inputs var; cd inputs; \
|
||||||
|
git clone https://github.com/monero-project/monero
|
||||||
|
|
||||||
|
CMD ["sleep", "infinity"]
|
||||||
|
EOF
|
||||||
|
|
||||||
|
docker build --pull -f ${TAG}.Dockerfile -t $TAG .
|
||||||
|
|
||||||
|
cd ..
|
||||||
|
docker run -v /var/run/docker.sock:/var/run/docker.sock -d --name gitrun $TAG
|
||||||
|
if [ -f MacOSX10.11.sdk.tar.gz ]; then
|
||||||
|
docker cp MacOSX10.11.sdk.tar.gz gitrun:$WORKDIR/builder/inputs/
|
||||||
|
else
|
||||||
|
echo "No MacOS SDK found, Mac builds will be omitted"
|
||||||
|
fi
|
||||||
|
|
||||||
|
fi
|
||||||
|
|
||||||
|
IMAGE=`docker images | grep $TAG2`
|
||||||
|
if [ -z "$IMAGE" ]; then
|
||||||
|
mkdir -p docker
|
||||||
|
cd docker
|
||||||
|
|
||||||
|
# container for actually running each build
|
||||||
|
cat <<EOF > ${TAG2}.Dockerfile
|
||||||
|
FROM ubuntu:bionic
|
||||||
|
|
||||||
|
ENV DEBIAN_FRONTEND=noninteractive
|
||||||
|
RUN echo 'Acquire::http { Proxy "http://172.17.0.1:3142"; };' > /etc/apt/apt.conf.d/50cacher
|
||||||
|
RUN apt-get update && apt-get --no-install-recommends -y install build-essential git language-pack-en \
|
||||||
|
wget lsb-release curl gcc-7 g++-7 gcc g++ binutils-gold pkg-config autoconf libtool automake faketime \
|
||||||
|
bsdmainutils ca-certificates python cmake gperf
|
||||||
|
|
||||||
|
RUN useradd -ms /bin/bash -U ubuntu
|
||||||
|
USER ubuntu:ubuntu
|
||||||
|
WORKDIR $WORKDIR
|
||||||
|
|
||||||
|
CMD ["sleep", "infinity"]
|
||||||
|
EOF
|
||||||
|
|
||||||
|
docker build --pull -f ${TAG2}.Dockerfile -t $TAG2 .
|
||||||
|
|
||||||
|
cd ..
|
||||||
|
|
||||||
|
fi
|
||||||
|
|
||||||
|
RUNNING=`docker ps | grep gitrun`
|
||||||
|
if [ -z "$RUNNING" ]; then
|
||||||
|
BUILT=`docker ps -a | grep gitrun`
|
||||||
|
if [ -z "$BUILT" ]; then
|
||||||
|
docker run -v /var/run/docker.sock:/var/run/docker.sock -d --name gitrun $TAG
|
||||||
|
else
|
||||||
|
docker start gitrun
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
docker cp gitian-build.py gitrun:$WORKDIR/
|
||||||
|
docker exec -t gitrun ./gitian-build.py -d -b -D -n $OPT $GH_USER $VERSION
|
||||||
|
RC=$?
|
||||||
|
if [ $RC != 0 ]; then
|
||||||
|
exit $RC
|
||||||
|
fi
|
||||||
|
echo "\nBuild Results:\n"
|
||||||
|
docker exec gitrun sh -c "sha256sum out/$VERSION/*"
|
||||||
|
echo "\nIf these results look correct, type \"sign\" to sign them, otherwise ^C to stop now."
|
||||||
|
read check
|
||||||
|
if [ "$check" != "sign" ]; then
|
||||||
|
echo "Not signing, bye."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -d sigs ]; then
|
||||||
|
git clone https://github.com/monero-project/gitian.sigs.git sigs
|
||||||
|
cd sigs
|
||||||
|
git remote add $GH_USER git@github.com:$GH_USER/gitian.sigs.git
|
||||||
|
cd ..
|
||||||
|
fi
|
||||||
|
|
||||||
|
DIRS=`docker exec gitrun sh -c "echo sigs/$VERSION-*"`
|
||||||
|
for i in $DIRS; do
|
||||||
|
docker cp gitrun:$WORKDIR/$i sigs
|
||||||
|
gpg --detach-sign $i/$GH_USER/*.assert
|
||||||
|
done
|
||||||
|
|
||||||
|
cd sigs
|
||||||
|
git checkout -b $VERSION
|
||||||
|
git add $VERSION-*
|
||||||
|
git commit -S -m "Add $GH_USER $VERSION"
|
||||||
|
git push --set-upstream $GH_USER $VERSION
|
||||||
Loading…
Reference in New Issue