Adapt Readme and script to monero gitian build signing

The sigs should be produced in a seperate step by default.
Remove windows and osx sig options that are not needed for monero.
This commit is contained in:
TheCharlatan 2018-10-11 02:05:00 +02:00
parent 9617fad070
commit 8f96c718bc
5 changed files with 43 additions and 77 deletions

View File

@ -3,8 +3,8 @@ Gitian building
*Setup instructions for a Gitian build of Monero using a VM or physical system.* *Setup instructions for a Gitian build of Monero using a VM or physical system.*
Gitian is the deterministic build process that is used to build the Bitcoin Gitian is the deterministic build process that is used to build the Monero CLI
Core executables. It provides a way to be reasonably sure that the executables. It provides a way to be reasonably sure that the
executables are really built from the git source. It also makes sure that executables are really built from the git source. It also makes sure that
the same, tested dependencies are used and statically built into the executable. the same, tested dependencies are used and statically built into the executable.
@ -22,8 +22,7 @@ Table of Contents
Please note that these instructions have been forked from bitcoin's gitian build Please note that these instructions have been forked from bitcoin's gitian build
instructions. Please also consult their documentation, when running into problems. instructions. Please also consult their documentation, when running into problems.
The signing is left as inherited from bitcoin at the moment, since building currently The signing is left as inherited from bitcoin at the moment.
still fails with libiconv.
- [Preparing the Gitian builder host](#preparing-the-gitian-builder-host) - [Preparing the Gitian builder host](#preparing-the-gitian-builder-host)
- [Getting and building the inputs](#getting-and-building-the-inputs) - [Getting and building the inputs](#getting-and-building-the-inputs)
@ -37,9 +36,10 @@ Preparing the Gitian builder host
The first step is to prepare the host environment that will be used to perform the Gitian builds. The first step is to prepare the host environment that will be used to perform the Gitian builds.
This guide explains how to set up the environment, and how to start the builds. This guide explains how to set up the environment, and how to start the builds.
Gitian builds are for now executed on Ubuntu 18.04 "Bionic Beaver". Please run Ubuntu in either a VM, or on your physical machine. Gitian builds are for now executed on Ubuntu 18.04 "Bionic Beaver". A solution is being worked on to run
it in docker in the future. Please run Ubuntu in either a VM, or on your physical machine.
You need to be logged in as the `gitianuser` in order to build gitian builds. If this user does not exist yet on your system, You need to be logged in as the `gitianuser` in order to build gitian builds. If this user does not exist yet on your system,
create him. create it.
Note that a version of `lxc-execute` higher or equal to 2.1.1 is required. Note that a version of `lxc-execute` higher or equal to 2.1.1 is required.
You can check the version with `lxc-execute --version`. You can check the version with `lxc-execute --version`.
@ -79,11 +79,8 @@ This setup is required to enable networking in the container.
Manual and Building Manual and Building
------------------- -------------------
The instructions below use the automated script [gitian-build.py](https://github.com/betcoin/bitcoin/blob/master/contrib/gitian-build.py) which only works in Ubuntu. For manual steps and instructions for fully offline signing, see [this guide](./gitian-building/gitian-building-manual.md). The instructions below use the automated script [gitian-build.py](https://github.com/betcoin/bitcoin/blob/master/contrib/gitian-build.py) which only works in Ubuntu.
It calls all available descriptors. Help for the build steps taken can be accessed with `./gitian-build.py --help`.
MacOS code signing
------------------
In order to sign builds for MacOS, you need to download the free SDK and extract a file. The steps are described [here](./gitian-building/gitian-building-mac-os-sdk.md). Alternatively, you can skip the OSX build by adding `--os=lw` below.
Initial Gitian Setup Initial Gitian Setup
-------------------- --------------------
@ -93,66 +90,57 @@ The `gitian-build.py` script will checkout different release tags, so it's best
cp monero/contrib/gitian/gitian-build.py . cp monero/contrib/gitian/gitian-build.py .
``` ```
You only need to do this once: Setup the required environment, you only need to do this once:
``` ```
./gitian-build.py --setup fluffypony 0.0.20 ./gitian-build.py --setup fluffypony 0.14.0
``` ```
Where `fluffypony` is your Github name and `0.0.20` is the most recent tag (without `v`). Where `fluffypony` is your Github name and `0.14.0` is the version tag you want to build (without `v`).
In order to sign gitian builds on your host machine, which has your PGP key, fork the gitian.sigs repository and clone it on your host machine: While gitian and this build script does provide a way for you to sign the build directly, it is recommended to sign in a seperate step.
This script is only there for convenience. Seperate steps for building can still be taken.
In order to sign gitian builds on your host machine, which has your PGP key,
fork the gitian.sigs repository and clone it on your host machine,
or pass the signed assert file back to your build machine.
``` ```
git clone git@github.com:bitcoin-core/gitian.sigs.git git clone git@github.com:monero-project/gitian.sigs.git
git remote add satoshi git@github.com:satoshi/gitian.sigs.git git remote add fluffypony git@github.com:fluffypony/gitian.sigs.git
``` ```
Build Binaries Build Binaries
----------------------------- -----------------------------
Windows and OSX have code signed binaries, but those won't be available until a few developers have gitian signed the non-codesigned binaries.
To build the most recent tag: To build the most recent tag:
`./gitian-build.py --detach-sign --no-commit -b fluffypony 0.0.20` `./gitian-build.py --detach-sign --no-commit -b fluffypony 0.14.0`
To speed up the build, use `-j 5 -m 5000` as the first arguments, where `5` is the number of CPU's you allocated to the VM plus one, and 5000 is a little bit less than then the MB's of RAM you allocated. If there is memory corruption on your machine, try to tweak these values. To speed up the build, use `-j 5 -m 5000` as the first arguments, where `5` is the number of CPU's you allocated to the VM plus one, and 5000 is a little bit less than then the MB's of RAM you allocated. If there is memory corruption on your machine, try to tweak these values.
If all went well, this produces a number of (uncommited) `.assert` files in the gitian.sigs repository. If all went well, this produces a number of (uncommited) `.assert` files in the gitian.sigs repository.
You need to copy these uncommited changes to your host machine, where you can sign them: If you do detached, offline signing, you need to copy these uncommited changes to your host machine, where you can sign them. For example:
``` ```
export NAME=satoshi export NAME=fluffypony
gpg --output $VERSION-linux/$NAME/bitcoin-linux-0.16-build.assert.sig --detach-sign 0.16.0rc1-linux/$NAME/bitcoin-linux-0.16-build.assert export VERSION=0.14
gpg --output $VERSION-osx-unsigned/$NAME/bitcoin-osx-0.16-build.assert.sig --detach-sign 0.16.0rc1-osx-unsigned/$NAME/bitcoin-osx-0.16-build.assert gpg --output $VERSION-linux/$NAME/monero-linux-$VERSION-build.assert.sig --detach-sign $VERSION-linux/$NAME/monero-linux-$VERSION-build.assert
gpg --output $VERSION-win-unsigned/$NAME/bitcoin-win-0.16-build.assert.sig --detach-sign 0.16.0rc1-win-unsigned/$NAME/bitcoin-win-0.16-build.assert gpg --output $VERSION-osx-unsigned/$NAME/monero-osx-$VERSION-build.assert.sig --detach-sign $VERSION-osx-unsigned/$NAME/monero-osx-$VERSION-build.assert
gpg --output $VERSION-win-unsigned/$NAME/monero-win-$VERSION-build.assert.sig --detach-sign $VERSION-win-unsigned/$NAME/monero-win-$VERSION-build.assert
``` ```
Make a PR (both the `.assert` and `.assert.sig` files) to the Make a pull request (both the `.assert` and `.assert.sig` files) to the
[bitcoin-core/gitian.sigs](https://github.com/bitcoin-core/gitian.sigs/) repository: [monero-project/gitian.sigs](https://github.com/monero-project/gitian.sigs/) repository:
``` ```
git checkout -b 0.0.20-not-codesigned git checkout -b 0.14.0
git commit -S -a -m "Add $NAME 0.0.20 non-code signed signatures" git commit -S -a -m "Add $NAME 0.14.0"
git push --set-upstream $NAME 0.0.20 git push --set-upstream $NAME 0.14.0
``` ```
You can also mail the files to Wladimir (laanwj@gmail.com) and he will commit them.
```bash ```bash
gpg --detach-sign ${VERSION}-linux/${SIGNER}/bitcoin-linux-*-build.assert gpg --detach-sign ${VERSION}-linux/${SIGNER}/monero-linux-*-build.assert
gpg --detach-sign ${VERSION}-win-unsigned/${SIGNER}/bitcoin-win-*-build.assert gpg --detach-sign ${VERSION}-win-unsigned/${SIGNER}/monero-win-*-build.assert
gpg --detach-sign ${VERSION}-osx-unsigned/${SIGNER}/bitcoin-osx-*-build.assert gpg --detach-sign ${VERSION}-osx-unsigned/${SIGNER}/monero-osx-*-build.assert
``` ```
You may have other .assert files as well (e.g. `signed` ones), in which case you should sign them too. You can see all of them by doing `ls ${VERSION}-*/${SIGNER}`.
This will create the `.sig` files that can be committed together with the `.assert` files to assert your
Gitian build.
`./gitian-build.py --detach-sign -s satoshi 0.16.0rc1 --nocommit`
Make another pull request for these.

View File

@ -65,13 +65,13 @@ def build():
if args.windows: if args.windows:
print('\nCompiling ' + args.version + ' Windows') print('\nCompiling ' + args.version + ' Windows')
subprocess.check_call(['bin/gbuild', '-j', args.jobs, '-m', args.memory, '--commit', 'monero='+args.commit, '--url', 'monero='+args.url, '../monero/contrib/gitian/gitian-win.yml']) subprocess.check_call(['bin/gbuild', '-j', args.jobs, '-m', args.memory, '--commit', 'monero='+args.commit, '--url', 'monero='+args.url, '../monero/contrib/gitian/gitian-win.yml'])
subprocess.check_call(['bin/gsign', '-p', args.sign_prog, '--signer', args.signer, '--release', args.version+'-win-unsigned', '--destination', '../gitian.sigs/', '../monero/contrib/gitian/gitian-win.yml']) subprocess.check_call(['bin/gsign', '-p', args.sign_prog, '--signer', args.signer, '--release', args.version+'-win', '--destination', '../gitian.sigs/', '../monero/contrib/gitian/gitian-win.yml'])
subprocess.check_call('mv build/out/monero*.zip ../monero-binaries/'+args.version, shell=True) subprocess.check_call('mv build/out/monero*.zip ../monero-binaries/'+args.version, shell=True)
if args.macos: if args.macos:
print('\nCompiling ' + args.version + ' MacOS') print('\nCompiling ' + args.version + ' MacOS')
subprocess.check_call(['bin/gbuild', '-j', args.jobs, '-m', args.memory, '--commit', 'monero='+args.commit, '--url', 'monero'+args.url, '../monero/contrib/gitian/gitian-osx.yml']) subprocess.check_call(['bin/gbuild', '-j', args.jobs, '-m', args.memory, '--commit', 'monero='+args.commit, '--url', 'monero'+args.url, '../monero/contrib/gitian/gitian-osx.yml'])
subprocess.check_call(['bin/gsign', '-p', args.sign_prog, '--signer', args.signer, '--release', args.version+'-osx-unsigned', '--destination', '../gitian.sigs/', '../monero/contrib/gitian/gitian-osx.yml']) subprocess.check_call(['bin/gsign', '-p', args.sign_prog, '--signer', args.signer, '--release', args.version+'-osx', '--destination', '../gitian.sigs/', '../monero/contrib/gitian/gitian-osx.yml'])
subprocess.check_call('mv build/out/monero*.tar.gz ../monero-binaries/'+args.version, shell=True) subprocess.check_call('mv build/out/monero*.tar.gz ../monero-binaries/'+args.version, shell=True)
os.chdir(workdir) os.chdir(workdir)
@ -80,24 +80,11 @@ def build():
print('\nCommitting '+args.version+' Unsigned Sigs\n') print('\nCommitting '+args.version+' Unsigned Sigs\n')
os.chdir('gitian.sigs') os.chdir('gitian.sigs')
subprocess.check_call(['git', 'add', args.version+'-linux/'+args.signer]) subprocess.check_call(['git', 'add', args.version+'-linux/'+args.signer])
subprocess.check_call(['git', 'add', args.version+'-win-unsigned/'+args.signer]) subprocess.check_call(['git', 'add', args.version+'-win/'+args.signer])
subprocess.check_call(['git', 'add', args.version+'-osx-unsigned/'+args.signer]) subprocess.check_call(['git', 'add', args.version+'-osx/'+args.signer])
subprocess.check_call(['git', 'commit', '-m', 'Add '+args.version+' unsigned sigs for '+args.signer]) subprocess.check_call(['git', 'commit', '-m', 'Add '+args.version+' unsigned sigs for '+args.signer])
os.chdir(workdir) os.chdir(workdir)
def sign():
global args, workdir
os.chdir('gitian-builder')
os.chdir(workdir)
if args.commit_files:
print('\nCommitting '+args.version+' Signed Sigs\n')
os.chdir('gitian.sigs')
subprocess.check_call(['git', 'add', args.version+'-win-signed/'+args.signer])
subprocess.check_call(['git', 'add', args.version+'-osx-signed/'+args.signer])
subprocess.check_call(['git', 'commit', '-a', '-m', 'Add '+args.version+' signed binary sigs for '+args.signer])
os.chdir(workdir)
def verify(): def verify():
global args, workdir global args, workdir
os.chdir('gitian-builder') os.chdir('gitian-builder')
@ -105,14 +92,9 @@ def verify():
print('\nVerifying v'+args.version+' Linux\n') print('\nVerifying v'+args.version+' Linux\n')
subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-linux', '../monero/contrib/gitian/gitian-linux.yml']) subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-linux', '../monero/contrib/gitian/gitian-linux.yml'])
print('\nVerifying v'+args.version+' Windows\n') print('\nVerifying v'+args.version+' Windows\n')
subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-win-unsigned', '../monero/contrib/gitian/gitian-win.yml']) subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-win', '../monero/contrib/gitian/gitian-win.yml'])
print('\nVerifying v'+args.version+' MacOS\n') print('\nVerifying v'+args.version+' MacOS\n')
subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-osx-unsigned', '../monero/contrib/gitian/gitian-osx.yml']) subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-osx', '../monero/contrib/gitian/gitian-osx.yml'])
print('\nVerifying v'+args.version+' Signed Windows\n')
subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-win-signed', '../monero/contrib/gitian/gitian-win-signer.yml'])
print('\nVerifying v'+args.version+' Signed MacOS\n')
subprocess.check_call(['bin/gverify', '-v', '-d', '../gitian.sigs/', '-r', args.version+'-osx-signed', '../monero/contrib/gitian/gitian-osx-signer.yml'])
os.chdir(workdir) os.chdir(workdir)
def main(): def main():
@ -124,7 +106,6 @@ def main():
parser.add_argument('-u', '--url', dest='url', default='https://github.com/monero-project/monero', help='Specify the URL of the repository. Default is %(default)s') parser.add_argument('-u', '--url', dest='url', default='https://github.com/monero-project/monero', help='Specify the URL of the repository. Default is %(default)s')
parser.add_argument('-v', '--verify', action='store_true', dest='verify', help='Verify the Gitian build') parser.add_argument('-v', '--verify', action='store_true', dest='verify', help='Verify the Gitian build')
parser.add_argument('-b', '--build', action='store_true', dest='build', help='Do a Gitian build') parser.add_argument('-b', '--build', action='store_true', dest='build', help='Do a Gitian build')
parser.add_argument('-s', '--sign', action='store_true', dest='sign', help='Make signed binaries for Windows and MacOS')
parser.add_argument('-B', '--buildsign', action='store_true', dest='buildsign', help='Build both signed and unsigned binaries') parser.add_argument('-B', '--buildsign', action='store_true', dest='buildsign', help='Build both signed and unsigned binaries')
parser.add_argument('-o', '--os', dest='os', default='lwm', help='Specify which Operating Systems the build is for. Default is %(default)s. l for Linux, w for Windows, m for MacOS') parser.add_argument('-o', '--os', dest='os', default='lwm', help='Specify which Operating Systems the build is for. Default is %(default)s. l for Linux, w for Windows, m for MacOS')
parser.add_argument('-j', '--jobs', dest='jobs', default='2', help='Number of processes to use. Default %(default)s') parser.add_argument('-j', '--jobs', dest='jobs', default='2', help='Number of processes to use. Default %(default)s')
@ -184,7 +165,7 @@ def main():
# Add leading 'v' for tags # Add leading 'v' for tags
if args.commit and args.pull: if args.commit and args.pull:
raise Exception('Cannot have both commit and pull') raise Exception('Cannot have both commit and pull')
args.commit = ('' if args.commit else 'v') + args.version args.commit = ('' if args.commit else) + args.version
if args.setup: if args.setup:
setup() setup()
@ -204,9 +185,6 @@ def main():
if args.build: if args.build:
build() build()
if args.sign:
sign()
if args.verify: if args.verify:
verify() verify()

View File

@ -1,5 +1,5 @@
--- ---
name: "monero-linux-0.18" name: "monero-linux-0.14"
enable_cache: true enable_cache: true
suites: suites:
- "bionic" - "bionic"

View File

@ -1,5 +1,5 @@
--- ---
name: "bitcoin-osx-0.18" name: "monero-osx-0.14"
enable_cache: true enable_cache: true
suites: suites:
- "bionic" - "bionic"

View File

@ -1,5 +1,5 @@
--- ---
name: "bitcoin-win-0.18" name: "monero-win-0.14"
enable_cache: true enable_cache: true
suites: suites:
- "bionic" - "bionic"