Commit Graph

233 Commits

Author SHA1 Message Date
Mathieu Velten 384dca53d6
complement: init postgres DB directly inside the target image (#13819)
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 17:12:45 +02:00
Erik Johnston c9b7e97355
Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Erik Johnston 18e4092801
Bump docker image to use stable poetry version (#13688) 2022-09-01 13:46:47 +01:00
reivilibre 2c5e2ae898
Document that the `DOCKER_BUILDKIT=1` flag is needed to build the docker image. (#13515) 2022-08-12 14:25:47 +01:00
Richard van der Hoff 166fafdf8d
synapse-workers docker: copy nginx and redis in from base images (#13447)
Part of my continuing quest to make the docker images build quicker: copy nginx and redis in from base docker images, rather than apt installing each time.
2022-08-04 12:59:27 +01:00
Jasper Spaans 503a95804e
Install cryptography build dependencies in requirements image. (#13372) 2022-08-03 11:16:32 +01:00
reivilibre 05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. (#13213)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-08-01 10:55:31 +00:00
Brendan Abolivier 10e4093839
Call out buildkit is required when building test docker images (#13338)
Co-authored-by: David Robertson <davidr@element.io>
2022-07-21 14:29:58 +02:00
David Robertson b977867358
Rate limit joins per-room (#13276) 2022-07-19 11:45:17 +00:00
David Robertson e9ce4d089b
Use and recommend poetry 1.1.14, up from 1.1.12 (#13285) 2022-07-15 16:18:47 +01:00
Richard van der Hoff 512486bbeb
Docker: copy postgres from base image (#13279)
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
2022-07-15 11:13:40 +01:00
reivilibre 65e675504f
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. (#13152) 2022-07-05 09:46:20 +00:00
Till fa10468eb4
[Complement] Allow device_name lookup over federation (#13167) 2022-07-04 12:34:50 +00:00
reivilibre c04e25789e
Enable Complement testing in the 'Twisted Trunk' CI runs. (#13079) 2022-07-01 15:42:49 +00:00
reivilibre 9667bad55d
Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. (#13127)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-30 11:58:12 +00:00
reivilibre 3c5549e74a
Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. (#13054)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-27 11:43:20 +01:00
reivilibre 90cadcd403
Add a Subject Alternative Name to the certificate generated for Complement tests. (#13071) 2022-06-16 12:43:21 +01:00
Richard van der Hoff 1e0044e8f9
Complement: use SQLite by default (#13075)
If no database is configured explicitly, use sqlite.

This means that you don't have to pass any variables into the image.
2022-06-16 12:12:26 +01:00
Erik Johnston c95b04bb0e
Change default `sync_response_cache_duration` (#13042) 2022-06-15 16:55:20 +00:00
reivilibre 212be2edc1
Use updated `update_user_directory_from_worker` options in Complement tests. (#13069) 2022-06-15 15:54:32 +01:00
reivilibre 538044ac01
Collapse Docker build commands in Complement CI runs to make the logs easier to read. (#13058) 2022-06-15 14:42:27 +00:00
reivilibre 67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. (#12881)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-08 09:57:05 +00:00
Nick Mills-Barrett 1f9013ce60
Add the `batch_send` endpoint to generic workers (#12868) 2022-05-25 09:51:07 +00:00
Erik Johnston f5b1c09909
Pin poetry.core in Docker images (#12853) 2022-05-24 11:35:08 +01:00
reivilibre 67aae05ece
Support registering Application Services when running with workers under Complement. (#12826)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-23 14:11:06 +01:00
reivilibre 438925c422
Fix Complement `TestCanRegisterAdmin` with workers, by adding Complement's shared registration secret. (#12819) 2022-05-23 12:20:30 +01:00
reivilibre 4fef76ca34
Remove Caddy from the Synapse workers image used in Complement. (#12818) 2022-05-23 10:29:24 +01:00
Till e8ae472d3b
Update configs used by Complement to allow more invites (#12731) 2022-05-13 16:45:47 +01:00
Richard van der Hoff 34e84fee68
Tweaks to workers-under-complement (#12637)
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
2022-05-09 22:41:06 +01:00
reivilibre c2d50e9f6c
Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452) 2022-05-06 11:43:53 +01:00
reivilibre d743b25c8f
Use supervisord to supervise Postgres and Caddy in the Complement image. (#12480)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-04-27 14:39:41 +01:00
Jason Robinson 706456de1f
Mark Dockerfile as requiring BuildKit (#12541)
Co-authored-by: David Robertson <davidr@element.io>
2022-04-26 15:31:52 +01:00
David Robertson 8bac3e0435
disallow-untyped-defs in `docker` and `stubs` directories (#12528) 2022-04-25 12:32:35 +00:00
Richard van der Hoff f5668f0b4a
Await un-partial-stating after a partial-state join (#12399)
When we join a room via the faster-joins mechanism, we end up with "partial
state" at some points on the event DAG. Many parts of the codebase need to
wait for the full state to load. So, we implement a mechanism to keep track of
which events have partial state, and wait for them to be fully-populated.
2022-04-21 07:42:03 +01:00
David Robertson e5a76ec00b
Dump setuptools; correct pyproject version number (#12478) 2022-04-20 17:33:20 +01:00
Richard van der Hoff aaaff98202
Dockerfile-workers: reduce the amount we install (#12464)
This is an attempt to reduce the rebuild time. In short, we reduce the amount
of stuff that the dockerfile installs, so as to give a faster startup.
2022-04-14 15:36:49 +01:00
Richard van der Hoff 960b4fb409
complement-synapse-workers: factor out separate entry point script (#12467)
... with a bit more verbosity.
2022-04-14 14:56:10 +01:00
Richard van der Hoff 8af8a9bce5
Dockerfile-workers: give the master its own log config (#12466)
When we run a worker-mode synapse under docker, everything gets logged to stdout. Currently, output from the workers is tacked with a worker name, for example:

```
2022-04-13 15:27:56,810 - worker:frontend_proxy1 - synapse.util.caches.lrucache - 154 - INFO - LruCache._expire_old_entries-0 - Dropped 0 items from caches
```

- note `worker:frontend_proxy1`. No such tag is applied to log lines from the master, which makes somewhat confusing reading.

To fix this, we generate a dedicated log config file for the master in the same way that we do for the workers, and use that.
2022-04-13 20:50:08 +01:00
David Robertson 5f8173dd80
Workaround pip bug installing latest treq and not-latest twisted from hashes (#12439)
The requirements file generated by `poetry export` isn't correctly processed by `pip install -r requirements.txt`. It contains twisted and treq, both pinned to 22.2.0.

When `pip` installs treq, it notices that `Twisted[tls]` is required. It then tries to acquire the latest twisted release, only to fail (because this hash isn't listed in the requirements file).From e.g. https://github.com/matrix-org/synapse/runs/5977154990?check_suite_focus=true

> ```
> #15 9.204 Collecting Twisted[tls]>=18.7.0
> #15 9.205 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
> #15 9.205     Twisted[tls]>=18.7.0 from 38622ff95be907db1987c4d92edd09/Twisted-22.4.0-py3-none-any.whl#sha256=f9f7a91f94932477a9fc3b169d57f54f96c6e74a23d78d9ce54039a7f48928a2 (from treq==22.2.0->-r /synapse/requirements.txt (line 724))
> #15 ERROR: executor failed running [/bin/sh -c pip install --prefix="/install" --no-warn-script-location -r /synapse/requirements.txt]: exit code: 1
> ```

The underlying pip issue is https://github.com/pypa/pip/issues/9644. A comment notes that one can avoid this behaviour with by `pip install`ing with the `--no-deps` flag. Let us do so.

(At first glance, the problem looks like https://github.com/python-poetry/poetry/issues/5311, but that was a bug in `poetry install`; this is `poetry export`, whose behaviour is fine AFAICS).
2022-04-12 10:16:01 +01:00
Richard van der Hoff 214f3b7d21
Enable certificate checking during complement tests (#12435) 2022-04-11 16:35:41 +01:00
Richard van der Hoff 3cdf5a1386
Fix up healthcheck generation for workers docker image (#12405)
This wasn't quite generating the right thing.
2022-04-11 13:38:58 +00:00
Richard van der Hoff 5f72ea1bde
Move complement setup stuff into the Synapse repo (#12404)
Fixes matrix-org/complement#330 (or it will, once we remove the old files).

It's not quite a lift-and-shift: I've also taken the opportunity to get rid of the custom CA that we used to use to sign the TLS certs, which has been superceded by the CA exposed by Complement.
2022-04-11 11:39:28 +01:00
Jorge Florian 78e4d96a4d
Add missing type definitions for scripts in docker folder (#12280)
Signed-off-by: Jorge Florian <jafn28@gmail.com>
2022-04-08 10:10:58 +00:00
David Robertson 3a7e97c7ad
Poetry: use locked environment in Docker images (#12385) 2022-04-07 11:43:31 +00:00
Richard van der Hoff 34a8370d7b
README-testing.md: fix minor error 2022-03-31 13:08:03 +01:00
David Robertson 15cdcf8f30
Remove `dockerfile-pgtests` (#12336) 2022-03-31 11:30:07 +01:00
David Robertson 4aeb00ca20
Move synctl into `synapse._scripts` and expose as an entrypoint (#12140) 2022-03-04 11:58:49 +00:00
David Robertson f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points (#12118)
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-03-02 13:00:16 +00:00
Andrew Morgan f26e390a40
Use Python 3.9 in Synapse dockerfiles by default (#12112) 2022-03-01 13:55:18 +00:00
Richard van der Hoff 5598556b77
Docker: remove `VOLUME` directive (#11997)
The driver for this is to stop Complement complaining about it, but as far as I can tell it was pointless and needed to go away anyway.

I'm a bit unclear about what exactly VOLUME does, but I think what it means is that, if you don't override it with an explicit -v argument, then docker run will create a temporary volume, and copy things into it. The temporary volume is then deleted when the container finishes.

That only sounds useful if your image has something to copy into it (otherwise you may as well just use the default root filesystem), and our image notably doesn't copy anything into /data.

So... this wasn't doing anything, except annoying Complement?
2022-02-15 13:59:15 +00:00
Shay 9006ee36d1
Drop support for and remove references to EOL Python 3.6 (#11683)
* remove reference in comments to python3.6

* upgrade tox python env in script

* bump python version in example for completeness

* upgrade python version requirement in setup doc

* upgrade necessary python version in __init__.py

* upgrade python version in setup.py

* newsfragment

* drops refs to bionic and replace with focal

* bump refs to postgres 9.6 to 10

* fix hanging ci

* try installing tzdata first

* revert change made in b979f336

* ignore new random mypy error while debugging other error

* fix lint error for temporary workaround

* revert change to install list

* try passing env var

* export debian frontend var?

* move line and add comment

* bump pillow dependency

* bump lxml depenency

* install libjpeg-dev for pillow

* bump automat version to one compatible with py3.8

* add libwebp for pillow

* bump twisted trunk python version

* change suffix of newsfragment

* remove redundant python 3.7 checks

* lint
2022-01-21 14:23:26 -08:00
Richard van der Hoff d41c4654db
Use buildkit's cache feature to speed up docker builds (#11691)
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
2022-01-12 10:37:57 +00:00
Callum Macdonald 7013e06e2f
Improve Docker docs for use with Postgres (#11640) 2022-01-05 10:50:28 +00:00
Shay 13c974ed35
Drop Bionic from Debian builds (#11633)
* update Trove classifiers to remove py36

* stop building bionic

* update dh-virtualenv

* newsfragment

* fix newsfragment

* update version refs

* another try at correct tag

* Update changelog
2022-01-03 11:17:16 -08:00
Michael Kaye e2c300e7e4
Create healthcheck script for synapse-workers container (#11429)
The intent is to iterate through all the worker ports and only
report healthy when all are healthy, starting with the main process.
2021-11-26 14:05:20 +00:00
Shay 7cebaf9644
Remove code invalidated by deprecated config flag 'trust_identity_servers_for_password_resets' (#11395)
* remove background update code related to deprecated config flag

* changelog entry

* update changelog

* Delete 11394.removal

Duplicate, wrong number

* add no-op background update and change newfragment so it will be consolidated with associated work

* remove unused code

* Remove code associated with deprecated flag from legacy docker dynamic config file

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-23 06:46:40 -08:00
Aaron R d993c3bb1e
Add support for `/_matrix/media/v3` APIs (#11371)
* Add support for `/_matrix/media/v3` APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Update `workers.md` to use v3 client and media APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-17 15:30:24 +00:00
Richard van der Hoff 0b99d4c8d2
Docker: avoid changing userid unnecessarily (#11209)
* Docker image: avoid changing user during `generate`

The intention was always that the config files get written as the initial user
(normally root) - only the data directory needs to be writable by Synapse. This
got changed in https://github.com/matrix-org/synapse/pull/5970, but that seems
to have been a mistake.

* Avoid changing user if no explicit UID is given

* changelog
2021-11-01 13:55:30 +00:00
Dan Callahan d7141e0b8b
Fix Shellcheck SC2006: Use $(...) notation
Use $(...) notation instead of legacy backticked `...`.

https://github.com/koalaman/shellcheck/wiki/SC2006

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:55 +01:00
Dan Callahan 13f084eb58
Fix Shellcheck SC2086: Quote to prevent splitting
Double quote to prevent globbing and word splitting.

https://github.com/koalaman/shellcheck/wiki/SC2086

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Dan Callahan 6c736fa472
Fix Shellcheck SC2154: variable possibly undefined
var is referenced but not assigned.

https://github.com/koalaman/shellcheck/wiki/SC2154

Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22 23:08:54 +01:00
Richard van der Hoff 09eff1b3db
fix relative link in docker readme (#11144)
relative links don't work when it's on dockerhub.
2021-10-21 13:50:43 +01:00
Erik Johnston a8bbf08576
Fix debian package builds. (#10931)
This was due to dh-virtualenv builds being broken due to Shpinx removing
deprecated APIs.
2021-09-28 12:13:51 +01:00
David Robertson ea01d4c2de
Update postgresql testing script (#10906)
- Use sytest:bionic. Sytest:latest is two years old (do we want
  CI to push out latest at all?) and comes with Python 3.5, which we
  explictly no longer support. The script now runs under PostgreSQL 10
  as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root

A few changes to make it speedier to re-run the tests:

- Create blank DB in the container, not the script, so we don't have to
  `initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
  fetch and install a bunch of packages from PyPI each time

Co-authored-by: reivilibre <olivier@librepush.net>
2021-09-24 14:27:09 +00:00
kegsay 01df612e1e
Fix frontend_proxy jinja script in docker workers (#10783) 2021-09-08 17:24:53 +01:00
reivilibre 5f7b1e1f27
Make `PeriodicallyFlushingMemoryHandler` the default logging handler. (#10518) 2021-08-17 13:13:11 +01:00
Richard van der Hoff 2afdb5c984
Fix deb build script to set prerelease flag correctly (#10500) 2021-08-01 10:47:36 +01:00
Richard van der Hoff 8ae0bdca75
Drop xenial-support hacks (#10429) 2021-07-21 21:25:28 +01:00
Richard van der Hoff c82eb02d64
Set section for prerelease debs (#10391)
This is part of fixing #6116: we want to put RC debs into a different place than release debs, so reprepro has to be able to tell them apart.
2021-07-14 14:41:40 +01:00
Dirk Klimpel 974261cd81
Fix broken links in INSTALL.md (#10331)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-07-08 15:46:13 +01:00
Brendan Abolivier 08c8469322
Remove support for ACME v1 (#10194)
Fixes #9778

ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
2021-06-17 18:56:48 +01:00
Chris Castle 543e423fce
Fix broken link to README at root of repo (#10132)
Signed-off-by: Chris Castle chris@crc.io
2021-06-07 16:31:39 +01:00
Patrick Cloke 7d90d6ce9b
Run complement with Synapse workers manually. (#10039)
Adds an option to complement.sh to run Synapse in worker
mode (instead of the default monolith mode).
2021-05-24 15:32:45 -04:00
Sergio Miguéns 22a8838f62
Fix docker image to not log at `/homeserver.log` (#10045)
Fixes #9970

Signed-off-by: Sergio Miguéns Iglesias lonyelon@lony.xyz
2021-05-24 20:23:54 +01:00
Christopher May-Townsend d5305000f1
Docker healthcheck timings - add startup delay and changed interval (#9913)
* Add healthcheck startup delay by 5secs and reduced interval check to 15s
to reduce waiting time for docker aware edge routers bringing an
instance online
2021-05-05 17:33:04 +02:00
Andrew Morgan 7e460ec2a5
Add a dockerfile for running a set of Synapse worker processes (#9162)
This PR adds a Dockerfile and some supporting files to the `docker/` directory. The Dockerfile's intention is to spin up a container with:

* A Synapse main process.
* Any desired worker processes, defined by a `SYNAPSE_WORKERS` environment variable supplied at runtime.
* A redis for worker communication.
* A nginx for routing traffic.
* A supervisord to start all worker processes and monitor them if any go down.

Note that **this is not currently intended to be used in production**. If you'd like to use Synapse workers with Docker, instead make use of the official image, with one worker per container. The purpose of this dockerfile is currently to allow testing Synapse in worker mode with the [Complement](https://github.com/matrix-org/complement/) test suite.

`configure_workers_and_start.py` is where most of the magic happens in this PR. It reads from environment variables (documented in the file) and creates all necessary config files for the processes. It is the entrypoint of the Dockerfile, and thus is run any time the docker container is spun up, recreating all config files in case you want to use a different set of workers. One can specify which workers they'd like to use by setting the `SYNAPSE_WORKERS` environment variable (as a comma-separated list of arbitrary worker names) or by setting it to `*` for all worker processes. We will be using the latter in CI.

Huge thanks to @MatMaul for helping get this all working 🎉 This PR is paired with its equivalent on the Complement side: https://github.com/matrix-org/complement/pull/62.

Note, for the purpose of testing this PR before it's merged: You'll need to (re)build the base Synapse docker image for everything to work (`matrixdotorg/synapse:latest`). Then build the worker-based docker image on top (`matrixdotorg/synapse:workers`).
2021-04-14 13:54:49 +01:00
Andrew Morgan f16c6cf59a
Add note to docker docs explaining platform support (#9801)
Context is in https://github.com/matrix-org/synapse/issues/9764#issuecomment-818615894.

I struggled to find a more official link for this. The problem occurs when using WSL1 instead of WSL2, which some Windows platforms (at least Server 2019) still don't have. Docker have updated their documentation to paint a much happier picture now given WSL2's support.

The last sentence here can probably be removed once WSL1 is no longer around... though that will likely not be for a very long time.
2021-04-14 12:06:19 +01:00
Johannes Wienke cb657eb2f8
Put opencontainers labels to the final image (#9765)
They don't make any sense on the intermediate builder image. The final
images needs them to be of use for anyone.

Signed-off-by: Johannes Wienke <languitar@semipol.de>
2021-04-08 13:49:14 +01:00
Richard van der Hoff f02663c4dd
Replace `room_invite_state_types` with `room_prejoin_state` (#9700)
`room_invite_state_types` was inconvenient as a configuration setting, because
anyone that ever set it would not receive any new types that were added to the
defaults. Here, we deprecate the old setting, and replace it with a couple of
new settings under `room_prejoin_state`.
2021-03-30 12:12:44 +01:00
Eric Eastwood 0a778c135f
Make pip install faster in Docker build for Complement testing (#9610)
Make pip install faster in Docker build for [Complement](https://github.com/matrix-org/complement) testing.

If files have changed in a `COPY` command, Docker will invalidate all of the layers below. So I changed the order of operations to install all dependencies before we `COPY synapse /synapse/synapse/`. This allows Docker to use our cached layer of dependencies even when we change the source of Synapse and speed up builds dramatically! `53.5s` -> `3.7s` builds 🤘

As an alternative, I did try using BuildKit caches but this still took 30 seconds overall on that step. 15 seconds to gather the dependencies from the cache and another 15 seconds to `Installing collected packages`.

Fix https://github.com/matrix-org/synapse/issues/9364
2021-03-26 18:42:58 +00:00
Erik Johnston 019010964d Merge branch 'master' into develop 2021-03-26 12:26:58 +00:00
Erik Johnston 12d6184713
Explicitly upgrade openssl in docker file and enforce new version of cryptography (#9697) 2021-03-26 12:00:25 +00:00
Quentin Gliech d4c4798a25
Use interpreter from $PATH instead of absolute paths in various scripts using /usr/bin/env (#9689)
On NixOS, `bash` isn't under `/bin/bash` but rather in some directory in `$PATH`. Locally, I've been patching those scripts to make them work.

`/usr/bin/env` seems to be the only [portable way](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my) to use binaries from the PATH as interpreters.

Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
2021-03-25 16:53:54 +00:00
Johannes Wienke 4612302399
Include opencontainers labels in Docker image (#9612)
Cf. https://github.com/opencontainers/image-spec/blob/master/annotations.md#pre-defined-annotation-keys

Signed-off-by: Johannes Wienke <languitar@semipol.de>
2021-03-22 15:31:00 +00:00
Mathieu Velten ccf1dc51d7
Install jemalloc in docker image (#8553)
Co-authored-by: Will Hunt <willh@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
2021-03-16 11:32:18 +00:00
Richard van der Hoff 1b2d6d55c5
Remove vestiges of uploads_path config (#9462)
`uploads_path` was a thing that was never used; most of it was removed in #6628
but a few vestiges remained.
2021-02-22 19:54:49 +00:00
Dan Callahan b8b172466f
Add rustc to Docker image build environment (#9405)
This is needed to build the cryptography library, since it does not
provide wheels for ARMv7.

Fixes #9403

Signed-off-by: Dan Callahan <danc@element.io>
2021-02-15 15:30:16 +00:00
Dan Callahan e19396d622
Fix Debian builds on Xenial (#9254)
Adds note about updating dh-virtualenv once we drop support for Xenial.

We can't update now, because it needs debhelper 12, while Xenial only
backports 10.

Signed-off-by: Dan Callahan <danc@element.io>
2021-01-29 14:56:04 +00:00
Andrew Morgan 94549771f7
Replace 'perspectives' config block with 'trusted_key_servers' in docker homeserver.yaml template (#9157) 2021-01-19 16:19:32 +00:00
Dan Callahan fa6deb298b
Fix failures in Debian packaging (#9079)
Debian package builds were failing for two reasons:

 1. Python versions prior to 3.7 throw exceptions when attempting to print
    Unicode characters under a "C" locale. (#9076)

 2. We depended on `dh-systemd` which no longer exists in Debian Bullseye, but
    is necessary in Ubuntu Xenial. (#9073)

Setting `LANG="C.UTF-8"` in the build environment fixes the first issue.
See also: https://bugs.python.org/issue19846

The second issue is a bit trickier. The dh-systemd package was merged into
debhelper version 9.20160709 and a transitional package left in its wake.

The transitional dh-systemd package was removed in Debian Bullseye.

However, Ubuntu Xenial ships an older debhelper, and still needs dh-systemd.

Thus, builds were failing on Bullseye since we depended on a package which had
ceased existing, but we couldn't remove it from the debian/control file and our
build scripts because we still needed it for Ubuntu Xenial.

We can fix the debian/control issue by listing dh-systemd as an alternative to
the newer versions of debhelper. Since dh-systemd declares that it depends on
debhelper, Ubuntu Xenial will select its older dh-systemd which will in turn
pull in its older debhelper, resulting in no change from the status quo. All
other supported releases will satisfy the debhelper dependency constraint and
skip the dh-systemd alternative.

Build scripts were fixed by unconditionally attempting to install dh-systemd on
all releases and suppressing failures.

Once we drop support for Ubuntu Xenial, we can revert most of this commit and
rely on the version constraint on debhelper in debian/control.

Fixes #9076
Fixes #9073

Signed-off-by: Dan Callahan <danc@element.io>
2021-01-12 14:15:04 +00:00
Erik Johnston 57068eae75 Add 'xmlsec1' to dependency list 2020-12-09 13:48:16 +00:00
Jordan Bancino 295c209cdd
Remove version pin prometheus_client dependency (#8875)
This removes the version pin of the `prometheus_client` dependency, in direct response to #8831. If merged, this will close #8831 

As far as I can tell, no other changes are needed, but as I'm no synapse expert, I'm relying heavily on CI and maintainer reviews for this. My very primitive test of synapse with prometheus_client v0.9.0 on my home server didn't bring up any issues, so we'll see what happens.

Signed-off-by: Jordan Bancino
2020-12-04 13:01:06 +00:00
Andrew Morgan c087f68053
Cap the version of prometheus_client to <v0.9.0 in the dockerfile (#8767)
Short-term fix for https://github.com/matrix-org/synapse/issues/8766.
2020-11-17 16:01:33 +00:00
Dan Callahan ca39e67f3d
Use Python 3.8 in Docker images by default (#8698)
This bumps us closer to current Python without going all the way to 3.9.

Fixes #8674

Signed-off-by: Dan Callahan <danc@element.io>
2020-11-02 16:33:06 +00:00
Samuel Philipp 6905f5751a
Docker: support passing additional commandline args to synapse (#8390) 2020-10-11 20:51:11 +01:00
Mateusz Przybyłowicz ca2db5dd0c
Increase default max_upload_size from 10M to 50M (#8502)
Signed-off-by: Mateusz Przybyłowicz <uamfhq@gmail.com>
2020-10-09 16:58:23 +01:00
Christopher May-Townsend ed18f32e1b
Add required Debian dependencies to allow docker builds on the arm platform (#8144)
Signed-off-by: Christopher May-Townsend <chris@maytownsend.co.uk>
2020-08-26 15:03:20 +01:00
Christopher May-Townsend 64e8a4697a
Add healthcheck for default localhost 8008 port on /health endpoint. (#8147) 2020-08-24 18:15:18 +01:00
Erik Johnston 0304ad0c3d Move setting of Filter into code.
We do this to prevent foot guns. The default config uses a MemoryFilter,
but users are free to change to logging to files directly. If they do
then they have to ensure to set the `filters: [context]` on the right
handler, otherwise records get written with the wrong context.

Instead we move the logic to happen when we generate a record, which is
when we *log* rather than *handle*.

(It's possible to add filters to loggers in the config, however they
don't apply to descendant loggers and so they have to be manually set on
*every* logger used in the code base)
2020-08-11 18:10:46 +01:00
Michael Kaye 1ec2961b3b
Add help for creating a user via docker (#7885) 2020-07-17 13:25:48 -04:00