Commit Graph

202 Commits

Author SHA1 Message Date
realtyem e9cbddc8e7
Modernize configure_workers_and_start.py bootstrapping script for Dockerfile-workers. (#14294) 2022-11-09 12:02:15 +00:00
realtyem d85cba1aa0
Add all Stream Writer worker types to configure_workers_and_start.py (#14197)
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-11-08 13:14:00 +00:00
realtyem 69814eb282
Allow override for requesting specific worker types for Complement on command line. (#14324)
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.

* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.

* Update docs to reflect this new ability.

* Changelog

* Don't rely on soft wrapping to format long strings

Good idea dklimpel. Thanks for catching that.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>

* Small nits just noticed in docs.

* Fixup new line in docs.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
2022-11-08 12:34:09 +00:00
Patrick Cloke 59ca73006c
Enable testing MSC3874 in complement. (#14339) 2022-11-01 13:26:28 -04:00
Jonathan de Jong e440f9674a
Enable URL previews in complement homeserver config. (#14198) 2022-10-18 09:52:23 -04:00
realtyem 6c5082f3e0
Flush stdout/err in Dockerfile-workers before replacing the current process (#14195)
Also update `subprocess.check_output` to the slightly newer `subprocess.run`.

Signed-off-by: Jason Little <realtyem@gmail.com>
2022-10-18 11:56:20 +00:00
Erik Johnston 6fee2f49f3
Cache Rust build cache when building docker images (#14130) 2022-10-17 17:21:14 +00:00
realtyem c75836fe77
Strip whitespace from worker types in Dockerfile-workers (#14165) 2022-10-14 19:38:04 +00:00
Patrick Cloke bc2bd92b93 Merge remote-tracking branch 'origin/release-v1.69' into develop 2022-10-14 14:11:27 -04:00
realtyem c7446906bd
Set LD_PRELOAD to load jemalloc in Dockerfile-workers. (#14182) 2022-10-14 13:29:49 +01:00
Patrick Cloke c3e4edb4d6
Stabilize the threads API. (#14175)
Stabilize the threads API (MSC3856) by supporting (only) the v1
path for the endpoint.

This also marks the API as safe for workers since it is a read-only
API.
2022-10-14 07:16:50 -04:00
Erik Johnston 29ee4b6698
Fix docker build OOMing in CI for arm64 builds (#14173)
Co-authored-by: David Robertson <davidr@element.io>
2022-10-13 18:16:21 +00:00
Patrick Cloke c604d2c218
Mark /relations endpoint as usable on workers. (#14028)
Co-authored-by: Eric Eastwood <erice@element.io>
2022-10-12 06:46:13 -04:00
David Robertson 3f057e4c54
Use minimal Rust installation in docker images and CI (#14141) 2022-10-12 10:47:02 +01:00
Erik Johnston d94bcbced3
Fix pinning Rust deps in docker images (#14129) 2022-10-11 10:53:34 +00:00
David Robertson 0c853e0970
Fix broken links to README (#14093) 2022-10-06 22:49:51 +00:00
Richard van der Hoff 51436c8dd5
Complement test image: capture logs from nginx (#14063)
Have nginx send its logs to stderr/out, so that we can debug
https://github.com/matrix-org/synapse/issues/13334.
2022-10-05 17:37:34 +01:00
Mathieu Velten 42dd992bb7
Dockerfile for tests: align comments with current behavior (#13867)
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-09-27 15:55:43 +02:00
Mathieu Velten d5292b8017
Fix Docker build when Rust .so has been build locally first (#13811)
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 15:38:54 +00:00
Mathieu Velten 384dca53d6
complement: init postgres DB directly inside the target image (#13819)
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 17:12:45 +02:00
Erik Johnston c9b7e97355
Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Erik Johnston 18e4092801
Bump docker image to use stable poetry version (#13688) 2022-09-01 13:46:47 +01:00
reivilibre 2c5e2ae898
Document that the `DOCKER_BUILDKIT=1` flag is needed to build the docker image. (#13515) 2022-08-12 14:25:47 +01:00
Richard van der Hoff 166fafdf8d
synapse-workers docker: copy nginx and redis in from base images (#13447)
Part of my continuing quest to make the docker images build quicker: copy nginx and redis in from base docker images, rather than apt installing each time.
2022-08-04 12:59:27 +01:00
Jasper Spaans 503a95804e
Install cryptography build dependencies in requirements image. (#13372) 2022-08-03 11:16:32 +01:00
reivilibre 05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. (#13213)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-08-01 10:55:31 +00:00
Brendan Abolivier 10e4093839
Call out buildkit is required when building test docker images (#13338)
Co-authored-by: David Robertson <davidr@element.io>
2022-07-21 14:29:58 +02:00
David Robertson b977867358
Rate limit joins per-room (#13276) 2022-07-19 11:45:17 +00:00
David Robertson e9ce4d089b
Use and recommend poetry 1.1.14, up from 1.1.12 (#13285) 2022-07-15 16:18:47 +01:00
Richard van der Hoff 512486bbeb
Docker: copy postgres from base image (#13279)
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
2022-07-15 11:13:40 +01:00
reivilibre 65e675504f
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. (#13152) 2022-07-05 09:46:20 +00:00
Till fa10468eb4
[Complement] Allow device_name lookup over federation (#13167) 2022-07-04 12:34:50 +00:00
reivilibre c04e25789e
Enable Complement testing in the 'Twisted Trunk' CI runs. (#13079) 2022-07-01 15:42:49 +00:00
reivilibre 9667bad55d
Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. (#13127)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-30 11:58:12 +00:00
reivilibre 3c5549e74a
Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. (#13054)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-27 11:43:20 +01:00
reivilibre 90cadcd403
Add a Subject Alternative Name to the certificate generated for Complement tests. (#13071) 2022-06-16 12:43:21 +01:00
Richard van der Hoff 1e0044e8f9
Complement: use SQLite by default (#13075)
If no database is configured explicitly, use sqlite.

This means that you don't have to pass any variables into the image.
2022-06-16 12:12:26 +01:00
Erik Johnston c95b04bb0e
Change default `sync_response_cache_duration` (#13042) 2022-06-15 16:55:20 +00:00
reivilibre 212be2edc1
Use updated `update_user_directory_from_worker` options in Complement tests. (#13069) 2022-06-15 15:54:32 +01:00
reivilibre 538044ac01
Collapse Docker build commands in Complement CI runs to make the logs easier to read. (#13058) 2022-06-15 14:42:27 +00:00
reivilibre 67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. (#12881)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-08 09:57:05 +00:00
Nick Mills-Barrett 1f9013ce60
Add the `batch_send` endpoint to generic workers (#12868) 2022-05-25 09:51:07 +00:00
Erik Johnston f5b1c09909
Pin poetry.core in Docker images (#12853) 2022-05-24 11:35:08 +01:00
reivilibre 67aae05ece
Support registering Application Services when running with workers under Complement. (#12826)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-23 14:11:06 +01:00
reivilibre 438925c422
Fix Complement `TestCanRegisterAdmin` with workers, by adding Complement's shared registration secret. (#12819) 2022-05-23 12:20:30 +01:00
reivilibre 4fef76ca34
Remove Caddy from the Synapse workers image used in Complement. (#12818) 2022-05-23 10:29:24 +01:00
Till e8ae472d3b
Update configs used by Complement to allow more invites (#12731) 2022-05-13 16:45:47 +01:00
Richard van der Hoff 34e84fee68
Tweaks to workers-under-complement (#12637)
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
2022-05-09 22:41:06 +01:00
reivilibre c2d50e9f6c
Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452) 2022-05-06 11:43:53 +01:00
reivilibre d743b25c8f
Use supervisord to supervise Postgres and Caddy in the Complement image. (#12480)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-04-27 14:39:41 +01:00