da040e167d
Tweak `instantiate_worker_template`, both in name, description and variable names
2023-12-06 15:18:47 +00:00
3318722753
Update comment on `merged`
2023-12-06 15:18:47 +00:00
a0d1348309
Fix comment and mutation bug on merge_worker_template_configs
2023-12-06 15:18:47 +00:00
f647745492
Docstring on WorkerTemplate
2023-12-06 15:14:41 +00:00
321d3590fe
Add a --generate-only option
2023-11-17 12:03:56 +00:00
fbafde81a1
Promote mark_filepath to constant
2023-11-17 12:03:56 +00:00
3bb21a9c26
Use `merge_into` when adding workers to the shared config
2023-11-17 12:03:56 +00:00
f49dbc7ba7
Add `sharding_allowed` to the WorkerTemplate rather than having a separate function for that
2023-11-17 12:03:56 +00:00
7d8824e2dc
Rename function to add_worker_to_instance_map given reduction of scope
2023-11-17 12:03:56 +00:00
f38297b619
Remove special logic for adding stream_writers: just make it part of the extra config template
2023-11-17 12:03:56 +00:00
8b7463957f
Add `merge_into`
2023-11-17 12:03:56 +00:00
26073fa778
Tweak comments
2023-11-17 12:03:56 +00:00
94a85b36f7
Convert listener_resources and endpoint_patterns to Set[str]
2023-11-17 12:03:56 +00:00
67d4fc8b99
Collapse WORKERS_CONFIG by removing entries with defaults
2023-11-17 12:03:56 +00:00
ba3b6a4dfd
Use a lambda for the worker name rather than search and replace later
2023-11-17 12:03:56 +00:00
a22eb7dc15
Convert worker templates into dataclass
2023-11-17 12:03:56 +00:00
b39a50a43b
Remove obsolete `"app"` from worker templates
2023-11-17 12:03:56 +00:00
8940d1b28e
Add `/notifications` endpoint to workers ( #16265 )
2023-09-07 09:26:07 +00:00
ad3f43be9a
Run pyupgrade for python 3.7 & 3.8. ( #16110 )
2023-08-15 08:11:20 -04:00
224ef0b669
Unix Sockets for HTTP Replication ( #15708 )
...
Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353 )), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at.
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
2023-07-11 13:08:06 -05:00
0f02f0b4da
Remove experimental MSC2716 implementation to incrementally import history into existing rooms ( #15748 )
...
Context for why we're removing the implementation:
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734
Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737
Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
30a5076da8
Log when events are (unexpectedly) filtered out of responses in tests ( #14213 )
...
See https://github.com/matrix-org/synapse/pull/14095#discussion_r990335492
This is useful because when see that a relevant event is an `outlier` or `soft-failed`, then that's a good unexpected indicator explaining why it's not showing up. `filter_events_for_client` is used in `/sync`, `/messages`, `/context` which are all common end-to-end assertion touch points (also notifications, relations).
2023-06-01 21:27:18 -05:00
e4f545c452
Remove `worker_replication_*` settings ( #15491 )
...
* Add master to the instance_map as part of Complement, have ReplicationEndpoint look at instance_map for master.
* Fix typo in drive by.
* Remove unnecessary worker_replication_* bits from unit tests and add master to instance_map(hopefully in the right place)
* Several updates:
1. Switch from master to main for naming the main process in the instance_map. Add useful constants for easier adjustment of names in the future.
2. Add backwards compatibility for worker_replication_* to allow time to transition to new style. Make sure to prioritize declaring main directly on the instance_map.
3. Clean up old comments/commented out code.
4. Adjust unit tests to match with new code.
5. Adjust Complement setup infrastructure to only add main to the instance_map if workers are used and remove now unused options from the worker.yaml template.
* Initial Docs upload
* Changelog
* Missed some commented out code that can go now
* Remove TODO comment that no longer holds true.
* Fix links in docs
* More docs
* Remove debug logging
* Apply suggestions from code review
Co-authored-by: reivilibre <olivier@librepush.net>
* Apply suggestions from code review
Co-authored-by: reivilibre <olivier@librepush.net>
* Update version to latest, include completeish before/after examples in upgrade notes.
* Fix up and docs too
---------
Co-authored-by: reivilibre <olivier@librepush.net>
2023-05-11 11:30:56 +01:00
e4a25d022c
Load `/capabilities` endpoint on workers ( #15436 )
2023-04-14 12:26:07 -04:00
4af0aec54d
Load `/directory/room/{roomAlias}` endpoint on workers ( #15333 )
...
* Enable `directory`
* move to worker store
* newsfile
* disable `ClientDirectoryListServer` and `ClientAppserviceDirectoryListServer` for workers
2023-04-14 10:24:06 +01:00
4fc85e5a92
Load `/password_policy` endpoint on workers. ( #15331 )
2023-03-27 07:37:17 -04:00
3d70cc393f
Load `/register/available` endpoint on workers ( #15268 )
2023-03-17 09:50:31 -04:00
66fc166b96
Make `configure_workers_and_start` script used in Complement tests compatible with older versions of Python. ( #15275 )
2023-03-17 09:02:30 -04:00
003a25ae5c
Additional functionality for declaring worker types in Complement ( #14921 )
...
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-03-14 16:29:33 +00:00
ecbe0ddbe7
Add support for knocking to workers. ( #15133 )
2023-03-02 12:59:53 -05:00
c4f4dc35cd
Dockerfile-workers: spell out when config isn't generated ( #15186 )
...
* Complement: Spell out when config isn't generated
* Changelog
2023-03-02 15:55:26 +00:00
682d31c702
Allow use of the `/filter` Client-Server APIs on workers. ( #15134 )
2023-02-28 16:37:19 +00:00
8f10c8b054
Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location ( #14471 )
...
Fix https://github.com/matrix-org/synapse/issues/14390
- Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
- Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`
Complement test changes: https://github.com/matrix-org/complement/pull/559
2022-11-28 15:54:18 -06:00
c15e9a0edb
Remove need for `worker_main_http_uri` setting to use /keys/upload. ( #14400 )
2022-11-16 22:16:25 +00:00
e9cbddc8e7
Modernize configure_workers_and_start.py bootstrapping script for Dockerfile-workers. ( #14294 )
2022-11-09 12:02:15 +00:00
d85cba1aa0
Add all Stream Writer worker types to configure_workers_and_start.py ( #14197 )
...
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-11-08 13:14:00 +00:00
6c5082f3e0
Flush stdout/err in Dockerfile-workers before replacing the current process ( #14195 )
...
Also update `subprocess.check_output` to the slightly newer `subprocess.run`.
Signed-off-by: Jason Little <realtyem@gmail.com>
2022-10-18 11:56:20 +00:00
c75836fe77
Strip whitespace from worker types in Dockerfile-workers ( #14165 )
2022-10-14 19:38:04 +00:00
c7446906bd
Set LD_PRELOAD to load jemalloc in Dockerfile-workers. ( #14182 )
2022-10-14 13:29:49 +01:00
c3e4edb4d6
Stabilize the threads API. ( #14175 )
...
Stabilize the threads API (MSC3856) by supporting (only) the v1
path for the endpoint.
This also marks the API as safe for workers since it is a read-only
API.
2022-10-14 07:16:50 -04:00
c604d2c218
Mark /relations endpoint as usable on workers. ( #14028 )
...
Co-authored-by: Eric Eastwood <erice@element.io>
2022-10-12 06:46:13 -04:00
65e675504f
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. ( #13152 )
2022-07-05 09:46:20 +00:00
9667bad55d
Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. ( #13127 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-30 11:58:12 +00:00
3c5549e74a
Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. ( #13054 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-27 11:43:20 +01:00
212be2edc1
Use updated `update_user_directory_from_worker` options in Complement tests. ( #13069 )
2022-06-15 15:54:32 +01:00
67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. ( #12881 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-08 09:57:05 +00:00
1f9013ce60
Add the `batch_send` endpoint to generic workers ( #12868 )
2022-05-25 09:51:07 +00:00
67aae05ece
Support registering Application Services when running with workers under Complement. ( #12826 )
...
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-23 14:11:06 +01:00
4fef76ca34
Remove Caddy from the Synapse workers image used in Complement. ( #12818 )
2022-05-23 10:29:24 +01:00
c2d50e9f6c
Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ( #12452 )
2022-05-06 11:43:53 +01:00
Olivier Wilkinson (reivilibre)