Commit Graph

1273 Commits

Author SHA1 Message Date
Patrick Cloke 2a51f3ec36
Implement MSC3952: Intentional mentions (#14823)
MSC3952 defines push rules which searches for mentions in a list of
Matrix IDs in the event body, instead of searching the entire event
body for display name / local part.

This is implemented behind an experimental configuration flag and
does not yet implement the backwards compatibility pieces of the MSC.
2023-01-27 10:16:21 -05:00
David Robertson 4607be0b7b
Request partial joins by default (#14905)
* Request partial joins by default

This is a little sloppy, but we are trying to gain confidence in faster
joins in the upcoming RC.

Admins can still opt out by adding the following to their Synapse
config:

```yaml
experimental:
    faster_joins: false
```

We may revert this change before the release proper, depending on how
testing in the wild goes.

* Changelog

* Try to fix the backfill test failures

* Upgrade notes

* Postgres compat?
2023-01-24 15:28:20 +00:00
Andrew Morgan a7b54ca8d8
Implement MSC3930: polls push rules (#14787) 2023-01-19 12:47:10 +00:00
Catalan Lover e8f2bf5c40
Change default room version to 10. Implements MSC3904 (#14111)
* Change Documentation to have v10 as default room version

* Change Default Room version to 10

* Add changelog entry for default room version swap

* Add changelog entry for v10 default room version in docs

* Clarify doc changelog entry

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Improve Documentation changes.

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update Changelog entry to have correct format

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update Spec Version to 1.5

* Only need 1 changelog.

* Fix test.

* Update "Changed in" line

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2023-01-18 18:59:48 +00:00
David Robertson 5b3af1c7d0
Stabilise serving partial join responses (#14839)
Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true.

Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
2023-01-17 12:44:15 +00:00
Andrew Morgan 54cd90ea60
Implement MSC3890: Remotely silence local notifications (#14775) 2023-01-13 19:32:10 +00:00
Erik Johnston b50c008453
Re-enable some linting (#14821)
* Re-enable some linting

* Newsfile

* Remove comment
2023-01-12 10:52:07 +00:00
Richard van der Hoff 06ab64f201
Implement MSC3925: changes to bundling of edits (#14811)
Two parts to this:

 * Bundle the whole of the replacement with any edited events. This is backwards-compatible so I haven't put it behind a flag.
 * Optionally, inhibit server-side replacement of edited events. This has scope to break things, so it is currently disabled by default.
2023-01-10 16:31:28 +00:00
Patrick Cloke 630d0aeaf6
Support RFC7636 PKCE in the OAuth 2.0 flow. (#14750)
PKCE can protect against certain attacks and is enabled by default. Support
can be controlled manually by setting the pkce_method of each oidc_providers
entry to 'auto' (default), 'always', or 'never'.

This is required by Twitter OAuth 2.0 support.
2023-01-04 14:58:08 -05:00
Andrew Morgan c4456114e1
Add experimental support for MSC3391: deleting account data (#14714) 2023-01-01 03:40:46 +00:00
Patrick Cloke 7010a3d015
Switch to ruff instead of flake8. (#14633)
ruff is a flake8-compatible Python linter written in Rust.
It supports the flake8 plugins that we use and is significantly
faster in testing.
2022-12-21 13:05:21 -05:00
Patrick Cloke 3aeca2588b
Add missing type hints to tests.config. (#14681) 2022-12-16 08:53:28 -05:00
David Robertson e2a1adbf5d
Allow selecting "prejoin" events by state keys (#14642)
* Declare new config

* Parse new config

* Read new config

* Don't use trial/our TestCase where it's not needed

Before:

```
$ time trial tests/events/test_utils.py > /dev/null

real	0m2.277s
user	0m2.186s
sys	0m0.083s
```

After:
```
$ time trial tests/events/test_utils.py > /dev/null

real	0m0.566s
user	0m0.508s
sys	0m0.056s
```

* Helper to upsert to event fields

without exceeding size limits.

* Use helper when adding invite/knock state

Now that we allow admins to include events in prejoin room state with
arbitrary state keys, be a good Matrix citizen and ensure they don't
accidentally create an oversized event.

* Changelog

* Move StateFilter tests

should have done this in #14668

* Add extra methods to StateFilter

* Use StateFilter

* Ensure test file enforces typed defs; alphabetise

* Workaround surprising get_current_state_ids

* Whoops, fix mypy
2022-12-13 00:54:46 +00:00
Will Hunt 71f3e53ad0
Add `push.enabled` option to disable push notification calculation (#14551)
* Add initial option

* changelog

* Some more linting
2022-12-01 13:46:24 +00:00
Travis Ralston dd51828120
Create MSC1767 (extensible events) room version; Implement MSC3932 (#14521)
* Add MSC1767's dedicated room version, based on v10

* Only enable MSC1767 room version if the config flag is on

Using a similar technique to knocking:
https://github.com/matrix-org/synapse/pull/6739/files#diff-3af529eedb0e00279bafb7369370c9654b37792af8eafa0925400e9281d57f0a

* Support MSC3932: Extensible events room version feature flag

* Changelog entry
2022-11-28 17:22:34 -07:00
Travis Ralston 3da6450327
Initial support for MSC3931: Room version push rule feature flags (#14520)
* Add support for MSC3931: Room Version Supports push rule condition

* Create experimental flag for future work, and use it to gate MSC3931

* Changelog entry
2022-11-28 16:29:53 -07:00
Eric Eastwood 8f10c8b054
Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location (#14471)
Fix https://github.com/matrix-org/synapse/issues/14390

 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
 - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`

Complement test changes: https://github.com/matrix-org/complement/pull/559
2022-11-28 15:54:18 -06:00
reivilibre 9af2be192a
Remove legacy Prometheus metrics names. They were deprecated in Synapse v1.69.0 and disabled by default in Synapse v1.71.0. (#14538) 2022-11-24 09:09:17 +00:00
realtyem df390a8e67
Refactor `federation_sender` and `pusher` configuration loading. (#14496)
To avoid duplicating the same logic for handling legacy configuration
settings.

This should help in applying similar logic to other worker types.
2022-11-22 21:33:58 +00:00
Sean Quah 78e23eea05
Reduce default third party invite rate limit to 216 invites per day (#14487)
The previous default was the same as the `rc_message` rate limit, which
defaults to 17,280 per day.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-18 18:10:01 +00:00
David Robertson 01a0527892
Fix version that `worker_main_http_uri` is redundant from (#14476)
* Fix version that `worker_main_http_uri` is redundant from

* Changelog
2022-11-17 16:11:08 +00:00
realtyem c15e9a0edb
Remove need for `worker_main_http_uri` setting to use /keys/upload. (#14400) 2022-11-16 22:16:25 +00:00
Patrick Cloke d8cc86eff4
Remove redundant types from comments. (#14412)
Remove type hints from comments which have been added
as Python type hints. This helps avoid drift between comments
and reality, as well as removing redundant information.

Also adds some missing type hints which were simple to fill in.
2022-11-16 15:25:24 +00:00
Tuomas Ojamies b5ab2c428a
Support using SSL on worker endpoints. (#14128)
* Fix missing SSL support in worker endpoints.

* Add changelog

* SSL for Replication endpoint

* Remove unit test change

* Refactor listener creation to reduce duplicated code

* Fix the logger message

* Update synapse/app/_base.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Update synapse/app/_base.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Update synapse/app/_base.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Add config documentation for new TLS option

Co-authored-by: Tuomas Ojamies <tojamies@palantir.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-11-15 12:55:00 +00:00
Sean Quah e5d18956b9 Synapse 1.71.0rc2 (2022-11-04)
==============================
 
 Please note that, as announced in the release notes for Synapse 1.69.0, legacy Prometheus metric names are now disabled by default.
 They will be removed altogether in Synapse 1.73.0.
 If not already done, server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names.
 See the [upgrade notes](https://matrix-org.github.io/synapse/v1.71/upgrade.html#upgrading-to-v1710) for more details.
 
 Improved Documentation
 ----------------------
 
 - Document the changes to monthly active user metrics due to deprecation of legacy Prometheus metric names. ([\#14358](https://github.com/matrix-org/synapse/issues/14358), [\#14360](https://github.com/matrix-org/synapse/issues/14360))
 
 Deprecations and Removals
 -------------------------
 
 - Disable legacy Prometheus metric names by default. They can still be re-enabled for now, but they will be removed altogether in Synapse 1.73.0. ([\#14353](https://github.com/matrix-org/synapse/issues/14353))
 
 Internal Changes
 ----------------
 
 - Run unit tests against Python 3.11. ([\#13812](https://github.com/matrix-org/synapse/issues/13812))
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmNlD5YACgkQ+84Kzgcy
 GG8T0Qv+PkTt9+lUbDAsqWnqWtSYCit4DEZU/g0G+/xUVdZypo+pIHYzzEHEEPoB
 Siinh5sLqUagoOn0i+YU+D6WZw+20F5+qdQi6QSkpPDorReYBl+4MvyRNqjn3lAE
 a/bfyHY3+88SfVMza+GRQS7pU06CCMD5aqUZABnYwV5fpKWdF2hOESSuzqEElQDR
 7zaZuc7I/8d/VNz79FmyDfGzYnvWS+tr0BcNIyye6OIzZ2AS+q8loBzQiJ5rd1kY
 ZiGeAHfeu/r05X25kzD1E1qr04ycM7mFjDgBa/aSdf5kkimNNUISo424sawoCVOU
 oDm+QVefH4zKK9b+DVAJ7eaHIaQsRjJir+h/umXrX09I/x/eMB7Zll1wy1T+KzUS
 Zf4GCVCI1kKY3FzcPWh2Jf1j899JtbkrKasA+UYf3+PEeFxQ4LyjYlVHC+J1sCXL
 NGn5YDZkvhfpTOR300eDisUsxKIBFPxL6FjzW5whOnBddKC+VVnlSQkAbn6yHQGE
 LU1xKhta
 =m6ze
 -----END PGP SIGNATURE-----

Merge tag 'v1.71.0rc2' into develop

Synapse 1.71.0rc2 (2022-11-04)
==============================

Please note that, as announced in the release notes for Synapse 1.69.0, legacy Prometheus metric names are now disabled by default.
They will be removed altogether in Synapse 1.73.0.
If not already done, server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names.
See the [upgrade notes](https://matrix-org.github.io/synapse/v1.71/upgrade.html#upgrading-to-v1710) for more details.

Improved Documentation
----------------------

- Document the changes to monthly active user metrics due to deprecation of legacy Prometheus metric names. ([\#14358](https://github.com/matrix-org/synapse/issues/14358), [\#14360](https://github.com/matrix-org/synapse/issues/14360))

Deprecations and Removals
-------------------------

- Disable legacy Prometheus metric names by default. They can still be re-enabled for now, but they will be removed altogether in Synapse 1.73.0. ([\#14353](https://github.com/matrix-org/synapse/issues/14353))

Internal Changes
----------------

- Run unit tests against Python 3.11. ([\#13812](https://github.com/matrix-org/synapse/issues/13812))
2022-11-04 15:22:06 +00:00
Brendan Abolivier 86c5a710d8
Implement MSC3912: Relation-based redactions (#14260)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-11-03 16:21:31 +00:00
reivilibre 6546308c1e
Disable legacy Prometheus metric names by default. They can still be re-enabled for now, but they will be removed altogether in Synapse 1.73.0. (#14353) 2022-11-02 17:33:45 +00:00
Quentin Gliech cc3a52b33d
Support OIDC backchannel logouts (#11414)
If configured an OIDC IdP can log a user's session out of
Synapse when they log out of the identity provider.

The IdP sends a request directly to Synapse (and must be
configured with an endpoint) when a user logs out.
2022-10-31 13:07:30 -04:00
Dirk Klimpel 1357ae869f
Add workers settings to configuration manual (#14086)
* Add workers settings to configuration manual
* Update `pusher_instances`
* update url to python logger
* update headlines
* update links after headline change
* remove link from `daemon process`

There is no docs in Synapse for this

* extend example for `federation_sender_instances` and `pusher_instances`
* more infos about stream writers
* add link to DAG
* update `pusher_instances`
* update `worker_listeners`
* update `stream_writers`
* Update `worker_name`

Co-authored-by: David Robertson <davidr@element.io>
2022-10-27 14:39:47 +01:00
DeepBlueV7.X 2d0ba3f89a
Implementation for MSC3664: Pushrules for relations (#11804) 2022-10-25 14:38:01 +01:00
Hugh Nimmo-Smith 4eaf3eb840
Implementation of HTTP 307 response for MSC3886 POST endpoint (#14018)
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: Andrew Morgan <andrewm@element.io>
2022-10-18 15:52:25 +00:00
David Robertson 8e50299d8b
Fix `track_memory_usage` on poetry-core 1.3.x installations (#14221)
* Fix `track_memory_usage` on poetry-core 1.3.x installations

The same kind of problem as discussed in #14085:

1. we defined an extra with an underscore
2. we look it up at runtime with an underscore
3. but poetry-core 1.3.x. installs it with a dash, causing (2) to fail.

Fix by using a dash everywhere.

* Changelog
2022-10-18 13:59:04 +01:00
Patrick Cloke 4283bd1cf9
Support filtering the /messages API by relation type (MSC3874). (#14148)
Gated behind an experimental configuration flag.
2022-10-17 11:32:11 -04:00
Patrick Cloke bc2bd92b93 Merge remote-tracking branch 'origin/release-v1.69' into develop 2022-10-14 14:11:27 -04:00
Patrick Cloke c3e4edb4d6
Stabilize the threads API. (#14175)
Stabilize the threads API (MSC3856) by supporting (only) the v1
path for the endpoint.

This also marks the API as safe for workers since it is a read-only
API.
2022-10-14 07:16:50 -04:00
Patrick Cloke 3bbe532abb
Add an API for listing threads in a room. (#13394)
Implement the /threads endpoint from MSC3856.

This is currently unstable and behind an experimental configuration
flag.

It includes a background update to backfill data, results from
the /threads endpoint will be partial until that finishes.
2022-10-13 08:02:11 -04:00
Patrick Cloke 09be8ab5f9
Remove the experimental implementation of MSC3772. (#14094)
MSC3772 has been abandoned.
2022-10-12 06:26:39 -04:00
Patrick Cloke 6136768e76
Remove the groups config code. (#14142)
This has been unused for a long time, but missed removal in #11584.
2022-10-11 13:14:57 -04:00
Patrick Cloke e03d7c5fd0
Remove support for the unstable dir flag on relations. (#14106)
From MSC3715, this was unused by clients (and there was no
way for clients to know it was supported).

Matrix 1.4 defines the stable field.
2022-10-07 12:38:46 -04:00
Quentin Gliech dc37b68a25
Parse SYNAPSE_ASYNC_IO_REACTOR env variable & log the reactor on startup (#14092) 2022-10-07 15:19:59 +01:00
David Robertson f1673866ed
Unpin build-system requirements, but impose an upper-bound (#14085)
* Revert to prior build-system requirements

This reverts #14080.

* Use normalised extra name, which poetry-core 1.3 will generate anyway

* Changelog

* Upper bound build-system requirements

* Remove upgrade note; expand changelog entry a little.

* Fix typo in build-system comment

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-10-07 15:15:10 +01:00
Patrick Cloke 66a7857334
Use stable identifiers for MSC3771 & MSC3773. (#14050)
These are both part of Matrix 1.4 which has now been released.

For now, support both the unstable and stable identifiers.
2022-10-07 09:26:40 -04:00
Patrick Cloke b4ec4f5e71
Track notification counts per thread (implement MSC3773). (#13776)
When retrieving counts of notifications segment the results based on the
thread ID, but choose whether to return them as individual threads or as
a single summed field by letting the client opt-in via a sync flag.

The summarization code is also updated to be per thread, instead of per
room.
2022-10-04 09:47:04 -04:00
Patrick Cloke e70c6b720e
Disable pushing for server ACL events (MSC3786). (#13997)
Switches to the stable identifier for MSC3786 and enables it
by default.

This disables pushes of m.room.server_acl events.
2022-10-04 07:08:27 -04:00
reivilibre b381701f8c
Announce that legacy metric names are deprecated, will be turned off by default in Synapse v1.71.0 and removed altogether in Synapse v1.73.0. (#14024) 2022-10-03 17:16:15 +00:00
Brendan Abolivier be76cd8200
Allow admins to require a manual approval process before new accounts can be used (using MSC3866) (#13556) 2022-09-29 15:23:24 +02:00
Patrick Cloke efd108b45d
Accept & store thread IDs for receipts (implement MSC3771). (#13782)
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
2022-09-23 14:33:28 +00:00
reivilibre c06b2b7142
Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823) 2022-09-23 11:47:16 +01:00
Hugh Nimmo-Smith 0fd2f2d460
Implementation of MSC3882 login token request (#13722) 2022-09-21 15:12:29 +00:00
Brendan Abolivier 8ae42ab8fa
Support enabling/disabling pushers (from MSC3881) (#13799)
Partial implementation of MSC3881
2022-09-21 14:39:01 +00:00