Commit Graph

3317 Commits

Author SHA1 Message Date
Dirk Klimpel de4390cd40
Convert async to normal tests in `TestSSOHandler` (#15433)
* Convert async to normal tests in `TestSSOHandler`

* newsfile
2023-04-14 10:48:35 +00:00
reivilibre edae20f926
Improve robustness when handling a perspective key response by deduplicating received server keys. (#15423)
* Change `store_server_verify_keys` to take a `Mapping[(str, str), FKR]`

This is because we already can't handle duplicate keys — leads to cardinality violation

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-04-13 15:35:03 +01:00
Patrick Cloke 2503126d52
Implement MSC2174: move redacts to a content property. (#15395)
This moves `redacts` from being a top-level property to
a `content` property in a new room version.

MSC2176 (which was previously implemented) states to not
`redact` this property.
2023-04-13 13:47:07 +00:00
Will Hunt 253e86a72e
Throw if the appservice config list is the wrong type (#15425)
* raise a ConfigError on an invalid app_service_config_files

* changelog

* Move config check to read_config

* Add test

* Ensure list also contains strings
2023-04-12 11:28:46 +00:00
Patrick Cloke 83649b891d
Implement MSC3989 to redact the origin field. (#15393)
This will be done in a future room version, for now an unstable
room version is added which redacts the origin field.
2023-04-05 14:42:46 -04:00
Shay 6b23d74ad1
Delete server-side backup keys when deactivating an account. (#15181) 2023-04-04 20:16:08 +00:00
Patrick Cloke cf2f2934ad
Call appservices on modern paths, falling back to legacy paths. (#15317)
This uses the specced /_matrix/app/v1/... paths instead of the
"legacy" paths. If the homeserver receives an error it will retry
using the legacy path.
2023-04-03 13:20:32 -04:00
Erik Johnston 6204c3663e
Revert pruning of old devices (#15360)
* Revert "Fix registering a device on an account with lots of devices (#15348)"

This reverts commit f0d8f66eaa.

* Revert "Delete stale non-e2e devices for users, take 3 (#15183)"

This reverts commit 78cdb72cd6.
2023-03-31 13:51:51 +01:00
Mathieu Velten 6f68e32bfb
to_device updates could be dropped when consuming the replication stream (#15349)
Co-authored-by: reivilibre <oliverw@matrix.org>
2023-03-30 19:41:14 +02:00
Erik Johnston 91c3f32673
Speed up SQLite unit test CI (#15334)
Tests now take 40% of the time.
2023-03-30 16:21:12 +01:00
Patrick Cloke ae4acda1bb
Implement MSC3984 to proxy /keys/query requests to appservices. (#15321)
If enabled, for users which are exclusively owned by an application
service then the appservice will be queried for devices in addition
to any information stored in the Synapse database.
2023-03-30 08:39:38 -04:00
Sean Quah d9f694932c
Fix spinloop during partial state sync when a prev event is in backoff (#15351)
Previously, we would spin in a tight loop until
`update_state_for_partial_state_event` stopped raising
`FederationPullAttemptBackoffError`s. Replace the spinloop with a wait
until the backoff period has expired.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-03-30 13:36:41 +01:00
Warren Bailey a3bad89d57
Add the ability to enable/disable registrations when in the OIDC flow (#14978)
Signed-off-by: Warren Bailey <warren@warrenbailey.net>
2023-03-30 11:09:41 +00:00
Erik Johnston f0d8f66eaa
Fix registering a device on an account with lots of devices (#15348)
Fixes up #15183
2023-03-29 13:37:06 +00:00
Erik Johnston 78cdb72cd6
Delete stale non-e2e devices for users, take 3 (#15183)
This should help reduce the number of devices e.g. simple bots the repeatedly login rack up.

We only delete non-e2e devices as they should be safe to delete, whereas if we delete e2e devices for a user we may accidentally break their ability to receive e2e keys for a message.
2023-03-29 12:07:14 +01:00
Patrick Cloke 5282ba1e2b
Implement MSC3983 to proxy /keys/claim queries to appservices. (#15314)
Experimental support for MSC3983 is behind a configuration flag.
If enabled, for users which are exclusively owned by an application
service then the appservice will be queried for one-time keys *if*
there are none uploaded to Synapse.
2023-03-28 18:26:27 +00:00
dependabot[bot] bd4d958aaf
Bump ruff from 0.0.252 to 0.0.259 (#15328)
* Bump ruff from 0.0.252 to 0.0.259

Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.252 to 0.0.259.
- [Release notes](https://github.com/charliermarsh/ruff/releases)
- [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md)
- [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.252...v0.0.259)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix new warnings

* Mypy

* Newsfile

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
2023-03-28 09:46:47 +01:00
Quentin Gliech 5b70f240cf
Make cleaning up pushers depend on the device_id instead of the token_id (#15280)
This makes it so that we rely on the `device_id` to delete pushers on logout,
instead of relying on the `access_token_id`. This ensures we're not removing
pushers on token refresh, and prepares for a world without access token IDs
(also known as the OIDC).

This actually runs the `set_device_id_for_pushers` background update, which
was forgotten in #13831.

Note that for backwards compatibility it still deletes pushers based on the
`access_token` until the background update finishes.
2023-03-24 11:09:39 -04:00
Patrick Cloke 68a6717312
Reject mentions on the C-S API which are invalid. (#15311)
Invalid mentions data received over the Client-Server API should
be rejected with a 400 error. This will hopefully stop clients from
sending invalid data, although does not help with data received
over federation.
2023-03-24 08:31:14 -04:00
David Robertson 3b0083c92a
Use immutabledict instead of frozendict (#15113)
Additionally:

* Consistently use `freeze()` in test

---------

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-03-22 17:15:34 +00:00
Andrew Morgan ec9224bf9a
Make `POST /_matrix/client/v3/rooms/{roomId}/report/{eventId}` endpoint return 404 if event exists, but the user lacks access (#15300) 2023-03-21 13:24:03 +00:00
Andrew Morgan b6aef59334
Make `EventHandler.get_event` return `None` when the requested event is not found (#15298) 2023-03-21 13:23:47 +00:00
Erik Johnston 827f198177
Fix error when sending message into deleted room. (#15235)
When a room is deleted in Synapse we remove the event forward
extremities in the room, so if (say a bot) tries to send a message into
the room we error out due to not being able to calculate prev events for
the new event *before* we check if the sender is in the room.

Fixes #8094
2023-03-21 09:13:43 +00:00
Patrick Cloke a5fb382a29
Separate HTTP preview code and URL previewer. (#15269)
Separates REST layer code from the actual URL previewing.
2023-03-20 14:32:26 -04:00
Shay 5ab7146e19
Add Synapse-Trace-Id to access-control-expose-headers header (#14974) 2023-03-20 11:14:05 -07:00
Patrick Cloke afb216c202
Remove no-op send_command for Redis replication. (#15274)
With Redis commands do not need to be re-issued by the main
process (they fan-out to all processes at once) and thus it is no
longer necessary to worry about them reflecting recursively forever.
2023-03-16 11:13:30 -04:00
reivilibre 1f5473465d
Refresh remote profiles that have been marked as stale, in order to fill the user directory. [rei:userdirpriv] (#14756)
* Scaffolding for background process to refresh profiles

* Add scaffolding for background process to refresh profiles for a given server

* Implement the code to select servers to refresh from

* Ensure we don't build up multiple looping calls

* Make `get_profile` able to respect backoffs

* Add logic for refreshing users

* When backing off, schedule a refresh when the backoff is over

* Wake up the background processes when we receive an interesting state event

* Add tests

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Add comment about 1<<62

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-03-16 11:44:11 +00:00
Andrew Morgan 4953cd71df
Move Account Validity callbacks to a dedicated file (#15237) 2023-03-16 10:35:31 +00:00
David Robertson a1c9869394
Merge branch 'release-v1.79' into develop 2023-03-13 18:35:21 +00:00
David Robertson c071cd5a0e
Ensure fed-sender catchup does not block for full state (#15248)
* Reproduce bad scenario in test
* Avoid catchup optimisation for partial state rooms
2023-03-13 12:31:19 +00:00
David Robertson 4bb26c95a9
Refactor `filter_events_for_server` (#15240)
* Tweak docstring and type hint

* Flip logic and provide better name

* Separate decision from action

* Track a set of strings, not EventBases

* Require explicit boolean options from callers

* Add explicit option for partial state rooms

* Changelog

* Rename param
2023-03-10 15:31:25 +00:00
Andrew Morgan e157c63f68
Fix missing conditional for registering `on_remove_user_third_party_identifier` module api callbacks (#15227 2023-03-10 10:35:18 +00:00
Patrick Cloke e7c3832ba6
Pull in netaddr type hints. (#15231)
And fix any issues from having those type hints.
2023-03-09 07:09:49 -05:00
Patrick Cloke 20ed8c926b
Stabilize support for MSC3873: disambuguated event push keys. (#15190)
This removes the experimental configuration option and
always escapes the push rule condition keys.

Also escapes any (experimental) push rule condition keys
in the base rules which contain dot in a field name.
2023-03-07 11:27:57 -05:00
Quentin Gliech 47bc84dd53
Pass the Requester down to the HttpTransactionCache. (#15200) 2023-03-07 16:05:22 +00:00
Patrick Cloke 820f02b70b
Stabilize support for MSC3966: event_property_contains push condition. (#15187)
This removes the configuration flag & updates the identifiers to
use the stable version.
2023-03-07 10:06:02 -05:00
Patrick Cloke 05e0a4089a
Stop applying edits to event contents (MSC3925). (#15193)
Enables MSC3925 support by default, which:

* Includes the full edit event in the bundled aggregations of an
  edited event.
* Stops modifying the original event's content to return the new
  content from the edit event.

This is a backwards-incompatible change that is considered to be
"correct" by the spec.
2023-03-06 09:43:01 -05:00
Patrick Cloke fd9cadcf53
Stabilize support for MSC3758: event_property_is push condition (#15185)
This removes the configuration flag & updates the identifiers to
use the stable version.
2023-03-06 08:38:01 -05:00
6543 6b6e91e610
Fix ICU tests on alpine / macOS. (#15177)
The word boundary behaviour is slightly different, consider it
acceptable for the tests.
2023-03-03 14:22:06 +00:00
Patrick Cloke 8ef324ea6f
Update intentional mentions (MSC3952) to depend on `exact_event_property_contains` (MSC3966). (#15051)
This replaces the specific `is_user_mention` push rule condition
used in MSC3952 with the generic `exact_event_property_contains`
push rule condition from MSC3966.
2023-03-02 08:30:51 -05:00
Hugh Nimmo-Smith 916b8061d2
Implementation of MSC3967: Don't require UIA for initial upload of cross signing keys (#15077) 2023-03-02 10:34:59 +00:00
Richard van der Hoff 2b78981736
Remove support for aggregating reactions (#15172)
It turns out that no clients rely on server-side aggregation of `m.annotation`
relationships: it's just not very useful as currently implemented.

It's also non-trivial to calculate.

I want to remove it from MSC2677, so to keep the implementation in line, let's
remove it here.
2023-02-28 18:49:28 +00:00
H. Shay b2fd03d075 Merge branch 'master' into develop 2023-02-28 10:14:20 -08:00
reivilibre d62cd940cb
Fix a long-standing bug where an initial sync would not respond to changes to the list of ignored users if there was an initial sync cached. (#15163) 2023-02-28 17:11:26 +00:00
Patrick Cloke e746f80b4f
Do not accept pattern_type from user input in push rules. (#15088)
Internally the push rules module uses a `pattern_type` property for `event_match`
conditions (and `related_event_match`) to mark the condition as matching the
current user's Matrix ID or localpart.

This is leaky to the Client-Server API where a user can successfully set a condition
which provides `pattern_type` instead of `pattern` (note that there's no benefit to
doing this -- the user can just use their own Matrix ID or localpart instead). When
serializing back to the client the `pattern_type` property is converted into a proper
`pattern`.

The following changes are made to avoid this:

* Separate the `KnownCondition::EventMatch` enum value into `EventMatch`
  and `EventMatchType`, each with their own expected properties. (Note that a
  similar change is made for `RelatedEventMatch`.)
* Make it such that the `pattern_type` variants serialize to the same condition kind,
  but cannot be deserialized (since they're only provided by base rules).
* As a final tweak, convert `user_id` vs. `user_localpart` values into an enum.
2023-02-28 10:11:20 -05:00
Dirk Klimpel 93f7955eba
Admin API endpoint to delete a reported event (#15116)
* Admin api to delete event report

* lint +  tests

* newsfile

* Apply suggestions from code review

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* revert changes - move to WorkerStore

* update unit test

* Note that timestamp is in millseconds

---------

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2023-02-28 12:09:10 +00:00
Andrew Morgan b40657314e
Add module API callbacks for adding and deleting local 3PID associations (#15044 2023-02-27 14:19:19 +00:00
Patrick Cloke 4fc8875876
Refactor media modules. (#15146)
* Removes the `v1` directory from `test.rest.media.v1`.
* Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`.
* Flatten the `v1` directory from `synapse.rest.media`,  but leave compatiblity
  with 3rd party media repositories and spam checkers.
2023-02-27 08:26:05 -05:00
Shay 1c95ddd09b
Batch up storing state groups when creating new room (#14918) 2023-02-24 13:15:29 -08:00
Erik Johnston b2357a898c
Fix bug where 5s delays would occasionally happen. (#15150)
This only affects deployments using workers.
2023-02-24 14:39:50 +00:00