Commit Graph

15323 Commits

Author SHA1 Message Date
reivilibre 1f5473465d
Refresh remote profiles that have been marked as stale, in order to fill the user directory. [rei:userdirpriv] (#14756)
* Scaffolding for background process to refresh profiles

* Add scaffolding for background process to refresh profiles for a given server

* Implement the code to select servers to refresh from

* Ensure we don't build up multiple looping calls

* Make `get_profile` able to respect backoffs

* Add logic for refreshing users

* When backing off, schedule a refresh when the backoff is over

* Wake up the background processes when we receive an interesting state event

* Add tests

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Add comment about 1<<62

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-03-16 11:44:11 +00:00
Andrew Morgan 4953cd71df
Move Account Validity callbacks to a dedicated file (#15237) 2023-03-16 10:35:31 +00:00
reivilibre f54f877f27
Preparatory work to fix the user directory assuming that any remote membership state events represent a profile change. [rei:userdirpriv] (#14755)
* Remove special-case method for new memberships only, use more generic method

* Only collect profiles from state events in public rooms

* Add a table to track stale remote user profiles

* Add store methods to set and delete rows in this new table

* Mark remote profiles as stale when a member state event comes in to a private room

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Simplify by removing Optionality of `event_id`

* Replace names and avatars with None if they're set to dodgy things

I think this makes more sense anyway.

* Move schema delta to 74 (I missed the boat?)

* Turns out these can be None after all

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-03-16 09:55:19 +00:00
Patrick Cloke 3bf973edc7
Remove unused class: DirectTcpReplicationClientFactory. (#15272) 2023-03-15 15:42:20 -04:00
reivilibre 63d87c08c8
Add schema comments about the `destinations` and `destination_rooms` tables. (#15247) 2023-03-15 09:25:58 +00:00
reivilibre d0fe417f5c
Remove unused store method `_set_destination_retry_timings_emulated`. (#15266) 2023-03-14 17:32:46 +00:00
Patrick Cloke e7b559d2ca
Avoid unneeded work if auto-join rooms aren't configured. (#15262)
It is not necessary to reach out to the database to check some
parameters if the auto-join rooms are not configured, or (in some cases)
if auto-create rooms is not configured.
2023-03-14 08:18:49 -04:00
David Robertson a1c9869394
Merge branch 'release-v1.79' into develop 2023-03-13 18:35:21 +00:00
David Robertson c071cd5a0e
Ensure fed-sender catchup does not block for full state (#15248)
* Reproduce bad scenario in test
* Avoid catchup optimisation for partial state rooms
2023-03-13 12:31:19 +00:00
David Robertson 4bb26c95a9
Refactor `filter_events_for_server` (#15240)
* Tweak docstring and type hint

* Flip logic and provide better name

* Separate decision from action

* Track a set of strings, not EventBases

* Require explicit boolean options from callers

* Add explicit option for partial state rooms

* Changelog

* Rename param
2023-03-10 15:31:25 +00:00
Andrew Morgan e157c63f68
Fix missing conditional for registering `on_remove_user_third_party_identifier` module api callbacks (#15227 2023-03-10 10:35:18 +00:00
David Robertson ce54477f6f
Give PyCharm some help with `@cache_in_self` (#15238)
* Give PyCharm some help with `@cache_in_self`

* Changelog

* Fix import for old python versions
2023-03-09 19:12:09 +00:00
Sean Quah caf43c3d7c
Faster joins: Fix spurious errors on incremental sync (#15232)
When pushing events in partial state rooms down incremental /sync, we
try to find the `m.room.member` state event for their senders by digging
through their auth events, so that we can present the membership to the
client. Events usually have a membership event in their auth events,
with the exception of the `m.room.create` event and a user's first join
into the room.

When implementing #13477, we took the case of a user's first join into
account, but forgot to handle the `m.room.create` case. This change
fixes that.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-03-09 14:18:39 +00:00
Patrick Cloke 3d060eae6c
Add missing type hints to `synapse.storage.database`. (#15230) 2023-03-09 07:10:09 -05:00
Patrick Cloke e7c3832ba6
Pull in netaddr type hints. (#15231)
And fix any issues from having those type hints.
2023-03-09 07:09:49 -05:00
Shay be4ea209e8
Add topic and name events to group of events that are batch persisted when creating a room. (#15229) 2023-03-08 19:27:20 -08:00
Patrick Cloke 88efc75bab
Include the room ID in more purge room log lines. (#15222) 2023-03-08 20:08:56 +00:00
Shay a368d30c1c
More speedups/fixes to creating batched events (#15195) 2023-03-07 13:54:39 -08:00
Patrick Cloke 20ed8c926b
Stabilize support for MSC3873: disambuguated event push keys. (#15190)
This removes the experimental configuration option and
always escapes the push rule condition keys.

Also escapes any (experimental) push rule condition keys
in the base rules which contain dot in a field name.
2023-03-07 11:27:57 -05:00
Quentin Gliech 47bc84dd53
Pass the Requester down to the HttpTransactionCache. (#15200) 2023-03-07 16:05:22 +00:00
Patrick Cloke 820f02b70b
Stabilize support for MSC3966: event_property_contains push condition. (#15187)
This removes the configuration flag & updates the identifiers to
use the stable version.
2023-03-07 10:06:02 -05:00
Erik Johnston c69aae94cd
Split up txn for fetching device keys (#15215)
We look up keys in batches, but we should do that outside of the
transaction to avoid starving the database pool.
2023-03-07 08:51:34 +00:00
Quentin Gliech 41f127e068
Pass the requester during event serialization. (#15174)
This allows Synapse to properly include the transaction ID in the
unsigned data of events.
2023-03-06 16:08:39 +00:00
Patrick Cloke 05e0a4089a
Stop applying edits to event contents (MSC3925). (#15193)
Enables MSC3925 support by default, which:

* Includes the full edit event in the bundled aggregations of an
  edited event.
* Stops modifying the original event's content to return the new
  content from the edit event.

This is a backwards-incompatible change that is considered to be
"correct" by the spec.
2023-03-06 09:43:01 -05:00
Patrick Cloke fd9cadcf53
Stabilize support for MSC3758: event_property_is push condition (#15185)
This removes the configuration flag & updates the identifiers to
use the stable version.
2023-03-06 08:38:01 -05:00
Patrick Cloke 02f74f3a99
Combine AbstractStreamIdTracker and AbstractStreamIdGenerator. (#15192)
AbstractStreamIdTracker (now) has only a single sub-class: AbstractStreamIdGenerator,
combine them to simplify some code and remove any direct references to
AbstractStreamIdTracker.
2023-03-03 08:13:37 -05:00
Quentin Gliech 848f7e3d5f
Remove unspecced and buggy `PUT` method on the unstable `/rooms/<room_id>/batch_send` endpoint. (#15199) 2023-03-03 12:22:49 +00:00
Andrew Morgan 15e975f68f
Experimental MSC3890 Implementation: Fix deleting account data when using an account data writer worker (#14869) 2023-03-03 10:51:57 +00:00
Andrew Morgan 1eea662780
Add a `get_next_txn` method to `StreamIdGenerator` to match `MultiWriterIdGenerator` (#15191 2023-03-02 18:27:00 +00:00
Dirk Klimpel ecbe0ddbe7
Add support for knocking to workers. (#15133) 2023-03-02 12:59:53 -05:00
Quentin Gliech c8665dd25d
Remove the unspecced and bugged PUT /knock/{roomIdOrAlias} endpoint (#15189) 2023-03-02 17:16:54 +00:00
Patrick Cloke 8ef324ea6f
Update intentional mentions (MSC3952) to depend on `exact_event_property_contains` (MSC3966). (#15051)
This replaces the specific `is_user_mention` push rule condition
used in MSC3952 with the generic `exact_event_property_contains`
push rule condition from MSC3966.
2023-03-02 08:30:51 -05:00
Patrick Cloke 33a85cf08c
Fix conflicting URLs for dehydrated devices. (#15180) 2023-03-02 07:24:29 -05:00
Dirk Klimpel 65f10afb64
Move event_reports to `RoomWorkerStore` (#15165) 2023-03-02 10:38:46 +00:00
Hugh Nimmo-Smith 916b8061d2
Implementation of MSC3967: Don't require UIA for initial upload of cross signing keys (#15077) 2023-03-02 10:34:59 +00:00
Richard van der Hoff 2b78981736
Remove support for aggregating reactions (#15172)
It turns out that no clients rely on server-side aggregation of `m.annotation`
relationships: it's just not very useful as currently implemented.

It's also non-trivial to calculate.

I want to remove it from MSC2677, so to keep the implementation in line, let's
remove it here.
2023-02-28 18:49:28 +00:00
H. Shay b2fd03d075 Merge branch 'master' into develop 2023-02-28 10:14:20 -08:00
reivilibre d62cd940cb
Fix a long-standing bug where an initial sync would not respond to changes to the list of ignored users if there was an initial sync cached. (#15163) 2023-02-28 17:11:26 +00:00
reivilibre 682d31c702
Allow use of the `/filter` Client-Server APIs on workers. (#15134) 2023-02-28 16:37:19 +00:00
Patrick Cloke c369d82df0
Add missing type hints to InsecureInterceptableContextFactory. (#15164) 2023-02-28 10:17:55 -05:00
Dirk Klimpel 93f7955eba
Admin API endpoint to delete a reported event (#15116)
* Admin api to delete event report

* lint +  tests

* newsfile

* Apply suggestions from code review

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* revert changes - move to WorkerStore

* update unit test

* Note that timestamp is in millseconds

---------

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2023-02-28 12:09:10 +00:00
Travis Ralston 189a878a35
Remove dangling reference to being a reference implementation (#15167)
* Remove dangling reference to being a reference implementation

* Create 15167.misc
2023-02-27 20:08:18 +00:00
Andrew Morgan b40657314e
Add module API callbacks for adding and deleting local 3PID associations (#15044 2023-02-27 14:19:19 +00:00
Patrick Cloke 4fc8875876
Refactor media modules. (#15146)
* Removes the `v1` directory from `test.rest.media.v1`.
* Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`.
* Flatten the `v1` directory from `synapse.rest.media`,  but leave compatiblity
  with 3rd party media repositories and spam checkers.
2023-02-27 08:26:05 -05:00
Andrew Morgan 3f2ef205e2
Small fixes to `MatrixFederationHttpClient` docstrings (#15148) 2023-02-27 13:03:22 +00:00
Shay 1c95ddd09b
Batch up storing state groups when creating new room (#14918) 2023-02-24 13:15:29 -08:00
Erik Johnston b2357a898c
Fix bug where 5s delays would occasionally happen. (#15150)
This only affects deployments using workers.
2023-02-24 14:39:50 +00:00
Sean Quah 335f52d595
Improve handling of non-ASCII characters in user directory search (#15143)
* Fix a long-standing bug where non-ASCII characters in search terms,
  including accented letters, would not match characters in a different
  case.
* Fix a long-standing bug where search terms using combining accents
  would not match display names using precomposed accents and vice
  versa.

To fully take effect, the user directory must be rebuilt after this
change.

Fixes #14630.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-02-24 13:39:45 +00:00
Patrick Cloke 682151a464
Do not fail completely if oEmbed autodiscovery fails. (#15092)
Previously if an autodiscovered oEmbed request failed (e.g. the
oEmbed endpoint is down or does not exist) then the entire URL
preview would fail. Instead we now return everything we can, even
if this additional request fails.
2023-02-23 16:08:53 -05:00
Patrick Cloke f8a584ed02
Stop parsing the unspecced type parameter on thumbnail requests. (#15137)
Ideally we would replace this with parsing of the Accept header
or something else, but for now just make Synapse spec compliant
by ignoring the unspecced parameter.

It does not seem that this is ever sent by a client, and even if it is
there's a reasonable fallback.
2023-02-23 16:07:46 -05:00