Commit Graph

21270 Commits

Author SHA1 Message Date
Brendan Abolivier d24a1486e5 Fixup changelog 2022-05-18 11:46:05 +01:00
Brendan Abolivier 1aa30f7b3e 1.59.1 2022-05-18 11:41:53 +01:00
David Robertson c22314c4e8 Discard null-containing strings before updating the user directory (#12762) 2022-05-18 11:40:09 +01:00
David Robertson d4713d3e33
Discard null-containing strings before updating the user directory (#12762) 2022-05-18 11:28:14 +01:00
Hubert Chathi 8afb7b55d0
Make handling of federation Authorization header (more) compliant with RFC7230 (#12774)
The main differences are:
- values with delimiters (such as colons) should be quoted, so always
  quote the origin, since it could contain a colon followed by a port
  number
- should allow more than one space after "X-Matrix"
- quoted values with backslash-escaped characters should be unescaped
- names should be case insensitive
2022-05-18 11:19:30 +01:00
Adam 37935b5183
Move methods that call add_push_rule to PushRuleStore (#12772)
Signed-off-by: Adam Roddick <ajroddick@tuta.io>
2022-05-18 09:37:48 +00:00
David Robertson 0d17357fcd
Suggest using docker when testing against postgres (#12765)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-05-17 18:05:53 +00:00
Mathieu Velten 182ca78a12
Delete events from federation_inbound_events_staging table on purge (#12770) 2022-05-17 17:01:06 +00:00
David Robertson 5331fb5b47
allow `on_invalidate=None` in `@cached` methods (#12769) 2022-05-17 16:06:45 +00:00
Dirk Klimpel 6edefef602
Add some type hints to datastore (#12717) 2022-05-17 15:29:06 +01:00
Travis Ralston 942c30b16b
Add a new room version for MSC3787's knock+restricted join rule (#12623) 2022-05-17 10:41:39 +00:00
Richard van der Hoff 24b590de32
Remove code which updates `application_services_state.last_txn` (#12680)
This column is unused as of #12209, so let's stop writing to it.
2022-05-17 11:07:18 +01:00
SpiritCroc a34a41f135
Fix push for m.read events (#12721)
badge_count_last_call was always zero when the response for push
notifications included a "rejected" key which mapped to an empty list.
2022-05-17 11:03:07 +01:00
David Robertson 1402159bb8
Merge branch 'master' into develop 2022-05-17 11:00:54 +01:00
Erik Johnston 32ef24fbd7
Add index to cache invalidations (#12747)
For workers that rarely write to the cache the `get_all_updated_caches`
query can become expensive if the worker falls behind when reading the
cache.
2022-05-17 09:34:59 +00:00
Erik Johnston fcf951d5dc
Track in memory events using weakrefs (#10533) 2022-05-17 10:34:27 +01:00
David Robertson 44d7bb13c3
version tweak in changelog 2022-05-17 10:30:31 +01:00
David Robertson 5c3d525cad
1.59.0 2022-05-17 10:27:51 +01:00
David Robertson 1fe202a1a3
Tidy up and type-hint the database engine modules (#12734)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-05-17 00:34:38 +01:00
Andrew Morgan 6d8d1218dd
Fix typo in name of 'run_background_tasks_on' option in config manual (#12749) 2022-05-16 17:31:12 +00:00
Andrew Morgan 3eafee629d Revert "changelog"
This reverts commit e24c11afd6.

whoops...
2022-05-16 17:52:22 +01:00
Andrew Morgan e24c11afd6 changelog 2022-05-16 17:51:43 +01:00
Andrew Morgan 83be72d76c
Add `StreamKeyType` class and replace string literals with constants (#12567) 2022-05-16 15:35:31 +00:00
Erik Johnston 4ea546067d
Fix query performance for /sync (#12745) 2022-05-16 16:30:35 +01:00
Šimon Brandner 3ce15cc7be
Avoid unnecessary copies when filtering private read receipts. (#12711)
A minor optimization to avoid unnecessary copying/building
identical dictionaries when filtering private read receipts.

Also clarifies comments and cleans-up some tests.
2022-05-16 15:06:23 +00:00
David Robertson b4eb163434 Synapse 1.59.0rc2 (2022-05-16)
==============================
 
 Synapse 1.59 makes several changes that server administrators should be aware of:
 
 - Device name lookup over federation is now disabled by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616))
 - The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](https://github.com/matrix-org/synapse/issues/12452), [\#12654](https://github.com/matrix-org/synapse/issues/12654))
 
 See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590) for more details.
 
 Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](https://github.com/matrix-org/synapse/issues/12597))
 
 Bugfixes
 --------
 
 - Fix a bug introduced in Synapse 1.58.0 where `/sync` would fail if the most recent event in a room was rejected. ([\#12729](https://github.com/matrix-org/synapse/issues/12729))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmKCTJQACgkQkD7OEIo5
 3t1JFw/+NCKc2jBSXQHfcN6cSqfmBFd3sCLDkTiVtQy8k55OW4ioE+qvL7NlMWQW
 4Qon/DpCGV5KV3mJ3CdqaUqaA6B69L68JSUAkDgIlMfPNOsAPHvojp+Pf1uSPApc
 PXjhHhiV/BHRLjNmfoavbWB1XfvVfDXCg/9hCH7+EFktmsCAyyzJSb/oaFLKtP/l
 dUCivzAe2JcQcCSx/cO0zU4NqIoeSymK/ffjvHdEr+FHGR6vyT//7t/71Rm2tbZ6
 6j2utZ4VlT9gXylcQKP37Q2HKMP3pgocDwXWTLk2ZJU47oYc9S+iXGwP75fX900L
 bkmxzx2opKw6LCyvU3sMcw+GHpqfcOG1A6uJEGP6JzVj27xDEOto4EYWkhnSz5Cn
 LMHHSh1ZPNtq/aqm6SxlSdgs6uWpyxTAxeniBItSL80HgtOGCeFbddepKu/hcU1G
 V97AdP5+K4aXC36yVGibsFooBgcXRb+xdzrmQoeYHj4Ky3sybBVkBJ7uS4tp5TB5
 h+YbYSpWLDZy/ldbOL3NAqbjhg+l87QnKHAUhpdU1T48a672mE6hWSRq41N0HyP+
 ldEtgzfyjGFMOQ3lvzZ9HUweivgyBQZKxhXKHTB7IQQDtNAKR14YWx3syA76tJki
 RhOdcNwAmVB8L0BEGBVYWKLnw7qoU838hcFMZFBRAkNAY8xZjXk=
 =iA2S
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmKCV9EACgkQkD7OEIo5
 3t0sIQ//RR816yuZT2Ln4hFyHLxafO7wyzmSCLFMaMixJNPYrq3dwGTz92mjHUY/
 tV0XY3Yjg9YqzHps/eQy9hvhnZao0RZDxoXRPnXlDqMYiHsIe61UmYH0F1E6Nz1U
 xln5T/Sl+iIlfR3NskOxRaMk/PIbQYKHb0tQIDhkQo5KkuZZsrf0t/IL+9RO6Tmy
 DFYOpSs7L5U+NufeO2/xR4jCltLb1zunypjNIOBb0G3LJvyTtRSoFscaYdSFEFEJ
 v9lPfJwhUnKJ/NWiH0CgViboN5Z2Igk1jlPdb8o2p7ketD58w4gGMn67Eq0hotau
 esy8FA/sx5/V+pf8mYDeZNTdgBk03W2A5XxYB4JUsCQChtMaELqMZCL6Vk8hi1oi
 S8WnfFoc9EWAbOh/J4bBv00zzA54vGf8ra1qeirndKDyd5pkZnsMaJxohU9LhEJb
 OSTED1NNZuhuO+phwVx0CtnsSkf2Sz2xc8E5sm891s/dU0cRZmXgCt7rMFreEsD3
 IdzUd1oDQ47JhV8EMntRw1Y8/CDPrml5wZ9On18btxfTPTjUJNtLSaIkZ462/AgZ
 naBk1OlzaPV8QwerDqrxy0c891CVqrr3iJ62luFb40kTyy85Facdfw8OyrCtMmvL
 TEgXM5xaFxfydjX2Sfz3Y8QAAokEzLhVlj3sR93acSEDdzN+Gms=
 =JmDV
 -----END PGP SIGNATURE-----

Merge tag 'v1.59.0rc2' into develop

Synapse 1.59.0rc2 (2022-05-16)
==============================

Synapse 1.59 makes several changes that server administrators should be aware of:

- Device name lookup over federation is now disabled by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616))
- The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](https://github.com/matrix-org/synapse/issues/12452), [\#12654](https://github.com/matrix-org/synapse/issues/12654))

See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590) for more details.

Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](https://github.com/matrix-org/synapse/issues/12597))

Bugfixes
--------

- Fix a bug introduced in Synapse 1.58.0 where `/sync` would fail if the most recent event in a room was rejected. ([\#12729](https://github.com/matrix-org/synapse/issues/12729))
2022-05-16 14:55:18 +01:00
Dirk Klimpel 8060034612
Fix typo in listener config (#12742) 2022-05-16 13:50:07 +00:00
Sean Quah a5c26750b5
Fix room upgrades creating an empty room when auth fails (#12696)
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-16 14:06:04 +01:00
Patrick Cloke 86a515ccbf
Consolidate logic for parsing relations. (#12693)
Parse the `m.relates_to` event content field (which describes relations)
in a single place, this is used during:

* Event persistence.
* Validation of the Client-Server API.
* Fetching bundled aggregations.
* Processing of push rules.

Each of these separately implement the logic and each made slightly
different assumptions about what was valid. Some had minor / potential
bugs.
2022-05-16 12:42:45 +00:00
David Robertson 6f04ae7033
Move 1.59 warning to the top 2022-05-16 12:53:10 +01:00
David Robertson c3b232cb39
1.59.0rc2 2022-05-16 12:52:29 +01:00
Erik Johnston 8689230a55
Fix bug /sync returning 404 (#12729)
* Fix bug /sync returning 404

Fixes #12571
2022-05-16 12:06:56 +01:00
Shay cde8af9a49
Add config flags to allow for cache auto-tuning (#12701) 2022-05-13 12:32:39 -07:00
Till e8ae472d3b
Update configs used by Complement to allow more invites (#12731) 2022-05-13 16:45:47 +01:00
Brendan Abolivier 9013104429
Don't create an empty room when checking for MAU limits (#12713) 2022-05-13 15:30:15 +02:00
David Robertson aec69d2481
Another batch of type annotations (#12726) 2022-05-13 12:35:31 +01:00
Jess Porter 39bed28b28
SpamChecker metrics (#12513)
* add Measure blocks all over SpamChecker

Signed-off-by: jesopo <github@lolnerd.net>

* fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck

* better changelog entry
2022-05-13 12:17:38 +01:00
Niklas c9fc2c0d22
Update issuer URL in example OIDC Keycloak config (#12727)
* Update openid.md

Newer versions of keycloak returning a 404 when using the `/auth` prefix.

Related: https://github.com/matrix-org/synapse/issues/12714
2022-05-13 10:15:51 +00:00
Andrew Morgan 57f6c496d0
URL preview cache expiry logs: INFO -> DEBUG, text clarifications (#12720) 2022-05-12 18:16:32 +01:00
David Robertson 17e1eb7749
Reduce the number of "untyped defs" (#12716) 2022-05-12 14:33:50 +00:00
Andy Balaam de1e599b9d
add default_power_level_content_override config option. (#12618)
Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2022-05-12 10:41:35 +00:00
Andrew Morgan 409573f6d0
Fix reference to the wrong symbol in the media admin api docs (#12715) 2022-05-12 09:29:37 +01:00
Sean Quah bf7ce92bf7
Enable cancellation of `GET /members` and `GET /state` requests (#12708)
Enable cancellation of `GET /rooms/$room_id/members`,
`GET /rooms/$room_id/state` and
`GET /rooms/$room_id/state/$state_key/*` requests.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 17:22:34 +01:00
David Robertson db10f2c037
No longer permit empty body when sending receipts (#12709) 2022-05-11 15:34:17 +00:00
Sean Quah 6ee61b9052
Complain if a federation endpoint has the `@cancellable` flag (#12705)
`BaseFederationServlet` wraps its endpoints in a bunch of async code
that has not been vetted for compatibility with cancellation.
Fail CI if a `@cancellable` flag is applied to a federation endpoint.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 14:52:26 +01:00
David Robertson d38d242411
Reload cache factors from disk on SIGHUP (#12673) 2022-05-11 13:43:22 +00:00
Sean Quah a559c8b0d9
Respect the `@cancellable` flag for `ReplicationEndpoint`s (#12700)
While `ReplicationEndpoint`s register themselves via `JsonResource`,
they pass a method that calls the handler, instead of the handler itself,
to `register_paths`. As a result, `JsonResource` will not correctly pick
up the `@cancellable` flag and we have to apply it ourselves.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 12:25:39 +01:00
Sean Quah 9d8e380d2e
Respect the `@cancellable` flag for `RestServlet`s and `BaseFederationServlet`s (#12699)
Both `RestServlet`s and `BaseFederationServlet`s register their handlers
with `HttpServer.register_paths` / `JsonResource.register_paths`. Update
`JsonResource` to respect the `@cancellable` flag on handlers registered
in this way.

Although `ReplicationEndpoint` also registers itself using
`register_paths`, it does not pass the handler method that would have the
`@cancellable` flag directly, and so needs separate handling.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 12:25:13 +01:00
Sean Quah dffecade7d
Respect the `@cancellable` flag for `DirectServe{Html,Json}Resource`s (#12698)
`DirectServeHtmlResource` and `DirectServeJsonResource` both inherit
from `_AsyncResource`. These classes expect to be subclassed with
`_async_render_*` methods.

This commit has no effect on `JsonResource`, despite inheriting from
`_AsyncResource`. `JsonResource` has its own `_async_render` override
which will need to be updated separately.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 12:24:48 +01:00
Patrick Cloke a4c75918b3
Remove unneeded `ActionGenerator` class. (#12691)
It simply passes through to `BulkPushRuleEvaluator`, which can be
called directly instead.
2022-05-11 07:15:21 -04:00