Commit Graph

179 Commits

Author SHA1 Message Date
H. Shay ea45257199 1.75.0rc2 2023-01-12 10:30:54 -08:00
Erik Johnston b50c008453
Re-enable some linting (#14821)
* Re-enable some linting

* Newsfile

* Remove comment
2023-01-12 10:52:07 +00:00
David Robertson 9a4c69f59f
1.75.0rc1 2023-01-10 12:18:50 +00:00
dependabot[bot] 32c2ff8eab
Bump ruff from 0.0.206 to 0.0.215 (#14796)
* Bump ruff from 0.0.206 to 0.0.215

Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.206 to 0.0.215.
- [Release notes](https://github.com/charliermarsh/ruff/releases)
- [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md)
- [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.206...v0.0.215)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Changelog

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-01-09 10:08:43 +00:00
dependabot[bot] f79ef37b8c
Bump ruff from 0.0.189 to 0.0.206 (#14759) 2023-01-05 09:49:50 +00:00
Patrick Cloke 7010a3d015
Switch to ruff instead of flake8. (#14633)
ruff is a flake8-compatible Python linter written in Rust.
It supports the flake8 plugins that we use and is significantly
faster in testing.
2022-12-21 13:05:21 -05:00
Patrick Cloke 5831bed450
Bump minimum PyYAML to 3.13. (#14720)
PyYAML 3.13 fixes some issues with Python 3.7 compatibility
and was released in 2018.
2022-12-21 12:29:19 -05:00
Sean Quah 774e20b570 1.74.0 2022-12-20 16:08:33 +00:00
Sean Quah b8cf480fa9 1.74.0rc1 2022-12-13 13:30:07 +00:00
Brendan Abolivier 2a3cd59dd0
Add optional ICU support for user search (#14464)
Fixes #13655

This change uses ICU (International Components for Unicode) to improve boundary detection in user search.

This change also adds a new dependency on libicu-dev and pkg-config for the Debian packages, which are available in all supported distros.
2022-12-12 13:21:17 +01:00
Olivier Wilkinson (reivilibre) bb9f156978 Merge branch 'master' into develop 2022-12-06 13:37:49 +00:00
Olivier Wilkinson (reivilibre) dc6b60f68d 1.73.0 2022-12-06 11:49:37 +00:00
Patrick Cloke 6a8310f3df
Compare to the earliest known stream pos in the stream change cache. (#14435)
The internal methods of the StreamChangeCache were inconsistently
treating the earliest known stream position as valid. It is now treated as
invalid, meaning the cache cannot determine if an entity at the earliest
known stream position has changed or not.
2022-12-05 09:00:59 -05:00
David Robertson 6a41e5022e
1.73.0rc2 2022-12-01 10:02:56 +00:00
David Robertson 8c5b8e6d40
1.73.0rc1 2022-11-29 12:32:02 +00:00
David Robertson 706b6a1ebb
1.72.0 2022-11-22 10:59:39 +00:00
Erik Johnston 1a8cd8bec0 1.72.0rc1 2022-11-16 15:11:06 +00:00
Sean Quah 404404733c 1.71.0 2022-11-08 10:38:16 +00:00
Sean Quah b00294b8b1 1.71.0rc2 2022-11-04 12:01:17 +00:00
David Robertson ddbba28d52
1.71.0rc1 2022-11-01 12:10:51 +00:00
Olivier Wilkinson (reivilibre) 1335367ca7 Merge branch 'master' into develop 2022-10-28 15:59:51 +01:00
Olivier Wilkinson (reivilibre) cc3a04876f 1.70.1 2022-10-28 12:10:37 +01:00
Olivier Wilkinson (reivilibre) 86b7d9b886 Merge branch 'master' into develop 2022-10-26 13:05:09 +01:00
Olivier Wilkinson (reivilibre) 23fa636ed7 1.70.0 2022-10-26 11:13:29 +01:00
Olivier Wilkinson (reivilibre) 85fcbba595 Merge branch 'release-v1.70' into develop 2022-10-25 15:39:35 +01:00
Quentin Gliech 9192d74b0b
Refactor OIDC tests to better mimic an actual OIDC provider. (#13910)
This implements a fake OIDC server, which intercepts calls to the HTTP client.
Improves accuracy of tests by covering more internal methods.

One particular example was the ID token validation, which previously mocked.

This uncovered an incorrect dependency: Synapse actually requires at least
authlib 0.15.1, not 0.14.0.
2022-10-25 14:25:02 +00:00
Olivier Wilkinson (reivilibre) 8c8fcdb87d 1.70.0rc2 2022-10-25 11:02:55 +01:00
Erik Johnston 1c642156d7
Only build aarch64 wheels for cpython manylinux (#14259) 2022-10-21 14:22:26 +00:00
Erik Johnston 5f77b74215
Try building ABI3 wheels for cpython (#14253) 2022-10-21 14:17:19 +00:00
Erik Johnston 15a240f135 1.70.0rc1 2022-10-19 14:12:03 +01:00
David Robertson 844ce47b9b
Don't pin dev-deps in pyproject; use lower bounds (#14227)
* Don't pin dev-deps in pyproject; use lower bounds

This makes it slightly less tedious to update these things via
successive dependabot updates, by reducing the likelihood of a merge
conflict.

* Changelog

* Changelog
2022-10-18 16:44:43 +01:00
David Robertson 8e50299d8b
Fix `track_memory_usage` on poetry-core 1.3.x installations (#14221)
* Fix `track_memory_usage` on poetry-core 1.3.x installations

The same kind of problem as discussed in #14085:

1. we defined an extra with an underscore
2. we look it up at runtime with an underscore
3. but poetry-core 1.3.x. installs it with a dash, causing (2) to fail.

Fix by using a dash everywhere.

* Changelog
2022-10-18 13:59:04 +01:00
Ivan Shapovalov 4af93bd7f6
Allow poetry-core 1.3.2 (#14217)
Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>
2022-10-18 10:38:58 +01:00
dependabot[bot] c101fc6568
Bump click from 8.1.1 to 8.1.3 (#14201) 2022-10-17 17:54:11 +01:00
Erik Johnston 130668b66c Merge branch 'master' into develop 2022-10-17 17:04:40 +01:00
Erik Johnston 6b097a3e17 1.69.0 2022-10-17 11:31:15 +01:00
Patrick Cloke bc2bd92b93 Merge remote-tracking branch 'origin/release-v1.69' into develop 2022-10-14 14:11:27 -04:00
Erik Johnston b43be004b4 1.69.0rc4 2022-10-14 15:05:04 +01:00
Erik Johnston 19eb23bf32 1.69.0rc3 2022-10-12 13:24:22 +01:00
David Robertson 3f057e4c54
Use minimal Rust installation in docker images and CI (#14141) 2022-10-12 10:47:02 +01:00
David Robertson f1673866ed
Unpin build-system requirements, but impose an upper-bound (#14085)
* Revert to prior build-system requirements

This reverts #14080.

* Use normalised extra name, which poetry-core 1.3 will generate anyway

* Changelog

* Upper bound build-system requirements

* Remove upgrade note; expand changelog entry a little.

* Fix typo in build-system comment

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-10-07 15:15:10 +01:00
Sean Quah 44741aa85b Synapse 1.69.0rc2 (2022-10-06)
==============================
 
 Please note that legacy Prometheus metric names are now deprecated and will be removed in Synapse 1.73.0.
 Server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names.
 See the [upgrade notes](https://matrix-org.github.io/synapse/v1.69/upgrade.html#upgrading-to-v1690) for more details.
 
 Deprecations and Removals
 -------------------------
 
 - Deprecate the `generate_short_term_login_token` method in favor of an async `create_login_token` method in the Module API. ([\#13842](https://github.com/matrix-org/synapse/issues/13842))
 
 Internal Changes
 ----------------
 
 - Ensure Synapse v1.69 works with upcoming database changes in v1.70. ([\#14045](https://github.com/matrix-org/synapse/issues/14045))
 - Fix a bug introduced in Synapse v1.68.0 where messages could not be sent in rooms with non-integer `notifications` power level. ([\#14073](https://github.com/matrix-org/synapse/issues/14073))
 - Temporarily pin build-system requirements to workaround an incompatibility with poetry-core 1.3.0. This will be reverted before the v1.69.0 release proper, see [\#14079](https://github.com/matrix-org/synapse/issues/14079). ([\#14080](https://github.com/matrix-org/synapse/issues/14080))
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmM+674ACgkQ+84Kzgcy
 GG+TdQwAm3EswLr9gq+PPD59ewt7BI6DgdrW0aw+OS0qbYBEg2BklNk8RpQsELFC
 U/FT+Ty9VakI+/ID4W0mHhPDc9xtZz09NE+fZqrSNxel7rdL8SfQu3gsrT433o53
 FElQbeknuspiX8kokBFnzuFKR+HzZsZq5Of9SqSy6ItO/YsV8kFdx5QSYolzWZdP
 AU/HNrfEBog/blFJEx504XDg7SgQRjAQM4uOYS1twSlKdG19yDm9+N1kLxdH4nyy
 K5bxKQMARzk35QgM5JQIfryMKVo5rcfB5uotxUXd8s1mlHGxAVAb+jfXdSSvHrKq
 SSUD5cII2XQmkZbsqig1fQaknh8RbwR4d3y+3O1Rjm1JlCQ4vjPMZ7KVyTS2nmVA
 wdo9bEMbtP2qMMoq5Mlj9QpMg3VMU+ZpRDIY6zoa8oWHUCAL4Q/jxeDb4XtvLn3h
 MuE6yRHca390jeYIJnSy1zZjoacIAAEBy3he1oTTor4KhCZV4jLRRtcc/nmR5cZ8
 sgW+7wFh
 =Iu7b
 -----END PGP SIGNATURE-----

Merge tag 'v1.69.0rc2' into develop

Synapse 1.69.0rc2 (2022-10-06)
==============================

Please note that legacy Prometheus metric names are now deprecated and will be removed in Synapse 1.73.0.
Server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names.
See the [upgrade notes](https://matrix-org.github.io/synapse/v1.69/upgrade.html#upgrading-to-v1690) for more details.

Deprecations and Removals
-------------------------

- Deprecate the `generate_short_term_login_token` method in favor of an async `create_login_token` method in the Module API. ([\#13842](https://github.com/matrix-org/synapse/issues/13842))

Internal Changes
----------------

- Ensure Synapse v1.69 works with upcoming database changes in v1.70. ([\#14045](https://github.com/matrix-org/synapse/issues/14045))
- Fix a bug introduced in Synapse v1.68.0 where messages could not be sent in rooms with non-integer `notifications` power level. ([\#14073](https://github.com/matrix-org/synapse/issues/14073))
- Temporarily pin build-system requirements to workaround an incompatibility with poetry-core 1.3.0. This will be reverted before the v1.69.0 release proper, see [\#14079](https://github.com/matrix-org/synapse/issues/14079). ([\#14080](https://github.com/matrix-org/synapse/issues/14080))
2022-10-06 18:48:47 +01:00
David Robertson 720b12c209
Pin build-system requirements (#14080)
* Pin build-system requirements

* Changelog
2022-10-06 12:55:07 +01:00
Sean Quah f6f6bdc7b3 1.69.0rc2 2022-10-06 10:33:23 +01:00
Sean Quah f0019f3f3b Synapse 1.69.0rc1 (2022-10-04)
==============================
 
 Please note that legacy Prometheus metric names are now deprecated and will be removed in Synapse 1.73.0.
 Server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names.
 See the [upgrade notes](https://matrix-org.github.io/synapse/v1.69/upgrade.html#upgrading-to-v1690) for more details.
 
 Features
 --------
 
 - Allow application services to set the `origin_server_ts` of a state event by providing the query parameter `ts` in [`PUT /_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}`](https://spec.matrix.org/v1.4/client-server-api/#put_matrixclientv3roomsroomidstateeventtypestatekey), per [MSC3316](https://github.com/matrix-org/matrix-doc/pull/3316). Contributed by @lukasdenk. ([\#11866](https://github.com/matrix-org/synapse/issues/11866))
 - Allow server admins to require a manual approval process before new accounts can be used (using [MSC3866](https://github.com/matrix-org/matrix-spec-proposals/pull/3866)). ([\#13556](https://github.com/matrix-org/synapse/issues/13556))
 - Exponentially backoff from backfilling the same event over and over. ([\#13635](https://github.com/matrix-org/synapse/issues/13635), [\#13936](https://github.com/matrix-org/synapse/issues/13936))
 - Add cache invalidation across workers to module API. ([\#13667](https://github.com/matrix-org/synapse/issues/13667), [\#13947](https://github.com/matrix-org/synapse/issues/13947))
 - Experimental implementation of [MSC3882](https://github.com/matrix-org/matrix-spec-proposals/pull/3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\#13722](https://github.com/matrix-org/synapse/issues/13722), [\#13868](https://github.com/matrix-org/synapse/issues/13868))
 - Experimental support for thread-specific receipts ([MSC3771](https://github.com/matrix-org/matrix-spec-proposals/pull/3771)). ([\#13782](https://github.com/matrix-org/synapse/issues/13782), [\#13893](https://github.com/matrix-org/synapse/issues/13893), [\#13932](https://github.com/matrix-org/synapse/issues/13932), [\#13937](https://github.com/matrix-org/synapse/issues/13937), [\#13939](https://github.com/matrix-org/synapse/issues/13939))
 - Add experimental support for [MSC3881: Remotely toggle push notifications for another client](https://github.com/matrix-org/matrix-spec-proposals/pull/3881). ([\#13799](https://github.com/matrix-org/synapse/issues/13799), [\#13831](https://github.com/matrix-org/synapse/issues/13831), [\#13860](https://github.com/matrix-org/synapse/issues/13860))
 - Keep track when an event pulled over federation fails its signature check so we can intelligently back-off in the future. ([\#13815](https://github.com/matrix-org/synapse/issues/13815))
 - Improve validation for the unspecced, internal-only `_matrix/client/unstable/add_threepid/msisdn/submit_token` endpoint. ([\#13832](https://github.com/matrix-org/synapse/issues/13832))
 - Faster remote room joins: record _when_ we first partial-join to a room. ([\#13892](https://github.com/matrix-org/synapse/issues/13892))
 - Support a `dir` parameter on the `/relations` endpoint per [MSC3715](https://github.com/matrix-org/matrix-doc/pull/3715). ([\#13920](https://github.com/matrix-org/synapse/issues/13920))
 - Ask mail servers receiving emails from Synapse to not send automatic replies (e.g. out-of-office responses). ([\#13957](https://github.com/matrix-org/synapse/issues/13957))
 
 Bugfixes
 --------
 
 - Send push notifications for invites received over federation. ([\#13719](https://github.com/matrix-org/synapse/issues/13719), [\#14014](https://github.com/matrix-org/synapse/issues/14014))
 - Fix a long-standing bug where typing events would be accepted from remote servers not present in a room. Also fix a bug where incoming typing events would cause other incoming events to get stuck during a fast join. ([\#13830](https://github.com/matrix-org/synapse/issues/13830))
 - Fix a bug introduced in Synapse v1.53.0 where the experimental implementation of [MSC3715](https://github.com/matrix-org/matrix-spec-proposals/pull/3715) would give incorrect results when paginating forward. ([\#13840](https://github.com/matrix-org/synapse/issues/13840))
 - Fix access token leak to logs from proxy agent. ([\#13855](https://github.com/matrix-org/synapse/issues/13855))
 - Fix `have_seen_event` cache not being invalidated after we persist an event which causes inefficiency effects like extra `/state` federation calls. ([\#13863](https://github.com/matrix-org/synapse/issues/13863))
 - Faster room joins: Fix a bug introduced in 1.66.0 where an error would be logged when syncing after joining a room. ([\#13872](https://github.com/matrix-org/synapse/issues/13872))
 - Fix a bug introduced in 1.66.0 where some required fields in the pushrules sent to clients were not present anymore. Contributed by Nico. ([\#13904](https://github.com/matrix-org/synapse/issues/13904))
 - Fix packaging to include `Cargo.lock` in `sdist`. ([\#13909](https://github.com/matrix-org/synapse/issues/13909))
 - Fix a long-standing bug where device updates could cause delays sending out to-device messages over federation. ([\#13922](https://github.com/matrix-org/synapse/issues/13922))
 - Fix a bug introduced in v1.68.0 where Synapse would require `setuptools_rust` at runtime, even though the package is only required at build time. ([\#13952](https://github.com/matrix-org/synapse/issues/13952))
 - Fix a long-standing bug where `POST /_matrix/client/v3/keys/query` requests could result in excessively large SQL queries. ([\#13956](https://github.com/matrix-org/synapse/issues/13956))
 - Fix a performance regression in the `get_users_in_room` database query. Introduced in v1.67.0. ([\#13972](https://github.com/matrix-org/synapse/issues/13972))
 - Fix a bug introduced in v1.68.0 bug where Rust extension wasn't built in `release` mode when using `poetry install`. ([\#14009](https://github.com/matrix-org/synapse/issues/14009))
 - Do not return an unspecified `original_event` field when using the stable `/relations` endpoint. Introduced in Synapse v1.57.0. ([\#14025](https://github.com/matrix-org/synapse/issues/14025))
 - Correctly handle a race with device lists when a remote user leaves during a partial join. ([\#13885](https://github.com/matrix-org/synapse/issues/13885))
 - Correctly handle sending local device list updates to remote servers during a partial join. ([\#13934](https://github.com/matrix-org/synapse/issues/13934))
 
 Improved Documentation
 ----------------------
 
 - Add `worker_main_http_uri` for the worker generator bash script. ([\#13772](https://github.com/matrix-org/synapse/issues/13772))
 - Update URL for the NixOS module for Synapse. ([\#13818](https://github.com/matrix-org/synapse/issues/13818))
 - Fix a mistake in sso_mapping_providers.md: `map_user_attributes` is expected to return `display_name`, not `displayname`. ([\#13836](https://github.com/matrix-org/synapse/issues/13836))
 - Fix a cross-link from the registration admin API to the `registration_shared_secret` configuration documentation. ([\#13870](https://github.com/matrix-org/synapse/issues/13870))
 - Update the man page for the `hash_password` script to correct the default number of bcrypt rounds performed. ([\#13911](https://github.com/matrix-org/synapse/issues/13911), [\#13930](https://github.com/matrix-org/synapse/issues/13930))
 - Emphasize the right reasons when to use `(room_id, event_id)` in a database schema. ([\#13915](https://github.com/matrix-org/synapse/issues/13915))
 - Add instruction to contributing guide for running unit tests in parallel. Contributed by @ashfame. ([\#13928](https://github.com/matrix-org/synapse/issues/13928))
 - Clarify that the `auto_join_rooms` config option can also be used with Space aliases. ([\#13931](https://github.com/matrix-org/synapse/issues/13931))
 - Add some cross references to worker documentation. ([\#13974](https://github.com/matrix-org/synapse/issues/13974))
 - Linkify urls in config documentation. ([\#14003](https://github.com/matrix-org/synapse/issues/14003))
 
 Deprecations and Removals
 -------------------------
 
 - Remove the `complete_sso_login` method from the Module API which was deprecated in Synapse 1.13.0. ([\#13843](https://github.com/matrix-org/synapse/issues/13843))
 - Announce that legacy metric names are deprecated, will be turned off by default in Synapse v1.71.0 and removed altogether in Synapse v1.73.0. See the upgrade notes for more information. ([\#14024](https://github.com/matrix-org/synapse/issues/14024))
 
 Internal Changes
 ----------------
 
 - Speed up creation of DM rooms. ([\#13487](https://github.com/matrix-org/synapse/issues/13487), [\#13800](https://github.com/matrix-org/synapse/issues/13800))
 - Port push rules to using Rust. ([\#13768](https://github.com/matrix-org/synapse/issues/13768), [\#13838](https://github.com/matrix-org/synapse/issues/13838), [\#13889](https://github.com/matrix-org/synapse/issues/13889))
 - Optimise get rooms for user calls. Contributed by Nick @ Beeper (@fizzadar). ([\#13787](https://github.com/matrix-org/synapse/issues/13787))
 - Update the script which makes full schema dumps. ([\#13792](https://github.com/matrix-org/synapse/issues/13792))
 - Use shared methods for cache invalidation when persisting events, remove duplicate codepaths. Contributed by Nick @ Beeper (@fizzadar). ([\#13796](https://github.com/matrix-org/synapse/issues/13796))
 - Improve the `synapse.api.auth.Auth` mock used in unit tests. ([\#13809](https://github.com/matrix-org/synapse/issues/13809))
 - Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. ([\#13823](https://github.com/matrix-org/synapse/issues/13823))
 - Carry IdP Session IDs through user-mapping sessions. ([\#13839](https://github.com/matrix-org/synapse/issues/13839))
 - Fix the release script not publishing binary wheels. ([\#13850](https://github.com/matrix-org/synapse/issues/13850))
 - Raise issue if complement fails with latest deps. ([\#13859](https://github.com/matrix-org/synapse/issues/13859))
 - Correct the comments in the complement dockerfile. ([\#13867](https://github.com/matrix-org/synapse/issues/13867))
 - Create a new snapshot of the database schema. ([\#13873](https://github.com/matrix-org/synapse/issues/13873))
 - Faster room joins: Send device list updates to most servers in rooms with partial state. ([\#13874](https://github.com/matrix-org/synapse/issues/13874), [\#14013](https://github.com/matrix-org/synapse/issues/14013))
 - Add comments to the Prometheus recording rules to make it clear which set of rules you need for Grafana or Prometheus Console. ([\#13876](https://github.com/matrix-org/synapse/issues/13876))
 - Only pull relevant backfill points from the database based on the current depth and limit (instead of all) every time we want to `/backfill`. ([\#13879](https://github.com/matrix-org/synapse/issues/13879))
 - Faster room joins: Avoid waiting for full state when processing `/keys/changes` requests. ([\#13888](https://github.com/matrix-org/synapse/issues/13888))
 - Improve backfill robustness by trying more servers when we get a `4xx` error back. ([\#13890](https://github.com/matrix-org/synapse/issues/13890))
 - Fix mypy errors with canonicaljson 1.6.3. ([\#13905](https://github.com/matrix-org/synapse/issues/13905))
 - Faster remote room joins: correctly handle remote device list updates during a partial join. ([\#13913](https://github.com/matrix-org/synapse/issues/13913))
 - Complement image: propagate SIGTERM to all workers. ([\#13914](https://github.com/matrix-org/synapse/issues/13914))
 - Update an innaccurate comment in Synapse's upsert database helper. ([\#13924](https://github.com/matrix-org/synapse/issues/13924))
 - Update mypy (0.950 -> 0.981) and mypy-zope (0.3.7 -> 0.3.11). ([\#13925](https://github.com/matrix-org/synapse/issues/13925), [\#13993](https://github.com/matrix-org/synapse/issues/13993))
 - Use dedicated `get_local_users_in_room(room_id)` function to find local users when calculating users to copy over during a room upgrade. ([\#13960](https://github.com/matrix-org/synapse/issues/13960))
 - Refactor language in user directory `_track_user_joined_room` code to make it more clear that we use both local and remote users. ([\#13966](https://github.com/matrix-org/synapse/issues/13966))
 - Revert catch-all exceptions being recorded as event pull attempt failures (only handle what we know about). ([\#13969](https://github.com/matrix-org/synapse/issues/13969))
 - Speed up calculating push actions in large rooms. ([\#13973](https://github.com/matrix-org/synapse/issues/13973), [\#13992](https://github.com/matrix-org/synapse/issues/13992))
 - Enable update notifications from Github's dependabot. ([\#13976](https://github.com/matrix-org/synapse/issues/13976))
 - Prototype a workflow to automatically add changelogs to dependabot PRs. ([\#13998](https://github.com/matrix-org/synapse/issues/13998), [\#14011](https://github.com/matrix-org/synapse/issues/14011), [\#14017](https://github.com/matrix-org/synapse/issues/14017), [\#14021](https://github.com/matrix-org/synapse/issues/14021), [\#14027](https://github.com/matrix-org/synapse/issues/14027))
 - Fix type annotations to be compatible with new annotations in development versions of twisted. ([\#14012](https://github.com/matrix-org/synapse/issues/14012))
 - Clear out stale entries in `event_push_actions_staging` table. ([\#14020](https://github.com/matrix-org/synapse/issues/14020))
 - Bump versions of GitHub actions. ([\#13978](https://github.com/matrix-org/synapse/issues/13978), [\#13979](https://github.com/matrix-org/synapse/issues/13979), [\#13980](https://github.com/matrix-org/synapse/issues/13980), [\#13982](https://github.com/matrix-org/synapse/issues/13982), [\#14015](https://github.com/matrix-org/synapse/issues/14015), [\#14019](https://github.com/matrix-org/synapse/issues/14019), [\#14022](https://github.com/matrix-org/synapse/issues/14022), [\#14023](https://github.com/matrix-org/synapse/issues/14023))
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmM8IQMACgkQ+84Kzgcy
 GG+eewv/fl+SluS/J5xQ/bJU/rCp1TREd29vafWbCapjLFI4g/2CsLjWXXfP+zFi
 xHbGA/lWK7Vju/SV/9cURbouosxeCtbc5MrB5l2WmQB21Uw1Ql2DUkGBPVQKb45/
 OkCG3J5qAID9r+OsGf6Ir1pJ6+38r5zliEuTn7557B5Fuawjbx7BWX5dfotjH49r
 WvhyKV+2EzVenmA82iLwC+RSykYcbTkCvKyoNd4Ry6oBOIBDFgLQIVivnQp6W/RK
 CneP7Q64Ga9+WvLUt9Xgp0B1vIWJcPoM9uPeeifb9Pzq7m7jPo48Zld6Sxdp/zAs
 NrLFdTQa+YfgvyvtxDXr8fLPjJ91F1Gv6wyPPtd57tZDLgcn3U2UMifDAjMgCWKB
 7IEKNOH5fSY00q50cnSr2wS2azYPAS4KFfgXArZ0t1RaU2LDkjyZVjJj7eI7zuex
 rlqVqvZtqjdreOVUBkf401zTD/hD4+Vld8BnbHX3liAcGObOUIRSsGCwhFh6bquw
 8hYy9V6Y
 =wV0r
 -----END PGP SIGNATURE-----

Merge tag 'v1.69.0rc1' into develop

Synapse 1.69.0rc1 (2022-10-04)
==============================

Please note that legacy Prometheus metric names are now deprecated and will be removed in Synapse 1.73.0.
Server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names.
See the [upgrade notes](https://matrix-org.github.io/synapse/v1.69/upgrade.html#upgrading-to-v1690) for more details.

Features
--------

- Allow application services to set the `origin_server_ts` of a state event by providing the query parameter `ts` in [`PUT /_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}`](https://spec.matrix.org/v1.4/client-server-api/#put_matrixclientv3roomsroomidstateeventtypestatekey), per [MSC3316](https://github.com/matrix-org/matrix-doc/pull/3316). Contributed by @lukasdenk. ([\#11866](https://github.com/matrix-org/synapse/issues/11866))
- Allow server admins to require a manual approval process before new accounts can be used (using [MSC3866](https://github.com/matrix-org/matrix-spec-proposals/pull/3866)). ([\#13556](https://github.com/matrix-org/synapse/issues/13556))
- Exponentially backoff from backfilling the same event over and over. ([\#13635](https://github.com/matrix-org/synapse/issues/13635), [\#13936](https://github.com/matrix-org/synapse/issues/13936))
- Add cache invalidation across workers to module API. ([\#13667](https://github.com/matrix-org/synapse/issues/13667), [\#13947](https://github.com/matrix-org/synapse/issues/13947))
- Experimental implementation of [MSC3882](https://github.com/matrix-org/matrix-spec-proposals/pull/3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\#13722](https://github.com/matrix-org/synapse/issues/13722), [\#13868](https://github.com/matrix-org/synapse/issues/13868))
- Experimental support for thread-specific receipts ([MSC3771](https://github.com/matrix-org/matrix-spec-proposals/pull/3771)). ([\#13782](https://github.com/matrix-org/synapse/issues/13782), [\#13893](https://github.com/matrix-org/synapse/issues/13893), [\#13932](https://github.com/matrix-org/synapse/issues/13932), [\#13937](https://github.com/matrix-org/synapse/issues/13937), [\#13939](https://github.com/matrix-org/synapse/issues/13939))
- Add experimental support for [MSC3881: Remotely toggle push notifications for another client](https://github.com/matrix-org/matrix-spec-proposals/pull/3881). ([\#13799](https://github.com/matrix-org/synapse/issues/13799), [\#13831](https://github.com/matrix-org/synapse/issues/13831), [\#13860](https://github.com/matrix-org/synapse/issues/13860))
- Keep track when an event pulled over federation fails its signature check so we can intelligently back-off in the future. ([\#13815](https://github.com/matrix-org/synapse/issues/13815))
- Improve validation for the unspecced, internal-only `_matrix/client/unstable/add_threepid/msisdn/submit_token` endpoint. ([\#13832](https://github.com/matrix-org/synapse/issues/13832))
- Faster remote room joins: record _when_ we first partial-join to a room. ([\#13892](https://github.com/matrix-org/synapse/issues/13892))
- Support a `dir` parameter on the `/relations` endpoint per [MSC3715](https://github.com/matrix-org/matrix-doc/pull/3715). ([\#13920](https://github.com/matrix-org/synapse/issues/13920))
- Ask mail servers receiving emails from Synapse to not send automatic replies (e.g. out-of-office responses). ([\#13957](https://github.com/matrix-org/synapse/issues/13957))

Bugfixes
--------

- Send push notifications for invites received over federation. ([\#13719](https://github.com/matrix-org/synapse/issues/13719), [\#14014](https://github.com/matrix-org/synapse/issues/14014))
- Fix a long-standing bug where typing events would be accepted from remote servers not present in a room. Also fix a bug where incoming typing events would cause other incoming events to get stuck during a fast join. ([\#13830](https://github.com/matrix-org/synapse/issues/13830))
- Fix a bug introduced in Synapse v1.53.0 where the experimental implementation of [MSC3715](https://github.com/matrix-org/matrix-spec-proposals/pull/3715) would give incorrect results when paginating forward. ([\#13840](https://github.com/matrix-org/synapse/issues/13840))
- Fix access token leak to logs from proxy agent. ([\#13855](https://github.com/matrix-org/synapse/issues/13855))
- Fix `have_seen_event` cache not being invalidated after we persist an event which causes inefficiency effects like extra `/state` federation calls. ([\#13863](https://github.com/matrix-org/synapse/issues/13863))
- Faster room joins: Fix a bug introduced in 1.66.0 where an error would be logged when syncing after joining a room. ([\#13872](https://github.com/matrix-org/synapse/issues/13872))
- Fix a bug introduced in 1.66.0 where some required fields in the pushrules sent to clients were not present anymore. Contributed by Nico. ([\#13904](https://github.com/matrix-org/synapse/issues/13904))
- Fix packaging to include `Cargo.lock` in `sdist`. ([\#13909](https://github.com/matrix-org/synapse/issues/13909))
- Fix a long-standing bug where device updates could cause delays sending out to-device messages over federation. ([\#13922](https://github.com/matrix-org/synapse/issues/13922))
- Fix a bug introduced in v1.68.0 where Synapse would require `setuptools_rust` at runtime, even though the package is only required at build time. ([\#13952](https://github.com/matrix-org/synapse/issues/13952))
- Fix a long-standing bug where `POST /_matrix/client/v3/keys/query` requests could result in excessively large SQL queries. ([\#13956](https://github.com/matrix-org/synapse/issues/13956))
- Fix a performance regression in the `get_users_in_room` database query. Introduced in v1.67.0. ([\#13972](https://github.com/matrix-org/synapse/issues/13972))
- Fix a bug introduced in v1.68.0 bug where Rust extension wasn't built in `release` mode when using `poetry install`. ([\#14009](https://github.com/matrix-org/synapse/issues/14009))
- Do not return an unspecified `original_event` field when using the stable `/relations` endpoint. Introduced in Synapse v1.57.0. ([\#14025](https://github.com/matrix-org/synapse/issues/14025))
- Correctly handle a race with device lists when a remote user leaves during a partial join. ([\#13885](https://github.com/matrix-org/synapse/issues/13885))
- Correctly handle sending local device list updates to remote servers during a partial join. ([\#13934](https://github.com/matrix-org/synapse/issues/13934))

Improved Documentation
----------------------

- Add `worker_main_http_uri` for the worker generator bash script. ([\#13772](https://github.com/matrix-org/synapse/issues/13772))
- Update URL for the NixOS module for Synapse. ([\#13818](https://github.com/matrix-org/synapse/issues/13818))
- Fix a mistake in sso_mapping_providers.md: `map_user_attributes` is expected to return `display_name`, not `displayname`. ([\#13836](https://github.com/matrix-org/synapse/issues/13836))
- Fix a cross-link from the registration admin API to the `registration_shared_secret` configuration documentation. ([\#13870](https://github.com/matrix-org/synapse/issues/13870))
- Update the man page for the `hash_password` script to correct the default number of bcrypt rounds performed. ([\#13911](https://github.com/matrix-org/synapse/issues/13911), [\#13930](https://github.com/matrix-org/synapse/issues/13930))
- Emphasize the right reasons when to use `(room_id, event_id)` in a database schema. ([\#13915](https://github.com/matrix-org/synapse/issues/13915))
- Add instruction to contributing guide for running unit tests in parallel. Contributed by @ashfame. ([\#13928](https://github.com/matrix-org/synapse/issues/13928))
- Clarify that the `auto_join_rooms` config option can also be used with Space aliases. ([\#13931](https://github.com/matrix-org/synapse/issues/13931))
- Add some cross references to worker documentation. ([\#13974](https://github.com/matrix-org/synapse/issues/13974))
- Linkify urls in config documentation. ([\#14003](https://github.com/matrix-org/synapse/issues/14003))

Deprecations and Removals
-------------------------

- Remove the `complete_sso_login` method from the Module API which was deprecated in Synapse 1.13.0. ([\#13843](https://github.com/matrix-org/synapse/issues/13843))
- Announce that legacy metric names are deprecated, will be turned off by default in Synapse v1.71.0 and removed altogether in Synapse v1.73.0. See the upgrade notes for more information. ([\#14024](https://github.com/matrix-org/synapse/issues/14024))

Internal Changes
----------------

- Speed up creation of DM rooms. ([\#13487](https://github.com/matrix-org/synapse/issues/13487), [\#13800](https://github.com/matrix-org/synapse/issues/13800))
- Port push rules to using Rust. ([\#13768](https://github.com/matrix-org/synapse/issues/13768), [\#13838](https://github.com/matrix-org/synapse/issues/13838), [\#13889](https://github.com/matrix-org/synapse/issues/13889))
- Optimise get rooms for user calls. Contributed by Nick @ Beeper (@fizzadar). ([\#13787](https://github.com/matrix-org/synapse/issues/13787))
- Update the script which makes full schema dumps. ([\#13792](https://github.com/matrix-org/synapse/issues/13792))
- Use shared methods for cache invalidation when persisting events, remove duplicate codepaths. Contributed by Nick @ Beeper (@fizzadar). ([\#13796](https://github.com/matrix-org/synapse/issues/13796))
- Improve the `synapse.api.auth.Auth` mock used in unit tests. ([\#13809](https://github.com/matrix-org/synapse/issues/13809))
- Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. ([\#13823](https://github.com/matrix-org/synapse/issues/13823))
- Carry IdP Session IDs through user-mapping sessions. ([\#13839](https://github.com/matrix-org/synapse/issues/13839))
- Fix the release script not publishing binary wheels. ([\#13850](https://github.com/matrix-org/synapse/issues/13850))
- Raise issue if complement fails with latest deps. ([\#13859](https://github.com/matrix-org/synapse/issues/13859))
- Correct the comments in the complement dockerfile. ([\#13867](https://github.com/matrix-org/synapse/issues/13867))
- Create a new snapshot of the database schema. ([\#13873](https://github.com/matrix-org/synapse/issues/13873))
- Faster room joins: Send device list updates to most servers in rooms with partial state. ([\#13874](https://github.com/matrix-org/synapse/issues/13874), [\#14013](https://github.com/matrix-org/synapse/issues/14013))
- Add comments to the Prometheus recording rules to make it clear which set of rules you need for Grafana or Prometheus Console. ([\#13876](https://github.com/matrix-org/synapse/issues/13876))
- Only pull relevant backfill points from the database based on the current depth and limit (instead of all) every time we want to `/backfill`. ([\#13879](https://github.com/matrix-org/synapse/issues/13879))
- Faster room joins: Avoid waiting for full state when processing `/keys/changes` requests. ([\#13888](https://github.com/matrix-org/synapse/issues/13888))
- Improve backfill robustness by trying more servers when we get a `4xx` error back. ([\#13890](https://github.com/matrix-org/synapse/issues/13890))
- Fix mypy errors with canonicaljson 1.6.3. ([\#13905](https://github.com/matrix-org/synapse/issues/13905))
- Faster remote room joins: correctly handle remote device list updates during a partial join. ([\#13913](https://github.com/matrix-org/synapse/issues/13913))
- Complement image: propagate SIGTERM to all workers. ([\#13914](https://github.com/matrix-org/synapse/issues/13914))
- Update an innaccurate comment in Synapse's upsert database helper. ([\#13924](https://github.com/matrix-org/synapse/issues/13924))
- Update mypy (0.950 -> 0.981) and mypy-zope (0.3.7 -> 0.3.11). ([\#13925](https://github.com/matrix-org/synapse/issues/13925), [\#13993](https://github.com/matrix-org/synapse/issues/13993))
- Use dedicated `get_local_users_in_room(room_id)` function to find local users when calculating users to copy over during a room upgrade. ([\#13960](https://github.com/matrix-org/synapse/issues/13960))
- Refactor language in user directory `_track_user_joined_room` code to make it more clear that we use both local and remote users. ([\#13966](https://github.com/matrix-org/synapse/issues/13966))
- Revert catch-all exceptions being recorded as event pull attempt failures (only handle what we know about). ([\#13969](https://github.com/matrix-org/synapse/issues/13969))
- Speed up calculating push actions in large rooms. ([\#13973](https://github.com/matrix-org/synapse/issues/13973), [\#13992](https://github.com/matrix-org/synapse/issues/13992))
- Enable update notifications from Github's dependabot. ([\#13976](https://github.com/matrix-org/synapse/issues/13976))
- Prototype a workflow to automatically add changelogs to dependabot PRs. ([\#13998](https://github.com/matrix-org/synapse/issues/13998), [\#14011](https://github.com/matrix-org/synapse/issues/14011), [\#14017](https://github.com/matrix-org/synapse/issues/14017), [\#14021](https://github.com/matrix-org/synapse/issues/14021), [\#14027](https://github.com/matrix-org/synapse/issues/14027))
- Fix type annotations to be compatible with new annotations in development versions of twisted. ([\#14012](https://github.com/matrix-org/synapse/issues/14012))
- Clear out stale entries in `event_push_actions_staging` table. ([\#14020](https://github.com/matrix-org/synapse/issues/14020))
- Bump versions of GitHub actions. ([\#13978](https://github.com/matrix-org/synapse/issues/13978), [\#13979](https://github.com/matrix-org/synapse/issues/13979), [\#13980](https://github.com/matrix-org/synapse/issues/13980), [\#13982](https://github.com/matrix-org/synapse/issues/13982), [\#14015](https://github.com/matrix-org/synapse/issues/14015), [\#14019](https://github.com/matrix-org/synapse/issues/14019), [\#14022](https://github.com/matrix-org/synapse/issues/14022), [\#14023](https://github.com/matrix-org/synapse/issues/14023))
2022-10-04 15:26:37 +01:00
dependabot[bot] 94017e867d
Bump isort from 5.7.0 to 5.10.1 (#13988)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-10-04 13:19:40 +01:00
Sean Quah 471e13a103 1.69.0rc1 2022-10-04 11:18:37 +01:00
David Robertson 1254eb2247
1.68.0 2022-09-27 12:03:58 +01:00
Erik Johnston 6d3b1497c3
Correctly add Cargo.lock to sdsit (#13909) 2022-09-26 15:12:57 +01:00
Erik Johnston 11ebcc8a3a 1.68.0rc2 2022-09-23 09:40:16 +01:00
Erik Johnston 9615e23c29
Fix sdist to include rust build script (#13866)
* Fix sdist to include rust build script

Fixes #13851

* Newsfile
2022-09-22 11:59:23 +01:00
Olivier Wilkinson (reivilibre) e124b24ad9 1.68.0rc1 2022-09-20 11:30:48 +01:00
Andrew Morgan 918c74bfb5
Add a `MXCUri` class to make working with mxc uri's easier. (#13162) 2022-09-15 12:57:16 +00:00
Erik Johnston 9772e362aa Merge branch 'master' into develop 2022-09-13 12:11:53 +01:00
Erik Johnston 4b678b20a2 1.67.0 2022-09-13 09:20:28 +01:00
Erik Johnston c9b7e97355
Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Erik Johnston 3d20115115
Fix trial-olddeps (#13725) 2022-09-06 14:21:55 +00:00
Erik Johnston cdf7fb737b 1.67.0rc1 2022-09-06 09:01:26 +01:00
Sean Quah ef88bc0775 1.66.0 2022-08-31 11:21:09 +01:00
David Robertson 8f6aa015a8
1.66.0rc2 2022-08-30 12:25:44 +01:00
David Robertson f0b23927fc
1.66.0rc1 2022-08-23 09:49:51 +01:00
Andrew Morgan 5ace5d7b15 Merge branch 'master' into develop 2022-08-16 17:32:20 +01:00
Andrew Morgan 738c11729a 1.65.0 2022-08-16 16:52:07 +01:00
David Robertson d642ce4b32
Use Pydantic to systematically validate a first batch of endpoints in `synapse.rest.client.account`. (#13188) 2022-08-15 19:05:57 +00:00
Olivier Wilkinson (reivilibre) 7a25b4302c 1.65.0rc2 2022-08-11 11:40:40 +01:00
Olivier Wilkinson (reivilibre) 827f0669bf 1.65.0rc1 2022-08-09 11:43:30 +01:00
Olivier Wilkinson (reivilibre) 4e80ca2243 1.64.0 2022-08-02 11:04:08 +01:00
Richard van der Hoff 6b4fd8b430 1.64.0rc2 2022-07-29 12:23:13 +01:00
Olivier Wilkinson (reivilibre) 641412decd 1.64.0rc1 2022-07-26 12:12:22 +01:00
Erik Johnston d3995049a8 Merge remote-tracking branch 'origin/master' into develop 2022-07-20 14:59:43 +01:00
Erik Johnston 93740cae57 1.63.1 2022-07-20 13:37:00 +01:00
Brendan Abolivier 47822fd2e8
Merge branch 'master' into develop 2022-07-19 16:14:02 +02:00
Brendan Abolivier 097afd0e0b 1.63.0 2022-07-19 14:43:28 +02:00
Brendan Abolivier 8c60c572f0
Up the dependency on canonicaljson to ^1.5.0 (#13172)
Co-authored-by: David Robertson <davidr@element.io>
2022-07-18 17:30:59 +02:00
Sean Quah 6173d585df 1.63.0rc1 2022-07-12 11:26:25 +01:00
David Robertson cf63d57dce
1.62.0 2022-07-05 11:14:27 +01:00
Andrew Morgan 046d87756b 1.62.0rc3 2022-07-04 16:16:47 +01:00
Andrew Morgan d40b2708cf 1.62.0rc2 2022-07-01 11:42:57 +01:00
Andrew Morgan b210146fd9 1.62.0rc1 2022-06-28 16:42:44 +01:00
Andrew Morgan 6cba6a51af Merge branch 'master' into develop 2022-06-28 15:19:48 +01:00
Andrew Morgan ea10cdbea7 1.61.1 2022-06-28 14:37:35 +01:00
David Robertson f33356e8f8
Use caret (semver bounds) for matrix.org packages (#13082) 2022-06-17 19:07:04 +01:00
Hannes Lerchl 7d99414edf
Replace pyjwt with authlib in `org.matrix.login.jwt` (#13011) 2022-06-15 16:45:16 +00:00
Erik Johnston de334ac183
Add a CI job to check that schema deltas are in the correct folder. (#13063) 2022-06-15 16:27:18 +01:00
Erik Johnston 5d139f578d Merge branch 'release-v1.61' into develop 2022-06-14 12:00:02 +01:00
Erik Johnston d580014e22 1.61.0 2022-06-14 11:44:27 +01:00
Olivier Wilkinson (reivilibre) 00d915b2a7 Merge branch 'master' into develop 2022-06-07 16:59:41 +01:00
David Robertson f30bcbd84a
Fix Synapse git info missing in version strings (#12973) 2022-06-07 15:24:11 +01:00
Olivier Wilkinson (reivilibre) 495fbf5d63 1.61.0rc1 2022-06-07 13:18:02 +01:00
Jacek Kuśnierz 88193f2125
Remove direct refeferences to PyNaCl (use signedjson instead). (#12902) 2022-06-01 07:32:35 -04:00
Erik Johnston 5984ada6bb 1.60.0 2022-05-31 13:41:49 +01:00
Sean Quah e409ab8e92 1.60.0rc2 2022-05-27 11:06:45 +01:00
Sean Quah a670b5cda2 1.60.0rc1 2022-05-24 12:05:33 +01:00
Brendan Abolivier 1aa30f7b3e 1.59.1 2022-05-18 11:41:53 +01:00
David Robertson 5c3d525cad
1.59.0 2022-05-17 10:27:51 +01:00
David Robertson c3b232cb39
1.59.0rc2 2022-05-16 12:52:29 +01:00
David Robertson e5fd23fb6f
1.59.0rc1 2022-05-10 10:45:13 +01:00
David Robertson fa0eab9c8e
Use `ParamSpec` in a few places (#12667) 2022-05-09 10:27:39 +00:00
David Robertson 26c1ad71c5
Use `Concatenate` to annotate `do_execute` (#12666) 2022-05-09 10:28:38 +01:00
David Robertson 0ce2201932
Move `pympler` back into the `all` extras (#12652)
* Move `pympler` back into the `all` extras

Undoes a change I made in #12381. I can't fully remember my reasoning,
but this changed the contents of the debian packages in a backwards
incompatible way. We're not aware of anyone who's been bitten by this,
but we still want to fix it.

To the reviewer: please be convinced that the debian packages will still
contain pympler after this change.

* Debian changelog entry to keep the linter happy
2022-05-07 13:40:58 +01:00