Commit Graph

1349 Commits

Author SHA1 Message Date
reivilibre 67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. (#12881)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-08 09:57:05 +00:00
Andrew Morgan a47636c570
Prevent local quarantined media from being claimed by media retention (#12972) 2022-06-07 10:53:47 +00:00
Jan Christian Grünhage fcd8703508
Allow updating passwords using the admin api without logging out devices (#12952) 2022-06-06 12:10:13 +01:00
Patrick Cloke 7bc08f3201
Remove remaining bits of groups code. (#12936)
* Update worker docs to remove group endpoints.
* Removes an unused parameter to `ApplicationService`.
* Break dependency between media repo and groups.
* Avoid copying `m.room.related_groups` state events during room upgrades.
2022-06-01 09:41:25 -04:00
Andrew Morgan 2fc787c341
Add config options for media retention (#12732) 2022-05-31 16:35:29 +00:00
Erik Johnston 3594f6c1f3 Merge branch 'master' into develop 2022-05-31 14:48:22 +01:00
Brendan Abolivier 8fd87739bf
Fix import in module_api module and docs on the new check_event_for_spam signature (#12918)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-05-31 12:04:53 +02:00
Brendan Abolivier 28989cb301
Add a background job to automatically delete stale devices (#12855)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-27 17:47:32 +02:00
Sean Quah 053ca5f3ca Synapse 1.60.0rc2 (2022-05-27)
==============================
 
 This release of Synapse adds a unique index to the `state_group_edges` table, in
 order to prevent accidentally introducing duplicate information (for example,
 because a database backup was restored multiple times). If your Synapse database
 already has duplicate rows in this table, this could fail with an error and
 require manual remediation.
 
 Additionally, the signature of the `check_event_for_spam` module callback has changed.
 The previous signature has been deprecated and remains working for now. Module authors
 should update their modules to use the new signature where possible.
 
 See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
 for more details.
 
 Features
 --------
 
 - Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))
 
 Bugfixes
 --------
 
 - Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))
 
 Internal Changes
 ----------------
 
 - Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmKQqMcACgkQ+84Kzgcy
 GG9Z2Av+N+b/fvaB3D56UkFqTW/xLmCEyri65njcXU8625bWiLSPM6hssmyJB1FA
 xlc2RBKr8QxlnHRS/v31wDtONC8YZ2O3fyzYPFfY1fF5Ul7Kg3XCzLeUH4/j1/Ar
 5bqriDqaN9FQ/6QJybShXlA4l7lY1Fs0C4P23jDBgqfKjnlToeVLqhVA70dDaFu/
 ir+vVprKCkQI1iqnYXwIxGRmgBzLWGoVqQFGbSI6hugGwXpGIyX7+2I+0v8tI6vA
 SZ99vLFWcvnd6DJTyBhIeV22Ff4qA7eQsyPvSrMETdsaZmrxGlG+t332HNCgplv8
 gv2gUpJL0br++5WTAX+nRc7HpfKo/74vKeTktqPmlvFP8kUOg+PbzmoJFUu21PhA
 rnq5TzgsPHK0dqBhM1RC2vtOiJ5v3ZBqzJJzSRXl6lsFpWxxFmwesEcIDAYS0Nmh
 QoJb7/L8cPCHksHvZM76bzB465tSH9NhuFYZQoLGHcpxa0kYekrdlYasP8U0FU7L
 nF3C0Pgw
 =D3F+
 -----END PGP SIGNATURE-----

Merge tag 'v1.60.0rc2' into develop

Synapse 1.60.0rc2 (2022-05-27)
==============================

This release of Synapse adds a unique index to the `state_group_edges` table, in
order to prevent accidentally introducing duplicate information (for example,
because a database backup was restored multiple times). If your Synapse database
already has duplicate rows in this table, this could fail with an error and
require manual remediation.

Additionally, the signature of the `check_event_for_spam` module callback has changed.
The previous signature has been deprecated and remains working for now. Module authors
should update their modules to use the new signature where possible.

See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
for more details.

Features
--------

- Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))

Bugfixes
--------

- Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))

Internal Changes
----------------

- Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
2022-05-27 12:07:18 +01:00
reivilibre 7b88f5a107
Add an option allowing users to use their password to reauthenticate even though password authentication is disabled. (#12883) 2022-05-27 09:44:51 +00:00
Patrick Cloke 1885ee0113
Remove unstable APIs for /hierarchy. (#12851)
Removes the unstable endpoint as well as a duplicated field
which was modified during stabilization.
2022-05-26 07:10:28 -04:00
Patrick Cloke a8db8c6eba
Remove user-visible groups/communities code (#12553)
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:

* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Nick Mills-Barrett 1f9013ce60
Add the `batch_send` endpoint to generic workers (#12868) 2022-05-25 09:51:07 +00:00
Nick Mills-Barrett 2e5f88b5e6
Add the `/account/whoami` endpoint to generic workers (#12866) 2022-05-25 10:41:41 +01:00
Nick Mills-Barrett b4fab0b14f
Fix incorrect worker-allowed path in documentation (#12867) 2022-05-25 09:20:34 +00:00
Dirk Klimpel 298911555c
Fix typos in documentation (#12863) 2022-05-25 10:14:03 +01:00
Richard van der Hoff 0b3423fd51
contributing_guide.md: fix link to DCO 2022-05-24 11:48:11 +01:00
David Teller 28199e9357
Uniformize spam-checker API, part 2: check_event_for_spam (#12808)
Signed-off-by: David Teller <davidt@element.io>
2022-05-23 17:27:39 +00:00
Jess Porter a608ac847b
add SpamChecker callback for silently dropping inbound federated events (#12744)
Signed-off-by: jesopo <github@lolnerd.net>
2022-05-23 16:36:21 +00:00
Shay eb4aaa1b4b
Add detail to `cache_autotuning` config option documentation (#12776) 2022-05-19 07:47:07 -07:00
Shay ab2a615cfb
Update configuration manual to document size-related suffixes (#12777) 2022-05-19 07:46:33 -07:00
reivilibre 66a5f6c400
Add a unique index to `state_group_edges` to prevent duplicates being accidentally introduced and the consequential impact to performance. (#12687) 2022-05-19 14:16:49 +01:00
Sami Olmari 47619017f9
Add missing user directory search endpoint to the generic worker documentation (#12773)
Signed-off-by: Sami Olmari <sami@olmari.fi>
2022-05-19 12:03:12 +01:00
Richard van der Hoff 5675cebfaa
openid.md: fix some links
docbook doesn't auto-linkify links
2022-05-19 10:28:18 +01:00
Patrick Cloke a1cb05b3e8
Fix federation in demo scripts. (#12783) 2022-05-18 14:49:33 -04:00
Andrew Morgan 0fce474a40
Fix YAML parsing error in `url_preview_accept_language` (#12785) 2022-05-18 17:24:44 +00:00
Sean Quah 3d8839c30c
Add documentation for cancellation of request processing (#12761)
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-05-18 17:56:23 +01:00
Andrew Morgan a167304c8b
Switch the 'Configuration' link in the docs homepage to the config manual (#12748) 2022-05-18 11:29:32 +00:00
Andrew Morgan deca250e3f
Add some documentation around the `rc_invites` option to the config docs (#12759) 2022-05-18 12:21:32 +01:00
David Robertson 0d17357fcd
Suggest using docker when testing against postgres (#12765)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-05-17 18:05:53 +00:00
Andrew Morgan 6d8d1218dd
Fix typo in name of 'run_background_tasks_on' option in config manual (#12749) 2022-05-16 17:31:12 +00:00
Dirk Klimpel 8060034612
Fix typo in listener config (#12742) 2022-05-16 13:50:07 +00:00
Shay cde8af9a49
Add config flags to allow for cache auto-tuning (#12701) 2022-05-13 12:32:39 -07:00
Niklas c9fc2c0d22
Update issuer URL in example OIDC Keycloak config (#12727)
* Update openid.md

Newer versions of keycloak returning a 404 when using the `/auth` prefix.

Related: https://github.com/matrix-org/synapse/issues/12714
2022-05-13 10:15:51 +00:00
Andy Balaam de1e599b9d
add default_power_level_content_override config option. (#12618)
Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2022-05-12 10:41:35 +00:00
Andrew Morgan 409573f6d0
Fix reference to the wrong symbol in the media admin api docs (#12715) 2022-05-12 09:29:37 +01:00
David Robertson d38d242411
Reload cache factors from disk on SIGHUP (#12673) 2022-05-11 13:43:22 +00:00
reivilibre 699192fc1a
Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. (#12654)
Co-authored-by: Shay <hillerys@element.io>
2022-05-10 11:08:45 +01:00
Shay d80a7ab151
Update `replication.md` with info on TCP module structure (#12621) 2022-05-09 14:46:43 -07:00
Dirk Klimpel 615d96ad6e
Update SQL statements in docs for Synapse Admins (#12536) 2022-05-09 14:43:02 -07:00
Eric Eastwood 18d6c18aa1
Fix docs on how to run specific Complement tests after recent `complement.sh` change (#12664) 2022-05-09 10:38:32 +01:00
reivilibre c2d50e9f6c
Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452) 2022-05-06 11:43:53 +01:00
Will Hunt 2d74a8c178
Add `mau_appservice_trial_days` config (#12619)
* Add mau_appservice_trial_days

* Add a test

* Tweaks

* changelog

* Ensure we sync after the delay

* Fix types

* Add config statement

* Fix test

* Reinstate logging that got removed

* Fix feature name
2022-05-04 19:33:26 +01:00
Andrew Morgan 332cce8dcf
Disable device name lookup over federation by default (#12616) 2022-05-04 16:41:40 +01:00
Patrick Cloke ba3fd54bad
Remove unstable/unspecced login types. (#12597)
* `m.login.jwt`, which was never specced and has been deprecated
  since Synapse 1.16.0. (`org.matrix.login.jwt` can be used instead.)
* `uk.half-shot.msc2778.login.application_service`, which was
  stabilized as part of the Matrix spec v1.2 release.
2022-05-04 13:53:21 +00:00
Sean Quah b2df0716bc
Improve logging for cancelled requests (#12587)
Don't log stack traces for cancelled requests and use a custom HTTP
status code of 499.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-04 13:38:55 +01:00
Richard van der Hoff 8d156ec0ba
Remove special-case for `twisted` logger (#12589)
This was originally added when we first added a `MemoryHandler` to the default
log config back in https://github.com/matrix-org/synapse/pull/8040, to ensure
that we didn't explode with an infinite loop if there was an error formatting
the logs.

Since then, we made additional improvements to logging which make this
workaround redundant. In particular:

 * we no longer attempt to log un-UTF8-decodable byte sequences, which were the
   most likely cause of an error in the first place.

 * https://github.com/matrix-org/synapse/pull/8268 ensures that in the unlikely
   case that there *is* an error, it won't cause an infinite loop.
2022-04-29 22:05:18 +01:00
Will Hunt e8d1ec0e92
Add option to enable token registration without requiring 3pids (#12526) 2022-04-27 12:57:53 +00:00
Shay e75c7e3b6d
Add a table of contents to config manual (#12527)
* Update config_documentation.md
2022-04-25 11:43:59 -07:00
Sami Olmari 813d728d09
Correct typo in user_admin_api.md device deletion JSON (#12533)
Signed-off-by: Sami Olmari <sami@olmari.fi>
2022-04-25 12:39:15 +00:00
villepeh d9b71410c2
Add HAProxy delegation example to docs (#12501)
Signed-off-by: Ville Petteri Huh
2022-04-25 13:18:18 +01:00
Dirk Klimpel a36a38b1ca
Add some example configurations for worker (#12492)
Signed-off-by: Dirk Klimpel <dirk@klimpel.org>
2022-04-25 13:17:03 +01:00
Richard van der Hoff f46b223354
turn-howto: fix some links 2022-04-22 14:23:40 +01:00
David Robertson ecef741add
Recommend poetry in docs (#12475)
* Recommend poetry in docs

- readme
- contributor guide
- upgrade notes
- new dev cheat sheet for poetry

Co-authored-by: Shay <hillerys@element.io>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-04-20 15:18:21 +01:00
Olivier Wilkinson (reivilibre) 0921d93dcd Merge branch 'master' into develop 2022-04-19 13:28:28 +01:00
Richard van der Hoff 9f512ff537
fix typo in upgrade notes 2022-04-19 11:25:12 +01:00
Shay 3c758d9808
Add a manual documenting config file options (#12368) 2022-04-18 11:32:30 -07:00
reivilibre 7efddbebef
Update documentation to reflect that `run_background_tasks_on` is no longer experimental. (#12451)
* Background workers aren't experimental anymore

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Stream writers aren't experimental either
2022-04-14 15:25:22 +01:00
Dirk Klimpel a743f7d33e
Replace `federation_reader` with `generic_worker` in docs (#12457) 2022-04-14 13:09:07 +01:00
David Robertson 535a689cfc
Reintroduce the lint targets in the linter script (#12455) 2022-04-14 11:33:06 +01:00
Andrew Morgan 0922462fc7
docs: Don't render the table of contents on the print page (#12340) 2022-04-13 08:27:51 -07:00
Erik Johnston a468768104 Merge branch 'release-v1.57' into develop 2022-04-12 15:28:41 +01:00
Patrick Cloke 9535fd0f9c
Disable groups/communities by default. (#12344)
This disables the endpoints (and sync response fields) for
groups/communities by default.
2022-04-12 10:20:46 -04:00
Richard van der Hoff 643c0c50c1
Update UPGRADE notes to include more details about v1.57 upgrade failure mode (#12448) 2022-04-12 14:56:05 +01:00
Erik Johnston e31d06f6f0 Split changelogs by year 2022-04-12 13:54:36 +01:00
David Robertson dd5cc37aa4
Stop maintaining a list of lint targets in `lint.sh` (#12420) 2022-04-08 14:36:09 +01:00
Eric Eastwood 350062661c
Clarify that we mark as outliers because we don't have any state for them (#12345)
As discussed at https://github.com/matrix-org/synapse/pull/12179#discussion_r837263852
2022-04-06 16:34:33 -05:00
Patrick Cloke 6902e9ff2b
Remove docs for converting a legacy structured logging config. (#12392)
And update the upgrade notes to point at old versions of the documentation.
2022-04-06 13:41:14 -04:00
Sean Quah ab3fdcf960 Merge branch 'master' into develop 2022-04-05 15:14:53 +01:00
reivilibre 708d88b1a2
Allow specifying the Postgres database's port when running unit tests with Postgres. (#12376) 2022-04-05 12:44:48 +01:00
Shay f608e6c8cf
Update broken link in `workers.md` (#12369) 2022-04-04 17:40:31 -07:00
Nick Mills-Barrett 993d90f82b
Use a sequence to generate AS transaction IDs, drop `last_txn` AS state (#12209)
Switching to a sequence means there's no need to track `last_txn` on the
AS state table to generate new TXN IDs. This also means that there is
no longer contention between the AS scheduler and AS handler on updates
to the `application_services_state` table, which will prevent serialization
errors during the complete AS txn transaction.
2022-04-01 13:33:25 +00:00
Brendan Abolivier e4409301ba
Add a module callback to react to account data changes (#12327)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-04-01 11:22:48 +02:00
Brendan Abolivier 5e88143dff
Add a callback to react to 3PID associations (#12302) 2022-03-31 18:27:21 +02:00
David Robertson 15cdcf8f30
Remove `dockerfile-pgtests` (#12336) 2022-03-31 11:30:07 +01:00
reivilibre 2fc15ac718
Flesh out documentation for running SyTest against Synapse, including use of Postgres and worker mode. (#12271)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-30 12:55:05 +00:00
Brendan Abolivier 437a8ed9ef
Add a configuration to exclude rooms from sync response (#12310) 2022-03-30 09:43:04 +00:00
Andrew Morgan 013f3f5e44
Move warning of open registration to v1.56 upgrade notes (#12322) 2022-03-29 14:12:41 +01:00
villepeh a4643a685c
HAProxy guide update (#12279)
HAproxy reverse proxy guide update to stop sending IPv4-mapped address to HS.
2022-03-28 10:48:18 +00:00
Shay 3c41d87b67
Add restrictions by default to open registration in Synapse (#12091) 2022-03-25 10:11:01 -07:00
IronTooch 61aae18d45
Authentik OpenID minor doc update (#12275) 2022-03-25 12:40:10 +00:00
Brendan Abolivier 5859e2fe0c
Mention the new behaviour on unsafe database locale in the upgrade notes (#12288)
Co-authored-by: Shay <hillerys@element.io>
2022-03-25 10:56:18 +01:00
Shay e78d4f61fc
Refuse to start if DB has an unsafe locale (#12262) 2022-03-23 10:23:05 -07:00
Jonathan de Jong c5776780f0
Remove mutual_rooms `update_user_directory` check, and add extra documentation (#12038)
Resolves #10339
2022-03-23 12:47:07 +00:00
Patrick Cloke 2177e356bc
Sync more worker regexes in the documentation. (#12243) 2022-03-18 12:51:27 -04:00
David Robertson 872dbb0181
Correct `check_username_for_spam` annotations and docs (#12246)
* Formally type the UserProfile in user searches
* export UserProfile in synapse.module_api
* Update docs

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-03-18 13:51:41 +00:00
Patrick Cloke 54f674f7a9
Deprecate the groups/communities endpoints and add an experimental configuration flag. (#12200) 2022-03-12 13:23:37 -05:00
Shay ef3619e61d
Add config settings for background update parameters (#11980) 2022-03-11 10:46:45 -08:00
Brendan Abolivier e6a106fd5e
Implement a Jinja2 filter to extract localparts from email addresses (#12212) 2022-03-11 15:15:11 +00:00
Brendan Abolivier 003cc6910a
Update the SSO username picker template to comply with SIWA guidelines (#12210)
Fixes https://github.com/matrix-org/synapse/issues/12205
2022-03-11 13:20:00 +00:00
Andrew Morgan 3b12f6d61b
Note that contributors can sign off privately (#12204)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-11 11:10:20 +00:00
~creme 7577894bec
Document that most streams can only have a single writer. (#12196)
This includes the `typing`, `to_device`, `account_data`, `receipts`, and `presence`
streams (really anything except the `events` stream).
2022-03-10 18:15:19 +00:00
reivilibre 72e7f1c420
Remove workaround introduced in Synapse v1.50.0rc1 for Mjolnir compatibility. Breaks compatibility with Mjolnir v1.3.1 and earlier. (#11700) 2022-03-10 15:53:23 +00:00
Richard van der Hoff 52a947dc46
Updates to the Room DAG concepts development document (#12179)
Some stuff that came up while we were talking about #12173.
2022-03-10 15:18:31 +00:00
Will Hunt 15382b1afa
Add third_party module callbacks to check if a user can delete a room and deactivate a user (#12028)
* Add check_can_deactivate_user

* Add check_can_shutdown_rooms

* Documentation

* callbacks, not functions

* Various suggested tweaks

* Add tests for test_check_can_shutdown_room and test_check_can_deactivate_user

* Update check_can_deactivate_user to not take a Requester

* Fix check_can_shutdown_room docs

* Renegade and use `by_admin` instead of `admin_user_id`

* fix lint

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-03-09 18:23:57 +00:00
Patrick Cloke 9a0172d49f
Clean-up demo scripts & documentation (#12143)
* Rewrites the demo documentation to be clearer, accurate, and moves it to our documentation tree.
* Improvements to the demo scripts:
	* `clean.sh` now runs `stop.sh` first to avoid zombie processes.
	* Uses more modern Synapse configuration (and removes some obsolete configuration).
	* Consistently use the HTTP ports for server name, etc.
	* Remove the `demo/etc` directory and place everything into the `demo/808x` directories.
2022-03-08 15:02:59 -05:00
David Robertson 4aeb00ca20
Move synctl into `synapse._scripts` and expose as an entrypoint (#12140) 2022-03-04 11:58:49 +00:00
David Robertson 1fbe0316a9
Add suffices to scripts in scripts-dev (#12137)
* Rename scripts-dev to have suffices

* Update references to `scripts-dev`

* Changelog

* These scripts don't pass mypy
2022-03-02 18:00:26 +00:00
Andrew Morgan b4461e7d8a
Enable complexity checking in complexity checking docs example (#11998) 2022-03-02 16:11:16 +00:00
David Robertson f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points (#12118)
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-03-02 13:00:16 +00:00