Mirrors
/
synapse-old
mirror of https://github.com/matrix-org/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,252 Commits 842 Branches 671 Tags 438 MiB
Commit Graph

1044 Commits

Author SHA1 Message Date
David Teller f14428b25c
Allow spam-checker modules to be provide async methods. (#8890) 
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
 2020-12-11 14:05:15 -05:00
Dirk Klimpel 0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Dirk Klimpel a5f7aff5e5
Deprecate Shutdown Room and Purge Room Admin API (#8829) 
Deprecate both APIs in favour of the Delete Room API.

Related: #8663 and #8810
 2020-12-10 11:42:48 +00:00
Patrick Cloke 344ab0b53a
Default to blacklisting reserved IP ranges and add a whitelist. (#8870) 
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
 2020-12-09 13:56:06 -05:00
Dirk Klimpel 43bf3c5178
Combine related media admin API docs (#8839) 
Related: #8810
Also a few small improvements.

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-12-09 16:19:57 +00:00
Richard van der Hoff 025fa06fc7
Clarify config template comments (#8891) 2020-12-08 14:03:08 +00:00
Patrick Cloke 96358cb424
Add authentication to replication endpoints. (#8853) 
Authentication is done by checking a shared secret provided
in the Synapse configuration file.
 2020-12-04 10:56:28 -05:00
Patrick Cloke 112f6bd49e Synapse 1.24.0rc2 (2020-12-04) 
==============================
 
 Bugfixes
 --------
 
 - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))
 
 Internal Changes
 ----------------
 
 - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/KQ6MACgkQM/xY9qcR
 MEjqOg/9Fd4Ow0QdnK3O6J0bmXP6+g2GfjJCAKlYlF/nwX1ez9jZVed00D2h9/hD
 YCUJCEv+XZpbv5F4usiYwoLGithbueo7AKEOQJQMEuQBHU1/E6TJ9iiKR3lTBNNw
 ccuyMMqqEatYxoa4DIO/lrwm2mhwRNdMPt0r1DPlML13bb/TVbeXXykbOCyZSKm1
 YuXpxYDngRPL9o46I77l7/KNTSpYYeZi2qwp2orl6GBfa0KSvR2Oi6uohKYzuoHC
 +XLvHSFMpHAZkaZsIMxU5hRVH6jtkb/D37CABuKQsdlvHcooGK4Sdt6aoUyhPtoh
 6YwajSazWKEJ/9YO5JV22qkWFPi9T6s/lPZfjOSu/euwMw7DDQJ/8t8Sm6N7sJDS
 A/5FctOAd1jd9BiAaqkHB2zLHZSomIHQpJJfDyE3UgAGQQ6fm2Wg7SkLByDx/MYC
 4D74dBuVBPENM2VxNoyjHbpVSeQi8t0RgWb44/PUAOtpwm3f3eVRuS8zY6uR2bZr
 YZUbJylqKhRW6dBdZH+EzVolRGvBgLkc27IzRCQ3kGUEo2Wem05s6MuKMvLxoIvN
 NHLWNClUWITT8FgkfFP3c6CK40dwIGJzYryI+71qPW/R7snf2fazuoHlH1pAmFh0
 E0/nBMTw1deixOYXbG1ah63AhJ1NjdTfimDsKmWZLqVc6g7g4d4=
 =is1Y
 -----END PGP SIGNATURE-----

Merge tag 'v1.24.0rc2' into develop

Synapse 1.24.0rc2 (2020-12-04)
==============================

Bugfixes
--------

- Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))

Internal Changes
----------------

- Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
 2020-12-04 09:14:31 -05:00
Patrick Cloke 22c6c19f91
Fix a regression that mapping providers should be able to redirect users. (#8878) 
This was broken in #8801.
 2020-12-04 08:25:15 -05:00
Richard van der Hoff 6e4f71c057
Fix a buglet in the SAML username mapping provider doc (#8873) 
the constructor is called with a `module_api`.
 2020-12-04 10:14:15 +00:00
Patrick Cloke 30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821) 
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
 2020-12-02 11:09:24 -05:00
Andrew Morgan d1be293f00 Fix typo in password_auth_providers doc 
A word got removed accidentally in 83434df381.
 2020-12-01 10:34:52 +00:00
Andrew Morgan 17fa58bdd1
Add a config option to change whether unread push notification counts are per-message or per-room (#8820) 
This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55).

However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).
 2020-11-30 18:43:54 +00:00
Richard van der Hoff a090b86209
Add `force_purge` option to delete-room admin api. (#8843) 2020-11-30 16:48:12 +00:00
Dirk Klimpel 14f81a6d24
Improve documentation how to configure prometheus for workers (#8822) 2020-11-26 10:42:55 +00:00
Dirk Klimpel 3f0ff53158
Remove deprecated `/_matrix/client/*/admin` endpoints (#8785) 
These are now only available via `/_synapse/admin/v1`.
 2020-11-25 16:26:11 -05:00
Andrew Morgan 2b110dda2a
Fix the formatting of push config section (#8818) 
This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).
 2020-11-25 21:02:53 +00:00
Patrick Cloke 4fd222ad70
Support trying multiple localparts for OpenID Connect. (#8801) 
Abstracts the SAML and OpenID Connect code which attempts to regenerate
the localpart of a matrix ID if it is already in use.
 2020-11-25 10:04:22 -05:00
Dirk Klimpel b08dc7effe
Clarify documentation of the admin list media API (#8795) 
Clarify that the list media API only shows media from unencrypted events.
 2020-11-24 09:04:51 -05:00
Richard van der Hoff e3d7806704
Update turn-howto (#8779) 
Some hopefully-useful notes on setting up a turnserver.
 2020-11-24 12:52:22 +00:00
Patrick Cloke 79bfe966e0
Improve error checking for OIDC/SAML mapping providers (#8774) 
Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.
 2020-11-19 14:25:17 -05:00
Ben Banfield-Zanin 53a6f5ddf0
SAML: Allow specifying the IdP entityid to use. (#8630) 
If the SAML metadata includes multiple IdPs it is necessary to
specify which IdP to redirect users to for authentication.
 2020-11-19 09:57:13 -05:00
Marcus Schopen d356588339
SAML: Document allowing a clock/time difference from IdP (#8731) 
Updates the sample configuration with the pysaml2 configuration for
accepting clock skew/drift between the homeserver and IdP.
 2020-11-18 07:36:28 -05:00
chagai95 e487d9fabc
a comma too much (#8771) 
Signed-off-by: Chagai Friedlander chagai95@gmail.com
 2020-11-17 14:13:56 +00:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Adrian Wannenmacher f1de4bb58b
Clarify the usecase for an msisdn delegate (#8734) 
Signed-off-by: Adrian Wannenmacher <tfld@tfld.dev>
 2020-11-14 23:09:36 +00:00
Dirk Klimpel 023f791143
Migrate documentation `docs/admin_api/event_reports` to markdown (#8742) 
Related to #8714. `event_reports.rst` was introduced in Synapse 1.21.0.
 2020-11-13 13:57:55 +00:00
Marcus Schopen 68fc0dcb5a
SAML: add <mdui:UIInfo> element examples (#8718) 
add some mdui:UIInfo element examples for saml2_config in homeserver.yaml
 2020-11-13 12:07:50 +00:00
Marcus Schopen c059413001
Notes on SSO logins and media_repository worker (#8701) 
If SSO login is used (e.g. SAML) in a multi worker setup, it should be mentioned that currently all SAML logins must run on the same worker, see https://github.com/matrix-org/synapse/issues/7530

Also, if you are using different ports (for example 443 and 8448) in a reverse proxy for client and federation, the path `/_matrix/media` on the client and federation port must point to the listener of the `media_repository` worker, otherwise you'll get a 404 on the federation port for the path `/_matrix/media`, if a remote server is trying to get the media object on federation port, see https://github.com/matrix-org/synapse/issues/8695
 2020-11-06 14:33:07 +00:00
Dirk Klimpel c3119d1536
Add an admin API for users' media statistics (#8700) 
Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-11-05 18:59:12 +00:00
Dirk Klimpel e4676bd877
Add `displayname` to Shared-Secret Registration for admins (#8722) 
Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`
 2020-11-05 13:55:45 +00:00
Dirk Klimpel 4fda58ddd2
Remove the "draft" status of the Room Details Admin API (#8702) 
Fixes #8550
 2020-11-03 12:48:25 +00:00
Erik Johnston 4b09b7438e
Document how to set up multiple event persisters (#8706) 2020-11-03 10:27:11 +00:00
Matthew Hodgson d04c2d19b3 grammar 2020-11-02 21:22:36 +00:00
Matthew Hodgson 11fd90a2b7 typo 2020-11-02 13:33:56 +00:00
Andrew Morgan 26b46796ea Fix typos in systemd-with-workers doc 2020-11-02 12:56:16 +00:00
Andrew Morgan 305545682d Fix typo in workers doc 2020-11-02 12:36:18 +00:00
Patrick Cloke 8f1aefa694
Improve the sample config for SSO (OIDC, SAML, and CAS). (#8635) 2020-10-30 10:01:59 -04:00
Patrick Cloke 00b24aa545
Support generating structured logs in addition to standard logs. (#8607) 
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
 2020-10-29 07:27:37 -04:00
Dirk Klimpel 2239813278
Add an admin APIs to allow server admins to list users' pushers (#8610) 
Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers
 2020-10-28 15:02:42 +00:00
Michael Kaye f49c2093b5
Cross-link documentation to the prometheus recording rules. (#8667) 2020-10-27 15:29:50 -04:00
Dirk Klimpel 9b7c28283a
Add admin API to list users' local media (#8647) 
Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.
 2020-10-27 14:12:31 +00:00
Dirk Klimpel 66e6801c3e
Split admin API for reported events into a detail and a list view (#8539) 
Split admin API for reported events in detail und list view.
API was introduced with #8217 in synapse v.1.21.0.

It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview.
The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`.
It is similar to room and users API.

It is a kind of regression in `GET /_synapse/admin/v1/event_reports`.  `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec).

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-10-26 18:16:37 +00:00
Peter Krantz 6c9ab61df5
Added basic instructions for Azure AD to OpenId documentation (#8582) 
Signed-off-by: Peter Krantz peter.krantz@gmail.com
 2020-10-26 17:49:55 +00:00
Dirk Klimpel 49d72dea2a
Add an admin api to delete local media. (#8519) 
Related to: #6459, #3479

Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete
a single file from server.
 2020-10-26 17:02:28 +00:00
Andrew Morgan f6a3859a73
Fix filepath of Dex example config (#8657) 2020-10-26 16:53:11 +00:00
Andrew Morgan 73d8209694
Correct the package name in OpenID Connect install instructions (#8634) 
The OpenID Connect install instructions suggested installing `synapse[oidc]`, but our PyPI package is called `matrix-synapse`.
 2020-10-26 14:45:33 +00:00
Dirk Klimpel 913f8a06e4
Add field `total` to device list in admin API (#8644) 2020-10-26 14:07:51 +00:00
Patrick Cloke 34a5696f93
Fix typos and spelling errors. (#8639) 2020-10-23 12:38:40 -04:00
Andrew Morgan 4fb7a68a65 Correct the package name in authlib install instructions 2020-10-22 18:25:58 +01:00
Christopher May-Townsend 1cf4a68108
Add note to manhole.md about bind_address when using with docker (#8526) 
Signed-off-by: Christopher May-Townsend <chris@maytownsend.co.uk>
 2020-10-14 15:28:59 +01:00
Brendan Abolivier 9e66f3761c
Update documentation on retention policies limits (#8529) 
* Update documentation on retention policies limits

Document the changes from https://github.com/matrix-org/synapse/pull/8104
 2020-10-14 15:00:49 +01:00
Erik Johnston 8de3703d21
Make event persisters periodically announce position over replication. (#8499) 
Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress.

This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.
 2020-10-12 15:51:41 +01:00
Mateusz Przybyłowicz ca2db5dd0c
Increase default max_upload_size from 10M to 50M (#8502) 
Signed-off-by: Mateusz Przybyłowicz <uamfhq@gmail.com>
 2020-10-09 16:58:23 +01:00
Richard van der Hoff 4f0637346a
Combine `SpamCheckerApi` with the more generic `ModuleApi`. (#8464) 
Lots of different module apis is not easy to maintain.

Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.
 2020-10-07 12:03:26 +01:00
Andrew Morgan 01f82bfe32
Remove docs/sphinx and related references (#8480) 
https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...
 2020-10-07 11:45:31 +01:00
Richard van der Hoff 785437dc0d
Update default room version to 6 (#8461) 
Per https://github.com/matrix-org/matrix-doc/pull/2788
 2020-10-05 21:40:51 +01:00
Patrick Cloke f64c6aae68
Update manhole documentation for async/await. (#8462) 2020-10-05 09:40:19 -04:00
Patrick Cloke 62894673e6
Allow background tasks to be run on a separate worker. (#8369) 2020-10-02 08:23:15 -04:00
BBBSnowball 05ee048f2c
Add config option for always using "userinfo endpoint" for OIDC (#7658) 
This allows for connecting to certain IdPs, e.g. GitLab.
 2020-10-01 13:54:35 -04:00
Richard van der Hoff c1ef579b63
Add prometheus metrics to track federation delays (#8430) 
Add a pair of federation metrics to track the delays in sending PDUs to/from 
particular servers.
 2020-10-01 11:09:12 +01:00
Patrick Cloke 8b40843392
Allow additional SSO properties to be passed to the client (#8413) 2020-09-30 13:02:43 -04:00
Aaron Raimist 8238b55e08
Update description of server_name config option (#8415) 2020-09-29 13:50:25 -04:00
Erik Johnston bd380d942f
Add checks for postgres sequence consistency (#8402) 2020-09-28 18:00:30 +01:00
Tdxdxoz abd04b6af0
Allow existing users to login via OpenID Connect. (#8345) 
Co-authored-by: Benjamin Koch <bbbsnowball@gmail.com>

This adds configuration flags that will match a user to pre-existing users
when logging in via OpenID Connect. This is useful when switching to
an existing SSO system.
 2020-09-25 07:01:45 -04:00
Julian Fietkau a4e63e5a47
Add note to reverse_proxy.md about disabling Apache's mod_security2 (#8375) 
This change adds a note and a few lines of configuration settings for Apache users to disable ModSecurity for Synapse's virtual hosts. With ModSecurity enabled and running with its default settings, Matrix clients are unable to send chat messages through the Synapse installation. With this change, ModSecurity can be disabled only for the Synapse virtual hosts.
 2020-09-23 11:14:08 +01:00
Dirk Klimpel 4da01f9c61
Admin API for reported events (#8217) 
Add an admin API to read entries of table `event_reports`. API: `GET /_synapse/admin/v1/event_reports`
 2020-09-22 18:15:04 +01:00
Dirk Klimpel d688b4bafc
Admin API for querying rooms where a user is a member (#8306) 
Add a new admin API `GET /_synapse/admin/v1/users/<user_id>/joined_rooms` to
list all rooms where a user is a member.
 2020-09-18 15:26:36 +01:00
Tulir Asokan b82d68c0bd
Add the topic and avatar to the room details admin API (#8305) 2020-09-14 10:07:04 -04:00
Patrick Cloke 6605470bfb
Improve SAML error messages (#8248) 2020-09-14 09:05:36 -04:00
Patrick Cloke a9dbe98ef9 Synapse 1.20.0rc3 (2020-09-11) 
==============================
 
 Bugfixes
 --------
 
 - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9bbA8ACgkQM/xY9qcR
 MEjjJg//ZGcvPLr8y3F5JXptcjWA3AFja3DgztBA3uNFoDrwVhR9m3+F4rL/Fbat
 gnaccWsKIXOjZw5WOThQEUIfYMKmHEsaDl7xtlUPC4ylmSV+31ypu1KiRSxND2kT
 TmOupvqw+4/E4RajpYX6WT3e2oQUpIcBAKsUyZL3ZXECR/yNYrA6w2w7i4wrv3cU
 QDzBFrCcr5aJql7VUc88BlaZUG6xom2/kkPtjmO6imPnPGBHLN22uPU7zQ4n7Vgi
 Nrg45v5WHVCx57WoXqAEap1zdKgoRCna1x0NkqYh0OAXq1l6aVwlf/Pdynt91f2x
 yyaMYjMskjyatHlzONMV4kz0w03dUrGJXiAx2ldEDd32SKBUxLJ/CYtNEdoXZ4Mm
 o0OJfDbaSJwlqK7FhdZJE3kCSkUvlmVasDErXfQRDVHy5sx9Ufr4P0kkaZyXDys1
 UAWFcCJwy9dgFR2679PoXk1s/gHf10wuk6FOs8ESbcJJOvCljxWrOIqKNzSUL3HQ
 Hj5V1zHYXB6HtgiI7tbKxSn/d2uajHI9b8LVOkyV9RKsb1DGGRgNQQ5Kz4Zud7dC
 vuQ2jVw7JIo41W3QIXvcL1KdQeje2nwYwuwNREZaEDZJsGLjxaJ6dNyBVgkmYYFP
 gxhplSOSsFjQRGEsBrc2utWxDnLVGPHEAt7iHVSqjwVUyARvxjo=
 =e2T1
 -----END PGP SIGNATURE-----

Merge tag 'v1.20.0rc3' into develop

Synapse 1.20.0rc3 (2020-09-11)
==============================

Bugfixes
--------

- Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
 2020-09-11 08:30:36 -04:00
Andrew Morgan 95d869c357
Add /_synapse/client to the reverse proxy docs (#8227) 
This PR adds a information about forwarding `/_synapse/client` endpoints through your reverse proxy. The first of these endpoints are introduced in https://github.com/matrix-org/synapse/pull/8004.
 2020-09-10 13:26:34 +01:00
Andrew Morgan 192e98111d
Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290) 2020-09-10 13:08:08 +01:00
Andrew Morgan a3a90ee031
Show a confirmation page during user password reset (#8004) 
This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset.

This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.
 2020-09-10 11:45:12 +01:00
Andrew Morgan 094896a69d
Add a config option for validating 'next_link' parameters against a domain whitelist (#8275) 
This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285

They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality.

This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint.

This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.
 2020-09-08 16:03:09 +01:00
Richard van der Hoff ad28030c12
Systemd docs: configure workers to start after main process. (#8276) 2020-09-08 10:57:43 +01:00
Will Hunt b257c788c0
Add /user/{user_id}/shared_rooms/ api (#7785) 
* Add shared_rooms api

* Add changelog

* Add .

* Wrap response in {"rooms": }

* linting

* Add unstable_features key

* Remove options from isort that aren't part of 5.x

`-y` and `-rc` are now default behaviour and no longer exist.

`dont-skip` is no longer required

https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020

* Update imports to make isort happy

* Add changelog

* Update tox.ini file with correct invocation

* fix linting again for isort

* Vendor prefix unstable API

* Fix to match spec

* import Codes

* import Codes

* Use FORBIDDEN

* Update changelog.d/7785.feature

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Implement get_shared_rooms_for_users

* a comma

* trailing whitespace

* Handle the easy feedback

* Switch to using runInteraction

* Add tests

* Feedback

* Seperate unstable endpoint from v2

* Add upgrade node

* a line

* Fix style by adding a blank line at EOF.

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Tulir Asokan <tulir@maunium.net>

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update UPGRADE.rst

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Fix UPGRADE/CHANGELOG unstable paths

unstable unstable unstable

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
 2020-09-02 13:18:40 +01:00
Brendan Abolivier b5133dd97f
Explain better what GDPR-erased means (#8189) 
Fixes https://github.com/matrix-org/synapse/issues/8185
 2020-09-01 16:31:59 +01:00
Andrew Morgan 74bf8d4d06
Wording fixes to 'name' user admin api filter (#8163) 
Some fixes to wording I noticed after merging #7377.
 2020-08-25 15:03:24 +01:00
Andrew Morgan 79ac619403 Fix missing double-backtick in RST document 2020-08-25 14:24:06 +01:00
Manuel Stahl 97962ad17b
Search in columns 'name' and 'displayname' in the admin users endpoint (#7377) 
* Search in columns 'name' and 'displayname' in the admin users endpoint

Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
 2020-08-25 14:18:14 +01:00
Brendan Abolivier 420484a334
Allow capping a room's retention policy (#8104) 2020-08-24 18:21:04 +01:00
Ryan Cole cfeb37f039
Updated docs: Added note about missing 308 redirect support. (#8120) 
* Updated docs: Added note about missing 308 redirect support.

* Added changelog
 2020-08-19 12:26:50 +01:00
Patrick Cloke acfb7c3b5d
Add a link to the matrix-synapse-rest-password-provider. (#8111) 2020-08-18 09:54:35 -04:00
Andrew Morgan e04e465b4d
Use the default templates when a custom template file cannot be found (#8037) 
Fixes https://github.com/matrix-org/synapse/issues/6583
 2020-08-17 17:05:00 +01:00
Erik Johnston 0304ad0c3d Move setting of Filter into code. 
We do this to prevent foot guns. The default config uses a MemoryFilter,
but users are free to change to logging to files directly. If they do
then they have to ensure to set the `filters: [context]` on the right
handler, otherwise records get written with the wrong context.

Instead we move the logic to happen when we generate a record, which is
when we *log* rather than *handle*.

(It's possible to add filters to loggers in the config, however they
don't apply to descendant loggers and so they have to be manually set on
*every* logger used in the code base)
 2020-08-11 18:10:46 +01:00
Erik Johnston db131b6b22
Change the default log config to reduce disk I/O and storage (#8040) 
* Change default log config to buffer by default.

This batches up writes to the filesystem, which is more efficient for
disk I/O. This means that it can take some time for logs to get written
to disk. Note that ERROR logs (and above) immediately flush the buffer.

This only effects new installs, as we only write the log config if
started with `--generate-config` (in the same way we do for generating
signing keys).

* Default to keeping last 4 days of logs.

This hopefully reduces the amount of logs kept for new servers. Keeping
the last 1GB of logs is likely overkill for new servers, but equally may
not be enough for busy ones.

Instead, we keep the last four days worth of logs, enough so that admins
can investigate any problems that happened over e.g. a long weekend.
 2020-08-11 18:09:46 +01:00
Richard van der Hoff 0cb169900e
Implement login blocking based on SAML attributes (#8052) 
Hopefully this mostly speaks for itself. I also did a bit of cleaning up of the
error handling.

Fixes #8047
 2020-08-11 16:08:10 +01:00
Travis Ralston 1048ed2afa
Clarify that undoing a shutdown might not be possible (#8010) 2020-08-07 17:16:24 +01:00
Erik Johnston 7620912d84
Add health check endpoint (#8048) 2020-08-07 14:21:24 +01:00
Erik Johnston 079bc3c8e3
Fixup worker doc (again) (#8000) 2020-08-06 10:35:59 +01:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Erik Johnston faba873d4b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins 2020-07-31 15:07:01 +01:00
Erik Johnston 18de00adb4 Add ratelimiting on joins 2020-07-31 15:06:56 +01:00
Travis Ralston e2a4ba6f9b
Add docs for undoing room shutdowns (#7998) 
These docs were tested successfully in production by a customer, so it's probably fine.
 2020-07-31 04:41:44 +01:00
Stuart Mumford 6d4b790021
Update workers docs (#7990) 2020-07-30 17:30:11 +01:00
Erik Johnston 606805bf06
Fix typo in docs/workers.md (#7992) 2020-07-30 16:28:36 +01:00
Olivier Wilkinson (reivilibre) 3aa36b782c Merge branch 'master' into develop 2020-07-30 15:18:36 +01:00
Erik Johnston 2c1b9d6763
Update worker docs with recent enhancements (#7969) 2020-07-29 23:22:13 +01:00
Aaron Raimist 2184f61fae
Various improvements to the docs (#7899) 2020-07-29 10:35:44 -04:00
Dirk Klimpel e866e3b896
Add an option to disable purge in delete room admin API (#7964) 
Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete```
Fixes: #3761

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-07-28 20:08:23 +01:00
lugino-emeritus 3857de2194
Option to allow server admins to join complex rooms (#7902) 
Fixes #7901.

Signed-off-by: Niklas Tittjung <nik_t.01@web.de>
 2020-07-28 13:41:44 +01:00
Erik Johnston aaf9ce72a0
Fix typo in metrics docs (#7966) 2020-07-28 10:03:18 +01:00
Patrick Cloke 83434df381
Update the auth providers to be async. (#7935) 2020-07-23 15:45:39 -04:00
Brendan Abolivier 55f2617f8c
Update the dates for ACME v1 EOL 
As per https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430
 2020-07-22 16:18:40 +01:00
Adrian 64d2280299
Fix a typo in the sample config. (#7890) 2020-07-20 13:42:52 -04:00
Andrew Morgan 5ecf98f59e
Change sample config's postgres user to synapse_user (#7889) 
The [postgres setup docs](https://github.com/matrix-org/synapse/blob/develop/docs/postgres.md#set-up-database) recommend setting up your database with user `synapse_user`.

However, uncommenting the postgres defaults in the sample config leave you with user `synapse`.

This PR switches the sample config to recommend `synapse_user`. Took a me a second to figure this out, so assume this will beneficial to others.
 2020-07-20 18:29:25 +01:00
Patrick Cloke 852930add7
Add a default limit (of 100) to get/sync operations. (#7858) 2020-07-17 07:59:23 -04:00
Michael Albert fff483ea96
Add admin endpoint to get members in a room. (#7842) 2020-07-16 16:43:23 -04:00
Luke Faraone b0f031f92a
Combine nginx federation server blocks (#7823) 
I'm pretty sure there's no technical reason these have to be distinct server blocks, so collapse into one and go with the more terse location block.

Signed-off-by: Luke W Faraone <luke@faraone.cc>
 2020-07-16 16:01:45 +01:00
Patrick Cloke 8c7d0f163d
Allow accounts to be re-activated from the admin APIs. (#7847) 2020-07-15 11:00:21 -04:00
Patrick Cloke 111e70d75c
Return the proper 403 Forbidden error during errors with JWT logins. (#7844) 2020-07-15 07:10:21 -04:00
Brendan Abolivier 85223106f3
Allow email subjects to be customised through Synapse's configuration (#7846) 2020-07-14 19:10:42 +01:00
Dirk Klimpel 491f0dab1b
Add delete room admin endpoint (#7613) 
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API.

Fixes: #6425 

It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105)

Related to:
- #5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-07-14 12:36:23 +01:00
Patrick Cloke 77d2c05410
Add the option to validate the `iss` and `aud` claims for JWT logins. (#7827) 2020-07-14 07:16:43 -04:00
Erik Johnston f299441cc6
Add ability to shard the federation sender (#7798) 2020-07-10 18:26:36 +01:00
Nicolai Søborg 96bb01d8ec
Change Caddy links (old is deprecated) (#7789) 
* Change Caddy links

Current links points to Caddy v1 which is deprecated.

Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
 2020-07-08 10:09:16 +01:00
Patrick Cloke 2a266f4511
Add documentation for JWT login type and improve sample config. (#7776) 2020-07-06 08:31:51 -04:00
Patrick Cloke 71cccf1593
Additional configuration options for auto-join rooms (#7763) 2020-06-30 15:41:36 -04:00
Erik Johnston b44bdd7f7b
Support running multiple media repos. (#7706) 
This requires a new config option to specify which media repo should be
responsible for running background jobs to e.g. clear out expired URL
preview caches.
 2020-06-17 14:13:30 +01:00
Richard van der Hoff e452973fd2
fix broken link in sample config (#7712) 2020-06-16 19:50:16 +01:00
hungrymonkey 5c5516f80e
Add instructions for authing with Keycloak via OpenID (#7659) 2020-06-16 11:28:21 -04:00
Patrick Cloke b9df7f70bb
Increase the default SAML session expirary time to 15 minutes. (#7664) 2020-06-11 07:55:45 -04:00
wondratsch c746889bb0
fix typo in sample_config.yaml (#7652) 
Just a simple typo fix.

Signed-off-by: wondratsch 28294257+wondratsch@users.noreply.github.com
 2020-06-11 11:51:10 +01:00
Andrew Morgan fcd6961441
Add option to enable encryption by default for new rooms (#7639) 
Fixes https://github.com/matrix-org/synapse/issues/2431

Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.

Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637

Based on #7637
 2020-06-10 17:44:34 +01:00
Travis Ralston 09099313e6
Add an option to disable autojoin for guest accounts (#6637) 
Fixes https://github.com/matrix-org/synapse/issues/3177
 2020-06-05 18:18:15 +01:00
Richard van der Hoff 1bc00fd76d
Clarifications to the admin api documentation (#7647) 
* Clarify how to authenticate
* path params are not the same thing as query params
* Fix documentation for `/_synapse/admin/v2/users/<user_id>`
 2020-06-05 17:31:05 +01:00
Dirk Klimpel 2970ce8367
Add device management to admin API (#7481) 
- Admin is able to
  - change displaynames
  - delete devices
  - list devices
  - get device informations

Fixes #7330
 2020-06-05 13:07:22 +01:00
Richard van der Hoff 11de843626
Cleanups to the OpenID Connect integration (#7628) 
docs, default configs, comments. Nothing very significant.
 2020-06-03 21:13:17 +01:00
Richard van der Hoff 1bbc9e2df6
Clean up exception handling in SAML2ResponseResource (#7614) 
* Expose `return_html_error`, and allow it to take a Jinja2 template instead of a raw string

* Clean up exception handling in SAML2ResponseResource

  * use the existing code in `return_html_error` instead of re-implementing it
    (giving it a jinja2 template rather than inventing a new form of template)

  * do the exception-catching in the REST layer rather than in the handler
    layer, to make sure we catch all exceptions.
 2020-06-03 10:41:12 +01:00
Christopher Cooper c4a820b32a
allow emails to be passed through SAML (#7385) 
Signed-off-by: Christopher Cooper <cooperc@ocf.berkeley.edu>
 2020-05-27 17:40:08 +01:00
Jason Robinson 4be968d05d
Fix sample config docs error (#7581) 
'client_auth_method' commented out value was erronously 'client_auth_basic',
when code and docstring says it should be 'client_secret_basic'.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2020-05-27 13:52:18 +01:00
Erik Johnston d7d8a2e7ee Fix up comments 2020-05-27 13:34:46 +01:00
Erik Johnston 4ba55559ac
Fix specifying cache factors via env vars with * in name. (#7580) 
This mostly applise to `*stateGroupCache*` and co.

Broke in #6391.
 2020-05-27 13:17:01 +01:00
Richard van der Hoff 66a564c859
Fix some DETECTED VIOLATIONS in the config file (#7550) 
consistency ftw
 2020-05-22 10:11:50 +01:00
David Vo d74cdc1a42
Ensure worker config exists in systemd service (#7528) 2020-05-21 13:47:23 +01:00
Richard van der Hoff 4fa74c7606
Minor clarifications to the TURN docs (#7533) 2020-05-20 11:04:34 +01:00
Paul Tötterman ab3e19d814
Improve API doc readability (#7527) 2020-05-19 11:20:23 +01:00
Richard van der Hoff 24d9151a08
Formatting for reverse-proxy docs (#7514) 
also a small clarification to nginx
 2020-05-15 15:13:39 +01:00
Jeff Peeler 572b444dab
Add Caddy 2 example (#7463) 
The specific headers that are passed using this new configuration format
are Host and X-Forwarded-For, which should be all that's required.

Note that for production another matcher should be added in the first
section to properly handle the base_url lookup:
reverse_proxy /.well-known/matrix/* http://localhost:8008

Signed-off-by: Jeff Peeler <jpeeler@gmail.com>
 2020-05-15 14:36:01 +01:00
Richard van der Hoff ec0b72bc4e Merge branch 'master' into develop 2020-05-14 18:12:00 +01:00
Richard van der Hoff 66d03639dc
Notes on using git (#7496) 
* general updates to CONTRIBUTING.md
* notes on updating your PR
* Notes on squash-merging or otherwise
* document git branching model
 2020-05-14 18:03:10 +01:00
Patrick Cloke 51fb0fc2e5
Update documentation about SSO mapping providers (#7458) 2020-05-12 10:51:07 -04:00
Amber Brown 7cb8b4bc67
Allow configuration of Synapse's cache without using synctl or environment variables (#6391) 2020-05-11 18:45:23 +01:00
Andrew Morgan 5cf758cdd6 Merge branch 'release-v1.13.0' into develop 
* release-v1.13.0:
  Don't UPGRADE database rows
  RST indenting
  Put rollback instructions in upgrade notes
  Fix changelog typo
  Oh yeah, RST
  Absolute URL it is then
  Fix upgrade notes link
  Provide summary of upgrade issues in changelog. Fix )
  Move next version notes from changelog to upgrade notes
  Changelog fixes
  1.13.0rc1
  Documentation on setting up redis (#7446)
  Rework UI Auth session validation for registration (#7455)
  Fix errors from malformed log line (#7454)
  Drop support for redis.dbid (#7450)
 2020-05-11 16:46:33 +01:00
Neil Johnson 85155654c5
Documentation on setting up redis (#7446) 2020-05-11 13:21:15 +01:00
Andrew Morgan 67feea8044
Extend spam checker to allow for multiple modules (#7435) 2020-05-08 19:25:48 +01:00
Quentin Gliech 616af44137
Implement OpenID Connect-based login (#7256) 2020-05-08 08:30:40 -04:00
Manuel Stahl a4a5ec4096
Add room details admin endpoint (#7317) 2020-05-07 15:33:07 -04:00
Brendan Abolivier 5bb26b7c4f Merge branch 'release-v1.13.0' into develop 2020-05-07 17:31:19 +02:00
Erik Johnston d7983b63a6
Support any process writing to cache invalidation stream. (#7436) 2020-05-07 13:51:08 +01:00
Brendan Abolivier d9b8d27494
Add a configuration setting for the dummy event threshold (#7422) 
Add dummy_events_threshold which allows configuring the number of forward extremities a room needs for Synapse to send forward extremities in it.
 2020-05-07 10:35:23 +01:00
Richard van der Hoff 207b1737ee
Update reverse_proxy.md 
a couple of cleanups
 2020-05-05 11:29:29 +01:00
Erik Johnston 37f6823f5b
Add instance name to RDATA/POSITION commands (#7364) 
This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
 2020-04-29 16:23:08 +01:00
Andrew Morgan c58ae367d8
Clean up admin api docs (#7361) 2020-04-28 20:06:03 +01:00
Manuel Stahl 04dd7d182d
Return total number of users and profile attributes in admin users endpoint (#6881) 
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
 2020-04-28 18:19:36 +01:00
Brendan Abolivier 036fab5d8a
Document monitoring workers (#7357) 
It doesn't seem to be documented anywhere and means that you suddenly start losing metrics without any obvious reason when you go from monolith to workers (e.g. #7312).
 2020-04-27 21:36:47 +02:00
lub aa2492907f
Add some explanation to application_services.md (#7091) 
Signed-off-by: Simon Körner <git@lubiland.de>
 2020-04-27 15:03:09 +01:00
Patrick Cloke 7bfe0902ce
Add documentation to the sample config about the templates for SSO. (#7343) 2020-04-24 15:03:49 -04:00
Patrick Cloke 204664d1ad Synapse v1.12.4 
Features:
 
 * Always send users their own device updates. (#7160)
 * Add support for handling GET requests for account_data on a worker. (#7311)
 
 Bugfixes:
 
 * Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255)
 * Do not treat display names as globs in push rules. (#7271)
 * Fix a bug with cross-signing devices belonging to remote users who did not share a
   room with any user on the local homeserver. (#7289)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl6hs9EACgkQM/xY9qcR
 MEjq2w/9EC6m4Bx/2bDlohUy3GKfe9BUpZnMDGSgQR1v+J+OIA4yZ1s16hmY95fr
 rGjNaZlcUbLUFj9GfaLOOZjvuXQ0KslO2ojP1lcSaKCfx9WK1CH0DtCNVB0lAYQv
 sfu2C2I91XetkiMHo5GqYBP1d6IK3OTI7YRz72sHOQTVPFcSB1/wCUPuan7P90HE
 gBdfxtl7Xl1TU0tfvJoeRLVarktVnE4LiXREfm1eGpRECPclewu7sbx5p1ESN5mg
 /RBIh0FP4rOL4cP4Qt5qiCo68woJmBc4CwiV6pvcOyUMtMGqmwO9K/wJxqY+pxnh
 JKBNqv90BCSxD55Nywslx9yA52kshuRdcdqHuOVYOHo4Dcwu+ecyySpmzuZ2EJPc
 hDEkf3dnpfaVoMn3uesbwGNvHEr69QXxR8yE1XZvlAdKzvy4Po/0qRAZuD+NG9qf
 YjN94bDbkASlFQgPn3wF7R+4RA/HdOf/1Ns6YKt2dFRikHb18WCBPWxHcSwscrgE
 W8xma8rVTlqfRBuAcZ6y4K3KSHFX6dxLUNxqUngyAhjTInSPuqt2EzlAdqRpnJBS
 t+hhj+zOh2ixxbz4ZrGdMx0jPF2XARr38x3F/LIjhHLcgOlHdCl4WWkxi9BSXOTd
 cscvQ8GJBH9c0ANy7PGBntXlohQiW3eVVqblWRbV95TpHVoimME=
 =Hc/7
 -----END PGP SIGNATURE-----

Merge tag 'v1.12.4'

Synapse v1.12.4

Features:

* Always send users their own device updates. (#7160)
* Add support for handling GET requests for account_data on a worker. (#7311)

Bugfixes:

* Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255)
* Do not treat display names as globs in push rules. (#7271)
* Fix a bug with cross-signing devices belonging to remote users who did not share a
  room with any user on the local homeserver. (#7289)
 2020-04-23 12:03:33 -04:00
Brendan Abolivier 2e3b9a0fcb
Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token"" 
This reverts commit 1adf6a5587.
 2020-04-23 11:23:53 +02:00
Brendan Abolivier fb825759e3 Merge branch 'master' into develop 2020-04-23 11:23:33 +02:00
Brendan Abolivier 1adf6a5587
Revert "Merge pull request #7315 from matrix-org/babolivier/request_token" 
This reverts commit 6f4319368b, reversing
changes made to 0d775fcc2d.
 2020-04-23 11:23:10 +02:00
Brendan Abolivier 6f4319368b
Merge pull request #7315 from matrix-org/babolivier/request_token 
Config option to inhibit 3PID errors on /requestToken
 2020-04-23 10:38:57 +02:00
Richard van der Hoff 71a1abb8a1
Stop the master relaying USER_SYNC for other workers (#7318) 
Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication.

In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits.

Fixes (I hope) #7257.
 2020-04-22 22:39:04 +01:00
Brendan Abolivier 69ad7cc13b
Config option to inhibit 3PID errors on /requestToken 
Adds a request_token_inhibit_errors configuration flag (disabled by
default) which, if enabled, change the behaviour of all /requestToken
endpoints so that they return a 200 and a fake sid if the 3PID was/was
not found associated with an account (depending on the endpoint),
instead of an error.

Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2020-04-22 23:38:42 +02:00
Dirk Klimpel 6b6685db9f
Extend room admin api with additional attributes (#7225) 2020-04-22 13:38:41 +01:00
Richard van der Hoff 2aa5bf13c8 Merge branch 'release-v1.12.4' into develop 2020-04-22 13:09:23 +01:00
Richard van der Hoff 974c0d726a
Support GET account_data requests on a worker (#7311) 2020-04-21 10:46:30 +01:00
Lars Franke 13917232d5
Fix indention in generated config file (#7300) 
Also adjust sample_config.yaml

Signed-off-by: Lars Franke <frcl@mailbox.org>
 2020-04-20 16:51:27 +01:00
nataraj-hates-MS-for-stealing-github 0d775fcc2d
Improve example TURN configuration in documentation (#7284) 2020-04-17 08:04:23 -04:00
Tristan Lins c07fca9e2f
Clarify the comments for media_storage_providers options (#7272) 2020-04-17 07:09:33 -04:00
Andrew Morgan a48138784e
Allow specifying the value of Accept-Language header for URL previews (#7265) 2020-04-15 13:35:29 +01:00
Ryan Hovland 4a0dadafbe
Add setting to nginx configuration to allow larger file uploads (#7251) 2020-04-13 17:23:36 +01:00
Andrew Morgan a026bdaab7
Add matrix-synapse-shared-secret-auth as an example password provider (#7248) 2020-04-09 12:49:05 +01:00
Richard van der Hoff cae4121484
Make systemd-with-workers doc official (#7234) 
Simplify and update this documentation, and make it part of the core dist.
 2020-04-08 11:59:26 +01:00
Andrew Morgan 29b7e22b93
Add documentation to password_providers config option (#7238) 2020-04-08 00:46:50 +01:00
Martin Milata b0db928c63
Extend web_client_location to handle absolute URLs (#7006) 
Log warning when filesystem path is used.

Signed-off-by: Martin Milata <martin@martinmilata.cz>
 2020-04-03 11:57:34 -04:00
Richard van der Hoff 0122ef1037 Revert "Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback" 
This was incorrectly merged to master.

This reverts commit 319c41f573, reversing
changes made to 229eb81498.
 2020-04-03 11:17:39 +01:00
siroccal 250f87d0de
Update postgres.md (#7119) 2020-04-01 12:44:51 +01:00
Jostein Kjønigsen 2e826cd80c
Improve TURN documentation. (#7167) 2020-03-31 15:50:48 +01:00
Andrew Morgan d9f29f8dae
Fix a small typo in the `metrics_flags` config option. (#7171) 2020-03-30 17:38:21 +01:00
Erik Johnston 4f21c33be3
Remove usage of "conn_id" for presence. (#7128) 
* Remove `conn_id` usage for UserSyncCommand.

Each tcp replication connection is assigned a "conn_id", which is used
to give an ID to a remotely connected worker. In a redis world, there
will no longer be a one to one mapping between connection and instance,
so instead we need to replace such usages with an ID generated by the
remote instances and included in the replicaiton commands.

This really only effects UserSyncCommand.

* Add CLEAR_USER_SYNCS command that is sent on shutdown.

This should help with the case where a synchrotron gets restarted
gracefully, rather than rely on 5 minute timeout.
 2020-03-30 16:37:24 +01:00
Patrick Cloke c5f89fba55
Add developer documentation for running a local CAS server (#7147) 2020-03-30 07:28:42 -04:00
Richard van der Hoff b7da598a61 Always whitelist the login fallback for SSO (#7153) 
That fallback sets the redirect URL to itself (so it can process the login
token then return gracefully to the client). This would make it pointless to
ask the user for confirmation, since the URL the confirmation page would be
showing wouldn't be the client's.
 2020-03-27 20:24:52 +00:00
Dirk Klimpel fb69690761
Admin API to join users to a room. (#7051) 2020-03-27 19:16:43 +00:00
Dirk Klimpel 8327eb9280
Add options to prevent users from changing their profile. (#7096) 2020-03-27 19:15:23 +00:00
Brendan Abolivier 319c41f573
Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback 
Always whitelist the login fallback for SSO
 2020-03-27 15:34:41 +01:00
Brendan Abolivier 63aea691a7
Update the wording of the config comment 2020-03-27 15:09:12 +01:00
Brendan Abolivier 7083147961
Regenerate sample config 2020-03-26 19:01:54 +01:00
Dirk Klimpel e8e2ddb60a
Allow server admins to define and enforce a password policy (MSC2000). (#7118) 2020-03-26 16:51:13 +00:00
Aaron Raimist 6ca5e56fd1
Remove unused captcha_bypass_secret option (#7137) 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2020-03-25 17:49:34 +00:00
Erik Johnston 4cff617df1
Move catchup of replication streams to worker. (#7024) 
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
 2020-03-25 14:54:01 +00:00
Richard van der Hoff 39230d2171
Clean up some LoggingContext stuff (#7120) 
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
 2020-03-24 14:45:33 +00:00
Richard van der Hoff 5126cb1253 Merge branch 'master' into develop 2020-03-23 13:54:29 +00:00
Richard van der Hoff 229eb81498 Synapse 1.12.0 (2020-03-23) 
===========================
 
 No significant changes since 1.12.0rc1.
 
 Debian packages and Docker images are rebuilt using the latest versions of
 dependency libraries, including Twisted 20.3.0. **Please see security advisory
 below**.
 
 Security advisory
 -----------------
 
 Synapse may be vulnerable to request-smuggling attacks when it is used with a
 reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are
 described in
 [CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108)
 and
 [CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109).
 For a good introduction to this class of request-smuggling attacks, see
 https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn.
 
 We are not aware of these vulnerabilities being exploited in the wild, and
 do not believe that they are exploitable with current versions of any reverse
 proxies. Nevertheless, we recommend that all Synapse administrators ensure that
 they have the latest versions of the Twisted library to ensure that their
 installation remains secure.
 
 * Administrators using the [`matrix.org` Docker
   image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
   packages from
   `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
   should ensure that they have version 1.12.0 installed: these images include
   Twisted 20.3.0.
 * Administrators who have [installed Synapse from
   source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
   should upgrade Twisted within their virtualenv by running:
   ```sh
   <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0'
   ```
 * Administrators who have installed Synapse from distribution packages should
   consult the information from their distributions.
 
 The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities.
 
 Advance notice of change to the default `git` branch for Synapse
 ----------------------------------------------------------------
 
 Currently, the default `git` branch for Synapse is `master`, which tracks the
 latest release.
 
 After the release of Synapse 1.13.0, we intend to change this default to
 `develop`, which is the development tip. This is more consistent with common
 practice and modern `git` usage.
 
 Although we try to keep `develop` in a stable state, there may be occasions
 where regressions creep in. Developers and distributors who have scripts which
 run builds using the default branch of `Synapse` should therefore consider
 pinning their scripts to `master`.
 
 Synapse 1.12.0rc1 (2020-03-19)
 ==============================
 
 Features
 --------
 
 - Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)):
   - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965))
   - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971))
   - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986))
   - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037))
   - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941))
   - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966))
   - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970))
   - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034))
 - Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315))
 - Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982))
 - Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030))
 - Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067))
 - Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085))
 - Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106))
 - Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094))
 
 Bugfixes
 --------
 
 - When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572))
 - Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962))
 - Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967))
 - Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968))
 - Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990))
 - Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018))
 - Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035))
 - Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044))
 - Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066))
 - Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070))
 - Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074))
 
 Improved Documentation
 ----------------------
 
 - Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925))
 - Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984))
 - Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015))
 - Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048))
 
 Deprecations and Removals
 -------------------------
 
 - Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026))
 
 Internal Changes
 ----------------
 
 - Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309))
 - Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615))
 - Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003))
 - Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095))
 - Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953))
 - Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954))
 - Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956))
 - Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957))
 - Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104))
 - Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979))
 - Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985))
 - Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987))
 - Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019))
 - Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995))
 - Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020))
 - Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045))
 - Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl54vN0ACgkQOSor00I9
 eP/LOgf/U22VypPu2Cl4vofxigeeUL+ph0gEQDCsF7i3EPi9ObeTi4aUggC95dBs
 MY4JQENmVrk1NhVWQpC08mjehsr4xUzJuwnPPIeGQ6X5U/2BS4YUasxOt9A+iLUz
 8IxTzUgx4T+CTfibZvn6xdE/tZK/b2N3BoOikesutD2aQGC5Fm6w2HReoY4Qrdgw
 AwHlsIV22PYgEn0RL5y6DJ2NUU9SdeSmPjKNe+R1rHDlTpvH7LSyINhOFkYDgRPY
 xmlH+Ek5+7vLi3AlWg6pA001mMWGADlC4T84URcf1fQv6hXT1iM+A9CqC57jGlfT
 nHDphCtz0Uk9kmgT4To+hclLDWecYw==
 =iSEk
 -----END PGP SIGNATURE-----

Merge tag 'v1.12.0'

Synapse 1.12.0 (2020-03-23)
===========================

No significant changes since 1.12.0rc1.

Debian packages and Docker images are rebuilt using the latest versions of
dependency libraries, including Twisted 20.3.0. **Please see security advisory
below**.

Security advisory
-----------------

Synapse may be vulnerable to request-smuggling attacks when it is used with a
reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are
described in
[CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108)
and
[CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109).
For a good introduction to this class of request-smuggling attacks, see
https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn.

We are not aware of these vulnerabilities being exploited in the wild, and
do not believe that they are exploitable with current versions of any reverse
proxies. Nevertheless, we recommend that all Synapse administrators ensure that
they have the latest versions of the Twisted library to ensure that their
installation remains secure.

* Administrators using the [`matrix.org` Docker
  image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
  packages from
  `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
  should ensure that they have version 1.12.0 installed: these images include
  Twisted 20.3.0.
* Administrators who have [installed Synapse from
  source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
  should upgrade Twisted within their virtualenv by running:
  ```sh
  <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0'
  ```
* Administrators who have installed Synapse from distribution packages should
  consult the information from their distributions.

The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities.

Advance notice of change to the default `git` branch for Synapse
----------------------------------------------------------------

Currently, the default `git` branch for Synapse is `master`, which tracks the
latest release.

After the release of Synapse 1.13.0, we intend to change this default to
`develop`, which is the development tip. This is more consistent with common
practice and modern `git` usage.

Although we try to keep `develop` in a stable state, there may be occasions
where regressions creep in. Developers and distributors who have scripts which
run builds using the default branch of `Synapse` should therefore consider
pinning their scripts to `master`.

Synapse 1.12.0rc1 (2020-03-19)
==============================

Features
--------

- Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)):
  - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965))
  - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971))
  - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986))
  - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037))
  - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941))
  - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966))
  - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970))
  - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034))
- Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315))
- Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982))
- Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030))
- Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067))
- Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085))
- Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106))
- Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094))

Bugfixes
--------

- When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572))
- Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962))
- Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967))
- Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968))
- Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990))
- Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018))
- Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035))
- Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044))
- Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066))
- Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070))
- Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074))

Improved Documentation
----------------------

- Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925))
- Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984))
- Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015))
- Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048))

Deprecations and Removals
-------------------------

- Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026))

Internal Changes
----------------

- Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309))
- Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615))
- Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003))
- Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095))
- Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953))
- Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954))
- Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956))
- Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957))
- Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104))
- Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979))
- Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985))
- Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987))
- Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019))
- Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995))
- Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020))
- Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045))
- Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063))
 2020-03-23 13:54:17 +00:00
Richard van der Hoff c165c1233b
Improve database configuration docs (#6988) 
Attempts to clarify the sample config for databases, and add some stuff about
tcp keepalives to `postgres.md`.
 2020-03-20 15:24:22 +00:00
Patrick Cloke 88b41986db
Add an option to the set password API to choose whether to logout other devices. (#7085) 2020-03-18 07:50:00 -04:00
Richard van der Hoff 4ce50519cd
Update postgres.md 
fix broken link
 2020-03-17 18:08:43 +00:00
Richard van der Hoff 6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)" 
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
 2020-03-17 11:25:01 +00:00
Brendan Abolivier f9e98176bf
Put the file in the templates directory 2020-03-11 20:31:42 +00:00
Brendan Abolivier 900bca9707
Update wording and config 2020-03-11 19:40:30 +00:00
Brendan Abolivier 54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
Dirk Klimpel 751d51dd12
Update sample_config.yaml 2020-03-10 21:41:25 +01:00
Brendan Abolivier 51c094c4ac
Update sample config 2020-03-10 14:00:29 +00:00
dklimpel 885134529f updates after review 2020-03-09 22:09:29 +01:00
Patrick Cloke 66315d862f
Update routing of fallback auth in the worker docs. (#7048) 2020-03-09 07:19:24 -04:00
dklimpel 99bbe177b6 add disable_3pid_changes 2020-03-08 21:58:12 +01:00
dklimpel 20545a2199 lint2 2020-03-08 15:28:00 +01:00
dklimpel fb078f921b changelog 2020-03-08 15:19:07 +01:00
Brendan Abolivier 43f874055d
Merge branch 'master' into develop 2020-03-03 15:20:49 +00:00
Richard van der Hoff b68041df3d Add a whitelist for the SSO confirmation step. 2020-03-02 17:05:09 +00:00
Brendan Abolivier b2bd54a2e3 Add a confirmation step to the SSO login flow 2020-03-02 16:36:32 +00:00
Richard van der Hoff 4c2ed3f20e
Fix minor issues with email config (#6962) 
* Give `notif_template_html`, `notif_template_text` default values (fixes #6960)
 * Don't complain if `smtp_host` and `smtp_port` are unset, since they have sensible defaults (fixes #6961)
 * Set the example for `enable_notifs` to `True`, for consistency and because it's more useful
 * Raise errors as ConfigError rather than RuntimeError for nicer formatting
 2020-02-24 15:18:38 +00:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) 
Ensure good comprehension hygiene using flake8-comprehensions.
 2020-02-21 07:15:07 -05:00
Brendan Abolivier bbe39f808c
Merge pull request #6940 from matrix-org/babolivier/federate.md 
Clean up and update federation docs
 2020-02-19 10:58:59 +00:00
Brendan Abolivier 71d65407e7
Incorporate review 2020-02-19 10:03:19 +00:00
Brendan Abolivier 61b457e3ec
Incorporate review 2020-02-18 17:20:03 +00:00
Brendan Abolivier a0d2f9d089
Phrasing 2020-02-18 16:16:49 +00:00
Brendan Abolivier d484126bf7
Merge pull request #6907 from matrix-org/babolivier/acme-config 
Add mention and warning about ACME v1 deprecation to the TLS config
 2020-02-18 16:11:31 +00:00
Erik Johnston 818def8248
Fix worker docs to point `/publicised_groups` API correctly. (#6938) 2020-02-18 15:27:45 +00:00
Brendan Abolivier d009535639
Add mention of SRV records as an advanced topic 2020-02-18 14:07:41 +00:00
Brendan Abolivier ba7a523854
Argh trailing spaces 2020-02-18 13:57:15 +00:00
Brendan Abolivier e837be5b5c
Fix links in the reverse proxy doc 2020-02-18 13:53:58 +00:00
Brendan Abolivier 3c67eee6dc
Make federate.md more of a sumary of the steps to follow to set up replication 2020-02-18 13:51:03 +00:00
Brendan Abolivier 8ee0d74516
Split the delegating documentation out of federate.md and trim it down 2020-02-18 12:05:45 +00:00
Richard van der Hoff 97a42bbc3a
Add a warning about indentation to generated config (#6920) 
Fixes #6916.
 2020-02-14 16:22:30 +00:00
Patrick Cloke 49f877d32e
Filter the results of user directory searching via the spam checker (#6888) 
Add a method to the spam checker to filter the user directory results.
 2020-02-14 07:17:54 -05:00
Brendan Abolivier 5820ed905f
Add mention and warning about ACME v1 deprecation to the Synapse config 2020-02-13 14:20:08 +00:00
Patrick Cloke 361de49c90
Add documentation for the spam checker module (#6906) 
Add documentation for the spam checker.
 2020-02-13 07:40:57 -05:00
Brendan Abolivier 862669d6cc
Update docs/ACME.md 2020-02-13 11:29:08 +00:00
Brendan Abolivier 459d089af7
Mention that using Synapse to serve certificates requires restarts 2020-02-12 21:05:30 +00:00
Brendan Abolivier e45a7c0939
Remove duplicated info about certbot et al 2020-02-12 20:14:59 +00:00
Brendan Abolivier f092029d2d
Update ACME.md to mention ACME v1 deprecation 2020-02-12 20:14:16 +00:00
Brendan Abolivier 6cd34da8b1
Merge pull request #6891 from matrix-org/babolivier/retention-doc-amend 
Spell out that the last event sent to a room won't be deleted by a purge
 2020-02-12 20:12:20 +00:00
Brendan Abolivier 08e050c3fd
Rephrase 2020-02-12 15:39:40 +00:00
Brendan Abolivier 47acbc519f Merge branch 'master' into develop 2020-02-12 13:24:09 +00:00
Brendan Abolivier d9239b5257 Synapse 1.10.0 (2020-02-12) 
===========================
 
 **WARNING to client developers**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details.
 
 Updates to the Docker image
 ---------------------------
 
 - Update the docker images to Alpine Linux 3.11. ([\#6897](https://github.com/matrix-org/synapse/issues/6897))
 
 Synapse 1.10.0rc5 (2020-02-11)
 ==============================
 
 Bugfixes
 --------
 
 - Fix the filtering introduced in 1.10.0rc3 to also apply to the state blocks returned by `/sync`. ([\#6884](https://github.com/matrix-org/synapse/issues/6884))
 
 Synapse 1.10.0rc4 (2020-02-11)
 ==============================
 
 This release candidate was built incorrectly and is superceded by 1.10.0rc5.
 
 Synapse 1.10.0rc3 (2020-02-10)
 ==============================
 
 Features
 --------
 
 - Filter out `m.room.aliases` from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878))
 
 Internal Changes
 ----------------
 
 - Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880))
 
 Synapse 1.10.0rc2 (2020-02-06)
 ==============================
 
 Bugfixes
 --------
 
 - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844))
 - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848))
 
 Internal Changes
 ----------------
 
 - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))
 
 Synapse 1.10.0rc1 (2020-01-31)
 ==============================
 
 Features
 --------
 
 - Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794))
 
 Bugfixes
 --------
 
 - Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734))
 - Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761))
 - Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767))
 - Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771))
 - Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795))
 - Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796))
 - Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800))
 - Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801))
 - Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811))
 
 Improved Documentation
 ----------------------
 
 - Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775))
 
 Internal Changes
 ----------------
 
 - Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810))
 - Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748))
 - Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751))
 - Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757))
 - Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819))
 - Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786))
 - Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792))
 - When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797))
 - Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816))
 - Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAl5D780ACgkQHgFcFF8Z
 Fs32MxAApvBbTGMhC5E2SLt8Yh4DSfDmg8yCPLMRVTdJuUQyUVgKYFSl9LqzqCGe
 oLpzwbXXDpN1FYGuO3daFBSN3RaU1XCQPMayekTaF6Wvq9N9rNW1tAi6cQTjUf20
 bTAxZljXAIkfXOMbahlFIqAtL+ItTbECKPW4zbndIbC4TiWw/bJPFHeSuoQbHbsI
 TOgPjM4q2aydHFeTDL2Wqqm99pobPvnEji5sBzHOK+IyIaN0HZI7BZAsm38SDSEA
 HQTW342RNl7Vm0t2KPdhBq/43r4ENKSUbZjCctFZA+iqbV2mmE96zd2pLOIh9OLP
 7KoCDKlplQjInJjN3MQvPzn/cxHnZOmt3AwhQjfhOMcJWcwz00ouWp38oszgZi1c
 8y1htf1/Tr26veYxF7umlPHT0h8LNj4Wi7ceQjoIDPu4X70X6vXOk5sJmeJ+KqI5
 PJ4yUgw0erWX0I7jybavIvruyA9ZqCwk5FIIyG54IBIZLfughi/fcce49gfvJTa6
 Rdjvp0g28qxynkRKq9hVOBSX/TRXTedHe8tHBvSzoDRScvI2aDWQYjJkmGzzLF6L
 +eWiZ2ICJ2fs7XYIxPhEr/vSpmc/1R+k6vZIPeBTAs0muH8WoK2en6KZPmJ7C25L
 2FHdA+1S1Ybqsvyue5S5Sl6yGcvpzGzc1R4mHoB0jZki2v3Cwcs=
 =j25Z
 -----END PGP SIGNATURE-----

Merge tag 'v1.10.0'

Synapse 1.10.0 (2020-02-12)
===========================

**WARNING to client developers**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details.

Updates to the Docker image
---------------------------

- Update the docker images to Alpine Linux 3.11. ([\#6897](https://github.com/matrix-org/synapse/issues/6897))

Synapse 1.10.0rc5 (2020-02-11)
==============================

Bugfixes
--------

- Fix the filtering introduced in 1.10.0rc3 to also apply to the state blocks returned by `/sync`. ([\#6884](https://github.com/matrix-org/synapse/issues/6884))

Synapse 1.10.0rc4 (2020-02-11)
==============================

This release candidate was built incorrectly and is superceded by 1.10.0rc5.

Synapse 1.10.0rc3 (2020-02-10)
==============================

Features
--------

- Filter out `m.room.aliases` from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878))

Internal Changes
----------------

- Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880))

Synapse 1.10.0rc2 (2020-02-06)
==============================

Bugfixes
--------

- Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844))
- Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848))

Internal Changes
----------------

- Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))

Synapse 1.10.0rc1 (2020-01-31)
==============================

Features
--------

- Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794))

Bugfixes
--------

- Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734))
- Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761))
- Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767))
- Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771))
- Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795))
- Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796))
- Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800))
- Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801))
- Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811))

Improved Documentation
----------------------

- Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775))

Internal Changes
----------------

- Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810))
- Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748))
- Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751))
- Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757))
- Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819))
- Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786))
- Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792))
- When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797))
- Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816))
- Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820))
 2020-02-12 13:23:22 +00:00
Brendan Abolivier 6b21986e4e
Also spell it out in the purge history API doc 2020-02-11 17:56:04 +00:00
Brendan Abolivier a443d2a25d
Spell out that Synapse never purges the last event sent in a room 2020-02-11 17:37:09 +00:00
Erik Johnston 21db35f77e
Add support for putting fed user query API on workers (#6873) 2020-02-07 15:45:39 +00:00
Erik Johnston de2d267375
Allow moving group read APIs to workers (#6866) 2020-02-07 11:14:19 +00:00
Dirk Klimpel 56ca93ef59
Admin api to add an email address (#6789) 2020-02-07 10:29:36 +00:00
Robin Vleij f0561fcffd
Update documentation (#6859) 
Update documentation to reflect the correct format of user_id (fully qualified).
 2020-02-05 21:27:38 +00:00
Erik Johnston 02b44db922
Warn if postgres database has non-C locale. (#6734) 
As using non-C locale can cause issues on upgrading OS.
 2020-01-28 13:44:21 +00:00
Jason Robinson 1fe5001369 Fix federation_reader listeners doc as per PR review 
Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2020-01-27 10:21:25 +02:00
Jason Robinson aa6ad288f1 Clarifications to the workers documentation 
* Add note that user_dir requires disabling user dir
  updates from the main synapse process.
* Add note that federation_reader should have
  the federation listener resource.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2020-01-24 11:08:50 +02:00
Andrew Morgan d31f5f4d89
Update admin room docs with correct endpoints (#6770) 2020-01-23 11:37:26 +00:00
Andrew Morgan 90a28fb475
Admin API to list, filter and sort rooms (#6720) 2020-01-22 13:36:43 +00:00
Erik Johnston 5d7a6ad223
Allow streaming cache invalidate all to workers. (#6749) 2020-01-22 10:37:00 +00:00