Mirrors
/
synapse
mirror of https://github.com/element-hq/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,139 Commits 932 Branches 727 Tags 442 MiB
Commit Graph

1349 Commits

Author SHA1 Message Date
Andrew Morgan d688a6dee5 fix a small typo in the delete room api docs 2021-11-03 11:09:00 +00:00
Dirk Klimpel 7537201840
Add search by room ID and room alias to List Room admin API (#11099) 
Fixes: #10874
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-11-02 10:01:13 +00:00
reivilibre 69ab3dddbc
Make `check_event_allowed` module API callback not fail open (accept events) when an exception is raised (#11033) 2021-11-01 15:45:56 +00:00
Dirk Klimpel 66bdca3e31
Remove deprecated delete room admin API (#11213) 
Remove deprecated delete room admin API,
`POST /_synapse/admin/v1/rooms/<room_id>/delete`
 2021-11-01 15:11:24 +00:00
Richard van der Hoff 71f9966f27
Support for serving server well-known files (#11211) 
Fixes https://github.com/matrix-org/synapse/issues/8308
 2021-11-01 15:10:16 +00:00
Brett Bethke 2014098d01
Add domain specific matching for haproxy config (#11128) 2021-11-01 14:16:02 +00:00
Sumner Evans ece84f2c45
Improve code formatting and fix a few typos in docs (#11221) 
* Labeled a lot more code blocks with the appropriate type
* Fixed a couple of minor typos (missing/extraneous commas)

Signed-off-by: Sumner Evans <me@sumnerevans.com>
 2021-11-01 11:35:55 +00:00
Brendan Abolivier e320f5dba3
Deprecate user_may_create_room_with_invites (#11206) 2021-11-01 10:46:08 +01:00
Sean Quah 3ed17ff651
Clarify lack of Windows support in documentation (#11198) 2021-10-29 14:03:58 +01:00
Skyler Mäntysaari a19bf32a03
docs/openid: Add Authentik documentation. (#11151) 2021-10-28 10:31:22 +00:00
Brendan Abolivier c7a5e49664
Implement an `on_new_event` callback (#11126) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-10-26 15:17:36 +02:00
Jason Robinson 63cbdd8af0
Enable changing user type via users admin API (#11174) 
Users admin API can now also modify user
type in addition to allowing it to be
set on user creation.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-10-26 09:01:06 +00:00
Dirk Klimpel b0f03aeb6a
Add more information what happens when a user is deactivated (#11083) 2021-10-22 09:00:28 +01:00
Aaron R 2c61a318cc
Show error when timestamp in seconds is provided to the /purge_media_cache API (#11101) 2021-10-20 14:41:48 +00:00
Travis Ralston 106d99b8cd
Remove false warning about copying the log config to a homeserver.yaml (#11092) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-10-20 11:48:15 +00:00
Brendan Abolivier 78d5896d19
Document the version of Synapse each module callback was introduced in (#11132) 
* Mention callbacks introduced in v1.37.0

According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062

* Mention callbacks introduced in v1.39.0

According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884

* Mention callbacks introduced in v1.42.0

According to https://github.com/matrix-org/synapse/pull/10524

* Mention callbacks introduced in v1.44.0 and v1.45.0

As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894

* Mention callbacks introduced in v1.46.0

According to https://github.com/matrix-org/synapse/pull/10548
 2021-10-20 11:04:27 +00:00
Andrew Morgan 5e0e683541
Fix instances of [example]{.title-ref} in the upgrade notes (#11118) 2021-10-19 14:13:56 +01:00
Brendan Abolivier 73743b8ad1
Document Synapse's behaviour when dealing with multiple modules (#11096) 
Document Synapse's behaviour when multiple modules register the same
callback/web resource/etc.

Co-authored-by: reivilibre <oliverw@matrix.org>
 2021-10-18 18:26:52 +02:00
Patrick Cloke 5573133348
Move experimental & retention config out of the server module. (#11070) 2021-10-15 14:30:48 +00:00
Dirk Klimpel 013e0f9cae
Update doc of the allowed characters for registration tokens (#11093) 
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-10-15 09:56:39 +00:00
David Robertson 404444260a
Fix upgrade dead links (#11069) 2021-10-13 14:32:43 +01:00
Azrenbeth cdd308845b
Port the Password Auth Providers module interface to the new generic interface (#10548) 
Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-10-13 11:21:52 +00:00
Brendan Abolivier 9e13cd98af
Update upgrade notes 2021-10-12 11:23:51 +01:00
reivilibre 5e29d417fc
Include the requirements for [mypy,lint] in [dev] (#11034) 2021-10-11 16:34:31 +01:00
Olivier Wilkinson (reivilibre) 593eeac19e Revert accidental push to develop. 2021-10-08 17:17:22 +01:00
Olivier Wilkinson (reivilibre) 9f23ff78da Update contributing guide to use [all,dev] 2021-10-08 17:11:32 +01:00
Andrew Morgan e564bdd127
Add content to the Synapse documentation intro page (#10990) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-10-06 17:09:35 +00:00
Brendan Abolivier 4e51621064
Add a spamchecker method to allow or deny 3pid invites (#10894) 
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.

Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.
 2021-10-06 17:18:13 +02:00
Brendan Abolivier 829f2a82b0
Add a spamchecker callback to allow or deny room joins (#10910) 
Co-authored-by: Erik Johnston <erik@matrix.org>
 2021-10-06 14:32:16 +00:00
Max Kratz 38b7db5885
Updated development doc on samling environment for testing. (#10973) 2021-10-06 07:20:41 -04:00
Max Kratz 6744273f0b
Remove "reference" wording according Synapse homeserver (#10971) 2021-10-06 10:05:07 +00:00
Dirk Klimpel 7036a7a60a
Update links to MSCs in documentation (#10991) 
Based on matrix-doc switching from master -> main and
MSCs being merged.
 2021-10-05 11:35:19 +00:00
Richard van der Hoff e32b9f44ee
Update installation instructions (#10919) 
Various updates to the install docs.
 2021-09-29 10:57:53 +00:00
Sean Quah f7768f62cb
Avoid storing URL cache files in storage providers (#10911) 
URL cache files are short-lived and it does not make sense to offload
them (eg. to the cloud) or back them up.
 2021-09-27 12:55:27 +01:00
Brendan Abolivier b10257e879
Add a spamchecker callback to allow or deny room creation based on invites (#10898) 
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.

This adds a callback that's fairly similar to user_may_create_room except it also allows processing based on the invites sent at room creation.
 2021-09-24 16:38:23 +02:00
David Robertson ea01d4c2de
Update postgresql testing script (#10906) 
- Use sytest:bionic. Sytest:latest is two years old (do we want
  CI to push out latest at all?) and comes with Python 3.5, which we
  explictly no longer support. The script now runs under PostgreSQL 10
  as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root

A few changes to make it speedier to re-run the tests:

- Create blank DB in the container, not the script, so we don't have to
  `initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
  fetch and install a bunch of packages from PyPI each time

Co-authored-by: reivilibre <olivier@librepush.net>
 2021-09-24 14:27:09 +00:00
Patrick Cloke 52913d56a5
Add documentation for experimental feature flags. (#10865) 2021-09-22 13:41:42 +00:00
Patrick Cloke ba7a91aea5
Refactor oEmbed previews (#10814) 
The major change is moving the decision of whether to use oEmbed
further up the call-stack. This reverts the _download_url method to
being a "dumb" functionwhich takes a single URL and downloads it
(as it was before #7920).

This also makes more minor refactorings:

* Renames internal variables for clarity.
* Factors out shared code between the HTML and rich oEmbed
  previews.
* Fixes tests to preview an oEmbed image.
 2021-09-21 16:09:57 +00:00
David Robertson 60453315bd
Always add local users to the user directory (#10796) 
It's a simplification, but one that'll help make the user directory logic easier
to follow with the other changes upcoming. It's not strictly required for those
changes, but this will help simplify the resulting logic that listens for
`m.room.member` events and generally make the logic easier to follow.

This means the config option `search_all_users` ends up controlling the
search query only, and not the data we store. The cost of doing so is an
extra row in the `user_directory` and `user_directory_search` tables for
each local user which

- belongs to no public rooms
- belongs to no private rooms of size ≥ 2

I think the cost of this will be marginal (since they'll already have entries
 in `users` and `profiles` anyway).

As a small upside, a homeserver whose directory was built with this
change can toggle `search_all_users` without having to rebuild their
directory.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-09-21 12:02:34 +00:00
Charles Wright 6b6bb81b23
Fix #10837 by adding JSON encoding/decoding to the Module API example… (#10845) 2021-09-17 18:04:37 +01:00
BramvdnHeuvel 9f111075e8
Fix copy-paste error in the password section of the sample-config. (#10804) 2021-09-13 08:58:34 -04:00
reivilibre ceab5a4bfa
Fix 2 typos in docs/log_contexts.md (#10795) 2021-09-10 16:33:36 +01:00
David Robertson 318162f5de
Easy refactors of the user directory (#10789) 
No functional changes here. This came out as I was working to tackle #5677
 2021-09-10 10:54:38 +01:00
Brendan Abolivier 03caba6577
Improve the modules doc (#10758) 
* Split up the documentation in several files rather than one huge one
* Add examples for each callback category
* Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632
* Add titles to callbacks so they're easier to find in the navigation panels and link to
 2021-09-08 17:14:54 +00:00
Patrick Cloke 1fdf2cf8e8
Document that /spaces & /hierarchy can be routed to workers. (#10648) 
Also refactors some of the registration of endpoints on workers.
 2021-09-08 11:02:31 -04:00
Patrick Cloke 89ba834818
Use attrs internally for the URL preview code & add documentation. (#10753) 2021-09-07 13:10:34 +00:00
Azrenbeth 6e895366ea
Add config option to use non-default manhole password and keys (#10643) 2021-09-06 16:08:03 +01:00
David Teller e1641b46d1
Doc: Clarifying undoing room shutdowns (#10480) 2021-09-06 14:24:31 +01:00
cuttingedge1109 00640ee71a
Fix documentation of directory name for remote thumbnails (#10556) 2021-09-02 14:07:53 +01:00
Olivier Wilkinson (reivilibre) e059094119 Synapse 1.42.0rc1 (2021-09-01) 
==============================
 
 Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
 
 Features
 --------
 
 - Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
 - Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
 - Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
 - Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))
 
 Bugfixes
 --------
 
 - Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
 - Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
 - Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
 - Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
 - Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
 - Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
 - Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
 - Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
 - Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
 - Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
 - Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))
 
 Improved Documentation
 ----------------------
 
 - Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
 - Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
 - Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
 - Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
 - Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))
 
 Deprecations and Removals
 -------------------------
 
 - Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))
 
 Internal Changes
 ----------------
 
 - Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
 - Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
 - Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
 - Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
 - Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
 - Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
 - Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
 - Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
 - Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
 - Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
 - Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
 - Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
 - Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
 - Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
 - Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
 - Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmEvd3cACgkQLS76LzL7
 4EcF0hAAvOjDeH2hn0eI4YX03/BTCesJ5pCTbEJeC9Hsui3SbWamyYarH0Ae0gM6
 BP9x5/XkCdsZhKT2OJedtZQUQ0cUws5FdJP87NnE1m7KBW9djbTxmkue3JKaHNTT
 ajvEoLe8UjOFbjvyMc+188uWVgtKQ6KLcetwllTpYY3FJ3+yk9CQP1ydYzwfHcHe
 hXW3b20WoRdAmH0B27mj1oQzCeXclbj1kCthcBcxH5RytA8bBc1itBtpAP2FFaeS
 Ml6paoXSs0wZaXXL5T1MhQtcVOOG5Yh7EPsemdyfqkwhIk9QtV3nmGfC7K5iEeq0
 DldTgBNE0wLO+ycdgEZ2oCvitCLT4WDoKkVqKPA91wpuvCMeK5lnDdqbFUSN3hfd
 l8Oz+ydhjsCx7+pBBopnsyGrPIHNaN/vySYGF3lPa3ANKS2J0IVlAaJ59bZqzg2C
 vzdyMBCnn0fhnXMkADPLnIYqtebvC09JlPBRGo5BKmwmlM8Ea1dLQ+Lcy5vXalcw
 +GaYw3U805c03EvsWQxWchjwq6IwMeP06YpK/1ruMjlrCWYJp3V7f/4hXjlTdYm5
 OEWc/kHOnlLfH2cA4aS0sV5WsnEuftotQIzBXuXWw5hSMdFmH7eAviPg0vXgfeA/
 i0FOkOgEYR8O3RMh4Cip8+h44t8SPNjYUFJK9ip3DmNateeUCH0=
 =XxMc
 -----END PGP SIGNATURE-----

Merge tag 'v1.42.0rc1' into develop

Synapse 1.42.0rc1 (2021-09-01)
==============================

Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).

Features
--------

- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))

Bugfixes
--------

- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))

Improved Documentation
----------------------

- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))

Deprecations and Removals
-------------------------

- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))

Internal Changes
----------------

- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 2021-09-01 14:58:14 +01:00
Olivier Wilkinson (reivilibre) 6b2aca473a 1.42.0rc1 2021-09-01 11:47:24 +01:00
Patrick Cloke e2481dbe93
Allow configuration of the oEmbed URLs. (#10714) 
This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.
 2021-08-31 18:37:07 -04:00
David Robertson 46ff99ef95
Advertise matrix-org.github.io/synapse docs (#10595) 
Point to the book where possible, and use hyperlinks to github to refer to files not included in the book.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-08-31 11:29:27 +01:00
Azrenbeth 54aa7047eb
Removed page summaries from the top of installation and contributing doc pages (#10711) 
- Removed page summaries from CONTRIBUTING and installation pages as 
this information was already in the table of contents on the right hand side
- Fixed some broken links in CONTRIBUTING
- Added margin-right tag for when table of contents is being shown
(otherwise the text in the page sometimes overlaps with it)
 2021-08-27 14:19:17 +00:00
Richard van der Hoff 051ddac53b
Clarifications to reverse_proxy.md (#10708) 
* Update reverse_proxy.md

* Create 10708.doc
 2021-08-27 12:54:21 +01:00
Azrenbeth ad17fbd20e
Remove pushers when deleting 3pid from account (#10581) 
When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)
 2021-08-26 13:53:57 +01:00
Hugo DELVAL 86415f162d
doc: add django-oauth-toolkit to oidc doc (#10192) 
Signed-off-by: Hugo Delval <hugo.delval@gmail.com>
 2021-08-23 17:12:36 +00:00
Andrew Morgan 3e83f97154
Fix the titles in the OIDC documentation (#10639) 
* Fix the titles in the OIDC documentation

Having them as links broke the table-of-contents rendering in mdbook.
Plus there's no reason for only some of the provider titles to be links.

* Changelog

* Add link to google idp docs
 2021-08-23 14:58:31 +01:00
Callum Brown 947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142) 
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
 2021-08-21 22:14:43 +01:00
Dirk Klimpel 220f901229
Remove not needed database updates in modify user admin API (#10627) 2021-08-19 10:25:05 +01:00
Erik Johnston 78a70a2e0b Merge branch 'release-v1.41' into develop 2021-08-18 17:02:47 +01:00
Erik Johnston b9c35586a4 Update docs/upgrade.md with new version 2021-08-18 16:59:36 +01:00
Andrew Morgan 3692f7fd33
Mount /_synapse/admin/v1/users/{userId}/media admin API on media workers only (#10628) 
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
 2021-08-18 13:25:12 +01:00
Callum Brown 6e613a10d0
Display an error page during failure of fallback UIA. (#10561) 2021-08-18 08:13:35 -04:00
Dirk Klimpel 430241a1e9
Remove deprecated Shutdown Room and Purge Room Admin API (#8830) 2021-08-17 21:19:13 +01:00
Andrew Morgan 703e3a9e85
Allow /createRoom to be run on workers (#10564) 
Fixes https://github.com/matrix-org/synapse/issues/7867
 2021-08-17 14:33:16 +01:00
Azrenbeth 1a9f531c79
Port the PresenceRouter module interface to the new generic interface (#10524) 
Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
 2021-08-17 13:22:45 +00:00
Brendan Abolivier 5639759980
Centralise the custom template directory (#10596) 
Several configuration sections are using separate settings for custom template directories, which can be confusing. This PR adds a new top-level configuration for a custom template directory which is then used for every module. The only exception is the consent templates, since the consent template directory require a specific hierarchy, so it's probably better that it stays separate from everything else.
 2021-08-17 14:45:24 +02:00
reivilibre 5f7b1e1f27
Make `PeriodicallyFlushingMemoryHandler` the default logging handler. (#10518) 2021-08-17 13:13:11 +01:00
Dirk Klimpel 3bcd525b46
Allow to edit `external_ids` by Edit User admin API (#10598) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-08-17 11:56:11 +01:00
Will Hunt a933c2c7d8
Add an admin API to check if a username is available (#10578) 
This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.
 2021-08-17 10:52:38 +01:00
reivilibre 19e51b14d2
Manhole: wrap coroutines in `defer.ensureDeferred` automatically (#10602) 2021-08-16 18:11:48 +01:00
Dirk Klimpel 915b37e5ef
Admin API to delete media for a specific user (#10558) 2021-08-11 19:29:59 +00:00
Dirk Klimpel 339c3918e1
support federation queries through http connect proxy (#10475) 
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-08-11 15:34:59 +01:00
Drew Short 189c055eb6
Moved homeserver documentation above reverse proxy examples (#10551) 
Signed-off-by: Drew Short <warrick@sothr.com>
 2021-08-09 15:12:53 +00:00
Richard van der Hoff 74d7336686
Add a setting to disable TLS for sending email (#10546) 
This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.
 2021-08-06 10:13:34 +00:00
Richard van der Hoff 0cb4274dbf Merge remote-tracking branch 'origin/release-v1.40' into develop 2021-08-05 19:01:03 +01:00
Dirk Klimpel 4578531002
fix broken links in `upgrade.md` (#10543) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-08-05 19:00:44 +01:00
Dirk Klimpel 834cdc3606
Add documentation for configuring a forward proxy. (#10443) 2021-08-05 07:20:05 -04:00
Kento Okamoto 72935b7c50
Add warnings to ip_range_blacklist usage with proxies (#10129) 
Per issue #9812 using `url_preview_ip_range_blacklist` with a proxy via `HTTPS_PROXY` or `HTTP_PROXY` environment variables has some inconsistent bahavior than mentioned. This PR changes the following:

- Changes the Sample Config file to include a note mentioning that `url_preview_ip_range_blacklist` and `ip_range_blacklist` is ignored when using a proxy
- Changes some logic in synapse/config/repository.py to send a warning when both `*ip_range_blacklist` configs and a proxy environment variable are set and but no longer throws an error.

Signed-off-by: Kento Okamoto <kentokamoto@protonmail.com>
 2021-08-03 18:13:34 +00:00
Richard van der Hoff 4b10880da3
Make sync response cache time configurable. (#10513) 2021-08-03 14:45:04 +01:00
reivilibre f4ac934afe
Revert use of PeriodicallyFlushingMemoryHandler by default (#10515) 2021-08-03 11:30:39 +01:00
Eric Eastwood 2bae2c632f
Add developer documentation to explain room DAG concepts like `outliers` and `state_groups` (#10464) 2021-08-03 10:08:57 +00:00
Toni Spets ba5287f5e8
Allow setting transaction limit for db connections (#10440) 
Setting the value will help PostgreSQL free up memory by recycling
the connections in the connection pool.

Signed-off-by: Toni Spets <toni.spets@iki.fi>
 2021-08-02 13:24:43 +00:00
reivilibre e16eab29d6
Add a PeriodicallyFlushingMemoryHandler to prevent logging silence (#10407) 
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
 2021-07-27 14:32:05 +01:00
Dirk Klimpel 89c4ca81bb
Add `creation_ts` to list users admin API (#10448) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-07-22 16:05:16 +02:00
Eric Eastwood d518b05a86
Move dev/ docs to development/ (#10453) 2021-07-22 12:58:24 +02:00
Patrick Cloke 5b68816de9
Fix the hierarchy of OpenID providers in the docs. (#10445) 2021-07-21 13:48:06 -04:00
Brendan Abolivier a743bf4694
Port the ThirdPartyEventRules module interface to the new generic interface (#10386) 
Port the third-party event rules interface to the generic module interface introduced in v1.37.0
 2021-07-20 12:39:46 +02:00
Brendan Abolivier 36dc15412d
Add a module type for account validity (#9884) 
This adds an API for third-party plugin modules to implement account validity, so they can provide this feature instead of Synapse. The module implementing the current behaviour for this feature can be found at https://github.com/matrix-org/synapse-email-account-validity.

To allow for a smooth transition between the current feature and the new module, hooks have been added to the existing account validity endpoints to allow their behaviours to be overridden by a module.
 2021-07-16 18:11:53 +02:00
Brendan Abolivier 9f497024aa Merge branch 'master' into develop 2021-07-15 14:54:45 +01:00
Brendan Abolivier 3fffb71254
Make deprecation notice of the spam checker doc more obvious (#10395) 2021-07-15 15:54:22 +02:00
Luke Walsh 23a90a6a5c
Updating install prerequisites for newer macOS & ARM Macs. (#9971) 2021-07-15 13:18:58 +01:00
Andrew Morgan c141455049
Docs: Use something other than the document name to describe a page (#10399) 
Our documentation has a history of using a document's name as a way to link to it, such as "See [workers.md]() for details". This makes sense when you're traversing a directory of files, but less sense when the files are abstracted away - as they are on the documentation website.

This PR changes the links to various documentation pages to something that fits better into the surrounding sentence, as you would when making any hyperlink on the web.
 2021-07-15 12:47:55 +01:00
Brendan Abolivier 5ecad4e7a5
Update the logcontext doc (#10353) 
By referring to awaitables instead of deferreds.
 2021-07-15 11:38:05 +01:00
Moritz Dietz 0ae95b3847
doc: Add delegation example to the caddy reverse proxy section (#10368) 2021-07-14 18:50:30 +02:00
Richard van der Hoff 28ffff73c1
Instructions on installing RC debs (#10396) 2021-07-14 17:12:01 +01:00
Dirk Klimpel e938f69697
Fix some links in `docs` and `contrib` (#10370) 2021-07-13 11:55:48 +01:00
Erik Johnston 251cfc4e09 Synapse 1.38.0rc2 (2021-07-09) 
==============================
 
 Bugfixes
 --------
 
 - Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](https://github.com/matrix-org/synapse/issues/10336))
 
 Improved Documentation
 ----------------------
 
 - Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](https://github.com/matrix-org/synapse/issues/10287))
 - Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](https://github.com/matrix-org/synapse/issues/10331))
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAmDoH+4QHGVyaWtAbWF0
 cml4Lm9yZwAKCRClQuTtGw+sCXxYCACneuRvkdvYqiH+PhPe8tXqhhJIifH1LecY
 FlJqp4OJPR2VFzio1btsgpRPQyLBLHZkJ9pgWsXAETbYOO+hSeOc4nIHsyqlSJhe
 v01sCUE4sle3DBrw15fG4XpercsiM3TFMyR9pV9laq9nIn8j+CY5K6W5t12/mYGy
 asHS0IKilCMhJlFwgE3eBr6P6fywi0JoIrr8EpfIs4eC2qDFpUlsrAQSkbE1JvdP
 O4BGZJKVysg3a6WYSWdJytqLYe942k8qUF4B4h4VmQi0xbuKSsTLiK/cFC8ohRMv
 E+O5O/KgwqwE/XOcukbsjlHxuiiFZTq6154PwLxXUpNnsMNn2/ph
 =6iBw
 -----END PGP SIGNATURE-----

Merge tag 'v1.38.0rc2' into develop

Synapse 1.38.0rc2 (2021-07-09)
==============================

Bugfixes
--------

- Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](https://github.com/matrix-org/synapse/issues/10336))

Improved Documentation
----------------------

- Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](https://github.com/matrix-org/synapse/issues/10287))
- Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](https://github.com/matrix-org/synapse/issues/10331))
 2021-07-09 11:26:17 +01:00
Brendan Abolivier 33ae301fee
Fix formatting in the logcontext doc (#10337) 2021-07-08 18:16:30 +02:00
Cristina f6767abc05
Remove functionality associated with unused historical stats tables (#9721) 
Fixes #9602
 2021-07-08 16:57:13 +01:00
Dirk Klimpel 974261cd81
Fix broken links in INSTALL.md (#10331) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-07-08 15:46:13 +01:00
Richard van der Hoff 189652b2fe
Fix a broken link in the admin api docs (#10322) 
* Fix a broken link in the admin api docs

* Rename 10321.doc to 10321.docs

* Rename 10321.docs to 10322.doc
 2021-07-07 12:54:57 +01:00
Richard van der Hoff 240b3ce253 Merge remote-tracking branch 'origin/release-v1.38' into develop 
merge @dklimpel's fixes to the sample config
 2021-07-07 12:37:10 +01:00
Dirk Klimpel 56fd5fa8e1
Update links to documentation in sample config (#10287) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-07-07 12:35:45 +01:00
Dirk Klimpel 2d044667cf
Simplify structure of room admin API docs (#10313) 2021-07-07 12:18:36 +01:00
Richard van der Hoff d7a94a7dcc
Add upgrade notes about disk space for events migration (#10314) 2021-07-06 11:00:05 +01:00
Erik Johnston 7a5873277e
Add support for evicting cache entries based on last access time. (#10205) 2021-07-05 16:32:12 +01:00
Brendan Abolivier 10671da05b
Fix bad link in modules documentation (#10302) 
Fix link in modules doc to point at instructions on registering a callback instead of ones on registering a web resource.
 2021-07-02 13:20:43 +02:00
Dirk Klimpel 6c02cca95f
Add SSO `external_ids` to Query User Account admin API (#10261) 
Related to #10251
 2021-07-01 11:26:24 +02:00
Andrew Morgan 04c8f308f4
Fix the homeserver config example in presence router docs (#10288) 
The presence router docs include some sample homeserver config. At some point we changed the name of the [config option](859dc05b36/docs/sample_config.yaml (L104-L113)), but forgot to update the docs.

I've also added `presence.enabled: true` to the example, as that's the new way to enable presence (the `presence_enabled` option has been deprecated).
 2021-06-30 23:43:58 +01:00
Andrew Morgan 7f25d73859
Convert UPGRADE.rst to markdown (#10166) 
This PR:

* Converts UPGRADE.rst to markdown and moves the contents into the `docs/` directory.
* Updates the contents of UPGRADE.rst to point to the website instead.
* Updates links around the codebase that point to UPGRADE.rst.

`pandoc` + some manual editing was used to convert from RST to md.
 2021-06-23 16:57:57 +01:00
jkanefendt 756fd513df
Implement config option `sso.update_profile_information` (#10108) 
Implemented config option sso.update_profile_information to keep user's display name in sync with the SSO displayname.

Signed-off-by: Johannes Kanefendt <johannes.kanefendt@krzn.de>
 2021-06-21 23:48:57 +01:00
Brendan Abolivier e9f2ad8603
Describe callbacks signatures as async in new modules doc (#10206) 2021-06-18 16:55:53 +02:00
Brendan Abolivier 1b3e398bea
Standardise the module interface (#10062) 
This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
 2021-06-18 12:15:52 +01:00
Brendan Abolivier 08c8469322
Remove support for ACME v1 (#10194) 
Fixes #9778

ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
 2021-06-17 18:56:48 +01:00
Patrick Cloke 76f9c701c3
Always require users to re-authenticate for dangerous operations. (#10184) 
Dangerous actions means deactivating an account, modifying an account
password, or adding a 3PID.

Other actions (deleting devices, uploading keys) can re-use the same UI
auth session if ui_auth.session_timeout is configured.
 2021-06-16 11:07:28 -04:00
Michael Kaye b8b282aa32
A guide to the request log lines format. (#8436) 
This doc is short but a useful guide to what the request log lines mean.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
Co-authored-by: Daniele Sluijters <daenney@users.noreply.github.com>
 2021-06-16 13:31:55 +01:00
Dirk Klimpel 0adc2882c1
Fix broken links in documentation (#10180) 
* Fix broken links in documentation

* newsfile
 2021-06-16 13:15:52 +01:00
Richard van der Hoff 13577aa55e
Notes on boolean columns in database schemas (#10164) 2021-06-11 17:13:56 +01:00
Richard van der Hoff c1b9922498
Support for database schema version ranges (#9933) 
This is essentially an implementation of the proposal made at https://hackmd.io/@richvdh/BJYXQMQHO, though the details have ended up looking slightly different.
 2021-06-11 14:45:53 +01:00
Aaron Raimist e6245e6d48
Mention that you need to configure max upload size in reverse proxy as well (#10122) 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2021-06-10 11:40:24 +01:00
Dan Callahan 7dc14730d9
Name release branches just after major.minor (#10013) 
With the prior format, 1.33.0 / 1.33.1 / 1.33.2 got separate branches:

    release-v1.33.0
    release-v1.33.1
    release-v1.33.2

Under the new model, all three would share a common branch:

    release-v1.33

As before, RCs and actual releases exist as tags on these branches.

This better reflects our support model, e.g., that the "1.33" series had
a formal release followed by two patches / updates.

Signed-off-by: Dan Callahan <danc@element.io>
 2021-06-08 11:44:50 +01:00
Rohan Sharma beb251e3ee
Make link in docs use HTTPS (#10130) 
Fixes #10121

Signed-off-by: Rohan Sharma <rhnsharma5113@gmail.com>
 2021-06-07 16:35:02 +01:00
Andrew Morgan fd9856e4a9
Compile and render Synapse's docs into a browsable, mobile-friendly and searchable website (#10086) 2021-06-03 17:20:40 +01:00
Andrew Morgan 73636cab69
Convert admin api docs to markdown (#10089) 
So that they render nicely in mdbook (see #10086), and so that we no longer have a mix of structured text languages in our documentation (excluding files outside of `docs/`).
 2021-06-03 14:06:03 +01:00
Dirk Klimpel 0284d2a297
Add new admin APIs to remove media by media ID from quarantine. (#10044) 
Related to: #6681, #5956, #10040

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-06-02 18:50:35 +01:00
Erik Johnston 5eed6348ce
Move some more endpoints off master (#10084) 2021-05-27 22:45:43 +01:00
Callum Brown 8fb9af570f
Make reason and score optional for report_event (#10077) 
Implements MSC2414: https://github.com/matrix-org/matrix-doc/pull/2414
See #8551 

Signed-off-by: Callum Brown <callum@calcuode.com>
 2021-05-27 18:42:23 +01:00
Patrick Cloke f42e4c4eb9
Remove the experimental spaces enabled flag. (#10063) 
In lieu of just always enabling the unstable spaces endpoint and
unstable room version.
 2021-05-26 14:35:16 -04:00
Dirk Klimpel 65e6c64d83
Add an admin API for unprotecting local media from quarantine (#10040) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-05-26 11:19:47 +01:00
Aaron Raimist 3e1beb75e6
Update CAPTCHA documentation to mention turning off verify origin feature (#10046) 
* Update CAPTCHA documentation to mention turning off verify origin

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2021-05-26 10:55:30 +01:00
Jerin J Titus 057ce7b754
Remove tls_fingerprints option (#9280) 
Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
 2021-05-24 17:43:30 +01:00
Patrick Cloke 316f89e87f
Enable experimental spaces by default. (#10011) 
The previous spaces_enabled flag now defaults to true and
is exposed in the sample config.
 2021-05-24 08:57:14 -04:00
Dirk Klimpel 387c297489
Add missing entry to the table of contents of room admin API (#10043) 2021-05-24 07:37:30 -04:00
Andrew Morgan d983ced596 Merge branch 'master' of github.com:matrix-org/synapse into develop 2021-05-20 14:39:46 +01:00
Javier Junquera Sánchez 141b073c7b
Update user_directory.md (#10016) 
Signed-off-by: Javier Junquera Sánchez <javier@junquera.io>
 2021-05-20 14:24:19 +01:00
Savyasachee Jha 5bba1b4905
Hardened systemd unit files (#9803) 
Signed-off-by: Savyasachee Jha savya.jha@hawkradius.com
 2021-05-19 11:44:16 +01:00
Andrew Morgan 4d6e5a5e99
Use a database table to hold the users that should have full presence sent to them, instead of something in-memory (#9823) 2021-05-18 14:13:45 +01:00
Richard van der Hoff 6660912226
Update postgres docs (#9989) 2021-05-14 13:14:48 +01:00
Richard van der Hoff c14f99be46
Support enabling opentracing by user (#9978) 
Add a config option which allows enabling opentracing by user id, eg for
debugging requests made by a test user.
 2021-05-14 10:51:08 +01:00
Andrew Morgan 976216959b
Update minimum supported version in postgres.md (#9988) 2021-05-14 09:21:00 +01:00
Patrick Cloke d19bccdbec
Update SSO mapping providers documentation about unique IDs. (#9980) 2021-05-13 14:37:20 -04:00
Patrick Cloke 28c6841102
Send the `m.room.create` stripped event with invites (support MSC1772). (#9966) 
MSC1772 specifies the m.room.create event should be sent as part
of the invite_state. This was done optionally behind an experimental
flag, but is now done by default due to MSC1772 being approved.
 2021-05-11 10:58:58 -04:00
Aaron Raimist dc6366a9bd
Add config option to hide device names over federation (#9945) 
Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers.

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2021-05-11 14:03:23 +01:00
Erik Johnston 1fb9a2d0bf
Limit how often GC happens by time. (#9902) 
Synapse can be quite memory intensive, and unless care is taken to tune
the GC thresholds it can end up thrashing, causing noticable performance
problems for large servers. We fix this by limiting how often we GC a
given generation, regardless of current counts/thresholds.

This does not help with the reverse problem where the thresholds are set
too high, but that should only happen in situations where they've been
manually configured.

Adds a `gc_min_seconds_between` config option to override the defaults.

Fixes #9890.
 2021-05-05 16:53:45 +01:00
Andrew Morgan b85821aca2
Add port parameter to the sample config for psycopg2 args (#9911) 
Adds the `port` option with the default value to the sample config file.
 2021-05-04 13:28:59 +01:00
ThibF 0085dc5abc
Delete room endpoint (#9889) 
Support the delete of a room through DELETE request and mark
previous request as deprecated through documentation.

Signed-off-by: Thibault Ferrante <thibault.ferrante@pm.me>
 2021-04-29 10:31:45 +01:00
Patrick Cloke 683d6f75af
Rename handler and config modules which end in handler/config. (#9816) 2021-04-20 14:55:20 -04:00
Andrew Morgan 71f0623de9
Port "Allow users to click account renewal links multiple times without hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832) 
This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).
 2021-04-19 19:16:34 +01:00
Dirk Klimpel 1fc97ee876
Add an admin API to manage ratelimit for a specific user (#9648) 2021-04-13 10:26:37 +01:00
Andrew Morgan 04819239ba
Add a Synapse Module for configuring presence update routing (#9491) 
At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though.

This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around.

A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. 

The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being:

* Sending state for a specific set or all known users to a defined set of local and remote users.
* The ability to trigger an initial sync for specific users, so they receive all current state.
 2021-04-06 14:38:30 +01:00
Erik Johnston 5688a74cf3 Merge branch 'master' into develop 2021-04-06 13:29:29 +01:00
Erik Johnston a888cbdd31
Add deprecation policy doc (#9723) 
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2021-04-06 13:07:36 +01:00
Dirk Klimpel bb0fe02a52
Add `order_by` to list user admin API (#9691) 2021-04-01 11:28:53 +01:00
Denis Kasak 5ff8eb97c6
Make sample config allowed_local_3pids regex stricter. (#9719) 
The regex should be terminated so that subdomain matches of another
domain are not accepted. Just ensuring that someone doesn't shoot
themselves in the foot by copying our example.

Signed-off-by: Denis Kasak <dkasak@termina.org.uk>
 2021-03-31 12:27:20 +00:00
Richard van der Hoff f02663c4dd
Replace `room_invite_state_types` with `room_prejoin_state` (#9700) 
`room_invite_state_types` was inconvenient as a configuration setting, because
anyone that ever set it would not receive any new types that were added to the
defaults. Here, we deprecate the old setting, and replace it with a couple of
new settings under `room_prejoin_state`.
 2021-03-30 12:12:44 +01:00
Richard van der Hoff 4bbd535450
Update the OIDC sample config (#9695) 
I've reiterated the advice about using `oidc` to migrate, since I've seen a few
people caught by this.

I've also removed a couple of the examples as they are duplicating the OIDC
documentation, and I think they might be leading people astray.
 2021-03-29 15:40:11 +01:00
Paul Tötterman d7d4232a2d
Preserve host in example apache config (#9696) 
Fixes redirect loop

Signed-off-by: Paul Tötterman <paul.totterman@iki.fi>
 2021-03-26 10:38:31 +00:00
Ben Banfield-Zanin 4655d2221e
docs: fallback/web endpoint does not appear to be mounted on workers (#9679) 2021-03-24 11:43:04 +00:00
Erik Johnston e550ab17ad
Increase default join burst ratelimiting (#9674) 
It's legitimate behaviour to try and join a bunch of rooms at once.
 2021-03-23 14:52:20 +00:00
Hubbe dd5e5dc1d6
Add SSO attribute requirements for OIDC providers (#9609) 
Allows limiting who can login using OIDC via the claims
made from the IdP.
 2021-03-16 11:46:07 -04:00
Andrew Morgan 847ecdd8fa
Pass SSO IdP information to spam checker's registration function (#9626) 
Fixes https://github.com/matrix-org/synapse/issues/9572

When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time.

This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.
 2021-03-16 12:41:41 +00:00
Richard van der Hoff dd69110d95
Add support for stable MSC2858 API (#9617) 
The stable format uses different brand identifiers, so we need to support two
identifiers for each IdP.
 2021-03-16 11:21:26 +00:00
Richard van der Hoff 5b5bc188cf
Clean up config settings for stats (#9604) 
... and complain if people try to turn it off.
 2021-03-16 10:57:54 +00:00
Jason Robinson 17cd48fe51
Fix spam checker modules documentation example (#9580) 
Mention that parse_config must exist and note the
check_media_file_for_spam method.
 2021-03-10 10:42:51 -05:00
Andrew Morgan e9df3f496b
Link to the List user's media admin API from media Admin API docs (#9571) 
Earlier [I was convinced](https://github.com/matrix-org/synapse/issues/9565) that we didn't have an Admin API for listing media uploaded by a user. Foolishly I was looking under the Media Admin API documentation, instead of the User Admin API documentation.

I thought it'd be helpful to link to the latter so others don't hit the same dead end :)
 2021-03-09 15:15:52 +00:00
Richard van der Hoff eaada74075
JWT OIDC secrets for Sign in with Apple (#9549) 
Apple had to be special. They want a client secret which is generated from an EC key.

Fixes #9220. Also fixes #9212 while I'm here.
 2021-03-09 15:03:37 +00:00
Erik Johnston 576c91c7c1 Fixup sample config 
After 0764d0c6e5
 2021-03-09 11:40:45 +00:00
Leo Bärring 0fc4eb103a
Update reverse proxy to add OpenBSD relayd example configuration. (#9508) 
Update reverse proxy to add OpenBSD relayd example configuration.

Signed-off-by: Leo Bärring <leo.barring@protonmail.com>
 2021-03-06 11:49:19 +00:00
Richard van der Hoff a5daae2a5f
Update nginx reverse-proxy docs (#9512) 
Turns out nginx overwrites the Host header by default.
 2021-03-03 11:08:11 +00:00
Richard van der Hoff d8e95e5452
Add support for X-Forwarded-Proto (#9472) 
rewrite XForwardedForRequest to set `isSecure()` based on
`X-Forwarded-Proto`. Also implement `getClientAddress()` while we're here.
 2021-02-24 18:11:33 +00:00
Andrew Morgan 00bf80cb8e Fix typo in spam checker documentation 2021-02-24 17:51:52 +00:00
Erik Johnston 66f4949e7f
Fix deleting pushers when using sharded pushers. (#9465) 2021-02-22 21:14:42 +00:00
Dirk Klimpel 71c9f8de6d
Add an `order_by` field to list users' media admin API. (#8978) 2021-02-22 14:38:51 -05:00
Richard van der Hoff 70ea9593ff
example systemd config: propagate reloads to units (#9463) 
It should be possible to reload `synapse.target` to have the reload propagate
to all the synapse units.
 2021-02-22 18:43:17 +00:00
Andrew Morgan e22b71810e
Clean up the user directory sample config section (#9385) 
The user directory sample config section was a little messy, and didn't adhere to our [recommended config format guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).

This PR cleans that up a bit.
 2021-02-22 11:44:31 +00:00
Andrew Morgan 13e9029f44
Add a config option to prioritise local users in user directory search results (#9383) 
This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default.

Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical.

This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.
 2021-02-19 11:02:03 +00:00
AndrewFerr 9bc74743d5
Add configs to make profile data more private (#9203) 
Add off-by-default configuration settings to:
- disable putting an invitee's profile info in invite events
- disable profile lookup via federation

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
 2021-02-19 09:50:41 +00:00
Rishabh Arya e17553e185
Parse ui_auth.session_timeout as a duration (instead of treating it as ms) (#9426) 2021-02-18 09:18:14 -05:00
David Vo bb2577f6b7
Add http2 to the nginx example config (#9390) 2021-02-18 08:46:16 -05:00
Richard van der Hoff 626afd7e89 Revert "Update workers.md" 
This reverts commit a8878960c0.
 2021-02-18 11:56:25 +00:00
Dirk Klimpel c8d9383cfb
Add the shadow-banning status to the display user admin API. (#9400) 2021-02-17 15:19:23 -05:00
Richard van der Hoff 53f1c4da81
Update workers.md 2021-02-17 17:14:23 +00:00
Richard van der Hoff a8878960c0
Update workers.md 
tiny typo in sso paths
 2021-02-17 17:11:24 +00:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381) 
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
 2021-02-16 22:32:34 +00:00
Marcus 5636e597c3
Fix OIDC gitiea redirect URL. (#9404) 
Fixes a "conflict" from 846b9d3df0
and d1f13c7485.
 2021-02-16 14:06:55 -05:00
Erik Johnston ddfdf94506
Document that pusher instances are shardable (#9407) 2021-02-16 13:27:49 +00:00
Andrew Morgan ff40c8099d Fix sample config 
Just a small change missed in 7950aa8a27.
 2021-02-12 22:18:40 +00:00
Patrick Cloke 2c9b4a5f16 Synapse 1.27.0rc2 (2021-02-11) 
==============================
 
 Features
 --------
 
 - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297))
 
 Bugfixes
 --------
 
 - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302))
 - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313))
 
 Improved Documentation
 ----------------------
 
 - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAlX4AACgkQM/xY9qcR
 MEjxhg/+JzOGDmgxy4Vm9oU84htkkJsflU755ykxhwbIjVy2j+07pMgtWMmebXIS
 /tWjinwRB9OgKIK+j5RWdYzz/GRTkLCbafTLhhNOWsnaCO7nnAfZFozLOMd9g2qN
 309cS8efZkxGCISGSRm1QQKjQDx6HHx7nZVAwpqb778Q9TMrBYRAQeCk9yws7FfL
 GsP5YrBI42n84YhvflWA8J8QRYkeJJaggVWzE0XvwdI6raY1BmxMKcDvyl9iKpGb
 /QXsdketD3eJxdOlU92O5ZxRXmL00bdyAFdw2+J3Y+wjoGBnC5njKSFG3j8Z2UcP
 xIdB6w/zyGoPLKd4s7tkDI3axE7qrwFetA4NytannWGEHZ9q5tgOd5PA1kTeKYvn
 ao2os4pKMjWQOHiWLskqZVXgmoW5Kb2zgyZU/vhFAz8ImuidFULOjpEytAWqtt1B
 iuL/fRlM3z0BpwQNd3NBsblYTpZri8gdTp7ULJYtaKpT2MeG0sg3swJAptUjw9L4
 awbQuBFZ8o/E/0xwiGieSxbR6b6Nz5WiCyLaeMz6b3Y4YdomttOAZ6pL6E9y8ygU
 2wKYleSmg+tbaBKKZR5CSCwXOduSYPhWw7oqsfEfvT0NCQLuItfGNFZqxRzsf73H
 lCV0wwxNZBv8kRyMs6KivMvSI6SKinAjXS8AjzHE402ozdwOCv0=
 =NSyp
 -----END PGP SIGNATURE-----

Merge tag 'v1.27.0rc2' into develop

Synapse 1.27.0rc2 (2021-02-11)
==============================

Features
--------

- Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297))

Bugfixes
--------

- Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302))
- Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313))

Improved Documentation
----------------------

- Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))
 2021-02-11 11:56:03 -05:00
Patrick Cloke e40d88cff3
Backout changes for automatically calculating the public baseurl. (#9313) 
This breaks some people's configurations (if their Client-Server API
is not accessed via port 443).
 2021-02-11 11:16:54 -05:00
Thomas Mortagne 5f716fa777
Add XWiki OIDC provider example. (#9324) 2021-02-09 11:54:52 -05:00
David Teller b0b2cac057
Merge pull request #9150 from Yoric/develop-context 
New API /_synapse/admin/rooms/{roomId}/context/{eventId}
 2021-02-08 15:53:44 +01:00
Erik Johnston adc96d4236 Merge branch 'erikj/media_spam_checker' into develop 2021-02-04 17:01:59 +00:00
Erik Johnston 7e8083eb48 Add check_media_file_for_spam spam checker hook 2021-02-04 17:01:30 +00:00
dykstranet 982d9eb211
Correct `matrix-synapse.service` reference in TURN howto docs. (#9308) 2021-02-04 11:22:44 -05:00
Patrick Cloke 2ab6e67ab7
Fix escaping of braces in OIDC sample config. (#9317) 
This fixes the Jinja2 templates for the mapping provider.
 2021-02-04 09:06:20 -05:00
Richard van der Hoff 3f534d3fdf Merge branch 'social_login_hotfixes' into develop 2021-02-03 20:34:27 +00:00
Patrick Cloke afa18f1baa
Clarify documentation about escaping URLs in templates. (#9310) 2021-02-03 14:51:38 -05:00
dykstranet e4cdecb310
config: Add detail to auto_join_rooms comment (#9291) 
config: Add detail to auto_join_rooms comment

Signed-off-by: Gary Dykstra <gary@dykstranet.com>
 2021-02-03 15:21:30 +00:00
Patrick Cloke 4ca054a4ea
Convert blacklisted IPv4 addresses to compatible IPv6 addresses. (#9240) 
Also add a few more IP ranges to the default blacklist.
 2021-02-03 07:13:46 -05:00
Travis Ralston b60bb28bbc
Add an admin API to get the current room state (#9168) 
This could arguably replace the existing admin API for `/members`, however that is out of scope of this change.

This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.
 2021-02-02 11:16:29 +00:00
Richard van der Hoff 8f75bf1df7
Put SAML callback URI under /_synapse/client. (#9289) 2021-02-02 09:43:50 +00:00
Richard van der Hoff 846b9d3df0
Put OIDC callback URI under /_synapse/client. (#9288) 2021-02-01 22:56:01 +00:00
Oliver Hanikel d1f13c7485
Add an OpenID example config for Gitea. (#9134) 2021-02-01 16:21:09 -05:00
Richard van der Hoff 5963426b95 Merge branch 'social_login' into develop 2021-02-01 18:46:12 +00:00
Richard van der Hoff c543bf87ec
Collect terms consent from the user during SSO registration (#9276) 2021-02-01 18:37:41 +00:00
Richard van der Hoff e5d70c8a82
Improve styling and wording of SSO UIA templates (#9286) 
fixes #9171
 2021-02-01 18:36:04 +00:00
Richard van der Hoff 18ab35284a Merge branch 'social_login' into develop 2021-02-01 17:28:37 +00:00
Richard van der Hoff 4167494c90
Replace username picker with a template (#9275) 
There's some prelimiary work here to pull out the construction of a jinja environment to a separate function.

I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.
 2021-02-01 15:52:50 +00:00
Richard van der Hoff 8aed29dc61
Improve styling and wording of SSO redirect confirm template (#9272) 2021-02-01 15:50:56 +00:00
Richard van der Hoff 9c715a5f19
Fix SSO on workers (#9271) 
Fixes #8966.

* Factor out build_synapse_client_resource_tree

Start a function which will mount resources common to all workers.

* Move sso init into build_synapse_client_resource_tree

... so that we don't have to do it for each worker

* Fix SSO-login-via-a-worker

Expose the SSO login endpoints on workers, like the documentation says.

* Update workers config for new endpoints

Add documentation for endpoints recently added (#8942, #9017, #9262)

* remove submit_token from workers endpoints list

this *doesn't* work on workers (yet).

* changelog

* Add a comment about the odd path for SAML2Resource
 2021-02-01 15:47:59 +00:00
Erik Johnston f2c1560eca
Ratelimit invites by room and target user (#9258) 2021-01-29 16:38:29 +00:00
Richard van der Hoff 0d81a6fa3e Merge branch 'social_login' into develop 2021-01-28 22:08:11 +00:00
Erik Johnston 4b73488e81
Ratelimit 3PID /requestToken API (#9238) 2021-01-28 17:39:21 +00:00
David Teller fe52dae6bd FIXUP: Documenting /_synapse/admin/v1/rooms/<room_id>/context/<event_id> 2021-01-28 12:30:21 +01:00
Richard van der Hoff 34efb4c604
Add notes on integrating with Facebook for SSO login. (#9244) 2021-01-27 22:57:16 +00:00
Richard van der Hoff a083aea396
Add 'brand' field to MSC2858 response (#9242) 
We've decided to add a 'brand' field to help clients decide how to style the
buttons.

Also, fix up the allowed characters for idp_id, while I'm in the area.
 2021-01-27 21:31:45 +00:00
Richard van der Hoff 869667760f
Support for scraping email addresses from OIDC providers (#9245) 2021-01-27 21:28:59 +00:00
Erik Johnston 93b61589b0
Add a note to changelog about redis usage (#9227) 2021-01-27 14:06:27 +00:00
Patrick Cloke 26837d5dbe
Do not require the CAS service URL setting (use public_baseurl instead). (#9199) 
The current configuration is handled for backwards compatibility,
but is considered deprecated.
 2021-01-26 10:49:25 -05:00
Jason Robinson cee4010f94 Merge branch 'develop' into jaywink/admin-forward-extremities 
# Conflicts:
#	synapse/rest/admin/__init__.py
 2021-01-26 10:15:32 +02:00
Patrick Cloke 4a55d267ee
Add an admin API for shadow-banning users. (#9209) 
This expands the current shadow-banning feature to be usable via
the admin API and adds documentation for it.

A shadow-banned users receives successful responses to their
client-server API requests, but the events are not propagated into rooms.

Shadow-banning a user should be used as a tool of last resort and may lead
to confusing or broken behaviour for the client.
 2021-01-25 14:49:39 -05:00
Jason Robinson 8965b6cfec Merge branch 'develop' into jaywink/admin-forward-extremities 2021-01-23 21:41:35 +02:00
Jason Robinson 930ba00971 Add depth and received_ts to forward_extremities admin API response 
Also add a warning on the admin API documentation.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2021-01-23 21:34:32 +02:00
Richard van der Hoff b5120f09f1 Merge remote-tracking branch 'origin/release-v1.26.0' into develop 2021-01-21 13:17:07 +00:00
Richard van der Hoff 7447f19702
Prefix idp_id with "oidc-" (#9189) 
... to avoid clashes with other SSO mechanisms
 2021-01-21 12:25:02 +00:00
Patrick Cloke f81d02d75b Synapse 1.26.0rc1 (2021-01-20) 
==============================
 
 This release brings a new schema version for Synapse and rolling back to a previous
 verious is not trivial.  Please review [UPGRADE.rst](UPGRADE.rst) for more details
 on these changes  and for general upgrade guidance.
 
 Features
 --------
 
 - Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177))
 - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091))
 - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159))
 - Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024))
 - Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984))
 - Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086))
 - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932))
 - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948))
 - Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130))
 - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068))
 - Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092))
 - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166))
 
 Bugfixes
 --------
 
 - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023))
 - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028))
 - Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051))
 - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053))
 - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054))
 - Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059))
 - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070))
 - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071))
 - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116))
 - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117))
 - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128))
 - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108))
 - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145))
 - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161))
 
 Improved Documentation
 ----------------------
 
 - Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997))
 - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035))
 - Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040))
 - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057))
 - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151))
 
 Deprecations and Removals
 -------------------------
 
 - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039))
 
 Internal Changes
 ----------------
 
 - Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124))
 - Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939))
 - Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016))
 - Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018))
 - Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025))
 - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030))
 - Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031))
 - Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033))
 - Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038))
 - Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041))
 - Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055))
 - Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058))
 - Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063))
 - Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069))
 - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080))
 - Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093))
 - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098))
 - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106))
 - Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112))
 - Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125))
 - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144))
 - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146))
 - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAIVFkACgkQM/xY9qcR
 MEhLfQ//Z4xnP5Icx732k2xpdR+JaXX6heNVFm/UuLNd6wq+uXjO/Dgc9IKQ5LXD
 kfG+/OnNGDr/eUk4mQ5s/ccBvaTrOid2IIemJd4lUBy/Q5LvPvfKIp08QFr72WJT
 7Y4Ma54nNSSMUAdESgj/aFAHjATbfHhxPOZ5OdQHGLAuJ/OUwR2ksasVNCuINbgh
 8vrxrzjeYn1Zl3UTBrsdCcat7AFqYjxK/Y0i6JxfSgRwDZjUbW4M2C4OGWZJPUrU
 2YTPHangxd+22+HObE2KDEzaHDdCR/Kj4dhCiJBynZ5iucuHSS+mgf39RtbjqWML
 01cmXXUV+FH7rnCilmF9rYdmHZ6L9BJxtrQDQU4dB106BX9G7hKR6rYHc9Z6tjOh
 3tg0cKWYMXOsTNnvRqiFNh2q0yBKS228HRAwyGosW/6NiYj43ArtYP6knX7dQyBX
 0mYNkfry2dyX6Kj5el0i9MOTHYQxfc8apsBm2M3OTNYukgQKA0NhbAquibaAxEEW
 2qIqbSp4CUlGPvM7+u+ZGTu0hEbVKKNjqMfXhuY8A4/BWiV2mSqKcEirR5cE/LqY
 mq5Ac7vbO4Uh+1xiRw/G9ITi1dqAjYIzhBawlhdUPOh+aINWTUikYYzjUSmm4PbN
 H9BTPLA9iAdVRfuWJ9um2G0DdS8Qpx/aIRh3MScXVly6cGCC6Do=
 =kgqn
 -----END PGP SIGNATURE-----

Merge tag 'v1.26.0rc1' into develop

Synapse 1.26.0rc1 (2021-01-20)
==============================

This release brings a new schema version for Synapse and rolling back to a previous
verious is not trivial.  Please review [UPGRADE.rst](UPGRADE.rst) for more details
on these changes  and for general upgrade guidance.

Features
--------

- Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177))
- During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091))
- Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159))
- Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024))
- Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984))
- Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086))
- Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932))
- Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948))
- Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130))
- Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068))
- Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092))
- Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166))

Bugfixes
--------

- Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023))
- Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028))
- Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051))
- Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053))
- Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054))
- Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059))
- Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070))
- Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071))
- Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116))
- Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117))
- Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128))
- Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108))
- Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145))
- Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161))

Improved Documentation
----------------------

- Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997))
- Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035))
- Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040))
- Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057))
- Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151))

Deprecations and Removals
-------------------------

- Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039))

Internal Changes
----------------

- Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124))
- Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939))
- Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016))
- Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018))
- Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025))
- Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030))
- Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031))
- Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033))
- Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038))
- Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041))
- Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055))
- Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058))
- Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063))
- Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069))
- Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080))
- Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093))
- Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098))
- Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106))
- Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112))
- Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125))
- Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144))
- Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146))
- Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157))
 2021-01-20 11:27:39 -05:00
Richard van der Hoff 0cd2938bc8
Support icons for Identity Providers (#9154) 2021-01-20 08:15:14 -05:00
rht a5b9c87ac6
docs: Add link to Matrix VoIP tester for turn-howto (#9135) 
Signed-off-by: rht <rhtbot@protonmail.com>
 2021-01-20 12:41:57 +00:00
Richard van der Hoff fa50e4bf4d
Give `public_baseurl` a default value (#9159) 2021-01-20 12:30:41 +00:00
Patrick Cloke de45bf5b5b
Quote pip install with brackets to avoid shell interpretation. (#9151) 2021-01-18 11:12:20 -05:00
Matthew Hodgson 883d4e6f2b link to the scalability blog post from workers.md 2021-01-18 00:27:27 +00:00
Richard van der Hoff 9de6b94117
Land support for multiple OIDC providers (#9110) 
This is the final step for supporting multiple OIDC providers concurrently.

First of all, we reorganise the config so that you can specify a list of OIDC providers, instead of a single one. Before:

    oidc_config:
       enabled: true
       issuer: "https://oidc_provider"
       # etc

After:

    oidc_providers:
     - idp_id: prov1
       issuer: "https://oidc_provider"

     - idp_id: prov2
       issuer: "https://another_oidc_provider"

The old format is still grandfathered in.

With that done, it's then simply a matter of having OidcHandler instantiate a new OidcProvider for each configured provider.
 2021-01-15 16:55:29 +00:00
Patrick Cloke 3e4cdfe5d9
Add an admin API endpoint to protect media. (#9086) 
Protecting media stops it from being quarantined when
e.g. all media in a room is quarantined. This is useful
for sticker packs and other media that is uploaded by
server administrators, but used by many people.
 2021-01-15 11:18:09 -05:00
Richard van der Hoff 5310808d3b Give the user a better error when they present bad SSO creds 
If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO
IdP, try to give them a better error.

Previously, the UIA would claim to be successful, but then the operation in
question would simply fail with "auth fail". Instead, serve up an error page
which explains the failure.
 2021-01-13 20:22:41 +00:00
Patrick Cloke d1eb1b96e8
Register the /devices endpoint on workers. (#9092) 2021-01-13 12:35:40 -05:00
Dirk Klimpel 7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932) 
This only applies if the user's data is to be erased.
 2021-01-12 16:30:15 -05:00
Jason Robinson da16d06301 Address pr feedback 
* docs updates
* prettify SQL
* add missing copyright
* cursor_to_dict
* update touched files copyright years

Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2021-01-11 23:43:58 +02:00
Jason Robinson 0b77329fe2
Clarify rooms.md 
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2021-01-11 23:05:36 +02:00
David Teller b161528fcc
Also support remote users on the joined_rooms admin API. (#8948) 
For remote users, only the rooms which the server knows about are returned.
Local users have all of their joined rooms returned.
 2021-01-11 14:32:17 -05:00
Erik Johnston 1315a2e8be
Use a chain cover index to efficiently calculate auth chain difference (#8868) 2021-01-11 16:09:22 +00:00
Jason Robinson e2c16edc78 Add changelog and admin API docs 
Signed-off-by: Jason Robinson <jasonr@matrix.org>
 2021-01-09 22:58:29 +02:00
Christopher Rücker bce0c91d9a
Keycloak mapping_provider example (#9037) (#9057) 
This PR adds the missing user_mapping_provider section in oidc.md

Signed-off-by: Christopher Rücker chris-ruecker@protonmail.com
 2021-01-08 18:29:30 +00:00
Emelie 9066c2fd7f
Fix typo in docs/systemd-with-workers/README.md (#9035) 
Signed-off-by: Emelie em@nao.sh
 2021-01-07 15:31:01 +00:00
Richard van der Hoff 111b673fc1
Add initial support for a "pick your IdP" page (#9017) 
During login, if there are multiple IdPs enabled, offer the user a choice of
IdPs.
 2021-01-05 11:25:28 +00:00
Jerin J Titus cfcf5541b4
Update the value of group_creation_prefix in sample config. (#8992) 
Removes the trailing slash with causes issues with matrix.to/Element.
 2020-12-29 09:30:48 -05:00
Patrick Cloke 68bb26da69
Allow redacting events on workers (#8994) 
Adds the redacts endpoint to workers that have the client listener.
 2020-12-29 07:40:12 -05:00
Patrick Cloke 4218473f9e
Refactor the CAS handler in prep for using the abstracted SSO code. (#8958) 
This makes the CAS handler look more like the SAML/OIDC handlers:

* Render errors to users instead of throwing JSON errors.
* Internal reorganization.
 2020-12-18 13:09:45 -05:00
Patrick Cloke 56e00ca85e
Send the location of the web client to the IS when inviting via 3PIDs. (#8930) 
Adds a new setting `email.invite_client_location` which, if defined, is
passed to the identity server during invites.
 2020-12-18 11:01:57 -05:00
Erik Johnston d781a81e69
Allow server admin to get admin bit in rooms where local user is an admin (#8756) 
This adds an admin API that allows a server admin to get power in a room if a local user has power in a room. Will also invite the user if they're not in the room and its a private room. Can specify another user (rather than the admin user) to be granted power.

Co-authored-by: Matthew Hodgson <matthew@matrix.org>
 2020-12-18 15:37:19 +00:00
Richard van der Hoff 28877fade9
Implement a username picker for synapse (#8942) 
The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow
`UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser
back to the client, we redirect to a username-picker resource, which allows the user to enter a username.
We *then* complete the SSO flow (including doing the client permission checks).

The static resources for the username picker itself (in 
https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker)
are essentially lifted wholesale from
https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. 
As the comment says, we might want to think about making them customisable, but that can be a follow-up. 

Fixes #8876.
 2020-12-18 14:19:46 +00:00
Patrick Cloke 5d4c330ed9
Allow re-using a UI auth validation for a period of time (#8970) 2020-12-18 07:33:57 -05:00
Dirk Klimpel 06006058d7
Make search statement in List Room and User Admin API case-insensitive (#8931) 2020-12-17 10:43:37 +00:00
Patrick Cloke 44b7d4c6d6
Fix the sample config location for the ip_range_whitelist setting. (#8954) 
Move it from the federation section to the server section to match
ip_range_blacklist.
 2020-12-16 14:40:47 -05:00
David Teller f14428b25c
Allow spam-checker modules to be provide async methods. (#8890) 
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
 2020-12-11 14:05:15 -05:00
Dirk Klimpel 0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Dirk Klimpel a5f7aff5e5
Deprecate Shutdown Room and Purge Room Admin API (#8829) 
Deprecate both APIs in favour of the Delete Room API.

Related: #8663 and #8810
 2020-12-10 11:42:48 +00:00
Patrick Cloke 344ab0b53a
Default to blacklisting reserved IP ranges and add a whitelist. (#8870) 
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
 2020-12-09 13:56:06 -05:00
Dirk Klimpel 43bf3c5178
Combine related media admin API docs (#8839) 
Related: #8810
Also a few small improvements.

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-12-09 16:19:57 +00:00
Richard van der Hoff 025fa06fc7
Clarify config template comments (#8891) 2020-12-08 14:03:08 +00:00
Patrick Cloke 96358cb424
Add authentication to replication endpoints. (#8853) 
Authentication is done by checking a shared secret provided
in the Synapse configuration file.
 2020-12-04 10:56:28 -05:00
Patrick Cloke 112f6bd49e Synapse 1.24.0rc2 (2020-12-04) 
==============================
 
 Bugfixes
 --------
 
 - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))
 
 Internal Changes
 ----------------
 
 - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/KQ6MACgkQM/xY9qcR
 MEjqOg/9Fd4Ow0QdnK3O6J0bmXP6+g2GfjJCAKlYlF/nwX1ez9jZVed00D2h9/hD
 YCUJCEv+XZpbv5F4usiYwoLGithbueo7AKEOQJQMEuQBHU1/E6TJ9iiKR3lTBNNw
 ccuyMMqqEatYxoa4DIO/lrwm2mhwRNdMPt0r1DPlML13bb/TVbeXXykbOCyZSKm1
 YuXpxYDngRPL9o46I77l7/KNTSpYYeZi2qwp2orl6GBfa0KSvR2Oi6uohKYzuoHC
 +XLvHSFMpHAZkaZsIMxU5hRVH6jtkb/D37CABuKQsdlvHcooGK4Sdt6aoUyhPtoh
 6YwajSazWKEJ/9YO5JV22qkWFPi9T6s/lPZfjOSu/euwMw7DDQJ/8t8Sm6N7sJDS
 A/5FctOAd1jd9BiAaqkHB2zLHZSomIHQpJJfDyE3UgAGQQ6fm2Wg7SkLByDx/MYC
 4D74dBuVBPENM2VxNoyjHbpVSeQi8t0RgWb44/PUAOtpwm3f3eVRuS8zY6uR2bZr
 YZUbJylqKhRW6dBdZH+EzVolRGvBgLkc27IzRCQ3kGUEo2Wem05s6MuKMvLxoIvN
 NHLWNClUWITT8FgkfFP3c6CK40dwIGJzYryI+71qPW/R7snf2fazuoHlH1pAmFh0
 E0/nBMTw1deixOYXbG1ah63AhJ1NjdTfimDsKmWZLqVc6g7g4d4=
 =is1Y
 -----END PGP SIGNATURE-----

Merge tag 'v1.24.0rc2' into develop

Synapse 1.24.0rc2 (2020-12-04)
==============================

Bugfixes
--------

- Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))

Internal Changes
----------------

- Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
 2020-12-04 09:14:31 -05:00
Patrick Cloke 22c6c19f91
Fix a regression that mapping providers should be able to redirect users. (#8878) 
This was broken in #8801.
 2020-12-04 08:25:15 -05:00
Richard van der Hoff 6e4f71c057
Fix a buglet in the SAML username mapping provider doc (#8873) 
the constructor is called with a `module_api`.
 2020-12-04 10:14:15 +00:00
Patrick Cloke 30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821) 
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
 2020-12-02 11:09:24 -05:00
Andrew Morgan d1be293f00 Fix typo in password_auth_providers doc 
A word got removed accidentally in 83434df381.
 2020-12-01 10:34:52 +00:00
Andrew Morgan 17fa58bdd1
Add a config option to change whether unread push notification counts are per-message or per-room (#8820) 
This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55).

However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).
 2020-11-30 18:43:54 +00:00
Richard van der Hoff a090b86209
Add `force_purge` option to delete-room admin api. (#8843) 2020-11-30 16:48:12 +00:00
Dirk Klimpel 14f81a6d24
Improve documentation how to configure prometheus for workers (#8822) 2020-11-26 10:42:55 +00:00
Dirk Klimpel 3f0ff53158
Remove deprecated `/_matrix/client/*/admin` endpoints (#8785) 
These are now only available via `/_synapse/admin/v1`.
 2020-11-25 16:26:11 -05:00
Andrew Morgan 2b110dda2a
Fix the formatting of push config section (#8818) 
This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).
 2020-11-25 21:02:53 +00:00
Patrick Cloke 4fd222ad70
Support trying multiple localparts for OpenID Connect. (#8801) 
Abstracts the SAML and OpenID Connect code which attempts to regenerate
the localpart of a matrix ID if it is already in use.
 2020-11-25 10:04:22 -05:00
Dirk Klimpel b08dc7effe
Clarify documentation of the admin list media API (#8795) 
Clarify that the list media API only shows media from unencrypted events.
 2020-11-24 09:04:51 -05:00
Richard van der Hoff e3d7806704
Update turn-howto (#8779) 
Some hopefully-useful notes on setting up a turnserver.
 2020-11-24 12:52:22 +00:00
Patrick Cloke 79bfe966e0
Improve error checking for OIDC/SAML mapping providers (#8774) 
Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.
 2020-11-19 14:25:17 -05:00
Ben Banfield-Zanin 53a6f5ddf0
SAML: Allow specifying the IdP entityid to use. (#8630) 
If the SAML metadata includes multiple IdPs it is necessary to
specify which IdP to redirect users to for authentication.
 2020-11-19 09:57:13 -05:00
Marcus Schopen d356588339
SAML: Document allowing a clock/time difference from IdP (#8731) 
Updates the sample configuration with the pysaml2 configuration for
accepting clock skew/drift between the homeserver and IdP.
 2020-11-18 07:36:28 -05:00
chagai95 e487d9fabc
a comma too much (#8771) 
Signed-off-by: Chagai Friedlander chagai95@gmail.com
 2020-11-17 14:13:56 +00:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Adrian Wannenmacher f1de4bb58b
Clarify the usecase for an msisdn delegate (#8734) 
Signed-off-by: Adrian Wannenmacher <tfld@tfld.dev>
 2020-11-14 23:09:36 +00:00
Dirk Klimpel 023f791143
Migrate documentation `docs/admin_api/event_reports` to markdown (#8742) 
Related to #8714. `event_reports.rst` was introduced in Synapse 1.21.0.
 2020-11-13 13:57:55 +00:00
Marcus Schopen 68fc0dcb5a
SAML: add <mdui:UIInfo> element examples (#8718) 
add some mdui:UIInfo element examples for saml2_config in homeserver.yaml
 2020-11-13 12:07:50 +00:00
Marcus Schopen c059413001
Notes on SSO logins and media_repository worker (#8701) 
If SSO login is used (e.g. SAML) in a multi worker setup, it should be mentioned that currently all SAML logins must run on the same worker, see https://github.com/matrix-org/synapse/issues/7530

Also, if you are using different ports (for example 443 and 8448) in a reverse proxy for client and federation, the path `/_matrix/media` on the client and federation port must point to the listener of the `media_repository` worker, otherwise you'll get a 404 on the federation port for the path `/_matrix/media`, if a remote server is trying to get the media object on federation port, see https://github.com/matrix-org/synapse/issues/8695
 2020-11-06 14:33:07 +00:00
Dirk Klimpel c3119d1536
Add an admin API for users' media statistics (#8700) 
Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-11-05 18:59:12 +00:00
Dirk Klimpel e4676bd877
Add `displayname` to Shared-Secret Registration for admins (#8722) 
Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`
 2020-11-05 13:55:45 +00:00
Dirk Klimpel 4fda58ddd2
Remove the "draft" status of the Room Details Admin API (#8702) 
Fixes #8550
 2020-11-03 12:48:25 +00:00
Erik Johnston 4b09b7438e
Document how to set up multiple event persisters (#8706) 2020-11-03 10:27:11 +00:00
Matthew Hodgson d04c2d19b3 grammar 2020-11-02 21:22:36 +00:00
Matthew Hodgson 11fd90a2b7 typo 2020-11-02 13:33:56 +00:00
Andrew Morgan 26b46796ea Fix typos in systemd-with-workers doc 2020-11-02 12:56:16 +00:00
Andrew Morgan 305545682d Fix typo in workers doc 2020-11-02 12:36:18 +00:00
Patrick Cloke 8f1aefa694
Improve the sample config for SSO (OIDC, SAML, and CAS). (#8635) 2020-10-30 10:01:59 -04:00
Patrick Cloke 00b24aa545
Support generating structured logs in addition to standard logs. (#8607) 
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
 2020-10-29 07:27:37 -04:00
Dirk Klimpel 2239813278
Add an admin APIs to allow server admins to list users' pushers (#8610) 
Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers
 2020-10-28 15:02:42 +00:00
Michael Kaye f49c2093b5
Cross-link documentation to the prometheus recording rules. (#8667) 2020-10-27 15:29:50 -04:00
Dirk Klimpel 9b7c28283a
Add admin API to list users' local media (#8647) 
Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.
 2020-10-27 14:12:31 +00:00
Dirk Klimpel 66e6801c3e
Split admin API for reported events into a detail and a list view (#8539) 
Split admin API for reported events in detail und list view.
API was introduced with #8217 in synapse v.1.21.0.

It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview.
The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`.
It is similar to room and users API.

It is a kind of regression in `GET /_synapse/admin/v1/event_reports`.  `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec).

Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2020-10-26 18:16:37 +00:00
Peter Krantz 6c9ab61df5
Added basic instructions for Azure AD to OpenId documentation (#8582) 
Signed-off-by: Peter Krantz peter.krantz@gmail.com
 2020-10-26 17:49:55 +00:00
Dirk Klimpel 49d72dea2a
Add an admin api to delete local media. (#8519) 
Related to: #6459, #3479

Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete
a single file from server.
 2020-10-26 17:02:28 +00:00