Commit Graph

18037 Commits

Author SHA1 Message Date
Richard van der Hoff 8388a7fb3a Make `_make_callback_with_userinfo` async
... so that we can test its behaviour when it raises.

Also pull it out to the top level so that I can use it from other test classes.
2020-12-15 23:10:59 +00:00
Richard van der Hoff c1883f042d Remove spurious mocking of complete_sso_login
The tests that need this all do it already.
2020-12-15 23:10:59 +00:00
Richard van der Hoff 2dd2e90e2b Test `get_extra_attributes` fallback
despite the warnings saying "don't implement get_extra_attributes", we had
implemented it, so the tests weren't doing what we thought they were.
2020-12-15 23:10:59 +00:00
Richard van der Hoff 01333681bc
Preparatory refactoring of the SamlHandlerTestCase (#8938)
* move simple_async_mock to test_utils

... so that it can be re-used

* Remove references to `SamlHandler._map_saml_response_to_user` from tests

This method is going away, so we can no longer use it as a test point. Instead,
factor out a higher-level method which takes a SAML object, and verify correct
behaviour by mocking out `AuthHandler.complete_sso_login`.

* changelog
2020-12-15 20:56:10 +00:00
Patrick Cloke b3a4b53587
Fix handling of stream tokens for push. (#8943)
Removes faulty assertions and fixes the logic to ensure the max
stream token is always set.
2020-12-15 10:41:34 -05:00
Richard van der Hoff 6d02eb22df
Fix startup failure with localdb_enabled: False (#8937) 2020-12-14 20:42:03 +00:00
Patrick Cloke 1619802228
Various clean-ups to the logging context code (#8935) 2020-12-14 14:19:47 -05:00
Richard van der Hoff 895e04319b
Preparatory refactoring of the OidcHandlerTestCase (#8911)
* Remove references to handler._auth_handler

(and replace them with hs.get_auth_handler)

* Factor out a utility function for building Requests

* Remove mocks of `OidcHandler._map_userinfo_to_user`

This method is going away, so mocking it out is no longer a valid approach.

Instead, we mock out lower-level methods (eg _remote_id_from_userinfo), or
simply allow the regular implementation to proceed and update the expectations
accordingly.

* Remove references to `OidcHandler._map_userinfo_to_user` from tests

This method is going away, so we can no longer use it as a test point. Instead
we build mock "callback" requests which we pass into `handle_oidc_callback`,
and verify correct behaviour by mocking out `AuthHandler.complete_sso_login`.
2020-12-14 11:38:50 +00:00
David Teller f14428b25c
Allow spam-checker modules to be provide async methods. (#8890)
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
2020-12-11 14:05:15 -05:00
Patrick Cloke 5d34f40d49
Add type hints to the push module. (#8901) 2020-12-11 11:43:53 -05:00
Erik Johnston a8eceb01e5
Honour AS ratelimit settings for /login requests (#8920)
Fixes #8846.
2020-12-11 16:33:31 +00:00
Patrick Cloke 3af0672350
Improve tests for structured logging. (#8916) 2020-12-11 07:25:01 -05:00
Dirk Klimpel 0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Erik Johnston 1d55c7b567
Don't ratelimit autojoining of rooms (#8921)
Fixes #8866
2020-12-11 10:17:49 +00:00
Richard van der Hoff dc016c66ae
Don't publish `latest` docker image until all archs are built (#8909) 2020-12-10 17:00:29 +00:00
Erik Johnston 80a992d7b9
Fix deadlock on SIGHUP (#8918)
Fixes #8892
2020-12-10 16:56:05 +00:00
Richard van der Hoff c64002e1c1
Refactor `SsoHandler.get_mxid_from_sso` (#8900)
* Factor out _call_attribute_mapper and _register_mapped_user

This is mostly an attempt to simplify `get_mxid_from_sso`.

* Move mapping_lock down into SsoHandler.
2020-12-10 12:43:58 +00:00
Richard van der Hoff 1821f7cc26
Fix buglet in DirectRenderJsonResource (#8897)
this was using `canonical_json` without setting it, so when you used it as a
standalone class, you would get exceptions.
2020-12-10 12:42:55 +00:00
Dirk Klimpel a5f7aff5e5
Deprecate Shutdown Room and Purge Room Admin API (#8829)
Deprecate both APIs in favour of the Delete Room API.

Related: #8663 and #8810
2020-12-10 11:42:48 +00:00
Patrick Cloke 344ab0b53a
Default to blacklisting reserved IP ranges and add a whitelist. (#8870)
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
2020-12-09 13:56:06 -05:00
Patrick Cloke 6ff34e00d9
Skip the SAML tests if xmlsec1 isn't available. (#8905) 2020-12-09 12:23:30 -05:00
Dirk Klimpel 43bf3c5178
Combine related media admin API docs (#8839)
Related: #8810
Also a few small improvements.

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-12-09 16:19:57 +00:00
Richard van der Hoff a4a5c7a35e Merge remote-tracking branch 'origin/master' into develop 2020-12-09 16:13:52 +00:00
Richard van der Hoff 3e8292d483
Merge pull request #8906 from matrix-org/rav/fix_multiarch_builds
Pin the docker version for multiarch builds
2020-12-09 16:03:12 +00:00
Erik Johnston cf7d3c90d6 Merge branch 'release-v1.24.0' into develop 2020-12-09 16:01:12 +00:00
Richard van der Hoff 9bbbb11ac2 Pin the docker version for multiarch builds
It seems that letting CircleCI use its default docker version (17.09.0-ce,
apparently) did not interact well with multiarch builds: in particular, we saw
weird effects where running an amd64 build at the same time as an arm64 build
caused the arm64 builds to fail with:

   Error while loading /usr/sbin/dpkg-deb: No such file or directory
2020-12-09 15:51:11 +00:00
Erik Johnston 57068eae75 Add 'xmlsec1' to dependency list 2020-12-09 13:48:16 +00:00
Erik Johnston fd83debcc0 Merge branch 'master' into develop 2020-12-09 11:30:08 +00:00
Erik Johnston 320e8c8064 Synapse 1.23.1 (2020-12-09)
===========================
 
 Due to the two security issues highlighted below, server administrators are
 encouraged to update Synapse. We are not aware of these vulnerabilities being
 exploited in the wild.
 
 Security advisory
 -----------------
 
 The following issues are fixed in v1.23.1 and v1.24.0.
 
 - There is a denial of service attack
   ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
   against the federation APIs in which future events will not be correctly sent
   to other servers over federation. This affects all servers that participate in
   open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).
 
 - Synapse may be affected by OpenSSL
   [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
   Synapse administrators should ensure that they have the latest versions of
   the cryptography Python package installed.
 
 To upgrade Synapse along with the cryptography package:
 
 * Administrators using the [`matrix.org` Docker
   image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
   packages from
   `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
   should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
   the updated packages.
 * Administrators who have [installed Synapse from
   source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
   should upgrade the cryptography package within their virtualenv by running:
   ```sh
   <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
   ```
 * Administrators who have installed Synapse from distribution packages should
   consult the information from their distributions.
 
 Bugfixes
 --------
 
 - Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))
 
 Internal Changes
 ----------------
 
 - Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAl/QsOYQHGVyaWtAbWF0
 cml4Lm9yZwAKCRClQuTtGw+sCZTkCACEDbyMY/UCqJaUILxtYeBE7K4GvOqPPHyo
 2VLjyitI7XWVzB/paUOPxAtOtiwXS0GOrL+UsW6Lky2HIjafjLe1Z3LHzATQwF2I
 J2bZWTY1Y4v3y8B7noPmp7+QFIBIey++09BY+MwzT3EQYnXt6lvoHmEvPH/htzjg
 LfdZpSj4WrJr4S2/W0rVlkGSuIShN0Tnv6pTgbGRZMt1N4JH2mo65mCGt3xrMS7E
 us+xqStGh5Q+9g3F913iIJ8noUMeCvTT7hbr1eonhZ3MIKWG30z+zcXwmGb0t3B8
 zvTFXqdbZPSw+ZZdxaZwZuJzNCnYOu6t0JuzXqDoE0xsHb8RVUe9
 =Z9US
 -----END PGP SIGNATURE-----

Merge tag 'v1.23.1'

Synapse 1.23.1 (2020-12-09)
===========================

Due to the two security issues highlighted below, server administrators are
encouraged to update Synapse. We are not aware of these vulnerabilities being
exploited in the wild.

Security advisory
-----------------

The following issues are fixed in v1.23.1 and v1.24.0.

- There is a denial of service attack
  ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
  against the federation APIs in which future events will not be correctly sent
  to other servers over federation. This affects all servers that participate in
  open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).

- Synapse may be affected by OpenSSL
  [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
  Synapse administrators should ensure that they have the latest versions of
  the cryptography Python package installed.

To upgrade Synapse along with the cryptography package:

* Administrators using the [`matrix.org` Docker
  image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
  packages from
  `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
  should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
  the updated packages.
* Administrators who have [installed Synapse from
  source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
  should upgrade the cryptography package within their virtualenv by running:
  ```sh
  <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
  ```
* Administrators who have installed Synapse from distribution packages should
  consult the information from their distributions.

Bugfixes
--------

- Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))

Internal Changes
----------------

- Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
2020-12-09 11:29:56 +00:00
Erik Johnston adfc9cb53d Merge branch 'master' into develop 2020-12-09 11:26:48 +00:00
Erik Johnston 1cec3d1457 1.23.1 2020-12-09 11:07:41 +00:00
Erik Johnston 9b26a4ac87 1.24.0 2020-12-09 11:07:24 +00:00
Patrick Cloke 0eb9b2f866 Fix installing pysaml2 on Python 3.5. (#8898)
This pins pysaml2 to < 6.4.0 on Python 3.5, as the last known working version.
2020-12-09 10:38:46 +00:00
Richard van der Hoff 3ce2f303f1 Consistently use room_id from federation request body (#8776)
* Consistently use room_id from federation request body

Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.

* Kill off some references to "context"

Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-12-09 10:38:39 +00:00
Aaron Raimist cd9e72b185
Add X-Robots-Tag header to stop crawlers from indexing media (#8887)
Fixes / related to: https://github.com/matrix-org/synapse/issues/6533

This should do essentially the same thing as a robots.txt file telling robots to not index the media repo. https://developers.google.com/search/reference/robots_meta_tag

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-12-08 22:51:03 +00:00
Patrick Cloke 1a9553045c
Fix installing pysaml2 on Python 3.5. (#8898)
This pins pysaml2 to < 6.4.0 on Python 3.5, as the last known working version.
2020-12-08 13:41:25 -05:00
Richard van der Hoff ab7a24cc6b
Better formatting for config errors from modules (#8874)
The idea is that the parse_config method of extension modules can raise either a ConfigError or a JsonValidationError,
and it will be magically turned into a legible error message. There's a few components to it:

* Separating the "path" and the "message" parts of a ConfigError, so that we can fiddle with the path bit to turn it
   into an absolute path.
* Generally improving the way ConfigErrors get printed.
* Passing in the config path to load_module so that it can wrap any exceptions that get caught appropriately.
2020-12-08 14:04:35 +00:00
Richard van der Hoff 36ba73f53d
Simplify the flow for SSO UIA (#8881)
* SsoHandler: remove inheritance from BaseHandler

* Simplify the flow for SSO UIA

We don't need to do all the magic for mapping users when we are doing UIA, so
let's factor that out.
2020-12-08 14:03:38 +00:00
Richard van der Hoff 025fa06fc7
Clarify config template comments (#8891) 2020-12-08 14:03:08 +00:00
Will Hunt ff1f0ee094
Call set_avatar_url with target_user, not user_id (#8872)
* Call set_avatar_url with target_user, not user_id

Fixes https://github.com/matrix-org/synapse/issues/8871

* Create 8872.bugfix

* Update synapse/rest/admin/users.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Testing

* Update changelog.d/8872.bugfix

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-12-07 19:13:07 +00:00
Patrick Cloke 1f3748f033
Do not raise a 500 exception when previewing empty media. (#8883) 2020-12-07 10:00:08 -05:00
Patrick Cloke 92d87c6882
Add type hints for HTTP and email pushers. (#8880) 2020-12-07 09:59:38 -05:00
Patrick Cloke 02e588856a
Add type hints to the push mailer module. (#8882) 2020-12-07 07:10:22 -05:00
Patrick Cloke 96358cb424
Add authentication to replication endpoints. (#8853)
Authentication is done by checking a shared secret provided
in the Synapse configuration file.
2020-12-04 10:56:28 -05:00
Erik Johnston df4b1e9c74
Pass room_id to get_auth_chain_difference (#8879)
This is so that we can choose which algorithm to use based on the room ID.
2020-12-04 15:52:49 +00:00
Patrick Cloke b774c555d8
Add additional validation to pusher URLs. (#8865)
Pusher URLs now must end in `/_matrix/push/v1/notify` per the
specification.
2020-12-04 10:51:56 -05:00
Patrick Cloke df3e6a23a7
Do not 500 if the content-length is not provided when uploading media. (#8862)
Instead return the proper 400 error.
2020-12-04 10:26:09 -05:00
Patrick Cloke 112f6bd49e Synapse 1.24.0rc2 (2020-12-04)
==============================
 
 Bugfixes
 --------
 
 - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))
 
 Internal Changes
 ----------------
 
 - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/KQ6MACgkQM/xY9qcR
 MEjqOg/9Fd4Ow0QdnK3O6J0bmXP6+g2GfjJCAKlYlF/nwX1ez9jZVed00D2h9/hD
 YCUJCEv+XZpbv5F4usiYwoLGithbueo7AKEOQJQMEuQBHU1/E6TJ9iiKR3lTBNNw
 ccuyMMqqEatYxoa4DIO/lrwm2mhwRNdMPt0r1DPlML13bb/TVbeXXykbOCyZSKm1
 YuXpxYDngRPL9o46I77l7/KNTSpYYeZi2qwp2orl6GBfa0KSvR2Oi6uohKYzuoHC
 +XLvHSFMpHAZkaZsIMxU5hRVH6jtkb/D37CABuKQsdlvHcooGK4Sdt6aoUyhPtoh
 6YwajSazWKEJ/9YO5JV22qkWFPi9T6s/lPZfjOSu/euwMw7DDQJ/8t8Sm6N7sJDS
 A/5FctOAd1jd9BiAaqkHB2zLHZSomIHQpJJfDyE3UgAGQQ6fm2Wg7SkLByDx/MYC
 4D74dBuVBPENM2VxNoyjHbpVSeQi8t0RgWb44/PUAOtpwm3f3eVRuS8zY6uR2bZr
 YZUbJylqKhRW6dBdZH+EzVolRGvBgLkc27IzRCQ3kGUEo2Wem05s6MuKMvLxoIvN
 NHLWNClUWITT8FgkfFP3c6CK40dwIGJzYryI+71qPW/R7snf2fazuoHlH1pAmFh0
 E0/nBMTw1deixOYXbG1ah63AhJ1NjdTfimDsKmWZLqVc6g7g4d4=
 =is1Y
 -----END PGP SIGNATURE-----

Merge tag 'v1.24.0rc2' into develop

Synapse 1.24.0rc2 (2020-12-04)
==============================

Bugfixes
--------

- Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))

Internal Changes
----------------

- Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
2020-12-04 09:14:31 -05:00
Patrick Cloke 2602514f34 Minor update to CHANGES. 2020-12-04 09:00:32 -05:00
Patrick Cloke 693dab487c 1.24.0rc2 2020-12-04 08:48:04 -05:00