Commit Graph

18381 Commits

Author SHA1 Message Date
Patrick Cloke 3e4cdfe5d9
Add an admin API endpoint to protect media. (#9086)
Protecting media stops it from being quarantined when
e.g. all media in a room is quarantined. This is useful
for sticker packs and other media that is uploaded by
server administrators, but used by many people.
2021-01-15 11:18:09 -05:00
Patrick Cloke 74dd906041
Avoid raising the body exceeded error multiple times. (#9108)
Previously this code generated unreferenced `Deferred` instances
which caused "Unhandled Deferreds" errors to appear in error
situations.
2021-01-15 11:00:13 -05:00
Richard van der Hoff 9ffac2bef1
Remote dependency on distutils (#9125)
`distutils` is pretty much deprecated these days, and replaced with
`setuptools`. It's also annoying because it's you can't `pip install` it, and
it's hard to figure out which debian package we should depend on to make sure
it's there.

Since we only use it for a tiny function anyway, let's just vendor said
function into our codebase.
2021-01-15 15:59:20 +00:00
Patrick Cloke d34c6e1279
Add type hints to media rest resources. (#9093) 2021-01-15 10:57:37 -05:00
Richard van der Hoff 0dd2649c12
Improve UsernamePickerTestCase (#9112)
* make the OIDC bits of the test work at a higher level - via the REST api instead of poking the OIDCHandler directly.
* Move it to test_login.py, where I think it fits better.
2021-01-15 13:45:13 +00:00
Richard van der Hoff 4575ad0b1e
Store an IdP ID in the OIDC session (#9109)
Again in preparation for handling more than one OIDC provider, add a new caveat to the macaroon used as an OIDC session cookie, which remembers which OIDC provider we are talking to. In future, when we get a callback, we'll need it to make sure we talk to the right IdP.

As part of this, I'm adding an idp_id and idp_name field to the OIDC configuration object. They aren't yet documented, and we'll just use the old values by default.
2021-01-15 13:22:12 +00:00
Eric Eastwood 20af310889
Add some extra notes for getting Synapse running on macOS. (#8997) 2021-01-15 06:58:31 -05:00
Richard van der Hoff 14950a45d6
Merge pull request #9091 from matrix-org/rav/error_on_bad_sso
Give the user a better error when they present bad SSO creds
2021-01-15 00:27:13 +00:00
Erik Johnston 1a08e0cdab
Fix event chain bg update. (#9118)
We passed in a graph to `sorted_topologically` which didn't have an
entry for each node (as we dropped nodes with no edges).
2021-01-14 18:57:32 +00:00
Erik Johnston d2479c6870
Fix perf of get_cross_signing_keys (#9116) 2021-01-14 17:57:09 +00:00
Erik Johnston 659c415ed4
Fix chain cover background update to work with split out event persisters (#9115) 2021-01-14 17:19:35 +00:00
Erik Johnston 631dd06f2c
Fix get destinations to catch up query. (#9114)
t was doing a sequential scan on `destination_rooms`, which took
minutes.
2021-01-14 16:47:21 +00:00
Erik Johnston 7036e24e98
Add background update for add chain cover index (#9029) 2021-01-14 15:18:27 +00:00
Richard van der Hoff 21a296cd5a
Split OidcProvider out of OidcHandler (#9107)
The idea here is that we will have an instance of OidcProvider for each
configured IdP, with OidcHandler just doing the marshalling of them.

For now it's still hardcoded with a single provider.
2021-01-14 13:29:17 +00:00
Tim Leung 12702be951
Fix wrong arguments being passed to BlacklistingAgentWrapper (#9098)
A reactor was being passed instead of a whitelist for the BlacklistingAgentWrapper
used by the WellyKnownResolver. This coulld cause exceptions when attempting to
connect to IP addresses that are blacklisted, but in reality this did not have any
observable affect since this code is not used for IP literals.
2021-01-14 06:59:26 -05:00
Richard van der Hoff 26d10331e5 Add a test for wrong user returned by SSO 2021-01-13 20:22:41 +00:00
Richard van der Hoff 420031906a Move `complete_sso_ui_auth` into SSOHandler
since we're hacking on this code anyway, may as well move it out of the
cluttered AuthHandler.
2021-01-13 20:22:41 +00:00
Richard van der Hoff 5310808d3b Give the user a better error when they present bad SSO creds
If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO
IdP, try to give them a better error.

Previously, the UIA would claim to be successful, but then the operation in
question would simply fail with "auth fail". Instead, serve up an error page
which explains the failure.
2021-01-13 20:22:41 +00:00
Richard van der Hoff 233c8b9fce
Add a test for UI-Auth-via-SSO (#9082)
* Add complete test for UI-Auth-via-SSO.

* review comments
2021-01-13 20:21:55 +00:00
Richard van der Hoff d02e4b2825
Merge pull request #9105 from matrix-org/rav/multi_idp/oidc_provider_config
Enhancements to OIDC configuration handling
2021-01-13 19:51:46 +00:00
Patrick Cloke aee8e6a95d
Reduce scope of exception handler. (#9106)
Removes a bare `except Exception` clause and replaces it with
catching a specific exception around the portion that might throw.
2021-01-13 13:27:49 -05:00
Richard van der Hoff ef410232f3 changelog 2021-01-13 17:47:27 +00:00
Richard van der Hoff dc3c83a933 Add jsonschema verification for the oidc provider config 2021-01-13 17:47:27 +00:00
Patrick Cloke d1eb1b96e8
Register the /devices endpoint on workers. (#9092) 2021-01-13 12:35:40 -05:00
Richard van der Hoff 7cc9509eca Extract OIDCProviderConfig object
Collect all the config options which related to an OIDC provider into a single
object.
2021-01-13 16:40:02 +00:00
Patrick Cloke 98a64b7f7f
Add basic domain validation for `DomainSpecificString.is_valid`. (#9071)
This checks that the domain given to `DomainSpecificString.is_valid` (e.g.
`UserID`, `RoomAlias`, etc.) is of a valid form. Previously some validation
was done on the localpart (e.g. the sigil), but not the domain portion.
2021-01-13 07:05:16 -05:00
Erik Johnston aa4d8c1f9a Merge branch 'master' into develop 2021-01-13 10:36:55 +00:00
Erik Johnston ebd534b58d Move removal warning up changelog 2021-01-13 10:31:27 +00:00
Erik Johnston 891c925b88 Link to GH profile and fix tense 2021-01-13 10:28:03 +00:00
Erik Johnston f7478d5cc6 Fix link in changelog 2021-01-13 10:26:25 +00:00
Richard van der Hoff bc4bf7b384
Preparatory refactors of OidcHandler (#9067)
Some light refactoring of OidcHandler, in preparation for bigger things:

  * remove inheritance from deprecated BaseHandler
  * add an object to hold the things that go into a session cookie
  * factor out a separate class for manipulating said cookies
2021-01-13 10:26:12 +00:00
Erik Johnston 429c339de8 Fixup changelog 2021-01-13 10:23:16 +00:00
Erik Johnston 3dd6ba135e 1.25.0 2021-01-13 10:19:12 +00:00
Dirk Klimpel 7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932)
This only applies if the user's data is to be erased.
2021-01-12 16:30:15 -05:00
Dan Callahan 6d91e6ca5f
Announce Python / PostgreSQL deprecation policies (#9085)
Fixes #8782
2021-01-12 20:11:15 +00:00
Richard van der Hoff 789d9ebad3
UI Auth via SSO: redirect the user to an appropriate SSO. (#9081)
If we have integrations with multiple identity providers, when the user does a UI Auth, we need to redirect them to the right one.

There are a few steps to this. First of all we actually need to store the userid of the user we are trying to validate in the UIA session, since the /auth/sso/fallback/web request is unauthenticated.

Then, once we get the /auth/sso/fallback/web request, we can fish the user id out of the session, and use it to look up the external id mappings, and hence pick an SSO provider for them.
2021-01-12 17:38:03 +00:00
Marcus e385c8b473
Don't apply the IP range blacklist to proxy connections (#9084)
It is expected that the proxy would be on a private IP address so the
configured proxy should be connected to regardless of the IP range
blacklist.
2021-01-12 12:20:30 -05:00
Patrick Cloke 723b19748a
Handle bad JSON data being returned from the federation API. (#9070) 2021-01-12 11:07:01 -05:00
Dan Callahan fa6deb298b
Fix failures in Debian packaging (#9079)
Debian package builds were failing for two reasons:

 1. Python versions prior to 3.7 throw exceptions when attempting to print
    Unicode characters under a "C" locale. (#9076)

 2. We depended on `dh-systemd` which no longer exists in Debian Bullseye, but
    is necessary in Ubuntu Xenial. (#9073)

Setting `LANG="C.UTF-8"` in the build environment fixes the first issue.
See also: https://bugs.python.org/issue19846

The second issue is a bit trickier. The dh-systemd package was merged into
debhelper version 9.20160709 and a transitional package left in its wake.

The transitional dh-systemd package was removed in Debian Bullseye.

However, Ubuntu Xenial ships an older debhelper, and still needs dh-systemd.

Thus, builds were failing on Bullseye since we depended on a package which had
ceased existing, but we couldn't remove it from the debian/control file and our
build scripts because we still needed it for Ubuntu Xenial.

We can fix the debian/control issue by listing dh-systemd as an alternative to
the newer versions of debhelper. Since dh-systemd declares that it depends on
debhelper, Ubuntu Xenial will select its older dh-systemd which will in turn
pull in its older debhelper, resulting in no change from the status quo. All
other supported releases will satisfy the debhelper dependency constraint and
skip the dh-systemd alternative.

Build scripts were fixed by unconditionally attempting to install dh-systemd on
all releases and suppressing failures.

Once we drop support for Ubuntu Xenial, we can revert most of this commit and
rely on the version constraint on debhelper in debian/control.

Fixes #9076
Fixes #9073

Signed-off-by: Dan Callahan <danc@element.io>
2021-01-12 14:15:04 +00:00
Richard van der Hoff 0f8945e166
Kill off `HomeServer.get_ip_from_request()` (#9080)
Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.
2021-01-12 12:48:12 +00:00
Richard van der Hoff 2ec8ca5e60
Remove SynapseRequest.get_user_agent (#9069)
SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests",
which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the 
LoggingContext and write the right entries to the request log).

Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a
SynapseRequest when there is nothing synapse-specific about the Request at all, and any old
twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult.

In short: move get_user_agent out to a utility function.
2021-01-12 12:34:16 +00:00
Jason Robinson c177faf5a9 Remove trailing whitespace to appease the linter
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:55:44 +02:00
Jason Robinson 49c619a9a2 Simplify delete_forward_extremities_for_room_txn SQL
As per feedback.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:49:58 +02:00
Jason Robinson da16d06301 Address pr feedback
* docs updates
* prettify SQL
* add missing copyright
* cursor_to_dict
* update touched files copyright years

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:43:58 +02:00
Jason Robinson 0b77329fe2
Clarify rooms.md
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-01-11 23:05:36 +02:00
David Teller b161528fcc
Also support remote users on the joined_rooms admin API. (#8948)
For remote users, only the rooms which the server knows about are returned.
Local users have all of their joined rooms returned.
2021-01-11 14:32:17 -05:00
Erik Johnston c9195744a4
Move more encryption endpoints off master (#9068) 2021-01-11 18:01:27 +00:00
Dirk Klimpel 42d3a28d8b
Removes unnecessary declarations in the tests for the admin API. (#9063) 2021-01-11 11:15:54 -05:00
Erik Johnston 1315a2e8be
Use a chain cover index to efficiently calculate auth chain difference (#8868) 2021-01-11 16:09:22 +00:00
Richard van der Hoff 671138f658
Clean up exception handling in the startup code (#9059)
Factor out the exception handling in the startup code to a utility function,
and fix the some logging and exit code stuff.
2021-01-11 15:55:05 +00:00