Commit Graph

18156 Commits

Author SHA1 Message Date
Erik Johnston 631dd06f2c
Fix get destinations to catch up query. (#9114)
t was doing a sequential scan on `destination_rooms`, which took
minutes.
2021-01-14 16:47:21 +00:00
Erik Johnston 7036e24e98
Add background update for add chain cover index (#9029) 2021-01-14 15:18:27 +00:00
Richard van der Hoff 21a296cd5a
Split OidcProvider out of OidcHandler (#9107)
The idea here is that we will have an instance of OidcProvider for each
configured IdP, with OidcHandler just doing the marshalling of them.

For now it's still hardcoded with a single provider.
2021-01-14 13:29:17 +00:00
Tim Leung 12702be951
Fix wrong arguments being passed to BlacklistingAgentWrapper (#9098)
A reactor was being passed instead of a whitelist for the BlacklistingAgentWrapper
used by the WellyKnownResolver. This coulld cause exceptions when attempting to
connect to IP addresses that are blacklisted, but in reality this did not have any
observable affect since this code is not used for IP literals.
2021-01-14 06:59:26 -05:00
Richard van der Hoff 233c8b9fce
Add a test for UI-Auth-via-SSO (#9082)
* Add complete test for UI-Auth-via-SSO.

* review comments
2021-01-13 20:21:55 +00:00
Richard van der Hoff d02e4b2825
Merge pull request #9105 from matrix-org/rav/multi_idp/oidc_provider_config
Enhancements to OIDC configuration handling
2021-01-13 19:51:46 +00:00
Patrick Cloke aee8e6a95d
Reduce scope of exception handler. (#9106)
Removes a bare `except Exception` clause and replaces it with
catching a specific exception around the portion that might throw.
2021-01-13 13:27:49 -05:00
Richard van der Hoff ef410232f3 changelog 2021-01-13 17:47:27 +00:00
Richard van der Hoff dc3c83a933 Add jsonschema verification for the oidc provider config 2021-01-13 17:47:27 +00:00
Patrick Cloke d1eb1b96e8
Register the /devices endpoint on workers. (#9092) 2021-01-13 12:35:40 -05:00
Richard van der Hoff 7cc9509eca Extract OIDCProviderConfig object
Collect all the config options which related to an OIDC provider into a single
object.
2021-01-13 16:40:02 +00:00
Patrick Cloke 98a64b7f7f
Add basic domain validation for `DomainSpecificString.is_valid`. (#9071)
This checks that the domain given to `DomainSpecificString.is_valid` (e.g.
`UserID`, `RoomAlias`, etc.) is of a valid form. Previously some validation
was done on the localpart (e.g. the sigil), but not the domain portion.
2021-01-13 07:05:16 -05:00
Erik Johnston aa4d8c1f9a Merge branch 'master' into develop 2021-01-13 10:36:55 +00:00
Erik Johnston ebd534b58d Move removal warning up changelog 2021-01-13 10:31:27 +00:00
Erik Johnston 891c925b88 Link to GH profile and fix tense 2021-01-13 10:28:03 +00:00
Erik Johnston f7478d5cc6 Fix link in changelog 2021-01-13 10:26:25 +00:00
Richard van der Hoff bc4bf7b384
Preparatory refactors of OidcHandler (#9067)
Some light refactoring of OidcHandler, in preparation for bigger things:

  * remove inheritance from deprecated BaseHandler
  * add an object to hold the things that go into a session cookie
  * factor out a separate class for manipulating said cookies
2021-01-13 10:26:12 +00:00
Erik Johnston 429c339de8 Fixup changelog 2021-01-13 10:23:16 +00:00
Erik Johnston 3dd6ba135e 1.25.0 2021-01-13 10:19:12 +00:00
Dirk Klimpel 7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932)
This only applies if the user's data is to be erased.
2021-01-12 16:30:15 -05:00
Dan Callahan 6d91e6ca5f
Announce Python / PostgreSQL deprecation policies (#9085)
Fixes #8782
2021-01-12 20:11:15 +00:00
Richard van der Hoff 789d9ebad3
UI Auth via SSO: redirect the user to an appropriate SSO. (#9081)
If we have integrations with multiple identity providers, when the user does a UI Auth, we need to redirect them to the right one.

There are a few steps to this. First of all we actually need to store the userid of the user we are trying to validate in the UIA session, since the /auth/sso/fallback/web request is unauthenticated.

Then, once we get the /auth/sso/fallback/web request, we can fish the user id out of the session, and use it to look up the external id mappings, and hence pick an SSO provider for them.
2021-01-12 17:38:03 +00:00
Marcus e385c8b473
Don't apply the IP range blacklist to proxy connections (#9084)
It is expected that the proxy would be on a private IP address so the
configured proxy should be connected to regardless of the IP range
blacklist.
2021-01-12 12:20:30 -05:00
Patrick Cloke 723b19748a
Handle bad JSON data being returned from the federation API. (#9070) 2021-01-12 11:07:01 -05:00
Dan Callahan fa6deb298b
Fix failures in Debian packaging (#9079)
Debian package builds were failing for two reasons:

 1. Python versions prior to 3.7 throw exceptions when attempting to print
    Unicode characters under a "C" locale. (#9076)

 2. We depended on `dh-systemd` which no longer exists in Debian Bullseye, but
    is necessary in Ubuntu Xenial. (#9073)

Setting `LANG="C.UTF-8"` in the build environment fixes the first issue.
See also: https://bugs.python.org/issue19846

The second issue is a bit trickier. The dh-systemd package was merged into
debhelper version 9.20160709 and a transitional package left in its wake.

The transitional dh-systemd package was removed in Debian Bullseye.

However, Ubuntu Xenial ships an older debhelper, and still needs dh-systemd.

Thus, builds were failing on Bullseye since we depended on a package which had
ceased existing, but we couldn't remove it from the debian/control file and our
build scripts because we still needed it for Ubuntu Xenial.

We can fix the debian/control issue by listing dh-systemd as an alternative to
the newer versions of debhelper. Since dh-systemd declares that it depends on
debhelper, Ubuntu Xenial will select its older dh-systemd which will in turn
pull in its older debhelper, resulting in no change from the status quo. All
other supported releases will satisfy the debhelper dependency constraint and
skip the dh-systemd alternative.

Build scripts were fixed by unconditionally attempting to install dh-systemd on
all releases and suppressing failures.

Once we drop support for Ubuntu Xenial, we can revert most of this commit and
rely on the version constraint on debhelper in debian/control.

Fixes #9076
Fixes #9073

Signed-off-by: Dan Callahan <danc@element.io>
2021-01-12 14:15:04 +00:00
Richard van der Hoff 0f8945e166
Kill off `HomeServer.get_ip_from_request()` (#9080)
Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.
2021-01-12 12:48:12 +00:00
Richard van der Hoff 2ec8ca5e60
Remove SynapseRequest.get_user_agent (#9069)
SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests",
which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the 
LoggingContext and write the right entries to the request log).

Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a
SynapseRequest when there is nothing synapse-specific about the Request at all, and any old
twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult.

In short: move get_user_agent out to a utility function.
2021-01-12 12:34:16 +00:00
David Teller b161528fcc
Also support remote users on the joined_rooms admin API. (#8948)
For remote users, only the rooms which the server knows about are returned.
Local users have all of their joined rooms returned.
2021-01-11 14:32:17 -05:00
Erik Johnston c9195744a4
Move more encryption endpoints off master (#9068) 2021-01-11 18:01:27 +00:00
Dirk Klimpel 42d3a28d8b
Removes unnecessary declarations in the tests for the admin API. (#9063) 2021-01-11 11:15:54 -05:00
Erik Johnston 1315a2e8be
Use a chain cover index to efficiently calculate auth chain difference (#8868) 2021-01-11 16:09:22 +00:00
Richard van der Hoff 671138f658
Clean up exception handling in the startup code (#9059)
Factor out the exception handling in the startup code to a utility function,
and fix the some logging and exit code stuff.
2021-01-11 15:55:05 +00:00
Erik Johnston 4e04435bda
Remove old tables after schema version bump (#9055)
These tables are unused, and can be dropped now the schema version has been bumped.
2021-01-11 13:58:19 +00:00
Erik Johnston 63f4990298
Ensure rejected events get added to some metadata tables (#9016)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-01-11 13:57:33 +00:00
0xflotus 2fb1c2b6e6
Fix a typo in the install docs. (#9040) 2021-01-11 07:42:18 -05:00
Richard van der Hoff 7db2622d30
Remove unused SynapseService (#9058) 2021-01-11 10:24:22 +00:00
Jerin J Titus c21d8f1c1d
Drop last_used column from access_tokens (#9025)
* Dropped last_used column from access_tokens

Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
2021-01-11 10:23:49 +00:00
Matthew Hodgson ef0388a648 fix spurious MD in README.rst 2021-01-10 23:40:12 +00:00
Christopher Rücker bce0c91d9a
Keycloak mapping_provider example (#9037) (#9057)
This PR adds the missing user_mapping_provider section in oidc.md

Signed-off-by: Christopher Rücker chris-ruecker@protonmail.com
2021-01-08 18:29:30 +00:00
Erik Johnston a03d71dc9d
Fix "Starting metrics collection from sentinel context" errors (#9053) 2021-01-08 14:33:53 +00:00
Richard van der Hoff 12f79da587
Merge pull request #9036 from matrix-org/rav/multi_idp/tests
Add tests for the IdP picker
2021-01-08 14:24:41 +00:00
Richard van der Hoff d32870ffa5
Fix validate_config on nested objects (#9054) 2021-01-08 14:23:04 +00:00
Erik Johnston fa5f5cbc74
Fix error handling during insertion of client IPs (#9051)
You can't continue using a transaction once an exception has been
raised, so catching and dropping the error here is pointless and just
causes more errors.
2021-01-08 14:15:20 +00:00
Richard van der Hoff 195adf4025
Remove broken and unmaintained 'webserver.py' script (#9039)
I'm not even sure what this was supposed to do, but the fact it has python2isms
and nobody has noticed suggests it's not terribly important.

It doesn't seem to have been used since ff23e5ba37.
2021-01-08 14:09:06 +00:00
Richard van der Hoff 23a59d24ae
Run the linters on a consistent list of files (#9038)
We were running some linters on some files and some on others. Extract a common
setting and use it everywhere.
2021-01-08 14:08:44 +00:00
Erik Johnston b530eaa262
Allow running sendToDevice on workers (#9044) 2021-01-07 20:19:26 +00:00
Erik Johnston 5e99a94502
Support routing edu's to multiple instances (#9042)
This is in preparation for moving `SendToDeviceServlet` off master
2021-01-07 18:07:28 +00:00
Erik Johnston e34df813ce
Ensure that remote users' device list resyncing always happens on master (#9043)
Currently `DeviceMessageHandler` only ever exists on master, but that is about to change.
2021-01-07 18:06:52 +00:00
Erik Johnston 63593134a1
Some cleanups to device inbox store. (#9041) 2021-01-07 17:20:44 +00:00
Emelie 9066c2fd7f
Fix typo in docs/systemd-with-workers/README.md (#9035)
Signed-off-by: Emelie em@nao.sh
2021-01-07 15:31:01 +00:00