Commit Graph

18173 Commits

Author SHA1 Message Date
Richard van der Hoff 90cf1eec44 Remove redundant mocking 2020-12-02 17:53:38 +00:00
Richard van der Hoff 7ea85302f3 fix up various test cases
A few test cases were relying on being able to mount non-client servlets on the
test resource. it's better to give them their own Resources.
2020-12-02 16:30:01 +00:00
Patrick Cloke 30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Erik Johnston c5b6abd53d
Correctly handle unpersisted events when calculating auth chain difference. (#8827)
We do state res with unpersisted events when calculating the new current state of the room, so that should be the only thing impacted. I don't think this is tooooo big of a deal as:

1. the next time a state event happens in the room the current state should correct itself;
2. in the common case all the unpersisted events' auth events will be pulled in by other state, so will still return the correct result (or one which is sufficiently close to not affect the result); and
3. we mostly use the state at an event to do important operations, which isn't affected by this.
2020-12-02 15:22:37 +00:00
Richard van der Hoff 693516e756 Add `create_resource_dict` method to HomeserverTestCase
Rather than using a single JsonResource, construct a resource tree, as we do in
the prod code, and allow testcases to add extra resources by overriding
`create_resource_dict`.
2020-12-02 15:21:00 +00:00
Johanna Dorothea Reichmann 0fed46ebe5
Add missing prometheus rules for persisted events (#8802)
The official dashboard uses data from these rules, but they were never added to the synapse-v2.rules. They are mentioned in this issue: https://github.com/matrix-org/synapse/issues/7917#issuecomment-661330409, but never got added to the rules.

Adding them results in all graphs in the "Event persist rate" section to function as intended.

Signed-off-by: Johanna Dorothea Reichmann <transcaffeine@finallycoffee.eu>
2020-12-02 15:18:41 +00:00
David Florness c4675e1b24
Add additional validation for the admin register endpoint. (#8837)
Raise a proper 400 error if the `mac` field is missing.
2020-12-02 10:01:15 -05:00
Patrick Cloke e41720d85f Minor changes to the CHANGES doc. 2020-12-02 09:17:42 -05:00
Patrick Cloke c67af840aa Minor fixes to changelog. 2020-12-02 09:03:12 -05:00
Patrick Cloke 53b12688dd 1.24.0rc1 2020-12-02 08:57:51 -05:00
Patrick Cloke 8388384a64
Fix a regression when grandfathering SAML users. (#8855)
This was broken in #8801 when abstracting code shared with OIDC.

After this change both SAML and OIDC have a concept of
grandfathering users, but with different implementations.
2020-12-02 07:45:42 -05:00
Patrick Cloke c21bdc813f
Add basic SAML tests for mapping users. (#8800) 2020-12-02 07:09:21 -05:00
Richard van der Hoff d3ed93504b
Create a `PasswordProvider` wrapper object (#8849)
The idea here is to abstract out all the conditional code which tests which
methods a given password provider has, to provide a consistent interface.
2020-12-02 10:38:50 +00:00
Andrew Morgan edb3d3f827
Allow specifying room version in 'RestHelper.create_room_as' and add typing (#8854)
This PR adds a `room_version` argument to the `RestHelper`'s `create_room_as` function for tests. I plan to use this for testing knocking, which currently uses an unstable room version.
2020-12-02 10:38:18 +00:00
Richard van der Hoff 4d9496559d
Support "identifier" dicts in UIA (#8848)
The spec requires synapse to support `identifier` dicts for `m.login.password`
user-interactive auth, which it did not (instead, it required an undocumented
`user` parameter.)

To fix this properly, we need to pull the code that interprets `identifier`
into `AuthHandler.validate_login` so that it can be called from the UIA code.

Fixes #5665.
2020-12-01 17:42:26 +00:00
Richard van der Hoff 9edff901d1
Add missing `ordering` to background updates (#8850)
It's important that we make sure our background updates happen in a defined
order, to avoid disasters like #6923.

Add an ordering to all of the background updates that have landed since #7190.
2020-12-01 15:52:49 +00:00
Nicolas Chamo 3f0cba657c
Allow Date header through CORS (#8804) 2020-12-01 13:24:56 +00:00
Richard van der Hoff 89f7930730
Don't offer password login when it is disabled (#8835)
Fix a minor bug where we would offer "m.login.password" login if a custom auth provider supported it, even if password login was disabled.
2020-12-01 13:04:03 +00:00
Richard van der Hoff ddc4343683
Add some tests for `password_auth_providers` (#8819)
These things seemed to be completely untested, so I added a load of tests for
them.
2020-12-01 11:10:42 +00:00
Richard van der Hoff 09ac0569fe
Fix broken testcase (#8851)
This test was broken by #8565. It doesn't need to set set `self.clock`
here anyway - that is done by `setUp`.
2020-12-01 11:04:57 +00:00
Andrew Morgan d1be293f00 Fix typo in password_auth_providers doc
A word got removed accidentally in 83434df381.
2020-12-01 10:34:52 +00:00
Richard van der Hoff 59e18a1333
Simplify appservice login code (#8847)
we don't need to support legacy login dictionaries here.
2020-11-30 19:20:56 +00:00
Mathieu Velten 9f0f274fe0
Allow per-room profile to be used for server notice user (#8799)
This applies even if the feature is disabled at the server level with `allow_per_room_profiles`.
The server notice not being a real user it doesn't have an user profile.
2020-11-30 18:59:29 +00:00
Richard van der Hoff f8d13ca13d
Drop (almost) unused index on event_json (#8845) 2020-11-30 18:44:09 +00:00
Andrew Morgan 17fa58bdd1
Add a config option to change whether unread push notification counts are per-message or per-room (#8820)
This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55).

However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).
2020-11-30 18:43:54 +00:00
Jonathan de Jong ca60822b34
Simplify the way the `HomeServer` object caches its internal attributes. (#8565)
Changes `@cache_in_self` to use underscore-prefixed attributes.
2020-11-30 13:28:44 -05:00
Richard van der Hoff a090b86209
Add `force_purge` option to delete-room admin api. (#8843) 2020-11-30 16:48:12 +00:00
Tulir Asokan 856eab606b
Remove special case of pretty printing JSON responses for curl (#8833)
* Remove special case of pretty printing JSON responses for curl

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2020-11-27 12:37:55 +00:00
Andrew Morgan 5cbe8d93fe
Add typing to membership Replication class methods (#8809)
This PR grew out of #6739, and adds typing to some method arguments

You'll notice that there are a lot of `# type: ignores` in here. This is due to the base methods not matching the overloads here. This is necessary to stop mypy complaining, but a better solution is #8828.
2020-11-27 10:49:38 +00:00
Dirk Klimpel 1cd356765e
Update example prometheus console (#8824)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-11-26 17:41:20 +00:00
Erik Johnston 382b4e83f1
Defer SIGHUP handlers to reactor. (#8817)
We can get a SIGHUP at any point, including times where we are not in a
sane state. By deferring calling the handlers until the next reactor
tick we ensure that we don't get unexpected conflicts, e.g. trying to
flush logs from the signal handler while the code was in the process of
writing a log entry.

Fixes #8769.
2020-11-26 11:18:10 +00:00
Dmitry Borodaenko 7c43447477
Strip trailing / from server_url in register_new_matrix_user (#8823)
When server URL provided to register_new_matrix_user includes path
component (e.g. "http://localhost:8008/"), the command fails with
"ERROR! Received 400 Bad Request". Stripping trailing slash from the
server_url command argument makes sure combined endpoint URL remains
valid.

Signed-off-by: Dmitry Borodaenko angdraug@debian.org
2020-11-26 10:57:26 +00:00
Dirk Klimpel 14f81a6d24
Improve documentation how to configure prometheus for workers (#8822) 2020-11-26 10:42:55 +00:00
Dirk Klimpel 3f0ff53158
Remove deprecated `/_matrix/client/*/admin` endpoints (#8785)
These are now only available via `/_synapse/admin/v1`.
2020-11-25 16:26:11 -05:00
Andrew Morgan 2b110dda2a
Fix the formatting of push config section (#8818)
This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).
2020-11-25 21:02:53 +00:00
Andrew Morgan d963c69ba5
Speed up remote invite rejection database call (#8815)
This is another PR that grew out of #6739.

The existing code for checking whether a user is currently invited to a room when they want to leave the room looks like the following:

f737368a26/synapse/handlers/room_member.py (L518-L540)

It calls `get_invite_for_local_user_in_room`, which will actually query *all* rooms the user has been invited to, before iterating over them and matching via the room ID. It will then return a tuple of a lot of information which we pull the event ID out of.

I need to do a similar check for knocking, but this code wasn't very efficient. I then tried to write a different implementation using `StateHandler.get_current_state` but this actually didn't work as we haven't *joined* the room yet - we've only been invited to it. That means that only certain tables in Synapse have our desired `invite` membership state. One of those tables is `local_current_membership`.

So I wrote a store method that just queries that table instead
2020-11-25 20:06:13 +00:00
Patrick Cloke 968939bdac
Add additional type hints to HTTP client. (#8812)
This also removes some duplicated code between the simple
HTTP client and matrix federation client.
2020-11-25 13:30:47 -05:00
Patrick Cloke 4fd222ad70
Support trying multiple localparts for OpenID Connect. (#8801)
Abstracts the SAML and OpenID Connect code which attempts to regenerate
the localpart of a matrix ID if it is already in use.
2020-11-25 10:04:22 -05:00
Patrick Cloke f38676d161
Add type hints to matrix federation client / agent. (#8806) 2020-11-25 07:07:21 -05:00
Dirk Klimpel b08dc7effe
Clarify documentation of the admin list media API (#8795)
Clarify that the list media API only shows media from unencrypted events.
2020-11-24 09:04:51 -05:00
Richard van der Hoff 97b35ee259
Add a script to sign arbitrary json objects. (#8772) 2020-11-24 12:53:00 +00:00
Richard van der Hoff e3d7806704
Update turn-howto (#8779)
Some hopefully-useful notes on setting up a turnserver.
2020-11-24 12:52:22 +00:00
Patrick Cloke 6fde6aa9c0
Properly report user-agent/IP during registration of SSO users. (#8784)
This also expands type-hints to the SSO and registration code.

Refactors the CAS code to more closely match OIDC/SAML.
2020-11-23 13:28:03 -05:00
Waylon Cude 7127855741
Fix synctl and duplicate worker spawning (#8798)
Synctl did not check if a worker thread was already running when using
`synctl start` and would naively start a fresh copy. This would
sometimes lead to cases where many duplicate copies of a single worker
would run.

This fix adds a pid check when starting worker threads and synctl will
now refuse to start individual workers if they're already running.
2020-11-23 15:20:49 +00:00
Andrew Morgan 59a995f38d
Improve logging of the mapping from SSO IDs to Matrix IDs. (#8773) 2020-11-23 08:45:23 -05:00
Daniele Sluijters 8ca120df7c
INSTALL: Fix setting content-type on well-known (#8793)
When using `add_header` nginx will literally add a header. If a
`content-type` header is already configured (for example through a
server wide default), this means we end up with 2 content-type headers,
like so:

```
content-type: text/html
content-type: application/json
access-control-allow-origin: *
```

That doesn't make sense. Instead, we want the content type of that
block to only be `application/json` which we can achieve using
`default_type` instead.

Signed-off-by: Daniele Sluijters <daenney@users.noreply.github.com>
2020-11-23 13:01:18 +00:00
Matthew Hodgson 476b8c0ae6 fix MD 2020-11-22 00:30:13 +00:00
Matthew Hodgson 1091bcea3e fix ancient changelog to be MD 2020-11-22 00:29:05 +00:00
Patrick Cloke 79bfe966e0
Improve error checking for OIDC/SAML mapping providers (#8774)
Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.
2020-11-19 14:25:17 -05:00
Ben Banfield-Zanin 53a6f5ddf0
SAML: Allow specifying the IdP entityid to use. (#8630)
If the SAML metadata includes multiple IdPs it is necessary to
specify which IdP to redirect users to for authentication.
2020-11-19 09:57:13 -05:00