Commit Graph

919 Commits

Author SHA1 Message Date
Patrick Cloke d1bf5f7c9d
Strip "join_authorised_via_users_server" from join events which do not need it. (#10933)
This fixes a "Event not signed by authorising server" error when
transition room member from join -> join, e.g. when updating a
display name or avatar URL for restricted rooms.
2021-09-30 11:13:59 -04:00
Jason Robinson fa74536384
Fix AuthBlocking check when requester is appservice (#10881)
If the MAU count had been reached, Synapse incorrectly blocked appservice users even though they've been explicitly configured not to be tracked (the default). This was due to bypassing the relevant if as it was chained behind another earlier hit if as an elif.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-09-24 10:41:18 +01:00
Patrick Cloke 47854c71e9
Use direct references for configuration variables (part 4). (#10893) 2021-09-23 12:03:01 -04:00
Eric Eastwood 51e2db3598
Rename MSC2716 things from `chunk` to `batch` to match `/batch_send` endpoint (#10838)
See https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r684574497

Dropping support for older MSC2716 room versions so we don't have to worry about
supporting both chunk and batch events.
2021-09-21 15:06:28 -05:00
Patrick Cloke 8c7a531e27
Use direct references for some configuration variables (part 2) (#10812) 2021-09-15 08:34:52 -04:00
Patrick Cloke 01c88a09cd
Use direct references for some configuration variables (#10798)
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
reivilibre 524b8ead77
Add types to synapse.util. (#10601) 2021-09-10 17:03:18 +01:00
Patrick Cloke a621ba0259
Prefer room v9 for restricted rooms. (#10772)
Hint to clients via the room capabilities API (MSC3244) that
room version 9 should be preferred for creating a room with
restricted join rules (instead of room version 8).
2021-09-09 07:37:33 -04:00
Patrick Cloke 0288e6033b
Add a constant for m.federate. (#10775) 2021-09-08 10:00:43 -04:00
Brendan Abolivier 287108fb2e Merge tag 'v1.42.0rc2' into develop
Synapse 1.42.0rc2 (2021-09-06)
==============================

This version of Synapse removes deprecated room-management admin APIs, removes out-of-date
email pushers, and improves error handling for fallback templates for user-interactive
authentication. For more information on these points, server administrators are
encouraged to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).

Features
--------

- Support room version 9 from [MSC3375](https://github.com/matrix-org/matrix-doc/pull/3375). ([\#10747](https://github.com/matrix-org/synapse/issues/10747))

Internal Changes
----------------

- Print a warning when using one of the deprecated `template_dir` settings. ([\#10768](https://github.com/matrix-org/synapse/issues/10768))
2021-09-06 16:29:59 +01:00
Richard van der Hoff 56e2a30634
Move `maybe_kick_guest_users` out of `BaseHandler` (#10744)
This is part of my ongoing war against BaseHandler. I've moved kick_guest_users into RoomMemberHandler (since it calls out to that handler anyway), and split maybe_kick_guest_users into the two places it is called.
2021-09-06 12:17:16 +01:00
Patrick Cloke ae3c16318b
Support MSC3375: room version 9. (#10747) 2021-09-03 12:51:15 -04:00
Eric Eastwood dc75fb7f05
Populate `rooms.creator` field for easy lookup (#10697)
Part of https://github.com/matrix-org/synapse/pull/10566

 - Fill in creator whenever we insert into the rooms table
 - Add background update to backfill any missing creator values
2021-09-01 16:27:58 +01:00
Erik Johnston cd22fb568a Merge branch 'master' into develop 2021-08-24 17:13:29 +01:00
Callum Brown 947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142)
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
2021-08-21 22:14:43 +01:00
David Robertson ee3b2ac59a
Validate device_keys for C-S /keys/query requests (#10593)
* Validate device_keys for C-S /keys/query requests

Closes #10354

A small, not particularly critical fix. I'm interested in seeing if we
can find a more systematic approach though. #8445 is the place for any discussion.
2021-08-20 15:47:03 +01:00
Patrick Cloke 5cda75fede
Set room version 8 as preferred for restricted rooms. (#10571) 2021-08-20 07:17:50 -04:00
Eric Eastwood 7afb615839
When redacting, keep event fields around that maintain the historical event structure intact (MSC2716) (#10538)
* Keep event fields that maintain the historical event structure intact

Fix https://github.com/matrix-org/synapse/issues/10521

* Add changelog

* Bump room version

* Better changelog text

* Fix up room version after develop merge
2021-08-09 20:23:31 -05:00
Patrick Cloke 0c246dd4a0
Support MSC3289: Room version 8 (#10449)
This adds support for MSC3289: room version 8. This is room version 7 + MSC3083.
2021-08-09 10:46:39 +02:00
Eric Eastwood d0b294ad97
Make historical events discoverable from backfill for servers without any scrollback history (MSC2716) (#10245)
* Make historical messages available to federated servers

Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716

Follow-up to https://github.com/matrix-org/synapse/pull/9247

* Debug message not available on federation

* Add base starting insertion point when no chunk ID is provided

* Fix messages from multiple senders in historical chunk

Follow-up to https://github.com/matrix-org/synapse/pull/9247

Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716

---

Previously, Synapse would throw a 403,
`Cannot force another user to join.`,
because we were trying to use `?user_id` from a single virtual user
which did not match with messages from other users in the chunk.

* Remove debug lines

* Messing with selecting insertion event extremeties

* Move db schema change to new version

* Add more better comments

* Make a fake requester with just what we need

See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080

* Store insertion events in table

* Make base insertion event float off on its own

See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889

Conflicts:
	synapse/rest/client/v1/room.py

* Validate that the app service can actually control the given user

See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455

Conflicts:
	synapse/rest/client/v1/room.py

* Add some better comments on what we're trying to check for

* Continue debugging

* Share validation logic

* Add inserted historical messages to /backfill response

* Remove debug sql queries

* Some marker event implemntation trials

* Clean up PR

* Rename insertion_event_id to just event_id

* Add some better sql comments

* More accurate description

* Add changelog

* Make it clear what MSC the change is part of

* Add more detail on which insertion event came through

* Address review and improve sql queries

* Only use event_id as unique constraint

* Fix test case where insertion event is already in the normal DAG

* Remove debug changes

* Switch to chunk events so we can auth via power_levels

Previously, we were using `content.chunk_id` to connect one
chunk to another. But these events can be from any `sender`
and we can't tell who should be able to send historical events.
We know we only want the application service to do it but these
events have the sender of a real historical message, not the
application service user ID as the sender. Other federated homeservers
also have no indicator which senders are an application service on
the originating homeserver.

So we want to auth all of the MSC2716 events via power_levels
and have them be sent by the application service with proper
PL levels in the room.

* Switch to chunk events for federation

* Add unstable room version to support new historical PL

* Fix federated events being rejected for no state_groups

Add fix from https://github.com/matrix-org/synapse/pull/10439
until it merges.

* Only connect base insertion event to prev_event_ids

Per discussion with @erikjohnston,
https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$12bTUiObDFdHLAYtT7E-BvYRp3k_xv8w0dUQHibasJk?via=jki.re&via=matrix.org

* Make it possible to get the room_version with txn

* Allow but ignore historical events in unsupported room version

See https://github.com/matrix-org/synapse/pull/10245#discussion_r675592489

We can't reject historical events on unsupported room versions because homeservers without knowledge of MSC2716 or the new room version don't reject historical events either.

Since we can't rely on the auth check here to stop historical events on unsupported room versions, I've added some additional checks in the processing/persisting code (`synapse/storage/databases/main/events.py` ->  `_handle_insertion_event` and `_handle_chunk_event`). I've had to do some refactoring so there is method to fetch the room version by `txn`.

* Move to unique index syntax

See https://github.com/matrix-org/synapse/pull/10245#discussion_r675638509

* High-level document how the insertion->chunk lookup works

* Remove create_event fallback for room_versions

See https://github.com/matrix-org/synapse/pull/10245/files#r677641879

* Use updated method name
2021-07-28 10:46:37 -05:00
Erik Johnston 8c201c97ec Synapse 1.39.0rc3 (2021-07-28)
==============================
 
 Bugfixes
 --------
 
 - Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477))
 - Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485))
 - Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486))
 
 Internal Changes
 ----------------
 
 - Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461))
 - Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465))
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAmEBUJUQHGVyaWtAbWF0
 cml4Lm9yZwAKCRClQuTtGw+sCWi6CACfA3B7rXfQIO893mctSZkNhtAG/e4o310M
 etvjAtovKmKeFqBvm89FPmFoXvGA2ypoz7Jz2fdkP56DNKIXvihnDc8EeQ3gGawf
 hQ8GBjICOL1e7R/4qzuOCWYdppBGoGNAbz7qyxAUxZ/XOzsPJ2VBGkgyXWZkyPe/
 nJYsuMuMp117HUwTqPqs3oZuTN0MRTy6xgaDfbFbcX9UC/amLmGIWOhlme8iV15r
 HQAW0X7F2Un5h7eQwogWTDb9gBHKnJ4ApaSJLfZl9E72i0Sv7o9UFcLIdy2DRVSq
 gxlRBVzVU6ZndCQ4vQXFSjQ20VGNXc2vpKVRE1D6YXxmxVNbBQB4
 =lb6H
 -----END PGP SIGNATURE-----

Merge tag 'v1.39.0rc3' into develop

Synapse 1.39.0rc3 (2021-07-28)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477))
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485))
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486))

Internal Changes
----------------

- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461))
- Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465))
2021-07-28 15:53:53 +01:00
Šimon Brandner c3b037795a
Support for MSC2285 (hidden read receipts) (#10413)
Implementation of matrix-org/matrix-doc#2285
2021-07-28 10:05:11 +02:00
Andrew Morgan 74d09a43d9
Always communicate device OTK counts to clients (#10485)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-07-27 14:36:38 +01:00
Patrick Cloke 228decfce1
Update the MSC3083 support to verify if joins are from an authorized server. (#10254) 2021-07-26 12:17:00 -04:00
Eric Eastwood 2d89c66b88
Switch to `chunk` events so we can auth via power_levels (MSC2716) (#10432)
Previously, we were using `content.chunk_id` to connect one
chunk to another. But these events can be from any `sender`
and we can't tell who should be able to send historical events.
We know we only want the application service to do it but these
events have the sender of a real historical message, not the
application service user ID as the sender. Other federated homeservers
also have no indicator which senders are an application service on
the originating homeserver.

So we want to auth all of the MSC2716 events via power_levels
and have them be sent by the application service with proper
PL levels in the room.
2021-07-21 10:29:57 +00:00
Michael Telatynski 69226c1ab4
MSC3244 room capabilities implementation (#10283) 2021-07-20 12:59:23 +01:00
Brendan Abolivier 36dc15412d
Add a module type for account validity (#9884)
This adds an API for third-party plugin modules to implement account validity, so they can provide this feature instead of Synapse. The module implementing the current behaviour for this feature can be found at https://github.com/matrix-org/synapse-email-account-validity.

To allow for a smooth transition between the current feature and the new module, hooks have been added to the existing account validity endpoints to allow their behaviours to be overridden by a module.
2021-07-16 18:11:53 +02:00
Jonathan de Jong bf72d10dbf
Use inline type hints in various other places (in `synapse/`) (#10380) 2021-07-15 11:02:43 +01:00
Eric Eastwood 0d5b08ac7a
Fix messages from multiple senders in historical chunk (MSC2716) (#10276)
Fix messages from multiple senders in historical chunk. This also means that an app service does not need to define `?user_id` when using this endpoint.

Follow-up to https://github.com/matrix-org/synapse/pull/9247

Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
2021-07-13 14:12:33 -05:00
Patrick Cloke 8d609435c0
Move methods involving event authentication to EventAuthHandler. (#10268)
Instead of mixing them with user authentication methods.
2021-07-01 14:25:37 -04:00
Patrick Cloke aaf7d1acb8
Correct type hints for synapse.event_auth. (#10253) 2021-06-30 07:08:42 -04:00
Patrick Cloke f55836929d
Do not recurse into non-spaces in the spaces summary. (#10256)
Previously m.child.room events in non-space rooms would be
treated as part of the room graph, but this is no longer
supported.
2021-06-29 12:00:04 -04:00
Quentin Gliech bd4919fb72
MSC2918 Refresh tokens implementation (#9450)
This implements refresh tokens, as defined by MSC2918

This MSC has been implemented client side in Hydrogen Web: vector-im/hydrogen-web#235

The basics of the MSC works: requesting refresh tokens on login, having the access tokens expire, and using the refresh token to get a new one.

Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
2021-06-24 14:33:20 +01:00
Eric Eastwood 96f6293de5
Add endpoints for backfilling history (MSC2716) (#9247)
Work on https://github.com/matrix-org/matrix-doc/pull/2716
2021-06-22 10:02:53 +01:00
Patrick Cloke 8c97d5863f
Update MSC3083 support per changes in the MSC. (#10189)
Adds a "type" field and generalize "space" to "room_id".
2021-06-17 12:53:27 -04:00
Richard van der Hoff 9e405034e5
Make opentracing trace into event persistence (#10134)
* Trace event persistence

When we persist a batch of events, set the parent opentracing span to the that
from the request, so that we can trace all the way in.

* changelog

* When we force tracing, set a baggage item

... so that we can check again later.

* Link in both directions between persist_events spans
2021-06-16 11:41:15 +01:00
Patrick Cloke 4911f7931d
Remove support for unstable MSC1772 prefixes. (#10161)
The stable prefixes have been supported since v1.34.0. The unstable
prefixes are not supported by any known clients.
2021-06-15 08:03:17 -04:00
Patrick Cloke 9e5ab6dd58
Remove the experimental flag for knocking and use stable prefixes / endpoints. (#10167)
* Room version 7 for knocking.
* Stable prefixes and endpoints (both client and federation) for knocking.
* Removes the experimental configuration flag.
2021-06-15 07:45:14 -04:00
Sorunome d936371b69
Implement knock feature (#6739)
This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403

Signed-off-by: Sorunome mail@sorunome.de
Signed-off-by: Andrew Morgan andrewm@element.io
2021-06-09 19:39:51 +01:00
Richard van der Hoff ed53bf314f
Set opentracing priority before setting other tags (#10092)
... because tags on spans which aren't being sampled get thrown away.
2021-05-28 16:14:08 +01:00
Patrick Cloke f42e4c4eb9
Remove the experimental spaces enabled flag. (#10063)
In lieu of just always enabling the unstable spaces endpoint and
unstable room version.
2021-05-26 14:35:16 -04:00
Richard van der Hoff c14f99be46
Support enabling opentracing by user (#9978)
Add a config option which allows enabling opentracing by user id, eg for
debugging requests made by a test user.
2021-05-14 10:51:08 +01:00
Brendan Abolivier a683028d81
Correctly ratelimit invites when creating a room (#9968)
* Correctly ratelimit invites when creating a room

Also allow ratelimiting for more than one action at a time.
2021-05-12 16:05:28 +02:00
Richard van der Hoff 7967b36efe
Fix `m.room_key_request` to-device messages (#9961)
fixes #9960
2021-05-11 11:02:56 +01:00
Patrick Cloke 70f0ffd2fc Follow-up to #9915 to correct the identifier for room types. 2021-05-05 16:31:16 -04:00
Patrick Cloke e2a443550e
Support stable MSC1772 spaces identifiers. (#9915)
Support both the unstable and stable identifiers. A future release
will disable the unstable identifiers.
2021-05-05 11:56:51 -04:00
Richard van der Hoff 3ff2251754
Improved validation for received requests (#9817)
* Simplify `start_listening` callpath

* Correctly check the size of uploaded files
2021-04-23 19:20:44 +01:00
Patrick Cloke e83627926f
Add type hints to auth and auth_blocking. (#9876) 2021-04-23 12:02:16 -04:00
Patrick Cloke d924827da1
Check for space membership during a remote join of a restricted room (#9814)
When receiving a /send_join request for a room with join rules set to 'restricted',
check if the user is a member of the spaces defined in the 'allow' key of the join rules.

This only applies to an experimental room version, as defined in MSC3083.
2021-04-23 07:05:51 -04:00
Andrew Morgan 71f0623de9
Port "Allow users to click account renewal links multiple times without hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832)
This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).
2021-04-19 19:16:34 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Will Hunt e300ef64b1
Require AppserviceRegistrationType (#9548)
This change ensures that the appservice registration behaviour follows the spec. We decided to do this for Dendrite, so it made sense to also make a PR for synapse to correct the behaviour.
2021-04-12 15:13:55 +01:00
Patrick Cloke 35c5ef2d24
Add an experimental room version to support restricted join rules. (#9717)
Per MSC3083.
2021-03-31 16:39:08 -04:00
Erik Johnston 963f4309fe
Make RateLimiter class check for ratelimit overrides (#9711)
This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited.

We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits.

Fixes #9663
2021-03-30 12:06:09 +01:00
Erik Johnston b5efcb577e
Make it possible to use dmypy (#9692)
Running `dmypy run` will do a `mypy` check while spinning up a daemon
that makes rerunning `dmypy run` a lot faster.

`dmypy` doesn't support `follow_imports = silent` and has
`local_partial_types` enabled, so this PR enables those options and
fixes the issues that were newly raised. Note that `local_partial_types`
will be enabled by default in upcoming mypy releases.
2021-03-26 16:49:46 +00:00
Brendan Abolivier 592d6305fd
Merge branch 'develop' into babolivier/msc3026 2021-03-19 16:12:40 +01:00
Richard van der Hoff 004234f03a
Initial spaces summary API (#9643)
This is very bare-bones for now: federation will come soon, while pagination is descoped for now but will come later.
2021-03-18 18:24:16 +00:00
Brendan Abolivier 405aeb0b2c
Implement MSC3026: busy presence state 2021-03-18 16:34:47 +01:00
Patrick Cloke 55da8df078
Fix additional type hints from Twisted 21.2.0. (#9591) 2021-03-12 11:37:57 -05:00
Richard van der Hoff 7eb6e39a8f
Record the SSO Auth Provider in the login token (#9510)
This great big stack of commits is a a whole load of hoop-jumping to make it easier to store additional values in login tokens, and then to actually store the SSO Identity Provider in the login token. (Making use of that data will follow in a subsequent PR.)
2021-03-04 14:44:22 +00:00
Patrick Cloke fc8b3d8809
Ratelimit cross-user key sharing requests. (#8957) 2021-02-19 13:20:34 -05:00
Patrick Cloke d2f0ec12d5
Add type hints to groups code. (#9393) 2021-02-17 08:41:47 -05:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke e40d88cff3
Backout changes for automatically calculating the public baseurl. (#9313)
This breaks some people's configurations (if their Client-Server API
is not accessed via port 443).
2021-02-11 11:16:54 -05:00
Richard van der Hoff fa50e4bf4d
Give `public_baseurl` a default value (#9159) 2021-01-20 12:30:41 +00:00
Richard van der Hoff 0f8945e166
Kill off `HomeServer.get_ip_from_request()` (#9080)
Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.
2021-01-12 12:48:12 +00:00
Richard van der Hoff 2ec8ca5e60
Remove SynapseRequest.get_user_agent (#9069)
SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests",
which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the 
LoggingContext and write the right entries to the request log).

Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a
SynapseRequest when there is nothing synapse-specific about the Request at all, and any old
twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult.

In short: move get_user_agent out to a utility function.
2021-01-12 12:34:16 +00:00
Patrick Cloke 9dde9c9f01
Implement MSC2176: Updated redaction rules (#8984)
An experimental room version ("org.matrix.msc2176") contains
the new redaction rules for testing.
2021-01-05 07:41:48 -05:00
Erik Johnston a7a913918c Merge remote-tracking branch 'origin/erikj/as_mau_block' into develop 2020-12-18 09:51:56 +00:00
Erik Johnston 70586aa63e
Try and drop stale extremities. (#8929)
If we see stale extremities while persisting events, and notice that
they don't change the result of state resolution, we drop them.
2020-12-18 09:49:18 +00:00
Erik Johnston 7932d4e9f7 Don't MAU limit AS ghost users 2020-12-17 12:04:14 +00:00
Patrick Cloke be2db93b3c
Do not assume that the contents dictionary includes history_visibility. (#8945) 2020-12-16 08:46:37 -05:00
Erik Johnston a8eceb01e5
Honour AS ratelimit settings for /login requests (#8920)
Fixes #8846.
2020-12-11 16:33:31 +00:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Erik Johnston f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Erik Johnston c850dd9a8e
Fix handling of User-Agent headers with bad utf-8. (#8632) 2020-10-23 17:12:59 +01:00
Richard van der Hoff 0ec0bc3886 type annotations for LruCache 2020-10-16 15:56:39 +01:00
Richard van der Hoff 3ee17585cd
Make LruCache register its own metrics (#8561)
rather than have everything that instantiates an LruCache manage metrics
separately, have LruCache do it itself.
2020-10-16 15:51:57 +01:00
Patrick Cloke c5251c6fbd
Do not assume that account data is of the correct form. (#8454)
This fixes a bug where `m.ignored_user_list` was assumed to be a dict,
leading to odd behavior for users who set it to something else.
2020-10-05 09:28:05 -04:00
Mathieu Velten 916bb9d0d1
Don't push if an user account has expired (#8353) 2020-09-23 16:06:28 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Andrew Morgan a3a90ee031
Show a confirmation page during user password reset (#8004)
This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset.

This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.
2020-09-10 11:45:12 +01:00
Patrick Cloke 72bec36d50
Directly import json from the standard library. (#8259)
By importing from canonicaljson the simplejson module was still being used
in some situations. After this change the std lib json is consistenty used
throughout Synapse.
2020-09-08 07:33:48 -04:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Will Hunt cbbf9126cb
Do not apply ratelimiting on joins to appservices (#8139)
Add new method ratelimiter.can_requester_do_action and ensure that appservices are exempt from being ratelimited.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
2020-08-21 15:07:56 +01:00
Patrick Cloke e259d63f73
Stop shadow-banned users from sending invites. (#8095) 2020-08-20 15:07:42 -04:00
Patrick Cloke eebf52be06
Be stricter about JSON that is accepted by Synapse (#8106) 2020-08-19 07:26:03 -04:00
Patrick Cloke ad6190c925
Convert stream database to async/await. (#8074) 2020-08-17 07:24:46 -04:00
Patrick Cloke ac77cdb64e
Add a shadow-banned flag to users. (#8092) 2020-08-14 12:37:59 -04:00
Patrick Cloke d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Patrick Cloke 66f24449dd
Improve performance of the register endpoint (#8009) 2020-08-06 08:09:55 -04:00
Patrick Cloke 8553f46498
Convert a synapse.events to async/await. (#7949) 2020-07-27 13:40:22 -04:00
Patrick Cloke b975fa2e99
Convert state resolution to async/await (#7942) 2020-07-24 10:59:51 -04:00
Patrick Cloke 35450519de
Ensure that calls to `json.dumps` are compatible with the standard library json. (#7836) 2020-07-15 13:40:54 -04:00
Patrick Cloke 4db1509516
Improve the type hints of synapse.api.errors. (#7820) 2020-07-14 07:03:58 -04:00
Patrick Cloke 66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802) 2020-07-10 14:30:08 -04:00
Patrick Cloke 38e1fac886
Fix some spelling mistakes / typos. (#7811) 2020-07-09 09:52:58 -04:00
Will Hunt 62b1ce8539
isort 5 compatibility (#7786)
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Patrick Cloke bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. (#7692) 2020-06-15 07:03:36 -04:00
Andrew Morgan fcd6961441
Add option to enable encryption by default for new rooms (#7639)
Fixes https://github.com/matrix-org/synapse/issues/2431

Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.

Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637

Based on #7637
2020-06-10 17:44:34 +01:00
Patrick Cloke a0d2d81cf9
Update to the stable SSO prefix for UI Auth. (#7630) 2020-06-05 10:50:08 -04:00
Richard van der Hoff eea124370b
Fix type information on `assert_*_is_admin` methods (#7645)
These things don't return Deferreds.
2020-06-05 14:33:49 +01:00
Richard van der Hoff b4f8dcb4bd
Remove some unused constants. (#7644) 2020-06-05 14:33:35 +01:00
Andrew Morgan f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595)
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
Patrick Cloke 08bc80ef09
Implement room version 6 (MSC2240). (#7506) 2020-05-15 09:30:10 -04:00
Patrick Cloke 56b66db78a
Strictly enforce canonicaljson requirements in a new room version (#7381) 2020-05-14 13:24:01 -04:00
Patrick Cloke fef3ff5cc4
Enforce MSC2209: auth rules for notifications in power level event (#7502)
In a new room version, the "notifications" key of power level events are
subject to restricted auth rules.
2020-05-14 12:38:17 -04:00
Andrew Morgan 225c165087
Allow expired accounts to logout (#7443) 2020-05-14 16:32:49 +01:00
Amber Brown 7cb8b4bc67
Allow configuration of Synapse's cache without using synctl or environment variables (#6391) 2020-05-11 18:45:23 +01:00
Andrew Morgan aee9130a83
Stop Auth methods from polling the config on every req. (#7420) 2020-05-06 15:54:58 +01:00
Andrew Morgan 6b22921b19
async/await is_server_admin (#7363) 2020-05-01 15:15:36 +01:00
Richard van der Hoff 71a1abb8a1
Stop the master relaying USER_SYNC for other workers (#7318)
Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication.

In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits.

Fixes (I hope) #7257.
2020-04-22 22:39:04 +01:00
Andrew Morgan 07b88c546d
Convert http.HTTPStatus objects to their int equivalent (#7188) 2020-04-03 14:26:07 +01:00
Patrick Cloke b9930d24a0
Support SAML in the user interactive authentication workflow. (#7102) 2020-04-01 08:48:00 -04:00
Dirk Klimpel e8e2ddb60a
Allow server admins to define and enforce a password policy (MSC2000). (#7118) 2020-03-26 16:51:13 +00:00
Patrick Cloke 06eb5cae08
Remove special auth and redaction rules for aliases events in experimental room ver. (#7037) 2020-03-09 08:58:25 -04:00
Patrick Cloke 13892776ef
Allow deleting an alias if the user has sufficient power level (#6986) 2020-03-04 11:30:46 -05:00
Patrick Cloke 7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Patrick Cloke 7936d2a96e
Publishing/removing from the directory requires a power level greater than canonical aliases. 2020-02-21 07:18:33 -05:00
Richard van der Hoff a0a1fd0bec Add `allow_departed_users` param to `check_in_room_or_world_readable`
... and set it everywhere it's called.

while we're here, rename it for consistency with `check_user_in_room` (and to
help check that I haven't missed any instances)
2020-02-19 08:52:51 +00:00
Richard van der Hoff b58d17e44f Refactor the membership check methods in Auth
these were getting a bit unwieldy, so let's combine `check_joined_room` and
`check_user_was_in_room` into a single `check_user_in_room`.
2020-02-18 23:21:44 +00:00
Erik Johnston 6475382d80
Fix detecting unknown devices from remote encrypted events. (#6848)
We were looking at the wrong event type (`m.room.encryption` vs
`m.room.encrypted`).

Also fixup the duplicate `EvenTypes` entries.

Introduced in #6776.
2020-02-04 17:25:54 +00:00
Richard van der Hoff 08f41a6f05 Add `get_room_version` method
So that we can start factoring out some of this boilerplatey boilerplate.
2020-01-31 10:28:15 +00:00
Richard van der Hoff 49d3bca37b Implement updated auth rules from MSC2260 2020-01-28 14:20:10 +00:00
Richard van der Hoff a8ce7aeb43
Pass room version object into event_auth.check and check_redaction (#6788)
These are easier to work with than the strings and we normally have one around.

This fixes `FederationHander._persist_auth_tree` which was passing a
RoomVersion object into event_auth.check instead of a string.
2020-01-28 14:18:29 +00:00
Erik Johnston 0f6e525be3
Fixup synapse.api to pass mypy (#6733) 2020-01-20 17:34:13 +00:00
Erik Johnston d386f2f339
Add StateMap type alias (#6715) 2020-01-16 13:31:22 +00:00
Richard van der Hoff 8f5d7302ac
Implement RedirectException (#6687)
Allow REST endpoint implemnentations to raise a RedirectException, which will
redirect the user's browser to a given location.
2020-01-15 15:58:55 +00:00
Richard van der Hoff 326c893d24
Kill off RegistrationError (#6691)
This is pretty pointless. Let's just use SynapseError.
2020-01-13 12:48:22 +00:00
Richard van der Hoff 98247c4a0e
Remove unused, undocumented "content repo" resource (#6628)
This looks like it got half-killed back in #888.

Fixes #6567.
2020-01-03 17:10:52 +00:00
Erik Johnston fa780e9721
Change EventContext to use the Storage class (#6564) 2019-12-20 10:32:02 +00:00
Erik Johnston 3fbe5b7ec3
Add auth events as per spec. (#6556)
Previously we tried to be clever and filter out some unnecessary event
IDs to keep the auth chain small, but that had some annoying
interactions with state res v2 so we stop doing that for now.
2019-12-16 16:59:32 +00:00
Brendan Abolivier c530f9af4d
Merge pull request #6329 from matrix-org/babolivier/context_filters
Filter state, events_before and events_after in /context requests
2019-12-04 15:24:16 +00:00
Brendan Abolivier 9dc84b7989
Merge branch 'develop' into babolivier/context_filters 2019-12-04 14:23:44 +00:00
Brendan Abolivier 54dd5dc12b
Add ephemeral messages support (MSC2228) (#6409)
Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). The parts that differ are:

* the feature is hidden behind a configuration flag (`enable_ephemeral_messages`)
* self-destruction doesn't happen for state events
* only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one)
* doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
2019-12-03 19:19:45 +00:00
Brendan Abolivier 9e937c28ee Merge branch 'develop' into babolivier/message_retention 2019-11-26 17:53:57 +00:00
Andrew Morgan bc29a19731 Replace instance variations of homeserver with correct case/spacing 2019-11-12 13:08:12 +00:00
Brendan Abolivier 8822b33111
Update copyrights 2019-11-05 14:46:35 +00:00
Brendan Abolivier 09957ce0e4
Implement per-room message retention policies 2019-11-04 17:09:22 +00:00
Brendan Abolivier f496d25877
Merge pull request #6301 from matrix-org/babolivier/msc2326
Implement MSC2326 (label based filtering)
2019-11-01 17:04:45 +00:00
Brendan Abolivier 988d8d6507
Incorporate review 2019-11-01 16:22:44 +00:00
Brendan Abolivier 57cdb046e4
Lint 2019-11-01 10:39:14 +00:00
Brendan Abolivier c6dbca2422
Incorporate review 2019-11-01 10:30:51 +00:00
Andrew Morgan 54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Brendan Abolivier e7943f660a
Add unit tests 2019-10-30 16:15:04 +00:00
Brendan Abolivier acd16ad86a
Implement filtering 2019-10-30 15:56:33 +00:00
Brendan Abolivier fa0dcbc8fa
Store labels for new events 2019-10-30 14:27:15 +00:00
Neil Johnson 2794b79052 Option to suppress resource exceeded alerting (#6173)
The expected use case is to suppress MAU limiting on small instances
2019-10-24 11:48:46 +01:00
Richard van der Hoff 93eaeec75a
Remove Auth.check method (#6217)
This method was somewhat redundant, and confusing.
2019-10-18 19:43:36 +02:00