Commit Graph

3034 Commits

Author SHA1 Message Date
Richard van der Hoff 8c97f6414c
Remove non-functional 'expire_access_token' setting (#5782)
The `expire_access_token` didn't do what it sounded like it should do. What it
actually did was make Synapse enforce the 'time' caveat on macaroons used as
access tokens, but since our access token macaroons never contained such a
caveat, it was always a no-op.

(The code to add 'time' caveats was removed back in v0.18.5, in #1656)
2019-07-30 08:25:02 +01:00
Amber Brown 865077f1d1
Room Complexity Client Implementation (#5783) 2019-07-30 02:47:27 +10:00
Erik Johnston aecae8f397 Correctly handle errors doing requests to group servers 2019-07-29 17:21:57 +01:00
Amber Brown 97a8b4caf7
Move some timeout checking logs to DEBUG #5785 2019-07-30 02:02:18 +10:00
Jorik Schellekens 85b0bd8fe0 Update the device list cache when keys/query is called (#5693) 2019-07-29 16:34:44 +01:00
Richard van der Hoff 1a93daf353
Merge pull request #5744 from matrix-org/erikj/log_leave_origin_mismatch
Log when we receive a /make_* request from a different origin
2019-07-26 12:38:37 +01:00
Richard van der Hoff d1020653fc Log when we receive a /make_* request from a different origin 2019-07-26 10:08:22 +01:00
Hubert Chathi c659b9f94f allow uploading keys for cross-signing 2019-07-25 11:08:24 -04:00
Erik Johnston 2276936bac
Merge pull request #5743 from matrix-org/erikj/log_origin_receipts_mismatch
Log when we receive receipt from a different origin
2019-07-24 13:27:57 +01:00
Richard van der Hoff f30a71a67b
Stop trying to fetch events with event_id=None. (#5753)
`None` is not a valid event id, so queuing up a database fetch for it seems
like a silly thing to do.

I considered making `get_event` return `None` if `event_id is None`, but then
its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
2019-07-24 13:16:18 +01:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Erik Johnston 3db1377b26 Log when we receive receipt from a different origin 2019-07-23 13:31:03 +01:00
Richard van der Hoff 9c70a02a9c
Ignore redactions of m.room.create events (#5701) 2019-07-17 19:08:02 +01:00
Andrew Morgan 18c516698e
Return a different error from Invalid Password when a user is deactivated (#5674)
Return `This account has been deactivated` instead of `Invalid password` when a user is deactivated.
2019-07-15 11:45:29 +01:00
Erik Johnston d86321300a
Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_data
Add basic function to get all data for a user out of synapse
2019-07-15 10:04:02 +01:00
Richard van der Hoff 5f158ec039
Implement access token expiry (#5660)
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Richard van der Hoff 1890cfcf82 Inline issue_access_token (#5659)
this is only used in one place, so it's clearer if we inline it and reduce the
API surface.

Also, fixes a buglet where we would create an access token even if we were
about to block the user (we would never return the AT, so the user could never
use it, but it was still created and added to the db.)
2019-07-11 04:10:07 +10:00
Brendan Abolivier 8ab3444fdf
Merge pull request #5658 from matrix-org/babolivier/is-json
Send 3PID bind requests as JSON data
2019-07-10 17:01:26 +01:00
Richard van der Hoff 953dbb7980
Remove access-token support from RegistrationStore.register (#5642)
The 'token' param is no longer used anywhere except the tests, so let's kill
that off too.
2019-07-10 16:26:49 +01:00
Brendan Abolivier b2a2e96ea6
Typo 2019-07-10 15:56:21 +01:00
Brendan Abolivier f77e997619
Send 3PID bind requests as JSON data 2019-07-10 15:46:42 +01:00
Brendan Abolivier 65434da75d
Merge pull request #5638 from matrix-org/babolivier/invite-json
Use JSON when querying the IS's /store-invite endpoint
2019-07-09 18:48:38 +01:00
Hubert Chathi 7b3bc755a3 remove unused and unnecessary check for FederationDeniedError (#5645)
FederationDeniedError is a subclass of SynapseError, which is a subclass of
CodeMessageException, so if e is a FederationDeniedError, then this check for
FederationDeniedError will never be reached since it will be caught by the
check for CodeMessageException above.  The check for CodeMessageException does
almost the same thing as this check (since FederationDeniedError initialises
with code=403 and msg="Federation denied with %s."), so may as well just keep
allowing it to handle this case.
2019-07-09 18:37:39 +01:00
Brendan Abolivier af67c7c1de
Merge pull request #5644 from matrix-org/babolivier/profile-allow-self
Allow newly-registered users to lookup their own profiles
2019-07-09 10:25:40 +01:00
Richard van der Hoff 824707383b
Remove access-token support from RegistrationHandler.register (#5641)
Nothing uses this now, so we can remove the dead code, and clean up the
API.

Since we're changing the shape of the return value anyway, we take the
opportunity to give the method a better name.
2019-07-08 19:01:08 +01:00
Brendan Abolivier 7556851665
Allow newly-registered users to lookup their own profiles
When a user creates an account and the 'require_auth_for_profile_requests' config flag is set, and a client that performed the registration wants to lookup the newly-created profile, the request will be denied because the user doesn't share a room with themselves yet.
2019-07-08 17:31:00 +01:00
Richard van der Hoff b70e080b59
Better logging for auto-join. (#5643)
It was pretty unclear what was going on, so I've added a couple of log lines.
2019-07-08 17:14:51 +01:00
Brendan Abolivier 57eacee4f4 Merge branch 'develop' into babolivier/invite-json 2019-07-08 15:49:23 +01:00
Richard van der Hoff 1af2fcd492 Move get_or_create_user to test code (#5628)
This is only used in tests, so...
2019-07-08 23:52:26 +10:00
Brendan Abolivier f05c7d62bc
Lint 2019-07-08 14:29:27 +01:00
Brendan Abolivier 1a807dfe68
Use application/json when querying the IS's /store-invite endpoint 2019-07-08 14:19:39 +01:00
Richard van der Hoff 80cc82a445
Remove support for invite_3pid_guest. (#5625)
This has never been documented, and I'm not sure it's ever been used outside
sytest.

It's quite a lot of poorly-maintained code, so I'd like to get rid of it.

For now I haven't removed the database table; I suggest we leave that for a
future clearout.
2019-07-05 16:47:58 +01:00
Erik Johnston b4f5416dd9 pep8 2019-07-05 14:41:29 +01:00
Erik Johnston eadb13d2e9 Remove FileExfiltrationWriter 2019-07-05 14:15:00 +01:00
Erik Johnston 7f0d8e4288 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_exfiltrate_data 2019-07-05 14:08:21 +01:00
Erik Johnston 9ccea16d45 Assume key existence. Update docstrings 2019-07-05 14:07:56 +01:00
Erik Johnston c061d4f237 Fixup from review comments. 2019-07-04 11:41:06 +01:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Erik Johnston d0b849c86d
Apply comment fixups from code review
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-03 15:03:38 +01:00
Richard van der Hoff 91753cae59
Fix a number of "Starting txn from sentinel context" warnings (#5605)
Fixes #5602, #5603
2019-07-03 09:31:27 +01:00
Erik Johnston 8ee69f299c Add basic function to get all data for a user out of synapse 2019-07-02 12:09:04 +01:00
Richard van der Hoff 6eecb6e500
Complete the SAML2 implementation (#5422)
* SAML2 Improvements and redirect stuff

Signed-off-by: Alexander Trost <galexrt@googlemail.com>

* Code cleanups and simplifications.

Also: share the saml client between redirect and response handlers.

* changelog

* Revert redundant changes to static js

* Move all the saml stuff out to a centralised handler

* Add support for tracking SAML2 sessions.

This allows us to correctly handle `allow_unsolicited: False`.

* update sample config

* cleanups

* update sample config

* rename BaseSSORedirectServlet for consistency

* Address review comments
2019-07-02 11:18:11 +01:00
Richard van der Hoff b4fd86a9b4 Merge branch 'develop' into rav/saml2_client 2019-07-01 14:21:03 +01:00
Richard van der Hoff 3bcb13edd0 Address review comments 2019-07-01 12:13:22 +01:00
Erik Johnston 04196a4dae
Merge pull request #5507 from matrix-org/erikj/presence_sync_tighloop
Fix sync tightloop bug.
2019-07-01 11:43:10 +01:00
Erik Johnston 915280f1ed Fixup comment 2019-07-01 10:22:42 +01:00
Brendan Abolivier c7ff297dde
Merge pull request #5576 from matrix-org/babolivier/3pid-invite-ratelimit
Don't update the ratelimiter before sending a 3PID invite
2019-06-28 17:43:48 +01:00
Brendan Abolivier 15d9fc31bd
Only ratelimit when sending the email
If we do the opposite, an event can arrive after or while sending the email and the 3PID invite event will get ratelimited.
2019-06-28 16:04:05 +01:00
Brendan Abolivier 01d0f8e701
Don't update the ratelimiter before sending a 3PID invite
This would cause emails being sent, but Synapse responding with a 429 when creating the event. The client would then retry, and with bad timing the same scenario would happen again. Some testing I did ended up sending me 10 emails for one single invite because of this.
2019-06-28 15:22:16 +01:00
Daniel Hoffend 9646a593ac Added possibilty to disable local password authentication (#5092)
Signed-off-by: Daniel Hoffend <dh@dotlan.net>
2019-06-27 18:37:29 +01:00
Erik Johnston 729f5a4fb6 Review comments 2019-06-27 16:06:23 +01:00
Richard van der Hoff 28db0ae537 cleanups 2019-06-27 00:37:41 +01:00
Richard van der Hoff 36f4953dec Add support for tracking SAML2 sessions.
This allows us to correctly handle `allow_unsolicited: False`.
2019-06-26 23:50:55 +01:00
Richard van der Hoff 3705322103 Move all the saml stuff out to a centralised handler 2019-06-26 22:52:02 +01:00
Erik Johnston 8624db3194 Refactor and comment sync device list code 2019-06-26 19:39:49 +01:00
Erik Johnston 806a06daf2 Rename get_users_whose_devices_changed 2019-06-26 19:39:19 +01:00
Erik Johnston a2f6d31a63 Refactor get_user_ids_changed to pull less from DB
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.

This PR changes this to only check for changes for users the client is
actually interested in.

Fixes #5553
2019-06-26 12:03:44 +01:00
Andrew Morgan ef8c62758c
Prevent multiple upgrades on the same room at once (#5051)
Closes #4583

Does slightly less than #5045, which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time.

Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
2019-06-25 14:19:21 +01:00
Erik Johnston 25433f212d
Merge pull request #5531 from matrix-org/erikj/workers_pagination_token
Fix /messages on workers when no from param specified.
2019-06-24 15:30:10 +01:00
Richard van der Hoff edea4bb5be Allow configuration of the path used for ACME account keys.
Because sticking it in the same place as the config isn't necessarily the right
thing to do.
2019-06-24 13:51:22 +01:00
Richard van der Hoff 21bf4318b5
Factor acme bits out to a separate file (#5521)
This makes some of the conditional-import hoop-jumping easier.
2019-06-24 11:33:56 +01:00
Erik Johnston dddf20e8e1 Fix /messages on workers when no from param specified.
If no `from` param is specified we calculate and use the "current
token" that inlcuded typing, presence, etc. These are unused during
pagination and are not available on workers, so we simply don't
calculate them.
2019-06-24 10:06:51 +01:00
Erik Johnston 8181e290a9 Fix sync tightloop bug.
If, for some reason, presence updates take a while to persist then it
can trigger clients to tightloop calling `/sync` due to the presence
handler returning updates but not advancing the stream token.

Fixes #5503.
2019-06-21 11:10:27 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston 7dcf984075
Merge pull request #5042 from matrix-org/erikj/fix_get_missing_events_error
Handle the case of `get_missing_events` failing
2019-06-19 13:20:09 +01:00
Erik Johnston e0be8d7016
Merge pull request #5480 from matrix-org/erikj/extremities_dummy_events
Add experimental option to reduce extremities.
2019-06-19 13:19:18 +01:00
Erik Johnston 554609288b Run as background process and fix comments 2019-06-19 11:33:03 +01:00
David Baker f2d2ae03da Add some logging to 3pid invite sig verification (#5015)
I had to add quite a lot of logging to diagnose a problem with 3pid
invites - we only logged the one failure which isn't all that
informative.

NB. I'm not convinced the logic of this loop is right: I think it
should just accept a single valid signature from a trusted source
rather than fail if *any* signature is invalid. Also it should
probably not skip the rest of middle loop if a check fails? However,
I'm deliberately not changing the logic here.
2019-06-18 22:51:24 +01:00
Erik Johnston 2b20d0fb59 Fix logline 2019-06-18 16:12:53 +01:00
Erik Johnston 19b80fe68a Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_get_missing_events_error 2019-06-18 16:11:43 +01:00
Erik Johnston b42f90470f Add experimental option to reduce extremities.
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.

THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
Brendan Abolivier 112cf5a73a Add third party rules hook for 3PID invites 2019-06-17 17:39:09 +01:00
Erik Johnston 187d2837a9 Add third party rules hook into create room 2019-06-17 16:41:19 +01:00
Erik Johnston 6840ebeef8
Merge pull request #5385 from matrix-org/erikj/reduce_http_exceptions
Handle HttpResponseException when using federation client.
2019-06-17 13:54:47 +01:00
Erik Johnston a9dab970b8
Merge pull request #5464 from matrix-org/erikj/3pid_remote_invite_state
Fix 3PID invite room state over federation.
2019-06-17 10:18:28 +01:00
Brendan Abolivier f12e1f029c
Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules
Allow server admins to define implementations of extra rules for allowing or denying incoming events
2019-06-14 19:37:59 +01:00
Brendan Abolivier f874b16b2e Add plugin APIs for implementations of custom event rules. 2019-06-14 18:16:03 +01:00
Erik Johnston 304a1376c2 Fix 3PID invite room state over federation.
Fixes that when a user exchanges a 3PID invite for a proper invite over
federation it does not include the `invite_room_state` key.

This was due to synapse incorrectly sending out two invite requests.
2019-06-14 16:19:11 +01:00
Brendan Abolivier 6d56a694f4 Don't send renewal emails to deactivated users 2019-06-14 15:05:56 +01:00
Brendan Abolivier d0530382ee
Track deactivated accounts in the database (#5378) 2019-06-14 13:18:24 +01:00
Erik Johnston 49e01e5710 Fix defaults on checking threepids 2019-06-10 23:09:31 +01:00
Erik Johnston a46ef1e3a4 Handle HttpResponseException when using federation client.
Otherwise we just log exceptions everywhere.
2019-06-07 10:29:35 +01:00
Andrew Morgan 3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377)
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Richard van der Hoff 71063a69b8
Fix missing logcontext for PresenceHandler.on_shutdown. (#5369)
Fixes some warnings, and a scary-looking stacktrace when sytest kills the
process.
2019-06-06 14:45:17 +01:00
Brendan Abolivier 8f06344e11
Merge pull request #5089 from dnaf/m-heroes-empty-room-name
Make /sync return heroes if room name or canonical alias are empty
2019-06-06 11:18:13 +01:00
Brendan Abolivier 64fa928792
Simplify condition 2019-06-06 10:34:12 +01:00
Brendan Abolivier b9c43c8463
Merge pull request #5355 from matrix-org/babolivier/heroes_left_members
Include left members in room summaries' heroes
2019-06-06 10:05:27 +01:00
Erik Johnston 75538813fc Fix background updates to handle redactions/rejections (#5352)
* Fix background updates to handle redactions/rejections

In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
2019-06-06 00:45:46 +10:00
Brendan Abolivier 0a2f522644
Simplify condition 2019-06-05 14:02:29 +01:00
Brendan Abolivier 40596aec0e Merge branch 'develop' into m-heroes-empty-room-name 2019-06-05 13:38:01 +01:00
Brendan Abolivier dbbaf25dd3
Do user_id != me checks before deciding whether we should pick heroes from the joined members or the parted ones 2019-06-05 11:50:27 +01:00
Erik Johnston bc3d6b918b Add logging when request fails and clarify we ignore errors. 2019-06-05 11:37:51 +01:00
Erik Johnston 220a733d73 Fix handling of failures when calling /event_auth.
When processing an incoming event over federation, we may try and
resolve any unexpected differences in auth events. This is a
non-essential process and so should not stop the processing of the event
if it fails (e.g. due to the remote disappearing or not implementing the
necessary endpoints).

Fixes #3330
2019-06-03 09:56:45 +01:00
Richard van der Hoff d828d1dc57
Merge pull request #5309 from matrix-org/rav/limit_displayname_length
Limit displaynames and avatar URLs
2019-06-01 11:34:50 +01:00
Richard van der Hoff d16c6375fe Limit displaynames and avatar URLs
These end up in join events everywhere, so let's limit them.

Fixes #5079
2019-06-01 10:44:36 +01:00
Richard van der Hoff 5726378ece
Fix "db txn 'update_presence' from sentinel context" log messages (#5275)
Fixes #4414.
2019-05-28 21:20:11 +01:00
Erik Johnston d16f5574b6
Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events
Don't bundle aggregations with events in /sync or /events or state queries
2019-05-24 10:36:31 +01:00
Erik Johnston 4cb577c23f Don't bundle aggs for /state and /members etc APIs 2019-05-24 09:52:33 +01:00
Andrew Morgan 6368150a74
Add config option for setting homeserver's default room version (#5223)
Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present.

That hardcoded value is now located in the server.py config file.
2019-05-23 15:00:20 +01:00
Richard van der Hoff 85d1e03b9d
Simplifications and comments in do_auth (#5227)
I was staring at this function trying to figure out wtf it was actually
doing. This is (hopefully) a non-functional refactor which makes it a bit
clearer.
2019-05-23 11:17:42 +01:00
Amber Brown 4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
Erik Johnston 9526aa96a6
Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactions
Block attempts to annotate the same event twice
2019-05-21 17:08:14 +01:00
Erik Johnston 44b8ba484e Fix words 2019-05-21 16:51:45 +01:00
Richard van der Hoff c4aef549ad
Exclude soft-failed events from fwd-extremity candidates. (#5146)
When considering the candidates to be forward-extremities, we must exclude soft
failures.

Hopefully fixes #5090.
2019-05-21 16:10:54 +01:00
Erik Johnston de7672b78f Don't bundle events in /sync or /events
As we'll send down the annotations too anyway, so this just ends up
confusing clients.
2019-05-21 13:54:09 +01:00
Erik Johnston c7ec06e8a6 Block attempts to annotate the same event twice 2019-05-20 17:39:05 +01:00
ReidAnderson 3787133c9e Limit UserIds to a length that fits in a state key (#5198) 2019-05-20 11:20:08 +01:00
bytepoets-blo 291e1eea5e fix mapping of return values for get_or_register_3pid_guest (#5177)
* fix mapping of return values for get_or_register_3pid_guest
2019-05-17 17:27:14 +01:00
Brendan Abolivier cd32375846 Add option to disable per-room profiles 2019-05-16 14:34:28 +01:00
Matthew Hodgson 4a6d5de98c Make /sync attempt to return device updates for both joined and invited users (#3484) 2019-05-16 13:23:43 +01:00
Amber Brown f1e5b41388
Make all the rate limiting options more consistent (#5181) 2019-05-15 12:06:04 -05:00
Erik Johnston b54b03f9e1 Allow client event serialization to be async 2019-05-14 11:58:01 +01:00
Matthew Hodgson c0e0740bef add options to require an access_token to GET /profile and /publicRooms on CS API (#5083)
This commit adds two config options:

* `restrict_public_rooms_to_local_users`

Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.

* `require_auth_for_profile_requests`

When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.

MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.

Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
Brendan Abolivier 1473058b5e
Do checks on aliases for incoming m.room.aliases events (#5128)
Follow-up to #5124

Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
2019-05-08 17:01:30 +01:00
Erik Johnston de655e669a
Merge pull request #5104 from matrix-org/erikj/ratelimit_3pid_invite
Ratelimit 3pid invites
2019-05-07 10:12:49 +01:00
Erik Johnston 176f31c2e3 Rate limit early 2019-05-02 15:23:08 +01:00
Brendan Abolivier 84196cb231 Add some limitations to alias creation 2019-05-02 11:05:11 +01:00
Erik Johnston d6118c5be6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ratelimit_3pid_invite 2019-04-26 18:14:23 +01:00
Erik Johnston 28a81ed62f Ratelimit 3pid invites
We do ratelimit sending the 3PID invite events, but that happens after
spamming the identity server.
2019-04-26 18:06:25 +01:00
Richard van der Hoff bd0d45ca69 Fix infinite loop in presence handler
Fixes #5102
2019-04-26 11:14:49 +01:00
Katie Wolfe 7e07dc429f
Lint
I probably should've just run autopep8 in the first place...

Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:43:18 -04:00
Katie Wolfe b3e5db402d
Clean up code
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:55 -04:00
Katie Wolfe 5d3ed79944
Show heroes if room name or canonical alias are empty
Fixes #4194

Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:53 -04:00
Katie Wolfe 7f025eb425
Show heroes if room name or canonical alias are empty
Fixes #4194

Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:29 -04:00
Erik Johnston ca90336a69 Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration 2019-04-17 19:44:40 +01:00
Brendan Abolivier eaf41a943b Add management endpoints for account validity 2019-04-17 19:34:45 +01:00
Brendan Abolivier 91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier 20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00
Erik Johnston c132c8e505 Handle the case of `get_missing_events` failing
Currently if a call to `/get_missing_events` fails we log an exception
and stop processing the top level event we received over federation.
Instead let's try and handle it sensibly given it is a somewhat expected
failure mode.
2019-04-10 10:39:54 +01:00
Brendan Abolivier 8e85493b0c
Add config option to block users from looking up 3PIDs (#5010) 2019-04-04 17:25:47 +01:00
Andrew Morgan db265f0642
Prevent kicking users who aren't in the room (#4999)
Prevent kick events from succeeding if the user is not currently in the room.
2019-04-04 13:05:51 +01:00
Andrew Morgan 4a4d5c4fd6
Fix grammar and document get_current_users_in_room (#4998) 2019-04-03 14:32:20 +01:00
Neil Johnson e8419554ff
Remove presence lists (#4989)
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
Erik Johnston 8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915
Implement MSC1915 - 3PID unbind APIs
2019-04-03 11:07:09 +01:00
Andrew Morgan 66e78700a2
Transfer related groups on room upgrade (#4990)
Transfers the m.room.related_groups state event on room upgrade.
2019-04-02 17:15:24 +01:00
Erik Johnston 4ef5d17b96 Correctly handle id_server param 2019-04-02 11:20:09 +01:00
Erik Johnston 24232514bf Remove threepid binding if id server returns 400/404/501 2019-04-02 11:20:09 +01:00
Erik Johnston c75e2017f1 Fixup docstrings 2019-04-02 11:20:06 +01:00
Erik Johnston 057715aaa2 Allowing specifying IS to use in unbind API.
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.

Implements MSC915.
2019-04-01 15:25:18 +01:00
Erik Johnston 9fbbc3d9e5 For unbind poke IS used during binding of 3PID
This changes the behaviour from using the server specified trusted
identity server to using the IS that used during the binding of the
3PID, if known.

This is the behaviour specified by MSC1915.
2019-04-01 15:23:30 +01:00
Erik Johnston 1666c0696a Track IS used to bind 3PIDs
This will then be used to know which IS to default to when unbinding the
threepid.
2019-04-01 15:23:01 +01:00
Richard van der Hoff 54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Andrew Morgan 7a91b9d81c
Allow password providers to bind emails (#4947)
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston 4e5f0f7ca0 Use an assert 2019-03-28 14:05:05 +00:00
Erik Johnston 40e56997bc Review comments 2019-03-28 13:48:41 +00:00
Erik Johnston 197fae1639 Use event streams to calculate presence
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962
2019-03-27 13:41:36 +00:00
Andrew Morgan bbd244c7b2
Support 3PID login in password providers (#4931)
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown ac396a0d32
Refactor out state delta handling into its own class (#4917) 2019-03-25 20:37:08 +11:00
Erik Johnston 01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Erik Johnston 3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston 536a266520 Deny peeking into rooms that have been blocked 2019-03-21 11:20:13 +00:00
Erik Johnston 09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Erik Johnston cd62981a6a Revert spurious delete 2019-03-20 17:51:27 +00:00
Erik Johnston aa959a6c07 Use flags 2019-03-20 17:40:29 +00:00
Erik Johnston 7d47cc1305 Move requester check into assert_accepted_privacy_policy 2019-03-20 17:08:36 +00:00
Erik Johnston 74c46d81fa Only require consent for events with an associated request
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Richard van der Hoff a902d13180
Batch up outgoing read-receipts to reduce federation traffic. (#4890)
Rate-limit outgoing read-receipts as per #4730.
2019-03-20 16:02:25 +00:00
Erik Johnston cc197a61a1 Disable publishing to room list when its disabled 2019-03-20 14:30:36 +00:00
Erik Johnston 2c90422146 Pull out config option 2019-03-20 14:25:58 +00:00
Erik Johnston 7529038e66 Return before we log 2019-03-20 14:25:28 +00:00
Erik Johnston 213c98c00a Add option to disable search room lists
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Neil Johnson 88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Amber Brown 282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Brendan Abolivier 651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier 899e523d6d
Add ratelimiting on login (#4821)
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff 2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston 9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Richard van der Hoff fdcad8eabd Move client receipt processing to federation sender worker.
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff eed7271b3b declare a ReadReceipt class
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan d42c81d724
Transfer local user's push rules on room upgrade (#4838)
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown c0332d095f fixup 2019-03-13 01:30:54 +11:00
Amber Brown 10480c4348 fixup 2019-03-12 21:47:14 +11:00
Amber Brown 5ba8ceab4c fixes 2019-03-12 00:35:31 +11:00
Amber Brown 4abc988c6a
initial 2019-03-11 21:11:36 +11:00
Erik Johnston 8c4896668f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl 2019-03-08 11:44:20 +00:00
Erik Johnston 0ff8163eae Factor out soft fail checks 2019-03-08 11:26:33 +00:00
Erik Johnston 469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston d42b41544a When re-syncing device lists reset the state
We keep track of what stream IDs we've seen so that we know what updates
we've handled or missed. If we re-sync we don't know if the updates
we've seen are included in the re-sync (there may be a race), so we
should reset the seen updates.
2019-03-07 16:04:24 +00:00
Erik Johnston c633fc02d7 Add some debug logging for device list handling 2019-03-07 15:58:03 +00:00
Amber Brown f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Erik Johnston 0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston 8b7790e68f Port #4422 debug logging from hotfixes 2019-03-06 17:29:15 +00:00
Erik Johnston 6d13bdec91 Add docstrings from matrix-org-hotfixes 2019-03-06 17:24:26 +00:00
Erik Johnston a9de04be72 Implement soft fail 2019-03-06 16:22:16 +00:00
Brendan Abolivier 6f3cde8b25 Make registration ratelimiter separate from the main events one 2019-03-06 11:02:42 +00:00
Brendan Abolivier f4195f4118 Revert "Split ratelimiters in two (one for events, one for registration)"
This reverts commit d7dbad3526.
2019-03-06 10:55:22 +00:00
Brendan Abolivier d7dbad3526 Split ratelimiters in two (one for events, one for registration) 2019-03-05 18:41:27 +00:00
Brendan Abolivier a4c3a361b7
Add rate-limiting on registration (#4735)
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
2019-03-05 14:25:33 +00:00
Erik Johnston b050a10871
Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston aa06d26ae0 clarify comments 2019-03-05 09:16:35 +00:00
Erik Johnston c3c542bb4a
Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Erik Johnston bfa7d46a10 Allow /keys/{changes,query} API to run on worker 2019-03-04 18:30:01 +00:00
Erik Johnston 157e5a8f27 Split DeviceHandler into master and worker 2019-03-04 18:29:26 +00:00
Richard van der Hoff b29693a30b Clean up read-receipt handling.
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.

We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Erik Johnston d1523aed6b Only check history visibility when filtering
When filtering events to send to server we check more than just history
visibility. However when deciding whether to backfill or not we only
care about the history visibility.
2019-03-04 14:43:42 +00:00
Richard van der Hoff 856c83f5f8
Avoid rebuilding Edu objects in worker mode (#4770)
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston 8b63fe4c26 s/get_forward_events/get_successor_events/ 2019-03-04 11:56:03 +00:00
Erik Johnston fbc047f2a5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room 2019-03-04 11:54:58 +00:00
Richard van der Hoff 3064952939
Fix incorrect log about not persisting duplicate state event. (#4776)
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Erik Johnston b183fef9ac Update comments 2019-02-27 13:06:10 +00:00