Commit Graph

518 Commits

Author SHA1 Message Date
Hubert Chathi 4cb44a1585
Add support for MSC2697: Dehydrated devices (#8380)
This allows a user to store an offline device on the server and
then restore it at a subsequent login.
2020-10-07 08:00:17 -04:00
Richard van der Hoff 43c622885c
Merge pull request #8463 from matrix-org/rav/clean_up_event_handling
Reduce inconsistencies between codepaths for membership and non-membership events.
2020-10-07 12:20:44 +01:00
Hubert Chathi 3cd78bbe9e
Add support for MSC2732: olm fallback keys (#8312) 2020-10-06 13:26:29 -04:00
Richard van der Hoff e775b5bb5b kill off `send_nonmember_event`
This is now redundant, and we can just call `handle_new_client_event` directly.
2020-10-05 19:04:10 +01:00
BBBSnowball 05ee048f2c
Add config option for always using "userinfo endpoint" for OIDC (#7658)
This allows for connecting to certain IdPs, e.g. GitLab.
2020-10-01 13:54:35 -04:00
Patrick Cloke 8b40843392
Allow additional SSO properties to be passed to the client (#8413) 2020-09-30 13:02:43 -04:00
Tdxdxoz abd04b6af0
Allow existing users to login via OpenID Connect. (#8345)
Co-authored-by: Benjamin Koch <bbbsnowball@gmail.com>

This adds configuration flags that will match a user to pre-existing users
when logging in via OpenID Connect. This is useful when switching to
an existing SSO system.
2020-09-25 07:01:45 -04:00
Dionysis Grigoropoulos 37ca5924bd
Create function to check for long names in devices (#8364)
* Create a new function to verify that the length of a device name is
under a certain threshold.
* Refactor old code and tests to use said function.
* Verify device name length during registration of device
* Add a test for the above

Signed-off-by: Dionysis Grigoropoulos <dgrig@erethon.com>
2020-09-22 11:42:55 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
reivilibre 576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop (#8272) 2020-09-15 09:07:19 +01:00
Erik Johnston c9dbee50ae
Fixup pusher pool notifications (#8287)
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
2020-09-09 16:56:08 +01:00
Erik Johnston dc9dcdbd59 Revert "Fixup pusher pool notifications"
This reverts commit e7fd336a53.
2020-09-09 16:19:22 +01:00
Erik Johnston e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Patrick Cloke cef00211c8
Allow for make_awaitable's return value to be re-used. (#8261) 2020-09-08 07:26:55 -04:00
Andrew Morgan 68cdb3708e
Rename 'populate_stats_process_rooms_2' background job back to 'populate_stats_process_rooms' again (#8243)
Fixes https://github.com/matrix-org/synapse/issues/8238

Alongside the delta file, some changes were also necessary to the codebase to remove references to the now defunct `populate_stats_process_rooms_2` background job. Thankfully the latter doesn't seem to have made it into any documentation yet :)
2020-09-08 11:05:59 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke b055dc9322
Ensure that the OpenID Connect remote ID is a string. (#8190) 2020-08-28 08:56:36 -04:00
Patrick Cloke e00816ad98
Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
Patrick Cloke b49a5b9307
Convert stats and related calls to async/await (#8192) 2020-08-27 17:24:37 -04:00
Patrick Cloke 4a739c73b4
Convert simple_update* and simple_select* to async (#8173) 2020-08-27 07:08:38 -04:00
Andrew Morgan a466b67972
Reduce run-times of tests by advancing the reactor less (#7757) 2020-08-27 11:39:53 +01:00
Patrick Cloke 6fe12c9512
Do not propagate typing notifications from shadow-banned users. (#8176) 2020-08-26 12:05:36 -04:00
Patrick Cloke 4c6c56dc58
Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
Patrick Cloke 3f91638da6
Allow denying or shadow banning registrations via the spam checker (#8034) 2020-08-20 15:42:58 -04:00
Patrick Cloke 050e20e7ca
Convert some of the general database methods to async (#8100) 2020-08-17 12:18:01 -04:00
Patrick Cloke ad6190c925
Convert stream database to async/await. (#8074) 2020-08-17 07:24:46 -04:00
Patrick Cloke 5ecc8b5825
Convert devices database to async/await. (#8069) 2020-08-12 10:51:42 -04:00
Patrick Cloke d68e10f308
Convert account data, device inbox, and censor events databases to async/await (#8063) 2020-08-12 09:29:06 -04:00
Patrick Cloke a0acdfa9e9
Converts event_federation and registration databases to async/await (#8061) 2020-08-11 17:21:13 -04:00
Patrick Cloke 7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042) 2020-08-07 13:36:29 -04:00
Patrick Cloke d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Andrew Morgan 5d92a1428c
Prevent join->join membership transitions changing member count (#7977)
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.

However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.

This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.

Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).

Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
2020-08-03 21:54:24 +01:00
Patrick Cloke c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Patrick Cloke 4cce8ef74e
Convert appservice to async. (#7973) 2020-07-30 07:27:39 -04:00
Patrick Cloke 6b3ac3b8cd
Convert device handler to async/await (#7871) 2020-07-17 07:09:25 -04:00
Patrick Cloke 6fca1b3506
Convert _base, profile, and _receipts handlers to async/await (#7860) 2020-07-17 07:08:30 -04:00
Erik Johnston 9006e125af Fix tests 2020-07-15 15:47:27 +01:00
Patrick Cloke b11450dedc
Convert E2E key and room key handlers to async/await. (#7851) 2020-07-15 08:48:58 -04:00
Patrick Cloke 6d687ebba1
Convert the appservice handler to async/await. (#7775) 2020-07-06 07:40:35 -04:00
Will Hunt 62b1ce8539
isort 5 compatibility (#7786)
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Patrick Cloke 71cccf1593
Additional configuration options for auto-join rooms (#7763) 2020-06-30 15:41:36 -04:00
Patrick Cloke 3630825612
Convert the typing handler to async/await. (#7679) 2020-06-17 10:37:59 -04:00
Hubert Chathi 2b2344652b
Ensure etag is a string for GET room_keys/version response (#7691) 2020-06-15 13:42:44 -04:00
Andrew Morgan fcd6961441
Add option to enable encryption by default for new rooms (#7639)
Fixes https://github.com/matrix-org/synapse/issues/2431

Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.

Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637

Based on #7637
2020-06-10 17:44:34 +01:00
Patrick Cloke 737b4a936e
Convert user directory handler and related classes to async/await. (#7640) 2020-06-05 14:42:55 -04:00
Travis Ralston 09099313e6
Add an option to disable autojoin for guest accounts (#6637)
Fixes https://github.com/matrix-org/synapse/issues/3177
2020-06-05 18:18:15 +01:00
Andrew Morgan f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595)
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
Andrew Morgan c389bfb6ea
Fix encryption algorithm typos in tests/comments (#7637)
@uhoreg has confirmed these were both typos. They are only in comments and tests though, rather than anything critical.

Introduced in:

* https://github.com/matrix-org/synapse/pull/7157
* https://github.com/matrix-org/synapse/pull/5726
2020-06-04 20:03:40 +01:00
Erik Johnston 1531b214fc
Add ability to wait for replication streams (#7542)
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).

Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.

People probably want to look at this commit by commit.
2020-05-22 14:21:54 +01:00
Patrick Cloke a3cf36f76e
Support UI Authentication for OpenID Connect accounts (#7457) 2020-05-15 12:26:02 -04:00
Patrick Cloke 08bc80ef09
Implement room version 6 (MSC2240). (#7506) 2020-05-15 09:30:10 -04:00
Patrick Cloke 56b66db78a
Strictly enforce canonicaljson requirements in a new room version (#7381) 2020-05-14 13:24:01 -04:00
Andrew Morgan 67feea8044
Extend spam checker to allow for multiple modules (#7435) 2020-05-08 19:25:48 +01:00
Quentin Gliech 616af44137
Implement OpenID Connect-based login (#7256) 2020-05-08 08:30:40 -04:00
Andrew Morgan aee9130a83
Stop Auth methods from polling the config on every req. (#7420) 2020-05-06 15:54:58 +01:00
Andrew Morgan 6b22921b19
async/await is_server_admin (#7363) 2020-05-01 15:15:36 +01:00
Patrick Cloke eed7c5b89e
Convert auth handler to async/await (#7261) 2020-04-15 12:40:18 -04:00
Patrick Cloke 468dcc767b
Allow admins to create aliases when they are not in the room (#7191) 2020-04-01 08:27:05 -04:00
Richard van der Hoff 6486c96b65
Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_tests
Add tests for outbound device pokes
2020-03-30 13:59:07 +01:00
Dirk Klimpel 8327eb9280
Add options to prevent users from changing their profile. (#7096) 2020-03-27 19:15:23 +00:00
Richard van der Hoff 28d9d6e8a9 Remove spurious "name" parameter to `default_config`
this is never set to anything other than "test", and is a source of unnecessary
boilerplate.
2020-03-24 18:33:49 +00:00
Richard van der Hoff 6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)"
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
2020-03-17 11:25:01 +00:00
Brendan Abolivier 54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
dklimpel 885134529f updates after review 2020-03-09 22:09:29 +01:00
dklimpel 7e5f40e771 fix tests 2020-03-09 21:00:36 +01:00
dklimpel 04f4b5f6f8 add tests 2020-03-09 19:51:31 +01:00
dklimpel 1f5f3ae8b1 Add options to disable setting profile info for prevent changes. 2020-03-08 14:49:33 +01:00
Patrick Cloke 13892776ef
Allow deleting an alias if the user has sufficient power level (#6986) 2020-03-04 11:30:46 -05:00
Patrick Cloke 7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Richard van der Hoff 3e99528f2b
Store room version on invite (#6983)
When we get an invite over federation, store the room version in the rooms table.

The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
2020-02-26 16:58:33 +00:00
Erik Johnston 1f773eec91
Port PresenceHandler to async/await (#6991) 2020-02-26 15:33:26 +00:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff b58d17e44f Refactor the membership check methods in Auth
these were getting a bit unwieldy, so let's combine `check_joined_room` and
`check_user_was_in_room` into a single `check_user_in_room`.
2020-02-18 23:21:44 +00:00
Patrick Cloke fe3941f6e3
Stop sending events when creating or deleting aliases (#6904)
Stop sending events when creating or deleting associations (room aliases). Send an updated canonical alias event if one of the alt_aliases is deleted.
2020-02-18 07:29:44 -05:00
Patrick Cloke 02e89021f5
Convert the directory handler tests to use HomeserverTestCase (#6919)
Convert directory handler tests to use HomeserverTestCase.
2020-02-14 09:05:43 -05:00
Patrick Cloke 49f877d32e
Filter the results of user directory searching via the spam checker (#6888)
Add a method to the spam checker to filter the user directory results.
2020-02-14 07:17:54 -05:00
Patrick Cloke a92e703ab9
Reject device display names that are too long (#6882)
* Reject device display names that are too long.

Too long is currently defined as 100 characters in length.

* Add a regression test for rejecting a too long device display name.
2020-02-10 16:35:26 -05:00
Erik Johnston b08b0a22d5
Add typing to synapse.federation.sender (#6871) 2020-02-07 13:56:38 +00:00
Richard van der Hoff 928edef979 Pass room_version into `event_from_pdu_json`
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
Richard van der Hoff a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3
Pass room_version into add_hashes_and_signatures
2020-01-31 10:57:03 +00:00
Richard van der Hoff d7bf793cc1 s/get_room_version/get_room_version_id/
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
Richard van der Hoff ef6bdafb29 Store the room version in EventBuilder 2020-01-30 22:15:50 +00:00
Erik Johnston b660327056
Resync remote device list when detected as stale. (#6786) 2020-01-30 17:06:38 +00:00
Erik Johnston 28c98e51ff
Add `local_current_membership` table (#6655)
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Richard van der Hoff 326c893d24
Kill off RegistrationError (#6691)
This is pretty pointless. Let's just use SynapseError.
2020-01-13 12:48:22 +00:00
Erik Johnston 2284eb3a53
Add database config class (#6513)
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
Hubert Chathi cb2db17994
look up cross-signing keys from the DB in bulk (#6486) 2019-12-12 12:03:28 -05:00
Neil Johnson adfdd82b21
Back out perf regression from get_cross_signing_keys_from_cache. (#6494)
Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression.
2019-12-09 13:59:27 +00:00
Erik Johnston a9b393340f
Merge pull request #6484 from matrix-org/erikj/port_sync_handler
Port SyncHandler to async/await
2019-12-09 11:32:44 +00:00
Erik Johnston b3a4e35ca8 Fixup functions to consistently return deferreds 2019-12-06 10:40:05 +00:00
Erik Johnston 8437e2383e Port SyncHandler to async/await 2019-12-05 17:58:25 +00:00
Erik Johnston 4a33a6dd19 Move background update handling out of store 2019-12-05 11:11:26 +00:00
Erik Johnston 756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Erik Johnston ee86abb2d6 Remove underscore from SQLBaseStore functions 2019-12-04 16:23:43 +00:00
Amber Brown 0f87b912ab
Implementation of MSC2314 (#6176) 2019-11-28 08:54:07 +11:00
Hubert Chathi 0d27aba900
add etag and count to key backup endpoints (#5858) 2019-11-27 16:14:44 -05:00
Richard van der Hoff 807ec3bd99
Fix bug which caused rejected events to be stored with the wrong room state (#6320)
Fixes a bug where rejected events were persisted with the wrong state group.

Also fixes an occasional internal-server-error when receiving events over
federation which are rejected and (possibly because they are
backwards-extremities) have no prev_group.

Fixes #6289.
2019-11-06 10:01:39 +00:00
Hubert Chathi bb6cec27a5 rename get_devices_by_remote to get_device_updates_by_remote 2019-10-30 14:57:34 -04:00
Erik Johnston 3f33879be4 Port federation_server to async/await 2019-10-29 14:13:08 +00:00