04faa0bfa9
Convert tags and metrics databases to async/await ( #8062 )
2020-08-11 17:21:20 -04:00
a0acdfa9e9
Converts event_federation and registration databases to async/await ( #8061 )
2020-08-11 17:21:13 -04:00
fcbab08cbd
Add an assertion on prev_events in create_new_client_event ( #8041 )
...
I think this would have caught all the cases in
https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes
more sense here than a 403
2020-08-10 12:29:47 +01:00
7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async ( #8042 )
2020-08-07 13:36:29 -04:00
f3fe6961b2
Convert additional database stores to async/await ( #8045 )
2020-08-07 12:17:17 -04:00
7620912d84
Add health check endpoint ( #8048 )
2020-08-07 14:21:24 +01:00
2ffd6783c7
Revert #7736 ( #8039 )
2020-08-06 17:15:35 +01:00
fe6cfc80ec
Convert some util functions to async ( #8035 )
2020-08-06 08:39:35 -04:00
d4a7829b12
Convert synapse.api to async/await ( #8031 )
2020-08-06 08:30:06 -04:00
66f24449dd
Improve performance of the register endpoint ( #8009 )
2020-08-06 08:09:55 -04:00
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
e19de43eb5
Convert streams to async. ( #8014 )
2020-08-04 07:21:47 -04:00
5d92a1428c
Prevent join->join membership transitions changing member count ( #7977 )
...
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.
However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.
This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.
Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).
Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338 .
2020-08-03 21:54:24 +01:00
6812509807
Implement handling of HTTP HEAD requests. ( #7999 )
2020-08-03 08:45:42 -04:00
2a89ce8cd4
Convert the crypto module to async/await. ( #8003 )
2020-08-03 08:29:01 -04:00
faba873d4b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins
2020-07-31 15:07:01 +01:00
18de00adb4
Add ratelimiting on joins
2020-07-31 15:06:56 +01:00
c978f6c451
Convert federation client to async/await. ( #7975 )
2020-07-30 08:01:33 -04:00
4cce8ef74e
Convert appservice to async. ( #7973 )
2020-07-30 07:27:39 -04:00
b3a97d6dac
Convert some of the data store to async. ( #7976 )
2020-07-30 07:20:41 -04:00
8dff4a1242
Re-implement unread counts ( #7736 )
2020-07-29 18:26:55 +01:00
3345c166a4
Convert storage layer to async/await. ( #7963 )
2020-07-28 16:09:53 -04:00
e866e3b896
Add an option to disable purge in delete room admin API ( #7964 )
...
Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete```
Fixes : #3761
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-28 20:08:23 +01:00
3857de2194
Option to allow server admins to join complex rooms ( #7902 )
...
Fixes #7901 .
Signed-off-by: Niklas Tittjung <nik_t.01@web.de>
2020-07-28 13:41:44 +01:00
8553f46498
Convert a synapse.events to async/await. ( #7949 )
2020-07-27 13:40:22 -04:00
5f65e62681
Convert groups and visibility code to async / await. ( #7951 )
2020-07-27 12:32:08 -04:00
8144bc26a7
Convert push to async/await. ( #7948 )
2020-07-27 12:21:34 -04:00
c4268e3da6
Convert tests/rest/admin/test_room.py to unix file endings ( #7953 )
...
Converts tests/rest/admin/test_room.py to have unix file endings after they were accidentally changed in #7613 .
Keeping the same changelog as #7613 as it hasn't gone out in a release yet.
2020-07-27 13:22:52 +01:00
3fc8fdd150
Support oEmbed for media previews. ( #7920 )
...
Fixes previews of Twitter URLs by using their oEmbed endpoint to grab content.
2020-07-27 07:50:44 -04:00
b975fa2e99
Convert state resolution to async/await ( #7942 )
2020-07-24 10:59:51 -04:00
5ea29d7f85
Convert more of the media code to async/await ( #7873 )
2020-07-24 09:39:02 -04:00
6a080ea184
Return an empty body for OPTIONS requests. ( #7886 )
2020-07-24 07:08:07 -04:00
1ec688bf21
Downgrade warning on client disconnect to INFO ( #7928 )
...
Clients disconnecting before we finish processing the request happens from time
to time. We don't need to yell about it
2020-07-24 09:55:47 +01:00
68cd935826
Convert the federation agent and related code to async/await. ( #7874 )
2020-07-23 07:05:57 -04:00
cc9bb3dc3f
Convert the message handler to async/await. ( #7884 )
2020-07-22 12:29:15 -04:00
6b3ac3b8cd
Convert device handler to async/await ( #7871 )
2020-07-17 07:09:25 -04:00
6fca1b3506
Convert _base, profile, and _receipts handlers to async/await ( #7860 )
2020-07-17 07:08:30 -04:00
fff483ea96
Add admin endpoint to get members in a room. ( #7842 )
2020-07-16 16:43:23 -04:00
649a7ead5c
Add ability to run multiple pusher instances ( #7855 )
...
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
97e1159ac1
Merge branch 'erikj/faster_typing' of github.com:matrix-org/synapse into develop
2020-07-15 16:54:30 +01:00
8c7d0f163d
Allow accounts to be re-activated from the admin APIs. ( #7847 )
2020-07-15 11:00:21 -04:00
9006e125af
Fix tests
2020-07-15 15:47:27 +01:00
f13061d515
Fix client reader sharding tests ( #7853 )
...
* Fix client reader sharding tests
* Newsfile
* Fix typing
* Update changelog.d/7853.misc
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Move mocking of http_client to tests
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-07-15 15:27:35 +01:00
b11450dedc
Convert E2E key and room key handlers to async/await. ( #7851 )
2020-07-15 08:48:58 -04:00
111e70d75c
Return the proper 403 Forbidden error during errors with JWT logins. ( #7844 )
2020-07-15 07:10:21 -04:00
491f0dab1b
Add delete room admin endpoint ( #7613 )
...
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md )" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md )" API.
Fixes : #6425
It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105)#5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-14 12:36:23 +01:00
77d2c05410
Add the option to validate the `iss` and `aud` claims for JWT logins. ( #7827 )
2020-07-14 07:16:43 -04:00
457096e6df
Support handling registration requests across multiple client readers. ( #7830 )
2020-07-13 13:31:46 -04:00
66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. ( #7802 )
2020-07-10 14:30:08 -04:00
f299441cc6
Add ability to shard the federation sender ( #7798 )
2020-07-10 18:26:36 +01:00
38e1fac886
Fix some spelling mistakes / typos. ( #7811 )
2020-07-09 09:52:58 -04:00
67593b1728
Add `HomeServer.signing_key` property ( #7805 )
...
... instead of duplicating `config.signing_key[0]` everywhere
2020-07-08 17:51:56 +01:00
76dbd7b8d6
Stop populating unused table `local_invites`. ( #7793 )
...
This table is no longer used, so we may as well stop populating it. Removing it
would prevent people rolling back to older releases of Synapse, so that can
happen in a future release.
2020-07-07 14:20:40 +01:00
6d687ebba1
Convert the appservice handler to async/await. ( #7775 )
2020-07-06 07:40:35 -04:00
57feeab364
Don't ignore `set_tweak` actions with no explicit `value`. ( #7766 )
...
* Fix spec compliance; tweaks without values are valid
(default to True, which is only concretely specified for
`highlight`, but it seems only reasonable to generalise)
* Changelog for 7766.
* Add documentation to `tweaks_for_actions`
May as well tidy up when I'm here.
* Add a test for `tweaks_for_actions`
2020-07-06 11:43:41 +01:00
62b1ce8539
isort 5 compatibility ( #7786 )
...
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
5cdca53aa0
Merge different Resource implementation classes ( #7732 )
2020-07-03 19:02:19 +01:00
21a212f8e5
Fix inconsistent handling of upper and lower cases of email addresses. ( #7021 )
...
fixes #7016
2020-07-03 14:03:13 +01:00
e5808c4cfb
Hack to add push priority to push notifications ( #7765 )
...
* Remove obsolete comment about ancient temporary code
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Implement hack to set push priority
based on whether the tweaks indicate the event might cause
effects.
* Changelog for 7765
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Antilint
* Add tests for push priority
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Update synapse/push/httppusher.py
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Antilint
* Remove needless invites from tests.
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2020-07-01 17:02:31 +01:00
74d3e177f0
Back out MSC2625 implementation ( #7761 )
2020-07-01 11:08:25 +01:00
71cccf1593
Additional configuration options for auto-join rooms ( #7763 )
2020-06-30 15:41:36 -04:00
0e0a2817a2
Yield during large v2 state res. ( #7735 )
...
State res v2 across large data sets can be very CPU intensive, and if
all the relevant events are in the cache the algorithm will run from
start to finish within a single reactor tick. This can result in
blocking the reactor tick for several seconds, which can have major
repercussions on other requests.
To fix this we simply add the occaisonal `sleep(0)` during iterations to
yield execution until the next reactor tick. The aim is to only do this
for large data sets so that we don't impact otherwise quick resolutions.=
2020-06-24 18:48:18 +01:00
6920e58136
add org.matrix.login.jwt so that m.login.jwt can be deprecated ( #7675 )
2020-06-24 10:23:55 +01:00
95e41f368b
Allow local media to be marked as safe from being quarantined. ( #7718 )
2020-06-22 08:04:14 -04:00
3630825612
Convert the typing handler to async/await. ( #7679 )
2020-06-17 10:37:59 -04:00
434716e1d3
Fetch from the r0 media path instead of the unspecced v1. ( #7714 )
2020-06-17 08:36:46 -04:00
46613aaf79
Implement unread counter (MSC2625) ( #7673 )
...
Implementation of https://github.com/matrix-org/matrix-doc/pull/2625
2020-06-17 10:58:32 +01:00
ac51bd581a
Include a user agent in federation requests. ( #7677 )
2020-06-16 10:43:29 -04:00
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents ( #7704 )
2020-06-16 08:51:47 -04:00
03619324fc
Create a ListenerConfig object ( #7681 )
...
This ended up being a bit more invasive than I'd hoped for (not helped by
generic_worker duplicating some of the code from homeserver), but hopefully
it's an improvement.
The idea is that, rather than storing unstructured `dict`s in the config for
the listener configurations, we instead parse it into a structured
`ListenerConfig` object.
2020-06-16 12:44:07 +01:00
789606577a
Merge branch 'master' into develop
2020-06-16 10:36:29 +01:00
3e6b5bba71
Wrap register_device coroutine in an ensureDeferred ( #7684 )
...
Fixes https://github.com/matrix-org/synapse/issues/7683
Broke in: #7649
We had a `yield` acting on a coroutine. To be fair this one is a bit difficult to notice as there's a function in the middle that just passes the coroutine along.
2020-06-16 10:13:59 +01:00
cc32fa7358
Ensure the body is a string before comparing push rules. ( #7701 )
2020-06-15 16:20:34 -04:00
2b2344652b
Ensure etag is a string for GET room_keys/version response ( #7691 )
2020-06-15 13:42:44 -04:00
6efb2b0ad4
Merge branch 'develop' into babolivier/mark_unread
2020-06-15 16:37:52 +01:00
c2b4621630
Merge branch 'develop' into babolivier/mark_unread
2020-06-15 16:37:08 +01:00
7d2532be36
Discard RDATA from already seen positions. ( #7648 )
2020-06-15 08:44:54 -04:00
7e80c84902
Lint
2020-06-12 11:31:11 +01:00
6b1fa3293d
Test that a mark_unread action updates the right counter when using a slave store
2020-06-12 11:28:26 +01:00
63d9a00bf1
Remove debug logging
2020-06-12 11:13:30 +01:00
2a07c5ded6
Test that a mark_unread action updates the right counter
2020-06-12 11:08:05 +01:00
476a89707a
Fix tests
2020-06-10 17:55:03 +01:00
fcd6961441
Add option to enable encryption by default for new rooms ( #7639 )
...
Fixes https://github.com/matrix-org/synapse/issues/2431
Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.
Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637
Based on #7637
2020-06-10 17:44:34 +01:00
737b4a936e
Convert user directory handler and related classes to async/await. ( #7640 )
2020-06-05 14:42:55 -04:00
09099313e6
Add an option to disable autojoin for guest accounts ( #6637 )
...
Fixes https://github.com/matrix-org/synapse/issues/3177
2020-06-05 18:18:15 +01:00
908f9e2d24
Allow new users to be registered via the admin API even if the monthly active user limit has been reached ( #7263 )
2020-06-05 13:08:49 +01:00
2970ce8367
Add device management to admin API ( #7481 )
...
- Admin is able to
- change displaynames
- delete devices
- list devices
- get device informations
Fixes #7330
2020-06-05 13:07:22 +01:00
e55ee7c32f
Add support for webp thumbnailing ( #7586 )
...
Closes #4382
Signed-off-by: Maxim Plotnikov <wgh@torlan.ru>
2020-06-05 11:54:27 +01:00
f4e6495b5d
Performance improvements and refactor of Ratelimiter ( #7595 )
...
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:
* Rather undocumented, and
* causing a *lot* of config checks
This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation.
Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
c389bfb6ea
Fix encryption algorithm typos in tests/comments ( #7637 )
...
@uhoreg has confirmed these were both typos. They are only in comments and tests though, rather than anything critical.
Introduced in:
* https://github.com/matrix-org/synapse/pull/7157
* https://github.com/matrix-org/synapse/pull/5726
2020-06-04 20:03:40 +01:00
0188daf32c
Replace instances of reactor pumping with get_success. ( #7619 )
...
Calls `self.get_success` on all deferred methods instead of abusing `self.pump()`. This has the benefit of working with coroutines, as well as checking that method execution completed successfully.
There are also a few small cleanups that I made in the process.
2020-06-03 16:39:30 +01:00
fe434cd3c9
Fix a bug in automatic user creation with m.login.jwt. ( #7585 )
2020-06-01 12:55:07 -04:00
33c39ab93c
Process cross-signing keys when resyncing device lists ( #7594 )
...
It looks like `user_device_resync` was ignoring cross-signing keys from the results received from the remote server. This patch fixes this, by processing these keys using the same process `_handle_signing_key_updates` does (and effectively factor that part out of that function).
2020-06-01 17:47:30 +02:00
901b1fa561
Email notifications for new users when creating via the Admin API. ( #7267 )
2020-06-01 15:34:33 +01:00
a72d5f39db
Add test for Linearizer.is_queued(..)
2020-05-27 19:41:06 +01:00
4ba55559ac
Fix specifying cache factors via env vars with * in name. ( #7580 )
...
This mostly applise to `*stateGroupCache*` and co.
Broke in #6391 .
2020-05-27 13:17:01 +01:00
eefc6b3a0d
Don't apply cache factor to event cache. ( #7578 )
...
This is already correctly done when we instansiate the cache, but wasn't
when it got reloaded (which always happens at least once on startup).
2020-05-27 12:04:37 +01:00
00db90f409
Fix recording of federation stream token ( #7564 )
...
A couple of changes of significance:
* remove the `_last_ack < federation_position` condition, so that
updates will still be correctly processed after restart
* Correctly wire up send_federation_ack to the right class.
2020-05-26 11:41:38 +01:00
f4269694ce
Optimise some references to hs.config ( #7546 )
...
These are surprisingly expensive, and we only really need to do them at startup.
2020-05-22 21:47:07 +01:00
2901f54359
Fix missing CORS headers on OPTION responses ( #7560 )
...
Broke in #7534 .
2020-05-22 17:42:39 +01:00
Patrick Cloke