Commit Graph

1093 Commits

Author SHA1 Message Date
Erik Johnston ba47fea528
Allow multiple workers to write to receipts stream. (#16432)
Fixes #16417
2023-10-25 16:16:19 +01:00
Erik Johnston 3bc23cc45c
Fix bug that could cause a `/sync` to tightloop with sqlite after restart (#16540)
This could happen if the last rows in the account data stream were inserted into `account_data`. After a restart the max account ID would be calculated without looking at the `account_data` table, and so have an old ID.
2023-10-23 13:39:25 +00:00
Patrick Cloke 12ca87f5ea
Remove the last reference to event_txn_id. (#16521)
This table was no longer used, except for a background process
which purged old entries in it.
2023-10-23 07:37:45 -04:00
Erik Johnston e9069c9f91
Mark sync as limited if there is a gap in the timeline (#16485)
This splits thinsg into two queries, but most of the time we won't have
new event backwards extremities so this shouldn't actually add an extra
RTT for the majority of cases.

Note this removes the check for events with no prev events, but that was
part of MSC2716 work that has since been removed.
2023-10-19 15:04:18 +01:00
Patrick Cloke 49c9745b45
Avoid sending massive replication updates when purging a room. (#16510) 2023-10-18 12:26:01 -04:00
Mathieu Velten bcff01b406
Improve performance of delete device messages query (#16492) 2023-10-18 16:42:01 +01:00
Patrick Cloke 6ad1f9eac2
Convert DeviceLastConnectionInfo to attrs. (#16507)
To improve type safety & memory usage.
2023-10-17 12:47:42 +00:00
Patrick Cloke e3e0ae4ab1
Convert state delta processing from a dict to attrs. (#16469)
For improved type checking & memory usage.
2023-10-16 07:35:22 -04:00
Mathieu Velten eee6474bce
Remove useless async job to delete device messages on sync (#16491) 2023-10-16 11:06:27 +01:00
Patrick Cloke cc865fffc0
Convert user_get_threepids response to attrs. (#16468)
This improves type annotations by not having a dictionary of Any values.
2023-10-11 20:08:11 -04:00
Patrick Cloke a4904dcb04
Convert simple_select_many_batch, simple_select_many_txn to tuples. (#16444) 2023-10-11 13:24:56 -04:00
Patrick Cloke f1e43018b7
Inline simple_search_list/simple_search_list_txn. (#16434)
This only has a single use and is over abstracted. Inline it so that
we can improve type hints.
2023-10-10 12:16:36 -04:00
David Robertson 28fd28e92e
Add DB indices to speed up purging rooms (#16457) 2023-10-10 10:33:39 +01:00
reivilibre 8902b3031d
Disable statement timeout whilst purging rooms (#16455)
* Disable statement timeout whilst purging rooms

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Note the introduction version

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-10-09 15:41:17 +01:00
Patrick Cloke 06bbf1029c
Convert simple_select_list_paginate_txn to return tuples. (#16433) 2023-10-06 11:41:57 -04:00
Patrick Cloke 7615e2bf48
Return ThumbnailInfo in more places (#16438)
Improves type hints by using concrete types instead of
dictionaries.
2023-10-06 10:12:43 -04:00
Patrick Cloke 3555790b27
Remove unused method. (#16435) 2023-10-05 17:42:44 -04:00
Patrick Cloke fa907025f4
Remove manys calls to cursor_to_dict (#16431)
This avoids calling cursor_to_dict and then immediately
unpacking the values in the dict for other users. By not
creating the intermediate dictionary we can avoid allocating
the dictionary and strings for the keys, which should generally
be more performant.

Additionally this improves type hints by avoid Dict[str, Any]
dictionaries coming out of the database layer.
2023-10-05 11:07:38 -04:00
Erik Johnston 009b47badf
Factor out `MultiWriter` token from `RoomStreamToken` (#16427) 2023-10-05 10:46:28 +01:00
Erik Johnston 80ec81dcc5
Some refactors around receipts stream (#16426) 2023-10-04 16:28:40 +01:00
David Robertson 1026776380
mypy plugin to check `@cached` return types (#14911)
Co-authored-by: David Robertson <davidr@element.io>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>

Assert that the return type of callables wrapped in @cached
and @cachedList are cachable (aka immutable).
2023-10-02 14:22:36 +00:00
Patrick Cloke 17800a0e97
Implement MSC4028: push all encrypted events. (#16361)
This unstable push rule is implemented behind an experimental
configuration flag.
2023-09-26 11:52:19 -04:00
Patrick Cloke 2763c49eca
Improve comments in StateGroupBackgroundUpdateStore. (#16383) 2023-09-25 18:50:47 +00:00
Patrick Cloke 7ec0a141b4
Convert more cached return values to immutable types (#16356) 2023-09-20 07:48:55 -04:00
Patrick Cloke d7c89c5908
Return immutable objects for cachedList decorators (#16350) 2023-09-19 15:26:44 -04:00
dependabot[bot] eee2b6642d
Bump ruff from 0.0.286 to 0.0.290 (#16342)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2023-09-18 11:30:43 -04:00
Hanadi eef2b9e344
Filter locked users in the admin API (#16328)
Co-authored-by: Hanadi Tamimi <hanadi.tamimi@sdui.de>
2023-09-18 15:37:51 +01:00
Patrick Cloke c1e244c8f7
Make cached account data/tags/admin types immutable (#16325) 2023-09-18 09:55:04 -04:00
Patrick Cloke 85bfd4735e
Return an immutable value from get_latest_event_ids_in_room. (#16326) 2023-09-18 09:29:05 -04:00
Jason Little 2a0f86f88f
Convert `_insert_graph_receipts_txn` to `simple_upsert` (#16299) 2023-09-15 09:16:45 +01:00
Erik Johnston 329597022e
Some minor performance fixes for task schedular (#16313) 2023-09-14 16:20:47 +01:00
Erik Johnston 954921736b
Refactor `get_user_by_id` (#16316) 2023-09-14 12:46:30 +01:00
Hanadi 7afb5e0410
Fix using dehydrated devices (MSC2697) & refresh tokens (#16288)
Refresh tokens were not correctly moved to the rehydrated
device (similar to how the access token is currently handled).
This resulted in invalid refresh tokens after rehydration.
2023-09-13 08:33:39 -04:00
Patrick Cloke d38d0dffc9
Use StrCollection in additional places. (#16301) 2023-09-13 07:57:19 -04:00
Patrick Cloke 16ef6f1e3c
Stop purging tables which are slated for removal. (#16273) 2023-09-12 07:12:31 -04:00
Erik Johnston 2b35626b6b
Refactor storing of server keys (#16261) 2023-09-12 11:08:04 +01:00
Erik Johnston 1cd410a783
Recheck if remote device is cached before requesting it (#16252)
This fixes a bug where we could get stuck re-requesting the device over
replication again and again.
2023-09-07 12:45:43 +00:00
Erik Johnston 8940d1b28e
Add `/notifications` endpoint to workers (#16265) 2023-09-07 09:26:07 +00:00
Patrick Cloke 32fb264120 Merge remote-tracking branch 'origin/release-v1.92' into develop 2023-09-06 13:08:22 -04:00
Patrick Cloke 55c20da4a3 Merge remote-tracking branch 'origin/release-v1.91' into release-v1.92 2023-09-06 11:25:28 -04:00
Quentin Gliech 1940d990a3
Revert MSC3861 introspection cache, admin impersonation and account lock (#16258) 2023-09-06 15:19:51 +01:00
Mathieu Velten 4f1840a88a
Delete device messages asynchronously and in staged batches (#16240) 2023-09-06 09:30:53 +02:00
Will Hunt 1e571cd664
Fix appservices being unable to handle to_device messages for multiple users (#16251) 2023-09-05 15:46:57 -04:00
Erik Johnston c9cec2daed
Fix bug where we kept re-requesting a remote server's key repeatedly. (#16257)
* Correctly handle multiple rows per server/key

* Newsfile
2023-09-05 20:27:41 +01:00
Erik Johnston d35bed8369
Don't wake up destination transaction queue if they're not due for retry. (#16223) 2023-09-04 17:14:09 +01:00
Mathieu Velten dcb2778341
Add last_seen_ts to the admin users API (#16218) 2023-09-04 18:13:28 +02:00
Erik Johnston a2e0d4cd60
Fix rare bug that broke looping calls (#16210)
* Fix rare bug that broke looping calls

We can't interact with the reactor from the main thread via looping
call.

Introduced in v1.90.0 / #15791.

* Newsfile
2023-08-30 14:18:42 +01:00
Patrick Cloke ebd8374fb5
Stop writing to the event_txn_id table (#16175) 2023-08-30 11:10:56 +01:00
dependabot[bot] 001fc7bd19
Bump ruff from 0.0.277 to 0.0.286 (#16198)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-29 09:41:43 -04:00
Patrick Cloke 33fa82a34c
Stabilize support for MSC3958 (suppress notifications from edits). (#16113) 2023-08-23 13:22:34 -04:00