8f35f8148e
Fix bug where a new writer advances their token too quickly ( #16473 )
...
* Fix bug where a new writer advances their token too quickly
When starting a new writer (for e.g. persisting events), the
`MultiWriterIdGenerator` doesn't have a minimum token for it as there
are no rows matching that new writer in the DB.
This results in the the first stream ID it acquired being announced as
persisted *before* it actually finishes persisting, if another writer
gets and persists a subsequent stream ID. This is due to the logic of
setting the minimum persisted position to the minimum known position of
across all writers, and the new writer starts off not being considered.
* Fix sending out POSITIONs when our token advances without update
Broke in #14820
* For replication HTTP requests, only wait for minimal position
2023-10-23 16:57:30 +01:00
3bc23cc45c
Fix bug that could cause a `/sync` to tightloop with sqlite after restart ( #16540 )
...
This could happen if the last rows in the account data stream were inserted into `account_data`. After a restart the max account ID would be calculated without looking at the `account_data` table, and so have an old ID.
2023-10-23 13:39:25 +00:00
12ca87f5ea
Remove the last reference to event_txn_id. ( #16521 )
...
This table was no longer used, except for a background process
which purged old entries in it.
2023-10-23 07:37:45 -04:00
e9069c9f91
Mark sync as limited if there is a gap in the timeline ( #16485 )
...
This splits thinsg into two queries, but most of the time we won't have
new event backwards extremities so this shouldn't actually add an extra
RTT for the majority of cases.
Note this removes the check for events with no prev events, but that was
part of MSC2716 work that has since been removed.
2023-10-19 15:04:18 +01:00
49c9745b45
Avoid sending massive replication updates when purging a room. ( #16510 )
2023-10-18 12:26:01 -04:00
bcff01b406
Improve performance of delete device messages query ( #16492 )
2023-10-18 16:42:01 +01:00
6ad1f9eac2
Convert DeviceLastConnectionInfo to attrs. ( #16507 )
...
To improve type safety & memory usage.
2023-10-17 12:47:42 +00:00
e3e0ae4ab1
Convert state delta processing from a dict to attrs. ( #16469 )
...
For improved type checking & memory usage.
2023-10-16 07:35:22 -04:00
eee6474bce
Remove useless async job to delete device messages on sync ( #16491 )
2023-10-16 11:06:27 +01:00
4cc729d480
Revert "Drop unused tables & unneeded access token ID for events. ( #16268 )" ( #16465 )
...
This reverts commit cabd577460
2023-10-12 08:56:10 -04:00
cc865fffc0
Convert user_get_threepids response to attrs. ( #16468 )
...
This improves type annotations by not having a dictionary of Any values.
2023-10-11 20:08:11 -04:00
a4904dcb04
Convert simple_select_many_batch, simple_select_many_txn to tuples. ( #16444 )
2023-10-11 13:24:56 -04:00
f1e43018b7
Inline simple_search_list/simple_search_list_txn. ( #16434 )
...
This only has a single use and is over abstracted. Inline it so that
we can improve type hints.
2023-10-10 12:16:36 -04:00
28fd28e92e
Add DB indices to speed up purging rooms ( #16457 )
2023-10-10 10:33:39 +01:00
8902b3031d
Disable statement timeout whilst purging rooms ( #16455 )
...
* Disable statement timeout whilst purging rooms
* Newsfile
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Note the introduction version
---------
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-10-09 15:41:17 +01:00
06bbf1029c
Convert simple_select_list_paginate_txn to return tuples. ( #16433 )
2023-10-06 11:41:57 -04:00
7615e2bf48
Return ThumbnailInfo in more places ( #16438 )
...
Improves type hints by using concrete types instead of
dictionaries.
2023-10-06 10:12:43 -04:00
cabd577460
Drop unused tables & unneeded access token ID for events. ( #16268 )
...
Drop the event_txn_id table and the tables related to MSC2716,
which is no longer supported in Synapse.
2023-10-06 08:29:33 -04:00
3555790b27
Remove unused method. ( #16435 )
2023-10-05 17:42:44 -04:00
fa907025f4
Remove manys calls to cursor_to_dict ( #16431 )
...
This avoids calling cursor_to_dict and then immediately
unpacking the values in the dict for other users. By not
creating the intermediate dictionary we can avoid allocating
the dictionary and strings for the keys, which should generally
be more performant.
Additionally this improves type hints by avoid Dict[str, Any]
dictionaries coming out of the database layer.
2023-10-05 11:07:38 -04:00
009b47badf
Factor out `MultiWriter` token from `RoomStreamToken` ( #16427 )
2023-10-05 10:46:28 +01:00
80ec81dcc5
Some refactors around receipts stream ( #16426 )
2023-10-04 16:28:40 +01:00
1026776380
mypy plugin to check `@cached` return types ( #14911 )
...
Co-authored-by: David Robertson <davidr@element.io>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
Assert that the return type of callables wrapped in @cached
and @cachedList are cachable (aka immutable).
2023-10-02 14:22:36 +00:00
f84da3c32e
Add a cache around server ACL checking ( #16360 )
...
* Pre-compiles the server ACLs onto an object per room and
invalidates them when new events come in.
* Converts the server ACL checking into Rust.
2023-09-26 11:57:50 -04:00
17800a0e97
Implement MSC4028: push all encrypted events. ( #16361 )
...
This unstable push rule is implemented behind an experimental
configuration flag.
2023-09-26 11:52:19 -04:00
2763c49eca
Improve comments in StateGroupBackgroundUpdateStore. ( #16383 )
2023-09-25 18:50:47 +00:00
b225acf3e6
Bump types-psycopg2 from 2.9.21.11 to 2.9.21.14 ( #16381 )
...
* Bump types-psycopg2 from 2.9.21.11 to 2.9.21.14
Bumps [types-psycopg2](https://github.com/python/typeshed ) from 2.9.21.11 to 2.9.21.14.
- [Commits](https://github.com/python/typeshed/commits )
---
updated-dependencies:
- dependency-name: types-psycopg2
dependency-type: direct:development
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Relax the annotation of Cursor.description
See
https://github.com/matrix-org/synapse/pull/16343#issuecomment-1726083384
for rationale.
* Changelog
* Changelog
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David Robertson <davidr@element.io>
2023-09-25 17:48:42 +01:00
12611bfcdd
Add support for pydantic v2 via pydantic.v1 compat module ( #16332 )
...
While maintaining support with pydantic v1.
2023-09-25 15:19:08 +00:00
7ec0a141b4
Convert more cached return values to immutable types ( #16356 )
2023-09-20 07:48:55 -04:00
d7c89c5908
Return immutable objects for cachedList decorators ( #16350 )
2023-09-19 15:26:44 -04:00
eee2b6642d
Bump ruff from 0.0.286 to 0.0.290 ( #16342 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2023-09-18 11:30:43 -04:00
eef2b9e344
Filter locked users in the admin API ( #16328 )
...
Co-authored-by: Hanadi Tamimi <hanadi.tamimi@sdui.de>
2023-09-18 15:37:51 +01:00
c1e244c8f7
Make cached account data/tags/admin types immutable ( #16325 )
2023-09-18 09:55:04 -04:00
85bfd4735e
Return an immutable value from get_latest_event_ids_in_room. ( #16326 )
2023-09-18 09:29:05 -04:00
2a0f86f88f
Convert `_insert_graph_receipts_txn` to `simple_upsert` ( #16299 )
2023-09-15 09:16:45 +01:00
329597022e
Some minor performance fixes for task schedular ( #16313 )
2023-09-14 16:20:47 +01:00
954921736b
Refactor `get_user_by_id` ( #16316 )
2023-09-14 12:46:30 +01:00
7afb5e0410
Fix using dehydrated devices (MSC2697) & refresh tokens ( #16288 )
...
Refresh tokens were not correctly moved to the rehydrated
device (similar to how the access token is currently handled).
This resulted in invalid refresh tokens after rehydration.
2023-09-13 08:33:39 -04:00
d38d0dffc9
Use StrCollection in additional places. ( #16301 )
2023-09-13 07:57:19 -04:00
16ef6f1e3c
Stop purging tables which are slated for removal. ( #16273 )
2023-09-12 07:12:31 -04:00
2b35626b6b
Refactor storing of server keys ( #16261 )
2023-09-12 11:08:04 +01:00
edd83f23b7
Improve type hints for attrs classes ( #16276 )
2023-09-08 19:29:38 +01:00
aa483cb4c9
Update ruff config ( #16283 )
...
Enable additional checks & clean-up unneeded configuration.
2023-09-08 11:24:36 -04:00
c1c6c95d72
Log values at DEBUG level with execute_values ( #16281 )
2023-09-08 14:50:13 +01:00
1cd410a783
Recheck if remote device is cached before requesting it ( #16252 )
...
This fixes a bug where we could get stuck re-requesting the device over
replication again and again.
2023-09-07 12:45:43 +00:00
8940d1b28e
Add `/notifications` endpoint to workers ( #16265 )
2023-09-07 09:26:07 +00:00
32fb264120
Merge remote-tracking branch 'origin/release-v1.92' into develop
2023-09-06 13:08:22 -04:00
55c20da4a3
Merge remote-tracking branch 'origin/release-v1.91' into release-v1.92
2023-09-06 11:25:28 -04:00
1940d990a3
Revert MSC3861 introspection cache, admin impersonation and account lock ( #16258 )
2023-09-06 15:19:51 +01:00
4f1840a88a
Delete device messages asynchronously and in staged batches ( #16240 )
2023-09-06 09:30:53 +02:00
Erik Johnston