Commit Graph

52 Commits

Author SHA1 Message Date
David Robertson 80d44060c9
Faster joins: omit partial rooms from eager syncs until the resync completes (#14870)
* Allow `AbstractSet` in `StrCollection`

Or else frozensets are excluded. This will be useful in an upcoming
commit where I plan to change a function that accepts `List[str]` to
accept `StrCollection` instead.

* `rooms_to_exclude` -> `rooms_to_exclude_globally`

I am about to make use of this exclusion mechanism to exclude rooms for
a specific user and a specific sync. This rename helps to clarify the
distinction between the global config and the rooms to exclude for a
specific sync.

* Better function names for internal sync methods

* Track a list of excluded rooms on SyncResultBuilder

I plan to feed a list of partially stated rooms for this sync to ignore

* Exclude partial state rooms during eager sync

using the mechanism established in the previous commit

* Track un-partial-state stream in sync tokens

So that we can work out which rooms have become fully-stated during a
given sync period.

* Fix mutation of `@cached` return value

This was fouling up a complement test added alongside this PR.
Excluding a room would mean the set of forgotten rooms in the cache
would be extended. This means that room could be erroneously considered
forgotten in the future.

Introduced in #12310, Synapse 1.57.0. I don't think this had any
user-visible side effects (until now).

* SyncResultBuilder: track rooms to force as newly joined

Similar plan as before. We've omitted rooms from certain sync responses;
now we establish the mechanism to reintroduce them into future syncs.

* Read new field, to present rooms as newly joined

* Force un-partial-stated rooms to be newly-joined

for eager incremental syncs only, provided they're still fully stated

* Notify user stream listeners to wake up long polling syncs

* Changelog

* Typo fix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Unnecessary list cast

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Rephrase comment

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Another comment

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Fixup merge(?)

* Poke notifier when receiving un-partial-stated msg over replication

* Fixup merge whoops

Thanks MV :)

Co-authored-by: Mathieu Velen <mathieuv@matrix.org>

Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-01-23 15:44:39 +00:00
Benjamin Kampmann f6c74d1cb2
Implement message forward pagination from start when no from is given, fixes #12383 (#14149)
Fixes https://github.com/matrix-org/synapse/issues/12383
2022-11-24 09:10:51 +00:00
David Robertson 2cc592584a
Remove unused type-ignores (#14433)
* Remove unused type-ignores

Oversights in #14427 and #14429.

* Changelog
2022-11-14 13:46:29 +00:00
Eric Eastwood 92d21faf12
Instrument `/messages` for understandable traces in Jaeger (#13368)
In Jaeger:

 - Before: huge list of uncategorized database calls
 - After: nice and collapsible into units of work
2022-08-03 10:57:38 -05:00
Patrick Cloke e541bb9eed
Rework stream token to stop caring about groups. (#12897) 2022-05-31 07:42:50 -04:00
Eric Eastwood 793d03e2c5
Generate historic pagination token for `/messages` when no `?from` token provided (#12370) 2022-04-06 11:40:28 +01:00
Richard van der Hoff e24ff8ebe3
Remove `HomeServer.get_datastore()` (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
Sean Quah e6acd3cf4f
Upgrade mypy to version 0.931 (#12030)
Upgrade mypy to 0.931, mypy-zope to 0.3.5 and fix new complaints.
2022-02-18 15:57:26 +00:00
Patrick Cloke 4054dfa409
Add type hints for event streams. (#10856) 2021-09-21 13:34:26 -04:00
Jonathan de Jong bf72d10dbf
Use inline type hints in various other places (in `synapse/`) (#10380) 2021-07-15 11:02:43 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Erik Johnston c9c544cda5
Remove `ChainedIdGenerator`. (#8123)
It's just a thin wrapper around two ID gens to make `get_current_token`
and `get_next` return tuples. This can easily be replaced by calling the
appropriate methods on the underlying ID gens directly.
2020-08-19 13:41:51 +01:00
Patrick Cloke e19de43eb5
Convert streams to async. (#8014) 2020-08-04 07:21:47 -04:00
Patrick Cloke 38e1fac886
Fix some spelling mistakes / typos. (#7811) 2020-07-09 09:52:58 -04:00
Erik Johnston 495005360c Bump version of mypy 2019-12-12 15:21:12 +00:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Erik Johnston dddf20e8e1 Fix /messages on workers when no from param specified.
If no `from` param is specified we calculate and use the "current
token" that inlcuded typing, presence, etc. These are unused during
pagination and are not available on workers, so we simply don't
calculate them.
2019-06-24 10:06:51 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Erik Johnston 4d793626ff Fix bug in generating current token 2017-07-20 16:42:44 +01:00
Erik Johnston c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston 2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Erik Johnston b6800a8ecd Actually use the new function 2016-10-24 13:39:49 +01:00
Erik Johnston d04e2ff3a4 Fix incredubly slow back pagination query
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
2016-10-24 13:35:51 +01:00
Mark Haines e993925279 Add store-and-forward direct-to-device messaging 2016-08-25 17:35:37 +01:00
Mark Haines ddf9e7b302 Hook up the push rules to the notifier 2016-03-03 14:57:45 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Mark Haines 1c960fbb80 s/private_user_data/account_data/ 2015-11-18 15:31:04 +00:00
Mark Haines f40b0ed5e1 Inform the client of new room tags using v1 /events 2015-10-29 15:21:09 +00:00
Mark Haines bb4dddd6c4 Move NullSource out of synapse and into tests since it is only used by the tests 2015-09-22 18:33:34 +01:00
Mark Haines a247729806 synapse/streams/events.py:StreamSource was unused 2015-09-22 18:19:49 +01:00
Erik Johnston ca041d5526 Wire together receipts and the notifer/federation 2015-07-07 15:25:30 +01:00
Erik Johnston ddf7979531 Add receipts_key to StreamToken 2015-07-02 11:45:44 +01:00
Erik Johnston 4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Paul "LeoNerd" Evans d6bcffa929 Construct a source-specific 'SourcePaginationConfig' to pass into get_pagination_rows; meaning each source doesn't have to care about its own name any more 2014-10-29 16:16:01 +00:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans a8e8d1d06c Renamed get_current_token_part to get_current_key 2014-08-29 19:15:23 +01:00
Paul "LeoNerd" Evans eec67a675f Have EventSource's get_new_events_for_user() API work only on keys within that source, not overall eventstream tokens 2014-08-29 19:13:55 +01:00
Paul "LeoNerd" Evans 4bfdec1eb2 Rename 'events_key' to 'room_key' so it matches the name of the event source 2014-08-29 19:00:55 +01:00
Paul "LeoNerd" Evans 6dd50da54e Define a new event stream data source for typing notifications (currently null) 2014-08-29 17:39:33 +01:00
Paul "LeoNerd" Evans 20d0db6cfb Move the *EventSource classes into the handlers they relate to, so it's easier to find the code 2014-08-29 17:10:28 +01:00
Erik Johnston f3f32addca Fix typo in NullSource.get_pagination_rows. Remove unused import. 2014-08-28 10:57:53 +01:00
Paul "LeoNerd" Evans 407c86c013 Define a NullSource useful for unit-testing 2014-08-27 18:30:09 +01:00
Erik Johnston bfe9faad5a Index sources in a nicer fashion. 2014-08-27 15:33:52 +01:00
Erik Johnston 05672a6a8c Convert get_paginat_rows to use PaginationConfig. This allows people to supply directions. 2014-08-27 15:25:27 +01:00
Erik Johnston 77a255c7c3 PEP8 tweaks. 2014-08-27 14:19:39 +01:00
Erik Johnston bd16b93e8f Implement presence event source. Change the way the notifier indexes listeners 2014-08-27 14:03:27 +01:00
Erik Johnston 67c5f89244 Enable presence again. Fix up api to match old api. 2014-08-26 19:40:29 +01:00