e92fb00f32
sync auth blocking
2018-08-08 17:54:49 +01:00
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
1a01a5b964
clarify comment on p_ids
2018-07-24 14:03:15 +01:00
eb1d911ab7
rather than adding ll_ids, remove them from p_ids
2018-07-24 13:40:49 +01:00
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
254fb430d1
incorporate review
2018-07-23 19:21:20 +01:00
bcaec2915a
incorporate review
2018-07-19 19:03:50 +01:00
924eb34d94
add a filtered_types param to limit filtering to specific types
2018-07-19 18:32:02 +01:00
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
6eb861b67f
typo
2018-06-25 17:37:16 +01:00
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
5f6122fe10
more comments
2018-06-04 00:08:52 +03:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
a6c8f7c875
add pydoc
2018-05-29 01:09:55 +01:00
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
d26bec8a43
Add comment to sync as to why code path is split
2018-05-09 15:56:07 +01:00
e2accd7f1d
Refactor sync APIs to reuse pagination API
...
The sync API often returns events in a topological rather than stream
ordering, e.g. when the user joined the room or on initial sync. When
this happens we can reuse existing pagination storage functions.
2018-05-09 13:43:39 +01:00
c4af4c24ca
Refactor get_recent_events_for_room return type
...
There is no reason to return a tuple of tokens when the last token is
always the token passed as an argument. Changing it makes it consistent
with other storage APIs
2018-05-09 11:55:34 +01:00
8dc4a6144b
Merge pull request #3107 from NotAFile/py3-bool-nonzero
...
add __bool__ alias to __nonzero__ methods
2018-04-20 15:43:39 +01:00
f63ff73c7f
add __bool__ alias to __nonzero__ methods
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
b78395b7fe
Refactor ResponseCache usage
...
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.
This will be largely non-functional, but does include the following functional
changes:
* federation_server.on_context_state_request: drops use of _server_linearizer
which looked redundant and could cause incorrect cache misses by yielding
between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
on production.
2018-04-12 13:02:15 +01:00
b3384232a0
Add metrics for ResponseCache
2018-04-10 23:14:47 +01:00
b2f2282947
make lazy_load_members configurable in filters
2018-03-19 01:15:13 +00:00
478af0f720
reshuffle todo & comments
2018-03-19 01:00:12 +00:00
366f730bf6
only get member state IDs for incremental syncs if we're filtering
2018-03-18 21:40:43 +00:00
fc5397fdf5
remove debug
2018-03-16 01:44:55 +00:00
4f0493c850
fix tsm search again
2018-03-16 01:43:37 +00:00
f7dcc404f2
add state_ids for timeline entries
2018-03-16 01:37:53 +00:00
5b3b3aada8
simplify timeline_start_members
2018-03-16 01:17:34 +00:00
3bc5bd2d22
make incr syncs work
2018-03-16 00:52:04 +00:00
4d0cfef6ee
add copyright to nudge CI
2018-03-14 00:02:20 +00:00
c9d72e4571
oops
2018-03-13 23:46:45 +00:00
ccca02846d
make it work
2018-03-13 22:31:41 +00:00
14a9d2f73d
ensure we always include the members for a given timeline block
2018-03-13 22:03:42 +00:00
8713365265
typos
2018-03-11 20:10:25 +00:00
9b334b3f97
WIP experiment in lazyloading room members
2018-03-11 20:01:41 +00:00
a56d54dcb7
Fix up log message
2018-03-07 11:55:31 +00:00
02a1296ad6
Fix typo
2018-03-07 11:55:31 +00:00
8cb44da4aa
Fix race in sync when joining room
...
The race happens when the user joins a room at the same time as doing a
sync. We fetch the current token and then get the rooms the user is in.
If the join happens after the current token, but before we get the rooms
we end up sending down a partial room entry in the sync.
This is fixed by looking at the stream ordering of the membership
returned by get_rooms_for_user, and handling the case when that stream
ordering is after the current token.
2018-03-07 11:55:31 +00:00
d4ffe61d4f
Remove ability for AS users to call /events and /sync
...
This functionality has been deprecated for a while as well as being
broken for a while. Instead of fixing it lets just remove it entirely.
See: https://github.com/matrix-org/matrix-doc/issues/1144
2018-03-05 15:44:46 +00:00
2a7e9faeec
Do logcontexts outside ResponseCache
2017-10-25 15:21:08 +01:00
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
e3edca3b5d
Refactor to speed up incremental syncs
2017-09-25 17:35:39 +01:00
3166ed55b2
Fix device list when rejoining room ( #2461 )
2017-09-22 14:44:17 +01:00
93e504d04e
Ensure that creator of group sees group down /sync
2017-09-19 11:08:16 +01:00
b5feaa5a49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged
2017-09-19 11:07:45 +01:00
2d1b7955ae
Don't filter out current state events from timeline
2017-09-18 17:13:03 +01:00
3a0cee28d6
Actually hook leave notifs up
2017-09-14 11:49:37 +01:00
473700f016
Get left rooms
2017-09-13 15:13:41 +01:00
9ce866ed4f
In sync handle device lists for newly joined/left rooms
2017-09-12 16:44:26 +01:00
69ef4987a6
Add left section to /keys/changes
2017-09-08 14:44:36 +01:00
53cc8ad35a
Send down device list change notif when member leaves/rejoins room
2017-09-07 15:08:39 +01:00
d5e32c843f
Correctly add joins to correct segment
2017-07-24 13:31:26 +01:00
139fe30f47
Remember to cast to bool
2017-07-20 16:47:35 +01:00
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
e9aec001f4
Use less DB for device list handling in sync
2017-07-12 10:30:10 +01:00
b4f59c7e27
Add count of one time keys to sync stream
2017-05-19 15:47:55 +01:00
da146657c9
Comments
2017-03-16 13:04:07 +00:00
a158c36a8a
Comment
2017-03-16 11:57:45 +00:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
2ccf3b241c
Implement no op for room stream in sync
2017-03-16 11:06:41 +00:00
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
64d62e41b8
Noop repated delete device inbox calls from /sync
2017-03-10 10:36:43 +00:00
a597994fb6
Measure new device list stuff
2017-02-02 18:36:17 +00:00
6d6591880e
Wake sync up for device changes
2017-02-01 15:15:16 +00:00
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
edc6a1e4f9
Add some logging for syncing to_device events
...
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
2016-12-15 18:16:10 +00:00
be14c24cea
Fix rare notifier bug where listeners dont timeout
...
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00
83bcdcee61
Return early on /sync code paths if a '*' filter is used
...
This is currently very conservative in that it only does this if there is no
`since` token. This limits the risk to clients likely to be doing one-off
syncs (like bridges), but does mean that normal human clients won't benefit
from the time savings here. If the savings are large enough, I would consider
generalising this to just check the filter.
2016-11-22 16:38:35 +00:00
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
65fa37ac5e
Only return new device messages in /sync
2016-09-02 15:50:37 +01:00
c8cd87b21b
Comment about message deletion
2016-08-30 11:23:26 +01:00
1ee6285905
Fix check
2016-08-30 11:17:46 +01:00
68e1a872fd
Noop get_new_messages_for_device if token hasn't changed
2016-08-30 10:58:46 +01:00
55fc17cf4b
Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
...
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
778fa85f47
Make sync not pull out full state
2016-08-25 18:59:44 +01:00
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
afbf6b33fc
defer.returnValue must not be called within Measure
2016-08-19 18:23:44 +01:00
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
661a540dd1
Deduplicate presence entries in sync ( #818 )
2016-06-02 15:20:28 +01:00
faad233ea6
Change short circuit path
2016-05-24 14:27:19 +01:00
6900303997
Don't send down all ephemeral events
2016-05-24 11:44:55 +01:00
1c5ed2a19b
Only work out newly_joined_users for incremental sync
2016-05-24 11:21:34 +01:00
b08ad0389e
Only include non-offline presence in initial sync
2016-05-24 11:15:05 +01:00
be2c677386
Spell builder correctly
2016-05-24 10:53:03 +01:00
79bea8ab9a
Inline function. Make load_filtered_recents private
2016-05-24 10:22:24 +01:00
84f94e4cbb
Add comments
2016-05-24 10:14:53 +01:00
137e6a4557
Shuffle things room
2016-05-24 09:50:55 +01:00
c0c79ef444
Add back concurrently_execute
2016-05-23 18:21:27 +01:00
b5605dfecc
Refactor SyncHandler
2016-05-23 18:08:18 +01:00
f68eea808a
Move SyncHandler out of the Handlers object
2016-05-16 20:19:26 +01:00
53e171f345
Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler
2016-05-16 20:08:32 +01:00
80cb9becd8
Remove get_joined_rooms_for_user from RoomMemberHandler
2016-05-16 20:06:55 +01:00
Neil Johnson