4548d1f87e
Remove unnecessary parentheses around return statements ( #5931 )
...
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
e3d3fbf63f
Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys
2019-08-28 17:36:46 -07:00
748aa38378
Remove logging for #5407 and update comments
2019-08-15 12:02:18 +01:00
6881f21f3e
Handle TimelineBatch being limited and empty.
...
This hopefully addresses #5407 by gracefully handling an empty but
limited TimelineBatch. We also add some logging to figure out how this
is happening.
2019-08-06 12:59:00 +01:00
336c546d6a
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-08-01 16:31:40 -04:00
c659b9f94f
allow uploading keys for cross-signing
2019-07-25 11:08:24 -04:00
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
729f5a4fb6
Review comments
2019-06-27 16:06:23 +01:00
8624db3194
Refactor and comment sync device list code
2019-06-26 19:39:49 +01:00
806a06daf2
Rename get_users_whose_devices_changed
2019-06-26 19:39:19 +01:00
a2f6d31a63
Refactor get_user_ids_changed to pull less from DB
...
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.
This PR changes this to only check for changes for users the client is
actually interested in.
Fixes #5553
2019-06-26 12:03:44 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
8f06344e11
Merge pull request #5089 from dnaf/m-heroes-empty-room-name
...
Make /sync return heroes if room name or canonical alias are empty
2019-06-06 11:18:13 +01:00
64fa928792
Simplify condition
2019-06-06 10:34:12 +01:00
0a2f522644
Simplify condition
2019-06-05 14:02:29 +01:00
40596aec0e
Merge branch 'develop' into m-heroes-empty-room-name
2019-06-05 13:38:01 +01:00
dbbaf25dd3
Do user_id != me checks before deciding whether we should pick heroes from the joined members or the parted ones
2019-06-05 11:50:27 +01:00
4a6d5de98c
Make /sync attempt to return device updates for both joined and invited users ( #3484 )
2019-05-16 13:23:43 +01:00
7e07dc429f
Lint
...
I probably should've just run autopep8 in the first place...
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:43:18 -04:00
b3e5db402d
Clean up code
...
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:55 -04:00
5d3ed79944
Show heroes if room name or canonical alias are empty
...
Fixes #4194
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:53 -04:00
7f025eb425
Show heroes if room name or canonical alias are empty
...
Fixes #4194
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:29 -04:00
4a4d5c4fd6
Fix grammar and document get_current_users_in_room ( #4998 )
2019-04-03 14:32:20 +01:00
0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
...
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
8b7790e68f
Port #4422 debug logging from hotfixes
2019-03-06 17:29:15 +00:00
6d13bdec91
Add docstrings from matrix-org-hotfixes
2019-03-06 17:24:26 +00:00
07dfe148de
Add some debug for membership syncing issues ( #4538 )
...
I can't figure out what's going on with #4422 and #4436 ; perhaps this will help.
2019-01-31 18:30:40 +00:00
ed8c5e4cda
Fix remote invite rejections not comming down sync
...
This was broken in PR #4405 , commit 886e5ac
2019-01-30 15:46:27 +00:00
81b513416e
Fixed line length
2018-12-19 14:12:33 -05:00
668e6625b0
Fix the variable names used for account_data
2018-12-19 06:53:02 -05:00
cb53ce9d64
Refactor state group lookup to reduce DB hits ( #4011 )
...
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
7e561b5c1a
Add description to counter metric
2018-10-10 11:41:15 +01:00
3cbe8331e6
Track number of non-empty sync responses instead
2018-10-10 11:23:17 +01:00
395276b405
Append _total to metric and fix up spelling
2018-10-10 09:24:39 +01:00
bdc27d6716
Add metric to count lazy member sync requests
2018-10-09 14:17:52 +01:00
82f922b4af
Fix lazy loaded sync with rejected state events
...
In particular, we assume that the name and canonical alias events in
the state have not been rejected. In practice this may not be the case
(though we should probably think about fixing that) so lets ensure that
we gracefully handle that case, rather than 404'ing the sync request
like we do now.
2018-10-01 14:19:40 +01:00
787d22ed6c
Only lazy load self-members on initial sync
...
Given we have disabled lazy loading for incr syncs in #3840 , we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
2018-09-25 00:49:26 +01:00
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients ( #3916 )
...
Should fix https://github.com/vector-im/riot-web/issues/7209
2018-09-20 21:21:54 +01:00
2ac1abbc7e
show heroes if a room has a 'deleted' name/canonical_alias ( #3851 )
2018-09-12 17:11:05 +01:00
0403cf0783
argh pep8
2018-09-12 16:54:28 +01:00
0e200e366d
correctly log gappy sync metrics
2018-09-12 16:47:20 +01:00
11bfc2af1c
fix logline
2018-09-12 16:45:42 +01:00
b041115415
Speed up lazy loading ( #3827 )
...
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats (#3840 )
2018-09-12 00:50:39 +01:00
f60c9e2a01
Don't send empty tags list down sync
2018-09-06 17:01:41 +01:00
7baf66ef5d
Send existing room tags down sync on join
...
When a user joined a room any existing tags were not sent down the sync
stream. Ordinarily this isn't a problem because the user needs to be in
the room to have set tags in it, however synapse will sometimes add tags
for a user to a room, e.g. for server notices, which need to come down
sync.
2018-09-06 16:46:51 +01:00
ee7c8bd2b5
Merge pull request #3795 from matrix-org/erikj/faster_sync_state
...
User iter* during sync state calculations
2018-09-07 00:24:43 +10:00
2608ebc04c
Port handlers/ to Python 3 ( #3803 )
2018-09-07 00:22:23 +10:00
7419764351
User iter* during sync state calculations
2018-09-05 16:19:50 +01:00
b1580f50fe
don't return non-LL-member state in incremental sync state blocks ( #3760 )
...
don't return non-LL-member state in incremental sync state blocks
2018-08-28 23:25:58 +01:00
c334ca67bb
Integrate presence from hotfixes ( #3694 )
2018-08-18 01:08:45 +10:00
3f543dc021
initial cut at a room summary API ( #3574 )
2018-08-16 09:46:50 +01:00
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block
...
Block ability to read via sync if mau limit exceeded
2018-08-14 15:21:31 +00:00
b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
09cf130898
only block on sync where user is not part of the mau cohort
2018-08-09 17:39:12 +01:00
69ce057ea6
block sync if auth checks fail
2018-08-09 12:26:27 +01:00
e92fb00f32
sync auth blocking
2018-08-08 17:54:49 +01:00
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
1a01a5b964
clarify comment on p_ids
2018-07-24 14:03:15 +01:00
eb1d911ab7
rather than adding ll_ids, remove them from p_ids
2018-07-24 13:40:49 +01:00
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
254fb430d1
incorporate review
2018-07-23 19:21:20 +01:00
bcaec2915a
incorporate review
2018-07-19 19:03:50 +01:00
924eb34d94
add a filtered_types param to limit filtering to specific types
2018-07-19 18:32:02 +01:00
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
6eb861b67f
typo
2018-06-25 17:37:16 +01:00
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
5f6122fe10
more comments
2018-06-04 00:08:52 +03:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
a6c8f7c875
add pydoc
2018-05-29 01:09:55 +01:00
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
d26bec8a43
Add comment to sync as to why code path is split
2018-05-09 15:56:07 +01:00
e2accd7f1d
Refactor sync APIs to reuse pagination API
...
The sync API often returns events in a topological rather than stream
ordering, e.g. when the user joined the room or on initial sync. When
this happens we can reuse existing pagination storage functions.
2018-05-09 13:43:39 +01:00
c4af4c24ca
Refactor get_recent_events_for_room return type
...
There is no reason to return a tuple of tokens when the last token is
always the token passed as an argument. Changing it makes it consistent
with other storage APIs
2018-05-09 11:55:34 +01:00
8dc4a6144b
Merge pull request #3107 from NotAFile/py3-bool-nonzero
...
add __bool__ alias to __nonzero__ methods
2018-04-20 15:43:39 +01:00
f63ff73c7f
add __bool__ alias to __nonzero__ methods
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
b78395b7fe
Refactor ResponseCache usage
...
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.
This will be largely non-functional, but does include the following functional
changes:
* federation_server.on_context_state_request: drops use of _server_linearizer
which looked redundant and could cause incorrect cache misses by yielding
between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
on production.
2018-04-12 13:02:15 +01:00
b3384232a0
Add metrics for ResponseCache
2018-04-10 23:14:47 +01:00
b2f2282947
make lazy_load_members configurable in filters
2018-03-19 01:15:13 +00:00
478af0f720
reshuffle todo & comments
2018-03-19 01:00:12 +00:00
366f730bf6
only get member state IDs for incremental syncs if we're filtering
2018-03-18 21:40:43 +00:00
fc5397fdf5
remove debug
2018-03-16 01:44:55 +00:00
4f0493c850
fix tsm search again
2018-03-16 01:43:37 +00:00
f7dcc404f2
add state_ids for timeline entries
2018-03-16 01:37:53 +00:00
5b3b3aada8
simplify timeline_start_members
2018-03-16 01:17:34 +00:00
3bc5bd2d22
make incr syncs work
2018-03-16 00:52:04 +00:00
4d0cfef6ee
add copyright to nudge CI
2018-03-14 00:02:20 +00:00
c9d72e4571
oops
2018-03-13 23:46:45 +00:00
ccca02846d
make it work
2018-03-13 22:31:41 +00:00
14a9d2f73d
ensure we always include the members for a given timeline block
2018-03-13 22:03:42 +00:00
8713365265
typos
2018-03-11 20:10:25 +00:00
9b334b3f97
WIP experiment in lazyloading room members
2018-03-11 20:01:41 +00:00
a56d54dcb7
Fix up log message
2018-03-07 11:55:31 +00:00
02a1296ad6
Fix typo
2018-03-07 11:55:31 +00:00
8cb44da4aa
Fix race in sync when joining room
...
The race happens when the user joins a room at the same time as doing a
sync. We fetch the current token and then get the rooms the user is in.
If the join happens after the current token, but before we get the rooms
we end up sending down a partial room entry in the sync.
This is fixed by looking at the stream ordering of the membership
returned by get_rooms_for_user, and handling the case when that stream
ordering is after the current token.
2018-03-07 11:55:31 +00:00
d4ffe61d4f
Remove ability for AS users to call /events and /sync
...
This functionality has been deprecated for a while as well as being
broken for a while. Instead of fixing it lets just remove it entirely.
See: https://github.com/matrix-org/matrix-doc/issues/1144
2018-03-05 15:44:46 +00:00
2a7e9faeec
Do logcontexts outside ResponseCache
2017-10-25 15:21:08 +01:00
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
e3edca3b5d
Refactor to speed up incremental syncs
2017-09-25 17:35:39 +01:00
3166ed55b2
Fix device list when rejoining room ( #2461 )
2017-09-22 14:44:17 +01:00
93e504d04e
Ensure that creator of group sees group down /sync
2017-09-19 11:08:16 +01:00
b5feaa5a49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged
2017-09-19 11:07:45 +01:00
2d1b7955ae
Don't filter out current state events from timeline
2017-09-18 17:13:03 +01:00
3a0cee28d6
Actually hook leave notifs up
2017-09-14 11:49:37 +01:00
473700f016
Get left rooms
2017-09-13 15:13:41 +01:00
9ce866ed4f
In sync handle device lists for newly joined/left rooms
2017-09-12 16:44:26 +01:00
69ef4987a6
Add left section to /keys/changes
2017-09-08 14:44:36 +01:00
53cc8ad35a
Send down device list change notif when member leaves/rejoins room
2017-09-07 15:08:39 +01:00
d5e32c843f
Correctly add joins to correct segment
2017-07-24 13:31:26 +01:00
139fe30f47
Remember to cast to bool
2017-07-20 16:47:35 +01:00
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
e9aec001f4
Use less DB for device list handling in sync
2017-07-12 10:30:10 +01:00
b4f59c7e27
Add count of one time keys to sync stream
2017-05-19 15:47:55 +01:00
da146657c9
Comments
2017-03-16 13:04:07 +00:00
a158c36a8a
Comment
2017-03-16 11:57:45 +00:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
2ccf3b241c
Implement no op for room stream in sync
2017-03-16 11:06:41 +00:00
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
64d62e41b8
Noop repated delete device inbox calls from /sync
2017-03-10 10:36:43 +00:00
a597994fb6
Measure new device list stuff
2017-02-02 18:36:17 +00:00
6d6591880e
Wake sync up for device changes
2017-02-01 15:15:16 +00:00
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
edc6a1e4f9
Add some logging for syncing to_device events
...
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
2016-12-15 18:16:10 +00:00
be14c24cea
Fix rare notifier bug where listeners dont timeout
...
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00
83bcdcee61
Return early on /sync code paths if a '*' filter is used
...
This is currently very conservative in that it only does this if there is no
`since` token. This limits the risk to clients likely to be doing one-off
syncs (like bridges), but does mean that normal human clients won't benefit
from the time savings here. If the savings are large enough, I would consider
generalising this to just check the filter.
2016-11-22 16:38:35 +00:00
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
65fa37ac5e
Only return new device messages in /sync
2016-09-02 15:50:37 +01:00
c8cd87b21b
Comment about message deletion
2016-08-30 11:23:26 +01:00
1ee6285905
Fix check
2016-08-30 11:17:46 +01:00
68e1a872fd
Noop get_new_messages_for_device if token hasn't changed
2016-08-30 10:58:46 +01:00
55fc17cf4b
Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
...
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
778fa85f47
Make sync not pull out full state
2016-08-25 18:59:44 +01:00
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
afbf6b33fc
defer.returnValue must not be called within Measure
2016-08-19 18:23:44 +01:00
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
661a540dd1
Deduplicate presence entries in sync ( #818 )
2016-06-02 15:20:28 +01:00
faad233ea6
Change short circuit path
2016-05-24 14:27:19 +01:00
6900303997
Don't send down all ephemeral events
2016-05-24 11:44:55 +01:00
1c5ed2a19b
Only work out newly_joined_users for incremental sync
2016-05-24 11:21:34 +01:00
b08ad0389e
Only include non-offline presence in initial sync
2016-05-24 11:15:05 +01:00
be2c677386
Spell builder correctly
2016-05-24 10:53:03 +01:00
79bea8ab9a
Inline function. Make load_filtered_recents private
2016-05-24 10:22:24 +01:00
84f94e4cbb
Add comments
2016-05-24 10:14:53 +01:00
Andrew Morgan