Mirrors
/
synapse
mirror of https://github.com/element-hq/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Pull less state out if we fail to backfill (#16788) 

Sometimes we fail to fetch events during backfill due to missing state,
and we often end up querying the same bad events periodically (as people
backpaginate). In such cases its likely we will continue to fail to get
the state, and therefore we should try *before* loading the state that
we have from the DB (as otherwise it's wasted DB and memory).

---------

Co-authored-by: reivilibre <oliverw@matrix.org>
This commit is contained in:
Erik Johnston 2024-01-10 14:42:13 +00:00 committed by GitHub
parent 1485cfd0f2
commit 0a96fa52a2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 13 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
changelog.d/16788.misc Normal file
View File

@ -0,0 +1 @@
Pull less state out of the DB when we retry fetching old events during backfill.

21
synapse/handlers/federation_event.py
View File

@ -1141,16 +1141,8 @@ class FederationEventHandler:
partial_state_flags = await self._store.get_partial_state_events(seen) partial_state_flags = await self._store.get_partial_state_events(seen)
partial_state = any(partial_state_flags.values()) partial_state = any(partial_state_flags.values())
# Get the state of the events we know about
ours = await self._state_storage_controller.get_state_groups_ids(
room_id, seen, await_full_state=False
)
# state_maps is a list of mappings from (type, state_key) to event_id # state_maps is a list of mappings from (type, state_key) to event_id
state_maps: List[StateMap[str]] = list(ours.values()) state_maps: List[StateMap[str]] = []
# we don't need this any more, let's delete it.
del ours
# Ask the remote server for the states we don't # Ask the remote server for the states we don't
# know about # know about
@ -1169,6 +1161,17 @@ class FederationEventHandler:
state_maps.append(remote_state_map) state_maps.append(remote_state_map)
# Get the state of the events we know about. We do this *after*
# trying to fetch missing state over federation as that might fail
# and then we can skip loading the local state.
ours = await self._state_storage_controller.get_state_groups_ids(
room_id, seen, await_full_state=False
)
state_maps.extend(ours.values())
# we don't need this any more, let's delete it.
del ours
room_version = await self._store.get_room_version_id(room_id) room_version = await self._store.get_room_version_id(room_id)
state_map = await self._state_resolution_handler.resolve_events_with_store( state_map = await self._state_resolution_handler.resolve_events_with_store(
room_id, room_id,