Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (take 2, now with no added deadlocks!) (#16658)
* Add `ALTER TABLE ... REPLICA IDENTITY ...` for individual tables We can't combine them into one file as it makes it likely to hit a deadlock if Synapse is running, as it only takes one other transaction to access two tables in a different order to the schema delta. * Add notes * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Re-introduce REPLICA IDENTITY test --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
This commit is contained in:
parent
0aa4d3b6f7
commit
51e4e35653
|
@ -0,0 +1 @@
|
|||
Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication.
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE applied_module_schemas REPLICA IDENTITY USING INDEX applied_module_schemas_module_name_file_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE applied_schema_deltas REPLICA IDENTITY USING INDEX applied_schema_deltas_version_file_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE background_updates REPLICA IDENTITY USING INDEX background_updates_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE schema_compat_version REPLICA IDENTITY USING INDEX schema_compat_version_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE schema_version REPLICA IDENTITY USING INDEX schema_version_lock_key;
|
|
@ -0,0 +1,13 @@
|
|||
The `10_replica_identity_xxx.sql.postgres` series of schema deltas adds replica identities for tables that do not have one implicitly as a result of having a primary key.
|
||||
|
||||
This is needed to use logical replication with Synapse (at least without `UPDATE` and `DELETE` statements failing!).
|
||||
|
||||
Where possible, we use an existing `UNIQUE` index on `NOT NULL` columns as the replica identity. Otherwise, we have to fall back to using the full row as a replica identity.
|
||||
|
||||
Unfortunately, by running all the `ALTER TABLE` statements in one schema delta per database, it was too likely to hit a deadlock as it would only take
|
||||
one other transaction from a running Synapse worker to access the tables out of order and trigger a deadlock.
|
||||
|
||||
By having each statement in its own delta file, each one is run in its own transaction and only needs to take a very brief (instant) lock on the table but no other tables,
|
||||
so there should be no chance of deadlock.
|
||||
|
||||
Like many schema deltas we already apply to Synapse, it is probably blocked by an ongoing `pg_dump`.
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE account_data REPLICA IDENTITY USING INDEX account_data_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE application_services_txns REPLICA IDENTITY USING INDEX application_services_txns_as_id_txn_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE appservice_room_list REPLICA IDENTITY USING INDEX appservice_room_list_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE appservice_stream_position REPLICA IDENTITY USING INDEX appservice_stream_position_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE batch_events REPLICA IDENTITY USING INDEX chunk_events_event_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE blocked_rooms REPLICA IDENTITY USING INDEX blocked_rooms_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE cache_invalidation_stream_by_instance REPLICA IDENTITY USING INDEX cache_invalidation_stream_by_instance_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE current_state_delta_stream REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE current_state_events REPLICA IDENTITY USING INDEX current_state_events_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE deleted_pushers REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_auth_providers REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_federation_inbox REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_federation_outbox REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_inbox REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_changes_converted_stream_position REPLICA IDENTITY USING INDEX device_lists_changes_converted_stream_position_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_changes_in_room REPLICA IDENTITY USING INDEX device_lists_changes_in_stream_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_outbound_last_success REPLICA IDENTITY USING INDEX device_lists_outbound_last_success_unique_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_outbound_pokes REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_remote_cache REPLICA IDENTITY USING INDEX device_lists_remote_cache_unique_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_remote_extremeties REPLICA IDENTITY USING INDEX device_lists_remote_extremeties_unique_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_remote_resync REPLICA IDENTITY USING INDEX device_lists_remote_resync_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE device_lists_stream REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE devices REPLICA IDENTITY USING INDEX device_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_cross_signing_keys REPLICA IDENTITY USING INDEX e2e_cross_signing_keys_stream_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_cross_signing_signatures REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_device_keys_json REPLICA IDENTITY USING INDEX e2e_device_keys_json_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_fallback_keys_json REPLICA IDENTITY USING INDEX e2e_fallback_keys_json_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_one_time_keys_json REPLICA IDENTITY USING INDEX e2e_one_time_keys_json_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_room_keys REPLICA IDENTITY USING INDEX e2e_room_keys_with_version_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE e2e_room_keys_versions REPLICA IDENTITY USING INDEX e2e_room_keys_versions_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE erased_users REPLICA IDENTITY USING INDEX erased_users_user;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_auth REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_auth_chain_links REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_backward_extremities REPLICA IDENTITY USING INDEX event_backward_extremities_event_id_room_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_edges REPLICA IDENTITY USING INDEX event_edges_event_id_prev_event_id_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_forward_extremities REPLICA IDENTITY USING INDEX event_forward_extremities_event_id_room_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_json REPLICA IDENTITY USING INDEX event_json_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_push_actions REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_push_actions_staging REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_push_summary REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_push_summary_last_receipt_stream_id REPLICA IDENTITY USING INDEX event_push_summary_last_receipt_stream_id_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_push_summary_stream_ordering REPLICA IDENTITY USING INDEX event_push_summary_stream_ordering_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_relations REPLICA IDENTITY USING INDEX event_relations_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_search REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_to_state_groups REPLICA IDENTITY USING INDEX event_to_state_groups_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_txn_id REPLICA IDENTITY USING INDEX event_txn_id_event_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE event_txn_id_device_id REPLICA IDENTITY USING INDEX event_txn_id_device_id_event_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE events REPLICA IDENTITY USING INDEX events_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE federation_inbound_events_staging REPLICA IDENTITY USING INDEX federation_inbound_events_staging_instance_event;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE federation_stream_position REPLICA IDENTITY USING INDEX federation_stream_position_instance;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE ignored_users REPLICA IDENTITY USING INDEX ignored_users_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE insertion_event_edges REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE insertion_event_extremities REPLICA IDENTITY USING INDEX insertion_event_extremities_event_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE insertion_events REPLICA IDENTITY USING INDEX insertion_events_event_id;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE local_current_membership REPLICA IDENTITY USING INDEX local_current_membership_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE local_media_repository REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE local_media_repository_thumbnails REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE local_media_repository_url_cache REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE monthly_active_users REPLICA IDENTITY USING INDEX monthly_active_users_users;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE partial_state_events REPLICA IDENTITY USING INDEX partial_state_events_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE partial_state_rooms_servers REPLICA IDENTITY USING INDEX partial_state_rooms_servers_room_id_server_name_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE presence_stream REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE profiles REPLICA IDENTITY USING INDEX profiles_user_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE push_rules_stream REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE ratelimit_override REPLICA IDENTITY USING INDEX ratelimit_override_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE receipts_graph REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE receipts_linearized REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE received_transactions REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE redactions REPLICA IDENTITY USING INDEX redactions_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE registration_tokens REPLICA IDENTITY USING INDEX registration_tokens_token_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE rejections REPLICA IDENTITY USING INDEX rejections_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE remote_media_cache REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE remote_media_cache_thumbnails REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_account_data REPLICA IDENTITY USING INDEX room_account_data_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_alias_servers REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_aliases REPLICA IDENTITY USING INDEX room_aliases_room_alias_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_depth REPLICA IDENTITY USING INDEX room_depth_room_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_forgetter_stream_pos REPLICA IDENTITY USING INDEX room_forgetter_stream_pos_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_memberships REPLICA IDENTITY USING INDEX room_memberships_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_stats_earliest_token REPLICA IDENTITY USING INDEX room_stats_earliest_token_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_stats_state REPLICA IDENTITY USING INDEX room_stats_state_room;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_tags REPLICA IDENTITY USING INDEX room_tag_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE room_tags_revisions REPLICA IDENTITY USING INDEX room_tag_revisions_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE server_keys_json REPLICA IDENTITY USING INDEX server_keys_json_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE server_signature_keys REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE sessions REPLICA IDENTITY USING INDEX sessions_session_type_session_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE state_events REPLICA IDENTITY USING INDEX state_events_event_id_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE stats_incremental_position REPLICA IDENTITY USING INDEX stats_incremental_position_lock_key;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE stream_ordering_to_exterm REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE stream_positions REPLICA IDENTITY USING INDEX stream_positions_idx;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE threads REPLICA IDENTITY USING INDEX threads_uniqueness;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE threepid_guest_access_tokens REPLICA IDENTITY FULL;
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE timeline_gaps REPLICA IDENTITY FULL;
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue