Commit Graph

45 Commits

Author SHA1 Message Date
Erik Johnston 9ac263ed1b Add new storage functions to slave store 2017-05-04 14:29:03 +01:00
Erik Johnston e4f3431116 Remove unused cache 2017-04-24 13:27:38 +01:00
Erik Johnston 29574fd5b3 Reduce federation presence replication traffic
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
Erik Johnston 3a1f3f8388 Change slave storage to use new replication interface
As the TCP replication uses a slightly different API and streams than
the HTTP replication.

This breaks HTTP replication.
2017-04-03 15:34:19 +01:00
Erik Johnston d58b1ffe94 Replace some calls to cursor_to_dict
cursor_to_dict can be surprisinglh expensive for large result sets, so lets
only call it when we need to.
2017-03-24 11:07:02 +00:00
Erik Johnston aac6d1fc9b PEP8 2017-03-20 13:47:56 +00:00
Erik Johnston 61f471f779 Don't send the full event json over replication 2017-03-17 15:50:01 +00:00
Erik Johnston 45c7f12d2a Add new storage function to slave store 2017-03-13 16:26:44 +00:00
Erik Johnston 095b45c165 Aggregate event push actions 2017-02-14 13:39:41 +00:00
Erik Johnston 0f3e296cb7 Fix replication 2017-02-02 15:02:03 +00:00
Erik Johnston 458b6f4733 Only invalidate membership caches based on the cache stream
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
Erik Johnston a55fa2047f Insert delta of current_state_events to be more efficient 2017-01-20 17:10:18 +00:00
Erik Johnston 7c9cdb2245 Store federation stream positions in the database 2016-11-21 11:33:08 +00:00
Erik Johnston f8ee66250a Handle sending events and device messages over federation 2016-11-17 15:48:04 +00:00
Erik Johnston 418bcd4309 Add new storage function to slave store 2016-09-16 08:37:39 +01:00
Erik Johnston cb3edec6af Use stream_change cache to make get_forward_extremeties_for_room cache more effective 2016-09-15 14:28:13 +01:00
Erik Johnston 55e6fc917c Add cache to get_forward_extremeties_for_room 2016-09-15 14:04:28 +01:00
Erik Johnston 211786ecd6 Stream public room changes down replication 2016-09-15 11:47:23 +01:00
Erik Johnston a99e933550 Add upgrade script that will slowly prune state_groups_state entries 2016-09-05 10:05:36 +01:00
Erik Johnston 826ca61745 Add storage function to SlaveStore 2016-08-31 14:45:04 +01:00
Erik Johnston 41788bba50 Add to slave store 2016-08-30 09:55:17 +01:00
Erik Johnston bed10f9880 Use state handler instead of get_users_in_room/get_joined_hosts 2016-08-26 14:54:30 +01:00
Erik Johnston 4daa397a00 Add is_host_joined to slave storage 2016-08-26 13:10:56 +01:00
Erik Johnston 50943ab942 Add new state storage funcs to replication 2016-08-26 09:57:32 +01:00
Erik Johnston 5aa024e501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader 2016-07-29 11:24:56 +01:00
Mark Haines 0a7d3cd00f Create separate methods for getting messages to push
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.

The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.

The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
Erik Johnston 0fcbca531f Add get_auth_chain to slave store 2016-07-28 16:36:28 +01:00
Erik Johnston aede7248ab Split out a FederationReader process 2016-07-21 17:37:44 +01:00
David Baker 46b7362304 pep8 2016-06-24 11:44:57 +01:00
David Baker 870c45913e Use similar naming we use in email notifs for push
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
Erik Johnston 05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
Mark Haines 81cf449daa Add methods to events, account data and receipt slaves
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
Mark Haines 0466454b00 Assert that stream replicated stream positions are ints 2016-05-13 17:33:44 +01:00
Mark Haines 1f71f386f6 Merge branch 'develop' into dbkr/email_notifs_on_pusher 2016-05-13 16:59:56 +01:00
Mark Haines 206eb9fd94 Shift some of the state_group methods into the SlavedEventStore 2016-05-13 16:58:14 +01:00
Mark Haines 3547e66bc6 Make sure we advance our stream position 2016-05-13 11:53:00 +01:00
David Baker 5f46be19a7 Pass through get_events to pusher too 2016-05-10 18:43:40 +02:00
David Baker d46b18a00f Pass through _get_event_txn 2016-05-10 18:27:06 +02:00
Mark Haines 8a65666454 Fix backfill replication to advance the stream correctly 2016-04-27 15:38:43 +01:00
Mark Haines c0d8e0eb63 Replicate push actions 2016-04-21 15:25:58 +01:00
Mark Haines e99365f601 Replicate get_invited_rooms_for_user 2016-04-19 15:22:14 +01:00
Mark Haines ceb599e789 Add tests for redactions 2016-04-07 16:52:07 +01:00
Mark Haines 60ec9793fb Add tests for get_latest_event_ids_in_room and get_current_state 2016-04-07 13:17:56 +01:00
Mark Haines 6bfec56796 Test that room membership is replicated 2016-04-06 16:20:13 +01:00
Mark Haines 75fb9ac1be Add a slaved events store class
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00