4cff617df1
Move catchup of replication streams to worker. ( #7024 )
...
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
2020-03-25 14:54:01 +00:00
6e6476ef07
Comments from review
2020-03-18 10:13:55 +00:00
9ce4e344a8
Change device list replication to match new semantics.
...
Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).
2020-02-28 11:25:34 +00:00
c3c6c0e622
Add 'device_lists_outbound_pokes' as extra table.
...
This makes sure we check all the relevant tables to get the current max
stream ID.
Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.
2020-02-28 11:15:11 +00:00
bbf8886a05
Merge worker apps into one. ( #6964 )
2020-02-25 16:56:55 +00:00
de2d267375
Allow moving group read APIs to workers ( #6866 )
2020-02-07 11:14:19 +00:00
e17a110661
Detect unknown remote devices and mark cache as stale ( #6776 )
...
We just mark the fact that the cache may be stale in the database for
now.
2020-01-28 14:43:21 +00:00
5d7a6ad223
Allow streaming cache invalidate all to workers. ( #6749 )
2020-01-22 10:37:00 +00:00
28c98e51ff
Add `local_current_membership` table ( #6655 )
...
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
e8b68a4e4b
Fixup synapse.replication to pass mypy checks ( #6667 )
2020-01-14 14:08:06 +00:00
9a4fb457cf
Change DataStores to accept 'database' param.
2019-12-06 13:30:06 +00:00
a7f20500ff
_CURRENT_STATE_CACHE_NAME is public
2019-12-04 15:45:42 +00:00
1056d6885a
Move cache invalidation to main data store
2019-12-04 15:21:14 +00:00
cc6243b4c0
document the REPLICATE command a bit better ( #6305 )
...
since I found myself wonder how it works
2019-11-04 12:40:18 +00:00
f7e4a582ef
clean up code a bit
2019-10-31 12:01:00 -04:00
998f7fe7d4
make user signatures a separate stream
2019-10-30 17:22:52 -04:00
8ac766c44a
make notification of signatures work with workers
2019-10-24 22:14:58 -04:00
bb6264be0b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 10:41:18 +01:00
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
a22d58c96c
add user signature stream change cache to slaved device store
2019-09-04 19:32:35 -04:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
5dbff34509
Fixup bsaed on review comments
2019-05-17 15:48:04 +01:00
d46aab3fa8
Add basic editing support
2019-05-16 16:54:45 +01:00
b5c62c6b26
Fix relations in worker mode
2019-05-16 10:38:13 +01:00
f50efcb65d
Replace SlavedKeyStore with a shim
...
since we're pulling everything out of KeyStore anyway, we may as well simplify
it.
2019-04-08 23:59:07 +01:00
3352baac4b
Remove unused server_tls_certificates functions ( #5028 )
...
These have been unused since #4120 , and with the demise of perspectives, it is
unlikely that they will ever be used again.
2019-04-08 21:50:18 +01:00
e8419554ff
Remove presence lists ( #4989 )
...
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
297bf2547e
Fix sync bug when accepting invites ( #4956 )
...
Hopefully this time we really will fix #4422 .
We need to make sure that the cache on
`get_rooms_for_user_with_stream_ordering` is invalidated *before* the
SyncHandler is notified for the new events, and we can now do so reliably via
the `events` stream.
2019-04-02 12:42:39 +01:00
1f6d6f918a
Make EventStream rows have a type
...
... as a precursor to combining it with the CurrentStateDelta stream.
2019-03-27 22:07:05 +00:00
face0c5b3c
Prefill client IPs cache on workers
2019-03-06 17:39:32 +00:00
a84b8d56c2
Fixup slave stores
2019-03-04 18:04:57 +00:00
1e315017d3
When presence is enabled don't send over replication
2019-02-27 13:53:46 +00:00
80467bbac3
Fix state cache invalidation on workers
2019-02-22 14:38:14 +00:00
a9b5ea6fc1
Batch cache invalidation over replication
...
Currently whenever the current state changes in a room invalidate a lot
of caches, which cause *a lot* of traffic over replication. Instead,
lets batch up all those invalidations and send a single poke down
the replication streams.
Hopefully this will reduce load on the master process by substantially
reducing traffic.
2019-02-18 17:53:31 +00:00
381d2cfdf0
Make workers work on Py3 ( #4027 )
2018-10-13 00:14:08 +11:00
7c27c4d51c
merge ( #3576 )
2018-09-14 03:11:11 +10:00
773db62a22
Rename slave TransactionStore to SlaveTransactionStore
2018-08-15 14:17:06 +01:00
5785b93711
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-09 10:16:16 +01:00
1e2bed9656
Import all functions from TransactionStore
2018-08-06 15:23:38 +01:00
0ca459ea33
Basic support for room versioning
...
This is the first tranche of support for room versioning. It includes:
* setting the default room version in the config file
* new room_version param on the createRoom API
* storing the version of newly-created rooms in the m.room.create event
* fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
05f5dabc10
Use stream cache in get_linearized_receipts_for_room
...
This avoids us from uncessarily hitting the database when there has been
no change for the room
2018-07-10 17:22:42 +01:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
eb50c44eaf
Add UserErasureWorkerStore to workers
2018-06-25 14:22:24 +01:00
3518d0ea8f
Split up ProfileStore
2018-03-13 17:36:50 +00:00
2e223163ff
Split Directory store
2018-03-05 15:11:30 +00:00
fafa3e7114
Split registration store
2018-03-02 13:48:27 +00:00
1a6c7cdf54
Merge pull request #2928 from matrix-org/erikj/read_marker_caches
...
Fix typo in getting replication account data processing
2018-03-01 17:56:14 +00:00
89b7232ff8
Fix typo in getting replication account data processing
2018-03-01 17:50:30 +00:00
1773df0632
Merge pull request #2925 from matrix-org/erikj/split_sig_fed
...
Split out SignatureStore and EventFederationStore
2018-03-01 17:32:58 +00:00
65cf454fd1
Remove unused DataStore
2018-03-01 17:27:53 +00:00
Erik Johnston