Commit Graph

242 Commits

Author SHA1 Message Date
Erik Johnston dfeca6cf40 Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
Erik Johnston 6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston 052c5d19d5 Merge pull request #2281 from matrix-org/erikj/phone_home_stats
Fix phone home stats
2017-06-15 12:46:23 +01:00
Erik Johnston 4b461a6931 Add some more stats 2017-06-15 09:39:39 +01:00
Erik Johnston 617304b2cf Fix phone home stats 2017-06-14 19:47:15 +01:00
Erik Johnston d53fe399eb Add cache for is_host_joined 2017-06-13 09:56:18 +01:00
Erik Johnston c62c480dc6 Merge pull request #2259 from matrix-org/erikj/fix_state_woes
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
Erik Johnston 197bd126f0 Fix bug where state_group tables got corrupted
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.

We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
Erik Johnston 4abcff0177 Fix typo 2017-05-31 16:22:36 +01:00
Erik Johnston 304880d185 Add stream change cache 2017-05-31 15:46:36 +01:00
Erik Johnston 04095f7581 Add clobbered event_id 2017-05-30 14:53:01 +01:00
Erik Johnston a584a81b3e Add current_state_delta_stream table 2017-05-30 14:44:09 +01:00
Erik Johnston c049472b8a Only store event_auth for state events 2017-05-24 15:23:31 +01:00
Erik Johnston b8492b6c2f Merge pull request #2224 from matrix-org/erikj/prefill_state
Prefill state caches
2017-05-16 15:50:11 +01:00
Erik Johnston 331570ea6f Remove spurious merge artifacts 2017-05-16 15:33:07 +01:00
Erik Johnston 871605f4e2 Comments 2017-05-15 15:11:30 +01:00
Erik Johnston e0d2f6d5b0 Add more granular event send metrics 2017-05-15 15:11:30 +01:00
Erik Johnston bfbc907cec Prefill state caches 2017-05-15 15:11:13 +01:00
Richard van der Hoff 34194aaff7 Don't create event_search index on sqlite
... because the table is virtual
2017-05-11 12:46:55 +01:00
Richard van der Hoff 114f290947 Add more logging for purging
Log the number of events we will be deleting at info.
2017-05-11 12:08:47 +01:00
Richard van der Hoff baafb85ba4 Add an index to event_search
- to make the purge API quicker
2017-05-11 12:05:22 +01:00
Richard van der Hoff dc026bb16f Tidy purge code and add some comments
Try to make this clearer with more comments and some variable renames
2017-05-11 10:56:12 +01:00
Richard van der Hoff 8e345ce465 Don't de-delta state groups we're about to delete 2017-05-10 18:44:22 +01:00
Richard van der Hoff b64d312421 add some logging to purge_history 2017-05-10 18:44:22 +01:00
Erik Johnston 093f7e47cc Expand docstring a bit 2017-05-08 16:14:46 +01:00
Erik Johnston ffad4fe35b Don't update event cache hit ratio from get_joined_users
Otherwise the hit ration of plain get_events gets completely skewed by
calls to get_joined_users* functions.
2017-05-08 16:06:17 +01:00
Erik Johnston 587f07543f Revert "Prefill state caches" 2017-05-04 15:07:27 +01:00
Erik Johnston 1827057acc Comments 2017-05-03 09:56:05 +01:00
Erik Johnston 8346e6e696 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state 2017-05-03 09:46:40 +01:00
Erik Johnston 3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Erik Johnston a2c89a225c Prefill state caches 2017-05-02 10:40:31 +01:00
Luke Barnard 6a70647d45 Correct logic in is_event_after 2017-04-13 13:46:17 +01:00
Luke Barnard b9557064bf Simplify is_event_after logic 2017-04-12 14:36:20 +01:00
Luke Barnard 7f94709066 travis flake8.. 2017-04-11 18:35:45 +01:00
Luke Barnard 73880268ef Refactor event ordering check to events store 2017-04-11 17:34:09 +01:00
Erik Johnston 24d35ab47b Add new storage functions for new replication
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
2017-03-30 11:48:35 +01:00
Erik Johnston bac9bf1b12 Typo 2017-03-27 18:02:17 +01:00
Erik Johnston d82c42837f Short circuit if all new events have same state group 2017-03-27 18:00:47 +01:00
Erik Johnston e71940aa64 Use iter(items|values) 2017-03-24 10:57:02 +00:00
Erik Johnston 00957d1aa4 User Cursor.__iter__ instead of fetchall
This prevents unnecessary construction of lists
2017-03-23 17:53:49 +00:00
Erik Johnston 37a187bfab Merge pull request #2033 from matrix-org/erikj/repl_speed
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
Erik Johnston 61f471f779 Don't send the full event json over replication 2017-03-17 15:50:01 +00:00
Richard van der Hoff 0c01f829ae Avoid resetting state on rejected events
When we get a rejected event, give it the same state_group as its prev_event,
rather than no state_group at all.

This should fix https://github.com/matrix-org/synapse/issues/1935.
2017-03-17 15:06:08 +00:00
Richard van der Hoff 5068fb16a5 Refactoring and cleanups
A few non-functional changes:

* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
  readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
  mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
  mind-bending conditional.
2017-03-17 15:06:07 +00:00
Erik Johnston bb256ac96f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed 2017-03-14 11:35:05 +00:00
Erik Johnston cc7a294e2e Fix current_state_events table to not lie
If we try and persist two state events that have the same ancestor we
calculate the wrong current state when persisting those events.
2017-03-14 10:57:43 +00:00
Erik Johnston 8ffbe43ba1 Get current state by using current_state_events table 2017-03-10 17:39:35 +00:00
Erik Johnston a41dce8f8a Remove needless check 2017-02-27 18:54:43 +00:00
Erik Johnston c0d6045776 It should be all 2017-02-27 18:45:24 +00:00
Erik Johnston 49f4bc4709 Don't fetch current state in common case
Currently we fetch the list of current state events whenever we send
something in a room. This is overkill for the common case of persisting
a simple chain of non-state events, so lets handle that case specially.
2017-02-27 18:33:41 +00:00