Commit Graph

134 Commits

Author SHA1 Message Date
Erik Johnston 2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Erik Johnston b2c7bd4b09 Cache get_recent_events_for_room 2015-08-07 14:42:34 +01:00
Erik Johnston 07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
Erik Johnston a2c4f3f150 Fix daedlock 2015-05-15 10:54:04 +01:00
Erik Johnston f6f902d459 Move fetching of events into their own transactions 2015-05-14 13:45:48 +01:00
Erik Johnston 4071f29653 Fetch events from events_id in their own transactions 2015-05-13 16:59:41 +01:00
Erik Johnston 4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Erik Johnston 17653a5dfe Move storage.stream._StreamToken to types.RoomStreamToken 2015-05-11 18:01:01 +01:00
Erik Johnston 6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston 2ded344620 Remove unused import 2015-04-09 13:46:06 +01:00
Erik Johnston 8ad0f4912e
Stream ordering and out of order insertions.
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston 9517f4da4d Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement 2015-03-20 16:02:47 +00:00
Paul "LeoNerd" Evans ceb61daa70 Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream 2015-03-20 15:44:06 +00:00
Erik Johnston 87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Kegan Dougal 377ae369c1 Wrap all of get_app_service_rooms in a txn. 2015-03-02 11:20:51 +00:00
Kegan Dougal b216b36892 JOIN state_events rather than parsing unrecognized_keys to pull out member state_keys 2015-03-02 10:41:35 +00:00
Kegan Dougal ebc4830666 PR tweaks: set earlier on and use 'as json' for compat 2015-03-02 09:53:00 +00:00
Kegan Dougal 806a6c886a PEP8 2015-02-27 09:48:57 +00:00
Kegan Dougal f0995436e7 Check for membership invite events correctly. 2015-02-26 17:21:17 +00:00
Kegan Dougal dcec7175dc Finish impl to get new events for AS. ASes should now be able to poll /events 2015-02-26 16:23:01 +00:00
Kegan Dougal 978ce87c86 Comment unused variables. 2015-02-25 17:37:48 +00:00
Kegan Dougal 2d20466f9a Add stub functions and work out execution flow to implement AS event stream polling. 2015-02-25 15:00:59 +00:00
Mark Haines 8498d348d8 Fix token formatting 2015-01-30 11:42:09 +00:00
Mark Haines 22dd1cde2d Filter the recent events before applying the limit when doing an incremental sync with a gap 2015-01-30 11:32:35 +00:00
Mark Haines 396a67a09a Merge branch 'client_v2_filter' into client_v2_sync
Conflicts:
	synapse/rest/client/v2_alpha/__init__.py
2015-01-29 14:58:00 +00:00
Mark Haines e016f4043b Use get_room_events_stream to get changes to the rooms if the number of changes is small 2015-01-29 14:40:28 +00:00
Mark Haines b0b80074e0 SYN-252: Supply the stream and topological parts in the correct order to the constructor 2015-01-29 01:48:48 +00:00
Mark Haines e020574d65 Fix Formatting 2015-01-27 20:19:36 +00:00
Mark Haines a56008842b Start implementing incremental initial sync 2015-01-27 16:24:22 +00:00
Mark Haines fda63064fc get_room_events isn't called anywhere 2015-01-13 14:43:26 +00:00
Mark Haines 895fcb377e Fix stream token ordering 2015-01-13 14:38:53 +00:00
Kegan Dougal c43d898119 SYN-178: Fix off by one. 2015-01-12 17:38:40 +00:00
Erik Johnston fd9a8db7ea Only fetch the columns we need. 2015-01-06 15:59:31 +00:00
Mark Haines 5e23a19204 Merge pull request #28 from matrix-org/erikj-perf
Database performance improvements.
2015-01-06 13:33:40 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Erik Johnston af1c7c7808 PEP8 2015-01-06 13:13:17 +00:00
Erik Johnston 98933e3db6 Only fetch prev_content when a client is streaming/paginating. Use transactions for event streams. 2015-01-06 13:03:23 +00:00
Erik Johnston 1505055334 Don't return outliers when we get recent events for rooms. 2014-11-27 16:38:50 +00:00
Mark Haines db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines 1731af3f29 SYN-104: When going backwards the end token should be before the last event 2014-11-18 16:45:06 +00:00
Erik Johnston 65f846ade0 Notify users about invites. 2014-11-10 11:15:02 +00:00
Erik Johnston da1dda3e1d Add transaction level logging and timing information. Add a _simple_delete method 2014-10-28 11:18:04 +00:00
Erik Johnston c818aa13eb Add LIMIT to scalar subquery 2014-09-25 15:51:21 +01:00
Erik Johnston 70899d3ab2 Rename deletions to redactions 2014-09-24 15:27:59 +01:00
Erik Johnston 78af6bbb98 Add m.room.deletion. If an event is deleted it will be returned to clients 'pruned', i.e. all client specified keys will be removed. 2014-09-23 15:28:32 +01:00
Erik Johnston 28bcd01e8d SYN-47: Fix bug where we still returned events for rooms we had left.
SYN-47 #resolve
2014-09-19 14:45:21 +01:00
Paul "LeoNerd" Evans e53d77b501 Add a .runInteraction() method on SQLBaseStore itself to wrap the .db_pool 2014-09-12 14:28:07 +01:00
Erik Johnston 781ff713ba When getting a state event also include the previous content 2014-09-06 02:23:36 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Erik Johnston ee06023573 Get the equalities right. 2014-08-29 13:28:06 +01:00
Paul "LeoNerd" Evans 2c7c12bc6e Initial room event stream token must be s0, not s1, or everyone will miss the very first room event 2014-08-28 17:39:34 +01:00
Mark Haines 2aeaa7b77c Merge branch 'develop' into storage_transactions
Conflicts:
	synapse/handlers/room.py
	synapse/storage/stream.py
2014-08-27 17:15:58 +01:00
Erik Johnston 3a2a5b959c WIP: Completely change how event streaming and pagination work. This reflects the change in the underlying storage model. 2014-08-26 18:57:46 +01:00
Mark Haines 4b63b06cad Merge branch 'develop' into storage_transactions
Conflicts:
	synapse/api/auth.py
	synapse/handlers/room.py
	synapse/storage/__init__.py
2014-08-26 16:07:05 +01:00
Mark Haines ac21dfff6d Fix pyflakes errors 2014-08-26 16:02:59 +01:00
Mark Haines 4b2ad549d5 Move the event storage into a single transaction 2014-08-26 14:36:03 +01:00
Erik Johnston be6abdff19 Order 'get_recent_events_for_room' correctly. 2014-08-26 09:22:58 +01:00
Erik Johnston 808f663ed1 Don't return state event outlier's when paginating. 2014-08-22 13:06:07 +01:00
Erik Johnston e7ee0b9fc1 Change IM sync api to also return the current presence list. 2014-08-21 16:40:21 +01:00
Erik Johnston ebd3c41ede Make event stream storage return all membership events about the user, regardless of if they were in the room or not. 2014-08-20 17:09:44 +01:00
Erik Johnston d4fb1c8a92 Only hit get_room_events_stream if we have a valid user_id 2014-08-19 17:18:19 +01:00
Erik Johnston ae493c9418 Fix token to correct format 2014-08-19 16:45:55 +01:00
Erik Johnston eea2dc7dde Remove debug logging from token parsing funcs. 2014-08-19 16:40:38 +01:00
Erik Johnston d94765999d Add comment about what strorage.stream does 2014-08-19 16:40:25 +01:00
Erik Johnston 598a1d8ff9 Change the way pagination works to support out of order events. 2014-08-19 14:19:48 +01:00
Erik Johnston 1422a22970 Fix typos in SQL and where we still had rowid's (which no longer exist) 2014-08-18 16:25:18 +01:00
Erik Johnston 4eb8f84aa8 Make snapshot_all_rooms return results in the correct form, including start and end tokens. 2014-08-18 16:20:21 +01:00
Erik Johnston fc26275bb3 Add two different columns for ordering the events table, one which can be used for pagination and one which can be as tokens for notifying clients. Also add a 'processed' field which is currently always set to True 2014-08-18 15:50:41 +01:00
Erik Johnston d260a42ca2 PEP8 cleanups 2014-08-15 16:17:36 +01:00
Erik Johnston 8fa3cc37f9 Comment. 2014-08-15 16:11:25 +01:00
Erik Johnston cd2967d271 Fix bug when generating a key when get_room_events_stream returned zero rows 2014-08-15 16:05:46 +01:00
Erik Johnston 86be66c34e Actually use MAX_STREAM_SIZE constant. 2014-08-15 16:04:54 +01:00
Erik Johnston 8d1f763209 Fix pagination to work with new db schema 2014-08-15 15:53:06 +01:00
Erik Johnston 01f089d9fb Correctly return new token when returning events. Serialize events correctly. 2014-08-15 15:28:54 +01:00
Erik Johnston 114984a236 Start chagning the events stream to work with the new DB schema 2014-08-15 13:58:28 +01:00
Erik Johnston 2c46bb6208 Fix up typos and correct sql queries 2014-08-14 18:40:50 +01:00
Erik Johnston 7e681ad778 Update StreamStore 2014-08-14 18:01:39 +01:00
Matthew Hodgson f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00