Mirrors
/
synapse-old
mirror of https://github.com/matrix-org/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,004 Commits 842 Branches 671 Tags 438 MiB
Commit Graph

877 Commits

Author SHA1 Message Date
Erik Johnston 8888982db3 Don't insert None 2015-05-13 15:43:32 +01:00
Erik Johnston cf706cc6ef Don't return None 2015-05-13 15:31:25 +01:00
Erik Johnston 5971d240d4 Limit batch size 2015-05-13 15:26:49 +01:00
Erik Johnston ca4f458787 Fetch events in bulk 2015-05-13 15:13:42 +01:00
Erik Johnston 6edff11a88 Don't fetch redaction and rejection stuff for each event, so we can use index only scan 2015-05-13 14:39:05 +01:00
Mark Haines 63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Erik Johnston 02590c3e1d Temp turn off checking for rejections and redactions 2015-05-13 11:31:28 +01:00
Erik Johnston 619a21812b defer.gatherResults loop 2015-05-13 11:29:03 +01:00
Erik Johnston fec4485e28 Batch fetching of events for state groups 2015-05-13 11:22:42 +01:00
Erik Johnston 409bcc76bd Load events for state group seperately 2015-05-13 11:13:31 +01:00
Erik Johnston e122685978 You need to call contextmanager 2015-05-12 16:12:37 +01:00
Mark Haines a6fb2aa2a5 Merge pull request #144 from matrix-org/erikj/logging_context 
Preserving logging contexts
 2015-05-12 15:23:50 +01:00
Mark Haines 1fce36b111 Merge pull request #149 from matrix-org/erikj/backfill 
Backfill support
 2015-05-12 15:20:32 +01:00
Erik Johnston 8b28209c60 Err, delete the right stuff 2015-05-12 15:02:53 +01:00
Erik Johnston 30c72d377e Newlines 2015-05-12 14:47:40 +01:00
Erik Johnston e4eddf9b36 We do actually want to delete rows out of event_backward_extremities 2015-05-12 14:47:23 +01:00
Erik Johnston c1779a79bc Fix up _handle_prev_events to not try to insert duplicate rows 2015-05-12 14:41:50 +01:00
Erik Johnston 74850d7f75 Do state groups persistence /after/ checking if we have already persisted the event 2015-05-12 14:14:58 +01:00
Erik Johnston da6a7bbdde Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context 2015-05-12 13:10:42 +01:00
Mark Haines 5002056b16 SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes. 2015-05-12 11:20:40 +01:00
Erik Johnston 4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Erik Johnston 84e6b4001f Initial hack at wiring together pagination and backfill 2015-05-11 18:01:31 +01:00
Erik Johnston 17653a5dfe Move storage.stream._StreamToken to types.RoomStreamToken 2015-05-11 18:01:01 +01:00
Erik Johnston b036596b75 Prefer to use _simple_*. 2015-05-11 11:24:01 +01:00
Erik Johnston cd525c0f5a push_rules table expects an 'id' field 2015-05-11 11:24:01 +01:00
Erik Johnston 22d2f498fa Fix push rule bug: can't insert bool into small int column 2015-05-10 10:50:51 +01:00
Erik Johnston 476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Mark Haines 1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines 9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines 84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Mark Haines 6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston 0c4ac271ca Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371 2015-05-05 18:21:19 +01:00
Erik Johnston 0cf7e480b4 And use buffer(...) there as well 2015-05-05 18:20:01 +01:00
Erik Johnston ed2584050f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany 2015-05-05 18:15:20 +01:00
Erik Johnston 977338a7af Use buffer(...) when inserting into bytea column 2015-05-05 18:12:53 +01:00
Mark Haines 31049c4d72 Merge pull request #139 from matrix-org/bugs/SYN-369 
Fix race with cache invalidation. SYN-369
 2015-05-05 17:46:13 +01:00
Mark Haines deb0237166 Add some doc-string 2015-05-05 17:45:11 +01:00
Mark Haines d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Erik Johnston 9951542393 Add a comment about the zip(*[zip(sorted(...),...)]) 2015-05-05 17:06:55 +01:00
Mark Haines 041b6cba61 SYN-369: Add comments to the sequence number logic in the cache 2015-05-05 16:32:44 +01:00
Mark Haines 63075118a5 Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access 2015-05-05 16:24:04 +01:00
Erik Johnston 531d7955fd Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely 2015-05-05 16:12:28 +01:00
Mark Haines bfa4a7f8b0 Invalidate the room_member cache if the current state events updates 2015-05-05 15:43:49 +01:00
Mark Haines d0fece8d3c Missing return for when the event was already persisted 2015-05-05 15:39:09 +01:00
Erik Johnston bdcd7693c8 Fix indentation 2015-05-05 15:14:48 +01:00
Erik Johnston 43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston 1692dc019d Don't call 'encode_parameter' no-op 2015-05-05 15:00:30 +01:00
Mark Haines a9aea68fd5 Invalidate the caches from the correct thread 2015-05-05 14:57:08 +01:00
Mark Haines 261d809a47 Sequence the modifications to the cache so that selects don't race with inserts 2015-05-05 14:13:50 +01:00
Erik Johnston d9cc5de9e5 Correctly name transaction 2015-05-05 10:24:10 +01:00
Erik Johnston b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
David Baker eb9bd2d949 user_id now in user_threepids 2015-05-01 15:04:37 +01:00
David Baker 9eeb03c0dd Don't use self.execute: it's designed for fetching stuff 2015-05-01 14:21:25 +01:00
David Baker 7693f24792 No id field on user 2015-05-01 13:55:42 +01:00
David Baker 92b20713d7 More missed get_user_by_id API changes 2015-05-01 13:45:54 +01:00
Erik Johnston 4011cf1c42 Cache latest_event_ids_in_room 2015-05-01 13:06:26 +01:00
Erik Johnston 657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston fabb7acd45 Fix bug where we reconnected to the database on every query. 2015-05-01 10:24:24 +01:00
Erik Johnston 23c639ff32 Split a storage function in two so that we don't have to do extra work. 2015-05-01 10:17:19 +01:00
Erik Johnston 8be5284e91 Remove pointless join 2015-04-30 18:49:26 +01:00
Erik Johnston 503e4d3d52 Fix broken SQL 2015-04-30 18:44:47 +01:00
Erik Johnston 00718ae7a9 Need more yield 2015-04-30 18:43:39 +01:00
Erik Johnston 0465560c1a Add missing param 2015-04-30 18:42:44 +01:00
Erik Johnston 61d05daab1 More join conditions 2015-04-30 18:41:05 +01:00
Erik Johnston 6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Erik Johnston f383d5a801 Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction 2015-04-30 17:12:52 +01:00
Erik Johnston 69d4063651 Add get_rooms_for_user cache 2015-04-30 16:47:51 +01:00
Erik Johnston 3c4c229788 Don't use sub queries, it makes postgres sad 2015-04-30 16:16:53 +01:00
Mark Haines 29400b45b9 SYN-367: Use upsert rather than insert_or_replace 2015-04-30 15:21:31 +01:00
Erik Johnston 2366d28780 Don't needlessly join on state_events 2015-04-30 14:02:06 +01:00
Erik Johnston 1d7702833d Make simple query rather than long one and then throw away half the results 2015-04-30 10:16:12 +01:00
Erik Johnston b1ca784aca Correctly decode, for sqlite and postgres, rows from pushers table 2015-04-29 19:41:14 +01:00
Erik Johnston 4a9dc5b2f5 pushkey' are also bytes. 2015-04-29 19:27:02 +01:00
Erik Johnston 0ade2712d1 Typo 2015-04-29 19:17:25 +01:00
Erik Johnston 50f96f256f Also remove yield from within lock in the other generator 2015-04-29 19:17:00 +01:00
Erik Johnston d2d61a8288 Fix deadlock in id_generators. No idea why this was an actual deadlock. 2015-04-29 19:15:23 +01:00
Erik Johnston 3864b3a8e6 Actually return rows 2015-04-29 18:07:36 +01:00
Erik Johnston 0618978238 Typo, args wrong way round 2015-04-29 18:04:35 +01:00
Erik Johnston 09177f4f2e Decode buffers in same thread 2015-04-29 18:03:42 +01:00
Erik Johnston 472be88674 We store pusher data as bytes 2015-04-29 17:43:46 +01:00
David Baker 12d381bd5d Decode the data json in the storage layer (was moved but this part was missed) 2015-04-29 17:13:51 +01:00
David Baker f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker 61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker 4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston 0337eaf321 txn.execute doesn't return cursors 2015-04-29 16:43:39 +01:00
Erik Johnston 884fb88e28 txn.execute doesn't return cursors 2015-04-29 16:35:20 +01:00
Erik Johnston d76c058eea Fix invalid SQL to work in postgres land 2015-04-29 16:30:25 +01:00
Erik Johnston c8ed9bd278 pushers table requires an access_token 2015-04-29 15:33:27 +01:00
Erik Johnston 970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Erik Johnston 64991b0c8b Merge pull request #129 from matrix-org/key_distribution 
Key distribution v2
 2015-04-29 13:34:38 +01:00
Mark Haines e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines 4ad8b45155 Merge branch 'develop' into key_distribution 
Conflicts:
	synapse/config/homeserver.py
 2015-04-29 13:15:14 +01:00
Erik Johnston cd0864121b Make postgres database error slightly more helpful 2015-04-29 12:12:25 +01:00
Erik Johnston 4932a7e2d9 Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__ 2015-04-29 12:12:25 +01:00
Erik Johnston a08bf11138 Appease PEP8 2015-04-29 11:44:48 +01:00
Erik Johnston 204132a998 Check that postgres database has correct charset set 2015-04-29 11:42:28 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston 18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines 9182f87664 Merge pull request #126 from matrix-org/csauth 
Client / Server Auth Refactor
 2015-04-28 11:00:27 +01:00
Mark Haines 55e1bc8920 And don't bump the schema version unnecessarily 2015-04-28 10:54:15 +01:00
Mark Haines b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Erik Johnston 8558e1ec73 Make get_max_token into inlineCallbacks so that the lock works. 2015-04-27 15:19:44 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Erik Johnston 2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston e4c4664d73 Handle the fact that postgres databases can be restarted from under us 2015-04-27 12:40:49 +01:00
David Baker 8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
David Baker f8152f2708 rename db method to be more informative 2015-04-27 10:16:26 +01:00
David Baker 2f475bd5d5 pep8 2015-04-24 18:15:07 +01:00
David Baker a7b51f4539 Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266 2015-04-24 18:11:21 +01:00
Mark Haines eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines 4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
Mark Haines f30d47c876 Implement remote key lookup api 2015-04-22 14:21:08 +01:00
David Baker 91c8f828e1 pep8 2015-04-17 19:56:04 +01:00
David Baker 8db6832db8 Password reset, finally. 2015-04-17 19:53:47 +01:00
David Baker 117f35ac4a Add endpoint to get threepids from server 2015-04-17 17:20:18 +01:00
David Baker 4eea5cf6c2 pep8 2015-04-17 16:46:45 +01:00
David Baker f96ab9d18d make add3pid servlet work 2015-04-17 16:44:49 +01:00
Erik Johnston e3417bbbe0 Revert needless change to storage.profile 2015-04-17 16:13:14 +01:00
David Baker cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston 0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
Erik Johnston e1c0970c11 PEP8 2015-04-16 11:18:45 +01:00
Erik Johnston b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston bc9e69e160 Move encoding and decoding of JSON into storage layer 2015-04-16 11:01:09 +01:00
Erik Johnston ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00
Erik Johnston cf04cedf21 Change full_schemas/11 to work with postgres 2015-04-15 16:53:47 +01:00
Erik Johnston 5b31afcbd1 Remove debug logging 2015-04-15 16:27:04 +01:00
Erik Johnston 6e91f14d09 Add missing yield 2015-04-15 16:25:07 +01:00
Erik Johnston ed26e4012b pushers table requires a unique id. 2015-04-15 16:24:14 +01:00
Erik Johnston a5c72780e6 Don't pass in removed flag 2015-04-15 15:13:22 +01:00
Erik Johnston 4af32a2817 Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction 2015-04-15 14:51:21 +01:00
Erik Johnston 25d2914fba Turn off persistance of sent_transactions.response_json 2015-04-15 14:09:35 +01:00
Erik Johnston cce5d057d3 Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses 2015-04-15 11:43:25 +01:00
Erik Johnston a971fa9d58 Use try..finally in contextlib.contextmanager 2015-04-15 10:25:43 +01:00
Erik Johnston ded4128965 Use True for True rather than 1 2015-04-15 10:24:24 +01:00
Erik Johnston f9e12f79ca Add missing yield in storage func 2015-04-15 10:24:07 +01:00
Erik Johnston c756dfeb14 Correctly identify deadlocks 2015-04-15 10:23:42 +01:00
Mark Haines 32e14d8181 Return a sha256 fingerprint rather than the entire tls certificate 2015-04-14 19:10:09 +01:00
Erik Johnston 127fad17dd Add postgres database engine 2015-04-14 14:50:29 +01:00
Erik Johnston 5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston 3c741682e5 Correctly increment the _next_id initially 2015-04-14 09:54:44 +01:00
Erik Johnston 86fc9b617c For backwards compat, make state_groups.id have a type of int, not varchar 2015-04-13 17:03:49 +01:00
Erik Johnston 93937b2b31 Remove duplicate rows 2015-04-10 18:47:03 +01:00
Erik Johnston c5365dee56 Use case sensitive collations 2015-04-10 18:46:33 +01:00
Erik Johnston 4103b1c470 DROP indexes before recreating them 2015-04-10 18:46:09 +01:00