Commit Graph

979 Commits

Author SHA1 Message Date
Erik Johnston 4071f29653 Fetch events from events_id in their own transactions 2015-05-13 16:59:41 +01:00
Erik Johnston a988361aea Typo 2015-05-13 15:44:15 +01:00
Erik Johnston 8888982db3 Don't insert None 2015-05-13 15:43:32 +01:00
Erik Johnston cf706cc6ef Don't return None 2015-05-13 15:31:25 +01:00
Erik Johnston 5971d240d4 Limit batch size 2015-05-13 15:26:49 +01:00
Erik Johnston ca4f458787 Fetch events in bulk 2015-05-13 15:13:42 +01:00
Erik Johnston 6edff11a88 Don't fetch redaction and rejection stuff for each event, so we can use index only scan 2015-05-13 14:39:05 +01:00
Mark Haines 63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Erik Johnston 02590c3e1d Temp turn off checking for rejections and redactions 2015-05-13 11:31:28 +01:00
Erik Johnston 619a21812b defer.gatherResults loop 2015-05-13 11:29:03 +01:00
Erik Johnston fec4485e28 Batch fetching of events for state groups 2015-05-13 11:22:42 +01:00
Erik Johnston 409bcc76bd Load events for state group seperately 2015-05-13 11:13:31 +01:00
Erik Johnston e122685978 You need to call contextmanager 2015-05-12 16:12:37 +01:00
Mark Haines a6fb2aa2a5 Merge pull request #144 from matrix-org/erikj/logging_context
Preserving logging contexts
2015-05-12 15:23:50 +01:00
Mark Haines 1fce36b111 Merge pull request #149 from matrix-org/erikj/backfill
Backfill support
2015-05-12 15:20:32 +01:00
Erik Johnston 8b28209c60 Err, delete the right stuff 2015-05-12 15:02:53 +01:00
Erik Johnston 30c72d377e Newlines 2015-05-12 14:47:40 +01:00
Erik Johnston e4eddf9b36 We do actually want to delete rows out of event_backward_extremities 2015-05-12 14:47:23 +01:00
Erik Johnston c1779a79bc Fix up _handle_prev_events to not try to insert duplicate rows 2015-05-12 14:41:50 +01:00
Erik Johnston 74850d7f75 Do state groups persistence /after/ checking if we have already persisted the event 2015-05-12 14:14:58 +01:00
Erik Johnston da6a7bbdde Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context 2015-05-12 13:10:42 +01:00
Mark Haines 5002056b16 SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes. 2015-05-12 11:20:40 +01:00
Erik Johnston 4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Erik Johnston 84e6b4001f Initial hack at wiring together pagination and backfill 2015-05-11 18:01:31 +01:00
Erik Johnston 17653a5dfe Move storage.stream._StreamToken to types.RoomStreamToken 2015-05-11 18:01:01 +01:00
Erik Johnston b036596b75 Prefer to use _simple_*. 2015-05-11 11:24:01 +01:00
Erik Johnston cd525c0f5a push_rules table expects an 'id' field 2015-05-11 11:24:01 +01:00
Erik Johnston 22d2f498fa Fix push rule bug: can't insert bool into small int column 2015-05-10 10:50:51 +01:00
Erik Johnston 476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Mark Haines 1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines 9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines 84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Mark Haines 6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston 0c4ac271ca Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371 2015-05-05 18:21:19 +01:00
Erik Johnston 0cf7e480b4 And use buffer(...) there as well 2015-05-05 18:20:01 +01:00
Erik Johnston ed2584050f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany 2015-05-05 18:15:20 +01:00
Erik Johnston 977338a7af Use buffer(...) when inserting into bytea column 2015-05-05 18:12:53 +01:00
Mark Haines 31049c4d72 Merge pull request #139 from matrix-org/bugs/SYN-369
Fix race with cache invalidation. SYN-369
2015-05-05 17:46:13 +01:00
Mark Haines deb0237166 Add some doc-string 2015-05-05 17:45:11 +01:00
Mark Haines d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Erik Johnston 9951542393 Add a comment about the zip(*[zip(sorted(...),...)]) 2015-05-05 17:06:55 +01:00
Mark Haines 041b6cba61 SYN-369: Add comments to the sequence number logic in the cache 2015-05-05 16:32:44 +01:00
Mark Haines 63075118a5 Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access 2015-05-05 16:24:04 +01:00
Erik Johnston 531d7955fd Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely 2015-05-05 16:12:28 +01:00
Mark Haines bfa4a7f8b0 Invalidate the room_member cache if the current state events updates 2015-05-05 15:43:49 +01:00
Mark Haines d0fece8d3c Missing return for when the event was already persisted 2015-05-05 15:39:09 +01:00
Erik Johnston bdcd7693c8 Fix indentation 2015-05-05 15:14:48 +01:00
Erik Johnston 43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston 1692dc019d Don't call 'encode_parameter' no-op 2015-05-05 15:00:30 +01:00
Mark Haines a9aea68fd5 Invalidate the caches from the correct thread 2015-05-05 14:57:08 +01:00
Mark Haines 261d809a47 Sequence the modifications to the cache so that selects don't race with inserts 2015-05-05 14:13:50 +01:00
Erik Johnston d9cc5de9e5 Correctly name transaction 2015-05-05 10:24:10 +01:00
Erik Johnston b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
David Baker eb9bd2d949 user_id now in user_threepids 2015-05-01 15:04:37 +01:00
David Baker 9eeb03c0dd Don't use self.execute: it's designed for fetching stuff 2015-05-01 14:21:25 +01:00
David Baker 7693f24792 No id field on user 2015-05-01 13:55:42 +01:00
David Baker 92b20713d7 More missed get_user_by_id API changes 2015-05-01 13:45:54 +01:00
Erik Johnston 4011cf1c42 Cache latest_event_ids_in_room 2015-05-01 13:06:26 +01:00
Erik Johnston 657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston fabb7acd45 Fix bug where we reconnected to the database on every query. 2015-05-01 10:24:24 +01:00
Erik Johnston 23c639ff32 Split a storage function in two so that we don't have to do extra work. 2015-05-01 10:17:19 +01:00
Erik Johnston 8be5284e91 Remove pointless join 2015-04-30 18:49:26 +01:00
Erik Johnston 503e4d3d52 Fix broken SQL 2015-04-30 18:44:47 +01:00
Erik Johnston 00718ae7a9 Need more yield 2015-04-30 18:43:39 +01:00
Erik Johnston 0465560c1a Add missing param 2015-04-30 18:42:44 +01:00
Erik Johnston 61d05daab1 More join conditions 2015-04-30 18:41:05 +01:00
Erik Johnston 6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Erik Johnston f383d5a801 Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction 2015-04-30 17:12:52 +01:00
Erik Johnston 69d4063651 Add get_rooms_for_user cache 2015-04-30 16:47:51 +01:00
Erik Johnston 3c4c229788 Don't use sub queries, it makes postgres sad 2015-04-30 16:16:53 +01:00
Mark Haines 29400b45b9 SYN-367: Use upsert rather than insert_or_replace 2015-04-30 15:21:31 +01:00
Erik Johnston 2366d28780 Don't needlessly join on state_events 2015-04-30 14:02:06 +01:00
Erik Johnston 1d7702833d Make simple query rather than long one and then throw away half the results 2015-04-30 10:16:12 +01:00
Erik Johnston b1ca784aca Correctly decode, for sqlite and postgres, rows from pushers table 2015-04-29 19:41:14 +01:00
Erik Johnston 4a9dc5b2f5 pushkey' are also bytes. 2015-04-29 19:27:02 +01:00
Erik Johnston 0ade2712d1 Typo 2015-04-29 19:17:25 +01:00
Erik Johnston 50f96f256f Also remove yield from within lock in the other generator 2015-04-29 19:17:00 +01:00
Erik Johnston d2d61a8288 Fix deadlock in id_generators. No idea why this was an actual deadlock. 2015-04-29 19:15:23 +01:00
Erik Johnston 3864b3a8e6 Actually return rows 2015-04-29 18:07:36 +01:00
Erik Johnston 0618978238 Typo, args wrong way round 2015-04-29 18:04:35 +01:00
Erik Johnston 09177f4f2e Decode buffers in same thread 2015-04-29 18:03:42 +01:00
Erik Johnston 472be88674 We store pusher data as bytes 2015-04-29 17:43:46 +01:00
David Baker 12d381bd5d Decode the data json in the storage layer (was moved but this part was missed) 2015-04-29 17:13:51 +01:00
David Baker f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker 61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker 4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston 0337eaf321 txn.execute doesn't return cursors 2015-04-29 16:43:39 +01:00
Erik Johnston 884fb88e28 txn.execute doesn't return cursors 2015-04-29 16:35:20 +01:00
Erik Johnston d76c058eea Fix invalid SQL to work in postgres land 2015-04-29 16:30:25 +01:00
Erik Johnston c8ed9bd278 pushers table requires an access_token 2015-04-29 15:33:27 +01:00
Erik Johnston 970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Erik Johnston 64991b0c8b Merge pull request #129 from matrix-org/key_distribution
Key distribution v2
2015-04-29 13:34:38 +01:00
Mark Haines e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines 4ad8b45155 Merge branch 'develop' into key_distribution
Conflicts:
	synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Erik Johnston cd0864121b Make postgres database error slightly more helpful 2015-04-29 12:12:25 +01:00
Erik Johnston 4932a7e2d9 Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__ 2015-04-29 12:12:25 +01:00
Erik Johnston a08bf11138 Appease PEP8 2015-04-29 11:44:48 +01:00
Erik Johnston 204132a998 Check that postgres database has correct charset set 2015-04-29 11:42:28 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston 18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines 9182f87664 Merge pull request #126 from matrix-org/csauth
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines 55e1bc8920 And don't bump the schema version unnecessarily 2015-04-28 10:54:15 +01:00
Mark Haines b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Erik Johnston 8558e1ec73 Make get_max_token into inlineCallbacks so that the lock works. 2015-04-27 15:19:44 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Erik Johnston 2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston e4c4664d73 Handle the fact that postgres databases can be restarted from under us 2015-04-27 12:40:49 +01:00
David Baker 8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
David Baker f8152f2708 rename db method to be more informative 2015-04-27 10:16:26 +01:00
David Baker 2f475bd5d5 pep8 2015-04-24 18:15:07 +01:00
David Baker a7b51f4539 Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266 2015-04-24 18:11:21 +01:00
Mark Haines eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines 4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
Mark Haines f30d47c876 Implement remote key lookup api 2015-04-22 14:21:08 +01:00
David Baker 91c8f828e1 pep8 2015-04-17 19:56:04 +01:00
David Baker 8db6832db8 Password reset, finally. 2015-04-17 19:53:47 +01:00
David Baker 117f35ac4a Add endpoint to get threepids from server 2015-04-17 17:20:18 +01:00
David Baker 4eea5cf6c2 pep8 2015-04-17 16:46:45 +01:00
David Baker f96ab9d18d make add3pid servlet work 2015-04-17 16:44:49 +01:00
Erik Johnston e3417bbbe0 Revert needless change to storage.profile 2015-04-17 16:13:14 +01:00
David Baker cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston 0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
Erik Johnston e1c0970c11 PEP8 2015-04-16 11:18:45 +01:00
Erik Johnston b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston bc9e69e160 Move encoding and decoding of JSON into storage layer 2015-04-16 11:01:09 +01:00
Erik Johnston ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00
Erik Johnston cf04cedf21 Change full_schemas/11 to work with postgres 2015-04-15 16:53:47 +01:00
Erik Johnston 5b31afcbd1 Remove debug logging 2015-04-15 16:27:04 +01:00
Erik Johnston 6e91f14d09 Add missing yield 2015-04-15 16:25:07 +01:00
Erik Johnston ed26e4012b pushers table requires a unique id. 2015-04-15 16:24:14 +01:00
Erik Johnston a5c72780e6 Don't pass in removed flag 2015-04-15 15:13:22 +01:00
Erik Johnston 4af32a2817 Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction 2015-04-15 14:51:21 +01:00
Erik Johnston 25d2914fba Turn off persistance of sent_transactions.response_json 2015-04-15 14:09:35 +01:00
Erik Johnston cce5d057d3 Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses 2015-04-15 11:43:25 +01:00
Erik Johnston a971fa9d58 Use try..finally in contextlib.contextmanager 2015-04-15 10:25:43 +01:00
Erik Johnston ded4128965 Use True for True rather than 1 2015-04-15 10:24:24 +01:00
Erik Johnston f9e12f79ca Add missing yield in storage func 2015-04-15 10:24:07 +01:00
Erik Johnston c756dfeb14 Correctly identify deadlocks 2015-04-15 10:23:42 +01:00
Mark Haines 32e14d8181 Return a sha256 fingerprint rather than the entire tls certificate 2015-04-14 19:10:09 +01:00
Erik Johnston 127fad17dd Add postgres database engine 2015-04-14 14:50:29 +01:00
Erik Johnston 5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston 3c741682e5 Correctly increment the _next_id initially 2015-04-14 09:54:44 +01:00
Erik Johnston 86fc9b617c For backwards compat, make state_groups.id have a type of int, not varchar 2015-04-13 17:03:49 +01:00
Erik Johnston 93937b2b31 Remove duplicate rows 2015-04-10 18:47:03 +01:00
Erik Johnston c5365dee56 Use case sensitive collations 2015-04-10 18:46:33 +01:00
Erik Johnston 4103b1c470 DROP indexes before recreating them 2015-04-10 18:46:09 +01:00
Erik Johnston 4d5b098626 Use LONGBLOB and TEXT for arbitary length rows 2015-04-10 18:45:09 +01:00
Erik Johnston 7ed2ec3061 Handle the fact that in sqlite binary data might be stored as unicode or bytes 2015-04-10 13:41:54 +01:00
Erik Johnston ce797ad373 Bump schema version 2015-04-10 11:22:30 +01:00
Erik Johnston 7e863c51e6 Use unsigned bigint 2015-04-10 11:22:04 +01:00
Erik Johnston 0f12772e32 SQLite wants INTEGER and not BIGINT for primary keys 2015-04-10 11:16:09 +01:00
Erik Johnston d5d4281647 Update full_schemas/16 to match delta files. Add delta/16 scripts 2015-04-10 10:59:46 +01:00
Erik Johnston cda4a6f93f Revert non-trivial changes to upgrade scripts 2015-04-10 10:19:50 +01:00
Erik Johnston e2722f58ee Fix schema again 2015-04-10 10:16:29 +01:00
Erik Johnston a1665c5094 Revert non-trivial schema changes and move them to a new schema version. 2015-04-10 10:05:44 +01:00
Erik Johnston 2ded344620 Remove unused import 2015-04-09 13:46:06 +01:00
Erik Johnston 9707acfc40 Remove spurious spaces 2015-04-09 13:45:20 +01:00
Erik Johnston 8bf285e082 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-09 13:13:26 +01:00
Erik Johnston 8ad0f4912e
Stream ordering and out of order insertions.
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Erik Johnston 279a547a8b Use generic db exceptions rather than sqlite3 specific ones 2015-04-08 16:53:48 +01:00
Mark Haines 83f5125d52 Merge pull request #114 from matrix-org/improve_get_event_cache
Improve get event cache
2015-04-08 16:50:27 +01:00
Erik Johnston 3887350e47 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 14:11:07 +01:00
Erik Johnston c1b34af441 Move database timer logging to seperate logger 2015-04-08 13:31:19 +01:00
Erik Johnston 9a05795619 Retry transaction, not SQL query 2015-04-08 13:11:28 +01:00
Erik Johnston 24d8134ac1 Fix maria engine to correctly recognize deadlocks 2015-04-08 13:10:54 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston 49d6aa1394 Retry on deadlock 2015-04-07 15:28:37 +01:00
Erik Johnston 0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston 6bc9edd8b2 Fix prepare_sqlite3_database's convert_param_style 2015-04-07 12:13:58 +01:00
Erik Johnston 05a35d62b6 Bump database version 2015-04-07 12:10:15 +01:00
Erik Johnston 8574bf62dc Add index to presence table 2015-04-07 12:09:36 +01:00
Erik Johnston 0af5f5efaf Don't use multiple UNIQUE constraints; it will cause deadlocks 2015-04-07 12:08:35 +01:00
Erik Johnston c8d3f6486d Implement or_ignore flag on inserts 2015-04-07 12:06:22 +01:00
Erik Johnston 304111afd0 Don't use AUTOINCREMENT, use an in memory version 2015-04-07 12:05:36 +01:00
Erik Johnston 779f7b0f44 Fix unicode support 2015-04-02 10:06:22 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal 5e88a09a42 Add same user_id char checks as registration. 2015-03-31 14:00:25 +01:00
Kegan Dougal cf1fa59f4b Use a sender localpart instead of a user ID.
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
Kegan Dougal b59aa74556 Fix tests and missing returns on deferreds. 2015-03-31 11:35:45 +01:00
Kegan Dougal d33ae65efc Remove more reg/unreg methods. Read config not database for cache. 2015-03-31 11:00:00 +01:00
Paul "LeoNerd" Evans 3e420aebd8 Revert "Add another @cached wrapper, this time on get_presence_state()"
This reverts commit ff1fa0fbf8.
2015-03-27 16:16:58 +00:00
Paul "LeoNerd" Evans ff1fa0fbf8 Add another @cached wrapper, this time on get_presence_state() 2015-03-27 15:57:16 +00:00
David Baker a32e876ef4 Delete pushers when changing password 2015-03-26 13:40:16 +00:00
Paul "LeoNerd" Evans a198894bf7 Appease pep8 2015-03-26 11:53:58 +00:00
Kegan Dougal 32206dde3f Fixes from PR comments 2015-03-26 10:11:52 +00:00
Kegan Dougal 4edcbcee3b Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans 953e40f9dc Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one 2015-03-25 19:12:16 +00:00
David Baker c1a256cc4c Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API. 2015-03-25 19:06:22 +00:00
Paul "LeoNerd" Evans f173d40a32 Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 1b988b051b Store the rejected reason in (Frozen)Event structs 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 033a517feb Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
David Baker c7023f2155 1) Pushers are now associated with an access token
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston 0ba393924a Escape non printing ascii character 2015-03-24 16:31:52 +00:00
Erik Johnston f488293d96 Don't reinsert into event_edges 2015-03-24 16:20:26 +00:00
Erik Johnston 1aa44939fc Fix bugs in transactions storage 2015-03-24 16:20:05 +00:00
Erik Johnston 5a447098dd Don't use room hosts table 2015-03-24 16:19:24 +00:00
Erik Johnston 9e98f1022a Don't order by rowid 2015-03-24 16:19:01 +00:00
Erik Johnston 9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
David Baker d19e79ecc9 Make deleting other access tokens when you change your password actually work 2015-03-24 15:33:48 +00:00
Erik Johnston 6e7131f02f Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable. 2015-03-23 15:38:56 +00:00
Erik Johnston 9a7f496298 Sanitize RoomMemberStore 2015-03-23 15:29:04 +00:00
David Baker d98660a60d Implement password changing (finally) along with a start on making client/server auth more general. 2015-03-23 14:20:28 +00:00
Erik Johnston d5272b1d2c Use 'update or insert' rather than on 'conflict replace' 2015-03-23 14:02:34 +00:00
Erik Johnston 278149f533 Sanitize TransactionStore 2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans 72d8406409 Put a cache on get_aliases_for_room 2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans 0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans b1022ed8b5 func(*EXPR) is valid Python syntax, really... 2015-03-20 17:47:45 +00:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston 80cd08c190 PEP8 2015-03-20 16:03:25 +00:00
Erik Johnston 9517f4da4d Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement 2015-03-20 16:02:47 +00:00
Erik Johnston dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans ceb61daa70 Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream 2015-03-20 15:44:06 +00:00
Erik Johnston fce0114005 Start removing Tables 2015-03-20 15:05:44 +00:00
Erik Johnston 7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans 91cb46191d Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though 2015-03-20 14:59:45 +00:00
Erik Johnston 87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston d288d273e1 Generate transaction id in code 2015-03-20 10:57:44 +00:00
Erik Johnston d4f50f3ae5 decode_result takes an iterable 2015-03-20 10:57:26 +00:00
Erik Johnston 0d0610870d Fix up schemas some more 2015-03-20 10:55:31 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Erik Johnston 58ed393235 Remove redundant key 2015-03-19 15:12:05 +00:00
Erik Johnston fae059cc18 Fix up schemas to work with mariadb 2015-03-19 13:42:39 +00:00
Erik Johnston bb24609158 Clean out event_forward_extremities table when the server rejoins the room 2015-03-18 11:19:47 +00:00
Erik Johnston 89036579ed Update schema to work with mariadb 2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans 93978c5e2b @cached() annotate get_user_by_token() - achieves a minor DB performance improvement 2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans 1489521ee5 Be polite and ensure we use @functools.wraps() when creating a function decorator 2015-03-17 17:19:22 +00:00
Paul "LeoNerd" Evans be170b1426 Add a metric for the scheduling latency of SQL queries 2015-03-16 17:21:59 +00:00
Erik Johnston cd2539ab2a Merge pull request #110 from matrix-org/fix_ban
Fix ban
2015-03-16 15:36:52 +00:00
Kegan Dougal f0d6f724a2 Set the service ID as soon as it is known. 2015-03-16 15:24:32 +00:00
Erik Johnston f1d2b94e0b Copy dict of context.current_state before changing it. 2015-03-16 15:13:05 +00:00
Kegan Dougal 835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal f9232c7917 Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans e75fa8bbbf Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 63cb7ece62 Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f1fbe3e09f Rename TimerMetric to DistributionMetric; as it could count more than just time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 642f725fd7 Pretend the 'getEvent' cache is just another cache in the set of all the others for metric 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans fa319a5786 Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely 2015-03-12 16:24:51 +00:00