Mirrors
/
synapse-old
mirror of https://github.com/matrix-org/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,784 Commits 842 Branches 671 Tags 438 MiB
Commit Graph

1074 Commits

Author SHA1 Message Date
Erik Johnston 2f39dc19a2 Remove more references to dead PDU tables 2014-10-31 14:27:14 +00:00
Erik Johnston bfa36a72b9 Remove PDU tables. 2014-10-31 14:00:32 +00:00
Erik Johnston 21fe249d62 Actually don't store any PDUs 2014-10-31 10:47:34 +00:00
Erik Johnston 841df4da71 Don't store any PDUs 2014-10-31 09:59:59 +00:00
Erik Johnston f2de2d644a Move the impl of backfill to use events. 2014-10-31 09:59:02 +00:00
Erik Johnston ef9c4476a0 Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 2014-10-30 11:18:28 +00:00
Mark Haines 7c06399512 Merge branch 'develop' into request_logging 
Conflicts:
	synapse/config/logger.py
 2014-10-30 11:13:58 +00:00
Mark Haines 7d709542ca Fix pep8 warnings 2014-10-30 11:10:17 +00:00
Erik Johnston aa80900a8e Fix SQL so that accepts we may want to persist events twice. 2014-10-30 10:11:06 +00:00
Mark Haines b29517bd01 Add a request-id to each log line 2014-10-30 01:21:33 +00:00
Erik Johnston e7858b6d7e Start filling out and using new events tables 2014-10-29 16:59:24 +00:00
Erik Johnston a10c2ec88d Don't reference PDU when persisting event 2014-10-28 17:15:32 +00:00
Erik Johnston 2d1dfb3b34 Begin implementing all the PDU storage stuff in Events land 2014-10-28 16:42:35 +00:00
Erik Johnston da1dda3e1d Add transaction level logging and timing information. Add a _simple_delete method 2014-10-28 11:18:04 +00:00
Erik Johnston 967ce43b59 Clean up LoggingTransaction 2014-10-28 10:53:11 +00:00
Erik Johnston 8e358ef35a Add timer to LoggingTransaction 2014-10-28 10:34:05 +00:00
Erik Johnston ad9226eeec Merge branch 'event_signing' of github.com:matrix-org/synapse into federation_authorization 
Conflicts:
	synapse/storage/__init__.py
 2014-10-27 11:58:32 +00:00
Mark Haines 5e2236f9ff fix pyflakes warnings 2014-10-27 11:19:15 +00:00
Mark Haines acb2d171e8 Merge branch 'develop' into event_signing 2014-10-27 11:14:11 +00:00
Mark Haines d56e389a95 Fix pyflakes warnings 2014-10-27 10:33:17 +00:00
Erik Johnston bb4a20174c Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 
Conflicts:
	synapse/federation/transport.py
	synapse/handlers/message.py
 2014-10-27 10:20:44 +00:00
Mark Haines db2e350e29 Wrap preparing the database in a transaction. Otherwise it will take many seconds to complete because sqlite will create a transaction per statement 2014-10-24 19:04:26 +01:00
Erik Johnston 5662be894e Bump database version number. 2014-10-17 20:26:18 +01:00
Erik Johnston 5ffe5ab43f Use state groups to get current state. Make join dance actually work. 2014-10-17 18:56:42 +01:00
Mark Haines dc3c2823ac Merge branch 'develop' into event_signing 
Conflicts:
	synapse/federation/replication.py
 2014-10-17 17:33:58 +01:00
Mark Haines 82c5820767 keep 'origin_server_ts' as 'ts' in the database to avoid needlessly updating schema 2014-10-17 17:31:48 +01:00
Mark Haines f5cf7ac25b SPEC-7: Rename 'ts' to 'origin_server_ts' 2014-10-17 17:12:25 +01:00
Mark Haines c8f996e29f Hash the same content covered by the signature when referencing previous PDUs rather than reusing the PDU content hashes 2014-10-17 11:40:35 +01:00
Mark Haines bb04447c44 Include hashes of previous pdus when referencing them 2014-10-16 23:25:12 +01:00
Mark Haines 66104da10c Sign outgoing PDUs. 2014-10-16 00:09:48 +01:00
Mark Haines 1c445f88f6 persist hashes and origin signatures for PDUs 2014-10-15 17:09:04 +01:00
Erik Johnston e7bc1291a0 Begin making auth use event.old_state_events 2014-10-15 16:06:59 +01:00
Erik Johnston 80472ac198 Add missing package storate.state 2014-10-15 10:04:55 +01:00
Erik Johnston 5fefc12d1e Begin implementing state groups. 2014-10-14 16:59:51 +01:00
Mark Haines 3dac27a8a9 Storage for pdu signatures 2014-10-14 14:58:31 +01:00
Mark Haines 07639c79d9 Respond with more helpful error messages for unsigned requests 2014-10-13 16:39:15 +01:00
Mark Haines 9605593d11 Merge branch 'develop' into server2server_signing 
Conflicts:
	synapse/storage/__init__.py
	tests/rest/test_presence.py
 2014-09-30 17:55:06 +01:00
Mark Haines b95a178584 SYN-75 Verify signatures on server to server transactions 2014-09-30 15:15:10 +01:00
Erik Johnston fbf6320614 pyflakes cleanup 2014-09-30 12:38:38 +01:00
Erik Johnston e06adc6d7e SYN-2: Allow server admins to delete room aliases 2014-09-30 11:31:42 +01:00
Erik Johnston 7151615260 Update docstring 2014-09-29 15:35:54 +01:00
Erik Johnston 1550ab9e2f SYN-48: Delete dead code 2014-09-29 15:04:47 +01:00
Erik Johnston 3ccb17ce59 SYN-48: Implement WHOIS rest servlet 2014-09-29 14:59:52 +01:00
Erik Johnston c65306f877 Add auth check to test if a user is an admin or not. 2014-09-29 13:35:38 +01:00
Erik Johnston f7d80930f2 SYN-48: Track User-Agents as well as IPs for client devices. 2014-09-29 13:35:15 +01:00
Erik Johnston 0fdf308874 Track the IP users connect with. Add an admin column to users table. 2014-09-26 16:36:24 +01:00
Erik Johnston 37bfe44046 Merge branch 'deletions' of github.com:matrix-org/synapse into develop 2014-09-25 17:02:53 +01:00
Erik Johnston c818aa13eb Add LIMIT to scalar subquery 2014-09-25 15:51:21 +01:00
Erik Johnston ba87eb6753 Fix bug where we tried to insert state events with null state key 2014-09-25 14:45:27 +01:00
Erik Johnston 70899d3ab2 Rename deletions to redactions 2014-09-24 15:27:59 +01:00
Erik Johnston 7d9a84a445 Make deleting deletes not undelete 2014-09-24 14:18:08 +01:00
Erik Johnston 1e6c5b205c Fix bug where we didn't correctly pull out the event_id of the deletion 2014-09-24 13:29:20 +01:00
Erik Johnston 4354590a69 Add v4 deltas to current sql. 2014-09-24 11:06:41 +01:00
Erik Johnston efea61dc50 Rename 'pruned' to 'pruned_because' 2014-09-23 17:40:58 +01:00
Erik Johnston bc250a6afa SYN-12: Implement auth for deletion by adding a 'delete_level' on the ops levels event 
SYN-12 # comment Auth has been added.
 2014-09-23 17:36:24 +01:00
Erik Johnston b99f6eb904 Make sure we don't persist the 'pruned' key 2014-09-23 15:29:27 +01:00
Erik Johnston 78af6bbb98 Add m.room.deletion. If an event is deleted it will be returned to clients 'pruned', i.e. all client specified keys will be removed. 2014-09-23 15:28:32 +01:00
Erik Johnston 231afe464a Add a deletions table 2014-09-22 13:42:52 +01:00
Erik Johnston 28bcd01e8d SYN-47: Fix bug where we still returned events for rooms we had left. 
SYN-47 #resolve
 2014-09-19 14:45:21 +01:00
Paul "LeoNerd" Evans 4571cf7baa Merge branch 'develop' into test-sqlite-memory 2014-09-17 18:27:47 +01:00
Erik Johnston 59516a8bb1 Correctly handle receiving 'missing' Pdus from federation, rather than just discarding them. 2014-09-15 16:40:44 +01:00
Paul "LeoNerd" Evans b0406b9ead Merge remote-tracking branch 'origin/develop' into test-sqlite-memory 2014-09-15 14:15:10 +01:00
Erik Johnston 5bd9369a62 Correctly handle the 'age' key in events and pdus 2014-09-15 13:26:11 +01:00
Erik Johnston 14975ce5bc Fix bug where we relied on the current_state_events being updated when we are handling type specific persistence 2014-09-12 17:57:02 +01:00
Erik Johnston 667e747ed1 Fix bug where we no longer stored user_id on Pdus 2014-09-12 17:56:21 +01:00
Paul "LeoNerd" Evans 1c51c8ab7d Merge remote-tracking branch 'origin/develop' into test-sqlite-memory 
Conflicts:
	synapse/storage/pdu.py
 2014-09-12 17:20:06 +01:00
Erik Johnston 39e3fc69e5 Make the state resolution use actual power levels rather than taking them from a Pdu key. 2014-09-12 17:11:09 +01:00
Erik Johnston b42fe05c51 Fix bug where we incorrectly removed a remote host from the list of hosts in a room when any user from that host left that room even if they weren't the last user from that host in that room 2014-09-12 17:11:09 +01:00
Erik Johnston ca1ae7cf9b Fix bug where we didn't return a tuple when expected. 2014-09-12 17:11:09 +01:00
Paul "LeoNerd" Evans aa525e4a63 More accurate docs / clearer paramter names in RoomMemberStore 2014-09-12 16:43:49 +01:00
Paul "LeoNerd" Evans a87eac4308 Revert recent changes to RoomMemberStore 2014-09-12 15:51:51 +01:00
Paul "LeoNerd" Evans a840ff8f3f Now don't need the other logger.debug() call in _execute 2014-09-12 14:38:27 +01:00
Paul "LeoNerd" Evans 1c20249884 Logging of all SQL queries via the 'synapse.storage.SQL' logger 2014-09-12 14:37:55 +01:00
Paul "LeoNerd" Evans e53d77b501 Add a .runInteraction() method on SQLBaseStore itself to wrap the .db_pool 2014-09-12 14:28:07 +01:00
Paul "LeoNerd" Evans 249e8f2277 Add a better _store_room_member_txn() method that takes separated fields instead of an event object; also add FIXME comment about a big bug in the logic 2014-09-11 18:52:35 +01:00
Paul "LeoNerd" Evans aaf9ab68c6 Rename _store_room_member_txn to _store_room_member_from_event_txn so we can create another, more sensible function of that name 2014-09-11 18:44:04 +01:00
Paul "LeoNerd" Evans 55397f6347 prepare_database() on db_conn, not plain name, so we can pass in the connection from outside 2014-09-10 16:23:58 +01:00
Paul "LeoNerd" Evans ce55a8cc4b Move database preparing code out of homserver.py into storage where it belongs 2014-09-10 15:42:15 +01:00
Erik Johnston a75f8686ba Fix bug where we used an unbound local variable if we ended up rolling back the persist_event transaction 2014-09-09 16:27:59 +01:00
Erik Johnston e062f2dfa8 Apparently we can't do txn.rollback(), so raise and catch an exception instead. 2014-09-08 22:37:19 +01:00
Erik Johnston 83ce57302d Fix bug in state handling where we incorrectly identified a missing pdu. Update tests to catch this case. 2014-09-08 19:50:59 +01:00
Erik Johnston c0577ea87a Rollback if we try and insert duplicate events 2014-09-08 18:34:18 +01:00
Erik Johnston 781ff713ba When getting a state event also include the previous content 2014-09-06 02:23:36 +01:00
Erik Johnston 9dd4570b68 Generate m.room.aliases event when the HS creates a room alias 2014-09-05 21:35:56 +01:00
Paul "LeoNerd" Evans d703e712f7 Add support to _simple_insert() to do INSERT OR REPLACE 2014-09-03 18:18:41 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans 756e171ad0 Store SQL DDL deltas as well; attempt to upgrade the database on startup if it's too old 2014-09-03 14:14:39 +01:00
Paul "LeoNerd" Evans da3e04df8b Rename the 'do_users_share_a_room' to something slightly less verb-sounding 2014-09-03 10:40:48 +01:00
Mark Haines c6eafdfbaf Add copyright notices and fix pyflakes errors 2014-09-03 09:43:11 +01:00
Erik Johnston 4b8244fbf8 Fix bug where we didn't correctly store the ops power levels event. 2014-09-02 12:12:14 +01:00
Erik Johnston f452899fe2 Merge branch 'develop' of github.com:matrix-org/synapse into room_config 2014-09-01 19:57:28 +01:00
Erik Johnston b8ab9f1c0a Add all the necessary checks to make banning work. 2014-09-01 18:24:56 +01:00
Erik Johnston db7109c43b Add beginnings of ban support. 2014-09-01 16:15:34 +01:00
Mark Haines 9ea1de432d Fix homeserver config parsing 2014-09-01 15:51:15 +01:00
Erik Johnston 865469f233 Implement power level lists, default power levels and send_evnet_level/add_state_level events. 2014-09-01 14:39:09 +01:00
Mark Haines 3eb45eba0e Merge branch 'develop' into server2server_tls 2014-08-31 16:08:20 +01:00
Erik Johnston 1118f02689 Start adding storage for new events. 2014-08-29 15:18:30 +01:00
Erik Johnston ee06023573 Get the equalities right. 2014-08-29 13:28:06 +01:00
Mark Haines d7ae9b90a0 Add store for server certificates and keys 2014-08-28 18:19:47 +01:00
Paul "LeoNerd" Evans 2c7c12bc6e Initial room event stream token must be s0, not s1, or everyone will miss the very first room event 2014-08-28 17:39:34 +01:00
Mark Haines 2aeaa7b77c Merge branch 'develop' into storage_transactions 
Conflicts:
	synapse/handlers/room.py
	synapse/storage/stream.py
 2014-08-27 17:15:58 +01:00
Mark Haines f54b70520a Return the store_id from persist_event 2014-08-27 17:03:45 +01:00
Mark Haines b30358f439 add _get_room_member, fix datastore methods 2014-08-27 16:51:54 +01:00
Mark Haines a03c7f27a8 Fill out prev_events before calling persist_event 2014-08-27 14:32:19 +01:00
Erik Johnston 47519cd8c2 Merge branch 'develop' of github.com:matrix-org/synapse into stream_refactor 
Conflicts:
	synapse/handlers/events.py
	synapse/rest/events.py
	synapse/rest/room.py
 2014-08-27 14:13:06 +01:00
Mark Haines 474dcecb11 Remove unused populate_previous_pdus 2014-08-27 11:34:31 +01:00
Mark Haines bf05218c4b Merge branch 'develop' into storage_transactions 2014-08-27 11:19:37 +01:00
Kegan Dougal c65885e166 Added support for GET /events/$eventid with auth checks. 2014-08-27 10:33:01 +01:00
Mark Haines d2798de660 Fold federation/handler into handlers/federation 2014-08-26 19:49:42 +01:00
Erik Johnston 93cff1668c Merge branch 'develop' of github.com:matrix-org/synapse into stream_refactor 2014-08-26 18:57:55 +01:00
Erik Johnston 3a2a5b959c WIP: Completely change how event streaming and pagination work. This reflects the change in the underlying storage model. 2014-08-26 18:57:46 +01:00
Mark Haines 64e2a5d58e Move pdu and event persistence into a single persist_event function 2014-08-26 18:01:36 +01:00
Kegan Dougal 5a3df1d029 Feedback: Removed FeedbackRestServlet. Modified keys on FeedbackEvent. Expanded the feedback constants to fully explain what type of feedback they are. 2014-08-26 17:49:46 +01:00
Mark Haines 4b63b06cad Merge branch 'develop' into storage_transactions 
Conflicts:
	synapse/api/auth.py
	synapse/handlers/room.py
	synapse/storage/__init__.py
 2014-08-26 16:07:05 +01:00
Mark Haines ac21dfff6d Fix pyflakes errors 2014-08-26 16:02:59 +01:00
Mark Haines 32347bfcc9 fix a few pyflakes errors 2014-08-26 16:01:29 +01:00
Mark Haines 4b2ad549d5 Move the event storage into a single transaction 2014-08-26 14:36:03 +01:00
Erik Johnston a664ec20e0 Add a do_users_share_a_room method and use that in the presence handler. 2014-08-26 13:38:26 +01:00
Kegan Dougal 47c3a089c5 Merge branch 'develop' of github.com:matrix-org/synapse into client_server_url_rename 2014-08-26 09:26:33 +01:00
Kegan Dougal cab3095803 Removed member list servlet: now using generic state paths. 2014-08-26 09:26:07 +01:00
Erik Johnston be6abdff19 Order 'get_recent_events_for_room' correctly. 2014-08-26 09:22:58 +01:00
Erik Johnston 9d86c8c7a6 Add a unique constraint on the room hosts table 2014-08-24 11:29:29 +01:00
Erik Johnston 537ecd4e99 Turn off spammy logging 2014-08-22 18:12:38 +01:00
Erik Johnston 9f514915af Add indices to schema 2014-08-22 18:03:39 +01:00
Mark Haines 1379dcae6f Take a snapshot of the state of the room before performing updates 2014-08-22 17:00:10 +01:00
Erik Johnston 808f663ed1 Don't return state event outlier's when paginating. 2014-08-22 13:06:07 +01:00
Erik Johnston e7ee0b9fc1 Change IM sync api to also return the current presence list. 2014-08-21 16:40:21 +01:00
Erik Johnston 063e1b22e6 Stop internal keys from getting into SynapseEvents 2014-08-21 15:06:00 +01:00
Erik Johnston ebd3c41ede Make event stream storage return all membership events about the user, regardless of if they were in the room or not. 2014-08-20 17:09:44 +01:00
Erik Johnston e8244c23ba Give the event_id of the failed event 2014-08-20 15:53:07 +01:00
Erik Johnston d4fb1c8a92 Only hit get_room_events_stream if we have a valid user_id 2014-08-19 17:18:19 +01:00
Erik Johnston ae493c9418 Fix token to correct format 2014-08-19 16:45:55 +01:00
Erik Johnston eea2dc7dde Remove debug logging from token parsing funcs. 2014-08-19 16:40:38 +01:00
Erik Johnston d94765999d Add comment about what strorage.stream does 2014-08-19 16:40:25 +01:00
Erik Johnston 840771190f Fix bug where we sometimes set min_token to None. 2014-08-19 14:32:47 +01:00
Erik Johnston 75b6d982a0 Add a 'backfill room' button 2014-08-19 14:20:03 +01:00
Erik Johnston 598a1d8ff9 Change the way pagination works to support out of order events. 2014-08-19 14:19:48 +01:00
Erik Johnston 1422a22970 Fix typos in SQL and where we still had rowid's (which no longer exist) 2014-08-18 16:25:18 +01:00
Erik Johnston 4eb8f84aa8 Make snapshot_all_rooms return results in the correct form, including start and end tokens. 2014-08-18 16:20:21 +01:00
Erik Johnston 709a92cee8 SQL doesn't allow AUTOINCREMENT on non PRIMARY KEY columns. 2014-08-18 16:00:46 +01:00
Erik Johnston fc26275bb3 Add two different columns for ordering the events table, one which can be used for pagination and one which can be as tokens for notifying clients. Also add a 'processed' field which is currently always set to True 2014-08-18 15:50:41 +01:00
Erik Johnston 1c2caacd67 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor 2014-08-18 11:14:57 +01:00
Matthew Hodgson 0f9b633af7 retcols is mandatory. i have no idea how this could ever have worked? 2014-08-16 01:46:35 +01:00
Erik Johnston 0e938b1ff7 Rename method name to not clash with other ones in storage. 2014-08-15 16:47:48 +01:00
Erik Johnston 506711749f We no longer need to special case room config events. 2014-08-15 16:45:16 +01:00
Erik Johnston d260a42ca2 PEP8 cleanups 2014-08-15 16:17:36 +01:00
Erik Johnston 8fa3cc37f9 Comment. 2014-08-15 16:11:25 +01:00
Erik Johnston cd2967d271 Fix bug when generating a key when get_room_events_stream returned zero rows 2014-08-15 16:05:46 +01:00
Erik Johnston 86be66c34e Actually use MAX_STREAM_SIZE constant. 2014-08-15 16:04:54 +01:00
Erik Johnston 8d1f763209 Fix pagination to work with new db schema 2014-08-15 15:53:06 +01:00
Paul "LeoNerd" Evans 3c532314ec Fix imsync's SELECT query to only find the rooms I'm actually joined in, not every room I have ever joined 2014-08-15 15:44:53 +01:00
Erik Johnston 01f089d9fb Correctly return new token when returning events. Serialize events correctly. 2014-08-15 15:28:54 +01:00
Erik Johnston 114984a236 Start chagning the events stream to work with the new DB schema 2014-08-15 13:58:28 +01:00
Erik Johnston d72f897f07 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor 
Conflicts:
	synapse/storage/stream.py
 2014-08-15 11:50:14 +01:00
Erik Johnston 5002efa31b Reimplement the get public rooms api to work with new DB schema 2014-08-15 10:26:35 +01:00
Erik Johnston 2c46bb6208 Fix up typos and correct sql queries 2014-08-14 18:40:50 +01:00
Erik Johnston 7e681ad778 Update StreamStore 2014-08-14 18:01:39 +01:00
Erik Johnston 661c711765 Start fixing places that use the data store. 2014-08-14 17:34:37 +01:00
Erik Johnston 78b501eba6 Fix typo 2014-08-14 17:09:28 +01:00
Erik Johnston 2529f2bc01 Rename _execute_query 2014-08-14 16:58:51 +01:00
Paul "LeoNerd" Evans 93a8be7bef We really don't need debug logging of all the SQL statements we execute; we're quite happy these all work now 2014-08-14 16:15:57 +01:00
Kegan Dougal fb93e14e53 Be more helpful when failing to register/login, stating why (communication error, user in user, wrong credentials, etc). Make the HS send M_USER_IN_USE. 2014-08-14 16:03:04 +01:00
Erik Johnston 937c175029 Fix up RoomMemberStore to work with the new schema. 2014-08-14 16:02:10 +01:00
Erik Johnston 6d6a1c3454 Actually encode dicts as json in the DB 2014-08-14 14:30:25 +01:00
Erik Johnston 10294b6082 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor 
Conflicts:
	synapse/storage/_base.py
 2014-08-14 10:01:04 +01:00
Paul "LeoNerd" Evans d05aa651f8 An initial hack at storing presence state-change mtimes in database and presenting age durations to clients/federation events 2014-08-13 19:19:15 +01:00
Paul "LeoNerd" Evans baf04be5cf Set datastore's .hs field in SQLBaseStore rather than in the toplevel DataStore mixed-in result class 2014-08-13 19:19:15 +01:00
Matthew Hodgson 59dfbaba3b when we're talking about backfilling data in federation, call it backfilling - not pagination. 2014-08-13 18:17:09 +01:00
Erik Johnston beaf4384d9 Make feedback table also store sender. 2014-08-13 18:03:41 +01:00
Erik Johnston 336987bb8d Initial stab at refactoring the SQL tables, including rejigging some of the storage layer. 2014-08-13 18:03:41 +01:00
Erik Johnston 3dfa84bec8 Convert im schema to a 'one' table structure 2014-08-13 18:03:41 +01:00
Kegan Dougal 9b3702c924 Return the room_alias when GETing public rooms. 2014-08-13 11:57:49 +01:00
Matthew Hodgson f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00