Commit Graph

1151 Commits

Author SHA1 Message Date
Erik Johnston 6138584651 Don't return anything from _handle_new_pdu, since we ignore the return value anyway 2015-02-16 14:08:02 +00:00
Erik Johnston 789251afa7 Fix logging 2015-02-12 19:29:43 +00:00
Erik Johnston 58d848adc0 Parrellize fetching of events 2015-02-12 18:35:36 +00:00
Erik Johnston 963256638d Correctly handle all the places that can throw exceptions 2015-02-12 18:17:11 +00:00
Erik Johnston c52e8d395b Merge pull request #61 from matrix-org/timeout-federation-requests
Timeout federation requests
2015-02-11 17:10:33 +00:00
Erik Johnston ef276e8770 Fix so timing out connections to actually work. 2015-02-11 16:48:05 +00:00
Erik Johnston ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00
Erik Johnston 4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Erik Johnston c8e1da930d Log all the exits from _attempt_new_transaction 2015-02-10 17:30:46 +00:00
Erik Johnston 697ab75a34 Sign auth_chains when returned by /state/ requests 2015-02-10 15:46:24 +00:00
Erik Johnston e9c85a4d5a Connection errors in twisted aren't RuntimeErrors 2015-02-05 13:50:15 +00:00
Erik Johnston e1515c3e91 Pass through list of room hosts from room alias query to federation so that it can retry against different room hosts 2015-02-05 13:44:42 +00:00
Erik Johnston ae46f10fc5 Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations 2015-02-04 16:28:12 +00:00
Erik Johnston ff78eded01 Retry make_join 2015-02-04 13:55:10 +00:00
Erik Johnston 3c39f42a05 New line 2015-02-03 16:14:19 +00:00
Erik Johnston 9bace3a367 Actually, the old prune_event function was non-deterministic, so no point keeping it around :( 2015-02-03 15:32:17 +00:00
Erik Johnston 8dae5c8108 Remove unused imports 2015-02-03 15:01:12 +00:00
Erik Johnston 7b810e136e Add new FederationBase 2015-02-03 15:00:42 +00:00
Erik Johnston 0dd3aea319 Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers 2015-02-03 14:58:30 +00:00
Erik Johnston 0f48e22ef6 PEP8 2015-02-03 10:43:29 +00:00
Erik Johnston 40c6fe1b81 Don't bother requesting PDUs with bad signatures from the same server 2015-02-02 17:06:37 +00:00
Erik Johnston 941f59101b Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator. 2015-02-02 16:56:01 +00:00
Erik Johnston 776ac820f9 Briefly doc structure of query_auth API. 2015-01-30 15:58:28 +00:00
Erik Johnston a70a801184 Fix bug where we superfluously asked for current state. Change API of /query_auth/ so that we don't duplicate events in the response. 2015-01-30 13:34:01 +00:00
Erik Johnston c1d860870b Fix regression where we no longer correctly handled the case of gaps in our event graph 2015-01-30 10:48:47 +00:00
Erik Johnston 78015948a7 Initial implementation of auth conflict resolution 2015-01-29 16:52:33 +00:00
Erik Johnston 0ef5bfd6a9 Start implementing auth conflict res 2015-01-28 16:16:53 +00:00
Erik Johnston c92d64a6c3 Make it the responsibility of the replication layer to check signature and hashes. 2015-01-26 14:33:11 +00:00
Erik Johnston 7b88619241 Split up replication_layer module into client, server and transaction queue 2015-01-26 10:45:24 +00:00
Erik Johnston ca65a9d03e Split out TransactionQueue from replication layer 2015-01-22 16:37:08 +00:00
Mark Haines 5fed042640 Finish renaming "context" to "room_id" in federation codebase 2015-01-16 19:01:03 +00:00
Mark Haines 2408c4b0a4 Fold _do_request_for_transaction into the methods that called it since it was a trivial wrapper around client.get_json 2015-01-16 19:01:03 +00:00
Mark Haines 602684eac5 Split transport layer into client and server parts 2015-01-16 19:01:03 +00:00
Matrix 0529a7e2e9 Add some logging for when we are sending transactions. 2015-01-06 14:06:25 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Mark Haines 041ac476a5 Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them 2014-12-18 18:47:13 +00:00
Erik Johnston 52f99243ab Use is_outlier() so that we don't get AttributeError 2014-12-16 18:33:50 +00:00
Erik Johnston 3c77d13aa5 Kill off synapse.api.events.* 2014-12-16 11:29:05 +00:00
Erik Johnston aae8a37e63 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-10 13:18:40 +00:00
Erik Johnston 95aa903ffa Try and figure out how and why signatures are being changed. 2014-12-10 11:37:47 +00:00
Erik Johnston b8d30899b1 Code style. 2014-12-10 10:16:09 +00:00
Matthew Hodgson 71da2bed55 plateau retries after 1h 2014-12-10 00:18:44 +00:00
Matthew Hodgson faf12b64f8 add errbacks to enqueue_pdu deferreds; change logging for failed federation sends to warn rather than exception 2014-12-10 00:12:51 +00:00
Matthew Hodgson 2b1acb7671 squidge to 79 columns as per pep8 2014-12-10 00:03:55 +00:00
Matthew Hodgson 8529fba02d fix a million stupid bugs and make it actually work 2014-12-08 19:34:51 +00:00
Erik Johnston ee3df06183 More bug fixes 2014-12-08 14:50:48 +00:00
Erik Johnston d044121168 Various typos and bug fixes. 2014-12-08 09:08:26 +00:00
Matthew Hodgson 9c43b258ec actually reset retry schedule if we can successfuly talk to it 2014-12-08 00:17:12 +00:00
Matthew Hodgson 5cd43d4b9f fix stupid syntax thinkos 2014-12-07 23:44:16 +00:00
Matthew Hodgson aed62a3583 track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover. 2014-12-07 02:26:07 +00:00
Erik Johnston c31dba86ec Convert rest and handlers to use new event structure 2014-12-04 15:50:01 +00:00
Erik Johnston 5d7c9ab789 Begin converting things to use the new Event structure 2014-12-04 11:27:59 +00:00
Erik Johnston d06dfc70b0 Fix bug where did not always resolve all the deferreds in _attempt_new_transaction 2014-12-03 11:50:23 +00:00
Erik Johnston 07699b5871 Change the way we get missing auth and state events 2014-11-27 14:31:43 +00:00
Erik Johnston 3598c11c8d Correctly handle the case where we get an event for an unknown room, which turns out we are actually in 2014-11-26 10:41:08 +00:00
Erik Johnston 64fc859dac Fix bugs in invite/join dances.
We now do more implement more of the auth on the events so that we
don't reject valid events.
2014-11-25 17:59:49 +00:00
Erik Johnston 4961a4fab1 Mark the auth events as possible outlier 2014-11-24 13:55:49 +00:00
Erik Johnston 2bca242fdc Ask for any auth events that we don't have 2014-11-24 13:46:41 +00:00
Erik Johnston 4bd0ab76c6 We don't always want to Auth get_persisted_pdu 2014-11-24 12:56:17 +00:00
Mark Haines db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines 32090aee16 Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts 2014-11-20 16:24:00 +00:00
Erik Johnston 4eada9a908 Fix backfill request 2014-11-19 17:22:37 +00:00
Mark Haines c5eabe3143 replace user_id with sender 2014-11-19 16:38:40 +00:00
Mark Haines 428581dd05 SYN-144: Remove bad keys from pdu json objects, convert age_ts to age
for all pdus sent.
2014-11-18 19:20:25 +00:00
Mark Haines db7e8b5619 SYN-141: Decode the query params as UTF-8 2014-11-18 17:17:57 +00:00
Mark Haines cb4b6c844a Merge PDUs and Events into one object 2014-11-14 21:25:02 +00:00
Mark Haines 8d8a133c89 SYN-103: Remove "origin" and "destination" keys from edus 2014-11-13 15:49:03 +00:00
Erik Johnston 3db0efa69f Fix pyflake warnings and add a FIXME comment to deal with auth_chains received when joining 2014-11-12 11:27:02 +00:00
Erik Johnston 6fea478d2e Fix bugs with invites/joins across federatiom.
Both in terms of auth and not trying to fetch missing PDUs for invites,
joins etc.
2014-11-12 11:24:11 +00:00
Erik Johnston 092979b8cc Fix bugs which broke federation due to changes in function signatures. 2014-11-11 14:19:13 +00:00
Erik Johnston 5d439b127b PEP8 2014-11-10 13:46:44 +00:00
Erik Johnston c46088405a Remove useless comments 2014-11-10 13:39:33 +00:00
Erik Johnston 003668cfaa Add auth to the various server-server APIs 2014-11-10 13:37:24 +00:00
Erik Johnston 6447db063a Fix backfill to work. Add auth to backfill request 2014-11-10 11:59:51 +00:00
Erik Johnston 1c06806f90 Finish redaction algorithm. 2014-11-10 10:21:32 +00:00
Erik Johnston 02c3b1c9e2 Add '/event_auth/' federation api 2014-11-07 15:35:53 +00:00
Erik Johnston d2fb2b8095 Implement invite part of invite join dance 2014-11-07 13:41:00 +00:00
Erik Johnston 328dab2463 Remove /context/ request 2014-11-07 11:40:38 +00:00
Erik Johnston 3b4dec442d Return auth chain when handling send_join 2014-11-07 11:22:12 +00:00
Erik Johnston a5a4ef3fd7 Fix bug in replication 2014-11-04 15:16:43 +00:00
Erik Johnston fc7b2b11a2 PEP8 2014-11-04 15:09:34 +00:00
Erik Johnston 440cbd5235 Add support for sending failures 2014-11-04 14:17:55 +00:00
Erik Johnston d7412c4df1 Remove unused interface 2014-11-04 14:16:19 +00:00
Erik Johnston aa76bf39ab Remove unused imports 2014-11-04 14:14:02 +00:00
Erik Johnston 68698e0ac8 Fix bugs in generating event signatures and hashing 2014-11-03 17:51:42 +00:00
Erik Johnston ad6eacb3e9 Rename PDU fields to match that of events. 2014-11-03 13:06:58 +00:00
Erik Johnston d59aa6af25 For now, don't store txn -> pdu mappings. 2014-11-03 11:35:19 +00:00
Erik Johnston 2f39dc19a2 Remove more references to dead PDU tables 2014-10-31 14:27:14 +00:00
Erik Johnston bfa36a72b9 Remove PDU tables. 2014-10-31 14:00:32 +00:00
Erik Johnston 21fe249d62 Actually don't store any PDUs 2014-10-31 10:47:34 +00:00
Erik Johnston 841df4da71 Don't store any PDUs 2014-10-31 09:59:59 +00:00
Erik Johnston f2de2d644a Move the impl of backfill to use events. 2014-10-31 09:59:02 +00:00
Erik Johnston 12ce441e67 Convert event ids to be of the form :example.com 2014-10-30 17:00:11 +00:00
Erik Johnston e7858b6d7e Start filling out and using new events tables 2014-10-29 16:59:24 +00:00
Erik Johnston 2d1dfb3b34 Begin implementing all the PDU storage stuff in Events land 2014-10-28 16:42:35 +00:00
Erik Johnston ad9226eeec Merge branch 'event_signing' of github.com:matrix-org/synapse into federation_authorization
Conflicts:
	synapse/storage/__init__.py
2014-10-27 11:58:32 +00:00
Mark Haines 5e2236f9ff fix pyflakes warnings 2014-10-27 11:19:15 +00:00
Mark Haines acb2d171e8 Merge branch 'develop' into event_signing 2014-10-27 11:14:11 +00:00
Erik Johnston bb4a20174c Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
Conflicts:
	synapse/federation/transport.py
	synapse/handlers/message.py
2014-10-27 10:20:44 +00:00
Erik Johnston ac9345b47a Check that we have auth headers and fail nicely 2014-10-17 21:00:58 +01:00
Erik Johnston cd198dfea8 More log lines. 2014-10-17 20:58:47 +01:00
Mark Haines 8afbece683 Remove signatures from pdu when computing hashes to use for prev pdus, make sure is_state is a boolean. 2014-10-17 19:41:32 +01:00
Erik Johnston 5ffe5ab43f Use state groups to get current state. Make join dance actually work. 2014-10-17 18:56:42 +01:00
Mark Haines dc3c2823ac Merge branch 'develop' into event_signing
Conflicts:
	synapse/federation/replication.py
2014-10-17 17:33:58 +01:00
Mark Haines 82c5820767 keep 'origin_server_ts' as 'ts' in the database to avoid needlessly updating schema 2014-10-17 17:31:48 +01:00
Mark Haines f5cf7ac25b SPEC-7: Rename 'ts' to 'origin_server_ts' 2014-10-17 17:12:25 +01:00
Mark Haines c5cec1cc77 Rename 'meta' to 'unsigned' 2014-10-17 16:50:04 +01:00
Mark Haines 4d1a7624f4 move 'age' into 'meta' subdict so that it is clearer that it is not part of the signed data 2014-10-17 15:27:11 +01:00
Erik Johnston f71627567b Finish implementing the new join dance. 2014-10-17 15:04:17 +01:00
Mark Haines c8f996e29f Hash the same content covered by the signature when referencing previous PDUs rather than reusing the PDU content hashes 2014-10-17 11:40:35 +01:00
Mark Haines bb04447c44 Include hashes of previous pdus when referencing them 2014-10-16 23:25:12 +01:00
Erik Johnston 1116f5330e Start implementing the invite/join dance. Continue moving auth to use event.state_events 2014-10-16 16:56:51 +01:00
Mark Haines 66104da10c Sign outgoing PDUs. 2014-10-16 00:09:48 +01:00
Mark Haines 1c445f88f6 persist hashes and origin signatures for PDUs 2014-10-15 17:09:04 +01:00
Mark Haines 9aed791fc3 SYN-103: Ignore the 'origin' key in received EDUs. Instead take the origin from the transaction itself 2014-10-14 16:44:27 +01:00
Mark Haines 07639c79d9 Respond with more helpful error messages for unsigned requests 2014-10-13 16:39:15 +01:00
Mark Haines 25d80f35f1 Raise a SynapseError if the authorisation header is missing or malformed 2014-10-13 15:53:18 +01:00
Mark Haines 75e517a2da Remove debug logging, raise a proper SynapseError if the auth header is missing 2014-10-13 15:41:20 +01:00
Mark Haines 6684855767 Verify signatures for server2server requests 2014-10-13 14:37:46 +01:00
Mark Haines 10ef8e6e4b SYN-75 sign at the request level rather than the transaction level 2014-10-13 11:49:55 +01:00
Mark Haines 984e207b59 Merge branch develop into server2server_signing
Conflicts:
	synapse/app/homeserver.py
2014-10-13 10:58:50 +01:00
Mark Haines 574377636e Add a keyword argument to get_json to avoid retrying on DNS failures. Rather than passing MatrixHttpClient.RETRY_DNS_LOOKUP_FAILURES as a fake query string parameter 2014-10-02 14:26:13 +01:00
Mark Haines b95a178584 SYN-75 Verify signatures on server to server transactions 2014-09-30 15:15:10 +01:00
Mark Haines 52ca867670 Sign federation transactions 2014-09-24 17:25:41 +01:00
Erik Johnston 40d2f38abe Fix bug where we incorrectly calculated 'age_ts' from 'age' key rather than the reverse. Don't transmit age_ts to clients for now. 2014-09-15 16:55:39 +01:00
Erik Johnston 6ac0b4ade8 Fix 'age' key to update on retries 2014-09-15 16:24:03 +01:00
Erik Johnston 5bd9369a62 Correctly handle the 'age' key in events and pdus 2014-09-15 13:26:11 +01:00
Erik Johnston 39e3fc69e5 Make the state resolution use actual power levels rather than taking them from a Pdu key. 2014-09-12 17:11:09 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans 86d6232236 Don't eat federation transmit errors during unit tests; fix remote presence EDU-sending test because of this 2014-09-03 13:46:52 +01:00
Erik Johnston b8ab9f1c0a Add all the necessary checks to make banning work. 2014-09-01 18:24:56 +01:00
Erik Johnston 5dd38d579b Fix a couple of bugs in presence handler related to pushing updatesto the correct user. Fix presence tests. 2014-08-29 12:08:33 +01:00
Paul "LeoNerd" Evans b1da3fa0a7 Avoid AlreadyCalledError from EDU sending failures 2014-08-28 16:19:16 +01:00
Mark Haines 474dcecb11 Remove unused populate_previous_pdus 2014-08-27 11:34:31 +01:00
Mark Haines d2798de660 Fold federation/handler into handlers/federation 2014-08-26 19:49:42 +01:00
Mark Haines 6966971a28 Use store.persist_event rather than pdu_actions.persist_outgoing/pdu_actions.persist_received 2014-08-26 18:57:17 +01:00
Mark Haines 64e2a5d58e Move pdu and event persistence into a single persist_event function 2014-08-26 18:01:36 +01:00
Mark Haines 4b63b06cad Merge branch 'develop' into storage_transactions
Conflicts:
	synapse/api/auth.py
	synapse/handlers/room.py
	synapse/storage/__init__.py
2014-08-26 16:07:05 +01:00
Erik Johnston 2c4908ed26 Ensure that we don't have duplicate hosts in the pdu destinations list 2014-08-24 14:35:13 +01:00
Mark Haines 1379dcae6f Take a snapshot of the state of the room before performing updates 2014-08-22 17:00:10 +01:00
Erik Johnston 347242a5c4 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor
Conflicts:
	tests/rest/test_presence.py
	tests/rest/test_rooms.py
	tests/utils.py
2014-08-19 14:48:19 +01:00
Paul "LeoNerd" Evans 992782b9f5 Ensure that federation's .send_edu() returns a Deferred 2014-08-19 14:26:06 +01:00
Erik Johnston 75b6d982a0 Add a 'backfill room' button 2014-08-19 14:20:03 +01:00
Erik Johnston fc26275bb3 Add two different columns for ordering the events table, one which can be used for pagination and one which can be as tokens for notifying clients. Also add a 'processed' field which is currently always set to True 2014-08-18 15:50:41 +01:00
Kegan Dougal c75add6ec8 Added a urls module for keeping client and federation prefixes. 2014-08-14 11:52:56 +01:00
Kegan Dougal 29aa13f0d4 Make federation use resource_for_federation as well. 2014-08-14 10:05:06 +01:00
Matthew Hodgson 59dfbaba3b when we're talking about backfilling data in federation, call it backfilling - not pagination. 2014-08-13 18:17:09 +01:00
Paul "LeoNerd" Evans 827de7cee9 Define the concept of a 'federation Query'; creating API for making and handling Queries on the Federation's increasingly-inaccurately-named ReplicationLayer 2014-08-13 17:23:49 +01:00
Paul "LeoNerd" Evans 820ed34abe Namespace all the Federation HTTP URLs to /matrix/federation/v1/... 2014-08-13 15:07:55 +01:00
Matthew Hodgson f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00