93d90765c4
Initial implementation of federation server rate limiting
2015-02-26 16:15:26 +00:00
59362454dd
Must update pending_transactions map before yield'ing
2015-02-26 15:47:35 +00:00
db215b7e00
Implement and use new batched get missing pdu
2015-02-23 13:58:02 +00:00
0ac2a79faa
Initial stab at implementing a batched get_missing_pdus request
2015-02-19 17:24:14 +00:00
a7925259a1
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1
2015-02-18 13:57:55 +00:00
446ef58992
Add errback to all deferreds in transaction_queue
2015-02-18 12:03:26 +00:00
2462aacd77
Restrict the destinations that synapse can talk to
2015-02-18 11:52:51 +00:00
b68e4a729f
Discard destination 'localhost'
2015-02-18 11:32:39 +00:00
47d3ff4cf8
Don't send failure to self
2015-02-18 11:30:37 +00:00
e482541e1d
Fix pyflakes
2015-02-18 10:44:22 +00:00
1be67eca8a
Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop
2015-02-18 10:34:40 +00:00
fe672a04f7
Merge pull request #77 from matrix-org/failures
...
Failures
2015-02-18 10:29:29 +00:00
08f804208b
Merge pull request #79 from matrix-org/get_pdu_limiting
...
Get pdu limiting
2015-02-18 10:29:10 +00:00
ec847059f3
Rename _fail_fetch_pdu_cache to _get_pdu_cache
2015-02-18 10:14:10 +00:00
9371019133
Try to only back off if we think we failed to connect to the remote
2015-02-17 18:13:34 +00:00
2b8f1a956c
Add per server retry limiting.
...
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
676e8ee78a
Remove debug raise
2015-02-17 15:22:45 +00:00
fa6c93bd26
Merge branch 'consumeErrors' of github.com:matrix-org/synapse into develop
2015-02-17 15:18:17 +00:00
c02da58a9d
Merge branch 'develop' of github.com:matrix-org/synapse into failures
2015-02-17 15:15:07 +00:00
659ead082f
Format the response of transaction request in a nicer way
2015-02-17 15:11:44 +00:00
c82e26ad4b
Actually respond with JSON to incoming transaction
2015-02-17 13:24:13 +00:00
02bfa889de
Handle recieving failures in transactions
2015-02-17 13:13:14 +00:00
72a4de2ce6
Use consumeErrors=True on all DeferredLists.
...
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
0194e71e99
Merge branch 'develop' of github.com:matrix-org/synapse into get_pdu_limiting
2015-02-17 09:48:23 +00:00
baa5b9a975
Cache results of get_pdu.
2015-02-16 18:02:39 +00:00
91fc5eef1d
Mark old events as outliers.
...
This is to fix the issue where if a remote server sends an event
that references a really "old" event, then the local server will pull
that in and send to all clients.
We decide if an event is old if its depth is less than the minimum depth
of the room.
2015-02-16 14:27:40 +00:00
6138584651
Don't return anything from _handle_new_pdu, since we ignore the return value anyway
2015-02-16 14:08:02 +00:00
789251afa7
Fix logging
2015-02-12 19:29:43 +00:00
58d848adc0
Parrellize fetching of events
2015-02-12 18:35:36 +00:00
963256638d
Correctly handle all the places that can throw exceptions
2015-02-12 18:17:11 +00:00
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
ef276e8770
Fix so timing out connections to actually work.
2015-02-11 16:48:05 +00:00
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
c8e1da930d
Log all the exits from _attempt_new_transaction
2015-02-10 17:30:46 +00:00
697ab75a34
Sign auth_chains when returned by /state/ requests
2015-02-10 15:46:24 +00:00
e9c85a4d5a
Connection errors in twisted aren't RuntimeErrors
2015-02-05 13:50:15 +00:00
e1515c3e91
Pass through list of room hosts from room alias query to federation so that it can retry against different room hosts
2015-02-05 13:44:42 +00:00
ae46f10fc5
Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations
2015-02-04 16:28:12 +00:00
ff78eded01
Retry make_join
2015-02-04 13:55:10 +00:00
3c39f42a05
New line
2015-02-03 16:14:19 +00:00
9bace3a367
Actually, the old prune_event function was non-deterministic, so no point keeping it around :(
2015-02-03 15:32:17 +00:00
8dae5c8108
Remove unused imports
2015-02-03 15:01:12 +00:00
7b810e136e
Add new FederationBase
2015-02-03 15:00:42 +00:00
0dd3aea319
Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers
2015-02-03 14:58:30 +00:00
0f48e22ef6
PEP8
2015-02-03 10:43:29 +00:00
40c6fe1b81
Don't bother requesting PDUs with bad signatures from the same server
2015-02-02 17:06:37 +00:00
941f59101b
Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator.
2015-02-02 16:56:01 +00:00
776ac820f9
Briefly doc structure of query_auth API.
2015-01-30 15:58:28 +00:00
a70a801184
Fix bug where we superfluously asked for current state. Change API of /query_auth/ so that we don't duplicate events in the response.
2015-01-30 13:34:01 +00:00
c1d860870b
Fix regression where we no longer correctly handled the case of gaps in our event graph
2015-01-30 10:48:47 +00:00
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
0ef5bfd6a9
Start implementing auth conflict res
2015-01-28 16:16:53 +00:00
c92d64a6c3
Make it the responsibility of the replication layer to check signature and hashes.
2015-01-26 14:33:11 +00:00
7b88619241
Split up replication_layer module into client, server and transaction queue
2015-01-26 10:45:24 +00:00
ca65a9d03e
Split out TransactionQueue from replication layer
2015-01-22 16:37:08 +00:00
5fed042640
Finish renaming "context" to "room_id" in federation codebase
2015-01-16 19:01:03 +00:00
2408c4b0a4
Fold _do_request_for_transaction into the methods that called it since it was a trivial wrapper around client.get_json
2015-01-16 19:01:03 +00:00
602684eac5
Split transport layer into client and server parts
2015-01-16 19:01:03 +00:00
0529a7e2e9
Add some logging for when we are sending transactions.
2015-01-06 14:06:25 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
041ac476a5
Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them
2014-12-18 18:47:13 +00:00
52f99243ab
Use is_outlier() so that we don't get AttributeError
2014-12-16 18:33:50 +00:00
3c77d13aa5
Kill off synapse.api.events.*
2014-12-16 11:29:05 +00:00
aae8a37e63
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
2014-12-10 13:18:40 +00:00
95aa903ffa
Try and figure out how and why signatures are being changed.
2014-12-10 11:37:47 +00:00
b8d30899b1
Code style.
2014-12-10 10:16:09 +00:00
71da2bed55
plateau retries after 1h
2014-12-10 00:18:44 +00:00
faf12b64f8
add errbacks to enqueue_pdu deferreds; change logging for failed federation sends to warn rather than exception
2014-12-10 00:12:51 +00:00
2b1acb7671
squidge to 79 columns as per pep8
2014-12-10 00:03:55 +00:00
8529fba02d
fix a million stupid bugs and make it actually work
2014-12-08 19:34:51 +00:00
ee3df06183
More bug fixes
2014-12-08 14:50:48 +00:00
d044121168
Various typos and bug fixes.
2014-12-08 09:08:26 +00:00
9c43b258ec
actually reset retry schedule if we can successfuly talk to it
2014-12-08 00:17:12 +00:00
5cd43d4b9f
fix stupid syntax thinkos
2014-12-07 23:44:16 +00:00
aed62a3583
track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover.
2014-12-07 02:26:07 +00:00
c31dba86ec
Convert rest and handlers to use new event structure
2014-12-04 15:50:01 +00:00
5d7c9ab789
Begin converting things to use the new Event structure
2014-12-04 11:27:59 +00:00
d06dfc70b0
Fix bug where did not always resolve all the deferreds in _attempt_new_transaction
2014-12-03 11:50:23 +00:00
07699b5871
Change the way we get missing auth and state events
2014-11-27 14:31:43 +00:00
3598c11c8d
Correctly handle the case where we get an event for an unknown room, which turns out we are actually in
2014-11-26 10:41:08 +00:00
64fc859dac
Fix bugs in invite/join dances.
...
We now do more implement more of the auth on the events so that we
don't reject valid events.
2014-11-25 17:59:49 +00:00
4961a4fab1
Mark the auth events as possible outlier
2014-11-24 13:55:49 +00:00
2bca242fdc
Ask for any auth events that we don't have
2014-11-24 13:46:41 +00:00
4bd0ab76c6
We don't always want to Auth get_persisted_pdu
2014-11-24 12:56:17 +00:00
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
32090aee16
Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts
2014-11-20 16:24:00 +00:00
4eada9a908
Fix backfill request
2014-11-19 17:22:37 +00:00
c5eabe3143
replace user_id with sender
2014-11-19 16:38:40 +00:00
428581dd05
SYN-144: Remove bad keys from pdu json objects, convert age_ts to age
...
for all pdus sent.
2014-11-18 19:20:25 +00:00
db7e8b5619
SYN-141: Decode the query params as UTF-8
2014-11-18 17:17:57 +00:00
cb4b6c844a
Merge PDUs and Events into one object
2014-11-14 21:25:02 +00:00
8d8a133c89
SYN-103: Remove "origin" and "destination" keys from edus
2014-11-13 15:49:03 +00:00
3db0efa69f
Fix pyflake warnings and add a FIXME comment to deal with auth_chains received when joining
2014-11-12 11:27:02 +00:00
6fea478d2e
Fix bugs with invites/joins across federatiom.
...
Both in terms of auth and not trying to fetch missing PDUs for invites,
joins etc.
2014-11-12 11:24:11 +00:00
092979b8cc
Fix bugs which broke federation due to changes in function signatures.
2014-11-11 14:19:13 +00:00
5d439b127b
PEP8
2014-11-10 13:46:44 +00:00
c46088405a
Remove useless comments
2014-11-10 13:39:33 +00:00
003668cfaa
Add auth to the various server-server APIs
2014-11-10 13:37:24 +00:00
6447db063a
Fix backfill to work. Add auth to backfill request
2014-11-10 11:59:51 +00:00
1c06806f90
Finish redaction algorithm.
2014-11-10 10:21:32 +00:00
02c3b1c9e2
Add '/event_auth/' federation api
2014-11-07 15:35:53 +00:00
d2fb2b8095
Implement invite part of invite join dance
2014-11-07 13:41:00 +00:00
328dab2463
Remove /context/ request
2014-11-07 11:40:38 +00:00
3b4dec442d
Return auth chain when handling send_join
2014-11-07 11:22:12 +00:00
a5a4ef3fd7
Fix bug in replication
2014-11-04 15:16:43 +00:00
fc7b2b11a2
PEP8
2014-11-04 15:09:34 +00:00
440cbd5235
Add support for sending failures
2014-11-04 14:17:55 +00:00
d7412c4df1
Remove unused interface
2014-11-04 14:16:19 +00:00
aa76bf39ab
Remove unused imports
2014-11-04 14:14:02 +00:00
68698e0ac8
Fix bugs in generating event signatures and hashing
2014-11-03 17:51:42 +00:00
ad6eacb3e9
Rename PDU fields to match that of events.
2014-11-03 13:06:58 +00:00
d59aa6af25
For now, don't store txn -> pdu mappings.
2014-11-03 11:35:19 +00:00
2f39dc19a2
Remove more references to dead PDU tables
2014-10-31 14:27:14 +00:00
bfa36a72b9
Remove PDU tables.
2014-10-31 14:00:32 +00:00
21fe249d62
Actually don't store any PDUs
2014-10-31 10:47:34 +00:00
841df4da71
Don't store any PDUs
2014-10-31 09:59:59 +00:00
f2de2d644a
Move the impl of backfill to use events.
2014-10-31 09:59:02 +00:00
12ce441e67
Convert event ids to be of the form :example.com
2014-10-30 17:00:11 +00:00
e7858b6d7e
Start filling out and using new events tables
2014-10-29 16:59:24 +00:00
2d1dfb3b34
Begin implementing all the PDU storage stuff in Events land
2014-10-28 16:42:35 +00:00
ad9226eeec
Merge branch 'event_signing' of github.com:matrix-org/synapse into federation_authorization
...
Conflicts:
synapse/storage/__init__.py
2014-10-27 11:58:32 +00:00
5e2236f9ff
fix pyflakes warnings
2014-10-27 11:19:15 +00:00
acb2d171e8
Merge branch 'develop' into event_signing
2014-10-27 11:14:11 +00:00
bb4a20174c
Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
...
Conflicts:
synapse/federation/transport.py
synapse/handlers/message.py
2014-10-27 10:20:44 +00:00
ac9345b47a
Check that we have auth headers and fail nicely
2014-10-17 21:00:58 +01:00
cd198dfea8
More log lines.
2014-10-17 20:58:47 +01:00
8afbece683
Remove signatures from pdu when computing hashes to use for prev pdus, make sure is_state is a boolean.
2014-10-17 19:41:32 +01:00
5ffe5ab43f
Use state groups to get current state. Make join dance actually work.
2014-10-17 18:56:42 +01:00
dc3c2823ac
Merge branch 'develop' into event_signing
...
Conflicts:
synapse/federation/replication.py
2014-10-17 17:33:58 +01:00
82c5820767
keep 'origin_server_ts' as 'ts' in the database to avoid needlessly updating schema
2014-10-17 17:31:48 +01:00
f5cf7ac25b
SPEC-7: Rename 'ts' to 'origin_server_ts'
2014-10-17 17:12:25 +01:00
c5cec1cc77
Rename 'meta' to 'unsigned'
2014-10-17 16:50:04 +01:00
4d1a7624f4
move 'age' into 'meta' subdict so that it is clearer that it is not part of the signed data
2014-10-17 15:27:11 +01:00
f71627567b
Finish implementing the new join dance.
2014-10-17 15:04:17 +01:00
c8f996e29f
Hash the same content covered by the signature when referencing previous PDUs rather than reusing the PDU content hashes
2014-10-17 11:40:35 +01:00
bb04447c44
Include hashes of previous pdus when referencing them
2014-10-16 23:25:12 +01:00
1116f5330e
Start implementing the invite/join dance. Continue moving auth to use event.state_events
2014-10-16 16:56:51 +01:00
66104da10c
Sign outgoing PDUs.
2014-10-16 00:09:48 +01:00
1c445f88f6
persist hashes and origin signatures for PDUs
2014-10-15 17:09:04 +01:00
9aed791fc3
SYN-103: Ignore the 'origin' key in received EDUs. Instead take the origin from the transaction itself
2014-10-14 16:44:27 +01:00
07639c79d9
Respond with more helpful error messages for unsigned requests
2014-10-13 16:39:15 +01:00
25d80f35f1
Raise a SynapseError if the authorisation header is missing or malformed
2014-10-13 15:53:18 +01:00
75e517a2da
Remove debug logging, raise a proper SynapseError if the auth header is missing
2014-10-13 15:41:20 +01:00
6684855767
Verify signatures for server2server requests
2014-10-13 14:37:46 +01:00
10ef8e6e4b
SYN-75 sign at the request level rather than the transaction level
2014-10-13 11:49:55 +01:00
984e207b59
Merge branch develop into server2server_signing
...
Conflicts:
synapse/app/homeserver.py
2014-10-13 10:58:50 +01:00
574377636e
Add a keyword argument to get_json to avoid retrying on DNS failures. Rather than passing MatrixHttpClient.RETRY_DNS_LOOKUP_FAILURES as a fake query string parameter
2014-10-02 14:26:13 +01:00
b95a178584
SYN-75 Verify signatures on server to server transactions
2014-09-30 15:15:10 +01:00
52ca867670
Sign federation transactions
2014-09-24 17:25:41 +01:00
40d2f38abe
Fix bug where we incorrectly calculated 'age_ts' from 'age' key rather than the reverse. Don't transmit age_ts to clients for now.
2014-09-15 16:55:39 +01:00
6ac0b4ade8
Fix 'age' key to update on retries
2014-09-15 16:24:03 +01:00
5bd9369a62
Correctly handle the 'age' key in events and pdus
2014-09-15 13:26:11 +01:00
39e3fc69e5
Make the state resolution use actual power levels rather than taking them from a Pdu key.
2014-09-12 17:11:09 +01:00
8a7c1d6a00
fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch.
2014-09-03 17:31:57 +01:00
86d6232236
Don't eat federation transmit errors during unit tests; fix remote presence EDU-sending test because of this
2014-09-03 13:46:52 +01:00
b8ab9f1c0a
Add all the necessary checks to make banning work.
2014-09-01 18:24:56 +01:00
5dd38d579b
Fix a couple of bugs in presence handler related to pushing updatesto the correct user. Fix presence tests.
2014-08-29 12:08:33 +01:00
b1da3fa0a7
Avoid AlreadyCalledError from EDU sending failures
2014-08-28 16:19:16 +01:00
474dcecb11
Remove unused populate_previous_pdus
2014-08-27 11:34:31 +01:00
d2798de660
Fold federation/handler into handlers/federation
2014-08-26 19:49:42 +01:00
6966971a28
Use store.persist_event rather than pdu_actions.persist_outgoing/pdu_actions.persist_received
2014-08-26 18:57:17 +01:00
64e2a5d58e
Move pdu and event persistence into a single persist_event function
2014-08-26 18:01:36 +01:00
4b63b06cad
Merge branch 'develop' into storage_transactions
...
Conflicts:
synapse/api/auth.py
synapse/handlers/room.py
synapse/storage/__init__.py
2014-08-26 16:07:05 +01:00
2c4908ed26
Ensure that we don't have duplicate hosts in the pdu destinations list
2014-08-24 14:35:13 +01:00
1379dcae6f
Take a snapshot of the state of the room before performing updates
2014-08-22 17:00:10 +01:00
347242a5c4
Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor
...
Conflicts:
tests/rest/test_presence.py
tests/rest/test_rooms.py
tests/utils.py
2014-08-19 14:48:19 +01:00
992782b9f5
Ensure that federation's .send_edu() returns a Deferred
2014-08-19 14:26:06 +01:00
75b6d982a0
Add a 'backfill room' button
2014-08-19 14:20:03 +01:00
fc26275bb3
Add two different columns for ordering the events table, one which can be used for pagination and one which can be as tokens for notifying clients. Also add a 'processed' field which is currently always set to True
2014-08-18 15:50:41 +01:00
c75add6ec8
Added a urls module for keeping client and federation prefixes.
2014-08-14 11:52:56 +01:00
29aa13f0d4
Make federation use resource_for_federation as well.
2014-08-14 10:05:06 +01:00
59dfbaba3b
when we're talking about backfilling data in federation, call it backfilling - not pagination.
2014-08-13 18:17:09 +01:00
827de7cee9
Define the concept of a 'federation Query'; creating API for making and handling Queries on the Federation's increasingly-inaccurately-named ReplicationLayer
2014-08-13 17:23:49 +01:00
820ed34abe
Namespace all the Federation HTTP URLs to /matrix/federation/v1/...
2014-08-13 15:07:55 +01:00
f98e6380f1
add in whitespace after copyright statements to improve legibility
2014-08-13 03:14:34 +01:00
4f475c7697
Reference Matrix Home Server
2014-08-12 15:10:52 +01:00
Erik Johnston