2a28b79e04
Batch sending of device list pokes
2017-03-24 14:44:49 +00:00
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
29ed09e80a
Fix assertion to stop transaction queue getting wedged
...
... and update some docstrings to correctly reflect the types being used.
get_new_device_msgs_for_remote can return a long under some circumstances,
which was being stored in last_device_list_stream_id_by_dest, and was then
upsetting things on the next loop.
2017-03-15 12:16:55 +00:00
29235901b8
Move FederationServer._handle_new_pdu to FederationHandler
...
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.
Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09 16:20:13 +00:00
e8b1721290
Move sig check out of _handle_new_pdu
...
When we receive PDUs via `get_missing_events`, we have already checked their
sigs, so there is no need to do it again.
2017-03-09 15:50:44 +00:00
3406333a58
Factor _get_missing_events_for_pdu out of _handle_new_pdu
...
This should be functionally identical: it just seeks to improve readability by
reducing indentation.
2017-03-09 15:50:44 +00:00
45d173a59a
Fix docstring
2017-03-09 15:50:29 +00:00
0c4cf9372b
Fix a race in transaction queue
...
It was theoretically possible for a PDU to get queued and not sent for ages. On
closer inspection I think there were bigger problems elsewhere, but we might as
well fix this since it's easy.
2017-02-20 16:46:25 +00:00
df4ecff5a9
Correctly raise exceptions for ratelimitng. Ratelimit on 401
2017-02-01 15:42:19 +00:00
ae7a132f38
Better handle 404 response for federation /send/
2017-01-31 13:40:09 +00:00
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
51e9fe36e4
Fix up sending of m.device_list_update edus
2017-01-25 16:55:21 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
fc7cae8aa3
Merge pull request #1824 from matrix-org/erikj/retry_host_log
...
Lower the not retrying host log line to debug
2017-01-18 11:07:51 +00:00
f878f64f43
Lower the not retrying host log line to debug
2017-01-17 17:20:39 +00:00
e6153e1bd1
Fix couple of federation state bugs
2017-01-17 13:22:34 +00:00
f7085ac84f
Name linearizer's for better logs
2017-01-09 17:17:10 +00:00
4304e7e593
do the discard check in the right place to avoid grabbing dependent events
2017-01-07 03:44:18 +00:00
e10c527930
Discard PDUs from invalid origins due to #1753 in 0.18.[56]
2017-01-07 02:13:14 +00:00
468749c9fc
fix comment
2017-01-05 12:00:11 +00:00
eedf400d05
limit total timeout for get_missing_events to 10s
2017-01-05 11:58:15 +00:00
f784980d2b
Only send events that originate on this server.
...
Or events that are sent via the federation "send_join" API.
This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
e02bdaf08b
Get the destinations from the state from before the event
...
Rather than the state after then event.
2017-01-04 15:17:15 +00:00
b6b67715ed
Send ALL membership events to the server that was affected.
...
Send all membership changes to the server that was affected.
This ensures that if the last member of a room on a server
was kicked or banned they get told about it.
2017-01-04 13:56:20 +00:00
62ce3034f3
s/aquire/acquire/g
2016-12-30 20:04:44 +00:00
0aff09f6c9
Add more useful logging when we block fetching events
2016-12-30 20:00:44 +00:00
cc50b1ae53
Remove fallback from get_missing_events.
...
get_missing_events used to fallback to fetching the missing events
individually requesting from every server in the room, one by one.e
This could be unacceptably slow, possibly causing #1732
2016-12-30 18:13:15 +00:00
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
aaecffba3a
Correctly handle 500's and 429 on federation
2016-11-24 15:04:49 +00:00
feec718265
Shuffle receipt handler around so that worker apps don't need to load it
2016-11-23 15:14:24 +00:00
4c79a63fd7
Explicit federation ack
2016-11-23 10:40:44 +00:00
51e89709aa
Comments
2016-11-21 17:59:39 +00:00
88d85ebae1
Add some metrics
2016-11-21 17:36:05 +00:00
50934ce460
Comments
2016-11-21 16:55:23 +00:00
9687e039e7
Remove explicit calls to send_pdu
2016-11-21 14:48:51 +00:00
524d61bf7e
Fix tests
2016-11-21 11:53:02 +00:00
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
ed787cf09e
Hook up the send queue and create a federation sender worker
2016-11-16 17:34:44 +00:00
1587b5a033
Add initial cut of federation send queue
2016-11-16 14:47:52 +00:00
59ef517e6b
Use new federation_sender DI
2016-11-16 14:47:52 +00:00
847d5db1d1
Add transaction queue and transport layer to DI
2016-11-16 14:47:52 +00:00
daec6fc355
Move logic into transaction_queue
2016-11-16 14:47:52 +00:00
0e830d3770
Rename transaction queue functions to send_*
2016-11-16 14:47:52 +00:00
22578545a0
Time out typing over federation
2016-09-23 14:00:52 +01:00
23b6701a28
Support filtering remote room lists
2016-09-16 10:24:15 +01:00
5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
76b09c29b0
Merge pull request #1110 from matrix-org/markjh/e2e_timeout
...
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
706b5d76ed
Fix backfill when cannot find an event.
...
`get_pdu` can succeed but return None.
2016-09-12 14:59:51 +01:00
af4701b311
Fix incorrect attribute name
2016-09-09 17:36:56 +01:00
464ffd1b5e
Comment
2016-09-09 17:17:23 +01:00
327425764e
Add edu.type as part of key. Remove debug logging
2016-09-09 17:13:30 +01:00
52b2318777
Clobber EDUs in send queue
2016-09-09 15:59:08 +01:00
ab80d5e0a9
Drop replication log levels
2016-09-09 14:56:50 +01:00
a6c6750166
Check if destination is ready for retry earlier
2016-09-09 13:46:05 +01:00
4598682b43
Fix tightloop on sending transaction
2016-09-09 13:12:53 +01:00
d2688d7f03
Correctly guard against multiple concurrent transactions
2016-09-09 11:44:36 +01:00
b390756150
Update last_device_stream_id_by_dest if there is nothing to send
2016-09-09 11:00:15 +01:00
43954d000e
Add a new method to enqueue the device messages rather than sending a dummy EDU
2016-09-07 16:10:51 +01:00
cb98ac261b
Move the check for federated device_messages.
...
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
2854ee2a52
Only pull out IDs from DB for /state_ids/ request
2016-09-02 10:53:36 +01:00
f4778d4cd9
Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into develop
2016-09-01 14:40:36 +01:00
516a272aca
Ensure we only return a validated pdu in get_pdu
2016-09-01 10:55:02 +01:00
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
c315922b5f
PEP8
2016-08-10 16:34:10 +01:00
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
5aeadb7414
Merge pull request #999 from matrix-org/erikj/measure_more
...
Measure federation send transaction resources
2016-08-10 14:16:14 +01:00
487bc49bf8
Don't stop on 4xx series errors
2016-08-10 13:39:12 +01:00
ea8c4094db
Also pull out rejected events
2016-08-10 13:26:13 +01:00
f91df1f761
Store if we fail to fetch an event from a destination
2016-08-10 11:31:46 +01:00
3bc9629be5
Measure federation send transaction resources
2016-08-10 10:56:38 +01:00
f5deaff424
Merge pull request #991 from matrix-org/erikj/retry_make
...
Retry joining via other servers if first one failed. Fix some other bugs.
2016-08-05 18:21:27 +01:00
5f360182c6
Fix a couple of python bugs
2016-08-05 18:08:32 +01:00
46453bfc2f
Retry joining via other servers if first one failed
2016-08-05 18:02:03 +01:00
24f36469bc
Add federation /version API
2016-08-05 16:36:07 +01:00
597c79be10
Change the way we specify if we require auth or not
2016-08-05 16:17:04 +01:00
2d4de61fb7
Fix typo
2016-08-05 10:48:56 +01:00
fccadb7719
Check if we already have the events returned by /state/
2016-08-05 10:43:47 +01:00
1515d1b581
Fallback to /state/ on both 400 and 404
2016-08-05 10:24:23 +01:00
a2b7102eea
Tweak integrity error recovery to work as intended
2016-08-04 20:38:08 -07:00
257c41cc2e
Fix typos.
2016-08-04 14:05:45 +01:00
b4e2290d89
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api
2016-08-04 14:04:35 +01:00
e3ee63578f
Tidy up get_events
2016-08-04 14:01:18 +01:00
edb33eb163
Rename fields to _ids
2016-08-03 17:19:15 +01:00
bcc9cda8ca
Fix copy + paste fails
2016-08-03 17:17:26 +01:00
4c56bedee3
Actually call get_room_state
2016-08-03 15:04:29 +01:00
520ee9bd2c
Fix syntax error
2016-08-03 15:03:15 +01:00
a60a2eaa02
Comment
2016-08-03 14:52:43 +01:00
e3a720217a
Add /state_ids federation API
...
The new API only returns the event_ids for the state, as most
requesters will already have the vast majority of the events already.
2016-08-03 14:47:37 +01:00
921f17f938
Merge branch 'develop' into rav/refactor_device_query
2016-08-03 11:12:47 +01:00
1efee2f52b
E2E keys: Make federation query share code with client query
...
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
c9154b970c
Don't double wrap 200
2016-08-02 16:45:53 +01:00
b3d5c4ad9d
Fix response cache
2016-08-02 16:42:21 +01:00
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
8f4a9bbc16
Linearize some federation endpoints based on (origin, room_id)
2016-06-17 16:43:45 +01:00
Erik Johnston