660b77f362
Add missing docstring detail
2019-03-13 18:38:16 +00:00
5526b054aa
Fix syntax issues
2019-03-13 18:35:21 +00:00
09626bfd39
Switch to wrapper function around _send_request
2019-03-13 18:26:06 +00:00
7d053cfe10
Retry on 400:M_UNRECOGNIZED
2019-03-13 12:10:33 +00:00
7e75d9644b
Fix paranthesis indent
2019-03-13 11:15:23 +00:00
cf301e37d8
Add workaround note
2019-03-13 11:14:43 +00:00
0ea8582f8b
Cleaner way of implementing trailing slashes
2019-03-12 14:11:11 +00:00
64ff11019e
Retry certain federation requests on 404
2019-03-08 18:22:47 +00:00
c88bc53903
Fix TaskStopped exceptions when outbound requests time out ( #4690 )
2019-02-20 11:35:52 +00:00
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates ( #4611 )
...
It's nothing to do with refreshing the certificates. No idea why it was here.
2019-02-11 18:03:30 +00:00
f0ba34f581
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
...
Fixes #4003
2019-02-01 12:22:57 +00:00
8520bc3109
Fix Host header sent by MatrixFederationAgent ( #4468 )
...
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
7871146667
Make MatrixFederationClient use MatrixFederationAgent
...
... instead of the matrix_federation_endpoint
2019-01-22 20:34:35 +00:00
de6888e7ce
Remove redundant WrappedConnection ( #4409 )
...
* Remove redundant WrappedConnection
The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
2019-01-18 23:07:38 +11:00
9feb5d0b71
sign_request -> build_auth_headers ( #4408 )
...
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
34ea14139d
Fixup docstrings for matrixfederationclient
2019-01-09 09:25:59 +00:00
d422570e9b
Use RequestSendFailed when fail to parse content type headers
2019-01-08 14:22:18 +00:00
b970cb0e96
Refactor request sending to have better excpetions ( #4358 )
...
* Correctly retry and back off if we get a HTTPerror response
* Refactor request sending to have better excpetions
MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).
This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
...
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
b8a5b0097c
Various cleanups in the federation client code ( #4031 )
...
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
219606a6ed
Fix exception documentation in matrixfederationclient.py
2018-09-26 13:26:27 -06:00
a334e1cace
Update to use new timeout function everywhere.
...
The existing deferred timeout helper function (and the one into twisted)
suffer from a bug when a deferred's canceller throws an exception, #3842 .
The new helper function doesn't suffer from this problem.
2018-09-19 10:39:40 +01:00
35aec19f0a
Destination is a string
2018-09-18 15:29:30 -06:00
31c15dcb80
Refactor matrixfederationclient to fix logging ( #3906 )
...
We want to wait until we have read the response body before we log the request
as complete, otherwise a confusing thing happens where the request appears to
have completed, but we later fail it.
To do this, we factor the salient details of a request out to a separate
object, which can then keep track of the txn_id, so that it can be logged.
2018-09-18 18:17:15 +01:00
f00a9d2636
Fix some b'abcd' noise in logs and metrics
...
Python 3 compatibility: make sure that we decode some byte sequences before we
use them to create log lines and metrics labels.
2018-09-17 16:15:42 +01:00
fcfe7a850d
Add an awful secondary timeout to fix wedged requests
...
This is an attempt to mitigate #3842 by adding yet-another-timeout
2018-09-14 19:23:07 +01:00
3e6e94fe9f
Merge pull request #3872 from matrix-org/hawkowl/timeouts-2
...
timeouts 2: electric boogaloo
2018-09-14 16:58:44 +01:00
9e2f9a7b57
Measure outbound requests
2018-09-14 15:11:26 +01:00
63755fa4c2
we do that higher up
2018-09-14 03:21:47 +10:00
1c3f4d9ca5
buffer?
2018-09-14 03:09:13 +10:00
bfa0b759e0
Attempt to figure out what's going on with timeouts ( #3857 )
2018-09-14 00:15:51 +10:00
8c5b84441b
Log outbound requests when we retry
2018-09-12 16:22:14 +01:00
4084a774a8
Timeout reading body for outbound HTTP requests
2018-09-12 10:10:20 +01:00
2d2828dcbc
Port http/ to Python 3 ( #3771 )
2018-09-06 00:10:47 +10:00
1d67b13674
Fix bug when federation_domain_whitelist is an emtpy list
...
Outbound federation were incorrectly allowed when the config option was
set to an empty list
2018-08-24 12:13:12 +01:00
b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
3c0213a217
Merge pull request #3439 from vojeroen/send_sni_for_federation_requests
...
send SNI for federation requests
2018-08-10 12:23:54 +01:00
f900d50824
include known room versions in outgoing make_joins
2018-08-06 13:45:37 +01:00
505530f36a
Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests
...
# Conflicts:
# synapse/crypto/context_factory.py
2018-07-14 20:24:46 +02:00
b5e157d895
Merge branch 'develop' into send_sni_for_federation_requests
...
# Conflicts:
# synapse/http/endpoint.py
2018-07-09 08:51:11 +02:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
07b4f88de9
formatting changes for pep8
2018-06-25 12:31:16 +02:00
3d605853c8
send SNI for federation requests
2018-06-24 22:38:43 +02:00
77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
187a546bff
Merge pull request #3276 from matrix-org/dbkr/unbind
...
Remove email addresses / phone numbers from ID servers when they're removed from synapse
2018-06-11 16:02:00 +01:00
bf54c1cf6c
pep8
2018-06-06 10:15:33 +01:00
3e4bc4488c
More doc fixes
2018-06-06 09:44:10 +01:00
607bd27c83
fix pep8
2018-06-05 18:10:35 +01:00
d62162bbec
doc fixes
2018-06-05 18:09:13 +01:00
c5930d513a
Docstring
2018-06-04 12:05:58 +01:00
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
9e2601f830
Merge pull request #3108 from NotAFile/py3-six-urlparse
...
Use six.moves.urlparse
2018-04-30 00:33:05 +01:00
9d2c1b8429
Backport deferred.addTimeout
...
Twisted 16.0 doesn't have addTimeout, so let's backport it.
2018-04-27 12:52:30 +01:00
1ea904b9f0
Use deferred.addTimeout instead of time_bound_deferred
...
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
2a3c33ff03
Use six.moves.urlparse
...
The imports were shuffled around a bunch in py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 21:22:43 +02:00
1d71f484d4
use PUT instead of POST for federating groups/m.join_policy
2018-04-06 12:54:09 +02:00
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
e43de3ae4b
Improve logging of failures in matrixfederationclient
...
* don't log exception types twice
* not all exceptions have a meaningful 'message'. Use the repr rather than
attempting to build a string ourselves.
2017-09-28 15:38:09 +01:00
e884ff31d8
Add DELETE
2017-07-18 16:41:44 +01:00
05c13f6c22
Add 'args' param to post_json
2017-07-18 16:40:21 +01:00
4903ccf159
Fix some lies, and other clarifications, in docstrings
...
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
a380f041c2
try not to drop context after federation requests
...
preserve_context_over_fn uses a ContextPreservingDeferred, which only restores
context for the duration of its callbacks, which isn't really correct, and
means that subsequent operations in the same request can end up without their
logcontexts.
2017-03-23 22:36:21 +00:00
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
ad8a26e361
MatrixFederationHttpClient: clean up
...
rename _create_request to _request, and push ascii-encoding of `destination`
and `path` down into it
2017-03-23 00:27:04 +00:00
170ccc9de5
Fix routing loop when fetching remote media
...
When we proxy a media request to a remote server, add a query-param, which will
tell the remote server to 404 if it doesn't recognise the server_name.
This should fix a routing loop where the server keeps forwarding back to
itself.
Also improves the error handling on remote media fetches, so that we don't
always return a rather obscure 502.
2017-03-13 16:30:36 +00:00
b4bc6fef5b
Respect long_retries param and default to off
2016-12-29 00:58:34 +00:00
b7336ff32d
Clean up
2016-12-29 00:09:33 +00:00
5b6672c66d
Wrap connections in an N minute timeout to ensure they get reaped correctly
2016-12-29 00:06:53 +00:00
b5b3a7e867
More intelligent Content-Type parsing
...
Content-Type is allowed to contain options (`; charset=utf-8`, for
instance). We should allow that.
2016-11-30 15:07:32 +00:00
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
0eabfa55f6
Fix typo
2015-11-20 17:17:58 +00:00
6408541075
Don't limit connections to perspective servers
2015-11-20 17:15:44 +00:00
2f2bbb4d06
Use min and not max to set an upper bound on retry interval
2015-11-20 09:34:58 +00:00
cbf3cd6151
Add comment
2015-11-17 18:29:29 +00:00
cf4ef5f3c7
Only retry federation requests for a long time for background requests
2015-11-17 18:26:50 +00:00
afdfd12bdf
Implement required method 'resumeProducing'
2015-11-17 16:57:06 +00:00
bceec65913
Slightly more aggressive retry timers at HTTP level
2015-11-17 15:10:05 +00:00
eacb068ac2
Retry dead servers a lot less often
2015-11-02 16:56:30 +00:00
81a93ddcc8
Allow configuration to ignore invalid SSL certs
...
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
2015-09-09 12:02:07 +01:00
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
adbd720fab
PEP8
2015-08-13 11:47:38 +01:00
998a72d4d9
Merge branch 'develop' into markjh/twisted-15
...
Conflicts:
synapse/http/matrixfederationclient.py
2015-08-12 17:21:14 +01:00
653533a3da
Fix log context when sending requests
2015-06-19 11:46:49 +01:00
18968efa0a
Remove stale debug lines
2015-06-19 10:18:02 +01:00
9d112f4440
Add IDs to outbound transactions
2015-06-19 10:13:03 +01:00
90abdaf3bc
Use Twisted-15.2.1, Use Agent.usingEndpointFactory rather than implement our own Agent
2015-06-01 10:51:50 +01:00
79e37a7ecb
Correctly pass connection pool parameter
2015-05-28 16:48:53 +01:00
11f51e6ded
Up maxPersistentPerHost count
2015-05-28 15:45:46 +01:00
291e942332
Use connection pool for federation connections
2015-05-28 15:43:21 +01:00
284f55a7fb
Add doc strings
2015-05-22 15:18:04 +01:00
5b1631a4a9
Add a timeout param to get_event
2015-05-19 14:53:32 +01:00
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
7d72e44eb9
Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code
2015-03-12 16:24:50 +00:00
7025781df8
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1
2015-02-18 17:37:43 +00:00
5358966a87
Use git aware version string in User-Agent and Server headers
2015-02-18 16:52:04 +00:00
7e9d59f3b4
Don't convert DNSLookupError to a 4xx SynapseError
2015-02-18 10:58:13 +00:00
65ca713ff5
Add .__name__ after type(e)
2015-02-18 10:51:32 +00:00
5e24471469
Fix up ResponseNeverReceived to str
2015-02-18 10:50:10 +00:00
0db52d43fa
strings.join() expects iterable of strings
2015-02-18 10:41:46 +00:00
0647e27a41
Remove unused import
2015-02-17 15:19:54 +00:00
4de93001bf
Make matrixfederationclient log more nicely
2015-02-17 15:12:06 +00:00
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
fb233dc40b
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 10:33:19 +00:00
05b961d7e3
PEP8
2015-02-11 10:28:46 +00:00
dcf52469e8
Move time_bound_deferred into Clock
2015-02-11 10:25:06 +00:00
0e6b3e4e40
Time out HTTP federation requests
2015-02-10 18:17:27 +00:00
9f2573eea1
Return body of response in HttpResponseException
2015-02-06 10:55:01 +00:00
ae46f10fc5
Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations
2015-02-04 16:28:12 +00:00
5a3a15f5c1
Make post_json(...) actually send data.
2015-01-29 13:58:22 +00:00
c183cec8f6
Add post_json(...) method to federation client
2015-01-29 13:44:52 +00:00
5fed042640
Finish renaming "context" to "room_id" in federation codebase
2015-01-16 19:01:03 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
24b5d01853
Include version in User-Agent and Server headers
2014-12-22 10:16:02 +00:00
d80d505b1f
Limit the size of images that are thumbnailed serverside. Limit the size of file that a server will download from a remote server
2014-12-11 14:19:32 +00:00
4f37c0ea9d
Merge branch 'develop' into media_repository
2014-12-10 16:55:06 +00:00
0f4dcab238
turn back on per-request transaction retries, so that every time we try to hit a dead server we actually end up hammering 5 times :|
2014-12-10 10:28:27 +00:00
b8d30899b1
Code style.
2014-12-10 10:16:09 +00:00
faf12b64f8
add errbacks to enqueue_pdu deferreds; change logging for failed federation sends to warn rather than exception
2014-12-10 00:12:51 +00:00
2b1acb7671
squidge to 79 columns as per pep8
2014-12-10 00:03:55 +00:00
aed62a3583
track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover.
2014-12-07 02:26:07 +00:00
c01fd5573c
Implement download support for media_repository
2014-12-04 14:22:31 +00:00
610c2ea131
Fix pep8 and pyflakes warnings
2014-11-20 18:00:10 +00:00
e377d33652
Separate out the matrix http client completely because just about all of its code it now separate from the simple case we need for standard HTTP(S)
2014-11-20 17:41:56 +00:00
Andrew Morgan