Commit Graph

321 Commits

Author SHA1 Message Date
Erik Johnston fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston a8ce159be4 Replace some ujson with simplejson to make it work 2018-03-16 00:27:09 +00:00
Erik Johnston 926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Richard van der Hoff 58dd148c4f Add some docstrings to help figure this out 2018-03-09 18:05:41 +00:00
Richard van der Hoff 88541f9009 Add a metric which increments when a request is received
It's useful to know when there are peaks in incoming requests - which isn't
quite the same as there being peaks in outgoing responses, due to the time
taken to handle requests.
2018-03-09 16:30:26 +00:00
Richard van der Hoff dbe80a286b refactor JsonResource
rephrase the OPTIONS and unrecognised request handling so that they look
similar to the common flow.
2018-03-09 16:22:16 +00:00
Richard van der Hoff 10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff 74fcbf741b delete_local_events for purge_history
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Richard van der Hoff 78e7e05188
Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_fail
Remove spurious log argument
2018-01-31 22:18:46 +01:00
Richard van der Hoff 421d68ca8c
Merge pull request #2817 from matrix-org/rav/http_conn_pool
Use a connection pool for the SimpleHttpClient
2018-01-31 22:14:22 +01:00
Richard van der Hoff af19f5e9aa Remove spurious log argument
... which would cause scary-looking and unhelpful errors in the log on dns fail
2018-01-30 17:52:03 +00:00
Richard van der Hoff 02ba118f81 Increase http conn pool size 2018-01-29 14:30:15 +00:00
Matthew Hodgson ab9f844aaf
Add federation_domain_whitelist option (#2820)
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Richard van der Hoff 2c8526cac7 Use a connection pool for the SimpleHttpClient
In particular I hope this will help the pusher, which makes many requests to
sygnal, and is currently negotiating SSL for each one.
2018-01-20 00:55:44 +00:00
Richard van der Hoff e8f7541d3f Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling 2018-01-17 14:01:57 +00:00
Richard van der Hoff fb6563b4be
Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis
Track db txn time in millisecs
2018-01-17 13:52:42 +00:00
Richard van der Hoff 936482d507 Fix 'NoneType' object has no attribute 'writeHeaders'
Avoid throwing a (harmless) exception when we try to write an error response to
an http request where the client has disconnected.

This comes up as a CRITICAL error in the logs which tends to mislead people
into thinking there's an actual problem
2018-01-16 17:58:16 +00:00
Richard van der Hoff 3d12d97415 Track DB scheduling delay per-request
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
Richard van der Hoff 6324b65f08 Track db txn time in millisecs
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
Richard van der Hoff 39f4e29d01 Reorganise request and block metrics
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.

The following are all deprecated, and replaced with synapse_util_metrics_block_count:
  synapse_util_metrics_block_timer:count
  synapse_util_metrics_block_ru_utime:count
  synapse_util_metrics_block_ru_stime:count
  synapse_util_metrics_block_db_txn_count:count
  synapse_util_metrics_block_db_txn_duration:count

The following are all deprecated, and replaced with synapse_http_server_response_count:
   synapse_http_server_requests
   synapse_http_server_response_time:count
   synapse_http_server_response_ru_utime:count
   synapse_http_server_response_ru_stime:count
   synapse_http_server_response_db_txn_count:count
   synapse_http_server_response_db_txn_duration:count

The following are renamed (the old metrics are kept for now, but deprecated):

  synapse_util_metrics_block_timer:total ->
     synapse_util_metrics_block_time_seconds

  synapse_util_metrics_block_ru_utime:total ->
     synapse_util_metrics_block_ru_utime_seconds

  synapse_util_metrics_block_ru_stime:total ->
     synapse_util_metrics_block_ru_stime_seconds

  synapse_util_metrics_block_db_txn_count:total ->
     synapse_util_metrics_block_db_txn_count

  synapse_util_metrics_block_db_txn_duration:total ->
     synapse_util_metrics_block_db_txn_duration_seconds

  synapse_http_server_response_time:total ->
     synapse_http_server_response_time_seconds

  synapse_http_server_response_ru_utime:total ->
     synapse_http_server_response_ru_utime_seconds

  synapse_http_server_response_ru_stime:total ->
     synapse_http_server_response_ru_stime_seconds

   synapse_http_server_response_db_txn_count:total ->
      synapse_http_server_response_db_txn_count

   synapse_http_server_response_db_txn_duration:total
      synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
Richard van der Hoff d90e8ea444 Update http request metrics before calling servlet
Make sure that we set the servlet name in the metrics object *before* calling
the servlet, in case the servlet throws an exception.
2018-01-09 18:27:35 +00:00
Richard van der Hoff df6c72ede3
Merge pull request #2711 from matrix-org/rav/fix_dns_errhandler
Fix error handling on dns lookup
2017-11-27 12:19:18 +00:00
Richard van der Hoff 6be01f599b Improve tracebacks on exceptions
Use failure.Failure to recover our failure, which will give us a useful
stacktrace, unlike the rethrown exception.
2017-11-27 12:05:58 +00:00
Richard van der Hoff 8b38096a89 Fix error handling on dns lookup
pass the right arguments to the errback handler

Fixes "TypeError('eb() takes exactly 2 arguments (1 given)',)"
2017-11-24 16:47:48 +00:00
Richard van der Hoff f90649eb2b Fix 500 on invalid utf-8 in request
If somebody sends us a request where the the body is invalid utf-8, we should
return a 400 rather than a 500. (json.loads throws a UnicodeError in this
situation)

We might as well catch all Exceptions here: it seems very unlikely that we
would get a request that *isn't caused by invalid json.
2017-11-10 09:15:39 +00:00
Richard van der Hoff fcdfc911ee Add a hook for custom rest endpoints
Let the user specify custom modules which can be used for implementing extra
endpoints.
2017-11-02 14:36:55 +00:00
Richard van der Hoff 8b56977b6f Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header
Front-end proxy: pass through auth header
2017-10-27 11:01:50 +01:00
Richard van der Hoff 173567a7f2 Docstring for post_urlencoded_get_json 2017-10-27 10:59:50 +01:00
Richard van der Hoff 0d8e3ad48b Fix logcontext leaks in httpclient
`preserve_context_over_fn` is borked
2017-10-26 18:17:10 +01:00
Richard van der Hoff 12ef02dc3d SimpleHTTPClient: add support for headers
Sometimes we need to pass headers into these methods
2017-10-26 17:59:50 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
David Baker 27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Richard van der Hoff e43de3ae4b Improve logging of failures in matrixfederationclient
* don't log exception types twice
* not all exceptions have a meaningful 'message'. Use the repr rather than
  attempting to build a string ourselves.
2017-09-28 15:38:09 +01:00
Richard van der Hoff 75e67b9ee4 Handle SERVFAILs when doing AAAA lookups for federation (#2477)
... to cope with people with broken dnssec setups, mostly
2017-09-28 15:24:00 +01:00
Richard van der Hoff f65e31d22f Do an AAAA lookup on SRV record targets (#2462)
Support SRV records which point at AAAA records, as well as A records.

Fixes https://github.com/matrix-org/synapse/issues/2405
2017-09-22 20:26:47 +01:00
Erik Johnston e884ff31d8 Add DELETE 2017-07-18 16:41:44 +01:00
Erik Johnston 05c13f6c22 Add 'args' param to post_json 2017-07-18 16:40:21 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Krombel 1ff419d343 allow Authorization header which handling got implemented in #1098
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
Kegan Dougal d0debb2116 Remember how twisted works 2017-05-05 11:00:21 +01:00
Kegan Dougal c2ddd773bc Include the clock 2017-05-05 10:52:46 +01:00
Kegan Dougal 7dd3bf5e24 Rewrite SimpleHttpClient.request to include timeouts
Fixes #2191
2017-05-05 10:49:19 +01:00
David Baker 482a2ad122 No need for the exception variable 2017-05-03 11:02:59 +01:00
David Baker c0380402bc List caught expection types 2017-05-03 10:56:22 +01:00
David Baker 5fd12dce01 Remove debugging 2017-04-26 12:36:26 +01:00
David Baker 81804909d3 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-26 11:31:55 +01:00
David Baker c366276056 Fix get_json 2017-04-26 10:07:01 +01:00
David Baker 1a9255c12e Use CodeMessageException subclass instead
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
David Baker a46982cee9 Need the HTTP status code 2017-04-21 16:20:12 +01:00
David Baker 70caf49914 Do the same for get_json 2017-04-21 16:09:03 +01:00