c6f730282c
Move RequestMetrics handling into SynapseRequest.processing()
...
It fits quite nicely here, and opens the path to getting rid of the
"include_metrics" mess.
2018-05-10 12:19:51 +01:00
09b29f9c4a
Make RequestMetrics take a raw time rather than a clock
...
... which is going to make it easier to move around.
2018-05-10 12:18:52 +01:00
8460e48d06
Move request_id management into SynapseRequest
2018-05-10 11:48:17 +01:00
18e144fe08
Move RequestsMetrics to its own file
...
This is useful in its own right, because server.py is full of stuff; but more
importantly, I want to do some refactoring that will cause a circular reference
as it is.
2018-05-09 19:55:03 +01:00
e9143b6593
more bytes strings
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-29 00:13:57 +02:00
87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings
...
Add b prefixes to some strings that are bytes in py3
2018-04-09 23:54:05 +01:00
13decdbf96
Revert "Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics"
...
We aren't ready to release this yet, so I'm reverting it for now.
This reverts commit d1679a4ed7e089100c62
2018-04-09 12:59:12 +01:00
7b824f1475
Add response size metrics
2018-04-06 13:20:11 +01:00
518f6de088
Remove redundant metrics which were deprecated in 0.27.0.
2018-04-04 19:46:28 +01:00
6168351877
Add b prefixes to some strings that are bytes in py3
...
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-04 13:48:51 +02:00
1c41b05c8c
Add Cache-Control headers to all JSON APIs
...
It is especially important that sync requests don't get cached, as if a
sync returns the same token given then the client will call sync with
the same parameters again. If the previous response was cached it will
get reused, resulting in the client tight looping making the same
request and never making any progress.
In general, clients will expect to get up to date data when requesting
APIs, and so its safer to do a blanket no cache policy than only
whitelisting APIs that we know will break things if they get cached.
2018-03-21 17:46:26 +00:00
fa72803490
Merge branch 'master' of github.com:matrix-org/synapse into develop
2018-03-19 11:41:01 +00:00
a8ce159be4
Replace some ujson with simplejson to make it work
2018-03-16 00:27:09 +00:00
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
58dd148c4f
Add some docstrings to help figure this out
2018-03-09 18:05:41 +00:00
88541f9009
Add a metric which increments when a request is received
...
It's useful to know when there are peaks in incoming requests - which isn't
quite the same as there being peaks in outgoing responses, due to the time
taken to handle requests.
2018-03-09 16:30:26 +00:00
dbe80a286b
refactor JsonResource
...
rephrase the OPTIONS and unrecognised request handling so that they look
similar to the common flow.
2018-03-09 16:22:16 +00:00
e8f7541d3f
Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling
2018-01-17 14:01:57 +00:00
fb6563b4be
Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis
...
Track db txn time in millisecs
2018-01-17 13:52:42 +00:00
936482d507
Fix 'NoneType' object has no attribute 'writeHeaders'
...
Avoid throwing a (harmless) exception when we try to write an error response to
an http request where the client has disconnected.
This comes up as a CRITICAL error in the logs which tends to mislead people
into thinking there's an actual problem
2018-01-16 17:58:16 +00:00
3d12d97415
Track DB scheduling delay per-request
...
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
6324b65f08
Track db txn time in millisecs
...
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
39f4e29d01
Reorganise request and block metrics
...
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.
The following are all deprecated, and replaced with synapse_util_metrics_block_count:
synapse_util_metrics_block_timer:count
synapse_util_metrics_block_ru_utime:count
synapse_util_metrics_block_ru_stime:count
synapse_util_metrics_block_db_txn_count:count
synapse_util_metrics_block_db_txn_duration:count
The following are all deprecated, and replaced with synapse_http_server_response_count:
synapse_http_server_requests
synapse_http_server_response_time:count
synapse_http_server_response_ru_utime:count
synapse_http_server_response_ru_stime:count
synapse_http_server_response_db_txn_count:count
synapse_http_server_response_db_txn_duration:count
The following are renamed (the old metrics are kept for now, but deprecated):
synapse_util_metrics_block_timer:total ->
synapse_util_metrics_block_time_seconds
synapse_util_metrics_block_ru_utime:total ->
synapse_util_metrics_block_ru_utime_seconds
synapse_util_metrics_block_ru_stime:total ->
synapse_util_metrics_block_ru_stime_seconds
synapse_util_metrics_block_db_txn_count:total ->
synapse_util_metrics_block_db_txn_count
synapse_util_metrics_block_db_txn_duration:total ->
synapse_util_metrics_block_db_txn_duration_seconds
synapse_http_server_response_time:total ->
synapse_http_server_response_time_seconds
synapse_http_server_response_ru_utime:total ->
synapse_http_server_response_ru_utime_seconds
synapse_http_server_response_ru_stime:total ->
synapse_http_server_response_ru_stime_seconds
synapse_http_server_response_db_txn_count:total ->
synapse_http_server_response_db_txn_count
synapse_http_server_response_db_txn_duration:total
synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
d90e8ea444
Update http request metrics before calling servlet
...
Make sure that we set the servlet name in the metrics object *before* calling
the servlet, in case the servlet throws an exception.
2018-01-09 18:27:35 +00:00
6be01f599b
Improve tracebacks on exceptions
...
Use failure.Failure to recover our failure, which will give us a useful
stacktrace, unlike the rethrown exception.
2017-11-27 12:05:58 +00:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
1ff419d343
allow Authorization header which handling got implemented in #1098
...
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
b1c27975d0
Set CORs headers on responses from the media repo
2016-11-02 11:29:25 +00:00
33f3624ff7
Add exception logging. Fix typo
2016-08-22 10:49:31 +01:00
8c52160b07
Allow request handlers to override metric name
2016-08-22 10:44:45 +01:00
a093fab253
Use top level measure
2016-08-22 10:18:12 +01:00
27e0178da9
Add a top level measure
2016-08-19 18:49:37 +01:00
6d1a94d218
Remove redundant measure
2016-08-19 18:40:31 +01:00
39b900b316
Measure http.server render
2016-08-19 18:23:44 +01:00
33d08e8433
Log when adding listeners
2016-07-25 17:22:15 +01:00
351b50a887
Fix more typos in per-request metrics
2016-04-28 15:29:46 +01:00
dcfc10b129
Fix typo in request metrics
2016-04-28 15:11:06 +01:00
1a12766e3b
Add a comment explaining why automatic metric reporting is disabled for JsonResource
2016-04-28 12:31:26 +01:00
6037349512
Check if report_metrics is True
2016-04-28 12:26:07 +01:00
8d7ad44331
Report per request metrics for all of the things using request_handler
2016-04-28 10:57:49 +01:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
58c9f20692
Catch the exceptions thrown by twisted when you write to a closed connection
2016-02-12 13:46:59 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
13e6262659
Add metrics to pushers
2016-02-08 14:26:45 +00:00
d4f72a5bfb
Allowing tagging log contexts
2016-02-03 13:52:27 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
2b0f8a9482
Fix typo
2015-12-16 17:59:44 +01:00
338c0a8a69
Include errcode on Internal Server Error
2015-12-14 13:50:50 +00:00
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
99e1d6777f
Add metrics to track the cpu on the main thread consumed by each type of request
2015-12-04 14:42:24 +00:00
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
a7dcbfe430
Don't 500 if a group is missing from the regex
2015-07-06 16:47:17 +01:00
aaa749d366
Disable twisted access logging. Move access logging to SynapseRequest object
2015-06-15 18:18:05 +01:00
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
b5209c5744
Create SynapseRequest that overrides __repr__ to not print access_token
2015-06-15 16:37:04 +01:00
f00f8346f1
Make http.server request logging more verbose, but redact access_tokens
2015-06-15 13:37:58 +01:00
d15f166093
Remove log line
2015-05-29 15:03:24 +01:00
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
2236ef6c92
Fix up leak. Add warnings.
2015-05-08 19:53:34 +01:00
0c1b7f843b
Unused import
2015-04-30 13:33:30 +01:00
4b46fbec5b
Doesn't look like this is used anymore
2015-04-30 12:04:08 +01:00
6532b6e607
Merge branch 'develop' into csauth
...
Conflicts:
synapse/http/server.py
2015-04-24 09:37:54 +01:00
1967650bc4
Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator
2015-04-21 16:35:53 +01:00
1ebff9736b
Split out the JsonResource request logging and error handling into a separate wrapper function
2015-04-21 16:07:20 +01:00
c5bf0343e8
Explain how I justified to myself making JsonResource not always send JSON.
2015-04-01 15:13:14 +01:00
e9c908ebc0
Completely replace fallback auth for C/S V2:
...
* Now only the auth part goes to fallback, not the whole operation
* Auth fallback is a normal API endpoint, not a static page
* Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
f129ee1e18
Make docs a bit more true
2015-03-31 18:25:10 +01:00
d18e7779ca
Grammar and deduplication
2015-03-31 14:40:02 +01:00
3f7a31d366
Add a DistributionMetric to HTTP request/response processing time in the server
2015-03-16 18:31:29 +00:00
c8032aec17
actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0'
2015-03-14 00:12:20 +00:00
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
7d72e44eb9
Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code
2015-03-12 16:24:50 +00:00
9d9d39536b
Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through
2015-03-05 16:24:13 +00:00
5358966a87
Use git aware version string in User-Agent and Server headers
2015-02-18 16:52:04 +00:00
6370cffbbf
Fix bug where variable was not always defined
2015-02-11 10:34:41 +00:00
24cc6979fb
Log when we receive a request, when we send a response and how long it took to process it.
2015-02-09 13:46:22 +00:00
ede491b4e0
Oops: second part of commit dc938606
2015-01-22 17:38:53 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
773de09774
Set a content-length for JSON responses
2015-01-06 13:05:19 +00:00
24b5d01853
Include version in User-Agent and Server headers
2014-12-22 10:16:02 +00:00
1fc2a0e33e
Fix tests and remove debug logging
2014-12-12 15:08:29 +00:00
7b43a503f3
Consistently url decode and decode as utf 8 the URL parts
2014-12-12 15:05:37 +00:00
5da65085d1
Get uploads working with new media repo
2014-12-02 19:51:47 +00:00
279c48c8b4
Write the upload portion of version 1 of the media repository
2014-12-02 17:13:14 +00:00
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
dfdda2c871
Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
2014-11-20 17:10:37 +00:00
b29517bd01
Add a request-id to each log line
2014-10-30 01:21:33 +00:00
8a7c1d6a00
fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch.
2014-09-03 17:31:57 +01:00
bf10cf5f1a
move contentrepo class to it's own file
2014-09-03 11:10:44 +01:00
112c7ea315
Set status message for ratelimit error responses
2014-09-03 09:37:44 +01:00
68a04b9282
given everything's apparently gone https:// by default for c-s, change this hardcoding...
2014-09-01 23:28:43 +01:00
2b7918bd6f
missed a s#/matrix#/_matrix/g
2014-08-31 14:54:58 +01:00
0c3b4a1f63
For the content repo, don't just use homeserver.hostname as that might not include the port due to SRV.
2014-08-24 11:56:55 +01:00
acf5127604
Make the content repo work with in daemon mode. Return the full url on upload. Update the webclient to use new content repo api.
2014-08-22 10:25:32 +01:00
f48792eec4
Reduce the amount of incredibly spammy stack traces. Expected errors (e.g. SynapseErrors) shouldn't have their full trace logged every time. Don't send responses to disconnected requests.
2014-08-19 10:56:43 +01:00
e37b040bc3
Small amounts of cleanup and bonus round comments.
2014-08-18 17:22:31 +01:00
58548ab557
Implemented GETs for the ContentRepoResource. It all actually appears to be working.
2014-08-18 17:18:54 +01:00
590ab24c85
hs: Make the uploads directory if it doesn't exist. Namespace uploads by the base64 encoded user id of the uploader. Make a reasonable attempt to retry clashing upload paths. Try to guess a sensible file extension depending on the content type.
2014-08-18 17:18:54 +01:00
35da1bf4a3
Auth content uploads. Added a mapping function from request > filename. Added exception handling for content uploads. webclient: Only prefix the client API path on doRequest, not doBaseRequest (this would've broken the identity server auth too). Added matrixService.uploadContent. May not require mFileUpload anymore.
2014-08-18 17:18:54 +01:00
a18b1a649c
Added /matrix/content path, HS resource_for_content_repo attribute and FileUploadResource. Added stub methods.
2014-08-18 17:18:54 +01:00
9fd445eb92
If the web client is enabled, automatically redirect root '/' to the web client path.
2014-08-14 11:37:13 +01:00
9a1638ed21
Removed http_server from HomeServer. Updated unit tests to use either resource_for_federation or resource_for_client depending on what is being tested.
2014-08-14 10:18:54 +01:00
9f863d3466
Start phasing out HttpServer: we should be using Resources instead. Added resource_for_client/federation/web_client to the HomeServer and hooked the C-S servlets to operate on resource_for_client. Dynamically construct the Resource tree.
2014-08-14 09:55:16 +01:00
f98e6380f1
add in whitespace after copyright statements to improve legibility
2014-08-13 03:14:34 +01:00
4add1c70e9
Just say yes to OPTIONS requests, since we want to turn on CORS headers for all paths
2014-08-12 17:17:14 +01:00
4f475c7697
Reference Matrix Home Server
2014-08-12 15:10:52 +01:00
Richard van der Hoff