Commit Graph

653 Commits

Author SHA1 Message Date
Patrick Cloke ff5c4da128
Add a maximum size for well-known lookups. (#8950) 2020-12-16 17:25:24 -05:00
Patrick Cloke 1619802228
Various clean-ups to the logging context code (#8935) 2020-12-14 14:19:47 -05:00
Richard van der Hoff 1821f7cc26
Fix buglet in DirectRenderJsonResource (#8897)
this was using `canonical_json` without setting it, so when you used it as a
standalone class, you would get exceptions.
2020-12-10 12:42:55 +00:00
Patrick Cloke 30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Nicolas Chamo 3f0cba657c
Allow Date header through CORS (#8804) 2020-12-01 13:24:56 +00:00
Tulir Asokan 856eab606b
Remove special case of pretty printing JSON responses for curl (#8833)
* Remove special case of pretty printing JSON responses for curl

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2020-11-27 12:37:55 +00:00
Patrick Cloke 968939bdac
Add additional type hints to HTTP client. (#8812)
This also removes some duplicated code between the simple
HTTP client and matrix federation client.
2020-11-25 13:30:47 -05:00
Patrick Cloke f38676d161
Add type hints to matrix federation client / agent. (#8806) 2020-11-25 07:07:21 -05:00
Andrew Morgan eedaf90c84
Better error message when a remote resource uses invalid Content-Type (#8719) 2020-11-11 14:22:40 +00:00
Erik Johnston f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Richard van der Hoff c97da1e45d
Merge pull request #8678 from matrix-org/rav/fix_frozen_events
Fix serialisation errors when using third-party event rules.
2020-10-28 20:41:42 +00:00
Richard van der Hoff e80eb69887 remove unused imports 2020-10-28 16:18:05 +00:00
Richard van der Hoff b6ca69e4f1 Remove frozendict_json_encoder and support frozendicts everywhere
Not being able to serialise `frozendicts` is fragile, and it's annoying to have
to think about which serialiser you want. There's no real downside to
supporting frozendicts, so let's just have one json encoder.
2020-10-28 15:56:57 +00:00
Dan Callahan aff1eb7c67
Tell Black to format code for Python 3.5 (#8664)
This allows trailing commas in multi-line arg lists.

Minor, but we might as well keep our formatting current with regard to
our minimum supported Python version.

Signed-off-by: Dan Callahan <danc@element.io>
2020-10-27 23:26:36 +00:00
Patrick Cloke 34a5696f93
Fix typos and spelling errors. (#8639) 2020-10-23 12:38:40 -04:00
Erik Johnston c850dd9a8e
Fix handling of User-Agent headers with bad utf-8. (#8632) 2020-10-23 17:12:59 +01:00
Patrick Cloke 514a240aed
Remove unused OPTIONS handlers. (#8621)
The handling of OPTIONS requests was consolidated in #7534, but the endpoint
specific handlers were not removed.
2020-10-22 08:35:55 -04:00
Andrew Morgan 3e58ce72b4
Don't bother responding to client requests that have already disconnected (#8465)
This PR ports the quick fix from https://github.com/matrix-org/synapse/pull/2796 to further methods which handle media, URL preview and `/key/v2/server` requests. This prevents a harmless `ERROR` that comes up in the logs when we were unable to respond to a client request when the client had already disconnected. In this case we simply bail out if the client has already done so.

This is the 'simple fix' as suggested by https://github.com/matrix-org/synapse/issues/5304#issuecomment-574740003.

Fixes https://github.com/matrix-org/synapse/issues/6700
Fixes https://github.com/matrix-org/synapse/issues/5304
2020-10-06 10:03:39 +01:00
Patrick Cloke 4ff0201e62
Enable mypy checking for unreachable code and fix instances. (#8432) 2020-10-01 08:09:18 -04:00
Richard van der Hoff 1c262431f9
Fix handling of connection timeouts in outgoing http requests (#8400)
* Remove `on_timeout_cancel` from `timeout_deferred`

The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a
timeout (in particular if the canceller raised an exception), so it was
unreliable. It was also only used in one place, and to be honest it's easier to
do what it does a different way.

* Fix handling of connection timeouts in outgoing http requests

Turns out that if we get a timeout during connection, then a different
exception is raised, which wasn't always handled correctly.

To fix it, catch the exception in SimpleHttpClient and turn it into a
RequestTimedOutError (which is already a documented exception).

Also add a description to RequestTimedOutError so that we can see which stage
it failed at.

* Fix incorrect handling of timeouts reading federation responses

This was trapping the wrong sort of TimeoutError, so was never being hit.

The effect was relatively minor, but we should fix this so that it does the
expected thing.

* Fix inconsistent handling of `timeout` param between methods

`get_json`, `put_json` and `delete_json` were applying a different timeout to
the response body to `post_json`; bring them in line and test.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
2020-09-29 10:29:21 +01:00
Richard van der Hoff 450ec48445
A pair of tiny cleanups in the federation request code. (#8401) 2020-09-28 13:15:00 +01:00
Richard van der Hoff 11c9e17738
Add type annotations to SimpleHttpClient (#8372) 2020-09-24 15:47:20 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke aec294ee0d
Use slots in attrs classes where possible (#8296)
slots use less memory (and attribute access is faster) while slightly
limiting the flexibility of the class attributes. This focuses on objects
which are instantiated "often" and for short periods of time.
2020-09-14 12:50:06 -04:00
Patrick Cloke b86764662b
Fix the exception that is raised when invalid JSON is encountered. (#8291) 2020-09-10 14:55:25 -04:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke 5bf8e5f55b
Convert the well known resolver to async (#8214) 2020-09-01 09:15:22 -04:00
Patrick Cloke f594e434c3
Switch the JSON byte producer from a pull to a push producer. (#8116) 2020-08-19 08:07:57 -04:00
Patrick Cloke eebf52be06
Be stricter about JSON that is accepted by Synapse (#8106) 2020-08-19 07:26:03 -04:00
Patrick Cloke 2f4d60a5ba
Iteratively encode JSON responses to avoid blocking the reactor. (#8013) 2020-08-18 08:49:59 -04:00
Erik Johnston a0f574f3c2
Reduce INFO logging (#8050)
c.f. #8021 

A lot of the code here is to change the `Completed 200 OK` logging to include the request URI so that we can drop the `Sending request...` log line.

Some notes:

1. We won't log retries, which may be confusing considering the time taken log line includes retries and sleeps.
2. The `_send_request_with_optional_trailing_slash` will always be logged *without* the forward slash, even if it succeeded only with the forward slash.
2020-08-11 18:10:07 +01:00
Erik Johnston 2f9fd5ab00
Don't log OPTIONS request at INFO (#8049) 2020-08-07 14:53:05 +01:00
Erik Johnston 7620912d84
Add health check endpoint (#8048) 2020-08-07 14:21:24 +01:00
David Vo 4dd27e6d11
Reduce unnecessary whitespace in JSON. (#7372) 2020-08-07 08:02:55 -04:00
Patrick Cloke c36228c403
Convert run_as_background_process inner function to async. (#8032) 2020-08-06 08:20:42 -04:00
Patrick Cloke 88a3ff12f0
Convert the SimpleHttpClient to async. (#8016) 2020-08-04 07:22:04 -04:00
Patrick Cloke 6812509807
Implement handling of HTTP HEAD requests. (#7999) 2020-08-03 08:45:42 -04:00
Patrick Cloke c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Patrick Cloke a53e0160a2
Ensure the msg property of HttpResponseException is a string. (#7979) 2020-07-29 13:56:06 -04:00
Patrick Cloke 6a080ea184
Return an empty body for OPTIONS requests. (#7886) 2020-07-24 07:08:07 -04:00
Richard van der Hoff 1ec688bf21
Downgrade warning on client disconnect to INFO (#7928)
Clients disconnecting before we finish processing the request happens from time
to time. We don't need to yell about it
2020-07-24 09:55:47 +01:00
Patrick Cloke 68cd935826
Convert the federation agent and related code to async/await. (#7874) 2020-07-23 07:05:57 -04:00
Richard van der Hoff 2ccd48e921 fix an incorrect comment 2020-07-22 00:24:56 +01:00
Patrick Cloke 35450519de
Ensure that calls to `json.dumps` are compatible with the standard library json. (#7836) 2020-07-15 13:40:54 -04:00
Erik Johnston f13061d515
Fix client reader sharding tests (#7853)
* Fix client reader sharding tests

* Newsfile

* Fix typing

* Update changelog.d/7853.misc

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Move mocking of http_client to tests

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-07-15 15:27:35 +01:00
Patrick Cloke 66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802) 2020-07-10 14:30:08 -04:00
Patrick Cloke d9e47af617
Add types to the server code and remove unused parameter (#7813) 2020-07-10 14:28:42 -04:00
Richard van der Hoff 67593b1728
Add `HomeServer.signing_key` property (#7805)
... instead of duplicating `config.signing_key[0]` everywhere
2020-07-08 17:51:56 +01:00
Patrick Cloke ff0680f69d
Stop passing bytes when dumping JSON (#7799) 2020-07-08 07:14:56 -04:00
Erik Johnston 5cdca53aa0
Merge different Resource implementation classes (#7732) 2020-07-03 19:02:19 +01:00