Commit Graph

64 Commits

Author SHA1 Message Date
Richard van der Hoff a0ed0f363e
Soft-fail spammy events received over federation (#10263) 2021-06-29 11:08:06 +01:00
Erik Johnston c842c581ed
When joining a remote room limit the number of events we concurrently check signatures/hashes for (#10117)
If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
2021-06-08 11:07:46 +01:00
Erik Johnston 1c6a19002c
Add `Keyring.verify_events_for_server` and reduce memory usage (#10018)
Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.
2021-05-20 16:25:11 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
David Teller f14428b25c
Allow spam-checker modules to be provide async methods. (#8890)
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
2020-12-11 14:05:15 -05:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Patrick Cloke bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. (#7692) 2020-06-15 07:03:36 -04:00
Patrick Cloke 56b66db78a
Strictly enforce canonicaljson requirements in a new room version (#7381) 2020-05-14 13:24:01 -04:00
Richard van der Hoff 39230d2171
Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Patrick Cloke c2db6599c8
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors (#7089). 2020-03-19 08:22:56 -04:00
Patrick Cloke 7581d30e9f
Remove unused federation endpoint (`query_auth`) (#7026) 2020-03-17 08:04:49 -04:00
Patrick Cloke 12d4259000
Add some type annotations to the federation base & client classes (#6995) 2020-02-28 07:31:07 -05:00
Patrick Cloke 380122866f
Cast a coroutine into a Deferred in the federation base (#6996)
Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
2020-02-26 11:32:13 -05:00
Richard van der Hoff 799001f2c0
Add a `make_event_from_dict` method (#6858)
... and use it in places where it's trivial to do so.

This will make it easier to pass room versions into the FrozenEvent
constructors.
2020-02-07 15:30:04 +00:00
Richard van der Hoff 928edef979 Pass room_version into `event_from_pdu_json`
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
Andrew Morgan 54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Richard van der Hoff 172f264ed3
Improve signature checking on some federation APIs (#6262)
Make sure that we check that events sent over /send_join, /send_leave, and
/invite, are correctly signed and come from the expected servers.
2019-10-28 12:43:23 +00:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Richard van der Hoff 2f8491daef
Fix logging error when a tampered event is detected. (#5500) 2019-06-21 15:11:42 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
Richard van der Hoff 14f13babb0
Add a test room version where we enforce key validity (#5348) 2019-06-05 10:38:25 +01:00
Richard van der Hoff fec2dcb1a5
Enforce validity period on server_keys for fed requests. (#5321)
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.

We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
Richard van der Hoff 837d7f85a9 more logging improvements 2019-04-25 22:17:59 +01:00
Richard van der Hoff fd8fb32bdd remove extraneous exception logging 2019-04-25 22:02:03 +01:00
Richard van der Hoff 7ca638c761 Clarify logging when PDU signature checking fails 2019-04-25 20:55:12 +01:00
Richard van der Hoff 54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Erik Johnston 0b24d58e05 No vdh tests! 2019-01-29 23:11:48 +00:00
Erik Johnston a1b0e1879b Enable room version v3 2019-01-29 23:09:10 +00:00
Erik Johnston ff2f65d737 Update comment 2019-01-29 22:35:36 +00:00
Erik Johnston 840068bd78 Only check event ID domain for signatures for V1 events
In future version events won't have an event ID, so we won't be able to
do this check.
2019-01-29 18:02:02 +00:00
Erik Johnston a50cf929c1 Require event format version to parse or create events 2019-01-25 10:32:19 +00:00
Erik Johnston be6a7e47fa
Revert "Require event format version to parse or create events" 2019-01-25 10:23:51 +00:00
Erik Johnston f431ff3fb8 Require event format version to parse or create events 2019-01-23 20:21:33 +00:00
Erik Johnston 6a41d2a187 Add room_version param to get_pdu
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
Erik Johnston 89a76d1889 Fix handling of redacted events from federation
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.

This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
2018-09-13 15:44:12 +01:00
Richard van der Hoff cd7ef43872 clearer logging when things fail, too 2018-09-06 23:56:47 +01:00
Richard van der Hoff 804dd41e18 Check that signatures on events are valid
We should check that both the sender's server, and the server which created the
event_id (which may be different from whatever the remote server has told us
the origin is), have signed the event.
2018-09-05 13:08:07 +01:00
Krombel 3366b9c534 rename assert_params_in_request to assert_params_in_dict
the method "assert_params_in_request" does handle dicts and not
requests. A request body has to be parsed to json before this method
can be used
2018-07-13 21:53:01 +02:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Richard van der Hoff 33f469ba19 Apply some limits to depth to counter abuse
* When creating a new event, cap its depth to 2^63 - 1
* When receiving events, reject any without a sensible depth

As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI
2018-05-01 17:54:19 +01:00
Erik Johnston e05bf34117 Move property setting from ReplicationLayer to FederationBase 2018-03-13 10:51:30 +00:00
Richard van der Hoff bd91857028 Check missing fields in event_from_pdu_json
Return a 400 rather than a 500 when somebody messes up their send_join
2017-12-30 18:40:19 +00:00
Richard van der Hoff 3079f80d4a Factor out `event_from_pdu_json`
turns out we have two copies of this, and neither needs to be an instance
method
2017-12-30 18:40:19 +00:00
David Baker 6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
Richard van der Hoff 6de74ea6d7 Fix logcontexts in _check_sigs_and_hashes 2017-09-20 01:32:42 +01:00
Richard van der Hoff fcf2c0fd1a Remove redundant `preserve_fn`
preserve_fn is a no-op unless the wrapped function returns a
Deferred. verify_json_objects_for_server returns a list, so this is doing
nothing.
2017-09-20 01:32:42 +01:00
Richard van der Hoff 5ed109d59f PoC for filtering spammy events (#2456)
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00