Commit Graph

377 Commits

Author SHA1 Message Date
Richard van der Hoff da0090fdff
Fix modifying events in `ThirdPartyRules` modules (#8564)
EventBuilder.build wants auth events these days
2020-10-16 13:39:46 +01:00
Richard van der Hoff 5649669c3c
Merge pull request #8535 from matrix-org/rav/third_party_events_updates
Support modifying event content from ThirdPartyRules modules
2020-10-15 20:55:41 +01:00
Richard van der Hoff a34b17e492 Simplify `_locally_reject_invite`
Update `EventCreationHandler.create_event` to accept an auth_events param, and
use it in `_locally_reject_invite` instead of reinventing the wheel.
2020-10-13 23:58:48 +01:00
Richard van der Hoff 617e8a4653 Allow ThirdPartyRules modules to replace event content
Support returning a new event dict from `check_event_allowed`.
2020-10-13 23:24:50 +01:00
Richard van der Hoff d9d86c2996 Remove redundant `token_id` parameter to create_event
this is always the same as requester.access_token_id.
2020-10-13 23:06:36 +01:00
Richard van der Hoff 123711ed19 Move third_party_rules check to event creation time
Rather than waiting until we handle the event, call the ThirdPartyRules check
when we fist create the event.
2020-10-13 21:38:48 +01:00
Patrick Cloke 629a951b49
Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
Erik Johnston b2486f6656
Fix message duplication if something goes wrong after persisting the event (#8476)
Should fix #3365.
2020-10-13 12:07:56 +01:00
Andrew Morgan 66ac4b1e34
Allow modules to create and send events into rooms (#8479)
This PR allows Synapse modules making use of the `ModuleApi` to create and send non-membership events into a room. This can useful to have modules send messages, or change power levels in a room etc. Note that they must send event through a user that's already in the room.

The non-membership event limitation is currently arbitrary, as it's another chunk of work and not necessary at the moment.
2020-10-09 13:46:36 +01:00
Patrick Cloke c9c0ad5e20
Remove the deprecated Handlers object (#8494)
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Richard van der Hoff 903fcd2d35 update wording 2020-10-07 11:28:05 +01:00
Richard van der Hoff e775b5bb5b kill off `send_nonmember_event`
This is now redundant, and we can just call `handle_new_client_event` directly.
2020-10-05 19:04:10 +01:00
Richard van der Hoff fd0282201e pull up event.sender assertion 2020-10-05 19:00:50 +01:00
Richard van der Hoff 2ee302d016 Move shadow-ban check down into `handle_new_client_event`. 2020-10-05 18:55:06 +01:00
Richard van der Hoff b520a1bf5a De-duplicate duplicate handling
move the "duplicate state event" handling down into `handle_new_client_event`
where it can be shared between multiple call paths.
2020-10-05 18:38:25 +01:00
Richard van der Hoff f31f8e6319
Remove stream ordering from Metadata dict (#8452)
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.

This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Erik Johnston ac11fcbbb8
Add EventStreamPosition type (#8388)
The idea is to remove some of the places we pass around `int`, where it can represent one of two things:

1. the position of an event in the stream; or
2. a token that partitions the stream, used as part of the stream tokens.

The valid operations are then:

1. did a position happen before or after a token;
2. get all events that happened before or after a token; and
3. get all events between two tokens.

(Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
2020-09-24 13:24:17 +01:00
Richard van der Hoff 2983049a77
Factor out `_send_dummy_event_for_room` (#8370)
this makes it possible to use from the manhole, and seems cleaner anyway.
2020-09-23 18:18:43 +01:00
Erik Johnston 04cc249b43
Add experimental support for sharding event persister. Again. (#8294)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-14 10:16:41 +01:00
Erik Johnston fe8ed1b46f
Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281) 2020-09-11 12:22:55 +01:00
Erik Johnston 5d3e306d9f
Clean up `Notifier.on_new_room_event` code path (#8288)
The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it.

This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens.

The valid operations here are:

1. Is a position before or after a token
2. Fetching all events between two tokens
3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C.

Future PR will change the token type to a dedicated type.
2020-09-10 13:24:43 +01:00
Erik Johnston c9dbee50ae
Fixup pusher pool notifications (#8287)
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
2020-09-09 16:56:08 +01:00
Erik Johnston dc9dcdbd59 Revert "Fixup pusher pool notifications"
This reverts commit e7fd336a53.
2020-09-09 16:19:22 +01:00
Erik Johnston e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Brendan Abolivier 9f8abdcc38
Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
* Revert "Add experimental support for sharding event persister. (#8170)"

This reverts commit 82c1ee1c22.

* Changelog
2020-09-04 10:19:42 +01:00
Erik Johnston 82c1ee1c22
Add experimental support for sharding event persister. (#8170)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-02 15:48:37 +01:00
Patrick Cloke da77520cd1
Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
Andrew Morgan e0d6244beb
Remove unused parameter from, and add safeguard in, get_room_data (#8174)
Small cleanup PR.

* Removed the unused `is_guest` argument
* Added a safeguard to a (currently) impossible code path, fixing static checking at the same time.
2020-08-26 15:07:35 +01:00
Patrick Cloke 5099bd68da
Do not allow send_nonmember_event to be called with shadow-banned users. (#8158) 2020-08-25 10:52:15 -04:00
Patrick Cloke cbd8d83da7
Stop shadow-banned users from sending non-member events. (#8142) 2020-08-24 13:58:56 -04:00
Patrick Cloke eebf52be06
Be stricter about JSON that is accepted by Synapse (#8106) 2020-08-19 07:26:03 -04:00
Patrick Cloke f40645e60b
Convert events worker database to async/await. (#8071) 2020-08-18 16:20:49 -04:00
Patrick Cloke 3c01724b33
Fix the return type of send_nonmember_events. (#8112) 2020-08-18 09:53:13 -04:00
Patrick Cloke 25e55d2598
Return the previous stream token if a non-member event is a duplicate. (#8093) 2020-08-18 07:53:23 -04:00
Richard van der Hoff 53834bb9c4
Run `remove_push_actions_from_staging` in foreground (#8081)
If we got an error persisting an event, we would try to remove the push actions
asynchronously, which would lead to a 'Re-starting finished log context'
warning.

I don't think there's any need for this to be asynchronous.
2020-08-13 17:05:31 +01:00
Erik Johnston 5dd73d029e
Add type hints to handlers.message and events.builder (#8067) 2020-08-12 15:05:50 +01:00
Richard van der Hoff fcbab08cbd
Add an assertion on prev_events in create_new_client_event (#8041)
I think this would have caught all the cases in
https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes
more sense here than a 403
2020-08-10 12:29:47 +01:00
Patrick Cloke d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Patrick Cloke d8a9cd8d3e
Remove hacky error handling for inlineDeferreds. (#7950) 2020-07-27 08:35:56 -04:00
Patrick Cloke cc9bb3dc3f
Convert the message handler to async/await. (#7884) 2020-07-22 12:29:15 -04:00
Patrick Cloke 6fca1b3506
Convert _base, profile, and _receipts handlers to async/await (#7860) 2020-07-17 07:08:30 -04:00
Richard van der Hoff 2ab0b021f1
Generate real events when we reject invites (#7804)
Fixes #2181. 

The basic premise is that, when we
fail to reject an invite via the remote server, we can generate our own
out-of-band leave event and persist it as an outlier, so that we have something
to send to the client.
2020-07-09 10:40:19 +01:00
Patrick Cloke e060bf4462
Convert directory handler to async/await (#7727) 2020-06-22 07:18:00 -04:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Patrick Cloke bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. (#7692) 2020-06-15 07:03:36 -04:00
Andrew Morgan f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595)
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
Richard van der Hoff f4269694ce
Optimise some references to hs.config (#7546)
These are surprisingly expensive, and we only really need to do them at startup.
2020-05-22 21:47:07 +01:00
Erik Johnston e5c67d04db
Add option to move event persistence off master (#7517) 2020-05-22 16:11:35 +01:00