Commit Graph

463 Commits

Author SHA1 Message Date
Adrian Tschira d9fe2b2d9d Replace some more comparisons with six
plus a bonus b"" string I missed last time

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
Richard van der Hoff 950a32eb47
Merge pull request #3152 from NotAFile/py3-local-imports
make imports local
2018-04-30 01:28:13 +01:00
Adrian Tschira 57b58e2174 make imports local
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:41:41 +02:00
Richard van der Hoff fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff 2a13af23bc Use run_in_background in preference to preserve_fn
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Richard van der Hoff 9c3da24561
Merge pull request #3138 from matrix-org/rav/catch_unhandled_exceptions
Improve exception handling for background processes
2018-04-27 11:47:49 +01:00
Richard van der Hoff 605defb9e4 Add missing consumeErrors
In general we want defer.gatherResults to consumeErrors, rather than having
exceptions hanging around and getting logged as CRITICAL unhandled errors.
2018-04-27 11:16:28 +01:00
Richard van der Hoff 9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Erik Johnston 573712da6b Update comments 2018-02-21 11:29:49 +00:00
Erik Johnston 6ff8c87484 Batch inserts into event_push_actions_staging 2018-02-20 11:33:07 +00:00
Erik Johnston 012e8e142a Comments 2018-02-16 11:35:01 +00:00
Erik Johnston 4810f7effd Remove context.push_actions 2018-02-15 15:47:06 +00:00
Erik Johnston acac21248c Store push actions in staging area 2018-02-15 15:47:04 +00:00
Richard van der Hoff 03dd745fe2 Better logging when pushes fail 2018-01-29 15:49:06 +00:00
Richard van der Hoff e051abd20b add appid/device_display_name to to pusher logging 2018-01-29 15:04:16 +00:00
Richard van der Hoff 4528dd2443 Fix logging and add user_id 2018-01-22 20:15:42 +00:00
Richard van der Hoff 93efd7eb04 logging and debug for http pusher 2018-01-22 18:14:10 +00:00
Richard van der Hoff 2c6d63922a Remove pushers when deleting access tokens
Whenever an access token is invalidated, we should remove the associated
pushers.
2017-11-29 16:44:35 +00:00
Richard van der Hoff 1b83c09c03
Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs
Remove preserve_context_over_{fn, deferred}
2017-11-15 11:13:53 +00:00
Richard van der Hoff 7e6fa29cb5 Remove preserve_context_over_{fn, deferred}
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
David Baker 2a98ba0ed3 Rename redact_content option to include_content
The redact_content option never worked because it read the wrong config
section. The PR introducing it
(https://github.com/matrix-org/synapse/pull/2301) had feedback suggesting the
name be changed to not re-use the term 'redact' but this wasn't
incorporated.

This reanmes the option to give it a less confusing name, and also
means that people who've set the redact_content option won't suddenly
see a behaviour change when upgrading synapse, but instead can set
include_content if they want to.

This PR also updates the wording of the config comment to clarify
that this has no effect on event_id_only push.

Includes https://github.com/matrix-org/synapse/pull/2422
2017-11-08 10:35:30 +00:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
David Baker 9342bcfce0 Omit the *s for @room notifications
They're just redundant
2017-10-16 13:38:10 +01:00
David Baker 81a5e0073c pep8 2017-10-10 15:53:34 +01:00
David Baker ab1bc9bf5f Don't KeyError if no power_levels event 2017-10-10 15:34:05 +01:00
David Baker 0f1eb3e914 Use notification levels in power_levels
Rather than making the condition directly require a specific power
level. This way the level require to notify a room can be configured
per room.
2017-10-10 15:23:00 +01:00
David Baker c9f034b4ac There was already a constant for this
also update copyright
2017-10-10 11:47:10 +01:00
David Baker a9f9d68631 More optimisation 2017-10-10 11:38:31 +01:00
David Baker 707374d5dc What year is it!? Who's the president!? 2017-10-10 11:21:41 +01:00
David Baker 269af961e9 Make be faster 2017-10-05 13:27:12 +01:00
David Baker ed80c6b6cc Add fastpath optimisation 2017-10-05 13:20:22 +01:00
David Baker e433393c4f pep8 2017-10-05 13:08:02 +01:00
David Baker 985ce80375 They're called rooms 2017-10-05 13:03:44 +01:00
David Baker b9b9714fd5 Get rule type right 2017-10-05 13:02:19 +01:00
David Baker fa969cfdde Support for channel notifications
Add condition type to check the sender's power level and add a base
rule using it for @channel notifications.
2017-10-05 12:39:18 +01:00
David Baker 0c8da8b519 Use better method for word boundary searching
From ebc95667b8
2017-10-05 11:57:43 +01:00
David Baker cbe3c3fdd4 pep8 2017-10-05 11:43:10 +01:00
David Baker 6748f0a579 Fix notif kws that start/end with non-word chars
Only prepend / append word bounary characters if the search
expression starts or ends with a word character, otherwise they
don't work because there's no word bounary between whitespace and
a non-word char.
2017-10-05 11:33:30 +01:00
David Baker 436ee0a2ea Also include the room_id
as really it's part of the event ID
2017-09-18 15:58:38 +01:00
David Baker b393f5db51 Use .get - it's much shorter 2017-09-18 15:50:26 +01:00
David Baker a2562f9d74 Add support for event_id_only push format
Param in the data dict of a pusher that tells an HTTP pusher to
send just the event_id of the event it's notifying about and the
notification counts. For clients that want to go & fetch the body
of the event themselves anyway.
2017-09-18 15:39:39 +01:00
Erik Johnston bfde076022 Increase cache hit ratio for push
We don't update the cache in all code paths, which causes subsequent
calls to miss the cache
2017-07-14 16:11:26 +01:00
Erik Johnston 8d26385d76 Add more metrics to push rule evaluation 2017-07-13 14:37:30 +01:00
Erik Johnston f60218ec41 Push: Don't acquire lock unless necessary 2017-07-13 11:23:53 +01:00
Erik Johnston f502b0dea1 Perf: Don't filter events for push
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
2017-07-07 14:04:40 +01:00
Richard van der Hoff 3d31b39297 Merge pull request #2332 from matrix-org/rav/fix_pushes
Fix caching error in the push evaluator
2017-07-05 11:10:53 +01:00
Richard van der Hoff 73cfe48031 Fix caching error in the push evaluator
Initialising `result` to `{}` in the parameters meant that every call to
_flatten_dict used the *same* target dictionary.

I'm hopeful this will fix https://github.com/matrix-org/synapse/issues/2270,
but I suspect it won't. (This code seems to have been here since forever,
unlike the bug, and I don't really think it explains the observed
behaviour). Still, it makes it hard to investigate the problem.
2017-07-05 00:28:43 +01:00
Caleb James DeLisle 75eba3b07d Fix TravisCI tests for PR #2301 2017-06-23 15:15:18 +02:00
Caleb James DeLisle 1591eddaea Add configuration parameter to allow redaction of content from push messages for google/apple devices 2017-06-23 13:01:04 +02:00
Erik Johnston cca94272fa Fix typo when getting app name 2017-06-06 11:50:07 +01:00
Erik Johnston 66b121b2fc Fix wrong number of arguments 2017-06-06 11:46:38 +01:00
Erik Johnston 11c2a3655f Only load jinja2 templates once
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
2017-05-22 17:48:58 +01:00
Erik Johnston 6489455bed Comment 2017-05-22 16:22:04 +01:00
Erik Johnston d668caa79c Remove spurious log level guards 2017-05-22 16:21:06 +01:00
Erik Johnston 7fb80b5eae Check if current event is a membership event 2017-05-22 15:02:12 +01:00
Erik Johnston 2d17b09a6d Add debug logging 2017-05-22 15:01:36 +01:00
Erik Johnston 24c8f38784 Comment 2017-05-22 14:59:27 +01:00
Erik Johnston 25f03cf8e9 Use tuple unpacking 2017-05-22 14:58:22 +01:00
Erik Johnston 270e1c904a Speed up calculating push rules 2017-05-19 16:51:05 +01:00
Erik Johnston 1c1c0257f4 Move invalidation cb to its own structure 2017-05-19 11:44:11 +01:00
Erik Johnston cafe659f72 Store ActionGenerator in HomeServer 2017-05-19 10:09:56 +01:00
Erik Johnston 72ed8196b3 Don't push users who have left 2017-05-18 17:48:36 +01:00
Erik Johnston 760625acba Make get_if_app_services_interested_in_user faster 2017-05-18 16:34:44 +01:00
Erik Johnston c57789d138 Remove size of push get_rules cache 2017-05-18 16:17:23 +01:00
Erik Johnston 056ba9b795 Add comment 2017-05-18 11:45:56 +01:00
Erik Johnston 66d8ffabbd Faster push rule calculation via push specific cache
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
Erik Johnston 78f306a6f7 Revert "Speed up filtering of a single event in push"
This reverts commit 421fdf7460.
2017-05-08 13:07:41 +01:00
Erik Johnston 421fdf7460 Speed up filtering of a single event in push 2017-04-28 09:52:36 +01:00
Erik Johnston acb58bfb6a fix up 2017-04-25 15:39:19 +01:00
Erik Johnston f7181615f2 Don't specify default as dict 2017-04-25 15:22:59 +01:00
Erik Johnston d9aa645f86 Reduce size of joined_user cache
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.

We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Daniel Dent 5058292537 Support authenticated SMTP
Closes (SYN-714) #1385

Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>
2017-04-05 21:01:08 -07:00
Erik Johnston 9cee0ce7db Merge pull request #2075 from matrix-org/erikj/cache_speed
Speed up cached function access
2017-03-31 10:10:56 +01:00
Erik Johnston 86780a8bc3 Don't convert to deferreds when not necessary 2017-03-30 14:14:36 +01:00
Erik Johnston a3810136fe Cache glob to regex at a higher level for push 2017-03-29 15:53:14 +01:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston 8ffbe43ba1 Get current state by using current_state_events table 2017-03-10 17:39:35 +00:00
Erik Johnston 7455ba436a Ensure we pass positive ints to delay function 2017-02-22 12:08:14 +00:00
Erik Johnston 51adaac953 Fix email push in pusher worker
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
David Baker 97d3918377 Merge pull request #1811 from aperezdc/unhardcode-riot-urls
Allow configuring the Riot URL used in notification emails
2017-01-18 14:38:49 +00:00
Erik Johnston 04006bb7f0 Get state at event rather than for room in push 2017-01-17 14:31:21 +00:00
Adrian Perez de Castro a3e4a198e3
Allow configuring the Riot URL used in notification emails
The URLs used for notification emails were hardcoded to use either matrix.to
or vector.im; but for self-hosted setups where Riot is also self-hosted it
may be desirable to allow configuring an alternative Riot URL.

Fixes #1809.

Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>
2017-01-13 17:12:04 +02:00
Erik Johnston 52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston 8ca05b5755 Fix push notifications for a single unread message 2016-10-18 10:57:33 +01:00
Erik Johnston f0ca088280 Reduce redundant database work in email pusher
Update the last stream ordering if the
`get_unread_push_actions_for_user_in_range_for_email` returns no new
push actions. This reduces the range that it needs to check next
iteration.
2016-10-18 10:52:47 +01:00
Erik Johnston 1d107d8484 Fix email push notifs being dropped
A lot of email push notifications were failing to be sent due to an
exception being thrown along one of the (many) paths. This was due to a
change where we moved from pulling out the full state for each room, but
rather pulled out the event ids for the state and separately loaded the
full events when needed.
2016-10-13 13:40:38 +01:00
Erik Johnston 49cf205dc7 _id field must uniquely identify different conditions 2016-09-19 10:34:01 +01:00
Matthew Hodgson 3f6ec271ba proposal for notifying on e2e events 2016-09-17 22:05:06 +01:00
Erik Johnston 6a5ded5988 Ensure we don't mutate the cache of push rules 2016-09-15 09:16:13 +01:00
Erik Johnston 3847fa38c4 Make bulk_get_push_rules_for_room use get_joined_users_from_context cache 2016-09-05 10:02:38 +01:00
Erik Johnston 44982606ee Merge pull request #1060 from matrix-org/erikj/state_ids
Assign state groups in state handler.
2016-09-01 14:20:42 +01:00
Erik Johnston c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston d80f64d370 Fix email notifs by adding missing param 2016-08-30 21:46:39 +01:00
Erik Johnston 55e17d3697 Fix push room names for rooms with only an alias 2016-08-30 11:19:59 +01:00
Erik Johnston bed10f9880 Use state handler instead of get_users_in_room/get_joined_hosts 2016-08-26 14:54:30 +01:00
Erik Johnston a234e895cf Fix room name in email notifs 2016-08-26 14:10:21 +01:00
Erik Johnston c7cd35d682 Typo 2016-08-26 11:23:58 +01:00
Erik Johnston 7356d52e73 Fix up push to use get_current_state_ids 2016-08-25 18:35:49 +01:00
Erik Johnston a3dc1e9cbe Replace context.current_state with context.current_state_ids 2016-08-25 17:32:22 +01:00
Erik Johnston 9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00