Commit Graph

155 Commits

Author SHA1 Message Date
Erik Johnston 9d1e4942ab
Fix typing for notifier (#8064) 2020-08-12 14:03:08 +01:00
Erik Johnston a1e9bb9eae
Add typing info to Notifier (#8058) 2020-08-11 19:40:02 +01:00
Patrick Cloke e19de43eb5
Convert streams to async. (#8014) 2020-08-04 07:21:47 -04:00
Patrick Cloke 38e1fac886
Fix some spelling mistakes / typos. (#7811) 2020-07-09 09:52:58 -04:00
Erik Johnston 1a1da60ad2
Fix new flake8 errors (#7470) 2020-05-12 11:20:48 +01:00
Patrick Cloke b0cbc57375
Convert the synapse.notifier module to async/await. (#7395) 2020-05-01 15:14:49 -04:00
Erik Johnston 3eab76ad43
Don't relay REMOTE_SERVER_UP cmds to same conn. (#7352)
For direct TCP connections we need the master to relay REMOTE_SERVER_UP
commands to the other connections so that all instances get notified
about it. The old implementation just relayed to all connections,
assuming that sending back to the original sender of the command was
safe. This is not true for redis, where commands sent get echoed back to
the sender, which was causing master to effectively infinite loop
sending and then re-receiving REMOTE_SERVER_UP commands that it sent.

The fix is to ensure that we only relay to *other* connections and not
to the connection we received the notification from.

Fixes #7334.
2020-04-29 14:10:59 +01:00
Erik Johnston a8a50f5b57
Wake up transaction queue when remote server comes back online (#6706)
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
2020-01-17 10:27:19 +00:00
Erik Johnston 8437e2383e Port SyncHandler to async/await 2019-12-05 17:58:25 +00:00
Erik Johnston 69f0054ce6 Port to use state storage 2019-10-30 14:46:54 +00:00
Andrew Morgan 4548d1f87e
Remove unnecessary parentheses around return statements (#5931)
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff c7325776a7 Remove redundant PreserveLoggingContext
Both (!) things that register as replication listeners do the right thing wrt
logcontexts, so this is redundant.
2019-03-04 18:31:18 +00:00
Richard van der Hoff daa10e3e66 Remove unused `wait_for_replication` method
I guess this was used once? It's not now, anyway.
2019-03-04 18:27:32 +00:00
Amber Brown b69216f768
Make the metrics less racy (#4061) 2018-10-19 21:45:45 +11:00
Richard van der Hoff a87d419a85 Run notify_app_services as a bg process
This ensures that its resource usage metrics get recorded somewhere rather than
getting lost.

(It also fixes an error when called from a nested logging context which
completes before the bg process)
2018-09-26 17:00:40 +01:00
Erik Johnston 80d2d50f47 Fixup 2018-09-19 11:19:47 +01:00
Erik Johnston 9407bcf37a Replace custom DeferredTimeoutError with defer.TimeoutError 2018-09-19 11:07:29 +01:00
Erik Johnston a334e1cace Update to use new timeout function everywhere.
The existing deferred timeout helper function (and the one into twisted)
suffer from a bug when a deferred's canceller throws an exception, #3842.

The new helper function doesn't suffer from this problem.
2018-09-19 10:39:40 +01:00
Amber Brown b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Matthew Hodgson 5797f5542b WIP to announce deleted devices over federation
Previously we queued up the poke correctly when the device was deleted,
but then the actual EDU wouldn't get sent, as the device was no longer known.
Instead, we now send EDUs for deleted devices too if there's a poke for them.
2018-07-12 01:32:39 +01:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown 77ac14b960
Pass around the reactor explicitly (#3385) 2018-06-22 09:37:10 +01:00
Amber Brown 071206304d cleanup pep8 errors 2018-05-22 16:54:22 -05:00
Amber Brown 85ba83eb51 fixes 2018-05-22 16:28:23 -05:00
Amber Brown df9f72d9e5 replacing portions 2018-05-21 19:47:37 -05:00
Richard van der Hoff 6146332387 Merge remote-tracking branch 'origin/develop' into rav/deferred_timeout 2018-04-27 14:18:00 +01:00
Richard van der Hoff 9d2c1b8429 Backport deferred.addTimeout
Twisted 16.0 doesn't have addTimeout, so let's backport it.
2018-04-27 12:52:30 +01:00
Richard van der Hoff 9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Richard van der Hoff 1ea904b9f0 Use deferred.addTimeout instead of time_bound_deferred
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
Adrian Tschira f63ff73c7f add __bool__ alias to __nonzero__ methods
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
Richard van der Hoff d4fb4f7c52 Clear logcontext before starting fed txn queue runner
These processes take a long time compared to the request, so there is lots of
"Entering|Restoring dead context" in the logs. Let's try to shut it up a bit.
2017-11-28 15:26:14 +00:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston ecdd2a3658 Don't start user_directory handling on workers 2017-06-07 12:02:53 +01:00
Erik Johnston eeb2f9e546 Add user_directory to database 2017-05-31 11:51:01 +01:00
Erik Johnston 62b89daac6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server 2017-04-04 09:46:16 +01:00
Richard van der Hoff 65e1683680 Remove spurious PreserveLoggingContext
In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its
subroutines with the logcontext set.
2017-04-03 15:42:38 +01:00
Richard van der Hoff feb496056e preserve_fn some deferred-returning things
In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which
return deferreds, so that it is safe to call it with an active logcontext.
2017-04-03 15:41:17 +01:00
Richard van der Hoff 773e1c6d68 Remove spurious @preserve_fn decorators
Remove `@preserve_fn` decorators on `on_new_room_event`,
`_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and
`on_new_replication_data` - none of these functions return a deferred, and the
decorator does nothing unless the wrapped function returns a deferred, so the
decorator was a no-op.
2017-04-03 15:14:11 +01:00
Erik Johnston 1df7c28661 Use callbacks to notify tcp replication rather than deferreds 2017-03-31 15:42:51 +01:00
Erik Johnston 7984708a55 Add a simple hook to wait for replication traffic 2017-03-30 11:57:52 +01:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston 6c82de5100 Format presence events on the edges instead of reformatting them multiple times 2017-03-15 14:27:34 +00:00
Erik Johnston 872e75a3d5 Add some metrics on notifier 2017-03-15 10:56:51 +00:00
Erik Johnston fd2eef49c8 Reduce spurious calls to generate sync 2017-03-14 16:33:45 +00:00
Erik Johnston 54a79c1d37 Make presence.get_new_events a bit faster
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
Erik Johnston fbaf868f62 Correctly handle timeout errors 2016-12-09 16:30:29 +00:00
Erik Johnston be14c24cea Fix rare notifier bug where listeners dont timeout
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00