Commit Graph

6850 Commits

Author SHA1 Message Date
Erik Johnston 31aa7bd8d1 Move type into key 2017-10-13 13:47:38 +01:00
Erik Johnston ad1911bbf4 Comment 2017-10-13 13:47:05 +01:00
Erik Johnston c021c39cbd Remove spurious addition 2017-10-13 13:46:53 +01:00
Erik Johnston 1f43d22397 Don't needlessly rename variable 2017-10-13 11:42:07 +01:00
Erik Johnston a675bd08bd Add paths back in... 2017-10-13 11:41:06 +01:00
Erik Johnston 4d7e1dde70 Remove unnecessary diff 2017-10-13 11:36:32 +01:00
Erik Johnston ae5d18617a Make things be absolute paths again 2017-10-13 11:35:44 +01:00
Erik Johnston 9732ec6797 s/write_to_file/write_to_file_and_backup/ 2017-10-13 11:34:41 +01:00
Erik Johnston 0e28281a02 Fix up 2017-10-13 11:33:49 +01:00
Erik Johnston 505371414f Fix up thumbnailing function 2017-10-13 11:23:53 +01:00
Erik Johnston e3428d26ca Fix typo 2017-10-13 10:39:59 +01:00
Erik Johnston 35332298ef Fix up comments 2017-10-13 10:39:32 +01:00
Erik Johnston 64db043a71 Move makedirs to thread 2017-10-13 10:25:01 +01:00
Erik Johnston b60859d6cc Use make_deferred_yieldable 2017-10-13 10:24:19 +01:00
Erik Johnston d76621a47b Fix comments 2017-10-12 18:16:25 +01:00
Erik Johnston 4ae85ae121 Don't close prematurely.. 2017-10-12 17:57:31 +01:00
Erik Johnston cc505b4b5e getvalue closes buffer 2017-10-12 17:52:30 +01:00
Erik Johnston 1259a76047 Get len before close 2017-10-12 17:39:23 +01:00
Erik Johnston 802ca12d05 Don't close file prematurely 2017-10-12 17:37:21 +01:00
Erik Johnston e283b555b1 Copy everything to backup 2017-10-12 17:31:24 +01:00
Erik Johnston b77a13812c Typo 2017-10-12 15:32:32 +01:00
Erik Johnston 6dfde6d485 Remove dead code 2017-10-12 15:30:26 +01:00
Erik Johnston c8eeef6947 Fix typos 2017-10-12 15:28:24 +01:00
Erik Johnston 67cb89fbdf Fix typo 2017-10-12 15:23:41 +01:00
Erik Johnston bf4fb1fb40 Basic implementation of backup media store 2017-10-12 15:20:59 +01:00
hera f807f7f804 log when we get an exception handling replication updates 2017-10-12 11:51:24 +01:00
David Baker b8d8ed1ba9 Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages
Allow error strings from spam checker
2017-10-12 10:31:03 +01:00
Richard van der Hoff cc794d60e7 Merge pull request #2532 from matrix-org/rav/fix_linearizer
Fix stackoverflow and logcontexts from linearizer
2017-10-11 17:29:32 +01:00
Richard van der Hoff 8dd0c85ac5 Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling
log pdu_failures from incoming transactions
2017-10-11 17:29:14 +01:00
Richard van der Hoff 76fa695241 Merge pull request #2515 from matrix-org/rav/fix_receipt_logcontext
A logformatter which includes the stack when the exception was caught when
logging exceptions.
2017-10-11 17:28:01 +01:00
Richard van der Hoff f30c4ed2bc logformatter: fix AttributeError
make sure we have the relevant fields before we try to log them.
2017-10-11 17:26:17 +01:00
Erik Johnston b752507b48 Fix fetching remote summaries 2017-10-11 16:59:18 +01:00
Erik Johnston 818b08d0e4 peeeeeeeeep8888888888888888888888888888 2017-10-11 15:54:00 +01:00
Erik Johnston ea18996f54 Fix group stream replication
The stream update functions expect the storage function to return a list
of tuples.
2017-10-11 15:44:39 +01:00
Richard van der Hoff 68fd82e840 Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext
fix a logcontext leak in read receipt handling
2017-10-11 15:08:53 +01:00
Richard van der Hoff 4fad8efbfb Fix stackoverflow and logcontexts from linearizer
1. make it not blow out the stack when there are more than 50 things waiting
   for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505.

2. Make it not mess up the log contexts.
2017-10-11 15:05:05 +01:00
David Baker b78bae2d51 fix isinstance 2017-10-11 14:49:09 +01:00
Erik Johnston 271f5601f3 Fix typo in invite to group 2017-10-11 14:45:33 +01:00
David Baker c3b7a45e84 Allow error strings from spam checker 2017-10-11 14:39:22 +01:00
Richard van der Hoff c3e190ce67 fix a logcontext leak in read receipt handling 2017-10-11 14:37:20 +01:00
Richard van der Hoff b75d443caf log pdu_failures from incoming transactions
... even if we have no EDUs.

This appears to have been introduced in
476899295f.
2017-10-11 14:36:13 +01:00
Erik Johnston 27e727a146 Fix typo 2017-10-11 14:32:40 +01:00
Erik Johnston 4ce4379235 Fix attestations to check correct server name 2017-10-11 14:11:43 +01:00
Erik Johnston c2c47550f9 Fix schema delta versions 2017-10-11 13:23:15 +01:00
Erik Johnston 535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston dfbf73408c Merge pull request #2501 from matrix-org/dbkr/channel_notifications
Support for channel notifications
2017-10-11 13:19:29 +01:00
Erik Johnston bc7f3eb32f Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
Erik Johnston ec954f47fb Validate room ids 2017-10-11 13:15:44 +01:00
David Baker 81a5e0073c pep8 2017-10-10 15:53:34 +01:00
David Baker ab1bc9bf5f Don't KeyError if no power_levels event 2017-10-10 15:34:05 +01:00
David Baker 0f1eb3e914 Use notification levels in power_levels
Rather than making the condition directly require a specific power
level. This way the level require to notify a room can be configured
per room.
2017-10-10 15:23:00 +01:00
Erik Johnston 84e27a592d Merge pull request #2490 from matrix-org/erikj/drop_left_room_events
Ignore incoming events for rooms that we have left
2017-10-10 11:58:32 +01:00
David Baker c9f034b4ac There was already a constant for this
also update copyright
2017-10-10 11:47:10 +01:00
David Baker a9f9d68631 More optimisation 2017-10-10 11:38:31 +01:00
David Baker 707374d5dc What year is it!? Who's the president!? 2017-10-10 11:21:41 +01:00
David Baker 89fa00ddff Merge branch 'develop' into dbkr/channel_notifications 2017-10-10 11:20:17 +01:00
Richard van der Hoff 6a6cc27aee fed server: process PDUs for different rooms in parallel
With luck, this will give a real-time improvement when there are many rooms and
the server ends up calling out to fetch missing events.
2017-10-09 18:30:31 +01:00
Richard van der Hoff 4c7c4d4061 Fed server: use a linearizer for ongoing transactions
We don't want to process the same transaction multiple times concurrently, so
use a linearizer.
2017-10-09 18:30:10 +01:00
Richard van der Hoff ba5b9b80a5 fed server: refactor on_incoming_transaction
Move as much as possible to after the have_responded check, and reduce the
number of times we iterate over the pdu list.
2017-10-09 18:10:53 +01:00
Richard van der Hoff a6e3222fe5 Fed server: Move origin-check code to _handle_received_pdu
The response-building code expects there to be an entry in the `results` list
for each entry in the pdu_list, so the early `continue` was messing this
up. That doesn't really matter, because all that the federation client does is
log any errors, but it's pretty poor form.
2017-10-09 17:53:32 +01:00
Richard van der Hoff 3cc852d339 Fancy logformatter to format exceptions better
This is a bit of an experimental change at this point; the idea is to see if it
helps us track down where our stack overflows are coming from by logging the
stack when the exception was caught and turned into a Failure. (We'll also need
edf2704420).

If we deploy this, we'll be able to enable it via the log config yaml.
2017-10-09 17:44:42 +01:00
Richard van der Hoff 0eeaa25694 Merge pull request #2508 from matrix-org/rav/federation_queue_logcontexts
Fix up logcontext handling in (federation) TransactionQueue
2017-10-09 17:43:48 +01:00
Richard van der Hoff aa3fac8057 Merge pull request #2507 from matrix-org/rav/execute_concurrently_log_contexts
Fix logcontext handling for concurrently_execute
2017-10-09 17:43:32 +01:00
Erik Johnston e8496efe84 Fix up comment 2017-10-09 15:17:34 +01:00
Richard van der Hoff 01bbacf3c4 Fix up logcontext handling in (federation) TransactionQueue
Avoid using preserve_context_over_function, which has problems with respect to
logcontexts.
2017-10-06 22:39:25 +01:00
Richard van der Hoff 148428ce76 Fix logcontext handling for concurrently_execute
Avoid preserve_context_over_deferred, which is broken.
2017-10-06 22:24:28 +01:00
Richard van der Hoff c8f568ddf9 Fix up deferred handling in federation.py
* Avoid preserve_context_over_deferred, which is broken

* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
  failure" erros
2017-10-06 22:14:24 +01:00
Richard van der Hoff 3ddda939d3 some comments in the state res code 2017-10-05 14:58:17 +01:00
David Baker f878e6f8af Spam checking: add the invitee to user_may_invite 2017-10-05 14:02:28 +01:00
David Baker 269af961e9 Make be faster 2017-10-05 13:27:12 +01:00
David Baker ed80c6b6cc Add fastpath optimisation 2017-10-05 13:20:22 +01:00
David Baker e433393c4f pep8 2017-10-05 13:08:02 +01:00
David Baker 985ce80375 They're called rooms 2017-10-05 13:03:44 +01:00
David Baker b9b9714fd5 Get rule type right 2017-10-05 13:02:19 +01:00
David Baker fa969cfdde Support for channel notifications
Add condition type to check the sender's power level and add a base
rule using it for @channel notifications.
2017-10-05 12:39:18 +01:00
David Baker 0c8da8b519 Use better method for word boundary searching
From ebc95667b8
2017-10-05 11:57:43 +01:00
Erik Johnston eaaa837e00 Don't corrupt cache 2017-10-05 11:43:22 +01:00
David Baker cbe3c3fdd4 pep8 2017-10-05 11:43:10 +01:00
David Baker 6748f0a579 Fix notif kws that start/end with non-word chars
Only prepend / append word bounary characters if the search
expression starts or ends with a word character, otherwise they
don't work because there's no word bounary between whitespace and
a non-word char.
2017-10-05 11:33:30 +01:00
David Baker d8ce68b09b spam check room publishing 2017-10-04 14:29:33 +01:00
David Baker 78d4ced829 un-double indent 2017-10-04 12:44:27 +01:00
David Baker 197c14dbcf Add room creation checks to spam checker
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
David Baker 1e2ac54351 s/roomid/room_id/ 2017-10-03 17:41:38 +01:00
David Baker 1e375468de pass room id too 2017-10-03 17:13:14 +01:00
David Baker c2c188b699 Federation was passing strings anyway
so pass string everywhere
2017-10-03 15:46:19 +01:00
David Baker c46a0d7eb4 this shouldn't be debug 2017-10-03 15:20:14 +01:00
David Baker bd769a81e1 better logging 2017-10-03 15:16:40 +01:00
David Baker 537088e7dc Actually write warpper function 2017-10-03 14:28:12 +01:00
David Baker 41fd9989a2 Skip spam check for admin users 2017-10-03 14:17:44 +01:00
Erik Johnston 11d62f43c9 Invalidate cache 2017-10-03 14:12:28 +01:00
Erik Johnston e4ab96021e Update comments 2017-10-03 14:10:41 +01:00
David Baker 2a7ed700d5 Fix param name & lint 2017-10-03 14:04:10 +01:00
David Baker 84716d267c Allow spam checker to reject invites too 2017-10-03 13:56:43 +01:00
Erik Johnston f2da6df568 Remove spurious line feed 2017-10-03 11:31:06 +01:00
Erik Johnston 30848c0fcd Ignore incoming events for rooms that we have left
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.

However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.

This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
Erik Johnston e585c83209 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-10-02 18:11:24 +01:00
Erik Johnston 6c1bb1601e Bump version and changelog 2017-10-02 18:05:17 +01:00
Erik Johnston ea87cb1ba5 Make 'affinity' package optional 2017-10-02 18:03:59 +01:00
Erik Johnston 3fed5bb25f Move quit_with_error 2017-10-02 17:59:34 +01:00
David Baker 27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00