Commit Graph

11394 Commits

Author SHA1 Message Date
Erik Johnston bb3d536087 Newsfile 2018-07-10 17:28:31 +01:00
Erik Johnston 05f5dabc10 Use stream cache in get_linearized_receipts_for_room
This avoids us from uncessarily hitting the database when there has been
no change for the room
2018-07-10 17:22:42 +01:00
Richard van der Hoff c3c29aa196
Attempt to include db threads in cpu usage stats (#3496)
Let's try to include time spent in the DB threads in the per-request/block cpu
usage metrics.
2018-07-10 16:12:36 +01:00
Richard van der Hoff 55370331da
Refactor logcontext resource usage tracking (#3501)
Factor out the resource usage tracking out to a separate object, which can be
passed around and copied independently of the logcontext itself.
2018-07-10 13:56:07 +01:00
Matthew Hodgson 16b10666e7 another typo 2018-07-10 12:28:42 +01:00
Matthew Hodgson 4ea391a6ae typo (i think) 2018-07-10 12:08:09 +01:00
Richard van der Hoff b1fe697b3c
Merge pull request #3497 from matrix-org/rav/measure_fetch_event_loop
Add CPU metrics for _fetch_event_list
2018-07-10 10:00:24 +01:00
Oleg Girko 6c1ec5a1bd Use more portable syntax using attrs package.
Newer syntax

    attr.ib(factory=dict)

is just a syntactic sugar for

    attr.ib(default=attr.Factory(dict))

It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.

We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-07-10 00:38:49 +01:00
Richard van der Hoff f3b3b9dd8f changelog 2018-07-09 18:16:52 +01:00
Richard van der Hoff e31e5dee38 Add CPU metrics for _fetch_event_list
add a Measure block on _fetch_event_list, in the hope that we can better
measure CPU usage here.
2018-07-09 18:15:54 +01:00
Richard van der Hoff 395fa8d1fd
Merge pull request #3464 from matrix-org/hawkowl/isort-run
Run isort on Synapse
2018-07-09 10:24:43 +01:00
Amber Brown 09477bd884 changelog 2018-07-09 16:09:37 +10:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown 2ee9f1bd1a
Add an isort configuration (#3463) 2018-07-09 16:05:21 +10:00
Amber Brown 1241156c82 changelog 2018-07-07 10:48:30 +10:00
Amber Brown 3060bcc8e9 version 2018-07-07 10:48:06 +10:00
Amber Brown e845fd41c2
Correct attrs package name in requirements (#3492) 2018-07-07 10:46:59 +10:00
Richard van der Hoff 20ff89d6e1 Synapse 0.32.1 (2018-07-06)
===========================
 
 Bugfixes
 --------
 
 - Add explicit dependency on netaddr ([#3488](https://github.com/matrix-org/synapse/issues/3488))
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJbP5AkAAoJEIofk9V1tejV7a0H/A2Xn3IMCA/tUMi8J2WFXnR/
 gpyprut24fAvU+NruDDKwYs81vr5pvYHpfc6TTEhKnrna5V26S+hXb15ZwVUm5VK
 drjCt1iexe2sZ0foeK1kmUmMMKbg8QXh+qUO2vXFRyRiOs9ly5JPehuNr7AgJron
 htjlGakD5Kmc5lnQHMK6nnR6nEzs6gQnIUHkshcPNxKRZ8lwUm/0PT2zGsspvjjJ
 oI/pKp7PCwTBZAauX+eT2h+AGLgvZxurht8N/9c4ls8JyoL50/g1c3eb58ieOOcm
 I8cbkLGFPe1YXKRm+lyIMJGvbk3c8uDOl7IeXi+qyekluOqdjy0c2ftLzlY98rc=
 =mORK
 -----END PGP SIGNATURE-----

Merge tag 'v0.32.1'

Synapse 0.32.1 (2018-07-06)
===========================

Bugfixes
--------

- Add explicit dependency on netaddr ([#3488](https://github.com/matrix-org/synapse/issues/3488))
2018-07-06 16:56:23 +01:00
Richard van der Hoff 1cfc2c4790 Prepare 0.32.1 release 2018-07-06 16:50:52 +01:00
Richard van der Hoff 2087d5d046
Merge pull request #3488 from matrix-org/rav/fix-netaddr-dep
Add explicit dependency on netaddr
2018-07-06 16:39:56 +01:00
Richard van der Hoff 1464a0578a Add explicit dependency on netaddr
the dependencies file, causing failures on upgrade (and presumably for new
installs).
2018-07-06 16:27:17 +01:00
Neil Johnson 277c561766 0.32.0 version bump, update changelog 2018-07-06 15:07:29 +01:00
Amber Brown 89690aaaeb changelog 2018-07-05 20:46:40 +10:00
Amber Brown be8b32dbc2 ACL changelog 2018-07-05 20:45:12 +10:00
Amber Brown d196fe42a9 bump version to 0.32.0rc1 2018-07-05 20:22:35 +10:00
Neil Johnson feef8461d1 Merge remote-tracking branch 'hera/rav/server_acls' into develop 2018-07-05 11:04:01 +01:00
Erik Johnston 1a88640677
Merge pull request #3483 from matrix-org/rav/more_server_name_validation
More server_name validation
2018-07-05 10:04:20 +01:00
Richard van der Hoff 3cf3e08a97 Implementation of server_acls
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
2018-07-04 19:06:20 +01:00
Richard van der Hoff 546bc9e28b More server_name validation
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.

Also, use it to verify the server name in the config file.
2018-07-04 18:59:51 +01:00
Richard van der Hoff f192a93875
Merge pull request #3481 from matrix-org/rav/fix_cachedescriptor_test
Reinstate lost run_on_reactor in unit test
2018-07-04 18:55:33 +01:00
Erik Johnston 13f7adf84b
Merge pull request #3473 from matrix-org/erikj/thread_cache
Invalidate cache on correct thread
2018-07-04 10:11:38 +01:00
Erik Johnston 40252d13d1
Merge pull request #3474 from matrix-org/erikj/py3_auth
Fix up auth check
2018-07-04 09:41:33 +01:00
Richard van der Hoff ea555d5633 Reinstate lost run_on_reactor in unit test
a61738b removed a call to run_on_reactor from a unit test, but that call was
doing something useful, in making the function in question asynchronous.

Reinstate the call and add a check that we are testing what we wanted to be
testing.
2018-07-04 09:40:01 +01:00
Richard van der Hoff c4e7ad0e0f Add changelog 2018-07-04 09:15:45 +01:00
Richard van der Hoff a4ab491371
Merge branch 'develop' into rav/drop_re_signing_hacks 2018-07-04 07:13:38 +01:00
Richard van der Hoff 508196e08a
Reject invalid server names (#3480)
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
2018-07-03 14:36:14 +01:00
Richard van der Hoff f741630847
Merge pull request #3470 from matrix-org/matthew/fix-utf8-logging
don't mix unicode strings with utf8-in-byte-strings
2018-07-02 15:25:36 +01:00
Matthew Hodgson 6ec3aa2f72 news snippet 2018-07-02 13:43:34 +01:00
Erik Johnston abb183438c Correct newsfile 2018-07-02 13:12:38 +01:00
Erik Johnston f88dea577d Newsfile 2018-07-02 11:46:32 +01:00
Erik Johnston cea4662b13 Newsfile 2018-07-02 11:46:20 +01:00
Erik Johnston 2c33b55738 Avoid relying on int vs None comparison
Python 3 doesn't support comparing None to ints
2018-07-02 11:40:32 +01:00
Erik Johnston cbf82dddf1 Ensure that we define sender_domain 2018-07-02 11:37:57 +01:00
Erik Johnston 3905c693c5 Invalidate cache on correct thread 2018-07-02 11:36:44 +01:00
Matthew Hodgson fc4f8f33be replace invalid utf8 with \ufffd 2018-07-02 11:33:02 +01:00
Matthew Hodgson 1c867f5391 a fix which doesn't NPE everywhere 2018-07-01 11:56:33 +01:00
Matthew Hodgson f131bf8d3e don't mix unicode strings with utf8-in-byte-strings
otherwise we explode with:

```
Traceback (most recent call last):
  File /usr/lib/python2.7/logging/handlers.py, line 78, in emit
    logging.FileHandler.emit(self, record)
  File /usr/lib/python2.7/logging/__init__.py, line 950, in emit
    StreamHandler.emit(self, record)
  File /usr/lib/python2.7/logging/__init__.py, line 887, in emit
    self.handleError(record)
  File /usr/lib/python2.7/logging/__init__.py, line 810, in handleError
    None, sys.stderr)
  File /usr/lib/python2.7/traceback.py, line 124, in print_exception
    _print(file, 'Traceback (most recent call last):')
  File /usr/lib/python2.7/traceback.py, line 13, in _print
    file.write(str+terminator)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_io.py, line 170, in write
    self.log.emit(self.level, format=u{log_io}, log_io=line)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_logger.py, line 144, in emit
    self.observer(event)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_observer.py, line 136, in __call__
    errorLogger = self._errorLoggerForObserver(brokenObserver)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_observer.py, line 156, in _errorLoggerForObserver
    if obs is not observer
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_observer.py, line 81, in __init__
    self.log = Logger(observer=self)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_logger.py, line 64, in __init__
    namespace = self._namespaceFromCallingContext()
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_logger.py, line 42, in _namespaceFromCallingContext
    return currentframe(2).f_globals[__name__]
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/compat.py, line 93, in currentframe
    for x in range(n + 1):
RuntimeError: maximum recursion depth exceeded while calling a Python object
Logged from file site.py, line 129
  File /usr/lib/python2.7/logging/__init__.py, line 859, in emit
    msg = self.format(record)
  File /usr/lib/python2.7/logging/__init__.py, line 732, in format
    return fmt.format(record)
  File /usr/lib/python2.7/logging/__init__.py, line 471, in format
    record.message = record.getMessage()
  File /usr/lib/python2.7/logging/__init__.py, line 335, in getMessage
    msg = msg % self.args
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4: ordinal not in range(128)
Logged from file site.py, line 129
```

...where the logger apparently recurses whilst trying to log the error, hitting the
maximum recursion depth and killing everything badly.
2018-07-01 05:08:58 +01:00
Matthew Hodgson 75d4986a8c
Merge pull request #3467 from matrix-org/hawkowl/contributor-requirements
Clarify "real name" in contributor requirements
2018-06-30 00:21:10 +01:00
Amber Brown 7c0cdd330f topfile 2018-06-29 14:13:15 +01:00
Erik Johnston e3b4043800
Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking
Check the state of prev_events a bit more thoroughly when coming over federation
2018-06-29 13:55:02 +01:00