Bugfixes

-------- - Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723)) -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEIQBQJ4l+yK4dlKkFIwi0edOSShEFAlt6/yMACgkQIwi0edOS ShFrzAgAhLA74fClydGLfhPNCXIw12Amv2ASA6+r0ukqjYQ/9MZ0CRsL+PMZJTk6 Z75PK3TkX38zSpPaUx9HdMQO/yE+rpI99ZxlBM+8jzsf1jyF8ljqxSfixXUlaXX9 698PQQ3f+IDoSs59NKrHvoWhUThaBpf/1GwRxG5mUcm2oKJItNMHk/uLurNCaIIZ YSwWcxvZdluOMX0WrlF/S524DqtgULcRf3E9x3mumobkVGxI3UH73hXgGUcPi7XV ttLuy7fVbV2Cun/wvjBqD7kc8LNbKwSaT9y+/xS0D5QYjph4rpQiTG7S5rtPLPMU Z2lL1C5ggBCkD43g3orDvehbNWCL5g== =0vEe -----END PGP SIGNATURE----- Merge tag 'v0.33.3rc2' into develop Bugfixes -------- - Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723))
2018-08-21 20:30:09 +01:00 · 2018-08-21 20:30:09 +01:00 · a52f276990
parent 46c832eaac 3b5b64ac99
commit a52f276990
46 changed files with 91 additions and 46 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -1,3 +1,76 @@
 Synapse 0.33.3rc2 (2018-08-21)
 ==============================
 Bugfixes
 --------
 - Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723))
 Synapse 0.33.3rc1 (2018-08-21)
 ==============================
 Features
 --------
 - Add support for the SNI extension to federation TLS connections ([\#1491](https://github.com/matrix-org/synapse/issues/1491))
 - Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184))
 - speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568))
 - implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574))
 - Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589))
 - Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633))
 - Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653))
 - Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654))
 - Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655))
 - Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662))
 - Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664))
 - Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670))
 - set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687))
 - Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694))
 - For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708))
 Bugfixes
 --------
 - Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658))
 - Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661))
 - Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676))
 - Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677))
 - Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681))
 - Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689))
 - Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692))
 - Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700))
 - Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701))
 - Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705), [\#3713](https://github.com/matrix-org/synapse/issues/3713))
 - Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710))
 - Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719))
 Deprecations and Removals
 -------------------------
 - The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst). ([\#3703](https://github.com/matrix-org/synapse/issues/3703))
 Internal Changes
 ----------------
 - The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423))
 - Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632))
 - Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647))
 - Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660))
 - Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668))
 - Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669))
 - Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678))
 - Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679))
 - Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684))
 - Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690))
 - add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707))
 - Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709))
 - Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712))
 Synapse 0.33.2 (2018-08-09)
 ===========================
--- a/changelog.d/1491.feature
+++ b/changelog.d/1491.feature
@ -1 +0,0 @@
 Add support for the SNI extension to federation TLS connections
--- a/changelog.d/3184.feature
+++ b/changelog.d/3184.feature
@ -1 +0,0 @@
 Add /_media/r0/config
--- a/changelog.d/3423.misc
+++ b/changelog.d/3423.misc
@ -1 +0,0 @@
 The test suite now can run under PostgreSQL.
--- a/changelog.d/3568.feature
+++ b/changelog.d/3568.feature
@ -1 +0,0 @@
 speed up /members API and add `at` and `membership` params as per MSC1227
--- a/changelog.d/3574.feature
+++ b/changelog.d/3574.feature
@ -1 +0,0 @@
 implement `summary` block in /sync response as per MSC688
--- a/changelog.d/3589.feature
+++ b/changelog.d/3589.feature
@ -1 +0,0 @@
 Add lazy-loading support to /messages as per MSC1227
--- a/changelog.d/3632.misc
+++ b/changelog.d/3632.misc
@ -1 +0,0 @@
 Refactor HTTP replication endpoints to reduce code duplication
--- a/changelog.d/3633.feature
+++ b/changelog.d/3633.feature
@ -1 +0,0 @@
 Add ability to limit number of monthly active users on the server
--- a/changelog.d/3647.misc
+++ b/changelog.d/3647.misc
@ -1 +0,0 @@
 Tests now correctly execute on Python 3.
--- a/changelog.d/3653.feature
+++ b/changelog.d/3653.feature
@ -1 +0,0 @@
 Support more federation endpoints on workers
--- a/changelog.d/3654.feature
+++ b/changelog.d/3654.feature
@ -1 +0,0 @@
 Basic support for room versioning
--- a/changelog.d/3655.feature
+++ b/changelog.d/3655.feature
@ -1 +0,0 @@
 Ability to disable client/server Synapse via conf toggle
--- a/changelog.d/3658.bugfix
+++ b/changelog.d/3658.bugfix
@ -1 +0,0 @@
 Fix occasional glitches in the synapse_event_persisted_position metric
--- a/changelog.d/3660.misc
+++ b/changelog.d/3660.misc
@ -1 +0,0 @@
 Sytests can now be run inside a Docker container.
--- a/changelog.d/3661.bugfix
+++ b/changelog.d/3661.bugfix
@ -1 +0,0 @@
 Fix bug on deleting 3pid when using identity servers that don't support unbind API
--- a/changelog.d/3662.feature
+++ b/changelog.d/3662.feature
@ -1 +0,0 @@
 Ability to whitelist specific threepids against monthly active user limiting
--- a/changelog.d/3664.feature
+++ b/changelog.d/3664.feature
@ -1 +0,0 @@
 Add some metrics for the appservice and federation event sending loops
--- a/changelog.d/3668.misc
+++ b/changelog.d/3668.misc
@ -1 +0,0 @@
 Port over enough to Python 3 to allow the sytests to start.
--- a/changelog.d/3669.misc
+++ b/changelog.d/3669.misc
@ -1 +0,0 @@
 Update docker base image from alpine 3.7 to 3.8.
--- a/changelog.d/3670.feature
+++ b/changelog.d/3670.feature
@ -1 +0,0 @@
 Where server is disabled, block ability for locked out users to read new messages
--- a/changelog.d/3676.bugfix
+++ b/changelog.d/3676.bugfix
@ -1 +0,0 @@
 Make the tests pass on Twisted < 18.7.0
--- a/changelog.d/3677.bugfix
+++ b/changelog.d/3677.bugfix
@ -1 +0,0 @@
 Don’t ship recaptcha_ajax.js, use it directly from Google
--- a/changelog.d/3678.misc
+++ b/changelog.d/3678.misc
@ -1 +0,0 @@
 Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7.
--- a/changelog.d/3679.misc
+++ b/changelog.d/3679.misc
@ -1 +0,0 @@
 Synapse's tests are now formatted with the black autoformatter.
--- a/changelog.d/3681.bugfix
+++ b/changelog.d/3681.bugfix
@ -1 +0,0 @@
 Fixes test_reap_monthly_active_users so it passes under postgres
--- a/changelog.d/3684.misc
+++ b/changelog.d/3684.misc
@ -1 +0,0 @@
 Implemented a new testing base class to reduce test boilerplate.
--- a/changelog.d/3687.feature
+++ b/changelog.d/3687.feature
@ -1 +0,0 @@
 set admin uri via config, to be used in error messages where the user should contact the administrator
--- a/changelog.d/3689.bugfix
+++ b/changelog.d/3689.bugfix
@ -1 +0,0 @@
 Fix mau blocking calulation bug on login
--- a/changelog.d/3690.misc
+++ b/changelog.d/3690.misc
@ -1 +0,0 @@
 Rename MAU prometheus metrics
--- a/changelog.d/3692.bugfix
+++ b/changelog.d/3692.bugfix
@ -1 +0,0 @@
 Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users
--- a/changelog.d/3694.feature
+++ b/changelog.d/3694.feature
@ -1 +0,0 @@
 Synapse's presence functionality can now be disabled with the "use_presence" configuration option.
--- a/changelog.d/3700.bugfix
+++ b/changelog.d/3700.bugfix
@ -1 +0,0 @@
 Improve HTTP request logging to include all requests
--- a/changelog.d/3701.bugfix
+++ b/changelog.d/3701.bugfix
@ -1 +0,0 @@
 Avoid timing out requests while we are streaming back the response
--- a/changelog.d/3703.removal
+++ b/changelog.d/3703.removal
@ -1 +0,0 @@
 The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst).
--- a/changelog.d/3705.bugfix
+++ b/changelog.d/3705.bugfix
@ -1 +0,0 @@
 Support more federation endpoints on workers
--- a/changelog.d/3707.misc
+++ b/changelog.d/3707.misc
@ -1 +0,0 @@
 add new error type ResourceLimit
--- a/changelog.d/3708.feature
+++ b/changelog.d/3708.feature
@ -1 +0,0 @@
 For resource limit blocked users, prevent writing into rooms
--- a/changelog.d/3709.misc
+++ b/changelog.d/3709.misc
@ -1 +0,0 @@
 Logcontexts for replication command handlers
--- a/changelog.d/3710.bugfix
+++ b/changelog.d/3710.bugfix
@ -1 +0,0 @@
 Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning
--- a/changelog.d/3712.misc
+++ b/changelog.d/3712.misc
@ -1 +0,0 @@
 Update admin register API documentation to reference a real user ID.
--- a/changelog.d/3713.bugfix
+++ b/changelog.d/3713.bugfix
@ -1 +0,0 @@
 Support more federation endpoints on workers
--- a/changelog.d/3719.bugfix
+++ b/changelog.d/3719.bugfix
@ -1 +0,0 @@
 Fix bug where `state_cache` cache factor ignored environment variables
--- a/synapse/init.py
+++ b/synapse/init.py
@ -17,4 +17,4 @@
 """ This is a reference implementation of a Matrix home server.
 """
-__version__ = "0.33.2"
+__version__ = "0.33.3rc2"
--- a/synapse/http/site.py
+++ b/synapse/http/site.py
@ -182,7 +182,7 @@ class SynapseRequest(Request):
        # the client disconnects.
        with PreserveLoggingContext(self.logcontext):
            logger.warn(
-                "Error processing request: %s %s", reason.type, reason.value,
+                "Error processing request %r: %s %s", self, reason.type, reason.value,
            )
            if not self._is_processing:
@ -219,6 +219,12 @@ class SynapseRequest(Request):
        """Log the completion of this request and update the metrics
        """
        if self.logcontext is None:
            # this can happen if the connection closed before we read the
            # headers (so render was never called). In that case we'll already
            # have logged a warning, so just bail out.
            return
        usage = self.logcontext.get_resource_usage()
        if self._processing_finished_time is None:
--- a/synapse/util/logcontext.py
+++ b/synapse/util/logcontext.py
@ -385,7 +385,13 @@ class LoggingContextFilter(logging.Filter):
        context = LoggingContext.current_context()
        for key, value in self.defaults.items():
            setattr(record, key, value)
-        context.copy_to(record)
+
        # context should never be None, but if it somehow ends up being, then
        # we end up in a death spiral of infinite loops, so let's check, for
        # robustness' sake.
        if context is not None:
            context.copy_to(record)
        return True
@ -396,7 +402,9 @@ class PreserveLoggingContext(object):
    __slots__ = ["current_context", "new_context", "has_parent"]
-    def __init__(self, new_context=LoggingContext.sentinel):
+    def __init__(self, new_context=None):
        if new_context is None:
            new_context = LoggingContext.sentinel
        self.new_context = new_context
    def __enter__(self):
		`@ -1 +0,0 @@`
			`Add support for the SNI extension to federation TLS connections`
		`@ -1 +0,0 @@`
			`The test suite now can run under PostgreSQL.`
		`@ -1 +0,0 @@`
			speed up /members API and add `at` and `membership` params as per MSC1227
		`@ -1 +0,0 @@`
			implement `summary` block in /sync response as per MSC688
		`@ -1 +0,0 @@`
			`Add lazy-loading support to /messages as per MSC1227`
		`@ -1 +0,0 @@`
			`Refactor HTTP replication endpoints to reduce code duplication`
		`@ -1 +0,0 @@`
			`Add ability to limit number of monthly active users on the server`
		`@ -1 +0,0 @@`
			`Support more federation endpoints on workers`
		`@ -1 +0,0 @@`
			`Ability to disable client/server Synapse via conf toggle`
		`@ -1 +0,0 @@`
			`Fix occasional glitches in the synapse_event_persisted_position metric`