Synapse 1.7.0 (2019-12-13)
========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497)) - Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355)) - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409)) - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420)) - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491)) - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490)) - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388)) - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443)) - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266)) - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392)) - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483)) - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484)) -----BEGIN PGP SIGNATURE----- iQJEBAABCgAuFiEEumuwyPtYLL2OMhYdOtoG7cdT0R4FAl3zbaUQHGVyaWtAbWF0 cml4Lm9yZwAKCRA62gbtx1PRHqxED/9Ldjx5pxRm99Rkfhpv6qaw79GaQsAERW+i rPzdC0u59PRDLyfB2MRGU+q4oTTmBN3APPuW79mhY4CRisKfUvw6tuaJM5avvRsF vlY6m4ObewPXvK9dumlKD4ORqks45Sv0VGLHoZH4MLvWu/3Z+ycUn0qf+KNy0JrV TNNnNfsEeo3FXrkLU/12TiEBqNTVCU2vi/nZaHoGlHtZGVoykyFk4z2i9hLKENH4 ScwaN4zeFRTIWYMiEb+kI/6QQLTASvar5FstwXPpM2F+aZhrMc5M7YsEe16jx+bQ jOM1L67hMetcWWmE9QrgP030clIDg30xhNbVxFSD4WoGgEPFkf/zE91v0CF5ZMRJ u83hlhe8tHoO8DOfVl2DNw3x6Evx4Vb5P8bvszNwKAyVDanvkSe0GvYm84B0Tat6 2UCyPR5mbU3fyiwpouaNYun8MppyMRQ/fCnNElvZwwoj9EQQcJ8QOnMK0pFv1AXV n4wwXqRb+WvvbhnKPxTsiTK0DPBbIiGqo1TIONSbhPF+wySR3YV80HCWofd+qlRT V+GbZihk4SCcmamgqYLeQYneOrIu6UWGP348t3scCz/RCUp7hmxaJhJ3GAC3hOmU alN6tDJn7Lo0CwY2frX8xPS6S31sq9IZyDhHkaL0chsEa1rgWJMOK3FRp/b+c8yU b3CEJObIjw== =RQVG -----END PGP SIGNATURE----- Merge tag 'v1.7.0' Synapse 1.7.0 (2019-12-13) ========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497)) - Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355)) - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409)) - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420)) - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491)) - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490)) - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388)) - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443)) - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266)) - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392)) - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483)) - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484))
This commit is contained in:
Erik Johnston
commit
bee1982d17
|
|
@ -1,6 +1,6 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
set -ex
|
set -e
|
||||||
|
|
||||||
if [[ "$BUILDKITE_BRANCH" =~ ^(develop|master|dinsic|shhs|release-.*)$ ]]; then
|
if [[ "$BUILDKITE_BRANCH" =~ ^(develop|master|dinsic|shhs|release-.*)$ ]]; then
|
||||||
echo "Not merging forward, as this is a release branch"
|
echo "Not merging forward, as this is a release branch"
|
||||||
|
|
@ -18,6 +18,8 @@ else
|
||||||
GITBASE=$BUILDKITE_PULL_REQUEST_BASE_BRANCH
|
GITBASE=$BUILDKITE_PULL_REQUEST_BASE_BRANCH
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
echo "--- merge_base_branch $GITBASE"
|
||||||
|
|
||||||
# Show what we are before
|
# Show what we are before
|
||||||
git --no-pager show -s
|
git --no-pager show -s
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# Configuration file used for testing the 'synapse_port_db' script.
|
# Configuration file used for testing the 'synapse_port_db' script.
|
||||||
# Tells the script to connect to the postgresql database that will be available in the
|
# Tells the script to connect to the postgresql database that will be available in the
|
||||||
# CI's Docker setup at the point where this file is considered.
|
# CI's Docker setup at the point where this file is considered.
|
||||||
server_name: "test"
|
server_name: "localhost:8800"
|
||||||
|
|
||||||
signing_key_path: "/src/.buildkite/test.signing.key"
|
signing_key_path: "/src/.buildkite/test.signing.key"
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# Configuration file used for testing the 'synapse_port_db' script.
|
# Configuration file used for testing the 'synapse_port_db' script.
|
||||||
# Tells the 'update_database' script to connect to the test SQLite database to upgrade its
|
# Tells the 'update_database' script to connect to the test SQLite database to upgrade its
|
||||||
# schema and run background updates on it.
|
# schema and run background updates on it.
|
||||||
server_name: "test"
|
server_name: "localhost:8800"
|
||||||
|
|
||||||
signing_key_path: "/src/.buildkite/test.signing.key"
|
signing_key_path: "/src/.buildkite/test.signing.key"
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -28,3 +28,39 @@ User sees updates to presence from other users in the incremental sync.
|
||||||
Gapped incremental syncs include all state changes
|
Gapped incremental syncs include all state changes
|
||||||
|
|
||||||
Old members are included in gappy incr LL sync if they start speaking
|
Old members are included in gappy incr LL sync if they start speaking
|
||||||
|
|
||||||
|
# new failures as of https://github.com/matrix-org/sytest/pull/732
|
||||||
|
Device list doesn't change if remote server is down
|
||||||
|
Remote servers cannot set power levels in rooms without existing powerlevels
|
||||||
|
Remote servers should reject attempts by non-creators to set the power levels
|
||||||
|
|
||||||
|
# new failures as of https://github.com/matrix-org/sytest/pull/753
|
||||||
|
GET /rooms/:room_id/messages returns a message
|
||||||
|
GET /rooms/:room_id/messages lazy loads members correctly
|
||||||
|
Read receipts are sent as events
|
||||||
|
Only original members of the room can see messages from erased users
|
||||||
|
Device deletion propagates over federation
|
||||||
|
If user leaves room, remote user changes device and rejoins we see update in /sync and /keys/changes
|
||||||
|
Changing user-signing key notifies local users
|
||||||
|
Newly updated tags appear in an incremental v2 /sync
|
||||||
|
Server correctly handles incoming m.device_list_update
|
||||||
|
Local device key changes get to remote servers with correct prev_id
|
||||||
|
AS-ghosted users can use rooms via AS
|
||||||
|
Ghost user must register before joining room
|
||||||
|
Test that a message is pushed
|
||||||
|
Invites are pushed
|
||||||
|
Rooms with aliases are correctly named in pushed
|
||||||
|
Rooms with names are correctly named in pushed
|
||||||
|
Rooms with canonical alias are correctly named in pushed
|
||||||
|
Rooms with many users are correctly pushed
|
||||||
|
Don't get pushed for rooms you've muted
|
||||||
|
Rejected events are not pushed
|
||||||
|
Test that rejected pushers are removed.
|
||||||
|
Events come down the correct room
|
||||||
|
|
||||||
|
# https://buildkite.com/matrix-dot-org/sytest/builds/326#cca62404-a88a-4fcb-ad41-175fd3377603
|
||||||
|
Presence changes to UNAVAILABLE are reported to remote room members
|
||||||
|
If remote user leaves room, changes device and rejoins we see update in sync
|
||||||
|
uploading self-signing key notifies over federation
|
||||||
|
Inbound federation can receive redacted events
|
||||||
|
Outbound federation can request missing events
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,8 @@
|
||||||
### Pull Request Checklist
|
### Pull Request Checklist
|
||||||
|
|
||||||
<!-- Please read CONTRIBUTING.rst before submitting your pull request -->
|
<!-- Please read CONTRIBUTING.md before submitting your pull request -->
|
||||||
|
|
||||||
* [ ] Pull request is based on the develop branch
|
* [ ] Pull request is based on the develop branch
|
||||||
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#changelog)
|
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.md#changelog)
|
||||||
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#sign-off)
|
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.md#sign-off)
|
||||||
* [ ] Code style is correct (run the [linters](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#code-style))
|
* [ ] Code style is correct (run the [linters](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.md#code-style))
|
||||||
|
|
|
||||||
95
CHANGES.md
95
CHANGES.md
|
|
@ -1,3 +1,98 @@
|
||||||
|
Synapse 1.7.0 (2019-12-13)
|
||||||
|
==========================
|
||||||
|
|
||||||
|
This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details.
|
||||||
|
|
||||||
|
Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11.
|
||||||
|
|
||||||
|
Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite.
|
||||||
|
|
||||||
|
No significant changes since 1.7.0rc2.
|
||||||
|
|
||||||
|
|
||||||
|
Synapse 1.7.0rc2 (2019-12-11)
|
||||||
|
=============================
|
||||||
|
|
||||||
|
Bugfixes
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497))
|
||||||
|
- Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499))
|
||||||
|
- Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509))
|
||||||
|
|
||||||
|
|
||||||
|
Synapse 1.7.0rc1 (2019-12-09)
|
||||||
|
=============================
|
||||||
|
|
||||||
|
Features
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436))
|
||||||
|
- Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858))
|
||||||
|
- Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925))
|
||||||
|
- Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119))
|
||||||
|
- Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176))
|
||||||
|
- Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355))
|
||||||
|
- Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409))
|
||||||
|
- Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434))
|
||||||
|
|
||||||
|
|
||||||
|
Bugfixes
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237))
|
||||||
|
- Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241))
|
||||||
|
- Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329))
|
||||||
|
- Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332))
|
||||||
|
- Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333))
|
||||||
|
- Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406))
|
||||||
|
- Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408))
|
||||||
|
- Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420))
|
||||||
|
- Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421))
|
||||||
|
- Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449))
|
||||||
|
- Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451))
|
||||||
|
- Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462))
|
||||||
|
- Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470))
|
||||||
|
- Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472))
|
||||||
|
- Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491))
|
||||||
|
- Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493))
|
||||||
|
- Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494))
|
||||||
|
|
||||||
|
|
||||||
|
Improved Documentation
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
- Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490))
|
||||||
|
- Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388))
|
||||||
|
- Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390))
|
||||||
|
- Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443))
|
||||||
|
- Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458))
|
||||||
|
- Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461))
|
||||||
|
|
||||||
|
|
||||||
|
Deprecations and Removals
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
- Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925))
|
||||||
|
- Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488))
|
||||||
|
|
||||||
|
|
||||||
|
Internal Changes
|
||||||
|
----------------
|
||||||
|
|
||||||
|
- Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266))
|
||||||
|
- Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322))
|
||||||
|
- Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480))
|
||||||
|
- Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362))
|
||||||
|
- Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379))
|
||||||
|
- Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392))
|
||||||
|
- Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423))
|
||||||
|
- Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429))
|
||||||
|
- Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487))
|
||||||
|
- Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482))
|
||||||
|
- Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483))
|
||||||
|
- Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484))
|
||||||
|
|
||||||
Synapse 1.6.1 (2019-11-28)
|
Synapse 1.6.1 (2019-11-28)
|
||||||
==========================
|
==========================
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,210 @@
|
||||||
|
# Contributing code to Matrix
|
||||||
|
|
||||||
|
Everyone is welcome to contribute code to Matrix
|
||||||
|
(https://github.com/matrix-org), provided that they are willing to license
|
||||||
|
their contributions under the same license as the project itself. We follow a
|
||||||
|
simple 'inbound=outbound' model for contributions: the act of submitting an
|
||||||
|
'inbound' contribution means that the contributor agrees to license the code
|
||||||
|
under the same terms as the project's overall 'outbound' license - in our
|
||||||
|
case, this is almost always Apache Software License v2 (see [LICENSE](LICENSE)).
|
||||||
|
|
||||||
|
## How to contribute
|
||||||
|
|
||||||
|
The preferred and easiest way to contribute changes to Matrix is to fork the
|
||||||
|
relevant project on github, and then [create a pull request](
|
||||||
|
https://help.github.com/articles/using-pull-requests/) to ask us to pull
|
||||||
|
your changes into our repo.
|
||||||
|
|
||||||
|
**The single biggest thing you need to know is: please base your changes on
|
||||||
|
the develop branch - *not* master.**
|
||||||
|
|
||||||
|
We use the master branch to track the most recent release, so that folks who
|
||||||
|
blindly clone the repo and automatically check out master get something that
|
||||||
|
works. Develop is the unstable branch where all the development actually
|
||||||
|
happens: the workflow is that contributors should fork the develop branch to
|
||||||
|
make a 'feature' branch for a particular contribution, and then make a pull
|
||||||
|
request to merge this back into the matrix.org 'official' develop branch. We
|
||||||
|
use github's pull request workflow to review the contribution, and either ask
|
||||||
|
you to make any refinements needed or merge it and make them ourselves. The
|
||||||
|
changes will then land on master when we next do a release.
|
||||||
|
|
||||||
|
We use [Buildkite](https://buildkite.com/matrix-dot-org/synapse) for continuous
|
||||||
|
integration. If your change breaks the build, this will be shown in GitHub, so
|
||||||
|
please keep an eye on the pull request for feedback.
|
||||||
|
|
||||||
|
To run unit tests in a local development environment, you can use:
|
||||||
|
|
||||||
|
- ``tox -e py35`` (requires tox to be installed by ``pip install tox``)
|
||||||
|
for SQLite-backed Synapse on Python 3.5.
|
||||||
|
- ``tox -e py36`` for SQLite-backed Synapse on Python 3.6.
|
||||||
|
- ``tox -e py36-postgres`` for PostgreSQL-backed Synapse on Python 3.6
|
||||||
|
(requires a running local PostgreSQL with access to create databases).
|
||||||
|
- ``./test_postgresql.sh`` for PostgreSQL-backed Synapse on Python 3.5
|
||||||
|
(requires Docker). Entirely self-contained, recommended if you don't want to
|
||||||
|
set up PostgreSQL yourself.
|
||||||
|
|
||||||
|
Docker images are available for running the integration tests (SyTest) locally,
|
||||||
|
see the [documentation in the SyTest repo](
|
||||||
|
https://github.com/matrix-org/sytest/blob/develop/docker/README.md) for more
|
||||||
|
information.
|
||||||
|
|
||||||
|
## Code style
|
||||||
|
|
||||||
|
All Matrix projects have a well-defined code-style - and sometimes we've even
|
||||||
|
got as far as documenting it... For instance, synapse's code style doc lives
|
||||||
|
[here](docs/code_style.md).
|
||||||
|
|
||||||
|
To facilitate meeting these criteria you can run `scripts-dev/lint.sh`
|
||||||
|
locally. Since this runs the tools listed in the above document, you'll need
|
||||||
|
python 3.6 and to install each tool:
|
||||||
|
|
||||||
|
```
|
||||||
|
# Install the dependencies
|
||||||
|
pip install -U black flake8 isort
|
||||||
|
|
||||||
|
# Run the linter script
|
||||||
|
./scripts-dev/lint.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note that the script does not just test/check, but also reformats code, so you
|
||||||
|
may wish to ensure any new code is committed first**. By default this script
|
||||||
|
checks all files and can take some time; if you alter only certain files, you
|
||||||
|
might wish to specify paths as arguments to reduce the run-time:
|
||||||
|
|
||||||
|
```
|
||||||
|
./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
|
||||||
|
```
|
||||||
|
|
||||||
|
Before pushing new changes, ensure they don't produce linting errors. Commit any
|
||||||
|
files that were corrected.
|
||||||
|
|
||||||
|
Please ensure your changes match the cosmetic style of the existing project,
|
||||||
|
and **never** mix cosmetic and functional changes in the same commit, as it
|
||||||
|
makes it horribly hard to review otherwise.
|
||||||
|
|
||||||
|
|
||||||
|
## Changelog
|
||||||
|
|
||||||
|
All changes, even minor ones, need a corresponding changelog / newsfragment
|
||||||
|
entry. These are managed by [Towncrier](https://github.com/hawkowl/towncrier).
|
||||||
|
|
||||||
|
To create a changelog entry, make a new file in the `changelog.d` directory named
|
||||||
|
in the format of `PRnumber.type`. The type can be one of the following:
|
||||||
|
|
||||||
|
* `feature`
|
||||||
|
* `bugfix`
|
||||||
|
* `docker` (for updates to the Docker image)
|
||||||
|
* `doc` (for updates to the documentation)
|
||||||
|
* `removal` (also used for deprecations)
|
||||||
|
* `misc` (for internal-only changes)
|
||||||
|
|
||||||
|
The content of the file is your changelog entry, which should be a short
|
||||||
|
description of your change in the same style as the rest of our [changelog](
|
||||||
|
https://github.com/matrix-org/synapse/blob/master/CHANGES.md). The file can
|
||||||
|
contain Markdown formatting, and should end with a full stop ('.') for
|
||||||
|
consistency.
|
||||||
|
|
||||||
|
Adding credits to the changelog is encouraged, we value your
|
||||||
|
contributions and would like to have you shouted out in the release notes!
|
||||||
|
|
||||||
|
For example, a fix in PR #1234 would have its changelog entry in
|
||||||
|
`changelog.d/1234.bugfix`, and contain content like "The security levels of
|
||||||
|
Florbs are now validated when received over federation. Contributed by Jane
|
||||||
|
Matrix.".
|
||||||
|
|
||||||
|
## Debian changelog
|
||||||
|
|
||||||
|
Changes which affect the debian packaging files (in `debian`) are an
|
||||||
|
exception.
|
||||||
|
|
||||||
|
In this case, you will need to add an entry to the debian changelog for the
|
||||||
|
next release. For this, run the following command:
|
||||||
|
|
||||||
|
```
|
||||||
|
dch
|
||||||
|
```
|
||||||
|
|
||||||
|
This will make up a new version number (if there isn't already an unreleased
|
||||||
|
version in flight), and open an editor where you can add a new changelog entry.
|
||||||
|
(Our release process will ensure that the version number and maintainer name is
|
||||||
|
corrected for the release.)
|
||||||
|
|
||||||
|
If your change affects both the debian packaging *and* files outside the debian
|
||||||
|
directory, you will need both a regular newsfragment *and* an entry in the
|
||||||
|
debian changelog. (Though typically such changes should be submitted as two
|
||||||
|
separate pull requests.)
|
||||||
|
|
||||||
|
## Sign off
|
||||||
|
|
||||||
|
In order to have a concrete record that your contribution is intentional
|
||||||
|
and you agree to license it under the same terms as the project's license, we've adopted the
|
||||||
|
same lightweight approach that the Linux Kernel
|
||||||
|
[submitting patches process](
|
||||||
|
https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin>),
|
||||||
|
[Docker](https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other
|
||||||
|
projects use: the DCO (Developer Certificate of Origin:
|
||||||
|
http://developercertificate.org/). This is a simple declaration that you wrote
|
||||||
|
the contribution or otherwise have the right to contribute it to Matrix:
|
||||||
|
|
||||||
|
```
|
||||||
|
Developer Certificate of Origin
|
||||||
|
Version 1.1
|
||||||
|
|
||||||
|
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
|
||||||
|
660 York Street, Suite 102,
|
||||||
|
San Francisco, CA 94110 USA
|
||||||
|
|
||||||
|
Everyone is permitted to copy and distribute verbatim copies of this
|
||||||
|
license document, but changing it is not allowed.
|
||||||
|
|
||||||
|
Developer's Certificate of Origin 1.1
|
||||||
|
|
||||||
|
By making a contribution to this project, I certify that:
|
||||||
|
|
||||||
|
(a) The contribution was created in whole or in part by me and I
|
||||||
|
have the right to submit it under the open source license
|
||||||
|
indicated in the file; or
|
||||||
|
|
||||||
|
(b) The contribution is based upon previous work that, to the best
|
||||||
|
of my knowledge, is covered under an appropriate open source
|
||||||
|
license and I have the right under that license to submit that
|
||||||
|
work with modifications, whether created in whole or in part
|
||||||
|
by me, under the same open source license (unless I am
|
||||||
|
permitted to submit under a different license), as indicated
|
||||||
|
in the file; or
|
||||||
|
|
||||||
|
(c) The contribution was provided directly to me by some other
|
||||||
|
person who certified (a), (b) or (c) and I have not modified
|
||||||
|
it.
|
||||||
|
|
||||||
|
(d) I understand and agree that this project and the contribution
|
||||||
|
are public and that a record of the contribution (including all
|
||||||
|
personal information I submit with it, including my sign-off) is
|
||||||
|
maintained indefinitely and may be redistributed consistent with
|
||||||
|
this project or the open source license(s) involved.
|
||||||
|
```
|
||||||
|
|
||||||
|
If you agree to this for your contribution, then all that's needed is to
|
||||||
|
include the line in your commit or pull request comment:
|
||||||
|
|
||||||
|
```
|
||||||
|
Signed-off-by: Your Name <your@email.example.org>
|
||||||
|
```
|
||||||
|
|
||||||
|
We accept contributions under a legally identifiable name, such as
|
||||||
|
your name on government documentation or common-law names (names
|
||||||
|
claimed by legitimate usage or repute). Unfortunately, we cannot
|
||||||
|
accept anonymous contributions at this time.
|
||||||
|
|
||||||
|
Git allows you to add this signoff automatically when using the `-s`
|
||||||
|
flag to `git commit`, which uses the name and email set in your
|
||||||
|
`user.name` and `user.email` git configs.
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
That's it! Matrix is a very open and collaborative project as you might expect
|
||||||
|
given our obsession with open communication. If we're going to successfully
|
||||||
|
matrix together all the fragmented communication technologies out there we are
|
||||||
|
reliant on contributions and collaboration from the community to do so. So
|
||||||
|
please get involved - and we hope you have as much fun hacking on Matrix as we
|
||||||
|
do!
|
||||||
206
CONTRIBUTING.rst
206
CONTRIBUTING.rst
|
|
@ -1,206 +0,0 @@
|
||||||
Contributing code to Matrix
|
|
||||||
===========================
|
|
||||||
|
|
||||||
Everyone is welcome to contribute code to Matrix
|
|
||||||
(https://github.com/matrix-org), provided that they are willing to license
|
|
||||||
their contributions under the same license as the project itself. We follow a
|
|
||||||
simple 'inbound=outbound' model for contributions: the act of submitting an
|
|
||||||
'inbound' contribution means that the contributor agrees to license the code
|
|
||||||
under the same terms as the project's overall 'outbound' license - in our
|
|
||||||
case, this is almost always Apache Software License v2 (see LICENSE).
|
|
||||||
|
|
||||||
How to contribute
|
|
||||||
~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
The preferred and easiest way to contribute changes to Matrix is to fork the
|
|
||||||
relevant project on github, and then create a pull request to ask us to pull
|
|
||||||
your changes into our repo
|
|
||||||
(https://help.github.com/articles/using-pull-requests/)
|
|
||||||
|
|
||||||
**The single biggest thing you need to know is: please base your changes on
|
|
||||||
the develop branch - /not/ master.**
|
|
||||||
|
|
||||||
We use the master branch to track the most recent release, so that folks who
|
|
||||||
blindly clone the repo and automatically check out master get something that
|
|
||||||
works. Develop is the unstable branch where all the development actually
|
|
||||||
happens: the workflow is that contributors should fork the develop branch to
|
|
||||||
make a 'feature' branch for a particular contribution, and then make a pull
|
|
||||||
request to merge this back into the matrix.org 'official' develop branch. We
|
|
||||||
use github's pull request workflow to review the contribution, and either ask
|
|
||||||
you to make any refinements needed or merge it and make them ourselves. The
|
|
||||||
changes will then land on master when we next do a release.
|
|
||||||
|
|
||||||
We use `Buildkite <https://buildkite.com/matrix-dot-org/synapse>`_ for
|
|
||||||
continuous integration. Buildkite builds need to be authorised by a
|
|
||||||
maintainer. If your change breaks the build, this will be shown in GitHub, so
|
|
||||||
please keep an eye on the pull request for feedback.
|
|
||||||
|
|
||||||
To run unit tests in a local development environment, you can use:
|
|
||||||
|
|
||||||
- ``tox -e py35`` (requires tox to be installed by ``pip install tox``)
|
|
||||||
for SQLite-backed Synapse on Python 3.5.
|
|
||||||
- ``tox -e py36`` for SQLite-backed Synapse on Python 3.6.
|
|
||||||
- ``tox -e py36-postgres`` for PostgreSQL-backed Synapse on Python 3.6
|
|
||||||
(requires a running local PostgreSQL with access to create databases).
|
|
||||||
- ``./test_postgresql.sh`` for PostgreSQL-backed Synapse on Python 3.5
|
|
||||||
(requires Docker). Entirely self-contained, recommended if you don't want to
|
|
||||||
set up PostgreSQL yourself.
|
|
||||||
|
|
||||||
Docker images are available for running the integration tests (SyTest) locally,
|
|
||||||
see the `documentation in the SyTest repo
|
|
||||||
<https://github.com/matrix-org/sytest/blob/develop/docker/README.md>`_ for more
|
|
||||||
information.
|
|
||||||
|
|
||||||
Code style
|
|
||||||
~~~~~~~~~~
|
|
||||||
|
|
||||||
All Matrix projects have a well-defined code-style - and sometimes we've even
|
|
||||||
got as far as documenting it... For instance, synapse's code style doc lives
|
|
||||||
at https://github.com/matrix-org/synapse/tree/master/docs/code_style.md.
|
|
||||||
|
|
||||||
To facilitate meeting these criteria you can run ``scripts-dev/lint.sh``
|
|
||||||
locally. Since this runs the tools listed in the above document, you'll need
|
|
||||||
python 3.6 and to install each tool. **Note that the script does not just
|
|
||||||
test/check, but also reformats code, so you may wish to ensure any new code is
|
|
||||||
committed first**. By default this script checks all files and can take some
|
|
||||||
time; if you alter only certain files, you might wish to specify paths as
|
|
||||||
arguments to reduce the run-time.
|
|
||||||
|
|
||||||
Please ensure your changes match the cosmetic style of the existing project,
|
|
||||||
and **never** mix cosmetic and functional changes in the same commit, as it
|
|
||||||
makes it horribly hard to review otherwise.
|
|
||||||
|
|
||||||
Before doing a commit, ensure the changes you've made don't produce
|
|
||||||
linting errors. You can do this by running the linters as follows. Ensure to
|
|
||||||
commit any files that were corrected.
|
|
||||||
|
|
||||||
::
|
|
||||||
# Install the dependencies
|
|
||||||
pip install -U black flake8 isort
|
|
||||||
|
|
||||||
# Run the linter script
|
|
||||||
./scripts-dev/lint.sh
|
|
||||||
|
|
||||||
Changelog
|
|
||||||
~~~~~~~~~
|
|
||||||
|
|
||||||
All changes, even minor ones, need a corresponding changelog / newsfragment
|
|
||||||
entry. These are managed by Towncrier
|
|
||||||
(https://github.com/hawkowl/towncrier).
|
|
||||||
|
|
||||||
To create a changelog entry, make a new file in the ``changelog.d`` file named
|
|
||||||
in the format of ``PRnumber.type``. The type can be one of the following:
|
|
||||||
|
|
||||||
* ``feature``.
|
|
||||||
* ``bugfix``.
|
|
||||||
* ``docker`` (for updates to the Docker image).
|
|
||||||
* ``doc`` (for updates to the documentation).
|
|
||||||
* ``removal`` (also used for deprecations).
|
|
||||||
* ``misc`` (for internal-only changes).
|
|
||||||
|
|
||||||
The content of the file is your changelog entry, which should be a short
|
|
||||||
description of your change in the same style as the rest of our `changelog
|
|
||||||
<https://github.com/matrix-org/synapse/blob/master/CHANGES.md>`_. The file can
|
|
||||||
contain Markdown formatting, and should end with a full stop ('.') for
|
|
||||||
consistency.
|
|
||||||
|
|
||||||
Adding credits to the changelog is encouraged, we value your
|
|
||||||
contributions and would like to have you shouted out in the release notes!
|
|
||||||
|
|
||||||
For example, a fix in PR #1234 would have its changelog entry in
|
|
||||||
``changelog.d/1234.bugfix``, and contain content like "The security levels of
|
|
||||||
Florbs are now validated when recieved over federation. Contributed by Jane
|
|
||||||
Matrix.".
|
|
||||||
|
|
||||||
Debian changelog
|
|
||||||
----------------
|
|
||||||
|
|
||||||
Changes which affect the debian packaging files (in ``debian``) are an
|
|
||||||
exception.
|
|
||||||
|
|
||||||
In this case, you will need to add an entry to the debian changelog for the
|
|
||||||
next release. For this, run the following command::
|
|
||||||
|
|
||||||
dch
|
|
||||||
|
|
||||||
This will make up a new version number (if there isn't already an unreleased
|
|
||||||
version in flight), and open an editor where you can add a new changelog entry.
|
|
||||||
(Our release process will ensure that the version number and maintainer name is
|
|
||||||
corrected for the release.)
|
|
||||||
|
|
||||||
If your change affects both the debian packaging *and* files outside the debian
|
|
||||||
directory, you will need both a regular newsfragment *and* an entry in the
|
|
||||||
debian changelog. (Though typically such changes should be submitted as two
|
|
||||||
separate pull requests.)
|
|
||||||
|
|
||||||
Sign off
|
|
||||||
~~~~~~~~
|
|
||||||
|
|
||||||
In order to have a concrete record that your contribution is intentional
|
|
||||||
and you agree to license it under the same terms as the project's license, we've adopted the
|
|
||||||
same lightweight approach that the Linux Kernel
|
|
||||||
`submitting patches process <https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin>`_, Docker
|
|
||||||
(https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other
|
|
||||||
projects use: the DCO (Developer Certificate of Origin:
|
|
||||||
http://developercertificate.org/). This is a simple declaration that you wrote
|
|
||||||
the contribution or otherwise have the right to contribute it to Matrix::
|
|
||||||
|
|
||||||
Developer Certificate of Origin
|
|
||||||
Version 1.1
|
|
||||||
|
|
||||||
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
|
|
||||||
660 York Street, Suite 102,
|
|
||||||
San Francisco, CA 94110 USA
|
|
||||||
|
|
||||||
Everyone is permitted to copy and distribute verbatim copies of this
|
|
||||||
license document, but changing it is not allowed.
|
|
||||||
|
|
||||||
Developer's Certificate of Origin 1.1
|
|
||||||
|
|
||||||
By making a contribution to this project, I certify that:
|
|
||||||
|
|
||||||
(a) The contribution was created in whole or in part by me and I
|
|
||||||
have the right to submit it under the open source license
|
|
||||||
indicated in the file; or
|
|
||||||
|
|
||||||
(b) The contribution is based upon previous work that, to the best
|
|
||||||
of my knowledge, is covered under an appropriate open source
|
|
||||||
license and I have the right under that license to submit that
|
|
||||||
work with modifications, whether created in whole or in part
|
|
||||||
by me, under the same open source license (unless I am
|
|
||||||
permitted to submit under a different license), as indicated
|
|
||||||
in the file; or
|
|
||||||
|
|
||||||
(c) The contribution was provided directly to me by some other
|
|
||||||
person who certified (a), (b) or (c) and I have not modified
|
|
||||||
it.
|
|
||||||
|
|
||||||
(d) I understand and agree that this project and the contribution
|
|
||||||
are public and that a record of the contribution (including all
|
|
||||||
personal information I submit with it, including my sign-off) is
|
|
||||||
maintained indefinitely and may be redistributed consistent with
|
|
||||||
this project or the open source license(s) involved.
|
|
||||||
|
|
||||||
If you agree to this for your contribution, then all that's needed is to
|
|
||||||
include the line in your commit or pull request comment::
|
|
||||||
|
|
||||||
Signed-off-by: Your Name <your@email.example.org>
|
|
||||||
|
|
||||||
We accept contributions under a legally identifiable name, such as
|
|
||||||
your name on government documentation or common-law names (names
|
|
||||||
claimed by legitimate usage or repute). Unfortunately, we cannot
|
|
||||||
accept anonymous contributions at this time.
|
|
||||||
|
|
||||||
Git allows you to add this signoff automatically when using the ``-s``
|
|
||||||
flag to ``git commit``, which uses the name and email set in your
|
|
||||||
``user.name`` and ``user.email`` git configs.
|
|
||||||
|
|
||||||
Conclusion
|
|
||||||
~~~~~~~~~~
|
|
||||||
|
|
||||||
That's it! Matrix is a very open and collaborative project as you might expect
|
|
||||||
given our obsession with open communication. If we're going to successfully
|
|
||||||
matrix together all the fragmented communication technologies out there we are
|
|
||||||
reliant on contributions and collaboration from the community to do so. So
|
|
||||||
please get involved - and we hope you have as much fun hacking on Matrix as we
|
|
||||||
do!
|
|
||||||
16
INSTALL.md
16
INSTALL.md
|
|
@ -109,8 +109,8 @@ Installing prerequisites on Ubuntu or Debian:
|
||||||
|
|
||||||
```
|
```
|
||||||
sudo apt-get install build-essential python3-dev libffi-dev \
|
sudo apt-get install build-essential python3-dev libffi-dev \
|
||||||
python-pip python-setuptools sqlite3 \
|
python3-pip python3-setuptools sqlite3 \
|
||||||
libssl-dev python-virtualenv libjpeg-dev libxslt1-dev
|
libssl-dev python3-virtualenv libjpeg-dev libxslt1-dev
|
||||||
```
|
```
|
||||||
|
|
||||||
#### ArchLinux
|
#### ArchLinux
|
||||||
|
|
@ -133,9 +133,9 @@ sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
|
||||||
sudo yum groupinstall "Development Tools"
|
sudo yum groupinstall "Development Tools"
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Mac OS X
|
#### macOS
|
||||||
|
|
||||||
Installing prerequisites on Mac OS X:
|
Installing prerequisites on macOS:
|
||||||
|
|
||||||
```
|
```
|
||||||
xcode-select --install
|
xcode-select --install
|
||||||
|
|
@ -144,6 +144,14 @@ sudo pip install virtualenv
|
||||||
brew install pkg-config libffi
|
brew install pkg-config libffi
|
||||||
```
|
```
|
||||||
|
|
||||||
|
On macOS Catalina (10.15) you may need to explicitly install OpenSSL
|
||||||
|
via brew and inform `pip` about it so that `psycopg2` builds:
|
||||||
|
|
||||||
|
```
|
||||||
|
brew install openssl@1.1
|
||||||
|
export LDFLAGS=-L/usr/local/Cellar/openssl\@1.1/1.1.1d/lib/
|
||||||
|
```
|
||||||
|
|
||||||
#### OpenSUSE
|
#### OpenSUSE
|
||||||
|
|
||||||
Installing prerequisites on openSUSE:
|
Installing prerequisites on openSUSE:
|
||||||
|
|
|
||||||
17
UPGRADE.rst
17
UPGRADE.rst
|
|
@ -75,6 +75,23 @@ for example:
|
||||||
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
||||||
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
||||||
|
|
||||||
|
Upgrading to v1.7.0
|
||||||
|
===================
|
||||||
|
|
||||||
|
In an attempt to configure Synapse in a privacy preserving way, the default
|
||||||
|
behaviours of ``allow_public_rooms_without_auth`` and
|
||||||
|
``allow_public_rooms_over_federation`` have been inverted. This means that by
|
||||||
|
default, only authenticated users querying the Client/Server API will be able
|
||||||
|
to query the room directory, and relatedly that the server will not share
|
||||||
|
room directory information with other servers over federation.
|
||||||
|
|
||||||
|
If your installation does not explicitly set these settings one way or the other
|
||||||
|
and you want either setting to be ``true`` then it will necessary to update
|
||||||
|
your homeserver configuration file accordingly.
|
||||||
|
|
||||||
|
For more details on the surrounding context see our `explainer
|
||||||
|
<https://matrix.org/blog/2019/11/09/avoiding-unwelcome-visitors-on-private-matrix-servers>`_.
|
||||||
|
|
||||||
|
|
||||||
Upgrading to v1.5.0
|
Upgrading to v1.5.0
|
||||||
===================
|
===================
|
||||||
|
|
|
||||||
|
|
@ -25,7 +25,7 @@ Restart=on-abort
|
||||||
User=synapse
|
User=synapse
|
||||||
Group=nogroup
|
Group=nogroup
|
||||||
|
|
||||||
WorkingDirectory=/opt/synapse
|
WorkingDirectory=/home/synapse/synapse
|
||||||
ExecStart=/home/synapse/synapse/env/bin/python -m synapse.app.homeserver --config-path=/home/synapse/synapse/homeserver.yaml
|
ExecStart=/home/synapse/synapse/env/bin/python -m synapse.app.homeserver --config-path=/home/synapse/synapse/homeserver.yaml
|
||||||
SyslogIdentifier=matrix-synapse
|
SyslogIdentifier=matrix-synapse
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,9 @@
|
||||||
|
matrix-synapse-py3 (1.7.0) stable; urgency=medium
|
||||||
|
|
||||||
|
* New synapse release 1.7.0.
|
||||||
|
|
||||||
|
-- Synapse Packaging team <packages@matrix.org> Fri, 13 Dec 2019 10:19:38 +0000
|
||||||
|
|
||||||
matrix-synapse-py3 (1.6.1) stable; urgency=medium
|
matrix-synapse-py3 (1.6.1) stable; urgency=medium
|
||||||
|
|
||||||
* New synapse release 1.6.1.
|
* New synapse release 1.6.1.
|
||||||
|
|
|
||||||
|
|
@ -130,3 +130,15 @@ docker run -it --rm \
|
||||||
This will generate the same configuration file as the legacy mode used, but
|
This will generate the same configuration file as the legacy mode used, but
|
||||||
will store it in `/data/homeserver.yaml` instead of a temporary location. You
|
will store it in `/data/homeserver.yaml` instead of a temporary location. You
|
||||||
can then use it as shown above at [Running synapse](#running-synapse).
|
can then use it as shown above at [Running synapse](#running-synapse).
|
||||||
|
|
||||||
|
## Building the image
|
||||||
|
|
||||||
|
If you need to build the image from a Synapse checkout, use the following `docker
|
||||||
|
build` command from the repo's root:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker build -t matrixdotorg/synapse -f docker/Dockerfile .
|
||||||
|
```
|
||||||
|
|
||||||
|
You can choose to build a different docker image by changing the value of the `-f` flag to
|
||||||
|
point to another Dockerfile.
|
||||||
|
|
|
||||||
|
|
@ -21,3 +21,20 @@ It returns a JSON body like the following:
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
# Quarantine media in a room
|
||||||
|
|
||||||
|
This API 'quarantines' all the media in a room.
|
||||||
|
|
||||||
|
The API is:
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /_synapse/admin/v1/quarantine_media/<room_id>
|
||||||
|
|
||||||
|
{}
|
||||||
|
```
|
||||||
|
|
||||||
|
Quarantining media means that it is marked as inaccessible by users. It applies
|
||||||
|
to any local media, and any locally-cached copies of remote media.
|
||||||
|
|
||||||
|
The media file itself (and any thumbnails) is not deleted from the server.
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,48 @@
|
||||||
|
List Accounts
|
||||||
|
=============
|
||||||
|
|
||||||
|
This API returns all local user accounts.
|
||||||
|
|
||||||
|
The api is::
|
||||||
|
|
||||||
|
GET /_synapse/admin/v2/users?from=0&limit=10&guests=false
|
||||||
|
|
||||||
|
including an ``access_token`` of a server admin.
|
||||||
|
The parameters ``from`` and ``limit`` are required only for pagination.
|
||||||
|
By default, a ``limit`` of 100 is used.
|
||||||
|
The parameter ``user_id`` can be used to select only users with user ids that
|
||||||
|
contain this value.
|
||||||
|
The parameter ``guests=false`` can be used to exclude guest users,
|
||||||
|
default is to include guest users.
|
||||||
|
The parameter ``deactivated=true`` can be used to include deactivated users,
|
||||||
|
default is to exclude deactivated users.
|
||||||
|
If the endpoint does not return a ``next_token`` then there are no more users left.
|
||||||
|
It returns a JSON body like the following:
|
||||||
|
|
||||||
|
.. code:: json
|
||||||
|
|
||||||
|
{
|
||||||
|
"users": [
|
||||||
|
{
|
||||||
|
"name": "<user_id1>",
|
||||||
|
"password_hash": "<password_hash1>",
|
||||||
|
"is_guest": 0,
|
||||||
|
"admin": 0,
|
||||||
|
"user_type": null,
|
||||||
|
"deactivated": 0
|
||||||
|
}, {
|
||||||
|
"name": "<user_id2>",
|
||||||
|
"password_hash": "<password_hash2>",
|
||||||
|
"is_guest": 0,
|
||||||
|
"admin": 1,
|
||||||
|
"user_type": null,
|
||||||
|
"deactivated": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"next_token": "100"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
Query Account
|
Query Account
|
||||||
=============
|
=============
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -54,15 +54,16 @@ pid_file: DATADIR/homeserver.pid
|
||||||
#
|
#
|
||||||
#require_auth_for_profile_requests: true
|
#require_auth_for_profile_requests: true
|
||||||
|
|
||||||
# If set to 'false', requires authentication to access the server's public rooms
|
# If set to 'true', removes the need for authentication to access the server's
|
||||||
# directory through the client API. Defaults to 'true'.
|
# public rooms directory through the client API, meaning that anyone can
|
||||||
|
# query the room directory. Defaults to 'false'.
|
||||||
#
|
#
|
||||||
#allow_public_rooms_without_auth: false
|
#allow_public_rooms_without_auth: true
|
||||||
|
|
||||||
# If set to 'false', forbids any other homeserver to fetch the server's public
|
# If set to 'true', allows any other homeserver to fetch the server's public
|
||||||
# rooms directory via federation. Defaults to 'true'.
|
# rooms directory via federation. Defaults to 'false'.
|
||||||
#
|
#
|
||||||
#allow_public_rooms_over_federation: false
|
#allow_public_rooms_over_federation: true
|
||||||
|
|
||||||
# The default room version for newly created rooms.
|
# The default room version for newly created rooms.
|
||||||
#
|
#
|
||||||
|
|
@ -328,6 +329,69 @@ listeners:
|
||||||
#
|
#
|
||||||
#user_ips_max_age: 14d
|
#user_ips_max_age: 14d
|
||||||
|
|
||||||
|
# Message retention policy at the server level.
|
||||||
|
#
|
||||||
|
# Room admins and mods can define a retention period for their rooms using the
|
||||||
|
# 'm.room.retention' state event, and server admins can cap this period by setting
|
||||||
|
# the 'allowed_lifetime_min' and 'allowed_lifetime_max' config options.
|
||||||
|
#
|
||||||
|
# If this feature is enabled, Synapse will regularly look for and purge events
|
||||||
|
# which are older than the room's maximum retention period. Synapse will also
|
||||||
|
# filter events received over federation so that events that should have been
|
||||||
|
# purged are ignored and not stored again.
|
||||||
|
#
|
||||||
|
retention:
|
||||||
|
# The message retention policies feature is disabled by default. Uncomment the
|
||||||
|
# following line to enable it.
|
||||||
|
#
|
||||||
|
#enabled: true
|
||||||
|
|
||||||
|
# Default retention policy. If set, Synapse will apply it to rooms that lack the
|
||||||
|
# 'm.room.retention' state event. Currently, the value of 'min_lifetime' doesn't
|
||||||
|
# matter much because Synapse doesn't take it into account yet.
|
||||||
|
#
|
||||||
|
#default_policy:
|
||||||
|
# min_lifetime: 1d
|
||||||
|
# max_lifetime: 1y
|
||||||
|
|
||||||
|
# Retention policy limits. If set, a user won't be able to send a
|
||||||
|
# 'm.room.retention' event which features a 'min_lifetime' or a 'max_lifetime'
|
||||||
|
# that's not within this range. This is especially useful in closed federations,
|
||||||
|
# in which server admins can make sure every federating server applies the same
|
||||||
|
# rules.
|
||||||
|
#
|
||||||
|
#allowed_lifetime_min: 1d
|
||||||
|
#allowed_lifetime_max: 1y
|
||||||
|
|
||||||
|
# Server admins can define the settings of the background jobs purging the
|
||||||
|
# events which lifetime has expired under the 'purge_jobs' section.
|
||||||
|
#
|
||||||
|
# If no configuration is provided, a single job will be set up to delete expired
|
||||||
|
# events in every room daily.
|
||||||
|
#
|
||||||
|
# Each job's configuration defines which range of message lifetimes the job
|
||||||
|
# takes care of. For example, if 'shortest_max_lifetime' is '2d' and
|
||||||
|
# 'longest_max_lifetime' is '3d', the job will handle purging expired events in
|
||||||
|
# rooms whose state defines a 'max_lifetime' that's both higher than 2 days, and
|
||||||
|
# lower than or equal to 3 days. Both the minimum and the maximum value of a
|
||||||
|
# range are optional, e.g. a job with no 'shortest_max_lifetime' and a
|
||||||
|
# 'longest_max_lifetime' of '3d' will handle every room with a retention policy
|
||||||
|
# which 'max_lifetime' is lower than or equal to three days.
|
||||||
|
#
|
||||||
|
# The rationale for this per-job configuration is that some rooms might have a
|
||||||
|
# retention policy with a low 'max_lifetime', where history needs to be purged
|
||||||
|
# of outdated messages on a very frequent basis (e.g. every 5min), but not want
|
||||||
|
# that purge to be performed by a job that's iterating over every room it knows,
|
||||||
|
# which would be quite heavy on the server.
|
||||||
|
#
|
||||||
|
#purge_jobs:
|
||||||
|
# - shortest_max_lifetime: 1d
|
||||||
|
# longest_max_lifetime: 3d
|
||||||
|
# interval: 5m:
|
||||||
|
# - shortest_max_lifetime: 3d
|
||||||
|
# longest_max_lifetime: 1y
|
||||||
|
# interval: 24h
|
||||||
|
|
||||||
|
|
||||||
## TLS ##
|
## TLS ##
|
||||||
|
|
||||||
|
|
@ -1270,8 +1334,23 @@ password_config:
|
||||||
# smtp_user: "exampleusername"
|
# smtp_user: "exampleusername"
|
||||||
# smtp_pass: "examplepassword"
|
# smtp_pass: "examplepassword"
|
||||||
# require_transport_security: false
|
# require_transport_security: false
|
||||||
|
#
|
||||||
|
# # notif_from defines the "From" address to use when sending emails.
|
||||||
|
# # It must be set if email sending is enabled.
|
||||||
|
# #
|
||||||
|
# # The placeholder '%(app)s' will be replaced by the application name,
|
||||||
|
# # which is normally 'app_name' (below), but may be overridden by the
|
||||||
|
# # Matrix client application.
|
||||||
|
# #
|
||||||
|
# # Note that the placeholder must be written '%(app)s', including the
|
||||||
|
# # trailing 's'.
|
||||||
|
# #
|
||||||
# notif_from: "Your Friendly %(app)s homeserver <noreply@example.com>"
|
# notif_from: "Your Friendly %(app)s homeserver <noreply@example.com>"
|
||||||
# app_name: Matrix
|
#
|
||||||
|
# # app_name defines the default value for '%(app)s' in notif_from. It
|
||||||
|
# # defaults to 'Matrix'.
|
||||||
|
# #
|
||||||
|
# #app_name: my_branded_matrix_server
|
||||||
#
|
#
|
||||||
# # Enable email notifications by default
|
# # Enable email notifications by default
|
||||||
# #
|
# #
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,6 @@ who are present in a publicly viewable room present on the server.
|
||||||
|
|
||||||
The directory info is stored in various tables, which can (typically after
|
The directory info is stored in various tables, which can (typically after
|
||||||
DB corruption) get stale or out of sync. If this happens, for now the
|
DB corruption) get stale or out of sync. If this happens, for now the
|
||||||
solution to fix it is to execute the SQL here
|
solution to fix it is to execute the SQL [here](../synapse/storage/data_stores/main/schema/delta/53/user_dir_populate.sql)
|
||||||
https://github.com/matrix-org/synapse/blob/master/synapse/storage/schema/delta/53/user_dir_populate.sql
|
|
||||||
and then restart synapse. This should then start a background task to
|
and then restart synapse. This should then start a background task to
|
||||||
flush the current tables and regenerate the directory.
|
flush the current tables and regenerate the directory.
|
||||||
|
|
|
||||||
|
|
@ -27,7 +27,7 @@ class Store(object):
|
||||||
"_store_pdu_reference_hash_txn"
|
"_store_pdu_reference_hash_txn"
|
||||||
]
|
]
|
||||||
_store_prev_pdu_hash_txn = SignatureStore.__dict__["_store_prev_pdu_hash_txn"]
|
_store_prev_pdu_hash_txn = SignatureStore.__dict__["_store_prev_pdu_hash_txn"]
|
||||||
_simple_insert_txn = SQLBaseStore.__dict__["_simple_insert_txn"]
|
simple_insert_txn = SQLBaseStore.__dict__["simple_insert_txn"]
|
||||||
|
|
||||||
|
|
||||||
store = Store()
|
store = Store()
|
||||||
|
|
|
||||||
|
|
@ -58,10 +58,10 @@ if __name__ == "__main__":
|
||||||
" on it."
|
" on it."
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
parser.add_argument("-v", action='store_true')
|
parser.add_argument("-v", action="store_true")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--database-config",
|
"--database-config",
|
||||||
type=argparse.FileType('r'),
|
type=argparse.FileType("r"),
|
||||||
required=True,
|
required=True,
|
||||||
help="A database config file for either a SQLite3 database or a PostgreSQL one.",
|
help="A database config file for either a SQLite3 database or a PostgreSQL one.",
|
||||||
)
|
)
|
||||||
|
|
@ -101,10 +101,7 @@ if __name__ == "__main__":
|
||||||
|
|
||||||
# Instantiate and initialise the homeserver object.
|
# Instantiate and initialise the homeserver object.
|
||||||
hs = MockHomeserver(
|
hs = MockHomeserver(
|
||||||
config,
|
config, database_engine, db_conn, db_config=config.database_config,
|
||||||
database_engine,
|
|
||||||
db_conn,
|
|
||||||
db_config=config.database_config,
|
|
||||||
)
|
)
|
||||||
# setup instantiates the store within the homeserver object.
|
# setup instantiates the store within the homeserver object.
|
||||||
hs.setup()
|
hs.setup()
|
||||||
|
|
@ -112,13 +109,13 @@ if __name__ == "__main__":
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def run_background_updates():
|
def run_background_updates():
|
||||||
yield store.run_background_updates(sleep=False)
|
yield store.db.updates.run_background_updates(sleep=False)
|
||||||
# Stop the reactor to exit the script once every background update is run.
|
# Stop the reactor to exit the script once every background update is run.
|
||||||
reactor.stop()
|
reactor.stop()
|
||||||
|
|
||||||
# Apply all background updates on the database.
|
# Apply all background updates on the database.
|
||||||
reactor.callWhenRunning(lambda: run_as_background_process(
|
reactor.callWhenRunning(
|
||||||
"background_updates", run_background_updates
|
lambda: run_as_background_process("background_updates", run_background_updates)
|
||||||
))
|
)
|
||||||
|
|
||||||
reactor.run()
|
reactor.run()
|
||||||
|
|
|
||||||
|
|
@ -47,6 +47,7 @@ from synapse.storage.data_stores.main.media_repository import (
|
||||||
from synapse.storage.data_stores.main.registration import (
|
from synapse.storage.data_stores.main.registration import (
|
||||||
RegistrationBackgroundUpdateStore,
|
RegistrationBackgroundUpdateStore,
|
||||||
)
|
)
|
||||||
|
from synapse.storage.data_stores.main.room import RoomBackgroundUpdateStore
|
||||||
from synapse.storage.data_stores.main.roommember import RoomMemberBackgroundUpdateStore
|
from synapse.storage.data_stores.main.roommember import RoomMemberBackgroundUpdateStore
|
||||||
from synapse.storage.data_stores.main.search import SearchBackgroundUpdateStore
|
from synapse.storage.data_stores.main.search import SearchBackgroundUpdateStore
|
||||||
from synapse.storage.data_stores.main.state import StateBackgroundUpdateStore
|
from synapse.storage.data_stores.main.state import StateBackgroundUpdateStore
|
||||||
|
|
@ -54,6 +55,7 @@ from synapse.storage.data_stores.main.stats import StatsStore
|
||||||
from synapse.storage.data_stores.main.user_directory import (
|
from synapse.storage.data_stores.main.user_directory import (
|
||||||
UserDirectoryBackgroundUpdateStore,
|
UserDirectoryBackgroundUpdateStore,
|
||||||
)
|
)
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.storage.engines import create_engine
|
from synapse.storage.engines import create_engine
|
||||||
from synapse.storage.prepare_database import prepare_database
|
from synapse.storage.prepare_database import prepare_database
|
||||||
from synapse.util import Clock
|
from synapse.util import Clock
|
||||||
|
|
@ -131,54 +133,22 @@ class Store(
|
||||||
EventsBackgroundUpdatesStore,
|
EventsBackgroundUpdatesStore,
|
||||||
MediaRepositoryBackgroundUpdateStore,
|
MediaRepositoryBackgroundUpdateStore,
|
||||||
RegistrationBackgroundUpdateStore,
|
RegistrationBackgroundUpdateStore,
|
||||||
|
RoomBackgroundUpdateStore,
|
||||||
RoomMemberBackgroundUpdateStore,
|
RoomMemberBackgroundUpdateStore,
|
||||||
SearchBackgroundUpdateStore,
|
SearchBackgroundUpdateStore,
|
||||||
StateBackgroundUpdateStore,
|
StateBackgroundUpdateStore,
|
||||||
UserDirectoryBackgroundUpdateStore,
|
UserDirectoryBackgroundUpdateStore,
|
||||||
StatsStore,
|
StatsStore,
|
||||||
):
|
):
|
||||||
def __init__(self, db_conn, hs):
|
|
||||||
super().__init__(db_conn, hs)
|
|
||||||
self.db_pool = hs.get_db_pool()
|
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
|
||||||
def runInteraction(self, desc, func, *args, **kwargs):
|
|
||||||
def r(conn):
|
|
||||||
try:
|
|
||||||
i = 0
|
|
||||||
N = 5
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
txn = conn.cursor()
|
|
||||||
return func(
|
|
||||||
LoggingTransaction(txn, desc, self.database_engine, [], []),
|
|
||||||
*args,
|
|
||||||
**kwargs
|
|
||||||
)
|
|
||||||
except self.database_engine.module.DatabaseError as e:
|
|
||||||
if self.database_engine.is_deadlock(e):
|
|
||||||
logger.warning("[TXN DEADLOCK] {%s} %d/%d", desc, i, N)
|
|
||||||
if i < N:
|
|
||||||
i += 1
|
|
||||||
conn.rollback()
|
|
||||||
continue
|
|
||||||
raise
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug("[TXN FAIL] {%s} %s", desc, e)
|
|
||||||
raise
|
|
||||||
|
|
||||||
with PreserveLoggingContext():
|
|
||||||
return (yield self.db_pool.runWithConnection(r))
|
|
||||||
|
|
||||||
def execute(self, f, *args, **kwargs):
|
def execute(self, f, *args, **kwargs):
|
||||||
return self.runInteraction(f.__name__, f, *args, **kwargs)
|
return self.db.runInteraction(f.__name__, f, *args, **kwargs)
|
||||||
|
|
||||||
def execute_sql(self, sql, *args):
|
def execute_sql(self, sql, *args):
|
||||||
def r(txn):
|
def r(txn):
|
||||||
txn.execute(sql, args)
|
txn.execute(sql, args)
|
||||||
return txn.fetchall()
|
return txn.fetchall()
|
||||||
|
|
||||||
return self.runInteraction("execute_sql", r)
|
return self.db.runInteraction("execute_sql", r)
|
||||||
|
|
||||||
def insert_many_txn(self, txn, table, headers, rows):
|
def insert_many_txn(self, txn, table, headers, rows):
|
||||||
sql = "INSERT INTO %s (%s) VALUES (%s)" % (
|
sql = "INSERT INTO %s (%s) VALUES (%s)" % (
|
||||||
|
|
@ -221,7 +191,7 @@ class Porter(object):
|
||||||
def setup_table(self, table):
|
def setup_table(self, table):
|
||||||
if table in APPEND_ONLY_TABLES:
|
if table in APPEND_ONLY_TABLES:
|
||||||
# It's safe to just carry on inserting.
|
# It's safe to just carry on inserting.
|
||||||
row = yield self.postgres_store._simple_select_one(
|
row = yield self.postgres_store.db.simple_select_one(
|
||||||
table="port_from_sqlite3",
|
table="port_from_sqlite3",
|
||||||
keyvalues={"table_name": table},
|
keyvalues={"table_name": table},
|
||||||
retcols=("forward_rowid", "backward_rowid"),
|
retcols=("forward_rowid", "backward_rowid"),
|
||||||
|
|
@ -231,12 +201,14 @@ class Porter(object):
|
||||||
total_to_port = None
|
total_to_port = None
|
||||||
if row is None:
|
if row is None:
|
||||||
if table == "sent_transactions":
|
if table == "sent_transactions":
|
||||||
forward_chunk, already_ported, total_to_port = (
|
(
|
||||||
yield self._setup_sent_transactions()
|
forward_chunk,
|
||||||
)
|
already_ported,
|
||||||
|
total_to_port,
|
||||||
|
) = yield self._setup_sent_transactions()
|
||||||
backward_chunk = 0
|
backward_chunk = 0
|
||||||
else:
|
else:
|
||||||
yield self.postgres_store._simple_insert(
|
yield self.postgres_store.db.simple_insert(
|
||||||
table="port_from_sqlite3",
|
table="port_from_sqlite3",
|
||||||
values={
|
values={
|
||||||
"table_name": table,
|
"table_name": table,
|
||||||
|
|
@ -266,7 +238,7 @@ class Porter(object):
|
||||||
|
|
||||||
yield self.postgres_store.execute(delete_all)
|
yield self.postgres_store.execute(delete_all)
|
||||||
|
|
||||||
yield self.postgres_store._simple_insert(
|
yield self.postgres_store.db.simple_insert(
|
||||||
table="port_from_sqlite3",
|
table="port_from_sqlite3",
|
||||||
values={"table_name": table, "forward_rowid": 1, "backward_rowid": 0},
|
values={"table_name": table, "forward_rowid": 1, "backward_rowid": 0},
|
||||||
)
|
)
|
||||||
|
|
@ -320,7 +292,7 @@ class Porter(object):
|
||||||
if table == "user_directory_stream_pos":
|
if table == "user_directory_stream_pos":
|
||||||
# We need to make sure there is a single row, `(X, null), as that is
|
# We need to make sure there is a single row, `(X, null), as that is
|
||||||
# what synapse expects to be there.
|
# what synapse expects to be there.
|
||||||
yield self.postgres_store._simple_insert(
|
yield self.postgres_store.db.simple_insert(
|
||||||
table=table, values={"stream_id": None}
|
table=table, values={"stream_id": None}
|
||||||
)
|
)
|
||||||
self.progress.update(table, table_size) # Mark table as done
|
self.progress.update(table, table_size) # Mark table as done
|
||||||
|
|
@ -361,7 +333,9 @@ class Porter(object):
|
||||||
|
|
||||||
return headers, forward_rows, backward_rows
|
return headers, forward_rows, backward_rows
|
||||||
|
|
||||||
headers, frows, brows = yield self.sqlite_store.runInteraction("select", r)
|
headers, frows, brows = yield self.sqlite_store.db.runInteraction(
|
||||||
|
"select", r
|
||||||
|
)
|
||||||
|
|
||||||
if frows or brows:
|
if frows or brows:
|
||||||
if frows:
|
if frows:
|
||||||
|
|
@ -375,7 +349,7 @@ class Porter(object):
|
||||||
def insert(txn):
|
def insert(txn):
|
||||||
self.postgres_store.insert_many_txn(txn, table, headers[1:], rows)
|
self.postgres_store.insert_many_txn(txn, table, headers[1:], rows)
|
||||||
|
|
||||||
self.postgres_store._simple_update_one_txn(
|
self.postgres_store.db.simple_update_one_txn(
|
||||||
txn,
|
txn,
|
||||||
table="port_from_sqlite3",
|
table="port_from_sqlite3",
|
||||||
keyvalues={"table_name": table},
|
keyvalues={"table_name": table},
|
||||||
|
|
@ -414,7 +388,7 @@ class Porter(object):
|
||||||
|
|
||||||
return headers, rows
|
return headers, rows
|
||||||
|
|
||||||
headers, rows = yield self.sqlite_store.runInteraction("select", r)
|
headers, rows = yield self.sqlite_store.db.runInteraction("select", r)
|
||||||
|
|
||||||
if rows:
|
if rows:
|
||||||
forward_chunk = rows[-1][0] + 1
|
forward_chunk = rows[-1][0] + 1
|
||||||
|
|
@ -431,8 +405,8 @@ class Porter(object):
|
||||||
rows_dict = []
|
rows_dict = []
|
||||||
for row in rows:
|
for row in rows:
|
||||||
d = dict(zip(headers, row))
|
d = dict(zip(headers, row))
|
||||||
if "\0" in d['value']:
|
if "\0" in d["value"]:
|
||||||
logger.warning('dropping search row %s', d)
|
logger.warning("dropping search row %s", d)
|
||||||
else:
|
else:
|
||||||
rows_dict.append(d)
|
rows_dict.append(d)
|
||||||
|
|
||||||
|
|
@ -452,7 +426,7 @@ class Porter(object):
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
|
|
||||||
self.postgres_store._simple_update_one_txn(
|
self.postgres_store.db.simple_update_one_txn(
|
||||||
txn,
|
txn,
|
||||||
table="port_from_sqlite3",
|
table="port_from_sqlite3",
|
||||||
keyvalues={"table_name": "event_search"},
|
keyvalues={"table_name": "event_search"},
|
||||||
|
|
@ -502,17 +476,14 @@ class Porter(object):
|
||||||
self.progress.set_state("Preparing %s" % config["name"])
|
self.progress.set_state("Preparing %s" % config["name"])
|
||||||
conn = self.setup_db(config, engine)
|
conn = self.setup_db(config, engine)
|
||||||
|
|
||||||
db_pool = adbapi.ConnectionPool(
|
db_pool = adbapi.ConnectionPool(config["name"], **config["args"])
|
||||||
config["name"], **config["args"]
|
|
||||||
)
|
|
||||||
|
|
||||||
hs = MockHomeserver(self.hs_config, engine, conn, db_pool)
|
hs = MockHomeserver(self.hs_config, engine, conn, db_pool)
|
||||||
|
|
||||||
store = Store(conn, hs)
|
store = Store(Database(hs), conn, hs)
|
||||||
|
|
||||||
yield store.runInteraction(
|
yield store.db.runInteraction(
|
||||||
"%s_engine.check_database" % config["name"],
|
"%s_engine.check_database" % config["name"], engine.check_database,
|
||||||
engine.check_database,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
return store
|
return store
|
||||||
|
|
@ -520,7 +491,9 @@ class Porter(object):
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def run_background_updates_on_postgres(self):
|
def run_background_updates_on_postgres(self):
|
||||||
# Manually apply all background updates on the PostgreSQL database.
|
# Manually apply all background updates on the PostgreSQL database.
|
||||||
postgres_ready = yield self.postgres_store.has_completed_background_updates()
|
postgres_ready = (
|
||||||
|
yield self.postgres_store.db.updates.has_completed_background_updates()
|
||||||
|
)
|
||||||
|
|
||||||
if not postgres_ready:
|
if not postgres_ready:
|
||||||
# Only say that we're running background updates when there are background
|
# Only say that we're running background updates when there are background
|
||||||
|
|
@ -528,9 +501,9 @@ class Porter(object):
|
||||||
self.progress.set_state("Running background updates on PostgreSQL")
|
self.progress.set_state("Running background updates on PostgreSQL")
|
||||||
|
|
||||||
while not postgres_ready:
|
while not postgres_ready:
|
||||||
yield self.postgres_store.do_next_background_update(100)
|
yield self.postgres_store.db.updates.do_next_background_update(100)
|
||||||
postgres_ready = yield (
|
postgres_ready = yield (
|
||||||
self.postgres_store.has_completed_background_updates()
|
self.postgres_store.db.updates.has_completed_background_updates()
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
|
|
@ -539,7 +512,9 @@ class Porter(object):
|
||||||
self.sqlite_store = yield self.build_db_store(self.sqlite_config)
|
self.sqlite_store = yield self.build_db_store(self.sqlite_config)
|
||||||
|
|
||||||
# Check if all background updates are done, abort if not.
|
# Check if all background updates are done, abort if not.
|
||||||
updates_complete = yield self.sqlite_store.has_completed_background_updates()
|
updates_complete = (
|
||||||
|
yield self.sqlite_store.db.updates.has_completed_background_updates()
|
||||||
|
)
|
||||||
if not updates_complete:
|
if not updates_complete:
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
"Pending background updates exist in the SQLite3 database."
|
"Pending background updates exist in the SQLite3 database."
|
||||||
|
|
@ -580,22 +555,22 @@ class Porter(object):
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
yield self.postgres_store.runInteraction("alter_table", alter_table)
|
yield self.postgres_store.db.runInteraction("alter_table", alter_table)
|
||||||
except Exception:
|
except Exception:
|
||||||
# On Error Resume Next
|
# On Error Resume Next
|
||||||
pass
|
pass
|
||||||
|
|
||||||
yield self.postgres_store.runInteraction(
|
yield self.postgres_store.db.runInteraction(
|
||||||
"create_port_table", create_port_table
|
"create_port_table", create_port_table
|
||||||
)
|
)
|
||||||
|
|
||||||
# Step 2. Get tables.
|
# Step 2. Get tables.
|
||||||
self.progress.set_state("Fetching tables")
|
self.progress.set_state("Fetching tables")
|
||||||
sqlite_tables = yield self.sqlite_store._simple_select_onecol(
|
sqlite_tables = yield self.sqlite_store.db.simple_select_onecol(
|
||||||
table="sqlite_master", keyvalues={"type": "table"}, retcol="name"
|
table="sqlite_master", keyvalues={"type": "table"}, retcol="name"
|
||||||
)
|
)
|
||||||
|
|
||||||
postgres_tables = yield self.postgres_store._simple_select_onecol(
|
postgres_tables = yield self.postgres_store.db.simple_select_onecol(
|
||||||
table="information_schema.tables",
|
table="information_schema.tables",
|
||||||
keyvalues={},
|
keyvalues={},
|
||||||
retcol="distinct table_name",
|
retcol="distinct table_name",
|
||||||
|
|
@ -685,11 +660,11 @@ class Porter(object):
|
||||||
rows = txn.fetchall()
|
rows = txn.fetchall()
|
||||||
headers = [column[0] for column in txn.description]
|
headers = [column[0] for column in txn.description]
|
||||||
|
|
||||||
ts_ind = headers.index('ts')
|
ts_ind = headers.index("ts")
|
||||||
|
|
||||||
return headers, [r for r in rows if r[ts_ind] < yesterday]
|
return headers, [r for r in rows if r[ts_ind] < yesterday]
|
||||||
|
|
||||||
headers, rows = yield self.sqlite_store.runInteraction("select", r)
|
headers, rows = yield self.sqlite_store.db.runInteraction("select", r)
|
||||||
|
|
||||||
rows = self._convert_rows("sent_transactions", headers, rows)
|
rows = self._convert_rows("sent_transactions", headers, rows)
|
||||||
|
|
||||||
|
|
@ -722,7 +697,7 @@ class Porter(object):
|
||||||
next_chunk = yield self.sqlite_store.execute(get_start_id)
|
next_chunk = yield self.sqlite_store.execute(get_start_id)
|
||||||
next_chunk = max(max_inserted_rowid + 1, next_chunk)
|
next_chunk = max(max_inserted_rowid + 1, next_chunk)
|
||||||
|
|
||||||
yield self.postgres_store._simple_insert(
|
yield self.postgres_store.db.simple_insert(
|
||||||
table="port_from_sqlite3",
|
table="port_from_sqlite3",
|
||||||
values={
|
values={
|
||||||
"table_name": "sent_transactions",
|
"table_name": "sent_transactions",
|
||||||
|
|
@ -735,7 +710,7 @@ class Porter(object):
|
||||||
txn.execute(
|
txn.execute(
|
||||||
"SELECT count(*) FROM sent_transactions" " WHERE ts >= ?", (yesterday,)
|
"SELECT count(*) FROM sent_transactions" " WHERE ts >= ?", (yesterday,)
|
||||||
)
|
)
|
||||||
size, = txn.fetchone()
|
(size,) = txn.fetchone()
|
||||||
return int(size)
|
return int(size)
|
||||||
|
|
||||||
remaining_count = yield self.sqlite_store.execute(get_sent_table_size)
|
remaining_count = yield self.sqlite_store.execute(get_sent_table_size)
|
||||||
|
|
@ -782,10 +757,13 @@ class Porter(object):
|
||||||
def _setup_state_group_id_seq(self):
|
def _setup_state_group_id_seq(self):
|
||||||
def r(txn):
|
def r(txn):
|
||||||
txn.execute("SELECT MAX(id) FROM state_groups")
|
txn.execute("SELECT MAX(id) FROM state_groups")
|
||||||
next_id = txn.fetchone()[0] + 1
|
curr_id = txn.fetchone()[0]
|
||||||
|
if not curr_id:
|
||||||
|
return
|
||||||
|
next_id = curr_id + 1
|
||||||
txn.execute("ALTER SEQUENCE state_group_id_seq RESTART WITH %s", (next_id,))
|
txn.execute("ALTER SEQUENCE state_group_id_seq RESTART WITH %s", (next_id,))
|
||||||
|
|
||||||
return self.postgres_store.runInteraction("setup_state_group_id_seq", r)
|
return self.postgres_store.db.runInteraction("setup_state_group_id_seq", r)
|
||||||
|
|
||||||
|
|
||||||
##############################################
|
##############################################
|
||||||
|
|
@ -866,7 +844,7 @@ class CursesProgress(Progress):
|
||||||
duration = int(now) - int(self.start_time)
|
duration = int(now) - int(self.start_time)
|
||||||
|
|
||||||
minutes, seconds = divmod(duration, 60)
|
minutes, seconds = divmod(duration, 60)
|
||||||
duration_str = '%02dm %02ds' % (minutes, seconds)
|
duration_str = "%02dm %02ds" % (minutes, seconds)
|
||||||
|
|
||||||
if self.finished:
|
if self.finished:
|
||||||
status = "Time spent: %s (Done!)" % (duration_str,)
|
status = "Time spent: %s (Done!)" % (duration_str,)
|
||||||
|
|
@ -876,7 +854,7 @@ class CursesProgress(Progress):
|
||||||
left = float(self.total_remaining) / self.total_processed
|
left = float(self.total_remaining) / self.total_processed
|
||||||
|
|
||||||
est_remaining = (int(now) - self.start_time) * left
|
est_remaining = (int(now) - self.start_time) * left
|
||||||
est_remaining_str = '%02dm %02ds remaining' % divmod(est_remaining, 60)
|
est_remaining_str = "%02dm %02ds remaining" % divmod(est_remaining, 60)
|
||||||
else:
|
else:
|
||||||
est_remaining_str = "Unknown"
|
est_remaining_str = "Unknown"
|
||||||
status = "Time spent: %s (est. remaining: %s)" % (
|
status = "Time spent: %s (est. remaining: %s)" % (
|
||||||
|
|
@ -962,7 +940,7 @@ if __name__ == "__main__":
|
||||||
description="A script to port an existing synapse SQLite database to"
|
description="A script to port an existing synapse SQLite database to"
|
||||||
" a new PostgreSQL database."
|
" a new PostgreSQL database."
|
||||||
)
|
)
|
||||||
parser.add_argument("-v", action='store_true')
|
parser.add_argument("-v", action="store_true")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--sqlite-database",
|
"--sqlite-database",
|
||||||
required=True,
|
required=True,
|
||||||
|
|
@ -971,12 +949,12 @@ if __name__ == "__main__":
|
||||||
)
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--postgres-config",
|
"--postgres-config",
|
||||||
type=argparse.FileType('r'),
|
type=argparse.FileType("r"),
|
||||||
required=True,
|
required=True,
|
||||||
help="The database config file for the PostgreSQL database",
|
help="The database config file for the PostgreSQL database",
|
||||||
)
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--curses", action='store_true', help="display a curses based progress UI"
|
"--curses", action="store_true", help="display a curses based progress UI"
|
||||||
)
|
)
|
||||||
|
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
|
|
@ -1052,3 +1030,4 @@ if __name__ == "__main__":
|
||||||
if end_error_exec_info:
|
if end_error_exec_info:
|
||||||
exc_type, exc_value, exc_traceback = end_error_exec_info
|
exc_type, exc_value, exc_traceback = end_error_exec_info
|
||||||
traceback.print_exception(exc_type, exc_value, exc_traceback)
|
traceback.print_exception(exc_type, exc_value, exc_traceback)
|
||||||
|
sys.exit(5)
|
||||||
|
|
|
||||||
|
|
@ -36,7 +36,7 @@ try:
|
||||||
except ImportError:
|
except ImportError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
__version__ = "1.6.1"
|
__version__ = "1.7.0"
|
||||||
|
|
||||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||||
# We import here so that we don't have to install a bunch of deps when
|
# We import here so that we don't have to install a bunch of deps when
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,8 @@
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
# Copyright 2014-2016 OpenMarket Ltd
|
# Copyright 2014-2016 OpenMarket Ltd
|
||||||
# Copyright 2017 Vector Creations Ltd
|
# Copyright 2017 Vector Creations Ltd
|
||||||
# Copyright 2018 New Vector Ltd
|
# Copyright 2018-2019 New Vector Ltd
|
||||||
|
# Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||||
#
|
#
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
# you may not use this file except in compliance with the License.
|
# you may not use this file except in compliance with the License.
|
||||||
|
|
@ -94,6 +95,8 @@ class EventTypes(object):
|
||||||
ServerACL = "m.room.server_acl"
|
ServerACL = "m.room.server_acl"
|
||||||
Pinned = "m.room.pinned_events"
|
Pinned = "m.room.pinned_events"
|
||||||
|
|
||||||
|
Retention = "m.room.retention"
|
||||||
|
|
||||||
|
|
||||||
class RejectedReason(object):
|
class RejectedReason(object):
|
||||||
AUTH_ERROR = "auth_error"
|
AUTH_ERROR = "auth_error"
|
||||||
|
|
@ -145,3 +148,7 @@ class EventContentFields(object):
|
||||||
|
|
||||||
# Labels for the event, cf https://github.com/matrix-org/matrix-doc/pull/2326
|
# Labels for the event, cf https://github.com/matrix-org/matrix-doc/pull/2326
|
||||||
LABELS = "org.matrix.labels"
|
LABELS = "org.matrix.labels"
|
||||||
|
|
||||||
|
# Timestamp to delete the event after
|
||||||
|
# cf https://github.com/matrix-org/matrix-doc/pull/2228
|
||||||
|
SELF_DESTRUCT_AFTER = "org.matrix.self_destruct_after"
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,8 @@
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
# Copyright 2015, 2016 OpenMarket Ltd
|
# Copyright 2015, 2016 OpenMarket Ltd
|
||||||
|
# Copyright 2017 Vector Creations Ltd
|
||||||
|
# Copyright 2018-2019 New Vector Ltd
|
||||||
|
# Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||||
#
|
#
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
# you may not use this file except in compliance with the License.
|
# you may not use this file except in compliance with the License.
|
||||||
|
|
|
||||||
|
|
@ -269,7 +269,7 @@ def start(hs, listeners=None):
|
||||||
|
|
||||||
# It is now safe to start your Synapse.
|
# It is now safe to start your Synapse.
|
||||||
hs.start_listening(listeners)
|
hs.start_listening(listeners)
|
||||||
hs.get_datastore().start_profiling()
|
hs.get_datastore().db.start_profiling()
|
||||||
|
|
||||||
setup_sentry(hs)
|
setup_sentry(hs)
|
||||||
setup_sdnotify(hs)
|
setup_sdnotify(hs)
|
||||||
|
|
|
||||||
|
|
@ -40,6 +40,7 @@ from synapse.replication.slave.storage.transactions import SlavedTransactionStor
|
||||||
from synapse.replication.tcp.client import ReplicationClientHandler
|
from synapse.replication.tcp.client import ReplicationClientHandler
|
||||||
from synapse.replication.tcp.streams._base import ReceiptsStream
|
from synapse.replication.tcp.streams._base import ReceiptsStream
|
||||||
from synapse.server import HomeServer
|
from synapse.server import HomeServer
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.storage.engines import create_engine
|
from synapse.storage.engines import create_engine
|
||||||
from synapse.types import ReadReceipt
|
from synapse.types import ReadReceipt
|
||||||
from synapse.util.async_helpers import Linearizer
|
from synapse.util.async_helpers import Linearizer
|
||||||
|
|
@ -59,8 +60,8 @@ class FederationSenderSlaveStore(
|
||||||
SlavedDeviceStore,
|
SlavedDeviceStore,
|
||||||
SlavedPresenceStore,
|
SlavedPresenceStore,
|
||||||
):
|
):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(FederationSenderSlaveStore, self).__init__(db_conn, hs)
|
super(FederationSenderSlaveStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
# We pull out the current federation stream position now so that we
|
# We pull out the current federation stream position now so that we
|
||||||
# always have a known value for the federation position in memory so
|
# always have a known value for the federation position in memory so
|
||||||
|
|
@ -69,7 +70,7 @@ class FederationSenderSlaveStore(
|
||||||
self.federation_out_pos_startup = self._get_federation_out_pos(db_conn)
|
self.federation_out_pos_startup = self._get_federation_out_pos(db_conn)
|
||||||
|
|
||||||
def _get_federation_out_pos(self, db_conn):
|
def _get_federation_out_pos(self, db_conn):
|
||||||
sql = "SELECT stream_id FROM federation_stream_position" " WHERE type = ?"
|
sql = "SELECT stream_id FROM federation_stream_position WHERE type = ?"
|
||||||
sql = self.database_engine.convert_param_style(sql)
|
sql = self.database_engine.convert_param_style(sql)
|
||||||
|
|
||||||
txn = db_conn.cursor()
|
txn = db_conn.cursor()
|
||||||
|
|
|
||||||
|
|
@ -68,9 +68,9 @@ from synapse.rest.key.v2 import KeyApiV2Resource
|
||||||
from synapse.rest.media.v0.content_repository import ContentRepoResource
|
from synapse.rest.media.v0.content_repository import ContentRepoResource
|
||||||
from synapse.rest.well_known import WellKnownResource
|
from synapse.rest.well_known import WellKnownResource
|
||||||
from synapse.server import HomeServer
|
from synapse.server import HomeServer
|
||||||
from synapse.storage import DataStore, are_all_users_on_domain
|
from synapse.storage import DataStore
|
||||||
from synapse.storage.engines import IncorrectDatabaseSetup, create_engine
|
from synapse.storage.engines import IncorrectDatabaseSetup, create_engine
|
||||||
from synapse.storage.prepare_database import UpgradeDatabaseException, prepare_database
|
from synapse.storage.prepare_database import UpgradeDatabaseException
|
||||||
from synapse.util.caches import CACHE_SIZE_FACTOR
|
from synapse.util.caches import CACHE_SIZE_FACTOR
|
||||||
from synapse.util.httpresourcetree import create_resource_tree
|
from synapse.util.httpresourcetree import create_resource_tree
|
||||||
from synapse.util.manhole import manhole
|
from synapse.util.manhole import manhole
|
||||||
|
|
@ -294,22 +294,6 @@ class SynapseHomeServer(HomeServer):
|
||||||
else:
|
else:
|
||||||
logger.warning("Unrecognized listener type: %s", listener["type"])
|
logger.warning("Unrecognized listener type: %s", listener["type"])
|
||||||
|
|
||||||
def run_startup_checks(self, db_conn, database_engine):
|
|
||||||
all_users_native = are_all_users_on_domain(
|
|
||||||
db_conn.cursor(), database_engine, self.hostname
|
|
||||||
)
|
|
||||||
if not all_users_native:
|
|
||||||
quit_with_error(
|
|
||||||
"Found users in database not native to %s!\n"
|
|
||||||
"You cannot changed a synapse server_name after it's been configured"
|
|
||||||
% (self.hostname,)
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
database_engine.check_database(db_conn.cursor())
|
|
||||||
except IncorrectDatabaseSetup as e:
|
|
||||||
quit_with_error(str(e))
|
|
||||||
|
|
||||||
|
|
||||||
# Gauges to expose monthly active user control metrics
|
# Gauges to expose monthly active user control metrics
|
||||||
current_mau_gauge = Gauge("synapse_admin_mau:current", "Current MAU")
|
current_mau_gauge = Gauge("synapse_admin_mau:current", "Current MAU")
|
||||||
|
|
@ -357,16 +341,12 @@ def setup(config_options):
|
||||||
|
|
||||||
synapse.config.logger.setup_logging(hs, config, use_worker_options=False)
|
synapse.config.logger.setup_logging(hs, config, use_worker_options=False)
|
||||||
|
|
||||||
logger.info("Preparing database: %s...", config.database_config["name"])
|
logger.info("Setting up server")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with hs.get_db_conn(run_new_connection=False) as db_conn:
|
hs.setup()
|
||||||
prepare_database(db_conn, database_engine, config=config)
|
except IncorrectDatabaseSetup as e:
|
||||||
database_engine.on_new_connection(db_conn)
|
quit_with_error(str(e))
|
||||||
|
|
||||||
hs.run_startup_checks(db_conn, database_engine)
|
|
||||||
|
|
||||||
db_conn.commit()
|
|
||||||
except UpgradeDatabaseException:
|
except UpgradeDatabaseException:
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
"\nFailed to upgrade database.\n"
|
"\nFailed to upgrade database.\n"
|
||||||
|
|
@ -375,9 +355,6 @@ def setup(config_options):
|
||||||
)
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
logger.info("Database prepared in %s.", config.database_config["name"])
|
|
||||||
|
|
||||||
hs.setup()
|
|
||||||
hs.setup_master()
|
hs.setup_master()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
|
|
@ -436,7 +413,7 @@ def setup(config_options):
|
||||||
_base.start(hs, config.listeners)
|
_base.start(hs, config.listeners)
|
||||||
|
|
||||||
hs.get_pusherpool().start()
|
hs.get_pusherpool().start()
|
||||||
hs.get_datastore().start_doing_background_updates()
|
hs.get_datastore().db.updates.start_doing_background_updates()
|
||||||
except Exception:
|
except Exception:
|
||||||
# Print the exception and bail out.
|
# Print the exception and bail out.
|
||||||
print("Error during startup:", file=sys.stderr)
|
print("Error during startup:", file=sys.stderr)
|
||||||
|
|
@ -542,8 +519,8 @@ def phone_stats_home(hs, stats, stats_process=_stats_process):
|
||||||
# Database version
|
# Database version
|
||||||
#
|
#
|
||||||
|
|
||||||
stats["database_engine"] = hs.get_datastore().database_engine_name
|
stats["database_engine"] = hs.database_engine.module.__name__
|
||||||
stats["database_server_version"] = hs.get_datastore().get_server_version()
|
stats["database_server_version"] = hs.database_engine.server_version
|
||||||
logger.info("Reporting stats to %s: %s" % (hs.config.report_stats_endpoint, stats))
|
logger.info("Reporting stats to %s: %s" % (hs.config.report_stats_endpoint, stats))
|
||||||
try:
|
try:
|
||||||
yield hs.get_proxied_http_client().put_json(
|
yield hs.get_proxied_http_client().put_json(
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,7 @@ from synapse.replication.slave.storage.account_data import SlavedAccountDataStor
|
||||||
from synapse.replication.slave.storage.events import SlavedEventStore
|
from synapse.replication.slave.storage.events import SlavedEventStore
|
||||||
from synapse.replication.slave.storage.pushers import SlavedPusherStore
|
from synapse.replication.slave.storage.pushers import SlavedPusherStore
|
||||||
from synapse.replication.slave.storage.receipts import SlavedReceiptsStore
|
from synapse.replication.slave.storage.receipts import SlavedReceiptsStore
|
||||||
|
from synapse.replication.slave.storage.room import RoomStore
|
||||||
from synapse.replication.tcp.client import ReplicationClientHandler
|
from synapse.replication.tcp.client import ReplicationClientHandler
|
||||||
from synapse.server import HomeServer
|
from synapse.server import HomeServer
|
||||||
from synapse.storage import DataStore
|
from synapse.storage import DataStore
|
||||||
|
|
@ -45,7 +46,11 @@ logger = logging.getLogger("synapse.app.pusher")
|
||||||
|
|
||||||
|
|
||||||
class PusherSlaveStore(
|
class PusherSlaveStore(
|
||||||
SlavedEventStore, SlavedPusherStore, SlavedReceiptsStore, SlavedAccountDataStore
|
SlavedEventStore,
|
||||||
|
SlavedPusherStore,
|
||||||
|
SlavedReceiptsStore,
|
||||||
|
SlavedAccountDataStore,
|
||||||
|
RoomStore,
|
||||||
):
|
):
|
||||||
update_pusher_last_stream_ordering_and_success = __func__(
|
update_pusher_last_stream_ordering_and_success = __func__(
|
||||||
DataStore.update_pusher_last_stream_ordering_and_success
|
DataStore.update_pusher_last_stream_ordering_and_success
|
||||||
|
|
|
||||||
|
|
@ -151,7 +151,7 @@ class SynchrotronPresence(object):
|
||||||
|
|
||||||
def set_state(self, user, state, ignore_status_msg=False):
|
def set_state(self, user, state, ignore_status_msg=False):
|
||||||
# TODO Hows this supposed to work?
|
# TODO Hows this supposed to work?
|
||||||
pass
|
return defer.succeed(None)
|
||||||
|
|
||||||
get_states = __func__(PresenceHandler.get_states)
|
get_states = __func__(PresenceHandler.get_states)
|
||||||
get_state = __func__(PresenceHandler.get_state)
|
get_state = __func__(PresenceHandler.get_state)
|
||||||
|
|
|
||||||
|
|
@ -43,6 +43,7 @@ from synapse.replication.tcp.streams.events import (
|
||||||
from synapse.rest.client.v2_alpha import user_directory
|
from synapse.rest.client.v2_alpha import user_directory
|
||||||
from synapse.server import HomeServer
|
from synapse.server import HomeServer
|
||||||
from synapse.storage.data_stores.main.user_directory import UserDirectoryStore
|
from synapse.storage.data_stores.main.user_directory import UserDirectoryStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.storage.engines import create_engine
|
from synapse.storage.engines import create_engine
|
||||||
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
||||||
from synapse.util.httpresourcetree import create_resource_tree
|
from synapse.util.httpresourcetree import create_resource_tree
|
||||||
|
|
@ -60,11 +61,11 @@ class UserDirectorySlaveStore(
|
||||||
UserDirectoryStore,
|
UserDirectoryStore,
|
||||||
BaseSlavedStore,
|
BaseSlavedStore,
|
||||||
):
|
):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(UserDirectorySlaveStore, self).__init__(db_conn, hs)
|
super(UserDirectorySlaveStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
events_max = self._stream_id_gen.get_current_token()
|
events_max = self._stream_id_gen.get_current_token()
|
||||||
curr_state_delta_prefill, min_curr_state_delta_id = self._get_cache_dict(
|
curr_state_delta_prefill, min_curr_state_delta_id = self.db.get_cache_dict(
|
||||||
db_conn,
|
db_conn,
|
||||||
"current_state_delta_stream",
|
"current_state_delta_stream",
|
||||||
entity_column="room_id",
|
entity_column="room_id",
|
||||||
|
|
|
||||||
|
|
@ -185,7 +185,7 @@ class ApplicationServiceApi(SimpleHttpClient):
|
||||||
|
|
||||||
if not _is_valid_3pe_metadata(info):
|
if not _is_valid_3pe_metadata(info):
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"query_3pe_protocol to %s did not return a" " valid result", uri
|
"query_3pe_protocol to %s did not return a valid result", uri
|
||||||
)
|
)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -134,7 +134,7 @@ def _load_appservice(hostname, as_info, config_filename):
|
||||||
for regex_obj in as_info["namespaces"][ns]:
|
for regex_obj in as_info["namespaces"][ns]:
|
||||||
if not isinstance(regex_obj, dict):
|
if not isinstance(regex_obj, dict):
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
"Expected namespace entry in %s to be an object," " but got %s",
|
"Expected namespace entry in %s to be an object, but got %s",
|
||||||
ns,
|
ns,
|
||||||
regex_obj,
|
regex_obj,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -146,6 +146,8 @@ class EmailConfig(Config):
|
||||||
if k not in email_config:
|
if k not in email_config:
|
||||||
missing.append("email." + k)
|
missing.append("email." + k)
|
||||||
|
|
||||||
|
# public_baseurl is required to build password reset and validation links that
|
||||||
|
# will be emailed to users
|
||||||
if config.get("public_baseurl") is None:
|
if config.get("public_baseurl") is None:
|
||||||
missing.append("public_baseurl")
|
missing.append("public_baseurl")
|
||||||
|
|
||||||
|
|
@ -305,8 +307,23 @@ class EmailConfig(Config):
|
||||||
# smtp_user: "exampleusername"
|
# smtp_user: "exampleusername"
|
||||||
# smtp_pass: "examplepassword"
|
# smtp_pass: "examplepassword"
|
||||||
# require_transport_security: false
|
# require_transport_security: false
|
||||||
|
#
|
||||||
|
# # notif_from defines the "From" address to use when sending emails.
|
||||||
|
# # It must be set if email sending is enabled.
|
||||||
|
# #
|
||||||
|
# # The placeholder '%(app)s' will be replaced by the application name,
|
||||||
|
# # which is normally 'app_name' (below), but may be overridden by the
|
||||||
|
# # Matrix client application.
|
||||||
|
# #
|
||||||
|
# # Note that the placeholder must be written '%(app)s', including the
|
||||||
|
# # trailing 's'.
|
||||||
|
# #
|
||||||
# notif_from: "Your Friendly %(app)s homeserver <noreply@example.com>"
|
# notif_from: "Your Friendly %(app)s homeserver <noreply@example.com>"
|
||||||
# app_name: Matrix
|
#
|
||||||
|
# # app_name defines the default value for '%(app)s' in notif_from. It
|
||||||
|
# # defaults to 'Matrix'.
|
||||||
|
# #
|
||||||
|
# #app_name: my_branded_matrix_server
|
||||||
#
|
#
|
||||||
# # Enable email notifications by default
|
# # Enable email notifications by default
|
||||||
# #
|
# #
|
||||||
|
|
|
||||||
|
|
@ -106,6 +106,13 @@ class RegistrationConfig(Config):
|
||||||
account_threepid_delegates = config.get("account_threepid_delegates") or {}
|
account_threepid_delegates = config.get("account_threepid_delegates") or {}
|
||||||
self.account_threepid_delegate_email = account_threepid_delegates.get("email")
|
self.account_threepid_delegate_email = account_threepid_delegates.get("email")
|
||||||
self.account_threepid_delegate_msisdn = account_threepid_delegates.get("msisdn")
|
self.account_threepid_delegate_msisdn = account_threepid_delegates.get("msisdn")
|
||||||
|
if self.account_threepid_delegate_msisdn and not self.public_baseurl:
|
||||||
|
raise ConfigError(
|
||||||
|
"The configuration option `public_baseurl` is required if "
|
||||||
|
"`account_threepid_delegate.msisdn` is set, such that "
|
||||||
|
"clients know where to submit validation tokens to. Please "
|
||||||
|
"configure `public_baseurl`."
|
||||||
|
)
|
||||||
|
|
||||||
self.default_identity_server = config.get("default_identity_server")
|
self.default_identity_server = config.get("default_identity_server")
|
||||||
self.allow_guest_access = config.get("allow_guest_access", False)
|
self.allow_guest_access = config.get("allow_guest_access", False)
|
||||||
|
|
|
||||||
|
|
@ -170,7 +170,7 @@ class _RoomDirectoryRule(object):
|
||||||
self.action = action
|
self.action = action
|
||||||
else:
|
else:
|
||||||
raise ConfigError(
|
raise ConfigError(
|
||||||
"%s rules can only have action of 'allow'" " or 'deny'" % (option_name,)
|
"%s rules can only have action of 'allow' or 'deny'" % (option_name,)
|
||||||
)
|
)
|
||||||
|
|
||||||
self._alias_matches_all = alias == "*"
|
self._alias_matches_all = alias == "*"
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,7 @@ import logging
|
||||||
import os.path
|
import os.path
|
||||||
import re
|
import re
|
||||||
from textwrap import indent
|
from textwrap import indent
|
||||||
from typing import List
|
from typing import Dict, List, Optional
|
||||||
|
|
||||||
import attr
|
import attr
|
||||||
import yaml
|
import yaml
|
||||||
|
|
@ -118,15 +118,16 @@ class ServerConfig(Config):
|
||||||
self.allow_public_rooms_without_auth = False
|
self.allow_public_rooms_without_auth = False
|
||||||
self.allow_public_rooms_over_federation = False
|
self.allow_public_rooms_over_federation = False
|
||||||
else:
|
else:
|
||||||
# If set to 'False', requires authentication to access the server's public
|
# If set to 'true', removes the need for authentication to access the server's
|
||||||
# rooms directory through the client API. Defaults to 'True'.
|
# public rooms directory through the client API, meaning that anyone can
|
||||||
|
# query the room directory. Defaults to 'false'.
|
||||||
self.allow_public_rooms_without_auth = config.get(
|
self.allow_public_rooms_without_auth = config.get(
|
||||||
"allow_public_rooms_without_auth", True
|
"allow_public_rooms_without_auth", False
|
||||||
)
|
)
|
||||||
# If set to 'False', forbids any other homeserver to fetch the server's public
|
# If set to 'true', allows any other homeserver to fetch the server's public
|
||||||
# rooms directory via federation. Defaults to 'True'.
|
# rooms directory via federation. Defaults to 'false'.
|
||||||
self.allow_public_rooms_over_federation = config.get(
|
self.allow_public_rooms_over_federation = config.get(
|
||||||
"allow_public_rooms_over_federation", True
|
"allow_public_rooms_over_federation", False
|
||||||
)
|
)
|
||||||
|
|
||||||
default_room_version = config.get("default_room_version", DEFAULT_ROOM_VERSION)
|
default_room_version = config.get("default_room_version", DEFAULT_ROOM_VERSION)
|
||||||
|
|
@ -223,7 +224,7 @@ class ServerConfig(Config):
|
||||||
self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])
|
self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
raise ConfigError(
|
raise ConfigError(
|
||||||
"Invalid range(s) provided in " "federation_ip_range_blacklist: %s" % e
|
"Invalid range(s) provided in federation_ip_range_blacklist: %s" % e
|
||||||
)
|
)
|
||||||
|
|
||||||
if self.public_baseurl is not None:
|
if self.public_baseurl is not None:
|
||||||
|
|
@ -246,6 +247,124 @@ class ServerConfig(Config):
|
||||||
# events with profile information that differ from the target's global profile.
|
# events with profile information that differ from the target's global profile.
|
||||||
self.allow_per_room_profiles = config.get("allow_per_room_profiles", True)
|
self.allow_per_room_profiles = config.get("allow_per_room_profiles", True)
|
||||||
|
|
||||||
|
retention_config = config.get("retention")
|
||||||
|
if retention_config is None:
|
||||||
|
retention_config = {}
|
||||||
|
|
||||||
|
self.retention_enabled = retention_config.get("enabled", False)
|
||||||
|
|
||||||
|
retention_default_policy = retention_config.get("default_policy")
|
||||||
|
|
||||||
|
if retention_default_policy is not None:
|
||||||
|
self.retention_default_min_lifetime = retention_default_policy.get(
|
||||||
|
"min_lifetime"
|
||||||
|
)
|
||||||
|
if self.retention_default_min_lifetime is not None:
|
||||||
|
self.retention_default_min_lifetime = self.parse_duration(
|
||||||
|
self.retention_default_min_lifetime
|
||||||
|
)
|
||||||
|
|
||||||
|
self.retention_default_max_lifetime = retention_default_policy.get(
|
||||||
|
"max_lifetime"
|
||||||
|
)
|
||||||
|
if self.retention_default_max_lifetime is not None:
|
||||||
|
self.retention_default_max_lifetime = self.parse_duration(
|
||||||
|
self.retention_default_max_lifetime
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
self.retention_default_min_lifetime is not None
|
||||||
|
and self.retention_default_max_lifetime is not None
|
||||||
|
and (
|
||||||
|
self.retention_default_min_lifetime
|
||||||
|
> self.retention_default_max_lifetime
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise ConfigError(
|
||||||
|
"The default retention policy's 'min_lifetime' can not be greater"
|
||||||
|
" than its 'max_lifetime'"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
self.retention_default_min_lifetime = None
|
||||||
|
self.retention_default_max_lifetime = None
|
||||||
|
|
||||||
|
self.retention_allowed_lifetime_min = retention_config.get(
|
||||||
|
"allowed_lifetime_min"
|
||||||
|
)
|
||||||
|
if self.retention_allowed_lifetime_min is not None:
|
||||||
|
self.retention_allowed_lifetime_min = self.parse_duration(
|
||||||
|
self.retention_allowed_lifetime_min
|
||||||
|
)
|
||||||
|
|
||||||
|
self.retention_allowed_lifetime_max = retention_config.get(
|
||||||
|
"allowed_lifetime_max"
|
||||||
|
)
|
||||||
|
if self.retention_allowed_lifetime_max is not None:
|
||||||
|
self.retention_allowed_lifetime_max = self.parse_duration(
|
||||||
|
self.retention_allowed_lifetime_max
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
self.retention_allowed_lifetime_min is not None
|
||||||
|
and self.retention_allowed_lifetime_max is not None
|
||||||
|
and self.retention_allowed_lifetime_min
|
||||||
|
> self.retention_allowed_lifetime_max
|
||||||
|
):
|
||||||
|
raise ConfigError(
|
||||||
|
"Invalid retention policy limits: 'allowed_lifetime_min' can not be"
|
||||||
|
" greater than 'allowed_lifetime_max'"
|
||||||
|
)
|
||||||
|
|
||||||
|
self.retention_purge_jobs = [] # type: List[Dict[str, Optional[int]]]
|
||||||
|
for purge_job_config in retention_config.get("purge_jobs", []):
|
||||||
|
interval_config = purge_job_config.get("interval")
|
||||||
|
|
||||||
|
if interval_config is None:
|
||||||
|
raise ConfigError(
|
||||||
|
"A retention policy's purge jobs configuration must have the"
|
||||||
|
" 'interval' key set."
|
||||||
|
)
|
||||||
|
|
||||||
|
interval = self.parse_duration(interval_config)
|
||||||
|
|
||||||
|
shortest_max_lifetime = purge_job_config.get("shortest_max_lifetime")
|
||||||
|
|
||||||
|
if shortest_max_lifetime is not None:
|
||||||
|
shortest_max_lifetime = self.parse_duration(shortest_max_lifetime)
|
||||||
|
|
||||||
|
longest_max_lifetime = purge_job_config.get("longest_max_lifetime")
|
||||||
|
|
||||||
|
if longest_max_lifetime is not None:
|
||||||
|
longest_max_lifetime = self.parse_duration(longest_max_lifetime)
|
||||||
|
|
||||||
|
if (
|
||||||
|
shortest_max_lifetime is not None
|
||||||
|
and longest_max_lifetime is not None
|
||||||
|
and shortest_max_lifetime > longest_max_lifetime
|
||||||
|
):
|
||||||
|
raise ConfigError(
|
||||||
|
"A retention policy's purge jobs configuration's"
|
||||||
|
" 'shortest_max_lifetime' value can not be greater than its"
|
||||||
|
" 'longest_max_lifetime' value."
|
||||||
|
)
|
||||||
|
|
||||||
|
self.retention_purge_jobs.append(
|
||||||
|
{
|
||||||
|
"interval": interval,
|
||||||
|
"shortest_max_lifetime": shortest_max_lifetime,
|
||||||
|
"longest_max_lifetime": longest_max_lifetime,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
if not self.retention_purge_jobs:
|
||||||
|
self.retention_purge_jobs = [
|
||||||
|
{
|
||||||
|
"interval": self.parse_duration("1d"),
|
||||||
|
"shortest_max_lifetime": None,
|
||||||
|
"longest_max_lifetime": None,
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
self.listeners = [] # type: List[dict]
|
self.listeners = [] # type: List[dict]
|
||||||
for listener in config.get("listeners", []):
|
for listener in config.get("listeners", []):
|
||||||
if not isinstance(listener.get("port", None), int):
|
if not isinstance(listener.get("port", None), int):
|
||||||
|
|
@ -372,6 +491,8 @@ class ServerConfig(Config):
|
||||||
"cleanup_extremities_with_dummy_events", True
|
"cleanup_extremities_with_dummy_events", True
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self.enable_ephemeral_messages = config.get("enable_ephemeral_messages", False)
|
||||||
|
|
||||||
def has_tls_listener(self) -> bool:
|
def has_tls_listener(self) -> bool:
|
||||||
return any(l["tls"] for l in self.listeners)
|
return any(l["tls"] for l in self.listeners)
|
||||||
|
|
||||||
|
|
@ -500,15 +621,16 @@ class ServerConfig(Config):
|
||||||
#
|
#
|
||||||
#require_auth_for_profile_requests: true
|
#require_auth_for_profile_requests: true
|
||||||
|
|
||||||
# If set to 'false', requires authentication to access the server's public rooms
|
# If set to 'true', removes the need for authentication to access the server's
|
||||||
# directory through the client API. Defaults to 'true'.
|
# public rooms directory through the client API, meaning that anyone can
|
||||||
|
# query the room directory. Defaults to 'false'.
|
||||||
#
|
#
|
||||||
#allow_public_rooms_without_auth: false
|
#allow_public_rooms_without_auth: true
|
||||||
|
|
||||||
# If set to 'false', forbids any other homeserver to fetch the server's public
|
# If set to 'true', allows any other homeserver to fetch the server's public
|
||||||
# rooms directory via federation. Defaults to 'true'.
|
# rooms directory via federation. Defaults to 'false'.
|
||||||
#
|
#
|
||||||
#allow_public_rooms_over_federation: false
|
#allow_public_rooms_over_federation: true
|
||||||
|
|
||||||
# The default room version for newly created rooms.
|
# The default room version for newly created rooms.
|
||||||
#
|
#
|
||||||
|
|
@ -761,6 +883,69 @@ class ServerConfig(Config):
|
||||||
# Defaults to `28d`. Set to `null` to disable clearing out of old rows.
|
# Defaults to `28d`. Set to `null` to disable clearing out of old rows.
|
||||||
#
|
#
|
||||||
#user_ips_max_age: 14d
|
#user_ips_max_age: 14d
|
||||||
|
|
||||||
|
# Message retention policy at the server level.
|
||||||
|
#
|
||||||
|
# Room admins and mods can define a retention period for their rooms using the
|
||||||
|
# 'm.room.retention' state event, and server admins can cap this period by setting
|
||||||
|
# the 'allowed_lifetime_min' and 'allowed_lifetime_max' config options.
|
||||||
|
#
|
||||||
|
# If this feature is enabled, Synapse will regularly look for and purge events
|
||||||
|
# which are older than the room's maximum retention period. Synapse will also
|
||||||
|
# filter events received over federation so that events that should have been
|
||||||
|
# purged are ignored and not stored again.
|
||||||
|
#
|
||||||
|
retention:
|
||||||
|
# The message retention policies feature is disabled by default. Uncomment the
|
||||||
|
# following line to enable it.
|
||||||
|
#
|
||||||
|
#enabled: true
|
||||||
|
|
||||||
|
# Default retention policy. If set, Synapse will apply it to rooms that lack the
|
||||||
|
# 'm.room.retention' state event. Currently, the value of 'min_lifetime' doesn't
|
||||||
|
# matter much because Synapse doesn't take it into account yet.
|
||||||
|
#
|
||||||
|
#default_policy:
|
||||||
|
# min_lifetime: 1d
|
||||||
|
# max_lifetime: 1y
|
||||||
|
|
||||||
|
# Retention policy limits. If set, a user won't be able to send a
|
||||||
|
# 'm.room.retention' event which features a 'min_lifetime' or a 'max_lifetime'
|
||||||
|
# that's not within this range. This is especially useful in closed federations,
|
||||||
|
# in which server admins can make sure every federating server applies the same
|
||||||
|
# rules.
|
||||||
|
#
|
||||||
|
#allowed_lifetime_min: 1d
|
||||||
|
#allowed_lifetime_max: 1y
|
||||||
|
|
||||||
|
# Server admins can define the settings of the background jobs purging the
|
||||||
|
# events which lifetime has expired under the 'purge_jobs' section.
|
||||||
|
#
|
||||||
|
# If no configuration is provided, a single job will be set up to delete expired
|
||||||
|
# events in every room daily.
|
||||||
|
#
|
||||||
|
# Each job's configuration defines which range of message lifetimes the job
|
||||||
|
# takes care of. For example, if 'shortest_max_lifetime' is '2d' and
|
||||||
|
# 'longest_max_lifetime' is '3d', the job will handle purging expired events in
|
||||||
|
# rooms whose state defines a 'max_lifetime' that's both higher than 2 days, and
|
||||||
|
# lower than or equal to 3 days. Both the minimum and the maximum value of a
|
||||||
|
# range are optional, e.g. a job with no 'shortest_max_lifetime' and a
|
||||||
|
# 'longest_max_lifetime' of '3d' will handle every room with a retention policy
|
||||||
|
# which 'max_lifetime' is lower than or equal to three days.
|
||||||
|
#
|
||||||
|
# The rationale for this per-job configuration is that some rooms might have a
|
||||||
|
# retention policy with a low 'max_lifetime', where history needs to be purged
|
||||||
|
# of outdated messages on a very frequent basis (e.g. every 5min), but not want
|
||||||
|
# that purge to be performed by a job that's iterating over every room it knows,
|
||||||
|
# which would be quite heavy on the server.
|
||||||
|
#
|
||||||
|
#purge_jobs:
|
||||||
|
# - shortest_max_lifetime: 1d
|
||||||
|
# longest_max_lifetime: 3d
|
||||||
|
# interval: 5m:
|
||||||
|
# - shortest_max_lifetime: 3d
|
||||||
|
# longest_max_lifetime: 1y
|
||||||
|
# interval: 24h
|
||||||
"""
|
"""
|
||||||
% locals()
|
% locals()
|
||||||
)
|
)
|
||||||
|
|
@ -787,14 +972,14 @@ class ServerConfig(Config):
|
||||||
"--print-pidfile",
|
"--print-pidfile",
|
||||||
action="store_true",
|
action="store_true",
|
||||||
default=None,
|
default=None,
|
||||||
help="Print the path to the pidfile just" " before daemonizing",
|
help="Print the path to the pidfile just before daemonizing",
|
||||||
)
|
)
|
||||||
server_group.add_argument(
|
server_group.add_argument(
|
||||||
"--manhole",
|
"--manhole",
|
||||||
metavar="PORT",
|
metavar="PORT",
|
||||||
dest="manhole",
|
dest="manhole",
|
||||||
type=int,
|
type=int,
|
||||||
help="Turn on the twisted telnet manhole" " service on the given port.",
|
help="Turn on the twisted telnet manhole service on the given port.",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@
|
||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from six import string_types
|
from six import integer_types, string_types
|
||||||
|
|
||||||
from synapse.api.constants import MAX_ALIAS_LENGTH, EventTypes, Membership
|
from synapse.api.constants import MAX_ALIAS_LENGTH, EventTypes, Membership
|
||||||
from synapse.api.errors import Codes, SynapseError
|
from synapse.api.errors import Codes, SynapseError
|
||||||
|
|
@ -22,11 +22,12 @@ from synapse.types import EventID, RoomID, UserID
|
||||||
|
|
||||||
|
|
||||||
class EventValidator(object):
|
class EventValidator(object):
|
||||||
def validate_new(self, event):
|
def validate_new(self, event, config):
|
||||||
"""Validates the event has roughly the right format
|
"""Validates the event has roughly the right format
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
event (FrozenEvent)
|
event (FrozenEvent): The event to validate.
|
||||||
|
config (Config): The homeserver's configuration.
|
||||||
"""
|
"""
|
||||||
self.validate_builder(event)
|
self.validate_builder(event)
|
||||||
|
|
||||||
|
|
@ -67,6 +68,99 @@ class EventValidator(object):
|
||||||
Codes.INVALID_PARAM,
|
Codes.INVALID_PARAM,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if event.type == EventTypes.Retention:
|
||||||
|
self._validate_retention(event, config)
|
||||||
|
|
||||||
|
def _validate_retention(self, event, config):
|
||||||
|
"""Checks that an event that defines the retention policy for a room respects the
|
||||||
|
boundaries imposed by the server's administrator.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
event (FrozenEvent): The event to validate.
|
||||||
|
config (Config): The homeserver's configuration.
|
||||||
|
"""
|
||||||
|
min_lifetime = event.content.get("min_lifetime")
|
||||||
|
max_lifetime = event.content.get("max_lifetime")
|
||||||
|
|
||||||
|
if min_lifetime is not None:
|
||||||
|
if not isinstance(min_lifetime, integer_types):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg="'min_lifetime' must be an integer",
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
config.retention_allowed_lifetime_min is not None
|
||||||
|
and min_lifetime < config.retention_allowed_lifetime_min
|
||||||
|
):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg=(
|
||||||
|
"'min_lifetime' can't be lower than the minimum allowed"
|
||||||
|
" value enforced by the server's administrator"
|
||||||
|
),
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
config.retention_allowed_lifetime_max is not None
|
||||||
|
and min_lifetime > config.retention_allowed_lifetime_max
|
||||||
|
):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg=(
|
||||||
|
"'min_lifetime' can't be greater than the maximum allowed"
|
||||||
|
" value enforced by the server's administrator"
|
||||||
|
),
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
|
if max_lifetime is not None:
|
||||||
|
if not isinstance(max_lifetime, integer_types):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg="'max_lifetime' must be an integer",
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
config.retention_allowed_lifetime_min is not None
|
||||||
|
and max_lifetime < config.retention_allowed_lifetime_min
|
||||||
|
):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg=(
|
||||||
|
"'max_lifetime' can't be lower than the minimum allowed value"
|
||||||
|
" enforced by the server's administrator"
|
||||||
|
),
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
config.retention_allowed_lifetime_max is not None
|
||||||
|
and max_lifetime > config.retention_allowed_lifetime_max
|
||||||
|
):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg=(
|
||||||
|
"'max_lifetime' can't be greater than the maximum allowed"
|
||||||
|
" value enforced by the server's administrator"
|
||||||
|
),
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
|
if (
|
||||||
|
min_lifetime is not None
|
||||||
|
and max_lifetime is not None
|
||||||
|
and min_lifetime > max_lifetime
|
||||||
|
):
|
||||||
|
raise SynapseError(
|
||||||
|
code=400,
|
||||||
|
msg="'min_lifetime' can't be greater than 'max_lifetime",
|
||||||
|
errcode=Codes.BAD_JSON,
|
||||||
|
)
|
||||||
|
|
||||||
def validate_builder(self, event):
|
def validate_builder(self, event):
|
||||||
"""Validates that the builder/event has roughly the right format. Only
|
"""Validates that the builder/event has roughly the right format. Only
|
||||||
checks values that we expect a proto event to have, rather than all the
|
checks values that we expect a proto event to have, rather than all the
|
||||||
|
|
|
||||||
|
|
@ -324,87 +324,32 @@ class FederationClient(FederationBase):
|
||||||
A list of events in the state, and a list of events in the auth chain
|
A list of events in the state, and a list of events in the auth chain
|
||||||
for the given event.
|
for the given event.
|
||||||
"""
|
"""
|
||||||
try:
|
result = yield self.transport_layer.get_room_state_ids(
|
||||||
# First we try and ask for just the IDs, as thats far quicker if
|
|
||||||
# we have most of the state and auth_chain already.
|
|
||||||
# However, this may 404 if the other side has an old synapse.
|
|
||||||
result = yield self.transport_layer.get_room_state_ids(
|
|
||||||
destination, room_id, event_id=event_id
|
|
||||||
)
|
|
||||||
|
|
||||||
state_event_ids = result["pdu_ids"]
|
|
||||||
auth_event_ids = result.get("auth_chain_ids", [])
|
|
||||||
|
|
||||||
fetched_events, failed_to_fetch = yield self.get_events_from_store_or_dest(
|
|
||||||
destination, room_id, set(state_event_ids + auth_event_ids)
|
|
||||||
)
|
|
||||||
|
|
||||||
if failed_to_fetch:
|
|
||||||
logger.warning(
|
|
||||||
"Failed to fetch missing state/auth events for %s: %s",
|
|
||||||
room_id,
|
|
||||||
failed_to_fetch,
|
|
||||||
)
|
|
||||||
|
|
||||||
event_map = {ev.event_id: ev for ev in fetched_events}
|
|
||||||
|
|
||||||
pdus = [event_map[e_id] for e_id in state_event_ids if e_id in event_map]
|
|
||||||
auth_chain = [
|
|
||||||
event_map[e_id] for e_id in auth_event_ids if e_id in event_map
|
|
||||||
]
|
|
||||||
|
|
||||||
auth_chain.sort(key=lambda e: e.depth)
|
|
||||||
|
|
||||||
return pdus, auth_chain
|
|
||||||
except HttpResponseException as e:
|
|
||||||
if e.code == 400 or e.code == 404:
|
|
||||||
logger.info("Failed to use get_room_state_ids API, falling back")
|
|
||||||
else:
|
|
||||||
raise e
|
|
||||||
|
|
||||||
result = yield self.transport_layer.get_room_state(
|
|
||||||
destination, room_id, event_id=event_id
|
destination, room_id, event_id=event_id
|
||||||
)
|
)
|
||||||
|
|
||||||
room_version = yield self.store.get_room_version(room_id)
|
state_event_ids = result["pdu_ids"]
|
||||||
format_ver = room_version_to_event_format(room_version)
|
auth_event_ids = result.get("auth_chain_ids", [])
|
||||||
|
|
||||||
pdus = [
|
fetched_events, failed_to_fetch = yield self.get_events_from_store_or_dest(
|
||||||
event_from_pdu_json(p, format_ver, outlier=True) for p in result["pdus"]
|
destination, room_id, set(state_event_ids + auth_event_ids)
|
||||||
]
|
|
||||||
|
|
||||||
auth_chain = [
|
|
||||||
event_from_pdu_json(p, format_ver, outlier=True)
|
|
||||||
for p in result.get("auth_chain", [])
|
|
||||||
]
|
|
||||||
|
|
||||||
seen_events = yield self.store.get_events(
|
|
||||||
[ev.event_id for ev in itertools.chain(pdus, auth_chain)]
|
|
||||||
)
|
)
|
||||||
|
|
||||||
signed_pdus = yield self._check_sigs_and_hash_and_fetch(
|
if failed_to_fetch:
|
||||||
destination,
|
logger.warning(
|
||||||
[p for p in pdus if p.event_id not in seen_events],
|
"Failed to fetch missing state/auth events for %s: %s",
|
||||||
outlier=True,
|
room_id,
|
||||||
room_version=room_version,
|
failed_to_fetch,
|
||||||
)
|
)
|
||||||
signed_pdus.extend(
|
|
||||||
seen_events[p.event_id] for p in pdus if p.event_id in seen_events
|
|
||||||
)
|
|
||||||
|
|
||||||
signed_auth = yield self._check_sigs_and_hash_and_fetch(
|
event_map = {ev.event_id: ev for ev in fetched_events}
|
||||||
destination,
|
|
||||||
[p for p in auth_chain if p.event_id not in seen_events],
|
|
||||||
outlier=True,
|
|
||||||
room_version=room_version,
|
|
||||||
)
|
|
||||||
signed_auth.extend(
|
|
||||||
seen_events[p.event_id] for p in auth_chain if p.event_id in seen_events
|
|
||||||
)
|
|
||||||
|
|
||||||
signed_auth.sort(key=lambda e: e.depth)
|
pdus = [event_map[e_id] for e_id in state_event_ids if e_id in event_map]
|
||||||
|
auth_chain = [event_map[e_id] for e_id in auth_event_ids if e_id in event_map]
|
||||||
|
|
||||||
return signed_pdus, signed_auth
|
auth_chain.sort(key=lambda e: e.depth)
|
||||||
|
|
||||||
|
return pdus, auth_chain
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def get_events_from_store_or_dest(self, destination, room_id, event_ids):
|
def get_events_from_store_or_dest(self, destination, room_id, event_ids):
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
# Copyright 2015, 2016 OpenMarket Ltd
|
# Copyright 2015, 2016 OpenMarket Ltd
|
||||||
# Copyright 2018 New Vector Ltd
|
# Copyright 2018 New Vector Ltd
|
||||||
|
# Copyright 2019 Matrix.org Federation C.I.C
|
||||||
#
|
#
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
# you may not use this file except in compliance with the License.
|
# you may not use this file except in compliance with the License.
|
||||||
|
|
@ -73,6 +74,7 @@ class FederationServer(FederationBase):
|
||||||
|
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.handler = hs.get_handlers().federation_handler
|
self.handler = hs.get_handlers().federation_handler
|
||||||
|
self.state = hs.get_state_handler()
|
||||||
|
|
||||||
self._server_linearizer = Linearizer("fed_server")
|
self._server_linearizer = Linearizer("fed_server")
|
||||||
self._transaction_linearizer = Linearizer("fed_txn_handler")
|
self._transaction_linearizer = Linearizer("fed_txn_handler")
|
||||||
|
|
@ -264,9 +266,6 @@ class FederationServer(FederationBase):
|
||||||
await self.registry.on_edu(edu_type, origin, content)
|
await self.registry.on_edu(edu_type, origin, content)
|
||||||
|
|
||||||
async def on_context_state_request(self, origin, room_id, event_id):
|
async def on_context_state_request(self, origin, room_id, event_id):
|
||||||
if not event_id:
|
|
||||||
raise NotImplementedError("Specify an event")
|
|
||||||
|
|
||||||
origin_host, _ = parse_server_name(origin)
|
origin_host, _ = parse_server_name(origin)
|
||||||
await self.check_server_matches_acl(origin_host, room_id)
|
await self.check_server_matches_acl(origin_host, room_id)
|
||||||
|
|
||||||
|
|
@ -280,13 +279,18 @@ class FederationServer(FederationBase):
|
||||||
# - but that's non-trivial to get right, and anyway somewhat defeats
|
# - but that's non-trivial to get right, and anyway somewhat defeats
|
||||||
# the point of the linearizer.
|
# the point of the linearizer.
|
||||||
with (await self._server_linearizer.queue((origin, room_id))):
|
with (await self._server_linearizer.queue((origin, room_id))):
|
||||||
resp = await self._state_resp_cache.wrap(
|
resp = dict(
|
||||||
(room_id, event_id),
|
await self._state_resp_cache.wrap(
|
||||||
self._on_context_state_request_compute,
|
(room_id, event_id),
|
||||||
room_id,
|
self._on_context_state_request_compute,
|
||||||
event_id,
|
room_id,
|
||||||
|
event_id,
|
||||||
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
room_version = await self.store.get_room_version(room_id)
|
||||||
|
resp["room_version"] = room_version
|
||||||
|
|
||||||
return 200, resp
|
return 200, resp
|
||||||
|
|
||||||
async def on_state_ids_request(self, origin, room_id, event_id):
|
async def on_state_ids_request(self, origin, room_id, event_id):
|
||||||
|
|
@ -306,7 +310,11 @@ class FederationServer(FederationBase):
|
||||||
return 200, {"pdu_ids": state_ids, "auth_chain_ids": auth_chain_ids}
|
return 200, {"pdu_ids": state_ids, "auth_chain_ids": auth_chain_ids}
|
||||||
|
|
||||||
async def _on_context_state_request_compute(self, room_id, event_id):
|
async def _on_context_state_request_compute(self, room_id, event_id):
|
||||||
pdus = await self.handler.get_state_for_pdu(room_id, event_id)
|
if event_id:
|
||||||
|
pdus = await self.handler.get_state_for_pdu(room_id, event_id)
|
||||||
|
else:
|
||||||
|
pdus = (await self.state.get_current_state(room_id)).values()
|
||||||
|
|
||||||
auth_chain = await self.store.get_auth_chain([pdu.event_id for pdu in pdus])
|
auth_chain = await self.store.get_auth_chain([pdu.event_id for pdu in pdus])
|
||||||
|
|
||||||
return {
|
return {
|
||||||
|
|
|
||||||
|
|
@ -44,7 +44,7 @@ class TransactionActions(object):
|
||||||
response code and response body.
|
response code and response body.
|
||||||
"""
|
"""
|
||||||
if not transaction.transaction_id:
|
if not transaction.transaction_id:
|
||||||
raise RuntimeError("Cannot persist a transaction with no " "transaction_id")
|
raise RuntimeError("Cannot persist a transaction with no transaction_id")
|
||||||
|
|
||||||
return self.store.get_received_txn_response(transaction.transaction_id, origin)
|
return self.store.get_received_txn_response(transaction.transaction_id, origin)
|
||||||
|
|
||||||
|
|
@ -56,7 +56,7 @@ class TransactionActions(object):
|
||||||
Deferred
|
Deferred
|
||||||
"""
|
"""
|
||||||
if not transaction.transaction_id:
|
if not transaction.transaction_id:
|
||||||
raise RuntimeError("Cannot persist a transaction with no " "transaction_id")
|
raise RuntimeError("Cannot persist a transaction with no transaction_id")
|
||||||
|
|
||||||
return self.store.set_received_txn_response(
|
return self.store.set_received_txn_response(
|
||||||
transaction.transaction_id, origin, code, response
|
transaction.transaction_id, origin, code, response
|
||||||
|
|
|
||||||
|
|
@ -49,7 +49,7 @@ sent_pdus_destination_dist_count = Counter(
|
||||||
|
|
||||||
sent_pdus_destination_dist_total = Counter(
|
sent_pdus_destination_dist_total = Counter(
|
||||||
"synapse_federation_client_sent_pdu_destinations:total",
|
"synapse_federation_client_sent_pdu_destinations:total",
|
||||||
"" "Total number of PDUs queued for sending across all destinations",
|
"Total number of PDUs queued for sending across all destinations",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -84,7 +84,7 @@ class TransactionManager(object):
|
||||||
txn_id = str(self._next_txn_id)
|
txn_id = str(self._next_txn_id)
|
||||||
|
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"TX [%s] {%s} Attempting new transaction" " (pdus: %d, edus: %d)",
|
"TX [%s] {%s} Attempting new transaction (pdus: %d, edus: %d)",
|
||||||
destination,
|
destination,
|
||||||
txn_id,
|
txn_id,
|
||||||
len(pdus),
|
len(pdus),
|
||||||
|
|
@ -103,7 +103,7 @@ class TransactionManager(object):
|
||||||
self._next_txn_id += 1
|
self._next_txn_id += 1
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"TX [%s] {%s} Sending transaction [%s]," " (PDUs: %d, EDUs: %d)",
|
"TX [%s] {%s} Sending transaction [%s], (PDUs: %d, EDUs: %d)",
|
||||||
destination,
|
destination,
|
||||||
txn_id,
|
txn_id,
|
||||||
transaction.transaction_id,
|
transaction.transaction_id,
|
||||||
|
|
|
||||||
|
|
@ -38,30 +38,6 @@ class TransportLayerClient(object):
|
||||||
self.server_name = hs.hostname
|
self.server_name = hs.hostname
|
||||||
self.client = hs.get_http_client()
|
self.client = hs.get_http_client()
|
||||||
|
|
||||||
@log_function
|
|
||||||
def get_room_state(self, destination, room_id, event_id):
|
|
||||||
""" Requests all state for a given room from the given server at the
|
|
||||||
given event.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
destination (str): The host name of the remote homeserver we want
|
|
||||||
to get the state from.
|
|
||||||
context (str): The name of the context we want the state of
|
|
||||||
event_id (str): The event we want the context at.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Deferred: Results in a dict received from the remote homeserver.
|
|
||||||
"""
|
|
||||||
logger.debug("get_room_state dest=%s, room=%s", destination, room_id)
|
|
||||||
|
|
||||||
path = _create_v1_path("/state/%s", room_id)
|
|
||||||
return self.client.get_json(
|
|
||||||
destination,
|
|
||||||
path=path,
|
|
||||||
args={"event_id": event_id},
|
|
||||||
try_trailing_slash_on_400=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
@log_function
|
@log_function
|
||||||
def get_room_state_ids(self, destination, room_id, event_id):
|
def get_room_state_ids(self, destination, room_id, event_id):
|
||||||
""" Requests all state for a given room from the given server at the
|
""" Requests all state for a given room from the given server at the
|
||||||
|
|
|
||||||
|
|
@ -421,7 +421,7 @@ class FederationEventServlet(BaseFederationServlet):
|
||||||
return await self.handler.on_pdu_request(origin, event_id)
|
return await self.handler.on_pdu_request(origin, event_id)
|
||||||
|
|
||||||
|
|
||||||
class FederationStateServlet(BaseFederationServlet):
|
class FederationStateV1Servlet(BaseFederationServlet):
|
||||||
PATH = "/state/(?P<context>[^/]*)/?"
|
PATH = "/state/(?P<context>[^/]*)/?"
|
||||||
|
|
||||||
# This is when someone asks for all data for a given context.
|
# This is when someone asks for all data for a given context.
|
||||||
|
|
@ -429,7 +429,7 @@ class FederationStateServlet(BaseFederationServlet):
|
||||||
return await self.handler.on_context_state_request(
|
return await self.handler.on_context_state_request(
|
||||||
origin,
|
origin,
|
||||||
context,
|
context,
|
||||||
parse_string_from_args(query, "event_id", None, required=True),
|
parse_string_from_args(query, "event_id", None, required=False),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1360,7 +1360,7 @@ class RoomComplexityServlet(BaseFederationServlet):
|
||||||
FEDERATION_SERVLET_CLASSES = (
|
FEDERATION_SERVLET_CLASSES = (
|
||||||
FederationSendServlet,
|
FederationSendServlet,
|
||||||
FederationEventServlet,
|
FederationEventServlet,
|
||||||
FederationStateServlet,
|
FederationStateV1Servlet,
|
||||||
FederationStateIdsServlet,
|
FederationStateIdsServlet,
|
||||||
FederationBackfillServlet,
|
FederationBackfillServlet,
|
||||||
FederationQueryServlet,
|
FederationQueryServlet,
|
||||||
|
|
|
||||||
|
|
@ -56,7 +56,7 @@ class AdminHandler(BaseHandler):
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def get_users(self):
|
def get_users(self):
|
||||||
"""Function to reterive a list of users in users table.
|
"""Function to retrieve a list of users in users table.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
Returns:
|
Returns:
|
||||||
|
|
@ -67,19 +67,22 @@ class AdminHandler(BaseHandler):
|
||||||
return ret
|
return ret
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def get_users_paginate(self, order, start, limit):
|
def get_users_paginate(self, start, limit, name, guests, deactivated):
|
||||||
"""Function to reterive a paginated list of users from
|
"""Function to retrieve a paginated list of users from
|
||||||
users list. This will return a json object, which contains
|
users list. This will return a json list of users.
|
||||||
list of users and the total number of users in users table.
|
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
order (str): column name to order the select by this column
|
|
||||||
start (int): start number to begin the query from
|
start (int): start number to begin the query from
|
||||||
limit (int): number of rows to reterive
|
limit (int): number of rows to retrieve
|
||||||
|
name (string): filter for user names
|
||||||
|
guests (bool): whether to in include guest users
|
||||||
|
deactivated (bool): whether to include deactivated users
|
||||||
Returns:
|
Returns:
|
||||||
defer.Deferred: resolves to json object {list[dict[str, Any]], count}
|
defer.Deferred: resolves to json list[dict[str, Any]]
|
||||||
"""
|
"""
|
||||||
ret = yield self.store.get_users_paginate(order, start, limit)
|
ret = yield self.store.get_users_paginate(
|
||||||
|
start, limit, name, guests, deactivated
|
||||||
|
)
|
||||||
|
|
||||||
return ret
|
return ret
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -119,7 +119,7 @@ class DirectoryHandler(BaseHandler):
|
||||||
if not service.is_interested_in_alias(room_alias.to_string()):
|
if not service.is_interested_in_alias(room_alias.to_string()):
|
||||||
raise SynapseError(
|
raise SynapseError(
|
||||||
400,
|
400,
|
||||||
"This application service has not reserved" " this kind of alias.",
|
"This application service has not reserved this kind of alias.",
|
||||||
errcode=Codes.EXCLUSIVE,
|
errcode=Codes.EXCLUSIVE,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
|
|
|
||||||
|
|
@ -30,6 +30,7 @@ from twisted.internet import defer
|
||||||
from synapse.api.errors import CodeMessageException, Codes, NotFoundError, SynapseError
|
from synapse.api.errors import CodeMessageException, Codes, NotFoundError, SynapseError
|
||||||
from synapse.logging.context import make_deferred_yieldable, run_in_background
|
from synapse.logging.context import make_deferred_yieldable, run_in_background
|
||||||
from synapse.logging.opentracing import log_kv, set_tag, tag_args, trace
|
from synapse.logging.opentracing import log_kv, set_tag, tag_args, trace
|
||||||
|
from synapse.replication.http.devices import ReplicationUserDevicesResyncRestServlet
|
||||||
from synapse.types import (
|
from synapse.types import (
|
||||||
UserID,
|
UserID,
|
||||||
get_domain_from_id,
|
get_domain_from_id,
|
||||||
|
|
@ -53,6 +54,12 @@ class E2eKeysHandler(object):
|
||||||
|
|
||||||
self._edu_updater = SigningKeyEduUpdater(hs, self)
|
self._edu_updater = SigningKeyEduUpdater(hs, self)
|
||||||
|
|
||||||
|
self._is_master = hs.config.worker_app is None
|
||||||
|
if not self._is_master:
|
||||||
|
self._user_device_resync_client = ReplicationUserDevicesResyncRestServlet.make_client(
|
||||||
|
hs
|
||||||
|
)
|
||||||
|
|
||||||
federation_registry = hs.get_federation_registry()
|
federation_registry = hs.get_federation_registry()
|
||||||
|
|
||||||
# FIXME: switch to m.signing_key_update when MSC1756 is merged into the spec
|
# FIXME: switch to m.signing_key_update when MSC1756 is merged into the spec
|
||||||
|
|
@ -191,9 +198,15 @@ class E2eKeysHandler(object):
|
||||||
# probably be tracking their device lists. However, we haven't
|
# probably be tracking their device lists. However, we haven't
|
||||||
# done an initial sync on the device list so we do it now.
|
# done an initial sync on the device list so we do it now.
|
||||||
try:
|
try:
|
||||||
user_devices = yield self.device_handler.device_list_updater.user_device_resync(
|
if self._is_master:
|
||||||
user_id
|
user_devices = yield self.device_handler.device_list_updater.user_device_resync(
|
||||||
)
|
user_id
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
user_devices = yield self._user_device_resync_client(
|
||||||
|
user_id=user_id
|
||||||
|
)
|
||||||
|
|
||||||
user_devices = user_devices["devices"]
|
user_devices = user_devices["devices"]
|
||||||
for device in user_devices:
|
for device in user_devices:
|
||||||
results[user_id] = {device["device_id"]: device["keys"]}
|
results[user_id] = {device["device_id"]: device["keys"]}
|
||||||
|
|
@ -251,7 +264,6 @@ class E2eKeysHandler(object):
|
||||||
|
|
||||||
return ret
|
return ret
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
|
||||||
def get_cross_signing_keys_from_cache(self, query, from_user_id):
|
def get_cross_signing_keys_from_cache(self, query, from_user_id):
|
||||||
"""Get cross-signing keys for users from the database
|
"""Get cross-signing keys for users from the database
|
||||||
|
|
||||||
|
|
@ -271,35 +283,14 @@ class E2eKeysHandler(object):
|
||||||
self_signing_keys = {}
|
self_signing_keys = {}
|
||||||
user_signing_keys = {}
|
user_signing_keys = {}
|
||||||
|
|
||||||
for user_id in query:
|
# Currently a stub, implementation coming in https://github.com/matrix-org/synapse/pull/6486
|
||||||
# XXX: consider changing the store functions to allow querying
|
return defer.succeed(
|
||||||
# multiple users simultaneously.
|
{
|
||||||
key = yield self.store.get_e2e_cross_signing_key(
|
"master_keys": master_keys,
|
||||||
user_id, "master", from_user_id
|
"self_signing_keys": self_signing_keys,
|
||||||
)
|
"user_signing_keys": user_signing_keys,
|
||||||
if key:
|
}
|
||||||
master_keys[user_id] = key
|
)
|
||||||
|
|
||||||
key = yield self.store.get_e2e_cross_signing_key(
|
|
||||||
user_id, "self_signing", from_user_id
|
|
||||||
)
|
|
||||||
if key:
|
|
||||||
self_signing_keys[user_id] = key
|
|
||||||
|
|
||||||
# users can see other users' master and self-signing keys, but can
|
|
||||||
# only see their own user-signing keys
|
|
||||||
if from_user_id == user_id:
|
|
||||||
key = yield self.store.get_e2e_cross_signing_key(
|
|
||||||
user_id, "user_signing", from_user_id
|
|
||||||
)
|
|
||||||
if key:
|
|
||||||
user_signing_keys[user_id] = key
|
|
||||||
|
|
||||||
return {
|
|
||||||
"master_keys": master_keys,
|
|
||||||
"self_signing_keys": self_signing_keys,
|
|
||||||
"user_signing_keys": user_signing_keys,
|
|
||||||
}
|
|
||||||
|
|
||||||
@trace
|
@trace
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
# Copyright 2017, 2018 New Vector Ltd
|
# Copyright 2017, 2018 New Vector Ltd
|
||||||
|
# Copyright 2019 Matrix.org Foundation C.I.C.
|
||||||
#
|
#
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
# you may not use this file except in compliance with the License.
|
# you may not use this file except in compliance with the License.
|
||||||
|
|
@ -103,14 +104,35 @@ class E2eRoomKeysHandler(object):
|
||||||
rooms
|
rooms
|
||||||
session_id(string): session ID to delete keys for, for None to delete keys
|
session_id(string): session ID to delete keys for, for None to delete keys
|
||||||
for all sessions
|
for all sessions
|
||||||
|
Raises:
|
||||||
|
NotFoundError: if the backup version does not exist
|
||||||
Returns:
|
Returns:
|
||||||
A deferred of the deletion transaction
|
A dict containing the count and etag for the backup version
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# lock for consistency with uploading
|
# lock for consistency with uploading
|
||||||
with (yield self._upload_linearizer.queue(user_id)):
|
with (yield self._upload_linearizer.queue(user_id)):
|
||||||
|
# make sure the backup version exists
|
||||||
|
try:
|
||||||
|
version_info = yield self.store.get_e2e_room_keys_version_info(
|
||||||
|
user_id, version
|
||||||
|
)
|
||||||
|
except StoreError as e:
|
||||||
|
if e.code == 404:
|
||||||
|
raise NotFoundError("Unknown backup version")
|
||||||
|
else:
|
||||||
|
raise
|
||||||
|
|
||||||
yield self.store.delete_e2e_room_keys(user_id, version, room_id, session_id)
|
yield self.store.delete_e2e_room_keys(user_id, version, room_id, session_id)
|
||||||
|
|
||||||
|
version_etag = version_info["etag"] + 1
|
||||||
|
yield self.store.update_e2e_room_keys_version(
|
||||||
|
user_id, version, None, version_etag
|
||||||
|
)
|
||||||
|
|
||||||
|
count = yield self.store.count_e2e_room_keys(user_id, version)
|
||||||
|
return {"etag": str(version_etag), "count": count}
|
||||||
|
|
||||||
@trace
|
@trace
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def upload_room_keys(self, user_id, version, room_keys):
|
def upload_room_keys(self, user_id, version, room_keys):
|
||||||
|
|
@ -138,6 +160,9 @@ class E2eRoomKeysHandler(object):
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
A dict containing the count and etag for the backup version
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
NotFoundError: if there are no versions defined
|
NotFoundError: if there are no versions defined
|
||||||
RoomKeysVersionError: if the uploaded version is not the current version
|
RoomKeysVersionError: if the uploaded version is not the current version
|
||||||
|
|
@ -171,59 +196,62 @@ class E2eRoomKeysHandler(object):
|
||||||
else:
|
else:
|
||||||
raise
|
raise
|
||||||
|
|
||||||
# go through the room_keys.
|
# Fetch any existing room keys for the sessions that have been
|
||||||
# XXX: this should/could be done concurrently, given we're in a lock.
|
# submitted. Then compare them with the submitted keys. If the
|
||||||
|
# key is new, insert it; if the key should be updated, then update
|
||||||
|
# it; otherwise, drop it.
|
||||||
|
existing_keys = yield self.store.get_e2e_room_keys_multi(
|
||||||
|
user_id, version, room_keys["rooms"]
|
||||||
|
)
|
||||||
|
to_insert = [] # batch the inserts together
|
||||||
|
changed = False # if anything has changed, we need to update the etag
|
||||||
for room_id, room in iteritems(room_keys["rooms"]):
|
for room_id, room in iteritems(room_keys["rooms"]):
|
||||||
for session_id, session in iteritems(room["sessions"]):
|
for session_id, room_key in iteritems(room["sessions"]):
|
||||||
yield self._upload_room_key(
|
log_kv(
|
||||||
user_id, version, room_id, session_id, session
|
{
|
||||||
|
"message": "Trying to upload room key",
|
||||||
|
"room_id": room_id,
|
||||||
|
"session_id": session_id,
|
||||||
|
"user_id": user_id,
|
||||||
|
}
|
||||||
)
|
)
|
||||||
|
current_room_key = existing_keys.get(room_id, {}).get(session_id)
|
||||||
|
if current_room_key:
|
||||||
|
if self._should_replace_room_key(current_room_key, room_key):
|
||||||
|
log_kv({"message": "Replacing room key."})
|
||||||
|
# updates are done one at a time in the DB, so send
|
||||||
|
# updates right away rather than batching them up,
|
||||||
|
# like we do with the inserts
|
||||||
|
yield self.store.update_e2e_room_key(
|
||||||
|
user_id, version, room_id, session_id, room_key
|
||||||
|
)
|
||||||
|
changed = True
|
||||||
|
else:
|
||||||
|
log_kv({"message": "Not replacing room_key."})
|
||||||
|
else:
|
||||||
|
log_kv(
|
||||||
|
{
|
||||||
|
"message": "Room key not found.",
|
||||||
|
"room_id": room_id,
|
||||||
|
"user_id": user_id,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
log_kv({"message": "Replacing room key."})
|
||||||
|
to_insert.append((room_id, session_id, room_key))
|
||||||
|
changed = True
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
if len(to_insert):
|
||||||
def _upload_room_key(self, user_id, version, room_id, session_id, room_key):
|
yield self.store.add_e2e_room_keys(user_id, version, to_insert)
|
||||||
"""Upload a given room_key for a given room and session into a given
|
|
||||||
version of the backup. Merges the key with any which might already exist.
|
|
||||||
|
|
||||||
Args:
|
version_etag = version_info["etag"]
|
||||||
user_id(str): the user whose backup we're setting
|
if changed:
|
||||||
version(str): the version ID of the backup we're updating
|
version_etag = version_etag + 1
|
||||||
room_id(str): the ID of the room whose keys we're setting
|
yield self.store.update_e2e_room_keys_version(
|
||||||
session_id(str): the session whose room_key we're setting
|
user_id, version, None, version_etag
|
||||||
room_key(dict): the room_key being set
|
|
||||||
"""
|
|
||||||
log_kv(
|
|
||||||
{
|
|
||||||
"message": "Trying to upload room key",
|
|
||||||
"room_id": room_id,
|
|
||||||
"session_id": session_id,
|
|
||||||
"user_id": user_id,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
# get the room_key for this particular row
|
|
||||||
current_room_key = None
|
|
||||||
try:
|
|
||||||
current_room_key = yield self.store.get_e2e_room_key(
|
|
||||||
user_id, version, room_id, session_id
|
|
||||||
)
|
|
||||||
except StoreError as e:
|
|
||||||
if e.code == 404:
|
|
||||||
log_kv(
|
|
||||||
{
|
|
||||||
"message": "Room key not found.",
|
|
||||||
"room_id": room_id,
|
|
||||||
"user_id": user_id,
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
else:
|
|
||||||
raise
|
|
||||||
|
|
||||||
if self._should_replace_room_key(current_room_key, room_key):
|
count = yield self.store.count_e2e_room_keys(user_id, version)
|
||||||
log_kv({"message": "Replacing room key."})
|
return {"etag": str(version_etag), "count": count}
|
||||||
yield self.store.set_e2e_room_key(
|
|
||||||
user_id, version, room_id, session_id, room_key
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
log_kv({"message": "Not replacing room_key."})
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _should_replace_room_key(current_room_key, room_key):
|
def _should_replace_room_key(current_room_key, room_key):
|
||||||
|
|
@ -314,6 +342,8 @@ class E2eRoomKeysHandler(object):
|
||||||
raise NotFoundError("Unknown backup version")
|
raise NotFoundError("Unknown backup version")
|
||||||
else:
|
else:
|
||||||
raise
|
raise
|
||||||
|
|
||||||
|
res["count"] = yield self.store.count_e2e_room_keys(user_id, res["version"])
|
||||||
return res
|
return res
|
||||||
|
|
||||||
@trace
|
@trace
|
||||||
|
|
|
||||||
|
|
@ -16,8 +16,6 @@
|
||||||
import logging
|
import logging
|
||||||
import random
|
import random
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.constants import EventTypes, Membership
|
from synapse.api.constants import EventTypes, Membership
|
||||||
from synapse.api.errors import AuthError, SynapseError
|
from synapse.api.errors import AuthError, SynapseError
|
||||||
from synapse.events import EventBase
|
from synapse.events import EventBase
|
||||||
|
|
@ -50,9 +48,8 @@ class EventStreamHandler(BaseHandler):
|
||||||
self._server_notices_sender = hs.get_server_notices_sender()
|
self._server_notices_sender = hs.get_server_notices_sender()
|
||||||
self._event_serializer = hs.get_event_client_serializer()
|
self._event_serializer = hs.get_event_client_serializer()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
|
||||||
@log_function
|
@log_function
|
||||||
def get_stream(
|
async def get_stream(
|
||||||
self,
|
self,
|
||||||
auth_user_id,
|
auth_user_id,
|
||||||
pagin_config,
|
pagin_config,
|
||||||
|
|
@ -69,17 +66,17 @@ class EventStreamHandler(BaseHandler):
|
||||||
"""
|
"""
|
||||||
|
|
||||||
if room_id:
|
if room_id:
|
||||||
blocked = yield self.store.is_room_blocked(room_id)
|
blocked = await self.store.is_room_blocked(room_id)
|
||||||
if blocked:
|
if blocked:
|
||||||
raise SynapseError(403, "This room has been blocked on this server")
|
raise SynapseError(403, "This room has been blocked on this server")
|
||||||
|
|
||||||
# send any outstanding server notices to the user.
|
# send any outstanding server notices to the user.
|
||||||
yield self._server_notices_sender.on_user_syncing(auth_user_id)
|
await self._server_notices_sender.on_user_syncing(auth_user_id)
|
||||||
|
|
||||||
auth_user = UserID.from_string(auth_user_id)
|
auth_user = UserID.from_string(auth_user_id)
|
||||||
presence_handler = self.hs.get_presence_handler()
|
presence_handler = self.hs.get_presence_handler()
|
||||||
|
|
||||||
context = yield presence_handler.user_syncing(
|
context = await presence_handler.user_syncing(
|
||||||
auth_user_id, affect_presence=affect_presence
|
auth_user_id, affect_presence=affect_presence
|
||||||
)
|
)
|
||||||
with context:
|
with context:
|
||||||
|
|
@ -91,7 +88,7 @@ class EventStreamHandler(BaseHandler):
|
||||||
# thundering herds on restart.
|
# thundering herds on restart.
|
||||||
timeout = random.randint(int(timeout * 0.9), int(timeout * 1.1))
|
timeout = random.randint(int(timeout * 0.9), int(timeout * 1.1))
|
||||||
|
|
||||||
events, tokens = yield self.notifier.get_events_for(
|
events, tokens = await self.notifier.get_events_for(
|
||||||
auth_user,
|
auth_user,
|
||||||
pagin_config,
|
pagin_config,
|
||||||
timeout,
|
timeout,
|
||||||
|
|
@ -112,14 +109,14 @@ class EventStreamHandler(BaseHandler):
|
||||||
# Send down presence.
|
# Send down presence.
|
||||||
if event.state_key == auth_user_id:
|
if event.state_key == auth_user_id:
|
||||||
# Send down presence for everyone in the room.
|
# Send down presence for everyone in the room.
|
||||||
users = yield self.state.get_current_users_in_room(
|
users = await self.state.get_current_users_in_room(
|
||||||
event.room_id
|
event.room_id
|
||||||
)
|
)
|
||||||
states = yield presence_handler.get_states(users, as_event=True)
|
states = await presence_handler.get_states(users, as_event=True)
|
||||||
to_add.extend(states)
|
to_add.extend(states)
|
||||||
else:
|
else:
|
||||||
|
|
||||||
ev = yield presence_handler.get_state(
|
ev = await presence_handler.get_state(
|
||||||
UserID.from_string(event.state_key), as_event=True
|
UserID.from_string(event.state_key), as_event=True
|
||||||
)
|
)
|
||||||
to_add.append(ev)
|
to_add.append(ev)
|
||||||
|
|
@ -128,7 +125,7 @@ class EventStreamHandler(BaseHandler):
|
||||||
|
|
||||||
time_now = self.clock.time_msec()
|
time_now = self.clock.time_msec()
|
||||||
|
|
||||||
chunks = yield self._event_serializer.serialize_events(
|
chunks = await self._event_serializer.serialize_events(
|
||||||
events,
|
events,
|
||||||
time_now,
|
time_now,
|
||||||
as_client_event=as_client_event,
|
as_client_event=as_client_event,
|
||||||
|
|
@ -151,8 +148,7 @@ class EventHandler(BaseHandler):
|
||||||
super(EventHandler, self).__init__(hs)
|
super(EventHandler, self).__init__(hs)
|
||||||
self.storage = hs.get_storage()
|
self.storage = hs.get_storage()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def get_event(self, user, room_id, event_id):
|
||||||
def get_event(self, user, room_id, event_id):
|
|
||||||
"""Retrieve a single specified event.
|
"""Retrieve a single specified event.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -167,15 +163,15 @@ class EventHandler(BaseHandler):
|
||||||
AuthError if the user does not have the rights to inspect this
|
AuthError if the user does not have the rights to inspect this
|
||||||
event.
|
event.
|
||||||
"""
|
"""
|
||||||
event = yield self.store.get_event(event_id, check_room_id=room_id)
|
event = await self.store.get_event(event_id, check_room_id=room_id)
|
||||||
|
|
||||||
if not event:
|
if not event:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
users = yield self.store.get_users_in_room(event.room_id)
|
users = await self.store.get_users_in_room(event.room_id)
|
||||||
is_peeking = user.to_string() not in users
|
is_peeking = user.to_string() not in users
|
||||||
|
|
||||||
filtered = yield filter_events_for_client(
|
filtered = await filter_events_for_client(
|
||||||
self.storage, user.to_string(), [event], is_peeking=is_peeking
|
self.storage, user.to_string(), [event], is_peeking=is_peeking
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -19,11 +19,13 @@
|
||||||
|
|
||||||
import itertools
|
import itertools
|
||||||
import logging
|
import logging
|
||||||
|
from typing import Dict, Iterable, Optional, Sequence, Tuple
|
||||||
|
|
||||||
import six
|
import six
|
||||||
from six import iteritems, itervalues
|
from six import iteritems, itervalues
|
||||||
from six.moves import http_client, zip
|
from six.moves import http_client, zip
|
||||||
|
|
||||||
|
import attr
|
||||||
from signedjson.key import decode_verify_key_bytes
|
from signedjson.key import decode_verify_key_bytes
|
||||||
from signedjson.sign import verify_signed_json
|
from signedjson.sign import verify_signed_json
|
||||||
from unpaddedbase64 import decode_base64
|
from unpaddedbase64 import decode_base64
|
||||||
|
|
@ -45,6 +47,7 @@ from synapse.api.errors import (
|
||||||
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersions
|
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS, RoomVersions
|
||||||
from synapse.crypto.event_signing import compute_event_signature
|
from synapse.crypto.event_signing import compute_event_signature
|
||||||
from synapse.event_auth import auth_types_for_event
|
from synapse.event_auth import auth_types_for_event
|
||||||
|
from synapse.events import EventBase
|
||||||
from synapse.events.snapshot import EventContext
|
from synapse.events.snapshot import EventContext
|
||||||
from synapse.events.validator import EventValidator
|
from synapse.events.validator import EventValidator
|
||||||
from synapse.logging.context import (
|
from synapse.logging.context import (
|
||||||
|
|
@ -72,6 +75,23 @@ from ._base import BaseHandler
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@attr.s
|
||||||
|
class _NewEventInfo:
|
||||||
|
"""Holds information about a received event, ready for passing to _handle_new_events
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
event: the received event
|
||||||
|
|
||||||
|
state: the state at that event
|
||||||
|
|
||||||
|
auth_events: the auth_event map for that event
|
||||||
|
"""
|
||||||
|
|
||||||
|
event = attr.ib(type=EventBase)
|
||||||
|
state = attr.ib(type=Optional[Sequence[EventBase]], default=None)
|
||||||
|
auth_events = attr.ib(type=Optional[Dict[Tuple[str, str], EventBase]], default=None)
|
||||||
|
|
||||||
|
|
||||||
def shortstr(iterable, maxitems=5):
|
def shortstr(iterable, maxitems=5):
|
||||||
"""If iterable has maxitems or fewer, return the stringification of a list
|
"""If iterable has maxitems or fewer, return the stringification of a list
|
||||||
containing those items.
|
containing those items.
|
||||||
|
|
@ -121,6 +141,7 @@ class FederationHandler(BaseHandler):
|
||||||
self.pusher_pool = hs.get_pusherpool()
|
self.pusher_pool = hs.get_pusherpool()
|
||||||
self.spam_checker = hs.get_spam_checker()
|
self.spam_checker = hs.get_spam_checker()
|
||||||
self.event_creation_handler = hs.get_event_creation_handler()
|
self.event_creation_handler = hs.get_event_creation_handler()
|
||||||
|
self._message_handler = hs.get_message_handler()
|
||||||
self._server_notices_mxid = hs.config.server_notices_mxid
|
self._server_notices_mxid = hs.config.server_notices_mxid
|
||||||
self.config = hs.config
|
self.config = hs.config
|
||||||
self.http_client = hs.get_simple_http_client()
|
self.http_client = hs.get_simple_http_client()
|
||||||
|
|
@ -141,6 +162,8 @@ class FederationHandler(BaseHandler):
|
||||||
|
|
||||||
self.third_party_event_rules = hs.get_third_party_event_rules()
|
self.third_party_event_rules = hs.get_third_party_event_rules()
|
||||||
|
|
||||||
|
self._ephemeral_messages_enabled = hs.config.enable_ephemeral_messages
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def on_receive_pdu(self, origin, pdu, sent_to_us_directly=False):
|
def on_receive_pdu(self, origin, pdu, sent_to_us_directly=False):
|
||||||
""" Process a PDU received via a federation /send/ transaction, or
|
""" Process a PDU received via a federation /send/ transaction, or
|
||||||
|
|
@ -594,14 +617,14 @@ class FederationHandler(BaseHandler):
|
||||||
for e in auth_chain
|
for e in auth_chain
|
||||||
if e.event_id in auth_ids or e.type == EventTypes.Create
|
if e.event_id in auth_ids or e.type == EventTypes.Create
|
||||||
}
|
}
|
||||||
event_infos.append({"event": e, "auth_events": auth})
|
event_infos.append(_NewEventInfo(event=e, auth_events=auth))
|
||||||
seen_ids.add(e.event_id)
|
seen_ids.add(e.event_id)
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"[%s %s] persisting newly-received auth/state events %s",
|
"[%s %s] persisting newly-received auth/state events %s",
|
||||||
room_id,
|
room_id,
|
||||||
event_id,
|
event_id,
|
||||||
[e["event"].event_id for e in event_infos],
|
[e.event.event_id for e in event_infos],
|
||||||
)
|
)
|
||||||
yield self._handle_new_events(origin, event_infos)
|
yield self._handle_new_events(origin, event_infos)
|
||||||
|
|
||||||
|
|
@ -792,9 +815,9 @@ class FederationHandler(BaseHandler):
|
||||||
|
|
||||||
a.internal_metadata.outlier = True
|
a.internal_metadata.outlier = True
|
||||||
ev_infos.append(
|
ev_infos.append(
|
||||||
{
|
_NewEventInfo(
|
||||||
"event": a,
|
event=a,
|
||||||
"auth_events": {
|
auth_events={
|
||||||
(
|
(
|
||||||
auth_events[a_id].type,
|
auth_events[a_id].type,
|
||||||
auth_events[a_id].state_key,
|
auth_events[a_id].state_key,
|
||||||
|
|
@ -802,7 +825,7 @@ class FederationHandler(BaseHandler):
|
||||||
for a_id in a.auth_event_ids()
|
for a_id in a.auth_event_ids()
|
||||||
if a_id in auth_events
|
if a_id in auth_events
|
||||||
},
|
},
|
||||||
}
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# Step 1b: persist the events in the chunk we fetched state for (i.e.
|
# Step 1b: persist the events in the chunk we fetched state for (i.e.
|
||||||
|
|
@ -814,10 +837,10 @@ class FederationHandler(BaseHandler):
|
||||||
assert not ev.internal_metadata.is_outlier()
|
assert not ev.internal_metadata.is_outlier()
|
||||||
|
|
||||||
ev_infos.append(
|
ev_infos.append(
|
||||||
{
|
_NewEventInfo(
|
||||||
"event": ev,
|
event=ev,
|
||||||
"state": events_to_state[e_id],
|
state=events_to_state[e_id],
|
||||||
"auth_events": {
|
auth_events={
|
||||||
(
|
(
|
||||||
auth_events[a_id].type,
|
auth_events[a_id].type,
|
||||||
auth_events[a_id].state_key,
|
auth_events[a_id].state_key,
|
||||||
|
|
@ -825,7 +848,7 @@ class FederationHandler(BaseHandler):
|
||||||
for a_id in ev.auth_event_ids()
|
for a_id in ev.auth_event_ids()
|
||||||
if a_id in auth_events
|
if a_id in auth_events
|
||||||
},
|
},
|
||||||
}
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
yield self._handle_new_events(dest, ev_infos, backfilled=True)
|
yield self._handle_new_events(dest, ev_infos, backfilled=True)
|
||||||
|
|
@ -1428,9 +1451,9 @@ class FederationHandler(BaseHandler):
|
||||||
return event
|
return event
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def do_remotely_reject_invite(self, target_hosts, room_id, user_id):
|
def do_remotely_reject_invite(self, target_hosts, room_id, user_id, content):
|
||||||
origin, event, event_format_version = yield self._make_and_verify_event(
|
origin, event, event_format_version = yield self._make_and_verify_event(
|
||||||
target_hosts, room_id, user_id, "leave"
|
target_hosts, room_id, user_id, "leave", content=content,
|
||||||
)
|
)
|
||||||
# Mark as outlier as we don't have any state for this event; we're not
|
# Mark as outlier as we don't have any state for this event; we're not
|
||||||
# even in the room.
|
# even in the room.
|
||||||
|
|
@ -1710,7 +1733,12 @@ class FederationHandler(BaseHandler):
|
||||||
return context
|
return context
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def _handle_new_events(self, origin, event_infos, backfilled=False):
|
def _handle_new_events(
|
||||||
|
self,
|
||||||
|
origin: str,
|
||||||
|
event_infos: Iterable[_NewEventInfo],
|
||||||
|
backfilled: bool = False,
|
||||||
|
):
|
||||||
"""Creates the appropriate contexts and persists events. The events
|
"""Creates the appropriate contexts and persists events. The events
|
||||||
should not depend on one another, e.g. this should be used to persist
|
should not depend on one another, e.g. this should be used to persist
|
||||||
a bunch of outliers, but not a chunk of individual events that depend
|
a bunch of outliers, but not a chunk of individual events that depend
|
||||||
|
|
@ -1720,14 +1748,14 @@ class FederationHandler(BaseHandler):
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def prep(ev_info):
|
def prep(ev_info: _NewEventInfo):
|
||||||
event = ev_info["event"]
|
event = ev_info.event
|
||||||
with nested_logging_context(suffix=event.event_id):
|
with nested_logging_context(suffix=event.event_id):
|
||||||
res = yield self._prep_event(
|
res = yield self._prep_event(
|
||||||
origin,
|
origin,
|
||||||
event,
|
event,
|
||||||
state=ev_info.get("state"),
|
state=ev_info.state,
|
||||||
auth_events=ev_info.get("auth_events"),
|
auth_events=ev_info.auth_events,
|
||||||
backfilled=backfilled,
|
backfilled=backfilled,
|
||||||
)
|
)
|
||||||
return res
|
return res
|
||||||
|
|
@ -1741,7 +1769,7 @@ class FederationHandler(BaseHandler):
|
||||||
|
|
||||||
yield self.persist_events_and_notify(
|
yield self.persist_events_and_notify(
|
||||||
[
|
[
|
||||||
(ev_info["event"], context)
|
(ev_info.event, context)
|
||||||
for ev_info, context in zip(event_infos, contexts)
|
for ev_info, context in zip(event_infos, contexts)
|
||||||
],
|
],
|
||||||
backfilled=backfilled,
|
backfilled=backfilled,
|
||||||
|
|
@ -1843,7 +1871,14 @@ class FederationHandler(BaseHandler):
|
||||||
yield self.persist_events_and_notify([(event, new_event_context)])
|
yield self.persist_events_and_notify([(event, new_event_context)])
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def _prep_event(self, origin, event, state, auth_events, backfilled):
|
def _prep_event(
|
||||||
|
self,
|
||||||
|
origin: str,
|
||||||
|
event: EventBase,
|
||||||
|
state: Optional[Iterable[EventBase]],
|
||||||
|
auth_events: Optional[Dict[Tuple[str, str], EventBase]],
|
||||||
|
backfilled: bool,
|
||||||
|
):
|
||||||
"""
|
"""
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -1851,7 +1886,7 @@ class FederationHandler(BaseHandler):
|
||||||
event:
|
event:
|
||||||
state:
|
state:
|
||||||
auth_events:
|
auth_events:
|
||||||
backfilled (bool)
|
backfilled:
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Deferred, which resolves to synapse.events.snapshot.EventContext
|
Deferred, which resolves to synapse.events.snapshot.EventContext
|
||||||
|
|
@ -1887,15 +1922,16 @@ class FederationHandler(BaseHandler):
|
||||||
return context
|
return context
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def _check_for_soft_fail(self, event, state, backfilled):
|
def _check_for_soft_fail(
|
||||||
|
self, event: EventBase, state: Optional[Iterable[EventBase]], backfilled: bool
|
||||||
|
):
|
||||||
"""Checks if we should soft fail the event, if so marks the event as
|
"""Checks if we should soft fail the event, if so marks the event as
|
||||||
such.
|
such.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
event (FrozenEvent)
|
event
|
||||||
state (dict|None): The state at the event if we don't have all the
|
state: The state at the event if we don't have all the event's prev events
|
||||||
event's prev events
|
backfilled: Whether the event is from backfill
|
||||||
backfilled (bool): Whether the event is from backfill
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Deferred
|
Deferred
|
||||||
|
|
@ -2040,8 +2076,10 @@ class FederationHandler(BaseHandler):
|
||||||
auth_events (dict[(str, str)->synapse.events.EventBase]):
|
auth_events (dict[(str, str)->synapse.events.EventBase]):
|
||||||
Map from (event_type, state_key) to event
|
Map from (event_type, state_key) to event
|
||||||
|
|
||||||
What we expect the event's auth_events to be, based on the event's
|
Normally, our calculated auth_events based on the state of the room
|
||||||
position in the dag. I think? maybe??
|
at the event's position in the DAG, though occasionally (eg if the
|
||||||
|
event is an outlier), may be the auth events claimed by the remote
|
||||||
|
server.
|
||||||
|
|
||||||
Also NB that this function adds entries to it.
|
Also NB that this function adds entries to it.
|
||||||
Returns:
|
Returns:
|
||||||
|
|
@ -2091,35 +2129,35 @@ class FederationHandler(BaseHandler):
|
||||||
origin (str):
|
origin (str):
|
||||||
event (synapse.events.EventBase):
|
event (synapse.events.EventBase):
|
||||||
context (synapse.events.snapshot.EventContext):
|
context (synapse.events.snapshot.EventContext):
|
||||||
|
|
||||||
auth_events (dict[(str, str)->synapse.events.EventBase]):
|
auth_events (dict[(str, str)->synapse.events.EventBase]):
|
||||||
|
Map from (event_type, state_key) to event
|
||||||
|
|
||||||
|
Normally, our calculated auth_events based on the state of the room
|
||||||
|
at the event's position in the DAG, though occasionally (eg if the
|
||||||
|
event is an outlier), may be the auth events claimed by the remote
|
||||||
|
server.
|
||||||
|
|
||||||
|
Also NB that this function adds entries to it.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
defer.Deferred[EventContext]: updated context
|
defer.Deferred[EventContext]: updated context
|
||||||
"""
|
"""
|
||||||
event_auth_events = set(event.auth_event_ids())
|
event_auth_events = set(event.auth_event_ids())
|
||||||
|
|
||||||
if event.is_state():
|
# missing_auth is the set of the event's auth_events which we don't yet have
|
||||||
event_key = (event.type, event.state_key)
|
# in auth_events.
|
||||||
else:
|
|
||||||
event_key = None
|
|
||||||
|
|
||||||
# if the event's auth_events refers to events which are not in our
|
|
||||||
# calculated auth_events, we need to fetch those events from somewhere.
|
|
||||||
#
|
|
||||||
# we start by fetching them from the store, and then try calling /event_auth/.
|
|
||||||
missing_auth = event_auth_events.difference(
|
missing_auth = event_auth_events.difference(
|
||||||
e.event_id for e in auth_events.values()
|
e.event_id for e in auth_events.values()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# if we have missing events, we need to fetch those events from somewhere.
|
||||||
|
#
|
||||||
|
# we start by checking if they are in the store, and then try calling /event_auth/.
|
||||||
if missing_auth:
|
if missing_auth:
|
||||||
# TODO: can we use store.have_seen_events here instead?
|
have_events = yield self.store.have_seen_events(missing_auth)
|
||||||
have_events = yield self.store.get_seen_events_with_rejections(missing_auth)
|
logger.debug("Events %s are in the store", have_events)
|
||||||
logger.debug("Got events %s from store", have_events)
|
missing_auth.difference_update(have_events)
|
||||||
missing_auth.difference_update(have_events.keys())
|
|
||||||
else:
|
|
||||||
have_events = {}
|
|
||||||
|
|
||||||
have_events.update({e.event_id: "" for e in auth_events.values()})
|
|
||||||
|
|
||||||
if missing_auth:
|
if missing_auth:
|
||||||
# If we don't have all the auth events, we need to get them.
|
# If we don't have all the auth events, we need to get them.
|
||||||
|
|
@ -2165,19 +2203,18 @@ class FederationHandler(BaseHandler):
|
||||||
except AuthError:
|
except AuthError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
have_events = yield self.store.get_seen_events_with_rejections(
|
|
||||||
event.auth_event_ids()
|
|
||||||
)
|
|
||||||
except Exception:
|
except Exception:
|
||||||
# FIXME:
|
|
||||||
logger.exception("Failed to get auth chain")
|
logger.exception("Failed to get auth chain")
|
||||||
|
|
||||||
if event.internal_metadata.is_outlier():
|
if event.internal_metadata.is_outlier():
|
||||||
|
# XXX: given that, for an outlier, we'll be working with the
|
||||||
|
# event's *claimed* auth events rather than those we calculated:
|
||||||
|
# (a) is there any point in this test, since different_auth below will
|
||||||
|
# obviously be empty
|
||||||
|
# (b) alternatively, why don't we do it earlier?
|
||||||
logger.info("Skipping auth_event fetch for outlier")
|
logger.info("Skipping auth_event fetch for outlier")
|
||||||
return context
|
return context
|
||||||
|
|
||||||
# FIXME: Assumes we have and stored all the state for all the
|
|
||||||
# prev_events
|
|
||||||
different_auth = event_auth_events.difference(
|
different_auth = event_auth_events.difference(
|
||||||
e.event_id for e in auth_events.values()
|
e.event_id for e in auth_events.values()
|
||||||
)
|
)
|
||||||
|
|
@ -2191,53 +2228,58 @@ class FederationHandler(BaseHandler):
|
||||||
different_auth,
|
different_auth,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# XXX: currently this checks for redactions but I'm not convinced that is
|
||||||
|
# necessary?
|
||||||
|
different_events = yield self.store.get_events_as_list(different_auth)
|
||||||
|
|
||||||
|
for d in different_events:
|
||||||
|
if d.room_id != event.room_id:
|
||||||
|
logger.warning(
|
||||||
|
"Event %s refers to auth_event %s which is in a different room",
|
||||||
|
event.event_id,
|
||||||
|
d.event_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
# don't attempt to resolve the claimed auth events against our own
|
||||||
|
# in this case: just use our own auth events.
|
||||||
|
#
|
||||||
|
# XXX: should we reject the event in this case? It feels like we should,
|
||||||
|
# but then shouldn't we also do so if we've failed to fetch any of the
|
||||||
|
# auth events?
|
||||||
|
return context
|
||||||
|
|
||||||
|
# now we state-resolve between our own idea of the auth events, and the remote's
|
||||||
|
# idea of them.
|
||||||
|
|
||||||
|
local_state = auth_events.values()
|
||||||
|
remote_auth_events = dict(auth_events)
|
||||||
|
remote_auth_events.update({(d.type, d.state_key): d for d in different_events})
|
||||||
|
remote_state = remote_auth_events.values()
|
||||||
|
|
||||||
room_version = yield self.store.get_room_version(event.room_id)
|
room_version = yield self.store.get_room_version(event.room_id)
|
||||||
|
new_state = yield self.state_handler.resolve_events(
|
||||||
|
room_version, (local_state, remote_state), event
|
||||||
|
)
|
||||||
|
|
||||||
different_events = yield make_deferred_yieldable(
|
logger.info(
|
||||||
defer.gatherResults(
|
"After state res: updating auth_events with new state %s",
|
||||||
[
|
{
|
||||||
run_in_background(
|
(d.type, d.state_key): d.event_id
|
||||||
self.store.get_event, d, allow_none=True, allow_rejected=False
|
for d in new_state.values()
|
||||||
)
|
if auth_events.get((d.type, d.state_key)) != d
|
||||||
for d in different_auth
|
},
|
||||||
if d in have_events and not have_events[d]
|
)
|
||||||
],
|
|
||||||
consumeErrors=True,
|
|
||||||
)
|
|
||||||
).addErrback(unwrapFirstError)
|
|
||||||
|
|
||||||
if different_events:
|
auth_events.update(new_state)
|
||||||
local_view = dict(auth_events)
|
|
||||||
remote_view = dict(auth_events)
|
|
||||||
remote_view.update(
|
|
||||||
{(d.type, d.state_key): d for d in different_events if d}
|
|
||||||
)
|
|
||||||
|
|
||||||
new_state = yield self.state_handler.resolve_events(
|
context = yield self._update_context_for_auth_events(
|
||||||
room_version,
|
event, context, auth_events
|
||||||
[list(local_view.values()), list(remote_view.values())],
|
)
|
||||||
event,
|
|
||||||
)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"After state res: updating auth_events with new state %s",
|
|
||||||
{
|
|
||||||
(d.type, d.state_key): d.event_id
|
|
||||||
for d in new_state.values()
|
|
||||||
if auth_events.get((d.type, d.state_key)) != d
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
auth_events.update(new_state)
|
|
||||||
|
|
||||||
context = yield self._update_context_for_auth_events(
|
|
||||||
event, context, auth_events, event_key
|
|
||||||
)
|
|
||||||
|
|
||||||
return context
|
return context
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def _update_context_for_auth_events(self, event, context, auth_events, event_key):
|
def _update_context_for_auth_events(self, event, context, auth_events):
|
||||||
"""Update the state_ids in an event context after auth event resolution,
|
"""Update the state_ids in an event context after auth event resolution,
|
||||||
storing the changes as a new state group.
|
storing the changes as a new state group.
|
||||||
|
|
||||||
|
|
@ -2246,18 +2288,21 @@ class FederationHandler(BaseHandler):
|
||||||
|
|
||||||
context (synapse.events.snapshot.EventContext): initial event context
|
context (synapse.events.snapshot.EventContext): initial event context
|
||||||
|
|
||||||
auth_events (dict[(str, str)->str]): Events to update in the event
|
auth_events (dict[(str, str)->EventBase]): Events to update in the event
|
||||||
context.
|
context.
|
||||||
|
|
||||||
event_key ((str, str)): (type, state_key) for the current event.
|
|
||||||
this will not be included in the current_state in the context.
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Deferred[EventContext]: new event context
|
Deferred[EventContext]: new event context
|
||||||
"""
|
"""
|
||||||
|
# exclude the state key of the new event from the current_state in the context.
|
||||||
|
if event.is_state():
|
||||||
|
event_key = (event.type, event.state_key)
|
||||||
|
else:
|
||||||
|
event_key = None
|
||||||
state_updates = {
|
state_updates = {
|
||||||
k: a.event_id for k, a in iteritems(auth_events) if k != event_key
|
k: a.event_id for k, a in iteritems(auth_events) if k != event_key
|
||||||
}
|
}
|
||||||
|
|
||||||
current_state_ids = yield context.get_current_state_ids(self.store)
|
current_state_ids = yield context.get_current_state_ids(self.store)
|
||||||
current_state_ids = dict(current_state_ids)
|
current_state_ids = dict(current_state_ids)
|
||||||
|
|
||||||
|
|
@ -2459,7 +2504,7 @@ class FederationHandler(BaseHandler):
|
||||||
room_version, event_dict, event, context
|
room_version, event_dict, event, context
|
||||||
)
|
)
|
||||||
|
|
||||||
EventValidator().validate_new(event)
|
EventValidator().validate_new(event, self.config)
|
||||||
|
|
||||||
# We need to tell the transaction queue to send this out, even
|
# We need to tell the transaction queue to send this out, even
|
||||||
# though the sender isn't a local user.
|
# though the sender isn't a local user.
|
||||||
|
|
@ -2574,7 +2619,7 @@ class FederationHandler(BaseHandler):
|
||||||
event, context = yield self.event_creation_handler.create_new_client_event(
|
event, context = yield self.event_creation_handler.create_new_client_event(
|
||||||
builder=builder
|
builder=builder
|
||||||
)
|
)
|
||||||
EventValidator().validate_new(event)
|
EventValidator().validate_new(event, self.config)
|
||||||
return (event, context)
|
return (event, context)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
|
|
@ -2708,6 +2753,11 @@ class FederationHandler(BaseHandler):
|
||||||
event_and_contexts, backfilled=backfilled
|
event_and_contexts, backfilled=backfilled
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if self._ephemeral_messages_enabled:
|
||||||
|
for (event, context) in event_and_contexts:
|
||||||
|
# If there's an expiry timestamp on the event, schedule its expiry.
|
||||||
|
self._message_handler.maybe_schedule_expiry(event)
|
||||||
|
|
||||||
if not backfilled: # Never notify for backfilled events
|
if not backfilled: # Never notify for backfilled events
|
||||||
for event, _ in event_and_contexts:
|
for event, _ in event_and_contexts:
|
||||||
yield self._notify_persisted_event(event, max_stream_id)
|
yield self._notify_persisted_event(event, max_stream_id)
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@
|
||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
import logging
|
import logging
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
from six import iteritems, itervalues, string_types
|
from six import iteritems, itervalues, string_types
|
||||||
|
|
||||||
|
|
@ -22,9 +23,16 @@ from canonicaljson import encode_canonical_json, json
|
||||||
|
|
||||||
from twisted.internet import defer
|
from twisted.internet import defer
|
||||||
from twisted.internet.defer import succeed
|
from twisted.internet.defer import succeed
|
||||||
|
from twisted.internet.interfaces import IDelayedCall
|
||||||
|
|
||||||
from synapse import event_auth
|
from synapse import event_auth
|
||||||
from synapse.api.constants import EventTypes, Membership, RelationTypes, UserTypes
|
from synapse.api.constants import (
|
||||||
|
EventContentFields,
|
||||||
|
EventTypes,
|
||||||
|
Membership,
|
||||||
|
RelationTypes,
|
||||||
|
UserTypes,
|
||||||
|
)
|
||||||
from synapse.api.errors import (
|
from synapse.api.errors import (
|
||||||
AuthError,
|
AuthError,
|
||||||
Codes,
|
Codes,
|
||||||
|
|
@ -62,6 +70,17 @@ class MessageHandler(object):
|
||||||
self.storage = hs.get_storage()
|
self.storage = hs.get_storage()
|
||||||
self.state_store = self.storage.state
|
self.state_store = self.storage.state
|
||||||
self._event_serializer = hs.get_event_client_serializer()
|
self._event_serializer = hs.get_event_client_serializer()
|
||||||
|
self._ephemeral_events_enabled = hs.config.enable_ephemeral_messages
|
||||||
|
self._is_worker_app = bool(hs.config.worker_app)
|
||||||
|
|
||||||
|
# The scheduled call to self._expire_event. None if no call is currently
|
||||||
|
# scheduled.
|
||||||
|
self._scheduled_expiry = None # type: Optional[IDelayedCall]
|
||||||
|
|
||||||
|
if not hs.config.worker_app:
|
||||||
|
run_as_background_process(
|
||||||
|
"_schedule_next_expiry", self._schedule_next_expiry
|
||||||
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def get_room_data(
|
def get_room_data(
|
||||||
|
|
@ -138,7 +157,7 @@ class MessageHandler(object):
|
||||||
raise NotFoundError("Can't find event for token %s" % (at_token,))
|
raise NotFoundError("Can't find event for token %s" % (at_token,))
|
||||||
|
|
||||||
visible_events = yield filter_events_for_client(
|
visible_events = yield filter_events_for_client(
|
||||||
self.storage, user_id, last_events
|
self.storage, user_id, last_events, apply_retention_policies=False
|
||||||
)
|
)
|
||||||
|
|
||||||
event = last_events[0]
|
event = last_events[0]
|
||||||
|
|
@ -225,6 +244,100 @@ class MessageHandler(object):
|
||||||
for user_id, profile in iteritems(users_with_profile)
|
for user_id, profile in iteritems(users_with_profile)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def maybe_schedule_expiry(self, event):
|
||||||
|
"""Schedule the expiry of an event if there's not already one scheduled,
|
||||||
|
or if the one running is for an event that will expire after the provided
|
||||||
|
timestamp.
|
||||||
|
|
||||||
|
This function needs to invalidate the event cache, which is only possible on
|
||||||
|
the master process, and therefore needs to be run on there.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
event (EventBase): The event to schedule the expiry of.
|
||||||
|
"""
|
||||||
|
assert not self._is_worker_app
|
||||||
|
|
||||||
|
expiry_ts = event.content.get(EventContentFields.SELF_DESTRUCT_AFTER)
|
||||||
|
if not isinstance(expiry_ts, int) or event.is_state():
|
||||||
|
return
|
||||||
|
|
||||||
|
# _schedule_expiry_for_event won't actually schedule anything if there's already
|
||||||
|
# a task scheduled for a timestamp that's sooner than the provided one.
|
||||||
|
self._schedule_expiry_for_event(event.event_id, expiry_ts)
|
||||||
|
|
||||||
|
@defer.inlineCallbacks
|
||||||
|
def _schedule_next_expiry(self):
|
||||||
|
"""Retrieve the ID and the expiry timestamp of the next event to be expired,
|
||||||
|
and schedule an expiry task for it.
|
||||||
|
|
||||||
|
If there's no event left to expire, set _expiry_scheduled to None so that a
|
||||||
|
future call to save_expiry_ts can schedule a new expiry task.
|
||||||
|
"""
|
||||||
|
# Try to get the expiry timestamp of the next event to expire.
|
||||||
|
res = yield self.store.get_next_event_to_expire()
|
||||||
|
if res:
|
||||||
|
event_id, expiry_ts = res
|
||||||
|
self._schedule_expiry_for_event(event_id, expiry_ts)
|
||||||
|
|
||||||
|
def _schedule_expiry_for_event(self, event_id, expiry_ts):
|
||||||
|
"""Schedule an expiry task for the provided event if there's not already one
|
||||||
|
scheduled at a timestamp that's sooner than the provided one.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
event_id (str): The ID of the event to expire.
|
||||||
|
expiry_ts (int): The timestamp at which to expire the event.
|
||||||
|
"""
|
||||||
|
if self._scheduled_expiry:
|
||||||
|
# If the provided timestamp refers to a time before the scheduled time of the
|
||||||
|
# next expiry task, cancel that task and reschedule it for this timestamp.
|
||||||
|
next_scheduled_expiry_ts = self._scheduled_expiry.getTime() * 1000
|
||||||
|
if expiry_ts < next_scheduled_expiry_ts:
|
||||||
|
self._scheduled_expiry.cancel()
|
||||||
|
else:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Figure out how many seconds we need to wait before expiring the event.
|
||||||
|
now_ms = self.clock.time_msec()
|
||||||
|
delay = (expiry_ts - now_ms) / 1000
|
||||||
|
|
||||||
|
# callLater doesn't support negative delays, so trim the delay to 0 if we're
|
||||||
|
# in that case.
|
||||||
|
if delay < 0:
|
||||||
|
delay = 0
|
||||||
|
|
||||||
|
logger.info("Scheduling expiry for event %s in %.3fs", event_id, delay)
|
||||||
|
|
||||||
|
self._scheduled_expiry = self.clock.call_later(
|
||||||
|
delay,
|
||||||
|
run_as_background_process,
|
||||||
|
"_expire_event",
|
||||||
|
self._expire_event,
|
||||||
|
event_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
@defer.inlineCallbacks
|
||||||
|
def _expire_event(self, event_id):
|
||||||
|
"""Retrieve and expire an event that needs to be expired from the database.
|
||||||
|
|
||||||
|
If the event doesn't exist in the database, log it and delete the expiry date
|
||||||
|
from the database (so that we don't try to expire it again).
|
||||||
|
"""
|
||||||
|
assert self._ephemeral_events_enabled
|
||||||
|
|
||||||
|
self._scheduled_expiry = None
|
||||||
|
|
||||||
|
logger.info("Expiring event %s", event_id)
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Expire the event if we know about it. This function also deletes the expiry
|
||||||
|
# date from the database in the same database transaction.
|
||||||
|
yield self.store.expire_event(event_id)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Could not expire event %s: %r", event_id, e)
|
||||||
|
|
||||||
|
# Schedule the expiry of the next event to expire.
|
||||||
|
yield self._schedule_next_expiry()
|
||||||
|
|
||||||
|
|
||||||
# The duration (in ms) after which rooms should be removed
|
# The duration (in ms) after which rooms should be removed
|
||||||
# `_rooms_to_exclude_from_dummy_event_insertion` (with the effect that we will try
|
# `_rooms_to_exclude_from_dummy_event_insertion` (with the effect that we will try
|
||||||
|
|
@ -250,6 +363,8 @@ class EventCreationHandler(object):
|
||||||
self.config = hs.config
|
self.config = hs.config
|
||||||
self.require_membership_for_aliases = hs.config.require_membership_for_aliases
|
self.require_membership_for_aliases = hs.config.require_membership_for_aliases
|
||||||
|
|
||||||
|
self.room_invite_state_types = self.hs.config.room_invite_state_types
|
||||||
|
|
||||||
self.send_event_to_master = ReplicationSendEventRestServlet.make_client(hs)
|
self.send_event_to_master = ReplicationSendEventRestServlet.make_client(hs)
|
||||||
|
|
||||||
# This is only used to get at ratelimit function, and maybe_kick_guest_users
|
# This is only used to get at ratelimit function, and maybe_kick_guest_users
|
||||||
|
|
@ -295,6 +410,10 @@ class EventCreationHandler(object):
|
||||||
5 * 60 * 1000,
|
5 * 60 * 1000,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self._message_handler = hs.get_message_handler()
|
||||||
|
|
||||||
|
self._ephemeral_events_enabled = hs.config.enable_ephemeral_messages
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def create_event(
|
def create_event(
|
||||||
self,
|
self,
|
||||||
|
|
@ -417,7 +536,7 @@ class EventCreationHandler(object):
|
||||||
403, "You must be in the room to create an alias for it"
|
403, "You must be in the room to create an alias for it"
|
||||||
)
|
)
|
||||||
|
|
||||||
self.validator.validate_new(event)
|
self.validator.validate_new(event, self.config)
|
||||||
|
|
||||||
return (event, context)
|
return (event, context)
|
||||||
|
|
||||||
|
|
@ -634,7 +753,7 @@ class EventCreationHandler(object):
|
||||||
if requester:
|
if requester:
|
||||||
context.app_service = requester.app_service
|
context.app_service = requester.app_service
|
||||||
|
|
||||||
self.validator.validate_new(event)
|
self.validator.validate_new(event, self.config)
|
||||||
|
|
||||||
# If this event is an annotation then we check that that the sender
|
# If this event is an annotation then we check that that the sender
|
||||||
# can't annotate the same way twice (e.g. stops users from liking an
|
# can't annotate the same way twice (e.g. stops users from liking an
|
||||||
|
|
@ -799,7 +918,7 @@ class EventCreationHandler(object):
|
||||||
state_to_include_ids = [
|
state_to_include_ids = [
|
||||||
e_id
|
e_id
|
||||||
for k, e_id in iteritems(current_state_ids)
|
for k, e_id in iteritems(current_state_ids)
|
||||||
if k[0] in self.hs.config.room_invite_state_types
|
if k[0] in self.room_invite_state_types
|
||||||
or k == (EventTypes.Member, event.sender)
|
or k == (EventTypes.Member, event.sender)
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
@ -877,6 +996,10 @@ class EventCreationHandler(object):
|
||||||
event, context=context
|
event, context=context
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if self._ephemeral_events_enabled:
|
||||||
|
# If there's an expiry timestamp on the event, schedule its expiry.
|
||||||
|
self._message_handler.maybe_schedule_expiry(event)
|
||||||
|
|
||||||
yield self.pusher_pool.on_new_notifications(event_stream_id, max_stream_id)
|
yield self.pusher_pool.on_new_notifications(event_stream_id, max_stream_id)
|
||||||
|
|
||||||
def _notify():
|
def _notify():
|
||||||
|
|
|
||||||
|
|
@ -15,12 +15,15 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
|
from six import iteritems
|
||||||
|
|
||||||
from twisted.internet import defer
|
from twisted.internet import defer
|
||||||
from twisted.python.failure import Failure
|
from twisted.python.failure import Failure
|
||||||
|
|
||||||
from synapse.api.constants import EventTypes, Membership
|
from synapse.api.constants import EventTypes, Membership
|
||||||
from synapse.api.errors import SynapseError
|
from synapse.api.errors import SynapseError
|
||||||
from synapse.logging.context import run_in_background
|
from synapse.logging.context import run_in_background
|
||||||
|
from synapse.metrics.background_process_metrics import run_as_background_process
|
||||||
from synapse.storage.state import StateFilter
|
from synapse.storage.state import StateFilter
|
||||||
from synapse.types import RoomStreamToken
|
from synapse.types import RoomStreamToken
|
||||||
from synapse.util.async_helpers import ReadWriteLock
|
from synapse.util.async_helpers import ReadWriteLock
|
||||||
|
|
@ -80,6 +83,109 @@ class PaginationHandler(object):
|
||||||
self._purges_by_id = {}
|
self._purges_by_id = {}
|
||||||
self._event_serializer = hs.get_event_client_serializer()
|
self._event_serializer = hs.get_event_client_serializer()
|
||||||
|
|
||||||
|
self._retention_default_max_lifetime = hs.config.retention_default_max_lifetime
|
||||||
|
|
||||||
|
if hs.config.retention_enabled:
|
||||||
|
# Run the purge jobs described in the configuration file.
|
||||||
|
for job in hs.config.retention_purge_jobs:
|
||||||
|
self.clock.looping_call(
|
||||||
|
run_as_background_process,
|
||||||
|
job["interval"],
|
||||||
|
"purge_history_for_rooms_in_range",
|
||||||
|
self.purge_history_for_rooms_in_range,
|
||||||
|
job["shortest_max_lifetime"],
|
||||||
|
job["longest_max_lifetime"],
|
||||||
|
)
|
||||||
|
|
||||||
|
@defer.inlineCallbacks
|
||||||
|
def purge_history_for_rooms_in_range(self, min_ms, max_ms):
|
||||||
|
"""Purge outdated events from rooms within the given retention range.
|
||||||
|
|
||||||
|
If a default retention policy is defined in the server's configuration and its
|
||||||
|
'max_lifetime' is within this range, also targets rooms which don't have a
|
||||||
|
retention policy.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
min_ms (int|None): Duration in milliseconds that define the lower limit of
|
||||||
|
the range to handle (exclusive). If None, it means that the range has no
|
||||||
|
lower limit.
|
||||||
|
max_ms (int|None): Duration in milliseconds that define the upper limit of
|
||||||
|
the range to handle (inclusive). If None, it means that the range has no
|
||||||
|
upper limit.
|
||||||
|
"""
|
||||||
|
# We want the storage layer to to include rooms with no retention policy in its
|
||||||
|
# return value only if a default retention policy is defined in the server's
|
||||||
|
# configuration and that policy's 'max_lifetime' is either lower (or equal) than
|
||||||
|
# max_ms or higher than min_ms (or both).
|
||||||
|
if self._retention_default_max_lifetime is not None:
|
||||||
|
include_null = True
|
||||||
|
|
||||||
|
if min_ms is not None and min_ms >= self._retention_default_max_lifetime:
|
||||||
|
# The default max_lifetime is lower than (or equal to) min_ms.
|
||||||
|
include_null = False
|
||||||
|
|
||||||
|
if max_ms is not None and max_ms < self._retention_default_max_lifetime:
|
||||||
|
# The default max_lifetime is higher than max_ms.
|
||||||
|
include_null = False
|
||||||
|
else:
|
||||||
|
include_null = False
|
||||||
|
|
||||||
|
rooms = yield self.store.get_rooms_for_retention_period_in_range(
|
||||||
|
min_ms, max_ms, include_null
|
||||||
|
)
|
||||||
|
|
||||||
|
for room_id, retention_policy in iteritems(rooms):
|
||||||
|
if room_id in self._purges_in_progress_by_room:
|
||||||
|
logger.warning(
|
||||||
|
"[purge] not purging room %s as there's an ongoing purge running"
|
||||||
|
" for this room",
|
||||||
|
room_id,
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
max_lifetime = retention_policy["max_lifetime"]
|
||||||
|
|
||||||
|
if max_lifetime is None:
|
||||||
|
# If max_lifetime is None, it means that include_null equals True,
|
||||||
|
# therefore we can safely assume that there is a default policy defined
|
||||||
|
# in the server's configuration.
|
||||||
|
max_lifetime = self._retention_default_max_lifetime
|
||||||
|
|
||||||
|
# Figure out what token we should start purging at.
|
||||||
|
ts = self.clock.time_msec() - max_lifetime
|
||||||
|
|
||||||
|
stream_ordering = yield self.store.find_first_stream_ordering_after_ts(ts)
|
||||||
|
|
||||||
|
r = yield self.store.get_room_event_after_stream_ordering(
|
||||||
|
room_id, stream_ordering,
|
||||||
|
)
|
||||||
|
if not r:
|
||||||
|
logger.warning(
|
||||||
|
"[purge] purging events not possible: No event found "
|
||||||
|
"(ts %i => stream_ordering %i)",
|
||||||
|
ts,
|
||||||
|
stream_ordering,
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
(stream, topo, _event_id) = r
|
||||||
|
token = "t%d-%d" % (topo, stream)
|
||||||
|
|
||||||
|
purge_id = random_string(16)
|
||||||
|
|
||||||
|
self._purges_by_id[purge_id] = PurgeStatus()
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"Starting purging events in room %s (purge_id %s)" % (room_id, purge_id)
|
||||||
|
)
|
||||||
|
|
||||||
|
# We want to purge everything, including local events, and to run the purge in
|
||||||
|
# the background so that it's not blocking any other operation apart from
|
||||||
|
# other purges in the same room.
|
||||||
|
run_as_background_process(
|
||||||
|
"_purge_history", self._purge_history, purge_id, room_id, token, True,
|
||||||
|
)
|
||||||
|
|
||||||
def start_purge_history(self, room_id, token, delete_local_events=False):
|
def start_purge_history(self, room_id, token, delete_local_events=False):
|
||||||
"""Start off a history purge on a room.
|
"""Start off a history purge on a room.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -266,7 +266,7 @@ class RegistrationHandler(BaseHandler):
|
||||||
}
|
}
|
||||||
|
|
||||||
# Bind email to new account
|
# Bind email to new account
|
||||||
yield self._register_email_threepid(user_id, threepid_dict, None, False)
|
yield self._register_email_threepid(user_id, threepid_dict, None)
|
||||||
|
|
||||||
return user_id
|
return user_id
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
# Copyright 2014 - 2016 OpenMarket Ltd
|
# Copyright 2014 - 2016 OpenMarket Ltd
|
||||||
# Copyright 2018 New Vector Ltd
|
# Copyright 2018-2019 New Vector Ltd
|
||||||
|
# Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||||
#
|
#
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
# you may not use this file except in compliance with the License.
|
# you may not use this file except in compliance with the License.
|
||||||
|
|
@ -198,21 +199,21 @@ class RoomCreationHandler(BaseHandler):
|
||||||
# finally, shut down the PLs in the old room, and update them in the new
|
# finally, shut down the PLs in the old room, and update them in the new
|
||||||
# room.
|
# room.
|
||||||
yield self._update_upgraded_room_pls(
|
yield self._update_upgraded_room_pls(
|
||||||
requester, old_room_id, new_room_id, old_room_state
|
requester, old_room_id, new_room_id, old_room_state,
|
||||||
)
|
)
|
||||||
|
|
||||||
return new_room_id
|
return new_room_id
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def _update_upgraded_room_pls(
|
def _update_upgraded_room_pls(
|
||||||
self, requester, old_room_id, new_room_id, old_room_state
|
self, requester, old_room_id, new_room_id, old_room_state,
|
||||||
):
|
):
|
||||||
"""Send updated power levels in both rooms after an upgrade
|
"""Send updated power levels in both rooms after an upgrade
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
requester (synapse.types.Requester): the user requesting the upgrade
|
requester (synapse.types.Requester): the user requesting the upgrade
|
||||||
old_room_id (unicode): the id of the room to be replaced
|
old_room_id (str): the id of the room to be replaced
|
||||||
new_room_id (unicode): the id of the replacement room
|
new_room_id (str): the id of the replacement room
|
||||||
old_room_state (dict[tuple[str, str], str]): the state map for the old room
|
old_room_state (dict[tuple[str, str], str]): the state map for the old room
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
|
|
@ -298,7 +299,7 @@ class RoomCreationHandler(BaseHandler):
|
||||||
tombstone_event_id (unicode|str): the ID of the tombstone event in the old
|
tombstone_event_id (unicode|str): the ID of the tombstone event in the old
|
||||||
room.
|
room.
|
||||||
Returns:
|
Returns:
|
||||||
Deferred[None]
|
Deferred
|
||||||
"""
|
"""
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
|
|
@ -333,6 +334,7 @@ class RoomCreationHandler(BaseHandler):
|
||||||
(EventTypes.Encryption, ""),
|
(EventTypes.Encryption, ""),
|
||||||
(EventTypes.ServerACL, ""),
|
(EventTypes.ServerACL, ""),
|
||||||
(EventTypes.RelatedGroups, ""),
|
(EventTypes.RelatedGroups, ""),
|
||||||
|
(EventTypes.PowerLevels, ""),
|
||||||
)
|
)
|
||||||
|
|
||||||
old_room_state_ids = yield self.store.get_filtered_current_state_ids(
|
old_room_state_ids = yield self.store.get_filtered_current_state_ids(
|
||||||
|
|
@ -346,6 +348,31 @@ class RoomCreationHandler(BaseHandler):
|
||||||
if old_event:
|
if old_event:
|
||||||
initial_state[k] = old_event.content
|
initial_state[k] = old_event.content
|
||||||
|
|
||||||
|
# Resolve the minimum power level required to send any state event
|
||||||
|
# We will give the upgrading user this power level temporarily (if necessary) such that
|
||||||
|
# they are able to copy all of the state events over, then revert them back to their
|
||||||
|
# original power level afterwards in _update_upgraded_room_pls
|
||||||
|
|
||||||
|
# Copy over user power levels now as this will not be possible with >100PL users once
|
||||||
|
# the room has been created
|
||||||
|
|
||||||
|
power_levels = initial_state[(EventTypes.PowerLevels, "")]
|
||||||
|
|
||||||
|
# Calculate the minimum power level needed to clone the room
|
||||||
|
event_power_levels = power_levels.get("events", {})
|
||||||
|
state_default = power_levels.get("state_default", 0)
|
||||||
|
ban = power_levels.get("ban")
|
||||||
|
needed_power_level = max(state_default, ban, max(event_power_levels.values()))
|
||||||
|
|
||||||
|
# Raise the requester's power level in the new room if necessary
|
||||||
|
current_power_level = power_levels["users"][requester.user.to_string()]
|
||||||
|
if current_power_level < needed_power_level:
|
||||||
|
# Assign this power level to the requester
|
||||||
|
power_levels["users"][requester.user.to_string()] = needed_power_level
|
||||||
|
|
||||||
|
# Set the power levels to the modified state
|
||||||
|
initial_state[(EventTypes.PowerLevels, "")] = power_levels
|
||||||
|
|
||||||
yield self._send_events_for_new_room(
|
yield self._send_events_for_new_room(
|
||||||
requester,
|
requester,
|
||||||
new_room_id,
|
new_room_id,
|
||||||
|
|
@ -874,6 +901,10 @@ class RoomContextHandler(object):
|
||||||
room_id, event_id, before_limit, after_limit, event_filter
|
room_id, event_id, before_limit, after_limit, event_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if event_filter:
|
||||||
|
results["events_before"] = event_filter.filter(results["events_before"])
|
||||||
|
results["events_after"] = event_filter.filter(results["events_after"])
|
||||||
|
|
||||||
results["events_before"] = yield filter_evts(results["events_before"])
|
results["events_before"] = yield filter_evts(results["events_before"])
|
||||||
results["events_after"] = yield filter_evts(results["events_after"])
|
results["events_after"] = yield filter_evts(results["events_after"])
|
||||||
results["event"] = event
|
results["event"] = event
|
||||||
|
|
@ -902,7 +933,12 @@ class RoomContextHandler(object):
|
||||||
state = yield self.state_store.get_state_for_events(
|
state = yield self.state_store.get_state_for_events(
|
||||||
[last_event_id], state_filter=state_filter
|
[last_event_id], state_filter=state_filter
|
||||||
)
|
)
|
||||||
results["state"] = list(state[last_event_id].values())
|
|
||||||
|
state_events = list(state[last_event_id].values())
|
||||||
|
if event_filter:
|
||||||
|
state_events = event_filter.filter(state_events)
|
||||||
|
|
||||||
|
results["state"] = state_events
|
||||||
|
|
||||||
# We use a dummy token here as we only care about the room portion of
|
# We use a dummy token here as we only care about the room portion of
|
||||||
# the token, which we replace.
|
# the token, which we replace.
|
||||||
|
|
|
||||||
|
|
@ -94,7 +94,9 @@ class RoomMemberHandler(object):
|
||||||
raise NotImplementedError()
|
raise NotImplementedError()
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
def _remote_reject_invite(self, requester, remote_room_hosts, room_id, target):
|
def _remote_reject_invite(
|
||||||
|
self, requester, remote_room_hosts, room_id, target, content
|
||||||
|
):
|
||||||
"""Attempt to reject an invite for a room this server is not in. If we
|
"""Attempt to reject an invite for a room this server is not in. If we
|
||||||
fail to do so we locally mark the invite as rejected.
|
fail to do so we locally mark the invite as rejected.
|
||||||
|
|
||||||
|
|
@ -104,6 +106,7 @@ class RoomMemberHandler(object):
|
||||||
reject invite
|
reject invite
|
||||||
room_id (str)
|
room_id (str)
|
||||||
target (UserID): The user rejecting the invite
|
target (UserID): The user rejecting the invite
|
||||||
|
content (dict): The content for the rejection event
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Deferred[dict]: A dictionary to be returned to the client, may
|
Deferred[dict]: A dictionary to be returned to the client, may
|
||||||
|
|
@ -471,7 +474,7 @@ class RoomMemberHandler(object):
|
||||||
# send the rejection to the inviter's HS.
|
# send the rejection to the inviter's HS.
|
||||||
remote_room_hosts = remote_room_hosts + [inviter.domain]
|
remote_room_hosts = remote_room_hosts + [inviter.domain]
|
||||||
res = yield self._remote_reject_invite(
|
res = yield self._remote_reject_invite(
|
||||||
requester, remote_room_hosts, room_id, target
|
requester, remote_room_hosts, room_id, target, content,
|
||||||
)
|
)
|
||||||
return res
|
return res
|
||||||
|
|
||||||
|
|
@ -971,13 +974,15 @@ class RoomMemberMasterHandler(RoomMemberHandler):
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@defer.inlineCallbacks
|
||||||
def _remote_reject_invite(self, requester, remote_room_hosts, room_id, target):
|
def _remote_reject_invite(
|
||||||
|
self, requester, remote_room_hosts, room_id, target, content
|
||||||
|
):
|
||||||
"""Implements RoomMemberHandler._remote_reject_invite
|
"""Implements RoomMemberHandler._remote_reject_invite
|
||||||
"""
|
"""
|
||||||
fed_handler = self.federation_handler
|
fed_handler = self.federation_handler
|
||||||
try:
|
try:
|
||||||
ret = yield fed_handler.do_remotely_reject_invite(
|
ret = yield fed_handler.do_remotely_reject_invite(
|
||||||
remote_room_hosts, room_id, target.to_string()
|
remote_room_hosts, room_id, target.to_string(), content=content,
|
||||||
)
|
)
|
||||||
return ret
|
return ret
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|
|
||||||
|
|
@ -55,7 +55,9 @@ class RoomMemberWorkerHandler(RoomMemberHandler):
|
||||||
|
|
||||||
return ret
|
return ret
|
||||||
|
|
||||||
def _remote_reject_invite(self, requester, remote_room_hosts, room_id, target):
|
def _remote_reject_invite(
|
||||||
|
self, requester, remote_room_hosts, room_id, target, content
|
||||||
|
):
|
||||||
"""Implements RoomMemberHandler._remote_reject_invite
|
"""Implements RoomMemberHandler._remote_reject_invite
|
||||||
"""
|
"""
|
||||||
return self._remote_reject_client(
|
return self._remote_reject_client(
|
||||||
|
|
@ -63,6 +65,7 @@ class RoomMemberWorkerHandler(RoomMemberHandler):
|
||||||
remote_room_hosts=remote_room_hosts,
|
remote_room_hosts=remote_room_hosts,
|
||||||
room_id=room_id,
|
room_id=room_id,
|
||||||
user_id=target.to_string(),
|
user_id=target.to_string(),
|
||||||
|
content=content,
|
||||||
)
|
)
|
||||||
|
|
||||||
def _user_joined_room(self, target, room_id):
|
def _user_joined_room(self, target, room_id):
|
||||||
|
|
|
||||||
|
|
@ -22,8 +22,6 @@ from six import iteritems, itervalues
|
||||||
|
|
||||||
from prometheus_client import Counter
|
from prometheus_client import Counter
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.constants import EventTypes, Membership
|
from synapse.api.constants import EventTypes, Membership
|
||||||
from synapse.logging.context import LoggingContext
|
from synapse.logging.context import LoggingContext
|
||||||
from synapse.push.clientformat import format_push_rules_for_user
|
from synapse.push.clientformat import format_push_rules_for_user
|
||||||
|
|
@ -241,8 +239,7 @@ class SyncHandler(object):
|
||||||
expiry_ms=LAZY_LOADED_MEMBERS_CACHE_MAX_AGE,
|
expiry_ms=LAZY_LOADED_MEMBERS_CACHE_MAX_AGE,
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def wait_for_sync_for_user(
|
||||||
def wait_for_sync_for_user(
|
|
||||||
self, sync_config, since_token=None, timeout=0, full_state=False
|
self, sync_config, since_token=None, timeout=0, full_state=False
|
||||||
):
|
):
|
||||||
"""Get the sync for a client if we have new data for it now. Otherwise
|
"""Get the sync for a client if we have new data for it now. Otherwise
|
||||||
|
|
@ -255,9 +252,9 @@ class SyncHandler(object):
|
||||||
# not been exceeded (if not part of the group by this point, almost certain
|
# not been exceeded (if not part of the group by this point, almost certain
|
||||||
# auth_blocking will occur)
|
# auth_blocking will occur)
|
||||||
user_id = sync_config.user.to_string()
|
user_id = sync_config.user.to_string()
|
||||||
yield self.auth.check_auth_blocking(user_id)
|
await self.auth.check_auth_blocking(user_id)
|
||||||
|
|
||||||
res = yield self.response_cache.wrap(
|
res = await self.response_cache.wrap(
|
||||||
sync_config.request_key,
|
sync_config.request_key,
|
||||||
self._wait_for_sync_for_user,
|
self._wait_for_sync_for_user,
|
||||||
sync_config,
|
sync_config,
|
||||||
|
|
@ -267,8 +264,9 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
return res
|
return res
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _wait_for_sync_for_user(
|
||||||
def _wait_for_sync_for_user(self, sync_config, since_token, timeout, full_state):
|
self, sync_config, since_token, timeout, full_state
|
||||||
|
):
|
||||||
if since_token is None:
|
if since_token is None:
|
||||||
sync_type = "initial_sync"
|
sync_type = "initial_sync"
|
||||||
elif full_state:
|
elif full_state:
|
||||||
|
|
@ -283,7 +281,7 @@ class SyncHandler(object):
|
||||||
if timeout == 0 or since_token is None or full_state:
|
if timeout == 0 or since_token is None or full_state:
|
||||||
# we are going to return immediately, so don't bother calling
|
# we are going to return immediately, so don't bother calling
|
||||||
# notifier.wait_for_events.
|
# notifier.wait_for_events.
|
||||||
result = yield self.current_sync_for_user(
|
result = await self.current_sync_for_user(
|
||||||
sync_config, since_token, full_state=full_state
|
sync_config, since_token, full_state=full_state
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
|
|
@ -291,7 +289,7 @@ class SyncHandler(object):
|
||||||
def current_sync_callback(before_token, after_token):
|
def current_sync_callback(before_token, after_token):
|
||||||
return self.current_sync_for_user(sync_config, since_token)
|
return self.current_sync_for_user(sync_config, since_token)
|
||||||
|
|
||||||
result = yield self.notifier.wait_for_events(
|
result = await self.notifier.wait_for_events(
|
||||||
sync_config.user.to_string(),
|
sync_config.user.to_string(),
|
||||||
timeout,
|
timeout,
|
||||||
current_sync_callback,
|
current_sync_callback,
|
||||||
|
|
@ -314,15 +312,13 @@ class SyncHandler(object):
|
||||||
"""
|
"""
|
||||||
return self.generate_sync_result(sync_config, since_token, full_state)
|
return self.generate_sync_result(sync_config, since_token, full_state)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def push_rules_for_user(self, user):
|
||||||
def push_rules_for_user(self, user):
|
|
||||||
user_id = user.to_string()
|
user_id = user.to_string()
|
||||||
rules = yield self.store.get_push_rules_for_user(user_id)
|
rules = await self.store.get_push_rules_for_user(user_id)
|
||||||
rules = format_push_rules_for_user(user, rules)
|
rules = format_push_rules_for_user(user, rules)
|
||||||
return rules
|
return rules
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def ephemeral_by_room(self, sync_result_builder, now_token, since_token=None):
|
||||||
def ephemeral_by_room(self, sync_result_builder, now_token, since_token=None):
|
|
||||||
"""Get the ephemeral events for each room the user is in
|
"""Get the ephemeral events for each room the user is in
|
||||||
Args:
|
Args:
|
||||||
sync_result_builder(SyncResultBuilder)
|
sync_result_builder(SyncResultBuilder)
|
||||||
|
|
@ -343,7 +339,7 @@ class SyncHandler(object):
|
||||||
room_ids = sync_result_builder.joined_room_ids
|
room_ids = sync_result_builder.joined_room_ids
|
||||||
|
|
||||||
typing_source = self.event_sources.sources["typing"]
|
typing_source = self.event_sources.sources["typing"]
|
||||||
typing, typing_key = yield typing_source.get_new_events(
|
typing, typing_key = await typing_source.get_new_events(
|
||||||
user=sync_config.user,
|
user=sync_config.user,
|
||||||
from_key=typing_key,
|
from_key=typing_key,
|
||||||
limit=sync_config.filter_collection.ephemeral_limit(),
|
limit=sync_config.filter_collection.ephemeral_limit(),
|
||||||
|
|
@ -365,7 +361,7 @@ class SyncHandler(object):
|
||||||
receipt_key = since_token.receipt_key if since_token else "0"
|
receipt_key = since_token.receipt_key if since_token else "0"
|
||||||
|
|
||||||
receipt_source = self.event_sources.sources["receipt"]
|
receipt_source = self.event_sources.sources["receipt"]
|
||||||
receipts, receipt_key = yield receipt_source.get_new_events(
|
receipts, receipt_key = await receipt_source.get_new_events(
|
||||||
user=sync_config.user,
|
user=sync_config.user,
|
||||||
from_key=receipt_key,
|
from_key=receipt_key,
|
||||||
limit=sync_config.filter_collection.ephemeral_limit(),
|
limit=sync_config.filter_collection.ephemeral_limit(),
|
||||||
|
|
@ -382,8 +378,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
return now_token, ephemeral_by_room
|
return now_token, ephemeral_by_room
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _load_filtered_recents(
|
||||||
def _load_filtered_recents(
|
|
||||||
self,
|
self,
|
||||||
room_id,
|
room_id,
|
||||||
sync_config,
|
sync_config,
|
||||||
|
|
@ -415,10 +410,10 @@ class SyncHandler(object):
|
||||||
# ensure that we always include current state in the timeline
|
# ensure that we always include current state in the timeline
|
||||||
current_state_ids = frozenset()
|
current_state_ids = frozenset()
|
||||||
if any(e.is_state() for e in recents):
|
if any(e.is_state() for e in recents):
|
||||||
current_state_ids = yield self.state.get_current_state_ids(room_id)
|
current_state_ids = await self.state.get_current_state_ids(room_id)
|
||||||
current_state_ids = frozenset(itervalues(current_state_ids))
|
current_state_ids = frozenset(itervalues(current_state_ids))
|
||||||
|
|
||||||
recents = yield filter_events_for_client(
|
recents = await filter_events_for_client(
|
||||||
self.storage,
|
self.storage,
|
||||||
sync_config.user.to_string(),
|
sync_config.user.to_string(),
|
||||||
recents,
|
recents,
|
||||||
|
|
@ -449,14 +444,14 @@ class SyncHandler(object):
|
||||||
# Otherwise, we want to return the last N events in the room
|
# Otherwise, we want to return the last N events in the room
|
||||||
# in toplogical ordering.
|
# in toplogical ordering.
|
||||||
if since_key:
|
if since_key:
|
||||||
events, end_key = yield self.store.get_room_events_stream_for_room(
|
events, end_key = await self.store.get_room_events_stream_for_room(
|
||||||
room_id,
|
room_id,
|
||||||
limit=load_limit + 1,
|
limit=load_limit + 1,
|
||||||
from_key=since_key,
|
from_key=since_key,
|
||||||
to_key=end_key,
|
to_key=end_key,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
events, end_key = yield self.store.get_recent_events_for_room(
|
events, end_key = await self.store.get_recent_events_for_room(
|
||||||
room_id, limit=load_limit + 1, end_token=end_key
|
room_id, limit=load_limit + 1, end_token=end_key
|
||||||
)
|
)
|
||||||
loaded_recents = sync_config.filter_collection.filter_room_timeline(
|
loaded_recents = sync_config.filter_collection.filter_room_timeline(
|
||||||
|
|
@ -468,10 +463,10 @@ class SyncHandler(object):
|
||||||
# ensure that we always include current state in the timeline
|
# ensure that we always include current state in the timeline
|
||||||
current_state_ids = frozenset()
|
current_state_ids = frozenset()
|
||||||
if any(e.is_state() for e in loaded_recents):
|
if any(e.is_state() for e in loaded_recents):
|
||||||
current_state_ids = yield self.state.get_current_state_ids(room_id)
|
current_state_ids = await self.state.get_current_state_ids(room_id)
|
||||||
current_state_ids = frozenset(itervalues(current_state_ids))
|
current_state_ids = frozenset(itervalues(current_state_ids))
|
||||||
|
|
||||||
loaded_recents = yield filter_events_for_client(
|
loaded_recents = await filter_events_for_client(
|
||||||
self.storage,
|
self.storage,
|
||||||
sync_config.user.to_string(),
|
sync_config.user.to_string(),
|
||||||
loaded_recents,
|
loaded_recents,
|
||||||
|
|
@ -498,8 +493,7 @@ class SyncHandler(object):
|
||||||
limited=limited or newly_joined_room,
|
limited=limited or newly_joined_room,
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def get_state_after_event(self, event, state_filter=StateFilter.all()):
|
||||||
def get_state_after_event(self, event, state_filter=StateFilter.all()):
|
|
||||||
"""
|
"""
|
||||||
Get the room state after the given event
|
Get the room state after the given event
|
||||||
|
|
||||||
|
|
@ -511,7 +505,7 @@ class SyncHandler(object):
|
||||||
Returns:
|
Returns:
|
||||||
A Deferred map from ((type, state_key)->Event)
|
A Deferred map from ((type, state_key)->Event)
|
||||||
"""
|
"""
|
||||||
state_ids = yield self.state_store.get_state_ids_for_event(
|
state_ids = await self.state_store.get_state_ids_for_event(
|
||||||
event.event_id, state_filter=state_filter
|
event.event_id, state_filter=state_filter
|
||||||
)
|
)
|
||||||
if event.is_state():
|
if event.is_state():
|
||||||
|
|
@ -519,8 +513,9 @@ class SyncHandler(object):
|
||||||
state_ids[(event.type, event.state_key)] = event.event_id
|
state_ids[(event.type, event.state_key)] = event.event_id
|
||||||
return state_ids
|
return state_ids
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def get_state_at(
|
||||||
def get_state_at(self, room_id, stream_position, state_filter=StateFilter.all()):
|
self, room_id, stream_position, state_filter=StateFilter.all()
|
||||||
|
):
|
||||||
""" Get the room state at a particular stream position
|
""" Get the room state at a particular stream position
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -536,13 +531,13 @@ class SyncHandler(object):
|
||||||
# get_recent_events_for_room operates by topo ordering. This therefore
|
# get_recent_events_for_room operates by topo ordering. This therefore
|
||||||
# does not reliably give you the state at the given stream position.
|
# does not reliably give you the state at the given stream position.
|
||||||
# (https://github.com/matrix-org/synapse/issues/3305)
|
# (https://github.com/matrix-org/synapse/issues/3305)
|
||||||
last_events, _ = yield self.store.get_recent_events_for_room(
|
last_events, _ = await self.store.get_recent_events_for_room(
|
||||||
room_id, end_token=stream_position.room_key, limit=1
|
room_id, end_token=stream_position.room_key, limit=1
|
||||||
)
|
)
|
||||||
|
|
||||||
if last_events:
|
if last_events:
|
||||||
last_event = last_events[-1]
|
last_event = last_events[-1]
|
||||||
state = yield self.get_state_after_event(
|
state = await self.get_state_after_event(
|
||||||
last_event, state_filter=state_filter
|
last_event, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -551,8 +546,7 @@ class SyncHandler(object):
|
||||||
state = {}
|
state = {}
|
||||||
return state
|
return state
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def compute_summary(self, room_id, sync_config, batch, state, now_token):
|
||||||
def compute_summary(self, room_id, sync_config, batch, state, now_token):
|
|
||||||
""" Works out a room summary block for this room, summarising the number
|
""" Works out a room summary block for this room, summarising the number
|
||||||
of joined members in the room, and providing the 'hero' members if the
|
of joined members in the room, and providing the 'hero' members if the
|
||||||
room has no name so clients can consistently name rooms. Also adds
|
room has no name so clients can consistently name rooms. Also adds
|
||||||
|
|
@ -574,7 +568,7 @@ class SyncHandler(object):
|
||||||
# FIXME: we could/should get this from room_stats when matthew/stats lands
|
# FIXME: we could/should get this from room_stats when matthew/stats lands
|
||||||
|
|
||||||
# FIXME: this promulgates https://github.com/matrix-org/synapse/issues/3305
|
# FIXME: this promulgates https://github.com/matrix-org/synapse/issues/3305
|
||||||
last_events, _ = yield self.store.get_recent_event_ids_for_room(
|
last_events, _ = await self.store.get_recent_event_ids_for_room(
|
||||||
room_id, end_token=now_token.room_key, limit=1
|
room_id, end_token=now_token.room_key, limit=1
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -582,7 +576,7 @@ class SyncHandler(object):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
last_event = last_events[-1]
|
last_event = last_events[-1]
|
||||||
state_ids = yield self.state_store.get_state_ids_for_event(
|
state_ids = await self.state_store.get_state_ids_for_event(
|
||||||
last_event.event_id,
|
last_event.event_id,
|
||||||
state_filter=StateFilter.from_types(
|
state_filter=StateFilter.from_types(
|
||||||
[(EventTypes.Name, ""), (EventTypes.CanonicalAlias, "")]
|
[(EventTypes.Name, ""), (EventTypes.CanonicalAlias, "")]
|
||||||
|
|
@ -590,7 +584,7 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
|
|
||||||
# this is heavily cached, thus: fast.
|
# this is heavily cached, thus: fast.
|
||||||
details = yield self.store.get_room_summary(room_id)
|
details = await self.store.get_room_summary(room_id)
|
||||||
|
|
||||||
name_id = state_ids.get((EventTypes.Name, ""))
|
name_id = state_ids.get((EventTypes.Name, ""))
|
||||||
canonical_alias_id = state_ids.get((EventTypes.CanonicalAlias, ""))
|
canonical_alias_id = state_ids.get((EventTypes.CanonicalAlias, ""))
|
||||||
|
|
@ -608,12 +602,12 @@ class SyncHandler(object):
|
||||||
# calculating heroes. Empty strings are falsey, so we check
|
# calculating heroes. Empty strings are falsey, so we check
|
||||||
# for the "name" value and default to an empty string.
|
# for the "name" value and default to an empty string.
|
||||||
if name_id:
|
if name_id:
|
||||||
name = yield self.store.get_event(name_id, allow_none=True)
|
name = await self.store.get_event(name_id, allow_none=True)
|
||||||
if name and name.content.get("name"):
|
if name and name.content.get("name"):
|
||||||
return summary
|
return summary
|
||||||
|
|
||||||
if canonical_alias_id:
|
if canonical_alias_id:
|
||||||
canonical_alias = yield self.store.get_event(
|
canonical_alias = await self.store.get_event(
|
||||||
canonical_alias_id, allow_none=True
|
canonical_alias_id, allow_none=True
|
||||||
)
|
)
|
||||||
if canonical_alias and canonical_alias.content.get("alias"):
|
if canonical_alias and canonical_alias.content.get("alias"):
|
||||||
|
|
@ -678,7 +672,7 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
|
|
||||||
missing_hero_state = yield self.store.get_events(missing_hero_event_ids)
|
missing_hero_state = await self.store.get_events(missing_hero_event_ids)
|
||||||
missing_hero_state = missing_hero_state.values()
|
missing_hero_state = missing_hero_state.values()
|
||||||
|
|
||||||
for s in missing_hero_state:
|
for s in missing_hero_state:
|
||||||
|
|
@ -697,8 +691,7 @@ class SyncHandler(object):
|
||||||
logger.debug("found LruCache for %r", cache_key)
|
logger.debug("found LruCache for %r", cache_key)
|
||||||
return cache
|
return cache
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def compute_state_delta(
|
||||||
def compute_state_delta(
|
|
||||||
self, room_id, batch, sync_config, since_token, now_token, full_state
|
self, room_id, batch, sync_config, since_token, now_token, full_state
|
||||||
):
|
):
|
||||||
""" Works out the difference in state between the start of the timeline
|
""" Works out the difference in state between the start of the timeline
|
||||||
|
|
@ -759,16 +752,16 @@ class SyncHandler(object):
|
||||||
|
|
||||||
if full_state:
|
if full_state:
|
||||||
if batch:
|
if batch:
|
||||||
current_state_ids = yield self.state_store.get_state_ids_for_event(
|
current_state_ids = await self.state_store.get_state_ids_for_event(
|
||||||
batch.events[-1].event_id, state_filter=state_filter
|
batch.events[-1].event_id, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
state_ids = yield self.state_store.get_state_ids_for_event(
|
state_ids = await self.state_store.get_state_ids_for_event(
|
||||||
batch.events[0].event_id, state_filter=state_filter
|
batch.events[0].event_id, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
else:
|
else:
|
||||||
current_state_ids = yield self.get_state_at(
|
current_state_ids = await self.get_state_at(
|
||||||
room_id, stream_position=now_token, state_filter=state_filter
|
room_id, stream_position=now_token, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -783,13 +776,13 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
elif batch.limited:
|
elif batch.limited:
|
||||||
if batch:
|
if batch:
|
||||||
state_at_timeline_start = yield self.state_store.get_state_ids_for_event(
|
state_at_timeline_start = await self.state_store.get_state_ids_for_event(
|
||||||
batch.events[0].event_id, state_filter=state_filter
|
batch.events[0].event_id, state_filter=state_filter
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# We can get here if the user has ignored the senders of all
|
# We can get here if the user has ignored the senders of all
|
||||||
# the recent events.
|
# the recent events.
|
||||||
state_at_timeline_start = yield self.get_state_at(
|
state_at_timeline_start = await self.get_state_at(
|
||||||
room_id, stream_position=now_token, state_filter=state_filter
|
room_id, stream_position=now_token, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -807,19 +800,19 @@ class SyncHandler(object):
|
||||||
# about them).
|
# about them).
|
||||||
state_filter = StateFilter.all()
|
state_filter = StateFilter.all()
|
||||||
|
|
||||||
state_at_previous_sync = yield self.get_state_at(
|
state_at_previous_sync = await self.get_state_at(
|
||||||
room_id, stream_position=since_token, state_filter=state_filter
|
room_id, stream_position=since_token, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
if batch:
|
if batch:
|
||||||
current_state_ids = yield self.state_store.get_state_ids_for_event(
|
current_state_ids = await self.state_store.get_state_ids_for_event(
|
||||||
batch.events[-1].event_id, state_filter=state_filter
|
batch.events[-1].event_id, state_filter=state_filter
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# Its not clear how we get here, but empirically we do
|
# Its not clear how we get here, but empirically we do
|
||||||
# (#5407). Logging has been added elsewhere to try and
|
# (#5407). Logging has been added elsewhere to try and
|
||||||
# figure out where this state comes from.
|
# figure out where this state comes from.
|
||||||
current_state_ids = yield self.get_state_at(
|
current_state_ids = await self.get_state_at(
|
||||||
room_id, stream_position=now_token, state_filter=state_filter
|
room_id, stream_position=now_token, state_filter=state_filter
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -843,7 +836,7 @@ class SyncHandler(object):
|
||||||
# So we fish out all the member events corresponding to the
|
# So we fish out all the member events corresponding to the
|
||||||
# timeline here, and then dedupe any redundant ones below.
|
# timeline here, and then dedupe any redundant ones below.
|
||||||
|
|
||||||
state_ids = yield self.state_store.get_state_ids_for_event(
|
state_ids = await self.state_store.get_state_ids_for_event(
|
||||||
batch.events[0].event_id,
|
batch.events[0].event_id,
|
||||||
# we only want members!
|
# we only want members!
|
||||||
state_filter=StateFilter.from_types(
|
state_filter=StateFilter.from_types(
|
||||||
|
|
@ -883,7 +876,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
state = {}
|
state = {}
|
||||||
if state_ids:
|
if state_ids:
|
||||||
state = yield self.store.get_events(list(state_ids.values()))
|
state = await self.store.get_events(list(state_ids.values()))
|
||||||
|
|
||||||
return {
|
return {
|
||||||
(e.type, e.state_key): e
|
(e.type, e.state_key): e
|
||||||
|
|
@ -892,10 +885,9 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def unread_notifs_for_room_id(self, room_id, sync_config):
|
||||||
def unread_notifs_for_room_id(self, room_id, sync_config):
|
|
||||||
with Measure(self.clock, "unread_notifs_for_room_id"):
|
with Measure(self.clock, "unread_notifs_for_room_id"):
|
||||||
last_unread_event_id = yield self.store.get_last_receipt_event_id_for_user(
|
last_unread_event_id = await self.store.get_last_receipt_event_id_for_user(
|
||||||
user_id=sync_config.user.to_string(),
|
user_id=sync_config.user.to_string(),
|
||||||
room_id=room_id,
|
room_id=room_id,
|
||||||
receipt_type="m.read",
|
receipt_type="m.read",
|
||||||
|
|
@ -903,7 +895,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
notifs = []
|
notifs = []
|
||||||
if last_unread_event_id:
|
if last_unread_event_id:
|
||||||
notifs = yield self.store.get_unread_event_push_actions_by_room_for_user(
|
notifs = await self.store.get_unread_event_push_actions_by_room_for_user(
|
||||||
room_id, sync_config.user.to_string(), last_unread_event_id
|
room_id, sync_config.user.to_string(), last_unread_event_id
|
||||||
)
|
)
|
||||||
return notifs
|
return notifs
|
||||||
|
|
@ -912,8 +904,9 @@ class SyncHandler(object):
|
||||||
# count is whatever it was last time.
|
# count is whatever it was last time.
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def generate_sync_result(
|
||||||
def generate_sync_result(self, sync_config, since_token=None, full_state=False):
|
self, sync_config, since_token=None, full_state=False
|
||||||
|
):
|
||||||
"""Generates a sync result.
|
"""Generates a sync result.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -928,7 +921,7 @@ class SyncHandler(object):
|
||||||
# this is due to some of the underlying streams not supporting the ability
|
# this is due to some of the underlying streams not supporting the ability
|
||||||
# to query up to a given point.
|
# to query up to a given point.
|
||||||
# Always use the `now_token` in `SyncResultBuilder`
|
# Always use the `now_token` in `SyncResultBuilder`
|
||||||
now_token = yield self.event_sources.get_current_token()
|
now_token = await self.event_sources.get_current_token()
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"Calculating sync response for %r between %s and %s",
|
"Calculating sync response for %r between %s and %s",
|
||||||
|
|
@ -944,10 +937,9 @@ class SyncHandler(object):
|
||||||
# See https://github.com/matrix-org/matrix-doc/issues/1144
|
# See https://github.com/matrix-org/matrix-doc/issues/1144
|
||||||
raise NotImplementedError()
|
raise NotImplementedError()
|
||||||
else:
|
else:
|
||||||
joined_room_ids = yield self.get_rooms_for_user_at(
|
joined_room_ids = await self.get_rooms_for_user_at(
|
||||||
user_id, now_token.room_stream_id
|
user_id, now_token.room_stream_id
|
||||||
)
|
)
|
||||||
|
|
||||||
sync_result_builder = SyncResultBuilder(
|
sync_result_builder = SyncResultBuilder(
|
||||||
sync_config,
|
sync_config,
|
||||||
full_state,
|
full_state,
|
||||||
|
|
@ -956,11 +948,11 @@ class SyncHandler(object):
|
||||||
joined_room_ids=joined_room_ids,
|
joined_room_ids=joined_room_ids,
|
||||||
)
|
)
|
||||||
|
|
||||||
account_data_by_room = yield self._generate_sync_entry_for_account_data(
|
account_data_by_room = await self._generate_sync_entry_for_account_data(
|
||||||
sync_result_builder
|
sync_result_builder
|
||||||
)
|
)
|
||||||
|
|
||||||
res = yield self._generate_sync_entry_for_rooms(
|
res = await self._generate_sync_entry_for_rooms(
|
||||||
sync_result_builder, account_data_by_room
|
sync_result_builder, account_data_by_room
|
||||||
)
|
)
|
||||||
newly_joined_rooms, newly_joined_or_invited_users, _, _ = res
|
newly_joined_rooms, newly_joined_or_invited_users, _, _ = res
|
||||||
|
|
@ -970,13 +962,13 @@ class SyncHandler(object):
|
||||||
since_token is None and sync_config.filter_collection.blocks_all_presence()
|
since_token is None and sync_config.filter_collection.blocks_all_presence()
|
||||||
)
|
)
|
||||||
if self.hs_config.use_presence and not block_all_presence_data:
|
if self.hs_config.use_presence and not block_all_presence_data:
|
||||||
yield self._generate_sync_entry_for_presence(
|
await self._generate_sync_entry_for_presence(
|
||||||
sync_result_builder, newly_joined_rooms, newly_joined_or_invited_users
|
sync_result_builder, newly_joined_rooms, newly_joined_or_invited_users
|
||||||
)
|
)
|
||||||
|
|
||||||
yield self._generate_sync_entry_for_to_device(sync_result_builder)
|
await self._generate_sync_entry_for_to_device(sync_result_builder)
|
||||||
|
|
||||||
device_lists = yield self._generate_sync_entry_for_device_list(
|
device_lists = await self._generate_sync_entry_for_device_list(
|
||||||
sync_result_builder,
|
sync_result_builder,
|
||||||
newly_joined_rooms=newly_joined_rooms,
|
newly_joined_rooms=newly_joined_rooms,
|
||||||
newly_joined_or_invited_users=newly_joined_or_invited_users,
|
newly_joined_or_invited_users=newly_joined_or_invited_users,
|
||||||
|
|
@ -987,11 +979,11 @@ class SyncHandler(object):
|
||||||
device_id = sync_config.device_id
|
device_id = sync_config.device_id
|
||||||
one_time_key_counts = {}
|
one_time_key_counts = {}
|
||||||
if device_id:
|
if device_id:
|
||||||
one_time_key_counts = yield self.store.count_e2e_one_time_keys(
|
one_time_key_counts = await self.store.count_e2e_one_time_keys(
|
||||||
user_id, device_id
|
user_id, device_id
|
||||||
)
|
)
|
||||||
|
|
||||||
yield self._generate_sync_entry_for_groups(sync_result_builder)
|
await self._generate_sync_entry_for_groups(sync_result_builder)
|
||||||
|
|
||||||
# debug for https://github.com/matrix-org/synapse/issues/4422
|
# debug for https://github.com/matrix-org/synapse/issues/4422
|
||||||
for joined_room in sync_result_builder.joined:
|
for joined_room in sync_result_builder.joined:
|
||||||
|
|
@ -1015,18 +1007,17 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
|
|
||||||
@measure_func("_generate_sync_entry_for_groups")
|
@measure_func("_generate_sync_entry_for_groups")
|
||||||
@defer.inlineCallbacks
|
async def _generate_sync_entry_for_groups(self, sync_result_builder):
|
||||||
def _generate_sync_entry_for_groups(self, sync_result_builder):
|
|
||||||
user_id = sync_result_builder.sync_config.user.to_string()
|
user_id = sync_result_builder.sync_config.user.to_string()
|
||||||
since_token = sync_result_builder.since_token
|
since_token = sync_result_builder.since_token
|
||||||
now_token = sync_result_builder.now_token
|
now_token = sync_result_builder.now_token
|
||||||
|
|
||||||
if since_token and since_token.groups_key:
|
if since_token and since_token.groups_key:
|
||||||
results = yield self.store.get_groups_changes_for_user(
|
results = await self.store.get_groups_changes_for_user(
|
||||||
user_id, since_token.groups_key, now_token.groups_key
|
user_id, since_token.groups_key, now_token.groups_key
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
results = yield self.store.get_all_groups_for_user(
|
results = await self.store.get_all_groups_for_user(
|
||||||
user_id, now_token.groups_key
|
user_id, now_token.groups_key
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1059,8 +1050,7 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
|
|
||||||
@measure_func("_generate_sync_entry_for_device_list")
|
@measure_func("_generate_sync_entry_for_device_list")
|
||||||
@defer.inlineCallbacks
|
async def _generate_sync_entry_for_device_list(
|
||||||
def _generate_sync_entry_for_device_list(
|
|
||||||
self,
|
self,
|
||||||
sync_result_builder,
|
sync_result_builder,
|
||||||
newly_joined_rooms,
|
newly_joined_rooms,
|
||||||
|
|
@ -1108,32 +1098,32 @@ class SyncHandler(object):
|
||||||
# room with by looking at all users that have left a room plus users
|
# room with by looking at all users that have left a room plus users
|
||||||
# that were in a room we've left.
|
# that were in a room we've left.
|
||||||
|
|
||||||
users_who_share_room = yield self.store.get_users_who_share_room_with_user(
|
users_who_share_room = await self.store.get_users_who_share_room_with_user(
|
||||||
user_id
|
user_id
|
||||||
)
|
)
|
||||||
|
|
||||||
# Step 1a, check for changes in devices of users we share a room with
|
# Step 1a, check for changes in devices of users we share a room with
|
||||||
users_that_have_changed = yield self.store.get_users_whose_devices_changed(
|
users_that_have_changed = await self.store.get_users_whose_devices_changed(
|
||||||
since_token.device_list_key, users_who_share_room
|
since_token.device_list_key, users_who_share_room
|
||||||
)
|
)
|
||||||
|
|
||||||
# Step 1b, check for newly joined rooms
|
# Step 1b, check for newly joined rooms
|
||||||
for room_id in newly_joined_rooms:
|
for room_id in newly_joined_rooms:
|
||||||
joined_users = yield self.state.get_current_users_in_room(room_id)
|
joined_users = await self.state.get_current_users_in_room(room_id)
|
||||||
newly_joined_or_invited_users.update(joined_users)
|
newly_joined_or_invited_users.update(joined_users)
|
||||||
|
|
||||||
# TODO: Check that these users are actually new, i.e. either they
|
# TODO: Check that these users are actually new, i.e. either they
|
||||||
# weren't in the previous sync *or* they left and rejoined.
|
# weren't in the previous sync *or* they left and rejoined.
|
||||||
users_that_have_changed.update(newly_joined_or_invited_users)
|
users_that_have_changed.update(newly_joined_or_invited_users)
|
||||||
|
|
||||||
user_signatures_changed = yield self.store.get_users_whose_signatures_changed(
|
user_signatures_changed = await self.store.get_users_whose_signatures_changed(
|
||||||
user_id, since_token.device_list_key
|
user_id, since_token.device_list_key
|
||||||
)
|
)
|
||||||
users_that_have_changed.update(user_signatures_changed)
|
users_that_have_changed.update(user_signatures_changed)
|
||||||
|
|
||||||
# Now find users that we no longer track
|
# Now find users that we no longer track
|
||||||
for room_id in newly_left_rooms:
|
for room_id in newly_left_rooms:
|
||||||
left_users = yield self.state.get_current_users_in_room(room_id)
|
left_users = await self.state.get_current_users_in_room(room_id)
|
||||||
newly_left_users.update(left_users)
|
newly_left_users.update(left_users)
|
||||||
|
|
||||||
# Remove any users that we still share a room with.
|
# Remove any users that we still share a room with.
|
||||||
|
|
@ -1143,8 +1133,7 @@ class SyncHandler(object):
|
||||||
else:
|
else:
|
||||||
return DeviceLists(changed=[], left=[])
|
return DeviceLists(changed=[], left=[])
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _generate_sync_entry_for_to_device(self, sync_result_builder):
|
||||||
def _generate_sync_entry_for_to_device(self, sync_result_builder):
|
|
||||||
"""Generates the portion of the sync response. Populates
|
"""Generates the portion of the sync response. Populates
|
||||||
`sync_result_builder` with the result.
|
`sync_result_builder` with the result.
|
||||||
|
|
||||||
|
|
@ -1165,14 +1154,14 @@ class SyncHandler(object):
|
||||||
# We only delete messages when a new message comes in, but that's
|
# We only delete messages when a new message comes in, but that's
|
||||||
# fine so long as we delete them at some point.
|
# fine so long as we delete them at some point.
|
||||||
|
|
||||||
deleted = yield self.store.delete_messages_for_device(
|
deleted = await self.store.delete_messages_for_device(
|
||||||
user_id, device_id, since_stream_id
|
user_id, device_id, since_stream_id
|
||||||
)
|
)
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"Deleted %d to-device messages up to %d", deleted, since_stream_id
|
"Deleted %d to-device messages up to %d", deleted, since_stream_id
|
||||||
)
|
)
|
||||||
|
|
||||||
messages, stream_id = yield self.store.get_new_messages_for_device(
|
messages, stream_id = await self.store.get_new_messages_for_device(
|
||||||
user_id, device_id, since_stream_id, now_token.to_device_key
|
user_id, device_id, since_stream_id, now_token.to_device_key
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1190,8 +1179,7 @@ class SyncHandler(object):
|
||||||
else:
|
else:
|
||||||
sync_result_builder.to_device = []
|
sync_result_builder.to_device = []
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _generate_sync_entry_for_account_data(self, sync_result_builder):
|
||||||
def _generate_sync_entry_for_account_data(self, sync_result_builder):
|
|
||||||
"""Generates the account data portion of the sync response. Populates
|
"""Generates the account data portion of the sync response. Populates
|
||||||
`sync_result_builder` with the result.
|
`sync_result_builder` with the result.
|
||||||
|
|
||||||
|
|
@ -1209,25 +1197,25 @@ class SyncHandler(object):
|
||||||
(
|
(
|
||||||
account_data,
|
account_data,
|
||||||
account_data_by_room,
|
account_data_by_room,
|
||||||
) = yield self.store.get_updated_account_data_for_user(
|
) = await self.store.get_updated_account_data_for_user(
|
||||||
user_id, since_token.account_data_key
|
user_id, since_token.account_data_key
|
||||||
)
|
)
|
||||||
|
|
||||||
push_rules_changed = yield self.store.have_push_rules_changed_for_user(
|
push_rules_changed = await self.store.have_push_rules_changed_for_user(
|
||||||
user_id, int(since_token.push_rules_key)
|
user_id, int(since_token.push_rules_key)
|
||||||
)
|
)
|
||||||
|
|
||||||
if push_rules_changed:
|
if push_rules_changed:
|
||||||
account_data["m.push_rules"] = yield self.push_rules_for_user(
|
account_data["m.push_rules"] = await self.push_rules_for_user(
|
||||||
sync_config.user
|
sync_config.user
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
(
|
(
|
||||||
account_data,
|
account_data,
|
||||||
account_data_by_room,
|
account_data_by_room,
|
||||||
) = yield self.store.get_account_data_for_user(sync_config.user.to_string())
|
) = await self.store.get_account_data_for_user(sync_config.user.to_string())
|
||||||
|
|
||||||
account_data["m.push_rules"] = yield self.push_rules_for_user(
|
account_data["m.push_rules"] = await self.push_rules_for_user(
|
||||||
sync_config.user
|
sync_config.user
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1242,8 +1230,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
return account_data_by_room
|
return account_data_by_room
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _generate_sync_entry_for_presence(
|
||||||
def _generate_sync_entry_for_presence(
|
|
||||||
self, sync_result_builder, newly_joined_rooms, newly_joined_or_invited_users
|
self, sync_result_builder, newly_joined_rooms, newly_joined_or_invited_users
|
||||||
):
|
):
|
||||||
"""Generates the presence portion of the sync response. Populates the
|
"""Generates the presence portion of the sync response. Populates the
|
||||||
|
|
@ -1271,7 +1258,7 @@ class SyncHandler(object):
|
||||||
presence_key = None
|
presence_key = None
|
||||||
include_offline = False
|
include_offline = False
|
||||||
|
|
||||||
presence, presence_key = yield presence_source.get_new_events(
|
presence, presence_key = await presence_source.get_new_events(
|
||||||
user=user,
|
user=user,
|
||||||
from_key=presence_key,
|
from_key=presence_key,
|
||||||
is_guest=sync_config.is_guest,
|
is_guest=sync_config.is_guest,
|
||||||
|
|
@ -1283,12 +1270,12 @@ class SyncHandler(object):
|
||||||
|
|
||||||
extra_users_ids = set(newly_joined_or_invited_users)
|
extra_users_ids = set(newly_joined_or_invited_users)
|
||||||
for room_id in newly_joined_rooms:
|
for room_id in newly_joined_rooms:
|
||||||
users = yield self.state.get_current_users_in_room(room_id)
|
users = await self.state.get_current_users_in_room(room_id)
|
||||||
extra_users_ids.update(users)
|
extra_users_ids.update(users)
|
||||||
extra_users_ids.discard(user.to_string())
|
extra_users_ids.discard(user.to_string())
|
||||||
|
|
||||||
if extra_users_ids:
|
if extra_users_ids:
|
||||||
states = yield self.presence_handler.get_states(extra_users_ids)
|
states = await self.presence_handler.get_states(extra_users_ids)
|
||||||
presence.extend(states)
|
presence.extend(states)
|
||||||
|
|
||||||
# Deduplicate the presence entries so that there's at most one per user
|
# Deduplicate the presence entries so that there's at most one per user
|
||||||
|
|
@ -1298,8 +1285,9 @@ class SyncHandler(object):
|
||||||
|
|
||||||
sync_result_builder.presence = presence
|
sync_result_builder.presence = presence
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _generate_sync_entry_for_rooms(
|
||||||
def _generate_sync_entry_for_rooms(self, sync_result_builder, account_data_by_room):
|
self, sync_result_builder, account_data_by_room
|
||||||
|
):
|
||||||
"""Generates the rooms portion of the sync response. Populates the
|
"""Generates the rooms portion of the sync response. Populates the
|
||||||
`sync_result_builder` with the result.
|
`sync_result_builder` with the result.
|
||||||
|
|
||||||
|
|
@ -1321,7 +1309,7 @@ class SyncHandler(object):
|
||||||
if block_all_room_ephemeral:
|
if block_all_room_ephemeral:
|
||||||
ephemeral_by_room = {}
|
ephemeral_by_room = {}
|
||||||
else:
|
else:
|
||||||
now_token, ephemeral_by_room = yield self.ephemeral_by_room(
|
now_token, ephemeral_by_room = await self.ephemeral_by_room(
|
||||||
sync_result_builder,
|
sync_result_builder,
|
||||||
now_token=sync_result_builder.now_token,
|
now_token=sync_result_builder.now_token,
|
||||||
since_token=sync_result_builder.since_token,
|
since_token=sync_result_builder.since_token,
|
||||||
|
|
@ -1333,16 +1321,16 @@ class SyncHandler(object):
|
||||||
since_token = sync_result_builder.since_token
|
since_token = sync_result_builder.since_token
|
||||||
if not sync_result_builder.full_state:
|
if not sync_result_builder.full_state:
|
||||||
if since_token and not ephemeral_by_room and not account_data_by_room:
|
if since_token and not ephemeral_by_room and not account_data_by_room:
|
||||||
have_changed = yield self._have_rooms_changed(sync_result_builder)
|
have_changed = await self._have_rooms_changed(sync_result_builder)
|
||||||
if not have_changed:
|
if not have_changed:
|
||||||
tags_by_room = yield self.store.get_updated_tags(
|
tags_by_room = await self.store.get_updated_tags(
|
||||||
user_id, since_token.account_data_key
|
user_id, since_token.account_data_key
|
||||||
)
|
)
|
||||||
if not tags_by_room:
|
if not tags_by_room:
|
||||||
logger.debug("no-oping sync")
|
logger.debug("no-oping sync")
|
||||||
return [], [], [], []
|
return [], [], [], []
|
||||||
|
|
||||||
ignored_account_data = yield self.store.get_global_account_data_by_type_for_user(
|
ignored_account_data = await self.store.get_global_account_data_by_type_for_user(
|
||||||
"m.ignored_user_list", user_id=user_id
|
"m.ignored_user_list", user_id=user_id
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1352,18 +1340,18 @@ class SyncHandler(object):
|
||||||
ignored_users = frozenset()
|
ignored_users = frozenset()
|
||||||
|
|
||||||
if since_token:
|
if since_token:
|
||||||
res = yield self._get_rooms_changed(sync_result_builder, ignored_users)
|
res = await self._get_rooms_changed(sync_result_builder, ignored_users)
|
||||||
room_entries, invited, newly_joined_rooms, newly_left_rooms = res
|
room_entries, invited, newly_joined_rooms, newly_left_rooms = res
|
||||||
|
|
||||||
tags_by_room = yield self.store.get_updated_tags(
|
tags_by_room = await self.store.get_updated_tags(
|
||||||
user_id, since_token.account_data_key
|
user_id, since_token.account_data_key
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
res = yield self._get_all_rooms(sync_result_builder, ignored_users)
|
res = await self._get_all_rooms(sync_result_builder, ignored_users)
|
||||||
room_entries, invited, newly_joined_rooms = res
|
room_entries, invited, newly_joined_rooms = res
|
||||||
newly_left_rooms = []
|
newly_left_rooms = []
|
||||||
|
|
||||||
tags_by_room = yield self.store.get_tags_for_user(user_id)
|
tags_by_room = await self.store.get_tags_for_user(user_id)
|
||||||
|
|
||||||
def handle_room_entries(room_entry):
|
def handle_room_entries(room_entry):
|
||||||
return self._generate_room_entry(
|
return self._generate_room_entry(
|
||||||
|
|
@ -1376,7 +1364,7 @@ class SyncHandler(object):
|
||||||
always_include=sync_result_builder.full_state,
|
always_include=sync_result_builder.full_state,
|
||||||
)
|
)
|
||||||
|
|
||||||
yield concurrently_execute(handle_room_entries, room_entries, 10)
|
await concurrently_execute(handle_room_entries, room_entries, 10)
|
||||||
|
|
||||||
sync_result_builder.invited.extend(invited)
|
sync_result_builder.invited.extend(invited)
|
||||||
|
|
||||||
|
|
@ -1410,8 +1398,7 @@ class SyncHandler(object):
|
||||||
newly_left_users,
|
newly_left_users,
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _have_rooms_changed(self, sync_result_builder):
|
||||||
def _have_rooms_changed(self, sync_result_builder):
|
|
||||||
"""Returns whether there may be any new events that should be sent down
|
"""Returns whether there may be any new events that should be sent down
|
||||||
the sync. Returns True if there are.
|
the sync. Returns True if there are.
|
||||||
"""
|
"""
|
||||||
|
|
@ -1422,7 +1409,7 @@ class SyncHandler(object):
|
||||||
assert since_token
|
assert since_token
|
||||||
|
|
||||||
# Get a list of membership change events that have happened.
|
# Get a list of membership change events that have happened.
|
||||||
rooms_changed = yield self.store.get_membership_changes_for_user(
|
rooms_changed = await self.store.get_membership_changes_for_user(
|
||||||
user_id, since_token.room_key, now_token.room_key
|
user_id, since_token.room_key, now_token.room_key
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1435,8 +1422,7 @@ class SyncHandler(object):
|
||||||
return True
|
return True
|
||||||
return False
|
return False
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _get_rooms_changed(self, sync_result_builder, ignored_users):
|
||||||
def _get_rooms_changed(self, sync_result_builder, ignored_users):
|
|
||||||
"""Gets the the changes that have happened since the last sync.
|
"""Gets the the changes that have happened since the last sync.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -1461,7 +1447,7 @@ class SyncHandler(object):
|
||||||
assert since_token
|
assert since_token
|
||||||
|
|
||||||
# Get a list of membership change events that have happened.
|
# Get a list of membership change events that have happened.
|
||||||
rooms_changed = yield self.store.get_membership_changes_for_user(
|
rooms_changed = await self.store.get_membership_changes_for_user(
|
||||||
user_id, since_token.room_key, now_token.room_key
|
user_id, since_token.room_key, now_token.room_key
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1499,11 +1485,11 @@ class SyncHandler(object):
|
||||||
continue
|
continue
|
||||||
|
|
||||||
if room_id in sync_result_builder.joined_room_ids or has_join:
|
if room_id in sync_result_builder.joined_room_ids or has_join:
|
||||||
old_state_ids = yield self.get_state_at(room_id, since_token)
|
old_state_ids = await self.get_state_at(room_id, since_token)
|
||||||
old_mem_ev_id = old_state_ids.get((EventTypes.Member, user_id), None)
|
old_mem_ev_id = old_state_ids.get((EventTypes.Member, user_id), None)
|
||||||
old_mem_ev = None
|
old_mem_ev = None
|
||||||
if old_mem_ev_id:
|
if old_mem_ev_id:
|
||||||
old_mem_ev = yield self.store.get_event(
|
old_mem_ev = await self.store.get_event(
|
||||||
old_mem_ev_id, allow_none=True
|
old_mem_ev_id, allow_none=True
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1536,13 +1522,13 @@ class SyncHandler(object):
|
||||||
newly_left_rooms.append(room_id)
|
newly_left_rooms.append(room_id)
|
||||||
else:
|
else:
|
||||||
if not old_state_ids:
|
if not old_state_ids:
|
||||||
old_state_ids = yield self.get_state_at(room_id, since_token)
|
old_state_ids = await self.get_state_at(room_id, since_token)
|
||||||
old_mem_ev_id = old_state_ids.get(
|
old_mem_ev_id = old_state_ids.get(
|
||||||
(EventTypes.Member, user_id), None
|
(EventTypes.Member, user_id), None
|
||||||
)
|
)
|
||||||
old_mem_ev = None
|
old_mem_ev = None
|
||||||
if old_mem_ev_id:
|
if old_mem_ev_id:
|
||||||
old_mem_ev = yield self.store.get_event(
|
old_mem_ev = await self.store.get_event(
|
||||||
old_mem_ev_id, allow_none=True
|
old_mem_ev_id, allow_none=True
|
||||||
)
|
)
|
||||||
if old_mem_ev and old_mem_ev.membership == Membership.JOIN:
|
if old_mem_ev and old_mem_ev.membership == Membership.JOIN:
|
||||||
|
|
@ -1566,7 +1552,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
if leave_events:
|
if leave_events:
|
||||||
leave_event = leave_events[-1]
|
leave_event = leave_events[-1]
|
||||||
leave_stream_token = yield self.store.get_stream_token_for_event(
|
leave_stream_token = await self.store.get_stream_token_for_event(
|
||||||
leave_event.event_id
|
leave_event.event_id
|
||||||
)
|
)
|
||||||
leave_token = since_token.copy_and_replace(
|
leave_token = since_token.copy_and_replace(
|
||||||
|
|
@ -1603,7 +1589,7 @@ class SyncHandler(object):
|
||||||
timeline_limit = sync_config.filter_collection.timeline_limit()
|
timeline_limit = sync_config.filter_collection.timeline_limit()
|
||||||
|
|
||||||
# Get all events for rooms we're currently joined to.
|
# Get all events for rooms we're currently joined to.
|
||||||
room_to_events = yield self.store.get_room_events_stream_for_rooms(
|
room_to_events = await self.store.get_room_events_stream_for_rooms(
|
||||||
room_ids=sync_result_builder.joined_room_ids,
|
room_ids=sync_result_builder.joined_room_ids,
|
||||||
from_key=since_token.room_key,
|
from_key=since_token.room_key,
|
||||||
to_key=now_token.room_key,
|
to_key=now_token.room_key,
|
||||||
|
|
@ -1652,8 +1638,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
return room_entries, invited, newly_joined_rooms, newly_left_rooms
|
return room_entries, invited, newly_joined_rooms, newly_left_rooms
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _get_all_rooms(self, sync_result_builder, ignored_users):
|
||||||
def _get_all_rooms(self, sync_result_builder, ignored_users):
|
|
||||||
"""Returns entries for all rooms for the user.
|
"""Returns entries for all rooms for the user.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -1677,7 +1662,7 @@ class SyncHandler(object):
|
||||||
Membership.BAN,
|
Membership.BAN,
|
||||||
)
|
)
|
||||||
|
|
||||||
room_list = yield self.store.get_rooms_for_user_where_membership_is(
|
room_list = await self.store.get_rooms_for_user_where_membership_is(
|
||||||
user_id=user_id, membership_list=membership_list
|
user_id=user_id, membership_list=membership_list
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1700,7 +1685,7 @@ class SyncHandler(object):
|
||||||
elif event.membership == Membership.INVITE:
|
elif event.membership == Membership.INVITE:
|
||||||
if event.sender in ignored_users:
|
if event.sender in ignored_users:
|
||||||
continue
|
continue
|
||||||
invite = yield self.store.get_event(event.event_id)
|
invite = await self.store.get_event(event.event_id)
|
||||||
invited.append(InvitedSyncResult(room_id=event.room_id, invite=invite))
|
invited.append(InvitedSyncResult(room_id=event.room_id, invite=invite))
|
||||||
elif event.membership in (Membership.LEAVE, Membership.BAN):
|
elif event.membership in (Membership.LEAVE, Membership.BAN):
|
||||||
# Always send down rooms we were banned or kicked from.
|
# Always send down rooms we were banned or kicked from.
|
||||||
|
|
@ -1726,8 +1711,7 @@ class SyncHandler(object):
|
||||||
|
|
||||||
return room_entries, invited, []
|
return room_entries, invited, []
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _generate_room_entry(
|
||||||
def _generate_room_entry(
|
|
||||||
self,
|
self,
|
||||||
sync_result_builder,
|
sync_result_builder,
|
||||||
ignored_users,
|
ignored_users,
|
||||||
|
|
@ -1769,7 +1753,7 @@ class SyncHandler(object):
|
||||||
since_token = room_builder.since_token
|
since_token = room_builder.since_token
|
||||||
upto_token = room_builder.upto_token
|
upto_token = room_builder.upto_token
|
||||||
|
|
||||||
batch = yield self._load_filtered_recents(
|
batch = await self._load_filtered_recents(
|
||||||
room_id,
|
room_id,
|
||||||
sync_config,
|
sync_config,
|
||||||
now_token=upto_token,
|
now_token=upto_token,
|
||||||
|
|
@ -1796,7 +1780,7 @@ class SyncHandler(object):
|
||||||
# tag was added by synapse e.g. for server notice rooms.
|
# tag was added by synapse e.g. for server notice rooms.
|
||||||
if full_state:
|
if full_state:
|
||||||
user_id = sync_result_builder.sync_config.user.to_string()
|
user_id = sync_result_builder.sync_config.user.to_string()
|
||||||
tags = yield self.store.get_tags_for_room(user_id, room_id)
|
tags = await self.store.get_tags_for_room(user_id, room_id)
|
||||||
|
|
||||||
# If there aren't any tags, don't send the empty tags list down
|
# If there aren't any tags, don't send the empty tags list down
|
||||||
# sync
|
# sync
|
||||||
|
|
@ -1821,7 +1805,7 @@ class SyncHandler(object):
|
||||||
):
|
):
|
||||||
return
|
return
|
||||||
|
|
||||||
state = yield self.compute_state_delta(
|
state = await self.compute_state_delta(
|
||||||
room_id, batch, sync_config, since_token, now_token, full_state=full_state
|
room_id, batch, sync_config, since_token, now_token, full_state=full_state
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1844,7 +1828,7 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
or since_token is None
|
or since_token is None
|
||||||
):
|
):
|
||||||
summary = yield self.compute_summary(
|
summary = await self.compute_summary(
|
||||||
room_id, sync_config, batch, state, now_token
|
room_id, sync_config, batch, state, now_token
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -1861,7 +1845,7 @@ class SyncHandler(object):
|
||||||
)
|
)
|
||||||
|
|
||||||
if room_sync or always_include:
|
if room_sync or always_include:
|
||||||
notifs = yield self.unread_notifs_for_room_id(room_id, sync_config)
|
notifs = await self.unread_notifs_for_room_id(room_id, sync_config)
|
||||||
|
|
||||||
if notifs is not None:
|
if notifs is not None:
|
||||||
unread_notifications["notification_count"] = notifs["notify_count"]
|
unread_notifications["notification_count"] = notifs["notify_count"]
|
||||||
|
|
@ -1887,8 +1871,7 @@ class SyncHandler(object):
|
||||||
else:
|
else:
|
||||||
raise Exception("Unrecognized rtype: %r", room_builder.rtype)
|
raise Exception("Unrecognized rtype: %r", room_builder.rtype)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def get_rooms_for_user_at(self, user_id, stream_ordering):
|
||||||
def get_rooms_for_user_at(self, user_id, stream_ordering):
|
|
||||||
"""Get set of joined rooms for a user at the given stream ordering.
|
"""Get set of joined rooms for a user at the given stream ordering.
|
||||||
|
|
||||||
The stream ordering *must* be recent, otherwise this may throw an
|
The stream ordering *must* be recent, otherwise this may throw an
|
||||||
|
|
@ -1903,7 +1886,7 @@ class SyncHandler(object):
|
||||||
Deferred[frozenset[str]]: Set of room_ids the user is in at given
|
Deferred[frozenset[str]]: Set of room_ids the user is in at given
|
||||||
stream_ordering.
|
stream_ordering.
|
||||||
"""
|
"""
|
||||||
joined_rooms = yield self.store.get_rooms_for_user_with_stream_ordering(user_id)
|
joined_rooms = await self.store.get_rooms_for_user_with_stream_ordering(user_id)
|
||||||
|
|
||||||
joined_room_ids = set()
|
joined_room_ids = set()
|
||||||
|
|
||||||
|
|
@ -1921,10 +1904,10 @@ class SyncHandler(object):
|
||||||
|
|
||||||
logger.info("User joined room after current token: %s", room_id)
|
logger.info("User joined room after current token: %s", room_id)
|
||||||
|
|
||||||
extrems = yield self.store.get_forward_extremeties_for_room(
|
extrems = await self.store.get_forward_extremeties_for_room(
|
||||||
room_id, stream_ordering
|
room_id, stream_ordering
|
||||||
)
|
)
|
||||||
users_in_room = yield self.state.get_current_users_in_room(room_id, extrems)
|
users_in_room = await self.state.get_current_users_in_room(room_id, extrems)
|
||||||
if user_id in users_in_room:
|
if user_id in users_in_room:
|
||||||
joined_room_ids.add(room_id)
|
joined_room_ids.add(room_id)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -313,7 +313,7 @@ class TypingNotificationEventSource(object):
|
||||||
|
|
||||||
events.append(self._make_event_for(room_id))
|
events.append(self._make_event_for(room_id))
|
||||||
|
|
||||||
return events, handler._latest_room_serial
|
return defer.succeed((events, handler._latest_room_serial))
|
||||||
|
|
||||||
def get_current_key(self):
|
def get_current_key(self):
|
||||||
return self.get_typing_handler()._latest_room_serial
|
return self.get_typing_handler()._latest_room_serial
|
||||||
|
|
|
||||||
|
|
@ -96,7 +96,7 @@ def parse_boolean_from_args(args, name, default=None, required=False):
|
||||||
return {b"true": True, b"false": False}[args[name][0]]
|
return {b"true": True, b"false": False}[args[name][0]]
|
||||||
except Exception:
|
except Exception:
|
||||||
message = (
|
message = (
|
||||||
"Boolean query parameter %r must be one of" " ['true', 'false']"
|
"Boolean query parameter %r must be one of ['true', 'false']"
|
||||||
) % (name,)
|
) % (name,)
|
||||||
raise SynapseError(400, message)
|
raise SynapseError(400, message)
|
||||||
else:
|
else:
|
||||||
|
|
|
||||||
|
|
@ -261,6 +261,18 @@ def parse_drain_configs(
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class StoppableLogPublisher(LogPublisher):
|
||||||
|
"""
|
||||||
|
A log publisher that can tell its observers to shut down any external
|
||||||
|
communications.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
for obs in self._observers:
|
||||||
|
if hasattr(obs, "stop"):
|
||||||
|
obs.stop()
|
||||||
|
|
||||||
|
|
||||||
def setup_structured_logging(
|
def setup_structured_logging(
|
||||||
hs,
|
hs,
|
||||||
config,
|
config,
|
||||||
|
|
@ -336,7 +348,7 @@ def setup_structured_logging(
|
||||||
# We should never get here, but, just in case, throw an error.
|
# We should never get here, but, just in case, throw an error.
|
||||||
raise ConfigError("%s drain type cannot be configured" % (observer.type,))
|
raise ConfigError("%s drain type cannot be configured" % (observer.type,))
|
||||||
|
|
||||||
publisher = LogPublisher(*observers)
|
publisher = StoppableLogPublisher(*observers)
|
||||||
log_filter = LogLevelFilterPredicate()
|
log_filter = LogLevelFilterPredicate()
|
||||||
|
|
||||||
for namespace, namespace_config in log_config.get(
|
for namespace, namespace_config in log_config.get(
|
||||||
|
|
|
||||||
|
|
@ -17,25 +17,29 @@
|
||||||
Log formatters that output terse JSON.
|
Log formatters that output terse JSON.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
import sys
|
import sys
|
||||||
|
import traceback
|
||||||
from collections import deque
|
from collections import deque
|
||||||
from ipaddress import IPv4Address, IPv6Address, ip_address
|
from ipaddress import IPv4Address, IPv6Address, ip_address
|
||||||
from math import floor
|
from math import floor
|
||||||
from typing import IO
|
from typing import IO, Optional
|
||||||
|
|
||||||
import attr
|
import attr
|
||||||
from simplejson import dumps
|
|
||||||
from zope.interface import implementer
|
from zope.interface import implementer
|
||||||
|
|
||||||
from twisted.application.internet import ClientService
|
from twisted.application.internet import ClientService
|
||||||
|
from twisted.internet.defer import Deferred
|
||||||
from twisted.internet.endpoints import (
|
from twisted.internet.endpoints import (
|
||||||
HostnameEndpoint,
|
HostnameEndpoint,
|
||||||
TCP4ClientEndpoint,
|
TCP4ClientEndpoint,
|
||||||
TCP6ClientEndpoint,
|
TCP6ClientEndpoint,
|
||||||
)
|
)
|
||||||
|
from twisted.internet.interfaces import IPushProducer, ITransport
|
||||||
from twisted.internet.protocol import Factory, Protocol
|
from twisted.internet.protocol import Factory, Protocol
|
||||||
from twisted.logger import FileLogObserver, ILogObserver, Logger
|
from twisted.logger import FileLogObserver, ILogObserver, Logger
|
||||||
from twisted.python.failure import Failure
|
|
||||||
|
_encoder = json.JSONEncoder(ensure_ascii=False, separators=(",", ":"))
|
||||||
|
|
||||||
|
|
||||||
def flatten_event(event: dict, metadata: dict, include_time: bool = False):
|
def flatten_event(event: dict, metadata: dict, include_time: bool = False):
|
||||||
|
|
@ -141,11 +145,49 @@ def TerseJSONToConsoleLogObserver(outFile: IO[str], metadata: dict) -> FileLogOb
|
||||||
|
|
||||||
def formatEvent(_event: dict) -> str:
|
def formatEvent(_event: dict) -> str:
|
||||||
flattened = flatten_event(_event, metadata)
|
flattened = flatten_event(_event, metadata)
|
||||||
return dumps(flattened, ensure_ascii=False, separators=(",", ":")) + "\n"
|
return _encoder.encode(flattened) + "\n"
|
||||||
|
|
||||||
return FileLogObserver(outFile, formatEvent)
|
return FileLogObserver(outFile, formatEvent)
|
||||||
|
|
||||||
|
|
||||||
|
@attr.s
|
||||||
|
@implementer(IPushProducer)
|
||||||
|
class LogProducer(object):
|
||||||
|
"""
|
||||||
|
An IPushProducer that writes logs from its buffer to its transport when it
|
||||||
|
is resumed.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
buffer: Log buffer to read logs from.
|
||||||
|
transport: Transport to write to.
|
||||||
|
"""
|
||||||
|
|
||||||
|
transport = attr.ib(type=ITransport)
|
||||||
|
_buffer = attr.ib(type=deque)
|
||||||
|
_paused = attr.ib(default=False, type=bool, init=False)
|
||||||
|
|
||||||
|
def pauseProducing(self):
|
||||||
|
self._paused = True
|
||||||
|
|
||||||
|
def stopProducing(self):
|
||||||
|
self._paused = True
|
||||||
|
self._buffer = None
|
||||||
|
|
||||||
|
def resumeProducing(self):
|
||||||
|
self._paused = False
|
||||||
|
|
||||||
|
while self._paused is False and (self._buffer and self.transport.connected):
|
||||||
|
try:
|
||||||
|
event = self._buffer.popleft()
|
||||||
|
self.transport.write(_encoder.encode(event).encode("utf8"))
|
||||||
|
self.transport.write(b"\n")
|
||||||
|
except Exception:
|
||||||
|
# Something has gone wrong writing to the transport -- log it
|
||||||
|
# and break out of the while.
|
||||||
|
traceback.print_exc(file=sys.__stderr__)
|
||||||
|
break
|
||||||
|
|
||||||
|
|
||||||
@attr.s
|
@attr.s
|
||||||
@implementer(ILogObserver)
|
@implementer(ILogObserver)
|
||||||
class TerseJSONToTCPLogObserver(object):
|
class TerseJSONToTCPLogObserver(object):
|
||||||
|
|
@ -165,8 +207,9 @@ class TerseJSONToTCPLogObserver(object):
|
||||||
metadata = attr.ib(type=dict)
|
metadata = attr.ib(type=dict)
|
||||||
maximum_buffer = attr.ib(type=int)
|
maximum_buffer = attr.ib(type=int)
|
||||||
_buffer = attr.ib(default=attr.Factory(deque), type=deque)
|
_buffer = attr.ib(default=attr.Factory(deque), type=deque)
|
||||||
_writer = attr.ib(default=None)
|
_connection_waiter = attr.ib(default=None, type=Optional[Deferred])
|
||||||
_logger = attr.ib(default=attr.Factory(Logger))
|
_logger = attr.ib(default=attr.Factory(Logger))
|
||||||
|
_producer = attr.ib(default=None, type=Optional[LogProducer])
|
||||||
|
|
||||||
def start(self) -> None:
|
def start(self) -> None:
|
||||||
|
|
||||||
|
|
@ -187,38 +230,44 @@ class TerseJSONToTCPLogObserver(object):
|
||||||
factory = Factory.forProtocol(Protocol)
|
factory = Factory.forProtocol(Protocol)
|
||||||
self._service = ClientService(endpoint, factory, clock=self.hs.get_reactor())
|
self._service = ClientService(endpoint, factory, clock=self.hs.get_reactor())
|
||||||
self._service.startService()
|
self._service.startService()
|
||||||
|
self._connect()
|
||||||
|
|
||||||
def _write_loop(self) -> None:
|
def stop(self):
|
||||||
|
self._service.stopService()
|
||||||
|
|
||||||
|
def _connect(self) -> None:
|
||||||
"""
|
"""
|
||||||
Implement the write loop.
|
Triggers an attempt to connect then write to the remote if not already writing.
|
||||||
"""
|
"""
|
||||||
if self._writer:
|
if self._connection_waiter:
|
||||||
return
|
return
|
||||||
|
|
||||||
self._writer = self._service.whenConnected()
|
self._connection_waiter = self._service.whenConnected(failAfterFailures=1)
|
||||||
|
|
||||||
@self._writer.addBoth
|
@self._connection_waiter.addErrback
|
||||||
|
def fail(r):
|
||||||
|
r.printTraceback(file=sys.__stderr__)
|
||||||
|
self._connection_waiter = None
|
||||||
|
self._connect()
|
||||||
|
|
||||||
|
@self._connection_waiter.addCallback
|
||||||
def writer(r):
|
def writer(r):
|
||||||
if isinstance(r, Failure):
|
# We have a connection. If we already have a producer, and its
|
||||||
r.printTraceback(file=sys.__stderr__)
|
# transport is the same, just trigger a resumeProducing.
|
||||||
self._writer = None
|
if self._producer and r.transport is self._producer.transport:
|
||||||
self.hs.get_reactor().callLater(1, self._write_loop)
|
self._producer.resumeProducing()
|
||||||
|
self._connection_waiter = None
|
||||||
return
|
return
|
||||||
|
|
||||||
try:
|
# If the producer is still producing, stop it.
|
||||||
for event in self._buffer:
|
if self._producer:
|
||||||
r.transport.write(
|
self._producer.stopProducing()
|
||||||
dumps(event, ensure_ascii=False, separators=(",", ":")).encode(
|
|
||||||
"utf8"
|
|
||||||
)
|
|
||||||
)
|
|
||||||
r.transport.write(b"\n")
|
|
||||||
self._buffer.clear()
|
|
||||||
except Exception as e:
|
|
||||||
sys.__stderr__.write("Failed writing out logs with %s\n" % (str(e),))
|
|
||||||
|
|
||||||
self._writer = False
|
# Make a new producer and start it.
|
||||||
self.hs.get_reactor().callLater(1, self._write_loop)
|
self._producer = LogProducer(buffer=self._buffer, transport=r.transport)
|
||||||
|
r.transport.registerProducer(self._producer, True)
|
||||||
|
self._producer.resumeProducing()
|
||||||
|
self._connection_waiter = None
|
||||||
|
|
||||||
def _handle_pressure(self) -> None:
|
def _handle_pressure(self) -> None:
|
||||||
"""
|
"""
|
||||||
|
|
@ -277,4 +326,4 @@ class TerseJSONToTCPLogObserver(object):
|
||||||
self._logger.failure("Failed clearing backpressure")
|
self._logger.failure("Failed clearing backpressure")
|
||||||
|
|
||||||
# Try and write immediately.
|
# Try and write immediately.
|
||||||
self._write_loop()
|
self._connect()
|
||||||
|
|
|
||||||
|
|
@ -175,4 +175,4 @@ class ModuleApi(object):
|
||||||
Returns:
|
Returns:
|
||||||
Deferred[object]: result of func
|
Deferred[object]: result of func
|
||||||
"""
|
"""
|
||||||
return self._store.runInteraction(desc, func, *args, **kwargs)
|
return self._store.db.runInteraction(desc, func, *args, **kwargs)
|
||||||
|
|
|
||||||
|
|
@ -304,8 +304,7 @@ class Notifier(object):
|
||||||
without waking up any of the normal user event streams"""
|
without waking up any of the normal user event streams"""
|
||||||
self.notify_replication()
|
self.notify_replication()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def wait_for_events(
|
||||||
def wait_for_events(
|
|
||||||
self, user_id, timeout, callback, room_ids=None, from_token=StreamToken.START
|
self, user_id, timeout, callback, room_ids=None, from_token=StreamToken.START
|
||||||
):
|
):
|
||||||
"""Wait until the callback returns a non empty response or the
|
"""Wait until the callback returns a non empty response or the
|
||||||
|
|
@ -313,9 +312,9 @@ class Notifier(object):
|
||||||
"""
|
"""
|
||||||
user_stream = self.user_to_user_stream.get(user_id)
|
user_stream = self.user_to_user_stream.get(user_id)
|
||||||
if user_stream is None:
|
if user_stream is None:
|
||||||
current_token = yield self.event_sources.get_current_token()
|
current_token = await self.event_sources.get_current_token()
|
||||||
if room_ids is None:
|
if room_ids is None:
|
||||||
room_ids = yield self.store.get_rooms_for_user(user_id)
|
room_ids = await self.store.get_rooms_for_user(user_id)
|
||||||
user_stream = _NotifierUserStream(
|
user_stream = _NotifierUserStream(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
rooms=room_ids,
|
rooms=room_ids,
|
||||||
|
|
@ -344,11 +343,11 @@ class Notifier(object):
|
||||||
self.hs.get_reactor(),
|
self.hs.get_reactor(),
|
||||||
)
|
)
|
||||||
with PreserveLoggingContext():
|
with PreserveLoggingContext():
|
||||||
yield listener.deferred
|
await listener.deferred
|
||||||
|
|
||||||
current_token = user_stream.current_token
|
current_token = user_stream.current_token
|
||||||
|
|
||||||
result = yield callback(prev_token, current_token)
|
result = await callback(prev_token, current_token)
|
||||||
if result:
|
if result:
|
||||||
break
|
break
|
||||||
|
|
||||||
|
|
@ -364,12 +363,11 @@ class Notifier(object):
|
||||||
# This happened if there was no timeout or if the timeout had
|
# This happened if there was no timeout or if the timeout had
|
||||||
# already expired.
|
# already expired.
|
||||||
current_token = user_stream.current_token
|
current_token = user_stream.current_token
|
||||||
result = yield callback(prev_token, current_token)
|
result = await callback(prev_token, current_token)
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def get_events_for(
|
||||||
def get_events_for(
|
|
||||||
self,
|
self,
|
||||||
user,
|
user,
|
||||||
pagination_config,
|
pagination_config,
|
||||||
|
|
@ -391,15 +389,14 @@ class Notifier(object):
|
||||||
"""
|
"""
|
||||||
from_token = pagination_config.from_token
|
from_token = pagination_config.from_token
|
||||||
if not from_token:
|
if not from_token:
|
||||||
from_token = yield self.event_sources.get_current_token()
|
from_token = await self.event_sources.get_current_token()
|
||||||
|
|
||||||
limit = pagination_config.limit
|
limit = pagination_config.limit
|
||||||
|
|
||||||
room_ids, is_joined = yield self._get_room_ids(user, explicit_room_id)
|
room_ids, is_joined = await self._get_room_ids(user, explicit_room_id)
|
||||||
is_peeking = not is_joined
|
is_peeking = not is_joined
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def check_for_updates(before_token, after_token):
|
||||||
def check_for_updates(before_token, after_token):
|
|
||||||
if not after_token.is_after(before_token):
|
if not after_token.is_after(before_token):
|
||||||
return EventStreamResult([], (from_token, from_token))
|
return EventStreamResult([], (from_token, from_token))
|
||||||
|
|
||||||
|
|
@ -415,7 +412,7 @@ class Notifier(object):
|
||||||
if only_keys and name not in only_keys:
|
if only_keys and name not in only_keys:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
new_events, new_key = yield source.get_new_events(
|
new_events, new_key = await source.get_new_events(
|
||||||
user=user,
|
user=user,
|
||||||
from_key=getattr(from_token, keyname),
|
from_key=getattr(from_token, keyname),
|
||||||
limit=limit,
|
limit=limit,
|
||||||
|
|
@ -425,7 +422,7 @@ class Notifier(object):
|
||||||
)
|
)
|
||||||
|
|
||||||
if name == "room":
|
if name == "room":
|
||||||
new_events = yield filter_events_for_client(
|
new_events = await filter_events_for_client(
|
||||||
self.storage,
|
self.storage,
|
||||||
user.to_string(),
|
user.to_string(),
|
||||||
new_events,
|
new_events,
|
||||||
|
|
@ -461,7 +458,7 @@ class Notifier(object):
|
||||||
user_id_for_stream,
|
user_id_for_stream,
|
||||||
)
|
)
|
||||||
|
|
||||||
result = yield self.wait_for_events(
|
result = await self.wait_for_events(
|
||||||
user_id_for_stream,
|
user_id_for_stream,
|
||||||
timeout,
|
timeout,
|
||||||
check_for_updates,
|
check_for_updates,
|
||||||
|
|
|
||||||
|
|
@ -386,15 +386,7 @@ class RulesForRoom(object):
|
||||||
"""
|
"""
|
||||||
sequence = self.sequence
|
sequence = self.sequence
|
||||||
|
|
||||||
rows = yield self.store._simple_select_many_batch(
|
rows = yield self.store.get_membership_from_event_ids(member_event_ids.values())
|
||||||
table="room_memberships",
|
|
||||||
column="event_id",
|
|
||||||
iterable=member_event_ids.values(),
|
|
||||||
retcols=("user_id", "membership", "event_id"),
|
|
||||||
keyvalues={},
|
|
||||||
batch_size=500,
|
|
||||||
desc="_get_rules_for_member_event_ids",
|
|
||||||
)
|
|
||||||
|
|
||||||
members = {row["event_id"]: (row["user_id"], row["membership"]) for row in rows}
|
members = {row["event_id"]: (row["user_id"], row["membership"]) for row in rows}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -246,7 +246,7 @@ class HttpPusher(object):
|
||||||
# fixed, we don't suddenly deliver a load
|
# fixed, we don't suddenly deliver a load
|
||||||
# of old notifications.
|
# of old notifications.
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"Giving up on a notification to user %s, " "pushkey %s",
|
"Giving up on a notification to user %s, pushkey %s",
|
||||||
self.user_id,
|
self.user_id,
|
||||||
self.pushkey,
|
self.pushkey,
|
||||||
)
|
)
|
||||||
|
|
@ -299,8 +299,7 @@ class HttpPusher(object):
|
||||||
# for sanity, we only remove the pushkey if it
|
# for sanity, we only remove the pushkey if it
|
||||||
# was the one we actually sent...
|
# was the one we actually sent...
|
||||||
logger.warning(
|
logger.warning(
|
||||||
("Ignoring rejected pushkey %s because we" " didn't send it"),
|
("Ignoring rejected pushkey %s because we didn't send it"), pk,
|
||||||
pk,
|
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
logger.info("Pushkey %s was rejected: removing", pk)
|
logger.info("Pushkey %s was rejected: removing", pk)
|
||||||
|
|
|
||||||
|
|
@ -43,7 +43,7 @@ logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
MESSAGE_FROM_PERSON_IN_ROOM = (
|
MESSAGE_FROM_PERSON_IN_ROOM = (
|
||||||
"You have a message on %(app)s from %(person)s " "in the %(room)s room..."
|
"You have a message on %(app)s from %(person)s in the %(room)s room..."
|
||||||
)
|
)
|
||||||
MESSAGE_FROM_PERSON = "You have a message on %(app)s from %(person)s..."
|
MESSAGE_FROM_PERSON = "You have a message on %(app)s from %(person)s..."
|
||||||
MESSAGES_FROM_PERSON = "You have messages on %(app)s from %(person)s..."
|
MESSAGES_FROM_PERSON = "You have messages on %(app)s from %(person)s..."
|
||||||
|
|
@ -55,7 +55,7 @@ MESSAGES_FROM_PERSON_AND_OTHERS = (
|
||||||
"You have messages on %(app)s from %(person)s and others..."
|
"You have messages on %(app)s from %(person)s and others..."
|
||||||
)
|
)
|
||||||
INVITE_FROM_PERSON_TO_ROOM = (
|
INVITE_FROM_PERSON_TO_ROOM = (
|
||||||
"%(person)s has invited you to join the " "%(room)s room on %(app)s..."
|
"%(person)s has invited you to join the %(room)s room on %(app)s..."
|
||||||
)
|
)
|
||||||
INVITE_FROM_PERSON = "%(person)s has invited you to chat on %(app)s..."
|
INVITE_FROM_PERSON = "%(person)s has invited you to chat on %(app)s..."
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,14 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.http.server import JsonResource
|
from synapse.http.server import JsonResource
|
||||||
from synapse.replication.http import federation, login, membership, register, send_event
|
from synapse.replication.http import (
|
||||||
|
devices,
|
||||||
|
federation,
|
||||||
|
login,
|
||||||
|
membership,
|
||||||
|
register,
|
||||||
|
send_event,
|
||||||
|
)
|
||||||
|
|
||||||
REPLICATION_PREFIX = "/_synapse/replication"
|
REPLICATION_PREFIX = "/_synapse/replication"
|
||||||
|
|
||||||
|
|
@ -30,3 +37,4 @@ class ReplicationRestResource(JsonResource):
|
||||||
federation.register_servlets(hs, self)
|
federation.register_servlets(hs, self)
|
||||||
login.register_servlets(hs, self)
|
login.register_servlets(hs, self)
|
||||||
register.register_servlets(hs, self)
|
register.register_servlets(hs, self)
|
||||||
|
devices.register_servlets(hs, self)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,73 @@
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
# Copyright 2018 New Vector Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from synapse.replication.http._base import ReplicationEndpoint
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class ReplicationUserDevicesResyncRestServlet(ReplicationEndpoint):
|
||||||
|
"""Ask master to resync the device list for a user by contacting their
|
||||||
|
server.
|
||||||
|
|
||||||
|
This must happen on master so that the results can be correctly cached in
|
||||||
|
the database and streamed to workers.
|
||||||
|
|
||||||
|
Request format:
|
||||||
|
|
||||||
|
POST /_synapse/replication/user_device_resync/:user_id
|
||||||
|
|
||||||
|
{}
|
||||||
|
|
||||||
|
Response is equivalent to ` /_matrix/federation/v1/user/devices/:user_id`
|
||||||
|
response, e.g.:
|
||||||
|
|
||||||
|
{
|
||||||
|
"user_id": "@alice:example.org",
|
||||||
|
"devices": [
|
||||||
|
{
|
||||||
|
"device_id": "JLAFKJWSCS",
|
||||||
|
"keys": { ... },
|
||||||
|
"device_display_name": "Alice's Mobile Phone"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
NAME = "user_device_resync"
|
||||||
|
PATH_ARGS = ("user_id",)
|
||||||
|
CACHE = False
|
||||||
|
|
||||||
|
def __init__(self, hs):
|
||||||
|
super(ReplicationUserDevicesResyncRestServlet, self).__init__(hs)
|
||||||
|
|
||||||
|
self.device_list_updater = hs.get_device_handler().device_list_updater
|
||||||
|
self.store = hs.get_datastore()
|
||||||
|
self.clock = hs.get_clock()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _serialize_payload(user_id):
|
||||||
|
return {}
|
||||||
|
|
||||||
|
async def _handle_request(self, request, user_id):
|
||||||
|
user_devices = await self.device_list_updater.user_device_resync(user_id)
|
||||||
|
|
||||||
|
return 200, user_devices
|
||||||
|
|
||||||
|
|
||||||
|
def register_servlets(hs, http_server):
|
||||||
|
ReplicationUserDevicesResyncRestServlet(hs).register(http_server)
|
||||||
|
|
@ -93,6 +93,7 @@ class ReplicationRemoteRejectInviteRestServlet(ReplicationEndpoint):
|
||||||
{
|
{
|
||||||
"requester": ...,
|
"requester": ...,
|
||||||
"remote_room_hosts": [...],
|
"remote_room_hosts": [...],
|
||||||
|
"content": { ... }
|
||||||
}
|
}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
@ -107,7 +108,7 @@ class ReplicationRemoteRejectInviteRestServlet(ReplicationEndpoint):
|
||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _serialize_payload(requester, room_id, user_id, remote_room_hosts):
|
def _serialize_payload(requester, room_id, user_id, remote_room_hosts, content):
|
||||||
"""
|
"""
|
||||||
Args:
|
Args:
|
||||||
requester(Requester)
|
requester(Requester)
|
||||||
|
|
@ -118,12 +119,14 @@ class ReplicationRemoteRejectInviteRestServlet(ReplicationEndpoint):
|
||||||
return {
|
return {
|
||||||
"requester": requester.serialize(),
|
"requester": requester.serialize(),
|
||||||
"remote_room_hosts": remote_room_hosts,
|
"remote_room_hosts": remote_room_hosts,
|
||||||
|
"content": content,
|
||||||
}
|
}
|
||||||
|
|
||||||
async def _handle_request(self, request, room_id, user_id):
|
async def _handle_request(self, request, room_id, user_id):
|
||||||
content = parse_json_object_from_request(request)
|
content = parse_json_object_from_request(request)
|
||||||
|
|
||||||
remote_room_hosts = content["remote_room_hosts"]
|
remote_room_hosts = content["remote_room_hosts"]
|
||||||
|
event_content = content["content"]
|
||||||
|
|
||||||
requester = Requester.deserialize(self.store, content["requester"])
|
requester = Requester.deserialize(self.store, content["requester"])
|
||||||
|
|
||||||
|
|
@ -134,7 +137,7 @@ class ReplicationRemoteRejectInviteRestServlet(ReplicationEndpoint):
|
||||||
|
|
||||||
try:
|
try:
|
||||||
event = await self.federation_handler.do_remotely_reject_invite(
|
event = await self.federation_handler.do_remotely_reject_invite(
|
||||||
remote_room_hosts, room_id, user_id
|
remote_room_hosts, room_id, user_id, event_content,
|
||||||
)
|
)
|
||||||
ret = event.get_pdu_json()
|
ret = event.get_pdu_json()
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,9 @@ from typing import Dict
|
||||||
|
|
||||||
import six
|
import six
|
||||||
|
|
||||||
from synapse.storage._base import _CURRENT_STATE_CACHE_NAME, SQLBaseStore
|
from synapse.storage._base import SQLBaseStore
|
||||||
|
from synapse.storage.data_stores.main.cache import CURRENT_STATE_CACHE_NAME
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.storage.engines import PostgresEngine
|
from synapse.storage.engines import PostgresEngine
|
||||||
|
|
||||||
from ._slaved_id_tracker import SlavedIdTracker
|
from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
@ -34,8 +36,8 @@ def __func__(inp):
|
||||||
|
|
||||||
|
|
||||||
class BaseSlavedStore(SQLBaseStore):
|
class BaseSlavedStore(SQLBaseStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(BaseSlavedStore, self).__init__(db_conn, hs)
|
super(BaseSlavedStore, self).__init__(database, db_conn, hs)
|
||||||
if isinstance(self.database_engine, PostgresEngine):
|
if isinstance(self.database_engine, PostgresEngine):
|
||||||
self._cache_id_gen = SlavedIdTracker(
|
self._cache_id_gen = SlavedIdTracker(
|
||||||
db_conn, "cache_invalidation_stream", "stream_id"
|
db_conn, "cache_invalidation_stream", "stream_id"
|
||||||
|
|
@ -62,7 +64,7 @@ class BaseSlavedStore(SQLBaseStore):
|
||||||
if stream_name == "caches":
|
if stream_name == "caches":
|
||||||
self._cache_id_gen.advance(token)
|
self._cache_id_gen.advance(token)
|
||||||
for row in rows:
|
for row in rows:
|
||||||
if row.cache_func == _CURRENT_STATE_CACHE_NAME:
|
if row.cache_func == CURRENT_STATE_CACHE_NAME:
|
||||||
room_id = row.keys[0]
|
room_id = row.keys[0]
|
||||||
members_changed = set(row.keys[1:])
|
members_changed = set(row.keys[1:])
|
||||||
self._invalidate_state_caches(room_id, members_changed)
|
self._invalidate_state_caches(room_id, members_changed)
|
||||||
|
|
|
||||||
|
|
@ -18,15 +18,16 @@ from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||||
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
|
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
|
||||||
from synapse.storage.data_stores.main.account_data import AccountDataWorkerStore
|
from synapse.storage.data_stores.main.account_data import AccountDataWorkerStore
|
||||||
from synapse.storage.data_stores.main.tags import TagsWorkerStore
|
from synapse.storage.data_stores.main.tags import TagsWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
|
|
||||||
class SlavedAccountDataStore(TagsWorkerStore, AccountDataWorkerStore, BaseSlavedStore):
|
class SlavedAccountDataStore(TagsWorkerStore, AccountDataWorkerStore, BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
self._account_data_id_gen = SlavedIdTracker(
|
self._account_data_id_gen = SlavedIdTracker(
|
||||||
db_conn, "account_data_max_stream_id", "stream_id"
|
db_conn, "account_data_max_stream_id", "stream_id"
|
||||||
)
|
)
|
||||||
|
|
||||||
super(SlavedAccountDataStore, self).__init__(db_conn, hs)
|
super(SlavedAccountDataStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
def get_max_account_data_stream_id(self):
|
def get_max_account_data_stream_id(self):
|
||||||
return self._account_data_id_gen.get_current_token()
|
return self._account_data_id_gen.get_current_token()
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,7 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage.data_stores.main.client_ips import LAST_SEEN_GRANULARITY
|
from synapse.storage.data_stores.main.client_ips import LAST_SEEN_GRANULARITY
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.util.caches import CACHE_SIZE_FACTOR
|
from synapse.util.caches import CACHE_SIZE_FACTOR
|
||||||
from synapse.util.caches.descriptors import Cache
|
from synapse.util.caches.descriptors import Cache
|
||||||
|
|
||||||
|
|
@ -21,8 +22,8 @@ from ._base import BaseSlavedStore
|
||||||
|
|
||||||
|
|
||||||
class SlavedClientIpStore(BaseSlavedStore):
|
class SlavedClientIpStore(BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedClientIpStore, self).__init__(db_conn, hs)
|
super(SlavedClientIpStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
self.client_ip_last_seen = Cache(
|
self.client_ip_last_seen = Cache(
|
||||||
name="client_ip_last_seen", keylen=4, max_entries=50000 * CACHE_SIZE_FACTOR
|
name="client_ip_last_seen", keylen=4, max_entries=50000 * CACHE_SIZE_FACTOR
|
||||||
|
|
|
||||||
|
|
@ -16,13 +16,14 @@
|
||||||
from synapse.replication.slave.storage._base import BaseSlavedStore
|
from synapse.replication.slave.storage._base import BaseSlavedStore
|
||||||
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
|
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
|
||||||
from synapse.storage.data_stores.main.deviceinbox import DeviceInboxWorkerStore
|
from synapse.storage.data_stores.main.deviceinbox import DeviceInboxWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.util.caches.expiringcache import ExpiringCache
|
from synapse.util.caches.expiringcache import ExpiringCache
|
||||||
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
||||||
|
|
||||||
|
|
||||||
class SlavedDeviceInboxStore(DeviceInboxWorkerStore, BaseSlavedStore):
|
class SlavedDeviceInboxStore(DeviceInboxWorkerStore, BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedDeviceInboxStore, self).__init__(db_conn, hs)
|
super(SlavedDeviceInboxStore, self).__init__(database, db_conn, hs)
|
||||||
self._device_inbox_id_gen = SlavedIdTracker(
|
self._device_inbox_id_gen = SlavedIdTracker(
|
||||||
db_conn, "device_max_stream_id", "stream_id"
|
db_conn, "device_max_stream_id", "stream_id"
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -18,12 +18,13 @@ from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
|
||||||
from synapse.replication.tcp.streams._base import DeviceListsStream, UserSignatureStream
|
from synapse.replication.tcp.streams._base import DeviceListsStream, UserSignatureStream
|
||||||
from synapse.storage.data_stores.main.devices import DeviceWorkerStore
|
from synapse.storage.data_stores.main.devices import DeviceWorkerStore
|
||||||
from synapse.storage.data_stores.main.end_to_end_keys import EndToEndKeyWorkerStore
|
from synapse.storage.data_stores.main.end_to_end_keys import EndToEndKeyWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
||||||
|
|
||||||
|
|
||||||
class SlavedDeviceStore(EndToEndKeyWorkerStore, DeviceWorkerStore, BaseSlavedStore):
|
class SlavedDeviceStore(EndToEndKeyWorkerStore, DeviceWorkerStore, BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedDeviceStore, self).__init__(db_conn, hs)
|
super(SlavedDeviceStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -31,6 +31,7 @@ from synapse.storage.data_stores.main.signatures import SignatureWorkerStore
|
||||||
from synapse.storage.data_stores.main.state import StateGroupWorkerStore
|
from synapse.storage.data_stores.main.state import StateGroupWorkerStore
|
||||||
from synapse.storage.data_stores.main.stream import StreamWorkerStore
|
from synapse.storage.data_stores.main.stream import StreamWorkerStore
|
||||||
from synapse.storage.data_stores.main.user_erasure_store import UserErasureWorkerStore
|
from synapse.storage.data_stores.main.user_erasure_store import UserErasureWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
from ._base import BaseSlavedStore
|
from ._base import BaseSlavedStore
|
||||||
from ._slaved_id_tracker import SlavedIdTracker
|
from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
@ -59,13 +60,13 @@ class SlavedEventStore(
|
||||||
RelationsWorkerStore,
|
RelationsWorkerStore,
|
||||||
BaseSlavedStore,
|
BaseSlavedStore,
|
||||||
):
|
):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
self._stream_id_gen = SlavedIdTracker(db_conn, "events", "stream_ordering")
|
self._stream_id_gen = SlavedIdTracker(db_conn, "events", "stream_ordering")
|
||||||
self._backfill_id_gen = SlavedIdTracker(
|
self._backfill_id_gen = SlavedIdTracker(
|
||||||
db_conn, "events", "stream_ordering", step=-1
|
db_conn, "events", "stream_ordering", step=-1
|
||||||
)
|
)
|
||||||
|
|
||||||
super(SlavedEventStore, self).__init__(db_conn, hs)
|
super(SlavedEventStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
# Cached functions can't be accessed through a class instance so we need
|
# Cached functions can't be accessed through a class instance so we need
|
||||||
# to reach inside the __dict__ to extract them.
|
# to reach inside the __dict__ to extract them.
|
||||||
|
|
|
||||||
|
|
@ -14,13 +14,14 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage.data_stores.main.filtering import FilteringStore
|
from synapse.storage.data_stores.main.filtering import FilteringStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
from ._base import BaseSlavedStore
|
from ._base import BaseSlavedStore
|
||||||
|
|
||||||
|
|
||||||
class SlavedFilteringStore(BaseSlavedStore):
|
class SlavedFilteringStore(BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedFilteringStore, self).__init__(db_conn, hs)
|
super(SlavedFilteringStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
# Filters are immutable so this cache doesn't need to be expired
|
# Filters are immutable so this cache doesn't need to be expired
|
||||||
get_user_filter = FilteringStore.__dict__["get_user_filter"]
|
get_user_filter = FilteringStore.__dict__["get_user_filter"]
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,7 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage import DataStore
|
from synapse.storage import DataStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
||||||
|
|
||||||
from ._base import BaseSlavedStore, __func__
|
from ._base import BaseSlavedStore, __func__
|
||||||
|
|
@ -21,8 +22,8 @@ from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
||||||
|
|
||||||
class SlavedGroupServerStore(BaseSlavedStore):
|
class SlavedGroupServerStore(BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedGroupServerStore, self).__init__(db_conn, hs)
|
super(SlavedGroupServerStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@
|
||||||
|
|
||||||
from synapse.storage import DataStore
|
from synapse.storage import DataStore
|
||||||
from synapse.storage.data_stores.main.presence import PresenceStore
|
from synapse.storage.data_stores.main.presence import PresenceStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
from synapse.util.caches.stream_change_cache import StreamChangeCache
|
||||||
|
|
||||||
from ._base import BaseSlavedStore, __func__
|
from ._base import BaseSlavedStore, __func__
|
||||||
|
|
@ -22,8 +23,8 @@ from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
||||||
|
|
||||||
class SlavedPresenceStore(BaseSlavedStore):
|
class SlavedPresenceStore(BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedPresenceStore, self).__init__(db_conn, hs)
|
super(SlavedPresenceStore, self).__init__(database, db_conn, hs)
|
||||||
self._presence_id_gen = SlavedIdTracker(db_conn, "presence_stream", "stream_id")
|
self._presence_id_gen = SlavedIdTracker(db_conn, "presence_stream", "stream_id")
|
||||||
|
|
||||||
self._presence_on_startup = self._get_active_presence(db_conn)
|
self._presence_on_startup = self._get_active_presence(db_conn)
|
||||||
|
|
|
||||||
|
|
@ -15,17 +15,18 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage.data_stores.main.push_rule import PushRulesWorkerStore
|
from synapse.storage.data_stores.main.push_rule import PushRulesWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
from ._slaved_id_tracker import SlavedIdTracker
|
from ._slaved_id_tracker import SlavedIdTracker
|
||||||
from .events import SlavedEventStore
|
from .events import SlavedEventStore
|
||||||
|
|
||||||
|
|
||||||
class SlavedPushRuleStore(SlavedEventStore, PushRulesWorkerStore):
|
class SlavedPushRuleStore(SlavedEventStore, PushRulesWorkerStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
self._push_rules_stream_id_gen = SlavedIdTracker(
|
self._push_rules_stream_id_gen = SlavedIdTracker(
|
||||||
db_conn, "push_rules_stream", "stream_id"
|
db_conn, "push_rules_stream", "stream_id"
|
||||||
)
|
)
|
||||||
super(SlavedPushRuleStore, self).__init__(db_conn, hs)
|
super(SlavedPushRuleStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
def get_push_rules_stream_token(self):
|
def get_push_rules_stream_token(self):
|
||||||
return (
|
return (
|
||||||
|
|
|
||||||
|
|
@ -15,14 +15,15 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage.data_stores.main.pusher import PusherWorkerStore
|
from synapse.storage.data_stores.main.pusher import PusherWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
from ._base import BaseSlavedStore
|
from ._base import BaseSlavedStore
|
||||||
from ._slaved_id_tracker import SlavedIdTracker
|
from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
||||||
|
|
||||||
class SlavedPusherStore(PusherWorkerStore, BaseSlavedStore):
|
class SlavedPusherStore(PusherWorkerStore, BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(SlavedPusherStore, self).__init__(db_conn, hs)
|
super(SlavedPusherStore, self).__init__(database, db_conn, hs)
|
||||||
self._pushers_id_gen = SlavedIdTracker(
|
self._pushers_id_gen = SlavedIdTracker(
|
||||||
db_conn, "pushers", "id", extra_tables=[("deleted_pushers", "stream_id")]
|
db_conn, "pushers", "id", extra_tables=[("deleted_pushers", "stream_id")]
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,7 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage.data_stores.main.receipts import ReceiptsWorkerStore
|
from synapse.storage.data_stores.main.receipts import ReceiptsWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
from ._base import BaseSlavedStore
|
from ._base import BaseSlavedStore
|
||||||
from ._slaved_id_tracker import SlavedIdTracker
|
from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
@ -29,14 +30,14 @@ from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
||||||
|
|
||||||
class SlavedReceiptsStore(ReceiptsWorkerStore, BaseSlavedStore):
|
class SlavedReceiptsStore(ReceiptsWorkerStore, BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
# We instantiate this first as the ReceiptsWorkerStore constructor
|
# We instantiate this first as the ReceiptsWorkerStore constructor
|
||||||
# needs to be able to call get_max_receipt_stream_id
|
# needs to be able to call get_max_receipt_stream_id
|
||||||
self._receipts_id_gen = SlavedIdTracker(
|
self._receipts_id_gen = SlavedIdTracker(
|
||||||
db_conn, "receipts_linearized", "stream_id"
|
db_conn, "receipts_linearized", "stream_id"
|
||||||
)
|
)
|
||||||
|
|
||||||
super(SlavedReceiptsStore, self).__init__(db_conn, hs)
|
super(SlavedReceiptsStore, self).__init__(database, db_conn, hs)
|
||||||
|
|
||||||
def get_max_receipt_stream_id(self):
|
def get_max_receipt_stream_id(self):
|
||||||
return self._receipts_id_gen.get_current_token()
|
return self._receipts_id_gen.get_current_token()
|
||||||
|
|
|
||||||
|
|
@ -14,14 +14,15 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from synapse.storage.data_stores.main.room import RoomWorkerStore
|
from synapse.storage.data_stores.main.room import RoomWorkerStore
|
||||||
|
from synapse.storage.database import Database
|
||||||
|
|
||||||
from ._base import BaseSlavedStore
|
from ._base import BaseSlavedStore
|
||||||
from ._slaved_id_tracker import SlavedIdTracker
|
from ._slaved_id_tracker import SlavedIdTracker
|
||||||
|
|
||||||
|
|
||||||
class RoomStore(RoomWorkerStore, BaseSlavedStore):
|
class RoomStore(RoomWorkerStore, BaseSlavedStore):
|
||||||
def __init__(self, db_conn, hs):
|
def __init__(self, database: Database, db_conn, hs):
|
||||||
super(RoomStore, self).__init__(db_conn, hs)
|
super(RoomStore, self).__init__(database, db_conn, hs)
|
||||||
self._public_room_id_gen = SlavedIdTracker(
|
self._public_room_id_gen = SlavedIdTracker(
|
||||||
db_conn, "public_room_list_stream", "stream_id"
|
db_conn, "public_room_list_stream", "stream_id"
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -88,8 +88,7 @@ TagAccountDataStreamRow = namedtuple(
|
||||||
"TagAccountDataStreamRow", ("user_id", "room_id", "data") # str # str # dict
|
"TagAccountDataStreamRow", ("user_id", "room_id", "data") # str # str # dict
|
||||||
)
|
)
|
||||||
AccountDataStreamRow = namedtuple(
|
AccountDataStreamRow = namedtuple(
|
||||||
"AccountDataStream",
|
"AccountDataStream", ("user_id", "room_id", "data_type") # str # str # str
|
||||||
("user_id", "room_id", "data_type", "data"), # str # str # str # dict
|
|
||||||
)
|
)
|
||||||
GroupsStreamRow = namedtuple(
|
GroupsStreamRow = namedtuple(
|
||||||
"GroupsStreamRow",
|
"GroupsStreamRow",
|
||||||
|
|
@ -421,8 +420,8 @@ class AccountDataStream(Stream):
|
||||||
|
|
||||||
results = list(room_results)
|
results = list(room_results)
|
||||||
results.extend(
|
results.extend(
|
||||||
(stream_id, user_id, None, account_data_type, content)
|
(stream_id, user_id, None, account_data_type)
|
||||||
for stream_id, user_id, account_data_type, content in global_results
|
for stream_id, user_id, account_data_type in global_results
|
||||||
)
|
)
|
||||||
|
|
||||||
return results
|
return results
|
||||||
|
|
|
||||||
|
|
@ -34,12 +34,12 @@ from synapse.rest.admin.server_notice_servlet import SendServerNoticeServlet
|
||||||
from synapse.rest.admin.users import (
|
from synapse.rest.admin.users import (
|
||||||
AccountValidityRenewServlet,
|
AccountValidityRenewServlet,
|
||||||
DeactivateAccountRestServlet,
|
DeactivateAccountRestServlet,
|
||||||
GetUsersPaginatedRestServlet,
|
|
||||||
ResetPasswordRestServlet,
|
ResetPasswordRestServlet,
|
||||||
SearchUsersRestServlet,
|
SearchUsersRestServlet,
|
||||||
UserAdminServlet,
|
UserAdminServlet,
|
||||||
UserRegisterServlet,
|
UserRegisterServlet,
|
||||||
UsersRestServlet,
|
UsersRestServlet,
|
||||||
|
UsersRestServletV2,
|
||||||
WhoisRestServlet,
|
WhoisRestServlet,
|
||||||
)
|
)
|
||||||
from synapse.util.versionstring import get_version_string
|
from synapse.util.versionstring import get_version_string
|
||||||
|
|
@ -191,6 +191,7 @@ def register_servlets(hs, http_server):
|
||||||
SendServerNoticeServlet(hs).register(http_server)
|
SendServerNoticeServlet(hs).register(http_server)
|
||||||
VersionServlet(hs).register(http_server)
|
VersionServlet(hs).register(http_server)
|
||||||
UserAdminServlet(hs).register(http_server)
|
UserAdminServlet(hs).register(http_server)
|
||||||
|
UsersRestServletV2(hs).register(http_server)
|
||||||
|
|
||||||
|
|
||||||
def register_servlets_for_client_rest_resource(hs, http_server):
|
def register_servlets_for_client_rest_resource(hs, http_server):
|
||||||
|
|
@ -201,7 +202,6 @@ def register_servlets_for_client_rest_resource(hs, http_server):
|
||||||
PurgeHistoryRestServlet(hs).register(http_server)
|
PurgeHistoryRestServlet(hs).register(http_server)
|
||||||
UsersRestServlet(hs).register(http_server)
|
UsersRestServlet(hs).register(http_server)
|
||||||
ResetPasswordRestServlet(hs).register(http_server)
|
ResetPasswordRestServlet(hs).register(http_server)
|
||||||
GetUsersPaginatedRestServlet(hs).register(http_server)
|
|
||||||
SearchUsersRestServlet(hs).register(http_server)
|
SearchUsersRestServlet(hs).register(http_server)
|
||||||
ShutdownRoomRestServlet(hs).register(http_server)
|
ShutdownRoomRestServlet(hs).register(http_server)
|
||||||
UserRegisterServlet(hs).register(http_server)
|
UserRegisterServlet(hs).register(http_server)
|
||||||
|
|
|
||||||
|
|
@ -25,6 +25,7 @@ from synapse.api.errors import Codes, SynapseError
|
||||||
from synapse.http.servlet import (
|
from synapse.http.servlet import (
|
||||||
RestServlet,
|
RestServlet,
|
||||||
assert_params_in_dict,
|
assert_params_in_dict,
|
||||||
|
parse_boolean,
|
||||||
parse_integer,
|
parse_integer,
|
||||||
parse_json_object_from_request,
|
parse_json_object_from_request,
|
||||||
parse_string,
|
parse_string,
|
||||||
|
|
@ -59,71 +60,45 @@ class UsersRestServlet(RestServlet):
|
||||||
return 200, ret
|
return 200, ret
|
||||||
|
|
||||||
|
|
||||||
class GetUsersPaginatedRestServlet(RestServlet):
|
class UsersRestServletV2(RestServlet):
|
||||||
"""Get request to get specific number of users from Synapse.
|
PATTERNS = (re.compile("^/_synapse/admin/v2/users$"),)
|
||||||
This needs user to have administrator access in Synapse.
|
|
||||||
Example:
|
|
||||||
http://localhost:8008/_synapse/admin/v1/users_paginate/
|
|
||||||
@admin:user?access_token=admin_access_token&start=0&limit=10
|
|
||||||
Returns:
|
|
||||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
|
||||||
"""
|
|
||||||
|
|
||||||
PATTERNS = historical_admin_path_patterns(
|
"""Get request to list all local users.
|
||||||
"/users_paginate/(?P<target_user_id>[^/]*)"
|
This needs user to have administrator access in Synapse.
|
||||||
)
|
|
||||||
|
GET /_synapse/admin/v2/users?from=0&limit=10&guests=false
|
||||||
|
|
||||||
|
returns:
|
||||||
|
200 OK with list of users if success otherwise an error.
|
||||||
|
|
||||||
|
The parameters `from` and `limit` are required only for pagination.
|
||||||
|
By default, a `limit` of 100 is used.
|
||||||
|
The parameter `user_id` can be used to filter by user id.
|
||||||
|
The parameter `guests` can be used to exclude guest users.
|
||||||
|
The parameter `deactivated` can be used to include deactivated users.
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, hs):
|
def __init__(self, hs):
|
||||||
self.store = hs.get_datastore()
|
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.handlers = hs.get_handlers()
|
self.admin_handler = hs.get_handlers().admin_handler
|
||||||
|
|
||||||
async def on_GET(self, request, target_user_id):
|
async def on_GET(self, request):
|
||||||
"""Get request to get specific number of users from Synapse.
|
|
||||||
This needs user to have administrator access in Synapse.
|
|
||||||
"""
|
|
||||||
await assert_requester_is_admin(self.auth, request)
|
await assert_requester_is_admin(self.auth, request)
|
||||||
|
|
||||||
target_user = UserID.from_string(target_user_id)
|
start = parse_integer(request, "from", default=0)
|
||||||
|
limit = parse_integer(request, "limit", default=100)
|
||||||
|
user_id = parse_string(request, "user_id", default=None)
|
||||||
|
guests = parse_boolean(request, "guests", default=True)
|
||||||
|
deactivated = parse_boolean(request, "deactivated", default=False)
|
||||||
|
|
||||||
if not self.hs.is_mine(target_user):
|
users = await self.admin_handler.get_users_paginate(
|
||||||
raise SynapseError(400, "Can only users a local user")
|
start, limit, user_id, guests, deactivated
|
||||||
|
)
|
||||||
|
ret = {"users": users}
|
||||||
|
if len(users) >= limit:
|
||||||
|
ret["next_token"] = str(start + len(users))
|
||||||
|
|
||||||
order = "name" # order by name in user table
|
|
||||||
start = parse_integer(request, "start", required=True)
|
|
||||||
limit = parse_integer(request, "limit", required=True)
|
|
||||||
|
|
||||||
logger.info("limit: %s, start: %s", limit, start)
|
|
||||||
|
|
||||||
ret = await self.handlers.admin_handler.get_users_paginate(order, start, limit)
|
|
||||||
return 200, ret
|
|
||||||
|
|
||||||
async def on_POST(self, request, target_user_id):
|
|
||||||
"""Post request to get specific number of users from Synapse..
|
|
||||||
This needs user to have administrator access in Synapse.
|
|
||||||
Example:
|
|
||||||
http://localhost:8008/_synapse/admin/v1/users_paginate/
|
|
||||||
@admin:user?access_token=admin_access_token
|
|
||||||
JsonBodyToSend:
|
|
||||||
{
|
|
||||||
"start": "0",
|
|
||||||
"limit": "10
|
|
||||||
}
|
|
||||||
Returns:
|
|
||||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
|
||||||
"""
|
|
||||||
await assert_requester_is_admin(self.auth, request)
|
|
||||||
UserID.from_string(target_user_id)
|
|
||||||
|
|
||||||
order = "name" # order by name in user table
|
|
||||||
params = parse_json_object_from_request(request)
|
|
||||||
assert_params_in_dict(params, ["limit", "start"])
|
|
||||||
limit = params["limit"]
|
|
||||||
start = params["start"]
|
|
||||||
logger.info("limit: %s, start: %s", limit, start)
|
|
||||||
|
|
||||||
ret = await self.handlers.admin_handler.get_users_paginate(order, start, limit)
|
|
||||||
return 200, ret
|
return 200, ret
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -16,8 +16,6 @@
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import (
|
from synapse.api.errors import (
|
||||||
AuthError,
|
AuthError,
|
||||||
Codes,
|
Codes,
|
||||||
|
|
@ -47,17 +45,15 @@ class ClientDirectoryServer(RestServlet):
|
||||||
self.handlers = hs.get_handlers()
|
self.handlers = hs.get_handlers()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, room_alias):
|
||||||
def on_GET(self, request, room_alias):
|
|
||||||
room_alias = RoomAlias.from_string(room_alias)
|
room_alias = RoomAlias.from_string(room_alias)
|
||||||
|
|
||||||
dir_handler = self.handlers.directory_handler
|
dir_handler = self.handlers.directory_handler
|
||||||
res = yield dir_handler.get_association(room_alias)
|
res = await dir_handler.get_association(room_alias)
|
||||||
|
|
||||||
return 200, res
|
return 200, res
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, room_alias):
|
||||||
def on_PUT(self, request, room_alias):
|
|
||||||
room_alias = RoomAlias.from_string(room_alias)
|
room_alias = RoomAlias.from_string(room_alias)
|
||||||
|
|
||||||
content = parse_json_object_from_request(request)
|
content = parse_json_object_from_request(request)
|
||||||
|
|
@ -77,26 +73,25 @@ class ClientDirectoryServer(RestServlet):
|
||||||
|
|
||||||
# TODO(erikj): Check types.
|
# TODO(erikj): Check types.
|
||||||
|
|
||||||
room = yield self.store.get_room(room_id)
|
room = await self.store.get_room(room_id)
|
||||||
if room is None:
|
if room is None:
|
||||||
raise SynapseError(400, "Room does not exist")
|
raise SynapseError(400, "Room does not exist")
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
|
|
||||||
yield self.handlers.directory_handler.create_association(
|
await self.handlers.directory_handler.create_association(
|
||||||
requester, room_alias, room_id, servers
|
requester, room_alias, room_id, servers
|
||||||
)
|
)
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_DELETE(self, request, room_alias):
|
||||||
def on_DELETE(self, request, room_alias):
|
|
||||||
dir_handler = self.handlers.directory_handler
|
dir_handler = self.handlers.directory_handler
|
||||||
|
|
||||||
try:
|
try:
|
||||||
service = yield self.auth.get_appservice_by_req(request)
|
service = await self.auth.get_appservice_by_req(request)
|
||||||
room_alias = RoomAlias.from_string(room_alias)
|
room_alias = RoomAlias.from_string(room_alias)
|
||||||
yield dir_handler.delete_appservice_association(service, room_alias)
|
await dir_handler.delete_appservice_association(service, room_alias)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Application service at %s deleted alias %s",
|
"Application service at %s deleted alias %s",
|
||||||
service.url,
|
service.url,
|
||||||
|
|
@ -107,12 +102,12 @@ class ClientDirectoryServer(RestServlet):
|
||||||
# fallback to default user behaviour if they aren't an AS
|
# fallback to default user behaviour if they aren't an AS
|
||||||
pass
|
pass
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
user = requester.user
|
user = requester.user
|
||||||
|
|
||||||
room_alias = RoomAlias.from_string(room_alias)
|
room_alias = RoomAlias.from_string(room_alias)
|
||||||
|
|
||||||
yield dir_handler.delete_association(requester, room_alias)
|
await dir_handler.delete_association(requester, room_alias)
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"User %s deleted alias %s", user.to_string(), room_alias.to_string()
|
"User %s deleted alias %s", user.to_string(), room_alias.to_string()
|
||||||
|
|
@ -130,32 +125,29 @@ class ClientDirectoryListServer(RestServlet):
|
||||||
self.handlers = hs.get_handlers()
|
self.handlers = hs.get_handlers()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, room_id):
|
||||||
def on_GET(self, request, room_id):
|
room = await self.store.get_room(room_id)
|
||||||
room = yield self.store.get_room(room_id)
|
|
||||||
if room is None:
|
if room is None:
|
||||||
raise NotFoundError("Unknown room")
|
raise NotFoundError("Unknown room")
|
||||||
|
|
||||||
return 200, {"visibility": "public" if room["is_public"] else "private"}
|
return 200, {"visibility": "public" if room["is_public"] else "private"}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, room_id):
|
||||||
def on_PUT(self, request, room_id):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
|
|
||||||
content = parse_json_object_from_request(request)
|
content = parse_json_object_from_request(request)
|
||||||
visibility = content.get("visibility", "public")
|
visibility = content.get("visibility", "public")
|
||||||
|
|
||||||
yield self.handlers.directory_handler.edit_published_room_list(
|
await self.handlers.directory_handler.edit_published_room_list(
|
||||||
requester, room_id, visibility
|
requester, room_id, visibility
|
||||||
)
|
)
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_DELETE(self, request, room_id):
|
||||||
def on_DELETE(self, request, room_id):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
|
|
||||||
yield self.handlers.directory_handler.edit_published_room_list(
|
await self.handlers.directory_handler.edit_published_room_list(
|
||||||
requester, room_id, "private"
|
requester, room_id, "private"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -181,15 +173,14 @@ class ClientAppserviceDirectoryListServer(RestServlet):
|
||||||
def on_DELETE(self, request, network_id, room_id):
|
def on_DELETE(self, request, network_id, room_id):
|
||||||
return self._edit(request, network_id, room_id, "private")
|
return self._edit(request, network_id, room_id, "private")
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _edit(self, request, network_id, room_id, visibility):
|
||||||
def _edit(self, request, network_id, room_id, visibility):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
if not requester.app_service:
|
if not requester.app_service:
|
||||||
raise AuthError(
|
raise AuthError(
|
||||||
403, "Only appservices can edit the appservice published room list"
|
403, "Only appservices can edit the appservice published room list"
|
||||||
)
|
)
|
||||||
|
|
||||||
yield self.handlers.directory_handler.edit_published_appservice_room_list(
|
await self.handlers.directory_handler.edit_published_appservice_room_list(
|
||||||
requester.app_service.id, network_id, room_id, visibility
|
requester.app_service.id, network_id, room_id, visibility
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -16,8 +16,6 @@
|
||||||
"""This module contains REST servlets to do with event streaming, /events."""
|
"""This module contains REST servlets to do with event streaming, /events."""
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import SynapseError
|
from synapse.api.errors import SynapseError
|
||||||
from synapse.http.servlet import RestServlet
|
from synapse.http.servlet import RestServlet
|
||||||
from synapse.rest.client.v2_alpha._base import client_patterns
|
from synapse.rest.client.v2_alpha._base import client_patterns
|
||||||
|
|
@ -36,9 +34,8 @@ class EventStreamRestServlet(RestServlet):
|
||||||
self.event_stream_handler = hs.get_event_stream_handler()
|
self.event_stream_handler = hs.get_event_stream_handler()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||||
requester = yield self.auth.get_user_by_req(request, allow_guest=True)
|
|
||||||
is_guest = requester.is_guest
|
is_guest = requester.is_guest
|
||||||
room_id = None
|
room_id = None
|
||||||
if is_guest:
|
if is_guest:
|
||||||
|
|
@ -57,7 +54,7 @@ class EventStreamRestServlet(RestServlet):
|
||||||
|
|
||||||
as_client_event = b"raw" not in request.args
|
as_client_event = b"raw" not in request.args
|
||||||
|
|
||||||
chunk = yield self.event_stream_handler.get_stream(
|
chunk = await self.event_stream_handler.get_stream(
|
||||||
requester.user.to_string(),
|
requester.user.to_string(),
|
||||||
pagin_config,
|
pagin_config,
|
||||||
timeout=timeout,
|
timeout=timeout,
|
||||||
|
|
@ -83,14 +80,13 @@ class EventRestServlet(RestServlet):
|
||||||
self.event_handler = hs.get_event_handler()
|
self.event_handler = hs.get_event_handler()
|
||||||
self._event_serializer = hs.get_event_client_serializer()
|
self._event_serializer = hs.get_event_client_serializer()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, event_id):
|
||||||
def on_GET(self, request, event_id):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
event = await self.event_handler.get_event(requester.user, None, event_id)
|
||||||
event = yield self.event_handler.get_event(requester.user, None, event_id)
|
|
||||||
|
|
||||||
time_now = self.clock.time_msec()
|
time_now = self.clock.time_msec()
|
||||||
if event:
|
if event:
|
||||||
event = yield self._event_serializer.serialize_event(event, time_now)
|
event = await self._event_serializer.serialize_event(event, time_now)
|
||||||
return 200, event
|
return 200, event
|
||||||
else:
|
else:
|
||||||
return 404, "Event not found."
|
return 404, "Event not found."
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,6 @@
|
||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.http.servlet import RestServlet, parse_boolean
|
from synapse.http.servlet import RestServlet, parse_boolean
|
||||||
from synapse.rest.client.v2_alpha._base import client_patterns
|
from synapse.rest.client.v2_alpha._base import client_patterns
|
||||||
|
|
@ -29,13 +28,12 @@ class InitialSyncRestServlet(RestServlet):
|
||||||
self.initial_sync_handler = hs.get_initial_sync_handler()
|
self.initial_sync_handler = hs.get_initial_sync_handler()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
as_client_event = b"raw" not in request.args
|
as_client_event = b"raw" not in request.args
|
||||||
pagination_config = PaginationConfig.from_request(request)
|
pagination_config = PaginationConfig.from_request(request)
|
||||||
include_archived = parse_boolean(request, "archived", default=False)
|
include_archived = parse_boolean(request, "archived", default=False)
|
||||||
content = yield self.initial_sync_handler.snapshot_all_rooms(
|
content = await self.initial_sync_handler.snapshot_all_rooms(
|
||||||
user_id=requester.user.to_string(),
|
user_id=requester.user.to_string(),
|
||||||
pagin_config=pagination_config,
|
pagin_config=pagination_config,
|
||||||
as_client_event=as_client_event,
|
as_client_event=as_client_event,
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,6 @@ import xml.etree.ElementTree as ET
|
||||||
|
|
||||||
from six.moves import urllib
|
from six.moves import urllib
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
from twisted.web.client import PartialDownloadError
|
from twisted.web.client import PartialDownloadError
|
||||||
|
|
||||||
from synapse.api.errors import Codes, LoginError, SynapseError
|
from synapse.api.errors import Codes, LoginError, SynapseError
|
||||||
|
|
@ -130,8 +129,7 @@ class LoginRestServlet(RestServlet):
|
||||||
def on_OPTIONS(self, request):
|
def on_OPTIONS(self, request):
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
self._address_ratelimiter.ratelimit(
|
self._address_ratelimiter.ratelimit(
|
||||||
request.getClientIP(),
|
request.getClientIP(),
|
||||||
time_now_s=self.hs.clock.time(),
|
time_now_s=self.hs.clock.time(),
|
||||||
|
|
@ -145,11 +143,11 @@ class LoginRestServlet(RestServlet):
|
||||||
if self.jwt_enabled and (
|
if self.jwt_enabled and (
|
||||||
login_submission["type"] == LoginRestServlet.JWT_TYPE
|
login_submission["type"] == LoginRestServlet.JWT_TYPE
|
||||||
):
|
):
|
||||||
result = yield self.do_jwt_login(login_submission)
|
result = await self.do_jwt_login(login_submission)
|
||||||
elif login_submission["type"] == LoginRestServlet.TOKEN_TYPE:
|
elif login_submission["type"] == LoginRestServlet.TOKEN_TYPE:
|
||||||
result = yield self.do_token_login(login_submission)
|
result = await self.do_token_login(login_submission)
|
||||||
else:
|
else:
|
||||||
result = yield self._do_other_login(login_submission)
|
result = await self._do_other_login(login_submission)
|
||||||
except KeyError:
|
except KeyError:
|
||||||
raise SynapseError(400, "Missing JSON keys.")
|
raise SynapseError(400, "Missing JSON keys.")
|
||||||
|
|
||||||
|
|
@ -158,8 +156,7 @@ class LoginRestServlet(RestServlet):
|
||||||
result["well_known"] = well_known_data
|
result["well_known"] = well_known_data
|
||||||
return 200, result
|
return 200, result
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _do_other_login(self, login_submission):
|
||||||
def _do_other_login(self, login_submission):
|
|
||||||
"""Handle non-token/saml/jwt logins
|
"""Handle non-token/saml/jwt logins
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
|
|
@ -219,20 +216,20 @@ class LoginRestServlet(RestServlet):
|
||||||
(
|
(
|
||||||
canonical_user_id,
|
canonical_user_id,
|
||||||
callback_3pid,
|
callback_3pid,
|
||||||
) = yield self.auth_handler.check_password_provider_3pid(
|
) = await self.auth_handler.check_password_provider_3pid(
|
||||||
medium, address, login_submission["password"]
|
medium, address, login_submission["password"]
|
||||||
)
|
)
|
||||||
if canonical_user_id:
|
if canonical_user_id:
|
||||||
# Authentication through password provider and 3pid succeeded
|
# Authentication through password provider and 3pid succeeded
|
||||||
|
|
||||||
result = yield self._complete_login(
|
result = await self._complete_login(
|
||||||
canonical_user_id, login_submission, callback_3pid
|
canonical_user_id, login_submission, callback_3pid
|
||||||
)
|
)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
# No password providers were able to handle this 3pid
|
# No password providers were able to handle this 3pid
|
||||||
# Check local store
|
# Check local store
|
||||||
user_id = yield self.hs.get_datastore().get_user_id_by_threepid(
|
user_id = await self.hs.get_datastore().get_user_id_by_threepid(
|
||||||
medium, address
|
medium, address
|
||||||
)
|
)
|
||||||
if not user_id:
|
if not user_id:
|
||||||
|
|
@ -280,7 +277,7 @@ class LoginRestServlet(RestServlet):
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
canonical_user_id, callback = yield self.auth_handler.validate_login(
|
canonical_user_id, callback = await self.auth_handler.validate_login(
|
||||||
identifier["user"], login_submission
|
identifier["user"], login_submission
|
||||||
)
|
)
|
||||||
except LoginError:
|
except LoginError:
|
||||||
|
|
@ -297,13 +294,12 @@ class LoginRestServlet(RestServlet):
|
||||||
)
|
)
|
||||||
raise
|
raise
|
||||||
|
|
||||||
result = yield self._complete_login(
|
result = await self._complete_login(
|
||||||
canonical_user_id, login_submission, callback
|
canonical_user_id, login_submission, callback
|
||||||
)
|
)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def _complete_login(
|
||||||
def _complete_login(
|
|
||||||
self, user_id, login_submission, callback=None, create_non_existant_users=False
|
self, user_id, login_submission, callback=None, create_non_existant_users=False
|
||||||
):
|
):
|
||||||
"""Called when we've successfully authed the user and now need to
|
"""Called when we've successfully authed the user and now need to
|
||||||
|
|
@ -337,15 +333,15 @@ class LoginRestServlet(RestServlet):
|
||||||
)
|
)
|
||||||
|
|
||||||
if create_non_existant_users:
|
if create_non_existant_users:
|
||||||
user_id = yield self.auth_handler.check_user_exists(user_id)
|
user_id = await self.auth_handler.check_user_exists(user_id)
|
||||||
if not user_id:
|
if not user_id:
|
||||||
user_id = yield self.registration_handler.register_user(
|
user_id = await self.registration_handler.register_user(
|
||||||
localpart=UserID.from_string(user_id).localpart
|
localpart=UserID.from_string(user_id).localpart
|
||||||
)
|
)
|
||||||
|
|
||||||
device_id = login_submission.get("device_id")
|
device_id = login_submission.get("device_id")
|
||||||
initial_display_name = login_submission.get("initial_device_display_name")
|
initial_display_name = login_submission.get("initial_device_display_name")
|
||||||
device_id, access_token = yield self.registration_handler.register_device(
|
device_id, access_token = await self.registration_handler.register_device(
|
||||||
user_id, device_id, initial_display_name
|
user_id, device_id, initial_display_name
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -357,23 +353,21 @@ class LoginRestServlet(RestServlet):
|
||||||
}
|
}
|
||||||
|
|
||||||
if callback is not None:
|
if callback is not None:
|
||||||
yield callback(result)
|
await callback(result)
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def do_token_login(self, login_submission):
|
||||||
def do_token_login(self, login_submission):
|
|
||||||
token = login_submission["token"]
|
token = login_submission["token"]
|
||||||
auth_handler = self.auth_handler
|
auth_handler = self.auth_handler
|
||||||
user_id = yield auth_handler.validate_short_term_login_token_and_get_user_id(
|
user_id = await auth_handler.validate_short_term_login_token_and_get_user_id(
|
||||||
token
|
token
|
||||||
)
|
)
|
||||||
|
|
||||||
result = yield self._complete_login(user_id, login_submission)
|
result = await self._complete_login(user_id, login_submission)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def do_jwt_login(self, login_submission):
|
||||||
def do_jwt_login(self, login_submission):
|
|
||||||
token = login_submission.get("token", None)
|
token = login_submission.get("token", None)
|
||||||
if token is None:
|
if token is None:
|
||||||
raise LoginError(
|
raise LoginError(
|
||||||
|
|
@ -397,7 +391,7 @@ class LoginRestServlet(RestServlet):
|
||||||
raise LoginError(401, "Invalid JWT", errcode=Codes.UNAUTHORIZED)
|
raise LoginError(401, "Invalid JWT", errcode=Codes.UNAUTHORIZED)
|
||||||
|
|
||||||
user_id = UserID(user, self.hs.hostname).to_string()
|
user_id = UserID(user, self.hs.hostname).to_string()
|
||||||
result = yield self._complete_login(
|
result = await self._complete_login(
|
||||||
user_id, login_submission, create_non_existant_users=True
|
user_id, login_submission, create_non_existant_users=True
|
||||||
)
|
)
|
||||||
return result
|
return result
|
||||||
|
|
@ -460,8 +454,7 @@ class CasTicketServlet(RestServlet):
|
||||||
self._sso_auth_handler = SSOAuthHandler(hs)
|
self._sso_auth_handler = SSOAuthHandler(hs)
|
||||||
self._http_client = hs.get_proxied_http_client()
|
self._http_client = hs.get_proxied_http_client()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
|
||||||
client_redirect_url = parse_string(request, "redirectUrl", required=True)
|
client_redirect_url = parse_string(request, "redirectUrl", required=True)
|
||||||
uri = self.cas_server_url + "/proxyValidate"
|
uri = self.cas_server_url + "/proxyValidate"
|
||||||
args = {
|
args = {
|
||||||
|
|
@ -469,12 +462,12 @@ class CasTicketServlet(RestServlet):
|
||||||
"service": self.cas_service_url,
|
"service": self.cas_service_url,
|
||||||
}
|
}
|
||||||
try:
|
try:
|
||||||
body = yield self._http_client.get_raw(uri, args)
|
body = await self._http_client.get_raw(uri, args)
|
||||||
except PartialDownloadError as pde:
|
except PartialDownloadError as pde:
|
||||||
# Twisted raises this error if the connection is closed,
|
# Twisted raises this error if the connection is closed,
|
||||||
# even if that's being used old-http style to signal end-of-data
|
# even if that's being used old-http style to signal end-of-data
|
||||||
body = pde.response
|
body = pde.response
|
||||||
result = yield self.handle_cas_response(request, body, client_redirect_url)
|
result = await self.handle_cas_response(request, body, client_redirect_url)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def handle_cas_response(self, request, cas_response_body, client_redirect_url):
|
def handle_cas_response(self, request, cas_response_body, client_redirect_url):
|
||||||
|
|
@ -555,8 +548,7 @@ class SSOAuthHandler(object):
|
||||||
self._registration_handler = hs.get_registration_handler()
|
self._registration_handler = hs.get_registration_handler()
|
||||||
self._macaroon_gen = hs.get_macaroon_generator()
|
self._macaroon_gen = hs.get_macaroon_generator()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_successful_auth(
|
||||||
def on_successful_auth(
|
|
||||||
self, username, request, client_redirect_url, user_display_name=None
|
self, username, request, client_redirect_url, user_display_name=None
|
||||||
):
|
):
|
||||||
"""Called once the user has successfully authenticated with the SSO.
|
"""Called once the user has successfully authenticated with the SSO.
|
||||||
|
|
@ -582,9 +574,9 @@ class SSOAuthHandler(object):
|
||||||
"""
|
"""
|
||||||
localpart = map_username_to_mxid_localpart(username)
|
localpart = map_username_to_mxid_localpart(username)
|
||||||
user_id = UserID(localpart, self._hostname).to_string()
|
user_id = UserID(localpart, self._hostname).to_string()
|
||||||
registered_user_id = yield self._auth_handler.check_user_exists(user_id)
|
registered_user_id = await self._auth_handler.check_user_exists(user_id)
|
||||||
if not registered_user_id:
|
if not registered_user_id:
|
||||||
registered_user_id = yield self._registration_handler.register_user(
|
registered_user_id = await self._registration_handler.register_user(
|
||||||
localpart=localpart, default_display_name=user_display_name
|
localpart=localpart, default_display_name=user_display_name
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,8 +15,6 @@
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.http.servlet import RestServlet
|
from synapse.http.servlet import RestServlet
|
||||||
from synapse.rest.client.v2_alpha._base import client_patterns
|
from synapse.rest.client.v2_alpha._base import client_patterns
|
||||||
|
|
||||||
|
|
@ -35,17 +33,16 @@ class LogoutRestServlet(RestServlet):
|
||||||
def on_OPTIONS(self, request):
|
def on_OPTIONS(self, request):
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
|
|
||||||
if requester.device_id is None:
|
if requester.device_id is None:
|
||||||
# the acccess token wasn't associated with a device.
|
# the acccess token wasn't associated with a device.
|
||||||
# Just delete the access token
|
# Just delete the access token
|
||||||
access_token = self.auth.get_access_token_from_request(request)
|
access_token = self.auth.get_access_token_from_request(request)
|
||||||
yield self._auth_handler.delete_access_token(access_token)
|
await self._auth_handler.delete_access_token(access_token)
|
||||||
else:
|
else:
|
||||||
yield self._device_handler.delete_device(
|
await self._device_handler.delete_device(
|
||||||
requester.user.to_string(), requester.device_id
|
requester.user.to_string(), requester.device_id
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -64,17 +61,16 @@ class LogoutAllRestServlet(RestServlet):
|
||||||
def on_OPTIONS(self, request):
|
def on_OPTIONS(self, request):
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
# first delete all of the user's devices
|
# first delete all of the user's devices
|
||||||
yield self._device_handler.delete_all_devices_for_user(user_id)
|
await self._device_handler.delete_all_devices_for_user(user_id)
|
||||||
|
|
||||||
# .. and then delete any access tokens which weren't associated with
|
# .. and then delete any access tokens which weren't associated with
|
||||||
# devices.
|
# devices.
|
||||||
yield self._auth_handler.delete_access_tokens_for_user(user_id)
|
await self._auth_handler.delete_access_tokens_for_user(user_id)
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -19,8 +19,6 @@ import logging
|
||||||
|
|
||||||
from six import string_types
|
from six import string_types
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import AuthError, SynapseError
|
from synapse.api.errors import AuthError, SynapseError
|
||||||
from synapse.handlers.presence import format_user_presence_state
|
from synapse.handlers.presence import format_user_presence_state
|
||||||
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
||||||
|
|
@ -40,27 +38,25 @@ class PresenceStatusRestServlet(RestServlet):
|
||||||
self.clock = hs.get_clock()
|
self.clock = hs.get_clock()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, user_id):
|
||||||
def on_GET(self, request, user_id):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
|
|
||||||
if requester.user != user:
|
if requester.user != user:
|
||||||
allowed = yield self.presence_handler.is_visible(
|
allowed = await self.presence_handler.is_visible(
|
||||||
observed_user=user, observer_user=requester.user
|
observed_user=user, observer_user=requester.user
|
||||||
)
|
)
|
||||||
|
|
||||||
if not allowed:
|
if not allowed:
|
||||||
raise AuthError(403, "You are not allowed to see their presence.")
|
raise AuthError(403, "You are not allowed to see their presence.")
|
||||||
|
|
||||||
state = yield self.presence_handler.get_state(target_user=user)
|
state = await self.presence_handler.get_state(target_user=user)
|
||||||
state = format_user_presence_state(state, self.clock.time_msec())
|
state = format_user_presence_state(state, self.clock.time_msec())
|
||||||
|
|
||||||
return 200, state
|
return 200, state
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, user_id):
|
||||||
def on_PUT(self, request, user_id):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
|
|
||||||
if requester.user != user:
|
if requester.user != user:
|
||||||
|
|
@ -86,7 +82,7 @@ class PresenceStatusRestServlet(RestServlet):
|
||||||
raise SynapseError(400, "Unable to parse state")
|
raise SynapseError(400, "Unable to parse state")
|
||||||
|
|
||||||
if self.hs.config.use_presence:
|
if self.hs.config.use_presence:
|
||||||
yield self.presence_handler.set_state(user, state)
|
await self.presence_handler.set_state(user, state)
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -14,8 +14,8 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
""" This module contains REST servlets to do with profile: /profile/<paths> """
|
""" This module contains REST servlets to do with profile: /profile/<paths> """
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
|
from synapse.api.errors import Codes, SynapseError
|
||||||
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
||||||
from synapse.rest.client.v2_alpha._base import client_patterns
|
from synapse.rest.client.v2_alpha._base import client_patterns
|
||||||
from synapse.types import UserID
|
from synapse.types import UserID
|
||||||
|
|
@ -30,19 +30,18 @@ class ProfileDisplaynameRestServlet(RestServlet):
|
||||||
self.profile_handler = hs.get_profile_handler()
|
self.profile_handler = hs.get_profile_handler()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, user_id):
|
||||||
def on_GET(self, request, user_id):
|
|
||||||
requester_user = None
|
requester_user = None
|
||||||
|
|
||||||
if self.hs.config.require_auth_for_profile_requests:
|
if self.hs.config.require_auth_for_profile_requests:
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester_user = requester.user
|
requester_user = requester.user
|
||||||
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
|
|
||||||
yield self.profile_handler.check_profile_query_allowed(user, requester_user)
|
await self.profile_handler.check_profile_query_allowed(user, requester_user)
|
||||||
|
|
||||||
displayname = yield self.profile_handler.get_displayname(user)
|
displayname = await self.profile_handler.get_displayname(user)
|
||||||
|
|
||||||
ret = {}
|
ret = {}
|
||||||
if displayname is not None:
|
if displayname is not None:
|
||||||
|
|
@ -50,11 +49,10 @@ class ProfileDisplaynameRestServlet(RestServlet):
|
||||||
|
|
||||||
return 200, ret
|
return 200, ret
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, user_id):
|
||||||
def on_PUT(self, request, user_id):
|
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||||
requester = yield self.auth.get_user_by_req(request, allow_guest=True)
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
is_admin = yield self.auth.is_server_admin(requester.user)
|
is_admin = await self.auth.is_server_admin(requester.user)
|
||||||
|
|
||||||
content = parse_json_object_from_request(request)
|
content = parse_json_object_from_request(request)
|
||||||
|
|
||||||
|
|
@ -63,7 +61,7 @@ class ProfileDisplaynameRestServlet(RestServlet):
|
||||||
except Exception:
|
except Exception:
|
||||||
return 400, "Unable to parse name"
|
return 400, "Unable to parse name"
|
||||||
|
|
||||||
yield self.profile_handler.set_displayname(user, requester, new_name, is_admin)
|
await self.profile_handler.set_displayname(user, requester, new_name, is_admin)
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
|
|
@ -80,19 +78,18 @@ class ProfileAvatarURLRestServlet(RestServlet):
|
||||||
self.profile_handler = hs.get_profile_handler()
|
self.profile_handler = hs.get_profile_handler()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, user_id):
|
||||||
def on_GET(self, request, user_id):
|
|
||||||
requester_user = None
|
requester_user = None
|
||||||
|
|
||||||
if self.hs.config.require_auth_for_profile_requests:
|
if self.hs.config.require_auth_for_profile_requests:
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester_user = requester.user
|
requester_user = requester.user
|
||||||
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
|
|
||||||
yield self.profile_handler.check_profile_query_allowed(user, requester_user)
|
await self.profile_handler.check_profile_query_allowed(user, requester_user)
|
||||||
|
|
||||||
avatar_url = yield self.profile_handler.get_avatar_url(user)
|
avatar_url = await self.profile_handler.get_avatar_url(user)
|
||||||
|
|
||||||
ret = {}
|
ret = {}
|
||||||
if avatar_url is not None:
|
if avatar_url is not None:
|
||||||
|
|
@ -100,19 +97,22 @@ class ProfileAvatarURLRestServlet(RestServlet):
|
||||||
|
|
||||||
return 200, ret
|
return 200, ret
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, user_id):
|
||||||
def on_PUT(self, request, user_id):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
is_admin = yield self.auth.is_server_admin(requester.user)
|
is_admin = await self.auth.is_server_admin(requester.user)
|
||||||
|
|
||||||
content = parse_json_object_from_request(request)
|
content = parse_json_object_from_request(request)
|
||||||
try:
|
try:
|
||||||
new_name = content["avatar_url"]
|
new_avatar_url = content["avatar_url"]
|
||||||
except Exception:
|
except KeyError:
|
||||||
return 400, "Unable to parse name"
|
raise SynapseError(
|
||||||
|
400, "Missing key 'avatar_url'", errcode=Codes.MISSING_PARAM
|
||||||
|
)
|
||||||
|
|
||||||
yield self.profile_handler.set_avatar_url(user, requester, new_name, is_admin)
|
await self.profile_handler.set_avatar_url(
|
||||||
|
user, requester, new_avatar_url, is_admin
|
||||||
|
)
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
|
|
@ -129,20 +129,19 @@ class ProfileRestServlet(RestServlet):
|
||||||
self.profile_handler = hs.get_profile_handler()
|
self.profile_handler = hs.get_profile_handler()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, user_id):
|
||||||
def on_GET(self, request, user_id):
|
|
||||||
requester_user = None
|
requester_user = None
|
||||||
|
|
||||||
if self.hs.config.require_auth_for_profile_requests:
|
if self.hs.config.require_auth_for_profile_requests:
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester_user = requester.user
|
requester_user = requester.user
|
||||||
|
|
||||||
user = UserID.from_string(user_id)
|
user = UserID.from_string(user_id)
|
||||||
|
|
||||||
yield self.profile_handler.check_profile_query_allowed(user, requester_user)
|
await self.profile_handler.check_profile_query_allowed(user, requester_user)
|
||||||
|
|
||||||
displayname = yield self.profile_handler.get_displayname(user)
|
displayname = await self.profile_handler.get_displayname(user)
|
||||||
avatar_url = yield self.profile_handler.get_avatar_url(user)
|
avatar_url = await self.profile_handler.get_avatar_url(user)
|
||||||
|
|
||||||
ret = {}
|
ret = {}
|
||||||
if displayname is not None:
|
if displayname is not None:
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,6 @@
|
||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import (
|
from synapse.api.errors import (
|
||||||
NotFoundError,
|
NotFoundError,
|
||||||
|
|
@ -46,8 +45,7 @@ class PushRuleRestServlet(RestServlet):
|
||||||
self.notifier = hs.get_notifier()
|
self.notifier = hs.get_notifier()
|
||||||
self._is_worker = hs.config.worker_app is not None
|
self._is_worker = hs.config.worker_app is not None
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, path):
|
||||||
def on_PUT(self, request, path):
|
|
||||||
if self._is_worker:
|
if self._is_worker:
|
||||||
raise Exception("Cannot handle PUT /push_rules on worker")
|
raise Exception("Cannot handle PUT /push_rules on worker")
|
||||||
|
|
||||||
|
|
@ -57,7 +55,7 @@ class PushRuleRestServlet(RestServlet):
|
||||||
except InvalidRuleException as e:
|
except InvalidRuleException as e:
|
||||||
raise SynapseError(400, str(e))
|
raise SynapseError(400, str(e))
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
|
|
||||||
if "/" in spec["rule_id"] or "\\" in spec["rule_id"]:
|
if "/" in spec["rule_id"] or "\\" in spec["rule_id"]:
|
||||||
raise SynapseError(400, "rule_id may not contain slashes")
|
raise SynapseError(400, "rule_id may not contain slashes")
|
||||||
|
|
@ -67,7 +65,7 @@ class PushRuleRestServlet(RestServlet):
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
if "attr" in spec:
|
if "attr" in spec:
|
||||||
yield self.set_rule_attr(user_id, spec, content)
|
await self.set_rule_attr(user_id, spec, content)
|
||||||
self.notify_user(user_id)
|
self.notify_user(user_id)
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
|
|
@ -91,7 +89,7 @@ class PushRuleRestServlet(RestServlet):
|
||||||
after = _namespaced_rule_id(spec, after)
|
after = _namespaced_rule_id(spec, after)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
yield self.store.add_push_rule(
|
await self.store.add_push_rule(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
rule_id=_namespaced_rule_id_from_spec(spec),
|
rule_id=_namespaced_rule_id_from_spec(spec),
|
||||||
priority_class=priority_class,
|
priority_class=priority_class,
|
||||||
|
|
@ -108,20 +106,19 @@ class PushRuleRestServlet(RestServlet):
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_DELETE(self, request, path):
|
||||||
def on_DELETE(self, request, path):
|
|
||||||
if self._is_worker:
|
if self._is_worker:
|
||||||
raise Exception("Cannot handle DELETE /push_rules on worker")
|
raise Exception("Cannot handle DELETE /push_rules on worker")
|
||||||
|
|
||||||
spec = _rule_spec_from_path([x for x in path.split("/")])
|
spec = _rule_spec_from_path([x for x in path.split("/")])
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
namespaced_rule_id = _namespaced_rule_id_from_spec(spec)
|
namespaced_rule_id = _namespaced_rule_id_from_spec(spec)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
yield self.store.delete_push_rule(user_id, namespaced_rule_id)
|
await self.store.delete_push_rule(user_id, namespaced_rule_id)
|
||||||
self.notify_user(user_id)
|
self.notify_user(user_id)
|
||||||
return 200, {}
|
return 200, {}
|
||||||
except StoreError as e:
|
except StoreError as e:
|
||||||
|
|
@ -130,15 +127,14 @@ class PushRuleRestServlet(RestServlet):
|
||||||
else:
|
else:
|
||||||
raise
|
raise
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, path):
|
||||||
def on_GET(self, request, path):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
# we build up the full structure and then decide which bits of it
|
# we build up the full structure and then decide which bits of it
|
||||||
# to send which means doing unnecessary work sometimes but is
|
# to send which means doing unnecessary work sometimes but is
|
||||||
# is probably not going to make a whole lot of difference
|
# is probably not going to make a whole lot of difference
|
||||||
rules = yield self.store.get_push_rules_for_user(user_id)
|
rules = await self.store.get_push_rules_for_user(user_id)
|
||||||
|
|
||||||
rules = format_push_rules_for_user(requester.user, rules)
|
rules = format_push_rules_for_user(requester.user, rules)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,8 +15,6 @@
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import Codes, StoreError, SynapseError
|
from synapse.api.errors import Codes, StoreError, SynapseError
|
||||||
from synapse.http.server import finish_request
|
from synapse.http.server import finish_request
|
||||||
from synapse.http.servlet import (
|
from synapse.http.servlet import (
|
||||||
|
|
@ -39,12 +37,11 @@ class PushersRestServlet(RestServlet):
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user = requester.user
|
user = requester.user
|
||||||
|
|
||||||
pushers = yield self.hs.get_datastore().get_pushers_by_user_id(user.to_string())
|
pushers = await self.hs.get_datastore().get_pushers_by_user_id(user.to_string())
|
||||||
|
|
||||||
allowed_keys = [
|
allowed_keys = [
|
||||||
"app_display_name",
|
"app_display_name",
|
||||||
|
|
@ -78,9 +75,8 @@ class PushersSetRestServlet(RestServlet):
|
||||||
self.notifier = hs.get_notifier()
|
self.notifier = hs.get_notifier()
|
||||||
self.pusher_pool = self.hs.get_pusherpool()
|
self.pusher_pool = self.hs.get_pusherpool()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user = requester.user
|
user = requester.user
|
||||||
|
|
||||||
content = parse_json_object_from_request(request)
|
content = parse_json_object_from_request(request)
|
||||||
|
|
@ -91,7 +87,7 @@ class PushersSetRestServlet(RestServlet):
|
||||||
and "kind" in content
|
and "kind" in content
|
||||||
and content["kind"] is None
|
and content["kind"] is None
|
||||||
):
|
):
|
||||||
yield self.pusher_pool.remove_pusher(
|
await self.pusher_pool.remove_pusher(
|
||||||
content["app_id"], content["pushkey"], user_id=user.to_string()
|
content["app_id"], content["pushkey"], user_id=user.to_string()
|
||||||
)
|
)
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
@ -117,14 +113,14 @@ class PushersSetRestServlet(RestServlet):
|
||||||
append = content["append"]
|
append = content["append"]
|
||||||
|
|
||||||
if not append:
|
if not append:
|
||||||
yield self.pusher_pool.remove_pushers_by_app_id_and_pushkey_not_user(
|
await self.pusher_pool.remove_pushers_by_app_id_and_pushkey_not_user(
|
||||||
app_id=content["app_id"],
|
app_id=content["app_id"],
|
||||||
pushkey=content["pushkey"],
|
pushkey=content["pushkey"],
|
||||||
not_user_id=user.to_string(),
|
not_user_id=user.to_string(),
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
yield self.pusher_pool.add_pusher(
|
await self.pusher_pool.add_pusher(
|
||||||
user_id=user.to_string(),
|
user_id=user.to_string(),
|
||||||
access_token=requester.access_token_id,
|
access_token=requester.access_token_id,
|
||||||
kind=content["kind"],
|
kind=content["kind"],
|
||||||
|
|
@ -164,16 +160,15 @@ class PushersRemoveRestServlet(RestServlet):
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.pusher_pool = self.hs.get_pusherpool()
|
self.pusher_pool = self.hs.get_pusherpool()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(request, rights="delete_pusher")
|
||||||
requester = yield self.auth.get_user_by_req(request, rights="delete_pusher")
|
|
||||||
user = requester.user
|
user = requester.user
|
||||||
|
|
||||||
app_id = parse_string(request, "app_id", required=True)
|
app_id = parse_string(request, "app_id", required=True)
|
||||||
pushkey = parse_string(request, "pushkey", required=True)
|
pushkey = parse_string(request, "pushkey", required=True)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
yield self.pusher_pool.remove_pusher(
|
await self.pusher_pool.remove_pusher(
|
||||||
app_id=app_id, pushkey=pushkey, user_id=user.to_string()
|
app_id=app_id, pushkey=pushkey, user_id=user.to_string()
|
||||||
)
|
)
|
||||||
except StoreError as se:
|
except StoreError as se:
|
||||||
|
|
|
||||||
|
|
@ -714,7 +714,7 @@ class RoomMembershipRestServlet(TransactionRestServlet):
|
||||||
target = UserID.from_string(content["user_id"])
|
target = UserID.from_string(content["user_id"])
|
||||||
|
|
||||||
event_content = None
|
event_content = None
|
||||||
if "reason" in content and membership_action in ["kick", "ban"]:
|
if "reason" in content:
|
||||||
event_content = {"reason": content["reason"]}
|
event_content = {"reason": content["reason"]}
|
||||||
|
|
||||||
await self.room_member_handler.update_membership(
|
await self.room_member_handler.update_membership(
|
||||||
|
|
|
||||||
|
|
@ -17,8 +17,6 @@ import base64
|
||||||
import hashlib
|
import hashlib
|
||||||
import hmac
|
import hmac
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.http.servlet import RestServlet
|
from synapse.http.servlet import RestServlet
|
||||||
from synapse.rest.client.v2_alpha._base import client_patterns
|
from synapse.rest.client.v2_alpha._base import client_patterns
|
||||||
|
|
||||||
|
|
@ -31,9 +29,8 @@ class VoipRestServlet(RestServlet):
|
||||||
self.hs = hs
|
self.hs = hs
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(
|
||||||
requester = yield self.auth.get_user_by_req(
|
|
||||||
request, self.hs.config.turn_allow_guests
|
request, self.hs.config.turn_allow_guests
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -78,7 +78,7 @@ def interactive_auth_handler(orig):
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def wrapped(*args, **kwargs):
|
def wrapped(*args, **kwargs):
|
||||||
res = defer.maybeDeferred(orig, *args, **kwargs)
|
res = defer.ensureDeferred(orig(*args, **kwargs))
|
||||||
res.addErrback(_catch_incomplete_interactive_auth)
|
res.addErrback(_catch_incomplete_interactive_auth)
|
||||||
return res
|
return res
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,8 +18,6 @@ import logging
|
||||||
|
|
||||||
from six.moves import http_client
|
from six.moves import http_client
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.constants import LoginType
|
from synapse.api.constants import LoginType
|
||||||
from synapse.api.errors import Codes, SynapseError, ThreepidValidationError
|
from synapse.api.errors import Codes, SynapseError, ThreepidValidationError
|
||||||
from synapse.config.emailconfig import ThreepidBehaviour
|
from synapse.config.emailconfig import ThreepidBehaviour
|
||||||
|
|
@ -67,8 +65,7 @@ class EmailPasswordRequestTokenRestServlet(RestServlet):
|
||||||
template_text=template_text,
|
template_text=template_text,
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
|
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
|
||||||
if self.config.local_threepid_handling_disabled_due_to_email_config:
|
if self.config.local_threepid_handling_disabled_due_to_email_config:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
|
|
@ -95,7 +92,7 @@ class EmailPasswordRequestTokenRestServlet(RestServlet):
|
||||||
Codes.THREEPID_DENIED,
|
Codes.THREEPID_DENIED,
|
||||||
)
|
)
|
||||||
|
|
||||||
existing_user_id = yield self.hs.get_datastore().get_user_id_by_threepid(
|
existing_user_id = await self.hs.get_datastore().get_user_id_by_threepid(
|
||||||
"email", email
|
"email", email
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -106,7 +103,7 @@ class EmailPasswordRequestTokenRestServlet(RestServlet):
|
||||||
assert self.hs.config.account_threepid_delegate_email
|
assert self.hs.config.account_threepid_delegate_email
|
||||||
|
|
||||||
# Have the configured identity server handle the request
|
# Have the configured identity server handle the request
|
||||||
ret = yield self.identity_handler.requestEmailToken(
|
ret = await self.identity_handler.requestEmailToken(
|
||||||
self.hs.config.account_threepid_delegate_email,
|
self.hs.config.account_threepid_delegate_email,
|
||||||
email,
|
email,
|
||||||
client_secret,
|
client_secret,
|
||||||
|
|
@ -115,7 +112,7 @@ class EmailPasswordRequestTokenRestServlet(RestServlet):
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# Send password reset emails from Synapse
|
# Send password reset emails from Synapse
|
||||||
sid = yield self.identity_handler.send_threepid_validation(
|
sid = await self.identity_handler.send_threepid_validation(
|
||||||
email,
|
email,
|
||||||
client_secret,
|
client_secret,
|
||||||
send_attempt,
|
send_attempt,
|
||||||
|
|
@ -153,8 +150,7 @@ class PasswordResetSubmitTokenServlet(RestServlet):
|
||||||
[self.config.email_password_reset_template_failure_html],
|
[self.config.email_password_reset_template_failure_html],
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, medium):
|
||||||
def on_GET(self, request, medium):
|
|
||||||
# We currently only handle threepid token submissions for email
|
# We currently only handle threepid token submissions for email
|
||||||
if medium != "email":
|
if medium != "email":
|
||||||
raise SynapseError(
|
raise SynapseError(
|
||||||
|
|
@ -176,7 +172,7 @@ class PasswordResetSubmitTokenServlet(RestServlet):
|
||||||
# Attempt to validate a 3PID session
|
# Attempt to validate a 3PID session
|
||||||
try:
|
try:
|
||||||
# Mark the session as valid
|
# Mark the session as valid
|
||||||
next_link = yield self.store.validate_threepid_session(
|
next_link = await self.store.validate_threepid_session(
|
||||||
sid, client_secret, token, self.clock.time_msec()
|
sid, client_secret, token, self.clock.time_msec()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -218,8 +214,7 @@ class PasswordRestServlet(RestServlet):
|
||||||
self._set_password_handler = hs.get_set_password_handler()
|
self._set_password_handler = hs.get_set_password_handler()
|
||||||
|
|
||||||
@interactive_auth_handler
|
@interactive_auth_handler
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
|
|
||||||
# there are two possibilities here. Either the user does not have an
|
# there are two possibilities here. Either the user does not have an
|
||||||
|
|
@ -233,14 +228,14 @@ class PasswordRestServlet(RestServlet):
|
||||||
# In the second case, we require a password to confirm their identity.
|
# In the second case, we require a password to confirm their identity.
|
||||||
|
|
||||||
if self.auth.has_access_token(request):
|
if self.auth.has_access_token(request):
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
params = yield self.auth_handler.validate_user_via_ui_auth(
|
params = await self.auth_handler.validate_user_via_ui_auth(
|
||||||
requester, body, self.hs.get_ip_from_request(request)
|
requester, body, self.hs.get_ip_from_request(request)
|
||||||
)
|
)
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
else:
|
else:
|
||||||
requester = None
|
requester = None
|
||||||
result, params, _ = yield self.auth_handler.check_auth(
|
result, params, _ = await self.auth_handler.check_auth(
|
||||||
[[LoginType.EMAIL_IDENTITY]], body, self.hs.get_ip_from_request(request)
|
[[LoginType.EMAIL_IDENTITY]], body, self.hs.get_ip_from_request(request)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -254,7 +249,7 @@ class PasswordRestServlet(RestServlet):
|
||||||
# (See add_threepid in synapse/handlers/auth.py)
|
# (See add_threepid in synapse/handlers/auth.py)
|
||||||
threepid["address"] = threepid["address"].lower()
|
threepid["address"] = threepid["address"].lower()
|
||||||
# if using email, we must know about the email they're authing with!
|
# if using email, we must know about the email they're authing with!
|
||||||
threepid_user_id = yield self.datastore.get_user_id_by_threepid(
|
threepid_user_id = await self.datastore.get_user_id_by_threepid(
|
||||||
threepid["medium"], threepid["address"]
|
threepid["medium"], threepid["address"]
|
||||||
)
|
)
|
||||||
if not threepid_user_id:
|
if not threepid_user_id:
|
||||||
|
|
@ -267,7 +262,7 @@ class PasswordRestServlet(RestServlet):
|
||||||
assert_params_in_dict(params, ["new_password"])
|
assert_params_in_dict(params, ["new_password"])
|
||||||
new_password = params["new_password"]
|
new_password = params["new_password"]
|
||||||
|
|
||||||
yield self._set_password_handler.set_password(user_id, new_password, requester)
|
await self._set_password_handler.set_password(user_id, new_password, requester)
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
|
|
@ -286,8 +281,7 @@ class DeactivateAccountRestServlet(RestServlet):
|
||||||
self._deactivate_account_handler = hs.get_deactivate_account_handler()
|
self._deactivate_account_handler = hs.get_deactivate_account_handler()
|
||||||
|
|
||||||
@interactive_auth_handler
|
@interactive_auth_handler
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
erase = body.get("erase", False)
|
erase = body.get("erase", False)
|
||||||
if not isinstance(erase, bool):
|
if not isinstance(erase, bool):
|
||||||
|
|
@ -297,19 +291,19 @@ class DeactivateAccountRestServlet(RestServlet):
|
||||||
Codes.BAD_JSON,
|
Codes.BAD_JSON,
|
||||||
)
|
)
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
|
|
||||||
# allow ASes to dectivate their own users
|
# allow ASes to dectivate their own users
|
||||||
if requester.app_service:
|
if requester.app_service:
|
||||||
yield self._deactivate_account_handler.deactivate_account(
|
await self._deactivate_account_handler.deactivate_account(
|
||||||
requester.user.to_string(), erase
|
requester.user.to_string(), erase
|
||||||
)
|
)
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
yield self.auth_handler.validate_user_via_ui_auth(
|
await self.auth_handler.validate_user_via_ui_auth(
|
||||||
requester, body, self.hs.get_ip_from_request(request)
|
requester, body, self.hs.get_ip_from_request(request)
|
||||||
)
|
)
|
||||||
result = yield self._deactivate_account_handler.deactivate_account(
|
result = await self._deactivate_account_handler.deactivate_account(
|
||||||
requester.user.to_string(), erase, id_server=body.get("id_server")
|
requester.user.to_string(), erase, id_server=body.get("id_server")
|
||||||
)
|
)
|
||||||
if result:
|
if result:
|
||||||
|
|
@ -346,8 +340,7 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
|
||||||
template_text=template_text,
|
template_text=template_text,
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
|
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
|
||||||
if self.config.local_threepid_handling_disabled_due_to_email_config:
|
if self.config.local_threepid_handling_disabled_due_to_email_config:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
|
|
@ -371,7 +364,7 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
|
||||||
Codes.THREEPID_DENIED,
|
Codes.THREEPID_DENIED,
|
||||||
)
|
)
|
||||||
|
|
||||||
existing_user_id = yield self.store.get_user_id_by_threepid(
|
existing_user_id = await self.store.get_user_id_by_threepid(
|
||||||
"email", body["email"]
|
"email", body["email"]
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -382,7 +375,7 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
|
||||||
assert self.hs.config.account_threepid_delegate_email
|
assert self.hs.config.account_threepid_delegate_email
|
||||||
|
|
||||||
# Have the configured identity server handle the request
|
# Have the configured identity server handle the request
|
||||||
ret = yield self.identity_handler.requestEmailToken(
|
ret = await self.identity_handler.requestEmailToken(
|
||||||
self.hs.config.account_threepid_delegate_email,
|
self.hs.config.account_threepid_delegate_email,
|
||||||
email,
|
email,
|
||||||
client_secret,
|
client_secret,
|
||||||
|
|
@ -391,7 +384,7 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# Send threepid validation emails from Synapse
|
# Send threepid validation emails from Synapse
|
||||||
sid = yield self.identity_handler.send_threepid_validation(
|
sid = await self.identity_handler.send_threepid_validation(
|
||||||
email,
|
email,
|
||||||
client_secret,
|
client_secret,
|
||||||
send_attempt,
|
send_attempt,
|
||||||
|
|
@ -414,8 +407,7 @@ class MsisdnThreepidRequestTokenRestServlet(RestServlet):
|
||||||
self.store = self.hs.get_datastore()
|
self.store = self.hs.get_datastore()
|
||||||
self.identity_handler = hs.get_handlers().identity_handler
|
self.identity_handler = hs.get_handlers().identity_handler
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
assert_params_in_dict(
|
assert_params_in_dict(
|
||||||
body, ["client_secret", "country", "phone_number", "send_attempt"]
|
body, ["client_secret", "country", "phone_number", "send_attempt"]
|
||||||
|
|
@ -435,7 +427,7 @@ class MsisdnThreepidRequestTokenRestServlet(RestServlet):
|
||||||
Codes.THREEPID_DENIED,
|
Codes.THREEPID_DENIED,
|
||||||
)
|
)
|
||||||
|
|
||||||
existing_user_id = yield self.store.get_user_id_by_threepid("msisdn", msisdn)
|
existing_user_id = await self.store.get_user_id_by_threepid("msisdn", msisdn)
|
||||||
|
|
||||||
if existing_user_id is not None:
|
if existing_user_id is not None:
|
||||||
raise SynapseError(400, "MSISDN is already in use", Codes.THREEPID_IN_USE)
|
raise SynapseError(400, "MSISDN is already in use", Codes.THREEPID_IN_USE)
|
||||||
|
|
@ -450,7 +442,7 @@ class MsisdnThreepidRequestTokenRestServlet(RestServlet):
|
||||||
"Adding phone numbers to user account is not supported by this homeserver",
|
"Adding phone numbers to user account is not supported by this homeserver",
|
||||||
)
|
)
|
||||||
|
|
||||||
ret = yield self.identity_handler.requestMsisdnToken(
|
ret = await self.identity_handler.requestMsisdnToken(
|
||||||
self.hs.config.account_threepid_delegate_msisdn,
|
self.hs.config.account_threepid_delegate_msisdn,
|
||||||
country,
|
country,
|
||||||
phone_number,
|
phone_number,
|
||||||
|
|
@ -484,8 +476,7 @@ class AddThreepidEmailSubmitTokenServlet(RestServlet):
|
||||||
[self.config.email_add_threepid_template_failure_html],
|
[self.config.email_add_threepid_template_failure_html],
|
||||||
)
|
)
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
|
||||||
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
|
if self.config.threepid_behaviour_email == ThreepidBehaviour.OFF:
|
||||||
if self.config.local_threepid_handling_disabled_due_to_email_config:
|
if self.config.local_threepid_handling_disabled_due_to_email_config:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
|
|
@ -508,7 +499,7 @@ class AddThreepidEmailSubmitTokenServlet(RestServlet):
|
||||||
# Attempt to validate a 3PID session
|
# Attempt to validate a 3PID session
|
||||||
try:
|
try:
|
||||||
# Mark the session as valid
|
# Mark the session as valid
|
||||||
next_link = yield self.store.validate_threepid_session(
|
next_link = await self.store.validate_threepid_session(
|
||||||
sid, client_secret, token, self.clock.time_msec()
|
sid, client_secret, token, self.clock.time_msec()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -558,8 +549,7 @@ class AddThreepidMsisdnSubmitTokenServlet(RestServlet):
|
||||||
self.store = hs.get_datastore()
|
self.store = hs.get_datastore()
|
||||||
self.identity_handler = hs.get_handlers().identity_handler
|
self.identity_handler = hs.get_handlers().identity_handler
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
if not self.config.account_threepid_delegate_msisdn:
|
if not self.config.account_threepid_delegate_msisdn:
|
||||||
raise SynapseError(
|
raise SynapseError(
|
||||||
400,
|
400,
|
||||||
|
|
@ -571,7 +561,7 @@ class AddThreepidMsisdnSubmitTokenServlet(RestServlet):
|
||||||
assert_params_in_dict(body, ["client_secret", "sid", "token"])
|
assert_params_in_dict(body, ["client_secret", "sid", "token"])
|
||||||
|
|
||||||
# Proxy submit_token request to msisdn threepid delegate
|
# Proxy submit_token request to msisdn threepid delegate
|
||||||
response = yield self.identity_handler.proxy_msisdn_submit_token(
|
response = await self.identity_handler.proxy_msisdn_submit_token(
|
||||||
self.config.account_threepid_delegate_msisdn,
|
self.config.account_threepid_delegate_msisdn,
|
||||||
body["client_secret"],
|
body["client_secret"],
|
||||||
body["sid"],
|
body["sid"],
|
||||||
|
|
@ -591,17 +581,15 @@ class ThreepidRestServlet(RestServlet):
|
||||||
self.auth_handler = hs.get_auth_handler()
|
self.auth_handler = hs.get_auth_handler()
|
||||||
self.datastore = self.hs.get_datastore()
|
self.datastore = self.hs.get_datastore()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
|
|
||||||
threepids = yield self.datastore.user_get_threepids(requester.user.to_string())
|
threepids = await self.datastore.user_get_threepids(requester.user.to_string())
|
||||||
|
|
||||||
return 200, {"threepids": threepids}
|
return 200, {"threepids": threepids}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
|
|
||||||
|
|
@ -615,11 +603,11 @@ class ThreepidRestServlet(RestServlet):
|
||||||
client_secret = threepid_creds["client_secret"]
|
client_secret = threepid_creds["client_secret"]
|
||||||
sid = threepid_creds["sid"]
|
sid = threepid_creds["sid"]
|
||||||
|
|
||||||
validation_session = yield self.identity_handler.validate_threepid_session(
|
validation_session = await self.identity_handler.validate_threepid_session(
|
||||||
client_secret, sid
|
client_secret, sid
|
||||||
)
|
)
|
||||||
if validation_session:
|
if validation_session:
|
||||||
yield self.auth_handler.add_threepid(
|
await self.auth_handler.add_threepid(
|
||||||
user_id,
|
user_id,
|
||||||
validation_session["medium"],
|
validation_session["medium"],
|
||||||
validation_session["address"],
|
validation_session["address"],
|
||||||
|
|
@ -642,9 +630,9 @@ class ThreepidAddRestServlet(RestServlet):
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.auth_handler = hs.get_auth_handler()
|
self.auth_handler = hs.get_auth_handler()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
@interactive_auth_handler
|
||||||
def on_POST(self, request):
|
async def on_POST(self, request):
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
|
|
||||||
|
|
@ -652,11 +640,15 @@ class ThreepidAddRestServlet(RestServlet):
|
||||||
client_secret = body["client_secret"]
|
client_secret = body["client_secret"]
|
||||||
sid = body["sid"]
|
sid = body["sid"]
|
||||||
|
|
||||||
validation_session = yield self.identity_handler.validate_threepid_session(
|
await self.auth_handler.validate_user_via_ui_auth(
|
||||||
|
requester, body, self.hs.get_ip_from_request(request)
|
||||||
|
)
|
||||||
|
|
||||||
|
validation_session = await self.identity_handler.validate_threepid_session(
|
||||||
client_secret, sid
|
client_secret, sid
|
||||||
)
|
)
|
||||||
if validation_session:
|
if validation_session:
|
||||||
yield self.auth_handler.add_threepid(
|
await self.auth_handler.add_threepid(
|
||||||
user_id,
|
user_id,
|
||||||
validation_session["medium"],
|
validation_session["medium"],
|
||||||
validation_session["address"],
|
validation_session["address"],
|
||||||
|
|
@ -678,8 +670,7 @@ class ThreepidBindRestServlet(RestServlet):
|
||||||
self.identity_handler = hs.get_handlers().identity_handler
|
self.identity_handler = hs.get_handlers().identity_handler
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
|
|
||||||
assert_params_in_dict(body, ["id_server", "sid", "client_secret"])
|
assert_params_in_dict(body, ["id_server", "sid", "client_secret"])
|
||||||
|
|
@ -688,10 +679,10 @@ class ThreepidBindRestServlet(RestServlet):
|
||||||
client_secret = body["client_secret"]
|
client_secret = body["client_secret"]
|
||||||
id_access_token = body.get("id_access_token") # optional
|
id_access_token = body.get("id_access_token") # optional
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
yield self.identity_handler.bind_threepid(
|
await self.identity_handler.bind_threepid(
|
||||||
client_secret, sid, user_id, id_server, id_access_token
|
client_secret, sid, user_id, id_server, id_access_token
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -708,12 +699,11 @@ class ThreepidUnbindRestServlet(RestServlet):
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.datastore = self.hs.get_datastore()
|
self.datastore = self.hs.get_datastore()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
"""Unbind the given 3pid from a specific identity server, or identity servers that are
|
"""Unbind the given 3pid from a specific identity server, or identity servers that are
|
||||||
known to have this 3pid bound
|
known to have this 3pid bound
|
||||||
"""
|
"""
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
assert_params_in_dict(body, ["medium", "address"])
|
assert_params_in_dict(body, ["medium", "address"])
|
||||||
|
|
||||||
|
|
@ -723,7 +713,7 @@ class ThreepidUnbindRestServlet(RestServlet):
|
||||||
|
|
||||||
# Attempt to unbind the threepid from an identity server. If id_server is None, try to
|
# Attempt to unbind the threepid from an identity server. If id_server is None, try to
|
||||||
# unbind from all identity servers this threepid has been added to in the past
|
# unbind from all identity servers this threepid has been added to in the past
|
||||||
result = yield self.identity_handler.try_unbind_threepid(
|
result = await self.identity_handler.try_unbind_threepid(
|
||||||
requester.user.to_string(),
|
requester.user.to_string(),
|
||||||
{"address": address, "medium": medium, "id_server": id_server},
|
{"address": address, "medium": medium, "id_server": id_server},
|
||||||
)
|
)
|
||||||
|
|
@ -738,16 +728,15 @@ class ThreepidDeleteRestServlet(RestServlet):
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.auth_handler = hs.get_auth_handler()
|
self.auth_handler = hs.get_auth_handler()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
assert_params_in_dict(body, ["medium", "address"])
|
assert_params_in_dict(body, ["medium", "address"])
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
requester = await self.auth.get_user_by_req(request)
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
ret = yield self.auth_handler.delete_threepid(
|
ret = await self.auth_handler.delete_threepid(
|
||||||
user_id, body["medium"], body["address"], body.get("id_server")
|
user_id, body["medium"], body["address"], body.get("id_server")
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
|
|
@ -772,9 +761,8 @@ class WhoamiRestServlet(RestServlet):
|
||||||
super(WhoamiRestServlet, self).__init__()
|
super(WhoamiRestServlet, self).__init__()
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
|
|
||||||
return 200, {"user_id": requester.user.to_string()}
|
return 200, {"user_id": requester.user.to_string()}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,8 +15,6 @@
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import AuthError, NotFoundError, SynapseError
|
from synapse.api.errors import AuthError, NotFoundError, SynapseError
|
||||||
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
||||||
|
|
||||||
|
|
@ -41,15 +39,14 @@ class AccountDataServlet(RestServlet):
|
||||||
self.store = hs.get_datastore()
|
self.store = hs.get_datastore()
|
||||||
self.notifier = hs.get_notifier()
|
self.notifier = hs.get_notifier()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, user_id, account_data_type):
|
||||||
def on_PUT(self, request, user_id, account_data_type):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
if user_id != requester.user.to_string():
|
if user_id != requester.user.to_string():
|
||||||
raise AuthError(403, "Cannot add account data for other users.")
|
raise AuthError(403, "Cannot add account data for other users.")
|
||||||
|
|
||||||
body = parse_json_object_from_request(request)
|
body = parse_json_object_from_request(request)
|
||||||
|
|
||||||
max_id = yield self.store.add_account_data_for_user(
|
max_id = await self.store.add_account_data_for_user(
|
||||||
user_id, account_data_type, body
|
user_id, account_data_type, body
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -57,13 +54,12 @@ class AccountDataServlet(RestServlet):
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, user_id, account_data_type):
|
||||||
def on_GET(self, request, user_id, account_data_type):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
if user_id != requester.user.to_string():
|
if user_id != requester.user.to_string():
|
||||||
raise AuthError(403, "Cannot get account data for other users.")
|
raise AuthError(403, "Cannot get account data for other users.")
|
||||||
|
|
||||||
event = yield self.store.get_global_account_data_by_type_for_user(
|
event = await self.store.get_global_account_data_by_type_for_user(
|
||||||
account_data_type, user_id
|
account_data_type, user_id
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -91,9 +87,8 @@ class RoomAccountDataServlet(RestServlet):
|
||||||
self.store = hs.get_datastore()
|
self.store = hs.get_datastore()
|
||||||
self.notifier = hs.get_notifier()
|
self.notifier = hs.get_notifier()
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_PUT(self, request, user_id, room_id, account_data_type):
|
||||||
def on_PUT(self, request, user_id, room_id, account_data_type):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
if user_id != requester.user.to_string():
|
if user_id != requester.user.to_string():
|
||||||
raise AuthError(403, "Cannot add account data for other users.")
|
raise AuthError(403, "Cannot add account data for other users.")
|
||||||
|
|
||||||
|
|
@ -106,7 +101,7 @@ class RoomAccountDataServlet(RestServlet):
|
||||||
" Use /rooms/!roomId:server.name/read_markers",
|
" Use /rooms/!roomId:server.name/read_markers",
|
||||||
)
|
)
|
||||||
|
|
||||||
max_id = yield self.store.add_account_data_to_room(
|
max_id = await self.store.add_account_data_to_room(
|
||||||
user_id, room_id, account_data_type, body
|
user_id, room_id, account_data_type, body
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -114,13 +109,12 @@ class RoomAccountDataServlet(RestServlet):
|
||||||
|
|
||||||
return 200, {}
|
return 200, {}
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request, user_id, room_id, account_data_type):
|
||||||
def on_GET(self, request, user_id, room_id, account_data_type):
|
requester = await self.auth.get_user_by_req(request)
|
||||||
requester = yield self.auth.get_user_by_req(request)
|
|
||||||
if user_id != requester.user.to_string():
|
if user_id != requester.user.to_string():
|
||||||
raise AuthError(403, "Cannot get account data for other users.")
|
raise AuthError(403, "Cannot get account data for other users.")
|
||||||
|
|
||||||
event = yield self.store.get_account_data_for_room_and_type(
|
event = await self.store.get_account_data_for_room_and_type(
|
||||||
user_id, room_id, account_data_type
|
user_id, room_id, account_data_type
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,8 +15,6 @@
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.errors import AuthError, SynapseError
|
from synapse.api.errors import AuthError, SynapseError
|
||||||
from synapse.http.server import finish_request
|
from synapse.http.server import finish_request
|
||||||
from synapse.http.servlet import RestServlet
|
from synapse.http.servlet import RestServlet
|
||||||
|
|
@ -45,13 +43,12 @@ class AccountValidityRenewServlet(RestServlet):
|
||||||
self.success_html = hs.config.account_validity.account_renewed_html_content
|
self.success_html = hs.config.account_validity.account_renewed_html_content
|
||||||
self.failure_html = hs.config.account_validity.invalid_token_html_content
|
self.failure_html = hs.config.account_validity.invalid_token_html_content
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_GET(self, request):
|
||||||
def on_GET(self, request):
|
|
||||||
if b"token" not in request.args:
|
if b"token" not in request.args:
|
||||||
raise SynapseError(400, "Missing renewal token")
|
raise SynapseError(400, "Missing renewal token")
|
||||||
renewal_token = request.args[b"token"][0]
|
renewal_token = request.args[b"token"][0]
|
||||||
|
|
||||||
token_valid = yield self.account_activity_handler.renew_account(
|
token_valid = await self.account_activity_handler.renew_account(
|
||||||
renewal_token.decode("utf8")
|
renewal_token.decode("utf8")
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -67,7 +64,6 @@ class AccountValidityRenewServlet(RestServlet):
|
||||||
request.setHeader(b"Content-Length", b"%d" % (len(response),))
|
request.setHeader(b"Content-Length", b"%d" % (len(response),))
|
||||||
request.write(response.encode("utf8"))
|
request.write(response.encode("utf8"))
|
||||||
finish_request(request)
|
finish_request(request)
|
||||||
defer.returnValue(None)
|
|
||||||
|
|
||||||
|
|
||||||
class AccountValiditySendMailServlet(RestServlet):
|
class AccountValiditySendMailServlet(RestServlet):
|
||||||
|
|
@ -85,18 +81,17 @@ class AccountValiditySendMailServlet(RestServlet):
|
||||||
self.auth = hs.get_auth()
|
self.auth = hs.get_auth()
|
||||||
self.account_validity = self.hs.config.account_validity
|
self.account_validity = self.hs.config.account_validity
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request):
|
||||||
def on_POST(self, request):
|
|
||||||
if not self.account_validity.renew_by_email_enabled:
|
if not self.account_validity.renew_by_email_enabled:
|
||||||
raise AuthError(
|
raise AuthError(
|
||||||
403, "Account renewal via email is disabled on this server."
|
403, "Account renewal via email is disabled on this server."
|
||||||
)
|
)
|
||||||
|
|
||||||
requester = yield self.auth.get_user_by_req(request, allow_expired=True)
|
requester = await self.auth.get_user_by_req(request, allow_expired=True)
|
||||||
user_id = requester.user.to_string()
|
user_id = requester.user.to_string()
|
||||||
yield self.account_activity_handler.send_renewal_email_to_user(user_id)
|
await self.account_activity_handler.send_renewal_email_to_user(user_id)
|
||||||
|
|
||||||
defer.returnValue((200, {}))
|
return 200, {}
|
||||||
|
|
||||||
|
|
||||||
def register_servlets(hs, http_server):
|
def register_servlets(hs, http_server):
|
||||||
|
|
|
||||||
|
|
@ -15,8 +15,6 @@
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from twisted.internet import defer
|
|
||||||
|
|
||||||
from synapse.api.constants import LoginType
|
from synapse.api.constants import LoginType
|
||||||
from synapse.api.errors import SynapseError
|
from synapse.api.errors import SynapseError
|
||||||
from synapse.api.urls import CLIENT_API_PREFIX
|
from synapse.api.urls import CLIENT_API_PREFIX
|
||||||
|
|
@ -171,8 +169,7 @@ class AuthRestServlet(RestServlet):
|
||||||
else:
|
else:
|
||||||
raise SynapseError(404, "Unknown auth stage type")
|
raise SynapseError(404, "Unknown auth stage type")
|
||||||
|
|
||||||
@defer.inlineCallbacks
|
async def on_POST(self, request, stagetype):
|
||||||
def on_POST(self, request, stagetype):
|
|
||||||
|
|
||||||
session = parse_string(request, "session")
|
session = parse_string(request, "session")
|
||||||
if not session:
|
if not session:
|
||||||
|
|
@ -186,7 +183,7 @@ class AuthRestServlet(RestServlet):
|
||||||
|
|
||||||
authdict = {"response": response, "session": session}
|
authdict = {"response": response, "session": session}
|
||||||
|
|
||||||
success = yield self.auth_handler.add_oob_auth(
|
success = await self.auth_handler.add_oob_auth(
|
||||||
LoginType.RECAPTCHA, authdict, self.hs.get_ip_from_request(request)
|
LoginType.RECAPTCHA, authdict, self.hs.get_ip_from_request(request)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -215,7 +212,7 @@ class AuthRestServlet(RestServlet):
|
||||||
session = request.args["session"][0]
|
session = request.args["session"][0]
|
||||||
authdict = {"session": session}
|
authdict = {"session": session}
|
||||||
|
|
||||||
success = yield self.auth_handler.add_oob_auth(
|
success = await self.auth_handler.add_oob_auth(
|
||||||
LoginType.TERMS, authdict, self.hs.get_ip_from_request(request)
|
LoginType.TERMS, authdict, self.hs.get_ip_from_request(request)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue