Commit Graph

2964 Commits

Author SHA1 Message Date
Katie Wolfe 7e07dc429f
Lint
I probably should've just run autopep8 in the first place...

Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:43:18 -04:00
Katie Wolfe b3e5db402d
Clean up code
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:55 -04:00
Katie Wolfe 5d3ed79944
Show heroes if room name or canonical alias are empty
Fixes #4194

Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:53 -04:00
Katie Wolfe 7f025eb425
Show heroes if room name or canonical alias are empty
Fixes #4194

Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:29 -04:00
Erik Johnston ca90336a69 Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration 2019-04-17 19:44:40 +01:00
Brendan Abolivier eaf41a943b Add management endpoints for account validity 2019-04-17 19:34:45 +01:00
Brendan Abolivier 91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier 20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00
Erik Johnston c132c8e505 Handle the case of `get_missing_events` failing
Currently if a call to `/get_missing_events` fails we log an exception
and stop processing the top level event we received over federation.
Instead let's try and handle it sensibly given it is a somewhat expected
failure mode.
2019-04-10 10:39:54 +01:00
Brendan Abolivier 8e85493b0c
Add config option to block users from looking up 3PIDs (#5010) 2019-04-04 17:25:47 +01:00
Andrew Morgan db265f0642
Prevent kicking users who aren't in the room (#4999)
Prevent kick events from succeeding if the user is not currently in the room.
2019-04-04 13:05:51 +01:00
Andrew Morgan 4a4d5c4fd6
Fix grammar and document get_current_users_in_room (#4998) 2019-04-03 14:32:20 +01:00
Neil Johnson e8419554ff
Remove presence lists (#4989)
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
Erik Johnston 8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915
Implement MSC1915 - 3PID unbind APIs
2019-04-03 11:07:09 +01:00
Andrew Morgan 66e78700a2
Transfer related groups on room upgrade (#4990)
Transfers the m.room.related_groups state event on room upgrade.
2019-04-02 17:15:24 +01:00
Erik Johnston 4ef5d17b96 Correctly handle id_server param 2019-04-02 11:20:09 +01:00
Erik Johnston 24232514bf Remove threepid binding if id server returns 400/404/501 2019-04-02 11:20:09 +01:00
Erik Johnston c75e2017f1 Fixup docstrings 2019-04-02 11:20:06 +01:00
Erik Johnston 057715aaa2 Allowing specifying IS to use in unbind API.
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.

Implements MSC915.
2019-04-01 15:25:18 +01:00
Erik Johnston 9fbbc3d9e5 For unbind poke IS used during binding of 3PID
This changes the behaviour from using the server specified trusted
identity server to using the IS that used during the binding of the
3PID, if known.

This is the behaviour specified by MSC1915.
2019-04-01 15:23:30 +01:00
Erik Johnston 1666c0696a Track IS used to bind 3PIDs
This will then be used to know which IS to default to when unbinding the
threepid.
2019-04-01 15:23:01 +01:00
Richard van der Hoff 54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Andrew Morgan 7a91b9d81c
Allow password providers to bind emails (#4947)
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston 4e5f0f7ca0 Use an assert 2019-03-28 14:05:05 +00:00
Erik Johnston 40e56997bc Review comments 2019-03-28 13:48:41 +00:00
Erik Johnston 197fae1639 Use event streams to calculate presence
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962
2019-03-27 13:41:36 +00:00
Andrew Morgan bbd244c7b2
Support 3PID login in password providers (#4931)
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown ac396a0d32
Refactor out state delta handling into its own class (#4917) 2019-03-25 20:37:08 +11:00
Erik Johnston 01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Erik Johnston 3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston 536a266520 Deny peeking into rooms that have been blocked 2019-03-21 11:20:13 +00:00
Erik Johnston 09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Erik Johnston cd62981a6a Revert spurious delete 2019-03-20 17:51:27 +00:00
Erik Johnston aa959a6c07 Use flags 2019-03-20 17:40:29 +00:00
Erik Johnston 7d47cc1305 Move requester check into assert_accepted_privacy_policy 2019-03-20 17:08:36 +00:00
Erik Johnston 74c46d81fa Only require consent for events with an associated request
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Richard van der Hoff a902d13180
Batch up outgoing read-receipts to reduce federation traffic. (#4890)
Rate-limit outgoing read-receipts as per #4730.
2019-03-20 16:02:25 +00:00
Erik Johnston cc197a61a1 Disable publishing to room list when its disabled 2019-03-20 14:30:36 +00:00
Erik Johnston 2c90422146 Pull out config option 2019-03-20 14:25:58 +00:00
Erik Johnston 7529038e66 Return before we log 2019-03-20 14:25:28 +00:00
Erik Johnston 213c98c00a Add option to disable search room lists
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Neil Johnson 88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Amber Brown 282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Brendan Abolivier 651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier 899e523d6d
Add ratelimiting on login (#4821)
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff 2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston 9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Richard van der Hoff fdcad8eabd Move client receipt processing to federation sender worker.
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff eed7271b3b declare a ReadReceipt class
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan d42c81d724
Transfer local user's push rules on room upgrade (#4838)
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown c0332d095f fixup 2019-03-13 01:30:54 +11:00
Amber Brown 10480c4348 fixup 2019-03-12 21:47:14 +11:00
Amber Brown 5ba8ceab4c fixes 2019-03-12 00:35:31 +11:00
Amber Brown 4abc988c6a
initial 2019-03-11 21:11:36 +11:00
Erik Johnston 8c4896668f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl 2019-03-08 11:44:20 +00:00
Erik Johnston 0ff8163eae Factor out soft fail checks 2019-03-08 11:26:33 +00:00
Erik Johnston 469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston d42b41544a When re-syncing device lists reset the state
We keep track of what stream IDs we've seen so that we know what updates
we've handled or missed. If we re-sync we don't know if the updates
we've seen are included in the re-sync (there may be a race), so we
should reset the seen updates.
2019-03-07 16:04:24 +00:00
Erik Johnston c633fc02d7 Add some debug logging for device list handling 2019-03-07 15:58:03 +00:00
Amber Brown f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Erik Johnston 0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston 8b7790e68f Port #4422 debug logging from hotfixes 2019-03-06 17:29:15 +00:00
Erik Johnston 6d13bdec91 Add docstrings from matrix-org-hotfixes 2019-03-06 17:24:26 +00:00
Erik Johnston a9de04be72 Implement soft fail 2019-03-06 16:22:16 +00:00
Brendan Abolivier 6f3cde8b25 Make registration ratelimiter separate from the main events one 2019-03-06 11:02:42 +00:00
Brendan Abolivier f4195f4118 Revert "Split ratelimiters in two (one for events, one for registration)"
This reverts commit d7dbad3526.
2019-03-06 10:55:22 +00:00
Brendan Abolivier d7dbad3526 Split ratelimiters in two (one for events, one for registration) 2019-03-05 18:41:27 +00:00
Brendan Abolivier a4c3a361b7
Add rate-limiting on registration (#4735)
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
2019-03-05 14:25:33 +00:00
Erik Johnston b050a10871
Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston aa06d26ae0 clarify comments 2019-03-05 09:16:35 +00:00
Erik Johnston c3c542bb4a
Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Erik Johnston bfa7d46a10 Allow /keys/{changes,query} API to run on worker 2019-03-04 18:30:01 +00:00
Erik Johnston 157e5a8f27 Split DeviceHandler into master and worker 2019-03-04 18:29:26 +00:00
Richard van der Hoff b29693a30b Clean up read-receipt handling.
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.

We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Erik Johnston d1523aed6b Only check history visibility when filtering
When filtering events to send to server we check more than just history
visibility. However when deciding whether to backfill or not we only
care about the history visibility.
2019-03-04 14:43:42 +00:00
Richard van der Hoff 856c83f5f8
Avoid rebuilding Edu objects in worker mode (#4770)
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston 8b63fe4c26 s/get_forward_events/get_successor_events/ 2019-03-04 11:56:03 +00:00
Erik Johnston fbc047f2a5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room 2019-03-04 11:54:58 +00:00
Richard van der Hoff 3064952939
Fix incorrect log about not persisting duplicate state event. (#4776)
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Erik Johnston b183fef9ac Update comments 2019-02-27 13:06:10 +00:00
Andrew Morgan 802884d4ee Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop 2019-02-26 14:23:40 +00:00
Andrew Morgan 6fcb25202f Put function def back to the way it was 2019-02-26 14:17:38 +00:00
Andrew Morgan 7a4632af9c Prevent showing non-fed rooms in fed /publicRooms 2019-02-26 13:37:24 +00:00
Andrew Morgan c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms" 2019-02-26 13:20:38 +00:00
Andrew Morgan a1a6473293
Merge pull request #4736 from matrix-org/anoa/public_rooms_federate
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:07:15 +00:00
Andrew Morgan c4414768af Cleaner chunk logic 2019-02-26 12:22:34 +00:00
Andrew Morgan a712aa3a9c Correct indent 2019-02-26 12:13:55 +00:00
Andrew Morgan 16565e67db Correct docstring types and chunk logic 2019-02-26 12:12:48 +00:00
Andrew Morgan 40c2271680 Clean up room chunk logic 2019-02-26 12:04:34 +00:00
Andrew Morgan 6728bf3940 Make not showing non-federated rooms the default 2019-02-26 11:52:52 +00:00
Andrew Morgan 6946c20111 Result may be None 2019-02-26 11:27:19 +00:00
Andrew Morgan 71669a0fba Address rich comments 2019-02-26 11:25:00 +00:00
Matthew Hodgson 70ea2f4e1d switch from google.com to recaptcha.net for reCAPTCHA (#4731)
* add trivial clarification about jemalloc

* switch from google.com to recaptcha.net

because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally
2019-02-25 11:15:36 -08:00
Andrew Morgan 1330aa4a8f elif not else if 2019-02-25 17:28:19 +00:00
Erik Johnston 65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston c7b333c545 Log tracebacks correctly 2019-02-25 16:56:41 +00:00
Erik Johnston 9c598dddcb Fix typo 2019-02-25 16:32:02 +00:00
Andrew Morgan 8aaf7ffc44 syntax derp 2019-02-25 15:27:17 +00:00
Andrew Morgan 84c0a20dfe Simplify call to generate_room_entry 2019-02-25 15:23:27 +00:00
Erik Johnston 4b9e5076c4 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate 2019-02-25 15:08:18 +00:00
Andrew Morgan 07493607a8 Docs and arg name clarification 2019-02-25 15:04:46 +00:00
Andrew Morgan bd398b874e Don't restrict non-fed rooms over client APIs 2019-02-25 15:04:46 +00:00
Andrew Morgan e4b078a600 Config option to prevent showing non-fed rooms in fed /publicRooms 2019-02-25 15:04:46 +00:00
Erik Johnston d730c2c22b More comments 2019-02-25 14:45:02 +00:00
Erik Johnston 890cb048fd Assert rather than clobber the values 2019-02-25 14:42:11 +00:00
Erik Johnston 9342cc6ab1 Add comments and paranoia 2019-02-25 10:02:12 +00:00
Erik Johnston 7b288826b7 Fix backfill storing incorrect state for events 2019-02-22 11:33:51 +00:00
Richard van der Hoff 6d65659b62
Run push_receipts_to_remotes as background job (#4707)
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston b9d6756b14
Merge pull request #4263 from rkfg/develop
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston 71b625d808 Stop backpaginating when events not visible 2019-02-20 18:14:12 +00:00
Erik Johnston dbdc565dfd Fix registration on workers (#4682)
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
2019-02-20 18:47:31 +11:00
Brendan Abolivier a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Brendan Abolivier a8626901cd Fetch ACME domain into an instance member 2019-02-19 10:54:33 +00:00
Andrew Morgan 968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Andrew Morgan 34ac75ce2c lint 2019-02-18 18:23:37 +00:00
Andrew Morgan f8b9ca53ce Move member event processing and changelog fix 2019-02-18 17:28:52 +00:00
Erik Johnston fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston 459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston af691e415c Move register_device into handler 2019-02-18 16:49:38 +00:00
Erik Johnston c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Brendan Abolivier 45bb55c6de Use a configuration parameter to give the domain to generate a certificate for 2019-02-18 15:46:23 +00:00
Erik Johnston 94960cef03 pep8 2019-02-18 15:24:13 +00:00
Erik Johnston e07cc31cb8 Correctly handle HttpResponseException 2019-02-18 14:55:09 +00:00
Brendan Abolivier 68a53f825f Merge branch 'develop' into babolivier/acme-delegated 2019-02-18 14:52:23 +00:00
Erik Johnston 32e54b472a Fix kicking guest users in worker mode
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.

Presumably broken in #4141
2019-02-18 14:16:07 +00:00
Andrew Morgan 915421065b Membership events are done later 2019-02-18 14:02:09 +00:00
Erik Johnston d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan 9caab0c364 Transfer bans on room upgrade 2019-02-18 13:52:51 +00:00
Erik Johnston eb2b8523ae Split out registration to worker
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.

Note: due to the in-memory session map all registration requests must be
handled by the same worker.
2019-02-18 12:12:57 +00:00
Andrew Morgan 6d02a13d81
Typo in info log
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:34 +00:00
Brendan Abolivier f86b695cbd
Various cosmetics to make TravisCI happy 2019-02-15 12:29:34 +00:00
Brendan Abolivier af8a2f679b
Remove unused import 2019-02-15 12:27:43 +00:00
Brendan Abolivier 1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Erik Johnston f61b2068e6 Only fetch aliases when publishing rooms 2019-02-14 18:08:09 +00:00
Erik Johnston eaf4d11af9 Add configurable room list publishing rules
This allows specifying who and what is allowed to be published onto the
public room list
2019-02-14 16:02:23 +00:00
Erik Johnston 7fc1196a36 Correctly handle RequestSendFailed exceptions
This mainly reduces the number of exceptions we log.
2019-02-14 14:01:04 +00:00
Amber Brown bb4fd8f927
Run `black` on user directory code (#4635) 2019-02-13 23:05:32 +11:00
Erik Johnston ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Andrew Morgan a126f86eec Transfer Server ACLs on room upgrade 2019-02-11 11:30:37 +00:00
Hubert Chathi afae8442b5 make sure version is in body and wrap in linearizer queue
also add tests
2019-02-08 01:32:45 -05:00
Hubert Chathi d9e424bf64 re-try to make isort happy 2019-02-06 22:18:41 -05:00
Hubert Chathi 9ff620a518 fix import to make isort happy 2019-02-06 21:32:52 -05:00
Hubert Chathi 8248637173 add new endpoint to update backup versions 2019-02-06 17:57:10 -05:00
Hubert Chathi 664c81e8b7 return proper error codes for some 404s 2019-02-06 17:47:22 -05:00
Andrew Morgan c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
Richard van der Hoff 07dfe148de
Add some debug for membership syncing issues (#4538)
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan 3ed3cb4339 New function for getting room's create event 2019-01-31 18:21:39 +00:00
Andrew Morgan bbb97a35fd Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable 2019-01-31 18:11:00 +00:00
Andrew Morgan fb50934b8f lint 2019-01-31 11:34:45 +00:00
Andrew Morgan d621c5562e Copy over non-federatable trait on room upgrade 2019-01-30 16:33:51 +00:00
Erik Johnston ed8c5e4cda Fix remote invite rejections not comming down sync
This was broken in PR #4405, commit 886e5ac, where we changed remote
rejections to be outliers.

The fix is to explicitly add the leave event in when we know its an out
of band invite. We can't always add the event as if the server is/was in
the room there might be more events to send down the sync than just the
leave.
2019-01-30 15:46:27 +00:00
Richard van der Hoff 7615a8ced1 ACME config cleanups (#4525)
* Handle listening for ACME requests on IPv6 addresses

the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.

* Clean up the default ACME config

make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.

* newsfile
2019-01-30 14:17:55 +00:00
Amber Brown 3f189c902e
Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Erik Johnston 67b82f1336 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:58:38 +00:00
Erik Johnston a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston 55d9024835 Use snder and not event ID domain to check if ours
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
2019-01-29 16:54:23 +00:00
Erik Johnston b1fffca345 Remove event ID usage when checking if new room
The event ID is changing, so we can no longer get the domain from it. On
the other hand, the check is unnecessary.
2019-01-29 16:15:02 +00:00
Erik Johnston b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
Erik Johnston 5891a6edc8 Correctly set context.app_service 2019-01-29 12:09:10 +00:00
Erik Johnston 7709d2bd16 Implement rechecking of redactions 2019-01-29 11:56:20 +00:00
Erik Johnston 6598992b01 Fixup calls to `comput_event_signature`
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.

This fixes PR #4493 commit 855a151
2019-01-29 11:41:58 +00:00
Erik Johnston be47cfa9c9 Refactor event building into EventBuilder
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
Erik Johnston 073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Erik Johnston f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Erik Johnston d758d5310e Correctly use default room version if none is set 2019-01-28 17:26:39 +00:00
Erik Johnston b872c7b1b4 Split up event validation between event and builder
The validator was being run on the EventBuilder objects, and so the
validator only checked a subset of fields. With the upcoming
EventBuilder refactor even fewer fields will be there to validate.

To get around this we split the validation into those that can be run
against an EventBuilder and those run against a fully fledged event.
2019-01-28 17:00:14 +00:00
Andrew Morgan f0e96ab66a Change return syntax in doc string 2019-01-28 14:09:45 +00:00
Andrew Morgan 4026d555fa Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-28 14:08:24 +00:00
Andrew Morgan 1ce463963d Reuse predecessor method 2019-01-28 14:08:18 +00:00
Erik Johnston ae2a957dba Pass through room version to event auth 2019-01-25 18:31:41 +00:00
Erik Johnston b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Andrew Morgan 9244a3089e Fixes 2019-01-25 11:48:38 +00:00
Andrew Morgan da0d2219d2 Clean up direct_rooms access 2019-01-25 11:37:12 +00:00
Andrew Morgan 8265995498 Use python magic 2019-01-25 11:26:06 +00:00
Andrew Morgan c4cdafa81f Destructure account data tuple before use 2019-01-25 11:24:28 +00:00
Andrew Morgan 516456b763 Remove unnecessary null check 2019-01-25 11:22:14 +00:00
Andrew Morgan 6f3fda79ce Move room_tag declaration to be closer to its use 2019-01-25 11:21:25 +00:00
Andrew Morgan 821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Andrew Morgan 0862d35b8e Move tag and direct state copying into separate function 2019-01-25 11:09:34 +00:00
Andrew Morgan b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston a50cf929c1 Require event format version to parse or create events 2019-01-25 10:32:19 +00:00
Erik Johnston 62514bb81b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 2019-01-25 10:07:08 +00:00
Erik Johnston edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston 80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston d148c43050 Review comments 2019-01-24 18:31:23 +00:00
Erik Johnston 9139b87be4 Remove unecessary setting of outlier bit 2019-01-24 18:04:02 +00:00
Erik Johnston b8082a5445 Use term 'out of band membership' instead 2019-01-24 17:33:19 +00:00
Richard van der Hoff 03c85335d1
Apply suggestions from code review
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
Andrew Morgan 075ff3ede9 Change default timeout value from 0 to None 2019-01-24 15:10:22 +00:00
Andrew Morgan 5b1dc94083 Use self.clock instead of datetime 2019-01-24 14:59:50 +00:00
Andrew Morgan a2d85144e5 isort 2019-01-24 14:22:26 +00:00
Andrew Morgan 5541645e80 lint 2019-01-24 12:45:32 +00:00
Andrew Morgan 068aa1d228 Time out filtered room dir queries after 60s 2019-01-24 12:44:27 +00:00
Erik Johnston 7c288c2250 Clarify the invite flows 2019-01-23 20:07:47 +00:00
Erik Johnston 07f62da55a Remove unnecessary '_sign_event' 2019-01-23 20:04:57 +00:00
Erik Johnston 886e5acc76 Store rejected remote invite events as outliers
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.

In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
Erik Johnston 6a41d2a187 Add room_version param to get_pdu
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
Erik Johnston 67cd4dad81 Implement MSC 1813 - Add room version to make APIs
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
Andrew Morgan 82a92ba535 Add metric for user dir current event stream position 2019-01-23 15:34:47 +00:00
Amber Brown 6129e52f43
Support ACME for certificate provisioning (#4384) 2019-01-23 19:39:06 +11:00
Andrew Morgan 766a172b99 lint 2019-01-22 13:51:40 +00:00
Andrew Morgan c4875d8c76 Prevent duplicate room IDs in m.direct 2019-01-22 12:13:46 +00:00
Andrew Morgan c9bfb058d8 Fix a bug with single-room search searching all rooms
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
Andrew Morgan 8086a5c05e Fix comments 2019-01-22 11:16:23 +00:00
Andrew Morgan 48951f437f Join logic covers both room creator and arbitrary users 2019-01-22 11:00:04 +00:00
Andrew Morgan 8c85f0833d tags, m.direct copying over correctly 2019-01-22 11:00:04 +00:00
Andrew Morgan 25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Andrew Morgan ea8903fcc9 Migrating dm and room tags work for migrator 2019-01-22 11:00:04 +00:00
Andrew Morgan 887ca93a1b Prevent crash on user who doesn't have any direct rooms 2019-01-22 11:00:04 +00:00
Andrew Morgan 4ff6d22245 Preserve DM status of a room on upgrade
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-22 11:00:04 +00:00
Andrew Morgan 75942af1db Fix typo 2019-01-22 11:00:04 +00:00
Andrew Morgan 702c4b750c Migrate encryption state on room upgrade (#4411)
* Migrate encryption state on room upgrade

Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>

* Add changelog file
2019-01-21 20:42:58 +11:00
Andrew Morgan df3a661e4a Search for messages across predecessor rooms
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
Richard van der Hoff 9feb5d0b71
sign_request -> build_auth_headers (#4408)
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
Richard van der Hoff 05e1296649
don't store more remote device lists if they have more than 1K devices (#4397) 2019-01-16 23:14:11 +00:00
Matthew Hodgson 27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Nathan Pennie 81b513416e
Fixed line length 2018-12-19 14:12:33 -05:00
Nathan Pennie 668e6625b0
Fix the variable names used for account_data 2018-12-19 06:53:02 -05:00
Neil Johnson d2f7c4e6b1
create support user (#4141)
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Richard van der Hoff 30da50a5b8
Initialise user displayname from SAML2 data (#4272)
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
Richard van der Hoff c588b9b9e4
Factor SSO success handling out of CAS login (#4264)
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
rkfg ae19a7db8c Prevent crash on pagination. 2018-12-06 13:32:05 +03:00
Richard van der Hoff b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265)
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
Travis Ralston 158ffb92f1 Add an option to disable search for homeservers which may not be interested in it (#4230)
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
Neil Johnson 7039ece8fb Neilj/fix autojoin (#4223)
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
Travis Ralston ab4526a153 Remove duplicate slashes in generated consent URLs 2018-11-15 20:41:53 -07:00
Erik Johnston dc59ad5334 Remove hack to support rejoining rooms 2018-11-09 14:58:09 +00:00
Erik Johnston 30dd27afff Simplify to always drop events if server isn't in the room 2018-11-09 11:36:45 +00:00
Travis Ralston 0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142)
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
Hubert Chathi f1087106cf
handle empty backups according to latest spec proposal (#4123)
fixes #4056
2018-11-05 17:59:29 -05:00
Erik Johnston bc80b3f454 Add helpers for getting prev and auth events (#4139)
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Travis Ralston a8d41c6aff Include a version query string arg for the consent route 2018-10-31 13:19:28 -06:00
Travis Ralston d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Richard van der Hoff 1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
Richard van der Hoff 0f8591a5a8 Avoid else clause on exception for clarity 2018-10-31 15:43:57 +00:00
Richard van der Hoff 94c7fadc98 Attempt to move room aliases on room upgrades 2018-10-31 15:43:57 +00:00
Richard van der Hoff 9b827c40ca
Log some bits about event creation (#4121)
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Amber Brown 086e1a8f3e Fix search 500ing (#4122) 2018-10-31 06:26:46 +11:00
Amber Brown 3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown 4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff 7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
Richard van der Hoff db24d7f15e Better handling of odd PLs during room upgrades
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
Richard van der Hoff 5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00
Richard van der Hoff 54bbe71867 optimise state copying 2018-10-26 22:51:34 +01:00
Richard van der Hoff 193cadc988 Address review comments
Improve comments, get old room state from the context we already have
2018-10-26 17:10:30 +01:00
Richard van der Hoff e6babc27d5 restrict PLs in old room 2018-10-25 19:18:25 +01:00
Richard van der Hoff 3a263bf3ae copy state 2018-10-25 19:18:25 +01:00
Richard van der Hoff 1b9f253e20 preserve PLs 2018-10-25 19:10:24 +01:00
Richard van der Hoff 4cda300058 preserve room visibility 2018-10-25 19:10:24 +01:00