Commit Graph

1122 Commits

Author SHA1 Message Date
Erik Johnston 073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Amber Brown 94fb63e44f
Fix typo in upserts code (#4505)
* fix obvious problem :|

* changelog
2019-01-29 10:04:23 +00:00
Erik Johnston 17709f8f9c
Merge pull request #4493 from matrix-org/erikj/refactor_event_signing
Refactor event signing to work on dicts
2019-01-29 09:52:54 +00:00
Richard van der Hoff f2b553d656 Use SimpleResolverComplexifier in tests (#4497)
two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes
unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we
rely on to give consistent behaviour between python 2 and 3.
2019-01-29 09:38:29 +00:00
Erik Johnston f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Travis Ralston c4045647eb
Create 4498.misc 2019-01-28 12:16:39 -07:00
Erik Johnston 8cbc99cc19 Newsfile 2019-01-28 17:36:06 +00:00
Amber Brown 5d976c0c7c
Fix worker TLS (#4492)
* load cert

* changelog

* fix
2019-01-28 17:18:33 +00:00
Erik Johnston 1977a9b006 Newsfile 2019-01-28 17:05:04 +00:00
Erik Johnston b8bea3424f Newsfile 2019-01-28 16:47:12 +00:00
Amber Brown 7072fe3084
Fix UPSERTs on SQLite 3.24+ (#4477) 2019-01-28 15:43:32 +00:00
Andrew Morgan 4026d555fa Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-28 14:08:24 +00:00
Aaron Raimist 57a3e96e8e Remove --process-dependency-links from UPGRADE.rst (#4485)
* Remove --process-dependency-links from UPGRADE.rst

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-28 13:54:25 +00:00
Richard van der Hoff ff05ad147a changelog 2019-01-28 10:34:30 +00:00
Richard van der Hoff d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487)
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.

There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.

The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
2019-01-28 09:56:59 +00:00
Erik Johnston f01c7488ab Newsfile 2019-01-25 18:32:50 +00:00
Erik Johnston b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Erik Johnston 431e485914 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert 2019-01-25 14:11:17 +00:00
Richard van der Hoff 4a3f138832
Fix quoting for allowed_local_3pids example config (#4476)
If you use double-quotes here, you have to escape your backslashes. It's much
easier with single-quotes.

(Note that the existing double-backslashes are already interpreted by python's
""" parsing.)
2019-01-25 13:57:52 +00:00
Richard van der Hoff 8520bc3109
Fix Host header sent by MatrixFederationAgent (#4468)
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
Andrew Morgan 0b3fd1401f Don't require sqlite3 when using postgres (#4466) 2019-01-25 22:25:02 +11:00
Neil Johnson a3f0556bea towncrier 2019-01-25 11:15:41 +00:00
Andrew Morgan 821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Andrew Morgan b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston 8dcfa6e75c Newsfile 2019-01-25 10:48:40 +00:00
Erik Johnston 5d881cbcb8 Newsfile 2019-01-25 10:37:13 +00:00
Erik Johnston 62514bb81b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 2019-01-25 10:07:08 +00:00
Erik Johnston edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston 80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston 664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2
Add support for persisting event format versions
2019-01-24 18:32:30 +00:00
Andrew Morgan 50c396a7ee
Merge pull request #4461 from matrix-org/anoa/room_dir_quick_fix
Add a 60s timeout to filtered room directory queries
2019-01-24 15:24:35 +00:00
Richard van der Hoff 4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup
MatrixFederationAgent: Look up the right SRV record
2019-01-24 13:57:51 +00:00
Richard van der Hoff 8c58c10697
Generate the debian config during build (#4444)
Rather than hardcoding a config which we always forget to update, generate it
from the default config.
2019-01-24 13:39:01 +00:00
Richard van der Hoff f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json (#4460)
This could cause exceptions if the id server returned 4xx responses.
2019-01-24 13:38:29 +00:00
Richard van der Hoff afd69a0920 Look up the right SRV record 2019-01-24 13:31:43 +00:00
Neil Johnson 10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check
Neilj/fix threepid auth check
2019-01-24 13:02:50 +00:00
Andrew Morgan 2a360e834f Add changelog 2019-01-24 12:47:35 +00:00
Neil Johnson 5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc
Clarify docs for public_baseurl
2019-01-24 12:36:09 +00:00
Amber Brown 0e27501ee5
Fix UPSERT check (#4459) 2019-01-24 22:57:41 +11:00
David Baker 1f2058fca5 Changelog 2019-01-24 10:57:12 +00:00
Amber Brown 58f6c48183
Use native UPSERTs where possible (#4306) 2019-01-24 21:31:54 +11:00
Richard van der Hoff 97fd29c019
Don't send IP addresses as SNI (#4452)
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
2019-01-24 09:34:44 +00:00
Erik Johnston 183738f469 Newsfile 2019-01-23 20:04:57 +00:00
Willem Mulder d528406cb8 Fix error message for optional dependencies
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
2019-01-23 18:44:57 +01:00
Erik Johnston c5a125b24f Update newsfile 2019-01-23 17:26:06 +00:00
Erik Johnston 4a8b715679 Newsfile 2019-01-23 17:25:27 +00:00
Erik Johnston a4ef8d8dd5 Newsfile 2019-01-23 17:24:44 +00:00
Andrew Morgan 6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport
Add metric for user dir current event stream position
2019-01-23 15:58:40 +00:00
Richard van der Hoff a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent
Move SRV magic into an Agent-like thing
2019-01-23 15:50:25 +00:00
Andrew Morgan cb0e637a94 Add changelog 2019-01-23 15:38:27 +00:00
Richard van der Hoff 2f88881c93
debian package: symlink to python-3.X (#4433)
In the debian package, make the virtualenv symlink python to /usr/bin/python3.X
rather than /usr/bin/python3. Also make sure we depend on the right python3.x
package.

This might help a bit with subtle failures when people install a package from
the wrong distro (https://github.com/matrix-org/synapse/issues/4431).
2019-01-23 11:43:04 +00:00
Erik Johnston 4cd50d983d Newsfile 2019-01-23 11:30:24 +00:00
Erik Johnston 90743c9d89 Fixup removal of duplicate `user_ips` rows (#4432)
* Remove unnecessary ORDER BY clause

* Add logging

* Newsfile
2019-01-23 19:45:18 +11:00
Amber Brown 6129e52f43
Support ACME for certificate provisioning (#4384) 2019-01-23 19:39:06 +11:00
Jason Robinson db33634b1d Collapse changelog to one line
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson 1d2c69fee8 Add changelog for openid resource addition
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Richard van der Hoff c66f4bf7f1 changelog 2019-01-22 20:34:35 +00:00
Neil Johnson c99c2d58d7 move guard out of is_threepid_reserved and into register.py 2019-01-22 17:47:00 +00:00
Neil Johnson d619b113ed Fix None guard in config.server.is_threepid_reserved 2019-01-22 16:52:29 +00:00
Erik Johnston 12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup
Fix bug when removing duplicate rows from user_ips
2019-01-22 16:51:57 +00:00
Neil Johnson 388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration
Config option to disable requesting MSISDN on registration
2019-01-22 16:23:08 +00:00
Erik Johnston c658425e6f Newsfile 2019-01-22 13:34:10 +00:00
Andrew Morgan 117bc94cd2 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-22 13:16:51 +00:00
Richard van der Hoff 6bfa735a69
Make key fetches use regular federation client (#4426)
All this magic is redundant.
2019-01-22 11:04:20 +00:00
Andrew Morgan 25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Andrew Morgan 1f18c7cfc9 Add changelog 2019-01-22 11:00:04 +00:00
Richard van der Hoff 33a55289cb
Refactor and bugfix for resove_service (#4427) 2019-01-22 10:59:27 +00:00
Amber Brown 23b0813599
Require ECDH key exchange & remove dh_params (#4429)
* remove dh_params and set better cipher string
2019-01-22 21:58:50 +11:00
Neil Johnson 5349262302 Config option to disable requesting MSISDN on registration 2019-01-21 14:59:37 +00:00
Erik Johnston 35e1d67b4e Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_v2_invite_server 2019-01-21 14:04:19 +00:00
Erik Johnston 5f54765587
Merge pull request #4390 from matrix-org/erikj/versioned_fed_apis
Add groundwork for new versions of federation APIs
2019-01-21 11:44:05 +00:00
Andrew Morgan 702c4b750c Migrate encryption state on room upgrade (#4411)
* Migrate encryption state on room upgrade

Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>

* Add changelog file
2019-01-21 20:42:58 +11:00
Erik Johnston 25dd56ace3 Fix race when persisting create event (#4404)
* Fix race when persisting create event

When persisting a chunk of DAG it is sometimes requried to do a state
resolution, which requires knowledge of the room version. If this
happens while we're persisting the create event then we need to use that
event rather than attempting to look it up in the database.
2019-01-18 23:17:04 +11:00
Erik Johnston 71b94eac46 Tweak code coverage settings (#4400)
* Tweak code coverage settings

* Fix manifest

* Newsfile

* Fix commit status?
2019-01-18 23:13:14 +11:00
Richard van der Hoff de6888e7ce Remove redundant WrappedConnection (#4409)
* Remove redundant WrappedConnection

The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
2019-01-18 23:07:38 +11:00
Andrew Morgan cb80db8941 Add changelog 2019-01-18 11:22:00 +00:00
Richard van der Hoff 676cf2ee26
Fix incorrect logcontexts after a Deferred was cancelled (#4407) 2019-01-17 14:00:23 +00:00
Richard van der Hoff 9feb5d0b71
sign_request -> build_auth_headers (#4408)
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
Richard van der Hoff 3982a6ee07
Changing macaroon_secret_key no longer logs you out (#4387) 2019-01-16 23:14:41 +00:00
Richard van der Hoff 05e1296649
don't store more remote device lists if they have more than 1K devices (#4397) 2019-01-16 23:14:11 +00:00
Erik Johnston f788c9eb70 Newsfile 2019-01-16 13:46:36 +00:00
Andrej Shadura 3b31a54a6e
Add a changelog entry
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:54:43 +01:00
Neil Johnson 9ec56d6935 ALL_USER_TYPES should be a tuple 2019-01-15 14:38:15 +00:00
Erik Johnston 1a8f4139a5 Newsfile 2019-01-15 11:21:52 +00:00
Richard van der Hoff 34b25dcc8e Silence travis-ci build warnings by removing non-functional python3.6 (#4377)
* Remove non-functional python3.6 in travis env

* changelog
2019-01-12 06:22:56 +11:00
Amber Brown a35c66a00b
Remove duplicates in the user_ips table and add an index (#4370) 2019-01-12 06:21:50 +11:00
Richard van der Hoff 8c818af38e
Merge pull request #4342 from aaronraimist/new-virtualenv
Update README to use new virtualenv (#4328)
2019-01-10 14:59:33 +00:00
Richard van der Hoff d566e6b17a Merge branch 'master' into develop 2019-01-09 15:00:46 +00:00
Richard van der Hoff 998f5225c1 0.34.1 2019-01-09 14:53:54 +00:00
Erik Johnston 55c3e853c5
Merge pull request #4368 from matrix-org/erikj/better_errors
Fixup docstrings for matrixfederationclient
2019-01-09 11:39:28 +00:00
Erik Johnston 484867d35d Merge branch 'develop' into release-v0.34.1 2019-01-09 11:38:13 +00:00
Amber Brown 7960c26fda
Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. (#4369) 2019-01-09 22:26:25 +11:00
Erik Johnston 0dce21ba77 Newsfile 2019-01-09 09:27:03 +00:00
Erik Johnston d91b99abe0 Newsfile 2019-01-08 14:24:05 +00:00
Erik Johnston 83c50bf752 Newsfile 2019-01-08 12:28:35 +00:00
Richard van der Hoff bc1a4b5576 changelog 2019-01-08 11:14:59 +00:00
Erik Johnston b970cb0e96 Refactor request sending to have better excpetions (#4358)
* Correctly retry and back off if we get a HTTPerror response

* Refactor request sending to have better excpetions

MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).

This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
Amber Brown 1dcb086f33
Fix synapse.config.__main__ on python 3 (#4356) 2019-01-08 10:03:09 +11:00
Jason Robinson 26e5abf20d Fix command hint to generate a config file (#4353)
* Fix command hint to generate a config file

When trying to start Synapse without a config file, it will complain
and give a hint towards what command to run. This hinted command
is missing the "report_stats" parameter, which is required with either
yes or no value. Add this to the command.

Not an ideal situation but makes the given command work without the
user getting another error, even though it might be unclear what
"report_stats" represents.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-07 16:28:40 +00:00
Travis Ralston bc1fa8cd01 Add GET account data routes (#4303)
As per https://github.com/matrix-org/matrix-doc/issues/1339
2019-01-07 10:27:54 +00:00
Richard van der Hoff b7c0218812
Check jinja version for consent resource (#4327)
* Raise a ConfigError if an invalid resource is specified

* Require Jinja 2.9 for the consent resource

* changelog
2019-01-07 10:14:31 +00:00
Michael Telatynski a27e501b09 fix the check for whether `is_url` to match all the other ones in codebase (#3405)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2019-01-06 21:43:25 +00:00
Richard van der Hoff 29f20a8a1a
Update debian Conflicts specifications (#4349)
...  to allow installation alongside our matrix-synapse transitional package.
2019-01-04 17:24:13 +00:00
Matthew Hodgson 27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Aaron Raimist 37f8bdc1d5 Update README to not lie about required restart (#4343)
* Update README to not lie about required restart

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-02 16:32:37 +00:00
Neil Johnson 84b6fae1f5 Ensure synchrotrons can access is_support_user in the storage layer 2019-01-02 10:19:59 +00:00
Aaron Raimist 252c0c81fa Update PR template to use absolute links (#4341)
* Update PR template to use absolute links

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-02 07:33:13 +00:00
Richard van der Hoff e9cdfedff3
Avoid packaging _trial_temp directory (#4326)
Make sure we don't put the _trial_temp directory in the package target
directory.

Fixes https://github.com/matrix-org/synapse/issues/4322
2019-01-02 07:30:31 +00:00
Richard van der Hoff 7134832c01
Install the optional dependencies into the debian package (#4325)
since #4298, the optional dependencies are no longer installed with a simple
`pip install .`, which meant that they were not being included in the debian
package.

The easy fix to that is dh_virtualenv --extras, but that needs dh_virtualenv
1.1...
2019-01-02 07:17:39 +00:00
Aaron Raimist 7975d39cbd Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-01 15:40:55 -06:00
Amber Brown d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00
Krithin Sitaram 98df67a8de Remove mention of lt-cred-mech in the sample coturn config. (#4333)
* Remove mention of lt-cred-mech in the sample coturn config.

See https://github.com/coturn/coturn/pull/262 for more context.
Also clean up some minor formatting issues while I'm here.

* Add changelog.

Signed-off-by: Krithin Sitaram <krithin@gmail.com>
2018-12-28 23:31:49 +00:00
Will Hunt 0708f437cc Log roomid along with Unknown room (#4297) 2018-12-24 10:49:42 +01:00
Richard van der Hoff cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms
Add v2 room version
2018-12-24 10:48:59 +01:00
Richard van der Hoff f85676cc93
Return well_known in /login response (#4319)
... as per MSC1730.
2018-12-24 10:44:33 +01:00
Richard van der Hoff 3355c1a9ec
Merge pull request #4317 from de-vri-es/test-metric-prometheus-0.5
Fix test_metrics.py compatibility prometheus_client 0.5
2018-12-23 00:14:47 +01:00
Richard van der Hoff 597dafb5e8
Merge pull request #4316 from matrix-org/rav/fix_docker_upload
Fix circleci config for synapse:latest docker upload
2018-12-21 17:33:34 +01:00
Richard van der Hoff 9c2af7b2c5 Add a script to generate a clean config file (#4315) 2018-12-22 02:04:57 +11:00
Richard van der Hoff f3561f8d86 Fix indentation in default config (#4313)
These settings are not supposed to be under 'listeners'.
2018-12-22 02:01:43 +11:00
Amber Brown ea6abf6724
Fix IP URL previews on Python 3 (#4215) 2018-12-22 01:56:13 +11:00
Amber Brown c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools (#4298) 2018-12-22 01:37:26 +11:00
Maarten de Vries 48b7ff7a35 Fix test_metrics.py compatibility prometheus_client 0.5
prometheus_client 0.5 has a named-tuple Sample type with more member
than the old plain tuple had. This commit makes sure the unit test
detects this and changes the way it reads the sample.

Signed-off-by: Maarten de Vries <maarten@de-vri.es>
2018-12-21 01:53:57 +01:00
Richard van der Hoff d9aaf26539 Fix circleci config for synapse:latest docker upload
Give the image the right tag, so that we can push it.
2018-12-21 00:22:31 +00:00
Nathan Pennie da8628ba2c
Create 4309.bugfix 2018-12-19 14:42:49 -05:00
Erik Johnston 1dc7492ce5 Newsfile 2018-12-18 18:11:56 +00:00
Neil Johnson 7e22cd90f5
ensure can report mau stats when hs.config.mau_stats_only is set (#4305)
* ensure can report mau stats when hs.config.mau_stats_only is set
2018-12-18 14:36:11 +00:00
Neil Johnson d2f7c4e6b1
create support user (#4141)
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Amber Brown e93a0ebf50
Settings Fix deleting e2e room keys on xenial (#4295) 2018-12-14 05:10:31 +11:00
Nad Chishtie cef8ae272a Improved welcome page (#4294)
* Improved static/index.html look, feel and content.

* Added accessibility tags to in-line SVG.

* Grammar.
2018-12-14 02:42:51 +11:00
Richard van der Hoff a83826ae99
Merge pull request #4274 from matrix-org/michaelkaye/update_kernel_dco_link
Update link to kernel.org DCO usage
2018-12-11 20:55:43 +01:00
Richard van der Hoff f208f608cb Merge branch 'release-v0.34.0' into develop 2018-12-11 15:43:20 +00:00
Richard van der Hoff eb835bf65b 0.34.0rc2 2018-12-11 15:35:47 +00:00
Richard van der Hoff 03116da984 remove changelog files 2018-12-11 15:35:10 +00:00
Richard van der Hoff 188945713e
Merge pull request #4290 from matrix-org/rav/remove_webclient
Stop installing Matrix Console by default
2018-12-11 16:24:15 +01:00
Richard van der Hoff bd52978fd7 Merge branch 'rav/disable_pager_in_ci' into rav/welcome_page 2018-12-11 14:13:02 +00:00
Richard van der Hoff 2755a0d48a Disable pager for ci script
... otherwise it hangs
2018-12-11 14:12:22 +00:00
Richard van der Hoff f537432ef9 Add a welcome page to the static resources
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.

The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.

It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.

It's based on the work done by @krombel for #2601.
2018-12-11 13:26:22 +00:00
Richard van der Hoff df96177ca7 Stop installing Matrix Console by default
This is based on the work done by @krombel in #2601.
2018-12-11 13:20:33 +00:00
David Baker 89ac2a5bdb Add 'sandbox' to CSP for media repo (#4284)
* Add 'sandbox' to the CSP for media repo

* Changelog
2018-12-11 04:05:02 +11:00
Will Hunt 989f1167af
Merge pull request #4279 from matrix-org/hs/fix-config-cors
Make /config more CORS-y
2018-12-10 15:24:03 +00:00
Erik Johnston 74c3606c53 Newsfile 2018-12-10 14:15:43 +00:00
Will Hunt 91206e09f2 changelog & isort 2018-12-09 17:39:44 +00:00
Michael Kaye 912a843294
changelog 2018-12-07 17:50:52 +00:00
Richard van der Hoff 30da50a5b8
Initialise user displayname from SAML2 data (#4272)
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
Richard van der Hoff 35e13477cf
Update the example systemd config to use a virtualenv (#4273)
If you're installing as a system package, the system package should have set up
the systemd config, so it's more useful to give an example of running in a
virtualenv here.
2018-12-07 14:43:41 +01:00
Richard van der Hoff c7401a697f
Implement SAML2 authentication (#4267)
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.

What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
2018-12-07 13:11:11 +01:00
Richard van der Hoff c588b9b9e4
Factor SSO success handling out of CAS login (#4264)
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
rkfg ae19a7db8c Prevent crash on pagination. 2018-12-06 13:32:05 +03:00
Richard van der Hoff b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265)
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
Richard van der Hoff 9a3e24a13d drop undocumented dependency on dateutil (#4266)
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 04:52:42 +11:00
Richard van der Hoff e8d98466b0
Implement .well-known handling (#4262)
Sometimes it's useful for synapse to generate its own .well-known file.
2018-12-05 14:38:58 +01:00
Richard van der Hoff a2ed0f287e
Merge pull request #4260 from matrix-org/rav/python3
Notes on upgrading to python3, and README updates.
2018-12-04 14:46:31 +01:00
Richard van der Hoff 75937e9033 Remove obsolete settings from docker homeserver.yaml
These aren't used, because we have a `log_config` setting.
2018-12-04 12:31:00 +00:00
Richard van der Hoff 4acd1a3549 Notes on upgrading to python3, and README updates. 2018-12-04 12:28:24 +00:00
Travis Ralston 1737753a62 Add an option to enable recording IPs for appservice users (#3831) 2018-12-04 12:44:41 +01:00
Amber Brown fd96dd75a3 Fix non-ASCII pushrules (#4248) 2018-12-04 12:44:02 +01:00
Travis Ralston 158ffb92f1 Add an option to disable search for homeservers which may not be interested in it (#4230)
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
Aaron Raimist 512e94d230 Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers (#4224)
* Add note to UPGRADE.rst about removing riot.im from list of trusted identity servers

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-04 11:59:09 +01:00
Ben Parsons b5ac0ffa0a add more detail to logging regarding "More than one row matched" error (#4234) 2018-12-04 11:57:39 +01:00
Richard van der Hoff ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english (#4253)
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
2018-12-04 11:55:52 +01:00
Richard van der Hoff f144c0a210
Merge pull request #4244 from aaronraimist/drop-sent-txt
Drop sent_transactions
2018-12-04 11:41:28 +01:00
Richard van der Hoff 48972ce9d1
Patch defer.inlineCallbacks to check logcontexts in tests (#4205) 2018-12-04 11:30:32 +01:00
Richard van der Hoff a077e710a3
Merge pull request #4250 from matrix-org/hawkowl/pusher-remove-py3
Fix removing pushers on python 3
2018-12-04 11:22:46 +01:00
Richard van der Hoff a484735bb0
Merge pull request #4257 from aaronraimist/add-editorconfig
Add a basic .editorconfig
2018-12-04 11:10:02 +01:00
Richard van der Hoff 52e87fbfbe
Run the AS senders as background processes (#4189)
This should fix some "Starting db connection from sentinel context" warnings,
and will mean we get metrics for these processes.
2018-12-04 10:53:49 +01:00
Aaron Raimist 3518c28aa8
Add a basic .editorconfig
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-03 22:38:47 -06:00
Amber Brown 998ba41493 changelog 2018-12-03 22:28:12 +11:00
Richard van der Hoff c03324294d Workaround for non-ascii event ids (#4241)
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.

Fixes #4226
2018-12-03 21:47:48 +11:00
Aaron Raimist 44dc4c365b
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:10:21 -06:00
Neil Johnson 7039ece8fb Neilj/fix autojoin (#4223)
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
Amber Brown 8ca53fb53e
Report combined coverage to codecov (#4225) 2018-11-28 20:59:31 +11:00
Neil Johnson f9b136a886 Neilj/fix mau initial reserved users (#4211)
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail

* towncrier

* white space
2018-11-28 20:33:41 +11:00
Richard van der Hoff 944d524f18 Support m.login.sso (#4220)
* Clean up the CSS for the fallback login form

I was finding this hard to work with, so simplify a bunch of things. Each
flow is now a form inside a div of class login_flow.

The login_flow class now has a fixed width, as that looks much better than each
flow having a differnt width.

* Support m.login.sso

MSC1721 renames m.login.cas to m.login.sso. This implements the change
(retaining support for m.login.cas for older clients).

* changelog
2018-11-27 18:51:52 +11:00
Richard van der Hoff a44c0a096f Check logcontexts before and after each test (#4190)
* Add better diagnostics to flakey keyring test

* fix interpolation fail

* Check logcontexts before and after each test

* update changelog

* update changelog
2018-11-27 13:47:18 +11:00
Richard van der Hoff 80527b568d Fix more logcontext leaks in tests (#4209) 2018-11-27 13:01:04 +11:00
Richard van der Hoff de8772a655 Do a GC after each test to fix logcontext leaks (#4227)
* Some words about garbage collections and logcontexts

* Do a GC after each test to fix logcontext leaks

This feels like an awful hack, but...

* changelog
2018-11-27 13:00:33 +11:00
Richard van der Hoff 6c18cc4b50 Ignore __pycache__ directories in schema delta dir
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
2018-11-20 22:52:34 +00:00
Neil Johnson 78ba0e7ab8 Remove riot.im from the list of trusted Identity Servers in the default configuration (#4207) 2018-11-20 12:29:25 +01:00
Richard van der Hoff 416c671474
Merge pull request #4204 from matrix-org/rav/logcontext_leak_fixes
Fix some logcontext leaks
2018-11-20 12:19:19 +01:00
Amber Brown 31425d82a3 Merge remote-tracking branch 'origin/master' into develop 2018-11-19 12:55:25 -06:00
Amber Brown 80cac86b2c
Fix fallback auth on Python 3 (#4197) 2018-11-19 12:27:33 -06:00
Richard van der Hoff 0c05da2e2e changelog 2018-11-19 17:07:42 +00:00
Richard van der Hoff 10cdf519aa
Merge pull request #4182 from aaronraimist/update-issue-template
Add a pull request template and add multiple issue templates
2018-11-19 14:24:30 +01:00
Aaron Raimist cc2cf2da97
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-11-18 12:42:08 -06:00
Amber Brown 4285c818ec
Merge pull request #4193 from kivikakk/add-openbsd-prereq
add jpeg to OpenBSD prereq list
2018-11-17 14:27:53 -06:00
Ashe Connor ceca3b2f30
add changelog.d entry 2018-11-17 15:01:02 +11:00
Travis Ralston d75db3df59 Changelog 2018-11-15 20:44:57 -07:00
Amber Brown 8b1affe7d5
Fix Content-Disposition in media repository (#4176) 2018-11-15 15:55:58 -06:00
Travis Ralston 835779f7fb Add option to track MAU stats (but not limit people) (#3830) 2018-11-15 18:08:27 +00:00
Amber Brown df758e155d
Use <meta> tags to discover the per-page encoding of html previews (#4183) 2018-11-15 11:05:08 -06:00
Amber Brown a51288e5d6
Add a coveragerc (#4180) 2018-11-15 10:50:08 -06:00
Richard van der Hoff 4f8bb633c7 Update README for #1491 fix 2018-11-15 10:03:36 +00:00
Richard van der Hoff 0c4dc6fd76 changelog 2018-11-14 10:48:08 +00:00
David Baker 0869566ad3
Merge pull request #4113 from matrix-org/dbkr/e2e_backup_versions_are_numbers
Make e2e backup versions numeric in the DB
2018-11-14 07:55:48 +00:00
Aaron Raimist 9ca1215582
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-11-13 21:46:48 -06:00
David Baker bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
Erik Johnston db5a1c059a
Merge pull request #4166 from matrix-org/erikj/drop_unknown_events
Drop incoming events from federation for unknown rooms
2018-11-09 17:59:34 +00:00
Erik Johnston 7b22421a7b
Merge pull request #4164 from matrix-org/erikj/fix_device_comparison
Fix noop checks when updating device keys
2018-11-08 14:37:20 +00:00
Erik Johnston 94896d7ffe Newsfile 2018-11-08 12:30:25 +00:00
Erik Johnston 06c3d8050f Newsfile 2018-11-08 12:18:41 +00:00
Richard van der Hoff de6223836e changelog 2018-11-08 11:06:28 +00:00
Amber Brown 264cb14402
Port hash_password to Python 3 (#4161)
* port hash_password

* changelog
2018-11-08 04:57:28 +11:00
Amber Brown b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157) 2018-11-08 01:37:43 +11:00
Richard van der Hoff c8ba79327b
Merge pull request #4155 from rubo77/purge-api
add purge_history.sh and purge_remote_media.sh scripts
2018-11-07 14:06:41 +00:00
rubo77 2904d133f3 add purge_history.sh and purge_remote_media.sh scripts to contrib/purge_api/
Signed-off-by: Ruben Barkow <github@r.z11.de>
2018-11-07 14:02:41 +01:00
Amber Brown e62f7f17b3
Remove some boilerplate in tests (#4156) 2018-11-07 03:00:00 +11:00
Travis Ralston 0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142)
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
Hubert Chathi f1087106cf
handle empty backups according to latest spec proposal (#4123)
fixes #4056
2018-11-05 17:59:29 -05:00
Amber Brown efdcbbe46b
Tests for user consent resource (#4140) 2018-11-06 05:53:44 +11:00
Amber Brown 5a63589e80
Add some tests for the HTTP pusher (#4149) 2018-11-06 05:53:24 +11:00
Erik Johnston bc80b3f454 Add helpers for getting prev and auth events (#4139)
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown 0467384d2f
Set the encoding to UTF8 in the default logconfig (#4138) 2018-11-03 02:28:07 +11:00
Erik Johnston 90d713b8c6
Merge pull request #4137 from matrix-org/erikj/clean_up_events
Clean up event accesses and tests
2018-11-02 14:12:49 +00:00
Erik Johnston 76cd7de108 Newsfile 2018-11-02 13:45:56 +00:00
Amber Brown cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Richard van der Hoff efb9343c8c
Merge pull request #4132 from matrix-org/rav/fix_device_list_locking
Fix locked upsert on device_lists_remote_cache
2018-11-02 10:50:53 +00:00
Richard van der Hoff 00f12e00f8
Merge pull request #4133 from matrix-org/travis/fix-terms-auth
Fix logic error that prevented guests from seeing the privacy policy
2018-11-02 10:50:43 +00:00
Erik Johnston b199534518
Merge pull request #4135 from matrix-org/erikj/fix_state_res_none
Fix None exception in state res v2
2018-11-02 10:45:57 +00:00
Richard van der Hoff 1cc6671ec4 changelog 2018-11-02 10:36:13 +00:00
Erik Johnston f05d97e283 Newsfile 2018-11-02 10:32:06 +00:00
Travis Ralston 552f090f62 Changelog 2018-11-01 16:51:11 -06:00
Richard van der Hoff 3149d55b7d
Merge pull request #3778 from z3ntu/patch-1
Fix build of Docker image with docker-compose
2018-11-01 17:34:56 +00:00
Travis Ralston c68aab1536
Merge pull request #4004 from matrix-org/travis/login-terms
Add m.login.terms to the registration flow
2018-11-01 11:03:38 -06:00
Erik Johnston 1b21e771d0
Merge pull request #4128 from matrix-org/erikj/state_res_v2_version
Add STATE_V2_TEST room version
2018-11-01 13:17:57 +00:00
Erik Johnston 62d683161e Newsfile 2018-11-01 11:44:44 +00:00
Amber Brown 073d400b84 Merge branch 'master' into develop 2018-11-01 21:32:12 +11:00
Travis Ralston d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Richard van der Hoff 1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
Richard van der Hoff 94c7fadc98 Attempt to move room aliases on room upgrades 2018-10-31 15:43:57 +00:00
Richard van der Hoff 9b827c40ca
Log some bits about event creation (#4121)
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Richard van der Hoff 60f128a401
Merge pull request #4124 from matrix-org/rav/fix_tox
Attempt to fix tox installs
2018-10-31 15:41:55 +00:00
Amber Brown 916efc8249
Remove fetching keys via the deprecated v1 kex method (#4120) 2018-10-31 23:14:39 +11:00
Amber Brown f79f454485
Remove deprecated v1 key exchange endpoint (#4119) 2018-10-31 22:29:02 +11:00
Richard van der Hoff a2d8bff0dc changelog 2018-10-30 21:21:05 +00:00
Amber Brown e615e95590 changelog 2018-10-31 06:28:11 +11:00
Amber Brown 086e1a8f3e Fix search 500ing (#4122) 2018-10-31 06:26:46 +11:00
Amber Brown 3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown 2e223a8c22
Remove the unused /pull federation API (#4118) 2018-10-31 04:24:59 +11:00
Erik Johnston 0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups
Delete unreferenced state groups during purge
2018-10-30 16:58:22 +00:00
Amber Brown 0dce9e1379
Write some tests for the email pusher (#4095) 2018-10-30 23:55:43 +11:00
David Baker 4eacf0f200 news fragment 2018-10-30 10:05:51 +00:00
Erik Johnston 169851b412
Merge pull request #4109 from matrix-org/erikj/repl_devices
A couple of replication fixes for device lists
2018-10-29 18:16:48 +00:00
Erik Johnston 4f0fa7a120 Newsfile 2018-10-29 18:15:42 +00:00
Erik Johnston 39f419868f Newsfile 2018-10-29 17:38:09 +00:00
Amber Brown 4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff 7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
Richard van der Hoff 56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls
Better handling of odd PLs during room upgrades
2018-10-29 12:48:51 +00:00
Richard van der Hoff bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades
Room version upgrade support
2018-10-29 12:47:20 +00:00
Amber Brown c4b3698a80
Make the replication logger quieter (#4108) 2018-10-29 22:59:44 +11:00
Amber Brown 3b0a85fc8e changelog 2018-10-29 21:54:01 +11:00
Richard van der Hoff db24d7f15e Better handling of odd PLs during room upgrades
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
Richard van der Hoff 5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00
Erik Johnston 03e634dad4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-26 16:22:45 +01:00
Amber Brown 77d70a7646
Port register_new_matrix_user to Python 3 and add tests (#4085) 2018-10-26 22:05:22 +11:00
Richard van der Hoff 6cb2e2448a
Merge pull request #4089 from dekonnection/master
Make Docker image listen on ipv6 as well as ipv4
2018-10-25 19:23:37 +01:00
Richard van der Hoff 68c0ce62d8 changelog 2018-10-25 19:18:25 +01:00
Cédric Laudrel 379376e5e6 Make Docker image listening on ipv6 as well as ipv4
Signed-off-by: Cédric Laudrel <dek@iono.me>
2018-10-25 20:03:47 +02:00
Erik Johnston cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011)
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
Richard van der Hoff e5da60d75d Merge remote-tracking branch 'origin/master' into develop 2018-10-25 17:08:09 +01:00
Erik Johnston c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list
Add config option to control alias creation
2018-10-25 17:04:59 +01:00
Neil Johnson 95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init
fix race condiftion in calling initialise_reserved_users
2018-10-25 16:33:40 +01:00
Erik Johnston b94a43d5b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list 2018-10-25 15:25:31 +01:00
Neil Johnson f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Travis Ralston 54def42c19 Merge branch 'develop' into travis/login-terms 2018-10-24 13:22:59 -06:00
Neil Johnson ea69a84bbb fix style inconsistencies 2018-10-24 17:18:08 +01:00
Neil Johnson 07126e43a4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init 2018-10-24 16:25:39 +01:00
Richard van der Hoff 9f72c209ee
Update changelog.d/3975.feature
Co-Authored-By: neilisfragile <neil@matrix.org>
2018-10-24 14:37:36 +01:00
Richard van der Hoff 3ad359e5be Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation 2018-10-24 11:23:49 +01:00
Richard van der Hoff 7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8
Fix a number of flake8 errors
2018-10-24 11:23:35 +01:00
Erik Johnston 3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase
Add v2 state resolution algorithm
2018-10-24 11:12:12 +01:00
Richard van der Hoff 7e07d25ed6 Allow backslashes in event field filters
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.

The intention here is to allow single-backslashes, but disallow
double-backslashes.
2018-10-24 11:11:24 +01:00
Richard van der Hoff ef771cc4c2 Fix a number of flake8 errors
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
Richard van der Hoff e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts
Clean up the way logcontexts and threads work in the pushers
2018-10-24 09:53:57 +01:00
Erik Johnston cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation
Clean up room alias creation
2018-10-24 09:40:27 +01:00
Neil Johnson a67d8ace9b remove errant exception and style 2018-10-23 17:44:39 +01:00
Travis Ralston 43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
Neil Johnson 329d18b39c remove white space 2018-10-23 15:27:20 +01:00
Neil Johnson 6105c6101f fix race condiftion in calling initialise_reserved_users 2018-10-23 15:24:58 +01:00
Richard van der Hoff b3f6dddad2
Give some more things logcontexts (#4077) 2018-10-23 14:29:17 +01:00
Richard van der Hoff 5c445114d3
Correctly account for cpu usage by background threads (#4074)
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
2018-10-23 13:12:32 +01:00
Richard van der Hoff 1fe6bbb555
Merge pull request #3698 from spantaleev/add-matrix-docker-ansible-deploy
Add information about the matrix-docker-ansible-deploy playbook
2018-10-23 10:11:58 +01:00
Richard van der Hoff abd9914683 Changelog 2018-10-22 16:12:11 +01:00
Richard van der Hoff 7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts
Run MAU queries as background processes
2018-10-22 14:46:59 +01:00
Richard van der Hoff 911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil
Make psutil an explicit dependency
2018-10-22 12:33:21 +01:00
Richard van der Hoff a6f421e812 Run MAU queries as background processes
Fixes #3820
2018-10-20 02:14:35 +01:00
Amber Brown e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
steamport b85fe45f46
Add CL 2018-10-19 21:55:38 +00:00
Richard van der Hoff e5b52d0f94 Make psutil an explicit dependency
As of #4027, we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992.

Fixes #4062.
2018-10-19 21:51:15 +01:00
Richard van der Hoff f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065
Calculate absolute path for email templates
2018-10-19 16:33:54 +01:00
Erik Johnston 056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
Richard van der Hoff cc325c7069 Calculate absolute path for email templates 2018-10-19 14:01:59 +01:00
Amber Brown e404ba9aac
Fix manhole on py3 (pt 2) (#4067) 2018-10-19 22:26:00 +11:00
Amber Brown b69216f768
Make the metrics less racy (#4061) 2018-10-19 21:45:45 +11:00
Erik Johnston f9d6c677ea Newsfile 2018-10-19 10:22:45 +01:00
Erik Johnston 0d31109ed5 Newsfile 2018-10-19 10:14:29 +01:00
Amber Brown 1d17fc52ae changelog 2018-10-19 09:27:10 +11:00
Richard van der Hoff c69026a758 Use the right python when starting workers
We should use the same python to start the workers as we do for the main
synapse (ie, the same one used to run synctl.)
2018-10-18 21:06:30 +01:00
Travis Ralston 49a044aa5f Merge branch 'develop' into travis/login-terms 2018-10-18 09:57:58 -06:00
Richard van der Hoff 03287c350e remove redundant changelog file
this change has been released
2018-10-18 15:09:34 +01:00
Richard van der Hoff c632bc8654 Merge branch 'master' into develop 2018-10-18 15:07:03 +01:00
Erik Johnston e77f24d80a
Merge pull request #4049 from matrix-org/erikj/synctl_colour
Only colourise synctl output when attached to tty
2018-10-18 10:51:14 +01:00
Richard van der Hoff 52e3d3813b prep changelog 2018-10-17 17:40:01 +01:00
Richard van der Hoff 0fd2321629 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-17 17:35:26 +01:00
Richard van der Hoff 1519572961 Ship the email templates as package_data
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.

In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
2018-10-17 16:46:02 +01:00
Erik Johnston 3a5d8d5891 Newsfile 2018-10-17 16:12:45 +01:00
Erik Johnston 1af16acd4c Newsfile 2018-10-17 13:48:09 +01:00
Will Hunt d6a7797dd1 Fix roomlist since tokens on Python 3 (#4046)
Thanks @Half-Shot !!!
2018-10-17 23:04:55 +11:00
Richard van der Hoff 6ec9d8ba0a
Merge pull request #4045 from matrix-org/rav/fix_get_missing_events
Fix incorrect truncation in get_missing_events
2018-10-17 11:40:03 +01:00
Neil Johnson c6584f4b5f clean up config error logic and imports 2018-10-17 11:36:41 +01:00
Richard van der Hoff fc0f13dd03 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Richard van der Hoff 017eb9d17a changelog 2018-10-16 16:31:47 +01:00
Erik Johnston fc954960e9 Newsfile 2018-10-16 16:28:42 +01:00
Richard van der Hoff b8a5b0097c
Various cleanups in the federation client code (#4031)
- Improve logging: log things in the right order, include destination and txids
  in all log lines, don't log successful responses twice

- Fix the docstring on TransportLayerClient.send_transaction

- Don't use treq.request, which is overcomplicated for our purposes: just use a
  twisted.web.client.Agent.

- simplify the logic for setting up the bodyProducer

- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
Travis Ralston f293d124b6 Merge branch 'develop' into travis/login-terms 2018-10-15 14:44:32 -06:00
Amber Brown 24bc15eab4 update changelog 2018-10-15 22:23:13 +11:00
Amber Brown 4e50fe3edb update changelog 2018-10-15 22:22:49 +11:00
David Baker 03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
E2E backups
2018-10-15 10:19:43 +01:00
Slavi Pantaleev c187638ee9 Add information about the matrix-docker-ansible-deploy playbook
Signed-off-by: Slavi Pantaleev <slavi@devture.com>
2018-10-14 21:50:18 +03:00
Ivan Shapovalov 06bc8d2fe5 synapse/app: frontend_proxy.py: actually make workers work on py3 2018-10-14 20:08:39 +03:00
Ivan Shapovalov fb216a22db synapse/visibility.py: fix SyntaxError on py3.7 2018-10-14 20:08:17 +03:00
Travis Ralston 7ede650956 Merge branch 'develop' into travis/login-terms 2018-10-12 16:24:07 -06:00
Travis Ralston e3586f7c06 Merge branch 'develop' into travis/fix-federated-group-requests 2018-10-12 14:49:58 -06:00
Neil Johnson a2bfb778c8 improve auto room join logic, comments and tests 2018-10-12 18:17:36 +01:00
Amber Brown 381d2cfdf0
Make workers work on Py3 (#4027) 2018-10-13 00:14:08 +11:00
Erik Johnston 49840f5ab2 Update newsfile 2018-10-10 11:31:02 +01:00
Erik Johnston 20733857ab Newsfile 2018-10-09 14:18:16 +01:00
David Baker d34657e1f2 Add changelog 2018-10-09 11:15:54 +01:00
Erik Johnston 9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions
Log looping call exceptions
2018-10-09 10:12:10 +01:00
Richard van der Hoff 495975e231 Optimisation for filter_events_for_server
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.

Also, iteritems > items.
2018-10-08 13:46:52 +01:00
Erik Johnston 8164f6daf3 Newsfile 2018-10-05 11:25:09 +01:00
Neil Johnson ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin 2018-10-04 17:26:59 +01:00
Richard van der Hoff c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers
Fix bug in forward_extremity update logic
2018-10-04 16:27:05 +01:00
Erik Johnston d9f3db5081 Newsfile 2018-10-04 16:03:08 +01:00
Amber Brown d86794325f Merge branch 'master' into develop 2018-10-04 22:41:52 +10:00
Amber Brown 92faeb2a3f changelog 2018-10-04 22:38:10 +10:00
Travis Ralston 158d6c75b6 Changelog 2018-10-03 17:54:08 -06:00
Richard van der Hoff a59d899668 Pin to prometheus_client<0.4 to avoid renaming all of our metrics 2018-10-03 17:20:15 +01:00
Erik Johnston 8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion
Fix handling of rejected threepid invites
2018-10-03 14:47:41 +01:00
Erik Johnston 81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration
Fix exception in background metrics collection
2018-10-03 14:14:38 +01:00
Erik Johnston 93a8603904 Newsfile 2018-10-03 11:59:05 +01:00
Erik Johnston 6e0c66f651 Newsfile 2018-10-03 11:40:02 +01:00
Erik Johnston c69faf8c4a Newsfile 2018-10-03 11:29:44 +01:00
Richard van der Hoff 2a4ea3baa8 fix newsfile name 2018-10-03 07:09:25 +01:00
Richard van der Hoff ae61ade891 Fix bug in forward_extremity update logic
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
2018-10-02 23:36:08 +01:00
Amber Brown ccce6f26a6 changelog 2018-10-03 02:16:21 +10:00
Erik Johnston 25baf3b2ac
Merge pull request #3991 from matrix-org/erikj/fix_pop_retry_cache
Fix bug when invalidating destination retry timings
2018-10-02 16:42:28 +01:00
Amber Brown 058a4c665e
Remove Jenkins & other old dev junk (#3988) 2018-10-03 00:59:11 +10:00
Erik Johnston 8f614f842d Newsfile 2018-10-02 15:54:31 +01:00
Richard van der Hoff 2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error
Fix error handling for missing auth_event
2018-10-02 13:57:54 +01:00
Amber Brown 7232917f12
Disable frozen dicts by default (#3987) 2018-10-02 22:53:47 +10:00