Commit Graph

1005 Commits

Author SHA1 Message Date
Richard van der Hoff c475275926
Clarifications for reverse proxy docs (#4607)
Factor out the reverse proxy info to a separate file, add some more info on
reverse-proxying the federation port.
2019-02-11 11:44:28 +00:00
Andrew Morgan eff2042217 Changelog 2019-02-11 11:41:57 +00:00
Amber Brown 6e2a5aa050 ACME Reprovisioning (#4522) 2019-02-11 10:36:26 +00:00
Amber Brown 4ffd10f46d Be tolerant of blank TLS fingerprints config (#4589) 2019-02-11 10:04:27 +00:00
Erik Johnston b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Valentin Anger 2dc2b6e9f1 Allow "unavailable" presence status for /sync (#4592)
* Allow "unavailable" presence status for /sync

Closes #3772, closes #3779

Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>

* Add changelog for PR 4592
2019-02-08 21:09:56 +00:00
Richard van der Hoff 56710c7df5
Fix 'no unique or exclusion constraint' error (#4591)
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
2019-02-08 18:30:46 +00:00
Amber Brown 9cd33d2f4b
Deduplicate some code in synapse.app (#4567) 2019-02-08 17:25:57 +00:00
Erik Johnston acb2ac5863 Update MSC1711 FAQ to be explicit about well-known (#4584)
A surprising number of people are using the well-known method, and are
simply copying the example configuration. This is problematic as the
example includes an explicit port, which causes inbound federation
requests to have the HTTP Host header include the port, upsetting some
reverse proxies.

Given that, we update the well-known example to be more explicit about
the various ways you can set it up, and the consequence of using an
explict port.
2019-02-07 19:30:32 +00:00
Richard van der Hoff 7a22a645b5 Merge branch 'master' into develop 2019-02-07 19:18:48 +00:00
Richard van der Hoff 624b172e08 Merge remote-tracking branch 'origin/release-v0.99.0' 2019-02-07 19:18:26 +00:00
Andrew Morgan c17b128b83 Update ACME docs to include port instructions (#4578) 2019-02-07 19:18:08 +00:00
Hubert Chathi 51b73be63b add changelog entry 2019-02-06 21:39:56 -05:00
Richard van der Hoff 7fe407a87a Merge branch 'master' into develop 2019-02-06 09:50:54 +00:00
Richard van der Hoff d8e63846e2
Fix docker upload job to push -py2 images (#4576) 2019-02-06 09:41:54 +00:00
Richard van der Hoff 2475434080 Merge branch 'master' into develop 2019-02-05 18:44:49 +00:00
Richard van der Hoff 3bd9daf4b8 v0.99.0 2019-02-05 18:33:02 +00:00
Neil Johnson 6585ef4799
Neilj/1711faq (#4572)
MSC1711 certificates FAQ
2019-02-05 17:19:28 +00:00
Richard van der Hoff 40b35fb875
Enable ACME support in the docker image (#4566)
Also:

* Fix wrapping in docker readme
* Clean up some docs on the docker image
* a workaround for #4554
2019-02-05 13:42:21 +00:00
Andrew Morgan 627ecd358e Filter user directory state query to a subset of state events (#4462)
* Filter user directory state query to a subset of state events

* Add changelog
2019-02-05 12:16:28 +00:00
Richard van der Hoff 3ef71a6ea0
Docker: only copy what we need to the build image (#4562)
There are two reasons this is a good thing:

 * first, it means that you don't end up with stuff kicking around your working
   copy ending up in the build image by mistake (which can upset the pip
   install process)

 * second: it means that the docker image cache is more effective, and we can
   reuse docker images when iterating on the docker stuff.
2019-02-05 11:44:40 +00:00
Richard van der Hoff bf1e4d96ad
Fix default ACME config for py2 (#4564)
Fixes #4559
2019-02-05 11:37:33 +00:00
Richard van der Hoff 9a75c0b52e
switch docker image to py3 by default (#4558)
Switch the matrixdotorg/synapse:latest Docker image to use python 3
2019-02-05 11:33:26 +00:00
Richard van der Hoff d7e27a1f08
fix typo in config comments (#4557) 2019-02-05 11:32:45 +00:00
Andrew Morgan bcc78bb0b8 Merge branch 'release-v0.99.0' of github.com:matrix-org/synapse into anoa/acme_docs 2019-02-01 15:57:32 +00:00
Richard van der Hoff 30fd2f89db 0.99.0rc4 2019-02-01 15:52:28 +00:00
Andrew Morgan c5fc09322c Add changelog 2019-02-01 15:05:10 +00:00
Richard van der Hoff ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
2019-02-01 14:37:22 +00:00
Richard van der Hoff f8db967d5a Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-02-01 13:20:15 +00:00
Richard van der Hoff 9763a73af0 Merge branch 'release-v0.99.0' into develop 2019-02-01 12:30:22 +00:00
Richard van der Hoff f0ba34f581 Fix noisy "twisted.internet.task.TaskStopped" errors in logs
Fixes #4003
2019-02-01 12:22:57 +00:00
Richard van der Hoff 8a21b03fba Treat an invalid .well-known the same as an absent one
... basically, carry on and fall back to SRV etc.
2019-02-01 11:37:31 +00:00
Richard van der Hoff 0390c961ac changelog 2019-02-01 09:40:58 +00:00
Richard van der Hoff d428b46346 Update federation routing logic to check .well-known before SRV 2019-01-31 23:14:18 +00:00
Richard van der Hoff 85129d7068 v0.99.0rc3 2019-01-31 18:35:38 +00:00
Richard van der Hoff 07dfe148de
Add some debug for membership syncing issues (#4538)
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan bbb97a35fd Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable 2019-01-31 18:11:00 +00:00
Richard van der Hoff e707e7b38d
Fix infinite loop when an event is redacted in a v3 room (#4535) 2019-01-31 15:34:17 +00:00
Andrew Morgan 563f6a832b Reject large transactions on federation (#4513)
* Reject large transactions on federation

* Add changelog

* lint

* Simplify large transaction handling
2019-01-31 11:44:04 +00:00
Richard van der Hoff 35f544410a update debian installation instructions (#4526)
* update debian installation instructions

* docs PR is docs
2019-01-31 10:29:15 +00:00
Andrew Morgan cf9a2676d0 Add changelog 2019-01-30 19:04:48 +00:00
Richard van der Hoff 6fba9fd20c Merge remote-tracking branch 'origin/release-v0.99.0' into develop 2019-01-30 17:02:32 +00:00
Richard van der Hoff b8b898666e v0.99.0rc2 2019-01-30 16:31:07 +00:00
Matthew Hodgson ad7ac8853c by default include m.room.encryption on invites (#3902)
* by default include m.room.encryption on invites

* fix constant

* changelog
2019-01-30 16:26:13 +00:00
Richard van der Hoff c74b96755c Merge remote-tracking branch 'origin/develop' into release-v0.99.0 2019-01-30 16:23:28 +00:00
Amber Brown fbaee26c68 ACME Upgrade Docs (#4528) 2019-01-30 16:22:37 +00:00
Erik Johnston e25ab58c5e Newsfile 2019-01-30 15:50:28 +00:00
Richard van der Hoff a5d0c771a3 0.99.0rc1 2019-01-30 15:11:18 +00:00
Erik Johnston 6587b0b89b
Merge pull request #4472 from matrix-org/neilj/room_capabilities
Server capabilities support
2019-01-30 14:26:56 +00:00
Erik Johnston a4f52a33fe Fix replication for room v3 (#4523)
* Fix replication for room v3

We were not correctly quoting the path fragments over http replication,
which meant that it exploded when the event IDs had a slash in them

* Newsfile
2019-01-30 14:19:52 +00:00
Richard van der Hoff 7615a8ced1 ACME config cleanups (#4525)
* Handle listening for ACME requests on IPv6 addresses

the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.

* Clean up the default ACME config

make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.

* newsfile
2019-01-30 14:17:55 +00:00
Erik Johnston e6a7a15f93 Newsfile 2019-01-30 12:17:38 +00:00
Richard van der Hoff a79034aedf
Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups
Tiny .well-known fixes
2019-01-30 11:47:24 +00:00
Richard van der Hoff c7b24ac3d0
Follow redirects on .well-known (#4520) 2019-01-30 11:43:33 +00:00
Amber Brown f6813919e8
SIGHUP for TLS cert reloading (#4495) 2019-01-30 11:00:02 +00:00
Richard van der Hoff 283753c33a newsfile 2019-01-30 10:59:21 +00:00
Richard van der Hoff bc5f6e1797
Add a caching layer to .well-known responses (#4516) 2019-01-30 10:55:25 +00:00
Amber Brown 3f189c902e
Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Neil Johnson ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities 2019-01-30 10:28:08 +00:00
Richard van der Hoff 457fbfaf22
Merge pull request #4486 from xperimental/workaround-4216
Implement workaround for login error.
2019-01-30 07:06:11 +00:00
Robert Jacob 2a7f0b8953 Implement workaround for login error.
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
2019-01-30 01:06:39 +01:00
Erik Johnston ebcffbc3eb Newsfile 2019-01-29 23:09:10 +00:00
Erik Johnston 67b82f1336 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:58:38 +00:00
Erik Johnston 7740eddd04
Merge pull request #4514 from matrix-org/erikj/remove_event_id
Remove usages of event ID's domain
2019-01-29 22:54:25 +00:00
Erik Johnston a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston 6f9cdc2d47
Merge pull request #4483 from matrix-org/erikj/event_v2
Implement event format V2
2019-01-29 21:40:00 +00:00
Andrew Morgan 03b086647f
Merge pull request #4512 from matrix-org/anoa/consent_dir
Check consent dir path on startup
2019-01-29 20:08:18 +00:00
Erik Johnston 3680bc18e9 Newsfile 2019-01-29 18:06:11 +00:00
Erik Johnston b40abe0724 Newsfile 2019-01-29 18:02:26 +00:00
Richard van der Hoff cc2d650ef7
Relax requirement for a content-type on .well-known (#4511) 2019-01-29 16:49:17 +00:00
Travis Ralston d02c5ccb11
Merge pull request #4498 from matrix-org/travis/fix-docs-public_baseurl
Don't recommend :8448 to people on public_baseurl
2019-01-29 09:06:16 -07:00
Neil Johnson 19259d903c update to reflect broadening scope 2019-01-29 16:01:46 +00:00
Andrew Morgan 9adbc912b3 Add changelog 2019-01-29 15:34:06 +00:00
Amber Brown 6bd4374636
Do not generate self-signed TLS certificates by default. (#4509) 2019-01-29 14:09:10 +00:00
Erik Johnston b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
Richard van der Hoff 99e36d5e24
Implement MSC1708 (.well-known lookups for server routing) (#4489) 2019-01-29 13:53:02 +00:00
Erik Johnston 2562319821
Merge pull request #4510 from matrix-org/erikj/fixup_compute_event_signature
Fixup calls to `comput_event_signature`
2019-01-29 13:35:19 +00:00
Richard van der Hoff 5488cadaae
Enable configuring test log level via env var (#4506)
I got fed up with always adding '@unittest.DEBUG' every time I needed to debug a test.
2019-01-29 12:07:00 +00:00
Erik Johnston 7d1024d574 Newsfile 2019-01-29 11:58:16 +00:00
Erik Johnston 0c55b7701c Newsfile 2019-01-29 11:42:33 +00:00
Erik Johnston ff37acb8ce
Merge pull request #4496 from matrix-org/erikj/invite_fallback
Implement fallback for V2 invite API
2019-01-29 11:28:23 +00:00
Erik Johnston a388d59d44 Newsfile 2019-01-29 11:13:08 +00:00
Amber Brown f815bd7feb
Make linearizer more quiet (#4507) 2019-01-29 11:05:31 +00:00
Erik Johnston 073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Amber Brown 94fb63e44f
Fix typo in upserts code (#4505)
* fix obvious problem :|

* changelog
2019-01-29 10:04:23 +00:00
Erik Johnston 17709f8f9c
Merge pull request #4493 from matrix-org/erikj/refactor_event_signing
Refactor event signing to work on dicts
2019-01-29 09:52:54 +00:00
Richard van der Hoff f2b553d656 Use SimpleResolverComplexifier in tests (#4497)
two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes
unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we
rely on to give consistent behaviour between python 2 and 3.
2019-01-29 09:38:29 +00:00
Erik Johnston f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Travis Ralston c4045647eb
Create 4498.misc 2019-01-28 12:16:39 -07:00
Erik Johnston 8cbc99cc19 Newsfile 2019-01-28 17:36:06 +00:00
Amber Brown 5d976c0c7c
Fix worker TLS (#4492)
* load cert

* changelog

* fix
2019-01-28 17:18:33 +00:00
Erik Johnston 1977a9b006 Newsfile 2019-01-28 17:05:04 +00:00
Erik Johnston b8bea3424f Newsfile 2019-01-28 16:47:12 +00:00
Amber Brown 7072fe3084
Fix UPSERTs on SQLite 3.24+ (#4477) 2019-01-28 15:43:32 +00:00
Andrew Morgan 4026d555fa Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-28 14:08:24 +00:00
Aaron Raimist 57a3e96e8e Remove --process-dependency-links from UPGRADE.rst (#4485)
* Remove --process-dependency-links from UPGRADE.rst

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-01-28 13:54:25 +00:00
Richard van der Hoff ff05ad147a changelog 2019-01-28 10:34:30 +00:00
Richard van der Hoff d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487)
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.

There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.

The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
2019-01-28 09:56:59 +00:00
Erik Johnston f01c7488ab Newsfile 2019-01-25 18:32:50 +00:00
Erik Johnston b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Erik Johnston 431e485914 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert 2019-01-25 14:11:17 +00:00
Richard van der Hoff 4a3f138832
Fix quoting for allowed_local_3pids example config (#4476)
If you use double-quotes here, you have to escape your backslashes. It's much
easier with single-quotes.

(Note that the existing double-backslashes are already interpreted by python's
""" parsing.)
2019-01-25 13:57:52 +00:00
Richard van der Hoff 8520bc3109
Fix Host header sent by MatrixFederationAgent (#4468)
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
Andrew Morgan 0b3fd1401f Don't require sqlite3 when using postgres (#4466) 2019-01-25 22:25:02 +11:00
Neil Johnson a3f0556bea towncrier 2019-01-25 11:15:41 +00:00
Andrew Morgan 821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Andrew Morgan b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston 8dcfa6e75c Newsfile 2019-01-25 10:48:40 +00:00
Erik Johnston 5d881cbcb8 Newsfile 2019-01-25 10:37:13 +00:00
Erik Johnston 62514bb81b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 2019-01-25 10:07:08 +00:00
Erik Johnston edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston 80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston 664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2
Add support for persisting event format versions
2019-01-24 18:32:30 +00:00
Andrew Morgan 50c396a7ee
Merge pull request #4461 from matrix-org/anoa/room_dir_quick_fix
Add a 60s timeout to filtered room directory queries
2019-01-24 15:24:35 +00:00
Richard van der Hoff 4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup
MatrixFederationAgent: Look up the right SRV record
2019-01-24 13:57:51 +00:00
Richard van der Hoff 8c58c10697
Generate the debian config during build (#4444)
Rather than hardcoding a config which we always forget to update, generate it
from the default config.
2019-01-24 13:39:01 +00:00
Richard van der Hoff f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json (#4460)
This could cause exceptions if the id server returned 4xx responses.
2019-01-24 13:38:29 +00:00
Richard van der Hoff afd69a0920 Look up the right SRV record 2019-01-24 13:31:43 +00:00
Neil Johnson 10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check
Neilj/fix threepid auth check
2019-01-24 13:02:50 +00:00
Andrew Morgan 2a360e834f Add changelog 2019-01-24 12:47:35 +00:00
Neil Johnson 5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc
Clarify docs for public_baseurl
2019-01-24 12:36:09 +00:00
Amber Brown 0e27501ee5
Fix UPSERT check (#4459) 2019-01-24 22:57:41 +11:00
David Baker 1f2058fca5 Changelog 2019-01-24 10:57:12 +00:00
Amber Brown 58f6c48183
Use native UPSERTs where possible (#4306) 2019-01-24 21:31:54 +11:00
Richard van der Hoff 97fd29c019
Don't send IP addresses as SNI (#4452)
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
2019-01-24 09:34:44 +00:00
Erik Johnston 183738f469 Newsfile 2019-01-23 20:04:57 +00:00
Willem Mulder d528406cb8 Fix error message for optional dependencies
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
2019-01-23 18:44:57 +01:00
Erik Johnston c5a125b24f Update newsfile 2019-01-23 17:26:06 +00:00
Erik Johnston 4a8b715679 Newsfile 2019-01-23 17:25:27 +00:00
Erik Johnston a4ef8d8dd5 Newsfile 2019-01-23 17:24:44 +00:00
Andrew Morgan 6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport
Add metric for user dir current event stream position
2019-01-23 15:58:40 +00:00
Richard van der Hoff a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent
Move SRV magic into an Agent-like thing
2019-01-23 15:50:25 +00:00
Andrew Morgan cb0e637a94 Add changelog 2019-01-23 15:38:27 +00:00
Richard van der Hoff 2f88881c93
debian package: symlink to python-3.X (#4433)
In the debian package, make the virtualenv symlink python to /usr/bin/python3.X
rather than /usr/bin/python3. Also make sure we depend on the right python3.x
package.

This might help a bit with subtle failures when people install a package from
the wrong distro (https://github.com/matrix-org/synapse/issues/4431).
2019-01-23 11:43:04 +00:00
Erik Johnston 4cd50d983d Newsfile 2019-01-23 11:30:24 +00:00
Erik Johnston 90743c9d89 Fixup removal of duplicate `user_ips` rows (#4432)
* Remove unnecessary ORDER BY clause

* Add logging

* Newsfile
2019-01-23 19:45:18 +11:00
Amber Brown 6129e52f43
Support ACME for certificate provisioning (#4384) 2019-01-23 19:39:06 +11:00
Jason Robinson db33634b1d Collapse changelog to one line
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson 1d2c69fee8 Add changelog for openid resource addition
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Richard van der Hoff c66f4bf7f1 changelog 2019-01-22 20:34:35 +00:00
Neil Johnson c99c2d58d7 move guard out of is_threepid_reserved and into register.py 2019-01-22 17:47:00 +00:00
Neil Johnson d619b113ed Fix None guard in config.server.is_threepid_reserved 2019-01-22 16:52:29 +00:00
Erik Johnston 12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup
Fix bug when removing duplicate rows from user_ips
2019-01-22 16:51:57 +00:00
Neil Johnson 388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration
Config option to disable requesting MSISDN on registration
2019-01-22 16:23:08 +00:00
Erik Johnston c658425e6f Newsfile 2019-01-22 13:34:10 +00:00
Andrew Morgan 117bc94cd2 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-22 13:16:51 +00:00
Richard van der Hoff 6bfa735a69
Make key fetches use regular federation client (#4426)
All this magic is redundant.
2019-01-22 11:04:20 +00:00
Andrew Morgan 25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Andrew Morgan 1f18c7cfc9 Add changelog 2019-01-22 11:00:04 +00:00
Richard van der Hoff 33a55289cb
Refactor and bugfix for resove_service (#4427) 2019-01-22 10:59:27 +00:00