075ff3ede9
Change default timeout value from 0 to None
2019-01-24 15:10:22 +00:00
5b1dc94083
Use self.clock instead of datetime
2019-01-24 14:59:50 +00:00
a2d85144e5
isort
2019-01-24 14:22:26 +00:00
4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup
...
MatrixFederationAgent: Look up the right SRV record
2019-01-24 13:57:51 +00:00
f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json ( #4460 )
...
This could cause exceptions if the id server returned 4xx responses.
2019-01-24 13:38:29 +00:00
afd69a0920
Look up the right SRV record
2019-01-24 13:31:43 +00:00
10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check
...
Neilj/fix threepid auth check
2019-01-24 13:02:50 +00:00
5541645e80
lint
2019-01-24 12:45:32 +00:00
068aa1d228
Time out filtered room dir queries after 60s
2019-01-24 12:44:27 +00:00
5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc
...
Clarify docs for public_baseurl
2019-01-24 12:36:09 +00:00
0e27501ee5
Fix UPSERT check ( #4459 )
2019-01-24 22:57:41 +11:00
92d8a068ad
Clarify docs for public_baseurl
...
This is leading to problems with people upgrading to clients that
support MSC1730 because people have this misconfigured, so try
to make the docs completely unambiguous.
2019-01-24 10:52:06 +00:00
58f6c48183
Use native UPSERTs where possible ( #4306 )
2019-01-24 21:31:54 +11:00
97fd29c019
Don't send IP addresses as SNI ( #4452 )
...
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files ; let's do the same here.
2019-01-24 09:34:44 +00:00
7c288c2250
Clarify the invite flows
2019-01-23 20:07:47 +00:00
07f62da55a
Remove unnecessary '_sign_event'
2019-01-23 20:04:57 +00:00
886e5acc76
Store rejected remote invite events as outliers
...
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
6a41d2a187
Add room_version param to get_pdu
...
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
67cd4dad81
Implement MSC 1813 - Add room version to make APIs
...
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport
...
Add metric for user dir current event stream position
2019-01-23 15:58:40 +00:00
a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent
...
Move SRV magic into an Agent-like thing
2019-01-23 15:50:25 +00:00
82a92ba535
Add metric for user dir current event stream position
2019-01-23 15:34:47 +00:00
be1065af59
isort
2019-01-23 11:48:16 +00:00
c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
90743c9d89
Fixup removal of duplicate `user_ips` rows ( #4432 )
...
* Remove unnecessary ORDER BY clause
* Add logging
* Newsfile
2019-01-23 19:45:18 +11:00
6129e52f43
Support ACME for certificate provisioning ( #4384 )
2019-01-23 19:39:06 +11:00
7021784d46
put resolve_service in an object
...
this makes it easier to stub things out for tests.
2019-01-22 20:35:12 +00:00
53a327b4d5
Require that service_name be a byte string
...
it is only ever a bytes now, so let's enforce that.
2019-01-22 20:35:12 +00:00
fe212bbe4a
Kill off matrix_federation_endpoint
...
this thing is now redundant.
2019-01-22 20:34:35 +00:00
7871146667
Make MatrixFederationClient use MatrixFederationAgent
...
... instead of the matrix_federation_endpoint
2019-01-22 20:34:35 +00:00
44be7513bf
MatrixFederationAgent
...
Pull the magic that is currently in matrix_federation_endpoint and friends into
an agent-like thing
2019-01-22 20:34:35 +00:00
c99c2d58d7
move guard out of is_threepid_reserved and into register.py
2019-01-22 17:47:00 +00:00
d619b113ed
Fix None guard in config.server.is_threepid_reserved
2019-01-22 16:52:29 +00:00
12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup
...
Fix bug when removing duplicate rows from user_ips
2019-01-22 16:51:57 +00:00
7f503f83b9
Refactor to rewrite the SQL instead
2019-01-22 16:31:05 +00:00
388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration
...
Config option to disable requesting MSISDN on registration
2019-01-22 16:23:08 +00:00
1c9704f8ab
Don't shadow params
2019-01-22 16:20:33 +00:00
766a172b99
lint
2019-01-22 13:51:40 +00:00
2557531f0f
Fix bug when removing duplicate rows from user_ips
...
This was caused by accidentally overwritting a `last_seen` variable
in a for loop, causing the wrong value to be written to the progress
table. The result of which was that we didn't scan sections of the table
when searching for duplicates, and so some duplicates did not get
deleted.
2019-01-22 13:33:46 +00:00
117bc94cd2
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-22 13:16:51 +00:00
277e50462d
Do not return in a deferred function
2019-01-22 12:40:26 +00:00
c4875d8c76
Prevent duplicate room IDs in m.direct
2019-01-22 12:13:46 +00:00
c433f61091
Ensure new filter is actually created
2019-01-22 12:06:36 +00:00
c9bfb058d8
Fix a bug with single-room search searching all rooms
...
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
8086a5c05e
Fix comments
2019-01-22 11:16:23 +00:00
6bfa735a69
Make key fetches use regular federation client ( #4426 )
...
All this magic is redundant.
2019-01-22 11:04:20 +00:00
48951f437f
Join logic covers both room creator and arbitrary users
2019-01-22 11:00:04 +00:00
8c85f0833d
tags, m.direct copying over correctly
2019-01-22 11:00:04 +00:00
25d64a846a
Fix typos
2019-01-22 11:00:04 +00:00
ea8903fcc9
Migrating dm and room tags work for migrator
2019-01-22 11:00:04 +00:00
887ca93a1b
Prevent crash on user who doesn't have any direct rooms
2019-01-22 11:00:04 +00:00
4ff6d22245
Preserve DM status of a room on upgrade
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-22 11:00:04 +00:00
75942af1db
Fix typo
2019-01-22 11:00:04 +00:00
33a55289cb
Refactor and bugfix for resove_service ( #4427 )
2019-01-22 10:59:27 +00:00
23b0813599
Require ECDH key exchange & remove dh_params ( #4429 )
...
* remove dh_params and set better cipher string
2019-01-22 21:58:50 +11:00
1b53cc3cb4
fix line length
2019-01-21 15:17:20 +00:00
5349262302
Config option to disable requesting MSISDN on registration
2019-01-21 14:59:37 +00:00
35e1d67b4e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_v2_invite_server
2019-01-21 14:04:19 +00:00
5f54765587
Merge pull request #4390 from matrix-org/erikj/versioned_fed_apis
...
Add groundwork for new versions of federation APIs
2019-01-21 11:44:05 +00:00
702c4b750c
Migrate encryption state on room upgrade ( #4411 )
...
* Migrate encryption state on room upgrade
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
* Add changelog file
2019-01-21 20:42:58 +11:00
25dd56ace3
Fix race when persisting create event ( #4404 )
...
* Fix race when persisting create event
When persisting a chunk of DAG it is sometimes requried to do a state
resolution, which requires knowledge of the room version. If this
happens while we're persisting the create event then we need to use that
event rather than attempting to look it up in the database.
2019-01-18 23:17:04 +11:00
de6888e7ce
Remove redundant WrappedConnection ( #4409 )
...
* Remove redundant WrappedConnection
The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
2019-01-18 23:07:38 +11:00
df3a661e4a
Search for messages across predecessor rooms
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
676cf2ee26
Fix incorrect logcontexts after a Deferred was cancelled ( #4407 )
2019-01-17 14:00:23 +00:00
9feb5d0b71
sign_request -> build_auth_headers ( #4408 )
...
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
3982a6ee07
Changing macaroon_secret_key no longer logs you out ( #4387 )
2019-01-16 23:14:41 +00:00
05e1296649
don't store more remote device lists if they have more than 1K devices ( #4397 )
2019-01-16 23:14:11 +00:00
64cf6788d9
Depend on pymacaroons >= 0.13.0 instead on pymacaroons-pynacl
...
Since 0.13.0, pymacaroons works correctly with pynacl, so there
isn’t any more reason to depend on an outdated pynacl fork.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:54:41 +01:00
fab948120f
Use msgpack instead of msgpack-python
...
The package msgpack-python has been deprecated.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:53:49 +01:00
9ec56d6935
ALL_USER_TYPES should be a tuple
2019-01-15 14:38:15 +00:00
4a4d2e17bc
Add /v2/invite federation API
2019-01-15 13:22:44 +00:00
bb63e7ca4f
Add groundwork for new versions of federation APIs
2019-01-15 11:14:34 +00:00
a35c66a00b
Remove duplicates in the user_ips table and add an index ( #4370 )
2019-01-12 06:21:50 +11:00
522dada206
Merge remote-tracking branch 'origin/master' into develop
2019-01-11 02:22:48 +11:00
6dc06c3775
version
2019-01-11 01:56:37 +11:00
e0910d0145
Merge branch rav/macaroon_key_fix_0.34 into rav/macaroon_key_fix_0.34.1
...
Fixes #4371
2019-01-10 14:12:50 +00:00
ba41aeed6a
Revert "Fix macaroon_secret_key fallback logic"
...
This is already fixed in 0.34.1, by 59f93bbefc522c55e
2019-01-10 14:09:26 +00:00
aa70d24125
Merge branch 'rav/macaroon_key_fix' into rav/macaroon_key_fix_0.34
2019-01-10 12:58:33 +00:00
efc522c55e
Fix macaroon_secret_key fallback logic
2019-01-10 12:57:27 +00:00
566947ff34
Skip macaroon check for access tokens in the db
2019-01-10 12:57:21 +00:00
353f2407b7
Fix fallback to signing key for macaroon-secret-key
2019-01-10 12:42:56 +00:00
d566e6b17a
Merge branch 'master' into develop
2019-01-09 15:00:46 +00:00
998f5225c1
0.34.1
2019-01-09 14:53:54 +00:00
55c3e853c5
Merge pull request #4368 from matrix-org/erikj/better_errors
...
Fixup docstrings for matrixfederationclient
2019-01-09 11:39:28 +00:00
484867d35d
Merge branch 'develop' into release-v0.34.1
2019-01-09 11:38:13 +00:00
7960c26fda
Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. ( #4369 )
2019-01-09 22:26:25 +11:00
34ea14139d
Fixup docstrings for matrixfederationclient
2019-01-09 09:25:59 +00:00
d422570e9b
Use RequestSendFailed when fail to parse content type headers
2019-01-08 14:22:18 +00:00
1371d5b798
Don't log stack traces for HTTP error responses
2019-01-08 12:28:30 +00:00
ba85786d71
0.34.1rc1
2019-01-08 11:15:34 +00:00
b970cb0e96
Refactor request sending to have better excpetions ( #4358 )
...
* Correctly retry and back off if we get a HTTPerror response
* Refactor request sending to have better excpetions
MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).
This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
1dcb086f33
Fix synapse.config.__main__ on python 3 ( #4356 )
2019-01-08 10:03:09 +11:00
bc1fa8cd01
Add GET account data routes ( #4303 )
...
As per https://github.com/matrix-org/matrix-doc/issues/1339
2019-01-07 10:27:54 +00:00
b7c0218812
Check jinja version for consent resource ( #4327 )
...
* Raise a ConfigError if an invalid resource is specified
* Require Jinja 2.9 for the consent resource
* changelog
2019-01-07 10:14:31 +00:00
a27e501b09
fix the check for whether `is_url` to match all the other ones in codebase ( #3405 )
...
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2019-01-06 21:43:25 +00:00
27128145e6
fix NPE in /messages by checking if all events were filtered out ( #4330 )
2019-01-02 23:37:39 +00:00
84b6fae1f5
Ensure synchrotrons can access is_support_user in the storage layer
2019-01-02 10:19:59 +00:00
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console ( #4334 )
2018-12-29 23:12:30 +11:00
0708f437cc
Log roomid along with Unknown room ( #4297 )
2018-12-24 10:49:42 +01:00
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms
...
Add v2 room version
2018-12-24 10:48:59 +01:00
f85676cc93
Return well_known in /login response ( #4319 )
...
... as per MSC1730.
2018-12-24 10:44:33 +01:00
9c2af7b2c5
Add a script to generate a clean config file ( #4315 )
2018-12-22 02:04:57 +11:00
f3561f8d86
Fix indentation in default config ( #4313 )
...
These settings are not supposed to be under 'listeners'.
2018-12-22 02:01:43 +11:00
ea6abf6724
Fix IP URL previews on Python 3 ( #4215 )
2018-12-22 01:56:13 +11:00
c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools ( #4298 )
2018-12-22 01:37:26 +11:00
0b26feb422
Merge branch 'master' into develop
2018-12-20 16:48:47 +00:00
6dd7271243
Prepare 0.34 release
2018-12-20 12:07:15 +00:00
d3c9c562c6
Debian packaging via dh_virtualenv
2018-12-20 11:35:23 +00:00
1a6d5bfa08
Debian packaging via dh_virtualenv ( #4285 )
2018-12-20 11:33:29 +00:00
1c0051114a
Add 'sandbox' to CSP for media repo ( #4284 )
...
* Add 'sandbox' to the CSP for media repo
* Changelog
2018-12-20 11:09:18 +00:00
81b513416e
Fixed line length
2018-12-19 14:12:33 -05:00
668e6625b0
Fix the variable names used for account_data
2018-12-19 06:53:02 -05:00
df89f8afb8
Add v2 room version
2018-12-18 18:10:37 +00:00
7e22cd90f5
ensure can report mau stats when hs.config.mau_stats_only is set ( #4305 )
...
* ensure can report mau stats when hs.config.mau_stats_only is set
2018-12-18 14:36:11 +00:00
d2f7c4e6b1
create support user ( #4141 )
...
Allow for the creation of a support user.
A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
128294d3e5
Improved welcome page ( #4294 )
...
* Improved static/index.html look, feel and content.
* Added accessibility tags to in-line SVG.
* Grammar.
2018-12-14 05:13:56 +11:00
c7144b105c
Settings Fix deleting e2e room keys on xenial ( #4295 )
2018-12-14 05:13:33 +11:00
e93a0ebf50
Settings Fix deleting e2e room keys on xenial ( #4295 )
2018-12-14 05:10:31 +11:00
cef8ae272a
Improved welcome page ( #4294 )
...
* Improved static/index.html look, feel and content.
* Added accessibility tags to in-line SVG.
* Grammar.
2018-12-14 02:42:51 +11:00
f208f608cb
Merge branch 'release-v0.34.0' into develop
2018-12-11 15:43:20 +00:00
eb835bf65b
0.34.0rc2
2018-12-11 15:35:47 +00:00
188945713e
Merge pull request #4290 from matrix-org/rav/remove_webclient
...
Stop installing Matrix Console by default
2018-12-11 16:24:15 +01:00
95c1f6500b
Merge pull request #4289 from matrix-org/rav/welcome_page
...
Add a welcome page to the static resources
2018-12-11 16:24:06 +01:00
f537432ef9
Add a welcome page to the static resources
...
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.
The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.
It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.
It's based on the work done by @krombel for #2601 .
2018-12-11 13:26:22 +00:00
df96177ca7
Stop installing Matrix Console by default
...
This is based on the work done by @krombel in #2601 .
2018-12-11 13:20:33 +00:00
89ac2a5bdb
Add 'sandbox' to CSP for media repo ( #4284 )
...
* Add 'sandbox' to the CSP for media repo
* Changelog
2018-12-11 04:05:02 +11:00
fee831c040
Move imports to one line
2018-12-10 13:52:33 +00:00
466c1f3e01
Use `send_cors`
2018-12-10 13:11:37 +00:00
91206e09f2
changelog & isort
2018-12-09 17:39:44 +00:00
dbf736ba66
Make /config more CORS-y
2018-12-09 13:27:22 +00:00
30da50a5b8
Initialise user displayname from SAML2 data ( #4272 )
...
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
c7401a697f
Implement SAML2 authentication ( #4267 )
...
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.
What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
2018-12-07 13:11:11 +01:00
c588b9b9e4
Factor SSO success handling out of CAS login ( #4264 )
...
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
b0c24a66ec
Rip out half-implemented m.login.saml2 support ( #4265 )
...
* Rip out half-implemented m.login.saml2 support
This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.
* drop undocumented dependency on dateutil
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
9a3e24a13d
drop undocumented dependency on dateutil ( #4266 )
...
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 04:52:42 +11:00
e8d98466b0
Implement .well-known handling ( #4262 )
...
Sometimes it's useful for synapse to generate its own .well-known file.
2018-12-05 14:38:58 +01:00
e55983defe
Prepare 0.34.0rc1
2018-12-04 13:52:16 +00:00
1737753a62
Add an option to enable recording IPs for appservice users ( #3831 )
2018-12-04 12:44:41 +01:00
fd96dd75a3
Fix non-ASCII pushrules ( #4248 )
2018-12-04 12:44:02 +01:00
158ffb92f1
Add an option to disable search for homeservers which may not be interested in it ( #4230 )
...
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
b5ac0ffa0a
add more detail to logging regarding "More than one row matched" error ( #4234 )
2018-12-04 11:57:39 +01:00
ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english ( #4253 )
...
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252 . Fundamentally the right answer is to drop support for Python 2.
2018-12-04 11:55:52 +01:00
f144c0a210
Merge pull request #4244 from aaronraimist/drop-sent-txt
...
Drop sent_transactions
2018-12-04 11:41:28 +01:00
a077e710a3
Merge pull request #4250 from matrix-org/hawkowl/pusher-remove-py3
...
Fix removing pushers on python 3
2018-12-04 11:22:46 +01:00
52e87fbfbe
Run the AS senders as background processes ( #4189 )
...
This should fix some "Starting db connection from sentinel context" warnings,
and will mean we get metrics for these processes.
2018-12-04 10:53:49 +01:00
d3c61ef906
fix type error
2018-12-03 22:27:41 +11:00
c03324294d
Workaround for non-ascii event ids ( #4241 )
...
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.
Fixes #4226
2018-12-03 21:47:48 +11:00
704c5298f0
Drop sent_transactions
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:07:35 -06:00
7039ece8fb
Neilj/fix autojoin ( #4223 )
...
* Fix auto join failures for servers that require user consent
* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
f9b136a886
Neilj/fix mau initial reserved users ( #4211 )
...
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail
* towncrier
* white space
2018-11-28 20:33:41 +11:00
Andrew Morgan