3bc238629e
0.99.1rc2
2019-02-13 14:46:18 +00:00
c1dfd6a18a
Merge remote-tracking branch 'origin/release-v0.99.1' into develop
2019-02-13 14:27:45 +00:00
309f3bb322
Update synapse/app/_base.py
...
Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>
2019-02-13 13:24:27 +00:00
bb4fd8f927
Run `black` on user directory code ( #4635 )
2019-02-13 23:05:32 +11:00
767686af48
Use `listen_tcp` for the replication listener
...
Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with
what we do elsewhere.
2019-02-13 11:59:04 +00:00
2a5a15aff8
Improve logging around listening services
...
I wanted to bring listen_tcp into line with listen_ssl in terms of returning a
list of ports, and wanted to check that was a safe thing to do - hence the
logging in `refresh_certificate`.
Also, pull the 'Synapse now listening' message up to homeserver.py, because it
was being duplicated everywhere else.
2019-02-13 11:58:54 +00:00
e3a0300431
Special-case the default bind_addresses for metrics listener
...
turns out it doesn't really support ipv6, so let's hack around that by only
listening on ipv4 by default.
2019-02-13 11:48:56 +00:00
93f7d2df3e
Comments
2019-02-12 16:03:40 +00:00
6a8f902edb
Raise an appropriate error message if sentry_sdk missing
2019-02-12 16:01:41 +00:00
ef2228c890
Basic sentry integration
2019-02-12 13:55:58 +00:00
d2fa7b7e99
Update changelog and version
2019-02-12 13:22:25 +00:00
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
...
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze
...
Analyze user_ips before running deduplication
2019-02-12 13:05:09 +00:00
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe
...
Reduce user_ips bloat during dedupe background update
2019-02-12 13:02:58 +00:00
495ea92350
Fix pep8
2019-02-12 12:40:42 +00:00
483ba85c7a
Analyze user_ips before running deduplication
...
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
362d80b770
Reduce user_ips bloat during dedupe background update
...
The background update to remove duplicate rows naively deleted and
reinserted the duplicates. For large tables with a large number of
duplicates this causes a lot of bloat (with postgres), as the inserted
rows are appended to the table, since deleted rows will not be
overwritten until a VACUUM has happened.
This should hopefully also help ensure that the query in the last batch
uses the correct index, as inserting a large number of new rows without
analyzing will upset the query planner.
2019-02-12 11:39:34 +00:00
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings
...
fix self-signed cert notice from generate-config
2019-02-12 11:24:45 +00:00
a4ce91396b
Disable TLS by default ( #4614 )
2019-02-12 10:52:08 +00:00
32b781bfe2
Fix error when loading cert if tls is disabled ( #4618 )
...
If TLS is disabled, it should not be an error if no cert is given.
Fixes #4554 .
2019-02-12 10:51:31 +00:00
dfc846a316
fix self-signed cert notice from generate-config
...
fixes #4620
2019-02-12 10:37:59 +00:00
0ca2908653
fix tests
2019-02-11 22:01:27 +00:00
4fddf8fc77
Infer no_tls from presence of TLS listeners
...
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
2019-02-11 21:39:14 +00:00
15272f837c
Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work
2019-02-11 21:34:19 +00:00
9645728619
Don't create server contexts when TLS is disabled
...
we aren't going to use them anyway.
2019-02-11 21:32:01 +00:00
be794c7cf7
Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work
2019-02-11 21:16:00 +00:00
2129dd1a02
Fail cleanly if listener config lacks a 'port'
...
... otherwise we would fail with a mysterious KeyError or something later.
2019-02-11 21:15:01 +00:00
086f6f27d4
Logging improvements around TLS certs
...
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
2019-02-11 21:02:06 +00:00
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates ( #4611 )
...
It's nothing to do with refreshing the certificates. No idea why it was here.
2019-02-11 18:03:30 +00:00
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating
...
add updating of backup versions
2019-02-11 13:45:49 +00:00
24b7f3916d
Clean up default listener configuration ( #4586 )
...
Rearrange the comments to try to clarify them, and expand on what some of it
means.
Use a sensible default 'bind_addresses' setting.
For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
2019-02-11 12:50:30 +00:00
a126f86eec
Transfer Server ACLs on room upgrade
2019-02-11 11:30:37 +00:00
6e2a5aa050
ACME Reprovisioning ( #4522 )
2019-02-11 10:36:26 +00:00
4ffd10f46d
Be tolerant of blank TLS fingerprints config ( #4589 )
2019-02-11 10:04:27 +00:00
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
...
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
2dc2b6e9f1
Allow "unavailable" presence status for /sync ( #4592 )
...
* Allow "unavailable" presence status for /sync
Closes #3772 , closes #3779
Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>
* Add changelog for PR 4592
2019-02-08 21:09:56 +00:00
56710c7df5
Fix 'no unique or exclusion constraint' error ( #4591 )
...
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
2019-02-08 18:30:46 +00:00
9cd33d2f4b
Deduplicate some code in synapse.app ( #4567 )
2019-02-08 17:25:57 +00:00
afae8442b5
make sure version is in body and wrap in linearizer queue
...
also add tests
2019-02-08 01:32:45 -05:00
d9e424bf64
re-try to make isort happy
2019-02-06 22:18:41 -05:00
9ff620a518
fix import to make isort happy
2019-02-06 21:32:52 -05:00
8248637173
add new endpoint to update backup versions
2019-02-06 17:57:10 -05:00
664c81e8b7
return proper error codes for some 404s
2019-02-06 17:47:22 -05:00
2475434080
Merge branch 'master' into develop
2019-02-05 18:44:49 +00:00
3bd9daf4b8
v0.99.0
2019-02-05 18:33:02 +00:00
627ecd358e
Filter user directory state query to a subset of state events ( #4462 )
...
* Filter user directory state query to a subset of state events
* Add changelog
2019-02-05 12:16:28 +00:00
bf1e4d96ad
Fix default ACME config for py2 ( #4564 )
...
Fixes #4559
2019-02-05 11:37:33 +00:00
d7e27a1f08
fix typo in config comments ( #4557 )
2019-02-05 11:32:45 +00:00
30fd2f89db
0.99.0rc4
2019-02-01 15:52:28 +00:00
ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation
...
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
2019-02-01 14:37:22 +00:00
f8db967d5a
Merge remote-tracking branch 'origin/release-v0.99.0' into develop
2019-02-01 13:20:15 +00:00
fa794980ec
Merge pull request #4544 from matrix-org/rav/skip_invalid_well_known
...
Treat an invalid .well-known the same as an absent one
2019-02-01 13:18:36 +00:00
e9779a6f8f
Fix b'ab' noise in logs
2019-02-01 12:34:31 +00:00
9763a73af0
Merge branch 'release-v0.99.0' into develop
2019-02-01 12:30:22 +00:00
f0ba34f581
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
...
Fixes #4003
2019-02-01 12:22:57 +00:00
8a21b03fba
Treat an invalid .well-known the same as an absent one
...
... basically, carry on and fall back to SRV etc.
2019-02-01 11:37:31 +00:00
3c8a41140e
Cache failures to parse .well-known
...
Also add a Measure block around the .well-known fetch
2019-02-01 00:37:52 +00:00
24d59c7568
better logging for federation connections
2019-01-31 23:18:20 +00:00
d428b46346
Update federation routing logic to check .well-known before SRV
2019-01-31 23:14:18 +00:00
c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
...
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
625385d684
Merge branch 'release-v0.99.0' into develop
2019-01-31 18:43:20 +00:00
85129d7068
v0.99.0rc3
2019-01-31 18:35:38 +00:00
d239f67c25
Raise an exception instead of returning None
2019-01-31 18:34:15 +00:00
07dfe148de
Add some debug for membership syncing issues ( #4538 )
...
I can't figure out what's going on with #4422 and #4436 ; perhaps this will help.
2019-01-31 18:30:40 +00:00
3ed3cb4339
New function for getting room's create event
2019-01-31 18:21:39 +00:00
bbb97a35fd
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable
2019-01-31 18:11:00 +00:00
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room ( #4535 )
2019-01-31 15:34:17 +00:00
563f6a832b
Reject large transactions on federation ( #4513 )
...
* Reject large transactions on federation
* Add changelog
* lint
* Simplify large transaction handling
2019-01-31 11:44:04 +00:00
fb50934b8f
lint
2019-01-31 11:34:45 +00:00
6fba9fd20c
Merge remote-tracking branch 'origin/release-v0.99.0' into develop
2019-01-30 17:02:32 +00:00
d621c5562e
Copy over non-federatable trait on room upgrade
2019-01-30 16:33:51 +00:00
b8b898666e
v0.99.0rc2
2019-01-30 16:31:07 +00:00
ad7ac8853c
by default include m.room.encryption on invites ( #3902 )
...
* by default include m.room.encryption on invites
* fix constant
* changelog
2019-01-30 16:26:13 +00:00
c74b96755c
Merge remote-tracking branch 'origin/develop' into release-v0.99.0
2019-01-30 16:23:28 +00:00
81b7e7eed3
Update constants.py
...
remove trailing ,
2019-01-30 16:11:36 +00:00
ed8c5e4cda
Fix remote invite rejections not comming down sync
...
This was broken in PR #4405 , commit 886e5ac
2019-01-30 15:46:27 +00:00
a5d0c771a3
0.99.0rc1
2019-01-30 15:11:18 +00:00
6587b0b89b
Merge pull request #4472 from matrix-org/neilj/room_capabilities
...
Server capabilities support
2019-01-30 14:26:56 +00:00
a4f52a33fe
Fix replication for room v3 ( #4523 )
...
* Fix replication for room v3
We were not correctly quoting the path fragments over http replication,
which meant that it exploded when the event IDs had a slash in them
* Newsfile
2019-01-30 14:19:52 +00:00
7615a8ced1
ACME config cleanups ( #4525 )
...
* Handle listening for ACME requests on IPv6 addresses
the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.
* Clean up the default ACME config
make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.
* newsfile
2019-01-30 14:17:55 +00:00
e87d7a4b0f
Raise ConfigError instead
2019-01-30 12:48:09 +00:00
270f212a2a
_listener_http should return a list
2019-01-30 12:14:50 +00:00
a79034aedf
Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups
...
Tiny .well-known fixes
2019-01-30 11:47:24 +00:00
c7b24ac3d0
Follow redirects on .well-known ( #4520 )
2019-01-30 11:43:33 +00:00
f6813919e8
SIGHUP for TLS cert reloading ( #4495 )
2019-01-30 11:00:02 +00:00
09a1a6b55e
fix exception text
2019-01-30 10:58:52 +00:00
928c50b59a
Also jitter the invalid cache period
2019-01-30 10:58:52 +00:00
b37e8c9572
Merge branch 'neilj/room_capabilities' of github.com:matrix-org/synapse into neilj/room_capabilities
2019-01-30 10:56:47 +00:00
f834d98402
isort
2019-01-30 10:55:42 +00:00
bc5f6e1797
Add a caching layer to .well-known responses ( #4516 )
2019-01-30 10:55:25 +00:00
3f189c902e
Fix flake8 ( #4519 )
2019-01-30 10:53:17 +00:00
ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities
2019-01-30 10:28:08 +00:00
c5a0f82cca
define room dispositions for use in exposing room capabilities
2019-01-30 10:24:24 +00:00
9c850d9d5e
formatting and use constants where available
2019-01-30 10:23:26 +00:00
2f46804055
Populate default room version from Constants
2019-01-30 09:39:10 +00:00
c7837dce24
reflect that rooms v3 is a stable room version
2019-01-30 09:33:30 +00:00
457fbfaf22
Merge pull request #4486 from xperimental/workaround-4216
...
Implement workaround for login error.
2019-01-30 07:06:11 +00:00
2a7f0b8953
Implement workaround for login error.
...
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
2019-01-30 01:06:39 +01:00
0b24d58e05
No vdh tests!
2019-01-29 23:11:48 +00:00
a1b0e1879b
Enable room version v3
2019-01-29 23:09:10 +00:00
67b82f1336
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:58:38 +00:00
afeea319df
Fixup comment
2019-01-29 22:55:29 +00:00
7740eddd04
Merge pull request #4514 from matrix-org/erikj/remove_event_id
...
Remove usages of event ID's domain
2019-01-29 22:54:25 +00:00
ff2f65d737
Update comment
2019-01-29 22:35:36 +00:00
655ce037fd
check event format version not room version
2019-01-29 22:33:43 +00:00
f46a818ce5
kill vdh test some more
2019-01-29 22:02:58 +00:00
a696c48133
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:00:33 +00:00
c21b7cbc09
Update synapse/storage/events_worker.py
2019-01-29 21:53:48 +00:00
4db252c073
Check redaction state when event is pulled out of the database
2019-01-29 21:48:36 +00:00
6d23ec2111
Fix typo
2019-01-29 21:45:53 +00:00
b5d510ad64
Remove unused arg
2019-01-29 21:45:28 +00:00
6f9cdc2d47
Merge pull request #4483 from matrix-org/erikj/event_v2
...
Implement event format V2
2019-01-29 21:40:00 +00:00
47e2dd1994
Drop vdh support
2019-01-29 21:24:34 +00:00
38590a4870
Add docstring
2019-01-29 21:22:47 +00:00
82165eeb05
Update synapse/storage/events_worker.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-01-29 21:14:39 +00:00
03b086647f
Merge pull request #4512 from matrix-org/anoa/consent_dir
...
Check consent dir path on startup
2019-01-29 20:08:18 +00:00
84af577356
Implement event format v2
2019-01-29 18:06:11 +00:00
e4bef9d470
rework format of change password capability
2019-01-29 18:04:56 +00:00
610f0830b0
Don't assert an event must have an event ID
2019-01-29 18:02:26 +00:00
840068bd78
Only check event ID domain for signatures for V1 events
...
In future version events won't have an event ID, so we won't be able to
do this check.
2019-01-29 18:02:02 +00:00
8e3d34e3c5
Use event origin for filtering incoming events
...
We only process events sent to us from a server if the event ID matches
the server, to help guard against federation storms. We replace this
with a check against the event origin.
2019-01-29 16:57:00 +00:00
55d9024835
Use snder and not event ID domain to check if ours
...
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
2019-01-29 16:54:23 +00:00
cc2d650ef7
Relax requirement for a content-type on .well-known ( #4511 )
2019-01-29 16:49:17 +00:00
b1fffca345
Remove event ID usage when checking if new room
...
The event ID is changing, so we can no longer get the domain from it. On
the other hand, the check is unnecessary.
2019-01-29 16:15:02 +00:00
770b823445
Only check event IDs domain signed event for V1 and V2
...
Since newer versions of events don't have the same format for event ID.
2019-01-29 16:15:00 +00:00
d02c5ccb11
Merge pull request #4498 from matrix-org/travis/fix-docs-public_baseurl
...
Don't recommend :8448 to people on public_baseurl
2019-01-29 09:06:16 -07:00
f03b3a7a3a
support change_password in capabilities end-point
2019-01-29 15:58:37 +00:00
e65a17b26f
Check consent dir path on startup
2019-01-29 15:30:33 +00:00
6bd4374636
Do not generate self-signed TLS certificates by default. ( #4509 )
2019-01-29 14:09:10 +00:00
b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
...
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
99e36d5e24
Implement MSC1708 (.well-known lookups for server routing) ( #4489 )
2019-01-29 13:53:02 +00:00
b82a76c384
Finish comment...
2019-01-29 13:50:59 +00:00
2562319821
Merge pull request #4510 from matrix-org/erikj/fixup_compute_event_signature
...
Fixup calls to `comput_event_signature`
2019-01-29 13:35:19 +00:00
5891a6edc8
Correctly set context.app_service
2019-01-29 12:09:10 +00:00
fb99dae9c8
Don't set event_id twice
2019-01-29 12:08:23 +00:00
64c1bd1d21
Remove dead function
2019-01-29 12:06:28 +00:00
7709d2bd16
Implement rechecking of redactions
2019-01-29 11:56:20 +00:00
7a3ec5b022
Add RoomVersions.V3 constant, without enabling it
...
We add the constant, but don't add it to the known room versions. This
lets us start adding V3 logic, but the servers will never join or create
V3 rooms
2019-01-29 11:55:33 +00:00
6598992b01
Fixup calls to `comput_event_signature`
...
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.
This fixes PR #4493 commit 855a151
2019-01-29 11:41:58 +00:00
a124025dab
enforce auth for capabilities endpoint
2019-01-29 11:37:56 +00:00
ff37acb8ce
Merge pull request #4496 from matrix-org/erikj/invite_fallback
...
Implement fallback for V2 invite API
2019-01-29 11:28:23 +00:00
5180f12bae
Replace usage of builder.user_id with builder.sender
...
`.user_id` is proxed to `.sender` in FrozenEvent, so this has no
functional change
2019-01-29 11:18:38 +00:00
be47cfa9c9
Refactor event building into EventBuilder
...
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
554ca58ea1
Make add_hashes_and_signatures operate on dicts
2019-01-29 11:12:38 +00:00
f815bd7feb
Make linearizer more quiet ( #4507 )
2019-01-29 11:05:31 +00:00
073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
...
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
40638ae7f5
Remove duplicate checks
2019-01-29 10:37:40 +00:00
9fa3c6ffa3
Fix up error messages
2019-01-29 10:36:46 +00:00
28efc80723
Fold validate into validate_new
2019-01-29 10:34:49 +00:00
b6b73a0bcf
Fix receiving events from federation via a worker
...
This bug was introduced in PR #4470 , commit 678a92cb56
2019-01-29 10:30:26 +00:00
327b992e17
register capabilities servlet
2019-01-29 10:28:35 +00:00
94fb63e44f
Fix typo in upserts code ( #4505 )
...
* fix obvious problem :|
* changelog
2019-01-29 10:04:23 +00:00
17709f8f9c
Merge pull request #4493 from matrix-org/erikj/refactor_event_signing
...
Refactor event signing to work on dicts
2019-01-29 09:52:54 +00:00
f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
...
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
6901ac7e9d
Don't recommend :8448 to people on public_baseurl
2019-01-28 12:15:22 -07:00
d414f30019
Implement fallback for V2 invite API
...
If the room version is either 1 or 2 then a server should retry failed
`/v2/invite` requests with the v1 API
2019-01-28 17:33:25 +00:00
d758d5310e
Correctly use default room version if none is set
2019-01-28 17:26:39 +00:00
5d976c0c7c
Fix worker TLS ( #4492 )
...
* load cert
* changelog
* fix
2019-01-28 17:18:33 +00:00
b872c7b1b4
Split up event validation between event and builder
...
The validator was being run on the EventBuilder objects, and so the
validator only checked a subset of fields. With the upcoming
EventBuilder refactor even fewer fields will be there to validate.
To get around this we split the validation into those that can be run
against an EventBuilder and those run against a fully fledged event.
2019-01-28 17:00:14 +00:00
855a151015
Refactor event signing to work on dicts
...
This is in preparation for making EventBuilder format agnostic, which
means event signing should be done against the event dict rather than
the EventBuilder object.
2019-01-28 16:42:10 +00:00
7072fe3084
Fix UPSERTs on SQLite 3.24+ ( #4477 )
2019-01-28 15:43:32 +00:00
f0e96ab66a
Change return syntax in doc string
2019-01-28 14:09:45 +00:00
4026d555fa
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-28 14:08:24 +00:00
1ce463963d
Reuse predecessor method
2019-01-28 14:08:18 +00:00
3bd0f1a4a3
docstrings for _RoutingResult
2019-01-28 12:43:09 +00:00
0fd5b3b53e
Handle IP literals explicitly
...
We don't want to be doing .well-known lookups on these guys.
2019-01-28 10:34:30 +00:00
51958df766
MatrixFederationAgent: factor out routing logic
...
This is going to get too big and unmanageable.
2019-01-28 10:34:30 +00:00
d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent ( #4487 )
...
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.
There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.
The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
2019-01-28 09:56:59 +00:00
ae2a957dba
Pass through room version to event auth
2019-01-25 18:31:41 +00:00
893107be78
backout v3
2019-01-25 17:27:36 +00:00
b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
...
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
431e485914
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert
2019-01-25 14:11:17 +00:00
4a3f138832
Fix quoting for allowed_local_3pids example config ( #4476 )
...
If you use double-quotes here, you have to escape your backslashes. It's much
easier with single-quotes.
(Note that the existing double-backslashes are already interpreted by python's
""" parsing.)
2019-01-25 13:57:52 +00:00
8520bc3109
Fix Host header sent by MatrixFederationAgent ( #4468 )
...
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
9244a3089e
Fixes
2019-01-25 11:48:38 +00:00
da0d2219d2
Clean up direct_rooms access
2019-01-25 11:37:12 +00:00
8265995498
Use python magic
2019-01-25 11:26:06 +00:00
0b3fd1401f
Don't require sqlite3 when using postgres ( #4466 )
2019-01-25 22:25:02 +11:00
c4cdafa81f
Destructure account data tuple before use
2019-01-25 11:24:28 +00:00
516456b763
Remove unnecessary null check
2019-01-25 11:22:14 +00:00
6f3fda79ce
Move room_tag declaration to be closer to its use
2019-01-25 11:21:25 +00:00
95f871fc0d
Support room version capabilities in CS API (MSC1804)
2019-01-25 11:16:29 +00:00
821b65aeb5
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-25 11:09:53 +00:00
0862d35b8e
Move tag and direct state copying into separate function
2019-01-25 11:09:34 +00:00
53ef4da8c2
track unstable room v3
2019-01-25 11:04:11 +00:00
b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
...
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
1953067136
Disable native upserts for sqlite, as they don't work
2019-01-25 10:46:49 +00:00
678a92cb56
Replace missed usages of FrozenEvent
2019-01-25 10:32:30 +00:00
a50cf929c1
Require event format version to parse or create events
2019-01-25 10:32:19 +00:00
62514bb81b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813
2019-01-25 10:07:08 +00:00
edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
...
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
...
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2
...
Add support for persisting event format versions
2019-01-24 18:32:30 +00:00
d148c43050
Review comments
2019-01-24 18:31:23 +00:00
26f44164c8
Review comments
2019-01-24 18:28:00 +00:00
5ee1f997a8
Update make_membership_event docs
2019-01-24 18:08:08 +00:00
9139b87be4
Remove unecessary setting of outlier bit
2019-01-24 18:04:02 +00:00
b8082a5445
Use term 'out of band membership' instead
2019-01-24 17:33:19 +00:00
a383289b0d
Merge branch 'anoa/full_search_upgraded_rooms' of github.com:matrix-org/synapse into anoa/full_search_upgraded_rooms
2019-01-24 17:23:51 +00:00
e1781b043b
Remove redundant create event None check
2019-01-24 17:23:39 +00:00
03c85335d1
Apply suggestions from code review
...
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
8ea509a935
Update synapse/api/filtering.py
...
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:21:35 +00:00
50c396a7ee
Merge pull request #4461 from matrix-org/anoa/room_dir_quick_fix
...
Add a 60s timeout to filtered room directory queries
2019-01-24 15:24:35 +00:00
075ff3ede9
Change default timeout value from 0 to None
2019-01-24 15:10:22 +00:00
5b1dc94083
Use self.clock instead of datetime
2019-01-24 14:59:50 +00:00
a2d85144e5
isort
2019-01-24 14:22:26 +00:00
4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup
...
MatrixFederationAgent: Look up the right SRV record
2019-01-24 13:57:51 +00:00
f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json ( #4460 )
...
This could cause exceptions if the id server returned 4xx responses.
2019-01-24 13:38:29 +00:00
afd69a0920
Look up the right SRV record
2019-01-24 13:31:43 +00:00
10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check
...
Neilj/fix threepid auth check
2019-01-24 13:02:50 +00:00
5541645e80
lint
2019-01-24 12:45:32 +00:00
068aa1d228
Time out filtered room dir queries after 60s
2019-01-24 12:44:27 +00:00
5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc
...
Clarify docs for public_baseurl
2019-01-24 12:36:09 +00:00
0e27501ee5
Fix UPSERT check ( #4459 )
2019-01-24 22:57:41 +11:00
92d8a068ad
Clarify docs for public_baseurl
...
This is leading to problems with people upgrading to clients that
support MSC1730 because people have this misconfigured, so try
to make the docs completely unambiguous.
2019-01-24 10:52:06 +00:00
58f6c48183
Use native UPSERTs where possible ( #4306 )
2019-01-24 21:31:54 +11:00
97fd29c019
Don't send IP addresses as SNI ( #4452 )
...
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files ; let's do the same here.
2019-01-24 09:34:44 +00:00
7c288c2250
Clarify the invite flows
2019-01-23 20:07:47 +00:00
07f62da55a
Remove unnecessary '_sign_event'
2019-01-23 20:04:57 +00:00
886e5acc76
Store rejected remote invite events as outliers
...
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
d528406cb8
Fix error message for optional dependencies
...
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
2019-01-23 18:44:57 +01:00
6a41d2a187
Add room_version param to get_pdu
...
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
67cd4dad81
Implement MSC 1813 - Add room version to make APIs
...
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport
...
Add metric for user dir current event stream position
2019-01-23 15:58:40 +00:00
a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent
...
Move SRV magic into an Agent-like thing
2019-01-23 15:50:25 +00:00
82a92ba535
Add metric for user dir current event stream position
2019-01-23 15:34:47 +00:00
be1065af59
isort
2019-01-23 11:48:16 +00:00
c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
6f680241bd
Fix flake8 issues
...
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:53:48 +02:00
90743c9d89
Fixup removal of duplicate `user_ips` rows ( #4432 )
...
* Remove unnecessary ORDER BY clause
* Add logging
* Newsfile
2019-01-23 19:45:18 +11:00
6129e52f43
Support ACME for certificate provisioning ( #4384 )
2019-01-23 19:39:06 +11:00
0516dc4d85
Remove openid resource from default config
...
Instead document it commented out.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
d39b7b6d38
Document `servlet_groups` parameters
...
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
82e13662c0
Split federation OpenID userinfo endpoint out of the federation resource
...
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.
The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
a17bac171f
Make SynapseHomeServer _http_listener use self.get_reactor()
...
For all the homeserver classes, only the FrontendProxyServer passes
its reactor when doing the http listen. Looking at previous PR's looks
like this was introduced to make it possible to write a test, otherwise
when you try to run a test with the test homeserver it tries to
do a real bind to a port. Passing the reactor that the homeserver
is instantiated with should probably be the right thing to do anyway?
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
4f8f41c824
Make FederationReaderServer _http_listen use self.get_reactor()
...
For all the homeserver classes, only the FrontendProxyServer passes
its reactor when doing the http listen. Looking at previous PR's looks
like this was introduced to make it possible to write a test, otherwise
when you try to run a test with the test homeserver it tries to
do a real bind to a port. Passing the reactor that the homeserver
is instantiated with should probably be the right thing to do anyway?
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
899e60be80
Add parameterized Python module to test dependencies
...
Allows running parameterized tests. BSD license.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
7021784d46
put resolve_service in an object
...
this makes it easier to stub things out for tests.
2019-01-22 20:35:12 +00:00
53a327b4d5
Require that service_name be a byte string
...
it is only ever a bytes now, so let's enforce that.
2019-01-22 20:35:12 +00:00
fe212bbe4a
Kill off matrix_federation_endpoint
...
this thing is now redundant.
2019-01-22 20:34:35 +00:00
7871146667
Make MatrixFederationClient use MatrixFederationAgent
...
... instead of the matrix_federation_endpoint
2019-01-22 20:34:35 +00:00
44be7513bf
MatrixFederationAgent
...
Pull the magic that is currently in matrix_federation_endpoint and friends into
an agent-like thing
2019-01-22 20:34:35 +00:00
c99c2d58d7
move guard out of is_threepid_reserved and into register.py
2019-01-22 17:47:00 +00:00
d619b113ed
Fix None guard in config.server.is_threepid_reserved
2019-01-22 16:52:29 +00:00
12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup
...
Fix bug when removing duplicate rows from user_ips
2019-01-22 16:51:57 +00:00
7f503f83b9
Refactor to rewrite the SQL instead
2019-01-22 16:31:05 +00:00
388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration
...
Config option to disable requesting MSISDN on registration
2019-01-22 16:23:08 +00:00
1c9704f8ab
Don't shadow params
2019-01-22 16:20:33 +00:00
766a172b99
lint
2019-01-22 13:51:40 +00:00
2557531f0f
Fix bug when removing duplicate rows from user_ips
...
This was caused by accidentally overwritting a `last_seen` variable
in a for loop, causing the wrong value to be written to the progress
table. The result of which was that we didn't scan sections of the table
when searching for duplicates, and so some duplicates did not get
deleted.
2019-01-22 13:33:46 +00:00
117bc94cd2
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-22 13:16:51 +00:00
277e50462d
Do not return in a deferred function
2019-01-22 12:40:26 +00:00
c4875d8c76
Prevent duplicate room IDs in m.direct
2019-01-22 12:13:46 +00:00
c433f61091
Ensure new filter is actually created
2019-01-22 12:06:36 +00:00
c9bfb058d8
Fix a bug with single-room search searching all rooms
...
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
8086a5c05e
Fix comments
2019-01-22 11:16:23 +00:00
6bfa735a69
Make key fetches use regular federation client ( #4426 )
...
All this magic is redundant.
2019-01-22 11:04:20 +00:00
48951f437f
Join logic covers both room creator and arbitrary users
2019-01-22 11:00:04 +00:00
8c85f0833d
tags, m.direct copying over correctly
2019-01-22 11:00:04 +00:00
25d64a846a
Fix typos
2019-01-22 11:00:04 +00:00
ea8903fcc9
Migrating dm and room tags work for migrator
2019-01-22 11:00:04 +00:00
887ca93a1b
Prevent crash on user who doesn't have any direct rooms
2019-01-22 11:00:04 +00:00
4ff6d22245
Preserve DM status of a room on upgrade
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-22 11:00:04 +00:00
75942af1db
Fix typo
2019-01-22 11:00:04 +00:00
33a55289cb
Refactor and bugfix for resove_service ( #4427 )
2019-01-22 10:59:27 +00:00
23b0813599
Require ECDH key exchange & remove dh_params ( #4429 )
...
* remove dh_params and set better cipher string
2019-01-22 21:58:50 +11:00
1b53cc3cb4
fix line length
2019-01-21 15:17:20 +00:00
5349262302
Config option to disable requesting MSISDN on registration
2019-01-21 14:59:37 +00:00
35e1d67b4e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_v2_invite_server
2019-01-21 14:04:19 +00:00
5f54765587
Merge pull request #4390 from matrix-org/erikj/versioned_fed_apis
...
Add groundwork for new versions of federation APIs
2019-01-21 11:44:05 +00:00
702c4b750c
Migrate encryption state on room upgrade ( #4411 )
...
* Migrate encryption state on room upgrade
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
* Add changelog file
2019-01-21 20:42:58 +11:00
25dd56ace3
Fix race when persisting create event ( #4404 )
...
* Fix race when persisting create event
When persisting a chunk of DAG it is sometimes requried to do a state
resolution, which requires knowledge of the room version. If this
happens while we're persisting the create event then we need to use that
event rather than attempting to look it up in the database.
2019-01-18 23:17:04 +11:00
de6888e7ce
Remove redundant WrappedConnection ( #4409 )
...
* Remove redundant WrappedConnection
The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
2019-01-18 23:07:38 +11:00
df3a661e4a
Search for messages across predecessor rooms
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
676cf2ee26
Fix incorrect logcontexts after a Deferred was cancelled ( #4407 )
2019-01-17 14:00:23 +00:00
9feb5d0b71
sign_request -> build_auth_headers ( #4408 )
...
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
3982a6ee07
Changing macaroon_secret_key no longer logs you out ( #4387 )
2019-01-16 23:14:41 +00:00
05e1296649
don't store more remote device lists if they have more than 1K devices ( #4397 )
2019-01-16 23:14:11 +00:00
64cf6788d9
Depend on pymacaroons >= 0.13.0 instead on pymacaroons-pynacl
...
Since 0.13.0, pymacaroons works correctly with pynacl, so there
isn’t any more reason to depend on an outdated pynacl fork.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:54:41 +01:00
fab948120f
Use msgpack instead of msgpack-python
...
The package msgpack-python has been deprecated.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:53:49 +01:00
9ec56d6935
ALL_USER_TYPES should be a tuple
2019-01-15 14:38:15 +00:00
4a4d2e17bc
Add /v2/invite federation API
2019-01-15 13:22:44 +00:00
bb63e7ca4f
Add groundwork for new versions of federation APIs
2019-01-15 11:14:34 +00:00
a35c66a00b
Remove duplicates in the user_ips table and add an index ( #4370 )
2019-01-12 06:21:50 +11:00
522dada206
Merge remote-tracking branch 'origin/master' into develop
2019-01-11 02:22:48 +11:00
6dc06c3775
version
2019-01-11 01:56:37 +11:00
e0910d0145
Merge branch rav/macaroon_key_fix_0.34 into rav/macaroon_key_fix_0.34.1
...
Fixes #4371
2019-01-10 14:12:50 +00:00
ba41aeed6a
Revert "Fix macaroon_secret_key fallback logic"
...
This is already fixed in 0.34.1, by 59f93bbefc522c55e
2019-01-10 14:09:26 +00:00
aa70d24125
Merge branch 'rav/macaroon_key_fix' into rav/macaroon_key_fix_0.34
2019-01-10 12:58:33 +00:00
efc522c55e
Fix macaroon_secret_key fallback logic
2019-01-10 12:57:27 +00:00
566947ff34
Skip macaroon check for access tokens in the db
2019-01-10 12:57:21 +00:00
353f2407b7
Fix fallback to signing key for macaroon-secret-key
2019-01-10 12:42:56 +00:00
d566e6b17a
Merge branch 'master' into develop
2019-01-09 15:00:46 +00:00
998f5225c1
0.34.1
2019-01-09 14:53:54 +00:00
55c3e853c5
Merge pull request #4368 from matrix-org/erikj/better_errors
...
Fixup docstrings for matrixfederationclient
2019-01-09 11:39:28 +00:00
484867d35d
Merge branch 'develop' into release-v0.34.1
2019-01-09 11:38:13 +00:00
7960c26fda
Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. ( #4369 )
2019-01-09 22:26:25 +11:00
34ea14139d
Fixup docstrings for matrixfederationclient
2019-01-09 09:25:59 +00:00
d422570e9b
Use RequestSendFailed when fail to parse content type headers
2019-01-08 14:22:18 +00:00
1371d5b798
Don't log stack traces for HTTP error responses
2019-01-08 12:28:30 +00:00
ba85786d71
0.34.1rc1
2019-01-08 11:15:34 +00:00
b970cb0e96
Refactor request sending to have better excpetions ( #4358 )
...
* Correctly retry and back off if we get a HTTPerror response
* Refactor request sending to have better excpetions
MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).
This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
1dcb086f33
Fix synapse.config.__main__ on python 3 ( #4356 )
2019-01-08 10:03:09 +11:00
bc1fa8cd01
Add GET account data routes ( #4303 )
...
As per https://github.com/matrix-org/matrix-doc/issues/1339
2019-01-07 10:27:54 +00:00
b7c0218812
Check jinja version for consent resource ( #4327 )
...
* Raise a ConfigError if an invalid resource is specified
* Require Jinja 2.9 for the consent resource
* changelog
2019-01-07 10:14:31 +00:00
a27e501b09
fix the check for whether `is_url` to match all the other ones in codebase ( #3405 )
...
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2019-01-06 21:43:25 +00:00
27128145e6
fix NPE in /messages by checking if all events were filtered out ( #4330 )
2019-01-02 23:37:39 +00:00
84b6fae1f5
Ensure synchrotrons can access is_support_user in the storage layer
2019-01-02 10:19:59 +00:00
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console ( #4334 )
2018-12-29 23:12:30 +11:00
0708f437cc
Log roomid along with Unknown room ( #4297 )
2018-12-24 10:49:42 +01:00
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms
...
Add v2 room version
2018-12-24 10:48:59 +01:00
f85676cc93
Return well_known in /login response ( #4319 )
...
... as per MSC1730.
2018-12-24 10:44:33 +01:00
9c2af7b2c5
Add a script to generate a clean config file ( #4315 )
2018-12-22 02:04:57 +11:00
f3561f8d86
Fix indentation in default config ( #4313 )
...
These settings are not supposed to be under 'listeners'.
2018-12-22 02:01:43 +11:00
ea6abf6724
Fix IP URL previews on Python 3 ( #4215 )
2018-12-22 01:56:13 +11:00
c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools ( #4298 )
2018-12-22 01:37:26 +11:00
0b26feb422
Merge branch 'master' into develop
2018-12-20 16:48:47 +00:00
6dd7271243
Prepare 0.34 release
2018-12-20 12:07:15 +00:00
d3c9c562c6
Debian packaging via dh_virtualenv
2018-12-20 11:35:23 +00:00
1a6d5bfa08
Debian packaging via dh_virtualenv ( #4285 )
2018-12-20 11:33:29 +00:00
1c0051114a
Add 'sandbox' to CSP for media repo ( #4284 )
...
* Add 'sandbox' to the CSP for media repo
* Changelog
2018-12-20 11:09:18 +00:00
81b513416e
Fixed line length
2018-12-19 14:12:33 -05:00
668e6625b0
Fix the variable names used for account_data
2018-12-19 06:53:02 -05:00
df89f8afb8
Add v2 room version
2018-12-18 18:10:37 +00:00
7e22cd90f5
ensure can report mau stats when hs.config.mau_stats_only is set ( #4305 )
...
* ensure can report mau stats when hs.config.mau_stats_only is set
2018-12-18 14:36:11 +00:00
d2f7c4e6b1
create support user ( #4141 )
...
Allow for the creation of a support user.
A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
128294d3e5
Improved welcome page ( #4294 )
...
* Improved static/index.html look, feel and content.
* Added accessibility tags to in-line SVG.
* Grammar.
2018-12-14 05:13:56 +11:00
c7144b105c
Settings Fix deleting e2e room keys on xenial ( #4295 )
2018-12-14 05:13:33 +11:00
e93a0ebf50
Settings Fix deleting e2e room keys on xenial ( #4295 )
2018-12-14 05:10:31 +11:00
cef8ae272a
Improved welcome page ( #4294 )
...
* Improved static/index.html look, feel and content.
* Added accessibility tags to in-line SVG.
* Grammar.
2018-12-14 02:42:51 +11:00
f208f608cb
Merge branch 'release-v0.34.0' into develop
2018-12-11 15:43:20 +00:00
eb835bf65b
0.34.0rc2
2018-12-11 15:35:47 +00:00
188945713e
Merge pull request #4290 from matrix-org/rav/remove_webclient
...
Stop installing Matrix Console by default
2018-12-11 16:24:15 +01:00
95c1f6500b
Merge pull request #4289 from matrix-org/rav/welcome_page
...
Add a welcome page to the static resources
2018-12-11 16:24:06 +01:00
f537432ef9
Add a welcome page to the static resources
...
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.
The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.
It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.
It's based on the work done by @krombel for #2601 .
2018-12-11 13:26:22 +00:00
df96177ca7
Stop installing Matrix Console by default
...
This is based on the work done by @krombel in #2601 .
2018-12-11 13:20:33 +00:00
89ac2a5bdb
Add 'sandbox' to CSP for media repo ( #4284 )
...
* Add 'sandbox' to the CSP for media repo
* Changelog
2018-12-11 04:05:02 +11:00
fee831c040
Move imports to one line
2018-12-10 13:52:33 +00:00
466c1f3e01
Use `send_cors`
2018-12-10 13:11:37 +00:00
91206e09f2
changelog & isort
2018-12-09 17:39:44 +00:00
dbf736ba66
Make /config more CORS-y
2018-12-09 13:27:22 +00:00
30da50a5b8
Initialise user displayname from SAML2 data ( #4272 )
...
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
c7401a697f
Implement SAML2 authentication ( #4267 )
...
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.
What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
2018-12-07 13:11:11 +01:00
c588b9b9e4
Factor SSO success handling out of CAS login ( #4264 )
...
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
ae19a7db8c
Prevent crash on pagination.
2018-12-06 13:32:05 +03:00
b0c24a66ec
Rip out half-implemented m.login.saml2 support ( #4265 )
...
* Rip out half-implemented m.login.saml2 support
This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.
* drop undocumented dependency on dateutil
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
9a3e24a13d
drop undocumented dependency on dateutil ( #4266 )
...
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 04:52:42 +11:00
e8d98466b0
Implement .well-known handling ( #4262 )
...
Sometimes it's useful for synapse to generate its own .well-known file.
2018-12-05 14:38:58 +01:00
e55983defe
Prepare 0.34.0rc1
2018-12-04 13:52:16 +00:00
1737753a62
Add an option to enable recording IPs for appservice users ( #3831 )
2018-12-04 12:44:41 +01:00
fd96dd75a3
Fix non-ASCII pushrules ( #4248 )
2018-12-04 12:44:02 +01:00
158ffb92f1
Add an option to disable search for homeservers which may not be interested in it ( #4230 )
...
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
b5ac0ffa0a
add more detail to logging regarding "More than one row matched" error ( #4234 )
2018-12-04 11:57:39 +01:00
ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english ( #4253 )
...
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252 . Fundamentally the right answer is to drop support for Python 2.
2018-12-04 11:55:52 +01:00
f144c0a210
Merge pull request #4244 from aaronraimist/drop-sent-txt
...
Drop sent_transactions
2018-12-04 11:41:28 +01:00
a077e710a3
Merge pull request #4250 from matrix-org/hawkowl/pusher-remove-py3
...
Fix removing pushers on python 3
2018-12-04 11:22:46 +01:00
52e87fbfbe
Run the AS senders as background processes ( #4189 )
...
This should fix some "Starting db connection from sentinel context" warnings,
and will mean we get metrics for these processes.
2018-12-04 10:53:49 +01:00
d3c61ef906
fix type error
2018-12-03 22:27:41 +11:00
c03324294d
Workaround for non-ascii event ids ( #4241 )
...
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.
Fixes #4226
2018-12-03 21:47:48 +11:00
704c5298f0
Drop sent_transactions
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:07:35 -06:00
7039ece8fb
Neilj/fix autojoin ( #4223 )
...
* Fix auto join failures for servers that require user consent
* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
f9b136a886
Neilj/fix mau initial reserved users ( #4211 )
...
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail
* towncrier
* white space
2018-11-28 20:33:41 +11:00
944d524f18
Support m.login.sso ( #4220 )
...
* Clean up the CSS for the fallback login form
I was finding this hard to work with, so simplify a bunch of things. Each
flow is now a form inside a div of class login_flow.
The login_flow class now has a fixed width, as that looks much better than each
flow having a differnt width.
* Support m.login.sso
MSC1721 renames m.login.cas to m.login.sso. This implements the change
(retaining support for m.login.cas for older clients).
* changelog
2018-11-27 18:51:52 +11:00
6c18cc4b50
Ignore __pycache__ directories in schema delta dir
...
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
2018-11-20 22:52:34 +00:00
78ba0e7ab8
Remove riot.im from the list of trusted Identity Servers in the default configuration ( #4207 )
2018-11-20 12:29:25 +01:00
416c671474
Merge pull request #4204 from matrix-org/rav/logcontext_leak_fixes
...
Fix some logcontext leaks
2018-11-20 12:19:19 +01:00
31425d82a3
Merge remote-tracking branch 'origin/master' into develop
2018-11-19 12:55:25 -06:00
d102e19e47
version
2018-11-19 12:42:49 -06:00
80cac86b2c
Fix fallback auth on Python 3 ( #4197 )
2018-11-19 12:27:33 -06:00
f5faf6bc14
Fix logcontext leak in EmailPusher
2018-11-19 17:07:01 +00:00
ab4526a153
Remove duplicate slashes in generated consent URLs
2018-11-15 20:41:53 -07:00
8b1affe7d5
Fix Content-Disposition in media repository ( #4176 )
2018-11-15 15:55:58 -06:00
835779f7fb
Add option to track MAU stats (but not limit people) ( #3830 )
2018-11-15 18:08:27 +00:00
df758e155d
Use <meta> tags to discover the per-page encoding of html previews ( #4183 )
2018-11-15 11:05:08 -06:00
bf648c37e7
release 0.33.9rc1
2018-11-14 11:45:52 +00:00
c1efcd7c6a
Add a test for the public T&Cs form
2018-11-14 10:46:27 +00:00
83a5f459aa
Fix an internal server error when viewing the public privacy policy
2018-11-14 10:21:07 +00:00
0869566ad3
Merge pull request #4113 from matrix-org/dbkr/e2e_backup_versions_are_numbers
...
Make e2e backup versions numeric in the DB
2018-11-14 07:55:48 +00:00
bca3b91c2d
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers
2018-11-09 18:35:02 +00:00
db5a1c059a
Merge pull request #4166 from matrix-org/erikj/drop_unknown_events
...
Drop incoming events from federation for unknown rooms
2018-11-09 17:59:34 +00:00
dc59ad5334
Remove hack to support rejoining rooms
2018-11-09 14:58:09 +00:00
d44dea0223
pep8
2018-11-09 14:38:31 +00:00
4f93abd62d
add docs
2018-11-09 13:25:38 +00:00
30dd27afff
Simplify to always drop events if server isn't in the room
2018-11-09 11:36:45 +00:00
3cecf5340d
Update synapse/federation/federation_server.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2018-11-09 11:28:25 +00:00
9bce065a53
Update synapse/federation/federation_server.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2018-11-09 11:28:22 +00:00
d3fa6194f7
Remove unnecessary str()
2018-11-09 11:11:31 +00:00
7b22421a7b
Merge pull request #4164 from matrix-org/erikj/fix_device_comparison
...
Fix noop checks when updating device keys
2018-11-08 14:37:20 +00:00
5ebed18692
Lets convert bytes to unicode instead
2018-11-08 12:33:13 +00:00
b1a22b24ab
Fix noop checks when updating device keys
...
Clients often reupload their device keys (for some reason) so its
important for the server to check for no-ops before sending out device
list update notifications.
The check is broken in python 3 due to the fact comparing bytes and
unicode always fails, and that we write bytes to the DB but get unicode
when we read.
2018-11-08 12:18:38 +00:00
9417986f77
Drop PDUs of unknown rooms
...
When we receive events over federation we will need to know the room
version to be able to correctly handle them, e.g. once we start changing
event formats. Currently, we attempt to handle events in unknown rooms.
2018-11-08 12:11:20 +00:00
0a1fc52971
fix parse_string docstring
2018-11-08 11:12:29 +00:00
2b075fb03a
Fix encoding error for consent form on python3
...
The form was rendering this as "b'01234....'".
-- richvdh
2018-11-08 11:05:39 +00:00
b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) ( #4157 )
2018-11-08 01:37:43 +11:00
0f5e51f726
Add config variables for enabling terms auth and the policy name ( #4142 )
...
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
f1087106cf
handle empty backups according to latest spec proposal ( #4123 )
...
fixes #4056
2018-11-05 17:59:29 -05:00
efdcbbe46b
Tests for user consent resource ( #4140 )
2018-11-06 05:53:44 +11:00
bc80b3f454
Add helpers for getting prev and auth events ( #4139 )
...
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
0467384d2f
Set the encoding to UTF8 in the default logconfig ( #4138 )
2018-11-03 02:28:07 +11:00
90d713b8c6
Merge pull request #4137 from matrix-org/erikj/clean_up_events
...
Clean up event accesses and tests
2018-11-02 14:12:49 +00:00
b86d05a279
Clean up event accesses and tests
...
This is in preparation to refactor FrozenEvent to support different
event formats for different room versions
2018-11-02 13:44:14 +00:00
cb7a6b2379
Fix typing being reset causing infinite syncs ( #4127 )
2018-11-03 00:19:23 +11:00
efb9343c8c
Merge pull request #4132 from matrix-org/rav/fix_device_list_locking
...
Fix locked upsert on device_lists_remote_cache
2018-11-02 10:50:53 +00:00
00f12e00f8
Merge pull request #4133 from matrix-org/travis/fix-terms-auth
...
Fix logic error that prevented guests from seeing the privacy policy
2018-11-02 10:50:43 +00:00
b199534518
Merge pull request #4135 from matrix-org/erikj/fix_state_res_none
...
Fix None exception in state res v2
2018-11-02 10:45:57 +00:00
350f654e7b
Add unique indexes to a couple of tables
...
The indexes on device_lists_remote_extremeties can be unique, and they
therefore should, to ensure that the db remains consistent.
2018-11-02 10:36:13 +00:00
50e328d1e7
Remove redundant database locks for device list updates
...
We can rely on the application-level per-user linearizer.
2018-11-02 10:36:13 +00:00
54aec35867
Fix None exception in state res v2
2018-11-02 10:29:19 +00:00
642505abc3
Fix logic error that prevented guests from seeing the privacy policy
2018-11-01 16:48:32 -06:00
c68aab1536
Merge pull request #4004 from matrix-org/travis/login-terms
...
Add m.login.terms to the registration flow
2018-11-01 11:03:38 -06:00
1b21e771d0
Merge pull request #4128 from matrix-org/erikj/state_res_v2_version
...
Add STATE_V2_TEST room version
2018-11-01 13:17:57 +00:00
b3dd6fa981
Add STATE_V2_TEST room version
2018-11-01 11:43:46 +00:00
073d400b84
Merge branch 'master' into develop
2018-11-01 21:32:12 +11:00
aa98e38896
version
2018-11-01 21:28:35 +11:00
a8d41c6aff
Include a version query string arg for the consent route
2018-10-31 13:19:28 -06:00
d1e7b9c44c
Merge branch 'develop' into travis/login-terms
2018-10-31 13:15:14 -06:00
1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
...
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
0f8591a5a8
Avoid else clause on exception for clarity
2018-10-31 15:43:57 +00:00
94c7fadc98
Attempt to move room aliases on room upgrades
2018-10-31 15:43:57 +00:00
9b827c40ca
Log some bits about event creation ( #4121 )
...
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
916efc8249
Remove fetching keys via the deprecated v1 kex method ( #4120 )
2018-10-31 23:14:39 +11:00
f79f454485
Remove deprecated v1 key exchange endpoint ( #4119 )
2018-10-31 22:29:02 +11:00
67c1924899
version bump
2018-10-31 06:27:05 +11:00
086e1a8f3e
Fix search 500ing ( #4122 )
2018-10-31 06:26:46 +11:00
3bade14ec0
Fix search 500ing ( #4122 )
2018-10-31 04:33:41 +11:00
2e223a8c22
Remove the unused /pull federation API ( #4118 )
2018-10-31 04:24:59 +11:00
0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups
...
Delete unreferenced state groups during purge
2018-10-30 16:58:22 +00:00
0dce9e1379
Write some tests for the email pusher ( #4095 )
2018-10-30 23:55:43 +11:00
e0934acdbb
Cast to int here too
2018-10-30 11:12:23 +00:00
12941f5f8b
Cast bacjup version to int when querying
2018-10-30 11:01:07 +00:00
2f0f911c52
Convert version back to a string
2018-10-30 10:35:18 +00:00
64fa557f80
Try & make it work on postgres
2018-10-30 09:51:04 +00:00
563f9b61b1
Make e2e backup versions numeric in the DB
...
We were doing max(version) which does not do what we wanted
on a column of type TEXT.
2018-10-29 21:01:22 +00:00
88e5ffe6fe
Deduplicate device updates sent over replication
...
We currently send several kHz of device list updates over replication
occisonally, which often causes the replications streams to lag and then
get dropped.
A lot of those updates will actually be duplicates, since we don't send
e.g. device_ids across replication, so let's deduplicate it when we pull
them out of the database.
2018-10-29 17:34:34 +00:00
a163b748a5
Don't truncate command name in metrics
2018-10-29 17:34:21 +00:00
ad88460e0d
Move _find_unreferenced_groups
2018-10-29 14:24:19 +00:00
664b192a3b
Fix set operations thinko
2018-10-29 14:21:43 +00:00
f4f223aa44
Don't make temporary list
2018-10-29 14:01:49 +00:00
b2399f6281
Make SQL a bit cleaner
2018-10-29 14:01:11 +00:00
4cd1c9f2ff
Delete the disused & unspecced identicon functionality ( #4106 )
2018-10-29 23:57:24 +11:00
7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
...
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls
...
Better handling of odd PLs during room upgrades
2018-10-29 12:48:51 +00:00
bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades
...
Room version upgrade support
2018-10-29 12:47:20 +00:00
c4b3698a80
Make the replication logger quieter ( #4108 )
2018-10-29 22:59:44 +11:00
2b791865c4
version bump
2018-10-29 21:52:52 +11:00
db24d7f15e
Better handling of odd PLs during room upgrades
...
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
5caf79b312
Remember to copy the avatar on room upgrades
2018-10-26 23:56:40 +01:00
54bbe71867
optimise state copying
2018-10-26 22:51:34 +01:00
193cadc988
Address review comments
...
Improve comments, get old room state from the context we already have
2018-10-26 17:10:30 +01:00
03e634dad4
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups
2018-10-26 16:22:45 +01:00
77d70a7646
Port register_new_matrix_user to Python 3 and add tests ( #4085 )
2018-10-26 22:05:22 +11:00
e6babc27d5
restrict PLs in old room
2018-10-25 19:18:25 +01:00
3a263bf3ae
copy state
2018-10-25 19:18:25 +01:00
1b9f253e20
preserve PLs
2018-10-25 19:10:24 +01:00
4cda300058
preserve room visibility
2018-10-25 19:10:24 +01:00
0f7d1c9906
Basic initial support for room upgrades
...
Currently just creates a new, empty, room, and sends a tombstone in the old
room.
2018-10-25 19:10:24 +01:00
e1948175ee
Allow power_level_content_override=None for _send_events_for_new_room
2018-10-25 19:10:24 +01:00
7f7b2cd3de
Make room_member_handler a member of RoomCreationHandler
...
... to save passing it into `_send_events_for_new_room`
2018-10-25 19:10:18 +01:00
871c4abfec
Factor _generate_room_id out of create_room
...
we're going to need this for room upgrades.
2018-10-25 18:23:09 +01:00
cb53ce9d64
Refactor state group lookup to reduce DB hits ( #4011 )
...
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list
...
Add config option to control alias creation
2018-10-25 17:04:59 +01:00
95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init
...
fix race condiftion in calling initialise_reserved_users
2018-10-25 16:33:40 +01:00
fcbd488e9a
add new line
2018-10-25 16:13:43 +01:00
b94a43d5b5
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list
2018-10-25 15:25:31 +01:00
e5481b22aa
Use allow/deny
2018-10-25 15:25:21 +01:00
f8fe98812b
improve comments
2018-10-25 14:58:59 +01:00
f7f487e14c
Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin
2018-10-25 14:40:06 +01:00
54def42c19
Merge branch 'develop' into travis/login-terms
2018-10-24 13:22:59 -06:00
ea69a84bbb
fix style inconsistencies
2018-10-24 17:18:08 +01:00
663d9db8e7
commit transaction before closing
2018-10-24 17:17:30 +01:00
07126e43a4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init
2018-10-24 16:25:39 +01:00
9ec2186586
isort
2018-10-24 16:09:21 +01:00
9532caf6ef
remove trailing whiter space
2018-10-24 16:08:25 +01:00
94a49e0636
fix tuple
...
Co-Authored-By: neilisfragile <neil@matrix.org>
2018-10-24 14:39:23 +01:00
3ad359e5be
Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation
2018-10-24 11:23:49 +01:00
7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8
...
Fix a number of flake8 errors
2018-10-24 11:23:35 +01:00
3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase
...
Add v2 state resolution algorithm
2018-10-24 11:12:12 +01:00
7e07d25ed6
Allow backslashes in event field filters
...
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.
The intention here is to allow single-backslashes, but disallow
double-backslashes.
2018-10-24 11:11:24 +01:00
ef771cc4c2
Fix a number of flake8 errors
...
Broadly three things here:
* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
them
* use `r""` for strings which include backslashes
Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
47a9ba435d
Use match rather than search
2018-10-24 09:54:57 +01:00
e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts
...
Clean up the way logcontexts and threads work in the pushers
2018-10-24 09:53:57 +01:00
dacbeb2e03
Comment
2018-10-24 09:47:49 +01:00
810715f79a
Rename resolve_events_with_factory
2018-10-24 09:44:22 +01:00
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation
...
Clean up room alias creation
2018-10-24 09:40:27 +01:00
c573794b22
Fix start_pushers vs _start_pushers confusion
2018-10-24 09:24:55 +01:00
e564306e31
sanity-check the is_processing flag
...
... and rename it, for even more sanity
2018-10-24 09:23:33 +01:00
a67d8ace9b
remove errant exception and style
2018-10-23 17:44:39 +01:00
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
...
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
3e704822be
Comments help
2018-10-23 10:25:31 -06:00
329d18b39c
remove white space
2018-10-23 15:27:20 +01:00
6105c6101f
fix race condiftion in calling initialise_reserved_users
2018-10-23 15:24:58 +01:00
b3f6dddad2
Give some more things logcontexts ( #4077 )
2018-10-23 14:29:17 +01:00
5c445114d3
Correctly account for cpu usage by background threads ( #4074 )
...
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.
While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.
Fixes #4064
2018-10-23 13:12:32 +01:00
026cd91ac8
Run PusherPool.start as a background process
...
We don't do anything with the result, so this is needed to give this code a
logcontext.
2018-10-22 16:12:11 +01:00
f749607c91
Make on_started synchronous too
...
This brings it into line with on_new_notifications and on_new_receipts. It
requires a little bit of hoop-jumping in EmailPusher to load the throttle
params before the first loop.
2018-10-22 16:12:11 +01:00
e7a16c6210
Remove redundant run_as_background_process() from pusherpool
...
`on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher`
now always return synchronously, so we can remove the `defer.gatherResults` on
their results, and the `run_as_background_process` wrappers can be removed too
because the PusherPool methods will now complete quickly enough.
2018-10-22 16:12:11 +01:00
c7273c11bc
Give pushers their own background logcontext
...
Each pusher has its own loop which runs for as long as it has work to do. This
should run in its own background thread with its own logcontext, as other
similar loops elsewhere in the system do - which means that CPU usage is
consistently attributed to that loop, rather than to whatever request happened
to start the loop.
2018-10-22 16:12:11 +01:00
5110f4e425
move get_all_pushers call down
...
simplifies the interface to _start_pushers
2018-10-22 16:12:11 +01:00
04277d0ed8
Factor PusherPool._start_pusher out of _start_pushers
...
... and use it from start_pusher_by_id. This mostly simplifies
start_pusher_by_id.
2018-10-22 16:12:11 +01:00
3e8b02c939
Rename _refresh_pusher
...
This is public (or at least, called from outside the class), so ought to have a
better name.
2018-10-22 16:12:11 +01:00
7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts
...
Run MAU queries as background processes
2018-10-22 14:46:59 +01:00
911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil
...
Make psutil an explicit dependency
2018-10-22 12:33:21 +01:00
a6f421e812
Run MAU queries as background processes
...
Fixes #3820
2018-10-20 02:14:35 +01:00
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) ( #4068 )
2018-10-20 11:16:55 +11:00
e5b52d0f94
Make psutil an explicit dependency
...
As of #4027 , we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992 .
Fixes #4062 .
2018-10-19 21:51:15 +01:00
f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065
...
Calculate absolute path for email templates
2018-10-19 16:33:54 +01:00
67f7b9cb50
pep8
2018-10-19 16:06:59 +01:00
056f099126
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups
2018-10-19 15:48:59 +01:00
47a9da28ca
Batch process handling state groups
2018-10-19 15:48:15 +01:00
cc325c7069
Calculate absolute path for email templates
2018-10-19 14:01:59 +01:00
e404ba9aac
Fix manhole on py3 (pt 2) ( #4067 )
2018-10-19 22:26:00 +11:00
b69216f768
Make the metrics less racy ( #4061 )
2018-10-19 21:45:45 +11:00
9fafdfa97d
Anchor returned regex to start and end of string
2018-10-19 10:22:45 +01:00
084046456e
Add config option to control alias creation
2018-10-19 10:22:45 +01:00
74e7617083
Clean up room alias creation
2018-10-19 10:11:56 +01:00
a36b0ec195
make a bytestring
2018-10-19 09:24:00 +11:00
49a044aa5f
Merge branch 'develop' into travis/login-terms
2018-10-18 09:57:58 -06:00
c632bc8654
Merge branch 'master' into develop
2018-10-18 15:07:03 +01:00
926da4dda8
0.33.7
2018-10-18 14:57:32 +01:00
c7d0f34a3c
v0.33.7rc2
2018-10-17 17:40:19 +01:00
0fd2321629
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-17 17:35:26 +01:00
c8f2c19991
Put the warning blob at the top of the file
2018-10-17 16:56:22 +01:00
1519572961
Ship the email templates as package_data
...
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.
In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
2018-10-17 16:46:02 +01:00
f6a0a02a62
Fix bug where we raised StopIteration in a generator
...
This made python 3.7 unhappy
2018-10-17 16:10:52 +01:00
d6a7797dd1
Fix roomlist since tokens on Python 3 ( #4046 )
...
Thanks @Half-Shot !!!
2018-10-17 23:04:55 +11:00
c6584f4b5f
clean up config error logic and imports
2018-10-17 11:36:41 +01:00
fc0f13dd03
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
4a28d3d36f
Update event_auth table for rejected events
2018-10-16 16:28:42 +01:00
15133477ee
Fix up use of resolve_events_with_factory
2018-10-16 16:28:42 +01:00
6bd856caa2
User event.sender rather than alias event.user_id
2018-10-16 16:16:13 +01:00
e238013c44
Add v2 state res algorithm.
...
We hook this up to the vdh test room version.
2018-10-16 16:16:13 +01:00
b8a5b0097c
Various cleanups in the federation client code ( #4031 )
...
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
a8ed93a4b5
pep8
2018-10-15 16:10:29 -06:00
442734ff9e
Ensure the terms params are actually provided
2018-10-15 14:56:13 -06:00
762a0982aa
Python is hard
2018-10-15 14:46:09 -06:00
f293d124b6
Merge branch 'develop' into travis/login-terms
2018-10-15 14:44:32 -06:00
f726f2dc6c
version bump
2018-10-15 22:21:45 +11:00
03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
...
E2E backups
2018-10-15 10:19:43 +01:00
06bc8d2fe5
synapse/app: frontend_proxy.py: actually make workers work on py3
2018-10-14 20:08:39 +03:00
fb216a22db
synapse/visibility.py: fix SyntaxError on py3.7
2018-10-14 20:08:17 +03:00
1ccafb0c5e
no need to join room if creator
2018-10-13 21:14:21 +01:00
dd99db846d
Update login terms structure for the proposed language support
2018-10-12 18:03:27 -06:00
5119818e9d
Rely on the lack of ?u to represent public access
...
also general cleanup
2018-10-12 18:03:17 -06:00
7ede650956
Merge branch 'develop' into travis/login-terms
2018-10-12 16:24:07 -06:00
164f8e4843
isort
2018-10-12 15:11:59 -06:00
7bb651de6a
More sane handling of group errors and pep8
2018-10-12 14:53:30 -06:00
e3586f7c06
Merge branch 'develop' into travis/fix-federated-group-requests
2018-10-12 14:49:58 -06:00
a2bfb778c8
improve auto room join logic, comments and tests
2018-10-12 18:17:36 +01:00
a45f2c3a00
missed one
2018-10-12 14:33:55 +01:00
381d2cfdf0
Make workers work on Py3 ( #4027 )
2018-10-13 00:14:08 +11:00
67a1e315cc
Fix up comments
2018-10-12 13:49:48 +01:00
8c0ff0287a
Linting soothes the savage PEP8 monster
2018-10-12 13:47:43 +01:00
306361b31b
Misc PR feedback bits
2018-10-12 11:48:56 +01:00
bddfad253a
Don't mangle exceptions
2018-10-12 11:48:02 +01:00
86ef9760a7
Split /room_keys/version into 2 servlets
2018-10-12 11:35:08 +01:00
83e72bb2f0
PR feedback pt. 1
2018-10-12 11:26:18 +01:00
8ddd0f273c
Comments on get_all_new_events_stream
...
just some docstrings to clarify the behaviour here
2018-10-12 09:55:41 +01:00
7e561b5c1a
Add description to counter metric
2018-10-10 11:41:15 +01:00
3cbe8331e6
Track number of non-empty sync responses instead
2018-10-10 11:23:17 +01:00
395276b405
Append _total to metric and fix up spelling
2018-10-10 09:24:39 +01:00
b8d9e108be
Fix mergefail
2018-10-09 18:04:21 +01:00
bdc27d6716
Add metric to count lazy member sync requests
2018-10-09 14:17:52 +01:00
d3464ce708
isort
2018-10-09 10:33:59 +01:00
9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions
...
Log looping call exceptions
2018-10-09 10:12:10 +01:00
dc045ef202
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
2018-10-09 10:05:02 +01:00
f4a4dbcad1
Apparently this blank line is Very Important
2018-10-09 09:47:04 +01:00
0c905ee015
be python3 compatible
2018-10-09 09:39:13 +01:00
6982320572
Remove unnecessary extra function call layer
2018-10-08 14:06:19 +01:00
495975e231
Optimisation for filter_events_for_server
...
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.
Also, iteritems > items.
2018-10-08 13:46:52 +01:00
8a1817f0d2
Use errback pattern and catch async failures
2018-10-08 13:29:47 +01:00
497444f1fd
Don't reuse backup versions
...
Since we don't actually delete the keys, just mark the versions
as deleted in the db rather than actually deleting them, then we
won't reuse versions.
Fixes https://github.com/vector-im/riot-web/issues/7448
2018-10-05 15:08:36 +01:00
f7199e8734
Log looping call exceptions
...
If a looping call function errors, then it kills the loop entirely.
Currently it throws away the exception logs, so we should make it
actually log them.
Fixes #3929
2018-10-05 11:24:12 +01:00
ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin
2018-10-04 17:26:59 +01:00
2dadc092b8
move logic into register, fix room alias localpart bug, tests
2018-10-04 17:00:27 +01:00
c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers
...
Fix bug in forward_extremity update logic
2018-10-04 16:27:05 +01:00
4917ff5523
Add state_group index to event_to_state_groups
...
This is needed to efficiently check for unreferenced state groups during
purge.
2018-10-04 16:03:08 +01:00
17d585753f
Delete unreferened state groups during purge
2018-10-04 16:03:06 +01:00
d86794325f
Merge branch 'master' into develop
2018-10-04 22:41:52 +10:00
dd59dfc51f
full version
2018-10-04 22:37:55 +10:00
537d0b7b36
Use a flag rather than a new route for the public policy
...
This also means that the template now has optional parameters, which will need to be documented somehow.
2018-10-03 17:50:11 -06:00
f9d34a763c
Auto-consent to the privacy policy if the user registered with terms
2018-10-03 17:39:45 -06:00
dfcad5fad5
Make the terms flow requried
2018-10-03 17:39:00 -06:00
3099d96dba
Flesh out the fallback auth for terms
2018-10-03 17:39:00 -06:00
149c4f1765
Supply params for terms auth stage
...
As per https://github.com/matrix-org/matrix-doc/pull/1692
2018-10-03 15:57:42 -06:00
fd99787162
Incorporate Dave's work for GDPR login flows
...
As per https://github.com/vector-im/riot-web/issues/7168#issuecomment-419996117
2018-10-03 15:57:42 -06:00
a59d899668
Pin to prometheus_client<0.4 to avoid renaming all of our metrics
2018-10-03 17:20:15 +01:00
8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion
...
Fix handling of rejected threepid invites
2018-10-03 14:47:41 +01:00
81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration
...
Fix exception in background metrics collection
2018-10-03 14:14:38 +01:00
52e6e815be
Sanitise error messages when user doesn't have permission to invite
2018-10-03 14:13:07 +01:00
69e857853f
Fix handling of rejected threepid invites
2018-10-03 11:57:30 +01:00
495a9d06bb
Fix exception handling in fetching remote profiles
2018-10-03 11:34:30 +01:00
7c570bff74
Fix exception in background metrics collection
...
We attempted to iterate through a list on a separate thread without
doing the necessary copying.
2018-10-03 11:28:01 +01:00
9693625e55
actually exclude outliers
2018-10-03 10:19:41 +01:00
3e39783d5d
remove debugging
2018-10-02 23:44:14 +01:00
ae61ade891
Fix bug in forward_extremity update logic
...
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
2018-10-02 23:36:08 +01:00
da8f82008d
version
2018-10-03 02:15:48 +10:00
7258d081a5
Fix bug when invalidating destination retry timings
2018-10-02 15:47:57 +01:00
2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error
...
Fix error handling for missing auth_event
2018-10-02 13:57:54 +01:00
7232917f12
Disable frozen dicts by default ( #3987 )
2018-10-02 22:53:47 +10:00
334e075dd8
Fix error when logging incomplete requests
...
If a connection is lost before a request is read from Request, Twisted
sets `method` (and `uri`) attributes to dummy values. These dummy values
have incorrect types (i.e. they're not bytes), and so things like
`__repr__` would raise an exception.
To fix this we had a helper method to return the method with a
consistent type.
2018-10-02 12:06:22 +01:00
8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces
...
Avoid reraise, to improve stacktraces
2018-10-02 10:47:38 +01:00
bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state
...
Fix lazy loaded sync with rejected state events
2018-10-02 11:38:35 +02:00
8174c6725b
Avoid reraise, to improve stacktraces
2018-10-01 18:50:34 +01:00
5908a8f6f5
Merge branch 'develop' into rav/fix_missing_create_event_error
2018-10-01 17:16:20 +01:00
b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors
...
Fix exceptions when handling incoming transactions
2018-10-01 15:54:24 +01:00
6e05fd032c
Fix userconsent on Python 3 ( #3938 )
2018-10-02 00:11:58 +10:00
82f922b4af
Fix lazy loaded sync with rejected state events
...
In particular, we assume that the name and canonical alias events in
the state have not been rejected. In practice this may not be the case
(though we should probably think about fixing that) so lets ensure that
we gracefully handle that case, rather than 404'ing the sync request
like we do now.
2018-10-01 14:19:40 +01:00
8f5c23d0cd
Merge pull request #3933 from matrix-org/erikj/destination_retry_cache
...
Add a five minute cache to get_destination_retry_timings
2018-10-01 14:20:30 +02:00
4f3e3ac192
Correctly match 'dict.pop' api
2018-10-01 12:25:27 +01:00
8ea887856c
Don't update eviction metrics on explicit removal
2018-10-01 12:00:58 +01:00
23b6a0537f
emit room aliases event
2018-09-29 02:19:37 +01:00
5b68f29f48
fix thinkos
2018-09-29 02:14:40 +01:00
8f646f2d04
fix UTs
2018-09-28 15:37:28 +01:00
3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks
...
remove spurious federation checks on localhost
2018-09-28 13:35:47 +01:00
965154d60a
Fix complete fail to do the right thing
2018-09-28 12:45:54 +01:00
19475cf337
Remove redundant call to start_get_pdu_cache
...
I think this got forgotten in #3932 . We were getting away with it because it
was the last call in this function.
2018-09-28 12:01:23 +01:00
9c8cec5dab
Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache
2018-09-28 10:51:09 +01:00
07340cdaca
untested stab at autocreating autojoin rooms
2018-09-28 01:42:53 +01:00
f094f715cf
Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors
2018-09-27 15:18:21 +01:00
36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing
...
Improve performance of getting typing updates for replication
2018-09-27 15:14:26 +01:00
e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups
...
Clarifications in FederationHandler
2018-09-27 15:06:59 +01:00
6de3884e5e
Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_process
...
Run notify_app_services as a bg process
2018-09-27 23:40:30 +10:00
a512e637ac
Merge pull request #3970 from schnuffle/develop-py3
...
Replaced all occurences of e.message with str(e)
2018-09-27 23:39:45 +10:00
b3064532d0
Run our oldest supported configuration in CI ( #3952 )
2018-09-27 23:21:54 +10:00
dc5db01ff2
Replaced all occurences of e.message with str(e)
...
Signed-off-by: Schnuffle <schnuffle@github.com>
2018-09-27 13:38:50 +02:00
51d33d5178
Merge pull request #3961 from matrix-org/neilj/lock_mau_upserts
...
fix #3854 MAU transaction errors
2018-09-27 12:31:37 +01:00
333bee27f5
Include event when resolving state for missing prevs
...
If we have a forward extremity for a room as `E`, and you receive `A`, `B`,
s.t. `A -> B -> E`, and `B` also points to an unknown event `X`, then we need
to do state res between `X` and `E`.
When that happens, we need to make sure we include `X` in the state that goes
into the state res alg.
Fixes #3934 .
2018-09-27 11:37:39 +01:00
bd61c82bdf
Include state from remote servers in pdu handling
...
If we've fetched state events from remote servers in order to resolve the state
for a new event, we need to actually pass those events into
resolve_events_with_factory (so that it can do the state res) and then persist
the ones we need - otherwise other bits of the codebase get confused about why
we have state groups pointing to non-existent events.
2018-09-27 11:37:39 +01:00
a215b698c4
Fix "unhashable type: 'list'" exception in federation handling
...
get_state_groups returns a map from state_group_id to a list of FrozenEvents,
so was very much the wrong thing to be putting as one of the entries in the
list passed to resolve_events_with_factory (which expects maps from
(event_type, state_key) to event id).
We actually want get_state_groups_ids().values() rather than
get_state_groups().
This fixes the main problem in #3923 , but there are other problems with this
bit of code which get discovered once you do so.
2018-09-27 11:37:39 +01:00
28223841e0
more comments
2018-09-27 11:31:51 +01:00
e3c159863d
Clarifications in FederationHandler
...
* add some comments on things that look a bit bogus
* rename this `state` variable to avoid confusion with the `state` used
elsewhere in this function. (There was no actual conflict, but it was
a confusing bit of spaghetti.)
2018-09-27 11:31:51 +01:00
92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2
...
Logging improvements
2018-09-27 11:27:46 +01:00
4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions ( #3959 )
...
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.
This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
2018-09-27 11:25:34 +01:00
ae6ad4cf41
docstrings and unittests for storage.state ( #3958 )
...
I spent ages trying to figure out how I was going mad...
2018-09-27 11:22:25 +01:00
82fa31799c
Remove debugging statement
2018-09-26 14:01:02 -06:00
b4c3bc1734
Handle HttpResponseException more safely for federated groups
2018-09-26 13:48:04 -06:00
219606a6ed
Fix exception documentation in matrixfederationclient.py
2018-09-26 13:26:27 -06:00
e70b4ce069
Logging improvements
...
Some logging tweaks to help with debugging incoming federation transactions
2018-09-26 17:36:14 +01:00
a87d419a85
Run notify_app_services as a bg process
...
This ensures that its resource usage metrics get recorded somewhere rather than
getting lost.
(It also fixes an error when called from a nested logging context which
completes before the bg process)
2018-09-26 17:00:40 +01:00
9453c65948
remove spurious federation checks on localhost
...
There's really no point in checking for destinations called "localhost" because
there is nothing stopping people creating other DNS entries which point to
127.0.0.1. The right fix for this is
https://github.com/matrix-org/synapse/issues/3953 .
Blocking localhost, on the other hand, means that you get a surprise when
trying to connect a test server on localhost to an existing server (with a
'normal' server_name).
2018-09-26 16:53:52 +01:00
607eec0456
fix docstring for FederationClient.get_state_for_room
...
trivial fixes for docstring
2018-09-26 16:52:24 +01:00
28781b65e7
fix #3854
2018-09-26 16:16:41 +01:00
8afddf7afe
Fix error handling for missing auth_event
...
When we were authorizing an event, if there was no `m.room.create` in its
auth_events, we would raise a SynapseError with a cryptic message, which then
meant that we would bail out of processing any incoming events, rather than
storing a rejection for the faulty event and moving on.
We should treat the absent event the same as any other auth failure, by
raising an AuthError, so that the event is marked as rejected.
2018-09-26 14:40:16 +01:00
4e8276a34a
Merge pull request #3956 from matrix-org/rav/fix_expiring_cache_len
...
Fix ExpiringCache.__len__ to be accurate
2018-09-26 13:10:13 +01:00
5b4028fa78
Merge branch 'rav/fix_expiring_cache_len' into erikj/destination_retry_cache
2018-09-26 12:55:53 +01:00
7ee94fc1ba
Log which cache is throwing exceptions
2018-09-26 12:43:08 +01:00
66a1d57adb
Merge pull request #3948 from matrix-org/rav/no_symlink_synctl
...
Move synctl into top dir to avoid a symlink
2018-09-26 21:41:58 +10:00
c2185f14d7
Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pdu
...
Comments and interface cleanup for on_receive_pdu
2018-09-26 21:41:26 +10:00
3baf6e1667
Fix ExpiringCache.__len__ to be accurate
...
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.
(This appears to have been a longstanding bug, but one which has been made more
of a problem by #3932 and #3933 ).
(This was originally done by Erik as part of #3933 . I'm cherry-picking it
because really it's a fix in its own right)
2018-09-26 12:32:29 +01:00
a1cd37390f
Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache
2018-09-25 12:03:54 +01:00
4c3e7eeec5
Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches
...
Fix some instances of ExpiringCache not expiring cache items
2018-09-25 12:02:57 +01:00
6cf261930a
added "media-src: 'self'" to CSP for resources ( #3578 )
...
Synapse doesn’t allow for media resources to be played directly from
Chrome. It is a problem for users on other networks (e.g. IRC)
communicating with Matrix users through a gateway. The gateway sends
them the raw URL for the resource when a Matrix user uploads a video
and the video cannot be played directly in Chrome using that URL.
Chrome argues it is not authorized to play the video because of the
Content Security Policy. Chrome checks for the "media-src" policy which
is missing, and defauts to the "default-src" policy which is "none".
As Synapse already sends "object-src: 'self'" I thought it wouldn’t be
a problem to add "media-src: 'self'" to the CSP to fix this problem.
2018-09-25 11:55:02 +01:00
94f7befc31
Merge pull request #3925 from matrix-org/erikj/fix_producers_unregistered
...
Fix spurious exceptions when client closes conncetion
2018-09-25 11:52:06 +01:00
c53336986d
Move synctl into top dir to avoid a symlink
...
symlinks apparently break setuptools on python3 and alpine
(https://bugs.python.org/issue31940 ), so let's stop using a symlink and just
use the file directly.
2018-09-25 11:19:27 +01:00
a9d84f4e44
We require attrs 16.0.0
...
Ref: https://github.com/matrix-org/synapse/issues/3945
2018-09-25 10:43:39 +01:00
787d22ed6c
Only lazy load self-members on initial sync
...
Given we have disabled lazy loading for incr syncs in #3840 , we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
2018-09-25 00:49:26 +01:00
fbe5ba25f6
Merge branch 'master' into develop
2018-09-25 03:10:01 +10:00
6b6cb32297
bump version
2018-09-25 02:54:34 +10:00
04eed80a73
Merge branch 'master' into develop
2018-09-24 23:42:25 +10:00
e302f40e20
update version
2018-09-24 23:40:05 +10:00
19dc676d1a
Fix ExpiringCache.__len__ to be accurate
...
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.
2018-09-21 16:25:42 +01:00
fdd1a62e8d
Add a five minute cache to get_destination_retry_timings
...
Hopefully helps with #3931
2018-09-21 14:56:12 +01:00
79eded1ae4
Make ExpiringCache slightly more performant
2018-09-21 14:52:21 +01:00
8601c24287
Fix some instances of ExpiringCache not expiring cache items
...
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
2018-09-21 14:19:46 +01:00
ad53a5497d
Merge pull request #3927 from matrix-org/erikj/handle_background_errors
...
Handle exceptions thrown by background tasks
2018-09-21 09:26:30 +01:00
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients ( #3916 )
...
Should fix https://github.com/vector-im/riot-web/issues/7209
2018-09-20 21:21:54 +01:00
94ae1dea3c
Add missing logger
2018-09-20 17:05:34 +01:00
9ea408441f
Handle exceptions thrown by background tasks
...
Fixes #3921
2018-09-20 16:15:21 +01:00
b28a7ed503
Fix spurious exceptions when client closes conncetion
...
If a HTTP handler throws an exception while processing a request we
automatically write a JSON error response. If the handler had already
started writing a response twisted throws an exception.
We should check for this case and simple abort the connection if there
was an error after the response had started being written.
2018-09-20 13:44:20 +01:00
23b53b4ef8
Merge pull request #3868 from matrix-org/neilj/fix_room_invite_mail_links
...
Neilj/fix room invite mail links
2018-09-20 13:32:38 +01:00
703de4ec13
Comments and interface cleanup for on_receive_pdu
...
Add some informative comments about what's going on here.
Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart
from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of
`get_missing` and use `sent_to_us_directly` consistently.
2018-09-20 13:06:55 +01:00
1f3f5fcf52
Fix client IPs being broken on Python 3 ( #3908 )
2018-09-20 20:14:34 +10:00
3fd68d533b
Merge pull request #3914 from matrix-org/erikj/remove_retry_cache
...
Remove get_destination_retry_timings cache
2018-09-20 10:54:49 +01:00
aeca5a5ed5
Add a regression test for logging on failed connections ( #3912 )
2018-09-20 16:28:18 +10:00
642199570c
Improve the logging when handling a federation transaction ( #3904 )
...
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:
- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
them into other, more cryptic, exceptions.
2018-09-19 17:28:18 +01:00
ce846bb620
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typing
2018-09-19 15:08:36 +01:00
bbab6ebfd9
Fix up changelog and remove spurious comment
2018-09-19 14:45:14 +01:00
392a54128c
pep8
2018-09-19 14:37:49 +01:00
b9158ac2bf
Remove get_destination_retry_timings cache
...
Currently we rely on the master to invalidate this cache promptly.
However, after having moved most federation endpoints off of master this
no longer happens, causing outbound fedeariont to get blackholed.
Fixes #3798
2018-09-19 14:22:57 +01:00
80d2d50f47
Fixup
2018-09-19 11:19:47 +01:00
9407bcf37a
Replace custom DeferredTimeoutError with defer.TimeoutError
2018-09-19 11:07:29 +01:00
6c48aa0256
Run canceller first to allow it to generate correct error
2018-09-19 11:07:27 +01:00
a334e1cace
Update to use new timeout function everywhere.
...
The existing deferred timeout helper function (and the one into twisted)
suffer from a bug when a deferred's canceller throws an exception, #3842 .
The new helper function doesn't suffer from this problem.
2018-09-19 10:39:40 +01:00
47c02e6332
Merge pull request #3909 from turt2live/travis/fix-logging-1
...
Fix matrixfederationclient.py logging: Destination is a string
2018-09-19 18:14:47 +10:00
3d6b24fb1b
Merge pull request #3907 from matrix-org/rav/set_sni_to_server_name
...
Set SNI to the server_name, not whatever was in the SRV record
2018-09-19 17:59:33 +10:00
f773ecbd61
Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeout
...
Bump timeout on get_missing_events request
2018-09-19 17:57:48 +10:00
35aec19f0a
Destination is a string
2018-09-18 15:29:30 -06:00
38ead946a9
Merge remote-tracking branch 'origin/develop' into neilj/fix_room_invite_mail_links
2018-09-18 19:02:45 +01:00
a219ce8726
Use directory server for room joins ( #3899 )
...
When we do a join, always try the server we used for the alias lookup first.
Fixes #2418
2018-09-18 18:27:37 +01:00
31c15dcb80
Refactor matrixfederationclient to fix logging ( #3906 )
...
We want to wait until we have read the response body before we log the request
as complete, otherwise a confusing thing happens where the request appears to
have completed, but we later fail it.
To do this, we factor the salient details of a request out to a separate
object, which can then keep track of the txn_id, so that it can be logged.
2018-09-18 18:17:15 +01:00
c600886d47
Merge pull request #3894 from matrix-org/hs/phone_home_py_version
...
Add python_version phone home stat
2018-09-19 02:40:04 +10:00
b3097396e7
Set SNI to the server_name, not whatever was in the SRV record
...
Fixes #3843
2018-09-18 17:01:12 +01:00
550007cb0e
Bump timeout on get_missing_events request
2018-09-18 15:02:51 +01:00
286d6930b7
Merge pull request #3879 from matrix-org/matthew/fix-autojoin
...
don't ratelimit autojoins
2018-09-18 13:07:01 +01:00
1e09a1d48a
Merge pull request #3889 from matrix-org/rav/404_on_remove_unknown_alias
...
Return a 404 when deleting unknown room alias
2018-09-18 12:59:30 +01:00
5baa087312
typo
2018-09-17 17:37:56 +01:00
b58714789f
make pip happy?
2018-09-17 17:35:54 +01:00
ac80cb08fe
Fix more b'abcd' noise in metrics
2018-09-17 17:16:50 +01:00
9a1cceeca9
Use a string for versions
2018-09-17 17:09:06 +01:00
f75b9961c6
Reinstate missing null check
2018-09-17 16:52:02 +01:00
2b39494cd5
Add python_version phone home stat
2018-09-17 16:35:18 +01:00
f00a9d2636
Fix some b'abcd' noise in logs and metrics
...
Python 3 compatibility: make sure that we decode some byte sequences before we
use them to create log lines and metrics labels.
2018-09-17 16:15:42 +01:00
fe88907d04
version
2018-09-17 22:33:22 +10:00
85a43f4167
Return a 404 when deleting unknown room alias
...
As per https://github.com/matrix-org/matrix-doc/issues/1675
Fixes https://github.com/matrix-org/synapse/issues/2782
2018-09-17 13:19:00 +01:00
d42d79e3c3
don't ratelimit autojoins
2018-09-15 22:27:41 +01:00
24efb2a70d
Fix timeout function
...
Turns out deferred.cancel sometimes throws, so we do that last to ensure
that we always do resolve the new deferred.
2018-09-15 11:38:39 +01:00
fcfe7a850d
Add an awful secondary timeout to fix wedged requests
...
This is an attempt to mitigate #3842 by adding yet-another-timeout
2018-09-14 19:23:07 +01:00
024be6cf18
don't filter membership events based on history visibility ( #3874 )
...
don't filter membership events based on history visibility
as we will already have filtered the messages in the timeline, and state events
are always visible.
and because @erikjohnston said so.
2018-09-14 18:12:52 +01:00
3e6e94fe9f
Merge pull request #3872 from matrix-org/hawkowl/timeouts-2
...
timeouts 2: electric boogaloo
2018-09-14 16:58:44 +01:00
bc9af88a2d
fix
2018-09-15 00:26:00 +10:00
d0f6c1ce21
Remove spurious comment
2018-09-14 15:12:36 +01:00
9e2f9a7b57
Measure outbound requests
2018-09-14 15:11:26 +01:00
0a81038ea0
Add in flight real time metrics for Measure blocks
2018-09-14 15:08:37 +01:00
7de1989ea2
fix link for case that config.email_riot_base_url is set
2018-09-13 22:43:50 +01:00
c971aa7b9d
fix
2018-09-14 03:57:02 +10:00
8f08d848f5
fix
2018-09-14 03:53:56 +10:00
f1a7264663
Fix minor typo in exception
2018-09-13 11:51:12 -06:00
7c33ab76da
redact better
2018-09-14 03:45:34 +10:00
63755fa4c2
we do that higher up
2018-09-14 03:21:47 +10:00
73884ebac5
Merge remote-tracking branch 'origin/develop' into hawkowl/timeouts-2
2018-09-14 03:11:25 +10:00
7c27c4d51c
merge ( #3576 )
2018-09-14 03:11:11 +10:00
1c3f4d9ca5
buffer?
2018-09-14 03:09:13 +10:00
bc74925c5b
WIP e2e key backups
...
Continues from uhoreg's branch
This just fixed the errcode on /room_keys/version if no backup and
updates the schema delta to be on the latest so it gets run
2018-09-13 17:02:59 +01:00
6c0f8d9d50
Merge pull request #3856 from matrix-org/erikj/speed_up_purge
...
Make purge history slightly faster
2018-09-13 16:14:46 +01:00
ed5331a627
comment
2018-09-13 16:10:56 +01:00
89a76d1889
Fix handling of redacted events from federation
...
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.
This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
2018-09-13 15:44:12 +01:00
bfa0b759e0
Attempt to figure out what's going on with timeouts ( #3857 )
2018-09-14 00:15:51 +10:00
9cbd0094f0
pep8
2018-09-13 15:15:35 +01:00
9dbe38ea7d
Create indices after insertion
2018-09-13 15:05:52 +01:00
93139a1fb8
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_purge
2018-09-13 12:57:09 +01:00
c857f5ef9b
Make purge history slightly faster
...
Don't pull out events that are outliers and won't be deleted, as nothing
should happen to them.
2018-09-13 12:48:10 +01:00
b7d2fb5eb9
Remove some superfluous logging ( #3855 )
2018-09-13 19:59:32 +10:00
f30a303590
Merge pull request #3846 from matrix-org/neilj/expose-registered-users
...
expose number of real reserved users
2018-09-12 17:14:04 +01:00
2ac1abbc7e
show heroes if a room has a 'deleted' name/canonical_alias ( #3851 )
2018-09-12 17:11:05 +01:00
fa0d464fa4
Merge pull request #3853 from matrix-org/erikj/log_outbound_each_time
...
Log outbound requests when we retry
2018-09-12 16:55:40 +01:00
0403cf0783
argh pep8
2018-09-12 16:54:28 +01:00
0e200e366d
correctly log gappy sync metrics
2018-09-12 16:47:20 +01:00
11bfc2af1c
fix logline
2018-09-12 16:45:42 +01:00
8decd6233d
improve naming
2018-09-12 16:22:15 +01:00
8c5b84441b
Log outbound requests when we retry
2018-09-12 16:22:14 +01:00
54f8616d2c
Merge pull request #3841 from matrix-org/erikj/manhole_key_length
...
Change the manhole SSH key to have more bits
2018-09-12 14:33:39 +01:00
7ca097f77e
Port federation/ to py3 ( #3847 )
2018-09-12 23:23:32 +10:00
0ddf486724
expose number of real reserved users
2018-09-12 11:58:52 +01:00
546aee7e52
Merge pull request #3835 from krombel/fix_3821
...
fix VOIP crashes under Python 3
2018-09-12 20:44:18 +10:00
33716c4aea
Merge pull request #3826 from matrix-org/rav/logging_for_keyring
...
add some logging for the keyring queue
2018-09-12 20:43:47 +10:00
bc635026c5
Merge pull request #3824 from matrix-org/rav/fix_jwt_import
...
Fix jwt import check
2018-09-12 20:41:57 +10:00
02aa41809b
Port rest/ to Python 3 ( #3823 )
2018-09-12 20:41:31 +10:00
8fd93b5eea
Port crypto/ to Python 3 ( #3822 )
2018-09-12 20:16:31 +10:00
4084a774a8
Timeout reading body for outbound HTTP requests
2018-09-12 10:10:20 +01:00
b041115415
Speed up lazy loading ( #3827 )
...
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats (#3840 )
2018-09-12 00:50:39 +01:00
9e05c8d309
Change the manhole SSH key to have more bits
...
Newer versions of openssh client refuse to connect to the old key due to
its length.
2018-09-11 10:42:10 +01:00
7bc22539ff
fix VOIP crashes under Python 3 ( #3821 )
2018-09-10 14:30:08 +02:00
9a5ea511b5
Merge pull request #3810 from matrix-org/erikj/send_tags_down_sync_on_join
...
Send existing room tags down sync on join
2018-09-07 23:28:42 +10:00
b8ad756bd0
Fix jwt import check
...
This handy code attempted to check that we could import jwt, but utterly failed
to check it was the right jwt.
Fixes https://github.com/matrix-org/synapse/issues/3793
2018-09-07 14:20:54 +01:00
771d213ac5
Merge branch 'master' into develop
2018-09-07 21:45:38 +10:00
6febd8e8f7
version
2018-09-07 21:40:57 +10:00
cd7ef43872
clearer logging when things fail, too
2018-09-06 23:56:47 +01:00
806964b5de
add some logging for the keyring queue
...
why is it so damn slow?
2018-09-06 18:51:06 +01:00
c5440b2ca0
Merge pull request #3800 from matrix-org/neilj/remove-guests-from-mau-count
...
guest users should not be part of mau total
2018-09-06 17:45:55 +01:00
84a750e0c3
ensure guests never enter mau list
2018-09-06 17:22:53 +01:00
f60c9e2a01
Don't send empty tags list down sync
2018-09-06 17:01:41 +01:00
7baf66ef5d
Send existing room tags down sync on join
...
When a user joined a room any existing tags were not sent down the sync
stream. Ordinarily this isn't a problem because the user needs to be in
the room to have set tags in it, however synapse will sometimes add tags
for a user to a room, e.g. for server notices, which need to come down
sync.
2018-09-06 16:46:51 +01:00
3801b8aa03
try to make flake8 and isort happy
2018-09-06 11:35:19 -04:00
654324eded
Merge pull request #3805 from matrix-org/erikj/limit_transaction_pdus_edus
...
Limit the number of PDUs/EDUs per fedreation transaction
2018-09-07 01:33:31 +10:00
70fd75cd1d
Merge pull request #3788 from matrix-org/erikj/remove_conn_id
...
Remove conn_id from repl prometheus metrics
2018-09-07 00:48:19 +10:00
ee7c8bd2b5
Merge pull request #3795 from matrix-org/erikj/faster_sync_state
...
User iter* during sync state calculations
2018-09-07 00:24:43 +10:00
6707a3212c
Limit the number of PDUs/EDUs per fedreation transaction
2018-09-06 15:23:55 +01:00
135f3b4390
Merge pull request #3804 from matrix-org/rav/fix_openssl_dep
...
bump dep on pyopenssl to 16.x
2018-09-07 00:23:39 +10:00
2608ebc04c
Port handlers/ to Python 3 ( #3803 )
2018-09-07 00:22:23 +10:00
417e7077aa
Bump version and changelog
2018-09-06 14:08:55 +01:00
d64b24dfe6
Synapse 0.33.3.1 (2018-09-06)
...
=============================
SECURITY FIXES
--------------
- Fix an issue where event signatures were not always correctly validated ([\#3796](https://github.com/matrix-org/synapse/issues/3796 ))
- Fix an issue where server_acls could be circumvented for incoming events ([\#3796](https://github.com/matrix-org/synapse/issues/3796 ))
Internal Changes
----------------
- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802 ))
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAABCAAGBQJbkPLrAAoJEIofk9V1tejV7K4IAItYIX98DKN9x3FNs9Hd69Pw
mZtqmuJ12YAyiVZoR7IJ5GfyctHCYUUXcqmNR1O+2/IRvezFnU6ZTcPW3OfNfnuD
vlnMiK53F21T96ul3Wu47Z2wyO+WkoKeXdvlqt3Wa8HpnPbU6y5CSac2vK57ppTU
DcMZKXaDae6vA7bsjryYgTRRplH6eFWZEexjbuZudbDvqkySi2zGUTs6SUesMC5B
FRPgI6p6sCeQgGMJY+d1i+ZdzWkZhH5OLH2icf2MI1hnhSnRSJ/scSGm5OYDCwYy
03hOy0ZVMr8SB5/j0TjSaFXonKsmPKqxnU2g7crJmvFP4BfJrEbym0y9YhYXnVQ=
=8PDU
-----END PGP SIGNATURE-----
Merge tag 'v0.33.3.1' into release-v0.33.4
Synapse 0.33.3.1 (2018-09-06)
=============================
SECURITY FIXES
--------------
- Fix an issue where event signatures were not always correctly validated ([\#3796](https://github.com/matrix-org/synapse/issues/3796 ))
- Fix an issue where server_acls could be circumvented for incoming events ([\#3796](https://github.com/matrix-org/synapse/issues/3796 ))
Internal Changes
----------------
- Unignore synctl in .dockerignore to fix docker builds ([\#3802](https://github.com/matrix-org/synapse/issues/3802 ))
2018-09-06 14:08:33 +01:00
4f8baab0c4
Merge branch 'master' into develop
2018-09-06 13:05:22 +01:00
625542878d
bump dep on pyopenssl to 16.x
2018-09-06 12:53:15 +01:00
80189ed27c
prepare v0.33.3.1
2018-09-06 10:26:23 +01:00
61b05727fa
guest users should not be part of mau total
2018-09-05 22:30:36 +01:00
dfba1d843d
Merge pull request #3790 from matrix-org/rav/respect_event_format_in_filter
...
Implement 'event_format' filter param in /sync
2018-09-05 16:24:14 +01:00
7419764351
User iter* during sync state calculations
2018-09-05 16:19:50 +01:00
2d2828dcbc
Port http/ to Python 3 ( #3771 )
2018-09-06 00:10:47 +10:00
c127c8d042
Fix origin handling for pushed transactions
...
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
2018-09-05 13:08:07 +01:00
804dd41e18
Check that signatures on events are valid
...
We should check that both the sender's server, and the server which created the
event_id (which may be different from whatever the remote server has told us
the origin is), have signed the event.
2018-09-05 13:08:07 +01:00
5f02017aea
Improve performance of getting typing updates for replication
...
Fetching the list of all new typing notifications involved iterating
over all rooms and comparing their serial. Lets move to using a stream
change cache, like we do for other streams.
2018-09-05 10:20:40 +01:00
87c18d12ee
Implement 'event_format' filter param in /sync
...
This has been specced and part-implemented; let's implement it for /sync (but
no other endpoints yet :/).
2018-09-04 15:20:09 +01:00
a6cf7d9d9a
Merge pull request #3789 from matrix-org/neilj/improve_threepid_error_strings
...
improve human readable error messages
2018-09-04 13:16:00 +00:00
7e9ced4178
version and towncrier
2018-09-04 21:12:04 +10:00
bae37cd811
improve human readable error message
2018-09-04 12:07:00 +01:00
c42f7fd7b9
improve human readable error messages
2018-09-04 12:03:17 +01:00
3e242dc149
Remove conn_id
2018-09-04 11:45:52 +01:00
b13836da7f
Remove conn_id from repl prometheus metrics
...
`conn_id` gets set to a random string, and so we end up filling up
prometheus with tonnes of data series, which is bad.
2018-09-03 17:22:49 +01:00
567363e497
Merge pull request #3737 from matrix-org/erikj/remove_redundant_state_func
...
Remove unnecessary resolve_events_with_state_map
2018-09-03 16:19:41 +01:00
301cb60d0b
assert rather than warn
2018-08-31 17:29:35 +01:00
0b01281e77
move threepid checker to config, add missing yields
2018-08-31 17:11:11 +01:00
e8e540630e
fix reference to is_threepid_reserved
2018-08-31 16:09:15 +01:00
09f3cf1a7e
ensure post registration auth checks do not fail erroneously
2018-08-31 15:42:51 +01:00
ea068d6f3c
fix bug where preserved threepid user comes to sign up and server is mau blocked
2018-08-31 10:49:14 +01:00
14e4d4f4bf
Port storage/ to Python 3 ( #3725 )
2018-08-31 00:19:58 +10:00
475253a88e
Merge pull request #3764 from matrix-org/rav/close_db_conn_after_init
...
Make sure that we close db connections opened during init
2018-08-30 10:47:27 +01:00
79a8a347a6
fix #3445
...
itervalues(d) calls d.itervalues() [PY2] and d.values() [PY3]
but SortedDict only implements d.values()
2018-08-29 16:28:25 +02:00
b1580f50fe
don't return non-LL-member state in incremental sync state blocks ( #3760 )
...
don't return non-LL-member state in incremental sync state blocks
2018-08-28 23:25:58 +01:00
32eb1dedd2
use abc.abstractproperty
...
This gives clearer messages when someone gets it wrong
2018-08-28 17:10:43 +01:00
0b07f02e19
Make sure that we close db connections opened during init
...
We should explicitly close any db connections we open, because failing to do so
can block other transactions as per
https://github.com/matrix-org/synapse/issues/3682 .
Let's also try to factor out some of the boilerplate by having server classes
define their datastore class rather than duplicating the whole of `setup`.
2018-08-28 13:39:49 +01:00
16a31c6fce
update to newer Synapse APIs
2018-08-24 22:51:25 -04:00
9db2476991
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_contact
2018-08-24 17:00:37 +01:00
05077e06fa
Change admin_uri to admin_contact in config and errors
2018-08-24 16:51:27 +01:00
83caead95a
Merge branch 'develop' into e2e_backups
2018-08-24 11:44:26 -04:00
01a5a8b9e3
Fix checking if service notice room is already tagged
...
This manifested in synapse repeatedly setting the tag for the room
2018-08-24 16:22:37 +01:00
cdd24449ee
Ensure we wake up /sync when we add tag to notice room
2018-08-24 14:50:03 +01:00
14d49c51db
Make content of tag an empty object rather than null
2018-08-24 14:44:16 +01:00
84b4e76fed
Merge pull request #3754 from matrix-org/erikj/fix_whitelist
...
Allow federation_domain_whitelist to be emtpy list
2018-08-24 12:23:39 +01:00
1d67b13674
Fix bug when federation_domain_whitelist is an emtpy list
...
Outbound federation were incorrectly allowed when the config option was
set to an empty list
2018-08-24 12:13:12 +01:00
92d50e3c2a
Merge pull request #3753 from matrix-org/erikj/fix_no_server_noticse
...
Fix bug where we broke sync when using limit_usage_by_mau
2018-08-24 11:56:08 +01:00
e94cdbaecf
Merge pull request #3751 from matrix-org/rav/twisted_17
...
Pin to twisted 17.1 or later
2018-08-24 11:55:52 +01:00
7f147d623b
Fix bug where we broke sync when using limit_usage_by_mau
...
We assumed that we always had service notices configured, but that is
not always true
2018-08-24 11:33:50 +01:00
f584d6108f
Pin to twisted 17.1 or later
...
Fixes https://github.com/matrix-org/synapse/issues/3741 .
2018-08-24 10:02:31 +01:00
cd77270a66
Implement trail users
2018-08-23 19:17:19 +01:00
7e6e588e60
Fix bug where we resent "limit exceeded" server notices
...
This was due to a bug where we mutated a cached event's contents
2018-08-23 16:21:20 +01:00
7a0da69eee
Add missing yield
2018-08-23 10:28:12 +01:00
fd2dbf1836
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-22 17:06:10 +01:00
9643a6f7f2
Update notice format
2018-08-22 17:00:29 +01:00
5c261107c9
Remove unnecessary resolve_events_with_state_map
...
We only ever used the synchronous resolve_events_with_state_map in one
place, which is trivial to replace with the async version.
2018-08-22 15:41:15 +01:00
c7181dcc6c
Merge branch 'master' into develop
2018-08-22 14:37:11 +01:00
74854a9719
Use recaptcha_ajax.js directly from Google
...
This was originally done in commit c75b71a397#3677 ) being based on the wrong
branch.
We're ready to merge this to master now, so let's make it match
release-v0.33.3.
2018-08-22 14:30:49 +01:00
48fec67536
Features
...
--------
- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439 ))
- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184 ))
- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568 ))
- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574 ))
- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589 ))
- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633 ))
- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653 ))
- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654 ))
- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655 ))
- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662 ))
- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664 ))
- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670 ))
- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687 ))
- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694 ))
- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708 ))
Bugfixes
--------
- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658 ))
- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661 ))
- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676 ))
- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677 ))
- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681 ))
- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689 ))
- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692 ))
- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700 ))
- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701 ))
- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705 ), [\#3713](https://github.com/matrix-org/synapse/issues/3713 ))
- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710 ))
- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719 ))
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723 ))
- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732 ))
Deprecations and Removals
-------------------------
- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst ). ([\#3703](https://github.com/matrix-org/synapse/issues/3703 ))
Internal Changes
----------------
- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423 ))
- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632 ))
- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647 ))
- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660 ))
- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668 ))
- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669 ))
- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678 ))
- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679 ))
- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684 ))
- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690 ))
- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707 ))
- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709 ))
- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712 ))
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJbfWKDAAoJEIofk9V1tejVHOMH/RmP+axF2K+Socnnb5Fa6RTo
pDJXVwsvNQxcwzZIJXh2TRilVr88jx7A1PHUde0TmkHRmis/bDzZxa59QRA4rFEy
fLYMx9Tyt0RzAJrEd0p7jbIkJxxPm5PbMWfCgVfzg44q0kepRW6ZVl1+kFHDrsCo
ocE03WdLe2sRHSudaVtBLOVMA1lDE22v40/7xiu9IbUOnF+WZ3cjXYX+SNr5eb6g
7YGbXOk4/E8XoUnpzH5oEnNtiDK5piRrr7o1282kecY20ZMNzKgseBHU6nYMv5ZO
TUpCPwbukmzeGHITupY1RkzpmBbEIJTCnmLP8fiUw+RWznYyFC/aRFMiWTzle3A=
=ngmn
-----END PGP SIGNATURE-----
Merge tag 'v0.33.3'
Features
--------
- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439 ))
- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184 ))
- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568 ))
- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574 ))
- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589 ))
- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633 ))
- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653 ))
- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654 ))
- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655 ))
- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662 ))
- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664 ))
- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670 ))
- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687 ))
- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694 ))
- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708 ))
Bugfixes
--------
- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658 ))
- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661 ))
- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676 ))
- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677 ))
- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681 ))
- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689 ))
- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692 ))
- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700 ))
- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701 ))
- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705 ), [\#3713](https://github.com/matrix-org/synapse/issues/3713 ))
- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710 ))
- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719 ))
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723 ))
- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732 ))
Deprecations and Removals
-------------------------
- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst ). ([\#3703](https://github.com/matrix-org/synapse/issues/3703 ))
Internal Changes
----------------
- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423 ))
- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632 ))
- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647 ))
- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660 ))
- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668 ))
- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669 ))
- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678 ))
- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679 ))
- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684 ))
- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690 ))
- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707 ))
- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709 ))
- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712 ))
2018-08-22 14:28:55 +01:00
4e5a4549b6
bump version to 0.33.3
2018-08-22 14:07:10 +01:00
db10f553ba
Merge pull request #3724 from Half-Shot/hs/guest-fetch-event
...
Allow guests to use /rooms/:roomId/event/:eventId
2018-08-22 13:41:08 +01:00
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles
...
Allow profile updates to happen on workers
2018-08-22 11:35:55 +01:00
8432e2ebd7
Rename WorkerProfileHandler to BaseProfileHandler
2018-08-22 10:13:40 +01:00
a81f140880
Add assert to ensure handler is only run on master
2018-08-22 10:11:21 +01:00
47b25ba5f3
Remove redundant vars
2018-08-22 10:09:05 +01:00
3bf8bab8f9
Merge pull request #3673 from matrix-org/erikj/refactor_state_handler
...
Refactor state module to support multiple room versions
2018-08-22 10:04:55 +01:00
a4cf660a32
Merge pull request #3735 from matrix-org/travis/federation-spelling
...
limt -> limit
2018-08-22 09:34:21 +01:00
0d568ff403
Merge remote-tracking branch 'origin/release-v0.33.3' into develop
2018-08-22 09:15:44 +01:00
bb81e78ec6
Split the state_group_cache in two ( #3726 )
...
Splits the state_group_cache in two.
One half contains normal state events; the other contains member events.
The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache. This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching.
Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow. Hopefully fixes https://github.com/matrix-org/synapse/issues/3720 .
2018-08-22 00:56:37 +02:00
f7bf181a90
fix another consent encoding fail
2018-08-21 23:14:25 +01:00
f7baff6f7b
Fix 500 error from /consent form
...
Fixes #3731
2018-08-21 22:47:07 +01:00
a52f276990
Bugfixes
...
--------
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs
([\#3723](https://github.com/matrix-org/synapse/issues/3723 ))
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEIQBQJ4l+yK4dlKkFIwi0edOSShEFAlt6/yMACgkQIwi0edOS
ShFrzAgAhLA74fClydGLfhPNCXIw12Amv2ASA6+r0ukqjYQ/9MZ0CRsL+PMZJTk6
Z75PK3TkX38zSpPaUx9HdMQO/yE+rpI99ZxlBM+8jzsf1jyF8ljqxSfixXUlaXX9
698PQQ3f+IDoSs59NKrHvoWhUThaBpf/1GwRxG5mUcm2oKJItNMHk/uLurNCaIIZ
YSwWcxvZdluOMX0WrlF/S524DqtgULcRf3E9x3mumobkVGxI3UH73hXgGUcPi7XV
ttLuy7fVbV2Cun/wvjBqD7kc8LNbKwSaT9y+/xS0D5QYjph4rpQiTG7S5rtPLPMU
Z2lL1C5ggBCkD43g3orDvehbNWCL5g==
=0vEe
-----END PGP SIGNATURE-----
Merge tag 'v0.33.3rc2' into develop
Bugfixes
--------
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs
([\#3723](https://github.com/matrix-org/synapse/issues/3723 ))
2018-08-21 20:30:09 +01:00
42a394caa2
allow session_data to be any JSON instead of just a string
2018-08-21 14:51:34 -04:00
46c832eaac
Merge pull request #3727 from matrix-org/erikj/dont_error_on_missing_keys
...
Don't log exceptions when failing to fetch server keys
2018-08-21 17:07:20 +01:00
cd6937fb26
Fix typo
2018-08-21 16:28:10 +01:00
8550a7e9c2
allow auth_data to be any JSON instead of a string
2018-08-21 10:38:00 -04:00
c2c153dd3b
Log more detail when we fail to authenticate request
2018-08-21 11:42:49 +01:00
808d8e06aa
Don't log exceptions when failing to fetch server keys
...
Not being able to resolve or connect to remote servers is an expected
error, so we shouldn't log at ERROR with stacktraces.
2018-08-21 11:19:26 +01:00
3f6762f0bb
isort
2018-08-21 09:38:38 +01:00
3b5b64ac99
changelog
2018-08-21 03:48:55 +10:00
f89f6b7c09
Allow guests to access /rooms/:roomId/event/:eventId
2018-08-20 18:25:54 +01:00
be6527325a
Fix exceptions when a connection is closed before we read the headers
...
This fixes bugs introduced in #3700 , by making sure that we behave sanely
when an incoming connection is closed before the headers are read.
2018-08-20 18:21:10 +01:00
55e6bdf287
Robustness fix for logcontext filter
...
Make the logcontext filter not explode if it somehow ends up with a logcontext
of None, since that infinite-loops the whole logging system.
2018-08-20 18:20:07 +01:00
b01a755498
Make the in flight requests metrics thread safe
2018-08-20 17:27:52 +01:00
1058d14127
Make the in flight background process metrics thread safe
2018-08-20 17:27:24 +01:00
9a2f960736
version
2018-08-21 00:00:19 +10:00
324525f40c
Port over enough to get some sytests running on Python 3 ( #3668 )
2018-08-20 23:54:49 +10:00
4d664278af
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler
2018-08-20 14:49:43 +01:00
8dee601054
Remove redundant room_version checks
2018-08-20 14:48:53 +01:00
e21c368b8b
Revert spurious change
2018-08-20 13:54:51 +01:00
f2a48d87df
Use get_cache_factor_for function for `state_cache`
...
This allows the cache factor for `state_cache` to be individually
specified in the enviroment
2018-08-20 13:01:46 +01:00
2aa7cc6a46
Merge pull request #3713 from matrix-org/erikj/fixup_fed_logging
...
Fix logging bug in EDU handling over replication
2018-08-20 10:51:45 +01:00
e07970165f
rename error code
2018-08-18 14:39:45 +01:00
c5171bf171
special case server_notices_mxid
2018-08-18 12:33:07 +01:00
ba1fbf7d5b
special case server_notices_mxid
2018-08-18 12:31:08 +01:00
3cef867cc1
Merge pull request #3709 from matrix-org/rav/logcontext_for_replication_commands
...
Logcontexts for replication command handlers
2018-08-17 16:22:07 +01:00
c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates
...
Fix logcontexts for running pushers
2018-08-17 16:21:49 +01:00
c334ca67bb
Integrate presence from hotfixes ( #3694 )
2018-08-18 01:08:45 +10:00
04f5d2db62
Remove v1/register's broken shared secret functionality ( #3703 )
2018-08-18 00:55:01 +10:00
ab822a2d1f
Add some fixmes
2018-08-17 15:31:50 +01:00
91cdb6de08
Call UserDirectoryHandler methods directly
...
Turns out that the user directory handling is fairly racey as a bunch
of stuff assumes that the processing happens on master, which it doesn't
when there is a synapse.app.user_dir worker. So lets just call the
function directly until we actually get round to fixing it, since it
doesn't make the situation any worse.
2018-08-17 15:26:13 +01:00
d49b77404b
clean up, no functional changes
2018-08-17 15:21:34 +01:00
63260397c6
Merge pull request #3701 from matrix-org/rav/use_producer_for_responses
...
Use a producer to stream back responses
2018-08-17 14:58:45 +01:00
3f8709ffe4
Merge pull request #3700 from matrix-org/rav/wait_for_producers
...
Refactor request logging code
2018-08-17 14:57:45 +01:00
3ee57bdcbb
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-17 14:34:10 +01:00
782689bd40
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-17 14:15:48 +01:00
ca87ad1def
Split ProfileHandler into master and worker
2018-08-17 14:15:14 +01:00
b5f638f1f4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-17 14:04:15 +01:00
9fd161c6fb
Merge branch 'neilj/limit_exceeded_error' of github.com:matrix-org/synapse into neilj/limit_exceeded_error
2018-08-17 13:58:40 +01:00
0195dfbf52
server limits config docs
2018-08-17 13:58:25 +01:00
69c49d3fa3
Merge branch 'develop' into neilj/limit_exceeded_error
2018-08-17 12:44:26 +00:00
38f708a2bb
Remote profile cache should remain in master worker
2018-08-17 11:37:42 +01:00
3b2dcfff78
Fix logging bug in EDU handling over replication
2018-08-17 11:11:06 +01:00
521d369e7a
remove errant yield
2018-08-17 10:12:11 +01:00
0e8d78f6aa
Logcontexts for replication command handlers
...
Run the handlers for replication commands as background processes. This should
improve the visibility in our metrics, and reduce the number of "running db
transaction from sentinel context" warnings.
Ideally it means converting the things that fire off deferreds into the night
into things that actually return a Deferred when they are done. I've made a bit
of a stab at this, but it will probably be leaky.
2018-08-17 00:43:43 +01:00
66f7dc8c87
Fix logcontexts for running pushers
...
First of all, avoid resetting the logcontext before running the pushers, to fix
the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning.
Instead, give them their own "background process" logcontexts.
2018-08-17 00:32:39 +01:00
bcfeb44afe
call reap on start up and fix under reaping bug
2018-08-16 22:55:32 +01:00
372bf073c1
block event creation and room creation on hitting resource limits
2018-08-16 21:25:16 +01:00
13ad9930c8
add new error type ResourceLimit
2018-08-16 18:02:02 +01:00
51b17ec566
flake8
2018-08-16 17:32:22 +01:00
3c1080b6e4
refactor for readability, and reuse caching for setting tags
2018-08-16 17:02:04 +01:00
eff3ae3b9a
add room tagging
2018-08-16 15:48:34 +01:00
7d5b1a60a3
Fix inbound federation on reader worker
...
Inbound federation requires calculating push, which in turn relies on
having access to account data.
2018-08-16 15:32:20 +01:00
bfb6c58624
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-16 15:10:16 +01:00
a675f9c556
check for room state before deciding on action
2018-08-16 14:53:35 +01:00
c151b32b1d
Add GET media/v1/config ( #3184 )
2018-08-16 14:23:38 +01:00
762a758fea
lazyload aware /messages ( #3589 )
2018-08-16 14:22:47 +01:00
25d2b5d55f
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-16 11:37:17 +01:00
df1e4f259f
WIP impl commiting to get feedback
2018-08-16 11:10:53 +01:00
c055c91655
fix case where empty string state check is evaulated as False
2018-08-16 11:10:19 +01:00
3f543dc021
initial cut at a room summary API ( #3574 )
2018-08-16 09:46:50 +01:00
8cfad2e686
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-15 17:20:38 +01:00
81d727efa9
Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus
...
Fix Mau off by one errors
2018-08-15 16:19:41 +00:00
2f78f432c4
speed up /members and add at= and membership params ( #3568 )
2018-08-15 16:35:22 +01:00
fc5d937550
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-15 16:31:40 +01:00
86a00e05e1
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-15 16:27:08 +01:00
eabc5f8271
wip cut at sending resource server notices
2018-08-15 15:04:52 +01:00
c24fc9797b
add new event types
2018-08-15 15:04:30 +01:00
afcd655ab6
Use a producer to stream back responses
...
The problem with dumping all of the json response into the Request object at
once is that doing so starts the timeout for the next request to be received:
so if it takes longer than 60s to stream back the response to the client, the
client never gets it.
The correct solution is to use a Producer; then the timeout is only started
once all of the content is sent over the TCP connection.
2018-08-15 15:04:16 +01:00
dc56c47dc0
Merge pull request #3653 from matrix-org/erikj/split_federation
...
Move more federation APIs to workers
2018-08-15 14:59:02 +01:00
4601129c44
Merge pull request #3687 from matrix-org/neilj/admin_email
...
support admin_email config and pass through into blocking errors,
2018-08-15 13:52:25 +00:00
ef184caf30
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-15 14:25:46 +01:00
488ffe6fdb
Use federation handler function rather than duplicate
...
This involves renaming _persist_events to be a public function.
2018-08-15 14:17:18 +01:00
773db62a22
Rename slave TransactionStore to SlaveTransactionStore
2018-08-15 14:17:06 +01:00
eaaa2248ff
Refactor request logging code
...
This commit moves a bunch of the logic for deciding when to log the receipt and
completion of HTTP requests into SynapseRequest, rather than in the request
handling wrappers.
Advantages of this are:
* we get logs for *all* requests (including OPTIONS and HEADs), rather than
just those that end up hitting handlers we've remembered to decorate
correctly.
* when a request handler wires up a Producer (as the media stuff does
currently, and as other things will do soon), we log at the point that all
of the traffic has been sent to the client.
2018-08-15 13:47:52 +01:00
87a824bad1
clean up AuthError
2018-08-15 11:58:03 +01:00
c4eb97518f
Merge branch 'neilj/update_limits_error_codes' of github.com:matrix-org/synapse into neilj/admin_email
2018-08-15 11:53:01 +01:00
55afba0fc5
update admin email to uri
2018-08-15 11:41:18 +01:00
1c5e690a6b
Merge pull request #3690 from matrix-org/neilj/change_prometheus_mau_metric_name
...
combine mau metrics into one group
2018-08-15 09:53:59 +00:00
fef2e65d12
Merge pull request #3667 from matrix-org/erikj/fixup_unbind
...
Don't fail requests to unbind 3pids for non supporting ID servers
2018-08-15 10:32:12 +01:00
b8429c7c81
update error codes for resource limiting
2018-08-15 10:19:48 +01:00
ab035bdeac
replace admin_email with admin_uri for greater flexibility
2018-08-15 10:16:41 +01:00
19b433e3f4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email
2018-08-14 17:44:46 +01:00
b586b8b986
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 17:43:22 +01:00
70e48cbbb1
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/change_prometheus_mau_metric_name
2018-08-14 17:40:59 +01:00
9ecbaf8ba8
adding missing yield
2018-08-14 16:55:28 +01:00
e5962f845c
pep8
2018-08-14 16:36:14 +01:00
e7d091fb86
combine mau metrics into one group
2018-08-14 16:26:55 +01:00
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block
...
Block ability to read via sync if mau limit exceeded
2018-08-14 15:21:31 +00:00
2545993ce4
make comments clearer
2018-08-14 15:48:12 +01:00
8f9a7eb58d
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-14 15:11:54 +01:00
c74c71128d
remove blank line
2018-08-14 15:06:24 +01:00
ed4bc3d2fc
fix off by 1s on mau
2018-08-14 15:04:48 +01:00
bd92c8eaa7
Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:56:45 +01:00
99ebaed8e6
Update register.py
...
remove comments
2018-08-14 14:55:55 +01:00
9b5bf3d858
Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:51:38 +01:00
e25d87d97b
Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:32:18 +01:00
e2c9fe0a6a
backout ability to pass in event type to server notices
2018-08-14 13:32:56 +01:00
591bf87c6a
Merge remote-tracking branch 'origin/develop' into neilj/fix_reap_users_in_postgres
2018-08-14 20:56:23 +10:00
9b75c78b4d
support server notice state events for resource limits
2018-08-14 11:20:41 +01:00
63417c31e9
fix typo
2018-08-13 22:36:52 +01:00
f4b49152e2
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-13 21:09:47 +01:00
50bcaf1a27
Merge pull request #3677 from andrewshadura/master
...
Use recaptcha_ajax.js directly from Google
2018-08-13 18:52:56 +01:00
34f51babc5
Revert "Use recaptcha_ajax.js directly from Google"
2018-08-13 18:48:03 +01:00
ce7de9ae6b
Revert "support admin_email config and pass through into blocking errors, return AuthError in all cases"
...
This reverts commit 0d43f991a1
2018-08-13 18:06:18 +01:00
0d43f991a1
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-13 18:00:23 +01:00
99dd975dae
Run tests under PostgreSQL ( #3423 )
2018-08-13 16:47:46 +10:00
4f7064f6b5
missing import
2018-08-12 19:14:31 -04:00
f0cede5556
missing import
2018-08-12 19:14:31 -04:00
54ac18e832
use parse_string
2018-08-12 19:14:31 -04:00
66a4ca1d28
404 nicely if you try to interact with a missing current version
2018-08-12 19:14:31 -04:00
72788cf9c1
support DELETE /version with no args
2018-08-12 19:14:31 -04:00
edc427a351
flake8
2018-08-12 19:14:31 -04:00
fe87890b18
implement remaining tests and make them work
2018-08-12 19:14:31 -04:00
15d513f16f
fix idiocies and so make tests pass
2018-08-12 19:14:31 -04:00
b5eee511c7
don't needlessly return user_id
2018-08-12 19:14:31 -04:00
93d174bcc4
improve docstring
2018-08-12 19:14:31 -04:00
5e42c45c96
switch get_current_version_info back to being get_version_info
2018-08-12 19:14:31 -04:00
982edca380
fix flakes
2018-08-12 19:14:31 -04:00
234611f347
fix typos
2018-08-12 19:14:31 -04:00
14b3da63a3
add a tonne of docstring; make upload_room_keys properly assert version
2018-08-12 19:14:31 -04:00
9f0791b7bd
add a tonne of docstring; make upload_room_keys properly assert version
2018-08-12 19:14:31 -04:00
9f500cb39e
more docstring for the e2e_room_keys rest
2018-08-12 19:14:31 -04:00
8d14598e90
add storage docstring; remove unused set_e2e_room_keys
2018-08-12 19:14:31 -04:00
ca0b052307
fix factoring out of _should_replace_room_key
2018-08-12 19:14:31 -04:00
cac0253799
rename room_key_version table correctly, and fix opt args
2018-08-12 19:14:31 -04:00
0abb205b47
blindly incorporate PR review - needs testing & fixing
2018-08-12 19:14:31 -04:00
69e51c7ba4
make /room_keys/version work
2018-08-12 19:14:31 -04:00
8ae64b270f
implement /room_keys/version too (untested)
2018-08-12 19:14:31 -04:00
cf1e2000f6
document the API
2018-08-12 19:13:09 -04:00
6b8c07abc2
make it work and fix pep8
2018-08-12 19:13:09 -04:00
0bc4627a73
interim WIP checkin; doesn't build yet
2018-08-12 18:23:10 -04:00
53ace904b2
total WIP skeleton for /room_keys API
2018-08-12 18:23:10 -04:00
31fa743567
fix sqlite/postgres incompatibility in reap_monthly_active_users
2018-08-11 22:38:34 +01:00
6c6aba76e1
implementation of server notices to alert on hitting resource limits
2018-08-10 15:12:59 +01:00
c31793a784
Merge branch 'rav/fix_linearizer_cancellation' into develop
2018-08-10 14:57:27 +01:00
c08f9d95b2
log *after* reloading log config
...
... because logging *before* reloading means the log message gets lost in the old MemoryLogger
2018-08-10 14:56:48 +01:00
b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
c75b71a397
Use recaptcha_ajax.js directly from Google
...
The script recaptcha_ajax.js contains minified non-free code which
we probably cannot redistribute. Since it talks to Google servers
anyway, it is better to just download it from Google directly instead
of shipping it.
This fixes #1932 .
2018-08-10 13:36:14 +02:00
3c0213a217
Merge pull request #3439 from vojeroen/send_sni_for_federation_requests
...
send SNI for federation requests
2018-08-10 12:23:54 +01:00
638d35ef08
Fix linearizer cancellation on twisted < 18.7
...
Turns out that cancellation of inlineDeferreds didn't really work properly
until Twisted 18.7. This commit refactors Linearizer.queue to avoid
inlineCallbacks.
2018-08-10 10:59:09 +01:00
01021c812f
wip at implementing MSC 7075
2018-08-09 22:16:00 +01:00
2e9c73e8ca
more generic conversion of str/bytes to unicode
2018-08-09 21:31:26 +02:00
64899341dc
include private functions from twisted
2018-08-09 21:04:22 +02:00
885ea9c602
rename _user_last_seen_monthly_active
2018-08-09 18:02:12 +01:00
c1f9dec92a
fix errant parenthesis
2018-08-09 17:43:26 +01:00
09cf130898
only block on sync where user is not part of the mau cohort
2018-08-09 17:39:12 +01:00
bb99b1f550
Add fast path in state res for zero prev events
2018-08-09 14:58:49 +01:00
ce6db0e547
Choose state algorithm based on room version
2018-08-09 14:58:47 +01:00
152c0aa58e
Add constants for room versions
2018-08-09 14:55:47 +01:00
119451dcd1
Refactor state module
...
We split out the actual state resolution algorithm to prepare for having
multiple versions.
2018-08-09 14:55:47 +01:00
69ce057ea6
block sync if auth checks fail
2018-08-09 12:26:27 +01:00
3dce9050cf
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block
2018-08-09 11:42:02 +01:00
0ad98e38d0
Merge pull request #3655 from matrix-org/neilj/disable_hs
...
Flag to disable HS without disabling federation
2018-08-09 10:41:43 +00:00
a5ef110749
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block
2018-08-09 11:40:37 +01:00
a6c813761a
Docstrings
2018-08-09 10:41:08 +01:00
b179537f2a
Move clean_room_for_join to master
2018-08-09 10:37:38 +01:00
72d1902bbe
Fixup doc comments
2018-08-09 10:23:49 +01:00
984376745b
Merge branch 'master' into develop
2018-08-09 19:23:47 +10:00
dafe90a6be
0.33.2
2018-08-09 19:20:41 +10:00
484a0ebdfc
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-09 10:16:29 +01:00
5785b93711
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-09 10:16:16 +01:00
bf7598f582
Log when we 3pid/unbind request fails
2018-08-09 10:09:56 +01:00
2bdafaf3c1
Merge pull request #3632 from matrix-org/erikj/refactor_repl_servlet
...
Add helper base class for generating new replication endpoints
2018-08-09 10:06:23 +01:00
62564797f5
Fixup wording and remove dead code
2018-08-09 09:56:10 +01:00
bb89c84614
Merge pull request #3664 from matrix-org/rav/federation_metrics
...
more metrics for the federation and appservice senders
2018-08-08 23:16:58 +01:00
d5c0ce4cad
updated docstring for ServerContextFactory
2018-08-08 19:25:01 +02:00
e92fb00f32
sync auth blocking
2018-08-08 17:54:49 +01:00
839a317c96
fix pep8 too many lines
2018-08-08 17:39:04 +01:00
5298d79fb5
Merge branch 'develop' into neilj/disable_hs
2018-08-08 16:13:03 +00:00
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions
...
Support for room versioning
2018-08-08 17:10:53 +01:00
d2f3ef98ac
Merge branch 'develop' into neilj/disable_hs
2018-08-08 15:55:47 +00:00
2b5baebeba
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/disable_hs
2018-08-08 13:47:07 +01:00
be59910b93
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/reserved_users
2018-08-08 13:45:21 +01:00
990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker
...
API for monthly_active_users table
2018-08-08 12:44:15 +00:00
312ae74746
typos
2018-08-08 13:33:16 +01:00
360ba89c50
Don't fail requests to unbind 3pids for non supporting ID servers
...
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.
Fixes #3661
2018-08-08 12:06:18 +01:00
bebe325e6c
Rename POST param to METHOD
2018-08-08 10:36:18 +01:00
5011417632
Fixup logging and docstrings
2018-08-08 10:29:58 +01:00
bab94da79c
fix metric name
2018-08-07 22:11:45 +01:00
53bca4690b
more metrics for the federation and appservice senders
2018-08-07 19:09:48 +01:00
ef3589063a
prevent total number of reserved users being too large
2018-08-07 17:59:27 +01:00
e8eba2b4e3
implement reserved users for mau limits
2018-08-07 17:49:43 +01:00
3523f5432a
Don't expose default_room_version as config opt
2018-08-07 12:51:57 +01:00
848431be1d
fix for rc1
2018-08-07 21:13:22 +10:00
3b9662339b
version
2018-08-07 21:10:52 +10:00
cd9765805e
Allow ratelimiting on workers
2018-08-07 10:50:28 +01:00
495cb100d1
Allow profile changes to happen on workers
2018-08-07 10:50:26 +01:00
ca9bc1f4fe
Fix occasional glitches in the synapse_event_persisted_position metric
...
Every so often this metric glitched to a negative number. I'm assuming it was
due to backfilled events.
2018-08-07 10:00:03 +01:00
a74b25faaa
WIP building out mau reserved users
2018-08-06 23:25:25 +01:00
7daa8a78c5
load mau limit threepids
2018-08-06 22:55:05 +01:00
89834d9a29
Merge branch 'neilj/mau_tracker' of github.com:matrix-org/synapse into neilj/disable_hs
2018-08-06 21:56:56 +01:00
e54794f5b6
Fix postgres compatibility bug
2018-08-06 21:55:54 +01:00
7bcf126b18
Merge branch 'neilj/mau_tracker' of github.com:matrix-org/synapse into neilj/disable_hs
2018-08-06 21:39:44 +01:00
1911c037cb
update comments to reflect new sig
2018-08-06 18:01:46 +01:00
33bd07d062
Merge branch 'neilj/mau_tracker' of github.com:matrix-org/synapse into neilj/disable_hs
2018-08-06 17:52:28 +01:00
16d78be315
make use of _simple_select_one_onecol, improved comments
2018-08-06 17:51:15 +01:00
19a17068f1
Check m.room.create for sane room_versions
2018-08-06 16:11:24 +01:00
96a9a29645
Pull in necessary stores in federation_reader
2018-08-06 15:23:57 +01:00
62ace05c45
Move necessary storage functions to worker classes
2018-08-06 15:23:38 +01:00
1e2bed9656
Import all functions from TransactionStore
2018-08-06 15:23:38 +01:00
a3f5bf79a0
Add EDU/query handling over replication
2018-08-06 15:23:31 +01:00
e26dbd82ef
Add replication APIs for persisting federation events
2018-08-06 15:02:28 +01:00
051a99c400
Fix isort
2018-08-06 14:29:31 +01:00
f900d50824
include known room versions in outgoing make_joins
2018-08-06 13:45:37 +01:00
42c6823827
disable HS from config
2018-08-04 22:07:04 +01:00
e40a510fbf
py3 fix
2018-08-03 23:19:13 +01:00
886be75ad1
bug fixes
2018-08-03 22:29:03 +01:00
16d9701892
Return M_NOT_FOUND when a profile could not be found. ( #3596 )
2018-08-03 19:08:05 +01:00
e10830e976
wip commit - tests failing
2018-08-03 17:55:50 +01:00
3777fa26aa
sanity check response from make_join
2018-08-03 16:08:32 +01:00
0d63d93ca8
Enforce compatibility when processing make_join requests
...
Reject make_join requests from servers which do not support the room version.
Also include the room version in the response.
2018-08-03 16:08:32 +01:00
0ca459ea33
Basic support for room versioning
...
This is the first tranche of support for room versioning. It includes:
* setting the default room version in the config file
* new room_version param on the createRoom API
* storing the version of newly-created rooms in the m.room.create event
* fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
15c1ae45e5
Docstrings for BaseFederationServlet
...
... to save me reverse-engineering this stuff again.
2018-08-03 16:08:32 +01:00
5593ff6773
fix (lots of) py3 test failures
2018-08-03 14:59:17 +01:00
950807d93a
fix caching and tests
2018-08-03 13:49:53 +01:00
897c51d274
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-03 13:40:47 +01:00
cb298ff623
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_repl_servlet
2018-08-03 09:25:15 +01:00
c0affa7b4f
update generate_monthly_active_users, and reap_monthly_active_users
2018-08-02 23:03:01 +01:00
5e2f7b8084
typo
2018-08-02 22:47:48 +01:00
4ecb4bdac9
wip attempt at caching
2018-08-02 22:41:05 +01:00
1fa98495d0
Merge pull request #3639 from matrix-org/rav/refactor_error_handling
...
Clean up handling of errors from outbound requests
2018-08-02 17:38:24 +01:00
bdae8f2e68
Merge pull request #3638 from matrix-org/rav/refactor_federation_client_exception_handling
...
Factor out exception handling in federation_client
2018-08-02 17:37:46 +01:00
74b1d46ad9
do mau checks based on monthly_active_users table
2018-08-02 16:57:35 +01:00
9180061b49
remove unused count_monthly_users
2018-08-02 15:55:29 +01:00
704c3e6239
Merge branch 'master' into develop
2018-08-02 15:43:30 +01:00
43ecfe0b10
Synapse 0.33.1 (2018-08-02)
...
===========================
SECURITY FIXES
--------------
- Fix a potential issue where servers could request events for rooms they have not joined. (`#3641 <https://github.com/matrix-org/synapse/issues/3641 >`_)
- Fix a potential issue where users could see events in private rooms before they joined. (`#3642 <https://github.com/matrix-org/synapse/issues/3642 >`_)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJbYxcCAAoJEIofk9V1tejVg8YIAJU1xoZ2zSnordJczpvVZbzx
Rsstk/wvE2dSGJL22gfSLzQNCbQvF4aOul1LAGJfbEQWc46BcNpyuWUxjTd3FaF7
SfbWdkDh1w3ROSuiPA8j4CnH6EwP0w2itIBhFG7JpVEXjRgCyc1iMUl/oQESv82g
UQyr6t/G68mE8xHm0eqvNfCjVAMSZDAnOzYZa7vfZJBqlZiGHB6Z1nsvsS3nZqQd
BQQi/pQTXxsIL2egSvuycykiZtEZcm+QDRFi6hIatv5LRFOsQUqKVTB/D7XOoSbP
RPOzcaSjv4mZ9a4NBzFMVVzzTRbr2EnipPd2ePrnJoOnl2eNRO4Won8zl4h+OQc=
=U/ir
-----END PGP SIGNATURE-----
Merge tag 'v0.33.1'
Synapse 0.33.1 (2018-08-02)
===========================
SECURITY FIXES
--------------
- Fix a potential issue where servers could request events for rooms they have not joined. (`#3641 <https://github.com/matrix-org/synapse/issues/3641 >`_)
- Fix a potential issue where users could see events in private rooms before they joined. (`#3642 <https://github.com/matrix-org/synapse/issues/3642 >`_)
2018-08-02 15:40:44 +01:00
14a4e7d5a4
Prepare 0.33.1
2018-08-02 15:31:04 +01:00
0bf5ec0db7
Check room visibility for /event/ requests
...
Make sure that the user has permission to view the requeseted event for
/event/{eventId} and /room/{roomId}/event/{eventId} requests.
Also check that the event is in the given room for
/room/{roomId}/event/{eventId}, for sanity.
2018-08-02 15:03:27 +01:00
14fa9d4d92
Avoid extra db lookups
...
Since we're about to look up the events themselves anyway, we can skip the
extra db queries here.
2018-08-02 13:55:51 +01:00
0a65450d04
Validation for events/rooms in fed requests
...
When we get a federation request which refers to an event id, make sure that
said event is in the room the caller claims it is in.
(patch supplied by @turt2live)
2018-08-02 13:48:40 +01:00
00f99f74b1
insertion into monthly_active_users
2018-08-02 13:47:19 +01:00
4a6725d9d1
Merge branch 'neilj/mau_tracker' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-02 11:04:18 +01:00
165e067033
Revert "change monthly_active_users table to be a single column"
...
This reverts commit ec716a35b2
2018-08-02 10:59:58 +01:00
40c1c59cf4
Merge pull request #3621 from matrix-org/erikj/split_fed_store
...
Split out DB writes in federation handler
2018-08-02 10:41:42 +01:00
08281fe6b7
self.db_conn unused
2018-08-01 23:26:24 +01:00
c21d82bab3
normalise reaping query
2018-08-01 23:24:38 +01:00
ec716a35b2
change monthly_active_users table to be a single column
2018-08-01 17:54:37 +01:00
d766f26de9
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-01 17:49:41 +01:00
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits
...
Initial impl of capping MAU
2018-08-01 15:58:45 +00:00
b8d7d3996b
Merge pull request #3620 from fuzzmz/return-404-room-not-found
...
return 404 if room not found
2018-08-01 16:34:32 +01:00
b7f203a566
count_monthly_users is now async
2018-08-01 16:17:42 +01:00
7ff44d9215
improve clarity
2018-08-01 16:17:00 +01:00
01e93f48ed
Kill off MatrixCodeMessageException
...
This code brings the SimpleHttpClient into line with the
MatrixFederationHttpClient by having it raise HttpResponseExceptions when a
request fails (rather than trying to parse for matrix errors and maybe raising
MatrixCodeMessageException).
Then, whenever we were checking for MatrixCodeMessageException and turning them
into SynapseErrors, we now need to check for HttpResponseExceptions and call
to_synapse_error.
2018-08-01 16:02:46 +01:00
018d75a148
Refactor code for turning HttpResponseException into SynapseError
...
This commit replaces SynapseError.from_http_response_exception with
HttpResponseException.to_synapse_error.
The new method actually returns a ProxiedRequestError, which allows us to pass
through additional metadata from the API call.
2018-08-01 16:02:46 +01:00
fa7dc889f1
Be more careful which errors we send back over the C-S API
...
We really shouldn't be sending all CodeMessageExceptions back over the C-S API;
it will include things like 401s which we shouldn't proxy.
That means that we need to explicitly turn a few HttpResponseExceptions into
SynapseErrors in the federation layer.
The effect of the latter is that the matrix errcode will get passed through
correctly to calling clients, which might help with some of the random
M_UNKNOWN errors when trying to join rooms.
2018-08-01 16:02:38 +01:00
c82ccd3027
Factor out exception handling in federation_client
...
Factor out the error handling from make_membership_event, send_join, and
send_leave, so that it can be shared.
2018-08-01 16:01:04 +01:00
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
303f1c851f
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits
2018-08-01 13:42:50 +01:00
a6d7b74915
update docs
2018-08-01 13:39:14 +01:00
4b256b9271
_persist_auth_tree no longer returns anything
2018-08-01 13:39:07 +01:00
4e5ac901dd
clean up
2018-08-01 12:03:57 +01:00
f9f5559971
fix comment
2018-08-01 12:03:42 +01:00
0aba3d361a
count_monthly_users() async
2018-08-01 11:47:58 +01:00
2c54f1c225
remove need to plot limit_usage_by_mau
2018-08-01 11:46:59 +01:00
f49147d14f
Merge pull request #3634 from matrix-org/rav/wtf_is_a_replication_layer
...
rename replication_layer to federation_client
2018-08-01 10:29:29 +01:00
cab782c17e
Merge pull request #3384 from matrix-org/rav/rewrite_cachedlist_decorator
...
Rewrite cache list decorator
2018-08-01 10:28:56 +01:00
6023cdd227
remove errant print
2018-08-01 10:27:17 +01:00
7931393495
make count_monthly_users async synapse/handlers/auth.py
2018-08-01 10:21:56 +01:00
c507fa15ce
only need to loop if mau limiting is enabled
2018-08-01 10:20:42 +01:00
37be52ac34
limt -> limit
2018-07-31 16:29:09 -06:00
70af98e361
return NotFoundError if room not found
...
Per the Client-Server API[0] we should return
`M_NOT_FOUND` if the room isn't found instead
of generic SynapseError.
This ensures that /directory/list API returns
404 for room not found instead of 400.
[0]: https://matrix.org/docs/spec/client_server/unstable.html#get-matrix-client-r0-directory-list-room-roomid
Signed-off-by: Serban Constantin <serban.constantin@gmail.com>
2018-07-31 21:47:23 +03:00
5e2ee64660
Merge pull request #3628 from turt2live/travis/goodby-pdu-failures
...
Remove pdu_failures from transactions
2018-07-31 12:13:09 -06:00
6ef983ce5c
api into monthly_active_users table
2018-07-31 16:36:24 +01:00
bdbdceeafa
rename replication_layer to federation_client
...
I have HAD ENOUGH of trying to remember wtf a replication layer is in terms of
classes.
2018-07-31 15:44:05 +01:00
443da003bc
Use new helper base class for membership requests
2018-07-31 14:32:23 +01:00
729b672823
Use new helper base class for ReplicationSendEventRestServlet
2018-07-31 14:32:23 +01:00
d81602b75a
Add helper base class for generating new replication endpoints
...
This will hopefully reduce the boiler plate required to implement new
internal HTTP requests.
2018-07-31 14:32:20 +01:00
5de936caa1
Merge pull request #3612 from matrix-org/rav/store_heirarchy
...
Make EventStore inherit from EventFederationStore
2018-07-31 13:44:04 +01:00
df2235e7fa
coding style
2018-07-31 13:16:20 +01:00
0bc9b9e397
reinstate explicit include of EventsWorkerStore
2018-07-31 13:11:04 +01:00
e908b86832
Remove pdu_failures from transactions
...
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
2018-07-30 16:28:47 -06:00
254e8267e2
Only import secrets when available
...
secrets got introduced in python 3.6 so this class is not available
in 3.5 and before.
This now checks for the current running version and only tries using
secrets if the version is 3.6 or above
Signed-Off-By: Matthias Kesler <krombel@krombel.de>
2018-07-30 23:59:02 +02:00
21276ff846
remove errant logging
2018-07-30 22:42:12 +01:00
fef7e58ac6
actually close conn
2018-07-30 22:29:44 +01:00
9b13817e06
factor out metrics from __init__ to app/homeserver
2018-07-30 22:07:07 +01:00
251e6c1210
limit register and sign in on number of monthly users
2018-07-30 15:55:57 +01:00
143f1a2532
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_fed_store
2018-07-30 09:56:18 +01:00
2903e65aff
fix isort
2018-07-29 19:47:08 +02:00
a8cbce0ced
fix invalidation
2018-07-27 16:17:17 +01:00
e9b2d047f6
make /context lazyload & filter aware ( #3567 )
...
make /context lazyload & filter aware.
2018-07-27 15:12:50 +01:00
f102c05856
Rewrite cache list decorator
...
Because it was complicated and annoyed me. I suspect this will be more
efficient too.
2018-07-27 13:47:04 +01:00
8e3f75b39a
fix accidental removal of hs
2018-07-27 12:17:31 +02:00
b0b5566f36
Merge pull request #3391 from t3chguy/t3chguy/default_inviter_display_name_3pid
...
if inviter_display_name == ""||None then default to inviter MXID
2018-07-27 10:08:39 +01:00
7041cd872b
Merge branch 'develop' into send_sni_for_federation_requests
2018-07-27 09:17:11 +01:00
d42455cdc9
Merge pull request #3616 from matrix-org/travis/event_id_send_leave
...
Update the send_leave path to be an event_id
2018-07-26 23:25:17 +01:00
65c8dee900
Merge pull request #3614 from matrix-org/rav/stop_populating_event_content
...
Stop populating events.content
2018-07-26 22:54:08 +01:00
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
9e68b1bd2d
Merge pull request #3613 from matrix-org/rav/stop_using_event_edges_room_id
...
Remove some redundant joins on event_edges.room_id
2018-07-26 22:31:01 +01:00
7d32f0d745
Update the send_leave path to be an event_id
...
It's still not used, however the parameter is an event ID not a transaction ID.
2018-07-26 14:41:59 -06:00
ef9d51b081
Merge pull request #3610 from matrix-org/rav/fix_looping_calls
...
Fix some looping_call calls which were broken in #3604
2018-07-26 14:55:57 +01:00
51d7df1915
Create the column nullable
...
There's no real point in ever making the column non-nullable, and doing so
breaks the sytests.
2018-07-26 14:54:04 +01:00
5c1d301fd9
Stop populating events.content
...
This field is no longer read from, so we should stop populating it. Once we're
happy that this doesn't break everything, and a rollback is unlikely, we can
think about dropping the column.
2018-07-26 14:43:02 +01:00
bd4b25f4d0
Remove some redundant joins on event_edges.room_id
...
We've long passed the point where it's possible to have the same event_id in
different tables, so these join conditions are redundant: we can just join on
event_id.
event_edges is of non-trivial size, and the room_id column is wasteful, so
let's stop reading from it. In future, we can stop writing to it, and then drop
it.
2018-07-26 13:19:08 +01:00
1b4d73fa52
comment on event_edges
2018-07-26 12:53:51 +01:00
21e878ebb6
Make EventStore inherit from EventFederationStore
...
(since it uses methods therein)
Turns out that we had a bunch of things which were incorrectly importing
EventWorkerStore from events.py rather than events_worker.py, which broke once
I removed the import into events.py.
2018-07-26 12:48:51 +01:00
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
1bcd0490c2
Merge pull request #2970 from matrix-org/matthew/filter_members
...
Implement the lazy_load_members room state filter parameter
2018-07-26 00:03:01 +01:00
bc7944e6d2
switch missing_types to be a bool
2018-07-25 23:36:31 +01:00
d8e65ed7e1
Fix a minor documentation typo in on_make_leave
2018-07-25 15:44:41 -06:00
2565804030
Merge branch 'develop' into matthew/filter_members
2018-07-25 17:27:49 +01:00
7d9fb88617
incorporate more review.
2018-07-25 16:33:50 +01:00
78a691d005
Split out DB writes in federation handler
...
This will allow us to easily add an internal replication API to proxy
these reqeusts to master, so that we can move federation APIs to
workers.
2018-07-25 16:22:56 +01:00
3849f7f69f
Merge pull request #3603 from matrix-org/erikj/handle_outliers
...
Correctly handle outliers during persist events
2018-07-25 13:24:04 +01:00
cee1ae1b72
Merge pull request #3606 from matrix-org/rav/logcontext_fixes_once_more
...
Fix another logcontext leak in _persist_events
2018-07-25 11:56:00 +01:00
32b30e15f2
Merge pull request #3607 from matrix-org/rav/fix_persist_events_integrity_error
...
Fix occasional 'tuple index out of range' error
2018-07-25 11:55:45 +01:00
4081bd1560
Merge pull request #3605 from matrix-org/rav/fix_update_remote_profile_cache
...
Fix updating of cached remote profiles
2018-07-25 11:54:46 +01:00
7780a7b47c
Actually fix it by adding continue
2018-07-25 11:13:20 +01:00
1be94440d3
Fix occasional 'tuple index out of range' error
...
This fixes a bug in _delete_existing_rows_txn which was introduced in #3435
(though it's been on matrix-org-hotfixes for *years*). This code is only called
when there is some sort of conflict the first time we try to persist an event,
so it only happens rarely. Still, the exceptions are annoying.
2018-07-25 11:05:58 +01:00
07defd5fe6
Fix another logcontext leak in _persist_events
...
We need to run the errback in the sentinel context to avoid losing our own
context.
Also: add logging to runInteraction to help identify where "Starting db
connection from sentinel context" warnings are coming from
2018-07-25 10:53:23 +01:00
55acd6856c
Fix updating of cached remote profiles
...
_update_remote_profile_cache was missing its `defer.inlineCallbacks`, so when
it was called, would just return a generator object, without actually running
any of the method body.
2018-07-25 10:34:48 +01:00
f59be4eb0e
Fix unit tests
...
on_notifier_poke no longer runs synchonously, so we have to do a different hack
to make sure that the replication data has been sent. Let's actually listen for
its arrival.
2018-07-25 10:30:36 +01:00
a297ff2b16
Fix typo in conditional
2018-07-25 09:48:01 +01:00
371da42ae4
Wrap a number of things that run in the background
...
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
2018-07-25 09:41:12 +01:00
ec56121b0d
Correctly handle outliers during persist events
...
We incorrectly asserted that all contexts must have a non None state
group without consider outliers. This would usually be fine as the
assertion would never be hit, as there is a shortcut during persistence
if the forward extremities don't change.
However, if the outlier is being persisted with non-outlier events, the
function would be called and the assertion would be hit.
Fixes #3601
2018-07-25 09:35:02 +01:00
cb5c37a57c
handle the edge case for _get_some_state_from_cache where types is []
2018-07-24 20:34:45 +01:00
87951d3891
Merge branch 'develop' of github.com:matrix-org/synapse into t3chguy/default_inviter_display_name_3pid
2018-07-24 17:17:46 +01:00
8b8c4f34a3
Replace usage of get_current_toke with StreamToken.START
...
This allows us to handle /context/ requests on the client_reader worker
without having to pull in all the various stream handlers (e.g.
precence, typing, pushers etc). The only thing the token gets used for
is pagination, and that ignores everything but the room portion of the
token.
2018-07-24 16:49:17 +01:00
3188973857
Pull out did_forget to worker store
2018-07-24 16:49:14 +01:00
60a1d147a7
Merge pull request #3595 from matrix-org/erikj/use_deltas
...
Use deltas to calculate current state deltas
2018-07-24 15:41:18 +01:00
709c309b0e
Expand on docstring comment about return value
2018-07-24 15:12:50 +01:00
8f65ab98d2
Remove unnecessary iteritems
2018-07-24 15:08:01 +01:00
ed0dd68731
Fixup comment (and indent)
2018-07-24 14:31:38 +01:00
811ac73a42
Don't fetch state from the database unless needed
2018-07-24 14:18:23 +01:00
a79410e7b8
Have _get_new_state_after_events return delta
...
If we have a delta from the existing to new current state, then we can
reuse that rather than manually working it out by fetching both lots of
state.
2018-07-24 14:14:17 +01:00
81946db9cf
Merge pull request #3587 from matrix-org/rav/better_exception_logging
...
Improve logging for exceptions when handling PDUs
2018-07-24 14:12:12 +01:00
a321f78991
Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groups
...
Fixes and optimisations for resolve_state_groups
2018-07-24 14:11:45 +01:00
93b0722c50
Merge pull request #3583 from matrix-org/rav/remove_who_forgot_in_room
...
Remove redundant checks on room forgottenness
2018-07-24 14:11:11 +01:00
454f59b7ad
Merge branch 'develop' into matthew/filter_members
2018-07-24 14:03:37 +01:00
1a01a5b964
clarify comment on p_ids
2018-07-24 14:03:15 +01:00
223341205e
Don't require to_delete to have event_ids
2018-07-24 14:02:40 +01:00
e22700c3dd
consider non-filter_type types as wildcards, thus missing from the state-group-cache
2018-07-24 13:59:07 +01:00
eb1d911ab7
rather than adding ll_ids, remove them from p_ids
2018-07-24 13:40:49 +01:00
d19fba3655
Merge branch 'develop' into matthew/filter_members
2018-07-24 12:39:54 +01:00
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
30bfed5aa5
Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room
2018-07-24 11:46:09 +01:00
97acd385a3
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_calculate_state_delta
2018-07-24 11:32:13 +01:00
0fa73e4a63
Remove unnecessary if
2018-07-24 11:19:23 +01:00
ff5426f6b8
Speed up _calculate_state_delta
2018-07-24 10:55:11 +01:00
a678145010
Merge branch 'develop' into rav/logcontext_fixes
2018-07-24 10:43:30 +01:00
d436ad332c
Merge pull request #3555 from matrix-org/erikj/client_apis_move
...
Make client_reader support some more read only APIs
2018-07-24 10:42:28 +01:00
536bc63a4e
Merge branch 'develop' into erikj/client_apis_move
2018-07-24 09:57:05 +01:00
cf2d15c6a9
another couple of logcontext leaks
2018-07-24 00:57:48 +01:00
8dff6e0322
Logcontext fixes
...
Fix some random logcontext leaks.
2018-07-24 00:37:17 +01:00
69fb5dbdab
fix idiocy
2018-07-24 00:04:44 +01:00
1938cffaea
Add some measure blocks to persist_events
...
... to help us figure out where 40% of CPU is going
2018-07-23 23:48:19 +01:00
efcdacad7d
handle case where types is [] on postgres correctly
2018-07-23 22:41:05 +01:00
ce0c18dec5
Improve logging for exceptions handling PDUs
...
when we get an exception handling a federation PDU, log the whole stacktrace.
2018-07-23 22:13:19 +01:00
c1f80effbe
Handle delta_ids being None in _update_context_for_auth_events
...
it's easier to create the new state group as a delta from the existing one.
(There's an outside chance this will help with
https://github.com/matrix-org/synapse/issues/3364 )
2018-07-23 22:06:50 +01:00
adfe29ec0b
Merge branch 'develop' into matthew/filter_members
2018-07-23 19:21:37 +01:00
254fb430d1
incorporate review
2018-07-23 19:21:20 +01:00
5c705f70c9
Fixes and optimisations for resolve_state_groups
...
First of all, fix the logic which looks for identical input state groups so
that we actually use them. This turned out to be most easily done by factoring
the relevant code out to a separate function so that we could do an early
return.
Secondly, avoid building the whole `conflicted_state` dict (which was only ever
used as a boolean flag).
Thirdly, replace the construction of the `state` dict (which mapped from keys
to events that set them), with an optimistic construction of the resolution
result assuming there will be no conflicts. This should be no slower than
building the old `state` dict, and:
- in the conflicted case, we'll short-cut it, saving part of the work
- in the unconflicted case, it saves rebuilding the resolution from the
`state` dict.
Finally, do a couple of s/values/itervalues/.
2018-07-23 19:10:11 +01:00
8b9f164fff
Comments
2018-07-23 17:43:01 +01:00
50c60e5fad
Only get cached state from context in persist_event
...
We don't want to bother pulling out the current state from the DB since
until we know we have to. Checking the context for state is just an
optimisation.
2018-07-23 17:21:40 +01:00
4f5cc8e4e7
Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room
2018-07-23 17:15:12 +01:00
dae6dc1e77
Remove redundant checks on room forgottenness
...
Fixes #3550
2018-07-23 17:13:34 +01:00
0faa3223cd
Fix missing attributes on workers.
...
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
999bcf9d01
Fix EventContext when using workers
...
We were:
1. Not correctly setting all attributes
2. Using defer.inlineCallbacks in a non-generator
2018-07-23 15:24:21 +01:00
4797ed000e
Update docstrings to make sense
2018-07-23 15:05:56 +01:00
0b0b24cb82
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move
2018-07-23 13:21:15 +01:00
027bc01a1b
Add support for updating state
2018-07-23 13:17:25 +01:00
e42510ba63
Use new getters
2018-07-23 13:17:22 +01:00
440b8845b5
Make EventContext lazy load state
2018-07-23 12:56:56 +01:00
842cdece42
pep8
2018-07-23 12:31:35 +01:00
acbfdc3442
Refcator EventContext to accept state during init
2018-07-23 12:20:55 +01:00
9b34f3ea3a
Merge branch 'develop' into matthew/sync_deleted_devices
2018-07-23 10:03:28 +01:00
c1bf2b587e
add trailing comma
2018-07-23 09:56:23 +01:00
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
5c88bb722f
Move PaginationHandler to its own file
2018-07-20 15:32:23 +01:00
0ecf68aedc
Move check_in_room_or_world_readable to Auth
2018-07-20 15:30:59 +01:00
3d6df84658
Test and fix support for cancellation in Linearizer
2018-07-20 13:59:55 +01:00
4f67623674
Merge pull request #3571 from matrix-org/rav/limiter_fixes
...
A set of improvements to the Limiter
2018-07-20 13:53:03 +01:00
e1a237eaab
Admin API for creating new users ( #3415 )
2018-07-20 22:41:13 +10:00
7c712f95bb
Combine Limiter and Linearizer
...
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
2018-07-20 13:11:43 +01:00
8462c26485
Improvements to the Limiter
...
* give them names, to improve logging
* use a deque rather than a list for efficiency
2018-07-20 12:50:27 +01:00
d7275eecf3
Add a sleep to the Limiter to fix stack overflows.
...
Fixes #3570
2018-07-20 12:37:12 +01:00
2f558300cc
fix thinkos; unbreak tests
2018-07-19 19:22:27 +01:00
bcaec2915a
incorporate review
2018-07-19 19:03:50 +01:00
924eb34d94
add a filtered_types param to limit filtering to specific types
2018-07-19 18:32:02 +01:00
5f3658baf5
Merge pull request #3377 from Valodim/note-affinity
...
document that the affinity package is required for the cpu_affinity setting
2018-07-19 14:35:06 +01:00
9a8acdc720
Merge branch 'master' into develop
2018-07-19 21:16:44 +10:00
38f53399a2
0.33 final
2018-07-19 21:11:40 +10:00
95ccb6e2ec
Don't spew errors because we can't save metrics ( #3563 )
2018-07-19 20:58:18 +10:00
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
9e40834f74
yes, we do need to invalidate the device_id_exists_cache when deleting a remote device
2018-07-19 11:15:10 +01:00
f1a15ea206
revert 00bc979
...
... we've fixed the things that caused the warnings, so we should reinstate the
warning.
2018-07-19 11:14:20 +01:00
1ffb7bec20
Merge remote-tracking branch 'origin/release-v0.33.0' into develop
2018-07-19 11:12:33 +01:00
c754e006f4
Merge pull request #3556 from matrix-org/rav/background_processes
...
Run things as background processes
2018-07-19 11:04:18 +01:00
a97c845271
Move v1-only APIs into their own module & isolate deprecated ones ( #3460 )
2018-07-19 20:03:33 +10:00
c0685f67c0
spell out that include_deleted_devices requires include_all_devices
2018-07-19 10:59:02 +01:00
00bc979137
Disable logcontext warning
...
Temporary workaround to #3518 while we release 0.33.0.
2018-07-19 10:51:15 +01:00
6f62a6ef21
Merge pull request #3554 from matrix-org/erikj/response_metrics_code
...
Add response code to response timer metrics
2018-07-19 10:25:18 +01:00
8c69b735e3
Make Distributor run its processes as a background process
...
This is more involved than it might otherwise be, because the current
implementation just drops its logcontexts and runs everything in the sentinel
context.
It turns out that we aren't actually using a bunch of the functionality here
(notably suppress_failures and the fact that Distributor.fire returns a
deferred), so the easiest way to fix this is actually by simplifying a bunch of
code.
2018-07-18 20:55:05 +01:00
667fba68f3
Run things as background processes
...
This fixes #3518 , and ensures that we get useful logs and metrics for lots of
things that happen in the background.
(There are certainly more things that happen in the background; these are just
the common ones I've found running a single-process synapse locally).
2018-07-18 20:55:05 +01:00
9b596177ae
Add some room read only APIs to client_reader
2018-07-18 15:33:03 +01:00
bacdf0cbf9
Move RoomContextHandler out of Handlers
...
This is in preparation for moving GET /context/ to a worker
2018-07-18 15:33:03 +01:00
8cb8df55e9
Split MessageHandler into read only and writers
...
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
2018-07-18 15:33:03 +01:00
65d6a0e477
Merge pull request #3553 from matrix-org/rav/background_process_tracking
...
Resource tracking for background processes
2018-07-18 14:27:50 +01:00
5bd0a47fcd
pep8
2018-07-18 14:19:00 +01:00
e45a46b6e4
Add response code to response timer metrics
2018-07-18 13:59:36 +01:00
dab00faa83
Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacks
...
Remove event re-signing hacks
2018-07-18 12:46:27 +01:00
c91a44572e
Merge pull request #3514 from matrix-org/dbkr/turn_dont_add_defaults
...
Comment dummy TURN parameters in default config
2018-07-18 11:56:18 +01:00
6e3fc657b4
Resource tracking for background processes
...
This introduces a mechanism for tracking resource usage by background
processes, along with an example of how it will be used.
This will help address #3518 , but more importantly will give us better insights
into things which are happening but not being shown up by the request metrics.
We *could* do this with Measure blocks, but:
- I think having them pulled out as a completely separate metric class will
make it easier to distinguish top-level processes from those which are
nested.
- I want to be able to report on in-flight background processes, and I don't
think we want to do this for *all* Measure blocks.
2018-07-18 10:50:33 +01:00
5f3d02f6eb
bump to 0.33.0rc1
2018-07-18 12:52:56 +10:00
79eb339c66
add a comment
2018-07-17 14:53:34 +01:00
d897be6a98
Fix visibility of events from erased users over federation
2018-07-17 14:02:07 +01:00
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
...
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
94440ae994
fix imports
2018-07-17 11:51:26 +01:00
bc006b3c9d
Refactor REST API tests to use explicit reactors ( #3351 )
2018-07-17 20:43:18 +10:00
2172a3d8cb
add a comment
2018-07-17 11:13:57 +01:00
b2aa05a8d6
Use efficient .intersection
2018-07-17 11:07:04 +01:00
547b1355d3
Fix perf regression in PR #3530
...
The get_entities_changed function was changed to return all changed
entities since the given stream position, rather than only those changed
from a given list of entities. This resulted in the function incorrectly
returning large numbers of entities that, for example, caused large
increases in database usage.
2018-07-17 10:27:51 +01:00
3fe0938b76
Merge pull request #3530 from matrix-org/erikj/stream_cache
...
Don't return unknown entities in get_entities_changed
2018-07-17 13:44:46 +10:00
09e29fb58b
Attempt to make _filter_events_for_server more efficient
2018-07-16 14:06:09 +01:00
78a9ddcf9a
rerun isort with latest version
2018-07-16 14:23:25 +02:00
ea69d35651
Move filter_events_for_server out of FederationHandler
...
for easier unit testing.
2018-07-16 13:06:24 +01:00
4a27000548
check isort by travis
2018-07-16 13:57:33 +02:00
Richard van der Hoff