c53336986d
Move synctl into top dir to avoid a symlink
...
symlinks apparently break setuptools on python3 and alpine
(https://bugs.python.org/issue31940 ), so let's stop using a symlink and just
use the file directly.
2018-09-25 11:19:27 +01:00
a9d84f4e44
We require attrs 16.0.0
...
Ref: https://github.com/matrix-org/synapse/issues/3945
2018-09-25 10:43:39 +01:00
787d22ed6c
Only lazy load self-members on initial sync
...
Given we have disabled lazy loading for incr syncs in #3840 , we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
2018-09-25 00:49:26 +01:00
fbe5ba25f6
Merge branch 'master' into develop
2018-09-25 03:10:01 +10:00
6b6cb32297
bump version
2018-09-25 02:54:34 +10:00
04eed80a73
Merge branch 'master' into develop
2018-09-24 23:42:25 +10:00
e302f40e20
update version
2018-09-24 23:40:05 +10:00
19dc676d1a
Fix ExpiringCache.__len__ to be accurate
...
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.
2018-09-21 16:25:42 +01:00
fdd1a62e8d
Add a five minute cache to get_destination_retry_timings
...
Hopefully helps with #3931
2018-09-21 14:56:12 +01:00
79eded1ae4
Make ExpiringCache slightly more performant
2018-09-21 14:52:21 +01:00
8601c24287
Fix some instances of ExpiringCache not expiring cache items
...
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
2018-09-21 14:19:46 +01:00
ad53a5497d
Merge pull request #3927 from matrix-org/erikj/handle_background_errors
...
Handle exceptions thrown by background tasks
2018-09-21 09:26:30 +01:00
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients ( #3916 )
...
Should fix https://github.com/vector-im/riot-web/issues/7209
2018-09-20 21:21:54 +01:00
94ae1dea3c
Add missing logger
2018-09-20 17:05:34 +01:00
9ea408441f
Handle exceptions thrown by background tasks
...
Fixes #3921
2018-09-20 16:15:21 +01:00
b28a7ed503
Fix spurious exceptions when client closes conncetion
...
If a HTTP handler throws an exception while processing a request we
automatically write a JSON error response. If the handler had already
started writing a response twisted throws an exception.
We should check for this case and simple abort the connection if there
was an error after the response had started being written.
2018-09-20 13:44:20 +01:00
23b53b4ef8
Merge pull request #3868 from matrix-org/neilj/fix_room_invite_mail_links
...
Neilj/fix room invite mail links
2018-09-20 13:32:38 +01:00
703de4ec13
Comments and interface cleanup for on_receive_pdu
...
Add some informative comments about what's going on here.
Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart
from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of
`get_missing` and use `sent_to_us_directly` consistently.
2018-09-20 13:06:55 +01:00
1f3f5fcf52
Fix client IPs being broken on Python 3 ( #3908 )
2018-09-20 20:14:34 +10:00
3fd68d533b
Merge pull request #3914 from matrix-org/erikj/remove_retry_cache
...
Remove get_destination_retry_timings cache
2018-09-20 10:54:49 +01:00
aeca5a5ed5
Add a regression test for logging on failed connections ( #3912 )
2018-09-20 16:28:18 +10:00
642199570c
Improve the logging when handling a federation transaction ( #3904 )
...
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:
- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
them into other, more cryptic, exceptions.
2018-09-19 17:28:18 +01:00
ce846bb620
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typing
2018-09-19 15:08:36 +01:00
bbab6ebfd9
Fix up changelog and remove spurious comment
2018-09-19 14:45:14 +01:00
392a54128c
pep8
2018-09-19 14:37:49 +01:00
b9158ac2bf
Remove get_destination_retry_timings cache
...
Currently we rely on the master to invalidate this cache promptly.
However, after having moved most federation endpoints off of master this
no longer happens, causing outbound fedeariont to get blackholed.
Fixes #3798
2018-09-19 14:22:57 +01:00
80d2d50f47
Fixup
2018-09-19 11:19:47 +01:00
9407bcf37a
Replace custom DeferredTimeoutError with defer.TimeoutError
2018-09-19 11:07:29 +01:00
6c48aa0256
Run canceller first to allow it to generate correct error
2018-09-19 11:07:27 +01:00
a334e1cace
Update to use new timeout function everywhere.
...
The existing deferred timeout helper function (and the one into twisted)
suffer from a bug when a deferred's canceller throws an exception, #3842 .
The new helper function doesn't suffer from this problem.
2018-09-19 10:39:40 +01:00
47c02e6332
Merge pull request #3909 from turt2live/travis/fix-logging-1
...
Fix matrixfederationclient.py logging: Destination is a string
2018-09-19 18:14:47 +10:00
3d6b24fb1b
Merge pull request #3907 from matrix-org/rav/set_sni_to_server_name
...
Set SNI to the server_name, not whatever was in the SRV record
2018-09-19 17:59:33 +10:00
f773ecbd61
Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeout
...
Bump timeout on get_missing_events request
2018-09-19 17:57:48 +10:00
35aec19f0a
Destination is a string
2018-09-18 15:29:30 -06:00
38ead946a9
Merge remote-tracking branch 'origin/develop' into neilj/fix_room_invite_mail_links
2018-09-18 19:02:45 +01:00
a219ce8726
Use directory server for room joins ( #3899 )
...
When we do a join, always try the server we used for the alias lookup first.
Fixes #2418
2018-09-18 18:27:37 +01:00
31c15dcb80
Refactor matrixfederationclient to fix logging ( #3906 )
...
We want to wait until we have read the response body before we log the request
as complete, otherwise a confusing thing happens where the request appears to
have completed, but we later fail it.
To do this, we factor the salient details of a request out to a separate
object, which can then keep track of the txn_id, so that it can be logged.
2018-09-18 18:17:15 +01:00
c600886d47
Merge pull request #3894 from matrix-org/hs/phone_home_py_version
...
Add python_version phone home stat
2018-09-19 02:40:04 +10:00
b3097396e7
Set SNI to the server_name, not whatever was in the SRV record
...
Fixes #3843
2018-09-18 17:01:12 +01:00
550007cb0e
Bump timeout on get_missing_events request
2018-09-18 15:02:51 +01:00
286d6930b7
Merge pull request #3879 from matrix-org/matthew/fix-autojoin
...
don't ratelimit autojoins
2018-09-18 13:07:01 +01:00
1e09a1d48a
Merge pull request #3889 from matrix-org/rav/404_on_remove_unknown_alias
...
Return a 404 when deleting unknown room alias
2018-09-18 12:59:30 +01:00
5baa087312
typo
2018-09-17 17:37:56 +01:00
b58714789f
make pip happy?
2018-09-17 17:35:54 +01:00
ac80cb08fe
Fix more b'abcd' noise in metrics
2018-09-17 17:16:50 +01:00
9a1cceeca9
Use a string for versions
2018-09-17 17:09:06 +01:00
f75b9961c6
Reinstate missing null check
2018-09-17 16:52:02 +01:00
2b39494cd5
Add python_version phone home stat
2018-09-17 16:35:18 +01:00
f00a9d2636
Fix some b'abcd' noise in logs and metrics
...
Python 3 compatibility: make sure that we decode some byte sequences before we
use them to create log lines and metrics labels.
2018-09-17 16:15:42 +01:00
fe88907d04
version
2018-09-17 22:33:22 +10:00
85a43f4167
Return a 404 when deleting unknown room alias
...
As per https://github.com/matrix-org/matrix-doc/issues/1675
Fixes https://github.com/matrix-org/synapse/issues/2782
2018-09-17 13:19:00 +01:00
d42d79e3c3
don't ratelimit autojoins
2018-09-15 22:27:41 +01:00
24efb2a70d
Fix timeout function
...
Turns out deferred.cancel sometimes throws, so we do that last to ensure
that we always do resolve the new deferred.
2018-09-15 11:38:39 +01:00
fcfe7a850d
Add an awful secondary timeout to fix wedged requests
...
This is an attempt to mitigate #3842 by adding yet-another-timeout
2018-09-14 19:23:07 +01:00
024be6cf18
don't filter membership events based on history visibility ( #3874 )
...
don't filter membership events based on history visibility
as we will already have filtered the messages in the timeline, and state events
are always visible.
and because @erikjohnston said so.
2018-09-14 18:12:52 +01:00
3e6e94fe9f
Merge pull request #3872 from matrix-org/hawkowl/timeouts-2
...
timeouts 2: electric boogaloo
2018-09-14 16:58:44 +01:00
bc9af88a2d
fix
2018-09-15 00:26:00 +10:00
d0f6c1ce21
Remove spurious comment
2018-09-14 15:12:36 +01:00
9e2f9a7b57
Measure outbound requests
2018-09-14 15:11:26 +01:00
0a81038ea0
Add in flight real time metrics for Measure blocks
2018-09-14 15:08:37 +01:00
7de1989ea2
fix link for case that config.email_riot_base_url is set
2018-09-13 22:43:50 +01:00
c971aa7b9d
fix
2018-09-14 03:57:02 +10:00
8f08d848f5
fix
2018-09-14 03:53:56 +10:00
f1a7264663
Fix minor typo in exception
2018-09-13 11:51:12 -06:00
7c33ab76da
redact better
2018-09-14 03:45:34 +10:00
63755fa4c2
we do that higher up
2018-09-14 03:21:47 +10:00
73884ebac5
Merge remote-tracking branch 'origin/develop' into hawkowl/timeouts-2
2018-09-14 03:11:25 +10:00
7c27c4d51c
merge ( #3576 )
2018-09-14 03:11:11 +10:00
1c3f4d9ca5
buffer?
2018-09-14 03:09:13 +10:00
6c0f8d9d50
Merge pull request #3856 from matrix-org/erikj/speed_up_purge
...
Make purge history slightly faster
2018-09-13 16:14:46 +01:00
ed5331a627
comment
2018-09-13 16:10:56 +01:00
89a76d1889
Fix handling of redacted events from federation
...
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.
This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
2018-09-13 15:44:12 +01:00
bfa0b759e0
Attempt to figure out what's going on with timeouts ( #3857 )
2018-09-14 00:15:51 +10:00
9cbd0094f0
pep8
2018-09-13 15:15:35 +01:00
9dbe38ea7d
Create indices after insertion
2018-09-13 15:05:52 +01:00
93139a1fb8
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_purge
2018-09-13 12:57:09 +01:00
c857f5ef9b
Make purge history slightly faster
...
Don't pull out events that are outliers and won't be deleted, as nothing
should happen to them.
2018-09-13 12:48:10 +01:00
b7d2fb5eb9
Remove some superfluous logging ( #3855 )
2018-09-13 19:59:32 +10:00
f30a303590
Merge pull request #3846 from matrix-org/neilj/expose-registered-users
...
expose number of real reserved users
2018-09-12 17:14:04 +01:00
2ac1abbc7e
show heroes if a room has a 'deleted' name/canonical_alias ( #3851 )
2018-09-12 17:11:05 +01:00
fa0d464fa4
Merge pull request #3853 from matrix-org/erikj/log_outbound_each_time
...
Log outbound requests when we retry
2018-09-12 16:55:40 +01:00
0403cf0783
argh pep8
2018-09-12 16:54:28 +01:00
0e200e366d
correctly log gappy sync metrics
2018-09-12 16:47:20 +01:00
11bfc2af1c
fix logline
2018-09-12 16:45:42 +01:00
8decd6233d
improve naming
2018-09-12 16:22:15 +01:00
8c5b84441b
Log outbound requests when we retry
2018-09-12 16:22:14 +01:00
54f8616d2c
Merge pull request #3841 from matrix-org/erikj/manhole_key_length
...
Change the manhole SSH key to have more bits
2018-09-12 14:33:39 +01:00
7ca097f77e
Port federation/ to py3 ( #3847 )
2018-09-12 23:23:32 +10:00
0ddf486724
expose number of real reserved users
2018-09-12 11:58:52 +01:00
546aee7e52
Merge pull request #3835 from krombel/fix_3821
...
fix VOIP crashes under Python 3
2018-09-12 20:44:18 +10:00
33716c4aea
Merge pull request #3826 from matrix-org/rav/logging_for_keyring
...
add some logging for the keyring queue
2018-09-12 20:43:47 +10:00
bc635026c5
Merge pull request #3824 from matrix-org/rav/fix_jwt_import
...
Fix jwt import check
2018-09-12 20:41:57 +10:00
02aa41809b
Port rest/ to Python 3 ( #3823 )
2018-09-12 20:41:31 +10:00
8fd93b5eea
Port crypto/ to Python 3 ( #3822 )
2018-09-12 20:16:31 +10:00
4084a774a8
Timeout reading body for outbound HTTP requests
2018-09-12 10:10:20 +01:00
b041115415
Speed up lazy loading ( #3827 )
...
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats (#3840 )
2018-09-12 00:50:39 +01:00
9e05c8d309
Change the manhole SSH key to have more bits
...
Newer versions of openssh client refuse to connect to the old key due to
its length.
2018-09-11 10:42:10 +01:00
7bc22539ff
fix VOIP crashes under Python 3 ( #3821 )
2018-09-10 14:30:08 +02:00
9a5ea511b5
Merge pull request #3810 from matrix-org/erikj/send_tags_down_sync_on_join
...
Send existing room tags down sync on join
2018-09-07 23:28:42 +10:00
b8ad756bd0
Fix jwt import check
...
This handy code attempted to check that we could import jwt, but utterly failed
to check it was the right jwt.
Fixes https://github.com/matrix-org/synapse/issues/3793
2018-09-07 14:20:54 +01:00
Richard van der Hoff