Commit Graph

12276 Commits

Author SHA1 Message Date
Richard van der Hoff 484a9b8c81 Remove redundant, failing, test
This test didn't do what it claimed to do, and what it claimed to do was the
same as test_cant_hide_direct_ancestors anyway.

This stuff is tested by sytest anyway.
2018-09-27 13:11:23 +01:00
Schnuffle a873896096 Added changelog fragment 2018-09-27 14:01:44 +02:00
Schnuffle dc5db01ff2 Replaced all occurences of e.message with str(e)
Signed-off-by: Schnuffle  <schnuffle@github.com>
2018-09-27 13:38:50 +02:00
Richard van der Hoff 51d33d5178
Merge pull request #3961 from matrix-org/neilj/lock_mau_upserts
fix #3854 MAU transaction errors
2018-09-27 12:31:37 +01:00
Michael Kaye 74bbdd0412 Do the changelog.d dance 2018-09-27 12:01:43 +01:00
Michael Kaye 8607397ea7 Make a ":latest" tag, and a SHA1 commit ID one too.
Latest is horrible and makes debugging what has happened anywhere a
nightmare. We push a latest because of demand for it, but we'll also
push a SHA1 commit id so those wanting to know what they're running
(and be able to roll back if required) can use those instead.

Note that latest here is defined as "most recent master commit" not
"most recent released version", as the actual semantics of making latest
correct while still being able to build bugfixed releases of previous
versions is just ARGH. So we define it as "master" not "latest release".
2018-09-27 12:01:41 +01:00
Michael Kaye a3c11f7320 Also, don't run this job on any branches 2018-09-27 12:01:09 +01:00
Michael Kaye 73a089f461 Make username configurable in the UI too 2018-09-27 12:00:40 +01:00
Michael Kaye 3852c2bc39 Fix typo 2018-09-27 12:00:01 +01:00
Michael Kaye 0d36fe3563 Build and push docker image to hub 2018-09-27 11:59:17 +01:00
Richard van der Hoff 948a6d8776 changelog 2018-09-27 11:37:39 +01:00
Richard van der Hoff 333bee27f5 Include event when resolving state for missing prevs
If we have a forward extremity for a room as `E`, and you receive `A`, `B`,
s.t. `A -> B -> E`, and `B` also points to an unknown event `X`, then we need
to do state res between `X` and `E`.

When that happens, we need to make sure we include `X` in the state that goes
into the state res alg.

Fixes #3934.
2018-09-27 11:37:39 +01:00
Richard van der Hoff bd61c82bdf Include state from remote servers in pdu handling
If we've fetched state events from remote servers in order to resolve the state
for a new event, we need to actually pass those events into
resolve_events_with_factory (so that it can do the state res) and then persist
the ones we need - otherwise other bits of the codebase get confused about why
we have state groups pointing to non-existent events.
2018-09-27 11:37:39 +01:00
Richard van der Hoff a215b698c4 Fix "unhashable type: 'list'" exception in federation handling
get_state_groups returns a map from state_group_id to a list of FrozenEvents,
so was very much the wrong thing to be putting as one of the entries in the
list passed to resolve_events_with_factory (which expects maps from
(event_type, state_key) to event id).

We actually want get_state_groups_ids().values() rather than
get_state_groups().

This fixes the main problem in #3923, but there are other problems with this
bit of code which get discovered once you do so.
2018-09-27 11:37:39 +01:00
Richard van der Hoff 28223841e0 more comments 2018-09-27 11:31:51 +01:00
Richard van der Hoff ad8e137062 changelog 2018-09-27 11:31:51 +01:00
Richard van der Hoff e3c159863d Clarifications in FederationHandler
* add some comments on things that look a bit bogus
* rename this `state` variable to avoid confusion with the `state` used
  elsewhere in this function. (There was no actual conflict, but it was
  a confusing bit of spaghetti.)
2018-09-27 11:31:51 +01:00
Richard van der Hoff 92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2
Logging improvements
2018-09-27 11:27:46 +01:00
Richard van der Hoff 4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions (#3959)
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.

This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
2018-09-27 11:25:34 +01:00
Richard van der Hoff ae6ad4cf41
docstrings and unittests for storage.state (#3958)
I spent ages trying to figure out how I was going mad...
2018-09-27 11:22:25 +01:00
Amber Brown 2c695fd1aa
Merge pull request #3963 from matrix-org/rav/get_state_for_room_docstring
fix docstring for FederationClient.get_state_for_room
2018-09-27 18:06:38 +10:00
Richard van der Hoff e70b4ce069 Logging improvements
Some logging tweaks to help with debugging incoming federation transactions
2018-09-26 17:36:14 +01:00
Richard van der Hoff 3c37c7e45b changelog 2018-09-26 17:01:30 +01:00
Richard van der Hoff a87d419a85 Run notify_app_services as a bg process
This ensures that its resource usage metrics get recorded somewhere rather than
getting lost.

(It also fixes an error when called from a nested logging context which
completes before the bg process)
2018-09-26 17:00:40 +01:00
Richard van der Hoff 0c4a99ea2d changelog 2018-09-26 16:54:54 +01:00
Richard van der Hoff 9453c65948 remove spurious federation checks on localhost
There's really no point in checking for destinations called "localhost" because
there is nothing stopping people creating other DNS entries which point to
127.0.0.1. The right fix for this is
https://github.com/matrix-org/synapse/issues/3953.

Blocking localhost, on the other hand, means that you get a surprise when
trying to connect a test server on localhost to an existing server (with a
'normal' server_name).
2018-09-26 16:53:52 +01:00
Richard van der Hoff ab59f3d8da changelog 2018-09-26 16:52:24 +01:00
Richard van der Hoff 607eec0456 fix docstring for FederationClient.get_state_for_room
trivial fixes for docstring
2018-09-26 16:52:24 +01:00
Neil Johnson d514608b5c towncrier 2018-09-26 16:19:53 +01:00
Neil Johnson 28781b65e7 fix #3854 2018-09-26 16:16:41 +01:00
Amber Brown d4e0861ff9
Reduce the load on our CI (#3957)
* changelog

* reduce circleci config

* plus a handy script

* fix regex
2018-09-27 00:23:21 +10:00
Richard van der Hoff a5e70b31a1 changelog 2018-09-26 14:41:14 +01:00
Richard van der Hoff 8afddf7afe Fix error handling for missing auth_event
When we were authorizing an event, if there was no `m.room.create` in its
auth_events, we would raise a SynapseError with a cryptic message, which then
meant that we would bail out of processing any incoming events, rather than
storing a rejection for the faulty event and moving on.

We should treat the absent event the same as any other auth failure, by
raising an AuthError, so that the event is marked as rejected.
2018-09-26 14:40:16 +01:00
Richard van der Hoff bf01efb864 Merge branch 'develop' into rav/hacky_cache_factor_fix 2018-09-26 13:24:07 +01:00
Erik Johnston 8834396b8a Actuall set cache factors in workers 2018-09-26 13:23:02 +01:00
Richard van der Hoff 4e8276a34a
Merge pull request #3956 from matrix-org/rav/fix_expiring_cache_len
Fix ExpiringCache.__len__ to be accurate
2018-09-26 13:10:13 +01:00
Richard van der Hoff 5b4028fa78 Merge branch 'rav/fix_expiring_cache_len' into erikj/destination_retry_cache 2018-09-26 12:55:53 +01:00
Richard van der Hoff 7ee94fc1ba Log which cache is throwing exceptions 2018-09-26 12:43:08 +01:00
Amber Brown 66a1d57adb
Merge pull request #3948 from matrix-org/rav/no_symlink_synctl
Move synctl into top dir to avoid a symlink
2018-09-26 21:41:58 +10:00
Amber Brown c2185f14d7
Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pdu
Comments and interface cleanup for on_receive_pdu
2018-09-26 21:41:26 +10:00
Richard van der Hoff 8ac9fa7375 changelog 2018-09-26 12:35:10 +01:00
Erik Johnston 3baf6e1667 Fix ExpiringCache.__len__ to be accurate
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.

(This appears to have been a longstanding bug, but one which has been made more
of a problem by #3932 and #3933).

(This was originally done by Erik as part of #3933. I'm cherry-picking it
because really it's a fix in its own right)
2018-09-26 12:32:29 +01:00
Richard van der Hoff f65163627f
Merge pull request #3911 from matrix-org/jcgruenhage/docker-support-python3
make python 3 work in the docker container
2018-09-25 15:18:09 +01:00
Jan Christian Grünhage df55a943ca
Update Dockerfile 2018-09-25 14:33:38 +02:00
Jan Christian Grünhage e7fa032126
Update .dockerignore 2018-09-25 14:33:19 +02:00
Richard van der Hoff 0649306fde Update grafana dashboard 2018-09-25 13:29:28 +01:00
Richard van der Hoff a1cd37390f Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-25 12:03:54 +01:00
Richard van der Hoff 4c3e7eeec5
Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches
Fix some instances of ExpiringCache not expiring cache items
2018-09-25 12:02:57 +01:00
Jérémy Farnaud 6cf261930a added "media-src: 'self'" to CSP for resources (#3578)
Synapse doesn’t allow for media resources to be played directly from
Chrome. It is a problem for users on other networks (e.g. IRC)
communicating with Matrix users through a gateway. The gateway sends
them the raw URL for the resource when a Matrix user uploads a video
and the video cannot be played directly in Chrome using that URL.

Chrome argues it is not authorized to play the video because of the
Content Security Policy. Chrome checks for the "media-src" policy which
is missing, and defauts to the "default-src" policy which is "none".

As Synapse already sends "object-src: 'self'" I thought it wouldn’t be
a problem to add "media-src: 'self'" to the CSP to fix this problem.
2018-09-25 11:55:02 +01:00
Richard van der Hoff 94f7befc31
Merge pull request #3925 from matrix-org/erikj/fix_producers_unregistered
Fix spurious exceptions when client closes conncetion
2018-09-25 11:52:06 +01:00