Commit Graph

135 Commits

Author SHA1 Message Date
Jorik Schellekens cf2972c818
Fix servlet metric names (#5734)
* Fix servlet metric names

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Remove redundant check

* Cover all return paths
2019-07-24 13:07:35 +01:00
Richard van der Hoff fa8271c5ac
Convert synapse.federation.transport.server to async (#5689)
* Convert BaseFederationServlet._wrap to async

Empirically, this fixes some lost stacktraces. It should be safe because the
wrapped function is called from JsonResource._async_render, which is already
async.

* Convert the rest of synapse.federation.transport.server to async

We may as well do the whole file while we're here.

* changelog

* flake8
2019-07-18 11:46:47 +01:00
Jorik Schellekens 38a6d3eea7
Add basic opentracing support (#5544)
* Configure and initialise tracer

Includes config options for the tracer and sets up JaegerClient.

* Scope manager using LogContexts

We piggy-back our tracer scopes by using log context.
The current log context gives us the current scope. If new scope is
created we create a stack of scopes in the context.

* jaeger is a dependency now

* Carrier inject and extraction for Twisted Headers

* Trace federation requests on the way in and out.

The span is created in _started_processing and closed in
_finished_processing because we need a meaningful log context.

* Create logcontext for new scope.

Instead of having a stack of scopes in a logcontext we create a new
context for a new scope if the current logcontext already has a scope.

* Remove scope from logcontext if logcontext is top level

* Disable tracer if not configured

* typo

* Remove dependence on jaeger internals

* bools

* Set service name

* :Explicitely state that the tracer is disabled

* Black is the new black

* Newsfile

* Code style

* Use the new config setup.

* Generate config.

* Copyright

* Rename config to opentracing

* Remove user whitelisting

* Empty whitelist by default

* User ConfigError instead of RuntimeError

* Use isinstance

* Use tag constants for opentracing.

* Remove debug comment and no need to explicitely record error

* Two errors a "s(c)entry"

* Docstrings!

* Remove debugging brainslip

* Homeserver Whitlisting

* Better opentracing config comment

* linting

* Inclue worker name in service_name

* Make opentracing an optional dependency

* Neater config retreival

* Clean up dummy tags

* Instantiate tracing as object instead of global class

* Inlcude opentracing as a homeserver member.

* Thread opentracing to the request level

* Reference opetnracing through hs

* Instantiate dummy opentracin g for tests.

* About to revert, just keeping the unfinished changes just in case

* Revert back to global state, commit number:

9ce4a3d906

* Use class level methods in tracerutils

* Start and stop requests spans in a place where we
have access to the authenticated entity

* Seen it, isort it

* Make sure to close the active span.

* I'm getting black and blue from this.

* Logger formatting

Co-Authored-By: Erik Johnston <erik@matrix.org>

* Outdated comment

* Import opentracing at the top

* Return a contextmanager

* Start tracing client requests from the servlet

* Return noop context manager if not tracing

* Explicitely say that these are federation requests

* Include servlet name in client requests

* Use context manager

* Move opentracing to logging/

* Seen it, isort it again!

* Ignore twisted return exceptions on context exit

* Escape the scope

* Scopes should be entered to make them useful.

* Nicer decorator names

* Just one init, init?

* Don't need to close something that isn't open

* Docs make you smarter
2019-07-11 10:36:03 +01:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Brendan Abolivier bfe84e051e Split public rooms directory auth config in two 2019-06-24 15:42:31 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
Richard van der Hoff fec2dcb1a5
Enforce validity period on server_keys for fed requests. (#5321)
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.

We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
Amber Brown 46c8f7a517
Implement the SHHS complexity API (#5216) 2019-05-30 01:47:16 +10:00
Amber Brown f1e5b41388
Make all the rate limiting options more consistent (#5181) 2019-05-15 12:06:04 -05:00
Matthew Hodgson c0e0740bef add options to require an access_token to GET /profile and /publicRooms on CS API (#5083)
This commit adds two config options:

* `restrict_public_rooms_to_local_users`

Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.

* `require_auth_for_profile_requests`

When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.

MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.

Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
Richard van der Hoff 54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Andrew Morgan 4a125be138
Make federation endpoints more tolerant of trailing slashes v2 (#4935)
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
Erik Johnston 271cb1998b Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)"
This reverts commit 290552fd83.
2019-03-14 14:30:54 +00:00
Andrew Morgan 290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)
Server side of a solution towards #3622.
2019-03-11 17:44:03 +00:00
Andrew Morgan 336de1d45b Remove unnecessary dollar signs
A dollar sign is already appended to the end of each PATH, so there's
no need to add one in the PATH declaration as well.
2019-03-04 15:25:12 +00:00
Amber Brown b131cc77df
Make 'event_id' a required parameter in federated state requests (#4741)
* make 'event_id' a required parameter in federated state requests

As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40

Signed-off-by: Joseph Weston <joseph@weston.cloud>

* add changelog entry for bugfix

Signed-off-by: Joseph Weston <joseph@weston.cloud>

* Update server.py
2019-02-27 14:35:47 -08:00
Andrew Morgan 802884d4ee Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop 2019-02-26 14:23:40 +00:00
Andrew Morgan 7a4632af9c Prevent showing non-fed rooms in fed /publicRooms 2019-02-26 13:37:24 +00:00
Andrew Morgan c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms" 2019-02-26 13:20:38 +00:00
Erik Johnston 4b9e5076c4 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate 2019-02-25 15:08:18 +00:00
Andrew Morgan bd398b874e Don't restrict non-fed rooms over client APIs 2019-02-25 15:04:46 +00:00
Erik Johnston b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Erik Johnston a50cf929c1 Require event format version to parse or create events 2019-01-25 10:32:19 +00:00
Erik Johnston be6a7e47fa
Revert "Require event format version to parse or create events" 2019-01-25 10:23:51 +00:00
Erik Johnston f431ff3fb8 Require event format version to parse or create events 2019-01-23 20:21:33 +00:00
Jason Robinson d39b7b6d38 Document `servlet_groups` parameters
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson 82e13662c0 Split federation OpenID userinfo endpoint out of the federation resource
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.

The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Erik Johnston 4a4d2e17bc Add /v2/invite federation API 2019-01-15 13:22:44 +00:00
Erik Johnston bb63e7ca4f Add groundwork for new versions of federation APIs 2019-01-15 11:14:34 +00:00
Amber Brown 2e223a8c22
Remove the unused /pull federation API (#4118) 2018-10-31 04:24:59 +11:00
Richard van der Hoff 0fd2321629 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-17 17:35:26 +01:00
Richard van der Hoff fc0f13dd03 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Amber Brown 7ca097f77e
Port federation/ to py3 (#3847) 2018-09-12 23:23:32 +10:00
Richard van der Hoff 4f8baab0c4 Merge branch 'master' into develop 2018-09-06 13:05:22 +01:00
Richard van der Hoff c127c8d042 Fix origin handling for pushed transactions
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
2018-09-05 13:08:07 +01:00
Erik Johnston c2c153dd3b Log more detail when we fail to authenticate request 2018-08-21 11:42:49 +01:00
Erik Johnston 808d8e06aa Don't log exceptions when failing to fetch server keys
Not being able to resolve or connect to remote servers is an expected
error, so we shouldn't log at ERROR with stacktraces.
2018-08-21 11:19:26 +01:00
Richard van der Hoff 0d63d93ca8 Enforce compatibility when processing make_join requests
Reject make_join requests from servers which do not support the room version.

Also include the room version in the response.
2018-08-03 16:08:32 +01:00
Richard van der Hoff 15c1ae45e5 Docstrings for BaseFederationServlet
... to save me reverse-engineering this stuff again.
2018-08-03 16:08:32 +01:00
Amber Brown da7785147d
Python 3: Convert some unicode/bytes uses (#3569) 2018-08-02 00:54:06 +10:00
Travis Ralston e908b86832 Remove pdu_failures from transactions
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
2018-07-30 16:28:47 -06:00
Travis Ralston 7d32f0d745
Update the send_leave path to be an event_id
It's still not used, however the parameter is an event ID not a transaction ID.
2018-07-26 14:41:59 -06:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Richard van der Hoff 3cf3e08a97 Implementation of server_acls
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
2018-07-04 19:06:20 +01:00
Richard van der Hoff 546bc9e28b More server_name validation
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.

Also, use it to verify the server name in the config file.
2018-07-04 18:59:51 +01:00
Richard van der Hoff 508196e08a
Reject invalid server names (#3480)
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
2018-07-03 14:36:14 +01:00
Richard van der Hoff 9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Matthew Hodgson 78a9698650 fix federation_domain_whitelist
we were checking the wrong server_name on inbound requests
2018-04-13 15:47:43 +01:00
Matthew Hodgson 25b0ba30b1 revert last to PR properly 2018-04-13 15:46:37 +01:00