Commit Graph

298 Commits

Author SHA1 Message Date
Andrew Morgan 7a4632af9c Prevent showing non-fed rooms in fed /publicRooms 2019-02-26 13:37:24 +00:00
Andrew Morgan c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms" 2019-02-26 13:20:38 +00:00
Erik Johnston 4b9e5076c4 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate 2019-02-25 15:08:18 +00:00
Andrew Morgan bd398b874e Don't restrict non-fed rooms over client APIs 2019-02-25 15:04:46 +00:00
Erik Johnston b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Erik Johnston d414f30019 Implement fallback for V2 invite API
If the room version is either 1 or 2 then a server should retry failed
`/v2/invite` requests with the v1 API
2019-01-28 17:33:25 +00:00
Erik Johnston a50cf929c1 Require event format version to parse or create events 2019-01-25 10:32:19 +00:00
Erik Johnston be6a7e47fa
Revert "Require event format version to parse or create events" 2019-01-25 10:23:51 +00:00
Erik Johnston f431ff3fb8 Require event format version to parse or create events 2019-01-23 20:21:33 +00:00
Jason Robinson d39b7b6d38 Document `servlet_groups` parameters
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson 82e13662c0 Split federation OpenID userinfo endpoint out of the federation resource
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.

The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Erik Johnston 4a4d2e17bc Add /v2/invite federation API 2019-01-15 13:22:44 +00:00
Erik Johnston bb63e7ca4f Add groundwork for new versions of federation APIs 2019-01-15 11:14:34 +00:00
Amber Brown 2e223a8c22
Remove the unused /pull federation API (#4118) 2018-10-31 04:24:59 +11:00
Richard van der Hoff fc0f13dd03 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Richard van der Hoff b8a5b0097c
Various cleanups in the federation client code (#4031)
- Improve logging: log things in the right order, include destination and txids
  in all log lines, don't log successful responses twice

- Fix the docstring on TransportLayerClient.send_transaction

- Don't use treq.request, which is overcomplicated for our purposes: just use a
  twisted.web.client.Agent.

- simplify the logic for setting up the bodyProducer

- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
Amber Brown 7ca097f77e
Port federation/ to py3 (#3847) 2018-09-12 23:23:32 +10:00
Richard van der Hoff 4f8baab0c4 Merge branch 'master' into develop 2018-09-06 13:05:22 +01:00
Richard van der Hoff c127c8d042 Fix origin handling for pushed transactions
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
2018-09-05 13:08:07 +01:00
Richard van der Hoff a4cf660a32
Merge pull request #3735 from matrix-org/travis/federation-spelling
limt -> limit
2018-08-22 09:34:21 +01:00
Erik Johnston c2c153dd3b Log more detail when we fail to authenticate request 2018-08-21 11:42:49 +01:00
Erik Johnston 808d8e06aa Don't log exceptions when failing to fetch server keys
Not being able to resolve or connect to remote servers is an expected
error, so we shouldn't log at ERROR with stacktraces.
2018-08-21 11:19:26 +01:00
Richard van der Hoff f900d50824 include known room versions in outgoing make_joins 2018-08-06 13:45:37 +01:00
Richard van der Hoff 0d63d93ca8 Enforce compatibility when processing make_join requests
Reject make_join requests from servers which do not support the room version.

Also include the room version in the response.
2018-08-03 16:08:32 +01:00
Richard van der Hoff 15c1ae45e5 Docstrings for BaseFederationServlet
... to save me reverse-engineering this stuff again.
2018-08-03 16:08:32 +01:00
Amber Brown da7785147d
Python 3: Convert some unicode/bytes uses (#3569) 2018-08-02 00:54:06 +10:00
Travis Ralston 37be52ac34
limt -> limit 2018-07-31 16:29:09 -06:00
Travis Ralston e908b86832 Remove pdu_failures from transactions
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
2018-07-30 16:28:47 -06:00
Travis Ralston 7d32f0d745
Update the send_leave path to be an event_id
It's still not used, however the parameter is an event ID not a transaction ID.
2018-07-26 14:41:59 -06:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Richard van der Hoff 3cf3e08a97 Implementation of server_acls
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
2018-07-04 19:06:20 +01:00
Richard van der Hoff 546bc9e28b More server_name validation
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.

Also, use it to verify the server name in the config file.
2018-07-04 18:59:51 +01:00
Richard van der Hoff 508196e08a
Reject invalid server names (#3480)
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
2018-07-03 14:36:14 +01:00
Richard van der Hoff 9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Matthew Hodgson 78a9698650 fix federation_domain_whitelist
we were checking the wrong server_name on inbound requests
2018-04-13 15:47:43 +01:00
Matthew Hodgson 25b0ba30b1 revert last to PR properly 2018-04-13 15:46:37 +01:00
Matthew Hodgson f8d46cad3c correctly auth inbound federation_domain_whitelist reqs 2018-04-13 15:41:52 +01:00
Erik Johnston 11d2609da7 Ensure slashes are escaped 2018-04-10 11:24:40 +01:00
Erik Johnston dab87b84a3 URL quote path segments over federation 2018-04-10 11:16:08 +01:00
Luke Barnard 112c2253e2 pep8 2018-04-06 15:43:27 +01:00
Luke Barnard f8d1917fce Fix federation client `set_group_joinable` typo 2018-04-06 15:43:27 +01:00
David Baker b370fe61c0 Implement group join API 2018-04-06 15:43:27 +01:00
Krombel 1d71f484d4 use PUT instead of POST for federating groups/m.join_policy 2018-04-06 12:54:09 +02:00
Luke Barnard 104c0bc1d5 Use "/settings/" (plural) 2018-04-05 14:07:16 +01:00
Luke Barnard eb8d8d6f57 Use join_policy API instead of joinable
The API is now under
 /groups/$group_id/setting/m.join_policy

and expects a JSON blob of the shape

```json
{
  "m.join_policy": {
    "type": "invite"
  }
}
```

where "invite" could alternatively be "open".
2018-04-03 16:16:40 +01:00
David Baker 32260baa41 pep8 2018-03-28 14:29:42 +01:00
David Baker 79452edeee Add joinability for groups
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
2018-03-28 14:03:37 +01:00
Erik Johnston cea462e285 s/replication_server/federation_server 2018-03-13 13:22:21 +00:00
Erik Johnston 265b993b8a Split replication layer into two 2018-03-13 10:55:47 +00:00
Matthew Hodgson ab9f844aaf
Add federation_domain_whitelist option (#2820)
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Erik Johnston 82e4bfb53d Add brackets 2017-11-09 10:06:42 +00:00
Erik Johnston e8814410ef Have an explicit API to update room config 2017-11-08 16:13:27 +00:00
Erik Johnston 94ff2cda73
Revert "Modify group room association API to allow modification of is_public" 2017-11-08 15:43:34 +00:00
Luke Barnard 207fabbc6a Update docs for updating room group association 2017-11-01 09:35:15 +00:00
Luke Barnard 13b3d7b4a0 Flake8 2017-10-31 17:20:11 +00:00
Luke Barnard 20fe347906 Modify group room association API to allow modification of is_public
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Erik Johnston 39dc52157d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile 2017-10-24 09:16:20 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston ce6d4914f4 Correctly wire in update group profile over federation 2017-10-23 15:21:24 +01:00
Erik Johnston 011d03a0f6 Fix typo 2017-10-19 11:22:48 +01:00
Erik Johnston 9ab859f27b Fix typo in group attestation handling 2017-10-19 10:55:52 +01:00
Luke Barnard 85f5674e44 Delint 2017-10-16 15:52:17 +01:00
Luke Barnard 2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Erik Johnston 271f5601f3 Fix typo in invite to group 2017-10-11 14:45:33 +01:00
Erik Johnston 17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston 3431ec55dc Comments 2017-07-18 17:23:50 +01:00
Erik Johnston 94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston 12ed4ee48e Correctly parse query params 2017-07-18 15:33:09 +01:00
Erik Johnston e5ea6dd021 Add client apis 2017-07-18 14:37:06 +01:00
Erik Johnston cccfcfa7b9 Comments 2017-07-18 10:35:18 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston cb3aee8219 Ensure category and role ids are non-null 2017-07-14 14:06:55 +01:00
Erik Johnston 8575e3160f Comments 2017-07-13 13:52:41 +01:00
Erik Johnston 7a39da8cc6 Add summary APIs to federation 2017-07-13 11:13:19 +01:00
Erik Johnston a62406aaa5 Add group summary APIs 2017-07-12 11:36:15 +01:00
Erik Johnston fe4e885f54 Add federation API for adding room to group 2017-07-11 14:35:07 +01:00
Erik Johnston 6322fbbd41 Comment 2017-07-11 11:52:03 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Erik Johnston 5e49a57ecc Separate federation servlet into different lists 2017-07-05 14:32:24 +01:00
Erik Johnston 310b1ccdc1 Use preserve_fn and add logs 2017-05-05 13:41:19 +01:00
Erik Johnston db7d0c3127 Always mark remotes as up if we receive a signed request from them 2017-05-05 10:34:53 +01:00
Richard van der Hoff 91b3981800 Try harder when sending leave events
When we're rejecting invites, ignore the backoff data, so that we have a better
chance of not getting the room out of sync.
2017-04-21 01:50:36 +01:00
Richard van der Hoff 4903ccf159 Fix some lies, and other clarifications, in docstrings
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
Richard van der Hoff 5a16cb4bf0 Ignore backoff history for invites, aliases, and roomdirs
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
Richard van der Hoff 4bd597d9fc push federation retry limiter down to matrixfederationclient
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
Erik Johnston c974116f19 Implement device key caching over federation 2017-01-26 16:07:24 +00:00
Matthew Hodgson 555d702e34 limit total timeout for get_missing_events to 10s 2016-12-31 15:21:37 +00:00
Erik Johnston f32fb65552 Add new API appservice specific public room list 2016-12-06 16:12:27 +00:00
Erik Johnston 23b6701a28 Support filtering remote room lists 2016-09-16 10:24:15 +01:00
Erik Johnston 5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Mark Haines 949c2c5435 Add a timeout parameter for end2end key queries.
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Erik Johnston 24f36469bc Add federation /version API 2016-08-05 16:36:07 +01:00
Erik Johnston 597c79be10 Change the way we specify if we require auth or not 2016-08-05 16:17:04 +01:00
Erik Johnston b4e2290d89 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api 2016-08-04 14:04:35 +01:00
Erik Johnston e3a720217a Add /state_ids federation API
The new API only returns the event_ids for the state, as most
requesters will already have the vast majority of the events already.
2016-08-03 14:47:37 +01:00
Richard van der Hoff 1efee2f52b E2E keys: Make federation query share code with client query
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
Erik Johnston 8f4a9bbc16 Linearize some federation endpoints based on (origin, room_id) 2016-06-17 16:43:45 +01:00
Erik Johnston 120c238705 Disable responding with canonical json for federation 2016-06-17 16:10:37 +01:00
Erik Johnston d88faf92d1 Fix up federation PublicRoomList 2016-06-08 14:39:31 +01:00
Erik Johnston 1fd6eb695d Enable auth on federation PublicRoomList 2016-06-08 14:15:18 +01:00
David Baker 6ecb2ca4ec pep8 2016-06-01 09:48:55 +01:00
David Baker d240796ded Basic, un-cached support for secondary_directory_servers 2016-05-31 17:20:07 +01:00
David Baker 70ecb415f5 Fix c+p fail 2016-05-31 12:00:54 +01:00
David Baker e1625d62a8 Add federation room list servlet 2016-05-31 11:55:57 +01:00
Mark Haines 9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston 62b51b8452 Fix typo in event_auth servlet path 2016-04-29 12:00:51 +01:00
Erik Johnston 914f1eafac Lower timeout for make_membership_event
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
2016-04-15 11:22:23 +01:00
Erik Johnston acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Mark Haines e9c1cabac2 Use parse_json_object_from_request to parse JSON out of request bodies 2016-03-11 16:41:03 +00:00
Erik Johnston ed61a49169 Add profile information to invites 2016-03-04 14:35:02 +00:00
Daniel Wagner-Hall 577951b032 Allow third_party_signed to be specified on /join 2016-02-23 15:11:25 +00:00
Erik Johnston 9959d9ece8 Remove redundated BaseHomeServer 2016-01-26 13:52:29 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall 14d7acfad4 Host /unstable and /r0 versions of r0 APIs 2015-12-01 17:34:32 +00:00
Erik Johnston cf4ef5f3c7 Only retry federation requests for a long time for background requests 2015-11-17 18:26:50 +00:00
Daniel Wagner-Hall 2cebe53545 Exchange 3pid invites for m.room.member invites 2015-11-05 16:43:19 +00:00
Daniel Wagner-Hall 137fafce4e Allow rejecting invites
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
2015-10-20 11:58:58 +01:00
Daniel Wagner-Hall 5b3e9713dd Implement third party identifier invites 2015-10-01 17:49:52 +01:00
Mark Haines 2da3b1e60b Get the end-to-end key federation working 2015-07-24 18:26:46 +01:00
Mark Haines 62c010283d Add federation support for end-to-end key requests 2015-07-23 16:03:38 +01:00
Erik Johnston cee69441d3 Log more when we have processed the request 2015-06-15 17:11:44 +01:00
Erik Johnston 3483b78d1a Log where a request came from in federation 2015-06-02 18:15:13 +01:00
Mark Haines 36317f3dad Merge pull request #156 from matrix-org/erikj/join_perf
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
Erik Johnston 284f55a7fb Add doc strings 2015-05-22 15:18:04 +01:00
Erik Johnston b21d015c55 Log origin and stats of incoming transactions 2015-05-22 14:44:25 +01:00
Erik Johnston 5b1631a4a9 Add a timeout param to get_event 2015-05-19 14:53:32 +01:00
Erik Johnston c178e4e6ca Add missing servlet to list 2015-03-19 17:48:21 +00:00
Paul "LeoNerd" Evans 1748605c5d Count incoming HTTP requests per servlet that responds 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans d79d91a4a7 Appease pep8 2015-03-05 20:55:40 +00:00
Paul "LeoNerd" Evans 5eab2549ab Append a $ on PATH at registration time, meaning each PATH attribute doesn't need it 2015-03-05 20:36:05 +00:00
Paul "LeoNerd" Evans 7644cb79b2 Slightly neater(?) arrangement of authentication wrapper for HTTP servlet methods 2015-03-05 20:33:16 +00:00
Paul "LeoNerd" Evans ba8ac996f9 Remove the dead 'rate_limit_origin' method from TransportLayerServer 2015-03-05 19:43:17 +00:00
Paul "LeoNerd" Evans a901ed16b5 Move federation API responding code out of weird mix of lambdas into Servlet-style methods on instances 2015-03-05 19:10:57 +00:00
Erik Johnston 0a036944bd Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu 2015-03-02 13:53:30 +00:00
Erik Johnston 9d9b230501 Make the federation server ratelimiting configurable. 2015-03-02 11:33:45 +00:00
Erik Johnston 0554d07082 Move federation rate limiting out of transport layer 2015-02-27 15:41:52 +00:00
Erik Johnston 9dc9118e55 Document FederationRateLimiter 2015-02-27 15:16:47 +00:00
Erik Johnston 93d90765c4 Initial implementation of federation server rate limiting 2015-02-26 16:15:26 +00:00
Erik Johnston db215b7e00 Implement and use new batched get missing pdu 2015-02-23 13:58:02 +00:00
Erik Johnston 4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Erik Johnston ae46f10fc5 Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations 2015-02-04 16:28:12 +00:00
Erik Johnston 78015948a7 Initial implementation of auth conflict resolution 2015-01-29 16:52:33 +00:00
Mark Haines 5fed042640 Finish renaming "context" to "room_id" in federation codebase 2015-01-16 19:01:03 +00:00
Mark Haines 2408c4b0a4 Fold _do_request_for_transaction into the methods that called it since it was a trivial wrapper around client.get_json 2015-01-16 19:01:03 +00:00
Mark Haines 602684eac5 Split transport layer into client and server parts 2015-01-16 19:01:03 +00:00