f1e5b41388
Make all the rate limiting options more consistent ( #5181 )
2019-05-15 12:06:04 -05:00
c0e0740bef
add options to require an access_token to GET /profile and /publicRooms on CS API ( #5083 )
...
This commit adds two config options:
* `restrict_public_rooms_to_local_users`
Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.
* `require_auth_for_profile_requests`
When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.
MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.
Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
54a87a7b08
Collect room-version variations into one place ( #4969 )
...
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 ( #4935 )
...
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 ( #4840 )
...
As per #3622 , we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
271cb1998b
Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )"
...
This reverts commit 290552fd83
2019-03-14 14:30:54 +00:00
09626bfd39
Switch to wrapper function around _send_request
2019-03-13 18:26:06 +00:00
0ea8582f8b
Cleaner way of implementing trailing slashes
2019-03-12 14:11:11 +00:00
f18dca26da
Merge branch 'develop' into anoa/trailing_slashes_client
2019-03-11 17:44:29 +00:00
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )
...
Server side of a solution towards #3622 .
2019-03-11 17:44:03 +00:00
a5dd335cd8
lint
2019-03-08 18:25:59 +00:00
64ff11019e
Retry certain federation requests on 404
2019-03-08 18:22:47 +00:00
525dd02bbe
Remove trailing slashes from outbound federation requests
2019-03-08 16:55:52 +00:00
336de1d45b
Remove unnecessary dollar signs
...
A dollar sign is already appended to the end of each PATH, so there's
no need to add one in the PATH declaration as well.
2019-03-04 15:25:12 +00:00
b131cc77df
Make 'event_id' a required parameter in federated state requests ( #4741 )
...
* make 'event_id' a required parameter in federated state requests
As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* add changelog entry for bugfix
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* Update server.py
2019-02-27 14:35:47 -08:00
802884d4ee
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop
2019-02-26 14:23:40 +00:00
7a4632af9c
Prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:37:24 +00:00
c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms"
2019-02-26 13:20:38 +00:00
4b9e5076c4
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate
2019-02-25 15:08:18 +00:00
bd398b874e
Don't restrict non-fed rooms over client APIs
2019-02-25 15:04:46 +00:00
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
...
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
d414f30019
Implement fallback for V2 invite API
...
If the room version is either 1 or 2 then a server should retry failed
`/v2/invite` requests with the v1 API
2019-01-28 17:33:25 +00:00
a50cf929c1
Require event format version to parse or create events
2019-01-25 10:32:19 +00:00
be6a7e47fa
Revert "Require event format version to parse or create events"
2019-01-25 10:23:51 +00:00
f431ff3fb8
Require event format version to parse or create events
2019-01-23 20:21:33 +00:00
d39b7b6d38
Document `servlet_groups` parameters
...
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
82e13662c0
Split federation OpenID userinfo endpoint out of the federation resource
...
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.
The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
4a4d2e17bc
Add /v2/invite federation API
2019-01-15 13:22:44 +00:00
bb63e7ca4f
Add groundwork for new versions of federation APIs
2019-01-15 11:14:34 +00:00
2e223a8c22
Remove the unused /pull federation API ( #4118 )
2018-10-31 04:24:59 +11:00
fc0f13dd03
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
b8a5b0097c
Various cleanups in the federation client code ( #4031 )
...
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
7ca097f77e
Port federation/ to py3 ( #3847 )
2018-09-12 23:23:32 +10:00
4f8baab0c4
Merge branch 'master' into develop
2018-09-06 13:05:22 +01:00
c127c8d042
Fix origin handling for pushed transactions
...
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
2018-09-05 13:08:07 +01:00
a4cf660a32
Merge pull request #3735 from matrix-org/travis/federation-spelling
...
limt -> limit
2018-08-22 09:34:21 +01:00
c2c153dd3b
Log more detail when we fail to authenticate request
2018-08-21 11:42:49 +01:00
808d8e06aa
Don't log exceptions when failing to fetch server keys
...
Not being able to resolve or connect to remote servers is an expected
error, so we shouldn't log at ERROR with stacktraces.
2018-08-21 11:19:26 +01:00
f900d50824
include known room versions in outgoing make_joins
2018-08-06 13:45:37 +01:00
0d63d93ca8
Enforce compatibility when processing make_join requests
...
Reject make_join requests from servers which do not support the room version.
Also include the room version in the response.
2018-08-03 16:08:32 +01:00
15c1ae45e5
Docstrings for BaseFederationServlet
...
... to save me reverse-engineering this stuff again.
2018-08-03 16:08:32 +01:00
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
37be52ac34
limt -> limit
2018-07-31 16:29:09 -06:00
e908b86832
Remove pdu_failures from transactions
...
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
2018-07-30 16:28:47 -06:00
7d32f0d745
Update the send_leave path to be an event_id
...
It's still not used, however the parameter is an event ID not a transaction ID.
2018-07-26 14:41:59 -06:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
3cf3e08a97
Implementation of server_acls
...
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw .
2018-07-04 19:06:20 +01:00
546bc9e28b
More server_name validation
...
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.
Also, use it to verify the server name in the config file.
2018-07-04 18:59:51 +01:00
508196e08a
Reject invalid server names ( #3480 )
...
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
2018-07-03 14:36:14 +01:00
9255a6cb17
Improve exception handling for background processes
...
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
78a9698650
fix federation_domain_whitelist
...
we were checking the wrong server_name on inbound requests
2018-04-13 15:47:43 +01:00
25b0ba30b1
revert last to PR properly
2018-04-13 15:46:37 +01:00
f8d46cad3c
correctly auth inbound federation_domain_whitelist reqs
2018-04-13 15:41:52 +01:00
11d2609da7
Ensure slashes are escaped
2018-04-10 11:24:40 +01:00
dab87b84a3
URL quote path segments over federation
2018-04-10 11:16:08 +01:00
112c2253e2
pep8
2018-04-06 15:43:27 +01:00
f8d1917fce
Fix federation client `set_group_joinable` typo
2018-04-06 15:43:27 +01:00
b370fe61c0
Implement group join API
2018-04-06 15:43:27 +01:00
1d71f484d4
use PUT instead of POST for federating groups/m.join_policy
2018-04-06 12:54:09 +02:00
104c0bc1d5
Use "/settings/" (plural)
2018-04-05 14:07:16 +01:00
eb8d8d6f57
Use join_policy API instead of joinable
...
The API is now under
/groups/$group_id/setting/m.join_policy
and expects a JSON blob of the shape
```json
{
"m.join_policy": {
"type": "invite"
}
}
```
where "invite" could alternatively be "open".
2018-04-03 16:16:40 +01:00
32260baa41
pep8
2018-03-28 14:29:42 +01:00
79452edeee
Add joinability for groups
...
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
2018-03-28 14:03:37 +01:00
cea462e285
s/replication_server/federation_server
2018-03-13 13:22:21 +00:00
265b993b8a
Split replication layer into two
2018-03-13 10:55:47 +00:00
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
82e4bfb53d
Add brackets
2017-11-09 10:06:42 +00:00
e8814410ef
Have an explicit API to update room config
2017-11-08 16:13:27 +00:00
94ff2cda73
Revert "Modify group room association API to allow modification of is_public"
2017-11-08 15:43:34 +00:00
207fabbc6a
Update docs for updating room group association
2017-11-01 09:35:15 +00:00
13b3d7b4a0
Flake8
2017-10-31 17:20:11 +00:00
20fe347906
Modify group room association API to allow modification of is_public
...
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
39dc52157d
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile
2017-10-24 09:16:20 +01:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
ce6d4914f4
Correctly wire in update group profile over federation
2017-10-23 15:21:24 +01:00
011d03a0f6
Fix typo
2017-10-19 11:22:48 +01:00
9ab859f27b
Fix typo in group attestation handling
2017-10-19 10:55:52 +01:00
85f5674e44
Delint
2017-10-16 15:52:17 +01:00
2c5972f87f
Implement GET /groups/$groupId/invited_users
2017-10-16 15:31:11 +01:00
271f5601f3
Fix typo in invite to group
2017-10-11 14:45:33 +01:00
17b8e2bd02
Add remove room API
2017-09-26 15:52:41 +01:00
ef8e578677
Add bulk group publicised lookup API
2017-08-09 13:36:22 +01:00
6f443a74cf
Add update group profile API
2017-07-20 09:46:33 +01:00
14a34f12d7
Comments
2017-07-18 17:28:42 +01:00
3431ec55dc
Comments
2017-07-18 17:23:50 +01:00
94ecd871a0
Fix typos
2017-07-18 16:38:54 +01:00
12ed4ee48e
Correctly parse query params
2017-07-18 15:33:09 +01:00
e5ea6dd021
Add client apis
2017-07-18 14:37:06 +01:00
cccfcfa7b9
Comments
2017-07-18 10:35:18 +01:00
2f9eafdd36
Add local group server support
2017-07-17 12:03:49 +01:00
cb3aee8219
Ensure category and role ids are non-null
2017-07-14 14:06:55 +01:00
8575e3160f
Comments
2017-07-13 13:52:41 +01:00
7a39da8cc6
Add summary APIs to federation
2017-07-13 11:13:19 +01:00
a62406aaa5
Add group summary APIs
2017-07-12 11:36:15 +01:00
fe4e885f54
Add federation API for adding room to group
2017-07-11 14:35:07 +01:00
6322fbbd41
Comment
2017-07-11 11:52:03 +01:00
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
5e49a57ecc
Separate federation servlet into different lists
2017-07-05 14:32:24 +01:00
310b1ccdc1
Use preserve_fn and add logs
2017-05-05 13:41:19 +01:00
db7d0c3127
Always mark remotes as up if we receive a signed request from them
2017-05-05 10:34:53 +01:00
91b3981800
Try harder when sending leave events
...
When we're rejecting invites, ignore the backoff data, so that we have a better
chance of not getting the room out of sync.
2017-04-21 01:50:36 +01:00
4903ccf159
Fix some lies, and other clarifications, in docstrings
...
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
555d702e34
limit total timeout for get_missing_events to 10s
2016-12-31 15:21:37 +00:00
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
23b6701a28
Support filtering remote room lists
2016-09-16 10:24:15 +01:00
5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
24f36469bc
Add federation /version API
2016-08-05 16:36:07 +01:00
597c79be10
Change the way we specify if we require auth or not
2016-08-05 16:17:04 +01:00
b4e2290d89
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api
2016-08-04 14:04:35 +01:00
e3a720217a
Add /state_ids federation API
...
The new API only returns the event_ids for the state, as most
requesters will already have the vast majority of the events already.
2016-08-03 14:47:37 +01:00
1efee2f52b
E2E keys: Make federation query share code with client query
...
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
8f4a9bbc16
Linearize some federation endpoints based on (origin, room_id)
2016-06-17 16:43:45 +01:00
120c238705
Disable responding with canonical json for federation
2016-06-17 16:10:37 +01:00
d88faf92d1
Fix up federation PublicRoomList
2016-06-08 14:39:31 +01:00
1fd6eb695d
Enable auth on federation PublicRoomList
2016-06-08 14:15:18 +01:00
6ecb2ca4ec
pep8
2016-06-01 09:48:55 +01:00
d240796ded
Basic, un-cached support for secondary_directory_servers
2016-05-31 17:20:07 +01:00
70ecb415f5
Fix c+p fail
2016-05-31 12:00:54 +01:00
e1625d62a8
Add federation room list servlet
2016-05-31 11:55:57 +01:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
62b51b8452
Fix typo in event_auth servlet path
2016-04-29 12:00:51 +01:00
914f1eafac
Lower timeout for make_membership_event
...
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
2016-04-15 11:22:23 +01:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
ed61a49169
Add profile information to invites
2016-03-04 14:35:02 +00:00
577951b032
Allow third_party_signed to be specified on /join
2016-02-23 15:11:25 +00:00
9959d9ece8
Remove redundated BaseHomeServer
2016-01-26 13:52:29 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
cf4ef5f3c7
Only retry federation requests for a long time for background requests
2015-11-17 18:26:50 +00:00
2cebe53545
Exchange 3pid invites for m.room.member invites
2015-11-05 16:43:19 +00:00
137fafce4e
Allow rejecting invites
...
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
2015-10-20 11:58:58 +01:00
5b3e9713dd
Implement third party identifier invites
2015-10-01 17:49:52 +01:00
2da3b1e60b
Get the end-to-end key federation working
2015-07-24 18:26:46 +01:00
62c010283d
Add federation support for end-to-end key requests
2015-07-23 16:03:38 +01:00
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
3483b78d1a
Log where a request came from in federation
2015-06-02 18:15:13 +01:00
36317f3dad
Merge pull request #156 from matrix-org/erikj/join_perf
...
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
284f55a7fb
Add doc strings
2015-05-22 15:18:04 +01:00
b21d015c55
Log origin and stats of incoming transactions
2015-05-22 14:44:25 +01:00
5b1631a4a9
Add a timeout param to get_event
2015-05-19 14:53:32 +01:00
c178e4e6ca
Add missing servlet to list
2015-03-19 17:48:21 +00:00
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
d79d91a4a7
Appease pep8
2015-03-05 20:55:40 +00:00
5eab2549ab
Append a $ on PATH at registration time, meaning each PATH attribute doesn't need it
2015-03-05 20:36:05 +00:00
7644cb79b2
Slightly neater(?) arrangement of authentication wrapper for HTTP servlet methods
2015-03-05 20:33:16 +00:00
ba8ac996f9
Remove the dead 'rate_limit_origin' method from TransportLayerServer
2015-03-05 19:43:17 +00:00
a901ed16b5
Move federation API responding code out of weird mix of lambdas into Servlet-style methods on instances
2015-03-05 19:10:57 +00:00
0a036944bd
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
2015-03-02 13:53:30 +00:00
9d9b230501
Make the federation server ratelimiting configurable.
2015-03-02 11:33:45 +00:00
0554d07082
Move federation rate limiting out of transport layer
2015-02-27 15:41:52 +00:00
9dc9118e55
Document FederationRateLimiter
2015-02-27 15:16:47 +00:00
93d90765c4
Initial implementation of federation server rate limiting
2015-02-26 16:15:26 +00:00
db215b7e00
Implement and use new batched get missing pdu
2015-02-23 13:58:02 +00:00
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
ae46f10fc5
Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations
2015-02-04 16:28:12 +00:00
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
5fed042640
Finish renaming "context" to "room_id" in federation codebase
2015-01-16 19:01:03 +00:00
2408c4b0a4
Fold _do_request_for_transaction into the methods that called it since it was a trivial wrapper around client.get_json
2015-01-16 19:01:03 +00:00
602684eac5
Split transport layer into client and server parts
2015-01-16 19:01:03 +00:00
Amber Brown