01e93f48ed
Kill off MatrixCodeMessageException
...
This code brings the SimpleHttpClient into line with the
MatrixFederationHttpClient by having it raise HttpResponseExceptions when a
request fails (rather than trying to parse for matrix errors and maybe raising
MatrixCodeMessageException).
Then, whenever we were checking for MatrixCodeMessageException and turning them
into SynapseErrors, we now need to check for HttpResponseExceptions and call
to_synapse_error.
2018-08-01 16:02:46 +01:00
018d75a148
Refactor code for turning HttpResponseException into SynapseError
...
This commit replaces SynapseError.from_http_response_exception with
HttpResponseException.to_synapse_error.
The new method actually returns a ProxiedRequestError, which allows us to pass
through additional metadata from the API call.
2018-08-01 16:02:46 +01:00
fa7dc889f1
Be more careful which errors we send back over the C-S API
...
We really shouldn't be sending all CodeMessageExceptions back over the C-S API;
it will include things like 401s which we shouldn't proxy.
That means that we need to explicitly turn a few HttpResponseExceptions into
SynapseErrors in the federation layer.
The effect of the latter is that the matrix errcode will get passed through
correctly to calling clients, which might help with some of the random
M_UNKNOWN errors when trying to join rooms.
2018-08-01 16:02:38 +01:00
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
251e6c1210
limit register and sign in on number of monthly users
2018-07-30 15:55:57 +01:00
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
d19fba3655
Merge branch 'develop' into matthew/filter_members
2018-07-24 12:39:54 +01:00
536bc63a4e
Merge branch 'develop' into erikj/client_apis_move
2018-07-24 09:57:05 +01:00
adfe29ec0b
Merge branch 'develop' into matthew/filter_members
2018-07-23 19:21:37 +01:00
e42510ba63
Use new getters
2018-07-23 13:17:22 +01:00
0ecf68aedc
Move check_in_room_or_world_readable to Auth
2018-07-20 15:30:59 +01:00
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
33b60c01b5
Make auth & transactions more testable ( #3499 )
2018-07-14 07:34:49 +10:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
3cf3e08a97
Implementation of server_acls
...
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw .
2018-07-04 19:06:20 +01:00
f4f1cda928
add ip_range_whitelist parameter to limit where ASes can connect from
2018-06-28 20:32:00 +01:00
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
ec1e799e17
Don't print invalid access tokens in the logs
...
Tokens shouldn't be appearing the logs, valid or invalid.
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-24 12:17:01 -06:00
557b686eac
Refactor get_send_level to take a power_levels event
...
it makes it easier for me to reason about
2018-06-14 11:26:27 +01:00
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
9bbb9f5556
add lazy_load_members to the filter json schema
2018-05-29 04:26:10 +01:00
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
33f469ba19
Apply some limits to depth to counter abuse
...
* When creating a new event, cap its depth to 2^63 - 1
* When receiving events, reject any without a sensible depth
As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI
2018-05-01 17:54:19 +01:00
36c59ce669
Use six.itervalues in some places
...
There's more where that came from
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:39:43 +02:00
6168351877
Add b prefixes to some strings that are bytes in py3
...
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-04 13:48:51 +02:00
fcfe7f6ad3
Use simplejson throughout
...
Let's use simplejson rather than json, for consistency.
2018-03-29 22:45:52 +01:00
fa72803490
Merge branch 'master' of github.com:matrix-org/synapse into develop
2018-03-19 11:41:01 +00:00
b2f2282947
make lazy_load_members configurable in filters
2018-03-19 01:15:13 +00:00
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
28a6ccb49c
add registrations_require_3pid
...
lets homeservers specify a whitelist for 3PIDs that users are allowed to associate with.
Typically useful for stopping people from registering with non-work emails
2018-01-19 00:19:58 +00:00
d5f9fb06b0
Refactor UI auth implementation
...
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
2017-12-05 09:40:05 +00:00
da562bd6a1
Improve comments on get_user_by_access_token
...
because I have to reverse-engineer this every time.
2017-11-29 15:52:41 +00:00
aa620d09a0
Add a config option to block all room invites ( #2457 )
...
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
2017-09-19 16:08:14 +01:00
ed9a7f5436
Merge pull request #2309 from matrix-org/erikj/user_ip_repl
...
Fix up user_ip replication commands
2017-07-06 14:33:14 +01:00
2c365f4723
Cache macaroon parse and validation
...
Turns out this can be quite expensive for requests, and is easily
cachable. We don't cache the lookup to the DB so invalidation still
works.
2017-06-29 14:50:18 +01:00
8c23221666
Fix up
2017-06-27 15:53:45 +01:00
ed3d0170d9
Batch upsert user ips
2017-06-27 13:37:04 +01:00
0185b75381
Change is_host_joined to use current_state table
...
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
1a9255c12e
Use CodeMessageException subclass instead
...
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
566641a0b5
use jsonschema.FormatChecker for RoomID and UserID strings
...
* use a valid filter in rest/client/v2_alpha test
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:41 -03:00
acafcf1c5b
Add valid filter tests, flake8, fix typo
...
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:10 -03:00
e56c79c114
check_valid_filter using JSONSchema
...
* add invalid filter tests
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:07 -03:00
19b9366d73
Fix a couple of logcontext leaks
...
Use preserve_fn to correctly manage the logcontexts around things we don't want
to yield on.
2017-03-23 00:17:46 +00:00
a8f96c63aa
Comment
2017-03-15 16:01:01 +00:00
e892457a03
Comment
2017-03-15 15:01:39 +00:00
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
7827251daf
Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
...
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
1d09586599
Address review comments
...
- don't blindly proxy all HTTPRequestExceptions
- log unexpected exceptions at error
- avoid `isinstance`
- improve docs on `from_http_response_exception`
2017-03-14 14:15:37 +00:00
7f237800e9
re-refactor exception heirarchy
...
Give CodeMessageException back its `msg` attribute, and use that to hold the
HTTP status message for HttpResponseException.
2017-03-14 14:15:37 +00:00
73a5f06652
Support registration / login with phone number
...
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
170ccc9de5
Fix routing loop when fetching remote media
...
When we proxy a media request to a remote server, add a query-param, which will
tell the remote server to 404 if it doesn't recognise the server_name.
This should fix a routing loop where the server keeps forwarding back to
itself.
Also improves the error handling on remote media fetches, so that we don't
always return a rather obscure 502.
2017-03-13 16:30:36 +00:00
7eae6eaa2f
Revert "Support registration & login with phone number"
2017-03-13 09:59:33 +00:00
ce3e583d94
WIP support for msisdn 3pid proxy methods
2017-02-14 15:05:55 +00:00
063a1251a9
Remove a few aspirational but unused constants
...
from the Kegan era
2017-02-08 11:36:08 +00:00
5f027d1fc5
Change resolve_state_groups call site logging to DEBUG
2017-01-17 17:07:15 +00:00
e178feca3f
Remove unused function
2017-01-13 15:16:45 +00:00
8b2fa38256
Split event auth code into seperate module
2017-01-13 15:07:32 +00:00
7e6c2937c3
Split out static auth methods from Auth object
2017-01-10 18:16:54 +00:00
c18f7fc410
Fix flake8 and update changelog
2017-01-05 13:50:22 +00:00
d79d165761
add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works.
2017-01-05 13:40:39 +00:00
1529c19675
Prevent user tokens being used as guest tokens ( #1675 )
...
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
2016-12-06 15:31:37 +00:00
aa09d6b8f0
Rip out more refresh_token code
...
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
321fe5c44c
Merge pull request #1656 from matrix-org/rav/remove_time_caveat
...
Stop putting a time caveat on access tokens
2016-11-30 16:53:20 +00:00
4febfe47f0
Comments
...
Update comments in verify_macaroon
2016-11-30 07:36:32 +00:00
77eca2487c
Merge pull request #1653 from matrix-org/rav/guest_e2e
...
Implement E2E for guests
2016-11-29 17:41:35 +00:00
1c4f05db41
Stop putting a time caveat on access tokens
...
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.
Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
b6146537d2
Merge pull request #1655 from matrix-org/rav/remove_redundant_macaroon_checks
...
Remove redundant list of known caveat prefixes
2016-11-25 16:57:19 +00:00
7f02e4d008
Give guest users a device_id
...
We need to create devices for guests so that they can use e2e, but we don't
have anywhere to store it, so just use a fixed one.
2016-11-25 15:25:30 +00:00
e1d7c96814
Remove redundant list of known caveat prefixes
...
Also add some comments.
2016-11-24 12:38:17 +00:00
83bcdcee61
Return early on /sync code paths if a '*' filter is used
...
This is currently very conservative in that it only does this if there is no
`since` token. This limits the risk to clients likely to be doing one-off
syncs (like bridges), but does mean that normal human clients won't benefit
from the time savings here. If the savings are large enough, I would consider
generalising this to just check the filter.
2016-11-22 16:38:35 +00:00
d4a459f7cb
Merge pull request #1638 from matrix-org/kegan/sync-event-fields
...
Implement "event_fields" in filters
2016-11-22 14:02:38 +00:00
cea4e4e7b2
Glue only_event_fields into the sync rest servlet
2016-11-22 10:14:05 +00:00
f97511a1f3
Move event_fields filtering to serialize_event
...
Also make it an inclusive not exclusive filter, as the spec demands.
2016-11-21 17:42:16 +00:00
e90fcd9edd
Add filter_event_fields and filter_field to FilterCollection
2016-11-21 15:18:18 +00:00
a2a6c1c22f
Fail with a coherent error message if `/sync?filter=` is invalid
2016-11-21 13:15:25 +00:00
177f104432
Merge pull request #1098 from matrix-org/markjh/bearer_token
...
Allow clients to supply access_tokens as headers
2016-10-25 17:33:15 +01:00
e01a1bc92d
Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting
...
Allow Configurable Rate Limiting Per AS
2016-10-20 16:21:10 +01:00
07caa749bf
Closing brace on following line
2016-10-20 12:07:16 +01:00
f09db236b1
as_user->app_service, less redundant comments, better positioned comments
2016-10-20 12:04:54 +01:00
8bfd01f619
flake8
2016-10-20 11:52:46 +01:00
1b17d1a106
Use real AS object by passing it through the requester
...
This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user
2016-10-20 11:43:05 +01:00
550308c7a1
Check whether to ratelimit sooner to avoid work
2016-10-19 10:45:24 +01:00
a2f2516199
Merge pull request #1157 from Rugvip/nolimit
...
Remove rate limiting from app service senders and fix get_or_create_user requester
2016-10-11 11:20:54 +01:00
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
24a70e19c7
api/auth: fix for not being allowed to set your own state_key
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-09-30 13:08:25 +02:00
f96020550f
Update comments
2016-09-22 12:54:22 +01:00
2e9ee30969
Add comments
2016-09-22 11:59:46 +01:00
a61e4522b5
Shuffle things around to make unit tests work
2016-09-22 11:08:12 +01:00
1168cbd54d
Allow invites via 3pid to bypass sender sig check
...
When a server sends a third party invite another server may be the one
that the inviting user registers with. In this case it is that remote
server that will issue an actual invitation, and wants to do it "in the
name of" the original invitee. However, the new proper invite will not
be signed by the original server, and thus other servers would reject
the invite if it was seen as coming from the original user.
To fix this, a special case has been added to the auth rules whereby
another server can send an invite "in the name of" another server's
user, so long as that user had previously issued a third party invite
that is now being accepted.
2016-09-22 10:56:53 +01:00
ec609f8094
Fix unit tests
2016-09-12 10:46:02 +01:00
8e01263587
Allow clients to supply access_tokens as headers
...
Clients can continue to supply access tokens as query parameters
or can supply the token as a header:
Authorization: Bearer <access_token_goes_here>
This matches the ouath2 format of
https://tools.ietf.org/html/rfc6750#section-2.1
2016-09-09 18:17:42 +01:00
8aee5aa068
Add helper function for getting access_tokens from requests
...
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
ed7a703d4c
Handle the fact that workers can't generate state groups
2016-08-31 15:53:19 +01:00
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
1ccdc1e93a
Cache check_host_in_room
2016-08-26 10:59:40 +01:00
25414b44a2
Add measure on check_host_in_room
2016-08-26 10:47:00 +01:00
30961182f2
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids
2016-08-26 09:48:13 +01:00
1294d4a329
Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant
2016-08-25 18:34:47 +01:00
0e1900d819
Pull out full state less
2016-08-25 18:15:51 +01:00
142983b4ea
APP_SERVICE_PREFIX is never used; don't bother
2016-08-25 18:06:05 +01:00
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
6fe6a6f029
Fix login with m.login.token
...
login with token (as used by CAS auth) was broken by 067596d
2016-08-08 16:40:39 +01:00
c63b1697f4
Merge pull request #952 from matrix-org/markjh/more_fixes
...
Check if the user is banned when handling 3pid invites
2016-07-26 19:20:56 +01:00
eb359eced4
Add `create_requester` function
...
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
c824b29e77
Check if the user is banned when handling 3pid invites
2016-07-26 16:39:14 +01:00
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
053e83dafb
More doc-comments
...
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
d137e03231
Fix 500 ISE when sending alias event without a state_key
2016-07-15 18:58:25 +01:00
bd7c51921d
Merge pull request #919 from matrix-org/erikj/auth_fix
...
Various auth.py fixes.
2016-07-15 11:38:33 +01:00
ebdafd8114
Check sender signed event
2016-07-14 17:03:24 +01:00
209e04fa11
Merge pull request #918 from negzi/bugfix_for_token_expiry
...
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
e5142f65a6
Add 'contains_url' to filter
2016-07-14 15:35:48 +01:00
0136a522b1
Bug fix: expire invalid access tokens
2016-07-13 15:00:37 +02:00
2cb758ac75
Check if alias event's state_key matches sender's domain
2016-07-13 13:12:25 +01:00
560c71c735
Check creation event's room_id domain matches sender's
2016-07-13 13:07:19 +01:00
385aec4010
Implement https://github.com/matrix-org/matrix-doc/pull/346/files
2016-07-08 17:42:48 +01:00
58930da52b
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-07-08 14:11:37 +01:00
067596d341
Fix bug where we did not correctly explode when multiple user_ids were set in macaroon
2016-07-07 16:22:24 +01:00
be8be535f7
requestToken update
...
Don't send requestToken request to untrusted ID servers
Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
2016-06-30 17:51:28 +01:00
1f31cc37f8
Working unsubscribe links going straight to the HS
...
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
812b5de0fe
Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe
2016-06-02 15:33:28 +01:00
aaa70e26a2
special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208
2016-06-01 22:13:47 +01:00
991af8b0d6
WIP on unsubscribing email notifs without logging in
2016-06-01 17:40:52 +01:00
1a3a2002ff
Spell "domain" correctly
...
s/domian/domain/g
2016-05-16 19:17:23 +01:00
40aa6e8349
Create user with expiry
...
- Add unittests for client, api and handler
Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
c9eb6dfc1b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-09 13:21:06 +01:00
08dfa8eee2
Add and use get_domian_from_id
2016-05-09 10:36:03 +01:00
a438a6d2bc
Implement basic ignore user
2016-05-04 10:16:46 +01:00
0f2ca8cde1
Measure Auth.check
2016-04-13 11:15:59 +01:00
c53f9d561e
Don't auto log failed auth checks
2016-04-13 11:11:46 +01:00
3e7fac0d56
Add published room list edit API
2016-03-21 15:06:07 +00:00
874fd43257
Send the user ID matching the guest access token, since there is no Matrix API to discover what user ID an access token is for.
2016-03-07 17:13:56 +00:00
577951b032
Allow third_party_signed to be specified on /join
2016-02-23 15:11:25 +00:00
e5ad2e5267
Merge pull request #582 from matrix-org/erikj/presence
...
Rewrite presence for performance.
2016-02-19 09:37:50 +00:00
114b929f8b
Check presence state is a valid one
2016-02-18 09:16:32 +00:00
536f949a1a
api/filtering: don't assume that event content will always be a dict
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-17 12:59:41 +01:00
4de08a4672
Revert "Merge two of the room join codepaths"
...
This reverts commit cf81375b94
2016-02-12 16:17:24 +00:00
cf81375b94
Merge two of the room join codepaths
...
There's at least one more to merge in.
Side-effects:
* Stop reporting None as displayname and avatar_url in some cases
* Joining a room by alias populates guest-ness in join event
* Remove unspec'd PUT version of /join/<room_id_or_alias> which has not
been called on matrix.org according to logs
* Stop recording access_token_id on /join/room_id - currently we don't
record it on /join/room_alias; I can try to thread it through at some
point.
2016-02-12 15:11:49 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
737c4223ef
Host /media/r0 as well as /media/v1
2016-02-05 10:47:46 +00:00
2df6114bc4
Log more diagnostics for unrecognised access tokens
2016-02-02 19:21:49 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
Richard van der Hoff