Commit Graph

4912 Commits

Author SHA1 Message Date
Richard van der Hoff eb359eced4 Add `create_requester` function
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
Mark Haines c824b29e77 Check if the user is banned when handling 3pid invites 2016-07-26 16:39:14 +01:00
Richard van der Hoff 33d7776473 Fix typo 2016-07-26 13:32:15 +01:00
Richard van der Hoff 9ad8d9b17c Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device 2016-07-26 13:29:46 +01:00
Richard van der Hoff 5b1825ba5b Merge pull request #951 from matrix-org/rav/flake8
Fix flake8 noise
2016-07-26 13:27:54 +01:00
Mark Haines 9c4cf83259 Merge pull request #948 from matrix-org/markjh/auth_fixes
Don't add rejections to the state_group, persist all rejections
2016-07-26 13:22:57 +01:00
Richard van der Hoff 05e7e5e972 Fix flake8 violation
Apparently flake8 v3 puts the error on a different line to v2. Easiest way to
make sure that happens is by putting the whole statement on one line :)
2016-07-26 11:59:08 +01:00
Richard van der Hoff 8e02494166 Delete refresh tokens when deleting devices 2016-07-26 11:10:37 +01:00
Mark Haines a6f06ce3e2 Fix how push_actions are redacted. 2016-07-26 11:05:39 +01:00
David Baker d34e9f93b7 Merge pull request #949 from matrix-org/rav/update_devices
Implement updates and deletes for devices
2016-07-26 10:49:55 +01:00
Mark Haines efeb6176c1 Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically 2016-07-26 10:49:52 +01:00
Matthew Hodgson 242c52d607 typo 2016-07-26 10:09:25 +02:00
Richard van der Hoff 012b4c1913 Implement updating devices
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff 436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
Mark Haines 1b3c3e6d68 Only update the events and event_json tables for rejected events 2016-07-25 18:44:30 +01:00
Richard van der Hoff 33d08e8433 Log when adding listeners 2016-07-25 17:22:15 +01:00
Mark Haines 8f7f4cb92b Don't add the events to forward extremities if the event is rejected 2016-07-25 17:13:37 +01:00
Mark Haines 2623cec874 Don't add rejections to the state_group, persist all rejections 2016-07-25 16:12:16 +01:00
David Baker 4fcdf7b4b2 Merge pull request #946 from matrix-org/dbkr/log_recaptcha_hostname
Log the hostname the reCAPTCHA was completed on
2016-07-25 16:10:39 +01:00
Mark Haines 955ef1f06c fix: defer.returnValue takes one argument 2016-07-25 16:04:45 +01:00
Richard van der Hoff 2ee4c9ee02 background updates: fix assert again 2016-07-25 16:01:46 +01:00
Richard van der Hoff 9dbd903f41 background updates: Fix assertion to do something 2016-07-25 14:05:23 +01:00
Richard van der Hoff 465117d7ca Fix background_update tests
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
David Baker 7ed58bb347 Use get to avoid KeyErrors 2016-07-22 17:18:50 +01:00
David Baker dad2da7e54 Log the hostname the reCAPTCHA was completed on
This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.
2016-07-22 17:00:56 +01:00
Richard van der Hoff 363786845b PEP8 2016-07-22 13:21:07 +01:00
Richard van der Hoff ec5717caf5 Create index on user_ips in the background
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.

I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Erik Johnston d26b660aa6 Cache getPeer 2016-07-21 17:38:51 +01:00
Erik Johnston aede7248ab Split out a FederationReader process 2016-07-21 17:37:44 +01:00
Richard van der Hoff 55abbe1850 make /devices return a list
Turns out I specced this to return a list of devices rather than a dict of them
2016-07-21 15:57:28 +01:00
David Baker 2c28e25bda Merge pull request #943 from matrix-org/rav/get_device_api
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
David Baker 1e6e370b76 Merge pull request #942 from matrix-org/rav/fix_register_deviceid
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
Richard van der Hoff 1c3c202b96 Fix PEP8 errors 2016-07-21 13:15:15 +01:00
Richard van der Hoff 406f7aa0f6 Implement GET /device/{deviceId} 2016-07-21 12:00:29 +01:00
Richard van der Hoff c445f5fec7 storage/client_ips: remove some dead code 2016-07-21 11:58:47 +01:00
Erik Johnston cf94a78872 Set host not path 2016-07-21 11:45:53 +01:00
Richard van der Hoff 1a64dffb00 Preserve device_id from first call to /register
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
Erik Johnston 081e5d55e6 Send the correct host header when fetching keys 2016-07-21 11:14:54 +01:00
Erik Johnston 248e6770ca Cache federation state responses 2016-07-21 10:30:12 +01:00
Richard van der Hoff 7314bf4682 Merge branch 'develop' into rav/get_devices_api
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff e9e3eaa67d Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips
Record device_id in client_ips
2016-07-20 17:38:45 +01:00
Erik Johnston d36b1d849d Don't explode if we have no snapshots yet 2016-07-20 16:59:52 +01:00
David Baker 742056be0d Merge pull request #937 from matrix-org/rav/register_device_on_register
Register a device_id in the /v2/register flow.
2016-07-20 16:51:27 +01:00
Richard van der Hoff bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff ec041b335e Record device_id in client_ips
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff 053e83dafb More doc-comments
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
Richard van der Hoff b97a1356b1 Register a device_id in the /v2/register flow.
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.

Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
Erik Johnston b73dc0ef4d Merge pull request #936 from matrix-org/erikj/log_rss
Add metrics for psutil derived memory usage
2016-07-20 16:32:38 +01:00
Erik Johnston 66868119dc Add metrics for psutil derived memory usage 2016-07-20 16:00:21 +01:00
Erik Johnston aba0b2a39b Merge pull request #935 from matrix-org/erikj/backfill_notifs
Don't notify pusher pool for backfilled events
2016-07-20 13:39:16 +01:00
Erik Johnston 57dca35692 Don't notify pusher pool for backfilled events 2016-07-20 13:25:06 +01:00
Richard van der Hoff c68518dfbb Merge pull request #933 from matrix-org/rav/type_annotations
Type annotations
2016-07-20 12:26:32 +01:00
David Baker e967bc86e7 Merge pull request #932 from matrix-org/rav/register_refactor
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston 1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Erik Johnston f91faf09b3 Comment 2016-07-20 10:18:09 +01:00
Richard van der Hoff 3413f1e284 Type annotations
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
Richard van der Hoff 40cbffb2d2 Further registration refactoring
* `RegistrationHandler.appservice_register` no longer issues an access token:
  instead it is left for the caller to do it. (There are two of these, one in
  `synapse/rest/client/v1/register.py`, which now simply calls
  `AuthHandler.issue_access_token`, and the other in
  `synapse/rest/client/v2_alpha/register.py`, which is covered below).

* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
  access_tokens into `_create_registration_details`. This means that the normal
  flow no longer needs to call `AuthHandler.issue_access_token`; the
  shared-secret flow can tell `RegistrationHandler.register` not to generate a
  token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
David Baker b9e997f561 Merge pull request #931 from matrix-org/rav/refactor_register
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
Richard van der Hoff 9a7a77a22a Merge pull request #929 from matrix-org/rav/support_deviceid_in_login
Add device_id support to /login
2016-07-19 15:53:04 +01:00
Richard van der Hoff 8f6281ab0c Don't bind email unless threepid contains expected fields 2016-07-19 15:50:01 +01:00
Richard van der Hoff 0da0d0a29d rest/client/v2_alpha/register.py: Refactor flow somewhat.
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.

The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
Richard van der Hoff 022b9176fe schema fix
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Mark Haines c41d52a042 Summary line 2016-07-19 10:28:27 +01:00
Mark Haines 7e554aac86 Update docstring on Handlers.
To indicate it is deprecated.
2016-07-19 10:20:58 +01:00
Richard van der Hoff f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Richard van der Hoff 93efcb8526 Merge pull request #928 from matrix-org/rav/refactor_login
Refactor login flow
2016-07-18 16:12:35 +01:00
Richard van der Hoff dcfd71aa4c Refactor login flow
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.

Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
Erik Johnston fca90b3445 Merge pull request #924 from matrix-org/erikj/purge_history
Fix /purge_history bug
2016-07-18 15:11:11 +01:00
Mark Haines a292454aa1 Merge pull request #925 from matrix-org/markjh/auth_fix
Fix 500 ISE when sending alias event without a state_key
2016-07-18 15:04:47 +01:00
Erik Johnston 4f81edbd4f Merge pull request #927 from Half-Shot/develop
Fall back to 'username' if 'user' is not given for appservice registration.
2016-07-18 10:44:56 +01:00
Richard van der Hoff 6344db659f Fix a doc-comment
The `store` in a handler is a generic DataStore, not just an events.StateStore.
2016-07-18 09:48:10 +01:00
Will Hunt 511a52afc8 Use body.get to check for 'user' 2016-07-16 18:44:08 +01:00
Will Hunt e885e2a623 Fall back to 'username' if 'user' is not given for appservice reg. 2016-07-16 18:33:48 +01:00
Mark Haines d137e03231 Fix 500 ISE when sending alias event without a state_key 2016-07-15 18:58:25 +01:00
Erik Johnston f52565de50 Fix /purge_history bug
This was caused by trying to insert duplicate backward extremeties
2016-07-15 14:23:15 +01:00
Erik Johnston a2d288c6a9 Merge pull request #923 from matrix-org/erikj/purge_history
Various purge_history fixes
2016-07-15 13:23:29 +01:00
Erik Johnston bd7c51921d Merge pull request #919 from matrix-org/erikj/auth_fix
Various auth.py fixes.
2016-07-15 11:38:33 +01:00
Erik Johnston 978fa53cc2 Pull out min stream_ordering from ex_outlier_stream 2016-07-15 10:22:30 +01:00
Erik Johnston eec9609e96 event_backwards_extremeties may not be empty 2016-07-15 10:22:09 +01:00
Erik Johnston 9e1b43bcbf Comment 2016-07-15 09:29:54 +01:00
Erik Johnston a3036ac37e Merge pull request #921 from matrix-org/erikj/account_deactivate
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
Erik Johnston ebdafd8114 Check sender signed event 2016-07-14 17:03:24 +01:00
Erik Johnston a98d215204 Add filter param to /messages API 2016-07-14 16:30:56 +01:00
Erik Johnston d554ca5e1d Add support for filters in paginate_room_events 2016-07-14 15:59:04 +01:00
Erik Johnston 209e04fa11 Merge pull request #918 from negzi/bugfix_for_token_expiry
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
Erik Johnston e5142f65a6 Add 'contains_url' to filter 2016-07-14 15:35:48 +01:00
Erik Johnston b64aa6d687 Add sender and contains_url field to events table 2016-07-14 15:35:43 +01:00
Erik Johnston 848d3bf2e1 Add hs object 2016-07-14 10:25:52 +01:00
Erik Johnston b55c770271 Only accept password auth 2016-07-14 10:00:38 +01:00
Erik Johnston d543b72562 Add an /account/deactivate endpoint 2016-07-14 09:56:53 +01:00
Negar Fazeli 0136a522b1 Bug fix: expire invalid access tokens 2016-07-13 15:00:37 +02:00
Erik Johnston 2cb758ac75 Check if alias event's state_key matches sender's domain 2016-07-13 13:12:25 +01:00
Erik Johnston 560c71c735 Check creation event's room_id domain matches sender's 2016-07-13 13:07:19 +01:00
David Baker a37ee2293c Merge pull request #915 from matrix-org/dbkr/more_requesttokens
Add requestToken endpoints
2016-07-13 11:51:46 +01:00
David Baker c55ad2e375 be more pythonic 2016-07-12 14:15:10 +01:00
David Baker aaa9d9f0e1 on_OPTIONS isn't neccessary 2016-07-12 14:13:14 +01:00
David Baker 75fa7f6b3c Remove other debug logging 2016-07-12 14:08:57 +01:00
David Baker a5db0026ed Separate out requestTokens to separate handlers 2016-07-11 09:57:07 +01:00
David Baker 9c491366c5 Oops, remove debug logging 2016-07-11 09:07:40 +01:00
David Baker 385aec4010 Implement https://github.com/matrix-org/matrix-doc/pull/346/files 2016-07-08 17:42:48 +01:00
Mark Haines dfde67a6fe Add a comment explaining allow_none 2016-07-08 15:57:06 +01:00
Mark Haines 10c843fcfb Ensure that the guest user is in the database when upgrading accounts 2016-07-08 15:15:55 +01:00
Erik Johnston 58930da52b Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-07-08 14:11:37 +01:00
Erik Johnston 0870588c20 Merge branch 'hotfixes-v0.16.1' 2016-07-08 13:22:32 +01:00
Erik Johnston f90cf150e2 Bump version and changelog 2016-07-07 16:33:00 +01:00
Erik Johnston 067596d341 Fix bug where we did not correctly explode when multiple user_ids were set in macaroon 2016-07-07 16:22:24 +01:00
Erik Johnston b92e7955be Comment 2016-07-07 11:42:15 +01:00
Erik Johnston c98e1479bd Return 400 rather than 500 2016-07-07 11:41:07 +01:00
Erik Johnston 67f2c901ea Add rest servlet. Fix SQL. 2016-07-06 15:56:59 +01:00
Erik Johnston eef7778af9 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/test2 2016-07-06 14:50:22 +01:00
Erik Johnston a17e7caeb7 Merge branch 'erikj/shared_secret' into erikj/test2 2016-07-06 14:46:31 +01:00
Erik Johnston f0c06ac65c Merge pull request #909 from matrix-org/erikj/shared_secret
Add an admin option to shared secret registration (breaks backwards compat)
2016-07-06 14:08:51 +01:00
Erik Johnston 76b18df3d9 Check that there are no null bytes in user and passsword 2016-07-06 11:17:53 +01:00
Erik Johnston 0da24cac8b Add null separator to hmac 2016-07-06 11:05:16 +01:00
Kent Shikama 8d9a884cee
Update password config comment
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-06 12:18:19 +09:00
Erik Johnston be3548f7e1 Remove spurious txn 2016-07-05 17:46:51 +01:00
Erik Johnston 4adf93e0f7 Fix for postgres 2016-07-05 17:34:25 +01:00
Erik Johnston 651faee698 Add an admin option to shared secret registration 2016-07-05 17:30:22 +01:00
Erik Johnston caf33b2d9b Protect password when registering using shared secret 2016-07-05 17:18:19 +01:00
Erik Johnston 8f8798bc0d Add ReadWriteLock for pagination and history prune 2016-07-05 15:30:25 +01:00
Erik Johnston 7335f0adda Add ReadWriteLock 2016-07-05 15:23:17 +01:00
David Baker ef535178ff Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server
requestToken update
2016-07-05 15:13:34 +01:00
Mark Haines 04dee11e97 Merge pull request #906 from matrix-org/markjh/faster_events_around
Use a query that postgresql optimises better for get_events_around
2016-07-05 14:48:34 +01:00
Mark Haines dd2ccee27d Fix typo 2016-07-05 14:06:07 +01:00
Mark Haines b6b0132ac7 Make get_events_around more efficient on sqlite3 2016-07-05 13:55:18 +01:00
Kent Shikama 252ee2d979
Remove default password pepper string 2016-07-05 19:15:51 +09:00
Kent Shikama 14362bf359
Fix password config 2016-07-05 19:12:53 +09:00
Kent Shikama 1ee2584307
Fix pep8 2016-07-05 19:01:00 +09:00
Kent Shikama 507b8bb091
Add comment to prompt changing of pepper 2016-07-05 18:42:35 +09:00
Mark Haines d44d11d864 Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer 2016-07-05 10:39:13 +01:00
Erik Johnston 2d21d43c34 Add purge_history API 2016-07-05 10:28:51 +01:00
Mark Haines 0fb76c71ac Use different SQL for postgres and sqlite3 for when using multicolumn indexes 2016-07-04 19:44:55 +01:00
Kent Shikama 8bdaf5f7af
Add pepper to password hashing
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-05 02:13:52 +09:00
Erik Johnston a67bf0b074 Add storage function to purge history for a room 2016-07-04 16:02:50 +01:00
Mark Haines f18d7546c6 Use a query that postgresql optimises better for get_events_around 2016-07-04 15:48:25 +01:00
Kent Shikama bb069079bb
Fix style violations
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-04 22:07:11 +09:00
Kent Shikama 2e5a31f197 Use .get() instead of [] to access password_hash 2016-07-04 22:00:13 +09:00
Kent Shikama fc8007dbec
Optionally include password hash in createUser endpoint
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
David Baker be8be535f7 requestToken update
Don't send requestToken request to untrusted ID servers

Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
2016-06-30 17:51:28 +01:00
Erik Johnston f328d95cef Feature: Add deactivate account admin API
Allows server admins to "deactivate" accounts, which:

- Revokes all access tokens
- Removes all threepids
- Removes password

The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Erik Johnston f52cb4cd78 Remove race 2016-06-29 15:24:50 +01:00
Mark Haines 6783534a0f Merge pull request #886 from matrix-org/markjh/async_commit
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
Erik Johnston a70688445d Implement purge_media_cache admin API 2016-06-29 14:57:59 +01:00
Erik Johnston 314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
David Baker db74dcda5b Merge pull request #894 from matrix-org/dbkr/push_room_naming
Use similar naming we use in email notifs for push
2016-06-28 10:12:24 +01:00
Matthew Hodgson 63bb8f0df9 remove vector.im from default secondary DS list 2016-06-27 13:13:33 +04:00
David Baker 0b640aa56b even more pep8 2016-06-24 11:47:11 +01:00
David Baker aa3a4944d5 more pep8 2016-06-24 11:45:23 +01:00
David Baker 46b7362304 pep8 2016-06-24 11:44:57 +01:00
David Baker 870c45913e Use similar naming we use in email notifs for push
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00