99bbd90b0d
Always run txn.after_callbacks
2016-08-15 09:45:44 +01:00
8a57cc3123
Add missing database corruption recovery case
...
Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
2016-08-14 11:50:22 -07:00
dfaf0fee31
Log the value which is observed in the first place.
...
The name 'result' is of bool type and has no len property,
resulting in a TypeError. Futhermore in the flow control
conn.response is observed and hence should be reported.
Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:49:05 +02:00
e380538b59
Fix AttributeError when bind_dn is not defined.
...
In case one does not define bind_dn in ldap configuration, filter
attribute is not declared. Since auth code only uses ldap_filter attribute
when according LDAP mode is selected, it is safe to only declare the
attribute in that case.
Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:48:33 +02:00
4e1cebd56f
Make synchrotron accept /events
2016-08-12 15:31:44 +01:00
866a5320de
Dont invoke get_handlers fromClientV1RestServlet
...
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
448ac6cf0d
Only process one local membership event per room at a time
2016-08-12 09:32:19 +01:00
832799dbff
Merge pull request #997 from Half-Shot/develop
...
Don't change status_msg on /sync
2016-08-11 14:10:55 +01:00
b4ecf0b886
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-11 14:09:13 +01:00
5b5148b7ec
Synced up synchrotron set_state with PresenceHandler set_state
2016-08-11 11:48:30 +01:00
1d40373c9d
Include prev_content in redacted state events
2016-08-11 10:24:41 +01:00
c315922b5f
PEP8
2016-08-10 16:34:10 +01:00
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
5aeadb7414
Merge pull request #999 from matrix-org/erikj/measure_more
...
Measure federation send transaction resources
2016-08-10 14:16:14 +01:00
487bc49bf8
Don't stop on 4xx series errors
2016-08-10 13:39:12 +01:00
739ea29d1e
Also check if server is in the room
2016-08-10 13:32:23 +01:00
ea8c4094db
Also pull out rejected events
2016-08-10 13:26:13 +01:00
7f41bcbeec
Correctly auth /event/ requests
2016-08-10 13:22:20 +01:00
11fdfaf03b
Only resign our own events
2016-08-10 13:16:58 +01:00
2510db3e76
Don't change status_msg on /sync
2016-08-10 12:59:59 +01:00
f91df1f761
Store if we fail to fetch an event from a destination
2016-08-10 11:31:46 +01:00
3bc9629be5
Measure federation send transaction resources
2016-08-10 10:56:38 +01:00
d45489474d
Merge pull request #996 from matrix-org/erikj/tls_error
...
Don't print stack traces when failing to get remote keys
2016-08-10 10:56:08 +01:00
fa1ce4d8ad
Don't print stack traces when failing to get remote keys
2016-08-10 10:44:37 +01:00
79ebfbe7c6
/login: Respond with a 403 when we get an invalid m.login.token
2016-08-09 16:29:28 +01:00
cd41c6ece2
Merge pull request #995 from matrix-org/rav/clean_up_cas_login
...
Clean up CAS login code
2016-08-09 10:21:56 +01:00
27771b2495
Merge pull request #994 from matrix-org/rav/fix_cas_login
...
Fix CAS login
2016-08-08 17:47:45 +01:00
d3250499c1
Merge pull request #993 from matrix-org/rav/fix_token_login
...
Fix token login
2016-08-08 17:43:02 +01:00
65666fedd5
Clean up CAS login code
...
Remove some apparently unused code.
Clean up parse_cas_response, mostly to catch the exception if the CAS response
isn't valid XML.
2016-08-08 17:17:25 +01:00
0682ca04b3
Fix CAS login
...
Attempting to log in with CAS was giving a 500 error.
2016-08-08 17:01:30 +01:00
6fe6a6f029
Fix login with m.login.token
...
login with token (as used by CAS auth) was broken by 067596d
2016-08-08 16:40:39 +01:00
27fe3e2d4f
Bump changelog and version
2016-08-08 11:31:12 +01:00
3410142741
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0
2016-08-08 11:24:53 +01:00
7c1a92274c
Make psutil optional
2016-08-08 11:12:21 +01:00
f5deaff424
Merge pull request #991 from matrix-org/erikj/retry_make
...
Retry joining via other servers if first one failed. Fix some other bugs.
2016-08-05 18:21:27 +01:00
5f360182c6
Fix a couple of python bugs
2016-08-05 18:08:32 +01:00
46453bfc2f
Retry joining via other servers if first one failed
2016-08-05 18:02:03 +01:00
6bf6bc1d1d
Merge pull request #990 from matrix-org/erikj/fed_vers
...
Add federation /version API
2016-08-05 16:59:16 +01:00
24f36469bc
Add federation /version API
2016-08-05 16:36:07 +01:00
597c79be10
Change the way we specify if we require auth or not
2016-08-05 16:17:04 +01:00
e6021c370e
Merge pull request #989 from matrix-org/erikj/raise_404
...
Raise 404 when couldn't find event
2016-08-05 15:54:16 +01:00
a8b946decb
Raise 404 when couldn't find event
2016-08-05 15:31:02 +01:00
5bcccfde6c
Don't include html comments in description
2016-08-05 14:45:11 +01:00
4d87d3659a
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0
2016-08-05 13:19:22 +01:00
93acf49e9b
Fix backfill auth events
2016-08-05 12:59:04 +01:00
b2c290a6e5
Bump version and changelog
2016-08-05 11:16:54 +01:00
499dc1b349
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0
2016-08-05 11:15:48 +01:00
2d4de61fb7
Fix typo
2016-08-05 10:48:56 +01:00
fccadb7719
Check if we already have the events returned by /state/
2016-08-05 10:43:47 +01:00
f0fa66f495
Delete more tables
2016-08-05 10:40:08 +01:00
1515d1b581
Fallback to /state/ on both 400 and 404
2016-08-05 10:24:23 +01:00
a2b7102eea
Tweak integrity error recovery to work as intended
2016-08-04 20:38:08 -07:00
a5d7968b3e
Merge pull request #973 from matrix-org/erikj/xpath_fix
...
Change the way we summarize URLs
2016-08-04 16:37:25 +01:00
b5525c76d1
Typo
2016-08-04 16:10:08 +01:00
e97648c4e2
Test summarization
2016-08-04 16:09:09 +01:00
835ceeee76
Merge pull request #983 from matrix-org/erikj/retry_on_integrity_error
...
Retry event persistence on IntegrityError
2016-08-04 15:40:03 +01:00
b3682df2ca
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/xpath_fix
2016-08-04 15:29:45 +01:00
8ad8490cff
Fix typo
2016-08-04 15:21:29 +01:00
59fa91fe88
Retry event persistence on IntegrityError
...
Due to a bug in the porting script some backfilled events were not
correctly persisted, causing irrecoverable IntegrityErrors on future
attempts to persist those events.
This commit adds a retry mechanism invoked upon IntegrityError,
where when retried the tables are purged for all references to the
events being persisted.
2016-08-04 15:02:15 +01:00
257c41cc2e
Fix typos.
2016-08-04 14:05:45 +01:00
b4e2290d89
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api
2016-08-04 14:04:35 +01:00
e3ee63578f
Tidy up get_events
2016-08-04 14:01:18 +01:00
f131cd9e53
keys/query: Omit device displayname if null
...
... which makes it more consistent with user displaynames.
2016-08-04 10:59:51 +01:00
edb33eb163
Rename fields to _ids
2016-08-03 17:19:15 +01:00
bcc9cda8ca
Fix copy + paste fails
2016-08-03 17:17:26 +01:00
98385888b8
PEP8
2016-08-03 15:42:08 +01:00
68264d7404
Include device name in /keys/query response
...
Add an 'unsigned' section which includes the device display name.
2016-08-03 15:42:08 +01:00
91fa69e029
keys/query: return all users which were asked for
...
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
2016-08-03 15:41:44 +01:00
4c56bedee3
Actually call get_room_state
2016-08-03 15:04:29 +01:00
520ee9bd2c
Fix syntax error
2016-08-03 15:03:15 +01:00
a60a2eaa02
Comment
2016-08-03 14:52:43 +01:00
e3a720217a
Add /state_ids federation API
...
The new API only returns the event_ids for the state, as most
requesters will already have the vast majority of the events already.
2016-08-03 14:47:37 +01:00
530bc862dc
Merge branch 'rav/null_default_device_displayname' into develop
2016-08-03 14:30:32 +01:00
a6f5cc65d9
PEP8
2016-08-03 14:30:06 +01:00
e555bc6551
Merge branch 'rav/refactor_device_query' into develop
2016-08-03 14:26:01 +01:00
a843868fe9
E2eKeysHandler: minor tweaks
...
PR feedback
2016-08-03 14:24:33 +01:00
80ad710217
Remove other bit of deduplication
2016-08-03 13:25:59 +01:00
4fec5e57be
Default device_display_name to null
...
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
a8a32d2714
Ensure we only persist an event once at a time
2016-08-03 11:23:39 +01:00
921f17f938
Merge branch 'develop' into rav/refactor_device_query
2016-08-03 11:12:47 +01:00
58c9653c6b
Don't infer paragrahs from newlines
2016-08-02 18:50:24 +01:00
6b58ade2f0
Comment on why we clone
2016-08-02 18:41:22 +01:00
9e66c58ceb
Spelling.
2016-08-02 18:37:31 +01:00
f83f5fbce8
Make it actually compile
2016-08-02 18:32:42 +01:00
aecaec3e10
Change the way we summarize URLs
...
Using XPath is slow on some machines (for unknown reasons), so use a
different approach to get a list of text nodes.
Try to generate a summary that respect paragraph and then word
boundaries, adding ellipses when appropriate.
2016-08-02 18:25:53 +01:00
1efee2f52b
E2E keys: Make federation query share code with client query
...
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
49e047c55e
Bump version and changelog
2016-08-02 17:10:26 +01:00
59a2c6d60e
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0
2016-08-02 17:07:54 +01:00
c9154b970c
Don't double wrap 200
2016-08-02 16:45:53 +01:00
b3d5c4ad9d
Fix response cache
2016-08-02 16:42:21 +01:00
8f650bd338
Bump changeog and version
2016-08-02 15:43:52 +01:00
7b0f6293f2
Merge pull request #940 from matrix-org/erikj/fed_state_cache
...
Cache federation state responses
2016-08-02 15:21:37 +01:00
342e072024
Merge pull request #967 from matrix-org/erikj/fed_reader
...
Split out the federation reading portions into a separate.
2016-08-02 13:59:47 +01:00
54de6a812a
Merge branch 'develop' into dbkr/fix_add_email_on_register
2016-08-02 10:18:15 +01:00
733bf44290
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader
2016-08-02 09:56:31 +01:00
986615b0b2
Move e2e query logic into a handler
2016-08-01 18:02:07 +01:00
b260f92936
Ignore AlreadyCalled errors on timer cancel
2016-07-31 16:00:12 +01:00
271d3e7865
Fix adding emails on registration
...
Synapse was not adding email addresses to accounts registered with an email address, due to too many different variables called 'result'. Rename both of them. Also remove the defer.returnValue() with no params because that's not a thing.
2016-07-29 15:25:24 +01:00
18b7eb830b
Merge pull request #958 from matrix-org/paul/SYN-738
...
Forbid non-ASes from registering users whose names begin with '_'
2016-07-29 14:10:45 +01:00
2a9ce8c422
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader
2016-07-29 11:33:46 +01:00
cbea0c7044
Merge pull request #964 from matrix-org/erikj/fed_join_fix
...
Handle the case of missing auth events when joining a room
2016-07-29 11:33:16 +01:00
5aa024e501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader
2016-07-29 11:24:56 +01:00
c51a52f300
Mention that func will fetch auth events
2016-07-29 11:17:04 +01:00
3d13c3a295
Update docstring
2016-07-29 10:45:05 +01:00
8dad08a950
Fix SQL to supply arguments in the same order
2016-07-29 09:57:13 +01:00
0a7d3cd00f
Create separate methods for getting messages to push
...
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.
The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.
The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
ec8b217722
Add destination retry to slave store
2016-07-28 17:35:53 +01:00
76b89d0edb
Add slace storage functions for public room list
2016-07-28 17:03:40 +01:00
370135ad0b
Comment get_unread_push_actions_for_user_in_range function
2016-07-28 16:47:37 +01:00
0fcbca531f
Add get_auth_chain to slave store
2016-07-28 16:36:28 +01:00
1e2740caab
Handle the case of missing auth events when joining a room
2016-07-28 16:08:33 +01:00
6ede23ff1b
Add more key storage funcs into slave store
2016-07-28 15:41:26 +01:00
5fb41a955c
Merge branch 'release-v0.17.0' of github.com:matrix-org/synapse into develop
2016-07-28 14:56:32 +01:00
4329f20e3f
Merge pull request #962 from matrix-org/markjh/retry
...
Fix retry utils to check if the exception is a subclass of CME
2016-07-28 11:12:19 +01:00
a285194021
Merge branch 'erikj/key_client_fix' of github.com:matrix-org/synapse into release-v0.17.0
2016-07-28 10:47:06 +01:00
bf81e38d36
Fix retry utils to check if the exception is a subclass of CME
2016-07-28 10:47:02 +01:00
78cac3e594
Merge pull request #941 from matrix-org/erikj/key_client_fix
...
Send the correct host header when fetching keys
2016-07-28 10:46:36 +01:00
7871790db1
Bump version and changelog
2016-07-28 10:38:56 +01:00
18e044628e
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0
2016-07-28 10:37:43 +01:00
b557b682d9
Merge pull request #961 from matrix-org/dbkr/fix_push_invite_name
...
Don't include name of room for invites in push
2016-07-28 10:36:41 +01:00
389c890f14
Don't include name of room for invites in push
...
Avoids insane pushes like, "Bob invited you to invite from Bob"
2016-07-28 10:20:47 +01:00
cd8738ab63
Merge pull request #960 from matrix-org/rav/support_r0.2
...
Add r0.2.0 to the "supported versions" list
2016-07-28 10:14:33 +01:00
f6f8f81a48
Add r0.1.0 to the "supported versions" list
2016-07-28 10:14:07 +01:00
ecd5e6bfa4
Typo
2016-07-28 10:04:52 +01:00
fda078f995
Add r0.2.0 to the "supported versions" list
2016-07-28 09:14:21 +01:00
40e539683c
Merge pull request #956 from matrix-org/rav/check_device_id_on_key_upload
...
Make the device id on e2e key upload optional
2016-07-27 18:16:57 +01:00
05f6447301
Forbid non-ASes from registering users whose names begin with '_' (SYN-738)
2016-07-27 17:54:26 +01:00
5238960850
Bump CHANGES and version
2016-07-27 17:33:09 +01:00
ccec25e2c6
key upload tweaks
...
1. Add v2_alpha URL back in, since things seem to be using it.
2. Don't reject the request if the device_id in the upload request fails to
match that in the access_token.
2016-07-27 16:41:06 +01:00
29b25d59c6
Merge branch 'develop' into markjh/verify
...
Conflicts:
synapse/crypto/keyring.py
2016-07-27 15:11:02 +01:00
884b800899
Merge pull request #955 from matrix-org/markjh/only_from2
...
Add a couple more checks to the keyring
2016-07-27 15:08:22 +01:00
09d31815b4
Merge pull request #954 from matrix-org/markjh/even_more_fixes
...
Fix a couple of bugs in the transaction and keyring code
2016-07-27 15:08:14 +01:00
fe1b369946
Clean up verify_json_objects_for_server
2016-07-27 14:10:43 +01:00
26cb0efa88
SQL syntax fix
2016-07-27 12:30:22 +01:00
d47115ff8b
Delete e2e keys on device delete
2016-07-27 12:24:52 +01:00
2e3d90d67c
Make the device id on e2e key upload optional
...
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.
For active access_tokens without an associated device_id, we ought to register
the device in the devices table.
Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
a4b06b619c
Add a couple more checks to the keyring
2016-07-26 19:50:11 +01:00
c63b1697f4
Merge pull request #952 from matrix-org/markjh/more_fixes
...
Check if the user is banned when handling 3pid invites
2016-07-26 19:20:56 +01:00
87ffd21b29
Fix a couple of bugs in the transaction and keyring code
2016-07-26 19:19:08 +01:00
eb359eced4
Add `create_requester` function
...
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
c824b29e77
Check if the user is banned when handling 3pid invites
2016-07-26 16:39:14 +01:00
33d7776473
Fix typo
2016-07-26 13:32:15 +01:00
9ad8d9b17c
Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device
2016-07-26 13:29:46 +01:00
5b1825ba5b
Merge pull request #951 from matrix-org/rav/flake8
...
Fix flake8 noise
2016-07-26 13:27:54 +01:00
9c4cf83259
Merge pull request #948 from matrix-org/markjh/auth_fixes
...
Don't add rejections to the state_group, persist all rejections
2016-07-26 13:22:57 +01:00
05e7e5e972
Fix flake8 violation
...
Apparently flake8 v3 puts the error on a different line to v2. Easiest way to
make sure that happens is by putting the whole statement on one line :)
2016-07-26 11:59:08 +01:00
8e02494166
Delete refresh tokens when deleting devices
2016-07-26 11:10:37 +01:00
a6f06ce3e2
Fix how push_actions are redacted.
2016-07-26 11:05:39 +01:00
d34e9f93b7
Merge pull request #949 from matrix-org/rav/update_devices
...
Implement updates and deletes for devices
2016-07-26 10:49:55 +01:00
efeb6176c1
Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically
2016-07-26 10:49:52 +01:00
242c52d607
typo
2016-07-26 10:09:25 +02:00
012b4c1913
Implement updating devices
...
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
1b3c3e6d68
Only update the events and event_json tables for rejected events
2016-07-25 18:44:30 +01:00
33d08e8433
Log when adding listeners
2016-07-25 17:22:15 +01:00
8f7f4cb92b
Don't add the events to forward extremities if the event is rejected
2016-07-25 17:13:37 +01:00
2623cec874
Don't add rejections to the state_group, persist all rejections
2016-07-25 16:12:16 +01:00
4fcdf7b4b2
Merge pull request #946 from matrix-org/dbkr/log_recaptcha_hostname
...
Log the hostname the reCAPTCHA was completed on
2016-07-25 16:10:39 +01:00
955ef1f06c
fix: defer.returnValue takes one argument
2016-07-25 16:04:45 +01:00
2ee4c9ee02
background updates: fix assert again
2016-07-25 16:01:46 +01:00
9dbd903f41
background updates: Fix assertion to do something
2016-07-25 14:05:23 +01:00
465117d7ca
Fix background_update tests
...
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
7ed58bb347
Use get to avoid KeyErrors
2016-07-22 17:18:50 +01:00
dad2da7e54
Log the hostname the reCAPTCHA was completed on
...
This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.
2016-07-22 17:00:56 +01:00
363786845b
PEP8
2016-07-22 13:21:07 +01:00
ec5717caf5
Create index on user_ips in the background
...
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.
I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
d26b660aa6
Cache getPeer
2016-07-21 17:38:51 +01:00
aede7248ab
Split out a FederationReader process
2016-07-21 17:37:44 +01:00
55abbe1850
make /devices return a list
...
Turns out I specced this to return a list of devices rather than a dict of them
2016-07-21 15:57:28 +01:00
2c28e25bda
Merge pull request #943 from matrix-org/rav/get_device_api
...
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
1e6e370b76
Merge pull request #942 from matrix-org/rav/fix_register_deviceid
...
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
1c3c202b96
Fix PEP8 errors
2016-07-21 13:15:15 +01:00
406f7aa0f6
Implement GET /device/{deviceId}
2016-07-21 12:00:29 +01:00
c445f5fec7
storage/client_ips: remove some dead code
2016-07-21 11:58:47 +01:00
cf94a78872
Set host not path
2016-07-21 11:45:53 +01:00
1a64dffb00
Preserve device_id from first call to /register
...
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
081e5d55e6
Send the correct host header when fetching keys
2016-07-21 11:14:54 +01:00
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
e9e3eaa67d
Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips
...
Record device_id in client_ips
2016-07-20 17:38:45 +01:00
d36b1d849d
Don't explode if we have no snapshots yet
2016-07-20 16:59:52 +01:00
742056be0d
Merge pull request #937 from matrix-org/rav/register_device_on_register
...
Register a device_id in the /v2/register flow.
2016-07-20 16:51:27 +01:00
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
053e83dafb
More doc-comments
...
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
b97a1356b1
Register a device_id in the /v2/register flow.
...
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.
Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
b73dc0ef4d
Merge pull request #936 from matrix-org/erikj/log_rss
...
Add metrics for psutil derived memory usage
2016-07-20 16:32:38 +01:00
66868119dc
Add metrics for psutil derived memory usage
2016-07-20 16:00:21 +01:00
aba0b2a39b
Merge pull request #935 from matrix-org/erikj/backfill_notifs
...
Don't notify pusher pool for backfilled events
2016-07-20 13:39:16 +01:00
57dca35692
Don't notify pusher pool for backfilled events
2016-07-20 13:25:06 +01:00
c68518dfbb
Merge pull request #933 from matrix-org/rav/type_annotations
...
Type annotations
2016-07-20 12:26:32 +01:00
e967bc86e7
Merge pull request #932 from matrix-org/rav/register_refactor
...
Further registration refactoring
2016-07-20 11:03:33 +01:00
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
f91faf09b3
Comment
2016-07-20 10:18:09 +01:00
3413f1e284
Type annotations
...
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
40cbffb2d2
Further registration refactoring
...
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
b9e997f561
Merge pull request #931 from matrix-org/rav/refactor_register
...
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
9a7a77a22a
Merge pull request #929 from matrix-org/rav/support_deviceid_in_login
...
Add device_id support to /login
2016-07-19 15:53:04 +01:00
8f6281ab0c
Don't bind email unless threepid contains expected fields
2016-07-19 15:50:01 +01:00
0da0d0a29d
rest/client/v2_alpha/register.py: Refactor flow somewhat.
...
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.
The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
022b9176fe
schema fix
...
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
c41d52a042
Summary line
2016-07-19 10:28:27 +01:00
7e554aac86
Update docstring on Handlers.
...
To indicate it is deprecated.
2016-07-19 10:20:58 +01:00
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
93efcb8526
Merge pull request #928 from matrix-org/rav/refactor_login
...
Refactor login flow
2016-07-18 16:12:35 +01:00
dcfd71aa4c
Refactor login flow
...
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.
Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
fca90b3445
Merge pull request #924 from matrix-org/erikj/purge_history
...
Fix /purge_history bug
2016-07-18 15:11:11 +01:00
a292454aa1
Merge pull request #925 from matrix-org/markjh/auth_fix
...
Fix 500 ISE when sending alias event without a state_key
2016-07-18 15:04:47 +01:00
4f81edbd4f
Merge pull request #927 from Half-Shot/develop
...
Fall back to 'username' if 'user' is not given for appservice registration.
2016-07-18 10:44:56 +01:00
6344db659f
Fix a doc-comment
...
The `store` in a handler is a generic DataStore, not just an events.StateStore.
2016-07-18 09:48:10 +01:00
511a52afc8
Use body.get to check for 'user'
2016-07-16 18:44:08 +01:00
e885e2a623
Fall back to 'username' if 'user' is not given for appservice reg.
2016-07-16 18:33:48 +01:00
d137e03231
Fix 500 ISE when sending alias event without a state_key
2016-07-15 18:58:25 +01:00
f52565de50
Fix /purge_history bug
...
This was caused by trying to insert duplicate backward extremeties
2016-07-15 14:23:15 +01:00
a2d288c6a9
Merge pull request #923 from matrix-org/erikj/purge_history
...
Various purge_history fixes
2016-07-15 13:23:29 +01:00
bd7c51921d
Merge pull request #919 from matrix-org/erikj/auth_fix
...
Various auth.py fixes.
2016-07-15 11:38:33 +01:00
978fa53cc2
Pull out min stream_ordering from ex_outlier_stream
2016-07-15 10:22:30 +01:00
eec9609e96
event_backwards_extremeties may not be empty
2016-07-15 10:22:09 +01:00
9e1b43bcbf
Comment
2016-07-15 09:29:54 +01:00
a3036ac37e
Merge pull request #921 from matrix-org/erikj/account_deactivate
...
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
ebdafd8114
Check sender signed event
2016-07-14 17:03:24 +01:00
a98d215204
Add filter param to /messages API
2016-07-14 16:30:56 +01:00
d554ca5e1d
Add support for filters in paginate_room_events
2016-07-14 15:59:04 +01:00
209e04fa11
Merge pull request #918 from negzi/bugfix_for_token_expiry
...
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
e5142f65a6
Add 'contains_url' to filter
2016-07-14 15:35:48 +01:00
b64aa6d687
Add sender and contains_url field to events table
2016-07-14 15:35:43 +01:00
848d3bf2e1
Add hs object
2016-07-14 10:25:52 +01:00
b55c770271
Only accept password auth
2016-07-14 10:00:38 +01:00
d543b72562
Add an /account/deactivate endpoint
2016-07-14 09:56:53 +01:00
0136a522b1
Bug fix: expire invalid access tokens
2016-07-13 15:00:37 +02:00
2cb758ac75
Check if alias event's state_key matches sender's domain
2016-07-13 13:12:25 +01:00
560c71c735
Check creation event's room_id domain matches sender's
2016-07-13 13:07:19 +01:00
a37ee2293c
Merge pull request #915 from matrix-org/dbkr/more_requesttokens
...
Add requestToken endpoints
2016-07-13 11:51:46 +01:00
c55ad2e375
be more pythonic
2016-07-12 14:15:10 +01:00
aaa9d9f0e1
on_OPTIONS isn't neccessary
2016-07-12 14:13:14 +01:00
75fa7f6b3c
Remove other debug logging
2016-07-12 14:08:57 +01:00
a5db0026ed
Separate out requestTokens to separate handlers
2016-07-11 09:57:07 +01:00
9c491366c5
Oops, remove debug logging
2016-07-11 09:07:40 +01:00
385aec4010
Implement https://github.com/matrix-org/matrix-doc/pull/346/files
2016-07-08 17:42:48 +01:00
dfde67a6fe
Add a comment explaining allow_none
2016-07-08 15:57:06 +01:00
10c843fcfb
Ensure that the guest user is in the database when upgrading accounts
2016-07-08 15:15:55 +01:00
58930da52b
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-07-08 14:11:37 +01:00
0870588c20
Merge branch 'hotfixes-v0.16.1'
2016-07-08 13:22:32 +01:00
f90cf150e2
Bump version and changelog
2016-07-07 16:33:00 +01:00
067596d341
Fix bug where we did not correctly explode when multiple user_ids were set in macaroon
2016-07-07 16:22:24 +01:00
b92e7955be
Comment
2016-07-07 11:42:15 +01:00
c98e1479bd
Return 400 rather than 500
2016-07-07 11:41:07 +01:00
67f2c901ea
Add rest servlet. Fix SQL.
2016-07-06 15:56:59 +01:00
eef7778af9
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/test2
2016-07-06 14:50:22 +01:00
a17e7caeb7
Merge branch 'erikj/shared_secret' into erikj/test2
2016-07-06 14:46:31 +01:00
f0c06ac65c
Merge pull request #909 from matrix-org/erikj/shared_secret
...
Add an admin option to shared secret registration (breaks backwards compat)
2016-07-06 14:08:51 +01:00
76b18df3d9
Check that there are no null bytes in user and passsword
2016-07-06 11:17:53 +01:00
0da24cac8b
Add null separator to hmac
2016-07-06 11:05:16 +01:00
8d9a884cee
Update password config comment
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-06 12:18:19 +09:00
be3548f7e1
Remove spurious txn
2016-07-05 17:46:51 +01:00
4adf93e0f7
Fix for postgres
2016-07-05 17:34:25 +01:00
651faee698
Add an admin option to shared secret registration
2016-07-05 17:30:22 +01:00
caf33b2d9b
Protect password when registering using shared secret
2016-07-05 17:18:19 +01:00
8f8798bc0d
Add ReadWriteLock for pagination and history prune
2016-07-05 15:30:25 +01:00
7335f0adda
Add ReadWriteLock
2016-07-05 15:23:17 +01:00
ef535178ff
Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server
...
requestToken update
2016-07-05 15:13:34 +01:00
04dee11e97
Merge pull request #906 from matrix-org/markjh/faster_events_around
...
Use a query that postgresql optimises better for get_events_around
2016-07-05 14:48:34 +01:00
dd2ccee27d
Fix typo
2016-07-05 14:06:07 +01:00
b6b0132ac7
Make get_events_around more efficient on sqlite3
2016-07-05 13:55:18 +01:00
252ee2d979
Remove default password pepper string
2016-07-05 19:15:51 +09:00
14362bf359
Fix password config
2016-07-05 19:12:53 +09:00
1ee2584307
Fix pep8
2016-07-05 19:01:00 +09:00
507b8bb091
Add comment to prompt changing of pepper
2016-07-05 18:42:35 +09:00
d44d11d864
Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer
2016-07-05 10:39:13 +01:00
2d21d43c34
Add purge_history API
2016-07-05 10:28:51 +01:00
0fb76c71ac
Use different SQL for postgres and sqlite3 for when using multicolumn indexes
2016-07-04 19:44:55 +01:00
8bdaf5f7af
Add pepper to password hashing
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-05 02:13:52 +09:00
a67bf0b074
Add storage function to purge history for a room
2016-07-04 16:02:50 +01:00
f18d7546c6
Use a query that postgresql optimises better for get_events_around
2016-07-04 15:48:25 +01:00
bb069079bb
Fix style violations
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-04 22:07:11 +09:00
2e5a31f197
Use .get() instead of [] to access password_hash
2016-07-04 22:00:13 +09:00
fc8007dbec
Optionally include password hash in createUser endpoint
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
be8be535f7
requestToken update
...
Don't send requestToken request to untrusted ID servers
Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
2016-06-30 17:51:28 +01:00
f328d95cef
Feature: Add deactivate account admin API
...
Allows server admins to "deactivate" accounts, which:
- Revokes all access tokens
- Removes all threepids
- Removes password
The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
f52cb4cd78
Remove race
2016-06-29 15:24:50 +01:00
6783534a0f
Merge pull request #886 from matrix-org/markjh/async_commit
...
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
a70688445d
Implement purge_media_cache admin API
2016-06-29 14:57:59 +01:00
314b146b2e
Track approximate last access time for remote media
2016-06-29 11:41:20 +01:00
db74dcda5b
Merge pull request #894 from matrix-org/dbkr/push_room_naming
...
Use similar naming we use in email notifs for push
2016-06-28 10:12:24 +01:00
63bb8f0df9
remove vector.im from default secondary DS list
2016-06-27 13:13:33 +04:00
0b640aa56b
even more pep8
2016-06-24 11:47:11 +01:00
aa3a4944d5
more pep8
2016-06-24 11:45:23 +01:00
46b7362304
pep8
2016-06-24 11:44:57 +01:00
870c45913e
Use similar naming we use in email notifs for push
...
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
05f1a4596a
Merge branch 'master' into develop
2016-06-23 11:17:48 +01:00
b5fb7458d5
Actually we need to order these properly
...
otherwise we'll end up returning the wrong 20
2016-06-22 18:07:14 +01:00
f73fdb04a6
Style
2016-06-22 17:51:40 +01:00
3a4120e49a
Put most recent 20 messages in notif
...
Fixes https://github.com/vector-im/vector-web/issues/1648
2016-06-22 17:47:18 +01:00
0a32208e5d
Rework ldap integration with ldap3
...
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.
Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
employing the `user_filter`, then tries the bind when a single
matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
localpart and `user_base`
Offer support for STARTTLS on a plain connection.
The configuration was changed to reflect these new possibilities.
Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2016-06-22 17:51:59 +02:00
774f3a692c
Merge pull request #889 from matrix-org/markjh/synctl_workers
...
Optionally start or stop workers in synctl.
2016-06-21 17:58:19 +01:00
5cc7564c5c
Optionally start or stop workers in synctl.
...
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.
The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
2016-06-21 16:38:05 +01:00
0fe0b0eeb6
Merge pull request #888 from matrix-org/markjh/content_repo
...
Remove the legacy v0 content upload API.
2016-06-21 14:01:01 +01:00
13e334506c
Remove the legacy v0 content upload API.
...
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
d5fb561709
Optionally make committing to postgres asynchronous.
...
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
d8ec81cc31
Merge pull request #879 from matrix-org/erikj/linearize_fed_server
...
Linearize some federation endpoints based on (origin, room_id)
2016-06-20 17:34:29 +01:00
bc72d381b2
Merge branch 'release-v0.16.1' of github.com:matrix-org/synapse
2016-06-20 14:18:04 +01:00
4d362a61ea
Bump version and changelog
2016-06-20 14:17:42 +01:00
41e4b2efea
Add the create_profile method back since the tests use it
2016-06-17 19:20:47 +01:00
0c13d45522
Add a comment on why we don't create a profile for upgrading users
2016-06-17 19:18:53 +01:00
9f1800fba8
Remove registered_users from the distributor.
...
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.
This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
8f4a9bbc16
Linearize some federation endpoints based on (origin, room_id)
2016-06-17 16:43:45 +01:00
9ba2bf1570
Merge pull request #878 from matrix-org/erikj/ujson
...
Disable responding with canonical json for federation
2016-06-17 16:22:12 +01:00
120c238705
Disable responding with canonical json for federation
2016-06-17 16:10:37 +01:00
1c1f633b13
Merge pull request #877 from matrix-org/erikj/frozen_default
...
Turn use_frozen_events off by default
2016-06-17 15:33:55 +01:00
6660f37558
Merge pull request #876 from matrix-org/erikj/sign_own
...
Only re-sign our own events
2016-06-17 15:23:20 +01:00
3e41de05cc
Turn use_frozen_events off by default
2016-06-17 15:11:22 +01:00
2884712ca7
Only re-sign our own events
2016-06-17 14:47:33 +01:00
ded01c3bf6
Fix ``KeyError: 'msgtype'``. Use ``.get``
...
Fixes a key error where the mailer tried to get the ``msgtype`` of an
event that was missing a ``msgtype``.
```
File "synapse/push/mailer.py", line 264, in get_notif_vars
File "synapse/push/mailer.py", line 285, in get_message_vars
File ".../frozendict/__init__.py", line 10, in __getitem__
return self.__dict[key]
KeyError: 'msgtype'
```
2016-06-17 13:49:16 +01:00
8c75040c25
Fix setting gc thresholds in the workers
2016-06-17 11:48:12 +01:00
a352b68acf
Use worker_ prefixes for worker config, use existing support for multiple config files
2016-06-16 17:29:50 +01:00
364d616792
Access the event_cache_size directly from the server object.
...
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
bde13833cb
Access replication_url from the worker config directly
2016-06-16 12:44:40 +01:00
80a1bc7db5
Comment on what's going on in clobber_with_worker_config
2016-06-16 11:29:45 +01:00
f1f70bf4b5
Merge remote-tracking branch 'origin/develop' into markjh/worker_config
2016-06-16 11:20:17 +01:00
dbb5a39b64
Add worker config module
2016-06-16 11:09:15 +01:00
885ee861f7
Inline the synchrotron and pusher configs into the main config
2016-06-16 11:06:12 +01:00
486b9a6a2d
Merge pull request #873 from vt0r/bugfix/bcrypt-utf8-encode
...
Fix TypeError in call to bcrypt.hashpw
2016-06-16 10:12:17 +01:00
ed5f43a55a
Fix TypeError in call to bcrypt.hashpw
...
- At the very least, this TypeError caused logins to fail on my own
running instance of Synapse, and the simple (explicit) UTF-8
conversion resolved login errors for me.
Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>
2016-06-16 00:43:42 -04:00
09a17f965c
Line lengths
2016-06-15 16:58:12 +01:00
1e9026e484
Handle floats as img widths
2016-06-15 16:58:05 +01:00
a60169ea09
Handle og props with not content
2016-06-15 16:57:48 +01:00
0ef0655b83
Bump version and changelog
2016-06-15 15:50:48 +01:00
d41a1a91d3
Linearize fetching of gaps on incoming events
...
This potentially stops the server from doing multiple requests for the
same data.
2016-06-15 15:16:14 +01:00
15bf3e3376
Merge pull request #870 from matrix-org/rav/work_around_tls_bug
...
Work around TLS bug in twisted
2016-06-15 13:28:13 +01:00
b31c49d676
Correctly mark backfilled events as backfilled
2016-06-15 10:59:08 +01:00
255c229f23
Work around TLS bug in twisted
...
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473 . Hopefully this fixes
https://matrix.org/jira/browse/SYN-700 .
2016-06-15 10:39:08 +01:00
d12134ce37
Merge pull request #868 from matrix-org/erikj/invalid_id
...
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 15:09:07 +01:00
36e2aade87
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 13:25:29 +01:00
33546b58aa
point to the CAPTCHA docs
2016-06-12 23:11:29 +01:00
41b35412bf
Merge pull request #863 from matrix-org/markjh/load_config
...
Add function to load config without generating it
2016-06-10 11:12:32 +01:00
7dbb473339
Add function to load config without generating it
...
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
16a8884233
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-06-09 16:57:33 +01:00
a31befbcd0
Bump version and changelog
2016-06-09 13:23:41 +01:00
5ee5b655b2
Merge pull request #862 from matrix-org/erikj/media_remote_error
...
502 on /thumbnail when can't contact remote server
2016-06-09 13:11:02 +01:00
0a43219a27
Merge pull request #860 from negzi/bug_fix_get_or_create_user
...
Fix a bug caused by a change in auth_handler function
2016-06-09 11:40:12 +01:00
eba4ff1bcb
502 on /thumbnail when can't contact remote server
2016-06-09 11:29:43 +01:00
95f305c35a
Remove redundant exception log in /events
2016-06-09 11:15:04 +01:00
6e7dc7c7dd
Fix a bug caused by a change in auth_handler function
...
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
b7fbc9bd95
Merge pull request #859 from matrix-org/erikj/public_room_performance
...
Pull full state for each room all at once
2016-06-08 16:36:06 +01:00
919a2c74f6
Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop
2016-06-08 16:26:26 +01:00
81c07a32fd
Pull full state for each room all at once
2016-06-08 15:51:49 +01:00
690029d1a3
Don't make rooms visibile by default
2016-06-08 14:47:42 +01:00
d88faf92d1
Fix up federation PublicRoomList
2016-06-08 14:39:31 +01:00
958c968d02
Merge pull request #856 from matrix-org/erikj/fed_pub_rooms
...
Enable auth on /publicRoom endpoints
2016-06-08 14:36:09 +01:00
746b2f5657
Merge pull request #854 from matrix-org/erikj/federation_logging
...
Add some logging for when servers ask for missing events
2016-06-08 14:31:41 +01:00
efeabd3180
Log user that is making /publicRooms calls
2016-06-08 14:23:15 +01:00
1fd6eb695d
Enable auth on federation PublicRoomList
2016-06-08 14:15:18 +01:00
17aab5827a
Add some logging for when servers ask for missing events
2016-06-08 11:55:31 +01:00
1a815fb04f
Don't hit DB for noop replications queries
2016-06-08 11:33:30 +01:00
bab916bccc
Bump version and changelog to v0.16.0-rc2
2016-06-08 11:05:45 +01:00
e0fda29f94
Merge pull request #850 from matrix-org/erikj/gc_threshold
...
Add gc_threshold to pusher and synchrotron
2016-06-08 10:05:56 +01:00
0f2165ccf4
Don't track total objects as its too expensive to calculate
2016-06-07 17:00:45 +01:00
18f0cc7d99
Record some more GC metrics
2016-06-07 16:55:49 +01:00
2d1d1025fa
Add gc_threshold to pusher and synchrotron
2016-06-07 16:26:25 +01:00
b01e71e719
Merge pull request #849 from matrix-org/erikj/gc_threshold
...
Allow setting of gc.set_thresholds
2016-06-07 16:07:54 +01:00
dded389ac1
Allow setting of gc.set_thresholds
2016-06-07 15:45:56 +01:00
0b2158719c
Remove dead code.
...
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.
The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.
The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
48e65099b5
Also record number of unreachable objects
2016-06-07 13:40:22 +01:00
75331c5fca
Change the way we do stats
2016-06-07 13:33:13 +01:00
8c966fbd51
Merge pull request #771 from matrix-org/erikj/gc_tick
...
Manually run GC on reactor tick.
2016-06-07 13:18:36 +01:00
88625db05f
Notify users for events in rooms they join.
...
Change how the notifier updates the map from room_id to user streams on
receiving a join event. Make it update the map when it notifies for the
join event, rather than using the "user_joined_room" distributor signal
2016-06-07 11:33:36 +01:00
84379062f9
Fix AS retries, but with correct ordering
2016-06-07 10:24:50 +01:00
310197bab5
Fix AS retries
2016-06-07 09:34:50 +01:00
b0932b34cb
Merge pull request #845 from matrix-org/markjh/synchrotron_presence
...
Fix a KeyError in the synchrotron presence
2016-06-06 16:52:27 +01:00
4a5bbb1941
Fix a KeyError in the synchrotron presence
2016-06-06 16:37:12 +01:00
87f60e7053
Merge pull request #844 from matrix-org/markjh/yield_on_sleep
...
Yield on the sleeps intended to backoff replication
2016-06-06 16:20:54 +01:00
5ef84da4f1
Yield on the sleeps intended to backoff replication
2016-06-06 16:05:28 +01:00
216a05b3e3
.values() returns list of sets
2016-06-06 16:00:09 +01:00
96dc600579
Fix typos
2016-06-06 15:44:41 +01:00
377eb480ca
Fire after 30s not 8h
2016-06-06 15:14:21 +01:00
e4134c5e13
Merge pull request #841 from matrix-org/erikj/event_counter
...
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
7aa778fba9
Add metric counter for number of persisted events
2016-06-06 11:58:09 +01:00
70aee0717c
Add events to cache when we persist them
2016-06-06 11:34:53 +01:00
3210f4c385
Merge pull request #836 from matrix-org/erikj/change_event_cache
...
Change the way we cache events
2016-06-03 18:47:46 +01:00
cffe46408f
Don't rely on options when inserting event into cache
2016-06-03 18:25:21 +01:00
ac9716f154
Fix spelling
2016-06-03 18:10:00 +01:00
8f79084bd4
Add get_presence_list_accepted to the broken caches in synchrotron
2016-06-03 18:03:40 +01:00
10ea3f46ba
Change the way we cache events
2016-06-03 17:57:50 +01:00
f6be734be9
Merge pull request #835 from matrix-org/erikj/get_event_txn
...
Remove event fetching from DB threads
2016-06-03 17:30:00 +01:00
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
72c4d482e9
3rd time lucky: we'd already calculated it above
2016-06-03 16:39:50 +01:00
fbf608decb
Oops, we're using the dict form
2016-06-03 16:38:39 +01:00
06d40c8b98
Add substitutions to email notif From
2016-06-03 16:31:23 +01:00
5f88549f4a
Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop
2016-06-03 16:27:24 +01:00
ca457f594e
Merge pull request #831 from matrix-org/markjh/synchrotronII
...
Add a separate process that can handle /sync requests
2016-06-03 16:06:25 +01:00
21961c93c7
Bump changelog and version
2016-06-03 15:33:14 +01:00
48340e4f13
Clear the list of ongoing syncs on shutdown
2016-06-03 15:02:27 +01:00
fcbc282f56
Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0
2016-06-03 15:02:04 +01:00
51773bcbaf
Merge pull request #832 from matrix-org/erikj/presence_coount
...
Change def of small delta in presence stream. Add metrics.
2016-06-03 14:57:00 +01:00
da491e75b2
Appease flake8
2016-06-03 14:56:36 +01:00
0b3c80a234
Use ClientIpStore to record client ips
2016-06-03 14:55:01 +01:00
dd6f62ed99
Merge branch 'develop' into markjh/synchrotronII
2016-06-03 14:51:33 +01:00
eef541a291
Move insert_client_ip to a separate class
2016-06-03 14:42:35 +01:00
80aade3805
Send updates to the syncing users every ten seconds or immediately if they've just come online
2016-06-03 14:24:19 +01:00
ab116bdb0c
Fix typo
2016-06-03 14:03:42 +01:00
4ce84a1acd
Change metric style
2016-06-03 13:49:16 +01:00
a7ff5a1770
Presence metrics. Change def of small delta
2016-06-03 13:40:55 +01:00
8d740132f4
Merge branch 'develop' into matthew/brand-from-header
2016-06-03 12:14:18 +01:00
3b096c5f5c
Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop
2016-06-03 12:00:33 +01:00
43b7f371f5
Merge pull request #830 from matrix-org/erikj/metrics_perf
...
Change CacheMetrics to be quicker
2016-06-03 11:57:29 +01:00
abb151f3c9
Add a separate process that can handle /sync requests
2016-06-03 11:57:26 +01:00
4982b28868
Merge pull request #829 from matrix-org/erikj/poke_notifier
...
Poke notifier on next reactor tick
2016-06-03 11:52:10 +01:00
d06f2a229e
Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
...
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
58a224a651
Pull out update_results_dict
2016-06-03 11:47:07 +01:00
20eccd84d4
Merge pull request #827 from matrix-org/markjh/more_slaved_methods
...
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
722472b48c
Merge pull request #825 from matrix-org/erikj/cache_push_rules
...
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
73c7112433
Change CacheMetrics to be quicker
...
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
b09f348530
Merge pull request #824 from matrix-org/markjh/slaved_presence_store
...
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
4c04222fa5
Poke notifier on next reactor tick
2016-06-03 11:24:16 +01:00
ccb56fc24b
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:20:23 +01:00
81cf449daa
Add methods to events, account data and receipt slaves
...
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
e043ede4a2
Small optimisation to CacheListDescriptor
2016-06-03 11:19:22 +01:00
b821c839dc
Merge pull request #823 from matrix-org/markjh/more_slaved_stores
...
Add slaved stores for filters, tokens, and push rules
2016-06-03 11:17:43 +01:00
597013caa5
Make cachedList go a bit faster
2016-06-03 11:13:29 +01:00
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
59f2d73522
Remove unnecessary sets
2016-06-03 11:05:45 +01:00
9c26b390a2
Only get local users
2016-06-03 11:04:31 +01:00
f88d747f79
Add a comment explaining why the filter cache doesn't need exipiring
2016-06-03 11:03:10 +01:00
065e739d6e
Merge pull request #811 from matrix-org/erikj/state_users_in_room
...
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
696d7c5937
Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
...
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
0eae075723
Add slaved stores for filters, tokens, and push rules
2016-06-03 10:58:03 +01:00
79d1f072f4
brand the email from header
2016-06-02 21:34:40 +01:00
6bb9aacf9d
Merge pull request #821 from matrix-org/dbkr/email_unsubscribe
...
Email unsubscribe links that don't require logging in
2016-06-02 17:44:55 +01:00
745ddb4dd0
peppate
2016-06-02 17:38:41 +01:00
7a5a5f2df2
Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error
...
Fix error in email notification string formatting
2016-06-02 17:26:06 +01:00
1f31cc37f8
Working unsubscribe links going straight to the HS
...
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
2675c1e40e
add some branding debugging
2016-06-02 17:21:12 +01:00
c71177f285
Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe
2016-06-02 17:20:56 +01:00
07a5559916
Fix error in email notification string formatting
2016-06-02 17:17:16 +01:00
56d15a0530
Store the typing users as user_id strings. ( #819 )
...
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
812b5de0fe
Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe
2016-06-02 15:33:28 +01:00
80f34d7b57
Fix setting the _clock in SQLBaseStore
2016-06-02 15:23:56 +01:00
661a540dd1
Deduplicate presence entries in sync ( #818 )
2016-06-02 15:20:28 +01:00
70599ce925
Allow external processes to mark a user as syncing. ( #812 )
...
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
2016-06-02 15:20:15 +01:00
fb2193cc63
Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler
...
Split out the auth handler
2016-06-02 14:31:35 +01:00
356f13c069
Disable INCLUDE_ALL_UNREAD_NOTIFS
2016-06-02 14:07:38 +01:00
02ac463dbf
Merge pull request #800 from matrix-org/erikj/sync_refactor
...
Refactor SyncHandler
2016-06-02 14:02:13 +01:00
c5af1b6b00
Merge pull request #814 from matrix-org/matthew/3pid_invite_auth
...
special case m.room.third_party_invite event auth to match invites,
2016-06-02 13:47:40 +01:00
3a3fb2f6f9
Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe
2016-06-02 13:35:25 +01:00
4a10510cd5
Split out the auth handler
2016-06-02 13:31:45 +01:00
f84b89f0c6
if an email pusher specifies a brand param, use it
2016-06-02 13:29:48 +01:00
a15ad60849
Email unsubscribing that may in theory, work
...
Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.
2016-06-02 11:44:15 +01:00
e793866398
Use user_id in email greeting if display name is null
2016-06-02 09:41:13 +01:00
aaa70e26a2
special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208
2016-06-01 22:13:47 +01:00
a04a2d043c
Merge pull request #807 from matrix-org/erikj/push_rules_cache
...
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
0f06b496d1
Merge pull request #806 from matrix-org/erikj/hash_cache
...
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
e0deeff23e
Fix room list spidering
2016-06-01 17:58:58 +01:00
991af8b0d6
WIP on unsubscribing email notifs without logging in
2016-06-01 17:40:52 +01:00
00c487a8db
Merge pull request #808 from matrix-org/dbkr/room_list_spider
...
Add secondary_directory_servers option to fetch room list from other servers
2016-06-01 15:32:52 +01:00
c8285564a3
Use state to calculate get_users_in_room
2016-06-01 15:25:25 +01:00
d60eed0710
Limit number of notifications in an email notification
2016-06-01 11:45:43 +01:00
43db0d9f6a
Add get_users_with_read_receipts_in_room cache
2016-06-01 10:54:32 +01:00
8e539f13c0
Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider
2016-06-01 09:54:36 +01:00
6ecb2ca4ec
pep8
2016-06-01 09:48:55 +01:00
2a449fec4d
Add cache to remote room lists
...
Poll for updates from remote servers, waiting for the poll if there's no cache entry.
2016-05-31 18:27:23 +01:00
6ca4d3ae9a
Add vector.im to default secondary_directory_servers and add comment explaining it's not a permanent solution
2016-05-31 17:24:50 +01:00
dea9f20f8c
Force boolean
2016-05-31 17:24:30 +01:00
963e3ed282
Apparently I am not permitted to have two blank lines here
2016-05-31 17:22:53 +01:00
d240796ded
Basic, un-cached support for secondary_directory_servers
2016-05-31 17:20:07 +01:00
c8c5bf950a
Fix synapse/storage/schema/delta/30/as_users.py
2016-05-31 17:10:40 +01:00
c9ca285d33
Merge pull request #805 from matrix-org/erikj/push_rules_cache
...
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
1d4ee854e2
Fix typo
2016-05-31 15:45:53 +01:00
cca0093fa9
Change fix
2016-05-31 15:44:08 +01:00
4efa389299
Fix GET /push_rules
2016-05-31 15:37:53 +01:00
aefd2d1cbc
Cache get_event_reference_hashes
2016-05-31 15:32:32 +01:00
10de8c2631
Merge pull request #804 from matrix-org/erikj/push_rules_cache
...
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
c626fc576a
Move the AS handler out of the Handlers object.
...
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
e5b0bbcd33
Add caches to bulk_get_push_rules*
2016-05-31 13:46:58 +01:00
70ecb415f5
Fix c+p fail
2016-05-31 12:00:54 +01:00
e1625d62a8
Add federation room list servlet
2016-05-31 11:55:57 +01:00
887c6e6f05
Split out the room list handler
...
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
85b992f621
Fix to allow start with postgres
2016-05-27 10:44:44 +01:00
cc84f7cb8e
Send down correct error response if user not found
2016-05-27 10:35:15 +01:00
209ba4d024
Merge pull request #795 from matrix-org/dbkr/delete_push_actions_after_a_month
...
Only delete push actions after 30 days
2016-05-24 16:22:13 +01:00
b007ee4606
Check for presence of 'avatar_url' key
2016-05-24 15:12:05 +01:00
cbf8d146ac
Merge pull request #799 from matrix-org/matthew/quieter-email-notifs
...
Tune email notifs to make them quieter:
2016-05-24 14:30:51 +01:00
faad233ea6
Change short circuit path
2016-05-24 14:27:19 +01:00
6900303997
Don't send down all ephemeral events
2016-05-24 11:44:55 +01:00
37b7e84620
Include the ts the notif was received at
2016-05-24 11:33:32 +01:00
1c5ed2a19b
Only work out newly_joined_users for incremental sync
2016-05-24 11:21:34 +01:00
b08ad0389e
Only include non-offline presence in initial sync
2016-05-24 11:15:05 +01:00
be2c677386
Spell builder correctly
2016-05-24 10:53:03 +01:00
79bea8ab9a
Inline function. Make load_filtered_recents private
2016-05-24 10:22:24 +01:00
84f94e4cbb
Add comments
2016-05-24 10:14:53 +01:00
137e6a4557
Shuffle things room
2016-05-24 09:50:55 +01:00
680f1d9387
catch thinko in presentable names
2016-05-23 22:55:11 +01:00
cb8a321bdd
fix NPE in room ordering
2016-05-23 22:54:56 +01:00
88ea5ab2c3
consistency is the better part of valour
2016-05-23 19:33:45 +01:00
989bdc9e56
Tune email notifs to make them quieter:
...
* After initial 10 minute window, only alert every 24h for room notifs
* Reset room state after 6h of idleness
* Synchronise throttles for messages sent in the same notif, so the 24 hourly notifs 'line up'
* Fix the email subjects to say what triggered the notification
* Order the rooms in reverse activity order in the email, so the 'reason' room should always come first
2016-05-23 19:24:11 +01:00
6fe04ffef2
Fix set profile error with Requester.
...
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
b791a530da
Actually make the 'read' flag correct
2016-05-23 18:48:02 +01:00
a24bc5b2dc
Add GET /notifications API
2016-05-23 18:33:51 +01:00
c0c79ef444
Add back concurrently_execute
2016-05-23 18:21:27 +01:00
b5605dfecc
Refactor SyncHandler
2016-05-23 18:08:18 +01:00
31b5395ab6
Remove debug logging
2016-05-23 16:32:01 +01:00
c2da3406fc
Oops, missing comma
2016-05-20 18:03:31 +01:00
ccffb0965d
Remove stale line
2016-05-20 17:59:10 +01:00
18d68bfee4
Handle empty events table
2016-05-20 17:58:09 +01:00
d4503e25ed
Make deleting push actions more efficient
...
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
149fa411e2
Only delete push actions after 30 days
2016-05-20 15:25:12 +01:00
332d7e9b97
Allow clients to specify a server_name to avoid 'No known servers'
...
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
6fb51eaf7b
Merge pull request #793 from matrix-org/matthew/one-push-badge-per-convo
...
increment badge count per missed convo, not per msg
2016-05-18 13:56:38 +01:00
e837df6adb
increment badge count per missed convo, not per msg
2016-05-18 11:53:25 +01:00
42368ea8db
Add desc to get_presence_for_users
2016-05-18 11:38:10 +01:00
0cb441fedd
Move typing handler out of the Handlers object
2016-05-17 15:58:46 +01:00
6a30a0bfd3
Move the functions for parsing app service config
2016-05-17 11:28:58 +01:00
523d5bcd0b
Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler
2016-05-17 10:43:58 +01:00
43e1e0489c
Merge pull request #786 from matrix-org/matthew/email_notifs_tuning
...
tune email notifs, fix CSS a bit, and add debugging details
2016-05-17 10:43:24 +01:00
526bf8126f
Remove unused get_joined_rooms_for_user
2016-05-17 10:20:51 +01:00
425e6b4983
Merge branch 'develop' into markjh/member_cleanup
2016-05-17 10:13:16 +01:00
b153f5b150
Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
...
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
3fd8a07fca
Merge pull request #788 from matrix-org/markjh/domian
...
Spell "domain" correctly
2016-05-17 10:09:34 +01:00
f68eea808a
Move SyncHandler out of the Handlers object
2016-05-16 20:19:26 +01:00
53e171f345
Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler
2016-05-16 20:08:32 +01:00
80cb9becd8
Remove get_joined_rooms_for_user from RoomMemberHandler
2016-05-16 20:06:55 +01:00
816df9f267
get_room_members is unused now
2016-05-16 19:51:43 +01:00
821306120a
Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room
2016-05-16 19:48:07 +01:00
1a3a2002ff
Spell "domain" correctly
...
s/domian/domain/g
2016-05-16 19:17:23 +01:00
e168abbcff
Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler
2016-05-16 19:08:40 +01:00
cbd2adc95e
tune email notifs, fix CSS a bit, and add debugging details
2016-05-16 18:58:38 +01:00
3b86ecfa79
Move the presence handler out of the Handlers object
2016-05-16 18:56:37 +01:00
647781ca56
Fix emailpusher import
...
Try importing at the root level rather than conditionally importing, as per comment
2016-05-16 18:41:32 +01:00
c39f305067
os.environ requires a string
2016-05-16 17:21:30 +01:00
678c8a7f1e
Merge pull request #785 from matrix-org/erikj/cache_factor_synctly
...
Make synctl read a cache factor from config file
2016-05-16 17:07:35 +01:00
c5c5a7403b
Make synctl read a cache factor from config file
2016-05-16 17:01:57 +01:00
eb79110beb
Clean up the blacklist/whitelist handling.
...
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
dd95eb4cb5
Merge branch 'develop' into matthew/preview_url_ip_whitelist
2016-05-16 12:59:41 +01:00
60d53f9e95
Count number of GC collects
2016-05-16 09:34:42 +01:00
782471b7e1
fix matrix.to URLs
2016-05-13 17:50:16 +01:00
0466454b00
Assert that stream replicated stream positions are ints
2016-05-13 17:33:44 +01:00
b3f29dc1e5
Manually expire broken caches like the who_forgot_in_room
2016-05-13 17:16:27 +01:00
f03ddc98ec
Use the SlavedAccountDataStore
2016-05-13 17:01:28 +01:00
1f71f386f6
Merge branch 'develop' into dbkr/email_notifs_on_pusher
2016-05-13 16:59:56 +01:00
206eb9fd94
Shift some of the state_group methods into the SlavedEventStore
2016-05-13 16:58:14 +01:00
7d6e89ed22
Add a comment
2016-05-13 16:31:08 +01:00
21018c2c13
Merge pull request #783 from matrix-org/markjh/slave_account_data
...
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
b7381d5338
Allow receipts for events we haven't seen in the db
2016-05-13 15:46:41 +01:00
3abab26458
Add a slaved datastore for account data
2016-05-13 15:34:06 +01:00
99b5a2e560
Merge pull request #741 from negzi/create_user_with_expiry
...
Create user with expiry
2016-05-13 14:46:53 +01:00
ba5c616ff4
Merge pull request #778 from matrix-org/erikj/add_pusher
...
Fixup add_pusher
2016-05-13 14:43:23 +01:00
0c11c1be88
Spelling
2016-05-13 14:42:25 +01:00
e00e8f2166
Merge pull request #769 from matrix-org/erikj/push_actions_delete
...
Delete old pushers
2016-05-13 14:41:36 +01:00
fd8e921b6e
Merge pull request #779 from matrix-org/erikj/receipts
...
Use tree cache for get_linearized_receipts_for_room
2016-05-13 14:41:21 +01:00
c9aff0736c
Remove topics table
2016-05-13 14:40:38 +01:00
Erik Johnston