5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
f3eead0660
Allow paginating both forwards and backwards
2016-09-15 10:15:37 +01:00
4131381123
Remove support for aggregate room lists
2016-09-15 09:28:15 +01:00
6a5ded5988
Ensure we don't mutate the cache of push rules
2016-09-15 09:16:13 +01:00
4f181f361d
Accept optional token to public room list
2016-09-15 09:08:57 +01:00
c566f0ee17
Calculate the public room list from a stream_ordering
2016-09-14 17:42:47 +01:00
772c6067a3
Refactor public rooms to not pull out the full state for each room
2016-09-14 17:29:25 +01:00
baffe96d95
Add a room visibility stream
2016-09-14 17:29:19 +01:00
264a48aedf
Merge pull request #1117 from matrix-org/erikj/fix_state
...
Ensure we don't mutate state cache entries
2016-09-14 16:50:37 +01:00
ed992ae6ba
Add a DB index to figure out past state at a stream ordering in a room
2016-09-14 16:20:27 +01:00
e0b6db29ed
Split out public room list into a worker process
2016-09-14 14:42:51 +01:00
a70a43bc51
Move RoomListHandler into a separate file
2016-09-14 14:07:37 +01:00
f2b2cd8eb4
Amalgamate two identical consecutive if statements
2016-09-14 11:16:22 +01:00
00f51493f5
Fix reindex
2016-09-14 10:18:30 +01:00
d5ae1f1291
Ensure we don't mutate state cache entries
2016-09-14 10:03:48 +01:00
1b01488d27
Merge pull request #1111 from matrix-org/matthew/device-ids
...
make device IDs more useful for human disambiguation
2016-09-14 01:18:02 +01:00
0f73f0e70e
Merge pull request #1116 from matrix-org/paul/tiny-fixes
...
Fix typo "persiting"
2016-09-13 13:27:42 +01:00
ca35e54d6b
Fix typo "persiting"
2016-09-13 13:26:33 +01:00
ad816b0add
Limit how often we ask for keys from dead servers
2016-09-13 11:53:50 +01:00
43253c10b8
Remove redundant event_auth index
2016-09-13 11:47:48 +01:00
18ab019a4a
Move the E2E key handling into the e2e handler
2016-09-13 11:35:35 +01:00
76b09c29b0
Merge pull request #1110 from matrix-org/markjh/e2e_timeout
...
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
ba6bc2faa0
Merge pull request #1109 from matrix-org/erikj/partial_indices
...
Add WHERE clause support to index creation
2016-09-13 09:06:16 +01:00
edbcb4152b
make device IDs more useful for human disambiguation
2016-09-13 00:02:39 +01:00
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
b17af156c7
Remove where clause
2016-09-12 17:05:54 +01:00
1c9da43a95
Merge pull request #1108 from matrix-org/erikj/create_dm
...
Add is_direct param to /createRoom
2016-09-12 16:57:16 +01:00
0b32bb20bb
Index contains_url for file search queries
2016-09-12 16:57:05 +01:00
c94de0ab60
Add WHERE clause support to index creation
2016-09-12 16:55:01 +01:00
502c901e11
Merge pull request #1107 from matrix-org/erikj/backfill_none
...
Fix backfill when cannot find an event.
2016-09-12 16:48:01 +01:00
48a5a7552d
Add is_direct param to /createRoom
2016-09-12 16:34:20 +01:00
706b5d76ed
Fix backfill when cannot find an event.
...
`get_pdu` can succeed but return None.
2016-09-12 14:59:51 +01:00
7c679b1118
Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
...
Create new index concurrently
2016-09-12 14:38:40 +01:00
d080b3425c
Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
...
Optimise /notifications query
2016-09-12 14:34:12 +01:00
03a98aff3c
Create new index concurrently
2016-09-12 14:27:01 +01:00
fa20c9ce94
Change the index to be stream_ordering, highlight
2016-09-12 14:04:08 +01:00
5ef5435529
Remove unused import
2016-09-12 13:32:58 +01:00
aa7b890cfe
Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
...
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
7cd6edb947
Use register_background_index_update
2016-09-12 12:54:48 +01:00
0294c14ec4
Add back in query change
2016-09-12 12:43:56 +01:00
7fe42cf949
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast
2016-09-12 12:37:09 +01:00
15ca0c6a4d
Make reindex happen in bg
2016-09-12 12:36:36 +01:00
a232e06100
Fix direct to device messages recieved over federation to notify sync
2016-09-12 12:30:46 +01:00
4a32d25d4c
Merge branch 'develop' into markjh/bearer_token
2016-09-12 11:14:56 +01:00
31f85f9db9
Add comments to existing schema deltas that used "CREATE INDEX" directly
2016-09-12 11:00:26 +01:00
ec609f8094
Fix unit tests
2016-09-12 10:46:02 +01:00
54417999b6
Revert "Add index to event_push_actions"
2016-09-12 10:39:55 +01:00
45dc260060
Merge pull request #1101 from matrix-org/erikj/state_types_idx
...
Change state fetch query for postgres to be faster
2016-09-12 10:20:38 +01:00
d1c217c823
Merge pull request #1097 from matrix-org/erikj/replication_typing_rest
...
Correctly handle typing stream id resetting
2016-09-12 10:10:15 +01:00
897d57bc58
Change state fetch query for postgres to be faster
...
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
555460ae1b
Merge pull request #1095 from matrix-org/erikj/batch_edus
...
Clobber EDUs in send queue
2016-09-12 08:04:15 +01:00
4162f820ff
Merge pull request #1100 from VShell/fix-cas
...
Conform better to the CAS protocol specification
2016-09-09 21:57:42 +01:00
29205e9596
Conform better to the CAS protocol specification
...
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.
Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
b91e2833b3
Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast
2016-09-09 19:11:34 +01:00
f2acc3dcf9
Add index to event_push_actions
...
and remove room_id caluse so it uses it
Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
3ddec016ff
Merge branch 'develop' into markjh/bearer_token
2016-09-09 18:51:22 +01:00
8e01263587
Allow clients to supply access_tokens as headers
...
Clients can continue to supply access tokens as query parameters
or can supply the token as a header:
Authorization: Bearer <access_token_goes_here>
This matches the ouath2 format of
https://tools.ietf.org/html/rfc6750#section-2.1
2016-09-09 18:17:42 +01:00
3265def8c7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus
2016-09-09 18:06:01 +01:00
af4701b311
Fix incorrect attribute name
2016-09-09 17:36:56 +01:00
44330a21e9
Comment
2016-09-09 17:22:07 +01:00
464ffd1b5e
Comment
2016-09-09 17:17:23 +01:00
327425764e
Add edu.type as part of key. Remove debug logging
2016-09-09 17:13:30 +01:00
dbff7e9436
Merge pull request #1096 from matrix-org/markjh/get_access_token
...
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
a4339de9de
Correctly handle typing stream id resetting
2016-09-09 16:44:26 +01:00
8aee5aa068
Add helper function for getting access_tokens from requests
...
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
52b2318777
Clobber EDUs in send queue
2016-09-09 15:59:08 +01:00
56f38d1776
Merge pull request #1091 from matrix-org/paul/third-party-lookup
...
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
8cb252d00c
Merge pull request #1094 from matrix-org/paul/get-state-whole-event
...
Allow clients to ask for the whole of a single state event
2016-09-09 15:26:17 +01:00
776594f99d
Log if rejecting 3PE query metadata result due to type check
2016-09-09 15:11:31 +01:00
ed44c475d8
Reject malformed 3PE query metadata results earlier in AS API handling code
2016-09-09 15:07:04 +01:00
ab80d5e0a9
Drop replication log levels
2016-09-09 14:56:50 +01:00
f25d74f69c
Minor fixes from PR comments
2016-09-09 14:54:16 +01:00
d271383e63
Filter returned events for client-facing format
2016-09-09 14:40:15 +01:00
0fc0a3bdff
Allow clients to specify the format a room state event is returned in
2016-09-09 14:34:29 +01:00
6c4d582144
Deduplicate presence in _update_states
2016-09-09 14:28:22 +01:00
685da5a3b0
Merge pull request #1092 from matrix-org/erikj/transaction_queue_check
...
Check if destination is ready for retry earlier
2016-09-09 13:49:56 +01:00
a6c6750166
Check if destination is ready for retry earlier
2016-09-09 13:46:05 +01:00
bdbcfc2a80
appease pep8
2016-09-09 13:31:39 +01:00
6eb0c8a2e4
Python isn't JavaScript; have to quote dict keys
2016-09-09 13:31:17 +01:00
6b54fa81de
Merge pull request #1089 from matrix-org/markjh/direct_to_device_stream
...
Track the max device stream_id in a separate table,
2016-09-09 13:31:07 +01:00
25eb769b26
Efficiency fix for lookups of a single protocol
2016-09-09 13:25:02 +01:00
3328428d05
Allow lookup of a single 3PE protocol query metadata
2016-09-09 13:19:04 +01:00
4598682b43
Fix tightloop on sending transaction
2016-09-09 13:12:53 +01:00
033d43e419
Don't corrupt shared cache on subsequent protocol requests
2016-09-09 13:10:36 +01:00
647c724573
Use the previous MAX value if any to set the stream_id
2016-09-09 11:52:44 +01:00
a15ba15e64
Merge pull request #1088 from matrix-org/erikj/transaction_queue_check
...
Correctly guard against multiple concurrent transactions
2016-09-09 11:49:06 +01:00
6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
d2688d7f03
Correctly guard against multiple concurrent transactions
2016-09-09 11:44:36 +01:00
303b6f29f0
Merge pull request #1087 from matrix-org/markjh/reapply_delta
...
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
1fe7ca1362
Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop
2016-09-09 11:15:40 +01:00
66efcbbff1
Bump changelog and version
2016-09-09 11:07:39 +01:00
0877157353
Just move the schema and add some DROPs
2016-09-09 11:04:47 +01:00
2ffec928e2
Reduce batch size to be under SQL limit
2016-09-09 11:03:31 +01:00
b390756150
Update last_device_stream_id_by_dest if there is nothing to send
2016-09-09 11:00:15 +01:00
b8f84f99ff
Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
...
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
43b77c5d97
Only catch databas errors
2016-09-08 17:44:21 +01:00
2f267ee160
Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client
2016-09-08 17:43:53 +01:00
7d5b142547
Add a stub run_upgrade
2016-09-08 17:39:11 +01:00
c3276aef25
Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from
...
Fix /notifications API when used with `from` param
2016-09-08 17:35:35 +01:00
fa722a699c
Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop
2016-09-08 17:35:16 +01:00
5c688739d6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups
2016-09-08 16:52:09 +01:00
ebb46497ba
Add delta file
2016-09-08 16:38:54 +01:00
91ec972277
Merge pull request #1084 from matrix-org/markjh/direct_to_device_wildcard
...
Support wildcard device_ids for direct to device messages
2016-09-08 16:28:08 +01:00
5beda10bbd
Reindex state_groups_state after pruning
2016-09-08 16:18:01 +01:00
257025ac89
Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
...
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
3f9889bfd6
Use parse_string
2016-09-08 15:51:10 +01:00
caa22334b3
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-09-08 15:33:00 +01:00
b152ee71fe
Bump version and changelog
2016-09-08 15:23:44 +01:00
a1c8f268e5
Support wildcard device_ids for direct to device messages
2016-09-08 15:13:05 +01:00
8b93af662d
Check the user_id for presence/typing matches origin
2016-09-08 15:07:38 +01:00
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
4ef222ab61
Implement `only=highlight` on `/notifications`
2016-09-08 13:43:35 +01:00
61cd9af09b
Log delta files we're applying
2016-09-08 13:40:46 +01:00
791658b576
Add server param to /publicRooms
2016-09-08 11:53:05 +01:00
c5b49eb7ca
Fix /notifications API when used with `from` param
2016-09-08 09:40:10 +01:00
b568ca309c
Temporarily disable sequential scans for state fetching
2016-09-08 09:38:54 +01:00
3c320c006c
Merge pull request #1077 from matrix-org/markjh/device_logging
...
Log the types and values when failing to store devices
2016-09-07 18:24:24 +01:00
85b51fdd6b
Log the types and values when failing to store devices
2016-09-07 17:19:18 +01:00
43954d000e
Add a new method to enqueue the device messages rather than sending a dummy EDU
2016-09-07 16:10:51 +01:00
2a0159b8ae
Fix the stream change cache to work over replication
2016-09-07 15:58:00 +01:00
cb98ac261b
Move the check for federated device_messages.
...
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
513188aa56
Comment
2016-09-07 14:53:23 +01:00
fadb01551a
Add appopriate framing clause
2016-09-07 14:39:01 +01:00
d25c20ccbe
Use windowing function to make use of index
2016-09-07 14:22:22 +01:00
7d893beebe
Comment the add_messages storage functions
2016-09-07 12:03:37 +01:00
94a83b534f
Merge pull request #1065 from matrix-org/erikj/state_storage
...
Move to storing state_groups_state as deltas
2016-09-07 09:39:58 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
e020834e4f
Add storage methods for federated device messages
2016-09-06 15:12:13 +01:00
2ad72da931
Add tables for federated device messages
...
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
3c4208a057
Record counts of state changes
2016-09-06 11:31:01 +01:00
f4164edb70
Move _add_messages_to_device_inbox_txn into a separate method
2016-09-06 11:26:37 +01:00
2eed4d7af4
Merge pull request #1072 from matrix-org/erikj/presence_fiddle
...
Fiddle should_notify to better report stats
2016-09-06 10:57:48 +01:00
438ef47637
Short circuit if presence is the same
2016-09-06 10:28:35 +01:00
74a3b4a650
Fiddle should_notify to better report stats
2016-09-06 10:23:38 +01:00
d51b8a1674
Add quotes and be explicity about script-src
2016-09-05 17:35:01 +01:00
662b031a30
Allow PDF to be rendered from media repo
2016-09-05 17:25:26 +01:00
0595413c0f
Scale the batch size so that we're not bitten by the minimum
2016-09-05 15:49:57 +01:00
a7032abb2e
Correctly handle reindexing state groups that already have an edge
2016-09-05 15:07:23 +01:00
9e6d88f4e2
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-05 15:01:33 +01:00
8c93e0bae7
Merge pull request #1070 from matrix-org/erikj/presence_stats
...
Record why we have chosen to notify
2016-09-05 15:01:30 +01:00
70332a12dd
Take value in a better way
2016-09-05 14:57:14 +01:00
373654c635
Comment about sqlite and WITH RECURSIVE
2016-09-05 14:50:36 +01:00
485d999c8a
Correctly delete old state groups in purge history API
2016-09-05 14:49:08 +01:00
69054e3d4c
Record why we have chosen to notify
2016-09-05 14:12:11 +01:00
69a2d4e38c
Use get_joined_users_from_context instead of manually looking up hosts
2016-09-05 13:44:40 +01:00
f12993ec16
Bump changelog and version
2016-09-05 13:01:14 +01:00
bb36b93f71
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-05 11:53:11 +01:00
d87b87adf7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/bulk_push
2016-09-05 11:52:49 +01:00
caed150363
Remove unused imports
2016-09-05 10:52:01 +01:00
80a6a445fa
Only fetch local pushers
2016-09-05 10:43:32 +01:00
628e65721b
Add comments
2016-09-05 10:41:27 +01:00
274c2f50a5
Merge pull request #1067 from matrix-org/markjh/idempotent
...
Fix membership changes to be idempotent
2016-09-05 10:21:25 +01:00
a99e933550
Add upgrade script that will slowly prune state_groups_state entries
2016-09-05 10:05:36 +01:00
3847fa38c4
Make bulk_get_push_rules_for_room use get_joined_users_from_context cache
2016-09-05 10:02:38 +01:00
f2690c6423
Fix membership changes to be idempotent
2016-09-02 19:23:22 +01:00
81b94c5750
Merge pull request #1066 from matrix-org/markjh/direct_to_device_lowerbound
...
Only return new device messages in /sync
2016-09-02 16:18:34 +01:00
65fa37ac5e
Only return new device messages in /sync
2016-09-02 15:50:37 +01:00
3baf641a48
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-02 14:54:07 +01:00
c0238ecbed
Explicitly specify state_key for history_visibility fetching
2016-09-02 14:53:46 +01:00
273b6bcf22
Merge pull request #1064 from matrix-org/erikj/on_receive_check
...
Only check if host is in room if we have state and auth_chain
2016-09-02 14:43:35 +01:00
f7f1027d3d
Comment on when auth chain and state are None
2016-09-02 14:42:38 +01:00
34e5e17f91
Comment
2016-09-02 14:26:07 +01:00
b96c6c3185
Docstrings
2016-09-02 14:19:22 +01:00
1ffe9578d1
Merge pull request #1063 from matrix-org/erikj/pull_out_ids_only
...
Only pull out IDs from DB for /state_ids/ request
2016-09-02 14:16:28 +01:00
cce957e254
Bump max_entries on get_destination_retry_timings
2016-09-02 14:08:33 +01:00
bd9b8d87ae
Only check if host is in room if we have state and auth_chain
2016-09-02 13:40:28 +01:00
657847e4c6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-02 11:04:48 +01:00
965168a842
Merge branch 'develop' into markjh/direct_to_device_synchrotron
2016-09-02 10:59:24 +01:00
2854ee2a52
Only pull out IDs from DB for /state_ids/ request
2016-09-02 10:53:36 +01:00
598317927c
Limit the length of state chains
2016-09-02 10:41:38 +01:00
7ed5acacf4
Fix up the calls to the notifier for device messages
2016-09-01 18:08:40 +01:00
051a9ea921
Linearize state resolution to help caches
2016-09-01 14:55:03 +01:00
265d847ffd
Fix typo in log line
2016-09-01 14:50:06 +01:00
f4778d4cd9
Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into develop
2016-09-01 14:40:36 +01:00
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
44982606ee
Merge pull request #1060 from matrix-org/erikj/state_ids
...
Assign state groups in state handler.
2016-09-01 14:20:42 +01:00
516a272aca
Ensure we only return a validated pdu in get_pdu
2016-09-01 10:55:02 +01:00
0cfd6c3161
Use state_groups table to test existence
2016-08-31 16:25:57 +01:00
5405351b14
Lower get_linearized_receipts_for_room cache size
2016-08-31 16:19:44 +01:00
ed7a703d4c
Handle the fact that workers can't generate state groups
2016-08-31 15:53:19 +01:00
1671913287
Merge pull request #1059 from matrix-org/erikj/sent_transaction_delete
...
Clean up old sent transactions
2016-08-31 15:07:33 +01:00
f51888530d
Always specify state_group so that its in the cache
2016-08-31 14:58:11 +01:00
826ca61745
Add storage function to SlaveStore
2016-08-31 14:45:04 +01:00
fbd2615de4
Merge pull request #1057 from matrix-org/erikj/fix_email_name
...
Fix email notifs by adding missing param
2016-08-31 14:33:45 +01:00
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
ef0cc648cf
Clean up old sent transactions
2016-08-31 11:12:02 +01:00
761f9fccff
Merge pull request #1058 from matrix-org/markjh/direct_to_device_synchrotron
...
Add a replication stream for direct to device messages
2016-08-31 11:08:18 +01:00
a662252758
Return the current stream position from add_messages_to_device_inbox
2016-08-31 10:42:52 +01:00
1aa3e1d287
Add a replication stream for direct to device messages
2016-08-31 10:38:58 +01:00
1bb8ec296d
Generate state group ids in state layer
2016-08-31 10:09:46 +01:00
d80f64d370
Fix email notifs by adding missing param
2016-08-30 21:46:39 +01:00
c882783535
flake8
2016-08-30 17:20:31 +01:00
572acde483
Use None instead of the empty string
...
Change how we validate the 'url' field as a result.
2016-08-30 17:16:00 +01:00
5dc2a702cf
Make _state_groups_id_gen a normal IdGenerator
2016-08-30 16:55:11 +01:00
3e784eff74
Remove state replication stream
2016-08-30 16:51:36 +01:00
16b652f0a3
Flake8
2016-08-30 16:30:12 +01:00
e82247f990
Allow application services to have an optional 'url'
...
If 'url' is not specified, they will not be pushed for events or queries. This
is useful for bots who simply wish to reserve large chunks of user/alias
namespace, and don't care about being pushed for events.
2016-08-30 16:21:16 +01:00
c7f665d700
Merge pull request #1055 from matrix-org/erikj/occaisonally_persist
...
Occaisonally persist unpersisted presence updates
2016-08-30 15:59:00 +01:00
097330bae8
Check correct variable
2016-08-30 15:50:20 +01:00
Erik Johnston