Commit Graph

1732 Commits

Author SHA1 Message Date
Erik Johnston 5875a65253 Merge pull request #1145 from matrix-org/erikj/fix_reindex
Fix background reindex of origin_server_ts
2016-09-29 13:53:48 +01:00
Erik Johnston 9040c9ffa1 Fix background reindex of origin_server_ts
The storage function `_get_events_txn` was removed everywhere except
from this background reindex. The function was removed due to it being
(almost) completely unused while also being large and complex.
Therefore, instead of resurrecting `_get_events_txn` we manually
reimplement the bits that are needed directly.
2016-09-27 11:23:49 +01:00
Erik Johnston 4974147aa3 Remove duplication 2016-09-27 09:27:54 +01:00
Erik Johnston 13122e5e24 Remove unused variable 2016-09-27 09:21:51 +01:00
Erik Johnston cf3e1cc200 Fix perf of fetching state in SQLite 2016-09-26 17:16:24 +01:00
Erik Johnston 8009d84364 Match against event_id, rather than room_id 2016-09-21 16:46:59 +01:00
Erik Johnston dc692556d6 Remove spurious AS clause 2016-09-21 16:28:47 +01:00
Erik Johnston dc78db8c56 Update correct table 2016-09-21 15:52:44 +01:00
Erik Johnston 4f78108d8c Readd entries to public_room_list_stream that were deleted 2016-09-21 15:24:22 +01:00
Erik Johnston 0b78d8adf2 Fix _delete_old_forward_extrem_cache query 2016-09-21 15:20:56 +01:00
Erik Johnston a68807d426 Comment 2016-09-16 11:36:20 +01:00
Erik Johnston e58a9d781c Filter remote rooms lists locally 2016-09-16 10:19:32 +01:00
Erik Johnston de4f798f01 Handling expiring stream extrems correctly. 2016-09-15 17:34:59 +01:00
Erik Johnston 955f34d23e Change get_pos_of_last_change to return upper bound 2016-09-15 15:12:07 +01:00
Erik Johnston cb3edec6af Use stream_change cache to make get_forward_extremeties_for_room cache more effective 2016-09-15 14:28:13 +01:00
Erik Johnston 55e6fc917c Add cache to get_forward_extremeties_for_room 2016-09-15 14:04:28 +01:00
Erik Johnston e457034e99 Merge pull request #1121 from matrix-org/erikj/public_room_paginate
Add pagination support to publicRooms
2016-09-15 13:27:09 +01:00
Erik Johnston 211786ecd6 Stream public room changes down replication 2016-09-15 11:47:23 +01:00
Erik Johnston 4fb65a1091 Base public room list off of public_rooms stream 2016-09-15 11:27:04 +01:00
Erik Johnston c566f0ee17 Calculate the public room list from a stream_ordering 2016-09-14 17:42:47 +01:00
Erik Johnston baffe96d95 Add a room visibility stream 2016-09-14 17:29:19 +01:00
Erik Johnston ed992ae6ba Add a DB index to figure out past state at a stream ordering in a room 2016-09-14 16:20:27 +01:00
Erik Johnston 00f51493f5 Fix reindex 2016-09-14 10:18:30 +01:00
Erik Johnston d5ae1f1291 Ensure we don't mutate state cache entries 2016-09-14 10:03:48 +01:00
Erik Johnston b17af156c7 Remove where clause 2016-09-12 17:05:54 +01:00
Erik Johnston 0b32bb20bb Index contains_url for file search queries 2016-09-12 16:57:05 +01:00
Erik Johnston c94de0ab60 Add WHERE clause support to index creation 2016-09-12 16:55:01 +01:00
Erik Johnston 7c679b1118 Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
Create new index concurrently
2016-09-12 14:38:40 +01:00
Erik Johnston d080b3425c Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
Optimise /notifications query
2016-09-12 14:34:12 +01:00
Erik Johnston 03a98aff3c Create new index concurrently 2016-09-12 14:27:01 +01:00
Erik Johnston fa20c9ce94 Change the index to be stream_ordering, highlight 2016-09-12 14:04:08 +01:00
Erik Johnston 5ef5435529 Remove unused import 2016-09-12 13:32:58 +01:00
Mark Haines aa7b890cfe Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
Erik Johnston 7cd6edb947 Use register_background_index_update 2016-09-12 12:54:48 +01:00
Erik Johnston 0294c14ec4 Add back in query change 2016-09-12 12:43:56 +01:00
Erik Johnston 7fe42cf949 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast 2016-09-12 12:37:09 +01:00
Erik Johnston 15ca0c6a4d Make reindex happen in bg 2016-09-12 12:36:36 +01:00
Mark Haines a232e06100 Fix direct to device messages recieved over federation to notify sync 2016-09-12 12:30:46 +01:00
Mark Haines 31f85f9db9 Add comments to existing schema deltas that used "CREATE INDEX" directly 2016-09-12 11:00:26 +01:00
Erik Johnston 54417999b6 Revert "Add index to event_push_actions" 2016-09-12 10:39:55 +01:00
Erik Johnston 897d57bc58 Change state fetch query for postgres to be faster
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
David Baker b91e2833b3 Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast 2016-09-09 19:11:34 +01:00
David Baker f2acc3dcf9 Add index to event_push_actions
and remove room_id caluse so it uses it

Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
Mark Haines 647c724573 Use the previous MAX value if any to set the stream_id 2016-09-09 11:52:44 +01:00
Mark Haines 6a6cbfcf1e Track the max_stream_device_id in a separate table, since we delete from the inbox table 2016-09-09 11:48:23 +01:00
Mark Haines 303b6f29f0 Merge pull request #1087 from matrix-org/markjh/reapply_delta
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
Erik Johnston 1fe7ca1362 Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop 2016-09-09 11:15:40 +01:00
Mark Haines 0877157353 Just move the schema and add some DROPs 2016-09-09 11:04:47 +01:00
Erik Johnston 2ffec928e2 Reduce batch size to be under SQL limit 2016-09-09 11:03:31 +01:00
Matthew Hodgson b8f84f99ff Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00