00cb3eb24b
Cache tags and account data
2016-01-28 16:37:41 +00:00
c23a8c7833
Ensure keys to RoomStreamChangeCache are ints
2016-01-28 15:55:26 +00:00
e1941442d4
Invalidate caches properly. Remove unused arg
2016-01-28 15:02:41 +00:00
19fd425928
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync
2016-01-28 14:46:08 +00:00
167d1df699
Merge pull request #534 from matrix-org/erikj/setup
...
Add a Homeserver.setup method
2016-01-28 14:45:36 +00:00
7ed2bbeb11
Clean up a bit. Add comment
2016-01-28 14:32:05 +00:00
4e7948b47a
Allow paginating backwards from stream token
2016-01-28 11:52:34 +00:00
ba8931829b
Return correct type of token
2016-01-28 11:34:17 +00:00
61eaa6ec64
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync
2016-01-28 10:50:57 +00:00
c5e7c0e436
Up get_rooms_for_user cache size
2016-01-28 09:58:45 +00:00
e7febf4fbb
PEP8
2016-01-27 17:33:27 +00:00
aca3193efb
Use the same path for incremental with gap or without gap
2016-01-27 17:33:27 +00:00
b97f6626b6
Add cache to room stream
2016-01-27 17:33:26 +00:00
a955cbfa49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/setup
2016-01-27 17:22:35 +00:00
8c94833b72
Fix adding push rules relative to other rules
2016-01-27 10:24:20 +00:00
87f9477b10
Add a Homeserver.setup method.
...
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
2016-01-26 15:51:06 +00:00
8431f62ebb
Merge pull request #525 from matrix-org/erikj/select_many
...
Implement a `_simple_select_many_batch`
2016-01-25 16:30:36 +00:00
f091b73e69
Merge pull request #527 from matrix-org/erikj/push_cache
...
Push: Use storage apis that are cached
2016-01-25 16:16:34 +00:00
aea5da0ef6
Guard against empty iterables
2016-01-25 15:59:29 +00:00
1ebf5e3d03
Correct docstring
2016-01-25 15:53:36 +00:00
86896408b0
Add index to event_push_actions
2016-01-25 15:30:32 +00:00
53cb173663
Push: Use storage apis that are cached
2016-01-25 13:55:18 +00:00
ddd25def01
Implement a _simple_select_many_batch
2016-01-25 13:36:02 +00:00
52bdd1b834
Overlay the push_rules_enabled map for users, otherwise they won't be able to disable server default rules.
2016-01-22 14:58:19 +00:00
10f76dc5da
Make LRU cache not default to treecache & add options to use it
2016-01-22 12:10:33 +00:00
42eae4634f
Use new invalidate_many cache invalidation to invalidate the event_push_actions cache appropriately.
2016-01-22 11:22:48 +00:00
8f66fe6392
Cache get_unread_event_push_actions_by_room_for_user
2016-01-21 15:02:07 +00:00
367cfab4e6
peppate
2016-01-20 16:05:09 +00:00
d4315bbf6b
Add index by user id on receipts_linearized
2016-01-20 15:33:27 +00:00
3fa344c037
Add storage function to get all receipts for a user. Also add some cache invalidation to the receipts storage because there wasn't any, and remove a method that was unused.
2016-01-20 15:30:31 +00:00
5a7d1ecffc
Add regex cache. Only caculate push actions for users that have sent read receipts, and are on that server
2016-01-19 16:01:05 +00:00
3adcc4c86a
Return highlight_count in /sync
2016-01-19 11:35:50 +00:00
5cd2126a6a
Remove dead code
2016-01-18 16:49:46 +00:00
2c176e02ae
Make unit tests work
2016-01-18 14:48:50 +00:00
f59b564507
Make notifications go quicker
2016-01-18 14:48:29 +00:00
cc66a9a5e3
Allow filtering events for multiple users at once
2016-01-18 14:43:50 +00:00
2680043bc6
Require ID and as_token be unique for ASs
...
Defaults ID to as_token if not specified. This will change
when IDs are fully supported.
2016-01-14 14:34:01 +00:00
27927463a1
Merge pull request #494 from matrix-org/daniel/2
...
Don't start server if ASes are invalidly configured
2016-01-14 11:06:19 +00:00
fcb6df45e5
Merge pull request #493 from matrix-org/daniel/1
...
Delete unused code
2016-01-14 11:06:14 +00:00
f6fcff3602
Don't start server if ASes are invalidly configured
2016-01-13 17:09:24 +00:00
244b356a37
Delete unused code
2016-01-13 17:03:58 +00:00
9c1f853d58
Rename 'user_name' to 'user_id' in push to make it consistent with the rest of the code
2016-01-13 13:32:59 +00:00
f4dad9f639
Merge remote-tracking branch 'origin/erikj/bulk_get_push_rules' into markjh/table_name
...
Conflicts:
synapse/storage/push_rule.py
2016-01-13 11:46:07 +00:00
8740e4e94a
bulk_get_push_rules should handle empty lists
2016-01-13 11:37:17 +00:00
c0a279e808
Delete the table objects from TransactionStore
2016-01-13 11:15:20 +00:00
96e400fee5
Remove the RoomsTable object
2016-01-13 11:07:32 +00:00
a8e9e0b916
Remove the PushersTable and EventPushActionsTable objects
2016-01-12 14:41:26 +00:00
31de2953a3
Remove the PushRuleTable and PushRuleEnableTable objects
2016-01-12 14:36:16 +00:00
b5d33a656f
Postgres doesn't like booleans
2016-01-11 17:13:52 +00:00
fe56138142
Remove rogue 'admin'
2016-01-11 17:09:03 +00:00
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
daadcf36c0
This comma is actually important
2016-01-07 10:15:35 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
442fcc02f7
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-06 17:28:55 +00:00
b6a585348a
Adding is_guest here won't work because it just constructs a dict of uid -> password hash
2016-01-06 17:16:02 +00:00
5880de186b
Log when we skip daily messages
2016-01-06 13:52:16 +00:00
992928304f
Delete notifications for redacted events
2016-01-06 11:58:46 +00:00
ae1262a241
Add schema change file for is_guest flag
2016-01-06 11:58:20 +00:00
c79f221192
Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions.
2016-01-06 11:38:09 +00:00
eb03625626
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-05 18:39:50 +00:00
cfd07aafff
Allow guests to upgrade their accounts
2016-01-05 18:01:18 +00:00
4eb7b950c8
= not == in sql
2016-01-04 18:11:17 +00:00
c914d67cda
Rename event-actions to event_push_actions as per PR request
2016-01-04 14:05:37 +00:00
3051c9d002
Address minor PR issues
2016-01-04 13:39:29 +00:00
4c8f6a7e42
Insert push actions in a single db query rather than one per user/profile_tag
2015-12-22 17:04:31 +00:00
65c451cb38
Add bulk push rule evaluator which actually still evaluates rules one by one, but does far fewer db queries to fetch the rules
2015-12-22 15:19:34 +00:00
c061b47c57
Merge remote-tracking branch 'origin/develop' into store_event_actions
2015-12-21 15:30:26 +00:00
091c545c4f
pep8
2015-12-21 10:14:57 +00:00
413d0d6a24
Make unread notification count sending work: put the correct count in incremental syncs too, where necessary, and fix silly bugs like only select the event actions for that user...
2015-12-18 17:47:00 +00:00
a64f9bbfe0
Fix 500 error when back-paginating search results
...
We were mistakenly adding pagination clauses to the count query, which then
failed because the count query doesn't join to the events table.
2015-12-17 12:50:46 +00:00
42ad49f5b7
still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs).
2015-12-16 18:42:09 +00:00
4fab578b43
Merge branch 'release-v0.12.0' into develop
2015-12-16 11:18:45 +00:00
dcfc70e8ed
Allow users to change which account a 3pid is bound to
2015-12-15 17:02:21 +00:00
15e9885197
Merge branch 'release-v0.12.0' into develop
2015-12-14 14:46:55 +00:00
98dfa7d24f
Skip events that where the body, name or topic isn't a string when back populating the FTS index
2015-12-14 13:55:46 +00:00
76e69cc8de
Fix typo
2015-12-14 12:38:55 +00:00
bfc52a2342
Fix typo in sql for full text search on sqlite3
2015-12-14 11:38:11 +00:00
deeebbfcb7
Merge branch 'release-v0.12.0' into develop
2015-12-12 14:21:49 +00:00
d9a5c56930
Include approximate count of search results
2015-12-11 11:40:23 +00:00
51fb590c0e
Use more efficient query form
2015-12-11 11:12:57 +00:00
aa667ee396
Save event actions to the db
2015-12-10 17:51:15 +00:00
7d6b313312
Add caches for whether a room has been forgotten by a user
2015-12-10 17:49:34 +00:00
21f135ba76
Very first cut of calculating actions for events as they come in. Doesn't store them yet. Not very efficient.
2015-12-10 16:26:08 +00:00
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
d57c5cda71
Bump schema version.
...
As we released version 26 in v0.11.1
2015-12-04 15:28:39 +00:00
b8680b82c3
Merge pull request #414 from matrix-org/erikj/if_not_exists
...
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-03 13:21:09 +00:00
e515b48929
Just replace the table definition with the one from full_schema 16
2015-12-02 17:23:52 +00:00
8810eb8c39
Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15...
2015-12-02 17:19:11 +00:00
f721fdbf87
Merge pull request #412 from matrix-org/erikj/search
...
Search: Add prefix matching support
2015-12-02 13:56:41 +00:00
976cb5aaa8
Throw if unrecognized DB type
2015-12-02 13:50:43 +00:00
b2def42bfd
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-02 13:29:14 +00:00
b9acef5301
Fix so highlight matching works again
2015-12-02 13:28:13 +00:00
7dd6e5efca
Remove deuplication. Add comment about regex.
2015-12-02 13:09:37 +00:00
477da77b46
Search: Add prefix matching support
2015-12-02 11:40:52 +00:00
95f30ecd1f
Add API for setting account_data globaly or on a per room basis
2015-12-01 18:41:32 +00:00
4dcaa42b6d
Allow paginating search ordered by recents
2015-11-30 17:45:31 +00:00
76936f43ae
Return words to highlight in search results
2015-11-27 16:40:42 +00:00
df7cf6c0eb
Fix SQL for postgres again
2015-11-23 18:54:41 +00:00
3e573a5c6b
Fix SQL for postgres
2015-11-23 18:48:53 +00:00
7dfa455508
Remove size specifier for database column
...
Postgres doesn't support them like this.
We don't have a bool type in common between postgres and sqlite.
2015-11-23 18:35:25 +00:00
95c3306798
Merge branch 'daniel/forgetrooms' of github.com:matrix-org/synapse into daniel/forgetrooms
2015-11-19 15:00:14 -05:00
df6824a008
Ignore forgotten rooms in v2 sync
2015-11-19 14:54:47 -05:00
1cfda3d2d8
Merge branch 'develop' into daniel/forgetrooms
2015-11-19 16:53:13 +00:00
9da4c5340d
Simplify code
2015-11-19 10:07:21 -05:00
bed7889703
Apply forgetting properly to historical events
2015-11-18 18:11:12 -05:00
d153f482dd
Rename the database table
2015-11-18 15:33:02 +00:00
1c960fbb80
s/private_user_data/account_data/
2015-11-18 15:31:04 +00:00
ba26eb3d5d
Allow users to forget rooms
2015-11-17 17:17:30 -05:00
e4d622aaaf
Implementation of state rollback in /sync
...
Implementation of SPEC-254: roll back the state dictionary to how it looked at
the start of the timeline.
Merged PR https://github.com/matrix-org/synapse/pull/373
2015-11-13 10:58:56 +00:00
fddedd51d9
Fix a few race conditions in the state calculation
...
Be a bit more careful about how we calculate the state to be returned by
/sync. In a few places, it was possible for /sync to return slightly later
state than that represented by the next_batch token and the timeline. In
particular, the following cases were susceptible:
* On a full state sync, for an active room
* During a per-room incremental sync with a timeline gap
* When the user has just joined a room. (Refactor check_joined_room to make it
less magical)
Also, use store.get_state_for_events() (and thus the existing stategroups) to
calculate the state corresponding to a particular sync position, rather than
state_handler.compute_event_context(), which recalculates from first principles
(and tends to miss some state).
Merged from PR https://github.com/matrix-org/synapse/pull/372
2015-11-13 10:39:09 +00:00
3de46c7755
Trailing whitespace
2015-11-12 15:36:43 +00:00
8fd8e72cec
Expand comment
2015-11-12 15:33:47 +00:00
764e79d051
Comment
2015-11-12 15:19:56 +00:00
320408ef47
Fix SQL syntax
2015-11-12 15:09:45 +00:00
14a9d805b9
Use a (hopefully) more efficient SQL query for doing recency based room search
2015-11-12 14:48:39 +00:00
39de87869c
Fix bug where assumed dict was namedtuple
2015-11-12 14:47:48 +00:00
e21cef9bb5
Merge pull request #359 from matrix-org/markjh/incremental_indexing
...
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
e1627388d1
Fix param style to work on both sqlite and postgres
2015-11-11 17:14:56 +00:00
940a161192
Fix the background update
2015-11-11 13:59:40 +00:00
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
90b503216c
Use a background task to update databases to use the full text search
2015-11-10 16:20:13 +00:00
a412b9a465
Run the background updates when starting synapse.
2015-11-10 15:50:58 +00:00
2ede7aa8a1
Add background update task for reindexing event search
2015-11-09 19:29:32 +00:00
c6a01f2ed0
Add storage module for tracking background updates.
...
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
bbe10e8be7
Merge branch 'develop' into markjh/SYN-513
...
Conflicts:
synapse/storage/tags.py
2015-11-09 15:01:59 +00:00
c4135d85e1
SYN-513: Include updates for rooms that have had all their tags deleted
2015-11-09 14:53:08 +00:00
dd40fb68e4
fix comedy important missing comma breaking recent-ordered FTS on sqlite
2015-11-08 16:04:37 +00:00
767c20a869
add a key existence check to tags_by_room to avoid /events 500'ing when testing against vector
2015-11-06 20:49:57 +01:00
f2c4ee41b9
Remove accidentally added ID column
2015-11-06 14:27:49 +00:00
3640ddfbf6
Error handling
2015-11-05 16:10:54 +00:00
729ea933ea
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-11-05 15:43:52 +00:00
7301e05122
Implement basic pagination for search results
2015-11-05 15:04:08 +00:00
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
05c326d445
Implement order and group by
2015-11-04 17:57:44 +00:00
4e62ffdb21
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-11-04 17:31:01 +00:00
f522f50a08
Allow guests to register and call /events?room_id=
...
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
2015-11-04 17:29:07 +00:00
33b3e04049
Merge branch 'develop' into daniel/removesomelies
...
Conflicts:
synapse/notifier.py
2015-11-04 16:01:00 +00:00
6a3a840b19
Merge pull request #343 from matrix-org/erikj/fix_retries
...
Fix broken cache for getting retry times.
2015-11-03 17:51:49 +00:00
6797fcd9ab
Merge pull request #335 from matrix-org/markjh/room_tags
...
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
5897e773fd
Spell "deferred" more correctly
2015-11-03 14:27:35 +00:00
771ca56c88
Remove more unused parameters
2015-11-02 15:31:57 +00:00
ddd8566f41
Store room tag content and return the content in the m.tag event
2015-11-02 15:11:31 +00:00
0e36756383
Merge branch 'develop' into markjh/room_tags
2015-11-02 10:57:00 +00:00
621e84d9a0
Add missing column
2015-10-30 16:25:53 +00:00
f40b0ed5e1
Inform the client of new room tags using v1 /events
2015-10-29 15:21:09 +00:00
a89b86dc47
Fix pyflakes errors
2015-10-28 16:45:57 +00:00
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
56dbcd1524
Docs
2015-10-28 14:05:50 +00:00
5cb298c934
Add room context api
2015-10-28 13:45:56 +00:00
4cf633d5e9
Pull out sender when computing search results
2015-10-23 15:41:36 +01:00
0c36098c1f
Implement rank function for SQLite FTS
2015-10-23 13:23:48 +01:00
671ac699f1
Actually filter results
2015-10-22 16:54:56 +01:00
fb0fecd0b9
LESS THAN
2015-10-22 16:18:35 +01:00
232beb3a3c
Use namedtuple as return value
2015-10-22 15:02:35 +01:00
ba02bba88c
Limit max number of SQL vars
2015-10-22 13:25:27 +01:00
8a98f0dc5b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-22 13:16:35 +01:00
f142898f52
PEP8
2015-10-22 11:18:01 +01:00
4d25bc6c92
Move FTS to delta 25
2015-10-22 11:12:28 +01:00
e94ffd89d6
Merge pull request #316 from matrix-org/markjh/v2_sync_archived
...
Add rooms that the user has left under archived in v2 sync.
2015-10-21 15:46:41 +01:00
e3d75f564a
Include banned rooms in the archived section of v2 sync
2015-10-21 11:15:48 +01:00
68b7fc3e2b
Add rooms that the user has left under archived in v2 sync.
2015-10-19 17:26:18 +01:00
46d39343d9
Explicitly check for Sqlite3Engine
2015-10-16 16:58:00 +01:00
f2d698cb52
Typing
2015-10-16 16:46:48 +01:00
33646eb000
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-16 15:35:35 +01:00
fc012aa8dc
Fix FilteringStore.get_user_filter to work with postgres
2015-10-16 15:28:43 +01:00
12122bfc36
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-16 14:46:32 +01:00
edb998ba23
Explicitly check for Sqlite3Engine
2015-10-16 14:37:14 +01:00
3cf9948b8d
Add docstring
2015-10-16 11:28:12 +01:00
73260ad01f
Comment on the LIMIT 500
2015-10-16 11:24:02 +01:00
22a8c91448
Split up run_upgrade
2015-10-16 11:19:44 +01:00
6296590bf7
Encode the filter JSON as UTF-8 before storing in the database.
...
Because we are using a binary column type to store the filter JSON.
2015-10-16 10:50:32 +01:00
bcfb653816
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-15 16:37:32 +01:00
a059760954
Merge pull request #305 from matrix-org/markjh/v2_sync_api
...
Update the v2 sync API to work as specified in the current spec.
2015-10-14 13:56:23 +01:00
f50c43464c
Merge branch 'develop' into markjh/v2_sync_api
2015-10-14 11:40:45 +01:00
99c7fbfef7
Fix to work with SQLite
2015-10-14 09:52:40 +01:00
858634e1d0
Remove unused room_id arg
2015-10-14 09:31:20 +01:00
7ecd11accb
Add paranoia limit
2015-10-13 15:50:56 +01:00
3e2a1297b5
Remove constraints in preperation of using filters
2015-10-13 15:22:14 +01:00
cacf0688c6
Add a get_invites_for_user method to the storage to find out the rooms a user is invited to
2015-10-13 14:08:38 +01:00
88971fd034
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 14:03:30 +01:00
17c80c8a3d
rename schema_prepare to prepare_database
2015-10-13 13:56:22 +01:00
cfd39d6b55
Add SQLite support
2015-10-13 13:47:50 +01:00
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
40b6a5aad1
Split out the schema preparation and update logic into its own module
2015-10-13 11:38:48 +01:00
1a40afa756
Add sqlite schema
2015-10-13 10:36:25 +01:00
ca53ad7425
Filter events to only thsoe that the user is allowed to see
2015-10-12 15:52:55 +01:00
927004e349
Remove unused room_id parameter
2015-10-12 15:06:14 +01:00
61561b9df7
Keep FTS indexes up to date. Only search through rooms currently joined
2015-10-12 10:49:53 +01:00
c85c912562
Add basic full text search impl.
2015-10-09 15:48:31 +01:00
301141515a
Merge pull request #288 from matrix-org/markjh/unused_definitions
...
Remove some of the unused definitions from synapse
2015-09-28 14:22:44 +01:00
cf1100887b
Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn
2015-09-24 17:35:10 +01:00
1cd65a8d1e
synapse/storage/state.py: _make_group_id was unused
2015-09-23 10:37:58 +01:00
973ebb66ba
Remove unused functions from synapse/storage/signatures.py
2015-09-23 10:36:33 +01:00
e51aa4be96
synapse/storage/roommember.py:_get_members_query was unused
2015-09-23 10:35:10 +01:00
92d8d724c5
Remove unused functions from synapse/storage/events.py
2015-09-23 10:33:06 +01:00
c292dba70c
Remove unused functions from synapse/storage/event_federation.py
2015-09-23 10:31:25 +01:00
396834f1c0
synapse/storage/_base.py:_simple_max_id was unused
2015-09-23 10:30:38 +01:00
1d9036aff2
synapse/storage/_base.py:_simple_delete was unused
2015-09-23 10:30:25 +01:00
Erik Johnston