70d93cafdb
Update insert
2019-11-06 10:59:03 +00:00
807ec3bd99
Fix bug which caused rejected events to be stored with the wrong room state ( #6320 )
...
Fixes a bug where rejected events were persisted with the wrong state group.
Also fixes an occasional internal-server-error when receiving events over
federation which are rejected and (possibly because they are
backwards-extremities) have no prev_group.
Fixes #6289 .
2019-11-06 10:01:39 +00:00
8822b33111
Update copyrights
2019-11-05 14:46:35 +00:00
c2203bea57
Re-add docstring, with caveats detailed
2019-11-04 18:17:11 +00:00
0287d033ee
Transfer upgraded rooms on groups
2019-11-04 18:08:50 +00:00
09957ce0e4
Implement per-room message retention policies
2019-11-04 17:09:22 +00:00
7134ca7daa
Change to not require a state_groups.room_id index.
...
This does mean that we won't clean up orphaned state groups (i.e. state
groups that were persisted but the associated event wasn't).
2019-11-04 13:36:57 +00:00
6a0092d371
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_purge_history
2019-11-04 13:29:35 +00:00
3b29a73f9f
Print out the actual number of affected rows
2019-11-04 09:56:11 +00:00
824bba2f78
Correctly order results
2019-11-04 09:56:11 +00:00
49008e674f
TODO
2019-11-04 09:56:11 +00:00
1586f2c7e7
Fix exit condition
2019-11-04 09:56:11 +00:00
1c1268245d
Lint
2019-11-04 09:56:11 +00:00
911b03ca31
Don't try to process events we already have a label for
2019-11-04 09:56:10 +00:00
07cb38e965
Use a sensible default value for labels
2019-11-04 09:56:10 +00:00
a46574281d
Use the right format for rows
2019-11-04 09:56:10 +00:00
c9a1b80a74
MSC2326: Add background update to take previous events into account
2019-11-04 09:56:04 +00:00
f496d25877
Merge pull request #6301 from matrix-org/babolivier/msc2326
...
Implement MSC2326 (label based filtering)
2019-11-01 17:04:45 +00:00
988d8d6507
Incorporate review
2019-11-01 16:22:44 +00:00
5598445655
Update synapse/storage/data_stores/main/schema/delta/56/event_labels.sql
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-11-01 16:18:34 +00:00
fa7e52caf1
Merge pull request #6313 from matrix-org/uhoreg/cross_signing_fix_sqlite_schema
...
fix hidden field in devices table for older sqlite
2019-11-01 10:52:46 -04:00
a2c63c619a
Add more data to the event_labels table and fix the indexes
2019-11-01 11:47:28 +00:00
669b6cbda3
Fix up comment
2019-11-01 11:32:20 +00:00
57cdb046e4
Lint
2019-11-01 10:39:14 +00:00
c6dbca2422
Incorporate review
2019-11-01 10:30:51 +00:00
c61db13183
fix hidden field in devices table for older sqlite
2019-10-31 22:52:55 -04:00
c3fc176c60
Update synapse/storage/data_stores/main/devices.py
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-10-31 22:49:48 -04:00
6f4bc6d01d
Merge branch 'develop' into cross-signing_federation
2019-10-31 22:38:21 -04:00
9c94b48bf1
Merge branch 'develop' into uhoreg/cross_signing_fix_workers_notify
2019-10-31 12:32:07 -04:00
fb1a6914cf
Update log line to lie a little less
2019-10-31 15:45:48 +00:00
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
61be1a2926
Add state_groups.room_id index
2019-10-31 15:39:26 +00:00
f91f2a1f92
Docstrings
2019-10-31 15:26:00 +00:00
8f5bbdb987
Fix purge room API
2019-10-31 15:22:08 +00:00
cd581338cf
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_purge_history
2019-10-31 15:19:26 +00:00
dfe0cd71b6
Merge pull request #6294 from matrix-org/erikj/add_state_storage
...
Add StateGroupStorage interface
2019-10-31 16:17:53 +01:00
64f2b8c3d8
Apply suggestions from code review
...
Fix docstring
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-10-31 15:44:31 +01:00
c6bcd38841
Fix /purge_room API.
...
It fails trying to clean the `topic` table which was recently removed.
2019-10-31 11:17:23 +00:00
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
998f7fe7d4
make user signatures a separate stream
2019-10-30 17:22:52 -04:00
670972c0e1
Merge branch 'develop' into uhoreg/cross_signing_fix_workers_notify
2019-10-30 16:46:31 -04:00
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
0467f33584
fix delete_existing for _persist_events ( #6300 )
...
this is part of _retry_on_integrity_error, so should only be on _persist_events_and_state_updates
2019-10-30 18:05:00 +00:00
dcc069a2e2
Lint
2019-10-30 18:01:56 +00:00
fe51d6cacf
Add more integration testing
2019-10-30 17:28:41 +00:00
233b14ebe1
Add index on label
2019-10-30 15:58:05 +00:00
acd16ad86a
Implement filtering
2019-10-30 15:56:33 +00:00
7c8c97e635
Split purge API into events vs state
2019-10-30 15:23:37 +00:00
5db03535d5
Add StateGroupStorage interface
2019-10-30 14:46:49 +00:00
fa0dcbc8fa
Store labels for new events
2019-10-30 14:27:15 +00:00
7d7eac61be
Merge branch 'develop' into cross-signing_federation
2019-10-30 10:17:10 -04:00
d78b1e339d
apply changes as a result of PR review
2019-10-30 10:01:53 -04:00
ec6de1cc7d
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_persistence_store
2019-10-30 13:37:04 +00:00
a8d16f6c00
Review comments
2019-10-30 13:36:12 +00:00
47f767269c
Add database table for keeping track of labels on events
2019-10-29 16:56:22 +00:00
2c35ffead2
Port receipt and read markers to async/wait
2019-10-29 15:08:22 +00:00
561133c3c5
Merge pull request #6263 from matrix-org/erikj/caches_return_deferreds
...
Quick fix to ensure cache descriptors always return deferreds
2019-10-29 12:53:21 +01:00
e419c44ba4
Merge branch 'release-v1.5.0' of github.com:matrix-org/synapse into develop
2019-10-29 11:41:27 +00:00
342d871d76
Make room directory search case insensitive
2019-10-29 10:24:02 +00:00
14504ad573
Add CI for synapse_port_db ( #6140 )
...
This adds:
* a test sqlite database
* a configuration file for the sqlite database
* a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`)
as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that:
1. installs Synapse
2. updates the test sqlite database to the latest schema and runs background updates on it
3. creates an empty postgresql database
4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage)
Step `2` is done via a new script located at `scripts-dev/update_database`.
The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.
2019-10-28 17:45:32 +00:00
d0d8a22c13
Quick fix to ensure cache descriptors always return deferreds
2019-10-28 13:33:04 +00:00
9aee28927b
Convert EventContext to attrs ( #6218 )
...
* make EventContext use an attr
2019-10-28 14:29:55 +02:00
da78f61778
Merge pull request #6253 from matrix-org/uhoreg/e2e_backup_delete_keys
...
delete keys when deleting backup versions
2019-10-25 11:28:11 -04:00
a71b8c87ec
Merge branch 'release-v1.5.0' of github.com:matrix-org/synapse into develop
2019-10-25 11:32:24 +01:00
a411f2b177
Fix /keys/query API on workers.
...
The necessary getters were added only to the master store and not the
worker stores.
2019-10-25 11:08:03 +01:00
44ab048cfe
Merge pull request #6251 from matrix-org/michaelkaye/debug_guard_logging
...
Reduce debug logging overhead
2019-10-25 10:05:44 +01:00
c40d7244f8
Merge branch 'develop' into cross-signing_federation
2019-10-24 22:31:25 -04:00
8ac766c44a
make notification of signatures work with workers
2019-10-24 22:14:58 -04:00
848cd388d9
delete keys when deleting backups
2019-10-24 21:21:51 -04:00
e4d98188da
Address codestyle concerns
2019-10-24 18:43:13 +01:00
9eebc1e73b
use %r to __repr__ objects
...
This avoids calculating __repr__ unless we are going to log.
2019-10-24 18:18:56 +01:00
39266a9c9f
Make user/room stats log line less verbose.
2019-10-24 17:55:53 +01:00
3aa2a90556
Move schema delta files to the correct data store.
...
They were put in the global schema delta directory due to a bad merge.
2019-10-24 16:45:03 +01:00
73cf63784b
Add DataStores and Storage classes.
2019-10-23 16:15:03 +01:00
dc2cd6f79d
move get_e2e_cross_signing_key to EndToEndKeyWorkerStore so it works with workers
2019-10-23 09:13:47 -04:00
22a9847670
Move persist_events out from main data store.
...
This is in preparation for splitting out of state_groups_state from the
main store into it own one, as persisting events depends on calculating
state.
2019-10-23 13:29:44 +01:00
404e8c8532
vendor-prefix the EDU name until MSC1756 is merged into the spec
2019-10-22 22:33:23 -04:00
1fabf82d50
update to work with newer code, and fix formatting
2019-10-22 21:44:58 -04:00
cfdb84422d
make black happy
2019-10-22 19:06:06 -04:00
a1aaf3eea6
don't crash if the user doesn't have cross-signing keys
2019-10-22 19:04:37 -04:00
8d3542a64e
implement federation parts of cross-signing
2019-10-22 19:04:35 -04:00
c17efdc01c
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 18:44:29 +01:00
23d62eded2
Clean up prepare_database.py a bit and add comments
2019-10-22 18:43:31 +01:00
6cc497f99b
Delete background_update table creation in main data_store
2019-10-22 18:02:50 +01:00
1bbc5444a8
Move README into synapse/storage/__init__.py
2019-10-22 17:59:31 +01:00
acf47c7698
Add a basic README to synapse.storage
2019-10-22 11:55:46 +01:00
bb6264be0b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 10:41:18 +01:00
ffd24545bb
Fix schema management to work with multiple data stores.
2019-10-21 16:08:40 +01:00
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
36adfaedab
Merge branch 'develop' into cross-signing_sig_upload
2019-10-18 18:34:42 +01:00
cfc28325a6
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-10-18 11:39:40 +01:00
f0f6a2b360
use the right function for when we're already in runInteraction
2019-10-18 10:56:54 +01:00
06fc66c81e
Merge pull request #6193 from matrix-org/uhoreg/interpret_device_key_in_storage
...
make storage layer in charge of interpreting the device key data
2019-10-11 15:19:06 -04:00
132b251e29
expand on comment
2019-10-11 14:24:52 -04:00
a0d0ba7862
Fix MAU reaping where reserved users are specified. ( #6168 )
2019-10-11 09:38:26 +01:00
7a0dce9259
make sure we actually return something
2019-10-10 20:31:30 -04:00
4908fb3b30
make storage layer in charge of interpreting the device key data
2019-10-10 19:15:30 -04:00
83d86106a8
Merge pull request #6156 from matrix-org/erikj/postgres_any
...
Use Postgres ANY for selecting many values.
2019-10-10 16:41:36 +01:00
b54b1e759a
Fix SQLite take 2
2019-10-10 16:19:40 +01:00
afb6d9d53b
Fix SQLite
2019-10-10 15:55:41 +01:00
Brendan Abolivier