76ec154e95
We don't need the full events for get_rooms_for_user_where_membership_is
2015-01-06 14:37:00 +00:00
5e23a19204
Merge pull request #28 from matrix-org/erikj-perf
...
Database performance improvements.
2015-01-06 13:33:40 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
af1c7c7808
PEP8
2015-01-06 13:13:17 +00:00
12819d5082
Remove debug lines
2015-01-06 13:12:30 +00:00
52d8519008
Don't do batching when getting events.
2015-01-06 13:10:27 +00:00
98933e3db6
Only fetch prev_content when a client is streaming/paginating. Use transactions for event streams.
2015-01-06 13:03:23 +00:00
3e26720e05
Temporarily turn off 'redacted_because' and 'prev_content' keys
2015-01-06 11:26:58 +00:00
f4ea78e9e2
More debug logging
2015-01-06 11:24:18 +00:00
753126b8cc
Add some debug logging
2015-01-06 11:18:12 +00:00
d7e8ea67b3
Reformat
2015-01-06 11:18:02 +00:00
f0128f9600
Add RoomMemberStore.get_users_in_room, so that we can get the list of joined users without having to retrieve the full events
2015-01-06 10:55:43 +00:00
d61109f578
Merge branch 'hotfixes-v0.6.0' of github.com:matrix-org/synapse into erikj-perf
2014-12-19 16:37:08 +00:00
9c71d945d6
Look for name, topic in the event content rather than the event itself when persisting room name and topic events
2014-12-19 15:16:48 +00:00
b56730bb6e
Merge branch 'develop' into pushers
...
Conflicts:
synapse/api/errors.py
synapse/server.py
synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
afa953a293
schema version is now 10
2014-12-18 15:11:06 +00:00
0a6664493a
Merge branch 'master' into pushers
2014-12-18 15:06:11 +00:00
4c7ad50f6e
Thank you, pyflakes
2014-12-18 14:55:04 +00:00
173264b656
...and bump SCHEMA_VERSION
2014-12-18 14:53:10 +00:00
fc7c5e9cd7
Rename the pusher SQL delta to v9 which the next free one
2014-12-18 14:51:29 +00:00
9728c305a3
after a few rethinks, a working implementation of pushers.
2014-12-18 14:49:22 +00:00
f3788e3c78
Test some ideas that might help performance a bit
2014-12-17 23:37:08 +00:00
dec5b62339
Use _get_events_txn instead of _parse_events_txn
2014-12-16 19:16:41 +00:00
2215faa361
Merge branch 'hotfixes-v0.5.4a' of github.com:matrix-org/synapse into release-v0.6.0
2014-12-16 19:11:13 +00:00
96779d2490
Fix bug where we did not send the full auth chain to people that joined over federation
2014-12-16 18:57:36 +00:00
f76269392b
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.6.0
...
Conflicts:
synapse/state.py
2014-12-16 18:35:46 +00:00
52f99243ab
Use is_outlier() so that we don't get AttributeError
2014-12-16 18:33:50 +00:00
5b39cfff69
Don't assume an event exists
2014-12-16 18:25:24 +00:00
56db465047
Merge branch 'release-v0.6.0' into develop
2014-12-16 17:29:49 +00:00
627e4f01d2
Remove send_message since nothing was calling it. Remove Snapshot because only send_message was using it
2014-12-16 16:07:41 +00:00
42b725ce52
Fix upgrade script to run all the missing deltas.
2014-12-16 15:13:34 +00:00
b3c793e362
Do run all deltas up to missing delta 10
2014-12-16 14:44:53 +00:00
2e44714214
Make failure to run appropraite upgrade scripts more helpful.
2014-12-16 14:20:32 +00:00
ef5a141050
Bump database version
2014-12-16 13:57:47 +00:00
35f4f6b070
Update upgrade script
2014-12-16 13:27:53 +00:00
882dc8dcab
Persist internal_metadata
2014-12-16 13:17:09 +00:00
3c77d13aa5
Kill off synapse.api.events.*
2014-12-16 11:29:05 +00:00
f280929a12
Use frozenutils
2014-12-15 17:31:36 +00:00
c8dd3314d6
Fix bug where we ignored event_edge_hashes table
2014-12-15 13:55:22 +00:00
c39beb5559
Store json as UTF-8 and not bytes
2014-12-12 14:53:37 +00:00
75085bb4d1
Pyflakes
2014-12-12 14:34:34 +00:00
cde840a82c
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
...
Conflicts:
setup.py
2014-12-11 17:48:48 +00:00
0b04369238
Fix public room joining by making sure replaces_state never points to itself.
2014-12-11 15:56:06 +00:00
9191292b0f
Fix prev_content
2014-12-11 15:16:55 +00:00
8cdebce470
Fix redactions. Fix 'age' key
2014-12-11 13:25:19 +00:00
02e4c18171
Remove dead code
2014-12-10 18:00:36 +00:00
4f37c0ea9d
Merge branch 'develop' into media_repository
2014-12-10 16:55:06 +00:00
61fc37e467
Merge branch 'develop' into media_repository
2014-12-10 16:14:17 +00:00
1d2a0040cf
Fix bug where we clobbered old state group values
2014-12-10 15:55:03 +00:00
e5275d856e
Get the code actually working
2014-12-10 15:46:18 +00:00
cc84d3ea78
Thumbnail uploaded and cached images
2014-12-10 15:40:52 +00:00
02db7eb209
Fix bug when uploading state with empty state_key
2014-12-10 14:02:48 +00:00
aae8a37e63
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
2014-12-10 13:18:40 +00:00
95aa903ffa
Try and figure out how and why signatures are being changed.
2014-12-10 11:37:47 +00:00
b8d30899b1
Code style.
2014-12-10 10:16:09 +00:00
2b1acb7671
squidge to 79 columns as per pep8
2014-12-10 00:03:55 +00:00
aa3f66cf7f
Change the way we implement get_events to be less sucky
2014-12-09 13:35:26 +00:00
8529fba02d
fix a million stupid bugs and make it actually work
2014-12-08 19:34:51 +00:00
0d3fa1ac6e
add a write-through cache on the retry schedule
2014-12-08 17:48:57 +00:00
ba3d1e2fc0
Remove unused import
2014-12-08 12:01:25 +00:00
d044121168
Various typos and bug fixes.
2014-12-08 09:08:26 +00:00
5cd43d4b9f
fix stupid syntax thinkos
2014-12-07 23:44:16 +00:00
aed62a3583
track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover.
2014-12-07 02:26:07 +00:00
a953be097f
Add a method field to thumbnail storage
2014-12-05 16:31:56 +00:00
6630e1b579
Start making more things use EventContext rather than event.*
2014-12-05 16:20:48 +00:00
c31dba86ec
Convert rest and handlers to use new event structure
2014-12-04 15:50:01 +00:00
c01fd5573c
Implement download support for media_repository
2014-12-04 14:22:31 +00:00
5d7c9ab789
Begin converting things to use the new Event structure
2014-12-04 11:27:59 +00:00
88af58d41d
Update to app_id / app_instance_id (partially) and mangle to be PEP8 compliant.
2014-12-03 13:37:02 +00:00
6941a19715
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
2014-12-03 11:56:49 +00:00
10eb8f070c
Workaround for non-uniqueness of room member events in the database confusing HAVING COUNT() test of room membership intersection (with thanks to Tom Molesworth)
2014-12-02 21:36:53 +00:00
2f804a7072
Fix pyflakes and pep8 warnings
2014-12-02 19:55:18 +00:00
5da65085d1
Get uploads working with new media repo
2014-12-02 19:51:47 +00:00
279c48c8b4
Write the upload portion of version 1 of the media repository
2014-12-02 17:13:14 +00:00
7642d95d5e
Merge branch 'develop' into pushers
2014-12-02 13:50:05 +00:00
ec2b5d8c28
Store full JSON of events in db
2014-12-01 16:22:07 +00:00
1505055334
Don't return outliers when we get recent events for rooms.
2014-11-27 16:38:50 +00:00
4e2ffe79a4
Don't delete the entire current_state_events table
2014-11-26 15:17:08 +00:00
cb76945688
Add update delta for schema change
2014-11-26 11:17:19 +00:00
87538711b6
Update schema to support multiple signatures
2014-11-26 11:14:30 +00:00
3598c11c8d
Correctly handle the case where we get an event for an unknown room, which turns out we are actually in
2014-11-26 10:41:08 +00:00
a46e5ef621
SYN-163: Add an order by rowid to selects.
...
This should fix the bug where the edges of the graph get returned in a
different order than they were inserted in, and so no get_event no
longer returned the exact same JSON as was inserted. This meant that
signature checks failed.
2014-11-24 10:56:36 +00:00
eb6aedf92c
More work on pushers. Attempt to do HTTP pokes. Not sure if the actual HTTP pokes work or not yet but the retry semantics are pretty good.
2014-11-21 12:21:00 +00:00
23465a30b6
Merge branch 'develop' into pushers
2014-11-20 18:17:46 +00:00
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
dfdda2c871
Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
2014-11-20 17:10:37 +00:00
32090aee16
Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts
2014-11-20 16:24:00 +00:00
74c3879760
Start creating a module to do generic notifications (just prints them to stdout currently!)
2014-11-19 18:20:59 +00:00
5b46ce579b
Bump version, changelog and upgrade.rst
2014-11-19 18:00:57 +00:00
1731af3f29
SYN-104: When going backwards the end token should be before the last event
2014-11-18 16:45:06 +00:00
ae9c2ab165
SYN-149: Send join event immediately after the room create event
2014-11-18 15:29:48 +00:00
a5b88c489e
Split out sending the room alias events from creating the alias so that we can do them in the right point when creating a room
2014-11-18 15:03:13 +00:00
cb4b6c844a
Merge PDUs and Events into one object
2014-11-14 21:25:02 +00:00
8c2b5ea7c4
Fix PDU and event signatures
2014-11-14 19:11:04 +00:00
e903c941cb
Merge branch 'develop' into request_logging
...
Conflicts:
setup.py
synapse/storage/_base.py
synapse/util/async.py
2014-11-14 11:16:50 +00:00
f04b3d5042
Store all signatures on events rather than just dropping them
2014-11-12 17:02:34 +00:00
e715741abc
Update some of the docs in event_federation
2014-11-12 16:20:30 +00:00
e24d5cb97d
Document StateStore and use transactions
2014-11-12 14:33:48 +00:00
58c0ef90c9
Add indices to state group tables
2014-11-12 14:33:48 +00:00
6fea478d2e
Fix bugs with invites/joins across federatiom.
...
Both in terms of auth and not trying to fetch missing PDUs for invites,
joins etc.
2014-11-12 11:24:11 +00:00
5ff0bfb81d
Fix bug where we /always/ created a new state group
2014-11-11 14:16:41 +00:00
a8e565eca8
Add an EventValidator. Fix bugs in auth ++ storage
2014-11-10 18:25:42 +00:00
cdc1b5d629
Fix regression where we did not return redacted events.
2014-11-10 15:21:30 +00:00
5d439b127b
PEP8
2014-11-10 13:46:44 +00:00
6447db063a
Fix backfill to work. Add auth to backfill request
2014-11-10 11:59:51 +00:00
65f846ade0
Notify users about invites.
2014-11-10 11:15:02 +00:00
6cb6cb9e69
Tidy up some of the unused sql tables
2014-11-10 10:31:00 +00:00
1c06806f90
Finish redaction algorithm.
2014-11-10 10:21:32 +00:00
02c3b1c9e2
Add '/event_auth/' federation api
2014-11-07 15:35:53 +00:00
16a0815fac
Fix bug in _get_auth_chain_txn
2014-11-07 11:21:20 +00:00
49948d72f3
Fix joining over federation
2014-11-07 10:53:38 +00:00
8b0e96474b
Implement method to get auth_chain from a given event_id
2014-11-07 10:53:38 +00:00
bf6b72eb55
Start implementing auth chains
2014-11-07 10:53:38 +00:00
351c64e99e
Amalgamate all power levels.
...
Remove concept of reqired power levels, something similiar can be done
using the new power level event.
2014-11-06 16:59:13 +00:00
4317c8e583
Implement new replace_state and changed prev_state
...
`prev_state` is now a list of previous state ids, similiar to
prev_events. `replace_state` now points to what we think was replaced.
2014-11-06 15:10:55 +00:00
cc44ecc62f
Get correct prev_events
2014-11-05 13:23:35 +00:00
aa76bf39ab
Remove unused imports
2014-11-04 14:14:02 +00:00
d59aa6af25
For now, don't store txn -> pdu mappings.
2014-11-03 11:35:19 +00:00
f139c02e95
Formatting
2014-11-03 11:34:49 +00:00
ecabff7eb4
Sign evnets
2014-10-31 17:08:52 +00:00
80b2710e6f
Remove unused signature storage methods
2014-10-31 17:08:36 +00:00
fb3a01fa3a
Remove unused sql file.
2014-10-31 16:04:27 +00:00
d30d79b5be
Make prev_event signing work again.
2014-10-31 15:35:39 +00:00
2f39dc19a2
Remove more references to dead PDU tables
2014-10-31 14:27:14 +00:00
bfa36a72b9
Remove PDU tables.
2014-10-31 14:00:32 +00:00
21fe249d62
Actually don't store any PDUs
2014-10-31 10:47:34 +00:00
841df4da71
Don't store any PDUs
2014-10-31 09:59:59 +00:00
f2de2d644a
Move the impl of backfill to use events.
2014-10-31 09:59:02 +00:00
ef9c4476a0
Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
2014-10-30 11:18:28 +00:00
7c06399512
Merge branch 'develop' into request_logging
...
Conflicts:
synapse/config/logger.py
2014-10-30 11:13:58 +00:00
7d709542ca
Fix pep8 warnings
2014-10-30 11:10:17 +00:00
aa80900a8e
Fix SQL so that accepts we may want to persist events twice.
2014-10-30 10:11:06 +00:00
b29517bd01
Add a request-id to each log line
2014-10-30 01:21:33 +00:00
e7858b6d7e
Start filling out and using new events tables
2014-10-29 16:59:24 +00:00
a10c2ec88d
Don't reference PDU when persisting event
2014-10-28 17:15:32 +00:00
2d1dfb3b34
Begin implementing all the PDU storage stuff in Events land
2014-10-28 16:42:35 +00:00
da1dda3e1d
Add transaction level logging and timing information. Add a _simple_delete method
2014-10-28 11:18:04 +00:00
967ce43b59
Clean up LoggingTransaction
2014-10-28 10:53:11 +00:00
8e358ef35a
Add timer to LoggingTransaction
2014-10-28 10:34:05 +00:00
ad9226eeec
Merge branch 'event_signing' of github.com:matrix-org/synapse into federation_authorization
...
Conflicts:
synapse/storage/__init__.py
2014-10-27 11:58:32 +00:00
5e2236f9ff
fix pyflakes warnings
2014-10-27 11:19:15 +00:00
acb2d171e8
Merge branch 'develop' into event_signing
2014-10-27 11:14:11 +00:00
d56e389a95
Fix pyflakes warnings
2014-10-27 10:33:17 +00:00
bb4a20174c
Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
...
Conflicts:
synapse/federation/transport.py
synapse/handlers/message.py
2014-10-27 10:20:44 +00:00
db2e350e29
Wrap preparing the database in a transaction. Otherwise it will take many seconds to complete because sqlite will create a transaction per statement
2014-10-24 19:04:26 +01:00
5662be894e
Bump database version number.
2014-10-17 20:26:18 +01:00
5ffe5ab43f
Use state groups to get current state. Make join dance actually work.
2014-10-17 18:56:42 +01:00
dc3c2823ac
Merge branch 'develop' into event_signing
...
Conflicts:
synapse/federation/replication.py
2014-10-17 17:33:58 +01:00
82c5820767
keep 'origin_server_ts' as 'ts' in the database to avoid needlessly updating schema
2014-10-17 17:31:48 +01:00
f5cf7ac25b
SPEC-7: Rename 'ts' to 'origin_server_ts'
2014-10-17 17:12:25 +01:00
c8f996e29f
Hash the same content covered by the signature when referencing previous PDUs rather than reusing the PDU content hashes
2014-10-17 11:40:35 +01:00
bb04447c44
Include hashes of previous pdus when referencing them
2014-10-16 23:25:12 +01:00
66104da10c
Sign outgoing PDUs.
2014-10-16 00:09:48 +01:00
1c445f88f6
persist hashes and origin signatures for PDUs
2014-10-15 17:09:04 +01:00
e7bc1291a0
Begin making auth use event.old_state_events
2014-10-15 16:06:59 +01:00
80472ac198
Add missing package storate.state
2014-10-15 10:04:55 +01:00
5fefc12d1e
Begin implementing state groups.
2014-10-14 16:59:51 +01:00
3dac27a8a9
Storage for pdu signatures
2014-10-14 14:58:31 +01:00
07639c79d9
Respond with more helpful error messages for unsigned requests
2014-10-13 16:39:15 +01:00
9605593d11
Merge branch 'develop' into server2server_signing
...
Conflicts:
synapse/storage/__init__.py
tests/rest/test_presence.py
2014-09-30 17:55:06 +01:00
b95a178584
SYN-75 Verify signatures on server to server transactions
2014-09-30 15:15:10 +01:00
fbf6320614
pyflakes cleanup
2014-09-30 12:38:38 +01:00
e06adc6d7e
SYN-2: Allow server admins to delete room aliases
2014-09-30 11:31:42 +01:00
7151615260
Update docstring
2014-09-29 15:35:54 +01:00
1550ab9e2f
SYN-48: Delete dead code
2014-09-29 15:04:47 +01:00
3ccb17ce59
SYN-48: Implement WHOIS rest servlet
2014-09-29 14:59:52 +01:00
c65306f877
Add auth check to test if a user is an admin or not.
2014-09-29 13:35:38 +01:00
f7d80930f2
SYN-48: Track User-Agents as well as IPs for client devices.
2014-09-29 13:35:15 +01:00
0fdf308874
Track the IP users connect with. Add an admin column to users table.
2014-09-26 16:36:24 +01:00
37bfe44046
Merge branch 'deletions' of github.com:matrix-org/synapse into develop
2014-09-25 17:02:53 +01:00
c818aa13eb
Add LIMIT to scalar subquery
2014-09-25 15:51:21 +01:00
ba87eb6753
Fix bug where we tried to insert state events with null state key
2014-09-25 14:45:27 +01:00
70899d3ab2
Rename deletions to redactions
2014-09-24 15:27:59 +01:00
7d9a84a445
Make deleting deletes not undelete
2014-09-24 14:18:08 +01:00
1e6c5b205c
Fix bug where we didn't correctly pull out the event_id of the deletion
2014-09-24 13:29:20 +01:00
4354590a69
Add v4 deltas to current sql.
2014-09-24 11:06:41 +01:00
efea61dc50
Rename 'pruned' to 'pruned_because'
2014-09-23 17:40:58 +01:00
bc250a6afa
SYN-12: Implement auth for deletion by adding a 'delete_level' on the ops levels event
...
SYN-12 # comment Auth has been added.
2014-09-23 17:36:24 +01:00
b99f6eb904
Make sure we don't persist the 'pruned' key
2014-09-23 15:29:27 +01:00
78af6bbb98
Add m.room.deletion. If an event is deleted it will be returned to clients 'pruned', i.e. all client specified keys will be removed.
2014-09-23 15:28:32 +01:00
231afe464a
Add a deletions table
2014-09-22 13:42:52 +01:00
28bcd01e8d
SYN-47: Fix bug where we still returned events for rooms we had left.
...
SYN-47 #resolve
2014-09-19 14:45:21 +01:00
4571cf7baa
Merge branch 'develop' into test-sqlite-memory
2014-09-17 18:27:47 +01:00
59516a8bb1
Correctly handle receiving 'missing' Pdus from federation, rather than just discarding them.
2014-09-15 16:40:44 +01:00
b0406b9ead
Merge remote-tracking branch 'origin/develop' into test-sqlite-memory
2014-09-15 14:15:10 +01:00
5bd9369a62
Correctly handle the 'age' key in events and pdus
2014-09-15 13:26:11 +01:00
14975ce5bc
Fix bug where we relied on the current_state_events being updated when we are handling type specific persistence
2014-09-12 17:57:02 +01:00
667e747ed1
Fix bug where we no longer stored user_id on Pdus
2014-09-12 17:56:21 +01:00
1c51c8ab7d
Merge remote-tracking branch 'origin/develop' into test-sqlite-memory
...
Conflicts:
synapse/storage/pdu.py
2014-09-12 17:20:06 +01:00
39e3fc69e5
Make the state resolution use actual power levels rather than taking them from a Pdu key.
2014-09-12 17:11:09 +01:00
b42fe05c51
Fix bug where we incorrectly removed a remote host from the list of hosts in a room when any user from that host left that room even if they weren't the last user from that host in that room
2014-09-12 17:11:09 +01:00
ca1ae7cf9b
Fix bug where we didn't return a tuple when expected.
2014-09-12 17:11:09 +01:00
aa525e4a63
More accurate docs / clearer paramter names in RoomMemberStore
2014-09-12 16:43:49 +01:00
a87eac4308
Revert recent changes to RoomMemberStore
2014-09-12 15:51:51 +01:00
a840ff8f3f
Now don't need the other logger.debug() call in _execute
2014-09-12 14:38:27 +01:00
1c20249884
Logging of all SQL queries via the 'synapse.storage.SQL' logger
2014-09-12 14:37:55 +01:00
e53d77b501
Add a .runInteraction() method on SQLBaseStore itself to wrap the .db_pool
2014-09-12 14:28:07 +01:00
249e8f2277
Add a better _store_room_member_txn() method that takes separated fields instead of an event object; also add FIXME comment about a big bug in the logic
2014-09-11 18:52:35 +01:00
aaf9ab68c6
Rename _store_room_member_txn to _store_room_member_from_event_txn so we can create another, more sensible function of that name
2014-09-11 18:44:04 +01:00
55397f6347
prepare_database() on db_conn, not plain name, so we can pass in the connection from outside
2014-09-10 16:23:58 +01:00
Erik Johnston