Commit Graph

662 Commits

Author SHA1 Message Date
David Baker ad42322257 Add migration script
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
Erik Johnston d9664344ec Rename table. Add docs. 2016-08-15 11:45:57 +01:00
Erik Johnston 4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Richard van der Hoff 4fec5e57be Default device_display_name to null
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Richard van der Hoff 26cb0efa88 SQL syntax fix 2016-07-27 12:30:22 +01:00
Richard van der Hoff 2e3d90d67c Make the device id on e2e key upload optional
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.

For active access_tokens without an associated device_id, we ought to register
the device in the devices table.

Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Richard van der Hoff 436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
Richard van der Hoff ec5717caf5 Create index on user_ips in the background
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.

I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Richard van der Hoff bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Erik Johnston 1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Richard van der Hoff 022b9176fe schema fix
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Richard van der Hoff f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Erik Johnston b64aa6d687 Add sender and contains_url field to events table 2016-07-14 15:35:43 +01:00
Erik Johnston 314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
Mark Haines c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Mark Haines 9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston 5e50058473 Remove unused indices
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Matrix 3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
David Baker 997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
Mark Haines 9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston 8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
Erik Johnston 984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
Erik Johnston ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston 8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston 51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00
Erik Johnston c877f0f034 Optimise event_search in postgres 2016-04-21 16:56:14 +01:00
David Baker 07d765209d First bits of emailpusher
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
Erik Johnston 96bcfb29c7 Add index 2016-04-14 11:26:33 +01:00
David Baker 2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Matthew Hodgson 6ff7a79308 move local_media_repository_url_cache.sql to schema v31 2016-04-08 19:09:02 +01:00
Matthew Hodgson d6e7333ae4 Merge branch 'develop' into matthew/preview_urls 2016-04-07 17:26:44 +01:00
David Baker 05d044aac3 pep8 2016-04-07 16:45:38 +01:00
David Baker 2d5c693fd3 Fix port script for changes merged from develop 2016-04-07 16:43:54 +01:00
David Baker 9c99ab4572 Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions 2016-04-07 16:35:22 +01:00
David Baker 7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
* Make badges work again
 * Remove old, unused code
2016-04-06 15:42:15 +01:00
Erik Johnston 8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Erik Johnston 0c53d750e7 Docs and indents 2016-04-04 18:02:48 +01:00
Erik Johnston 92ab45a330 Add upgrade path, rename table 2016-04-04 17:07:43 +01:00
Erik Johnston 3d76b7cb2b Store invites in a separate table. 2016-04-04 16:30:15 +01:00
Matthew Hodgson 9f7dc2bef7 Merge branch 'develop' into matthew/preview_urls 2016-04-04 00:38:21 +01:00
Matthew Hodgson 7426c86eb8 add a persistent cache of URL lookups, and fix up the in-memory one to work 2016-04-03 00:31:57 +01:00
Mark Haines 1fbb094c6f Add replication streams for ex outliers and current state resets 2016-03-30 17:19:56 +01:00
Erik Johnston b275765545 Comment about weird SQL 2016-03-23 14:15:32 +00:00
Erik Johnston 0c1a27b787 SQLite and postgres doesn't share a true literal 2016-03-23 14:10:49 +00:00
Erik Johnston 84afeb41f3 Ensure all old public rooms have aliases 2016-03-23 13:59:34 +00:00
Mark Haines ba660ecde2 Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id) 2016-03-16 10:35:00 +00:00
Mark Haines b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Erik Johnston 9669a99d1a Update users table in a batched manner 2016-03-10 15:12:19 +00:00
Daniel Wagner-Hall 3b97797c8d Merge branch 'develop' into daniel/ick 2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall edca2d9891 Idempotent-ise schema update script
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
2016-03-08 17:32:29 +00:00
Mark Haines ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Erik Johnston dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete
Allow alias creators to delete aliases
2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall 95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Mark Haines a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Erik Johnston f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Daniel Wagner-Hall 33300673b7 Generate guest access token on 3pid invites
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall 763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston 8450114098 Merge pull request #554 from matrix-org/erikj/event_push
Change event_push_actions_rm_tokens schema
2016-02-03 15:02:47 +00:00
Erik Johnston 771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Erik Johnston 65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
Erik Johnston fdca8ec418 Add events index 2016-01-28 16:41:59 +00:00
Erik Johnston 86896408b0 Add index to event_push_actions 2016-01-25 15:30:32 +00:00
David Baker d4315bbf6b Add index by user id on receipts_linearized 2016-01-20 15:33:27 +00:00
David Baker fe56138142 Remove rogue 'admin' 2016-01-11 17:09:03 +00:00
David Baker c232780081 Merge pull request #456 from matrix-org/store_event_actions
Send unread notification counts
2016-01-08 14:47:15 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
David Baker ae1262a241 Add schema change file for is_guest flag 2016-01-06 11:58:20 +00:00
David Baker c79f221192 Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions. 2016-01-06 11:38:09 +00:00
David Baker eb03625626 Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall cfd07aafff Allow guests to upgrade their accounts 2016-01-05 18:01:18 +00:00
David Baker c914d67cda Rename event-actions to event_push_actions as per PR request 2016-01-04 14:05:37 +00:00
David Baker 42ad49f5b7 still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs). 2015-12-16 18:42:09 +00:00
David Baker aa667ee396 Save event actions to the db 2015-12-10 17:51:15 +00:00
Mark Haines d57c5cda71 Bump schema version.
As we released version 26 in v0.11.1
2015-12-04 15:28:39 +00:00
Erik Johnston b8680b82c3 Merge pull request #414 from matrix-org/erikj/if_not_exists
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-03 13:21:09 +00:00
David Baker e515b48929 Just replace the table definition with the one from full_schema 16 2015-12-02 17:23:52 +00:00
David Baker 8810eb8c39 Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15... 2015-12-02 17:19:11 +00:00
Erik Johnston b2def42bfd Older versions of SQLite don't like IF NOT EXISTS in virtual tables 2015-12-02 13:29:14 +00:00
Mark Haines 95f30ecd1f Add API for setting account_data globaly or on a per room basis 2015-12-01 18:41:32 +00:00
Erik Johnston 4dcaa42b6d Allow paginating search ordered by recents 2015-11-30 17:45:31 +00:00
Daniel Wagner-Hall 7dfa455508 Remove size specifier for database column
Postgres doesn't support them like this.

We don't have a bool type in common between postgres and sqlite.
2015-11-23 18:35:25 +00:00
Paul "LeoNerd" Evans 1cfda3d2d8 Merge branch 'develop' into daniel/forgetrooms 2015-11-19 16:53:13 +00:00
Mark Haines d153f482dd Rename the database table 2015-11-18 15:33:02 +00:00
Daniel Wagner-Hall ba26eb3d5d Allow users to forget rooms 2015-11-17 17:17:30 -05:00
Erik Johnston e21cef9bb5 Merge pull request #359 from matrix-org/markjh/incremental_indexing
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
Mark Haines e1627388d1 Fix param style to work on both sqlite and postgres 2015-11-11 17:14:56 +00:00
Mark Haines 940a161192 Fix the background update 2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall cf437900e0 Return world_readable and guest_can_join in /publicRooms 2015-11-10 17:10:27 +00:00
Mark Haines 90b503216c Use a background task to update databases to use the full text search 2015-11-10 16:20:13 +00:00
Mark Haines c6a01f2ed0 Add storage module for tracking background updates.
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
Daniel Wagner-Hall f2c4ee41b9 Remove accidentally added ID column 2015-11-06 14:27:49 +00:00
Daniel Wagner-Hall ca2f90742d Open up /events to anonymous users for room events only
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Mark Haines 6797fcd9ab Merge pull request #335 from matrix-org/markjh/room_tags
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
Mark Haines ddd8566f41 Store room tag content and return the content in the m.tag event 2015-11-02 15:11:31 +00:00
Daniel Wagner-Hall 621e84d9a0 Add missing column 2015-10-30 16:25:53 +00:00
Mark Haines 892e70ec84 Add APIs for adding and removing tags from rooms 2015-10-28 16:06:57 +00:00
Erik Johnston 4cf633d5e9 Pull out sender when computing search results 2015-10-23 15:41:36 +01:00
Erik Johnston 0c36098c1f Implement rank function for SQLite FTS 2015-10-23 13:23:48 +01:00
Erik Johnston f142898f52 PEP8 2015-10-22 11:18:01 +01:00
Erik Johnston 4d25bc6c92 Move FTS to delta 25 2015-10-22 11:12:28 +01:00
Erik Johnston 22a8c91448 Split up run_upgrade 2015-10-16 11:19:44 +01:00
Erik Johnston 88971fd034 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 14:03:30 +01:00
Erik Johnston f9340ea0d5 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 13:29:02 +01:00
Erik Johnston 1a40afa756 Add sqlite schema 2015-10-13 10:36:25 +01:00
Erik Johnston 61561b9df7 Keep FTS indexes up to date. Only search through rooms currently joined 2015-10-12 10:49:53 +01:00
Erik Johnston c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Daniel Wagner-Hall 7213588083 Implement configurable stats reporting
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Erik Johnston dffc9c4ae0 Drop unused index 2015-09-14 14:41:37 +01:00
Mark Haines 037481a033 Remove autoincrement since we incrementing the ID in the storage layer 2015-08-24 17:48:57 +01:00
Daniel Wagner-Hall e8cf77fa49 Merge branch 'develop' into refresh
Conflicts:
	synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Mark Haines a0b8e5f2fe Merge pull request #211 from matrix-org/email_in_use
Changes for unique emails
2015-08-20 10:04:04 +01:00
Erik Johnston 8f4165628b Add index receipts_linearized_room_stream 2015-08-17 14:43:54 +01:00
David Baker c77048e12f Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server. 2015-08-04 14:37:09 +01:00
Erik Johnston 5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston 0d7f0febf4 Uniquely name unique constraint 2015-07-10 13:43:03 +01:00
Mark Haines bf0d59ed30 Don't bother with a timeout for one time keys on the server. 2015-07-09 14:04:03 +01:00
Erik Johnston 1af188209a Change format of receipts to allow inclusion of data 2015-07-09 11:39:30 +01:00
Mark Haines b5770f8947 Add store for client end to end keys 2015-07-06 18:46:47 +01:00
Erik Johnston ac78e60de6 Add stream_id index 2015-07-02 13:18:41 +01:00
Erik Johnston 80a61330ee Add basic storage functions for handling of receipts 2015-07-01 17:19:12 +01:00
Erik Johnston 901f56fa63 Add tables for receipts 2015-06-30 15:29:47 +01:00
Erik Johnston 2eda996a63 Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py 2015-06-13 12:21:58 +01:00
David Baker da84946de4 pep8 2015-06-04 16:43:45 +01:00
David Baker 63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston c71176858b Newline, remove debug logging 2015-05-18 10:11:14 +01:00
Erik Johnston 4f1d984e56 Add index on events 2015-05-13 17:22:26 +01:00
Mark Haines 1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines 9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines 6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
David Baker f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker 61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker 4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston c8ed9bd278 pushers table requires an access_token 2015-04-29 15:33:27 +01:00
Mark Haines e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines 4ad8b45155 Merge branch 'develop' into key_distribution
Conflicts:
	synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston 18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines 9182f87664 Merge pull request #126 from matrix-org/csauth
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Mark Haines eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines 4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
David Baker cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston 0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
Erik Johnston b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00
Erik Johnston cf04cedf21 Change full_schemas/11 to work with postgres 2015-04-15 16:53:47 +01:00
Erik Johnston cce5d057d3 Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses 2015-04-15 11:43:25 +01:00
Mark Haines 32e14d8181 Return a sha256 fingerprint rather than the entire tls certificate 2015-04-14 19:10:09 +01:00
Erik Johnston 5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston 86fc9b617c For backwards compat, make state_groups.id have a type of int, not varchar 2015-04-13 17:03:49 +01:00
Erik Johnston 93937b2b31 Remove duplicate rows 2015-04-10 18:47:03 +01:00
Erik Johnston 4103b1c470 DROP indexes before recreating them 2015-04-10 18:46:09 +01:00
Erik Johnston 4d5b098626 Use LONGBLOB and TEXT for arbitary length rows 2015-04-10 18:45:09 +01:00
Erik Johnston 7e863c51e6 Use unsigned bigint 2015-04-10 11:22:04 +01:00
Erik Johnston 0f12772e32 SQLite wants INTEGER and not BIGINT for primary keys 2015-04-10 11:16:09 +01:00
Erik Johnston d5d4281647 Update full_schemas/16 to match delta files. Add delta/16 scripts 2015-04-10 10:59:46 +01:00
Erik Johnston cda4a6f93f Revert non-trivial changes to upgrade scripts 2015-04-10 10:19:50 +01:00
Erik Johnston e2722f58ee Fix schema again 2015-04-10 10:16:29 +01:00
Erik Johnston a1665c5094 Revert non-trivial schema changes and move them to a new schema version. 2015-04-10 10:05:44 +01:00
Erik Johnston 9707acfc40 Remove spurious spaces 2015-04-09 13:45:20 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston 8574bf62dc Add index to presence table 2015-04-07 12:09:36 +01:00
Erik Johnston 0af5f5efaf Don't use multiple UNIQUE constraints; it will cause deadlocks 2015-04-07 12:08:35 +01:00
Erik Johnston 779f7b0f44 Fix unicode support 2015-04-02 10:06:22 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
David Baker c1a256cc4c Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API. 2015-03-25 19:06:22 +00:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
David Baker c7023f2155 1) Pushers are now associated with an access token
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston 0d0610870d Fix up schemas some more 2015-03-20 10:55:31 +00:00
Erik Johnston 58ed393235 Remove redundant key 2015-03-19 15:12:05 +00:00
Erik Johnston fae059cc18 Fix up schemas to work with mariadb 2015-03-19 13:42:39 +00:00
Erik Johnston 89036579ed Update schema to work with mariadb 2015-03-18 11:18:49 +00:00
Kegan Dougal b98cd03193 Use event IDs instead of dumping event content in the txns table. 2015-03-09 17:25:20 +00:00
Kegan Dougal 0a60bbf4fa Finish appservice txn storage impl and tests. 2015-03-09 15:53:03 +00:00
Kegan Dougal 34ce2ca62f Merge branch 'develop' into application-services-txn-reliability 2015-03-06 17:28:49 +00:00
Kegan Dougal 01c099d9ef Add appservice txns sql schema 2015-03-06 17:16:47 +00:00
David Baker cf66ddc1b4 Schema change as delta in v14 2015-03-06 14:11:49 +00:00
David Baker ed72fc3a50 Merge branch 'develop' into pushrules2
Conflicts:
	synapse/storage/schema/pusher.sql
2015-03-04 15:24:21 +00:00
Erik Johnston 17d319a20d s/schema_deltas/applied_schema_deltas/ 2015-03-04 15:06:22 +00:00
Erik Johnston 5eefd1f618 Add unique constraint on schema_version.lock schema. Use conflict clause in sql. 2015-03-04 13:52:18 +00:00
Erik Johnston b4c38738f4 Change to use logger in db upgrade script 2015-03-04 13:43:35 +00:00
Erik Johnston 8c8354e85a Actually add full_schemas dir 2015-03-04 13:34:38 +00:00
Erik Johnston c3530c3fb3 More docs. Rename 'schema/current' to 'schema/full_schemas' 2015-03-04 13:34:11 +00:00
Erik Johnston 82b34e813d SYN-67: Finish up implementing new database schema management 2015-03-04 12:04:19 +00:00
David Baker 94fa334b01 Add enable/disable overlay for push rules (REST API not yet hooked up) 2015-02-25 19:17:07 +00:00
Erik Johnston 1a989c436c Bump schema version 2015-02-17 15:45:55 +00:00
Mark Haines 0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Erik Johnston 83d41f25d8 Set database schema version in delta 2015-02-12 10:05:47 +00:00
Kegan Dougal 53557fc532 Merge branch 'develop' into application-services 2015-02-09 15:20:56 +00:00
Kegan Dougal ac3183caaa Register a user account for the AS when the AS registers. Add 'sender' column to AS table. 2015-02-09 12:03:37 +00:00
Erik Johnston 0cd66885e3 Move delta/v13.sql to delta/v12.sql 2015-02-06 14:38:04 +00:00
Kegan Dougal a3c6010718 Add delta sql file. 2015-02-05 16:48:57 +00:00
Kegan Dougal 951690e54d Merge branch 'develop' into application-services 2015-02-05 14:28:03 +00:00
Kegan Dougal 27091f146a Add hs_token column and generate a different token f.e application service. 2015-02-05 10:08:12 +00:00
David Baker dc7bb70f22 s/instance_handle/profile_tag/ 2015-02-03 16:51:07 +00:00
Kegan Dougal c059c9fea5 Merge branch 'develop' into application-services
Conflicts:
	synapse/handlers/__init__.py
	synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
Erik Johnston fdd2ac495a Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
Erik Johnston 472cf532b7 Put CREATE rejections into seperate .sql 2015-01-30 14:48:03 +00:00
Erik Johnston e0b7c521cb Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Mark Haines 9d8f798a3f Merge changes from develop 2015-01-29 14:55:27 +00:00
Mark Haines e4f50fa0aa Move bump schema delta 2015-01-29 14:53:18 +00:00
Kegan Dougal 3773759c0f Also edit the filter column on the delta SQL 2015-01-29 09:15:33 +00:00
Kegan Dougal 11634017f4 s/definition/filter_json/ since definition is now used to mean a component of the filter, rather than the complete json 2015-01-28 17:42:19 +00:00
David Baker 60b143a52e Move pushers delta to v12 and bump schema version 2015-01-28 15:48:28 +00:00
Kegan Dougal 42876969b9 Add basic application_services SQL, and hook up parts of the appservice store to read from it. 2015-01-28 11:59:38 +00:00
Paul "LeoNerd" Evans 8398f19bce Created schema delta 2015-01-27 19:00:09 +00:00
Paul "LeoNerd" Evans 06cc147012 Initial stab at real SQL storage implementation of user filter definitions 2015-01-27 18:46:03 +00:00
David Baker ede491b4e0 Oops: second part of commit dc938606 2015-01-22 17:38:53 +00:00
Erik Johnston b1b85753d7 Add support for storing rejected events in EventContext and data stores 2015-01-22 15:50:17 +00:00
David Baker afb714f7be add instance_handles to pushers so we have a way to refer to them even if the push token changes. 2015-01-20 11:49:48 +00:00
David Baker 2d2953cf5f Require device language when adding a pusher.
Because this seems like it might be useful to do sooner rather
than later.
2015-01-16 11:24:10 +00:00
David Baker 2cb30767fa Honour the 'rejected' return from push gateways
Add a timestamp to push tokens so we know the last time they we
got them from the device. Send it to the push gateways so it can
determine whether its failure is more recent than the token.
Stop and remove pushers that have been rejected.
2015-01-13 19:48:37 +00:00
David Baker c06a9063e1 Merge branch 'develop' into pushers 2015-01-13 13:15:51 +00:00
Erik Johnston a01416cf21 Add delta and bump DB version 2015-01-06 15:42:18 +00:00
Erik Johnston f6da237c35 Add index on transaction_id to sent_transcations 2015-01-06 15:40:38 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
David Baker b56730bb6e Merge branch 'develop' into pushers
Conflicts:
	synapse/api/errors.py
	synapse/server.py
	synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
David Baker afa953a293 schema version is now 10 2014-12-18 15:11:06 +00:00
David Baker fc7c5e9cd7 Rename the pusher SQL delta to v9 which the next free one 2014-12-18 14:51:29 +00:00
David Baker 9728c305a3 after a few rethinks, a working implementation of pushers. 2014-12-18 14:49:22 +00:00
Erik Johnston 42b725ce52 Fix upgrade script to run all the missing deltas. 2014-12-16 15:13:34 +00:00
Erik Johnston 882dc8dcab Persist internal_metadata 2014-12-16 13:17:09 +00:00
Erik Johnston cde840a82c Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
Conflicts:
	setup.py
2014-12-11 17:48:48 +00:00
Mark Haines 4f37c0ea9d Merge branch 'develop' into media_repository 2014-12-10 16:55:06 +00:00
Mark Haines 61fc37e467 Merge branch 'develop' into media_repository 2014-12-10 16:14:17 +00:00
Erik Johnston 1d2a0040cf Fix bug where we clobbered old state group values 2014-12-10 15:55:03 +00:00
Erik Johnston 02db7eb209 Fix bug when uploading state with empty state_key 2014-12-10 14:02:48 +00:00
Erik Johnston aae8a37e63 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-10 13:18:40 +00:00
Matthew Hodgson aed62a3583 track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover. 2014-12-07 02:26:07 +00:00
Mark Haines a953be097f Add a method field to thumbnail storage 2014-12-05 16:31:56 +00:00
David Baker 88af58d41d Update to app_id / app_instance_id (partially) and mangle to be PEP8 compliant. 2014-12-03 13:37:02 +00:00
Mark Haines 279c48c8b4 Write the upload portion of version 1 of the media repository 2014-12-02 17:13:14 +00:00
David Baker 7642d95d5e Merge branch 'develop' into pushers 2014-12-02 13:50:05 +00:00
Erik Johnston ec2b5d8c28 Store full JSON of events in db 2014-12-01 16:22:07 +00:00
Erik Johnston cb76945688 Add update delta for schema change 2014-11-26 11:17:19 +00:00
Erik Johnston 87538711b6 Update schema to support multiple signatures 2014-11-26 11:14:30 +00:00
David Baker eb6aedf92c More work on pushers. Attempt to do HTTP pokes. Not sure if the actual HTTP pokes work or not yet but the retry semantics are pretty good. 2014-11-21 12:21:00 +00:00
David Baker 74c3879760 Start creating a module to do generic notifications (just prints them to stdout currently!) 2014-11-19 18:20:59 +00:00
Erik Johnston f04b3d5042 Store all signatures on events rather than just dropping them 2014-11-12 17:02:34 +00:00
Erik Johnston 58c0ef90c9 Add indices to state group tables 2014-11-12 14:33:48 +00:00
Erik Johnston 6cb6cb9e69 Tidy up some of the unused sql tables 2014-11-10 10:31:00 +00:00
Erik Johnston bf6b72eb55 Start implementing auth chains 2014-11-07 10:53:38 +00:00
Erik Johnston 4317c8e583 Implement new replace_state and changed prev_state
`prev_state` is now a list of previous state ids, similiar to
prev_events. `replace_state` now points to what we think was replaced.
2014-11-06 15:10:55 +00:00
Erik Johnston fb3a01fa3a Remove unused sql file. 2014-10-31 16:04:27 +00:00
Erik Johnston 2f39dc19a2 Remove more references to dead PDU tables 2014-10-31 14:27:14 +00:00
Erik Johnston bfa36a72b9 Remove PDU tables. 2014-10-31 14:00:32 +00:00
Erik Johnston e7858b6d7e Start filling out and using new events tables 2014-10-29 16:59:24 +00:00
Erik Johnston 2d1dfb3b34 Begin implementing all the PDU storage stuff in Events land 2014-10-28 16:42:35 +00:00
Erik Johnston ad9226eeec Merge branch 'event_signing' of github.com:matrix-org/synapse into federation_authorization
Conflicts:
	synapse/storage/__init__.py
2014-10-27 11:58:32 +00:00
Mark Haines acb2d171e8 Merge branch 'develop' into event_signing 2014-10-27 11:14:11 +00:00
Erik Johnston bb4a20174c Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
Conflicts:
	synapse/federation/transport.py
	synapse/handlers/message.py
2014-10-27 10:20:44 +00:00
Erik Johnston 5662be894e Bump database version number. 2014-10-17 20:26:18 +01:00
Mark Haines 82c5820767 keep 'origin_server_ts' as 'ts' in the database to avoid needlessly updating schema 2014-10-17 17:31:48 +01:00
Mark Haines f5cf7ac25b SPEC-7: Rename 'ts' to 'origin_server_ts' 2014-10-17 17:12:25 +01:00
Mark Haines c8f996e29f Hash the same content covered by the signature when referencing previous PDUs rather than reusing the PDU content hashes 2014-10-17 11:40:35 +01:00
Mark Haines bb04447c44 Include hashes of previous pdus when referencing them 2014-10-16 23:25:12 +01:00
Mark Haines 1c445f88f6 persist hashes and origin signatures for PDUs 2014-10-15 17:09:04 +01:00
Erik Johnston e7bc1291a0 Begin making auth use event.old_state_events 2014-10-15 16:06:59 +01:00
Erik Johnston 5fefc12d1e Begin implementing state groups. 2014-10-14 16:59:51 +01:00
Mark Haines 3dac27a8a9 Storage for pdu signatures 2014-10-14 14:58:31 +01:00
Mark Haines 9605593d11 Merge branch 'develop' into server2server_signing
Conflicts:
	synapse/storage/__init__.py
	tests/rest/test_presence.py
2014-09-30 17:55:06 +01:00
Mark Haines b95a178584 SYN-75 Verify signatures on server to server transactions 2014-09-30 15:15:10 +01:00
Erik Johnston 3ccb17ce59 SYN-48: Implement WHOIS rest servlet 2014-09-29 14:59:52 +01:00
Erik Johnston f7d80930f2 SYN-48: Track User-Agents as well as IPs for client devices. 2014-09-29 13:35:15 +01:00
Erik Johnston 0fdf308874 Track the IP users connect with. Add an admin column to users table. 2014-09-26 16:36:24 +01:00
Erik Johnston 70899d3ab2 Rename deletions to redactions 2014-09-24 15:27:59 +01:00
Erik Johnston 4354590a69 Add v4 deltas to current sql. 2014-09-24 11:06:41 +01:00
Erik Johnston bc250a6afa SYN-12: Implement auth for deletion by adding a 'delete_level' on the ops levels event
SYN-12 # comment Auth has been added.
2014-09-23 17:36:24 +01:00
Erik Johnston 78af6bbb98 Add m.room.deletion. If an event is deleted it will be returned to clients 'pruned', i.e. all client specified keys will be removed. 2014-09-23 15:28:32 +01:00
Erik Johnston 231afe464a Add a deletions table 2014-09-22 13:42:52 +01:00
Erik Johnston 9dd4570b68 Generate m.room.aliases event when the HS creates a room alias 2014-09-05 21:35:56 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans 756e171ad0 Store SQL DDL deltas as well; attempt to upgrade the database on startup if it's too old 2014-09-03 14:14:39 +01:00
Mark Haines c6eafdfbaf Add copyright notices and fix pyflakes errors 2014-09-03 09:43:11 +01:00
Erik Johnston f452899fe2 Merge branch 'develop' of github.com:matrix-org/synapse into room_config 2014-09-01 19:57:28 +01:00
Erik Johnston b8ab9f1c0a Add all the necessary checks to make banning work. 2014-09-01 18:24:56 +01:00
Erik Johnston db7109c43b Add beginnings of ban support. 2014-09-01 16:15:34 +01:00
Erik Johnston 865469f233 Implement power level lists, default power levels and send_evnet_level/add_state_level events. 2014-09-01 14:39:09 +01:00
Erik Johnston 1118f02689 Start adding storage for new events. 2014-08-29 15:18:30 +01:00
Mark Haines d7ae9b90a0 Add store for server certificates and keys 2014-08-28 18:19:47 +01:00
Erik Johnston 9d86c8c7a6 Add a unique constraint on the room hosts table 2014-08-24 11:29:29 +01:00
Erik Johnston 9f514915af Add indices to schema 2014-08-22 18:03:39 +01:00
Erik Johnston 808f663ed1 Don't return state event outlier's when paginating. 2014-08-22 13:06:07 +01:00
Erik Johnston 598a1d8ff9 Change the way pagination works to support out of order events. 2014-08-19 14:19:48 +01:00
Erik Johnston 709a92cee8 SQL doesn't allow AUTOINCREMENT on non PRIMARY KEY columns. 2014-08-18 16:00:46 +01:00
Erik Johnston fc26275bb3 Add two different columns for ordering the events table, one which can be used for pagination and one which can be as tokens for notifying clients. Also add a 'processed' field which is currently always set to True 2014-08-18 15:50:41 +01:00
Erik Johnston 114984a236 Start chagning the events stream to work with the new DB schema 2014-08-15 13:58:28 +01:00
Erik Johnston 5002efa31b Reimplement the get public rooms api to work with new DB schema 2014-08-15 10:26:35 +01:00
Erik Johnston 2c46bb6208 Fix up typos and correct sql queries 2014-08-14 18:40:50 +01:00
Erik Johnston 937c175029 Fix up RoomMemberStore to work with the new schema. 2014-08-14 16:02:10 +01:00
Erik Johnston 10294b6082 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor
Conflicts:
	synapse/storage/_base.py
2014-08-14 10:01:04 +01:00
Paul "LeoNerd" Evans d05aa651f8 An initial hack at storing presence state-change mtimes in database and presenting age durations to clients/federation events 2014-08-13 19:19:15 +01:00
Erik Johnston beaf4384d9 Make feedback table also store sender. 2014-08-13 18:03:41 +01:00
Erik Johnston 336987bb8d Initial stab at refactoring the SQL tables, including rejigging some of the storage layer. 2014-08-13 18:03:41 +01:00
Erik Johnston 3dfa84bec8 Convert im schema to a 'one' table structure 2014-08-13 18:03:41 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00