Commit Graph

571 Commits

Author SHA1 Message Date
Erik Johnston f44f1d2e83 Fix errors storing large retry intervals.
We have set the max retry interval to a value larger than a postgres or
sqlite int can hold, which caused exceptions when updating the
destinations table.

To fix postgres we need to change the column to a bigint, and for sqlite
we lower the max interval to 2**62 (which is still incredibly long).
2019-10-02 10:36:27 +01:00
Erik Johnston 898dde981b Add received_ts column to redactions.
This will allow us to efficiently search for uncensored redactions in
the DB before a given time.
2019-10-01 13:43:48 +01:00
Erik Johnston d2bd0bc6b1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cleanup_user_ips 2019-09-25 17:16:28 +01:00
Richard van der Hoff ed8b92f0d2 Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work 2019-09-24 12:57:32 +01:00
Erik Johnston ed80231ade Add BG update to populate devices last seen info 2019-09-23 16:59:45 +01:00
Erik Johnston 2ade05dca3 Add last seen info to devices table.
This allows us to purge old user_ips entries without having to preserve
the latest last seen info for active devices.
2019-09-23 16:59:35 +01:00
Richard van der Hoff b65327ff66 Merge branch 'develop' into rav/saml_mapping_work 2019-09-19 18:13:31 +01:00
Richard van der Hoff bcd9132869
Undo the deletion of some tables (#6047)
This is a partial revert of #5893. The problem is that if we drop these tables
in the same release as removing the code that writes to them, it prevents users
users from being able to roll back to a previous release.

So let's leave the tables in place for now, and remember to drop them in a
subsequent release.

(Note that these tables haven't been *read* for *years*, so any missing rows
resulting from a temporary upgrade to vNext won't cause a problem.)
2019-09-19 15:06:27 +01:00
Richard van der Hoff 1e19ce00bf
Add 'failure_ts' column to 'destinations' table (#6016)
Track the time that a server started failing at, for general analysis purposes.
2019-09-17 11:41:54 +01:00
Richard van der Hoff a8ac40445c Record mappings from saml users in an external table
We want to assign unique mxids to saml users based on an incrementing
suffix. For that to work, we need to record the allocated mxid in a separate
table.
2019-09-13 16:01:46 +01:00
Erik Johnston 591d82f06b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/censor_redactions 2019-09-05 17:27:46 +01:00
Erik Johnston 6e834e94fc
Fix and refactor room and user stats (#5971)
Previously the stats were not being correctly populated.
2019-09-04 13:04:27 +01:00
Erik Johnston a4bf72c30c Censor redactions in DB after a month 2019-08-30 11:29:17 +01:00
Jorik Schellekens 8767b63a82
Propagate opentracing contexts through EDUs (#5852)
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-08-22 18:21:10 +01:00
Richard van der Hoff c9f11d09fc
Add missing index on users_in_public_rooms. (#5894) 2019-08-22 10:43:13 +01:00
Richard van der Hoff 4dab867288
Drop some unused tables. (#5893)
These tables are never used, so we may as well drop them.
2019-08-21 13:16:28 +01:00
Hubert Chathi b7f7cc7ace add the version field to the index for e2e_room_keys 2019-08-14 17:14:40 -07:00
Erik Johnston 123c04daa7 Don't recreate column 2019-07-30 16:42:48 +01:00
Erik Johnston 15056ca208 Fix current_state_events membership background update.
Turns out not all rooms are in `rooms`, so lets fetch the room list from
`current_state_events`. We move the delta file to force it to be run
again.
2019-07-30 14:51:41 +01:00
Erik Johnston 105e7f6ed3 Remove lost comment 2019-07-29 16:09:48 +01:00
Erik Johnston 32768e96d4 Add function to get all forgotten rooms for user
This will allow us to efficiently filter out rooms that have been
forgotten in other queries without having to join against the
`room_memberships` table.
2019-07-24 11:44:23 +01:00
Erik Johnston ebc5ed1296 Update comment for new column 2019-07-19 13:29:02 +01:00
Erik Johnston c618a5d348 Add background update for current_state_events.membership column 2019-07-18 14:16:01 +01:00
Erik Johnston 6de09e07a6 Add membership column to current_state_events table.
It turns out that doing a join is surprisingly expensive for the DB to
do when room_membership table is larger than the disk cache.
2019-07-18 14:15:57 +01:00
Richard van der Hoff 5f158ec039
Implement access token expiry (#5660)
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Brendan Abolivier d0530382ee
Track deactivated accounts in the database (#5378) 2019-06-14 13:18:24 +01:00
Erik Johnston 1b8cb64393
Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1
Make a full SQL schema
2019-06-06 18:35:38 +01:00
Amber H. Brown ed872db8df fix maybe 2019-06-07 02:53:47 +10:00
Andrew Morgan 3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377)
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Amber H. Brown 837e32ef55 just user dir? 2019-06-07 01:49:25 +10:00
Amber H. Brown 3b6645d3bf remove background updates that arent needed 2019-06-07 01:20:58 +10:00
Amber Brown 6362e3af14 add more comments 2019-06-06 04:20:35 +10:00
Amber Brown 2198b7ce2a add stuff in bg updates 2019-06-04 01:06:00 +10:00
Amber Brown 4e75c5e02a WHY IS THIS CALLED A SLIGHTLY DIFFERENT THING 2019-06-03 22:42:12 +10:00
Amber Brown be452fc9ac more fix 2019-06-03 22:24:23 +10:00
Amber Brown 7f81b967ca fix schemas 2019-06-03 22:23:40 +10:00
Amber Brown dc72b90cd6 full schema 2019-06-03 22:03:28 +10:00
Erik Johnston 31d44ec4bd
Merge pull request #5294 from matrix-org/erikj/speed_up_room_stats
Speed up room stats background update
2019-05-31 10:48:51 +01:00
Erik Johnston 5037326d66 Add indices. Remove room_ids accidentally added
We have to do this by re-inserting a background update and recreating
tables, as the tables only get created during a background update and
will later be deleted.

We also make sure that we remove any entries that should have been
removed but weren't due to a race that has been fixed in a previous
commit.
2019-05-31 10:26:56 +01:00
Erik Johnston 6cdfb0207e Add index to temp table 2019-05-30 15:02:31 +01:00
Erik Johnston 640fcbb07f Fixup comments and logging 2019-05-30 10:55:55 +01:00
Erik Johnston d79c9994f4 Add DB bg update to cleanup extremities.
Due to #5269 we may have extremities in our DB that we shouldn't have,
so lets add a cleanup task such to remove those.
2019-05-29 15:11:26 +01:00
Brendan Abolivier a97d4e218a
Merge pull request #5268 from matrix-org/babolivier/account_validity_fix_schema
Fix schema update for account validity
2019-05-28 10:30:07 +01:00
Brendan Abolivier ba17de7fbc
Fix schema update for account validity 2019-05-28 10:11:38 +01:00
Richard van der Hoff b75537beaf Store key validity time in the storage layer
This is a first step to checking that the key is valid at the required moment.

The idea here is that, rather than passing VerifyKey objects in and out of the
storage layer, we instead pass FetchKeyResult objects, which simply wrap the
VerifyKey and add a valid_until_ts field.
2019-05-23 11:52:22 +01:00
Amber Brown 4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
Erik Johnston efe3c7977a Add simple send_relation API and track in DB 2019-05-15 13:36:51 +01:00
Brendan Abolivier 91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier 20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00