Commit Graph

1269 Commits

Author SHA1 Message Date
Mark Haines a612ce6659 Merge pull request #489 from matrix-org/markjh/replication
Add a /replication API for extracting the updates that happened on synapse.
2016-03-01 15:08:24 +00:00
Mark Haines 60a0f81c7a Add a /replication API for extracting the updates that happened on
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Mark Haines 54172924c8 Load the current id in the IdGenerator constructor
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Mark Haines de27f7fc79 Add support for changing the actions for default rules
See matrix-org/matrix-doc#283

Works by adding dummy rules to the push rules table with a negative priority class and then using those rules to clobber the default rule actions when adding the default rules in ``list_with_base_rules``
2016-02-26 14:28:19 +00:00
Daniel Wagner-Hall 33300673b7 Generate guest access token on 3pid invites
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Erik Johnston 6451fcd085 Create a new stream_id per presence update 2016-02-23 15:51:39 +00:00
Erik Johnston e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence
Rewrite presence for performance.
2016-02-19 09:37:50 +00:00
Erik Johnston e6c5e3f28a Close cursor 2016-02-18 16:39:28 +00:00
Erik Johnston 42109a62a4 Remove unused param from get_max_token 2016-02-18 16:37:28 +00:00
Mark Haines b9977ea667 Remove dead code for setting device specific rules.
It wasn't possible to hit the code from the API because of a typo
in parsing the request path. Since no-one was using the feature
we might as well remove the dead code.
2016-02-18 16:05:13 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Daniel Wagner-Hall 7e90fb6a57 Merge branch 'develop' into daniel/roomcleanupincremental
Conflicts:
	synapse/rest/client/v1/room.py
2016-02-17 15:53:59 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall a4e278bfe7 Respond to federated invite with non-empty context
Currently, we magically perform an extra database hit to find the
inviter, and use this to guess where we should send the event. Instead,
fill in a valid context, so that other callers relying on the context
actually have one.
2016-02-17 15:25:12 +00:00
Mark Haines a9c9868957 Make adding push rules idempotent
Also remove the **kwargs from the add_push_rule method.

Fixes https://matrix.org/jira/browse/SYN-391
2016-02-16 15:53:38 +00:00
Erik Johnston ce14c7a995 Fix SYN-627, events are in incorrect room in /sync 2016-02-11 15:02:56 +00:00
Erik Johnston 0eff740523 Return events in correct order for /events 2016-02-11 10:07:27 +00:00
Erik Johnston 5189bfdef4 Batch fetch _get_state_groups_from_groups 2016-02-10 13:24:42 +00:00
Erik Johnston 24f00a6c33 Use _simple_select_many for _get_state_group_for_events 2016-02-10 12:57:50 +00:00
Erik Johnston f7ef5c1d57 Merge pull request #568 from matrix-org/erikj/unread_notif
Atomically persit push actions when we persist the event
2016-02-10 11:25:32 +00:00
Erik Johnston 0214745239 Rename functions 2016-02-10 11:09:56 +00:00
Erik Johnston 7b0d846407 Atomically persit push actions when we persist the event 2016-02-09 16:19:15 +00:00
Erik Johnston 78d6c1b5be Change a log from debug to info 2016-02-09 14:44:12 +00:00
Erik Johnston 70a8608749 Invalidate get_last_receipt_event_id_for_user cache 2016-02-09 14:27:29 +00:00
Erik Johnston eff12e838c Don't load all ephemeral state for a room on every sync 2016-02-09 13:55:59 +00:00
Erik Johnston 2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall 79a1c0574b Allocate guest user IDs numericcally
The current random IDs are ugly and confusing when presented in UIs.
This makes them prettier and easier to read.

Also, disable non-automated registration of numeric IDs so that we don't
need to worry so much about people carving out our automated address
space and us needing to keep retrying ID registration.
2016-02-05 11:22:30 +00:00
Erik Johnston 4d36e73230 Actually return something sensible 2016-02-03 16:35:00 +00:00
Erik Johnston aa4af94c69 We return dicts now. 2016-02-03 16:29:32 +00:00
Erik Johnston b84d59c5f0 Add descriptions 2016-02-03 16:22:35 +00:00
Erik Johnston 8450114098 Merge pull request #554 from matrix-org/erikj/event_push
Change event_push_actions_rm_tokens schema
2016-02-03 15:02:47 +00:00
Erik Johnston f8aae79a72 Simplify get_rooms 2016-02-03 13:24:35 +00:00
Erik Johnston 6f52e90065 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_room_fix 2016-02-03 11:06:29 +00:00
Erik Johnston 771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Erik Johnston b32121a5d1 Unused import 2016-02-03 10:30:56 +00:00
Daniel Wagner-Hall d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston 8a391e33ae s/get_room_changes_for_user/get_membership_changes_for_user/ 2016-02-02 16:12:10 +00:00
Erik Johnston 477b1ed6cf Fetch events in a separate transaction.
This has a couple of benefits:

- It reduces the time of transactions, allowing other database requests
  to run.
- Fetching events is given a dedicated database thread, and so can't
  starve other database requests.
2016-02-02 15:58:14 +00:00
Erik Johnston 65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
Erik Johnston 89b40b225c Order things correctly 2016-02-01 16:32:46 +00:00
Erik Johnston 4bf448be25 Switch over /events to use per room caches 2016-02-01 16:26:51 +00:00
Erik Johnston 2d3837bec7 Merge pull request #543 from matrix-org/erikj/sync
Cache get_room_changes_for_user
2016-02-01 15:05:06 +00:00
Erik Johnston ceb6b8680a Only use room_ids if in get_room_events_stream if is_guest 2016-02-01 10:33:52 +00:00
Erik Johnston 25c311eaf6 Cache get_room_changes_for_user 2016-01-29 16:52:48 +00:00
Erik Johnston cc9c97e0dc Invalidate _account_data_stream_cache correctly 2016-01-29 16:41:51 +00:00
Erik Johnston 8da95b6f1b Comment. Remove superfluous order by 2016-01-29 15:39:17 +00:00
Erik Johnston b5dbced938 Don't prefill account data 2016-01-29 14:53:59 +00:00
Erik Johnston 3d60686c0c Actually use cache 2016-01-29 14:49:11 +00:00
Erik Johnston 45488e0ffa Max is not a function 2016-01-29 14:42:01 +00:00
Erik Johnston f67d60496a Convert param style 2016-01-29 14:41:16 +00:00