Commit Graph

5228 Commits

Author SHA1 Message Date
Kent Shikama fc8007dbec
Optionally include password hash in createUser endpoint
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
David Baker be8be535f7 requestToken update
Don't send requestToken request to untrusted ID servers

Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
2016-06-30 17:51:28 +01:00
Erik Johnston f328d95cef Feature: Add deactivate account admin API
Allows server admins to "deactivate" accounts, which:

- Revokes all access tokens
- Removes all threepids
- Removes password

The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Erik Johnston f52cb4cd78 Remove race 2016-06-29 15:24:50 +01:00
Mark Haines 6783534a0f Merge pull request #886 from matrix-org/markjh/async_commit
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
Erik Johnston a70688445d Implement purge_media_cache admin API 2016-06-29 14:57:59 +01:00
Erik Johnston 314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
David Baker db74dcda5b Merge pull request #894 from matrix-org/dbkr/push_room_naming
Use similar naming we use in email notifs for push
2016-06-28 10:12:24 +01:00
Matthew Hodgson 63bb8f0df9 remove vector.im from default secondary DS list 2016-06-27 13:13:33 +04:00
David Baker 0b640aa56b even more pep8 2016-06-24 11:47:11 +01:00
David Baker aa3a4944d5 more pep8 2016-06-24 11:45:23 +01:00
David Baker 46b7362304 pep8 2016-06-24 11:44:57 +01:00
David Baker 870c45913e Use similar naming we use in email notifs for push
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
Mark Haines 05f1a4596a Merge branch 'master' into develop 2016-06-23 11:17:48 +01:00
David Baker b5fb7458d5 Actually we need to order these properly
otherwise we'll end up returning the wrong 20
2016-06-22 18:07:14 +01:00
David Baker f73fdb04a6 Style 2016-06-22 17:51:40 +01:00
David Baker 3a4120e49a Put most recent 20 messages in notif
Fixes https://github.com/vector-im/vector-web/issues/1648
2016-06-22 17:47:18 +01:00
Martin Weinelt 0a32208e5d Rework ldap integration with ldap3
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.

Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
  employing the `user_filter`, then tries the bind when a single
  matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
  localpart and `user_base`

Offer support for STARTTLS on a plain connection.

The configuration was changed to reflect these new possibilities.

Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2016-06-22 17:51:59 +02:00
Mark Haines 774f3a692c Merge pull request #889 from matrix-org/markjh/synctl_workers
Optionally start or stop workers in synctl.
2016-06-21 17:58:19 +01:00
Mark Haines 5cc7564c5c Optionally start or stop workers in synctl.
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.

The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
2016-06-21 16:38:05 +01:00
Mark Haines 0fe0b0eeb6 Merge pull request #888 from matrix-org/markjh/content_repo
Remove the legacy v0 content upload API.
2016-06-21 14:01:01 +01:00
Mark Haines 13e334506c Remove the legacy v0 content upload API.
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
Mark Haines d5fb561709 Optionally make committing to postgres asynchronous.
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
Erik Johnston d8ec81cc31 Merge pull request #879 from matrix-org/erikj/linearize_fed_server
Linearize some federation endpoints based on (origin, room_id)
2016-06-20 17:34:29 +01:00
Erik Johnston bc72d381b2 Merge branch 'release-v0.16.1' of github.com:matrix-org/synapse 2016-06-20 14:18:04 +01:00
Erik Johnston 4d362a61ea Bump version and changelog 2016-06-20 14:17:42 +01:00
Mark Haines 41e4b2efea Add the create_profile method back since the tests use it 2016-06-17 19:20:47 +01:00
Mark Haines 0c13d45522 Add a comment on why we don't create a profile for upgrading users 2016-06-17 19:18:53 +01:00
Mark Haines 9f1800fba8 Remove registered_users from the distributor.
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.

This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
Erik Johnston 8f4a9bbc16 Linearize some federation endpoints based on (origin, room_id) 2016-06-17 16:43:45 +01:00
Erik Johnston 9ba2bf1570 Merge pull request #878 from matrix-org/erikj/ujson
Disable responding with canonical json for federation
2016-06-17 16:22:12 +01:00
Erik Johnston 120c238705 Disable responding with canonical json for federation 2016-06-17 16:10:37 +01:00
Erik Johnston 1c1f633b13 Merge pull request #877 from matrix-org/erikj/frozen_default
Turn use_frozen_events off by default
2016-06-17 15:33:55 +01:00
Erik Johnston 6660f37558 Merge pull request #876 from matrix-org/erikj/sign_own
Only re-sign our own events
2016-06-17 15:23:20 +01:00
Erik Johnston 3e41de05cc Turn use_frozen_events off by default 2016-06-17 15:11:22 +01:00
Erik Johnston 2884712ca7 Only re-sign our own events 2016-06-17 14:47:33 +01:00
Mark Haines ded01c3bf6 Fix ``KeyError: 'msgtype'``. Use ``.get``
Fixes a key error where the mailer tried to get the ``msgtype`` of an
event that was missing a ``msgtype``.

```
 File "synapse/push/mailer.py", line 264, in get_notif_vars
 File "synapse/push/mailer.py", line 285, in get_message_vars
 File ".../frozendict/__init__.py", line 10, in __getitem__
    return self.__dict[key]
    KeyError: 'msgtype'
```
2016-06-17 13:49:16 +01:00
Mark Haines 8c75040c25 Fix setting gc thresholds in the workers 2016-06-17 11:48:12 +01:00
Mark Haines a352b68acf Use worker_ prefixes for worker config, use existing support for multiple config files 2016-06-16 17:29:50 +01:00
Mark Haines 364d616792 Access the event_cache_size directly from the server object.
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
Mark Haines bde13833cb Access replication_url from the worker config directly 2016-06-16 12:44:40 +01:00
Mark Haines 80a1bc7db5 Comment on what's going on in clobber_with_worker_config 2016-06-16 11:29:45 +01:00
Mark Haines f1f70bf4b5 Merge remote-tracking branch 'origin/develop' into markjh/worker_config 2016-06-16 11:20:17 +01:00
Mark Haines dbb5a39b64 Add worker config module 2016-06-16 11:09:15 +01:00
Mark Haines 885ee861f7 Inline the synchrotron and pusher configs into the main config 2016-06-16 11:06:12 +01:00
Erik Johnston 486b9a6a2d Merge pull request #873 from vt0r/bugfix/bcrypt-utf8-encode
Fix TypeError in call to bcrypt.hashpw
2016-06-16 10:12:17 +01:00
Salvatore LaMendola ed5f43a55a
Fix TypeError in call to bcrypt.hashpw
- At the very least, this TypeError caused logins to fail on my own
  running instance of Synapse, and the simple (explicit) UTF-8
  conversion resolved login errors for me.

Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>
2016-06-16 00:43:42 -04:00
Erik Johnston 09a17f965c Line lengths 2016-06-15 16:58:12 +01:00
Erik Johnston 1e9026e484 Handle floats as img widths 2016-06-15 16:58:05 +01:00
Erik Johnston a60169ea09 Handle og props with not content 2016-06-15 16:57:48 +01:00
Erik Johnston 0ef0655b83 Bump version and changelog 2016-06-15 15:50:48 +01:00
Erik Johnston d41a1a91d3 Linearize fetching of gaps on incoming events
This potentially stops the server from doing multiple requests for the
same data.
2016-06-15 15:16:14 +01:00
Richard van der Hoff 15bf3e3376 Merge pull request #870 from matrix-org/rav/work_around_tls_bug
Work around TLS bug in twisted
2016-06-15 13:28:13 +01:00
Erik Johnston b31c49d676 Correctly mark backfilled events as backfilled 2016-06-15 10:59:08 +01:00
Richard van der Hoff 255c229f23 Work around TLS bug in twisted
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473. Hopefully this fixes
https://matrix.org/jira/browse/SYN-700.
2016-06-15 10:39:08 +01:00
Erik Johnston d12134ce37 Merge pull request #868 from matrix-org/erikj/invalid_id
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 15:09:07 +01:00
Erik Johnston 36e2aade87 Make get_domain_from_id throw SynapseError on invalid ID 2016-06-14 13:25:29 +01:00
Matthew Hodgson 33546b58aa point to the CAPTCHA docs 2016-06-12 23:11:29 +01:00
Mark Haines 41b35412bf Merge pull request #863 from matrix-org/markjh/load_config
Add function to load config without generating it
2016-06-10 11:12:32 +01:00
Mark Haines 7dbb473339 Add function to load config without generating it
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.

The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.

As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
Erik Johnston 16a8884233 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-06-09 16:57:33 +01:00
Erik Johnston a31befbcd0 Bump version and changelog 2016-06-09 13:23:41 +01:00
Erik Johnston 5ee5b655b2 Merge pull request #862 from matrix-org/erikj/media_remote_error
502 on /thumbnail when can't contact remote server
2016-06-09 13:11:02 +01:00
Erik Johnston 0a43219a27 Merge pull request #860 from negzi/bug_fix_get_or_create_user
Fix a bug caused by a change in auth_handler function
2016-06-09 11:40:12 +01:00
Erik Johnston eba4ff1bcb 502 on /thumbnail when can't contact remote server 2016-06-09 11:29:43 +01:00
Erik Johnston 95f305c35a Remove redundant exception log in /events 2016-06-09 11:15:04 +01:00
Negar Fazeli 6e7dc7c7dd Fix a bug caused by a change in auth_handler function
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
Erik Johnston b7fbc9bd95 Merge pull request #859 from matrix-org/erikj/public_room_performance
Pull full state for each room all at once
2016-06-08 16:36:06 +01:00
Erik Johnston 919a2c74f6 Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop 2016-06-08 16:26:26 +01:00
Erik Johnston 81c07a32fd Pull full state for each room all at once 2016-06-08 15:51:49 +01:00
Erik Johnston 690029d1a3 Don't make rooms visibile by default 2016-06-08 14:47:42 +01:00
Erik Johnston d88faf92d1 Fix up federation PublicRoomList 2016-06-08 14:39:31 +01:00
Erik Johnston 958c968d02 Merge pull request #856 from matrix-org/erikj/fed_pub_rooms
Enable auth on /publicRoom endpoints
2016-06-08 14:36:09 +01:00
Erik Johnston 746b2f5657 Merge pull request #854 from matrix-org/erikj/federation_logging
Add some logging for when servers ask for missing events
2016-06-08 14:31:41 +01:00
Erik Johnston efeabd3180 Log user that is making /publicRooms calls 2016-06-08 14:23:15 +01:00
Erik Johnston 1fd6eb695d Enable auth on federation PublicRoomList 2016-06-08 14:15:18 +01:00
Erik Johnston 17aab5827a Add some logging for when servers ask for missing events 2016-06-08 11:55:31 +01:00
Erik Johnston 1a815fb04f Don't hit DB for noop replications queries 2016-06-08 11:33:30 +01:00
Erik Johnston bab916bccc Bump version and changelog to v0.16.0-rc2 2016-06-08 11:05:45 +01:00
Erik Johnston e0fda29f94 Merge pull request #850 from matrix-org/erikj/gc_threshold
Add gc_threshold to pusher and synchrotron
2016-06-08 10:05:56 +01:00
Erik Johnston 0f2165ccf4 Don't track total objects as its too expensive to calculate 2016-06-07 17:00:45 +01:00
Erik Johnston 18f0cc7d99 Record some more GC metrics 2016-06-07 16:55:49 +01:00
Erik Johnston 2d1d1025fa Add gc_threshold to pusher and synchrotron 2016-06-07 16:26:25 +01:00
Erik Johnston b01e71e719 Merge pull request #849 from matrix-org/erikj/gc_threshold
Allow setting of gc.set_thresholds
2016-06-07 16:07:54 +01:00
Erik Johnston dded389ac1 Allow setting of gc.set_thresholds 2016-06-07 15:45:56 +01:00
Mark Haines 0b2158719c Remove dead code.
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.

The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.

The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston 48e65099b5 Also record number of unreachable objects 2016-06-07 13:40:22 +01:00
Erik Johnston 75331c5fca Change the way we do stats 2016-06-07 13:33:13 +01:00
Erik Johnston 8c966fbd51 Merge pull request #771 from matrix-org/erikj/gc_tick
Manually run GC on reactor tick.
2016-06-07 13:18:36 +01:00
Mark Haines 88625db05f Notify users for events in rooms they join.
Change how the notifier updates the map from room_id to user streams on
receiving a join event. Make it update the map when it notifies for the
join event, rather than using the "user_joined_room" distributor signal
2016-06-07 11:33:36 +01:00
Erik Johnston 84379062f9 Fix AS retries, but with correct ordering 2016-06-07 10:24:50 +01:00
Erik Johnston 310197bab5 Fix AS retries 2016-06-07 09:34:50 +01:00
Mark Haines b0932b34cb Merge pull request #845 from matrix-org/markjh/synchrotron_presence
Fix a KeyError in the synchrotron presence
2016-06-06 16:52:27 +01:00
Mark Haines 4a5bbb1941 Fix a KeyError in the synchrotron presence 2016-06-06 16:37:12 +01:00
Mark Haines 87f60e7053 Merge pull request #844 from matrix-org/markjh/yield_on_sleep
Yield on the sleeps intended to backoff replication
2016-06-06 16:20:54 +01:00
Mark Haines 5ef84da4f1 Yield on the sleeps intended to backoff replication 2016-06-06 16:05:28 +01:00
Erik Johnston 216a05b3e3 .values() returns list of sets 2016-06-06 16:00:09 +01:00
Erik Johnston 96dc600579 Fix typos 2016-06-06 15:44:41 +01:00
Erik Johnston 377eb480ca Fire after 30s not 8h 2016-06-06 15:14:21 +01:00
Erik Johnston e4134c5e13 Merge pull request #841 from matrix-org/erikj/event_counter
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
Erik Johnston 7aa778fba9 Add metric counter for number of persisted events 2016-06-06 11:58:09 +01:00
Erik Johnston 70aee0717c Add events to cache when we persist them 2016-06-06 11:34:53 +01:00
Erik Johnston 3210f4c385 Merge pull request #836 from matrix-org/erikj/change_event_cache
Change the way we cache events
2016-06-03 18:47:46 +01:00
Erik Johnston cffe46408f Don't rely on options when inserting event into cache 2016-06-03 18:25:21 +01:00
Mark Haines ac9716f154 Fix spelling 2016-06-03 18:10:00 +01:00
Mark Haines 8f79084bd4 Add get_presence_list_accepted to the broken caches in synchrotron 2016-06-03 18:03:40 +01:00
Erik Johnston 10ea3f46ba Change the way we cache events 2016-06-03 17:57:50 +01:00
Erik Johnston f6be734be9 Merge pull request #835 from matrix-org/erikj/get_event_txn
Remove event fetching from DB threads
2016-06-03 17:30:00 +01:00
Erik Johnston 05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
David Baker 72c4d482e9 3rd time lucky: we'd already calculated it above 2016-06-03 16:39:50 +01:00
David Baker fbf608decb Oops, we're using the dict form 2016-06-03 16:38:39 +01:00
David Baker 06d40c8b98 Add substitutions to email notif From 2016-06-03 16:31:23 +01:00
Erik Johnston 5f88549f4a Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop 2016-06-03 16:27:24 +01:00
Mark Haines ca457f594e Merge pull request #831 from matrix-org/markjh/synchrotronII
Add a separate process that can handle /sync requests
2016-06-03 16:06:25 +01:00
Erik Johnston 21961c93c7 Bump changelog and version 2016-06-03 15:33:14 +01:00
Mark Haines 48340e4f13 Clear the list of ongoing syncs on shutdown 2016-06-03 15:02:27 +01:00
Erik Johnston fcbc282f56 Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0 2016-06-03 15:02:04 +01:00
Erik Johnston 51773bcbaf Merge pull request #832 from matrix-org/erikj/presence_coount
Change def of small delta in presence stream. Add metrics.
2016-06-03 14:57:00 +01:00
Mark Haines da491e75b2 Appease flake8 2016-06-03 14:56:36 +01:00
Mark Haines 0b3c80a234 Use ClientIpStore to record client ips 2016-06-03 14:55:01 +01:00
Mark Haines dd6f62ed99 Merge branch 'develop' into markjh/synchrotronII 2016-06-03 14:51:33 +01:00
Mark Haines eef541a291 Move insert_client_ip to a separate class 2016-06-03 14:42:35 +01:00
Mark Haines 80aade3805 Send updates to the syncing users every ten seconds or immediately if they've just come online 2016-06-03 14:24:19 +01:00
Erik Johnston ab116bdb0c Fix typo 2016-06-03 14:03:42 +01:00
Erik Johnston 4ce84a1acd Change metric style 2016-06-03 13:49:16 +01:00
Erik Johnston a7ff5a1770 Presence metrics. Change def of small delta 2016-06-03 13:40:55 +01:00
Matthew Hodgson 8d740132f4 Merge branch 'develop' into matthew/brand-from-header 2016-06-03 12:14:18 +01:00
Erik Johnston 3b096c5f5c Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop 2016-06-03 12:00:33 +01:00
Erik Johnston 43b7f371f5 Merge pull request #830 from matrix-org/erikj/metrics_perf
Change CacheMetrics to be quicker
2016-06-03 11:57:29 +01:00
Mark Haines abb151f3c9 Add a separate process that can handle /sync requests 2016-06-03 11:57:26 +01:00
Erik Johnston 4982b28868 Merge pull request #829 from matrix-org/erikj/poke_notifier
Poke notifier on next reactor tick
2016-06-03 11:52:10 +01:00
Erik Johnston d06f2a229e Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston 58a224a651 Pull out update_results_dict 2016-06-03 11:47:07 +01:00
Mark Haines 20eccd84d4 Merge pull request #827 from matrix-org/markjh/more_slaved_methods
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
Erik Johnston 722472b48c Merge pull request #825 from matrix-org/erikj/cache_push_rules
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Erik Johnston 73c7112433 Change CacheMetrics to be quicker
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines b09f348530 Merge pull request #824 from matrix-org/markjh/slaved_presence_store
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston 4c04222fa5 Poke notifier on next reactor tick 2016-06-03 11:24:16 +01:00
Erik Johnston ccb56fc24b Make get_joined_hosts_for_room use get_users_in_room 2016-06-03 11:20:23 +01:00
Mark Haines 81cf449daa Add methods to events, account data and receipt slaves
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
Erik Johnston e043ede4a2 Small optimisation to CacheListDescriptor 2016-06-03 11:19:22 +01:00
Mark Haines b821c839dc Merge pull request #823 from matrix-org/markjh/more_slaved_stores
Add slaved stores for filters, tokens, and push rules
2016-06-03 11:17:43 +01:00
Erik Johnston 597013caa5 Make cachedList go a bit faster 2016-06-03 11:13:29 +01:00
Erik Johnston 6a0afa582a Load push rules in storage layer, so that they get cached 2016-06-03 11:10:00 +01:00
Mark Haines 3ae915b27e Add a slaved store for presence 2016-06-03 11:05:53 +01:00
Erik Johnston 59f2d73522 Remove unnecessary sets 2016-06-03 11:05:45 +01:00
Erik Johnston 9c26b390a2 Only get local users 2016-06-03 11:04:31 +01:00
Mark Haines f88d747f79 Add a comment explaining why the filter cache doesn't need exipiring 2016-06-03 11:03:10 +01:00
Erik Johnston 065e739d6e Merge pull request #811 from matrix-org/erikj/state_users_in_room
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston 696d7c5937 Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines 0eae075723 Add slaved stores for filters, tokens, and push rules 2016-06-03 10:58:03 +01:00
Matthew Hodgson 79d1f072f4 brand the email from header 2016-06-02 21:34:40 +01:00
David Baker 6bb9aacf9d Merge pull request #821 from matrix-org/dbkr/email_unsubscribe
Email unsubscribe links that don't require logging in
2016-06-02 17:44:55 +01:00
David Baker 745ddb4dd0 peppate 2016-06-02 17:38:41 +01:00
David Baker 7a5a5f2df2 Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error
Fix error in email notification string formatting
2016-06-02 17:26:06 +01:00
David Baker 1f31cc37f8 Working unsubscribe links going straight to the HS
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
Matthew Hodgson 2675c1e40e add some branding debugging 2016-06-02 17:21:12 +01:00
David Baker c71177f285 Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe 2016-06-02 17:20:56 +01:00
David Baker 07a5559916 Fix error in email notification string formatting 2016-06-02 17:17:16 +01:00
Mark Haines 56d15a0530 Store the typing users as user_id strings. (#819)
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
David Baker 812b5de0fe Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe 2016-06-02 15:33:28 +01:00
Mark Haines 80f34d7b57 Fix setting the _clock in SQLBaseStore 2016-06-02 15:23:56 +01:00
Mark Haines 661a540dd1 Deduplicate presence entries in sync (#818) 2016-06-02 15:20:28 +01:00
Mark Haines 70599ce925 Allow external processes to mark a user as syncing. (#812)
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
2016-06-02 15:20:15 +01:00
David Baker fb2193cc63 Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler
Split out the auth handler
2016-06-02 14:31:35 +01:00
Erik Johnston 356f13c069 Disable INCLUDE_ALL_UNREAD_NOTIFS 2016-06-02 14:07:38 +01:00
Erik Johnston 02ac463dbf Merge pull request #800 from matrix-org/erikj/sync_refactor
Refactor SyncHandler
2016-06-02 14:02:13 +01:00
Matthew Hodgson c5af1b6b00 Merge pull request #814 from matrix-org/matthew/3pid_invite_auth
special case m.room.third_party_invite event auth to match invites,
2016-06-02 13:47:40 +01:00
David Baker 3a3fb2f6f9 Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe 2016-06-02 13:35:25 +01:00
David Baker 4a10510cd5 Split out the auth handler 2016-06-02 13:31:45 +01:00
Matthew Hodgson f84b89f0c6 if an email pusher specifies a brand param, use it 2016-06-02 13:29:48 +01:00
David Baker a15ad60849 Email unsubscribing that may in theory, work
Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.
2016-06-02 11:44:15 +01:00
David Baker e793866398 Use user_id in email greeting if display name is null 2016-06-02 09:41:13 +01:00
Matthew Hodgson aaa70e26a2 special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208 2016-06-01 22:13:47 +01:00
Erik Johnston a04a2d043c Merge pull request #807 from matrix-org/erikj/push_rules_cache
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
Erik Johnston 0f06b496d1 Merge pull request #806 from matrix-org/erikj/hash_cache
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
David Baker e0deeff23e Fix room list spidering 2016-06-01 17:58:58 +01:00
David Baker 991af8b0d6 WIP on unsubscribing email notifs without logging in 2016-06-01 17:40:52 +01:00
David Baker 00c487a8db Merge pull request #808 from matrix-org/dbkr/room_list_spider
Add secondary_directory_servers option to fetch room list from other servers
2016-06-01 15:32:52 +01:00
Erik Johnston c8285564a3 Use state to calculate get_users_in_room 2016-06-01 15:25:25 +01:00
David Baker d60eed0710 Limit number of notifications in an email notification 2016-06-01 11:45:43 +01:00
Erik Johnston 43db0d9f6a Add get_users_with_read_receipts_in_room cache 2016-06-01 10:54:32 +01:00
David Baker 8e539f13c0 Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider 2016-06-01 09:54:36 +01:00
David Baker 6ecb2ca4ec pep8 2016-06-01 09:48:55 +01:00
David Baker 2a449fec4d Add cache to remote room lists
Poll for updates from remote servers, waiting for the poll if there's no cache entry.
2016-05-31 18:27:23 +01:00
David Baker 6ca4d3ae9a Add vector.im to default secondary_directory_servers and add comment explaining it's not a permanent solution 2016-05-31 17:24:50 +01:00
Erik Johnston dea9f20f8c Force boolean 2016-05-31 17:24:30 +01:00
David Baker 963e3ed282 Apparently I am not permitted to have two blank lines here 2016-05-31 17:22:53 +01:00
David Baker d240796ded Basic, un-cached support for secondary_directory_servers 2016-05-31 17:20:07 +01:00
Mark Haines c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston c9ca285d33 Merge pull request #805 from matrix-org/erikj/push_rules_cache
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
Erik Johnston 1d4ee854e2 Fix typo 2016-05-31 15:45:53 +01:00
Erik Johnston cca0093fa9 Change fix 2016-05-31 15:44:08 +01:00
Erik Johnston 4efa389299 Fix GET /push_rules 2016-05-31 15:37:53 +01:00
Erik Johnston aefd2d1cbc Cache get_event_reference_hashes 2016-05-31 15:32:32 +01:00
Erik Johnston 10de8c2631 Merge pull request #804 from matrix-org/erikj/push_rules_cache
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
Mark Haines c626fc576a Move the AS handler out of the Handlers object.
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Erik Johnston e5b0bbcd33 Add caches to bulk_get_push_rules* 2016-05-31 13:46:58 +01:00
David Baker 70ecb415f5 Fix c+p fail 2016-05-31 12:00:54 +01:00
David Baker e1625d62a8 Add federation room list servlet 2016-05-31 11:55:57 +01:00
David Baker 887c6e6f05 Split out the room list handler
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
Erik Johnston 85b992f621 Fix to allow start with postgres 2016-05-27 10:44:44 +01:00
Erik Johnston cc84f7cb8e Send down correct error response if user not found 2016-05-27 10:35:15 +01:00
David Baker 209ba4d024 Merge pull request #795 from matrix-org/dbkr/delete_push_actions_after_a_month
Only delete push actions after 30 days
2016-05-24 16:22:13 +01:00
David Baker b007ee4606 Check for presence of 'avatar_url' key 2016-05-24 15:12:05 +01:00
David Baker cbf8d146ac Merge pull request #799 from matrix-org/matthew/quieter-email-notifs
Tune email notifs to make them quieter:
2016-05-24 14:30:51 +01:00
Erik Johnston faad233ea6 Change short circuit path 2016-05-24 14:27:19 +01:00
Erik Johnston 6900303997 Don't send down all ephemeral events 2016-05-24 11:44:55 +01:00
David Baker 37b7e84620 Include the ts the notif was received at 2016-05-24 11:33:32 +01:00
Erik Johnston 1c5ed2a19b Only work out newly_joined_users for incremental sync 2016-05-24 11:21:34 +01:00
Erik Johnston b08ad0389e Only include non-offline presence in initial sync 2016-05-24 11:15:05 +01:00
Erik Johnston be2c677386 Spell builder correctly 2016-05-24 10:53:03 +01:00
Erik Johnston 79bea8ab9a Inline function. Make load_filtered_recents private 2016-05-24 10:22:24 +01:00
Erik Johnston 84f94e4cbb Add comments 2016-05-24 10:14:53 +01:00
Erik Johnston 137e6a4557 Shuffle things room 2016-05-24 09:50:55 +01:00
Matthew Hodgson 680f1d9387 catch thinko in presentable names 2016-05-23 22:55:11 +01:00
Matthew Hodgson cb8a321bdd fix NPE in room ordering 2016-05-23 22:54:56 +01:00
Matthew Hodgson 88ea5ab2c3 consistency is the better part of valour 2016-05-23 19:33:45 +01:00
Matthew Hodgson 989bdc9e56 Tune email notifs to make them quieter:
* After initial 10 minute window, only alert every 24h for room notifs
 * Reset room state after 6h of idleness
 * Synchronise throttles for messages sent in the same notif, so the 24 hourly notifs 'line up'
 * Fix the email subjects to say what triggered the notification
 * Order the rooms in reverse activity order in the email, so the 'reason' room should always come first
2016-05-23 19:24:11 +01:00
Negi Fazeli 6fe04ffef2 Fix set profile error with Requester.
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
David Baker b791a530da Actually make the 'read' flag correct 2016-05-23 18:48:02 +01:00
David Baker a24bc5b2dc Add GET /notifications API 2016-05-23 18:33:51 +01:00
Erik Johnston c0c79ef444 Add back concurrently_execute 2016-05-23 18:21:27 +01:00
Erik Johnston b5605dfecc Refactor SyncHandler 2016-05-23 18:08:18 +01:00
David Baker 31b5395ab6 Remove debug logging 2016-05-23 16:32:01 +01:00
David Baker c2da3406fc Oops, missing comma 2016-05-20 18:03:31 +01:00
David Baker ccffb0965d Remove stale line 2016-05-20 17:59:10 +01:00
David Baker 18d68bfee4 Handle empty events table 2016-05-20 17:58:09 +01:00
David Baker d4503e25ed Make deleting push actions more efficient
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
David Baker 149fa411e2 Only delete push actions after 30 days 2016-05-20 15:25:12 +01:00
Kegan Dougal 332d7e9b97 Allow clients to specify a server_name to avoid 'No known servers'
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
Matthew Hodgson 6fb51eaf7b Merge pull request #793 from matrix-org/matthew/one-push-badge-per-convo
increment badge count per missed convo, not per msg
2016-05-18 13:56:38 +01:00
Matthew Hodgson e837df6adb increment badge count per missed convo, not per msg 2016-05-18 11:53:25 +01:00
Erik Johnston 42368ea8db Add desc to get_presence_for_users 2016-05-18 11:38:10 +01:00
Mark Haines 0cb441fedd Move typing handler out of the Handlers object 2016-05-17 15:58:46 +01:00
Mark Haines 6a30a0bfd3 Move the functions for parsing app service config 2016-05-17 11:28:58 +01:00
Mark Haines 523d5bcd0b Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler 2016-05-17 10:43:58 +01:00
Matthew Hodgson 43e1e0489c Merge pull request #786 from matrix-org/matthew/email_notifs_tuning
tune email notifs, fix CSS a bit, and add debugging details
2016-05-17 10:43:24 +01:00
Mark Haines 526bf8126f Remove unused get_joined_rooms_for_user 2016-05-17 10:20:51 +01:00
Mark Haines 425e6b4983 Merge branch 'develop' into markjh/member_cleanup 2016-05-17 10:13:16 +01:00
Mark Haines b153f5b150 Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
Mark Haines 3fd8a07fca Merge pull request #788 from matrix-org/markjh/domian
Spell "domain" correctly
2016-05-17 10:09:34 +01:00
Mark Haines f68eea808a Move SyncHandler out of the Handlers object 2016-05-16 20:19:26 +01:00
Mark Haines 53e171f345 Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler 2016-05-16 20:08:32 +01:00
Mark Haines 80cb9becd8 Remove get_joined_rooms_for_user from RoomMemberHandler 2016-05-16 20:06:55 +01:00
Mark Haines 816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines 821306120a Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room 2016-05-16 19:48:07 +01:00
Mark Haines 1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines e168abbcff Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler 2016-05-16 19:08:40 +01:00
Matthew Hodgson cbd2adc95e tune email notifs, fix CSS a bit, and add debugging details 2016-05-16 18:58:38 +01:00
Mark Haines 3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
David Baker 647781ca56 Fix emailpusher import
Try importing at the root level rather than conditionally importing, as per comment
2016-05-16 18:41:32 +01:00
Erik Johnston c39f305067 os.environ requires a string 2016-05-16 17:21:30 +01:00
Erik Johnston 678c8a7f1e Merge pull request #785 from matrix-org/erikj/cache_factor_synctly
Make synctl read a cache factor from config file
2016-05-16 17:07:35 +01:00
Erik Johnston c5c5a7403b Make synctl read a cache factor from config file 2016-05-16 17:01:57 +01:00
Mark Haines eb79110beb Clean up the blacklist/whitelist handling.
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
Mark Haines dd95eb4cb5 Merge branch 'develop' into matthew/preview_url_ip_whitelist 2016-05-16 12:59:41 +01:00
Erik Johnston 60d53f9e95 Count number of GC collects 2016-05-16 09:34:42 +01:00
Matthew Hodgson 782471b7e1 fix matrix.to URLs 2016-05-13 17:50:16 +01:00
Mark Haines 0466454b00 Assert that stream replicated stream positions are ints 2016-05-13 17:33:44 +01:00
Mark Haines b3f29dc1e5 Manually expire broken caches like the who_forgot_in_room 2016-05-13 17:16:27 +01:00
Mark Haines f03ddc98ec Use the SlavedAccountDataStore 2016-05-13 17:01:28 +01:00
Mark Haines 1f71f386f6 Merge branch 'develop' into dbkr/email_notifs_on_pusher 2016-05-13 16:59:56 +01:00
Mark Haines 206eb9fd94 Shift some of the state_group methods into the SlavedEventStore 2016-05-13 16:58:14 +01:00
Erik Johnston 7d6e89ed22 Add a comment 2016-05-13 16:31:08 +01:00
Mark Haines 21018c2c13 Merge pull request #783 from matrix-org/markjh/slave_account_data
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
Mark Haines b7381d5338 Allow receipts for events we haven't seen in the db 2016-05-13 15:46:41 +01:00
Mark Haines 3abab26458 Add a slaved datastore for account data 2016-05-13 15:34:06 +01:00
Erik Johnston 99b5a2e560 Merge pull request #741 from negzi/create_user_with_expiry
Create user with expiry
2016-05-13 14:46:53 +01:00
Erik Johnston ba5c616ff4 Merge pull request #778 from matrix-org/erikj/add_pusher
Fixup add_pusher
2016-05-13 14:43:23 +01:00
Erik Johnston 0c11c1be88 Spelling 2016-05-13 14:42:25 +01:00
Erik Johnston e00e8f2166 Merge pull request #769 from matrix-org/erikj/push_actions_delete
Delete old pushers
2016-05-13 14:41:36 +01:00
Erik Johnston fd8e921b6e Merge pull request #779 from matrix-org/erikj/receipts
Use tree cache for get_linearized_receipts_for_room
2016-05-13 14:41:21 +01:00
Erik Johnston c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Negi Fazeli 40aa6e8349 Create user with expiry
- Add unittests for client, api and handler

Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
Mark Haines 9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston 5e50058473 Remove unused indices
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Mark Haines cdda850ce1 Merge pull request #781 from matrix-org/markjh/replication_problems
Fix a bug in replication that was causing the pusher to tight loop
2016-05-13 12:05:19 +01:00
Mark Haines 0e792e7903 Log the stream IDs in an order that makes sense 2016-05-13 11:54:44 +01:00
Mark Haines 3547e66bc6 Make sure we advance our stream position 2016-05-13 11:53:00 +01:00
Erik Johnston 6da7f39d95 Use tree cache for get_linearized_receipts_for_room 2016-05-13 11:41:23 +01:00
David Baker b5e646a18c Make email notifs work on the pusher synapse
Plus general bugfix to email notif code
2016-05-13 11:36:50 +01:00
Erik Johnston 13d37c3c56 Fixup add_pusher 2016-05-13 11:25:02 +01:00
Mark Haines a458a40337 missed a spot 2016-05-12 18:19:58 +01:00
Mark Haines 7e23476814 move filter_events_for_client out of base handler 2016-05-11 13:42:37 +01:00
Mark Haines 1620578b13 Shuffle when we get the signing_key attribute.
Wait until we sign a message to get the signing key from the homeserver
config. This means that the message handler can be created without
having a signing key in the config which means that separate processes
like the pusher that don't send messages and don't need to sign them can
still access the handlers.
2016-05-11 12:20:57 +01:00
Erik Johnston 108434e53d Merge pull request #775 from matrix-org/erikj/password_hash
Correctly handle NULL password hashes from the database
2016-05-11 12:18:13 +01:00
Erik Johnston 1400bb1663 Correctly handle NULL password hashes from the database 2016-05-11 12:06:02 +01:00
Mark Haines 458a435114 Fix typo 2016-05-11 10:35:33 +01:00
Mark Haines 30057b1e15 Move _create_new_client_event and handle_new_client_event out of base handler 2016-05-11 09:09:20 +01:00
David Baker ae1af262f6 Pass through _get_state_group_for_events 2016-05-10 19:18:03 +02:00
David Baker 90afc07f39 StateStore, not EventsStore 2016-05-10 19:10:46 +02:00
David Baker 89b5ef7c4b Cached functions must be accessed through the dict 2016-05-10 19:05:22 +02:00
David Baker 35b6e6d2a8 Pass though _get_state_group_for_events 2016-05-10 18:56:40 +02:00
David Baker 3367e65476 Pass through get_state_groups 2016-05-10 18:53:15 +02:00
David Baker 0c4ccdcb83 Also pass through get_profile_displayname 2016-05-10 18:51:14 +02:00
David Baker f28643cea9 Uncommit accidentally commited edit to cipher list 2016-05-10 18:44:32 +02:00
David Baker 5f46be19a7 Pass through get_events to pusher too 2016-05-10 18:43:40 +02:00
David Baker d46b18a00f Pass through _get_event_txn 2016-05-10 18:27:06 +02:00
Matrix 3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
Matthew Hodgson fe97b81c09 Merge pull request #759 from matrix-org/dbkr/email_notifs
Send email notifications for missed messages
2016-05-10 16:30:05 +02:00
David Baker 997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
David Baker c00b484eff More consistent config naming 2016-05-10 14:39:16 +02:00
David Baker 94040b0798 Add config option to not send email notifs for new users 2016-05-10 14:34:53 +02:00
David Baker e04b1d6b0a Make pep8 happy 2016-05-10 14:23:16 +02:00
Matthew Hodgson 5599608887 Switch from CSS to Table layout for HTML mails so they work in Outlook aka Word
Remove templates-vector and theme templates with variables instead
    Switch to matrix.to URLs by default for links
2016-05-10 00:14:48 +02:00
Erik Johnston c9eb6dfc1b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:21:06 +01:00
Erik Johnston 3f84da139c Merge pull request #773 from matrix-org/erikj/get_domian_from_id
Add and use get_domain_from_id
2016-05-09 13:21:00 +01:00
Erik Johnston def64d6ef3 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:05:09 +01:00
Erik Johnston 100e2c42f6 Merge pull request #764 from matrix-org/erikj/replication_logging
Add some log information at returned replication streams
2016-05-09 11:18:00 +01:00
Erik Johnston 8715731559 Merge pull request #772 from matrix-org/erikj/get_user_cache
Add cache to get_user_by_id
2016-05-09 11:12:11 +01:00
Erik Johnston 34b3af3363 Merge pull request #770 from matrix-org/erikj/transaction_reactor
Run transaction queue on reactor
2016-05-09 11:12:06 +01:00
Erik Johnston 08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
Erik Johnston 1f1dee94f6 Manually run GC on reactor tick.
This also adds a metric for amount of time spent in GC.
2016-05-09 10:13:25 +01:00
Erik Johnston f6ebaf4a32 Run transaction queue on reactor
This ensures that any CPU work that happens doesn't block message
sending.
2016-05-09 10:10:06 +01:00
Erik Johnston 4ea762c1a2 Add cache to get_user_by_id 2016-05-09 10:08:21 +01:00
Erik Johnston 012cb5416c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete 2016-05-06 15:59:20 +01:00
Erik Johnston fcb2c3f0db Remove unused import 2016-05-06 15:47:40 +01:00
Erik Johnston fd85b167ec Pull loop one level up 2016-05-06 15:38:42 +01:00
Erik Johnston b6e0be701e Queue events for persistence 2016-05-06 14:31:44 +01:00
Erik Johnston d13459636f Pull prev txn from in memory 2016-05-06 11:30:55 +01:00
Erik Johnston 1d275dba69 Don't needlessly enter transaction 2016-05-06 11:25:58 +01:00
Erik Johnston 56b5e83e36 Reduce database inserts when sending transactions 2016-05-06 11:20:18 +01:00
Matthew Hodgson 53ca739f1f better mail subject lines 2016-05-05 15:55:44 +01:00
Matthew Hodgson 81c2176cba fix layout; handle app naming in synapse, not jinja 2016-05-05 15:54:29 +01:00
Mark Haines 573ef3f1c9 Rename openid/token to openid/request_token 2016-05-05 15:15:00 +01:00
Erik Johnston 8940281d1b Don't warn 2016-05-05 15:10:03 +01:00
Mark Haines 9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston 5d8a93a10e Add some log information at returned replication streams 2016-05-05 10:29:21 +01:00
Erik Johnston 1f0f5ffa1e Add bulk fetch storage API 2016-05-05 10:03:15 +01:00
Matthew Hodgson 634efb65f1 pep8 2016-05-05 02:10:57 +01:00
Matthew Hodgson ce81ccb063 handle fragments correctly on mxc URLs.
switch to vector.im permalinks as matrix.to isn't ready yet.
merge overlapping notifications together.
give one message of context after a notification (in the unlikely event it exists, but it's possible thanks to throttling).
include name of app in mail templates
2016-05-05 02:00:33 +01:00
Matthew Hodgson 1cf5c379cb spell out emailpusher full path 2016-05-05 01:59:39 +01:00
Erik Johnston fee1118a20 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-04 19:08:27 +01:00
Erik Johnston 97b9141245 Merge pull request #762 from matrix-org/erikj/report_event
Add /report endpoint
2016-05-04 19:05:49 +01:00
Erik Johnston fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston 8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
David Baker 39d0a99972 Include no context
until we can de-dup between the context and other notifs
2016-05-04 14:52:49 +01:00
David Baker 9ef05a12c3 Add date header & message id 2016-05-04 14:52:10 +01:00
David Baker 80be396464 Correct SQL statement for postgres
In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).
2016-05-04 13:19:59 +01:00
Erik Johnston 5650e38e7d Move event_id to path 2016-05-04 13:19:39 +01:00
David Baker 8cc82aad87 Add db functions used for email to the pusher app 2016-05-04 11:47:59 +01:00
David Baker de22001ab5 pep8 2016-05-04 11:41:35 +01:00
Matthew Hodgson f1026418ea copyright 2016-05-04 11:38:01 +01:00
Matthew Hodgson 17cbf773b9 fix assorted typos in default config 2016-05-04 11:38:01 +01:00
Erik Johnston 984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker e6bffa4475 Unused import 2016-05-04 11:26:58 +01:00
David Baker 92f0f3d21d Catch all exceptions when creating a pusher 2016-05-04 11:24:07 +01:00
Erik Johnston a438a6d2bc Implement basic ignore user 2016-05-04 10:16:46 +01:00
Erik Johnston 7ea3b4118d Merge pull request #757 from matrix-org/erikj/event_auth_typo
Fix typo in event_auth servlet path
2016-05-03 14:23:15 +01:00
Erik Johnston 183f23f10d Delete old pushers 2016-05-03 14:22:33 +01:00
Matthew Hodgson 792def4928 add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space 2016-05-01 12:44:24 +01:00
David Baker 2df75de505 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 20:28:47 +01:00
David Baker b084e4d963 Add constant for throttle multiplier 2016-04-29 20:14:55 +01:00
David Baker 35b7b8e4bc Remove unused function 2016-04-29 20:10:34 +01:00
David Baker 6b9b6a9169 Remove unused arg 2016-04-29 20:02:52 +01:00
David Baker c7c75e87dc Docstring 2016-04-29 19:47:35 +01:00
David Baker b0a1036d93 Use explicit join 2016-04-29 19:28:56 +01:00
David Baker 8f99cd5996 Oops, actually specify the user id 2016-04-29 19:27:03 +01:00
David Baker 60f44c098d Remove unnecessary if 2016-04-29 19:17:10 +01:00
David Baker 50ad8005e4 Put spaces at start of line 2016-04-29 19:16:15 +01:00
David Baker 83618d719a Try imports in config 2016-04-29 19:13:52 +01:00
David Baker e7a76b5123 Use the constant 2016-04-29 19:10:45 +01:00
David Baker 29c8cf8db8 Avoid `vars` builtin 2016-04-29 19:09:28 +01:00
David Baker d3da5294e8 Use named parameter format 2016-04-29 19:04:40 +01:00
David Baker 765f2b8446 Default enable email notifs to False 2016-04-29 14:46:18 +01:00
David Baker 311b5ce051 pep8 2016-04-29 14:37:30 +01:00
David Baker 3facde2536 Remove rather pointless get function 2016-04-29 14:36:45 +01:00
David Baker 4364ea1272 Stop processing notifs once we've sent a mail 2016-04-29 14:31:27 +01:00
David Baker 4b0c3a3270 Correct public_baseurl default 2016-04-29 14:30:15 +01:00
David Baker 5048455965 Nicer get() shorthand 2016-04-29 14:27:40 +01:00
David Baker 6c8957be7f Remove redundant docstring 2016-04-29 14:25:28 +01:00
David Baker 18ce88bd2d Correct default template and add text template 2016-04-29 14:24:25 +01:00
David Baker 40d40e470d Send mail notifs with a plaintext part too 2016-04-29 13:56:21 +01:00
David Baker dc2c527ce9 Fix password reset
Default requester to None, otherwise it isn't defined when resetting using email auth
2016-04-29 12:07:54 +01:00
Erik Johnston 62b51b8452 Fix typo in event_auth servlet path 2016-04-29 12:00:51 +01:00
David Baker b2c04da8dc Add an email pusher for new users
If they registered with an email address and email notifs are enabled on the HS
2016-04-29 11:43:57 +01:00
David Baker ec9cbe847d pep8 newline 2016-04-29 10:07:30 +01:00
David Baker acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
David Baker e800ee2f63 May as well always include room link 2016-04-28 17:28:27 +01:00
David Baker cc0874cf71 Put back real delay before mailing 2016-04-28 17:00:40 +01:00
David Baker 68f8fc2f14 Support file messages & fix plain text 2016-04-28 16:59:57 +01:00
David Baker 4845c7359d Support image notifs 2016-04-28 15:55:53 +01:00
Mark Haines 351b50a887 Fix more typos in per-request metrics 2016-04-28 15:29:46 +01:00
David Baker 60f86fc876 pep8 2016-04-28 15:16:30 +01:00
David Baker 937c407eef Only import email pusher if email notifs are on 2016-04-28 15:12:14 +01:00
Mark Haines dcfc10b129 Fix typo in request metrics 2016-04-28 15:11:06 +01:00
Matthew Hodgson aebd0c9717 fix typo 2016-04-28 15:09:25 +01:00
Mark Haines 1a12766e3b Add a comment explaining why automatic metric reporting is disabled for JsonResource 2016-04-28 12:31:26 +01:00
Mark Haines 6037349512 Check if report_metrics is True 2016-04-28 12:26:07 +01:00
David Baker ebbabc4986 Handle room invites in email notifs 2016-04-28 11:49:36 +01:00
Mark Haines 8d7ad44331 Report per request metrics for all of the things using request_handler 2016-04-28 10:57:49 +01:00
David Baker 9dba1b668c Linkify plain text messages too 2016-04-28 10:55:08 +01:00
David Baker 424a7f48f8 Run filter_events_for_client
so we don't accidentally mail out events people shouldn't see
2016-04-27 17:50:49 +01:00
David Baker 4ed1e45869 Make html messages work 2016-04-27 17:18:51 +01:00
Mark Haines 21d188bf95 Merge pull request #755 from matrix-org/markjh/right_direction
Fix backfill replication to advance the stream correctly
2016-04-27 15:54:48 +01:00
Mark Haines 8a65666454 Fix backfill replication to advance the stream correctly 2016-04-27 15:38:43 +01:00
David Baker 8781083960 Better grammar for multiple messages in a room
Say who the messages are from if there's no room name, otherwise it's a bit nonsensical
2016-04-27 15:30:41 +01:00
David Baker fa12209c1b Hopefully all remaining bits for email notifs
Add public facing base url to the server so synapse knows what URL to use when converting mxc to http urls for use in emails
2016-04-27 15:09:55 +01:00
Mark Haines 871357d539 Check that somethign has happend before running the selects 2016-04-27 11:54:13 +01:00
Mark Haines 71df327190 Actually start the pusher daemon 2016-04-26 17:07:09 +01:00
Mark Haines c9eab73f2a Fix typo in default pusher config 2016-04-26 17:06:18 +01:00
Mark Haines b80b93ea0f Add a log context to the daemonized pusher 2016-04-26 15:57:28 +01:00
Mark Haines 6df5a6a833 Optionally daemonize the pusher 2016-04-26 15:37:41 +01:00
Erik Johnston 5164ccc3e5 Bump changelog and version 2016-04-26 11:26:32 +01:00
Erik Johnston 3306cf45ca Merge pull request #750 from matrix-org/erikj/jwt_optional
Make pyjwt dependency optional
2016-04-26 11:07:22 +01:00
Mark Haines 9c417c54d4 Add a couple of update methods to the PusherSlaveStore 2016-04-26 10:45:02 +01:00
David Baker 7b4715bad7 More variable calculation for email notifs
Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.
2016-04-25 18:27:04 +01:00
Mark Haines f15e9e8de4 Remove the uncomments from the comments 2016-04-25 17:56:24 +01:00
Mark Haines 72e2fafa20 Add a metrics listener and a ssh listener to the pusher 2016-04-25 17:34:25 +01:00
Mark Haines f22f46f4f9 Move the listenTCP call outside the manhole function 2016-04-25 14:59:21 +01:00
David Baker 290f125a13 Typo 2016-04-25 14:42:59 +01:00
Erik Johnston 52ecbc2843 Make pyjwt dependency optional 2016-04-25 14:30:15 +01:00
David Baker 05e49ffbdf No we don't: it's just the display name 2016-04-22 18:44:17 +01:00
David Baker bd0f9c2065 Actually do UTF8 correctly 2016-04-22 18:42:00 +01:00
David Baker c5b3c6e101 Sort member events
So names of people in a room are given in order
2016-04-22 18:33:36 +01:00
David Baker 83bf65297a Mime part is binary so encode it first.
Doesn't get character enocind right yet but makes it not error.
2016-04-22 18:31:47 +01:00
David Baker e8701e64b9 Implement group-of-people names 2016-04-22 17:28:42 +01:00
David Baker c553797c4f No inlineCallbacks necessary on this 2016-04-22 17:27:54 +01:00
Mark Haines 5905f36f05 Split out setting up the manhole to a separate file 2016-04-22 17:09:15 +01:00
Mark Haines 62607d5452 Merge branch 'develop' into markjh/split_out_site.py
Conflicts:
	synapse/app/homeserver.py
2016-04-22 16:26:57 +01:00
Mark Haines e856036f4c Move SynapseSite to its own file 2016-04-22 16:09:55 +01:00
Mark Haines 9e7aa98c22 Split out create_resource_tree to a separate file 2016-04-22 15:40:51 +01:00
Mark Haines 2022ae0fb9 Merge pull request #746 from matrix-org/markjh/split_out_pusher
Optionally split out the pushers into a separate process
2016-04-22 11:34:08 +01:00
Erik Johnston 64ec3493c1 Merge pull request #745 from matrix-org/erikj/search-index
Optimise event_search in postgres
2016-04-22 11:23:49 +01:00
Erik Johnston 183cacac90 Simplify query and handle finishing correctly 2016-04-22 10:01:57 +01:00
David Baker c10ed26c30 Flesh out email templating
Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.
2016-04-21 19:19:07 +01:00
Erik Johnston ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston 3ddbb1687c Fix query 2016-04-21 18:02:36 +01:00
Erik Johnston 8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston b57dcb4b51 Typo 2016-04-21 17:49:00 +01:00
Erik Johnston 26db18bc90 Need to do _background_update_progress_txn in actual transaction 2016-04-21 17:45:56 +01:00
Erik Johnston b9675ef6e6 Merge pull request #687 from nikriek/jwt-fix
Fix issues with JWT login
2016-04-21 17:42:25 +01:00
Erik Johnston e395eb1108 Update progress when creating index 2016-04-21 17:39:24 +01:00
Erik Johnston 3b0fa77f50 Fix SQL statement 2016-04-21 17:37:42 +01:00
Erik Johnston 129e403487 Create index must be on a conn 2016-04-21 17:35:51 +01:00
Mark Haines a3ac837599 Optionally split out the pushers into a separate process 2016-04-21 17:22:37 +01:00
Mark Haines 78741cf025 Merge pull request #743 from matrix-org/markjh/slave_pushers
Replicate the pushers
2016-04-21 17:21:29 +01:00
Erik Johnston 51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00
Mark Haines 31719ad124 Merge pull request #744 from matrix-org/markjh/replication_remove_pusher
Add a replication endpoint for deleting pushers
2016-04-21 17:10:49 +01:00
Niklas Riekenbrauck 565c2edb0a Fix issues with JWT login 2016-04-21 18:10:48 +02:00
Erik Johnston c877f0f034 Optimise event_search in postgres 2016-04-21 16:56:14 +01:00
Mark Haines cfe1ff4bdb Add a replication endpoint for deleting pushers 2016-04-21 16:33:05 +01:00
Mark Haines d4823efad9 Replicate the pushers 2016-04-21 16:18:00 +01:00
Mark Haines c0d8e0eb63 Replicate push actions 2016-04-21 15:25:58 +01:00
David Baker 2ed0adb075 Generate mails from a template 2016-04-20 18:35:29 +01:00
David Baker 05adc6c2de more pep8 2016-04-20 13:02:45 +01:00
David Baker f63bd4ff47 Send a rather basic email notif
Also pep8 fixes
2016-04-20 13:02:01 +01:00
Erik Johnston 5bbc321588 Always use state cache entry if it exists
Also check if the resolved state matches an existing state group.
2016-04-20 11:49:10 +01:00
Erik Johnston 4cf4320593 Add some logging to state resolve_events 2016-04-20 11:06:02 +01:00
Erik Johnston eab47ea1e5 Merge pull request #739 from matrix-org/erikj/cache_get_state_groups_for_groups
Add cache to _get_state_groups_from_groups
2016-04-19 17:37:19 +01:00
Mark Haines f52dd35ac3 Merge pull request #738 from matrix-org/markjh/slaved_receipts
Add a slaved receipts store
2016-04-19 17:31:59 +01:00
Erik Johnston 61c7edfd34 Add cache to _get_state_groups_from_groups 2016-04-19 17:22:03 +01:00
Mark Haines 5bbd424ee0 Add a slaved receipts store 2016-04-19 17:14:08 +01:00
Erik Johnston 6ac40f7b65 Merge pull request #737 from matrix-org/erikj/spider_ssl_factory
Use tls_server_context_factory for SpiderEndpoint
2016-04-19 16:22:05 +01:00
Erik Johnston f505575f69 Make InsecureInterceptableContextFactory work with SpiderEndpoint 2016-04-19 16:08:14 +01:00
Mark Haines e99365f601 Replicate get_invited_rooms_for_user 2016-04-19 15:22:14 +01:00
David Baker e2a01455af Add single instance & logging stuff
Copy the stuff over from http pusher that prevents multiple instances of process running at once and sets up logging and measure blocks.
2016-04-19 14:52:58 +01:00
Erik Johnston e8884e5e9c Add self.media_repo to PreviewUrlResource 2016-04-19 14:51:34 +01:00
Erik Johnston a7001c311b _make_dirs was moved to MediaRepository 2016-04-19 14:49:31 +01:00
Erik Johnston 9181e2f4c7 Add store to PreviewUrlResource 2016-04-19 14:48:24 +01:00
Erik Johnston fb76a81ff7 Reorder imports 2016-04-19 14:45:05 +01:00
David Baker 07d765209d First bits of emailpusher
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
Erik Johnston 0c93df89b6 Move MediaRepository to media_repository module 2016-04-19 11:31:43 +01:00
Erik Johnston 43f0941e8f Split out BaseMediaResource into MediaRepository
This is so that a single MediaRepository can be shared across all
resources, rather than having a "copy" per resource.

In particular this allows us to guard against both the thumbnail and
download resource triggering a download of remote content at the same
time.
2016-04-19 11:24:59 +01:00
Erik Johnston eb8619e256 Create log context in Measure if one doesn't exist 2016-04-18 16:08:32 +01:00
Erik Johnston 4ef7a25c10 Merge pull request #733 from matrix-org/erikj/make_member_timeout
Lower timeout for make_membership_event
2016-04-18 15:08:05 +01:00
Erik Johnston 3727a15764 Merge pull request #732 from matrix-org/erikj/login
Simplify _check_password
2016-04-18 15:07:57 +01:00
Matthew Hodgson aaabbd3e9e explicitly pass in the charset from Content-Type to lxml to fix cyrillic woes better 2016-04-15 14:32:25 +01:00
Matthew Hodgson 84f9cac4d0 fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, rather than relying on lxml's heuristics which seem to get it wrong 2016-04-15 13:20:08 +01:00
Erik Johnston 914f1eafac Lower timeout for make_membership_event
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
2016-04-15 11:22:23 +01:00
Erik Johnston 6fd2f685fe Simplify _check_password 2016-04-15 11:17:18 +01:00
Erik Johnston cb9c465707 Use SynapseError 504 for Timeout errors 2016-04-15 10:21:32 +01:00
Mark Haines 3c79bdd7a0 Fix check_password rather than inverting the meaning of _check_local_password (#730) 2016-04-14 19:00:21 +01:00
David Baker a4c56bf67b Merge pull request #729 from matrix-org/dbkr/fix_login_nonexistent_user
Fix login to error for nonexistent users
2016-04-14 18:46:45 +01:00
David Baker 4c1b32d7e2 Fix login to error for nonexistent users
Fixes SYN-680
2016-04-14 18:28:42 +01:00
Matthew Hodgson f78b479118 fix urlparse import thinko breaking tiny URLs 2016-04-14 15:23:55 +01:00
Kegan Dougal 83776d6219 Make v2_alpha reg follow the AS API specification
The spec is clear the key should be 'user' not 'username' and this is indeed
the case for v1. This is not true for v2_alpha though, which is what this
commit is fixing.
2016-04-14 14:52:26 +01:00
Matthew Hodgson bd77216d06 comment out 2c838f6459 due to risk of https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve 2016-04-14 14:39:24 +01:00
Erik Johnston 9ae64c9910 Measure push action generator 2016-04-14 13:42:22 +01:00
Erik Johnston b42ad359e9 Merge pull request #725 from matrix-org/dbkr/push_only_joined
Don't push for everyone who ever sent an RR to the room
2016-04-14 12:05:13 +01:00
David Baker 757e2c79b4 Don't push for everyone who ever sent an RR to the room 2016-04-14 12:02:50 +01:00
Erik Johnston 86e9bbc74e Add missing yield 2016-04-14 11:56:52 +01:00
Erik Johnston e40f25ebe1 Rename log context 2016-04-14 11:54:14 +01:00
Erik Johnston 2ae91a9e2f Make send_badge private 2016-04-14 11:37:50 +01:00
Erik Johnston d213d69fe3 Add desc arg 2016-04-14 11:36:23 +01:00
Erik Johnston 56da835eaf Add necessary logging contexts 2016-04-14 11:33:50 +01:00
Erik Johnston 96bcfb29c7 Add index 2016-04-14 11:26:33 +01:00
Erik Johnston 7be1065b8f Add extra Measure 2016-04-14 11:26:15 +01:00
Erik Johnston a2546b9082 Fix query for get_unread_push_actions_for_user_in_range 2016-04-14 11:08:31 +01:00
Erik Johnston ceeb5b909f Merge pull request #721 from matrix-org/erikj/spider
Sanitize the optional dependencies for spider API
2016-04-14 09:59:29 +01:00
David Baker 43a89cca8e Merge pull request #722 from matrix-org/dbkr/only_unread_event_actions
Only return unread notifications
2016-04-13 14:54:26 +01:00
Erik Johnston f338bf9257 Give install requirements 2016-04-13 14:33:48 +01:00
David Baker 767fc0b739 pep8 2016-04-13 14:23:27 +01:00
David Baker 54d08c8868 Only return unread notifications
Make get_unread_push_actions_for_user_in_range only return unread event actions, being more true to its name. Done in two separate sql queries to get actions after a read receipt and those in a room wiht no receipt at all. SQL queries by Erik.
2016-04-13 14:16:45 +01:00
Erik Johnston 5880bc5417 Merge pull request #718 from matrix-org/erikj/public_room_list
Don't return empty public rooms
2016-04-13 14:07:26 +01:00
Erik Johnston f613a3e332 Merge pull request #720 from matrix-org/erikj/auth_chec
Don't auto log failed auth checks
2016-04-13 14:07:23 +01:00
Erik Johnston bfe586843f Add back in helpful description for missing url_preview_ip_range_blacklist 2016-04-13 13:52:57 +01:00
Erik Johnston d0633e6dbe Sanitize the optional dependencies for spider API 2016-04-13 13:38:09 +01:00
Erik Johnston 0f2ca8cde1 Measure Auth.check 2016-04-13 11:15:59 +01:00
Erik Johnston c53f9d561e Don't auto log failed auth checks 2016-04-13 11:11:46 +01:00
David Baker 65141161f6 Unused member variable 2016-04-12 16:25:26 +01:00
Erik Johnston 72f454b752 Don't return empty public rooms 2016-04-12 16:06:18 +01:00
David Baker d33d623f0d Merge pull request #716 from matrix-org/dbkr/get_pushers
Add get endpoint for pushers
2016-04-12 14:40:37 +01:00
David Baker 7984ffdc6a Unneccessarywhitespaceisunnecessary 2016-04-12 13:55:57 +01:00
David Baker c1267d04c5 Oops, forgot the desc. 2016-04-12 13:55:32 +01:00
David Baker a04c076b7f Make the /set part mandatory 2016-04-12 13:54:41 +01:00
David Baker 44891b4a0a Tidy up get_pusher functions
Decodes pushers rows on the main thread rather than the db thread and uses _simple_select_list. Also do the same to the function I copied and factor out the duplication into a helper function.
2016-04-12 13:47:17 +01:00
David Baker 7b39bcdaae Mis-named function 2016-04-12 13:35:08 +01:00
David Baker d937f342bb Split into separate servlet classes 2016-04-12 13:33:30 +01:00
Erik Johnston c48465dbaa More comments 2016-04-12 12:48:30 +01:00
Erik Johnston 8be1a37909 More comments 2016-04-12 12:04:19 +01:00
Erik Johnston d3d0be4167 Don't append to unused list 2016-04-12 11:59:00 +01:00
Erik Johnston 762ada1e07 Add back backfilled parameter that was removed 2016-04-12 11:58:04 +01:00
Erik Johnston 0d3da210f0 Add comment 2016-04-12 11:54:41 +01:00
Erik Johnston cccf86dd05 Check if we've already backfilled events 2016-04-12 11:19:32 +01:00
David Baker 8a76094965 Add get endpoint for pushers
As per https://github.com/matrix-org/matrix-doc/pull/308
2016-04-11 18:00:03 +01:00
Mark Haines 790f5848b2 Fix the rule_id for .m.rule.invite_for_me (#715) 2016-04-11 16:10:39 +01:00
Mark Haines 82d7eea7e3 Move the versionstring code out of app.homeserver into util 2016-04-11 14:57:09 +01:00
David Baker 2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
David Baker 9bb041791c Run unsafe proces in a loop until we've caught up
and wrap unsafe process in a try block
2016-04-11 12:48:30 +01:00
Erik Johnston 17515bae14 PEP8 2016-04-11 11:02:50 +01:00
Matthew Hodgson 4bd3d25218 Merge pull request #688 from matrix-org/matthew/preview_urls
URL previewing support
2016-04-11 10:40:29 +01:00
Matthew Hodgson 5ffacc5e84 fix typos and needless try/except from PR review 2016-04-11 10:39:16 +01:00
Matthew Hodgson 83b2f83da0 actually throw meaningful errors 2016-04-08 21:36:59 +01:00
Mark Haines b36270b5e1 Fix pep8 warning 2016-04-08 19:52:23 +01:00
Matthew Hodgson 6ff7a79308 move local_media_repository_url_cache.sql to schema v31 2016-04-08 19:09:02 +01:00
Matthew Hodgson af582b66bb fix typo 2016-04-08 19:08:47 +01:00
Matthew Hodgson 2460d904bd fix error checking for new SQL 2016-04-08 19:04:29 +01:00
Matthew Hodgson 1ccabe2965 more PR feedback 2016-04-08 18:58:08 +01:00
Matthew Hodgson fb83f6a1fc fix SQL based on PR feedback 2016-04-08 18:55:38 +01:00
Matthew Hodgson dafef5a688 Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off.
Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
2016-04-08 18:37:15 +01:00
David Baker d96a070a3a Actually check if we;re processing 2016-04-08 16:49:39 +01:00
David Baker ed3979df5f Fix invite pushes
* If the event is an invite event, add the invitee to list of user we run push rules for (if they have a pusher etc)
 * Move invite_for_me to be higher prio than member events otherwise member events matches them
 * Spell override right
2016-04-08 15:29:59 +01:00
Erik Johnston 79fc4ff6f9 Merge pull request #677 from matrix-org/erikj/dns_cache
Read from DNS cache if within TTL
2016-04-08 14:09:56 +01:00
David Baker 7b6d519482 Make sure max stream ordering only increases 2016-04-08 14:08:16 +01:00
David Baker 52d1008661 Unsafe process should call itself if the max has changed 2016-04-08 14:06:54 +01:00
Erik Johnston 96bd8ff57c Merge pull request #707 from matrix-org/markjh/remove_changed_presencelike_data
changed_presencelike_data isn't observed anywhere so can be removed
2016-04-08 14:04:54 +01:00
David Baker ce3fe52498 Comment why unsafe process is unsafe 2016-04-08 14:02:38 +01:00
Mark Haines 7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
Mark Haines b9ee5650b0 Move all the wrapper functions for distributor.fire
Move the functions inside the distributor and import them
where needed. This reduces duplication and makes it possible
for flake8 to detect when the functions aren't used in a
given file.
2016-04-08 11:01:38 +01:00
Mark Haines caef337587 changed_presencelike_data isn't observed anywhere in synapse so can be removed 2016-04-08 10:37:19 +01:00
Mark Haines b4a5002a6e Merge pull request #708 from matrix-org/markjh/remove_collect_presencelike_data
Call profile handler get_displayname directly
2016-04-08 09:51:36 +01:00
Mark Haines 86be915cce Call profile handler get_displayname directly rather than using collect_presencelike_data 2016-04-07 18:11:49 +01:00
David Baker d9f38561c8 Literally a dictionary 2016-04-07 17:45:01 +01:00
David Baker 4836864f56 generate id in the main thread 2016-04-07 17:38:48 +01:00
David Baker a4a31fa8dc Only pass in what we need 2016-04-07 17:37:19 +01:00
Erik Johnston f942980c0b Merge pull request #701 from DoubleMalt/ldap-auth
Add LDAP authentication
2016-04-07 17:35:28 +01:00
David Baker 3fb35cbd6f Oops, inequality fail 2016-04-07 17:33:37 +01:00
David Baker 15e0f1696f Wrap process in a flag so we don't process whist already processing. 2016-04-07 17:31:08 +01:00
Matthew Hodgson d6e7333ae4 Merge branch 'develop' into matthew/preview_urls 2016-04-07 17:26:44 +01:00
David Baker 6ec02e9ecf indenting 2016-04-07 17:24:05 +01:00
David Baker 25cd5bb697 defer.gatherResults rather than doing all the pokes in series 2016-04-07 17:22:14 +01:00
David Baker fa129ce5b5 Add measure blocks 2016-04-07 17:12:29 +01:00
David Baker e1e042f2a1 Add comments on min_stream_id
saying that the min stream id won't be completely accurate all the time
2016-04-07 17:09:36 +01:00
Mark Haines ceb599e789 Add tests for redactions 2016-04-07 16:52:07 +01:00
Mark Haines 8c82b06904 Merge pull request #704 from matrix-org/markh/slaveIII
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 16:49:34 +01:00
David Baker 05d044aac3 pep8 2016-04-07 16:45:38 +01:00
David Baker 2d5c693fd3 Fix port script for changes merged from develop 2016-04-07 16:43:54 +01:00
Mark Haines 57fa1801c3 Add sensible __eq__ operators inside the tests.
Rather than adding them globally. This limits the changes to only
affect the tests.
2016-04-07 16:41:37 +01:00
Erik Johnston a294b04bf0 Merge pull request #700 from matrix-org/erikj/deduplicate_joins
Deduplicate membership changes
2016-04-07 16:35:40 +01:00
David Baker 9c99ab4572 Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions 2016-04-07 16:35:22 +01:00
David Baker d549fdfa22 Remove code that's now been obsoleted or moved elsewhere 2016-04-07 16:31:38 +01:00
Erik Johnston 95ac3078da Rename things 2016-04-07 16:07:16 +01:00
David Baker 92e3071623 Send badge count pushes.
Also fix bugs with retrying.
2016-04-07 15:39:53 +01:00
Erik Johnston ee5aef6c72 Log contexts and squash things together 2016-04-07 15:34:21 +01:00
Erik Johnston 639cd07d6d Add comment 2016-04-07 14:24:12 +01:00
Erik Johnston af03ecf352 Deduplicate joins 2016-04-07 14:19:02 +01:00
Mark Haines 60ec9793fb Add tests for get_latest_event_ids_in_room and get_current_state 2016-04-07 13:17:56 +01:00
Erik Johnston a28d066732 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache 2016-04-07 11:11:17 +01:00
Erik Johnston 1ef0365670 Set profile information when joining rooms remotely 2016-04-07 09:42:52 +01:00
Richard van der Hoff 87a30890a3 Merge pull request #699 from matrix-org/rav/show_own_leave_event
Let users see their own leave events
2016-04-06 17:57:06 +01:00
Christoph Witzany ed4d18f516 fix check for failed authentication 2016-04-06 18:30:11 +02:00
Christoph Witzany 9c62fcdb68 remove line 2016-04-06 18:23:46 +02:00
Christoph Witzany 27a0c21c38 make tests for ldap more specific to not be fooled by Mocks 2016-04-06 18:23:46 +02:00
Christoph Witzany 3555a659ec output ldap version for info and to pacify pep8 2016-04-06 18:23:46 +02:00
Christoph Witzany 4c5e8adf8b conditionally import ldap 2016-04-06 18:23:46 +02:00
Christoph Witzany 875ed05bdc fix pep8 2016-04-06 18:23:46 +02:00
Christoph Witzany 67f3a50e9a fix exception handling 2016-04-06 18:23:46 +02:00
Christoph Witzany afff321e9a code style 2016-04-06 18:23:46 +02:00
Christoph Witzany 8f0e47fae8 cleanup 2016-04-06 18:23:45 +02:00
Christoph Witzany 823b8be4b7 add tls property and twist my head around twisted 2016-04-06 18:23:45 +02:00
Christoph Witzany 92767dd703 add tls property 2016-04-06 18:23:45 +02:00
Christoph Witzany 7b9319b1c8 move LDAP authentication to AuthenticationHandler 2016-04-06 18:23:45 +02:00
Christoph Witzany 3d95405e5f Introduce LDAP authentication 2016-04-06 18:23:45 +02:00
David Baker 0fd1cd2400 pep8 2016-04-06 16:50:47 +01:00
Mark Haines 6bfec56796 Test that room membership is replicated 2016-04-06 16:20:13 +01:00
Mark Haines e815763b7f Merge pull request #697 from matrix-org/markjh/slaveI
Add a slaved events store class
2016-04-06 16:19:25 +01:00
David Baker 7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
* Make badges work again
 * Remove old, unused code
2016-04-06 15:42:15 +01:00
Richard van der Hoff 1e05637e37 Let users see their own leave events
... otherwise clients get confused.

Fixes https://matrix.org/jira/browse/SYN-662,
https://github.com/vector-im/vector-web/issues/368
2016-04-06 15:36:19 +01:00
Erik Johnston b713934b2e Merge pull request #698 from matrix-org/erikj/port_script_fix
Don't require config to create database
2016-04-06 14:32:45 +01:00
Mark Haines 75fb9ac1be Add a slaved events store class
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
Erik Johnston 8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Mark Haines 7d11f825aa Merge pull request #694 from matrix-org/markjh/caches
Move _get_cache_dict into the SQLBaseStore
2016-04-06 13:21:25 +01:00
Mark Haines 87f2dec8d4 Make the cache objects be per instance rather than being global 2016-04-06 13:08:05 +01:00
Mark Haines a1e0d316ea Move _get_cache_dict into the SQLBaseStore 2016-04-06 13:05:19 +01:00
Mark Haines 2e308a3a38 Merge pull request #692 from matrix-org/markjh/replicate_reshuffle
Separate generating the replication response...
2016-04-05 13:23:36 +01:00
Erik Johnston c2b429ab24 Merge pull request #693 from matrix-org/erikj/backfill_self
Don't backfill from self
2016-04-05 13:04:36 +01:00
Erik Johnston 6222ae51ce Don't backfill from self 2016-04-05 12:56:29 +01:00