d6af5512bb
Merge pull request #2809 from matrix-org/rav/metrics_errors
...
better exception logging in callbackmetrics
2018-01-18 11:46:37 +00:00
ce236f8ac8
better exception logging in callbackmetrics
...
when we fail to render a metric, give a clue as to which metric it was
2018-01-18 11:30:49 +00:00
dc519602ac
Ensure we registerProducer isn't called twice
2018-01-18 11:07:17 +00:00
17b54389fe
Fix _notify_empty typo
2018-01-18 11:05:34 +00:00
28b338ed9b
Move definition of paused_producer to __init__
2018-01-18 11:04:41 +00:00
a177325b49
Fix comments
2018-01-18 11:02:43 +00:00
36da256cc6
Merge pull request #2805 from matrix-org/rav/log_state_res
...
Log room when doing state resolution
2018-01-17 18:05:04 +00:00
1224612a79
Log room when doing state resolution
...
Mostly because it helps figure out what is prompting the resolution
2018-01-17 17:11:59 +00:00
bc67e7d260
Add decent impl of a FileConsumer
...
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.
Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
2018-01-17 16:43:03 +00:00
a87006f9c7
Merge pull request #2783 from matrix-org/erikj/media_last_accessed
...
Keep track of last access time for local media
2018-01-17 16:39:02 +00:00
06db5c4b76
Merge pull request #2803 from matrix-org/matthew/fix-userdir-sql
...
fix SQL when searching all users
2018-01-17 16:27:54 +00:00
2d9ab533f9
fix SQL when searching all users
2018-01-17 15:58:52 +00:00
390093d45e
Split resolve_events into two functions
...
... so that the return type doesn't depend on the arg types
2018-01-17 15:44:31 +00:00
2fb3a28c98
Remove lost comment
2018-01-17 14:59:44 +00:00
a7e4ff9cca
Merge pull request #2795 from matrix-org/rav/track_db_scheduling
...
Track DB scheduling delay per-request
2018-01-17 14:29:37 +00:00
f884cfffb9
Merge pull request #2797 from matrix-org/rav/user_id_checking
...
Sanity checking for user ids
2018-01-17 14:29:12 +00:00
a5213df1f7
Sanity checking for user ids
...
Check the user_id passed to a couple of APIs for validity, to avoid
"IndexError: list index out of range" exception which looks scary and results
in a 500 rather than a more useful error.
Fixes #1432 , among other things
2018-01-17 14:28:54 +00:00
e8f7541d3f
Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling
2018-01-17 14:01:57 +00:00
fb6563b4be
Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis
...
Track db txn time in millisecs
2018-01-17 13:52:42 +00:00
1954e867b4
Merge pull request #2796 from matrix-org/rav/fix_closed_connection_errors
...
Fix 'NoneType' object has no attribute 'writeHeaders'
2018-01-17 13:52:24 +00:00
11ab2f56f5
Merge branch 'develop' into rav/fix_closed_connection_errors
2018-01-17 11:25:11 +00:00
0486a7814a
Merge branch 'develop' into rav/rework_run_interaction
2018-01-17 11:24:38 +00:00
90c14da992
Merge branch 'develop' into rav/db_txn_time_in_millis
2018-01-17 11:19:55 +00:00
38506773eb
Merge remote-tracking branch 'origin/develop' into rav/optimise_logging_context
2018-01-17 10:57:13 +00:00
300edc2348
Update last access time when thumbnails are viewed
2018-01-17 10:24:43 +00:00
05f98a2224
Keep track of last access time for local media
2018-01-17 10:24:43 +00:00
3cb2dabaad
Merge pull request #2789 from matrix-org/erikj/fix_thumbnails
...
Fix thumbnailing remote files
2018-01-17 10:22:20 +00:00
d728c47142
Add docstring
2018-01-17 10:06:14 +00:00
936482d507
Fix 'NoneType' object has no attribute 'writeHeaders'
...
Avoid throwing a (harmless) exception when we try to write an error response to
an http request where the client has disconnected.
This comes up as a CRITICAL error in the logs which tends to mislead people
into thinking there's an actual problem
2018-01-16 17:58:16 +00:00
3d12d97415
Track DB scheduling delay per-request
...
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
0f5d2cc37c
Merge branch 'rav/db_txn_time_in_millis' into rav/track_db_scheduling
2018-01-16 17:09:15 +00:00
8615f19d20
rework runInteraction in terms of runConnection
...
... so that we can share the code
2018-01-16 17:08:29 +00:00
5e97ca7ee6
fix typo
2018-01-16 16:52:35 +00:00
d863f68cab
Use local vars
2018-01-16 16:24:15 +00:00
6368e5c0ab
Change _generate_thumbnails to take media_type
2018-01-16 16:17:38 +00:00
0a90d9ede4
Move setting of file_id up to caller
2018-01-16 16:03:05 +00:00
6324b65f08
Track db txn time in millisecs
...
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
44a498418c
Optimise LoggingContext creation and copying
...
It turns out that the only thing we use the __dict__ of LoggingContext for is
`request`, and given we create lots of LoggingContexts and then copy them every
time we do a db transaction or log line, using the __dict__ seems a bit
redundant. Let's try to optimise things by making the request attribute
explicit.
2018-01-16 15:49:42 +00:00
5dfc83704b
Fix typo
2018-01-16 14:32:56 +00:00
febdca4b37
Merge pull request #2785 from matrix-org/rav/reorganise_metrics_again
...
Reorganise request and block metrics
2018-01-16 14:10:21 +00:00
f5f89fda21
Merge pull request #2790 from matrix-org/rav/preserve_event_logcontext_leak
...
Fix a logcontext leak in persist_events
2018-01-16 14:09:13 +00:00
307f88dfb6
Fix up log lines
2018-01-16 13:53:52 +00:00
807e848f0f
Merge pull request #2787 from matrix-org/rav/worker_event_counts
...
Metrics for events processed in appservice and fed sender
2018-01-16 13:14:09 +00:00
4a31a61ef9
Merge pull request #2786 from matrix-org/rav/rdata_metrics
...
Metrics for number of RDATA commands received
2018-01-16 13:13:53 +00:00
9795b9ebb1
Correctly use server_name/file_id when generating/fetching remote thumbnails
2018-01-16 12:02:06 +00:00
c5b589f2e8
Log when we respond with 404
2018-01-16 12:01:40 +00:00
64ddec1bc0
Fix a logcontext leak in persist_events
...
ObserveableDeferred expects its callbacks to be called without any
logcontexts, whereas it turns out we were calling them with the logcontext of
the request which initiated the persistence loop.
It seems wrong that we are attributing work done in the persistence loop to the
request that happened to initiate it, so let's solve this by dropping the
logcontext for it.
(I'm not sure this actually causes any real problems other than messages in the
debug log, but let's clean it up anyway)
2018-01-16 11:47:36 +00:00
a4c5e4a645
Fix thumbnailing remote files
2018-01-16 11:37:50 +00:00
1159abbdd2
Merge pull request #2767 from matrix-org/erikj/media_storage_refactor
...
Refactor MediaRepository to separate out storage
2018-01-16 10:23:50 +00:00
a027c2af8d
Metrics for events processed in appservice and fed sender
...
More metrics I wished I'd had
2018-01-15 18:23:24 +00:00
5c3c32f16f
Metrics for number of RDATA commands received
...
I found myself wishing we had this.
2018-01-15 17:45:55 +00:00
39f4e29d01
Reorganise request and block metrics
...
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.
The following are all deprecated, and replaced with synapse_util_metrics_block_count:
synapse_util_metrics_block_timer:count
synapse_util_metrics_block_ru_utime:count
synapse_util_metrics_block_ru_stime:count
synapse_util_metrics_block_db_txn_count:count
synapse_util_metrics_block_db_txn_duration:count
The following are all deprecated, and replaced with synapse_http_server_response_count:
synapse_http_server_requests
synapse_http_server_response_time:count
synapse_http_server_response_ru_utime:count
synapse_http_server_response_ru_stime:count
synapse_http_server_response_db_txn_count:count
synapse_http_server_response_db_txn_duration:count
The following are renamed (the old metrics are kept for now, but deprecated):
synapse_util_metrics_block_timer:total ->
synapse_util_metrics_block_time_seconds
synapse_util_metrics_block_ru_utime:total ->
synapse_util_metrics_block_ru_utime_seconds
synapse_util_metrics_block_ru_stime:total ->
synapse_util_metrics_block_ru_stime_seconds
synapse_util_metrics_block_db_txn_count:total ->
synapse_util_metrics_block_db_txn_count
synapse_util_metrics_block_db_txn_duration:total ->
synapse_util_metrics_block_db_txn_duration_seconds
synapse_http_server_response_time:total ->
synapse_http_server_response_time_seconds
synapse_http_server_response_ru_utime:total ->
synapse_http_server_response_ru_utime_seconds
synapse_http_server_response_ru_stime:total ->
synapse_http_server_response_ru_stime_seconds
synapse_http_server_response_db_txn_count:total ->
synapse_http_server_response_db_txn_count
synapse_http_server_response_db_txn_duration:total
synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
992018d1c0
mechanism to render metrics with alternative names
2018-01-15 17:04:39 +00:00
80fa610f9c
Add some comments to metrics classes
2018-01-15 16:52:52 +00:00
19d274085f
Make Counter render floats
...
Prometheus handles all metrics as floats, and sometimes we store non-integer
values in them (notably, durations in seconds), so let's render them as floats
too.
(Note that the standard client libraries also treat Counters as floats.)
2018-01-12 23:49:44 +00:00
21bf87a146
Reinstate media download on thumbnail request
...
We need to actually download the remote media when we get a request for a
thumbnail.
2018-01-12 15:38:06 +00:00
694f1c1b18
Fix up comments
2018-01-12 15:02:46 +00:00
e21370ba54
Correctly reraise exception
2018-01-12 14:44:02 +00:00
85a4d78213
Make Responder a context manager
2018-01-12 13:32:03 +00:00
dcc8eded41
Add missing class var
2018-01-12 13:16:27 +00:00
fefeb0ab0e
Merge pull request #2774 from matrix-org/erikj/synctl
...
When using synctl with workers, don't start the main synapse automatically
2018-01-12 12:00:07 +00:00
81391fa162
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/media_storage_refactor
2018-01-12 11:28:49 +00:00
1e4edd1717
Remove unnecessary condition
2018-01-12 11:28:32 +00:00
c6c009603c
Remove unused variables
2018-01-12 11:24:05 +00:00
4d88958cf6
Make class var local
2018-01-12 11:23:54 +00:00
227c491510
Comments
2018-01-12 11:22:41 +00:00
f4d93ae424
Actually make it work
2018-01-12 10:39:27 +00:00
f68e4cf690
Refactor
2018-01-12 10:11:12 +00:00
5f23b6d5ea
Merge pull request #2766 from matrix-org/rav/room_event
...
Add /room/{id}/event/{id} to synapse
2018-01-11 14:47:18 +00:00
7cd34512d8
When using synctl with workers, don't start the main synapse automatically
2018-01-11 11:37:39 +00:00
07ab948c38
Merge branch 'master' of github.com:matrix-org/synapse into develop
2018-01-11 11:23:40 +00:00
825a07a974
Merge pull request #2773 from matrix-org/erikj/hash_bg
...
Do bcrypt hashing in a background thread
2018-01-10 18:11:41 +00:00
f8e1ab5fee
Do bcrypt hashing in a background thread
2018-01-10 18:01:28 +00:00
5f07f5694c
fix order of operations derp and also use `.get` to default to {}
...
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-01-10 15:11:35 +00:00
8c9d5b4873
Fix publicised groups API (singular) over federation
...
which was missing its fed client API, since there is no other API
it might as well reuse the bulk one and unwrap it
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-01-10 15:04:51 +00:00
c175a5f0f2
Merge pull request #2770 from matrix-org/rav/fix_request_metrics
...
Update http request metrics before calling servlet
2018-01-10 14:53:40 +00:00
d90e8ea444
Update http request metrics before calling servlet
...
Make sure that we set the servlet name in the metrics object *before* calling
the servlet, in case the servlet throws an exception.
2018-01-09 18:27:35 +00:00
174eacc8ba
oops
2018-01-09 18:14:32 +00:00
a66f489678
fix GIST->GIN switch
2018-01-09 16:55:51 +00:00
e79db0a673
switch back from GIST to GIN indexes
2018-01-09 16:37:48 +00:00
e365ad329f
oops, tweak work_mem when actually storing
2018-01-09 16:30:30 +00:00
19f9227643
avoid 80s GIN inserts by tweaking work_mem
...
see https://github.com/matrix-org/synapse/issues/2753 for details
2018-01-09 16:25:04 +00:00
8f03aa9f61
Add StorageProvider concept
2018-01-09 16:16:12 +00:00
2442e9876c
Make PreviewUrlResource use MediaStorage
2018-01-09 16:15:07 +00:00
9d30a7691c
Make ThumbnailResource use MediaStorage
2018-01-09 16:15:07 +00:00
9e20840e02
Use MediaStorage for remote media
2018-01-09 16:15:07 +00:00
dd3092c3a3
Use MediaStorage for local files
2018-01-09 16:15:07 +00:00
ada470bccb
Add MediaStorage class
2018-01-09 16:15:07 +00:00
1ee787912b
Add some helper classes
2018-01-09 16:15:07 +00:00
47ca5eb882
Split out add_file_headers
2018-01-09 16:15:07 +00:00
b6c9deffda
Remove dead TODO
2018-01-09 15:53:23 +00:00
51c9d9ed65
Add /room/{id}/event/{id} to synapse
...
Turns out that there is a valid usecase for retrieving event by id (notably
having received a push), but event ids should be scoped to room, so /event/{id}
is wrong.
2018-01-09 14:39:12 +00:00
b30cd5b107
Remove dead code related to default thumbnails
2018-01-09 14:38:33 +00:00
cb66a2d387
Merge pull request #2763 from matrix-org/rav/fix_config_uts
...
Fix broken config UTs
2018-01-09 12:08:08 +00:00
0211464ba2
Fix broken config UTs
...
https://github.com/matrix-org/synapse/pull/2755 broke log-config generation,
which in turn broke the unit tests.
2018-01-09 11:28:33 +00:00
3a556f1ea0
Make indentation of generated log config consistent
...
(we had a mix of 2- and 4-space indents)
2018-01-09 11:27:19 +00:00
eccfc8e928
Fix templating error with unban permission message
...
Fixes https://github.com/matrix-org/synapse/issues/2759
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-01-07 19:52:58 -07:00
840f72356e
Remove 'verbosity'/'log_file' from generated cfg
...
... because these only really exist to confuse people nowadays.
Also bring log config more into line with the generated log config, by making `level_for_storage`
apply to the `synapse.storage.SQL` logger rather than `synapse.storage`.
2018-01-05 12:30:28 +00:00
864a6d2977
Bump version and changelog
2018-01-05 10:54:01 +00:00
6e375f4597
Merge pull request #2744 from matrix-org/rav/login_logging
...
Better logging when login can't find a 3pid
2018-01-05 09:54:39 +00:00
bd91857028
Check missing fields in event_from_pdu_json
...
Return a 400 rather than a 500 when somebody messes up their send_join
2017-12-30 18:40:19 +00:00
3079f80d4a
Factor out `event_from_pdu_json`
...
turns out we have two copies of this, and neither needs to be an instance
method
2017-12-30 18:40:19 +00:00
65abc90fb6
federation_server: clean up imports
2017-12-30 18:40:19 +00:00
a7b726ad18
federation_client: clean up imports
2017-12-30 18:40:19 +00:00
75c1b8df01
Better logging when login can't find a 3pid
2017-12-20 19:31:00 +00:00
3f9f1c50f3
Merge pull request #2683 from seckrv/fix_pwd_auth_prov_typo
...
synapse/config/password_auth_providers: Fixed bracket typo
2017-12-18 22:37:21 +00:00
df0f602796
Implement listen_tcp method in remaining workers
...
Signed-off-by: Silke <silke@slxh.eu>
2017-12-18 20:00:42 +01:00
26cd3f5690
Remove logger argument and do not catch replication listener
...
Signed-off-by: Silke <silke@slxh.eu>
2017-12-18 20:00:42 +01:00
ed48ecc58c
Add methods for listening on multiple addresses
...
Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP
and reactor.listenSSL for multiple addresses.
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2017-12-17 13:15:48 +01:00
37d1a90025
Allow binds to both :: and 0.0.0.0
...
Binding on 0.0.0.0 when :: is specified in the bind_addresses is now allowed.
This causes a warning explaining the behaviour.
Configuration changed to match.
See #2232
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2017-12-17 13:10:31 +01:00
3e59143ba8
Adapt the default config to bind on IPv6.
...
Most deployments are on Linux (or Mac OS), so this would actually bind
on both IPv4 and IPv6.
Resolves #1886 .
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
2017-12-17 13:07:37 +01:00
069ae2a5d6
Bump changelog and version
2017-12-13 15:11:22 +00:00
d8a6c734fa
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/createroom_content
2017-12-07 14:24:01 +00:00
ef045dcd71
Copy dict in update_membership too
2017-12-07 14:17:15 +00:00
33cb7ef0b7
Merge pull request #2723 from matrix-org/matthew/search-all-local-users
...
Add all local users to the user_directory and optionally search them
2017-12-05 11:09:47 +00:00
cdc2cb5d11
fix StoreError syntax
2017-12-05 11:09:31 +00:00
16ec3805e5
Fix error when deleting devices
...
This was introduced in d7ea8c4#2728
2017-12-05 09:49:22 +00:00
da1010c83a
support custom login types for validating users
...
Wire the custom login type support from password providers into the UI-auth
user-validation flows.
2017-12-05 09:43:30 +00:00
d7ea8c4800
Factor out a validate_user_via_ui_auth method
...
Collect together all the places that validate a logged-in user via UI auth.
2017-12-05 09:42:30 +00:00
d5f9fb06b0
Refactor UI auth implementation
...
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
2017-12-05 09:40:05 +00:00
c22e73293a
speed up the rate of initial spam for users
2017-12-04 18:05:28 +00:00
b11dca2025
better doc
2017-12-04 17:51:33 +00:00
74e0cc74ce
fix pep8 and tests
2017-12-04 15:11:38 +00:00
1bd40ca73e
switch to a simpler 'search_all_users' button as per review feedback
2017-12-04 14:58:39 +00:00
f397153dfc
Merge branch 'develop' into matthew/search-all-local-users
2017-11-30 01:51:38 +00:00
5406392f8b
specify default user_directory_include_pattern
2017-11-30 01:45:34 +00:00
f61e107f63
remove null constraint on user_dir.room_id
2017-11-30 01:43:50 +00:00
4b1fceb913
fix alternation operator for FTS4 - how did this ever work!?
2017-11-30 01:34:03 +00:00
a4bb133b68
fix thinkos galore
2017-11-30 01:17:15 +00:00
cd3697e8b7
kick the user_directory index when new users register
2017-11-29 18:33:34 +00:00
3241c7aac3
untested WIP but might actually work
2017-11-29 18:27:05 +00:00
624c46eb06
Merge pull request #2721 from matrix-org/rav/get_user_by_access_token_comments
...
Improve comments on get_user_by_access_token
2017-11-29 17:57:06 +00:00
47d99a20d5
Add user_directory_include_pattern config param to expand search results to additional users
...
Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive.
It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users...
...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too.
Committing it here as a WIP
2017-11-29 16:46:45 +00:00
ad7e570d07
Delete devices in various logout situations
...
Make sure that we delete devices whenever a user is logged out due to any of
the following situations:
* /logout
* /logout_all
* change password
* deactivate account (by the user or by an admin)
* invalidate access token from a dynamic module
Fixes #2672 .
2017-11-29 16:44:35 +00:00
ae31f8ce45
Move set_password into its own handler
...
Non-functional refactoring to move set_password. This means that we'll be able
to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
7ca5c68233
Move deactivate_account into its own handler
...
Non-functional refactoring to move deactivate_account. This means that we'll be
able to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
2c6d63922a
Remove pushers when deleting access tokens
...
Whenever an access token is invalidated, we should remove the associated
pushers.
2017-11-29 16:44:35 +00:00
97d1a1dc01
Merge pull request #2718 from matrix-org/rav/notify_logcontexts
...
Clear logcontext before starting fed txn queue runner
2017-11-29 16:01:46 +00:00
8b45de90a4
Merge pull request #2719 from matrix-org/rav/handle_missing_hashes
...
Fix 500 when joining matrix-dev
2017-11-29 16:01:33 +00:00
7303ed65e1
Fix 500 when joining matrix-dev
...
matrix-dev has an event (`$/6ANj/9QWQyd71N6DpRQPf+SDUu11+HVMeKSpMzBCwM:zemos.net`)
which has no `hashes` member.
Check for missing `hashes` element in events.
2017-11-29 16:00:46 +00:00
da562bd6a1
Improve comments on get_user_by_access_token
...
because I have to reverse-engineer this every time.
2017-11-29 15:52:41 +00:00
d4fb4f7c52
Clear logcontext before starting fed txn queue runner
...
These processes take a long time compared to the request, so there is lots of
"Entering|Restoring dead context" in the logs. Let's try to shut it up a bit.
2017-11-28 15:26:14 +00:00
dfbc45302e
PEP8
2017-11-28 15:23:26 +00:00
c4c1d170af
Fix wrong avatars when inviting multiple users when creating room
...
We reused the `content` dictionary between invite requests, which meant they could end up reusing the profile info for a previous user
2017-11-28 15:19:15 +00:00
ab1b2d0ff2
Allow guest access to group APIs for reading
2017-11-28 11:23:00 +00:00
5a4da5bf78
Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_error
...
Fix error on sqlite 3.7
2017-11-27 12:25:48 +00:00
84b31a3e7a
Merge pull request #2713 from matrix-org/rav/no_upsert_forever
...
Avoid retrying forever on IntegrityError
2017-11-27 12:19:35 +00:00
df6c72ede3
Merge pull request #2711 from matrix-org/rav/fix_dns_errhandler
...
Fix error handling on dns lookup
2017-11-27 12:19:18 +00:00
04bb79f139
Merge pull request #2710 from matrix-org/rav/remove_dead_code
...
Tiny code cleanups
2017-11-27 12:15:44 +00:00
e828a7380a
Merge pull request #2708 from matrix-org/rav/replication_logcontext_leaks
...
Fix some logcontext leaks in replication resource
2017-11-27 12:15:33 +00:00
7ef22a41a3
Merge pull request #2707 from matrix-org/rav/fix_urlpreview
...
Fix OPTIONS on preview_url
2017-11-27 12:15:14 +00:00
6be01f599b
Improve tracebacks on exceptions
...
Use failure.Failure to recover our failure, which will give us a useful
stacktrace, unlike the rethrown exception.
2017-11-27 12:05:58 +00:00
63ccaa5873
Avoid retrying forever on IntegrityError
2017-11-27 12:00:07 +00:00
8b38096a89
Fix error handling on dns lookup
...
pass the right arguments to the errback handler
Fixes "TypeError('eb() takes exactly 2 arguments (1 given)',)"
2017-11-24 16:47:48 +00:00
795b0849f3
Add a comment which might save some confusion
2017-11-24 00:34:56 +00:00
7f14f0ae38
Remove dead sync_callback
...
This is never used; let's remove it to stop confusing things.
2017-11-24 00:32:04 +00:00
0edf085b68
Fix some logcontext leaks in replication resource
...
The @measure_func annotations rely on the wrapped function respecting the
logcontext rules. Add the necessary yields to make this work.
2017-11-23 23:19:43 +00:00
8132a6b7ac
Fix OPTIONS on preview_url
...
Fixes #2706
2017-11-23 17:52:31 +00:00
6b48b3e277
fix sql fails
2017-11-22 18:06:24 +00:00
2908f955d1
Check database in has_completed_background_updates
...
so that the right thing happens on workers.
2017-11-22 18:02:15 +00:00
68ca864141
Add config option to disable media_repo on main synapse
...
... to stop us doing the cache cleanup jobs on the master.
2017-11-22 16:20:27 +00:00
e1fd4751de
Build MediaRepositoryResource as a homeserver dependency
...
This avoids the scenario where we have four different PreviewUrlResources
configured on a single app, each of which have their own caches and cache
clearing jobs.
2017-11-22 16:19:49 +00:00
d5a7c56ef9
Merge pull request #2698 from matrix-org/rav/remove_dead_dependencies
...
Clean up dependency list
2017-11-21 17:38:42 +00:00
0b4aa2dc21
Merge pull request #2689 from matrix-org/rav/unlock_account_data_upsert
...
Avoid locking account_data tables for upserts
2017-11-21 13:39:14 +00:00
7298ed7c51
Clean up dependency list
...
remove those that aren't used at all, and replace the ones that don't have
builders with simple getters rather than dynamically-generated methods.
2017-11-21 11:15:41 +00:00
7098b65cb8
Fix error on sqlite 3.7
...
Create the url_cache index on local_media_repository as a background update, so
that we can detect whether we are on sqlite or not and create a partial or
complete index accordingly.
To avoid running the cleanup job before we have built the index, add a bailout
which will defer the cleanup if the bg updates are still running.
Fixes https://github.com/matrix-org/synapse/issues/2572 .
2017-11-21 11:14:17 +00:00
83d8d4d8cd
Allow use of higher versions of saml2
...
The package was pinned to <4.0 with 07cf96eb
2017-11-20 11:14:39 +01:00
59a7275258
Merge pull request #2688 from matrix-org/rav/unlock_more_upsert
...
Avoid locking for upsert on pushers tables
2017-11-17 13:09:31 +00:00
d8a05418f9
Merge branch 'master' into develop
2017-11-17 12:13:14 +00:00
cdf6fc15b0
Merge pull request #2686 from matrix-org/luke/as-flair
...
Add automagical AS Publicised Group(s)
2017-11-17 10:13:46 +00:00
74bbeb4373
Bump version in __init__.py
2017-11-17 10:10:53 +00:00
eded7084d2
Fix auth handler #2678
2017-11-17 10:07:27 +00:00
34c3d0a386
typo
2017-11-17 01:54:02 +00:00
7fb0e98b03
Extract group_id from the dict for multiple use
2017-11-16 18:18:30 +00:00
c46139a17e
Avoid locking account_data tables for upserts
2017-11-16 18:08:01 +00:00
d8391f0541
Remove unused GROUP_ID_REGEX
2017-11-16 18:05:57 +00:00
4e8374856d
Document get_groups_for_user
2017-11-16 18:03:46 +00:00
270f9cd23a
Flake8
2017-11-16 18:03:31 +00:00
9d83d52027
Use a generator instead of a list
2017-11-16 17:57:34 +00:00
5b48eec4a1
Make sure we check AS groups for lookup on bulk
2017-11-16 17:55:15 +00:00
b1edf26051
Check group_id belongs to this domain
2017-11-16 17:54:27 +00:00
06e5bcfc83
Avoid locking for upsert on pushers tables
...
* replace the upsert into deleted_pushers with an insert
* no need to lock for upsert on pusher_throttle
2017-11-16 17:52:23 +00:00
624a8bbd67
Fix auth handler #2678
2017-11-16 17:19:02 +00:00
97bd18af4e
Add automagical AS Publicised Group(s)
...
via registration file "users" namespace:
```YAML
...
namespaces:
users:
- exclusive: true
regex: '.*luke.*'
group_id: '+all_the_lukes:hsdomain'
...
```
This is part of giving App Services their own groups for matching users. With this, ghost users will be given the appeareance that they are in a group and that they have publicised the fact, but _only_ from the perspective of the `get_publicised_groups_for_user` API.
2017-11-16 16:44:55 +00:00
77a1227870
Fix broken ref to IntegrityError
2017-11-16 16:03:38 +00:00
7ab2b69e18
Avoid locking `pushers` table on upsert
...
Now that _simple_upsert will retry on IntegrityError, we don't need to lock the
table.
2017-11-16 15:32:01 +00:00
10aaa1bc15
_simple_upsert: retry on IntegrityError
...
wrap the call to _simple_upsert_txn in a loop so that we retry on an
integrityerror: this means we can avoid locking the table provided there is an
unique index.
2017-11-16 15:30:15 +00:00
cdc9e50a5d
Cleanup in _simple_upsert_txn
...
Bail out early to reduce indentation
2017-11-16 15:29:10 +00:00
6f05de0e5e
synapse/config/password_auth_providers: Fixed bracket typo
...
Signed-off-by: Richard von Seck <richard.von-seck@gmx.net>
2017-11-16 15:59:38 +01:00
f959c01600
Merge pull request #2661 from matrix-org/rav/statereadstore
...
Pull out bits of StateStore to a mixin
2017-11-15 17:23:01 +00:00
30d2730ee2
Declare support for r0.3.0
2017-11-15 16:24:22 +00:00
1b83c09c03
Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs
...
Remove preserve_context_over_{fn, deferred}
2017-11-15 11:13:53 +00:00
7190a550dc
Merge pull request #2650 from matrix-org/dbkr/push_include_content_option
...
Rename redact_content option to include_content
2017-11-15 10:47:38 +00:00
b2cd6accf5
Remove __PreservingContextDeferred too
2017-11-14 23:00:10 +00:00
008aa2fc6d
Merge pull request #2671 from matrix-org/rav/room_list_fixes
...
Reshuffle room list request code
2017-11-14 18:11:02 +00:00
03feb7a34d
Bump version and changelog
2017-11-14 14:51:25 +00:00
35a4b63240
Pull out bits of StateStore to a mixin
...
... so that we don't need to secretly gut-wrench it for use in the slaved
stores. I haven't done the other stores yet, but we should. I'm tired of the
workers breaking every time we tweak the stores because I forgot to gut-wrench
the right method.
fixes https://github.com/matrix-org/synapse/issues/2655 .
2017-11-14 11:43:58 +00:00
4dd1bfa8c1
Revert "Revert "move _state_group_cache to statestore""
...
We're going to fix this properly on this branch, so that the _state_group_cache
can end up in StateGroupReadStore.
This reverts commit ab335edb02
2017-11-14 11:43:58 +00:00
6caa379ba1
Merge pull request #2658 from matrix-org/rav/store_heirarchy_init
...
Make __init__ consistent across Store hierarchy
2017-11-14 11:43:12 +00:00
7e6fa29cb5
Remove preserve_context_over_{fn, deferred}
...
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
44a1bfd6a6
Reshuffle room list request code
...
I'm not entirely sure if this will actually help anything, but it simplifies
the code and might give further clues about why room list search requests are
blowing out the get_current_state_ids caches.
2017-11-14 10:29:58 +00:00
1fc66c7460
Add a load of logging to the room_list handler
...
So we can see what it gets up to.
2017-11-14 10:23:47 +00:00
7bd6c87eca
Merge pull request #2668 from turt2live/travis/whoami
...
Add a route for determining who you are
2017-11-14 09:54:21 +00:00
812c191939
Remove redundent call
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2017-11-13 12:44:21 -07:00
c741ba59c9
Merge pull request #2669 from matrix-org/rav/cache_urlpreview_failure
...
Cache failures in url_preview handler
2017-11-13 18:36:24 +00:00
45ab288e07
Print instead of logging
...
because we had to wait until the logger was set up
2017-11-13 18:32:08 +00:00
8b33ac8f6c
Merge branch 'develop' into rav/invalid_request_utf8
2017-11-13 11:56:22 +00:00
6cfee09be9
Make __init__ consitstent across Store heirarchy
...
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
ab335edb02
Revert "move _state_group_cache to statestore"
...
This reverts commit f5cf3638e9
2017-11-13 10:05:33 +00:00
bfbf1e1f1a
Up cache size of get_global_account_data_by_type_for_user
2017-11-13 09:52:11 +00:00
2d314b771f
Add a route for determining who you are
...
Useful for applications which may have an access token, but no idea as to who owns it.
Signed-off-by: Travis Ralston <travpc@gmail.com>
2017-11-12 23:39:38 -07:00
5d15abb120
Bit more logging
2017-11-10 16:58:04 +00:00
46790f50cf
Cache failures in url_preview handler
...
Reshuffle the caching logic in the url_preview handler so that failures are
cached (and to generally simplify things and fix the logcontext leaks).
2017-11-10 16:50:50 +00:00
e508145c9b
Add some more comments appservice user registration
...
Explain why we don't validate userids registered via app services
2017-11-10 12:39:45 +00:00
e0ebd1e4bd
Downcase userids for shared-secret registration
2017-11-10 12:39:05 +00:00
f90649eb2b
Fix 500 on invalid utf-8 in request
...
If somebody sends us a request where the the body is invalid utf-8, we should
return a 400 rather than a 500. (json.loads throws a UnicodeError in this
situation)
We might as well catch all Exceptions here: it seems very unlikely that we
would get a request that *isn't caused by invalid json.
2017-11-10 09:15:39 +00:00
9b599bc18d
Downcase userid on registration
...
Force username to lowercase before attempting to register
https://github.com/matrix-org/synapse/issues/2660
2017-11-09 22:20:01 +00:00
9b803ccc98
Revert "Allow upper-case characters in mxids"
...
This reverts commit b70b646903
2017-11-09 21:57:24 +00:00
b70b646903
Allow upper-case characters in mxids
...
Because we're never going to be able to fix this :'(
2017-11-09 19:36:13 +00:00
2dce6b15c3
Fix typo
2017-11-09 15:56:16 +00:00
4e2b2508af
Register group servlet
2017-11-09 15:49:42 +00:00
13735843c7
Namespace visibility options for groups
2017-11-09 15:27:18 +00:00
618c7b816a
Merge pull request #2656 from matrix-org/rav/fix_deactivate
...
Fix 'NoneType' not iterable in /deactivate
2017-11-09 15:20:35 +00:00
889102315e
Fix 'NoneType' not iterable in /deactivate
...
make sure we actually return a value from user_delete_access_tokens
2017-11-09 15:15:33 +00:00
b2a788e902
Make the commented config have the default
2017-11-09 10:11:42 +00:00
82e4bfb53d
Add brackets
2017-11-09 10:06:42 +00:00
e8814410ef
Have an explicit API to update room config
2017-11-08 16:13:27 +00:00
94ff2cda73
Revert "Modify group room association API to allow modification of is_public"
2017-11-08 15:43:34 +00:00
d305987b40
Merge pull request #2631 from xyzz/fix_appservice_event_backlog
...
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-08 11:54:10 +00:00
167eb01d83
Merge pull request #2637 from spantaleev/avoid-noop-media-deletes
...
Avoid no-op media deletes
2017-11-08 11:53:27 +00:00
ad408beb66
better comments
2017-11-08 11:50:08 +00:00
1b870937ae
Log if any of the old config flags are set
2017-11-08 11:46:24 +00:00
2a98ba0ed3
Rename redact_content option to include_content
...
The redact_content option never worked because it read the wrong config
section. The PR introducing it
(https://github.com/matrix-org/synapse/pull/2301 ) had feedback suggesting the
name be changed to not re-use the term 'redact' but this wasn't
incorporated.
This reanmes the option to give it a less confusing name, and also
means that people who've set the redact_content option won't suddenly
see a behaviour change when upgrading synapse, but instead can set
include_content if they want to.
This PR also updates the wording of the config comment to clarify
that this has no effect on event_id_only push.
Includes https://github.com/matrix-org/synapse/pull/2422
2017-11-08 10:35:30 +00:00
02a9a93bde
Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_res
...
Fix bug in state group storage
2017-11-08 09:22:13 +00:00
e148438e97
s/items/iteritems/
2017-11-08 09:21:41 +00:00
d46386d57e
Remove useless assignment in notify_interested_services
2017-11-07 22:23:22 +03:00
228ccf1fe3
Merge pull request #2643 from matrix-org/matthew/user_dir_typos
...
Fix various embarrassing typos around user_directory and add some doc.
2017-11-07 17:31:11 +00:00
780dbb378f
Update deltas when doing auth resolution
...
Fixes a bug where the persisted state groups were different to those actually
being used after auth resolution.
2017-11-07 16:43:00 +00:00
1ca4288135
factor out _update_context_for_auth_events
...
This is duplicated, so let's factor it out before fixing it
2017-11-07 16:43:00 +00:00
f5cf3638e9
move _state_group_cache to statestore
...
this is internal to statestore, so let's keep it there.
2017-11-07 16:43:00 +00:00
5ef5e14ecc
Merge pull request #2636 from farialima/me-master
...
Fix for #2635 : correctly update rooms avatar/display name when modified by admin
2017-11-07 13:49:27 +00:00
76c9af193c
Revert "Merge branch 'master' of github.com:matrix-org/synapse into develop"
...
This reverts commit f9b255cd621bd654dabd
2017-11-07 13:32:35 +00:00
f9b255cd62
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-11-07 13:31:03 +00:00
38b265cb51
Remember to pick is_admin out of the db
2017-11-07 11:24:04 +00:00
5561c09091
Return whether a user is an admin within a group
2017-11-07 11:18:45 +00:00
3db5ff69b2
Merge pull request #2576 from maximevaillancourt/exclude-noscript-url-preview
...
Ignore <noscript> tags when generating URL preview descriptions
2017-11-07 11:09:22 +00:00
631fa4a1b7
create new indexes before dropping old ones to keep safetynet in place
2017-11-07 10:41:55 +00:00
bf993db11c
Logging and logcontext fixes for Limiter
...
Add some logging to the Limiter in a similar spirit to the Linearizer, to help
debug issues.
Also fix a logcontext leak.
Also refactor slightly to avoid throwing exceptions.
2017-11-07 00:48:57 +00:00
4ad883398f
s/users_in_pubic_room/users_in_public_rooms/g
2017-11-04 19:39:40 +00:00
d802e8ca6a
s/users_in_pubic_room/users_in_public_rooms/g
2017-11-04 19:38:13 +00:00
a100700630
fix copyright....
2017-11-04 19:35:49 +00:00
b6b075fd49
s/popualte/populate/
2017-11-04 19:35:33 +00:00
d1622e080f
s/intial/initial/
2017-11-04 19:35:14 +00:00
805196fbeb
Avoid no-op media deletes
...
If there are no media entries to delete,
avoid creating transactions, prepared statements
and unnecessary log entries.
Signed-off-by: Slavi Pantaleev <slavi@devture.com>
2017-11-04 09:50:15 +02:00
f103b91ffa
removed unused import flagged by flake8a
2017-11-03 18:45:49 +01:00
fa4f337b49
Fix for issue 2635: correctly update rooms avatar/display name when modified by admin
2017-11-03 18:25:04 +01:00
45fbe4ff67
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-02 22:49:43 +03:00
f851bc8182
Merge pull request #2630 from matrix-org/luke/fix-rooms-in-group
...
Make the get_rooms_in_group API more sane
2017-11-02 17:23:17 +00:00
9e09a1800b
Merge pull request #2629 from matrix-org/rav/register_inhibit_login
...
support inhibit_login in /register
2017-11-02 16:51:35 +00:00
a34c586a89
Make the get_rooms_in_group API more sane
...
Return entries with is_public = True when they're public and is_public = False otherwise.
2017-11-02 16:42:30 +00:00
6c3a02072b
support inhibit_login in /register
...
Allow things to pass inhibit_login when registering to ... inhibit logins.
2017-11-02 16:31:07 +00:00
4b36897cd9
Merge remote-tracking branch 'origin/develop' into rav/module_api_hooks
2017-11-02 15:19:17 +00:00
d4553818a0
Merge pull request #2627 from matrix-org/rav/custom_rest_endpoints
...
Add a hook for custom rest endpoints
2017-11-02 15:18:37 +00:00
6b6f03ae05
Merge pull request #2626 from matrix-org/rav/refactor_module_api
...
Factor _AccountHandler proxy out to ModuleApi
2017-11-02 15:15:30 +00:00
6b60f7dca0
Add more hooks to ModuleApi
...
add `get_user_by_req` and `invalidate_access_token`
2017-11-02 14:37:39 +00:00
fcdfc911ee
Add a hook for custom rest endpoints
...
Let the user specify custom modules which can be used for implementing extra
endpoints.
2017-11-02 14:36:55 +00:00
1189be43a2
Factor _AccountHandler proxy out to ModuleApi
...
We're going to need to use this from places that aren't password auth, so let's
move it to a proper class.
2017-11-02 14:36:11 +00:00
6650a07ede
Factor out _configure_named_resource
...
This was a bit of a code vomit, so let's factor it out to preserve some sanity
2017-11-02 14:33:37 +00:00
b19d9e2174
Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout
...
Notify auth providers on logout
2017-11-02 10:55:17 +00:00
1f080a6c97
Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers
...
Allow password_auth_providers to return a callback
2017-11-02 10:49:03 +00:00
04897c9dc1
Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers
...
Let auth providers get to the database
2017-11-02 10:41:25 +00:00
979eed4362
Fix user-interactive password auth
...
this got broken in the previous commit
2017-11-01 17:03:20 +00:00
bc8a5c0330
Notify auth providers on logout
...
Provide a hook by which auth providers can be notified of logouts.
2017-11-01 16:51:51 +00:00
4c8f94ac94
Allow password_auth_providers to return a callback
...
... so that they have a way to record access tokens.
2017-11-01 16:51:03 +00:00
846a94fbc9
Merge pull request #2620 from matrix-org/rav/auth_non_password
...
Let password auth providers handle arbitrary login types
2017-11-01 16:45:33 +00:00
3cd6b22c7b
Let password auth providers handle arbitrary login types
...
Provide a hook where password auth providers can say they know about other
login types, and get passed the relevant parameters
2017-11-01 16:43:57 +00:00
c9b9ef575b
Merge pull request #2621 from matrix-org/rav/refactor_accesstoken_delete
...
Move access token deletion into auth handler
2017-11-01 16:26:06 +00:00
275826f234
Merge pull request #2617 from matrix-org/matthew/auto-displayname
...
automatically set default displayname on register
2017-11-01 16:21:16 +00:00
4f0488b307
Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete
2017-11-01 16:20:19 +00:00
e5e930aec3
Merge pull request #2615 from matrix-org/rav/break_auth_device_dep
...
Break dependency of auth_handler on device_handler
2017-11-01 16:06:31 +00:00
fbbacb284e
Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens
...
Remove the last vestiges of refresh_tokens
2017-11-01 15:57:35 +00:00
9f7a555b4e
switch to setting default displayname in the storage layer
...
to avoid clobbering guest user displaynames on registration
2017-11-01 15:51:30 +00:00
dd13310fb8
Move access token deletion into auth handler
...
Also move duplicated deactivation code into the auth handler.
I want to add some hooks when we deactivate an access token, so let's bring it
all in here so that there's somewhere to put it.
2017-11-01 15:46:22 +00:00
0bb253f37b
Apparently this is python
2017-11-01 14:02:52 +00:00
59e7e62c4b
Log login requests
...
Carefully though, to avoid logging passwords
2017-11-01 13:58:01 +00:00
f8420d6279
automatically set default displayname on register
...
to avoid leaking ugly MXIDs and cluttering up the timeline with
displayname changes as well as membership joins for autojoin rooms
(e.g. the status autojoin rooms), automatically set the displayname
to match the localpart of the mxid upon registration.
2017-11-01 13:15:41 +00:00
99354b430e
Merge pull request #2612 from matrix-org/luke/groups-room-relationship-is-public
...
Modify group room association API to allow modification of is_public
2017-11-01 11:08:36 +00:00
74c56f794c
Break dependency of auth_handler on device_handler
...
I'm going to need to make the device_handler depend on the auth_handler, so I
need to break this dependency to avoid a cycle.
It turns out that the auth_handler was only using the device_handler in one
place which was an edge case which we can more elegantly handle by throwing an
error rather than fixing it up.
2017-11-01 10:27:06 +00:00
318a249c8b
Leave `is_public` as required argument of update_room_group_association
2017-11-01 09:36:01 +00:00
207fabbc6a
Update docs for updating room group association
2017-11-01 09:35:15 +00:00
356bcafc44
Remove the last vestiges of refresh_tokens
2017-10-31 20:35:58 +00:00
3e0aaad190
Let auth providers get to the database
...
Somewhat open to abuse, but also somewhat unavoidable :/
2017-10-31 17:22:29 +00:00
13b3d7b4a0
Flake8
2017-10-31 17:20:11 +00:00
20fe347906
Modify group room association API to allow modification of is_public
...
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
1650eb5847
DB schema interface for password auth providers
...
Provide an interface by which password auth providers can register db schema
files to be run at startup
2017-10-31 14:01:53 +00:00
c31a7c3ff6
Merge pull request #2609 from matrix-org/rav/refactor_login
...
Refactor some logic from LoginRestServlet into AuthHandler
2017-10-31 13:51:36 +00:00
1b65ae00ac
Refactor some logic from LoginRestServlet into AuthHandler
...
I'm going to need some more flexibility in handling login types in password
auth providers, so as a first step, move some stuff from LoginRestServlet into
AuthHandler.
In particular, we pass everything other than SAML, JWT and token logins down to
the AuthHandler, which now has responsibility for checking the login type and
fishing the password out of the login dictionary, as well as qualifying the
user_id if need be. Ideally SAML, JWT and token would go that way too, but
there's no real need for it right now and I'm trying to minimise impact.
This commit *should* be non-functional.
2017-10-31 10:48:41 +00:00
ffc574a6f9
Clean up backwards-compat hacks for ldap
...
try to make the backwards-compat flows follow the same code paths as the modern
impl.
This commit should be non-functional.
2017-10-31 10:47:02 +00:00
9bc17fc5fb
Fix wording on group creation error
2017-10-30 15:17:23 +00:00
208a6647f1
fix typo
2017-10-29 20:54:20 +00:00
d0abb4e8e6
Fix typo when checking if user is invited to group
2017-10-27 16:57:19 +01:00
977078f06d
Fix bad merge
2017-10-27 15:10:50 +01:00
6980c4557e
Merge branch 'erikj/attestation_jitter' of github.com:matrix-org/synapse into develop
2017-10-27 15:09:05 +01:00
af92f5b00f
Revert "Add jitter to validity period of attestations"
2017-10-27 15:07:21 +01:00
4ab8abbc2b
Merge branch 'erikj/attestation_local_fix' of github.com:matrix-org/synapse into develop
2017-10-27 15:07:08 +01:00
6af3656deb
Merge pull request #2595 from matrix-org/erikj/attestation_commnet
...
Add comment about attestations
2017-10-27 14:20:19 +01:00
4d83632009
Merge pull request #2591 from matrix-org/rav/device_delete_auth
...
Device deletion: check UI auth matches access token
2017-10-27 12:30:10 +01:00
110b373e9c
Merge pull request #2589 from matrix-org/rav/as_deactivate_account
...
Allow ASes to deactivate their own users
2017-10-27 12:29:32 +01:00
ca571b0ec3
Add jitter to validity period of attestations
...
This helps ensure that the renewals of attestations are spread out more
evenly.
2017-10-27 11:57:27 +01:00
d8c26162a1
Merge pull request #2582 from matrix-org/luke/group-is-public
...
Add is_public to groups table to allow for private groups
2017-10-27 11:41:13 +01:00
c067088747
Add comment about attestations
2017-10-27 11:35:41 +01:00
5451cc7792
Request is_public from database
2017-10-27 11:27:43 +01:00
124314672f
group is dict
2017-10-27 11:08:19 +01:00
6362298fa5
Create groups with is_public = True
2017-10-27 11:04:20 +01:00
8b56977b6f
Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header
...
Front-end proxy: pass through auth header
2017-10-27 11:01:50 +01:00
173567a7f2
Docstring for post_urlencoded_get_json
2017-10-27 10:59:50 +01:00
c7d9f25d22
Fix create_group to pass requester_user_id
2017-10-27 10:57:20 +01:00
e27b76d117
Import logger
2017-10-27 10:54:02 +01:00
8854c039f2
Merge pull request #2585 from matrix-org/rav/unstable_to_r0
...
Support /keys/upload on /r0 as well as /unstable
2017-10-27 10:53:48 +01:00
14f581abc2
Merge pull request #2584 from matrix-org/rav/fix_httpclient_logcontexts
...
Fix logcontext leaks in httpclient
2017-10-27 10:53:29 +01:00
2ca46c7afc
Correct logic for checking private group membership
2017-10-27 10:48:01 +01:00
82d8c1bacb
Fixup
2017-10-27 10:30:21 +01:00
2fd9831f7c
Merge pull request #2574 from matrix-org/erikj/room_list_fixes
...
Add logging and fix log contexts for publicRooms
2017-10-27 10:01:23 +01:00
195abfe7a5
Remove incorrect attestations
2017-10-27 09:58:13 +01:00
d8dde19f04
Log if we try to do attestations for our own user and group
2017-10-27 09:55:01 +01:00
585972b51a
Don't generate group attestations for local users
2017-10-27 09:46:56 +01:00
7a6546228b
Device deletion: check UI auth matches access token
...
(otherwise there's no point in the UI auth)
2017-10-27 00:04:31 +01:00
785bd7fd75
Allow ASes to deactivate their own users
2017-10-27 00:01:00 +01:00
c89e6aadff
Merge pull request #2581 from matrix-org/rav/fix_init_with_no_logfile
...
Fix error when running synapse with no logfile
2017-10-26 22:16:57 +01:00
54a2525133
Front-end proxy: pass through auth header
...
So that access-token-in-an-auth-header works.
2017-10-26 18:19:01 +01:00
0a5866bec9
Support /keys/upload on /r0 as well as /unstable
...
(So that we can stop riot relying on it in /unstable)
2017-10-26 18:18:23 +01:00
0d8e3ad48b
Fix logcontext leaks in httpclient
...
`preserve_context_over_fn` is borked
2017-10-26 18:17:10 +01:00
12ef02dc3d
SimpleHTTPClient: add support for headers
...
Sometimes we need to pass headers into these methods
2017-10-26 17:59:50 +01:00
69e8a05f35
Make it work
2017-10-26 17:55:58 +01:00
007cd48af6
Recreate groups table instead of adding column
...
Adding a column with non-constant default not possible in sqlite3
2017-10-26 17:55:22 +01:00
713e60b9b6
Awful hack to get default true
2017-10-26 17:38:14 +01:00
cfa4e658e0
Bump schema version to 46
2017-10-26 17:23:49 +01:00
595fe67f01
delint
2017-10-26 17:20:24 +01:00
9b2feef9eb
Add is_public to groups table to allow for private groups
...
Prevent group API access to non-members for private groups
Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.
2017-10-26 16:51:32 +01:00
f7f90e0c8d
Fix error when running synapse with no logfile
...
Fixes 'UnboundLocalError: local variable 'sighup' referenced before assignment'
2017-10-26 16:45:20 +01:00
8299b323ee
add release endpoints for /thirdparty
2017-10-26 16:58:20 +02:00
9b436c8b4c
register some /unstable endpoints in /r0 as well
2017-10-26 15:22:50 +02:00
566e21eac8
Update room_list.py
2017-10-26 11:39:54 +01:00
5287e57c86
Ignore noscript tags when generating URL previews
2017-10-25 20:44:34 -04:00
2a7e9faeec
Do logcontexts outside ResponseCache
2017-10-25 15:21:08 +01:00
1ad1ba9e6a
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-10-25 10:27:23 +01:00
33a9026cdf
Add logging and fix log contexts for publicRooms
2017-10-25 10:26:06 +01:00
efd0f5a3c5
tip for generating tls_fingerprints
2017-10-24 18:49:49 +01:00
6ba4fabdb9
Bump version and changelog
2017-10-24 14:15:27 +01:00
39dc52157d
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile
2017-10-24 09:16:20 +01:00
0be99858f3
fix vars named `l`
...
E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".
2017-10-23 15:56:38 +01:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
ce6d4914f4
Correctly wire in update group profile over federation
2017-10-23 15:21:24 +01:00
ecf198aab8
Merge pull request #2566 from matrix-org/rav/media_logcontext_leak
...
Fix a logcontext leak in the media repo
2017-10-23 14:47:49 +01:00
3267b81b81
Merge pull request #2561 from matrix-org/rav/id_checking
...
Updates to ID checking
2017-10-23 14:39:20 +01:00
d03cfc4258
Fix a logcontext leak in the media repo
2017-10-23 14:34:27 +01:00
1de557975f
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-10-23 13:18:12 +01:00
13e16cf302
Bump version and changelog
2017-10-23 13:13:31 +01:00
1135193dfd
Validate group ids when parsing
...
May as well do it whenever we parse a Group ID. We check the sigil and basic
structure here so it makes sense to check the grammar in the same place.
2017-10-21 00:30:39 +01:00
29812c628b
Allow = in mxids and groupids
...
... because the spec says we should.
2017-10-20 23:42:53 +01:00
58fbbe0f1d
Disallow capital letters in userids
...
Factor out a common function for checking user ids and group ids, which forbids
capitals.
2017-10-20 23:37:22 +01:00
631d7b87b5
Remove pointless create() method
...
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
6070647774
Correctly bump version
2017-10-19 16:40:20 +01:00
0ef0aeceac
Bump version and changelog
2017-10-19 13:48:33 +01:00
c7d46510d7
Flake8
2017-10-19 13:36:06 +01:00
ffd3f1a783
Add missing file...
2017-10-19 12:17:30 +01:00
29bafe2f7e
Add config to enable group creation
2017-10-19 12:13:44 +01:00
513c23bfd9
Enforce sensible group IDs
2017-10-19 12:01:01 +01:00
011d03a0f6
Fix typo
2017-10-19 11:22:48 +01:00
9ab859f27b
Fix typo in group attestation handling
2017-10-19 10:55:52 +01:00
bd5718d0ad
Fix typo in thumbnail generation
2017-10-19 10:27:18 +01:00
161a862ffb
Fix typo
2017-10-19 10:17:43 +01:00
69994c385a
Merge pull request #2553 from matrix-org/rav/fix_500_on_event_send
...
Fix 500 error when we get an error handling a PDU
2017-10-18 11:25:44 +01:00
b5dbbac308
Merge pull request #2552 from matrix-org/rav/fix_500_on_dodgy_powerlevels
...
Fix 500 error when fields missing from power_levels event
2017-10-17 20:53:30 +01:00
582bd19ee9
Fix 500 error when we get an error handling a PDU
...
FederationServer doesn't have a send_failure (and nor does its subclass,
ReplicationLayer), so this was failing.
I'm not really sure what the idea behind send_failure is, given (a) we don't do
anything at the other end with it except log it, and (b) we also send back the
failure via the transaction response. I suspect there's a whole lot of dead
code around it, but for now I'm just removing the broken bit.
2017-10-17 20:52:40 +01:00
74f99f227c
Doc some more dynamic Homeserver methods
2017-10-17 20:51:29 +01:00
c2bd177ea0
Fix 500 error when fields missing from power_levels event
...
If the users or events keys were missing from a power_levels event, then
we would throw 500s when trying to auth them.
2017-10-17 17:05:42 +01:00
fe6e9f580b
Merge pull request #2550 from krombel/fix_thumbnail_2548
...
fix thumbnailing (#2548 )
2017-10-17 15:35:18 +01:00
7216c76654
Improve error handling for missing files ( #2551 )
...
`os.path.exists` doesn't allow us to distinguish between permissions errors and
the path actually not existing, which repeatedly confuses people. It also means
that we try to overwrite existing key files, which is super-confusing. (cf
issues #2455 , #2379 ). Use os.stat instead.
Also, don't recomemnd the the use of --generate-config, which screws everything
up if you're using debian (cf #2455 ).
2017-10-17 14:46:17 +01:00
4cc8bb0767
Merge pull request #2549 from matrix-org/rav/event_persist_logcontexts
...
Fix logcontext handling for persist_events
2017-10-17 13:20:35 +01:00
4e242b3e20
Merge pull request #2545 from matrix-org/dbkr/auto_join_rooms
...
Add config option to auto-join new users to rooms
2017-10-17 11:45:49 +01:00
a6245478c8
fix thumbnailing ( #2548 )
...
in commit 0e28281a
2017-10-17 12:45:33 +02:00
2e9f5ea31a
Fix logcontext handling for persist_events
...
* don't use preserve_context_over_deferred, which is known broken.
* remove a redundant preserve_fn.
* add/improve some comments
2017-10-17 10:59:30 +01:00
a9c2e930ac
pep8
2017-10-17 10:13:13 +01:00
c05e6015cc
Add config option to auto-join new users to rooms
...
New users who register on the server will be dumped into all rooms in
auto_join_rooms in the config.
2017-10-16 17:57:27 +01:00
85f5674e44
Delint
2017-10-16 15:52:17 +01:00
c43e8a9736
Make it work. Warn about lack of user profile
2017-10-16 15:50:39 +01:00
a3ac4f6b0a
_create_rererouter for get_invited_users_in_group
2017-10-16 15:41:03 +01:00
5dfd0350c7
Merge branch 'develop' into luke/groups-invited-users
2017-10-16 15:34:36 +01:00
ca96d609e4
Merge pull request #2543 from matrix-org/luke/fix-on-group-invite-no-profile
...
Log a warning when no profile for invited member
2017-10-16 16:33:53 +02:00
2c5972f87f
Implement GET /groups/$groupId/invited_users
2017-10-16 15:31:11 +01:00
6079d0027a
Log a warning when no profile for invited member
...
And return empty profile
2017-10-16 14:20:45 +01:00
9342bcfce0
Omit the *s for @room notifications
...
They're just redundant
2017-10-16 13:38:10 +01:00
1b6b0b1e66
Add try/finally block to close t_byte_source
2017-10-13 15:34:08 +01:00
6b725cf56a
Remove old comment
2017-10-13 15:23:41 +01:00
2b24416e90
Don't reuse source but instead copy from primary media store to backup
2017-10-13 14:11:34 +01:00
b92a8e6e4a
PEP8
2017-10-13 13:58:57 +01:00
31aa7bd8d1
Move type into key
2017-10-13 13:47:38 +01:00
ad1911bbf4
Comment
2017-10-13 13:47:05 +01:00
c021c39cbd
Remove spurious addition
2017-10-13 13:46:53 +01:00
1f43d22397
Don't needlessly rename variable
2017-10-13 11:42:07 +01:00
a675bd08bd
Add paths back in...
2017-10-13 11:41:06 +01:00
4d7e1dde70
Remove unnecessary diff
2017-10-13 11:36:32 +01:00
ae5d18617a
Make things be absolute paths again
2017-10-13 11:35:44 +01:00
9732ec6797
s/write_to_file/write_to_file_and_backup/
2017-10-13 11:34:41 +01:00
0e28281a02
Fix up
2017-10-13 11:33:49 +01:00
505371414f
Fix up thumbnailing function
2017-10-13 11:23:53 +01:00
e3428d26ca
Fix typo
2017-10-13 10:39:59 +01:00
35332298ef
Fix up comments
2017-10-13 10:39:32 +01:00
64db043a71
Move makedirs to thread
2017-10-13 10:25:01 +01:00
b60859d6cc
Use make_deferred_yieldable
2017-10-13 10:24:19 +01:00
d76621a47b
Fix comments
2017-10-12 18:16:25 +01:00
4ae85ae121
Don't close prematurely..
2017-10-12 17:57:31 +01:00
cc505b4b5e
getvalue closes buffer
2017-10-12 17:52:30 +01:00
1259a76047
Get len before close
2017-10-12 17:39:23 +01:00
802ca12d05
Don't close file prematurely
2017-10-12 17:37:21 +01:00
e283b555b1
Copy everything to backup
2017-10-12 17:31:24 +01:00
b77a13812c
Typo
2017-10-12 15:32:32 +01:00
6dfde6d485
Remove dead code
2017-10-12 15:30:26 +01:00
c8eeef6947
Fix typos
2017-10-12 15:28:24 +01:00
67cb89fbdf
Fix typo
2017-10-12 15:23:41 +01:00
bf4fb1fb40
Basic implementation of backup media store
2017-10-12 15:20:59 +01:00
f807f7f804
log when we get an exception handling replication updates
2017-10-12 11:51:24 +01:00
b8d8ed1ba9
Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages
...
Allow error strings from spam checker
2017-10-12 10:31:03 +01:00
cc794d60e7
Merge pull request #2532 from matrix-org/rav/fix_linearizer
...
Fix stackoverflow and logcontexts from linearizer
2017-10-11 17:29:32 +01:00
8dd0c85ac5
Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling
...
log pdu_failures from incoming transactions
2017-10-11 17:29:14 +01:00
76fa695241
Merge pull request #2515 from matrix-org/rav/fix_receipt_logcontext
...
A logformatter which includes the stack when the exception was caught when
logging exceptions.
2017-10-11 17:28:01 +01:00
f30c4ed2bc
logformatter: fix AttributeError
...
make sure we have the relevant fields before we try to log them.
2017-10-11 17:26:17 +01:00
b752507b48
Fix fetching remote summaries
2017-10-11 16:59:18 +01:00
818b08d0e4
peeeeeeeeep8888888888888888888888888888
2017-10-11 15:54:00 +01:00
ea18996f54
Fix group stream replication
...
The stream update functions expect the storage function to return a list
of tuples.
2017-10-11 15:44:39 +01:00
68fd82e840
Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext
...
fix a logcontext leak in read receipt handling
2017-10-11 15:08:53 +01:00
4fad8efbfb
Fix stackoverflow and logcontexts from linearizer
...
1. make it not blow out the stack when there are more than 50 things waiting
for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505 .
2. Make it not mess up the log contexts.
2017-10-11 15:05:05 +01:00
b78bae2d51
fix isinstance
2017-10-11 14:49:09 +01:00
271f5601f3
Fix typo in invite to group
2017-10-11 14:45:33 +01:00
c3b7a45e84
Allow error strings from spam checker
2017-10-11 14:39:22 +01:00
c3e190ce67
fix a logcontext leak in read receipt handling
2017-10-11 14:37:20 +01:00
b75d443caf
log pdu_failures from incoming transactions
...
... even if we have no EDUs.
This appears to have been introduced in
476899295f
2017-10-11 14:36:13 +01:00
27e727a146
Fix typo
2017-10-11 14:32:40 +01:00
4ce4379235
Fix attestations to check correct server name
2017-10-11 14:11:43 +01:00
c2c47550f9
Fix schema delta versions
2017-10-11 13:23:15 +01:00
535cc49f27
Merge pull request #2466 from matrix-org/erikj/groups_merged
...
Initial Group Implementation
2017-10-11 13:20:07 +01:00
dfbf73408c
Merge pull request #2501 from matrix-org/dbkr/channel_notifications
...
Support for channel notifications
2017-10-11 13:19:29 +01:00
bc7f3eb32f
Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
...
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
ec954f47fb
Validate room ids
2017-10-11 13:15:44 +01:00
81a5e0073c
pep8
2017-10-10 15:53:34 +01:00
ab1bc9bf5f
Don't KeyError if no power_levels event
2017-10-10 15:34:05 +01:00
0f1eb3e914
Use notification levels in power_levels
...
Rather than making the condition directly require a specific power
level. This way the level require to notify a room can be configured
per room.
2017-10-10 15:23:00 +01:00
84e27a592d
Merge pull request #2490 from matrix-org/erikj/drop_left_room_events
...
Ignore incoming events for rooms that we have left
2017-10-10 11:58:32 +01:00
c9f034b4ac
There was already a constant for this
...
also update copyright
2017-10-10 11:47:10 +01:00
a9f9d68631
More optimisation
2017-10-10 11:38:31 +01:00
707374d5dc
What year is it!? Who's the president!?
2017-10-10 11:21:41 +01:00
89fa00ddff
Merge branch 'develop' into dbkr/channel_notifications
2017-10-10 11:20:17 +01:00
6a6cc27aee
fed server: process PDUs for different rooms in parallel
...
With luck, this will give a real-time improvement when there are many rooms and
the server ends up calling out to fetch missing events.
2017-10-09 18:30:31 +01:00
4c7c4d4061
Fed server: use a linearizer for ongoing transactions
...
We don't want to process the same transaction multiple times concurrently, so
use a linearizer.
2017-10-09 18:30:10 +01:00
ba5b9b80a5
fed server: refactor on_incoming_transaction
...
Move as much as possible to after the have_responded check, and reduce the
number of times we iterate over the pdu list.
2017-10-09 18:10:53 +01:00
a6e3222fe5
Fed server: Move origin-check code to _handle_received_pdu
...
The response-building code expects there to be an entry in the `results` list
for each entry in the pdu_list, so the early `continue` was messing this
up. That doesn't really matter, because all that the federation client does is
log any errors, but it's pretty poor form.
2017-10-09 17:53:32 +01:00
3cc852d339
Fancy logformatter to format exceptions better
...
This is a bit of an experimental change at this point; the idea is to see if it
helps us track down where our stack overflows are coming from by logging the
stack when the exception was caught and turned into a Failure. (We'll also need
edf2704420
2017-10-09 17:44:42 +01:00
0eeaa25694
Merge pull request #2508 from matrix-org/rav/federation_queue_logcontexts
...
Fix up logcontext handling in (federation) TransactionQueue
2017-10-09 17:43:48 +01:00
aa3fac8057
Merge pull request #2507 from matrix-org/rav/execute_concurrently_log_contexts
...
Fix logcontext handling for concurrently_execute
2017-10-09 17:43:32 +01:00
e8496efe84
Fix up comment
2017-10-09 15:17:34 +01:00
01bbacf3c4
Fix up logcontext handling in (federation) TransactionQueue
...
Avoid using preserve_context_over_function, which has problems with respect to
logcontexts.
2017-10-06 22:39:25 +01:00
148428ce76
Fix logcontext handling for concurrently_execute
...
Avoid preserve_context_over_deferred, which is broken.
2017-10-06 22:24:28 +01:00
c8f568ddf9
Fix up deferred handling in federation.py
...
* Avoid preserve_context_over_deferred, which is broken
* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
failure" erros
2017-10-06 22:14:24 +01:00
3ddda939d3
some comments in the state res code
2017-10-05 14:58:17 +01:00
f878e6f8af
Spam checking: add the invitee to user_may_invite
2017-10-05 14:02:28 +01:00
269af961e9
Make be faster
2017-10-05 13:27:12 +01:00
ed80c6b6cc
Add fastpath optimisation
2017-10-05 13:20:22 +01:00
e433393c4f
pep8
2017-10-05 13:08:02 +01:00
985ce80375
They're called rooms
2017-10-05 13:03:44 +01:00
b9b9714fd5
Get rule type right
2017-10-05 13:02:19 +01:00
fa969cfdde
Support for channel notifications
...
Add condition type to check the sender's power level and add a base
rule using it for @channel notifications.
2017-10-05 12:39:18 +01:00
0c8da8b519
Use better method for word boundary searching
...
From ebc95667b8
2017-10-05 11:57:43 +01:00
eaaa837e00
Don't corrupt cache
2017-10-05 11:43:22 +01:00
cbe3c3fdd4
pep8
2017-10-05 11:43:10 +01:00
6748f0a579
Fix notif kws that start/end with non-word chars
...
Only prepend / append word bounary characters if the search
expression starts or ends with a word character, otherwise they
don't work because there's no word bounary between whitespace and
a non-word char.
2017-10-05 11:33:30 +01:00
d8ce68b09b
spam check room publishing
2017-10-04 14:29:33 +01:00
78d4ced829
un-double indent
2017-10-04 12:44:27 +01:00
197c14dbcf
Add room creation checks to spam checker
...
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
1e2ac54351
s/roomid/room_id/
2017-10-03 17:41:38 +01:00
1e375468de
pass room id too
2017-10-03 17:13:14 +01:00
c2c188b699
Federation was passing strings anyway
...
so pass string everywhere
2017-10-03 15:46:19 +01:00
c46a0d7eb4
this shouldn't be debug
2017-10-03 15:20:14 +01:00
bd769a81e1
better logging
2017-10-03 15:16:40 +01:00
537088e7dc
Actually write warpper function
2017-10-03 14:28:12 +01:00
41fd9989a2
Skip spam check for admin users
2017-10-03 14:17:44 +01:00
11d62f43c9
Invalidate cache
2017-10-03 14:12:28 +01:00
e4ab96021e
Update comments
2017-10-03 14:10:41 +01:00
2a7ed700d5
Fix param name & lint
2017-10-03 14:04:10 +01:00
84716d267c
Allow spam checker to reject invites too
2017-10-03 13:56:43 +01:00
f2da6df568
Remove spurious line feed
2017-10-03 11:31:06 +01:00
30848c0fcd
Ignore incoming events for rooms that we have left
...
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.
However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.
This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
e585c83209
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-10-02 18:11:24 +01:00
6c1bb1601e
Bump version and changelog
2017-10-02 18:05:17 +01:00
ea87cb1ba5
Make 'affinity' package optional
2017-10-02 18:03:59 +01:00
3fed5bb25f
Move quit_with_error
2017-10-02 17:59:34 +01:00
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
90d70af269
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-10-02 16:20:23 +01:00
e4a709eda3
Bump version and change log
2017-10-02 13:51:38 +01:00
cafb8de132
Unfreeze event before serializing with ujson
...
In newer versions of https://github.com/esnme/ultrajson , ujson does not
serialize frozendicts (introduced in esnme/ultrajson@53f85b1 ). Although
the PyPI version is still 1.35, Fedora ships with a build from commit
esnme/ultrajson@2f1d487 . This causes the serialization to fail if the
distribution-provided package is used.
This runs the event through the unfreeze utility before serializing it.
Thanks to @ignatenkobrain for tracking down the root cause.
fixes #2351
Signed-off-by: Jeremy Cline <jeremy@jcline.org>
2017-09-30 11:22:37 -04:00
d5325d7ef1
Merge pull request #2480 from matrix-org/rav/federation_client_logging
...
Improve logging of failures in matrixfederationclient
2017-09-29 17:32:53 +01:00
d5694ac5fa
Only log if we've removed media
2017-09-28 16:08:08 +01:00
e43de3ae4b
Improve logging of failures in matrixfederationclient
...
* don't log exception types twice
* not all exceptions have a meaningful 'message'. Use the repr rather than
attempting to build a string ourselves.
2017-09-28 15:38:09 +01:00
75e67b9ee4
Handle SERVFAILs when doing AAAA lookups for federation ( #2477 )
...
... to cope with people with broken dnssec setups, mostly
2017-09-28 15:24:00 +01:00
768f00dedb
Up the limits on number of url cache entries to delete at one time
2017-09-28 14:27:27 +01:00
4dc07e93a8
Add old indices
2017-09-28 14:10:33 +01:00
7cc483aa0e
Clear up expired url cache every 10s
2017-09-28 13:56:53 +01:00
e1e7d76cf1
Actually assign result to variable
2017-09-28 13:55:29 +01:00
93247a424a
Only pull out local media that were for url cache
2017-09-28 13:48:14 +01:00
5f501ec7e2
Fix typo in url cache expiry timer
2017-09-28 12:59:01 +01:00
761d255fdf
Merge pull request #2479 from matrix-org/erikj/expire_url_cache_thumbnails
...
Support new and old style media id formats
2017-09-28 12:58:13 +01:00
ace8079086
Support new and old style media id formats
2017-09-28 12:52:51 +01:00
7a44c01d89
Fix typo
2017-09-28 12:46:04 +01:00
ae79764fe5
Change expires column to expires_ts
2017-09-28 12:37:53 +01:00
77f1d24de3
More brackets
2017-09-28 12:23:15 +01:00
9ccb4226ba
Delete expired url cache data
2017-09-28 12:18:06 +01:00
8090fd4664
Fix /joined_members to work with AS users
2017-09-28 10:09:32 +01:00
adec03395d
Fix bug where /joined_members didn't check user was in room
2017-09-27 15:14:39 +01:00
ef3a5ae787
Don't test is spam_checker not None
...
Sometimes it's a Mock object which is not none but is still not
what we're after
2017-09-27 11:24:19 +01:00
8c06dd6071
Remove unintentional debugging
2017-09-27 10:31:14 +01:00
60c78666ab
pep8
2017-09-27 10:26:13 +01:00
1786b0e768
Forgot the new file again :(
2017-09-27 10:22:54 +01:00
6cd5fcd536
Make the spam checker a module
2017-09-26 19:20:23 +01:00
9fd086e506
unnecessary parens
2017-09-26 17:59:46 +01:00
0b03a97708
Add module_loader.py
2017-09-26 17:56:41 +01:00
4824a33c31
Factor out module loading to a separate place
...
So it can be reused
2017-09-26 17:51:26 +01:00
17b8e2bd02
Add remove room API
2017-09-26 15:52:41 +01:00
a8e2a3df32
Add unique index to group_rooms table
2017-09-26 15:39:21 +01:00
95298783bb
Add is_publicised to group summary
2017-09-26 11:04:37 +01:00
1a398b19fd
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.23.0
2017-09-26 10:08:59 +01:00
f4c8cd5e85
Bump changelog and version
2017-09-26 10:02:48 +01:00
e3edca3b5d
Refactor to speed up incremental syncs
2017-09-25 17:35:39 +01:00
44edac0497
Merge branch 'release-v0.23.0' of github.com:matrix-org/synapse into develop
2017-09-25 14:52:46 +01:00
94133d7ce8
Merge branch 'develop' into develop
2017-09-25 11:50:11 +01:00
ba8fdc925c
Bump version and changes
2017-09-25 11:01:31 +01:00
79b3cf3e02
Fix logcontxt leak in keyclient ( #2465 )
...
preserve_context_over_function doesn't do what you want it to do.
2017-09-25 09:51:39 +01:00
f65e31d22f
Do an AAAA lookup on SRV record targets ( #2462 )
...
Support SRV records which point at AAAA records, as well as A records.
Fixes https://github.com/matrix-org/synapse/issues/2405
2017-09-22 20:26:47 +01:00
f496399ac4
fix thinko'd docstring
2017-09-22 15:34:14 +01:00
3166ed55b2
Fix device list when rejoining room ( #2461 )
2017-09-22 14:44:17 +01:00
e1dec2f1a7
Remove user from group summary when the leave the group
2017-09-21 16:09:57 +01:00
bb746a9de1
Revert: Keep room_id's in group summary
2017-09-21 15:57:22 +01:00
ae8d4bb0f0
Keep room_id's in group summary
2017-09-21 15:55:18 +01:00
197d82dc07
Correctly return next token
2017-09-20 11:12:11 +01:00
069ae2df12
Fix initial sync
2017-09-20 10:52:12 +01:00
6de74ea6d7
Fix logcontexts in _check_sigs_and_hashes
2017-09-20 01:32:42 +01:00
c5c24c239b
Fix logcontext handling in verify_json_objects_for_server
...
preserve_context_over_fn is essentially broken, because (a) it pointlessly
drops the current logcontext before calling its wrapped function, which means
we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the
resulting deferred in a _PreservingContextDeferred, which is very dangerous
because you then can't yield on it without leaking context back into the
reactor.
Instead, let's specify that the resultant deferreds call their callbacks with
no logcontext.
2017-09-20 01:32:42 +01:00
c5b0e9f485
Turn _start_key_lookups into an inlineCallbacks function
...
... which means that logcontexts can be correctly preserved for the stuff it
does.
get_server_verify_keys is now called with the logcontext, so needs to
preserve_fn when it fires off its nested inlineCallbacks function.
Also renames get_server_verify_keys to reflect the fact it's meant to be
private.
2017-09-20 01:32:42 +01:00
abdefb8a01
Fix potential race in _start_key_lookups
...
If the verify_request.deferred has already completed, then `remove_deferreds`
will be called immediately. It therefore might resolve the server_to_deferred
deferred while there are still other requests for that server in flight.
To avoid that, we should build the complete list of requests, and *then* add the
callbacks.
2017-09-20 01:32:42 +01:00
afbd773dc6
Add some comments to _start_key_lookups
2017-09-20 01:32:42 +01:00
2a4b9ea233
Consistency for how verify_request.deferred is called
...
Define that it is run with no log context, and make sure that happens.
If we aren't careful to reset the logcontext, we can't bung the deferreds into
defer.gatherResults etc. We don't actually do that directly, but we *do*
resolve other deferreds from affected callbacks (notably the server_to_deferred
map in _start_key_lookups), and those *do* get passed into
defer.gatherResults. It turns out that this way ends up being least confusing.
2017-09-20 01:32:42 +01:00
3b98439eca
Factor out _start_key_lookups
...
... to make it easier to see what's going on.
2017-09-20 01:32:42 +01:00
fde63b880d
Replace `server_and_json` with `verify_requests`
...
This is a precursor to factoring some of this code out.
2017-09-20 01:32:42 +01:00
2d511defd9
pull out handle_key_deferred to top level
...
There's no need for this to be a nested definition; pulling it out not only
makes it more efficient, but makes it easier to check that it's not accessing
any local variables it shouldn't be.
2017-09-20 01:32:42 +01:00
dd1ea9763a
Fix incorrect key_ids in error message
2017-09-20 01:32:42 +01:00
e76d1135dd
Invalidate signing key cache when we gat an update
...
This might make the cache slightly more efficient.
2017-09-20 01:32:42 +01:00
fcf2c0fd1a
Remove redundant `preserve_fn`
...
preserve_fn is a no-op unless the wrapped function returns a
Deferred. verify_json_objects_for_server returns a list, so this is doing
nothing.
2017-09-20 01:32:42 +01:00
9864efa532
Fix concurrent server_key requests ( #2458 )
...
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
2017-09-19 23:25:44 +01:00
aa620d09a0
Add a config option to block all room invites ( #2457 )
...
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
2017-09-19 16:08:14 +01:00
2eabdf3f98
add some comments to on_exchange_third_party_invite_request
2017-09-19 12:20:36 +01:00
5ed109d59f
PoC for filtering spammy events ( #2456 )
...
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
93e504d04e
Ensure that creator of group sees group down /sync
2017-09-19 11:08:16 +01:00
b5feaa5a49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged
2017-09-19 11:07:45 +01:00
3f405b34e9
Fix overzealous kicking of guest users ( #2453 )
...
We should only kick guest users if the guest access event is authorised.
2017-09-19 08:52:52 +01:00
290777b3d9
Clean up and document handling of logcontexts in Keyring ( #2452 )
...
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
2017-09-18 18:31:01 +01:00
77c81ca6ea
Merge pull request #2451 from matrix-org/erikj/add_state_to_timeline
...
Don't filter out current state events from timeline
2017-09-18 17:22:33 +01:00
2d1b7955ae
Don't filter out current state events from timeline
2017-09-18 17:13:03 +01:00
862c8da560
Merge pull request #2450 from matrix-org/dbkr/push_event_id_only
...
Add support for event_id_only push format
2017-09-18 16:41:29 +01:00
436ee0a2ea
Also include the room_id
...
as really it's part of the event ID
2017-09-18 15:58:38 +01:00
b393f5db51
Use .get - it's much shorter
2017-09-18 15:50:26 +01:00
a2562f9d74
Add support for event_id_only push format
...
Param in the data dict of a pusher that tells an HTTP pusher to
send just the event_id of the event it's notifying about and the
notification counts. For clients that want to go & fetch the body
of the event themselves anyway.
2017-09-18 15:39:39 +01:00
d6dadd95ac
Correctly handle leaving room in /key/changes
2017-09-18 15:38:22 +01:00
4a94eb3ea4
Fix typo
2017-09-15 09:56:54 +01:00
3a0cee28d6
Actually hook leave notifs up
2017-09-14 11:49:37 +01:00
4f845a0713
Handle joining/leaving rooms in /keys/changes
2017-09-13 16:28:08 +01:00
473700f016
Get left rooms
2017-09-13 15:13:41 +01:00
9ce866ed4f
In sync handle device lists for newly joined/left rooms
2017-09-12 16:44:26 +01:00
69ef4987a6
Add left section to /keys/changes
2017-09-08 14:44:36 +01:00
53cc8ad35a
Send down device list change notif when member leaves/rejoins room
2017-09-07 15:08:39 +01:00
4a9b1cf253
Add user profiles to summary from group server
2017-08-25 16:23:58 +01:00
258409ef61
Fix typos and reinherit
2017-08-25 14:45:20 +01:00
bf81f3cf2c
Split out profile handler to fix tests
2017-08-25 14:34:56 +01:00
27ebc5c8f2
Add remote profile cache
2017-08-25 11:25:47 +01:00
97c544f91f
Add _simple_update
2017-08-25 11:11:37 +01:00
a7effa8400
Merge pull request #2288 from kyrias/bcrypt
...
python_dependencies: Use bcrypt module instead of py-bcrypt
2017-08-23 14:14:56 +01:00
2800983f3e
Merge pull request #2410 from matrix-org/erikj/groups_publicise
...
Add ability to publicise group membership
2017-08-21 16:51:56 +01:00
8b50fe5330
Use BOOLEAN rather than TEXT type
2017-08-21 16:37:29 +01:00
175a01f56c
Groups: Fix mising json.load in initial sync
2017-08-21 14:45:56 +01:00
692250c6be
Fix user_dir startup
...
Add missing parameter to _base.start_worker_reactor
2017-08-16 15:11:29 +01:00
d2352347cf
Fix process startup
...
escape the % that got added in 92168cb
2017-08-16 14:57:35 +01:00
92168cbbc5
explain why CPU affinity is a good idea
2017-08-15 18:27:42 +01:00
10d8b701a1
Allow configuration of CPU affinity
...
Make it possible to set the CPU affinity in the config file, so that we don't
need to remember to do it manually every time.
2017-08-15 17:08:28 +01:00
543c794a76
Factor out common application start
...
We have 10 copies of this code, and I don't really want to update each one
separately.
2017-08-15 17:04:40 +01:00
ba3ff7918b
Fixup
2017-08-11 13:42:42 +01:00
ef8e578677
Add bulk group publicised lookup API
2017-08-09 13:36:22 +01:00
b880ff190a
Allow update group publicity
2017-08-08 14:19:41 +01:00
05e21285aa
Store whether the user wants to publicise their membership of a group
2017-08-08 13:01:46 +01:00
eae04f1952
fix english
2017-08-04 23:56:42 +01:00
5699b05072
typo
2017-08-04 23:44:37 +01:00
a1e67bcb97
Remove stale TODO comments
2017-08-04 10:07:10 +01:00
09552f9d9c
Reduce spammy log line in synchrotrons
2017-08-02 17:29:51 +01:00
f18373dc5d
Fix iteration of requests_missing_keys; list doesn't have .values()
...
Signed-off-by: Kenny Keslar <r3dey3@r3dey3.com>
2017-07-26 22:44:19 -05:00
966a70f1fa
Update comment
2017-07-24 17:49:39 +01:00
629cdfb124
Use join rather than joined, etc.
2017-07-24 14:54:05 +01:00
ed666d3969
Fix all the typos
2017-07-24 14:05:09 +01:00
b76ef6ccb8
Include users membership in group in summary API
2017-07-24 13:55:39 +01:00
851aeae7c7
Check users/rooms are in group before adding to summary
2017-07-24 13:40:56 +01:00
d5e32c843f
Correctly add joins to correct segment
2017-07-24 13:31:26 +01:00
96917d5552
Merge pull request #2378 from matrix-org/erikj/group_sync_support
...
Add groups to sync stream
2017-07-21 11:05:39 +01:00
0401604222
Merge pull request #2377 from matrix-org/erikj/group_profile_update
...
Add update group profile API
2017-07-20 17:53:39 +01:00
b238cf7f6b
Remove spurious content param
2017-07-20 17:49:55 +01:00
960dae3340
Add notifier
2017-07-20 17:14:44 +01:00
2cc998fed8
Fix replication. And notify
2017-07-20 17:13:18 +01:00
139fe30f47
Remember to cast to bool
2017-07-20 16:47:35 +01:00
4d793626ff
Fix bug in generating current token
2017-07-20 16:42:44 +01:00
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
0ab153d201
Check values are strings
2017-07-20 16:24:18 +01:00
8209b5f033
Fix a storage desc
2017-07-20 16:22:22 +01:00
57826d645b
Fix typo
2017-07-20 13:15:22 +01:00
6f443a74cf
Add update group profile API
2017-07-20 09:46:33 +01:00
14a34f12d7
Comments
2017-07-18 17:28:42 +01:00
3431ec55dc
Comments
2017-07-18 17:23:50 +01:00
6027b1992f
Fix permissions
2017-07-18 16:51:25 +01:00
e884ff31d8
Add DELETE
2017-07-18 16:41:44 +01:00
05c13f6c22
Add 'args' param to post_json
2017-07-18 16:40:21 +01:00
94ecd871a0
Fix typos
2017-07-18 16:38:54 +01:00
12ed4ee48e
Correctly parse query params
2017-07-18 15:33:09 +01:00
332839f6ea
Update federation client pokes
2017-07-18 14:45:37 +01:00
e5ea6dd021
Add client apis
2017-07-18 14:37:06 +01:00
cccfcfa7b9
Comments
2017-07-18 10:35:18 +01:00
68f34e85ce
Use transport client directly
2017-07-18 10:29:57 +01:00
3e703eb04e
Comment
2017-07-18 10:17:25 +01:00
508460f240
Remove sync stuff
2017-07-18 09:55:46 +01:00
6e9f147faa
Add GroupID type
2017-07-18 09:47:25 +01:00
4540730111
Remove unused tables
2017-07-18 09:38:15 +01:00
e96ee95a7e
Remove sync stuff
2017-07-18 09:38:08 +01:00
2f9eafdd36
Add local group server support
2017-07-17 12:03:49 +01:00
bfde076022
Increase cache hit ratio for push
...
We don't update the cache in all code paths, which causes subsequent
calls to miss the cache
2017-07-14 16:11:26 +01:00
cb3aee8219
Ensure category and role ids are non-null
2017-07-14 14:06:55 +01:00
85fda57208
Add DEFAULT_ROLE_ID
2017-07-14 14:03:54 +01:00
4b203bdba5
Correctly increment orders
2017-07-14 14:02:31 +01:00
8d26385d76
Add more metrics to push rule evaluation
2017-07-13 14:37:30 +01:00
3b0470dba5
Remove unused functions
2017-07-13 13:54:01 +01:00
8575e3160f
Comments
2017-07-13 13:52:41 +01:00
f60218ec41
Push: Don't acquire lock unless necessary
2017-07-13 11:23:53 +01:00
a78cda4baf
Remove TODO
2017-07-13 11:17:07 +01:00
7a39da8cc6
Add summary APIs to federation
2017-07-13 11:13:19 +01:00
5bbb53580a
raise NotImplementedError
2017-07-13 10:25:29 +01:00
26451a09eb
Comments
2017-07-12 14:47:18 +01:00
8d55877c9e
Simplify checking if admin
2017-07-12 11:43:39 +01:00
a62406aaa5
Add group summary APIs
2017-07-12 11:36:15 +01:00
e9aec001f4
Use less DB for device list handling in sync
2017-07-12 10:30:10 +01:00
28e8c46f29
Merge pull request #2352 from matrix-org/erikj/group_server_split
...
Initial Group Server
2017-07-12 10:26:29 +01:00
6d586dc05c
Comment
2017-07-12 09:58:37 +01:00
410b4e14a1
Move comment
2017-07-11 15:44:18 +01:00
fe4e885f54
Add federation API for adding room to group
2017-07-11 14:35:07 +01:00
bbb739d24a
Comment
2017-07-11 14:31:36 +01:00
26752df503
Typo
2017-07-11 14:29:03 +01:00
e52c391cd4
Rename column to attestation_json
2017-07-11 14:25:46 +01:00
0aac30d53b
Comments
2017-07-11 14:23:50 +01:00
0184a97dbd
Merge pull request #2354 from krombel/reduce_static_sync_reply
...
encode sync-response statically
2017-07-11 14:19:56 +01:00
85b9f76f1d
split out reducing stuff; just make encode_* static
2017-07-11 13:14:35 +02:00
6322fbbd41
Comment
2017-07-11 11:52:03 +01:00
8ba89f1050
Remove u/ requirement
2017-07-11 11:45:32 +01:00
429925a5e9
Lift out visibility parsing
2017-07-11 11:44:08 +01:00
83936293eb
Comments
2017-07-11 11:42:25 +01:00
e2cb760dcc
Merge pull request #2357 from matrix-org/erikj/push
...
Don't compute push actions for backfilled events
2017-07-11 10:53:22 +01:00
925b3638ff
Reduce log levels in tcp replication
2017-07-11 10:04:21 +01:00
9a6fd3ef29
Don't compute push actions for backfilled events
2017-07-11 10:02:21 +01:00
2f82de18ee
fix test
2017-07-10 17:34:58 +02:00
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
6e16aca8b0
encode sync-response statically; omit empty objects from sync-response
2017-07-10 16:42:17 +02:00
d4d12daed9
Include registration and as stores in frontend proxy
2017-07-07 18:36:45 +01:00
f467a8f66d
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-07-07 18:26:28 +01:00
1fc4a962e4
Add a frontend proxy
2017-07-07 18:19:46 +01:00
f502b0dea1
Perf: Don't filter events for push
...
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
2017-07-07 14:04:40 +01:00
76ed3476d3
Bump version and changelog
2017-07-06 18:11:22 +01:00
58dc1f2c78
Merge pull request #2342 from matrix-org/erikj/pusher_pool_instantiate
...
Fix bug where pusherpool didn't start and broke some rooms
2017-07-06 18:08:43 +01:00
5a7f561a9b
Fix bug where pusherpool didn't start and broke some rooms
...
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.
This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
ed9a7f5436
Merge pull request #2309 from matrix-org/erikj/user_ip_repl
...
Fix up user_ip replication commands
2017-07-06 14:33:14 +01:00
1f64207f26
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-07-06 13:57:45 +01:00
6264cf9666
Bump version and changelog
2017-07-06 10:35:56 +01:00
f386632800
Merge pull request #2334 from matrix-org/erikj/refactor_transport_server
...
Separate federation servlet into different lists
2017-07-05 17:09:07 +01:00
5e49a57ecc
Separate federation servlet into different lists
2017-07-05 14:32:24 +01:00
3d31b39297
Merge pull request #2332 from matrix-org/rav/fix_pushes
...
Fix caching error in the push evaluator
2017-07-05 11:10:53 +01:00
73cfe48031
Fix caching error in the push evaluator
...
Initialising `result` to `{}` in the parameters meant that every call to
_flatten_dict used the *same* target dictionary.
I'm hopeful this will fix https://github.com/matrix-org/synapse/issues/2270 ,
but I suspect it won't. (This code seems to have been here since forever,
unlike the bug, and I don't really think it explains the observed
behaviour). Still, it makes it hard to investigate the problem.
2017-07-05 00:28:43 +01:00
05538587ef
Bump version and changelog
2017-07-04 14:02:21 +01:00
f92d7416d7
Merge pull request #2330 from matrix-org/erikj/cache_size_factor
...
Increase default cache size
2017-07-04 10:51:21 +01:00
495f075b41
Increase default cache factor size.
2017-07-04 09:58:32 +01:00
b5e8d529e6
Define CACHE_SIZE_FACTOR once
2017-07-04 09:56:44 +01:00
3e279411fe
Improve the error handling for bad invites received over federation
2017-06-30 16:20:30 +01:00
6ff14ddd2e
Make into list
2017-06-29 15:47:37 +01:00
5946aa0877
Prefill forward extrems and event to state groups
2017-06-29 15:38:48 +01:00
2c365f4723
Cache macaroon parse and validation
...
Turns out this can be quite expensive for requests, and is easily
cachable. We don't cache the lookup to the DB so invalidation still
works.
2017-06-29 14:50:18 +01:00
c72058bcc6
Use an ExpiringCache for storing registration sessions
...
This is because pruning them was a significant performance drain on
matrix.org
2017-06-29 14:08:37 +01:00
27f26e48b7
Serialize user ip command as json
2017-06-27 16:25:38 +01:00
8c23221666
Fix up
2017-06-27 15:53:45 +01:00
731f3c37a0
Merge branch 'release-v0.22.0' of github.com:matrix-org/synapse into develop
2017-06-27 15:41:34 +01:00
4b444723f0
Merge pull request #2308 from matrix-org/erikj/user_ip_repl
...
Make workers report to master for user ip updates
2017-06-27 15:36:47 +01:00
816605a137
Merge pull request #2307 from matrix-org/erikj/user_ip_batch
...
Batch upsert user ips
2017-06-27 15:08:32 +01:00
78cefd78d6
Make workers report to master for user ip updates
2017-06-27 14:58:10 +01:00
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
ed3d0170d9
Batch upsert user ips
2017-06-27 13:37:04 +01:00
976128f368
Update version and changelog
2017-06-26 16:14:56 +01:00
d04d672a80
Merge pull request #2290 from matrix-org/erikj/ensure_round_trip
...
Reject local events that don't round trip the DB
2017-06-26 15:12:02 +01:00
1bce3e6b35
Remove unused variables
2017-06-26 14:03:27 +01:00
e3cbec10c1
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip
2017-06-26 14:02:44 +01:00
8abdd7b553
Fix up indices for users_who_share_rooms
2017-06-26 14:01:30 +01:00
ff13c5e7af
Merge pull request #2301 from xwiki-labs/push-redact-content
...
Add configuration parameter to allow redaction of content from push m…
2017-06-24 13:13:51 +01:00
27bd0b9a91
Change the config file generator to more descriptive explanation of push.redact_content
2017-06-24 10:32:12 +02:00
bce144595c
Fix TravisCI tests for PR #2301 - Fat finger mistake
2017-06-23 15:26:09 +02:00
75eba3b07d
Fix TravisCI tests for PR #2301
2017-06-23 15:15:18 +02:00
1591eddaea
Add configuration parameter to allow redaction of content from push messages for google/apple devices
2017-06-23 13:01:04 +02:00
7fe8ed1787
Store URL cache preview downloads seperately
...
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
44c722931b
Make some more params configurable
2017-06-22 14:59:52 +01:00
2d520a9826
Typo. ARGH.
2017-06-22 14:42:39 +01:00
24d894e2e2
Fix thinko in unhandled user spam
2017-06-22 14:39:05 +01:00
ccfcef6b59
Merge branch 'master' into develop
2017-06-22 13:03:44 +01:00
e0004aa28a
Add desc
2017-06-22 10:03:48 +01:00
b668112320
Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar
...
Don't work out users who share room with appservice users
2017-06-21 14:50:24 +01:00
dae9a00a28
Initialise exclusive_user_regex
2017-06-21 14:19:33 +01:00
8177563ebe
Fix for workers
2017-06-21 13:57:49 +01:00
4202fba82a
Merge branch 'develop' into avoid_duplicate_filters
2017-06-21 14:48:21 +02:00
812c030e87
replaced json.dumps with encode_canonical_json
2017-06-21 14:48:12 +02:00
1217c7da91
Don't work out users who share room with appservice users
2017-06-21 12:00:41 +01:00
385dcb7c60
Handle thumbnail urls
2017-06-19 17:48:28 +01:00
b8b936a6ea
Add API to quarantine media
2017-06-19 17:39:21 +01:00
e5ae386ea4
Handle all cases of sending membership events
2017-06-19 16:07:54 +01:00
36e51aad3c
Remove unused import
2017-06-19 14:42:21 +01:00
b490299a3b
Change to create new room and join other users
2017-06-19 14:10:13 +01:00
5db7070dd1
Forget room
2017-06-19 12:40:29 +01:00
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
fcf01dd88e
Reject local events that don't round trip the DB
2017-06-19 11:33:40 +01:00
4f66312df8
python_dependencies: Use bcrypt module instead of py-bcrypt
...
py-bcrypt has been unmaintained for a long while, while bcrypt is
actively maintained. And since ff8b87118d
2017-06-17 17:39:35 +02:00
dfeca6cf40
Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
...
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
6aa5bc8635
Initial worker impl
2017-06-16 11:47:11 +01:00
d8f47d2efa
Merge pull request #2280 from matrix-org/erikj/share_room_user_dir
...
Include users who you share a room with in user directory
2017-06-16 11:06:00 +01:00
1ff419d343
allow Authorization header which handling got implemented in #1098
...
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
fdf1ca30f0
Bump version and changelog
2017-06-15 12:59:06 +01:00
052c5d19d5
Merge pull request #2281 from matrix-org/erikj/phone_home_stats
...
Fix phone home stats
2017-06-15 12:46:23 +01:00
5ddd199870
Typo
2017-06-15 10:49:10 +01:00
a9d6fa8b2b
Include users who share room with requester in user directory
2017-06-15 10:17:21 +01:00
4564b05483
Implement updating users who share rooms on the fly
2017-06-15 10:17:17 +01:00
72613bc379
Implement initial population of users who share rooms table
2017-06-15 09:59:04 +01:00
ebcd55d641
Add DB schema for tracking users who share rooms
2017-06-15 09:45:48 +01:00
4b461a6931
Add some more stats
2017-06-15 09:39:39 +01:00
617304b2cf
Fix phone home stats
2017-06-14 19:47:15 +01:00
d9fd937e39
Fix user directory insertion due to missing room_id
2017-06-13 11:50:24 +01:00
505e7e8b9d
Fix up sql
2017-06-13 11:19:18 +01:00
6fd7e6db3d
Fix user dir to not assume existence of user
2017-06-13 11:11:26 +01:00
fdca6e36ee
Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined
...
Add cache for is_host_joined
2017-06-13 10:58:53 +01:00
90ae0cffec
Merge pull request #2275 from matrix-org/erikj/tweark_user_directory_search
...
Tweak the ranking of PG user dir search
2017-06-13 10:58:43 +01:00
de4cb50ca6
Merge pull request #2276 from matrix-org/erikj/fix_user_di
...
Don't assume existence of events when updating user directory
2017-06-13 10:55:41 +01:00
48d2949416
Throw exception when not retrying when downloading media
2017-06-13 10:23:14 +01:00
6ae8373d40
Don't assume existance of events when updating user directory
2017-06-13 10:19:26 +01:00
b58e24cc3c
Tweak the ranking of PG user dir search
2017-06-13 10:16:31 +01:00
d53fe399eb
Add cache for is_host_joined
2017-06-13 09:56:18 +01:00
a837765e8c
Merge pull request #2266 from matrix-org/erikj/host_in_room
...
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
8060974344
Fix replication
2017-06-09 16:40:52 +01:00
b0d975e216
Comments
2017-06-09 16:25:42 +01:00
e54d7d536e
Cache state deltas
2017-06-09 16:24:00 +01:00
1e9b4d5a95
Merge pull request #2268 from matrix-org/erikj/entity_has_changed
...
Fix has_any_entity_changed
2017-06-09 15:30:55 +01:00
efc2b7db95
Rewrite conditional
2017-06-09 13:35:15 +01:00
bfd68019c2
Merge pull request #2267 from matrix-org/erikj/missing_notifier
...
Fix removing of pushers when using workers
2017-06-09 13:07:29 +01:00
1664948e41
Comment
2017-06-09 13:05:05 +01:00
935e588799
Tweak SQL
2017-06-09 13:01:23 +01:00
eed59dcc1e
Fix has_any_entity_changed
...
Occaisonally has_any_entity_changed would throw the error: "Set changed
size during iteration" when taking the max of the `sorteddict`. While
its uncertain how that happens, its quite inefficient to iterate over
the entire dict anyway so we change to using the more traditional
`bisect_*` functions.
2017-06-09 11:44:01 +01:00
2cac7623a5
Add missing notifier
2017-06-09 11:24:41 +01:00
298d83b340
Fix replication
2017-06-09 11:01:28 +01:00
0185b75381
Change is_host_joined to use current_state table
...
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
7132e5cdff
Mark remote invite rejections as outliers
2017-06-09 10:08:18 +01:00
ea11ee09f3
Ensure we don't use unpersisted state group as prev group
2017-06-08 11:59:57 +01:00
c62c480dc6
Merge pull request #2259 from matrix-org/erikj/fix_state_woes
...
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
197bd126f0
Fix bug where state_group tables got corrupted
...
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.
We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
f45f07ab86
Merge pull request #2258 from matrix-org/erikj/user_dir
...
Don't start user_directory handling on workers
2017-06-07 14:04:50 +01:00
a053ff3979
Merge pull request #2248 from matrix-org/erikj/state_fixup
...
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
ecdd2a3658
Don't start user_directory handling on workers
2017-06-07 12:02:53 +01:00
2f34ad31ac
Add some logging to user directory
2017-06-07 11:50:44 +01:00
64ed74c01e
When pruning, delete from device_lists_outbound_last_success
2017-06-07 11:20:47 +01:00
1a81a1898e
Keep pruning background task
2017-06-07 11:16:56 +01:00
6ba21bf2b8
Comments
2017-06-07 11:08:36 +01:00
09e4bc0501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup
2017-06-07 11:05:23 +01:00
6e2a7ee1bc
Remove spurious log lines
2017-06-07 11:05:17 +01:00
65f0513a33
Split up device_lists_outbound_pokes table for faster updates.
2017-06-07 11:02:38 +01:00
6f83c4537c
Increase size of IP cache
2017-06-07 10:18:44 +01:00
cca94272fa
Fix typo when getting app name
2017-06-06 11:50:07 +01:00
66b121b2fc
Fix wrong number of arguments
2017-06-06 11:46:38 +01:00
1a01af079e
Handle profile updates in user directory
2017-06-01 15:39:51 +01:00
4d039aa2ca
Fix sqlite
2017-06-01 14:58:48 +01:00
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
d5477c7afd
Tweak search query
2017-06-01 13:28:01 +01:00
02a6108235
Tweak search query
2017-06-01 13:16:40 +01:00
7233341eac
Comments
2017-06-01 13:11:38 +01:00
8be6fd95a3
Check if host is still in room
2017-06-01 13:05:39 +01:00
59dbb47065
Remove spurious inlineCallbacks
2017-06-01 11:41:29 +01:00
9c7db2491b
Fix removing users
2017-06-01 11:36:50 +01:00
0fe6f3c521
Bug fixes and logging
...
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
036362ede6
Order by if they have profile info
2017-06-01 09:41:08 +01:00
a757dd4863
Use prefix matching
2017-06-01 09:40:37 +01:00
f5cc22bdc6
Comment on why arbitrary comments
2017-05-31 17:30:26 +01:00
5dd1b2c525
Use unique indices
2017-05-31 17:29:12 +01:00
cc7609aa9f
Comment briefly on how we keep user_directory up to date
2017-05-31 17:11:18 +01:00
f1378aef91
Convert to int
2017-05-31 17:03:08 +01:00
b2d8d07109
Lifts things into separate function
2017-05-31 17:00:24 +01:00
f9791498ae
Typos
2017-05-31 16:50:57 +01:00
f091061711
Fix tests
2017-05-31 16:34:40 +01:00
4abcff0177
Fix typo
2017-05-31 16:22:36 +01:00
63c58c2a3f
Limit number of things we fetch out of the db
2017-05-31 16:17:58 +01:00
304880d185
Add stream change cache
2017-05-31 15:46:36 +01:00
5d79d728f5
Split out directory and search tables
2017-05-31 15:23:49 +01:00
dc51af3d03
Pull max id from correct table
2017-05-31 15:13:49 +01:00
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
293ef29655
Weight differently
2017-05-31 14:29:32 +01:00
535c99f157
Use POST
2017-05-31 14:15:45 +01:00
45a5df5914
Add REST API
2017-05-31 14:11:55 +01:00
3b5f22ca40
Add search
2017-05-31 14:00:01 +01:00
b5db4ed5f6
Update room column when room becomes unpublic
2017-05-31 13:40:28 +01:00
168524543f
Add call later
2017-05-31 11:59:36 +01:00
3e123b8497
Start later
2017-05-31 11:56:27 +01:00
42137efde7
Don't go round in circles
2017-05-31 11:55:13 +01:00
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
dd48f7204c
Add comment
2017-05-30 15:01:22 +01:00
04095f7581
Add clobbered event_id
2017-05-30 14:53:01 +01:00
a584a81b3e
Add current_state_delta_stream table
2017-05-30 14:44:09 +01:00
619e8ecd0c
Handle None state group correctly
2017-05-26 10:46:03 +01:00
23da638360
Fix typing tests
2017-05-26 10:02:04 +01:00
dfbda5e025
Faster cache for get_joined_hosts
2017-05-25 17:24:44 +01:00
2b03751c3c
Don't return weird prev_group
2017-05-25 14:47:39 +01:00
dbc0dfd2d5
Remove unused options
2017-05-25 14:28:34 +01:00
11f139a647
Merge pull request #2247 from matrix-org/erikj/auth_event
...
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
6e614e9e10
Add background task to clear out old event_auth
2017-05-24 15:23:34 +01:00
c049472b8a
Only store event_auth for state events
2017-05-24 15:23:31 +01:00
9a804b2812
Merge pull request #2243 from matrix-org/matthew/fix-url-preview-length-again
...
actually trim oversize og:description meta
2017-05-23 13:26:28 +01:00
fbbc40f385
Merge pull request #2237 from matrix-org/erikj/sync_key_count
...
Add count of one time keys to sync stream
2017-05-23 11:18:13 +01:00
8cf9f0a3e7
Remove redundant invalidation
2017-05-23 09:46:59 +01:00
e6618ece2d
Missed an invalidation
2017-05-23 09:36:52 +01:00
836d5c44b6
actually trim oversize og:description meta
2017-05-22 21:14:20 +01:00
11c2a3655f
Only load jinja2 templates once
...
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
2017-05-22 17:48:58 +01:00
539aa4d333
Merge pull request #2241 from matrix-org/erikj/fix_notifs
...
Correctly calculate push rules for member events
2017-05-22 16:46:58 +01:00
f85a415279
Add missing storage function to slave store
2017-05-22 16:31:24 +01:00
6489455bed
Comment
2017-05-22 16:22:04 +01:00
d668caa79c
Remove spurious log level guards
2017-05-22 16:21:06 +01:00
74bf4ee7bf
Stream count_e2e_one_time_keys cache invalidation
2017-05-22 16:19:22 +01:00
33ba90c6e9
Merge pull request #2240 from matrix-org/erikj/cache_list_fix
...
Update list cache to handle one arg case
2017-05-22 16:10:46 +01:00
bd7bb5df71
Pull out if statement from for loop
2017-05-22 15:12:19 +01:00
e3417a06e2
Update list cache to handle one arg case
...
We update the normal cache descriptors to handle caches with a single
argument specially so that the key wasn't a 1-tuple. We need to update
the cache list to be aware of this.
2017-05-22 15:04:42 +01:00
7fb80b5eae
Check if current event is a membership event
2017-05-22 15:02:12 +01:00
2d17b09a6d
Add debug logging
2017-05-22 15:01:36 +01:00
24c8f38784
Comment
2017-05-22 14:59:27 +01:00
25f03cf8e9
Use tuple unpacking
2017-05-22 14:58:22 +01:00
270e1c904a
Speed up calculating push rules
2017-05-19 16:51:05 +01:00
b4f59c7e27
Add count of one time keys to sync stream
2017-05-19 15:47:55 +01:00
58ebb96cce
Fix invalidation of get_users_with_read_receipts_in_room
2017-05-19 14:38:50 +01:00
1c1c0257f4
Move invalidation cb to its own structure
2017-05-19 11:44:11 +01:00
cafe659f72
Store ActionGenerator in HomeServer
2017-05-19 10:09:56 +01:00
72ed8196b3
Don't push users who have left
2017-05-18 17:48:36 +01:00
107ac7ac96
Increase size of push rule caches
2017-05-18 17:17:53 +01:00
760625acba
Make get_if_app_services_interested_in_user faster
2017-05-18 16:34:44 +01:00
c57789d138
Remove size of push get_rules cache
2017-05-18 16:17:23 +01:00
f33df30732
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-05-18 13:56:37 +01:00
a5425b2e5b
Bump changelog and version
2017-05-18 13:53:48 +01:00
6e381180ae
Merge pull request #2177 from matrix-org/erikj/faster_push_rules
...
Make calculating push actions faster
2017-05-18 11:46:18 +01:00
056ba9b795
Add comment
2017-05-18 11:45:56 +01:00
d9e3a4b5db
Merge pull request #2230 from matrix-org/erikj/speed_up_get_state
...
Make get_state_groups_from_groups faster.
2017-05-17 17:23:04 +01:00
66d8ffabbd
Faster push rule calculation via push specific cache
...
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
ace23463c5
Merge pull request #2216 from slipeer/app_services_interested_in_user
...
Fix users claimed non-exclusively by an app service don't get notific…
2017-05-17 16:28:50 +01:00
bbfe4e996c
Make get_state_groups_from_groups faster.
...
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.
By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
2017-05-17 15:12:15 +01:00
a8bc7cae56
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.21.0
2017-05-17 13:11:43 +01:00
3a431a126d
Bump changelog and version
2017-05-17 11:26:57 +01:00
85e8092cca
Comment
2017-05-17 10:03:09 +01:00
ad53fc3cf4
Short circuit when we have delta ids
2017-05-17 09:57:34 +01:00
6fa8148ccb
Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts
...
Speed up get_joined_hosts
2017-05-16 17:40:55 +01:00
7c69849a0d
Merge pull request #2227 from matrix-org/erikj/presence_caches
...
Make presence use cached users/hosts in room
2017-05-16 17:40:47 +01:00
13f540ef1b
Speed up get_joined_hosts
2017-05-16 16:05:22 +01:00
ec5c4499f4
Make presence use cached users/hosts in room
2017-05-16 16:01:43 +01:00
f2a5b6dbfd
Speed up get_domain_from_id
2017-05-16 15:59:37 +01:00
b8492b6c2f
Merge pull request #2224 from matrix-org/erikj/prefill_state
...
Prefill state caches
2017-05-16 15:50:11 +01:00
331570ea6f
Remove spurious merge artifacts
2017-05-16 15:33:07 +01:00
55af207321
Merge branch 'develop' into avoid_duplicate_filters
2017-05-16 15:29:59 +02:00
d648f65aaf
Merge pull request #2218 from matrix-org/rav/event_search_index
...
Add an index to event_search
2017-05-16 13:26:07 +01:00
608b5a6317
Take a copy before prefilling, as it may be a frozendict
2017-05-16 12:55:29 +01:00
64953c8ed2
avoid access-error if no filter_id matches
2017-05-15 18:36:37 +02:00
f451b64c8f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state
2017-05-15 16:09:32 +01:00
2c9475b58e
Merge pull request #2221 from psaavedra/sync_timeline_limit_filter_by_name
...
Configurable maximum number of events requested by /sync and /messages
2017-05-15 16:08:46 +01:00
d12ae7fd1c
Don't log exceptions for NotRetryingDestination
2017-05-15 15:42:18 +01:00
224137fcf9
Fixed syntax nits
2017-05-15 16:21:02 +02:00
e4435b014e
Update comment
2017-05-15 15:11:30 +01:00
871605f4e2
Comments
2017-05-15 15:11:30 +01:00
e0d2f6d5b0
Add more granular event send metrics
2017-05-15 15:11:30 +01:00
bfbc907cec
Prefill state caches
2017-05-15 15:11:13 +01:00
627e6ea2b0
Fixed implementation errors
...
* Added HS as property in SyncRestServlet
* Fixed set_timeline_upper_limit function implementat¡ion
2017-05-15 14:51:43 +02:00
9da4316ca5
Configurable maximum number of events requested by /sync and /messages ( #2220 )
...
Set the limit on the returned events in the timeline in the get and sync
operations. The default value is -1, means no upper limit.
For example, using `filter_timeline_limit: 5000`:
POST /_matrix/client/r0/user/user:id/filter
{
room: {
timeline: {
limit: 1000000000000000000
}
}
}
GET /_matrix/client/r0/user/user:id/filter/filter:id
{
room: {
timeline: {
limit: 5000
}
}
}
The server cuts down the room.timeline.limit.
2017-05-13 18:17:54 +02:00
eb7cbf27bc
insert whitespace to fix travis build
2017-05-12 12:09:42 +02:00
6b95e35e96
add check to only add a new filter if the same filter does not exist previously
...
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-05-11 16:05:30 +02:00
ff3d810ea8
Add a comment to old delta
2017-05-11 12:48:50 +01:00
34194aaff7
Don't create event_search index on sqlite
...
... because the table is virtual
2017-05-11 12:46:55 +01:00
114f290947
Add more logging for purging
...
Log the number of events we will be deleting at info.
2017-05-11 12:08:47 +01:00
baafb85ba4
Add an index to event_search
...
- to make the purge API quicker
2017-05-11 12:05:22 +01:00
29ded770b1
Merge pull request #2214 from matrix-org/rav/hurry_up_purge
...
When purging, don't de-delta state groups we're about to delete
2017-05-11 12:04:25 +01:00
dc026bb16f
Tidy purge code and add some comments
...
Try to make this clearer with more comments and some variable renames
2017-05-11 10:56:12 +01:00
328378f9cb
Fix users claimed non-exclusively by an app service don't get notifications #2211
2017-05-11 11:42:08 +03:00
8e345ce465
Don't de-delta state groups we're about to delete
2017-05-10 18:44:22 +01:00
b64d312421
add some logging to purge_history
2017-05-10 18:44:22 +01:00
ccad2ed824
Modify condition on empty localpart
2017-05-10 17:34:30 +01:00
369195caa5
Modify register/available to be GET with query param
...
- GET is now the method for register/available
- a query parameter "username" is now used
Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
a3648f84b2
Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid
...
Add per user ratelimiting overrides
2017-05-10 15:54:48 +01:00
5331cd150a
Merge pull request #2206 from matrix-org/rav/one_time_key_upload_change_sig
...
Allow clients to upload one-time-keys with new sigs
2017-05-10 14:18:40 +01:00
f7278e612e
Change register/available to POST (from GET)
2017-05-10 11:40:18 +01:00
b990b2fce5
Add per user ratelimiting overrides
2017-05-10 11:05:43 +01:00
aedaba018f
Replace some instances of preserve_context_over_deferred
2017-05-09 19:04:56 +01:00
de042b3b88
Do some logging when one-time-keys get claimed
...
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
a7e9d8762d
Allow clients to upload one-time-keys with new sigs
...
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).
Fixes https://github.com/vector-im/riot-android/issues/1208 , hopefully.
2017-05-09 19:04:56 +01:00
ca238bc023
Merge branch 'release-v0.21.0' of github.com:matrix-org/synapse into develop
2017-05-08 17:35:11 +01:00
40dcf0d856
Merge pull request #2203 from matrix-org/erikj/event_cache_hit_ratio
...
Don't update event cache hit ratio from get_joined_users
2017-05-08 16:52:39 +01:00
d3c3026496
Merge pull request #2201 from matrix-org/erikj/store_device_cache
...
Cache check to see if device exists
2017-05-08 16:23:04 +01:00
093f7e47cc
Expand docstring a bit
2017-05-08 16:14:46 +01:00
6a12998a83
Add missing yields
2017-05-08 16:10:51 +01:00
ffad4fe35b
Don't update event cache hit ratio from get_joined_users
...
Otherwise the hit ration of plain get_events gets completely skewed by
calls to get_joined_users* functions.
2017-05-08 16:06:17 +01:00
94e6ad71f5
Invalidate cache on device deletion
2017-05-08 15:55:59 +01:00
8571f864d2
Cache one time key counts
2017-05-08 15:34:27 +01:00
fc6d4974a6
Comment
2017-05-08 15:33:57 +01:00
738ccf61c0
Cache check to see if device exists
2017-05-08 15:32:18 +01:00
dcabef952c
Increase client_ip cache size
2017-05-08 15:09:19 +01:00
771c8a83c7
Bump version and changelog
2017-05-08 13:23:46 +01:00
6631985990
Merge pull request #2200 from matrix-org/erikj/revert_push
...
Revert speed up push
2017-05-08 13:20:52 +01:00
e0f20e9425
Revert "Remove unused import"
...
This reverts commit ab37bef83b
2017-05-08 13:07:43 +01:00
fe7c1b969c
Revert "We don't care about forgotten rooms"
...
This reverts commit ad8b316939
2017-05-08 13:07:43 +01:00
78f306a6f7
Revert "Speed up filtering of a single event in push"
...
This reverts commit 421fdf7460
2017-05-08 13:07:41 +01:00
9ac98197bb
Bump version and changelog
2017-05-08 11:07:54 +01:00
27c28eaa27
Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more
...
Always mark remotes as up if we receive a signed request from them
2017-05-05 14:08:12 +01:00
be2672716d
Merge pull request #2189 from matrix-org/erikj/handle_remote_device_list
...
Handle exceptions thrown in handling remote device list updates
2017-05-05 14:01:27 +01:00
653d90c1a5
Comment
2017-05-05 14:01:17 +01:00
310b1ccdc1
Use preserve_fn and add logs
2017-05-05 13:41:19 +01:00
a59b0ad1a1
Merge pull request #2192 from matrix-org/kegan/simple-http-client-timeouts
...
Rewrite SimpleHttpClient.request to include timeouts
2017-05-05 11:52:43 +01:00
7b222fc56e
Remove redundant reset of destination timers
2017-05-05 11:14:09 +01:00
d0debb2116
Remember how twisted works
2017-05-05 11:00:21 +01:00
66f371e8b8
Merge pull request #2176 from matrix-org/erikj/faster_get_joined
...
Make get_joined_users faster
2017-05-05 10:59:55 +01:00
b843631d71
Add comment and TODO
2017-05-05 10:59:32 +01:00
c2ddd773bc
Include the clock
2017-05-05 10:52:46 +01:00
7dd3bf5e24
Rewrite SimpleHttpClient.request to include timeouts
...
Fixes #2191
2017-05-05 10:49:19 +01:00
db7d0c3127
Always mark remotes as up if we receive a signed request from them
2017-05-05 10:34:53 +01:00
f346048a6e
Handle exceptions thrown in handling remote device list updates
2017-05-05 10:34:10 +01:00
e3aa8a7aa8
Merge pull request #2185 from matrix-org/erikj/smaller_caches
...
Optimise caches for single key
2017-05-05 10:19:05 +01:00
cf589f2c1e
Fixes
2017-05-05 10:17:56 +01:00
8af4569583
Merge pull request #2174 from matrix-org/erikj/current_cache_hosts
...
Add cache for get_current_hosts_in_room
2017-05-05 10:15:24 +01:00
587f07543f
Revert "Prefill state caches"
2017-05-04 15:07:27 +01:00
aa93cb9f44
Add comment
2017-05-04 14:59:28 +01:00
537dbadea0
Intern host strings
2017-05-04 14:55:28 +01:00
07a07588a0
Make caches bigger
2017-05-04 14:52:28 +01:00
dfaa58f72d
Fix comment and num args
2017-05-04 14:50:24 +01:00
9ac263ed1b
Add new storage functions to slave store
2017-05-04 14:29:03 +01:00
d2d8ed4884
Optimise caches with single key
2017-05-04 14:18:46 +01:00
5d8290429c
Reduce size of get_users_in_room
2017-05-04 13:43:19 +01:00
6aa423a1a8
Merge pull request #2183 from matrix-org/luke/username-availability
...
Implement username availability checker
2017-05-04 09:58:40 +01:00
3669065466
Appease the flake8 gods
2017-05-03 18:05:49 +01:00
7ebf518c02
Make get_joined_users faster
2017-05-03 15:55:54 +01:00
34ed4f4206
Implement username availability checker
...
Outlined here: https://github.com/vector-im/riot-web/issues/3605#issuecomment-298679388
```HTTP
GET /_matrix/.../register/available
{
"username": "desiredlocalpart123"
}
```
If available, the response looks like
```HTTP
HTTP/1.1 200 OK
{
"available": true
}
```
Otherwise,
```HTTP
HTTP/1.1 429
{
"errcode": "M_LIMIT_EXCEEDED",
"error": "Too Many Requests",
"retry_after_ms": 2000
}
```
or
```HTTP
HTTP/1.1 400
{
"errcode": "M_USER_IN_USE",
"error": "User ID already taken."
}
```
or
```HTTP
HTTP/1.1 400
{
"errcode": "M_INVALID_USERNAME",
"error": "Some reason for username being invalid"
}
```
2017-05-03 12:04:12 +01:00
60833c8978
Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error
...
Propagate errors sensibly from proxied IS requests
2017-05-03 11:23:25 +01:00
482a2ad122
No need for the exception variable
2017-05-03 11:02:59 +01:00
c0380402bc
List caught expection types
2017-05-03 10:56:22 +01:00
cdbf38728d
Merge pull request #2175 from matrix-org/erikj/prefill_state
...
Prefill state caches
2017-05-03 10:54:11 +01:00
0c27383dd7
Merge pull request #2170 from matrix-org/erikj/fed_hole_state
...
Don't fetch state for missing events that we fetched
2017-05-03 10:49:37 +01:00
ef862186dd
Merge together redundant calculations/logging
2017-05-03 10:06:43 +01:00
2c2dcf81d0
Update comment
2017-05-03 10:00:29 +01:00
1827057acc
Comments
2017-05-03 09:56:05 +01:00
8346e6e696
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state
2017-05-03 09:46:40 +01:00
e4c15fcb5c
Merge pull request #2178 from matrix-org/erikj/message_metrics
...
Add more granular event send metrics
2017-05-02 17:57:34 +01:00
3e5a62ecd8
Add more granular event send metrics
2017-05-02 14:23:26 +01:00
82475a18d9
Merge pull request #2180 from matrix-org/rav/fix_timeout_on_timeout
...
Instantiate DeferredTimedOutError correctly
2017-05-02 13:32:58 +01:00
2e996271fe
Instantiate DeferredTimedOutError correctly
...
Call `super` correctly, so that we correctly initialise the `errcode` field.
Fixes https://github.com/matrix-org/synapse/issues/2179 .
2017-05-02 13:26:17 +01:00
a2c89a225c
Prefill state caches
2017-05-02 10:40:31 +01:00
7166854f41
Add cache for get_current_hosts_in_room
2017-05-02 10:36:35 +01:00
3033261891
Merge pull request #2080 from matrix-org/erikj/filter_speed
...
Speed up filtering of a single event in push
2017-04-28 14:17:13 +01:00
2347efc065
Fixup
2017-04-28 12:46:53 +01:00
9b147cd730
Remove unncessary call in _get_missing_events_for_pdu
2017-04-28 11:55:25 +01:00
3a9f5bf6dd
Don't fetch state for missing events that we fetched
2017-04-28 11:26:46 +01:00
ab37bef83b
Remove unused import
2017-04-28 09:57:23 +01:00
ad8b316939
We don't care about forgotten rooms
2017-04-28 09:52:36 +01:00
421fdf7460
Speed up filtering of a single event in push
2017-04-28 09:52:36 +01:00
25a96e0c63
Merge pull request #2163 from matrix-org/erikj/fix_invite_state
...
Fix invite state to always include all events
2017-04-27 17:36:30 +01:00
46826bb078
Comment and remove spurious logging
2017-04-27 17:25:44 +01:00
f87b287291
Merge pull request #2127 from APwhitehat/alreadystarted
...
print something legible if synapse already running
2017-04-27 15:46:53 +01:00
bb9246e525
Merge pull request #2131 from matthewjwolff/develop
...
web_client_location documentation fix
2017-04-27 15:46:40 +01:00
c84770b877
Fix bgupdate error if index already exists ( #2167 )
...
When creating a new table index in the background, guard against it existing already. Fixes
https://github.com/matrix-org/synapse/issues/2135 .
Also, make sure we restore the autocommit flag when we're done, otherwise we
get more failures from other operations later on. Fixes
https://github.com/matrix-org/synapse/issues/1890 (hopefully).
2017-04-27 15:27:48 +01:00
380fb87ecc
Merge pull request #2168 from matrix-org/erikj/federation_logging
...
Add some extra logging for edge cases of federation
2017-04-27 15:19:12 +01:00
87ae59f5e9
Typo
2017-04-27 15:16:21 +01:00
e42b4ebf0f
Add some extra logging for edge cases of federation
2017-04-27 14:38:21 +01:00
d3c150411c
Merge pull request #2130 from APwhitehat/roomexists
...
Check that requested room_id exists
2017-04-27 09:20:26 +01:00
1e166470ab
Fix tests
2017-04-26 16:23:30 +01:00
34e682d385
Fix invite state to always include all events
2017-04-26 16:18:08 +01:00
5fd12dce01
Remove debugging
2017-04-26 12:36:26 +01:00
82ae0238f9
Revert accidental commit
2017-04-26 11:43:16 +01:00
81804909d3
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-26 11:31:55 +01:00
c366276056
Fix get_json
2017-04-26 10:07:01 +01:00
1a9255c12e
Use CodeMessageException subclass instead
...
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
f053a1409e
Make state caches cache in ascii
2017-04-25 17:22:55 +01:00
acb58bfb6a
fix up
2017-04-25 15:39:19 +01:00
f7181615f2
Don't specify default as dict
2017-04-25 15:22:59 +01:00
f144365281
Comment
2017-04-25 15:18:26 +01:00
d9aa645f86
Reduce size of joined_user cache
...
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.
We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
22f3d3ae76
Reduce _get_state_group_for_event cache size
2017-04-25 11:43:03 +01:00
b4da08cad8
Merge pull request #2158 from matrix-org/erikj/reduce_cache_size
...
Reduce cache size by not storing deferreds
2017-04-25 11:18:07 +01:00
efab1dadde
Remove DEBUG_CACHES
2017-04-25 10:54:09 +01:00
119cb9bbcf
Reduce cache size by not storing deferreds
...
Currently the cache descriptors store deferreds rather than raw values,
this is a simple way of triggering only one database hit and sharing the
result if two callers attempt to get the same value.
However, there are a few caches that simply store a mapping from string
to string (or int). These caches can have a large number of entries,
under the assumption that each entry is small. However, the size of a
deferred (specifically the size of ObservableDeferred) is signigicantly
larger than that of the raw value, 2kb vs 32b.
This PR therefore changes the cache descriptors to store the raw values
rather than the deferreds.
As a side effect cached storage function now either return a deferred or
the actual value, as the cached list decriptor already does. This is
fine as we always end up just yield'ing on the returned value
eventually, which handles that case correctly.
2017-04-25 10:23:11 +01:00
e6e2627636
Fix code for reporting old verify keys in synapse
2017-04-24 18:51:25 +01:00
30f7bfa121
Merge pull request #2145 from matrix-org/rav/reject_invite_to_unreachable_server
...
Fix rejection of invites to unreachable servers
2017-04-24 15:20:52 +01:00
7af825bae4
Merge pull request #2155 from matrix-org/erikj/string_intern
...
Only intern ascii strings
2017-04-24 14:28:41 +01:00
d134d0935e
Only intern ascii strings
2017-04-24 14:07:48 +01:00
e4f3431116
Remove unused cache
2017-04-24 13:27:38 +01:00
a46982cee9
Need the HTTP status code
2017-04-21 16:20:12 +01:00
70caf49914
Do the same for get_json
2017-04-21 16:09:03 +01:00
a1595cec78
Don't error for 3xx responses
2017-04-21 11:51:17 +01:00
2e165295b7
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-21 11:35:52 +01:00
a90a0f5c8a
Propagate errors sensibly from proxied IS requests
...
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
2017-04-21 11:32:48 +01:00
91b3981800
Try harder when sending leave events
...
When we're rejecting invites, ignore the backoff data, so that we have a better
chance of not getting the room out of sync.
2017-04-21 01:50:36 +01:00
0cdb32fc43
Remove redundant try/except clauses
...
The `except SynapseError` clauses were pointless because the wrapped functions
would never throw a `SynapseError` (they either throw a `CodeMessageException`
or a `RuntimeError`).
The `except CodeMessageException` is now also pointless because the caller
treats all exceptions equally, so we may as well just throw the
`CodeMessageException`.
2017-04-21 01:32:01 +01:00
838810b76a
Broaden the conditions for locally_rejecting invites
...
The logic for marking invites as locally rejected was all well and good, but
didn't happen when the remote server returned a 500, or wasn't reachable, or
had no DNS, or whatever.
Just expand the except clause to catch everything.
Fixes https://github.com/matrix-org/synapse/issues/761 .
2017-04-21 01:31:37 +01:00
736b9a4784
Remove redundant function
...
inline `reject_remote_invite`, which only existed to make tracing the callflow
more difficult.
2017-04-21 01:31:09 +01:00
4903ccf159
Fix some lies, and other clarifications, in docstrings
...
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
d4040e9e28
Queried CONDITIONAL_REQUIREMENTS
2017-04-18 16:19:48 -05:00
3fb8784c92
m.read_marker -> m.fully_read ( #2128 )
...
Also:
- change the REST endpoint to have a "S" on the end (so it's now /read_markers)
- change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
54f9a4cb59
Fixed travis build failure
...
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
2017-04-17 01:38:27 -05:00
8e780b113d
web_server_root documentation fix
...
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
2017-04-17 00:49:11 -05:00
574d573ac2
Check that requested room_id exists
2017-04-14 23:50:59 +05:30
78f0ddbfad
Merge pull request #2120 from matrix-org/luke/read-markers
...
Implement Read Marker API
2017-04-13 14:21:31 +01:00
6a70647d45
Correct logic in is_event_after
2017-04-13 13:46:17 +01:00
c1f52a321d
synctl.py: Check if synapse is already running
2017-04-13 18:00:02 +05:30
b9557064bf
Simplify is_event_after logic
2017-04-12 14:36:20 +01:00
cf6121e3da
More null-guard changes
2017-04-12 14:02:03 +01:00
247c736b9b
Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
...
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
8fbc0d29ee
Merge pull request #2121 from matrix-org/paul/sent-transactions-metric
...
Add a counter metric for successfully-sent transactions
2017-04-12 11:04:31 +01:00
c06c00190f
Merge pull request #2116 from matrix-org/erikj/dedupe_federation_repl2
...
Dedupe KeyedEdu and Devices federation repl traffic
2017-04-12 10:57:24 +01:00
c0aba0a23e
Remove Unused ref to hs
2017-04-12 10:52:11 +01:00
b9676a75f6
Move a space
2017-04-12 10:51:17 +01:00
69a18514e9
Only notify user, not entire room
2017-04-12 10:50:37 +01:00
122cd52ce4
Remove comment, simplify null-guard
2017-04-12 10:48:32 +01:00
26ae5178a4
Add some comments
2017-04-12 10:36:29 +01:00
bf9060156a
Merge pull request #2117 from matrix-org/erikj/remove_http_replication
...
Remove HTTP replication APIs
2017-04-12 10:21:42 +01:00
1745069543
Comment
2017-04-12 10:17:10 +01:00
c7ddb5ef7a
Reuse get_interested_parties
2017-04-12 10:16:26 +01:00
7b41013102
Merge pull request #2118 from matrix-org/erikj/no_devices
...
Fix getting latest device IP for user with no devices
2017-04-12 10:14:32 +01:00
77fb2b72ae
Handle no previous RM
2017-04-12 09:47:29 +01:00
7f94709066
travis flake8..
2017-04-11 18:35:45 +01:00
867822fa1e
flake8
2017-04-11 17:36:04 +01:00
73880268ef
Refactor event ordering check to events store
2017-04-11 17:34:09 +01:00
131485ef66
Copyright
2017-04-11 17:33:51 +01:00
11dbceb761
Add a counter metric for successfully-sent transactions
2017-04-11 17:16:12 +01:00
0127423027
flake8
2017-04-11 17:07:07 +01:00
85657eedf8
Bail on where clause instead
2017-04-11 16:24:31 +01:00
b48045a8f5
Don't bother with outer check for now
2017-04-11 16:23:24 +01:00
9c712a366f
Move get_presence_list_* to SlaveStore
2017-04-11 16:07:33 +01:00
a8c8e4efd4
Comment
2017-04-11 15:35:49 +01:00
414522aed5
Move get_interested_parties
2017-04-11 15:33:26 +01:00
2be8a281d2
Comments
2017-04-11 15:28:24 +01:00
6308ac45b0
Move get_interested_remotes back to presence handler
2017-04-11 15:19:26 +01:00
b9b72bc6e2
Comments
2017-04-11 15:15:34 +01:00
d892079844
Finish implementing RM endpoint
...
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
- This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
e263c26690
Initial commit of RM server-side impl
...
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u )
2017-04-11 11:55:30 +01:00
f3cf3ff8b6
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-04-11 11:13:32 +01:00
34840cdcef
Fix getting latest device IP for user with no devices
2017-04-11 09:56:54 +01:00
28a4649785
Remove HTTP replication APIs
2017-04-11 09:52:11 +01:00
7c551ec445
trust a hypothetical future riot.im IS
2017-04-10 17:58:36 +01:00
84fbb80c8f
Use generators
2017-04-10 16:55:56 +01:00
40453b3f84
Dedupe KeyedEdu and Devices federation repl traffic
2017-04-10 16:49:51 +01:00
29574fd5b3
Reduce federation presence replication traffic
...
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
2e6f5a4910
Typo
2017-04-10 16:17:40 +01:00
405ba4178a
Merge pull request #2102 from DanielDent/add-auth-email
...
Support authenticated SMTP
2017-04-10 15:42:16 +01:00
efcb6db688
Merge pull request #2109 from matrix-org/erikj/send_queue_fix
...
Fix up federation SendQueue and document types
2017-04-10 13:09:25 +01:00
0018491af2
Rename variable
2017-04-10 12:44:43 +01:00
0364d23210
Up replication ping timeout
2017-04-10 11:32:05 +01:00
8c5f03cec7
Revert to sending the same data type as before
2017-04-10 10:07:18 +01:00
f8434db549
Change name
2017-04-10 10:03:07 +01:00
ab904caf33
Comments
2017-04-10 10:02:17 +01:00
64765e5199
When we do an invite rejection, save the signed leave event to the db
...
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.
We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
2017-04-07 14:39:32 +01:00
2a3e822f44
Comment
2017-04-07 13:47:04 +01:00
a828a64b75
Comment
2017-04-07 11:54:03 +01:00
d4d176e5d0
Add logging
2017-04-07 11:51:28 +01:00
449d1297ca
Fix up federation SendQueue and document types
2017-04-07 11:48:33 +01:00
d72667fcce
Speed up get_current_state_ids
...
Using _simple_select_list is fairly expensive for functions that return
a lot of rows and/or get called a lot. (This is because it carefully
constructs a list of dicts).
get_current_state_ids gets called a lot on startup and e.g. when the IRC
bridge decided to send tonnes of joins/leaves (as it invalidates the
cache). We therefore replace it with a custon txn function that builds
up the final result dict without building up and intermediate
representation.
2017-04-07 10:10:49 +01:00
a41fe500d6
Bump version and changelog
2017-04-07 10:03:48 +01:00
54f59bd7d4
Merge pull request #2107 from HarHarLinks/patch-1
...
fix typo in synctl help
2017-04-07 09:54:37 +01:00
98ce212093
Merge pull request #2103 from matrix-org/erikj/no-double-encode
...
Don't double encode replication data
2017-04-07 09:39:52 +01:00
8a1137ceab
fix typo in synctl help
2017-04-06 17:10:20 +02:00
877c029c16
Use iteritems
2017-04-06 15:51:22 +01:00
944692ef69
Merge pull request #2106 from matrix-org/erikj/reduce_user_sync
...
Reduce rate of USER_SYNC repl commands
2017-04-06 13:35:31 +01:00
391712a4f9
Comment
2017-04-06 13:35:00 +01:00
ad544c803a
Document types of the replication streams
2017-04-06 13:28:52 +01:00
dbf87282d3
Docs
2017-04-06 13:11:21 +01:00
69b3fd485d
Fix incorrect type when using InvalidateCacheCommand
2017-04-06 09:36:38 +01:00
5058292537
Support authenticated SMTP
...
Closes (SYN-714) #1385
Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>
2017-04-05 21:01:08 -07:00
fcc803b2bf
Add log lines
2017-04-05 17:13:44 +01:00
3f213d908d
Rearrange metrics
2017-04-05 14:15:09 +01:00
1ca0e78ca1
Fix typo
2017-04-05 13:43:39 +01:00
b43d3267e2
Fixup some metrics for tcp repl
2017-04-05 13:34:54 +01:00
b5cb6347a4
Don't immediately notify the master about users whose syncs have gone away
2017-04-05 13:25:40 +01:00
96b9b6c127
Don't double json encode typing replication data
2017-04-05 11:34:20 +01:00
f10ce8944b
Don't double json encode federation replication data
2017-04-05 11:10:28 +01:00
a5c401bd12
Merge pull request #2097 from matrix-org/erikj/repl_tcp_client
...
Move to using TCP replication
2017-04-05 09:36:21 +01:00
d1d5362267
Add comment
2017-04-04 16:41:03 +01:00
9f26d3b75b
Deduplicate new deviceinbox rows for replication
2017-04-04 16:21:21 +01:00
a76886726b
Merge pull request #2098 from matrix-org/erikj/repl_tcp_fix
...
Advance replication streams even if nothing is listening
2017-04-04 15:40:51 +01:00
ac66e11f2b
Add the appropriate amount of preserve_fn
2017-04-04 15:22:54 +01:00
4264ceb31c
Fiddle tcp replication logging
2017-04-04 14:14:03 +01:00
023ee197be
Advance replication streams even if nothing is listening
...
Otherwise the streams don't advance and steadily fall behind, so when a
worker does connect either a) they'll be streamed lots of old updates or
b) the connection will fail as the streams are too far behind.
2017-04-04 13:19:26 +01:00
d1605794ad
Remove unused worker config option
2017-04-04 11:17:00 +01:00
3376f16012
Shuffle and comment synchrotron presence
2017-04-04 11:14:16 +01:00
6ce6bbedcb
Move where we ack federation
2017-04-04 11:02:44 +01:00
62b89daac6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server
2017-04-04 09:46:16 +01:00
773e64cc1a
Merge pull request #2095 from matrix-org/rav/cull_log_preserves
...
Cull spurious PreserveLoggingContexts
2017-04-03 17:02:25 +01:00
2d05eb3cf5
Merge remote-tracking branch 'origin/release-v0.20.0' into develop
2017-04-03 16:23:23 +01:00
30bcbf775a
Accept join events from all servers
...
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.
(Fixes #1893 ).
2017-04-03 15:58:07 +01:00
7eb9f34cc3
Remove spurious yield
...
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
2017-04-03 15:44:19 +01:00
0b08c48fc5
Remove more spurious `PreserveLoggingContext`s
...
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
2017-04-03 15:43:37 +01:00
65e1683680
Remove spurious PreserveLoggingContext
...
In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its
subroutines with the logcontext set.
2017-04-03 15:42:38 +01:00
feb496056e
preserve_fn some deferred-returning things
...
In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which
return deferreds, so that it is safe to call it with an active logcontext.
2017-04-03 15:41:17 +01:00
e2eebf1696
Fix fixme in preserve_fn
...
`preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a
fixme therein.
2017-04-03 15:38:02 +01:00
36c28bc467
Update all the workers and master to use TCP replication
2017-04-03 15:35:52 +01:00
3a1f3f8388
Change slave storage to use new replication interface
...
As the TCP replication uses a slightly different API and streams than
the HTTP replication.
This breaks HTTP replication.
2017-04-03 15:34:19 +01:00
52bfa604e1
Add basic replication client handler and factory
2017-04-03 15:34:13 +01:00
0a6a966e2b
Always advance stream tokens
2017-04-03 15:22:56 +01:00
773e1c6d68
Remove spurious @preserve_fn decorators
...
Remove `@preserve_fn` decorators on `on_new_room_event`,
`_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and
`on_new_replication_data` - none of these functions return a deferred, and the
decorator does nothing unless the wrapped function returns a deferred, so the
decorator was a no-op.
2017-04-03 15:14:11 +01:00
0d1c85e643
Merge branch 'release-v0.20.0' of github.com:matrix-org/synapse into develop
2017-04-03 14:58:14 +01:00
1df7c28661
Use callbacks to notify tcp replication rather than deferreds
2017-03-31 15:42:51 +01:00
36d2b66f90
Add a timestamp to USER_SYNC command
...
This timestamp is used to indicate when the user last sync'd
2017-03-31 15:42:22 +01:00
8a240e4f9c
Merge pull request #2078 from APwhitehat/assertuserfriendly
...
add user friendly report of assertion error in synctl.py
2017-03-31 14:41:49 +01:00
142b6b4abf
Merge pull request #2011 from matrix-org/matthew/turn_allow_guests
...
add setting (on by default) to support TURN for guests
2017-03-31 14:37:09 +01:00
2a06b44be2
Merge pull request #1986 from matrix-org/matthew/enable_guest_3p
...
enable guest access for the 3pl/3pid APIs
2017-03-31 14:36:03 +01:00
9ee397b440
switch to allow_guest=True for authing 3Ps as per PR feedback
2017-03-31 13:54:26 +01:00
9d0170ac6c
Fix up presence
2017-03-31 11:36:32 +01:00
bfcf016714
Fix up docs
2017-03-31 11:19:24 +01:00
9cee0ce7db
Merge pull request #2075 from matrix-org/erikj/cache_speed
...
Speed up cached function access
2017-03-31 10:10:56 +01:00
350333a09a
Merge pull request #2076 from matrix-org/erikj/as_perf
...
Make AS's faster
2017-03-31 09:43:10 +01:00
4d17add8de
Remove unused instance variable
2017-03-31 09:38:27 +01:00
27b1b4a2c9
Speed up copy_and_replace
2017-03-30 17:50:31 +01:00
5b5b171f3e
Docs
2017-03-30 17:05:53 +01:00
b282fe7170
Revert log context change
2017-03-30 17:03:59 +01:00
9ff4e0e91b
Bump version and changelog
2017-03-30 16:37:40 +01:00
0834d1a70c
Merge pull request #2079 from matrix-org/erikj/push_regex_cache
...
Cache glob to regex at a higher level for push
2017-03-30 16:28:18 +01:00
63fcc42990
Remove user from process_presence when stops syncing
2017-03-30 14:26:08 +01:00
6194a64ae9
Doc new instance variables
2017-03-30 14:19:10 +01:00
014fee93b3
Manually calculate cache key as getcallargs is expensive
...
This is because getcallargs recomputes the getargspec, amongst other
things, which we don't need to do as its already been done
2017-03-30 14:14:46 +01:00
86780a8bc3
Don't convert to deferreds when not necessary
2017-03-30 14:14:36 +01:00
3ba2859e0c
Add tcp replication listener type and hook it up
2017-03-30 13:31:10 +01:00
e9dd8370b0
Add functions to presence to support remote syncs
...
The TCP replication protocol streams deltas of who has started or
stopped syncing. This is different from the HTTP API which periodically
sends the full list of users who are syncing. This commit adds support
for the new TCP style of sending deltas.
2017-03-30 13:25:14 +01:00
4d7fc7f977
Add server side resource for tcp replication
2017-03-30 13:24:45 +01:00
f9b4bb05e0
Fix the logcontext handling in the cache wrappers ( #2077 )
...
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
2017-03-30 13:22:24 +01:00
7450693435
Initial TCP protocol implementation
...
This defines the low level TCP replication protocol
2017-03-30 12:54:46 +01:00
8da6f0be48
Define the various streams we will replicate
2017-03-30 12:54:46 +01:00
11880103b1
Make federation send queue take the current position
2017-03-30 12:54:36 +01:00
7984708a55
Add a simple hook to wait for replication traffic
2017-03-30 11:57:52 +01:00
24d35ab47b
Add new storage functions for new replication
...
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
2017-03-30 11:48:35 +01:00
30348c924c
Use txn.fetchall() so we can reuse txn
2017-03-30 10:30:05 +01:00
6cdca71079
synctl.py: wait for synapse to stop before restarting ( #2020 )
2017-03-29 19:20:13 +01:00
305d16d612
add user friendly report of assertion error in synctl.py
...
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2017-03-29 20:41:39 +05:30
a3810136fe
Cache glob to regex at a higher level for push
2017-03-29 15:53:14 +01:00
3ce8d59176
Increase cache size for _get_state_group_for_event
2017-03-29 14:31:46 +01:00
b9c2ae6788
Merge pull request #1849 from matrix-org/erikj/state_typo
...
Fix bug where current_state_events renamed to current_state_ids
2017-03-29 11:53:42 +01:00
85be3dde81
Bail early if remote wouldn't be retried ( #2064 )
...
* Bail early if remote wouldn't be retried
* Don't always return true
* Just use get_retry_limiter
* Spelling
2017-03-29 11:48:27 +01:00
2f8b580b64
Merge pull request #2053 from matrix-org/erikj/e2e_one_time_upsert
...
Don't user upsert to persist new one time keys
2017-03-29 11:44:23 +01:00
c5b0bdd542
Merge pull request #2067 from matrix-org/erikj/notify_on_fed
...
Notify on new federation traffic
2017-03-29 11:41:37 +01:00
e4df0e189d
Decrank last commit
2017-03-29 11:02:35 +01:00
4ad613f6be
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/e2e_one_time_upsert
2017-03-29 10:57:19 +01:00
ac6bc55512
Correctly look up key
2017-03-29 10:56:26 +01:00
69efd77749
Add comment
2017-03-29 09:50:05 +01:00
51b156d48a
Cache whether an AS is interested based on members
2017-03-28 13:27:21 +01:00
30f5ffdca2
Remove param and cast at call site
2017-03-28 13:27:21 +01:00
650f0e69f2
Compile the regex's used in ASes
2017-03-28 13:27:21 +01:00
d28db583da
Merge pull request #2063 from matrix-org/erikj/device_list_batch
...
Batch sending of device list pokes
2017-03-28 11:35:41 +01:00
58a35366be
The algorithm is part of the key id
2017-03-28 11:34:37 +01:00
bac9bf1b12
Typo
2017-03-27 18:02:17 +01:00
d82c42837f
Short circuit if all new events have same state group
2017-03-27 18:00:47 +01:00
35b4aa04be
Notify on new federation traffic
2017-03-27 14:07:47 +01:00
2a28b79e04
Batch sending of device list pokes
2017-03-24 14:44:49 +00:00
281553afe6
Merge pull request #2062 from matrix-org/erikj/presence_replication
...
Use presence replication stream to invalidate cache
2017-03-24 13:57:45 +00:00
987f4945b4
Actually call invalidate
2017-03-24 13:28:20 +00:00
31d56c3fb5
Merge pull request #2061 from matrix-org/erikj/add_transaction_store
...
Add slave transaction store to workers who send federation requests
2017-03-24 13:24:27 +00:00
09f79aaad0
Use presence replication stream to invalidate cache
...
Instead of using the cache invalidation replication stream to invalidate
the _get_presence_cache, we can instead rely on the presence replication
stream. This reduces the amount of replication traffic considerably.
2017-03-24 13:21:08 +00:00
23e0ff840a
Merge pull request #2060 from matrix-org/erikj/cache_hosts_in_room
...
Cache hosts in room
2017-03-24 13:07:22 +00:00
48e7697911
Add slave transaction store
2017-03-24 13:05:30 +00:00
f136c89d5e
Merge pull request #2058 from matrix-org/rav/logcontext_leaks_2
...
try not to drop context after federation requests
2017-03-24 12:47:26 +00:00
01fc847f7f
Merge pull request #2057 from matrix-org/rav/missing_yield_2
...
Add another missing yield on check_device_registered
2017-03-24 12:46:43 +00:00
7fc1f1e2b6
Cache hosts in room
2017-03-24 11:46:24 +00:00
57cfa513f5
Merge pull request #2054 from matrix-org/erikj/user_iter_cursor
...
Reduce some CPU work on DB threads
2017-03-24 11:40:08 +00:00
d58b1ffe94
Replace some calls to cursor_to_dict
...
cursor_to_dict can be surprisinglh expensive for large result sets, so lets
only call it when we need to.
2017-03-24 11:07:02 +00:00
e71940aa64
Use iter(items|values)
2017-03-24 10:57:02 +00:00
e36950dec5
Merge pull request #2055 from matrix-org/dbkr/fix_add_msisdn_requestToken
...
Fix token request for addition of phone numbers
2017-03-24 10:32:57 +00:00
f902e89d4b
Merge pull request #2056 from matrix-org/dbkr/fix_invite_reject
...
Fix rejection of invites not reaching sync
2017-03-24 10:32:51 +00:00
a380f041c2
try not to drop context after federation requests
...
preserve_context_over_fn uses a ContextPreservingDeferred, which only restores
context for the duration of its callbacks, which isn't really correct, and
means that subsequent operations in the same request can end up without their
logcontexts.
2017-03-23 22:36:21 +00:00
9397edb28b
Merge pull request #2050 from matrix-org/rav/federation_backoff
...
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
06ce7335e9
Merge pull request #2052 from matrix-org/rav/time_bound_deferred
...
Fix time_bound_deferred to throw the right exception
2017-03-23 22:22:54 +00:00
13c8749ac9
Add another missing yield on check_device_registered
2017-03-23 22:18:53 +00:00
e1f1784f99
Fix rejection of invites not reaching sync
...
Always allow the user to see their own leave events, otherwise
they won't see the event if they reject an invite for a room whose
history visibility is set such that they cannot see events before
joining.
2017-03-23 18:50:31 +00:00
86e865d7d2
Oops, remove unintentional change
2017-03-23 18:48:30 +00:00
a2dfab12c5
Fix token request for addition of phone numbers
2017-03-23 18:46:17 +00:00
00957d1aa4
User Cursor.__iter__ instead of fetchall
...
This prevents unnecessary construction of lists
2017-03-23 17:53:49 +00:00
250ce11ab9
Add jsonschema to python_dependencies.py
...
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:47 -03:00
566641a0b5
use jsonschema.FormatChecker for RoomID and UserID strings
...
* use a valid filter in rest/client/v2_alpha test
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:41 -03:00
acafcf1c5b
Add valid filter tests, flake8, fix typo
...
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:10 -03:00
e56c79c114
check_valid_filter using JSONSchema
...
* add invalid filter tests
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:07 -03:00
6ebe2d23b1
Raise a more helpful exception
2017-03-23 13:48:30 +00:00
e64655c25d
Don't user upsert to persist new one time keys
...
Instead we no-op duplicate one time key uploads, an error if the key_id
already exists but encodes a different key.
2017-03-23 13:17:00 +00:00
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
b88a323ffb
Fix time_bound_deferred to throw the right exception
...
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
2017-03-23 12:07:11 +00:00
55366814a6
Merge pull request #2048 from matrix-org/rav/missing_yield
...
Add a missing yield in device key upload
2017-03-23 09:29:44 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
ad8a26e361
MatrixFederationHttpClient: clean up
...
rename _create_request to _request, and push ascii-encoding of `destination`
and `path` down into it
2017-03-23 00:27:04 +00:00
19b9366d73
Fix a couple of logcontext leaks
...
Use preserve_fn to correctly manage the logcontexts around things we don't want
to yield on.
2017-03-23 00:17:46 +00:00
e08f81d96a
Add a missing yield in device key upload
...
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
2017-03-23 00:16:43 +00:00
35e0cfb54d
Merge pull request #2044 from matrix-org/rav/crypto_docs
...
fix up some key verif docstrings
2017-03-22 17:12:55 +00:00
7b67848042
Merge pull request #2042 from matrix-org/rav/fix_key_caching
...
Fix caching of remote servers' signature keys
2017-03-22 17:11:55 +00:00
95f21c7a66
Fix caching of remote servers' signature keys
...
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.
By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
2017-03-22 15:11:30 +00:00
d101488c5f
Merge branch 'master' into develop
2017-03-21 22:48:21 +01:00
64778693be
fix up some key verif docstrings
2017-03-21 13:27:50 +00:00
37a187bfab
Merge pull request #2033 from matrix-org/erikj/repl_speed
...
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
633dcc316c
Bump changelog and version
2017-03-20 16:37:14 +00:00
c36d15d2de
Add some debug to help diagnose weird federation issue
2017-03-20 15:36:14 +00:00
aac6d1fc9b
PEP8
2017-03-20 13:47:56 +00:00
bd08ee7a46
Merge pull request #2026 from matrix-org/rav/logcontext_docs
...
Logcontext docs
2017-03-20 12:05:21 +00:00
eddce9d74a
Merge pull request #2027 from matrix-org/rav/logcontext_leaks
...
A few fixes to logcontext things
2017-03-20 11:53:36 +00:00
f40c2db05a
Stop preserve_fn leaking context into the reactor
...
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
067b00d49d
Run the reactor with the sentinel logcontext
...
This fixes a class of 'Unexpected logcontext' messages, which were happening
because the logcontext was somewhat arbitrarily swapping between the sentinel
and the `run` logcontext.
2017-03-18 00:07:43 +00:00
994d7ae7c5
Remove broken use of clock.call_later
...
background_updates was using `call_later` in a way that leaked the logcontext
into the reactor.
We could have rewritten it to do it properly, but given that we weren't using
the fancier facilities provided by `call_later`, we might as well just use
`async.sleep`, which does the logcontext stuff properly.
2017-03-18 00:01:37 +00:00
d2d146a314
Logcontext docs
2017-03-17 23:59:28 +00:00
61f471f779
Don't send the full event json over replication
2017-03-17 15:50:01 +00:00
0c01f829ae
Avoid resetting state on rejected events
...
When we get a rejected event, give it the same state_group as its prev_event,
rather than no state_group at all.
This should fix https://github.com/matrix-org/synapse/issues/1935 .
2017-03-17 15:06:08 +00:00
5068fb16a5
Refactoring and cleanups
...
A few non-functional changes:
* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
mind-bending conditional.
2017-03-17 15:06:07 +00:00
2abe85d50e
Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join
...
Queue up federation PDUs while a room join is in progress
2017-03-17 11:32:44 +00:00
da146657c9
Comments
2017-03-16 13:04:07 +00:00
a158c36a8a
Comment
2017-03-16 11:57:45 +00:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
2ccf3b241c
Implement no op for room stream in sync
2017-03-16 11:06:41 +00:00
9ce53a3861
Queue up federation PDUs while a room join is in progress
...
This just takes the existing `room_queues` logic and moves it out to
`on_receive_pdu` instead of `_process_received_pdu`, which ensures that we
don't start trying to fetch prev_events and whathaveyou until the join has
completed.
2017-03-15 18:01:11 +00:00
54d2b7e596
Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence
...
Add fallback to last_active_ts if it beats the last sync time on a presence timeout.
2017-03-15 17:37:15 +00:00
a8f96c63aa
Comment
2017-03-15 16:01:01 +00:00
f83ac78201
Cache set of users whose presence the other user should see
2017-03-15 15:29:19 +00:00
e6032054bf
Add a great comment to handle_timeout for active vs sync times.
2017-03-15 15:24:48 +00:00
ebf5a6b14c
Add fallback to last_active_ts if it beats the last sync time.
2017-03-15 15:17:16 +00:00
e892457a03
Comment
2017-03-15 15:01:39 +00:00
a297155a97
Remove unused import
2017-03-15 14:49:25 +00:00
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
0ad44acb5a
Merge pull request #1997 from matrix-org/dbkr/cas_partialdownload
...
Handle PartialDownloadError in CAS login
2017-03-15 13:52:34 +00:00
0970e0307e
typo
2017-03-15 12:40:42 +00:00
5aa42d4292
set default for turn_allow_guests correctly
2017-03-15 12:40:13 +00:00
e0ff66251f
add setting (on by default) to support TURN for guests
2017-03-15 12:22:18 +00:00
29ed09e80a
Fix assertion to stop transaction queue getting wedged
...
... and update some docstrings to correctly reflect the types being used.
get_new_device_msgs_for_remote can return a long under some circumstances,
which was being stored in last_device_list_stream_id_by_dest, and was then
upsetting things on the next loop.
2017-03-15 12:16:55 +00:00
872e75a3d5
Add some metrics on notifier
2017-03-15 10:56:51 +00:00
7827251daf
Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
...
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
b5d1c68beb
Implement reset_context_after_deferred
...
to correctly reset the context when we fire off a deferred we aren't going to
wait for.
2017-03-15 02:21:07 +00:00
f2ed64eaaf
Merge pull request #1992 from matrix-org/rav/fix_media_loop
...
Fix routing loop when fetching remote media
2017-03-14 23:40:35 +00:00
bad72b0b8e
Merge pull request #2002 from matrix-org/erikj/dont_sync_by_default
...
Reduce number of spurious sync result generations.
2017-03-14 16:58:45 +00:00
fd2eef49c8
Reduce spurious calls to generate sync
2017-03-14 16:33:45 +00:00
1d09586599
Address review comments
...
- don't blindly proxy all HTTPRequestExceptions
- log unexpected exceptions at error
- avoid `isinstance`
- improve docs on `from_http_response_exception`
2017-03-14 14:15:37 +00:00
7f237800e9
re-refactor exception heirarchy
...
Give CodeMessageException back its `msg` attribute, and use that to hold the
HTTP status message for HttpResponseException.
2017-03-14 14:15:37 +00:00
1ece06273e
Handle PartialDownloadError in CAS login
2017-03-14 13:37:36 +00:00
bb256ac96f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed
2017-03-14 11:35:05 +00:00
cc7a294e2e
Fix current_state_events table to not lie
...
If we try and persist two state events that have the same ancestor we
calculate the wrong current state when persisting those events.
2017-03-14 10:57:43 +00:00
7b6ed9871e
Use extend instead of +=
2017-03-14 10:49:55 +00:00
d79a687d85
Oops, remove print
2017-03-14 10:40:20 +00:00
f29d85d9e4
Merge pull request #1993 from matrix-org/luke/delete-devices
...
Implement delete_devices API
2017-03-14 10:02:56 +00:00
bbeeb97f75
Implement _simple_delete_many_txn, use it to delete devices
...
(But this doesn't implement the same for deleting access tokens or e2e keys.
Also respond to code review.
2017-03-13 17:53:23 +00:00
0a9945220e
Fix registration for broken clients
...
Only offer msisdn flows if the x_show_msisdn option is given.
2017-03-13 17:29:38 +00:00
73a5f06652
Support registration / login with phone number
...
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
c077c3277b
Flake
2017-03-13 16:45:38 +00:00
31f3ca1b2b
Merge pull request #1990 from matrix-org/rav/log_config_comments
...
Add helpful texts to logger config options
2017-03-13 16:42:12 +00:00
c81f33f73d
Implement delete_devices API
...
This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.
2017-03-13 16:33:51 +00:00
170ccc9de5
Fix routing loop when fetching remote media
...
When we proxy a media request to a remote server, add a query-param, which will
tell the remote server to 404 if it doesn't recognise the server_name.
This should fix a routing loop where the server keeps forwarding back to
itself.
Also improves the error handling on remote media fetches, so that we don't
always return a rather obscure 502.
2017-03-13 16:30:36 +00:00
45c7f12d2a
Add new storage function to slave store
2017-03-13 16:26:44 +00:00
6037a9804c
Add helpful texts to logger config options
2017-03-13 12:33:35 +00:00
3c69f32402
Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu
2017-03-13 12:20:47 +00:00
6bfe8e32b5
Merge pull request #1983 from matrix-org/rav/no_redirect_stdio
...
Add an option to disable stdio redirect
2017-03-13 12:20:07 +00:00
5fc9261929
Merge pull request #1982 from matrix-org/rav/sighup_for_logconfig
...
Reread log config on SIGHUP
2017-03-13 12:19:54 +00:00
0162994983
Comments
2017-03-13 11:53:26 +00:00
254b7c5b15
Bump changelog and versions
2017-03-13 10:02:58 +00:00
672dcf59d3
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.3
2017-03-13 09:59:54 +00:00
7eae6eaa2f
Revert "Support registration & login with phone number"
2017-03-13 09:59:33 +00:00
79926e016e
Assume rooms likely haven't changed
2017-03-13 09:50:10 +00:00
a61dd408ed
enable guest access for the 3pl/3pid APIs
2017-03-12 19:30:45 +00:00
8ffbe43ba1
Get current state by using current_state_events table
2017-03-10 17:39:35 +00:00
bcfa5cd00c
Add an option to disable stdio redirect
...
This makes it tractable to run synapse under pdb.
2017-03-10 15:38:29 +00:00
d84bd51e95
Refactor logger config for workers
...
- to make it easier to add more config options.
2017-03-10 15:34:01 +00:00
9072a8c627
Reread log config on SIGHUP
...
When we are using a log_config file, reread it on SIGHUP.
2017-03-10 15:29:55 +00:00
8f267fa8a8
Fix it for the workers
2017-03-10 11:22:25 +00:00
64d62e41b8
Noop repated delete device inbox calls from /sync
2017-03-10 10:36:43 +00:00
3545e17f43
Add setdefault key to ExpiringCache
2017-03-10 10:30:49 +00:00
29235901b8
Move FederationServer._handle_new_pdu to FederationHandler
...
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.
Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09 16:20:13 +00:00
e8b1721290
Move sig check out of _handle_new_pdu
...
When we receive PDUs via `get_missing_events`, we have already checked their
sigs, so there is no need to do it again.
2017-03-09 15:50:44 +00:00
3406333a58
Factor _get_missing_events_for_pdu out of _handle_new_pdu
...
This should be functionally identical: it just seeks to improve readability by
reducing indentation.
2017-03-09 15:50:44 +00:00
45d173a59a
Fix docstring
2017-03-09 15:50:29 +00:00
663396e45d
Merge pull request #1971 from matrix-org/dbkr/msisdn_signin
...
Support registration & login with phone number
2017-03-09 10:18:51 +00:00
ece7e00048
Comment when our 3pids would be incomplete
2017-03-08 19:07:18 +00:00
9d0d40fc15
Docs
2017-03-08 19:05:29 +00:00
3edc57296d
Incorrectly copied copyright
...
This file post-dates OM
2017-03-08 19:00:51 +00:00
727124a762
Not any more, it doesn't
2017-03-08 19:00:23 +00:00
6ad71cc29d
Remove spurious SQL logging ( #1972 )
...
looks like the upsert function was accidentally sending sql logging to the
general logger. We already log the sql in `txn.execute`.
2017-03-08 18:00:44 +00:00
d4d3629aaf
Better error message
2017-03-08 17:01:26 +00:00
3170c56e07
Bump changelog and version
2017-03-08 13:00:47 +00:00
1c99934b28
pep8
2017-03-08 11:58:20 +00:00
a9e2b9ec16
Add msisdn util file
2017-03-08 11:53:36 +00:00
85bb322333
Pull out datastore in initialiser
2017-03-08 11:51:25 +00:00
65d43f3ca5
Minor fixes from PR feedback
2017-03-08 11:48:43 +00:00
Richard van der Hoff