990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker
...
API for monthly_active_users table
2018-08-08 12:44:15 +00:00
360ba89c50
Don't fail requests to unbind 3pids for non supporting ID servers
...
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.
Fixes #3661
2018-08-08 12:06:18 +01:00
53bca4690b
more metrics for the federation and appservice senders
2018-08-07 19:09:48 +01:00
3523f5432a
Don't expose default_room_version as config opt
2018-08-07 12:51:57 +01:00
495cb100d1
Allow profile changes to happen on workers
2018-08-07 10:50:26 +01:00
a3f5bf79a0
Add EDU/query handling over replication
2018-08-06 15:23:31 +01:00
e26dbd82ef
Add replication APIs for persisting federation events
2018-08-06 15:02:28 +01:00
f900d50824
include known room versions in outgoing make_joins
2018-08-06 13:45:37 +01:00
e40a510fbf
py3 fix
2018-08-03 23:19:13 +01:00
886be75ad1
bug fixes
2018-08-03 22:29:03 +01:00
16d9701892
Return M_NOT_FOUND when a profile could not be found. ( #3596 )
2018-08-03 19:08:05 +01:00
0ca459ea33
Basic support for room versioning
...
This is the first tranche of support for room versioning. It includes:
* setting the default room version in the config file
* new room_version param on the createRoom API
* storing the version of newly-created rooms in the m.room.create event
* fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
897c51d274
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-03 13:40:47 +01:00
cb298ff623
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_repl_servlet
2018-08-03 09:25:15 +01:00
1fa98495d0
Merge pull request #3639 from matrix-org/rav/refactor_error_handling
...
Clean up handling of errors from outbound requests
2018-08-02 17:38:24 +01:00
74b1d46ad9
do mau checks based on monthly_active_users table
2018-08-02 16:57:35 +01:00
704c3e6239
Merge branch 'master' into develop
2018-08-02 15:43:30 +01:00
0bf5ec0db7
Check room visibility for /event/ requests
...
Make sure that the user has permission to view the requeseted event for
/event/{eventId} and /room/{roomId}/event/{eventId} requests.
Also check that the event is in the given room for
/room/{roomId}/event/{eventId}, for sanity.
2018-08-02 15:03:27 +01:00
14fa9d4d92
Avoid extra db lookups
...
Since we're about to look up the events themselves anyway, we can skip the
extra db queries here.
2018-08-02 13:55:51 +01:00
0a65450d04
Validation for events/rooms in fed requests
...
When we get a federation request which refers to an event id, make sure that
said event is in the room the caller claims it is in.
(patch supplied by @turt2live)
2018-08-02 13:48:40 +01:00
40c1c59cf4
Merge pull request #3621 from matrix-org/erikj/split_fed_store
...
Split out DB writes in federation handler
2018-08-02 10:41:42 +01:00
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits
...
Initial impl of capping MAU
2018-08-01 15:58:45 +00:00
01e93f48ed
Kill off MatrixCodeMessageException
...
This code brings the SimpleHttpClient into line with the
MatrixFederationHttpClient by having it raise HttpResponseExceptions when a
request fails (rather than trying to parse for matrix errors and maybe raising
MatrixCodeMessageException).
Then, whenever we were checking for MatrixCodeMessageException and turning them
into SynapseErrors, we now need to check for HttpResponseExceptions and call
to_synapse_error.
2018-08-01 16:02:46 +01:00
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
303f1c851f
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits
2018-08-01 13:42:50 +01:00
a6d7b74915
update docs
2018-08-01 13:39:14 +01:00
4b256b9271
_persist_auth_tree no longer returns anything
2018-08-01 13:39:07 +01:00
0aba3d361a
count_monthly_users() async
2018-08-01 11:47:58 +01:00
7931393495
make count_monthly_users async synapse/handlers/auth.py
2018-08-01 10:21:56 +01:00
bdbdceeafa
rename replication_layer to federation_client
...
I have HAD ENOUGH of trying to remember wtf a replication layer is in terms of
classes.
2018-07-31 15:44:05 +01:00
443da003bc
Use new helper base class for membership requests
2018-07-31 14:32:23 +01:00
729b672823
Use new helper base class for ReplicationSendEventRestServlet
2018-07-31 14:32:23 +01:00
df2235e7fa
coding style
2018-07-31 13:16:20 +01:00
251e6c1210
limit register and sign in on number of monthly users
2018-07-30 15:55:57 +01:00
143f1a2532
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_fed_store
2018-07-30 09:56:18 +01:00
e9b2d047f6
make /context lazyload & filter aware ( #3567 )
...
make /context lazyload & filter aware.
2018-07-27 15:12:50 +01:00
b0b5566f36
Merge pull request #3391 from t3chguy/t3chguy/default_inviter_display_name_3pid
...
if inviter_display_name == ""||None then default to inviter MXID
2018-07-27 10:08:39 +01:00
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
1bcd0490c2
Merge pull request #2970 from matrix-org/matthew/filter_members
...
Implement the lazy_load_members room state filter parameter
2018-07-26 00:03:01 +01:00
d8e65ed7e1
Fix a minor documentation typo in on_make_leave
2018-07-25 15:44:41 -06:00
2565804030
Merge branch 'develop' into matthew/filter_members
2018-07-25 17:27:49 +01:00
78a691d005
Split out DB writes in federation handler
...
This will allow us to easily add an internal replication API to proxy
these reqeusts to master, so that we can move federation APIs to
workers.
2018-07-25 16:22:56 +01:00
55acd6856c
Fix updating of cached remote profiles
...
_update_remote_profile_cache was missing its `defer.inlineCallbacks`, so when
it was called, would just return a generator object, without actually running
any of the method body.
2018-07-25 10:34:48 +01:00
87951d3891
Merge branch 'develop' of github.com:matrix-org/synapse into t3chguy/default_inviter_display_name_3pid
2018-07-24 17:17:46 +01:00
8b8c4f34a3
Replace usage of get_current_toke with StreamToken.START
...
This allows us to handle /context/ requests on the client_reader worker
without having to pull in all the various stream handlers (e.g.
precence, typing, pushers etc). The only thing the token gets used for
is pagination, and that ignores everything but the room portion of the
token.
2018-07-24 16:49:17 +01:00
a321f78991
Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groups
...
Fixes and optimisations for resolve_state_groups
2018-07-24 14:11:45 +01:00
1a01a5b964
clarify comment on p_ids
2018-07-24 14:03:15 +01:00
eb1d911ab7
rather than adding ll_ids, remove them from p_ids
2018-07-24 13:40:49 +01:00
d19fba3655
Merge branch 'develop' into matthew/filter_members
2018-07-24 12:39:54 +01:00
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
a678145010
Merge branch 'develop' into rav/logcontext_fixes
2018-07-24 10:43:30 +01:00
536bc63a4e
Merge branch 'develop' into erikj/client_apis_move
2018-07-24 09:57:05 +01:00
cf2d15c6a9
another couple of logcontext leaks
2018-07-24 00:57:48 +01:00
8dff6e0322
Logcontext fixes
...
Fix some random logcontext leaks.
2018-07-24 00:37:17 +01:00
c1f80effbe
Handle delta_ids being None in _update_context_for_auth_events
...
it's easier to create the new state group as a delta from the existing one.
(There's an outside chance this will help with
https://github.com/matrix-org/synapse/issues/3364 )
2018-07-23 22:06:50 +01:00
adfe29ec0b
Merge branch 'develop' into matthew/filter_members
2018-07-23 19:21:37 +01:00
254fb430d1
incorporate review
2018-07-23 19:21:20 +01:00
0faa3223cd
Fix missing attributes on workers.
...
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
0b0b24cb82
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move
2018-07-23 13:21:15 +01:00
027bc01a1b
Add support for updating state
2018-07-23 13:17:25 +01:00
e42510ba63
Use new getters
2018-07-23 13:17:22 +01:00
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
5c88bb722f
Move PaginationHandler to its own file
2018-07-20 15:32:23 +01:00
0ecf68aedc
Move check_in_room_or_world_readable to Auth
2018-07-20 15:30:59 +01:00
7c712f95bb
Combine Limiter and Linearizer
...
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
2018-07-20 13:11:43 +01:00
8462c26485
Improvements to the Limiter
...
* give them names, to improve logging
* use a deque rather than a list for efficiency
2018-07-20 12:50:27 +01:00
bcaec2915a
incorporate review
2018-07-19 19:03:50 +01:00
924eb34d94
add a filtered_types param to limit filtering to specific types
2018-07-19 18:32:02 +01:00
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
bacdf0cbf9
Move RoomContextHandler out of Handlers
...
This is in preparation for moving GET /context/ to a worker
2018-07-18 15:33:03 +01:00
8cb8df55e9
Split MessageHandler into read only and writers
...
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
2018-07-18 15:33:03 +01:00
dab00faa83
Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacks
...
Remove event re-signing hacks
2018-07-18 12:46:27 +01:00
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
...
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
ea69d35651
Move filter_events_for_server out of FederationHandler
...
for easier unit testing.
2018-07-16 13:06:24 +01:00
ba22b6a456
typo
2018-07-13 12:03:39 +01:00
ea752bdd99
s/becuase/because/g
2018-07-10 17:58:18 +01:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
a4ab491371
Merge branch 'develop' into rav/drop_re_signing_hacks
2018-07-04 07:13:38 +01:00
e3b4043800
Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking
...
Check the state of prev_events a bit more thoroughly when coming over federation
2018-06-29 13:55:02 +01:00
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
99800de63d
try and clean up
2018-06-27 11:40:27 +01:00
f03a5d1a17
pep8
2018-06-27 11:38:14 +01:00
a7ecf34b70
cleanups
2018-06-27 11:36:03 +01:00
77078d6c8e
handle federation not telling us about prev_events
2018-06-27 11:27:32 +01:00
c7f6b420ae
Merge pull request #3448 from matrix-org/matthew/gdpr-deactivate-admin-api
...
add GDPR erase param to deactivate API
2018-06-26 10:43:14 +01:00
1e788db430
add GDPR erase param to deactivate API
2018-06-26 10:26:54 +01:00
d72fb9a448
Merge pull request #3442 from matrix-org/matthew/allow-unconsented-parts
...
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 20:14:18 +01:00
df48f7ef37
Actually fix it
2018-06-25 20:03:41 +01:00
a0e8a53c6d
Comment
2018-06-25 19:57:38 +01:00
7bdc5c8fa3
Fix bug with assuming wrong type
2018-06-25 19:56:02 +01:00
ea7a9c0483
Add fast path to _filter_events_for_server
...
Most rooms have a trivial history visibility like "shared" or
"world_readable", especially large rooms, so lets not bother getting the
full membership of those rooms in that case.
2018-06-25 19:49:13 +01:00
0269367f18
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 17:56:10 +01:00
784189b1f4
typos
2018-06-25 17:37:16 +01:00
6eb861b67f
typo
2018-06-25 17:37:16 +01:00
33fdcfa957
Merge pull request #3441 from matrix-org/erikj/redo_erasure
...
Fix user erasure and re-enable
2018-06-25 14:37:01 +01:00
07cad26d65
Remove all global reactor imports & pass it around explicitly ( #3424 )
2018-06-25 14:08:28 +01:00
244484bf3c
Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility""
...
This reverts commit 1d009013b3
2018-06-25 13:42:55 +01:00
1d009013b3
Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"
...
This reverts commit ce0d911156b4a5d767a9
2018-06-22 16:35:10 +01:00
28ddc6cfbe
Also log number of events for serach context
2018-06-22 15:42:11 +01:00
4b4cec3989
Add some logging to search queries
2018-06-22 15:42:11 +01:00
ce0d911156
Merge pull request #3431 from matrix-org/rav/erasure_visibility
...
Support hiding events from deleted users
2018-06-22 15:06:44 +01:00
b4a5d767a9
Merge pull request #3428 from matrix-org/erikj/persisted_pdu
...
Simplify get_persisted_pdu
2018-06-22 14:47:43 +01:00
77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
02bfc581f8
Merge pull request #3399 from costacruise/master
...
Add error code to room creation error
2018-06-20 17:26:25 +01:00
19cd3120ec
Add error code to room creation error
...
This error code is mentioned in the documentation at https://matrix.org/docs/api/client-server/#!/Room32creation/createRoom
2018-06-14 14:08:40 +02:00
a61738b316
Remove run_on_reactor ( #3395 )
2018-06-14 18:27:37 +10:00
94700e55fa
if inviter_display_name == ""||None then default to inviter MXID
...
to prevent email invite from "None"
2018-06-13 10:31:01 +01:00
b6faef2ad7
Filter out erased messages
...
Redact any messges sent by erased users.
2018-06-12 09:53:18 +01:00
f1023ebf4b
mark accounts as erased when requested
2018-06-12 09:53:18 +01:00
9fc5b74b24
simplify get_persisted_pdu
...
it doesn't make much sense to use get_persisted_pdu on the receive path: just
get the event straight from the store.
2018-06-12 09:51:31 +01:00
187a546bff
Merge pull request #3276 from matrix-org/dbkr/unbind
...
Remove email addresses / phone numbers from ID servers when they're removed from synapse
2018-06-11 16:02:00 +01:00
c96d882a02
Merge branch 'develop' into matthew/filter_members
2018-06-10 12:26:14 +03:00
c6b1441c52
Fix event filtering in get_missing_events handler
2018-06-08 14:15:31 +01:00
0834b49c6a
Fix event filtering in get_missing_events handler
2018-06-08 11:34:46 +01:00
8503dd0047
Remove event re-signing hacks
...
These "temporary fixes" have been here three and a half years, and I can't find
any events in the matrix.org database where the calculated signature differs
from what's in the db. It's time for them to go away.
2018-06-07 16:08:29 +01:00
d62162bbec
doc fixes
2018-06-05 18:09:13 +01:00
6a29e815fc
Fix comment
2018-06-04 12:01:23 +01:00
e44150a6de
Missing yield
2018-06-04 12:01:13 +01:00
f731e42baf
docstring
2018-06-04 12:00:51 +01:00
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
5f6122fe10
more comments
2018-06-04 00:08:52 +03:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
4a9cbdbc15
Exempt AS-registered users from doing gdpr
2018-05-29 19:54:32 +01:00
a6c8f7c875
add pydoc
2018-05-29 01:09:55 +01:00
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room
...
Let users leave the server notice room after joining
2018-05-25 11:15:45 +01:00
757ed27258
Let users leave the server notice room after joining
...
They still can't reject invites, but we let them leave it.
2018-05-25 11:07:21 +01:00
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
7a1af504d7
Remove users from user directory on deactivate
2018-05-24 15:59:58 +01:00
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes
...
Fix federation backfill bugs
2018-05-24 14:52:03 +01:00
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter
...
Use iter* methods for _filter_events_for_server
2018-05-24 14:44:57 +01:00
9700d15611
pep8
2018-05-24 11:23:15 +01:00
a21a41bad7
comment
2018-05-24 11:19:59 +01:00
b3bff53178
Unbind 3pids when they're deleted too
2018-05-24 11:08:05 +01:00
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri
2018-05-23 16:06:03 +01:00
3ff6f50eac
Use the localpart in the consent uri
...
... because it's shorter.
2018-05-23 15:28:23 +01:00
82191b08f6
Support for putting %(consent_uri)s in messages
...
Make it possible to put the URI in the error message and the server notice that
get sent by the server
2018-05-23 15:24:31 +01:00
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
5aaa3189d5
s/values/itervalues/
2018-05-23 10:13:05 +01:00
0a4bca4134
Use iter* methods for _filter_events_for_server
2018-05-23 10:04:23 +01:00
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
e85b5a0ff7
Use iter* methods
2018-05-22 19:02:48 +01:00
586b66b197
Fix that states is a dict of dicts
2018-05-22 19:02:36 +01:00
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
0a078026ea
comment typo
2018-05-22 17:14:06 +01:00
Neil Johnson