8601c24287
Fix some instances of ExpiringCache not expiring cache items
...
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
2018-09-21 14:19:46 +01:00
1f3f5fcf52
Fix client IPs being broken on Python 3 ( #3908 )
2018-09-20 20:14:34 +10:00
aeca5a5ed5
Add a regression test for logging on failed connections ( #3912 )
2018-09-20 16:28:18 +10:00
31c15dcb80
Refactor matrixfederationclient to fix logging ( #3906 )
...
We want to wait until we have read the response body before we log the request
as complete, otherwise a confusing thing happens where the request appears to
have completed, but we later fail it.
To do this, we factor the salient details of a request out to a separate
object, which can then keep track of the txn_id, so that it can be logged.
2018-09-18 18:17:15 +01:00
f6e82dcddb
Tests
2018-09-14 15:08:37 +01:00
bfa0b759e0
Attempt to figure out what's going on with timeouts ( #3857 )
2018-09-14 00:15:51 +10:00
8decd6233d
improve naming
2018-09-12 16:22:15 +01:00
0ddf486724
expose number of real reserved users
2018-09-12 11:58:52 +01:00
52ec6e9dfa
Port tests/ to Python 3 ( #3808 )
2018-09-07 02:58:18 +10:00
c5440b2ca0
Merge pull request #3800 from matrix-org/neilj/remove-guests-from-mau-count
...
guest users should not be part of mau total
2018-09-06 17:45:55 +01:00
84a750e0c3
ensure guests never enter mau list
2018-09-06 17:22:53 +01:00
3801b8aa03
try to make flake8 and isort happy
2018-09-06 11:35:19 -04:00
4f8baab0c4
Merge branch 'master' into develop
2018-09-06 13:05:22 +01:00
61b05727fa
guest users should not be part of mau total
2018-09-05 22:30:36 +01:00
c127c8d042
Fix origin handling for pushed transactions
...
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
2018-09-05 13:08:07 +01:00
77055dba92
Fix tests on postgresql ( #3740 )
2018-09-04 02:21:48 +10:00
0b01281e77
move threepid checker to config, add missing yields
2018-08-31 17:11:11 +01:00
ea068d6f3c
fix bug where preserved threepid user comes to sign up and server is mau blocked
2018-08-31 10:49:14 +01:00
14e4d4f4bf
Port storage/ to Python 3 ( #3725 )
2018-08-31 00:19:58 +10:00
414fa36f3e
Fix up tests
2018-08-28 17:21:05 +01:00
16a31c6fce
update to newer Synapse APIs
2018-08-24 22:51:25 -04:00
9db2476991
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_contact
2018-08-24 17:00:37 +01:00
05077e06fa
Change admin_uri to admin_contact in config and errors
2018-08-24 16:51:27 +01:00
83caead95a
Merge branch 'develop' into e2e_backups
2018-08-24 11:44:26 -04:00
01a5a8b9e3
Fix checking if service notice room is already tagged
...
This manifested in synapse repeatedly setting the tag for the room
2018-08-24 16:22:37 +01:00
f2cbbda956
Unit tests
2018-08-23 19:17:19 +01:00
cd77270a66
Implement trail users
2018-08-23 19:17:19 +01:00
7e6e588e60
Fix bug where we resent "limit exceeded" server notices
...
This was due to a bug where we mutated a cached event's contents
2018-08-23 16:21:20 +01:00
c5842dff1a
Actually run the tests
2018-08-23 10:35:54 +01:00
fd2dbf1836
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-22 17:06:10 +01:00
9643a6f7f2
Update notice format
2018-08-22 17:00:29 +01:00
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles
...
Allow profile updates to happen on workers
2018-08-22 11:35:55 +01:00
3bf8bab8f9
Merge pull request #3673 from matrix-org/erikj/refactor_state_handler
...
Refactor state module to support multiple room versions
2018-08-22 10:04:55 +01:00
bb81e78ec6
Split the state_group_cache in two ( #3726 )
...
Splits the state_group_cache in two.
One half contains normal state events; the other contains member events.
The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache. This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching.
Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow. Hopefully fixes https://github.com/matrix-org/synapse/issues/3720 .
2018-08-22 00:56:37 +02:00
4d664278af
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler
2018-08-20 14:49:43 +01:00
e07970165f
rename error code
2018-08-18 14:39:45 +01:00
ba1fbf7d5b
special case server_notices_mxid
2018-08-18 12:31:08 +01:00
c334ca67bb
Integrate presence from hotfixes ( #3694 )
2018-08-18 01:08:45 +10:00
d49b77404b
clean up, no functional changes
2018-08-17 15:21:34 +01:00
63260397c6
Merge pull request #3701 from matrix-org/rav/use_producer_for_responses
...
Use a producer to stream back responses
2018-08-17 14:58:45 +01:00
3ee57bdcbb
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-17 14:34:10 +01:00
782689bd40
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-17 14:15:48 +01:00
ca87ad1def
Split ProfileHandler into master and worker
2018-08-17 14:15:14 +01:00
b5f638f1f4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-17 14:04:15 +01:00
69c49d3fa3
Merge branch 'develop' into neilj/limit_exceeded_error
2018-08-17 12:44:26 +00:00
bcfeb44afe
call reap on start up and fix under reaping bug
2018-08-16 22:55:32 +01:00
13ad9930c8
add new error type ResourceLimit
2018-08-16 18:02:02 +01:00
3c1080b6e4
refactor for readability, and reuse caching for setting tags
2018-08-16 17:02:04 +01:00
a675f9c556
check for room state before deciding on action
2018-08-16 14:53:35 +01:00
8cfad2e686
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-15 17:20:38 +01:00
81d727efa9
Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus
...
Fix Mau off by one errors
2018-08-15 16:19:41 +00:00
2f78f432c4
speed up /members and add at= and membership params ( #3568 )
2018-08-15 16:35:22 +01:00
fc5d937550
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking
2018-08-15 16:31:40 +01:00
86a00e05e1
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-15 16:27:08 +01:00
a87af25fbb
Fix the tests
2018-08-15 15:12:23 +01:00
75c663c7b9
update error codes
2018-08-15 11:27:48 +01:00
b8429c7c81
update error codes for resource limiting
2018-08-15 10:19:48 +01:00
ab035bdeac
replace admin_email with admin_uri for greater flexibility
2018-08-15 10:16:41 +01:00
19b433e3f4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email
2018-08-14 17:44:46 +01:00
b586b8b986
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 17:43:22 +01:00
7277216d01
fix setup_test_homeserver to be postgres compatible
2018-08-14 17:14:39 +01:00
1522ed9c07
in case max_mau is less than I think
2018-08-14 16:52:30 +01:00
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block
...
Block ability to read via sync if mau limit exceeded
2018-08-14 15:21:31 +00:00
06b331ff40
fix off by 1 errors
2018-08-14 15:28:15 +01:00
8f9a7eb58d
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-14 15:11:54 +01:00
ed4bc3d2fc
fix off by 1s on mau
2018-08-14 15:04:48 +01:00
9b5bf3d858
Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:51:38 +01:00
e25d87d97b
Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:32:18 +01:00
bdfbd934d6
Implement a new test baseclass to cut down on boilerplate ( #3684 )
2018-08-14 20:53:43 +10:00
f4b49152e2
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-13 21:09:47 +01:00
ce7de9ae6b
Revert "support admin_email config and pass through into blocking errors, return AuthError in all cases"
...
This reverts commit 0d43f991a1
2018-08-13 18:06:18 +01:00
0d43f991a1
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-13 18:00:23 +01:00
99dd975dae
Run tests under PostgreSQL ( #3423 )
2018-08-13 16:47:46 +10:00
66a4ca1d28
404 nicely if you try to interact with a missing current version
2018-08-12 19:14:31 -04:00
edc427a351
flake8
2018-08-12 19:14:31 -04:00
fe87890b18
implement remaining tests and make them work
2018-08-12 19:14:31 -04:00
f6a3067868
linting
2018-08-12 19:14:31 -04:00
15d513f16f
fix idiocies and so make tests pass
2018-08-12 19:14:31 -04:00
174be586e5
first cut at a UT
2018-08-12 19:14:31 -04:00
6c6aba76e1
implementation of server notices to alert on hitting resource limits
2018-08-10 15:12:59 +01:00
807449d8f2
fix up a forced long line
2018-08-11 00:01:43 +10:00
8b3d9b6b19
Run black.
2018-08-10 23:54:09 +10:00
b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
3c0213a217
Merge pull request #3439 from vojeroen/send_sni_for_federation_requests
...
send SNI for federation requests
2018-08-10 12:23:54 +01:00
885ea9c602
rename _user_last_seen_monthly_active
2018-08-09 18:02:12 +01:00
04df714259
fix imports
2018-08-09 17:41:52 +01:00
09cf130898
only block on sync where user is not part of the mau cohort
2018-08-09 17:39:12 +01:00
3e19beb941
Fix tests
2018-08-09 14:58:49 +01:00
69ce057ea6
block sync if auth checks fail
2018-08-09 12:26:27 +01:00
3dce9050cf
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block
2018-08-09 11:42:02 +01:00
0ad98e38d0
Merge pull request #3655 from matrix-org/neilj/disable_hs
...
Flag to disable HS without disabling federation
2018-08-09 10:41:43 +00:00
a5ef110749
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block
2018-08-09 11:40:37 +01:00
2511f3f8a0
Test fixes for Python 3 ( #3647 )
2018-08-09 12:22:01 +10:00
e92fb00f32
sync auth blocking
2018-08-08 17:54:49 +01:00
5298d79fb5
Merge branch 'develop' into neilj/disable_hs
2018-08-08 16:13:03 +00:00
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions
...
Support for room versioning
2018-08-08 17:10:53 +01:00
d2f3ef98ac
Merge branch 'develop' into neilj/disable_hs
2018-08-08 15:55:47 +00:00
7f3d897e7a
mock config.max_mau_value
2018-08-08 11:46:23 +01:00
e8eba2b4e3
implement reserved users for mau limits
2018-08-07 17:49:43 +01:00
fbe255f9a4
add default mau_limits_reserved_threepids
2018-08-06 23:24:54 +01:00
42c6823827
disable HS from config
2018-08-04 22:07:04 +01:00
d08296f9f2
remove unused import
2018-08-03 23:00:16 +01:00
886be75ad1
bug fixes
2018-08-03 22:29:03 +01:00
e10830e976
wip commit - tests failing
2018-08-03 17:55:50 +01:00
0ca459ea33
Basic support for room versioning
...
This is the first tranche of support for room versioning. It includes:
* setting the default room version in the config file
* new room_version param on the createRoom API
* storing the version of newly-created rooms in the m.room.create event
* fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
5593ff6773
fix (lots of) py3 test failures
2018-08-03 14:59:17 +01:00
b2aab04d2c
fix py3 test failure
2018-08-03 14:12:56 +01:00
950807d93a
fix caching and tests
2018-08-03 13:49:53 +01:00
74b1d46ad9
do mau checks based on monthly_active_users table
2018-08-02 16:57:35 +01:00
9180061b49
remove unused count_monthly_users
2018-08-02 15:55:29 +01:00
c4ffbecb68
fix test, update constructor call
2018-08-02 13:51:05 +01:00
00f99f74b1
insertion into monthly_active_users
2018-08-02 13:47:19 +01:00
165e067033
Revert "change monthly_active_users table to be a single column"
...
This reverts commit ec716a35b2
2018-08-02 10:59:58 +01:00
ec716a35b2
change monthly_active_users table to be a single column
2018-08-01 17:54:37 +01:00
d766f26de9
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-01 17:49:41 +01:00
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits
...
Initial impl of capping MAU
2018-08-01 15:58:45 +00:00
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
c480c4c962
fix isort
2018-08-01 14:25:58 +01:00
6eed16d8a2
fix test for py3
2018-08-01 14:02:10 +01:00
303f1c851f
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits
2018-08-01 13:42:50 +01:00
4e6e00152c
fix known broken test
2018-08-01 11:48:37 +01:00
cab782c17e
Merge pull request #3384 from matrix-org/rav/rewrite_cachedlist_decorator
...
Rewrite cache list decorator
2018-08-01 10:28:56 +01:00
7931393495
make count_monthly_users async synapse/handlers/auth.py
2018-08-01 10:21:56 +01:00
6ef983ce5c
api into monthly_active_users table
2018-07-31 16:36:24 +01:00
df2235e7fa
coding style
2018-07-31 13:16:20 +01:00
7d05406a07
fix user_ips counting
2018-07-31 12:03:23 +01:00
e908b86832
Remove pdu_failures from transactions
...
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
2018-07-30 16:28:47 -06:00
cefac79c10
monthly_active_tests
2018-07-30 22:08:09 +01:00
251e6c1210
limit register and sign in on number of monthly users
2018-07-30 15:55:57 +01:00
a8cbce0ced
fix invalidation
2018-07-27 16:17:17 +01:00
f102c05856
Rewrite cache list decorator
...
Because it was complicated and annoyed me. I suspect this will be more
efficient too.
2018-07-27 13:47:04 +01:00
7041cd872b
Merge branch 'develop' into send_sni_for_federation_requests
2018-07-27 09:17:11 +01:00
2565804030
Merge branch 'develop' into matthew/filter_members
2018-07-25 17:27:49 +01:00
0620d27f4d
flake8
2018-07-25 17:21:17 +01:00
0a7ee0ab8b
add tests for _get_some_state_from_cache
2018-07-25 16:33:52 +01:00
f59be4eb0e
Fix unit tests
...
on_notifier_poke no longer runs synchonously, so we have to do a different hack
to make sure that the replication data has been sent. Let's actually listen for
its arrival.
2018-07-25 10:30:36 +01:00
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
adfe29ec0b
Merge branch 'develop' into matthew/filter_members
2018-07-23 19:21:37 +01:00
8fbe418777
Fix unit tests
2018-07-23 13:33:49 +01:00
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
3d6df84658
Test and fix support for cancellation in Linearizer
2018-07-20 13:59:55 +01:00
4f67623674
Merge pull request #3571 from matrix-org/rav/limiter_fixes
...
A set of improvements to the Limiter
2018-07-20 13:53:03 +01:00
e1a237eaab
Admin API for creating new users ( #3415 )
2018-07-20 22:41:13 +10:00
7c712f95bb
Combine Limiter and Linearizer
...
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
2018-07-20 13:11:43 +01:00
d7275eecf3
Add a sleep to the Limiter to fix stack overflows.
...
Fixes #3570
2018-07-20 12:37:12 +01:00
650daf5628
make test work
2018-07-19 20:49:44 +01:00
1fa4f7e03e
first cut of a UT for testing state store (untested)
2018-07-19 20:19:32 +01:00
c754e006f4
Merge pull request #3556 from matrix-org/rav/background_processes
...
Run things as background processes
2018-07-19 11:04:18 +01:00
a97c845271
Move v1-only APIs into their own module & isolate deprecated ones ( #3460 )
2018-07-19 20:03:33 +10:00
8c69b735e3
Make Distributor run its processes as a background process
...
This is more involved than it might otherwise be, because the current
implementation just drops its logcontexts and runs everything in the sentinel
context.
It turns out that we aren't actually using a bunch of the functionality here
(notably suppress_failures and the fact that Distributor.fire returns a
deferred), so the easiest way to fix this is actually by simplifying a bunch of
code.
2018-07-18 20:55:05 +01:00
d897be6a98
Fix visibility of events from erased users over federation
2018-07-17 14:02:07 +01:00
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
...
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
94440ae994
fix imports
2018-07-17 11:51:26 +01:00
bc006b3c9d
Refactor REST API tests to use explicit reactors ( #3351 )
2018-07-17 20:43:18 +10:00
850238b4ef
Add unit test
2018-07-17 10:59:02 +01:00
3fe0938b76
Merge pull request #3530 from matrix-org/erikj/stream_cache
...
Don't return unknown entities in get_entities_changed
2018-07-17 13:44:46 +10:00
15b13b537f
Add a test which profiles filter_events_for_server in a large room
2018-07-16 14:06:09 +01:00
ea69d35651
Move filter_events_for_server out of FederationHandler
...
for easier unit testing.
2018-07-16 13:06:24 +01:00
505530f36a
Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests
...
# Conflicts:
# synapse/crypto/context_factory.py
2018-07-14 20:24:46 +02:00
33b60c01b5
Make auth & transactions more testable ( #3499 )
2018-07-14 07:34:49 +10:00
bc832f822f
Fixup unit test
2018-07-13 17:03:04 +01:00
3b391d9c45
Fix unit tests
2018-07-13 16:28:04 +01:00
6c1ec5a1bd
Use more portable syntax using attrs package.
...
Newer syntax
attr.ib(factory=dict)
is just a syntactic sugar for
attr.ib(default=attr.Factory(dict))
It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.
We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-07-10 00:38:49 +01:00
b5e157d895
Merge branch 'develop' into send_sni_for_federation_requests
...
# Conflicts:
# synapse/http/endpoint.py
2018-07-09 08:51:11 +02:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
feef8461d1
Merge remote-tracking branch 'hera/rav/server_acls' into develop
2018-07-05 11:04:01 +01:00
1a88640677
Merge pull request #3483 from matrix-org/rav/more_server_name_validation
...
More server_name validation
2018-07-05 10:04:20 +01:00
3cf3e08a97
Implementation of server_acls
...
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw .
2018-07-04 19:06:20 +01:00
546bc9e28b
More server_name validation
...
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.
Also, use it to verify the server name in the config file.
2018-07-04 18:59:51 +01:00
ea555d5633
Reinstate lost run_on_reactor in unit test
...
a61738b
2018-07-04 09:40:01 +01:00
508196e08a
Reject invalid server names ( #3480 )
...
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
2018-07-03 14:36:14 +01:00
e3b4043800
Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking
...
Check the state of prev_events a bit more thoroughly when coming over federation
2018-06-29 13:55:02 +01:00
f82cf3c7df
add test
2018-06-28 21:14:16 +01:00
e72234f6bd
fix tests
2018-06-28 20:56:07 +01:00
f03a5d1a17
pep8
2018-06-27 11:38:14 +01:00
94f09618e5
cleanups
2018-06-27 11:38:03 +01:00
35cc3e8b14
stylistic cleanup
2018-06-27 11:32:09 +01:00
8d62baa48c
cleanups
2018-06-27 11:31:48 +01:00
77078d6c8e
handle federation not telling us about prev_events
2018-06-27 11:27:32 +01:00
cd6bcdaf87
Better testing framework for homeserver-using things ( #3446 )
2018-06-27 10:37:24 +01:00
3d605853c8
send SNI for federation requests
2018-06-24 22:38:43 +02:00
f79abda87f
Merge pull request #3427 from matrix-org/erikj/remove_filters
...
remove dead filter_events_for_clients
2018-06-22 14:45:42 +01:00
43e02c409d
Disable partial state group caching for wildcard lookups
...
When _get_state_for_groups is given a wildcard filter, just do a complete
lookup. Hopefully this will give us the best of both worlds by not filling up
the ram if we only need one or two keys, but also making the cache still work
for the federation reader usecase.
2018-06-22 11:52:07 +01:00
77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
9a793f861c
Merge branch 'master' into develop
2018-06-14 16:36:01 +01:00
1e77ac66e3
Fix broken unit test
...
We need power levels for this test to do what it is supposed to do.
2018-06-14 14:21:29 +01:00
a502cfec00
remove spurious debug
2018-06-14 14:20:53 +01:00
5c9afd6f80
Make default state_default 50
...
Make it so that, before there is a power-levels event in the room, you need a
power level of at least 50 to send state.
Partially addresses https://github.com/matrix-org/matrix-doc/issues/1192
2018-06-14 12:38:09 +01:00
a61738b316
Remove run_on_reactor ( #3395 )
2018-06-14 18:27:37 +10:00
bd348f0af6
remove dead filter_events_for_clients
...
This is only used by filter_events_for_client, so we can simplify the whole
thing by just doing one user at a time, and removing a dead storage function to
boot.
2018-06-12 09:51:31 +01:00
f7869f8f8b
Port to sortedcontainers (with tests!) ( #3332 )
2018-06-06 00:13:57 +10:00
5dbf305444
Put python's logs into Trial when running unit tests ( #3319 )
2018-06-04 16:06:06 +10:00
a995fdae39
fix tests
2018-05-29 20:19:29 +01:00
adb6bac4d5
fix another dumb typo
2018-05-29 02:29:22 +01:00
284e36ad04
fix dumb typo
2018-05-29 02:23:11 +01:00
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
d5dca9a04f
Move consent config parsing into ConsentConfig
...
turns out we need to reuse this, so it's better in the config class.
2018-05-22 11:54:51 +01:00
9ea219c514
Send users a server notice about consent
...
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
fcc525b0b7
rest of the changes
2018-05-21 19:48:57 -05:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
6e1cb54a05
Fix logcontext leak in HttpTransactionCache
...
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
2018-05-21 16:58:20 +01:00
6d6e7288fe
Stop the transaction cache caching failures
...
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
2018-05-21 16:49:59 +01:00
6406b70aeb
Use stream rather depth ordering for push actions
...
This simplifies things as it is, but will also allow us to change the
way we traverse topologically without having to update the way push
actions work.
2018-05-11 15:30:11 +01:00
902673e356
Merge pull request #3161 from NotAFile/remove-v1auth
...
Make Client-Server API return 403 for invalid token
2018-05-03 10:10:57 +01:00
53a5fdf312
Merge pull request #3175 from matrix-org/erikj/escape_metric_values
...
Escape label values in prometheus metrics
2018-05-03 10:01:04 +01:00
32015e1109
Escape label values in prometheus metrics
2018-05-02 16:52:42 +01:00
415c6b672e
Merge branch 'develop' into rav/more_logcontext_leaks
2018-05-02 16:16:01 +01:00
11607006d9
Remove spurious unittest.DEBUG
2018-05-02 15:48:47 +01:00
46beeb9a30
Fix a couple of logcontext leaks in unit tests
...
... which were making other, innocent, tests, fail.
Plus remove a spurious unittest.DEBUG which was making the output noisy.
2018-05-02 15:46:22 +01:00
f22e7cda2c
Fix a class of logcontext leaks
...
So, it turns out that if you have a first `Deferred` `D1`, you can add a
callback which returns another `Deferred` `D2`, and `D2` must then complete
before any further callbacks on `D1` will execute (and later callbacks on `D1`
get the *result* of `D2` rather than `D2` itself).
So, `D1` might have `called=True` (as in, it has started running its
callbacks), but any new callbacks added to `D1` won't get run until `D2`
completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield`
will 'block'.
In conclusion: some of our assumptions in `logcontext` were invalid. We need to
make sure that we don't optimise out the logcontext juggling when this
situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
2018-05-02 11:58:00 +01:00
e482f8cd85
Fix incorrect reference to StringIO
...
This was introduced in 4f2f5171
2018-05-02 09:12:26 +01:00
6495dbb326
Burminate v1auth
...
This closes #2602
v1auth was created to account for the differences in status code between
the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid
tokens). However since those protocols were merged, this makes the r0
version/endpoint internally inconsistent, and violates the
specification for the r0 endpoint.
This might break clients that rely on this inconsistency with the
specification. This is said to affect the legacy angular reference
client. However, I feel that restoring parity with the spec is more
important. Either way, it is critical to inform developers about this
change, in case they rely on the illegal behaviour.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-30 22:20:43 +02:00
9e2601f830
Merge pull request #3108 from NotAFile/py3-six-urlparse
...
Use six.moves.urlparse
2018-04-30 00:33:05 +01:00
3d1ae61399
Merge branch 'develop' into rav/deferred_timeout
2018-04-27 12:54:43 +01:00
286e20f2bc
Merge pull request #3109 from NotAFile/py3-tests-fix
...
Make tests py3 compatible
2018-04-23 11:59:03 +01:00
1ea904b9f0
Use deferred.addTimeout instead of time_bound_deferred
...
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
d06a9ea5f7
Merge pull request #3104 from NotAFile/py3-unittest-config
...
Add some more variables to the unittest config
2018-04-20 15:35:58 +01:00
692a3cc806
Merge pull request #3103 from NotAFile/py3-baseexcepton-message
...
Use str(e) instead of e.message
2018-04-20 15:34:49 +01:00
639480e14a
Avoid creating events with huge numbers of prev_events
...
In most cases, we limit the number of prev_events for a given event to 10
events. This fixes a particular code path which created events with huge
numbers of prev_events.
2018-04-16 18:41:37 +01:00
a1a3c9660f
Make tests py3 compatible
...
This is a mixed commit that fixes various small issues
* print parentheses
* 01 is invalid syntax (it was octal in py2)
* [x for i in 1, 2] is invalid syntax
* six moves
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-16 00:39:32 +02:00
2a3c33ff03
Use six.moves.urlparse
...
The imports were shuffled around a bunch in py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 21:22:43 +02:00
cb9cdfecd0
Add some more variables to the unittest config
...
These worked accidentally before (python2 doesn't complain if you
compare incompatible types) but under py3 this blows up spectacularly
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:36:39 +02:00
1515560f5c
Use str(e) instead of e.message
...
Doing this I learned e.message was pretty shortlived, added in 2.6,
they realized it was a bad idea and deprecated it in 2.7
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:32:42 +02:00
e939f3bca6
Fix tests
2018-04-11 14:37:11 +01:00
a3599dda97
Merge pull request #2996 from krombel/allow_auto_join_rooms
...
move handling of auto_join_rooms to RegisterHandler
2018-04-10 01:11:00 +01:00
87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings
...
Add b prefixes to some strings that are bytes in py3
2018-04-09 23:54:05 +01:00
664adb4236
Merge pull request #3016 from silkeh/improve-service-lookups
...
Improve handling of SRV records for federation connections
2018-04-09 23:40:06 +01:00
01afc563c3
Fix overzealous cache invalidation
...
Fixes an issue where a cache invalidation would invalidate *all* pending
entries, rather than just the entry that we intended to invalidate.
2018-04-05 16:24:04 +01:00
6168351877
Add b prefixes to some strings that are bytes in py3
...
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-04 13:48:51 +02:00
72251d1b97
Remove address resolution of hosts in SRV records
...
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2018-04-04 12:26:50 +02:00
91ea0202e6
move handling of auto_join_rooms to RegisterHandler
...
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work
This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.
This is related to issue #2725
Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-14 16:45:37 +01:00
cb9f8e527c
s/replication_client/federation_client/
2018-03-13 13:26:52 +00:00
cea462e285
s/replication_server/federation_server
2018-03-13 13:22:21 +00:00
6ea27fafad
Fix tests
2018-03-13 10:55:47 +00:00
631a73f7ef
Fix tests
2018-03-13 10:39:19 +00:00
2c911d75e8
Fix comment typo
2018-03-05 12:24:49 +00:00
c818fcab11
Test and fix find_first_stream_ordering_after_ts
...
It seemed to suffer from a bunch of off-by-one errors.
2018-03-05 12:04:02 +00:00
784f036306
Move RoomMemberHandler out of Handlers
2018-03-01 14:36:50 +00:00
e5b4a208ce
Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actions
...
Batch inserts into event_push_actions_staging
2018-02-26 14:45:40 +00:00
25634ed152
Fix test
2018-02-20 12:40:44 +00:00
e440e28456
Fix unit tests
2018-02-20 11:41:40 +00:00
3a061cae26
Fix unit test
2018-02-15 16:31:59 +00:00
4810f7effd
Remove context.push_actions
2018-02-15 15:47:06 +00:00
617199d73d
Merge pull request #2847 from matrix-org/erikj/separate_event_creation
...
Split event creation into a separate handler
2018-02-06 17:01:17 +00:00
770b2252ca
s/_create_new_client_event/create_new_client_event/
2018-02-06 16:40:30 +00:00
3d33eef6fc
Store state groups separately from events ( #2784 )
...
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
25c0a020f4
Updates tests
2018-02-05 16:01:48 +00:00
db91e72ade
Merge pull request #2844 from matrix-org/rav/evicted_metrics
...
montoring metrics for number of cache evictions
2018-02-05 16:34:36 +01:00
bc496df192
report metrics on number of cache evictions
2018-02-05 15:34:01 +00:00
a1beca0e25
Fix broken unit test for media storage
2018-02-05 12:44:03 +00:00
1f881e0746
Merge pull request #2791 from matrix-org/erikj/media_storage_refactor
...
Ensure media is in local cache before thumbnailing
2018-02-05 11:28:52 +00:00
6da4c4d3bd
Factor out resolve_state_groups to a separate handler
...
We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.
2018-02-01 12:40:04 +00:00
d1f3490e75
Add tests for user directory search
2018-01-27 17:21:57 +00:00
88b9c5cbf0
Make it possible to run tests against postgres
2018-01-27 17:15:24 +00:00
d7eacc4f87
Create dbpool as normal in tests
...
... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a
bit of a mystery.
For some reason this makes one of the tests run slightly slower, so bump the
sleep(). Sorry.
2018-01-27 17:15:15 +00:00
b178eca261
Run on_new_connection for unit tests
...
Configure the connectionpool used for unit tests to run the `on_new_connection`
function.
2018-01-27 17:06:04 +00:00
6e9bf67f18
Remove unused/bitrotted MemoryDataStore
...
This isn't used, and looks thoroughly bitrotted.
2018-01-26 00:35:15 +00:00
2b91846497
Remove spurious unittest.DEBUG
2018-01-26 00:34:27 +00:00
b2da272b77
Merge pull request #2821 from matrix-org/rav/matthew_test_fixes
...
Matthew's fixes to the unit tests
2018-01-22 20:43:56 +00:00
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
5c431f421c
Matthew's fixes to the unit tests
...
Extracted from https://github.com/matrix-org/synapse/pull/2820
2018-01-22 16:46:16 +00:00
d84f65255e
Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid
...
add registrations_require_3pid and allow_local_3pids
2018-01-22 13:57:22 +00:00
81d037dbd8
mock registrations_require_3pid
2018-01-19 00:28:08 +00:00
ce4f66133e
Add unit tests
2018-01-18 16:32:11 +00:00
b6dc7044a9
Merge pull request #2804 from matrix-org/erikj/file_consumer
...
Add decent impl of a FileConsumer
2018-01-18 16:31:33 +00:00
1432f7ccd5
Move test stuff to tests
2018-01-18 11:57:57 +00:00
bc67e7d260
Add decent impl of a FileConsumer
...
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.
Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
2018-01-17 16:43:03 +00:00
44a498418c
Optimise LoggingContext creation and copying
...
It turns out that the only thing we use the __dict__ of LoggingContext for is
`request`, and given we create lots of LoggingContexts and then copy them every
time we do a db transaction or log line, using the __dict__ seems a bit
redundant. Let's try to optimise things by making the request attribute
explicit.
2018-01-16 15:49:42 +00:00
a767f06e3f
Merge pull request #2765 from matrix-org/rav/fix_room_uts
...
Fix flaky test_rooms UTs
2018-01-09 14:21:03 +00:00
f8fa5ae4af
enable twisted delayedcall debugging in UTs
2018-01-09 12:06:45 +00:00
374c4d4ced
Remove dead code
...
pointless function is pointless
2018-01-09 12:06:45 +00:00
142fb0a7d4
Disable user_directory updates for UTs
...
Fix flakiness in the UTs caused by the user_directory being updated in the
background
2018-01-09 12:06:45 +00:00
0211464ba2
Fix broken config UTs
...
https://github.com/matrix-org/synapse/pull/2755 broke log-config generation,
which in turn broke the unit tests.
2018-01-09 11:28:33 +00:00
33cb7ef0b7
Merge pull request #2723 from matrix-org/matthew/search-all-local-users
...
Add all local users to the user_directory and optionally search them
2017-12-05 11:09:47 +00:00
d5f9fb06b0
Refactor UI auth implementation
...
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
2017-12-05 09:40:05 +00:00
7b86c1fdcd
try make tests work a bit more...
2017-12-04 17:10:03 +00:00
95f8a713dc
erik told me to
2017-12-04 16:56:25 +00:00
ab1dc84779
Add extra space before inline comment
2017-11-16 18:22:40 +00:00
e836bdf734
Fix tests
2017-11-16 18:14:39 +00:00
63ef607f1f
Fix tests for Store.__init__ update
...
Fix the test to pass the right number of args to the Store constructors
2017-11-13 10:46:08 +00:00
8a4a0ddea6
Fix appservice tests to account for new behavior of notify_interested_services
2017-11-02 23:19:57 +03:00
02237ce725
Fix tests for refresh_token removal
2017-11-01 10:19:42 +00:00
9ded00f221
fix tests
2017-10-31 14:21:13 +00:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
1135193dfd
Validate group ids when parsing
...
May as well do it whenever we parse a Group ID. We check the sigil and basic
structure here so it makes sense to check the grammar in the same place.
2017-10-21 00:30:39 +01:00
dbdfd8967d
Merge pull request #2546 from matrix-org/rav/remove_dead_event_injector
...
Remove dead class
2017-10-17 13:21:22 +01:00
b8e40d146f
Merge pull request #2547 from matrix-org/rav/test_make_deferred_yieldable
...
Add some tests for make_deferred_yieldable
2017-10-17 13:21:09 +01:00
a6ad8148b9
Fix name of test_logcontext
...
The file under test is logcontext.py, not log_context.py
2017-10-17 10:53:34 +01:00
5b5f35ccc0
Add some tests for make_deferred_yieldable
2017-10-17 10:52:31 +01:00
9b714abf35
Remove dead class
...
This isn't used anywhere.
2017-10-17 10:43:36 +01:00
33122c5a1b
Fix test
2017-10-17 10:39:50 +01:00
cc794d60e7
Merge pull request #2532 from matrix-org/rav/fix_linearizer
...
Fix stackoverflow and logcontexts from linearizer
2017-10-11 17:29:32 +01:00
4fad8efbfb
Fix stackoverflow and logcontexts from linearizer
...
1. make it not blow out the stack when there are more than 50 things waiting
for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505 .
2. Make it not mess up the log contexts.
2017-10-11 15:05:05 +01:00
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
f65e31d22f
Do an AAAA lookup on SRV record targets ( #2462 )
...
Support SRV records which point at AAAA records, as well as A records.
Fixes https://github.com/matrix-org/synapse/issues/2405
2017-09-22 20:26:47 +01:00
72472456d8
Add some more tests for Keyring
2017-09-20 01:32:42 +01:00
9864efa532
Fix concurrent server_key requests ( #2458 )
...
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
2017-09-19 23:25:44 +01:00
aa620d09a0
Add a config option to block all room invites ( #2457 )
...
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
2017-09-19 16:08:14 +01:00
b5feaa5a49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged
2017-09-19 11:07:45 +01:00
290777b3d9
Clean up and document handling of logcontexts in Keyring ( #2452 )
...
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
2017-09-18 18:31:01 +01:00
bf81f3cf2c
Split out profile handler to fix tests
2017-08-25 14:34:56 +01:00
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
ed9a7f5436
Merge pull request #2309 from matrix-org/erikj/user_ip_repl
...
Fix up user_ip replication commands
2017-07-06 14:33:14 +01:00
29a4066a4d
Update test
2017-07-04 10:21:25 +01:00
8c23221666
Fix up
2017-06-27 15:53:45 +01:00
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
052c5d19d5
Merge pull request #2281 from matrix-org/erikj/phone_home_stats
...
Fix phone home stats
2017-06-15 12:46:23 +01:00
93e7a38370
Remove unhelpful test
2017-06-15 09:30:54 +01:00
23da638360
Fix typing tests
2017-05-26 10:02:04 +01:00
11c2a3655f
Only load jinja2 templates once
...
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
2017-05-22 17:48:58 +01:00
bbfe4e996c
Make get_state_groups_from_groups faster.
...
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.
By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
2017-05-17 15:12:15 +01:00
de042b3b88
Do some logging when one-time-keys get claimed
...
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
a7e9d8762d
Allow clients to upload one-time-keys with new sigs
...
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).
Fixes https://github.com/vector-im/riot-android/issues/1208 , hopefully.
2017-05-09 19:04:56 +01:00
3e5a62ecd8
Add more granular event send metrics
2017-05-02 14:23:26 +01:00
85a0d6c7ab
Remove test of replication resource
2017-04-11 10:59:27 +01:00
a5c401bd12
Merge pull request #2097 from matrix-org/erikj/repl_tcp_client
...
Move to using TCP replication
2017-04-05 09:36:21 +01:00
3a1f3f8388
Change slave storage to use new replication interface
...
As the TCP replication uses a slightly different API and streams than
the HTTP replication.
This breaks HTTP replication.
2017-04-03 15:34:19 +01:00
9cee0ce7db
Merge pull request #2075 from matrix-org/erikj/cache_speed
...
Speed up cached function access
2017-03-31 10:10:56 +01:00
350333a09a
Merge pull request #2076 from matrix-org/erikj/as_perf
...
Make AS's faster
2017-03-31 09:43:10 +01:00
eefd9fee81
Fix up tests
2017-03-30 14:14:46 +01:00
f9b4bb05e0
Fix the logcontext handling in the cache wrappers ( #2077 )
...
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
2017-03-30 13:22:24 +01:00
650f0e69f2
Compile the regex's used in ASes
2017-03-28 13:27:21 +01:00
57cfa513f5
Merge pull request #2054 from matrix-org/erikj/user_iter_cursor
...
Reduce some CPU work on DB threads
2017-03-24 11:40:08 +00:00
9397edb28b
Merge pull request #2050 from matrix-org/rav/federation_backoff
...
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
06ce7335e9
Merge pull request #2052 from matrix-org/rav/time_bound_deferred
...
Fix time_bound_deferred to throw the right exception
2017-03-23 22:22:54 +00:00
00957d1aa4
User Cursor.__iter__ instead of fetchall
...
This prevents unnecessary construction of lists
2017-03-23 17:53:49 +00:00
566641a0b5
use jsonschema.FormatChecker for RoomID and UserID strings
...
* use a valid filter in rest/client/v2_alpha test
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:41 -03:00
acafcf1c5b
Add valid filter tests, flake8, fix typo
...
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:10 -03:00
e56c79c114
check_valid_filter using JSONSchema
...
* add invalid filter tests
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:07 -03:00
0bfea9a2be
fix tests
2017-03-23 13:20:08 +00:00
b88a323ffb
Fix time_bound_deferred to throw the right exception
...
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
2017-03-23 12:07:11 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
95f21c7a66
Fix caching of remote servers' signature keys
...
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.
By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
2017-03-22 15:11:30 +00:00
37a187bfab
Merge pull request #2033 from matrix-org/erikj/repl_speed
...
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
737f283a07
Fix unit test
2017-03-20 14:03:58 +00:00
f40c2db05a
Stop preserve_fn leaking context into the reactor
...
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
c144292373
Modify test_user_sync so it doesn't look at last_active_ts over last_user_sync_ts
2017-03-15 15:38:57 +00:00
095b45c165
Aggregate event push actions
2017-02-14 13:39:41 +00:00
51adaac953
Fix email push in pusher worker
...
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
692daf6f54
Remote membership tests for replication
...
This is because it now relies of the caches stream, which only works on
postgres. We are trying to test with sqlite.
2017-01-31 16:10:16 +00:00
c7a26b7c32
Fix unit tests
2017-01-30 17:11:24 +00:00
b3e1f2aa7a
Fix unit tests
2017-01-26 17:16:24 +00:00
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
a55fa2047f
Insert delta of current_state_events to be more efficient
2017-01-20 17:10:18 +00:00
09eb08f910
Derive current_state_events from state groups
2017-01-20 11:52:51 +00:00
15f012032c
Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split
...
Optimise state resolution
2017-01-18 10:53:00 +00:00
5d6bad1b3c
Optimise state resolution
2017-01-17 13:22:19 +00:00
9e8e236d98
Tidy up test
2017-01-17 11:50:18 +00:00
f85b6ca494
Speed up cache size calculation
...
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
f2f179dce2
Add ExpiringCache tests
2017-01-16 15:33:34 +00:00
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
6823fe5241
Linearize updates to membership via PUT /state/
2017-01-09 18:25:13 +00:00
24c16fc349
Fix crash in url preview when html tag has no text
...
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
2016-12-14 22:38:18 +01:00
d53a80af25
Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
...
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
8b34f71bea
Fix unit tests
2016-12-09 16:48:48 +00:00
1529c19675
Prevent user tokens being used as guest tokens ( #1675 )
...
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
2016-12-06 15:31:37 +00:00
6c9a0ba415
test_preview: Fix incorrect wrapping
...
The old test expected an incorrect wrapping due to the preview function
not using unicode properly, so it got the wrong length.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-05 16:33:57 +01:00
0697bb2247
Make test_preview use unicode strings
...
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-05 16:33:57 +01:00
12f3b9000c
fix imports
2016-11-30 17:45:49 +00:00
aa09d6b8f0
Rip out more refresh_token code
...
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
dc4b23e1a1
Merge branch 'develop' into rav/no_more_refresh_tokens
2016-11-30 17:10:04 +00:00
1c4f05db41
Stop putting a time caveat on access tokens
...
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.
Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
5c4edc83b5
Stop generating refresh tokens
...
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
feec718265
Shuffle receipt handler around so that worker apps don't need to load it
2016-11-23 15:14:24 +00:00
ee5e8d71ac
Fix tests
2016-11-23 14:57:07 +00:00
b69f76c106
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn
2016-11-23 11:31:53 +00:00
54fed21c04
Fix tests and flake8
2016-11-22 18:18:31 +00:00
c3d963ac24
Review comments
2016-11-22 13:42:11 +00:00
0a8b0eeca1
More tests
2016-11-22 09:59:27 +00:00
53b27bbf06
Add remaining tests
2016-11-21 17:58:22 +00:00
70a2157b64
Start adding some tests
2016-11-21 17:52:45 +00:00
f97511a1f3
Move event_fields filtering to serialize_event
...
Also make it an inclusive not exclusive filter, as the spec demands.
2016-11-21 17:42:16 +00:00
524d61bf7e
Fix tests
2016-11-21 11:53:02 +00:00
3991b4cbdb
Clean transactions based on time. Add HttpTransactionCache tests.
2016-11-14 11:19:24 +00:00
d073cb7ead
Add Limiter: limit concurrent access to resource
2016-11-10 16:29:51 +00:00
177f104432
Merge pull request #1098 from matrix-org/markjh/bearer_token
...
Allow clients to supply access_tokens as headers
2016-10-25 17:33:15 +01:00
78c083f159
Merge pull request #1164 from pik/error-codes
...
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
e8b1d2a452
Refactor test_filter to use real DataStore
...
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
d43b63818c
Fix MockHttpRequest always returning M_UNKNOWN errcode in testing
2016-10-14 15:46:54 -05:00
d9350b0db8
Error codes for filters
...
* add tests
Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
2016-10-14 10:18:28 -05:00
35e2cc8b52
Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth
...
Implement pluggable password auth
2016-10-12 11:41:20 +01:00
3061dac53e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop
2016-10-11 14:08:29 +01:00
7b5546d077
rest/client/v1/register: use the correct requester in createUser
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
850b103b36
Implement pluggable password auth
...
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
2016-10-03 10:36:40 +01:00
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
22578545a0
Time out typing over federation
2016-09-23 14:00:52 +01:00
ec609f8094
Fix unit tests
2016-09-12 10:46:02 +01:00
74cbfdc7de
Fix unit tests
2016-09-06 18:30:03 +01:00
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
bc1a8b1f7a
Don't notify for online -> online transitions.
...
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
2016-08-30 15:05:32 +01:00
55fc17cf4b
Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
...
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
93b32d4515
Fix unit tests
2016-08-26 15:40:27 +01:00
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
3f11953fcb
Fix tests
2016-08-26 10:15:52 +01:00
3b8d0ceb22
More 0_0 in tests
2016-08-25 18:42:46 +01:00
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
45fd2c8942
Ensure invalidation list does not grow unboundedly
2016-08-19 16:09:16 +01:00
c0d7d9d642
Rename to on_invalidate
2016-08-19 15:13:58 +01:00
dc76a3e909
Make cache_context an explicit option
2016-08-19 15:02:38 +01:00
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
4161ff2fc4
Add concept of cache contexts
2016-08-19 14:17:07 +01:00
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
320dfe523c
Make notify_interested_services faster
2016-08-17 17:20:50 +01:00
7321f45457
Clean up _ServiceQueuer
2016-08-17 12:03:04 +01:00
b835ebcc79
Update unit tests
2016-08-17 11:22:11 +01:00
48b5829aea
Fix up preview URL API. Add tests.
...
This includes:
- Splitting out methods of a class into stand alone functions, to make
them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
a8bcc7274d
PEP8
2016-08-08 17:20:38 +01:00
6fe6a6f029
Fix login with m.login.token
...
login with token (as used by CAS auth) was broken by 067596d
2016-08-08 16:40:39 +01:00
e97648c4e2
Test summarization
2016-08-04 16:09:09 +01:00
98385888b8
PEP8
2016-08-03 15:42:08 +01:00
68264d7404
Include device name in /keys/query response
...
Add an 'unsigned' section which includes the device display name.
2016-08-03 15:42:08 +01:00
91fa69e029
keys/query: return all users which were asked for
...
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
2016-08-03 15:41:44 +01:00
0a7d3cd00f
Create separate methods for getting messages to push
...
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.
The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.
The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
eb359eced4
Add `create_requester` function
...
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
8e02494166
Delete refresh tokens when deleting devices
2016-07-26 11:10:37 +01:00
012b4c1913
Implement updating devices
...
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
bf3de7b90b
Merge pull request #945 from matrix-org/rav/background_reindex
...
Create index on user_ips in the background
2016-07-25 14:04:05 +01:00
42f4feb2b7
PEP8
2016-07-25 12:25:06 +01:00
f16f0e169d
Slightly saner logging for unittests
...
1. Give the handler used for logging in unit tests a formatter, so that the
output is slightly more meaningful
2. Log some synapse.storage stuff, because it's useful.
2016-07-25 12:12:47 +01:00
465117d7ca
Fix background_update tests
...
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
55abbe1850
make /devices return a list
...
Turns out I specced this to return a list of devices rather than a dict of them
2016-07-21 15:57:28 +01:00
1c3c202b96
Fix PEP8 errors
2016-07-21 13:15:15 +01:00
406f7aa0f6
Implement GET /device/{deviceId}
2016-07-21 12:00:29 +01:00
40a1c96617
Fix PEP8 errors
2016-07-20 18:06:28 +01:00
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
e9e3eaa67d
Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips
...
Record device_id in client_ips
2016-07-20 17:38:45 +01:00
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
b97a1356b1
Register a device_id in the /v2/register flow.
...
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.
Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
e967bc86e7
Merge pull request #932 from matrix-org/rav/register_refactor
...
Further registration refactoring
2016-07-20 11:03:33 +01:00
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
40cbffb2d2
Further registration refactoring
...
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
b9e997f561
Merge pull request #931 from matrix-org/rav/refactor_register
...
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
0da0d0a29d
rest/client/v2_alpha/register.py: Refactor flow somewhat.
...
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.
The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
a98d215204
Add filter param to /messages API
2016-07-14 16:30:56 +01:00
0136a522b1
Bug fix: expire invalid access tokens
2016-07-13 15:00:37 +02:00
7335f0adda
Add ReadWriteLock
2016-07-05 15:23:17 +01:00
2455ad8468
Remove room name & alias test
...
as get_room_name_and_alias is now gone
2016-06-24 13:34:20 +01:00
0a32208e5d
Rework ldap integration with ldap3
...
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.
Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
employing the `user_filter`, then tries the bind when a single
matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
localpart and `user_base`
Offer support for STARTTLS on a plain connection.
The configuration was changed to reflect these new possibilities.
Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2016-06-22 17:51:59 +02:00
0113ad36ee
Enable use_frozen_events in tests
2016-06-17 15:13:13 +01:00
7dbb473339
Add function to load config without generating it
...
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
6e7dc7c7dd
Fix a bug caused by a change in auth_handler function
...
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
84379062f9
Fix AS retries, but with correct ordering
2016-06-07 10:24:50 +01:00
310197bab5
Fix AS retries
2016-06-07 09:34:50 +01:00
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
73c7112433
Change CacheMetrics to be quicker
...
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
56d15a0530
Store the typing users as user_id strings. ( #819 )
...
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
70599ce925
Allow external processes to mark a user as syncing. ( #812 )
...
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
2016-06-02 15:20:15 +01:00
4a10510cd5
Split out the auth handler
2016-06-02 13:31:45 +01:00
195254cae8
Inject fake room list handler in tests
...
Otherwise it tries to start the remote public room list updating looping call which breaks.
2016-06-01 11:14:16 +01:00
c626fc576a
Move the AS handler out of the Handlers object.
...
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
6fe04ffef2
Fix set profile error with Requester.
...
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
0cb441fedd
Move typing handler out of the Handlers object
2016-05-17 15:58:46 +01:00
425e6b4983
Merge branch 'develop' into markjh/member_cleanup
2016-05-17 10:13:16 +01:00
816df9f267
get_room_members is unused now
2016-05-16 19:51:43 +01:00
821306120a
Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room
2016-05-16 19:48:07 +01:00
3b86ecfa79
Move the presence handler out of the Handlers object
2016-05-16 18:56:37 +01:00
21018c2c13
Merge pull request #783 from matrix-org/markjh/slave_account_data
...
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
3abab26458
Add a slaved datastore for account data
2016-05-13 15:34:06 +01:00
40aa6e8349
Create user with expiry
...
- Add unittests for client, api and handler
Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
c0d8e0eb63
Replicate push actions
2016-04-21 15:25:58 +01:00
f52dd35ac3
Merge pull request #738 from matrix-org/markjh/slaved_receipts
...
Add a slaved receipts store
2016-04-19 17:31:59 +01:00
5bbd424ee0
Add a slaved receipts store
2016-04-19 17:14:08 +01:00
e99365f601
Replicate get_invited_rooms_for_user
2016-04-19 15:22:14 +01:00
eb8619e256
Create log context in Measure if one doesn't exist
2016-04-18 16:08:32 +01:00
79fc4ff6f9
Merge pull request #677 from matrix-org/erikj/dns_cache
...
Read from DNS cache if within TTL
2016-04-08 14:09:56 +01:00
7e2f971c08
Remove some unused functions ( #711 )
...
* Remove some unused functions
* get_room_events_stream is only used in tests
* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
ceb599e789
Add tests for redactions
2016-04-07 16:52:07 +01:00
8c82b06904
Merge pull request #704 from matrix-org/markh/slaveIII
...
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 16:49:34 +01:00
57fa1801c3
Add sensible __eq__ operators inside the tests.
...
Rather than adding them globally. This limits the changes to only
affect the tests.
2016-04-07 16:41:37 +01:00
af03ecf352
Deduplicate joins
2016-04-07 14:19:02 +01:00
60ec9793fb
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 13:17:56 +01:00
a28d066732
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache
2016-04-07 11:11:17 +01:00
6bfec56796
Test that room membership is replicated
2016-04-06 16:20:13 +01:00
e815763b7f
Merge pull request #697 from matrix-org/markjh/slaveI
...
Add a slaved events store class
2016-04-06 16:19:25 +01:00
75fb9ac1be
Add a slaved events store class
...
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
8aab9d87fa
Don't require config to create database
2016-04-06 14:15:45 +01:00
11860637e1
Tests
2016-04-06 10:12:30 +01:00
c906f30661
Do checks for memberships before creating events
2016-04-01 16:17:32 +01:00
f699b8f997
Read from DNS cache if within TTL
2016-03-31 10:04:28 +01:00
31a9eceda5
Add a replication stream for state groups
2016-03-30 16:01:58 +01:00
673c96ce97
Remove dead code left over from presence changes
2016-03-17 11:03:47 +00:00
ff7d3dc3a0
Fix tests
2016-03-16 14:25:14 +00:00
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
1be438f2a6
Flake8 fix
2016-03-10 12:13:35 +03:00
40160e24ab
Register endpoint returns refresh_token
...
Guest registration still doesn't return refresh_token
2016-03-10 10:29:19 +03:00
316c00936f
Fix tests
2016-03-07 17:32:36 +00:00
3110c37d02
Fix unit tests
2016-03-04 14:48:17 +00:00
ec7460b4f2
Merge branch 'develop' into markjh/pushrule_stream
2016-03-04 14:44:34 +00:00
bb0e82fff1
tests/utils: added room_invite_state_types to test config
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-03-04 10:43:17 +01:00
b4022cc487
Pass whole requester to ratelimiting
...
This will enable more detailed decisions
2016-03-03 16:43:42 +00:00
95481e7ba7
Merge pull request #571 from matrix-org/daniel/asids
...
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
2223204eba
Hook push rules up to the replication API
2016-03-02 17:26:20 +00:00
2a78dac60d
Merge pull request #612 from matrix-org/erikj/cache_size
...
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-02 14:36:10 +00:00
60a0f81c7a
Add a /replication API for extracting the updates that happened on
...
synapse
This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.
This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.
Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.
Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.
The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
ce2cdced61
Move cache size fiddling to descriptors only. Fix tests
2016-03-01 13:21:46 +00:00
5e2890bd49
Check that the disable_registration config key is handled correctly
2016-02-22 16:01:29 +00:00
7641a90c34
Add a test for TreeCache.__contains__
2016-02-22 15:22:38 +00:00
5c79ef9396
Test Filter.filter_rooms
...
Also check that the __repr__ method for FilterCollection does something
sensible.
2016-02-19 17:55:28 +00:00
700487a7c7
Fix flake8 warnings for tests
2016-02-19 15:34:38 +00:00
929cb0ed7d
Don't set currently_active for remote presence
2016-02-19 10:58:27 +00:00
b4796a62ee
Add unit test
2016-02-18 11:52:33 +00:00
9da9826b85
Remove old tests.
2016-02-18 10:46:16 +00:00
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
9e7900da1e
Add wheeltimer impl
2016-02-17 14:29:28 +00:00
763360594d
Mark AS users with their AS's ID
2016-02-11 17:26:42 +00:00
e557dc80b8
Merge pull request #566 from matrix-org/erikj/logcontext
...
Don't bother copying records on parent context
2016-02-10 11:41:45 +00:00
78a5482267
Typo
2016-02-09 16:23:11 +00:00
e664e9737c
Fix test
2016-02-09 14:57:43 +00:00
13ba8d878c
Fix test
2016-02-09 14:55:21 +00:00
6a9f1209df
Error if macaroon key is missing from config
...
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.
I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.
Also, add some basic tests for config generation and loading.
2016-02-05 01:58:23 +00:00
66bb255fcd
Merge pull request #556 from matrix-org/daniel/config
...
Rename config field to reflect yaml name
2016-02-03 14:55:54 +00:00
5054806ec1
Rename config field to reflect yaml name
2016-02-03 14:42:01 +00:00
f8aae79a72
Simplify get_rooms
2016-02-03 13:24:35 +00:00
f2d5ff5bf2
Fix the mock homserver used in the tests
2016-01-29 14:53:14 +00:00
b18114e19e
Merge pull request #536 from matrix-org/erikj/sync
...
Make /sync "better".
2016-01-29 13:04:51 +00:00
02a9c3be6c
Merge pull request #538 from matrix-org/erikj/fix_lru_cache
...
Fix LruCache. Make TreeCache track its own size.
2016-01-29 11:53:55 +00:00
4fce59f274
Add tests
2016-01-29 11:33:11 +00:00
35981c8b71
Fix test
2016-01-28 17:20:05 +00:00
167d1df699
Merge pull request #534 from matrix-org/erikj/setup
...
Add a Homeserver.setup method
2016-01-28 14:45:36 +00:00
4e7948b47a
Allow paginating backwards from stream token
2016-01-28 11:52:34 +00:00
61eaa6ec64
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync
2016-01-28 10:50:57 +00:00
e26390ca46
Merge pull request #535 from matrix-org/rav/paginate_from_stream_token
...
Make it possible to paginate forwards from stream tokens
2016-01-28 09:49:46 +00:00
5cba88ea7c
Make it possible to paginate forwards from stream tokens
...
In order that we can fill the gap after a /sync, make it possible to paginate
forwards from a stream token.
2016-01-27 17:42:45 +00:00
0487c9441f
Fix tests
2016-01-27 17:33:13 +00:00
9959d9ece8
Remove redundated BaseHomeServer
2016-01-26 13:52:29 +00:00
8c6012a4af
Fix tests
2016-01-25 13:12:35 +00:00
88baa3865e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync
2016-01-22 15:57:12 +00:00
d552861346
Revert all the bits changing keys of eeverything that used LRUCaches to tuples
2016-01-22 12:18:14 +00:00
10f76dc5da
Make LRU cache not default to treecache & add options to use it
2016-01-22 12:10:33 +00:00
31a051b677
Test treecache directly
2016-01-22 11:22:00 +00:00
8f9c74e9f1
Fix tests
2016-01-22 10:48:27 +00:00
4efcaa43c8
Add tests for treecache directly and test del_multi at the LruCache level too.
2016-01-22 10:37:37 +00:00
f1f8122120
Change LRUCache to be tree-based so we can delete subtrees.
2016-01-21 19:16:25 +00:00
191070123d
Cache dns lookups, and use the cache if we fail to lookup servers later
2016-01-20 11:34:09 +00:00
2c176e02ae
Make unit tests work
2016-01-18 14:48:50 +00:00
ac5a4477ad
Require unbanning before other membership changes
2016-01-15 16:27:26 +00:00
2680043bc6
Require ID and as_token be unique for ASs
...
Defaults ID to as_token if not specified. This will change
when IDs are fully supported.
2016-01-14 14:34:01 +00:00
c0a279e808
Delete the table objects from TransactionStore
2016-01-13 11:15:20 +00:00
42aa1f3f33
Merge pull request #478 from matrix-org/daniel/userobject
...
Introduce a User object
I'm sick of passing around more and more things as tuple items around
the whole world, and needing to edit every call site every time there is
more information about a user. So pass them around together as an
object.
This object has incredibly poorly named fields because we have a
convention that `user` indicates a UserID object, and `user_id`
indicates a string. I tried to clean up the whole repo to fix this, but
gave up. So instead, I introduce a second convention. A user_object is a
User, and a user_id_object is a UserId. I may have cried a little bit.
2016-01-11 17:50:22 +00:00
2110e35fd6
Introduce a Requester object
...
This tracks data about the entity which made the request. This is
instead of passing around a tuple, which requires call-site
modifications every time a new piece of optional context is passed
around.
I tried to introduce a User object. I gave up.
2016-01-11 17:48:45 +00:00
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
0e48f7f245
fix tests
2016-01-06 16:46:41 +00:00
eb03625626
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-05 18:39:50 +00:00
cfd07aafff
Allow guests to upgrade their accounts
2016-01-05 18:01:18 +00:00
92a1e74b20
fix tests
2016-01-04 14:17:35 +00:00
928c575c6f
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-04 13:39:51 +00:00
7fa71e3267
Add a unit test for the snapshot cache
2015-12-23 11:48:03 +00:00
d79e90f078
Add mocks to make tests work again
2015-12-22 17:56:56 +00:00
d547afeae0
Merge remote-tracking branch 'origin/master' into develop
2015-12-07 13:13:43 +00:00
4013216fcc
Fix mock import in tests.
...
For some reason, one test imports Mock class from mock.mock
rather than from mock.
This change fixes this error.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
2015-12-06 20:50:11 +00:00
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
7b593af7e1
rename the method in the tests as well
2015-12-01 16:06:17 +00:00
bd3de8f39a
Update tests
2015-11-17 18:38:48 +00:00
468a2ed4ec
Return non-room events from guest /events calls
2015-11-12 16:45:28 +00:00
78f6010207
Fix an issue with ignoring power_level changes on divergent graphs
...
Changes to m.room.power_levels events are supposed to be handled at a high
priority; however a typo meant that the relevant bit of code was never
executed, so they were handled just like any other state change - which meant
that a bad person could cause room state changes by forking the graph from a
point in history when they were allowed to do so.
2015-11-12 15:24:59 +00:00
e21cef9bb5
Merge pull request #359 from matrix-org/markjh/incremental_indexing
...
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
36c58b18a3
Test for background updates
2015-11-10 15:51:40 +00:00
9107ed23b7
Add a couple of unit tests for room/<x>/messages
...
... merely because I was trying to figure out how it worked, and couldn't.
2015-11-09 16:16:43 +00:00
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
4e62ffdb21
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-11-04 17:31:01 +00:00
f522f50a08
Allow guests to register and call /events?room_id=
...
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
2015-11-04 17:29:07 +00:00
33b3e04049
Merge branch 'develop' into daniel/removesomelies
...
Conflicts:
synapse/notifier.py
2015-11-04 16:01:00 +00:00
c452dabc3d
Remove the LockManager class because it wasn't being used
2015-11-04 14:08:15 +00:00
771ca56c88
Remove more unused parameters
2015-11-02 15:31:57 +00:00
0e36756383
Merge branch 'develop' into markjh/room_tags
2015-11-02 10:57:00 +00:00
fb46937413
Support clients supplying older tokens, fix short poll test
2015-10-30 16:38:35 +00:00
f69a5c9134
Fix a 500 error resulting from empty room_ids
...
POST /_matrix/client/api/v1/rooms//send/a.b.c gave a 500 error, because we
assumed that rooms always had at least one character.
2015-10-26 18:44:03 +00:00
b0ac0a9438
Merge pull request #319 from matrix-org/erikj/filter_refactor
...
Refactor api.filtering to have a Filter API
2015-10-22 13:17:10 +01:00
87da71bace
Merge pull request #314 from matrix-org/paul/event-redaction
...
Add some unit tests of prune_events()
2015-10-22 11:07:20 +01:00
45cd2b0233
Refactor api.filtering to have a Filter API
2015-10-20 15:33:25 +01:00
f510586372
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-10-20 12:00:22 +01:00
137fafce4e
Allow rejecting invites
...
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
2015-10-20 11:58:58 +01:00
9ed784098a
Invoke EventBuilder directly instead of going via the EventBuilderFactory
2015-10-19 17:42:34 +01:00
531e3aa75e
Capture __init__.py
2015-10-19 17:37:35 +01:00
a8795c9644
Use assertIn() instead of assertTrue on the 'in' operator
2015-10-19 15:24:49 +01:00
07b58a431f
Another signing test vector using an 'm.room.message' with content, so that the implementation will have to redact it
2015-10-19 15:00:52 +01:00
0aab34004b
Initial minimial hack at a test of event hashing and signing
2015-10-19 14:40:15 +01:00
aff4d850bd
Add some unit tests of prune_events()
2015-10-16 19:56:46 +01:00
8897781558
update filtering tests
2015-10-13 14:13:51 +01:00
17c80c8a3d
rename schema_prepare to prepare_database
2015-10-13 13:56:22 +01:00
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
d5e081c7ae
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/unfederatable
2015-10-02 10:33:49 +01:00
1ee3d26432
synapse/storage/_base.py:_simple_selectupdate_one was unused
2015-09-23 10:30:03 +01:00
bb4dddd6c4
Move NullSource out of synapse and into tests since it is only used by the tests
2015-09-22 18:33:34 +01:00
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
ee2d722f0f
Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left
...
SPEC-216: Allow users to view the history of rooms that they have left.
2015-09-21 14:38:13 +01:00
728d07c8c1
Merge pull request #256 from matrix-org/auth
...
Attempt to validate macaroons
2015-09-14 18:09:33 +01:00
e2054ce21a
Allow users to GET individual state events for rooms that they have left
2015-09-10 15:06:47 +01:00
bc8b25eb56
Allow users that have left the room to view the member list from the point they left
2015-09-09 15:42:16 +01:00
187320b019
Merge branch 'erikj/check_room_exists' into erikj/unfederatable
2015-09-01 15:58:10 +01:00
b345853918
Check against sender rather than event_id
2015-09-01 15:57:35 +01:00
00149c063b
Fix tests
2015-09-01 15:42:03 +01:00
e255c2c32f
s/user_id/user/g for consistency
2015-09-01 12:41:16 +01:00
3063383547
Swap out bcrypt for md5 in tests
...
This reduces our ~8 second sequential test time down to ~7 seconds
2015-08-26 15:59:32 +01:00
81450fded8
Turn TODO into thing which actually will fail
2015-08-26 13:56:01 +01:00
6a4b650d8a
Attempt to validate macaroons
...
A couple of weird caveats:
* If we can't validate your macaroon, we fall back to checking that
your access token is in the DB, and ignoring the failure
* Even if we can validate your macaroon, we still have to hit the DB to
get the access token ID, which we pretend is a device ID all over the
codebase.
This mostly adds the interesting code, and points out the two pieces we
need to delete (and necessary conditions) in order to fix the above
caveats.
2015-08-26 13:22:23 +01:00
a9d8bd95e7
Stop looking up "admin", which we never read
2015-08-25 16:29:39 +01:00
a0b181bd17
Remove completely unused concepts from codebase
...
Removes device_id and ClientInfo
device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.
This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
e8cf77fa49
Merge branch 'develop' into refresh
...
Conflicts:
synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
cecbd636e9
/tokenrefresh POST endpoint
...
This allows refresh tokens to be exchanged for (access_token,
refresh_token).
It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
13a6517d89
s/by_token/by_access_token/g
...
We're about to have two kinds of token, access and refresh
2015-08-20 16:01:29 +01:00
617501dd2a
Move token generation to auth handler
...
I prefer the auth handler to worry about all auth, and register to call
into it as needed, than to smatter auth logic between the two.
2015-08-20 11:35:56 +01:00
a0b8e5f2fe
Merge pull request #211 from matrix-org/email_in_use
...
Changes for unique emails
2015-08-20 10:04:04 +01:00
70e265e695
Re-add whitespace around caveat operators
2015-08-19 14:30:31 +01:00
ce832c38d4
Remove padding space around caveat operators
2015-08-18 17:39:26 +01:00
42e858daeb
Fix units in test
...
I made the non-test seconds instead of ms, but not the test
2015-08-18 17:38:37 +01:00
3e6fdfda00
Fix some formatting to use tuples
2015-08-18 15:18:50 +01:00
1469141023
Merge branch 'develop' into auth
2015-08-18 14:43:44 +01:00
2d3462714e
Issue macaroons as opaque auth tokens
...
This just replaces random bytes with macaroons. The macaroons are not
inspected by the client or server.
In particular, they claim to have an expiry time, but nothing verifies
that they have not expired.
Follow-up commits will actually enforce the expiration, and allow for
token refresh.
See https://bit.ly/matrix-auth for more information
2015-08-18 14:22:02 +01:00
d7451e0f22
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-12 10:30:30 +01:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
4ff0228c25
Remove call to recently removed function in mock
2015-08-11 16:56:30 +01:00
dcd5983fe4
Remove call to recently removed function in mock
2015-08-11 16:54:06 +01:00
20addfa358
Change Cache to not use *args in its interface
2015-08-07 18:32:47 +01:00
b8e386db59
Change Cache to not use *args in its interface
2015-08-07 11:52:21 +01:00
f596ff402e
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 13:37:56 +01:00
2efb93af52
Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds
2015-08-06 13:35:28 +01:00
7eea3e356f
Make @cached cache deferreds rather than the deferreds' values
2015-08-06 13:33:34 +01:00
07507643cb
Use dictionary cache to do group -> state fetching
2015-08-05 15:11:42 +01:00
c67ba143fa
Move DictionaryCache
2015-08-04 15:58:28 +01:00
e7768e77f5
Add basic dictionary cache
2015-08-04 15:56:56 +01:00
07ad03d5df
Fix tests
2015-08-04 15:18:40 +01:00
11b0a34074
Use the same reg paths as register v1 for ASes.
...
Namely this means using registration_handler.appservice_register.
2015-07-29 10:00:54 +01:00
a4d62ba36a
Fix v2_alpha registration. Add unit tests.
...
V2 Registration forced everyone (including ASes) to create a password for a
user, when ASes should be able to omit passwords. Also unbreak AS registration
in general which checked too early if the given username was claimed by an AS;
it was checked before knowing if the AS was the one doing the registration! Add
unit tests for AS reg, user reg and disabled_registration flag.
2015-07-28 17:34:12 +01:00
f0dd6d4cbd
Fix test.
2015-07-07 16:18:36 +01:00
bd1236c0ee
Consolidate duplicate code in notifier
2015-07-02 11:46:05 +01:00
ddf7979531
Add receipts_key to StreamToken
2015-07-02 11:45:44 +01:00
d88e20cdb9
Fix bug where synapse was sending AS user queries incorrectly.
...
Bug introduced in 92b20713d7
2015-06-17 17:26:03 +01:00
261ccd7f5f
Fix tests
2015-06-12 17:17:29 +01:00
3bbd0d0e09
Merge pull request #180 from matrix-org/erikj/prev_state_context
...
Don't needlessly compute prev_state
2015-06-03 17:20:56 +01:00
9dda396baa
Merge pull request #179 from matrix-org/erikj/state_group_outliers
...
Don't compute EventContext for outliers.
2015-06-03 17:20:40 +01:00
55bf90b9e4
Don't needlessly compute prev_state
2015-06-03 16:44:24 +01:00
1c3d844e73
Don't needlessly compute context
2015-06-03 16:41:51 +01:00
d8866d7277
Caches should be bound to instances.
...
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
17167898c8
Fix the presence tests
2015-05-22 16:22:54 +01:00
49a2c10279
Merge pull request #157 from matrix-org/markjh/presence_performance
...
Improve presence performance in loadtest
2015-05-22 16:04:40 +01:00
f43544eecc
Make the appservice use 'users_in_room' rather than get_room_members since it is cached
2015-05-22 11:01:28 +01:00
88f1ea36ce
Oops, get_rooms_for_user returns a namedtuple, not a room_id
2015-05-21 15:23:58 +01:00
a551c5dad7
Merge pull request #155 from matrix-org/erikj/perf
...
Bulk and batch retrieval of events.
2015-05-21 14:54:40 +01:00
8eca5bd50a
Fix the presence tests
2015-05-20 13:22:18 +01:00
0b0033c40b
Merge branch 'develop' into notifier_performance
2015-05-18 13:50:01 +01:00
131bdf9bb1
Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf
2015-05-18 10:23:37 +01:00
a2c4f3f150
Fix daedlock
2015-05-15 10:54:04 +01:00
c5d1b4986b
Remove unused arguments and doc PresenceHandler.push_update_to_clients
2015-05-14 14:59:31 +01:00
f1b83d88a3
Discard unused NotifierUserStreams
2015-05-13 16:54:02 +01:00
63878c0379
Don't bother checking for updates if the stream token hasn't advanced for a user
2015-05-13 13:42:21 +01:00
43c2e8deae
Add support for using executemany
2015-05-05 15:13:25 +01:00
327ca883ec
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-28 13:39:42 +01:00
6f8e2d517e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-27 14:41:40 +01:00
6532b6e607
Merge branch 'develop' into csauth
...
Conflicts:
synapse/http/server.py
2015-04-24 09:37:54 +01:00
e1e5e53127
Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again
2015-04-23 19:01:37 +01:00
8a785c3006
Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261)
2015-04-23 18:40:19 +01:00
cb03fafdf1
Merge branch 'develop' into csauth
2015-04-17 13:51:10 +01:00
6ba2e3df4e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-17 11:22:31 +01:00
04c7f3576e
Various minor fixes to unit-test structure around typing notifications
2015-04-15 23:32:11 +01:00
0268d40281
Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility
2015-04-15 23:09:35 +01:00
22d7a59306
Fix tests after commit 9a0579
2015-04-08 16:57:14 +01:00
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
9fa8bda099
Merge branch 'develop' into application-services-registration-script
2015-04-01 10:19:17 +01:00
09cbff174a
Fix thinko whereby events *for the AS specifically* were not passed on.
...
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
2015-03-31 16:44:45 +01:00
cf1fa59f4b
Use a sender localpart instead of a user ID.
...
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
c217504949
Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.
2015-03-31 12:07:56 +01:00
b59aa74556
Fix tests and missing returns on deferreds.
2015-03-31 11:35:45 +01:00
d33ae65efc
Remove more reg/unreg methods. Read config not database for cache.
2015-03-31 11:00:00 +01:00
4edcbcee3b
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
9ba6487b3f
Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict()
2015-03-25 19:05:34 +00:00
7ab9f91a60
Unit-test that Cache() key eviction is ordered
2015-03-25 18:50:43 +00:00
ce2766d19c
Fix tests
2015-03-24 18:56:51 +00:00
438a21c87b
Don't test exact equality of the list: as long as it has the fields we expect, that's just fine. I added the user_id (as in database pkey) and it broke: no point testing what that comes out as: it's determined by the db.
2015-03-24 18:21:54 +00:00
0f86312c4c
Pull out the cache logic from the @cached wrapper into its own class we can reuse
2015-03-20 18:25:42 +00:00
7e282a53a5
Tidy up _simple_... methods
2015-03-20 15:05:10 +00:00
d04fa1f712
Implement ServiceQueuer with tests.
2015-03-16 14:03:16 +00:00
6279285b2a
Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer.
2015-03-16 13:15:40 +00:00
835e01fc70
Minor PR comment tweaks.
2015-03-16 10:16:59 +00:00
f9232c7917
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
c1cdd7954d
Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way
2015-03-12 16:24:51 +00:00
f1fbe3e09f
Rename TimerMetric to DistributionMetric; as it could count more than just time
2015-03-12 16:24:51 +00:00
cbc0406be8
Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that
2015-03-12 16:24:51 +00:00
0e847540c3
Prometheus needs "escaped" label values
2015-03-12 16:24:51 +00:00
b3a0179d64
Bugfix to rendering output of vectored TimerMetrics
2015-03-12 16:24:51 +00:00
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
72625f2f4d
Initial hack at a TimerMetric; for storing counts + duration accumulators
2015-03-12 16:24:50 +00:00
23ab0c68c2
Implement vector CallbackMetrics
2015-03-12 16:24:50 +00:00
8664599af7
Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache
2015-03-12 16:24:50 +00:00
d8caa5454d
Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges
2015-03-12 16:24:50 +00:00
ce8b5769f7
Create the concept of a cachecounter metric; generating two counters specific to caches
2015-03-12 16:24:50 +00:00
e7420a3bef
Initial tiny attempt at (vectorable) counter metrics
2015-03-12 16:24:50 +00:00
b98cd03193
Use event IDs instead of dumping event content in the txns table.
2015-03-09 17:25:20 +00:00
21fd84dcb8
Use seconds; start gluing in the AS scheduler into the AS handler.
2015-03-09 17:01:19 +00:00
6d74e46621
Fix tests
2015-03-09 17:01:11 +00:00
0a60bbf4fa
Finish appservice txn storage impl and tests.
2015-03-09 15:53:03 +00:00
1ead1caa18
Implement create_appservice_txn with tests.
2015-03-09 13:54:20 +00:00
1c2dcf762a
Partially implement txn store methods with tests.
2015-03-09 13:10:31 +00:00
10766f1e93
Update UTs
2015-03-06 16:17:01 +00:00
2602ddc379
Apply clarity and docstrings
2015-03-06 16:16:14 +00:00
0354659f9d
Finish synapse.appservice.scheduler implementation.
...
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
2015-03-06 16:09:05 +00:00
f260cb72cd
Flesh out more stub functions.
2015-03-06 15:12:24 +00:00
0fbfe1b08a
Add more tests; fix bugs.
2015-03-06 14:36:52 +00:00
192e228a98
Start adding some tests
2015-03-06 11:50:27 +00:00
8ad024ea80
Merge pull request #93 from matrix-org/application-services-exclusive
...
Application services exclusive flag support
2015-03-02 14:56:32 +00:00
0a036944bd
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
2015-03-02 13:53:30 +00:00
de190e49d5
Add more unit tests for exclusive namespaces.
2015-02-27 11:51:06 +00:00
127efeeb68
Update unit tests to use new format.
2015-02-27 11:10:48 +00:00
0ebd632d39
Fix unit tests
2015-02-27 09:46:38 +00:00
f53fcbce97
Use cache.pop() instead of a separate membership test + del []
2015-02-23 18:30:45 +00:00
bd84755e64
Merge remote-tracking branch 'origin/develop' into performance-cache-improvements
2015-02-23 17:16:03 +00:00
e76d485e29
Allow @cached-wrapped functions to have a prefill method for setting entries
2015-02-23 15:41:54 +00:00
852816befe
Fix presence tests
2015-02-23 15:14:09 +00:00
ebc3db295b
Take named arguments to @cached() decorator, add a 'max_entries' limit
2015-02-19 18:36:02 +00:00
5f4c28d313
Update tests
2015-02-19 14:34:32 +00:00
5358966a87
Use git aware version string in User-Agent and Server headers
2015-02-18 16:52:04 +00:00
0d872f5aa6
Merge pull request #50 from matrix-org/application-services
...
Application Services
2015-02-13 15:06:14 +00:00
021d93db11
Merge pull request #62 from matrix-org/state-chache
...
State chache
2015-02-11 17:10:28 +00:00
cb43fbeeb4
Fix tests which broke when event caching was introduced.
2015-02-11 16:46:01 +00:00
f2fdcb7c4b
Merge branch 'develop' into application-services
2015-02-11 16:43:26 +00:00
f5a70e0d2e
Add a cache for get_event
2015-02-11 15:01:15 +00:00
d8324d5a2b
Add a lru cache class
2015-02-11 14:52:23 +00:00
896253e085
Factor out some of the common homeserver setup code into a
...
setup_test_homeserver function in utils.
2015-02-11 11:37:30 +00:00
c7783d6fee
Notify ASes for events sent by other users in a room which an AS user is a part of.
2015-02-11 10:36:08 +00:00
164f6b9256
Fix tests
2015-02-09 14:23:57 +00:00
ab3c897ce1
Remove unused imports.
2015-02-09 14:16:36 +00:00
5a7dd05818
Modify auth.get_user_by_req for authing appservices directly.
...
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
2015-02-09 14:14:15 +00:00
11e6b3d18b
Dependency inject ApplicationServiceApi when creating ApplicationServicesHandler.
2015-02-05 17:04:59 +00:00
951690e54d
Merge branch 'develop' into application-services
2015-02-05 14:28:03 +00:00
131e036402
Fix unit tests.
2015-02-05 13:22:20 +00:00
bc658907f0
Add unit test for appservice_handler.query_room_alias_exists
2015-02-05 11:54:36 +00:00
f0c730252f
Add unknown user ID check. Use store.get_aliases_for_room(room_id) when searching for services by alias.
2015-02-05 11:25:32 +00:00
27091f146a
Add hs_token column and generate a different token f.e application service.
2015-02-05 10:08:12 +00:00
89f2e8fbdf
Fix bug in store defer. Add more unit tests.
2015-02-04 15:21:03 +00:00
525a218b2b
Begin to add unit tests for appservice glue and regex testing.
2015-02-04 12:24:20 +00:00
e7ca813dd4
Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering.
2015-02-03 10:39:41 +00:00
1bd540ef79
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/schema/im.sql
2015-01-30 15:16:38 +00:00
471c47441d
Merge pull request #37 from matrix-org/client_v2_filter
...
Client v2 filter
2015-01-30 14:56:08 +00:00
2aaedab203
Merge branch 'develop' of github.com:matrix-org/synapse into new_state_resolution
2015-01-30 14:09:32 +00:00
2ebf795c0a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
93ed31dda2
Create a separate filter object to do the actual filtering, so that we can
...
split the storage and management of filters from the actual filter code
and don't have to load a filter from the db each time we filter an event
2015-01-29 17:45:07 +00:00
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
33391db5f8
Merge in auth changes from develop
2015-01-29 15:54:54 +00:00
9d8f798a3f
Merge changes from develop
2015-01-29 14:55:27 +00:00
38b27bd2cb
Add filter_room_state unit tests.
2015-01-29 14:28:34 +00:00
Erik Johnston