Commit Graph

552 Commits

Author SHA1 Message Date
Paul "LeoNerd" Evans a8795c9644 Use assertIn() instead of assertTrue on the 'in' operator 2015-10-19 15:24:49 +01:00
Paul "LeoNerd" Evans 07b58a431f Another signing test vector using an 'm.room.message' with content, so that the implementation will have to redact it 2015-10-19 15:00:52 +01:00
Paul "LeoNerd" Evans 0aab34004b Initial minimial hack at a test of event hashing and signing 2015-10-19 14:40:15 +01:00
Paul "LeoNerd" Evans aff4d850bd Add some unit tests of prune_events() 2015-10-16 19:56:46 +01:00
Mark Haines 8897781558 update filtering tests 2015-10-13 14:13:51 +01:00
Erik Johnston 17c80c8a3d rename schema_prepare to prepare_database 2015-10-13 13:56:22 +01:00
Erik Johnston ec398af41c Expose error more nicely 2015-10-13 11:43:43 +01:00
Erik Johnston d5e081c7ae Merge branch 'develop' of github.com:matrix-org/synapse into erikj/unfederatable 2015-10-02 10:33:49 +01:00
Mark Haines 1ee3d26432 synapse/storage/_base.py:_simple_selectupdate_one was unused 2015-09-23 10:30:03 +01:00
Mark Haines bb4dddd6c4 Move NullSource out of synapse and into tests since it is only used by the tests 2015-09-22 18:33:34 +01:00
Daniel Wagner-Hall 7213588083 Implement configurable stats reporting
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Mark Haines ee2d722f0f Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left
SPEC-216: Allow users to view the history of rooms that they have left.
2015-09-21 14:38:13 +01:00
Daniel Wagner-Hall 728d07c8c1 Merge pull request #256 from matrix-org/auth
Attempt to validate macaroons
2015-09-14 18:09:33 +01:00
Mark Haines e2054ce21a Allow users to GET individual state events for rooms that they have left 2015-09-10 15:06:47 +01:00
Mark Haines bc8b25eb56 Allow users that have left the room to view the member list from the point they left 2015-09-09 15:42:16 +01:00
Erik Johnston 187320b019 Merge branch 'erikj/check_room_exists' into erikj/unfederatable 2015-09-01 15:58:10 +01:00
Erik Johnston b345853918 Check against sender rather than event_id 2015-09-01 15:57:35 +01:00
Erik Johnston 00149c063b Fix tests 2015-09-01 15:42:03 +01:00
Daniel Wagner-Hall e255c2c32f s/user_id/user/g for consistency 2015-09-01 12:41:16 +01:00
Daniel Wagner-Hall 3063383547 Swap out bcrypt for md5 in tests
This reduces our ~8 second sequential test time down to ~7 seconds
2015-08-26 15:59:32 +01:00
Daniel Wagner-Hall 81450fded8 Turn TODO into thing which actually will fail 2015-08-26 13:56:01 +01:00
Daniel Wagner-Hall 6a4b650d8a Attempt to validate macaroons
A couple of weird caveats:
 * If we can't validate your macaroon, we fall back to checking that
   your access token is in the DB, and ignoring the failure
 * Even if we can validate your macaroon, we still have to hit the DB to
   get the access token ID, which we pretend is a device ID all over the
   codebase.

This mostly adds the interesting code, and points out the two pieces we
need to delete (and necessary conditions) in order to fix the above
caveats.
2015-08-26 13:22:23 +01:00
Daniel Wagner-Hall a9d8bd95e7 Stop looking up "admin", which we never read 2015-08-25 16:29:39 +01:00
Daniel Wagner-Hall a0b181bd17 Remove completely unused concepts from codebase
Removes device_id and ClientInfo

device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.

This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
Daniel Wagner-Hall e8cf77fa49 Merge branch 'develop' into refresh
Conflicts:
	synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Daniel Wagner-Hall 13a6517d89 s/by_token/by_access_token/g
We're about to have two kinds of token, access and refresh
2015-08-20 16:01:29 +01:00
Daniel Wagner-Hall 617501dd2a Move token generation to auth handler
I prefer the auth handler to worry about all auth, and register to call
into it as needed, than to smatter auth logic between the two.
2015-08-20 11:35:56 +01:00
Mark Haines a0b8e5f2fe Merge pull request #211 from matrix-org/email_in_use
Changes for unique emails
2015-08-20 10:04:04 +01:00
Daniel Wagner-Hall 70e265e695 Re-add whitespace around caveat operators 2015-08-19 14:30:31 +01:00
Daniel Wagner-Hall ce832c38d4 Remove padding space around caveat operators 2015-08-18 17:39:26 +01:00
Daniel Wagner-Hall 42e858daeb Fix units in test
I made the non-test seconds instead of ms, but not the test
2015-08-18 17:38:37 +01:00
Daniel Wagner-Hall 3e6fdfda00 Fix some formatting to use tuples 2015-08-18 15:18:50 +01:00
Daniel Wagner-Hall 1469141023 Merge branch 'develop' into auth 2015-08-18 14:43:44 +01:00
Daniel Wagner-Hall 2d3462714e Issue macaroons as opaque auth tokens
This just replaces random bytes with macaroons. The macaroons are not
inspected by the client or server.

In particular, they claim to have an expiry time, but nothing verifies
that they have not expired.

Follow-up commits will actually enforce the expiration, and allow for
token refresh.

See https://bit.ly/matrix-auth for more information
2015-08-18 14:22:02 +01:00
Erik Johnston d7451e0f22 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-12 10:30:30 +01:00
Erik Johnston 2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Daniel Wagner-Hall 4ff0228c25 Remove call to recently removed function in mock 2015-08-11 16:56:30 +01:00
Daniel Wagner-Hall dcd5983fe4 Remove call to recently removed function in mock 2015-08-11 16:54:06 +01:00
Erik Johnston 20addfa358 Change Cache to not use *args in its interface 2015-08-07 18:32:47 +01:00
Erik Johnston b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston f596ff402e Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 13:37:56 +01:00
Erik Johnston 2efb93af52 Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds 2015-08-06 13:35:28 +01:00
Erik Johnston 7eea3e356f Make @cached cache deferreds rather than the deferreds' values 2015-08-06 13:33:34 +01:00
Erik Johnston 07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
Erik Johnston c67ba143fa Move DictionaryCache 2015-08-04 15:58:28 +01:00
Erik Johnston e7768e77f5 Add basic dictionary cache 2015-08-04 15:56:56 +01:00
David Baker 07ad03d5df Fix tests 2015-08-04 15:18:40 +01:00
Kegan Dougal 11b0a34074 Use the same reg paths as register v1 for ASes.
Namely this means using registration_handler.appservice_register.
2015-07-29 10:00:54 +01:00
Kegan Dougal a4d62ba36a Fix v2_alpha registration. Add unit tests.
V2 Registration forced everyone (including ASes) to create a password for a
user, when ASes should be able to omit passwords. Also unbreak AS registration
in general which checked too early if the given username was claimed by an AS;
it was checked before knowing if the AS was the one doing the registration! Add
unit tests for AS reg, user reg and disabled_registration flag.
2015-07-28 17:34:12 +01:00
Erik Johnston f0dd6d4cbd Fix test. 2015-07-07 16:18:36 +01:00
Erik Johnston bd1236c0ee Consolidate duplicate code in notifier 2015-07-02 11:46:05 +01:00
Erik Johnston ddf7979531 Add receipts_key to StreamToken 2015-07-02 11:45:44 +01:00
Kegan Dougal d88e20cdb9 Fix bug where synapse was sending AS user queries incorrectly.
Bug introduced in 92b20713d7
which reversed the comparison when checking if a user existed
in the users table. Added UTs to prevent this happening again.
2015-06-17 17:26:03 +01:00
Erik Johnston 261ccd7f5f Fix tests 2015-06-12 17:17:29 +01:00
Erik Johnston 3bbd0d0e09 Merge pull request #180 from matrix-org/erikj/prev_state_context
Don't needlessly compute prev_state
2015-06-03 17:20:56 +01:00
Erik Johnston 9dda396baa Merge pull request #179 from matrix-org/erikj/state_group_outliers
Don't compute EventContext for outliers.
2015-06-03 17:20:40 +01:00
Erik Johnston 55bf90b9e4 Don't needlessly compute prev_state 2015-06-03 16:44:24 +01:00
Erik Johnston 1c3d844e73 Don't needlessly compute context 2015-06-03 16:41:51 +01:00
Erik Johnston d8866d7277 Caches should be bound to instances.
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
Mark Haines 17167898c8 Fix the presence tests 2015-05-22 16:22:54 +01:00
Mark Haines 49a2c10279 Merge pull request #157 from matrix-org/markjh/presence_performance
Improve presence performance in loadtest
2015-05-22 16:04:40 +01:00
Mark Haines f43544eecc Make the appservice use 'users_in_room' rather than get_room_members since it is cached 2015-05-22 11:01:28 +01:00
Mark Haines 88f1ea36ce Oops, get_rooms_for_user returns a namedtuple, not a room_id 2015-05-21 15:23:58 +01:00
Erik Johnston a551c5dad7 Merge pull request #155 from matrix-org/erikj/perf
Bulk and batch retrieval of events.
2015-05-21 14:54:40 +01:00
Mark Haines 8eca5bd50a Fix the presence tests 2015-05-20 13:22:18 +01:00
Mark Haines 0b0033c40b Merge branch 'develop' into notifier_performance 2015-05-18 13:50:01 +01:00
Erik Johnston 131bdf9bb1 Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf 2015-05-18 10:23:37 +01:00
Erik Johnston a2c4f3f150 Fix daedlock 2015-05-15 10:54:04 +01:00
Mark Haines c5d1b4986b Remove unused arguments and doc PresenceHandler.push_update_to_clients 2015-05-14 14:59:31 +01:00
Mark Haines f1b83d88a3 Discard unused NotifierUserStreams 2015-05-13 16:54:02 +01:00
Mark Haines 63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Erik Johnston 43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
David Baker 6532b6e607 Merge branch 'develop' into csauth
Conflicts:
	synapse/http/server.py
2015-04-24 09:37:54 +01:00
Paul "LeoNerd" Evans e1e5e53127 Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again 2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans 8a785c3006 Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261) 2015-04-23 18:40:19 +01:00
David Baker cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston 6ba2e3df4e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-17 11:22:31 +01:00
Paul "LeoNerd" Evans 04c7f3576e Various minor fixes to unit-test structure around typing notifications 2015-04-15 23:32:11 +01:00
Paul "LeoNerd" Evans 0268d40281 Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility 2015-04-15 23:09:35 +01:00
Erik Johnston 22d7a59306 Fix tests after commit 9a0579 2015-04-08 16:57:14 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal 9fa8bda099 Merge branch 'develop' into application-services-registration-script 2015-04-01 10:19:17 +01:00
Kegan Dougal 09cbff174a Fix thinko whereby events *for the AS specifically* were not passed on.
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
2015-03-31 16:44:45 +01:00
Kegan Dougal cf1fa59f4b Use a sender localpart instead of a user ID.
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
Kegan Dougal b59aa74556 Fix tests and missing returns on deferreds. 2015-03-31 11:35:45 +01:00
Kegan Dougal d33ae65efc Remove more reg/unreg methods. Read config not database for cache. 2015-03-31 11:00:00 +01:00
Kegan Dougal 4edcbcee3b Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans 9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans 7ab9f91a60 Unit-test that Cache() key eviction is ordered 2015-03-25 18:50:43 +00:00
David Baker ce2766d19c Fix tests 2015-03-24 18:56:51 +00:00
David Baker 438a21c87b Don't test exact equality of the list: as long as it has the fields we expect, that's just fine. I added the user_id (as in database pkey) and it broke: no point testing what that comes out as: it's determined by the db. 2015-03-24 18:21:54 +00:00
Paul "LeoNerd" Evans 0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Erik Johnston 7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Kegan Dougal d04fa1f712 Implement ServiceQueuer with tests. 2015-03-16 14:03:16 +00:00
Kegan Dougal 6279285b2a Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer. 2015-03-16 13:15:40 +00:00
Kegan Dougal 835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal f9232c7917 Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans c1cdd7954d Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f1fbe3e09f Rename TimerMetric to DistributionMetric; as it could count more than just time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans cbc0406be8 Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 0e847540c3 Prometheus needs "escaped" label values 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b3a0179d64 Bugfix to rendering output of vectored TimerMetrics 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 72625f2f4d Initial hack at a TimerMetric; for storing counts + duration accumulators 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 23ab0c68c2 Implement vector CallbackMetrics 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 8664599af7 Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans d8caa5454d Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans ce8b5769f7 Create the concept of a cachecounter metric; generating two counters specific to caches 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e7420a3bef Initial tiny attempt at (vectorable) counter metrics 2015-03-12 16:24:50 +00:00
Kegan Dougal b98cd03193 Use event IDs instead of dumping event content in the txns table. 2015-03-09 17:25:20 +00:00
Kegan Dougal 21fd84dcb8 Use seconds; start gluing in the AS scheduler into the AS handler. 2015-03-09 17:01:19 +00:00
Erik Johnston 6d74e46621 Fix tests 2015-03-09 17:01:11 +00:00
Kegan Dougal 0a60bbf4fa Finish appservice txn storage impl and tests. 2015-03-09 15:53:03 +00:00
Kegan Dougal 1ead1caa18 Implement create_appservice_txn with tests. 2015-03-09 13:54:20 +00:00
Kegan Dougal 1c2dcf762a Partially implement txn store methods with tests. 2015-03-09 13:10:31 +00:00
Kegan Dougal 10766f1e93 Update UTs 2015-03-06 16:17:01 +00:00
Kegan Dougal 2602ddc379 Apply clarity and docstrings 2015-03-06 16:16:14 +00:00
Kegan Dougal 0354659f9d Finish synapse.appservice.scheduler implementation.
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
2015-03-06 16:09:05 +00:00
Kegan Dougal f260cb72cd Flesh out more stub functions. 2015-03-06 15:12:24 +00:00
Kegan Dougal 0fbfe1b08a Add more tests; fix bugs. 2015-03-06 14:36:52 +00:00
Kegan Dougal 192e228a98 Start adding some tests 2015-03-06 11:50:27 +00:00
Kegsay 8ad024ea80 Merge pull request #93 from matrix-org/application-services-exclusive
Application services exclusive flag support
2015-03-02 14:56:32 +00:00
Erik Johnston 0a036944bd Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu 2015-03-02 13:53:30 +00:00
Kegan Dougal de190e49d5 Add more unit tests for exclusive namespaces. 2015-02-27 11:51:06 +00:00
Kegan Dougal 127efeeb68 Update unit tests to use new format. 2015-02-27 11:10:48 +00:00
Kegan Dougal 0ebd632d39 Fix unit tests 2015-02-27 09:46:38 +00:00
Paul "LeoNerd" Evans f53fcbce97 Use cache.pop() instead of a separate membership test + del [] 2015-02-23 18:30:45 +00:00
Paul "LeoNerd" Evans bd84755e64 Merge remote-tracking branch 'origin/develop' into performance-cache-improvements 2015-02-23 17:16:03 +00:00
Paul "LeoNerd" Evans e76d485e29 Allow @cached-wrapped functions to have a prefill method for setting entries 2015-02-23 15:41:54 +00:00
Erik Johnston 852816befe Fix presence tests 2015-02-23 15:14:09 +00:00
Paul "LeoNerd" Evans ebc3db295b Take named arguments to @cached() decorator, add a 'max_entries' limit 2015-02-19 18:36:02 +00:00
Erik Johnston 5f4c28d313 Update tests 2015-02-19 14:34:32 +00:00
Erik Johnston 5358966a87 Use git aware version string in User-Agent and Server headers 2015-02-18 16:52:04 +00:00
Mark Haines 0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Erik Johnston 021d93db11 Merge pull request #62 from matrix-org/state-chache
State chache
2015-02-11 17:10:28 +00:00
Kegan Dougal cb43fbeeb4 Fix tests which broke when event caching was introduced. 2015-02-11 16:46:01 +00:00
Kegan Dougal f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Mark Haines f5a70e0d2e Add a cache for get_event 2015-02-11 15:01:15 +00:00
Mark Haines d8324d5a2b Add a lru cache class 2015-02-11 14:52:23 +00:00
Mark Haines 896253e085 Factor out some of the common homeserver setup code into a
setup_test_homeserver function in utils.
2015-02-11 11:37:30 +00:00
Kegan Dougal c7783d6fee Notify ASes for events sent by other users in a room which an AS user is a part of. 2015-02-11 10:36:08 +00:00
Erik Johnston 164f6b9256 Fix tests 2015-02-09 14:23:57 +00:00
Kegan Dougal ab3c897ce1 Remove unused imports. 2015-02-09 14:16:36 +00:00
Kegan Dougal 5a7dd05818 Modify auth.get_user_by_req for authing appservices directly.
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
2015-02-09 14:14:15 +00:00
Kegan Dougal 11e6b3d18b Dependency inject ApplicationServiceApi when creating ApplicationServicesHandler. 2015-02-05 17:04:59 +00:00