From d77ef276fa0391aedd331fc9ddfd9956275dcd6f Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 15 Aug 2016 00:27:30 +0100 Subject: [PATCH 1/3] increase RAM reqs --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index d658670835..172dd4dfa0 100644 --- a/README.rst +++ b/README.rst @@ -95,7 +95,7 @@ Synapse is the reference python/twisted Matrix homeserver implementation. System requirements: - POSIX-compliant system (tested on Linux & OS X) - Python 2.7 -- At least 512 MB RAM. +- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org Synapse is written in python but some of the libraries is uses are written in C. So before we can install synapse itself we need a working C compiler and the From b7ffa0e2cdfd6b6568f91e0848e0f7f0d327210d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 19 Aug 2016 18:55:57 +0100 Subject: [PATCH 2/3] quick guide to synapse scalability via workers --- docs/workers.rst | 93 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/workers.rst diff --git a/docs/workers.rst b/docs/workers.rst new file mode 100644 index 0000000000..36d633e947 --- /dev/null +++ b/docs/workers.rst @@ -0,0 +1,93 @@ +Scaling synapse via workers +--------------------------- + +Synapse has experimental support for splitting out functionality into +multiple separate python processes, helping greatly with scalability. These +processes are called 'workers', and are (eventually) intended to scale +horizontally independently. + +All processes continue to share the same database instance, and as such, workers +only work with postgres based synapse deployments (sharing a single sqlite +across multiple processes is a recipe for disaster, plus you should be using +postgres anyway if you care about scalability). + +The workers communicate with the master synapse process via a synapse-specific +HTTP protocol called 'replication' - analogous to MySQL or Postgres style +database replication; feeding a stream of relevant data to the workers so they +can be kept in sync with the main synapse process and database state. + +To enable workers, you need to add a replication listener to the master synapse, e.g.:: + + listeners: + - port: 9092 + bind_address: '' + type: http + tls: false + x_forwarded: false + resources: + - names: [replication] + compress: false + +You then create a set of configs for the various worker processes. These should be +worker configuration files should be stored in a dedicated subdirectory, to allow +synctl to manipulate them. + +The current available worker applications are: + * synapse.app.pusher - handles sending push notifications to sygnal and email + * synapse.app.synchrotron - handles /sync endpoints. can scales horizontally through multiple instances. + * synapse.app.appservice - handles output traffic to Application Services + * synapse.app.federation_reader - handles receiving federation traffic (including public_rooms API) + * synapse.app.media_repository - handles the media repository. + +Each worker configuration file inherits the configuration of the main homeserver +configuration file. You can then override configuration specific to that worker, +e.g. the HTTP listener that it provides (if any); logging configuration; etc. +You should minimise the number of overrides though to maintain a usable config. + +You must specify the type of worker application (worker_app) and the replication +endpoint that it's talking to on the main synapse process (worker_replication_url). + +For instance:: + + worker_app: synapse.app.synchrotron + + # The replication listener on the synapse to talk to. + worker_replication_url: http://127.0.0.1:9092/_synapse/replication + + worker_listeners: + - type: http + port: 8083 + resources: + - names: + - client + + worker_daemonize: True + worker_pid_file: /home/matrix/synapse/synchrotron.pid + worker_log_config: /home/matrix/synapse/config/synchrotron_log_config.yaml + +...is a full configuration for a synchotron worker instance, which will expose a +plain HTTP /sync endpoint on port 8083 separately from the /sync endpoint provided +by the main synapse. + +Obviously you should configure your loadbalancer to route the /sync endpoint to +the synchotron instance(s) in this instance. + +Finally, to actually run your worker-based synapse, you must pass synctl the -a +commandline option to tell it to operate on all the worker configurations found +in the given directory, e.g.:: + + synctl -a $CONFIG/workers start + +Currently one should always restart all workers when restarting or upgrading +synapse, unless you explicitly know it's safe not to. For instance, restarting +synapse without restarting all the synchotrons may result in broken typing +notifications. + +To manipulate a specific worker, you pass the -w option to synctl:: + + synctl -w $CONFIG/workers/synchotron.yaml restart + +All of the above is highly experimental and subject to change as Synapse evolves, +but documenting it here to help folks needing highly scalable Synapses similar +to the one running matrix.org! + From 58d6c93103919f91ebb0cf3a08f0c02c90d93a8d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 19 Aug 2016 19:16:55 +0100 Subject: [PATCH 3/3] PR feedback --- docs/workers.rst | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/docs/workers.rst b/docs/workers.rst index 36d633e947..4eb05b0e59 100644 --- a/docs/workers.rst +++ b/docs/workers.rst @@ -20,7 +20,7 @@ To enable workers, you need to add a replication listener to the master synapse, listeners: - port: 9092 - bind_address: '' + bind_address: '127.0.0.1' type: http tls: false x_forwarded: false @@ -28,6 +28,10 @@ To enable workers, you need to add a replication listener to the master synapse, - names: [replication] compress: false +Under **no circumstances** should this replication API listener be exposed to the +public internet; it currently implements no authentication whatsoever and is +unencrypted HTTP. + You then create a set of configs for the various worker processes. These should be worker configuration files should be stored in a dedicated subdirectory, to allow synctl to manipulate them. @@ -65,12 +69,12 @@ For instance:: worker_pid_file: /home/matrix/synapse/synchrotron.pid worker_log_config: /home/matrix/synapse/config/synchrotron_log_config.yaml -...is a full configuration for a synchotron worker instance, which will expose a +...is a full configuration for a synchrotron worker instance, which will expose a plain HTTP /sync endpoint on port 8083 separately from the /sync endpoint provided by the main synapse. Obviously you should configure your loadbalancer to route the /sync endpoint to -the synchotron instance(s) in this instance. +the synchrotron instance(s) in this instance. Finally, to actually run your worker-based synapse, you must pass synctl the -a commandline option to tell it to operate on all the worker configurations found @@ -80,12 +84,12 @@ in the given directory, e.g.:: Currently one should always restart all workers when restarting or upgrading synapse, unless you explicitly know it's safe not to. For instance, restarting -synapse without restarting all the synchotrons may result in broken typing +synapse without restarting all the synchrotrons may result in broken typing notifications. To manipulate a specific worker, you pass the -w option to synctl:: - synctl -w $CONFIG/workers/synchotron.yaml restart + synctl -w $CONFIG/workers/synchrotron.yaml restart All of the above is highly experimental and subject to change as Synapse evolves, but documenting it here to help folks needing highly scalable Synapses similar