From 13577aa55ebe6087e8b813c0643bbb53148e9510 Mon Sep 17 00:00:00 2001
From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Date: Fri, 11 Jun 2021 17:13:56 +0100
Subject: [PATCH] Notes on boolean columns in database schemas (#10164)

---
 changelog.d/10164.misc              |  1 +
 docs/development/database_schema.md | 42 +++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)
 create mode 100644 changelog.d/10164.misc

diff --git a/changelog.d/10164.misc b/changelog.d/10164.misc
new file mode 100644
index 0000000000..a98f1e7c7a
--- /dev/null
+++ b/changelog.d/10164.misc
@@ -0,0 +1 @@
+Add some developer documentation about boolean columns in database schemas.
diff --git a/docs/development/database_schema.md b/docs/development/database_schema.md
index 7fe8ec63e1..20740cf5ac 100644
--- a/docs/development/database_schema.md
+++ b/docs/development/database_schema.md
@@ -93,3 +93,45 @@ Ensure postgres is installed, then run:
 
 NB at the time of writing, this script predates the split into separate `state`/`main`
 databases so will require updates to handle that correctly.
+
+## Boolean columns
+
+Boolean columns require special treatment, since SQLite treats booleans the
+same as integers.
+
+There are three separate aspects to this:
+
+ * Any new boolean column must be added to the `BOOLEAN_COLUMNS` list in
+   `scripts/synapse_port_db`. This tells the port script to cast the integer
+   value from SQLite to a boolean before writing the value to the postgres
+   database.
+
+ * Before SQLite 3.23, `TRUE` and `FALSE` were not recognised as constants by
+   SQLite, and the `IS [NOT] TRUE`/`IS [NOT] FALSE` operators were not
+   supported. This makes it necessary to avoid using `TRUE` and `FALSE`
+   constants in SQL commands.
+
+   For example, to insert a `TRUE` value into the database, write:
+
+   ```python
+   txn.execute("INSERT INTO tbl(col) VALUES (?)", (True, ))
+   ```
+
+ * Default values for new boolean columns present a particular
+   difficulty. Generally it is best to create separate schema files for
+   Postgres and SQLite. For example:
+
+   ```sql
+   # in 00delta.sql.postgres:
+   ALTER TABLE tbl ADD COLUMN col BOOLEAN DEFAULT FALSE;
+   ```
+
+   ```sql
+   # in 00delta.sql.sqlite:
+   ALTER TABLE tbl ADD COLUMN col BOOLEAN DEFAULT 0;
+   ```
+
+   Note that there is a particularly insidious failure mode here: the Postgres
+   flavour will be accepted by SQLite 3.22, but will give a column whose
+   default value is the **string** `"FALSE"` - which, when cast back to a boolean
+   in Python, evaluates to `True`.