Fix database migrations failing on platforms where SQLite is configured with SQLITE_DBCONFIG_DEFENSIVE by default, such as macOS. (#19690)

Fixes: #19616

This caused 2+ people trouble now, so worth batting away with a
low-effort change if we can.

Only seen on macOS so far, but nothing stops SQLite being configured in
defensive mode by default on other platforms, so it is not necessarily
entirely specific to macOS.

We *could* also do this for Python < 3.12 but it'd be more effort and I
don't know if it's worth it.

(For context @kegsay says the interpreter with this problem was
installed through `pyenv install`.)

---------

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>

changelog.d/19690.bugfix

@@ -0,0 +1 @@
+Fix database migrations failing on platforms where SQLite is configured with `SQLITE_DBCONFIG_DEFENSIVE` by default, such as macOS.

docs/development/contributing_guide.md

@@ -309,21 +309,6 @@ works without further arguments).
 Your Postgres account needs to be able to create databases; see the postgres
 docs for [`ALTER ROLE`](https://www.postgresql.org/docs/current/sql-alterrole.html).
 
-### Running the tests on MacOS
-
-To run the unit tests with SQLite on MacOS you will need to swap out the default MacOS
-python interpreter with the homebrew version. This is due to the default using a
-locked down version of Python and sqlite3. After installing python from homebrew:
-
-```
-  poetry env remove --all
-  poetry env use /opt/homebrew/bin/python3.13
-  poetry install
-  poetry run trial tests
-```
-
-This example uses python 3.13, but choose whichever version you want.
-
 ## Run the integration tests ([Sytest](https://github.com/matrix-org/sytest)).
 
 The integration tests are a more comprehensive suite of tests. They

synapse/storage/engines/sqlite.py

@@ -21,6 +21,7 @@
 import platform
 import sqlite3
 import struct
+import sys
 import threading
 from typing import TYPE_CHECKING, Any, Mapping
 
@@ -90,6 +91,20 @@ class Sqlite3Engine(BaseDatabaseEngine[sqlite3.Connection, sqlite3.Cursor]):
         # We need to import here to avoid an import loop.
         from synapse.storage.prepare_database import prepare_database
 
+        if sys.version_info >= (3, 12):
+            # Opportunistically disable the SQLITE_DBCONFIG_DEFENSIVE
+            # flag on the database, as some of our database migrations
+            # alter the schema and this is forbidden in defensive mode.
+            #
+            # This is only known to be necessary on macOS, though SQLite can
+            # in theory be configured to be defensive by default on any
+            # platform.
+            #
+            # The constant for this is only exposed in the sqlite3 module
+            # on Python >= 3.12.
+            assert isinstance(db_conn.conn, sqlite3.Connection)
+            db_conn.conn.setconfig(sqlite3.SQLITE_DBCONFIG_DEFENSIVE, False)
+
         if self._is_in_memory:
             # In memory databases need to be rebuilt each time. Ideally we'd
             # reuse the same connection as we do when starting up, but that