diff options
| -rw-r--r-- | proposals/2022-01-21_read-only-db.md | 60 |
1 files changed, 60 insertions, 0 deletions
diff --git a/proposals/2022-01-21_read-only-db.md b/proposals/2022-01-21_read-only-db.md new file mode 100644 index 00000000..16d3d166 --- /dev/null +++ b/proposals/2022-01-21_read-only-db.md @@ -0,0 +1,60 @@ + +status: in-progress + +Database Read-Only Mode +======================= + +Fatcat should have the ability to run in "read-only" mode, where all web and +API requests work as usual, except no modifications to the database are +possible. + +There are at least two specific use cases for this mode: + +- during certain maintenance operations, like migrating VMs or upgrading databases +- in the currently-planned high-availability mode, the "replica" VM (database, + API, and web interface) is read-only + +This proposal is to implement this at the SQL database level (using +`default_transaction_read_only` in PostgreSQL), have the API server detect this +error and report appropriately. Additionally, the web interface will have a +configuration variable that allows a site-wide alert message which will display +on every page, which will allow indicating things like upcoming downtime, read-only +mode, and read-replica responses. + +TODO: have fatcat API server handle the specific error when a write is +attempted on a read replica database (distinct from +`default_transaction_read_only`). + + +## Configure PostgreSQL Read-Only Mode + +Connect to database with `psql` (presumably as `postgres` superuser), and run: + + ALTER DATABASE fatcat_prod SET default_transaction_read_only = true; + +Then restart the API server (to force reconnect). + +Revert with the same process, but setting to `false`. + + +## Maintenance Process + +When database maintenance is in process, of a type that requires freezing +writes to the database, first a banner should be set up via +`FATCAT_ALERT_MESSAGE` and restarting `fatcat-web` (see below). + +Then, `default_transaction_read_only=true` can be set via `psql` (see +PostgreSQL section of this doc) and `fatcat-api` restarted. + +When maintenance is done, reverse the process. + + +## Read-Only Backup Web Configuration + +On replica (backup) servers, add a permanent `FATCAT_ALERT_MESSAGE` environment +variable for the `fatcat-web` service, which will impose a loud banner in the +web interface. HTML is allowed, so the message can link elsewhere for more +details. + +If the replica is ever promoted to primary, this environment variable should be +removed and the `fatcat-web` service restarted. |