database-migration-plan

# Database Migration Plan Skill

Produce a complete, safe database migration plan for a schema change. A migration plan is not just the SQL — it is a coordinated sequence of steps that ensures the application stays available, data stays consistent, and every step can be rolled back independently.

The expand/contract pattern is the default approach: expand the schema to support both old and new states, migrate the application, then contract to remove the old state. Never combine schema changes and data backfills in a single migration that runs during deployment.

## Required Inputs

Ask for these if not already provided:
- **Current schema state** — the DDL or description of the table(s) as they are now
- **Target schema state** — the DDL or description of what the table(s) should look like after migration
- **Migration reason** — why this change is being made (new feature, performance fix, normalization, compliance)
- **Database engine** — PostgreSQL, MySQL, SQLite, CockroachDB, etc.
- **Estimated data volume** — approximate number of rows in affected tables
- **Deployment constraints** — is any downtime allowed? What is the expected traffic level during migration? Are there multiple app instances running?
- **Rollback window** — how long after deploy can the team roll back before the migration becomes irreversible?

## Output Format

---

# Database Migration Plan: [Migration Name]

**Service:** [Name] | **Team:** [Team name]
**Author:** [Name] | **Reviewed by:** [Name / DBA]
**Date:** [Date] | **Target deploy date:** [Date]
**Database engine:** [PostgreSQL X.X / MySQL X.X]
**Ticket:** [JIRA-XXX]

---

## 1. Migration Overview

**What is changing:**
[1–2 sentences: the specific schema change — e.g. "Adding a non-nullable `organisation_id` column to the `users` table and backfilling it from the `accounts` table."]

**Why:**
[1–2 sentences: the business or technical reason driving the change.]

**Migration type:** [Additive only / Additive + backfill / Column rename / Column type change / Table restructure / Index change]

**Zero-downtime:** [Yes — using expand/contract / No — requires maintenance window — state duration]

**Estimated migration duration:**
- Expand phase: [~X minutes]
- Data backfill: [~X minutes/hours — based on X rows at Y rows/second]
- Contract phase: [~X minutes after app version deployed]

---

## 2. Backward Compatibility Analysis

Before writing a single line of SQL, assess whether each change is backward compatible with the currently deployed application code.

| Change | Backward compatible? | Risk | Notes |
|---|---|---|---|
| [e.g. Add nullable column `org_id`] | Yes | Low | Old app ignores new column |
| [e.g. Backfill `org_id`] | Yes | Medium | Old app unaffected; new app reads backfilled values |
| [e.g. Add NOT NULL constraint to `org_id`] | **No** | High | Old app that inserts without `org_id` will fail |
| [e.g. Drop old column `account_id`] | **No** | High | Old app that reads `account_id` will fail |
| [e.g. Add index on `org_id`] | Yes | Low | Additive; no breaking change |
| [e.g. Rename column] | **No** | High | Never rename in one step; use expand/contract |

**Summary:** [e.g. "This migration requires the expand/contract pattern across 3 deployment phases because steps 3 and 4 are not backward compatible."]

---

## 3. Expand/Contract Phases

### Phase Overview

```
Phase 1 — EXPAND
  Deploy migration: add new column (nullable), create new indexes
  Old app: continues to work (ignores new column)
  New app: not yet deployed
  Duration: [~X min] | Rollback: trivial — drop new column

       │
       ▼

Phase 2 — BACKFILL + DUAL-WRITE
  Deploy app update: writes to both old and new columns
  Run backfill: populate new column for existing rows
  Validate: confirm 100% of rows have non-null new column
  Duration: [~X hours depending on data volume]
  Rollback: deploy previous app version; new column is still nullable

       │
       ▼

Phase 3 — ENFORCE + SWITCH
  Deploy migration: add NOT NULL constraint, drop old column/index
  Deploy app update: reads only from new column
  Duration: [~X min] | Rollback: requires forward-fix (constraint must be dropped first)

       │
       ▼

Phase 4 — CONTRACT (optional cleanup)
  Deploy migration: drop deprecated columns, rename if needed
  Final state matches target schema
  Rollback: not recommended — contract changes are destructive
```

---

### Phase 1 — Expand Schema

**Goal:** Add the new column and structures without breaking the existing application.
**Deploy order:** Run migration first, then (optionally) deploy app.
**Application state:** Old app running; no app changes required yet.

```sql
-- Migration: 001_add_org_id_to_users.sql
BEGIN;

-- Add nullable column (safe — old app ignores it)
ALTER TABLE users
    ADD COLUMN org_id UUID NULL
        REFERENCES organisations(id) ON DELETE RESTRICT;

-- Add index NOW, not in Phase 3 — building index on large table during Phase 3 is risky
CREATE INDEX CONCURRENTLY users_org_id_idx ON users (org_id);

-- Note: CONCURRENTLY does not lock the table; safe on live traffic
-- Note: Cannot run CONCURRENTLY inside a transaction block; run separately if needed

COMMIT;
```

**Validation after Phase 1:**
```sql
-- Confirm column exists and is nullable
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name = 'users' AND column_name = 'org_id';
-- Expected: is_nullable = 'YES'

-- Confirm index exists
SELECT indexname, indexdef
FROM pg_indexes
WHERE tablename = 'users' AND indexname = 'users_org_id_idx';
```

**Rollback (Phase 1 only):**
```sql
BEGIN;
DROP INDEX CONCURRENTLY IF EXISTS users_org_id_idx;
ALTER TABLE users DROP COLUMN IF EXISTS org_id;
COMMIT;
```

---

### Phase 2 — Backfill Existing Data

**Goal:** Populate the new column for all existing rows before enforcing NOT NULL.
**When to run:** After Phase 1 is live and stable. Can be run as a background job or a one-time script.
**Application state:** Deploy app versi