Backup and restore drills

13 November 2026 · 5 MIN READ

A backup is not a backup until you have restored it

Why untested backups are just hope, and a simple monthly drill that turns yours into a fact.

The untested backup

Almost every backup strategy has one silent failure: it has never been restored. The job runs green, the storage fills up, the dashboard is happy — and the dump inside is truncated, encrypted with a lost key, or restores into a schema nothing can read.

The failure is only discovered the day it is needed, which is the most expensive possible moment to learn it.

Schedule the drill, not just the backup

A restore drill is small: take the latest dump, restore it to a scratch instance, and check it the way an application would use it. It does not need to be a full production recovery to catch 90% of the failure modes.

the drill · restore a sample dump
# On a scratch host, not production
createdb -h localhost restore_test
pg_restore -h localhost -d restore_test latest.dump

# Then verify: row counts, a login, an order flow —
# and only afterwards: dropdb restore_test

What the drill should measure

Two numbers, in plain words. RTO: how long from “we lost it” to “users are working” — the stopwatch starts at the outage, not at the restore command. RPO: how much data the last backup is allowed to miss.

  • Time to restore, from finding the dump to verified application access.
  • Whether the documentation was enough, or the drill only worked because you already knew the answer.
  • Anything the backup silently excludes: external volumes, secrets, certificates, cron jobs.

Automate the boring parts

The drill repeats, so it belongs in a script: dump, checksum, restore to scratch, count rows, report, clean up. A Bash script on a schedule turns the drill from a calendar promise into a monitored job — and a failing drill becomes an alert instead of a surprise.

Keep the backups out of the blast radius

Backups stored with the same credentials, on the same account, in the same region as production fail together. Separate storage account, separate access, retention that something destructive cannot shorten.

The principle behind the whole sequence: pair automation with checks and recovery instructions. The backup is half the system; the proven restore is the other half.

Alvin K Sabu ›Backup and restore drillsMade to be explored.

alvin — bash — 80 × 24

Welcome to Alvin’s desktop.

Type help to explore. This is a browser simulation.

esc to close

Search this desktop

↑ ↓ to navigate ↵ to openesc to close

A little desktop tour

Click an app in the dock or a folder in the sidebar. The window controls work, too.

Search everything
Ctrl / ⌘ K
Open terminal
`
Overview / Projects / About / Toolbox / Contact
1 — 5
Close a dialog or menu
Esc

Pipeline Lab and Terminal are local demos. They never connect to production infrastructure.