8.5 Backup, restore and disaster recovery

What this is and why it exists

Everybody has backups. Far fewer people have restores, and the difference between the two is discovered on the worst possible day. This lesson turns backup from a setting into two numbers you can defend, an arrangement that survives the failure it is meant to survive, and one exercise — a timed restore — that converts a belief into a fact.

The vocabulary

• RPO (recovery point objective) — how much data you can afford to lose, measured in time: the gap between the last backup and the failure.
• RTO (recovery time objective) — how long you can afford to be down while recovering.
• Retention — how far back your copies go.
• Immutability (or object lock) — a copy that cannot be altered or deleted before a stated date, even by an administrator.
• Multi-zone — copies in separate data centres in the same region.
• Multi-region — copies in a geographically separate region.
• Failover — switching to the standby.
• Restore drill — deliberately recovering from a backup, on a normal day, with a stopwatch.

The mental model

The two numbers come first, because they decide everything else and they are business questions rather than technical ones. Recovery point asks how much work may be lost — if backups run nightly, a failure at eight in the evening loses the day. Recovery time asks how long the service may be gone. Ask both about your own product plainly: for a learning plan, losing an hour of progress is annoying and losing a day is bad; for a payment ledger, losing anything is unacceptable and continuous backup is the only answer. State the two numbers, then choose the mechanism that meets them — daily snapshots, continuous point-in-time recovery, a warm standby — rather than choosing a mechanism and discovering afterwards what it implies.

Retention is the third dimension and it protects against a different enemy. Redundancy protects against loss; retention protects against damage you did not notice — a bad migration, a bug that corrupted records for a fortnight, a deletion nobody spotted. If your only copy is from last night and the corruption began three weeks ago, you have faithfully replicated the problem. A layered pattern is the usual answer: frequent copies kept briefly, daily copies kept for weeks, monthly copies kept for the period your obligations actually require.

Immutability is what makes backups survive an attacker rather than only an accident. Ransomware operators delete backups first, precisely because that is what turns an incident into a payment. A copy under an object lock cannot be deleted before its date even by an administrator with valid credentials — which is exactly the property you want, and it is also why the setting feels uncomfortable to configure. It should. That discomfort is the guarantee.

Where copies live decides which disasters they survive, and the test is simple: a copy that shares a fate with the original is not a copy. A snapshot in the same zone survives a machine failing. Another zone survives a data centre — power, cooling, a flood. Another region survives a regional outage, and is what a compliance question about resilience usually means. Each step costs more in transfer and in complexity, so the honest approach is to match it to what you actually need to survive, and to say out loud which disasters you have decided not to survive. That last sentence is the one most plans lack, and it is the one that makes a plan honest.

And then the exercise, which is the whole lesson in one instruction: restore something, time it, and write the number down. Not a test that the backup file exists — an actual recovery into a usable system. It will reveal things nothing else does: that the restore is slower than anyone guessed, that a step needs a credential nobody has, that a piece of configuration was never backed up at all, that the runbook refers to a person who left. Every one of those is cheap to find on a Tuesday and expensive to find during an outage. Do it on a schedule, and the number you write down becomes the recovery time you can honestly promise — which, until you have measured it, is a figure somebody made up.

What you should now be able to explain or do

Define recovery point and recovery time and state both for a product you know. Say what retention protects against that redundancy does not. Explain what immutability defends against and why the setting feels uncomfortable. Say which disaster each copy location survives, and name the disasters you have chosen not to survive. Run a restore drill and produce a number. Say why the recovery time you quote is fiction until that drill has happened.

Check yourself