WSS
Web Specification Studio Home
On this page
Reliability & Incident ResponseRequiredUpdated

Disaster recovery and RPO/RTO

The strategies and metrics used to restore business operations after a catastrophic event, focusing on acceptable data loss (RPO) and acceptable downtime (RTO).

What is RPO and RTO in disaster recovery?

When planning for disaster recovery, most definitions explain RPO and RTO in isolation. However, to truly define RPO and RTO, you must understand how they interact during an outage. The table below covers both side-by-side.

RPO — Recovery Point ObjectiveRTO — Recovery Time Objective
The question it answersHow much data can we afford to lose?How long can we afford to be offline?
Measured inTime (seconds, minutes, hours)Time (seconds, minutes, hours)
Driven byBackup frequency / replication lagInfrastructure rebuild speed
Who sets itBusiness, with input from the data teamBusiness, with input from the ops team
Example“We can lose at most 1 hour of orders”“We must be back online within 4 hours”

A shorter RPO requires more frequent data replication. A shorter RTO requires more pre-provisioned standby infrastructure. Both cost money. That cost-time tradeoff is the entire negotiation behind disaster recovery planning.

Reference Card

  • Origin/Prior Art: Enterprise IT continuity planning in the pre-cloud era.
  • Related Practices: Multi-region architecture, Automated backups, Chaos development.
  • Key Concepts: RTO (Recovery Time Objective), RPO (Recovery Point Objective), Active-Passive failover.
  • Primary Failure Modes: Backups failing to restore during an actual crisis.

The difference between DR and HA

A bug in a deployment is a standard incident. High Availability (HA) handles that — it’s about redundancy within a single environment. Two load balancers, multiple availability zones, automatic failover between healthy instances.

Disaster Recovery is what happens when the entire environment is gone. A cloud provider region goes down for 24 hours. A ransomware attack encrypts every database in the cluster. A datacenter flood takes out all physical hardware.

DR means rebuilding the application from scratch in a secondary, unaffected location. That rebuild has two governing metrics: how much data you lost while it was down (RPO), and how long it takes to come back (RTO).

A quick analogy:

  • HA is having two elevators. If one breaks, the other takes over. You barely notice.
  • DR is what happens when the entire building burns down. You must relocate to a new building, buy new desks, and resume operations there. RPO is how many paper files burned with the building. RTO is how long it takes before anyone can work again.

A worked example

An e-commerce platform processes orders 24 hours a day. Here’s how their RPO and RTO shake out under two different backup strategies:

Strategy A: Nightly database backup at midnight

  • A disaster strikes at 11:55 PM.
  • The backup from the previous midnight is restored.
  • RPO: ~24 hours — nearly a full day of order data is gone.
  • The team must manually provision new servers, restore the backup from cold storage, and update DNS. This takes 6 hours.
  • RTO: 6 hours

Strategy B: Continuous database replication to a warm standby in another region

  • A disaster strikes at 11:55 PM.
  • The standby database is promoted to primary. It was replicating in real time.
  • RPO: under 30 seconds — only the transactions in-flight during the outage are lost.
  • Infrastructure is already provisioned in the secondary region. DNS failover is automated and takes 3 minutes.
  • RTO: 3–5 minutes

Strategy B is not just better — it’s also significantly more expensive. That’s the core negotiation.

DR strategy tiers

The industry broadly categorizes DR approaches into four tiers, ordered from cheapest to most expensive:

StrategyRPORTOHow it worksRelative cost
Backup and RestoreHours to daysHours to daysPeriodic snapshots sent to cold storage (e.g., S3 Glacier). Restore from scratch during recovery.Lowest
Pilot LightMinutes to hours30 min – 4 hoursA minimal version of the environment runs continuously in the secondary region (database replication active, compute off). Scale up compute during recovery.Low–Medium
Warm StandbySeconds to minutes5–30 minutesA scaled-down but fully functional copy runs in the secondary region at all times. Scale it up to full production during recovery.Medium–High
Active-Active (Multi-Region)Near zeroNear zeroFull production capacity runs in multiple regions simultaneously. Traffic is split across them. Failover is automatic and invisible to users.Highest

Each tier is a business decision, not a technical one. The technical team’s job is to present the cost of each tier clearly and let the business decide how much downtime risk they’re willing to pay to eliminate.

Failure modes

Schrödinger’s Backup

The most catastrophic DR failure is discovering your backups are useless during an actual emergency. The backup script silently failed for six months. The encryption key required to read the backup was stored in the datacenter that just burned down. The restore process works in theory but takes 18 hours when you expected 2.

A backup does not exist until you have successfully restored it to a fresh environment. Monthly restore tests are not optional.

Over-promising to the business

Stakeholders will always want an RPO of 0 seconds and an RTO of 0 seconds. That requires Active-Active multi-region architecture, which is exponentially more complex than a warm standby. Development leadership must anchor the conversation in cost:

“An RTO under 5 minutes requires a warm standby that costs $X per month in idle infrastructure. A 4-hour RTO costs $Y per month. Which matches the risk we’re willing to accept?”

Without a concrete cost figure, the business defaults to asking for everything.

Security during a DR event

Under the pressure of a major outage, teams routinely bypass access controls to move faster — granting admin access to everyone, opening database ports to the public internet, disabling MFA. The cure becomes worse than the disease.

DR runbooks must include pre-approved “break glass” protocols: temporary, time-limited credentials that grant emergency access without requiring someone to improvise in the middle of a crisis. Those credentials should be heavily audited and automatically revoked after a set window.

Operational guidance

Test your DR plan annually

Run a “Game Day” — a controlled simulation of total primary environment failure. Track exactly how long it takes to execute your DR runbooks and bring the system online in the secondary region.

If the drill takes 6 hours but your SLA promises a 2-hour RTO, you have three options:

  1. Rewrite the runbooks to eliminate manual steps.
  2. Automate infrastructure provisioning with tools like Terraform so recovery is a single command, not a multi-step manual process.
  3. Renegotiate the SLA with the business using the actual drill data.

Define RPO and RTO per service, not per company

Not every service has the same recovery requirements. The payment processing service might need a 15-minute RTO. The internal reporting dashboard might be fine with a 48-hour RTO. Assign tiers to services based on their business impact, and allocate DR budget accordingly. Blanket company-wide targets lead to either overspending on low-value systems or under-protecting high-value ones.

Related topics

Sources & further reading