Google NALSD: a worked SRE design example

Google's SRE Workbook describes Non-Abstract Large System Design as a method for turning requirements into a concrete resource plan. It combines design iteration with capacity estimates and failure analysis.

Use the method to practise infrastructure design. The published chapter does not establish a universal interview stage, duration or scoring rubric; ask the recruiter what your own round covers.

For this editorial exercise, assume a platform has five decimal petabytes of data, all of which must be available within a four-hour recovery time objective. The only restore path is one dedicated 10Gbps link. No data is already at the recovery site.

Clarify the acceptable recovery point, whether all data is on the critical path and whether additional links or a smaller working set are possible. These answers can change the design before detailed sizing.

Five petabytes is 5,000,000 gigabytes using decimal units. A 10Gbps link carries at most 1.25GB/s, so transfer alone takes 5,000,000 / 1.25 = 4,000,000 seconds, about 46.3 days. At 100Gbps it is about 4.63 days. Both ignore protocol overhead and storage bottlenecks.

To move all five petabytes in four hours would require about 347GB/s, or 2.78Tbps of sustained payload bandwidth, before overhead. That rules out the stated single-link restore plan. It does not prove that every network-based recovery design is impossible.

One alternative is to keep a replica at the recovery site and rehearse promotion. Another is to recover a critical subset first, if the service objective permits it. Specify how replication lag is measured and what data loss the recovery point permits.

A standby requires storage and operating capacity. Do not assume an exact doubling of the bill: redundancy, compression, storage class and compute readiness change cost. Retain independent recovery points because logical corruption can propagate to a replica.

Suppose the product owner allows the most recent 2 TB to come back first. At a measured 70 percent of a 10 Gbit/s link, payload throughput is 0.875 GB/s. Transferring 2,000 GB takes about 2,286 seconds, or 38.1 minutes. This smaller transfer can fit inside four hours; the original 5 PB transfer cannot.

That result is only the network portion of recovery. If provisioning takes 25 minutes before transfer and validation takes 40 minutes afterward, the serial path is about 103 minutes. These timings and the 70 percent figure are exercise assumptions, not Google benchmarks. Check which steps can overlap and which share a bottleneck.

The reduced working set must still satisfy the service promise. Explain how requests for older data behave, when the rest becomes available, and how you prevent inconsistent reads during catch-up. A faster restore that loses required records has not met the recovery objective.

Keep a small capacity worksheet for the hardware and services in your design. Distinguish bits from bytes, decimal from binary units, random from sequential IO and throughput from latency.

Drive specifications and network benchmarks need their workload conditions. A single memorized IOPS number cannot size an entire storage system. Use rough bounds to eliminate unsuitable options, then validate the promising design.

Take a system you know and change one constraint: remove a region, multiply traffic or reduce the budget. State an initial estimate, explain its assumptions and identify the next measurement you would need.

When a number is wrong, correct it and revisit the decisions that used it. The purpose of this practice is to make assumptions and consequences visible to the person reviewing your design.

• Google SRE preparation and source scope
• Practise reliability arithmetic
• Apply the method to design questions