CASE STUDY

Reviewing an Existing Architecture for Risks

5 min read·990 words·Intermediate

How to use this case study

SDE-2 / Mid

Be able to walk a simple diagram (client, DNS, load balancer, service, database, third-party API) and name single points of failure, missing timeouts and missing monitoring.

SDE-3 / Senior

Prioritize risks by impact and likelihood, propose concrete fixes (replicas, retries with backoff, circuit breakers, caching) and explain how you would verify them.

Staff / Principal

Review a real design document critically, including unstated assumptions, data-loss risks, security and operability, and turn findings into a phased plan the team can execute.


0) Problem Restatement

Instead of designing from scratch, you are given an existing design and asked to review it. For example:

Client → DNS → Load Balancer → Application Service → Database, where the application also calls an external third-party API.

Or you get a written design document that has gaps and unsafe assumptions. Your job is to find the most important risks (scalability, reliability, security, operability), rank them, and propose a better design, plus a plan to prove the improvements work.

This tests judgment. Interviewers want a structured review, not a random list of buzzwords.

Asked at: Amazon, Anthropic, LinkedIn — 3 candidate reports between Jan 2026 and Apr 2026.

1) How to Structure the Review

Use a simple checklist and go layer by layer:

  1. Clarify first: what does the system do, how many users, what's the traffic pattern, what is "down" for the business, and what are the latency and availability targets?
  2. Follow one request end to end, and at each hop ask: what if this is slow? What if it fails? What if traffic is 10x?
  3. Group findings into availability, scalability, data safety, security, and operability (monitoring, deploys, on-call).
  4. Rank by impact × likelihood. Fix the things that can take the whole system down or lose data first.
  5. Propose fixes, then say how you would verify each one (load tests, failure drills, metrics).


2) Example Review of the Simple Diagram

2.1 Before

Architecture Diagram

flowchart LR
    C["Client"] --> DNS["DNS"]
    DNS --> LB["Single Load Balancer"]
    LB --> APP["App Service"]
    APP --> DB[("Single Database")]
    APP --> EXT["Third-party API"]

2.2 Findings (ranked)

#RiskWhy it mattersFix
1Single database, no replica or backup mentionedDB failure = full outage, possible data lossPrimary + standby replica with automatic failover, point-in-time backups, tested restores
2Third-party API called synchronously with no timeoutIf it gets slow, all app threads hang and the whole site goes downShort timeouts, retries with backoff and jitter, a circuit breaker, a fallback or async queue
3One load balancer, one zoneLB or zone failure = outageManaged LB across 2–3 availability zones; app instances spread across zones
4App tier size unknown, no autoscalingTraffic spikes overload itAt least 2–3 instances, autoscaling on CPU and latency, stateless app
5No cachingDB takes every read and becomes the bottleneckCache hot reads (Redis) and static content (CDN)
6No monitoring or alerting shownProblems found by users firstMetrics (latency, errors, saturation), logs, tracing, alerts, dashboards
7Security not describedData leaks, abuseTLS everywhere, WAF, rate limiting, secrets in a vault, least-privilege DB access
8Deploys not describedA bad deploy takes everything downRolling or canary deploys with automatic rollback

3) The Improved Design

Architecture Diagram

flowchart LR
    C["Client"] --> CDN["CDN + WAF"]
    CDN --> LB["Load Balancer - multi-zone"]
    LB --> A1["App - zone A"]
    LB --> A2["App - zone B"]
    A1 --> R[("Redis cache")]
    A2 --> R
    A1 --> P[("Primary DB")]
    A2 --> P
    P -->|"replication"| S[("Standby replica - other zone")]
    A1 --> CB["Circuit breaker + timeouts"]
    A2 --> CB
    CB --> EXT["Third-party API"]
    A1 --> Q[("Queue for non-urgent third-party calls")]

4) Deep Dive A — The third-party dependency

This is usually the most interesting finding, because it's easy to miss.

  • Timeouts: never wait forever. Set a timeout lower than your own request's deadline.
  • Retries with exponential backoff and jitter (wait 100 ms, 200 ms, 400 ms, with some randomness). Only retry safe, idempotent calls.
  • Circuit breaker: after many failures, stop calling the API for a short time and fail fast (or use a fallback). This protects your threads and gives the provider time to recover.
  • Bulkhead: give third-party calls their own limited thread or connection pool, so they can't use up all resources.
  • Make it async if the user doesn't need the answer right away (e.g., sending an email). Put it on a queue.
  • Cache responses that don't change often.
  • Ask about the provider's rate limits and SLA. Our availability can't be higher than theirs for features that depend on them.


5) Deep Dive B — Reviewing a design document

When given a written doc, look for:

  • Unstated assumptions: "the queue never loses messages", "clocks are in sync", "this runs once a day, so no concurrency". Ask what happens when each one is wrong.
  • Missing numbers: no QPS, data size or growth estimate, so you can't judge the design.
  • Data safety: backups, migrations, what happens on partial failure of multi-step writes (a need for idempotency or an outbox).
  • Operability: how will on-call know it's broken? How do we roll back?
  • Security and privacy: who can access what, and is PII encrypted and logged safely?
Then give a short, prioritized list. Don't rewrite everything. Keep what's good and explain why.


6) How to Verify the Improvements

  • Load test to 2–3x expected peak and watch latency and errors.
  • Failure drills (chaos testing): kill an app instance, fail over the DB, and make the third-party API slow. Check the system behaves as designed.
  • Backup restore test on a schedule, since a backup you never restored is not a backup.
  • SLOs and alerts: define "good" (e.g., 99.9% of requests under 300 ms) and alert when you're burning through the error budget.


7) Trade-offs to Mention

  • More replicas and zones cost more money. Match them to the business's actual availability needs.
  • Caching adds staleness and invalidation work.
  • Circuit breakers and fallbacks mean some users get a degraded experience instead of an error. Agree with product on what "degraded" looks like.


8) Wrap-Up

Review in a fixed order: clarify goals, trace a request, list risks by category, and rank them by impact. The usual top risks are a single database, a synchronous third-party call without timeouts, a single zone, and no monitoring. Fix them with replicas and backups, timeouts, retries, circuit breakers and queues, multi-zone deployment and observability, then prove the fixes with load tests and failure drills.

More Case Studies

Practice with a Mock Interview

Apply what you learned in a live system design mock interview with our AI interviewer.

Start System Design Interview →