Caching Strategies Explained

Caching stores frequently accessed data in fast storage (memory) to avoid expensive database reads. A well-designed cache can reduce response times from 100ms to sub-millisecond and cut database load by 90%+.

1. Why Caching Matters

graph LR
    Client["Client"]
    App["Application"]
    Cache["Cache
(Redis)
~1ms"]
    DB["Database
~100ms"]

    Client --> App
    App -->|"1. Check cache"| Cache
    Cache -->|"Hit: return cached data"| App
    Cache -->|"Miss: query DB"| App
    App -->|"2. On miss"| DB
    DB -->|"3. Populate cache"| Cache

    style Client fill:#FAF6EE,stroke:#E8DFC8
    style App fill:#D97A2B,stroke:#B86418,color:#fff
    style Cache fill:#D4EDDA,stroke:#28A745
    style DB fill:#FFF3CD,stroke:#FFC107

• Latency reduction: Redis reads take ~1ms vs ~100ms for a database query
• Throughput increase: Cache handles 100K+ reads/sec per node vs 1K-10K for databases
• Cost reduction: Fewer database queries = smaller database instances = lower cloud bills

2. Cache-Aside (Lazy Loading)

The most common pattern. The application is responsible for both reading and writing to the cache.

sequenceDiagram
    participant App as Application
    participant Cache as Redis Cache
    participant DB as Database

    App->>Cache: GET user:123
    alt Cache hit
        Cache-->>App: Return user data
    else Cache miss
        App->>DB: SELECT * FROM users WHERE id=123
        DB-->>App: Return user data
        App->>Cache: SET user:123 (TTL: 1hr)
    end
    App-->>Client: Return user data

function getUser(id: string) {
  // 1. Check cache
  let user = cache.get(`user:${id}`);
  if (user) return user;

  // 2. Cache miss : read from DB
  user = db.query("SELECT * FROM users WHERE id = ?", [id]);

  // 3. Populate cache with TTL
  cache.set(`user:${id}`, user, { ttl: 3600 });

  return user;
}

Pros: Only requested data is cached. Failed cache doesn't break the app (falls through to DB). Cons: Cache miss causes extra latency. Cold cache after restart means a thundering herd of DB reads.

3. Write-Through

The application writes to both the cache and the database simultaneously. The cache is always up-to-date.

sequenceDiagram
    participant App as Application
    participant Cache as Redis Cache
    participant DB as Database

    App->>DB: UPDATE users SET name="John" WHERE id=123
    App->>Cache: SET user:123 {name: "John"}
    Note over Cache,DB: Both writes happen synchronously
    App-->>Client: Write complete

function updateUser(id: string, data: UserUpdate) {
  db.update("UPDATE users SET ... WHERE id = ?", [id, data]);
  cache.set(`user:${id}`, data, { ttl: 3600 });
}

Pros: Cache is always consistent. No cache miss on reads. Cons: Write latency increases (two writes). Data that is never read still gets written to cache (wasted memory).

4. Write-Behind (Write-Back)

The application writes to the cache only. A background process asynchronously writes to the database. This reduces write latency dramatically.

sequenceDiagram
    participant App as Application
    participant Cache as Redis Cache
    participant Queue as Background Queue
    participant DB as Database

    App->>Cache: SET user:123 {name: "John"}
    Cache->>Queue: Queue DB write
    App-->>Client: Write complete (~1ms)
    Queue->>DB: Batch write (async)
    Note over Queue,DB: Happens in background, non-blocking

function updateUser(id: string, data: UserUpdate) {
  cache.set(`user:${id}`, data, { ttl: 3600 });
  // Background job writes to DB
  queue.push({ type: "db-write", id, data });
}

Pros: Extremely fast writes. Batches multiple updates. Cons: Risk of data loss if cache crashes before flush. Complexity of background processing.

5. Cache Invalidation

The hardest problem in computer science. When data changes, you must invalidate (remove) the cached version so stale data is not served.

graph TD
    Write["Data changes in DB"]
    Strategy{"Invalidation strategy?"}
    TTL["Time-based (TTL)
Set expiry time
Simple but stale window"]
    Event["Event-based
Listen to DB change events
Immediate invalidation"]
    Version["Version-based
Embed version in cache key
Old keys never accessed"]

    Write --> Strategy
    Strategy --> TTL
    Strategy --> Event
    Strategy --> Version

    style Write fill:#D97A2B,stroke:#B86418,color:#fff
    style Strategy fill:#FAF6EE,stroke:#E8DFC8
    style TTL fill:#FAF6EE,stroke:#E8DFC8
    style Event fill:#FAF6EE,stroke:#E8DFC8
    style Version fill:#FAF6EE,stroke:#E8DFC8

• Time-based expiry (TTL): Set an expiration time. Simple but may serve stale data until TTL expires.
• Event-based invalidation: Listen to database change events (CDC) and invalidate the corresponding cache key immediately.
• Version-based: Embed a version in the cache key (e.g., user:123:v3). When data changes, increment the version. Old keys are never accessed again.

6. Thundering Herd Problem

When a popular cache key expires, thousands of requests hit the DB simultaneously. The cache was absorbing all the traffic, and now the DB must handle it all at once.

graph TD
    CacheExpire["Popular key expires"]
    ThunderingHerd["1000s of requests
hit DB simultaneously"]
    DBOverload["DB overloaded
response time spikes"]
    Solution{"Solution?"}
    Mutex["Mutex lock
Only 1 request populates cache"]
    SWR["Stale-while-revalidate
Serve stale, refresh async"]

    CacheExpire --> ThunderingHerd
    ThunderingHerd --> DBOverload
    DBOverload --> Solution
    Solution --> Mutex
    Solution --> SWR

    style CacheExpire fill:#F8D7DA,stroke:#DC3545
    style ThunderingHerd fill:#F8D7DA,stroke:#DC3545
    style DBOverload fill:#F8D7DA,stroke:#DC3545
    style Solution fill:#D97A2B,stroke:#B86418,color:#fff
    style Mutex fill:#D4EDDA,stroke:#28A745
    style SWR fill:#D4EDDA,stroke:#28A745

Solutions:

• Mutex lock: Only one request populates the cache. Others wait or serve stale data.
• Stale-while-revalidate: Serve the stale cached value while refreshing in the background.
• Pre-warming: Refresh cache before it expires (e.g., at 80% TTL).

7. Redis vs Memcached

8. Common Interview Mistakes

• Not planning for cache invalidation: Stale data is worse than no cache. Always discuss your invalidation strategy.
• Ignoring thundering herd: When a popular cache key expires, thousands of requests hit the DB. Use mutex or stale-while-revalidate.
• Caching everything: Cache only hot data. Caching cold data wastes memory and adds complexity.
• Forgetting about cache size: Set max memory and eviction policy (LRU, LFU). Don't let the cache grow unbounded.
• Not monitoring cache hit rate: A hit rate below 80% means your caching strategy needs work.

9. Summary