Ad Frequency Capping Service (Netflix Ads)

0) Problem Restatement

A frequency cap limits how often one person sees an ad. For example: "show this campaign to a user at most 3 times in any 24 hours" or "at most 10 times per week per line item". Showing the same ad too often annoys viewers and wastes the advertiser's money.

Design the service the ad server calls on every ad request: "for this user, which of these candidate ads are still under their caps?" It must answer in a few milliseconds, and it must count every impression (a shown ad) correctly. This was asked at Netflix many times.

1) Requirements

1.1 Functional

• Configure caps at several levels: ad (creative), ad group / line item, campaign, order or advertiser, each with a count and a window (e.g., 3 per 24h rolling, 10 per 7 days).
• Check: given a user and ~50 candidate ads, return which are allowed.
• Record: count each impression once, when it's actually shown.

1.2 Non-Functional

• Latency: check in under ~5 ms at p99, since it sits inside ad serving (whose total budget is ~100 ms).
• Throughput: 100K+ ad requests/sec.
• Accuracy: small overshoot is tolerable, while big overshoot breaks advertiser trust.
• Available: if the service fails, ad serving must still work (with a safe fallback).

1.3 Scale Estimates

• 50M viewers, and the active capped entities per user are small (tens).
• Checks: 100K requests/sec × 50 candidates × ~3 cap levels = 15M counter reads/sec, but batched per user into 1–2 round trips.
• Impressions: ~20K/sec written.

1.4 API Design

• POST /v1/fcap/check { user_id, candidates: [{ ad_id, line_item_id, campaign_id }] } → { allowed: [ad_id, ...] }
• Impressions arrive as events (Kafka) from players or ad servers: { impression_id, user_id, ad_id, line_item_id, campaign_id, ts }
• Admin: caps are stored with the campaign configuration.

2) High-Level Architecture

2.1 Overview

• Cap config cache: caps per entity, loaded in memory in the fcap service (they change rarely).
• Counter store: Redis Cluster (or a similar in-memory KV), sharded by user_id, so all of one user's counters live on one shard and one call fetches them.
• FCap check service: stateless and close to the ad servers.
• Impression consumer: reads impression events, deduplicates and increments counters.

2.2 Architecture Diagram

flowchart LR
    AS["Ad Server"] -->|"check user + candidates"| FC["FCap Service - cap config in memory"]
    FC -->|"1 batched read per user"| R[("Redis - counters by user")]
    FC -->|"allowed ads"| AS
    PL["Players / ad server"] -->|"impression events"| K[("Kafka - by user_id")]
    K --> IC["Impression consumer - dedupe"]
    IC -->|"increment buckets"| R

3) Counting in Rolling Windows

"3 times in any 24 hours" is a rolling window. Storing every impression timestamp works but is heavy. Instead, use time buckets:

• For a 24h window, keep 24 hourly buckets per (user, entity). For a 7-day window, 7 daily buckets (or 28 six-hour buckets for better precision).
• Count in window = sum of the buckets inside the window. The oldest bucket is only partly inside the window. Either count all of it (a little conservative: may block slightly early) or weight it.

4) Key Flows

4.1 Check

1. Receive user + candidates. Look up each candidate's caps (ad, line item, campaign) from in-memory config.
2. Fetch the user's relevant hashes from Redis (1–2 round trips).
3. For each candidate, compute counts per cap level. The ad is allowed only if every level is under its cap.
4. Return the allowed list. Ad selection then picks from these.

4.2 Record

1. When the ad actually plays (not just when selected), an impression event is sent to Kafka with a unique impression_id.
2. The consumer drops duplicates (a SET NX on imp:{id} with a 2-day TTL), then HINCRBY the right buckets for each cap level.

5) Deep Dive A — The race between check and record

Between the check and the recorded impression there's a small delay (seconds), so a user rapidly loading pages could see a 4th impression. Options:

• Accept small overshoot (common): usually fine for 24h caps.
• Reserve on selection: when the ad server picks an ad, increment a "pending" counter immediately, then confirm or release it when the impression is (or isn't) logged. More accurate, but more writes.
• Per-user serialization: because counters are sharded by user and requests for one user are rare, conflicts are rare in practice.

6) Deep Dive B — Failures, regions and late events

• Redis slow or down: fcap must never block ad serving. Use a strict timeout (e.g., 3 ms). On failure, either serve without caps (risk of over-frequency) or only serve uncapped or house ads. It's a business decision, and a "fail open with alerts" default is typical.
• Multi-region: users usually stay in one region, so keep a user's counters in their home region, and replicate asynchronously for failover. Small inaccuracy during failover is acceptable.
• Late impressions (offline TV apps upload later): increment the bucket for the impression's own time, not arrival time. If the window already passed, it doesn't matter.
• Cap changes: caps live in config, and counters are just counts, so raising or lowering a cap applies immediately without migrating data.

7) Trade-offs & Alternatives

8) Common Follow-up Questions

• "Household caps?" Key counters by household ID instead of (or as well as) user ID.
• "Why not a database?" Millions of counter reads per second in under 5 ms need an in-memory store. A DB can hold the audit trail of impressions.
• "How do you test accuracy?" Replay impression logs offline, compute exact rolling counts, and compare them with the bucketed counts.

9) Wrap-Up

Keep caps in memory, keep per-user rolling-window counters as time buckets in Redis sharded by user, and answer each ad request with one batched read that checks every cap level. Count only confirmed impressions, deduplicate by impression ID and bucket by event time, accept small overshoot or use reservations for stricter caps, and use strict timeouts so frequency capping can never slow down or stop ad serving.