Recommendation System (Candidate Generation + Ranking)

0) Problem Restatement

Design a system that recommends items to users: posts, videos, games, local sports teams or weekly deals. When a user opens the app, we must pick the best 20–50 items out of millions, in about 100–200 ms, and the picks should feel personal, fresh and varied.

The standard answer splits the work into stages. Candidate generation quickly finds a few hundred possibly good items. Ranking then scores those few hundred carefully with a machine learning model.

1) Requirements

1.1 Functional

• Return a ranked list of items for a user (and optional context: location, time, page).
• Learn from user actions: views, clicks, likes, watch time, purchases, skips.
• Handle new users and new items (cold start).
• Avoid showing the same thing again and again. Keep variety.

1.2 Non-Functional

• Latency: under ~200 ms end to end.
• Scale: 100M daily users, peaks of 100K requests/sec.
• Freshness: new items and new user behavior should show up within minutes to hours.

1.3 Scale Estimates

• 50M items in the catalog. Scoring all of them per request is impossible (50M × 100K/sec), which is why candidate generation exists.
• Interaction events: 100M users × 50 actions = 5B events/day feeding training.

1.4 API Design

• GET /v1/recommendations?user_id=42&surface=home&limit=30 → [{ item_id, score, reason }]
• Events are logged separately: POST /v1/events { user_id, item_id, action, ts }

2) High-Level Architecture

2.1 Overview

• Event logging: user actions go to Kafka, then to a data lake.
• Offline training (daily): trains models such as a two-tower embedding model for candidates and a ranking model (gradient-boosted trees or a neural net).
• Embedding index: item vectors stored in an approximate nearest neighbor index (FAISS, ScaNN). This finds the items whose vectors are closest to the user's vector very quickly.
• Feature store: precomputed user and item features (e.g., "user's favorite categories", "item click rate last hour") for fast lookup.
• Recommendation service: collects candidates, fetches features, ranks, re-ranks, and returns.

2.2 Architecture Diagram

flowchart LR
    U["User opens app"] --> RS["Recommendation Service"]
    RS --> CG1["Candidates: embedding ANN"]
    RS --> CG2["Candidates: follows / friends"]
    RS --> CG3["Candidates: trending / local"]
    RS --> FS[("Feature Store")]
    RS --> RK["Ranking model server"]
    RK --> RR["Re-rank: diversity, filters"]
    RR --> U
    U -->|"clicks, views"| K[("Kafka")]
    K --> DL[("Data lake")]
    DL --> TR["Offline training"]
    TR --> CG1
    TR --> RK
    K --> FS

3) Stage 1 — Candidate Generation

Gather ~500–1,000 candidates from several simple sources:

• Collaborative filtering / embeddings: "people who liked what you liked also liked X". A two-tower model turns each user and item into a vector, and similar vectors mean a good match. Find the nearest items with the ANN index.
• Social: items from friends or accounts you follow.
• Content-based: items similar to ones you recently engaged with (same category, same artist).
• Context: trending now, popular near your location (e.g., local sports teams), new arrivals.

Using several sources makes the result robust. If one source is weak for a user, others fill the gap.

4) Stage 2 — Ranking

• For each candidate, fetch features: user features, item features, and user–item features (e.g., "user clicked this category 12 times this week").
• The ranking model predicts probabilities: will the user click, watch, like? Combine them into one score, e.g., score = 0.6 × P(click) + 0.4 × P(watch 30s).
• Sort by score.

• Diversity: no more than 2 items in a row from the same creator or category.
• Freshness: give a small boost to new items.
• Filters: remove already-seen, blocked or unavailable items.

5) Deep Dive A — Cold start

• New user: no history yet. Use context (location, device, sign-up answers such as "pick 3 interests") and popular items, then learn quickly from the first few clicks.
• New item: no clicks yet. Use its content (title, category, image embedding) to place it near similar items, and give it a small amount of exploration traffic so it gets a chance to prove itself.

6) Deep Dive B — Freshness and feedback loops

• Near-real-time features: stream processors update counters such as "item clicks in the last hour" in the feature store, so trending items rise quickly.
• Session signals: include the user's last few actions in the request, so recommendations react within the same session.
• Feedback loops: the model only learns from what it showed. Without care, popular items get more popular and new ones never appear. Fix this by showing a small percentage of exploratory items and logging which items were shown, not just clicked.
• Measurement: offline metrics (AUC, recall@K) help, but the real decision comes from A/B tests on engagement, retention and satisfaction.

7) Trade-offs & Alternatives

8) Common Follow-up Questions

• "How do you keep latency low?" Run candidate sources in parallel, cap candidates at ~500, batch feature lookups, and cache recommendations for a few minutes.
• "How do you explain recommendations?" Keep the candidate source as a reason ("Because you watched X", "Popular near you").
• "How do you avoid training/serving skew?" Compute features once in the feature store and log the exact features used at serving time for training.

9) Wrap-Up

Use two stages: gather a few hundred candidates from embeddings, the social graph, content similarity and trending lists, then rank them with an ML model using features from a feature store. Re-rank for diversity and freshness, solve cold start with content and exploration, and judge changes with A/B tests.