How to Design a News Feed System

Designing a news feed like Facebook, Twitter, or Instagram tests your understanding of fan-out strategies, caching, and real-time data distribution. The core challenge: when a user posts, how do you efficiently deliver it to millions of followers?

1. Requirements Clarification

Functional Requirements

• Post content (text, images, videos)
• Follow/unfollow users
• View personalized news feed (posts from people you follow)
• Like, comment, and share posts

Non-Functional Requirements

• Scale: 500M daily active users, 1M posts/minute
• Latency: Feed loads in <200ms
• Consistency: Eventual consistency is acceptable (feed can be a few seconds stale)

2. High-Level Architecture

graph LR
    Client["Client"]
    LB["Load Balancer"]
    PostService["Post Service"]
    FeedService["Feed Service"]
    FanOut["Fan-out Service"]
    Queue["Kafka Queue"]
    FeedCache["Feed Cache
(Redis)"]
    PostDB["Post DB
(Cassandra)"]
    UserGraph["User Graph
(Redis)"]

    Client -->|"1. Create post"| LB
    Client -->|"5. Load feed"| LB
    LB --> PostService
    LB --> FeedService
    PostService -->|"2. Store post"| PostDB
    PostService -->|"3. Publish event"| Queue
    Queue --> FanOut
    FanOut -->|"4. Push to followers"| FeedCache
    FanOut --> UserGraph
    FeedService -->|"6. Read from cache"| FeedCache
    FeedService --> FeedCache

    style Client fill:#FAF6EE,stroke:#E8DFC8
    style PostService fill:#D97A2B,stroke:#B86418,color:#fff
    style FeedService fill:#D97A2B,stroke:#B86418,color:#fff
    style FanOut fill:#FAF6EE,stroke:#E8DFC8
    style Queue fill:#FAF6EE,stroke:#E8DFC8
    style FeedCache fill:#FAF6EE,stroke:#E8DFC8
    style PostDB fill:#FAF6EE,stroke:#E8DFC8
    style UserGraph fill:#FAF6EE,stroke:#E8DFC8

Component Responsibilities

• Post Service: Handles post creation, validation, and storage. Stateless : horizontally scalable.
• Feed Service: Serves the personalized feed. Reads from Redis cache. Returns top 50 posts per request.
• Fan-out Service: The core. Receives post events from Kafka and pushes the post ID to each follower's feed cache. Decoupled from post creation.
• Feed Cache (Redis): Sorted set per user. Score = timestamp. Pre-computed top 500 posts per user for instant reads.
• User Graph: Stores follow relationships. Redis set per user: followers:{userId} = Set of follower IDs.

3. Sequence Diagrams

Post Creation + Fan-out Flow

sequenceDiagram
    participant A as User A
    participant PostSvc as Post Service
    participant DB as Cassandra
    participant Q as Kafka
    participant FanOut as Fan-out Service
    participant Cache as Redis Feed Cache
    participant B as User B (follower)
    participant C as User C (follower)

    A->>PostSvc: Create post {content, media}
    PostSvc->>DB: Store post
    PostSvc->>Q: Publish post-created event
    PostSvc-->>A: Post created (201)
    Q->>FanOut: Consume post event
    FanOut->>FanOut: Fetch follower list for User A
    FanOut->>Cache: ZADD feed:B post_id timestamp
    FanOut->>Cache: ZADD feed:C post_id timestamp
    Note over FanOut,Cache: B and C now see the post in their feed

Feed Load Flow

sequenceDiagram
    participant B as User B
    participant FeedSvc as Feed Service
    participant Cache as Redis Feed Cache
    participant DB as Cassandra

    B->>FeedSvc: GET /feed?cursor=abc
    FeedSvc->>Cache: ZREVRANGE feed:B 0 49
    alt Cache hit
        Cache-->>FeedSvc: Return top 50 post IDs
    else Cache miss
        FeedSvc->>DB: Fetch posts for user B
        DB-->>FeedSvc: Return posts
    end
    FeedSvc->>DB: Batch fetch post details
    DB-->>FeedSvc: Return post content
    FeedSvc-->>B: Return feed (posts + metadata)

4. Deep Dive: Fan-out Strategies

The central design decision. Two approaches, each with trade-offs:

graph TD
    A["User creates post"]
    Decision{"How many followers?"}
    LessThan1k["Regular user
(< 1000 followers)"]
    MoreThan1k["Celebrity
(> 1000 followers)"]
    PushModel["Fan-out on Write
Push to all followers' caches"]
    PullModel["Fan-out on Read
Fetch at feed load time"]
    Merge["Merge at read time
(pre-computed + on-demand)"]

    A --> Decision
    Decision -->|"99% of users"| LessThan1k
    Decision -->|"1% of users"| MoreThan1k
    LessThan1k --> PushModel
    MoreThan1k --> PullModel
    PushModel --> Merge
    PullModel --> Merge

    style Decision fill:#D97A2B,stroke:#B86418,color:#fff
    style PushModel fill:#D4EDDA,stroke:#28A745
    style PullModel fill:#FFF3CD,stroke:#FFC107
    style Merge fill:#FAF6EE,stroke:#E8DFC8

Fan-out on Write (Push Model)

When a user posts, immediately push the post to all their followers' feed caches. Reading the feed is a simple cache lookup : O(1) read. This works well when users have a moderate number of followers (most users).

Problem: Celebrities with millions of followers create a write amplification storm. A single post by Cristiano Ronaldo (500M followers) would require 500M cache writes.

Fan-out on Read (Pull Model)

When a user opens their feed, pull posts from all the people they follow and merge them. This avoids write amplification but makes reads expensive : potentially hundreds of database queries per feed load.

Hybrid Approach (What Facebook Uses)

Use fan-out on write for regular users (99% of users have <1000 followers). Use fan-out on read for celebrities (the top 1% with millions of followers). At feed load time, merge the pre-computed feed with celebrity posts fetched on demand.

5. Deep Dive: Feed Ranking

A chronological feed is simple but terrible for engagement. Modern feeds use ML-based ranking:

• Affinity score: How often does the user interact with this poster? (likes, comments, profile visits)
• Time decay: Recent posts rank higher. A post from 2 hours ago beats one from 2 days ago.
• Content type: Videos rank higher than text. Images rank higher than links.
• Engagement signals: Posts with many likes and comments get a boost.

// Simplified ranking score
score = (affinity × 0.4) + (engagement × 0.3) + (time_decay × 0.2) + (content_type × 0.1)

6. Deep Dive: Caching Strategy

// Redis feed cache per user
// Key: feed:{"{userId}"}
// Value: sorted set of post IDs (score = timestamp)

ZADD feed:user123 1725000000 post_abc
ZADD feed:user123 1725000060 post_def

// Fetch latest 50 posts
ZREVRANGE feed:user123 0 49

Pre-compute the top 500 posts per user in Redis. When the user scrolls past 500, fetch more from the database. This gives sub-100ms feed loads for 99% of sessions.

Cache Sizing

• 500M users × 500 posts × 8 bytes (post ID) = ~2 TB Redis
• With Redis Cluster (sharded across nodes), this fits in memory
• TTL: 7 days (old posts age out of cache)

7. Common Interview Mistakes

• Ignoring celebrity fan-out: A single post by a celebrity can crash the system if you fan-out to millions of followers on write. Always discuss the hybrid approach.
• Not caching feeds: Reading from the database on every feed load is too slow at scale.
• Over-ranking: If the algorithm is too aggressive, users miss content from friends and only see viral posts.
• Ignoring pagination: Never return the entire feed. Use cursor-based pagination for infinite scroll.
• Forgetting about unfollow: When User A unfollows User B, remove User B's posts from User A's feed cache.

8. Summary: Key Decisions