How to Design a URL Shortener

A URL shortener converts long URLs into short, shareable links : like bit.ly, tinyurl, or t.co. This is one of the most common system design interview questions because it tests hashing, database design, and API design fundamentals.

1. Requirements Clarification

Always start by clarifying scope with your interviewer. This shows you think before jumping into design.

Functional Requirements

• Given a long URL, generate a short URL (e.g., https://short.ly/abc123)
• Given a short URL, redirect to the original long URL
• Support custom aliases (e.g., short.ly/my-brand)
• URLs expire after a configurable TTL (default: 5 years)

Non-Functional Requirements

• Scale: 100M URLs generated per day, 10:1 read-to-write ratio (1B redirects/day)
• Latency: Redirect in <100ms (p99)
• Availability: 99.99% uptime (redirects must never fail)
• Durability: URLs cannot be lost once created

Back-of-Envelope Estimates

Writes: 100M / day = ~1,160 writes/sec
Reads:  1B / day   = ~11,600 reads/sec
Record size: ~500 bytes (shortCode, longUrl, userId, createdAt, expiresAt)
5-year storage: 100M × 365 × 5 × 500B = ~91 TB

2. API Design

Keep the API simple. Two endpoints cover the core functionality:

POST /api/shorten
Request:  { "long_url": "https://...", "custom_alias": "my-link", "expiry": "2027-01-01" }
Response: { "short_url": "https://short.ly/abc123", "expires_at": "2027-01-01" }

GET /:shortCode
Response: 301 Redirect to original URL

301 vs 302: Use 301 (Moved Permanently) if you want browsers to cache the redirect and reduce server load. Use 302 (Found) if you need to track every redirect for analytics. Most production systems use 302.

3. High-Level Architecture

Here's the complete system at a glance:

graph LR
    Client["Client (Browser/App)"]
    LB["Load Balancer"]
    API["API Server"]
    Cache["Redis Cache"]
    DB["DynamoDB"]
    Queue["Kafka Queue"]
    Analytics["Analytics DB"]

    Client -->|"1. POST /api/shorten"| LB
    Client -->|"3. GET /:code"| LB
    LB --> API
    API -->|"2. Check cache"| Cache
    API -->|"3. Read/Write"| DB
    API -->|"4. Emit click event"| Queue
    Queue --> Analytics

    style Client fill:#FAF6EE,stroke:#E8DFC8
    style LB fill:#FAF6EE,stroke:#E8DFC8
    style API fill:#D97A2B,stroke:#B86418,color:#fff
    style Cache fill:#FAF6EE,stroke:#E8DFC8
    style DB fill:#FAF6EE,stroke:#E8DFC8
    style Queue fill:#FAF6EE,stroke:#E8DFC8
    style Analytics fill:#FAF6EE,stroke:#E8DFC8

Component Responsibilities

• Load Balancer: Distributes requests across API servers. Use L7 (application) load balancer for URL-based routing.
• API Server: Stateless. Handles URL creation, validation, and redirection. Horizontally scalable.
• Redis Cache: Stores hot URLs. With 20% cache hit rate, you avoid ~200M DB reads/day.
• DynamoDB: Primary storage. Key-value lookups by shortCode. Handles massive write throughput.
• Kafka Queue: Decouples analytics from core path. Every redirect emits an event asynchronously.

4. Sequence Diagrams

URL Creation Flow

sequenceDiagram
    participant C as Client
    participant LB as Load Balancer
    participant API as API Server
    participant DB as DynamoDB
    participant Cache as Redis

    C->>LB: POST /api/shorten {long_url}
    LB->>API: Route to server
    API->>DB: Check if long_url exists
    DB-->>API: Return existing shortCode (or null)
    alt URL already exists
        API-->>C: Return existing short_url
    else New URL
        API->>API: Generate unique shortCode (base62)
        API->>DB: Store {shortCode, longUrl, userId, expiresAt}
        API->>Cache: SET shortCode → longUrl (TTL: 5yr)
        API-->>C: Return short_url
    end

URL Redirect Flow

sequenceDiagram
    participant C as Client
    participant LB as Load Balancer
    participant API as API Server
    participant Cache as Redis
    participant DB as DynamoDB
    participant Queue as Kafka

    C->>LB: GET /abc123
    LB->>API: Route to server
    API->>Cache: GET abc123
    alt Cache hit
        Cache-->>API: Return longUrl
    else Cache miss
        API->>DB: Query shortCode = abc123
        DB-->>API: Return {longUrl, expiresAt}
        API->>Cache: SET abc123 → longUrl
    end
    API->>Queue: Emit click event {shortCode, timestamp, geo}
    API-->>C: 302 Redirect to longUrl

5. Deep Dive: Short Code Generation

This is the core algorithm. You need to generate a unique, compact, human-readable code.

Approach: Auto-Increment ID + Base62

Use a distributed ID generator (like Snowflake or a simple DB auto-increment) to get a unique integer, then encode it in base62 (a-z, A-Z, 0-9).

const BASE62 = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";

function generateShortCode(id: number): string {
  let code = "";
  while (id > 0) {
    code = BASE62[id % 62] + code;
    id = Math.floor(id / 62);
  }
  return code;
}

// Examples:
// 0  → "a"
// 61 → "Z"
// 62 → "ba"
// 3,500,000,000 → "9XqWc0"

Why Base62 Over MD5/SHA?

• Base62: 7 characters → 62^7 = 3.5 trillion unique URLs. Predictable, reversible, no collisions.
• MD5/SHA: Produce 32-char hashes. You'd need to truncate, which causes collisions. Slower. Overkill for this use case.

Collision Handling

With auto-increment IDs, collisions are impossible : each ID maps to exactly one code. If you use random generation instead, you must check the DB for uniqueness before assigning. This adds latency and complexity.

6. Deep Dive: Database Schema

DynamoDB Table

Table: urls
Partition Key: shortCode (String)

Attributes:
  - shortCode: String     // "abc123"
  - longUrl: String       // "https://very-long-url.com/..."
  - userId: String        // who created it
  - createdAt: Number     // epoch ms
  - expiresAt: Number     // epoch ms (null = never)
  - clickCount: Number    // denormalized counter

Why DynamoDB?

• Key-value lookups are fast: O(1) by partition key. Perfect for shortCode → longUrl.
• Massive write throughput: Handles 1,160 writes/sec easily with on-demand capacity.
• Horizontal scaling: Auto-scales partitions. No sharding logic needed.
• TTL support: DynamoDB natively deletes expired items, so you don't need a cron job.

7. Deep Dive: Caching Strategy

graph TD
    Request["Incoming Request"]
    CacheCheck{"Redis Cache Hit?"}
    CacheHit["Return cached longUrl"]
    DBQuery["Query DynamoDB"]
    CacheUpdate["Cache result in Redis"]
    Response["Return longUrl"]

    Request --> CacheCheck
    CacheCheck -->|"Yes"| CacheHit
    CacheCheck -->|"No"| DBQuery
    DBQuery --> CacheUpdate
    CacheUpdate --> Response

    style Request fill:#FAF6EE,stroke:#E8DFC8
    style CacheCheck fill:#D97A2B,stroke:#B86418,color:#fff
    style CacheHit fill:#D4EDDA,stroke:#28A745
    style DBQuery fill:#FAF6EE,stroke:#E8DFC8
    style CacheUpdate fill:#FAF6EE,stroke:#E8DFC8
    style Response fill:#FAF6EE,stroke:#E8DFC8

Cache Configuration

// Redis cache settings
Key:    shortCode (e.g., "abc123")
Value:  longUrl (e.g., "https://...")
TTL:    Match URL expiry (5 years for non-expiring)
Eviction: LRU (Least Recently Used)
Max Memory: 50GB (holds ~100M entries at 500B each)

Why 20% hit rate matters: With 1B reads/day, even a 20% cache hit rate saves 200M DB reads/day. That's ~2,300 fewer reads/sec on your database. For a URL shortener, the 80/20 rule applies strongly : a small number of URLs (viral links, social media) account for most traffic.

8. Scaling Considerations

Write Scaling

• DynamoDB auto-scaling: Handles traffic spikes (e.g., viral tweet with a short link) automatically.
• Distributed ID generation: Use Snowflake IDs or a Redis-based counter to avoid DB bottlenecks on ID assignment.

Read Scaling

• Redis Cluster: Shard across multiple nodes. Each node handles a portion of the keyspace.
• Read replicas: DynamoDB DAX (DynamoDB Accelerator) provides an in-memory cache layer in front of DynamoDB for microsecond-latency reads.

Geographic Distribution

Deploy API servers and Redis caches in multiple regions. Use DNS-based routing (Route 53) to direct users to the nearest region. Each region has its own Redis cache but shares the same DynamoDB table (global tables).

9. Common Interview Mistakes

• Using MD5/SHA for hash generation: These are cryptographically secure but overkill. Base62 encoding of an auto-increment ID is simpler, faster, and collision-free.
• Not handling duplicate long URLs: If the same URL is submitted twice, return the existing short code instead of creating a new one. Check by longUrl before creating.
• Ignoring rate limiting: Without rate limiting, a single user can exhaust your short code space or cause a storage spike. Limit to 100 URLs/min per user.
• Not planning for expired URLs: Use DynamoDB TTL for automatic deletion. Don't rely on a cron job : it doesn't scale.
• Forgetting analytics: Every redirect should emit an event to Kafka. Don't write analytics synchronously : it adds latency to the redirect path.

10. Comparison: Approaches at a Glance