URL Shortener (TinyURL / bit.ly)

0) Problem Restatement

Design a service that turns a long URL like https://example.com/products/shoes?color=red&size=10 into a short one like https://sho.rt/aZ3kP9. When someone opens the short link, we send them to the original URL. Users can also pick a custom alias (sho.rt/summer-sale) and set an expiry date.

This is a read-heavy system. A link is created once but may be clicked millions of times, so redirects must be very fast.

1) Requirements

1.1 Functional

• Create a short URL for a long URL.
• Redirect a short URL to the original.
• Optional: custom alias, expiry date, and click counts.
• Shortening the same URL twice (by the same user) returns the same code.

1.2 Non-Functional

• Low latency: redirect in under ~50 ms.
• High availability: broken links are very visible to users.
• Short codes should not be guessable in order (so people cannot list all links).

1.3 Scale Estimates

• 100 million new URLs per month ≈ 40 writes/sec.
• Read:write ratio 100:1 → 4,000 redirects/sec on average, peaks of 20K/sec.
• Over 5 years: 6 billion URLs × ~500 bytes ≈ 3 TB.
• Code length: base62 (a–z, A–Z, 0–9) with 7 characters gives 62^7 ≈ 3.5 trillion codes, which is plenty.

1.4 API Design

• POST /v1/urls with { long_url, custom_alias?, expires_at? } → { short_url }
• GET /{code} → HTTP 301 or 302 redirect to the long URL.
• GET /v1/urls/{code}/stats → click counts.

2) High-Level Architecture

2.1 Overview

• Write service: validates the URL, generates a code, and saves the mapping.
• Redirect service: looks up the code (cache first, then DB) and returns the redirect.
• ID generator: hands out unique numbers that we convert to base62.
• Database: a key-value store (DynamoDB or Cassandra) keyed by code, which is ideal for simple lookups at huge scale.
• Cache (Redis): keeps popular codes in memory.
• Analytics pipeline: click events go to Kafka and are counted asynchronously, so redirects stay fast.

2.2 Architecture Diagram

flowchart LR
    U["User"] -->|"POST /urls"| WS["Write Service"]
    WS --> IDG["ID Range Allocator"]
    WS --> DB[("URL Store - code to long_url")]
    V["Visitor"] -->|"GET /aZ3kP9"| RS["Redirect Service"]
    RS --> C[("Redis Cache")]
    RS -->|"cache miss"| DB
    RS -->|"click event"| K[("Kafka")]
    K --> AN["Click Counter"]

3) Data Model

Table: urls   (key-value store, partition key = code)
  code         "aZ3kP9"
  long_url     "https://example.com/products/shoes?..."
  user_id      42
  created_at   2026-09-19
  expires_at   2027-09-19 (optional)

Table: url_by_user_hash   (to return the same code for a repeated URL)
  key = hash(user_id + long_url)  →  code

4) Generating Short Codes

This is the main discussion point.

• An ID Range Allocator (backed by a small strongly consistent store like ZooKeeper or a DB row) hands each write server a block of 1 million numbers.
• Each server uses its block locally with no network calls, and asks for a new block when it runs out.
• If a server crashes, the unused part of its block is skipped. That is fine, since there are trillions of numbers.

5) Key Flows

5.1 Create

1. Validate the URL (format, not on a malware blocklist).
2. If the user shortened this exact URL before, return the existing code.
3. Take the next number from the local block, scramble it, base62-encode it, and save it.

5.2 Redirect

1. Look up the code in Redis. On a hit, redirect immediately.
2. On a miss, read the DB, store the result in Redis with a TTL, then redirect.
3. If the code is missing or expired, return 404.
4. Send a click event to Kafka without waiting for it.

6) Deep Dive — 301 vs 302, caching and hot links

• 301 (permanent): browsers cache the redirect, so repeat visits never reach us. That means less load, but we cannot count those clicks.
• 302 (temporary): every click reaches us. Good for analytics. bit.ly-style products that sell analytics use 302.
• Hot links: a link in a viral tweet can get 50K clicks/sec. Redis handles this easily, and we can also cache at the CDN edge for a few seconds.
• Cache size: about 20% of links get 80% of clicks. Caching the top 20% of daily active codes (a few GB) gives a high hit rate.
• Expiry: check expires_at on read, and run a daily cleanup job (or use a DB TTL feature) to delete old rows.

7) Trade-offs & Alternatives

8) Common Follow-up Questions

• "How do you stop abuse?" Rate-limit link creation per user or IP, and check URLs against malware lists (like Google Safe Browsing) on creation and again periodically.
• "How do you go multi-region?" Give each region its own ID ranges so no coordination is needed, replicate the URL table to all regions, and serve redirects from the nearest one.
• "Why not just use a DB auto-increment?" One database becomes the bottleneck and a single point of failure. Ranges give the same result without that.

9) Wrap-Up

Hand out unique numbers in blocks to each server, scramble and base62-encode them into 7-character codes, and store code → long_url in a key-value store. Serve redirects from a Redis cache with a DB fallback, push click events to Kafka asynchronously, and choose 302 when analytics matter.