Pastebin (Share Text by Link)

0) Problem Restatement

Goldman Sachs asked (Superday): design Pastebin. Users paste text or code, click "create", and get a unique short link (e.g., paste.example/aB3xK9q). Anyone with the link can view it. Options: an expiry (10 minutes, 1 day, never), public or unlisted visibility, syntax highlighting, and a size limit (e.g., 10 MB). The interviewer expects the full flow: requirements, core entities, APIs, then the high-level design.

1) Requirements

• Functional: create a paste (text, title, language, expiry, visibility) → short link. View a paste. Optionally delete (by the owner) and list "my pastes".
• Non-functional: very read-heavy (a paste may be viewed thousands of times), low-latency reads, high availability, durable storage, and unguessable links for unlisted pastes.

1.1 Estimates

• 10M new pastes/day ≈ 115 writes/sec. Reads 10:1 → ~1,200/sec, with spikes for viral pastes.
• Average size 10 KB → 100 GB/day → ~36 TB/year, so store content in object storage, not the DB.

2) Core Entities and APIs

paste: key, owner_id?, title, language, visibility (public|unlisted), size, content_ref,
       created_at, expires_at

• POST /v1/pastes { content, title?, language?, expires_in?, visibility? } → { key, url }
• GET /v1/pastes/{key} → metadata + content (or a redirect to a CDN URL for the raw content)
• DELETE /v1/pastes/{key} (owner only)

3) High-Level Design

flowchart LR
    U["User"] --> API["Paste API"]
    API --> KG["Key generator - random base62"]
    API --> DB[("Metadata DB - key to paste")]
    API --> OS[("Object storage - content")]
    V["Viewer"] --> CDN["CDN / cache"]
    CDN --> API
    API --> C[("Redis - hot metadata")]
    JOB["Expiry cleanup job"] --> DB
    JOB --> OS

3.1 Create

1. Validate the size (≤ 10 MB) and rate-limit per IP or user.
2. Generate a key: 8 random base62 characters (62^8 ≈ 218 trillion), so it can't be guessed. On the rare collision, retry (insert with "if not exists").
3. Upload the content to object storage at pastes/{key}, then insert the metadata row (content first, so a paste never points at missing content).
4. Return the URL.

3.2 View

1. Request → CDN. Popular public pastes are cached at the edge (a TTL up to their expiry).
2. On a miss: the API reads metadata (Redis, then DB), checks expiry, and streams the content from object storage (or redirects to a signed CDN URL).
3. Syntax highlighting happens in the browser (a JS library), so the server just returns text and language.

4) Expiry and Cleanup

• On read, if now > expires_at, return 404 immediately, even before cleanup runs.
• A background job deletes expired metadata (in batches, via an index on expires_at) and the content objects, or uses the object store's lifecycle rules and the DB's TTL features.
• "Burn after reading" pastes: delete on the first successful view (a conditional delete).

5) Abuse and Safety

• Rate limits on creation, size limits, and CAPTCHAs for anonymous heavy users.
• Scan content for malware links or leaked secrets (API keys), and support takedown requests.
• Unlisted pastes aren't indexed or listed publicly, so only the random key grants access.

6) Wrap-Up

Store paste text in object storage and small metadata (key, visibility, expiry, language) in a database with a Redis cache, keyed by random 8-character base62 IDs generated with an insert-if-absent retry. Serve reads through a CDN and cache, since the load is heavily read-dominated, enforce expiry on read plus background cleanup, and protect the service with size limits, rate limiting and content scanning.