Anthropic System Design Questions

Design a Resilient Chat System

Design a real-time chat system like WhatsApp or Slack supporting one-to-one and group conversations (and, for Slack variants, workspaces, channels and threaded replies). Users send and receive messages with low latency, see…

Also reported as: Design a One-on-One Chat Service · Design a One-to-One Chat System · Design a one-to-one chat system +1 more

Design Large Model-Weight Distribution to GPU Workers

Design a system that distributes very large immutable model-weight artifacts (hundreds of GB, e.g., 500 GB) from a source to thousands of GPU workers as quickly and reliably as possible, for example when deploying a new model…

Also reported as: Design Peer-to-Peer Model Distribution Under a Shared Link Cap · Deploy a Large Model to GPU Workers · How to stream a large file to 1000 hosts fastest

Design a Dynamically Batched Inference API

Design a high-concurrency LLM inference API that dynamically batches compatible requests onto a limited GPU pool. Individual requests arrive continuously; the batching layer groups them (same model, compatible parameters) into…

Also reported as: Design an LLM Request Batching System

Design a Prompt Sharing Product

Design a prompt-sharing product where users publish prompts privately, to specific people, or publicly; others can view, run, fork or remix them. Requirements: stable immutable versions, a permission model (owner, editor, viewer,…

Also reported as: Design a prompt-sharing platform

Design a Concurrent Image Processing Service

Design an image-processing job service (e.g., resize, filter, or run a model on images) that starts with one worker and evolves safely to many concurrent processors and large batch jobs of millions of images. Requirements: job…

Also reported as: How would you scale batch image pipelines?

Design a prompt playground

Design a web-based prompt playground (like the OpenAI or Anthropic console) for developers and prompt engineers. Users write prompts, pick a model and generation parameters, run the prompt and watch the response stream in real…

Also reported as: Design An AI Playground For Very Large Prompts

Design Telemetry Ingestion and Reconcile Metric Names

Design an internal metrics monitoring platform like Datadog or Prometheus. Agents on hosts and services emit metrics (counters, gauges, histograms with tags); the system ingests millions of data points per second, stores them as…

Find a Distributed Mode Efficiently

Design an exact distributed algorithm to find the mode (most frequent value) of a huge multiset partitioned across many workers, without shipping every raw item to one machine. Discuss hash-partitioning values so each value's…

Design a Distributed Rate Limiter

Design a distributed rate limiter that protects APIs by limiting requests (or consumed units such as LLM billing tokens or storage quota) per user, API key, tenant, or IP. It must support configurable rules (e.g., 100…

Design Instagram (Feed, Photos, and Friend Recommendations)

Design the home feed of a social network like Instagram or Twitter. Users create posts (text, photos, video), follow other users, and open a personalized feed of recent posts from people they follow, possibly mixed with global or…

Scale Duplicate File Detection

Design a system that finds duplicate files across a very large file system or storage fleet (billions of files, petabytes). Return groups of files with identical content. Discuss a cheap-first filtering pipeline (group by size,…

Review and Improve a Flawed Design Document

You are given an existing architecture or design document (for example Client -> DNS -> Load Balancer -> Application Service -> Database, plus a call to a third-party API) that contains omissions and unsafe assumptions. Review…

Design guardrails and fallback for LLM reliability

Design the safety and reliability layer around an LLM-powered production system. Include input validation and prompt-injection detection, output validation (schema checks, toxicity and PII filters), policy enforcement, fallback…

Design a distributed web crawler

Design a distributed web crawler that starts from seed URLs and crawls about 10 billion pages (or all images reachable from given root URLs), with periodic recrawls. It must avoid fetching the same URL twice, obey robots.txt and…

Design a Crash-Resilient LRU Cache

Design and implement an LRU cache with fixed capacity and O(1) get/put (hash map plus doubly linked list). Then extend it: make it thread-safe for concurrent reads and writes in a web service, and make it crash-resilient so the…

Optimize HTTP requests for speed and limits

You need to make a very large number of HTTP requests to an external API as fast as possible while respecting its rate limits. Design the client: concurrency (async I/O, connection pooling), client-side rate limiting (token…