Street View Image Ingestion and Storage

0) Problem Restatement

Google asked: design the ingestion and storage system for Street View imagery. Cars (and backpacks, boats...) with camera rigs capture huge volumes of high-resolution images along with GPS, time and camera pose. The system must ingest the data, process it (stitch images into 360° panoramas, blur faces and license plates for privacy, align positions), index it by location, store petabytes durably and cheaply, and serve panoramas quickly to map users worldwide.

1) Scale Estimates

• A car captures ~7 cameras × a few frames per second × 8 hours → hundreds of GB to a few TB per car per day.
• A fleet of hundreds of cars → petabytes per month of raw data.
• Serving: millions of panorama views per day, each loading multiple image tiles.

2) Architecture

flowchart LR
    CAR["Capture vehicles - raw images + GPS/IMU"] -->|"ship drives / high-speed upload at depots"| ING["Ingestion stations"]
    ING --> RAW[("Raw storage - object store, cold tier")]
    ING --> Q[("Processing jobs queue")]
    Q --> P1["Pose alignment (GPS + IMU + SfM)"]
    P1 --> P2["Stitch 360 panoramas"]
    P2 --> P3["Privacy: detect + blur faces, plates"]
    P3 --> P4["Tile + multi-resolution encode"]
    P4 --> PUB[("Published panoramas - tiles")]
    P4 --> IDX[("Geo index - location to panorama ids")]
    PUB --> CDN["CDN"]
    U["Map users"] --> API["Panorama API"]
    API --> IDX
    U --> CDN

3) Ingestion

• Uploading TBs over cellular networks is impractical, so cars offload at depots (high-speed network) or ship drives to ingestion stations. Each capture session gets an ID, with a manifest of files and checksums.
• Verify the checksums, then write the raw data to object storage (a cheap cold tier). Raw data is kept, because better algorithms can reprocess it later.
• Metadata (session, time, GPS track, camera calibration) goes into a catalog DB.

4) Processing Pipeline (a DAG of batch jobs)

1. Pose alignment: combine GPS, the inertial sensor (IMU) and image matching (structure-from-motion) to get the precise position and orientation of every frame.
2. Stitching: merge the camera images into seamless 360° panoramas.
3. Privacy blurring: ML models detect faces and license plates, and blur them before anything is published. This is a hard requirement, and it also handles user blur requests (e.g., "blur my house").
4. Tiling: cut each panorama into tiles at multiple zoom levels (like map tiles), and encode them efficiently (WebP/AVIF).
5. Quality checks: blurry or dark images and failed stitches are flagged and dropped or re-queued.

• Each stage is idempotent and retried on failure, and progress is tracked per capture session, so a failure doesn't restart everything.
• Runs on large batch clusters (thousands of machines), prioritizing fresh areas or high-demand cities.

5) Storage and Indexing

• Tiers: raw (cold, cheapest, keep for reprocessing), processed panoramas (warm), and popular tiles cached at the CDN (hot).
• Geo index: panorama IDs indexed by location (S2 cells or geohash), plus the capture date, so "the nearest panorama to this point" and "older imagery (time travel)" are fast lookups.
• Graph of links: each panorama stores its neighbors, so users can "walk" along the street.

6) Serving

• The client asks the API for the panorama near a location → gets metadata and tile URLs → loads visible tiles from the CDN at the needed zoom (low resolution first, then sharper).
• Tiles are immutable (versioned URLs), which makes them ideal for long CDN caching.

7) Wrap-Up

Offload capture data at depots or via shipped drives, verify it and keep raw data in cold object storage with a metadata catalog. Run an idempotent, retryable batch DAG for pose alignment, stitching, mandatory face and plate blurring, quality checks and multi-resolution tiling. Publish immutable tiles behind a CDN, index panoramas by S2/geohash cell and date with neighbor links, tier storage by temperature, and reprocess from raw data when algorithms improve.