Interactive Map with 100 Million+ Points

0) Problem Restatement

Google asked: design an interactive map that shows more than 100 million data points (e.g., every store, sensor or event location). At zoomed-out levels, show aggregated information (a heat map or cluster bubbles with counts). At close zoom, show individual pins that can be clicked. Panning and zooming must be smooth.

1) Why It's Hard

• 100M points × ~20 bytes ≈ 2 GB of raw coordinates. A browser can't download or draw that.
• On screen, only a few thousand things can be shown usefully anyway.

2) Map Tiles (the core idea)

• The world map is split into tiles by zoom level z: zoom 0 = 1 tile for the whole world, zoom 1 = 4 tiles, zoom z = 4^z tiles. Each tile is addressed as z/x/y.
• The client asks only for tiles covering the current screen at the current zoom (typically 10–30 tiles).
• For each tile, we precompute what to show:
• Low zoom (0–8): aggregated heat map values or cluster counts per grid cell inside the tile.
• Mid zoom: clusters (e.g., "1,240 here").
• High zoom (14+): individual points in that small area (a few hundred per tile at most).

flowchart LR
    SRC[("100M points - DB / data lake")] --> TB["Tile builder - batch, per zoom"]
    TB --> TP[("Tile pyramid - vector tiles z/x/y")]
    TP --> CDN["CDN"]
    CDN --> CL["Browser - WebGL map"]
    CL -->|"click a pin"| API["Point details API"]
    API --> SRC
    UPD["Point updates"] --> DIRTY["Mark affected tiles dirty"]
    DIRTY --> TB

3) Building the Tile Pyramid

• A batch job (e.g., Spark, or tools like tippecanoe) assigns each point to its tile at the highest zoom, then rolls up counts to parent tiles zoom by zoom (each parent = the sum of its 4 children). This is cheap once the highest level is done.
• Store tiles as vector tiles (compact binary with points or cells and attributes), so the client can style them and show tooltips.
• Total tiles: only tiles with data are stored. Empty ocean tiles are skipped.
• Serve from object storage through a CDN. Tiles are static files, so they're very cacheable and cheap.

4) Client Rendering

• Use a WebGL map library (e.g., Mapbox GL / deck.gl) that can draw tens of thousands of shapes smoothly.
• As the user zooms, fetch the new zoom's tiles and fade between levels. Prefetch the neighboring tiles.
• Clicking a pin calls a details API by point ID (the tile only carries IDs and minimal attributes).

5) Updates and Filters

• Data changes: mark the tiles containing changed points as dirty (at all zoom levels up the chain) and rebuild them incrementally, with cache-busting via tile versions.
• Filters (e.g., "only category = restaurant"): options are:
• Precompute separate pyramids for a few common filters.
• Store per-category counts in each aggregated cell, and let the client combine selected categories.
• For arbitrary filters, generate tiles dynamically from a spatially indexed database (PostGIS with ST_AsMVT, or ClickHouse with geohash cells), cached per filter.

6) Wrap-Up

Split the world into z/x/y tiles and precompute a tile pyramid: individual points at high zoom, and cluster or heat-map aggregates rolled up zoom by zoom at lower levels, stored as vector tiles and served through a CDN. The WebGL client fetches only visible tiles for the current zoom and loads point details on click. Rebuild dirty tiles incrementally on data changes, and handle filters with per-category counts, extra pyramids or cached dynamic tiles from a spatial database.