Color Suggestion System (Adobe)

0) Problem Restatement

Adobe asked an ML engineer to design a color suggestion system, like Adobe Color. Given an input, suggest harmonious colors or palettes:

• from an image ("give me a palette from this photo"),
• from a partial palette ("I picked navy and orange; suggest 3 more"),
• or for a document ("suggest a background and text color that fit my design").

Suggestions should look good, be varied, respect accessibility (readable contrast), and improve from user feedback.

1) Approach Overview

Combine two parts:

1. Rule-based color theory (fast, explainable, works with no data): complementary, analogous, triadic, split-complementary and monochrome schemes, computed on the color wheel (HSL/HSV or the perceptual LAB/LCH color spaces).
2. Learned ranking (quality and taste): a model trained on millions of palettes people created, liked or used, which scores candidate palettes.

2) Extracting Colors from an Image

• Resize the image (e.g., to 200×200) and convert pixels to LAB color space, where distances match human perception.
• Run k-means clustering (k = 5–8) to find dominant colors. Weight by cluster size, and optionally by saliency (colors of the main subject matter more than a big blurry background).
• Remove near-duplicates (colors too close in LAB), and keep a balanced set (dominant, accent, neutral).

3) Generating and Ranking Palettes

1. Candidates: from the input colors, generate palettes using harmony rules, plus nearest neighbors from a library of popular palettes (search by palette embedding).
2. Features for each candidate: harmony type, contrast between colors, lightness spread, saturation balance, similarity to the input, popularity of similar palettes, and the user's past preferences.
3. Ranker: a gradient-boosted or small neural model predicts "likely to be saved or applied". It's trained on logs: shown vs clicked, saved or applied.
4. Re-rank for diversity (don't show 10 near-identical palettes) and accessibility: for text and background pairs, enforce WCAG contrast (e.g., at least 4.5:1).

flowchart LR
    IN["Input: image / partial palette / document"] --> EX["Color extraction - LAB k-means"]
    EX --> GEN["Candidate generation - harmony rules + similar palettes"]
    LIB[("Palette library + embeddings")] --> GEN
    GEN --> RANK["Ranker model"]
    UP[("User preferences")] --> RANK
    RANK --> RR["Re-rank: diversity, contrast rules"]
    RR --> OUT["Top palettes"]
    OUT -->|"saved / applied / ignored"| LOG[("Feedback logs")]
    LOG --> TRAIN["Retraining"]
    TRAIN --> RANK

4) Serving

• API: POST /v1/colors/suggest { image_url | colors[] | document_features, n: 10 } → palettes with hex codes and harmony labels.
• Latency target: ~200 ms. Image extraction is the costly part, so cache by image hash. The ranker is small, and candidate generation is cheap math.
• Run in the design app's backend. A tiny model could run on-device for instant results.

5) Evaluation

• Offline: ranking metrics (NDCG, recall@10) on held-out feedback, and designer ratings on a sample.
• Online A/B: palette apply rate, save rate, time to finish a design.
• Guardrails: contrast-compliance rate, and diversity of shown palettes.
• Cold start: a new user gets popular plus rule-based palettes, and personalization grows with their saves.

6) Wrap-Up

Extract dominant colors in perceptual LAB space with saliency-weighted k-means, generate candidate palettes from color-harmony rules and similar popular palettes, and rank them with a model trained on save and apply feedback plus user preferences. Re-rank for diversity and accessible contrast, cache expensive image work by hash, and evaluate with ranking metrics, designer ratings and A/B tests, retraining from feedback over time.