Incident Ticket Correlation Platform

0) Problem Restatement

Microsoft asked: design a ticket platform where every new support ticket is:

1. linked to the affected user,
2. correlated with all other relevant tickets for that user (and ideally similar tickets from other users),
3. mapped to the responsible system (the service or component that's probably broken),
4. enriched with that system's operational status at the time (was there an outage, a deploy, alerts?).

It must work under high concurrency: when a big service fails, thousands of tickets arrive within minutes.

1) Requirements

• Ingest tickets from email, portal, chat and API.
• Identify the user and tenant (from login, email, account ID).
• Correlate: same user, same issue, same time window, same system, and similar text.
• Classify the responsible system (a service catalog) and attach status snapshots (incidents, alerts, deploys).
• Group tickets into incidents during outages, so one fix closes many tickets.
• Fast search and a view for support engineers.

2) Architecture

flowchart LR
    IN["Email, portal, chat, API"] --> ING["Ingestion API"]
    ING --> DB[("Tickets DB")]
    ING --> K[("Ticket events")]
    K --> ENR["Enrichment workers"]
    ENR --> ID["Identity service - user, tenant"]
    ENR --> CLS["Classifier - responsible system"]
    ENR --> ST["Status history - incidents, alerts, deploys"]
    ENR --> COR["Correlator - rules + similarity search"]
    COR --> VEC[("Search / vector index of recent tickets")]
    COR --> INC[("Incident groups")]
    UI["Support engineer UI"] --> DB
    UI --> INC

3) Key Steps

1. Ingest: save the ticket immediately (with a status of new) and publish an event. Ingestion stays fast even in a spike, and the heavy work is async.
2. Identify the user: map the email or account to a user and tenant. Unknown senders are flagged for manual linking.
3. Classify the system: from product area fields, keywords and an ML text classifier trained on past tickets → a service in the service catalog (which lists owners and dependencies).
4. Status snapshot: query the status history (incident tracker, monitoring alerts, deploy log) for that service and its dependencies at the ticket's time, and store a copy on the ticket, so later readers see what was happening then.
5. Correlate:

• Same user: other open or recent tickets by that user (an index on user_id + time).
• Similar tickets: text embeddings or keyword search over tickets from the last few hours for the same service. Similar ones are linked.
• Incident grouping: if an active incident exists for the service, attach the ticket to it. If many similar tickets arrive quickly and there's no incident, propose a new incident to on-call.

4) Handling the Spike

• Queue-based enrichment scales with workers, and ingestion never blocks.
• Dedup and merge: the same user submitting twice → merge. Many users with the same error → group under one incident.
• Idempotent processing (by ticket_id) and conditional updates when attaching to incidents, to avoid race conditions in grouping.
• Bulk actions: resolving an incident resolves or notifies all attached tickets.

5) Data Model

tickets:        ticket_id, user_id, tenant_id, channel, subject, body, created_at, status,
                system_id, system_confidence, incident_id, status_snapshot (JSON)
ticket_links:   ticket_id, related_ticket_id, reason (same_user | similar | same_incident), score
incidents:      incident_id, system_id, started_at, status, ticket_count

6) Wrap-Up

Ingest tickets fast and enrich them asynchronously: resolve the user and tenant, classify the responsible system from the service catalog with rules and ML, snapshot that system's status at the ticket's time, and correlate with the user's other tickets and similar recent tickets via a search/vector index. Group bursts of similar tickets into incidents (joining active ones or proposing new ones), make processing idempotent to survive outage spikes, and give support engineers a linked, incident-centric view.