AI Service Desk Platform

Context

Industrial automation enterprise. IT service desk handling ~67,000 annual tickets across chat, the service portal, and other entry points. Greenfield data and AI platform build on Azure, delivered within a small product team: a tech lead and backend engineers built the chatbot orchestrator, a data analyst owned category data quality, and I owned the data and AI platform end-to-end: ingestion, transformation, retrieval, analytics, and infrastructure.

Problem Statement

The service desk operated without AI support. Tickets were manually triaged, routing had to contend with a taxonomy of several hundred classification groups, and the structured operational data accumulated in ServiceNow over years was inaccessible to automated systems. The client needed an AI assistant capable of answering queries and routing requests accurately, built on their existing ServiceNow data estate, with full observability into its performance from day one.

Engineering Approach

Data platform Incremental, watermark-based ingestion from the ServiceNow REST API into a Delta Lake lakehouse on Azure, curated daily by Spark pipelines into tables covering knowledge articles, ticket history, routing performance, and organisational structure. One platform feeds everything downstream: retrieval, classification, and performance measurement all build on the same curated layer instead of separate extracts.

AI retrieval layer Retrieval-augmented generation over the knowledge base using Azure AI Search for vector and semantic indexing and Azure OpenAI for query answering. Retrieval design decisions were made by measurement, not intuition: six chunking strategies were benchmarked against a frozen 75-question evaluation set before the winner was wired into production, and an information-retrieval evaluation framework compared index designs and retrieval modes (hybrid versus vector) to select the production configuration. Embedding runs incrementally: a hash-based diff detects new, changed, and deleted content, so unchanged rows are never re-embedded.

Intent classification The ticket taxonomy was distilled into an embedded category catalog with data-quality gating: only categories with sufficient ticket history and signal quality are eligible for AI routing. Incoming requests are classified by semantic search against this catalog and routed to the correct resolution path without manual triage.

Conversation analytics Conversation events are processed into sessions, turns, retrieval detail, and routing outcomes. Privacy by design: events carry no raw user text, and users are tracked through salted hashes, never raw identifiers. Routing accuracy is measured against ground truth by joining conversation outcomes to actual ticket assignments as tickets close, so accuracy figures improve automatically as reality catches up.

Infrastructure The entire platform is defined in Terraform and reproducible across environments. Every service authenticates through managed identities; no credentials flow through pipelines.

Outcome

• AI assistant handling query answering and intent-based routing across ~67,000 annual tickets, deployed as MVP
• Knowledge base and category indexes rebuilt automatically from live ServiceNow data, with incremental embedding and no manual curation
• Retrieval quality decisions (chunking strategy, index design, retrieval mode) backed by evaluation frameworks rather than judgment calls
• Conversation analytics measuring containment rate, escalation rate, and routing accuracy in place from initial deployment
• Infrastructure reproducible across environments via Terraform, with managed identities throughout

What Made This Complex

ServiceNow’s data model is highly relational. Flattening multi-table ticket, asset, and resolution data into retrieval-quality document chunks required a transformation strategy that preserved semantic context without inflating the index. Getting this right was the difference between an assistant that retrieved relevant content and one that didn’t.

The measurement problem was subtler than it looks. The assistant’s primary success signal, a session resolved without a ticket, leaves no trace in the ticketing system: ServiceNow only records what escalated. Measuring containment required designing a conversation event pipeline as the sole complete source of truth, with routing ground truth arriving late as tickets close and outcome tables rebuilt on every run to absorb it.