Blog

Building an AI Layer Across an Enterprise: What We Learned Building One

Two parallel workstreams, eight data sources, one Claude interface. An inside look at the architectural choices that make AI on enterprise data actually work, and what they tell you about how to do this properly.

I Estimated read time:
11 minutes

The Challenge

Our client is a field service management platform — cloud-based SaaS, serving businesses that run field operations at scale. Like most growing SaaS businesses, they had a problem familiar to anyone in this position: every team, from Customer Success to Support to Sales to Services to Product, needed a complete view of every customer. And no two teams were looking at the same picture.

The data existed. It was spread across their core platform (accounts, users, subscriptions), Zendesk, Jira, their Client Success system, their billing platform, their onboarding tooling, their royalty system, and Pendo for product usage. But there was no single layer that brought it all together. There was no master customer dimension that everything could be reconciled against. And there was certainly no way for a CSM to ask a question like “is this account at churn risk?” and get a properly synthesised answer drawn from across all that data in seconds.

The goal was to build that layer. Not just as a data project, but as an AI-accessible intelligence platform, joining the ranks of elite enterprise AI tools, with Claude as the front door.


Two Workstreams, One Platform

From the start, we split the build into two parallel workstreams. This separation is important because it reflects a principle we apply to most projects of this kind: the data foundations and the AI capabilities are different disciplines, and trying to do them as one undifferentiated build usually produces a poor outcome on both sides.

The first workstream was the data layer. This is the boring-but-essential work of getting data out of source systems, cleaning it, structuring it, and making it ready for consumption. Databricks Lakehouse (a cloud data platform for storing and processing large volumes of structured data) is the engine for this, with OpenSearch sitting alongside as the query and vector search layer — the component that makes it possible to search across data using meaning, not just exact keywords.

The second workstream was the AI build. This is the work of designing the AI agents for business, choosing the architecture, building the orchestration, and connecting it all to Claude through a custom MCP connector.

Both workstreams ran in parallel, but with the data layer always one step ahead. You cannot build useful AI on bad data, and trying to do them in the wrong order is a fast way to deliver something that looks impressive in a demo and falls apart in production.


The Data Layer: Foundations That Earn Trust

The first thing we built was a gold layer of 26 curated views spanning all the active source systems. These views are the clean, business-ready interface that everything downstream relies on. Whether it is a dashboard, an agent, or an analyst running an exploratory query, they all draw from the same governed set of definitions.

The most important piece of plumbing underneath this is a customer crosswalk. Every source system has its own way of identifying customers, and reconciling them is one of the hardest problems in enterprise data integration. We built an automated crosswalk that combines direct ID matching, fuzzy string matching, and AI-assisted matching to resolve every source to a single authoritative account ID. We are now achieving 99%+ match rates across all active accounts.

If your AI cannot answer “which Zendesk tickets belong to this customer,” the rest of the architecture does not matter.

From that foundation, we built two things that the AI layer would later consume. A curated Customer 360 dataset is generated daily for every active account, pulling together support history, CS notes, billing status, onboarding stage, product usage, and account flags into a single pre-built summary. And vector embeddings of unstructured data — primarily CS notes, Zendesk tickets, and Jira issues — are pushed into OpenSearch for semantic search. (Vector embeddings are a way of converting text into a numerical form that lets the system find conceptually similar content, even when the exact words don’t match.)

This combination, pre-built structured summaries plus searchable semantic indexes, turned out to be one of the most important design decisions in the whole build.


The AI Layer: A Router and a Roster of Specialists

The most distinctive part of the architecture is how the AI is structured. The instinct for many teams is to build a single, monolithic AI agent that tries to know everything and answer every kind of question. We did not do that.

Instead, the architecture uses a small Router. The Router is an LLM call whose only job is to classify each incoming question and dispatch it to the right specialist. The specialists are separate agents, each tuned for a specific kind of question and a specific data pattern.

The Customer 360 agent answers full-account-overview questions. It retrieves the pre-built dataset for the relevant account and synthesises a coherent narrative across support, CS, billing, onboarding, and account health.

The Usage 360 agent answers questions about how customers are using the product. It draws from Pendo for in-product usage signals (pages visited, features used, frequency) and from summaries of each customer’s own operational database — for example, how many work orders are being created each month and what value they represent. This gives a single, coherent view of both how the customer is using the platform and what they are achieving with it.

The Genie Proxy handles dataset-specific analytical questions (“how many tickets did this customer raise in Q1?”) by forwarding them directly to a Databricks Genie — a tool that translates plain-English questions into database queries and returns the result. No LLM call sits between the user and the answer, which keeps the data faithful and the response fast.

The Churn Analysis agent takes a different approach again. Churn risk is not a single query, it is a synthesis of many signals from different parts of the business. We built this agent as a Claude Skill — a packaged set of instructions and reasoning patterns that tells Claude how to approach a specific task, in this case churn risk assessment, in a consistent and repeatable way. The Skill calls out to a set of curated Genie queries to pull the supporting data (support trends, usage trajectories, CS notes, billing changes) and then reasons over that data to produce a structured churn risk assessment.

A planned RAG Search agent will handle open-ended exploratory questions using semantic search across the vector index, for the cases where someone wants to find anything relevant to a topic across all the unstructured data.

Each specialist is independently improvable. Adding a new capability does not mean rewriting the existing ones, it means adding a new specialist behind the Router. That is a fundamentally different shape from one big agent trying to do everything, and it has significant practical implications.


Why a Router Beats a Monolith in Enterprise AI tools

The Router pattern is not unique to us. Anthropic uses the same orchestrator-worker pattern in its own multi-agent systems, and AWS describes it in their guidance for production agentic AI. There are four reasons it consistently wins for this kind of build.

  • Better answers. A specialist tuned for one dataset and one task gives sharper results than a generalist trying to know everything. The Customer 360 agent does not need to know how to query Zendesk directly. The Genie Proxy does not need to know what a Customer 360 summary looks like.
  • Easier to extend. Adding a new agent means adding a new specialist and updating the Router’s prompt. Existing agents are untouched. New use cases ship faster.
  • Independent evaluation. When something goes wrong, it is either a routing failure or a specialist failure. The difference is observable and measurable. That makes the system improvable rather than mysterious.
  • Independent cost tuning. The Router runs on a cheap, fast model because it only has to classify. A specialist that needs deeper reasoning can use a stronger model. You spend tokens where they add value, not uniformly across every part of the system.

The trade-off is one extra LLM call per request. In cost and latency terms, that is small. In flexibility terms, it is enormous.


Pre-Built or Live: The Decision That Defines Performance

The second architectural choice worth understanding is how data is reached at query time. There are two patterns at play in the system, and they answer different kinds of questions.

For broad, narrative questions like “give me an overview of this account,” reaching live into five operational systems and joining them together is slow, fragile, and prone to data-model mismatches. So we do not do that. Instead, an offline pipeline assembles a curated dataset per customer overnight and writes it to OpenSearch, where it can be retrieved in a single fast read. The Customer 360 agent works against this pre-built dataset.

For specific analytical questions like “how many tickets did this customer raise last month,” a live query is fine and actually preferable, because the answer needs to be current to the minute. The Genie Proxy forwards these to a Databricks Genie, which generates SQL against the live tables and returns the result. No pre-aggregation needed.

This split, indexed for breadth, live for currency, is a pattern that mirrors how Microsoft Copilot and other enterprise AI products handle the same trade-off. It is the right answer because trying to do everything live ends up slow and fragile, and trying to pre-build everything ends up stale.

Claude as the Interface

A common option for a build like this would be a custom chat interface, a bespoke web app, with the company logo and brand. We did not build one. Instead, we exposed the agent platform as a Model Context Protocol (MCP) server — an open standard that lets external tools and data sources connect directly to Claude — and connected it to Claude, which the client already uses internally.

The benefits are significant. The chat surface is already familiar. It is already maintained externally. It is already integrated into how people work. Authentication runs through the company’s existing Okta setup, so joiners and leavers propagate automatically. And by exposing the platform via MCP, an open standard, it stays reachable from any future MCP-compatible client, not just Claude today.

The effort that would have gone into building and maintaining a UI instead went into the agent logic and the data integrations. That is the right trade-off for an internal AI tool, where what matters is the quality of the answers, not the polish of the chat box.

What This Unlocks

The point of all this architecture is what it enables. Once the data layer and the AI platform are in place, new capabilities ship as new specialists behind the Router, with no changes to anything that already works.

In the first releases, that meant a Customer 360 agent giving any team a full account summary in seconds, a Usage 360 agent showing how every customer is actually using the product and the value they are getting from it, churn risk assessment driven by a Claude Skill drawing on curated Genie queries, and natural language querying over Zendesk and Jira directly inside Claude. The next phase brings upsell and downsell signal detection, broader natural language querying across all the data, and proactive notifications when account health changes.

After that, support triage, automated CS notes generation, product feedback aggregation, and customer-facing usage reports. None of those require rebuilding the foundations. The data layer absorbs new sources, the curated views expand, and the Router gets a new specialist. The architecture grows with the use cases instead of being rebuilt for each one.

The Broader Lesson

What we built here is not unique to SaaS business intelligence, and it is not unique to a US business. The pattern, a unified data layer underneath, a Router and specialists on top, Claude or another LLM as the interface, applies to any organisation trying to put AI seriously across its business.

The lessons we would pull out for anyone planning something similar:

  • Treat the data layer and the AI layer as separate disciplines. Both matter, but they need different skills and different sequencing.
  • Resolve identity first. If your AI cannot tell which records belong to which customer, nothing else works.
  • Pre-build the breadth, live-query the depth. Account summaries are pre-aggregated. Specific analytical questions are answered live. The split is deliberate.
  • Avoid the monolithic agent. A small Router and a roster of specialists outperform a single generalist agent on every metric that matters.
  • Use managed services for everything that is not differentiating. The hosting, the model serving, the chat surface, the authentication, all of it. Your effort should go into the parts that are unique to your business.
  • Make the architecture additive. Every new capability should be a new specialist, not a rewrite. This is what lets the platform keep growing without becoming legacy.

None of this is theoretical. It is what we are building right now, in production, against real data, for a business that depends on it working. And the principles transfer cleanly to any organisation thinking about how to take AI from interesting demos to genuine business capability.

Thinking About a Build Like This?

InsyteGroup builds AI and data platforms for businesses that are serious about turning their data into operational intelligence. If you are looking at putting AI seriously across your business and want to understand what the right architecture looks like, we would be happy to share what we have learned. We can usually show within a workshop what a build like this could mean for your data and your teams.

Lets Work Together

We’re always happy to talk about what we do, what you’d like to achieve, and answer any questions.

Let’s start with a proof-of-concept to show you how we’d solve your BI needs.

Contact Us