Building an Enterprise AI Agent Before the Roadmap was Published
In early 2023 we designed and built a multi-capability AI agent inside Microsoft Teams for a tier 1 bank. At that time the term Agent had not beeen popularised; bot was the terminolgy of the time.
There was no roadmap to follow and very little quality guidance to lean on. We worked out the architecture ourselves, and the result rivalled Microsoft 365 Copilot as it stood at that time.
Because the client is a regulated financial institution, some details are confidential. What follows covers the engineering decisions we can share.
The Challenge
The brief was to give staff a multi tool assistant in Teams and to grow its abilities over several phases:
- One front door to many agents. Act as the entry point to multiple sub-agents built by other internal teams.
- Document summarisation and translation. Delivered with Azure resources, inside the bank’s own cloud boundary.
- RAG search over managed directories. Answers grounded in approved internal content, not the model’s best guess.
- Coding assistance and general knowledge. Combining what the foundation model knew with live web search. It was 2023, after all, and a model on its own was not enough.
All of this had to meet the security, compliance and change-control standards of a tier 1 bank.
Starting Without a Roadmap
Early 2023 was a different world for anyone building on large language models. Context windows were small. Frameworks were weeks or months old. The published examples were mostly demos that fell over under real use.
There were no accepted patterns for tool use, memory or agent-to-agent communication. Each of those had to be reasoned out from first principles, tested, and made dependable enough for a bank.
LangChain Inside the Microsoft Bot Framework
We integrated LangChain into the Microsoft Bot Framework. The Bot Framework gave us Teams as a channel, with enterprise identity and conversation handling. LangChain gave us the reasoning layer: prompts, tools and model orchestration.
It was an aggressive decision. LangChain was only a few months old and changing quickly, but we could see its potential. We could find no other example of anyone combining the two, so we built the integration ourselves.
The bet paid off. The agent logic stayed independent of the chat channel, and we could adopt new LangChain capabilities as they arrived without rebuilding the Teams side.
Two Problems We Had to Solve Ourselves
- Conversation memory in a tiny context window. The models of the day could hold only thirty thousand tokens and worked best when it was kept well below that limit. A long conversation about a document simply did not fit. We built historical context summarisation: older turns were condensed into a running summary, so the agent kept the thread of a conversation without exhausting its budget. This is standard practice now. It was not then.
- Reliable tool choice. Left alone, early models were erratic about which tool to call. We guided the agent’s choice with weighted decision algorithms, steering it towards the right capability for each request. Users got consistent behaviour, and the bank got an agent whose routing could be explained.
An Extensible Architecture, Ahead of the Standards
The agent was never meant to do everything itself. Several teams across the organisation were building specialist sub-agents, and each needed to develop and release on its own schedule.
We designed for loose coupling from the start. A sub-agent had to implement two basic interfaces:
- Discoverability. The sub-agent describes itself and any of its sub-agents it has, so the main agent knows it exists and what it is for.
- Tool and service exposure. The sub-agent publishes the capabilities it offers, so the main agent can call them as well as document/RAG stores.
Sub-agent's could be added by configuration in without changing the code. We built this in 2023, before today’s agent interoperability standards had become established. The same two ideas, discovery and capability exposure, now sit at the centre of those standards.
Keeping Pace: LangGraph and LangSmith
The platform did not stand still. In early 2024 we adopted LangGraph and LangSmith for workflows and evaluations, giving multi-step processes explicit, inspectable state in place of open-ended agent loops.
Using LangSmith, changes to prompts, tools and models could then be measured before release, which matters a great deal when the client is a bank.
What This Means for Your Project
We were building production agents in a regulated enterprise before most of the industry had
a name for them.
The lessons from that work carry over directly:
- Deterministic workflows beat unrestrained Agents. The agent is a tool, not a magic box. It should be easily debuggd and possible to explain what it did and why.
- Architecture that lasts. The design choices of 2023 still line up with where the industry has settled but we can still extend; good software architecture allows for dependencies to change over time.
- Don't be a bottle neck. The hero risk is a real thing; trying to own AI. is a real threat to delivery. From the start we communicated with the plan with other teams; we will solve the delivery and orchestaion so you can focus on an agent suited to your domain.
- Traceability and telemetry is key. Make every agent action observable and traceable end-to-end: Reasoning steps, tool calls, memory accesses, and inter-agent handoffs need a single trace fabric so the same telemetry can answer cost, latency, quality, security, and footprint questions. Observability of agentic behavior is qualitatively different from infrastructure monitoring.
Planning an AI agent for your organisation? Talk to Ninja Software.
Get in TouchCopyright @ 2026 Ninja Software All Rights Reserved
