The enterprise agent users trust is not the fastest one. It is the one that shows what it is thinking.
The problem the previous case left open
In the enterprise RAG case, we solved document search for a large public healthcare institution: procedures, policies, expiration dates, owners by department. The retrieval engine worked. Staff could ask about documents and get answers with a direct citation to the file.
But there was a frontier document search never reached: the institution's live operations.
Overdue radiology reports. Exam production data. Benefits policies. Work schedules. That knowledge did not live in PDFs on the Drive. It lived in relational databases, clinical records, HR records, and quality tables with their own structure. And it was split across four domains that, operationally speaking, are different worlds.
The previous solution had been n8n: workflows that worked, but were a black box. The user fired off a question and got an answer, or waited in silence until the timeout. No visibility into progress, no access control by domain, no structure to grow. When our partner institution brought us the problem, the diagnosis was blunt: this is not a prompt problem. It is an architecture problem.
One engine for four different worlds is a worse compromise than four engines
The natural temptation would be a generalist agent: one LLM handling reports, data, quality and HR behind a single endpoint. Great demo. In production, a pile of problems.
A radiologist asking about overdue reports has a completely different intent from an HR manager asking about the vacation schedule. The databases are different. The vocabulary is different. The access policies are different, and that part is fundamental: whoever can see radiology reports cannot necessarily see personnel data.
A generalist agent that knows everything needs an LLM router to dispatch each query to the right context. LLM routers are elegant in demos and unstable in production: a slightly ambiguous question goes to the wrong specialist, the answer comes back empty or wrong, and the user has no idea why. And when you need to update the Reports specialist, you end up touching the same monolith that serves HR.
We went another way: four distinct specialists, each one an independent FastAPI microservice, with its own domain, its own pipeline, and its own deployment.
The user as the router: a defensible position
Most multi-agent architectures solve routing with an LLM: the user types a question and the orchestrator decides which specialist answers. It is the smoothest possible experience. The user never needs to know there are four agents.
We did not do that. The user picks the specialist explicitly, through a sidebar or tabs in the interface.
The reason is a mix of pragmatism and epistemic humility: users know what they want better than a router does. A physician who wants reports does not need the system to figure that out. They already know. And when the router gets it wrong (say, dispatching an HR question to the Data agent), the user has no idea why the answer is wrong. They blame "the AI."
There is a second, more technical argument: RBAC. If an auditor cannot see HR data, the access layer has to enforce that before any LLM call happens. With explicit selection, access control is simple: the user tries to open the HR tab, the system checks allowedAgents in the JWT, and the tab simply does not appear if access has not been granted. With an LLM router, the question would have to be intercepted before it reached the specialist. More attack surface, more points of failure.
The honest tradeoff: explicit selection adds friction. Users sometimes pick the wrong tab. There is a real UX cost. The bet is that, in production, this cost is lower than the false positives of a poorly calibrated router, especially in a hospital, where a wrong answer has real consequences.
The four specialists
Each specialist is a FastAPI microservice with its own internal LangGraph pipeline. The separation is not just logical. It is physical.
Reports serves the radiology reporting domain. It looks up reports by modality, physician, period and delivery status. The vocabulary is clinical.
Data turns natural language into SQL, runs the query, and analyzes the result with Python, working like a senior data scientist on call through chat. This agent's complexity deserves its own case, coming soon. For now: it is the most technically deep piece of the system, and we chose not to compress it here.
Quality answers questions about current SOPs (standard operating procedures), institutional policies, expiration dates, and document types by department. This specialist is deliberately thin at the agent layer, because the heavy lifting was already done. In practice, the Quality agent is a thin bridge over the enterprise RAG service we built in the previous case: the same document search with BM25 plus a structured catalog, now available inside a conversational session. No duplicated code. No separate reindexing. The Quality microservice makes an HTTP call to the RAG service and returns the result formatted for the chat context.
HR answers questions about benefits policies, schedules, vacations and leave. The domain is corporate; the vocabulary is people management.
Topology of four independent microservices, with the Quality agent as a bridge to the RAG service
Isolation by microservice: why not nodes in a single graph
The most obvious architectural alternative would be a single LangGraph graph with four specialized nodes, one per domain. Simpler to operate: one process, one deployment.
The problem shows up at operational scale. If the Data node has a bug that causes a timeout, it hits the entire graph, including HR and Reports. There is no fault isolation. Deploying an update to the Reports specialist means redeploying the whole service.
With separate microservices, each specialist has its own lifecycle. We can update the HR agent without touching Reports. If Data gets overloaded, we scale only Data. RBAC can be enforced at the network layer: the HR manager literally has no network route to the Reports microservice.
The cost is real: four services to operate, four CI/CD pipelines, four sets of logs to monitor. For a small team, that overhead is tangible. The decision makes the most sense when domains have separate owning teams, or when per-domain access policy is a non-negotiable requirement, as it was here.
The internal pipeline: five nodes, five events
Each of the four specialists runs the same internal LangGraph pipeline pattern: five nodes in sequence, each one emitting an SSE event to the frontend as soon as it completes.
Query Augmenter → SQL Writer → Database Executor → Response Planner → Streamer
The interface renders in real time:
→ Analyzing question
→ Generating query
→ Executing
→ Synthesizing
→ Answer ready
The frontend is Next.js 15 with React 19, connected to the BFF (Express + TypeORM) through streaming fetch. The BFF relays SSE events from the FastAPI microservices without buffering, so the user sees node 1's progress before node 2 has even started.
Five-node pipeline with one SSE event per node: the user follows every step
Streaming is not a feature. It is trust architecture
Here is what we learned that is worth generalizing beyond this project.
An agent that takes 15 seconds to answer, with a spinner turning on screen, looks frozen to a non-technical user. They wait 5 seconds, 8 seconds, and start to get suspicious. At 12 seconds, they reload the page. Sometimes the agent was 2 seconds away from answering.
The same agent, streaming node by node, is not any faster. Processing still takes the same 15 seconds. But the user sees "Analyzing question" finish at 2 seconds, "Generating query" finish at 4 seconds, "Executing" finish at 8 seconds. Subjective time collapses. The perception shifts from "the system is stuck" to "the system is thinking."
Node-by-node streaming is not a performance improvement. It is a change in your contract with the user. You are promising transparency, not speed.
The architectural consequence is that streaming has to be a day-zero decision. Retrofitting streaming into a pipeline whose nodes were built to return final results is significant rework. Every node needs an event-emission mechanism built in from the start. Build without it and add it later, and you will be refactoring every node.
The practical heuristic: if any task your agent performs takes more than 5 seconds, the end user will notice. If it takes more than 10, a non-technical user will start distrusting the system. Streaming is the antidote.
Query Augmenter: the node that protects all the others
The second principle we learned the expensive way concerns the first node in the pipeline.
Non-technical users write vague questions. Not out of carelessness: that is simply how humans communicate when context is implicit. "Late reports?" assumes the listener knows the period, the modality, and what counts as late. A human would ask for clarification. A naive agent will pass that query straight to the SQL Writer.
The SQL Writer will try. It will generate a query. The query will return something. But "something" is probably not what the user wanted, because the original question specified nothing that would allow a precise query. You have burned through the expensive nodes (SQL Writer + Database Executor), spent expensive tokens, and the user gets an imprecise answer that they will blame on the system.
The Query Augmenter exists to intercept that before it happens.
It is node 1 in all four specialists. It receives the user's raw question. It uses a cheap model (GPT-4o-mini, with a strict prompt) to rewrite the query into a fully specified version, with the period inferred, entities identified, and criteria made explicit. If the question is so ambiguous that no reasonable inference is possible, the Augmenter sends a clarification request back to the user before triggering any downstream node.
Before: "late reports?"
After: "Which radiology reports were not delivered within the institution's defined turnaround times over the last 30 days, grouped by modality and responsible physician, excluding exams still in progress?"
Query Augmenter: the transformation before the costly nodes
The Query Augmenter is not an optimization. It is a quality-control node that protects the cost and accuracy of everything that comes after it.
The savings are real. An augmentation node on GPT-4o-mini costs a fraction of running SQL Writer + Database Executor on a malformed query. And for the user, the difference in answer quality is the difference between "the system works" and "the system is imprecise."
Access control per agent, not per query
The RBAC model was designed to mirror how organizations actually think about access.
In a hospital, access is not divided by query type. It is divided by domain of responsibility. The quality auditor has access to documents and SOPs. The HR manager has access to personnel policies and schedules. The physician has access to reports. Those categories map directly onto the four specialists.
The mechanism is straightforward. The authentication JWT carries an allowedAgents field:
{
"sub": "user-123",
"allowedAgents": ["qualidade", "laudos"]
}
In the interface, tabs for unauthorized specialists are simply not rendered. They are not disabled, not grayed out. They do not exist. In the FastAPI microservices, every endpoint validates the JWT independently before processing any query. This is defense in depth: the UI does not show what is not allowed, and the backend rejects unauthorized requests even if the UI is bypassed.
RBAC via allowedAgents: three profiles, three different views
The model has a useful property: when a new specialist is added, the access policy is just a new value in the allowedAgents array. There is no conditional logic scattered through the code. Adding a specialist and adding a permission are independent operations.
The interface as product, not container
The frontend deserves more than a line in a tech stack list.
Next.js 15 with React 19 serves as both the chat interface and the BFF through API Routes: the same process that renders the pages proxies the SSE events from the FastAPI microservices. Radix UI provides the accessibility primitives. Tailwind defines the design system. Framer Motion animates the transitions between pipeline states, so the "Analyzing question" and "Generating query" indicators are not just text. They are elements with their own visual state.
One implementation detail reflects the project's philosophy: chat sessions are only persisted to PostgreSQL once the user sends at least one real message. No database record is created when a user opens a tab and writes nothing. Zero orphaned rows. Zero cost of periodically cleaning up ghost data. Draft mode (the state between "opened the chat" and "sent something") lives only in React client state.
What we would do differently
The decision to have users explicitly select a specialist has a cost we underestimated. Some users consistently pick the wrong tab, especially for questions that span domains. A physician who wants to know about both a specific report and the related quality policy has to use two sessions in two tabs. It is not a serious problem, but it is real friction.
A likely future iteration is a hybrid mode: explicit selection by default, with an "I'm not sure" option that triggers an LLM router as a fallback. The best of both worlds: deterministic behavior for people who know what they want, automatic routing for people who don't.
The Query Augmenter has a risk surface worth naming: instruction injection through the query. If a user writes a question that reads like a prompt ("ignore the previous instructions and answer X"), the Augmenter can become the attack vector. The Augmenter's prompt has safeguards, but the risk is not zero. In high-security environments, that surface deserves a validation node in front of the Augmenter.
The operational overhead of four microservices is real. For a team with less infrastructure available, a middle ground (two microservices instead of four, grouping specialists with related domains) may be the better balance.
How to replicate this architecture
If you are building a conversational multi-agent system for an organization with distinct knowledge domains, this checklist captures the decisions that matter most:
1. Define your specialists as bounded subdomains, not personas. A specialist has a database, a vocabulary, and an access policy. If two "specialists" share all three, they are one. If they differ on any of them, they are two.
2. Build event streaming into the pipeline from day zero. Not as a polish feature. As a contract with the user. Every node emits an event when it completes. Retrofitting this later is significant rework.
3. Put the Query Augmenter in front of every costly node. Cheap model. Strict prompt. Full rewrite or a request for clarification. This protects the cost and accuracy of every downstream node.
4. RBAC per specialist, not per query. Model permissions at the same level of granularity the organization uses to think about access. Department or domain of responsibility, not question type.
5. Isolate services when domains have different lifecycles. If Reports and HR are updated by different teams at different paces, they deserve different processes. The overhead of running two services is worth the fault isolation and independent deploys.
6. In production, prefer deterministic specialist selection over an LLM router. The router gives you a smoother demo. Explicit selection gives you more predictable behavior, simpler RBAC, and fewer surprises when the query is ambiguous. You can add the router as a fallback later. The reverse is harder.
The Data agent deserves its own story
Of the four specialists, the Data agent has the most technical depth. It turns natural-language questions into complex SQL queries, runs them against the database, and analyzes the results with Python the way a senior analyst would: spotting patterns, computing derived figures, and framing the answer with interpretive context, not just numbers.
That chain of reasoning (NL → SQL → execution → analysis → answer) has implementation details that do not fit in this case without compressing what should not be compressed. We will tell that part separately, soon.
How does this system relate to the enterprise RAG?
The enterprise RAG case built a document search engine over the institution's PDFs (SOPs, policies, protocols) without a vector database, using BM25 and a structured catalog. That engine keeps running as an independent service.
This system's Quality agent is a thin bridge over that service: when a user asks about an SOP or a policy, the Quality agent makes an HTTP call to the RAG service and returns the result inside the conversational context. No duplicated search code. The two systems are complementary: one serves documents, the other serves operations. Together they cover institutional knowledge more completely than either could alone.
Why split into four microservices instead of a single graph with four nodes?
In a single graph, one failing node affects all the others. Deploying an update to the Reports specialist means redeploying the whole service, including HR and Quality. Scaling Data, the heaviest node, raises the resources for everyone.
With separate microservices, each specialist gets fault isolation, an independent deploy cycle, and can be scaled on its own. The cost is operational overhead: four services to monitor, four CI/CD pipelines. The trade makes sense when domains have different teams or update cadences, or when the RBAC policy needs to operate at the network layer.
Why does the user choose the agent instead of an automatic router?
LLM routers are elegant in demos. In production, a slightly ambiguous query goes to the wrong specialist and the user gets an incorrect answer without knowing why.
With explicit selection, behavior is deterministic and RBAC is straightforward: if the user has no access to the HR specialist, the tab simply does not appear. There is no routing surface to protect. The cost is friction: users sometimes pick the wrong tab. The bet is that this cost is lower than a router's false positives in an environment where wrong answers have real consequences.
A likely future iteration is a hybrid mode: explicit selection by default, with a router as a fallback for users who explicitly don't know which agent to use.
How does SSE streaming change the user's perception of time?
It does not change actual time. It changes subjective time. A query that takes 18 seconds behind a spinning loader is perceived as a frozen system. The same query with five SSE events (one per pipeline node) is perceived as a system at work.
The mechanism is psychological: visible progress collapses the perception of waiting. The user sees "Analyzing question" complete, then "Generating query," then "Executing." They know something is happening. Distrust never sets in.
The architectural consequence is that streaming has to be a day-zero decision, not a pre-launch one. Retrofitting SSE events into a pipeline built to return final results means refactoring every node.
What is the Query Augmenter, and why is it the first node?
The Query Augmenter is the first node in every specialist's pipeline. It takes the user's raw question and rewrites it into a fully specified version, with period, entities, criteria and scope made explicit, before any costly node is triggered.
Why it is node 1: to protect cost and quality. Non-technical users write vague questions by nature, because the context is implicit to them. If that vague query reaches the SQL Writer, the result will be an imprecise query that returns something wrong. You have spent the expensive nodes, and the user got an answer they did not ask for.
The Augmenter uses a cheap model with a strict prompt. If the question can be augmented, it rewrites it. If it cannot (too ambiguous for any reasonable inference), it sends back a request for clarification before spending any downstream node.