Governed GenAI Data Agents with LLM Evaluation

Problem

A retail and consumer-goods company with 8,000 office employees ran two pilot chatbots over its warehouse. One produced a wrong revenue figure in a board deck. The other leaked a supplier contract summary to the wrong team. The CIO froze all GenAI releases until agents can ship safely. That means governed data only and traceable answers. Each release needs a measurable evaluation. An auditor must be able to see how an answer was produced six months later. Three agents are in scope: an analytics assistant, a data-onboarding agent and an operations agent that drafts actions such as reorders.

Approach

Both incidents have the same root cause: the pilots wrote raw SQL against ungoverned tables and read documents with their own service account. The design narrows the surface the agent can touch rather than choosing a better model. Agents call three schema-validated tool surfaces, and identity travels with the user. No answer is released without a trace, and a release is gated by a policy file in CI.

Architecture

• Identity: APISIX and Keycloak issue a signed user claim that is forwarded to every tool call.
• Agents: a LangGraph runtime with an allow-list of tools per agent. A LiteLLM gateway fronts two commercial providers and an on-premises vLLM model.
• Tool surfaces: a metric API with certified definitions and per-user policy. Contracted data products in OpenMetadata. A document store filtered by access rights before ranking.
• Decision point: an Open Policy Agent sidecar decides agent, user and resource on every tool.
• Evidence: the answer, evidence set, tool-output hashes and prompt and model version go to PostgreSQL with an outbox row in one transaction. Iceberg keeps the archive for two years.
• Approvals: Flowable records who approved each action, with rollback.

Decisions and trade-offs

• No raw SQL tool at all. The wrong board-deck figure came from a model inventing a query on an ungoverned table. Now the agent picks a metric and filters; the layer compiles the SQL. An unknown metric fails validation, and the agent logs a request instead of guessing. A better model does not fix a wrong figure; a narrower surface does. The cost is coverage limited to certified metrics.
• No agent service account. The contract summary leaked because the agent had more rights than the person asking. Now a signed user claim is checked by the runtime, the tools and the policy decision. Answers are never cached across users. The cost: the agent can never help with data the user could not query alone.
• No answer without evidence. The evidence row and an outbox row are written in one PostgreSQL transaction, and the answer is released only after commit. Replay rebuilds the answer from stored tool outputs and never calls the model again. The guarantee is “same evidence, same assembly”, not “the model would say the same today”. The cost is running my own evidence store next to Langfuse.
• Design time versus run time. An LLM may draft pipeline code and contracts, and a person merges them. Production pipelines run without an LLM, so the same input gives the same output. The operations agent only drafts; Flowable records the human approval. The cost is a review step on every change.
• Evaluation gate. Each agent faces a 300-question golden set, tool-order checks, 200 injection cases, 100 exfiltration cases and a provider tolerance test. Thresholds sit in a versioned file, and a regression blocks the pipeline with no “just this once”. The cost is about 20 minutes per run, by the kata’s estimate, plus keeping the sets current.
• One model gateway, portability as a goal. Secrets, quotas and cost labels live in LiteLLM, so a provider switch is a routing change. If the switch drops golden accuracy by more than 2 points or tool-order match by more than 5%, the agent stays on its current model. Restricted documents can only go to the on-premises model. The cost is a gateway every call must pass.

What I would revisit

• Metric coverage. I assumed 400 certified metrics. How many questions fall outside them, and who handles the requests, within what time?
• Document labels. Are access lists and classes in the document system reliable? Unlabelled documents stay out by default.
• Provider agreements. Which data classes may go to an external provider, and is pseudonymisation enough for the impact assessment?
• Tolerance thresholds. I assumed 2 points and 5%. Do business owners accept them, or does each agent need its own?

Outcome notes (design assumptions, not measurements)

This is a design exercise, not a running agent platform. The output is a four-page diagram set, a one-page whiteboard version and a 16-page Turkish PDF. It includes an AI governance register and an EU AI Act mapping. Tolerance thresholds (2 points of golden accuracy, 5% trajectory) are assumptions awaiting business owners.

Sources

• Architecture PDF (Turkish, 16 pages).
• draw.io diagrams (English labels): four-page set and a one-page 30-minute version.