Learn
What is governed agentic analytics?
Plugging an LLM into your data is not an analytics strategy. Here’s what it takes to turn a confident-sounding AI into one that’s actually right.
The problem
“Analytics slop.”
In June 2026, Anthropic published a production write-up on their own agentic analytics implementation. The finding was brutally honest: without the surrounding system, their model scored 21% accuracy on internal analytics evaluations. With the full stack, accuracy moved above 95%.
The model didn’t change. The stack around it did.
“Analytics slop” — a term coined by Jose Parreño — is what you get when you skip the stack. Confident, well-formatted, plausible-looking answers that nobody has verified. It looks right until it isn’t. And in customer experience data, where a wrong number can route a budget or mislead a leadership team, that’s dangerous.
Concept ↔ entity ambiguity
“Active users” or “revenue” maps to dozens of plausible fields. The agent picks one. It’s often wrong.
“Detractor” could mean NPS 0–6, or anyone who complained, or anyone who rated below 8 — different answers, same word.
Data staleness
Schemas, definitions, and business rules change. Yesterday’s correct procedure quietly becomes wrong.
A column renamed last month. A filter that used to exclude test data but no longer does. The agent doesn’t know.
Retrieval failure
The right answer exists, but the search space is so large the agent builds confidently on a near miss.
The agent queries the wrong table, gets a plausible-looking result, and delivers it with full confidence.
The stack
Four layers between the model and the answer.
- Adversarial review
- Confidence scoring
- Provenance footers
- Correction loops
“Don’t self-certify.”
- Tool packs
- Procedural playbooks
- Bounded tools
“Semantic layer first, raw SQL as fallback.”
- Semantic layer
- Lineage
- Curated reference docs
“Let the LLM draft documentation. Let humans own meaning.”
- Canonical datasets
- Metadata as a product
- Quality checks
“Fewer, more heavily governed logical models.”
Layer 1 — Data foundations
One canonical dataset, not forty candidates. Every piece of feedback — from a survey, a review, a support ticket — is normalised into the same format. Automated quality checks run before any analysis touches it: completeness, valid ranges, anomaly detection. An agent asking about “detractor comments” doesn’t face forty plausible tables; it faces one governed dataset with a documented grain: one row, one piece of customer feedback.
Layer 2 — Sources of truth
The semantic layer. This is where governed definitions live: what “NPS detractor” means in YOUR organisation, what “last month” means under YOUR business calendar, what counts as a “valid response.” The LLM can draft this documentation. Humans approve it. No entry enters the agent’s context until a human says so. When schemas change, docs are regenerated and flagged needs_review — they can be briefly stale, but never silently wrong.
Layer 3 — Skills
Procedural knowledge about how to work, not just what data exists. Instead of a raw database connection, the agent gets dedicated tool packs: dataset exploration, analysis results, topics & risks. Each tool returns structured, governed output. Free-form SQL exists as an explicitly discouraged fallback: read-only, restricted to an allowlist of data tables. The 43 analysis types are encoded analyst playbooks — a senior CX analyst’s procedure, made reproducible.
Layer 4 — Validation
Before an answer is delivered, an adversarial review challenges the draft. A QA agent checks SQL, verifies assumptions against the Knowledge Pack, and flags findings lacking sufficient evidence. Every answer carries a provenance footer: source tier (knowledge pack vs. raw exploration), confidence rating, exact SQL executed, data freshness. It doesn’t make the answer more correct — it tells the reader how much to trust it.
Voice of the customer
Why this matters more for Voice-of-the-Customer data.
Anthropic’s analytics runs on structured data — tables, metrics, SQL. Customer feedback data is messier: open-ended survey verbatims, reviews, support tickets, social mentions. Unstructured text where ambiguity isn’t an edge case — it’s the raw material.
For structured data, the main governance challenge is concept ambiguity: “revenue” maps to the wrong column. For unstructured text, there’s a nastier cousin: the LLM invents categories.
An unconstrained LLM running sentiment analysis on 40,000 survey verbatims will produce plausible aspect categories — “Product Quality,” “Customer Service,” “Pricing” — that are different every run. Wave-over-wave comparison becomes meaningless. “Pricing” in Q1 doesn’t match “Pricing” in Q2 because the definition drifted.
See how governed taxonomies work in aspect-based sentiment analysis and the VOC analysis playbook.
The taxonomy problem
LLM invents categories → plausible, inconsistent, different every run → no comparability.
Humans own the taxonomy → LLM maps messy language onto approved categories → “Pricing” always means “Pricing.” Wave-over-wave comparison works.
Implementation
How InsightNarrator maps to the four layers.
| Layer | InsightNarrator implementation | What it does |
|---|---|---|
| Data foundations | Unified Dataset Format + Automated Quality Report | Every source normalised to one canonical format. Quality checks (completeness, ranges, anomalies) run before analysis. |
| Sources of truth | Knowledge Pack | Human-approved metrics, glossary, rules, caveats, example queries, reference docs. LLM drafts; humans approve. Staleness detection with needs_review flagging. |
| Skills | Tool packs (10 packs, 104 tools) + 43 analysis types | 43 encoded analyst playbooks across ten packs: analysis, datasets, dashboards, stories, library, capture, memory, topics/risks, live agent, SQL. |
| Validation | Multi-agent pipeline with QA agent + provenance footer | 9-agent pipeline. Final QA agent adversarially challenges findings. Every result carries confidence score, source tier, SQL executed, data freshness. |
Anthropic’s published 21% → 95% figures are theirs, measured on their own structured data. Our formal evaluation harness is on the roadmap; we don’t claim their numbers. Read the full architecture.
The alternative
Two scenarios. Same question. Same data. Different trust.
- Question
- “What’s driving detractors?”
- What the AI consults
- Raw data only — it guesses at tables and definitions
- How it answers
- Free-form SQL on whatever table looks plausible
- Validation
- None — the AI certifies itself
- Output
- “Pricing seems to be a factor” — no confidence, no evidence
- Provenance
- None
- If wrong
- You won’t know until it’s too late
- Question
- “What’s driving detractors?”
- What the AI consults
- Knowledge Pack: approved “detractor” definition (NPS 0–6), approved “this quarter” rule
- How it answers
- Dedicated analysis tool with governed context
- Validation
- QA agent challenges SQL and assumptions
- Output
- “Pricing → Plan Confusion is the #2 detractor driver, 80% negative sentiment, 19 verbatims. Confidence: high. Source: Knowledge Pack.”
- Provenance
- Source tier, SQL, row count, data freshness attached
- If wrong
- Provenance footer flags low-confidence answers for verification
FAQ
Questions about AI analytics governance
See governed analytics on your data.
Upload a CSV. Watch what happens when AI works within your definitions instead of inventing its own.
No credit card required · EU/Swiss hosting option · Cancel anytime