ClarityQ Agentic Data Analysis

A studio apologizes for a broken patch by sending players premium currency. Two weeks later, the agent flags runaway inflation. The agent wasn’t bad at SQL. The rule “never count compensation grants as a faucet” just lived in one person’s head. That’s what Edition 4 of Level Up is about: the rules your team applies by reflex and never wrote down, and what happens to them when an agent starts answering the questions. Inside: -> How to let corrections turn into rules, instead of trying to document everything up front -> Why every change to your metrics should go through review like a pull request -> How to use the agent’s own usage to find what’s missing in your context layer -> 8 questions to tell if your context is governed or just assembled

Scared of agentic data-analysis costs? It actually saves you money...

Everyone talks about how the context layer drives accuracy. Almost nobody talks about what it costs to use one. In our opinion, retrieval efficiency - how the agent finds what it needs in the layer - deserves the same attention as the context itself. The tokens spent on a question are set by how much the agent has to read before it finds the right definitions. Take "what was ARPU by region last quarter, excluding internal test accounts". Everything needed is in the layer. The cost depends entirely on how the agent gets to it. With naive retrieval, the agent is like a new employee on day one - smart, but with no one to ask, so it reads everything to answer anything: 1. Finds four revenue-related definitions. Reads all four to decide which is the source of truth. 2. Reads the dimension definitions to learn how region is represented, and whether country, geo or market is the live one. 3. Looks for the test-account rule. It is in the layer, but filed under "internal traffic exclusion", so the search misses it and the agent reads on. 4. Generates, hits an error on a column name it read fifty turns ago and no longer has in context. Re-reads, regenerates. Every step is paid for. The answer may well be correct, but it costs many times what it needed to, takes longer, and gets less reliable as the window fills with material the question never needed. This is why, in our view, retrieval is not a layer you bolt on. It is part of the harness - the tools the agent works through. How it searches. What it uses. When to search again, when to stop, and what to do when nothing is good enough. Done right, the agent receives, before generating anything: - The single source-of-truth definition, not four candidates - The valid dimensions and their real column names - The business rules in scope, even those stored under a different name - The relevant skills: proven ways to answer this, including the SQL All this searching and filtering runs as plain code, outside the model. Code is cheap. Model calls are not, so the less the model reads, the less the question costs. Three more things that keep the cost down: -> Work with the models' cache. Prompt caching rewards stable, well-ordered context. Making the repeating parts hit the cache is a real cost lever, and easy to get wrong. -> Turn research into context. When a metric is not in the layer, the agent researches it from scratch. If people keep asking, that research and the warehouse queries behind it repeat. The agent should notice and propose adding the metric to the layer. -> Promote repetition out of the agent. If "weekly retention by cohort" is asked every Monday, it should stop being an agent query. The agent can notice the pattern and propose a dashboard, same for repetitive reports. The savings depend on schema size and question complexity, but the gap between naive and efficient retrieval is real money. It is the difference between a tool everyone can use freely and one the company has to restrict.

Your context layer was correct on the day you built it. That much is most likely true, but here is what likely happened since: -> An engineer renames a column in your most used table. The layer still points at the old name. -> Finance changes how a subscription tier is bucketed. The metric definition is now silently wrong. -> A new SKU category launches. Every "by category" breakdown is missing it. None of these are edge cases, they are Tuesday morning cases. A context layer is as far from "one-off" effort as it can get. It's a living system with a maintenance surface, which means at minimum: 1. Schema drift detection. When the warehouse changes underneath, the affected definitions are flagged, not discovered by a wrong answer. 2. Change-aware analyses. If you rerun an analysis from last month and the context it depended on has changed since, you should see an alert, not a different number with no explanation. 3. Version history on every definition. Who changed what, when, and what did the previous version say. Rolling back a metric definition should be as easy as rolling back a commit. 4. Sync with whatever semantic or context layer already exists. Most companies already have some of the knowledge in tools like dbt, Tableau, or alike. And on top, a smart harness with an efficiency layer that tells the agent where exactly to dig in the context, to avoid draining the tokens. The uncomfortable truth: even the most sophisticated agent needs constant care - someone, or something, or both, monitoring and testing the context continuously.

Agentic data analysis is now also "an insight to action plan" and execution hub as well. See it in action now - Start free.

Agentic data analysis is now also "an insight to action plan" and execution hub as well. See it in action now - Start free.