The semantic layer predates most of the people arguing about it. It showed up in the BI tools of the early 90s, and for thirty years it did one job: give the business a single, trusted definition of what a number means. Then agents arrived, and the stakes changed.
David Mariani, co-founder and CTO of AtScale, has spent thirteen years building independent semantic layers. His argument in episode three is blunt. The semantic layer is the critical component of the context stack — the piece that keeps the actual numbers deterministic, even when the AI producing them is not. Let the agent be creative in the questions it asks. Never let it improvise when it's computing a number like revenue.
Austin Kronz pressed on where the semantic layer stops and the rest of the context stack takes over. Here's where Mariani drew the lines.
Q: The semantic layer has existed for thirty years. Why is it suddenly mission-critical in the AI era?
The goal hasn't changed since the BI era: one trusted definition of a metric, so finance and sales don't walk into a board meeting with two different revenue numbers. What changed is who's asking, and how fast.
In the BI era, humans queried through Tableau or Power BI. There was oversight, and a natural ceiling on how many questions got asked. Agents removed both. They ask far more questions, far faster, with no human checking the output before it lands.
Ontologies sit in the context stack too, but the semantic layer is the tier that makes the numbers deterministic — so you can trust them. The agent still gets to be creative. It just doesn't get to invent the numbers.
"You are giving it a curated set of metrics with which AI can be creative and probabilistic, which is what you want it to be. But not probabilistic when you're calculating revenue or gross margin."
Q: Where does the semantic layer start and end?
Mariani's definition hinges on one word: compute. A semantic layer has a semantic engine. It doesn't just describe revenue; it answers the query that calculates revenue. Some semantic layers only describe metrics. His argument is that a real one has to compute them.
What the semantic layer owns:
- Metric and calculation definitions for the business
- Aggregations, and the grain those aggregations run at
- The hard joins, including semi-additive measures. Inventory is the example: you don't sum a SKU across July, you take its level on July 1st and its level on July 31st.
- Deterministic query results
- Security and governance, so an agent inherits the same row-level access as the human it acts for. If Jimmy only sees the Pacific Northwest, the agent running as Jimmy only sees the Pacific Northwest.
What it deliberately doesn't do: storage and compute (that's the warehouse), cataloging, discovery, and lineage (that's Atlan), unstructured data and RAG, and data pipelines (that's dbt). It also doesn't decide which queries to run. That reasoning belongs to the agent.
"A semantic layer's job is to deliver deterministic query results. It's not its job to reason and plan on which queries to run. That's what the agent or the LLM does."
Q: How should a team get started without locking themselves in?
Start with metrics, not architecture. One AtScale customer went people-first: define the core metrics, name who owns each one, then encode them into semantic models so the owner is on record when a definition changes. Mariani's caution is that you can't skip that first step and hope the modeling fixes it later.
The bigger trap is where the semantics live. Hard-code them into your BI tool and you recreate the semantic sprawl of the BI era. Hard-code them into agents as skills, the way Anthropic described in a recent blog post, and it works for one use case but gets brittle fast across an enterprise. Build them into pipelines and they aren't queryable. Push them into a single data platform's semantic views and your semantics only work inside that platform.
His answer: semantics should live independent of the data and the consumption style. It's the same reason companies are landing data in Iceberg — they want to pick the cheapest, best engine instead of being locked to one platform. AtScale is part of the Open Semantic Interchange, now an Apache project, for the same portability reason.
Underneath all of it is a sharper point about ownership. Mariani called your semantics your competitive corporate corpus — the encoding of how your business actually runs. He pointed to Alex Karp's CNBC comments on AI sovereignty: hand that competitive knowledge to a single frontier model or platform, and you give it away.
"It's your competitive corporate corpus. It's the semantics of your business and how it runs. So why would you want to give that to a single vendor or a single platform?"
Q: Is a semantic layer enough for an agent to get context right?
No — and Mariani was clear that AtScale only owns one part of the stack. The semantic layer delivers analytical truth: which customers are likely to churn, what revenue was by region. It won't compile your documents, drive workflows, or send the emails. Every one of his customers runs a catalog like Atlan alongside AtScale, because the two are complementary.
Accuracy is where the argument gets pointed. He cited Databricks' own on-stage number: Genie's ontology gets it right 84.5% of the time. Being wrong more than 15% of the time isn't a rounding error — it's the kind of thing that gets someone fired. And one-shot accuracy hides a second problem: an LLM can get a question right on Monday and answer it differently on Thursday. So AtScale is now benchmarking determinism over time, not just accuracy on a single run.
Austin drew the other half of the boundary from the demo side. Atlan runs a fictional burger franchise, MEC Context, as its worked example for all of this — and there, a question like "why is drive-through time up this week?" needs the semantic definition of drive-through time. But "this week" is ambiguous, the answer depends on whether you're the franchise owner or the store manager, and the procedural knowledge and tone live outside the semantic model. Semantics are necessary. They aren't the whole context stack.
"Let them be creative, but don't let them be creative on units sold or revenue, or even what 2008 means. That's where you need the semantic engine."
Episode 4 turns to the layer Mariani kept circling back to. Jessica Talisman, who was building ontologies long before LLMs existed, joins Austin on July 30 to ask whether the context layer is a genuine evolution of the ontology or just a rebrand. Register here.
