The Post-AI Data Stack: A blueprint for analytics agents

0:00Zach MichelSo analytics agents, I love talking this topic. A little bit of background on who I am. I lead AI solution architecture here at Atlan and our support function. I've spent m- basically my entire career building systems of semantics. I stood up Atlan from the ground up and... Or sorry, Looker. Atlan, I didn't stand up. I love it here. But I helped stand up Looker from the ground up, and I've been working on systems of semantics for a long time. So I will talk to you about analytics agents. Afterwards, I'm gonna hand it over to Austin, our field CDO, who will be running a panel with Joe DosSantos from Workday, Piyush Bhargava from DocuSign, and Sowmya Parthan from Upwork, who will talk to you about putting analytics agents into practice.

0:43And so with that, as I said, I've been working on systems of semantics for a long time. We all, as data practitioners, have been trying to solve this qu- question of getting the business to be able to speak the language of data, and of course, everyone wants to be able to talk to their data. And with the evolution of GPTs and language models, it's, we can now start to think about teaching our data the language of the business. But it's a really complicated problem. Uh, a few examples I'll share with you from working with customers. One who'd built an agent and got it to about 89% accurate. That agent, however, still fizzled out and did not gain traction because that specific use case, from a data integrity point of view, needed 98% plus accuracy.

1:31A second customer who'd built dozen of semantic views, I think we've all gone down this path, one for each agent, and found it impossible to manage the improvements across all 12 of these views without duplication and somewhat of an overhead nightmare. And then a third, who was trying to solve this problem by hard-coding rules and procedures into each agent specifically, so they found that this proliferation was massive. So for us at Atlan, we spent the last 18 months trying to solve this in many ways, along with customers, but internally, we built our Atlan Labs function. We built context agents that do bootstrapping and context mining. We built semantic view generators, and we even built some in-house harnesses and models.

2:15Within Atlan Labs, we built a simulated environment that took really hard questions across an entire business and on top of a kind of messy data model, 'cause to be honest, there's no such thing as a clean data model nowadays. Uh, and I'm exci- I'm, I'm gonna share with you some of the science behind it, and then I'll demo at the end, uh, what it looks like and, and how we go about building it. So the first thing to start with our science and where our research started is where most people start, which is the model. And what we found is that the model is not the bottleneck for analytics workloads. If you take any model and you just run it on the bare schema, yes, you'll see an improvement as you get to more frontier models versus things like open-weight or small language models.

3:01The accuracy still doesn't get to par of what you need for production. So model is important, but it's not gonna solve the problem for you. So we started to ask, okay, if that's the case, then what context is needed, or what is needed at all to drive accuracy? So we built agents that mined across many different areas. Uh, we had about 12 to 14 different iterations of this. And what we found is that by mining query history and your existing BI layers, you a- can actually build very reputable context that helps you get to 83%, and that's without any human intervention. So I've seen a lot in the chat, you need humans to kind of take it that extra mile.

3:43That is true. Getting you to 83%, though, is with the right context. And to go a little further, we wanted to say, okay, let's try this on some harder questions. Let's involve multiple tables. Let's start to in- involve, uh, unclear joins. And what we found is that the gain was massive w- for this context on multi-table queries. And this was across models, so these numbers you're seeing are, we're testing across models and harness, trying to keep it consistent on every run, of course. Um, and so then we took that original thesis on our models that, hey, the model's not as important, and we ran it again with this context that we had mined, and we'd found that that thesis holds true.

4:29So an open-weight model with the right context can be as or more accurate than a frontier model. This is really exciting 'cause this means as models come out, yes, you can just cycle them through, but you don't have to be as worried, uh, about which model you need to use, and you can actually start to think about tokenomics and token maxing, maxing the model as long as your context is correct. And then the, the final thing we learned through all these labs is that context looks different when you're serving it to humans versus when you're serving it to an agent. And this, to me, was personally when I started to really lock into this thesis because, as I said, I've been building semantics, systems of semantics for well over almost two decades, and I've seen the value of them.

5:13So this really helped me land on why context for an agent may need to look different. So just to give you a couple examples. One, for a metric, you might talk about a metric, uh, f- from a, a human sense, from a logical sense. For an agent, though, it may not be as accurate just thinking logically of what the metric means. It actually needs the definition. It needs things like the grain, and it also needs to know where it can and can't look from a SQL sense. Similarly, with a table, if you take the description at a table written in natural language versus if you take the DDL and its traps noted, an agent's about 20%, 20 points, apologies, more accurate written with the DDL.

5:58And so the takeaway here that I found was- Great. We've been building systems of semantics and systems of context for years for humans, and for abstracting data for humans. Now, if we flip that on our- on its head, what does that look like? And what type of context are we talking about? Austin showed us this, this chart earlier. The three types of contexts I'm gonna talk about are metrics in SQL and, and how they can be structured for agents, ontology, whi- which Ashwin showed a little bit, so I'll fly through that. If you wanna hear more about ontology, listen to the WTF as the context layer, uh, from Austin. And then I'll talk about skills a little bit as well.

6:40So the first artifact is metrics. Metrics, as we all know and love, are near and dear to our heart as, as analysts, so hearing the term metric is probably, yeah, sure, of course. But a metric for an agent is much different than a metric written for a BI tool. Oftentimes in BI, you end up having to refine the metric down just to make sure you don't have an overhead or a maintenance nightmare. Here, writing a metric for an agent, you can store a lot more attributes for it and version it and manage it centrally so many agents can refer to the same metric, which has been approved and iterated on along with the SMEs. So things like the grain, the exclude statements, the filters, the dimensions, the time dimension of it, all of this needs to be stored along with the SQL and verified queries that allow the agent to make sure what it's querying is deterministic and not completely probabilistic and written at runtime.

7:43The second for why questions is the ontology. So for ontologies, y- knowing the relationship across the business and the edges and how they're connected helps the agent reason through some of the harder questions. For example, why did our viewership drop on our drama shows? First and foremost, you need to understand what viewership even means to the business. You need to know what does drama correlate to, and what is a show. All of those things aren't defined in today's understanding of semantic views. Oftentimes people shove them into instructions, and ontology gets to store those definitions and those linkages so the agent can figure that out itself. And then the third for the how questions, we use skills.

8:32So an example here is, how do, how do I, I investigate a revenue dip? In this case, there may be many branches that come about based on the ret- the metrics returned or the type of data that's found. You may also need to think of revenue dips specific to how your business operates. So our business, well, not ours specifically, but a business, for example, could have a dip in Europe in Q3, and that may always be a thing. So when this agent goes to investigate the dip, it needs to know, oh, I see this dip in Europe. It's in Q3. Let me not call that out and then just close out. Let me keep going deeper.

9:11So that's great. These are the three artifacts, metrics, ontology, and skills. How do we recommend going about building these agents? We have three easy steps. One, bootstrap the context. Uh, Atlan has context agents. We're also making our context agents available in local harnesses. Two, manage that context, so make sure it's versioned, owned, and tested. And then three, run hill climbing. So this is where evals come in. Make sure you have good evals in place so you know what great looks like. Then as you run, you can fix misses, and you can repeat as you go and then find new use cases. So this is all great to hear and see. I'm sure some of this you've heard before or some is new, but let's actually see it inside of Trident.

10:00So really quickly, when bootstrapping in Trident, you can run our context agents, like I was saying, from within the platform. You could also run it on your local harness like Claude, Codex or Cursor. Once the metrics are built, you can see here, as I was talking about, that we have all the definition, all the formula, all the dimensionality, all the grain. The dimensions are also managed. We have SQL examples, so the agent knows how to write this deterministically. And we also have dependencies, so we can see which metrics re- relate to each other and which SQL queries and releases. So you can manage it by the version. Um, you saw ontology earlier, and 'cause we're out of time, I'm gonna skip that.

10:47Uh, and the final piece I'll say from a bootstrapping point of view is you can also run this straight from your native warehouse. So we have native apps for Snowflake and Databricks, or you can bootstrap as well. For time, I will pass off now to Austin, who's gonna introduce our panel.

11:05Austin KronzCool. Thanks, Zach, and we'll share, um, you know, a video walkthrough of all this after as well, so we can have all this. So, uh, Zach, thank you so much for walking through that. Ashwin, Rohan walked through kind of personal productivity and operational agents. Zach walked through specifically the analytics workload. But now I, I'm really excited, uh, to bring our panel to the stage. Uh, please welcome Joe DosSantos, uh, from Workday. We have Sowmya from Upwork, and we have Piyush from DocuSign, who's gonna talk through just what they're seeing in their actual organizations and how the challenges, uh, that might come up in building these types of data agents. So Joe, you're gonna be our fearless leader for this panel.

11:43I will hand it over to you to get us going.

11:47Joe DosSantosWell, thanks, Austin. I think, uh, you know, when we were starting to get ready for this, you- we were reflecting on how we got started on this conversation, reflecting on what we talked about when we were together at the Snowflake Summit and the Workday journey. Um, we're data nerds, and we like things to have the right answer. We like the version of truth. We like that, all that stuff. And so, you know, we really started to, to invest in semantics and to make sure that, you know, when that agent asked that question, we're gonna give it the right answer. Uh, and then we figured out that some of the hardest part was having that agent navigate which of the semantics was there to use, and dawned on us that we really needed some skills and some context to actually help it navigate that.

12:27And so, you know, with that comes this, uh, expanding use case of what we're talking about here, about these four different use cases. So I guess my question for the panel here is, you know, we heard some frameworks today around analytical use cases, operational use cases, and different ways that kind of provided that aha moment for where context was gonna be a, a game changer. So S- uh, Sowmya, let me start with you. Like what, what was the first use case? How did you get started? And how did you-

12:53Sowmya ParthanYeah

12:53Joe DosSantos... Come across this aha moment?

12:55Sowmya ParthanYeah, it's, uh, it's great to be part, part of this panel. Uh, for Upwork, we started this as an analytics use case. We kind of started to, um, test the waters with things like metadata questions, right? Like, and then questions that have, like, really crisp one answer to that question, like numbers in response. Um, and then we actually expanded to say, okay, like, there is a much broader use case here where, um, you know, you then ask, uh, strategic questions and insights questions. So while our starting point was very focused and, you know, we were trying to give the right answer, we did start to expand this out to be a little bit more interrogative and to have conversational, um, back and forth, um, and to get into more like, you know, what happens if you changed this investment and what would the output look like?

13:45So we started to have those questions as well. Uh, and it was really interesting to go from really deterministic, um, highly reliable answers to a little bit more, um, you know, uh, back and forth type of, uh, navigation.

13:59Joe DosSantosWithin the context of a companion, like-

14:01Sowmya ParthanYeah, yeah

14:01Joe DosSantos... So you can, how can you support somebody in that context of a companion? Yeah. How about you-

14:05Sowmya ParthanBasically an office on Insights companion.

14:08Joe DosSantosMm-hmm. Great. And how about you, Piyush?

14:12Piyush BhargavaYeah, at DocuSign, you know, our initial use case was not analytics agent. We started with a, a specific business flow, uh, in go-to-market. Um, and it was, uh, how do we help SDR, uh, prepare to engage a prospect? And the person used to look in Salesforce, research the company, understand the product fit, then formulate an email. And we thought this is great use case for agent to take on this work and help scale and get productivity. And so the agent, you know, researches the customer, goes out to the web, understand the use cases, you know, see the product fit with what we are, uh, and, and, and doing, and then comes up with a, with a outreach strategy.

15:01Um, now even though this looks like a traditional workflow use case, but this is about, um, you know, compiling information which is fragmented across different sources, web documents, metrics and data and, you know, putting it all together. And it, it turns into analytics problem, uh, on, on how do you make this repeatable workflow, um, personalized, uh, for, for that particular role and, and deliver the, and, and deliver the value.

15:33Joe DosSantosYeah, your story sounds very similar to ours, Piyush, at Workday. When we started, we, I mean, we started with the use case of I'm a salesperson, and I'm gonna go visit a customer. What is the thing that I would like to know? I would probably like to know what the current CSAT situation is. How many open tickets are there? Are there any issues that I should know about? Uh, are there any upcoming renewals? And then it would craft certain kinds of deterministic and probabilistic things to service that need. Where we got to was the part that you can-- if you can identify a very, very narrow set of things that you want people to do, you're pretty, in pretty good shape.

16:06You can figure out a way to do that. As those questions become a little bit less expected, as the expectation is that people just have Google inside their enterprise can ask any question, then all of a sudden you need to have a much kind of a richer way to navigate all of that, uh, information. And, uh, Sowmya, you said something that I really wanted to double back to. You had this kind of distinction between what needed to be right and what needed to be okay. I'm paraphrasing, right? And I think that one of the things that we are trying to wrestle with is, how do we think about curation in this context? What is the right degree?

16:38What is the extent to which we curate data to get the right answer versus letting agents in to infer some of those right answers?

16:45Sowmya ParthanYeah. I think, um, you know, expecting an agent to get a hundred percent right answer all the time, uh, I think it's a barrier to entry into something like this, right? So you always have to think about, um, like how to curate, uh- With a starting point and then let the context grow, and the agent learns from that and becomes better at, like, having these back and forths, right? So when we started, we started with a set of, uh, ground truth that we do know. Like, we know that there are, uh, there are a set of questions that we do know the answers to already. Um, and then we also added very specific domain knowledge.

17:25Um, so Upwork is a marketplace business. We have freelancers and clients working with each other on our marketplace, and we know how these, uh, different entities relate to each other. So we built that ontology layer, and we built the, um, sort of the, uh, domain data that our analyt- analysts have deep knowledge of, and provided that as context to the agent, uh, as well as, uh, you know, the metrics that then define exact answers that you need to evaluate your business with. Um, so combined with that, we actually had, like, this, um, almost like a, a accuracy range, right? Like, you could go anywhere from seventy percent accurate to ninety-six percent accurate. In fact, there are domains that are highly accurate.

18:10We're at, like, ninety-six percent for three of our domains right now. And then there are domains that are becoming increasingly accurate. But we do make it very clear that, like, you know, this is a range of accuracy and that we provide that number back as we have these conversations and as we, uh, have the, have our analysts and our product managers work with our product. And just having folks get comfortable with the idea that you don't have to be one hundred percent all the time, I think is a really, really important thing to, um, just align your organization on and be okay with, frankly, right? Like, we're data people, and we are-- we try to aim for, like, ninety-nine plus percent accurate all the time.

18:49Um, but that shouldn't become a barrier for all the advantages that a product like this can bring.

18:54Joe DosSantosIt's fascinating. So kind of labeling the domains with some kind of expected, uh, accuracy score. If you're giving a financial metric, it should be right. If you're giving-

19:03Sowmya ParthanYeah

19:03Joe DosSantos... Something around, you know, perspec- perspective, you know, what might happen, it's not gonna... It's ish. Ish is okay, right?

19:09Sowmya ParthanIsh is, ish is fine- ... Depending on, like, the tier that you're kinda dealing with, I think.

19:14Joe DosSantosThat's a technical term, Austin. You can use that, uh, for your next conference. If you want the ish, you know, the, uh, the context there can help you with the ish. Uh, Piyush, how about you? How do you think about kind of data curation? Oh.

19:27Piyush BhargavaYeah, I think-

19:27Austin KronzNo, I'm, I'm writing down the ish, so...

19:29Joe DosSantosYeah, please, please. Technical term. Yeah.

19:32Piyush BhargavaUm, no, I think that's, that's probably a contribution to this, this, uh, this talk today. Uh, I think-

19:37Joe DosSantosTrademark pending

19:39Piyush Bhargava... To, uh, to me, I think it depends on the question you're asking. Like, for example, we rolled out a doctor to agent where a seller or executive leaders can ask about customer facts, like Customer 360. What's the usage? What's the ARR? Now, there the expectation is that, you know, the answer is going to be accurate. You know, just like you go to a dashboard and you see that, you don't want to come up with a different answer every time. And, and then, then there is, uh, you know, questions about, like, you're crafting an email for an SDR. You are crafting a call prep or meeting prep for a seller to go into it. Now, by definition, these are things which are not deterministic.

20:25You know, they'll, they'll, they'll have a range. And, and in things like that, you know, it's not just the facts, but, you know, the tone of the message, how things are coming together. So to me, I think the accuracy and the curation, it depends on the kind of questions you're asking. And for questions where the trust is high, it does require a lot of, um, putting in verified queries, putting in, you know, um, you know, going through rigorous testing, getting feedback, evals, and building that into the process. Um, but one of our learnings has always been to have the human in the loop. Even though if you're crafting, crafting an email, make sure you start with drafts, have the people become comfortable before you have the agents, uh, in an autonomous, autonomous manner.

21:19Joe DosSantosI think that it's interesting. We- we've started to think about this loop that you're describing in two different ways. There's a process, and the way you described it is the AI is helping you with something. Don't you want the human to verify what's happening and to make sure that it kind of fits the right kind of, uh, approach to how you wanna attack that, that task? We've also been thinking around, like, um, uh, Sowmya, you, you've introduced this concept of, like, sometimes I know and sometimes I don't know. Even if it's a de- even if it's a, a deterministic query, it's almost impossible to contemplate you having every single semantic at the ready for every single possible answer, right?

21:53So we've started to think about this as shouldn't the, shouldn't the agent tell you what it knows and infer what it can't, and tell you that it's actually inferring using these skills, right? And the key part is, "Hey, guy, I'm trying to piece together an answer for you. This is not a verified query. This is not a definitive source of truth, but I'm helping you out by constructing SQL basically on the fly that helps you through this process." Now, at that point, you have a natural feedback loop, right? You have a person on the other end that says, "Oh, I agree with the way that you're attacking that problem." You have an ability to then send that back to a governance, uh, framework that basically says, "Yes, I do in fact agree that this should be the go forward semantic." And in some respects, what you're doing is you're crowdsourcing your versions of truth so that over time, your accuracy rate can go higher and higher.

22:42And the trick is really how loose do you set that free before people give up and say, "This thing is stupid and doesn't give me the right answers," right? So it's a, it's a question of deployment. I guess, Austin, as you started to see some of the kind of deployment strategies in terms of getting things right, have you noticed any kind of strategies that people use that are about how do they, how do they allow skills to give people directionally correct answer in a way that doesn't necessarily blow up the whole thing and make the user groups reject what they're seeing?

23:13Austin KronzYeah, I, I think the starting point of that is what we were talking about at the beginning, which is just understanding the types of questions you're expecting this agent to answer. Or maybe more broadly, what, what is the use case that this analytics agent is supposed to accomplish? Because you mentioned ish, and I put in the chat, like, if you need a hundred percent accuracy on something, dare I say, you might need a dashboard or like take the LLM out of the equation, right? Like, uh, there are things where, yes, I wanna ask questions about biometric free form and, and things like that. And I think in a, in a call that this group had, um, when we were getting ready for this session, uh, Joe, it may have been you or it may have been a collective, but there was a really good fallback strategy that I've been telling customers to implement, um, pretty much across the board when it comes to analytics agents, which is you should always start with certified metrics where you have them, right?

24:06And this is, you know, Sowmya, to your point, different domains will eventually improve accuracy. Not saying a hundred percent is the goal, but start with certified metrics. The fallback then would be, okay, if the agent doesn't have a certified metric it can find for that use case, go and just browse the hopefully AI-ready data. So give the agent a chance to go and try and find something that might be relevant for the domain. But the third piece of the fallback strategy would be we need to make sure the agent, whether in the harness or in a skill or something, is coming back to the user saying, "Here's which route I took." Did you take the certified metrics route or did you take the I took a guess, here's the guess, and here's how I got to the guess.

24:46So that kind of three-part fallback strategy seemed to be a good way to stop that barrier to entry problem, but at the same time, give some users and just really leadership the confidence that, okay, this agent can continue to grow, you know, and expand its scope, but at the same time, we can give some accurate answers up front. So that's, that's what I would say.

25:07Piyush BhargavaYeah, I think-

25:07Joe DosSantosYeah, I think that we started to... Oh, please, Piyush, go ahead.

25:11Piyush BhargavaYou know, I think that's a great point. I think the expectations needs to be set that AI can have errors, and people need to give feedback. And getting that feedback and then building that and, and building on that is really important to, to get started on this. Yeah, go ahead, Joe.

25:28Joe DosSantosThe thing I've been thinking about is I, I tell a story a lot that when I started at Workday, we had a thousand Tableau dashboards, and each one of them was adamant that it was in fact the source of truth. And even despite the fact that, you know, they all disagreed with each other, right? So we're kind of right back where we are. The kind of overconfidence of an agent does not... It's no different for me than the overconfidence of the dashboard developer that did that, right? And the, the idea here for me is I actually think that we're in this place where the, the act of creating a dashboard is actually the end user's responsibility as opposed to the BI developers.

26:01Shouldn't a person be able to say, "Can you just show me a chart of revenue over the last four quarters?" Oh, make it a stacked bar chart. Make it a line chart. Oh, can you give me an analysis? And all of a sudden, these things are expected, and they're all anchored around the, around the truth, right? So I don't think, I don't think dashboards are dead. I think that they are a different construct that are available to the masses as opposed to the person that took eight weeks taking a BI development course. And I think that is a wild productivity measure, right? Um, and I will tell you that, uh, we were talking about this before, that I think that even when people see agents working, they need to have some kind of a fallback that says like, "How do you know that that's true?

26:38I don't like that number." And then you can kind of direct them to the dashboard that supports it and go like, "Oh, now I believe the number." So I think that there's always this kind of idea of the data that we push out and the intrinsic quality that's assumed with it and how it's tied to the dashboard, and then how other people can kind of create their own assets around it. It's changing. It's not BI developers creating dashboards. It's people creating-- using Cursor to throw up a dashboard together. So let me, um-

27:02Austin KronzAnd then, Joe-

27:03Joe DosSantosPlease

27:03Austin Kronz... Maybe very quickly, I'll just add that this is, this is where it's not like this is replacing analysts, right? Like, there is so much power that analysts can bring to the table, and they are the ones that are closest to the business, and they are the ones that are gonna tell you if this needs to be corrected. So that ability to, um, crowdsource that input context and to crowdsource that feedback and act on it is really critical. So just don't stop with metrics and context, but keep growing it.

27:31Joe DosSantosI love it. Crowdsourcing the context. So let me finish up with this question for this group, which is, um, there are, there are things that we have learned around data delivery over the past decade. What do we need to unlearn? I mean, we've touched upon some, which is like truth is truth, and, uh, failure to deliver truth is the enemy of my people, my data people, my data clan, right? So what are some other things that, that you would advise other people to kind of give up or rethink as opposed, as we, as we kind of navigate the AI era? Let me start with you, Piyush.

28:03Piyush BhargavaYeah. I think I tell this to my team all the time. Like, a lot of times when we think about our role, we often constrain ourselves to the warehouse. Hey, what's in the warehouse? Structure data. And I feel like data is a continuum now. You know, anywhere, like whether it's in your transitional systems, whether it's in your, you know, support cases, uh, Confluence, whatever. You know, our goal now is changed into how do we make enterprise data accessible and ready for agentic consumption? So I think we need to expand our horizons in how we think about our role. So, so that's number one. Number two, I think, is that data quality has never been, um, more important.

28:50It's always been clear, but earlier we had some constraint. We create data. People consume through dashboard. With agents, you know, as we make data more accessible, every quality challenge becomes more glaring and, and the mistakes and the cost of them become a lot higher. And, and the third thing I'll just reiterate, go back to, you know, how do you create that feedback loop with your users? Um- And, and learn from your mistakes. It's never gonna be perfect, but having that closed loop process and building around that, I think it's critical to success.

29:27Joe DosSantosTerrific. Um, Sowmya, what do you, what do you think?

29:32Sowmya ParthanUm, I think, you know, a- accuracy means zero error. I think we need to change that. We talked a little bit about that. Um, I think it is really important to measure the quality, as Piyush said, uh, when you build these data models and really provide the right inputs to the agent when you can. Uh, but also just constantly learning. I think that's one important thing. The second, um, uh, I think this is really important to open up how you, um... You know, it's not just the data team that is going to continue to maintain these, um, content, right? Like, so just really opening up the partnership and making sure that we are including folks that are closer to the business, whether these are analysts, product managers, et cetera, that contribute into that context layer and to democratize, right, while governing that context layer is really key.

30:19Um, so I'd say that's the, that's the second thing that we have to actively work on. Um, and, um, you know, it, it's interesting what we saw at Upwork. We released this as an insights agent, but we see our partners using these as connected experiences, right? So they, they use this agent to connect it with another agent, and then they, uh, do workflows with that. So just figuring out how to empower our stakeholders to, um, use these types of agents for their specific use cases and to invo- to adapt, um, is really key as well. So I- I'd say those are three things that are really important to keep in mind.

30:58Joe DosSantosI've been thinking a lot about what it means to be a data person, and I've always th- thought that there's kind of like this language that is spoken by data people that is different from the language that is spoken by business people. And so business people will say, "You know, I need a view on m- revenue increases," and then somebody will have to then dissect what table does that sit in, what data torturing do I need to do to squeeze out all, every last drop of badness out of that. And the standard is always remarkably high, right? The, the standard of an enterprise delivery has always been really high. Meanwhile, some guy with a spreadsheet just downloaded something from Salesforce and is giving it to his boss.

31:35And I actually think that the, the thing that I've been trying to rethink is how do we actually unify that experience? How do we actually tie that person out there who has a job to do, who knows what that business function needs, who knows what that term needs, and how do you rope them in so that they don't feel separate from the enterprise function? The enterprise function, it always feels slow and deliberate, and it takes weeks to develop a dashboard. And meanwhile, Johnny, who's sitting next to me, will whip that data up in 10 minutes. And I actually think this is a massive... The context to discussion is a massive way to bridge that gap.

32:07How do you actually get the data and the context in the hands of people who will use it and then provide a feedback loop for how it's being used, and then it kind of redefines what data democratization means. It's not data availability, it's metric availability, it's context availability, it's business, it's business domain, and all of that things can be crowdsourced. And the, we in the center, in the enterprise functions, can just sit by and, and help e- help everyone kind of broker their data to each other like a great big eBay of data. Um, what are you seeing, Austin, in terms of the, the CDO role and what they are trying to, uh, overcome from some of their mental blocks?

32:46Austin KronzSo for me, a- as much as things change with AI, some things stay the same, I think especially at the leadership level. And a lot of, a lot of it is how do you scope w- what I'm gonna put my name on as a data leader? You know, are you gonna say, "Yep, we'll be, we'll have talk to data for all of our data," right? Like, it, it still usually needs to start with some type of use case, some type of scoping. We can get it right. The obvious question would be, "But I don't just wanna have talk to data for one use case. I wanna be able to talk to all of my data." Of course, but there's gotta be some type of iterative process to expand, scope, test, approve, move.

33:24Like, it starts to sound like a classic development life cycle, release management process, you know, testing and, and moving. So I think setting yourself up for success means managing expectations of what a talk to data agent can do, and I think that is just fundamental. You know, I think we've always had to do that in the data space, even when it was mostly people. But now that agents will have access to this, like, can you commit to, "Yes, we'll support, um, you know, agent-ready metrics for our most trusted and valuable use cases. We'll support AI-ready data for everything else, and we'll work with you to kind of evolve based on your use cases." I think that's how D&A leaders can stay on top of business value, manage expectations, and also, like, hang your hat on a couple of high, high-value use cases, and then be able to be a support, kind of always on mechanism for the rest of the company.

34:15But that's, that's what I-

34:16Joe DosSantosAll right. So we're gonna wrap this up, and I think we've got a few big ideas. Number one, I think that we need to help the users get in the loop and start the crowdsourcing process for business context and data. Number two, we need to help people understand about what's good enough, um, and trademark-ish might be okay for certain kinds of contexts, especially as we get into, into models. And I think number three, we need to start to, to think about the engagement model with the business in terms of how, how the DevOps process can be applied for people that aren't familiar with that process in a way that doesn't slow them down. So thank you to my colleagues for having, spending this time with me talking data nerd stuff.

34:55It's always fantastic, and I think back to you, Austin.

34:59Austin KronzAwesome. Thank you so much. Uh, I think, you know, sharing insights, it, it's... You know, we can demo as much as we want, but the practical, like, what does this actually look like in an organization is, it's probably the most valuable. So Joe, Sowmya, Piyush, thank you so much, and, uh, we'll keep moving. Uh, you may have noticed that- We are slightly behind on the agenda. So I will share one last thing, and then we are going to jump into our final ecosystem panel right after this. So, um, Ashwin and Rohan shared, uh, different workloads from the personal productivity to the operational agents. Zach just talked about how metrics and ontology are key, uh, factors for the analytics workload, and I think with the panel we just had, it's clear that talk to data is a high priority use case because if your agents can't interact with enterprise data, you know, just like we needed data-driven people, we need data-driven agents.

35:53Uh, and so there's got to be a way to do this successfully. We've curated a lot of our learnings from our labs and working with customers into a real open source kind of playbook, so you'll see a couple things on the screen there. If you're interested in what our forward deployed engineers are doing with customers and the playbook they follow to get to value with analytics agents, you can check that out with these links, and we'll of course send all this in the chat and as follow-up. But again, exclusive access right now. And we even have an accelerator. If you really want to get hands-on and work with our forward deployed team and build some of these things, feel free to scan this QR code, follow the link in the chat, and sign up for our accelerator.

36:31Because we are skipping over that special announcement, uh, I'll just give a quick teaser again, you probably heard in the opening keynote. But we are open sourcing the spec for AI registry, so please go check this out. And stay tuned for Ben, who's going to join us after this executive panel to, uh, talk a bit more about how we're thinking about registry, what open sourcing it means, and how it relates to Trident overall at Atlan.