Skip to content
TrackPodcasts
newsSep 3, 202643:50

Who's coordinating your army of AI agents?

About this episode

Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work through two ideas from his latest research. 

The first is Conversation Experience Orchestration: the wall between conversational AI and conversation intelligence is coming down, and conversations are becoming the layer that decides what happens next. 

The second is the AI agent control plane: as enterprises get pitched agents by every vendor at once, they need a shared operating layer to keep them coordinated. 

Listen to the full episode, and learn more about dialog AI at https://poly.ai?utm_source=youtube&utm_medium=podcast&utm_campaign=podcast&utm_content=podcast

  • Follow PolyAI on LinkedIn
  • Watch this and other episodes of the Deep Learning pod on YouTube

Get every episode summarized

Each time Deep Learning with PolyAI publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.

Email me new episodes

Free for 3 shows. No card needed.

Hosts & guests

Transcript ready

689 searchable segments. Every word is indexed and playable.

Who's coordinating your army of AI agents?

Deep Learning with PolyAI

0:00
43:50

Full transcript

Deep Learning with PolyAIWho's coordinating your army of AI agents?. Machine-transcribed; use the interactive transcript above to jump the player to any line.

I've been arguing for a long time that we're not taking seriously employee agent, agent turnover or churn as a metric for success of conversational AI. If conversational AI succeeds, your rate of attrition of agents should go down because you're improving their jobs. And I don't think people are looking at it that way. I think that's an important metric that nobody is thinking about in terms of measuring how effective conversational AI is. Hello everyone and welcome to another episode of Deep Learning with Pali AI. Today, I've got Ian Jacobs from Opus Research with me. Before we start, please like, share, subscribe. But Ian, thank you so much for joining us on the show today. Yeah, thanks, Nikolayne. Thanks everyone for spending time with us. Absolutely. And look, I think maybe before we go into anything in a lot of detail, what's getting

you excited these days in the world of agent AI and what it can do for customer experience? It's kind of funny using the word excited because in a weird way, what's getting me excited is that people keep failing. And I feel like I have a better way that I can pitch them. And I don't want to be like profiting off of people's pains sort of. But one of the things that we hear over and over again and have seen data, including like a survey from PWC of CEOs, is that they're investing in AI, but they're not actually achieving ROI. And what's interesting in the PWC survey in particular, is they're not seeing the ROI from cost takeout or from revenue generation. And it was like, I don't remember the exact number. 55, 56% of CEOs said that in the 2026 survey. My hypothesis is that in the world of agentic and conversational AI, one of the major obstacles is that every webinar, every podcast like this, every speech you see at a conference,

every bender pitch tells brands, you've got all this high volume, low complexity work that we can automate. Let's start with the easy stuff and automate that. And I'm going to argue that that's actually a big cause of the lack of ROI. I mean, just on pure economics, Nikola, what human labor are you replacing? You're replacing the lowest cost outsourced labor there, the cheapest labor you have. If you started to do things that were more complex, more multi-step, it automated an entire real process and then not like a password reset process, but something much more hairy problem like now you're starting to talk about like, you know, L3 people in the contact center are much more expensive and not that you're going to replace them entirely. But even if you start to eat away 30% of their labor, that's going to give you much better ROI than automating 100% of the outsource person in the Philippines or Colombia or wherever

your outsourcing is, right? So that's what's getting me excited. This idea that we can work with clients to identify the most complex use cases where they'll actually get the initial ROI that's going to make people go, wow, and open up essentially the imagination required to really transform the businesses the way that brands like yours are saying that, you know, these companies can with the right technology, the right processes, the right people and the right use cases. So that's kind of where I'm getting excited. There's a lot of nuances to that argument. I mean, I'm being simplistic for the sake of a podcast, but like that's what's getting me revved up these days and the fact that that message is resonating with people, that's getting me excited. I don't know what I mean, you guys are talking clients all the time or how receptive would they be to that kind of message? Well, look, I think that there's a lot going on. I think that like right before the hike really came on when we were still in our very early

days, that messaging worked better. And I think the ROI was better because whoever was crazy enough to buy into the idea that I can automate much of their customer service was already like a crazy person, right? So they were willing to be the evangelist inside their company saying, hey, this is coming. And they were right. I'm happy to say if I started a company five years earlier, I would have been in wrong for next five years, right? Because I think it did take all of them to really deliver upon the full promise, but you know, it just meant that like they really had to work hard to articulate why something would work. And now I feel like a lot of vendors, you know, AIs and everything in everyone's an AI company and La La La. And because of all that, you kind of got to a point where the nearer it is, they've gotten good. Like the low-hang thing like, hey, think. Tell your board you bought AI so they can get off your back because let's be honest, you're not that evangelist. Like you're there to tell your board and your boss that you bought AI, they can get off

your back for another quarter and another quarter and another year, right? And I think because of your spot on, I completely agree with you. They are doing, you know, like ITSM scenarios, they're doing like simple FAQs and all of that. And at the end of the day, those are really short calls. Right? They're simple. But I mean, we used to have this term called zero touch resolution. It used to be easier to start with the simple stuff because people couldn't do anything, right? And it was a way to start without the IT buy-in, right? With start, like deliver a bit of value, but we pitched it as like minimal value. It was the opposite of that. It was like, hey, it's going to do very little, but it's going to get there quick, right? And you know, like that's a lot easier to do still. Like where people, you know, recently, I think we started working with a Tier 1 global bank and it was like financial health scenarios, really hard stuff. And you know, for like the guy who stole that is one of our best guys.

And he was like, I'm just going to go out here where like no one's going to be like, well, tomorrow I'll be able to do that with open AI or with another vendor. Like it was just so crazy that no one even wanted to compete for that deal because the average handle was 30 minutes. And like, you know, I did chief architect of the company was in a meeting with us. He looked at it. If you guys want to try, you know, it's kind of like a medieval quest like you've the night wants to go and fight the dragon. Like, Tier have a sword. Why not? You're going to lose anyway. You know, even if you automate half of the call, you've now automated, you know, five password reset calls. I mean, the equivalent. I've password reset calls, right? Or or 15 whizmocalls for a retailer, right? Like those are very short, right? So, right. Yeah, like that. That's the approach. And I like that you point it out that in many ways, the start with the easy stuff is kind of just a modern take or a modern gloss on get some quick wins under your belt so

that you can free up some investment later down the line at some point. But we are at the point where executives have seen that the simple stuff works. And of course, it works because they're things we already knew how to automate. Like we or L or M's, you guys could do a very good job of automating password reset. It's not the flow is the same almost every single time. Same with whizmo. You look up in the same system every single time, the order status and you flip it back to the customer. And that's, that's it. So we're automating that which was already automated and automatable, right? And that's not any way to transform your business. That's as you're saying, it's sort of a way to say, hey, look, we did something. We have some quick wins. But I'm saying we're past that time. Like we need to be past that time. One because the tools are going to allow you to do things that you haven't done before,

sell products that you hadn't sold before, sell services, which will require a new way to support those products and services. And a way to measure the impact of that. I mean, that's the other thing we're not talking about, right? If you automated 15 minutes of a 30 minute process, you still have the human involved and you have the AI involved and you probably have the AI augmenting the human and potentially the human augmenting the AI, how do you measure that? We don't, we don't have the measurement paradigm across the whole industry for that. Like we understand how you measure AHT. We even understand, although people don't do it very well, FCR and those kind of metrics, but what would a metric for this hybrid work? Like we need to start to develop that. That's the only way that brands are really going to start to see major transformation. So that's why I'm excited about it too. So you know, as a late, I mean, you've written quite a lot about like, kind of just the

conversational experience orchestration and like the agent control plane and stuff. And I've read some of that work. I think it'd be really interesting if you could like give us a bit of an overview of that because you mentioned like kind of humans helping AI, helping humans. Like how should people think about like the relative importance of automating agents assist the data flow between those? Like do you see it working better than before? Yeah. So let me take a step back since you mentioned CXO or conversation experience orchestration. It's not a radical concept. It's that the idea is that we used to treat conversational AI and conversation intelligence as separate worlds. They had separate buyers. They had separate user within the brands. And often the conversation intelligence when it was applied the service process improvement was applied to the human side. Right. So you would do automated quality and that kind of stuff with these tools.

My argument and actually the industry is proving out that we're right is that that was a false dichotomy to start with. Right. Like if you are a good conversational AI company, you've already proven that you understand how to understand the conversations. Otherwise you wouldn't be able to have them. Right. Like you understand the structure of conversations. You know how to analyze them. Otherwise you couldn't have continuous improvement of your own products. Like it wouldn't work. Is it necessarily true that if you know how to analyze the conversation, you know, if you're an analytics company, then you know how to have the conversation. Maybe not quite as much of it. But it's the coming true. Yeah. Right. And we're seeing acquisitions kind of make that case, right. Nice and cognitive. For example, nicely analytics world, cognitive and the conversational AI world. And they're trying to figure out how to bring those together so that they're not two separate products. Then you layer in a agent AI on top of that.

So now a big action. So what we're actually seeing is that this conversation experience orchestration creates the flywheel or the loop that brands really need to have the conversation, understand the conversation in order to improve the conversation. And yes, that does mean things like the agent augmentation. And it's interesting that we have this weird world where we're not always clear when we say agent, whether we mean human or AI. And in this case, it's kind of irrelevant because it could augment either way, right. So you could have the typical agent assist kind of version of agent augmentation, the human agent or you have an AI agent tackling a complex process that's following some guidance that says we always treat platinum customers well. But we have a policy that says I can't do this thing. Let me go to the human and say, can I make an exception? I can't do it because my, you know, the, the probably holistic part of me wants to do

it, but the deterministic rules that I have to follow says I can't so you can give me the thumbs up for the exception. And in that way, the human is augmenting the AI. And that's only possible when you have kind of this combination of the conversational AI technology and the conversation intelligence technology because you need to understand the conversation and be able to analyze it in real time to be able to say, oh, let you reach out to a human, not necessarily for disambiguation of an utterance that to the game was already right like more complex things like, can you give me the exception the, the okay for this exception because we want to treat our platinum diamond gold customers better than we treat Joe Shmo off the street, right? Like that kind of thing. So that's where we see that loop really starting to play. And that's how that data flow really would work. And I do think that the conversational AI companies like Paulie AI have seen that for

a long time because you've been doing the analytics again for your own continuous improvement. It's just now doing it in real time in the customers environment to improve their service flow and not just the agent performance so that the agent AI agent says the right thing doesn't have hallucinations like we're moving past that world into feeding the intelligence in to create better service process flows that that's again, that's exciting, isn't it? I mean, that's pretty cool stuff. No, 100%. I mean, look, I think that we just look at like the increased capabilities of these models. You know, we look at, you know, I just last week looked at probably about 10 of our largest customers and the repeat colors there and what's happening afterwards where the human journey goes that sometimes we have sometimes we don't have access to. And really it is that same journey mapping. And you know, for the longest of times, I think we've been liked by a lot of our customers for just like honestly telling them like, hey, look, we'll reach like a limit to what

we can automate. Not because we understand or don't understand, but because we'll be the annoying people flushing out like things that you never really fully settled in your SOPs. And you know, if you turns out that took me a few years to learn this from when we were very small, but if you say this outright, you warned them, then you showed them, then you actually get credit for versus getting blamed on your technology not working because, you know, you didn't hit this rate or that rate. And what's really interesting is like the tax on whether you need a human, whether you're an AI agent or like a tier one human agent is really that like spiritually, that digital transformation and the like, you know, canonization of what your rules are as a business. Because honestly that person, even if it's a tier three agent, is just free styling. So like it's really for them. The question is, who do you trust the freestyle? Right? If I like that idea of free styling. Yeah, when we're looking at some of our clients work in terms of humans augmenting AI,

one of the problems that they have today is that they don't have a role for that. They don't have like a job rack for what that looks like. They don't know how to manage those people because who do they report to? Senator Managers for it. For a while, I was calling them kind of coming up with a goofy name, calling them bot Wranglers because back then we were calling everything bots before agents, right? And they had to wrangle. And that was often about disambiguation of utterances and voice AI or stuff like that. But also as you're saying, there's like some canonical practices that the brand is instantiated in some policy document somewhere that's been fed into the training corpus of the conversational AI agent. And that's great, but it makes it very hard for it to freestyle. If that's, you know, if there's like a probabilistic layer over some deterministic flow and it has to follow that flow. So in that case, clearly the brands are trusting the humans to freestyle more.

I think that that's changing. I think it will, I think it will change. But this is one of those cases where you do need some kind of human backup to be there to get that trust level up to the point where you don't necessarily need the same level of human backup. Again, my example of a policy exception, right? We want to have human judgment there enough so that it becomes a training set that we can then use to say, okay, for this type of customer with this type of conversation where we've done this and in the past three weeks, they've had six different interactions with us about this thing. Sure, go ahead. You can give an exception, but in these other cases, no. But today, that's going to be a human kind of inserting human intelligence, human empathy, and human understanding of policy into those agent to AI flows.

But I do think it's interesting that we are getting to the point where we're actually going to have to remake the customer service organization to create these roles. Like what are they? Right? You know, we could come up with names. Bot Wrangler was me being silly, but we could have something that sounds more corporate about what that person does, what their KPIs are, what their management structure is, and then what their career path is. Because as I kind of described it, it's a self limiting role. You train yourself out of it and then open up the path to the next thing. I think it's one of the reasons, for example, that the BPO's are looking at that as a path for their agents. Because if they're going to automate away all of the L1 work, what do those people do? Well, maybe we can upskill them to be this bot Wrangler thing for the next three years, and then we'll have to think about what the next role would be for them and keep moving

them up. And if the BPO's are doing it, brands can certainly do the same thing, maybe not at the same scale, because they're managing 50,000 agents or 100,000 agents, but same idea probably applies. No, totally. I've always called these best kind of new air traffic control, where really, when we started a poly and there was a glory days of chatbots and no one was ever going to call again today, like a voice that's king and you can do no wrong in a voice. Exaggerated in both cases, people just want the best modality and sometimes it's voice, sometimes it's chat, sometimes it's an app. They like, for the most part, they should just not have the problem to begin with or if you know, services that are too complex, they said, so they just need that support because they're doing something for the first time. But what I find really interesting is like, it's really, and you know, the latest monocross context engineering, like those tier three agents and managers, they have context, right? And they're trusted with judgment over places where the codified word of law in the context

center has not caught up with a better judgment of like 10, 20 years in that organization, right? So to me, like, they're really the ones that are just kind of like trusted to like guide the agent in those settings. And ideally, and I think maybe, you know, people are not writing about this enough, it's really how in advance you provision up to capture that context in those situations that creates like a higher ordered canon law for the context center to move into a higher degree of automation. And so, you know, if you have that like higher ROI, but people kind of just go like, is the technology there or not? It's like, are you there or not? Like how many times have you applied your SOPs to get to something that could be automated by, or you know, if you brought in like 5,000 new people in your romantic center and everyone else just did not exist, could they restart it? And how quickly could they get to the same level of performance because it's really the same question. The flip side of that is that some things haven't changed from your chatbot days.

And that, for example, there are some use cases where the modern technology, right now, PolyAI, could automate some of these interactions and the brands don't want to or shouldn't want to. Right? So the example that I always bring up, because it's been 10 years at Farrister, we had a lot of insurance clients and talked to them. If somebody was making a claim on a life insurance policy, they didn't want automation within a thousand miles of the front end of that interaction, right? Because somebody close enough to that person to name them as a beneficiary on a life insurance policy had died and they just wanted a human voice. Now, it's true. That was before the world of voice AI and 11 labs coming out with Joe sympathy or whatever the name of the voice model is that's supposed to sound the most empathetic and sympathetic. But still, like there are some things where it's not just that it's like that tier three stuff where we haven't figured out how to instantiate that context and other keys that

those people have into the policies of the context center yet because right, they're being paid for context center work anyway very well because they have the human judgment then the brand trusts them. It's also we're still in that world where there are some things you probably still want human beings for and will for quite a while simply because of the nature of the type of interaction that you're having with a brand. I don't think that's changed. I mean, what's changed since the early chatbot days is you probably can now automate a lot of that stuff and it would have been more difficult, you know, 10 years ago to automate that. Well, look, I mean, I think it's like Q and far between the regular chance to do it. I think we have the first kind of heavy bereavement workload about three and a half years ago. It's been running ever since. It processes thousands of calls every day out of a workload of a few tens of thousands daily and we get higher CSIS course than humans do on that thing.

But you're right. People do not believe no matter. And you know, like this is a public case study. It doesn't matter. People just, they won't do this until they have to until the service levels with humans have collapsed really badly. Whatever reason that you know, life in that contact control had to that then they have no choice but to try. So like, I'm fully with you. I think that people just won't take the risk if they don't have to and then when they have to, they'll take all sorts of risks and AI can do things, but it doesn't have to. Right. I mean, to me, it's always been like, you don't have to automate everything. And you never will automate everything. And you know, when you automate everything, it will still be a few hundred people running those. Yeah. Yeah. It's not exactly the flip side, but sort of a different take on that coming out of from the other angle or the use cases where brands think that they probably need humans, but actually you'll see things like CSAT, but also other important metrics higher with automation. And those are the things where there's some sort of social friction that's possible.

Human human interaction that's then removed when there's some AI involved, right? So collections is an example. It's embarrassing. If you have to talk to a human and build out a payment plan, you're more likely to do it. So we've actually seen statistics from customers that promise to pay goes up when there's conversational AI. We amount. It actually is smaller because the AI isn't gilting the human the way that another human would, but almost to pay is like a key metric in that industry, right? It's like a huge one. And we see that it's actually higher with AI. And then in the less common use case, we've seen in the, I guess you'd call it the sexual health world, like the Planned Parenthoods and others of the world, like having a conversation with a teenager is very embarrassing. And it's not that you'll get better results. You'll actually get results like the, the teenagers may actually have that conversation with the conversational AI that they would never do with another human being.

So that's another area where you would think it would be the other way. Like that's one where you always want a human being. But the data kind of shows something different that like you have to actually start to analyze the social dynamics of human interpersonal relationships to be able to start to identify which use cases are going to really sing for your organization. Yeah. Yeah. No, no, no. I mean, I think that's completely right. I think that like, bereavement calls are the flip side of that where I actually, people forget that if you are a contact center agent, not that much above minimum wage, and like one in 10 calls are someone losing a loved one talking to you about it. You're not going to last very long in that job because no one wants to have a job where they have to hear that someone's not near father, mother, sister died. Like that's just not, you know, the honestly, the problem doesn't even have to do with how much you're paid. Like it's just not something that you want to do because it's not even support you just hear about, like sad things, right? It's really, yeah, STDs and stuff like that.

That's where like bots reign supreme because how would that happen? Well, you know what, like the core is going to be pretty awkward following the moment that you tell someone about love, whatever, right? So I want one that hilarious. We've had cases of that and more serious health issues as well, like drug addiction and stuff like that where it's your people don't feel the stigma of talking to a bot. And this does have real world business impacts. I mean, look at the financial results of the public BPO's over the last six months. The more traditional customer service lines of business are actually doing okay. They're stable in some really successful ones. They're growing the areas where their businesses are shrinking are the ones where there's a huge cost and toll on the human beings in those roles, content moderation, for example, for socials, right? It's not just that the AI has gotten better at that, which it has and can actually take over a lot of that functionality. It's that those people turn like crazy and the BPO's needed on staff, psychologists

and psychiatrists and therapists. So you can also flip the equation and turn the mirror inward and say like, what's the impact on your own people of having those as you were saying, right, around sexual health, same thing or out sexual health around bereavement, right? Same thing, like wearing on people. And if you can remove that, it's not just removing like the brain dead work, right? It's like removing the work that takes us to cost on your psyche. Like that's an area to where there's a lot of opportunity for investment in AI to help your own internal employees, like just have a better job. I've been arguing for a long time that we're not taking seriously employee agent, agent turnover or churn as a metric for success of conversational AI. If conversational AI succeeds, your rate of attrition of agents should go down because you're improving their jobs. And I don't think people are looking at it that way.

I think that's an important metric that nobody is thinking about in terms of measuring how effective conversational AI is. And obviously that has economic impacts, right? Because the cost of recruiting, hiring, training new people, especially when your attrition rate is topping 50% is very high. So any time you lower that rate, but also, frankly, it's just a more human thing to do. You're making the experience for your own employees better, you're giving them a better job, and that's why they stay. 100%. I mean, look, to me, like, one thing that's particularly exciting is, you know, the contact center is a hard place, right? Like sometimes people in contact centers don't get like sufficient opportunities. And since we released our coding agent, run, like there are people in those contact centers that spend a lot of time, like just improving the agent and doing an amount of self service that was previously unimaginable, just because honestly, the previous tools that we and everyone else had were not usable enough by not making people.

And now we get someone going, you know, 1500 turns back and forward, run in a month, which is more than so, my engineers do work cloud code. And that's like outstanding. It's really there, the new sales apps to Salesforce are, you know, these guys to us and to kind of like, conversationally, I as a whole, but as they all build things, like you've written quite a bit about agent control plane and agent sprawl, like we great to hear a bit more about your views and just like what enterprises are doing, how they're containing this role, are they containing this role? Like many anecdotes you have. Yeah. The idea that drove this research was, or the conversations that drove this research, the talking to clients who said, yeah, well, you know, I just in contact center and getting pitched AI agents from the likes of poly, but also the likes of Salesforce and the likes of service now and the likes of variant and I write to all different components. It could be on the analytic side, it could be on the workforce, then workforce planning

side, it could be on the interaction side, it could be on the agent augmentation side. And I know that in my organization, we're also hearing from SAP about AI agents to help our back office and ERP stuff and our HCM vendor is pitching us and blah, blah, blah, blah, so we're getting all of that while developing our own AI agents internally using cloud code or whatever tooling we're using to build our own AI agents. So it seemed pretty obvious that they were having the early signs of AI agents spraw. They're going to have an army of AI agents who are all in theory supposed to work to drive the outcomes that the brand wants. And many of these complex processes cross all of these different little silos that the AI agents are being pitched for. And so what we started to wonder is what would it take for a brand to think about managing

this AI agent sprawl, the AI agent workforce army, holistically, what are the shared layers that any AI agent that you have should be able to tap into to drive consistent outcomes that you're looking for in your brand? We came up with five of these kind of shared layers, but I bet anybody with a good brain could come up with five other ones as well. So I'm never going to say that this was a census of everything that is required. These are just kind of the obvious ones, especially from a CX perspective because the lens, those are the clients that we're working with. So that's the lens we're looking at. The first one, it's interesting that you mentioned context engineering. We're calling it journey in intense state, but really it's the context. What context do you need to persist throughout an entire journey across every agent that might touch that journey to drive the outcomes that you're looking for? The next one is maybe the best understood layer when it comes to humans and one of the

weaker ones when it comes to AI agents these days. And that's identity and consent, right? The ID and V and authentication. Great. We have voice bio everywhere for humans. What about the identity and consent of the agent as it tries to do something, especially when you have multiple agents that might have the ability to do the same task, right? Because they access the multiple different tools that touch different systems. So like what is the watermarking here of I was authorized to do this thing. I did it. You can track it back, right? The next one is something I mentioned in passing before, which is policy and guardrails. We think about the guardrails piece in AI. We've been wondering about that and not wondering working at that for a long time now. But the policy piece, how do I build a consistent policy layer again that any AI agent, whether it's the one that's managing the work schedule of the contact center agent or the one that's

actually interacting with the human customer right now or the AI agent that goes and does some back office task that then feeds data back to the contact center agent. How do you have one policy that they're all following, right? Again, how do we treat our platinum customers as a very simplistic example of a policy, right? Here are the rules about what we want. It doesn't mean that it's specifying every single interaction will go this way, but it's more like here is the tenor of every conversation that we're going to have with our customers. And here's where we're willing to bend and be flexible for them. And here's where we're not all of that kind of stuff. There's knowledge as well. We're not so much focused on knowledge as a layer, meaning you've got to aggregate all of your knowledge sources into some giant data repository, right? Data warehouse, data lake, whatever we come up with next. I don't know. Data ocean. However, it's not that it's more the governance of knowledge has to be consistent.

And I mentioned that the different agents may have access to different tools, although also have access to different knowledge. But what are our rules across the entire company for data freshness? Who determines that so that the AI agent is always using the right one? Or frankly, we've got three different knowledge repositories, the kind of the same thing, which one is the AI agent supposed to use at that point, right? So it's the governance around knowledge. And then the final one is something I know you guys have worked on a lot again internally. But this also needs to be external facing. And that's kind of the e-vow. Right? How do you actually a valuation and testing and move it from a post-hoc but pre-deployment one time thing? Yup, it works. So that's the ploy it to continuous throughout the entire process so that it's an endless loop. And again, when we're talking about an army of agents, it has to not just be focused on a single AI agent and how that AI agent works, but the interaction between the AI agents

and the interaction between the AI agents and the humans. Again, my example augment the AI augmenting human human augmenting AI. So you have to be able to test all of that. I know that's a lot that I said. But those are like the five layers that we came up with that sort of seem like prerequisite if you're really going to go from having an AI agent or two in your A-A-Ball organization and one or two in customer service to be having orchestrator agents that are orchestrating 50 different kinds of agents, 50 different kinds of tools. And that's just in the service organization. And you've got the same in the mid office and the back office and in the supply and HBM and in legal because all of that comes to play in complex interactions with customers right? All of those different organizations, it's not very hard in a B to B environment, for example, to see where a service agent would need to interact with legal AI agent.

That's not an uncommon use case, but how do you evaluate that interaction between the agents and make sure that it is optimized that it's always working? Anyway, that's the long-winded answer, but it's work we did. We published that a couple months ago and it was really just to plant the flag in the ground because it's something that we're paying a lot of attention to. And I know one of the things that is the typical question when we talk about this is, well, who do I buy that from? You know, I want me one of them. Give me one of them. Robelines and it's a complex answer is you guys will know because you operate in a computer system and you happen to be particularly partner friendly as a company, but not all of the AI vendors are. Some of the hyperscalers come to mind, right? Like they want you to operate inside their own environment or be CRM providers or whatever. So for you guys, it makes sense that like you're going to have to interoperate at this layer for some of the big guys.

Like if you're a sales force shop, sales forces sales guys are going to tell you they can do this because that's how they operate. Like it doesn't, you know, it could be vacuum cleaners. Yeah, we do that, right? I mean, I'm being facetious, but it's that I say more and you know, like say anything bad about it. But yeah, you're absolutely. I don't even mean it as a bad thing because what they end up doing is like they figure out where the demand is and then they buy or build something and it takes a couple of years, but they're there, right? And they try and, right. So my point is that this is complex. And for the most part brands, especially today are not going to get this from one vendor. They're going to be working with vendor who understands how to do the identity consent, verification, authentication in the human and AI world, right? You guys will tap in to that authentication. All you need for your AI agents is, yep, we can check that, you know, this is verified. This agent has to do this thing.

We don't need to be the one checking it. All we need is the output of that consent process for examples. To do honestly that I think that you're absolutely right. And the autonomy you have is really good because it's kind of like, you know, for different clients, you'll see just where they've made more progress for less. But you hear that you hear this almost cracked at all because honestly, at that point, it might as well build it all on their own with a hyperscaler because they're one of the few companies in the world that can. Yeah. Very few. And that you have to be at the same level of maturity across all of these layers, right? That's clearly not going to be the case as a brand. I mean, you were talking on the vendor side as well. But I mean, also for the brand, this isn't one where you get the gold check across each one at the same time. You got to focus it on the things that are probably causing you the most pain as you start to deploy multiple AI agents first and build out from there. And your provider, whoever that is is probably as you're saying, stronger in some of these

layers and had the technology potentially built in a way that you don't even see, right? It's already there. But you have to start as a brand. You have to start asking the questions about all of these different layers that we're talking about in the tech, the topics that are inherent in each of those layers. Again, this is how you scale without that, it's going to be very, very hard to scale to the point where people talk about replacing 90% of their contact center agents unless you've got a 10% contact center, right? If you've got a very large contact center, you're not going to get there unless because you have those corner cases, the edge cases, the all. The reason you have it is that you've done something for many years to led you to the point where you have 10,000 people doing that. That, that. Just that. Anyway, so we're calling that the AI agent control plane. I will point out that there are a couple of vendors out there who also use the control plane terminology. They tend to be focused in the Eval and testing world, right? Because they're saying they're going to be able to control the agent performance by doing

testing. But we're, I looked at networking, they call it a control plane, they talk about having different layers. Like, all right, I'm just going to steal that concept. So it may not be the name that this thing ends up being known by in the in the future, but analysts love to name things. That's one of the things that we do. So, the right world, right? Yeah, I mean, computer networking is always a good one through things from. I think my wife ran the news, I was like, well, how does the internet work? I was like, I've taken many of the work that I could tell you everything. She's like, go to work, go to work. I'm like, okay. Networking. That's a good thing to steal from, for sure, as a call. But, you know, one of the keys that we didn't talk about, and this is something that's true, I think, in your more traditional part of your business, is that what's implicit in what I said is that each of those layers has access to all of the enterprise systems and data that would require to power it the same way that you have to have the right system access

for an agentic that tool to do anything. It has to be able to access the right thing. And then the AI agents can come from anywhere on top of that, right? So it could be the commerce agent, it could be the legal agent, whatever it is, and they all should tap in. So I think of this as kind of the sandwich, these control plane layers between the AI agents and the enterprise systems, because they, in some ways, dictate the access to the system. If the policy says you can't touch that system, don't the agent won't touch that system, right? So it kind of sits between it. I have spent time in the tech vendor world, but I never had to create a market texture before, but basically that's what I created here. You're, I'm sure you're much better than that. Much better. I mean, we have, now that we have Claude, I'm very good at it, but it's used to be some run to my, to my co-founder for, you know, and he, he was always the one that would have to bear that across. But, again, I think we're, we're running out of time.

This was a real pleasure, and I hope we get to do it again. And, you know, I, we'll see where this role gets to, but the next time we get to get a chance to have this conversation. Yeah, it's possible that there will be some recognition of the need for this control plane. I think the CXO thing is a done deal. People just don't recognize it yet, but the control plane's going to take a while. I mean, I recognize that. It's, it's going to take a while. We'll try to do it. It starts you thinking about it and starts, you know, some questions in your head about where you need to go. But, Nicola, thank you so much for having me. I really appreciate the opportunity to be here. No, it was a pleasure. And always, always a joy of conversation. So Ian, thank you so much for joining us to everyone watching. Like, share, subscribe, and we'll see you in the next one. Thank you, Ian.

More episodes

More from Deep Learning with PolyAI

View all episodes →