Monarch Sydney Panel with BCG X and Airtree: When AI Moves Faster Than the Enterprise.

Sam Senior 0:00

Beautiful. Okay. So throughout, we've got a load of topics that we're gonna cover tonight, but I think there's a question I want everyone just to keep in their mind as we're talking. If you think that the wave of intelligence that we're seeing from the frontier labs, so Anthropic, OpenAI, Google, some of the Chinese labs as well, if you think that curve of improvement of intelligence even has a 10 or 20% probability of actually following the expected curve that we're all talking about and we're all seeing, would you continue operating your business with the same people, the same org structure, the same products, the same offerings that you do today if that is actually going to continue? I suspect not, and we're gonna talk a bit about that from here.

Michael Schniering 0:48

And Sam, I missed my hot take, so I've already failed...

Sam Senior 0:50

Yeah, please go ahead

Michael Schniering 0:51

The first question. And you said to disagree on some things.

Sam Senior 0:53

Yep, beautiful.

Michael Schniering 0:54

My hot take is I think for some enterprises I'm working with now in the Australian market, we're in a little bit of a reality vortex on the potential that you're describing and this curve around AI. And I think we are just bumping up to reality setting in of the very necessary investment and hard change in organizations to get the results, and it's gonna have a bit of a hard landing. That's my view in the next 6, 9, 12 months, and then we're gonna come out the curve. For me, the hot take is this is like no other technology that we've seen, for sure. But we're gonna go through the same change adoption curve that we've seen for other technologies.

Sam Senior 1:33

Yeah, I think I'll call that the messy middle as we go into this later on. And just to start, who here has a Microsoft Copilot license in their workplace? Okay, so quite a few people. Keep your hands up if you have a ChatGPT license. Okay, and Claude? Oh, wow, I'm surprised by how many people have Claude. Of the people who put your hands up just now, how many of you have actually seen significant material change to your revenue, to your cost, to your margin, to your offerings to your customers? One, 2, 3, 4, 5. Okay, so we're at about maybe 15, 20% of the people.

Michael Schniering 2:18

I have some clients that have. That was a notable audience.

Sam Senior 2:22

BCG's doing a great job out there. We have a little bit of competitiveness between us. I was at Bain for 6 years, in San Francisco. So I live in San Francisco and I'm talking to the labs on a regular basis, meeting with OpenAI and Anthropic, weekly actually, really talking about what they are seeing. And the biggest message that, Boris Cherny, who is the creator of Claude Code, had about 2 weeks ago, was that Anthropic cannot move fast enough to deliver to the enterprise what they actually need. Everyone is coming to them and saying, "How do we actually deploy this?"

And that is the big thing that we wanna talk about. So let's start with that. So one potential myth that I wanna bust a little bit is that buying AI tooling is going to lead to transformation. So with that, Michael, what separates the companies who are getting measurable returns from everyone who is just distributing licenses at the moment? Because there's such a large delta in this room.

Michael Schniering 3:20

Look, to be clear, I think some of the licensing and the tools we're giving to broad-based employees in companies that I know some of you work for, having met a few of you, before this, is necessary, is important. It's about generating excitement, adoption, upskilling. For me, it's, a necessary precondition, but doesn't get you to the benefit and the impact that, candidly, boards and executive teams are now being held to account on, from AI. So, it's an important step, and the literacy that comes around it will be, critical for organizations to reskill the future.

I'll come back a little bit on some of the changes we're seeing in skills and people. But to your question, Sam, we are starting to see a pattern emerging, and not just for companies that were born in this era of AI, but for incumbents that are coming through, investing over the long term and getting results for, from AI. We've seen it in our work and we've done We actually do Every year we publish a series called Built for the Future, which basically tries to quantify what we're seeing in our client work and says, "What is the difference between those, getting impact from our AI from those who aren't getting impact, from AI?"

And there's a few things that stand out. The first, it's a pretty obvious one, they invest more. So usually 2 to 3X investment, depending on how you classify it, in data technology and AI systems. And they're investing in, I would say, not the absolute frontier leading edge, but 20% in agentic and AI, so that the promise of the new technologies they are genuinely investing at real scale in. So they're investing more. They're doing that against an agreed vision for the company. So they have a North Star, they know where they're headed.

And then counterintuitively, they do less than others. So they pick 2 to 3 areas, they focus hard, they rethink from the workflow and how either a user or a customer is gonna, feel or experience, the technology and AI, and then they set out a multi-year, multi-stage journey to go after those results. And it's in stark contrast to companies that, you know, you used the example of licensing and just giving general tools, but it's we also see that in proof of concepts and pilots. You know, hundreds of pilots, none of them delivering any results.

The companies that get value focus on a few things, and they do it really well. Maybe one other success factor we've seen, and Claire, love your, some of your examples from your work, is they then make the really hard decision and do some of the things that we've seen in multiple waves of transformation prior to AI. They do things like, let's baseline the processes. They do things like, where are people spending their time and how do we create productivity and effectiveness? They do things like, what are gonna be the new KPIs and incentives and metrics, to do?

So all these things that we've done through core system transformation, through digital transformation, through cloud migration, although for some companies the benefits of those things are also less clear. But those things that, these leaders do, they set the foundations up well and then they execute relent relentlessly against the 2 to 3 things that matter. And we've now seen that pattern recurring, but we still see the failure mode a lot, which is doing too many things and thinking I can spread it thin.

Sam Senior 6:56

Yeah. Claire, obviously there's a whole lot of change management associated with that. How do you think about the org component to all of that?

Claire Smith 7:04

Yeah, I mean, maybe bridging off what Michael was saying, like prioritization is something that most large organizations are not brilliant at, and there's always stuff happening in the organization that's outside of that. But actually, where we've been able to be successful is on getting really clear on, you know, problem first thinking, like what are we actually trying to solve here? And creating genuine co-sponsorship at an executive leadership team level. So it's not a technology project or a this department project. You've got 2 or 3 people lined up who are really invested in making it successful and understand what it is, and you build a cross-functional team around that.

That brings together, you know, the art and science of actually what's gonna deliver value. Because there's what the tools can do and what the data can tell you, and then there's what the people know in the business. And when it's really successful, you bring those things together and that's when you get some differentiated outcomes. I think what's happening now is, you know, and particularly with some of the focused areas, there's really hard conversations that need to be had about what does this mean for operating model? What does this mean for skills and capability?

What does this mean for, actually what is the value we're providing to our customers? And I think the it's critical that people steer into that. And it's I don't see it consistently happening in organizations that let's bring us back to 3 things that we're gonna do together that matter that people have bought into. And, you know, because actually to Michael's point, there is a really tough bit. You get from one point to, yeah, we can see some value, but actually to scale that out gets really hard and it requires that real organizational persistence.

So you've gotta start with that co-sponsorship, and you've gotta be prepared to have the conversations about what are the implications of the capability that you're building.

Sam Senior 9:13

Yeah, I love that. And I think the thing that I'm seeing the most successful organizations do is they're not just setting a very large grand vision and then saying, "Oh, I expect the AI tooling can actually deliver this," 'cause you just have this chasm of expectation not being delivered upon. And the best teams, the analogy that one of my team members uses, you can't swallow an elephant in one go. It's one bite at a time. So you think about that you have this large opportunity in front of you. It's identifying perhaps that's the frontier of what you think is possible in your business today based on the information that you currently have.

And then how do you work back and say, "What is the smallest micro opportunity I have that starts to serve that frontier?" Because as soon as you solve that one opportunity in your business, your frontier has suddenly moved because you have a whole bunch of new information as to what's actually possible in your business. And so what we have seen successfully is, we're working with a large European, energy provider, and they have a huge renewal challenge. They easy to get customers, but they fall out of the bottom. And they were trying to use AI very quickly to basically build a churn predictive, predicting model.

It was not particularly successful because they didn't have data across,.. They're a highly acquisitive business, and so they maybe had 10 different companies that acquired over the last maybe 15, 20 years or so. And so all the data about their customers was sitting in all of these different places, and so they couldn't run these models across a similar ontology where the data actually makes sense to one another. And so we worked with them to say, "Okay, let's start with actually bringing these systems together to begin with. Identify one small problem, and then we can use that to bounce off onto the next."

'Cause once we solved one, we actually discovered there was 2 new problems. We solved those 2, we now discovered 5 new problems, and it started to replicate over and over. But every time we moved them closer to that frontier, which was how do we predict when our customers are gonna churn? How do we therefore get proactive in understanding that, well, they have these signals, so we're gonna reach out to them in this way. We're gonna offer them this product or service. And this was something they were never gonna be able to do if they were just throwing the tooling at the problem. And they'd been trying it for probably the last year or so using the Frontier tools, and it just wasn't moving.

Claire Smith 11:23

Yeah. And, like, churn models, like predictive churn models have been around for a long time. And, you know, these AI does enable them to be more sophisticated, but it's actually that how do I actually activate that at scale and thinking through So there's the data piece. I've gotta get the data together. But there's actually how do I actually get the tooling and the capability to actually activate that at the kind of scale that's really gonna actually deliver the business outcome that you want, which is more customer retention, greater customer value.

Michael Schniering 12:01

I didn't know when to bring this analogy in, but it's probably the right time now. And Dave, you'll like this one. I think this goes back to my hot take. Sorry, Dave and I are a friend, and we both follow Formula 1. And I'll try and keep the analogy short, but it's something I'm seeing very clearly in at least 3 pieces of work we're doing right now. If you think about a Formula 1 car, it's two and a half seconds 0 to 100. It's 2 seconds for the 100 to 200, so they actually get faster.

And then two to 300 is about 4 to 5 seconds. With these AI and agentic systems, you know, the 0 to 100 is super fast. Like, you get very interesting results from teams, often well set up, cross-functional teams with the right ambition in a very quick, short order. And so the executive team, you know, you do a great showcase. The team presents how quickly these models are working and generating something new and of value. I'll give an example in a second. And the executive team extrapolates that progress, the 0 to 100 out, and goes, "Whoa, we can probably be doing 350 or 400K an hour in like, I don't know, 2 sprints time," which is, you know, max velocity of the thing you're trying to solve for.

And I'm seeing live now the 100 to 200K an hour actually is much harder than everyone expected because of the lack of data, the right level of company context needed, the integration into existing workflows, the customization needed, the unique requirements of users of this system. And this, for me, is now gonna That's part of that vortex we're gonna have to break through because actually expectations of leadership teams are rising. The general market sentiment is there should be huge potential, value potential of AI. Teams are trying to deliver on that. Most organizations I'm working with are now investing.

But it's this period of really tough change, challenge, time, investment that teams are gonna have to stay the course. And, you know, one of the examples I'm working on, we're doing agentic in B2B selling. So we have a, you know, multi-stage LLM-enabled model to help salespeople generate simple stuff, but sales presentations understand how to present customer results and drive to customer success, then do obvious cross-sell, upsell of their product range. And the first sprint, it was literally 2 weeks, the results were the 0 to 100 100 were magical. It produced materials that, you know, looked great, could be used, et cetera.

So the leadership team said, went, "Right, let's pilot it with 100 salespeople." You go to 100 salespeople and you get in their context, in their shoes, saying, "Well, I have to front a customer next week that's in a certain sector, that has a certain context. I have a certain relationship and certain history." And it's like, "Hang on a second. Your system doesn't take into account that context. It doesn't speak in the language that I would speak in this sector. It doesn't have the tone I would normally use." And all of a sudden, for a salesperson whose success is obviously having a great customer conversation and growing and upselling, that 100 to 200 becomes incredibly hard, and it slows down dramatically.

And I think this is what we need to be prepared for and why the leaders getting impact from AI focus on 2, 3, 4 big things that they have thought through well. Luckily, in this case, we have rebased the workflows. We know where we're headed. This was sort of an obvious hurdle. But I think many organizations are arriving at that point now where the hype And it's good hype. I'm bullish, by the way, on AI, just to be clear. The hype generated by the frontier labs of the potential I think is gonna meet reality of governance, security, what users need the reality of our context.

And it's gonna be up to organizations how they then stay the course, see this through, support teams through that change. And leaders are gonna need to say, "Yep, we're gonna go through multiple iterations of this, and we have to get this right. We're gonna support and back the team." Because if the organization flips and stop, you know, moves between one thing and the next thing, you won't get the impact.

Sam Senior 16:16

Yeah. Claire, I'm curious, what would make you trust a pilot enough to actually put it into production at scale? What are the things you're looking for to some of the points that Michael is making?

Claire Smith 16:28

Yeah, and it's maybe actually what are you comparing it to I think gets lost a lot of the time. It's like, that wasn't the perfect sales presentation that came out in your 0 to 100 sprint, right? And people will naturally go to, "Well, you know, I would have done this way and this da." But actually, it's like, what are you trying to measure as the outcome that you're actually seeking? And probably doesn't work in that analogy, but, you know, parallel runs, pilots of subsets of, like, trying and getting really clear on measurement.

I had teams that, where I was working, you know, they complain about how much time they're spending on measurement. It's like, yep, that's okay, 'cause you've gotta get really clear on what am I comparing on value. So that'd be the value side of it. And a lot of work on measurement and how you evolve measurement over time. And then I think there's, there is, and it's imbalanced in a lot of organizations, but do you, do you understand how you're gonna identify the things that go wrong? What are your governance guardrails?

What are the Yeah, what are those things and how are you gonna monitor and identify them? And having those systems set up as well.

Sam Senior 18:01

I'm curious from the audience, how often are these AI initiatives that you're building out being run by the business unit, either by the GM, the owner of the P&L, versus developers or the IT teams? Is Business. Business? Business. Business? Is that common across everyone? Mixed? Okay. Yeah. I think that's a strong opinion I have, is that these things need to be run by the business units and by the teams, to the points that you're making. You might need expertise from other parts of the organization. However, the salesperson, when you're running that pilot, they need to have, they need to have full context as to what good looks like before they're running that out and scaling to 100 people.

They need to be able to walk over to their friend who built it out and say, "Hey, this thing you just did, it was great in these ways, but it didn't work for me over here." And you need to have that extremely tight feedback loop at all times with as much context as possible. Otherwise, you're gonna end up in these very long waterfall cycles and frustration, and you'll end up with these organizations who are cranky and angry at each other and wishing that they didn't have to work together because they don't have the context of each other's information. And so I think about, how do you change the org to actually be able to support this?

Claire Smith 19:19

Yeah. And what we've done, and it's been, is spin up cross-functional squads where you've got business and technology, using technology in the broadest sense, right together, and you're able to run that through. And that, you know, gets you to those kind of, those feedback loops and those iterations. But I think that is a very powerful build stage model. And where I think it becomes more challenging is you putting this into run, right? I've got to Michael's analogy, I know nothing about Formula One, by the way. I've got to 350 miles an hour and I believe that I've got the value I can from this for now, and I'm gonna put it into a run state.

And that is quite often when IP drifts away, and, you know, the, you just I've seen value lost at that point. And it's something that I think needs to be thought deeply about in terms of operating model and also capability build and how you maintain that organizational knowledge of what this thing can actually do and what it can't do. 'Cause quite often you're in this expectation gap where it's like, "Well, I thought it could do this." "And it's not doing it, so therefore I think it's useless." That kind of stuff happens.

Sam Senior 20:51

And to your point earlier, Michael, it sounds, seems like you want to add something there. I'm also curious, when what do you think meets the requirements to say, "Hey, we're actually gonna run a pilot against this"? Because I think we hear people running, you know, 10s or hundreds of pilots, however, they don't go anywhere. Is it because we're choosing bad pilots? Is it because we have bad success metrics? What do you think that reason is?

Michael Schniering 21:20

I personally, I just kill pilots in organizations. I don't think it's the right way to do it. I think you need things that test and learn and incrementally prove out either assumptions or results so you can get a good return on investment for an organization. That's true. But I think if you have done some of the other things we see leaders do, which is you pick 3, 4 big areas, if you do some of the hard stuff that is transformational, I've baselined it, I know where the workflow is going, I know where the value is, what you're building is just the st increment or the first slice toward that end state.

It may or may not work, but for me it's less a pilot, it's more the first iteration towards fundamentally rethinking and changing the function. And if I connect it to your last question, there's 2 things that I see organizations still, including clients I've worked with that I've pushed them hard on, hard decisions that they're not making. One, for these cross-functional teams, pulling the absolute best talent out of the line, senior, and having them lead those cross-functional teams. I don't know why. I keep saying it. I think I'm saying it wrong, but, it just doesn't work any other way.

It's the best salesperson, it's the best person that knows how to market to a customer, it's the best person who's leading the pricing function, whatever that is. Taking them out and saying, "You're like an internal founder. You have a new job, which is to transform the function." I know it seems really simple, but I just see time and time again they have that person play an expert role 5% of their time, and it's not sufficient. The second thing is they don't change the operating model around that team's context.

So set up great cross-functional teams and then they bump into Honestly, with full respect to teams running data infrastructure, running core systems, et cetera, and the reality sets in that, "Ah, the data I expected to have for this cross-functional team to make that progress is not there," or it's of poor quality or the integrations aren't set up. And that's the reality of delivery, and I think it's the second thing that organizations And I have a lot of empathy 'cause it is hard to rewire the organization when you're in transition. Don't make that operating model shift.

'Cause if you do those 2 things, you move away from pilots. You move to a mode of "I'm fundamentally gonna redefine how we do the pricing function," or, "I'm fundamentally gonna redefine how we show up to customers using agentic capabilities," et cetera. And I know it's gonna take me 3 years and I'm gonna do it in slices and get ROI. Those that do that, and I have seen Australian organizations, to pay credit where it's due, do that. They're that is the path to success. Without those preconditions, I think it gets, astronomically harder to have measurable impact.

Sam Senior 24:02

So I broadly agree with everything you just said. However, I would also encourage leaders to be looking not just to their star performer, but also looking to that person who might occasionally have those ideas that just seem orthogonal to what you're actually trying to do, who might be a little creative, who've solved this problem over here. Maybe they're just sometimes just downright lazy, but laziness often turns into someone who's trying to solve a problem in a different way 'cause they're frustrated by something. So, I think, yeah, we want our highest attaining rep because they know something that's unique and interesting.

We want them involved in something. However, I do think there is an opportunity for the people who are maybe your C-grade performers in some areas, but A++ in others, where they have a unique way of looking at the world and can bring that into this project.

Michael Schniering 24:49

That's a fair challenge. Let me quickly reply and then you go, Claire. You know, you've done a lot of this in action. You've seen what it takes. I think that's fair, 'cause you also need the changed mindset. You need the willingness to think about, "I'm gonna partner with team members like a data scientist or someone who knows the technology or someone who's redesigning the workflow to rethink " You know, it's to break the inertia. So I think that's fair. It needs a certain mindset as well.

Claire Smith 25:14

Agree with all of that. One thing that I think is very important in these people that we haven't talked about is actually their influence, because they are, you know, they're the change influencer back into the rest of the organization that you need to adopt whatever it is you're building. You know, we're building a promo optimization capability and we're gonna focus on this category for whatever reason, but actually we're gonna bring one category manager in. Actually, there's a bunch To get the value out of what's being built, other people need to adopt it. And so whoever you bring in has to have that influence.

Sam Senior 25:56

One thing I've noticed, so I probably spent the last month or so talking to about 30 or 40, leaders in Australian enterprises. There is a very interesting breakdown of how they're thinking about the usefulness of AI. There's a very large tranche who are primarily focused on productivity gains and cost takeout, and there seems to be somewhere in the 20 to 30% range that are thinking about revenue upside. And so I wanna talk a bit about that challenge and how much you think there is around cost takeout verse there being a lot of revenue opportunity.

How would you break it down? What are you seeing across your customers? Because I am pushing people every day to think about the upside opportunity, because I think the cost component is short-termism and you're, and you're thinking about it from an FTE out perspective a lot of the time, rather than thinking about, "How do I make this FTE not have to do the kind of 30, 50% of their job that is repetitive and could potentially be automated and actually enable them to move into a creative space, allow them to have joy in their work and deliver and focus on delivering customer value rather than following that weekly report that they need to build or whatever it is that they're doing?"

So I have very strong feelings about this, and I think people shouldn't be as worried about AI taking people's jobs, because I think the jobs will transform and they will turn into much more fulfilling jobs. I'm curious what your thoughts are on that.

Michael Schniering 27:21

I mean, you were gonna set this up so we could disagree, and now this is I think your observation's right. So, I can only agree now. I thought we were gonna disagree on this. I think, I hate to generalize, but, it is, I guess the average of what I'm seeing in the Australian market, call it sort of private ASX 50 and equivalent private held companies. Actually that's not true. I'd say more publicly listed falls into this category, where the predominant focus is productivity and cost. And I think that's a natural, spot for a management team to arrive where you look at context of core systems, digital transformation, cloud migration.

There's been a history of mixed results, candidly, of delivering an impact. There's now a huge push, to implement, AI or there's a, there's a impetus for management teams and boards to show progress on AI. And the most measurable, quote-unquote "easiest path" is in productivity. So I Look, I can't speak for every company, but at least in my experience from what I've seen, it is more focused on productivity. So think areas like support functions finance and people. And of course there's an effectiveness element to this, so how you reinve invest that productivity is a choice for the organization.

Areas like third party spend and procurement, is another one. And so it tends to be on the efficiency side. I think in Australia, and I'm doing some work in the Americas and you're in the US, I don't think we've been forced through enough competition yet to really think about the capabilities on the customer experience, the commercial, the, revenue generating activities, where I think there's enormous potential, but is typically harder to get at the value in my experience. Needs more iteration, there's more mistakes. I've done 3 different pricing engines in the last 2 years. They've all been hard.

They've all gone through a cycle of candidly almost not working before you come out the other side, and it's tough going to stay the course and get the ROI. So I think if I generalize and compare what we're doing in Australia versus other markets, we are, I think, a bit disproportionately focused on productivity initiatives. And I think that is a question for us, you know, either as playing a role as a startup looking to provide product to enterprises, being in the enterprise, me as an advisor to enterprise. How do you think about now moving to those customer and commercial and other functions that are gonna be the, I think, the sources of advantage?

Sam Senior 30:03

Yeah. I wanna talk about the competition thing really quickly, and then Claire, jump to you. So I was speaking to a senior partner at a large Australian law firm recently, and he was really focused on this. He'd just come back from the US and he felt like everyone was focused here in Australia, particularly on the operations side of how do we take out cost. And he said the conversations he was having in the US were very different. How do they look at changing the product that they're offering? How do they look at out-competing?

How do they think about their pricing? And suddenly they were taking this, these AI capabilities to how they actually deliver value to their customer. And the big thing he was talking about was, "Yeah, I can be enabling my teams with Harvey and Legora and all these sorts of tools to go faster. However, what's going to happen is one of my competitors is gonna come in and say, 'We can do this M&A transaction for $1 million.'" Historically, maybe that cost us $3 million and where you're just using AI to figure out ways to be able to do that cheaper and faster.

And now suddenly, all of our competitors are still pricing at $3 million and we've brought that down to $1 million now. And we can now do, you know, 5 times as many as we were previously because we've invested so deeply in how we operate, how we deliver to the customer, and then we've used that as a competitive advantage for our product, our services, and the way we out-compete. And as a result, he's really expecting that there's gonna be a whole bunch of law firms who are gonna fall apart because their economics don't make sense any longer when you have these AI-enabled firms as the primary dominant, company in the market.

And so I think that's the shift I'm really looking for people to change, is it's not just about how we operate, but it's actually the value we're delivering to the customer and what that offering truly is. Claire.

Claire Smith 31:44

I think, I think it's a good example and I, and I agree, right, that does seem to be where the focus is in Australia. And I think there is the history, antique investment, there's the economic climate. There's a bunch of things that drive that. I think your example, Sam, was a good one, but that is It is kind of productivity into price. And, organization, I don't know a huge amount about, but I do know a little bit about. So Orica, they sell explosives and chemicals and things to mining. And they've kind of got to the point where there's They don't They less and less think of themselves as that.

You know, because actually their data is enabling them to use less chemicals, use less explosives to achieve the same outcomes for their customers. So they're actually now, "We're a digital solution provider that's actually gonna help you to be more efficient and more safe in terms of how you achieve your outcomes." So that's kind of the And I think that, like, that sort of You've got to reimagine your business and it's more mentally effortful than, "We'll take cost out of this process." Right? And there is more risk involved, and more investment involved.

So I think it's, you know, all of those things play in, but I do believe and agree with you guys that the winners out of this era will be the ones who do that imagination, risk-taking investment to, you know, take it to that next frontier.

Sam Senior 33:23

Yeah. I, the I think we spoke about this a bit earlier, was we're working with an aerospace manufacturer. They make engines for commercial aircraft. And he was telling me that they actually sell the engines at a loss, essentially, because they make all their revenue on the other side through the maintenance. One of the major issues with a whole bunch of supply chain issues over the last, you know, through COVID, et cetera, that hasn't fully recovered for them, is that sometimes it can take 6 months, 2 years, 3 years for them to actually get the parts in.

And so they're not generating that revenue as quickly on the maintenance. And so what they've done is they have a team of about 300 people whose entire job is trying to model and predict what is going to happen to these engines. And now they've shifted from not only can we make these people faster in their jobs, but actually how do we simulate what is going to happen in the future? So instead of just looking at our backwards data, they're building digital twins of these engines. And we're essentially helping them not just think about how they build these They're building these digital twins without us necessarily, but what are the implications of that?

So once the digital twin has come back and said, "Oh, we're forecasting that when this plane ends up in this certain situation, they're gonna need this part, replaced," we're then helping them think through how they bring that into their supply chain, how do they think about predicting the inventory that they have, and being able to out-compete their competitors by saying, "You know, maybe it's gonna take 3 months for this part to be delivered when you go with this engine company, but if you work with us, it can take us 6 weeks."

And so suddenly the speed to revenue for them is substantially higher. They can charge more because now they're able to have shorter SLAs. And so they have gone from not only can we find ways to be more productive in our team today, they're looking at how they turn that productivity into substantial revenue for them. And so I wonder if you've seen anything like that happening throughout the Australian market or have any, ideas for the group here as to how you would actually generate that internally and find those opportunities?

Claire Smith 35:17

Yeah, and I think the Orica example is one, and I'd extend that, engine analogy to where's, you know, the person in that market who gets to the level of confidence who says, "I'm not gonna sell you an engine. I'm gonna sell you running time on the engine." Like, "I'm gonna sell you the outcome that you actually want as a customer." That's a big shift. So those are, like, 2 examples, but you've probably got others.

Michael Schniering 35:45

I mean, starting more smaller scale before you get to business model innovation, I've seen consumer goods companies, even early stage we're doing some work to help a company launch an adjacent business, so outside of their core, use AI for things like synthetic customer panels. So basically massively speed up test and research using first and third party data to simulate different customer segments, so you don't have to physically do it. You know, as well as then cute things like sending AI to do customer research and ethnographic research. So the AI interviews, you get research.

Now, I know it sounds like a small example, but if you think about, NPD for a consumer company or if you go to other... New product development. Thank you for keeping us in check. Or you know, we're doing work in the US with pharmaceutical companies on R&D where you then you exponentially expand this micro example that I'm giving you. The upfront R&D process and product development process can be massively, changed. So even before you get to you know, full business model reinvention. There are, I am seeing locally, but they, again, tend to be at the smaller scale.

The teams haven't yet completely re-thought how they can speed up, product development, customer testing. They're at the micro-experiment stage of going, "Oh, that's cool. I can speed that process up and get a few more insights." But I think the opportunity is much larger, and it comes back to what we said before, which is if that then becomes one of the big areas, it is about the reinvention,... And then working back from it.

Sam Senior 37:25

And so if you're a leader in this room thinking about this and you think there are some opportunities to reinvent the business, how are you actually having that conversation internally? How are you aligning the business around the change that's needed rather And winning the hearts and minds of the business? Like, what are you tactically doing or seeing with your clients who are successfully doing this?

Michael Schniering 37:46

Okay. Do you wanna start? Or I'll give you some thinking music.

Claire Smith 37:50

Yeah. No, thinking music, that's fine. I mean, I think, like, one of the things that, I've done quite a bit is, build up networks in other markets that are ahead who might not be in the same category, but they have something in common. Like, they're a multi-brand, single category retailer would be an example. And getting those people to talk to the leaders in the business. There is something very powerful about them not hearing it from you and about them being able to ask questions of these people about what's on their mind and kind of noodle it through and sort of see it and sort of imagine things differently.

And I think there's Just bring it like, the more you can bring the outside into your organization, and then try and do that at key times, like before strategy cycles, before budget cycles, when people are thinking more expansively about, "What are we gonna do next year? What are we gonna " You know? So there's kind of those moments in time where you can inject some of that. That thinking, I think, is very effective in building sponsorship, would be one tactic I've used a lot. I've got another...

Michael Schniering 39:09

I do wanna, I do wanna stop you.

Claire Smith 39:10

No. Keep going.

Michael Schniering 39:11

Okay. I mean, then it in That generates sort of the context, I think, for people to change and imagine a future. One mistake I've seen organizations make is they get a team excited. I think we've gone past the days of having to educate on what's a cross-functional team, how do you work agile. I think most organizations know how to do that. The mistake now is they start too quickly. They just get going. They do, you know they do the first sprint to just prove something or get going, and they miss the important step of bringing the team together, understanding the baseline, imagining a future.

And I'm not talking about trying to slow teams down for months, but doing the necessary setup. Because if that team is gonna be together over 12, 18 months to fundamentally transform something, they need to have come together as a team and understood what are they transforming to and how are they gonna transform. Because it is not just a few LLMs or some new technology that's gonna come together to create it. It is fundamentally redefining, you know, all the functional examples we've defined. And far too often I see teams just lurch and just get going versus doing that, call it an inception process, call it a planning step and then getting going.

And by the way, discovering some things like, okay, do we have to solve security? Do we have to solve, access to data infrastructure? Do we have to Do we have legacy systems that are gonna need, be needed in 2 iterations' time that actually the services aren't available? And it's this sort of thinking, that I don't see common and happen repeatedly every time. And I think for me it's that stuff that brings together a good team to then make progress.

Claire Smith 41:05

Yeah. And it's like with every wave of technology, we sort of have this collective amnesia and we need to go back to it's people, process, and technology. Or we kind of forget everything that we learn in the digital years around design thinking and how do you, how do you really get clear on the problem that you're trying to solve before you start solutioning, which is a version of what you're, what you're talking about. So agree that is important.

Michael Schniering 41:31

And the last one, sorry. If you are leaders, and some of you are on executive teams I know of these organizations, I have heard people say, "Look, can't we just vibe code this now? Like, isn't this just surely in 2 sprints you can do Like, why am I setting up an engineering process? Like, don't you really? Do I need people?" It's like the last thing these teams need to hear. Like, of course, in the end we'll get to harness engineering and all this stuff. But it I think how leaders interact now with their teams and what agenda and context they set is incredibly important.

And I It's the other thing I'm working with my clients on to say, "Look, broad-based upskilling around AI fluency is great, but you need to now train your next generation of leaders to understand how to lead these teams through this change." So for example, we at BCG, the senior team has a 4-level certification process now on AI, and there are carrots and sticks with this. So if I don't go through my certification, I don't get access to the latest tools. So I can't access Claude Code, co-work, et cetera. That's the highest certification if you're gonna use tools to code.

So I don't That's my carrot. I need to do that to get access to the tools. I also have sticks, and it's actually happening this month. They are starting to take things away. I won't explain what they are. I won't bore you. But real things that matter to us to ensure certification. And that's at the leadership level. That's the senior management of the company all the way through to the CEO. My view is Australian organizations need to send their leadership teams through similar things. And it's not just cursory bring a frontier lab and have a demo of how to use something and everyone gets excited. This is hardcore.

What does it mean to have a secure setup of this? How do I access infrastructure? What does good data management mean? Like, all this stuff that I'm guessing people in this room kind of know well, I think it's incumbent on executive teams to set the bar that high in their upskilling. I have seen some organizations, I won't call them out, that are at that level, and I think it's awesome. But generally, they aren't. And I think you can really misdirect large pools of team members trying to make progress if you don't, if you don't do that.

Sam Senior 43:39

So as a, as a founder who's unrelenting in the ambition that we have as a business, I absolutely overstepped this really poorly across my team, and I caused a bunch of chaos in the business for not a super long time, but probably a couple of weeks. Because I had the, all these visions of what was possible, and I was looking at our competitors' products, and I thought I could absolutely vibe code that on a weekend. And I did. And that's, but that's exactly what I did. I vibe coded it. And so it was probably 20 times bloated what it needed to be.

And I was telling our team, "Oh, we could put this in front of a customer. We could pilot." And they're like, "Mm, here are all the vulnerabilities that are in here when we look at this from a security perspective." And so what we ended up having to do is, and had to align my team on this, is I have full remit to have my vision, have my ambition, have my desire to build things, as long as it's not expected that I'm gonna be able to push that into production within a day or a week of doing this.

And so I could really set the bar for what good would look like and what's in my head and what I'm hearing back from customers, and have the agreement with my team that it's gonna go through a process between what I have done, what I think is perfect, and what actually is in our customers' hands. And so my I would think about this as, with my team of like, I built something that is 70% good enough. And I think we all know all the hard work is in the 90% to the 100%, and that actually takes substantially longer.

And so we now have very strong, firm agreements with each other as to what I should and I should not be doing, and what my expectations should be. And so I think it's really important we're not limiting the ambition of leaders who are out there talking to their customers, they're talking to other companies, and they see the opportunity that's in front of them, and they just want to get it on paper. Because once you do, your teams will move faster. They'll know exactly how to deliver the thing you're talking about, and you can't expect it to be at the same pace that you were doing it at. And so I think there's this really interesting tension that you need to make agreements with one another about as senior leaders.

Michael Schniering 45:34

Yeah. I mean, come on. We've all done the vibe coding and 0 to 100, and isn't this magical? And, you know, every consultancy and frontier lab and everyone is saying how exciting it is and how fast you can move. I'm trying not to. I'm trying I'm bullish, by the way. I'm not trying to drag us down. I'm just trying to say, you know, there's a reality of delivering results that I'm seeing now that is the hard stuff, and that's where it needs to come together. But this sort of speed to 0 to 100 is quite infectious.

You know, you get colleagues going, "Look what I vibe coded. Look what I quickly did. Look at the Copilot-generated emails that will send automatically that are super obviously written by Copilot." By the way, I'm so sick of it. I don't know. Others? Not just the em dashes, just the, "It's not this, it's that." It is killing me now reading these emails. Anyway, sidebar. But we've all done it. But I think how we interact with teams that actually have to build this at enterprise grade, touching a customer, that's, from a leadership perspective, sort of skills and capability we need to upskill to go, "Let's lead teams and organizations in a way that can get the best out of that and move the fastest in that context."

Sam Senior 46:41

Yeah. Okay. So we're gonna do Q&A in a moment, and so please think through if there's any questions. But we'll finish on one last topic, because I know Claire has to leave soon. So speaking about going faster and risk, I wanna talk, very quickly about moving faster does not necessarily mean you need to accept less control, though. So any thoughts on what you have seen in terms of being able to move responsibly at speed and where you should be accepting risk or where you are willing to push the boundaries or you've seen your customers do that and your clients do that?

Claire Smith 47:19

I think where some of this goes wrong is you get policies written by people who aren't close to the work. So when we were developing our sort of responsible use of data framework, and AI, it was actually done with the teams who were doing the work, with real problems that were on their minds, and then how do you codify that through? So I think that's probably the first one. It takes a bit of time to set it up, but if actually people it's in people's language and they understand it and they're being part of the process, your adoption is a lot quicker.

And then I think at a more macro scale, there's this big rethink that everybody needs to do of their governance processes inside organizations. Because what most organizations don't need is another one. It's how do you evolve your existing, you know, sort of architecture forums and those kind of things to bring in more of the roles that actually need to be involved in these kind of decision-making and actually change some of your existing governance forums to enable that. I think one of the things that is unfortunate is the tooling to actually monitor the effectiveness of the governance you have in place is running behind the deployment in most organizations.

And you do need some sort of deterministic layer. And some, you know, we were talking about it earlier, some really good cultural settings around you built and deployed the agent. It reports to you. You're accountable for its outcomes in the same way that you are as for one of your team members, and you need to provide it for feedback and goals and all this sort of stuff. And what it's delegation of authority is, like, some of that thinking needs to come in as well.

Sam Senior 49:15

Yeah. And I...

Claire Smith 49:16

So a few thoughts.

Sam Senior 49:17

Yeah. I wanna also finish. My final thought really is that the human should be in the lead of all of this. We should be moving from a human being in the loop, 'cause that was the thing for the last couple of years, really thinking, "Oh, when needed, a human could come in and intervene." But the human should really be leading this all along the way, to your point of, yeah, you have accountability. And so maybe it getting to 70% good enough on a workflow is actually good enough if the human is leading it and you're, and you have built systems and processes that ensure that you have full understanding and you can fully evaluate and benchmark whether the thing is improving and doing what you need across your business.

And so I think that is a concept that people are just starting to wrap their heads around, and I would really encourage others to really think about how is the human truly leading this rather than saying, "Oh, my Claude agent did this," or, "My ChatGPT agent did this." That's not an excuse for something going wrong. It is on you as the owner and the leader to be doing that. Any final thoughts there, Michael?

Michael Schniering 50:15

Just, I mean, just a quick one. We did, BCG did research with MIT Sloan on this topic, just a few months back actually. Really quickly, what it showed was 2022, about 50% of organizations had a responsible AI, framework capability policy. That's shifted to 85%. Sort of an obvious finding, I think, given the shift. Get below the surface, half of organizations that have that either were at low maturity or worse, actually, 25% of organizations, this was self-reported, but in some ways that's indicative, had scaled a scaled responsible AI, but not at the level of practices, workflows, embedding it with teams that actually had to execute.

It was, my words, not the, not the findings, a surface policy level that they had scaled. And I think that's a tricky and dangerous position to be in because it's a perception of having good governance and responsible AI without actually having it implemented. And I think leadership teams and boards need to be exposed to the reality of where are the vulnerabilities, just how hard it is to implement practices that work at the user and/or workflow level, and then make continuous improvement and continuous process to try and keep up with the pace of development.

'Cause the perfect prize is you have a great system that continuously improves, adapts, so you can enable your teams to move really fast. You set guardrails so that within those guardrails, they can move fast. We, as BCG, I think we've taken a very liberal approach to opening up all of the tools, but we've done that with certification, with very strict compliance and governance, with very strict rules on data. So we've put a lot of the scaffolding in to then move really quickly. Our organization, I think, is in some ways a bit simpler, candidly, than a bank or a telco or an airline.

So granted, we're a simpler organization, but we did that so we could then have our teams move quickly. And I mean, we are one of the biggest users of the LLMs, both paid You know, at one point we had the highest number of custom GPTs in OpenAI, like on the planet. So we are, we are moving really fast on this stuff internally to change our skills. So I think we're a small microcosm of a simpler organization that provided the scaffolding to then allow people to move quickly.