Microsoft AI CEO: The AI Future You’re Not Ready For

0:00:06 Welcome back to the Next Wave podcast, where we dive into the ideas shaping the future of
0:00:13 technology. Today’s guest is someone who’s been at the forefront of AI for over 15 years. He
0:00:20 co-founded DeepMind, he built Inflection AI, and now he’s leading Microsoft’s massive co-pilot
0:00:27 shift. Today, we’re talking to Mustafa Suleiman. In this conversation, you’ll hear why the so-called
0:00:33 AI training wall is a myth, how hallucinations might actually be a feature that we want,
0:00:41 and why the next software moat isn’t headcount or credentials. We also explore the agentic era of AI,
0:00:47 systems that don’t just chat, but plan, click, and get things done for you. And wait until you
0:00:53 hear the wild story about co-pilot Vision predicting the future of a flight that Mustafa was supposed to
0:00:59 be on. By the end of this episode, you’ll walk away with a practical playbook. You’ll know what
0:01:05 you can hand off to an AI agent this week, as well as how to think about moats in a world where anyone
0:01:11 can ship software in minutes. So if you’re living in AI or building right now, this one’s for you.
0:01:14 Let’s go chat with Mustafa Suleiman.
0:01:24 So I’m going to go ahead and start with your story a little bit. I know you’ve been in AI now for
0:01:30 over 15 years with DeepMind and Inflection and now Microsoft. I’m curious, what was it about
0:01:31 Microsoft that drew you to them?
0:01:38 You know, it is quite incredible to think that Microsoft is 50 years old, significantly older
0:01:46 than me. And, you know, for most of that time has been one of the leading tech companies in the world
0:01:53 and also adapted to every new tech revolution. So each new wave requires the company to pretty much
0:01:59 change its culture, sometimes changes business models, learn new programming languages, learn new
0:02:05 go-to-market motions and stuff like that. So I think I was always inspired by the adaptability of
0:02:11 it. And that’s kind of what we have to do now. We have to adapt to this co-pilot, agentic AI
0:02:15 companion era, which is just a totally different mindset to what the company is used to.
0:02:21 Yeah, absolutely. Have your beliefs or, you know, views on AI changed over the last 15 years since the
0:02:28 beginning of DeepMind to today? Yeah, I think when I originally got involved in AI back in 2010,
0:02:33 we were very, very obsessed with the AGI framing, artificial general intelligence.
0:02:39 Right. And I can still see that happening, but I think I’ve always been a bit more focused on what
0:02:46 happens day to day, practically with real users, you know, real consumers of technology and how that
0:02:51 makes people feel, you know, rather than like the super intelligence that solves all the grand
0:02:57 scientific problems. I still think that’s super cool, but I can also see it really living life
0:03:03 alongside us being an integral part of my day-to-day workflow as significant as your smartphone or your
0:03:08 tablet or having access to the internet. Right, right. So I’m curious about the whole like narrative
0:03:13 around AI training. There was a narrative not too long ago that the AI training was going to like hit
0:03:18 this wall and that we would sort of run out of things to train on. And then we saw the test time
0:03:24 compute and now they’re sort of moving that inference and the amount of compute going to that other end
0:03:29 of it. Do you think there is a wall? And do you think the answer to this stuff is just throwing more
0:03:35 compute at it all? Yeah, it’s interesting because the other side of that equation was people also saying
0:03:39 that there wasn’t going to be enough training data. Remember just before everyone was like, well,
0:03:44 pre-training is done and, you know, we’re going to hit this wall. And I think that the lesson is that
0:03:50 whatever obstacle, you know, comes up over a six to 12 month period, there are going to be five
0:03:55 different ways that people are trying to make the models more efficient and use compute in a different
0:04:01 way, generate data synthetically, which is what we’re all doing now. It’s like we’re producing human
0:04:07 quality data. Yeah, completely generated by one of these models for reinforcement learning from AI
0:04:14 feedback instead of from human feedback. So and if you look at like how big the models have got, yes,
0:04:20 they’ve got super huge, but they’ve also got tiny, right? You know, you can now train a GPT-3 capable model
0:04:29 at 100 times smaller in terms of the inference cost, you know, the cost that it takes you to produce an answer at test time than
0:04:34 it was when it was originally trained. That was only three years ago. So it’s getting cheaper, easier, smaller.
0:04:41 And it’s also getting bigger, more powerful, using more data and more compute. And that’s what you expect when, you know,
0:04:46 everything is working. It’s like having like struck an oil well. And now everywhere you look, it’s just like, okay,
0:04:52 we have another amazing capability that’s coming out. So do you think that like throwing more compute at it and just
0:04:58 kind of the bigger data centers, the more GPUs we put at it, that’s sort of how we get to the next level? Or is the next
0:05:04 level going to come from something entirely different? Yeah, I think that trend is definitely going to continue.
0:05:10 There’s going to be larger models, more compute. And we will definitely, you know, continue to get massive gains out of
0:05:17 that. But there’s also like an emerging craft now, which is in giving feedback to the models from other
0:05:24 models, right? So you get larger and more capable models to really efficiently teach smaller models
0:05:32 with very high quality data to perform a specific task really well, right? And I think that that trend
0:05:37 is also going to continue. And that’s where you get this kind of efficient frontier, like it’s very expensive
0:05:44 to build the biggest and best model first. And then it’s like 10x cheaper to build what was
0:05:50 frontier 12 months ago today, you know, just kind of amazing, right? So then you have this entire like
0:05:57 swarm of, you know, second, you know, phase or second wave model development capabilities, which are
0:06:02 coming. Very cool. So like stacking reinforcement learning on top of what we’re doing is kind of
0:06:09 what’s providing this sort of next era of AI. Yeah, simultaneously with just training larger models,
0:06:15 getting higher quality data, feedback from other AIs, you know, the kind of RL paradigm will still use,
0:06:21 you know, feedback from humans as well. So, you know, I think the lesson is that, you know, all of the
0:06:28 capabilities end up cumulatively being valuable. And you sort of chip away at the problem by both exploiting what
0:06:34 you know to be rewarding and exploring new techniques and new methods that then get piled onto the stack.
0:06:34 Right, right.
0:06:43 Everyone everywhere is talking about AI agents right now. But here’s the thing. Most companies
0:06:49 are going about it all wrong. HubSpot has gathered insights from top industry leaders who are implementing
0:06:55 AI agents the right way. You’ll discover which agent setups actually deliver ROI and how businesses are
0:07:02 automating their marketing, sales, and operations without replacing their teams. Get it right now by
0:07:06 scanning the code or clicking the link in the description. Now, let’s get back to the show.
0:07:11 So I want to talk about hallucinations for a second. So hallucinations, I guess, can kind of be
0:07:15 considered both a feature and a bug depending on how you’re using AI, right? So, you know,
0:07:19 if you need something factual, you kind of want to eliminate hallucinations. But if you want something
0:07:23 creative or you want to sort of solve new problems that nobody’s solved before, hallucinations are
0:07:29 something you want. How do we get to a place or do you see a pathway to getting to a place where we can
0:07:32 trust what’s coming out of these models. One of the things that I find really
0:07:40 interesting in technology is that sometimes a new trend gets a brand name or a label and then it
0:07:46 confuses everybody about what’s happened. You know, in some ways it causes like the entire field to
0:07:52 hallucinate about what they’re actually seeing. It’s a really weird word hallucinate because if you think
0:07:57 about the weaknesses of relational databases that have basically built pretty much all the value of
0:08:04 software for 40 or 50 years, it’s big weakness is that it can’t adapt. It can’t change. It doesn’t
0:08:10 have any fuzziness. You only get out precisely what you put in. Whereas, you know, this now kind of solves
0:08:15 a bit of that problem because it has this like fuzzy adaptive, you know, sort of abstraction that it can
0:08:20 transfer knowledge from one domain to another that it can interpolate. So, you know, you have two
0:08:24 knowledge points and it can join those knowledge points by finding the gap in the middle, which is
0:08:28 what you see when it’s generating a new image that’s in the style of something, for example.
0:08:34 So hallucination is just an unfortunate word. And I don’t really see that as a long term problem. I mean,
0:08:41 if you look at like three years ago, the models were really difficult to steer, right? Stylistically,
0:08:48 factually, they couldn’t draw on, you know, citations or reference documents that were reliable.
0:08:55 And now like two or three years later with more compute, they’re extremely controllable, right? They’re
0:09:01 very adherent to the behavior policies that we set for them. They’re very responsive to the stylistic
0:09:05 training. So they don’t actually produce a huge amount of bias. They still get some things wrong,
0:09:11 but it’s very, very different. And so I do think that that is one of the most promising signs of the
0:09:16 moment we’re in is that they’re getting easier to control and steer as they get bigger, which I think
0:09:19 was counterintuitive to a lot of people because a lot of people two or three years ago were like, oh,
0:09:23 these are always going to be misinformation machines. They’re going to be super chaotic. They’re
0:09:27 going to get it all wrong, et cetera, et cetera. And that’s turning out not to be true.
0:09:31 Yeah. And I feel like more and more now they’re sort of grounding themselves with citations,
0:09:34 right? So if they’re giving you factual information, you can see the citation and
0:09:37 actually go and look at the source and sort of double check things if you need to.
0:09:41 Right. Exactly. And that’s the definition of trust, right? If it knows when it doesn’t know
0:09:48 or, you know, when you go and like verify a claim that it makes, you feel confident that that’s a
0:09:52 sensible source, you’ll start to trust it more and more. And I think that’s, you know, also very,
0:09:57 very encouraging because the more people that use it, the better it will get. And then, you know,
0:09:59 again, more people will use it because it’s better quality.
0:10:04 Right, right. Now, I know you mentioned AGI is not really the area that you tend to focus on,
0:10:08 but I’m curious, do you have a definition for AGI? Because I feel like that term,
0:10:12 the definition is still fuzzy. And I feel like everybody who responds might have a little
0:10:14 different definition. So I’m curious what your definition would be.
0:10:18 I mean, that’s another challenge, right? Like hallucinations or, I mean,
0:10:24 words are difficult when you’re sort of trying to predict the future. Like, we don’t actually have good
0:10:29 definitions. So, you know, people talk about super intelligence or general intelligence.
0:10:34 You know, I like the method that we, or the definition that we use, the deep mind,
0:10:40 which is the ability to perform well across a wide range of environments. So the emphasis was on
0:10:47 generality and, you know, high quality performance, where the threshold is, you know, maybe it’s human
0:10:52 level performance in some tasks, maybe it’s all tasks, maybe it’s exceeding human level performance.
0:10:56 You know, I think we’ll have to just wait and see that the definition that I proposed in my book
0:11:03 was artificial capable intelligence. Because what we all really care about is not some abstract idea
0:11:07 of what intelligence is, which is really hard to define, right? It’s almost as hard as consciousness,
0:11:13 it’s really unclear, or soul or spirit, like, they’re very hard to define. But you can define
0:11:20 capabilities. You know, like, how much power does it draw? How many tokens does it produce per second?
0:11:24 Does it solve this specific challenge? Can it retrieve over documents in the web?
0:11:24 Right.
0:11:30 You know, can it take actions in your browser? Like, can it understand your scene and identify objects
0:11:35 when you use Copilot Vision to interact with the environment? They’re very measurable and specific
0:11:40 capabilities. So my mindset is always like, this, you know, month, this quarter, like, what are the
0:11:46 specific things that it can do? And, you know, that I think is a kind of practical way of handling that question.
0:11:53 Very cool. So I know you’re familiar with Jan Lacoon, right? He mentioned that he doesn’t believe that
0:12:00 LLMs are going to get us to AGI. He thinks it’s going to be some other type of AI model. Do you see
0:12:05 any other types of AI models after LLMs? What do you think is next beyond large language models?
0:12:14 I don’t. I think that we already have the kernel of very powerful systems because tool use is a meta
0:12:20 capability, right? You know, that’s kind of what has made us unique as a species. We’re able to, like,
0:12:25 pick things up, move them around, and then use that to manipulate our environment to augment our
0:12:29 intelligence, right? It makes us more smart and capable that we can make fire or that we can use a
0:12:37 stone axe or that we can, you know, create a GPU or invent electricity. So we now have this system that
0:12:42 obviously is nowhere near human level smart, but it can use tools. And we have a lot of very powerful
0:12:48 software tools around, right? And a lot of those are not yet hooked up. So I think that even if you sort
0:12:55 have stopped model development for a couple of years, you know, a core foundation model development
0:13:01 somehow magically, but just theoretically, then we’ve already got this kind of technological overhang
0:13:09 where the models have this amazing ability to use tools, talk to other AIs, source factual knowledge,
0:13:16 orchestrate things in sequence, right? And so, yeah, I don’t see it slowing down anytime soon.
0:13:21 Gotcha. I want to just touch on ASI really quickly. I was listening to an interview with
0:13:26 Naval Ravikant, and he said that he believes ASI, artificial superintelligence, is a fantasy. We’ll
0:13:29 never actually see it. I’m just curious what your take is.
0:13:34 I don’t know. I’m kind of one of these people who, whenever I hear never or fantasy, I’m just like,
0:13:39 okay, well, you know, it’s a bit like the yarn thing, like maybe saying that these models will never
0:13:46 scale to an AGI. I just think it’s really hard to say for sure either way. And I think that our
0:13:53 intuitions are changing every sort of year as we see these like massive wave of new advances coming
0:13:58 out. So, you know, I definitely think that there are certain capabilities, you know, with respect to
0:14:04 ASI that we need to be a little bit more careful of. For example, naturally, if you just give one of
0:14:10 these systems full autonomy with no human in the loop or on the loop or in any sense overseeing,
0:14:16 you know, it’s a pretty powerful system, right? So, I think we should think about these things in terms of
0:14:21 like, what are the specific capabilities we want to accelerate and engineer into the models? What are the
0:14:26 ones where we’re not sure, you know, and potentially could have very, very powerful consequences if we
0:14:30 mishaddle them? And really think about it from that perspective. Right, right. I also think it’s one of
0:14:34 those areas where we might see those goalposts continually move and nobody really knows how to
0:14:39 define it. So, it’s hard to go there, honestly. But what I want to talk about next is sort of some
0:14:43 of the concerns that pop up. You know, a lot of the things that I see like in the YouTube comments and
0:14:47 things like that when I make videos about AI. And, you know, one of the things that I see probably most
0:14:53 often is AI is going to take all of our jobs, right? AI is learning to code. AI is doing art. AI can be
0:14:59 your graphic designer. Should people be worried about AI taking their job? I do think the nature
0:15:03 of work is going to change fundamentally. I think there’s no question about that. You know, just as
0:15:09 50 years ago, there was no PC and everything was done on paper and we still relied on snail mail.
0:15:14 You know, I mean, we’re going through a big shift, which is going to affect how we work, what we do,
0:15:20 where we live. You know, that is a profound shift. And I think your day-to-day job has changed, just as
0:15:25 your job as a, you know, YouTuber and a media star now didn’t really exist five years ago.
0:15:25 Right, right.
0:15:29 Or maybe 10 years ago, it didn’t really exist. So, I think we’ve got to get ready for that transformation.
0:15:35 Gotcha. So, I want to ask really quickly about software companies. I’d like maybe some advice
0:15:38 from you on software companies, because there was this whole narrative that software companies don’t
0:15:43 have a moat because everybody can just sort of prompt a software into existence, or maybe one of the big
0:15:47 incumbent tech companies might just build it as a feature. What advice would you have to somebody
0:15:49 that’s trying to build a company right now?
0:15:54 Hmm, good question. I mean, it depends on the size of your company, but for smaller companies,
0:15:59 I think this is like an electric time. It’s pretty unbelievable. I mean, I was just playing with
0:16:05 this new version of GitHub Copilot now that has this unbelievable prompting ability, where in natural
0:16:10 language, you just give an instruction, or you just say what you want. What kind of app, or what kind of game,
0:16:15 what kind of program you want. And then obviously, it produces the code, you can then read all the code,
0:16:20 and then ask for specific chunks of it to change. So, in the process, you’re actually learning about
0:16:25 how code is written, where the bugs are, you know, it’s very, very kind of insightful. And that iterative
0:16:29 back and forth is a great way to prototype and actually build production-grade code and ship the
0:16:36 website, for example. So, the barrier to entry has never been lower, which means experimentation
0:16:41 by everybody is about to completely explode. What that means is that it’s going to be a very
0:16:45 competitive time for the big companies, medium companies, small companies, everybody, which I
0:16:51 think is an amazing thing for the consumer, because we’re going to get a lot of magical products and
0:16:56 experiences. But I think it is going to be a very competitive and explosive, you know, hard time
0:17:01 to really succeed in, because it’s going to be difficult to create those long-term moats, because
0:17:05 things are going to change so, so quickly. But I think what we’ve seen over and over with these
0:17:11 things is that people figure out ways of creating tremendous value, and that value always gets paid
0:17:16 out, you know, and people make, you know, big returns and have great outcomes. So,
0:17:20 I think it’s a really positive story. Yeah, I totally agree. So, last question,
0:17:24 I’m going to make it sort of a two-parter. What excites you most about what you can do with AI today,
0:17:29 and what excites you most about what we’ll be able to do with AI in the coming years?
0:17:34 You know, one of the things I really enjoy doing is just having a conversation with co-pilot
0:17:40 on voice on a commute on the way home. Like, at the end of the day, just chatting through a problem,
0:17:46 asking a question, rabbit-holing around something. I love basically learning. I love interactive
0:17:50 dialogue, and that’s kind of the way I learn. I mean, I love reading as well, but in some ways,
0:17:55 my books and my ideas just come alive because I’m able to kind of, you know, run down any corner of
0:18:00 a topic with co-pilot. So, I do love that experience. I’ve also been using the co-pilot
0:18:04 vision experience a lot. I mean, it sees everything that you see in real time. You can talk to it,
0:18:09 you know, about exactly what you’re both seeing. And that is quite a magical experience. I was
0:18:14 in LAX the other day and I opened co-pilot vision and it knew which airport I was in. It looked at
0:18:18 the gate number. It was like, oh, it looks like you’re heading to Seattle because it’s 4:30 on a
0:18:24 Sunday. So, there’s only one flight right now. By the way, did you know that your flight is delayed?
0:18:26 Oh, wow. And I was like, it’s not delayed,
0:18:31 copilot. You’ve obviously got it wrong. It’s hallucination. So, then I went to the front desk and
0:18:35 I was like, is the flight delayed? And they were like, oh, yeah, we’re just about to make an announcement
0:18:41 about it. How did you go? Yeah. Yeah. It’s like, that’s a magical experience. It is crazy. It’s
0:18:46 incredible. So, but the things I’m really excited about coming up are the actions. Copilot actions
0:18:53 is mind blowing. I mean, it will really just operate on your desktop in windows, highlighting areas where
0:18:58 you’re stuck, pointing at things, helping you with settings, or in your browser, like we’ll buy things,
0:19:06 book things, plan ahead. It’s quite magical to see copilot moving the mouse around, clicking on things,
0:19:12 opening new tabs, entering stuff in forms. You’re just like, we’re living in the future.
0:19:12 Yeah. Yeah. Yeah.
0:19:17 It’s amazing. It sounds like problem solving in windows is going to get automated pretty quickly
0:19:21 here. Exactly. Yeah. Amazing. Well, thank you so much for spending the time with me. This has been a
0:19:23 fascinating interview. I really, really appreciate you spending the time. Pleasure,
0:19:28 man. Thanks a lot. It’s really fun. Cheers. Awesome. If you remember one thing from today,
0:19:35 it’s that agents that do beat apps that wait. That’s the shift. So here’s your challenge this week,
0:19:41 pick one repetitive task. It could be booking a meeting, filling out a form, summarizing notes,
0:19:48 or cross-checking data and hand it off to an AI agent. Then let us know in the comments how it went.
0:19:54 All the links and resources are in the description. And if this episode helped you make sure to subscribe
0:19:58 and share with a builder who needs this playbook. Thanks so much for tuning in and hopefully we’ll
0:20:09 see you in the next one.