AI transcript
0:00:07 PUSHKIN
0:00:11 Hey, it’s Jacob.
0:00:16 There is no shortage of tech podcasts, but few of them actually give you a dose of the
0:00:17 future.
0:00:20 The A16Z podcast is the exception.
0:00:25 The chart-topping show from Andreessen Horowitz brings you to the world’s most influential
0:00:31 people, movers who have a track record of being both early and right, like Apple co-founder
0:00:37 Steve Wozniak, or Andreessen Horowitz co-founders Mark Andreessen and Ben Horowitz.
0:00:41 Not to mention folks you don’t typically get to hear from, from the very first CTO of
0:00:48 the CIA to the chief security officers behind OpenAI, Anthropic, and DeepMind.
0:00:54 These guests tackle the most important trends in technology, from space to supply chains.
0:00:58 Want to understand the technology, policy, and economy of deepfakes?
0:01:00 They just broke this down.
0:01:05 Curious what really happened with the recent data breach of 3 billion records?
0:01:07 They covered that too.
0:01:11 Are autonomous cars, boats, and drones still miles away?
0:01:13 Listen to find out.
0:01:20 So go ahead, eavesdrop on the future with the A16Z podcast.
0:01:24 Just a quick note at the start of the show, we are very eager to hear what you think of
0:01:29 the show and how we can make it better, how we can make a show that you like more than
0:01:30 the show we’re making now.
0:01:34 Tell us what we should do more of, what we should do less of, and what we should do
0:01:35 wildly differently.
0:01:39 Email us at problem@pushkin.fm.
0:01:47 Again, problem@pushkin.fm, I’m looking forward to reading every email.
0:01:53 Artificial intelligence feels very abstract, feels very ephemeral, feels more like an
0:01:56 idea than a thing.
0:02:02 But there is, in fact, a thing there, AI is rooted in the physical world.
0:02:08 The thing or things are these little pieces of silicon and metal called graphics processing
0:02:10 units, GPUs.
0:02:15 These GPUs are expensive, they cost thousands of dollars each.
0:02:19 If you want to build a state of the art AI model, you have to buy tens of thousands of
0:02:27 these GPUs, and most of the GPUs come from a single company, NVIDIA, which is why in
0:02:33 just the past few years NVIDIA has become one of the most valuable, most important companies
0:02:40 in the world.
0:02:43 I’m Jacob Goldstein, and this is What’s Your Problem.
0:02:45 My guest today is Tay Kim.
0:02:49 He is a staff writer at Barron’s and the author of a new book about NVIDIA.
0:02:53 The book’s called The NVIDIA Way.
0:02:57 The book, of course, winds up being a lot about Jensen Wong, who co-founded the company
0:03:06 back in 1993, and who is still the CEO, and who is really a wild, kind of terrifying, brilliant
0:03:07 figure.
0:03:12 And Tay and I talk a lot about Jensen later in the interview, but we started with this
0:03:19 moment in 2001, that led from NVIDIA being a company that made graphics cards that let
0:03:26 people play video games on computers to becoming this key company at the center of the AI revolution
0:03:30 today.
0:03:37 So you write about this moment in 2001 when a researcher, an academic at the University
0:03:42 of North Carolina, realizes he can basically hack these graphics cards, this hardware that’s
0:03:48 just made to make pretty graphics on the computer, to help in his research, which is like not
0:03:50 at all about graphics, right?
0:03:53 It’s like modeling the weather or something.
0:04:00 And it seems like this moment is a big, winds up being a big turning point in the history
0:04:01 of technology, really.
0:04:02 Tell me about that moment.
0:04:07 The key moment was with Mark Harris, where he was a researcher at the University of North
0:04:08 Carolina.
0:04:14 And him and a bunch of other academics realized there was all this computing power that you
0:04:21 can hack the algorithm to use that computing power to model thermodynamics of fluids inside
0:04:23 clouds, which was a thesis.
0:04:27 And he realized that worked better than the CPU than just sort of doing it the traditional
0:04:29 way through the computer.
0:04:35 And the GPU has all these different cores that run in parallel, and the CPU only typically
0:04:39 has four to eight cores, and GPUs have hundreds of thousands of cores.
0:04:42 What’s a CPU better at than a GPU?
0:04:46 Like why would you have four to eight when you could have thousands?
0:04:50 Well, it could run more complicated pieces of instructions and software.
0:04:54 And GPUs typically break down into much simpler tasks.
0:04:57 But the difference is you could run all those tasks across a thousand.
0:05:06 So for certain workloads like AI, it’s so much faster than a CPU, which has to do things
0:05:08 kind of one after another, seriously.
0:05:14 So Mark Harris and all these academics started hacking into the graphics algorithms and using
0:05:19 that power to do all this scientific high performance computing.
0:05:24 And Mark Harris started a website and congregated all these researchers and everyone sharing
0:05:28 the knowledge, sharing the tricks to actually hack into this.
0:05:33 So NVIDIA sees this, they’re seeing people use this to model stock options, to model
0:05:36 the weather, and saying, “Wow, this is actually really interesting.”
0:05:39 Yeah, that’s like your dream if you’re a company, right?
0:05:43 It’s like, “Oh, there’s all these other things you could do with this thing we’re already
0:05:44 making.”
0:05:45 Exactly.
0:05:49 And Jensen kind of had the foresight to see this, “Wow, this is a really big deal.
0:05:50 We should invest in this.”
0:05:56 So they wind up hiring Mark Harris and look at all the things that all these researchers
0:06:01 are doing, and this is the beginning of the development of CUDA, which is a programming
0:06:05 platform for general purpose GPU computing.
0:06:09 And just to be clear, CUDA is like, it’s a programming language, but in this case, they’re
0:06:13 programming what used to be just the thing for graphics, but it turns out to be good
0:06:14 for these other things.
0:06:18 And that language, it’s NVIDIA’s own language, right?
0:06:22 And this is, I feel like, going to be important in the business story for why NVIDIA is so
0:06:24 dominant today, right?
0:06:29 They write this language, they own the language, and it’s written just to work with their chips.
0:06:30 Is that right?
0:06:32 Yes, no one else can use this.
0:06:37 It’s extensions on a programming language that makes parallel computing easier for programmers
0:06:39 to make.
0:06:46 So Jensen just invested in this, and he actually thinks longer term than the average CEO.
0:06:51 So most CEOs are looking at maybe the next quarter, maybe a year or two, Jensen thinks
0:06:54 in five, 10, 15 year increments.
0:06:57 So he’s always looking out, what’s the next big computing shift?
0:07:00 What’s the next big technology phase?
0:07:05 So he saw CUDA as the main thing where eventually his GPUs, you have all this latent enormous
0:07:12 computing power, will be able to be used for science, research, all these other simulation
0:07:14 things.
0:07:15 And he didn’t give up.
0:07:19 Even when Wall Street went down on its next day, why are you wasting all this dye space?
0:07:21 It’s crushing your gross profit margins.
0:07:27 He said, no, this is the future computing, I’m going to invest in it, and it will work
0:07:28 out someday.
0:07:32 And when you say wasting all this dye space, dye space is like space on the chip, right?
0:07:38 He’s making a physical thing, and to commit to CUDA to have this dedicated programming
0:07:43 language, you actually have to give up space on the chip, so it’s costly, right?
0:07:48 Where margins go down, they’re not optimizing for profits at that moment.
0:07:50 It’s literal circuits called CUDA cores.
0:07:54 Their hardware circuits are optimized to run the CUDA language.
0:08:00 So even internally, executives are like, why are we spending all this money allocating
0:08:05 dye space for something that people really aren’t using that much, that isn’t generating
0:08:06 revenue?
0:08:11 But Jensen saw this as a future, I actually kind of bring up the analogy of Reed Hastings
0:08:12 and Netflix.
0:08:16 So when he started Netflix, the technology wasn’t ready, consumers didn’t really have
0:08:22 broadband, but he had this intuitive sense that someday video will be streamed over the
0:08:27 internet, and that was the future, and it’s so obvious now, but back then it wasn’t.
0:08:34 So Hastings positioned Netflix, he made money with DVD rentals for a while, and just stayed
0:08:37 on top of the technology, kept on investing and investing, and when the technology was
0:08:44 good enough, that’s when he really pivoted Netflix to dominate internet streaming.
0:08:49 Jensen did this multiple times with 3D graphics, video game graphics.
0:08:56 He knew that someday PC video games would be a big market, programmable GPUs, CUDA,
0:09:00 and later with AI on these full-stack data center AI servers.
0:09:05 He just sees the future and is willing to keep investing, even if it’s five, 10 years
0:09:06 out.
0:09:08 It started in 2006.
0:09:12 I mean, things got incrementally better for the next 10 years, but it didn’t really take
0:09:15 off till 2022.
0:09:19 Yeah, till chatGPT basically, 2022 is to chatGPT.
0:09:21 That’s when Nvidia goes totally bonkers.
0:09:26 And that was the power of the large language model of AI and all that stuff, and he’s investing
0:09:29 throughout this whole thing for 10, 15 years.
0:09:31 So there is a moment, there’s a moment in the middle there.
0:09:35 I do obviously want to get to the chatGPT moment and the present, but there is one more
0:09:41 moment I feel like in the middle where Nvidia is in the center of it, and that’s 2012.
0:09:47 So you have early 2001, people realize, “Oh, you can hack the GPU to do other things.”
0:09:48 And everybody’s like, “Oh, that’s interesting.
0:09:51 Let’s build the whole programming language so people can do that.”
0:10:01 And then in 2012, you have this moment that really seems like the birth of modern AI.
0:10:08 This moment when this AI model called AlexNet sort of emerges into the world, and everybody
0:10:14 in the AI world is like, “Oh my God, AI is here.”
0:10:16 Tell me about that moment.
0:10:23 So AlexNet was a program that was created by two researchers at the University of Toronto,
0:10:28 and they competed in this competition called ImageNet, which basically fed images into
0:10:34 the model, and they were able to recognize and categorize the image much more effectively
0:10:36 than any other model in the past.
0:10:39 And the breakthrough was they used GPUs for the first time.
0:10:41 And it was just a few, right?
0:10:46 They were grad students, and they got their hands on a few Nvidia GPUs and had them running
0:10:49 on servers in the hallway or something.
0:10:52 And these are video game GPUs to be clear.
0:10:54 This isn’t some enterprise complicated GPU.
0:10:59 They literally went to a store and bought a bunch of video game GPUs, and it turned out
0:11:06 to be very effective where they could do the power and effectiveness of 2,000 CPUs on only
0:11:08 12 GPUs.
0:11:12 And to be clear, that kind of vision AI, it’s machine learning.
0:11:18 It’s the same basic technology as is used in large language models, right?
0:11:19 It’s modern AI.
0:11:28 And so that moment is this moment when it’s like, “Oh, holy shit, GPUs are 100X better
0:11:30 than CPUs for AI.”
0:11:31 Yes.
0:11:32 That’s interesting.
0:11:33 That’s useful to know.
0:11:40 So it was a combination of GPUs, data, and algorithms that kind of combusted at that perfect
0:11:41 moment.
0:11:46 And then Jensen saw this, and he’s like, “This is a big deal, and someday AI is going to
0:11:49 be huge, and we need to invest big in this.”
0:11:56 So on top of CUDA, he invested in all these AI libraries that effectively managed the GPUs
0:12:00 to do AI workloads in the most effective manner.
0:12:05 And he invested in libraries, invested in software, he invested in researchers, and
0:12:11 allocated a lot of employees to this project, starting in around 2013.
0:12:16 And so that’s more of the like, it’s not just the chip, right?
0:12:23 It’s like the chip that is optimized to be not only efficient on the hardware level,
0:12:24 but there’s all this software.
0:12:31 So if you are building AI, if you’re writing code for AI, they make it really easy for
0:12:33 you to do it on NVIDIA GPUs.
0:12:40 So it’s all the math libraries, it’s all the AI frameworks work the best on NVIDIA GPUs.
0:12:44 And they added these things called tensor cores similar to CUDA cores.
0:12:51 They actually added hardware circuitry inside the GPUs that are optimized to run AI training
0:12:54 and workloads and all the software that you’re running.
0:12:59 And this is over again, this is over another nine, eight years where they’re building
0:13:05 all the software and hardware circuitry to run deep learning AI the best.
0:13:07 Okay.
0:13:08 It’s time.
0:13:15 2022, 2022, chat GPT comes out.
0:13:17 What’s that mean for NVIDIA?
0:13:24 So actually, chat GPT takes off and it takes about two quarters for the whole world to
0:13:26 realize this is a big deal.
0:13:32 This model of natural language processing where the computer actually understands what
0:13:40 you’re asking and that ability to draw insights and be effective with all that natural language
0:13:47 stuff just blows people away and companies and startups realize we have a new AI boom
0:13:56 here because it really unleashes a wave of capabilities that wasn’t doable before.
0:14:01 So about six months later, that’s when the big bang I call it for NVIDIA happens when
0:14:08 they say, “Oh my gosh, we’re going to beat our numbers that the street expects by $4
0:14:10 billion.”
0:14:14 And the stock went up like $170 billion in value because the world realized.
0:14:15 In like a day?
0:14:16 Yes, a day.
0:14:22 There was one day and it was spring of 2023, this is what you’re talking about.
0:14:28 And this is three weeks after I had the meeting with a book publisher who asked me if I could
0:14:29 do a book on NVIDIA.
0:14:30 But literally, I started.
0:14:35 Also, good timing for you, good timing for NVIDIA, good timing for you.
0:14:41 And right, no, I remember that day and there was a question then of like, “Oh my God, is
0:14:42 this a one-off?
0:14:44 Are they going to keep growing this way?”
0:14:50 Basically, what they said was, “We sold way, way more GPUs than we thought we were going
0:14:51 to.”
0:14:52 Right?
0:14:54 That’s the basic thing they reported in their earnings report.
0:14:59 And the subtext is, this is the world and in particular like big tech companies with
0:15:06 lots of money realizing, “Oh, we’ve got to get into this AI game more than we have been.”
0:15:11 And to do that, we got to buy a lot of really expensive GPUs from NVIDIA.
0:15:17 And Jensen is really smart at selling this stuff.
0:15:23 He’s very smart because every company, every company’s CEO, every startup, they saw this
0:15:25 risk as existential for them.
0:15:32 Because if your competitor comes out with the AI-featured offering that’s much better
0:15:39 than what you have today that isn’t advantaged by the AI models, that competitor could drive
0:15:40 you out of business.
0:15:46 So Jensen was very smart at selling to people that you need to get on board.
0:15:49 At making everybody scared of losing.
0:15:50 Yes.
0:15:57 And in particular, there is a small number of very large, very rich tech companies that
0:16:01 are a very big part of NVIDIA’s revenue.
0:16:09 It’s like, whatever, the ones you could think of, Google, Netta, Microsoft, Amazon, I guess
0:16:13 open AI is connected with.
0:16:17 Those companies are buying a huge share of these GPUs.
0:16:21 They account for a big, big chunk of NVIDIA’s revenue.
0:16:24 Those are good companies to have as customers because they’re super rich, right?
0:16:27 They can afford to pay you tens of billions of dollars for your GPUs.
0:16:33 But that being said, they’re reselling that GPU power to companies and startups.
0:16:39 So startups are buying that GPU computing capacity from these tech companies.
0:16:40 That’s a good point.
0:16:44 So they’re in a way not the end customer, they’re the sort of intermediate kind of service
0:16:45 providers.
0:16:49 They also use that for their own internal systems too.
0:16:55 Meta uses a ton of GPU power to make their advertising algorithms more effective and to
0:16:58 pick the videos like TikTok does.
0:17:03 And obviously Google search is now becoming more and more AI driven and there’s Gemini,
0:17:04 which is right.
0:17:10 I mean, there’s more direct use of AI by all of the big tech companies as well.
0:17:14 So there’s another piece of this, which is interesting, I mean, because there’s one
0:17:19 universe where it’s like, oh, we got to get in on AI, and they all buy the GPUs.
0:17:20 And then that’s kind of it.
0:17:24 It’s like a step function where like there’s this momentary rush where everybody buys the
0:17:25 GPUs and then whatever.
0:17:31 They just upgrade every couple of years and it’s more like a regular tech hardware business,
0:17:33 which is like good, but not amazing.
0:17:37 But there’s another piece of it that has been a huge deal for NVIDIA and that’s the scaling
0:17:40 law or the scaling hypothesis.
0:17:41 Let’s talk about that.
0:17:42 What’s the scaling hypothesis?
0:17:47 The scaling hypothesis is similar to what I said before is the combination of computing
0:17:53 power, the more computing power you add, the more data you increase, and the better ways
0:17:56 you figure out software algorithms.
0:18:01 For each of these three buckets, if you increase it, the AI model becomes more capable and
0:18:02 more effective.
0:18:07 And like just to be clear, like even if the algorithms aren’t getting that much better,
0:18:08 right?
0:18:10 Like my understanding is, and this is kind of a new idea.
0:18:17 But like, oh, if we can just have more data and more computing power, we’ll get better
0:18:18 results.
0:18:19 Yes.
0:18:22 So it’s great if you have all three doing, you know, basically going up.
0:18:29 But for this time period, the last few years, the scaling law has really taken off.
0:18:39 And companies are literally 10xing their compute power and these AI clusters going from 16,000
0:18:41 GPUs now to 100,000 GPUs.
0:18:46 And now people are saying they’re going to build one million GPU clusters in the next
0:18:47 two, three years.
0:18:51 And NVIDIA is selling most of those GPUs.
0:18:52 Yes.
0:18:56 So if you go from 16,000 to 100,000 GPUs in a couple of years, that’s a good business
0:18:57 to be in.
0:19:03 If you’re increasing the scale of the hardware by 10x every couple of years.
0:19:08 And the exciting thing for NVIDIA is that, like what I argue is that this thing is going
0:19:13 to, this trend is going to continue in the next few years because you have the scaling
0:19:14 laws.
0:19:18 Then you have this thing called multimodal where they’re using this GPUs, not just for
0:19:25 text like in chat tpt, you’re using it for video and images, generating those things.
0:19:30 And now there’s these two other things, two other waves of demand that are happening.
0:19:36 There’s this thing called AI agents, where these AI models can do multi-step tasks for
0:19:37 you.
0:19:38 Right.
0:19:43 You could be like, book me a ticket to San Diego some weekend in June, whenever it’s the
0:19:44 best deal.
0:19:45 Yes.
0:19:46 And then it does that.
0:19:49 So that’s going to happen in the next 12 months, yes.
0:19:54 These AI agents will eventually in the next year or two be able to do all that tedious
0:20:00 work automatically and probably with less errors than an actual human being.
0:20:05 So AI agents, multimodal, and now there’s this thing called test time compute, where the
0:20:11 AI models instead of just spinning back an instant reply can actually think about what’s
0:20:15 the best way to respond to your question and spend more time thinking about it and then
0:20:18 give you a higher quality answer.
0:20:25 So there’s all these things that all of these test time compute, AI agents, multimodal,
0:20:30 these are all things that need more computing power, they need more GPUs and these are all
0:20:36 things that are going to drive NVIDIA’s revenue in the next couple of years.
0:20:41 We’ll be back in a minute and we will talk, among other things, about Jensen Wong, one
0:20:47 of the most successful entrepreneurs of the 21st century and a man who once told a colleague
0:20:53 that he wakes up every morning, looks in the mirror and says to himself, “You suck.”
0:21:03 Hey it’s Jacob, I’m here with Rachel Botsman, Rachel Lectures on Trust at Oxford University
0:21:09 and she is the author of a new Pushkin audiobook called How to Trust and Be Trusted.
0:21:10 Hi Rachel.
0:21:11 Hi Jacob.
0:21:17 Rachel Botsman, tell me three things I need to know about trust.
0:21:21 Number one, do not mistake confidence for competence.
0:21:22 Big trust mistakes.
0:21:28 So when people are making trust decisions, they often look for confidence versus competence.
0:21:35 Number two, transparency doesn’t equal more trust, big myth and misconception and a real
0:21:37 problem actually in the tech world.
0:21:43 The reason why is because trust is a confident relationship with the unknown.
0:21:47 So what are you doing if you make things more transparent?
0:21:49 You’re reducing the need for trust.
0:21:55 And number three, become a stellar expectation setter.
0:22:00 Inconsistency with expectations really damages trust.
0:22:00 I love it.
0:22:04 Say the name of the book again and why everybody should listen to it.
0:22:07 So it’s called How to Trust and Be Trusted.
0:22:13 Intentionally it’s a two-way title because we have to give trust and we have to earn trust
0:22:19 and the reason why I wrote it is because we often hear about how trust is in a state of crisis
0:22:21 or how it’s in a state of decline.
0:22:26 But there’s lots of things that you can do to improve trust in your own lives,
0:22:30 to improve trust in your teams, trusting yourself to take more risks
0:22:33 or even making smarter trust decisions.
0:22:37 Rachel Botsman, the new audiobook is called How to Trust and Be Trusted.
0:22:38 Great to talk with you.
0:22:44 It’s so good to talk with you and I really hope listeners listen to it because it can change people’s lives.
0:22:48 I want to talk about Jensen a little bit.
0:22:53 I mean, your book, he’s obviously the main character in the history of NVIDIA.
0:22:54 He’s the main character in your book.
0:22:58 He’s been the CEO of NVIDIA for more than 30 years,
0:23:01 which is like longer than Bill Gates with the CEO of Microsoft,
0:23:06 longer than almost anybody in the S&P 500 has been a CEO at this point.
0:23:09 His childhood is quite interesting, right?
0:23:11 Like, tell me just a little bit about his childhood.
0:23:17 He was born in Taiwan and they moved around a little bit to Thailand.
0:23:23 And his father came to training in New York and fell in love with America.
0:23:26 This is like the great American dream story.
0:23:34 And the mother and father started teaching Jensen and his brother English 10 words a day,
0:23:38 and they sent him to his aunt and uncle when he was about age eight or nine.
0:23:43 And the aunt and uncle and the families, the funny thing is,
0:23:46 they sent him to a reform school in Kentucky by mistake,
0:23:51 thinking that he would get a great education at this boarding school in Kentucky.
0:23:53 But they just thought it was a boarding school.
0:23:56 Yes, but it turned out to be a school for troubled kids.
0:23:59 Which Jensen was not.
0:24:03 He was just like a smart kid with ambitious for him parents.
0:24:03 Yes.
0:24:09 But he talks about how his time at Oneida Baptist Institute in Kentucky was formational for him.
0:24:10 What was it like for him?
0:24:14 It was hard at the beginning, but he started befriending people.
0:24:19 He started playing chess with the janitor.
0:24:23 And he just learned how to deal with other people much better.
0:24:27 And he talks about how he learned his street fighter mentality.
0:24:28 Because he was literally fighting.
0:24:32 Yes, I mean, there were bullies and all that stuff,
0:24:38 but he just learned how to deal with people and the rough and tumble of kids back then.
0:24:44 Eventually, his parents came from abroad and they settled in Oregon.
0:24:49 And he learned his work ethic from working at Denny’s.
0:24:57 He talks about how he washed more dishes and cleaned more bathrooms than any CEO in the history of CEOs.
0:25:04 He says Denny’s helped them in terms of social skills and dealing with time pressure and dealing with customers.
0:25:09 But it’s the work ethic that really sets him apart.
0:25:14 So even at the beginning, he was working from 9am to midnight.
0:25:19 And he just set a culture where NVIDIA employees work really hard.
0:25:22 And I feel like at the beginning, that’s common, right?
0:25:25 Like that is the classic start-up story.
0:25:27 But the classic start-up story is you do that for five years.
0:25:33 And then either you get giant and you hire a grown-up CEO and you, you know,
0:25:37 go start your blimp company or whatever, or you sell or whatever.
0:25:39 He’s still doing that, right?
0:25:45 He’s 60 years old and like wildly rich and he’s still working that much.
0:25:46 He’s working all weekend.
0:25:48 He’s working Saturday, Sunday.
0:25:54 He talks about when he goes to a movie, he doesn’t remember the movie because he’s thinking about work.
0:25:57 He finds work relaxing and fun.
0:25:58 This is what drives him.
0:26:00 He loves working.
0:26:03 There’s a moment, you know, you don’t, you don’t write about his personal life at all
0:26:09 once he sort of grows up and starts NVIDIA, you know, reasonably because the book’s about NVIDIA.
0:26:13 And so I just assumed he just worked all the time and didn’t have a family.
0:26:18 And I think the only time in the book that his family comes up is there’s this scene
0:26:23 where he’s on vacation and he’s talking to some senior manager at NVIDIA on the phone
0:26:24 and they’re like, what are you doing?
0:26:29 And he’s like, I’m sitting here on the balcony watching my kids play in the sand and writing emails.
0:26:33 And it’s like, like, if it’s a movie, the kids are never on stage, right?
0:26:35 You just hear them offstage at that moment.
0:26:37 You’re like, oh, he has kids.
0:26:42 Like that was, that was a weird moment to me reading that, that line.
0:26:48 So both executives told me they hate it when Jensen so-called goes on vacation
0:26:53 because he winds up giving them more directives and orders
0:26:57 and more stuff to do when he’s on vacation because he’s emailing them, do this, do that.
0:27:00 And they, they actually yell at him.
0:27:02 So you play with your kids.
0:27:07 And he’s like, no, I could get real work done when I’m on vacation by doing his emails.
0:27:08 I mean-
0:27:11 So it shows you his obsession.
0:27:15 He’s constantly thinking, he’s constantly worried about what’s happening at NVIDIA.
0:27:17 I mean, it’s, it’s interesting, right?
0:27:25 Like, he’s not a balanced person, like, which is why, to some extent,
0:27:28 he has built the thing that he has built, right?
0:27:34 He’s completely obsessed with winning and he’s extremely competitive.
0:27:35 Yeah.
0:27:37 There’s a few other specific moments that really stood out to me.
0:27:43 There’s one where I think this, I think a salesperson told you this,
0:27:46 where they had just had a great quarter.
0:27:52 They’d sold a ton of GPUs and the sales guys talking to Jensen about how great they’re doing.
0:27:56 And Jensen says to the sales guy about Jensen, about himself.
0:28:01 Jensen says, I look in the mirror every morning and say, you suck.
0:28:05 So he, he’s almost like a self psychologist.
0:28:10 He knows if you start thinking that you’re, you’re the best and you’re, you’re hot stuff,
0:28:11 you might get complacent.
0:28:13 You start resting on your laurels.
0:28:14 You might not work as hard enough.
0:28:17 So he, he sees that, oh, we just had a blowout quarter.
0:28:19 What’s the risk here?
0:28:21 The risk is me getting complacent.
0:28:25 So I’m going to look myself in the mirror and say you suck and psych myself out.
0:28:29 I mean, that’s, that’s, that’s one reading of it.
0:28:31 That’s the like 10 dimensional chess reading of it.
0:28:33 I mean, there’s another reading of it, which is like,
0:28:38 he’s messed up in a way that makes him super driven and successful.
0:28:40 But it works.
0:28:41 He actually definitely works.
0:28:43 We can agree that it works.
0:28:45 But he does the opposite too.
0:28:50 So when things are really intimidating and he feels like, oh my gosh, how am I going to do this?
0:28:52 He tells himself, how hard can it be?
0:28:54 How hard can it be Jensen?
0:28:58 So he does it on both, both ends of this spectrum where he,
0:29:01 he cites himself out when he feels intimidated.
0:29:04 And when he feels like he’s on top of the world,
0:29:06 he tries to bring himself down back to the earth.
0:29:12 So the one other big piece of, of the Jensen Wong experience that we haven’t talked about
0:29:15 is the way he treats the people who work for him, right?
0:29:19 I want to talk about a particular scene,
0:29:22 because I think it, it puts a finer point on a scene from the book where
0:29:26 it’s like a company-wide meeting and it’s on like Zoom or whatever.
0:29:28 And he’s yelling at a guy in the meeting.
0:29:29 He’s yelling at him.
0:29:31 And then on top of that,
0:29:36 he keeps telling the person filming the meeting to zoom in on the guy he’s yelling at.
0:29:41 I had multiple sources tell me that it was the most humiliating thing they’ve ever seen.
0:29:43 I mean, it’s, that seems like bullying.
0:29:46 Like why zoom in on the guy?
0:29:48 Like what lesson is added by that?
0:29:52 So he kept on saying to Mr. Rayfield,
0:29:55 you got to get this chip back on track.
0:29:58 The chip that he was in charge of was behind schedule.
0:30:03 And he kept on pointing to him, zooming in his face and telling him,
0:30:04 you’ve got to get this.
0:30:05 This is not how you run a business.
0:30:08 You need to get this chip back on track.
0:30:11 And this kind of like high standard demanding attitude,
0:30:14 I think is effective.
0:30:15 It drives people.
0:30:18 If you get dressed down by Jensen,
0:30:23 the next time you’re going to work 10 times more to make sure that you do a better job.
0:30:27 Maybe I don’t want the world to be that way, but it is that way.
0:30:28 You know what I mean?
0:30:30 Like maybe I want to believe that like,
0:30:34 there’s a kinder world where people could do equally good work.
0:30:35 But maybe I’m wrong.
0:30:39 And Steve Jobs was the same way.
0:30:39 Yeah.
0:30:42 It wasn’t all sunshine and rainbows.
0:30:43 No, no.
0:30:46 I mean, the Isaacson book was really clear, right?
0:30:48 The Isaacson biography of Steve Jobs.
0:30:51 He did not seem like a good person in that book.
0:30:53 He seemed like really good at building amazing products,
0:30:59 but like not like a good human being by the sort of standard way we think of what makes a good person.
0:31:04 But I think the key point that we also have to think about is people at NVIDIA stay.
0:31:05 Yeah.
0:31:06 That’s really interesting.
0:31:11 The turnover is like one of the lowest in the industry at only 3% compared to the average of 13 to 15%.
0:31:18 I mean, certainly in the last few years, people stay because you get rich if you stay and you lose your equity if you leave.
0:31:21 But is that number true if you go back to when the stock was flat?
0:31:25 I don’t have the numbers, but a lot of the senior executives have been there
0:31:29 more so than any other company, 15, 20, 25 years.
0:31:30 Yeah, that’s interesting.
0:31:36 It’s you don’t really hear other than one instance of people leaving to become a CEO of a different company.
0:31:45 And I think part of it is people realize that NVIDIA or the Jensen way of doing things is super effective.
0:31:47 So people want to be on the winning team.
0:31:51 So in one sense, he drives people hard.
0:31:52 He dresses them down.
0:31:56 He’s very blunt and direct, but he compensates people.
0:31:58 No one ever hardly anyone can play.
0:31:59 It’s about compensation.
0:32:06 If you’re effective and you do a good job, he will like double your stock compensation on the spot.
0:32:10 So there’s this meritocracy that people really adore.
0:32:17 And if you see an effective leadership, a culture that’s based on meritocracy,
0:32:20 if you’re a top engineer, you stay at that company.
0:32:21 Yeah, yeah.
0:32:26 So you interviewed Jensen as you as you were working on the book.
0:32:28 Was it scary to interview him?
0:32:31 He was intimidating in the meeting.
0:32:33 Did he yell at you?
0:32:38 He didn’t yell at me, but a couple of times he was saying you don’t get NVIDIA in this line of questioning.
0:32:46 And it takes a while to get used to it, but you appreciate it because you see where the other person is.
0:32:50 Someone is blunt and direct like Jensen.
0:32:55 You know where he is at all times and then you can work at improving yourself.
0:32:59 So he talks about what are you optimizing for?
0:33:05 Are you optimizing for a person’s feelings or optimizing for what’s good for the company?
0:33:08 So that’s why he doesn’t do one-on-one meetings or career coaching.
0:33:14 Typically, at a large company, when someone’s doing poorly, the CEO will take them aside and say,
0:33:16 “Bob, you need to do this. You need to do that.”
0:33:17 Like if it’s a senior person.
0:33:23 Jensen’s like, “Why is Bob the only one that gets a learning here?”
0:33:29 If I am blunt and direct and show Bob what he’s doing incorrectly like that person at that meeting,
0:33:33 everyone in the room can learn.
0:33:36 All the employees up and down the ladder can learn.
0:33:40 So that’s his philosophy is everyone should learn from their mistakes
0:33:42 and everyone should know where they are at all times.
0:33:51 So, I mean, so you write in the book that NVIDIA really is like an extension of Jensen, right?
0:33:57 I think he used the metaphor of like the Formula One car that’s like optimized for him as the driver.
0:34:09 So, and he’s in his 60s, which is not old old, but definitely not young.
0:34:13 Like is he going to run NVIDIA for another 20 years?
0:34:14 What’s going to happen?
0:34:16 Like there’s no obvious successor.
0:34:17 Like where does that go?
0:34:27 I think there’s no one else at NVIDIA, I think, that can run NVIDIA as effectively as Jensen.
0:34:29 And he loves the company so much.
0:34:30 He loves what he’s doing.
0:34:33 They’re having an enormous impact with this AI wave.
0:34:38 And he’s so excited about the potential for curing cancer and digital biology,
0:34:46 the potential for robots, the potential for AI to kind of disrupt education and help kids learn better.
0:34:49 I don’t see him leaving anytime soon.
0:34:54 He loves his job so much and he can’t argue that he’s being ineffective.
0:35:01 So, I don’t think for the next few years there’s anyone that’s going to take over for him.
0:35:05 Someday, NVIDIA is going to have to come up with a new CEO or a successor to Jensen.
0:35:08 And that’s going to be a big question mark.
0:35:15 Is that next person going to be as effective as Jensen in terms of being able to have the technical skill
0:35:19 and the competency to steer NVIDIA in the right direction?
0:35:25 Will they have his business genius of coming up with all these new strategies
0:35:27 that he does time and time again?
0:35:29 That’s going to be a huge question mark.
0:35:39 I mean, people also talk about limits to scaling in AI.
0:35:45 We were talking earlier about how part of NVIDIA’s wild growth over the last couple of years
0:35:54 has come from this fact that you can just add more GPUs, basically, and get better results,
0:35:55 more GPUs, more data.
0:36:05 People talk about, A, running out of data, because they’re basically, as I understand it,
0:36:09 training on the whole internet right now, which is like, okay, it’s a lot of data.
0:36:14 Is that, it seems like nobody knows, right?
0:36:17 It seems like there are smart people who say, no, no, we’ll have synthetic data,
0:36:19 blah, blah, blah, we won’t hit a sort of scaling wall.
0:36:21 And then there are people who make the other argument.
0:36:22 I certainly don’t know.
0:36:23 But does that seem right?
0:36:25 Like, is that an open question right now?
0:36:27 And is that a meaningful question for NVIDIA?
0:36:29 I think it’s an open question.
0:36:35 But like I said, there’s the multimodal stuff, there’s AI agents,
0:36:38 and then there’s proprietary data inside companies.
0:36:44 So all these corporations have data going back decades.
0:36:51 Companies are going to use the power of these AI computing systems to go through all their data
0:36:57 internally, all their proprietary data, and have all that knowledge at employees’ fingertips.
0:37:03 So an employee can ask, what’s the best way to do this?
0:37:08 And the AI computer is going to be able to go back with 30 years of data,
0:37:11 and figure out the best piece of insight to help that employee.
0:37:13 And that’s not really being done today.
0:37:19 So NVIDIA designs their chips, their GPUs, but they don’t actually manufacture them, right?
0:37:24 Is it right that they’re made in Taiwan, where most cutting-edge chips are made now?
0:37:30 Yes. So TSMC actually makes and manufactures NVIDIA’s chips, ever since the late 1990s.
0:37:32 And they’re the best at doing this.
0:37:40 A lot of the fabless chip designers in California use TSMC, and NVIDIA uses them too.
0:37:49 And so, I mean, it is really interesting to me, to a lot of people, that Taiwan is this
0:37:55 incredibly fraught geopolitical place, right? China says Taiwan is part of China,
0:38:01 Taiwan says, no, we are not part of China, and that Taiwan is the only place in the world that
0:38:06 makes the most important physical thing in the world of technology today, right?
0:38:12 That’s wild. What do you make of that? And how does that fit with the NVIDIA story?
0:38:18 I think, eventually, NVIDIA chips will be made in the US, TSMC. I mean, that’s
0:38:22 all part of this CHIPS Act that the Biden administration has passed.
0:38:29 So this plant that TSMC is building in Arizona, is that advanced enough to make
0:38:35 like frontier NVIDIA chips? So the TSMC factories in the US
0:38:39 are always going to be one or two generations behind the factories in Taiwan.
0:38:46 But that doesn’t mean NVIDIA can’t use the factories in the US if they’re one or two
0:38:51 generations behind further older chips. So I think that will eventually happen.
0:38:57 It seems wild to me. I mean, it seems very possible that China will try and
0:39:02 make Taiwan be part of China, right? And we have all these export controls to try and
0:39:07 keep China from getting cutting-edge chips. It’s a really interesting complicated dynamic.
0:39:15 I think people make a big issue out of this, but I think if it happens, NVIDIA chips are not going
0:39:24 to be the main issue. It’ll cause a global calamity where we won’t be able to upgrade our cars,
0:39:32 laptops, nearly every computing device, every appliance will not work if we don’t have access
0:39:39 to Taiwan chips. I mean, presumably we could do some like sub-Manhattan project, scale Manhattan
0:39:47 project to get a state-of-the-art TSMC factory somewhere that is not Taiwan, right? Like one
0:39:50 would think. We’re doing some of that now, but it’s not going to be up to,
0:39:58 it’s going to be 10, 20% of the capacity we need. Yeah, I see the capacity, right? It takes a long
0:40:04 time. The fabs are wildly complicated to build and they cost billions of dollars and they take
0:40:08 years. So you couldn’t just do it. They cost 10 to 20 billion dollars to build in three to four years,
0:40:12 over three to four years. So it’s not something that can happen overnight.
0:40:17 And if something happens, it’s just going to be so terrible. It’ll be like a depression.
0:40:25 So hopefully it doesn’t happen. And the question isn’t going to be about Nvidia chips.
0:40:27 The question is going to be about the global economy of the app.
0:40:33 Why do you think it’s the biggest risk for Nvidia in the five-year timeframe?
0:40:39 It’s really tough to say now. I mean, I see the next few years with all the different AI
0:40:45 innovations and all the progress. But again, just like every big computing shift,
0:40:51 what’s the next big thing? Could it be quantum computing? It can be. Like who knows 5, 10,
0:40:58 15 years from now. And from the PCH with Microsoft and Intel, it’s called Wintel.
0:41:04 And then it went to smartphones where Apple dominated with the iPhone. And now,
0:41:10 Nvidia is dominating in terms of the AI computing shift. There’s going to be another shift.
0:41:14 And right now we’re at the early stages of the AI computing
0:41:22 movement. In five, 10 years, there might be the next big thing that I can’t even foresee right
0:41:29 now. And is Nvidia going to be able to see that coming? If Jensen’s around, I think he will.
0:41:36 But if Jensen’s not there, then just like every other major computer company in history,
0:41:41 say IBM, it’s very easy to get disrupted in the technology industry.
0:41:47 We’ll be back in a minute with the lightning round.
0:42:01 Hey, it’s Jacob. I’m here with Rachel Botsman. Rachel lectures on trust at Oxford University.
0:42:07 And she is the author of a new Pushkin audiobook called How to Trust and Be Trusted.
0:42:14 Hi, Rachel. Hi, Jacob. Rachel Botsman. Tell me three things I need to know about trust.
0:42:21 Number one, do not mistake confidence for competence. Big trust mistakes. So when people
0:42:28 are making trust decisions, they often look for confidence versus competence. Number two,
0:42:34 transparency doesn’t equal more trust. Big myth and misconception. And a real problem,
0:42:40 actually, in the tech world. The reason why is because trust is a confident relationship with
0:42:46 the unknown. So what are you doing if you make things more transparent? You’re reducing the need
0:42:55 for trust. And number three, become a stellar expectation setter. Inconsistency with expectations
0:43:02 really damages trust. I love it. Say the name of the book again and why everybody should listen to
0:43:08 it. So it’s called How to Trust and Be Trusted. Intentionally, it’s a two-way title because we
0:43:14 have to give trust and we have to earn trust. And the reason why I wrote it is because we often hear
0:43:20 about how trust is in a state of crisis or how it’s in a state of decline. But there’s lots of
0:43:26 things that you can do to improve trust in your own lives, to improve trust in your teams,
0:43:31 trust in yourself to take more risks, or even making smarter trust decisions.
0:43:36 Rachel Botsman, the new audiobook is called How to Trust and Be Trusted. Great to talk with you.
0:43:41 It’s so good to talk with you and I really hope listeners listen to it because it can change
0:43:49 people’s lives. I want to finish with the lightning round, which is going to be considerably more
0:43:56 random and digressive than the conversation to this point. I want to talk about video games
0:44:02 and your experience with video games. What was the first video game you ever loved?
0:44:10 I think it’s the first video I’ve ever tried, which was Combat for the Atari 2600.
0:44:17 I love that. I mean, I remember coming home and my dad buying the Atari 2600 connected to
0:44:22 a black and white TV and being able to use that joystick to control the little tank
0:44:28 going around the screen. That was an amazing moment. It was an incredible moment that this was
0:44:36 possible at the home. I had an Atari 2600. I remember the joystick. I remember Demon Attack.
0:44:43 It was a kind of second tier game, but I got really into it. What’s the game you spent the most hours on?
0:44:51 Probably this game called Lemmings. I don’t know. It’s a puzzle strategy game
0:44:58 where you control Lemmings and you guide them across a maze to get them to them.
0:45:02 Lemmings, the little creatures that are famed for jumping off cliffs, which maybe they don’t
0:45:08 actually do? Yes. If you guide them the wrong way, they’ll fall off a cliff and to their death.
0:45:14 They make this cute sound like, “Oh, no.” I played so many hours of that game.
0:45:20 A thousand hours? Probably hundreds of hours, I wouldn’t say a thousand.
0:45:25 What’s your most proud video game accomplishment?
0:45:34 I still remember playing the Legend of Zelda, the first one for the Nintendo Entertainment System,
0:45:37 and beating that game and just pumping my fists in the air.
0:45:46 I was very young then. What is the most exciting video game innovation
0:45:48 that’s going to happen in the next few years?
0:45:57 This is a little technical, but I think DLSS from the video is going to get even more powerful,
0:46:02 which is this upscaling technology that Jensen actually invented in a meeting
0:46:11 where they fill in the details using AI. The frame rates are able to go much faster and better.
0:46:17 So, it’s basically like interpolation. It’s sort of doing what AI always does for graphics.
0:46:20 It’s really effective. People can’t tell the difference between the real thing and the
0:46:28 interpolated AI graphics. As this technology gets better, that will allow the graphics to be
0:46:34 more photorealistic and the physics and the ray tracing better than ever before.
0:46:39 What game are you most excited to play in 2025?
0:46:44 I mean, Grand Theft Auto VI, if it comes out, but it might get delayed next year.
0:46:50 Okay, let’s do a few non-video game questions. What’s your favorite
0:46:59 tech book besides the one you wrote? I guess it’s still a tech book, The Innovator’s Dilemma,
0:47:05 which is actually one of Jensen’s favorite books. It talks about how companies get disrupted by
0:47:12 startups and people underneath them. It really goes into depth about how the disk drive industry,
0:47:16 every successive generation that disk drives, there was a new market leader,
0:47:22 because they couldn’t disrupt themselves to the new or smaller format.
0:47:27 I mean, to me, the really interesting insight of that book is the innovator
0:47:34 is actually making a crappier product. That’s the surprise. It’s not exactly like they come
0:47:39 along and do something better. They go to the crappy end of the market and they make a cheap
0:47:45 product. It’s not better than the thing the market leader is making. The market leader is like,
0:47:53 “Oh, that, who cares about that? That’s just some low margin thing.” The innovator comes up from
0:48:00 below. That to me is the really key insight of that book. They scale the volumes. Once you have
0:48:04 the volume, it pretty much becomes game over for the incumbent, because they can’t match
0:48:08 that scale and the economies of scale that comes with that.
0:48:15 Defend pineapple on pizza. It just tastes good. I love it.
0:48:20 I don’t know if you saw my tweets on it. I actually like pineapple on pizza.
0:48:23 Yes. I think it was Instagram. I think it was Instagram.
0:48:28 What’s the best deal you ever got at Costco?
0:48:32 It’s still the hot dog. I mean, you can’t beat the dollar fish.
0:48:47 Take Him is the author of The NVIDIA Way. Today’s show was produced by Gabriel Hunter Chang.
0:48:55 It was edited by Lydia Jean Cotte and engineered by Sarah Brugger. You can email us at problem@pushkin.fm.
0:49:02 I’m Jacob Goldstein and we’ll be back next week with another episode of What’s Your Problem?
0:49:05 What’s Your Problem?
0:49:14 Hey, it’s Jacob. I’m here with Rachel Botsman. Rachel lectures on trust at Oxford University
0:49:20 and she is the author of a new Pushkin audiobook called How to Trust and Be Trusted.
0:49:22 Hi, Rachel.
0:49:23 Hi, Jacob.
0:49:27 Rachel Botsman, tell me three things I need to know about trust.
0:49:34 Number one, do not mistake confidence for competence. Big trust mistakes. So when people
0:49:40 are making trust decisions, they often look for confidence versus competence.
0:49:46 Number two, transparency doesn’t equal more trust. Big myth and misconception.
0:49:52 And a real problem actually in the tech world. The reason why is because trust is a confident
0:49:57 relationship with the unknown. So what are you doing if you make things more transparent?
0:50:05 You’re reducing the need for trust. And number three, become a stellar expectation
0:50:11 setter. Inconsistency with expectations really damages trust.
0:50:16 I love it. Say the name of the book again and why everybody should listen to it.
0:50:21 So it’s called How to Trust and Be Trusted. Intentionally, it’s a two-way title because
0:50:27 we have to give trust and we have to earn trust. And the reason why I wrote it is because we often
0:50:33 hear about how trust is in a state of crisis or how it’s in a state of decline. But there’s lots
0:50:39 of things that you can do to improve trust in your own lives, to improve trust in your teams,
0:50:44 trusting yourself to take more risks, or even making smarter trust decisions.
0:50:49 Rachel Botsman, the new audiobook is called How to Trust and Be Trusted. Great to talk with you.
0:50:54 It’s so good to talk with you. And I really hope listeners listen to it because it can change
0:50:55 people’s lives.
0:00:11 Hey, it’s Jacob.
0:00:16 There is no shortage of tech podcasts, but few of them actually give you a dose of the
0:00:17 future.
0:00:20 The A16Z podcast is the exception.
0:00:25 The chart-topping show from Andreessen Horowitz brings you to the world’s most influential
0:00:31 people, movers who have a track record of being both early and right, like Apple co-founder
0:00:37 Steve Wozniak, or Andreessen Horowitz co-founders Mark Andreessen and Ben Horowitz.
0:00:41 Not to mention folks you don’t typically get to hear from, from the very first CTO of
0:00:48 the CIA to the chief security officers behind OpenAI, Anthropic, and DeepMind.
0:00:54 These guests tackle the most important trends in technology, from space to supply chains.
0:00:58 Want to understand the technology, policy, and economy of deepfakes?
0:01:00 They just broke this down.
0:01:05 Curious what really happened with the recent data breach of 3 billion records?
0:01:07 They covered that too.
0:01:11 Are autonomous cars, boats, and drones still miles away?
0:01:13 Listen to find out.
0:01:20 So go ahead, eavesdrop on the future with the A16Z podcast.
0:01:24 Just a quick note at the start of the show, we are very eager to hear what you think of
0:01:29 the show and how we can make it better, how we can make a show that you like more than
0:01:30 the show we’re making now.
0:01:34 Tell us what we should do more of, what we should do less of, and what we should do
0:01:35 wildly differently.
0:01:39 Email us at problem@pushkin.fm.
0:01:47 Again, problem@pushkin.fm, I’m looking forward to reading every email.
0:01:53 Artificial intelligence feels very abstract, feels very ephemeral, feels more like an
0:01:56 idea than a thing.
0:02:02 But there is, in fact, a thing there, AI is rooted in the physical world.
0:02:08 The thing or things are these little pieces of silicon and metal called graphics processing
0:02:10 units, GPUs.
0:02:15 These GPUs are expensive, they cost thousands of dollars each.
0:02:19 If you want to build a state of the art AI model, you have to buy tens of thousands of
0:02:27 these GPUs, and most of the GPUs come from a single company, NVIDIA, which is why in
0:02:33 just the past few years NVIDIA has become one of the most valuable, most important companies
0:02:40 in the world.
0:02:43 I’m Jacob Goldstein, and this is What’s Your Problem.
0:02:45 My guest today is Tay Kim.
0:02:49 He is a staff writer at Barron’s and the author of a new book about NVIDIA.
0:02:53 The book’s called The NVIDIA Way.
0:02:57 The book, of course, winds up being a lot about Jensen Wong, who co-founded the company
0:03:06 back in 1993, and who is still the CEO, and who is really a wild, kind of terrifying, brilliant
0:03:07 figure.
0:03:12 And Tay and I talk a lot about Jensen later in the interview, but we started with this
0:03:19 moment in 2001, that led from NVIDIA being a company that made graphics cards that let
0:03:26 people play video games on computers to becoming this key company at the center of the AI revolution
0:03:30 today.
0:03:37 So you write about this moment in 2001 when a researcher, an academic at the University
0:03:42 of North Carolina, realizes he can basically hack these graphics cards, this hardware that’s
0:03:48 just made to make pretty graphics on the computer, to help in his research, which is like not
0:03:50 at all about graphics, right?
0:03:53 It’s like modeling the weather or something.
0:04:00 And it seems like this moment is a big, winds up being a big turning point in the history
0:04:01 of technology, really.
0:04:02 Tell me about that moment.
0:04:07 The key moment was with Mark Harris, where he was a researcher at the University of North
0:04:08 Carolina.
0:04:14 And him and a bunch of other academics realized there was all this computing power that you
0:04:21 can hack the algorithm to use that computing power to model thermodynamics of fluids inside
0:04:23 clouds, which was a thesis.
0:04:27 And he realized that worked better than the CPU than just sort of doing it the traditional
0:04:29 way through the computer.
0:04:35 And the GPU has all these different cores that run in parallel, and the CPU only typically
0:04:39 has four to eight cores, and GPUs have hundreds of thousands of cores.
0:04:42 What’s a CPU better at than a GPU?
0:04:46 Like why would you have four to eight when you could have thousands?
0:04:50 Well, it could run more complicated pieces of instructions and software.
0:04:54 And GPUs typically break down into much simpler tasks.
0:04:57 But the difference is you could run all those tasks across a thousand.
0:05:06 So for certain workloads like AI, it’s so much faster than a CPU, which has to do things
0:05:08 kind of one after another, seriously.
0:05:14 So Mark Harris and all these academics started hacking into the graphics algorithms and using
0:05:19 that power to do all this scientific high performance computing.
0:05:24 And Mark Harris started a website and congregated all these researchers and everyone sharing
0:05:28 the knowledge, sharing the tricks to actually hack into this.
0:05:33 So NVIDIA sees this, they’re seeing people use this to model stock options, to model
0:05:36 the weather, and saying, “Wow, this is actually really interesting.”
0:05:39 Yeah, that’s like your dream if you’re a company, right?
0:05:43 It’s like, “Oh, there’s all these other things you could do with this thing we’re already
0:05:44 making.”
0:05:45 Exactly.
0:05:49 And Jensen kind of had the foresight to see this, “Wow, this is a really big deal.
0:05:50 We should invest in this.”
0:05:56 So they wind up hiring Mark Harris and look at all the things that all these researchers
0:06:01 are doing, and this is the beginning of the development of CUDA, which is a programming
0:06:05 platform for general purpose GPU computing.
0:06:09 And just to be clear, CUDA is like, it’s a programming language, but in this case, they’re
0:06:13 programming what used to be just the thing for graphics, but it turns out to be good
0:06:14 for these other things.
0:06:18 And that language, it’s NVIDIA’s own language, right?
0:06:22 And this is, I feel like, going to be important in the business story for why NVIDIA is so
0:06:24 dominant today, right?
0:06:29 They write this language, they own the language, and it’s written just to work with their chips.
0:06:30 Is that right?
0:06:32 Yes, no one else can use this.
0:06:37 It’s extensions on a programming language that makes parallel computing easier for programmers
0:06:39 to make.
0:06:46 So Jensen just invested in this, and he actually thinks longer term than the average CEO.
0:06:51 So most CEOs are looking at maybe the next quarter, maybe a year or two, Jensen thinks
0:06:54 in five, 10, 15 year increments.
0:06:57 So he’s always looking out, what’s the next big computing shift?
0:07:00 What’s the next big technology phase?
0:07:05 So he saw CUDA as the main thing where eventually his GPUs, you have all this latent enormous
0:07:12 computing power, will be able to be used for science, research, all these other simulation
0:07:14 things.
0:07:15 And he didn’t give up.
0:07:19 Even when Wall Street went down on its next day, why are you wasting all this dye space?
0:07:21 It’s crushing your gross profit margins.
0:07:27 He said, no, this is the future computing, I’m going to invest in it, and it will work
0:07:28 out someday.
0:07:32 And when you say wasting all this dye space, dye space is like space on the chip, right?
0:07:38 He’s making a physical thing, and to commit to CUDA to have this dedicated programming
0:07:43 language, you actually have to give up space on the chip, so it’s costly, right?
0:07:48 Where margins go down, they’re not optimizing for profits at that moment.
0:07:50 It’s literal circuits called CUDA cores.
0:07:54 Their hardware circuits are optimized to run the CUDA language.
0:08:00 So even internally, executives are like, why are we spending all this money allocating
0:08:05 dye space for something that people really aren’t using that much, that isn’t generating
0:08:06 revenue?
0:08:11 But Jensen saw this as a future, I actually kind of bring up the analogy of Reed Hastings
0:08:12 and Netflix.
0:08:16 So when he started Netflix, the technology wasn’t ready, consumers didn’t really have
0:08:22 broadband, but he had this intuitive sense that someday video will be streamed over the
0:08:27 internet, and that was the future, and it’s so obvious now, but back then it wasn’t.
0:08:34 So Hastings positioned Netflix, he made money with DVD rentals for a while, and just stayed
0:08:37 on top of the technology, kept on investing and investing, and when the technology was
0:08:44 good enough, that’s when he really pivoted Netflix to dominate internet streaming.
0:08:49 Jensen did this multiple times with 3D graphics, video game graphics.
0:08:56 He knew that someday PC video games would be a big market, programmable GPUs, CUDA,
0:09:00 and later with AI on these full-stack data center AI servers.
0:09:05 He just sees the future and is willing to keep investing, even if it’s five, 10 years
0:09:06 out.
0:09:08 It started in 2006.
0:09:12 I mean, things got incrementally better for the next 10 years, but it didn’t really take
0:09:15 off till 2022.
0:09:19 Yeah, till chatGPT basically, 2022 is to chatGPT.
0:09:21 That’s when Nvidia goes totally bonkers.
0:09:26 And that was the power of the large language model of AI and all that stuff, and he’s investing
0:09:29 throughout this whole thing for 10, 15 years.
0:09:31 So there is a moment, there’s a moment in the middle there.
0:09:35 I do obviously want to get to the chatGPT moment and the present, but there is one more
0:09:41 moment I feel like in the middle where Nvidia is in the center of it, and that’s 2012.
0:09:47 So you have early 2001, people realize, “Oh, you can hack the GPU to do other things.”
0:09:48 And everybody’s like, “Oh, that’s interesting.
0:09:51 Let’s build the whole programming language so people can do that.”
0:10:01 And then in 2012, you have this moment that really seems like the birth of modern AI.
0:10:08 This moment when this AI model called AlexNet sort of emerges into the world, and everybody
0:10:14 in the AI world is like, “Oh my God, AI is here.”
0:10:16 Tell me about that moment.
0:10:23 So AlexNet was a program that was created by two researchers at the University of Toronto,
0:10:28 and they competed in this competition called ImageNet, which basically fed images into
0:10:34 the model, and they were able to recognize and categorize the image much more effectively
0:10:36 than any other model in the past.
0:10:39 And the breakthrough was they used GPUs for the first time.
0:10:41 And it was just a few, right?
0:10:46 They were grad students, and they got their hands on a few Nvidia GPUs and had them running
0:10:49 on servers in the hallway or something.
0:10:52 And these are video game GPUs to be clear.
0:10:54 This isn’t some enterprise complicated GPU.
0:10:59 They literally went to a store and bought a bunch of video game GPUs, and it turned out
0:11:06 to be very effective where they could do the power and effectiveness of 2,000 CPUs on only
0:11:08 12 GPUs.
0:11:12 And to be clear, that kind of vision AI, it’s machine learning.
0:11:18 It’s the same basic technology as is used in large language models, right?
0:11:19 It’s modern AI.
0:11:28 And so that moment is this moment when it’s like, “Oh, holy shit, GPUs are 100X better
0:11:30 than CPUs for AI.”
0:11:31 Yes.
0:11:32 That’s interesting.
0:11:33 That’s useful to know.
0:11:40 So it was a combination of GPUs, data, and algorithms that kind of combusted at that perfect
0:11:41 moment.
0:11:46 And then Jensen saw this, and he’s like, “This is a big deal, and someday AI is going to
0:11:49 be huge, and we need to invest big in this.”
0:11:56 So on top of CUDA, he invested in all these AI libraries that effectively managed the GPUs
0:12:00 to do AI workloads in the most effective manner.
0:12:05 And he invested in libraries, invested in software, he invested in researchers, and
0:12:11 allocated a lot of employees to this project, starting in around 2013.
0:12:16 And so that’s more of the like, it’s not just the chip, right?
0:12:23 It’s like the chip that is optimized to be not only efficient on the hardware level,
0:12:24 but there’s all this software.
0:12:31 So if you are building AI, if you’re writing code for AI, they make it really easy for
0:12:33 you to do it on NVIDIA GPUs.
0:12:40 So it’s all the math libraries, it’s all the AI frameworks work the best on NVIDIA GPUs.
0:12:44 And they added these things called tensor cores similar to CUDA cores.
0:12:51 They actually added hardware circuitry inside the GPUs that are optimized to run AI training
0:12:54 and workloads and all the software that you’re running.
0:12:59 And this is over again, this is over another nine, eight years where they’re building
0:13:05 all the software and hardware circuitry to run deep learning AI the best.
0:13:07 Okay.
0:13:08 It’s time.
0:13:15 2022, 2022, chat GPT comes out.
0:13:17 What’s that mean for NVIDIA?
0:13:24 So actually, chat GPT takes off and it takes about two quarters for the whole world to
0:13:26 realize this is a big deal.
0:13:32 This model of natural language processing where the computer actually understands what
0:13:40 you’re asking and that ability to draw insights and be effective with all that natural language
0:13:47 stuff just blows people away and companies and startups realize we have a new AI boom
0:13:56 here because it really unleashes a wave of capabilities that wasn’t doable before.
0:14:01 So about six months later, that’s when the big bang I call it for NVIDIA happens when
0:14:08 they say, “Oh my gosh, we’re going to beat our numbers that the street expects by $4
0:14:10 billion.”
0:14:14 And the stock went up like $170 billion in value because the world realized.
0:14:15 In like a day?
0:14:16 Yes, a day.
0:14:22 There was one day and it was spring of 2023, this is what you’re talking about.
0:14:28 And this is three weeks after I had the meeting with a book publisher who asked me if I could
0:14:29 do a book on NVIDIA.
0:14:30 But literally, I started.
0:14:35 Also, good timing for you, good timing for NVIDIA, good timing for you.
0:14:41 And right, no, I remember that day and there was a question then of like, “Oh my God, is
0:14:42 this a one-off?
0:14:44 Are they going to keep growing this way?”
0:14:50 Basically, what they said was, “We sold way, way more GPUs than we thought we were going
0:14:51 to.”
0:14:52 Right?
0:14:54 That’s the basic thing they reported in their earnings report.
0:14:59 And the subtext is, this is the world and in particular like big tech companies with
0:15:06 lots of money realizing, “Oh, we’ve got to get into this AI game more than we have been.”
0:15:11 And to do that, we got to buy a lot of really expensive GPUs from NVIDIA.
0:15:17 And Jensen is really smart at selling this stuff.
0:15:23 He’s very smart because every company, every company’s CEO, every startup, they saw this
0:15:25 risk as existential for them.
0:15:32 Because if your competitor comes out with the AI-featured offering that’s much better
0:15:39 than what you have today that isn’t advantaged by the AI models, that competitor could drive
0:15:40 you out of business.
0:15:46 So Jensen was very smart at selling to people that you need to get on board.
0:15:49 At making everybody scared of losing.
0:15:50 Yes.
0:15:57 And in particular, there is a small number of very large, very rich tech companies that
0:16:01 are a very big part of NVIDIA’s revenue.
0:16:09 It’s like, whatever, the ones you could think of, Google, Netta, Microsoft, Amazon, I guess
0:16:13 open AI is connected with.
0:16:17 Those companies are buying a huge share of these GPUs.
0:16:21 They account for a big, big chunk of NVIDIA’s revenue.
0:16:24 Those are good companies to have as customers because they’re super rich, right?
0:16:27 They can afford to pay you tens of billions of dollars for your GPUs.
0:16:33 But that being said, they’re reselling that GPU power to companies and startups.
0:16:39 So startups are buying that GPU computing capacity from these tech companies.
0:16:40 That’s a good point.
0:16:44 So they’re in a way not the end customer, they’re the sort of intermediate kind of service
0:16:45 providers.
0:16:49 They also use that for their own internal systems too.
0:16:55 Meta uses a ton of GPU power to make their advertising algorithms more effective and to
0:16:58 pick the videos like TikTok does.
0:17:03 And obviously Google search is now becoming more and more AI driven and there’s Gemini,
0:17:04 which is right.
0:17:10 I mean, there’s more direct use of AI by all of the big tech companies as well.
0:17:14 So there’s another piece of this, which is interesting, I mean, because there’s one
0:17:19 universe where it’s like, oh, we got to get in on AI, and they all buy the GPUs.
0:17:20 And then that’s kind of it.
0:17:24 It’s like a step function where like there’s this momentary rush where everybody buys the
0:17:25 GPUs and then whatever.
0:17:31 They just upgrade every couple of years and it’s more like a regular tech hardware business,
0:17:33 which is like good, but not amazing.
0:17:37 But there’s another piece of it that has been a huge deal for NVIDIA and that’s the scaling
0:17:40 law or the scaling hypothesis.
0:17:41 Let’s talk about that.
0:17:42 What’s the scaling hypothesis?
0:17:47 The scaling hypothesis is similar to what I said before is the combination of computing
0:17:53 power, the more computing power you add, the more data you increase, and the better ways
0:17:56 you figure out software algorithms.
0:18:01 For each of these three buckets, if you increase it, the AI model becomes more capable and
0:18:02 more effective.
0:18:07 And like just to be clear, like even if the algorithms aren’t getting that much better,
0:18:08 right?
0:18:10 Like my understanding is, and this is kind of a new idea.
0:18:17 But like, oh, if we can just have more data and more computing power, we’ll get better
0:18:18 results.
0:18:19 Yes.
0:18:22 So it’s great if you have all three doing, you know, basically going up.
0:18:29 But for this time period, the last few years, the scaling law has really taken off.
0:18:39 And companies are literally 10xing their compute power and these AI clusters going from 16,000
0:18:41 GPUs now to 100,000 GPUs.
0:18:46 And now people are saying they’re going to build one million GPU clusters in the next
0:18:47 two, three years.
0:18:51 And NVIDIA is selling most of those GPUs.
0:18:52 Yes.
0:18:56 So if you go from 16,000 to 100,000 GPUs in a couple of years, that’s a good business
0:18:57 to be in.
0:19:03 If you’re increasing the scale of the hardware by 10x every couple of years.
0:19:08 And the exciting thing for NVIDIA is that, like what I argue is that this thing is going
0:19:13 to, this trend is going to continue in the next few years because you have the scaling
0:19:14 laws.
0:19:18 Then you have this thing called multimodal where they’re using this GPUs, not just for
0:19:25 text like in chat tpt, you’re using it for video and images, generating those things.
0:19:30 And now there’s these two other things, two other waves of demand that are happening.
0:19:36 There’s this thing called AI agents, where these AI models can do multi-step tasks for
0:19:37 you.
0:19:38 Right.
0:19:43 You could be like, book me a ticket to San Diego some weekend in June, whenever it’s the
0:19:44 best deal.
0:19:45 Yes.
0:19:46 And then it does that.
0:19:49 So that’s going to happen in the next 12 months, yes.
0:19:54 These AI agents will eventually in the next year or two be able to do all that tedious
0:20:00 work automatically and probably with less errors than an actual human being.
0:20:05 So AI agents, multimodal, and now there’s this thing called test time compute, where the
0:20:11 AI models instead of just spinning back an instant reply can actually think about what’s
0:20:15 the best way to respond to your question and spend more time thinking about it and then
0:20:18 give you a higher quality answer.
0:20:25 So there’s all these things that all of these test time compute, AI agents, multimodal,
0:20:30 these are all things that need more computing power, they need more GPUs and these are all
0:20:36 things that are going to drive NVIDIA’s revenue in the next couple of years.
0:20:41 We’ll be back in a minute and we will talk, among other things, about Jensen Wong, one
0:20:47 of the most successful entrepreneurs of the 21st century and a man who once told a colleague
0:20:53 that he wakes up every morning, looks in the mirror and says to himself, “You suck.”
0:21:03 Hey it’s Jacob, I’m here with Rachel Botsman, Rachel Lectures on Trust at Oxford University
0:21:09 and she is the author of a new Pushkin audiobook called How to Trust and Be Trusted.
0:21:10 Hi Rachel.
0:21:11 Hi Jacob.
0:21:17 Rachel Botsman, tell me three things I need to know about trust.
0:21:21 Number one, do not mistake confidence for competence.
0:21:22 Big trust mistakes.
0:21:28 So when people are making trust decisions, they often look for confidence versus competence.
0:21:35 Number two, transparency doesn’t equal more trust, big myth and misconception and a real
0:21:37 problem actually in the tech world.
0:21:43 The reason why is because trust is a confident relationship with the unknown.
0:21:47 So what are you doing if you make things more transparent?
0:21:49 You’re reducing the need for trust.
0:21:55 And number three, become a stellar expectation setter.
0:22:00 Inconsistency with expectations really damages trust.
0:22:00 I love it.
0:22:04 Say the name of the book again and why everybody should listen to it.
0:22:07 So it’s called How to Trust and Be Trusted.
0:22:13 Intentionally it’s a two-way title because we have to give trust and we have to earn trust
0:22:19 and the reason why I wrote it is because we often hear about how trust is in a state of crisis
0:22:21 or how it’s in a state of decline.
0:22:26 But there’s lots of things that you can do to improve trust in your own lives,
0:22:30 to improve trust in your teams, trusting yourself to take more risks
0:22:33 or even making smarter trust decisions.
0:22:37 Rachel Botsman, the new audiobook is called How to Trust and Be Trusted.
0:22:38 Great to talk with you.
0:22:44 It’s so good to talk with you and I really hope listeners listen to it because it can change people’s lives.
0:22:48 I want to talk about Jensen a little bit.
0:22:53 I mean, your book, he’s obviously the main character in the history of NVIDIA.
0:22:54 He’s the main character in your book.
0:22:58 He’s been the CEO of NVIDIA for more than 30 years,
0:23:01 which is like longer than Bill Gates with the CEO of Microsoft,
0:23:06 longer than almost anybody in the S&P 500 has been a CEO at this point.
0:23:09 His childhood is quite interesting, right?
0:23:11 Like, tell me just a little bit about his childhood.
0:23:17 He was born in Taiwan and they moved around a little bit to Thailand.
0:23:23 And his father came to training in New York and fell in love with America.
0:23:26 This is like the great American dream story.
0:23:34 And the mother and father started teaching Jensen and his brother English 10 words a day,
0:23:38 and they sent him to his aunt and uncle when he was about age eight or nine.
0:23:43 And the aunt and uncle and the families, the funny thing is,
0:23:46 they sent him to a reform school in Kentucky by mistake,
0:23:51 thinking that he would get a great education at this boarding school in Kentucky.
0:23:53 But they just thought it was a boarding school.
0:23:56 Yes, but it turned out to be a school for troubled kids.
0:23:59 Which Jensen was not.
0:24:03 He was just like a smart kid with ambitious for him parents.
0:24:03 Yes.
0:24:09 But he talks about how his time at Oneida Baptist Institute in Kentucky was formational for him.
0:24:10 What was it like for him?
0:24:14 It was hard at the beginning, but he started befriending people.
0:24:19 He started playing chess with the janitor.
0:24:23 And he just learned how to deal with other people much better.
0:24:27 And he talks about how he learned his street fighter mentality.
0:24:28 Because he was literally fighting.
0:24:32 Yes, I mean, there were bullies and all that stuff,
0:24:38 but he just learned how to deal with people and the rough and tumble of kids back then.
0:24:44 Eventually, his parents came from abroad and they settled in Oregon.
0:24:49 And he learned his work ethic from working at Denny’s.
0:24:57 He talks about how he washed more dishes and cleaned more bathrooms than any CEO in the history of CEOs.
0:25:04 He says Denny’s helped them in terms of social skills and dealing with time pressure and dealing with customers.
0:25:09 But it’s the work ethic that really sets him apart.
0:25:14 So even at the beginning, he was working from 9am to midnight.
0:25:19 And he just set a culture where NVIDIA employees work really hard.
0:25:22 And I feel like at the beginning, that’s common, right?
0:25:25 Like that is the classic start-up story.
0:25:27 But the classic start-up story is you do that for five years.
0:25:33 And then either you get giant and you hire a grown-up CEO and you, you know,
0:25:37 go start your blimp company or whatever, or you sell or whatever.
0:25:39 He’s still doing that, right?
0:25:45 He’s 60 years old and like wildly rich and he’s still working that much.
0:25:46 He’s working all weekend.
0:25:48 He’s working Saturday, Sunday.
0:25:54 He talks about when he goes to a movie, he doesn’t remember the movie because he’s thinking about work.
0:25:57 He finds work relaxing and fun.
0:25:58 This is what drives him.
0:26:00 He loves working.
0:26:03 There’s a moment, you know, you don’t, you don’t write about his personal life at all
0:26:09 once he sort of grows up and starts NVIDIA, you know, reasonably because the book’s about NVIDIA.
0:26:13 And so I just assumed he just worked all the time and didn’t have a family.
0:26:18 And I think the only time in the book that his family comes up is there’s this scene
0:26:23 where he’s on vacation and he’s talking to some senior manager at NVIDIA on the phone
0:26:24 and they’re like, what are you doing?
0:26:29 And he’s like, I’m sitting here on the balcony watching my kids play in the sand and writing emails.
0:26:33 And it’s like, like, if it’s a movie, the kids are never on stage, right?
0:26:35 You just hear them offstage at that moment.
0:26:37 You’re like, oh, he has kids.
0:26:42 Like that was, that was a weird moment to me reading that, that line.
0:26:48 So both executives told me they hate it when Jensen so-called goes on vacation
0:26:53 because he winds up giving them more directives and orders
0:26:57 and more stuff to do when he’s on vacation because he’s emailing them, do this, do that.
0:27:00 And they, they actually yell at him.
0:27:02 So you play with your kids.
0:27:07 And he’s like, no, I could get real work done when I’m on vacation by doing his emails.
0:27:08 I mean-
0:27:11 So it shows you his obsession.
0:27:15 He’s constantly thinking, he’s constantly worried about what’s happening at NVIDIA.
0:27:17 I mean, it’s, it’s interesting, right?
0:27:25 Like, he’s not a balanced person, like, which is why, to some extent,
0:27:28 he has built the thing that he has built, right?
0:27:34 He’s completely obsessed with winning and he’s extremely competitive.
0:27:35 Yeah.
0:27:37 There’s a few other specific moments that really stood out to me.
0:27:43 There’s one where I think this, I think a salesperson told you this,
0:27:46 where they had just had a great quarter.
0:27:52 They’d sold a ton of GPUs and the sales guys talking to Jensen about how great they’re doing.
0:27:56 And Jensen says to the sales guy about Jensen, about himself.
0:28:01 Jensen says, I look in the mirror every morning and say, you suck.
0:28:05 So he, he’s almost like a self psychologist.
0:28:10 He knows if you start thinking that you’re, you’re the best and you’re, you’re hot stuff,
0:28:11 you might get complacent.
0:28:13 You start resting on your laurels.
0:28:14 You might not work as hard enough.
0:28:17 So he, he sees that, oh, we just had a blowout quarter.
0:28:19 What’s the risk here?
0:28:21 The risk is me getting complacent.
0:28:25 So I’m going to look myself in the mirror and say you suck and psych myself out.
0:28:29 I mean, that’s, that’s, that’s one reading of it.
0:28:31 That’s the like 10 dimensional chess reading of it.
0:28:33 I mean, there’s another reading of it, which is like,
0:28:38 he’s messed up in a way that makes him super driven and successful.
0:28:40 But it works.
0:28:41 He actually definitely works.
0:28:43 We can agree that it works.
0:28:45 But he does the opposite too.
0:28:50 So when things are really intimidating and he feels like, oh my gosh, how am I going to do this?
0:28:52 He tells himself, how hard can it be?
0:28:54 How hard can it be Jensen?
0:28:58 So he does it on both, both ends of this spectrum where he,
0:29:01 he cites himself out when he feels intimidated.
0:29:04 And when he feels like he’s on top of the world,
0:29:06 he tries to bring himself down back to the earth.
0:29:12 So the one other big piece of, of the Jensen Wong experience that we haven’t talked about
0:29:15 is the way he treats the people who work for him, right?
0:29:19 I want to talk about a particular scene,
0:29:22 because I think it, it puts a finer point on a scene from the book where
0:29:26 it’s like a company-wide meeting and it’s on like Zoom or whatever.
0:29:28 And he’s yelling at a guy in the meeting.
0:29:29 He’s yelling at him.
0:29:31 And then on top of that,
0:29:36 he keeps telling the person filming the meeting to zoom in on the guy he’s yelling at.
0:29:41 I had multiple sources tell me that it was the most humiliating thing they’ve ever seen.
0:29:43 I mean, it’s, that seems like bullying.
0:29:46 Like why zoom in on the guy?
0:29:48 Like what lesson is added by that?
0:29:52 So he kept on saying to Mr. Rayfield,
0:29:55 you got to get this chip back on track.
0:29:58 The chip that he was in charge of was behind schedule.
0:30:03 And he kept on pointing to him, zooming in his face and telling him,
0:30:04 you’ve got to get this.
0:30:05 This is not how you run a business.
0:30:08 You need to get this chip back on track.
0:30:11 And this kind of like high standard demanding attitude,
0:30:14 I think is effective.
0:30:15 It drives people.
0:30:18 If you get dressed down by Jensen,
0:30:23 the next time you’re going to work 10 times more to make sure that you do a better job.
0:30:27 Maybe I don’t want the world to be that way, but it is that way.
0:30:28 You know what I mean?
0:30:30 Like maybe I want to believe that like,
0:30:34 there’s a kinder world where people could do equally good work.
0:30:35 But maybe I’m wrong.
0:30:39 And Steve Jobs was the same way.
0:30:39 Yeah.
0:30:42 It wasn’t all sunshine and rainbows.
0:30:43 No, no.
0:30:46 I mean, the Isaacson book was really clear, right?
0:30:48 The Isaacson biography of Steve Jobs.
0:30:51 He did not seem like a good person in that book.
0:30:53 He seemed like really good at building amazing products,
0:30:59 but like not like a good human being by the sort of standard way we think of what makes a good person.
0:31:04 But I think the key point that we also have to think about is people at NVIDIA stay.
0:31:05 Yeah.
0:31:06 That’s really interesting.
0:31:11 The turnover is like one of the lowest in the industry at only 3% compared to the average of 13 to 15%.
0:31:18 I mean, certainly in the last few years, people stay because you get rich if you stay and you lose your equity if you leave.
0:31:21 But is that number true if you go back to when the stock was flat?
0:31:25 I don’t have the numbers, but a lot of the senior executives have been there
0:31:29 more so than any other company, 15, 20, 25 years.
0:31:30 Yeah, that’s interesting.
0:31:36 It’s you don’t really hear other than one instance of people leaving to become a CEO of a different company.
0:31:45 And I think part of it is people realize that NVIDIA or the Jensen way of doing things is super effective.
0:31:47 So people want to be on the winning team.
0:31:51 So in one sense, he drives people hard.
0:31:52 He dresses them down.
0:31:56 He’s very blunt and direct, but he compensates people.
0:31:58 No one ever hardly anyone can play.
0:31:59 It’s about compensation.
0:32:06 If you’re effective and you do a good job, he will like double your stock compensation on the spot.
0:32:10 So there’s this meritocracy that people really adore.
0:32:17 And if you see an effective leadership, a culture that’s based on meritocracy,
0:32:20 if you’re a top engineer, you stay at that company.
0:32:21 Yeah, yeah.
0:32:26 So you interviewed Jensen as you as you were working on the book.
0:32:28 Was it scary to interview him?
0:32:31 He was intimidating in the meeting.
0:32:33 Did he yell at you?
0:32:38 He didn’t yell at me, but a couple of times he was saying you don’t get NVIDIA in this line of questioning.
0:32:46 And it takes a while to get used to it, but you appreciate it because you see where the other person is.
0:32:50 Someone is blunt and direct like Jensen.
0:32:55 You know where he is at all times and then you can work at improving yourself.
0:32:59 So he talks about what are you optimizing for?
0:33:05 Are you optimizing for a person’s feelings or optimizing for what’s good for the company?
0:33:08 So that’s why he doesn’t do one-on-one meetings or career coaching.
0:33:14 Typically, at a large company, when someone’s doing poorly, the CEO will take them aside and say,
0:33:16 “Bob, you need to do this. You need to do that.”
0:33:17 Like if it’s a senior person.
0:33:23 Jensen’s like, “Why is Bob the only one that gets a learning here?”
0:33:29 If I am blunt and direct and show Bob what he’s doing incorrectly like that person at that meeting,
0:33:33 everyone in the room can learn.
0:33:36 All the employees up and down the ladder can learn.
0:33:40 So that’s his philosophy is everyone should learn from their mistakes
0:33:42 and everyone should know where they are at all times.
0:33:51 So, I mean, so you write in the book that NVIDIA really is like an extension of Jensen, right?
0:33:57 I think he used the metaphor of like the Formula One car that’s like optimized for him as the driver.
0:34:09 So, and he’s in his 60s, which is not old old, but definitely not young.
0:34:13 Like is he going to run NVIDIA for another 20 years?
0:34:14 What’s going to happen?
0:34:16 Like there’s no obvious successor.
0:34:17 Like where does that go?
0:34:27 I think there’s no one else at NVIDIA, I think, that can run NVIDIA as effectively as Jensen.
0:34:29 And he loves the company so much.
0:34:30 He loves what he’s doing.
0:34:33 They’re having an enormous impact with this AI wave.
0:34:38 And he’s so excited about the potential for curing cancer and digital biology,
0:34:46 the potential for robots, the potential for AI to kind of disrupt education and help kids learn better.
0:34:49 I don’t see him leaving anytime soon.
0:34:54 He loves his job so much and he can’t argue that he’s being ineffective.
0:35:01 So, I don’t think for the next few years there’s anyone that’s going to take over for him.
0:35:05 Someday, NVIDIA is going to have to come up with a new CEO or a successor to Jensen.
0:35:08 And that’s going to be a big question mark.
0:35:15 Is that next person going to be as effective as Jensen in terms of being able to have the technical skill
0:35:19 and the competency to steer NVIDIA in the right direction?
0:35:25 Will they have his business genius of coming up with all these new strategies
0:35:27 that he does time and time again?
0:35:29 That’s going to be a huge question mark.
0:35:39 I mean, people also talk about limits to scaling in AI.
0:35:45 We were talking earlier about how part of NVIDIA’s wild growth over the last couple of years
0:35:54 has come from this fact that you can just add more GPUs, basically, and get better results,
0:35:55 more GPUs, more data.
0:36:05 People talk about, A, running out of data, because they’re basically, as I understand it,
0:36:09 training on the whole internet right now, which is like, okay, it’s a lot of data.
0:36:14 Is that, it seems like nobody knows, right?
0:36:17 It seems like there are smart people who say, no, no, we’ll have synthetic data,
0:36:19 blah, blah, blah, we won’t hit a sort of scaling wall.
0:36:21 And then there are people who make the other argument.
0:36:22 I certainly don’t know.
0:36:23 But does that seem right?
0:36:25 Like, is that an open question right now?
0:36:27 And is that a meaningful question for NVIDIA?
0:36:29 I think it’s an open question.
0:36:35 But like I said, there’s the multimodal stuff, there’s AI agents,
0:36:38 and then there’s proprietary data inside companies.
0:36:44 So all these corporations have data going back decades.
0:36:51 Companies are going to use the power of these AI computing systems to go through all their data
0:36:57 internally, all their proprietary data, and have all that knowledge at employees’ fingertips.
0:37:03 So an employee can ask, what’s the best way to do this?
0:37:08 And the AI computer is going to be able to go back with 30 years of data,
0:37:11 and figure out the best piece of insight to help that employee.
0:37:13 And that’s not really being done today.
0:37:19 So NVIDIA designs their chips, their GPUs, but they don’t actually manufacture them, right?
0:37:24 Is it right that they’re made in Taiwan, where most cutting-edge chips are made now?
0:37:30 Yes. So TSMC actually makes and manufactures NVIDIA’s chips, ever since the late 1990s.
0:37:32 And they’re the best at doing this.
0:37:40 A lot of the fabless chip designers in California use TSMC, and NVIDIA uses them too.
0:37:49 And so, I mean, it is really interesting to me, to a lot of people, that Taiwan is this
0:37:55 incredibly fraught geopolitical place, right? China says Taiwan is part of China,
0:38:01 Taiwan says, no, we are not part of China, and that Taiwan is the only place in the world that
0:38:06 makes the most important physical thing in the world of technology today, right?
0:38:12 That’s wild. What do you make of that? And how does that fit with the NVIDIA story?
0:38:18 I think, eventually, NVIDIA chips will be made in the US, TSMC. I mean, that’s
0:38:22 all part of this CHIPS Act that the Biden administration has passed.
0:38:29 So this plant that TSMC is building in Arizona, is that advanced enough to make
0:38:35 like frontier NVIDIA chips? So the TSMC factories in the US
0:38:39 are always going to be one or two generations behind the factories in Taiwan.
0:38:46 But that doesn’t mean NVIDIA can’t use the factories in the US if they’re one or two
0:38:51 generations behind further older chips. So I think that will eventually happen.
0:38:57 It seems wild to me. I mean, it seems very possible that China will try and
0:39:02 make Taiwan be part of China, right? And we have all these export controls to try and
0:39:07 keep China from getting cutting-edge chips. It’s a really interesting complicated dynamic.
0:39:15 I think people make a big issue out of this, but I think if it happens, NVIDIA chips are not going
0:39:24 to be the main issue. It’ll cause a global calamity where we won’t be able to upgrade our cars,
0:39:32 laptops, nearly every computing device, every appliance will not work if we don’t have access
0:39:39 to Taiwan chips. I mean, presumably we could do some like sub-Manhattan project, scale Manhattan
0:39:47 project to get a state-of-the-art TSMC factory somewhere that is not Taiwan, right? Like one
0:39:50 would think. We’re doing some of that now, but it’s not going to be up to,
0:39:58 it’s going to be 10, 20% of the capacity we need. Yeah, I see the capacity, right? It takes a long
0:40:04 time. The fabs are wildly complicated to build and they cost billions of dollars and they take
0:40:08 years. So you couldn’t just do it. They cost 10 to 20 billion dollars to build in three to four years,
0:40:12 over three to four years. So it’s not something that can happen overnight.
0:40:17 And if something happens, it’s just going to be so terrible. It’ll be like a depression.
0:40:25 So hopefully it doesn’t happen. And the question isn’t going to be about Nvidia chips.
0:40:27 The question is going to be about the global economy of the app.
0:40:33 Why do you think it’s the biggest risk for Nvidia in the five-year timeframe?
0:40:39 It’s really tough to say now. I mean, I see the next few years with all the different AI
0:40:45 innovations and all the progress. But again, just like every big computing shift,
0:40:51 what’s the next big thing? Could it be quantum computing? It can be. Like who knows 5, 10,
0:40:58 15 years from now. And from the PCH with Microsoft and Intel, it’s called Wintel.
0:41:04 And then it went to smartphones where Apple dominated with the iPhone. And now,
0:41:10 Nvidia is dominating in terms of the AI computing shift. There’s going to be another shift.
0:41:14 And right now we’re at the early stages of the AI computing
0:41:22 movement. In five, 10 years, there might be the next big thing that I can’t even foresee right
0:41:29 now. And is Nvidia going to be able to see that coming? If Jensen’s around, I think he will.
0:41:36 But if Jensen’s not there, then just like every other major computer company in history,
0:41:41 say IBM, it’s very easy to get disrupted in the technology industry.
0:41:47 We’ll be back in a minute with the lightning round.
0:42:01 Hey, it’s Jacob. I’m here with Rachel Botsman. Rachel lectures on trust at Oxford University.
0:42:07 And she is the author of a new Pushkin audiobook called How to Trust and Be Trusted.
0:42:14 Hi, Rachel. Hi, Jacob. Rachel Botsman. Tell me three things I need to know about trust.
0:42:21 Number one, do not mistake confidence for competence. Big trust mistakes. So when people
0:42:28 are making trust decisions, they often look for confidence versus competence. Number two,
0:42:34 transparency doesn’t equal more trust. Big myth and misconception. And a real problem,
0:42:40 actually, in the tech world. The reason why is because trust is a confident relationship with
0:42:46 the unknown. So what are you doing if you make things more transparent? You’re reducing the need
0:42:55 for trust. And number three, become a stellar expectation setter. Inconsistency with expectations
0:43:02 really damages trust. I love it. Say the name of the book again and why everybody should listen to
0:43:08 it. So it’s called How to Trust and Be Trusted. Intentionally, it’s a two-way title because we
0:43:14 have to give trust and we have to earn trust. And the reason why I wrote it is because we often hear
0:43:20 about how trust is in a state of crisis or how it’s in a state of decline. But there’s lots of
0:43:26 things that you can do to improve trust in your own lives, to improve trust in your teams,
0:43:31 trust in yourself to take more risks, or even making smarter trust decisions.
0:43:36 Rachel Botsman, the new audiobook is called How to Trust and Be Trusted. Great to talk with you.
0:43:41 It’s so good to talk with you and I really hope listeners listen to it because it can change
0:43:49 people’s lives. I want to finish with the lightning round, which is going to be considerably more
0:43:56 random and digressive than the conversation to this point. I want to talk about video games
0:44:02 and your experience with video games. What was the first video game you ever loved?
0:44:10 I think it’s the first video I’ve ever tried, which was Combat for the Atari 2600.
0:44:17 I love that. I mean, I remember coming home and my dad buying the Atari 2600 connected to
0:44:22 a black and white TV and being able to use that joystick to control the little tank
0:44:28 going around the screen. That was an amazing moment. It was an incredible moment that this was
0:44:36 possible at the home. I had an Atari 2600. I remember the joystick. I remember Demon Attack.
0:44:43 It was a kind of second tier game, but I got really into it. What’s the game you spent the most hours on?
0:44:51 Probably this game called Lemmings. I don’t know. It’s a puzzle strategy game
0:44:58 where you control Lemmings and you guide them across a maze to get them to them.
0:45:02 Lemmings, the little creatures that are famed for jumping off cliffs, which maybe they don’t
0:45:08 actually do? Yes. If you guide them the wrong way, they’ll fall off a cliff and to their death.
0:45:14 They make this cute sound like, “Oh, no.” I played so many hours of that game.
0:45:20 A thousand hours? Probably hundreds of hours, I wouldn’t say a thousand.
0:45:25 What’s your most proud video game accomplishment?
0:45:34 I still remember playing the Legend of Zelda, the first one for the Nintendo Entertainment System,
0:45:37 and beating that game and just pumping my fists in the air.
0:45:46 I was very young then. What is the most exciting video game innovation
0:45:48 that’s going to happen in the next few years?
0:45:57 This is a little technical, but I think DLSS from the video is going to get even more powerful,
0:46:02 which is this upscaling technology that Jensen actually invented in a meeting
0:46:11 where they fill in the details using AI. The frame rates are able to go much faster and better.
0:46:17 So, it’s basically like interpolation. It’s sort of doing what AI always does for graphics.
0:46:20 It’s really effective. People can’t tell the difference between the real thing and the
0:46:28 interpolated AI graphics. As this technology gets better, that will allow the graphics to be
0:46:34 more photorealistic and the physics and the ray tracing better than ever before.
0:46:39 What game are you most excited to play in 2025?
0:46:44 I mean, Grand Theft Auto VI, if it comes out, but it might get delayed next year.
0:46:50 Okay, let’s do a few non-video game questions. What’s your favorite
0:46:59 tech book besides the one you wrote? I guess it’s still a tech book, The Innovator’s Dilemma,
0:47:05 which is actually one of Jensen’s favorite books. It talks about how companies get disrupted by
0:47:12 startups and people underneath them. It really goes into depth about how the disk drive industry,
0:47:16 every successive generation that disk drives, there was a new market leader,
0:47:22 because they couldn’t disrupt themselves to the new or smaller format.
0:47:27 I mean, to me, the really interesting insight of that book is the innovator
0:47:34 is actually making a crappier product. That’s the surprise. It’s not exactly like they come
0:47:39 along and do something better. They go to the crappy end of the market and they make a cheap
0:47:45 product. It’s not better than the thing the market leader is making. The market leader is like,
0:47:53 “Oh, that, who cares about that? That’s just some low margin thing.” The innovator comes up from
0:48:00 below. That to me is the really key insight of that book. They scale the volumes. Once you have
0:48:04 the volume, it pretty much becomes game over for the incumbent, because they can’t match
0:48:08 that scale and the economies of scale that comes with that.
0:48:15 Defend pineapple on pizza. It just tastes good. I love it.
0:48:20 I don’t know if you saw my tweets on it. I actually like pineapple on pizza.
0:48:23 Yes. I think it was Instagram. I think it was Instagram.
0:48:28 What’s the best deal you ever got at Costco?
0:48:32 It’s still the hot dog. I mean, you can’t beat the dollar fish.
0:48:47 Take Him is the author of The NVIDIA Way. Today’s show was produced by Gabriel Hunter Chang.
0:48:55 It was edited by Lydia Jean Cotte and engineered by Sarah Brugger. You can email us at problem@pushkin.fm.
0:49:02 I’m Jacob Goldstein and we’ll be back next week with another episode of What’s Your Problem?
0:49:05 What’s Your Problem?
0:49:14 Hey, it’s Jacob. I’m here with Rachel Botsman. Rachel lectures on trust at Oxford University
0:49:20 and she is the author of a new Pushkin audiobook called How to Trust and Be Trusted.
0:49:22 Hi, Rachel.
0:49:23 Hi, Jacob.
0:49:27 Rachel Botsman, tell me three things I need to know about trust.
0:49:34 Number one, do not mistake confidence for competence. Big trust mistakes. So when people
0:49:40 are making trust decisions, they often look for confidence versus competence.
0:49:46 Number two, transparency doesn’t equal more trust. Big myth and misconception.
0:49:52 And a real problem actually in the tech world. The reason why is because trust is a confident
0:49:57 relationship with the unknown. So what are you doing if you make things more transparent?
0:50:05 You’re reducing the need for trust. And number three, become a stellar expectation
0:50:11 setter. Inconsistency with expectations really damages trust.
0:50:16 I love it. Say the name of the book again and why everybody should listen to it.
0:50:21 So it’s called How to Trust and Be Trusted. Intentionally, it’s a two-way title because
0:50:27 we have to give trust and we have to earn trust. And the reason why I wrote it is because we often
0:50:33 hear about how trust is in a state of crisis or how it’s in a state of decline. But there’s lots
0:50:39 of things that you can do to improve trust in your own lives, to improve trust in your teams,
0:50:44 trusting yourself to take more risks, or even making smarter trust decisions.
0:50:49 Rachel Botsman, the new audiobook is called How to Trust and Be Trusted. Great to talk with you.
0:50:54 It’s so good to talk with you. And I really hope listeners listen to it because it can change
0:50:55 people’s lives.
In the past few years, NVIDIA has become one of the most valuable and important companies in the world by making GPUs, the chips powering the AI boom. But where did the company come from, and why are NVIDIA chips the ones that dominate AI?
Tae Kim is the author of a new book called The Nvidia Way. In his book, he tells the story of how NVIDIA’s founder and CEO, Jensen Huang, set NVIDIA on the path to becoming what it is today.
See omnystudio.com/listener for privacy information.