AI transcript
0:00:01 (upbeat music)
0:00:02 – Hello, everyone.
0:00:04 Welcome back to the A16Z podcast.
0:00:05 This is Stuff.
0:00:07 Now, one of my favorite podcasts we’ve recorded
0:00:09 since I joined the team
0:00:11 was just about this time last year.
0:00:13 That episode was on autonomous vehicles,
0:00:17 but it was actually also in an autonomous vehicle.
0:00:20 That was my first ride in a self-driving car.
0:00:21 And over the last year,
0:00:23 I’ve seen so many others have their first
0:00:25 as Waymo has expanded to the public
0:00:27 in Phoenix and San Francisco,
0:00:30 while also placing its roots in Austin and LA.
0:00:32 In 2015, Waymo tested
0:00:35 its first fully driverless ride on public roads.
0:00:38 It then opened to the public in Phoenix in 2020,
0:00:40 but it wasn’t until 2022
0:00:43 that autonomous drives were offered in San Francisco.
0:00:45 And by the end of 2023,
0:00:48 it clocked in over 7 million driverless miles.
0:00:51 Slowly, then all at once.
0:00:53 So with this space moving so quickly,
0:00:54 we wanted to give you an update
0:00:57 on where this industry is today.
0:01:00 Passing the baton to properly introduce this episode,
0:01:02 here is our very own AI Revolution host
0:01:06 and A16Z General Partner, Sarah Wang.
0:01:08 As a reminder,
0:01:11 the content here is for informational purposes only.
0:01:13 Should not be taken as legal, business, tax
0:01:14 or investment advice,
0:01:16 or be used to evaluate any investment or security
0:01:18 and is not directed at any investors
0:01:21 or potential investors in any A16Z fund.
0:01:23 Please note that A16Z and its affiliates
0:01:24 may also maintain investments
0:01:27 in the companies discussed in this podcast.
0:01:28 For more details,
0:01:29 including a link to our investments,
0:01:33 please see A16Z.com/disclosures.
0:01:35 (upbeat music)
0:01:39 – Hey guys, I’m Sarah Wang,
0:01:42 General Partner on the A16Z growth team.
0:01:45 Welcome back to our AI Revolution series.
0:01:46 In this series,
0:01:48 we talk to the Gen AI builders
0:01:50 who are transforming our world to understand,
0:01:53 one, where we are, two, where we’re going
0:01:56 and three, the big open questions in the field.
0:01:59 Our guest this episode is Dmitry Dolgov,
0:02:01 the co-CEO of Waymo.
0:02:04 Dmitry has led Waymo to solve some of the biggest challenges
0:02:06 and bringing AI to the real world.
0:02:09 And after tens of millions of miles of testing,
0:02:11 Waymo’s vehicles have shown themselves
0:02:14 to be safer and more reliable than human drivers,
0:02:16 myself included.
0:02:18 Dmitry has a unique perspective,
0:02:20 given that his work has spanned multiple AI ML development
0:02:22 cycles across decades.
0:02:25 He was an early pioneer in self-driving cars,
0:02:28 working with Toyota and Stanford on DARPA’s grand challenge
0:02:30 before joining Google’s self-driving car project,
0:02:33 which then evolved into Waymo.
0:02:35 In this conversation from a closed door event
0:02:38 with A16Z General Partner, David George,
0:02:41 Dmitry talks about the potential of embodied AI,
0:02:43 the value of simulations and building training data,
0:02:46 and his approach to leading a company focused on solving
0:02:49 some of the world’s hardest problems.
0:02:50 Without further ado,
0:02:53 here’s Dmitry in conversation with David.
0:02:56 (dramatic music)
0:03:05 – Maybe to start, take us back to Stanford, if you will.
0:03:07 And that was when you first started working
0:03:09 on the DARPA project.
0:03:12 And maybe give us a little bit of your history
0:03:15 of how you ended up from there to here.
0:03:18 – My introduction to autonomous vehicles
0:03:21 was when I was doing a postdoc at Stanford,
0:03:23 that you just mentioned, David.
0:03:28 This was during, I got pretty lucky with the timing of it.
0:03:30 This was when the DARPA grand challenges were happening.
0:03:33 DARPA is the Defense Advanced Research Project Agency
0:03:35 that started these competitions with the goal
0:03:39 of boosting this field of autonomous vehicles.
0:03:44 And the one that I got involved in was in 2007,
0:03:46 that was called the DARPA Urban Challenge.
0:03:50 So the setup there was, it’s kind of like a toy version
0:03:53 of what we’ve been working on since then.
0:03:55 It was kind of supposed to mimic the driving
0:03:56 in urban environments.
0:04:00 So they kind of created a fake city on an abandoned airbase
0:04:03 and they populated it with a bunch of autonomous vehicles,
0:04:04 a bunch of human drivers,
0:04:07 and they had them do various tasks.
0:04:12 So that was kind of my introduction to this whole field.
0:04:14 And it was a bit of a, I think in a DARPA,
0:04:16 these challenges are often by people in the industry
0:04:19 considered kind of a foundational pivotal moment
0:04:22 for this whole field.
0:04:23 And it was definitely that for me.
0:04:27 It was like a light bulb, light switch moment
0:04:29 that really got me hooked.
0:04:31 – What was like the hardware and software
0:04:33 that you guys had at that point?
0:04:35 Is this 2007?
0:04:39 – Yeah, I know it’s a, at a very high level,
0:04:41 not unlike what we talk about today.
0:04:43 A car that has some instrumentation
0:04:45 so you can tell it what to do
0:04:47 and you get some feedback back.
0:04:50 Then you have kind of what’s called a post system,
0:04:54 a bunch of inertial measurement system accelerometers,
0:04:55 gyroscopes that kind of tell you in GPS,
0:04:57 tells you how you’re moving through space.
0:05:00 And it has sensors, radars, lighters and cameras,
0:05:02 those same stuff we still use today.
0:05:06 And then there’s a computer that gets the sensor data in
0:05:08 and then tells the car what to do.
0:05:10 And a bunch of software and software head,
0:05:13 perception components and decision-making planning
0:05:15 components and some AI.
0:05:17 But of course everything that we had,
0:05:20 like each one of those things over that,
0:05:21 how long has it been?
0:05:23 Almost 18 years, more than that.
0:05:24 It’s changed drastically, right?
0:05:26 So when we talk about AI today versus AI,
0:05:30 we had back in 2007, 2009, nothing in common.
0:05:31 And similarly, everything else has changed.
0:05:34 The sensors are not the same, computers are not the same.
0:05:35 – Yeah, of course.
0:05:37 So then, okay, so take us, so at that point,
0:05:40 that was the pivotal, that was like the light bulb moment.
0:05:43 And then at that point, you said, okay, I’m at Stanford,
0:05:45 I wanna make this my career, right?
0:05:47 Is that, and then it was Toyota,
0:05:49 and then where did it go from there?
0:05:51 – I don’t know if I thought about it in those terms.
0:05:55 I was like, this is the future, I wanna make it happen.
0:05:56 I wanna be building this thing, career.
0:05:58 Okay, you know, they can wait.
0:06:00 But it was, that was the next step.
0:06:03 That was the next big step is a number of us
0:06:06 from the DARPA Challenge competitions
0:06:09 started the Google self-driving project.
0:06:12 It was about a dozen of us, and then in 2009,
0:06:14 came together at Google with support,
0:06:16 an assignment from Larry and Sergey,
0:06:19 to see if we can take it to the next step.
0:06:24 And that then, we worked on it for a few years,
0:06:26 and that project then became Waymo in 2016,
0:06:29 and we’ve been on this path since then.
0:06:31 – Okay, so we have this new big breakthrough
0:06:33 in generative AI.
0:06:34 Some would say it’s new,
0:06:36 some would say it’s 70 years in the making.
0:06:40 How do you think about layering advances
0:06:43 that have come from generative AI
0:06:46 to what many would describe as more traditional AI
0:06:48 or machine learning techniques
0:06:50 that were kind of the building blocks
0:06:53 for self-driving technology up to that point?
0:06:54 – Oh yeah, great question.
0:06:56 So maybe generative AI is kind of a broad term.
0:06:59 So maybe you can take a little bit of a step back
0:07:02 and talk about the role that AI plays
0:07:05 in autonomous vehicles and kind of how we saw
0:07:07 the various breakthroughs in AI
0:07:09 map to the space of our task, right?
0:07:11 So like I mentioned,
0:07:16 AI has been part of self-driving autonomous vehicles
0:07:18 from the earliest days.
0:07:20 Back when we started, it was a very different kind of AI,
0:07:22 ML, kind of classical maintenance,
0:07:24 decision trees, classical computer visions
0:07:27 with kind of hand engineered features,
0:07:28 kernels and so forth.
0:07:34 And then one of the,
0:07:37 first really important breakthroughs
0:07:41 that happened in AI and computer vision
0:07:44 but really was important for our task
0:07:48 was the advancement in convolutional neural networks
0:07:51 right around 2012, right?
0:07:54 Many of you are probably familiar with AlexNet
0:07:55 and the ImageNet competition.
0:07:58 This is where AlexNet kind of blew away
0:08:01 out of the water all other approaches.
0:08:04 So that obviously has had very strong implications
0:08:06 for our domain, like how you do computer vision
0:08:07 and not just on cameras, right?
0:08:11 How you can use ConvNets to interpret what’s around you
0:08:13 and do kind of object detection and classification
0:08:14 from camera data, from LiDAR data,
0:08:16 from your imaging radars.
0:08:18 So that was kind of a big boost
0:08:22 around that 2012, 2013 timeframe.
0:08:24 And then we played with those approaches
0:08:27 and tried to extend the use of ConvNets to other domains,
0:08:30 just beyond perception with some interesting
0:08:32 but limited success.
0:08:36 Then another big, very important breakthrough
0:08:41 happened around 2017 when Transformers came around.
0:08:43 It had a really huge impact on language,
0:08:45 language understanding, language models,
0:08:48 machine translation, so forth.
0:08:52 And for us, it was a really important breakthrough
0:08:54 that really allowed us to take a mel in AI
0:08:58 to new areas well beyond perception.
0:09:00 And so if you think about, you know,
0:09:03 Transformers and the impact that they had on language,
0:09:06 ConvNets, the intuition is that they’re good at understanding
0:09:10 and predicting and generating sequences of words, right?
0:09:14 And in our case, we think about, in our domain,
0:09:17 about the tasks of understanding and predicting
0:09:20 what people will do, like other actors in the scene,
0:09:22 or the task of decision-making
0:09:23 and planning your own trajectories
0:09:26 or in simulation, generating generative AI,
0:09:29 or our version of generating behaviors
0:09:31 of how the world will evolve.
0:09:34 That kind of these behavioral,
0:09:37 like these sequences are not unlike sentences, right?
0:09:39 You’re kind of operating the state of objects, right?
0:09:40 And then there’s kind of local continuity,
0:09:42 but then the global context of the scene really matters.
0:09:44 So this is where we saw some really exciting breakthroughs
0:09:48 in behavior prediction and decision-making and simulation.
0:09:50 And then, you know, since then we’ve been on this trend
0:09:52 of, you know, models getting bigger.
0:09:56 People started building foundation models
0:09:57 for multi-tasks.
0:10:00 And most recently, all of the,
0:10:01 can I use the last couple of years,
0:10:04 all the breakthroughs in large language models.
0:10:08 You know, modern state, modern-day generative AI,
0:10:11 visual language models where you kind of align
0:10:13 image understanding and language understanding.
0:10:16 And there’s been, most recently,
0:10:17 one thing I’m pretty excited about
0:10:21 is kind of the intersection or combination of the two,
0:10:24 so that that’s what we’ve been very focused on
0:10:28 but Waymo most recently is taking kind of the AI backbone
0:10:32 and all of the AI, the Waymo AI that is over the years
0:10:35 we’ve built up that is really proficient
0:10:37 at this task of autonomous driving
0:10:41 and combining it with kind of the general world knowledge
0:10:44 and understanding of these, you know, VLMs.
0:10:45 – One of the things that you just mentioned
0:10:50 is the role of simulation and how that has been,
0:10:51 you guys have had major breakthroughs
0:10:54 in the use of simulation.
0:10:56 And this idea in, you know,
0:10:59 the recent breakthroughs in generative AI
0:11:02 around synthetic data and its usefulness
0:11:04 is somewhat in question.
0:11:06 I would say in your field,
0:11:08 this idea of synthetic data and simulation
0:11:11 is extremely useful and you’ve proven that.
0:11:12 So maybe you could just talk about
0:11:15 the simulation technology you guys have built,
0:11:16 how it’s allowed you to scale,
0:11:20 you know, build that real world understanding,
0:11:24 you know, and maybe how it’s changed in the last few years.
0:11:25 – Yeah, yeah, definitely.
0:11:27 It is super important in our field.
0:11:32 I mean, largely, if you think about this question
0:11:36 of evaluating the driver, like, you know, is it good enough?
0:11:38 It’s, you know, how do you answer that?
0:11:39 There’s, you know, a lot of metrics
0:11:43 and a lot of, you know, data sets you have to build up.
0:11:46 And then, you know, you,
0:11:49 how do you evaluate the latest version of your system?
0:11:51 You can’t just, you know, throw it on the physical world
0:11:54 and then, you know, see what happens.
0:11:55 You have to do a simulation.
0:11:59 But of course, the new system behaves differently
0:12:03 from what might have happened in the world otherwise.
0:12:05 So you have to have a realistic closed loop simulation
0:12:08 to give you, you know, confidence and value.
0:12:09 So that is one of the most important needs
0:12:10 for the simulation.
0:12:12 You’ve also mentioned synthetic data,
0:12:15 as that’s another area where simulation allows you
0:12:17 to have very high leverage.
0:12:20 And you just got to explore the long tail of that, right?
0:12:21 Maybe there’s something interesting
0:12:23 that you have seen in the physical world.
0:12:25 And, but, you know, you want to modify that scenario
0:12:28 and you want to kind of turn one event into thousands
0:12:30 or tens of thousands of variations of that scenario.
0:12:31 You know, how do you do that?
0:12:33 You know, this is where the simulation comes in.
0:12:34 And then, you know, lastly,
0:12:39 if you, you know, sometimes want to evaluate
0:12:43 and train on things that you’ve never seen.
0:12:47 You and I are very vast experience.
0:12:49 So this is where purely synthetic simulations come in
0:12:51 that are not based on anything that you have seen
0:12:53 in the physical world.
0:12:55 So in terms of technologies that go into play,
0:12:58 I mean, it’s a lot.
0:13:01 And that is like a huge generative AI problem.
0:13:04 But what’s really important is that that simulator
0:13:07 is realistic, right?
0:13:10 It has to be realistic in terms of your, you know,
0:13:12 sensor or perception realism, right?
0:13:17 Because it has to be realistic in terms of the behaviors
0:13:20 that you see from other dynamic actors, right?
0:13:23 You have, you know, if there are other actors
0:13:24 that are not behaving in an realistic way,
0:13:26 like if, you know, pedestrians are not walking
0:13:27 the way they do in the real world,
0:13:31 you need to be able to quantify the kind of the,
0:13:36 the scenarios that you create in simulation
0:13:39 to the realism and the rate of occurrence
0:13:40 in the physical world, right?
0:13:42 It’s, you know, very crazy to sample something
0:13:45 very, you know, easy to sample something totally crazy
0:13:47 in simulator, but then, you know, what do you do with that?
0:13:49 So I think that that brings me to the third point
0:13:52 of, you know, realism is that it has to be kind of realistic
0:13:54 and quantifiable at the macro level,
0:13:55 at the statistical level.
0:13:56 So there’s any, you can imagine,
0:13:58 there’s a lot of work that goes into building a simulator
0:14:01 that is, you know, large scale and has, you know,
0:14:03 that level of realism across those categories.
0:14:05 And if kind of intuitively you think about it, you know,
0:14:08 to build a good driver, you need to have a very good simulator,
0:14:09 but to have a good simulator,
0:14:11 you actually have to build models
0:14:13 of like realistic pedestrians and cyclists and drivers, right?
0:14:15 So it’s, you know, it kind of do that iteratively.
0:14:16 – Yeah, of course.
0:14:19 And then by having this simulation software
0:14:22 that is very good at mimicking real world
0:14:25 and very usable in the sense that you can
0:14:27 create variables in the scenes,
0:14:30 you can actually give the driver
0:14:32 multiples of the amount of experience
0:14:33 that they have on the road.
0:14:34 – That’s exactly right.
0:14:36 – In real miles, is that right?
0:14:36 – That’s exactly right.
0:14:40 We’ve driven, you know, tens of millions of miles
0:14:41 in the physical world.
0:14:44 And at this point we’ve driven more than 15 million miles
0:14:47 in full autonomy, we call it, you know, rider only mode.
0:14:48 But we’ve driven, you know,
0:14:49 tens of billions of miles of simulation.
0:14:52 So you get, you know, orders of magnitude of an amplifier.
0:14:56 – Speaking of multiples of miles driven,
0:15:01 one of the hotly debated topics in the AI world today
0:15:04 is this concept of scaling laws.
0:15:06 So how do you think about scaling laws
0:15:08 as it relates to autonomous driving?
0:15:09 Is it miles driven?
0:15:12 Is it certain experience had?
0:15:13 Is it compute?
0:15:15 Like, what are the ways that you think about that?
0:15:19 – So model size matters.
0:15:22 So we’re seeing, you know, scaling laws applied,
0:15:26 a lot of typical, you know,
0:15:29 old school models are severely under trained.
0:15:31 And so if you have a bigger model,
0:15:33 you have data that actually does help you.
0:15:36 You just have more capacity that generalize better.
0:15:39 So we are seeing the scaling laws apply there.
0:15:41 Data, of course, usually matters, right?
0:15:43 And, but it’s not just, you know,
0:15:46 counting the miles, right, or hours.
0:15:48 It has to be, you know,
0:15:50 the right kind of data that, you know,
0:15:53 teaches the models or trains the models to be, you know,
0:15:56 good at the rare cases that you care about.
0:15:58 And then, you know, there is a bit of a, you know,
0:16:00 a wrinkle ’cause then you have to,
0:16:02 you can build those very large models,
0:16:04 but in our space, it has to run on board the car, right?
0:16:07 So you are somewhat complicated, you have to distill it
0:16:10 into your, you know, onboard system.
0:16:13 But we do see trend, we just, you know,
0:16:15 common trend and we see that play out in our space
0:16:17 where you’re much better off training a huge model
0:16:19 and then distilling it into a small model
0:16:20 than just training small models.
0:16:22 – Yeah, I’m gonna shift gears a little bit
0:16:25 and I’m gonna do a sort of simplifying statement,
0:16:27 which is probably gonna drive you crazy.
0:16:32 But the DARPA School of Thought is, you know,
0:16:35 there’s sort of a rules-based approach, right?
0:16:39 A more traditional kind of AI-based approach
0:16:43 with a massive amount of volume and you document edge cases
0:16:46 and then the model then learns how to react to those.
0:16:51 The more recent approaches from some other large players
0:16:52 and startups would say,
0:16:54 hey, we just have AI from the start,
0:16:57 make all the decisions end to end.
0:17:00 You don’t need to have sort of all that pattern recognition
0:17:02 and learning, you know, like the end to end driving
0:17:04 that is kind of a tagline out there.
0:17:08 What is your interpretation of that approach
0:17:12 and what elements of that approach have you taken
0:17:14 and applied inside of Waymo?
0:17:16 – Yeah, you know, I think it’s kind of, you know,
0:17:19 sometimes it’s a, you know, the way people talk about it
0:17:23 is kind of this weird dichotomy is this or that.
0:17:24 – Yeah, of course.
0:17:27 – But it’s not, it’s that and then some, right?
0:17:30 So it is, you know, big models.
0:17:32 It is end to end models.
0:17:35 Yeah, it is a generative AI and combining, you know,
0:17:37 these models with VLMs, right?
0:17:41 But the problem is it’s not enough, right?
0:17:43 So I mean, like we all know the limitations
0:17:44 of those models, right?
0:17:47 And that’s, and we’ve seen, you know, through the years,
0:17:48 a lot of these breakthroughs in AI, right?
0:17:50 You know, Continuous Transformers,
0:17:52 you know, big end to end foundation models,
0:17:53 they’re huge boosts to us.
0:17:56 And you know, what we’ve been doing
0:17:58 at Waymo through the history of our project
0:18:03 is kind of constantly applying and pushing forward
0:18:04 these state-of-the-art techniques ourselves
0:18:06 in some cases, but then applying them to our domain.
0:18:08 And what we’ve been learning is that they really
0:18:11 give you a huge boost, but they’re just not enough, right?
0:18:14 So in the kind of, the theme has always been
0:18:16 that you can take, you know, your kind of latest
0:18:19 and greatest technology of the day
0:18:22 and it’s fairly easy to get started, right?
0:18:24 Like, you know, like the curves always look like that.
0:18:26 And they, like they’ve been kind of the curves
0:18:27 in their shaping, but the really hard problems
0:18:30 in that remaining point is 0.0001%.
0:18:32 And there it’s not enough, right?
0:18:34 So then you have to do stuff on top of that, right?
0:18:36 So yes, you can take, you know, nowadays,
0:18:38 you can take, you know, an end-to-end model,
0:18:41 go from sensor to, you know, trajectories or actuation.
0:18:42 You know, typically you don’t build them in one stage,
0:18:43 you build them in stages, but you know,
0:18:45 you can do, like, back prop through the whole thing.
0:18:48 So, you know, the concept is very, very valid.
0:18:50 You can, you know, combine it and, you know,
0:18:52 with a VLM and then, you know,
0:18:54 you add closed-loop simulations, some sort.
0:18:56 And, you know, you’re off to the races.
0:18:59 You can have a great demo, like, almost out of the box.
0:19:03 You can have, you know, an ADESP, or a driver assist system,
0:19:06 but that’s not enough to go all the way to full autonomy.
0:19:08 So that’s where really a lot of the hard work happens.
0:19:09 So I guess the question is, you know,
0:19:11 not is it this or that, it’s, you know, this,
0:19:14 and then what else do you need to take it all the way
0:19:16 to have the confidence in, you know,
0:19:18 so that you can actually remove the driver
0:19:19 and go for full autonomy.
0:19:20 And that’s a ton of work.
0:19:23 That’s a ton of work through the entire, kind of,
0:19:25 life cycle of these models and the entire system, right?
0:19:27 So it starts with training.
0:19:28 Like, how do you train?
0:19:29 How do you architect these models?
0:19:32 How do you, you know, evaluate them?
0:19:34 Then, you know, if you put in a bigger system,
0:19:35 the models themselves are not enough,
0:19:36 so you have to do things around them.
0:19:38 You have to, you know, they have,
0:19:40 modern genera of AI is great,
0:19:41 but there are some issues with, you know,
0:19:43 hallucinations, there are issues with, like,
0:19:45 – Explanability. – Exactly, exactly.
0:19:46 So, you know, they have some weaknesses
0:19:50 and kind of goal-oriented planning and policy-making
0:19:52 and kind of understanding this, you know,
0:19:55 3D space operating in this 3D spatial world, right?
0:19:57 So you have to add something on top of that.
0:19:58 We talked a little bit about the simulator.
0:20:00 That’s a really hard problem, you know, itself.
0:20:02 And then, you know, once you have something,
0:20:03 you know, once you deploy it
0:20:05 and you learn how do you feed that back.
0:20:07 So I guess this is where all of the really,
0:20:07 really hard work happens.
0:20:09 So it’s not, like, end-to-end versus something else.
0:20:12 It is end-to-end and, you know, big foundation models.
0:20:14 And then, like, and then the hard work.
0:20:15 – And then all the hard work.
0:20:17 Yeah, that totally makes sense.
0:20:19 That is a great segue into all of the progress
0:20:21 that you guys have made, right?
0:20:24 Writing in the Waymo for those who have done it
0:20:26 is an extraordinary experience.
0:20:28 It’s not to say that you have solved
0:20:29 all of these complex tasks,
0:20:31 but you’ve solved a lot of them.
0:20:36 What are some of the biggest AI or data problems
0:20:39 that you still feel like you’re facing today?
0:20:42 – The short answer is going to be, you know,
0:20:46 taking it to, you know, the next order of magnitude of scale.
0:20:47 Multiple orders of magnitude of scale.
0:20:49 And with that come, you know,
0:20:50 additional improvements that we need
0:20:53 to make it, you know, a great service, right?
0:20:57 But, you know, just to level-set in terms of
0:20:58 where we are today, you know,
0:21:03 we are, you know, driving in all kinds of conditions.
0:21:07 We’re driving, you know, 24/7 in San Francisco,
0:21:09 in Phoenix, you know, a little bit,
0:21:10 those are the most mature markets,
0:21:12 but also in LA and in Austin.
0:21:15 And, you know, all of the complexity that you see,
0:21:17 you know, go drive around the city, right?
0:21:18 All kinds of weather conditions,
0:21:21 whether it’s, you know, fog or, you know, storms
0:21:25 or dust storms or, you know, rainstorms down here,
0:21:27 like all of that, all of those are conditions
0:21:28 that we do operate in, right?
0:21:30 So then I think about, you know,
0:21:34 what makes it a great, you know, customer experience, right?
0:21:35 Like what does it take if you, you know,
0:21:39 grow by, you know, next, you know, orders of magnitude?
0:21:40 There’s a lot of improvements that we want to make
0:21:42 so that it becomes a better service for you
0:21:44 to get from point A to point B, right?
0:21:46 Like we asked for feedback from our writers.
0:21:49 A lot of feedback we get is, you know,
0:21:51 it has to do with the quality of your pickup
0:21:52 and drop-off locations, right?
0:21:53 So we’re learning from users.
0:21:55 Like no matter what, we want to make it a magical,
0:21:57 seamless, you know, delightful experience
0:21:59 from the time you kind of, you know,
0:22:00 start the app on your phone too,
0:22:01 when you get on the destination.
0:22:04 So that’s a lot of the work that we’re doing right now.
0:22:06 – Yeah, pick up and drop-off for what it’s worth
0:22:09 is an extraordinarily hard problem, right?
0:22:13 Like do you kind of block a little bit of a driveway
0:22:15 if you’re in an urban location
0:22:16 and then have a sensor that says,
0:22:19 oh, actually, I just saw somebody opening a garage door.
0:22:22 I need to get out of the way, you know,
0:22:24 how far down the street is acceptable to go pull.
0:22:25 Or if you’re in a parking lot,
0:22:27 where in the parking lot do you go?
0:22:29 Like this is an extraordinarily hard problem,
0:22:32 but to your point, it’s huge for user experience.
0:22:33 – That’s exactly right, right?
0:22:35 And just, you know, I think that’s a good example
0:22:36 of like just, hey, just one thing,
0:22:39 one of the many things that we have to build
0:22:41 in order for this to be an awesome product, right?
0:22:43 Not just like a technology demonstrator.
0:22:44 And I think you just like, you know,
0:22:49 hit exactly on a few things that make,
0:22:53 you know, something that kind of at the face of it
0:22:54 might seem fairly straightforward, right?
0:22:56 Okay, you know, I know there’s a place on the map
0:22:57 and I need to pull over.
0:22:59 It’s like, how hard can it be, right?
0:23:00 But really, if it’s a complicated, you know,
0:23:02 dense urban environment,
0:23:03 there’s a lot of these factors, right?
0:23:05 Is there like, you know, another vehicle
0:23:06 that you’re gonna be blocking?
0:23:08 Is there a garage door that’s opening, right?
0:23:10 Like, you know, what is the most convenient place
0:23:11 for the user to pick up?
0:23:14 What is, you know, so it really gets into this,
0:23:17 you know, the depth and the subtlety of understanding
0:23:21 the, you know, the semantics and the dynamic nature
0:23:23 of this driving task and, you know, doing things
0:23:25 that are, you know, safe, comfortable and predictable
0:23:28 and lead to a nice, seamless, pleasant,
0:23:30 delightful customer experience.
0:23:32 – Of course, okay, so you’ve mentioned this stat,
0:23:36 but 15 million miles, I know the number’s probably
0:23:39 a little bit bigger than that, but you released it Tuesday.
0:23:42 Yeah, it’s growing by the day.
0:23:46 15 million autonomous miles driven, that’s incredible.
0:23:49 Even more impressive, and you didn’t share this stat yet,
0:23:54 it results in 3.5 times fewer accidents
0:23:56 than human drivers, is that right?
0:23:59 – And I think 3.5 acts as the reduction in injury,
0:24:02 and it’s about 2x reduction in the police reportable
0:24:03 kind of lower severity incidents.
0:24:08 – This sort of comes to a question of both kind of
0:24:12 regulatory and, you know, business or ethical judgment.
0:24:15 What is the right level that you want to get to?
0:24:17 Obviously you want to constantly get better,
0:24:19 but is there a level at which you say,
0:24:21 “Okay, we’re good enough,” and that’s acceptable
0:24:22 to regulators?
0:24:24 – Yeah, so there’s no, you know,
0:24:28 simple, super simple short answer, right?
0:24:29 I think it starts with that.
0:24:31 It starts with those statistics that you just mentioned.
0:24:33 I can then have the day what I care about
0:24:35 is that roads are safer.
0:24:36 So when you look at those numbers,
0:24:38 yet, you know, when we operate today,
0:24:40 and we have, you know, strong empirical evidence
0:24:44 that our cars are in those areas safer than human drivers.
0:24:47 So on balance, that means a reduction in, you know,
0:24:49 collisions and harm.
0:24:54 Then, actually on top of the numbers
0:24:55 we’ve probably been publishing,
0:24:57 this is you’re quoting the latest numbers that we’ve shared.
0:24:58 – Yeah.
0:24:59 – Consistently, you know,
0:25:04 sharing numbers as our service scales up and grows.
0:25:06 You can also bring in, you know,
0:25:08 an additional lens of, you know,
0:25:11 how much did you contribute to a collision?
0:25:12 And we actually published, I think it was based
0:25:14 on about 4 million miles, 3.8 million miles.
0:25:17 We’ve published a joint study with Swiss ARRI,
0:25:20 which is, I think, the largest global reinsurer
0:25:20 in the world.
0:25:22 And the way they look at it is, you know,
0:25:24 who contributed to an event.
0:25:27 And there we saw like the same theme,
0:25:30 but the numbers were very strong.
0:25:35 That new field was a 76% reduction in property damage
0:25:39 collisions, and it was an 100% reduction
0:25:42 in claims around bodily injury.
0:25:43 So if you kind of bring in that lens,
0:25:45 I think the story becomes even more compelling.
0:25:46 – That is extremely compelling.
0:25:48 – Right, but there are some collisions where, you know,
0:25:51 we’d be, and that’s the bulk of the events that we see.
0:25:52 We’d be stopped at a red light,
0:25:54 and then somebody just plows into you, right?
0:25:55 – Sure.
0:25:58 – So, but then, like we, I think, you know,
0:26:01 we do know it’s a new technology, it’s a new product,
0:26:04 so it is held to a higher standard.
0:26:07 So we, when we think about our safety
0:26:09 and our readiness, you know, framing of methodology,
0:26:10 we don’t stop at just the race, right?
0:26:12 We build over the years as, you know,
0:26:16 one of the huge areas of investment
0:26:18 and experience over the years, like how, you know,
0:26:19 what else do you need?
0:26:20 So we have done, and we’ve done a number
0:26:21 of the other different things.
0:26:23 And we’ve published some of our methodologies,
0:26:25 we’ve shared our readiness framework, you know,
0:26:27 we do other things like we actually,
0:26:30 not just statistically, but on specific events,
0:26:34 we build models of an attentive, very good human driver,
0:26:36 like not distracted human, you know,
0:26:38 a good question whether such a driver exists, right?
0:26:41 But that’s kind of what we compare our driver to, right?
0:26:43 And it’s a model, like, it’s then it’s, you know,
0:26:45 in particular scenario, we evaluate ourselves
0:26:47 versus that model of a human driver,
0:26:49 and we hold ourselves to the bar of, you know,
0:26:51 doing well compared to that very high standard.
0:26:53 And then, you know, you pursue other, you know,
0:26:54 validation methodologies.
0:26:58 So that’s my answer is that it’s the, you know,
0:27:00 the aggregate of all of those methodologies
0:27:03 that we look at to decide that, yes, you know,
0:27:06 the system is ready enough to be deployed in scale.
0:27:09 – I’d love for you to talk about what you think,
0:27:10 maybe today and in the future,
0:27:13 about market structure, competition,
0:27:17 and what kind of role you envision Waymo playing.
0:27:20 – So the way we think about, you know,
0:27:22 Waymo and our company is that we are building
0:27:25 a generalize of both driver.
0:27:26 That’s the core of it.
0:27:28 And that’s the core of the mission
0:27:33 of making a transportation safe and accessible, right?
0:27:38 And we’re talking about right hailing today.
0:27:42 That’s our main, most mature primary application.
0:27:43 But, you know, we envision a future
0:27:46 where the Waymo driver will be deployed
0:27:47 in other commercial applications, right?
0:27:49 There’s deliveries, there’s trucking,
0:27:52 there’s personally owned vehicles, right?
0:27:54 So in all of those, you know,
0:27:57 our guiding principle would be to think
0:28:00 about the go-to-market strategy in a way
0:28:04 that accelerates access to this technology
0:28:08 and gets it deployed as, you know, broadly,
0:28:11 you know, of course, doing it gradually
0:28:12 and deliberately and safely, you know,
0:28:16 as quickly and broadly as possible.
0:28:18 So with that, as our guiding principle,
0:28:22 we’re gonna explore different commercial structures,
0:28:23 different partnership structures.
0:28:25 For example, in Phoenix today,
0:28:28 we have a partnership with Uber and Right Healing,
0:28:30 both in Uber Right Healing and in Uber Eats,
0:28:33 where, so in Phoenix, we have our own app.
0:28:35 You can download the Waymo app and, you know,
0:28:36 take a ride, an hour vehicle will show up
0:28:39 and take you where you wanna go.
0:28:41 That’s, you know, one way to experience our product.
0:28:43 Another one is through the Uber app.
0:28:45 We have a partnership where you can get through
0:28:48 the Uber app matched with our product,
0:28:49 the Waymo driver, the Waymo vehicle,
0:28:51 and it’s the same experience, right?
0:28:54 But this is another way for us to accelerate
0:28:56 and give more people to experience full autonomy.
0:28:59 And it gives us a chance to kind of, you know,
0:29:02 think about the different go-to-market strategies, right?
0:29:06 One is, you know, us having more of our own app.
0:29:07 The other one is more of a, you know,
0:29:09 driver-as-a-service for somebody else’s network.
0:29:11 So we’ll, you know, still early days,
0:29:13 but we will iterate and hold, you know,
0:29:15 in service of that main principle.
0:29:15 – That’s amazing.
0:29:18 Yeah, that’s gonna be exciting.
0:29:21 Maybe on back to the vehicle,
0:29:23 what about the hardware stack that you use?
0:29:24 You and I have talked a bunch about, you know,
0:29:28 you said like, hey, going all the way back to DARPA,
0:29:29 you know, it’s kind of the same stuff, right?
0:29:32 It’s, you know, it’s sensor, they’ve advanced
0:29:34 quite considerably, but, you know,
0:29:37 you still use, you know, radars and LiDAR.
0:29:41 Do you think that remains the future path
0:29:42 for autonomous driving?
0:29:43 LiDAR specifically?
0:29:48 – Yeah, no, I mean, the sensors are physically different,
0:29:53 right?
0:29:55 They have each one cameras, LiDARs, radar,
0:29:57 they have their, you know, benefits,
0:29:59 each one brings their own benefits, right?
0:30:00 You know, cameras obviously give you color
0:30:03 and they give you high, you know, very high resolution.
0:30:06 LiDARs kind of give you, you know,
0:30:09 a direct 3D measurement of your environment
0:30:11 and they’re an active sensor, right?
0:30:12 So it kind of brings their own energy,
0:30:15 pitch dark when there’s no, you know, external light source,
0:30:18 you know, you still get the seat just as well
0:30:20 as they do during the day, you know,
0:30:21 better in some cases.
0:30:24 And then, you know, radar is, you know,
0:30:26 very good at like, punching through just, you know,
0:30:28 different physics, different wavelengths, right?
0:30:32 If you build an imaging radar, which we do ourselves,
0:30:34 you know, it allows us to, you know,
0:30:37 give you an additional redundancy layer
0:30:39 and it has benefits, also an active sensor,
0:30:40 it can directly measure, you know,
0:30:42 through Doppler velocity of other objects
0:30:46 and it can, you know, degrades differently
0:30:48 and more gracefully in some other conditions.
0:30:50 Like, you know, very dense fog, you know,
0:30:52 or very dense rain.
0:30:53 So, you know, they’ll have their benefits.
0:30:58 So if you, you know, our approach has been to, you know,
0:31:00 use all of them, right?
0:31:02 And, you know, that’s how you have redundancy
0:31:04 and that’s how you get an extra boost
0:31:06 and capability of the system.
0:31:09 And, you know, we are on, you know,
0:31:11 today deployed in 5th and working to deploy
0:31:13 the sixth generation of our sensors.
0:31:15 And, you know, over those generations,
0:31:18 we’ve improved, you know, reliability,
0:31:20 we’ve improved, you know, capability and performance
0:31:23 and we’ve brought down the cost very significantly, right?
0:31:25 So, yeah, I think that the trend, you know,
0:31:27 for us that will, you know, using all three modalities
0:31:29 just makes a lot of sense.
0:31:31 Again, you know, you might make different trade-offs
0:31:33 if you are building a driver’s system
0:31:35 versus a fully autonomous vehicle where, you know,
0:31:38 that last 0.001% really, really matters.
0:31:39 – Yeah, absolutely.
0:31:44 One of the observations that we have
0:31:48 from the very early days of this wave of LLMs
0:31:53 is that there has been sort of already a massive
0:31:56 race of like cost reduction.
0:31:59 And many would argue that it’s sort of a process
0:32:02 of commoditization already, even though it’s very early days.
0:32:06 I would say the observation from autonomous driving
0:32:10 over many, many years now is kind of the opposite thing.
0:32:13 There’s been a thinning of the field, you know,
0:32:16 it’s proven to be much, much harder than expected.
0:32:19 Can you just talk about maybe why that’s the case?
0:32:21 – You know, they always have this property
0:32:22 that it’s very easy to get started,
0:32:25 but it’s very insanely difficult to get it, you know,
0:32:28 all the way, you know, to full autonomy
0:32:30 so that you can remove the driver.
0:32:34 And, you know, there’s maybe a few factors
0:32:36 that contribute to that.
0:32:40 One is, you know, compared to the LLMs and, you know,
0:32:43 it’s kind of AI in the digital world,
0:32:45 you have to operate in the physical world.
0:32:49 The physical world is messy, it is noisy,
0:32:50 and, you know, it can be quite humbling, right?
0:32:53 There’s all kinds of, you know, uncertainty and noise
0:32:57 that can kind of pull you out of distribution,
0:32:58 if you will, right? – Right, sure.
0:33:02 – So that’s one thing, that makes this very difficult.
0:33:07 And secondly, it’s safety, right?
0:33:09 – Sure.
0:33:12 – These, you know, AI systems, you know, in some domain,
0:33:16 you know, this is creativity, and it’s great.
0:33:18 You know, our domain, the cost of mistakes,
0:33:21 our lack of, you know, accuracy
0:33:22 has very serious consequences, right?
0:33:25 So that’s just the bar, very, very high.
0:33:27 Right, and then the last thing is that
0:33:30 it is, you know, you have to operate in real time.
0:33:33 You’re putting these systems on fast-moving vehicles,
0:33:35 and you have to, you know, milliseconds matter, right?
0:33:37 You have to make the decisions very quickly.
0:33:39 So I think it’s, you know, the combination of those factors
0:33:43 that really, you know, together lead to, you know,
0:33:45 the trend that you’ve been seeing is that,
0:33:47 like, you know, it’s an and, right?
0:33:48 You have to be excellent on this and this and this
0:33:49 and then, right?
0:33:50 It’s all of the bar.
0:33:52 The bar is very, very high for, you know,
0:33:54 every component of the system and how you put them together.
0:33:56 But, you know, there’s big advances,
0:33:57 and they, you know, boost you,
0:33:59 and they profile the system forward,
0:34:00 but there are no silver bullets, right?
0:34:03 And there’s no shortcuts if you’re talking about full autonomy.
0:34:06 And because of that lack of tolerance for errors,
0:34:08 you have a very high bar for safety.
0:34:12 You have a very high burden from regulators.
0:34:16 You know, it’s very costly to go through all those processes.
0:34:17 And so it makes sense.
0:34:20 And I’m very grateful that you guys have seen it through,
0:34:23 despite all the humbling experiences
0:34:25 that you had along the way.
0:34:29 It’s been a long journey, but it’s, you know,
0:34:31 for me and the many people at Weymo,
0:34:36 it is super exciting and very, very rewarding
0:34:38 to finally see it become reality.
0:34:41 Now, we talk about safety and AI in many contexts, right?
0:34:42 That’s a big question, right?
0:34:44 But, you know, here we are in this application of AI
0:34:46 in the physical world.
0:34:48 We have, you know, at this point a pretty robust
0:34:50 and increasing body of evidence that, you know,
0:34:53 we are seeing, like, tangible safety benefits.
0:34:54 So that’s very exciting.
0:34:56 Yeah, I always say to people,
0:34:58 it was a long journey
0:35:01 and very costly and expensive along the way.
0:35:04 But this is probably the most powerful manifestation of AI
0:35:08 that we have available to us in the world today.
0:35:09 I mean, you can get in a car without a driver,
0:35:11 and it’s safer than having a human.
0:35:12 And that’s just remarkable.
0:35:16 What were some of those humbling events along the way?
0:35:16 And those are early days?
0:35:17 Those are the first couple of years?
0:35:19 Early days.
0:35:24 Oh, I’m sorry, I remember one, there was one route.
0:35:29 That we did, that started, I think it started in Montmaville,
0:35:30 then went through Palo Alto,
0:35:33 then went, you know, through the mountains to Highway 1.
0:35:35 That took Highway 1 to San Francisco.
0:35:38 And I think, you know, went around the city a little bit
0:35:40 and, like, actually finished for Lombard Street.
0:35:42 So, like, in 2009, 10 people.
0:35:43 That is really complicated.
0:35:45 100 miles to the beginning to end, right?
0:35:47 I mean, as human drivers would fail at that task,
0:35:48 I think, so, yeah, I keep…
0:35:49 Yeah, yeah.
0:35:51 So, you know, we’re doing it one day,
0:35:52 and then we’re driving and kind of made it through
0:35:54 the Montmaville-Palo Alto part.
0:35:55 We’re driving through the mountains
0:35:57 and it’s foggy, it’s early morning.
0:36:00 And then we’re, like, seeing objects.
0:36:03 And, you know, our objects seem like random stuff
0:36:04 on the road in front of us.
0:36:06 There’s, like, a bucket and, like, a shoe.
0:36:08 And then there’s, like, at some point,
0:36:10 we come across, like, a, you know, a rusty bicycle.
0:36:12 Like, okay, what’s going on there?
0:36:13 And then we catch, you know, eventually.
0:36:16 And, like, the card, you know, doesn’t, you know,
0:36:17 handles it okay.
0:36:18 You know, maybe not super smoothly,
0:36:21 but, you know, we get stuck and we catch up to, like,
0:36:24 this dump truck that has all kind of stuff on it.
0:36:26 And, you know, periodically losing things
0:36:28 that person obstacles to the car.
0:36:30 This is, like, a cartoon, you know,
0:36:33 continuation of anomalies being thrown at you guys.
0:36:36 That’s pretty cool.
0:36:38 Okay, last question.
0:36:40 I’m going to tee you up to do some recruiting, probably.
0:36:45 But if you were in the shoes of the audience here
0:36:50 and just kind of seeking your first job,
0:36:52 I’m going to take something that you said, which is, like,
0:36:54 I can see your passion and excitement
0:36:56 for doing the start-up thing, right?
0:36:59 And, like, you know, kind of longing back for those days
0:37:01 is so cool.
0:37:04 What advice would you have for these folks
0:37:08 in where to go, whether it’s type of company,
0:37:13 type of role, industry, or anything else?
0:37:14 Way more?
0:37:16 That’s what I’m saying.
0:37:18 It’s the easiest to tee you right up.
0:37:20 You know, it’s a fine —
0:37:22 I mean, we’re talking about AI today,
0:37:24 but it’s a fine problem that matters.
0:37:26 You know, problem that matters to the world,
0:37:28 problem that matters to you.
0:37:31 Chances are it’s going to be a hard one.
0:37:33 Yeah.
0:37:37 Many things, you know, we’re doing have that property.
0:37:41 So don’t get discouraged by, you know,
0:37:43 the unknown by what others might tell you.
0:37:46 And, you know, start building.
0:37:49 And then, you know, keep building and don’t look back.
0:37:52 Huge congratulations on all the progress you guys have made.
0:37:56 And as a very happy customer, thank you for building it.
0:37:59 And we really appreciate you being here.
0:38:03 All right, that is all for today.
0:38:06 If you did make it this far, first of all, thank you.
0:38:08 We put a lot of thought into each of these episodes,
0:38:10 whether it’s guests, the calendar touchers,
0:38:12 the cycles with our amazing editor Tommy
0:38:14 until the music is just right.
0:38:16 So if you’d like what we put together,
0:38:20 consider dropping us a line at ratethispodcast.com/a16z.
0:38:23 And let us know what your favorite episode is.
0:38:26 It’ll make my day, and I’m sure Tommy’s too.
0:38:28 We’ll catch you on the flip side.
0:38:31 (upbeat music)
0:38:34 (crunching)
0:38:36 (upbeat music)
0:00:02 – Hello, everyone.
0:00:04 Welcome back to the A16Z podcast.
0:00:05 This is Stuff.
0:00:07 Now, one of my favorite podcasts we’ve recorded
0:00:09 since I joined the team
0:00:11 was just about this time last year.
0:00:13 That episode was on autonomous vehicles,
0:00:17 but it was actually also in an autonomous vehicle.
0:00:20 That was my first ride in a self-driving car.
0:00:21 And over the last year,
0:00:23 I’ve seen so many others have their first
0:00:25 as Waymo has expanded to the public
0:00:27 in Phoenix and San Francisco,
0:00:30 while also placing its roots in Austin and LA.
0:00:32 In 2015, Waymo tested
0:00:35 its first fully driverless ride on public roads.
0:00:38 It then opened to the public in Phoenix in 2020,
0:00:40 but it wasn’t until 2022
0:00:43 that autonomous drives were offered in San Francisco.
0:00:45 And by the end of 2023,
0:00:48 it clocked in over 7 million driverless miles.
0:00:51 Slowly, then all at once.
0:00:53 So with this space moving so quickly,
0:00:54 we wanted to give you an update
0:00:57 on where this industry is today.
0:01:00 Passing the baton to properly introduce this episode,
0:01:02 here is our very own AI Revolution host
0:01:06 and A16Z General Partner, Sarah Wang.
0:01:08 As a reminder,
0:01:11 the content here is for informational purposes only.
0:01:13 Should not be taken as legal, business, tax
0:01:14 or investment advice,
0:01:16 or be used to evaluate any investment or security
0:01:18 and is not directed at any investors
0:01:21 or potential investors in any A16Z fund.
0:01:23 Please note that A16Z and its affiliates
0:01:24 may also maintain investments
0:01:27 in the companies discussed in this podcast.
0:01:28 For more details,
0:01:29 including a link to our investments,
0:01:33 please see A16Z.com/disclosures.
0:01:35 (upbeat music)
0:01:39 – Hey guys, I’m Sarah Wang,
0:01:42 General Partner on the A16Z growth team.
0:01:45 Welcome back to our AI Revolution series.
0:01:46 In this series,
0:01:48 we talk to the Gen AI builders
0:01:50 who are transforming our world to understand,
0:01:53 one, where we are, two, where we’re going
0:01:56 and three, the big open questions in the field.
0:01:59 Our guest this episode is Dmitry Dolgov,
0:02:01 the co-CEO of Waymo.
0:02:04 Dmitry has led Waymo to solve some of the biggest challenges
0:02:06 and bringing AI to the real world.
0:02:09 And after tens of millions of miles of testing,
0:02:11 Waymo’s vehicles have shown themselves
0:02:14 to be safer and more reliable than human drivers,
0:02:16 myself included.
0:02:18 Dmitry has a unique perspective,
0:02:20 given that his work has spanned multiple AI ML development
0:02:22 cycles across decades.
0:02:25 He was an early pioneer in self-driving cars,
0:02:28 working with Toyota and Stanford on DARPA’s grand challenge
0:02:30 before joining Google’s self-driving car project,
0:02:33 which then evolved into Waymo.
0:02:35 In this conversation from a closed door event
0:02:38 with A16Z General Partner, David George,
0:02:41 Dmitry talks about the potential of embodied AI,
0:02:43 the value of simulations and building training data,
0:02:46 and his approach to leading a company focused on solving
0:02:49 some of the world’s hardest problems.
0:02:50 Without further ado,
0:02:53 here’s Dmitry in conversation with David.
0:02:56 (dramatic music)
0:03:05 – Maybe to start, take us back to Stanford, if you will.
0:03:07 And that was when you first started working
0:03:09 on the DARPA project.
0:03:12 And maybe give us a little bit of your history
0:03:15 of how you ended up from there to here.
0:03:18 – My introduction to autonomous vehicles
0:03:21 was when I was doing a postdoc at Stanford,
0:03:23 that you just mentioned, David.
0:03:28 This was during, I got pretty lucky with the timing of it.
0:03:30 This was when the DARPA grand challenges were happening.
0:03:33 DARPA is the Defense Advanced Research Project Agency
0:03:35 that started these competitions with the goal
0:03:39 of boosting this field of autonomous vehicles.
0:03:44 And the one that I got involved in was in 2007,
0:03:46 that was called the DARPA Urban Challenge.
0:03:50 So the setup there was, it’s kind of like a toy version
0:03:53 of what we’ve been working on since then.
0:03:55 It was kind of supposed to mimic the driving
0:03:56 in urban environments.
0:04:00 So they kind of created a fake city on an abandoned airbase
0:04:03 and they populated it with a bunch of autonomous vehicles,
0:04:04 a bunch of human drivers,
0:04:07 and they had them do various tasks.
0:04:12 So that was kind of my introduction to this whole field.
0:04:14 And it was a bit of a, I think in a DARPA,
0:04:16 these challenges are often by people in the industry
0:04:19 considered kind of a foundational pivotal moment
0:04:22 for this whole field.
0:04:23 And it was definitely that for me.
0:04:27 It was like a light bulb, light switch moment
0:04:29 that really got me hooked.
0:04:31 – What was like the hardware and software
0:04:33 that you guys had at that point?
0:04:35 Is this 2007?
0:04:39 – Yeah, I know it’s a, at a very high level,
0:04:41 not unlike what we talk about today.
0:04:43 A car that has some instrumentation
0:04:45 so you can tell it what to do
0:04:47 and you get some feedback back.
0:04:50 Then you have kind of what’s called a post system,
0:04:54 a bunch of inertial measurement system accelerometers,
0:04:55 gyroscopes that kind of tell you in GPS,
0:04:57 tells you how you’re moving through space.
0:05:00 And it has sensors, radars, lighters and cameras,
0:05:02 those same stuff we still use today.
0:05:06 And then there’s a computer that gets the sensor data in
0:05:08 and then tells the car what to do.
0:05:10 And a bunch of software and software head,
0:05:13 perception components and decision-making planning
0:05:15 components and some AI.
0:05:17 But of course everything that we had,
0:05:20 like each one of those things over that,
0:05:21 how long has it been?
0:05:23 Almost 18 years, more than that.
0:05:24 It’s changed drastically, right?
0:05:26 So when we talk about AI today versus AI,
0:05:30 we had back in 2007, 2009, nothing in common.
0:05:31 And similarly, everything else has changed.
0:05:34 The sensors are not the same, computers are not the same.
0:05:35 – Yeah, of course.
0:05:37 So then, okay, so take us, so at that point,
0:05:40 that was the pivotal, that was like the light bulb moment.
0:05:43 And then at that point, you said, okay, I’m at Stanford,
0:05:45 I wanna make this my career, right?
0:05:47 Is that, and then it was Toyota,
0:05:49 and then where did it go from there?
0:05:51 – I don’t know if I thought about it in those terms.
0:05:55 I was like, this is the future, I wanna make it happen.
0:05:56 I wanna be building this thing, career.
0:05:58 Okay, you know, they can wait.
0:06:00 But it was, that was the next step.
0:06:03 That was the next big step is a number of us
0:06:06 from the DARPA Challenge competitions
0:06:09 started the Google self-driving project.
0:06:12 It was about a dozen of us, and then in 2009,
0:06:14 came together at Google with support,
0:06:16 an assignment from Larry and Sergey,
0:06:19 to see if we can take it to the next step.
0:06:24 And that then, we worked on it for a few years,
0:06:26 and that project then became Waymo in 2016,
0:06:29 and we’ve been on this path since then.
0:06:31 – Okay, so we have this new big breakthrough
0:06:33 in generative AI.
0:06:34 Some would say it’s new,
0:06:36 some would say it’s 70 years in the making.
0:06:40 How do you think about layering advances
0:06:43 that have come from generative AI
0:06:46 to what many would describe as more traditional AI
0:06:48 or machine learning techniques
0:06:50 that were kind of the building blocks
0:06:53 for self-driving technology up to that point?
0:06:54 – Oh yeah, great question.
0:06:56 So maybe generative AI is kind of a broad term.
0:06:59 So maybe you can take a little bit of a step back
0:07:02 and talk about the role that AI plays
0:07:05 in autonomous vehicles and kind of how we saw
0:07:07 the various breakthroughs in AI
0:07:09 map to the space of our task, right?
0:07:11 So like I mentioned,
0:07:16 AI has been part of self-driving autonomous vehicles
0:07:18 from the earliest days.
0:07:20 Back when we started, it was a very different kind of AI,
0:07:22 ML, kind of classical maintenance,
0:07:24 decision trees, classical computer visions
0:07:27 with kind of hand engineered features,
0:07:28 kernels and so forth.
0:07:34 And then one of the,
0:07:37 first really important breakthroughs
0:07:41 that happened in AI and computer vision
0:07:44 but really was important for our task
0:07:48 was the advancement in convolutional neural networks
0:07:51 right around 2012, right?
0:07:54 Many of you are probably familiar with AlexNet
0:07:55 and the ImageNet competition.
0:07:58 This is where AlexNet kind of blew away
0:08:01 out of the water all other approaches.
0:08:04 So that obviously has had very strong implications
0:08:06 for our domain, like how you do computer vision
0:08:07 and not just on cameras, right?
0:08:11 How you can use ConvNets to interpret what’s around you
0:08:13 and do kind of object detection and classification
0:08:14 from camera data, from LiDAR data,
0:08:16 from your imaging radars.
0:08:18 So that was kind of a big boost
0:08:22 around that 2012, 2013 timeframe.
0:08:24 And then we played with those approaches
0:08:27 and tried to extend the use of ConvNets to other domains,
0:08:30 just beyond perception with some interesting
0:08:32 but limited success.
0:08:36 Then another big, very important breakthrough
0:08:41 happened around 2017 when Transformers came around.
0:08:43 It had a really huge impact on language,
0:08:45 language understanding, language models,
0:08:48 machine translation, so forth.
0:08:52 And for us, it was a really important breakthrough
0:08:54 that really allowed us to take a mel in AI
0:08:58 to new areas well beyond perception.
0:09:00 And so if you think about, you know,
0:09:03 Transformers and the impact that they had on language,
0:09:06 ConvNets, the intuition is that they’re good at understanding
0:09:10 and predicting and generating sequences of words, right?
0:09:14 And in our case, we think about, in our domain,
0:09:17 about the tasks of understanding and predicting
0:09:20 what people will do, like other actors in the scene,
0:09:22 or the task of decision-making
0:09:23 and planning your own trajectories
0:09:26 or in simulation, generating generative AI,
0:09:29 or our version of generating behaviors
0:09:31 of how the world will evolve.
0:09:34 That kind of these behavioral,
0:09:37 like these sequences are not unlike sentences, right?
0:09:39 You’re kind of operating the state of objects, right?
0:09:40 And then there’s kind of local continuity,
0:09:42 but then the global context of the scene really matters.
0:09:44 So this is where we saw some really exciting breakthroughs
0:09:48 in behavior prediction and decision-making and simulation.
0:09:50 And then, you know, since then we’ve been on this trend
0:09:52 of, you know, models getting bigger.
0:09:56 People started building foundation models
0:09:57 for multi-tasks.
0:10:00 And most recently, all of the,
0:10:01 can I use the last couple of years,
0:10:04 all the breakthroughs in large language models.
0:10:08 You know, modern state, modern-day generative AI,
0:10:11 visual language models where you kind of align
0:10:13 image understanding and language understanding.
0:10:16 And there’s been, most recently,
0:10:17 one thing I’m pretty excited about
0:10:21 is kind of the intersection or combination of the two,
0:10:24 so that that’s what we’ve been very focused on
0:10:28 but Waymo most recently is taking kind of the AI backbone
0:10:32 and all of the AI, the Waymo AI that is over the years
0:10:35 we’ve built up that is really proficient
0:10:37 at this task of autonomous driving
0:10:41 and combining it with kind of the general world knowledge
0:10:44 and understanding of these, you know, VLMs.
0:10:45 – One of the things that you just mentioned
0:10:50 is the role of simulation and how that has been,
0:10:51 you guys have had major breakthroughs
0:10:54 in the use of simulation.
0:10:56 And this idea in, you know,
0:10:59 the recent breakthroughs in generative AI
0:11:02 around synthetic data and its usefulness
0:11:04 is somewhat in question.
0:11:06 I would say in your field,
0:11:08 this idea of synthetic data and simulation
0:11:11 is extremely useful and you’ve proven that.
0:11:12 So maybe you could just talk about
0:11:15 the simulation technology you guys have built,
0:11:16 how it’s allowed you to scale,
0:11:20 you know, build that real world understanding,
0:11:24 you know, and maybe how it’s changed in the last few years.
0:11:25 – Yeah, yeah, definitely.
0:11:27 It is super important in our field.
0:11:32 I mean, largely, if you think about this question
0:11:36 of evaluating the driver, like, you know, is it good enough?
0:11:38 It’s, you know, how do you answer that?
0:11:39 There’s, you know, a lot of metrics
0:11:43 and a lot of, you know, data sets you have to build up.
0:11:46 And then, you know, you,
0:11:49 how do you evaluate the latest version of your system?
0:11:51 You can’t just, you know, throw it on the physical world
0:11:54 and then, you know, see what happens.
0:11:55 You have to do a simulation.
0:11:59 But of course, the new system behaves differently
0:12:03 from what might have happened in the world otherwise.
0:12:05 So you have to have a realistic closed loop simulation
0:12:08 to give you, you know, confidence and value.
0:12:09 So that is one of the most important needs
0:12:10 for the simulation.
0:12:12 You’ve also mentioned synthetic data,
0:12:15 as that’s another area where simulation allows you
0:12:17 to have very high leverage.
0:12:20 And you just got to explore the long tail of that, right?
0:12:21 Maybe there’s something interesting
0:12:23 that you have seen in the physical world.
0:12:25 And, but, you know, you want to modify that scenario
0:12:28 and you want to kind of turn one event into thousands
0:12:30 or tens of thousands of variations of that scenario.
0:12:31 You know, how do you do that?
0:12:33 You know, this is where the simulation comes in.
0:12:34 And then, you know, lastly,
0:12:39 if you, you know, sometimes want to evaluate
0:12:43 and train on things that you’ve never seen.
0:12:47 You and I are very vast experience.
0:12:49 So this is where purely synthetic simulations come in
0:12:51 that are not based on anything that you have seen
0:12:53 in the physical world.
0:12:55 So in terms of technologies that go into play,
0:12:58 I mean, it’s a lot.
0:13:01 And that is like a huge generative AI problem.
0:13:04 But what’s really important is that that simulator
0:13:07 is realistic, right?
0:13:10 It has to be realistic in terms of your, you know,
0:13:12 sensor or perception realism, right?
0:13:17 Because it has to be realistic in terms of the behaviors
0:13:20 that you see from other dynamic actors, right?
0:13:23 You have, you know, if there are other actors
0:13:24 that are not behaving in an realistic way,
0:13:26 like if, you know, pedestrians are not walking
0:13:27 the way they do in the real world,
0:13:31 you need to be able to quantify the kind of the,
0:13:36 the scenarios that you create in simulation
0:13:39 to the realism and the rate of occurrence
0:13:40 in the physical world, right?
0:13:42 It’s, you know, very crazy to sample something
0:13:45 very, you know, easy to sample something totally crazy
0:13:47 in simulator, but then, you know, what do you do with that?
0:13:49 So I think that that brings me to the third point
0:13:52 of, you know, realism is that it has to be kind of realistic
0:13:54 and quantifiable at the macro level,
0:13:55 at the statistical level.
0:13:56 So there’s any, you can imagine,
0:13:58 there’s a lot of work that goes into building a simulator
0:14:01 that is, you know, large scale and has, you know,
0:14:03 that level of realism across those categories.
0:14:05 And if kind of intuitively you think about it, you know,
0:14:08 to build a good driver, you need to have a very good simulator,
0:14:09 but to have a good simulator,
0:14:11 you actually have to build models
0:14:13 of like realistic pedestrians and cyclists and drivers, right?
0:14:15 So it’s, you know, it kind of do that iteratively.
0:14:16 – Yeah, of course.
0:14:19 And then by having this simulation software
0:14:22 that is very good at mimicking real world
0:14:25 and very usable in the sense that you can
0:14:27 create variables in the scenes,
0:14:30 you can actually give the driver
0:14:32 multiples of the amount of experience
0:14:33 that they have on the road.
0:14:34 – That’s exactly right.
0:14:36 – In real miles, is that right?
0:14:36 – That’s exactly right.
0:14:40 We’ve driven, you know, tens of millions of miles
0:14:41 in the physical world.
0:14:44 And at this point we’ve driven more than 15 million miles
0:14:47 in full autonomy, we call it, you know, rider only mode.
0:14:48 But we’ve driven, you know,
0:14:49 tens of billions of miles of simulation.
0:14:52 So you get, you know, orders of magnitude of an amplifier.
0:14:56 – Speaking of multiples of miles driven,
0:15:01 one of the hotly debated topics in the AI world today
0:15:04 is this concept of scaling laws.
0:15:06 So how do you think about scaling laws
0:15:08 as it relates to autonomous driving?
0:15:09 Is it miles driven?
0:15:12 Is it certain experience had?
0:15:13 Is it compute?
0:15:15 Like, what are the ways that you think about that?
0:15:19 – So model size matters.
0:15:22 So we’re seeing, you know, scaling laws applied,
0:15:26 a lot of typical, you know,
0:15:29 old school models are severely under trained.
0:15:31 And so if you have a bigger model,
0:15:33 you have data that actually does help you.
0:15:36 You just have more capacity that generalize better.
0:15:39 So we are seeing the scaling laws apply there.
0:15:41 Data, of course, usually matters, right?
0:15:43 And, but it’s not just, you know,
0:15:46 counting the miles, right, or hours.
0:15:48 It has to be, you know,
0:15:50 the right kind of data that, you know,
0:15:53 teaches the models or trains the models to be, you know,
0:15:56 good at the rare cases that you care about.
0:15:58 And then, you know, there is a bit of a, you know,
0:16:00 a wrinkle ’cause then you have to,
0:16:02 you can build those very large models,
0:16:04 but in our space, it has to run on board the car, right?
0:16:07 So you are somewhat complicated, you have to distill it
0:16:10 into your, you know, onboard system.
0:16:13 But we do see trend, we just, you know,
0:16:15 common trend and we see that play out in our space
0:16:17 where you’re much better off training a huge model
0:16:19 and then distilling it into a small model
0:16:20 than just training small models.
0:16:22 – Yeah, I’m gonna shift gears a little bit
0:16:25 and I’m gonna do a sort of simplifying statement,
0:16:27 which is probably gonna drive you crazy.
0:16:32 But the DARPA School of Thought is, you know,
0:16:35 there’s sort of a rules-based approach, right?
0:16:39 A more traditional kind of AI-based approach
0:16:43 with a massive amount of volume and you document edge cases
0:16:46 and then the model then learns how to react to those.
0:16:51 The more recent approaches from some other large players
0:16:52 and startups would say,
0:16:54 hey, we just have AI from the start,
0:16:57 make all the decisions end to end.
0:17:00 You don’t need to have sort of all that pattern recognition
0:17:02 and learning, you know, like the end to end driving
0:17:04 that is kind of a tagline out there.
0:17:08 What is your interpretation of that approach
0:17:12 and what elements of that approach have you taken
0:17:14 and applied inside of Waymo?
0:17:16 – Yeah, you know, I think it’s kind of, you know,
0:17:19 sometimes it’s a, you know, the way people talk about it
0:17:23 is kind of this weird dichotomy is this or that.
0:17:24 – Yeah, of course.
0:17:27 – But it’s not, it’s that and then some, right?
0:17:30 So it is, you know, big models.
0:17:32 It is end to end models.
0:17:35 Yeah, it is a generative AI and combining, you know,
0:17:37 these models with VLMs, right?
0:17:41 But the problem is it’s not enough, right?
0:17:43 So I mean, like we all know the limitations
0:17:44 of those models, right?
0:17:47 And that’s, and we’ve seen, you know, through the years,
0:17:48 a lot of these breakthroughs in AI, right?
0:17:50 You know, Continuous Transformers,
0:17:52 you know, big end to end foundation models,
0:17:53 they’re huge boosts to us.
0:17:56 And you know, what we’ve been doing
0:17:58 at Waymo through the history of our project
0:18:03 is kind of constantly applying and pushing forward
0:18:04 these state-of-the-art techniques ourselves
0:18:06 in some cases, but then applying them to our domain.
0:18:08 And what we’ve been learning is that they really
0:18:11 give you a huge boost, but they’re just not enough, right?
0:18:14 So in the kind of, the theme has always been
0:18:16 that you can take, you know, your kind of latest
0:18:19 and greatest technology of the day
0:18:22 and it’s fairly easy to get started, right?
0:18:24 Like, you know, like the curves always look like that.
0:18:26 And they, like they’ve been kind of the curves
0:18:27 in their shaping, but the really hard problems
0:18:30 in that remaining point is 0.0001%.
0:18:32 And there it’s not enough, right?
0:18:34 So then you have to do stuff on top of that, right?
0:18:36 So yes, you can take, you know, nowadays,
0:18:38 you can take, you know, an end-to-end model,
0:18:41 go from sensor to, you know, trajectories or actuation.
0:18:42 You know, typically you don’t build them in one stage,
0:18:43 you build them in stages, but you know,
0:18:45 you can do, like, back prop through the whole thing.
0:18:48 So, you know, the concept is very, very valid.
0:18:50 You can, you know, combine it and, you know,
0:18:52 with a VLM and then, you know,
0:18:54 you add closed-loop simulations, some sort.
0:18:56 And, you know, you’re off to the races.
0:18:59 You can have a great demo, like, almost out of the box.
0:19:03 You can have, you know, an ADESP, or a driver assist system,
0:19:06 but that’s not enough to go all the way to full autonomy.
0:19:08 So that’s where really a lot of the hard work happens.
0:19:09 So I guess the question is, you know,
0:19:11 not is it this or that, it’s, you know, this,
0:19:14 and then what else do you need to take it all the way
0:19:16 to have the confidence in, you know,
0:19:18 so that you can actually remove the driver
0:19:19 and go for full autonomy.
0:19:20 And that’s a ton of work.
0:19:23 That’s a ton of work through the entire, kind of,
0:19:25 life cycle of these models and the entire system, right?
0:19:27 So it starts with training.
0:19:28 Like, how do you train?
0:19:29 How do you architect these models?
0:19:32 How do you, you know, evaluate them?
0:19:34 Then, you know, if you put in a bigger system,
0:19:35 the models themselves are not enough,
0:19:36 so you have to do things around them.
0:19:38 You have to, you know, they have,
0:19:40 modern genera of AI is great,
0:19:41 but there are some issues with, you know,
0:19:43 hallucinations, there are issues with, like,
0:19:45 – Explanability. – Exactly, exactly.
0:19:46 So, you know, they have some weaknesses
0:19:50 and kind of goal-oriented planning and policy-making
0:19:52 and kind of understanding this, you know,
0:19:55 3D space operating in this 3D spatial world, right?
0:19:57 So you have to add something on top of that.
0:19:58 We talked a little bit about the simulator.
0:20:00 That’s a really hard problem, you know, itself.
0:20:02 And then, you know, once you have something,
0:20:03 you know, once you deploy it
0:20:05 and you learn how do you feed that back.
0:20:07 So I guess this is where all of the really,
0:20:07 really hard work happens.
0:20:09 So it’s not, like, end-to-end versus something else.
0:20:12 It is end-to-end and, you know, big foundation models.
0:20:14 And then, like, and then the hard work.
0:20:15 – And then all the hard work.
0:20:17 Yeah, that totally makes sense.
0:20:19 That is a great segue into all of the progress
0:20:21 that you guys have made, right?
0:20:24 Writing in the Waymo for those who have done it
0:20:26 is an extraordinary experience.
0:20:28 It’s not to say that you have solved
0:20:29 all of these complex tasks,
0:20:31 but you’ve solved a lot of them.
0:20:36 What are some of the biggest AI or data problems
0:20:39 that you still feel like you’re facing today?
0:20:42 – The short answer is going to be, you know,
0:20:46 taking it to, you know, the next order of magnitude of scale.
0:20:47 Multiple orders of magnitude of scale.
0:20:49 And with that come, you know,
0:20:50 additional improvements that we need
0:20:53 to make it, you know, a great service, right?
0:20:57 But, you know, just to level-set in terms of
0:20:58 where we are today, you know,
0:21:03 we are, you know, driving in all kinds of conditions.
0:21:07 We’re driving, you know, 24/7 in San Francisco,
0:21:09 in Phoenix, you know, a little bit,
0:21:10 those are the most mature markets,
0:21:12 but also in LA and in Austin.
0:21:15 And, you know, all of the complexity that you see,
0:21:17 you know, go drive around the city, right?
0:21:18 All kinds of weather conditions,
0:21:21 whether it’s, you know, fog or, you know, storms
0:21:25 or dust storms or, you know, rainstorms down here,
0:21:27 like all of that, all of those are conditions
0:21:28 that we do operate in, right?
0:21:30 So then I think about, you know,
0:21:34 what makes it a great, you know, customer experience, right?
0:21:35 Like what does it take if you, you know,
0:21:39 grow by, you know, next, you know, orders of magnitude?
0:21:40 There’s a lot of improvements that we want to make
0:21:42 so that it becomes a better service for you
0:21:44 to get from point A to point B, right?
0:21:46 Like we asked for feedback from our writers.
0:21:49 A lot of feedback we get is, you know,
0:21:51 it has to do with the quality of your pickup
0:21:52 and drop-off locations, right?
0:21:53 So we’re learning from users.
0:21:55 Like no matter what, we want to make it a magical,
0:21:57 seamless, you know, delightful experience
0:21:59 from the time you kind of, you know,
0:22:00 start the app on your phone too,
0:22:01 when you get on the destination.
0:22:04 So that’s a lot of the work that we’re doing right now.
0:22:06 – Yeah, pick up and drop-off for what it’s worth
0:22:09 is an extraordinarily hard problem, right?
0:22:13 Like do you kind of block a little bit of a driveway
0:22:15 if you’re in an urban location
0:22:16 and then have a sensor that says,
0:22:19 oh, actually, I just saw somebody opening a garage door.
0:22:22 I need to get out of the way, you know,
0:22:24 how far down the street is acceptable to go pull.
0:22:25 Or if you’re in a parking lot,
0:22:27 where in the parking lot do you go?
0:22:29 Like this is an extraordinarily hard problem,
0:22:32 but to your point, it’s huge for user experience.
0:22:33 – That’s exactly right, right?
0:22:35 And just, you know, I think that’s a good example
0:22:36 of like just, hey, just one thing,
0:22:39 one of the many things that we have to build
0:22:41 in order for this to be an awesome product, right?
0:22:43 Not just like a technology demonstrator.
0:22:44 And I think you just like, you know,
0:22:49 hit exactly on a few things that make,
0:22:53 you know, something that kind of at the face of it
0:22:54 might seem fairly straightforward, right?
0:22:56 Okay, you know, I know there’s a place on the map
0:22:57 and I need to pull over.
0:22:59 It’s like, how hard can it be, right?
0:23:00 But really, if it’s a complicated, you know,
0:23:02 dense urban environment,
0:23:03 there’s a lot of these factors, right?
0:23:05 Is there like, you know, another vehicle
0:23:06 that you’re gonna be blocking?
0:23:08 Is there a garage door that’s opening, right?
0:23:10 Like, you know, what is the most convenient place
0:23:11 for the user to pick up?
0:23:14 What is, you know, so it really gets into this,
0:23:17 you know, the depth and the subtlety of understanding
0:23:21 the, you know, the semantics and the dynamic nature
0:23:23 of this driving task and, you know, doing things
0:23:25 that are, you know, safe, comfortable and predictable
0:23:28 and lead to a nice, seamless, pleasant,
0:23:30 delightful customer experience.
0:23:32 – Of course, okay, so you’ve mentioned this stat,
0:23:36 but 15 million miles, I know the number’s probably
0:23:39 a little bit bigger than that, but you released it Tuesday.
0:23:42 Yeah, it’s growing by the day.
0:23:46 15 million autonomous miles driven, that’s incredible.
0:23:49 Even more impressive, and you didn’t share this stat yet,
0:23:54 it results in 3.5 times fewer accidents
0:23:56 than human drivers, is that right?
0:23:59 – And I think 3.5 acts as the reduction in injury,
0:24:02 and it’s about 2x reduction in the police reportable
0:24:03 kind of lower severity incidents.
0:24:08 – This sort of comes to a question of both kind of
0:24:12 regulatory and, you know, business or ethical judgment.
0:24:15 What is the right level that you want to get to?
0:24:17 Obviously you want to constantly get better,
0:24:19 but is there a level at which you say,
0:24:21 “Okay, we’re good enough,” and that’s acceptable
0:24:22 to regulators?
0:24:24 – Yeah, so there’s no, you know,
0:24:28 simple, super simple short answer, right?
0:24:29 I think it starts with that.
0:24:31 It starts with those statistics that you just mentioned.
0:24:33 I can then have the day what I care about
0:24:35 is that roads are safer.
0:24:36 So when you look at those numbers,
0:24:38 yet, you know, when we operate today,
0:24:40 and we have, you know, strong empirical evidence
0:24:44 that our cars are in those areas safer than human drivers.
0:24:47 So on balance, that means a reduction in, you know,
0:24:49 collisions and harm.
0:24:54 Then, actually on top of the numbers
0:24:55 we’ve probably been publishing,
0:24:57 this is you’re quoting the latest numbers that we’ve shared.
0:24:58 – Yeah.
0:24:59 – Consistently, you know,
0:25:04 sharing numbers as our service scales up and grows.
0:25:06 You can also bring in, you know,
0:25:08 an additional lens of, you know,
0:25:11 how much did you contribute to a collision?
0:25:12 And we actually published, I think it was based
0:25:14 on about 4 million miles, 3.8 million miles.
0:25:17 We’ve published a joint study with Swiss ARRI,
0:25:20 which is, I think, the largest global reinsurer
0:25:20 in the world.
0:25:22 And the way they look at it is, you know,
0:25:24 who contributed to an event.
0:25:27 And there we saw like the same theme,
0:25:30 but the numbers were very strong.
0:25:35 That new field was a 76% reduction in property damage
0:25:39 collisions, and it was an 100% reduction
0:25:42 in claims around bodily injury.
0:25:43 So if you kind of bring in that lens,
0:25:45 I think the story becomes even more compelling.
0:25:46 – That is extremely compelling.
0:25:48 – Right, but there are some collisions where, you know,
0:25:51 we’d be, and that’s the bulk of the events that we see.
0:25:52 We’d be stopped at a red light,
0:25:54 and then somebody just plows into you, right?
0:25:55 – Sure.
0:25:58 – So, but then, like we, I think, you know,
0:26:01 we do know it’s a new technology, it’s a new product,
0:26:04 so it is held to a higher standard.
0:26:07 So we, when we think about our safety
0:26:09 and our readiness, you know, framing of methodology,
0:26:10 we don’t stop at just the race, right?
0:26:12 We build over the years as, you know,
0:26:16 one of the huge areas of investment
0:26:18 and experience over the years, like how, you know,
0:26:19 what else do you need?
0:26:20 So we have done, and we’ve done a number
0:26:21 of the other different things.
0:26:23 And we’ve published some of our methodologies,
0:26:25 we’ve shared our readiness framework, you know,
0:26:27 we do other things like we actually,
0:26:30 not just statistically, but on specific events,
0:26:34 we build models of an attentive, very good human driver,
0:26:36 like not distracted human, you know,
0:26:38 a good question whether such a driver exists, right?
0:26:41 But that’s kind of what we compare our driver to, right?
0:26:43 And it’s a model, like, it’s then it’s, you know,
0:26:45 in particular scenario, we evaluate ourselves
0:26:47 versus that model of a human driver,
0:26:49 and we hold ourselves to the bar of, you know,
0:26:51 doing well compared to that very high standard.
0:26:53 And then, you know, you pursue other, you know,
0:26:54 validation methodologies.
0:26:58 So that’s my answer is that it’s the, you know,
0:27:00 the aggregate of all of those methodologies
0:27:03 that we look at to decide that, yes, you know,
0:27:06 the system is ready enough to be deployed in scale.
0:27:09 – I’d love for you to talk about what you think,
0:27:10 maybe today and in the future,
0:27:13 about market structure, competition,
0:27:17 and what kind of role you envision Waymo playing.
0:27:20 – So the way we think about, you know,
0:27:22 Waymo and our company is that we are building
0:27:25 a generalize of both driver.
0:27:26 That’s the core of it.
0:27:28 And that’s the core of the mission
0:27:33 of making a transportation safe and accessible, right?
0:27:38 And we’re talking about right hailing today.
0:27:42 That’s our main, most mature primary application.
0:27:43 But, you know, we envision a future
0:27:46 where the Waymo driver will be deployed
0:27:47 in other commercial applications, right?
0:27:49 There’s deliveries, there’s trucking,
0:27:52 there’s personally owned vehicles, right?
0:27:54 So in all of those, you know,
0:27:57 our guiding principle would be to think
0:28:00 about the go-to-market strategy in a way
0:28:04 that accelerates access to this technology
0:28:08 and gets it deployed as, you know, broadly,
0:28:11 you know, of course, doing it gradually
0:28:12 and deliberately and safely, you know,
0:28:16 as quickly and broadly as possible.
0:28:18 So with that, as our guiding principle,
0:28:22 we’re gonna explore different commercial structures,
0:28:23 different partnership structures.
0:28:25 For example, in Phoenix today,
0:28:28 we have a partnership with Uber and Right Healing,
0:28:30 both in Uber Right Healing and in Uber Eats,
0:28:33 where, so in Phoenix, we have our own app.
0:28:35 You can download the Waymo app and, you know,
0:28:36 take a ride, an hour vehicle will show up
0:28:39 and take you where you wanna go.
0:28:41 That’s, you know, one way to experience our product.
0:28:43 Another one is through the Uber app.
0:28:45 We have a partnership where you can get through
0:28:48 the Uber app matched with our product,
0:28:49 the Waymo driver, the Waymo vehicle,
0:28:51 and it’s the same experience, right?
0:28:54 But this is another way for us to accelerate
0:28:56 and give more people to experience full autonomy.
0:28:59 And it gives us a chance to kind of, you know,
0:29:02 think about the different go-to-market strategies, right?
0:29:06 One is, you know, us having more of our own app.
0:29:07 The other one is more of a, you know,
0:29:09 driver-as-a-service for somebody else’s network.
0:29:11 So we’ll, you know, still early days,
0:29:13 but we will iterate and hold, you know,
0:29:15 in service of that main principle.
0:29:15 – That’s amazing.
0:29:18 Yeah, that’s gonna be exciting.
0:29:21 Maybe on back to the vehicle,
0:29:23 what about the hardware stack that you use?
0:29:24 You and I have talked a bunch about, you know,
0:29:28 you said like, hey, going all the way back to DARPA,
0:29:29 you know, it’s kind of the same stuff, right?
0:29:32 It’s, you know, it’s sensor, they’ve advanced
0:29:34 quite considerably, but, you know,
0:29:37 you still use, you know, radars and LiDAR.
0:29:41 Do you think that remains the future path
0:29:42 for autonomous driving?
0:29:43 LiDAR specifically?
0:29:48 – Yeah, no, I mean, the sensors are physically different,
0:29:53 right?
0:29:55 They have each one cameras, LiDARs, radar,
0:29:57 they have their, you know, benefits,
0:29:59 each one brings their own benefits, right?
0:30:00 You know, cameras obviously give you color
0:30:03 and they give you high, you know, very high resolution.
0:30:06 LiDARs kind of give you, you know,
0:30:09 a direct 3D measurement of your environment
0:30:11 and they’re an active sensor, right?
0:30:12 So it kind of brings their own energy,
0:30:15 pitch dark when there’s no, you know, external light source,
0:30:18 you know, you still get the seat just as well
0:30:20 as they do during the day, you know,
0:30:21 better in some cases.
0:30:24 And then, you know, radar is, you know,
0:30:26 very good at like, punching through just, you know,
0:30:28 different physics, different wavelengths, right?
0:30:32 If you build an imaging radar, which we do ourselves,
0:30:34 you know, it allows us to, you know,
0:30:37 give you an additional redundancy layer
0:30:39 and it has benefits, also an active sensor,
0:30:40 it can directly measure, you know,
0:30:42 through Doppler velocity of other objects
0:30:46 and it can, you know, degrades differently
0:30:48 and more gracefully in some other conditions.
0:30:50 Like, you know, very dense fog, you know,
0:30:52 or very dense rain.
0:30:53 So, you know, they’ll have their benefits.
0:30:58 So if you, you know, our approach has been to, you know,
0:31:00 use all of them, right?
0:31:02 And, you know, that’s how you have redundancy
0:31:04 and that’s how you get an extra boost
0:31:06 and capability of the system.
0:31:09 And, you know, we are on, you know,
0:31:11 today deployed in 5th and working to deploy
0:31:13 the sixth generation of our sensors.
0:31:15 And, you know, over those generations,
0:31:18 we’ve improved, you know, reliability,
0:31:20 we’ve improved, you know, capability and performance
0:31:23 and we’ve brought down the cost very significantly, right?
0:31:25 So, yeah, I think that the trend, you know,
0:31:27 for us that will, you know, using all three modalities
0:31:29 just makes a lot of sense.
0:31:31 Again, you know, you might make different trade-offs
0:31:33 if you are building a driver’s system
0:31:35 versus a fully autonomous vehicle where, you know,
0:31:38 that last 0.001% really, really matters.
0:31:39 – Yeah, absolutely.
0:31:44 One of the observations that we have
0:31:48 from the very early days of this wave of LLMs
0:31:53 is that there has been sort of already a massive
0:31:56 race of like cost reduction.
0:31:59 And many would argue that it’s sort of a process
0:32:02 of commoditization already, even though it’s very early days.
0:32:06 I would say the observation from autonomous driving
0:32:10 over many, many years now is kind of the opposite thing.
0:32:13 There’s been a thinning of the field, you know,
0:32:16 it’s proven to be much, much harder than expected.
0:32:19 Can you just talk about maybe why that’s the case?
0:32:21 – You know, they always have this property
0:32:22 that it’s very easy to get started,
0:32:25 but it’s very insanely difficult to get it, you know,
0:32:28 all the way, you know, to full autonomy
0:32:30 so that you can remove the driver.
0:32:34 And, you know, there’s maybe a few factors
0:32:36 that contribute to that.
0:32:40 One is, you know, compared to the LLMs and, you know,
0:32:43 it’s kind of AI in the digital world,
0:32:45 you have to operate in the physical world.
0:32:49 The physical world is messy, it is noisy,
0:32:50 and, you know, it can be quite humbling, right?
0:32:53 There’s all kinds of, you know, uncertainty and noise
0:32:57 that can kind of pull you out of distribution,
0:32:58 if you will, right? – Right, sure.
0:33:02 – So that’s one thing, that makes this very difficult.
0:33:07 And secondly, it’s safety, right?
0:33:09 – Sure.
0:33:12 – These, you know, AI systems, you know, in some domain,
0:33:16 you know, this is creativity, and it’s great.
0:33:18 You know, our domain, the cost of mistakes,
0:33:21 our lack of, you know, accuracy
0:33:22 has very serious consequences, right?
0:33:25 So that’s just the bar, very, very high.
0:33:27 Right, and then the last thing is that
0:33:30 it is, you know, you have to operate in real time.
0:33:33 You’re putting these systems on fast-moving vehicles,
0:33:35 and you have to, you know, milliseconds matter, right?
0:33:37 You have to make the decisions very quickly.
0:33:39 So I think it’s, you know, the combination of those factors
0:33:43 that really, you know, together lead to, you know,
0:33:45 the trend that you’ve been seeing is that,
0:33:47 like, you know, it’s an and, right?
0:33:48 You have to be excellent on this and this and this
0:33:49 and then, right?
0:33:50 It’s all of the bar.
0:33:52 The bar is very, very high for, you know,
0:33:54 every component of the system and how you put them together.
0:33:56 But, you know, there’s big advances,
0:33:57 and they, you know, boost you,
0:33:59 and they profile the system forward,
0:34:00 but there are no silver bullets, right?
0:34:03 And there’s no shortcuts if you’re talking about full autonomy.
0:34:06 And because of that lack of tolerance for errors,
0:34:08 you have a very high bar for safety.
0:34:12 You have a very high burden from regulators.
0:34:16 You know, it’s very costly to go through all those processes.
0:34:17 And so it makes sense.
0:34:20 And I’m very grateful that you guys have seen it through,
0:34:23 despite all the humbling experiences
0:34:25 that you had along the way.
0:34:29 It’s been a long journey, but it’s, you know,
0:34:31 for me and the many people at Weymo,
0:34:36 it is super exciting and very, very rewarding
0:34:38 to finally see it become reality.
0:34:41 Now, we talk about safety and AI in many contexts, right?
0:34:42 That’s a big question, right?
0:34:44 But, you know, here we are in this application of AI
0:34:46 in the physical world.
0:34:48 We have, you know, at this point a pretty robust
0:34:50 and increasing body of evidence that, you know,
0:34:53 we are seeing, like, tangible safety benefits.
0:34:54 So that’s very exciting.
0:34:56 Yeah, I always say to people,
0:34:58 it was a long journey
0:35:01 and very costly and expensive along the way.
0:35:04 But this is probably the most powerful manifestation of AI
0:35:08 that we have available to us in the world today.
0:35:09 I mean, you can get in a car without a driver,
0:35:11 and it’s safer than having a human.
0:35:12 And that’s just remarkable.
0:35:16 What were some of those humbling events along the way?
0:35:16 And those are early days?
0:35:17 Those are the first couple of years?
0:35:19 Early days.
0:35:24 Oh, I’m sorry, I remember one, there was one route.
0:35:29 That we did, that started, I think it started in Montmaville,
0:35:30 then went through Palo Alto,
0:35:33 then went, you know, through the mountains to Highway 1.
0:35:35 That took Highway 1 to San Francisco.
0:35:38 And I think, you know, went around the city a little bit
0:35:40 and, like, actually finished for Lombard Street.
0:35:42 So, like, in 2009, 10 people.
0:35:43 That is really complicated.
0:35:45 100 miles to the beginning to end, right?
0:35:47 I mean, as human drivers would fail at that task,
0:35:48 I think, so, yeah, I keep…
0:35:49 Yeah, yeah.
0:35:51 So, you know, we’re doing it one day,
0:35:52 and then we’re driving and kind of made it through
0:35:54 the Montmaville-Palo Alto part.
0:35:55 We’re driving through the mountains
0:35:57 and it’s foggy, it’s early morning.
0:36:00 And then we’re, like, seeing objects.
0:36:03 And, you know, our objects seem like random stuff
0:36:04 on the road in front of us.
0:36:06 There’s, like, a bucket and, like, a shoe.
0:36:08 And then there’s, like, at some point,
0:36:10 we come across, like, a, you know, a rusty bicycle.
0:36:12 Like, okay, what’s going on there?
0:36:13 And then we catch, you know, eventually.
0:36:16 And, like, the card, you know, doesn’t, you know,
0:36:17 handles it okay.
0:36:18 You know, maybe not super smoothly,
0:36:21 but, you know, we get stuck and we catch up to, like,
0:36:24 this dump truck that has all kind of stuff on it.
0:36:26 And, you know, periodically losing things
0:36:28 that person obstacles to the car.
0:36:30 This is, like, a cartoon, you know,
0:36:33 continuation of anomalies being thrown at you guys.
0:36:36 That’s pretty cool.
0:36:38 Okay, last question.
0:36:40 I’m going to tee you up to do some recruiting, probably.
0:36:45 But if you were in the shoes of the audience here
0:36:50 and just kind of seeking your first job,
0:36:52 I’m going to take something that you said, which is, like,
0:36:54 I can see your passion and excitement
0:36:56 for doing the start-up thing, right?
0:36:59 And, like, you know, kind of longing back for those days
0:37:01 is so cool.
0:37:04 What advice would you have for these folks
0:37:08 in where to go, whether it’s type of company,
0:37:13 type of role, industry, or anything else?
0:37:14 Way more?
0:37:16 That’s what I’m saying.
0:37:18 It’s the easiest to tee you right up.
0:37:20 You know, it’s a fine —
0:37:22 I mean, we’re talking about AI today,
0:37:24 but it’s a fine problem that matters.
0:37:26 You know, problem that matters to the world,
0:37:28 problem that matters to you.
0:37:31 Chances are it’s going to be a hard one.
0:37:33 Yeah.
0:37:37 Many things, you know, we’re doing have that property.
0:37:41 So don’t get discouraged by, you know,
0:37:43 the unknown by what others might tell you.
0:37:46 And, you know, start building.
0:37:49 And then, you know, keep building and don’t look back.
0:37:52 Huge congratulations on all the progress you guys have made.
0:37:56 And as a very happy customer, thank you for building it.
0:37:59 And we really appreciate you being here.
0:38:03 All right, that is all for today.
0:38:06 If you did make it this far, first of all, thank you.
0:38:08 We put a lot of thought into each of these episodes,
0:38:10 whether it’s guests, the calendar touchers,
0:38:12 the cycles with our amazing editor Tommy
0:38:14 until the music is just right.
0:38:16 So if you’d like what we put together,
0:38:20 consider dropping us a line at ratethispodcast.com/a16z.
0:38:23 And let us know what your favorite episode is.
0:38:26 It’ll make my day, and I’m sure Tommy’s too.
0:38:28 We’ll catch you on the flip side.
0:38:31 (upbeat music)
0:38:34 (crunching)
0:38:36 (upbeat music)
Waymo’s autonomous vehicles have driven over 20 million miles on public roads and billions more in simulation.
In this episode, a16z General Partner David George sits down with Dmitri Dolgov, CTO at Waymo, to discuss the development of self-driving technology. Dmitri provides technical insights into the evolution of hardware and software, the impact of generative AI, and the safety standards that guide Waymo’s innovations.
This footage is from AI Revolution, an event that a16z recently hosted in San Francisco. Watch the full event here: a16z.com/dmitri-dolgov-waymo-ai
Resources:
Find Dmitri on Twitter: https://x.com/dmitri_dolgov
Find David George on Twitter: https://x.com/DavidGeorge83
Learn more about Waymo: https://waymo.com/
Stay Updated:
Let us know what you think: https://ratethispodcast.com/a16z
Find a16z on Twitter: https://twitter.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Subscribe on your favorite podcast app: https://a16z.simplecast.com/
Follow our host: https://twitter.com/stephsmithio
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.