AI transcript
0:00:01 This is an iHeart podcast.
0:00:38 When we started this show in 2022, the standard line about driverless cars was, driverless
0:00:41 cars have been five years away for the past 15 years.
0:00:47 Because it seemed like they were just always around the corner, always just a few years
0:00:49 away, but they never quite arrived.
0:00:51 Nobody says that anymore.
0:00:56 Today, in several cities around the country, getting a ride from a driverless car is just
0:00:58 a normal thing people do.
0:01:03 And it’ll become normal in more and more and more cities over the next few years.
0:01:05 Driverless cars are here now.
0:01:07 So now we can ask, what’s next?
0:01:17 I’m Jacob Goldstein, and this is What’s Your Problem, the show where I talk to people who
0:01:19 are trying to make technological progress.
0:01:21 My guest today is Boris Soffman.
0:01:28 He’s the co-founder and CEO of Bedrock Robotics, a company that’s figuring out how to retrofit
0:01:30 heavy equipment to make it work autonomously.
0:01:33 Boris’s problem is this.
0:01:39 How do you teach machines not just to drive, but to do things like grade roads and move heavy
0:01:40 things around construction sites?
0:01:46 Boris’s company is starting with excavators, and they plan to have their first commercial
0:01:51 excavators autonomously digging holes on construction projects next year.
0:01:54 Later in the interview, Boris goes big.
0:02:00 He argues that if Bedrock succeeds, the company could help push forward a broad wave of new
0:02:01 building in America.
0:02:08 But first, we talked about his time at Waymo and how the wild evolution of the autonomous vehicles
0:02:11 that he worked on there led him to start Bedrock.
0:02:17 The time at Waymo was this incredible period where I was there for about five years from
0:02:20 mid-2019 through spring of 24.
0:02:26 And that was this really great period where it was going through this 15 years of R&D, and
0:02:30 then it finally transitioned into this hockey stick of growth that is happening today.
0:02:33 So today, Waymo’s at over 100 million miles fully driverless.
0:02:35 It’s at five times safer than a human.
0:02:37 It’s millions of miles every single week.
0:02:38 And so it’s kind of scaling exponentially.
0:02:41 And it’s like a genuinely fantastic product.
0:02:45 But I was there when we were like stressing over the first 100 miles, and it felt like the
0:02:50 most incredible achievement to just go like 10, 50, 100 miles completely driverlessly.
0:02:53 Now that happens at like hundreds of thousands of miles every single day.
0:02:58 And so one of the things that really broke through and made that possible is this shift to
0:03:02 machine learning and data-driven approaches as a core of the autonomy stack.
0:03:08 And just to be clear, like that’s as opposed to a more like heuristics or rule-based kind
0:03:13 of model, like the old school 20th century AI is like, well, just tell the car all the
0:03:16 rules of how to drive, and then it’ll drive.
0:03:18 Like that’s what you’re comparing machine learning to?
0:03:20 Yeah, like know how to drive.
0:03:22 I’m going to like embed the cost functions and all this.
0:03:26 So yeah, so like large-scale search and heuristics and rules, and you can embed them
0:03:27 now inside those heuristics.
0:03:31 And so you can solve almost any given problem with that sort of approach.
0:03:32 Yeah.
0:03:37 So you can solve one problem, but you can’t solve like every single possible problem that
0:03:41 would ever arise when you’re driving, which is actually what you have to solve to do full
0:03:42 autonomy, right?
0:03:42 Right.
0:03:45 With activating whack-a-mole, where like you fix one problem and it becomes harder and
0:03:47 harder to kind of scale the other ones.
0:03:53 And so there was this really conscious shift at Waymo, which was kind of bold at the time
0:03:54 because it feels obvious in hindsight.
0:03:55 It wasn’t obvious at all back then.
0:04:00 The shift to like really embracing this as a data-driven solution where you’re learning
0:04:03 from human driving and human behavior.
0:04:08 And when you say you’re learning, you mean the machine learning model, the AI basically
0:04:09 is learning.
0:04:09 The machine learning model.
0:04:10 That’s right.
0:04:14 And so you’re basically taking giant scales of data, hundreds of thousands, millions of
0:04:19 miles, and you’re learning the model of how do you drive and how do you interpret this
0:04:24 like infinite capacity and kitchen sink of contacts, like all of this sensor data, the
0:04:26 road structures, the things you see around you, the way people are moving.
0:04:32 How do you go from a kind of engineered and, you know, kind of trained solution into one
0:04:34 that is like dominantly a learned solution?
0:04:40 I mean, as you’re doing that, you’re sort of riding the historic machine learning AI wave,
0:04:40 right?
0:04:41 Like-
0:04:41 That’s right.
0:04:47 Presumably you’re able to do that at that moment because of this explosion in AI, which
0:04:48 is basically machine learning, right?
0:04:49 You nailed it.
0:04:51 And you couldn’t have done that five, 10 years ago.
0:04:53 That’s what you’re kind of coming out of at Waymo.
0:04:55 That’s driving the success at Waymo.
0:04:58 How does that set you up for what you’re doing at Bedrock?
0:05:02 The most shocking thing was just how well this generalized from San Francisco to Los Angeles,
0:05:03 Phoenix, and Austin.
0:05:07 And then it becomes almost like a qualification problem eventually where you’re using data
0:05:09 to fill some gaps, but like you need less and less of it.
0:05:11 Less and less new things surprise you.
0:05:13 Like you’re just, your competency kind of expands.
0:05:18 And then we unified the technology stack between cars and trucks where even jumping from a car
0:05:23 to a truck was needed maybe 10, 15% more data, but it was fundamentally like you’re using
0:05:25 data to explain like how you operate a very different platform.
0:05:26 And this is like a big truck.
0:05:29 This is like an 18-wheel truck when Google was working on that.
0:05:30 Yeah, or Waymo was working on that.
0:05:33 Like 53-foot trailer, like 80,000 pounds.
0:05:39 And so that was the big moment where we started thinking about where else can you apply this?
0:05:43 What are the places where you have all this diversity of challenges and capabilities that
0:05:47 benefit from this type of versatility and also have the ability to jump between platforms
0:05:48 in this really natural way?
0:05:54 And so we looked at a lot of spaces and really settled on automation of specialized type of
0:05:54 machinery.
0:05:59 And so the types of machines that you see in construction, like excavators and wheel loaders
0:06:03 and bulldozers, but also, frankly, the sort of machines you see in all sorts of industries
0:06:07 like agriculture and mining and lumber and garbage movement.
0:06:12 And so you have these very diverse types of machines that are interacting with the world
0:06:12 around them.
0:06:14 They’re fairly slow moving.
0:06:15 They’re in semi-controlled environments.
0:06:20 And at the end of the day, there’s astronomical scale at which they operate at.
0:06:25 And a lot of the learnings that we experienced at Waymo actually transfer over incredibly well.
0:06:29 But the physics of the problem is a lot less hyperspiral to actually really tackle this and get
0:06:30 the market.
0:06:37 Basically, big vehicles operating in semi-constrained environments, doing things to the world,
0:06:41 interacting with the world in some physical way in addition to just driving across it.
0:06:41 That’s right.
0:06:46 And these are slow moving vehicles where you’re already on a closed site with people who are
0:06:49 assumed to be knowledgeable about the world around you.
0:06:51 And you’re up, you’re moving at like five miles an hour, for example.
0:06:53 You’re able to slow down and stop.
0:06:54 You’re able to minimize your exposure.
0:06:57 You always have a minimum safety condition of stopping.
0:07:01 Your complexity is less about interactions with others on the road and more about the interactions
0:07:02 with the world around you.
0:07:09 And so you can actually tackle safety not through this sort of statistical methods that drove
0:07:14 a lot of the mileage that we’d collect, but through a much more direct measure of your
0:07:19 competencies in order to just make sure that you’re like you’re actually capable of hurting
0:07:19 somebody.
0:07:22 Yeah, I mean, more like like an industrial robot almost, right?
0:07:23 That’s right.
0:07:23 Yeah.
0:07:25 It’s like a traditional system of engineering.
0:07:25 Yeah.
0:07:29 Where it’s like, look, even if it can’t do the thing every time, that’s fine.
0:07:32 Just make sure it’s not going to like go crazy and kill somebody.
0:07:32 That’s right.
0:07:35 And you can make it so that like your worst case is your productivity suffers, but safety
0:07:37 wise, you’re solid.
0:07:42 And so that’s like incredibly enabling because your long tail is now no longer safety, it’s
0:07:44 the versatility of what you can do.
0:07:45 Why did you start with excavators?
0:07:50 So excavators are the most highly utilized machine.
0:07:53 Like they’re usually the highest volume in fleets.
0:07:56 So between 20 and 25% of fleets are excavators.
0:07:59 Fleets of just like big construction, heavy equipment?
0:08:04 Yeah, like a general contractor that alone, like a thousand machines, 200 to 250 will
0:08:06 probably be excavators on average.
0:08:08 And it’s basically what a kid would call a digger, right?
0:08:12 It’s like got a bucket and an arm and it like digs stuff up.
0:08:13 It’s like the equivalent of an arm.
0:08:15 Like you can like dig stuff.
0:08:16 You can demolish stuff.
0:08:17 You can swap your tools.
0:08:19 You can like lift pipes and put them in a hole.
0:08:20 They can do a ton of stuff with it.
0:08:20 It’s kind of crazy.
0:08:22 It really is a versatile machine.
0:08:24 It also makes it very complicated to work.
0:08:26 So there’s like seven degrees of freedom, sometimes eight.
0:08:28 And so it’s one of the hardest machines to learn.
0:08:31 And it takes four to five years to really become an expert.
0:08:33 And there’s a huge difference between an expert and a novice.
0:08:39 And so you kind of have this situation where it’s a huge volume of work and it’s really hard
0:08:39 to learn.
0:08:41 And so you have a really deep pull in the market for it.
0:08:42 Pull meaning a lot of demand.
0:08:47 Like a lot of people want somebody who can drive one of these or a machine that could do
0:08:47 it.
0:08:49 Well, let me tell you about the demand.
0:08:54 I’ve never seen such a divergence of supply to demand, like in my career, in any case,
0:08:59 where on the demand side, you have this astronomical construction industry that’s already $2 trillion
0:09:00 a year in the U.S.
0:09:04 That’s obviously very heavily building and having machinery work.
0:09:07 And then you have this shortage of operators that already existed, but it’s going in the
0:09:11 wrong direction where 40% of construction workers is retiring in the next 10 years.
0:09:16 Our partners are consistently having trouble filling labor.
0:09:18 We’ve met some that have 100% turnover.
0:09:21 Does that mean everybody leaves every year?
0:09:25 It means like a third of people might be lifers, but then like two-thirds transition more than
0:09:27 once per year and so you’re constantly backfilling.
0:09:29 The skill sets vary a ton.
0:09:34 And one of them said that for every one person entering the workforce of the quality that they
0:09:36 look for, there’s seven leaving.
0:09:41 And so what you end up having is this shortage of ability to meet this demand.
0:09:43 And so prices go up, jobs don’t get done.
0:09:47 And what’s interesting is it’s not this like isolated industry that’s like just on its own
0:09:48 kind of like having these sort of challenges.
0:09:50 It’s a horizontal.
0:09:51 It supports every industry.
0:09:53 You can’t build data centers for AI without it.
0:09:54 You can’t build houses.
0:09:56 You can’t build energy facilities.
0:09:59 Kind of like a rate-limiting skill or rate-limiting machine.
0:10:00 It’s the whole country.
0:10:01 That’s exactly right.
0:10:05 And then you have this need that starts with labor, but then there’s safety.
0:10:07 It’s huge amounts of safety challenges.
0:10:10 You have, you know, huge predictability challenges.
0:10:15 If you actually could soften out some of these constraints, we would build more, more work
0:10:17 would get done, and it would like stimulate the whole economy.
0:10:19 And so that’s what’s actually pretty exciting about this opportunity.
0:10:21 It’s not a zero-sump game at all.
0:10:22 Good.
0:10:24 So like you decide to focus on excavators.
0:10:26 You, you know, you raise money for your company.
0:10:31 Like do you go out and buy a whatever, a million-dollar excavator?
0:10:35 You go to, what is it, bucketandshovel.com and buy yourself an excavator?
0:10:38 They’re like $300,000 to $500,000.
0:10:40 So they’re like, still expensive.
0:10:41 Yeah, they’re pretty expensive.
0:10:42 So, I mean, like, did you buy one?
0:10:43 We did.
0:10:43 Yeah.
0:10:45 Did you drive it?
0:10:45 Of course.
0:10:46 Like, you have to.
0:10:47 It’s like, it’s like a rite of passage.
0:10:48 They’re really fun.
0:10:49 I even took a lesson.
0:10:54 There’s a place in Vegas that lets you drive excavators and take a lesson from like a
0:10:54 train operator.
0:10:55 It was pretty fun.
0:10:56 Oh, that’s genius.
0:10:58 You get to break things.
0:11:03 I was actually, as I was walking here today to the train, there’s a playground and there
0:11:08 was an excavator just like breaking up the asphalt and then like prying it up.
0:11:09 And I was like, that looks awesome.
0:11:11 This is the funniest thing.
0:11:14 Anybody that gets an excavator, like your inner six-year-old comes out.
0:11:18 Suddenly, you have like the most mature, sophisticated person who’s trying to be all professional.
0:11:23 They get in an excavator and they just like get like a giant glob of mud and then like bring
0:11:26 it as high as they can and then like plop it down and see what happens.
0:11:27 It’s amazing.
0:11:29 And it’s like without fail, everybody kind of reverts back to this sort of a world.
0:11:30 Yeah, we drove it.
0:11:33 I think we bought like a half dozen excavators at this point.
0:11:38 And then we also can use a lot of excavators from our partners who are general contractors
0:11:39 and subcontractors.
0:11:41 And it’s just what, a year or so ago that you started?
0:11:43 Like not that long in this, right?
0:11:44 Less than a year and a half.
0:11:44 Yeah.
0:11:45 Like last May.
0:11:47 So it’s pretty been a good run.
0:11:51 How much is sort of commodified of autonomy, right?
0:11:54 How much is just like, well, we’re going to buy this, this, and this in terms of sort of
0:11:55 hardware and we know the software.
0:11:59 And then how much is like, oh, here’s the things we have to figure out that nobody knows
0:11:59 how to do.
0:12:04 So it’s one of the other enablers that’s like way better than what we would have had to go
0:12:06 through 10 years ago in the space.
0:12:11 We can use a lot of the existing components on LiDAR, on cameras, on IMUs, GPS.
0:12:16 There’s a lot of tailwind from automotive great cameras and compute that’s like accelerating.
0:12:21 And like, presumably they’re buying those things at scale so you can like go get them
0:12:23 cheap, essentially.
0:12:25 You’re like, give me one of those, one of those, one of those.
0:12:25 That’s right.
0:12:29 Because you’re going to go to millions and millions of units as like every car gets an autopilot
0:12:31 equivalent over the next two, three years.
0:12:33 All of that’s starting to kind of plummet at cost.
0:12:33 So that helps.
0:12:38 And then even the platform, we can retrofit existing machinery.
0:12:40 When you say the platform, what do you mean in this context?
0:12:43 Platform is like the car, the truck, the excavator.
0:12:45 So in this case, the excavator.
0:12:45 Yeah, the machine.
0:12:50 So like the excavator itself, construction machines, particularly this latest generation,
0:12:55 they’re really well designed to where they’re already effectively dried by wire, which means
0:12:57 that every signal going through the machine is electric.
0:13:05 And so we’re able to splice into it and both read and write to these signals and control them.
0:13:08 Electric meaning like computerized, basically?
0:13:11 Meaning like you have a joystick that operates the excavator.
0:13:14 That’s not physically connected to the hydraulics.
0:13:16 It’s an electric signal that’s connected to the hydraulics.
0:13:21 And so the same for every signal in the machine, like the sensors, the like pressure gauges.
0:13:36 And so we’re able to both get the data from the machine as well as control the machine through a non-invasive integration where we can up in a machine with our sensors and compute suite in like less than three hours and make it autonomy capable.
0:13:43 We could never do that with a car or a truck because the platforms were just not designed this way.
0:13:44 Yeah, so there’s a lot that’s there already.
0:13:50 The excavators themselves are sort of easy to integrate with, lots of off-the-shelf technology.
0:13:51 What’s not there?
0:13:55 Like when you’re coming in, what do you have to sort of build that nobody has built before?
0:14:05 Nobody has actually created autonomy that can solve the really nuanced problems that you need to solve in order to operate like an excavator in construction tasks.
0:14:14 So let’s talk like specifically, you buy your excavators, you got your hardware, you know, whatever, what basic AI model you’re going to use.
0:14:17 But like, what’s a specific thing you have to figure out?
0:14:18 Well, a lot of things.
0:14:27 So first of all, it’s not trivial to tap into these machines and to build a platform of kind of integration around it where you can read them, you can control them.
0:14:32 And so you have to basically create like a wrapper around the machine and then also design in a way that scales later to new machines, right?
0:14:35 So that part’s hard. It’s a big autonomy problem.
0:14:41 So in that respect, it’s no different than when we tackle the Waymo where you have to train massive scale models.
0:14:42 You have to collect a huge amount of data.
0:14:44 And in this case, it’s like about the digging?
0:14:46 It’s about the digging.
0:14:48 Dumb question. Is the digging the hard part?
0:14:49 Like what happens when the thing hits the ground?
0:14:51 There’s different kinds of stuff in the ground.
0:14:52 Like, tell me.
0:14:56 In the category of what’s new, suddenly a car doesn’t change the environment around it.
0:14:57 Yeah.
0:15:01 You have a really complicated, tough earth and soil,
0:15:03 and you’ve got to figure out the physics of how you dig through it.
0:15:06 How do you deal with clay versus topsoil?
0:15:08 How do you deal with rocks inside?
0:15:11 And so now if you want to use simulation to solve parts of these problems,
0:15:15 you have a very complicated simulation problem because you have to solve not just the sensor side,
0:15:17 but also the manipulation side.
0:15:24 You have to structure this problem in a way where you’re learning from the data you collect
0:15:27 in order to actually capture the nuances of how do you actually interact with the environment.
0:15:30 You have to then control it and execute it the right way.
0:15:32 You have to think about how do you actually define the goal?
0:15:36 So from the user point of view with an excavator, like, how does it look?
0:15:39 Like, I just, I want to dig a hole of this size at this spot?
0:15:40 Or like, what is it?
0:15:41 Yeah.
0:15:42 And this is a journey.
0:15:45 Like, fast forward, you know, next year where a customer does this, right?
0:15:51 So what they would do is they would give it a model of what they want the earth to be dug towards.
0:15:55 And so that would be like a 3D representation of the depth.
0:15:59 So width, length, depth, here’s the edges, here’s the constraints.
0:16:01 The pickups for the trucks are going to be over here.
0:16:02 Okay.
0:16:03 And then go to work.
0:16:07 And basically, these projects, like these machines will work for many months at a time,
0:16:11 continually just digging earth and working it towards the right foundation.
0:16:15 And the system will respect that boundary and basically dig down to that level.
0:16:18 It’ll have precision to the edges that you’ve defined.
0:16:22 It’ll do it in a sequence that makes sense so that you don’t trap yourself in a hole, for example.
0:16:27 And then you specify where you’re going to have dump truck pickups.
0:16:31 And there’s projects where this happens for nine months straight or 12 months straight with, like, many machines.
0:16:36 And so what you basically need to specify is what you want to dig to.
0:16:43 And what you want to dig to ends up basically being the foundation that you’re working towards for whatever you’re going to construct there.
0:16:45 And we want that to be versatile.
0:16:48 So sometimes you’re just taking off a layer of topsoil.
0:16:49 Sometimes you’re digging eight feet deep.
0:16:53 Sometimes you’re taking an existing stockpile and moving it somewhere else.
0:16:55 So there’s a lot of permutations of this.
0:17:00 Sometimes you’re taking rubble from a demolition job and loading it onto trucks, right?
0:17:05 And so what’s nice is that you start to see patterns over and over again in this sort of work.
0:17:10 We’ll be back in just a minute.
0:17:23 Run a business and not thinking about podcasting?
0:17:23 Think again.
0:17:28 More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora.
0:17:33 And as the number one podcaster, iHeart’s twice as large as the next two combined.
0:17:36 So whatever your customers listen to, they’ll hear your message.
0:17:40 Plus, only iHeart can extend your message to audiences across broadcast radio.
0:17:41 Think podcasting can help your business?
0:17:43 Think iHeart.
0:17:45 Streaming, radio, and podcasting.
0:17:48 Call 844-844-IHEART to get started.
0:17:50 That’s 844-844-IHEART.
0:17:55 So the data problem is interesting here, right?
0:17:59 Like you were talking about the physics, just the physics of an excavator is quite different, right?
0:18:02 Like it’s pushing down on the ground, which is pushing back on the excavator.
0:18:05 And like, as you say, the ground is changing because of its work.
0:18:08 And there’s not…
0:18:09 Well, where do you get the data?
0:18:11 Data is actually an interesting mix.
0:18:16 We did it both on our test sites and then also with our design partners.
0:18:20 So we’re working with general contractors and subcontractors.
0:18:24 Today we have five that we’re partnered with across southern states like Arizona, Texas.
0:18:25 And so we’re getting data on their sites.
0:18:27 We’re getting data on our sites.
0:18:29 It’s not that much data, right?
0:18:35 Like, I mean, I guess my reference point is always like ImageNet or, you know, the internet for large language models.
0:18:42 It does seem like a sort of recurring problem in robotics type AI applications is sparsity of data.
0:18:43 How do you get it?
0:18:43 Yeah.
0:18:44 Yeah.
0:18:47 And so this is where there’s a few kind of interesting things.
0:18:51 So first of all, like our partners together have thousands and thousands of machines.
0:18:53 And so there’s a lot of choices of which you’re a partner on.
0:18:57 The other thing that you can do is actually you can be very clever on a test site.
0:19:03 And when you’re on a real project, you’re kind of getting an unbiased sample.
0:19:06 You’re just getting a random distribution of the things you see, just like driving around on a road.
0:19:11 When you’re on a test site, you can actually upsample the things you actually want.
0:19:21 And you can go and you can collect 10 hours of data that’s representative of 5,000 hours of random data, which is particularly useful for things like safety situations, right?
0:19:27 You can actually create a much larger equivalent amount of data on a closed course through like kind of structured testing.
0:19:33 And so, for example, safety scenarios where you have weird interactions with people doing things they shouldn’t do.
0:19:35 You don’t wait to see that on an open site.
0:19:37 What is one of those?
0:19:41 Like what is your nightmare human behavior scenario in the field?
0:19:43 Nightmare human behavior is a human is curious.
0:19:45 They walk up to your machine.
0:19:46 Your machine stops.
0:19:47 And then they get really, really close.
0:19:50 And they’re now in a blind spot where you can’t see them.
0:19:52 But you still have to be smart enough to track them.
0:19:55 Or they’re in a hole in front of you while you’re like thinking about digging, right?
0:20:01 So occlusions from humans is probably a huge category, which is complex.
0:20:04 Occlusions meaning in your blind spot.
0:20:06 Humans in a place where the machine can’t see them.
0:20:06 Yeah.
0:20:11 Or usually your number one priority is anything that touches on human safety.
0:20:11 That’s sacred.
0:20:13 You never take any risks on that.
0:20:15 What’s something you haven’t figured out yet?
0:20:17 The things people do with excavators.
0:20:22 Like we’ve seen them do bizarre like clearing of debris.
0:20:26 We’ve seen them load a wheel loader with dirt.
0:20:30 We’ve seen them bang tools to change them.
0:20:30 We’ve seen them.
0:20:32 Wait, bang tools to change them?
0:20:33 What’s that one?
0:20:37 Like a tool gets stuck and they like bang it on the ground in order to get it loose.
0:20:43 They use their arm as a pivot point to turn when the wheels are like stuck in mud.
0:20:44 It’s kind of baller.
0:20:45 Like it’s pretty baller.
0:20:49 Yeah, it’s like, I mean, like when you see the like expert operators, they just show
0:20:50 off and it’s like, it’s incredible.
0:20:51 Like they’re awesome.
0:20:56 And so there’s like all these like weird subtleties of how they’ll use these tools in like really
0:21:01 subtle ways, which like the dimensions of on the product side of the use cases that blew
0:21:02 us away.
0:21:08 We thought about the obvious ones, but as we started like really going deeper and like studying
0:21:11 this, there’s so much diversity and interesting things that you can do with these machines.
0:21:12 It’s quite powerful.
0:21:17 I mean, presumably you don’t have to do all those to have a product people will pay you
0:21:18 for, right?
0:21:25 You can just have the like competent automaton that can dig a big hole just the way you want
0:21:25 it.
0:21:26 That’s correct.
0:21:30 And that there’s a huge amount of work even in those areas, but there is this like tail
0:21:32 that you go when you’re over time, go and have more and more capability.
0:21:35 And the nice thing is there’s just software updates and you get more.
0:21:39 It’s just like a, you know, the product ends up being the digital operator, which gets better
0:21:40 over time.
0:21:42 And so it’s similar to human getting better.
0:21:44 What’s the business model?
0:21:46 So we will go operator out next year.
0:21:52 So that’ll be our first like fully operatorless product that is in a spirit of like what this
0:21:53 is meant to scale as.
0:21:56 Operator out means driverless, means autonomous.
0:21:57 Is that what it means?
0:21:57 Yeah.
0:21:58 No, nobody there.
0:21:59 It’s just like operatorless.
0:21:59 Yeah.
0:22:03 And so our product is the digital operator.
0:22:07 So what I mean by this is that our customer is a general contractor or subcontractor.
0:22:10 So it’s the companies that already buy and manage these machines.
0:22:14 We are, we sell them a retrofit of these machines.
0:22:17 So we install an upfit that adds sensors and compute.
0:22:20 So the contractor already owns the excavator.
0:22:21 That’s right.
0:22:24 They’ve already bought a $500,000 machine or $300,000 machine.
0:22:27 And so this is an upgrade that enables autonomy.
0:22:30 And then our business model is selling labor.
0:22:36 So we’re effectively selling the labor and a variety of digital services around it that
0:22:37 operates this machine.
0:22:41 And for the surface area of types of tasks that it’s approved for, it’s completely driverless.
0:22:44 And then that surface area increases over time as software updates.
0:22:47 And for everything else, it can still be manually operated.
0:22:50 So is the core product, are they paying you by the hour for digging?
0:22:54 So we’re figuring it out, but it’ll be something that is either by the hour,
0:22:56 by the project, by subscriptions.
0:23:04 So it’s effectively the way that today projects are forecasted and billed by shifts for labor.
0:23:09 It’s a parallel of that, but it becomes a huge win because you have complete flexibility.
0:23:11 You can work 10 hours a day.
0:23:12 You can work 24 hours a day.
0:23:17 So there’s all sorts of benefits that operationally give you a lot of leeway on how you use it.
0:23:19 What might go wrong?
0:23:21 What might go wrong?
0:23:23 I’ll tell you the things that are really challenging.
0:23:30 The nuance and diversity of things are high.
0:23:36 As much as you want to just boil it down to a very simple dig and load sort of operation,
0:23:38 there’s always little corner cases.
0:23:40 There’s things you find in the ground.
0:23:44 There’s weird ways that trucks could interact with you.
0:23:45 There’s things people will do.
0:23:50 There’s varieties of machines, but there’s like 50 kinds of excavator models and sizes and so forth.
0:23:53 So I think the long tail is still challenging.
0:24:02 Presumably, you don’t have to figure out every edge case, but you probably have to figure out a lot for your thing to work in a functional sense, right?
0:24:07 In the meantime, you’re not just digging and loading.
0:24:08 You’ve got to reposition.
0:24:09 You’ve got to organize the earth.
0:24:10 You’ve got to think about the sequencing.
0:24:13 You’ve got to deal with daytime, nighttime, rain.
0:24:19 And so you have these types of really challenging diversities you have to think about and deal with.
0:24:27 So I think all in all, it’s still a complicated product area where there’s a huge amount of diversity of the things that need to be done.
0:24:41 But it’s one of those where I personally think there’s a handful of these holy grails of autonomy and physical industries that are like genuinely transformational opportunities for both positive impact to the country and the world,
0:24:45 and also just kind of scale of industries that are like double-digit percentages of GDP.
0:24:47 Transportation is without a doubt one of them.
0:24:49 Construction is one of them.
0:24:51 Agriculture is not far behind.
0:24:53 You know, mining is very, very significant as well.
0:24:55 Manufacturing is one of them.
0:25:06 And so I think that we’re going to see a wave over this next 10 years in autonomy, but it’s going to be tackling this like 75% of the world’s GDP that’s physical and not digital.
0:25:10 And there’s a lot of work, like a lot of positive impact that can happen across these spaces.
0:25:12 I mean, give me a little more on that one.
0:25:16 Pick a time in the future, five years, 10 years, not more than 10.
0:25:30 So my personal belief is that this idea, like there’s a lot of companies getting flooded for this, but the idea that like this giant brain for all of robotics, the foundation model for robotics, like I personally do not believe that that’s viable in the next 10 years.
0:25:51 Because you have such complexity in really understanding these verticals on the inputs, the hardware, the products, the neat use case, the customers, everything, that every single one of those has such complexity to get data, that the idea to get enough data that bulldozes through the generalization problem, that’s a long ways off.
0:26:03 But I do think that we’re at a perfect time for these vertical solutions, where if you have a focused solution where you’re trying to do construction, you’re trying to do fulfillment in a warehouse, you’re trying to do a very focused manufacturing solution.
0:26:08 I think that’s actually, there’s a giant step function in how powerful ML technologies are.
0:26:14 But if you’re trying to do a humanoid to do everything in a home for a consumer market, too far away.
0:26:15 Right.
0:26:18 Now, you’re saying you basically, you got to solve one problem at a time.
0:26:19 Yeah.
0:26:22 I want to go back to open road autonomy.
0:26:22 Almost done.
0:26:32 When do you think that most rides most people take in cars or trucks will be in driverless cars, autonomous vehicles?
0:26:33 Great question.
0:26:36 There’s a few prerequisites for most rides.
0:26:40 So first, all the geofences have to disappear, and the thing just works everywhere.
0:26:46 That’s on a trajectory of getting there because it’s getting more and more efficient as the scales across the country and then the world.
0:26:49 It has to go from ride hailing to personal car ownership.
0:26:55 That is both cost down and versatility and everything that, like, but that’s going to happen.
0:26:59 In that universe, when do they take the steering wheel out?
0:27:01 I just took a Waymo for the first time.
0:27:02 I was in San Francisco this summer.
0:27:03 It was awesome, right?
0:27:04 It was awesome, yeah.
0:27:08 And, like, the weirdest thing about it to me was the steering wheel.
0:27:08 Yeah.
0:27:08 Right?
0:27:14 Like, if it was just a box that I got in that looked like a train car or something, I would have been like, oh, sure, it’s like the air train or whatever.
0:27:17 But that there’s a steering wheel and it’s like a ghost is turning the steering wheel.
0:27:19 Like, what’s the steering wheel doing there?
0:27:25 So, the steering wheels will disappear pretty quickly because ride sharing autonomous cars have no use for it.
0:27:26 And then you need to wait.
0:27:27 Yeah, there’s, like, special cases.
0:27:28 We need to recover them.
0:27:30 But it’s a wasted space, right?
0:27:31 It’s a wasted spot.
0:27:33 So, that’ll happen soon.
0:27:38 But the idea of really deep penetration, I think it’s personal cars.
0:27:43 It’s beyond luxury, which is another generation, to, like, actually make it be something that’s affordable.
0:27:47 Then you’ve got to go through a buying cycle, which is, like, five, seven years.
0:27:52 And so, I think it starts to get serious penetration in the back half of the 30s.
0:27:59 And it’ll be the 40s where, like, okay, 50% of driving is kind of, like, autonomous.
0:28:01 I think it’s still 20 years off in my mind.
0:28:03 For example, you bought a car today.
0:28:06 It’s going to be in circulation for the next 12 to 15 years.
0:28:07 It just takes a while.
0:28:14 We’ll be back in a minute with The Lightning Round.
0:28:27 Run a business and not thinking about podcasting?
0:28:28 Think again.
0:28:33 More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora.
0:28:37 And as the number one podcaster, iHeart’s twice as large as the next two combined.
0:28:40 So, whatever your customers listen to, they’ll hear your message.
0:28:44 Plus, only iHeart can extend your message to audiences across broadcast radio.
0:28:46 Think podcasting can help your business?
0:28:47 Think iHeart.
0:28:49 Streaming, radio, and podcasting.
0:28:52 Call 844-844-IHEART to get started.
0:28:55 That’s 844-844-IHEART.
0:28:58 Okay, let’s finish with The Lightning Round.
0:29:02 If you could operate any machine, what would it be?
0:29:05 Oh, gosh.
0:29:06 This is a fun one.
0:29:10 If I could operate any machine at all in the world, not even in construction.
0:29:12 Oh, gosh.
0:29:12 Okay.
0:29:13 You know what?
0:29:18 I would love to either operate one of the, like, boring company gigantic drill machines.
0:29:18 Oh, good one.
0:29:19 Because they’re just, like, so astronomical.
0:29:25 Or there’s, like, mine trucks that are so astronomically big that they just dwarf any machine in the world.
0:29:26 And it costs, like, $5 million.
0:29:30 And just a skill that would be really, really fun to try at some point.
0:29:31 Just to be that high.
0:29:34 Just to have that much momentum, right?
0:29:36 That much mass at your disposal.
0:29:37 Yeah.
0:29:39 Like, literally, the tire is, like, three stories tall.
0:29:40 It’s, like, it’s wild.
0:29:41 Like, it’s absurd.
0:29:47 What’s one thing you remember about immigrating from the Soviet Union to the U.S. when you were, was it six?
0:29:48 I was six.
0:29:48 Yeah.
0:29:50 I was born in Moscow.
0:29:50 We immigrated.
0:29:51 I was super young.
0:30:00 I remember we ended up having a pit stop in Europe where there’s a standard path of going to Vienna, Venice, and Rome while your paperwork gets processed.
0:30:01 And then we went to New York.
0:30:12 I remember running around the rooftops of Venice with a friend of mine causing a bunch of, you know, trouble and running away and disappearing for long periods of time and having a blast.
0:30:20 And for whatever reason, running around the rooftops of Venice was ingrained in my memory, which was kind of a very positive memory while my parents were going through a whole bunch of stress.
0:30:25 Sounds very free, not to project East versus West language onto it, but it sounds very free.
0:30:27 It was very freeing.
0:30:28 It’s funny.
0:30:35 I just hit the age my dad was when they immigrated from the Soviet Union with two kids.
0:30:41 My sister was one month old, zero money, almost no English, having a fresh start.
0:30:42 So courageous, right?
0:30:44 Doesn’t it seem so brave?
0:30:46 Yeah, it does.
0:30:48 And they were trying to leave for like 10 years and couldn’t leave.
0:30:49 Yeah, it’s kind of fascinating.
0:30:52 So we’re very fortunate to not have to go through something like that.
0:30:56 Is that when you think about it that way, does that like put pressure on you?
0:30:58 You think, oh, my God, my parents did all this.
0:31:00 Like, I better, I better deliver.
0:31:02 Oh, it’s funny.
0:31:03 A little bit.
0:31:09 I mean, you kind of have this, I don’t know, like adventurous spirit, I guess, that maybe gets ingrained.
0:31:16 I think of it now for my kids where I’m like, okay, like now they’re in a really nice and comfortable environment growing up.
0:31:19 How do you like convey that edge and a little bit of that spirit?
0:31:21 How do you keep them from going soft?
0:31:21 Yeah.
0:31:28 And it’s not that you want them to like go through anything difficult because it’s not that I was, you know, for me, it was actually just an adventure.
0:31:29 My parents was difficult.
0:31:38 But part of it is just, yeah, like conveying that spirit of being able to like be comfortable trying to tackle something new and being thrown in a completely different environment.
0:31:43 It’s hard to force that or simulate that when you’re just growing up in the San Francisco area, right?
0:31:44 I appreciate your time.
0:31:45 Thanks for talking to me.
0:31:46 It’s a pleasure.
0:31:47 This was a lot of fun.
0:31:48 Thanks for having me.
0:31:59 Boris Hoffman is the co-founder and CEO of Bedrock Robotics.
0:32:06 Just a quick note, this is our last episode before a break of a couple of weeks, and then we’ll be back with more episodes.
0:32:10 Please email us at problem at Pushkin.fm.
0:32:13 We are always looking for new guests for the show.
0:32:17 Today’s show was produced by Trina Menino and Gabriel Hunter-Chang.
0:32:22 It was edited by Alexandra Gerriten and engineered by Sarah Bruguer.
0:32:26 I’m Jacob Goldstein, and we’ll be back next week with another episode of What’s Your Problem?
0:32:34 This is an iHeart Podcast.
0:00:38 When we started this show in 2022, the standard line about driverless cars was, driverless
0:00:41 cars have been five years away for the past 15 years.
0:00:47 Because it seemed like they were just always around the corner, always just a few years
0:00:49 away, but they never quite arrived.
0:00:51 Nobody says that anymore.
0:00:56 Today, in several cities around the country, getting a ride from a driverless car is just
0:00:58 a normal thing people do.
0:01:03 And it’ll become normal in more and more and more cities over the next few years.
0:01:05 Driverless cars are here now.
0:01:07 So now we can ask, what’s next?
0:01:17 I’m Jacob Goldstein, and this is What’s Your Problem, the show where I talk to people who
0:01:19 are trying to make technological progress.
0:01:21 My guest today is Boris Soffman.
0:01:28 He’s the co-founder and CEO of Bedrock Robotics, a company that’s figuring out how to retrofit
0:01:30 heavy equipment to make it work autonomously.
0:01:33 Boris’s problem is this.
0:01:39 How do you teach machines not just to drive, but to do things like grade roads and move heavy
0:01:40 things around construction sites?
0:01:46 Boris’s company is starting with excavators, and they plan to have their first commercial
0:01:51 excavators autonomously digging holes on construction projects next year.
0:01:54 Later in the interview, Boris goes big.
0:02:00 He argues that if Bedrock succeeds, the company could help push forward a broad wave of new
0:02:01 building in America.
0:02:08 But first, we talked about his time at Waymo and how the wild evolution of the autonomous vehicles
0:02:11 that he worked on there led him to start Bedrock.
0:02:17 The time at Waymo was this incredible period where I was there for about five years from
0:02:20 mid-2019 through spring of 24.
0:02:26 And that was this really great period where it was going through this 15 years of R&D, and
0:02:30 then it finally transitioned into this hockey stick of growth that is happening today.
0:02:33 So today, Waymo’s at over 100 million miles fully driverless.
0:02:35 It’s at five times safer than a human.
0:02:37 It’s millions of miles every single week.
0:02:38 And so it’s kind of scaling exponentially.
0:02:41 And it’s like a genuinely fantastic product.
0:02:45 But I was there when we were like stressing over the first 100 miles, and it felt like the
0:02:50 most incredible achievement to just go like 10, 50, 100 miles completely driverlessly.
0:02:53 Now that happens at like hundreds of thousands of miles every single day.
0:02:58 And so one of the things that really broke through and made that possible is this shift to
0:03:02 machine learning and data-driven approaches as a core of the autonomy stack.
0:03:08 And just to be clear, like that’s as opposed to a more like heuristics or rule-based kind
0:03:13 of model, like the old school 20th century AI is like, well, just tell the car all the
0:03:16 rules of how to drive, and then it’ll drive.
0:03:18 Like that’s what you’re comparing machine learning to?
0:03:20 Yeah, like know how to drive.
0:03:22 I’m going to like embed the cost functions and all this.
0:03:26 So yeah, so like large-scale search and heuristics and rules, and you can embed them
0:03:27 now inside those heuristics.
0:03:31 And so you can solve almost any given problem with that sort of approach.
0:03:32 Yeah.
0:03:37 So you can solve one problem, but you can’t solve like every single possible problem that
0:03:41 would ever arise when you’re driving, which is actually what you have to solve to do full
0:03:42 autonomy, right?
0:03:42 Right.
0:03:45 With activating whack-a-mole, where like you fix one problem and it becomes harder and
0:03:47 harder to kind of scale the other ones.
0:03:53 And so there was this really conscious shift at Waymo, which was kind of bold at the time
0:03:54 because it feels obvious in hindsight.
0:03:55 It wasn’t obvious at all back then.
0:04:00 The shift to like really embracing this as a data-driven solution where you’re learning
0:04:03 from human driving and human behavior.
0:04:08 And when you say you’re learning, you mean the machine learning model, the AI basically
0:04:09 is learning.
0:04:09 The machine learning model.
0:04:10 That’s right.
0:04:14 And so you’re basically taking giant scales of data, hundreds of thousands, millions of
0:04:19 miles, and you’re learning the model of how do you drive and how do you interpret this
0:04:24 like infinite capacity and kitchen sink of contacts, like all of this sensor data, the
0:04:26 road structures, the things you see around you, the way people are moving.
0:04:32 How do you go from a kind of engineered and, you know, kind of trained solution into one
0:04:34 that is like dominantly a learned solution?
0:04:40 I mean, as you’re doing that, you’re sort of riding the historic machine learning AI wave,
0:04:40 right?
0:04:41 Like-
0:04:41 That’s right.
0:04:47 Presumably you’re able to do that at that moment because of this explosion in AI, which
0:04:48 is basically machine learning, right?
0:04:49 You nailed it.
0:04:51 And you couldn’t have done that five, 10 years ago.
0:04:53 That’s what you’re kind of coming out of at Waymo.
0:04:55 That’s driving the success at Waymo.
0:04:58 How does that set you up for what you’re doing at Bedrock?
0:05:02 The most shocking thing was just how well this generalized from San Francisco to Los Angeles,
0:05:03 Phoenix, and Austin.
0:05:07 And then it becomes almost like a qualification problem eventually where you’re using data
0:05:09 to fill some gaps, but like you need less and less of it.
0:05:11 Less and less new things surprise you.
0:05:13 Like you’re just, your competency kind of expands.
0:05:18 And then we unified the technology stack between cars and trucks where even jumping from a car
0:05:23 to a truck was needed maybe 10, 15% more data, but it was fundamentally like you’re using
0:05:25 data to explain like how you operate a very different platform.
0:05:26 And this is like a big truck.
0:05:29 This is like an 18-wheel truck when Google was working on that.
0:05:30 Yeah, or Waymo was working on that.
0:05:33 Like 53-foot trailer, like 80,000 pounds.
0:05:39 And so that was the big moment where we started thinking about where else can you apply this?
0:05:43 What are the places where you have all this diversity of challenges and capabilities that
0:05:47 benefit from this type of versatility and also have the ability to jump between platforms
0:05:48 in this really natural way?
0:05:54 And so we looked at a lot of spaces and really settled on automation of specialized type of
0:05:54 machinery.
0:05:59 And so the types of machines that you see in construction, like excavators and wheel loaders
0:06:03 and bulldozers, but also, frankly, the sort of machines you see in all sorts of industries
0:06:07 like agriculture and mining and lumber and garbage movement.
0:06:12 And so you have these very diverse types of machines that are interacting with the world
0:06:12 around them.
0:06:14 They’re fairly slow moving.
0:06:15 They’re in semi-controlled environments.
0:06:20 And at the end of the day, there’s astronomical scale at which they operate at.
0:06:25 And a lot of the learnings that we experienced at Waymo actually transfer over incredibly well.
0:06:29 But the physics of the problem is a lot less hyperspiral to actually really tackle this and get
0:06:30 the market.
0:06:37 Basically, big vehicles operating in semi-constrained environments, doing things to the world,
0:06:41 interacting with the world in some physical way in addition to just driving across it.
0:06:41 That’s right.
0:06:46 And these are slow moving vehicles where you’re already on a closed site with people who are
0:06:49 assumed to be knowledgeable about the world around you.
0:06:51 And you’re up, you’re moving at like five miles an hour, for example.
0:06:53 You’re able to slow down and stop.
0:06:54 You’re able to minimize your exposure.
0:06:57 You always have a minimum safety condition of stopping.
0:07:01 Your complexity is less about interactions with others on the road and more about the interactions
0:07:02 with the world around you.
0:07:09 And so you can actually tackle safety not through this sort of statistical methods that drove
0:07:14 a lot of the mileage that we’d collect, but through a much more direct measure of your
0:07:19 competencies in order to just make sure that you’re like you’re actually capable of hurting
0:07:19 somebody.
0:07:22 Yeah, I mean, more like like an industrial robot almost, right?
0:07:23 That’s right.
0:07:23 Yeah.
0:07:25 It’s like a traditional system of engineering.
0:07:25 Yeah.
0:07:29 Where it’s like, look, even if it can’t do the thing every time, that’s fine.
0:07:32 Just make sure it’s not going to like go crazy and kill somebody.
0:07:32 That’s right.
0:07:35 And you can make it so that like your worst case is your productivity suffers, but safety
0:07:37 wise, you’re solid.
0:07:42 And so that’s like incredibly enabling because your long tail is now no longer safety, it’s
0:07:44 the versatility of what you can do.
0:07:45 Why did you start with excavators?
0:07:50 So excavators are the most highly utilized machine.
0:07:53 Like they’re usually the highest volume in fleets.
0:07:56 So between 20 and 25% of fleets are excavators.
0:07:59 Fleets of just like big construction, heavy equipment?
0:08:04 Yeah, like a general contractor that alone, like a thousand machines, 200 to 250 will
0:08:06 probably be excavators on average.
0:08:08 And it’s basically what a kid would call a digger, right?
0:08:12 It’s like got a bucket and an arm and it like digs stuff up.
0:08:13 It’s like the equivalent of an arm.
0:08:15 Like you can like dig stuff.
0:08:16 You can demolish stuff.
0:08:17 You can swap your tools.
0:08:19 You can like lift pipes and put them in a hole.
0:08:20 They can do a ton of stuff with it.
0:08:20 It’s kind of crazy.
0:08:22 It really is a versatile machine.
0:08:24 It also makes it very complicated to work.
0:08:26 So there’s like seven degrees of freedom, sometimes eight.
0:08:28 And so it’s one of the hardest machines to learn.
0:08:31 And it takes four to five years to really become an expert.
0:08:33 And there’s a huge difference between an expert and a novice.
0:08:39 And so you kind of have this situation where it’s a huge volume of work and it’s really hard
0:08:39 to learn.
0:08:41 And so you have a really deep pull in the market for it.
0:08:42 Pull meaning a lot of demand.
0:08:47 Like a lot of people want somebody who can drive one of these or a machine that could do
0:08:47 it.
0:08:49 Well, let me tell you about the demand.
0:08:54 I’ve never seen such a divergence of supply to demand, like in my career, in any case,
0:08:59 where on the demand side, you have this astronomical construction industry that’s already $2 trillion
0:09:00 a year in the U.S.
0:09:04 That’s obviously very heavily building and having machinery work.
0:09:07 And then you have this shortage of operators that already existed, but it’s going in the
0:09:11 wrong direction where 40% of construction workers is retiring in the next 10 years.
0:09:16 Our partners are consistently having trouble filling labor.
0:09:18 We’ve met some that have 100% turnover.
0:09:21 Does that mean everybody leaves every year?
0:09:25 It means like a third of people might be lifers, but then like two-thirds transition more than
0:09:27 once per year and so you’re constantly backfilling.
0:09:29 The skill sets vary a ton.
0:09:34 And one of them said that for every one person entering the workforce of the quality that they
0:09:36 look for, there’s seven leaving.
0:09:41 And so what you end up having is this shortage of ability to meet this demand.
0:09:43 And so prices go up, jobs don’t get done.
0:09:47 And what’s interesting is it’s not this like isolated industry that’s like just on its own
0:09:48 kind of like having these sort of challenges.
0:09:50 It’s a horizontal.
0:09:51 It supports every industry.
0:09:53 You can’t build data centers for AI without it.
0:09:54 You can’t build houses.
0:09:56 You can’t build energy facilities.
0:09:59 Kind of like a rate-limiting skill or rate-limiting machine.
0:10:00 It’s the whole country.
0:10:01 That’s exactly right.
0:10:05 And then you have this need that starts with labor, but then there’s safety.
0:10:07 It’s huge amounts of safety challenges.
0:10:10 You have, you know, huge predictability challenges.
0:10:15 If you actually could soften out some of these constraints, we would build more, more work
0:10:17 would get done, and it would like stimulate the whole economy.
0:10:19 And so that’s what’s actually pretty exciting about this opportunity.
0:10:21 It’s not a zero-sump game at all.
0:10:22 Good.
0:10:24 So like you decide to focus on excavators.
0:10:26 You, you know, you raise money for your company.
0:10:31 Like do you go out and buy a whatever, a million-dollar excavator?
0:10:35 You go to, what is it, bucketandshovel.com and buy yourself an excavator?
0:10:38 They’re like $300,000 to $500,000.
0:10:40 So they’re like, still expensive.
0:10:41 Yeah, they’re pretty expensive.
0:10:42 So, I mean, like, did you buy one?
0:10:43 We did.
0:10:43 Yeah.
0:10:45 Did you drive it?
0:10:45 Of course.
0:10:46 Like, you have to.
0:10:47 It’s like, it’s like a rite of passage.
0:10:48 They’re really fun.
0:10:49 I even took a lesson.
0:10:54 There’s a place in Vegas that lets you drive excavators and take a lesson from like a
0:10:54 train operator.
0:10:55 It was pretty fun.
0:10:56 Oh, that’s genius.
0:10:58 You get to break things.
0:11:03 I was actually, as I was walking here today to the train, there’s a playground and there
0:11:08 was an excavator just like breaking up the asphalt and then like prying it up.
0:11:09 And I was like, that looks awesome.
0:11:11 This is the funniest thing.
0:11:14 Anybody that gets an excavator, like your inner six-year-old comes out.
0:11:18 Suddenly, you have like the most mature, sophisticated person who’s trying to be all professional.
0:11:23 They get in an excavator and they just like get like a giant glob of mud and then like bring
0:11:26 it as high as they can and then like plop it down and see what happens.
0:11:27 It’s amazing.
0:11:29 And it’s like without fail, everybody kind of reverts back to this sort of a world.
0:11:30 Yeah, we drove it.
0:11:33 I think we bought like a half dozen excavators at this point.
0:11:38 And then we also can use a lot of excavators from our partners who are general contractors
0:11:39 and subcontractors.
0:11:41 And it’s just what, a year or so ago that you started?
0:11:43 Like not that long in this, right?
0:11:44 Less than a year and a half.
0:11:44 Yeah.
0:11:45 Like last May.
0:11:47 So it’s pretty been a good run.
0:11:51 How much is sort of commodified of autonomy, right?
0:11:54 How much is just like, well, we’re going to buy this, this, and this in terms of sort of
0:11:55 hardware and we know the software.
0:11:59 And then how much is like, oh, here’s the things we have to figure out that nobody knows
0:11:59 how to do.
0:12:04 So it’s one of the other enablers that’s like way better than what we would have had to go
0:12:06 through 10 years ago in the space.
0:12:11 We can use a lot of the existing components on LiDAR, on cameras, on IMUs, GPS.
0:12:16 There’s a lot of tailwind from automotive great cameras and compute that’s like accelerating.
0:12:21 And like, presumably they’re buying those things at scale so you can like go get them
0:12:23 cheap, essentially.
0:12:25 You’re like, give me one of those, one of those, one of those.
0:12:25 That’s right.
0:12:29 Because you’re going to go to millions and millions of units as like every car gets an autopilot
0:12:31 equivalent over the next two, three years.
0:12:33 All of that’s starting to kind of plummet at cost.
0:12:33 So that helps.
0:12:38 And then even the platform, we can retrofit existing machinery.
0:12:40 When you say the platform, what do you mean in this context?
0:12:43 Platform is like the car, the truck, the excavator.
0:12:45 So in this case, the excavator.
0:12:45 Yeah, the machine.
0:12:50 So like the excavator itself, construction machines, particularly this latest generation,
0:12:55 they’re really well designed to where they’re already effectively dried by wire, which means
0:12:57 that every signal going through the machine is electric.
0:13:05 And so we’re able to splice into it and both read and write to these signals and control them.
0:13:08 Electric meaning like computerized, basically?
0:13:11 Meaning like you have a joystick that operates the excavator.
0:13:14 That’s not physically connected to the hydraulics.
0:13:16 It’s an electric signal that’s connected to the hydraulics.
0:13:21 And so the same for every signal in the machine, like the sensors, the like pressure gauges.
0:13:36 And so we’re able to both get the data from the machine as well as control the machine through a non-invasive integration where we can up in a machine with our sensors and compute suite in like less than three hours and make it autonomy capable.
0:13:43 We could never do that with a car or a truck because the platforms were just not designed this way.
0:13:44 Yeah, so there’s a lot that’s there already.
0:13:50 The excavators themselves are sort of easy to integrate with, lots of off-the-shelf technology.
0:13:51 What’s not there?
0:13:55 Like when you’re coming in, what do you have to sort of build that nobody has built before?
0:14:05 Nobody has actually created autonomy that can solve the really nuanced problems that you need to solve in order to operate like an excavator in construction tasks.
0:14:14 So let’s talk like specifically, you buy your excavators, you got your hardware, you know, whatever, what basic AI model you’re going to use.
0:14:17 But like, what’s a specific thing you have to figure out?
0:14:18 Well, a lot of things.
0:14:27 So first of all, it’s not trivial to tap into these machines and to build a platform of kind of integration around it where you can read them, you can control them.
0:14:32 And so you have to basically create like a wrapper around the machine and then also design in a way that scales later to new machines, right?
0:14:35 So that part’s hard. It’s a big autonomy problem.
0:14:41 So in that respect, it’s no different than when we tackle the Waymo where you have to train massive scale models.
0:14:42 You have to collect a huge amount of data.
0:14:44 And in this case, it’s like about the digging?
0:14:46 It’s about the digging.
0:14:48 Dumb question. Is the digging the hard part?
0:14:49 Like what happens when the thing hits the ground?
0:14:51 There’s different kinds of stuff in the ground.
0:14:52 Like, tell me.
0:14:56 In the category of what’s new, suddenly a car doesn’t change the environment around it.
0:14:57 Yeah.
0:15:01 You have a really complicated, tough earth and soil,
0:15:03 and you’ve got to figure out the physics of how you dig through it.
0:15:06 How do you deal with clay versus topsoil?
0:15:08 How do you deal with rocks inside?
0:15:11 And so now if you want to use simulation to solve parts of these problems,
0:15:15 you have a very complicated simulation problem because you have to solve not just the sensor side,
0:15:17 but also the manipulation side.
0:15:24 You have to structure this problem in a way where you’re learning from the data you collect
0:15:27 in order to actually capture the nuances of how do you actually interact with the environment.
0:15:30 You have to then control it and execute it the right way.
0:15:32 You have to think about how do you actually define the goal?
0:15:36 So from the user point of view with an excavator, like, how does it look?
0:15:39 Like, I just, I want to dig a hole of this size at this spot?
0:15:40 Or like, what is it?
0:15:41 Yeah.
0:15:42 And this is a journey.
0:15:45 Like, fast forward, you know, next year where a customer does this, right?
0:15:51 So what they would do is they would give it a model of what they want the earth to be dug towards.
0:15:55 And so that would be like a 3D representation of the depth.
0:15:59 So width, length, depth, here’s the edges, here’s the constraints.
0:16:01 The pickups for the trucks are going to be over here.
0:16:02 Okay.
0:16:03 And then go to work.
0:16:07 And basically, these projects, like these machines will work for many months at a time,
0:16:11 continually just digging earth and working it towards the right foundation.
0:16:15 And the system will respect that boundary and basically dig down to that level.
0:16:18 It’ll have precision to the edges that you’ve defined.
0:16:22 It’ll do it in a sequence that makes sense so that you don’t trap yourself in a hole, for example.
0:16:27 And then you specify where you’re going to have dump truck pickups.
0:16:31 And there’s projects where this happens for nine months straight or 12 months straight with, like, many machines.
0:16:36 And so what you basically need to specify is what you want to dig to.
0:16:43 And what you want to dig to ends up basically being the foundation that you’re working towards for whatever you’re going to construct there.
0:16:45 And we want that to be versatile.
0:16:48 So sometimes you’re just taking off a layer of topsoil.
0:16:49 Sometimes you’re digging eight feet deep.
0:16:53 Sometimes you’re taking an existing stockpile and moving it somewhere else.
0:16:55 So there’s a lot of permutations of this.
0:17:00 Sometimes you’re taking rubble from a demolition job and loading it onto trucks, right?
0:17:05 And so what’s nice is that you start to see patterns over and over again in this sort of work.
0:17:10 We’ll be back in just a minute.
0:17:23 Run a business and not thinking about podcasting?
0:17:23 Think again.
0:17:28 More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora.
0:17:33 And as the number one podcaster, iHeart’s twice as large as the next two combined.
0:17:36 So whatever your customers listen to, they’ll hear your message.
0:17:40 Plus, only iHeart can extend your message to audiences across broadcast radio.
0:17:41 Think podcasting can help your business?
0:17:43 Think iHeart.
0:17:45 Streaming, radio, and podcasting.
0:17:48 Call 844-844-IHEART to get started.
0:17:50 That’s 844-844-IHEART.
0:17:55 So the data problem is interesting here, right?
0:17:59 Like you were talking about the physics, just the physics of an excavator is quite different, right?
0:18:02 Like it’s pushing down on the ground, which is pushing back on the excavator.
0:18:05 And like, as you say, the ground is changing because of its work.
0:18:08 And there’s not…
0:18:09 Well, where do you get the data?
0:18:11 Data is actually an interesting mix.
0:18:16 We did it both on our test sites and then also with our design partners.
0:18:20 So we’re working with general contractors and subcontractors.
0:18:24 Today we have five that we’re partnered with across southern states like Arizona, Texas.
0:18:25 And so we’re getting data on their sites.
0:18:27 We’re getting data on our sites.
0:18:29 It’s not that much data, right?
0:18:35 Like, I mean, I guess my reference point is always like ImageNet or, you know, the internet for large language models.
0:18:42 It does seem like a sort of recurring problem in robotics type AI applications is sparsity of data.
0:18:43 How do you get it?
0:18:43 Yeah.
0:18:44 Yeah.
0:18:47 And so this is where there’s a few kind of interesting things.
0:18:51 So first of all, like our partners together have thousands and thousands of machines.
0:18:53 And so there’s a lot of choices of which you’re a partner on.
0:18:57 The other thing that you can do is actually you can be very clever on a test site.
0:19:03 And when you’re on a real project, you’re kind of getting an unbiased sample.
0:19:06 You’re just getting a random distribution of the things you see, just like driving around on a road.
0:19:11 When you’re on a test site, you can actually upsample the things you actually want.
0:19:21 And you can go and you can collect 10 hours of data that’s representative of 5,000 hours of random data, which is particularly useful for things like safety situations, right?
0:19:27 You can actually create a much larger equivalent amount of data on a closed course through like kind of structured testing.
0:19:33 And so, for example, safety scenarios where you have weird interactions with people doing things they shouldn’t do.
0:19:35 You don’t wait to see that on an open site.
0:19:37 What is one of those?
0:19:41 Like what is your nightmare human behavior scenario in the field?
0:19:43 Nightmare human behavior is a human is curious.
0:19:45 They walk up to your machine.
0:19:46 Your machine stops.
0:19:47 And then they get really, really close.
0:19:50 And they’re now in a blind spot where you can’t see them.
0:19:52 But you still have to be smart enough to track them.
0:19:55 Or they’re in a hole in front of you while you’re like thinking about digging, right?
0:20:01 So occlusions from humans is probably a huge category, which is complex.
0:20:04 Occlusions meaning in your blind spot.
0:20:06 Humans in a place where the machine can’t see them.
0:20:06 Yeah.
0:20:11 Or usually your number one priority is anything that touches on human safety.
0:20:11 That’s sacred.
0:20:13 You never take any risks on that.
0:20:15 What’s something you haven’t figured out yet?
0:20:17 The things people do with excavators.
0:20:22 Like we’ve seen them do bizarre like clearing of debris.
0:20:26 We’ve seen them load a wheel loader with dirt.
0:20:30 We’ve seen them bang tools to change them.
0:20:30 We’ve seen them.
0:20:32 Wait, bang tools to change them?
0:20:33 What’s that one?
0:20:37 Like a tool gets stuck and they like bang it on the ground in order to get it loose.
0:20:43 They use their arm as a pivot point to turn when the wheels are like stuck in mud.
0:20:44 It’s kind of baller.
0:20:45 Like it’s pretty baller.
0:20:49 Yeah, it’s like, I mean, like when you see the like expert operators, they just show
0:20:50 off and it’s like, it’s incredible.
0:20:51 Like they’re awesome.
0:20:56 And so there’s like all these like weird subtleties of how they’ll use these tools in like really
0:21:01 subtle ways, which like the dimensions of on the product side of the use cases that blew
0:21:02 us away.
0:21:08 We thought about the obvious ones, but as we started like really going deeper and like studying
0:21:11 this, there’s so much diversity and interesting things that you can do with these machines.
0:21:12 It’s quite powerful.
0:21:17 I mean, presumably you don’t have to do all those to have a product people will pay you
0:21:18 for, right?
0:21:25 You can just have the like competent automaton that can dig a big hole just the way you want
0:21:25 it.
0:21:26 That’s correct.
0:21:30 And that there’s a huge amount of work even in those areas, but there is this like tail
0:21:32 that you go when you’re over time, go and have more and more capability.
0:21:35 And the nice thing is there’s just software updates and you get more.
0:21:39 It’s just like a, you know, the product ends up being the digital operator, which gets better
0:21:40 over time.
0:21:42 And so it’s similar to human getting better.
0:21:44 What’s the business model?
0:21:46 So we will go operator out next year.
0:21:52 So that’ll be our first like fully operatorless product that is in a spirit of like what this
0:21:53 is meant to scale as.
0:21:56 Operator out means driverless, means autonomous.
0:21:57 Is that what it means?
0:21:57 Yeah.
0:21:58 No, nobody there.
0:21:59 It’s just like operatorless.
0:21:59 Yeah.
0:22:03 And so our product is the digital operator.
0:22:07 So what I mean by this is that our customer is a general contractor or subcontractor.
0:22:10 So it’s the companies that already buy and manage these machines.
0:22:14 We are, we sell them a retrofit of these machines.
0:22:17 So we install an upfit that adds sensors and compute.
0:22:20 So the contractor already owns the excavator.
0:22:21 That’s right.
0:22:24 They’ve already bought a $500,000 machine or $300,000 machine.
0:22:27 And so this is an upgrade that enables autonomy.
0:22:30 And then our business model is selling labor.
0:22:36 So we’re effectively selling the labor and a variety of digital services around it that
0:22:37 operates this machine.
0:22:41 And for the surface area of types of tasks that it’s approved for, it’s completely driverless.
0:22:44 And then that surface area increases over time as software updates.
0:22:47 And for everything else, it can still be manually operated.
0:22:50 So is the core product, are they paying you by the hour for digging?
0:22:54 So we’re figuring it out, but it’ll be something that is either by the hour,
0:22:56 by the project, by subscriptions.
0:23:04 So it’s effectively the way that today projects are forecasted and billed by shifts for labor.
0:23:09 It’s a parallel of that, but it becomes a huge win because you have complete flexibility.
0:23:11 You can work 10 hours a day.
0:23:12 You can work 24 hours a day.
0:23:17 So there’s all sorts of benefits that operationally give you a lot of leeway on how you use it.
0:23:19 What might go wrong?
0:23:21 What might go wrong?
0:23:23 I’ll tell you the things that are really challenging.
0:23:30 The nuance and diversity of things are high.
0:23:36 As much as you want to just boil it down to a very simple dig and load sort of operation,
0:23:38 there’s always little corner cases.
0:23:40 There’s things you find in the ground.
0:23:44 There’s weird ways that trucks could interact with you.
0:23:45 There’s things people will do.
0:23:50 There’s varieties of machines, but there’s like 50 kinds of excavator models and sizes and so forth.
0:23:53 So I think the long tail is still challenging.
0:24:02 Presumably, you don’t have to figure out every edge case, but you probably have to figure out a lot for your thing to work in a functional sense, right?
0:24:07 In the meantime, you’re not just digging and loading.
0:24:08 You’ve got to reposition.
0:24:09 You’ve got to organize the earth.
0:24:10 You’ve got to think about the sequencing.
0:24:13 You’ve got to deal with daytime, nighttime, rain.
0:24:19 And so you have these types of really challenging diversities you have to think about and deal with.
0:24:27 So I think all in all, it’s still a complicated product area where there’s a huge amount of diversity of the things that need to be done.
0:24:41 But it’s one of those where I personally think there’s a handful of these holy grails of autonomy and physical industries that are like genuinely transformational opportunities for both positive impact to the country and the world,
0:24:45 and also just kind of scale of industries that are like double-digit percentages of GDP.
0:24:47 Transportation is without a doubt one of them.
0:24:49 Construction is one of them.
0:24:51 Agriculture is not far behind.
0:24:53 You know, mining is very, very significant as well.
0:24:55 Manufacturing is one of them.
0:25:06 And so I think that we’re going to see a wave over this next 10 years in autonomy, but it’s going to be tackling this like 75% of the world’s GDP that’s physical and not digital.
0:25:10 And there’s a lot of work, like a lot of positive impact that can happen across these spaces.
0:25:12 I mean, give me a little more on that one.
0:25:16 Pick a time in the future, five years, 10 years, not more than 10.
0:25:30 So my personal belief is that this idea, like there’s a lot of companies getting flooded for this, but the idea that like this giant brain for all of robotics, the foundation model for robotics, like I personally do not believe that that’s viable in the next 10 years.
0:25:51 Because you have such complexity in really understanding these verticals on the inputs, the hardware, the products, the neat use case, the customers, everything, that every single one of those has such complexity to get data, that the idea to get enough data that bulldozes through the generalization problem, that’s a long ways off.
0:26:03 But I do think that we’re at a perfect time for these vertical solutions, where if you have a focused solution where you’re trying to do construction, you’re trying to do fulfillment in a warehouse, you’re trying to do a very focused manufacturing solution.
0:26:08 I think that’s actually, there’s a giant step function in how powerful ML technologies are.
0:26:14 But if you’re trying to do a humanoid to do everything in a home for a consumer market, too far away.
0:26:15 Right.
0:26:18 Now, you’re saying you basically, you got to solve one problem at a time.
0:26:19 Yeah.
0:26:22 I want to go back to open road autonomy.
0:26:22 Almost done.
0:26:32 When do you think that most rides most people take in cars or trucks will be in driverless cars, autonomous vehicles?
0:26:33 Great question.
0:26:36 There’s a few prerequisites for most rides.
0:26:40 So first, all the geofences have to disappear, and the thing just works everywhere.
0:26:46 That’s on a trajectory of getting there because it’s getting more and more efficient as the scales across the country and then the world.
0:26:49 It has to go from ride hailing to personal car ownership.
0:26:55 That is both cost down and versatility and everything that, like, but that’s going to happen.
0:26:59 In that universe, when do they take the steering wheel out?
0:27:01 I just took a Waymo for the first time.
0:27:02 I was in San Francisco this summer.
0:27:03 It was awesome, right?
0:27:04 It was awesome, yeah.
0:27:08 And, like, the weirdest thing about it to me was the steering wheel.
0:27:08 Yeah.
0:27:08 Right?
0:27:14 Like, if it was just a box that I got in that looked like a train car or something, I would have been like, oh, sure, it’s like the air train or whatever.
0:27:17 But that there’s a steering wheel and it’s like a ghost is turning the steering wheel.
0:27:19 Like, what’s the steering wheel doing there?
0:27:25 So, the steering wheels will disappear pretty quickly because ride sharing autonomous cars have no use for it.
0:27:26 And then you need to wait.
0:27:27 Yeah, there’s, like, special cases.
0:27:28 We need to recover them.
0:27:30 But it’s a wasted space, right?
0:27:31 It’s a wasted spot.
0:27:33 So, that’ll happen soon.
0:27:38 But the idea of really deep penetration, I think it’s personal cars.
0:27:43 It’s beyond luxury, which is another generation, to, like, actually make it be something that’s affordable.
0:27:47 Then you’ve got to go through a buying cycle, which is, like, five, seven years.
0:27:52 And so, I think it starts to get serious penetration in the back half of the 30s.
0:27:59 And it’ll be the 40s where, like, okay, 50% of driving is kind of, like, autonomous.
0:28:01 I think it’s still 20 years off in my mind.
0:28:03 For example, you bought a car today.
0:28:06 It’s going to be in circulation for the next 12 to 15 years.
0:28:07 It just takes a while.
0:28:14 We’ll be back in a minute with The Lightning Round.
0:28:27 Run a business and not thinking about podcasting?
0:28:28 Think again.
0:28:33 More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora.
0:28:37 And as the number one podcaster, iHeart’s twice as large as the next two combined.
0:28:40 So, whatever your customers listen to, they’ll hear your message.
0:28:44 Plus, only iHeart can extend your message to audiences across broadcast radio.
0:28:46 Think podcasting can help your business?
0:28:47 Think iHeart.
0:28:49 Streaming, radio, and podcasting.
0:28:52 Call 844-844-IHEART to get started.
0:28:55 That’s 844-844-IHEART.
0:28:58 Okay, let’s finish with The Lightning Round.
0:29:02 If you could operate any machine, what would it be?
0:29:05 Oh, gosh.
0:29:06 This is a fun one.
0:29:10 If I could operate any machine at all in the world, not even in construction.
0:29:12 Oh, gosh.
0:29:12 Okay.
0:29:13 You know what?
0:29:18 I would love to either operate one of the, like, boring company gigantic drill machines.
0:29:18 Oh, good one.
0:29:19 Because they’re just, like, so astronomical.
0:29:25 Or there’s, like, mine trucks that are so astronomically big that they just dwarf any machine in the world.
0:29:26 And it costs, like, $5 million.
0:29:30 And just a skill that would be really, really fun to try at some point.
0:29:31 Just to be that high.
0:29:34 Just to have that much momentum, right?
0:29:36 That much mass at your disposal.
0:29:37 Yeah.
0:29:39 Like, literally, the tire is, like, three stories tall.
0:29:40 It’s, like, it’s wild.
0:29:41 Like, it’s absurd.
0:29:47 What’s one thing you remember about immigrating from the Soviet Union to the U.S. when you were, was it six?
0:29:48 I was six.
0:29:48 Yeah.
0:29:50 I was born in Moscow.
0:29:50 We immigrated.
0:29:51 I was super young.
0:30:00 I remember we ended up having a pit stop in Europe where there’s a standard path of going to Vienna, Venice, and Rome while your paperwork gets processed.
0:30:01 And then we went to New York.
0:30:12 I remember running around the rooftops of Venice with a friend of mine causing a bunch of, you know, trouble and running away and disappearing for long periods of time and having a blast.
0:30:20 And for whatever reason, running around the rooftops of Venice was ingrained in my memory, which was kind of a very positive memory while my parents were going through a whole bunch of stress.
0:30:25 Sounds very free, not to project East versus West language onto it, but it sounds very free.
0:30:27 It was very freeing.
0:30:28 It’s funny.
0:30:35 I just hit the age my dad was when they immigrated from the Soviet Union with two kids.
0:30:41 My sister was one month old, zero money, almost no English, having a fresh start.
0:30:42 So courageous, right?
0:30:44 Doesn’t it seem so brave?
0:30:46 Yeah, it does.
0:30:48 And they were trying to leave for like 10 years and couldn’t leave.
0:30:49 Yeah, it’s kind of fascinating.
0:30:52 So we’re very fortunate to not have to go through something like that.
0:30:56 Is that when you think about it that way, does that like put pressure on you?
0:30:58 You think, oh, my God, my parents did all this.
0:31:00 Like, I better, I better deliver.
0:31:02 Oh, it’s funny.
0:31:03 A little bit.
0:31:09 I mean, you kind of have this, I don’t know, like adventurous spirit, I guess, that maybe gets ingrained.
0:31:16 I think of it now for my kids where I’m like, okay, like now they’re in a really nice and comfortable environment growing up.
0:31:19 How do you like convey that edge and a little bit of that spirit?
0:31:21 How do you keep them from going soft?
0:31:21 Yeah.
0:31:28 And it’s not that you want them to like go through anything difficult because it’s not that I was, you know, for me, it was actually just an adventure.
0:31:29 My parents was difficult.
0:31:38 But part of it is just, yeah, like conveying that spirit of being able to like be comfortable trying to tackle something new and being thrown in a completely different environment.
0:31:43 It’s hard to force that or simulate that when you’re just growing up in the San Francisco area, right?
0:31:44 I appreciate your time.
0:31:45 Thanks for talking to me.
0:31:46 It’s a pleasure.
0:31:47 This was a lot of fun.
0:31:48 Thanks for having me.
0:31:59 Boris Hoffman is the co-founder and CEO of Bedrock Robotics.
0:32:06 Just a quick note, this is our last episode before a break of a couple of weeks, and then we’ll be back with more episodes.
0:32:10 Please email us at problem at Pushkin.fm.
0:32:13 We are always looking for new guests for the show.
0:32:17 Today’s show was produced by Trina Menino and Gabriel Hunter-Chang.
0:32:22 It was edited by Alexandra Gerriten and engineered by Sarah Bruguer.
0:32:26 I’m Jacob Goldstein, and we’ll be back next week with another episode of What’s Your Problem?
0:32:34 This is an iHeart Podcast.
Boris Sofman is the co-founder and CEO of Bedrock Robotics.
Boris’ problem is this: How do you teach machines not just to drive, but also to work: to grade roads, move heavy objects and dig big holes at construction sites.
On today’s show, Boris talks about how his work at Waymo led him to found Bedrock, and he explains how autonomous construction equipment could help unleash an American building boom.
See omnystudio.com/listener for privacy information.