Autonomy Across Air, Land, and Sea

0:00:06 There are now drones here in the Bay Area that are flown from somebody that’s 20, 30, 50 miles away.
0:00:13 Mining has had some form of trucks without any drivers in them since 2007-2008.
0:00:17 China outnumbers our shipbuilding capacity about 200 to 1.
0:00:21 Immediately, every Autel drone was just bricked in Taiwan.
0:00:29 Basic assumptions about how software is built for the kind of traditional SaaS world of the 2010s
0:00:32 just doesn’t work in the autonomy space.
0:00:35 You can actually create a really, really incredible reconstruction of the world
0:00:37 just using these video generative models.
0:00:39 And this is not hype-gen AI stuff.
0:00:43 The cars we buy in US, in Europe, etc.
0:00:48 They’re not delightful consumer products like when you brought your first iPhone.
0:00:51 This has been a big year for autonomy.
0:00:56 For example, the fully autonomous Waymo driver has done over 20 million miles.
0:01:03 The equivalent of driving to the moon and back 40 times, and is now doing more than 100,000 rides per week.
0:01:05 But it’s not just autonomy on land.
0:01:12 For example, the FAA granted several operators the ability to fly commercial drones without visual observers earlier this year.
0:01:14 And this is only just the beginning.
0:01:22 Now, in this live recording from SF Tech Week, we brought in experts from three domains, Air, Land and Sea, to discuss autonomous systems.
0:01:26 And we touched on the real world deployments, the latest chips and their impact on the economics,
0:01:32 building full stack, quantifying risk, and regulations role in advancing this frontier.
0:01:37 Moderating this panel was a 16Z partner, Aaron Price Wright, along with three panelists.
0:01:42 First up, we have Makario Nemi, Chief Marketing Officer of Skydio.
0:01:45 Skydio is a 10-year-old company based here in San Mateo.
0:01:49 We provide quadcopter drones, more specifically camera drones.
0:01:51 We shipped about 50,000.
0:01:54 We started in consumer, especially for outdoor enthusiasts.
0:02:01 We’ve transitioned exclusively to selling into private enterprises, state and local governments, as well as the federal government.
0:02:08 Next, we had Vijay Patnaik, Head of Product and Applied Intuition, but also previously spent five years at Waymo,
0:02:11 most recently as Head of Product of their self-driving truck division.
0:02:18 Applied Intuition is focused on providing developer tools and software to companies that are building wake-ups.
0:02:24 We provide simulation and data tools that are necessary for building autonomous systems.
0:02:31 And finally, Peter Bowman Davis, an engineering fellow now at A16Z, but previously worked on machine learning at Soronic.
0:02:34 Soronic is a full-stack maritime autonomy company.
0:02:38 We build boats that are autonomous and we sell directly to the government,
0:02:44 so that comprises everything from the whole manufacturing to the simulation side on the autonomy.
0:02:47 We also had some amazing questions from our live audience.
0:02:49 So stay tuned for that at the end.
0:02:53 And of course, if you’d like to attend events just like this in the future, make sure you’re subscribed.
0:02:56 And while you’re at it, leave a review.
0:02:58 All right, let’s get started.
0:03:03 As a reminder, the content here is for informational purposes only.
0:03:06 Should not be taken as legal, business, tax or investment advice,
0:03:13 or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund.
0:03:19 Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.
0:03:25 For more details, including a link to our investments, please see a16z.com/disclosures.
0:03:35 So I’m really excited because we have someone from the business, from the product,
0:03:39 and from the tech side here, as well as Air, Land and Sea.
0:03:48 So maybe to get started, I would love to hear how far along in the journey to autonomy you feel like we are in your industry,
0:03:54 maybe framing the levels of autonomy and where are we really all the way there and where do we have still some way to go.
0:04:00 So Skydio was sort of started on the foundation of autonomy and we did not invent the quadcopter.
0:04:02 Those existed prior to the founding of the business.
0:04:07 I think we’ve all experienced or seen them and even just walking around Best Buy sort of toy form,
0:04:12 where you have some sort of radio controlled mechanism to be able to tell the drone where to fly.
0:04:18 The challenge with the use of drones is that it always required a person to be there.
0:04:26 And if the more you go up in the use cases into more enterprise or government use cases, the more valuable the flying becomes.
0:04:33 And so if you’re going to go fly around a substation that belongs to an energy utility and you go crash that drone,
0:04:35 you could take out power into an entire neighborhood.
0:04:39 That’s not just something that’s cute that your kids did on Christmas afternoon.
0:04:43 That’s actually something that would cause you to lose your job because real problems for the community.
0:04:50 So the necessity for the person is not just to be there, but they have to be an expert pilot.
0:04:54 They have to be great at being able to fly in these really challenging environments.
0:04:56 And that was the premise of Skydio’s.
0:05:02 Can we build the skills of an expert pilot into the drone itself so that any one of us here in this room can actually pick up the drone.
0:05:07 It would be reasonably proficient and they become sort of democratized in terms of their access.
0:05:15 So we started day one from building an autonomy and the first iteration of that was the ability to follow somebody.
0:05:24 Since then, we’ve sort of upped our game quite a bit, not only beyond obstacle avoidance, but to be able to not have a person be present at all.
0:05:33 So going back to that substation example, there are now drones here in the Bay Area that are flown from somebody that’s 20, 30, 50 miles away.
0:05:36 And those drones complete automated inspection missions.
0:05:37 So we’re getting there.
0:05:43 The regulatory environment doesn’t allow us to just completely eliminate the person, but we’re certainly making quite a bit of progress.
0:05:53 In the automotive space, which is passenger cars at the lower levels of automation, what we would think of as for a user hands on system where you’re still paying attention to the vehicle.
0:06:00 Your hands are on the steering wheel, but the vehicle can do things like automatic emergency braking or what’s called cruise control.
0:06:05 Those systems I think are available on sort of any car you go and buy today in Europe.
0:06:08 Some of those are mandatory from a regulation standpoint to be there.
0:06:22 I think US regulations are slightly behind in mandating some of those systems, but generally those are available on vehicles where the next focus area for a lot of the OEMs is on what’s called sort of level two plus systems, which are these hands off system.
0:06:24 So as a user, you don’t need to keep your hands on the steering wheel.
0:06:27 You still need to pay attention and keep your eyes on the road.
0:06:30 And one level beyond that, but it’s eyes off as well.
0:06:32 So you can read a book, you can watch a movie.
0:06:38 Those systems aren’t yet deployed. That’s in R&D phase right now, just very early deployments right now.
0:06:46 And then on that level four side, I think in the city in San Francisco, we’ve all seen sort of Vemos robot axes and that’s the sort of main deployment on the car side right now.
0:06:50 And I think there are some deployments in China that we see on the level four side as well.
0:06:53 Taking a different industry, I think maybe construction and mining side.
0:07:01 What’s interesting is mining has had some form of trucks without any drivers in them since 2007, 2008.
0:07:06 It’s not the same level of autonomy as Vemo in the sense that those trucks are following up predefined paths.
0:07:10 They get a lot of support from the infrastructure in the mine in order to do that.
0:07:13 But there’s a clear business case and an ROI for that product.
0:07:20 And hence the mining industry has been investing in that the technology itself is far behind what’s available on today in automotive.
0:07:24 So there is a big focus to upgrading that technology and serving more of the use cases.
0:07:28 Maritime I would say takes a lot of inspiration from the self driving car community.
0:07:35 And the reason for that is effectively when maritime you have this kind of two dimensional plane that you’re moving around very similar to self driving cars, right?
0:07:36 You just have lateral planning.
0:07:40 It’s maybe a little bit more complex because you have the longitudinal axis as well.
0:07:43 And you also have much less things in your environment to ground you.
0:07:48 So when you’re thinking about like an autonomy model, the world doesn’t usually just like nicely translate around you.
0:07:53 Think about driving a car in a nice kind of like city skyscraper block.
0:07:56 As you take a left, the world nicely transforms around you.
0:08:00 But imagine you’re in the middle of the ocean and you take a left.
0:08:03 The ocean looks the exact same.
0:08:10 And so you actually get a lot less information per frame when you’re making these sort of like vision models or you’re making a sort of autonomy stack.
0:08:16 And then the other thing that I wanted to draw is the delineation between a perception stack and an autonomy stack.
0:08:20 For a lot of these use cases for us at least perception has mostly been solved.
0:08:23 That is to say object detection, object avoidance.
0:08:30 These are like very, very simple tasks and they can mostly be done with CNNs, which are 20, 30 years old.
0:08:36 Autonomy is a bit trickier because you actually have to take kind of elements from the perception stack and you have to make them actionable.
0:08:39 And that is to say you have to do a lot of long term planning.
0:08:42 You have to actually take in that data and make decisions based on it.
0:08:45 And that’s something I think that’s only been enabled in the last five.
0:08:47 Maybe you can make the argument even two years.
0:08:52 And so I’m super excited about a lot of the work that’s going into reinforcement learning and long term planning.
0:08:58 Because this is super, super important in the maritime domain because in maritime, you’re not often near the shore.
0:09:03 And in the defense application, you’re not often in communications with back home.
0:09:05 And so you need to make a lot of independent decisions.
0:09:09 And so these are the things that we’re thinking about in the maritime autonomy space.
0:09:12 Double clicking on some of those sort of technical breakthroughs.
0:09:17 Where do you think the latest developments in AI have impacted autonomy?
0:09:24 To what extent have they impacted autonomy and how much change are we seeing in the industry today versus let’s say five or 10 years ago?
0:09:32 Just based on the current state of the art and things like video language models or other new transformer based architectures that might not have existed.
0:09:36 For example, when Waymo got started or when Skydio got started.
0:09:41 Yeah, I think pretty significant impacts actually in terms of the latest developments.
0:09:46 I think foundation models do have an impact on the architecture of these autonomy systems on these ground based systems.
0:09:52 Where you think of foundation models as replacing a number of more task specific models that existed.
0:09:59 So we see more and more companies making the shift towards using or at least researching with how foundation models can be used.
0:10:04 The quote unquote frontier of research right now is end to end.
0:10:11 Even when I joined Waymo back in like 2016, there was a lot of hype around end to end driving, but it didn’t really materialize the technology wasn’t ready.
0:10:17 So now it’s almost a second version of that happening now where the research has progressed a little bit further.
0:10:23 So I do think the architectures are going to evolve significantly even though they’re not yet ready for production.
0:10:28 What that also means is that the tools and infrastructure needed to support that is evolving pretty quickly.
0:10:35 And so we are developing new kinds of simulators that are needed in order to support those advanced architectures.
0:10:43 And those simulators themselves require many of these generative techniques or neural rendering techniques that have shown good promise in research.
0:10:48 And third, I would say is just using some of the generative AI to simplify workflows.
0:10:56 As an example, if you’re not on the engineer, you spend a lot of time working with simulation and scenarios, but you don’t want to write a lot of complicated scenarios.
0:11:04 But we could generate them for you programmatically in a much more confident manner today than what was possible two or three years ago.
0:11:10 So just at different layers, whether it’s the autonomy stack, whether it’s the tools being used to develop that autonomy stack.
0:11:15 I think we see pretty fundamental impacts and that’s why we are investing pretty aggressively in utilizing these technologies.
0:11:19 Just double clicking on the idea of simulation for robotics or autonomy.
0:11:24 I think this is like a very non obvious point to people who haven’t worked in this industry for the last two years.
0:11:34 Because I think that we’ve started to use video generative models as dropping replacements for simulators like Unity or Ross or Unreal Engine.
0:11:42 And the reason you need a simulator to be clear is because, first of all, robots are really, really expensive and you don’t want to take them out to the field to have them break if you mess up the algorithm, basically.
0:11:51 And you also need many environments where you need to train on policy, meaning you basically need to run it in real time in order to make sure the thing works.
0:11:57 And so in recent years, they’ve started to use video generative models conditioned on actions or sensor data.
0:12:02 And you can actually create a really, really incredible reconstruction of the world just using these video generative models.
0:12:08 And this is not hype Gen AI stuff. This is used by Tesla, this is used by Wave, it’s pretty exciting stuff.
0:12:12 And I think that some people are being very quiet about it, but there’s some really awesome public releases.
0:12:25 I will say the rise of NVIDIA has been pretty instrumental for our kind of business to be able to put that kind of compute power in a four and a half pound form factor that can be sold for basically $11,000, $12,000 would still make some margin.
0:12:27 And that’s a very hard thing to do.
0:12:33 I’m curious, how do you think about the cost aspect when using the high performance chips that NVIDIA comes out with?
0:12:39 How does the sort of ROI for the business change when you’re putting the latest GPUs or compute onto the drones?
0:12:45 It’s so foundational to our business. We have to have high computes on the drones themselves.
0:12:51 What we did in our latest release that we announced about a year ago was we actually added excess capacity.
0:12:56 So basically trying to feature proof the hardware so we can do more over time. We haven’t fully used it.
0:13:03 We expect to be able to run additional models, including models that our customers build on the drone itself because they can do much deeper analysis on the things that matter to them.
0:13:07 What’s an example of a custom model that a customer might want to build?
0:13:14 So object detection is a really simple one, as you mentioned, but in our world we see everything.
0:13:21 It’s not just about, hey, people, right? It’s like, no, I want to be able to identify that particular transformer.
0:13:26 And by the way, PG&E’s transformers are different than Southern California Edison’s transformers, which are different than Baltimore Gas and Electric Transformers.
0:13:34 So they know their transformers and then once we actually identify the transformer, we need to be able to determine whether the thermal signature is actually saying there’s a problem or not.
0:13:44 That is so specific. We’re not the best ones to be able to build that. So that would be an example of how customers can build their own stuff and then basically run it so we can take immediate action on the drone itself.
0:13:50 That’s why we put in that excess capacity is to be able to do those kinds of things that have that kind of extensibility and platform.
0:14:01 Very cool. Actually, that brings me to another question, which is for Ceronic and Skydio, both companies are essentially building the full vertical stack where not just the algorithms around autonomy and perception,
0:14:09 but actually the full kind of product that goes into the world. Whereas Applied, you’re much more of a software provider working with companies who are building the hardware.
0:14:21 I’d be interested to hear some of the kind of trade-offs or what do you gain by owning the full end-to-end system and what’s harder as a result of having to manufacture and deliver the final product.
0:14:28 We think that the ultimate value of a vertically integrated stack is reliability. We’re very committed to it. We started with it.
0:14:36 And in fact, our primary competition in the world is DJI, the largest manufacturer of drones out there, and they are the complete opposite.
0:14:42 They’re happy to build the hardware and there’s a whole plethora of other companies to sort of build their ecosystem around DJI hardware.
0:14:48 And we see that those seams actually start to fracture when you get more and more complicated scenarios.
0:14:55 And so you add in autonomy there. What happens in the corner cases? What happens in the failure modes? How do things react?
0:15:00 That’s where we could completely control that and we can test for it and we build reliability around it.
0:15:07 So I think the downside is that we might go slightly slower, right? Because we do have finite resources.
0:15:16 We have to choose where our investments are. Whereas another company that may be all of what they do is to be able to build some kind of capability on top of another person’s hardware.
0:15:21 For us, we are okay to move a little slower to be able to deliver higher quality.
0:15:25 We’re kind of like a weird middle ground, I would say, because we own the full stack.
0:15:29 We also, though, have three types of boats, like small, medium and large, basically.
0:15:34 And these boats are very different in terms of the amount of cameras they have, the actuation techniques you use.
0:15:37 That being said, we still own all the hardware, especially all the compute.
0:15:44 And so the compute is mostly homogenous between them, but your actual kind of effector systems or your perception stack is going to be differentiated.
0:15:49 And it’s also differentiated in between boats sometimes, because we want to test things out, we want to add another camera or something.
0:15:54 And I would say it’s actually a powerful thing to be a little bit agnostic on the hardware and the software,
0:16:02 because it basically pushes you to create really, really solid abstractions for your perception stack or for your autonomy stack.
0:16:06 And it kind of pushes you to be a better engineer. This is at least what I’ve experienced in my engineering team.
0:16:10 We really liked to think about, okay, how is this going to work on boat N+1?
0:16:15 And so that’s been kind of a fun challenge to work on, but I do agree that it does slow things down sometimes.
0:16:18 Yeah, I completely agree with that point of abstraction.
0:16:21 So Applied started with being a tools provider.
0:16:26 So we would provide the simulation and data products and eventually our customers were like the tools are great,
0:16:30 but we don’t have the internal capability to build that autonomy stack using the tools.
0:16:31 Can you help us with that?
0:16:33 So that’s how Applied actually started.
0:16:35 We now provide an off-road autonomy stack.
0:16:42 We also have a trucking stack and to the point that Peter was making now more so than ever in autonomy.
0:16:48 I think these abstractions are possible where we have a stack that it’s not built for a single customer.
0:16:53 It’s meant to be reusable across customers and they can build their eventual application on top of it.
0:16:55 So if you only want to use the tools, that’s great.
0:17:01 But if you want an actual autonomy stack, either as your primary stack or to get kickstart your efforts internally,
0:17:09 there are providers like Applied or other ones that you can use with the right abstraction to significantly accelerate your program.
0:17:16 How involved is that sort of translation process to different form factors, vehicles, customer types?
0:17:22 I’m curious to hear how hard it is to get autonomy that works for one type of vehicle to work in a totally different setup.
0:17:26 I mean, I would love to tell you that it’s like seamless and just happens all by itself,
0:17:30 but there’s the engineering aspect of it and there’s the organizational aspect of it.
0:17:34 I think engineering to a certain extent, you can engineer the best APIs and abstractions.
0:17:41 There’s still customization that you would need to do, especially if it’s for the first time being deployed to different platforms.
0:17:48 But the organizational challenge is even more interesting because for some of the companies that we are working with,
0:17:56 they’re going through this big shift of going from hardware companies because vehicles were primarily hardware driven to becoming software companies.
0:18:03 So they’re reinventing themselves, hiring software engineers, hiring engineering leaders from Silicon Valley, etc.
0:18:09 And you actually need a very close partnership to make this work, especially if there’s various different vehicle form factors.
0:18:17 So it’s not just an engineering problem, it’s also how do you build up internal capabilities for customers who often will work with them
0:18:24 on like training their internal team, etc. in order to be able to work together on this transition of the stack to many different platforms.
0:18:30 Makes sense. I spent most of my career before investing in Palantir, so I’m very comfortable with that type of model.
0:18:37 What was your experience there in terms of how to help companies sort of make that shift to whether it’s autonomy, whether it’s software, data?
0:18:41 It wasn’t necessarily strictly autonomy, but I think that there are a lot of similarities.
0:18:45 The biggest thing was how you embed with the customer to drive cultural change,
0:18:54 which is the Ford-deployed engineer model really invented at Palantir where I would be flying out to Azerbaijan or somewhere in Oman or Trinidad
0:19:00 or really far-flung places to help their engineers really on the ground in the field figure out how to use the software.
0:19:02 And so maybe switching gears a little bit.
0:19:08 I’d love to talk a bit about the economics of autonomous systems, also how that relates to regulation.
0:19:14 So we’ve heard and read in the news a little bit lately about Waymo and the economics of a Waymo ride.
0:19:21 And it’s really exciting to see it start to actually make sense so that you’re not just pouring money into the system for the sake of R&D.
0:19:30 But every single Waymo ride is actually, I think, profitable if you don’t count the overall cost of the vehicle, which is a big cost to discount.
0:19:37 So I’d love to hear maybe starting with Skydio, like how do you think about the economics of the drone industry, where you’ve seen real success,
0:19:41 where it’s been slower, and maybe also how regulation has played a role in that.
0:19:46 We serve multiple different industries and their ROI calculations are going to be slightly different.
0:19:49 I’ve used a couple of examples where we talk about energy utilities.
0:19:58 And in that instance, it’s about looking at the people and labor it takes to go inspect the equipment, whether that’s physical substations or transmission lines or distribution lines.
0:20:04 And anytime you can save the cost of actually sending a truck on site where they can put a person in a bucket, get a buck in the air,
0:20:10 which can cost $2,000 per deployment, and you can instead put a drone up and get the exact same information in 60 seconds.
0:20:13 Just sort of a pretty obvious ROI there.
0:20:16 We also serve public safety, like first responders, police departments.
0:20:27 And I think you can’t really put a value on human life, but you can put a value on the insurance payouts that come with officer involved shootings.
0:20:32 It can easily go to $1 to $2 million every time there’s some kind of use of force.
0:20:36 And so being able to avoid that, being able to cut that down in half is really, really key.
0:20:40 Ultimately, we’re in the business of providing information for people to make better decisions.
0:20:50 And if you’re in a really high stakes scenario, better information about where the threat is can really make a big difference in terms of whether you end in a peaceful way or a tragic way.
0:20:56 Certainly some elements of economics, but a lot of it is just keeping people safe, keeping their officers safe, keeping the community safe across the board.
0:21:03 And how about with the regulatory piece of sort of whether an operator has a line of sight on a drone?
0:21:11 How maybe take a particular use case like police officers or something, like how they think about kind of fleet expansion as those regulations change?
0:21:19 One of the key areas of investments right now is a concept called drone as first responder, where you would pre-position docking stations on the rooftops of cities.
0:21:33 And so this is happening in New York, it’s being tested in other cities where when a 911 call comes in, the drone automatically deploys and goes and is effectively the first responder, the first person on site and is providing eyes back to a remote pilot.
0:21:34 That does two things.
0:21:43 One is the pilot is sitting in some nicer conditioned office and they can immediately relay information back to any responding officers so they can respond more appropriately.
0:21:49 They can control many, many drones at the same time, so it becomes a one to many versus a one to one argument.
0:21:51 And a lot of calls can be cleared.
0:21:56 These 911 calls for service can be cleared by basically the drone saying there’s no real issue here.
0:21:59 You don’t have to go out on site and that’s tremendous economic savings.
0:22:08 In order for this drone as first responder to happen, you have to have a regulatory environment that allows for a lack of visual observer on the rooftop.
0:22:14 And right now, just the 22nd primer on FAA regulations, if you’re putting anything in the air, it is regulated.
0:22:17 The FAA cares about airspace and they mostly care about just not hitting anything.
0:22:19 Don’t hit an airplane, don’t hit a helicopter.
0:22:26 And so they require you to stay below 400 feet and they require a person to be visually looking at that drone at all times.
0:22:27 That’s like the most basic rules.
0:22:31 So now you have a scenario where you might go further up.
0:22:34 You don’t have the available staff to put a person on the rooftop to be able to look out.
0:22:40 And the second is if you’re in some place like New York City, the top 100 buildings are over 600 feet.
0:22:42 So how are you going to be keeping everything below 400 feet?
0:22:50 So being able to sort of work with the FAA to be able to create these kinds of waivers to this traditional regulatory environment that allows for beyond visual line of sight,
0:22:59 beyond visual line of sight without a visual observer and to be able to actually stay within 50 feet of structure so you can actually go up and over a building as long as you’re within 50 feet.
0:23:02 And that was a waiver that just came out actually two, three weeks ago in New York City.
0:23:09 And it’s a fantastic way of having the FAA sort of work with these agencies to do us right for the community while still keeping the air safe.
0:23:11 Exciting. Congrats.
0:23:13 I’d love to hear the applied intuition.
0:23:23 The economic case where you see where is this still sort of a slightly money losing R&D exercise versus where are you actually starting to see the economics make sense?
0:23:26 And where, if at all, regulation plays a role in that?
0:23:36 Yeah, I mean, on passenger cars, they’ve had driver assistance systems for a long time as these systems become more capable OEMs have an ability to charge more for them, right?
0:23:39 That’s why the FSD system from Tesla when it came out was over $10,000.
0:23:42 There’s an Avid Tesla fossil driving user.
0:23:43 There you go.
0:23:44 I get a test.
0:23:45 Yeah.
0:23:48 So there’s a certain willingness to pay if the performance of the system is up.
0:23:58 They’re not just discounted that. So I think the industry is in a price finding mech sort of stage right now where obviously the best thing would be higher performance while keeping costs the same.
0:24:04 But as the cost of compute goes up, more sensors go into the vehicle, more software being deployed onto it because they are more complex.
0:24:06 The bomb cost for the OEM goes up.
0:24:08 So I think there’ll be some price increases.
0:24:13 OEMs already operate on somewhat thin margin compared to the margins we are used to in the Silicon Valley.
0:24:16 So they’re not in a position to keep losing money on each of these systems.
0:24:22 So I think the market will eventually find a price that makes it profitable for the OEM to ship these systems.
0:24:27 On the, I think areas like trucks and construction mining have a different unit economics framework.
0:24:35 So if you think about trucks, for example, and I’ll tie in regulation into this as well, we can think of unit economics as reducing the current costs.
0:24:39 But a big part of unit economics also can you increase the revenue per truck or not?
0:24:41 So the first argument is, hey, there’s a shortage.
0:24:48 So we’re losing like supply chain flexibility, like we do some work in Japan in providing our autonomous truck technology in Japan.
0:24:58 The government is actually pushing the commercial vehicle sector to invest more in autonomy because they’re seeing they call it the 2024 problem where the drivers are aging out and they are overworked and have health issues.
0:25:00 So the shortage problem is real.
0:25:05 But even if you look beyond the sort of, okay, how do we reduce the labor cost and the insurance cost?
0:25:14 If you have a network of autonomous trucks, you can actually optimize the entire logistics network such that you’re generating more revenue per truck, which makes unit economics better.
0:25:17 The regulation plays a part in that is twofold.
0:25:23 One is in a place like the US, each state can actually regulate how these driverless trucks are deployed.
0:25:31 And so you can have this weird mix where the states that your truck is traveling through have slightly different regulations and federal government has different regulations.
0:25:33 So you do need some consistency.
0:25:42 So if you want to have a truck grow from LA to Atlanta, which is actually a pretty heavy freight route, then California has to have consistent laws with Arizona and Texas that allows that.
0:25:44 And I think it’s not fully there yet, but getting there.
0:25:51 The other side of it, there are regulations like hours of service that determine how long a truck and a truck driver can drive.
0:25:54 That’s for their safety from fatigue, et cetera, and for road safety.
0:26:07 So the government has to be willing to be flexible on those such that those trucks can then operate 24 hours such that you can then reconfigure logistics networks and then make the supply chain more efficient and make the revenue and the unit economics work.
0:26:16 So I think those are some of the factors in trucking and mining, I think is a different sort of set of factors in the sense that the ROI is somewhat already clear to the industry.
0:26:23 That’s why they’ve been investing in autonomy for a long time and the trucks we’re talking about in mining on which these autonomy systems are being deployed.
0:26:25 These are like a $5 million machine, right?
0:26:29 So it’s not a big deal if you put a $100,000 lidar on it.
0:26:35 Of course, it adds $100,000 and that’s meaningful, but in relative to a $5 million machine, that’s not the main point.
0:26:46 The main point is if the system stops and it stops the mining operations for every minute you stop the mine, you’re losing them tens and hundreds of thousands of dollars.
0:26:51 So the unit economics game there is at a mine level, not necessarily at a truck level.
0:26:58 I also imagine replacement cycles are different. It’s not as easy to buy a new $5 million bulldozer as it is to buy a new $100,000 car.
0:27:05 That’s right, but the whole ROI calculation becomes a slightly different calculation, which is how many trucks do you deploy in a mine?
0:27:10 How do they impact the productivity, which increases the mine revenue and how do they impact the downtime, etc.
0:27:18 So we see slightly different applications, but I think generally in mining, as I said, OEMs and the mines, we talk to see the business case there.
0:27:23 It’s just improving the technologies so that it can reliably work and not bring downtime to the mines.
0:27:33 So I think related to some of the questions around regulation, I’d love to talk a little bit about geopolitics and some of the regional or global differences in our approach to autonomy.
0:27:40 I don’t know if this is a true story, but I did hear the story about DJI drones being used early on in Ukraine.
0:27:54 China had given Russians a backdoor to be able to determine the positions of Ukrainians on the ground using DJI drones, which was a big impetus for a lot of the drone activity that we’re seeing in Ukraine coming from the U.S.
0:27:58 So there’s a lot going on in this kind of geopolitical landscape.
0:28:13 And I kind of love to hear how, for all three of you actually, given Sironic as a defense contractor, I’d love to hear how kind of the role of geopolitics and how you think about autonomy and how it’s come up both as maybe a challenge and also a motivating factor.
0:28:19 So Sironic was basically founded on this idea that China outnumbers our shipbuilding capacity about 200 to one.
0:28:36 So the question is, what can we use as a sort of unfair advantage to leapfrog that rather than just compete on the basis of cost per ship? How can we make autonomous distributed systems deliver sort of an overmatch result over China’s kind of shipbuilding capacity?
0:28:49 And so this is obviously played a major role in motivating Sironic’s accompany. But sort of even beyond that, I think we’ve seen a lot of unmanned service vehicles, which is autonomous boats, be used in the Middle East as well as with Ukraine and Russia.
0:29:03 And so I think the sort of heating up of global proxy wars, not just great power conflict, has also led to a lot of expansion of use of autonomous systems as we see in Ukraine, as we see off the coast of Yemen and other countries like that.
0:29:05 But yeah, that’s the TLDR.
0:29:09 Given the sort of DJI angle here, we’d love to hear how Skydio thinks about this.
0:29:16 Geopolitics is a very important topic in our world. You mentioned Ukraine. We’ve been in Ukraine 30 times. I was there this past spring.
0:29:25 They use DJI’s in a very disposable fashion because it’s the cheapest they can get and they hack it. DJI’s effectively a hostile vendor to them.
0:29:31 And so they have to hack it quite extensively so that the drones can actually fly there and not be detected by the Russians.
0:29:45 And there’s certain techniques they have to do that. One of the challenges that the U.S. looks at is the threat of being wholly dependent on Chinese based technology for industries that they would consider to be critical to national security.
0:29:51 What Ukraine has shown to the world is how critical drones can be in any combat.
0:29:58 And if all of our drones and all of our drone parts are being held by Chinese manufacturers, then we are wholly dependent on them for such a critical technology.
0:30:02 And that is exactly what the U.S. is trying to avoid.
0:30:10 And so there’s laws being discussed within Congress now to start weaning the United States off of Chinese manufactured drones.
0:30:19 So it won’t be an immediate ban. It won’t happen overnight, but they can start sort of lessening their dependence and start encouraging U.S. and U.S. innovators to step into the fold, including ourselves.
0:30:33 We’re not asking for subsidies, but we do benefit from restrictions for sure that for some of these organizations like the federal government or state and local governments to choose western products instead of Chinese manufactured products.
0:30:41 And one small example of the kind of power that some of these manufacturers can have, Otel, which is sort of a smaller version of DJI based in the exact same city in China.
0:30:48 They decided one day that they would update their geofence, basically where you’re allowed to fly or not allowed to fly.
0:30:50 And they turned off all access in Taiwan.
0:30:54 So immediately every Otel drone was just bricked in Taiwan.
0:30:56 Now, why Taiwan?
0:31:07 So if you had an example where all of our critical industries are using a Chinese manufactured drone, and one day China says, I’m just going to turn them all off, they could immediately break them.
0:31:10 And that’s really the extent of the problem. It’s quite severe.
0:31:19 Yeah, I think China has a conversation with every CTO or CEO of any major OEM across industry that visits our office or we talk to.
0:31:26 And these are like weekly conversations just to take an example from automotive and maybe government because applied also does a bunch of work with the government.
0:31:30 In automotive, China is basically redefining the industry.
0:31:32 There’s no sort of lighter way to put it.
0:31:36 There’s a few things that are happening and the Beijing Auto Show was earlier this year.
0:31:38 It happened to be there in person for it.
0:31:40 All of the attention was on the local Chinese OEMs.
0:31:48 Like you could literally go to the booths of the international companies and consumers didn’t really have much interest in them except for a couple of brands.
0:31:54 And that’s because the product innovation in the China ecosystem is second to none right now.
0:32:04 The cars you can buy there, like buy, not just drive in a prototype, just vastly superior consumer experience compared to what you can get in the U.S. or Europe.
0:32:12 I was talking to Steph earlier about some of the experiences there in terms of you walk up to a car and you say, please unpark yourself, it unparts itself, opens a door for you.
0:32:17 You sit in the car, the car assistant can process commands from four people talking simultaneously.
0:32:21 It’s a delightful experience to sit in those. These are production cars.
0:32:23 And it’s not just about the product today.
0:32:27 It’s actually scary how fast the pace of innovation is every six months.
0:32:32 They’re innovating on that product and the cost at which they’re able to do it is very hard for the global industry to match.
0:32:38 So at one point you might think, well, there’s enough geopolitics that maybe the global economy is like somewhat insulated from that.
0:32:42 And we’re seeing that in the U.S. where there’s 100% tariffs on Chinese vehicles.
0:32:50 At the same time, you look at the sort of what’s happening in the market in China where these OEMs are facing a sort of flattening domestic demand.
0:32:51 They have oversupply.
0:32:58 So even though China is the largest automotive market already and the largest exporter, they all have ambitions of sort of going global, right?
0:33:07 And some of these OEMs have cost points that are so low like a BYD that despite tariffs in Europe and despite tariffs in U.S., they might actually be somewhat competitive.
0:33:17 And that’s why every automotive executive is worried about China and thinking about what’s our strategy, not just in China, but what’s our global strategy given what we are seeing in China.
0:33:24 I think on the government side, it’s similar honestly in the sense that the DoD used to be the driver of innovation, right?
0:33:32 There’s a lot of ties between the Silicon Valley history and DARPA and how autonomy came about and how a number of other technologies like even internet came about.
0:33:42 And now we’re at a point where I think the Pentagon recognizes that it’s been slow in moving towards software and moving towards autonomy and definitely slower than China.
0:33:47 Both in terms of sort of processes of like how do you do procurement is actually a big barrier today.
0:34:01 So we do this conference every year in DC focused on national security and the entire conversation is about how do we move faster and bring these technologies that are being deployed in the commercial world to national security much faster.
0:34:04 And how do we evolve our procurement processes to be able to do that?
0:34:12 So I think across industries, we see China being a major part of conversion, a major driver of strategy for companies.
0:34:17 I was going to end with the last question of what keeps you motivated, but I feel like that’s very motivating.
0:34:25 So with that, I think we have a few minutes left, so I wanted to just open it up if anyone in the room had any questions for our wonderful guests.
0:34:33 Alright, since this was a live recording and some of the questions were pretty long, we’ve condensed them for your ears and I’m going to be punching in with the voiceover.
0:34:39 Alright, first question. What are your experiences dealing with security guidelines and regulatory bodies?
0:34:48 The one thing that I would say is it’s a little bit unintuitive sometimes that designing autonomous systems, as long as they’re not like true kinetic effectors, sometimes it’s actually lower risk.
0:34:53 The example that I like to give a lot is imagine designing a drone versus a helicopter, right?
0:34:58 Helicopter, you’ve got to be pretty sure that that thing will not fall out of the sky, right?
0:35:02 A drone like falls out of the sky. Okay, that’s fine. We’ll iterate, we’ll keep going.
0:35:08 And so sometimes it actually helps me sleep a little bit better at night thinking, okay, at the end of the day, these systems are directing themselves.
0:35:12 If the boat sinks, it’s not the end of the world. And so that’s what all adds to it.
0:35:22 In terms of the actual reality, the fact that we’re designing for the DoD though, yes, these systems have to be absolutely like secure mission critical, especially in terms of reliability.
0:35:26 And to this extent, I think CERONIC as an engineering org is built up around this.
0:35:33 Like the way we do our systems engineering, the way we do our test and evaluation pipelines, I think is a lot more rigorous than a lot of other companies.
0:35:38 But at the end of the day, you can sleep a little bit nicer just knowing that they’re unmanned systems.
0:35:44 From a product design standpoint, in terms of safety, a lot goes into basically failure mode.
0:35:50 So what happens if the battery gets too low, no matter what inputs the operator gives, the drone is going to come back.
0:35:57 And it’s going to follow the path that it had so it can sort of navigate back from obstacles, no matter where it’s out to get back to its original location.
0:36:01 It can also basically land directly down. There’s a programming of safe landing zone.
0:36:07 So those are all things to try and keep people sort of safe in terms of more security, especially with the Department of Defense.
0:36:13 We sell a variant of our products that are effectively offline and offline is a bit of a misnomer.
0:36:16 They’re online, but they’re completely within the private network of the DoD.
0:36:23 So we have no access to it. It’s not cloud based. And those are instances where if the customer really, really needs that level of security, they have it.
0:36:29 And it basically means more engineering work for us because we have to carry two variants forward, but it’s necessary in order to circle federal government.
0:36:35 Yeah, generally, in sort of the automotive realm and sort of Vamo cars, etc.
0:36:45 What I would say is that what’s been done in the industry in the past, like all of the systems engineering practices that have been used to build aircraft like commercial airliners, etc.
0:36:55 Are necessary and are used, but not sufficient in the same way in some of the standards that have been put out ISO 26262 that has been used in the industry for a long time.
0:37:01 These are all inputs, but none of these individually or even cumulatively are sufficient.
0:37:07 And so if you look at some of the publications from Vamo and some of the work we do with our customers, we actually consistently tell our customers.
0:37:14 We actually need to do way more than what’s actually any regulatory bodies asking for or what aerospace engineering has done.
0:37:29 In order to prove the safety of the system that actually validating that these systems are safe to be deployed, whether that’s a passenger car, whether that’s a truck, whether it’s a mining agriculture defense application is actually one of the problems our customers struggle with the most,
0:37:32 because there is no standard blueprint for it.
0:37:43 It’s a combination of an engineering problem, plus a data science problem, plus a regulation problem, plus winning consumer trust problem, because eventually you have to convince yourself and the stakeholders that this is safe enough.
0:37:49 So it’s not a science problem, but in many ways it’s still the blueprint is yet to be clearly written for this.
0:37:53 And I just don’t think the regulations and standards are enough and the same applies for cybersecurity.
0:37:57 I think they’re very short of what actually needs to be put into these systems.
0:38:01 What best practices do you use to reduce stress in your technical debt?
0:38:06 If you ask any seronix software engineer what the three magic words are, they’ll tell you this.
0:38:12 The kind of design principles that we follow is simple, correct, fast in that order.
0:38:19 The idea is that first of all, you want to build a very, very simple system that you understand from the electron level.
0:38:24 We do not integrate any technology that we don’t understand from the electron level all the way up to the software level.
0:38:32 And so the TLDR is that we build a ton of stuff that doesn’t work at first, but that we understand super, super well.
0:38:38 And so the systems design like kind of philosophy pervades basically everything in our software spec.
0:38:43 So yeah, the simple, correct, fast is like the way we do it at Seronic. This is totally worked from my opinion.
0:38:46 I never heard of technical debt, what’s that?
0:38:54 So I think technical debt is inevitable in fast changing industries like the ones we play in.
0:38:59 So I think we were previously talking about how the underlying technology in these eventual systems is changing,
0:39:04 which means we have to rethink the products we are providing to the market quite a bit, right?
0:39:09 So you keep the current system stable and then you deploy a new system magically over time.
0:39:14 But we’ve all been doing engineering for a long time to know that it’s never as clear cut as that.
0:39:20 But I think we focus a lot on working with our customers on like joint roadmaps where we literally go to customers
0:39:25 and say, hey, we’re going to actually upgrade the architecture of this product because now we can actually make use of generative AI
0:39:28 or any other technology in order to improve these products.
0:39:35 And surprisingly, more often than not, if you have a reasonable roadmap, the customers are actually pretty reasonable.
0:39:41 It does require you to have a trusted relationship where it’s just not like here’s some money and you’re a vendor and a traditional software vendor.
0:39:48 So if you’re in commoditized markets where you’re just purchasing based on certain features and who’s the most cheapest to procure,
0:39:50 it’s hard to have that kind of relationship.
0:39:57 But for the industries where and where software we are providing them, either that’s the developer tools or the actual software that’s going into the vehicle.
0:40:00 This is cutting edge software and they know this is cutting edge software.
0:40:04 So it’s a very like collaborative relationship with the customer.
0:40:09 We’re often building new products with them and that allows us to have sort of these thoughtful roadmaps.
0:40:11 Man, I’ve been in tech 25 years.
0:40:13 There’s always technical debt.
0:40:15 The longer you’re around, the more it piles up.
0:40:17 I don’t know that I could add anything more.
0:40:24 The one thing that’s a little bit unique about Skydio is that we have these hardware platforms and that allow us to basically sort of reset.
0:40:29 Like the latest generation that we announced last year was three years and $80 million in the making.
0:40:31 That allowed us to basically reset a lot.
0:40:37 But we also have a pretty large cloud software component and there’s a hell of a lot of technical debt that’s been built up over the last seven, eight years.
0:40:40 I don’t have a magical answer on that one.
0:40:46 Some people think to get to self driving level four or five or a robot taxi status that we don’t need lighter.
0:40:48 What’s your take there?
0:40:50 All right, I can take that super easy question.
0:40:56 So I think generally one is somehow the industry has gotten into this.
0:40:59 Yes, LiDAR, no LiDAR, sort of binary camps.
0:41:09 But at the end of the day, a sensor is just one other piece of technology that you incorporate to achieve the ultimate goal of, in this case, level four robot axes.
0:41:15 So if we more had a path to deploying the vehicles that you see on the road without a LiDAR,
0:41:23 I guarantee you we would have taken that path because there’s a lot of engineering and science invention that went into building some of the LiDARs that we more has.
0:41:25 They really are like cutting edge LiDARs.
0:41:31 And so we must not building them for the sake of just, hey, it’s fun to work on these technical challenges.
0:41:42 There’s a portion of the safety case that goes back to the question of how do you certify that these are safe enough that relies on certain properties that only LiDARs have that today.
0:41:45 The other combination of other sensors cannot.
0:41:50 And even in cars in like China, for example, you see LiDARs in lower levels of autonomy.
0:41:54 So A, that allows them to sort of bring certain capability to market faster.
0:42:07 But also from a liability perspective, you have to really think from a manufacturer perspective, the vehicle does get into an accident and we will see accidents happen in the industry as sort of the technology is maturing.
0:42:21 Would you rather have gone the all the way and done whatever was possible from a safety and liability perspective to prevent that or or not is the question that OEMs have to face when they think about should I put a LiDAR in a passenger car or not.
0:42:36 So, ultimately, will we get to a point where systems without LiDARs are good enough to function in their respective design domain that respective design domain could be mining that respective design domain could be cars.
0:42:40 Some defense applications, etc. I think in the long term, we will get there.
0:42:56 But do I think that at the early stages of deployment of technology, taking the safest path is the right approach in order to build up trust in the technology in order to build up trust with residents of a city in order to build up trust with regulators.
0:42:59 I think that’s a pretty reasonable approach.
0:43:03 I’m super curious to hear your observations around the ecosystems you’re seeing emerging.
0:43:06 I’m sure that’s also really interesting from a venture perspective, right?
0:43:09 How do you see the next couple of years coming together there?
0:43:11 I’m happy to take a first crack at this.
0:43:13 So, I think it’s a really good question.
0:43:14 It’s something that we think about a lot.
0:43:32 Historically, I’d say the industry has mostly focused around with a few exceptions applied being a great example of an exception scaly eye being another one that both businesses really took off as a result of selling software into the sort of self driving early on at least into the self driving market.
0:43:46 Most autonomy companies for the last decade have been very tight coupling of hardware and software where all of the software was written almost to the firmware level for the specific hardware form factor.
0:44:01 And what we’re starting to see, I think part of it is like the last three years of developments in AI and part of it is the broad explosion of interest in autonomy across a lot of different use cases is more of a software tool chain emerging.
0:44:08 But there are a lot of unsolved challenges that we see pretty consistently across companies operating in autonomous spaces.
0:44:18 So even like basic assumptions about how software is built for the kind of traditional SaaS world of the 2010s just doesn’t work in the autonomy space.
0:44:28 You often don’t have Wi-Fi like you can’t expect that you can just ship releases like every five minutes to customers and they’ll be able to update.
0:44:31 How do you stream data from one place to another?
0:44:42 Like how do you decide what decision making happens on device on chip on the edge versus like in a cloud environment because it has to consider a swarm or a fleet of devices.
0:44:54 So those are all like really interesting technical challenges and problems that I think developers will start solving more at scale in the next several years as more and more autonomy use cases come up.
0:45:03 So I imagine that the kind of developer toolkit around building an autonomous system is going to be more decoupled from the hardware itself over the next several years.
0:45:08 There’s lots of opportunities to build new tech in those scenes coming up.
0:45:13 From our vantage point, autonomy is what democratizes access to the drones, but it’s not what people buy.
0:45:15 They don’t care about our autonomy.
0:45:17 They frankly don’t really care about our drones.
0:45:26 What they care about is the information that the drones give them and the easier way we can get that information, the faster way we can get that information.
0:45:41 Our next level of autonomy in the broader roadmap is going to be less about how do you fly and more about what mission are we trying to accomplish and how do you actually coordinate across multiple drones to be able to accomplish that mission in a safe way in a fast way.
0:45:45 And that becomes much more vertically specific and use case specific.
0:45:50 And that’s really to us where like the gold is and will create tremendous differentiation.
0:45:56 Our hope is that there is actually a very thriving drone industry in the United States to be able to combat the Chinese manufacturers.
0:45:59 It’s also extraordinarily capital intensive and that’s hard.
0:46:00 That’s a big barrier.
0:46:05 We’ve raised a lot of money and we’re the largest manufacturer in the United States, but we still have a long, long, long way to go.
0:46:18 I hope there’s an sort of proliferation of a number of players coming in and contributing because I think it’ll be overall best for consumers is in building applications for the vehicle.
0:46:23 So today, for the most part, the cars we buy in US, in Europe, etc.
0:46:30 They’re not delightful consumer products like when you brought your first iPhone and you had a magical moment of ours is great.
0:46:35 And now some of that has, I think, tapered off at iPhone 16 seems like the same as 15 and 14.
0:46:40 So there’s really not been a great consumer product in that sense for a while.
0:46:46 But when you go to China and you experience those, you get some of that wow and that moment of delight back.
0:46:50 And then you peel the layers and say, OK, why doesn’t this exist in the cars here?
0:46:53 The reality is that you peel back the layers behind your car today.
0:47:02 There’s 150 different suppliers that each provided a small ECU as mini computer and the OEM integrated that into sort of a functional experience.
0:47:05 But you almost have to redesign the car from the ground up.
0:47:07 And that’s what Tesla did because they could start fresh.
0:47:11 They could start with a software engineering talent and say, we don’t need to design with the way that we did.
0:47:16 And that’s a journey setting every single vehicle that moves is on today.
0:47:20 And that’s why Applied is sort of providing that operating system.
0:47:25 Because if we can provide the operating system and give you a nice SDK to build those consumer applications on top,
0:47:29 you can unleash your creativity and think of the car as your third space.
0:47:31 And what experiences would you want in that vehicle?
0:47:34 Is it theater mode? Is it something else, et cetera.
0:47:37 And so that’s where I hope the industry goes.
0:47:40 And I think overall, it’ll be just better for consumers.
0:47:46 All right, that is all for today.
0:47:49 If you did make it this far, first of all, thank you.
0:47:53 We put a lot of thought into each of these episodes, whether it’s guests, the calendar touchers,
0:47:57 the cycles with our amazing editor, Tommy, until the music is just right.
0:48:03 So if you like what we’ve put together, consider dropping us a line at ratethespodcast.com/a16z.
0:48:05 And let us know what your favorite episode is.
0:48:08 It’ll make my day, and I’m sure Tommy’s too.
0:48:10 We’ll catch you on the flip side.
0:48:13 [music]
0:48:15 [music fades out]
0:48:17 you