AI transcript
0:00:05 AI coding is the first really large market for AI.
0:00:07 When do we say, this is all agents?
0:00:12 We just, at the end of the value chain, are like, does this work or not work?
0:00:13 Click yes or no.
0:00:16 Agents, more than ever, need an environment to run these things.
0:00:19 Context engineering for both humans and agents.
0:00:22 Every single part of it is getting disrupted.
0:00:26 It’s not that there’s just somebody writing code, like your classical developers being
0:00:28 disrupted, but everybody along the value chain.
0:00:34 We’re resurfacing an episode from the AI plus A16Z podcast that we think deserves a wider
0:00:35 audience.
0:00:41 A16Z partners Yoko Lee and Guido Appenzeller make the case that AI coding is the first truly
0:00:44 massive market for AI, potentially worth trillions.
0:00:49 They break down what’s changing in the dev loop, where ROI is showing up first, and what
0:00:50 founders should build next.
0:00:51 Hope you enjoy.
0:00:59 So Yoko, we just launched this, I think, amazing new dev stack for the AI coding.
0:01:02 And I’m really, really excited about this.
0:01:02 Yeah.
0:01:07 I mean, let me start with a very high order pitch, why I think this is so incredibly exciting.
0:01:13 I think AI coding is the first really large market for AI, right?
0:01:15 I mean, we’ve seen there’s a ton of investment that has flown in.
0:01:18 The question now, to some degree, is where is the value, right?
0:01:19 Why are we doing all this?
0:01:23 AI coding can create an incredible amount of value.
0:01:27 If you think about this, we have about 30 million developers worldwide, roughly, right?
0:01:30 Let’s say each of them generates $100,000 in value.
0:01:34 In the United States, it may be low because many of them get paid a lot more.
0:01:36 But internationally, it might be a little high.
0:01:37 But I think in order of magnitude, it holds.
0:01:42 So in aggregate, the value we’re creating here is about 30 million times $100,000.
0:01:44 So $3 trillion.
0:01:47 I will argue even more because that’s just developers.
0:01:50 But then there’s also people who are development curious.
0:01:51 They were not developers.
0:01:54 Maybe they’re, I mean, design engineering now is a big thing.
0:01:57 Every designer should just, you know, write code.
0:01:57 Doc writers.
0:01:58 Exactly.
0:01:59 I mean, there’s so many of these effects.
0:02:03 But if you just take the $3 trillion figure, that’s about the GDP of France.
0:02:09 So the claim we’re making here, as crazy as it sounds, is that we’re saying the entire population
0:02:14 of the seventh or eighth, I think, largest economy on the planet generates about as much value
0:02:20 as a couple of startups that are reshaping the AI software development ecosystem plus the LLMs underneath.
0:02:20 Yeah.
0:02:21 That’s crazy.
0:02:25 Everything we see and touch and use nowadays are all software.
0:02:26 That’s right.
0:02:27 Yeah.
0:02:31 So we’ve, software has disrupted everything in the world.
0:02:35 And now software itself is getting massively disrupted.
0:02:35 Totally.
0:02:39 And then what you mentioned in the blog post is really interesting just because,
0:02:44 we now are more capable at using LLM to generate code and coding and produce software.
0:02:48 But then as a result, it’s not like there’s less jobs.
0:02:51 It’s actually more and more software is being produced.
0:02:56 Before, maybe as a SaaS service catering to hundreds of people’s needs, to thousands of
0:02:56 people’s needs.
0:03:00 Now you can really just vibe code things software for one.
0:03:00 Yeah.
0:03:01 Yeah.
0:03:01 I vibe code.
0:03:02 I do that.
0:03:02 Exactly.
0:03:03 You do that.
0:03:03 Exactly.
0:03:06 And then I vibe code my own email filter.
0:03:11 You know, I don’t do that so much of using LLM to reply to my email, but I have a filter
0:03:13 where I categorize the labels, things like that.
0:03:14 Only to some emails.
0:03:15 Only to some emails.
0:03:22 So the first question becomes like, how do you think the development loop is shifting?
0:03:24 I think the answer is complex.
0:03:28 And very frankly, it’s so early, I think, in this AI revolution.
0:03:30 We don’t have the full answer yet, right?
0:03:34 But I mean, we have our little stack, our little software development lifecycle post AI
0:03:35 in the blog post.
0:03:39 And I think the biggest learning from that is probably that every single part of it is getting
0:03:40 disrupted.
0:03:44 It’s not that there’s just somebody writing code, like your classical developers being
0:03:47 disrupted, but everybody along the value chain is getting disrupted.
0:03:49 What’s the most surprising part for you?
0:03:52 What’s the most disruptive field today in coding?
0:03:55 And what do you think AI will come after next?
0:03:59 So I think, well, we’ve seen the biggest growth, I can say to say, in the classic sort of coding,
0:04:02 IDE integrated coding assistance or more agentic coding assistance, right?
0:04:07 The cursors and devins and GitHub co-pilots and cloud codes of the world, right?
0:04:12 I think that’s where we see the most traction, where we see an incredible revenue growth.
0:04:16 I mean, I want to say that segment possibly has the fastest revenue growth of any startup
0:04:21 sector we’ve seen in the history of startups, which is, again, an incredible statement.
0:04:24 So I think this is currently the vanguard, right?
0:04:25 And everybody’s aware of it.
0:04:28 We’re seeing billion-dollar, you know, acqui-hires or takeover offers.
0:04:30 So that’s an incredibly vibrant sector.
0:04:32 Now, which one is next?
0:04:33 That’s a really good question.
0:04:38 So to be very specific, in a blog, we wrote about the basic loop.
0:04:41 The basics is you plan, you code, you review.
0:04:44 Where does LM coming in?
0:04:46 Where do you think more of the loop would be disrupted?
0:04:51 Do you think the loop will still be the same as what we used to have, the basic?
0:04:53 Or do you think it will look very different?
0:04:56 I think at this point, it’s very hard to speculate about the end state.
0:04:58 But if you, let’s assume you’ve seen the first email.
0:05:00 I’ll get there.
0:05:04 But if you looked at the first email sent over the internet, right?
0:05:06 You can sort of predict that probably we’ll have websites and these things.
0:05:08 Maybe if you’re good, you can, right?
0:05:12 But, you know, saying like, hey, the net effect of this is that everybody can rent out their
0:05:14 house and compete with hotels.
0:05:15 And there’s going to be the biggest hotel company on the planet.
0:05:17 You’d be like, well, that’s a little far-fetched.
0:05:18 But now we have Airbnb, right?
0:05:21 So these secondary effects, I think, are really hard to guess.
0:05:24 My current hypothesis is I think we’ll still have software developers.
0:05:26 I think they’re not going anywhere, right?
0:05:30 I think what they do will look completely different.
0:05:31 Right.
0:05:31 Right.
0:05:31 Yeah.
0:05:39 I think the CS education, frankly, any CS class taught today at any major university
0:05:42 is probably best seen as this historical relic from a bygone time, right?
0:05:47 I mean, if you look at the best-of-breed startups, what they’re doing, the loop that the developer
0:05:49 is in looks so different from what you did before, right?
0:05:53 You have multiple agents that you’re prompting, that you’re telling things, you know, and pull
0:05:53 that back into UI.
0:05:55 You’re trying to understand what they did.
0:05:56 You’re trying to put them back on the rails.
0:05:58 It’s a lot more thinking at a higher level.
0:06:02 I mean, all of coding sort of has been high levels of distractions, but I think we’re
0:06:03 making a huge leap here.
0:06:05 So how it’s going to look like, I have no idea.
0:06:08 My gut feeling is we’ll probably have more developers.
0:06:14 Look, this basic plan-execute cycle, there’s probably going to be some flavor of that still
0:06:15 around is my guess.
0:06:16 But what do you think?
0:06:21 So one of my top questions is that, would this be a step-by-step loop or would this
0:06:23 mesh into just one step?
0:06:29 So one example is, if my agent is writing a code, do I still need to review it or do I
0:06:31 have another agent that just reviews it?
0:06:34 If it’s the same agent, like implementation detail is one thing.
0:06:39 You can separate out the agent generating code from the agent reviewing code.
0:06:44 But then if it’s all agent, both generating and reviewing code, is it just the same step?
0:06:49 Do we actually disaggregate the step-by-step process and have human in the loop?
0:06:55 Whereas if as a human, I write code and I want agent to review code, that makes sense to me.
0:07:02 So I do wonder, when do we pull out something as this is an individual tool and individual
0:07:03 stuff an agent takes care of?
0:07:06 And when do we say, this is all agents?
0:07:10 We just, at the end of the value chain, we’re like, does this work or not work?
0:07:11 Click yes or no.
0:07:16 I think the time periods of which an agent can work autonomously will get longer.
0:07:21 You know, still, if somebody says, look, I want to write a complete ERP system for my
0:07:26 multinational enterprise goal, there’s no way I could imagine that it’ll just run and
0:07:29 back come software that actually fits the requirements.
0:07:33 And in part, I think it’s a problem that models are still very, very far away from being able
0:07:34 to run autonomously for that long.
0:07:38 But the other problem is, let’s assume this was an all-human team.
0:07:41 We wouldn’t understand all the challenges yet at the beginning, right?
0:07:42 We’d have to revisit the design.
0:07:45 We have to revisit the architecture that has cost implications and so on.
0:07:49 So at some point, you sort of need to go back to the architects and the product managers
0:07:53 and say, hey, we had a plan A, didn’t quite work, or we have found new challenges.
0:07:55 So here’s our updated plan A.
0:07:57 Is that the, you know, or plan B, right?
0:07:58 Is this what you want to do?
0:08:00 So I think the loop will still be there.
0:08:01 The timescales will probably change.
0:08:03 But yeah, it’s very, very hard to guess right now.
0:08:10 Another thing we start to see more often is, contrary to how much humans need to come and
0:08:15 intervene in the loop, we actually start to give agents tools for them to know what they
0:08:16 have to do.
0:08:22 One example is, this loop I see so often, like the agent wants to implement, say, clerk
0:08:23 auth in their app.
0:08:29 Now they need to go to Mitlify and context 7 to say, what is the latest version of clerk
0:08:32 and how can I implement it correctly and in what file?
0:08:38 I’m not going to copy paste it to cursor or give it to the agent because I’m too lazy as
0:08:39 a human now.
0:08:44 The agent should be able to call the API themselves to put stuff in the context to make it work.
0:08:48 This is just one example of what behavior change we’re seeing.
0:08:52 Because before, as developers, we’re so used to go back to the docs and refer to the docs
0:08:54 and tell the agent what to do.
0:08:55 Now agents can obviously just-
0:08:56 We cut off the middleman.
0:08:57 Right.
0:08:58 We cut off the middleman.
0:09:01 I don’t need to route all these requests for the agents anymore.
0:09:04 I think there’s other examples, which is verification.
0:09:10 As a human, before when I write code or review other people’s code, I pull out the code.
0:09:14 And then the first thing I do is actually not to review because I don’t like reading code.
0:09:15 I’m not a human compiler.
0:09:20 The first thing I do is to fork the change and see if it still works.
0:09:22 If it doesn’t work, I just do not review it.
0:09:28 Nowadays, there are opportunities to give just agents a native environment to first see,
0:09:29 does this work?
0:09:31 Does the UI still look good?
0:09:32 Do all the requests still check out?
0:09:33 Yeah, absolutely, yeah.
0:09:34 Did it break my build?
0:09:37 Before, the human needs to come in and review.
0:09:41 Maybe that manifests itself in part of the local development process.
0:09:44 Maybe it manifests itself in the PR review process.
0:09:49 But in any case, now agents, more than ever, need an environment to run these things.
0:09:52 When I used to write something just for myself, like a little script I need somewhere in the past,
0:09:54 I usually didn’t include unit tests, right?
0:09:55 For production code, it’s different.
0:09:58 But for just personal things, you know, it’s like, yeah, that’s a single developer.
0:09:59 I know what I’m doing here.
0:10:03 With agents, I now start including unit tests because they’re so much easier to write.
0:10:07 And they allow an agent, as you said, right, to understand if the changes that they did
0:10:08 broke anything else, right?
0:10:13 And they may not have the context how this originally was built anymore in an easily digestible form.
0:10:14 So that’s super valuable.
0:10:19 On the grand scheme of, like, how much economic value this generates,
0:10:23 where in the value chain do you think it’s growing the fastest?
0:10:28 Like, you see where the agents is producing so much more value than other areas.
0:10:30 But what are the areas you feel like would be the next takeoff?
0:10:36 So, look, I’m talking about 100 or so enterprises about this per year.
0:10:39 Just when we, you know, take our portfolio companies to them as potential customers.
0:10:44 What I’m hearing from them is that the number one use case in terms of ROI,
0:10:47 right now is legacy code porting, right?
0:10:48 It’s not super surprising.
0:10:50 Like, one of the first papers in the space from Google, right?
0:10:53 They wrote a fantastic paper on, you know, where they detailed on, you know,
0:10:58 just doing very mundane things like replacing a Java library across a very large code base,
0:10:59 right?
0:11:00 Not like millions of lines of code, right?
0:11:01 So it’s a very large code base.
0:11:04 What do you consider as legacy stack?
0:11:05 What do you consider as new stack?
0:11:07 Well, it totally depends on you.
0:11:10 But I mean, for the banks, it’s often COBOL or Fortran to do Java.
0:11:11 Oh, COBOL.
0:11:12 I haven’t heard that word for a long time.
0:11:17 You know, I actually wrote COBOL code once in the 90s.
0:11:19 It probably dates me at this point.
0:11:19 Okay.
0:11:22 Actually, how are LLMs with COBOL?
0:11:23 They’re apparently extremely good.
0:11:25 That’s a surprise.
0:11:26 So here’s the thing, right?
0:11:32 One of the hardest thing, if you implement code with LLMs, is just getting the specification precise, right?
0:11:36 If I can specify something very precisely, then usually the LL can do a good job at implementing it.
0:11:39 So many of these companies do is they take legacy code.
0:11:43 They have an LLM, write a specification that fits the legacy code.
0:11:45 And then they say, re-implement the specification.
0:11:50 And you may look at the code if there, you know, as a tiebreaker, if something is not clear, right?
0:11:52 And that seems to work incredibly well.
0:11:55 So I’m hearing that today, I heard from several.
0:12:00 sources now that you can get, you know, about a 2x speed up versus traditional processes for that, right?
0:12:02 And that’s amazing, right?
0:12:09 And what this has led to is that actually, of those enterprises that I’ve talked to, the majority says they’re currently accelerating,
0:12:15 at least the majority of those that are sophisticated about this, they’re saying they’re accelerating the developer hiring, right?
0:12:20 We don’t know if this is a long-term trend, but right now they’re busy saying, look, because we found so many low-hanging fruit type projects
0:12:24 where with a little bit upfront investment, we can then save infrastructure costs.
0:12:26 And that’s super exciting.
0:12:30 So, you know, what this will mean for the, you know, how much longer is the mainframe business going to be around?
0:12:31 Or, you know, I don’t know.
0:12:37 But there’s definitely a shift there where suddenly legacy code migration is much, much easier than it was before.
0:12:42 You know, I think that they’ll change a lot of the dynamics in the sort of classic enterprise software space.
0:12:42 That’s interesting.
0:12:46 I do wonder if we will get new mainframe code.
0:12:50 Because now, because before, no one knows how to program those things.
0:12:51 That could also be, yes.
0:12:56 And now you realize you can program mainframe using natural language.
0:12:56 Yeah, totally.
0:13:02 So, another possibility is that we get renaissance of, like, the underlying legacy coding languages.
0:13:08 I mean, it’s crazy to me how versatile these coding assistants are, right?
0:13:10 I mean, we’re seeing them write CUDA kernels.
0:13:10 Yeah.
0:13:12 Which is like, that is difficult stuff to write.
0:13:12 Right, totally.
0:13:13 By any metric.
0:13:13 Yeah.
0:13:18 I’ve tried them with a language which basically has no usable training data set.
0:13:26 And they’re still able to sort of abstract, you know, with a couple of examples, you know, of how the code will have to look like.
0:13:30 It’s not perfect, but so I think it’s a very, very broad technology for sure.
0:13:35 Recently, just like what we were talking about before, code reviews.
0:13:40 Because, like, to your point, LLMs are so good at coding and generating code.
0:13:43 Sometimes it’s beyond our comprehension.
0:13:46 We’ll take more time to review the code than the coding agents.
0:13:49 Like, that’s not a controversial opinion.
0:13:50 It’s just like how it is.
0:13:50 That’s the reality.
0:13:51 Yeah.
0:13:58 So, it does make me wonder how our development chain and steps are going to evolve from here.
0:14:04 How are we going to do PRs when we can’t possibly review thousands of lines of code as humans?
0:14:07 So, does that mean the right abstraction now is still code?
0:14:11 Or does it mean the right abstraction now is for us to review plans?
0:14:17 If that’s the case, is GitHub review still the right abstraction for that?
0:14:20 I think there’s still a role for review in general.
0:14:24 I think the question is, will humans do the review?
0:14:31 Like, right now, most of the code that LLM generates, you know, unless you’re deep in vibe coding territory, you’re just like, oh, this is a one-off.
0:14:33 I just want to try something out.
0:14:35 Maybe then you don’t review it.
0:14:36 You just hit accept and hope for the best.
0:14:41 But anything that, you know, anything else, you do review the, I still review the code line by line.
0:14:46 You know, that said, we’re starting to see really good tools that plug into your backend system, your GitHub.
0:14:55 Whenever a pull request comes in, they analyze it, they comment on it, they point out security vulnerabilities, they point out that the spec is different from the implementation.
0:15:00 You know, they point out that this creates dependencies, which may not be desired.
0:15:04 They, you know, enforce coding guidelines, which is very, very powerful, right?
0:15:06 I haven’t met anybody yet.
0:15:15 Tell me if you have, but I haven’t met anybody yet who basically has said, we’re going to rely purely on AI to review code, right?
0:15:17 We just, anything can go in if the AI checks off on it.
0:15:23 But I have seen, for example, companies that are saying before we had two developers review code, and now it’s one developer, right?
0:15:31 And, or, you know, cases where basically just the AI hangs out in the GitHub, you know, discussion and comments on things.
0:15:33 You know, if there’s, if they, you know, so you can basically delegate tasks.
0:15:34 It’s like, can you look at this?
0:15:36 It’s like, are we using this library somewhere else?
0:15:40 So basically you can, you suddenly have somebody who can, who can help with these tasks.
0:15:52 My hot take on this is actually, if PR is supposed to give us a context as developers, what did these other coding agents or my colleagues, you know, change that I should be aware of?
0:15:55 I don’t think code, reviewing code is the right abstraction.
0:16:00 Because it might be feature level, it might be performance level.
0:16:03 So now, now it might just become a one-liner.
0:16:09 This, you know, this agent improved your CUDA implementation.
0:16:14 And I, maybe I don’t even know CUDA, but I know the improvement when I can verify it.
0:16:20 So the question for me is, do I still need to go to the PR review every line?
0:16:27 Or should I just be given, you know, two sentences to know how this works and the environment to test it out?
0:16:28 And then I would just merge it.
0:16:30 If it’s the right two sentences that works.
0:16:30 Yeah.
0:16:33 Will the, will the LLM always pick the right two ones?
0:16:34 I don’t know yet.
0:16:34 Yeah, that’s true.
0:16:39 I mean, if you give it an environment, the question is like, can you verify the environment?
0:16:40 Against the two sentences?
0:16:41 Yeah.
0:16:43 If the answer is yes, that would be easier to solve.
0:16:46 If the answer is no, you know, that’s harder.
0:16:53 But I think there’s a bigger picture here, though, which is that LLMs are also very good in generating the documentation.
0:16:54 That’s true.
0:16:55 And description for the code, right?
0:17:05 So when I use something like Cursor for coding, and, you know, it generates code, I often ask it afterwards and now take the internal documentation and update it.
0:17:12 Because the internal documentation is important for me, but also for the coding agents because they want to be able to refer to it.
0:17:16 You don’t want every single time to take the entire code base and stick it into the context window.
0:17:17 It’s massively inefficient and slow.
0:17:22 So if instead you can just say, like, read this document that will explain to you the class hierarchy, right?
0:17:25 And then based on that, implement this new subclass.
0:17:27 So it’s much, much faster to do.
0:17:33 I think there’s a real opportunity there to get to much better documented code than previously, right?
0:17:33 Right.
0:17:34 That’s true.
0:17:39 A compiler is great, and then it takes a high-level abstraction, translates it into a lower-level one.
0:17:46 But now we have the ability that if somebody had in hand optimizes the lower-level one, we can use that to update the higher-level one.
0:17:46 That’s true.
0:17:50 What is the new compiler in the age of AI?
0:17:52 LLMs are sort of a compiler, right?
0:17:53 In some way.
0:17:53 That is true.
0:17:57 They take a higher-level description and fill it out.
0:17:58 I mean, I think the big difference…
0:18:01 I guess what’s missing is it doesn’t natively have an environment.
0:18:05 Compiler gives you, does this work or does it compile or does it not compile?
0:18:09 It doesn’t tell you, does this work, which is a very subject thing.
0:18:10 I think it’s right, right?
0:18:17 And like a compiler enforces, is a strict enforcement of certain things, right?
0:18:21 You can, I can rely on that, I don’t know, I’m using Rust and things will be typed, right?
0:18:23 So that’s a huge step forward, right?
0:18:24 And I can exclude certain bugs.
0:18:27 With an LLM, that’s not the case, right?
0:18:33 Now, that said, you sort of wonder, is this just an initial thing?
0:18:34 Is this just a…
0:18:36 I mean, LLMs over time…
0:18:42 Like, for example, we can give LLMs tools that allow them to syntactically parse code, right?
0:18:44 And then now suddenly they can start reasoning above code.
0:18:49 They can ask, are we sure that the representation of object X is the same across these different,
0:18:52 you know, modules, even if it gets serialized in between or something like that?
0:19:00 I don’t mean to make this episode disaggregating GitHub, but GitHub is so central to so much
0:19:01 of our workflow, right?
0:19:03 Everything goes through it.
0:19:07 From the social aspect of discovering what other developers are doing, to distributing the
0:19:12 software, you can download things, to keeping track of what you have changed, Git, right?
0:19:14 The integration of the build system, the backend, yeah.
0:19:22 Yeah, so now another interesting thing we start to see is people use Git repos very differently
0:19:22 now.
0:19:28 Before, it was like, oh, humans will make some changes, we’ll commit it, and then other people
0:19:28 will see it.
0:19:31 And then we, you know, open the PR, people see different revisions.
0:19:35 Now agents are doing so many changes.
0:19:38 It’s kind of counterproductive to commit everything.
0:19:45 And then repos usually have very low rate limits because it’s designed for humans to use.
0:19:51 So now the question becomes, what is the new repo, like, abstraction?
0:19:58 Like, that handles, like, a distributed, like, infrastructure, but also has, caters to high-frequency
0:19:58 commits.
0:20:04 And sometimes when agents doing these commits, they don’t even want to preserve this forever.
0:20:11 It’s more like an intermediate step so that they can explore five different paths, you know,
0:20:13 but then revert to any of them.
0:20:15 And then when they’re happy with it, they commit back to GitHub.
0:20:21 So one thing I’ve been playing around with is there’s this company called Relays, and then
0:20:27 their docs has this repos feature where you can’t give a repo to the agent.
0:20:31 The agent will, you know, commit stuff, like, very high-frequency, kind of works really well
0:20:32 with bi-coding agents.
0:20:34 And that has been such a great experience.
0:20:38 And in the wild, I start to see people building this internally, too.
0:20:40 It makes total sense.
0:20:47 Look, if we completely change how humans write code, or we’re shifting from humans to delegating
0:20:52 most of the writing to agents, I think it would be foolish to assume that the underlying
0:20:56 services that were a good fit for the human world are still a good fit for this new agent.
0:20:57 Yeah, for sure.
0:21:00 That’s almost certainly not the case, where we can do better in our cases.
0:21:03 I think you’re completely right, right?
0:21:06 For source code repositories, we want something that’s much more real-time.
0:21:09 Honestly, I mean, imagine, let’s take this to the limit, right?
0:21:14 Let’s assume I have now, I’m just me personally running 100 agents in parallel that all try to
0:21:15 implement features in my code base.
0:21:16 Yeah.
0:21:19 You probably need some kind of coordination mechanism between that.
0:21:21 They’re all not trying to edit the same file, right?
0:21:22 Oh, for sure.
0:21:24 You know, the rebase only carries you so far, right?
0:21:25 Right, yeah.
0:21:27 You just get too many collisions.
0:21:30 And they all need shared memory on the repo they’re working on.
0:21:31 That’s right.
0:21:33 Because they don’t want to reinstall the dependencies every time.
0:21:34 That’s right.
0:21:34 Yeah, exactly.
0:21:40 And so, you know, you probably need something that’s much, much more flexible and more real-time.
0:21:42 And, you know, I think we’re still figuring out what that is.
0:21:44 What Relays is doing there is amazing.
0:21:48 And I think it’s not just true for GitHub.
0:21:50 And GitHub is one of the big platforms we use.
0:21:54 But, you know, take the specification writing with, you know, say a Confluence or Jira, for
0:21:55 example, what you could use there.
0:22:01 If you would develop these systems bottoms up with AI in mind, they would probably look very,
0:22:03 very different, right?
0:22:06 I mean, it’s like, you know, should my story tracker have a function that looks at code and
0:22:07 updates stories accordingly?
0:22:10 I mean, yeah, that would be very natural, right?
0:22:12 So I think you can do it.
0:22:14 I mean, I think we’ve seen this for documentation, right?
0:22:18 I mean, something like Mintlify looks very, very different at the documentation problem than
0:22:20 previous generations.
0:22:21 I think we’re seeing it for testing.
0:22:23 We’re seeing it for PR reviews.
0:22:26 We’re rethinking this entire stack.
0:22:26 And that’s exciting, right?
0:22:27 That’s true.
0:22:32 Another behavior change we have seen, which is really interesting, because humans, like,
0:22:34 developers who are very lazy.
0:22:34 Yeah.
0:22:38 So if we don’t have to read it, we do not read it.
0:22:40 We only read the relevant parts in documentation.
0:22:42 We only skim through things.
0:22:48 So the net new behavior we see all the time with, you know, documentation, like, hosted on
0:22:53 Mintlify is that users who are net new just go in and ask questions.
0:22:54 Yeah.
0:22:57 And agents do not just ingest it anymore.
0:23:01 They will actually just do a query against this context.
0:23:07 So context engineering for both humans, because our brains are like, oh, so we need the context
0:23:11 and agents who need context, which is a very critical part.
0:23:15 It’s such a critical part of, like, development from here point on.
0:23:20 And then the question becomes, what are the other tools you think agents will need?
0:23:25 Like, we had this huge, you know, market map from the blog.
0:23:31 And then there’s, contrary to where, you know, all the previous market map suggests, which is
0:23:33 like, here are the developer tools.
0:23:35 Now there are agent tools to make it better.
0:23:38 In some cases, the same tool caters to both, right?
0:23:42 I mean, that’s a media documentation for agents and for humans.
0:23:42 Right.
0:23:47 So look, early days, but I think we’re seeing a couple of big categories emerge, right?
0:23:52 I mean, sandboxes are helpful to try out, you know, code snippets or build things.
0:23:53 How do you define a sandbox?
0:23:56 I think we can record the whole episode on it, but.
0:23:58 And I think we will at some point.
0:24:02 But look, at the end of the day, you need an environment with certain safety guarantees,
0:24:12 right, where LLMs hallucinate, LLMs can be with clever, you know, if they use external
0:24:15 sources, they can potentially be maliciously prompted to do bad things.
0:24:20 And you just want to have something that basically limits how much the blast radius, if anything
0:24:20 bad happens.
0:24:26 I think we’re seeing interesting search tools and parsing tools, like something like source
0:24:27 graph or so, right?
0:24:31 If I’m writing a couple of my code in a couple of files, we don’t need that.
0:24:35 But if we have a really large code base, you know, and suddenly the question is, look,
0:24:39 we’re looking for, we’re trying to replace, we’re trying to add a parameter to a function
0:24:41 in a library that’s widely used.
0:24:42 Where is this library used?
0:24:44 This is a really hard problem suddenly, right?
0:24:48 I mean, it’s like if you’re in Python or so, you can import something as something else.
0:24:51 So like a simple find operation will no longer find these things.
0:24:53 You probably need syntactic parsing to find those.
0:24:59 So I think we’re seeing documentation tools that are optimized for agents and allow agents
0:25:00 to look up these things.
0:25:04 I think it’s good if an agent can do things like web search, right?
0:25:07 So, you know, we’re seeing companies in that space.
0:25:07 What am I missing here?
0:25:08 There’s so many more.
0:25:13 I think there’s the other part of more, we’re seeing more specialized models.
0:25:18 Yeah, I mean, for things like code editing or, you know, re-ranking files and things like
0:25:19 that, right?
0:25:21 That’s definitely shaping up as a market.
0:25:29 If you take a big step back, if we assume there’s massive value creation here, that probably
0:25:31 creates an opportunity to create a very large number of startups.
0:25:35 I mean, if you would have asked me 18 months back, I would have, actually, if you had asked
0:25:38 me 24 months back, I would have said, like, look, DevTools, it’s a smallish kind of market.
0:25:39 How big can this be, right?
0:25:40 That’s not very exciting.
0:25:45 If you ask me today, it suddenly looks like this is, you know, a market that could go
0:25:47 in the hundreds of billions of dollars, right?
0:25:48 In theory, could it go to a trillion?
0:25:49 I don’t know.
0:25:51 So how many companies can you create in that space?
0:25:53 I have no idea, but probably hundreds.
0:25:56 And over time, it’ll, you know, consolidate.
0:26:01 But I expect this to be an ecosystem, not a business model.
0:26:08 I have a fun question, which is, going forward, so when GitHub came out, they have this commit
0:26:15 charts, it’s, it was on t-shirts, you know, like people compare the commit charts, people
0:26:17 like commit specific messages.
0:26:21 So the commit charts look like, like has a good graphics.
0:26:22 What?
0:26:28 Because before, commits are so tied to the value developers bring.
0:26:30 Like, how many commits do you make?
0:26:32 How many lines of code do you change?
0:26:35 I mean, we all know it’s not the best proxy for value.
0:26:37 Yes, not at all.
0:26:38 Right.
0:26:42 So what would be the next commit chart on GitHub?
0:26:44 What would that look like?
0:26:46 What’s the next?
0:26:47 That’s great.
0:26:48 Wow, that’s an amazing question.
0:26:50 Maybe how many tokens you burn?
0:26:54 If you come to the office, look, I burned like 10 million tokens over the weekend.
0:27:00 But tokens burned could also be very ineffective prompting or context engineering.
0:27:03 Yeah, I just stuck my entire code base in the context window, maybe.
0:27:06 Is it the number of agents you use?
0:27:08 That’s even more game of value.
0:27:16 What is the unit of value that’s the closest approximation to what you’ve delivered as value
0:27:16 as developer?
0:27:18 Non-cached input tokens?
0:27:21 Might be too complicated.
0:27:21 I don’t know.
0:27:22 Yeah.
0:27:24 I’m taking this to a higher level.
0:27:27 The metrics how we evaluate software development are changing.
0:27:28 Right?
0:27:34 Where potentially a big complex refactoring isn’t that much work anymore.
0:27:34 That’s true.
0:27:38 Because I can let the LLM do this and it’s structurally easy.
0:27:43 A specific optimization in a fairly obscure area where the LLM has no training data, I
0:27:45 may have to do by hand and it’s vastly more valuable.
0:27:47 So, yeah, it’s complicated.
0:27:49 Maybe it’s number of apps vibe coded.
0:28:00 I think something else, which is there’s on the market map agent toolboxes, there’s actually
0:28:02 a box I want to double click on.
0:28:04 There’s the agent orchestration.
0:28:11 You can now use not just one agent, but multiple agents, even different copies of the same agent
0:28:13 for them to parallelly do things.
0:28:16 What’s the implication of this?
0:28:21 What can you do with multiple agents orchestrating them together that you couldn’t do before?
0:28:24 I mean, we’re all very ADHD as developers.
0:28:27 I think there’s a couple of things.
0:28:28 Work faster, obviously, right?
0:28:32 But you can also try out multiple approaches in parallel and see which one works best.
0:28:35 I’ve seen approaches where people, for example, they want to optimize a certain code.
0:28:39 They fire off a couple of agents with slightly different approaches and then just measure which
0:28:39 one works best.
0:28:40 I see.
0:28:44 And there’s some startups proposing doing that in a more automated way even, right, where
0:28:49 you would just do this, you know, without human intervention.
0:28:50 You just say, optimize this.
0:28:52 And then, you know, this gets kicked off in the background.
0:28:56 And this all takes a crazy amount of tokens at the end, right?
0:29:03 So, I mean, I think this is a really, another really interesting trend where three months
0:29:07 ago, I don’t recall anybody talking about the cost of coding assistance.
0:29:08 Right.
0:29:08 Yeah.
0:29:10 I mean, three months ago, honestly, nothing, right?
0:29:14 You go to a Reddit forum on Cursor, so there was, you know, pretty much nothing posted there.
0:29:18 Today, there’s one of the number one topics in those forums.
0:29:20 Maybe the number one topic, right?
0:29:25 Because I think we figured out how with high-powered reasoning models and very large context windows,
0:29:31 we can have single tasks that suddenly cost dollars.
0:29:35 Not sure about tens of dollars, but at least, you know, dollars is very, very doable.
0:29:37 And that adds up, right?
0:29:41 It depends, you know, if you’re a super high-end programmer, maybe it doesn’t matter.
0:29:46 If anybody else, a couple of dollars an hour, you know, that’s a substantial expense.
0:29:52 If you’re in a low-cost location, it may end up costing more than what you’re making.
0:30:00 It used to be that if I was writing software, oversimplifying slightly, I had one expense,
0:30:01 which is people, right?
0:30:05 Okay, they needed, you know, a laptop and some connectivity and an office.
0:30:09 But in the grand scheme of things, the cost that really mattered, you know, at least in
0:30:13 the, you know, more high-cost locations was the compensation of the person.
0:30:16 That seems to have changed now.
0:30:20 We suddenly have infrastructure costs for a software engineer, right?
0:30:23 They need the constant feed of LLM tokens to keep them happy.
0:30:24 Otherwise, they’re not productive.
0:30:28 They’ll probably change the industry somehow, right?
0:30:30 And I think, unless we understand how, right?
0:30:33 We know people will be building more, that’s for sure.
0:30:34 That’s right.
0:30:39 And the question becomes, does building more correlate with more tokens burn to some extent,
0:30:40 right?
0:30:46 And I met so many great engineers who are burning, you know, like they’re the top token burning
0:30:48 engineer of the company and they’re just so effective.
0:30:49 Yeah.
0:30:52 They have like two laptops side by side and, you know, around coding agents that way.
0:30:57 It’s like the shift from digging versus driving the excavator.
0:30:58 Right.
0:31:00 It’s like breastfeed search instead of done.
0:31:01 But that changes the industry.
0:31:01 Right.
0:31:03 Probably the person is slightly happier, right?
0:31:04 I mean, that’s what happened in that case.
0:31:06 You need less of them.
0:31:09 People will, but you can also build a lot more, right?
0:31:11 So I think it’ll change the industry then.
0:31:16 I think there’s more customization to software, to your point about building more, because
0:31:21 there’s always one bespoke tool for any business out there.
0:31:23 You know, there’s HR software.
0:31:27 It covers 80% of what every company needs.
0:31:32 The 20%, like I know this really well because I used to be a PM on these enterprise softwares.
0:31:33 We build APIs.
0:31:36 So internal teams build their version.
0:31:36 That’s right.
0:31:36 Yeah.
0:31:39 And then now we just have turtles all the way down.
0:31:41 Like we build the base layer.
0:31:43 The internal team builds next layer.
0:31:45 And then developers are like, oh, it still doesn’t work.
0:31:46 Let me build something else.
0:31:52 But now with Vibe coding, I actually think customization is just easier and easier than ever.
0:31:58 You may or may not even need a centralized team to build that layer using, you know, a commercial
0:31:59 solutions APIs.
0:32:01 You can just code it up yourself like what I did.
0:32:08 Something I’ve been thinking about is what’s the next workflow or automation going to be?
0:32:14 Like before we have, you know, Zapier and other great RPA tools of the world to make it work.
0:32:19 Now with Vibe coding, obviously you still need to know code to some extent to make it work.
0:32:20 How would that change?
0:32:24 I think we’ll just end up having more workflows running somewhere.
0:32:36 The question is like, how can we unlock the net new, like new developers who are not traditionally technical, but now are writing code to implement these?
0:32:44 Maybe they don’t need a graphical interface or if they do need a graphical interface, it can be represented by JSON, which is more agent friendly.
0:32:50 Yeah, in fact, we’re starting to see almost self-extending software, right?
0:32:55 Software where a user with a prompt can add additional functionality.
0:32:56 Yeah, that’s so true.
0:32:57 Yeah, yeah.
0:32:58 Which is crazy.
0:32:59 Is that a trend?
0:33:01 I think it is a trend.
0:33:08 Will the next version of Microsoft Word, not the next version, but maybe a couple of versions on the road, have a, you know, add feature button and they help menu?
0:33:16 So software, I think the takeaway point is software is having more affordance than before because of the ability to integrate LLM.
0:33:24 And what I mean by that is before, if I’m a marketing company, I ship a feature so people can visualize six charts.
0:33:27 Now I ship a chat session with the LLM.
0:33:35 LLM can reach back to my data and the LLMs generate code to materialize whatever charts people want to see.
0:33:36 So it’s more than six.
0:33:41 It’s like thousands and hundreds and thousands of things that people will want to see.
0:33:52 So the interaction model between end user of the marketing software and LLM becomes, it can materialize, not new features using their own words.
0:33:59 So prompt, which is very different from the four software teams that are shipping feature by feature.
0:34:03 So now with all that, what do you think people want to build?
0:34:06 Or what do you think developers should be building?
0:34:07 What do you think the world needs?
0:34:09 It needs so much.
0:34:11 I mean, there’s two things I’m sure about.
0:34:22 One is, this is, I would say over the last three, four decades, probably the best moment in time to start a company in the development space, right?
0:34:31 If you have such a massive disruption, this is what allows a startup to really grow and scale and pick a battle with the incumbents.
0:34:39 And I think we’ve seen that with GitHub Copilot from Microsoft, first in the market, relationship to the number one model company with OpenAI.
0:34:41 You know, they have the number one source code repo.
0:34:42 They have the number one IDE.
0:34:43 They have the best enterprise sales force.
0:34:49 And still, we’re seeing, you know, a swarm of competitors that are all doing very, very well against them.
0:34:52 So this is really the time.
0:34:58 The second thing that I’m 100% convinced of is that the good ideas are not coming from the VCs, but from the entrepreneurs.
0:35:07 So, you know, if you spot an opportunity to do something better with AI right now, you can probably add value, right?
0:35:11 And then it’s about fast executions, about building a great team.
0:35:14 You know, it’s about running very, very fast.
0:35:24 But, you know, my prediction is I think we will fund dozens of companies in this space going forward.
0:35:27 Yeah, we are excited to just fund the next wave of startup.
0:35:31 But if you’re looking for ideas, here are two general directions.
0:35:35 The first one is what are the traditional workflows that you can now reinvent?
0:35:37 It might not be one-to-one.
0:35:40 So, like, a better Git may not be Git exactly.
0:35:43 It might be Git and something else, right?
0:35:50 And then if you just map out the value chain, like what we have on the blog post, you can pick a box and decide what you want to do with it.
0:35:50 I think that’s right, yeah.
0:35:52 That’s one way to do it.
0:35:55 The other way to do it is very differently from before.
0:36:01 Like, as product people, we used to only build for humans, other developers.
0:36:04 Now we actually build a lot for the agents.
0:36:05 Agents are the customers.
0:36:07 Does the agent need a better context?
0:36:08 That’s right, yeah.
0:36:09 Well, you should build for that.
0:36:13 Does the agent want, you know, lower latency for certain models?
0:36:19 Well, there are companies shipping, you know, code-apply models that, you know, operates way faster and with higher accuracy.
0:36:20 That’s also a need.
0:36:26 And when you look into where agents don’t yet work, there’s just plenty of things to work on.
0:36:30 You know, from just, like, easily resumable sandbox.
0:36:33 I know there are great companies already building that.
0:36:41 Or from, too, like, how do you enable the agent to kind of smash PR review process with the development process?
0:36:44 There’s just so much more there to reinvent how agents could work.
0:36:46 So, treat agents as your customer.
0:36:46 Yes.
0:36:47 And build for them.
0:36:49 The classic infrastructure, right?
0:36:49 Yeah.
0:36:51 Absolutely.
0:36:55 No, I think this is really an amazing time to start a company.
0:36:55 Yeah.
0:37:04 Thanks for listening to this episode of the A16Z podcast.
0:37:11 If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.
0:37:15 For more episodes, go to YouTube, Apple Podcasts, and Spotify.
0:37:21 Follow us on X and A16Z, and subscribe to our Substack at a16z.substack.com.
0:37:24 Thanks again for listening, and I’ll see you in the next episode.
0:37:39 As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.
0:37:44 Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.
0:37:51 For more details, including a link to our investments, please see A16Z.com forward slash disclosures.
0:00:07 When do we say, this is all agents?
0:00:12 We just, at the end of the value chain, are like, does this work or not work?
0:00:13 Click yes or no.
0:00:16 Agents, more than ever, need an environment to run these things.
0:00:19 Context engineering for both humans and agents.
0:00:22 Every single part of it is getting disrupted.
0:00:26 It’s not that there’s just somebody writing code, like your classical developers being
0:00:28 disrupted, but everybody along the value chain.
0:00:34 We’re resurfacing an episode from the AI plus A16Z podcast that we think deserves a wider
0:00:35 audience.
0:00:41 A16Z partners Yoko Lee and Guido Appenzeller make the case that AI coding is the first truly
0:00:44 massive market for AI, potentially worth trillions.
0:00:49 They break down what’s changing in the dev loop, where ROI is showing up first, and what
0:00:50 founders should build next.
0:00:51 Hope you enjoy.
0:00:59 So Yoko, we just launched this, I think, amazing new dev stack for the AI coding.
0:01:02 And I’m really, really excited about this.
0:01:02 Yeah.
0:01:07 I mean, let me start with a very high order pitch, why I think this is so incredibly exciting.
0:01:13 I think AI coding is the first really large market for AI, right?
0:01:15 I mean, we’ve seen there’s a ton of investment that has flown in.
0:01:18 The question now, to some degree, is where is the value, right?
0:01:19 Why are we doing all this?
0:01:23 AI coding can create an incredible amount of value.
0:01:27 If you think about this, we have about 30 million developers worldwide, roughly, right?
0:01:30 Let’s say each of them generates $100,000 in value.
0:01:34 In the United States, it may be low because many of them get paid a lot more.
0:01:36 But internationally, it might be a little high.
0:01:37 But I think in order of magnitude, it holds.
0:01:42 So in aggregate, the value we’re creating here is about 30 million times $100,000.
0:01:44 So $3 trillion.
0:01:47 I will argue even more because that’s just developers.
0:01:50 But then there’s also people who are development curious.
0:01:51 They were not developers.
0:01:54 Maybe they’re, I mean, design engineering now is a big thing.
0:01:57 Every designer should just, you know, write code.
0:01:57 Doc writers.
0:01:58 Exactly.
0:01:59 I mean, there’s so many of these effects.
0:02:03 But if you just take the $3 trillion figure, that’s about the GDP of France.
0:02:09 So the claim we’re making here, as crazy as it sounds, is that we’re saying the entire population
0:02:14 of the seventh or eighth, I think, largest economy on the planet generates about as much value
0:02:20 as a couple of startups that are reshaping the AI software development ecosystem plus the LLMs underneath.
0:02:20 Yeah.
0:02:21 That’s crazy.
0:02:25 Everything we see and touch and use nowadays are all software.
0:02:26 That’s right.
0:02:27 Yeah.
0:02:31 So we’ve, software has disrupted everything in the world.
0:02:35 And now software itself is getting massively disrupted.
0:02:35 Totally.
0:02:39 And then what you mentioned in the blog post is really interesting just because,
0:02:44 we now are more capable at using LLM to generate code and coding and produce software.
0:02:48 But then as a result, it’s not like there’s less jobs.
0:02:51 It’s actually more and more software is being produced.
0:02:56 Before, maybe as a SaaS service catering to hundreds of people’s needs, to thousands of
0:02:56 people’s needs.
0:03:00 Now you can really just vibe code things software for one.
0:03:00 Yeah.
0:03:01 Yeah.
0:03:01 I vibe code.
0:03:02 I do that.
0:03:02 Exactly.
0:03:03 You do that.
0:03:03 Exactly.
0:03:06 And then I vibe code my own email filter.
0:03:11 You know, I don’t do that so much of using LLM to reply to my email, but I have a filter
0:03:13 where I categorize the labels, things like that.
0:03:14 Only to some emails.
0:03:15 Only to some emails.
0:03:22 So the first question becomes like, how do you think the development loop is shifting?
0:03:24 I think the answer is complex.
0:03:28 And very frankly, it’s so early, I think, in this AI revolution.
0:03:30 We don’t have the full answer yet, right?
0:03:34 But I mean, we have our little stack, our little software development lifecycle post AI
0:03:35 in the blog post.
0:03:39 And I think the biggest learning from that is probably that every single part of it is getting
0:03:40 disrupted.
0:03:44 It’s not that there’s just somebody writing code, like your classical developers being
0:03:47 disrupted, but everybody along the value chain is getting disrupted.
0:03:49 What’s the most surprising part for you?
0:03:52 What’s the most disruptive field today in coding?
0:03:55 And what do you think AI will come after next?
0:03:59 So I think, well, we’ve seen the biggest growth, I can say to say, in the classic sort of coding,
0:04:02 IDE integrated coding assistance or more agentic coding assistance, right?
0:04:07 The cursors and devins and GitHub co-pilots and cloud codes of the world, right?
0:04:12 I think that’s where we see the most traction, where we see an incredible revenue growth.
0:04:16 I mean, I want to say that segment possibly has the fastest revenue growth of any startup
0:04:21 sector we’ve seen in the history of startups, which is, again, an incredible statement.
0:04:24 So I think this is currently the vanguard, right?
0:04:25 And everybody’s aware of it.
0:04:28 We’re seeing billion-dollar, you know, acqui-hires or takeover offers.
0:04:30 So that’s an incredibly vibrant sector.
0:04:32 Now, which one is next?
0:04:33 That’s a really good question.
0:04:38 So to be very specific, in a blog, we wrote about the basic loop.
0:04:41 The basics is you plan, you code, you review.
0:04:44 Where does LM coming in?
0:04:46 Where do you think more of the loop would be disrupted?
0:04:51 Do you think the loop will still be the same as what we used to have, the basic?
0:04:53 Or do you think it will look very different?
0:04:56 I think at this point, it’s very hard to speculate about the end state.
0:04:58 But if you, let’s assume you’ve seen the first email.
0:05:00 I’ll get there.
0:05:04 But if you looked at the first email sent over the internet, right?
0:05:06 You can sort of predict that probably we’ll have websites and these things.
0:05:08 Maybe if you’re good, you can, right?
0:05:12 But, you know, saying like, hey, the net effect of this is that everybody can rent out their
0:05:14 house and compete with hotels.
0:05:15 And there’s going to be the biggest hotel company on the planet.
0:05:17 You’d be like, well, that’s a little far-fetched.
0:05:18 But now we have Airbnb, right?
0:05:21 So these secondary effects, I think, are really hard to guess.
0:05:24 My current hypothesis is I think we’ll still have software developers.
0:05:26 I think they’re not going anywhere, right?
0:05:30 I think what they do will look completely different.
0:05:31 Right.
0:05:31 Right.
0:05:31 Yeah.
0:05:39 I think the CS education, frankly, any CS class taught today at any major university
0:05:42 is probably best seen as this historical relic from a bygone time, right?
0:05:47 I mean, if you look at the best-of-breed startups, what they’re doing, the loop that the developer
0:05:49 is in looks so different from what you did before, right?
0:05:53 You have multiple agents that you’re prompting, that you’re telling things, you know, and pull
0:05:53 that back into UI.
0:05:55 You’re trying to understand what they did.
0:05:56 You’re trying to put them back on the rails.
0:05:58 It’s a lot more thinking at a higher level.
0:06:02 I mean, all of coding sort of has been high levels of distractions, but I think we’re
0:06:03 making a huge leap here.
0:06:05 So how it’s going to look like, I have no idea.
0:06:08 My gut feeling is we’ll probably have more developers.
0:06:14 Look, this basic plan-execute cycle, there’s probably going to be some flavor of that still
0:06:15 around is my guess.
0:06:16 But what do you think?
0:06:21 So one of my top questions is that, would this be a step-by-step loop or would this
0:06:23 mesh into just one step?
0:06:29 So one example is, if my agent is writing a code, do I still need to review it or do I
0:06:31 have another agent that just reviews it?
0:06:34 If it’s the same agent, like implementation detail is one thing.
0:06:39 You can separate out the agent generating code from the agent reviewing code.
0:06:44 But then if it’s all agent, both generating and reviewing code, is it just the same step?
0:06:49 Do we actually disaggregate the step-by-step process and have human in the loop?
0:06:55 Whereas if as a human, I write code and I want agent to review code, that makes sense to me.
0:07:02 So I do wonder, when do we pull out something as this is an individual tool and individual
0:07:03 stuff an agent takes care of?
0:07:06 And when do we say, this is all agents?
0:07:10 We just, at the end of the value chain, we’re like, does this work or not work?
0:07:11 Click yes or no.
0:07:16 I think the time periods of which an agent can work autonomously will get longer.
0:07:21 You know, still, if somebody says, look, I want to write a complete ERP system for my
0:07:26 multinational enterprise goal, there’s no way I could imagine that it’ll just run and
0:07:29 back come software that actually fits the requirements.
0:07:33 And in part, I think it’s a problem that models are still very, very far away from being able
0:07:34 to run autonomously for that long.
0:07:38 But the other problem is, let’s assume this was an all-human team.
0:07:41 We wouldn’t understand all the challenges yet at the beginning, right?
0:07:42 We’d have to revisit the design.
0:07:45 We have to revisit the architecture that has cost implications and so on.
0:07:49 So at some point, you sort of need to go back to the architects and the product managers
0:07:53 and say, hey, we had a plan A, didn’t quite work, or we have found new challenges.
0:07:55 So here’s our updated plan A.
0:07:57 Is that the, you know, or plan B, right?
0:07:58 Is this what you want to do?
0:08:00 So I think the loop will still be there.
0:08:01 The timescales will probably change.
0:08:03 But yeah, it’s very, very hard to guess right now.
0:08:10 Another thing we start to see more often is, contrary to how much humans need to come and
0:08:15 intervene in the loop, we actually start to give agents tools for them to know what they
0:08:16 have to do.
0:08:22 One example is, this loop I see so often, like the agent wants to implement, say, clerk
0:08:23 auth in their app.
0:08:29 Now they need to go to Mitlify and context 7 to say, what is the latest version of clerk
0:08:32 and how can I implement it correctly and in what file?
0:08:38 I’m not going to copy paste it to cursor or give it to the agent because I’m too lazy as
0:08:39 a human now.
0:08:44 The agent should be able to call the API themselves to put stuff in the context to make it work.
0:08:48 This is just one example of what behavior change we’re seeing.
0:08:52 Because before, as developers, we’re so used to go back to the docs and refer to the docs
0:08:54 and tell the agent what to do.
0:08:55 Now agents can obviously just-
0:08:56 We cut off the middleman.
0:08:57 Right.
0:08:58 We cut off the middleman.
0:09:01 I don’t need to route all these requests for the agents anymore.
0:09:04 I think there’s other examples, which is verification.
0:09:10 As a human, before when I write code or review other people’s code, I pull out the code.
0:09:14 And then the first thing I do is actually not to review because I don’t like reading code.
0:09:15 I’m not a human compiler.
0:09:20 The first thing I do is to fork the change and see if it still works.
0:09:22 If it doesn’t work, I just do not review it.
0:09:28 Nowadays, there are opportunities to give just agents a native environment to first see,
0:09:29 does this work?
0:09:31 Does the UI still look good?
0:09:32 Do all the requests still check out?
0:09:33 Yeah, absolutely, yeah.
0:09:34 Did it break my build?
0:09:37 Before, the human needs to come in and review.
0:09:41 Maybe that manifests itself in part of the local development process.
0:09:44 Maybe it manifests itself in the PR review process.
0:09:49 But in any case, now agents, more than ever, need an environment to run these things.
0:09:52 When I used to write something just for myself, like a little script I need somewhere in the past,
0:09:54 I usually didn’t include unit tests, right?
0:09:55 For production code, it’s different.
0:09:58 But for just personal things, you know, it’s like, yeah, that’s a single developer.
0:09:59 I know what I’m doing here.
0:10:03 With agents, I now start including unit tests because they’re so much easier to write.
0:10:07 And they allow an agent, as you said, right, to understand if the changes that they did
0:10:08 broke anything else, right?
0:10:13 And they may not have the context how this originally was built anymore in an easily digestible form.
0:10:14 So that’s super valuable.
0:10:19 On the grand scheme of, like, how much economic value this generates,
0:10:23 where in the value chain do you think it’s growing the fastest?
0:10:28 Like, you see where the agents is producing so much more value than other areas.
0:10:30 But what are the areas you feel like would be the next takeoff?
0:10:36 So, look, I’m talking about 100 or so enterprises about this per year.
0:10:39 Just when we, you know, take our portfolio companies to them as potential customers.
0:10:44 What I’m hearing from them is that the number one use case in terms of ROI,
0:10:47 right now is legacy code porting, right?
0:10:48 It’s not super surprising.
0:10:50 Like, one of the first papers in the space from Google, right?
0:10:53 They wrote a fantastic paper on, you know, where they detailed on, you know,
0:10:58 just doing very mundane things like replacing a Java library across a very large code base,
0:10:59 right?
0:11:00 Not like millions of lines of code, right?
0:11:01 So it’s a very large code base.
0:11:04 What do you consider as legacy stack?
0:11:05 What do you consider as new stack?
0:11:07 Well, it totally depends on you.
0:11:10 But I mean, for the banks, it’s often COBOL or Fortran to do Java.
0:11:11 Oh, COBOL.
0:11:12 I haven’t heard that word for a long time.
0:11:17 You know, I actually wrote COBOL code once in the 90s.
0:11:19 It probably dates me at this point.
0:11:19 Okay.
0:11:22 Actually, how are LLMs with COBOL?
0:11:23 They’re apparently extremely good.
0:11:25 That’s a surprise.
0:11:26 So here’s the thing, right?
0:11:32 One of the hardest thing, if you implement code with LLMs, is just getting the specification precise, right?
0:11:36 If I can specify something very precisely, then usually the LL can do a good job at implementing it.
0:11:39 So many of these companies do is they take legacy code.
0:11:43 They have an LLM, write a specification that fits the legacy code.
0:11:45 And then they say, re-implement the specification.
0:11:50 And you may look at the code if there, you know, as a tiebreaker, if something is not clear, right?
0:11:52 And that seems to work incredibly well.
0:11:55 So I’m hearing that today, I heard from several.
0:12:00 sources now that you can get, you know, about a 2x speed up versus traditional processes for that, right?
0:12:02 And that’s amazing, right?
0:12:09 And what this has led to is that actually, of those enterprises that I’ve talked to, the majority says they’re currently accelerating,
0:12:15 at least the majority of those that are sophisticated about this, they’re saying they’re accelerating the developer hiring, right?
0:12:20 We don’t know if this is a long-term trend, but right now they’re busy saying, look, because we found so many low-hanging fruit type projects
0:12:24 where with a little bit upfront investment, we can then save infrastructure costs.
0:12:26 And that’s super exciting.
0:12:30 So, you know, what this will mean for the, you know, how much longer is the mainframe business going to be around?
0:12:31 Or, you know, I don’t know.
0:12:37 But there’s definitely a shift there where suddenly legacy code migration is much, much easier than it was before.
0:12:42 You know, I think that they’ll change a lot of the dynamics in the sort of classic enterprise software space.
0:12:42 That’s interesting.
0:12:46 I do wonder if we will get new mainframe code.
0:12:50 Because now, because before, no one knows how to program those things.
0:12:51 That could also be, yes.
0:12:56 And now you realize you can program mainframe using natural language.
0:12:56 Yeah, totally.
0:13:02 So, another possibility is that we get renaissance of, like, the underlying legacy coding languages.
0:13:08 I mean, it’s crazy to me how versatile these coding assistants are, right?
0:13:10 I mean, we’re seeing them write CUDA kernels.
0:13:10 Yeah.
0:13:12 Which is like, that is difficult stuff to write.
0:13:12 Right, totally.
0:13:13 By any metric.
0:13:13 Yeah.
0:13:18 I’ve tried them with a language which basically has no usable training data set.
0:13:26 And they’re still able to sort of abstract, you know, with a couple of examples, you know, of how the code will have to look like.
0:13:30 It’s not perfect, but so I think it’s a very, very broad technology for sure.
0:13:35 Recently, just like what we were talking about before, code reviews.
0:13:40 Because, like, to your point, LLMs are so good at coding and generating code.
0:13:43 Sometimes it’s beyond our comprehension.
0:13:46 We’ll take more time to review the code than the coding agents.
0:13:49 Like, that’s not a controversial opinion.
0:13:50 It’s just like how it is.
0:13:50 That’s the reality.
0:13:51 Yeah.
0:13:58 So, it does make me wonder how our development chain and steps are going to evolve from here.
0:14:04 How are we going to do PRs when we can’t possibly review thousands of lines of code as humans?
0:14:07 So, does that mean the right abstraction now is still code?
0:14:11 Or does it mean the right abstraction now is for us to review plans?
0:14:17 If that’s the case, is GitHub review still the right abstraction for that?
0:14:20 I think there’s still a role for review in general.
0:14:24 I think the question is, will humans do the review?
0:14:31 Like, right now, most of the code that LLM generates, you know, unless you’re deep in vibe coding territory, you’re just like, oh, this is a one-off.
0:14:33 I just want to try something out.
0:14:35 Maybe then you don’t review it.
0:14:36 You just hit accept and hope for the best.
0:14:41 But anything that, you know, anything else, you do review the, I still review the code line by line.
0:14:46 You know, that said, we’re starting to see really good tools that plug into your backend system, your GitHub.
0:14:55 Whenever a pull request comes in, they analyze it, they comment on it, they point out security vulnerabilities, they point out that the spec is different from the implementation.
0:15:00 You know, they point out that this creates dependencies, which may not be desired.
0:15:04 They, you know, enforce coding guidelines, which is very, very powerful, right?
0:15:06 I haven’t met anybody yet.
0:15:15 Tell me if you have, but I haven’t met anybody yet who basically has said, we’re going to rely purely on AI to review code, right?
0:15:17 We just, anything can go in if the AI checks off on it.
0:15:23 But I have seen, for example, companies that are saying before we had two developers review code, and now it’s one developer, right?
0:15:31 And, or, you know, cases where basically just the AI hangs out in the GitHub, you know, discussion and comments on things.
0:15:33 You know, if there’s, if they, you know, so you can basically delegate tasks.
0:15:34 It’s like, can you look at this?
0:15:36 It’s like, are we using this library somewhere else?
0:15:40 So basically you can, you suddenly have somebody who can, who can help with these tasks.
0:15:52 My hot take on this is actually, if PR is supposed to give us a context as developers, what did these other coding agents or my colleagues, you know, change that I should be aware of?
0:15:55 I don’t think code, reviewing code is the right abstraction.
0:16:00 Because it might be feature level, it might be performance level.
0:16:03 So now, now it might just become a one-liner.
0:16:09 This, you know, this agent improved your CUDA implementation.
0:16:14 And I, maybe I don’t even know CUDA, but I know the improvement when I can verify it.
0:16:20 So the question for me is, do I still need to go to the PR review every line?
0:16:27 Or should I just be given, you know, two sentences to know how this works and the environment to test it out?
0:16:28 And then I would just merge it.
0:16:30 If it’s the right two sentences that works.
0:16:30 Yeah.
0:16:33 Will the, will the LLM always pick the right two ones?
0:16:34 I don’t know yet.
0:16:34 Yeah, that’s true.
0:16:39 I mean, if you give it an environment, the question is like, can you verify the environment?
0:16:40 Against the two sentences?
0:16:41 Yeah.
0:16:43 If the answer is yes, that would be easier to solve.
0:16:46 If the answer is no, you know, that’s harder.
0:16:53 But I think there’s a bigger picture here, though, which is that LLMs are also very good in generating the documentation.
0:16:54 That’s true.
0:16:55 And description for the code, right?
0:17:05 So when I use something like Cursor for coding, and, you know, it generates code, I often ask it afterwards and now take the internal documentation and update it.
0:17:12 Because the internal documentation is important for me, but also for the coding agents because they want to be able to refer to it.
0:17:16 You don’t want every single time to take the entire code base and stick it into the context window.
0:17:17 It’s massively inefficient and slow.
0:17:22 So if instead you can just say, like, read this document that will explain to you the class hierarchy, right?
0:17:25 And then based on that, implement this new subclass.
0:17:27 So it’s much, much faster to do.
0:17:33 I think there’s a real opportunity there to get to much better documented code than previously, right?
0:17:33 Right.
0:17:34 That’s true.
0:17:39 A compiler is great, and then it takes a high-level abstraction, translates it into a lower-level one.
0:17:46 But now we have the ability that if somebody had in hand optimizes the lower-level one, we can use that to update the higher-level one.
0:17:46 That’s true.
0:17:50 What is the new compiler in the age of AI?
0:17:52 LLMs are sort of a compiler, right?
0:17:53 In some way.
0:17:53 That is true.
0:17:57 They take a higher-level description and fill it out.
0:17:58 I mean, I think the big difference…
0:18:01 I guess what’s missing is it doesn’t natively have an environment.
0:18:05 Compiler gives you, does this work or does it compile or does it not compile?
0:18:09 It doesn’t tell you, does this work, which is a very subject thing.
0:18:10 I think it’s right, right?
0:18:17 And like a compiler enforces, is a strict enforcement of certain things, right?
0:18:21 You can, I can rely on that, I don’t know, I’m using Rust and things will be typed, right?
0:18:23 So that’s a huge step forward, right?
0:18:24 And I can exclude certain bugs.
0:18:27 With an LLM, that’s not the case, right?
0:18:33 Now, that said, you sort of wonder, is this just an initial thing?
0:18:34 Is this just a…
0:18:36 I mean, LLMs over time…
0:18:42 Like, for example, we can give LLMs tools that allow them to syntactically parse code, right?
0:18:44 And then now suddenly they can start reasoning above code.
0:18:49 They can ask, are we sure that the representation of object X is the same across these different,
0:18:52 you know, modules, even if it gets serialized in between or something like that?
0:19:00 I don’t mean to make this episode disaggregating GitHub, but GitHub is so central to so much
0:19:01 of our workflow, right?
0:19:03 Everything goes through it.
0:19:07 From the social aspect of discovering what other developers are doing, to distributing the
0:19:12 software, you can download things, to keeping track of what you have changed, Git, right?
0:19:14 The integration of the build system, the backend, yeah.
0:19:22 Yeah, so now another interesting thing we start to see is people use Git repos very differently
0:19:22 now.
0:19:28 Before, it was like, oh, humans will make some changes, we’ll commit it, and then other people
0:19:28 will see it.
0:19:31 And then we, you know, open the PR, people see different revisions.
0:19:35 Now agents are doing so many changes.
0:19:38 It’s kind of counterproductive to commit everything.
0:19:45 And then repos usually have very low rate limits because it’s designed for humans to use.
0:19:51 So now the question becomes, what is the new repo, like, abstraction?
0:19:58 Like, that handles, like, a distributed, like, infrastructure, but also has, caters to high-frequency
0:19:58 commits.
0:20:04 And sometimes when agents doing these commits, they don’t even want to preserve this forever.
0:20:11 It’s more like an intermediate step so that they can explore five different paths, you know,
0:20:13 but then revert to any of them.
0:20:15 And then when they’re happy with it, they commit back to GitHub.
0:20:21 So one thing I’ve been playing around with is there’s this company called Relays, and then
0:20:27 their docs has this repos feature where you can’t give a repo to the agent.
0:20:31 The agent will, you know, commit stuff, like, very high-frequency, kind of works really well
0:20:32 with bi-coding agents.
0:20:34 And that has been such a great experience.
0:20:38 And in the wild, I start to see people building this internally, too.
0:20:40 It makes total sense.
0:20:47 Look, if we completely change how humans write code, or we’re shifting from humans to delegating
0:20:52 most of the writing to agents, I think it would be foolish to assume that the underlying
0:20:56 services that were a good fit for the human world are still a good fit for this new agent.
0:20:57 Yeah, for sure.
0:21:00 That’s almost certainly not the case, where we can do better in our cases.
0:21:03 I think you’re completely right, right?
0:21:06 For source code repositories, we want something that’s much more real-time.
0:21:09 Honestly, I mean, imagine, let’s take this to the limit, right?
0:21:14 Let’s assume I have now, I’m just me personally running 100 agents in parallel that all try to
0:21:15 implement features in my code base.
0:21:16 Yeah.
0:21:19 You probably need some kind of coordination mechanism between that.
0:21:21 They’re all not trying to edit the same file, right?
0:21:22 Oh, for sure.
0:21:24 You know, the rebase only carries you so far, right?
0:21:25 Right, yeah.
0:21:27 You just get too many collisions.
0:21:30 And they all need shared memory on the repo they’re working on.
0:21:31 That’s right.
0:21:33 Because they don’t want to reinstall the dependencies every time.
0:21:34 That’s right.
0:21:34 Yeah, exactly.
0:21:40 And so, you know, you probably need something that’s much, much more flexible and more real-time.
0:21:42 And, you know, I think we’re still figuring out what that is.
0:21:44 What Relays is doing there is amazing.
0:21:48 And I think it’s not just true for GitHub.
0:21:50 And GitHub is one of the big platforms we use.
0:21:54 But, you know, take the specification writing with, you know, say a Confluence or Jira, for
0:21:55 example, what you could use there.
0:22:01 If you would develop these systems bottoms up with AI in mind, they would probably look very,
0:22:03 very different, right?
0:22:06 I mean, it’s like, you know, should my story tracker have a function that looks at code and
0:22:07 updates stories accordingly?
0:22:10 I mean, yeah, that would be very natural, right?
0:22:12 So I think you can do it.
0:22:14 I mean, I think we’ve seen this for documentation, right?
0:22:18 I mean, something like Mintlify looks very, very different at the documentation problem than
0:22:20 previous generations.
0:22:21 I think we’re seeing it for testing.
0:22:23 We’re seeing it for PR reviews.
0:22:26 We’re rethinking this entire stack.
0:22:26 And that’s exciting, right?
0:22:27 That’s true.
0:22:32 Another behavior change we have seen, which is really interesting, because humans, like,
0:22:34 developers who are very lazy.
0:22:34 Yeah.
0:22:38 So if we don’t have to read it, we do not read it.
0:22:40 We only read the relevant parts in documentation.
0:22:42 We only skim through things.
0:22:48 So the net new behavior we see all the time with, you know, documentation, like, hosted on
0:22:53 Mintlify is that users who are net new just go in and ask questions.
0:22:54 Yeah.
0:22:57 And agents do not just ingest it anymore.
0:23:01 They will actually just do a query against this context.
0:23:07 So context engineering for both humans, because our brains are like, oh, so we need the context
0:23:11 and agents who need context, which is a very critical part.
0:23:15 It’s such a critical part of, like, development from here point on.
0:23:20 And then the question becomes, what are the other tools you think agents will need?
0:23:25 Like, we had this huge, you know, market map from the blog.
0:23:31 And then there’s, contrary to where, you know, all the previous market map suggests, which is
0:23:33 like, here are the developer tools.
0:23:35 Now there are agent tools to make it better.
0:23:38 In some cases, the same tool caters to both, right?
0:23:42 I mean, that’s a media documentation for agents and for humans.
0:23:42 Right.
0:23:47 So look, early days, but I think we’re seeing a couple of big categories emerge, right?
0:23:52 I mean, sandboxes are helpful to try out, you know, code snippets or build things.
0:23:53 How do you define a sandbox?
0:23:56 I think we can record the whole episode on it, but.
0:23:58 And I think we will at some point.
0:24:02 But look, at the end of the day, you need an environment with certain safety guarantees,
0:24:12 right, where LLMs hallucinate, LLMs can be with clever, you know, if they use external
0:24:15 sources, they can potentially be maliciously prompted to do bad things.
0:24:20 And you just want to have something that basically limits how much the blast radius, if anything
0:24:20 bad happens.
0:24:26 I think we’re seeing interesting search tools and parsing tools, like something like source
0:24:27 graph or so, right?
0:24:31 If I’m writing a couple of my code in a couple of files, we don’t need that.
0:24:35 But if we have a really large code base, you know, and suddenly the question is, look,
0:24:39 we’re looking for, we’re trying to replace, we’re trying to add a parameter to a function
0:24:41 in a library that’s widely used.
0:24:42 Where is this library used?
0:24:44 This is a really hard problem suddenly, right?
0:24:48 I mean, it’s like if you’re in Python or so, you can import something as something else.
0:24:51 So like a simple find operation will no longer find these things.
0:24:53 You probably need syntactic parsing to find those.
0:24:59 So I think we’re seeing documentation tools that are optimized for agents and allow agents
0:25:00 to look up these things.
0:25:04 I think it’s good if an agent can do things like web search, right?
0:25:07 So, you know, we’re seeing companies in that space.
0:25:07 What am I missing here?
0:25:08 There’s so many more.
0:25:13 I think there’s the other part of more, we’re seeing more specialized models.
0:25:18 Yeah, I mean, for things like code editing or, you know, re-ranking files and things like
0:25:19 that, right?
0:25:21 That’s definitely shaping up as a market.
0:25:29 If you take a big step back, if we assume there’s massive value creation here, that probably
0:25:31 creates an opportunity to create a very large number of startups.
0:25:35 I mean, if you would have asked me 18 months back, I would have, actually, if you had asked
0:25:38 me 24 months back, I would have said, like, look, DevTools, it’s a smallish kind of market.
0:25:39 How big can this be, right?
0:25:40 That’s not very exciting.
0:25:45 If you ask me today, it suddenly looks like this is, you know, a market that could go
0:25:47 in the hundreds of billions of dollars, right?
0:25:48 In theory, could it go to a trillion?
0:25:49 I don’t know.
0:25:51 So how many companies can you create in that space?
0:25:53 I have no idea, but probably hundreds.
0:25:56 And over time, it’ll, you know, consolidate.
0:26:01 But I expect this to be an ecosystem, not a business model.
0:26:08 I have a fun question, which is, going forward, so when GitHub came out, they have this commit
0:26:15 charts, it’s, it was on t-shirts, you know, like people compare the commit charts, people
0:26:17 like commit specific messages.
0:26:21 So the commit charts look like, like has a good graphics.
0:26:22 What?
0:26:28 Because before, commits are so tied to the value developers bring.
0:26:30 Like, how many commits do you make?
0:26:32 How many lines of code do you change?
0:26:35 I mean, we all know it’s not the best proxy for value.
0:26:37 Yes, not at all.
0:26:38 Right.
0:26:42 So what would be the next commit chart on GitHub?
0:26:44 What would that look like?
0:26:46 What’s the next?
0:26:47 That’s great.
0:26:48 Wow, that’s an amazing question.
0:26:50 Maybe how many tokens you burn?
0:26:54 If you come to the office, look, I burned like 10 million tokens over the weekend.
0:27:00 But tokens burned could also be very ineffective prompting or context engineering.
0:27:03 Yeah, I just stuck my entire code base in the context window, maybe.
0:27:06 Is it the number of agents you use?
0:27:08 That’s even more game of value.
0:27:16 What is the unit of value that’s the closest approximation to what you’ve delivered as value
0:27:16 as developer?
0:27:18 Non-cached input tokens?
0:27:21 Might be too complicated.
0:27:21 I don’t know.
0:27:22 Yeah.
0:27:24 I’m taking this to a higher level.
0:27:27 The metrics how we evaluate software development are changing.
0:27:28 Right?
0:27:34 Where potentially a big complex refactoring isn’t that much work anymore.
0:27:34 That’s true.
0:27:38 Because I can let the LLM do this and it’s structurally easy.
0:27:43 A specific optimization in a fairly obscure area where the LLM has no training data, I
0:27:45 may have to do by hand and it’s vastly more valuable.
0:27:47 So, yeah, it’s complicated.
0:27:49 Maybe it’s number of apps vibe coded.
0:28:00 I think something else, which is there’s on the market map agent toolboxes, there’s actually
0:28:02 a box I want to double click on.
0:28:04 There’s the agent orchestration.
0:28:11 You can now use not just one agent, but multiple agents, even different copies of the same agent
0:28:13 for them to parallelly do things.
0:28:16 What’s the implication of this?
0:28:21 What can you do with multiple agents orchestrating them together that you couldn’t do before?
0:28:24 I mean, we’re all very ADHD as developers.
0:28:27 I think there’s a couple of things.
0:28:28 Work faster, obviously, right?
0:28:32 But you can also try out multiple approaches in parallel and see which one works best.
0:28:35 I’ve seen approaches where people, for example, they want to optimize a certain code.
0:28:39 They fire off a couple of agents with slightly different approaches and then just measure which
0:28:39 one works best.
0:28:40 I see.
0:28:44 And there’s some startups proposing doing that in a more automated way even, right, where
0:28:49 you would just do this, you know, without human intervention.
0:28:50 You just say, optimize this.
0:28:52 And then, you know, this gets kicked off in the background.
0:28:56 And this all takes a crazy amount of tokens at the end, right?
0:29:03 So, I mean, I think this is a really, another really interesting trend where three months
0:29:07 ago, I don’t recall anybody talking about the cost of coding assistance.
0:29:08 Right.
0:29:08 Yeah.
0:29:10 I mean, three months ago, honestly, nothing, right?
0:29:14 You go to a Reddit forum on Cursor, so there was, you know, pretty much nothing posted there.
0:29:18 Today, there’s one of the number one topics in those forums.
0:29:20 Maybe the number one topic, right?
0:29:25 Because I think we figured out how with high-powered reasoning models and very large context windows,
0:29:31 we can have single tasks that suddenly cost dollars.
0:29:35 Not sure about tens of dollars, but at least, you know, dollars is very, very doable.
0:29:37 And that adds up, right?
0:29:41 It depends, you know, if you’re a super high-end programmer, maybe it doesn’t matter.
0:29:46 If anybody else, a couple of dollars an hour, you know, that’s a substantial expense.
0:29:52 If you’re in a low-cost location, it may end up costing more than what you’re making.
0:30:00 It used to be that if I was writing software, oversimplifying slightly, I had one expense,
0:30:01 which is people, right?
0:30:05 Okay, they needed, you know, a laptop and some connectivity and an office.
0:30:09 But in the grand scheme of things, the cost that really mattered, you know, at least in
0:30:13 the, you know, more high-cost locations was the compensation of the person.
0:30:16 That seems to have changed now.
0:30:20 We suddenly have infrastructure costs for a software engineer, right?
0:30:23 They need the constant feed of LLM tokens to keep them happy.
0:30:24 Otherwise, they’re not productive.
0:30:28 They’ll probably change the industry somehow, right?
0:30:30 And I think, unless we understand how, right?
0:30:33 We know people will be building more, that’s for sure.
0:30:34 That’s right.
0:30:39 And the question becomes, does building more correlate with more tokens burn to some extent,
0:30:40 right?
0:30:46 And I met so many great engineers who are burning, you know, like they’re the top token burning
0:30:48 engineer of the company and they’re just so effective.
0:30:49 Yeah.
0:30:52 They have like two laptops side by side and, you know, around coding agents that way.
0:30:57 It’s like the shift from digging versus driving the excavator.
0:30:58 Right.
0:31:00 It’s like breastfeed search instead of done.
0:31:01 But that changes the industry.
0:31:01 Right.
0:31:03 Probably the person is slightly happier, right?
0:31:04 I mean, that’s what happened in that case.
0:31:06 You need less of them.
0:31:09 People will, but you can also build a lot more, right?
0:31:11 So I think it’ll change the industry then.
0:31:16 I think there’s more customization to software, to your point about building more, because
0:31:21 there’s always one bespoke tool for any business out there.
0:31:23 You know, there’s HR software.
0:31:27 It covers 80% of what every company needs.
0:31:32 The 20%, like I know this really well because I used to be a PM on these enterprise softwares.
0:31:33 We build APIs.
0:31:36 So internal teams build their version.
0:31:36 That’s right.
0:31:36 Yeah.
0:31:39 And then now we just have turtles all the way down.
0:31:41 Like we build the base layer.
0:31:43 The internal team builds next layer.
0:31:45 And then developers are like, oh, it still doesn’t work.
0:31:46 Let me build something else.
0:31:52 But now with Vibe coding, I actually think customization is just easier and easier than ever.
0:31:58 You may or may not even need a centralized team to build that layer using, you know, a commercial
0:31:59 solutions APIs.
0:32:01 You can just code it up yourself like what I did.
0:32:08 Something I’ve been thinking about is what’s the next workflow or automation going to be?
0:32:14 Like before we have, you know, Zapier and other great RPA tools of the world to make it work.
0:32:19 Now with Vibe coding, obviously you still need to know code to some extent to make it work.
0:32:20 How would that change?
0:32:24 I think we’ll just end up having more workflows running somewhere.
0:32:36 The question is like, how can we unlock the net new, like new developers who are not traditionally technical, but now are writing code to implement these?
0:32:44 Maybe they don’t need a graphical interface or if they do need a graphical interface, it can be represented by JSON, which is more agent friendly.
0:32:50 Yeah, in fact, we’re starting to see almost self-extending software, right?
0:32:55 Software where a user with a prompt can add additional functionality.
0:32:56 Yeah, that’s so true.
0:32:57 Yeah, yeah.
0:32:58 Which is crazy.
0:32:59 Is that a trend?
0:33:01 I think it is a trend.
0:33:08 Will the next version of Microsoft Word, not the next version, but maybe a couple of versions on the road, have a, you know, add feature button and they help menu?
0:33:16 So software, I think the takeaway point is software is having more affordance than before because of the ability to integrate LLM.
0:33:24 And what I mean by that is before, if I’m a marketing company, I ship a feature so people can visualize six charts.
0:33:27 Now I ship a chat session with the LLM.
0:33:35 LLM can reach back to my data and the LLMs generate code to materialize whatever charts people want to see.
0:33:36 So it’s more than six.
0:33:41 It’s like thousands and hundreds and thousands of things that people will want to see.
0:33:52 So the interaction model between end user of the marketing software and LLM becomes, it can materialize, not new features using their own words.
0:33:59 So prompt, which is very different from the four software teams that are shipping feature by feature.
0:34:03 So now with all that, what do you think people want to build?
0:34:06 Or what do you think developers should be building?
0:34:07 What do you think the world needs?
0:34:09 It needs so much.
0:34:11 I mean, there’s two things I’m sure about.
0:34:22 One is, this is, I would say over the last three, four decades, probably the best moment in time to start a company in the development space, right?
0:34:31 If you have such a massive disruption, this is what allows a startup to really grow and scale and pick a battle with the incumbents.
0:34:39 And I think we’ve seen that with GitHub Copilot from Microsoft, first in the market, relationship to the number one model company with OpenAI.
0:34:41 You know, they have the number one source code repo.
0:34:42 They have the number one IDE.
0:34:43 They have the best enterprise sales force.
0:34:49 And still, we’re seeing, you know, a swarm of competitors that are all doing very, very well against them.
0:34:52 So this is really the time.
0:34:58 The second thing that I’m 100% convinced of is that the good ideas are not coming from the VCs, but from the entrepreneurs.
0:35:07 So, you know, if you spot an opportunity to do something better with AI right now, you can probably add value, right?
0:35:11 And then it’s about fast executions, about building a great team.
0:35:14 You know, it’s about running very, very fast.
0:35:24 But, you know, my prediction is I think we will fund dozens of companies in this space going forward.
0:35:27 Yeah, we are excited to just fund the next wave of startup.
0:35:31 But if you’re looking for ideas, here are two general directions.
0:35:35 The first one is what are the traditional workflows that you can now reinvent?
0:35:37 It might not be one-to-one.
0:35:40 So, like, a better Git may not be Git exactly.
0:35:43 It might be Git and something else, right?
0:35:50 And then if you just map out the value chain, like what we have on the blog post, you can pick a box and decide what you want to do with it.
0:35:50 I think that’s right, yeah.
0:35:52 That’s one way to do it.
0:35:55 The other way to do it is very differently from before.
0:36:01 Like, as product people, we used to only build for humans, other developers.
0:36:04 Now we actually build a lot for the agents.
0:36:05 Agents are the customers.
0:36:07 Does the agent need a better context?
0:36:08 That’s right, yeah.
0:36:09 Well, you should build for that.
0:36:13 Does the agent want, you know, lower latency for certain models?
0:36:19 Well, there are companies shipping, you know, code-apply models that, you know, operates way faster and with higher accuracy.
0:36:20 That’s also a need.
0:36:26 And when you look into where agents don’t yet work, there’s just plenty of things to work on.
0:36:30 You know, from just, like, easily resumable sandbox.
0:36:33 I know there are great companies already building that.
0:36:41 Or from, too, like, how do you enable the agent to kind of smash PR review process with the development process?
0:36:44 There’s just so much more there to reinvent how agents could work.
0:36:46 So, treat agents as your customer.
0:36:46 Yes.
0:36:47 And build for them.
0:36:49 The classic infrastructure, right?
0:36:49 Yeah.
0:36:51 Absolutely.
0:36:55 No, I think this is really an amazing time to start a company.
0:36:55 Yeah.
0:37:04 Thanks for listening to this episode of the A16Z podcast.
0:37:11 If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.
0:37:15 For more episodes, go to YouTube, Apple Podcasts, and Spotify.
0:37:21 Follow us on X and A16Z, and subscribe to our Substack at a16z.substack.com.
0:37:24 Thanks again for listening, and I’ll see you in the next episode.
0:37:39 As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.
0:37:44 Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.
0:37:51 For more details, including a link to our investments, please see A16Z.com forward slash disclosures.
Originally published on the a16z Infra podcast. We’re resurfacing it here for our main feed audience.
AI coding is already actively changing how software gets built.
a16z Infra Partners Yoko Li and Guido Appenzeller break down how “agents with environments” are changing the dev loop; why repos and PRs may need new abstractions; and where ROI is showing up first. We also cover token economics for engineering teams, the emerging agent toolbox, and founder opportunities when you treat agents as users, not just tools.
Â
Resources:
Follow Yoko on X: https://x.com/stuffyokodraws
Follow Guido on X: https://x.com/appenz
Â
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://x.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Leave a Reply
You must be logged in to post a comment.