The $700 Billion AI Productivity Problem No One’s Talking About

0:00:04 85% of the companies we talked to said they really believe they only have the next 18 months
0:00:07 to either become a leader or fall behind.
0:00:09 You know, we have our little group chat where we have another friend who’s like,
0:00:11 oh, all this stuff is overhyped and it’s going to zero.
0:00:12 Totally wrong.
0:00:13 Every time I use AI, it’s amazing.
0:00:17 There’s somebody at every big company who has figured out,
0:00:20 I could do something in one minute that used to take eight hours.
0:00:23 28-year-old guy who was using ChatGPT really, really well,
0:00:26 and they had him create a 30-slide deck,
0:00:29 and they did a global call for everyone in the investment bank
0:00:32 for this guy to spend an hour walking people through how to use ChatGPT.
0:00:34 But that’s absurd.
0:00:37 That’s an absurd way to hope people adopt world-changing technology.
0:00:39 Cursor has taken mediocre engineers and made them good,
0:00:42 but it’s taking amazing engineers and made them gods.
0:00:44 Every board meeting, I go in for my other four metrics,
0:00:47 I have some report of how are we doing against those reports.
0:00:49 And on AI, all I have is the amount of stuff we bought.
0:00:53 When a measure becomes a target, it is no longer accurate as a measure.
0:00:56 Even though we thought we had our quota set and we thought everyone was productive,
0:01:00 it turned out we thought we were productive, and actually it turned out we could be much more productive.
0:01:02 But compared to what?
0:01:06 Companies are spending $700 billion on AI this year.
0:01:08 Most know there’s waste, but don’t know how much.
0:01:13 And with AI budgets continuing to grow, this is no longer any old measurement problem.
0:01:15 It’s the measurement problem.
0:01:19 The one that will determine whether AI becomes the productivity revolution we are promised
0:01:22 or the most expensive placebo in corporate history.
0:01:24 Russ Frayden saw this movie once.
0:01:28 He was the first employee at the first online ad network in 1996,
0:01:33 when companies were pouring money into digital advertising with no clue if it worked.
0:01:36 The industry didn’t take off because the ads got better.
0:01:39 It took off because companies like Comscore built a boring infrastructure
0:01:41 to prove the ads worked at all.
0:01:44 Now he’s building Larradin to do the same thing for AI.
0:01:46 Not to sell you more AI tools,
0:01:49 to tell you if the ones you bought actually do anything.
0:01:50 The stakes are higher this time.
0:01:52 Ad budgets were millions.
0:01:53 AI budgets are billions.
0:01:55 And unlike banner ads,
0:01:59 AI is supposed to replace how your entire workforce operates.
0:02:06 In this episode, Russ and A16Z general partner Alex Rampel dig into the paradox at the heart of enterprise AI.
0:02:09 Everyone’s racing to adopt it, terrified of falling behind.
0:02:12 But almost no one can answer the most basic question.
0:02:13 Did it work?
0:02:18 I’m excited to be here with my friend Russ Frayden.
0:02:19 Yeah, good to see you.
0:02:21 I’ve known you for a long time.
0:02:21 That’s right.
0:02:24 And I think when I first met you, I still actually remember meeting you the first time.
0:02:26 It was from, I think, Josh McFarland.
0:02:29 And Josh was at Google and he was like, yeah, there’s this guy, Russ Frayden.
0:02:33 He started this company, Adify, and he sold it to Cox for all this money.
0:02:35 Back then, $300 million was a lot of money.
0:02:36 I know, it is amazing.
0:02:37 That was like a B route.
0:02:39 But like back then, like that was a huge acquisition.
0:02:44 And there was like, oh, like Russ, like I had, like amazing person that pulled this off.
0:02:47 And I think we met in Florida.
0:02:48 That’s right.
0:02:49 On a Silicon Valley bank trip.
0:02:52 And kind of everything comes full circle in the end.
0:02:55 But AI is probably the hottest thing in the history of the world.
0:02:59 But you also worked in what was the hottest thing in the history of the world in Web 1.0.
0:03:01 But now there’s this big question.
0:03:02 Actually, it reminds me of ad tech.
0:03:03 I think it’s a nice little segue.
0:03:06 It was like ad tech, you’re trying to figure out, does the advertising work?
0:03:09 A lot of ad tech is, here’s an advertisement.
0:03:11 And there’s this attribution problem.
0:03:12 Yep.
0:03:14 Of the sale happened.
0:03:16 Who is responsible for that sale?
0:03:18 Was it the banner ad on Yahoo?
0:03:21 Was it the last click that happened on Google?
0:03:24 Was it the coupon site that stuffed a cookie on a machine?
0:03:27 So part of ad tech is, like, I’m buying ads.
0:03:27 That’s part of it.
0:03:29 But part of it is also, did it work?
0:03:35 And AI, there’s all sorts of stuff around making AI work, which is, like, technically very, very challenging.
0:03:39 But then there’s the question of, did it actually yield a benefit?
0:03:39 Yep.
0:03:44 Which is probably the biggest question for, I mean, there’s a lot of, like, myths on this on both sides.
0:03:49 But we’d love to kind of hear about the origins of Laredin and how you think about it, even some similarities between the two.
0:03:50 Sure.
0:03:57 Yeah, there’s a lot of parallels, really, to what happened in the 90s with advertising and the growth of the Internet and what we’re seeing with AI.
0:03:59 I mean, forget the capital markets perspective.
0:04:06 It is funny to think about what is defined as big from an exit these days versus five years ago, 10 years ago, 20 years ago.
0:04:08 That’s kind of its own topic.
0:04:15 But just when I moved out here, I moved out to Silicon Valley in 1996, and I was the first guy at the first online ad network.
0:04:17 And in the early days, it was just there are websites.
0:04:19 We should put ads on them.
0:04:20 Great.
0:04:21 How do we do that at scale?
0:04:22 Great.
0:04:23 What are the metrics we should capture?
0:04:29 Then you saw the growth of things like Comscore and Nielsen as they moved into television to figure out, like, how do I actually plan this?
0:04:30 How do I spend this?
0:04:31 How do I give tools?
0:04:31 Right.
0:04:38 All of the money lived in TV or in radio, and there were these tools like Nielsen, Arbitron, IMS Health on the pharmaceutical side.
0:04:42 There were all these tools to help people understand what they were getting when they advertised on television.
0:04:44 You had to build that entire stack for the Internet.
0:04:53 You had companies like DoubleClick or Flycast, where I was, or companies like Omniture building a different part of the stack, companies like Comscore building a different part of the stack.
0:04:57 And those companies, obviously, Google and Facebook are two of the most amazing companies ever built.
0:05:01 But if it wasn’t for all of that infrastructure, their revenue just wouldn’t have grown as quickly.
0:05:04 And I really do think we’ll see the same thing in AI now.
0:05:06 The technology is unbelievable.
0:05:36 And my core thesis when I was thinking about starting Laredon after having been, you know, first guy at the first online ad network and having been maybe the first one of the first two executives at Comscore way, way 25 years ago back in the day, my partner Jim and I sat down and we said, look, every time there’s a tremendous shift in budget, and especially when it happens at a great pace, like what happened from TV to digital advertising, what’s happened in a lot of categories from client server to cloud.
0:05:40 Any time that happens, people need to rebuild all of the infrastructure.
0:05:48 There’s a great opportunity to build all of these tools around measurement, around governance, not with the goal of stopping anything, frankly, with the goal of accelerating it.
0:05:53 Because if I am a large company, yes, I’m going to experiment a ton with AI today.
0:05:58 It’s the most exciting thing that’s happened in the last 20 years from the technology standpoint.
0:05:59 It’s amazing.
0:05:59 It’s wonderful.
0:06:03 But also, there are very boring but important questions.
0:06:05 I have 35,000 people in my workforce.
0:06:09 They can’t all get retrained all at once with perfect knowledge and perfect security.
0:06:12 How does it affect my D&O insurance?
0:06:14 Was the project ultimately valuable?
0:06:23 And so, we really wanted to start a company about how would you build kind of the measurement and governance set of tools, not to be a gatekeeper, but to empower more of this spending.
0:06:27 I think as we grow, we will be the best friend to all of the AI companies.
0:06:35 Yeah, and maybe we can get into how you’re doing this, but just to kind of level set a little bit, and I love this framing that you gave me.
0:06:35 I’ve stolen it.
0:06:39 When I steal a phrase, it’s the most sincere form of flattery, of course.
0:06:42 But I just released a little video about how software is eating labor.
0:06:43 So, you know, software eats the world.
0:06:46 This was a thesis that our firm is founded on, but it’s eating labor.
0:06:48 But it doesn’t actually mean that, like, jobs are going to go away.
0:06:49 For sure.
0:06:56 Largely, what it means is that people are going to be, like, 10 times more productive where I can’t hire anybody to do this job, but I can hire AI to do it.
0:07:02 So, you have companies where their software budget is very, very small, but their labor budget is enormous.
0:07:10 And step one of the mega opportunity that excites us as a firm is that people say, oh, I’m going to start hiring software.
0:07:13 But now that means that your software budget is enormous.
0:07:25 Because right now, if you have, like, a $10 billion labor budget and, like, a $1 software budget, you’re not going to try to cut, you know, optimize your $1 software budget, but you’re really going to say, okay, do I need to hire more people?
0:07:26 Can I make people more productive?
0:07:29 You know, all these things that are going through people’s minds right now.
0:07:33 And this is yielding a lot of the mega growth curves of the AI software companies.
0:07:36 But now, this chart is going to be a little bit more balanced.
0:07:36 Sure.
0:07:42 Of, like, the $10 billion of labor, you know, maybe that goes to $8, and now you spend $1 billion on software.
0:07:48 So, like, the net spending for the company is actually lower, the company is more profitable, productivity gains galore.
0:07:51 But then, is this productive?
0:07:53 Like, I always want to know if the humans are productive.
0:07:57 But then, is the software yielding me more productivity, and how do I measure that?
0:08:04 So, everybody’s excited about this gold rush, I’m going to use these tools, but do they work, and how well do they work, and what’s the baseline?
0:08:04 Yes.
0:08:14 So, I’ve stolen your framing of it’s like, if Chase spends $18 billion on software or whatever, and now they’ve doubled that, they need to know if they’re getting their money’s worth.
0:08:17 They need to figure out if this is actually efficient spend.
0:08:17 Yes.
0:08:29 Look, a thing you will hear said frequently by the people running the largest AI companies in the world, the people running the largest firms investing in AI in the world, is they’ll say something along the lines of,
0:08:35 Today, global IT spend is $1 trillion, and we think because of AI and agents, that could go to $10 trillion.
0:08:37 And let’s ignore whether that’s true or false.
0:08:43 It’s certainly the bull case for NVIDIA, for OpenAI, for all of the other things that we spend all of our time doing.
0:08:54 And so, when you think about it, I think we said, I think, if I remember correctly, JPMorgan Chase’s global IT spend is on the order of $18 or $19 billion, and they spend a couple hundred billion dollars a year on people.
0:09:00 So, if you really think about that, well, is their IT spend going to go from $18 billion to $180 billion?
0:09:04 Seems unlikely in the next couple months, but it’s certainly going to go up.
0:09:08 And if it’s going to go up, what does the CFO need to understand?
0:09:18 And at the same time, because of the pace, a way I like to frame this that I think everyone knows, but I think it’s important to say out loud is, yes, there have been tons of shifts, right?
0:09:20 We’ve shifted society a million times.
0:09:22 We’ve shifted from farms to cities, right?
0:09:35 We all know all of these examples, but we’ve never had a time where we’ve expected the entire global workforce of knowledge workers to be retrained immediately on a new set of tools that didn’t exist six months ago, right?
0:09:40 And so, there is an element where everyone needs to figure this out as we go along.
0:09:42 So, what did we start with as a company, right?
0:09:48 Our first set of tools is just, what do you have in your company, and are people flat out using it?
0:09:49 You’ve spent all of this money.
0:09:51 Are people using it?
0:09:59 And what you find is 80-something percent of our customers find far more tools being used by their employees than they know about and they’ve licensed.
0:10:01 That doesn’t mean it was bad, by the way.
0:10:04 Some of those tools are dangerous, and they should worry about that.
0:10:08 Some of those tools might be very popular, and they need to bring them into the fold and understand what’s happening.
0:10:17 But from an IT standpoint, you normally don’t allow software to just be used across your organization with access to your organization’s data and have no idea what’s happening.
0:10:19 We’re letting that happen in AI all the time.
0:10:21 And I don’t really say that to our customers as a fear cell.
0:10:22 It’s to be expected.
0:10:23 Things are moving quickly.
0:10:24 You have to know what’s going on.
0:10:26 So, we start with the baseline of just flat out what’s happening.
0:10:36 The second set of things we try and solve is how do we get people using this stuff more in a productive way on the AI side, on the agent side?
0:10:38 How do we get people using this in their workflow?
0:10:43 I’m a marketer working at General Mills or something like that, right?
0:10:45 I’m a marketer working at General Mills.
0:10:48 How is General Mills going to help me use these tools?
0:10:56 And what I’ve generally found with employees, if you really want to drive employee usage of tools, you have to make them feel safe so they won’t look dumb.
0:11:00 And you have to make them understand that they can use this safely without getting fired.
0:11:07 Because, again, it’s one thing if you’re 22 years old and you’ve been using these tools effectively your entire life since high school.
0:11:20 But if you’re a 42-year-old person who’s been, you know, had a 20-something-year career and you’re working in your job every day, and by the way, you also have things you do at home and you have business travel.
0:11:21 You have all of these things you have to do.
0:11:22 Also, you have to become an AI expert.
0:11:24 You really would like to not look dumb.
0:11:27 And you’d like to accidentally not upload the wrong data and get yourself fired.
0:11:33 This is actually a bigger issue in some countries where there’s a bunch of EU regulations around AI that do matter.
0:11:36 And if I’m an employee at a company, I don’t want to look dumb.
0:11:38 So if I’m a CFO, we bought all these tools.
0:11:40 What did we actually buy, number one?
0:11:43 Number two, how do we get people actually using these tools?
0:11:50 Because the usage on these tools in the enterprise is less than people would think today, which makes sense, by the way.
0:11:52 I’m going to get to the productivity thing in a second.
0:12:01 But, you know, anyone listening to this, if you’ve ever been a part of any software rollout at any enterprise ever, a very boring but very important question is how do we drive actual usage?
0:12:03 And sure, everybody uses email.
0:12:06 People use Workday because if you don’t use Workday, you’re going to get fired, right?
0:12:07 You’re not going to get your paycheck.
0:12:17 But most enterprise software, your intranet software from SharePoint, things like that, are used by a relatively small set of the population that you wish were using it.
0:12:23 And so if the goal is to get people more productive using AI tools, you want to drive actual employee engagement.
0:12:25 So we built a suite of tools around that.
0:12:30 And then you have to get into productivity, which is did this get people actually more productive?
0:12:32 Is my organization actually more productive?
0:12:37 So today, I know where I want to go with Larratt.
0:12:45 And what I like to think about today is what we’re doing today on the productivity side is not as far as I’d like it to go, but it’s certainly better than anything that exists in the market.
0:12:53 So what we’re doing today is we’re marrying the behavioral data that no one else has, which is, is Alex a heavy user of ChatGPT or not?
0:12:54 Just flat out.
0:13:02 We’re not doing it at the individual level, but we’ll use that example for the podcast because we have to worry about the employee privacy concerns that companies have for their own employees.
0:13:14 But at the end of the day, I want to understand, did my users in the legal department that were using this expensive legal tool I bought, are they more productive than my users in the legal department that are not?
0:13:17 Because what I’ve definitely done is I’ve driven up my OPEX.
0:13:20 I bought this software, I’ve driven up my OPEX, but are they more productive?
0:13:24 Are my marketers that are using Claude or ChatGPT actually more productive?
0:13:25 And how do you measure that?
0:13:35 So today we do it the only way productivity research has ever existed so far, which is we take the normal productivity survey market research that people have done for 50 years.
0:13:37 Not ideal, but it is the gold standard.
0:13:39 It’s McKinsey, it’s Towers, Watson, it’s Accenture.
0:13:44 And we lay on top of it proprietary data that other folks don’t have, which is actual usage.
0:13:52 So the way I think of it is the worst way to measure productivity is I’m going to send a survey to my employees and say, do you feel more productive today from using ChatGPT?
0:13:54 First of all, there’s a definition issue.
0:13:57 Second of all, people are going to answer the way you hope they’ll answer.
0:13:59 But third, you have no idea if they’re actually using the tools.
0:14:02 So a better way to do that, I learned this years ago at Comscore.
0:14:07 One of the things, one of the many things I did at Comscore is I ran our survey market research group.
0:14:12 And one of the reasons the Comscore surveys were great is we had the behavioral data married with the actual survey responses.
0:14:13 We’re doing the same thing here.
0:14:18 Where I ultimately would like to get to is full passive measurement on productivity.
0:14:23 The truth with that is for enterprises, that’s going to require a level of additional data sharing that we’re not getting yet.
0:14:26 And yet from customers, we will eventually get there.
0:14:31 But to kind of like put a finer point on this, so I’m a lawyer.
0:14:34 I work at, you know, at some big company.
0:14:42 You know, productivity to a certain extent, like if I only have to work four hours a day versus eight hours a day, like that’s great for me.
0:14:46 Like I kind of feel like it’s a win because I often think about like the principal agent problem, right?
0:14:50 So everybody is an agent.
0:14:53 And then there’s the ethereal being of the corporation, which is the principal.
0:14:59 And it’s like, you know, yeah, I guess if I own stock in my corporation, I want it to be more profitable.
0:15:03 But really, I want to work as little as possible and get paid as much as possible.
0:15:06 Like that’s kind of every individual agent’s job.
0:15:08 And then you have these tools.
0:15:12 So like theoretically, it’s like everybody’s going to adopt these things if they get to be lazier.
0:15:13 Yep.
0:15:14 Like everybody wants to be lazier.
0:15:14 Sure.
0:15:15 Right?
0:15:16 They want to be lazier and richer.
0:15:19 I feel like these are like the universal like kind of human conditions.
0:15:22 There’s some small set that want promotion, but I agree for 90%.
0:15:22 That’s richer.
0:15:23 Yeah, that’s true.
0:15:24 That’s true.
0:15:27 So like if you can get the promotion by doing less work, I’m sure people would opt for that.
0:15:35 But I guess like how, like think about everybody will use these tools or like I think I was telling you the sad story because our kids go to the same school.
0:15:38 It’s like younger kid gets busted cheating, right?
0:15:39 With chat GPT.
0:15:41 Like clearly productivity gained for him.
0:15:41 Right.
0:15:42 Right?
0:15:47 Because it allowed him to be lazier and, you know, richer with his video game time until we confiscated his phone.
0:15:51 But also a set of rules that you can get in trouble for.
0:15:53 But like take that example.
0:16:01 Like so you can imagine the individual agent, you know, the human being, the lawyer in this example is benefiting, you know, but then does the company benefit?
0:16:03 Because to a certain extent, like I’m paying you the same amount of money.
0:16:06 I want you to work for eight hours a day.
0:16:19 So actually my expectation should be that if you’re whatever, I don’t know what the lawyer does, but like drafting legal drafts, like if you can now do it in four hours versus eight, you know, and spend four hours, you know, playing golf, like you’re thrilled.
0:16:20 You got a productivity gain.
0:16:22 The company didn’t actually benefit.
0:16:32 So what you kind of want is you want both parties to benefit, which is always tough because sometimes it’s very, very hard to sell products to people that eliminate their jobs.
0:16:34 That’s probably the hardest product to sell.
0:16:41 But like, I mean, maybe kind of taking this example and riffing on it, like now I can do in eight hours, like in four hours, I can do what used to take me eight.
0:16:42 Sure.
0:16:49 But how does the company, a company’s like, oh, wow, you’re still, you’re operating at your baseline, but actually you should be able to do twice as much with this tool.
0:16:51 So I guess like, how do you define the baseline?
0:16:53 How do you address that problem?
0:16:55 How do you think, am I framing it the right way?
0:16:56 I think you’re, I think you’re framing it.
0:17:00 I think you’re certainly framing it all right way for certain size of companies, right?
0:17:09 We all know for Silicon Valley, what you’re going to have just because of the competition and the equity form of compensation, what you’ll have is if I can get done in four hours, what I could have done in eight.
0:17:12 I’m just going to work four more hours and then another four.
0:17:13 And that, that’s very different.
0:17:22 That, that’s, there’s some subset of workers at all size companies, probably a larger percent in Silicon Valley, but a smaller percent in, at GE, right?
0:17:27 There are people at GE who want to one day become the CEO of GE and those people will work as much as they possibly can.
0:17:29 So there’s some subset of workers there.
0:17:34 For the rest, look, there’s an interesting question about how is management going to evolve overall, right?
0:17:39 I think behind all this, the first question is, the first question we’re trying to solve is, like I said, do people use these?
0:17:46 And from a corporation standpoint, for our measures of productivity, which is, we’re defining it with each of our customers, right?
0:17:52 For our measures of productivity as we ping folks, is there a difference in productivity between the heavy users and the lighter users?
0:17:56 What we want to measure with that, we want, we’re not doing this today.
0:18:00 What we want to measure with that is then some concept of raw tonnage of work, right?
0:18:07 The ultimate, there’s this lingua franca when we talk about kind of employees of FTE, right?
0:18:12 And we all know that you work different than I work and then, you know, various people work.
0:18:13 And we all know that.
0:18:16 Yet, if I’m the CFO of, I don’t know, let’s pick on JP Morgan again.
0:18:24 If I’m the CFO of JP Morgan, I have a fundamental, you know, horse sense for what do 1,000 FTE do versus 500 FTE versus 2,000 FTE.
0:18:27 And AI is going to break all of that, for sure.
0:18:37 And so our main goal today is just to build the baseline for our customers, which is, at the end of the day, are the people using these tools fundamentally more productive than the folks that aren’t?
0:18:41 Layer on top of that tonnage of amount of time worked.
0:18:43 You can get a pretty good – it’s never perfect.
0:18:44 People are on vacation.
0:18:46 You have to measure this as groups, right?
0:18:52 Any given person was out sick one day or was on a flight one day or was at a training one day that it’s impossible to measure.
0:18:56 And you – it seems from a system standpoint they weren’t working.
0:18:56 They actually were working.
0:18:57 They were doing a training, right?
0:18:59 So think of this as at the aggregate data.
0:19:00 It’s never useful.
0:19:03 None of this data is ever useful at the Russ Frayden level.
0:19:07 I mean to get existential, was I productive yesterday, is unknowable.
0:19:08 I can’t know if I was productive yesterday.
0:19:09 I think you were.
0:19:11 I was all for it.
0:19:24 But what we’re trying to do at the systems level for companies is understand, is there some correlation between specific use of these tools on advanced side, light side, heavy side, heavy user of the tool, lighter use of the tool?
0:19:28 Were the users more productive in their job?
0:19:29 Were the employees more productive in their job?
0:19:34 And then measure on top of that amount of time those segments of workers were actually working.
0:19:41 Because the goal, if I’m a CFO today, is not to understand, did Ben do a good job and did Tina do a good job?
0:19:47 The goal is to understand, I have definitely been asked to spend 50% more on OPEX.
0:19:47 Right.
0:19:50 Did I drive something?
0:19:51 Right.
0:19:53 And then, by the way, there are interesting questions.
0:19:58 We know this around staffing size and will companies get more done because people will actually work eight hours?
0:19:59 Yeah.
0:20:07 Look, it will turn out – I suspect it is true that – this is one of the things managers do.
0:20:14 I suspect it is true over time if it becomes clear that all of your employees are now working four hours a day instead of eight.
0:20:20 You will probably decide to have fewer employees and they will work – the remaining employees will work six hours a day.
0:20:28 So I’m not sure I really buy in the next couple of years you will see people in large companies actually just working half as much.
0:20:30 You know, sole proprietors is what it is.
0:20:37 If I were a sole proprietor lawyer, kind of my only measure of productivity today is to myself anyway.
0:20:37 Right.
0:20:40 It’s how hard do I want to work and how much money do I want to work.
0:20:41 Well, the principle is the agent.
0:20:41 Right.
0:20:42 So that’s fine.
0:20:46 This is why it’s like – but this is why it’s so important and this is why, I mean, candidly, I love what you do.
0:20:47 Sure.
0:20:47 Obviously, I love what you do.
0:20:48 That’s why you’re here.
0:20:52 But you – there is no baseline.
0:20:53 Like it’s like did this work?
0:20:57 Like, well, first I have to know how like you have to define the outputs.
0:21:00 You have the inputs, which are largely just like time and money.
0:21:00 Right.
0:21:02 And then you have the outputs.
0:21:06 And part of it is actually it is kind of complicated to come up with an output.
0:21:06 For sure.
0:21:08 Do you know of Goodhart’s Law?
0:21:09 Go ahead.
0:21:11 So Goodhart’s Law, I love this one.
0:21:18 It’s like when a target becomes a measure – sorry, when a measure becomes a target, it is no longer accurate as a measure.
0:21:18 Yes.
0:21:19 Right?
0:21:23 So if I say, okay, I’m going to judge you based on – I’m going to like how many emails are sent every day.
0:21:24 Well, that’s a measure.
0:21:28 But once it becomes a target, it’s like I want you to send more emails.
0:21:31 Well, you’re no longer – like the measurement gets corrupted.
0:21:37 Because now people decide to do more things to hit this target, and it’s no longer an objective measure.
0:21:42 So, you know, part of it is if I’m trying to figure out like, okay, there’s a product called Harvey.
0:21:46 A lot of people love Harvey, and it seems to make people a lot more productive.
0:21:48 But compared to what?
0:21:49 Right.
0:21:52 And so to me, the only way you answer that – we’ll talk about Harvey.
0:21:53 Nothing against Harvey.
0:21:54 I’m sure Harvey is amazing.
0:22:00 To me, the only way to really understand this – and that’s why I think the traditional way companies are doing this just doesn’t work at all,
0:22:03 which is, hey, let’s survey the people that use Harvey and ask them if they were productive.
0:22:07 And by the way, they will all say yes because no one ever answers they weren’t, number one.
0:22:10 And number two, my boss paid for the product.
0:22:12 I’m going to say it was a good product, right?
0:22:16 Unless we all universally hate it, which I assume is not true of Harvey because everyone seems to like Harvey.
0:22:16 So that’s wonderful.
0:22:21 So I think all you can actually do – it’s why I think the traditional way of measuring this is broken.
0:22:23 Look, it’s why we started learning.
0:22:31 All you can really do is understand without asking people how much usage of Harvey are these people actually doing, right?
0:22:33 We have five people.
0:22:34 We have six people.
0:22:35 Whatever they’d say on a survey.
0:22:38 Two have never logged in, right?
0:22:40 We’ve all seen the joke about, you know, your project is due.
0:22:41 It’s due in an hour.
0:22:42 You said you were caught up.
0:22:45 And then you – oh, shit, I have to ask permission for this Google Doc, right?
0:22:52 So there’s two of the six – I’m making these numbers up, of course – two of the six people actually signed up for Harvey the day they were told and then never went back to it all.
0:22:54 They’re very happy with the way they work.
0:22:55 They work that way all day, every day.
0:22:59 Two of the six log in and use a little bit, and two of the six use it all the time.
0:23:08 The only way I can even begin to understand if that software is valuable is by knowing that data passively without asking those folks a question.
0:23:14 And then asking everyone the same questions about productivity and measure it with amount of work actually output.
0:23:20 And if I take those three things together, then I can begin to form an understanding of was Harvey useful?
0:23:31 You and I had a discussion with someone where they were talking about one of the ways they incent their engineers at their company is they have a leaderboard of the amount of money each engineer spends on cloud code.
0:23:37 And the founder was talking about how he went to one of his best engineers and said, I don’t understand what’s happening.
0:23:38 You’re one of our best engineers.
0:23:40 Why aren’t you spending any money with cursor?
0:23:42 I’m sorry, not cloud code.
0:23:43 Why aren’t you spending any money with cursor?
0:23:45 I really don’t get what’s going on.
0:23:50 And so that was an example of for these companies where they’re very developer heavy.
0:23:50 Right.
0:23:51 You probably don’t need us.
0:24:03 If you’re a very developer heavy company, probably measuring amount of money spent on cursor plus your kind of normal management understanding of is this person actually working?
0:24:06 If they come in for two hours a day, you may be happy with that.
0:24:06 You may not.
0:24:09 That’s going to be company specific and lifestyle specific and culture specific.
0:24:11 But you’re in the office.
0:24:12 I see you’re there.
0:24:14 You’re not spending any money on cursor.
0:24:15 What’s up?
0:24:15 Right.
0:24:16 We have these metrics.
0:24:23 The issue, though, is we see this explosion of there’s hundreds of AI tools and companies have hundreds of roles.
0:24:31 And so that’s why we want to try and replace, you know, the McKinsey Corporate Health Index or the Tows Watson or the Accenture Service with some real useful data around AI.
0:24:39 But I think that kind of cursor example really crystallized in my mind what you’d want to be able to do for a whole company, which is how much did this person work?
0:24:42 So I have that quality, you know, I have that quantitative judgment.
0:24:45 Qualitatively as a manager, right, we’re not replacing this.
0:24:47 Did they do a good job?
0:24:49 And then fundamentally, did they use the tools?
0:24:51 And when you take those three things together, that’s the only way you’re going to have measurement.
0:25:05 And like I said, when you think about my, you know, micro world of if you really think JP Morgan is going to go from spending $18 billion in IT to $30 billion or $40 billion, the CFO is not just going to say no problem, right?
0:25:07 Today, our customer is the CIO.
0:25:10 I think over time, our customer becomes a partnership with the CIO and the CFO.
0:25:12 The numbers are just big.
0:25:13 It’s like cloud spend.
0:25:17 The numbers are just so big, people are going to pay attention.
0:25:18 It’s going way beyond experimental.
0:25:31 And obviously, the companies themselves, like if you ask any company that is trying to sell you anything and you ask them, does your product work, they will probably 99 times out of 99 say like, of course it does, right?
0:25:32 That’s like, it’s the best.
0:25:32 It’s the best.
0:25:34 You need to have an independent arbiter.
0:25:37 And that’s where you guys come in.
0:25:43 But kind of like double clicking on this point before, it’s almost like reinforcement learning at a company wide level, right?
0:25:45 Of what is the outcome that I’m looking for?
0:25:47 And sometimes it’s clear, right?
0:25:54 So, and this is where the measurement and target thing is also relevant because it’s like, I want you to write more lines of code.
0:25:57 Whoa, if that’s, there’s a measurement of like how many lines of code were written.
0:26:02 But if it becomes the target, then you’re just like writing gobbledygook code and like you’re, for sales, it’s very easy.
0:26:04 I want you to sell more stuff.
0:26:10 But there’s a lot of latency between like you go talk to a customer and then you go collect money.
0:26:12 So, you might have targets in between.
0:26:13 You might have measurements in between.
0:26:15 If you’re a lawyer, draft more contracts.
0:26:19 So, I guess, how do you try to define the goals?
0:26:23 Because some of them are just like, it’s kind of, I think of it as like background information that’s going through.
0:26:25 It’s like emails that are being sent.
0:26:25 Right.
0:26:30 Or, you know, Slacks that were sent or Google Docs that were edited.
0:26:33 Like there are these key, like there are these very, very clear measurements.
0:26:35 But those aren’t necessarily outputs.
0:26:48 So, first to your point on measurement, this is why I said earlier, if you think about any time true third-party measurement exists, there’s this interesting dynamic.
0:26:50 And we saw this at Comscore, but everyone has seen this.
0:26:53 Anytime they tried to build a third-party measurement company, Omniture saw this in the early days.
0:26:57 Google at some point fought it and then actually bought Urchin and built Google Analytics, right?
0:27:02 Because it turned out it’s actually good when your customers can track value if what you do is actually valuable.
0:27:09 And so, my general perspective is I think today a lot of the AI companies probably look askance at us.
0:27:14 But I think over time, certainly the AI tools that actually provide value are going to love us, right?
0:27:21 The way you will ultimately unlock real enterprise budget is because people believe these tools are actually valuable.
0:27:24 So, what we do today, and this is a journey, right?
0:27:25 The company is about a year old.
0:27:32 So, what we do today is we work with all of our customers and say, look, here are the baseline productivity questions that are gold standard that people have asked for 70.
0:27:36 You know, there’s pros and cons to them, but this is – you have to start somewhere.
0:27:37 This is where we start.
0:27:41 And let’s define a set of metrics for each of your departments.
0:27:49 One of the things we’ve found that actually seems to matter, not as a metric that companies share with their employees because then you have the good hearts a lot problem,
0:27:54 but as an actual reality on the ground is fundamental responsiveness.
0:28:02 There is an element of I spend some amount of money on my legal department and I am happy with the amount of productivity they do today.
0:28:10 So, there is an element of unless I’m trying to fire lawyers, which I’m not, you can argue how would I measure the value of software?
0:28:13 I guess my lawyers might be happier, but I don’t have a churn problem there.
0:28:15 So, frankly, why should I do this?
0:28:25 And so, what we found is one of the things we found is just almost an interdepartmental SLA, which is it turns out if I roll out these tools and I’m not firing employees,
0:28:29 because one way to look at this is could I fire half my lawyers, it turns out companies don’t really like firing people.
0:28:35 Companies do fire people if they have to, but I’ve actually never met a CFO that got excited about firing 30% of the workforce.
0:28:38 Outside of call centers, that’s a different issue we could talk about.
0:28:41 Like, companies treat their call center employees different from the rest of their employees.
0:28:47 But outside of call centers, I’ve never met a CFO who is, if you went to a CFO and said, you can fire half your FP&A people.
0:28:49 He doesn’t want to fire Tina.
0:28:49 He knows Tina.
0:28:51 He’s met Tina’s husband and children.
0:28:52 He doesn’t want to fire Tina.
0:28:55 He’d like Tina to be happier and more productive.
0:28:57 And actually, he’d like her to do a great job and never quit, right?
0:28:58 Companies don’t really like churn.
0:29:08 So, one of the metrics we found that people seem quite excited about is just did this raise or lower the interdepartmental responsiveness?
0:29:13 So, our measure would be am I now comfortable sending more things to legal, right?
0:29:17 If I’m going to keep my legal department the same size, I’m not going to start suing more people.
0:29:20 We’re talking about companies here, not law firms, where there’s a different measure of productivity, right?
0:29:21 There are cost centers, not profit centers.
0:29:30 So, one thing to do it is did over time, because my lawyers are now more productive, are other departments asking them more questions?
0:29:31 Are they getting their responses faster?
0:29:37 When I’m in product and I’m asking for input from engineers, are they responding more quickly, right?
0:29:42 That is a good way for me to see behaviorally we become more productive.
0:29:43 That’s not lines of code.
0:29:47 Now, by the way, I agree if you expose the metric and say, hey, you better be responsive.
0:29:47 People can lie.
0:29:49 They can send Slack messages back and forth.
0:29:55 But what I’d really like to understand is, as a map, which of my departments use these tools more?
0:29:58 And do they become more responsive to my other departments?
0:30:00 Because there’s an element of when you’re at a big company, people know this.
0:30:03 It’s one of the reasons small companies do so well in innovation.
0:30:07 There’s just a giant coordination problem for all of these companies, and we know this.
0:30:10 And, you know, in Silicon Valley, it’s fun to make fun of these companies.
0:30:14 But actually, every entrepreneur’s secret dream is to become so large that they have a giant bureaucratic company.
0:30:18 Of course, Google did not plan to have a giant bureaucracy 30 years ago.
0:30:20 They just became so successful, they now do have a giant bureaucracy.
0:30:27 And, you know, that’s kind of a good segue into perhaps the state of an AI enterprise, right?
0:30:29 So you went on this whole, like, you talked about 350 people?
0:30:30 Yeah.
0:30:34 We interviewed 350 heads of IT at major companies.
0:30:37 And across the whole gap, right?
0:30:37 Yeah.
0:30:41 It wasn’t just, you know, kind of Silicon Valley companies that…
0:30:49 Not, I mean, in all honesty, my whole career, basically, I spent a couple years helping my friend at Carbon, and I spent a year trying to fix wine.com.
0:30:53 But other than that, my whole career has been selling software to large companies.
0:30:56 Mostly large companies are older companies.
0:30:56 Right.
0:30:58 Yes, there’s the occasional Silicon Valley company that grows very quickly.
0:31:05 But if you’re in the Fortune 500, you are going to be 20-plus years old 99% of the time, right?
0:31:10 And so if you’re going to sell to someone with more than 1,000 employees, they’re almost, by definition, an older company.
0:31:13 So maybe give us the highlights of what you learned.
0:31:13 Sure.
0:31:16 We saw a bunch of different things, and, you know, people have seen this before.
0:31:18 I actually don’t think of this…
0:31:21 You’ll see people turn this into kind of click-baity, fear-mongering things.
0:31:22 I don’t really think of it that way.
0:31:29 So, first of all, we saw, like, we know this from Gartner, there’s like $700 billion being spent in enterprise AI.
0:31:31 It’s growing very, very quickly.
0:31:32 It’s going to keep growing quickly.
0:31:39 And one of the things we found is something like 70% of the leaders we talked to said, we are sure we are wasting money here.
0:31:41 It’s being spent so quickly.
0:31:42 And, by the way, shame on us.
0:31:44 We had no system to measure this in the first place.
0:31:47 I’ll get back to the report in a second, but I was talking to a customer today.
0:31:49 Why did we sign them as a customer?
0:31:53 They’re a very profitable business owned by a PE firm.
0:31:59 And their bosses, their PE owners, gave them five things they had to do this year.
0:32:02 And one of the five was adopt AI across the organization.
0:32:09 And he said, every board meeting I go in for my other four metrics, I have some report of how are we doing against those reports.
0:32:11 And on AI, all I have is the amount of stuff we bought.
0:32:12 It’s not.
0:32:13 So, yes.
0:32:15 Yes, I’m doing great.
0:32:15 But it turns out.
0:32:16 We have a large family of AI.
0:32:17 We adopted all these AI children.
0:32:18 We bought all of it.
0:32:18 It’s all great.
0:32:20 But it turns out we want to actually do it.
0:32:25 And so what we found is these leaders like maybe they are right that 70% of their projects are failing.
0:32:29 Regardless of their right, it’s a giant problem.
0:32:32 They feel that way because they have no system to figure it out in the first place.
0:32:34 No one believes 75% of their ad spend is failing.
0:32:39 It’s not because their ad planners are smarter than their AI buyers.
0:32:50 It’s because there are 20 years of systems in place to help me understand when I buy this ad campaign, when I spend this money, when I do this app install, whatever it is, did it actually drive value for me?
0:32:55 And we just don’t really have that in AI, like I said, outside of some very, very specific verticals.
0:32:58 And so really the biggest thing we found was kind of three things.
0:32:59 One, you saw the AI spend.
0:33:04 Two, something like I said, they believe 70-something percent of AI projects are wasted.
0:33:16 But the other thing we found is basically 80, 85 percent, I can’t remember, 80, 85 percent of the companies we talked to said they really believe they only have the next 18 months to either become a leader or fall behind.
0:33:26 So I think one of the things, one of the reasons you’ve seen this giant unlock and budget is there’s tremendous anxiety at these enterprises going like, we’re going to lose if we don’t adopt this stuff.
0:33:28 Yet, so we’re adopting it quickly.
0:33:30 We have no particular idea if it’s succeeding.
0:33:32 Our employees aren’t really using it.
0:33:40 By the way, a forgotten group in the company for all of this AI, and from my last company, we built a very, very large HR technology company.
0:33:44 We sold into heads of HR, touched all the employees in the company, but we sold into heads of HR.
0:33:53 And so as we’ve talked to a lot of our old customers, who aren’t really our customers today, but they’re influencers, what they will all say at all of these large companies is, hey, our employees are really worried.
0:33:55 It’s not even they’re worried they’re going to lose their job.
0:33:58 There’s a base level of worry about AI and the economy, all that stuff.
0:34:00 It’s not even they’re worried they’re going to lose their job.
0:34:03 It’s just they’re getting told to use a new system all day, every day, right?
0:34:07 Generally, if you work in a large company, there’s one or two new systems initiatives a year.
0:34:09 Now there’s 20 new tools.
0:34:09 They don’t know.
0:34:11 They don’t know what they’re allowed to do, and they have no training.
0:34:15 How do they actually – how do I get people using these tools?
0:34:18 And so you have this weird – you have this weird almost perfect storm.
0:34:20 It’s why we’re excited about Larrant.
0:34:21 It’s why you’re excited about Larrant.
0:34:29 You have this perfect storm of tremendous growth in budget, tremendous anxiety that none of it is working, tremendous anxiety from their employees about what they’re even allowed to do.
0:34:33 And so what we’re trying to do is – I don’t think we solve all of that.
0:34:34 That would be an absurd thing to say.
0:34:39 But I think we really help with all of that about, like, what is your plan to measure this in the first place?
0:34:39 Did anyone use it?
0:34:41 Did they become more productive than they did?
0:34:43 How do you give them the tools to use it more?
0:34:44 Yeah.
0:34:49 Well, and that kind of last point is super interesting as well because it’s like there’s the did it work?
0:34:50 How well did it work?
0:34:51 You know, what are the measurements?
0:34:53 Make sure that the measurements don’t become targets.
0:34:55 It’s like all the stuff that we just talked about.
0:35:04 And then to use the metaphor of my son who cheated on his math homework, there are people that are just like, wow, like, they’re the go-getters in the company.
0:35:08 This is actually why I am convinced that AI is under-hyped.
0:35:08 Yes.
0:35:13 You know, we have our little group chat where we have another friend who’s like, oh, all this stuff is over-hyped and it’s going to zero.
0:35:13 Totally wrong.
0:35:15 Every time I use AI, it’s amazing.
0:35:19 Because you go – it has not diffused.
0:35:25 You have like the 19-year-old kid or, you know, my 13-year-old son is like, wow, normally homework would take me two hours.
0:35:27 Now it takes me one second.
0:35:27 Right.
0:35:29 And obviously that’s bad, right?
0:35:33 Like I’m not using him as the – that’s why we confiscated his iPhone, right?
0:35:38 But there are these productivity unlocks where it’s probably not going to happen top down.
0:35:38 Sure.
0:35:46 It’s like somebody in the company – and sometimes, again, not to like oversimplify human behavior, but it’s like I want to be lazy and I want to be rich.
0:35:46 Yes.
0:35:47 Right?
0:35:49 Like these are the two things that are motivating people.
0:35:55 And like I found this tool that allows me to be lazier and richer that actually helps the company.
0:35:55 Yes.
0:35:57 So not the math – not the cheating, right?
0:36:03 It’s like you’re getting – like I can now – I know that my boss was thinking this would take eight hours.
0:36:05 I’ve figured out a way to do it in five seconds.
0:36:06 And it’s really good, by the way.
0:36:07 It’s amazing.
0:36:08 It’s really good.
0:36:11 And the worst thing that can happen – this is like the inverse of everything that we just talked about.
0:36:14 The worst thing that can happen is that guy keeps it a secret.
0:36:14 Right.
0:36:17 Because like – and you might be afraid.
0:36:19 It’s like, oh, am I allowed to use this?
0:36:33 But what you should do – this is how AI will go from underhyped to like correctly hyped and correctly diffused is it’s like there’s somebody at every big company who has figured out this like I could do something in one minute that used to take eight hours.
0:36:39 We need to like make this person a hero, like memorialize this, like memorialize this and push it out through the entire company.
0:36:41 So like how do you do that?
0:36:47 This was my point on the – what we’re doing on the AI engagement side and that’s a great question.
0:36:49 This is my point on what we’re doing on the AI engagement side.
0:36:53 This is one of these areas where everyone’s interests are aligned.
0:36:58 The employee that’s working very hard loves recognition and, by the way, he’d like his coworkers to come up to speed.
0:37:03 The employee that’s scared wants support and wants training.
0:37:06 And, by the way, companies actually want their employees to be more productive.
0:37:09 I know it’s a fun thing for a subset of people to tweet about.
0:37:15 But like I said, I have yet to find the CEO in all of – I’ve spent 30 years selling things to CEOs.
0:37:19 I have yet to find the CEO who wakes up in the morning and wants to run a smaller company.
0:37:21 He wants more employees and he wants more profit.
0:37:22 He wants more revenue.
0:37:25 But contrary to popular leap, they want more employees.
0:37:26 They like running big companies.
0:37:26 They do.
0:37:33 You can find old interviews of Larry Page talking about his plan for how Google was going to have a million employees one day.
0:37:39 And he was spending a lot of time thinking about self-driving cars to move the cars around the parking lot because this is before remote work.
0:37:42 And literally where were the million employees going to park all the cars?
0:37:44 And I remember I read that 15 years ago.
0:37:46 Like that stuck in the back of my mind.
0:37:49 I have never met a CEO who wants to run a smaller company.
0:37:52 It’s one of the reasons, by the way, totally unrelated point in the capital markets.
0:37:57 One of the reasons if you ever meet a CEO of a conglomerate, they never want to break up the conglomerate because they like running bigger companies.
0:37:59 It’s more fun, right?
0:38:01 I’ve run – I’ve had my companies grow.
0:38:02 They’re fun when they’re bigger.
0:38:03 It is.
0:38:03 It’s super cool.
0:38:13 And so from an employee standpoint, what we built with this Nexus product is effectively a product where we said – I’ll use an anecdote.
0:38:18 I was in the UK in July and I went on a bunch of sales calls.
0:38:28 And I was talking to someone at a bank, very large, very – European bank is among the most regulated folks in the world, at least, you know, fast to adopt new technology for good reasons, honestly.
0:38:34 And they were telling me a story about how they had – it was a 28-year-old guy.
0:38:36 I don’t remember what level that makes you in an investment bank.
0:38:37 So let’s say a director.
0:38:43 But 28-year-old guy who was using ChatGPT really, really well in the investment banking side of the business.
0:38:52 And they had him create a 30-slide deck and they did a global call for everyone in the investment bank for this guy to spend an hour walking people through how to use ChatGPT.
0:38:55 I’m sure that was very cool for him, but that’s absurd.
0:38:58 That’s an absurd way to hope people adopt world-changing technology.
0:39:13 Another absurd thing to do is to go out and buy some LMS course that HR is going to buy that, you know, the secret to a lot of LMS is other than things you must do or you will lose your job, like sexual harassment training, HIPAA training in certain orders, no one does it.
0:39:14 They just don’t go.
0:39:17 And so how do I actually get people using these tools?
0:39:21 And this was my point from earlier is you want to help them, A, not look dumb, and B, no, they won’t get fired.
0:39:29 And so what we effectively did is built these wrappers that exist around the models so we don’t tell people to use Quad or to use Gemini or to use ChatGPT.
0:39:30 Right.
0:39:33 Then the other thing we built because, again, people are also worried about getting fired.
0:39:37 They’re worried about getting fired because of the economy, because of AI, because whatever, it’s a new tool.
0:39:38 I would like to get fired.
0:39:46 By the way, when you’re talking about European banks, there’s a lot of regulation that is a legitimate concern of if our employees do the wrong thing, we will get fined.
0:39:47 Forget whether you fire them.
0:39:49 These companies don’t want to get fined.
0:39:59 And so the other thing we did is we basically trained our own kind of customized llama model to block people from asking questions that are illegal or the company doesn’t want you to.
0:40:01 So we’re not talking about hackers here.
0:40:03 True bad actors in the company is plenty of security solutions.
0:40:12 What we’re really talking about is the X percent of the people who I’m in people ops and HR at a large company, I’m supposed to do a workforce analysis.
0:40:19 Am I allowed to go into ChatGPT and load in our full employee database with race and gender?
0:40:20 I don’t know.
0:40:21 I would like to not get fired.
0:40:23 Maybe I’m allowed to and maybe I’m not.
0:40:28 And by the way, I think it’s incumbent on the company to say to our employees, here is a safe space.
0:40:31 Nothing you can do here is going to get you fired.
0:40:35 So we will – oh, Alex, you’re not allowed to – that has social security data.
0:40:36 Don’t share that.
0:40:42 You’re not allowed to ask that prompt because in Europe we’re not allowed to use HR – we’re not allowed to use AI to write employee reviews.
0:40:44 I don’t know if that’s a good law or a bad law.
0:40:45 I didn’t write the law.
0:41:01 But there are companies that look at the EU AI regulations and say to themselves, our read of the regulation – I’m not going to, you know, prosecute that – our read of the regulation is we believe it’s illegal and we will get fined if our employees use AI tools to do employee reviews.
0:41:07 So great, if I am a European-based company and I want my employees using AI, I have to block them from using it for those use cases.
0:41:16 And so what we’ve tried to build is this almost like harness to say you can be more productive, you’re not going to look dumb, you’re going to be more productive, and you’re not going to make any mistakes that get you fired.
0:41:22 And so what we found is that actually drives more AI usage, surprising literally no one.
0:41:24 And from a company standpoint, what do you want?
0:41:28 A, I want the usage, and B, I want to build up that IP of what really works for my company.
0:41:29 It’s a total unlock.
0:41:30 Same thing on the coding side, right?
0:41:37 You know, Cursor has taken mediocre engineers and made them good, but it’s taking amazing engineers and made them gods, right?
0:41:41 And so our goal should be how do we help people get much more productive with all of this?
0:41:44 How do we help them use Cursor more effectively, Harvey more effectively?
0:41:46 We started with all of the LLMs more effectively.
0:41:53 So maybe we could talk, I mean, this is a little bit philosophical, but future of work.
0:42:03 Because to a certain extent, like, all right, you’re the, like, if you’re the measurement, like the measurement inevitably will become a little bit more of a target.
0:42:11 And I always like to kind of remind people that I think it was 97, 98% of Americans when the Constitution was ratified were farmers.
0:42:11 Right.
0:42:14 And they all lost their jobs due to these pesky things.
0:42:15 And all worked out okay.
0:42:17 Like the tractor and fertilizer and all these things.
0:42:21 And I think the average life expectancy was, like, 35.
0:42:24 And most children died in childbirth or shortly thereafter.
0:42:29 It’s like, you know, things have changed, but this is what technology brings you.
0:42:39 I mean, nobody knows the answer to this, but given that you’re in charge of a company that’s measuring AI productivity and human productivity and kind of AI and humans working together.
0:42:43 I mean, what’s your timetable for, like, how fast things change?
0:42:46 Are we going to see, you know, net new jobs created?
0:42:52 Like every, and by the way, like, behind every one of these, there are all sorts of jobs that start becoming around that didn’t exist before.
0:42:54 So maybe that’s kind of part two of the question.
0:43:00 Because, like, the job that we have right now, like, you know, filming a podcast, like, that wasn’t a job like 200 years.
0:43:02 Like, there’s so many jobs that just, like, nobody could even think of.
0:43:11 So I guess, like, where do you think things are going and, like, what types of, maybe to put a nuance on it, like, what types of, like, future jobs do you see in and around this, like, new stuff?
0:43:20 So I don’t buy for a second there’s going to be large-scale job loss because of AI, frankly, because of what we’ve seen through all of history, which is just flat-out capitalism.
0:43:30 If my two choices are I can maintain my base level of productivity but fire a bunch of my employees and be more profitable, that is a fine idea in the short term.
0:43:39 If I’m – there’s probably a good idea for a PE firm to go along and go around and buy a bunch of minorly profitable companies, fire half their employees and make them more profitable.
0:43:45 But that’s what PE firms have done for a long time for noncompetitive companies anyway, yet employment has still increased, right?
0:43:56 So you can argue we’ve always had a function – or for the last 50 years, you can argue we’ve had a function whose goal is to take underperforming companies and fire a bunch of employees, right?
0:44:02 And let’s say that that’s what PE firms have done and that’s, you know, what AI could theoretically do, yet employment has increased.
0:44:10 So I don’t buy an AI – and look, it’s philosophical and, you know, I don’t have any special expertise because I’m building a measurement company.
0:44:14 But I don’t buy it because your competitor across the street is not going to fire all those employees.
0:44:18 He’s just going to do more with those employees and he’s going to kill your business, right?
0:44:21 I mean this is the Jeff Bezos, your margin is my opportunity line.
0:44:31 To the extent that AI is going to drive up your margin, that will be all of your competitor’s opportunity to be less profitable and compete with you.
0:44:39 So other than some very niche, monopolistic, I can fire everybody, you know, one-man firm, like will we have one-woman firms that do a billion in revenue?
0:44:40 Probably.
0:44:45 But today we have very profitable, you know, one-man, one-woman operations.
0:44:47 Not many people work at the Joe Rogan podcast.
0:44:52 I don’t think that many people work for Ben Thompson Incorporated, and yet I imagine those are quite profitable businesses, the best I can tell.
0:44:59 So that’s amazing and there will be a ton of opportunity to be a more successful solo entrepreneur.
0:45:02 So I absolutely believe there will be even more entrepreneurs.
0:45:11 But at a very high level, I just don’t believe the Fortune 500 will employ fewer people in 30 years than they do today because the ones that try and cut all the people will no longer be in the Fortune 500.
0:45:21 So I just flatly, because we live in a competitive world, we haven’t seen any proof yet that the economy is zero sum, right?
0:45:22 Maybe, right?
0:45:24 You can argue that, but we haven’t seen any proof yet.
0:45:25 GDP keeps increasing.
0:45:29 It increases slower in some places and faster in other places, but it’s generally grown.
0:45:30 Employment is generally grown.
0:45:35 I just don’t know why you’d believe that this time is different because of the competitive point of view.
0:45:37 The tech is different.
0:45:37 The tech is amazing.
0:45:54 But fundamentally, what will almost definitively happen, there is an interesting theoretical question that, you know, is more like a, you know, Ivy League grad school, you know, discussion about wouldn’t it be more fun as a society?
0:46:00 Wouldn’t we all be happier if everyone agreed we’d work half as much and be just as productive as we are today?
0:46:03 I don’t know, maybe, but that’s not human nature, right?
0:46:04 And so I’m not even sure that’s true.
0:46:08 I tend to believe in the Tyler Cowen point that all that really matters is growth.
0:46:18 And so my general perspective is you as a VC would just never get excited if one of your companies came in here and said, hey, we got to $100 million in revenue.
0:46:19 And you know what?
0:46:24 Because AI tools are so good, we’re going to fire 90% of our employees and we’re going to make $90 million in profit.
0:46:36 You would not be excited with that entrepreneur because you know that Sequoia is going to fund a direct competitor to that company who’s going to keep hiring, who’s going to be happy with 10% margins, and is going to destroy your company.
0:46:37 And we all know this.
0:46:46 So I don’t – it’s one of these things of like, you know, there’s a lot of fun headlines about OAI and then you have to have the counterpoint of, oh, it’s going to take the jobs and, oh, kids these days.
0:46:50 I don’t know, like when you – we all know this.
0:46:51 We’ve all seen this.
0:46:53 You can find articles about when the TV came out.
0:46:54 It was the end of reading.
0:46:56 When newspapers came out, it was the end of conversation.
0:47:04 So I do think it is scary that new tools are coming out and it is impacting the entire globe of all knowledge workers everywhere.
0:47:07 So what?
0:47:09 So I think there will be opportunities to be podcasters.
0:47:11 There probably will be more plumbers.
0:47:14 There will be a lot more employment around building data centers, right?
0:47:16 There’s going to be a whole set of engineers.
0:47:18 Maybe we will need a lot more astronauts.
0:47:19 Elon says we’re going to Mars.
0:47:27 Like someone is going to have to scrub the toilets in the space station and someone is going to have to pilot the ship to the space station.
0:47:28 The self-driving.
0:47:29 Yeah.
0:47:31 Self-driving spaceships.
0:47:34 So, by the way, there is a chance I will turn out to be wrong.
0:47:36 And in that case, I don’t know.
0:47:37 Maybe I’ll spend more time on vacation.
0:47:38 Well, it’s interesting.
0:47:41 I talked to this economist, Ed Glazier.
0:47:43 I think he’s at Harvard.
0:47:46 And he was saying – I asked him this question.
0:47:49 It’s like, you know, what’s going to happen with jobs?
0:47:51 And, you know, how do you compare this to everything else?
0:48:01 And he’s like, well, what’s really interesting is that this is arguably the first time that the job losses might be borne by, like, white-collar, super-educated people.
0:48:02 That’s why everybody gets scared.
0:48:04 Well, but so he actually had a different framing on it.
0:48:06 So, yes, agreed.
0:48:11 But almost tautologically, hyper-educated people are hyper-educated.
0:48:11 Sure.
0:48:22 So should be able to rejigger themselves and do something else versus in all of these previous revolutions where it’s like you have somebody that really has no skills.
0:48:22 Right.
0:48:26 And just, like, showed up at work with no skills and got paid.
0:48:27 Yes.
0:48:31 And there are a lot of jobs that look like that when there’s tremendous labor shortages.
0:48:31 Yeah.
0:48:36 So if you were doing anything in 1849, apparently, in California, like, boom.
0:48:37 Right.
0:48:39 It’s like there’s just like, oh, you’re a human.
0:48:39 Right.
0:48:40 You have a pulse.
0:48:41 You need someone with a pick.
0:48:43 Like, go do this.
0:48:45 Or just like, you know, you see that line over there?
0:48:47 Like, yeah, like, straighten it out.
0:49:01 So, but what’s different is that, yes, it’s scary for some people, but, like, everything right now, and maybe robots will work better in the future, but, like, everything right now is bit manipulation going after or augmenting.
0:49:08 I would argue more augmenting, white-collar, hyper-educated people by virtue of the fact that they’re hyper-educated.
0:49:08 Sure.
0:49:12 It might not, like, this does not mean that, like, you know, what happened to Detroit?
0:49:13 Sure.
0:49:13 Right?
0:49:15 It’s like that actually wasn’t about automation.
0:49:18 That was about, like, the Japanese built better cars.
0:49:18 Right.
0:49:19 Like, there are a lot of reasons why that happened.
0:49:27 But what do you do with somebody who had, like, a very, very high-paying job but actually didn’t have that many skills, and now they lost that job?
0:49:30 Well, because they don’t have any skills, they can’t find another job.
0:49:30 Right.
0:49:33 Whereas if you are highly skilled, you will find something else to do.
0:49:49 And I do think there’s an element of, look, there are certainly a set of people who were pretty highly educated, they were in good classes, they got into a good school, whatever that meant, they got a good job, they worked pretty hard in their 20s, a little less hard in their 30s, and a little less hard in their 40s.
0:49:51 But they’re paid pretty well.
0:50:00 And those people probably are a little uncomfortable today because their career, frankly, there’s some professions that just require continuing education.
0:50:07 By the way, if you’re an electrician or a plumber or a doctor or a lawyer, some of these professions just require constant upkeep and constant education.
0:50:10 That’s not true in a lot of professions.
0:50:15 There are a lot of jobs where you get to 40 or 50 and you can keep doing a good job, but you don’t really have to learn much new.
0:50:19 You can just keep doing a good job, doing what you’re doing, and you don’t have to learn much new.
0:50:20 And that’s probably quite uncomfortable.
0:50:23 I acknowledge it is quite uncomfortable for those people.
0:50:25 But to your point, they’re educated.
0:50:26 They have skills.
0:50:28 We have much more of a knowledge economy.
0:50:32 So I don’t necessarily have the issue of I literally have this house in Detroit.
0:50:35 The jobs are now in Knoxville, right?
0:50:36 Forget Japan.
0:50:37 The jobs are now in Knoxville.
0:50:38 I don’t want to move to Knoxville, right?
0:50:41 We all know the data on mobility and housing costs and all that.
0:50:41 So sure.
0:50:50 But at the end of the day, to your point, yes, there’s a set of people who’ve probably been slowly working less and pushing themselves less.
0:50:52 And now they have to push themselves more.
0:50:52 And that just is.
0:51:03 My joke all the time, I won’t say the company I use, but part of my sales pitch for Larrity and what I’m talking about is I say, look, when we talk to employees, you know, look, your average employee is a 42-year-old associate brand manager.
0:51:06 And if you ask them what they want out of AI, they do want it to go away.
0:51:09 Their number one wish would be that it would just go away.
0:51:11 I’d like that because I liked yesterday.
0:51:15 But we’d make a lot of money if we had the power to make AI go away, right?
0:51:18 Super big blackmail business, but we don’t have that power.
0:51:25 So all we can do is give you the tools to use these things better and help you be more productive and help you as a manager understand if your team is using these tools better.
0:51:32 And so, you know, my macro point is I just don’t really believe there will be wide-scale mass unemployment.
0:51:35 Might an individual have to push themselves more?
0:51:37 Yeah, for sure.
0:51:40 And some of those will be sad about that.
0:51:48 And, like, that same thing is true in the entertainment industry, right, that jobs have moved and jobs have shifted and people don’t watch movies the way they used to.
0:51:52 And, you know, TV seasons used to be 22 episodes and now they’re eight because consumer preferences have changed.
0:51:58 And so that is probably lousy if you were a guy who played a part on Law & Order, right?
0:52:00 That probably is uncomfortable.
0:52:01 I’m not being callous.
0:52:03 I’m sure there’s many ways it will impact my life negatively.
0:52:06 But I just don’t buy that they won’t be more educated.
0:52:08 Yeah, and a lot of this actually predates AI.
0:52:15 There was this great article or interview with the CEO of Waste Management before ChatGPT came out.
0:52:27 And he was saying, I have unlimited, like, I get resumes every day from somebody who has an MBA and wants to work, like, in our office for, and, like, they’re, like, negotiating against themselves.
0:52:29 Like, the price keeps going down there.
0:52:34 So 100 jobs for, 100 applications for every opening, or I forgot what he said.
0:52:36 I’m going to hire truck drivers.
0:52:36 Right.
0:52:42 Somebody who, and not just, like, self-driving, like, I need somebody who actually is collecting the trash.
0:52:43 Like, that’s what Waste Management does.
0:52:46 For $150,000 a year, I can’t find them.
0:52:49 So it is kind of interesting how things have flipped.
0:53:01 But I guess the other thing that I would say is, I would almost argue that a lot of AI’s problem right now, in terms of diffusing into the workplace, it’s almost a product marketing problem.
0:53:01 Sure.
0:53:01 Right?
0:53:04 Where it’s like, okay, AI can do anything.
0:53:04 Right.
0:53:05 Right?
0:53:06 But I’m not looking for anything.
0:53:09 Like, if I say, like, hey, I can do anything.
0:53:11 And you’re like, oh, I don’t need you.
0:53:14 It’s like, no, I can do this one thing very, very, oh, you do that?
0:53:28 And, like, that needs, like, this is kind of, I think, once you have more of these articulations of what can be done, and, like, the things that have really kind of gone hyper-growth, it’s like, oh, like, I have AI, it does everything.
0:53:30 Oh, I will help you code a hundred times better.
0:53:30 I will help you code better.
0:53:31 Oh, I want that.
0:53:38 I have this chat bot for, yeah, look, a long, long time ago, it’s funny when you get old, a long, long time ago, I was, like I said, I was the first guy at Comscore.
0:53:45 And Comscore’s sales pitch in the early days, Comscore, for those that don’t know, right, basically had all the data for everything that was happening on the internet.
0:53:51 And so the founders were true geniuses, and they basically knew everything that was happening everywhere on the internet.
0:53:56 And our sales pitch in the early days would be, we know everything.
0:53:59 I mean, obviously not, but would it basically be, like, we know everything, what would you like to know?
0:54:02 And it turned out that wasn’t a really good sales pitch.
0:54:06 You would sometimes accidentally run into someone who would go, oh, my God, I need to know this, can you do this?
0:54:08 And we’d say, yes, we could, and there you go.
0:54:11 But it turned out we only had a couple sellers that could figure that out in real time.
0:54:20 And then it turned out if we said, we can tell you the market share for Visa versus MasterCard versus others in Japan.
0:54:22 It turns out Visa really wants to know that.
0:54:28 But I can also tell you the share for your pharmaceutical drug versus others in research online.
0:54:29 It turns out they also want to know that.
0:54:36 After, I’m going to get it wrong, but Ford had some giant issue with Bridgestone tires setting on fire.
0:54:42 It turns out they really do want to know that, did employees, did search for Ford get worse because of Bridgestone, right?
0:54:43 So people do want to know these specifics.
0:54:47 And like I said, I think that’s exactly the right way to think about it is you need this product marketing problem.
0:54:49 Look, it’s why we’re so focused on what’s happening.
0:54:51 Are they more productive?
0:54:52 How do you get them to use it more?
0:54:56 We can actually do a lot of things, but you can’t sell things that way.
0:54:58 I mean, that’s more like general entrepreneurial advice.
0:55:05 But it turns out building something amazing that people don’t know how to use mostly doesn’t work.
0:55:07 Unless it does, which is ChatGPT, right?
0:55:09 So one in a million times it does work out.
0:55:11 Facebook, ChatGPT, on the consumer side.
0:55:12 Well, just like there, it’s just like it’s magic.
0:55:12 Yeah, it’s amazing.
0:55:16 If you show somebody a magic trick or you get somebody addicted.
0:55:16 Right.
0:55:20 And you’ll guess which one I’m referring to for which.
0:55:27 But if you ever watch Seinfeld, there’s this great episode where Jerry buys his father a sharp wizard,
0:55:29 which was like an early Palm Pilot.
0:55:32 It was like this early smart computer like in the 1990s.
0:55:33 Never went on to great things.
0:55:34 But it did everything.
0:55:35 It was like, you know, you could run these applications.
0:55:38 And Jerry’s trying to explain it to his dad.
0:55:39 And he’s like, well, I don’t get it.
0:55:39 What does it do?
0:55:40 He’s like, well, like, look here.
0:55:41 It has a tip calculator.
0:55:43 He’s like, oh, my God, a tip calculator?
0:55:44 And then he explains it to all of his friends.
0:55:45 He’s like, look at this.
0:55:47 My son, he’s a comedian.
0:55:47 He’s doing great.
0:55:49 He got me a tip calculator.
0:55:50 And Jerry’s like, no, it does other things.
0:55:56 And it often ends up being frustrating for the company that does the other things because
0:55:59 they aspire to have this like more broad horizontal platform.
0:56:02 But what we kind of need is more of these tip calculator things.
0:56:05 So I know we’re –
0:56:05 Yes, thank you.
0:56:09 But actually, why don’t we just – is there anything that we haven’t talked about that
0:56:13 you want to get in, like a little soliloquy that you can –
0:56:18 No, look, I think I’ll leave you with two thoughts, one related to something you brought
0:56:20 up in the conversation and the professor, but one related to Laredon.
0:56:26 So, look, my general perspective is anytime you see some giant shift in budget, you’re going
0:56:30 to build a set of very important but very boring tools.
0:56:31 What’s actually happening?
0:56:33 Are people more productive?
0:56:34 How do I get them to use it more?
0:56:37 And, you know, there’s a ton of business there.
0:56:41 And then I’ll leave you with an unrelated thought to your Harvard professor.
0:56:47 So, as you said, our kids go to the same school and my oldest kid is in 12th grade and just
0:56:49 got into college and he got into this top choice and he’s very proud of himself and it’s
0:56:51 a very highly rated school and I’m very happy for him.
0:56:53 And he came home and he said, Dad, look at this ranking.
0:56:57 And he showed me the new U.S. News World Reports ranking that showed the different rankings.
0:57:00 And I was like, his name’s Henry.
0:57:02 I said, Henry, I’m very proud of you.
0:57:06 There’s going to be a lot of different rankings over a lot of different years and there’s
0:57:07 only one thing you have to know for sure.
0:57:10 Whatever the ranking says, everyone knows number one is Harvard.
0:57:12 And so, it doesn’t matter.
0:57:13 Don’t get excited.
0:57:17 Whatever it says, wherever it puts your school, everyone always knows whatever the ranking
0:57:18 is, Harvard’s number one.
0:57:25 Thanks for listening to this episode of the A16Z podcast.
0:57:30 If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review
0:57:32 and share it with your friends and family.
0:57:36 For more episodes, go to YouTube, Apple Podcasts, and Spotify.
0:57:42 Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com.
0:57:45 Thanks again for listening, and I’ll see you in the next episode.
0:57:50 As a reminder, the content here is for informational purposes only.
0:57:54 It should not be taken as legal, business, tax, or investment advice, or be used to evaluate
0:57:59 any investment or security, and is not directed at any investors or potential investors in
0:58:00 any A16Z fund.
0:58:04 Please note that A16Z and its affiliates may also maintain investments in the companies
0:58:05 discussed in this podcast.
0:58:12 For more details, including a link to our investments, please see a16z.com forward slash disclosures.