0:00:07 Welcome to the next wave. I’m your host, Nathan Lanz. And today we’re going to talk all about
0:00:10 the future of shopping with AI. You know, in the last few months, we’ve seen perplexity
0:00:15 ad shopping. And now Chatsubiti has added shopping directly into Chatsubiti. You can
0:00:20 buy a product just by talking to your AI. Today, we’ve got on AJ Bam, the CEO of Viral,
0:00:24 a hot startup in Silicon Valley. It’s absolutely amazing because it makes video searchable.
0:00:28 And brands that are using this are already seeing their sales double. So I think you’re
0:00:32 going to learn a lot about where shopping is headed with AI, as well as ways you can take
0:00:36 advantage of this today in your business. So let’s just jump right into it.
0:00:43 Cutting your sales cycle in half sounds pretty impossible, but that’s exactly what
0:00:48 Sandler Training did with HubSpot. They used Breeze, HubSpot’s AI tools to tailor every customer
0:00:53 interaction without losing their personal touch. And the results were pretty incredible. Click-through
0:01:02 rates jumped 25%. And get this, qualified leads quadrupled. Who doesn’t want that? People
0:01:08 spent three times longer on their landing pages. It’s incredible. Go to HubSpot.com to see how
0:01:10 Breeze can help your business grow.
0:01:15 Hey, AJ. Great to have you here today.
0:01:17 Yeah, thank you. How are you?
0:01:23 I’m doing good. I’m doing good. It’s morning here in Kyoto, but waking up and excited to talk
0:01:25 with you today. You know, the reason I wanted to bring you on is, you know, I’ve been thinking
0:01:30 a lot lately about like what the future of shopping and e-commerce is going to look like with AI,
0:01:35 right? Like a few months back, you had Perplexity roll out. They’re like kind of AI-powered
0:01:38 shopping, which I thought was an okay experience. But I was like, okay, but I get where it could
0:01:44 go. And then OpenAI recently launched their thing, which I think they partnered with Shopify. And
0:01:50 they’re kind of like baking, shopping directly in to the LLM, which is like nuts. Like you can,
0:01:55 you know, I just imagine in the future, just being able to chat with my AI and get exactly what I want.
0:01:58 And I know you kind of play like a different role with your company, Viral, where you’re more on the
0:02:03 AI video side with the social shopping. So I just like love to hear like your thoughts on
0:02:05 the landscape and where things are at.
0:02:09 Yeah, absolutely. So first of all, thanks for having me on the show.
0:02:09 Yeah.
0:02:14 And I think you’re spot on that. I think what’s happening now is, you know, if you look at the
0:02:19 Gen Zs, they’re shopping now with all things video. And I think the proof in the pudding is TikTok,
0:02:20 right?
0:02:20 Right.
0:02:26 I mean, there’s a reason why TikTok became a phenomena. And TikTok proved that when you have
0:02:34 short form video at scale, and it’s authentic, interesting, funny, silly, perhaps as well,
0:02:40 it works. You know, I mean, just reflecting back on my company, right? Like what’s really
0:02:47 changed for us is life before TikTok and life after TikTok. So before TikTok, you know, I’d be knocking
0:02:52 doors, both with brands and retailers. And I’ve always believed that at the end of the day,
0:02:58 you know, people shop not because Kim Kardashian said you should buy a car, but people shop because
0:03:05 your neighbor who looks like you said, hey, Ajay, you know, I just bought my new EV and I love it.
0:03:11 And the reason I love it is it’s doing 200 miles on one charge. And so what really matters at the end
0:03:16 of the day is authentic opinions, right? Right. And where video is very transformative, where video
0:03:22 makes a huge difference is you can see the product, the person and the emotions inside the video.
0:03:22 Right.
0:03:28 So what that leads to really is high brand trust, high product trust, right? You can see the product in
0:03:34 action. So even like simple thing as my headset, right? Just being able to see in a video, whether
0:03:39 this fits, you know, loose or tight, I have a round face, right? Yeah. Like just being able to
0:03:45 see that versus someone telling you in a review that it’s in text makes a huge difference.
0:03:49 You know, it actually reminds me of is there’s this book I’ve been reading. It’s actually my second or
0:03:54 third time reading it, like Ogilvy on advertising. Oh, yeah, yeah, yeah. Of course. Yeah. Yeah. Yeah.
0:03:57 Maybe you’re even like kind of preaching that Bible. I don’t know. But they talk on that book about how
0:04:02 they found in all their years of doing advertising that people thought that hiring like a huge celebrity
0:04:07 celebrity work to promote something. But what they found was typically what happened is you pay those
0:04:12 people so much money. And if people remember the celebrity, not the product, you have to use a
0:04:16 person that people don’t know. It actually works better. Like they had like a, I think like a 80
0:04:21 year old grandma in France or something. And they had her doing, I think it was a butter commercial or
0:04:25 something like this. And that was like a huge hit for like 30 years or something. It was like a long
0:04:30 running ad they ran because people like authenticity. It’s just like a real person using a product.
0:04:34 Yeah. Yeah. Yeah, absolutely. I mean, I think you hit something very important here,
0:04:38 you know, and we’re seeing this trend as well in the market, which is brands are moving away from
0:04:45 very high paid influencers to more micro influencers and shoppers and authentic creators. Right. I mean,
0:04:52 today, in one of the reasons TikTok has exploded is also TikTok brings that average creator who
0:04:56 previously had to make a lot of effort on their phone to make a video. Now with TikTok tools,
0:05:02 anybody in the world can be a creator. Anybody can be an influencer. Right. So in general,
0:05:07 we’re seeing a trend where even in the market is, you know, I think between, I would say between
0:05:15 2018 and 2023, influencer marketing was all the craze. Right. And now what’s happening is, you know,
0:05:19 when you go online and you see another influencer, you’re like, oh, not again. You know, he was paid
0:05:24 to say this. Right. Right. And I think that’s where now there’s a massive tectonic shift. And I think
0:05:30 this is where again, you know, TikTok proved that when you have authentic content, it works. And I
0:05:35 think also what changes that now with, you know, we all have smartphones, I would say give and take
0:05:42 90% of all phones in the world can record pretty good these days, even in poor lighting. Right. So the
0:05:47 previous concerns that brands had about, you know, light being poor or the camera not being right or the
0:05:52 quality of the video not being good. And it’s not just the quality of the video as in capturing the
0:05:57 video. But even like with our 5G networks now, I mean, you live in Japan, Japan has had
0:06:04 fast, high speed phones. And, you know, Docomo was in fact, a leader when it came to mobile shopping.
0:06:07 You know, I lived in San Francisco for 13 years, and it’s crazy to me. I was like,
0:06:12 why is the internet not better in Silicon Valley? Like, what the hell is going on? You’re supposed to
0:06:16 be like in the Mecca of technology, then you go to other places. And it’s like, oh, they have better
0:06:20 technology, like infrastructure, they have at least better infrastructure, right? So it’s always kind
0:06:21 of shocking to me.
0:06:26 Right. I mean, Japan has been ahead in this game, to be honest with you, since 20 years ago. I even
0:06:32 hope you could pay in subway in Japan with your Docomo phone, which it’s only arrived now. I think
0:06:36 COVID was what changed. It took a virus to change our behaviors in the US, right?
0:06:37 Yeah.
0:06:40 But here we are now, I think Apple Pay has been accepted. But like I said,
0:06:45 tectonic shifts in your hardware, in your software, in the way you create content,
0:06:49 and in the way you consume content as well. So, you know, like watching a short form video today,
0:06:54 you can watch without disruption, right? I mean, previously, I remember like watching a video on
0:06:59 the phone, you would have to wait for that lag, you know, the latency on the video to happen,
0:07:04 right? And now I think, so all these factors have honestly have contributed to video really
0:07:07 taking off with when it comes to commerce. So what you’re saying makes me think that like
0:07:12 the future of, like, let’s say e-commerce 2.0 or whatever this is going to be, you know,
0:07:17 beyond just Amazon ruling everything, it feels like there’s probably two things. I assume that
0:07:22 eventually we’re going to have AIs that understand us as a person very well, and they’ll probably be
0:07:27 able to recommend things to us really well, maybe better than Amazon in the future. But then on the
0:07:32 discovery side, you know, if you’re somebody who wants to discover new things, you know, maybe that
0:07:36 starts leaning into the videos, right? Like, oh, I watched a video and, you know, maybe my wife saw
0:07:40 a video and she saw a purse and it’s like, who’s got that purse? There’s been a few times actually
0:07:44 where we’ve seen people that had a purse that she liked and literally once or twice, I asked them
0:07:48 where they got it. And she was shocked that I did that. She’s very introverted. I’m kind of,
0:07:52 I just started talking to this woman. I’m like, my wife loves your purse. Where did you get that?
0:07:56 And so I could see in the future that being like a main way that people discover it just through
0:08:01 social. And especially if you just make it simple and just click a button or something and you buy it,
0:08:05 that makes a ton of sense to me. Yeah. What we’re seeing is that I think this is what Instagram and
0:08:09 TikTok have really nailed, right? I mean, the algorithm is able to figure out, you know, who
0:08:14 you are, what your interests are. And based on your interests and your history, they’re able to
0:08:20 recommend certain products on your TikTok feed or your Instagram feed as well, right? People forget
0:08:24 that TikTok was like considered like the big AI startup as of like five years ago, right? It was like,
0:08:28 oh, they’ve got the best algorithm. They got the best AI. And then it just came out,
0:08:32 right? Yeah. Right, right. And I think the shift that’s happening now is this is moving now towards
0:08:38 retailers and like Amazon, right? I mean, if you go on shop on Amazon now, Amazon is leading all their
0:08:43 product pages now with video content. So there is the branded video at the top of the page. There is a
0:08:49 user-generated content in the middle of the page. And then Amazon runs its own retail media network as
0:08:54 well. They’re showing competitor ads, video ads on the bottom of the page, right? Okay. I mean,
0:09:01 my guesstimate is Amazon has amassed 250 million video reviews and video on their platform, right?
0:09:08 So that shift is now video is now jumping, I would say from social to retailers. And eventually we will
0:09:14 see video across, you know, all different retailer and DTC and other sites. And so essentially it’s
0:09:20 bringing social commerce to your retail and your DTC shopping experience, right? Now, there are some
0:09:26 challenges with that. And I think one of the big challenges with video in general is no one has time
0:09:32 to watch videos, right? So what I mean by that is when you’re in your shopping mindset, you do want to
0:09:37 watch the video, but you just want to watch the right video, right? So you’re buying a car now and you’re on
0:09:43 the BMW website. You have selected your car model and that 35 video on the page, right? There could be some
0:09:48 branded videos, BMW showcasing the product, the car features. There might be some testimonials from
0:09:54 customers, right? So let’s say you’re looking, Nathan, for a car with leather seats that is blue in color and
0:10:00 I’m looking for a car with child seats, right? With a female driver, right? So how do you know which video is
0:10:06 talking about leather seats versus child seats? So as we move towards video content consumption, I think the next
0:10:12 big problem is how do I find something in a video that’s relevant for me, right? Perhaps even personalized for me.
0:10:16 I mean, how many times have we been on a fashion website where you’re a guy and you’re being shown
0:10:22 videos featuring women, right? I think video personalization is not there yet, but it’s where
0:10:27 things are headed, right? So coming back to the BMW example, today most shoppers select a random video,
0:10:32 so they want to find that answer about leather seats. They select a random video based on a thumbnail.
0:10:37 A lot of videos today don’t have description and in fact, TikTok, Instagram videos don’t have a title
0:10:42 in description anymore, right? So how do you find something in a video? So I think finding ways to
0:10:48 help shoppers find what they need inside the video with video search and personalization recommendations
0:10:54 is going to be extremely important and key to really driving more conversion. And I’ll tell you this,
0:11:00 that video just, you know, from our own experience here at VARL, video increases brand trust by 20x.
0:11:00 Okay.
0:11:07 It increases conversion anywhere from, you know, 5 to 28% and it drives deeper engagement. So when
0:11:09 done right, it works, it’s magic.
0:11:12 You’re talking about like if somebody’s seeing a product, like if they see it, it’s someone using
0:11:15 a product versus just reading about it or whatever. It’s like a 20x jump.
0:11:22 Yeah, yeah, absolutely. Like, and again, brand trust, right? For someone to see how something works,
0:11:25 right? Whether it’s you’re putting your furniture together in the house and it’s a how-to video,
0:11:30 or perhaps it’s a makeup video and you’re seeing actually how you can apply makeup for your scan,
0:11:36 right? All of this, or you’re looking for a recipe and you’re just able to find in a 30-minute
0:11:40 recipe, you know, you want to know how to braise the chicken, right? If you’re able to find those
0:11:44 answers, that really creates a delight. And ultimately that’s going to drive a purchase, or
0:11:49 maybe people will spend more time on your site and do more, right? I mean, that’s the ultimate goal
0:11:54 that the brand. So search is a big problem. And the other big problem is no one has time to write
0:11:58 videos. They just want to, both on the brand side, you know, when they get a lot of video data,
0:12:03 they want to be able to find the video that’s going to help their shoppers, right? So how do
0:12:08 they find those nuggets? And on the shopper side, how do you find the video that’s going to answer
0:12:13 your burning question? So you can make a go, no, go, or even a comparison video, right?
0:12:17 Right. How do brands do it today? I know you guys have been working on a solution for this,
0:12:20 like before viral, like how would brands do that? How would they find that?
0:12:24 Yeah. Yeah. So to be honest with you, like, I think before viral, what we were seeing was
0:12:29 brands are creating content on their TikTok, YouTube, and Instagram, but we were seeing that
0:12:34 most customers, I would say both brands and retailers are not utilizing that content on the
0:12:40 e-commerce site, on their social site. So I would say the first thing is make sure you bring all your
0:12:44 video content that you’re producing even on your social media to your e-commerce site and website.
0:12:48 So we’re seeing that gap right now where they’re producing a lot of content for social, but hey,
0:12:53 they’re forgetting that at the end of the day, people do land on your website or your retailer on
0:12:57 your product page. And it’s the last mile where they make the product decision. So if you can also
0:13:02 influence them in that last mile in the shopper journey with video content, that’s going to be
0:13:07 extremely helpful. So I would say that’s number one. The number two is we see a lot of companies just
0:13:13 putting a video carousel and the video carousel is just placed there. And again, the problem is no one
0:13:17 has the 10 videos, right? I don’t have time to watch 10 videos. I just want to find that relevant
0:13:21 video, right? So how do you bring that relevancy is still a big problem on most sites. And then
0:13:27 brands are also, they have a lot of videos, how-to videos, support videos. And in most cases, we just
0:13:32 see a page with a list of videos. So again, they’re missing out by providing search, personalization,
0:13:39 some sort of recommendation engine. They’re missing out on really driving that self-service journey,
0:13:44 if you will, for the shopper to either buy your product or address a comment or concern they might
0:13:48 have with the video. So you’re talking about like a person on social media, they’re using a product and
0:13:52 now the brand is leveraging that video to promote the product. I wonder if there’s any way you could
0:13:58 create this kind of like, you know, loop where the person who created the video also benefits from that
0:14:02 being used somehow. Like there’s like a, you know, a link to them or something, because that could even
0:14:06 incentivize like, oh, now more influencers are going to want to use my product because, you know,
0:14:08 they might get mentioned now, right? They might get a boost from that.
0:14:11 Absolutely. Absolutely. I think that loop is coming circular, if you will.
0:14:17 So I’ll give you just a couple of examples. Now we’re like, we’re seeing now brands ask customers
0:14:21 for videos. So today it’s a bit of a manual process, but then there are tools like Varl,
0:14:27 like today after you make a purchase, now with Varl, you can set up a QR code or a message to invite
0:14:33 customers to make a video review, right? So what I’m going to do is I’m going to show first a demo example
0:14:38 on our site, and then I’m going to show you some live customers on how a number of our customers are
0:14:43 using the video carousel and more intelligent. I would say at Varl, we have built the next
0:14:49 generation video shopping experience, if you will. And really what we do is, so this is an example
0:14:54 where you’re buying an electronic toothbrush on an OLB website, you land on the product page,
0:14:57 and there are five video reviews of videos on the page.
0:15:03 So now, how do you know which of these videos are talking about what, right? So for example,
0:15:08 let’s say I want to know what people are saying about, it’s an electronic toothbrush, I want to know
0:15:13 what people are saying about the brush. So I can instantly search, voila, Varl found the videos now
0:15:19 talking about search. And you can see we generate all the key highlights, and this is all AI and ML driven.
0:15:20 Oh, that’s awesome.
0:15:25 So essentially, we make all your video, the audio, text, images, and transcription searchable,
0:15:29 and we generate the clips. We were searching for people are saying about the brush, you can instantly
0:15:34 find all the clips about the brush. So now before Varl, you would have to watch the entire video. Now
0:15:39 with Varl, you can just find the clips. And so what this does is this helps you find the answers you’re
0:15:43 looking for across one video or multiple videos. So if you look at the experience, right, we generate the
0:15:49 video summary, which is very extremely helpful for SEO. So Google can read the summary and the tags and
0:15:54 the transcript. We also generate highlights on top of the videos. So these are all the key highlights
0:15:59 that we have generated on top of this about what’s being said in the video as well. So if I go to the
0:16:04 next video, it will show you the next search result about talking about brush. And it also shows you the
0:16:08 highlights as well, right? And what’s cool is this is integrated with the shopping experience.
0:16:13 So as a shopper, you can just, or the shopper website, this is integrated with buy now. So
0:16:18 essentially a customer can hit buy now and add the product to the shopping cart and make a purchase.
0:16:24 Hey, we’ll be right back to the show. But first I want to tell you about another podcast I know
0:16:29 you’re going to love. It’s called Marketing Against the Grain. It’s hosted by Kip Bodner and
0:16:34 Kieran Flanagan. And it’s brought to you by the HubSpot Podcast Network, the audio destination for
0:16:38 business professionals. If you want to know what’s happening now in marketing, especially how to use AI
0:16:43 marketing, this is the podcast for you. Kip and Kieran share their marketing expertise,
0:16:48 unfiltered in the details, the truth, and like nobody else will tell it to you. They recently
0:16:56 had a great episode called Using ChatTBT03 to Plan Our 2025 Marketing Campaign. It was full of like
0:17:02 actual insights as well as just things I had not thought of about how to apply AI to marketing.
0:17:08 I highly suggest you check it out. Listen to Marketing Against the Grain wherever you get your podcasts.
0:17:15 So I’m going to just show you really quick a couple of other examples of live customers. So we have a
0:17:23 beauty brand, RX. They do beauty products for women, especially for curly hair. And they have now added
0:17:29 video content on all their sites. It’s a very rich, engaging experience. As you can see, you know,
0:17:32 the previous video didn’t have any speech, so we didn’t generate the highlights. But whenever there’s
0:17:37 speech, we generate the summary, we generate all the key highlights of what they’re saying. So even as a shopper,
0:17:42 you don’t have to like watch the entire video. Yeah. You can watch the clip that’s relevant for you
0:17:47 and make a purchase. This seems great for brands. You know, I think for average, like product videos
0:17:51 are so boring, right? Yeah. And usually you watch them as like, okay, whatever. It’s like highly
0:17:55 edited, you know, whatever. Yeah. But like something like this, like you actually see real people using
0:17:58 the product. Yeah. That would convince me more. Yeah. Also for a brand, that’s great, right? Because
0:18:01 they don’t even have to spend all this money on this advertisement. It’s not even going to work.
0:18:07 Exactly. Show the real people using it. That’s the ad. Right. And the beauty is, you know,
0:18:12 we have integrated with Instagram, TikTok, YouTube, Dropbox, Google. So we can bring all your TikTok
0:18:17 content that you’re producing. We can bring all that content to your e-commerce site. So essentially,
0:18:22 we have built a mechanism to pull in all your videos that you have already produced, or you can also
0:18:27 leverage our platform to capture video reviews as well. So here’s an example of a head. So this brand
0:18:32 has increased their site engagement by about 3.8x. They’ve increased their revenue overall
0:18:38 conversion on average by about 6% last quarter. So video has a direct impact because again, you can
0:18:42 see these are real testimonials from customers. They’re talking about how they can use the product
0:18:47 and more, right? So I’ll give you another example. So here’s a company that does hot sauce, right?
0:18:52 So they have a number of recipes. You know, there’s only so much heat I can bear. I don’t eat like very,
0:18:57 very hot foods. So I want to know, is there a comment about heat and about the sauce, right?
0:19:01 So instantly, it’ll find the video talking about heat, right? So this is a game changer. What is
0:19:06 this doing, right? This is going to help you instantly decide, like, for example, this comment
0:19:11 will help me decide whether, you know, where I want to use the sauce. Is it talking about heat? It even,
0:19:16 we generate all the key highlights about this hot sauce as well, right? So here we bring search,
0:19:22 personalization, SEO, recommendation engine, and buy now to all this content.
0:19:26 Yeah. I love that idea. I guess my one question there, it’s like, it’s almost like introducing a
0:19:30 new user behavior though. Like, are people interacting with that feature a lot, the search? Because
0:19:33 it’s an amazing feature. I’m just curious if people know how to use it, right?
0:19:37 Yeah. It’s a great question. So what’s happening right now is there’s a couple of things happening,
0:19:41 right? Like everything new, it’s very clear on top of the videos, right? That you can search,
0:19:46 right? So we’re also in the process of integrating this video search with your site search,
0:19:49 right? Okay. So where you have the site search, I mean, most people are familiar with tech search,
0:19:53 right? Right. So when you start typing the product name or you look for a keyword,
0:19:58 you will essentially find the videos. It will also surface video content as well. So we’re actually
0:20:01 building this as we speak. You should do probably like good suggestions as well, right? Like almost
0:20:05 like auto-complete. Absolutely. You’re searching for that and it’s like, how hot is it or whatever,
0:20:09 something like this. And then that’s, and then it shows the videos. Right. And I think to your point
0:20:12 though, we’re doing sort of a crawl, walk, run approach, right? The crawl approach is,
0:20:17 hey, let’s put an amazing video experience on your site. The walk approach is, let’s make sure we
0:20:22 integrate search across your entire site on all your pages as well, right? It could even, you might
0:20:27 even have a support page for your brand, right? So we have different implementations you can put
0:20:32 essentially on, even on your homepage or your product page. You can also put videos as well,
0:20:36 right? So essentially, Warwell has a set of e-commerce tools where you can bring search capability to
0:20:41 your, for your site search, for your homepage. We also bring a very rich video experience as I just
0:20:46 showed you on your site as well. It’s a whole new way of shopping that’s never happened before.
0:20:49 Yeah. It seems awesome. I guess one thing that keeps popping my head is like, I’m sitting here
0:20:54 thinking about like the future of the web and my slight concern is like, does ChatGPT eat the web?
0:20:59 Do people like, you know, basically live in ChatGPT? They’re not even, you know, maybe the future
0:21:03 during the browser, like you literally just like open up the ChatGPT app and that’s your like
0:21:07 surface into the web versus using a browser. Yeah. Kind of worry about that.
0:21:11 I think to be honest, like Nathan, it really is going to come down to trust. Yeah. Can you trust
0:21:16 ChatGPT to complete your transaction? I think there’s already enough scams and frauds with payments and
0:21:21 whatnot, right? Yeah. The challenge is with any LLM says, even if they are able to, I mean, you know,
0:21:26 like to your point, if you go on perplexity, it shows you, you know, I was looking for a role B
0:21:31 toothbrush. It essentially brings up a quick pop-up with the image. It summarizes the reviews
0:21:36 from multiple platforms. So I think there’s a benefit to getting that review, perhaps a summary
0:21:43 of the reviews on ChatGPT, but ultimately my reward points are tied to Amazon. My reward points are tied
0:21:48 to Target, right? You see where I’m going with this? So ultimately I believe that when a person is
0:21:52 shopping, they might even want to do comparison shopping. And, you know, as you can imagine,
0:21:56 retailers do a fairly decent job now showing you similar recommendations for similar products
0:22:01 across multiple brands, right? And it also will depend on the size of the purchase as well.
0:22:06 So for example, like, you know, people definitely buy, if the product is under $50, you’re very likely
0:22:11 to purchase it on TikTok, Instagram. I think the amount is much lower for trust and fraud to worry
0:22:17 about versus when you’re buying a car or when you’re buying a $300 cappuccino machine and really want to
0:22:21 see all the videos on how the machine works, the espresso machine works, right? So I would say that
0:22:25 the last miles still happen on the retailer or the brand website.
0:22:29 Yeah. I feel like what’s probably going to happen is ChatGPT will send people to websites still.
0:22:32 I hope that’s what’s going to happen, right? Like I’ve been seeing, that’s actually been a big thing
0:22:38 I’ve been thinking about and seeing people post on X about is that everyone’s saying that traffic from
0:22:43 ChatGPT is converting better than Google, which is just like, oh crap. So what does that mean?
0:22:47 That’s a huge disruptor to Google long-term, which Google’s now getting good at AI,
0:22:50 but it feels like people will still go to the website. And then when they’re looking for the products,
0:22:54 yeah, a lot of people would love to see a video of a real person using it.
0:22:58 And again, you know, also don’t forget in-store shopping experience. I would say 90% of still,
0:23:04 you know, big purchases happen in-store as well, right? So, but can bring video to in-store shopping,
0:23:10 you know, we’re seeing more and more packaging with QR codes. We’re seeing even on televisions in
0:23:15 stores as well, as they’re playing different product reviews and product videos, you are now actually
0:23:21 able to just scan the QR code and open a video experience to read the reviews or watch the
0:23:26 videos in-store as well, right? So my point is, I think it’s going to be an omni-channel experience
0:23:27 at the end of the day.
0:23:31 You know, there’s a few places in Japan, I’m not sure if America has this yet, but when you shop,
0:23:37 you just put the product down and then they know how many of the products you have. I’m not sure
0:23:38 exactly how they’re doing that.
0:23:42 You literally just put the products there and there’s no scanning. It just knows how many of
0:23:46 the products you have. That’d be really awesome in the future if there’s some area in stores where it’s
0:23:51 like, you just put the product down and then it’s just like, oh, here’s on social media, you know,
0:23:55 all these people. Maybe there’s like one famous person, but there’s people who are not famous and
0:23:58 it’s kind of a mixture and you could just click it and watch the videos. That’d be so cool.
0:24:02 Again, the future of commerce with video is going to be very different.
0:24:06 Right now, I would say it’s more two-dimensional video. You know, I wouldn’t be surprised if we
0:24:12 see 3D AR, VR experiences as well with video content and reviews, which you might be able to just
0:24:17 experience with your phone. And maybe, you know, as you’re seeing more smart eyewear come through,
0:24:23 I can see an application where you just stare at a product with your eyewear and it’s literally
0:24:27 pulling the video review on your screen. I’m not kidding, right? It’s coming. It’s a matter of time.
0:24:29 Yeah, I can definitely see that in the future. You’re looking at a product,
0:24:34 you see the reviews. I want to see videos as well. That’s so cool.
0:24:38 So I think this might be a good segue into talking about, you know, what Vyral does behind the scene.
0:24:39 Yeah, for sure.
0:24:42 I’ll do a quick demo of the dashboard, just a very teaser demo.
0:24:45 Yeah. So if you could show me how Vyral works, show me the tech behind the scenes,
0:24:48 you show me the widget. So that’s cool. But like, how would a brand actually use this?
0:24:54 Yeah, absolutely. So essentially, you know, the good news is most brands that do 5 million plus in
0:24:59 revenue already are producing some video content. So, you know, as I mentioned before,
0:25:04 the biggest challenge today is making all your video content searchable and making it useful.
0:25:11 So really what I’m showing you is a demo for Oral-B. It’s really a sample snapshot of some of the data.
0:25:17 So in this case, we captured 24 video reviews. The brand was interested in managing videos for a few
0:25:22 of their electronic toothbrush products. So what we did was we captured reviews from customers via QR code
0:25:28 campaigns. And we also captured reviews from social media. So in this case, these 24 video reviews came
0:25:33 in from either social media or via campaign. So essentially on the Oral platform, a brand can
0:25:38 essentially set up a QR code. So here’s an example where you can set up a QR code campaign and you can
0:25:44 invite your shoppers. So the brand actually sets this up and they set up the personalized experience.
0:25:48 And what you’re seeing is actually a mobile experience. They might issue, give you a reward
0:25:53 for making a video. It’s optional campaign description, brand instructions. And the big problem we have
0:25:59 solved, Nathan, is licensing. So the biggest fear that brands have is someone putting a video on their
0:26:01 site or something that has been unlicensed and they get into a lawsuit.
0:26:06 How do you capture video reviews at scale, right? Is the problem that the viral has solved. So now
0:26:12 with us, you can essentially create a campaign and it generates a QR code and you can tie this QR code
0:26:17 to your shopping experience. So you can, after a customer makes a purchase, you can invite the
0:26:22 customer to upload a video review. So when they scan the QR code, it will prompt them with the campaign
0:26:27 details and it will invite them to upload a video. And what happens is the video directly comes on the
0:26:32 viral dashboard and it gets analyzed. So in this case, we have 24 reviews that came in. The video is
0:26:36 social, we track your social media engagement. We track demographic who’s in the video by age,
0:26:40 ethnicity, gender. So you’ll notice at a glance that there are 40 to 50 year olds missing in the
0:26:45 video. So we have actually analyzed who is in the video. And by the way, we don’t store any personal
0:26:49 information. This is simply at a very high level, helping brands understand, you know, are people
0:26:53 making your videos, aren’t they on target with your demographic, right? So you notice there’s no
0:26:57 40 to 50 year olds in the video. So maybe if they’re targeting that demographic, they should have
0:27:02 videos featuring 40 to 50 years old. And then below, we launch a rating system.
0:27:06 So at a glance, brands can decide whether they want to publish the videos or not. So for these
0:27:11 products, for this SKU, we captured 11 reviews and they have a score of 89. So the higher the score,
0:27:16 better the video. So the score is about 60. The brands can promote the video. If the score is below
0:27:20 40, it’s very negative. And we even generate the highlights. So if you select the video, it will
0:27:24 take you inside the video to the key highlight. So as a brand manager, e-commerce team, you don’t have
0:27:29 to really spend time watching all your videos. Now, what makes Varo really special and unique,
0:27:35 and, you know, recently we just landed a contract with TikTok to power video reviews for TikTok and
0:27:40 for the 500,000 TikTok shops. And the reason we got that contract is we have 150 different filters
0:27:45 on the platform. So on the right, you see all these filters. So essentially, it’s a mechanism for brands
0:27:50 to moderate their video content. And all of this is offered via an API. So you can now search by positive
0:27:56 sentiment. You can say, show me everyone in the video that’s 18 to 24, 25 to 30, to 30 to 40 years
0:28:01 old. And I’m looking for a video that talks about battery life that I want to promote. So you can
0:28:05 instantly search the word battery life or a product feature. It’ll show you all the comments about
0:28:10 battery life. And not only can you see the comment, you can even open the video. It will take you inside
0:28:16 the video. So essentially, either TikTok will help capture a video after the purchase is made on TikTok
0:28:21 shop. Yeah. And then viral will analyze the video and we’ll have a merchant dashboard where TikTok
0:28:25 merchants can log into the viral dashboard. They can see all the insights for all the video reviews,
0:28:31 and they can then publish that content to the TikTok shop. So TikTok has given us exclusive right
0:28:35 to be able to publish video reviews that are vetted, right? Oh, that’s awesome. That’s huge.
0:28:40 Because TikTok has 30 rules. You know, you cannot mention the word Amazon in the video. The video has
0:28:44 to be less than three minutes long. You cannot have minors in the video. It has to be brand safe. It has to be
0:28:48 properly licensed, right? There should be no profanity in the video. So Vorl is building a
0:28:54 filter, a TikTok filter, an Amazon filter, or a Walmart filter to be able to vet this content with
0:28:58 these filters we have on the right, right? That’s the use case for TikTok. For other brands and retailers,
0:29:03 they can use our platform to manage all their video content across multiple SKUs, across multiple
0:29:08 platforms. So we have one dashboard. We are now going at Gentic as well. So we just launched an AI
0:29:14 e-commerce agent, where you can ask questions about your video data, and you can get answers. So here’s a couple of
0:29:19 examples. You know, I wanted to know what are some of the top topics being discussed in the videos, right?
0:29:19 Right.
0:29:22 So I can ask questions now. So it gives you an answer.
0:29:22 Oh, that’s awesome.
0:29:27 So now brands can, instead of having to watch videos, they can spend more time on action and
0:29:31 content. They can instantly get insights. So they want to create content. Well, they just launched a
0:29:35 new toothbrush. Hey, what are customers saying about cleaning? Right. They get a summary along with the
0:29:36 clips, right?
0:29:40 Right. You probably could get unique insights from that too. Like, oh, people in Japan are talking about
0:29:43 this or whatever. What are they saying? And like, you could actually learn about new opportunities for
0:29:44 your brand through that too.
0:29:48 And speaking of insights, these are all the insights we deliver. And I’ll just give you a quick
0:29:53 example of an insight. So if I open the speech report, by the way, brands love the speech report.
0:29:58 They use it for SEO, for identifying competitor mentions and trend analysis. So we can take you
0:30:01 inside the keyword and you can even play the clip. It’ll take you inside the video, right?
0:30:02 Right.
0:30:06 So we have different reports. And I want to show you one last thing, the level of insights we offer for
0:30:11 our customers and the way we make the video searchable. We generate the video summary. We detect the
0:30:14 sentiment score, whether people are saying positive, negative about your product or brand.
0:30:19 We detect languages, sentiment analysis, topics, demographic. We have marketing workflows.
0:30:23 You can do sentiment analysis of the audio. We even break it down by product feature,
0:30:28 which has never happened before. So we have built our own models. So it’s not LLMs. We actually have
0:30:33 our own AI, about 18 plus models that understand everything inside your video review and your video
0:30:34 content.
0:30:35 You say 18 models?
0:30:40 Yes. So that’s why we’ve been at this for a while. And we’re going deep with our models in e-commerce,
0:30:44 right? So here’s where, you know, we give you all the insights on your content. So we then have
0:30:48 a mechanism to publish content. So you can build your own carousel. So programmatically,
0:30:52 you can publish content to your site and you can customize the whole widget, the colors,
0:30:56 you know, whether you want the highlights, whether you want the search or not. And instantly,
0:31:00 and we can enable a search experience. So that we have solved the Holy King of Commerce,
0:31:05 which is in-video search, personalization, SEO, recommendations, and buy now inside the video.
0:31:06 So cool.
0:31:08 Yeah. So that’s a quick, very short demo.
0:31:12 Yeah. One thing that like really stuck with me, I was thinking one of the big values was like
0:31:17 having the more organic videos talking about a product. And so it seems like most brands are
0:31:21 only doing the opt-in. Is that like, they have like a campaign. Is that the main thing they’re
0:31:21 doing?
0:31:25 Well, today what happens is outside viral, if they’re not using viral, they usually do branded
0:31:27 and influencer videos. Right.
0:31:31 The problem is how do you capture a licensing rights from your shoppers at scale? Right.
0:31:32 Right.
0:31:37 So with viral, you can now program the QR code either with our API or with our campaigns on your
0:31:38 store. Yeah.
0:31:43 And after a customer makes a purchase, they get the QR code. So essentially the customer scans the QR code,
0:31:45 makes the video, accepts the licensing terms. Yeah.
0:31:48 They’re sharing their licensing terms, by the way. Right.
0:31:50 And essentially they’re giving the right to use the video at scale.
0:31:55 Yeah. Random idea. Like I would imagine like now you could take something like, I don’t know,
0:31:58 the new Gemini 2.5 or something like that. I’m not sure how well this would work at scale,
0:32:03 but it seems like a really easy experiment. But you could have that email or contact all these
0:32:08 people and like hype it up where your name’s going to be linked on there and stuff. So you may gain
0:32:12 some followers. Like if you’re someone who has like 200 followers or a thousand followers and like a
0:32:16 brand’s contacting you and it’s like, they’re going to show my face and they’re going to like link to me
0:32:21 somewhere. Like I think a lot of people, if there was like a one click opt-in, you know, and then you
0:32:24 got the contract, it’s like, okay, it’s fine now. It’s good. I’m not sure if you guys would do that.
0:32:26 That is something worth considering in the future.
0:32:30 Yeah. I mean, I’ll just, I’ll just give you an example. Like I think where things are headed,
0:32:34 I’ll give you a fun example. So imagine you just watched a Mission Impossible movie
0:32:39 and you’re coming out of the theater and you bought a ticket on Fandango or AMC theaters.
0:32:44 And imagine you get a quick nudge by the time you’re home from Tom Cruise saying, Hey, RJ,
0:32:51 do you have a minute to chat with me? So this is where future of commerce and conversational AI is
0:32:55 going with video reviews, right? So imagine you open the video and you’re just intrigued. Hey,
0:32:58 Tom wants to talk to me, right? So, you know, it’s a licensed avatar of Tom Cruise,
0:33:04 just to be clear. Right. And imagine like Tom Cruise says, Hey, I have three questions for you.
0:33:08 Do you think I should make Mission Impossible 5? Right. And how is the theater experience? Right.
0:33:14 And what else can we do better? Right. Or any other ideas? Right. So this is where brands can program
0:33:20 25 questions, right? Maybe Tom Cruise, his production company could have asked 25 questions to his fans and
0:33:23 followers. Yeah. That’s definitely going to happen. So I’m not sure if you know this,
0:33:27 but the original way I was using lore is I was actually lore.com is I was partnered with
0:33:31 Barry Osborne, the producer of Lord of the Rings and the Matrix. Yeah. And he helped create Weta with
0:33:36 Peter Jackson, right? The big special ethics company. They also do a lot of the gear. And so I got like
0:33:43 a VIP tour of Weta. And what I was shocked by was they had the facial scans of so many famous actors.
0:33:47 And apparently all of these studios and production companies, they already were thinking about AI.
0:33:51 Oh, yeah. Like for a long time now. They’re like, yeah, in the future, you’re going to want to have
0:33:55 Tom Cruise’s rights to be able to use it in other products. And you have some kind of revenue share
0:33:58 with him or whatever his estate in the future. That’s crazy. That’s coming.
0:34:03 Even with your podcast, I wouldn’t be surprised if another six months, a pop-up comes up and you’re
0:34:07 chatting with me on your site. You’re an avatar of you and saying, hey, RJ, what podcast should I be
0:34:11 making? What topics are you interested in? And you’re actually chatting with me to capture some
0:34:16 information from your visitor, site visitor, right? Right, right. So I’m telling you like where things
0:34:19 are headed, I think the future is going to be amazing. I can’t wait. It’s going to be fun.
0:34:24 It’s going to be fun. And also just last thing I would say is I think all these platforms, including
0:34:29 viral, you know, we have to be transparent and honest about what’s AI generated versus not, right? I mean,
0:34:35 just a quick point I’ll make is that, you know, when it comes to video reviews, people do not want AI
0:34:39 generated video reviews. I’ll tell you that. So we’re building tools to identify, to make sure that our
0:34:44 review is human generated and we’re validating it’s a human on the camera and not something else.
0:34:49 If you were building a business today, how would you be preparing for like these changes are coming
0:34:53 with shopping? Like how could you take advantage of the changes that are happening? Well, I think from
0:34:58 a company perspective, I would say that it’s really important now that all your employees are up to speed
0:35:03 and trained on AI and how what AI does, AI works. In fact, I would say that the first question in the
0:35:09 interview should be AI related. Right. So I would say that training is very important. The other thing is also
0:35:16 make sure you are on top of things like whatever domain you’re in. There’s plenty of newsletters. There’s plenty of
0:35:22 tools now. I would say be hands on. Like I think we live in an age right now where whether you’re a CEO or whether
0:35:28 you’re, you know, everyone in the company, right, at all levels, right, need to be playing with tools because and now,
0:35:33 you know, tools are free. Right. They’re easily accessible on your browser. There’s no reason to complain that you don’t have
0:35:39 access. Right. I think that excuse has gone away. Right. Right. And also like getting your employees to play with tools as
0:35:44 well. Let’s have people try different things. Right. Right. And in general, there’s a transformation happening
0:35:50 as well. Right. And then eventually figuring out, you know, what is going to make your job easy, cost-effective.
0:35:55 How can we bring more efficiencies for employees with AI? I think that’s starting to happen.
0:35:59 And we’re already seeing that in our company as well. Right. Yeah. I’ve been seeing tons of
0:36:05 different CEOs starting to share like almost like an AI first approach to hiring. Like not trying to have
0:36:09 less people, but make sure that everybody you’re hiring knows how to use AI because that’s just going
0:36:14 to amplify their outputs by, you know, so much more. Yeah. Well, it’s been awesome talking with you. I
0:36:19 think I learned a lot, especially about TikTok and about how people are finding products through social
0:36:23 videos. It makes a ton of sense. Like I think in the future, it’ll be like how I kind of described in the
0:36:27 beginning where the LLMs will know you very well. They’ll be really great for like buying basic
0:36:32 things. Yeah. But there’ll also be things where you want to discover new products and, you know,
0:36:37 sync social proof is huge there. It’s huge. I would say like there are five areas directly that will be
0:36:44 impacted in future with AI, like, or with chat TPD. And that is content generational, conversational AI
0:36:51 with video, personalization, SEO. You know, I think the reason now LLMs are doing a better job is
0:36:56 they’re able to better understand content than ever before. Right. I think that’s the problem
0:37:02 they’ve solved, right. Whether it’s video or audio or text. So SEO will get better. SEO will improve
0:37:08 significantly with LLMs as well. I’m now even like coining the word LEO, which is LLM based engine
0:37:13 optimization, if you will. Oh, okay. You know what I mean? Like, yeah, I’ve been saying AIO. That’s true.
0:37:17 Doesn’t exactly make sense, but it’s kind of catchy. Where should people check you out? Like,
0:37:20 should they check out your website or are you active on social media anywhere?
0:37:25 Absolutely. So, you know, we’re very active on LinkedIn. You can check out my website as well.
0:37:30 So if you just go to viral.com slash commerce, if you want to play with the experience, you can do
0:37:34 that. By the way, the word viral, it’s a play on the word viral. I do videos go viral, right? But it’s
0:37:41 spelled a little bit differently. It’s V-V-I-R-I-L-L. So it’s viral. Right. Again, V-V-I-R-I-L-L.com.
0:37:46 Cool. Yeah. We’ll put a link in the description. So yeah, it’s been great. And yeah. Hope to see
0:37:48 you again sometime. Thank you. Awesome.

0:00:06 everyone’s talking about vibe coding but the reality is for most things vibe coding doesn’t
0:00:11 work right now and even the guy who coined the term andre carpathy he recently posted that he’s
0:00:15 now trying to provide more context to models because he’s realized that’s what you have to
0:00:20 do to get good results back welcome to the next wave podcast i’m your host nathan lands and today
0:00:25 i’m going to show you the secret weapon that all the top ai coders are using you know everyone’s
0:00:29 talking about vibe coding this vibe code that but what they’re not telling you is that you can’t
0:00:34 vibe code most of anything that’s actually important right now for any important ai coding you want to
0:00:38 give it the proper context to know what it’s doing versus just throwing everything at which is what
0:00:42 cursor and windsurf and a lot of these other tools that everyone’s talking about do today i’ve got the
0:00:46 founder of repo prompt eric proventure on here and he’s going to show you how you can use repo prompt
0:00:54 to take your ai coding to the next level so let’s just jump right in cutting your cell cycle in half
0:00:59 sounds pretty impossible but that’s exactly what sandler training did with hubspot they used
0:01:04 breeze hubspot’s ai tools to tailor every customer interaction without losing their personal touch
0:01:13 and the results were pretty incredible click-through rates jumped 25 and get this qualified leads
0:01:19 quadrupled who doesn’t want that people spent three times longer on their landing pages it’s incredible
0:01:24 go to hubspot.com to see how breeze can help your business grow
0:01:34 hey we’ll be back to the pot in just a minute but first i want to tell you about something very
0:01:39 exciting happening at hubspot it’s no secret in business that the faster you can pivot the more
0:01:45 successful you’ll be and with how fast ai is changing everything we do you need tools that actually
0:01:50 deliver for you in record time enter hubspot spring spotlight where we just dropped hundreds of updates
0:01:56 that are completely changing the game we’re talking breeze agents that use ai to do in minutes what used
0:02:03 to take days workspaces that bring everything you need into one view and marketing hub features that use
0:02:09 ai to find your perfect audience what used to take weeks now happens in seconds and that changes
0:02:15 everything this isn’t just about moving fast it’s about moving fast in the right direction
0:02:22 visit hubspot.com forward slash spotlight and transform how your business grows starting today
0:02:28 thanks for coming on yeah yeah it’s nice uh you know finally put a face to it you know we tried it for a
0:02:33 while and uh it’s cool you’ve been using uh repo prompt for a few months now yeah yeah i’ve been telling
0:02:37 people about repo prompt for like the last you know probably six months or so kind of felt like it’s been
0:02:41 almost like my like ai coding secret weapon you know it’s like yeah everybody talking about cursor
0:02:47 and now windsurf and i do find cursor useful but i was like why is no one talking about repo prompt
0:02:51 because like for me every time i’d get into like a complicated project as soon as the project got a
0:02:55 little bit complicated the code from cursor would just stop working for me like it would just not know what
0:03:00 was going on you could tell it wasn’t like managing the context properly and then when 01 pro came
0:03:04 out that was when i really noticed repo prompt and started using it a lot yeah you had to go to 01 pro to
0:03:08 really get the best out of ai for coding at that point absolutely wouldn’t even work with the 01 pro
0:03:12 and so repo prompt was by far the best and it was just kind of shocking me like only like a few people
0:03:16 on x are talking about this yeah most people don’t know about it yeah i mean like it’s the only tool
0:03:21 that i use to work with ai and you know for a long time it was just sonnet and i would feel like i was
0:03:25 able to get a lot more out of sonnet than other tools just because you know the full context window
0:03:30 was there and you know i wasn’t bleeding through the nose with api costs uh doing using the the web chat and
0:03:34 just let me get to a place where i was able to get a tool that was able to do like not just like
0:03:38 putting context out but like taking the changes back in and applying them yeah i like to think i’m the
0:03:42 number one user but actually like look at the stats sometimes and i don’t think that’s even true anymore
0:03:46 yeah i mean i really wanted to bring you on after i saw that tweet from uh andre carpathy the day
0:03:52 so andre carpathy he used to be at tesla ai now he’s like one of the best educators about how lms work
0:03:56 and things like that he had his tweet saying noticing myself adopting a certain rhythm in ai
0:04:03 assisted coding i code i actually and professionally care about contrast to vibe code you know he coined
0:04:06 the term vibe code which everyone’s been using and then he basically goes on to talk about like
0:04:11 stuffing everything relevant into context all this i was like he literally he doesn’t know about repo
0:04:17 prompt i’m like how did this like top ai educator in the world top expert everything totally has no idea
0:04:21 about repro prompt i was like okay so i need to get eric on the podcast and we try to help with that
0:04:26 yeah i appreciate that yeah i mean yeah looking at that that tweet you see exactly like that flow that
0:04:29 like got me started like when you start getting serious about coding with ai like you start thinking
0:04:33 like it will how do i get the information to the ai model and like the ux on all these other tools is
0:04:38 just not cutting it you need a tool to just be able to quickly select search for your files like find
0:04:41 things and yeah you know i recently added the context builder i don’t know if you’ve tried that out but
0:04:45 maybe you know if you could explain like try to simplify it yeah and i think we should then just jump into a
0:04:49 demo and we can kind of just go from there sure thing sure thing yeah i mean the first thing you’re
0:04:53 going to do when you’re going to open up repo prompt is pick a folder so i can either open a folder
0:04:57 manually or just go to the last ones used but generally when you’re working with some code base
0:05:00 like this and flutter like this has a lot of like different build targets and things that are not like
0:05:03 relevant to working with flutter so if you’re not familiar with flutter it’s a way of working to
0:05:07 build multi-platform apps and so you can see it’s got like linux mac os web and all that stuff
0:05:12 but yeah like when you’re working in a repo like this you want to think through like what are the files that
0:05:16 are going through and if you’re using a coding agent like with cursor or whatever the first
0:05:20 thing they’re going to do when you ask a question is okay well let me go find what the user’s trying
0:05:25 to do let me search for files and pick those out and if you know what you’re doing with your code base
0:05:29 you tend to know like okay well i’m working on this button toolbar great so i’ll just like
0:05:33 clear the selection out and i’m just working on these these views here great so i’ve selected those
0:05:38 and that’s it so then i can see you know token use for those files it’s pretty small so i’m able to
0:05:44 just get to work type my prompt and paste that in here help me update all the docs pages so if i do
0:05:50 that and then i just do gemini flash quickly to show what that looks like so the context builder the way
0:05:56 that works is it will actually search for files using an llm based on the prompt that you’ve typed
0:06:00 out you know a big part of using repo prompt is that you have to know you know what it is that you’re
0:06:04 trying to select here right right and you know what i noticed a lot of users they were just putting
0:06:07 everything in they would just say like okay just select all and and that’d be it and you’d be
0:06:10 like okay we’ll get the first yeah i mean that’s the easy thing to do you’re like okay well there’s
0:06:14 the code base perfect but you know there’s plenty of tools that can just zip up your code base and
0:06:17 that’s easy but like the power of repo prompt is you can be selective you don’t have to select
0:06:22 everything so i can just hit replace here and then okay well what did that do okay well that actually
0:06:26 found all these files here that are related to my query put them in order of priority of like
0:06:30 importance based on what the llm’s judgment is and of course if you use gemini flash you’re not
0:06:34 going to get the best results compared to like using you know like a bigger model like gemini
0:06:39 2.5 pro but it’ll pick those out it’ll use something called code maps to help with that and you can see
0:06:45 the actual token file selection queries is just 6k tokens working with a code base if you’ve spent some
0:06:48 time you know programming in the past i know a lot of folks they’re not super familiar with all the
0:06:54 technicals there but like vibe coding yeah exactly exactly um so repo prompt has this code map feature
0:06:59 and what this will do is it will basically as you add files it’ll index them and extract what’s called
0:07:05 like um it’s a map but if you’ve used c++ before there’s like a header file and a cpp file and what
0:07:08 that is is basically you’re explaining to the compiler like what is all the definitions in this
0:07:12 file like you’ve got your functions you’ve got your variables and all that stuff and so it’s like a
0:07:17 high level extracted kind of an index like an index of your code base exactly yeah the context builder
0:07:21 uses that data to help you find what the relevant files are based on your query so it has like a kind
0:07:25 of peek inside the files without having all of the details and it’s able to kind of surface that
0:07:29 relevant information for you so that you can use that in a prompt one thing i love about
0:07:33 repo prompt so when i first started using it i had been like using just like a custom script i had
0:07:38 created to like take my code base and like and then like put you know the relevant you know context in
0:07:42 there which a lot of times i was just doing all of it i was literally putting all into a single file
0:07:47 and i’d copy and paste that into chat gbt yeah i think i tweet about this and someone told me like
0:07:51 oh you got to try repo prompt that’s when i tried repo prompt the fact that i could like see how
0:07:56 much context i was sharing yeah with the model was amazing and it seems like that’s super relevant too
0:07:59 because you know at least from the benchmarks i’ve seen you know everyone’s talking about how much
0:08:04 context you can put into their llm you know think of the benchmarks for llama for as soon as you went
0:08:10 over like 128k context like nowhere near the 10 million yeah like the quality just like dropped like
0:08:15 like a rock well until gemini 2.5 came out pretty much all the models you would really want to stay
0:08:21 below 32k tokens in general i find like over that you’re just losing a lot of intelligence so there’s this
0:08:25 concept of effective context you know the effective context window like at what point does the
0:08:29 intelligence stop being like as relevant for that model and for a lot of smaller models and local
0:08:34 models it’s a lot lower and you probably want to stay around 8k tokens but like for bigger models 32k
0:08:38 is a good number it’s only now with gemini that you’re able to kind of use the full package the full
0:08:43 context window but yeah so you’re using this context you’ve picked out your files say you you want
0:08:46 to use as many as you want 100k like what do you do with that so like you have a question like
0:08:54 um help me change how links are handled uh with my docs uh and so i have a question here i’m just
0:08:59 going to paste it to o3 and you’ll see like what is o3 getting out of this so it’s getting basically
0:09:04 this file tree so it’s getting a directory structure of this project it’s getting basically the high level
0:09:08 code maps of the files that i haven’t selected so basically when it’s set to complete everything that i
0:09:12 haven’t selected gets kind of shipped in and then you have the files that i did select and so then the
0:09:17 context is able to go ahead and is able to do that and so this is like a great way to kind of just get
0:09:21 this information into o3 get the most out of this model and o3 is an expensive model if you’re trying
0:09:26 to use it a lot like this is a great way to kind of get more value out of it move fast and get good
0:09:30 responses i think the average person like people who are just using chat tpt or even people who are
0:09:34 coding with cursor they don’t realize that you can do that that you can literally just copy and paste
0:09:40 all of that context in there and that the lm gets that and it understands what to do yes you know
0:09:44 in contrast to chat gpt claude is very good at following instructions like it’s the best model
0:09:48 at following instructions i find and i think this is another thing that repo prompt does quite well is
0:09:53 so it’s got like tools to kind of send information into the lm but it’s also got tools to go ahead
0:09:57 so it’s now it’s going to go ahead and write an xml plan and it’s going to create this theme selector
0:10:02 and it’s going to add these files and and change files for me and what’s cool with this is that i can
0:10:07 just go ahead and use claude with my subscription and then have it modify all these files so it’s
0:10:11 basically creating all these files and it can search and replace parts of files too so i don’t
0:10:15 have to re-update and re-upload the whole thing have it up with the complete code so a lot of models
0:10:19 struggle with you know people are noticing like oh this model is really lazy it’s not giving me the
0:10:23 whole code but like this kind of circumvents that issue because it lets the ai just kind of get an
0:10:26 escape patch and just do what it needs to do here right you know sometimes when i’m coding like this
0:10:31 i’ll iterate like so i pasted this question right with o3 and often what i’ll do is i’ll read through
0:10:37 the answer and then i’ll change my prompt and then paste again into a new chat and try and like see
0:10:41 where the result is different because basically i look at like here’s the output okay i actually don’t
0:10:46 care maybe about this copy link button okay then i’ll put specifically put a mention in my prompt to say
0:10:50 like let’s let’s kind of just focus on this part of the question and kind of reorient it and that’s the
0:10:54 nice thing with this is that i can just hit copy as many times i want if you’re paying for a pro sub
0:10:59 like there’s no cost to trying things there’s no cost to hitting paste again and you know you just try again
0:11:03 you just paste again let the model think again and try things and i think that’s like a really important
0:11:07 way of working with these models is to experiment and try things and and see how does changing the
0:11:11 context what files you have selected your prompt i use these stored prompts that come built in the app
0:11:15 so there’s the architect and engineer and these kind of help focus the model they give them roles
0:11:21 so like if i’m working on something complicated the architect prompt will kind of focus the model
0:11:26 on just the design and have it kind of not think about the code itself whereas the engineer is just the
0:11:30 code like don’t worry about the design just just kind of give me the code uh but just the things
0:11:34 that change maybe you should explain like when you say engineer prompt it’s literally you’re just adding
0:11:39 stuff that you copy and paste into the lm saying like you’re an expert engineer and this is what i
0:11:44 expect from you i expect for you to give me xml that’s your job do it and that’s literally how the lms
0:11:49 work like okay i’ll do it absolutely yeah giving them roles is is crucial telling them who they are
0:11:53 what their job description you know what what do i look for like giving them a performance review
0:11:57 evaluation uh all that stuff like i i find like the more detailed you are with your prompts the
0:12:01 more you can help like they kind of color the responses in an interesting way so just adding
0:12:05 the engineer prompt you see like it spent more time thinking about it so here this time it kind of said
0:12:09 okay this is the file tailwind here’s the change and this is the change that i’m going to do in a code
0:12:14 block so you know for the longest time before i had any of these xml features i was just kind of
0:12:18 using repo prompt and like getting these outputs and then just copying them back into my code base
0:12:22 manually and kind of reviewing them right i was like really the antithesis of vibe coding where
0:12:27 everything’s kind of automated yeah so i showed you a lot of stuff like pasting back seeing this xml
0:12:32 and then kind of putting it back in what’s really nice with repo prompts like chat flow is that all of
0:12:36 that is automated so if you want to vibe code and kind of think about it like just not think about
0:12:40 anything while being kind of cost effective too you can do that kind of work here and basically the way
0:12:47 this works here is i had gpt 4.1 as my main model this is all the context i gave it and then my pro
0:12:53 edit mode what it’ll do is it’ll actually ask a second model to apply the edits so i have gemini flash
0:12:58 that will go ahead and rewrite the file for me and just kind of do that work so i don’t have to manually
0:13:03 kind of incorporate these so if i was looking at here like okay this is the tailwind file i’d have to
0:13:08 open that up and then go ahead and introduce it in but having it kind of just go in the chat having
0:13:12 different models kind of do that work you know it makes a big difference working on repo prompt it’s
0:13:16 really like there’s building your context that’s like the biggest thing just picking what you want
0:13:20 you want to front load that work and you know in contrast to using agents you’re going to have those
0:13:25 agents kind of run off do a lot of work call a bunch of tool calls you see like oh three kind of
0:13:29 thought for 15 seconds thought through some tools to call it didn’t really make sense it just kind of
0:13:33 kept going and and ended up doing this and if you’ve used cursor a lot you know you’ll see like often
0:13:37 using oh three it’ll call tools that will like read this file read that file read this file
0:13:42 but if you just give it the files up front and you just kind of send it off to work with your
0:13:45 prompt you right away you get a response and you’re like okay well does this make sense to me am i able
0:13:49 to use this instead of letting it kind of serve for an hour yeah it’s a little bit more work at least
0:13:54 right now but it’s yeah i think you get a lot better results so it’s yeah yeah just front loading that
0:13:58 context being able to think through and iterate on that and that’s the whole philosophy around it is
0:14:03 just like thinking through like making this easy the context builder helps you find that context
0:14:08 you know eventually i’m going to add mcp support so you can query documentation find find things
0:14:14 related to your query as well and just spend time as an engineer sitting through what do i want the
0:14:18 llm to know and then what do i want it to do and then make that flow as quick and as painless as
0:14:22 possible and like that’s kind of everything and i think you know going forward and you know as you get
0:14:27 serious coding with ai like that’s what the human’s job is in this loop as engineer’s job is
0:14:30 figuring out the context i think that’s the new software engineering job
0:14:36 hey we’ll be right back to the show but first i’m going to tell you about another podcast i know
0:14:41 you’re going to love it’s called marketing against the grain it’s hosted by kip bodner and kieran
0:14:46 flanagan and it’s brought to you by the hubspot podcast network the audio destination for business
0:14:50 professionals if you want to know what’s happening now in marketing especially how to use ai
0:14:56 marketing this is the podcast for you kip and kieran share their marketing expertise unfiltered
0:15:01 in the details the truth and like nobody else will tell it to you they recently had a great episode
0:15:09 called using chat tbt 03 to plan our 2025 marketing campaign it was full of like actual insights as well
0:15:16 as just things i had not thought of about how to apply ai to marketing i highly suggest you check it out
0:15:20 listen to marketing against the grain wherever you get your podcasts
0:15:26 like i said before i was so surprised a lot of people haven’t talked about this because like
0:15:31 for me like right now cursor is good for like something very simple like okay change some
0:15:37 buttons or change some links or change whatever you know but anything complicated repo prompt i got like
0:15:44 way way better results so i’m curious like you know have you ever thought about like this being used
0:15:48 for things outside of coding and do you think would be useful for anything outside of coding yeah i mean
0:15:52 i’ve gotten academics reach out to me telling me they’re using it for their work uh there’s folks
0:15:57 in different fields for sure i think some of the ux has to probably improve a little bit but in general
0:16:02 like you know if you’re working with plain text files um you know repo prompt can service those use
0:16:07 cases for sure it’s all set up to read any kind of file and then apply edits to any kind of file too
0:16:12 like i don’t differentiate if i can read it then i’ll apply edits for you and i think a whole bunch of work
0:16:16 is around just like gathering context and kind of iterating on stuff like even you know in doing
0:16:21 legal work i do think you know a flow that is still missing from this app it’s just that like
0:16:25 kind of collaborative nature i think there’s still some work that needs to kind of be done to kind of
0:16:29 make this a more collaborative tool make this a tool that that kind of syncs a little bit better with
0:16:33 different things like for now like developers use git and like that’s that kind of collaboration
0:16:38 bedrock but i think like lawyers need other things yeah yeah that’s something i think too is like yeah
0:16:43 repo prompts super useful but you have to be a little bit more advanced like an average vibe coder
0:16:48 the average person using an llm and uh yeah you know no offense you can kind of tell one person has
0:16:52 built this you know it’s amazing but you can tell yeah yeah yeah no it’s all good i’m kind of curious
0:16:56 like why did you not go the vc route where’s repo prompt at right now like where is it now and what’s
0:17:00 your plan for it you know i’ve had a lot of folks you know bring that up to me and they’re kind of
0:17:04 thinking through like you know why not vc or whatever and i think it’s not something that the
0:17:10 door’s closed on forever it’s just i think right now it’s it’s work i’m able to build and you know i’m
0:17:14 able to kind of listen to my users and pay attention to what they need and i think it’s just
0:17:20 not super clear to me like where this all goes you know like this is an app that is like super useful
0:17:25 and it’s like helping me and i’m able to build it but like is it something that necessarily makes sense
0:17:29 to like have like you know a hundred million dollars invested into it to grow a huge team to like
0:17:34 build maybe i don’t know but like you know i want to kind of take things as they go as well and
0:17:38 you know right now i’m able to to monetize it a bit you know it’s got some passionate users
0:17:43 you know it’s working well this way but again like it’s all new you know to me like i’ve not
0:17:47 gone through this whole you know vc story myself i’ve had friends who kind of shy me away from it but
0:17:51 you know i i try to like listen to the folks around me too and see where yeah there’s pluses and minuses
0:17:56 to vc like you’ll hear on twitter and things like that like people who are like oh vc is horrible or oh
0:18:01 it’s amazing you know there’s good and bad to all of it yeah you know i feel like everything with ai
0:18:04 right now is like who knows what’s going to happen like yeah in a year everything could be
0:18:11 different in five years who the hell knows right yeah like right now because ai is such a big wave
0:18:15 that’s why we call the show the next wave right it’s such a large wave of transformation happening
0:18:21 that you are going to see the largest investments ever yeah i think in history yeah as well as the
0:18:25 largest acquisitions ever yeah and i think these are have yet to come yeah we’re like in the early
0:18:30 part of this transition i think the best two routes for you in my opinion would be either to go
0:18:35 really big and go the vc route or to go more like hey who knows what’s going to happen with it i just
0:18:39 want to like get my name out there and i can leverage my name for something else in the future
0:18:43 and like open source it that’s my kind of thought on strategically what i would do it’s like either go
0:18:48 really big or open source it and make it free and just put it out there and say yeah you know and get
0:18:52 some reputation benefit from it there is a free tier it’s not open source yeah but there is a feature you
0:18:56 know the thing about open source actually is something i’ve thought about a lot and the big issue with it
0:19:01 right now especially as people are building ai tools is that like it’s never been easier to fork
0:19:07 a project and kind of go off and just build it as a competitor if you’ve looked at client like client’s
0:19:11 a big tool you know that came around actually started around a similar time as me working on repo prompt
0:19:16 and uh if you’re not familiar the client is an ai agent that sits in vs code and it’s pretty cool but
0:19:21 the thing that is not so cool about it is that it eats your tokens for lunch like that thing will
0:19:25 churn through your wallet like faster than any other tool that exists just because it goes off
0:19:30 and reads files stuffs the context as big as possible so a lot of people really enjoy using
0:19:33 it because it has good results for certain things but yeah that cost is very high but the thing that
0:19:37 i was trying to bring up with this is that like so client was actually forked a few months ago by
0:19:41 another team of developers and it’s called bruise the alternative and if you look at open router
0:19:45 and some stats like bruise actually surpassing client and so you know that fork is now overtaking the
0:19:49 original and you know that’s the kind of space that we’re in where like different teams will kind of
0:19:52 take your code take it in their direction and then all of a sudden they’ve overtaken you and
0:19:57 you know you kind of lose track of you know where things are going there so like it’s a crazy space
0:20:01 it’s never been easier to open pull requests with ai you don’t need to understand the code you’re like
0:20:05 oh i have this open source project i’m just going to fork it and add my features and kind of go and
0:20:10 and it’s a tricky thing but like you know having a free version and kind of trying to ship and grow a
0:20:15 community of users who are passionate who like you can talk back to you and you know i mean that’s kind of
0:20:18 the route i’ve taken right now and it’s kind of been working so far i was in beta for a long time
0:20:22 yeah you know it’s still new figuring out where to go next with it and it’s mac only right now is that
0:20:26 correct yeah that’s true it’s mac only and i think a part of that is that i started off you know just
0:20:30 kind of trying to think about like you know how do i build this in a good way and the problem is like
0:20:35 i immediately ran into issues trying to build for different platforms and like i spent a bunch of time
0:20:40 debugging just getting svg icon rendering you know all these little things that are just like rabbit holes
0:20:44 and you’re like okay well you’re so abstracted from the base of like what’s happening and you spend a lot of time
0:20:49 just solving build issues that it’s like well i’m just gonna go ahead and do build native and just run with it
0:20:54 and have better performance doing so like you know if you open an ide like vs code you open up like a huge repo
0:21:00 what actually happens is that it’ll load the file tree and it will just kind of lazy load everything
0:21:04 like not everything needs to load because if you’re opening an ide you know as a coder traditionally
0:21:08 you only have a couple files open at a time maybe you have a dozen right you’re not going to be
0:21:13 processing 50 000 files at the same time but an ai model can you know if you give it to gemini like
0:21:17 gemini will want all those files it will want as much as you can give it because they can read all
0:21:23 of it and so you need a tool that is built different that is kind of organized in a way where it’s kind
0:21:29 of thinking first through that performance of mass data processing that you need to kind of do it’s a
0:21:33 whole different way of working that’s why it’s native because like i want that performance processing all
0:21:38 these files there’s all this concurrency happening where you’re like in parallel editing these files
0:21:42 like processing them and doing all this stuff like it’s very hard to do if you’re just you know using
0:21:45 javascript or typescript when i use repo prompt it seems like you’ve done a really great job of building
0:21:51 it it works really well it is all just you like right now yeah it is just me yeah i’ve been working
0:21:56 on it a lot yeah that’s crazy yeah it’s come a long way i iterated a lot on it you know but that’s
0:21:59 the power of dogfooding too like if you’re not feeling like folks listening dogfooding is when you like
0:22:04 kind of use your own product to iterate on it and build with it and you kind of make it a habit of
0:22:08 making sure that you’re a number one user of your app you know your own product to make sure that you
0:22:13 see all the stuff that sucks about it and for the longest time like you know it’s really sucked and
0:22:18 just that struggle and that that pain of using it and forcing yourself to feel that pain like that’s
0:22:22 what makes it good that’s where you’re able to kind of feel those things that the users using the
0:22:26 app will feel and and that’s when you end up with something that is great in the end so where do you
0:22:31 think repo prompt is going like long term which maybe now maybe long term now means like one year
0:22:35 where’s repo prompt going next year that’s long term it’s hard to say honestly like it’s weird you know
0:22:40 like in december like open ai announces oh three and they’re like oh it beats all the arc agi tests
0:22:44 and you’re like well is this agi like what is this like and then it kind of shifts and it’s like okay i
0:22:50 mean like it’s a better model it lies to you it’s not like uh the messiah you know right so it’s hard to
0:22:55 say like i don’t know like where we go like i have ideas on like where the future is one year from now
0:22:59 i think i’ll have to adapt this product and keep iterating on it to kind of stay relevant so it’s
0:23:04 going to keep changing but like i think that the flow i’m kind of pushing towards of that context
0:23:09 building i think that remains relevant for a while longer and what improves is the layers of automation
0:23:14 around that work yeah so i think like long term i still think that is kind of the vibe that i want
0:23:19 to go towards though i think just like integrating mcp just embracing that like universality of all
0:23:23 of these different tools so for folks listening if they’re not sure what is mcp is another acronym
0:23:29 we got lots of an ai so the idea there is traditionally if you use like claude or open ai they have tools
0:23:32 and those tools you know one of them could be like search the web or one of them could be like read the
0:23:37 files on your thing or look up documentation or these kinds of things and there’s this protocol mcp
0:23:43 that like creates like an abstraction layer so that any client app can implement this protocol
0:23:48 and then users can bring their own tools so if a user comes in and says like oh i want to use and
0:23:51 there’s this new one that’s really cool it’s called context seven where basically they’ve gone ahead
0:23:56 built a server that fetches the latest documentation for whatever programming language you’re using and
0:23:59 we’ll kind of pull that in as context so you can say okay great fetch the latest angular docs or
0:24:03 whatever docs you care about and then you can bring that in so that kind of work where you’re like
0:24:08 doing that context retrieval that’s super important or like stripe has one too where basically all the
0:24:12 docs for their tool is set up and you know you just plug in the stripe mcp and then
0:24:15 all of a sudden if you’re trying to vibe code your way through integrating stripe like that’s
0:24:20 super easy that the work is kind of handled you can plug in your api keys onto it so it can even talk
0:24:24 to the back end for you that whole work is kind of automated so it’s all about having tools for
0:24:28 folks using these models to kind of automate connecting to different services in this like
0:24:32 universe of all these different you know services that exist in the world yeah i kind of think of it
0:24:36 most i mean it’s different than xml but for me i think of it as almost more just kind of xml is like
0:24:41 the information language that ai can understand mcp is like the same thing with any service you want to
0:24:46 use or tool it knows how for the ai to know how to work with those things yeah and funny enough you
0:24:50 mention xml because that’s actually one of the things that i do a lot with rebomb is parsing xml
0:24:54 and i think one strength there that i have that like a lot of other tools are kind of ignoring
0:24:58 so traditionally when you’re working with these language models as a developer and you can see
0:25:03 this if you use chat you’d be like hey like um search the web it’s going to use the search tool
0:25:08 and you’ll see it say you call tool search and it’ll go through but what happens when it’s doing that
0:25:15 is that basically it calls that tool it stops waits for the result and then continues i think a bit like
0:25:20 the robot is kind of being reboot as a new session with that new context because basically every tool
0:25:25 call is a new query so you’re giving back the old information but you’re not necessarily talking to
0:25:29 that same instance it’s like a different ai instance that is answering your question from the
0:25:33 new checkpoint so like that’s like a weird thing so you know as you’re making all of these tool calls if
0:25:37 you use cursor you know it’ll make like 100 tool calls but by the end of it you know you’ve gone
0:25:41 through 25 different instances of these models and then you get a result at the end and you’re like
0:25:45 well you know it’s like weird like what actually happened you know there’s some data loss like weird
0:25:48 stuff you know we don’t know how this is yeah it doesn’t seem like that could create like reliability
0:25:52 issues right because like you know the lms like sometimes they give you amazing results and other
0:25:56 times yeah it’s like oh what is this and so every time you’re doing a new tool it sounds like you’re
0:26:00 almost recreating the chance of it going wrong in a way exactly yeah you’re you’re aggregating these
0:26:04 issues but you don’t even know where that info there could be different servers that are actually
0:26:08 processing all these different tool calls and yeah it’s weird sometimes you’ll have like oh that server
0:26:12 has some like chip issue on its memory and like that actually causes some weird issues where
0:26:16 claude is actually really dumb today um but on the other one it’s it’s a lot smarter because
0:26:20 their chip the memory chip is working fine you know you don’t know right so that kind of thing so
0:26:23 just to close that back you know what i’m doing yeah the way that i’ve kind of gone about this is
0:26:29 the way i call tools is you have your xml and the ai will just answer in one instance and it’ll
0:26:32 just give you the whole thing and it can call a bunch of tools in there it can be like hey like i
0:26:37 want to call this this do this and this and then i just parse that and then bulk call the tools
0:26:41 and then get the results and then we go another instance with the results and you can kind of
0:26:45 go back and forth like that so like not have to wait on each single one you’re actually just
0:26:49 bulk sending them out getting that data it’s a lot more efficient you’re able to process say like 25
0:26:54 queries you know get 2325 we’ll bring them all in you know let’s work from there and see how it goes
0:26:57 and so that kind of thinking so i think there’s a lot to kind of play with in terms of you know how
0:27:01 you’re even getting this data back and forth from the llms because at the end of the day it’s all text
0:27:05 you know text and images maybe um some video in some cases but like really text just for your coding
0:27:09 like that’s that’s the thing that you’re working with and you can do a lot with text manipulating
0:27:14 it and playing with it to kind of get good output so what do you think i’ve heard you know yc and
0:27:18 others i think gary tan said that i can’t remember what’s 80 but i think he said like 80 of the code
0:27:23 for the the startups going through yc right now is ai generated that number could be wrong do you think
0:27:28 in three years from now do we still have like normal engineers who don’t use ai at all is that a real
0:27:32 thing do you still have the holdouts well first of all like i think saying a percent like that of how
0:27:37 much of it is ai generated it’s a bit misleading to be hyped too yeah like i can go ahead and like
0:27:42 every line of code i could basically like type it in pseudocode to the ai model and like have it paste
0:27:47 it back in as like a fleshed out javascript function and say 100 of my code is written by ai
0:27:52 it really depends on how your workflow is what your pipeline looks like i do think fundamentally the job
0:27:56 of an engineer has changed it’s already done it’s already completely different like you can’t you can’t
0:28:00 work the same way but it depends on what point in the stack you’re working on like i i work for some folks
0:28:05 who do some like really low level you know graphics work and i talked to someone about like how they can’t
0:28:09 really use ai models because the ai models just hallucinates everything like it’s just not trained
0:28:14 on anything that they work on so it’s just useless for them but then if you work out someone who’s a
0:28:18 you know web developer well 100 of the code like like 98 of the training code is web code and web
0:28:23 framework code and so it’s like okay well yeah 100 of that work can be done by ai it’s really easy
0:28:28 so it really depends on where you are in the stack what kind of tool you’re working with and you know
0:28:33 how well the ai models can help you in that but i think like as we move forward more and more you’re
0:28:37 going to want to have ai models thinking through hard problems for you because it just happens much
0:28:41 faster as they get better at math at solving like you know connectivity architecture like architecture
0:28:47 something that like these 03 and 01 pro and hopefully 03 pro is just excel at they’re they’re very good
0:28:52 at finding good ways of like organizing your code and helping you plan how you connect things together
0:28:56 and i think that’s a big part of software engineering in general is just organizing your code because the
0:29:00 actual process of writing it like you know that’s not the fun part or even the interesting part it’s
0:29:05 that part of organizing and and i think a human job with this is to like iterate on those plans
0:29:09 iterate on these ideas because that’s like the kernel of what an ai will generate code with yeah so i
0:29:13 think that’s where the work is you know i used to open the editor like when i’m working on repo prompt
0:29:19 i don’t write a ton of code by hand like most of it is done by ai but like i spent a lot of time
0:29:24 thinking about architecture i spent a lot of time thinking about problems and debugging and thinking
0:29:28 through like i won’t just like hit the button and say like solve my problems fix my bug like that’s just
0:29:32 not helpful but like if i read through the code i’ll be like okay like i i think there’s like a
0:29:36 data race going on over here this part connecting to this part like there’s some concurrency issue
0:29:41 i’ll add some logs okay great i’ve got some idea of like what’s going on here perfect then you can
0:29:45 kind of feed that data into the ai model and have it kind of think through you know a resolution and often
0:29:50 once you’ve done those steps of troubleshooting the ai model can solve your problems but you have to sit
0:29:55 down and think through how things are connected and understand what is actually happening so i think
0:30:00 that’s kind of where that work changes that’s a great uh engineering answer yeah i’m looking for
0:30:04 the thing that goes viral on x right that you know like yeah yeah all engineers will be gone next year
0:30:10 this kind of thing you know listen the job is fully changed i think from today on like if you’re not
0:30:13 using these tools you’re not learning how they work like i think that’s like an issue because like i don’t
0:30:17 think you know a traditional engineer who spends his whole career just typing code up like that doesn’t
0:30:22 exist anymore but what does exist is someone who understands code and who can read it and who
0:30:26 understands you know what questions to ask and if you’re prompting like about the code if you
0:30:30 understand you know the connections that’s where you’re going to get the best results and that’s why
0:30:34 like a tool like repo prompt is so helpful because you’re able to do that and feed the right context in
0:30:39 but if you’re just saying like make the button blue or like move it over here i mean that works for to some
0:30:43 extent you know if as long as your instructions are simple enough and you know what you want you can get
0:30:47 there but like at a certain point you fall off and you know that’s when it stops working and maybe that
0:30:51 point where you fall off gets further and further as the models improve but i don’t think that like in
0:30:56 the next 10 years we get to a point where that point stops existing uh one thing that we didn’t talk
0:31:00 about that i was kind of curious to talk about was like what do you do at unity so what i do there is
0:31:05 i’ve been doing uh kind of xr research and xr engineering and so i work on a toolkit called the xr
0:31:11 interaction toolkit and basically it’s a framework for developers to build interactions with xr so if
0:31:16 you’re if you’re putting on an oculus quest or you’re a hololens or you know like uh apple vision
0:31:20 pro you want to basically interact with objects in your scene in your world you know like in ar if
0:31:24 you’re walking up and you want to pick up a virtual cube like how do you process that interaction of
0:31:28 you grabbing the cube and picking it up and looking at it so that’s like i’ve done a lot of research on
0:31:32 that that interaction of like input like i’ve written specs that are adopted for like the industry
0:31:37 in terms of hand interaction so like you know just tracking your hands how do you grasp something what
0:31:40 should you be doing if you want to poke a button that’s like not there like what does that look
0:31:44 like so that kind of stuff that’s that’s what i do there that’s amazing it’s like that’s a really
0:31:49 complicated engineering work how are you doing that doing a repo prompt and then you have a baby like
0:31:56 yeah how are you doing all this i mean i don’t have a lot of free time obviously i uh yeah yeah but
0:32:01 i’m passionate about what i do at work too and and then repo prompts you know this is my other baby
0:32:04 and i just think a big part of it is just you know when folks come to me and there’s like
0:32:09 something that’s like bugging them about the app you know i i just get like an itch and i have to
0:32:14 fix it for them yeah so like i just keep tricking on it and but i try to get some sleep in so i don’t
0:32:17 cut through that too much one thing i was thinking about too is like i have a son 11 year old you’ve
0:32:22 got a baby yeah this actually one reason i even like you know helped start this podcast was i’m
0:32:26 constantly thinking about where ai is going and wanting to stay ahead yeah and also think about what
0:32:31 does it mean for me and my family like quite honestly you know the selfish level and people used to
0:32:34 ask me like when my son was born because he was born in san francisco around tons of like
0:32:39 founders and vcs all the kind of people to be around like the birthday parties right it was all
0:32:43 people from like yc and people like that and it’d be asking me like you know what do you think your
0:32:48 son should do in the future what will his job be you know this is like 11 years ago and i was talking
0:32:52 about drones like he probably needs to be like a drone defense engineer like building anti-drone
0:32:57 systems or something it would be my common line that i would say at parties but now with ai like
0:33:02 because at that point we did not know ai would advance as fast as it has no it’s just happened
0:33:05 so fast right it was all just like a some stuff out of a book it was like oh yeah sure they’re
0:33:10 talking at stanford and they got some cool demos but like nothing’s working yeah now it’s working so
0:33:14 like with your child have you thought about that yet like oh of course what do you think they should
0:33:20 learn i have no idea yeah i have no idea it’s everyone right like like what do you even teach your
0:33:25 children like is it is it important to learn to code is it we teach them logic morals probably all of this
0:33:30 in more yeah flexible and super fluid i think so you know but it is funny on that topic i look at
0:33:36 engineers coming out and learning to code with ai around and i think they’re at a disadvantage you
0:33:40 know it’s unfortunate that like you know if you’re starting to code today and you have ai to lean on
0:33:44 you just don’t have that struggle you just don’t have the pain that like i had to go through when i
0:33:48 started to code when you know when engineers who’ve been in the field for so long that had to struggle
0:33:53 and not get the dopamine hit of a fixed problem right away like to study it and understand how it
0:33:57 works like that just doesn’t exist anymore because the ai just solves it for you and i think that’s
0:34:01 true in the code but it’s going to be more and more true in every field and so i think like there’s
0:34:06 going to be a need for people to have the restraint to kind of put aside these tools to struggle a little
0:34:12 bit i think there’s a ton of value in kind of using them to learn and grow but there’s also like that
0:34:17 restraint that you need to form to kind of have the struggle because that’s where the learning is and
0:34:21 it’s really tricky and i and i don’t know how you you solve that now because it’s it’s too easy not to
0:34:26 struggle now which which is a big problem yeah i’ve heard uh jonathan blow and if you know of him
0:34:31 of course yeah the game designer he talks about exactly what you’re saying that you know it’s in
0:34:35 the future like yeah sure ai could get amazing at coding in the future but it’s also going to create
0:34:39 issue where like just like you said people are not going to learn to properly code he was already
0:34:45 complaining about before ai the code was and then now with ai it’s like okay now we’re kind of
0:34:48 screwed i guess because like we’re gonna have a situation where like no one knows what’s going on
0:34:54 and like yeah you’re entirely dependent on the ai for everything yeah it’s a crutch so easy to reach
0:34:59 for and what do humans do but that’s the thing i think maybe that’s the middle part you know where
0:35:02 where we’re at this point where it’s like the ai is just not quite good enough to kind of solve
0:35:06 all the problems and you still have problems to solve and you still have people that need to kind
0:35:10 of work with the machines to kind of figure out how to go maybe at some point in the future
0:35:14 all of it is moot i know some folks think that and maybe it doesn’t matter but i think you know
0:35:17 there’s going to be some discomfort in the middle where you know the machines are not quite good
0:35:22 enough to solve every problem we lean on them as if they are and then you know we’re kind of atrophying
0:35:26 a lot of skills that we we’ve heard you know i haven’t driven in a tesla with fsd but i’ve heard
0:35:30 folks say the same thing there where like if they’re using it all the time they actually like suck at
0:35:34 driving without it and it’s like right you know like more and more that’s going to kind of be a thing
0:35:38 where we’re like you’re that that is the thing where we start to go like to like we’re like
0:35:42 almost living in like one of those sci-fi novels right like everything being super super safe you
0:35:46 know i live in japan everything you used to live in san francisco everything’s super safe in japan
0:35:51 and there’s one reason i like it but you do lose some freedom in that yeah but do i really want my son
0:35:57 you know driving now like if i really think about it there’s an alternative um yeah not necessarily you
0:36:01 know i agree i mean i have that same debate with my wife you know was saying like i don’t think our
0:36:05 daughter is gonna ever have a driver’s license and she’s like i don’t think so you know like we’ll see but
0:36:09 i don’t know like there is the safety part for sure and i think that’s like really interesting and
0:36:15 and hopefully like that is the case that like ai just does make it safer out yeah right so eric it’s
0:36:19 been awesome and uh maybe we should you know tell people where they can find you and uh where they
0:36:27 can find repo prompt and yeah so i’m uh puncher with a v on x so it’s like pvn ch er on x uh and
0:36:30 most most socialists my handle all over so you can reach out there my dms are open if you have
0:36:36 questions and repo prompts uh repo prompt.com so you can just head over there and uh find the app
0:36:40 free to download and uh nice discord community too if you want to hop over there and send me some
0:36:44 messages and tell me what you think like please do yeah thanks for having me on nathan it’s been
0:36:48 great chatting with you yeah appreciate it it’s been great yeah yeah yeah had a lot of fun cheers
0:36:50 likewise take care all right

0:00:06 Hey, welcome to the Next Way Podcast. I’m Matt Wolf. And today we’re talking with the
0:00:12 founder of Mindstream, a daily AI newsletter. And well, talking to him, I learned that he
0:00:19 hates making video. So he became an expert on all of the various AI avatar tools to help
0:00:24 him create videos. So in this episode, we’re going to dive down the rabbit hole of AI avatars,
0:00:28 how to use them, how to create them, how to make them the most effective you possibly can.
0:00:32 It’s an amazing episode. So let’s go ahead and dive in with Adam Biddlecomb.
0:00:38 Hey, we’ll be back to the pod in just a minute. But first, I wanted to tell you about something
0:00:43 very exciting happening at HubSpot. It’s no secret in business that the faster you can pivot,
0:00:47 the more successful you’ll be. And with how fast AI is changing everything we do,
0:00:53 you need tools that actually deliver for you in record time. Enter HubSpot’s spring spotlight,
0:00:57 where we just dropped hundreds of updates that are completely changing the game.
0:01:02 We’re talking breeze agents that use AI to do in minutes what used to take days,
0:01:08 workspaces that bring everything you need into one view, and marketing hub features that use AI to
0:01:15 find your perfect audience. What used to take weeks now happens in seconds. And that changes everything.
0:01:20 This isn’t just about moving fast. It’s about moving fast in the right direction. Visit
0:01:27 hubspot.com forward slash spotlight and transform how your business grows starting today.
0:01:31 Thanks for joining me today, Adam. I’m excited to dive in. How are you doing?
0:01:36 Yeah, I’m good. Thanks so much for having me on. I’ve been looking forward to this for a while.
0:01:41 And yeah, I’m really interested in these AI avatars. I’m someone who creates like a lot of content for
0:01:44 social. I’ve been building my following specifically on LinkedIn for a couple of years.
0:01:49 And dude, I just hate doing video. So anything that can like stop me setting up,
0:01:54 sitting down in front of a camera, I’m all in. Yeah, yeah. I mean, I really love the concept of
0:01:59 sort of making an AI avatar who can do, you know, maybe some of the short form video for you. But I’ve
0:02:05 been really, really sort of scared to do it myself because of how the audience might react, which I’m
0:02:10 sure we’ll get into some of that conversation a little bit deeper in. But maybe let’s start with
0:02:16 just sort of the landscape, like what tools are available out there to build this kind of stuff.
0:02:21 And maybe we’ll get into some of the pros and cons of each. Yeah, 100%. So there’s really two big
0:02:26 players, which is HeyGen and Synthesia. I kind of think of these as maybe like the ChatGPT and Claw.
0:02:31 They’re the ones that have a lot of funding, a lot of support behind them, a lot of usage. And they’re
0:02:35 kind of like, you know, generalist tools as such, like a lot of people are using them for all of
0:02:40 the different use cases. And I’ve spent most of my time playing around with HeyGen. At HubSpot,
0:02:45 we have some support from HeyGen. So I can get like that little bit of extra love that isn’t maybe
0:02:49 available to everyone else. But then there are some other tools. I’ve been playing around with Argyle
0:02:54 recently, which is specifically made more for people creating this short form content. So they’ve got a
0:02:59 bit more of like editing built in. And it’s just like very, very quick and easy to use. Whereas HeyGen and
0:03:01 Synthesia, there might be a little bit of a learning curve.
0:03:05 Gotcha. Like HeyGen, when they first came out, they actually were called something else. And then
0:03:12 they rebranded. But Synthesia was actually the first one that I came across early on and was like
0:03:17 super impressed by it. And the way they were sort of angling these things, like originally when they
0:03:23 first launched was this like marketing tool where you can make customized videos to your audience.
0:03:30 So let’s say somebody joins your email newsletter, you can actually email them and a personalized video
0:03:33 would be talking to you that would say like, Hey Adam, thank you so much for joining the Mindstream
0:03:38 newsletter. I really appreciate you joining. Here’s some things you can expect from us. And it would
0:03:43 like actually personalize that video. That was the use case that we’re pitching, but the use case that
0:03:51 it’s sort of evolved into has been more of this short form content, like AI avatar thing, which has been
0:03:56 really, really interesting to watch. You know, we were talking before we hit record about Rowan Chung,
0:04:01 who’s been doing a lot of this kind of stuff. And he kind of modeled that from Varun Maya.
0:04:06 And then we started to see a whole bunch of other channels pop up that are doing this kind of thing.
0:04:12 And it sort of blows my mind, like how well they do, right? Like I’m so impressed that they’re not
0:04:17 getting pushback or people going, Oh, this is gross. It’s an AI character, you know?
0:04:22 Yeah. Rowan Chung’s Instagram is insane. And he posted a really great case study on it that
0:04:28 really kind of inspired me. He spoke about like how many followers the account grew to 50,000 followers
0:04:32 in time of posting 7 million views. I think he’s like more than double that amount of followers now.
0:04:37 And it’s all exclusively these AI avatar videos. And generally what you find with these pieces of content,
0:04:43 if you haven’t seen them, is you’ll get kind of like 20% of the video will be the avatar speaking
0:04:48 to camera. And then 80% will be B-roll. Right. And I think part of the reason with that is if you’re
0:04:53 seeing the avatar in like quick cuts, it isn’t really on screen enough to kind of maybe put you off or make
0:04:59 you realize that it is an AI avatar because the fidelity is getting pretty good, but it’s not yet
0:05:06 kind of human. It’s maybe like 85%. Yeah. Um, so yeah, a lot of B-roll and a lot of cuts to kind of
0:05:10 get it to that level where the video is really captivating. Yeah. Yeah. When I first saw the
0:05:16 Varun Maya doing it, the first few videos I saw, I did not even realize they were AI. He actually had
0:05:21 to point out like, that’s actually not me speaking. That’s actually AI. There’s, I believe he used like
0:05:25 11 labs to do the voice, to make it sound like him. And it was, Hey Jen, to do the actual video.
0:05:29 And then Reed Hoffman, I don’t know if you’re familiar with, uh, you know, he was the founder
0:05:35 of LinkedIn. He actually made like AI virtual read and he does like interviews with himself.
0:05:40 And originally he was using some sort of like custom model, but it got to a point where Hey Jen got so
0:05:47 good. But now the whole virtual read is also just Hey Jen. It’s pretty crazy. That video was insane.
0:05:51 It was a full kind of like studio setup with him sat down, speaking to himself. When that came out,
0:05:55 that really blew my mind. Yeah. And you see these kinds of use cases out there, like with these
0:06:00 sort of leaders in the space. And like, you do know that they’re working very closely with the
0:06:04 technology leaders to get it to that level. Yeah. Yeah. They may even have access to like
0:06:07 a tier that might not be totally public yet. Some of these guys.
0:06:12 Yeah, I think so. What I’m really interested in. And I think a lot of people in this space is like,
0:06:18 what can we get easily? Like that’s where it’s going to become adopted by loads and loads and loads
0:06:23 of people. So like, where is the technology at with like a very, very basic input. Right.
0:06:26 And I’ve got some examples I can show you. Yeah. Yeah. I’d love that.
0:06:32 I’m going to show you like 10, 20 seconds of three models. So I’m going to start off with Hey Jen.
0:06:37 And they’re all a little bit different. But what we have with Hey Jen is the input recording that I’ve
0:06:41 used to train the avatar is the same across all of these three. So I recorded two and a half minutes.
0:06:45 The script was actually provided by Synthesia. And you just kind of like speak to camera and read
0:06:49 the script. And then I’ve trained the three models. This here is the Hey Jen test.
0:06:56 Do you think faking your Instagram birthday still works? Not anymore. Instagram has leveled up its AI
0:07:03 to spot underage users. Even if they lie about their age, it can read between the lines from a sweet 16
0:07:07 caption to user reports. Yeah, that’s really good. That was generally with Hey Jen.
0:07:12 Yeah. So the video is in Hey Jen. What I’ve done is I’ve actually uploaded my own audio,
0:07:18 like true audio recorded with into this microphone. So this, in my opinion, is one of the ways to get
0:07:24 like a real jump in the output. Hey Jen has this like really easily to do. I’ve tested the Hey Jen
0:07:30 audio, which I have elsewhere. But natively, the audio options from these tools, I don’t think are
0:07:34 great. You mentioned 11 labs earlier, they do kind of integrate with 11 labs. So you can go and train
0:07:40 a voice on 11 labs separately and like connect that in. But if you do have the time to record like a 60
0:07:45 second audio input, that’s probably where I’d go. And just for general context, we’re going to start
0:07:49 posting these clips like every day across the mainstream Instagram and LinkedIn actually starting
0:07:56 today. So this is what we’re going to be doing currently. So jumping over to Synthesia. Synthesia
0:08:02 is the same input, but in this case, we’re using their audio. Think faking your Insta birthday still
0:08:09 works? Not anymore. Instagram has leveled up its AI to spot underage users, even if they lie about their
0:08:15 age. It can read between the lines from a sweet 16 inches caption to user reports. And if it thinks
0:08:20 you’re under 16, you’re automatically placed in a teen account. So what I think is really interesting
0:08:26 is I actually think the lip syncing and the fidelity is better here, but maybe that’s because you’re using
0:08:28 their audio. Right.
0:08:33 Right. Whereas if you’re kind of uploading the audio separately, the model is kind of struggling to
0:08:38 maybe pick up your audio. Yeah. So you almost have this like trade-off of like, what’s more important,
0:08:43 the audio or the kind of lip syncing fidelity. Yeah. And it’s so funny. I remember when I was
0:08:48 listening to you speaking about Eleven Labs, you said that when you listen to your own voice, it sounds
0:08:53 rubbish. But when anyone else listens to it, they tell you it’s good. And I kind of think that it might
0:09:00 be the same with me critiquing the Adam AI avatar. And I probably need to give these to somebody else
0:09:04 to tell me which really is the better option. Yeah. Yeah. It’s a weird phenomenon that happens.
0:09:07 Yeah. With the Eleven Labs, I’m like, this doesn’t sound like me at all. And then all of the comments
0:09:12 like it sounds exactly like you. And when I watched this, I was really impressed with the voice that
0:09:16 came out of it. Yeah. I feel like maybe with Hey Jen, there was a little bit more motion in your
0:09:20 head where this one, it felt like you looked a little stiffer, you know, but the voice sounded
0:09:24 great to me. Yeah. And the other thing that I’ve learned kind of when playing around with these
0:09:30 models is the audio input, like when you’re using your own voice really dictates how the video output
0:09:36 comes out. So if you record like the audio input and be like really dynamic and energized, then the kind
0:09:39 of avatar will move around a little bit more. And if you speak a little bit slower and monotone,
0:09:44 it doesn’t give that level of dynamism. Right. Right. The third one I wanted to show you is
0:09:50 Argyle. And what I really liked about Argyle is I trained the model with the same input and then to
0:09:54 create the video, I put in the script. And then as you’re going through the creation process, it just
0:10:00 added subtitles for me. And it said to me, do you want to add B-roll? And I was like, sure. Okay. So
0:10:06 this output, I have done no editing. I have not like checked it or anything. I’ve just accepted the
0:10:11 subtitles, accepted the B-roll. And when you talk about speed to execution, this is really exciting.
0:10:12 Cool. Let’s check it out.
0:10:19 Think faking your Insta birthday still works? Not anymore. Instagram has leveled up its AI to spot
0:10:24 underage users. Even if they lie about their age, it can read between the lines from a sweet 16 caption
0:10:31 I think this is awesome. I think to get that level of output from just putting a script in, like it’s
0:10:36 reading the script, it’s finding B-roll that’s relevant. And like the B-roll is very relevant.
0:10:41 It’s added that music in. Obviously, this is all adjustable. But to go from like idea to posting
0:10:47 something in this format with no editor researching your own B-roll or whatever, I think this is a
0:10:50 pretty cool kind of like one-stop solution for this specific use case.
0:10:57 With the Argyle, is it its own custom model or is it using like a Haygen or Synthesia on the back end?
0:10:57 Do you know?
0:11:02 I don’t actually know. I should look that up. But yeah, it’s an interesting thing you raised,
0:11:06 because I think at the start of this, I mentioned how like, you know, I think a Haygen and Synthesia is
0:11:12 like the comparable to like ChatGPT, etc. I do think that like a lot of these kind of wrapper companies
0:11:17 are going to pop up with like specific use cases. I’ve got a friend who’s running one specifically for
0:11:22 coaching. So they’ve got like the avatar thing built in, but then they’ve also got in the software,
0:11:26 like all of the marketing tools that you need and like booking appointments and all of these things
0:11:31 specifically so you can create an AI coach of yourself. So yeah, I do think that’s interesting.
0:11:32 Go on.
0:11:36 Now, I’m curious, when you showed me the first one, the Haygen one, you said that you actually
0:11:42 recorded your own voice into it. Is there a benefit to doing that versus, you know, just flipping on the
0:11:47 camera and, you know, recording your voice to the camera? Because, you know, at the end of the day,
0:11:51 it’s probably the same amount of time that you put into like actually recording your voice
0:11:55 to feed it to Haygen versus just flipping on the camera and recording into a camera.
0:12:02 I think for myself, if I want to record a video, even if I use something like a teleprompter,
0:12:06 I personally just find it quite hard. Like first, I’ve got to like turn all my lights on and everything,
0:12:11 get my camera set up, and then I’ve got to deliver and read to camera and be dynamic and all of these
0:12:17 things. And for me to record like a 60-second clip like that might take 30, 40 minutes. But if I have a 60-second
0:12:22 script read, I can read that in one take pretty dynamically. Maybe two. But I feel quite
0:12:27 comfortable just recording audio compared to video. That’s just me. Obviously, I’m speaking to
0:12:31 you, Matt, who’s been doing YouTube forever. So for you, it probably seems like second nature to just
0:12:36 click with camera and click record. But I think where this is going to be particularly interesting
0:12:39 is for people like myself who are not so used to recording videos.
0:12:44 You know, there’s a lot of these things that have gone like really viral. Like we mentioned
0:12:47 Varun Maia, we mentioned Rowan Chung. There was another one that you mentioned earlier.
0:12:48 Ruben Hasid, yeah.
0:12:52 We mentioned a few others. I think maybe it’d be cool to show off some of these videos
0:12:56 that have gone viral so people could kind of see what sort of results others are getting with these.
0:12:59 A hundred percent. So here’s one of Rowan’s.
0:13:04 This AI can turn any photo into a 3D world you can explore. World Labs, founded by the godmother
0:13:10 of AI, Fei-Fei Li, created a system that transforms regular images into interactive 3D environments.
0:13:14 Their system lets you step inside the picture and look around as if you were really there.
0:13:20 You can also add real-time visual effects, change depth of field, and experiment with camera effects
0:13:20 like dolly zooms.
0:13:25 Very cool. Yeah, that one looks like he pretty much did the whole thing generatively.
0:13:28 You know, it sounded like the audio was probably like an 11 labs kind of audio.
0:13:32 And then he had all the B-roll. I don’t know if Rowan specifically
0:13:36 uses an AI tool that sort of sources the B-roll for him,
0:13:40 or if he does this sort of AI avatar, the whole video is shot,
0:13:45 and then he sends it to a team member who goes and finds B-roll to kind of hide any of the uncanniness, you know?
0:13:49 Yeah, the interesting thing is the fact that the avatar is in there for about two or three
0:13:56 seconds in the middle. What this really is, is like great short form storytelling video with B-roll.
0:14:01 And the avatar gives it that personal touch to kind of tie it back to like a personal brand,
0:14:06 or even if it was a business. Whereas like, you know, when I scroll through like my Instagram or
0:14:11 TikTok or whatever, there’s so many like great AI generated videos that don’t have any personality.
0:14:16 The recent viral ones I’ve seen, have you seen those like, you wake up as a France in 1800s?
0:14:23 Yeah. So like people are out there making like great AI videos, but here is a way to kind of take
0:14:27 these AI videos and give a little bit of personality towards it. And I think the interesting thing as
0:14:32 well is, you know, the specific use case we’re talking about is kind of me expanding and leveraging
0:14:38 my personal brand. But I think there’s opportunity for people here to create net new personal brands
0:14:42 that are completely fabricated, completely AI. Right. You can go and design an avatar around
0:14:46 someone and then kind of add that as an extra element to these like faceless YouTube channels
0:14:52 that you hear about. Yeah. Yeah. So you can almost be sort of anonymous yourself, but create a character
0:14:56 that’s out into the world that people kind of assume is a real character. I mean, we’ve actually been
0:15:02 seeing that quite a bit with the whole Instagram AI influencer thing, which, you know, I have very mixed
0:15:07 feelings about it. Right. I don’t totally get it, but a lot of people are doing, you know,
0:15:12 they’re really successful with these AI avatars and like having Instagram accounts with hundreds of
0:15:17 thousands of followers. And the character isn’t even a real person, which, you know, kind of blows my mind
0:15:21 still. It’s insane. Are you aware of little Michaela? I think she’s like the most famous one.
0:15:26 That was probably like the original, right? That has kind of like the genesis of it.
0:15:31 Yes. I was doing a bit of research. I think she launched in like 2016. Yeah. It was a startup
0:15:38 called Brood. Trevor McFedry’s and Sarah Deku, they developed and managed her persona, social media
0:15:44 presence, brand collaborations. And then this company was acquired in 2022. The crazy thing is,
0:15:51 is this little Michaela earns $10 million per year from brand partnerships. And she has some insane
0:15:57 collaborations, like with BMW of all companies, which shows that like real brands are kind of
0:16:03 willing to work with these AI personas. And I think the thing that’s interesting is that we talk about
0:16:09 this being launched in like 2016. This was pretty technologically revolutionary at the time, really,
0:16:14 like this is not easy to do. But now with some of these tools, like anyone can do this for like 50
0:16:21 bucks a month and go and create these personas. I think this AI influencer industry thing is really
0:16:26 going to expand. And yeah, I don’t really know what that means for like, you know, the future.
0:16:32 It’s quite scary. Yeah. I don’t know. I think there’s sort of like a generational thing, right?
0:16:37 Like I feel like, you know, at my age, like I don’t totally get it, but I feel like I might be sort of
0:16:41 like aged out of it, right? Because a lot of like younger generations are really into the whole like
0:16:47 character AI thing and, you know, chatting with fictional characters on character AI. And that seems to be a
0:16:52 popular thing. Obviously, a lot of these Instagram AI influencers get a lot of followers. For whatever
0:16:57 reason, it doesn’t click in my brain. Like when it comes to social media, I like to connect with other
0:17:02 humans. But I also, again, I think it’s a generational thing. I think as younger generations sort of grow up
0:17:07 with this kind of technology sort of being natively in their lives, it’s just going to become more and
0:17:12 more normal. I don’t know how I feel about that. Yeah, it’s very strange. I was kind of looking
0:17:17 through her Instagram earlier and this one video like really blew my mind. She was reviewing a
0:17:23 skincare brand. So it’s this digital avatar, like putting this kind of like skincare, like
0:17:26 makeup stuff on and be like, this is great. It’s going to make your skin really like clean and
0:17:31 whatever. And you kind of think the people who are watching that ad to go like, yes, I want to buy
0:17:36 that thing now. Like surely you need to see it on a human and see kind of, you know, before or after.
0:17:41 It’s better now. It’s all digital. Yeah. It’s so weird. So we were talking about Rowan earlier
0:17:46 and you mentioned he put out a report and in the report, he did comment about like how it was
0:17:51 perceived. What were some of the like things that were talked about? Because, you know, like we sort
0:17:57 of mentioned in the beginning, my sort of biggest worry around doing that kind of thing, a short form
0:18:01 video is, is just how it’s going to be perceived. The funny thing is I watch Rowan’s videos and I watch
0:18:06 Varun’s videos and I don’t really think twice about it, but I’m worried about doing it myself
0:18:11 because I’ve sort of put myself out there. I haven’t been an AI avatar. If I start doing it,
0:18:16 are people going to be like, oh, he’s starting to go the lazy route or whatever. You know, I don’t know
0:18:20 how it’ll be perceived. And that’s what worries me about it. Yeah. It’s wild. Like Rowan put out this
0:18:26 tweet and maybe we can link it below, but the kind of crux of it is he says like nobody cared. Like he
0:18:32 didn’t have any negative sentiment. And I think that is partly due to the AI thing,
0:18:36 but also partly because of the kind of quality of the work he did. I think if he was to start posting
0:18:41 like AI avatar content and like the B-roll wasn’t as kind of relevant and researched and all of those
0:18:47 things, like we said, and the script wasn’t so good at storytelling, you know, I think people need good
0:18:53 content. That’s what people are like desperate for. And if you can deliver that with this technology,
0:18:56 technology, then I don’t think people are going to be too unhappy. And the other thing he said,
0:19:01 which I think is crazy is he believes that the avatar is better than he is on camera. So again,
0:19:05 I think he’s potentially quite similar to me. Don’t really want to do video. Don’t have a bunch of
0:19:09 experience with it. Let’s look at this technology. Whereas again, for you, I think it’d be quite a
0:19:13 leap. Yeah. You know, biggest AI YouTuber. He’s had enough of doing video.
0:19:19 Yeah. I mean, I know Rowan personally pretty well, and I can, I can attest to the fact he doesn’t want
0:19:23 to be on camera. We’ve tried to get him on this podcast. I’m calling him out right now. We’ve tried to get him on
0:19:27 this podcast a handful of times and he’s always been like, yeah, I don’t really want to be on
0:19:33 camera. So yeah, I know that’s kind of the case with him is he prefers to be behind the camera,
0:19:39 run his newsletter and put it out videos like that. So I think, you know, like the sort of next logical
0:19:46 discussion here is like outside of this sort of virtual influencer kind of concept, maybe we can
0:19:51 sort of rattle off some of the various like other ways to use kind of this technology. I already
0:19:55 mentioned one in the very beginning when this stuff first came out, sort of personalized
0:20:00 videos that look like you created this video specifically for the person that just opted in,
0:20:06 right? There’s APIs that can automatically, whenever you opt into an email list, feed the person’s name
0:20:11 into like a Hey Jen generator. Hey Jen generates the videos. And then, you know, whatever, 30 minutes
0:20:17 later, you get a welcome email with a personalized video to you, which is really, really going to
0:20:22 probably increase retention because people feel like, Oh my God, this guy just sent me a video.
0:20:26 Like, even if they know it’s AI, they’ll be like, I can’t believe, you know, they sent me this
0:20:31 personalized thing. So that’s like one other use case that I’ve seen, but you know, what are some
0:20:36 other sort of business implications? This is obviously a HubSpot podcast. They’re a B2B company,
0:20:39 like anybody that’s listening to this, how else can they use this stuff?
0:20:44 Yeah. Just one thing I’ll say on that personalization. Like if you think about this AI avatar,
0:20:48 like say, Hey Jen, as part of like a big tool stack, personalization is getting more and more
0:20:51 important. And like when you receive an email in your inbox, that doesn’t have a level of
0:20:57 personalization, it’s almost an insult to like today. I’m really excited to see people integrating
0:21:02 this tool with something like Clay. So like you get the personalization of like, Hey Matt,
0:21:07 as an introduction, but then it can also bring in like extra personalization. Like I’ve scraped your
0:21:12 LinkedIn and I know everything about you and I can make that personal message specifically to you.
0:21:17 I think when you start getting videos like that land in your inbox. And the other thing is I get
0:21:21 these sales emails quite often now where someone’s like, they’re offering me like, let’s say LinkedIn
0:21:27 personal branding services. And they’ve done like a 20 second loom of them scrolling up and down my
0:21:32 LinkedIn account while they’re kind of playing me a not very personalized thing over the top.
0:21:37 I think personalization in outreach where you were using an AI avatar to give the personalization of
0:21:41 yourself, but you’re integrating with tools like Clay to get a lot of information on someone and have
0:21:44 very, very targeted outreach. I think that’s super exciting.
0:21:49 Yeah, no, that’s definitely another thing. I mean, you’ve got these tools like, um, I think it’s
0:21:54 pronounced N8N or Natan. I don’t know, but it’s like an AI automation tool where you can sort of
0:21:59 start to connect all of these various tools together. So it could do things like that. Like when somebody
0:22:05 opts in, go scrape their LinkedIn, add it to my CRM, go, you know, look at their Twitter bio, add it to my CRM.
0:22:11 You know, do they have a website, go add that info to my CRM. And then it can pull all that data, feed it
0:22:17 into, you know, chat GPT or clot or Gemini or whatever, and write up like a personalized message
0:22:22 about them. Like, Hey, I know you’re into guitar and surfing. That’s really cool. I’m into that too,
0:22:27 whatever. Right. And send like a personalized email based on all of this information that it grabbed
0:22:32 and then even feed it into HeyJet and create a personalized video with all of that information.
0:22:38 And I mean, right now I feel like that stuff’s still kind of expensive and still pretty slow, but I mean,
0:22:43 it’s the worst it’s ever going to be, right? Like as the saying goes, it’s only going to get easier and faster.
0:22:48 Yeah, a hundred percent. I think when you look at like how things are changing across any type of content creation.
0:22:55 So we’ve talked about social media, like content creators, you could look at onboarding documents for your team.
0:23:00 You can look at sales outreach. You can look at weekly reports in your Slack. You can look at investor emails,
0:23:08 like any of these sorts of things. You’re kind of starting to see often like a little bit at the top of the email where you can listen to the email.
0:23:15 I think in the future, there’s the option where you can watch the email. And I think the nice thing about the democratizing of content creation,
0:23:22 let’s say it enables people to create content for others in the way that they like to consume it.
0:23:28 You know, someone who is a newsletter operator. Now they have a video podcast with Hey Jen. They also have an audio podcast.
0:23:35 They also have short form content they’ve created and you’re just able to kind of meet your audience where they want to be met.
0:23:44 I wonder how long it’s going to be before, you know, companies like Twitter or Facebook, Instagram allow sort of this extra personalization on the video, right?
0:23:49 You’re scrolling your Instagram feed and it’s like, Hey Matt, stop for a sec. You know, Hey Adam, stop for a sec.
0:23:53 Have you checked out this elderberry supplement, whatever. Right.
0:24:01 But it’s like, it personalizes and stops you based on like, um, you know, knowing your name and all the details these companies have on you.
0:24:09 I bet you it’s not far off. Although I do feel some of those companies may like test that and then sort of pull back on it as people get creeped out by it.
0:24:15 Yeah, I think so. The adoption curve is going to take a bit of time. I’m pretty sure you might know about this.
0:24:21 I’m pretty sure I saw that Instagram was testing translation, which is another thing that these avatars can do.
0:24:28 Um, so, you know, if you’re putting out content generically in English, like it’s going to automatically translate it into Spanish or whatever for different audiences.
0:24:30 So that’s almost like the first step of that coming out.
0:24:36 Yeah. Yeah. And not only is it translating it, I don’t know if the Instagram specifically is doing it, but I know Hey Jen does it.
0:24:43 It actually lip syncs it. So it looks like you’re actually saying it in that other language where like, you know, YouTube has native translation.
0:24:50 Now I don’t think it’s rolled out to everybody yet, but they have this native audio translation feature, but it doesn’t sync anything up.
0:24:59 It just sort of overdubs like a, a translated voice over to the rest of the video, but with like Hey Jen and Synthesia and some of these tools that are out now, it actually changes your lips.
0:25:03 So it looks like you’re saying it in that language. It doesn’t actually look like a dub anymore.
0:25:18 Yeah. A hundred percent. And I think if you are a big content production, whatever, you know, even looking at a company like HubSpot that produces all of this video content, being able to distribute that worldwide to like every market is a huge, a huge piece of leverage of this technology.
0:25:35 Yeah. I mean, even on YouTube, there was an interview that Colin and Samir did with Mark Zuckerberg. And then later in the interview, Mr. Beast jumped on. Right. And when Mr. Beast was on this interview, he was telling Mark Zuckerberg that only about 30% of his audience speaks English.
0:25:50 And the reason he’s not putting more focus on Facebook is because Facebook doesn’t have that native translation and YouTube does. And, you know, when he puts it on YouTube again, 30% of people are listening in English. The other 70% are completely different languages.
0:25:57 So it’s like, if he puts it on Facebook, he’s like missing out on potentially 70% of the people that could watch his videos.
0:26:06 That’s insane. Mr. Beast is like so hot on this translation thing. He’s been looking at it for a while and he like has a separate company or part of his company that does this, this translation.
0:26:15 And he’ll like hire movie stars in the countries that he wants to distribute to kind of be him on these videos, which I think is so smart.
0:26:16 Yeah. Yeah.
0:26:21 The thing we should talk about is the negative use case, the possibility of scamming of scammers with this technology.
0:26:31 I was listening to an episode that you did recently where you’re talking about audio and scamming and like how, you know, you can personalize someone’s voice and call their mother or grandma or whatever.
0:26:42 I actually had this moment like a couple of days ago, I was trying to set up a new Google account for a new business I was starting and I tried my number for the 2FA and it was like, no, you’ve got too many Google emails with this number.
0:26:53 You can’t use it. I tried my partners, the same thing. So then I sent a 2FA code to my mom and text her saying, Hey mom, can you give me that code? And then I realized like, Oh wow, this is the kind of scam workflow.
0:27:01 So I sent my mom a picture of myself and said, it’s actually me. Send me the code. She sent me the code. That was just a selfie that I took in that moment.
0:27:08 You know, imagine if someone kind of got an access to a photo of me from anywhere, then they have that. That’s kind of how easy it is to get someone to send you a code.
0:27:16 But they don’t even need to get a selfie. They could probably generate one. If there’s enough images of you online to train in, you could probably, people will be able to generate them of you.
0:27:32 A hundred percent. So when you think about that and then you can have my voice perfectly emulated and then someone can also have my likeness personally emulated, sending a selfie video to my family member saying, I’ve just crashed my car. Can you send me some money? Like this is a pretty scary place we’re going towards.
0:27:44 Yeah. And I mean, we’ve already seen it in like scam ads as well. You know, we mentioned Mr. Beast, Joe Rogan apparently has had it happen to him. It’s kind of popped up quite a bit where, you know, they’ll start running Facebook ads, Instagram ads, Twitter ads.
0:27:57 And it looks like Joe Rogan or Mr. Beast or one of these big names is actually promoting a product, but they just used an AI tool, trained those person’s voice into it, and then generated an ad as those people.
0:28:13 I don’t think that’s going away anytime soon. I think a lot of these companies are going to try to figure out how to put guardrails up against it, but it’s going to be a constant cat and mouse game, right? Like whenever new guardrails come up, the people that are trying to do this stuff are just going to figure out more sophisticated ways to get around it.
0:28:22 But yeah, so we’re just kind of entering this world where you’ve got to be really careful. You know, you’ve got to know that this exists and kind of question everything almost, right?
0:28:32 Like you mentioned with the scenario with your mom, like I told my parents, if you ever get a call from me saying I’m in trouble and need money or something like that, ask for this passcode, right?
0:28:46 And then this is how you can confirm that it’s actually me is I will give you this word. If they can’t give you this word back, then you know, it’s not really me and it’s possibly a scam because look, I’m somebody who puts my likeness online a lot.
0:28:58 I’ve got thousands of hours of video, thousands of hours of audio. I’m probably one of the easier targets for some of this kind of stuff, but you know, it’s only going to get easier and easier for people to kind of do that sort of thing.
0:29:12 A hundred percent. And that’s a physical word that you’ve said to your mom in person or like written down somewhere. That’s really smart. I haven’t heard that before, but that is potentially one of the only ways to do it in this new world. We’re looking towards the other thing that’s interesting.
0:29:31 You mentioned there these kind of like fake and disingenuous ads, but I do think that these avatars are really interesting for UGC ads. There are a couple of companies that I noted down, Arcads and Creatify that are kind of specifically for these use cases where you can go in and use like hundreds of their UGC creators.
0:29:41 They’re kind of preloaded. I was looking through earlier and like, you know, you click through and it will be John and then you can have John sat laying in a hammock or sat in a podcast studio or like lying on his bed or whatever.
0:29:46 And, you know, you can put in scripts and then B-roll and you can have them kind of holding your product.
0:29:58 So I think when you look at like speed to market of testing and validating ideas, you can get pretty decent AI ads and test them without having to hire an actual UGC creator and ship products to people and all these sorts of things.
0:30:04 Yeah. Yeah. I was looking at Arcads the other day. I thought it was interesting that they actually used real actors, right?
0:30:09 They didn’t go and create like AI characters and now you can go and generate videos with those AI characters.
0:30:14 They actually got a whole bunch of actors to come in and actually do a bunch of training.
0:30:19 And then they use those actors inside, which if I was one of those actors, I don’t know, that would make me nervous, right?
0:30:23 Like you’re scrolling Twitter and you see yourself in a Viagra ad or something, you know?
0:30:27 That’s crazy. I didn’t, I didn’t realize they’d use real people to create their avatars.
0:30:39 That’s quite interesting that that has to tell you that it cannot create the likeness of a person as well as a real person, or at least you cannot create the likeness of a person to the level that an AI can mimic a person, which is quite interesting.
0:30:45 Yeah. Yeah. I think they had these people go and do a lot of the sort of reactions because if you look at Arcads, it’s a lot of reactions.
0:30:51 It’s people like pointing up or people doing like the shocked face or crying or, you know, laughing or that kind of thing.
0:31:00 And from what I understand, they brought in actors, had them do all of those things, and now you can prompt it and it will sort of mimic what they did.
0:31:02 But obviously, you know, they’re speaking involved and stuff like that.
0:31:14 So it’s sort of generating the reaction, but they actually have footage of those people doing those real reactions as well to sort of, you know, sort of fine tune it on those kinds of reactions.
0:31:17 That’s really smart. I’d love to know the deal that those UGC creators made.
0:31:22 Is it like a, is it a one-time fee that they got paid to kind of lease their likeness forever?
0:31:26 Are they getting like two cents every time someone makes an ad with their likeness?
0:31:28 Yeah, I have no clue.
0:31:42 I was actually reading a story the other day, though, about how a handful of Hollywood actors are all like really, really upset because they actually gave permission and gave people the ability to use their likeness in ads and things like that.
0:31:50 And now they’re all frustrated because they’re starting to see ads that they would have never actually approved being in spreading online.
0:31:55 So, yeah, it’s definitely a really weird and interesting world we’re entered into with this.
0:32:03 But I mean, the sort of like ethical use cases of making your own creators and making your own short form videos or creating ads for your own products.
0:32:10 To me, it’s really exciting, but, you know, you do have that sort of unethical counterbalance that needs to be figured out as well.
0:32:14 A hundred percent. I think that’s the case with kind of all of these new AI technologies.
0:32:19 The other thing I think is interesting is the idea of brands and creator-led brands.
0:32:27 And, you know, sometimes you will have a brand that kind of starts with a creator and then that, you know, creator will move on for whatever reason and the brand kind of gets left.
0:32:35 I think that there’s a world where we see kind of brands that create AI personas that become the creator for that brand.
0:32:37 But, you know, it’s not a real person. It’s forever licensed to the brand.
0:32:49 Like, even kind of like the Duolingo owl, like, is there a version of that where, you know, that owl is actually like an AI avatar, like creator type thing that’s doing all this, like, video content?
0:32:52 There’s obviously a range of, like, more and less human.
0:32:54 Well, like this Lil Mikaela example.
0:33:00 Like, if Lil Mikaela was to license herself forever to BMW and was forever the spokesperson to BMW,
0:33:06 you can kind of create these brands around creators and get the momentum of a creator-led brand,
0:33:09 but never, ever have the risk of that creator moving away from the brand.
0:33:15 Yeah, we were actually talking, we had Nikola from Wonder Studio on the show not too long ago.
0:33:23 And when we were talking to him, we were talking about this concept of, like, if you’re a brand, you can create your own, you know, Geico, Gecko, you know, Tony the Tiger,
0:33:30 you know, all of these companies that have, like, this mascot that’s not a real human, but people know the mascot, right?
0:33:35 Anybody can go and create that now and have that mascot do these ads for them.
0:33:41 You know, this was in the context of using Wonder Studio and actually creating, like, a 3D character model of it
0:33:44 and then putting them in a world using something like Wonder Studio.
0:33:49 But, yeah, I mean, like, anybody can go and do that now and have their own sort of mascot.
0:33:56 Like, for me, I can have, like, an animated cartoon wolf or something that pops up in my videos, and I own him forever.
0:33:57 That’s my IP.
0:33:59 He’s never going to, like, you know, go look for another job.
0:34:05 Yeah, and then when you think about where you can place that character, like, so, yeah, you pick up the phone to kind of, like, tell them you crashed your car
0:34:11 and it’s the Geico Gecko that answers you, you know, and you, like, do the chatbot on the website and, like, the avatar of them pops up
0:34:12 and it’s not just a chatbot.
0:34:14 It’s, like, the Gecko speaking to you, you know.
0:34:21 I think it’s a really smart opportunity for some brands to go out and really, really own this and put this, like, whatever it is, everywhere.
0:34:23 Like, all the touch points with this brand.
0:34:30 You can even imagine walking into a store, like, walking into the Apple store, and there’s just this, like, Apple, like, talking to you, like a hologram of an Apple.
0:34:38 The company, I think they’re called Hypervision, that are doing these holograms, which are similar to the avatars for, like, conference booths and, like, welcoming in stores.
0:34:41 So, I think that’s a kind of exciting tangent of it.
0:34:48 I went to a small meetup up in San Francisco a couple months ago, and as you walked in the door, it wasn’t a hologram.
0:34:51 It was, like, a giant flat screen TV that had, like, a camera on the top.
0:34:55 But as you walk in the door, it was like, hey, welcome into our store.
0:34:56 Oh, I love the shirt you’re wearing.
0:34:57 That plaid looks really good on you.
0:34:59 Oh, and you’ve got a nicely trimmed beard.
0:35:01 Thanks for joining us today, right?
0:35:05 And it was actually, like, commenting on your appearance as you walked in, right?
0:35:13 I can totally see stuff like that, either in hologram or, you know, in the beginning, maybe on just, like, big flat screen TVs that’s sort of interacting with you.
0:35:19 But it’s a character interacting with you and actually sort of giving feedback and actually responding to what it sees.
0:35:23 So, it’s actually specifically talking about you as you walk through the door.
0:35:24 100%.
0:35:27 It sounds insane, and it sounds like a Black Mirror episode.
0:35:32 But really, with the kind of, like, piecing these tools together, it could be, like, 6, 12 months away.
0:35:35 I think you’ll probably do it today, you know, if you wanted to.
0:35:37 I think it’s super exciting.
0:35:38 Yeah, yeah, for sure.
0:35:43 Well, is there any other avenues that we should travel down that we haven’t around some of these concepts?
0:35:47 I’ve got one more thing I can show if you like.
0:35:51 So, I’ve got a breakdown of the personal avatar compared to studio avatar done in HeyGen.
0:35:52 Okay, cool.
0:35:52 Yeah, yeah.
0:35:54 So, let’s start with the personal avatar first.
0:35:59 So, if anyone’s created an avatar with HeyGen or Synthesia before, you’ve probably created a personal avatar.
0:36:02 You kind of sit down, you can do it with your webcam, your phone.
0:36:07 You do a little 10-second recording of yourself saying, I give permission for Synthesia to make this.
0:36:10 And then, you’ll read a kind of two to three-minute script.
0:36:12 So, that’s the personal avatar.
0:36:19 So, this is, like, I think similar to the use cases that we’ve seen from Ruben previously and some of the tests I’ve done before.
0:36:23 DeepSeek R1, and it went viral overnight.
0:36:29 Built at a fraction of the cost, DeepSeek’s open-source model offers advanced math, coding, and reasoning skills.
0:36:31 Within days, it topped the app store charts.
0:36:33 So, that’s the idea of the personal avatar.
0:36:39 Now, the studio avatars, when I recorded this studio avatar, I went to a local studio.
0:36:50 I didn’t do it with HeyGen, but they gave us a very, very specific list where they said, use this camera, this light, stand in front of this type of green screen, have this type of microphone.
0:36:51 Like, it was very, very specific.
0:36:57 And we had very specific instructions on how to kind of speak and all of those sorts of things.
0:36:59 And the output of this one is pretty insane.
0:37:00 Let’s have a look.
0:37:08 DeepSeek, the Chinese AI startup, recently dropped its game-changing chatbot, DeepSeek R1, and it had gone viral overnight.
0:37:14 Built at a fraction of the cost, DeepSeek’s open-source model offers advanced math, coding, and reasoning skills.
0:37:17 Within days, it topped the app store charts.
0:37:18 So, there you go.
0:37:19 What do you make of the difference between those two, Matt?
0:37:21 Yeah, I mean, they look really good.
0:37:25 One thing I’ve noticed is that, you know, the second one, it had more of the full-body shot.
0:37:33 And one thing that I’ve seen, I think HeyGen is the one that does it, is they can actually do videos now of you, like, walking down the street.
0:37:39 And it sort of, like, changes your lips so it looks like you’re walking and talking to the camera, which sort of blows my mind.
0:37:42 But yeah, so you can record them full-body.
0:37:46 It’s funny, like, out of those two, the studio one was very expensive to record.
0:37:50 And it had a lot longer process, and we worked kind of more in collaboration with HeyGen.
0:38:02 But because you have the green screen, you can imagine that is amazing for, like, there’s so many use cases you can imagine that from, you know, changing the background all the time, presentations, you know, you could have someone kind of speaking over slides, like all of those sorts of things.
0:38:10 But for the specific use case that we’ve been working on, the short form video, the best background ever is the same background that I have when I’m on podcasts or whatever.
0:38:12 So that personal avatar actually works in that case.
0:38:17 I’m curious with the one that you showed, was the B-roll, did you go and do that, like, separately?
0:38:20 Or was that one of the tools that actually helped put the B-roll?
0:38:29 Because you had the, I don’t know what the sort of editing style is called, but when you have the words and it jumps between all the articles but stays focused on the same word, was that something that you, like, custom edited?
0:38:33 Or was that, like, an AI sort of function of one of these tools you’re using?
0:38:39 Yeah, so that one did run kind of, like, through an editor, and that is the workflow we’re working with now.
0:38:47 Obviously, we’re creating these videos on the Mindstream brand, which is owned by HubSpot, and we’ve got to be, like, really, really careful with what we put out and make sure it is the highest quality.
0:38:56 I haven’t spent a lot of time with, like, AI editing softwares, but I don’t know right now if we’re at the level that it kind of really can replace a full-time editor at the highest level.
0:38:56 Right.
0:39:01 In, like, kind of, like you say, matching all those and getting that really kind of, like, high level of video editing.
0:39:02 Yeah, yeah.
0:39:12 Descript actually put out a video about a new AI video agent that they’re calling the cursor for video editing, where you can basically say, hey, at, you know, this minute, go change this.
0:39:15 During this scene, add this B-roll into it.
0:39:16 At this scene, do this.
0:39:20 And you just keep on chatting with it, and it edits your video through natural language.
0:39:22 I mean, I haven’t gotten my hands on it.
0:39:26 I don’t know how effective it is, but the concept is really interesting to me.
0:39:28 Yeah, that’s really smart.
0:39:34 My editor has said to me that, like, Descript is his favorite, but it’s kind of like vibe editing, like we’re seeing this vibe coding now, right?
0:39:34 Yeah.
0:39:43 I think the magic of it with any of these things is if an AI tool can make it easier to do something, you still need to know what you’re trying to do to get there.
0:39:54 And someone who’s got a kind of 1,000 hours of video editing experience, put the AI tool in their hand, they’re going to get like a 10x output compared to myself who’s never edited videos.
0:39:56 And it’s like, oh, where do I even start?
0:40:04 Yeah, now that really, really solid editor can get you that edit back in three days instead of a week and a half, you know, but still just as quality.
0:40:11 You know, that’s what excites me about it is it sort of up levels the abilities of the people that actually know what they’re doing.
0:40:25 Like I’ve done a lot of the vibe coding stuff, but because I’m not that proficient of a coder, I know like a little bit, but because I’m not that proficient, I’ll run into bugs and I’ll sit there for like three hours trying to fix the smallest, tiniest bug.
0:40:30 That seems like it should be a simple fix, but the AI can’t figure out how to overcome that bug.
0:40:33 And because I don’t know enough about coding, I can’t get to where I want to go.
0:40:37 So, you know, vibe coding often turns into rage coding very, very quickly.
0:40:39 A hundred percent.
0:40:43 One more thing I’ll say that I found really interesting with these avatars.
0:40:48 And I do think it’s a little bit of a barrier when you’re just kind of like, you said the layman sitting down to make their first avatar.
0:40:52 When we created, we did the studio avatar first with Heygem.
0:40:55 We did that back in January and I’ve worked closely with them.
0:40:58 And recently we did a personal avatar.
0:40:59 So this is like the basic one.
0:41:05 But I did the recording with a Heygem kind of consultant on call with me.
0:41:09 So I went and did the two minute recording and he said, okay, now let’s change this thing.
0:41:14 And we did eight inputs until he said that input will create the best avatar.
0:41:19 And what that input is doesn’t really feel like it will create the best avatar.
0:41:23 You want to create an avatar that’s going to do kind of social media reels.
0:41:27 You’d expect the input to be like, hey guys, it’s Adam here.
0:41:29 And like, you know, do like the YouTube voice.
0:41:30 Right.
0:41:32 If you do that, it will break the avatar.
0:41:36 To get the best avatar output, you need to be the least human.
0:41:40 The take that Heygem, the kind of Heygem tech team were most happy with.
0:41:46 I was literally sitting there and going, hello, my name is Adam speaking really, really slowly.
0:41:53 And when you do a hand movement, they say like, bring it up, hold it for like three seconds and then take it down really slowly.
0:41:55 If you do this, it will break the model.
0:42:01 And so then you create this avatar in this really robotic way that is very, very unnatural.
0:42:08 And then when you kind of then do like the inputs, you know, I said earlier, it’s about the input audio dictates the kind of dynamism of the video.
0:42:16 And then you can, you know, with all these hand movements you do, you can kind of like set them as expressions and you can actually like retrofit them in to train the avatar.
0:42:21 But I think that’s really hard to communicate in like a kind of onboarding process.
0:42:27 Like, Hey guys, don’t record the avatar, how you want it to sound, record it like a robot to look human.
0:42:29 That’s interesting.
0:42:41 So, but you can still the output, you can still get something that looks excited and energetic, even though you train it on a sort of monotone, almost like boring sort of version of yourself.
0:42:41 Yeah.
0:42:44 And I’ll clarify, it’s not monotone or boring.
0:42:45 It’s very, very slow.
0:42:46 Okay.
0:42:46 Slow.
0:42:46 Okay.
0:42:47 Very slow.
0:42:57 And when you’re kind of doing your voice movement, if you’re doing like facial features or whatever, you kind of have to accentuate them and move slowly.
0:42:59 So it’s unnatural.
0:43:00 It’s very, very unnatural.
0:43:03 The kind of input you have to do.
0:43:09 And yeah, again, if you’re sitting down and you’re like, well, I want my avatar to be very, very professional or very dynamic or whatever.
0:43:15 That’s kind of all done in the edit, really not in the input recording of the avatar.
0:43:17 I think that’s a really, really good takeaway.
0:43:21 I’m glad you added that in because I think that’ll be like really, really helpful to people.
0:43:30 You know, one thing I mentioned sort of off recording is that I have issues because I have a beard and sometimes like the beard starts to look a little blurry and fuzzy around my lips as I start talking.
0:43:35 Or it looks like I have hair on my lips or, you know, something weird like that when I go to generate something.
0:43:39 But what you just described could be the solution to it, right?
0:43:42 I tend to get really excited and just talk really, really fast.
0:43:47 So when I go and train one of those models, I do it how I do it on video.
0:43:48 I talk fast.
0:43:48 I sound excited.
0:43:50 Maybe that’s the problem.
0:43:52 Maybe that’s what I’m doing wrong.
0:43:53 Potentially, potentially.
0:43:58 If you’re speaking kind of at the level you are now, I do think the model will struggle, but some would have been taught.
0:44:01 Another thing that’s obvious I haven’t mentioned, don’t ever cover your mouth.
0:44:03 So the way you’re sat now, you’re sat quite close to your microphone.
0:44:07 You know, if you did like kind of dip into that, that’s going to break it as well.
0:44:13 So yeah, like super slowly, you have to take long pauses kind of between the scripts they give you.
0:44:15 You also have to have a like home base.
0:44:18 So on the studio avatar, I was kind of holding my hands like that.
0:44:22 You know, I had to do that and move away from that, but always come back to that.
0:44:28 So, you know, if you don’t want your hands in shot, keep them out of shot, bring them into shot for a movement, but then take them back out.
0:44:33 You have to kind of have that home base that you always come back to, which will be like the home base of the avatar.
0:44:34 Right, right.
0:44:37 And same rules apply for either of the models that you’re using.
0:44:38 I believe so.
0:44:41 This, all this kind of learning I’ve got is from working with HeyGen.
0:44:45 I would assume it applies to Synthesia and Agile and all of them as well.
0:44:45 Awesome.
0:44:46 Well, very cool.
0:44:48 I think that’s super helpful for people.
0:44:50 So this has been super fascinating.
0:44:52 I really, really appreciate all the tips and insights.
0:45:01 And I think a lot of people are going to have a lot of ideas of how they can go and use this technology inside of their marketing, or if they want to go and become a creator or things like that.
0:45:05 If people want to go and learn more from you, learn more about what you’re up to, where can they go check you out?
0:45:07 So I’m very active on LinkedIn.
0:45:08 I post twice a day there.
0:45:09 It’s just my name, Adam Biddlecum.
0:45:12 But also, I’m still running the Mindstream newsletter.
0:45:14 It’s now owned by HubSpot.
0:45:18 So if you want kind of daily AI updates, make sure to subscribe to Mindstream.
0:45:18 Cool.
0:45:20 And where do they go to subscribe to it?
0:45:21 Mindstream.news.
0:45:23 You can tell I don’t do too many pods, can’t you?
0:45:26 This has been so much fun, Matt.
0:45:26 Thanks for having me on.
0:45:27 It’s been a blast.
0:45:28 Yeah, thanks for joining me.
0:45:36 And everybody who’s listening, if you enjoy this type of content, make sure you like this video and subscribe to this podcast because we’ve got more where that came from.
0:45:37 Thanks so much.
0:45:37 Thanks so much.

0:00:03 Hey, welcome to the Next Wave Podcast.
0:00:04 I’m Matt Wolfe.
0:00:06 I’m here with Nathan Lenz.
0:00:12 And today we’re going to try to do our best to demystify all of these recent launches with
0:00:19 OpenAI because they’ve released 4.5 and then 4.1 and then 04 and then 03.
0:00:21 And it’s just getting really, really confusing.
0:00:27 So this episode is designed to break it all down for you, tell you which each of these
0:00:32 models are good for, what they’re not good for, and we’ll even rank them from dumbest
0:00:35 to smartest at the end and tell you which ones to use for which things.
0:00:40 So without further ado, let’s just go ahead and jump in and talk about all these recent
0:00:40 OpenAI announcements.
0:00:47 I think probably the best place to start is with this one right here, because this is probably
0:00:52 the first piece of news in this sort of sprint that OpenAI is on of releasing news, which
0:00:56 is that they just rolled out this memory feature in OpenAI.
0:01:02 And to me, this is probably one of the coolest, most valuable features they’ve rolled out.
0:01:04 I found it really useful.
0:01:04 Yeah.
0:01:08 I know Nathan, before we hit record, you were saying something to the same effect of like,
0:01:10 out of everything, the memory is probably the coolest.
0:01:11 Yeah.
0:01:15 So it’s funny, the team at Grok, they reached out to me about what should they build, like
0:01:16 maybe like two months ago.
0:01:18 And I was like, focus on memory.
0:01:23 Nailed that first, because OpenAI has like a half-baked version of memory.
0:01:26 They’ll probably eventually get there, but who knows if it’s a priority or not.
0:01:28 If it’s not, you should get there first.
0:01:29 Right, right.
0:01:33 Unfortunately for Elon Musk and Grok, OpenAI definitely got there first.
0:01:35 And I think it’s going to be one of the biggest things long term.
0:01:38 It’s the kind of thing where people don’t realize why it’s important now.
0:01:40 But over time, it’s going to be clear why it’s important.
0:01:44 You know, I use ChatGPT a lot, like for so many different things, like personal stuff,
0:01:45 business, everything.
0:01:48 I use it for so many different things, probably more than the average person.
0:01:50 And I asked it to like, explain what do you know about me?
0:01:52 And I’m not going to show you what it said.
0:01:54 Something that was so revealing.
0:01:55 Was it not nice?
0:01:56 No, it was nice.
0:01:57 It was nice.
0:01:59 You know, because it knows a lot about my life too.
0:02:00 My life is pretty crazy.
0:02:05 And just like some of the insights that drew out of that were just kind of shocking and inspiring.
0:02:08 Honestly, it was like, yeah, I have done a lot of great things.
0:02:10 I have went through a lot of hard times in my life and all this stuff.
0:02:14 And it was just, it seemed to know things about what I want to accomplish in my life.
0:02:18 Things that I don’t think I’ve told it that were just kind of like, that’s me.
0:02:24 And so the idea that AIs are already starting to actually understand me maybe better than I understand myself.
0:02:24 Yeah.
0:02:26 And I mean, you’ve probably had a lot of chats.
0:02:33 I know you’ve mentioned that you’ve done even like voice chats where you go for like walks and just sort of like unload your thoughts into the voice chat.
0:02:35 And I mean, all of that is in memory now.
0:02:38 I mean, and just to be clear, you can turn off the memory.
0:02:41 If this feature freaks you out, there is an option to turn it off.
0:02:49 So like you don’t have to let it remember everything, but like, you know, all of those voice conversations where you’re just sort of like brain dumping.
0:02:53 That’s all in its context now when you chat with it.
0:02:54 Yeah.
0:02:57 I think long-term that’s how AI is going to get really useful for people.
0:03:04 Cause like people think of it right now, it’s just chat to you where you go in there and you copy and paste something in, maybe have it help you in some way, uh, or you just chat with it.
0:03:11 But eventually these models are going to understand maybe what you want or what you want to accomplish better than you yourself understand that.
0:03:13 Or even before you realize you want it.
0:03:14 Or before you do.
0:03:14 Right.
0:03:16 And this is like, this used to be like the dream.
0:03:19 Like when I first got involved in Silicon Valley, like we were all dreaming of this stuff.
0:03:21 We’re like, how do we make the web better?
0:03:23 Like, well, you got to make it more personalized while we actually understand people.
0:03:25 It’s like, how do you do that?
0:03:27 I guess machine learning or something.
0:03:28 And like, we didn’t know how to actually do it.
0:03:33 And people started doing it with like a very basic version of like, oh, well, Amazon’s kind of pulled off something like that.
0:03:36 Cause they kind of get what you want, but it’s not perfect.
0:03:37 Same as Netflix, not perfect.
0:03:42 Eventually all of this stuff, it’s just going to know about the kind of stuff you want to accomplish or do in your life.
0:03:46 And I think that’s where we’re going to get to a point where AI is going to be really helpful.
0:03:49 Anything you’re wanting to do, maybe it’s going to recommend it to you.
0:03:59 Beforehand, like you said, it’s going to know you or it’s going to just know the kind of stuff that you want to do and be able to help you better with more personalized help versus just some generic stuff that it says to everyone.
0:03:59 Yeah.
0:04:08 I mean, sort of random side topic, but since you mentioned Netflix, apparently Netflix is also going to be using OpenAI’s AI tech inside there.
0:04:12 So it will actually even better understand you and what movies to recommend.
0:04:13 But that makes sense.
0:04:17 Like I said, when I first got involved in Silicon Valley, everyone was trying to figure out how do you do that?
0:04:23 Cause it felt like the natural evolution of software products was to understand users and what they wanted better on a personal level.
0:04:25 And no one really nailed it.
0:04:33 Like I said, Amazon and Netflix got somewhat close and everyone’s just been like tweaking on what Amazon Netflix has been doing for like 10, 20 years now or something like that.
0:04:34 So yeah, it makes sense.
0:04:34 Yeah.
0:04:41 Well, I actually like the very first thing when this new memory feature rolled out was there was a button that says, do you want to try it?
0:04:42 And I said, yeah, sure.
0:04:42 Let’s try it.
0:04:45 And it automatically pasted in this prompt.
0:04:46 Describe me based on all our chats.
0:04:47 Make it catchy.
0:04:49 This is what it said about me.
0:04:50 I don’t mind sharing mine.
0:04:58 It says you’re Matt Wolf, a future forward creator, a high trailblazer, and the go-to tech Sherpa for over a half a million subscribers.
0:05:00 You’re not just riding the wave of innovation.
0:05:02 You’re helping build the surfboard.
0:05:07 Your YouTube channel is a hub for creators, entrepreneurs, and curious minds hungry for what’s next in AI and tech.
0:05:14 You break down the complex, spotlight the useful, and always bring the human angle all while rocking a retro futuristic vibe.
0:05:21 Whether it’s a short, punchy, real, or deep dive 20-minute banger, your goal is always the same.
0:05:24 Empower your audience to do more, create faster, and stay ahead.
0:05:29 Off camera, you’re a family guy with a white pup named Bowie, a deep love for camping,
0:05:32 and a soft spot for the San Diego Padres.
0:05:35 You’ve got a musician’s soul, a builder’s brain, and a storyteller’s heart.
0:05:39 Even when burnout hits or the algorithm plays rough, you keep showing up.
0:05:41 Driven by purpose, not just metrics.
0:05:43 You’re not just documenting the future.
0:05:45 You are part of shaping it.
0:05:48 I was pretty flattered by that, so I don’t mind sharing it.
0:05:49 Yeah, yeah.
0:05:53 I mean, like, parts of that were kind of generic, and then parts of it seemed like it was, you know, kind of personalized view.
0:06:00 I have a feeling I probably talked about more personal stuff with it because the stuff it said to me felt more personal than that.
0:06:00 Yeah.
0:06:06 And I mean, I also even gave it, like, a primer in the custom instructions, like, here’s who I am, and here’s, like, a general outline of my life.
0:06:08 Yeah, yours was probably like, and you keep refusing to get that wart checked out.
0:06:09 Go get it.
0:06:19 So, you know, I probably provided it a lot more about myself, but it said a lot of things, you know, it’s just, like, reading between the lines, but it said a lot of things that I did not tell it, though, like, exactly.
0:06:29 Like, it, like, kind of, like, understood things about me, about how I feel about myself, about my accomplishments, my failures, things I want to accomplish before I die, like, all these kind of things.
0:06:30 It was just kind of shocking.
0:06:32 And it was stuff you didn’t actually tell it?
0:06:34 It sort of figured it out?
0:06:34 Some of it.
0:06:35 Some of it.
0:06:39 And I was like, holy, you know, it’s, like, it’s connecting the dots.
0:06:45 It’s, like, understood my personality somewhat and the things that I get inspired by or that I want to do or that bother me or whatever.
0:06:45 Right.
0:06:52 But it definitely connected some dots and said some things that I did not explicitly tell it, which was just, holy crap, that’s amazing.
0:06:53 Yeah, yeah.
0:06:59 For me, this one feels, like, pretty on par with, like, the various discussions I’ve had, right?
0:07:02 I have, you know, talked about my YouTube channel a lot, right?
0:07:06 I use it a lot for, hey, I need an idea for this YouTube video.
0:07:07 Here’s the transcript.
0:07:08 Help me come up with a title.
0:07:09 All right.
0:07:11 Help me come up with some concepts for thumbnails, right?
0:07:14 So I go to it a lot for, like, ideation around my YouTube channel.
0:07:18 So it obviously knows a lot about the fact that I do stuff on YouTube.
0:07:18 Yeah.
0:07:25 It has all the transcripts of my YouTube videos because I always copy and paste them in and go, hey, how should I title this thing?
0:07:25 Right.
0:07:30 So a lot of this is based on probably transcripts from my YouTube videos.
0:07:30 Right.
0:07:36 It’s kind of random that, like, the one sort of personal thing it picked out about me is that I have a dog named Bowie.
0:07:36 Right.
0:07:40 Not that I’m, like, married with two kids or, you know, things like that.
0:07:41 Right.
0:07:43 But just I have a white dog named Bowie.
0:07:46 I’m like, I don’t actually remember telling you that, but it’s true.
0:07:49 So I’m sure it was in, like, one of my videos or something somewhere.
0:07:49 Right.
0:07:50 Yeah.
0:07:56 I mean, mine was more personal and, like, to the point where I didn’t cry, but it made me feel emotional, like, reading it.
0:07:57 And I was like, that is crazy.
0:07:58 And just imagine where that goes.
0:08:00 Like, we always say that’s the worst it’s ever going to be.
0:08:00 Right.
0:08:01 Right.
0:08:01 Right.
0:08:07 And the more you use it, the more it’s going to know about you and the better response you’re going to get on things like this.
0:08:11 We’re totally going to have the thing that, like, you know, in the future, when you pass away, you’ll have, like, a memory of who you were.
0:08:17 It won’t be perfect, but, God, it’ll probably be, like, a 90% of the essence of, like, who you were as a person.
0:08:21 And that is just shocking that, like, that technology is, like, basically, it’s almost there.
0:08:22 It’s, like, it’s right there.
0:08:24 It’s getting really close.
0:08:27 And, you know, this was sort of just the first announcement.
0:08:29 This announcement came out on April 10th.
0:08:29 Yeah.
0:08:30 Right.
0:08:33 So this was sort of, like, the first in a series of, like, almost every day.
0:08:33 Yeah.
0:08:36 The most important thing Grok could have built, they beat them to it.
0:08:37 Small little announcement.
0:08:39 And then on to the real things.
0:08:39 Yeah.
0:08:39 Yeah.
0:08:42 They didn’t even seem to make much of a big deal about it.
0:08:44 I don’t think, like, they even did a sort of live stream for this one.
0:08:47 Almost everything OpenAI does a live stream about in this one.
0:08:48 They didn’t even.
0:08:50 They’re just like, here, here’s a new feature that we rolled out.
0:08:53 Well, they’re talking about doing a social network and stuff now.
0:08:57 Which, if they do that, memory is a core feature of that of actually understanding who the people are.
0:08:57 Right?
0:08:59 That’s how you would build some kind of new social network.
0:09:03 Because it’d be highly customized based on the personality of the people in it.
0:09:03 Yeah.
0:09:07 I mean, I actually haven’t read too much into the whole social network thing.
0:09:10 I just pulled up an article from Reuters about it.
0:09:14 But, yeah, that’s another announcement that has sort of bubbled up.
0:09:19 Is that OpenAI is working on an X-like social media network, which is interesting, right?
0:09:25 Because there’s, like, a pretty big feud between Sam Altman and Elon Musk.
0:09:25 Yeah.
0:09:26 But it makes sense, though.
0:09:28 This is why I told the Gronk guys, like, that’s why they should do it.
0:09:30 It’s because they already have all that data.
0:09:30 Yeah.
0:09:32 About what people are sharing and how they’re interacting with people.
0:09:36 And so that would be their advantage over OpenAI is that data.
0:09:36 Yeah.
0:09:37 So what makes this one different?
0:09:39 Have you read into this one at all?
0:09:40 About what OpenAI is going to do?
0:09:41 Yeah.
0:09:43 Like, why would people use this over something like X?
0:09:45 It seems like it’s all rumors at this point.
0:09:46 I’m not sure if it’s actually even confirmed.
0:09:47 Yeah.
0:09:50 So it looks like it’s all sort of speculation at the moment.
0:09:50 Yeah.
0:09:51 Speculation.
0:09:53 I would assume that they wouldn’t be going after X.
0:09:55 They would be going after Facebook, if I had to guess.
0:09:59 Because Facebook is that kind of product where it’s all about, like, all your friends,
0:10:01 all these connections that you have, like, real life.
0:10:05 And I feel like OpenAI would be in a great spot to, like, revolutionize that kind of product
0:10:07 versus, like, competing with X.
0:10:08 Yeah.
0:10:08 I’m not sure.
0:10:10 I think there’s not a whole lot of details yet.
0:10:11 Yeah.
0:10:11 Super interesting.
0:10:14 We’ll kind of keep an eye on how that one plays out.
0:10:15 I’m sure Mark is, like, we’re super happy.
0:10:17 Like, his AI effort is totally floundering.
0:10:20 And then now Sam Altman may be coming to kill him.
0:10:22 I think it’s more likely to kill Facebook than X.
0:10:24 So, yeah, that’s a tough spot to be in.
0:10:24 Yeah, yeah.
0:10:29 So one of the other things that OpenAI has been doing is they’ve been releasing these new models,
0:10:31 but they’re also sunsetting a bunch of other models, right?
0:10:37 So apparently GPT-4 is going to be phased out on April 30th.
0:10:44 And then also GPT-4.5, which up until this week was, like, the newest model they released,
0:10:46 is also going to get phased out.
0:10:51 They sent out an email to all of the OpenAI API users, or maybe it was everybody in ChatGPD.
0:10:52 I don’t know.
0:10:59 But they sent out an email basically saying, we’re going to phase out 4.5 because 4.1 is better and faster.
0:11:02 Like, dude, OpenAI sucks at naming these things.
0:11:03 Yeah.
0:11:07 And they had said that they were going to solve that by doing GPT-5, which would be, like, the all-in-one,
0:11:12 the one, you know, the one model to rule them all kind of thing where, like, you just talk to the one model,
0:11:15 and then behind the scenes, it’s routing you to whatever is the best model probably is what they’re doing.
0:11:16 But they didn’t do it.
0:11:17 They, like, delayed it.
0:11:20 They’re like, no, sorry, we’re going to have naming problems for a few more months.
0:11:23 Like, give us, like, two or three more months, and then, you know, that’ll be solved.
0:11:27 So I kind of wonder, like, is 4.5, was it a failed run, or are they just finding that it’s so expensive
0:11:33 that people don’t know how to use it so it makes more sense to improve upon it with some kind of new reasoning model
0:11:37 or with the GPT-5 to have it part of that kind of stack of things that they might recommend?
0:11:41 Like, if you’re trying to write something, maybe behind the scenes you’re using 4.5.
0:11:45 But otherwise, it’s so costly to use it, why would you use it for the thing?
0:11:46 I don’t know.
0:11:48 I mean, 4.5 has been my favorite model.
0:11:53 When I’ve been using ChatGPT lately, I’ve been using 4.5 more than any other model.
0:11:53 Yeah.
0:11:56 But if I had to guess, I think it’s the cost, right?
0:12:01 Because the GPT-4.5, the cost of using it in the API was just, like, mind-blowing.
0:12:04 Like, nobody was using it because it was just way too expensive.
0:12:05 Yeah.
0:12:07 So I think cost was probably the biggest factor.
0:12:12 And supposedly, they rolled out 4.1 inside of the API.
0:12:14 So this is all super confusing, right?
0:12:21 Because 4.5 is available in ChatGPT, but 4.1 is not available in ChatGPT, but it is available in the API.
0:12:24 And they were pushing this, like, good for coding.
0:12:25 Yes.
0:12:27 I think they were pushing this one as the best one for coding.
0:12:33 Yeah, and so it’s kind of confusing, but, like, then if 0.3 is out, I assume it’s got to be better at coding.
0:12:35 So it’s, like, what’s the use case?
0:12:36 Or it’s confusing.
0:12:37 Yeah.
0:12:39 This one is available in the API.
0:12:41 This is GPT-4.1 in the API.
0:12:46 They released 4.1, 4.1 mini, and 4.1 nano.
0:12:49 They outperformed 4.0, 4.0 mini.
0:12:51 Let’s see.
0:12:55 Coding, it scores 54.6% on SWE bench verified.
0:13:00 And it’s an improvement over 4.0 and over 4.5.
0:13:05 So apparently it’s a lot better at coding than 4.5, but 4.5 wasn’t good at coding in the first place.
0:13:06 Right.
0:13:09 It felt more, like, human-like when you were chatting with it, I guess.
0:13:10 Yeah.
0:13:12 But it wasn’t really a model that was good with, like, logic and coding.
0:13:17 One thing I found interesting, though, is, like, yeah, 4.5 has been the best for writing, as far as I can tell.
0:13:21 It has the most personality, but they’ve just kept updating 4.0 behind the scenes, it feels like.
0:13:23 It felt like it just kept getting better and better.
0:13:29 It’s not as good as 4.5 in some ways, but ever since they released even, like, the image thing, it feels like they updated the model somehow.
0:13:30 Like, there’s something.
0:13:31 It’s smarter.
0:13:33 I’m not entirely sure.
0:13:35 You know, they haven’t really talked about that.
0:13:36 But, like, it just appears to be.
0:13:37 So I’m not sure what the takeaway is.
0:13:38 This is all super confusing.
0:13:40 It’s opening eyes fault.
0:13:44 I do have a little bit of a breakdown that we’ll get to once we’ve sort of covered all the announcements.
0:13:44 Yeah.
0:13:47 It sort of tries to organize it a little bit better.
0:13:56 But they did put out this graph, which is intelligence based on multilingual MMLU, massive multitask language understanding.
0:14:00 So this one’s more of, like, a general benchmark of just, like, intelligence.
0:14:06 So we can see that GPT 4.1 nano, the latency, that’s the speed, right?
0:14:09 So, like, how fast from the time you give it the prompt to get your response.
0:14:14 So GPT 4.1 nano is super fast, but not the smartest.
0:14:20 Mini is pretty smart, smarter and faster than 4.0 Mini.
0:14:27 And the new full GPT 4.1 is about the same speed as 4.0, but slightly smarter.
0:14:29 That’s what I’m making from that chart.
0:14:30 Oh, my God.
0:14:31 That’s super confusing.
0:14:34 So this is why they need the GPT 5.
0:14:35 It’s interesting.
0:14:38 They must have really realized, like, you do need different models for different use cases.
0:14:39 Yeah.
0:14:43 And so they’re just like, well, for now, let’s just keep giving them different crazy names.
0:14:45 But then we’ll have to make some kind of smart router.
0:14:47 So hopefully that comes soon.
0:14:47 Yeah.
0:14:55 Well, and again, I think, like, their ultimate goal is you enter a prompt and then it chooses for you the best model to use, right?
0:14:55 Yeah.
0:14:56 Well, that’s what GPT 5.
0:14:57 They’ve already said that’s what it’s going to be.
0:14:57 Okay.
0:14:57 Yeah, yeah.
0:14:59 That’s kind of the goal that they’re trying to get to.
0:14:59 Yeah.
0:15:04 So, you know, so Sweebench, this is basically how good it is at coding.
0:15:13 I believe this is the one where it goes on GitHub and tries to find, like, bugs that need to be fixed and how good is it at fixing them or something like that.
0:15:14 Yeah.
0:15:18 Given a code repository, an issue description must generate a patch to solve the issue.
0:15:22 So GPT 4.1 is way up here.
0:15:29 But one thing I’ve noticed is GPT and OpenAI has completely stopped comparing themselves to, like, external models.
0:15:32 Like, you notice how Gemini 2.5 is not on here.
0:15:35 Notice how Claude 3.7 is not on here.
0:15:40 Like, we don’t know how this compares based on this graph to models that aren’t OpenAI’s models.
0:15:40 Right.
0:15:41 Yeah.
0:15:42 And who knows with coding, too.
0:15:44 Like, things have changed a lot in the last month.
0:15:46 Like, all of a sudden, like, Google took over coding.
0:15:49 Like, Claude was, like, in a huge lead forever.
0:15:58 Then they released 3.7, which seemed good, but then kind of got overambitious because they would try to do tons of things for you and, like, seem to be making more mistakes than 3.5 did.
0:16:04 And then Google came out with, like, 2.5 Pro, their Gemini model, and that’s just been amazing.
0:16:05 Like, people haven’t talked about that enough.
0:16:08 Like, Google has taken over coding with AI in the last month.
0:16:12 I think most people probably don’t realize that, but, like, that has happened in the last month.
0:16:13 Yeah, no, that’s true.
0:16:15 And, I mean, I still do quite a bit of coding.
0:16:19 I’ve kind of been using WindSurf a little more than anything, a little more than Cursor, really.
0:16:25 And, yeah, I’ve been pretty much using Gemini 2.5 the most.
0:16:29 And every once in a while I’ll get hung up and I’ll go to Claude and be like, well, can Claude solve it?
0:16:32 And sometimes Claude can solve random stuff that Gemini can’t.
0:16:32 Yeah.
0:16:42 But I would say, like, 95% of my coding now is done with Gemini 2.5, which will be interesting because now OpenAI just released 03 and 04 Mini,
0:16:53 which this new 03 model is their most powerful model, but they also have 04, which is a bigger number than 03, but not as powerful as 03.
0:16:54 Right.
0:16:57 And, you know, Sam Allman had said a few things about these models.
0:17:04 Well, more recently, he said, the benchmarks you saw for 03, we found some ways to improve upon the model more than we anticipated.
0:17:11 So it’s actually better than what we showed, but also in the past, he hinted at 03 Pro being amazing.
0:17:15 And like you said, off camera, maybe 04 is 03 Pro.
0:17:15 Maybe.
0:17:15 Yeah.
0:17:16 I’m not sure.
0:17:16 Yeah.
0:17:16 Yeah.
0:17:20 Or maybe 03 Pro might be what they launch as GPT-5.
0:17:21 Who knows?
0:17:22 Yeah.
0:17:27 Or 03 Pro could be like in a week or two and it’s only for the professional, you know, users or something like that.
0:17:28 That makes sense.
0:17:28 Yeah.
0:17:32 This 03 is their current most powerful model that we have access to.
0:17:35 It’s available to both Pro and Plus users.
0:17:41 04 Mini, I believe, is a faster, less expensive model, but still a very impressive model.
0:17:42 Yeah.
0:17:44 We can kind of look at some of the charts that they show here.
0:17:50 So they actually had another chart in their video that they’re not showing here for some reason.
0:18:03 But you can see that like they’re 04 Mini with no tools scored 92.7 on competition math and they’re 03 with no tools scored 88.9.
0:18:08 So when it comes to math, I guess 04 Mini is better than 03.
0:18:09 Yeah.
0:18:21 But then they show this other chart in their launch video, which showed 03 with Python and it got like a 99.5% or something crazy like that on the competition math.
0:18:23 Basically like acing it.
0:18:23 Yeah.
0:18:25 That’s why they had to have something like GPT-5.
0:18:31 Like they’re going to have to have like one smart router to like, because all these models are good at different things.
0:18:34 And some of it, you discover it over time.
0:18:37 Like some of it, I’m just like, you learn that like, oh, this model is just slightly better for this one thing.
0:18:39 Yeah.
0:18:39 Yeah.
0:18:47 So we can see here that like 03 and 04 Mini, like when it comes to competition code here are like pretty equal.
0:18:59 And then down here, the PhD level science questions, 03 is actually better at that than 04 Mini, you know, but 04 Mini is better at math, but not as good at science.
0:19:02 How big is the jump in general from like 01 to 03?
0:19:06 So we’ve got the estimated inference cost on this axis here.
0:19:25 And then this is the math benchmark here and we can see 03 Mini, you know, is the lowest cost, but also didn’t score as well where you’ve got 04 Mini high on math here, pretty expensive comparatively, but also, you know, really smart.
0:19:25 Right.
0:19:36 And then we can sort of see it in comparison here to 03, which is not as good, but it’s not showing like the jump from like a one pro to 04, which is kind of frustrating.
0:19:37 Yeah.
0:19:41 Well, I mean, yeah, I think, I think 03 Pro be the more comparable there for a one pro.
0:19:43 I think they’ll probably come out of that soon.
0:19:49 But then here we can see when it comes to GPQA, what does that test for?
0:19:52 Graduate level Google proof.
0:19:58 So basically graduate level questions that you can’t just Google an answer for is essentially what that means.
0:20:08 So on that benchmark, 03 Mini actually performs about the same as 04 Mini high, which is interesting.
0:20:11 They both perform the same.
0:20:13 Wait, so the GPQA is actually physics, by the way.
0:20:15 So it’s like physics questions.
0:20:17 Because I was like, yeah, hard questions.
0:20:20 So cost performance, 01 and 03.
0:20:21 Okay, so here we go.
0:20:22 We can see this is math.
0:20:25 A-I-M-E is math questions, right?
0:20:29 So we can see on math, 03 for inference costs is down here.
0:20:32 03 high is way up here.
0:20:38 And the inference cost is like way cheaper than 01 Pro, which I believe is the same as 01 high.
0:20:39 Yeah.
0:20:41 I don’t know why they’re using different terminology.
0:20:43 I think 01 high is 01 pro.
0:20:58 I mean, I think, you know, remember when we talked about when like reasoning models first came out, I think when we were out in Boston, you know, for the HubSpot inbound conference, you know, we talked about like how things were going to speed up now that there was this new paradigm where it wasn’t just throwing more data at it, but you also could throw more compute power to.
0:21:01 And so we probably would see things improve dramatically faster.
0:21:09 It feels like we’re now seeing that because instead of the model getting, you know, twice as good in like, you know, two years, it’s like now it’s like what, like in three months or something.
0:21:10 Yeah.
0:21:10 Oh yeah.
0:21:15 The, the amount of time between like each of these new releases is just like compressing.
0:21:16 This is so fast.
0:21:17 I mean, here’s one that’s really impressive.
0:21:21 That same GPQ a here, here’s the curve of Oh three.
0:21:24 Look at where Oh one pro is basically down here.
0:21:34 The cost is like the highest possible cost and it’s dumber than Oh three low model or about on par with the Oh three low model here.
0:21:42 And the Oh three high is still maybe slightly cheaper than Oh one medium, but like, look at the jump when in intelligence way up here.
0:21:43 Wow.
0:21:43 Pretty wild.
0:21:47 The speed at which these leaps are happening is pretty crazy.
0:21:48 Yeah.
0:21:50 When we first started the show, we were talking about like how big of a change.
0:21:55 It just wasn’t, I think people probably thought that was hype and it’s actually happening now.
0:21:59 And I think people still don’t understand it’s actually happening and like how big of a deal it is.
0:22:02 Like Sam Altman was interviewed at Ted the other day.
0:22:02 Oh yeah.
0:22:03 I saw that.
0:22:03 Yeah.
0:22:09 And he was talking about how, I think he said that they have 500 million weekly active users now.
0:22:17 And then the Ted guy said, backstage, you told me that your numbers doubled in the last month since the whole Ghibli thing came out.
0:22:22 He was like, that was supposed to be off record, but yeah, you know, yeah, it’s, it’s going well.
0:22:28 I mean, so possibly open the eye is what in the ballpark of maybe 700 plus million users.
0:22:31 I mean, that is just Silicon Valley has never seen this before.
0:22:38 And it’s so wild that Sam Altman used to run YC, like the big incubator for Silicon Valley, that he ends up producing the biggest startup of all time, possibly.
0:22:40 I just wonder why he would want that to be off record.
0:22:43 That’s something that would probably attract investors pretty quickly.
0:22:44 Didn’t know those numbers.
0:22:45 Yeah.
0:22:51 But I think he is in a game with Google and Grok and Anthropic and everyone of like, you want to hype things up.
0:22:54 But also you have to keep some cards secret, right?
0:22:57 Like you don’t want them to know exactly everything.
0:23:00 I mean, there’s probably some misdirection and all kinds of stuff going on.
0:23:00 Yeah.
0:23:00 Yeah.
0:23:02 It’s wild how fast it’s improving.
0:23:09 And one thing that I found interesting is in the interview, he alluded to this being possibly like the most pivotal year for AI.
0:23:13 Like basically saying, and he’s also kind of like saying the best is yet to come kind of thing.
0:23:16 And also talking about the next big thing is scientific discoveries.
0:23:16 Yeah.
0:23:21 And the biggest talk on X over the last week that I’ve seen a lot of the rumors that were going around in some of them
0:23:26 were even shared, like there was like positive things being said by scientists too, kind of alluding to this being possibly true,
0:23:34 is that they’ve had early access to the new open AI models and possibly some new discoveries have been made.
0:23:39 Or at least experiments that they were not able to come up with that are now helping them possibly discover new things in science.
0:23:40 Yeah.
0:23:45 So possibly this is like having like really, really material impact on science for the first time ever.
0:23:46 Yeah.
0:23:46 No.
0:23:48 In fact, let me pull up an article.
0:23:50 I was actually reading this earlier.
0:23:52 It’s on the information here.
0:23:56 Open AI’s latest breakthrough AI that comes up with new ideas.
0:23:56 Yeah.
0:23:59 Now AI is getting good at brainstorming.
0:24:05 This came out a few days ago before this new, you know, O3 and O4 mini came out.
0:24:09 If the upcoming models dubbed O3 and O4 mini perform the way their early testers say they do,
0:24:15 the technology might soon come up with novel ideas for AI customers on how to tackle problems such as designing
0:24:20 or discovering new types of materials or drugs that could attract fortune 500 customers,
0:24:23 such as oil and gas companies and commercial drug developers.
0:24:28 in addition to research lab scientists, the apparent improvements highlight the benefit of AI models
0:24:32 focused on reasoning, which the chat GPT maker debuted in September.
0:24:37 But yeah, basically saying that these new models, the people that got early access have been saying
0:24:43 like this is helping us design and discover new materials and drugs, which is that’s kind of the
0:24:44 holy grail of what people want AI to do.
0:24:45 Yes.
0:24:49 I mean, it looks like we’re already there and we’ve just like scratched the surface of what this new
0:24:51 paradigm with reasoning models can do.
0:24:55 But it seems like the next several years, it’s going to keep accelerating and possibly,
0:24:57 you know, revolutionizing science.
0:24:59 So it’s like just, you know, it’s exciting.
0:24:59 Yeah.
0:25:00 Yeah.
0:25:03 And one of the things about these new models that we haven’t even touched on yet is that
0:25:05 they’re all being built multimodal, right?
0:25:09 So theoretically, you can give them videos, you can give them audio, you can give them text,
0:25:13 you can give them all the modalities, images, anything.
0:25:17 And it will actually be able to understand what’s going on within those.
0:25:21 But also they all have search now and they’re all getting the ability to use tools.
0:25:24 And so this chart I found pretty interesting.
0:25:27 This is one that they showed in their live stream, but they didn’t actually put on their
0:25:27 website.
0:25:33 You can see that when they gave Oh, for many tool use and it was using Python to help with
0:25:38 the math, it scored 99.5 on the competition math.
0:25:43 So like, yeah, basically acing competition math, when you give it access to be able to
0:25:48 use tools, it’s getting to that point where like these benchmarks aren’t good enough anymore.
0:25:53 We need new benchmarks for like, can AI find new discoveries for us?
0:25:54 Like, how do we make those kinds of benchmarks?
0:25:56 Right.
0:25:58 We’re getting more and more to a situation too.
0:26:01 Like when the Oh one came out, it took people a while to realize how good it was or like what
0:26:02 it meant.
0:26:03 You know, Sam Altman said this.
0:26:08 He said, his son is never going to grow up in a time where he’s smarter than AI.
0:26:12 Like for a child who’s being born right now, who’s young, that’s just not going to be possible
0:26:13 in their lifetime.
0:26:14 So like, that’s where we’re at.
0:26:18 Like these models, not in every single way, not in an entirely general way, every single
0:26:23 possible way, but in a lot of ways, they’re already more intelligent than most people.
0:26:25 And that’s just going to continue and accelerate.
0:26:29 So I’m constantly thinking about like, what does this mean for me personally in my business?
0:26:31 And like, what am I going to, how am I going to use it?
0:26:33 I saw Greg Eisenberg tweeting about this the other day.
0:26:36 If he’s telling the real story, if he was just trying to like set up something for social
0:26:40 media, but he’s talking about how to be kind of in a rut because things are changing so fast.
0:26:42 And I feel like there’s so many different things you could do.
0:26:46 And it’s like, what do you do in an environment where things are so dynamically changing?
0:26:48 We’ve never experienced this in human history.
0:26:53 It’s not clear how anyone exactly takes advantage of this because maybe by the time you do something,
0:26:54 it’s already changed again.
0:26:54 Yeah.
0:26:55 Yeah.
0:26:58 We’re entering completely new territory that nobody’s been in yet.
0:27:04 And even all the people that are trying to predict when things are going to happen or what’s going to happen next are all just making predictions.
0:27:07 Like they don’t even necessarily know what’s coming or where this is going to lead.
0:27:08 Yeah.
0:27:12 I’ve been preaching this for a while, but like, I really feel like major corporations are
0:27:13 not taking this seriously enough.
0:27:18 Like, you know, being a corporation who has too many meetings and overly like moves very
0:27:18 slow.
0:27:20 That’s a bad spot to be in.
0:27:24 I think people don’t realize how smart Elon Musk has been in this area where he’s like
0:27:26 really like made things more lean and like move fast.
0:27:31 Every big company should be doing that because otherwise, how do you survive in an environment
0:27:33 where things are dramatically changing?
0:27:33 Yeah.
0:27:33 Yeah.
0:27:37 Did you see Toby Lucky’s statement from Shopify?
0:27:38 Yeah.
0:27:38 Yeah.
0:27:42 I mean, he’s basically saying what you said, but the big point that he made is if you guys
0:27:48 want to hire anybody else in Shopify, you must prove that what you’re hiring for can’t be
0:27:49 done with AI first.
0:27:50 Right.
0:27:54 If you can prove AI cannot do it, then okay, you can make the hire, but you must prove to
0:27:56 me that AI can’t do it first.
0:27:56 Right.
0:28:00 And it’s also super clever because he’s also kind of like baking in AI into the culture
0:28:03 of the company where it’s like, look, if you’re involved in the company, you have to be thinking
0:28:06 about how you could be using AI to make the company better.
0:28:06 Yeah.
0:28:10 Because otherwise, how could you do that test if you’re not like actually using AI and like
0:28:11 actively understanding what it can do?
0:28:13 So that’s super brilliant.
0:28:16 And I don’t think he’s coming from a place of like, AI is going to replace all of you.
0:28:20 I think he’s coming from a place of like, this company can be so much more efficient
0:28:24 and so much better and get things done so much more smoothly if we’re all leveraging
0:28:25 AI together.
0:28:28 He’s talking about like an AI first company, like even though, even though they’re a large
0:28:32 org restructuring already to be an AI first company.
0:28:34 And most companies are not realizing this.
0:28:38 I mean, I, I think almost all major companies should be reorganizing right now ahead of time,
0:28:42 but you know, like people always, you know, they’re more reactive and proactive, like especially
0:28:46 big organizations, like all the companies, they will realize this like five or 10 years from
0:28:47 now.
0:28:50 And then some of them will not survive because of that, but it’s gonna be a wild time.
0:28:51 Yeah.
0:28:53 So I want to bring this one home with this.
0:28:54 Yeah.
0:28:59 So I actually asked open AI is deep research to sort of break down all of the models,
0:29:04 like help get past the confusion, do the deep research on which models are good at what,
0:29:09 and then give me like a ranked order of basically which was the dumbest to the smartest of all of
0:29:11 the available models right now.
0:29:21 It actually took, let’s see, 16 minutes for even a chat GPT to figure out, um, its own dang models
0:29:25 with how confusing it all is, but here’s the breakdown.
0:29:32 So GPT 3.5 is great at basic conversations, but substantially lower reasoning and knowledge
0:29:32 than the others.
0:29:38 Then GPT four, which came out in March, 2023 was a big leap from 3.5.
0:29:44 And then you have GPT four turbo slash GPT four Oh, which apparently are sort of tied.
0:29:51 Then you have GPT 4.1 nano, which is one that came out this week, which is a very small, but
0:29:57 surprisingly capable model, 80% of GPT fours ability, basically.
0:30:02 Then you have a GPT 4.1 mini slash GPT four Oh.
0:30:04 So GPT four one mini was one that came out this week.
0:30:08 GPT four Oh is the one that’s sort of been the default model for a while.
0:30:15 inside of chat GPT, but GPT four one mini, it says matches or exceeds GPT four Oh’s intelligence.
0:30:17 So both are kind of tied.
0:30:24 Then you have GPT 4.5, which was the one that they codenamed Orion GPT 4.5.
0:30:26 I know it’s been available in the $200 a month plan.
0:30:30 I can’t remember if it was available in the $20 a month plan, but that one’s been one that
0:30:32 was sort of more human.
0:30:37 like, I guess it’s good at writing stories, good at when you just want to have like a
0:30:37 chat with an AI.
0:30:42 That was the one that I always felt like was the best at having like a very human like conversation.
0:30:44 Then you have open AI’s Oh one.
0:30:49 This was their first reasoning model where it actually sort of thought through after you gave it the prompt.
0:30:53 And so during inference, it sort of did that sort of chain of thought thinking.
0:30:55 This was the first model we saw that was doing that.
0:31:05 Then you have Oh four mini, which was, I believe one that also came out this week is a little bit better than just the standard Oh one.
0:31:09 So it’s a new efficient reasoning model when run in high compute mode.
0:31:15 It’s extremely capable, likely outperforming Oh one and approaching the top models in STEM benchmarks.
0:31:26 Then you have GPT 4.1 full, which is another one that came out this week, which is open AI’s latest GPT model only available in the API right now, not available in chat GPT.
0:31:37 And this one’s more tuned for coding and following instructions, but this is why it’s a little bit confusing because we can see GPT 4.1 is actually a smarter model than GPT 4.5.
0:31:42 So even though the numbering scheme went backwards, it’s actually a more powerful model.
0:31:49 And then you have open AI Oh one pro, which for the longest time was open AI’s top dog model.
0:31:54 If you got into chat GPT and you had the $200 a month plan, Oh one pro was the best of the best.
0:31:58 It just kind of crushed anything you threw at it.
0:32:02 It was really good at coding took forever to get into the API, but that was their best for a long time.
0:32:06 And then you have open AI Oh three, which was also one that was just released this week.
0:32:08 It is available in chat GPT.
0:32:13 It’s available both in pro and plus so 20 and $200 a month plans.
0:32:19 This one is their new best, most powerful model they’re making available right now.
0:32:20 That’s open AI Oh three.
0:32:28 And then this list says their most intelligent, best possible model you can use is Oh three with tool use turned on.
0:32:35 So allowing Oh three to use tools like Python and things like that to help it with its, you know, math and coding and logic and things like that.
0:32:38 When you use Oh three with tools, it’s their best model.
0:32:40 And then also GPT 4.1.
0:32:44 They’ve also sort of tied with it depending on the use case.
0:32:46 So insanely confusing.
0:32:46 Yeah.
0:32:50 But this is the order it ranked it in based on all the deep research.
0:32:50 Yes.
0:32:54 I mean, so thankfully, like people won’t have to worry about this soon.
0:32:57 So like, hopefully in like two months, you won’t have to worry about this GPT five, hopefully be out.
0:32:59 It’ll manage all that for you.
0:33:00 You just talk to one model.
0:33:05 But right now, I think their descriptions actually are pretty good in line with my experience.
0:33:11 Like, I think there’s like three models people really would the average people would use right now, like for Oh, that’s what most people would use.
0:33:13 Like I said, that model keeps getting better.
0:33:13 Yeah.
0:33:16 And if you’re just using ChatGPT free, that’s what you’re using.
0:33:17 You’re using 4.0.
0:33:17 Yeah.
0:33:18 4.0 is great.
0:33:19 It just kept getting better.
0:33:23 They haven’t really like made a big fuss about it, but it seemed to have just kept getting smarter in some way.
0:33:25 I don’t think that’s entirely reflected in the benchmarks.
0:33:28 4.5 still seems to be the best for writing.
0:33:31 So if you’re wanting to edit something or write something, 4.5 is the best.
0:33:33 That one’s also getting phased out.
0:33:34 So use it while you can.
0:33:36 Use it while you can.
0:33:38 I think it’ll still be there in five.
0:33:39 I had to guess behind the scenes.
0:33:48 And then O3, which I haven’t used yet, based on the benchmarks and my understanding how the models work, I would assume that’s going to be the best if you have any complicated questions.
0:33:52 Like if there’s something that you’re thinking like, oh, the model might give me too surface level of an answer.
0:33:54 It’s not taking the time to think it through.
0:34:00 If you’re asking something complex, you probably want to hand it to O3 and give it the time to really think through its response to you.
0:34:03 And that’s also probably the one I would imagine you’re going to go to for coding as well.
0:34:04 Yeah, probably so.
0:34:06 You know, I haven’t had a chance to test any of it with coding yet.
0:34:12 There’s one last thing that we didn’t actually bring up about the new models that they just released, and that’s the context window.
0:34:12 Yeah.
0:34:15 One of these models is actually a 1 million token context window.
0:34:17 Is it a real one?
0:34:20 Like I remember that was a part of the whole fiasco with Meta as well.
0:34:21 It was like they said like 2 million context window.
0:34:27 And then apparently at all the benchmarks, it would like start to like fall apart after like 128K context or something.
0:34:28 I don’t know.
0:34:33 I actually kind of trust OpenAI’s word on this a little bit more than I trust Meta’s word on it.
0:34:33 Yeah, me too.
0:34:34 Me too.
0:34:35 Because that was just ridiculous.
0:34:36 Like people tested it.
0:34:40 It’s like they were hyping up like 2 million contacts or 4 million or whatever it was they hyped up.
0:34:43 And then the people tested it.
0:34:47 And as soon as it went beyond like 128K contacts, like the model just like became really dumb.
0:34:49 Yeah, I heard the same thing.
0:34:55 I mean, I’m sure there’ll be some diminishing returns like O1 Pro even like, you know, that’s why I was using a repo prompt is like, yeah, it could take a lot of context.
0:35:02 But you do want to kind of be careful about what you give it because if you give it more exactly what it needs, you do tend to get better results back.
0:35:09 But yeah, these context windows are getting to a point where like context length is not going to be an issue for very much longer.
0:35:09 Yeah.
0:35:13 Like you’re going to be able to plug in entire code bases and it’s going to be able to read the whole dang thing.
0:35:20 Yeah, and it is interesting that like Sam Altman kind of admittedly, well, he’s at number one, like people were asked about like DeepSeek and he’s like, well, it hasn’t impacted our growth at all.
0:35:22 So not really too worried about it.
0:35:28 But also he was saying that like moving forward, it’s probably more about the things they build on top of the models.
0:35:31 He thinks they’re going to continue to be in the lead.
0:35:35 He said, but that lead is going to narrow and it’s going to be like a small lead in the future.
0:35:37 So it’s all about what they build on top of it.
0:35:41 You know, it did seem early on that like Open the Eye was going to be incredibly ahead of the game.
0:35:44 But now it feels like there’s going to be lots of people who are pretty close to them.
0:35:45 So here we go.
0:35:50 So as of April 2025, here’s the context window sizes for OpenAI’s latest models.
0:35:55 So it was the GPT 4.1 series, the one that’s only available in the API.
0:35:56 Yeah.
0:36:03 That one is a 1 million token context window, which includes GPT 4.1, 4.1 Mini and 4.1 Nano.
0:36:05 So those are a million tokens.
0:36:08 GPT 4.0 is still 128,000.
0:36:11 O3 Mini supports up to 200,000.
0:36:16 O3 Full, likely maintaining or exceeding 200,000.
0:36:18 But it doesn’t actually say here.
0:36:22 O4 Mini released along O3 is a reduced version of a successor.
0:36:27 Specific context window sizes for O4 Mini have not been detailed.
0:36:31 So yeah, the O3 and O4, they haven’t actually said what the context window was.
0:36:38 But the new 4.1, which is the one that’s going to replace 4.5, is the million token context window.
0:36:42 So yeah, that’s pretty much a breakdown of all the OpenAI news that came out over the last few days.
0:36:46 There was one smaller update that’s just like a quality of life thing.
0:36:53 If you are generating images inside of OpenAI, they created this library page where you can see all the images that you’ve generated in just like one spot.
0:36:56 You know, kind of cool quality of life thing that they’ve added as well.
0:36:59 Yeah, there’s another move towards killing Mid-Journey possibly.
0:37:00 Yeah, very true.
0:37:02 Yeah, you know, Mid-Journey 7 came out.
0:37:06 Like people were expecting it was going to be way better at understanding images.
0:37:08 And it’s like slightly better.
0:37:08 Yeah, yeah.
0:37:10 It is still the most beautiful model.
0:37:11 And it got slightly more beautiful.
0:37:12 Yeah.
0:37:15 It more consistently beautiful and, you know, and things like that.
0:37:20 But it seems like they did not nail the thing where, oh, the characters are going to be completely consistent.
0:37:22 Like you can like reuse the characters.
0:37:22 Yeah.
0:37:24 As far as I can tell, they did not nail that.
0:37:26 Yeah, I played around with V7 a little bit.
0:37:29 I was actually about to make a video on it and didn’t even release the video.
0:37:32 Like I only want to put out videos about stuff that I’m excited about.
0:37:36 And like I was playing with it while recording and like it wasn’t really exciting me.
0:37:38 I’m like, it doesn’t feel like that big of a leap to me.
0:37:43 Yeah, the only cool part of it was the turbo thing where you can like rapidly try ideas.
0:37:44 That’s cool.
0:37:44 Yeah.
0:37:45 And you can do voice mode.
0:37:51 And so that was actually the only cool thing about version 7 was you could just press a voice mode button and just start chatting.
0:37:54 And as you’re chatting, it starts creating the images super fast.
0:37:55 That was cool.
0:37:57 Yeah, I actually didn’t play with that mode.
0:37:58 So that’s one I’ll have to mess with a little bit.
0:37:59 That’s the actually cool part of it.
0:38:00 Like everything else is like whatever.
0:38:01 Yeah.
0:38:03 I didn’t find it that impressive.
0:38:03 Yeah.
0:38:05 Ideogram has gotten really, really good.
0:38:08 The various Leonardo models have gotten really, really good.
0:38:08 Yeah.
0:38:11 Obviously, OpenAI’s model is really, really good.
0:38:18 Like all of these models have sort of caught up to each other where like mid-journey is just sort of in the mix with them as opposed to being the leader, you know?
0:38:19 Yeah.
0:38:29 But like I said in the past, I think people don’t understand like with OpenAI’s models, you know, with the reasoning models, they’re going to be able to actually understand what’s going on in the images and the videos they create.
0:38:29 Right.
0:38:32 Like that’s going to be the unlock, just like making something beautiful.
0:38:35 I’m sure you can do that, but do you understand what the hell is in the thing you just created?
0:38:36 And can you modify that?
0:38:38 That’s the thing that’s interesting.
0:38:42 And I think, I think OpenAI is like the one who’s like by far the leader in that right now.
0:38:43 Yeah.
0:38:43 Yeah.
0:38:44 They’re on the forefront of it for sure.
0:38:45 Yeah.
0:38:52 But yeah, I mean, I think that’s a pretty good breakdown of everything that’s been happening in the world of OpenAI this week.
0:38:52 Yeah.
0:38:58 We went off on a few little tangents, but, you know, hopefully we brought it all home by sort of showing you the ranking of intelligence.
0:39:02 And I think Nathan, you did a good job of saying like, all right, these are the three models you’re probably going to use.
0:39:14 So, you know, hopefully you found this informative and helpful trying to declutter your mind on all of these different OpenAI models because it’s still confusing to us.
0:39:17 I mean, we’re paying attention to it every day and we still get confused.
0:39:19 So, you know, hopefully this helped a little bit.
0:39:23 And thank you so much for tuning in to this episode.
0:39:24 Hopefully you enjoyed it.
0:39:27 If you like it, make sure that you subscribe ideally on YouTube.
0:39:30 That’s where we’re really trying to primarily grow the show.
0:39:35 But if you prefer audio, we’re available on Spotify and iTunes and wherever you listen to podcasts.
0:39:38 Thank you once again for tuning in and hopefully we’ll see you in the next one.
0:39:39 Goodbye.
0:40:01 Bye.

0:00:07 Hey, welcome to the Next Wave Podcast. I’m Matt Wolfe. I’m here with Nathan Lanz, and today we’re
0:00:13 diving into the world of AI and visual effects. Today, we’re chatting with Nikola Todorovic,
0:00:18 the CEO of Wonder Dynamics. Now, if you’re not familiar with Wonder Dynamics, it’s a tool
0:00:25 that allows you to film yourself or any human on video and then re-skin that video with like an AI
0:00:32 generated or 3D CG character. So if you think of like Lord of the Rings and you’ve got Golem and
0:00:38 Lord of the Rings, that was all acted by a human. And then special effects artists went in later and
0:00:45 sort of re-skinned it as a CG character as Golem. That’s what Flow Studio from Wonder Dynamics does
0:00:51 right now. And anybody can use it and it makes it super easy. And Nikola is actually going to demo
0:00:55 exactly how to do that in this video. And make sure you stick around as well, because if
0:00:59 you’ve got a business or are doing marketing or you’re a content creator, we’re going to dive
0:01:04 into all sorts of use cases for you as well. So let’s go ahead and jump in with Nikola from
0:01:09 Wonder Dynamics. Thanks so much for joining us on the show today. How are you doing?
0:01:11 Pretty good. Thanks for having me. It’s good to see you guys.
0:01:16 Yeah, thanks. You know, we were saying this right before we hit record, but the first time that we ever
0:01:22 saw the Wonder Dynamics studio tool, both of us thought it was fake. We saw the tool and went,
0:01:28 I don’t believe this. I think they’re pulling our legs. And then we finally got our hands on it a
0:01:32 couple of weeks later and we’re actually blown away that it did what it said it could do.
0:01:37 Yeah, I know. You know, as I said, I think a lot of people thought that, you know, a lot of people
0:01:41 thought we faked our demo, which is always a compliment, right? When people think that. So it’s
0:01:46 actually a nice thing. But I got to say, Matt, when you got access and did a coverage, our team was
0:01:51 really excited because a lot of people on our team follow you for news on tech and AI and all that
0:01:55 stuff. So they’re really pumped that we got that coverage because we were working in stealth mode for
0:02:00 about four years. Right. So one of those things with demos as well as, you know, you don’t know what the
0:02:04 reaction is going to be right when you’re so close with something for so long. So it’s very grateful,
0:02:08 you know, to get the coverage we did. Yeah. Yeah. I appreciate it. Yeah. I remember I made a video
0:02:12 where I was outside my yard and I was playing basketball. Right. And then I switched myself
0:02:17 with a robot. And then Elon Musk went and shared that video and said, maybe we’ll have this in real
0:02:23 life soon. I was like, oh, this is cool. That’s awesome. So very cool. Well, let’s get into your
0:02:28 background a little bit. Like how did Wonder Dynamics come to be? Were you in visual effects
0:02:34 and Hollywood? What’s the backstory there? Yeah. So I always wanted to be in filmmaking since I was,
0:02:38 you know, 12 years old and I come from a really small country. I was born in Bosnia, lived in
0:02:42 Croatia and Serbia all my life. You know, I couldn’t really afford to go to a film school or something
0:02:47 like that. But, you know, I started about 12, 13 years old, started watching YouTube tutorials,
0:02:52 you know, video co-pilot Andrew Kramer and those guys, you know, that was kind of my little VFX
0:02:58 school. And, you know, I moved to U.S. really to pursue career in filmmaking and, you know,
0:03:03 started working as a VFX artist, as a compositor first, worked for, you know, freelance and a lot
0:03:07 of different studios, you know, everything from ads to indie films. And then I started working as a
0:03:14 supervisor. And I met my co-founder, Ty Sheridan, who’s a young actor and a producer. And we really
0:03:19 started writing together. We wanted to, you know, tell our own films. Usually whatever we wrote,
0:03:24 it was, you know, sci-fi with robot characters. So we really wanted to tell the story about the near
0:03:29 future, you know, near future when we coexist with robotics. And every time we wrote something,
0:03:33 we realized, all right, this is about $200 million budget. There is no way we’re going to ever get
0:03:37 that, you know, and then we started looking into a bit of AI. You know, first we built something that’s
0:03:43 more interactive nature. About 2018, we built this product where you can have a conversation with the
0:03:48 character. So let’s say watching a murder mystery and a little switch happens from a stream into a
0:03:52 digital double. That’s a 3D representation that you can have a chat with. And that’s kind of when we,
0:03:56 you know, it was cognitive AI wasn’t there yet. This is before ChatGPT and all this push,
0:04:01 but we really saw a big opportunity in visual. We call it visual AI. There was no gen AI or any terms
0:04:05 like that. And we said, okay, this is going to change production. So I always say selfishly,
0:04:10 we wanted to just do it to make our own films. So we’re like, what’s the worst thing that could happen?
0:04:13 You know, and the worst thing could happen. We both concluded, we’re just going to learn what
0:04:17 the future of filmmaking is before other people. So we kind of get a little headstart. And then about
0:04:22 six months in, we realized, all right, this is bigger than just two of us. Let’s turn it into a tool and a
0:04:26 platform. I mean, that’s really how we started. We did a complete pivot, bootstrap first three years,
0:04:29 you know, kind of trying to find your way. What is it that you really want to do? And then,
0:04:33 yeah, 2018, 19, we really focused on this and kept growing the team and kept building.
0:04:37 What was the first prototype? And like, what made you confident that you guys could actually
0:04:38 build it?
0:04:45 You know, what was the first prototype? We actually did, if you know, the Spectre opening sequence in
0:04:50 Mexico, when James Bond walks on a ledge, we did that and test, we did kind of, you know,
0:04:54 just focus on the mocap, you know, AI mocap. And obviously the quality wasn’t even close to it. But
0:04:59 then we really realized the potential, you know, was huge. You know, we really started looking more
0:05:04 into kind of technology around, you know, self-driving vehicles and robotics, which is all
0:05:08 about, you know, understanding the world around you. You know, we call it scene understanding. How
0:05:13 much can I understand what I’m seeing in the pixels? And really, you know, as a VFX artist,
0:05:16 you’re always trying to do that. You’re trying to understand what looks real, what doesn’t,
0:05:22 what is it in 3D space from a 2D plate that I have, right? So that’s how we just kept building on top
0:05:22 of that.
0:05:27 Yeah. I wanted to share too, you mentioned your co-founder partner on this was Ty Sheridan.
0:05:31 Anybody who’s not familiar with them, if you’ve seen Ready Player One, Ty Sheridan’s the main
0:05:36 character in Ready Player One, right? So you’ve probably seen him and not even realized it,
0:05:41 but very cool. I would love to dive in and actually take a look and show off to anybody who actually
0:05:46 hasn’t seen this product yet, what it’s capable of. So, you know, if you wouldn’t mind sharing your
0:05:51 screen and showing us some of the sort of sizzle reel of what this thing could do, we’d love to just
0:05:53 kind of take a peek and chat about that.
0:05:57 Yeah, absolutely. So I can guide you a little bit on some capabilities here.
0:06:03 This is a flow studio and we have about four project types right now, which is live action,
0:06:07 which you mentioned earlier. And that’s really tailored about having CG character inside a live
0:06:12 action shot. And then we have a motion capture. That’s more just if I want to get the mock-up
0:06:16 performance out of a shot. And then recently we launched something called video to 3D scene,
0:06:20 which is more towards animation. But I’ll show you quickly of how it works.
0:06:25 We’ve really wanted to build an easy interface. So we have three steps. The idea behind Wonder was
0:06:30 always, how do I get someone who’s not that proficient in 3D to also get in the 3D? But then
0:06:35 also people that are proficient, how do they use it in their existing pipelines? This is a big issue I
0:06:41 saw, you know, in generative AI even early on. It was a lot of black boxes. You get certain results,
0:06:42 but you can’t really push it and edit it.
0:06:49 So we were from the get-go, we said, okay, it’s not there yet, but if I can get 60% there,
0:06:53 what is the data I can get out that I can then plug in an existing tools and push it there?
0:06:57 Because as you know, you know, being an artist is all about control of every single element,
0:07:01 whether you want to control your performance or animation, or you want to control the camera
0:07:05 or lighting, et cetera. So that was the idea behind it. But I’ll show you a quick kind of
0:07:09 workflow on that work. And then the other thing we wanted to do, we wanted to not just work shot by
0:07:14 shot, but we said, let’s enable it so it works in a sequence. So I’ll show you quickly a couple of
0:07:19 shots that you’ve probably seen, but it’s good to show. So, you know, a bit of a non-related shot.
0:07:23 And then we have one with a couple of connected shots with the same actor. So let’s say I’m
0:07:28 happy with my edit. I would go on my next. And then what I do is I scan frame for actors and then
0:07:34 it looks for the actor in the shot. And then we have characters here that you can apply. You know,
0:07:37 we have a couple of characters that we, a few characters we provide for people that don’t have
0:07:41 their own. But the idea is that you can upload your own characters. Right now we have a blender
0:07:47 and a Maya add-on to help you prep your character a bit easier in those. So the idea was always,
0:07:52 you know, create your character traditionally, as you do until AI gets there, you can completely
0:07:57 control it. So let’s say for this first one, we’ll assign this test crash dummy. And then
0:08:02 I have my second shot and we’ll come back to this one. And for this one, I’ll assign, we have
0:08:08 this little alien character. So I’ll just drag and drop. And then I have a couple of shots with the
0:08:12 same actor. I don’t have to go target for each one. I just need to target once. And then we use
0:08:17 something called re-ID that’s looking for the same actor in multiple shots. So let’s say we’ll do this
0:08:23 also, test crash dummy. And pretty much that’s it as far as interaction. Really where the power of the
0:08:29 software is, I can get a video out, but we say that’s post-viso best. So it’s just meant to show
0:08:34 you where the AI worked, where it didn’t work, right? It’s not meant to be your final VFX shot.
0:08:40 Really the power is in these elements and the scenes. So I can export my mocap. I can export clean plate,
0:08:46 alpha mass, camera track, and character pass. And then I can export, you know, a 3D scene out of it.
0:08:51 So essentially what it takes, it takes that plate. We have about 25 models in there. And these models
0:08:56 are everything from, you know, facial performance tracking, body pose tracking, camera tracking,
0:09:01 lighting estimation, and things like that. So then it takes basically from a 2D plate and puts it in
0:09:06 3D. In this case, let’s say I’ll export blender scene. And then something like this will take about,
0:09:10 you know, 70 minutes to process. I have it from before so I can show your results. You know,
0:09:14 and these shots are considered, if nothing, they’re considered really easy VFX shots. So you’re going to
0:09:19 get pretty decent results out of it, right? But for a trained eye, you’re going to be like, all right,
0:09:24 you know, obviously I can see a little leftover of the actor there. Not happy with the lighting here,
0:09:30 et cetera. So what I would do here is I would basically download my clean plate. So my compositor
0:09:34 can go in and clean what didn’t clean well, you know, on the actor. But most importantly,
0:09:39 I would really download these 3D scenes. So let’s say I download this 3D scene. So clean plate,
0:09:43 obviously pretty straightforward. You get, you know, image sequence of that shot. So obviously I can see
0:09:48 the actor remaining. So my compositor would clean that out. But when it comes to 3D scene,
0:09:54 essentially I’m getting each one of these shots in a separate 3D file. So if I open one,
0:10:00 what happens is it takes that performance and he estimates the animation and the camera. So it
0:10:04 actually tracks the camera and it tracks the animation, right? And then I have an option to
0:10:08 download my pass, which is my texture for the character. And I can download the clean plate,
0:10:14 which would be my background. Then I can download textures of that character. Now I have that
0:10:21 characters as a texture also, but then I’m also going to add that background because that clean plate
0:10:25 essentially, let’s say I cleaned it and I’m happy with it. That’s basically clean plate is my background,
0:10:30 right? Because it’s a 2D background. So essentially what that gives me, gives me that shot, right?
0:10:36 In elements I need it. So I have full animation data in it that I can control. I have obviously camera,
0:10:41 I have lighting info. So if I want to adjust my lighting, right? So I would really continue my
0:10:45 animation the way I do. So in this case, let’s say, you know, I want to control to the smallest
0:10:50 details. I want the character to actually look up a little bit higher. I can control that. So the idea
0:10:55 always has been around, how do I not lose control as an artist, right? When I do things like that. So
0:11:01 that’s why the data you get out of it is really the main thing of what we did inside of Full Studio
0:11:06 on that side. And then obviously one of the things that we always get from artists is, you know, I
0:11:10 don’t need all the passes. Can you just give me a camera track and clean plate? So we opened these
0:11:14 wonder tools as well that are essentially just these single models because we have so many models running
0:11:19 in the background. So this will do quicker as well. And let me show you a couple of other things that I
0:11:23 think is interesting. You know, you probably saw the demo we launched for animation use case.
0:11:31 The idea for it was that if I have multiple sequences in one space, can I calculate my camera setups?
0:11:37 So in this case, I have a shot, two shot that goes to one shot, another shot, and then it goes a little bit
0:11:43 two shot behind them. What it does, it actually calculates the footprint from those actors and the trajectory
0:11:49 of the cameras. And then this is all one camera, but I just edited it, right? But what animation use case does,
0:11:54 it really places it in 3D space based on the cuts, right? So it tries to guess where those cameras
0:11:59 are. So it pretty much sets you up with a kind of virtual production inside of Maya or Blender just
0:12:04 based on that sequence. So the idea was here was, you know, I also select my environment, not just the
0:12:09 characters, but more importantly, what we wanted to do here is, you know, what if I’m in my living room
0:12:13 and I want to frame my animation how I like it, but I also want to cut it how I like it, but I don’t want
0:12:18 to deal with each shot separately. I want to have that performance to be one continuous performance
0:12:22 with different camera setups throughout the shot, right? On the side. So there’s obviously a lot of
0:12:29 models working here together to try to calculate that. And one thing we saw is AI mocap, obviously,
0:12:33 you know, a pose estimation is just, you know, one to two models out of 20 something we have.
0:12:39 But the problem is with kind of when you’re doing markless mocap is you are only basically guessing
0:12:44 position of joints and everything just based on an image, right? So once you lose an actor or they’re
0:12:50 occluded by another actor, an object, it’s really hard not to have it break. Right? So one thing we
0:12:55 recently released after, you know, we’ve been building this for almost about 10 months to a year
0:13:00 is, you know, I have this issue is that anything you film, as you know, you’re always going to have
0:13:05 inclusion, right? So it’s very hard to rely only on an image. So we built something called motion
0:13:10 prediction that essentially predicts motion if you lose the character. So if it sees a few frames before
0:13:15 this tree, it will guess, okay, most likely it’s still walking, right? Or if it only sees half of
0:13:20 the subject, it’s still, you know, going to guess what it is. So in this position, I see upper body,
0:13:24 most likely it’s sitting. So I’m going to set up my lower body. Why is this important? Because,
0:13:29 you know, for live action, you kind of only care what you see in a frame, right? Because that’s what
0:13:34 you framed it. But for animation, you really want a full 3D body pose for that, you know, so you can
0:13:38 control everything and it’s going to affect your animation as well. So another thing also it does is
0:13:43 if I go in a closeup and you only see the top of the body, it will float, right? The bottom part
0:13:48 will float as you can see here. So even if it only sees the upper body like that on a closeup,
0:13:53 it’s still going to generate what the bottom body is doing, right? And that’s important for any animator,
0:13:58 right? They don’t want to be doing so much cleanup when the thing break on the side. So that’s a couple
0:14:05 of things, you know, we had added recently that really came as a natural flow of things that we were
0:14:11 looking to add for artists. We’ll be right back to the next wave. But first I want to tell you about
0:14:15 another podcast I know you’re going to love. It’s called Marketing Against the Grain, hosted by Kip
0:14:21 Bodner and Kieran Flanagan. It’s brought to you by the HubSpot Podcast Network, the audio destination
0:14:26 for business professionals. If you want to know what’s happening now in marketing, what’s coming,
0:14:30 and how you can lead the way, this is the podcast you want to check out. They recently did a great
0:14:34 episode where they show you how you can integrate AI into the workplace. Listen to Marketing Against
0:14:36 the Grain wherever you get your podcasts.
0:14:45 Very cool. I’m curious, like, how has the sort of reception been to this in like Hollywood? I know,
0:14:49 you know, there’s obviously when it comes to AI in general, there’s a lot of sort of fear around
0:14:52 job loss and things like that. How has the reception been?
0:14:57 Yeah, we were very cautious about it. I think if you notice in our demo, I always say we don’t
0:15:01 generate art. We’re accelerating, right? It’s still the artist that made that character,
0:15:06 still the artist that made the environment. It’s still we’re picking performance from the actor,
0:15:11 right? So we’re very cautious about how we build it. And obviously, you know, we’re a bit different
0:15:16 also because, you know, everything we train on is synthetic data, you know, because we’re not
0:15:20 generating art. We didn’t really have to rely on scraping the internet or similar. So, you know,
0:15:25 the way we started, we also had some really big names on the board, you know, from Russo Brothers to
0:15:30 Spielberg, etc. And Ty is an actor. So we come from this space, from the entertainment industry.
0:15:35 So as an artist, I saw the fine line, right? It’s a really tricky one. Sometimes you almost have to
0:15:39 resist building certain things just because of that. Right? So, but you know, in your question,
0:15:45 how did Hollywood react? I think this example of OpenAI with the recent release, it’s a good example of
0:15:50 how these reactions sometimes are very, you know, drastic. I’ve seen it early on. You’ve probably seen it,
0:15:57 you know, 2023. A lot of panic around it and how things are trained. And then it calmed down a
0:16:00 little bit and it comes back up. Yep. Seems to be waves for sure.
0:16:08 There’s waves. Yeah, there’s waves. I would say, you know, we see a lot of innovation and a lot of
0:16:13 directors that really accept it because it gives them an opportunity to, you know, make things quicker and
0:16:18 iterate quicker. We see people that are scared of it, you know, mostly sometimes lack of understanding.
0:16:23 Again, you know, Flow Studio is a bit different. So we’re in this interesting intersection where,
0:16:27 you know, we went through a bunch of studios we worked with. We had to go through rigorous security
0:16:31 process and stuff. So the fact that we’re not generating art is always something that is a big
0:16:38 relief. Right. But I’ve also seen studios, some accepting GNI, some not, you know, some can talk
0:16:43 about it, some cannot. So, but for me, we really did build this for indie artists, first and foremost,
0:16:48 even though we have studios as customers, you know, I always say studios will be okay. They’ll figure
0:16:53 out their way, no matter what the change is. To me, what’s exciting about AI, and I think for a lot of
0:16:58 people is it is a very hard industry to break into. You know, I got a lot of luck on my side, took many
0:17:03 years to break in the industry, but I know a lot of artists that are way better artists than me that
0:17:07 didn’t have these opportunities. Right. So I think that’s why people are excited because you’re really
0:17:11 going to open up opportunities for people to tell stories. And maybe I’m a little bit naive, but I
0:17:16 don’t think storytelling should be tied to any kind of socioeconomic status. It really, storytelling is
0:17:22 too important for you to be a good salesperson or meet a producer to be able to tell your story. Right.
0:17:26 So to me, it’s like, you know, it should be a clear path and easier to tell stories. So.
0:17:30 Yeah. Nicola, I spent a little bit of time in Hollywood. So I was partnered with Barry Osborne,
0:17:33 the producer of Lord of the Rings and The Matrix, and we tried to create a movies
0:17:36 studio together. And I, the same thing you’re talking about, like, you know, it takes like 200
0:17:41 million dollars to do a film with big special effects. I was like, this is nuts. If you want
0:17:44 to like start a new studio, you’re talking about like raising hundreds of millions of dollars.
0:17:49 That was what I found exciting about Wonder Studio. Just imagine like in the future, small
0:17:53 independent creators, whatever’s in their head, they’ll be able to get it out of their head and on
0:17:58 the film, which just has not been possible before. Yeah. And Nathan, you know this well. I mean,
0:18:02 there’s so much sacrifice from a script. It’s very hard to write scripts. I, you know, I, I,
0:18:07 used to write and I still write and to me start. Sometimes I say starting a company might be easier
0:18:12 than writing a script sometimes, but you know how much sacrifice you go from a script, what’s on paper
0:18:17 to what’s on screen. There’s always, you know, as they say, like you can push that imagination to
0:18:21 the screen. There’s so many technical challenges. And, and I don’t know, to me, it’s a little bit
0:18:25 ridiculous. You know, Ty and I always joke when we walk, like you were, I think we were in San
0:18:28 Francisco and we’re like, how much do you think that building costs? And we look it up,
0:18:33 it’s like $40 million. We’re like, someone makes one movie. That’s three of these buildings,
0:18:38 spend three years, and then has a weekend to potentially get the money back. Cause they have
0:18:43 to make double that amount. It’s such a broken system. I mean, think about it from a standpoint,
0:18:47 if you’re starting a company, a startup, if you have a startup that you built for five years
0:18:54 and you put $200 million and you only have that weekend to get your money back. And you have no
0:18:59 control of what’s going to happen that weekend. Nobody would fund it. Nobody would fund it. Right.
0:19:03 So I had a friend that said, he’s like, the difference between a tech and a film industry is
0:19:08 that, you know, in tech space, there’s obviously much more capital and film we do because of love
0:19:13 and occasionally a hit here and there. Yes. It’s also, it’s like sexy to be involved in,
0:19:16 right? Like people, some people invest cause like, if I get my money back, that’s cool,
0:19:20 but I get to be involved in around the Hollywood, which was slightly how I got involved through
0:19:24 friends in Silicon Valley, actually. Yeah. It’s, it’s interesting. I mean,
0:19:28 it’s such a, such a passion thing, right? Storytelling is very important. You, you know,
0:19:33 I was fortunate enough to work with a lot of people that are really, you know, not in it to become
0:19:36 famous or anything. They’re in it because they really believe their story can make a difference.
0:19:41 And, you know, that’s novel to me. So, you know, that’s one of the reasons I came into
0:19:45 it as well. Very cool. You mentioned that like some of your clients are big studios. Are there
0:19:50 any like films out in the wild or maybe even stuff in production right now where this technology is in
0:19:57 use? Yeah, we have a lot, but unfortunately we have hefty NDAs that you can imagine. No AI involved in
0:20:03 any of the films. Okay. Unfortunately, we do have some that when I really like Boxer studio did a
0:20:06 Superman and Lewis and did this case study, they used it for it. So one, that’s one of the public
0:20:11 ones that you can see for a TV show. And it’s very cool. I like those guys at Boxer studio. They’re
0:20:15 really innovative. They’re like throwing a lot of tools and playing with it and combining and being
0:20:20 more efficient on a turnaround. But obviously besides film and TV, we’ve also seen it being used in gaming,
0:20:26 you know, also marketing, a lot of brands as well on that side. And then obviously a lot of content
0:20:30 creators, you know, kind of individual content creators. And what’s exciting for me is we saw early on
0:20:36 people that don’t know what rigging is or what, you know, they would get into 3D and start learning
0:20:41 because it seems easier, right? Because to me, 3D is always, I used to do compositing. And then when I
0:20:47 switched to 3D, I was like, this is so overwhelming. It’s so hard to start. It’s so complicated. CG is so
0:20:52 slow. So I think that’s the beauty, you know, if we can make it easier for people to get in the field
0:20:58 and not be so, you know, scared of even going that way. As you said, Nathan, it’s like, sometimes when you
0:21:02 look at the process, filmmaking process from the outside, you’re like, oh my god, this is so hard.
0:21:07 This is so scary, right? Yeah. I got a tour of Weta and also like got to go backstage when they were
0:21:13 working on Mulan for Disney and got to see the early special effects and things like that. I was very
0:21:18 intimidated. Like, I was like, there’s so much money involved. It’s so hard to get involved. I wish I
0:21:21 could just be like, just raise like 5 million to be able to create like something cool just out of my
0:21:25 head versus having to depend on some gigantic studio like Disney.
0:21:29 Yeah. No, I think a lot of people feel like that. And also, I think a lot of artists are introverted
0:21:35 as well. So it’s very hard for you to go and pitch a project or, you know, go in a room and try to ask
0:21:40 for money and things like this. So a lot of true artists I know are like brilliant artists, but
0:21:44 they’re just not good at that part of the game. And unfortunately today, you have to be good at both
0:21:49 parts of the game. So I do see a lot of benefits when we see this AI, you know, I think, you know,
0:21:54 obviously it’s not there yet, whatever tool and you’re looking at, but it’s getting there. I’m
0:21:58 excited. And also it’s going to be much more global. You know, it’s going to be much more global right
0:22:03 now. It’s still very local. You have to be in a certain part of the world to be able to get funding
0:22:08 or greenlit on a project. I’m kind of curious what your thoughts are on like AI video. Cause I imagine
0:22:12 using something like wonder, or what are you calling it now? Wonder studio, or maybe it’s
0:22:18 flow studio. I imagine using something like that and like getting the shot I want and then
0:22:23 going into some kind of AI video tool in the future and like spicing it up and changing it.
0:22:27 Yeah. Yeah. We’ve seen people do it the other way. Also we’ve seen people generate AI video
0:22:34 and then getting that to use that for animation source so they can run and get mocap out of
0:22:38 it and then get a 3d source out of it. Right. So we’ve seen it both ways. Yeah. We’ve seen
0:22:41 it both ways. My problem still with AI video, I think it’s going to get there, but it’s really
0:22:46 that editability because it’s too general. I’m a big believer in separation of elements. Like I have
0:22:50 to control my character, but I have to control individual elements of my character, whether it’s
0:22:55 hair, whether it’s eyes, whether it’s right. And then, um, I’m also a big believer. You can’t prompt
0:23:00 the performance. Performance too subtle, especially reactions. Right. If you look at your reaction now,
0:23:05 Nathan, I can’t describe what you’re doing to the word, right? It’s very, very subtle and every actor
0:23:09 is going to do it differently. So I do think this multimodality is which really what we’re going,
0:23:14 you know, even though we marketed this kind of animation and live action, we really are building
0:23:19 foundational models so we can add these models on top of it that can help us get consistent in,
0:23:23 you know, spatial awareness. It’s a big issue, as you know, you know, cause you know, right now with
0:23:28 AI video is you can get a couple of good shots, but if I go from a wide to a closeup and then do five
0:23:33 step and then I cut to another shot, will it really be consistent of how much space it went,
0:23:39 right? This is the limitation of the training data really, because these, you know, AI video models are
0:23:44 trained on a 2D video that doesn’t have this 3D awareness, the world awareness. So we’re seeing a lot
0:23:50 of companies now understanding that we’ve been from the beginning kind of making a bet of let’s build this
0:23:55 world synthetic data. And so we can then have this consistency in 3D space, but then we can also
0:23:59 control it. But as I mentioned, I do think it’s going to take a little longer than people think.
0:24:06 So the pipelines will not change overnight, at least for professionals. You know, you still need
0:24:11 passes. You’re going to still get notes. You’re going to still have to control every single element
0:24:16 of your video. So I think you’ll get there. It’s just, you know, right now it probably is going to get
0:24:21 used a little bit quicker on, you know, social media, maybe advertising and et cetera. And then we’ll see
0:24:27 where we end up with copyright thing, because a lot of people that want to make commercial thing still
0:24:32 can’t touch it because it’s still being, you know, to be determined, you know, is fair use a real thing
0:24:34 or not a real thing. So we’ll see what happens.
0:24:39 Yeah. I still really feel like none of the AI video tools have nailed character consistency.
0:24:44 You know, I feel like a lot of them are starting to claim that they’ve got character consistency now,
0:24:46 but you watch them back and you’re like, eh, do they really though?
0:24:47 It’s a hard problem.
0:24:52 Yeah. I don’t really feel like they have. I do like that workflow though, of using something
0:24:56 like Flow Studio, generating sort of what you want to see, and then sort of doing
0:25:02 like a, you know, a video to video sort of AI transfer, that kind of stuff I think is going
0:25:06 to get really, really cool. Especially, you know, I know early on when I first heard about
0:25:12 Flow Studio, when it was Wonder Studio, I remember the Corridor Crew guys, they did some content around
0:25:18 it as well. And those guys also did this big video where they were trying to use AI to make an anime
0:25:23 film. And yeah, I mean, how much easier has that gotten now, right? You can actually use stuff like
0:25:28 Wonder Studio, Flow Studio to actually go and, you know, create all those scenes now,
0:25:34 run them through a video to video workflow and get a very sort of consistent look. So I think
0:25:36 these workflows are really, really improving right now.
0:25:41 Yeah. A hundred percent. That’s one thing we also want to do in a sense of like kind of having
0:25:46 at least that look, that sort of post-vis and then really push it forward more. I think the,
0:25:51 you know, kind of rendering approach will change. We’re going to see a lot of change in it. I love
0:25:57 what companies like Pika and Runway and Luma and those guys are doing and it’s really cool and it’s
0:26:01 progress. But I think what’s really exciting for me is that, you know, before this AI boom,
0:26:08 you had VFX studios that had engineers, maybe 30, 50 on a big studio, right? And that’s who was building
0:26:12 these tools. Like a bunch of these major tools like Nuke came out of digital domain.
0:26:18 It was engineers and TDs inside of these VFX houses that had to solve a problem. And then they built
0:26:23 a software that ended up being a global software, right? But now you have millions of people working
0:26:29 on these tools. So the pace of innovation, you know, is such a growth that we’re seeing. So that’s exciting.
0:26:34 I think we’re going to see much, much quicker. And I’m excited. Also, the content creation is
0:26:39 such a main thing. Yeah. You know, people was talking about, but it’s also a little bit ironic.
0:26:46 The idea behind AI and robotics has always been, you know, how do we help humanity not do monotone
0:26:52 things and focus on creativity, which really we’re, you know, meant to do. Right. And then the first
0:26:59 industry in danger is creative industry. So it’s like, okay, an irony. Right. So yeah. Yeah. I’ve
0:27:03 seen memes going around that are like, yeah, I want AI to, you know, do my dishes and fold laundry
0:27:08 so I can be creative. I don’t want AI to be creative. So I have to go do my dishes and fold laundry.
0:27:13 I know. I know. But I think also it’s a bit like, you know, Nathan, as you said,
0:27:17 if we can get quicker to a shot, we can iterate quicker. And really to me, that creative aspect,
0:27:20 you know, obviously you have to build it ethically. And I think we need to build it inside of our
0:27:25 industry as well. I think the best tool will be built by storytellers. And it’s a good question.
0:27:30 I’m interested in what you guys think. So do you believe that we’re going to have a future? Because
0:27:36 a lot of these AI video tools are generating humans, right? Do we believe that five years from now,
0:27:44 all the celebrities and actors on the box office, top five are all synthetic, right? Are we going to be
0:27:50 okay with that? I’m not a believer in that. I do think we love, take like a certain TV shows,
0:27:54 severance or things like this. We love talking about it because we also, we love those artists
0:27:59 behind it. Right. We love their performances. It’s so, I don’t know. I don’t know. I have this
0:28:03 conversation a lot lately. So interested to see what you guys think. I mean, I’ve got my take on it.
0:28:08 I don’t know if it’s the same as Nathan’s take or not, but I still believe that, you know,
0:28:14 humans appreciate the skills of other humans. You know, people go to live theater. I still go to the
0:28:18 theater with my wife and we watch like, you know, Hamilton and we just watched a Harry Potter play.
0:28:22 We’re going to see book of Mormon in a couple of weeks. Like we still go to live theater,
0:28:28 even though movies exist, even though I could watch it on my home TV. Like I still like that sort of
0:28:34 getting to see humans be talented in front of me. And I don’t think AI is going to take that away.
0:28:38 Personally. I think people are still going to really, really appreciate it. I can generate
0:28:45 any song with AI that I can imagine now with tools like Suno and UDO and things like that.
0:28:50 But I still love to go and watch live acts. I still want to hear music that I know was created by a
0:28:55 human. And I mean, I do think AI is going to get better and better, but I think just the idea of
0:29:01 knowing that a human created that is still an important factor to the listener and to the viewer.
0:29:05 And I don’t really see that going away personally.
0:29:09 Yeah, it might take pretty similar, right? I do think we’ll have a new genre in the future where
0:29:14 it is like entirely AI video generated, you know, entirely AI generated. It’ll be like
0:29:18 just crazy stuff that humans can barely even imagine, especially like in the core and sci-fi
0:29:23 and anime and some of these categories. I think AI video will be able to do some stuff that humans
0:29:27 just haven’t done yet. Like I’ve seen some things from AI video. It’s like that’s more terrifying than
0:29:32 anything I’ve ever seen in any horror film ever. Right. Right. And so I think in some of those areas,
0:29:36 it’ll be like new genres or it’ll, you know, I think overall, I think you’re right. I mean,
0:29:39 like the human performance is so important, but I think you’ll probably start to kind of like mix it
0:29:43 too. You’ll have human performances and then like probably stuff with like, you know, Flow Studio
0:29:48 and things like that. You’ll integrate that human performance into having AI enhance it in other
0:29:51 ways, different special effects or changing the environment or things like that.
0:29:57 Yeah. Yeah. I think so too. I think as long as to me, it’s like, okay, if you’re going to have a
0:30:01 character that doesn’t exist in this world, it makes sense, right? Whether you’re going to do traditional
0:30:06 CG or you’re going to kind of generate it. But if you want to just generate a human that doesn’t exist,
0:30:11 you know, that is where I’m a little bit like, okay, maybe we’re going to have few celebrities,
0:30:15 like little Michaela, if you remember, you know, it’s like kind of a CG, but I just don’t see.
0:30:19 And I also don’t hope we have a future where, because I know these technologies are starting
0:30:23 like this, but really, you know, they’re kind of same as we’re doing with, you know, getting facial
0:30:27 performance. Like you want to transfer that performance and drive a character by human
0:30:31 performance. That’s why I don’t, I think multimodality is really where to go, but I hope
0:30:36 we’re not in a future where, you know, certain actors are kind of licensing their likenesses,
0:30:40 sitting at home and doing five production at the same time. You know, that also to me,
0:30:45 it’s like a bit of a scary future that might, that might happen. You know, some people might do that.
0:30:49 It could. I mean, so like the one possible positive there, if that technology does get that
0:30:54 good to like, you know, recreate like human performance, you know, you imagine someone like
0:30:58 a Quentin Tarantino where they’re like a control freak. Well, now they can really be a control
0:31:02 freak and like actually control, you know, really control their actors. Like they want to, you know,
0:31:06 like, like here, you know, virtual character do it exactly this way. Oh, you didn’t do it right.
0:31:11 I want to tweak you a little bit, you know, on that side, I kind of find that exciting, but, uh,
0:31:15 I’m not sure if, you know, if we’ll get there. Yeah. But I also feel like a sort of future like
0:31:20 that could lead to just like too much choice that the cream kind of has to rise to the top. I mean,
0:31:24 if you look at like streaming services right now, you’ve got Netflix, Hulu, Disney plus,
0:31:31 HBO max, like the list goes on and on and on at any given moment, you have 2 million options of
0:31:36 something that you can watch right now. Yet we still all talk about severance, right? We all still
0:31:41 talk about breaking bad and the office and some of the cream that rose to the top. And so I think if
0:31:47 you kind of hit that future where, you know, the rock is in 17 releases all in the same month,
0:31:51 there’s going to be a lot of slot mixed in there and the cream will have to sort of rise to the top.
0:31:56 Yeah. I don’t know. I don’t see that being a future that like the majority of people want.
0:32:00 Yeah. No, I agree too. I mean, similar when digital cameras came out and DSLRs came out,
0:32:04 right? Everybody’s like, everybody’s going to be a filmmaker, but I do think the artistic eye,
0:32:09 and I see it with, with, uh, you know, artists and wonder, you know, we, you know, we have internal
0:32:15 artists and we worked on some projects and some big films really. And you can see how it takes so much
0:32:19 time to train artistic eye. It doesn’t matter that your tools sped you up from a week to a day.
0:32:24 You still need to recognize when a shot is good, when something’s good. And let’s say shot, maybe
0:32:30 recognition or shot being good is going to be easier. But when his story is good, when your beats are good,
0:32:34 when your performances are good, right? That’s really hard. That’s what I learned. Like, you know,
0:32:38 Nathan, you can probably back me up on this. Like, if you look at your work 10 years ago, you can be like,
0:32:42 I can’t believe I thought this was good. I can’t believe I actually showed this to someone.
0:32:47 It’s embarrassing. Right. So I think that’s what really comes down to, you know, it doesn’t matter
0:32:50 what tool it is. Like, can you recognize when there’s a good story that you made? Yeah.
0:32:55 Is this shot and sequence work together? Right. Does this, you know, structure of your story work
0:33:00 together? So that’s why I think I agree with you, Matt. I think it’s going to be, you know, kind of like
0:33:04 what happened with music as well. You have, everybody can release their own song on Spotify and self
0:33:08 release, but it’s one is you’re going to get, as you say, like the cream on top, but I think also
0:33:13 marketing is going to be such a hard thing. Like break, who can break through the noise? Right.
0:33:17 Right. Because it’s really hard to break through the noise on the side. So I hope it doesn’t happen
0:33:22 like music though. So I feel like music’s been a horrible spot for a long time. So when you say that,
0:33:25 that’s like kind of terrifying to me. Like it’s going to be like music. It’s like, oh, it’s like suck
0:33:26 for like 30 years. Okay.
0:33:34 It’s funny. But I, you know, I do look at like, you know, like Lord of the Rings is a great example,
0:33:40 right? You’ve got Gollum in Lord of the Rings, and we know that a human played all of that behind the
0:33:45 scenes, right? You can actually see the emotions in Gollum. You can see that acting come through,
0:33:51 even though it’s sort of a CG character overlaid on top of that. Right. And something like what flow
0:33:57 studio does now is it kind of democratizes being able to do that for more people and more companies,
0:34:02 like smaller budgets can do that same kind of thing now. Yeah. And that to me is exciting.
0:34:08 Regardless of wonder, I think in AI in general, I think if we can get more people we’ve had,
0:34:12 when we launched, especially a lot of smaller production tell us, hey, we had this project
0:34:16 forever. We’re a team of summit. We could have never do it. Thank you so much. Now we can finally do it.
0:34:21 And that’s so rewarding, you know, as a founder and a storyteller, it’s very rewarding to hear that
0:34:24 because you’re like, okay, I made something that, you know, we made something that people
0:34:29 actually can, you know, do something they couldn’t before because of the financial constraints.
0:34:34 And yeah, it’s nice to hear. Yeah, absolutely. I want to go back to something you said a little
0:34:38 bit earlier in the conversation. You mentioned that you’ve actually been seeing it used for,
0:34:43 you know, marketing and by content creators. And, you know, our audience on this show specifically
0:34:49 is a lot of those like solopreneurs, small businesses, content creators that makes up a lot of the
0:34:54 audience. So I’m curious if maybe you can share some of the like ways you’ve seen it used in those
0:35:01 worlds. Yeah. I think a lot of it’s been used by people. Well, let’s maybe separate YouTube and some
0:35:07 other social media platforms. We’ve seen a lot of use on kind of VFX creators. Like you have a lot of
0:35:12 these at home VFX creators that do incredible things. And I love seeing that because again, you know,
0:35:17 going back to video copilot, Andrew Kramer, that was my intro in visual effects. I saw someone,
0:35:21 you know, who was producing and making things that inspired you to follow. So we see it a lot. And,
0:35:25 you know, kind of this short form content where people are creating content with CG characters or
0:35:30 they’re just doing it to drive, you know, their camera or something they need out of the two elements.
0:35:36 And then on YouTube side, actually, we’ve also seen it a lot on, you know, obviously 3D content creators.
0:35:40 There’s a lot of, you know, subcategory in YouTube of 3D content creators, pretty large. We’ve also seen a lot
0:35:45 of kids animated shows, which is cool to see. You know, kind of some of these mainstream
0:35:52 kid animations. I think the reason why it lends itself naturally to YouTube as well is because,
0:35:57 you know, you guys know this well, my release time in a film is two to three years. My release time in a
0:36:03 TV show is a year or two. My release time on YouTube is a week. Right. I can’t really do CG much when I
0:36:07 only have a week. Right. And you mentioned Corridor Crew. That’s what they do. You know,
0:36:10 two, three weeks, they release a new video and it’s so impressive how quickly they do it. I mean,
0:36:15 I remember I was in some indie films and you can’t afford to do any roto because your budget is only
0:36:19 10 million. And you’re like, and the producer is like, no, we just can’t. We can’t even do roto,
0:36:25 not alone green screen. And then you’re so limited. You’re like, wow, like 10 million is little.
0:36:30 When you tell someone who’s not in the film space, they’re like, 10 million is a really small budget.
0:36:36 They’re like, 10 million dollars. Like, are we talking dollars? Right. So that’s that concept.
0:36:40 And I think this new generation of, you know, YouTubers, they’re just, they’re just so crafty
0:36:44 and they move much quicker. Right. And so they find the tools, they combine a couple of tools
0:36:48 and they’re releasing content. So I think that’s what’s going to happen. I think we’re going to see
0:36:52 a bit of shift of like, you had big studios. They’re only one that can do big visual film.
0:36:56 And then indie films cannot, you got a more grounded, more like live action. And then you’ll have,
0:37:00 you know, your social media content creators. I think what’s going to happen is like your indie
0:37:05 filmmakers are going to be able to produce now major visual stories. But to me, that’s exciting
0:37:09 because now you’re going to have grounded stories, like more character driven stories that you can make
0:37:13 higher risks because studios cannot really risk. If you make something for 200 million,
0:37:16 you can’t really make an art piece. Yeah. Finally can explore new ideas.
0:37:21 Like films have been repeating the same things with barely any new ideas for a long time. So yeah.
0:37:23 You said that Nathan, you said that.
0:37:28 It’s the same thing in the game industry, same thing in the game industry, you know,
0:37:33 so like in both of those, I’m excited for AI to like kind of change that and bring new ideas, hopefully.
0:37:37 So I do think like that where indie filmmakers are, we’re going to see, you know, kind of this
0:37:41 social media content creators really push it more. So they’re going to be creating. And then I think
0:37:45 studios will just push it higher. So that’s really exciting for me. It’s like kind of like
0:37:50 we’re shifting where everything was a little step forward. Right. And, you know, I don’t know,
0:37:54 I’ve seen some in the AI video space as well. I’ve seen some things that are so creative. You’re like,
0:38:00 wow, because if I had to do this traditionally, I would have spent so much time. And then during that
0:38:06 process, you still have to keep that vision and then creation of some element will take you forever.
0:38:11 You know, animating with hand, everything will take you forever. So you might not actually
0:38:15 keep that original vision because of the limitations will push you back and you might change. You’ll
0:38:22 sacrifice, you do something. So, you know, kind of that like artistic, I guess, instinct works fast.
0:38:26 You know, like when you’re ideating something, you’re like, oh, got it this. Right. But sometimes
0:38:30 that comes and then three months later, you’re still trying to work it. And now you question it
0:38:34 million times. You’re like, is that really what I wanted? Maybe this is better. And then this I can
0:38:40 because I don’t have money, et cetera. So. Yeah. Have you seen like any like brands or,
0:38:46 you know, e-commerce companies or, you know, any sort of like marketing use cases where maybe they’ve,
0:38:50 you know, had a alien selling their product or something like that. I’m just curious about some
0:38:55 of the like really fun sort of marketing. It’s out of this world. Yeah. Yeah, exactly.
0:39:00 We have, we have seen a bunch of it and we can recognize our characters and our characters,
0:39:05 you know, we open it free for license wise. People can download it. So we’ve seen it. And sometimes
0:39:10 it’s going to pop up on my YouTube as an ad and I’m like, oh, that’s our robot. I’ve also seen
0:39:16 ads on my Instagram where it’s like a, they’re advertising a tennis game. And like, you know,
0:39:20 those ads on Instagram where you like see it and it looks so good. Then you download it. It’s nothing
0:39:25 what they advertise. It’s like not even close. Right. I’ve seen one. It’s like tennis game.
0:39:29 And I’m like, I’m pretty sure this is for studio. I’m pretty sure this is ours. Cause I can recognize
0:39:34 animation. I can see where like little clean play didn’t go fully. And I can also, I’m a tennis player.
0:39:39 I can recognize this is real footage. This is based on real movements. This is not someone animating it.
0:39:43 So it’s funny. Like you see it in those aspects. And then we’ve had some major brands also that we
0:39:48 worked with closely that did it with certain characters as well. So it’s been fun. I think
0:39:54 advertising makes sense because you have a lot of kind of spokesperson, like, like, you know, like Geico,
0:39:58 but not every, you know, brand can afford, you know, spent that much money because Geico
0:40:04 traditionally is known, you know, how, how, um, high production and I love those ads. Right. So,
0:40:09 so, but it’s not cheap to make. So, yeah, it’s been, uh, interesting to see like how
0:40:11 some of the ads, but some of the ads are just, you know, also kind of terrible.
0:40:17 Yeah. I actually think that’s a really smart approach. I think like a company should start
0:40:21 making their own little like mascot and then, you know, like the Geico Gecko mascot or Tony,
0:40:27 the tiger kind of mascot and starts using something like flow studio to have that be their brand
0:40:28 representative. Yeah.
0:40:32 It’s safer than having a person, right? You know, a person could go off and say some crazy
0:40:36 things on social media or whatever. Yeah. It’s not us. It’s not us. It’s the lizard.
0:40:44 But it’s funny how we all remember these characters, right? They’re not real, but we all,
0:40:48 as you mentioned, you know, the, the tiger, a Gecko, it’s, there’s so many of them that we see.
0:40:51 What is the fox, the Carfax, right? It’s a, it’s a fox.
0:40:54 Yeah. You got a jack in the box, a jack character.
0:40:57 Yeah. Yeah. So it’s interesting how that stays with us. So yeah.
0:41:02 Yeah. Yeah. So this is sort of the last little rabbit hole I want to go down with you. And I
0:41:06 don’t know how deep you want to go on it, but I’m curious about the relationship with Autodesk. I know
0:41:11 you guys were recently acquired by Autodesk. How did that whole thing happen? Was there a reason you
0:41:13 decided that they’re the people we want to work with?
0:41:16 Yeah. Always happy to go down the rabbit hole with you, man.
0:41:17 How big was the check?
0:41:26 Well, I’d say, you know, we started the Autodesk partnership maybe a year before acquisition.
0:41:32 And we did it because we knew a lot of our users use Maya, obviously. Maya’s being, you know,
0:41:37 a leading tool in animation and character creation for so long. So, you know, we spent a year with working
0:41:44 with them and Diana, who runs the media entertainment part of it, we kind of really aligned on the vision.
0:41:47 And then one thing that was really important for me is, you know, you always have this perception
0:41:51 as a startup founder. Like once you go in a corporation, they’re going to tell you what to do.
0:41:55 It’s going to be like, you know, they’re going to turn you an add on or whatever. So from the get
0:42:00 go, Diana is very honest in like, hey, roadmaps, your product is yours. I believe in your vision.
0:42:05 You’re still running it as your startup. And that really has been true. And every decision has been,
0:42:10 you know, our team and mine on what we’re building, what the roadmap is going, how the product is going
0:42:14 to look. I mean, I just showed you, we use Blender as one of the outputs, you know, we actually doubled
0:42:18 down on that. You know, we were like, you know, let’s have an open ecosystem. So I’ve been very
0:42:22 fortunate on that side, I got to say, because I had a lot of friends as founders that really like
0:42:28 went into a bigger company and, you know, kind of lost that control completely. So, you know, from the get
0:42:32 go out, this came to us and said, Hey, you guys been building this for a while. We want to learn
0:42:36 from you guys. And, you know, we don’t want to be steering you in certain directions. We really want
0:42:40 you to be independent and we like the product and you should build the platform and the vision you have.
0:42:44 So we always went with like, here’s my five year vision. And, you know, so far that’s been really
0:42:50 supported on the side. And to us, it made sense because we are big believers in 3D. You know, like you
0:42:55 have your AI video, but I do think that 2D, 3D approach is needed for one another. You know, that kind of
0:42:59 consistency, whether it’s latent consistency or you’re talking a spatial consistency, it’s a big
0:43:04 problem to solve. And I think you need to be in 3D space. And I think you need to be in 3D space to
0:43:09 be able to control it. What I’ve seen is a lot of startups is this, they come from research space,
0:43:14 they create something that generates, and then you’re like, okay, how do I control animation?
0:43:19 Oh, what’s a rig? How do I control a body, right? Oh, how do I control a camera? So they’re kind of
0:43:23 learning the film terms as they go, right? Their research first, and then the film term.
0:43:27 We went kind of bottoms up. You know, I’m a big believer in like, okay, I have to be able to control
0:43:32 a camera in an inch. I have to be able to control a performance. And then I have to go back and forward
0:43:38 a lot. So for us, that made sense. And also, as I mentioned earlier, I don’t think the pipelines will
0:43:42 change. I don’t think it’s going to be one AI tool that’s going to replace it all. I think it’s going to be
0:43:46 a combo because they are built for a reason by very smart people for the past 30 years,
0:43:51 right? In the industry and how the creative process works. So I’m a big believer it’s going to be a
0:43:56 mix. And also we never wanted to build, you know, our product to be like, yeah, I’m going to disrupt
0:44:01 completely. So that’s what we build. Let us fit it in until some of this research gets better. So we
0:44:07 can take more part of the pipeline, but the control and editability being the main aspect of it. So
0:44:10 that’s where our vision is really aligned on that side.
0:44:12 Nicola, are you still going to make a studio one day or like?
0:44:18 We’re still going to make movies. Yeah. Ty and I are still writing. We’re still looking at projects,
0:44:23 you know, looking for projects to produce and also write and direct. So I’m still doing that. I’m still
0:44:26 writing. I don’t have as much time, obviously, you know, running a company and writing at the same time,
0:44:31 but you know, it’s my passion. I don’t think being a part of storytelling from one way for another,
0:44:36 that’s something I always do. Cool. Awesome. Well, so is wonderdynamics.com,
0:44:39 is that the best place for people to go check it out and use it themselves?
0:44:43 Yes, wonderdynamics.com. Cool. And if anybody wants to follow you personally,
0:44:46 do you have any sort of social platforms that you hang out on? Anything like that?
0:44:53 Oh man, I’m not big on that. Yeah. Yeah. I’m more of someone who like goes,
0:44:56 you know, on Twitter and Instagram to follow things than to post.
0:45:00 Gotcha. I’m not the best at it. I’d say follow wonderdynamics socials. Like there you go.
0:45:04 Follow wonderdynamics socials. Yeah. Awesome. You’re not going to learn anything too smart from
0:45:10 my socials. Well, amazing. This has been absolutely fascinating. Thank you so much for demoing everything
0:45:15 and going down these rabbit holes with us. We really, really appreciate it. And thanks for your time on the
0:45:19 episode today. Yeah. Thanks so much for having me guys. I’m a big fan of what you guys do. And I think
0:45:23 it’s important that you’re, you know, kind of educating people in this moving environment
0:45:26 we’re in. Appreciate it. Thank you. Yeah. Thanks guys. Thank you.
0:45:34 Thank you so much for tuning into this episode. If you haven’t already, make sure you go subscribe
0:45:39 on Apple or podcast or YouTube or wherever you like to listen to podcasts. And also one last thing,
0:45:47 this podcast is up for a Webby award in the business category. So if you can do us a huge favor and go cast
0:45:52 a vote for this podcast, we might actually win an award that we really appreciate it. So thanks again for
0:45:55 tuning in and hopefully we’ll see you in the next one.

0:00:04 Hey, welcome to the Next Wave Podcast.
0:00:08 I’m Matt Wolfe, and in today’s episode, we’re talking about AI agents.
0:00:15 I had the opportunity to go spend a few days with NVIDIA out at the NVIDIA GTC conference.
0:00:20 And in this episode, I’m going to deep dive with Amanda Saunders from NVIDIA.
0:00:26 She’s been at the heart of the AI agent revolution that’s happening right now while working with
0:00:30 NVIDIA, the company that’s enabling the AI revolution.
0:00:36 In this episode, we’ll unpack exactly what agentic AI is and how it’s reshaping industries
0:00:41 from healthcare and telecom to sports coaching, as well as why that’s just scratching the surface
0:00:42 of what’s possible.
0:00:48 You’ll also discover NVIDIA’s secret blueprint for easily building powerful AI agents, as
0:00:54 well as get into some of the fears that people have around AI agents, like is an AI agent going
0:00:56 to take your job and replace you?
0:00:57 Yeah, we’re going to go there.
0:01:01 And after the interview, I’m going to share some cool clips from NVIDIA’s GTC, where I
0:01:09 got Bob Petty, also from NVIDIA, to break down exactly what NVIDIA’s DGX Spark is and how
0:01:14 they’re going to be putting AI supercomputers in normal people’s homes.
0:01:18 You can have an AI supercomputer in your home by this time next year.
0:01:20 We’re going to get into that in today’s episode.
0:01:25 So without further ado, here’s my discussion with Amanda Saunders, followed by my tour of the
0:01:27 EGX Spark with Bob Petty.
0:01:28 Enjoy.
0:01:33 Marketing in 2025 is wild.
0:01:36 Savvy customers spot fake messaging instantly.
0:01:42 Privacy changes make ad targeting a nightmare, and everyone needs more content than ever.
0:01:46 That’s why you have to have HubSpot’s new Marketing Trends Report.
0:01:51 It doesn’t just show you what’s changing, it shows you exactly how to deal with it.
0:01:55 Everything’s backed by research, but focused on marketing plays you can use tomorrow.
0:01:58 Ready to turn marketing hurdles into results?
0:02:02 Go to click HubSpot.com slash marketing to download it for free.
0:02:07 Yeah, so my name’s Amanda Saunders.
0:02:11 I’m the director of enterprise AI software here at NVIDIA.
0:02:16 So you know a lot of people think about NVIDIA and all the amazing GPUs, CPUs, and DPUs that
0:02:18 we make that go into these powerful systems.
0:02:21 Well, actually, we have a lot of software that runs on that.
0:02:26 So me and my team, we focus on how do we tell that story and bring it to customers.
0:02:31 I’ve been at NVIDIA 10 years now, and I’ve covered everything from, you know, the graphic
0:02:34 side of our business into the data science.
0:02:37 And now, of course, the hottest topic, generative AI.
0:02:38 Very cool.
0:02:41 Let’s talk about agentic AI, because that’s not a hot topic.
0:02:45 It seems like in 2025, everybody’s saying this is the year of agentic AI.
0:02:48 How would you define agentic AI?
0:02:49 Absolutely.
0:02:52 So agents are helpful software tools.
0:02:56 They are digital employees that help us augment our work.
0:03:01 And what’s really important about agents and agentic AI is that it can perceive.
0:03:02 So it sees the world around it.
0:03:04 It sees the data that it has access to.
0:03:06 It takes that information.
0:03:08 It reasons about that information.
0:03:09 It thinks about it.
0:03:13 And then it can actually make actions based on that data.
0:03:16 So it can perceive, it can reason, and it can act.
0:03:20 And that’s what really sort of sets it apart from generative AI that we had in the past,
0:03:26 is being able to take those actions, whether that action is alerting a human or actually
0:03:30 actively using a tool and making something happen.
0:03:34 And so, yeah, I think they’re really exciting because they’re able to solve complex problems
0:03:37 that we’ve never been able to do with software before.
0:03:37 Very cool.
0:03:41 So what do you think makes agentic AI powerful right now?
0:03:42 Why not a year ago?
0:03:43 Why not two years ago?
0:03:46 Why is now the time everybody’s talking about it?
0:03:48 You know, I think this has been a journey that we’ve been on.
0:03:51 You know, it started with having to have the accelerated computing.
0:03:53 You had to have the computing power to do that.
0:03:56 And that’s a problem that NVIDIA has been working on for 30 years.
0:03:59 And then finally, we’ve got to the place where we have the computing systems.
0:04:01 We also then needed the software systems.
0:04:05 And so it started with the open models that became available.
0:04:09 I think LAMA was a huge advancement and step forward.
0:04:12 And more and more models have come out recently.
0:04:16 Reasoning models, in fact, are one of the big pieces that have sort of driven that
0:04:18 and sort of added to that.
0:04:22 And then just the full ecosystem of tools and things that are required.
0:04:28 I think AI is one of those really interesting fields where the more people use it, the more
0:04:31 they want to use more of it and they want to do more.
0:04:35 And so it’s really spurring this incredibly fast growth that we’re seeing, which is, you
0:04:36 know, breakneck speed.
0:04:42 But that’s what’s really driving us from, you know, the early onsets of generative AI,
0:04:46 where you’re just spitting out answers into these worlds now where they’re actually thinking
0:04:50 things through and making really smart actions, which is just sort of cool.
0:04:50 Yeah.
0:04:55 Or in my case, I got into AI and now I just make content about it because I’m obsessed with it.
0:04:56 It’s all I ever want to talk about.
0:05:00 Because yeah, once you get into it, you really, that’s all you want to do is you figure out
0:05:01 what it can do.
0:05:05 You learn where it’s, you know, boundaries are, and then you want to do more and more and
0:05:05 more.
0:05:10 And as it continues to, you know, get better, you can do more and more things with it.
0:05:10 So cool.
0:05:13 Well, can we walk through like a agentic workflow?
0:05:17 Like give some examples of like, this is the types of things an agent can do.
0:05:18 Yeah, absolutely.
0:05:23 I think, you know, some of the most basic things that we see from agents are really about
0:05:24 being able to talk to data.
0:05:28 I think that’s probably the first thing that we see most people try to do is they have their
0:05:34 own personal data, they connect an LLM to that data, and then they’re able to ask questions.
0:05:38 And I think, you know, more recently, they’re able to ask more and deeper questions.
0:05:40 Deep research has been a big topic.
0:05:42 It’s a big topic here at GTC.
0:05:48 And the more information you can get out of that data and the more quickly you can query that,
0:05:51 that’s a pretty standard agent that we would see out there.
0:05:54 Now, then they start to get a lot more complex.
0:05:59 I think there’s some really cool stuff going on in the telecom space where they’re actually
0:06:05 using agents to improve the network so they can actually predict when there might be network
0:06:12 outages coming and they can make recommendations to the human employees who, you know, maybe you
0:06:17 want to make these changes because maybe there’s a large show in town like GTC and there’s going
0:06:20 to be a lot of traffic on that network.
0:06:25 Here are the recommended changes so that you can actually, you know, still provide the service.
0:06:27 Is that kind of thing happening at GTC right now?
0:06:28 Are they using that?
0:06:29 I wish they would.
0:06:33 I think, I think the agents are just starting to be developed.
0:06:37 So maybe next year we’ll actually see that, but it’s definitely something that we’re going
0:06:37 to see more of.
0:06:38 Yeah.
0:06:38 But yeah.
0:06:41 So those I think are, you know, some varying levels of examples.
0:06:45 And then there’s obviously hundreds more in the healthcare space, you know, helping nurses
0:06:50 and doctors become more efficient because we know they need the help to, you know, everyday
0:06:51 people like you and I.
0:06:56 I mean, I use it for almost everything I do, whether it’s in my personal life or my
0:06:56 work life.
0:06:59 Are there any like agents that might surprise people?
0:07:04 Anything that’s like really sort of like interesting that, oh, I wouldn’t have thought people would
0:07:05 be using agents in that area.
0:07:07 It’s an interesting question.
0:07:12 I mean, the most interesting one for me on the agent side is really, as you started to
0:07:16 get into video and other types of data sources, like text, I think people have gotten the handle
0:07:21 on, but adding sort of video understanding and things coming through it.
0:07:25 Actually, one of the really cool use cases that we recently did was Jensen got to throw
0:07:27 out the first pitch at a baseball game.
0:07:28 I saw the clip of that.
0:07:36 And we used a video agent to be able to critique his performance, which I just think is really
0:07:36 cool.
0:07:41 So you can imagine for, you know, athletes, whether they’re amateurs or professionals, being
0:07:43 able to use that is really helpful.
0:07:43 Yeah.
0:07:44 Yeah.
0:07:46 I can see that like golfers and stuff like that.
0:07:47 Analyze my swing.
0:07:48 Tell me what I’m doing wrong.
0:07:48 Stuff like that.
0:07:53 And it’s probably too much for the AI on my swing, but you know, it still would be helpful
0:07:54 to know.
0:07:54 Yeah.
0:07:54 Yeah.
0:07:56 Well, let’s, let’s talk about the NVIDIA blueprints.
0:08:00 Because when I was at CES, that was a big topic during CES was the NVIDIA blueprints.
0:08:03 And to me, that’s sort of like the beginnings of agents, right?
0:08:06 But there’s sort of this pre-built agent.
0:08:07 You could probably explain it better than I can.
0:08:09 Well, exactly.
0:08:12 We tried to name them blueprints so that people would get the idea that these are reference
0:08:13 architectures.
0:08:18 And what they do is they take the different building blocks that NVIDIA is offering to
0:08:22 make these agents, and it helps you with a recipe on how to put them together.
0:08:25 So it starts out with our NIM microservices.
0:08:29 And so these are packaged, containerized, optimized models.
0:08:30 Right.
0:08:34 So all the leading open models that are out there in the market, we take them, we containerize
0:08:39 them, we add a standard API call to them so that people, you know, developers out there
0:08:40 can start building.
0:08:42 And we package those up.
0:08:43 So now you’ve got the model.
0:08:45 And now you need to connect that model.
0:08:46 Right.
0:08:46 It’s right.
0:08:47 A model and it’s only, only does so much.
0:08:48 So that’s…
0:08:54 So like a NIM would be like, you’ve got a video NIM that this one NIM can produce AI-generated
0:08:55 video.
0:08:57 This NIM could produce text-to-speech.
0:08:59 This NIM could do speech-to-text.
0:08:59 Exactly.
0:09:00 Right.
0:09:00 Okay.
0:09:00 Exactly.
0:09:05 So we actually have a hundred NIM that are now available that you can download and use.
0:09:10 And they do, there are 40 different sort of domains or modalities that these different
0:09:11 NIM can do.
0:09:12 So yeah.
0:09:14 So you can take a bunch of them, piece them together.
0:09:17 And so when we built those, everyone was like, this is great.
0:09:18 We need the models.
0:09:22 We need them to run really well on, you know, our NVIDIA GPUs.
0:09:24 But then they said, well, but how do we build them into the next thing?
0:09:26 And that’s where Blueprints came from.
0:09:26 Right.
0:09:31 So they are the recipe that allows you to take the NIM, start with this recipe, piece this
0:09:32 together, and you’ll get to an agent.
0:09:36 And then what’s even better is you can then customize them.
0:09:38 So you can, you know, add your own data sources.
0:09:40 You can add your own pieces in.
0:09:42 You can combine Blueprints together.
0:09:47 So maybe you started with a chatbot and you wanted to add a digital human on the front end.
0:09:49 We have a Blueprint for both of those.
0:09:52 You piece them together and all of a sudden you have a digital avatar.
0:09:57 Who can, you know, talk to you with a full, you know, face and expressions and natural language.
0:09:58 Probably even your voice if he wanted to.
0:09:59 Yeah.
0:09:59 Absolutely.
0:10:04 And you can, you can start with pieces from NVIDIA or you can start with pieces from the
0:10:04 ecosystem.
0:10:07 We try to make it really easy to piece it all together.
0:10:08 Super cool.
0:10:10 Let’s talk a little bit about like security.
0:10:15 I know when it comes to AI agents, they can be used for both good and bad.
0:10:19 What kind of things can we do to sort of protect and secure and make sure that people aren’t
0:10:23 going and using these agents for, you know, bad actor type stuff?
0:10:23 Absolutely.
0:10:30 I think, you know, what’s really cool about AI is where it raises concerns or security challenges
0:10:32 and it can actually also help answer them.
0:10:36 So there’s a Blueprint out there for container security that does a lot of those pieces.
0:10:43 But we also have other applications that can actually track activities and make alerts based
0:10:45 on detecting some anomaly, right?
0:10:49 So that’s tends to be how you recognize that something’s doing something it maybe wasn’t
0:10:50 supposed to.
0:10:54 And AIs are just really good at that because they can, you know, consume a lot of information.
0:11:00 So I think where, you know, AI potentially can raise some concerns, it’s also in some ways
0:11:03 the answer to addressing some of those concerns, which I think is really helpful.
0:11:03 Right.
0:11:09 So like the sort of like sci-fi movie scenario of like the AI rising up against us.
0:11:14 I mean, is this something that people should be concerned about or is this something that
0:11:16 you feel there’s pretty good guardrails in place for?
0:11:20 I think the builders of these apps need to make sure those guardrails are in place.
0:11:27 But I think, yes, in general, I think the tools exist and then it’s just about us as the application
0:11:29 builders being smart about how we deploy them.
0:11:30 Right.
0:11:33 You know, in general, I think agents are really powerful.
0:11:38 They’re incredible tools that help humans, but on their own, they’re not about to sort
0:11:39 of go off the rails.
0:11:41 It really takes a human to take it there.
0:11:43 So I think, again, it is a tool.
0:11:46 It is something that can, you know, work alongside you.
0:11:50 But I don’t see the rise of the machines quite yet.
0:11:55 So along similar lines, I think one of the fears a lot of people have is like, is an agent
0:11:56 going to take my job?
0:12:00 So, you know, what are your thoughts in that sort of realm?
0:12:06 I mean, for me personally, I see all of AI as like a tool, something that makes me a lot
0:12:09 more efficient and makes it so I can accomplish more in a day.
0:12:11 But I’m curious, like, what’s your take?
0:12:13 Do you think that, you know, people need to be concerned about that?
0:12:18 I think the best thing I’ve ever heard is an agent’s not going to take your job, but somebody
0:12:19 using an agent might.
0:12:21 And that’s always the thing that I think gets framed.
0:12:23 No, I think exactly that.
0:12:27 Like we as humans have the capacity to do so much more.
0:12:29 There’s just often so many hours in the day.
0:12:29 Right.
0:12:35 And so we don’t, you know, accomplish, you know, with agents, the work that we were trying
0:12:36 to do and then just stop.
0:12:36 Right.
0:12:39 No, it allows us to go and do more and try more.
0:12:42 And so I think, you know, for me, it’s not about replacing your job.
0:12:47 It’s about making you more effective at your job and being able to do more and, you know,
0:12:48 be more powerful.
0:12:53 I think, you know, healthcare for me is one of those industries that this is driven out there.
0:12:58 There’s actually a great partner of ours in the mental health space and they’re using
0:13:03 agents to free up therapist time from all the administrative work that they have to do.
0:13:05 We know their jobs are hard enough as it is.
0:13:11 If we can free that time up, it allows them to spend more time with their patients, actually
0:13:17 deeply understanding and working with them as opposed to worrying about calendars and taking
0:13:17 notes and scheduling.
0:13:22 So I think those are great examples where it’s, no, it’s something there that’s going
0:13:23 to help your job.
0:13:24 It’s not going to take your job.
0:13:24 Yeah.
0:13:28 I bumped into somebody on the street while I was walking yesterday and they had a very similar
0:13:29 concept for their company.
0:13:34 They work with therapists and they create chatbots specifically for the therapist so that their
0:13:38 customers can go and have the initial conversation with the chatbot.
0:13:40 But then the conversations get passed along to the therapist.
0:13:45 The therapist can kind of, you know, detect when some bigger issue that they need to address
0:13:47 comes up, but it’s not replacing the therapist.
0:13:50 It’s just, hey, I can now handle more patients, you know.
0:13:53 And give them more of my dedicated time.
0:13:53 Right.
0:13:57 And I think there’s something actually interesting as you start to see chatbots and particularly
0:14:03 those with digital avatars is humans will open up in some ways to one of these, you know,
0:14:07 chatbots or these avatars in ways that they may not necessarily feel comfortable opening up.
0:14:11 So I think it’s just giving us new avenues to do things that we were already doing anyway.
0:14:13 And I think that’s pretty cool.
0:14:13 Yeah.
0:14:17 This is sort of switching gears a little bit, but do you think there’s like a compute bottleneck
0:14:17 for agents?
0:14:21 Are we going to be able to scale agents and get to agents as fast as people would want
0:14:21 to?
0:14:25 There was a narrative, I feel like the narrative sort of faded a little bit, but there was
0:14:28 a narrative maybe six months ago that there’s a wall and AI is hitting a wall.
0:14:29 Do you think there’s a wall?
0:14:31 Do you think we’re going to hit some compute bottlenecks?
0:14:36 I think, you know, this is our life’s work at NVIDIA is to make sure that we have the compute
0:14:39 to, you know, drive the world’s agents.
0:14:44 And so I think, you know, a lot of the announcements we talked about today, both on the hardware and
0:14:49 on the software side are really focused on making these as efficient as possible.
0:14:55 And I think what’s really cool is as you’re looking at this AI space, when it first comes
0:14:58 out, it’s usually big, it’s boxy, it’s maybe not the most efficient.
0:15:03 And then over time, that efficiency comes in and allows us to do the next big leap.
0:15:06 And again, that starts out, you know, big and heavy.
0:15:09 And again, gets more and more efficient over time.
0:15:13 So I think I’m not seeing a bottleneck, but I do know it’s an area we need to continue
0:15:18 to drive because I do think the compute requirements are going to continue to grow.
0:15:18 Cool.
0:15:21 Well, so let’s extrapolate out a little bit.
0:15:24 So if you’re looking like five, 10 years down the road, what do you sort of envision with
0:15:25 AI agents?
0:15:26 Where do you think this is all headed?
0:15:29 And I know that’s hard because it’s sort of exponential technology.
0:15:31 It’s really hard for humans to grasp back credentials.
0:15:32 It’s incredibly hard.
0:15:36 I mean, I can barely keep six months out, you know, and new things keep popping up.
0:15:41 I mean, I think certainly the biggest things that we’re going to see are teams of agents.
0:15:44 And that’s going to be, you know, that we’re already seeing it today.
0:15:46 It’s starting now and it’s going to continue to grow.
0:15:52 And so, you know, the way that I see it is, you know, when we first got agents and we first
0:15:57 got, you know, chatbots and things, you could ask a question and it would respond to a question.
0:16:00 And then you could ask it to do a function and it could now do that function.
0:16:02 And then you can ask it to do a bigger job.
0:16:05 And these agents are just getting more and more sophisticated.
0:16:10 So five to 10 years out, you know, we may be able to give something, you know, as simple
0:16:14 as, you know, design me my, you know, retirement home.
0:16:18 And it will come back with all of the pieces involved in that without a human ever having
0:16:19 to do another prompt to follow it up.
0:16:21 And I think that could be pretty cool.
0:16:24 So that might even be just two or three years.
0:16:27 But yes, I certainly see that.
0:16:28 I think that’s where we’re going.
0:16:29 So you mentioned teams of agents.
0:16:30 That’s really fascinating to me.
0:16:35 Like, do you have any examples of like what sort of agents would team up to work together
0:16:36 and what sort of tasks will accomplish?
0:16:37 Yeah, absolutely.
0:16:38 So I work in marketing, right?
0:16:44 And so a lot of my job is about doing messaging, writing blogs, getting imagery, building
0:16:45 demo videos, things like that.
0:16:50 And today, you know, you start with one, you know, app that can help you write.
0:16:52 And then you go to another app that helps you build images.
0:16:55 And then you go to another app that helps you write the code for the website.
0:16:56 And then, you know, right.
0:16:58 And they’re all separate.
0:17:00 And each one of them returns to me.
0:17:03 And I, you know, move to the next step.
0:17:09 I think in the future, what we’re going to see is those agents will all be connected through
0:17:09 function calls.
0:17:14 And we actually, uh, at GTC announced a blueprint that’s going to help us do this.
0:17:15 It’s called IQ.
0:17:21 And so by connecting all these together, again, as a human, I’ll be able to start putting
0:17:23 my overall requests.
0:17:27 I need to build a new website for a new announcement that’s coming out and it will be able to do
0:17:29 all those functions together.
0:17:34 And I think what’s really cool about this is it’s going to allow us to design agents that
0:17:40 solve specific tasks, but by combining them together in that sort of that composable way,
0:17:43 they’ll be able to do bigger and bigger job functions.
0:17:43 Very cool.
0:17:49 So like what sort of bigger world plot problems do you see like AI and AI agents solving for
0:17:49 us?
0:17:54 I mean, I think networking is a really interesting one because anything that has that much data
0:18:01 that humans can’t solve, I think is, is amazing, you know, digital twins and simulation of those
0:18:06 types of environments, whether it’s, you know, the climate and weather, whether it’s, you know,
0:18:08 the businesses and things that we run.
0:18:13 I think all of these are areas where agents are just going to add to the ability to work with
0:18:14 these.
0:18:16 So, yeah, certainly I think those are big overall.
0:18:22 And then I think businesses, of course, it’s going to be about, you know, providing those
0:18:25 tools so that their employees are just that much more effective.
0:18:25 Yeah.
0:18:30 And that’s, that’s not a big world problem, but it’s a very common serious world problem.
0:18:30 Yeah.
0:18:34 One of the things that I think was really fascinating and I’ve heard, I’ve, I’ve made jokes about
0:18:38 how I feel like I’m on like the Jensen tour because I’ve actually seen his last like five
0:18:38 keynotes.
0:18:41 But one of the things that I’ve, I’ve seen him talk about that’s really fascinating to
0:18:47 me is the earth too, where it’s got the, they basically couldn’t map out the weather patterns
0:18:49 and figure out weather events a lot earlier.
0:18:54 So I’m really excited to see the sort of overlap of like the earth to concept and agents and solving
0:18:57 some of the more like bigger climate type issues as well.
0:18:58 A hundred percent.
0:19:03 I mean, I think one of the things when, when we introduced earth to is this idea that we,
0:19:08 we think a lot about these problems, we have conversations about them, but it’s really hard
0:19:13 as humans to sort of visualize what, you know, some of the changes that we make in government
0:19:16 and things like that are going to do in the next five to 10 years.
0:19:18 We’re very immediate creatures, right?
0:19:19 So that’s where we’re focused.
0:19:23 But I think, you know, imagine being able to go to earth too and through an agent say, Hey,
0:19:26 you know, if we made these changes, what would happen?
0:19:32 Or if we, you know, took these steps, how could that affect the world and what more can we
0:19:32 do?
0:19:37 And maybe there are things that we’re not even recognizing that the agent could recommend.
0:19:41 Because I think that’s, what’s really cool about these agents is they’re not humans.
0:19:43 They don’t think the way we do.
0:19:48 And so by giving them all that data and giving them an earth simulation, they might be able
0:19:49 to uncover things that we’ve never thought of.
0:19:50 Yeah.
0:19:50 Yeah.
0:19:51 I think that’s pretty cool.
0:19:51 Yeah.
0:19:52 I think that’s amazing.
0:19:57 What are some of the things in the AI world, you know, agents or otherwise that have you personally
0:19:58 really excited?
0:19:59 Like what sort of stuff do you use?
0:20:00 What do you play with?
0:20:02 Like, what’s your AI stack that you use?
0:20:05 I mean, I use as many as I can get my hands on.
0:20:08 We have a lot of them in NVIDIA.
0:20:12 I mean, one of my favorites, and I know Jensen talked about it a little bit on stage, is perplexity.
0:20:13 Yeah.
0:20:13 Oh, I love perplexity.
0:20:19 I love perplexity because I think it changes the dynamic of humans and how we interact.
0:20:25 Rather than searching for information and having to spend the thought process on that, it’s really
0:20:27 about who can ask the best questions.
0:20:33 And I think that’s a really powerful change and, you know, power dynamic that perplexity
0:20:34 is given to the users.
0:20:36 Now, if you can ask great questions, you can find great information.
0:20:38 So I love that tool.
0:20:44 We have AI agents in our company that help us with everything from, you know, our benefits
0:20:48 and understanding how to make the right decisions, you know, for each employee, which I think is
0:20:51 pretty cool, into, you know, how we do our jobs.
0:20:56 Whether that’s video creation, image creation, certainly content, which you have to write
0:20:57 a lot of.
0:20:58 So, yeah.
0:20:59 So all of those, I think, are great.
0:21:04 And then, you know, I take them into my personal life in terms of organizing and planning.
0:21:09 I’m very organized at work and I don’t always save a lot of that organization for my personal
0:21:10 life.
0:21:13 And so I can hand it off to, you know, any sorts of chatbots.
0:21:16 I think ChatTPT is excellent for this.
0:21:17 It’s just pretty cool.
0:21:18 And lately, deep research.
0:21:20 So I’ve been using a lot of that.
0:21:25 And again, we just introduced a blueprint that’s going to allow deep research on your
0:21:26 own personal things.
0:21:28 And so I can imagine that’s going to be pretty powerful.
0:21:33 So for people that want to sort of stay in the loop on AI that are curious, maybe they’re
0:21:33 worried about it.
0:21:37 Like, what sort of advice would you give them to sort of stay on top of things?
0:21:41 My best advice on AI is use it.
0:21:46 It sounds really obvious, but I think, one, I think it actually alleviates a lot of concerns
0:21:48 when you understand how the technology works.
0:21:54 And by using it, you can see what works today, where the limitations are, and how it kind
0:21:55 of functions.
0:22:02 And I think it lets people see it as that tool versus that, you know, something that might
0:22:02 be scary.
0:22:06 So that’s really the first piece of advice.
0:22:07 I think that’s really important.
0:22:12 And then I think it’s about, you know, trying to identify where many of those, you know,
0:22:17 concerns might be coming from, whether that’s data or security or things like that, and understand
0:22:19 how AI can also help with those.
0:22:23 And so that tends to be, you know, one of my best pieces of advice.
0:22:23 Cool.
0:22:24 Yeah.
0:22:26 Well, let’s talk about Llama 3 and Nemo Tron.
0:22:27 Yeah.
0:22:28 So Llama, Nemo Tron.
0:22:29 Llama, Nemo Tron.
0:22:30 Let’s talk about Llama, Nemo Tron.
0:22:31 Absolutely.
0:22:33 So Llama, Nemo Tron was a really cool announcement.
0:22:38 So NVIDIA, we work with all of the leading model builders that are out there, including Meta,
0:22:39 who released Llama.
0:22:46 And what’s really cool about Llama is there are, I think it’s 85,000 derivatives of this model.
0:22:51 And so, of course, NVIDIA, we’ve got a lot of really smart people who know how to optimize
0:22:53 and make models more efficient.
0:23:00 And when Reasoning came out and when DeepSeek introduced this reasoning wave into the open
0:23:06 source, we sort of said, well, how can we bring this and make it really efficient for,
0:23:08 you know, people who want to deploy this on NVIDIA?
0:23:15 And so by starting with the Llama model, which is an incredible model, we brought in our expertise
0:23:20 to train it so that a model that previously couldn’t do reasoning now could actually think
0:23:21 through problems.
0:23:26 And so we found, you know, the data set to go be able to train the model and how to do
0:23:27 this new task.
0:23:32 And we trained it and then we made the data source open, which I think is really cool.
0:23:36 So if others want to do their own training or if they want to train a different model or
0:23:38 anything like that, that data source is available.
0:23:44 But it’s just teaching a model a new skill is, I think, a really powerful thing to see because
0:23:47 it shows how quickly the space is evolving.
0:23:51 So is there anything actually happening like underneath Llama or is it just you’ve got
0:23:55 the Llama model, but now there’s a new layer on top of it that knows how to think?
0:23:57 Is there did any like new training happen to the model?
0:23:58 Yes.
0:24:00 So, yeah, the model was absolutely.
0:24:01 So we post train the model.
0:24:05 So this is what a lot of companies out there are doing today is they take a base model and
0:24:07 they actually train it with new data.
0:24:11 And in this case, sometimes you’re training it so it has more information on a particular
0:24:12 topic.
0:24:17 This is particularly popular when you’ve got domains and industries that have specific languages
0:24:20 that they speak, things like maybe the finance industry.
0:24:24 But in this case, we actually were training it on a skill.
0:24:27 And that skill is to think through problems.
0:24:31 And I think this is what’s really interesting about reasoning is the way a reasoning model
0:24:36 works is it starts by thinking through the question that you’re asking.
0:24:40 And it breaks down that question into multiple steps and multiple parts.
0:24:45 And then it actually goes through and comes up with answers for each of those parts.
0:24:51 And then it checks to say, OK, now that I’ve got those answers, does this actually, you know,
0:24:53 come back with the right question?
0:24:58 And it continues to do that until it gets to this highly accurate response.
0:25:05 And so not only are reasoning models really good for improving accuracy, which we all know
0:25:07 that when an AI model is useful, it’s when it’s accurate.
0:25:08 Right.
0:25:11 But it also allows us to solve problems we can never solve.
0:25:12 Right.
0:25:15 So a great example for me on this is I love puzzles.
0:25:20 I love all sorts of puzzles, but particularly Sudoku, because I hear it’s good for your brain.
0:25:27 Sudoku is a problem that humans can solve, but actually traditional LLMs couldn’t.
0:25:34 There are actually almost 80 different decisions that go into solving a Sudoku puzzle.
0:25:39 And each one of those affects the other decision, because obviously based on the rules that you
0:25:44 have to follow, reasoning models, and Lama Nibodron is a great example of this, can solve Sudoku.
0:25:45 Ooh, interesting.
0:25:48 Where a Lama model on its own couldn’t.
0:25:48 Wow.
0:25:50 And so again, that’s the type of thing.
0:25:54 It doesn’t sound like, you know, solving Sudoku puzzles is going to change the world.
0:25:59 But when you look at a Sudoku puzzle, there’s actually a lot of things that go on in the
0:26:01 world that are related to that.
0:26:02 Yeah.
0:26:03 I have a supply chain.
0:26:07 I’ve got to ship items and get them to different stores around the country.
0:26:12 I happen to know there’s a snowstorm coming in and I need to make sure that my trucks are
0:26:14 taking the most efficient routes.
0:26:18 All of those are steps that impact the other decisions that are being made.
0:26:23 And so it’s actually quite a complex problem that relates in some ways to Sudoku.
0:26:24 Yeah.
0:26:28 Well, it’s so interesting too, because you take a normal model and it can’t tell you
0:26:32 how many R’s are in strawberry, give it a thinking model, and it will actually count and then
0:26:33 double check.
0:26:34 Did I do that right?
0:26:34 And then answer.
0:26:36 And get the right answer.
0:26:36 Exactly.
0:26:40 It’s so fascinating because it seems like such a simple problem to a human brain.
0:26:44 But then that’s, I think, the thing is we look at models and we think of them like
0:26:46 we personify them as humans and they’re not.
0:26:47 Right.
0:26:49 And so, yeah, reasoning models are really cool for that.
0:26:53 And we’ve seen a lot of those great examples that come out of what these reasoning models
0:26:53 can do.
0:26:55 And it is.
0:26:57 It’s just, I think it’s really interesting to watch.
0:26:57 Yeah.
0:27:01 It’s so cool to actually see the thought process because you’ll actually see the models think
0:27:03 through something and then go, wait, that’s probably not right.
0:27:05 Let me think that through again.
0:27:08 And you actually see that text come out of it thinking through.
0:27:09 And that to me is fascinating.
0:27:10 It’s fascinating.
0:27:15 And it’s a great example of the compute story we were talking about, which is, you know, we do
0:27:17 need more compute so it can think.
0:27:19 The more it thinks, the more compute it requires.
0:27:24 So it’s this really interesting cycle that we’re watching these things go through.
0:27:28 One of my favorite things to do with reasoning models is to ask them to describe things to
0:27:29 me like a five-year-old.
0:27:29 Yes.
0:27:32 And it will come up with a description.
0:27:35 It will check if a five-year-old would understand that description.
0:27:36 It will then make changes.
0:27:39 And it’s really interesting to see what models think five-year-olds understand.
0:27:41 So at some point, I’ll have to go test it with a real question.
0:27:43 Yeah, read this.
0:27:44 Does this make sense to you?
0:27:45 Does this actually make sense?
0:27:46 Exactly, exactly.
0:27:47 Very cool.
0:27:50 Well, along the same line, real quick, let’s talk about hallucinations.
0:27:53 Do you see like a path to zero hallucinations?
0:27:55 Do we want to get rid of hallucinations completely?
0:27:56 Like, what are your thoughts on that?
0:28:01 I think, you know, there’s certainly things we can do to reduce hallucinations.
0:28:07 And some of those are, you know, as simple as putting a guardrail in place that says, if you’re
0:28:12 not 100% confident in the answer or 99% confident in the answer, don’t answer the question.
0:28:12 Right.
0:28:16 That sort of, it stops the model from doing things that it shouldn’t.
0:28:22 And Nemo guardrails, which is one of the products that NVIDIA offers, helps with building those.
0:28:26 But to your point about, do we want to stop hallucinations altogether?
0:28:31 Of course, we want it to give the right answers, but in being creative, we’re asking it to come
0:28:32 up with new things.
0:28:32 Right.
0:28:37 So, it’s, you have to be able to distinguish between a hallucination and a creative generative
0:28:37 response.
0:28:38 Right, right.
0:28:41 So, I think that’s where this balance plays.
0:28:47 So, there are absolutely steps that can be taken, and depending on how targeted and focused
0:28:51 you want the model to be, you can put more and more of those sort of guardrails or those
0:28:53 policies in place that will keep it from hallucinating.
0:28:54 Right.
0:28:57 Yeah, because I think in a lot of scenarios, hallucinations are a feature, not a bug, right?
0:29:01 If you wanted to write a short story for you, you wanted to hallucinate that short
0:29:01 story for you.
0:29:02 Exactly.
0:29:06 So, I think that’s where we have to understand what’s the hallucination versus what’s the model
0:29:07 doing what it’s supposed to do.
0:29:08 Right.
0:29:10 And again, and that’s where it becomes, what’s the use case?
0:29:12 What are you trying to have it do?
0:29:17 And really think those through, and then find the right tools from NVIDIA or others to be
0:29:19 able to actually, you know, go and do that.
0:29:20 Very cool.
0:29:24 Yeah, so if somebody wanted to get started with agentic AI, they want to start playing with
0:29:26 agents and testing the waters, what do they do?
0:29:27 What are our steps?
0:29:31 Well, so, from NVIDIA, we have something called build.nvidia.com.
0:29:33 We made the URL super easy.
0:29:36 If you’re trying to build something, we have a one-stop shop for you.
0:29:39 It’s got all the models on there so that you can test them out.
0:29:43 Whether it’s the new Llama Nemetron model with reasoning, you can actually turn reasoning on
0:29:43 and off.
0:29:45 It also has all the blueprints.
0:29:47 So, you can actually test them and experiment from them.
0:29:52 And then from there, there are also steps to go deploy them, test them out, and build them
0:29:53 yourself.
0:29:55 So, I think that’s a great starting point.
0:29:56 Very cool.
0:30:00 And for the more, like, technical people that maybe are trying to develop something, like,
0:30:02 is there a place they can go play with the APIs?
0:30:04 Like, what do we do there?
0:30:05 Build.nvidia.com.
0:30:05 Same place.
0:30:06 Same place.
0:30:11 It’s literally, whether you’re, you know, an enthusiast, whether you’re a developer, whether
0:30:16 you’re actually trying to, you know, build something to put in production, this is the one-stop
0:30:18 shop because everything’s on there.
0:30:20 You can take it as far as you want.
0:30:21 You can play around with the UI.
0:30:23 You can play around with the APIs.
0:30:28 You can actually download and deploy these models on any, you know, NVIDIA hardware.
0:30:30 It’s your one-stop shop for everything.
0:30:30 Yeah.
0:30:32 Actually, that brings me to another question.
0:30:35 Does this stuff work on older NVIDIA hardware?
0:30:40 If you have a, you know, a 3080 or a 4070, can I use this stuff on those as well?
0:30:41 Absolutely.
0:30:45 The only restriction is, does it fit within the memory of the GPU?
0:30:47 But if it fits, it ships.
0:30:52 So, yes, this will run on GPUs that are out there in the market today.
0:30:53 Very cool.
0:30:53 Awesome.
0:30:55 Well, thank you very much.
0:30:57 This has been amazing, fascinating.
0:31:00 I love talking AI and nerd now, especially agents.
0:31:01 Agents is the hottest topic.
0:31:02 So, really appreciate you taking the time with me.
0:31:03 Absolutely.
0:31:05 I could also show you that in the future one day.
0:31:06 So, yeah, have a good time.
0:31:07 Thank you.
0:31:11 We’ll be right back to the next wave.
0:31:14 But first, I want to talk about another podcast I know you’re going to love.
0:31:18 It’s called Marketing Against the Grain, hosted by Kip Bodnar and Kieran Flanagan.
0:31:24 It’s brought to you by the HubSpot Podcast Network, the audio destination for business professionals.
0:31:30 If you want to know what’s happening now in marketing, what’s coming, and how you can lead the way, this is the podcast you want to check out.
0:31:34 They recently did a great episode where they show you how you can integrate AI into the workplace.
0:31:37 Listen to Marketing Against the Grain wherever you get your podcasts.
0:31:46 Hi, this is Bob Petty, Vice President and General Manager of Enterprise Platforms at NVIDIA.
0:31:59 Our first effort with AI developers, right, whether they’re enthusiasts, prosumers, or professionals, was basically using our RTX Pro cards or GeForce cards, running Linux.
0:32:05 And for many people to run Linux, we did Windows subsystem for Linux, WSL2.
0:32:21 Worked with Microsoft to eliminate a lot of what was intended for just an emulator and really turn it in, eliminate the latency blocks so that you could use it to truly develop AI and evaluate AI, not just from an accuracy standpoint, but also from a latency standpoint.
0:32:26 So historically, we’ve gone out with AI workstations or AI PCs for that.
0:32:26 Right.
0:32:30 If you’re running Intel or AMD CPU, put those in there.
0:32:39 The new RTX 6000 Pro Blackwell is a 96 gigabyte frame buffer, so you can run 70B models, fine-tune, do a lot of it.
0:32:47 A lot of the infrastructure that people are buying today in the cloud or through some of our server partners is Grace Blackwell.
0:32:50 It was Grace Hopper, now Grace Blackwell.
0:32:50 Right.
0:32:52 Grace being the ARM CPU, right?
0:32:52 Right.
0:32:58 And that’s kind of the same technology that all the big, big, like OpenAI, those types of companies are using.
0:32:58 Exactly.
0:33:03 And the beauty of that is the ARM CPU uses up so much less power.
0:33:15 So if the majority of the workload was in the GPU and the CPU is kind of a traffic manager, you wouldn’t be able to do what you needed to do for as little as power as possible so you can put more GPUs in there.
0:33:17 So hence, Grace Blackwell.
0:33:18 Right.
0:33:24 Well, you can certainly develop AI on Windows workstations with RTX Pro or GeForce.
0:33:24 That’s all great.
0:33:33 But if you need an ARM port of your software or getting familiar, that’s where Spark fits the gap.
0:33:36 And this is the same thing as the Project Digits that was announced at CES, right?
0:33:37 This is Project Digits, yeah.
0:33:39 We finally chose the name Spark.
0:33:41 We had people send in a lot of comments.
0:33:55 But as Jensen mentioned in the keynote, you know, what was a box that was like this several years ago, 20-core CPU, one petaflop is now in this little 5×5 by less than 2-inch box.
0:34:00 It’s got the C2C memory between the Grace processor and the Blackwell GPU.
0:34:02 It was like 256 gigabytes a second.
0:34:08 You won’t have that on a traditional workstation because you’re going to go over the PCI bus, right?
0:34:16 So high memory bandwidth between the CPU and the GPU, 128 gigs of memory available from us online.
0:34:20 But that’s really for the enthusiast who want the gorgeous bezel.
0:34:26 So what kind of things can I do with this now that I can’t do with my 5090 at home?
0:34:27 Good question.
0:34:32 So from a frame buffer size, there are so many more models that you can run on this.
0:34:34 You can do fine-tuning on 70B models.
0:34:37 You can put two of these together with this cable.
0:34:39 This is a ConnectX Ethernet.
0:34:42 Put two of these together, you can run a 400 billion parameter model.
0:34:44 You can’t do that on the 5090.
0:34:44 Right.
0:34:48 You can’t do the 70B on a 5090, right?
0:34:48 Right.
0:34:51 And so the size of the model is very dependent.
0:34:56 The other thing with a 5090, your memory bandwidth between your CPU and your GPU is throttled by the PCI bus.
0:34:57 Okay, right.
0:35:02 And this has, you know, cache-coherent high-speed memory bandwidth.
0:35:11 So it really enables you to test what your code might look like running on one of the data center providers, OEMs out there.
0:35:17 Because same C2C memory, cache-coherency, speed, you can do more than just say it works.
0:35:18 Right.
0:35:25 You can get it to the point where you can remove a lot of the bottlenecks, whether you’re doing a vision language model or, you know, multimodal model.
0:35:27 That’s the biggest benefit.
0:35:33 One is getting your code ready for what’s the predominant AI infrastructure out there.
0:35:33 Right.
0:35:40 But the other is testing in a way that simulates how it’s going to run, you know, when you run it on a node there.
0:35:48 And then the idea is you’re not wasting data center time or cloud time just debugging, right, or illuminating bottlenecks.
0:35:52 When you’re here, you deploy, and you scale from one GPU to end GPU.
0:36:00 So the main purpose of this was really to help spread the Grace Blackwell ecosystem.
0:36:00 Right.
0:36:06 They’re going as fast as we can make them, but they’re not necessarily accessible to enthusiasts.
0:36:07 Right.
0:36:16 Who want to use FP4 features of Blackwell, which you can do on 5090, but want to use it with some of the more popular models that have higher parameter sizes.
0:36:22 So EZBox, the one terabyte version of storage, 128 gigs, is $29.99.
0:36:25 The four terabyte version is $39.99.
0:36:29 You can reserve on NVIDIA.com.
0:36:33 Our initial go-to-market partners are Dell, HP, ASUS.
0:36:38 They’ve got their own branded boxes without the gold foil, and then we’ll expand that.
0:36:45 Yeah, I remember Jensen said something to the effect of, like, imagine it’s a cloud computer just sitting on your desk.
0:36:46 It’s not going to the cloud.
0:36:49 You know, you don’t have to worry about internet connections, anything like that.
0:36:54 It’s just a cloud computer sitting on your desk that can do all of the inference right there on the bigger models.
0:36:57 It’s an AI supercomputer on your desk, and there’s a bigger one that we’ll walk to in a second.
0:37:04 But the other thing about this is we’re not suggesting that everybody just replace their existing, you know, laptop or workstation.
0:37:08 You’ve got a GeForce laptop or RTX Pro laptop.
0:37:10 You plug this guy into it.
0:37:15 So you might do everything you need to on a 5090, run your games and everything.
0:37:20 You do an AI development or want to write an AI inferencing that helps you on your 5090.
0:37:21 Plug that into it.
0:37:34 And that’s why Jensen showed the MacBook and the Spark, because it’s really meant to be both a plug-in to Uber Assist, maybe less capable machines, whether they have a GPU in them or not.
0:37:41 And it’s not using all the processing on your computer, so you can be running AI models on this while playing Cyberpunk on your computer.
0:37:42 Exactly, yeah, exactly.
0:37:49 And, you know, just format factor-wise, cost-wise, we think it would be easy for people to do it as an add-on.
0:37:52 But certainly there’s, you know, there’s a great GPU in here.
0:37:54 We’re running games on this.
0:38:04 I wouldn’t want it as my GeForce laptop, but I’d probably want to connect it to my GeForce if I was, you know, doing AI tuning for game development or things like that.
0:38:09 So that’s DGX Spark, who has walked this way.
0:38:11 This is the RTX Pro line.
0:38:18 Our 6000 line is the one that’s, you know, somewhat akin to the 5090 on the GeForce side.
0:38:22 The reason we have this Pro line, the manufacturing of it is a very precise bomb.
0:38:25 It’s not built by many different AICs.
0:38:34 There are computing benefits on here that we perceive the gaming community doesn’t need, so some of the high-end compute performance here is going to be much better.
0:38:37 The AI inferencing performance is about the same.
0:38:38 The big difference is frame buffer.
0:38:39 Right.
0:38:42 And doesn’t this one have, like, 96 gigs of RAM?
0:38:44 Yeah, 96 gigs of RAM.
0:38:48 And if you want to get full power, we’ve got the 600-watt version.
0:38:53 We’ve got a 300-watt version that most desktops can take today.
0:38:58 And then you get the same technology in the server version that would go in a rack here.
0:39:07 So we used to call these, in the past, we’ve had A40s or L40s based on Ada Lovelace, the Ampere Lovelace.
0:39:17 We’re going to call that B40 based on Blackwell, but kind of aligned around RTX Pro, Workstation, Max-Q.
0:39:22 Max-Q is that optimal PowerPoint, and the server edition.
0:39:24 So, again, same infrastructure.
0:39:29 You can code and develop and then deploy on your RTX server and a rack in the data center, meant to save time.
0:39:32 So, these are our DJX stations.
0:39:39 Initial partners, Dell, HP, Asus, Lambda, and Supermicro.
0:39:42 We show these two here because these are their boxes.
0:39:43 This is what it will look like.
0:39:47 If you look in here, this is a GB300 board.
0:39:50 So, a much more powerful Grace processor.
0:39:54 And then an extremely powerful B300.
0:39:58 B300 is the same GPU in the latest DJX.
0:39:58 Okay.
0:40:03 So, 784 gigabytes of memory, again.
0:40:10 So, the benefit is you’re doing ARM development, a lot of memory, and very, very high-speed memory bandwidth like Spark.
0:40:18 So, you’re not just seeing that code works, you’re seeing how well it works before you chew up time on your data center rack.
0:40:22 Now, does this need like a separate CPU, like an Intel AMD kind of thing?
0:40:26 No, it’s on both the Spark and the station, we’re providing the Grace CPU.
0:40:27 Okay.
0:40:30 So, great CPU and the Blackwell GPU there.
0:40:31 Graphics out.
0:40:34 We don’t put a big graphics card in here.
0:40:35 So, this one has a 4,000.
0:40:37 That’s a small form factor.
0:40:41 The reason being, we want this to plug into a standard 15-amp wall outlet.
0:40:42 Right.
0:40:44 Might need to be a dedicated one.
0:40:47 Because, you know, it’s at 1,650 watts, I guess.
0:40:49 And we’re going to come pretty close to that.
0:40:52 Which is why the manufacturers are liquid cooling this.
0:40:55 And they started doing that in the gaming side.
0:40:57 So, some of the Alienware chassis, you’ve seen the liquid cool.
0:40:58 Right.
0:41:00 So, they’re well adept at that.
0:41:05 They will liquid cool this so we can stay, you know, thermally be good and not need to take
0:41:07 up a lot more power with a lot of fans.
0:41:14 So, again, if you’re an enthusiast, just getting started, even an enterprise where you know the
0:41:17 size of your models and what you want to get done, easily connect that.
0:41:24 If your full-time job is, like, prepping AI, developing AI to deploy to the data center,
0:41:27 you have your choice of logging it to the data center.
0:41:31 Maybe you get a virtual workstation delivered back to you and you do your job.
0:41:33 Or putting this at your desk.
0:41:33 Okay.
0:41:38 And this is, literally, it’s like one of those B300 nodes.
0:41:39 Right, right.
0:41:46 It is a server node in a desktop with graphics out, right there from a data privacy standpoint,
0:41:48 from an IP protection standpoint.
0:41:50 I’m not sending anything anywhere.
0:41:50 Right.
0:41:51 It’s right there.
0:41:56 And that’s more important than just privacy and IP protection.
0:42:00 It’s just the time and the cost of transport of data.
0:42:00 Right.
0:42:06 You might run a, you know, a 600 billion parameter model, but the data that you’re running it on,
0:42:11 whether let’s say it’s Cosmos, the BLM, all the videos that you’re going to be processing
0:42:19 and the amount of data, you’d want to just sit here versus upload all that data or even
0:42:22 if your own dedicated data center and run it there and then download that.
0:42:27 So you’ve got ingress and egress costs of data transport, all happening right here.
0:42:31 That guy will be available in the summertime frame.
0:42:31 Okay.
0:42:33 Reservable today.
0:42:34 This guy will be late summer.
0:42:35 Okay.
0:42:38 We have a founder’s edition for that because it’s cool.
0:42:38 Right.
0:42:41 Enthusiasts are going to want something on their desk, right?
0:42:43 This one is only available for the OEMs.
0:42:44 Okay.
0:42:46 The way for us to scale our enterprise businesses through the OEMs.
0:42:52 And so if you were to go to the Dell booth today or the HPI booth or Asus, you’ll see
0:42:53 their versions of these here.
0:42:59 You’ll see their versions of the Spark with the Dell blue and the NVIDIA green LED and the
0:42:59 HPE blue.
0:43:06 And that’s the way to expand the ecosystem for Grace Arm development, Grace Blackwell
0:43:11 development, expand the access to technology that normally is only available if you’ve got
0:43:13 the capital expense to put one of these racks in.
0:43:15 And now it’s at your desktop.
0:43:16 Very cool.
0:43:17 Yeah.
0:43:17 Amazing.
0:43:18 Well, thanks, Bob.
0:43:21 This has been really informative and I appreciate it.
0:43:21 Thank you.
0:43:21 Yeah, no problem.
0:43:22 Thank you.
0:43:41 Thank you.