AI transcript
0:00:11 [music]
0:00:15 Hello and welcome to the NVIDIA AI podcast. I’m your host, Noah Kravitz.
0:00:21 The AI community is a buzz with agents. As NVIDIA CEO Jensen Huang said in his CES keynote this past
0:00:27 January, “Agentic AI represents the next wave in the evolution of generative AI and a multi-trillion
0:00:32 dollar opportunity at that.” AI agents enable applications to move beyond simple chatbot
0:00:38 interactions to tackle complex multi-step problems through sophisticated reasoning and planning.
0:00:42 They’re expected to become a centerpiece of enterprise AI systems going forward.
0:00:47 Our guest is John Heller, co-CEO and founder of First Hand, whose brand agent platform is
0:00:52 transforming how advertisers and publishers engage consumers online. First Hand is also a
0:00:57 member of Inception, NVIDIA’s program for startups. John, welcome and thanks so much for taking the
0:01:02 time to join the AI podcast. Hey Noah, thank you. It’s great to be here. Very excited.
0:01:06 So let’s start with the basics. If you don’t mind, tell us a little bit about First Hand.
0:01:12 Sure. As you said, First Hand is building the brand agent platform and that’s because when we
0:01:19 see AI, we believe it is itself a new medium, not just a technology. Perhaps I can explain that a
0:01:26 little bit about how we came to be. So I’d been working in advertising tech since the
0:01:32 internet started in the 90s and worked at DoubleClick for quite a long time. More recently,
0:01:37 it was the co-CEO and co-founder of FreeWheel, which is now Comcast ad-serving platform.
0:01:43 And I’ve been doing ad tech for a long time. Actually, when I left Comcast, I went back to
0:01:48 school finger quoted in AI, so online courses, because I wanted to work in something that was
0:01:54 much more broadly applicable and much earlier in its life. And it actually started by working on
0:02:00 reinforcement learning robots to act as minions and artificial teammates at video games.
0:02:05 And that’s how I started learning the guts of it. I’m biting my tongue, so we don’t take us off
0:02:09 course, because that sounds fascinating as well. Oh yeah, that could be a separate conversation.
0:02:14 But I’d been working in the gaming world and some of the generative AI abilities for gaming assets
0:02:21 when language models really came out. And something struck us, something very powerful,
0:02:29 which is, and this is a metaphor for the math inside, but AI now understands the ideas and
0:02:34 intense needs you may have from what you’re reading, what you’re watching, what you might
0:02:40 ask it outright. And it can go find the right response or take the right responding action.
0:02:43 And everything is presented to you in a very natural human way.
0:02:50 And if you back up the step and think of that happening all the way through a consumer’s use
0:02:54 of the digital world from when they’re searching and becoming aware of things they might need,
0:02:59 when they do some investigation and read up on products or services, when they go to
0:03:04 browse or shop, when they buy, all of those modes change pretty fundamentally. They don’t
0:03:10 replace, we think they get enhanced, because instead of the world of the past, where I maybe
0:03:16 did a search, got some directions and a link went to a place, read up on something,
0:03:20 browsed for something, went to saw an ad, maybe went to another place to try to find the version
0:03:27 I want. Those are all sort of separate hops, the internet where it’s the same content everybody
0:03:35 sees. AI instead is going to understand and learn at each moment what it is you need. And as with
0:03:41 most things AI data is the core, the people who have the most and best data about a product or
0:03:46 service are the brands. They are the retailers and the people who sell it. So they can create
0:03:52 brand agents, which means your experience on the internet at all of those moments in the journey
0:03:57 from first learning about it to figuring out what the right configuration is and comparing and
0:04:03 browsing and buying is going to adapt on the fly through these agents. So it doesn’t replace the
0:04:09 web, but it changes things from you looking at stuff someone wrote to something that’s
0:04:14 partially adapting to what you actually need, understanding your needs. But the agents that
0:04:19 are doing that for you are from the retailers and brands themselves, because it’s their data that
0:04:25 is what you need. And that sort of changes the internet and the kind of your internet for both
0:04:33 parties. And that really makes it a much more adaptive form fitting to what you need in enhancement
0:04:39 to what’s already there. But that from a media marketing perspective changes everything.
0:04:44 Yeah, sure. It’s interesting to hear you talk about that from the media marketing perspective.
0:04:50 We’re recording this in early 2025. Happy New Year everybody. Last year I had, especially towards
0:04:55 the end of the year, had a lot of conversations with people about agents and agent to AI,
0:05:02 a few of them on the podcast. And as a sort of consumer end user of things, I keep thinking about
0:05:07 the, and I like the way that you put it, changing it from the web to your web. I keep
0:05:13 thinking about it from the perspective of, oh, maybe in the future, I won’t have to go seek out
0:05:18 the information I want because my agents will know me and know what to go find and bring it back to
0:05:24 me. But it’s interesting to hear you talk about the agents actually being on, I guess, the advertiser
0:05:30 brand publisher side, because they already have the data about my habits. Well, it’s not that they
0:05:34 have the data about your habits. They haven’t about their products and services. So I’m honestly
0:05:39 correct. If you look ahead, consumers have, and we’ll have agents that understand them,
0:05:45 but that’s half the conversation, if you will. Got it. Okay. Those agents know what they need,
0:05:51 perhaps, and what they’re trying to do, but no one’s going to know more about the uses and
0:05:59 values of their sauces or beauty products or furnishings or cars than the brands and the
0:06:06 retailers that sell them. So we see sort of two halves to this. There’s the agents that consumers
0:06:12 may use, which may just be enhancements to the web as is. They may grow from that into something new,
0:06:16 but the folks who can answer their needs and help them not just to learn more, but actually
0:06:21 finish and convert, take the action are the brands and publishers. So for them to be able
0:06:26 to equip their agents, but then send them to all the different places consumers need,
0:06:30 that’s a big change. You’re kind of indicating the knowledge and expertise
0:06:33 out to where the people needed at their different moments on their journey,
0:06:37 which is wildly different than having to get them all to come to you.
0:06:43 Right. Got it. No, it makes a lot of sense. And so first hand is young. When was it founded and
0:06:48 what was kind of, you got into this a little bit, but what was kind of the moment where it came
0:06:53 together and you decided to launch the company? So I was working in generative AR trying to figure
0:06:59 out how to make gameplayable assets much more easy to produce and some commerce ideas around that.
0:07:05 When the language models first came out and we understood this change, that the folks who have
0:07:10 the best information about the products and services you might need are going to want to
0:07:14 speak in their own voice through their own agents, but everywhere. Right. And that that changes
0:07:19 everything and that the folks who have additional information, the publishers who have the expertise
0:07:26 on the trends or topics in general also want to have a voice in that conversation. That really
0:07:31 changed things again from that sort of the internet to your internet, then that sort of was the spark.
0:07:36 That’s like that’s white space. That is not how any advertising is done today. That’s not how the
0:07:40 technology stacks work. That’s not how the data management works. It’s not how the measurement
0:07:46 works. And from an entrepreneur’s point of view, white space is fantastic because you have to,
0:07:51 first of all, it’s just fun because you have to invent it. Right. But you’re not retrofitting or
0:07:56 dealing with, there’s less legacy to try to deal with. And you can, and at this point,
0:08:00 having done marketing for this many decades, there’s a lot of lessons learned about now
0:08:04 that I can start with a clean slate, I’m going to do it this way. And then the power in AI is just,
0:08:11 it’s astounding. So that was just very exciting. And it does change retail, commerce, advertising,
0:08:18 martech, customer research. So that’s a lot of playing fields. So we, I need co-founders that
0:08:22 was thought number one. But having done this for a while, I have just very good friends that I’ve
0:08:30 worked with for decades. And Michael Rubenstein, my co-CEO is someone I’ve known since late 90s,
0:08:35 double click, and way my CTO co-founder, someone that was instrumental in the engineering and
0:08:40 freewheel, you get to work with your friends that are industry experts in white space on
0:08:45 something that is this exciting, not just because of the change in the industry, but,
0:08:50 you know, just a little full disclosure here. The technology is just really fun to understand
0:08:56 and work with. Isn’t it? Yeah. It’s fantastic. Yeah. And then just, if I may, because advertising tech
0:09:02 is just a massive, massive scale, the inference volumes and speeds and things, just a total
0:09:07 different world. Working with companies like Nvidia on figuring out how to make that actually
0:09:13 properly for that kind of an industry. I mean, it sleeves rolled up thumb. Right. Yeah. We formed
0:09:24 in August of 2023. So not too too old, but we are 27 people now. We were live in the summer. We’ve
0:09:28 got, you know, we haven’t done case studies public yet, but we’ve got, you know, actual results that
0:09:35 when you actually adapt the content and the websites and the journey, if you will, for consumers
0:09:41 into something that’s fitting the needs for them, the effects are or multiples greater than
0:09:44 traditional advertising. I don’t want to harp on the nomenclature here, but I want to ask you a
0:09:50 little bit about the word agent and brand agents in particular. I read a definition of agents
0:09:55 in the intro, but, you know, for every conversation I’ve had with people about agents, there’s been
0:10:01 at least one new kind of working definition about what that might mean. Can you talk about the concept
0:10:06 of a brand agent as firsthand uses it? And specifically, how should companies be thinking
0:10:12 about brand agents as different from other types of AI agents? So I like to think of it from what
0:10:16 are you trying, what’s it supposed to accomplish? Right. So when I think about agents as I read
0:10:25 about them, most of what I’ve observed is here’s a somewhat autonomous execution engine that will
0:10:30 go create productivity by making something more efficient. That’s a solid definition. I’ll take
0:10:36 it. Yeah. That’s fantastic. That’s a wonderful, useful, make what you already do work better
0:10:41 use of AI. Right. But when you’re talking about how to participate in a brand new medium,
0:10:46 it’s a very different set of use cases because you’re talking about how do I help a prospect
0:10:51 and a consumer through, call it a connected journey from being aware of something to learning about
0:10:55 what you might want, to finding what version you want, to actually getting it. There’s many
0:11:01 different objectives and purposes there. So for us, an agent is actually a composable collection
0:11:06 of AI capabilities. So you might even think of an agent as containing many agents. We use the
0:11:13 right capability. And you would do one with a particular composition for say, awareness raising,
0:11:19 a different one for acquisition, maybe different one for conversion improvement in upsell. And
0:11:24 what that means is you do a few things. First, underneath the whole product is something called
0:11:31 Lakebed, which is a kind of intellectual property rights data management platform so that agents
0:11:37 in whatever form they take can cooperate on data across parties. And just to clarify, Lakebed is
0:11:41 a first-hand product. Yeah. It always comes back to data in order for an agent to do whatever its
0:11:47 job is. If it’s happening out, if it’s for a retailer and it’s happening out on a publisher,
0:11:52 it needs to be able to, because it benefits both parties, safely use the information of the
0:11:57 publisher about, if you’re on a website about decorating and decor and home and furnishings,
0:12:03 and a retailer wants to help you with that, then they need to know not only all of their products
0:12:08 and services and offerings, but everything about what you’re looking at and the content of the page,
0:12:12 and then it’s beneficial to what parties, if it’s done with full intellectual property and
0:12:17 privacy control rights, because no one exits an event with data they didn’t enter the event with,
0:12:22 if you will. But it makes the agent capable. And then that agent might help you instead of
0:12:27 seeing what you would traditionally see as an ad, it may say, “Look, are you trying to furnish a new
0:12:32 home? Are you trying to refresh for holiday entertaining? Are you downsizing and looking
0:12:37 to sort of deal with a smaller property?” And it learns, as I said, this is the new medium of
0:12:42 effect. That agent, even though it’s happening, retail agent is happening on the publisher property,
0:12:47 it’s learning about what you need, and it’s presenting the information. And here’s where
0:12:52 the capabilities come into play. It could simply be content presentation, if you want to have open
0:12:56 Q&A, you could add a chat capability. You could close the capabilities that make sense for what
0:13:01 you’re accomplishing. But it learns, and let’s say if someone is furnishing a new home, then it’s
0:13:05 going to present them with like, here’s the five key starter pieces you need, and here’s how to
0:13:11 turn a house into a home quickly with color and accessories. And then this is new medium again.
0:13:15 Once someone engages with that agent and the retailer, that’s the first party relationship
0:13:20 between the retailer and the consumer, which is an arcane element in marketing. But that means that
0:13:25 when that consumer leaves that agent and arrives at the retailer, the retailer is allowed to
0:13:30 understand everything that occurred in that agent out in the wild. So that means when they arrive,
0:13:35 instead of arriving in the past, it would be the landing page everybody sees, the landing page itself
0:13:41 can be another agent that is therefore composing the content of the page to fit the needs of what
0:13:47 brought you there in the first case, and offering up additional help to find the right configuration
0:13:52 that makes sense in this need, etc, etc, which is beneficial to the consumer, but obviously helps
0:13:56 sell more product and is beneficial to the retailer and obviously creating great value
0:14:00 for the publisher and selling that media opportunity. Everyone has to cooperate in a way
0:14:06 that’s safe for each other. And this is why we think we’re seeing results multiples and multiples
0:14:10 better than traditional ads, is because you’re creating a utility for the consumer that happens
0:14:16 to be directly beneficial to the publisher and retailer doing it, or the brand if they’re the
0:14:20 ones who are on the other side of the conversation. Right, right. I’m going to ask you one of those
0:14:24 questions where I think I know what the answer is, and usually those are the ones that are good to
0:14:30 ask because I’m often quite wrong. Can you kind of describe the difference sort of in as plain
0:14:36 terms as you like between what you’re describing and the chatbot experience that consumers may
0:14:41 be having now? So the key is that in different circumstances with different needs, if it’s
0:14:46 on a search engine, if it’s in social, if it’s on a publisher, if it’s on your own retail site or
0:14:51 the manufacturer brand’s home, their site, what you want to do, again, starts with what its intention
0:14:58 is, then we separate the brain of the agent, which is a few pieces from Lake Bend. What is it
0:15:03 allowed to know? So a retailer may want it to know only about their house brand products,
0:15:07 or they may want it to know about every product they offer. So that’s sort of a business choice.
0:15:11 Right, right. And then you choose the sets of capabilities you want it to produce.
0:15:17 So it may be simply content, the appropriate content selection and presentation. So it’s
0:15:22 essentially building a mini website, if you will, on the fly to help them with set up a choose your
0:15:28 own adventure structure. If you want to add to that chat so that they can ask a question you
0:15:32 didn’t interpret appropriately, then that’s just additional capability. And that’s the agent that
0:15:36 may happen out in the wild. You sort of wrap that up as a campaign and send it out there and say,
0:15:41 I want it to happen this many times in these places. And if someone goes through that agent
0:15:46 onto the destination site, so for example, and what we’ve seen so far, folks are, when they’re
0:15:51 looking at an agent, they’re going to the page about the actual thing that that agent talked to
0:15:56 them about multiples on multiples on multiples higher rates than an ad does. And they’re going
0:16:01 to many, many, many of them. So hundreds of different ones, which means you’re getting people
0:16:07 to what they actually need more efficiently. And that’s not a chat experience, it’s more interwoven
0:16:13 into the way people already serve, because that last part of the agent is something we call a
0:16:18 frame. And you can call that its UI. What is the user experience you want to present? Is it image
0:16:24 rich or is it text centric? It’s almost more of an app in a way than an agent, because how it
0:16:30 appears and the way you interact with it is at least as important, frankly, for marketing
0:16:34 as what it’s able to say. Right, no, that makes a lot of sense. And then, frankly, it continues its
0:16:40 help to you through your connected journey. So search from the publisher onto the retailer,
0:16:44 perhaps onto the brand, it has contacts throughout, which I think, you know, no one likes to start
0:16:50 from scratch again. Again, so an agent is this mix of what is it supposed to know? What is it
0:16:56 objective? Is it content presentation plus open dialogue? Is it actually constructing the page
0:17:02 you’re on while it’s talking to you? So you pick the set’s capabilities, then you layer on that UI.
0:17:06 But what that does is that’s the changing the internet to your internet, because it’s taking
0:17:12 those discrete hops. I searched for something, I read an article, I saw an ad, I clicked on it,
0:17:16 I went to the site, I started from the beginning, I found the product, and it’s turning that into
0:17:21 one connected journey. Right. It understands and helps me with what I need at each of those steps,
0:17:26 and it’s with me the whole way with full context, because that’s the relationship between me and
0:17:32 the retailer. The location is less important. So you’ve spoken to this, but maybe we can put a
0:17:37 point or you can put kind of multiple points on it. But what are some of the most exciting
0:17:44 areas of potential or even specific problems that brand agents can solve for marketers and kind of,
0:17:49 you know, ways that the retail experience, I mean, you’ve been talking about it with this idea of
0:17:55 composing, you know, your web on the fly to suit what you need. Are there other specific problems
0:17:59 or big things that you’re excited about when it comes to how the experience is going to be
0:18:05 transformed by brand agents? Well, I think the very, very fundamental change is being able to take
0:18:11 the expertise and the knowledge of the retailer or the market or the brand and put it in the moment
0:18:17 wherever the consumer is. So taking their ability and putting it into an agent and sending that
0:18:23 agent everywhere. So you’re helping the consumer at all those moments is a massive change because
0:18:29 that’s why instead of trying to sort of common denominator, pick them one message and figure
0:18:34 out the targeting about exactly what this one thing should match against the AI. This is what AI
0:18:39 is designed for. It understands and learns what’s the right thing in this situation, which means
0:18:45 because your internet is just as true for the marketer as it is for the consumer. So it’s taking
0:18:49 of all the things you could do. This is what is useful to this person here in this circumstance.
0:18:56 So that means instead of having to fine tune and sort of try to hit the bullseye, you can kind of
0:19:01 let this find all the bullseyes. And you can see that evidencing by the fact that it’s sending
0:19:06 people to hundreds of different destinations instead of just the one or two. But then if you
0:19:12 can continue that connected journey when they arrive and have that fit to what they need,
0:19:17 and that covers everything from like food and beauty to finance and auto and all forms of
0:19:21 retail, the odds of them getting to the thing they need and getting the version of it that
0:19:25 they need and being able to ask any follow on questions about how to actually pick the right
0:19:30 set of it. We don’t have data on this yet, but I cannot wait to do conversion tracking because
0:19:37 I am very excited about that. So I think that is a wonderful opportunity that has never existed
0:19:42 before as opposed to a problem solved. Right. Something you said earlier, the way you described
0:19:49 the current process of you have something that you want to find or you need or you’re decorating
0:19:54 your home and don’t know where to start. And so you do a web search and you get a list of results
0:20:00 and you pick one and you go to it and then you read the page, scan it to find what you’re looking for.
0:20:05 And the idea of that process changing, it makes me think of things that marketers, I’ve read and
0:20:13 talked to marketers saying that AI may fundamentally change the whole web experience starting with
0:20:18 breaking SEO. Do you have thoughts on that? So a couple things here. The world has been built
0:20:24 at this point where there is a fairly central place to go ask your question, looking backward.
0:20:33 Right. And that sent traffic out to the places to go read the details where people might run ads
0:20:38 to then get you to go to the place where you can consummate the purchase and learn more about the
0:20:44 purchase. But that hop, hop, hop is completely changing because the knowledge is pushing out
0:20:49 to the consumer at all those different places. They still want to go to publications and read
0:20:54 about the details and what’s the best version and then to understand what’s new and trendy. So
0:21:00 everything still exists. It’s just more adaptive and it’s more sort of distributed out. I think it’s
0:21:06 less centralized trying to get someone at that first question and get them at all the moments
0:21:10 of question throughout all their moments of need on that journey. And if you can make that smooth
0:21:15 for someone and in context throughout, that’s so much better for the consumer. But it is a,
0:21:19 it is still an enhancement to the internet. I think the way digital is great. It just makes it
0:21:23 much more, it’s the internet to your internet. It makes it richer and easier to do stuff.
0:21:30 Yeah. I’m speaking with John Heller. John is co-CEO and founder of First Hand. He’s an ad tech
0:21:37 veteran, started First Hand with friends from his ad tech days. And they are using AI, AI agents
0:21:43 specifically to transform the way that publishers and advertisers connect with consumers, engage
0:21:47 with them, help them throughout their journeys. Everything John’s been talking about to this
0:21:53 point. John, you’ve been speaking about all the different ways sort of conceptually and concretely
0:21:59 that AI, AI agents, the brand agent platform that First Hand is building can change the way
0:22:04 that consumers, you know, go on their journeys and that advertisers, publishers engage with them.
0:22:11 Are there specific industries, specific experience types? I mean, to me, this sounds like a
0:22:17 transformative thing that’s, you know, far reaching and then some. But are there certain areas that you
0:22:21 think are really, you know, right for transformation and you’re particularly excited about?
0:22:27 So one of the reasons this has been so much fun and starting something that’s so white-space
0:22:33 is so enjoyable is you are learning things that you didn’t understand. I started thinking this is
0:22:39 for high consideration stuff, financial services, automotive, telecom, consumer, things where you
0:22:46 have lots and lots of details to go figure out. Yeah. Nope. I mean, yes, but more, you know, food,
0:22:50 like what dishes would make sense for the holidays and how can I sort of help you figure out the
0:22:57 shopping list for a great Thanksgiving? Or did you know that these products just lead for balanced
0:23:04 healthy life? So what we found is as much interest in how this can benefit your lifestyle in that
0:23:09 path as much as how much this product is useful for its features. So the interest has been very
0:23:15 broad and it’s not just been broad from industries, it’s been broad for sort of where in the distribution
0:23:21 chain it fits as much on their own properties as it is distributed out across where they market.
0:23:26 So it’s as much customer as prospect. Right. We build a platform. This is the reason our
0:23:31 agents are very composable and collections of abilities as opposed to just an agent is we
0:23:36 thought that they would have broad application and the breadth surprised me. Yeah. Talk a little bit
0:23:43 about the experience of building brand agents and when, you know, in listening to the past 20 minutes,
0:23:50 I’ve been thinking about LLMs and the chatbot experience and using RAG to, you know, put
0:23:55 my company’s specific product information or what have you in there to let the chatbot find,
0:24:00 you know, the correct information. How is the process of, you know, thinking about and building
0:24:06 and deploying agents and brand agents specifically? What’s that like? Is it similar to building with
0:24:12 an LLM and RAG? Is it radically different? Talk about that if you would. It’s much more nuanced.
0:24:16 And the reason I say that, I’ll use an example. If it’s a retailer with several hundred thousand
0:24:23 SKUs, then the objective they’re trying to solve for is different than someone who sells
0:24:29 like three packages, an energy package, a sports package, and a family package. So that affects
0:24:36 how you retrieve, right? You actually, basically the platform we built is exceptionally composable
0:24:43 for many, many reasons because the thing you’re making the agent solve for affects the way you
0:24:47 embed and tokenize. It affects the way you retrieve. It expects the way you evaluate what you
0:24:54 retrieved and ranked. It affects how you generate and the instructions you give it and then how you
0:24:59 evaluate what was generated to make sure it was doing what you needed it to do. Because one of
0:25:04 the interesting things about marketing is there’s almost as many rules about what not to do as there
0:25:10 to do. So there’s quite a lot of highly flexible control layers around making sure it didn’t do
0:25:14 the thing it wasn’t supposed to do as well as did what you wanted. And all of that needs to be very
0:25:20 composable to the objectives of the parties involved. It’s always a supply chain across
0:25:27 multiple parties. So lots of rights management. And then it’s generating information that’s never
0:25:33 existed before because every agent, because that’s a first party interaction, is not just
0:25:37 generating metrics. How many times does someone engage with it? What did they click on? Did they
0:25:44 go somewhere? Did they click on a citation or call to action or a product image? But it’s recording
0:25:48 the transcripts of what was actually presented. What was the text of what you have a full survey
0:25:53 dialogue transcript of all these interactions, which is almost as if all your marketing is also
0:26:01 a customer research survey at the same time. And that can be used to learn how to be better.
0:26:06 Yeah, not so wealth of data. So it has feedback loops, again, completely IP rights managed
0:26:12 protected feedback loops, so that it’s improving for the person whose data it’s using to improve.
0:26:18 But those feedback loops make it understand in this context, this configuration for this piece
0:26:24 of the puzzle fits. So I think the key thing is high configurable composability. And then also,
0:26:30 just frankly, in enterprise world, people like to use like we made it so the foundation underlying
0:26:34 models you want to use or a choice, right, you know, the one you want as a cartridge, partly
0:26:39 because different companies have different desires, or maybe they’ve tuned a model, especially for
0:26:45 their world that they like. But this is going to change so fast, we need to be able to plug cartridge
0:26:50 A out and put improved cartridge B in and let the rest of the framework still operate. Right. So
0:26:56 it is, I say the composability and the learning and checking whether it did the right thing and
0:27:02 can it do better is the key part. When it comes to learning and checking to see if it did the
0:27:08 right thing, evaluation, and maybe even dealing with hallucinations, is there or are there,
0:27:13 you know, a genetic specific techniques that you’re either using or sort of discovering is,
0:27:19 you know, evaluating a response different when you’re working with agents? Or is it kind of the
0:27:24 same? I think the key difference is you have to have lots of different types of evaluations.
0:27:30 And this is, everything’s gold-dependent. So if it’s this type of a marketer with this type of
0:27:35 an objective, if it’s a recirculation agent to increase use of time on site for a publisher,
0:27:39 that’s a different case than if it’s a retailer trying to wrap people to the right product or a
0:27:45 brand trying to get people to understand that these are great recipe shopping lists for this holiday.
0:27:51 So that doesn’t just change what you retrieve and how you generate, but it changes which sets of
0:27:56 evaluations you invoke. So that configurability matters a great deal. And all these feedback
0:28:04 loops are sort of configuration specific and company specific because you would tune a house
0:28:10 brand, you know, holiday effort is different than what you would do for everything I have on my
0:28:16 shelves, just people to the site type of thing. So I think, again, it’s being able to manage all
0:28:22 that to fit its purpose that is proving really important. Perhaps a naive question here. So
0:28:29 forgive me, but is sort of troubleshooting and running tech support for your customers made
0:28:35 infinitely more complicated because of all these, you know, nuanced specific use cases? Or is it
0:28:40 just kind of a new version of things that people in the ad tech world have been dealing with for years?
0:28:47 Interestingly, we have an entire experimentation module, which means you can run this in sort of
0:28:54 laboratory mode, right, right, watch all of its behaviors before it goes in front of consumers,
0:28:59 where you get yet another important feedback loop, which is, you know, no one is more expert
0:29:04 in the quality of their output and the appropriateness of it than the editors and the marketing
0:29:10 staffs of the brand. So they get to go into the experimentation module, have it run in lab mode
0:29:14 and say, that was great. That was good. That was great. And that creates yet another feedback loop,
0:29:21 so it improves to past muster. Once you’ve got it to a state where you’re happy with what it does,
0:29:28 that reduces the once it’s live issue set. And again, to go back to the capabilities list,
0:29:33 this is how it’s different from chat. If you picked what we call a guided conversation,
0:29:37 which is sort of a choose your adventure motif, it’s only picking from the content you’ve already
0:29:41 told it’s allowed to know. And it’s simply surfacing the right things, phrase the right way in the
0:29:46 right moment. And this gets back to how agents are built. If you wanted to add chat to the bottom
0:29:51 of that, then someone can ask whatever they want. That obviously triggers a whole different set of
0:29:57 protections and evaluations to keep that on track. So again, that configurability is quite important.
0:30:01 Kind of at a really high level, how do brand agents, how does your technology
0:30:08 change the way that your customers, the brand’s publishers, think about how they want to engage
0:30:14 with customers? So in the past, you would try to get a list of target identities. So you’re
0:30:20 trying to find a list of folks, because that list is a proxy for flags that mean what they care about
0:30:26 and what they’re interested in. And then you would create a message that was appropriate to that list,
0:30:31 and then you got to go find where those folks might be and try to show that message to them.
0:30:36 And that’s when they’re out beyond your borders, if you will. When they arrive at your site,
0:30:40 you don’t know how they got there with any particular detail, and you’re sort of again
0:30:45 starting from scratch. So what we would say is you have a great set of products and services,
0:30:51 figure out how you want to talk to your different typical customer as opposed to specific target
0:30:59 list, and then let the AI do what it does. Put it in all the places where they might be
0:31:04 beyond your property and on your property, and then it will surface to them the things that
0:31:09 matter to them, and they’re going to end up coming to you already having indicated what’s
0:31:14 important and being qualified. So you receive them. So here’s a big change. You should have agents
0:31:21 present when they arrive that receive them with that knowledge and start from step five, not step
0:31:29 zero. And that changes how you think about campaigns, how you construct them. Interestingly,
0:31:35 this is a bit arcane for ad tech, it’s vastly simpler operationally to set this up than to try
0:31:42 to write like a hundred line item insertion order, trying to manually, this is a classic AI,
0:31:48 do I build a massive if then else statement, an IO with hundreds of line items,
0:31:55 or do I have an agent able to figure it out? Yeah. So what’s next? What’s next for firsthand?
0:32:00 What’s the future of AI agents like? Take it wherever you like. How do you envision the technology,
0:32:06 the use of it, the implications kind of evolving over the next, I’m going to say three to five years,
0:32:13 but you can tweak that to suit as you like. Well, I think the ability for the internet to come
0:32:18 your internet has a lot of the new medium, it has a lot of implications. So I think
0:32:23 where I see agents is they’ll start to be more wisdom around this is the right configuration
0:32:27 at this point in the customer journey. This is a productive configuration, these abilities,
0:32:32 it’s going to be this guided conversation with presentation, you know, when they arrive,
0:32:36 I want page composition, but only in these portions of the site. So I think the very
0:32:43 construction of how marketing campaigns and sites are put together changes. And when it
0:32:47 becomes this connected journey where I’ve understood the consumer all along the way,
0:32:54 and I’ve helped them all along the way, and I have that data, I think that data starts to drive
0:32:59 a whole lot of additional insight and therefore evolution. Obviously, it feedback loops into
0:33:04 making the models better, but if you can ask your marketing data, you know, what are people asking
0:33:08 me for? I don’t sell it. Why didn’t they like the new product launcher? Why did they like the
0:33:12 new product launcher? You look at all the things and say, what were the phrasings and offerings
0:33:16 that got a lot of engagement? What got no engagement? All of your marketing is customer
0:33:22 research at the same time. So you’re learning in census level what to tune next. That’s just
0:33:28 never existed before. Yeah, interesting times ahead, to say the least. John Heller, for folks
0:33:34 listening who would like to learn more about firsthand, obviously the website firsthand.ai.
0:33:39 Other places they can go online, social media accounts, a blog. Where would you direct them?
0:33:44 Well, we have a LinkedIn presence, but the website is probably the best place to start.
0:33:48 Fantastic. Thank you again so much for joining the podcast. Fascinating conversation, obviously,
0:33:53 with implications for retail and brands and ad tech, but just thinking about the future of
0:33:58 the web and the age of AI and now a gentek AI. It’s, I mean, just to echo what you said,
0:34:02 it must be a blast getting to build all this stuff from scratch.
0:34:08 It’s a rare opportunity to be able to work in a world that’s changing so much with something so
0:34:15 powerful and with partner companies that are just so capable and fantastic. And it is just fun.
0:34:19 Excellent. Well, best of luck to you and look forward to keeping track of your progress and
0:34:35 maybe catching up again down the road. Alrighty. Thank you so much.
0:34:45 So,
0:34:57 ,
0:35:13 you.
0:35:23 [BLANK_AUDIO]
0:00:15 Hello and welcome to the NVIDIA AI podcast. I’m your host, Noah Kravitz.
0:00:21 The AI community is a buzz with agents. As NVIDIA CEO Jensen Huang said in his CES keynote this past
0:00:27 January, “Agentic AI represents the next wave in the evolution of generative AI and a multi-trillion
0:00:32 dollar opportunity at that.” AI agents enable applications to move beyond simple chatbot
0:00:38 interactions to tackle complex multi-step problems through sophisticated reasoning and planning.
0:00:42 They’re expected to become a centerpiece of enterprise AI systems going forward.
0:00:47 Our guest is John Heller, co-CEO and founder of First Hand, whose brand agent platform is
0:00:52 transforming how advertisers and publishers engage consumers online. First Hand is also a
0:00:57 member of Inception, NVIDIA’s program for startups. John, welcome and thanks so much for taking the
0:01:02 time to join the AI podcast. Hey Noah, thank you. It’s great to be here. Very excited.
0:01:06 So let’s start with the basics. If you don’t mind, tell us a little bit about First Hand.
0:01:12 Sure. As you said, First Hand is building the brand agent platform and that’s because when we
0:01:19 see AI, we believe it is itself a new medium, not just a technology. Perhaps I can explain that a
0:01:26 little bit about how we came to be. So I’d been working in advertising tech since the
0:01:32 internet started in the 90s and worked at DoubleClick for quite a long time. More recently,
0:01:37 it was the co-CEO and co-founder of FreeWheel, which is now Comcast ad-serving platform.
0:01:43 And I’ve been doing ad tech for a long time. Actually, when I left Comcast, I went back to
0:01:48 school finger quoted in AI, so online courses, because I wanted to work in something that was
0:01:54 much more broadly applicable and much earlier in its life. And it actually started by working on
0:02:00 reinforcement learning robots to act as minions and artificial teammates at video games.
0:02:05 And that’s how I started learning the guts of it. I’m biting my tongue, so we don’t take us off
0:02:09 course, because that sounds fascinating as well. Oh yeah, that could be a separate conversation.
0:02:14 But I’d been working in the gaming world and some of the generative AI abilities for gaming assets
0:02:21 when language models really came out. And something struck us, something very powerful,
0:02:29 which is, and this is a metaphor for the math inside, but AI now understands the ideas and
0:02:34 intense needs you may have from what you’re reading, what you’re watching, what you might
0:02:40 ask it outright. And it can go find the right response or take the right responding action.
0:02:43 And everything is presented to you in a very natural human way.
0:02:50 And if you back up the step and think of that happening all the way through a consumer’s use
0:02:54 of the digital world from when they’re searching and becoming aware of things they might need,
0:02:59 when they do some investigation and read up on products or services, when they go to
0:03:04 browse or shop, when they buy, all of those modes change pretty fundamentally. They don’t
0:03:10 replace, we think they get enhanced, because instead of the world of the past, where I maybe
0:03:16 did a search, got some directions and a link went to a place, read up on something,
0:03:20 browsed for something, went to saw an ad, maybe went to another place to try to find the version
0:03:27 I want. Those are all sort of separate hops, the internet where it’s the same content everybody
0:03:35 sees. AI instead is going to understand and learn at each moment what it is you need. And as with
0:03:41 most things AI data is the core, the people who have the most and best data about a product or
0:03:46 service are the brands. They are the retailers and the people who sell it. So they can create
0:03:52 brand agents, which means your experience on the internet at all of those moments in the journey
0:03:57 from first learning about it to figuring out what the right configuration is and comparing and
0:04:03 browsing and buying is going to adapt on the fly through these agents. So it doesn’t replace the
0:04:09 web, but it changes things from you looking at stuff someone wrote to something that’s
0:04:14 partially adapting to what you actually need, understanding your needs. But the agents that
0:04:19 are doing that for you are from the retailers and brands themselves, because it’s their data that
0:04:25 is what you need. And that sort of changes the internet and the kind of your internet for both
0:04:33 parties. And that really makes it a much more adaptive form fitting to what you need in enhancement
0:04:39 to what’s already there. But that from a media marketing perspective changes everything.
0:04:44 Yeah, sure. It’s interesting to hear you talk about that from the media marketing perspective.
0:04:50 We’re recording this in early 2025. Happy New Year everybody. Last year I had, especially towards
0:04:55 the end of the year, had a lot of conversations with people about agents and agent to AI,
0:05:02 a few of them on the podcast. And as a sort of consumer end user of things, I keep thinking about
0:05:07 the, and I like the way that you put it, changing it from the web to your web. I keep
0:05:13 thinking about it from the perspective of, oh, maybe in the future, I won’t have to go seek out
0:05:18 the information I want because my agents will know me and know what to go find and bring it back to
0:05:24 me. But it’s interesting to hear you talk about the agents actually being on, I guess, the advertiser
0:05:30 brand publisher side, because they already have the data about my habits. Well, it’s not that they
0:05:34 have the data about your habits. They haven’t about their products and services. So I’m honestly
0:05:39 correct. If you look ahead, consumers have, and we’ll have agents that understand them,
0:05:45 but that’s half the conversation, if you will. Got it. Okay. Those agents know what they need,
0:05:51 perhaps, and what they’re trying to do, but no one’s going to know more about the uses and
0:05:59 values of their sauces or beauty products or furnishings or cars than the brands and the
0:06:06 retailers that sell them. So we see sort of two halves to this. There’s the agents that consumers
0:06:12 may use, which may just be enhancements to the web as is. They may grow from that into something new,
0:06:16 but the folks who can answer their needs and help them not just to learn more, but actually
0:06:21 finish and convert, take the action are the brands and publishers. So for them to be able
0:06:26 to equip their agents, but then send them to all the different places consumers need,
0:06:30 that’s a big change. You’re kind of indicating the knowledge and expertise
0:06:33 out to where the people needed at their different moments on their journey,
0:06:37 which is wildly different than having to get them all to come to you.
0:06:43 Right. Got it. No, it makes a lot of sense. And so first hand is young. When was it founded and
0:06:48 what was kind of, you got into this a little bit, but what was kind of the moment where it came
0:06:53 together and you decided to launch the company? So I was working in generative AR trying to figure
0:06:59 out how to make gameplayable assets much more easy to produce and some commerce ideas around that.
0:07:05 When the language models first came out and we understood this change, that the folks who have
0:07:10 the best information about the products and services you might need are going to want to
0:07:14 speak in their own voice through their own agents, but everywhere. Right. And that that changes
0:07:19 everything and that the folks who have additional information, the publishers who have the expertise
0:07:26 on the trends or topics in general also want to have a voice in that conversation. That really
0:07:31 changed things again from that sort of the internet to your internet, then that sort of was the spark.
0:07:36 That’s like that’s white space. That is not how any advertising is done today. That’s not how the
0:07:40 technology stacks work. That’s not how the data management works. It’s not how the measurement
0:07:46 works. And from an entrepreneur’s point of view, white space is fantastic because you have to,
0:07:51 first of all, it’s just fun because you have to invent it. Right. But you’re not retrofitting or
0:07:56 dealing with, there’s less legacy to try to deal with. And you can, and at this point,
0:08:00 having done marketing for this many decades, there’s a lot of lessons learned about now
0:08:04 that I can start with a clean slate, I’m going to do it this way. And then the power in AI is just,
0:08:11 it’s astounding. So that was just very exciting. And it does change retail, commerce, advertising,
0:08:18 martech, customer research. So that’s a lot of playing fields. So we, I need co-founders that
0:08:22 was thought number one. But having done this for a while, I have just very good friends that I’ve
0:08:30 worked with for decades. And Michael Rubenstein, my co-CEO is someone I’ve known since late 90s,
0:08:35 double click, and way my CTO co-founder, someone that was instrumental in the engineering and
0:08:40 freewheel, you get to work with your friends that are industry experts in white space on
0:08:45 something that is this exciting, not just because of the change in the industry, but,
0:08:50 you know, just a little full disclosure here. The technology is just really fun to understand
0:08:56 and work with. Isn’t it? Yeah. It’s fantastic. Yeah. And then just, if I may, because advertising tech
0:09:02 is just a massive, massive scale, the inference volumes and speeds and things, just a total
0:09:07 different world. Working with companies like Nvidia on figuring out how to make that actually
0:09:13 properly for that kind of an industry. I mean, it sleeves rolled up thumb. Right. Yeah. We formed
0:09:24 in August of 2023. So not too too old, but we are 27 people now. We were live in the summer. We’ve
0:09:28 got, you know, we haven’t done case studies public yet, but we’ve got, you know, actual results that
0:09:35 when you actually adapt the content and the websites and the journey, if you will, for consumers
0:09:41 into something that’s fitting the needs for them, the effects are or multiples greater than
0:09:44 traditional advertising. I don’t want to harp on the nomenclature here, but I want to ask you a
0:09:50 little bit about the word agent and brand agents in particular. I read a definition of agents
0:09:55 in the intro, but, you know, for every conversation I’ve had with people about agents, there’s been
0:10:01 at least one new kind of working definition about what that might mean. Can you talk about the concept
0:10:06 of a brand agent as firsthand uses it? And specifically, how should companies be thinking
0:10:12 about brand agents as different from other types of AI agents? So I like to think of it from what
0:10:16 are you trying, what’s it supposed to accomplish? Right. So when I think about agents as I read
0:10:25 about them, most of what I’ve observed is here’s a somewhat autonomous execution engine that will
0:10:30 go create productivity by making something more efficient. That’s a solid definition. I’ll take
0:10:36 it. Yeah. That’s fantastic. That’s a wonderful, useful, make what you already do work better
0:10:41 use of AI. Right. But when you’re talking about how to participate in a brand new medium,
0:10:46 it’s a very different set of use cases because you’re talking about how do I help a prospect
0:10:51 and a consumer through, call it a connected journey from being aware of something to learning about
0:10:55 what you might want, to finding what version you want, to actually getting it. There’s many
0:11:01 different objectives and purposes there. So for us, an agent is actually a composable collection
0:11:06 of AI capabilities. So you might even think of an agent as containing many agents. We use the
0:11:13 right capability. And you would do one with a particular composition for say, awareness raising,
0:11:19 a different one for acquisition, maybe different one for conversion improvement in upsell. And
0:11:24 what that means is you do a few things. First, underneath the whole product is something called
0:11:31 Lakebed, which is a kind of intellectual property rights data management platform so that agents
0:11:37 in whatever form they take can cooperate on data across parties. And just to clarify, Lakebed is
0:11:41 a first-hand product. Yeah. It always comes back to data in order for an agent to do whatever its
0:11:47 job is. If it’s happening out, if it’s for a retailer and it’s happening out on a publisher,
0:11:52 it needs to be able to, because it benefits both parties, safely use the information of the
0:11:57 publisher about, if you’re on a website about decorating and decor and home and furnishings,
0:12:03 and a retailer wants to help you with that, then they need to know not only all of their products
0:12:08 and services and offerings, but everything about what you’re looking at and the content of the page,
0:12:12 and then it’s beneficial to what parties, if it’s done with full intellectual property and
0:12:17 privacy control rights, because no one exits an event with data they didn’t enter the event with,
0:12:22 if you will. But it makes the agent capable. And then that agent might help you instead of
0:12:27 seeing what you would traditionally see as an ad, it may say, “Look, are you trying to furnish a new
0:12:32 home? Are you trying to refresh for holiday entertaining? Are you downsizing and looking
0:12:37 to sort of deal with a smaller property?” And it learns, as I said, this is the new medium of
0:12:42 effect. That agent, even though it’s happening, retail agent is happening on the publisher property,
0:12:47 it’s learning about what you need, and it’s presenting the information. And here’s where
0:12:52 the capabilities come into play. It could simply be content presentation, if you want to have open
0:12:56 Q&A, you could add a chat capability. You could close the capabilities that make sense for what
0:13:01 you’re accomplishing. But it learns, and let’s say if someone is furnishing a new home, then it’s
0:13:05 going to present them with like, here’s the five key starter pieces you need, and here’s how to
0:13:11 turn a house into a home quickly with color and accessories. And then this is new medium again.
0:13:15 Once someone engages with that agent and the retailer, that’s the first party relationship
0:13:20 between the retailer and the consumer, which is an arcane element in marketing. But that means that
0:13:25 when that consumer leaves that agent and arrives at the retailer, the retailer is allowed to
0:13:30 understand everything that occurred in that agent out in the wild. So that means when they arrive,
0:13:35 instead of arriving in the past, it would be the landing page everybody sees, the landing page itself
0:13:41 can be another agent that is therefore composing the content of the page to fit the needs of what
0:13:47 brought you there in the first case, and offering up additional help to find the right configuration
0:13:52 that makes sense in this need, etc, etc, which is beneficial to the consumer, but obviously helps
0:13:56 sell more product and is beneficial to the retailer and obviously creating great value
0:14:00 for the publisher and selling that media opportunity. Everyone has to cooperate in a way
0:14:06 that’s safe for each other. And this is why we think we’re seeing results multiples and multiples
0:14:10 better than traditional ads, is because you’re creating a utility for the consumer that happens
0:14:16 to be directly beneficial to the publisher and retailer doing it, or the brand if they’re the
0:14:20 ones who are on the other side of the conversation. Right, right. I’m going to ask you one of those
0:14:24 questions where I think I know what the answer is, and usually those are the ones that are good to
0:14:30 ask because I’m often quite wrong. Can you kind of describe the difference sort of in as plain
0:14:36 terms as you like between what you’re describing and the chatbot experience that consumers may
0:14:41 be having now? So the key is that in different circumstances with different needs, if it’s
0:14:46 on a search engine, if it’s in social, if it’s on a publisher, if it’s on your own retail site or
0:14:51 the manufacturer brand’s home, their site, what you want to do, again, starts with what its intention
0:14:58 is, then we separate the brain of the agent, which is a few pieces from Lake Bend. What is it
0:15:03 allowed to know? So a retailer may want it to know only about their house brand products,
0:15:07 or they may want it to know about every product they offer. So that’s sort of a business choice.
0:15:11 Right, right. And then you choose the sets of capabilities you want it to produce.
0:15:17 So it may be simply content, the appropriate content selection and presentation. So it’s
0:15:22 essentially building a mini website, if you will, on the fly to help them with set up a choose your
0:15:28 own adventure structure. If you want to add to that chat so that they can ask a question you
0:15:32 didn’t interpret appropriately, then that’s just additional capability. And that’s the agent that
0:15:36 may happen out in the wild. You sort of wrap that up as a campaign and send it out there and say,
0:15:41 I want it to happen this many times in these places. And if someone goes through that agent
0:15:46 onto the destination site, so for example, and what we’ve seen so far, folks are, when they’re
0:15:51 looking at an agent, they’re going to the page about the actual thing that that agent talked to
0:15:56 them about multiples on multiples on multiples higher rates than an ad does. And they’re going
0:16:01 to many, many, many of them. So hundreds of different ones, which means you’re getting people
0:16:07 to what they actually need more efficiently. And that’s not a chat experience, it’s more interwoven
0:16:13 into the way people already serve, because that last part of the agent is something we call a
0:16:18 frame. And you can call that its UI. What is the user experience you want to present? Is it image
0:16:24 rich or is it text centric? It’s almost more of an app in a way than an agent, because how it
0:16:30 appears and the way you interact with it is at least as important, frankly, for marketing
0:16:34 as what it’s able to say. Right, no, that makes a lot of sense. And then, frankly, it continues its
0:16:40 help to you through your connected journey. So search from the publisher onto the retailer,
0:16:44 perhaps onto the brand, it has contacts throughout, which I think, you know, no one likes to start
0:16:50 from scratch again. Again, so an agent is this mix of what is it supposed to know? What is it
0:16:56 objective? Is it content presentation plus open dialogue? Is it actually constructing the page
0:17:02 you’re on while it’s talking to you? So you pick the set’s capabilities, then you layer on that UI.
0:17:06 But what that does is that’s the changing the internet to your internet, because it’s taking
0:17:12 those discrete hops. I searched for something, I read an article, I saw an ad, I clicked on it,
0:17:16 I went to the site, I started from the beginning, I found the product, and it’s turning that into
0:17:21 one connected journey. Right. It understands and helps me with what I need at each of those steps,
0:17:26 and it’s with me the whole way with full context, because that’s the relationship between me and
0:17:32 the retailer. The location is less important. So you’ve spoken to this, but maybe we can put a
0:17:37 point or you can put kind of multiple points on it. But what are some of the most exciting
0:17:44 areas of potential or even specific problems that brand agents can solve for marketers and kind of,
0:17:49 you know, ways that the retail experience, I mean, you’ve been talking about it with this idea of
0:17:55 composing, you know, your web on the fly to suit what you need. Are there other specific problems
0:17:59 or big things that you’re excited about when it comes to how the experience is going to be
0:18:05 transformed by brand agents? Well, I think the very, very fundamental change is being able to take
0:18:11 the expertise and the knowledge of the retailer or the market or the brand and put it in the moment
0:18:17 wherever the consumer is. So taking their ability and putting it into an agent and sending that
0:18:23 agent everywhere. So you’re helping the consumer at all those moments is a massive change because
0:18:29 that’s why instead of trying to sort of common denominator, pick them one message and figure
0:18:34 out the targeting about exactly what this one thing should match against the AI. This is what AI
0:18:39 is designed for. It understands and learns what’s the right thing in this situation, which means
0:18:45 because your internet is just as true for the marketer as it is for the consumer. So it’s taking
0:18:49 of all the things you could do. This is what is useful to this person here in this circumstance.
0:18:56 So that means instead of having to fine tune and sort of try to hit the bullseye, you can kind of
0:19:01 let this find all the bullseyes. And you can see that evidencing by the fact that it’s sending
0:19:06 people to hundreds of different destinations instead of just the one or two. But then if you
0:19:12 can continue that connected journey when they arrive and have that fit to what they need,
0:19:17 and that covers everything from like food and beauty to finance and auto and all forms of
0:19:21 retail, the odds of them getting to the thing they need and getting the version of it that
0:19:25 they need and being able to ask any follow on questions about how to actually pick the right
0:19:30 set of it. We don’t have data on this yet, but I cannot wait to do conversion tracking because
0:19:37 I am very excited about that. So I think that is a wonderful opportunity that has never existed
0:19:42 before as opposed to a problem solved. Right. Something you said earlier, the way you described
0:19:49 the current process of you have something that you want to find or you need or you’re decorating
0:19:54 your home and don’t know where to start. And so you do a web search and you get a list of results
0:20:00 and you pick one and you go to it and then you read the page, scan it to find what you’re looking for.
0:20:05 And the idea of that process changing, it makes me think of things that marketers, I’ve read and
0:20:13 talked to marketers saying that AI may fundamentally change the whole web experience starting with
0:20:18 breaking SEO. Do you have thoughts on that? So a couple things here. The world has been built
0:20:24 at this point where there is a fairly central place to go ask your question, looking backward.
0:20:33 Right. And that sent traffic out to the places to go read the details where people might run ads
0:20:38 to then get you to go to the place where you can consummate the purchase and learn more about the
0:20:44 purchase. But that hop, hop, hop is completely changing because the knowledge is pushing out
0:20:49 to the consumer at all those different places. They still want to go to publications and read
0:20:54 about the details and what’s the best version and then to understand what’s new and trendy. So
0:21:00 everything still exists. It’s just more adaptive and it’s more sort of distributed out. I think it’s
0:21:06 less centralized trying to get someone at that first question and get them at all the moments
0:21:10 of question throughout all their moments of need on that journey. And if you can make that smooth
0:21:15 for someone and in context throughout, that’s so much better for the consumer. But it is a,
0:21:19 it is still an enhancement to the internet. I think the way digital is great. It just makes it
0:21:23 much more, it’s the internet to your internet. It makes it richer and easier to do stuff.
0:21:30 Yeah. I’m speaking with John Heller. John is co-CEO and founder of First Hand. He’s an ad tech
0:21:37 veteran, started First Hand with friends from his ad tech days. And they are using AI, AI agents
0:21:43 specifically to transform the way that publishers and advertisers connect with consumers, engage
0:21:47 with them, help them throughout their journeys. Everything John’s been talking about to this
0:21:53 point. John, you’ve been speaking about all the different ways sort of conceptually and concretely
0:21:59 that AI, AI agents, the brand agent platform that First Hand is building can change the way
0:22:04 that consumers, you know, go on their journeys and that advertisers, publishers engage with them.
0:22:11 Are there specific industries, specific experience types? I mean, to me, this sounds like a
0:22:17 transformative thing that’s, you know, far reaching and then some. But are there certain areas that you
0:22:21 think are really, you know, right for transformation and you’re particularly excited about?
0:22:27 So one of the reasons this has been so much fun and starting something that’s so white-space
0:22:33 is so enjoyable is you are learning things that you didn’t understand. I started thinking this is
0:22:39 for high consideration stuff, financial services, automotive, telecom, consumer, things where you
0:22:46 have lots and lots of details to go figure out. Yeah. Nope. I mean, yes, but more, you know, food,
0:22:50 like what dishes would make sense for the holidays and how can I sort of help you figure out the
0:22:57 shopping list for a great Thanksgiving? Or did you know that these products just lead for balanced
0:23:04 healthy life? So what we found is as much interest in how this can benefit your lifestyle in that
0:23:09 path as much as how much this product is useful for its features. So the interest has been very
0:23:15 broad and it’s not just been broad from industries, it’s been broad for sort of where in the distribution
0:23:21 chain it fits as much on their own properties as it is distributed out across where they market.
0:23:26 So it’s as much customer as prospect. Right. We build a platform. This is the reason our
0:23:31 agents are very composable and collections of abilities as opposed to just an agent is we
0:23:36 thought that they would have broad application and the breadth surprised me. Yeah. Talk a little bit
0:23:43 about the experience of building brand agents and when, you know, in listening to the past 20 minutes,
0:23:50 I’ve been thinking about LLMs and the chatbot experience and using RAG to, you know, put
0:23:55 my company’s specific product information or what have you in there to let the chatbot find,
0:24:00 you know, the correct information. How is the process of, you know, thinking about and building
0:24:06 and deploying agents and brand agents specifically? What’s that like? Is it similar to building with
0:24:12 an LLM and RAG? Is it radically different? Talk about that if you would. It’s much more nuanced.
0:24:16 And the reason I say that, I’ll use an example. If it’s a retailer with several hundred thousand
0:24:23 SKUs, then the objective they’re trying to solve for is different than someone who sells
0:24:29 like three packages, an energy package, a sports package, and a family package. So that affects
0:24:36 how you retrieve, right? You actually, basically the platform we built is exceptionally composable
0:24:43 for many, many reasons because the thing you’re making the agent solve for affects the way you
0:24:47 embed and tokenize. It affects the way you retrieve. It expects the way you evaluate what you
0:24:54 retrieved and ranked. It affects how you generate and the instructions you give it and then how you
0:24:59 evaluate what was generated to make sure it was doing what you needed it to do. Because one of
0:25:04 the interesting things about marketing is there’s almost as many rules about what not to do as there
0:25:10 to do. So there’s quite a lot of highly flexible control layers around making sure it didn’t do
0:25:14 the thing it wasn’t supposed to do as well as did what you wanted. And all of that needs to be very
0:25:20 composable to the objectives of the parties involved. It’s always a supply chain across
0:25:27 multiple parties. So lots of rights management. And then it’s generating information that’s never
0:25:33 existed before because every agent, because that’s a first party interaction, is not just
0:25:37 generating metrics. How many times does someone engage with it? What did they click on? Did they
0:25:44 go somewhere? Did they click on a citation or call to action or a product image? But it’s recording
0:25:48 the transcripts of what was actually presented. What was the text of what you have a full survey
0:25:53 dialogue transcript of all these interactions, which is almost as if all your marketing is also
0:26:01 a customer research survey at the same time. And that can be used to learn how to be better.
0:26:06 Yeah, not so wealth of data. So it has feedback loops, again, completely IP rights managed
0:26:12 protected feedback loops, so that it’s improving for the person whose data it’s using to improve.
0:26:18 But those feedback loops make it understand in this context, this configuration for this piece
0:26:24 of the puzzle fits. So I think the key thing is high configurable composability. And then also,
0:26:30 just frankly, in enterprise world, people like to use like we made it so the foundation underlying
0:26:34 models you want to use or a choice, right, you know, the one you want as a cartridge, partly
0:26:39 because different companies have different desires, or maybe they’ve tuned a model, especially for
0:26:45 their world that they like. But this is going to change so fast, we need to be able to plug cartridge
0:26:50 A out and put improved cartridge B in and let the rest of the framework still operate. Right. So
0:26:56 it is, I say the composability and the learning and checking whether it did the right thing and
0:27:02 can it do better is the key part. When it comes to learning and checking to see if it did the
0:27:08 right thing, evaluation, and maybe even dealing with hallucinations, is there or are there,
0:27:13 you know, a genetic specific techniques that you’re either using or sort of discovering is,
0:27:19 you know, evaluating a response different when you’re working with agents? Or is it kind of the
0:27:24 same? I think the key difference is you have to have lots of different types of evaluations.
0:27:30 And this is, everything’s gold-dependent. So if it’s this type of a marketer with this type of
0:27:35 an objective, if it’s a recirculation agent to increase use of time on site for a publisher,
0:27:39 that’s a different case than if it’s a retailer trying to wrap people to the right product or a
0:27:45 brand trying to get people to understand that these are great recipe shopping lists for this holiday.
0:27:51 So that doesn’t just change what you retrieve and how you generate, but it changes which sets of
0:27:56 evaluations you invoke. So that configurability matters a great deal. And all these feedback
0:28:04 loops are sort of configuration specific and company specific because you would tune a house
0:28:10 brand, you know, holiday effort is different than what you would do for everything I have on my
0:28:16 shelves, just people to the site type of thing. So I think, again, it’s being able to manage all
0:28:22 that to fit its purpose that is proving really important. Perhaps a naive question here. So
0:28:29 forgive me, but is sort of troubleshooting and running tech support for your customers made
0:28:35 infinitely more complicated because of all these, you know, nuanced specific use cases? Or is it
0:28:40 just kind of a new version of things that people in the ad tech world have been dealing with for years?
0:28:47 Interestingly, we have an entire experimentation module, which means you can run this in sort of
0:28:54 laboratory mode, right, right, watch all of its behaviors before it goes in front of consumers,
0:28:59 where you get yet another important feedback loop, which is, you know, no one is more expert
0:29:04 in the quality of their output and the appropriateness of it than the editors and the marketing
0:29:10 staffs of the brand. So they get to go into the experimentation module, have it run in lab mode
0:29:14 and say, that was great. That was good. That was great. And that creates yet another feedback loop,
0:29:21 so it improves to past muster. Once you’ve got it to a state where you’re happy with what it does,
0:29:28 that reduces the once it’s live issue set. And again, to go back to the capabilities list,
0:29:33 this is how it’s different from chat. If you picked what we call a guided conversation,
0:29:37 which is sort of a choose your adventure motif, it’s only picking from the content you’ve already
0:29:41 told it’s allowed to know. And it’s simply surfacing the right things, phrase the right way in the
0:29:46 right moment. And this gets back to how agents are built. If you wanted to add chat to the bottom
0:29:51 of that, then someone can ask whatever they want. That obviously triggers a whole different set of
0:29:57 protections and evaluations to keep that on track. So again, that configurability is quite important.
0:30:01 Kind of at a really high level, how do brand agents, how does your technology
0:30:08 change the way that your customers, the brand’s publishers, think about how they want to engage
0:30:14 with customers? So in the past, you would try to get a list of target identities. So you’re
0:30:20 trying to find a list of folks, because that list is a proxy for flags that mean what they care about
0:30:26 and what they’re interested in. And then you would create a message that was appropriate to that list,
0:30:31 and then you got to go find where those folks might be and try to show that message to them.
0:30:36 And that’s when they’re out beyond your borders, if you will. When they arrive at your site,
0:30:40 you don’t know how they got there with any particular detail, and you’re sort of again
0:30:45 starting from scratch. So what we would say is you have a great set of products and services,
0:30:51 figure out how you want to talk to your different typical customer as opposed to specific target
0:30:59 list, and then let the AI do what it does. Put it in all the places where they might be
0:31:04 beyond your property and on your property, and then it will surface to them the things that
0:31:09 matter to them, and they’re going to end up coming to you already having indicated what’s
0:31:14 important and being qualified. So you receive them. So here’s a big change. You should have agents
0:31:21 present when they arrive that receive them with that knowledge and start from step five, not step
0:31:29 zero. And that changes how you think about campaigns, how you construct them. Interestingly,
0:31:35 this is a bit arcane for ad tech, it’s vastly simpler operationally to set this up than to try
0:31:42 to write like a hundred line item insertion order, trying to manually, this is a classic AI,
0:31:48 do I build a massive if then else statement, an IO with hundreds of line items,
0:31:55 or do I have an agent able to figure it out? Yeah. So what’s next? What’s next for firsthand?
0:32:00 What’s the future of AI agents like? Take it wherever you like. How do you envision the technology,
0:32:06 the use of it, the implications kind of evolving over the next, I’m going to say three to five years,
0:32:13 but you can tweak that to suit as you like. Well, I think the ability for the internet to come
0:32:18 your internet has a lot of the new medium, it has a lot of implications. So I think
0:32:23 where I see agents is they’ll start to be more wisdom around this is the right configuration
0:32:27 at this point in the customer journey. This is a productive configuration, these abilities,
0:32:32 it’s going to be this guided conversation with presentation, you know, when they arrive,
0:32:36 I want page composition, but only in these portions of the site. So I think the very
0:32:43 construction of how marketing campaigns and sites are put together changes. And when it
0:32:47 becomes this connected journey where I’ve understood the consumer all along the way,
0:32:54 and I’ve helped them all along the way, and I have that data, I think that data starts to drive
0:32:59 a whole lot of additional insight and therefore evolution. Obviously, it feedback loops into
0:33:04 making the models better, but if you can ask your marketing data, you know, what are people asking
0:33:08 me for? I don’t sell it. Why didn’t they like the new product launcher? Why did they like the
0:33:12 new product launcher? You look at all the things and say, what were the phrasings and offerings
0:33:16 that got a lot of engagement? What got no engagement? All of your marketing is customer
0:33:22 research at the same time. So you’re learning in census level what to tune next. That’s just
0:33:28 never existed before. Yeah, interesting times ahead, to say the least. John Heller, for folks
0:33:34 listening who would like to learn more about firsthand, obviously the website firsthand.ai.
0:33:39 Other places they can go online, social media accounts, a blog. Where would you direct them?
0:33:44 Well, we have a LinkedIn presence, but the website is probably the best place to start.
0:33:48 Fantastic. Thank you again so much for joining the podcast. Fascinating conversation, obviously,
0:33:53 with implications for retail and brands and ad tech, but just thinking about the future of
0:33:58 the web and the age of AI and now a gentek AI. It’s, I mean, just to echo what you said,
0:34:02 it must be a blast getting to build all this stuff from scratch.
0:34:08 It’s a rare opportunity to be able to work in a world that’s changing so much with something so
0:34:15 powerful and with partner companies that are just so capable and fantastic. And it is just fun.
0:34:19 Excellent. Well, best of luck to you and look forward to keeping track of your progress and
0:34:35 maybe catching up again down the road. Alrighty. Thank you so much.
0:34:45 So,
0:34:57 ,
0:35:13 you.
0:35:23 [BLANK_AUDIO]
With AI agents, organizations can reshape the landscape in retail and beyond. In this episode of the NVIDIA AI Podcast, Jon Heller of Firsthand discusses how AI Brand Agents are transforming online shopping and digital marketing by personalizing customer journeys and turning marketing interactions into valuable research data.