AI transcript
0:00:10 [MUSIC]
0:00:13 Hello, and welcome to the NVIDIA AI podcast.
0:00:15 I’m your host, Noah Kravitz.
0:00:19 Digital humans and AI agents are poised to be big news this year.
0:00:21 They’re already making waves, in fact.
0:00:26 At CES 2025 in Las Vegas, Logitech G’s Stream Labs unveiled its intelligence
0:00:31 streaming assistant, powered by technologies from in-world AI and NVIDIA.
0:00:35 The intelligence streaming assistant is an AI agent designed to provide real-time
0:00:39 commentary during downtime and amplify excitement during high stakes moments
0:00:41 like boss fights or chases.
0:00:45 The collaboration brings together Stream Labs expertise in live streaming tools,
0:00:49 NVIDIA ace technology for digital humans, including AI vision models that can
0:00:53 understand what’s happening on screen, and in-world’s advanced generative AI
0:00:57 capabilities for perception, cognition, and adaptive output.
0:00:58 But what are digital humans?
0:01:00 Where are they going to be used?
0:01:03 And where are they going to make an impact in the enterprise and
0:01:05 in gaming and entertainment in particular?
0:01:08 And how does a designer design in the age of digital humans,
0:01:10 agentic AI, and beyond?
0:01:14 Chris Covert, director of product experiences in in-world AI,
0:01:18 who’s press release, by the way, or blog post, I paraphrased heartily from in
0:01:21 that introduction, so credit to.
0:01:24 Chris is here to dive into these points and more as we talk about some of the
0:01:28 technology that I think is really poised to really make a big impact this year
0:01:33 going forward and kind of shape this new area of digital experience for all of us
0:01:35 in what we’ll broadly call the AI age.
0:01:36 But at any rate, Chris is here.
0:01:41 So Chris, thank you so much for joining the AI podcast and welcome.
0:01:42 Thank you for having me today.
0:01:45 It’s a pleasure to be here, not only as a longtime partner of NVIDIA,
0:01:49 but also as a huge fan of your recent announcements in AI at CES this year as
0:01:53 well, genuinely amazing time to be talking about AI in this space.
0:01:54 It really is.
0:01:55 And so you were at the show.
0:01:56 You were in Vegas.
0:01:57 Couldn’t make it personally.
0:01:58 I’ll be doing dice.
0:02:00 I’ll be doing GDC, but our team was there.
0:02:02 We worked on the project.
0:02:03 All right, so let’s get into it.
0:02:08 Tell us a bit about in-world, in-world AI for listeners who may not know.
0:02:11 We can get deeper into the assistant in particular,
0:02:14 a little later in the conversation, if that’s cool, just so you can kind of set
0:02:18 the stage for all of us about what it is that we’re going to be talking about.
0:02:20 Yeah, so I, you know, extremely biased.
0:02:22 I have the best job in the world.
0:02:25 And in-world, we make the leading AI engine for video games.
0:02:28 And I get to work with the most creative minds in the industry,
0:02:31 most creative minds in the world of gaming and entertainment to answer the
0:02:35 question, how can we make fun more accessible to the people that make our
0:02:36 favorite experiences?
0:02:40 And I’m not going to tell you it sounds like a great job.
0:02:42 It is, it is.
0:02:43 And I’m not just blowing smoke here.
0:02:47 And we’ll definitely, as we go through this, you know, emphasize fun,
0:02:50 accessible in people, and why I think those are the most important here.
0:02:54 But in-world’s mission is to be that AI engine for this industry, right?
0:02:58 The more that the technology progresses, the lower the barrier
0:03:00 to entry to make AI experiences.
0:03:04 But we find there are still challenges, regardless of if you’re a studio
0:03:08 with a massive staff or you’re an AI kind of native company building
0:03:12 up these experiences from grassroots, that there’s a massive gap
0:03:14 between prototyping and deployment of AI systems.
0:03:19 And then to do that and find the fun for a user for an experience
0:03:20 is still incredibly challenging.
0:03:24 So we offer not only the platform, but the services to help you
0:03:26 create that fun in a deployable manner.
0:03:31 So kind of briefly run down maybe what you guys talked about at CES this year
0:03:34 and then we’ll put a pin in the assistant, like I said, and come back.
0:03:35 Yeah, that’s awesome.
0:03:38 So blog posts is out there, fantastic videos out there.
0:03:41 A lot of demonstrations of what happened, but succinctly, you know,
0:03:45 to plug the collaborative effort here in World NVIDIA and Streamlabs
0:03:50 got together for CES and we put to the test this, you know, streaming companion
0:03:52 that could not only act as a cohost to a streamer,
0:03:54 but also serve to support the creator in other ways,
0:03:58 like handling production for them on their stream as well.
0:04:01 We showed a demo of this happening at the conference, always a risk
0:04:04 because you’re playing in a big space with low, you know, internet bandwidth.
0:04:08 But we played this over Fortnite and the streamer, the demo where in this case
0:04:11 and their agentic cohost, you know, chatting during the game,
0:04:15 grilling each other for bad rounds, reacting to trends that are happening in chat,
0:04:17 cheering together when you manage to get kills.
0:04:21 But then interestingly enough, you know, when chat or when the streamer wants a replay,
0:04:24 streamer has to just ask the agent, they’ll clip it,
0:04:27 they’ll set up the screen for that replay and they’ll keep that stream feeling fluid.
0:04:30 So it’s one of those use cases, again, trying to align on like,
0:04:35 where do agentic systems actually play in enterprise and this entertainment industry
0:04:37 is we’re finding they’re hyper specialized.
0:04:40 And in this use case, we have this perfect, you know,
0:04:43 one streamer has to wear all of these different hats during their stream.
0:04:45 How can an agentic system come assist them
0:04:48 so that they can focus on making the best content
0:04:51 and all of the other things in managing that content can be done for them.
0:04:54 So back in the day, and this is part of why we got to come back to it.
0:04:58 I did a lot of YouTube stuff and we tried to do some live streaming.
0:05:00 This was in like the early mid 2000s.
0:05:05 And I remember setting up, you know, open source plugins to try to get graphics
0:05:06 over the live stream.
0:05:10 And then I had like a secondary camera, you know,
0:05:13 like a phone with a camera with a USB cable and all this stuff.
0:05:14 But that sounds amazing.
0:05:17 So I’m excited to talk about that to kind of then back into it
0:05:19 or start with the technology and go forward.
0:05:23 A genetic AI cohost digital humans.
0:05:24 Let’s talk about those things.
0:05:27 You want to start with digital humans because we’ve talked about a genetic AI
0:05:30 and we’ll get to it and it has the different, you know, what an agent is or isn’t.
0:05:33 I think it’s still a little bit malleable and context specific.
0:05:37 But when we talk about digital humans, when InWorld talks about digital humans,
0:05:38 what are we talking about?
0:05:39 You know, that’s such a great question.
0:05:41 And it brings up an important distinction.
0:05:44 When most people say digital humans, they’re thinking of chatbots.
0:05:48 Text-based tools that respond with mostly pre-programmed scripts
0:05:52 to help a specific task or guide users through a set of questions or instructions.
0:05:56 But at InWorld, we focus on AI agents that go far beyond simple chat.
0:05:59 These agents can autonomously plan, take actions,
0:06:01 and proactively engage with their environment.
0:06:07 Unlike a typical chatbot, which waits for a user to have some kind of question
0:06:12 or response in and then that AI agent provides a canned response,
0:06:15 our agents at InWorld are designed to interpret multiple types of inputs,
0:06:19 whether that’s speech or text or even vision or sensor data.
0:06:22 And then dynamically think and respond in real time.
0:06:23 They don’t just answer questions.
0:06:25 They can initiate tasks.
0:06:26 They can adapt to different contexts.
0:06:28 And they can carry out sophisticated interactions
0:06:31 that make sense within the context of what you’re doing.
0:06:34 So if you look at where digital humans are used mostly today,
0:06:35 and again, I’m super biased,
0:06:38 it’s often in the high volume chatbot or, you know,
0:06:40 personal digital assistance space.
0:06:43 But if you consider where they can have the biggest impact,
0:06:45 that’s when they function as these truly autonomous agents.
0:06:48 Planning and acting and proactively helping people
0:06:50 in ways that traditional chatbots can’t.
0:06:52 It’s not question and answer.
0:06:54 It’s answer or question that I didn’t even know I had yet.
0:06:56 And that’s the core of our focus at InWorld,
0:06:58 building AI agents that can not only look human,
0:07:02 but can think and react and solve problems like one,
0:07:05 using multiple modalities to interact with their world.
0:07:06 In our case, a digital one.
0:07:09 When it comes to building with agentic AI,
0:07:13 do you feel even like you have kind of a solid vision
0:07:15 of what is to come even near term?
0:07:20 Or is it still very early days of figuring out everything from,
0:07:23 you know, trying different types of infrastructure
0:07:26 to, you know, sort of the design and higher level thinking
0:07:28 about how to architect these things?
0:07:28 That’s a great question.
0:07:32 And the reality is we’re shifting toward an agentic AI framework,
0:07:33 as I mentioned before.
0:07:37 And shifting toward that framework is no longer really an option.
0:07:40 It’s becoming essential for us and our partners.
0:07:44 When you need any sense of autonomy or kind of real time
0:07:46 or runtime adaptability, latency optimization,
0:07:48 any sense of custom decision making,
0:07:52 you need a robust framework that lets you take control of that stack.
0:07:55 No one platform is going to serve the needs of our industry
0:07:56 as standalone.
0:07:58 So building a framework that is flexible,
0:08:00 that is robust to the needs and growing needs,
0:08:03 changing needs, super diverse needs of our industry
0:08:05 is incredibly important.
0:08:06 What we’re seeing more and more of is enterprises
0:08:09 that want to own that architecture end to end.
0:08:11 What I mean by that is they need the flexibility
0:08:14 to decide what models to use, where to use them,
0:08:16 how data is fed into their systems,
0:08:18 how to manage compliance and risk,
0:08:21 all of these factors that require really custom logic
0:08:22 and custom implementations.
0:08:24 So it’s adopting this, you know,
0:08:28 agentic AI framework that really puts the power in their hands.
0:08:30 It’s not just about future proofing.
0:08:33 It’s about giving them full control over how AI evolves
0:08:35 within their organization at any given time, right?
0:08:37 The industry is going to move quickly.
0:08:38 We want to make sure our partners can move
0:08:41 just as quickly as they get inspired.
0:08:45 But to me, you know, my favorite key differentiator here,
0:08:47 Future of Inworld, why the framework model,
0:08:50 why all of these changes is that such a framework
0:08:52 doesn’t just guarantee technical flexibility.
0:08:54 That’s great, and that’s going to make a lot of people happy,
0:08:57 but it also opens up the creative design space as well, right?
0:09:00 By collaborating with people that are thinking
0:09:02 of the most outrageously beautiful,
0:09:05 outrageously wonderfully weird experiences
0:09:07 you could possibly imagine and giving them tools
0:09:09 that are opened up to really let them craft
0:09:11 their own AI architectures.
0:09:14 We’re really, you know, helping push the boundaries
0:09:18 of what and how we can deliver innovative experiences, right?
0:09:21 We made the jokes earlier, but if we say that conversational AI,
0:09:22 you know, this like chat bot nature,
0:09:26 is now feeling very entry level, it’s feeling baseline,
0:09:28 then what we’re doing with this agentic framework
0:09:30 is building out that blueprint for the future
0:09:32 so that our partners and our customers
0:09:37 can help imagine, build, and deploy AI experiences
0:09:40 that genuinely didn’t feel possible even six months ago, right?
0:09:41 Taking that to a whole new level.
0:09:44 If we go more than like two years out,
0:09:47 I’m at a 10% confidence level of where this is actually going to be.
0:09:50 Yeah, I appreciate the candor.
0:09:52 Yeah, it’s just, you know,
0:09:54 and I’ve been doing this a couple of times in my career throughout
0:09:57 where I’ll work with somebody in an advisory capacity
0:10:00 where we’re looking at the future three to five years out
0:10:02 and by the time we finish our analysis,
0:10:04 the thing that we had two years out is now open sourced
0:10:06 and you’re like, cool, so back to the drawing board
0:10:09 because it’s just the industry moves so quickly
0:10:13 that what is possible, it’s actually really hard to nail down
0:10:16 and we will get into kind of, hopefully in this conversation,
0:10:20 how designing around AI and these systems is that challenge
0:10:23 because I think we’ve learned some fantastic lessons
0:10:25 that hopefully people listening can take away as well.
0:10:29 I really see there, you know, when it comes to trying not to sound too obvious
0:10:31 when we’re talking about agents,
0:10:34 the evolutionary chain of how an agent will grow,
0:10:36 how an agent, you know, the technology behind it
0:10:38 will proceed in even the near future.
0:10:41 It comes all down to agency and again, it’s agents.
0:10:44 It sounds obvious, but what I really mean by that
0:10:46 is the complexity of that cognition engine
0:10:49 and I’ll try and crack this net with an analogy
0:10:51 and I’ll try and make it fast because I could talk about this.
0:10:52 Now, take your time.
0:10:53 We’ll take your time with the analogy anyway.
0:10:58 We have this, you know, first again is that conversational AI phase
0:11:00 and I’ll use a gaming analogy, right?
0:11:04 The conversational AI phase gives avatars, gives agents,
0:11:05 I’ll use them interchangeably today,
0:11:08 extremely little agency in doing anything other than speaking, right?
0:11:12 It may be able to respond to my input if I ask it to do something,
0:11:14 but it’s not going to physically change the state of something
0:11:17 other than the dialogue it’s going to tell me back.
0:11:19 So, I think in terms of an analogy,
0:11:21 like a city-building simulation game,
0:11:24 in this phase, conversational AI,
0:11:25 I can ask it about architecture
0:11:27 and it could give me some facts about architecture.
0:11:30 I can ask it about plans I have to build this little area
0:11:31 and it could give me some advice.
0:11:34 I could ask it, “Hey, how do I limit the congestion
0:11:37 or how do I maximize traffic over this little commercial area?”
0:11:41 And it could tell me in words how to potentially do those things,
0:11:43 but it won’t be able to do things like place buildings
0:11:46 or reconstruct roads or even select the optimal facade
0:11:48 for the style of the region
0:11:50 because that’s completely out of its scope.
0:11:50 And not to…
0:11:51 It can’t act.
0:11:52 Yeah, it can’t act.
0:11:53 It can’t act at all.
0:11:53 It is…
0:11:55 Okay, just making sure that I’m fine.
0:11:55 Yeah.
0:11:58 It is a very rudimentary perception,
0:11:59 kind of what is it seeing,
0:12:02 what context does it have and cognition,
0:12:04 how is it planning, how is it reasoning,
0:12:06 very simple engines that drive that.
0:12:08 Really, I’m going to talk to it.
0:12:10 It’s going to say, “Oh, you know,
0:12:12 this person cares about architecture.
0:12:14 I’m going to pull everything I know about architecture,
0:12:16 turn that into a nice response,
0:12:19 weave together kind of their question into my response,
0:12:22 and then boom, now we have simple conversation AI.”
0:12:24 But not too far ahead of that.
0:12:26 And I say that lovingly and endearingly
0:12:29 is a simple task completion AI.
0:12:31 It’s very much still call and response.
0:12:33 You have an action engine.
0:12:34 You say, “I’m having an agent
0:12:37 that can only construct new buildings
0:12:38 exactly where and when I tell them.
0:12:40 I can say place a building at this intersection
0:12:41 and it does it, boom.
0:12:43 I can say change the building to this type
0:12:44 and boom, it does it.”
0:12:44 Right.
0:12:46 And while you argue that
0:12:47 that requires a little bit more perception,
0:12:49 maybe a little bit more reasoning,
0:12:50 definitely some action,
0:12:53 it’s limited to an extremely small set of actions
0:12:56 and it’s pretty much just scripting with extra steps.
0:12:58 It knows what it’s going to expect.
0:12:59 It’s waiting for me to say it in a way
0:13:02 that it can map to that is the place of building action.
0:13:04 I’m going to place a building done.
0:13:08 If a language model provides the description,
0:13:11 including like I wish I could change the background
0:13:13 to mountains or whatever, right?
0:13:15 Or envision a background with mountains
0:13:16 and it can’t act, right?
0:13:17 Can the system be automated?
0:13:18 What you’re talking about
0:13:23 so that the language model can tell the action cognition model?
0:13:24 What to do?
0:13:25 I forget what term you used.
0:13:28 I have a large action model in my head,
0:13:30 which yeah, I have thoughts about that terminology,
0:13:32 but I don’t know if they’re my own thoughts or not.
0:13:34 So we’ll get into it.
0:13:36 But you’re definitely hinting at, and again,
0:13:38 I should have upfront said this.
0:13:39 There are four phases.
0:13:41 We are at phase two.
0:13:43 I think a lot of virtual assistants are at that phase two.
0:13:46 What you’re describing in kind of intent recognition
0:13:49 or kind of passive intent recognition even
0:13:51 is there are no formal names for this,
0:13:54 but I’ll call this like an adaptive partner phase
0:13:56 where the AI is observing
0:13:58 and responding to changes on its own, right?
0:13:59 And this is the natural evolution
0:14:02 of where a lot of AI systems are heading today.
0:14:03 CES, a good example of that.
0:14:05 I have no doubt throughout the year,
0:14:07 you’ll get really close to getting here
0:14:09 as another core standard.
0:14:10 I think right now,
0:14:12 a lot of the kind of simple task completion AI
0:14:15 is a core standard across enterprise and gaming.
0:14:16 This adaptive partner phase
0:14:20 where this agent in this analogy, extended analogy,
0:14:22 would notice changes like newly built roads
0:14:25 or new residents influx into this tiny area
0:14:28 and automatically adapt construction plans.
0:14:30 It’s not micromanaging every decision,
0:14:33 but it feels like you’re collaborating with an agent
0:14:36 or a unit that has just enough context
0:14:37 to make smart decisions on its own,
0:14:40 like an evolution of a recommendation engine
0:14:43 being driven by a cognition engine here.
0:14:44 So it’s not just learning,
0:14:47 but it feels like it’s learning what we need
0:14:48 even before we ask it.
0:14:51 And I expect to see, again, a lot of this in the next year.
0:14:53 Again, I think that’s phase three.
0:14:55 I think there’s still a phase four.
0:14:57 And I think that’s a fully autonomous agent.
0:15:00 And that stage, again, continuing our analogy,
0:15:02 is a player two, right?
0:15:04 Where player three is it’s adapting to us.
0:15:07 Stage four is, hey, this thing is an agent all on its own.
0:15:10 It feels like I’m playing against another human.
0:15:11 It is making decisions that feel optimal
0:15:14 to its own objectives, alike to mine or not.
0:15:18 And I think this is where a lot of people think
0:15:19 about agentic AI.
0:15:21 They think we are there today.
0:15:24 There’s quite a big gap in the deployment of these systems
0:15:26 to get to that truly autonomous agent.
0:15:29 That is kind of the gold rush right now
0:15:32 is creating a cognition and a perception
0:15:34 and an action engine ecosystem
0:15:38 that can feel natural to a user.
0:15:39 I’m speaking with Chris Covert.
0:15:43 Chris is Director of Product Experiences at N-World AI.
0:15:45 I made him kind of, I made us kind of put a pin
0:15:47 in their product announcement to kind of bring this
0:15:50 into the, from the abstract a little bit into the concrete.
0:15:52 I’m glad you humor me, Chris, because I think kind of digging
0:15:54 into, well, what does this stuff actually mean?
0:15:55 And it’s sort of moving.
0:15:57 And one of the things I should say it’s kind of moving
0:15:58 and evolving.
0:16:00 Maybe the lingo is moving, targeting,
0:16:03 but the tech and what’s actually happening is evolving
0:16:04 and still so fast.
0:16:04 All right.
0:16:07 So let’s talk about the big announcement from CES.
0:16:10 You said you wanted to talk through kind of the design
0:16:12 process a little bit and how you make these choices.
0:16:14 So if that’s something that makes sense to you to do
0:16:17 in the context of talking about the streaming assistant,
0:16:18 great.
0:16:20 And if not, take it whichever way you want to go first.
0:16:21 That sounds perfect.
0:16:21 Yeah.
0:16:23 So I’ll be honest around here, right?
0:16:26 In any demo that you see publicly,
0:16:28 especially when you have more than one company demoing it,
0:16:31 there’s a lot of design, but also a lot of urgency
0:16:33 to show something quickly and cool.
0:16:37 So of the things that I’ll say are design axioms
0:16:40 that I love to live by, we had to sacrifice some
0:16:42 just to make sure that something awesome could
0:16:44 be demonstrated at CES.
0:16:46 Putting a timeline on creative vision is always challenging.
0:16:47 But–
0:16:47 Right.
0:16:49 And that’s like you can’t– we live in–
0:16:51 I mean, I don’t build software for a living.
0:16:52 You do.
0:16:55 But we live in such an age of like, we’re past this state,
0:16:58 where beta became, all right, everybody ships beta.
0:16:59 And then we got used to it.
0:17:01 And it’s like, right, there’ll be updates
0:17:03 and downloadable content and this and that.
0:17:05 But not CES.
0:17:05 That’s the date.
0:17:06 Show up or don’t.
0:17:07 So–
0:17:08 Exactly.
0:17:08 Exactly.
0:17:11 But in terms of design, and this might sound odd,
0:17:13 coming from a tech startup on the call here today.
0:17:16 But my first piece of advice, regardless of who we’re
0:17:18 working with, is always to ignore the technology
0:17:20 and focus solely on the experience
0:17:22 that you really want to create.
0:17:24 Where is the value in the experience?
0:17:25 The why?
0:17:28 And as soon as you can lead, or not as soon as you can lead
0:17:31 with the tech, but as soon as you do lead with the tech,
0:17:32 hey, I have an idea.
0:17:34 What if we had an LLM that could?
0:17:36 You already are risking thinking too narrowly
0:17:38 and missing bigger opportunities.
0:17:41 And I genuinely would love to say the reason behind this
0:17:42 is because it’s human-centered.
0:17:45 I come from the intersection of autonomous systems, AI,
0:17:46 and human-centered design.
0:17:49 So I would love to say that it’s a human-centered approach
0:17:50 to design, and that’s why I do it.
0:17:53 But in reality, is that the technology just advances
0:17:55 so quickly by the time your idea goes into production,
0:17:58 that tech-grounded idea that you had now
0:18:00 looks like a stone wheel compared to the race car
0:18:01 you thought it was six months ago.
0:18:05 So again, like the urgency behind some of these things
0:18:08 is challenging, especially when you come from a tech-grounded
0:18:10 design principle standard.
0:18:14 So I’m a firm believer in the moonshot first approach,
0:18:17 where you begin by clarifying why you want to build something
0:18:20 before how you decide to build it, especially for anything
0:18:22 with a shelf life greater than just a few months at this stage.
0:18:24 Again, if you start with how you end up
0:18:26 with a bunch of low-hanging fruit,
0:18:28 and a bunch of low-hanging fruit makes for a really bad smoothie.
0:18:32 So what is a moonshot experience?
0:18:35 Like, I often refer in design thinking workshops
0:18:38 to the classic impact versus feasibility matrix.
0:18:41 And this isn’t like a very exciting conversation.
0:18:42 So I’ll try and keep it high energy.
0:18:44 But typically, when you’re looking at impacts–
0:18:46 I was just saying the smoothie.
0:18:48 Sorry, I couldn’t let the smoothie go unacknowledged,
0:18:50 but I didn’t want to interrupt it.
0:18:51 Well, we’ll come back to smoothie right for now.
0:18:52 OK, good.
0:18:55 Because again, this is that it’s in that quadrant,
0:18:58 or it’s in that matrix, impact feasibility.
0:19:00 Your low-hanging fruit ideas are the things
0:19:03 that are going to be low feasibility or low impact.
0:19:06 But typically, you’re aiming for the upper right quadrant,
0:19:09 which is the highest feasibility and the highest value,
0:19:11 where impact and feasibility are so high,
0:19:13 the ideas feel easy to build
0:19:15 and have a lot of inherent value in them.
0:19:17 But what I’ve experienced when working with partners
0:19:20 and their AI ambitions is that that’s actually a trap,
0:19:23 that that quadrant right there is doomed for failure.
0:19:28 Because often the ideas that are the ones worthy of pursuit
0:19:31 the most are the ideas that almost feel impossible,
0:19:34 but if real would deliver extraordinary impact.
0:19:35 Right, that’s right.
0:19:37 As you get closer to building that value,
0:19:40 and you’re going from idea to prototype to implementation,
0:19:45 the technology is likely grown in not only capability,
0:19:47 efficiency, but also accessibility in ways
0:19:51 that probably outpace our own kind of traditional lens
0:19:54 of what is feasible and our imagination of where the tech can go.
0:19:56 So that sounds like a lot of nothing.
0:19:58 So my actual advice is,
0:20:01 when we’re designing these types of experiences,
0:20:03 start from that moonshot that feels impossible
0:20:06 and then break it down into a functional roadmap.
0:20:09 What you saw at CES in this demo with Streamlabs
0:20:11 and NVIDIA and in-world technology
0:20:13 creating the streamer assistant
0:20:15 is actually just Horizon 1, that insight.
0:20:17 What can we build that gives us insight
0:20:20 into whether this is something that provides value?
0:20:22 Where does it provide value and to who?
0:20:25 The thing we demonstrated at CES
0:20:30 was just the first pass proof of concept of this capability.
0:20:34 The kind of upper potential of where this can go,
0:20:35 where we want this to go,
0:20:37 and where we’re going to explore its value
0:20:42 is still very much architected out toward a much longer roadmap.
0:20:45 And again, showing what we showed at CES
0:20:47 and what I love about this industry right now
0:20:50 is we’re able to show really compelling integrations
0:20:53 with very meaningful use cases in this industry.
0:20:55 I’m not about to wax poetic,
0:20:58 like our industry is the same as healthcare
0:20:59 or things like that and enterprise,
0:21:02 but showing digital companions, virtual assistants,
0:21:05 digital humans at the state that we’re showing them now,
0:21:08 knowing what’s to come is an incredible place
0:21:10 to look ahead and say, “Okay, cool.
0:21:13 This scratches an itch that either the market needed
0:21:16 or that technologically is doing something
0:21:17 that could never be done traditionally before.
0:21:23 Where this moves in the next six months to six years
0:21:24 is anyone’s game.”
0:21:27 Okay. So two questions, but they’re related.
0:21:29 One’s easy. The other’s a follow-up.
0:21:32 What is the availability of the assistant?
0:21:34 It was demo state very early.
0:21:36 Do you have kind of a roadmap for that?
0:21:38 I shouldn’t say availability of a roadmap.
0:21:40 And that might be the answer to the second part,
0:21:44 but what else is on the horizon for you for in-world AI?
0:21:46 I mean, you talked a little bit before
0:21:49 about the broader horizon for agentic AI
0:21:51 and avatars and assistants and such,
0:21:52 but you can go back there if you like.
0:21:53 What’s coming up this year,
0:21:56 kind of maybe near-term that you’re excited about?
0:21:58 Well, man, this year being near-term
0:22:01 is funny at InWorld because we move very quickly.
0:22:03 This year is many near-terms stacked against each other.
0:22:06 Well, right. And I’m figuring we’re taping now,
0:22:08 but it’s going to go live.
0:22:10 You know, there’s a little bit of a buffer
0:22:12 and we reference CES.
0:22:14 Yeah. So we certainly have productization ambitions
0:22:15 for this demo.
0:22:18 What we’re doing post-CES is we’re taking the feedback
0:22:21 of what we’ve built and we’re augmenting it in many new ways.
0:22:24 Again, what was built for the demo was a proof of concept.
0:22:27 As many proof of concepts on show floors are,
0:22:29 it wasn’t robust to all the inputs we would want.
0:22:30 It wasn’t robust to all the games
0:22:32 that we would want it to be playable with.
0:22:34 So we’re trying to build out that ecosystem
0:22:35 in an intelligent strategic way
0:22:37 so that if it were to go to market,
0:22:40 it would be usable to as many streamers as possible
0:22:42 who wanted to leverage this type of technology.
0:22:46 So keep your ears on the beat for what’s about to come out.
0:22:50 I have no doubt that between Nvidia, InWorld, and Streamlabs,
0:22:52 all announcements and all possible show floors
0:22:54 that we can show our advancements on
0:22:56 will be shown at the right time.
0:22:57 So super exciting for that.
0:22:59 As it relates to InWorld, oh boy,
0:23:01 it’s such a fascinating question
0:23:03 because a lot of what we’re doing,
0:23:04 we’re doing with partners
0:23:07 with such fascinating game development lead times
0:23:09 that I hope that we’ll be playing
0:23:11 more in-world driven experiences
0:23:13 in the next, you know, when this releases,
0:23:15 but also over the next four or five years,
0:23:17 depending on the scale of the company we’re working with.
0:23:21 So I’m genuinely excited for what’s to hold
0:23:22 as our platform develops.
0:23:25 Again, as we’ve seen in just the last year or so
0:23:27 with more and more competitive players
0:23:30 in the space of providing AI tools,
0:23:32 and then just fantastic partners
0:23:35 that are helping this industry become more accessible
0:23:37 to people of all different backgrounds,
0:23:41 really hope to see the, I wouldn’t say consolidation of tools,
0:23:43 but the accessibility, I’ll keep using that word,
0:23:47 the accessibility of different AI platforms.
0:23:50 Hey, I want to use this model, but in this engine,
0:23:51 to become a lot easier.
0:23:53 And InWorld’s goal is certainly to make that happen
0:23:55 for as many industries as we can,
0:23:58 in particular the gaming and entertainment industry,
0:24:00 but it doesn’t happen without partners like Nvidia
0:24:02 and the work that you guys are doing with ACE.
0:24:05 So where I, you know, where I think InWorld is going
0:24:08 is to make that easier, to continue to work with studios
0:24:11 to find the fun and to convince every player.
0:24:14 And in particular, you know, let me be honest,
0:24:16 the YouTube commenters that, hey,
0:24:19 there is actually a world here where this technology
0:24:21 is not only fun and immersive,
0:24:23 but it’s something that the entire industry views
0:24:25 as it’s gold standard.
0:24:27 So I think we’re there a lot sooner than we think.
0:24:29 I think it’s right around the corner,
0:24:33 but could not be more excited to continue
0:24:35 to work with creatives to help them tell stories,
0:24:39 to help them, you know, flex and use their imagination
0:24:42 as much as possible to make the best possible experiences.
0:24:43 We do it every day.
0:24:45 We may not see it in the near term
0:24:47 because games take a while to make,
0:24:49 but genuinely excited.
0:24:51 Excellent. Well, I’ll offer more fun.
0:24:53 The world always needs more fun to counterbalance everything else.
0:24:56 Chris, for listeners who would like to learn more
0:24:59 about anything specific we might have talked about
0:25:01 or just broadly about InWorld,
0:25:02 where can they go online?
0:25:05 Website, obviously, but are there social handles?
0:25:08 Is there a separate blog or even research blog?
0:25:09 Where would you direct folks?
0:25:10 Yes, to all of the above.
0:25:13 You can find all of that at inworld.ai.
0:25:17 And we have our blog for partners and experiences there.
0:25:18 We have technical releases.
0:25:19 We have research done.
0:25:21 All of our socials are linked there.
0:25:22 If you want to stay up to date
0:25:24 with all the announcements that we make,
0:25:26 we have a lot of fun and we like to talk about it.
0:25:28 So definitely stay up to date.
0:25:30 Fantastic. Well, thanks for taking a minute
0:25:31 to come talk about it with us.
0:25:32 We appreciate it.
0:25:35 And best of luck to you and InWorld AI
0:25:36 with everything you’re doing this year.
0:25:39 And maybe we can do it again down the road.
0:25:41 I love it. Thank you so much, Noah. Appreciate it.
0:25:44 [MUSIC PLAYING]
0:25:48 [MUSIC PLAYING]
0:25:51 [MUSIC PLAYING]
0:25:55 [MUSIC PLAYING]
0:25:58 [MUSIC PLAYING]
0:26:02 [MUSIC PLAYING]
0:26:06 [MUSIC PLAYING]
0:26:09 [MUSIC PLAYING]
0:26:13 [MUSIC PLAYING]
0:26:16 [MUSIC PLAYING]
0:26:20 [MUSIC PLAYING]
0:26:23 [MUSIC PLAYING]
0:26:26 [MUSIC PLAYING]
0:26:30 [MUSIC PLAYING]
0:26:40 [BLANK_AUDIO]
0:00:13 Hello, and welcome to the NVIDIA AI podcast.
0:00:15 I’m your host, Noah Kravitz.
0:00:19 Digital humans and AI agents are poised to be big news this year.
0:00:21 They’re already making waves, in fact.
0:00:26 At CES 2025 in Las Vegas, Logitech G’s Stream Labs unveiled its intelligence
0:00:31 streaming assistant, powered by technologies from in-world AI and NVIDIA.
0:00:35 The intelligence streaming assistant is an AI agent designed to provide real-time
0:00:39 commentary during downtime and amplify excitement during high stakes moments
0:00:41 like boss fights or chases.
0:00:45 The collaboration brings together Stream Labs expertise in live streaming tools,
0:00:49 NVIDIA ace technology for digital humans, including AI vision models that can
0:00:53 understand what’s happening on screen, and in-world’s advanced generative AI
0:00:57 capabilities for perception, cognition, and adaptive output.
0:00:58 But what are digital humans?
0:01:00 Where are they going to be used?
0:01:03 And where are they going to make an impact in the enterprise and
0:01:05 in gaming and entertainment in particular?
0:01:08 And how does a designer design in the age of digital humans,
0:01:10 agentic AI, and beyond?
0:01:14 Chris Covert, director of product experiences in in-world AI,
0:01:18 who’s press release, by the way, or blog post, I paraphrased heartily from in
0:01:21 that introduction, so credit to.
0:01:24 Chris is here to dive into these points and more as we talk about some of the
0:01:28 technology that I think is really poised to really make a big impact this year
0:01:33 going forward and kind of shape this new area of digital experience for all of us
0:01:35 in what we’ll broadly call the AI age.
0:01:36 But at any rate, Chris is here.
0:01:41 So Chris, thank you so much for joining the AI podcast and welcome.
0:01:42 Thank you for having me today.
0:01:45 It’s a pleasure to be here, not only as a longtime partner of NVIDIA,
0:01:49 but also as a huge fan of your recent announcements in AI at CES this year as
0:01:53 well, genuinely amazing time to be talking about AI in this space.
0:01:54 It really is.
0:01:55 And so you were at the show.
0:01:56 You were in Vegas.
0:01:57 Couldn’t make it personally.
0:01:58 I’ll be doing dice.
0:02:00 I’ll be doing GDC, but our team was there.
0:02:02 We worked on the project.
0:02:03 All right, so let’s get into it.
0:02:08 Tell us a bit about in-world, in-world AI for listeners who may not know.
0:02:11 We can get deeper into the assistant in particular,
0:02:14 a little later in the conversation, if that’s cool, just so you can kind of set
0:02:18 the stage for all of us about what it is that we’re going to be talking about.
0:02:20 Yeah, so I, you know, extremely biased.
0:02:22 I have the best job in the world.
0:02:25 And in-world, we make the leading AI engine for video games.
0:02:28 And I get to work with the most creative minds in the industry,
0:02:31 most creative minds in the world of gaming and entertainment to answer the
0:02:35 question, how can we make fun more accessible to the people that make our
0:02:36 favorite experiences?
0:02:40 And I’m not going to tell you it sounds like a great job.
0:02:42 It is, it is.
0:02:43 And I’m not just blowing smoke here.
0:02:47 And we’ll definitely, as we go through this, you know, emphasize fun,
0:02:50 accessible in people, and why I think those are the most important here.
0:02:54 But in-world’s mission is to be that AI engine for this industry, right?
0:02:58 The more that the technology progresses, the lower the barrier
0:03:00 to entry to make AI experiences.
0:03:04 But we find there are still challenges, regardless of if you’re a studio
0:03:08 with a massive staff or you’re an AI kind of native company building
0:03:12 up these experiences from grassroots, that there’s a massive gap
0:03:14 between prototyping and deployment of AI systems.
0:03:19 And then to do that and find the fun for a user for an experience
0:03:20 is still incredibly challenging.
0:03:24 So we offer not only the platform, but the services to help you
0:03:26 create that fun in a deployable manner.
0:03:31 So kind of briefly run down maybe what you guys talked about at CES this year
0:03:34 and then we’ll put a pin in the assistant, like I said, and come back.
0:03:35 Yeah, that’s awesome.
0:03:38 So blog posts is out there, fantastic videos out there.
0:03:41 A lot of demonstrations of what happened, but succinctly, you know,
0:03:45 to plug the collaborative effort here in World NVIDIA and Streamlabs
0:03:50 got together for CES and we put to the test this, you know, streaming companion
0:03:52 that could not only act as a cohost to a streamer,
0:03:54 but also serve to support the creator in other ways,
0:03:58 like handling production for them on their stream as well.
0:04:01 We showed a demo of this happening at the conference, always a risk
0:04:04 because you’re playing in a big space with low, you know, internet bandwidth.
0:04:08 But we played this over Fortnite and the streamer, the demo where in this case
0:04:11 and their agentic cohost, you know, chatting during the game,
0:04:15 grilling each other for bad rounds, reacting to trends that are happening in chat,
0:04:17 cheering together when you manage to get kills.
0:04:21 But then interestingly enough, you know, when chat or when the streamer wants a replay,
0:04:24 streamer has to just ask the agent, they’ll clip it,
0:04:27 they’ll set up the screen for that replay and they’ll keep that stream feeling fluid.
0:04:30 So it’s one of those use cases, again, trying to align on like,
0:04:35 where do agentic systems actually play in enterprise and this entertainment industry
0:04:37 is we’re finding they’re hyper specialized.
0:04:40 And in this use case, we have this perfect, you know,
0:04:43 one streamer has to wear all of these different hats during their stream.
0:04:45 How can an agentic system come assist them
0:04:48 so that they can focus on making the best content
0:04:51 and all of the other things in managing that content can be done for them.
0:04:54 So back in the day, and this is part of why we got to come back to it.
0:04:58 I did a lot of YouTube stuff and we tried to do some live streaming.
0:05:00 This was in like the early mid 2000s.
0:05:05 And I remember setting up, you know, open source plugins to try to get graphics
0:05:06 over the live stream.
0:05:10 And then I had like a secondary camera, you know,
0:05:13 like a phone with a camera with a USB cable and all this stuff.
0:05:14 But that sounds amazing.
0:05:17 So I’m excited to talk about that to kind of then back into it
0:05:19 or start with the technology and go forward.
0:05:23 A genetic AI cohost digital humans.
0:05:24 Let’s talk about those things.
0:05:27 You want to start with digital humans because we’ve talked about a genetic AI
0:05:30 and we’ll get to it and it has the different, you know, what an agent is or isn’t.
0:05:33 I think it’s still a little bit malleable and context specific.
0:05:37 But when we talk about digital humans, when InWorld talks about digital humans,
0:05:38 what are we talking about?
0:05:39 You know, that’s such a great question.
0:05:41 And it brings up an important distinction.
0:05:44 When most people say digital humans, they’re thinking of chatbots.
0:05:48 Text-based tools that respond with mostly pre-programmed scripts
0:05:52 to help a specific task or guide users through a set of questions or instructions.
0:05:56 But at InWorld, we focus on AI agents that go far beyond simple chat.
0:05:59 These agents can autonomously plan, take actions,
0:06:01 and proactively engage with their environment.
0:06:07 Unlike a typical chatbot, which waits for a user to have some kind of question
0:06:12 or response in and then that AI agent provides a canned response,
0:06:15 our agents at InWorld are designed to interpret multiple types of inputs,
0:06:19 whether that’s speech or text or even vision or sensor data.
0:06:22 And then dynamically think and respond in real time.
0:06:23 They don’t just answer questions.
0:06:25 They can initiate tasks.
0:06:26 They can adapt to different contexts.
0:06:28 And they can carry out sophisticated interactions
0:06:31 that make sense within the context of what you’re doing.
0:06:34 So if you look at where digital humans are used mostly today,
0:06:35 and again, I’m super biased,
0:06:38 it’s often in the high volume chatbot or, you know,
0:06:40 personal digital assistance space.
0:06:43 But if you consider where they can have the biggest impact,
0:06:45 that’s when they function as these truly autonomous agents.
0:06:48 Planning and acting and proactively helping people
0:06:50 in ways that traditional chatbots can’t.
0:06:52 It’s not question and answer.
0:06:54 It’s answer or question that I didn’t even know I had yet.
0:06:56 And that’s the core of our focus at InWorld,
0:06:58 building AI agents that can not only look human,
0:07:02 but can think and react and solve problems like one,
0:07:05 using multiple modalities to interact with their world.
0:07:06 In our case, a digital one.
0:07:09 When it comes to building with agentic AI,
0:07:13 do you feel even like you have kind of a solid vision
0:07:15 of what is to come even near term?
0:07:20 Or is it still very early days of figuring out everything from,
0:07:23 you know, trying different types of infrastructure
0:07:26 to, you know, sort of the design and higher level thinking
0:07:28 about how to architect these things?
0:07:28 That’s a great question.
0:07:32 And the reality is we’re shifting toward an agentic AI framework,
0:07:33 as I mentioned before.
0:07:37 And shifting toward that framework is no longer really an option.
0:07:40 It’s becoming essential for us and our partners.
0:07:44 When you need any sense of autonomy or kind of real time
0:07:46 or runtime adaptability, latency optimization,
0:07:48 any sense of custom decision making,
0:07:52 you need a robust framework that lets you take control of that stack.
0:07:55 No one platform is going to serve the needs of our industry
0:07:56 as standalone.
0:07:58 So building a framework that is flexible,
0:08:00 that is robust to the needs and growing needs,
0:08:03 changing needs, super diverse needs of our industry
0:08:05 is incredibly important.
0:08:06 What we’re seeing more and more of is enterprises
0:08:09 that want to own that architecture end to end.
0:08:11 What I mean by that is they need the flexibility
0:08:14 to decide what models to use, where to use them,
0:08:16 how data is fed into their systems,
0:08:18 how to manage compliance and risk,
0:08:21 all of these factors that require really custom logic
0:08:22 and custom implementations.
0:08:24 So it’s adopting this, you know,
0:08:28 agentic AI framework that really puts the power in their hands.
0:08:30 It’s not just about future proofing.
0:08:33 It’s about giving them full control over how AI evolves
0:08:35 within their organization at any given time, right?
0:08:37 The industry is going to move quickly.
0:08:38 We want to make sure our partners can move
0:08:41 just as quickly as they get inspired.
0:08:45 But to me, you know, my favorite key differentiator here,
0:08:47 Future of Inworld, why the framework model,
0:08:50 why all of these changes is that such a framework
0:08:52 doesn’t just guarantee technical flexibility.
0:08:54 That’s great, and that’s going to make a lot of people happy,
0:08:57 but it also opens up the creative design space as well, right?
0:09:00 By collaborating with people that are thinking
0:09:02 of the most outrageously beautiful,
0:09:05 outrageously wonderfully weird experiences
0:09:07 you could possibly imagine and giving them tools
0:09:09 that are opened up to really let them craft
0:09:11 their own AI architectures.
0:09:14 We’re really, you know, helping push the boundaries
0:09:18 of what and how we can deliver innovative experiences, right?
0:09:21 We made the jokes earlier, but if we say that conversational AI,
0:09:22 you know, this like chat bot nature,
0:09:26 is now feeling very entry level, it’s feeling baseline,
0:09:28 then what we’re doing with this agentic framework
0:09:30 is building out that blueprint for the future
0:09:32 so that our partners and our customers
0:09:37 can help imagine, build, and deploy AI experiences
0:09:40 that genuinely didn’t feel possible even six months ago, right?
0:09:41 Taking that to a whole new level.
0:09:44 If we go more than like two years out,
0:09:47 I’m at a 10% confidence level of where this is actually going to be.
0:09:50 Yeah, I appreciate the candor.
0:09:52 Yeah, it’s just, you know,
0:09:54 and I’ve been doing this a couple of times in my career throughout
0:09:57 where I’ll work with somebody in an advisory capacity
0:10:00 where we’re looking at the future three to five years out
0:10:02 and by the time we finish our analysis,
0:10:04 the thing that we had two years out is now open sourced
0:10:06 and you’re like, cool, so back to the drawing board
0:10:09 because it’s just the industry moves so quickly
0:10:13 that what is possible, it’s actually really hard to nail down
0:10:16 and we will get into kind of, hopefully in this conversation,
0:10:20 how designing around AI and these systems is that challenge
0:10:23 because I think we’ve learned some fantastic lessons
0:10:25 that hopefully people listening can take away as well.
0:10:29 I really see there, you know, when it comes to trying not to sound too obvious
0:10:31 when we’re talking about agents,
0:10:34 the evolutionary chain of how an agent will grow,
0:10:36 how an agent, you know, the technology behind it
0:10:38 will proceed in even the near future.
0:10:41 It comes all down to agency and again, it’s agents.
0:10:44 It sounds obvious, but what I really mean by that
0:10:46 is the complexity of that cognition engine
0:10:49 and I’ll try and crack this net with an analogy
0:10:51 and I’ll try and make it fast because I could talk about this.
0:10:52 Now, take your time.
0:10:53 We’ll take your time with the analogy anyway.
0:10:58 We have this, you know, first again is that conversational AI phase
0:11:00 and I’ll use a gaming analogy, right?
0:11:04 The conversational AI phase gives avatars, gives agents,
0:11:05 I’ll use them interchangeably today,
0:11:08 extremely little agency in doing anything other than speaking, right?
0:11:12 It may be able to respond to my input if I ask it to do something,
0:11:14 but it’s not going to physically change the state of something
0:11:17 other than the dialogue it’s going to tell me back.
0:11:19 So, I think in terms of an analogy,
0:11:21 like a city-building simulation game,
0:11:24 in this phase, conversational AI,
0:11:25 I can ask it about architecture
0:11:27 and it could give me some facts about architecture.
0:11:30 I can ask it about plans I have to build this little area
0:11:31 and it could give me some advice.
0:11:34 I could ask it, “Hey, how do I limit the congestion
0:11:37 or how do I maximize traffic over this little commercial area?”
0:11:41 And it could tell me in words how to potentially do those things,
0:11:43 but it won’t be able to do things like place buildings
0:11:46 or reconstruct roads or even select the optimal facade
0:11:48 for the style of the region
0:11:50 because that’s completely out of its scope.
0:11:50 And not to…
0:11:51 It can’t act.
0:11:52 Yeah, it can’t act.
0:11:53 It can’t act at all.
0:11:53 It is…
0:11:55 Okay, just making sure that I’m fine.
0:11:55 Yeah.
0:11:58 It is a very rudimentary perception,
0:11:59 kind of what is it seeing,
0:12:02 what context does it have and cognition,
0:12:04 how is it planning, how is it reasoning,
0:12:06 very simple engines that drive that.
0:12:08 Really, I’m going to talk to it.
0:12:10 It’s going to say, “Oh, you know,
0:12:12 this person cares about architecture.
0:12:14 I’m going to pull everything I know about architecture,
0:12:16 turn that into a nice response,
0:12:19 weave together kind of their question into my response,
0:12:22 and then boom, now we have simple conversation AI.”
0:12:24 But not too far ahead of that.
0:12:26 And I say that lovingly and endearingly
0:12:29 is a simple task completion AI.
0:12:31 It’s very much still call and response.
0:12:33 You have an action engine.
0:12:34 You say, “I’m having an agent
0:12:37 that can only construct new buildings
0:12:38 exactly where and when I tell them.
0:12:40 I can say place a building at this intersection
0:12:41 and it does it, boom.
0:12:43 I can say change the building to this type
0:12:44 and boom, it does it.”
0:12:44 Right.
0:12:46 And while you argue that
0:12:47 that requires a little bit more perception,
0:12:49 maybe a little bit more reasoning,
0:12:50 definitely some action,
0:12:53 it’s limited to an extremely small set of actions
0:12:56 and it’s pretty much just scripting with extra steps.
0:12:58 It knows what it’s going to expect.
0:12:59 It’s waiting for me to say it in a way
0:13:02 that it can map to that is the place of building action.
0:13:04 I’m going to place a building done.
0:13:08 If a language model provides the description,
0:13:11 including like I wish I could change the background
0:13:13 to mountains or whatever, right?
0:13:15 Or envision a background with mountains
0:13:16 and it can’t act, right?
0:13:17 Can the system be automated?
0:13:18 What you’re talking about
0:13:23 so that the language model can tell the action cognition model?
0:13:24 What to do?
0:13:25 I forget what term you used.
0:13:28 I have a large action model in my head,
0:13:30 which yeah, I have thoughts about that terminology,
0:13:32 but I don’t know if they’re my own thoughts or not.
0:13:34 So we’ll get into it.
0:13:36 But you’re definitely hinting at, and again,
0:13:38 I should have upfront said this.
0:13:39 There are four phases.
0:13:41 We are at phase two.
0:13:43 I think a lot of virtual assistants are at that phase two.
0:13:46 What you’re describing in kind of intent recognition
0:13:49 or kind of passive intent recognition even
0:13:51 is there are no formal names for this,
0:13:54 but I’ll call this like an adaptive partner phase
0:13:56 where the AI is observing
0:13:58 and responding to changes on its own, right?
0:13:59 And this is the natural evolution
0:14:02 of where a lot of AI systems are heading today.
0:14:03 CES, a good example of that.
0:14:05 I have no doubt throughout the year,
0:14:07 you’ll get really close to getting here
0:14:09 as another core standard.
0:14:10 I think right now,
0:14:12 a lot of the kind of simple task completion AI
0:14:15 is a core standard across enterprise and gaming.
0:14:16 This adaptive partner phase
0:14:20 where this agent in this analogy, extended analogy,
0:14:22 would notice changes like newly built roads
0:14:25 or new residents influx into this tiny area
0:14:28 and automatically adapt construction plans.
0:14:30 It’s not micromanaging every decision,
0:14:33 but it feels like you’re collaborating with an agent
0:14:36 or a unit that has just enough context
0:14:37 to make smart decisions on its own,
0:14:40 like an evolution of a recommendation engine
0:14:43 being driven by a cognition engine here.
0:14:44 So it’s not just learning,
0:14:47 but it feels like it’s learning what we need
0:14:48 even before we ask it.
0:14:51 And I expect to see, again, a lot of this in the next year.
0:14:53 Again, I think that’s phase three.
0:14:55 I think there’s still a phase four.
0:14:57 And I think that’s a fully autonomous agent.
0:15:00 And that stage, again, continuing our analogy,
0:15:02 is a player two, right?
0:15:04 Where player three is it’s adapting to us.
0:15:07 Stage four is, hey, this thing is an agent all on its own.
0:15:10 It feels like I’m playing against another human.
0:15:11 It is making decisions that feel optimal
0:15:14 to its own objectives, alike to mine or not.
0:15:18 And I think this is where a lot of people think
0:15:19 about agentic AI.
0:15:21 They think we are there today.
0:15:24 There’s quite a big gap in the deployment of these systems
0:15:26 to get to that truly autonomous agent.
0:15:29 That is kind of the gold rush right now
0:15:32 is creating a cognition and a perception
0:15:34 and an action engine ecosystem
0:15:38 that can feel natural to a user.
0:15:39 I’m speaking with Chris Covert.
0:15:43 Chris is Director of Product Experiences at N-World AI.
0:15:45 I made him kind of, I made us kind of put a pin
0:15:47 in their product announcement to kind of bring this
0:15:50 into the, from the abstract a little bit into the concrete.
0:15:52 I’m glad you humor me, Chris, because I think kind of digging
0:15:54 into, well, what does this stuff actually mean?
0:15:55 And it’s sort of moving.
0:15:57 And one of the things I should say it’s kind of moving
0:15:58 and evolving.
0:16:00 Maybe the lingo is moving, targeting,
0:16:03 but the tech and what’s actually happening is evolving
0:16:04 and still so fast.
0:16:04 All right.
0:16:07 So let’s talk about the big announcement from CES.
0:16:10 You said you wanted to talk through kind of the design
0:16:12 process a little bit and how you make these choices.
0:16:14 So if that’s something that makes sense to you to do
0:16:17 in the context of talking about the streaming assistant,
0:16:18 great.
0:16:20 And if not, take it whichever way you want to go first.
0:16:21 That sounds perfect.
0:16:21 Yeah.
0:16:23 So I’ll be honest around here, right?
0:16:26 In any demo that you see publicly,
0:16:28 especially when you have more than one company demoing it,
0:16:31 there’s a lot of design, but also a lot of urgency
0:16:33 to show something quickly and cool.
0:16:37 So of the things that I’ll say are design axioms
0:16:40 that I love to live by, we had to sacrifice some
0:16:42 just to make sure that something awesome could
0:16:44 be demonstrated at CES.
0:16:46 Putting a timeline on creative vision is always challenging.
0:16:47 But–
0:16:47 Right.
0:16:49 And that’s like you can’t– we live in–
0:16:51 I mean, I don’t build software for a living.
0:16:52 You do.
0:16:55 But we live in such an age of like, we’re past this state,
0:16:58 where beta became, all right, everybody ships beta.
0:16:59 And then we got used to it.
0:17:01 And it’s like, right, there’ll be updates
0:17:03 and downloadable content and this and that.
0:17:05 But not CES.
0:17:05 That’s the date.
0:17:06 Show up or don’t.
0:17:07 So–
0:17:08 Exactly.
0:17:08 Exactly.
0:17:11 But in terms of design, and this might sound odd,
0:17:13 coming from a tech startup on the call here today.
0:17:16 But my first piece of advice, regardless of who we’re
0:17:18 working with, is always to ignore the technology
0:17:20 and focus solely on the experience
0:17:22 that you really want to create.
0:17:24 Where is the value in the experience?
0:17:25 The why?
0:17:28 And as soon as you can lead, or not as soon as you can lead
0:17:31 with the tech, but as soon as you do lead with the tech,
0:17:32 hey, I have an idea.
0:17:34 What if we had an LLM that could?
0:17:36 You already are risking thinking too narrowly
0:17:38 and missing bigger opportunities.
0:17:41 And I genuinely would love to say the reason behind this
0:17:42 is because it’s human-centered.
0:17:45 I come from the intersection of autonomous systems, AI,
0:17:46 and human-centered design.
0:17:49 So I would love to say that it’s a human-centered approach
0:17:50 to design, and that’s why I do it.
0:17:53 But in reality, is that the technology just advances
0:17:55 so quickly by the time your idea goes into production,
0:17:58 that tech-grounded idea that you had now
0:18:00 looks like a stone wheel compared to the race car
0:18:01 you thought it was six months ago.
0:18:05 So again, like the urgency behind some of these things
0:18:08 is challenging, especially when you come from a tech-grounded
0:18:10 design principle standard.
0:18:14 So I’m a firm believer in the moonshot first approach,
0:18:17 where you begin by clarifying why you want to build something
0:18:20 before how you decide to build it, especially for anything
0:18:22 with a shelf life greater than just a few months at this stage.
0:18:24 Again, if you start with how you end up
0:18:26 with a bunch of low-hanging fruit,
0:18:28 and a bunch of low-hanging fruit makes for a really bad smoothie.
0:18:32 So what is a moonshot experience?
0:18:35 Like, I often refer in design thinking workshops
0:18:38 to the classic impact versus feasibility matrix.
0:18:41 And this isn’t like a very exciting conversation.
0:18:42 So I’ll try and keep it high energy.
0:18:44 But typically, when you’re looking at impacts–
0:18:46 I was just saying the smoothie.
0:18:48 Sorry, I couldn’t let the smoothie go unacknowledged,
0:18:50 but I didn’t want to interrupt it.
0:18:51 Well, we’ll come back to smoothie right for now.
0:18:52 OK, good.
0:18:55 Because again, this is that it’s in that quadrant,
0:18:58 or it’s in that matrix, impact feasibility.
0:19:00 Your low-hanging fruit ideas are the things
0:19:03 that are going to be low feasibility or low impact.
0:19:06 But typically, you’re aiming for the upper right quadrant,
0:19:09 which is the highest feasibility and the highest value,
0:19:11 where impact and feasibility are so high,
0:19:13 the ideas feel easy to build
0:19:15 and have a lot of inherent value in them.
0:19:17 But what I’ve experienced when working with partners
0:19:20 and their AI ambitions is that that’s actually a trap,
0:19:23 that that quadrant right there is doomed for failure.
0:19:28 Because often the ideas that are the ones worthy of pursuit
0:19:31 the most are the ideas that almost feel impossible,
0:19:34 but if real would deliver extraordinary impact.
0:19:35 Right, that’s right.
0:19:37 As you get closer to building that value,
0:19:40 and you’re going from idea to prototype to implementation,
0:19:45 the technology is likely grown in not only capability,
0:19:47 efficiency, but also accessibility in ways
0:19:51 that probably outpace our own kind of traditional lens
0:19:54 of what is feasible and our imagination of where the tech can go.
0:19:56 So that sounds like a lot of nothing.
0:19:58 So my actual advice is,
0:20:01 when we’re designing these types of experiences,
0:20:03 start from that moonshot that feels impossible
0:20:06 and then break it down into a functional roadmap.
0:20:09 What you saw at CES in this demo with Streamlabs
0:20:11 and NVIDIA and in-world technology
0:20:13 creating the streamer assistant
0:20:15 is actually just Horizon 1, that insight.
0:20:17 What can we build that gives us insight
0:20:20 into whether this is something that provides value?
0:20:22 Where does it provide value and to who?
0:20:25 The thing we demonstrated at CES
0:20:30 was just the first pass proof of concept of this capability.
0:20:34 The kind of upper potential of where this can go,
0:20:35 where we want this to go,
0:20:37 and where we’re going to explore its value
0:20:42 is still very much architected out toward a much longer roadmap.
0:20:45 And again, showing what we showed at CES
0:20:47 and what I love about this industry right now
0:20:50 is we’re able to show really compelling integrations
0:20:53 with very meaningful use cases in this industry.
0:20:55 I’m not about to wax poetic,
0:20:58 like our industry is the same as healthcare
0:20:59 or things like that and enterprise,
0:21:02 but showing digital companions, virtual assistants,
0:21:05 digital humans at the state that we’re showing them now,
0:21:08 knowing what’s to come is an incredible place
0:21:10 to look ahead and say, “Okay, cool.
0:21:13 This scratches an itch that either the market needed
0:21:16 or that technologically is doing something
0:21:17 that could never be done traditionally before.
0:21:23 Where this moves in the next six months to six years
0:21:24 is anyone’s game.”
0:21:27 Okay. So two questions, but they’re related.
0:21:29 One’s easy. The other’s a follow-up.
0:21:32 What is the availability of the assistant?
0:21:34 It was demo state very early.
0:21:36 Do you have kind of a roadmap for that?
0:21:38 I shouldn’t say availability of a roadmap.
0:21:40 And that might be the answer to the second part,
0:21:44 but what else is on the horizon for you for in-world AI?
0:21:46 I mean, you talked a little bit before
0:21:49 about the broader horizon for agentic AI
0:21:51 and avatars and assistants and such,
0:21:52 but you can go back there if you like.
0:21:53 What’s coming up this year,
0:21:56 kind of maybe near-term that you’re excited about?
0:21:58 Well, man, this year being near-term
0:22:01 is funny at InWorld because we move very quickly.
0:22:03 This year is many near-terms stacked against each other.
0:22:06 Well, right. And I’m figuring we’re taping now,
0:22:08 but it’s going to go live.
0:22:10 You know, there’s a little bit of a buffer
0:22:12 and we reference CES.
0:22:14 Yeah. So we certainly have productization ambitions
0:22:15 for this demo.
0:22:18 What we’re doing post-CES is we’re taking the feedback
0:22:21 of what we’ve built and we’re augmenting it in many new ways.
0:22:24 Again, what was built for the demo was a proof of concept.
0:22:27 As many proof of concepts on show floors are,
0:22:29 it wasn’t robust to all the inputs we would want.
0:22:30 It wasn’t robust to all the games
0:22:32 that we would want it to be playable with.
0:22:34 So we’re trying to build out that ecosystem
0:22:35 in an intelligent strategic way
0:22:37 so that if it were to go to market,
0:22:40 it would be usable to as many streamers as possible
0:22:42 who wanted to leverage this type of technology.
0:22:46 So keep your ears on the beat for what’s about to come out.
0:22:50 I have no doubt that between Nvidia, InWorld, and Streamlabs,
0:22:52 all announcements and all possible show floors
0:22:54 that we can show our advancements on
0:22:56 will be shown at the right time.
0:22:57 So super exciting for that.
0:22:59 As it relates to InWorld, oh boy,
0:23:01 it’s such a fascinating question
0:23:03 because a lot of what we’re doing,
0:23:04 we’re doing with partners
0:23:07 with such fascinating game development lead times
0:23:09 that I hope that we’ll be playing
0:23:11 more in-world driven experiences
0:23:13 in the next, you know, when this releases,
0:23:15 but also over the next four or five years,
0:23:17 depending on the scale of the company we’re working with.
0:23:21 So I’m genuinely excited for what’s to hold
0:23:22 as our platform develops.
0:23:25 Again, as we’ve seen in just the last year or so
0:23:27 with more and more competitive players
0:23:30 in the space of providing AI tools,
0:23:32 and then just fantastic partners
0:23:35 that are helping this industry become more accessible
0:23:37 to people of all different backgrounds,
0:23:41 really hope to see the, I wouldn’t say consolidation of tools,
0:23:43 but the accessibility, I’ll keep using that word,
0:23:47 the accessibility of different AI platforms.
0:23:50 Hey, I want to use this model, but in this engine,
0:23:51 to become a lot easier.
0:23:53 And InWorld’s goal is certainly to make that happen
0:23:55 for as many industries as we can,
0:23:58 in particular the gaming and entertainment industry,
0:24:00 but it doesn’t happen without partners like Nvidia
0:24:02 and the work that you guys are doing with ACE.
0:24:05 So where I, you know, where I think InWorld is going
0:24:08 is to make that easier, to continue to work with studios
0:24:11 to find the fun and to convince every player.
0:24:14 And in particular, you know, let me be honest,
0:24:16 the YouTube commenters that, hey,
0:24:19 there is actually a world here where this technology
0:24:21 is not only fun and immersive,
0:24:23 but it’s something that the entire industry views
0:24:25 as it’s gold standard.
0:24:27 So I think we’re there a lot sooner than we think.
0:24:29 I think it’s right around the corner,
0:24:33 but could not be more excited to continue
0:24:35 to work with creatives to help them tell stories,
0:24:39 to help them, you know, flex and use their imagination
0:24:42 as much as possible to make the best possible experiences.
0:24:43 We do it every day.
0:24:45 We may not see it in the near term
0:24:47 because games take a while to make,
0:24:49 but genuinely excited.
0:24:51 Excellent. Well, I’ll offer more fun.
0:24:53 The world always needs more fun to counterbalance everything else.
0:24:56 Chris, for listeners who would like to learn more
0:24:59 about anything specific we might have talked about
0:25:01 or just broadly about InWorld,
0:25:02 where can they go online?
0:25:05 Website, obviously, but are there social handles?
0:25:08 Is there a separate blog or even research blog?
0:25:09 Where would you direct folks?
0:25:10 Yes, to all of the above.
0:25:13 You can find all of that at inworld.ai.
0:25:17 And we have our blog for partners and experiences there.
0:25:18 We have technical releases.
0:25:19 We have research done.
0:25:21 All of our socials are linked there.
0:25:22 If you want to stay up to date
0:25:24 with all the announcements that we make,
0:25:26 we have a lot of fun and we like to talk about it.
0:25:28 So definitely stay up to date.
0:25:30 Fantastic. Well, thanks for taking a minute
0:25:31 to come talk about it with us.
0:25:32 We appreciate it.
0:25:35 And best of luck to you and InWorld AI
0:25:36 with everything you’re doing this year.
0:25:39 And maybe we can do it again down the road.
0:25:41 I love it. Thank you so much, Noah. Appreciate it.
0:25:44 [MUSIC PLAYING]
0:25:48 [MUSIC PLAYING]
0:25:51 [MUSIC PLAYING]
0:25:55 [MUSIC PLAYING]
0:25:58 [MUSIC PLAYING]
0:26:02 [MUSIC PLAYING]
0:26:06 [MUSIC PLAYING]
0:26:09 [MUSIC PLAYING]
0:26:13 [MUSIC PLAYING]
0:26:16 [MUSIC PLAYING]
0:26:20 [MUSIC PLAYING]
0:26:23 [MUSIC PLAYING]
0:26:26 [MUSIC PLAYING]
0:26:30 [MUSIC PLAYING]
0:26:40 [BLANK_AUDIO]
AI agents with advanced perception and cognition capabilities are making digital experiences more dynamic and personalized across industries. In this episode of the NVIDIA AI Podcast, Inworld AI’s Chris Covert discusses how intelligent digital humans are reshaping interactive experiences, from gaming to healthcare, and emphasizes that the key to meaningful AI experiences lies in focusing on user value rather than just technology.