AI transcript
0:00:16 Hello, and welcome to the NVIDIA AI podcast. I’m your host, Noah Kravitz. Our guests today
0:00:20 have known and worked with one another for years and can speak to the ins and outs of
0:00:25 enterprise technology, including developing AI applications, from the perspective of brands,
0:00:29 vendors, developers, customers, and probably most anyone else you can think of.
0:00:34 Tony Ambrosie is Senior Vice President and Chief Digital and Technology Officer of Pharmacy and
0:00:41 Consumer Wellness for CVS Health, and Orijit Sengupta is Founder and CEO of Able. Tony and
0:00:48 are here to talk about developing AI apps at massive scale, using DGX Cloud to do it, and the current and
0:00:53 future of generative AI, agentech AI frameworks, and so much more. So I’m going to get out of the way
0:00:58 and introduce these two gentlemen to share some tales and drop some knowledge on us in the next
0:01:02 half an hour. Tony, Orijit, thank you so much for joining the AI podcast.
0:01:03 Well, thank you for inviting us.
0:01:10 So if you would, maybe we can start a little bit with your backgrounds and kind of how you’ve
0:01:15 been working together forever. So I’ll let you kind of start at the point that seems apt to get us into
0:01:17 what CVS and Able are doing now.
0:01:25 So I’ll start. So I’ve been doing this for technology and digital and then at some point data, machine
0:01:31 learning for quite a while, trying to be at the forefront of technology to support customers and
0:01:39 employees and colleagues and wherever else there. I’ve spent my last five years in healthcare, three years
0:01:52 in a hospital system in South Florida, obviously focused on patients and patients’ well-being. I’ve been with CVS for almost two years now, the same type of focus.
0:02:13 And I think Arjit and I kind of bumped into each other 2016, 2017, I think. And then we saw that it would be cool to do interesting things with this machine learning thing because we can transform a lot of, make a lot of things better. At that time, I was with Disney and make a lot of things much better for consumers and guests.
0:02:15 So, Arjit, where were you when you guys met?
0:02:33 So I had, right before that, sold my previous AI company to Salesforce, where it became Salesforce Einstein Discovery. So I started something about Leoncore, which was one of the first augmented analytics companies on the planet, and ended up having a crisis of faith. I was like, did I create value or did I just make some money?
0:02:56 Right. So I ended up writing a book called AI is a waste of money, where I’d taken a thousand AI projects and I laid out why these projects fail. And one of the most common reasons was disconnect between the business user and data scientists. And this is where Tony and I always had this fun thing, which was like, it doesn’t matter what you build unless the business stakeholders adopt it.
0:03:19 Right. So that’s how our partnership actually started. In fact, the first time I was actually trying to sell to him, I came to him and said, Tony, keep me honest here. Something like, we’ll do this prototype in 30 days so that your business users can see. And he said something like, do it in five days or something. And then we did it in two, three days. Tony, am I remembering this ballpark correct? Like it was something absurd like that.
0:03:30 I think so. So some of the folks who are listening to this find that that can you do it in less than, you know, in a week and five days, you’re absolutely right.
0:03:49 It’s coming. Yeah, you and I talked a lot. And I think you guys are very, very good at that, which is when you experiment, you try things. That’s where the discussion really is. If you have cycles of okay, we talked for a week about what we want to do.
0:04:06 And then we go and do it for two weeks or even more. Then we come back for another week and evaluate that is not what we wanted or not what the business wanted or the outcomes are not met. And then, so you have the cycle that takes months, everybody will forget about what they wanted to do in the first place.
0:04:20 So being able to slice things and then do something in five days, which is, I mean, how many of us remember what we did on a Thursday, three months ago?
0:04:23 You know, we do remember why we talked about it on Monday.
0:04:25 I’m just trying to remember if today is Thursday.
0:04:26 Yes.
0:04:41 I’m going to take that as a chance to segue and getting under the hood here, so to speak, with you guys. How do you approach that now when, let’s say you’re starting an AI project or, you know, further along in the cycle as well when you’re kind of iterating?
0:04:51 How do you approach that now, working together on projects for CVS? And I would imagine, you know, at that scale and all the other considerations, but how do you get started?
0:05:00 So there’s two key parts to it. One is for any use case, we go after 48-hour rapid prototype, no longer than that, which means from when we first start.
0:05:06 And we already implemented inside CVS of data protection and all that is very crucial to CVS.
0:05:12 This is all happening within CVS system control, right? That’s why you can kind of get going that fast.
0:05:18 But Tony’s team blesses a project, within 48 hours, a prototype has to be done.
0:05:18 Okay.
0:05:26 And that has to go in front of business for feedback. It’s not just IT or analytics, just thinking about it. If business is not involved, it doesn’t matter.
0:05:38 Separately, his teams are building their own prototypes, right? So my team will do soundish prototypes when Tony blesses it, but the teams have been allowed to build whatever agents they want to build, right?
0:05:48 And then once business blesses it, then you got to build it within 30 days, meaning something real has to be in the hands of business at scale in 30 days.
0:05:49 In 30 days.
0:05:55 When you set those kinds of expectations, AI is exciting. So people will sign up for projects, but they never finish it all.
0:05:56 Sure.
0:06:02 You’ve got to have this kind of, and sometimes they want an AI, they don’t really know what they mean and they don’t really know what they want, right?
0:06:03 Right.
0:06:13 When you get a prototype in front of them, they’ll either like it or dislike it, and they can tell you what they disliked about it. Then you iterate and improve it until you get to an alignment.
0:06:18 Right. You give somebody something to react to. It’s so different from even a blank check, you know, and they’ll, well, what do I write?
0:06:29 So you mentioned the Tony’s team and your teams. And I think in the beginning, talking about your origin story, so to speak, with Tony, I might’ve glossed over a little bit.
0:06:37 Able is providing the development platform services. What is Able providing? Kind of how did the Tony’s team and your team fit together?
0:06:46 We have an agent developing platform, but it’s a no code, low code. In fact, most of the time, it’s like no code at all kind of a platform.
0:06:46 Okay.
0:06:57 So we have the software that runs on top of NVIDIA. So for example, we are using things like NIMS and Nemos and all that good stuff, but Tony’s team doesn’t have to worry about that, right?
0:07:09 So we leverage all of NVIDIA’s capabilities within that. And then when his team comes in and says, hey, I want to look at what’s happened. Actually, I can’t mention specific use cases, but let’s say I want to understand what’s going on with this data.
0:07:12 The software does it, presents it back for feedback.
0:07:24 Yeah. And I would add that’s, that’s accurate. So we do a lot of work. Some of it. Thank you for helping other things, you know, some of the bigger things. There’s a lot of work.
0:07:41 What I would say is, and this is my boss always emphasizes, we have a particular, given that the domain where, you know, patients, consumers, we are have a particular focus on what we call responsible AI and maybe others call it bigger. So that is the fact there.
0:07:54 Before you delve into that, apologies, but just, just in case I’m thinking I grew up with CVS, but I think of it as, you know, the, the pharmacy, the store. CVS Health is the second largest healthcare provider in the United States under its full umbrella.
0:07:58 Right. But you’re focusing on the stores in the pharmacy.
0:07:59 Correct. Correct.
0:08:13 And, you know, obviously in, within that, because we have all the retail customers, you know, my focus with technology, with digital product is how do we make that experience for customers better?
0:08:18 Right. And of course, constantly innovating on how to make it, you know, prove the safety.
0:08:19 Right. Right.
0:08:21 Right. So the responsible AI started.
0:08:35 That’s where these, exactly. And also, how do we help our colleagues with things that probably I can do in terms of automation, while also, also always keeping the human in the loop and in control.
0:08:50 Yeah. So obviously we do some things that are pretty, pretty non, you know, non health, healthcare, a bunch of other things, you know, my finance partners and my procurement partners say, oh, can you write the agent for me to whatever, you know, summarize all the invoices?
0:08:54 Perfect. Sure. Of course. But there are others that we’re extraordinarily careful.
0:09:00 Right. Are there any specific applications that CVS has implemented that you can talk about?
0:09:18 There are quite a few. I think some of them probably I will not go into, but others, what we’re doing is, and it’s part of the, in progress of being implemented is, and that’s where NVIDIA plays a big part, is what we call a conversation AI.
0:09:39 Right. The way I look at it from a digital perspective is we want to help customers with whatever they need. Now, of course, we’re not talking about, you know, dispensing medication and, you know, the, what the pharmacist does, putting things in a bowl and making sure everything is right, but everything else, we want to make sure that customers can have access to us in all sorts of channels.
0:09:42 Right. So maybe reordering or refilling the prescription.
0:10:00 Absolutely. You know, you know, if somebody needs a more of a pharmacology or medical advice, that’s where for pharmacists and doctors, but things like that, I call some procedural, yes, like when is my prescription ready?
0:10:12 Those would try to, if they choose to go into the voice channel, we have, you know, AI being able to provide that with precision. And that’s a big thing because it’s a huge channel.
0:10:24 So, Tony, you mentioned AI agents, building an agent. I think the off-the-cuff example was to summarize receipts. I’m going to put you on the spot here. Can you first talk about what an AI agent is?
0:10:43 It’s sort of one of those things that building the definition as we fly the plane, maybe that kind of thing. And then get into, you know, in your experience and Orgid as well, obviously, some of the key considerations that go into thinking about, you know, and then designing and developing, I guess, agents for different use cases.
0:11:03 Sure. So, it’s interesting. And we talked earlier about how fast things move. All right. So, some of us got some inkling, probably Orgid more than I, you know, in 2018, 2019, about kind of what’s happening with these large models, right?
0:11:16 And the transformers, Google’s transformers in 2017, kind of opened up the avenue. And then, obviously, we got the chat GPT in 2022 and everything coming from there.
0:11:24 But it was an interesting thing that because these chatbots, and obviously, they were the API, they were the API behind them.
0:11:43 They could tell things, tell us things. They could think somehow, somewhat, but they couldn’t do anything, right? You know, you have the womb, but the human going in and asking a question and then taking that in some shape or fashion and doing something in some workflow.
0:11:52 Right. And then some people said, well, wait a minute, we’re kind of like the plumbers for AI, you know, there’s AI is thinking and we’re, you know, we’re doing.
0:12:09 So, agents came in the, and clearly, this has been a topic in the, you know, sci-fi for 50 years. Agents came at the intersection, really, of intelligent machines through large language models and the, let’s say, process automation.
0:12:18 And, you know, we’ve done LLMs for, for a few years now, a couple of years at least, you know, we’ve done RPA for 10 years.
0:12:27 And then, this is where we kind of bring them together and say, this, these things called AI agents and agenting AI.
0:12:35 By the way, don’t you like how we geeks in technology come up with all these fancy terms and cloud and agenting AI, all this stuff?
0:12:41 You know, I’m, I’m a writer at heart. So, so that maybe that’s what actually drew me to technology, I just didn’t realize.
0:12:53 So that’s kind of, we married those where the intelligence behind the, the LLMs now drives through the use of tools, process automation and process execution.
0:12:59 Because we had that gap of execution gap where those things were, were disconnected.
0:13:00 Right.
0:13:08 Of course, there are, I, I’ve seen, uh, researchers, uh, talk about different level of maturity, uh, like everything else.
0:13:17 You know, I think you’re starting with zero basically as manual work, um, going to all the way to some size, which is completely autonomous.
0:13:22 where basically every, you know, AI is doing everything, the AI agents are doing everything.
0:13:24 There’s no human in the loop at five.
0:13:28 Yeah. I don’t know what kind of world that would be in, but let’s not go there.
0:13:29 Right.
0:13:42 But I think most of us are somewhere in the two, two, three, you know, some level of, uh, some level of intelligence coupled with some level of doing through tools and, and, and within workflows.
0:13:54 When you mentioned tools and we talk about the agents being able to go and take actions and do things, what are some of the examples, you know, either within CVS or, or perhaps Arjit, uh, that, you know, ABLE has implemented.
0:14:04 And you’ve been thinking about just for folks who are maybe understand this idea of agents more in an abstract level and having, you know, gone hands on, what are some of these actions that we’re talking about?
0:14:16 Sure. So the, the tools enable the LLM to actually take their insight or their decision to us to operate the task and actually do it.
0:14:25 And I think there are plenty already out in the market where an agent can basically do whatever a user in front of a computer do.
0:14:35 So they can control the screen, they can navigate a browser, they can fill in forms, um, figuring out what the page is about.
0:14:43 And that’s kind of a little bit between the difference between RPA, where we would have to code all that in and say, Hey, you know, look into the HTML, uh, source.
0:14:51 And then when, whenever you find the, the label name, you put the name and then so on and so forth, the agents through that intelligence.
0:14:58 Cause they, what they do, they take a picture and they analyze and say, I think I know what name means is referring to that.
0:15:01 And then therefore I know what I figure out what to do with it.
0:15:19 Those would be, you know, one tool accessing out the other resources like APIs is another tool and accessing databases, whether internal or external, it’s, it’s another tool to bring all that together in order to execute, to get to the goal that’s given to that agent.
0:15:26 One way to think about tools for me, at least is think of the things that generative AI doesn’t do a good job of, right?
0:15:32 It actually, despite most of the research being focused on math, it actually can’t guarantee you is that the math is correct.
0:15:37 So that’s where, uh, like in Able, for example, we do a lot of augmented analytics.
0:15:41 So we’ll go in and look at a dataset and look at a million variable combinations.
0:15:49 Think of every possible group by and drill down chart, but you’re just doing the math in a span of one or two minutes.
0:15:54 So your snowflake, your BigQuery, your data warehouse just charges you for one or two minutes.
0:15:59 And then you’re combining that information with the language model to present it to the user.
0:16:03 The math is not being done by the generative AI.
0:16:07 The math is being done by a deterministic system backed up.
0:16:10 So if an auditor shows up and says, how did you get to that number?
0:16:14 You can point precisely to deterministic stuff, but you present it.
0:16:19 So anything the AI isn’t good at, that would be a truly calling piece.
0:16:24 But one of the things Tony was talking about, how we shifted from chat to something autonomous.
0:16:29 To me, our motto from the beginning has been this idea of I am able.
0:16:35 And what we felt was it has to be, AI has to empower the human, right?
0:16:38 Now think about what happens when you are doing a chat interface.
0:16:45 You’re taking the entire power of AI and putting it behind the human ability to ask questions.
0:16:46 That makes no sense, right?
0:16:52 You take this incredibly powerful thing and you say, I’m only going to sip from it every time a human asks a question.
0:16:57 What agents start doing is like, can I do things continuously?
0:16:59 Can I look out for stuff?
0:17:01 Can I be an advocate for the human?
0:17:06 Can I give the human superpowers so that we can do humanly impossible things?
0:17:13 And I think that shift is beginning to happen where the first phase of a lot of the use cases was, how can I automate the human?
0:17:14 Wrong mindset.
0:17:17 Think, how can I give every human a superpower?
0:17:18 And then you can do crazy things.
0:17:21 You said humans can do crazy things.
0:17:25 Let’s focus on the good, crazy things because they are humans now.
0:17:28 But I think, you know, you’re absolutely right, Arjit.
0:17:47 If you look at a bit back a long time ago, like basically back to 2022, a long time ago in AI history, what happened was, or even a bit earlier than that, folks figure out that, hey, when they ingest more data and some of the models get bigger because they were relatively small, right?
0:17:52 I think in the days of ML, you know, as maybe hundreds of parameters, not more.
0:17:56 The more you scale, you know, the better the output is.
0:18:06 And frankly, when we started with transformers is basically how do you predict probabilistically and I didn’t mention this versus deterministically what the word next word should be.
0:18:07 Right.
0:18:13 All right, but so we ingested all this data and the thing is, a lot of the chatbots are also optimized.
0:18:21 They ingest this and they optimize to basically provide the simplest responses for efficiency reasons, right?
0:18:26 We’re going to maybe talk about it and reasoning because that’s important.
0:18:29 So that’s kind of where we were in terms of history.
0:18:48 You both talked about probabilistic versus deterministic and, Arjit, when you were talking about it and this is in the context of the agentic systems and sort of sounded like you were talking about using the deterministic, other deterministic systems to do what the AI is not so good at.
0:19:02 When we’re talking about agents and thinking about that one to five scale and, you know, goal, not necessarily going all the way to five, right, but a goal being to get the system to do more and more of the work to free the human up, to go faster for multitude of reasons.
0:19:05 How do you go about designing?
0:19:06 I’m trying to even wrap my head around that.
0:19:21 How do you go about designing and sort of knowing what needs to be built deterministically to, and I don’t know if this is the right way to say it, but to kind of nudge the probabilistic system, you know, onto the right path?
0:19:24 So think of it as math and words.
0:19:24 Okay.
0:19:40 So when I was in business school, what our teachers, there’s a person named Das Narendra, who used to teach this and say that what a student needs to do, and I’m going to misquote him pretty badly, is they have to understand the math and do the analysis, and they have to be able to tell the story.
0:19:47 JANIA is brilliant at telling the story and for assimilating information together and present it in a very coherent way.
0:19:49 What it’s not very good at is doing the math.
0:19:54 And this is something like people are kind of hand-waving it, saying, no, no, no, we can do it.
0:20:01 What you’re actually going to find is if you want to be absolutely sure that the math is correct, at some point, they’re doing tool calling.
0:20:03 They’re going into a deterministic system.
0:20:09 Now, what we did is we said, look, our underlying platform actually has all kinds of AI.
0:20:10 We don’t just do general.
0:20:17 You want to go in and understand what’s happening in your data, like what’s driving my revenue, what’s driving my patient, no-shows, what’s driving my customer churn.
0:20:18 Guess what?
0:20:21 Have a deterministic system, ask millions of questions.
0:20:22 Don’t have the generativity.
0:20:28 But then if I give you a bunch of dashboards and a bunch of charts, you won’t consume it that easy.
0:20:32 How can I take all that and say, hey, here are five things you need to know?
0:20:34 And now you ask a follow-up and give you something more.
0:20:39 Now, one of the important parts, you ask, how do you decide what becomes a tool, what becomes an agent?
0:20:50 But on an ongoing basis, though, one really important thing of the reasoning model that Tony just mentioned a few minutes back is you can almost build humility into the AI.
0:20:58 Like the AI, when it explains its reasoning, it says, I did this, I did this, I’m a third step, I calculated gross margin this way.
0:21:02 It actually allows the human to then say, hey, you know what?
0:21:04 Your definition of gross margin was wrong.
0:21:07 Here, hamster, the AI says, what did I get wrong?
0:21:10 And this person says, you’ve got the gross margin definition.
0:21:12 That is where the market is going.
0:21:15 Up to now, it was like you’re training an AI like a pet.
0:21:16 Good job, bad job.
0:21:19 But that’s not very high resolution feedback.
0:21:24 The AI couldn’t explain itself to the human and the human couldn’t explain the feedback to the AI.
0:21:30 When you make that reasoning model and the feedback on the reasoning model happen, you close that loop.
0:21:36 And actually, I’m going to put Tony on the spot, but Tony was one of the advisors when we were doing this with NVIDIA,
0:21:42 like this reasoning model and the feedback cycle of that, which we did on BGX Cloud with a bunch of customers’ help.
0:21:44 One of them was Tony.
0:21:52 And Tony, is there anything you want to add about the reasoning model and that feedback, that teaching as an intern, if you feel comfortable saying, Shane?
0:21:53 Yeah, no, absolutely.
0:22:00 I will say this, that, you know, I think we all like the humility in AI, or I hope that that’s going to continue.
0:22:03 But I think, you know, reasoning is a big thing.
0:22:10 So back to what I was saying earlier, a bit earlier about, yes, everybody ingested huge quantities of data.
0:22:21 But frankly, at the beginning, years ago, I mean, a lifetime ago, three years ago, it was a relatively simple, you know, regurgitating of things, right?
0:22:25 If you think about the way everybody was going into chat GPT and asking simple questions.
0:22:34 And we then started having these conversations about elucidations, which most people think it’s, they’re a bug, but they’re not a bug.
0:22:35 They’re actually a feature.
0:22:47 And we figured out that if we force the models to explain what they do and how they think through things, they get to better outcomes, better results.
0:22:54 And plus, it allows us as humans to identify, just as Arjeet said, where do they go wrong?
0:22:54 Yeah.
0:22:56 And it’s a difference, let me put it very simply.
0:23:02 It’s a, let’s say you have a child, you know, young child who’s learning math.
0:23:03 I don’t know, five, six years old.
0:23:04 Okay.
0:23:06 And you say, okay, how much is one plus two?
0:23:08 And they would say five.
0:23:09 Why five?
0:23:17 Well, it’s interesting because the brain, his brain probably likes the number five for whatever reason, or they heard, they heard the number five.
0:23:18 And so that’s how they wrote it in.
0:23:18 Sure.
0:23:30 However, now, if you say, here’s, let’s do this exercise, you know, I had your both hands and, you know, in a fist and then each hand, you raise one finger for the number you want to add.
0:23:31 Yeah.
0:23:33 One on one hand and two on the other.
0:23:38 Now count your number, your raised fingers, then you get the right answer.
0:23:44 But also now the child understands better how that worked.
0:23:54 Another thing is through reasoning over time, and maybe we’re going to talk about memory, they learn how to do it better themselves.
0:23:57 But my kids are a little older, but it’s funny.
0:24:04 I was, I’m always trying to think of a metaphor as I’m listening to, you know, guests talk to, I don’t know, to prove that I understand you.
0:24:06 Hopefully to explain for the listener.
0:24:08 But I was thinking about debugging coding at first.
0:24:14 But then when you got to the thing about the example with the fingers, I thought, oh, my kids are a little older now.
0:24:16 But showing your work, right?
0:24:19 In math class, that’s always like, well, why do we have to show our work?
0:24:21 And I think you just explained it really well.
0:24:28 Yeah, and if we never challenge or never go back to them, they will never understand the mistake.
0:24:32 Now, I would say one thing that I probably should have started with.
0:24:38 The number one is we need to start thinking about AI and AI agents, not as tools.
0:24:43 Now, AI agents use tools like the one we described, but they’re not tools themselves.
0:24:50 So let’s think about them as, obviously, as an alternative intelligence, just like humans.
0:24:56 Yes, they’re silicon, they have different architecture, they have different capabilities, obviously, they’re fast.
0:25:01 But sometimes their behaviors are surprisingly human.
0:25:03 Surprisingly human.
0:25:06 And, you know, Arjeet was talking about humility.
0:25:07 Guess what?
0:25:13 They’re very, they’re kind of a hallucinate with a plum, which is like, absolutely, I’m accurate.
0:25:20 So that’s one thing that sometimes is we have to apply human psychology to what they do and how they do it.
0:25:21 That’s number one.
0:25:28 Now, the number two would be, would sound completely contrary, is after all these things are still machines.
0:25:36 At least from an implementation perspective, we need to make sure that how we implement them is well engineered.
0:25:49 Yes, they’re intelligent machines, but at the end, things like scalability, things like monitoring, things like error detection, and so on and so forth, that we have learned over the last 30 years.
0:25:53 Resiliency, you know, hallucination detection or error detection.
0:25:55 Those are still very important.
0:26:02 But in truth, if we think of them as humans, then maybe we understand them a little bit better.
0:26:16 You know, I use this symbolistic is think about as an intern, you get a new agent, you know, you’ve got a new intern, they have some training, just like the models have some training, and they have some experience.
0:26:20 The interns, the interns, just like the models have the memory, long-term memory, if they have it, right?
0:26:23 So, but there’s still a lot to be done.
0:26:33 You can’t bring in that intern, even if they went to do the MBA, and say, okay, go and rampage the company, and you’ll figure out how to do the best thing.
0:26:34 You still don’t do that.
0:26:53 It almost sounds originate like Abel’s business, in a metaphorical sense, is building a school or a development, I mean, development literally, right, but an environment for a precocious, but sometimes mercurial or unpredictable intern to grow and thrive in.
0:26:56 And the important thing to think about is how they evolve, right?
0:27:05 So, what would happen is, let’s say I start with an image intern, and by the way, that’s what we call our initial models, before they get specialized, we just call them interns.
0:27:06 You call them interns, okay.
0:27:15 Because the idea is, like, I hate using the term specialized agent, and the darn thing doesn’t know my data, doesn’t know my terminology, doesn’t know my way of doing things.
0:27:17 That ain’t a specialized agent yet, right?
0:27:26 So, you might start out with just an image one, and then the user comes in and says, I’m going to do a lost baggage detection use case.
0:27:30 Another person says, I’m going to do a personal protective equipment use case.
0:27:33 Third person says, I’m going to do a stock art use case.
0:27:37 They’re just providing feedback, and it became different, right?
0:27:46 But then a farmer comes to a personal protective equipment one and gives very different feedback than a construction company does, right?
0:27:52 So, what happens is, these interns evolve into many, many specialized agents.
0:27:57 So, over time, you might end up with thousands of AI agents.
0:28:02 And, by the way, Jensen has been talking about this, that how companies will have many, many agents.
0:28:12 But this is the only way to build many, many agents, through this kind of human feedback and evolution, where you have an automated system change based on their feedback.
0:28:21 One tactical thing, by the way, what we found, which was really interesting, when you give feedback on the reasoning steps, you need a lot less feedback to make the model much better.
0:28:23 And conceptually, that should make sense.
0:28:27 If you took the kid who gave the math wrong and he said, you got it wrong.
0:28:28 Well, he doesn’t know how it got it wrong.
0:28:34 But when you walk then through it and say, on this step, you got that wrong, it is easier for you to generalize.
0:28:37 So, a lot less feedback gets them better.
0:28:41 Secondly, users don’t have an incentive to give feedback.
0:28:43 If you’re a business user, why am I giving feedback?
0:28:45 I’m not getting any payback from it, right?
0:28:48 Maybe three months from now, the model gets better.
0:28:53 In a reasoning model, when you give feedback, we immediately make the AI update itself with that feedback.
0:28:54 Redo that work.
0:28:59 So, now you get immediate payoff, which is, the AI got it wrong, I give it feedback, it got it right.
0:29:00 Just like you do with an intern.
0:29:02 And that’s what we have to figure out.
0:29:06 As Tony was saying, he’s talking about thinking of AI as a human.
0:29:14 I actually am thinking about, how do I think of the human incentives in this world where humans and AI have to work together?
0:29:17 Because if you don’t figure out the human incentive, it’s not going to matter.
0:29:23 So, it’s a fine balance here with reasoning and what we’re trying.
0:29:31 I don’t know necessarily that, at least in the foreseeable future, of course, AGI, whenever that comes, who knows.
0:29:41 So, we have to, I believe, to understand that these things, yes, they learn, but they still have a large amount of variability.
0:29:46 And the balance or the paradox is, we do want them.
0:29:55 You know, you want an intern or a new employee to have ideas, interesting ideas, but we don’t want them to be damaging.
0:30:05 So, that’s this thing called stochasticity for agents, which is the amount of randomness, not in just generating responses.
0:30:10 Because these agents, especially the reasoning agents, what they will do, they’ll create a plan.
0:30:11 You give them a goal.
0:30:14 Go and book a vacation for me.
0:30:15 That’s the goal.
0:30:17 They’re going to create a plan.
0:30:21 And that’s where the reasoning comes, because you’re forcing them to create a plan and then they execute.
0:30:24 Then they figure out what tools to use and so on and so forth.
0:30:34 You want to control for bad things, but also you want that agent to have the creativity to try different things.
0:30:36 Okay, that airline is booked.
0:30:41 Therefore, let me try a different airline versus coming immediately back out, they’re booked.
0:30:45 I’m like, okay, I need to give another goal, try the other airline and so on and so forth.
0:30:47 That’s the interesting thing.
0:30:50 And it’s not just at the model level.
0:31:03 Sure, with models, we can adjust the temperature of them all, which is how wild I would, you know, obviously from between zero and one, how wild they are in terms of responses, which is the predictive of the next token.
0:31:11 But it’s also is how do we design the workflows that they follow in a way that they don’t go off the rails?
0:31:12 Yeah.
0:31:12 Right.
0:31:20 Because you can, and this is back to, I think Orgin said something earlier, which is at some point humans are going to be the problem.
0:31:21 And you know why?
0:31:25 Because we’re going to give confusing or overlapping goals.
0:31:26 All right.
0:31:30 So let’s assume an agent has two ways to accomplish a task.
0:31:44 You say, I want you to be resilient and that the agent will figure out because it is learning over time that the first way is probably not as available in terms of doesn’t get the answer back.
0:31:47 This is a external API or whatever it is.
0:31:53 And then starts learning that if it goes to the other one, it gets responses right away and systematically.
0:32:00 And it will, because the goal was, you know, be, be resilient, it will go to the second resource.
0:32:01 But guess what?
0:32:03 The second resource may be a lot more expensive.
0:32:03 Sure.
0:32:06 But we humans didn’t tell it that.
0:32:06 Right.
0:32:10 So now you get the huge bill from wherever for that second resource.
0:32:16 So that’s, that’s kind of like, we need to think through these implementations very well.
0:32:23 Just as it would, if you brought in a human and say, go and do this, they’ll go for the most expensive thing because they want the job done.
0:32:24 Right.
0:32:34 Thinking about all that and thinking about, you know, the importance of thinking these things through carefully from the design stage, implementation stage, the feedback stage, all of it.
0:32:44 As, you know, you’ve both alluded to in different contexts during this conversation, things are moving incredibly quickly right now in the world of AI, in the world in general.
0:32:48 What are the implications for AI?
0:32:57 You know, we sometimes phrase this as, what do you see happening in the next, you know, one to three years, three to five years, whatever time frame you like, in your industry?
0:33:05 But we’ve been talking about AI and agentic AI and intelligence kind of writ large here as much as about, Tony, your work at CVS Health.
0:33:10 So I’ll kind of leave it open-ended in that way, kind of as we wrap up.
0:33:14 What are the implications you see, you know, given how quickly the world’s moving?
0:33:26 Well, I would say that given the investments everybody across the world is making in AI, I don’t think, I don’t see this slowing down.
0:33:26 No.
0:33:36 You know, you heard Sam Altman of an AI saying, maybe six months ago, saying, hey, we’re running out of the data to ingest where our models are not going to evolve.
0:33:37 Right.
0:33:53 And then we said, wait a minute, we’re, if we’re forcing these models to ration, to reason, they’re actually getting more intelligent, even if they don’t have extra data, because they probably have enough data somewhere that when we force them to put it all together, they are more intelligent.
0:33:58 There’s a lot of investment, obviously NVIDIA is helping, you know, power these, these things.
0:34:05 I think now that is, I don’t see another winter, an AI winter in the, in the past.
0:34:12 I think that’s now back to us to say, how do we use these things in the most logical way?
0:34:17 You know, we were talking about which, which, how do we decide what to do with these things?
0:34:19 We were talking about prototypes.
0:34:23 Well, you need to figure out what brings the most value, what is most cost efficient.
0:34:25 And what’s doable at any one time.
0:34:37 We need to, we’re, you know, we have intelligent things now, but we still need to put all that good thinking through that humans are good when they want to, to make the best of it.
0:34:37 Yeah.
0:34:38 Or is it?
0:34:45 So first, let’s take AI in general, because when market conditions are constantly changing, all predictive models fail.
0:34:45 Right.
0:34:49 The reason is predictive models are based on the future looking like the past.
0:34:57 So if your past and future are completely, and we saw this during COVID times, for example, another rapidly changing world, all your models actually fail.
0:35:01 Now, second thing is what happens to things like natural language querying?
0:35:09 They also fail because we know what questions to ask of our data when the data matches our past experience.
0:35:14 When things are constantly changing, how would we ever know to ask the right questions?
0:35:15 Right.
0:35:27 So you really need to flip the funnel, shift from a human initiating a question to a world where automated systems are looking out for you, acting as your advocates, doing millions of things for you.
0:35:37 If you’re not starting from that mindset, if your frame of reference is just, I’m going to ask a chat, you’re going to fall behind because you will never find the unknown unknowns.
0:35:38 Think of a metal detector.
0:35:40 That’s what an AI should be.
0:35:41 It should find the metal for you.
0:35:44 You shouldn’t be hunting manually through stuff, right?
0:35:55 The other part of this, the reason I’m very excited about the reasoning model and the feedback cycle, because as we found that the cost of that feedback, fine tuning was very low.
0:36:01 And there’s a really cool Nemo customizer and evaluator stuff that NVIDIA has come up with that we were launch partners for.
0:36:05 But essentially, how can we constantly do feedback?
0:36:09 Every hundred observations, every hundred feedback, we’re going and doing fine tuning.
0:36:15 So that model is, and by the way, then we test it, because just because you fine tuned it doesn’t mean the model is better.
0:36:16 Maybe there was bad feedback.
0:36:17 Maybe the model is worse.
0:36:19 You’ve got to constantly test.
0:36:29 You move to a world where your models are constantly changing, constantly specializing, constantly evolving, but under those kind of control stone I was talking about.
0:36:31 Don’t let them go all over the place.
0:36:32 It’s still being monitored.
0:36:34 Everything is still getting logged.
0:36:37 You’re still looking for anomalous behavior, all automatically.
0:36:39 That’s where this market is going.
0:36:42 Not one big model to rule it all.
0:36:50 But each user benefiting from hundreds and thousands of autonomous agents, assisting them, almost unseen.
0:36:52 That’s how we give people superpowers.
0:36:55 That’s how you end up with the I am able mindset.
0:37:06 We humans, in the last 300 years, in the Industrial Revolution, we evolved faster than ever before when we figure out to make components.
0:37:08 In this case, specialized agents.
0:37:13 And bring them all together and build up in multiple ways.
0:37:19 And in ways that, you know, obviously using the integrations, that’s where the power will be.
0:37:25 I don’t know if with that, you, it’s very even hard to predict because it’s not going to be linear.
0:37:39 The more intelligence is, by the way, it’s interesting, the more people have served in studies that the more intelligent agents you have in the mix, the better the entire result is.
0:37:49 It’s not a sum of intelligence, because it’s interesting, sometimes the agents themselves negotiate with each other and each other’s reasoning.
0:37:53 And that’s an emergent capability that nobody can predict.
0:37:58 So who knows what’s in the future in terms of emergent capabilities.
0:38:00 And interestingly enough, they don’t do it in English quite often.
0:38:04 They’ll just make up their own language with each other.
0:38:04 It’s kind of fun.
0:38:11 For listeners who want to know more about what Able has been doing, where online can they go?
0:38:17 Well, for us, if you just go to able.com, we have a bunch of blogs there, including a bunch of stuff we have done with NVIDIA.
0:38:18 Perfect.
0:38:19 A-I-B-L-E.
0:38:20 A-I-B-L-E.com.
0:38:21 Fantastic.
0:38:26 Tony Ambrosie, Arjit Sengupta, thank you so much for joining the podcast.
0:38:29 Fascinating conversation, to say the least.
0:38:40 Not just about what businesses can do right now for their employees, developers, for their customers with AI, but where all of this is headed and how quickly it’s evolving.
0:38:42 So thank you again for joining the podcast.
0:38:43 Thank you.
0:39:25 Thank you.
0:39:26 Thank you.
0:00:20 have known and worked with one another for years and can speak to the ins and outs of
0:00:25 enterprise technology, including developing AI applications, from the perspective of brands,
0:00:29 vendors, developers, customers, and probably most anyone else you can think of.
0:00:34 Tony Ambrosie is Senior Vice President and Chief Digital and Technology Officer of Pharmacy and
0:00:41 Consumer Wellness for CVS Health, and Orijit Sengupta is Founder and CEO of Able. Tony and
0:00:48 are here to talk about developing AI apps at massive scale, using DGX Cloud to do it, and the current and
0:00:53 future of generative AI, agentech AI frameworks, and so much more. So I’m going to get out of the way
0:00:58 and introduce these two gentlemen to share some tales and drop some knowledge on us in the next
0:01:02 half an hour. Tony, Orijit, thank you so much for joining the AI podcast.
0:01:03 Well, thank you for inviting us.
0:01:10 So if you would, maybe we can start a little bit with your backgrounds and kind of how you’ve
0:01:15 been working together forever. So I’ll let you kind of start at the point that seems apt to get us into
0:01:17 what CVS and Able are doing now.
0:01:25 So I’ll start. So I’ve been doing this for technology and digital and then at some point data, machine
0:01:31 learning for quite a while, trying to be at the forefront of technology to support customers and
0:01:39 employees and colleagues and wherever else there. I’ve spent my last five years in healthcare, three years
0:01:52 in a hospital system in South Florida, obviously focused on patients and patients’ well-being. I’ve been with CVS for almost two years now, the same type of focus.
0:02:13 And I think Arjit and I kind of bumped into each other 2016, 2017, I think. And then we saw that it would be cool to do interesting things with this machine learning thing because we can transform a lot of, make a lot of things better. At that time, I was with Disney and make a lot of things much better for consumers and guests.
0:02:15 So, Arjit, where were you when you guys met?
0:02:33 So I had, right before that, sold my previous AI company to Salesforce, where it became Salesforce Einstein Discovery. So I started something about Leoncore, which was one of the first augmented analytics companies on the planet, and ended up having a crisis of faith. I was like, did I create value or did I just make some money?
0:02:56 Right. So I ended up writing a book called AI is a waste of money, where I’d taken a thousand AI projects and I laid out why these projects fail. And one of the most common reasons was disconnect between the business user and data scientists. And this is where Tony and I always had this fun thing, which was like, it doesn’t matter what you build unless the business stakeholders adopt it.
0:03:19 Right. So that’s how our partnership actually started. In fact, the first time I was actually trying to sell to him, I came to him and said, Tony, keep me honest here. Something like, we’ll do this prototype in 30 days so that your business users can see. And he said something like, do it in five days or something. And then we did it in two, three days. Tony, am I remembering this ballpark correct? Like it was something absurd like that.
0:03:30 I think so. So some of the folks who are listening to this find that that can you do it in less than, you know, in a week and five days, you’re absolutely right.
0:03:49 It’s coming. Yeah, you and I talked a lot. And I think you guys are very, very good at that, which is when you experiment, you try things. That’s where the discussion really is. If you have cycles of okay, we talked for a week about what we want to do.
0:04:06 And then we go and do it for two weeks or even more. Then we come back for another week and evaluate that is not what we wanted or not what the business wanted or the outcomes are not met. And then, so you have the cycle that takes months, everybody will forget about what they wanted to do in the first place.
0:04:20 So being able to slice things and then do something in five days, which is, I mean, how many of us remember what we did on a Thursday, three months ago?
0:04:23 You know, we do remember why we talked about it on Monday.
0:04:25 I’m just trying to remember if today is Thursday.
0:04:26 Yes.
0:04:41 I’m going to take that as a chance to segue and getting under the hood here, so to speak, with you guys. How do you approach that now when, let’s say you’re starting an AI project or, you know, further along in the cycle as well when you’re kind of iterating?
0:04:51 How do you approach that now, working together on projects for CVS? And I would imagine, you know, at that scale and all the other considerations, but how do you get started?
0:05:00 So there’s two key parts to it. One is for any use case, we go after 48-hour rapid prototype, no longer than that, which means from when we first start.
0:05:06 And we already implemented inside CVS of data protection and all that is very crucial to CVS.
0:05:12 This is all happening within CVS system control, right? That’s why you can kind of get going that fast.
0:05:18 But Tony’s team blesses a project, within 48 hours, a prototype has to be done.
0:05:18 Okay.
0:05:26 And that has to go in front of business for feedback. It’s not just IT or analytics, just thinking about it. If business is not involved, it doesn’t matter.
0:05:38 Separately, his teams are building their own prototypes, right? So my team will do soundish prototypes when Tony blesses it, but the teams have been allowed to build whatever agents they want to build, right?
0:05:48 And then once business blesses it, then you got to build it within 30 days, meaning something real has to be in the hands of business at scale in 30 days.
0:05:49 In 30 days.
0:05:55 When you set those kinds of expectations, AI is exciting. So people will sign up for projects, but they never finish it all.
0:05:56 Sure.
0:06:02 You’ve got to have this kind of, and sometimes they want an AI, they don’t really know what they mean and they don’t really know what they want, right?
0:06:03 Right.
0:06:13 When you get a prototype in front of them, they’ll either like it or dislike it, and they can tell you what they disliked about it. Then you iterate and improve it until you get to an alignment.
0:06:18 Right. You give somebody something to react to. It’s so different from even a blank check, you know, and they’ll, well, what do I write?
0:06:29 So you mentioned the Tony’s team and your teams. And I think in the beginning, talking about your origin story, so to speak, with Tony, I might’ve glossed over a little bit.
0:06:37 Able is providing the development platform services. What is Able providing? Kind of how did the Tony’s team and your team fit together?
0:06:46 We have an agent developing platform, but it’s a no code, low code. In fact, most of the time, it’s like no code at all kind of a platform.
0:06:46 Okay.
0:06:57 So we have the software that runs on top of NVIDIA. So for example, we are using things like NIMS and Nemos and all that good stuff, but Tony’s team doesn’t have to worry about that, right?
0:07:09 So we leverage all of NVIDIA’s capabilities within that. And then when his team comes in and says, hey, I want to look at what’s happened. Actually, I can’t mention specific use cases, but let’s say I want to understand what’s going on with this data.
0:07:12 The software does it, presents it back for feedback.
0:07:24 Yeah. And I would add that’s, that’s accurate. So we do a lot of work. Some of it. Thank you for helping other things, you know, some of the bigger things. There’s a lot of work.
0:07:41 What I would say is, and this is my boss always emphasizes, we have a particular, given that the domain where, you know, patients, consumers, we are have a particular focus on what we call responsible AI and maybe others call it bigger. So that is the fact there.
0:07:54 Before you delve into that, apologies, but just, just in case I’m thinking I grew up with CVS, but I think of it as, you know, the, the pharmacy, the store. CVS Health is the second largest healthcare provider in the United States under its full umbrella.
0:07:58 Right. But you’re focusing on the stores in the pharmacy.
0:07:59 Correct. Correct.
0:08:13 And, you know, obviously in, within that, because we have all the retail customers, you know, my focus with technology, with digital product is how do we make that experience for customers better?
0:08:18 Right. And of course, constantly innovating on how to make it, you know, prove the safety.
0:08:19 Right. Right.
0:08:21 Right. So the responsible AI started.
0:08:35 That’s where these, exactly. And also, how do we help our colleagues with things that probably I can do in terms of automation, while also, also always keeping the human in the loop and in control.
0:08:50 Yeah. So obviously we do some things that are pretty, pretty non, you know, non health, healthcare, a bunch of other things, you know, my finance partners and my procurement partners say, oh, can you write the agent for me to whatever, you know, summarize all the invoices?
0:08:54 Perfect. Sure. Of course. But there are others that we’re extraordinarily careful.
0:09:00 Right. Are there any specific applications that CVS has implemented that you can talk about?
0:09:18 There are quite a few. I think some of them probably I will not go into, but others, what we’re doing is, and it’s part of the, in progress of being implemented is, and that’s where NVIDIA plays a big part, is what we call a conversation AI.
0:09:39 Right. The way I look at it from a digital perspective is we want to help customers with whatever they need. Now, of course, we’re not talking about, you know, dispensing medication and, you know, the, what the pharmacist does, putting things in a bowl and making sure everything is right, but everything else, we want to make sure that customers can have access to us in all sorts of channels.
0:09:42 Right. So maybe reordering or refilling the prescription.
0:10:00 Absolutely. You know, you know, if somebody needs a more of a pharmacology or medical advice, that’s where for pharmacists and doctors, but things like that, I call some procedural, yes, like when is my prescription ready?
0:10:12 Those would try to, if they choose to go into the voice channel, we have, you know, AI being able to provide that with precision. And that’s a big thing because it’s a huge channel.
0:10:24 So, Tony, you mentioned AI agents, building an agent. I think the off-the-cuff example was to summarize receipts. I’m going to put you on the spot here. Can you first talk about what an AI agent is?
0:10:43 It’s sort of one of those things that building the definition as we fly the plane, maybe that kind of thing. And then get into, you know, in your experience and Orgid as well, obviously, some of the key considerations that go into thinking about, you know, and then designing and developing, I guess, agents for different use cases.
0:11:03 Sure. So, it’s interesting. And we talked earlier about how fast things move. All right. So, some of us got some inkling, probably Orgid more than I, you know, in 2018, 2019, about kind of what’s happening with these large models, right?
0:11:16 And the transformers, Google’s transformers in 2017, kind of opened up the avenue. And then, obviously, we got the chat GPT in 2022 and everything coming from there.
0:11:24 But it was an interesting thing that because these chatbots, and obviously, they were the API, they were the API behind them.
0:11:43 They could tell things, tell us things. They could think somehow, somewhat, but they couldn’t do anything, right? You know, you have the womb, but the human going in and asking a question and then taking that in some shape or fashion and doing something in some workflow.
0:11:52 Right. And then some people said, well, wait a minute, we’re kind of like the plumbers for AI, you know, there’s AI is thinking and we’re, you know, we’re doing.
0:12:09 So, agents came in the, and clearly, this has been a topic in the, you know, sci-fi for 50 years. Agents came at the intersection, really, of intelligent machines through large language models and the, let’s say, process automation.
0:12:18 And, you know, we’ve done LLMs for, for a few years now, a couple of years at least, you know, we’ve done RPA for 10 years.
0:12:27 And then, this is where we kind of bring them together and say, this, these things called AI agents and agenting AI.
0:12:35 By the way, don’t you like how we geeks in technology come up with all these fancy terms and cloud and agenting AI, all this stuff?
0:12:41 You know, I’m, I’m a writer at heart. So, so that maybe that’s what actually drew me to technology, I just didn’t realize.
0:12:53 So that’s kind of, we married those where the intelligence behind the, the LLMs now drives through the use of tools, process automation and process execution.
0:12:59 Because we had that gap of execution gap where those things were, were disconnected.
0:13:00 Right.
0:13:08 Of course, there are, I, I’ve seen, uh, researchers, uh, talk about different level of maturity, uh, like everything else.
0:13:17 You know, I think you’re starting with zero basically as manual work, um, going to all the way to some size, which is completely autonomous.
0:13:22 where basically every, you know, AI is doing everything, the AI agents are doing everything.
0:13:24 There’s no human in the loop at five.
0:13:28 Yeah. I don’t know what kind of world that would be in, but let’s not go there.
0:13:29 Right.
0:13:42 But I think most of us are somewhere in the two, two, three, you know, some level of, uh, some level of intelligence coupled with some level of doing through tools and, and, and within workflows.
0:13:54 When you mentioned tools and we talk about the agents being able to go and take actions and do things, what are some of the examples, you know, either within CVS or, or perhaps Arjit, uh, that, you know, ABLE has implemented.
0:14:04 And you’ve been thinking about just for folks who are maybe understand this idea of agents more in an abstract level and having, you know, gone hands on, what are some of these actions that we’re talking about?
0:14:16 Sure. So the, the tools enable the LLM to actually take their insight or their decision to us to operate the task and actually do it.
0:14:25 And I think there are plenty already out in the market where an agent can basically do whatever a user in front of a computer do.
0:14:35 So they can control the screen, they can navigate a browser, they can fill in forms, um, figuring out what the page is about.
0:14:43 And that’s kind of a little bit between the difference between RPA, where we would have to code all that in and say, Hey, you know, look into the HTML, uh, source.
0:14:51 And then when, whenever you find the, the label name, you put the name and then so on and so forth, the agents through that intelligence.
0:14:58 Cause they, what they do, they take a picture and they analyze and say, I think I know what name means is referring to that.
0:15:01 And then therefore I know what I figure out what to do with it.
0:15:19 Those would be, you know, one tool accessing out the other resources like APIs is another tool and accessing databases, whether internal or external, it’s, it’s another tool to bring all that together in order to execute, to get to the goal that’s given to that agent.
0:15:26 One way to think about tools for me, at least is think of the things that generative AI doesn’t do a good job of, right?
0:15:32 It actually, despite most of the research being focused on math, it actually can’t guarantee you is that the math is correct.
0:15:37 So that’s where, uh, like in Able, for example, we do a lot of augmented analytics.
0:15:41 So we’ll go in and look at a dataset and look at a million variable combinations.
0:15:49 Think of every possible group by and drill down chart, but you’re just doing the math in a span of one or two minutes.
0:15:54 So your snowflake, your BigQuery, your data warehouse just charges you for one or two minutes.
0:15:59 And then you’re combining that information with the language model to present it to the user.
0:16:03 The math is not being done by the generative AI.
0:16:07 The math is being done by a deterministic system backed up.
0:16:10 So if an auditor shows up and says, how did you get to that number?
0:16:14 You can point precisely to deterministic stuff, but you present it.
0:16:19 So anything the AI isn’t good at, that would be a truly calling piece.
0:16:24 But one of the things Tony was talking about, how we shifted from chat to something autonomous.
0:16:29 To me, our motto from the beginning has been this idea of I am able.
0:16:35 And what we felt was it has to be, AI has to empower the human, right?
0:16:38 Now think about what happens when you are doing a chat interface.
0:16:45 You’re taking the entire power of AI and putting it behind the human ability to ask questions.
0:16:46 That makes no sense, right?
0:16:52 You take this incredibly powerful thing and you say, I’m only going to sip from it every time a human asks a question.
0:16:57 What agents start doing is like, can I do things continuously?
0:16:59 Can I look out for stuff?
0:17:01 Can I be an advocate for the human?
0:17:06 Can I give the human superpowers so that we can do humanly impossible things?
0:17:13 And I think that shift is beginning to happen where the first phase of a lot of the use cases was, how can I automate the human?
0:17:14 Wrong mindset.
0:17:17 Think, how can I give every human a superpower?
0:17:18 And then you can do crazy things.
0:17:21 You said humans can do crazy things.
0:17:25 Let’s focus on the good, crazy things because they are humans now.
0:17:28 But I think, you know, you’re absolutely right, Arjit.
0:17:47 If you look at a bit back a long time ago, like basically back to 2022, a long time ago in AI history, what happened was, or even a bit earlier than that, folks figure out that, hey, when they ingest more data and some of the models get bigger because they were relatively small, right?
0:17:52 I think in the days of ML, you know, as maybe hundreds of parameters, not more.
0:17:56 The more you scale, you know, the better the output is.
0:18:06 And frankly, when we started with transformers is basically how do you predict probabilistically and I didn’t mention this versus deterministically what the word next word should be.
0:18:07 Right.
0:18:13 All right, but so we ingested all this data and the thing is, a lot of the chatbots are also optimized.
0:18:21 They ingest this and they optimize to basically provide the simplest responses for efficiency reasons, right?
0:18:26 We’re going to maybe talk about it and reasoning because that’s important.
0:18:29 So that’s kind of where we were in terms of history.
0:18:48 You both talked about probabilistic versus deterministic and, Arjit, when you were talking about it and this is in the context of the agentic systems and sort of sounded like you were talking about using the deterministic, other deterministic systems to do what the AI is not so good at.
0:19:02 When we’re talking about agents and thinking about that one to five scale and, you know, goal, not necessarily going all the way to five, right, but a goal being to get the system to do more and more of the work to free the human up, to go faster for multitude of reasons.
0:19:05 How do you go about designing?
0:19:06 I’m trying to even wrap my head around that.
0:19:21 How do you go about designing and sort of knowing what needs to be built deterministically to, and I don’t know if this is the right way to say it, but to kind of nudge the probabilistic system, you know, onto the right path?
0:19:24 So think of it as math and words.
0:19:24 Okay.
0:19:40 So when I was in business school, what our teachers, there’s a person named Das Narendra, who used to teach this and say that what a student needs to do, and I’m going to misquote him pretty badly, is they have to understand the math and do the analysis, and they have to be able to tell the story.
0:19:47 JANIA is brilliant at telling the story and for assimilating information together and present it in a very coherent way.
0:19:49 What it’s not very good at is doing the math.
0:19:54 And this is something like people are kind of hand-waving it, saying, no, no, no, we can do it.
0:20:01 What you’re actually going to find is if you want to be absolutely sure that the math is correct, at some point, they’re doing tool calling.
0:20:03 They’re going into a deterministic system.
0:20:09 Now, what we did is we said, look, our underlying platform actually has all kinds of AI.
0:20:10 We don’t just do general.
0:20:17 You want to go in and understand what’s happening in your data, like what’s driving my revenue, what’s driving my patient, no-shows, what’s driving my customer churn.
0:20:18 Guess what?
0:20:21 Have a deterministic system, ask millions of questions.
0:20:22 Don’t have the generativity.
0:20:28 But then if I give you a bunch of dashboards and a bunch of charts, you won’t consume it that easy.
0:20:32 How can I take all that and say, hey, here are five things you need to know?
0:20:34 And now you ask a follow-up and give you something more.
0:20:39 Now, one of the important parts, you ask, how do you decide what becomes a tool, what becomes an agent?
0:20:50 But on an ongoing basis, though, one really important thing of the reasoning model that Tony just mentioned a few minutes back is you can almost build humility into the AI.
0:20:58 Like the AI, when it explains its reasoning, it says, I did this, I did this, I’m a third step, I calculated gross margin this way.
0:21:02 It actually allows the human to then say, hey, you know what?
0:21:04 Your definition of gross margin was wrong.
0:21:07 Here, hamster, the AI says, what did I get wrong?
0:21:10 And this person says, you’ve got the gross margin definition.
0:21:12 That is where the market is going.
0:21:15 Up to now, it was like you’re training an AI like a pet.
0:21:16 Good job, bad job.
0:21:19 But that’s not very high resolution feedback.
0:21:24 The AI couldn’t explain itself to the human and the human couldn’t explain the feedback to the AI.
0:21:30 When you make that reasoning model and the feedback on the reasoning model happen, you close that loop.
0:21:36 And actually, I’m going to put Tony on the spot, but Tony was one of the advisors when we were doing this with NVIDIA,
0:21:42 like this reasoning model and the feedback cycle of that, which we did on BGX Cloud with a bunch of customers’ help.
0:21:44 One of them was Tony.
0:21:52 And Tony, is there anything you want to add about the reasoning model and that feedback, that teaching as an intern, if you feel comfortable saying, Shane?
0:21:53 Yeah, no, absolutely.
0:22:00 I will say this, that, you know, I think we all like the humility in AI, or I hope that that’s going to continue.
0:22:03 But I think, you know, reasoning is a big thing.
0:22:10 So back to what I was saying earlier, a bit earlier about, yes, everybody ingested huge quantities of data.
0:22:21 But frankly, at the beginning, years ago, I mean, a lifetime ago, three years ago, it was a relatively simple, you know, regurgitating of things, right?
0:22:25 If you think about the way everybody was going into chat GPT and asking simple questions.
0:22:34 And we then started having these conversations about elucidations, which most people think it’s, they’re a bug, but they’re not a bug.
0:22:35 They’re actually a feature.
0:22:47 And we figured out that if we force the models to explain what they do and how they think through things, they get to better outcomes, better results.
0:22:54 And plus, it allows us as humans to identify, just as Arjeet said, where do they go wrong?
0:22:54 Yeah.
0:22:56 And it’s a difference, let me put it very simply.
0:23:02 It’s a, let’s say you have a child, you know, young child who’s learning math.
0:23:03 I don’t know, five, six years old.
0:23:04 Okay.
0:23:06 And you say, okay, how much is one plus two?
0:23:08 And they would say five.
0:23:09 Why five?
0:23:17 Well, it’s interesting because the brain, his brain probably likes the number five for whatever reason, or they heard, they heard the number five.
0:23:18 And so that’s how they wrote it in.
0:23:18 Sure.
0:23:30 However, now, if you say, here’s, let’s do this exercise, you know, I had your both hands and, you know, in a fist and then each hand, you raise one finger for the number you want to add.
0:23:31 Yeah.
0:23:33 One on one hand and two on the other.
0:23:38 Now count your number, your raised fingers, then you get the right answer.
0:23:44 But also now the child understands better how that worked.
0:23:54 Another thing is through reasoning over time, and maybe we’re going to talk about memory, they learn how to do it better themselves.
0:23:57 But my kids are a little older, but it’s funny.
0:24:04 I was, I’m always trying to think of a metaphor as I’m listening to, you know, guests talk to, I don’t know, to prove that I understand you.
0:24:06 Hopefully to explain for the listener.
0:24:08 But I was thinking about debugging coding at first.
0:24:14 But then when you got to the thing about the example with the fingers, I thought, oh, my kids are a little older now.
0:24:16 But showing your work, right?
0:24:19 In math class, that’s always like, well, why do we have to show our work?
0:24:21 And I think you just explained it really well.
0:24:28 Yeah, and if we never challenge or never go back to them, they will never understand the mistake.
0:24:32 Now, I would say one thing that I probably should have started with.
0:24:38 The number one is we need to start thinking about AI and AI agents, not as tools.
0:24:43 Now, AI agents use tools like the one we described, but they’re not tools themselves.
0:24:50 So let’s think about them as, obviously, as an alternative intelligence, just like humans.
0:24:56 Yes, they’re silicon, they have different architecture, they have different capabilities, obviously, they’re fast.
0:25:01 But sometimes their behaviors are surprisingly human.
0:25:03 Surprisingly human.
0:25:06 And, you know, Arjeet was talking about humility.
0:25:07 Guess what?
0:25:13 They’re very, they’re kind of a hallucinate with a plum, which is like, absolutely, I’m accurate.
0:25:20 So that’s one thing that sometimes is we have to apply human psychology to what they do and how they do it.
0:25:21 That’s number one.
0:25:28 Now, the number two would be, would sound completely contrary, is after all these things are still machines.
0:25:36 At least from an implementation perspective, we need to make sure that how we implement them is well engineered.
0:25:49 Yes, they’re intelligent machines, but at the end, things like scalability, things like monitoring, things like error detection, and so on and so forth, that we have learned over the last 30 years.
0:25:53 Resiliency, you know, hallucination detection or error detection.
0:25:55 Those are still very important.
0:26:02 But in truth, if we think of them as humans, then maybe we understand them a little bit better.
0:26:16 You know, I use this symbolistic is think about as an intern, you get a new agent, you know, you’ve got a new intern, they have some training, just like the models have some training, and they have some experience.
0:26:20 The interns, the interns, just like the models have the memory, long-term memory, if they have it, right?
0:26:23 So, but there’s still a lot to be done.
0:26:33 You can’t bring in that intern, even if they went to do the MBA, and say, okay, go and rampage the company, and you’ll figure out how to do the best thing.
0:26:34 You still don’t do that.
0:26:53 It almost sounds originate like Abel’s business, in a metaphorical sense, is building a school or a development, I mean, development literally, right, but an environment for a precocious, but sometimes mercurial or unpredictable intern to grow and thrive in.
0:26:56 And the important thing to think about is how they evolve, right?
0:27:05 So, what would happen is, let’s say I start with an image intern, and by the way, that’s what we call our initial models, before they get specialized, we just call them interns.
0:27:06 You call them interns, okay.
0:27:15 Because the idea is, like, I hate using the term specialized agent, and the darn thing doesn’t know my data, doesn’t know my terminology, doesn’t know my way of doing things.
0:27:17 That ain’t a specialized agent yet, right?
0:27:26 So, you might start out with just an image one, and then the user comes in and says, I’m going to do a lost baggage detection use case.
0:27:30 Another person says, I’m going to do a personal protective equipment use case.
0:27:33 Third person says, I’m going to do a stock art use case.
0:27:37 They’re just providing feedback, and it became different, right?
0:27:46 But then a farmer comes to a personal protective equipment one and gives very different feedback than a construction company does, right?
0:27:52 So, what happens is, these interns evolve into many, many specialized agents.
0:27:57 So, over time, you might end up with thousands of AI agents.
0:28:02 And, by the way, Jensen has been talking about this, that how companies will have many, many agents.
0:28:12 But this is the only way to build many, many agents, through this kind of human feedback and evolution, where you have an automated system change based on their feedback.
0:28:21 One tactical thing, by the way, what we found, which was really interesting, when you give feedback on the reasoning steps, you need a lot less feedback to make the model much better.
0:28:23 And conceptually, that should make sense.
0:28:27 If you took the kid who gave the math wrong and he said, you got it wrong.
0:28:28 Well, he doesn’t know how it got it wrong.
0:28:34 But when you walk then through it and say, on this step, you got that wrong, it is easier for you to generalize.
0:28:37 So, a lot less feedback gets them better.
0:28:41 Secondly, users don’t have an incentive to give feedback.
0:28:43 If you’re a business user, why am I giving feedback?
0:28:45 I’m not getting any payback from it, right?
0:28:48 Maybe three months from now, the model gets better.
0:28:53 In a reasoning model, when you give feedback, we immediately make the AI update itself with that feedback.
0:28:54 Redo that work.
0:28:59 So, now you get immediate payoff, which is, the AI got it wrong, I give it feedback, it got it right.
0:29:00 Just like you do with an intern.
0:29:02 And that’s what we have to figure out.
0:29:06 As Tony was saying, he’s talking about thinking of AI as a human.
0:29:14 I actually am thinking about, how do I think of the human incentives in this world where humans and AI have to work together?
0:29:17 Because if you don’t figure out the human incentive, it’s not going to matter.
0:29:23 So, it’s a fine balance here with reasoning and what we’re trying.
0:29:31 I don’t know necessarily that, at least in the foreseeable future, of course, AGI, whenever that comes, who knows.
0:29:41 So, we have to, I believe, to understand that these things, yes, they learn, but they still have a large amount of variability.
0:29:46 And the balance or the paradox is, we do want them.
0:29:55 You know, you want an intern or a new employee to have ideas, interesting ideas, but we don’t want them to be damaging.
0:30:05 So, that’s this thing called stochasticity for agents, which is the amount of randomness, not in just generating responses.
0:30:10 Because these agents, especially the reasoning agents, what they will do, they’ll create a plan.
0:30:11 You give them a goal.
0:30:14 Go and book a vacation for me.
0:30:15 That’s the goal.
0:30:17 They’re going to create a plan.
0:30:21 And that’s where the reasoning comes, because you’re forcing them to create a plan and then they execute.
0:30:24 Then they figure out what tools to use and so on and so forth.
0:30:34 You want to control for bad things, but also you want that agent to have the creativity to try different things.
0:30:36 Okay, that airline is booked.
0:30:41 Therefore, let me try a different airline versus coming immediately back out, they’re booked.
0:30:45 I’m like, okay, I need to give another goal, try the other airline and so on and so forth.
0:30:47 That’s the interesting thing.
0:30:50 And it’s not just at the model level.
0:31:03 Sure, with models, we can adjust the temperature of them all, which is how wild I would, you know, obviously from between zero and one, how wild they are in terms of responses, which is the predictive of the next token.
0:31:11 But it’s also is how do we design the workflows that they follow in a way that they don’t go off the rails?
0:31:12 Yeah.
0:31:12 Right.
0:31:20 Because you can, and this is back to, I think Orgin said something earlier, which is at some point humans are going to be the problem.
0:31:21 And you know why?
0:31:25 Because we’re going to give confusing or overlapping goals.
0:31:26 All right.
0:31:30 So let’s assume an agent has two ways to accomplish a task.
0:31:44 You say, I want you to be resilient and that the agent will figure out because it is learning over time that the first way is probably not as available in terms of doesn’t get the answer back.
0:31:47 This is a external API or whatever it is.
0:31:53 And then starts learning that if it goes to the other one, it gets responses right away and systematically.
0:32:00 And it will, because the goal was, you know, be, be resilient, it will go to the second resource.
0:32:01 But guess what?
0:32:03 The second resource may be a lot more expensive.
0:32:03 Sure.
0:32:06 But we humans didn’t tell it that.
0:32:06 Right.
0:32:10 So now you get the huge bill from wherever for that second resource.
0:32:16 So that’s, that’s kind of like, we need to think through these implementations very well.
0:32:23 Just as it would, if you brought in a human and say, go and do this, they’ll go for the most expensive thing because they want the job done.
0:32:24 Right.
0:32:34 Thinking about all that and thinking about, you know, the importance of thinking these things through carefully from the design stage, implementation stage, the feedback stage, all of it.
0:32:44 As, you know, you’ve both alluded to in different contexts during this conversation, things are moving incredibly quickly right now in the world of AI, in the world in general.
0:32:48 What are the implications for AI?
0:32:57 You know, we sometimes phrase this as, what do you see happening in the next, you know, one to three years, three to five years, whatever time frame you like, in your industry?
0:33:05 But we’ve been talking about AI and agentic AI and intelligence kind of writ large here as much as about, Tony, your work at CVS Health.
0:33:10 So I’ll kind of leave it open-ended in that way, kind of as we wrap up.
0:33:14 What are the implications you see, you know, given how quickly the world’s moving?
0:33:26 Well, I would say that given the investments everybody across the world is making in AI, I don’t think, I don’t see this slowing down.
0:33:26 No.
0:33:36 You know, you heard Sam Altman of an AI saying, maybe six months ago, saying, hey, we’re running out of the data to ingest where our models are not going to evolve.
0:33:37 Right.
0:33:53 And then we said, wait a minute, we’re, if we’re forcing these models to ration, to reason, they’re actually getting more intelligent, even if they don’t have extra data, because they probably have enough data somewhere that when we force them to put it all together, they are more intelligent.
0:33:58 There’s a lot of investment, obviously NVIDIA is helping, you know, power these, these things.
0:34:05 I think now that is, I don’t see another winter, an AI winter in the, in the past.
0:34:12 I think that’s now back to us to say, how do we use these things in the most logical way?
0:34:17 You know, we were talking about which, which, how do we decide what to do with these things?
0:34:19 We were talking about prototypes.
0:34:23 Well, you need to figure out what brings the most value, what is most cost efficient.
0:34:25 And what’s doable at any one time.
0:34:37 We need to, we’re, you know, we have intelligent things now, but we still need to put all that good thinking through that humans are good when they want to, to make the best of it.
0:34:37 Yeah.
0:34:38 Or is it?
0:34:45 So first, let’s take AI in general, because when market conditions are constantly changing, all predictive models fail.
0:34:45 Right.
0:34:49 The reason is predictive models are based on the future looking like the past.
0:34:57 So if your past and future are completely, and we saw this during COVID times, for example, another rapidly changing world, all your models actually fail.
0:35:01 Now, second thing is what happens to things like natural language querying?
0:35:09 They also fail because we know what questions to ask of our data when the data matches our past experience.
0:35:14 When things are constantly changing, how would we ever know to ask the right questions?
0:35:15 Right.
0:35:27 So you really need to flip the funnel, shift from a human initiating a question to a world where automated systems are looking out for you, acting as your advocates, doing millions of things for you.
0:35:37 If you’re not starting from that mindset, if your frame of reference is just, I’m going to ask a chat, you’re going to fall behind because you will never find the unknown unknowns.
0:35:38 Think of a metal detector.
0:35:40 That’s what an AI should be.
0:35:41 It should find the metal for you.
0:35:44 You shouldn’t be hunting manually through stuff, right?
0:35:55 The other part of this, the reason I’m very excited about the reasoning model and the feedback cycle, because as we found that the cost of that feedback, fine tuning was very low.
0:36:01 And there’s a really cool Nemo customizer and evaluator stuff that NVIDIA has come up with that we were launch partners for.
0:36:05 But essentially, how can we constantly do feedback?
0:36:09 Every hundred observations, every hundred feedback, we’re going and doing fine tuning.
0:36:15 So that model is, and by the way, then we test it, because just because you fine tuned it doesn’t mean the model is better.
0:36:16 Maybe there was bad feedback.
0:36:17 Maybe the model is worse.
0:36:19 You’ve got to constantly test.
0:36:29 You move to a world where your models are constantly changing, constantly specializing, constantly evolving, but under those kind of control stone I was talking about.
0:36:31 Don’t let them go all over the place.
0:36:32 It’s still being monitored.
0:36:34 Everything is still getting logged.
0:36:37 You’re still looking for anomalous behavior, all automatically.
0:36:39 That’s where this market is going.
0:36:42 Not one big model to rule it all.
0:36:50 But each user benefiting from hundreds and thousands of autonomous agents, assisting them, almost unseen.
0:36:52 That’s how we give people superpowers.
0:36:55 That’s how you end up with the I am able mindset.
0:37:06 We humans, in the last 300 years, in the Industrial Revolution, we evolved faster than ever before when we figure out to make components.
0:37:08 In this case, specialized agents.
0:37:13 And bring them all together and build up in multiple ways.
0:37:19 And in ways that, you know, obviously using the integrations, that’s where the power will be.
0:37:25 I don’t know if with that, you, it’s very even hard to predict because it’s not going to be linear.
0:37:39 The more intelligence is, by the way, it’s interesting, the more people have served in studies that the more intelligent agents you have in the mix, the better the entire result is.
0:37:49 It’s not a sum of intelligence, because it’s interesting, sometimes the agents themselves negotiate with each other and each other’s reasoning.
0:37:53 And that’s an emergent capability that nobody can predict.
0:37:58 So who knows what’s in the future in terms of emergent capabilities.
0:38:00 And interestingly enough, they don’t do it in English quite often.
0:38:04 They’ll just make up their own language with each other.
0:38:04 It’s kind of fun.
0:38:11 For listeners who want to know more about what Able has been doing, where online can they go?
0:38:17 Well, for us, if you just go to able.com, we have a bunch of blogs there, including a bunch of stuff we have done with NVIDIA.
0:38:18 Perfect.
0:38:19 A-I-B-L-E.
0:38:20 A-I-B-L-E.com.
0:38:21 Fantastic.
0:38:26 Tony Ambrosie, Arjit Sengupta, thank you so much for joining the podcast.
0:38:29 Fascinating conversation, to say the least.
0:38:40 Not just about what businesses can do right now for their employees, developers, for their customers with AI, but where all of this is headed and how quickly it’s evolving.
0:38:42 So thank you again for joining the podcast.
0:38:43 Thank you.
0:39:25 Thank you.
0:39:26 Thank you.
Tony Ambrozie from CVS Health and Arijit Sengupta from Aible share how their partnership is transforming enterprise AI development through rapid prototyping and human-centered design. Discover their proven methodology for moving from concept to production in just 30 days, why they treat AI agents like interns who need training and feedback, and how reasoning models are creating more reliable and trustworthy AI systems. They also explore the future of autonomous agents, the importance of responsible AI in healthcare, and why the next wave of AI will focus on empowering humans rather than replacing them.
Learn more at: ai-podcast.nvidia.com
