How Visa Is Making Payments Safer and Smarter with AI – Ep. 256

0:00:15 Hello, and welcome to the NVIDIA AI podcast. I’m your host, Noah Kravitz.
0:00:19 AI’s impact on the financial services industry has already been quite significant.
0:00:24 From improving customer satisfaction and loyalty to streamlining operations and reducing costs,
0:00:30 artificial intelligence is transforming an industry that’s already been at the forefront of technology
0:00:34 innovation for quite some time now. Visa is leading the way in leveraging AI to transform
0:00:40 payment experiences. And with us to talk about how they’re doing it is Sarah Laszlo. Sarah is
0:00:44 senior director of Visa’s machine learning platform, where she’s architecting the infrastructure that
0:00:50 will power Visa’s future. Before joining Visa, Sarah, well, her resume is really impressive and
0:00:54 too long to get into right now. But I have to mention, she was on NPR Science Friday,
0:00:59 which is pretty cool. Sarah, welcome. Thank you so much for taking the time to join the AI podcast.
0:01:01 Yes, thank you so much.
0:01:04 So before we get into all the stuff you’re doing at Visa, all the stuff Visa’s doing,
0:01:08 would you tell us a little bit about your own journey, maybe how you got started in AI
0:01:11 and how you wound up in the role you’re in now?
0:01:18 Yeah. So one thing that I always like to remind people of these days is that, of course,
0:01:22 everybody thinks of artificial intelligence as something maybe that computer scientists do,
0:01:28 but we need to remember that the back propagation paper that opened the field back up again after
0:01:33 sort of the AI winter was, yes, Jeff Hinton was on there, but it was also co-authored by three
0:01:38 psychologists, David Rummel Hart. And I came up through that tradition of psychology.
0:01:39 Oh, no kidding.
0:01:44 Yeah. So my PhD is in psychology and I became interested in computational neuroscience.
0:01:48 There were many steps along the way, dot, dot, dot, but now I still do it.
0:01:50 Very cool. Did you practice as a psychologist?
0:01:55 I was not ever a clinical psychologist or therapist. I was a cognitive neuroscientist.
0:02:01 So, you know, I worked with human research participants, but never in a therapeutic context.
0:02:08 Got you. And so was computational neuroscience kind of your pathway leading to where you are now?
0:02:15 Yeah. So my postdoctoral advisor, David Plout, was appointed in both psychology and computer science
0:02:22 at Carnegie Mellon. And my academic grandfather in that line is Jeff Hinton. So most of the people who
0:02:28 who were in that line and now do this, because that was sort of, there were not that many people in 2009
0:02:33 that were doing deep learning. And so if you are one of those, you’re one of the only, you know,
0:02:35 20 people that have 15 years of deep learning experience.
0:02:39 Totally, totally. Incredible. I kind of hinted at it and we don’t want, we want to talk easy,
0:02:42 we want to talk about what you’re doing. How did you wind up with this role?
0:02:50 Yeah. So I was working in responsible AI at Google and I did that for a couple of years.
0:02:55 After doing that for a couple of years, I decided that I wanted to get out of California.
0:03:00 You know, I was 40 years old and I was making a lot of money, but I didn’t own a home.
0:03:07 My family was in Texas. I really wanted to get to Texas, but I still wanted to be sort of doing
0:03:12 hard engineering problems, doing artificial intelligence. And Visa has actually a huge
0:03:16 presence of our AI ML team in Austin. So that’s how I ended up at Visa.
0:03:20 So let’s get into that then, into Visa. What are you doing with Gen AI?
0:03:24 What are some of the exciting things, big problems you’re solving? What’s going on behind the scenes?
0:03:30 Yeah. So when Visa generates new technology, like we’re always first trying to find ways that we
0:03:35 can prevent fraud. There’s so much attempted fraud that even though we’re very good at it,
0:03:40 even marginal improvements in that actually prevent like large dollar amounts of fraud.
0:03:44 So we’re always looking at, okay, there’s a new computational technology. Can we prevent fraud
0:03:49 better? That’s always number one. Number two is we’re starting to get into personalization
0:03:55 for our card holders. So I tell this story, I won’t name the card, but I have a card that has an app on
0:03:59 my phone and it, the offers that it gives me are ridiculous. Like it, it offered me a Lamborghini
0:04:04 one time. I was like, this is, this is, we can do better. We can do better than this. So we would
0:04:09 like for your app to be able to like minimally not get offered a Lamborghini probably for most people.
0:04:14 Are we talking free Lamborghini? No, I don’t, I don’t, I think it was probably like get a free
0:04:19 pair of driving gloves if you test a Lamborghini. So I was trying to sell me a Lamborghini, which was
0:04:22 ludicrous. So we want those to be better. We would like to be able to make better recommendations. And
0:04:27 then of course, like one of the big themes of NVIDIA GTC this year, everybody’s talking about
0:04:33 agentic AI. And we are very invested in the concept of agentic commerce where we can have agents that
0:04:38 have access to our credit card credentials that can make purchases for us. So the example I’ve been
0:04:45 giving all day is, Hey agent, I want to go to Greece in August, do it. And it just books it for me.
0:04:50 Is the travel example because everybody likes, lots of people like to travel or is it because of the
0:04:57 word agent and we just latched on to travel agent? I don’t know because I didn’t think that I,
0:05:01 I thought I had come up with this travel example myself, but then I’ve heard every other exact here
0:05:07 using the same one. I think it might be because travel is like, you know, if you’re going to take a
0:05:11 flight, there’s only a certain number of flights you can take. If you’re going to stay in a hotel
0:05:16 in a city, there’s like the space that the agent needs to navigate is pretty constrained and it can
0:05:21 know, like, do I have Marriott points or do I have Hilton points? It can know that. So it seems like
0:05:24 it’s something that’s tractable as a first problem. I think that’s part of it.
0:05:30 That makes sense. So how far along is that development? When, when can I, you know,
0:05:33 have an agent book me a trip to Austin and Alambo?
0:05:38 Right. So we’re, we’re still closer to the recommendation part, like recommending offers,
0:05:43 recommending products. Right now, what we’re working on is trying to figure out the right
0:05:49 sort of user embeddings so that the agent can have access to an abstract representation of you,
0:05:54 the cardholder, without having access to your private data. And so balancing that is where we
0:05:54 are right now.
0:06:00 Right. Got it. I know financial services is a heavily regulated industry. So there are lots
0:06:05 of challenges that go along with that. When thinking about and, and actually developing and
0:06:11 deploying Gen AI apps in this space, what are some of the other unique challenges that, you know,
0:06:15 folks working in other spaces might not know about when it comes to financial?
0:06:20 Yeah. So one thing I’ve been talking to folks a lot about at GTC is that because we are so heavily
0:06:27 regulated and because Visa in particular has such a strong commitment to privacy and security of our
0:06:33 cardholder data, we have been slow adopters of cloud. So a lot of other, you know, Visa-sized
0:06:38 entities that are trying to adopt Gen AI are very likely going to be getting their GPUs through cloud.
0:06:38 Sure.
0:06:43 There’s a lot of reasons for that, but we don’t do that. We own and operate our own data center
0:06:49 and that then puts a lot of constraints on the kind of soft and hard infrastructure that we can have
0:06:55 and therefore the kinds of models that we can have. And that is something that other enterprises
0:06:56 of our size probably don’t face.
0:07:01 Right. And so do you get around that by just building things that fit your needs?
0:07:06 Yeah. So actually a lot of what my team is responsible for is figuring out how to squeeze
0:07:08 what we need out of the GPUs that we have.
0:07:09 Yeah. Yeah.
0:07:12 That’s fun. That’s, I love that actually. That’s the funnest part of my job.
0:07:15 Excellent. You have any best practices to share around that?
0:07:21 Yeah. So one of the things that we have had a lot of success with is we’ve started using
0:07:29 virtual GPUs. So we are able to isolate memory and compute into separate virtual GPU instances
0:07:37 on a single card. And the reason this is helpful is that very often our users request more GPU than
0:07:42 they actually need. And so if we can actually, instead of giving them a whole GPU, give them a
0:07:48 10th of the GPU, it really improves our cluster utilization with very little impact to the user
0:07:48 experience.
0:07:54 Right. Is that, is virtual GPUs or are virtual GPUs kind of a standard thing or becoming kind
0:07:55 of common practice now? Or is that?
0:08:00 I think they’re getting, I hear a lot of other like enterprises of our size is using that.
0:08:00 Yeah. Okay.
0:08:04 Because everyone has this problem. You know, it doesn’t take a whole GPU to run a Jupyter
0:08:08 notebook, but people use whole GPUs to run Jupyter notebooks and it’s inefficient,
0:08:11 it’s cost inefficient and it’s wasteful. So I think it’s getting to be more standard.
0:08:16 Yeah. No, that makes sense. You mentioned personalization card holders. That’s kind of,
0:08:20 you know, the trade-off, right? When I think about it from the consumer standpoint and not
0:08:24 with Visa or financial service or with anything is, okay, if I give you my info, I give you
0:08:29 my data, like how easy are you going to make it for me? I would imagine you mentioned, you
0:08:34 know, that working on the abstract representation. So the agent doesn’t actually see my card number
0:08:39 or what have you, right? How are you thinking about and what’s Visa doing and down the road
0:08:43 to provide personalized experiences? What are some of the, I don’t know, some of the cool
0:08:44 things you’re working on?
0:08:48 Yeah. So one of the best things about Visa and one of the biggest advantages we have is
0:08:50 that we have an incredible data set.
0:08:51 I can only imagine.
0:08:56 You can only imagine. And I think actually maybe more so than people are with the data,
0:09:00 like consumers or users are with the data that they provide to other large entities.
0:09:05 I think people are pretty comfortable with Visa having their data. I think everybody sort
0:09:09 of uses their credit card and they know that like Visa has been around for a hundred years
0:09:14 and that we keep the data very safe and that we don’t use it for lots of extraneous purposes.
0:09:21 And I think people feel comfortable with that. So we have that data set and we are using it
0:09:28 to make large models that can have very good abstract representations of consumers.
0:09:33 The reason the representations can be good is because of the magnitude of the data that we
0:09:38 have. Nobody else. So I come from Google research. I worked on petascale computing at Google. And so
0:09:43 not often did I think I was going to say that I was going to work somewhere else and have access to a
0:09:47 data resource that no, but that, that like Google doesn’t have, but nobody else has, but we have that.
0:09:52 And so that means that the quality of the embeddings can be really high because the magnitude of the
0:09:57 data is high. The granularity of the data is high. The computational resources that we can bring to
0:10:01 bear to process the data is high. The data scientists that we have are some of the best in the world.
0:10:06 And when you put all that together, you can come up with a consumer representation that outmatches
0:10:07 what anybody else has.
0:10:08 Sure.
0:10:13 The challenges with personalization, I think are, we don’t ever want to expose the raw cardholder data.
0:10:17 So like the raw data is probably like, you know, any situation may be the most powerful because
0:10:22 nothing has been removed from it yet. It’s still exactly what it was. And so probably one of the big
0:10:27 research challenges right now is figuring out like how to re-represent the data in a way that’s
0:10:31 effective, but that obscures the private information of the user.
0:10:33 Is that as big of a challenge as it sounds like?
0:10:34 It’s very hard. Yes.
0:10:39 Because, you know, it’s, it’s exactly the trade-off. It’s like the more obscure the data,
0:10:43 the personal information is like maybe the less effective the representation is. And so this is an
0:10:47 exact trade-off and it’s hard to figure out how to, it’s hard to figure out what to do. We have a whole
0:10:51 research team working on that. Right. Since we’re audio only for the audience, Sarah just made a
0:10:55 beautiful graph with her arms. You couldn’t see, but it explained everything perfectly.
0:10:58 Yes. If you could have seen it, you would understand what I was talking about.
0:11:02 What are some of the other things that are either happening or, you know, down the road in terms of
0:11:09 personalization? Yeah. So recommendations for offers, also recommendations for product. So one of the
0:11:15 things again about our data set is we don’t just know like what ads you clicked on or like what
0:11:21 websites you looked at. And so that means that we can make better product recommendations probably
0:11:25 than anyone else can because nobody else knows that. Right. We were talking a second ago about
0:11:30 virtual GPUs. As you alluded to with the data center, right, you’re working at incredible scale.
0:11:35 What are some of the things that you’ve had to do, innovations you came up with, challenges you got
0:11:39 around when it comes to, you know, just scaling up and offering reliable service?
0:11:45 Yeah. So especially as we move into sort of generative AI era, the models that need to be
0:11:50 trained are larger. That’s one part of the problem. But actually, the bigger part of the problem for
0:11:56 Visa is dealing with the data hungriness of the new models. So we have, you know, trillions of
0:12:03 observations in the petabyte regime size of data. And we want to be able to bring as much of that to
0:12:09 bear as possible. But that’s obviously a very high memory intensive process. And so part of the
0:12:16 innovation that my team has had to be doing lately is finding ways to like process data efficiently over
0:12:22 distributed GPU resources, keeping GPU utilization nice and high and consistent, like trying to stream
0:12:27 data where we can, trying to lazy load data where we can, trying to use just in time techniques where we
0:12:32 can. So as a platform engineer, not only does my team have to figure out how to do that, but then also
0:12:37 help our data scientists do it too. And so there’s the challenge, there’s sort of twofold challenge.
0:12:41 One is how do we do it? And two, how do we get our folks to adopt it?
0:12:47 So there are any examples of like big internal wins, kind of a, you know, use case that you can
0:12:51 share with us talking about how generative AI has just really made a measurable impact?
0:12:56 Yes, absolutely. So, you know, Visa has been doing AI and big data work for 50 years,
0:13:03 which means there’s some code hanging around in Visa that’s like pretty old. And we recently
0:13:09 determined that there was a big code base that was still revenue generating, but that was no longer
0:13:13 maintainable because it was written in a language and I won’t say which one. It was written in a language
0:13:19 that was no longer supportable. And so we had a big problem because this was still revenue generating.
0:13:24 It was no longer maintainable and we wanted to convert it to Python, but because it was written
0:13:29 in such an old language, we didn’t have anyone in the company who could both know that language and
0:13:34 Python. And we were like, oh no, what are we going to do? And this problem was given to my team. And so
0:13:41 we ended up using GPT-4 to convert the old code base into Python. This saved us $5 million. And we had
0:13:47 one engineer do it. He converted, I think so far he’s done 50 jobs. He did like 50 jobs in a quarter,
0:13:51 which is crazy because he was, you know, he was averaging like converting like almost one of these
0:13:56 jobs a day at a sustained rate for like at production quality at a sustained rate for a whole quarter
0:14:00 because we were able to use these Gen.AI tools. And as I said, that saved us about $5 million.
0:14:01 Fantastic.
0:14:04 Yeah. One guy with a transformer model can do a lot.
0:14:11 I’m speaking with Sarah Laszlo. Sarah is senior director of Visa’s machine learning platform where
0:14:16 she’s working on infrastructure, architecture to power Visa’s future, which as we’ve been talking
0:14:22 about is big wide future that touches a lot of people every day. There’s this concept of the AI
0:14:26 factory and it’s picked up some steam recently. What does that mean to you?
0:14:35 Yeah. So what that means to me is a single pipeline that goes from a data scientist with an idea about
0:14:40 a model they want to build all the way to the model running in production. That’s what it means to me.
0:14:46 I like that. That’s a, is Visa in on this model? Is it a new name for something you’ve already been
0:14:53 doing? When, when, when you think about kind of where this is headed inside of Visa or broadly for
0:14:57 that matter, you know, what, what comes to mind? Yeah. So it’s interesting. Cause I, I hadn’t really
0:15:02 thought much about this AI factory terminology. Like I had, I had heard it, but I hadn’t really
0:15:06 thought it was what I was doing until I came here to GTC and I started hearing other people talking
0:15:12 about it. And then I realized, oh, that’s what, that’s what my platform does. So, so, so my platform
0:15:18 recently we’ve adopted what we call the Ray everywhere strategy. So we use AnyScale’s Ray ecosystem
0:15:24 to do the whole thing, the whole shebang. So data conditioning, model training and model
0:15:29 serving, we do all in Ray. And it is, and it is intentionally trying to be more of this factory
0:15:36 concept where it’s just, it’s, there’s not a whole bunch of distinct parts or distinct tools that are
0:15:42 living in different places that work differently. It’s just one unified, consistent pipeline from start
0:15:48 to finish. Right. Is the notion of standardization in an AI world, I’ve had some guests on the pod
0:15:53 before talking about that. And some of them have said like, this is a big thing we’re kind of grappling
0:15:59 with now. What’s your, your take on that? And are we headed towards more standardization or how,
0:16:02 how might this play out? You know, it’s an interesting question. Cause I’m sort of,
0:16:08 of two minds about it. I’m as of one mind about it as a platform engineer for a fortune 500 company.
0:16:14 I’m of a different mind on it as a individual who works just because I’m a nerd on AI projects
0:16:20 and my side, like I go home and I do this some more as a platform engineer. Yes. It’s a big deal.
0:16:25 Like, because like as in the visa case, a lot of the models that we already have, so not even looking
0:16:29 into future models, the models that we already have, that we already have, that we already work
0:16:35 with big customers for the faster we can refresh them, the better they are. Part of the reason for that
0:16:40 is that visa and fraudsters are in sort of a constant cat and mouse game. Like it’s, you know,
0:16:43 we do something, there’s a countermeasure. We have to have a countermeasure for the countermeasure
0:16:47 or a countermeasure for the countermeasure. And it goes on forever. And so the faster we can refresh
0:16:52 things, like the more fraud we can prevent, the better it is. Standardization helps with that.
0:16:57 On the other side of it, as a, as an individual practitioner, that’s been doing this for a long time.
0:17:04 Like I have been a person that has said many times, like in my courses that I call it machine
0:17:09 learning still, really. I’m old school. Machine learning is more of an art than a science. And so
0:17:13 there’s a part of me that’s the individual practitioner that likes the sort of artful
0:17:20 and surprising and creative endeavor. I find machine learning to be a creative endeavor and I will miss
0:17:24 that. I think it’ll make me sad if it becomes a fully factorized kind of process.
0:17:30 Right, right, right. Yeah. No, that, that, I like that. That’s the notion of continuing to do it when
0:17:35 you get home from work. I’ve heard a lot of that lately from the folks we talked to on this show.
0:17:38 So what’s the, what’s the adage? If you love it, it’s not a job.
0:17:43 It’s not a job. And it’s, and right now is such an exciting time. Like things that I dreamed of doing
0:17:46 years ago, I can do now easily. It’s like, I can’t get enough.
0:17:48 Like what? Is there a big one that comes to mind?
0:17:54 Yeah. So I worked in image generation when I was, when I was in Google and just the improvement
0:17:59 that has come in image generation from, so I worked on Imagine One, the difference between like the
0:18:05 quality of something like Imagine One or Dali One compared to like what MidJourney can create now is
0:18:09 just, it’s, it’s, it’s, it’s, it’s incredible. Blows my mind. I can’t get enough.
0:18:15 Awesome. Have best practices emerged when it comes to kind of the broader picture of,
0:18:21 I’m working in financial services. Somehow I’ve been tasked with thinking about Gen AI. We got to
0:18:25 do it. Everybody’s doing it. We need to get in, but like, what do we do? What do we do internally?
0:18:29 How do we think about our customers? All of that. You’ve touched on a lot of this, you know, here and
0:18:35 there as we’ve talked, but do you have kind of learnings that, you know, to share that someone
0:18:41 else? Yeah. Yes. People have been asking me about this all day and all week because, because we are,
0:18:46 we are sort of at the front of this and I hope people will be able to learn from our mistakes.
0:18:50 We had a guest recently starting to interrupt you. It came to mind who said that, what is it that like
0:18:56 trends only happen because some people figured out, like started doing it before it was a trend,
0:19:00 right? So you have to be there. You have to be there. And hopefully, hopefully we sort of have
0:19:05 broken the trail. And so people, it’ll be easier for the people that come behind us. One thing that I’ve
0:19:10 been saying to people is that don’t discount open source models, like open, like using open source
0:19:13 for everything that you possibly can, I feel like is a good practice.
0:19:18 I was going to ask, is Visa building models or are you customizing like open weight models or
0:19:24 all of the above? We build our own, we fine tune and rag open models and we use proprietary models,
0:19:28 all of these. One of the best practices is that we do use open source for everything that we possibly
0:19:32 can. And that’s not just true for AI. That’s true. You know, we use open source Kubernetes,
0:19:38 we use Red Hat Linux, like we use open source. That’s part of our, we use, we’re huge. Actually,
0:19:42 Visa is one of the, you wouldn’t, maybe your listeners didn’t know this. Visa is one of the
0:19:47 largest contributors to the Apache open source project. Yeah. Like we’re huge Apache contributors.
0:19:53 So open source as much as possible is a good practice. I think it is very important for governance
0:20:00 and tech to have a good relationship from the start. When that is not true, like things can go badly.
0:20:05 So I think like making sure that governance and tech are talking to each other all the time from the
0:20:09 very beginning, like as soon as your organization has an AI governance, an AI governance function,
0:20:13 they should be talking to the tech people. Don’t wait. Like start that, start that right away. Like
0:20:21 I think that, I think that’s an important practice. I do think that being very aware of like what models
0:20:28 can and cannot do and what is and is not safe for them to do is important for like you, there needs to be
0:20:34 some employee education about like, you know, don’t type your tax return into the AI. Like people,
0:20:38 people, they’re excited to play with the tools and they do all kinds of stuff, but there needs to be
0:20:42 some like, here’s how you do it safely. I think that’s, I think that’s important too.
0:20:47 Sarah, this has been a pleasure. And one of these episodes, I think where there’s just so much that
0:20:51 you’ve said that can be kind of taken and examined and looked at. So this one’s going to have a lot of
0:20:56 replayability as we say. So it’s fantastic. Thank you. Before we wrap for listeners who
0:21:00 want to know more, is the best place to go just to go to the Visa website?
0:21:07 Yeah. Visa website is good. Recently, our CTO, Rajat Taneja, has done some really illuminating
0:21:11 interviews with the Wall Street Journal where he talks about his vision for us in the future.
0:21:15 So just look up Rajat in the Wall Street Journal and you can, you can hear about his vision too.
0:21:19 Fantastic. Sarah Laszlo, it was a pleasure to speak with you. Thanks again for coming on the AI Podcast.
0:21:20 Thank you.
0:22:10 Thank you.