AI transcript
0:00:16 Hello. Welcome to the NVIDIA AI Podcast. I’m your host, Noah Kravitz. Before we begin,
0:00:21 a quick reminder, if you’re enjoying the podcast, take a second to follow us wherever you get your
0:00:26 podcasts. It helps us out and it helps you out by making sure you never miss an episode that
0:00:32 just show up in your feed. My guest today is Shanae Levin. Shanae is the co-founder and CEO
0:00:38 of Impromptu AI. They’re focused on helping non-technical folks create AI products,
0:00:43 which is, as Shanae and I were just saying before we hit record, it’s where a lot of focus is these
0:00:48 days. So this is going to be a great conversation. And let’s get into it. Shanae, thank you for joining
0:00:52 the podcast. So happy to have you. Thank you so much for having me. It’s been such an honor.
0:00:56 So let’s start with a little bit about your background, if you don’t mind, and then getting
0:01:02 into what inspired you to co-found and build Impromptu and tell us what it’s all about.
0:01:09 Absolutely. So background is I studied business and computer science. And then I spent some time
0:01:17 at Google working on developer tools for Google Home and Android, helping build Android applications
0:01:21 for millions of developers all over the world. And then I spent some time at eBay working on
0:01:28 both ads and more traditional machine learning, which was super fun. And then I got tired of
0:01:35 just working at big companies. So I was like, I’m going to try this startup thing. And I then went on
0:01:41 to Cloudflare. And then I was a senior director of product at Docker. And then I was head of product
0:01:47 a series C company, startup company, and then I decided to go out on my own. I built a company
0:01:53 called Code C, which was helping developers to master the understanding of their code bases,
0:01:57 which back in 2019 was revolutionary.
0:02:04 It’s like the BT before Transformer era or something. I don’t know if we can call it.
0:02:08 Exactly. I’ve been doing this a really long time, helping people to understand their code.
0:02:13 And we built something called Code C AI, which basically allowed you to chat with your code base.
0:02:20 So I was in the GPT-2 beta, like way early in the process. And so we were building this generative
0:02:27 system back then. And then Code C got acquired and I decided to take some time off and travel around
0:02:36 the world. But then I ended up really starting off building just a very simple app with Lovable.
0:02:42 Tried to do that. And of course, being the technical person that I am, I really broke it. I just full
0:02:49 on broke it. And then I put a pause on that. I ended up taking a role for a short amount of time at a
0:02:54 subsidiary of Fox Sports to help them with their AI. And right around the same time,
0:03:01 I ended up meeting Sean, Dr. Sean Robinson, who was my co-founder. And I ended up, he ended up saying,
0:03:08 I invented this thing to get up to 98% accurate outputs out of AI. And I’ve looked at him and I
0:03:14 said, what, wait, what? Like, you can get up to 98% accurate outputs out of AI.
0:03:21 Like, what is going on? And so I was like, I needed that in a number of places at Cozy. It’s really hard
0:03:26 to get code. Like, right. It’s really hard to get AI outputs to be accurate. And that is the next
0:03:34 frontier. And so we decided to really team up and build and prompt you together. He is a computational
0:03:39 physicist and a researcher. And we’re always inventing new things. And it’s been really one of the
0:03:43 highlights of my entire life to be able to build this company for people.
0:03:47 That’s incredible. I have 9 million questions about your background that we don’t have time
0:03:52 for right now. So I’m going to narrow it down to just one. When you took time off and traveled,
0:03:53 favorite spot?
0:03:54 Oh my God.
0:03:55 You don’t have to pick one.
0:03:57 Okay. Okay. I’m going to pick two.
0:04:03 Okay. So it’s really hard. But I’m going to say I loved Japan.
0:04:03 Yeah.
0:04:12 Japan is amazing. And my husband convinced us to spend three nights in the Sahara desert or like
0:04:20 traveling from Fez to Morocco. And then we spent a night in the Sahara. And I’m definitely old enough
0:04:25 to remember, you know, Windows 95 screensavers. It looks exactly like that.
0:04:29 Like in real life, like a screensaver comes to life.
0:04:35 So when you’re done founding and selling successful startups, you can write some travel memoirs and
0:04:37 link them back to the tech experience. It is great.
0:04:38 Exactly.
0:04:44 So with all of your technical background, what made you, I mean, I don’t know what made you,
0:04:49 it makes sense to me why you wanted to found something for less technical people to be able
0:04:53 to create products, kind of get a handle on things, but sort of was that, I don’t know,
0:04:57 when you met Sean and the two of you decided to do this, was that the initial idea or did you kind
0:05:00 of iterate on some stuff before landing on Impromptu?
0:05:03 Yeah, no, actually that was not the original idea.
0:05:03 Okay.
0:05:12 We had no intentions on starting a huge company. We actually were started off by using the tech
0:05:19 that he invented the art, which we call now our AI core, which is our optimization tech. And we started
0:05:26 just building AI apps for people. Like people needed our help to build these AI applications. And we were
0:05:31 like, hey, how do we help them do it faster and more accurately? And that was always the goal. Like,
0:05:36 how do we make sure that we get good outputs? I mean, cause back then, like, you know, you would
0:05:43 get, you know, it would hallucinate like crazy. I mean, it kind of still does, but like it was wild
0:05:49 the responses. And so one of the things that we need to do is really build trust with AI. And so no,
0:05:55 we were just manually going and helping org people in our network build AI applications. And then after,
0:06:01 you know, over time, we ended up building a bunch of these and we kind of took a step back. I actually
0:06:08 literally have this recorded the moment that this happened. We’re like, oh, the infrastructure to
0:06:15 build AI applications across these, across all of our customers is exactly the same. I wonder if we
0:06:22 can agentically write this AI application. And Sean looks at me and goes, wait, you want an AI that
0:06:29 builds AI? And I was like, well, is that possible? And he’s like, no, it’s not possible. And then he does
0:06:35 the thing that he always does where he says no. And then, and then, you know, 20 minutes later,
0:06:42 he goes, well, maybe, maybe if we did this and this and this and fast forward till today. And here we
0:06:49 are, we have an AI that is very specialized, specialized builder to build actual AI applications.
0:06:55 And so you’re working primarily, your customers are primarily or entirely enterprise size clients?
0:06:59 Yeah. So they’re, they’re all, they’re businesses, businesses of different sizes,
0:07:06 right? So they’re mostly large enterprises and midsize enterprises. Typically, if you have a SaaS
0:07:12 application already, right? You know, you have a platform, you have a service that you’re doing with
0:07:19 the website, or you have, you know, a running business, how do I transform that into an AI
0:07:24 native company, right? And that’s really where the gap is, right? Like, you don’t, you need to start
0:07:25 from this legacy code base.
0:07:29 I was going to say, that’s, that’s the trillion dollar question, except, I don’t know, I keep
0:07:33 thinking about the 98% accuracy, and that might be the $2 trillion question.
0:07:39 Well, here’s the thing, both of those have to work together, right? You, you can’t have,
0:07:43 you know, it doesn’t matter if you can get 98% accuracy, if you have to throw out your entire
0:07:50 code base, right? It, and it doesn’t matter if you, you know, have to, you know, you, you’re starting
0:07:55 from a greenfield project, and then your output kind of sucks, right? So you need a lot of these
0:08:01 pieces to work together. And we’ve invented, you know, five new pieces of tech from adaptive context
0:08:09 engines to infinite memory, to being able to, to our optimization core, to be able to ingest a code
0:08:14 base and say, hey, I’m going to import this whole code base from GitHub and now add AI to it, right?
0:08:17 Like, that wasn’t possible before we invented the things that we did.
0:08:21 Right. To kind of focus on the enterprise scale for a minute here, when you’re working,
0:08:27 you know, with a client of that size and complexity and teams of data scientists and, you know,
0:08:32 so many stakeholders. Yeah. And you start working with somebody who’s less technical,
0:08:40 but the idea is to help them build something kind of in that context. How do you approach it? And I
0:08:44 don’t know if the best way is to kind of ask you questions piece by piece, or if you want to walk
0:08:49 through kind of an example project, but when I’m messing around with AI and having it build something
0:08:54 for my own purposes, right? I mean, obviously the, the threshold for success is much lower. It
0:08:58 doesn’t matter if it’s not perfect just for me, but also the process, you know, I can just kind of put
0:09:03 my head down and mess around and see what I come up with. I’m at work, I’m in the team. It’s a very
0:09:09 different situation, right? So, so what is that like for Impromptu and the folks you’re working with?
0:09:18 Well, it’s multifold, right? Because the mission of our company is to make AI accurate and accessible,
0:09:26 right? So how do we make sure to get this technology into the hands of people who, it’s not that, you
0:09:30 know, they haven’t coded before because I talked to 20 year developer, like people have been developing
0:09:36 for 20 years and this is a new technology that we’re all learning at the same time, right? And so
0:09:41 all of that comes with education, all of that. It’s like, if I said, Hey, you’re going to go walk
0:09:47 into Sephora and now you need to go and find sunscreen amongst a thousand products. How do you do that?
0:09:52 You know what I mean? So like, it’s, it’s just, I do. I tried to get something from Sephora once for
0:10:00 somebody else. There’s a lot going on. Exactly. It’s the same thing. Right. So it’s, it’s, it’s not a
0:10:05 less technical person per se, right? But it’s someone who’s less, less versed in what’s happening
0:10:13 in AI, which is a race to keep up with. Exactly. And so we, and we, we definitely think about like,
0:10:21 Hey, everybody is learning generative for the first time. What are the pieces to educate anybody at any
0:10:28 technical level? And so we do that not only with our UI where the UI of the builder allows the actually
0:10:34 knows what all seven pieces are to build the full seven pieces to make an AI application, but it asks
0:10:39 you back and forth questions to make sure that you have supplied it with the right information and it
0:10:47 goes and has a full conversation with you. And we also have a co-build model, right? So we as experts
0:10:54 will also alongside of our AI tech also build in collaboration with the organization and educate
0:11:00 their team, work alongside their teams so that, you know, it, it doesn’t feel like we’re just going
0:11:08 to kind of leave you in lurch, right? And so, cause enterprises we need, like everything is changing so
0:11:14 fast, right? You need to have someone who’s keeping up with this while you’re running your business and
0:11:20 running your team and running, you know, all of the things internally. Someone needs to keep up with
0:11:21 this stuff. And that’s basically us. Yeah.
0:11:26 What is that, you know, where we, I use the word transformation all the time, you know, you just
0:11:33 mentioned it and it’s accurate, but what does it mean in this context of, you know, transforming an
0:11:38 industry, transforming a business and the work that you’re doing sort of, you know, on the ground level,
0:11:42 right? I mean, you’ve got that, that high perspective from your background and everything,
0:11:46 but you’re right there doing it. So, you know, folks like me, you’re looking at kind of broader
0:11:51 trends and thinking like, wow, you know, the ability to, AI has really changed and it was really
0:11:56 changing the way search works, right? It’s just a giant example, right? Kind of a big glossy thing
0:12:02 to say, but what are you, you know, what are you seeing? Are there, are there certain trends that are
0:12:08 emerging, certain types of problems that people are having more success or less success, you know,
0:12:16 Yeah. So I, so the larger trends as we’re kind of working on these things together is a huge thing
0:12:23 in physical AI, right? Like how do you move things in a physical space? We work with a ton of folks in
0:12:31 that capacity. We also are really thinking about context, right? And memory and how, as you think
0:12:38 about like, if code is going to be this, you know, arbitrary commodity, what then becomes important?
0:12:44 And then that’s the data, right? How do you then tackle this next frontier to create an end-to-end
0:12:51 solution? So now on, you know, last week we announced our custom data models in straight into our builder.
0:12:57 So now you have, not only you can build an application, but it’s built right off of the custom data
0:13:02 and it’s trained in the exact way that you would want it to be trained. And so, so you get really
0:13:08 highly accurate responses. And then that becomes really, you take it one level up. So you start
0:13:13 thinking about multi-tenancy. You start thinking about specialized models. You start thinking about
0:13:18 AI doing decision-making for you. And that’s where you really, where you get into this thing called
0:13:22 provable AI. Like what’s actually happening? What decisions is it making? How do I control?
0:13:28 And then thinking about the governance plane. Well, if I have 10 applications across my
0:13:33 organization, how do I make sure that it follows these rules? And we’re tackling all of those things
0:13:40 in one fell swoop to be able to make sure that if you are going to go down this journey, that it is not
0:13:46 only accessible to anyone on your team and it scales appropriately, but it also feels like you have full
0:13:54 control over it. Right. How big is the gap for a, you know, first time generative AI builder, to put it
0:14:01 that way, from, you know, something that I might build and say like, oh, this like seems to work right in my
0:14:07 use case, going from that to something that’s production ready. How big is that gap and how do you help
0:14:14 help people bridge it? Yeah. So great question. We actually think about it slightly differently where I can
0:14:20 get you a chat bot in probably 10 minutes. It has all the production ready pieces that you need.
0:14:26 Right. But what, what you actually are now, what we’re thinking about is enterprise scale. How do you
0:14:33 do that with terabytes of data? How do you do that in real time across millions of users? Right. How do
0:14:39 you do that, you know, where you have full governance and control? Right. So like today in our self-serve
0:14:45 platform, if you’re on like a big enough plan, I can get you very fast, very accurate system
0:14:53 in probably 10 minutes, but it’s, it’s, it’s the bridging the gap as you grow and have multiple
0:14:59 interconnected AI workflows and, you know, being able to make sure that they’re all accurate and all
0:15:05 running to the best of their ability. Right. You use the word accessibility a minute ago and,
0:15:11 you know, talking about AI being accessible to first-timers, less technical people. What does
0:15:16 that mean, you know, kind of day-to-day in your work? And maybe like, are there ways that people
0:15:23 use it or kind of misconceptions people have about, about that term? Yeah. So I think AI accessibility
0:15:33 accessibility at this stage is allowing anyone with an idea, not just to use AI to build old
0:15:40 technologies, right? We, it’s, it’s not just using generative to make HTML, CSS, JavaScript, like all
0:15:46 of those are great, right? We absolutely need those. But how do we use AI to actually build AI, to put
0:15:54 generative, thinking, living, breathing applications into people’s hands? And so it’s not only new for
0:15:59 developers, but it’s new for the business owners. It’s new when you’re thinking about companies being
0:16:07 disrupted. It’s new for the venture community. It’s new for everyone. And so how do we democratize and
0:16:15 actually have this new technology to be disseminated to anyone who wants it? Because now, you know, I can,
0:16:22 we have customers who, you know, are doing these amazing things and now can enable these
0:16:30 even more amazing things for their customers. So, you know, we have a mom and daughter team who is,
0:16:37 you know, building financial literacy, or we have a CPG brand who takes ocean plastic and turns that into
0:16:44 active wear, right? Like, so not just the super technical large enterprise companies, but like every
0:16:50 across the spectrum, the example of the mom and daughter team, like, I can wrap my head around
0:16:56 that easily because I’ve, I’ve, you know, typed in like, help me code a dashboard for podcast analytics,
0:17:03 right? And I’ve gone on that journey. How does AI help? And, you know, impromptu, your experience,
0:17:08 how did it help the process of recycling ocean waste to create active wear?
0:17:14 So really thinking about how do you bring all of these systems together, right? How do you make
0:17:20 recommendations to, you know, the founders, how to, and their, you know, their teams internally to make
0:17:27 sure that, you know, if you’re spinning up an event to connect, collect the, uh, the, the bottles, like,
0:17:34 how do I spin up a new app, a new city across the country to do the exact same thing? How do I
0:17:41 operationalize this process? Tell me how to do it, AI, right? Um, and so putting all of, but you, in order to do
0:17:47 that, that’s very custom data, right? That you’re not just going to get from a general model, right? You need
0:17:53 their system and their models, um, because that’s not a very typical use case. And so, um, those are the sorts of things
0:18:02 that now you enable, I use AI every single day, uh, in my, in my system to build this company. Um,
0:18:09 it’s unheard of going from five years ago, building a company then until now, like my day-to-day is
0:18:14 completely different. Yeah. Well, it’s funny as you were saying about when you and Sean, I want to call
0:18:20 him Dr. Sean. You should, you should call him Dr. Sean. When, when you and Dr. Sean, when you and your
0:18:24 co-founder, I’ve got together, you know, in the question of, wait, you want me to build AI that
0:18:30 builds AI? And like you said, fast forward. And now, you know, I mean, I heard Jensen say it,
0:18:37 but lots of people are saying it, that AI is going to mostly generate tokens to be used by agents,
0:18:42 other AI systems, like, right? Like the majority of the tokens as we go forward are not going to be
0:18:48 creating output that humans sees, humans see, excuse me, but that, you know, another AI is going to act
0:18:55 with. Yeah. And it’s amazing. I’m sorry. Yeah, no, it’s, it’s truly, it’s truly amazing. You know,
0:19:02 we’re working with some customers who, and some, you know, people’s dreams who, you know, would have
0:19:08 never had the ability to, to do this. But then also you’re thinking about like, oh, actually I don’t
0:19:14 like, eventually we’re not going to need a full human into this at all. Right. And so we can actually
0:19:20 AI generate human in the loop systems, but we can also just, you know, generate a full end-to-end API
0:19:27 that, you know, goes out from the front end, does a whole system, does an AI generate and puts it back
0:19:32 as an API back into the customer system. And like, no one ever knows that we’re doing anything,
0:19:38 right. Cause it’s fully containerized and, and all of those things. So we’re already seeing
0:19:46 that day to day. My guest is Shania Levin. Shania is CEO and co-founder of Impromptu with an E,
0:19:52 E-M-P-R-O-M-P-T-U dot AI. I want to get that out there if you’re listening now and want to look it up
0:19:59 later. And we’re talking about Impromptu’s work, helping people of all technical abilities,
0:20:05 but certainly people who are less used to building with generative AI, build products and
0:20:10 at enterprise scale and production ready and, and the whole thing. Um, and so I want to ask you kind
0:20:17 of a more technical question here. Impromptu uses NVIDIA CUDA. Yeah. Can you talk about how, uh,
0:20:23 the CUDA libraries for embedding and classification in particular, improve your performance and make AI
0:20:28 efficient, all that good stuff, but how it actually translates to the work that you’re doing.
0:20:32 And then in particular, the work that your customers are doing and creating with these
0:20:40 tools. Yeah. So we consider Impromptu to be a mixed code builder instead of no code or pro code. It’s
0:20:44 kind of, we call it a mixed code under the hood though. You know, parts of it are essentially,
0:20:50 you know, a big embedding and classification machine, right? So every time, you know, we map a
0:20:56 new feature or iterator design, or, you know, you search your custom data model, you know, we’re turning
0:21:03 that into vectors and making bigger decisions on top of that. Now we have our own kind of custom models
0:21:09 and our own secret toss alongside of that, but we use NVIDIA’s code, uh, libraries so that all of the,
0:21:15 you know, the heavy math natively runs on GPUs instead of, you know, slowly on CPUs. And that really gives us
0:21:23 two big wins, right? First it’s performance. So like all of the everyday creators in our UI get that instant
0:21:28 feedback. They can, you know, request a feature or upload or tweak a prompt or, and the AI just kind
0:21:34 of responds. And so they can like iterate much faster, ship more, much more useful products.
0:21:41 Um, and then the second thing is efficiency, right? So, you know, CUDA lets us serve a ton of those
0:21:46 workloads in like a relatively small GPU footprint, uh, whether that’s on our, you know, in our cloud or
0:21:51 our customers VPC. Right. And how does CUDA help? I mean, you talked about this a little bit to,
0:21:56 to put a point on it. How does CUDA help kind of bridge the gap? And I’m wondering from your
0:22:02 perspective, how big or small that gap really is these days between kind of the, the cutting edge
0:22:07 AI research that’s being done and then the tools that, you know, your customers are actually able
0:22:13 to use to build things. Yeah. I think at this point, you know, the gap is mostly filled.
0:22:22 If I do say so myself, um, you know, you don’t write today, you know, need to, you know, a big
0:22:27 machine learning team to build something sophisticated. Um, you know, CUDA kind of helps us with that hard
0:22:32 math. And so our platform turns all of that into, you know, this fast, affordable, you know, just,
0:22:40 just works experience for, you know, non-technical and technical teams because like technical versus
0:22:46 non-technical, those now are, they go together now, right? It’s generative or not generative.
0:22:54 Right. And so as we’re thinking about that, like all of those things are now been able to do it in a,
0:23:00 you know, a self-service if you really want to do that, or if you want the pros to actually build it
0:23:05 for you, you can just throw that out, right? And we can actually just use our own systems to build
0:23:10 it on your behalf and do the integration for you. But all of the things under the hood are basically
0:23:15 all like taken care of. All right. So this is the point where I come back to this 98%
0:23:20 thing that, you know, I was like, I don’t want to ask up front because probably it’s secret sauce and
0:23:24 we’re not going to get too into it, but can you get into to any of the details of how it works?
0:23:31 Yeah. So it does work a little bit differently than most other systems. And what we found is that
0:23:35 a lot of people talk about benchmarks, model benchmarks, right?
0:23:35 Sure.
0:23:41 Model benchmarks are great. Everybody has a model benchmark. Excellent. But how do you translate
0:23:48 that model benchmark to the everyday enterprise? Who’s like, I have a goal. I have things to do.
0:23:57 Right. And so our accuracy is actually redefining that success, which is task success. And because
0:24:04 we work with so many use cases and so many customers and enterprises, we allow the user to define what
0:24:11 success means to them, what that task is, what success looks like. And we will use our full optimization
0:24:17 engine that optimizes the entire system, including the model, the data, the prompts, the evals, all of
0:24:23 the under secret sauce under the hood to optimize in real time towards that goal that the user sets.
0:24:30 So, and we measure it directly in our dashboard. So you’ll see a big number of how we improve that
0:24:35 accuracy towards the goal that the user sets. The other thing that might not be so obvious
0:24:43 is we have two modes. One is for, it’s called manual optimization, which are all for all of those
0:24:49 technical, deep technical developers who want to tinker and turn knobs just like myself, right? I want
0:24:55 to do that. But you can actually see the output towards that accuracy and it will actually give you
0:25:00 suggestions. You can click the button however many times you want to click it and, you know, improve it
0:25:06 however many times you want. Um, after about 30 runs though, we can turn on something called automatic
0:25:11 optimization. So if you’re less technical and you’re like, I don’t know what the heck that is,
0:25:17 that sounds smart. It is, but you don’t want to tinker with it yourself. We’ll just do the automatic
0:25:25 optimization for you up to, you know, 98% of that task accuracy. And we’re always improving it. It’s
0:25:30 always, you know, we’re always trying to get to, you know, three nines, which I definitely presume
0:25:35 that we will one day. And so that, you know, that speaks to, and you, and you brought this up before,
0:25:39 and obviously it’s a big, a big thing right now in the, in the industry and the marketplace
0:25:46 is building trust. Definitely. Are your, you know, your customers, like, are they looking at,
0:25:53 like, does that dashboard, that figure of optimization, does that do a lot to improve the trust? Is it really
0:25:59 just seeing the results and being able to say like, yes, it helped me get this project, you know,
0:26:05 off the ground and running and it’s working well? How do you build trust, you know, right now when
0:26:11 there’s kind of this tension between the race to build out all the infrastructure and the data centers and
0:26:16 everybody wants chips and can’t get them? And then, you know, everything that, that we’ve been talking
0:26:23 about. Yeah. So we subscribe to something called provable AI and we think that, um, that is the
0:26:31 absolute way to build trust, right. And, and make it so that it is accessible, right. If you can’t see it,
0:26:36 feel it, touch it in a theoretical sense, cause it’s all bites and it’s all numbers, ones and zeros,
0:26:41 right. Um, you know, until we get to physical AI, like you mentioned, and then, you know, right,
0:26:49 exactly. So, uh, thinking about that, you know, seeing, okay, what decisions is the AI making?
0:26:56 How do I roll back? Right. How do I, you know, if I have infinite memory, you know, how, like what,
0:27:03 you know, is involved in this decision, right? Like if I’m doing a custom data model, you know, how,
0:27:09 what data is kind of going in, being able to showcase, like here are all the runs that we took,
0:27:15 right. Um, showing those accuracy numbers in the dashboard, right. We always try to showcase,
0:27:22 you know, provable, Hey, this is the number. This is the suggestion. This is the, you know, uh, the,
0:27:28 the set that you need to be able to do now. Can you disconnect those things for privacy concerns?
0:27:36 Absolutely. But proving to the user that this is, we’re doing what we say we’re doing is of the
0:27:45 utmost importance. Yeah. Yeah. To switch gears a little bit as a woman in tech, right. Which like
0:27:50 everything else, particularly everything else in tech has changed a lot very quickly in certain ways.
0:27:56 And, and, you know, not to speak for you, but probably not so much in other ways, but then,
0:28:04 you know, working now with a lot of people who are, you know, the whole idea of making AI accessible,
0:28:11 right. How does your experience as you, as a woman in tech, you know, co-founding and running this
0:28:17 company now and working with folks, how does that inform kind of how you see, you know, barriers to
0:28:22 entry, whether it’s, Oh, I’m new to generative and I want to start working with generative or, you know,
0:28:29 Oh, I’ve never done something at production scale or, Oh, like I’m a woman and I’ve experienced what I’ve
0:28:36 experienced. And so I’m interested in this stuff, but I don’t know. Like, yeah, it definitely really,
0:28:43 you know, hits my heart. I can tell you a very quick story. So please, when I, uh, I said,
0:28:48 obviously studied business computer science, but I had a small web development company, um, that I
0:28:54 started when I was 19. And then my first big job, like big, big job after I was in an analyst was
0:29:04 working at Google and I was like, uh, so I had two choices, you know, uh, go to search Google search
0:29:11 at the time and then go to this thing called developer tools. And at the time I was like,
0:29:18 I just seen the internship, which I know kind of, but it felt like this really overwhelming technical,
0:29:23 like I’m going to be working with some of the biggest, smartest developers that, you know,
0:29:28 ever, it’s very intimidating. And then I ended up choosing to go to developer tools. Cause I thought
0:29:37 that would be easier. Um, but I’ve been obviously changed the course of my entire life, but, um, that
0:29:44 one decision, and I don’t want anyone else to make a technical decision like that out of fear.
0:29:51 Right. I don’t want women or kids that I talked to. I have 12 year olds emailing me,
0:29:59 right. Thinking about how to build generative systems. And I never want that to feel like
0:30:06 you have an idea and you can execute it. Right. And so we take great care with whatever technical
0:30:12 ability you have to take advantage of this technology. Cause I constantly think about,
0:30:18 you know, going into these rooms, right. With, with people. And I think luckily now I’ve,
0:30:25 I’ve built things for millions of people, but at the same time, I know deeply what that feels like
0:30:31 to be told that you’re not technical enough to be told or to have someone, you know, talk to Sean and not
0:30:38 talk to me. Right. And, you know, those sorts of things. So I think that it is, we are at a place in our
0:30:48 our evolution, um, that now that barrier is fully down, right. It’s fully, it’s fully unblocked.
0:30:54 Everybody is running. We are, you know, I’m talking to, you know, people who have gotten laid off from
0:31:02 jobs. You know, how do I take this knowledge that I’ve gained over the last, you know, 10 years and
0:31:09 start monetizing it into a real business and are making real money off of this using generative.
0:31:15 And then also completely on the other end, right. At these large enterprises where they’re like, Hey,
0:31:22 I know we use AI internally, but I still don’t know really how to use it. Right. Like, how do I set myself
0:31:27 up for success? How do I put this in front of users to help businesses reach their goals? I mean,
0:31:32 it’s all over the place. Right. So thinking about all of those different stakeholders is really
0:31:36 important to us. All right. So keep thinking about all those stakeholders, right? Yeah. The notion
0:31:45 that studying computer science, the 12 year olds, right. Studying computer science may not necessarily
0:31:51 be the route to take now. If you want to go into a, we’ll just call it a computer career, so to speak.
0:31:57 Right. If it’s not, if it is, tell me, but if it’s not, if coding skills aren’t necessarily like
0:32:05 the top thing looking forward that you need to be successful with AI, with technology, with generative,
0:32:11 everything, what are some of the skills, some of the mindsets, some of the knowledge that you would
0:32:17 advise the 12 year old or, you know, the, the 50 year old who got laid off and wants to monitor,
0:32:21 everybody wants to do something new. Yeah. What would you tell them to get into and how
0:32:28 would you guide them? So as the eldest of six children, I happen to have my youngest sister
0:32:36 as an intern at my company. And so I think about this all the time, you know, and I, I think that
0:32:44 for kids coming out of the, out of school today, um, I don’t, I, you know, I know that the headlines
0:32:49 are thinking about, you know, compute, like computer science is out and you can’t get jobs as junior
0:32:56 developers. But when I studied computer science, it was not just about coding, right? It wasn’t as
0:33:02 computer science is not, it’s about con is about cognition. It’s about thinking in systems. It’s about
0:33:08 critical thinking skills. It’s about, you know, how do you break problems down into small shippable
0:33:15 chunks? It’s about, you know, building on top of and merging that creative energy, that creative ability
0:33:23 with, you know, technology that’s always changing, right? Like, you know, I was in school and did
0:33:29 everything on a Java application and that allowed me to go into the Android system. But then Google decided
0:33:37 that we’re going to use Kotlin now, you know what I mean? So like, like, I took, I took intro to CS and,
0:33:42 in, uh, and it was taught in scheme. So yeah, you know, things change. Yeah. Right. Exactly. I mean,
0:33:47 the first class was like, we did assembly coach, you know what I mean? So like, I think that people are
0:33:54 underestimating the, the quality of a good computer science education, because I think that the critical
0:34:02 critical skills around how do I think about these larger problems that are going on in the world and
0:34:08 the world and how do I solve them needs critical thinking skills. And that needs to be taught and
0:34:14 practiced. Right. Right. Yeah. Even, even, even with the, you know, the metaphor, and this may have been
0:34:20 from another episode recently, um, the, the metaphor of like conducting the orchestra, right. You know,
0:34:26 you may not be writing all the code anymore, but you’re conducting the orchestra, humans, bots, a mix.
0:34:30 Absolutely. And that’s exactly what you’re talking about are those big picture thinking skills. Yeah.
0:34:36 Like even, even as a, you know, a PM for a very long time in my career, uh, all, you know, always on
0:34:41 very technical products, but thinking about system design, right. You’re going to need to read and
0:34:46 understand all of the code of the AI rights. And that is a skill in and of itself outside of writing
0:34:51 code. Right. So it, you know, it’s just like, if I decided to go read Japanese tomorrow,
0:34:57 right. Like I, I still need to practice that just the same way I need to learn to practice,
0:35:03 to read code and to keep that skill up to speed. And so from people coming out of college, I think
0:35:09 computer science education is not necessarily dead despite the headlines. I think that the types of
0:35:14 jobs and the types of skills that should be taught in computer science education just need to be, uh,
0:35:17 some need to be de-emphasized and some need to be re-emphasized.
0:35:21 That kind of leads us to, you know, sort of our last forward looking question here.
0:35:25 And, and given the conversation, I kind of want to ask you from the two points of view,
0:35:30 the individual or small team, and then the enterprise scale or, you know, the organization,
0:35:35 right. And of whatever size, but certainly the enterprise scale, where do you see the biggest
0:35:42 opportunities for AI to really make a difference and to empower folks and to, you know, lead to,
0:35:46 I mean, whatever the successes are, given the context, like, where do you see the biggest
0:35:52 opportunities, you know, in the next, whatever, I, two years is almost too long, right. But whatever
0:35:54 the next window in your head is.
0:36:04 Yeah. So I think the real opportunities are in fully remaking our entire world, honestly. Like,
0:36:12 I think that’s, I genuinely, I genuinely believe that, like, there are problems that if people.
0:36:15 Can you open source the plan when you and Dr. Sean saw on it?
0:36:20 Absolutely. Absolutely. I’ll tell you a small joke.
0:36:21 Sorry, I didn’t mean to interrupt you.
0:36:28 No, no, no worries. I, I, we had a customer who was, you know, just trying something out. And in our,
0:36:32 in our platform, they’re like, you know, you actually have to click the button that says build,
0:36:39 right, build it, tell it to start. And he filed a ticket that was like, it’s broken. And I was like,
0:36:46 you know, very soon, we should be able to have AI that reads minds. But today we can’t, you still
0:36:48 have to press the button.
0:36:49 You still have to kick the button.
0:36:57 And so maybe, I think that like, from every industry, it, you know, as we continue to work
0:37:06 on accuracy, right, as we continue to increase dexterity for robots, as we continue to, you know,
0:37:13 pull camera data and, you know, do analyses, as we continue to map networks, as we continue to add
0:37:21 data to these systems, I think that every industry will shift exactly the same way we did for mobile,
0:37:26 right? Mobile was a thing. We all had mobile web, it was kind of gross, then the apps came out.
0:37:31 And then, then we had social, and now we do everything, we can’t live without our phones,
0:37:38 right? And so it’s exactly the same technological evolution as this we’ve seen in history. And so
0:37:42 is it a big deal to have this shift as close to the last shift? Absolutely.
0:37:49 But I do think that we will continue to evolve every industry. And I think if certain, like,
0:37:57 people got out of the way, we, we could make sure to do like, your climate change, and we could,
0:38:03 you know, solve these huge existential crises. And we could, like, these are all, in my opinion,
0:38:10 I could be wrong about this, but really solvable problems. Problems are human problems that we keep
0:38:16 creating. But I feel like we, we’ve moved out of the way, we could absolutely, out of our own way,
0:38:20 we could really make sure to, like, make the world a truly better place.
0:38:24 I’m going to leave it there. You said truly better place. That’s a perfect spot to end on.
0:38:30 Shania, this is, this has been so enjoyable and, and so informative. Thank you for, for taking the time
0:38:36 again. For listeners who would like to find out more about the work you’re doing, there’s the website,
0:38:43 again, I’ll spell it. It’s with an E, impromptu, E-M-P-R-O-M-P-T-U dot AI. Other places folks should
0:38:50 go, social media, research papers. Yeah, absolutely. You can follow me on LinkedIn. I do a lot of yapping
0:38:53 about my thoughts all the time. Perfect.
0:39:00 I just started a TikTok about, uh, fun behind the scenes of building an AI company with AI.
0:39:01 Oh, cool.
0:39:06 And so, um, you can follow us there and we’ll be publishing, um, lots of things coming out really
0:39:08 soon in the near future.
0:39:10 Great. And the TikTok is under impromptu?
0:39:13 Act that AI girl, Shania, S-H-A-N-E-A.
0:39:19 Perfect. Well, Shania, again, thanks for taking the time. Um, this has been great. And, uh, I, for one,
0:39:25 look forward to keeping track of, of what, uh, you and Sean are up to because, um, certainly the
0:39:29 outcomes and everything you’re talking about on the technical side, but the approach that you bring
0:39:36 to it, I think is, it’s a great one to have an Amplify in the world right now. So thank you.
0:39:38 Thank you so much for having me.
0:39:55 Thank you.
0:39:57 Thank you.
0:39:59 Thank you.
0:40:07 Thank you.
0:40:07 Thank you.
0:00:21 a quick reminder, if you’re enjoying the podcast, take a second to follow us wherever you get your
0:00:26 podcasts. It helps us out and it helps you out by making sure you never miss an episode that
0:00:32 just show up in your feed. My guest today is Shanae Levin. Shanae is the co-founder and CEO
0:00:38 of Impromptu AI. They’re focused on helping non-technical folks create AI products,
0:00:43 which is, as Shanae and I were just saying before we hit record, it’s where a lot of focus is these
0:00:48 days. So this is going to be a great conversation. And let’s get into it. Shanae, thank you for joining
0:00:52 the podcast. So happy to have you. Thank you so much for having me. It’s been such an honor.
0:00:56 So let’s start with a little bit about your background, if you don’t mind, and then getting
0:01:02 into what inspired you to co-found and build Impromptu and tell us what it’s all about.
0:01:09 Absolutely. So background is I studied business and computer science. And then I spent some time
0:01:17 at Google working on developer tools for Google Home and Android, helping build Android applications
0:01:21 for millions of developers all over the world. And then I spent some time at eBay working on
0:01:28 both ads and more traditional machine learning, which was super fun. And then I got tired of
0:01:35 just working at big companies. So I was like, I’m going to try this startup thing. And I then went on
0:01:41 to Cloudflare. And then I was a senior director of product at Docker. And then I was head of product
0:01:47 a series C company, startup company, and then I decided to go out on my own. I built a company
0:01:53 called Code C, which was helping developers to master the understanding of their code bases,
0:01:57 which back in 2019 was revolutionary.
0:02:04 It’s like the BT before Transformer era or something. I don’t know if we can call it.
0:02:08 Exactly. I’ve been doing this a really long time, helping people to understand their code.
0:02:13 And we built something called Code C AI, which basically allowed you to chat with your code base.
0:02:20 So I was in the GPT-2 beta, like way early in the process. And so we were building this generative
0:02:27 system back then. And then Code C got acquired and I decided to take some time off and travel around
0:02:36 the world. But then I ended up really starting off building just a very simple app with Lovable.
0:02:42 Tried to do that. And of course, being the technical person that I am, I really broke it. I just full
0:02:49 on broke it. And then I put a pause on that. I ended up taking a role for a short amount of time at a
0:02:54 subsidiary of Fox Sports to help them with their AI. And right around the same time,
0:03:01 I ended up meeting Sean, Dr. Sean Robinson, who was my co-founder. And I ended up, he ended up saying,
0:03:08 I invented this thing to get up to 98% accurate outputs out of AI. And I’ve looked at him and I
0:03:14 said, what, wait, what? Like, you can get up to 98% accurate outputs out of AI.
0:03:21 Like, what is going on? And so I was like, I needed that in a number of places at Cozy. It’s really hard
0:03:26 to get code. Like, right. It’s really hard to get AI outputs to be accurate. And that is the next
0:03:34 frontier. And so we decided to really team up and build and prompt you together. He is a computational
0:03:39 physicist and a researcher. And we’re always inventing new things. And it’s been really one of the
0:03:43 highlights of my entire life to be able to build this company for people.
0:03:47 That’s incredible. I have 9 million questions about your background that we don’t have time
0:03:52 for right now. So I’m going to narrow it down to just one. When you took time off and traveled,
0:03:53 favorite spot?
0:03:54 Oh my God.
0:03:55 You don’t have to pick one.
0:03:57 Okay. Okay. I’m going to pick two.
0:04:03 Okay. So it’s really hard. But I’m going to say I loved Japan.
0:04:03 Yeah.
0:04:12 Japan is amazing. And my husband convinced us to spend three nights in the Sahara desert or like
0:04:20 traveling from Fez to Morocco. And then we spent a night in the Sahara. And I’m definitely old enough
0:04:25 to remember, you know, Windows 95 screensavers. It looks exactly like that.
0:04:29 Like in real life, like a screensaver comes to life.
0:04:35 So when you’re done founding and selling successful startups, you can write some travel memoirs and
0:04:37 link them back to the tech experience. It is great.
0:04:38 Exactly.
0:04:44 So with all of your technical background, what made you, I mean, I don’t know what made you,
0:04:49 it makes sense to me why you wanted to found something for less technical people to be able
0:04:53 to create products, kind of get a handle on things, but sort of was that, I don’t know,
0:04:57 when you met Sean and the two of you decided to do this, was that the initial idea or did you kind
0:05:00 of iterate on some stuff before landing on Impromptu?
0:05:03 Yeah, no, actually that was not the original idea.
0:05:03 Okay.
0:05:12 We had no intentions on starting a huge company. We actually were started off by using the tech
0:05:19 that he invented the art, which we call now our AI core, which is our optimization tech. And we started
0:05:26 just building AI apps for people. Like people needed our help to build these AI applications. And we were
0:05:31 like, hey, how do we help them do it faster and more accurately? And that was always the goal. Like,
0:05:36 how do we make sure that we get good outputs? I mean, cause back then, like, you know, you would
0:05:43 get, you know, it would hallucinate like crazy. I mean, it kind of still does, but like it was wild
0:05:49 the responses. And so one of the things that we need to do is really build trust with AI. And so no,
0:05:55 we were just manually going and helping org people in our network build AI applications. And then after,
0:06:01 you know, over time, we ended up building a bunch of these and we kind of took a step back. I actually
0:06:08 literally have this recorded the moment that this happened. We’re like, oh, the infrastructure to
0:06:15 build AI applications across these, across all of our customers is exactly the same. I wonder if we
0:06:22 can agentically write this AI application. And Sean looks at me and goes, wait, you want an AI that
0:06:29 builds AI? And I was like, well, is that possible? And he’s like, no, it’s not possible. And then he does
0:06:35 the thing that he always does where he says no. And then, and then, you know, 20 minutes later,
0:06:42 he goes, well, maybe, maybe if we did this and this and this and fast forward till today. And here we
0:06:49 are, we have an AI that is very specialized, specialized builder to build actual AI applications.
0:06:55 And so you’re working primarily, your customers are primarily or entirely enterprise size clients?
0:06:59 Yeah. So they’re, they’re all, they’re businesses, businesses of different sizes,
0:07:06 right? So they’re mostly large enterprises and midsize enterprises. Typically, if you have a SaaS
0:07:12 application already, right? You know, you have a platform, you have a service that you’re doing with
0:07:19 the website, or you have, you know, a running business, how do I transform that into an AI
0:07:24 native company, right? And that’s really where the gap is, right? Like, you don’t, you need to start
0:07:25 from this legacy code base.
0:07:29 I was going to say, that’s, that’s the trillion dollar question, except, I don’t know, I keep
0:07:33 thinking about the 98% accuracy, and that might be the $2 trillion question.
0:07:39 Well, here’s the thing, both of those have to work together, right? You, you can’t have,
0:07:43 you know, it doesn’t matter if you can get 98% accuracy, if you have to throw out your entire
0:07:50 code base, right? It, and it doesn’t matter if you, you know, have to, you know, you, you’re starting
0:07:55 from a greenfield project, and then your output kind of sucks, right? So you need a lot of these
0:08:01 pieces to work together. And we’ve invented, you know, five new pieces of tech from adaptive context
0:08:09 engines to infinite memory, to being able to, to our optimization core, to be able to ingest a code
0:08:14 base and say, hey, I’m going to import this whole code base from GitHub and now add AI to it, right?
0:08:17 Like, that wasn’t possible before we invented the things that we did.
0:08:21 Right. To kind of focus on the enterprise scale for a minute here, when you’re working,
0:08:27 you know, with a client of that size and complexity and teams of data scientists and, you know,
0:08:32 so many stakeholders. Yeah. And you start working with somebody who’s less technical,
0:08:40 but the idea is to help them build something kind of in that context. How do you approach it? And I
0:08:44 don’t know if the best way is to kind of ask you questions piece by piece, or if you want to walk
0:08:49 through kind of an example project, but when I’m messing around with AI and having it build something
0:08:54 for my own purposes, right? I mean, obviously the, the threshold for success is much lower. It
0:08:58 doesn’t matter if it’s not perfect just for me, but also the process, you know, I can just kind of put
0:09:03 my head down and mess around and see what I come up with. I’m at work, I’m in the team. It’s a very
0:09:09 different situation, right? So, so what is that like for Impromptu and the folks you’re working with?
0:09:18 Well, it’s multifold, right? Because the mission of our company is to make AI accurate and accessible,
0:09:26 right? So how do we make sure to get this technology into the hands of people who, it’s not that, you
0:09:30 know, they haven’t coded before because I talked to 20 year developer, like people have been developing
0:09:36 for 20 years and this is a new technology that we’re all learning at the same time, right? And so
0:09:41 all of that comes with education, all of that. It’s like, if I said, Hey, you’re going to go walk
0:09:47 into Sephora and now you need to go and find sunscreen amongst a thousand products. How do you do that?
0:09:52 You know what I mean? So like, it’s, it’s just, I do. I tried to get something from Sephora once for
0:10:00 somebody else. There’s a lot going on. Exactly. It’s the same thing. Right. So it’s, it’s, it’s not a
0:10:05 less technical person per se, right? But it’s someone who’s less, less versed in what’s happening
0:10:13 in AI, which is a race to keep up with. Exactly. And so we, and we, we definitely think about like,
0:10:21 Hey, everybody is learning generative for the first time. What are the pieces to educate anybody at any
0:10:28 technical level? And so we do that not only with our UI where the UI of the builder allows the actually
0:10:34 knows what all seven pieces are to build the full seven pieces to make an AI application, but it asks
0:10:39 you back and forth questions to make sure that you have supplied it with the right information and it
0:10:47 goes and has a full conversation with you. And we also have a co-build model, right? So we as experts
0:10:54 will also alongside of our AI tech also build in collaboration with the organization and educate
0:11:00 their team, work alongside their teams so that, you know, it, it doesn’t feel like we’re just going
0:11:08 to kind of leave you in lurch, right? And so, cause enterprises we need, like everything is changing so
0:11:14 fast, right? You need to have someone who’s keeping up with this while you’re running your business and
0:11:20 running your team and running, you know, all of the things internally. Someone needs to keep up with
0:11:21 this stuff. And that’s basically us. Yeah.
0:11:26 What is that, you know, where we, I use the word transformation all the time, you know, you just
0:11:33 mentioned it and it’s accurate, but what does it mean in this context of, you know, transforming an
0:11:38 industry, transforming a business and the work that you’re doing sort of, you know, on the ground level,
0:11:42 right? I mean, you’ve got that, that high perspective from your background and everything,
0:11:46 but you’re right there doing it. So, you know, folks like me, you’re looking at kind of broader
0:11:51 trends and thinking like, wow, you know, the ability to, AI has really changed and it was really
0:11:56 changing the way search works, right? It’s just a giant example, right? Kind of a big glossy thing
0:12:02 to say, but what are you, you know, what are you seeing? Are there, are there certain trends that are
0:12:08 emerging, certain types of problems that people are having more success or less success, you know,
0:12:16 Yeah. So I, so the larger trends as we’re kind of working on these things together is a huge thing
0:12:23 in physical AI, right? Like how do you move things in a physical space? We work with a ton of folks in
0:12:31 that capacity. We also are really thinking about context, right? And memory and how, as you think
0:12:38 about like, if code is going to be this, you know, arbitrary commodity, what then becomes important?
0:12:44 And then that’s the data, right? How do you then tackle this next frontier to create an end-to-end
0:12:51 solution? So now on, you know, last week we announced our custom data models in straight into our builder.
0:12:57 So now you have, not only you can build an application, but it’s built right off of the custom data
0:13:02 and it’s trained in the exact way that you would want it to be trained. And so, so you get really
0:13:08 highly accurate responses. And then that becomes really, you take it one level up. So you start
0:13:13 thinking about multi-tenancy. You start thinking about specialized models. You start thinking about
0:13:18 AI doing decision-making for you. And that’s where you really, where you get into this thing called
0:13:22 provable AI. Like what’s actually happening? What decisions is it making? How do I control?
0:13:28 And then thinking about the governance plane. Well, if I have 10 applications across my
0:13:33 organization, how do I make sure that it follows these rules? And we’re tackling all of those things
0:13:40 in one fell swoop to be able to make sure that if you are going to go down this journey, that it is not
0:13:46 only accessible to anyone on your team and it scales appropriately, but it also feels like you have full
0:13:54 control over it. Right. How big is the gap for a, you know, first time generative AI builder, to put it
0:14:01 that way, from, you know, something that I might build and say like, oh, this like seems to work right in my
0:14:07 use case, going from that to something that’s production ready. How big is that gap and how do you help
0:14:14 help people bridge it? Yeah. So great question. We actually think about it slightly differently where I can
0:14:20 get you a chat bot in probably 10 minutes. It has all the production ready pieces that you need.
0:14:26 Right. But what, what you actually are now, what we’re thinking about is enterprise scale. How do you
0:14:33 do that with terabytes of data? How do you do that in real time across millions of users? Right. How do
0:14:39 you do that, you know, where you have full governance and control? Right. So like today in our self-serve
0:14:45 platform, if you’re on like a big enough plan, I can get you very fast, very accurate system
0:14:53 in probably 10 minutes, but it’s, it’s, it’s the bridging the gap as you grow and have multiple
0:14:59 interconnected AI workflows and, you know, being able to make sure that they’re all accurate and all
0:15:05 running to the best of their ability. Right. You use the word accessibility a minute ago and,
0:15:11 you know, talking about AI being accessible to first-timers, less technical people. What does
0:15:16 that mean, you know, kind of day-to-day in your work? And maybe like, are there ways that people
0:15:23 use it or kind of misconceptions people have about, about that term? Yeah. So I think AI accessibility
0:15:33 accessibility at this stage is allowing anyone with an idea, not just to use AI to build old
0:15:40 technologies, right? We, it’s, it’s not just using generative to make HTML, CSS, JavaScript, like all
0:15:46 of those are great, right? We absolutely need those. But how do we use AI to actually build AI, to put
0:15:54 generative, thinking, living, breathing applications into people’s hands? And so it’s not only new for
0:15:59 developers, but it’s new for the business owners. It’s new when you’re thinking about companies being
0:16:07 disrupted. It’s new for the venture community. It’s new for everyone. And so how do we democratize and
0:16:15 actually have this new technology to be disseminated to anyone who wants it? Because now, you know, I can,
0:16:22 we have customers who, you know, are doing these amazing things and now can enable these
0:16:30 even more amazing things for their customers. So, you know, we have a mom and daughter team who is,
0:16:37 you know, building financial literacy, or we have a CPG brand who takes ocean plastic and turns that into
0:16:44 active wear, right? Like, so not just the super technical large enterprise companies, but like every
0:16:50 across the spectrum, the example of the mom and daughter team, like, I can wrap my head around
0:16:56 that easily because I’ve, I’ve, you know, typed in like, help me code a dashboard for podcast analytics,
0:17:03 right? And I’ve gone on that journey. How does AI help? And, you know, impromptu, your experience,
0:17:08 how did it help the process of recycling ocean waste to create active wear?
0:17:14 So really thinking about how do you bring all of these systems together, right? How do you make
0:17:20 recommendations to, you know, the founders, how to, and their, you know, their teams internally to make
0:17:27 sure that, you know, if you’re spinning up an event to connect, collect the, uh, the, the bottles, like,
0:17:34 how do I spin up a new app, a new city across the country to do the exact same thing? How do I
0:17:41 operationalize this process? Tell me how to do it, AI, right? Um, and so putting all of, but you, in order to do
0:17:47 that, that’s very custom data, right? That you’re not just going to get from a general model, right? You need
0:17:53 their system and their models, um, because that’s not a very typical use case. And so, um, those are the sorts of things
0:18:02 that now you enable, I use AI every single day, uh, in my, in my system to build this company. Um,
0:18:09 it’s unheard of going from five years ago, building a company then until now, like my day-to-day is
0:18:14 completely different. Yeah. Well, it’s funny as you were saying about when you and Sean, I want to call
0:18:20 him Dr. Sean. You should, you should call him Dr. Sean. When, when you and Dr. Sean, when you and your
0:18:24 co-founder, I’ve got together, you know, in the question of, wait, you want me to build AI that
0:18:30 builds AI? And like you said, fast forward. And now, you know, I mean, I heard Jensen say it,
0:18:37 but lots of people are saying it, that AI is going to mostly generate tokens to be used by agents,
0:18:42 other AI systems, like, right? Like the majority of the tokens as we go forward are not going to be
0:18:48 creating output that humans sees, humans see, excuse me, but that, you know, another AI is going to act
0:18:55 with. Yeah. And it’s amazing. I’m sorry. Yeah, no, it’s, it’s truly, it’s truly amazing. You know,
0:19:02 we’re working with some customers who, and some, you know, people’s dreams who, you know, would have
0:19:08 never had the ability to, to do this. But then also you’re thinking about like, oh, actually I don’t
0:19:14 like, eventually we’re not going to need a full human into this at all. Right. And so we can actually
0:19:20 AI generate human in the loop systems, but we can also just, you know, generate a full end-to-end API
0:19:27 that, you know, goes out from the front end, does a whole system, does an AI generate and puts it back
0:19:32 as an API back into the customer system. And like, no one ever knows that we’re doing anything,
0:19:38 right. Cause it’s fully containerized and, and all of those things. So we’re already seeing
0:19:46 that day to day. My guest is Shania Levin. Shania is CEO and co-founder of Impromptu with an E,
0:19:52 E-M-P-R-O-M-P-T-U dot AI. I want to get that out there if you’re listening now and want to look it up
0:19:59 later. And we’re talking about Impromptu’s work, helping people of all technical abilities,
0:20:05 but certainly people who are less used to building with generative AI, build products and
0:20:10 at enterprise scale and production ready and, and the whole thing. Um, and so I want to ask you kind
0:20:17 of a more technical question here. Impromptu uses NVIDIA CUDA. Yeah. Can you talk about how, uh,
0:20:23 the CUDA libraries for embedding and classification in particular, improve your performance and make AI
0:20:28 efficient, all that good stuff, but how it actually translates to the work that you’re doing.
0:20:32 And then in particular, the work that your customers are doing and creating with these
0:20:40 tools. Yeah. So we consider Impromptu to be a mixed code builder instead of no code or pro code. It’s
0:20:44 kind of, we call it a mixed code under the hood though. You know, parts of it are essentially,
0:20:50 you know, a big embedding and classification machine, right? So every time, you know, we map a
0:20:56 new feature or iterator design, or, you know, you search your custom data model, you know, we’re turning
0:21:03 that into vectors and making bigger decisions on top of that. Now we have our own kind of custom models
0:21:09 and our own secret toss alongside of that, but we use NVIDIA’s code, uh, libraries so that all of the,
0:21:15 you know, the heavy math natively runs on GPUs instead of, you know, slowly on CPUs. And that really gives us
0:21:23 two big wins, right? First it’s performance. So like all of the everyday creators in our UI get that instant
0:21:28 feedback. They can, you know, request a feature or upload or tweak a prompt or, and the AI just kind
0:21:34 of responds. And so they can like iterate much faster, ship more, much more useful products.
0:21:41 Um, and then the second thing is efficiency, right? So, you know, CUDA lets us serve a ton of those
0:21:46 workloads in like a relatively small GPU footprint, uh, whether that’s on our, you know, in our cloud or
0:21:51 our customers VPC. Right. And how does CUDA help? I mean, you talked about this a little bit to,
0:21:56 to put a point on it. How does CUDA help kind of bridge the gap? And I’m wondering from your
0:22:02 perspective, how big or small that gap really is these days between kind of the, the cutting edge
0:22:07 AI research that’s being done and then the tools that, you know, your customers are actually able
0:22:13 to use to build things. Yeah. I think at this point, you know, the gap is mostly filled.
0:22:22 If I do say so myself, um, you know, you don’t write today, you know, need to, you know, a big
0:22:27 machine learning team to build something sophisticated. Um, you know, CUDA kind of helps us with that hard
0:22:32 math. And so our platform turns all of that into, you know, this fast, affordable, you know, just,
0:22:40 just works experience for, you know, non-technical and technical teams because like technical versus
0:22:46 non-technical, those now are, they go together now, right? It’s generative or not generative.
0:22:54 Right. And so as we’re thinking about that, like all of those things are now been able to do it in a,
0:23:00 you know, a self-service if you really want to do that, or if you want the pros to actually build it
0:23:05 for you, you can just throw that out, right? And we can actually just use our own systems to build
0:23:10 it on your behalf and do the integration for you. But all of the things under the hood are basically
0:23:15 all like taken care of. All right. So this is the point where I come back to this 98%
0:23:20 thing that, you know, I was like, I don’t want to ask up front because probably it’s secret sauce and
0:23:24 we’re not going to get too into it, but can you get into to any of the details of how it works?
0:23:31 Yeah. So it does work a little bit differently than most other systems. And what we found is that
0:23:35 a lot of people talk about benchmarks, model benchmarks, right?
0:23:35 Sure.
0:23:41 Model benchmarks are great. Everybody has a model benchmark. Excellent. But how do you translate
0:23:48 that model benchmark to the everyday enterprise? Who’s like, I have a goal. I have things to do.
0:23:57 Right. And so our accuracy is actually redefining that success, which is task success. And because
0:24:04 we work with so many use cases and so many customers and enterprises, we allow the user to define what
0:24:11 success means to them, what that task is, what success looks like. And we will use our full optimization
0:24:17 engine that optimizes the entire system, including the model, the data, the prompts, the evals, all of
0:24:23 the under secret sauce under the hood to optimize in real time towards that goal that the user sets.
0:24:30 So, and we measure it directly in our dashboard. So you’ll see a big number of how we improve that
0:24:35 accuracy towards the goal that the user sets. The other thing that might not be so obvious
0:24:43 is we have two modes. One is for, it’s called manual optimization, which are all for all of those
0:24:49 technical, deep technical developers who want to tinker and turn knobs just like myself, right? I want
0:24:55 to do that. But you can actually see the output towards that accuracy and it will actually give you
0:25:00 suggestions. You can click the button however many times you want to click it and, you know, improve it
0:25:06 however many times you want. Um, after about 30 runs though, we can turn on something called automatic
0:25:11 optimization. So if you’re less technical and you’re like, I don’t know what the heck that is,
0:25:17 that sounds smart. It is, but you don’t want to tinker with it yourself. We’ll just do the automatic
0:25:25 optimization for you up to, you know, 98% of that task accuracy. And we’re always improving it. It’s
0:25:30 always, you know, we’re always trying to get to, you know, three nines, which I definitely presume
0:25:35 that we will one day. And so that, you know, that speaks to, and you, and you brought this up before,
0:25:39 and obviously it’s a big, a big thing right now in the, in the industry and the marketplace
0:25:46 is building trust. Definitely. Are your, you know, your customers, like, are they looking at,
0:25:53 like, does that dashboard, that figure of optimization, does that do a lot to improve the trust? Is it really
0:25:59 just seeing the results and being able to say like, yes, it helped me get this project, you know,
0:26:05 off the ground and running and it’s working well? How do you build trust, you know, right now when
0:26:11 there’s kind of this tension between the race to build out all the infrastructure and the data centers and
0:26:16 everybody wants chips and can’t get them? And then, you know, everything that, that we’ve been talking
0:26:23 about. Yeah. So we subscribe to something called provable AI and we think that, um, that is the
0:26:31 absolute way to build trust, right. And, and make it so that it is accessible, right. If you can’t see it,
0:26:36 feel it, touch it in a theoretical sense, cause it’s all bites and it’s all numbers, ones and zeros,
0:26:41 right. Um, you know, until we get to physical AI, like you mentioned, and then, you know, right,
0:26:49 exactly. So, uh, thinking about that, you know, seeing, okay, what decisions is the AI making?
0:26:56 How do I roll back? Right. How do I, you know, if I have infinite memory, you know, how, like what,
0:27:03 you know, is involved in this decision, right? Like if I’m doing a custom data model, you know, how,
0:27:09 what data is kind of going in, being able to showcase, like here are all the runs that we took,
0:27:15 right. Um, showing those accuracy numbers in the dashboard, right. We always try to showcase,
0:27:22 you know, provable, Hey, this is the number. This is the suggestion. This is the, you know, uh, the,
0:27:28 the set that you need to be able to do now. Can you disconnect those things for privacy concerns?
0:27:36 Absolutely. But proving to the user that this is, we’re doing what we say we’re doing is of the
0:27:45 utmost importance. Yeah. Yeah. To switch gears a little bit as a woman in tech, right. Which like
0:27:50 everything else, particularly everything else in tech has changed a lot very quickly in certain ways.
0:27:56 And, and, you know, not to speak for you, but probably not so much in other ways, but then,
0:28:04 you know, working now with a lot of people who are, you know, the whole idea of making AI accessible,
0:28:11 right. How does your experience as you, as a woman in tech, you know, co-founding and running this
0:28:17 company now and working with folks, how does that inform kind of how you see, you know, barriers to
0:28:22 entry, whether it’s, Oh, I’m new to generative and I want to start working with generative or, you know,
0:28:29 Oh, I’ve never done something at production scale or, Oh, like I’m a woman and I’ve experienced what I’ve
0:28:36 experienced. And so I’m interested in this stuff, but I don’t know. Like, yeah, it definitely really,
0:28:43 you know, hits my heart. I can tell you a very quick story. So please, when I, uh, I said,
0:28:48 obviously studied business computer science, but I had a small web development company, um, that I
0:28:54 started when I was 19. And then my first big job, like big, big job after I was in an analyst was
0:29:04 working at Google and I was like, uh, so I had two choices, you know, uh, go to search Google search
0:29:11 at the time and then go to this thing called developer tools. And at the time I was like,
0:29:18 I just seen the internship, which I know kind of, but it felt like this really overwhelming technical,
0:29:23 like I’m going to be working with some of the biggest, smartest developers that, you know,
0:29:28 ever, it’s very intimidating. And then I ended up choosing to go to developer tools. Cause I thought
0:29:37 that would be easier. Um, but I’ve been obviously changed the course of my entire life, but, um, that
0:29:44 one decision, and I don’t want anyone else to make a technical decision like that out of fear.
0:29:51 Right. I don’t want women or kids that I talked to. I have 12 year olds emailing me,
0:29:59 right. Thinking about how to build generative systems. And I never want that to feel like
0:30:06 you have an idea and you can execute it. Right. And so we take great care with whatever technical
0:30:12 ability you have to take advantage of this technology. Cause I constantly think about,
0:30:18 you know, going into these rooms, right. With, with people. And I think luckily now I’ve,
0:30:25 I’ve built things for millions of people, but at the same time, I know deeply what that feels like
0:30:31 to be told that you’re not technical enough to be told or to have someone, you know, talk to Sean and not
0:30:38 talk to me. Right. And, you know, those sorts of things. So I think that it is, we are at a place in our
0:30:48 our evolution, um, that now that barrier is fully down, right. It’s fully, it’s fully unblocked.
0:30:54 Everybody is running. We are, you know, I’m talking to, you know, people who have gotten laid off from
0:31:02 jobs. You know, how do I take this knowledge that I’ve gained over the last, you know, 10 years and
0:31:09 start monetizing it into a real business and are making real money off of this using generative.
0:31:15 And then also completely on the other end, right. At these large enterprises where they’re like, Hey,
0:31:22 I know we use AI internally, but I still don’t know really how to use it. Right. Like, how do I set myself
0:31:27 up for success? How do I put this in front of users to help businesses reach their goals? I mean,
0:31:32 it’s all over the place. Right. So thinking about all of those different stakeholders is really
0:31:36 important to us. All right. So keep thinking about all those stakeholders, right? Yeah. The notion
0:31:45 that studying computer science, the 12 year olds, right. Studying computer science may not necessarily
0:31:51 be the route to take now. If you want to go into a, we’ll just call it a computer career, so to speak.
0:31:57 Right. If it’s not, if it is, tell me, but if it’s not, if coding skills aren’t necessarily like
0:32:05 the top thing looking forward that you need to be successful with AI, with technology, with generative,
0:32:11 everything, what are some of the skills, some of the mindsets, some of the knowledge that you would
0:32:17 advise the 12 year old or, you know, the, the 50 year old who got laid off and wants to monitor,
0:32:21 everybody wants to do something new. Yeah. What would you tell them to get into and how
0:32:28 would you guide them? So as the eldest of six children, I happen to have my youngest sister
0:32:36 as an intern at my company. And so I think about this all the time, you know, and I, I think that
0:32:44 for kids coming out of the, out of school today, um, I don’t, I, you know, I know that the headlines
0:32:49 are thinking about, you know, compute, like computer science is out and you can’t get jobs as junior
0:32:56 developers. But when I studied computer science, it was not just about coding, right? It wasn’t as
0:33:02 computer science is not, it’s about con is about cognition. It’s about thinking in systems. It’s about
0:33:08 critical thinking skills. It’s about, you know, how do you break problems down into small shippable
0:33:15 chunks? It’s about, you know, building on top of and merging that creative energy, that creative ability
0:33:23 with, you know, technology that’s always changing, right? Like, you know, I was in school and did
0:33:29 everything on a Java application and that allowed me to go into the Android system. But then Google decided
0:33:37 that we’re going to use Kotlin now, you know what I mean? So like, like, I took, I took intro to CS and,
0:33:42 in, uh, and it was taught in scheme. So yeah, you know, things change. Yeah. Right. Exactly. I mean,
0:33:47 the first class was like, we did assembly coach, you know what I mean? So like, I think that people are
0:33:54 underestimating the, the quality of a good computer science education, because I think that the critical
0:34:02 critical skills around how do I think about these larger problems that are going on in the world and
0:34:08 the world and how do I solve them needs critical thinking skills. And that needs to be taught and
0:34:14 practiced. Right. Right. Yeah. Even, even, even with the, you know, the metaphor, and this may have been
0:34:20 from another episode recently, um, the, the metaphor of like conducting the orchestra, right. You know,
0:34:26 you may not be writing all the code anymore, but you’re conducting the orchestra, humans, bots, a mix.
0:34:30 Absolutely. And that’s exactly what you’re talking about are those big picture thinking skills. Yeah.
0:34:36 Like even, even as a, you know, a PM for a very long time in my career, uh, all, you know, always on
0:34:41 very technical products, but thinking about system design, right. You’re going to need to read and
0:34:46 understand all of the code of the AI rights. And that is a skill in and of itself outside of writing
0:34:51 code. Right. So it, you know, it’s just like, if I decided to go read Japanese tomorrow,
0:34:57 right. Like I, I still need to practice that just the same way I need to learn to practice,
0:35:03 to read code and to keep that skill up to speed. And so from people coming out of college, I think
0:35:09 computer science education is not necessarily dead despite the headlines. I think that the types of
0:35:14 jobs and the types of skills that should be taught in computer science education just need to be, uh,
0:35:17 some need to be de-emphasized and some need to be re-emphasized.
0:35:21 That kind of leads us to, you know, sort of our last forward looking question here.
0:35:25 And, and given the conversation, I kind of want to ask you from the two points of view,
0:35:30 the individual or small team, and then the enterprise scale or, you know, the organization,
0:35:35 right. And of whatever size, but certainly the enterprise scale, where do you see the biggest
0:35:42 opportunities for AI to really make a difference and to empower folks and to, you know, lead to,
0:35:46 I mean, whatever the successes are, given the context, like, where do you see the biggest
0:35:52 opportunities, you know, in the next, whatever, I, two years is almost too long, right. But whatever
0:35:54 the next window in your head is.
0:36:04 Yeah. So I think the real opportunities are in fully remaking our entire world, honestly. Like,
0:36:12 I think that’s, I genuinely, I genuinely believe that, like, there are problems that if people.
0:36:15 Can you open source the plan when you and Dr. Sean saw on it?
0:36:20 Absolutely. Absolutely. I’ll tell you a small joke.
0:36:21 Sorry, I didn’t mean to interrupt you.
0:36:28 No, no, no worries. I, I, we had a customer who was, you know, just trying something out. And in our,
0:36:32 in our platform, they’re like, you know, you actually have to click the button that says build,
0:36:39 right, build it, tell it to start. And he filed a ticket that was like, it’s broken. And I was like,
0:36:46 you know, very soon, we should be able to have AI that reads minds. But today we can’t, you still
0:36:48 have to press the button.
0:36:49 You still have to kick the button.
0:36:57 And so maybe, I think that like, from every industry, it, you know, as we continue to work
0:37:06 on accuracy, right, as we continue to increase dexterity for robots, as we continue to, you know,
0:37:13 pull camera data and, you know, do analyses, as we continue to map networks, as we continue to add
0:37:21 data to these systems, I think that every industry will shift exactly the same way we did for mobile,
0:37:26 right? Mobile was a thing. We all had mobile web, it was kind of gross, then the apps came out.
0:37:31 And then, then we had social, and now we do everything, we can’t live without our phones,
0:37:38 right? And so it’s exactly the same technological evolution as this we’ve seen in history. And so
0:37:42 is it a big deal to have this shift as close to the last shift? Absolutely.
0:37:49 But I do think that we will continue to evolve every industry. And I think if certain, like,
0:37:57 people got out of the way, we, we could make sure to do like, your climate change, and we could,
0:38:03 you know, solve these huge existential crises. And we could, like, these are all, in my opinion,
0:38:10 I could be wrong about this, but really solvable problems. Problems are human problems that we keep
0:38:16 creating. But I feel like we, we’ve moved out of the way, we could absolutely, out of our own way,
0:38:20 we could really make sure to, like, make the world a truly better place.
0:38:24 I’m going to leave it there. You said truly better place. That’s a perfect spot to end on.
0:38:30 Shania, this is, this has been so enjoyable and, and so informative. Thank you for, for taking the time
0:38:36 again. For listeners who would like to find out more about the work you’re doing, there’s the website,
0:38:43 again, I’ll spell it. It’s with an E, impromptu, E-M-P-R-O-M-P-T-U dot AI. Other places folks should
0:38:50 go, social media, research papers. Yeah, absolutely. You can follow me on LinkedIn. I do a lot of yapping
0:38:53 about my thoughts all the time. Perfect.
0:39:00 I just started a TikTok about, uh, fun behind the scenes of building an AI company with AI.
0:39:01 Oh, cool.
0:39:06 And so, um, you can follow us there and we’ll be publishing, um, lots of things coming out really
0:39:08 soon in the near future.
0:39:10 Great. And the TikTok is under impromptu?
0:39:13 Act that AI girl, Shania, S-H-A-N-E-A.
0:39:19 Perfect. Well, Shania, again, thanks for taking the time. Um, this has been great. And, uh, I, for one,
0:39:25 look forward to keeping track of, of what, uh, you and Sean are up to because, um, certainly the
0:39:29 outcomes and everything you’re talking about on the technical side, but the approach that you bring
0:39:36 to it, I think is, it’s a great one to have an Amplify in the world right now. So thank you.
0:39:38 Thank you so much for having me.
0:39:55 Thank you.
0:39:57 Thank you.
0:39:59 Thank you.
0:40:07 Thank you.
0:40:07 Thank you.
Empromptu CEO Shanea Leven shares how her company helps people without coding experience build meaningful, production-ready AI applications — fast and accurately. Powered by NVIDIA CUDA, Empromptu’s “AI that builds AI” platform is making cutting-edge technology accessible to all, enabling creators to turn bold ideas into real-world impact.
Listen to the full show archive at ai-podcast.nvidia.com
Leave a Reply
You must be logged in to post a comment.