AI transcript
0:00:10 [MUSIC]
0:00:13 >> Hello, and welcome to the Nvidia AI podcast.
0:00:15 I’m your host, Noah Kravitz.
0:00:18 Today, I’m joined by Barasch Kultekin,
0:00:20 the head of AI at Snowflake.
0:00:22 At Snowflake, he’s driving the development of
0:00:24 cutting-edge AI and ML products,
0:00:27 including Snowflake Cortex AI,
0:00:29 and ARTIC, their new foundational model.
0:00:33 But Barasch has also had a remarkable journey through AI himself,
0:00:35 having co-founded Google Now and
0:00:38 led AI efforts for Google Assistant.
0:00:40 So we’ve got a ton to talk about background,
0:00:42 the present at Snowflake, and of course,
0:00:44 we’re not going to let Barasch off the hook without asking
0:00:49 him to opine about the future of AI and the hot topic today.
0:00:51 What’s going to happen to all of us in the age of AI?
0:00:53 So I can’t wait to get into it.
0:00:55 Barasch, thank you so much for joining
0:00:57 the Nvidia AI podcast and welcome.
0:00:58 Thanks a lot, Noah.
0:01:01 I sometimes wonder if the pods we tape on a Friday,
0:01:03 let alone today, a Friday afternoon,
0:01:05 have a little bit looser of a feel than some of the other ones.
0:01:08 So we’ll have to do an exit survey at the end here.
0:01:10 But I’m super excited to talk to you.
0:01:12 We were talking offhand.
0:01:13 I’ve been tracking Snowflake,
0:01:18 especially over the past few years as interest in AI and data has exploded.
0:01:21 A good friend of mine has been with Snowflake now for a few years,
0:01:23 so I’ve been tracking her journey as well.
0:01:25 So I’m thrilled to have you here, Barasch,
0:01:28 and to learn more about Snowflake and your own journey.
0:01:32 Maybe we’ll start with Snowflake and we can get into your background as we go.
0:01:36 Can you start by telling the audience a little bit about what Snowflake is,
0:01:39 how long the company’s been around, what you do,
0:01:45 and obviously your role in the burgeoning AI explosion that we’re all part of?
0:01:48 Of course. So Snowflake is the AI data cloud.
0:01:51 We started the journey over a decade ago
0:01:55 focusing on how do we make data a lot more easily accessible,
0:02:00 how do we make data processing a lot more easily accessible to companies.
0:02:02 As the data volumes were growing,
0:02:07 there was a big innovation of separating the storage of data from compute
0:02:12 that allowed a massive unlock in the data space.
0:02:17 And since then, we’ve been evolving from a provider of a data warehouse
0:02:21 to a data cloud where that data can be easily shared.
0:02:23 And now we’re in the third phase of that journey
0:02:29 where you can now unlock a lot more value of that data with AI for a lot more users.
0:02:32 Right. Maybe just to set baselines,
0:02:36 can you quickly just talk about terms like data warehouse, data lake,
0:02:40 the AI data cloud, as you describe Snowflake,
0:02:44 and what those terms mean and maybe how they’ve evolved a little bit?
0:02:48 Of course, customers have a lot of structured data.
0:02:52 They need to have a way to bring all of that data
0:02:57 and then manage it, govern it, be able to run analysis on it.
0:03:02 So data warehouse allows our customers to very efficiently run
0:03:06 massive scale analysis across large volumes of data.
0:03:07 Right.
0:03:11 And that’s for structured data for tables.
0:03:17 With data lake, you expand that to bringing in more unstructured data
0:03:19 as structured data into the mix.
0:03:20 Right.
0:03:24 And so there’s issues at hand of the way that the data has stored itself,
0:03:27 obviously, the physical media it’s stored on.
0:03:31 But then, as with all things technology and certainly all things AI,
0:03:34 machine learning, deep learning, the software plays a role,
0:03:36 the interconnectivity plays a role.
0:03:39 Obviously, if it’s a data cloud, there’s all of the…
0:03:43 Everything that happens between your cloud and my workstation,
0:03:44 wherever in the world I am.
0:03:46 So there’s a lot going on.
0:03:50 So maybe we can dive in a little bit to some of the breakthroughs
0:03:54 that Snowflake has ushered in, is currently working on.
0:03:56 And kind of how data plays a role,
0:04:02 what the data cloud’s role is in the modern ALML stack, if you will.
0:04:03 So let’s talk a little better,
0:04:09 or if you would talk a little bit about how Snowflake works with enterprises
0:04:12 in helping them unlock their data, as you put it.
0:04:13 Yeah.
0:04:17 So first of all, we’re super excited about what AI is capable of,
0:04:18 what AI is bringing.
0:04:20 And of course, when we think about AI,
0:04:25 there is no AI strategy without a data strategy, is what they say.
0:04:27 AI is fueled with data.
0:04:32 So what our customers are interested in is they are trusting Snowflake with their data.
0:04:36 How can they now get the most out of this data
0:04:39 by bringing AI compute right next to where the data is?
0:04:43 So believe deeply in being able to bring compute to data
0:04:48 versus bringing massive amounts of data out into where the compute is.
0:04:52 So the way we work with our customers is that we built a AI platform.
0:04:58 And with this AI platform, our customers can run natural language analysis,
0:05:02 can build chatbots, can talk to their data in natural language.
0:05:06 And we make this platform available, and our customers are using it
0:05:08 to build all sorts of AI applications.
0:05:10 And so the customer stores their data with you,
0:05:13 but then also the training, the inference,
0:05:17 all the compute operations are done on your side as well.
0:05:18 That’s right, exactly.
0:05:21 So basically, our AI platform runs fully inside Snowflake.
0:05:25 So for Snowflake, it’s really important to govern the data,
0:05:28 as well as AI that’s running.
0:05:30 So we run everything inside Snowflake.
0:05:33 So maybe you can get into a little bit of some of the product offerings.
0:05:36 Do we want to start with Snowflake Cortex?
0:05:40 Yeah, so Snowflake Cortex is our managed service.
0:05:44 It is our offering where we’re running a series of large language models
0:05:45 inside Snowflake.
0:05:49 Our customers have super easy access to these large language models.
0:05:55 But the way we think about AI is we want to make AI easy, efficient, and trusted.
0:05:57 With Cortex, it is incredibly easy
0:06:00 because AI is running right next to where the data is.
0:06:02 So our customers don’t have to build data pipelines
0:06:06 to manage data in multiple places and govern it.
0:06:08 We were running it in a secure environment
0:06:11 and we have a series of very efficient models
0:06:13 from our own to a series of partner models.
0:06:14 We make that available.
0:06:17 And then Cortex also makes it very easy for our customers
0:06:21 to build, talk to my data experiences, if you will,
0:06:26 build chatbots, both for documents as well as for structured data.
0:06:30 What are some of the use cases that are kind of most popular, most seen?
0:06:35 I’ve heard people talk about, we’re recording in late June of 2024.
0:06:41 And I’ve heard people reference 2023 as being the year that generative AI
0:06:45 took over all the headlines and everybody was talking a lot,
0:06:47 but not really sure what to do.
0:06:53 And 2024 maybe being the year that businesses start to actually
0:06:57 develop applications or use applications others have developed,
0:07:02 but really start to do things using leveraging AI on their own data
0:07:07 to whatever it is, improve processes, try new ways of working,
0:07:08 all that kind of stuff.
0:07:12 What are you seeing on your end is the things that customers
0:07:15 and enterprise customers, other snowflake customers,
0:07:17 are interested in doing and then maybe on the flip side,
0:07:21 some of the things that, I don’t know, they’re concerned about
0:07:25 or don’t quite understand or still trying to kind of wrap
0:07:26 their collective heads around.
0:07:28 Yeah, so I agree with you.
0:07:32 In 2023 was, I’d say, the year of proof of concepts.
0:07:34 Right, well said.
0:07:38 We got their hands on AI and then started building demos.
0:07:43 And this year we’re starting to see these turn into real production use cases.
0:07:46 I can give a couple of examples.
0:07:50 We’re working with global pharmaceutical company Bayer
0:07:54 in building an experience where Bayer’s internal teams
0:07:56 and sales organizations, marketing organizations,
0:07:59 can ask questions of their structured data.
0:08:03 So Bayer believes that dashboards can only take them so far.
0:08:05 Dashboards tend to be very rigid.
0:08:08 And then first thing that happens when you see a dashboard
0:08:11 is you have three questions, four questions where you want to drill in
0:08:15 and figure out why something isn’t the way you want it to be or where.
0:08:19 So now we give that power to not only the analysts,
0:08:21 but to business users.
0:08:24 So business users can ask questions of their data, drill in in natural language.
0:08:26 And that’s super powerful.
0:08:29 We’ve been working with a series of companies and like Bayer,
0:08:34 they’re finding it very valuable to give democratized access to that kind of data.
0:08:35 Right.
0:08:39 Another interesting one is we’re working with kind of Siemens.
0:08:41 They have a large research organization.
0:08:45 They’ve just recently built a research chatbot
0:08:50 that has 700,000 pages of research that’s now unlocked
0:08:52 and available for this research organization.
0:08:52 Wow.
0:08:57 So, you know, instead of kind of figuring out how and where to get that data
0:09:01 to continue your research now, the team feels a lot more productive.
0:09:04 How many tokens is 700,000 pages?
0:09:05 It’s a lot of tokens, you know.
0:09:06 A lot of tokens.
0:09:08 A lot of tokens.
0:09:12 So when you’re working, and I’m sure the answer is different depending on the customer,
0:09:17 but when you’re working with a customer to do something like take 700,000 pages of documentation
0:09:20 and turn it into something that, you know, the average employee,
0:09:24 the average user can just speak to using natural language.
0:09:30 What’s the process like in terms of what you’re doing on the technical side?
0:09:32 Are you fine tuning the model?
0:09:35 Are you building kind of custom rag pipelines?
0:09:39 And again, you know, I’m sure it’s different with different use cases.
0:09:43 But what are some of the things that Snowflake does with a customer
0:09:48 that, you know, they couldn’t get just by sort of brute force
0:09:51 uploading these documents to a publicly available model?
0:09:57 So when we actually surveyed our customers, as they think about going from, you know,
0:10:01 these demos, proof of concepts to production, usually three big things emerge.
0:10:06 They are concerned about quality hallucinations.
0:10:06 Sure.
0:10:10 The second one is they’re concerned about kind of security of their data,
0:10:13 governance of that system, and finally the cost.
0:10:16 So those are the top three things that always emerge.
0:10:19 And then we try to address these three concerns head on.
0:10:23 Cortex Search is a new product offering that we are just recently releasing.
0:10:30 And we’ve tuned Cortex Search to be the highest quality in terms of a rag solution.
0:10:32 So we implement a custom rag solution.
0:10:36 We have our own embedding model, and we’ve built a hybrid search engine
0:10:38 that can provide high quality.
0:10:42 And we will tune the system so that it knows when not to answer questions,
0:10:44 you know, reducing hallucinations.
0:10:44 Got it.
0:10:49 And hybrid search meaning combines rag functionality with internet search?
0:10:50 Or–
0:10:50 Exactly.
0:10:54 Our hybrid search is basically combining vector search with the traditional keyword
0:10:55 based tech search.
0:10:56 Right, great.
0:10:56 Catch you.
0:10:57 Okay, cool.
0:11:02 On the hallucination point, any interesting learnings or insights when it comes to,
0:11:09 I would assume it’s more sophisticated than manually just line by line telling the model,
0:11:11 you know, don’t say this, don’t say that.
0:11:15 But how do you sort of coax a model into hallucinating less?
0:11:15 Oh, yeah.
0:11:18 So first of all, there is definitely model tuning.
0:11:20 That’s important in this.
0:11:26 But also, we just touched on the hybrid search element.
0:11:30 The nice thing about hybrid search is it can give you meaningful information
0:11:35 about whether the set of documents is relevant to the question.
0:11:36 Okay.
0:11:42 And, you know, usually, LLMs tend to hallucinate when they are not grounded on data.
0:11:46 So the system can know that the match to that question is low.
0:11:49 And rather than trying to answer the question, it should just reject it.
0:11:49 Got it.
0:11:53 The speaking of models, there are a bunch of Snowflake products.
0:11:54 We don’t have time to get into all of them, obviously.
0:11:58 And I’m going to leave room to bring up things that I didn’t ask about.
0:12:00 But I do want to ask you about Arctic.
0:12:05 So Arctic is an LLM that Snowflake built?
0:12:06 Or how would you describe it?
0:12:08 Yes, Arctic is our own language model.
0:12:10 It’s actually a family of language models.
0:12:17 You have the Arctic LLM as well as an embedding model and a document model.
0:12:25 So the LLM is an open source, large language model that is quite unique in its architecture
0:12:32 by combining both a mixture of experts model with a dense architecture.
0:12:36 We were able to have a very efficient and high quality model.
0:12:40 So we focused on what we’re calling enterprise intelligence,
0:12:46 being able to follow instructions, being able to do well in coding and SQL.
0:12:51 And then we were able to achieve the highest benchmarks in these categories
0:12:54 amongst open source models while being incredibly efficient.
0:12:59 We trained Arctic at about one eighth the cost of similar models.
0:13:03 And that means when we train custom models, for instance, for our customers,
0:13:07 we can be very, very cost effective while delivering very high quality.
0:13:10 Understood if these are trade secrets, you don’t want to divulge.
0:13:14 But how did you figure out how to train the model so much more efficiently?
0:13:19 So we actually really pride ourselves in our openness.
0:13:23 We’ve released cookbooks of our not only the model weights,
0:13:27 but also the research insights as well as our kind of data recipes.
0:13:28 Oh, fantastic.
0:13:28 Okay.
0:13:29 All of those are available.
0:13:31 And we’ve shared some of these insights.
0:13:40 It boiled down to having some of the best researchers that have pioneered MOE models,
0:13:43 a mixture of experts models, you know, back in the day,
0:13:48 along with some of the VLLM original team members.
0:13:51 And all working together to develop this architecture.
0:13:52 Very cool.
0:13:55 Again, for folks who might not be familiar and unfamiliar,
0:13:57 but I don’t fully understand how it works.
0:14:01 Someone asked, what is a mixture of experts approach?
0:14:02 What does that mean?
0:14:03 Was it entail?
0:14:05 Why is it different better than other approaches?
0:14:10 There are two major architectures that we’re seeing.
0:14:12 One is what’s called a dense model.
0:14:17 In the dense model, all of the parameters are active and they’re being used
0:14:18 when you’re doing inference.
0:14:21 So also during training, all of these parameters are active.
0:14:26 Mixture of experts model has larger set of parameters,
0:14:28 but only a subset of them gets used.
0:14:32 So you have different number of experts essentially that are,
0:14:35 you know, that are getting activated to answer one question.
0:14:35 Right.
0:14:37 That tends to be very efficient.
0:14:38 Right, got it.
0:14:40 So you can hone in on the accuracy that you’re looking for,
0:14:44 but then also it’s more efficient because you’re turning things on and off
0:14:46 as you need them instead of just leaving all the lights on.
0:14:49 Yeah, it’s efficient to train as well as efficient to run inference.
0:14:53 So it tends to be cheaper because it has lower number of active parameters.
0:14:53 Got it.
0:14:58 Any other specific products, innovations that Snowflake has put out
0:15:01 during your time that you’re particularly excited about?
0:15:05 I am excited about our Cortex Analyst product that we just recently announced.
0:15:15 As I was alluding to, there’s a lot of data gem that is locked in in very large amounts.
0:15:22 Bringing, allowing more people to have easy access to that data is really important.
0:15:27 So far, you know, data teams have to run kind of SQL analysis
0:15:30 to get insights from these data sets.
0:15:33 Large language models have been exciting to see,
0:15:36 hey, can we turn language into SQL?
0:15:38 And turns out that is a really, really difficult task
0:15:42 because, you know, the world of data tends to be massive.
0:15:47 You know, you have tens of thousands of tables, hundreds of thousands of columns,
0:15:50 really complex abbreviations of column names and so forth.
0:15:55 So we work really hard to have the world’s best text to SQL experience.
0:15:56 And then we’ve achieved it.
0:16:01 So we have the state of the art when it comes to text to SQL.
0:16:05 And that now becomes available to business users
0:16:07 who can now ask natural language questions.
0:16:09 And then we turn that into SQL.
0:16:11 We run that SQL and generate an answer.
0:16:15 So something like, how is my revenue growing in this region for this product
0:16:18 now becomes an easy question to ask for a business user.
0:16:19 Right, fantastic.
0:16:22 My guest today is Barish Gultekin.
0:16:27 Barish is the head of AI at Snowflake, the AI data cloud.
0:16:30 And we’ve been talking about, well, the role of data.
0:16:32 We always talk about the role of data on this show
0:16:33 because data is what fuels AI.
0:16:39 But in particular, Snowflake’s approaches to everything
0:16:43 from fine-tuning customer data, unlocking structured and unstructured data
0:16:47 so that the customers, the developers, folks on Snowflake’s ends
0:16:50 can turn the data into insights and applications.
0:16:54 And then also some of the innovative approaches that Snowflake has taken
0:16:58 to fine-tuning models, building models, converting text to SQL
0:16:59 as we were just talking about.
0:17:02 Let’s switch gears for a second, Barish, if we can,
0:17:06 and talk about your background in AI at Google, even before Google.
0:17:10 Have you always been a data nerd to put it that way?
0:17:12 You’ve always been interested in data, computer science.
0:17:14 Where did your journey start?
0:17:19 Yeah, I have been, actually, before it was cool to say AI.
0:17:24 I started this journey at Google a long time ago.
0:17:32 And at some point around 2010, 2011, we started building Google now.
0:17:35 And it was the traditional 20% project at Google,
0:17:39 where we basically thought our phones should do better than what they do today.
0:17:42 They should be able to give us the information we need when we need it.
0:17:47 So it was this proactive assistant, and we built that product.
0:17:52 Even though, at the time, the technology wasn’t quite there where it is now,
0:17:55 we were able to give helpful information like,
0:17:57 “Hey, there’s traffic on your commute,
0:18:01 and you should just take this alternate route or your flight is delayed.”
0:18:03 And all of that information felt magical
0:18:07 because of bringing context with kind of prediction.
0:18:08 Right, right, right.
0:18:10 Even though it was a set of heuristics,
0:18:12 it felt like, “Oh, there’s something intelligent here.”
0:18:15 And that was the beginning, and I loved Indians.
0:18:21 And then after that, I worked on Google Assistant.
0:18:25 And Google Assistant is, again, exciting because it understands language,
0:18:28 it can respond in natural language.
0:18:31 Early on, it is just a series of use cases
0:18:33 that are just coded one by one.
0:18:34 Right, right.
0:18:36 And now we’re at a point where, finally,
0:18:38 computers can understand language,
0:18:41 and you don’t have to kind of code each use case one at a time.
0:18:41 Right, right, right.
0:18:42 Exactly.
0:18:46 When you’re working on all the things that you work on at this scale that you do now,
0:18:49 or you did at Google, and now you do at Snowflake,
0:18:54 how much do you trust the answers given by a generative AI model?
0:18:58 And what sort of your own, I don’t know if it’s a workflow,
0:18:59 so much as just kind of a mental,
0:19:05 like, do you go back and verify results that you’re not sure about?
0:19:07 Have you kind of gotten to, you know,
0:19:11 do you have a feel for when something’s grounded versus hallucinated?
0:19:12 And this is a little bit more of a,
0:19:16 I don’t know, metaphysical question, perhaps, than the other stuff.
0:19:19 But I’m just wondering, someone with as much experience
0:19:22 and day-to-day working with this stuff as much as you do,
0:19:26 what your kind of feel is for where the systems are right now?
0:19:29 I try to see where we are.
0:19:34 This generative ability, the creativity, if you will,
0:19:36 a feature to a certain degree, right?
0:19:40 So the types of use cases that are great are,
0:19:43 when you ask the language models,
0:19:45 to generate something, to generate content.
0:19:48 And if my question is a factual question,
0:19:51 then I know to be careful.
0:19:54 But if it’s more of a help me brainstorm,
0:19:57 let’s think about this, how do you say this differently?
0:20:00 Those are the things where you’re leaning into the creativity,
0:20:03 into the hallucination as a feature, if you will.
0:20:04 Right, right, right.
0:20:08 And so then how does that translate to enterprise customers you’re working with?
0:20:10 I would imagine there’s a, you know,
0:20:15 they sort of run the gamut from folks who are really excited to work with this,
0:20:18 who maybe you have some customers who are a little more reticent,
0:20:20 but feel like they should be,
0:20:23 how do they relate to this whole notion of, you know,
0:20:26 hallucinations being part of the deal?
0:20:31 I think it’s incredibly important to know that,
0:20:35 right now, where the technology is, we need to build systems.
0:20:38 And these systems need to have grounding in them.
0:20:42 So work hard to provide technology to help our customers,
0:20:45 to make their systems, their products, their chatbots,
0:20:48 a lot more grounded with the data that they provide.
0:20:50 If an LLM is provided with grounding,
0:20:54 if an LLM is provided with the data, it does not hallucinate, right?
0:20:58 Only when there is lack of information, it then kind of makes it up sometimes.
0:21:00 So we work hard on those solutions.
0:21:01 We work with our customers.
0:21:05 We also want to make sure our customers are able to evaluate these models.
0:21:08 We’ve acquired a company called Truera just recently.
0:21:14 Truera is a company that focuses on ML, LLM observability,
0:21:19 being able to evaluate whether a chatbot that’s built is grounded,
0:21:23 whether the quality is right, whether the cost is, you know, how they want it.
0:21:26 So those are the technologies, tools that we’d like to offer to our customers.
0:21:28 And we work closely with them.
0:21:31 Right. And so along those lines, so that was an acquisition obviously,
0:21:36 but Snowflake’s partnering, you mentioned that kind of your company’s openness and transparency.
0:21:38 And there seems to be a spirit of that.
0:21:45 And perhaps because everyone’s laser focused now on this, you know, frontier technology that
0:21:50 inherently we’re all sort of figuring it out, whether as a user or developer to some degree.
0:21:56 What’s the nature of some of the other partnerships or sort of what’s Snowflake’s
0:22:00 role in working and partnering with some of the other tech giants and companies out there
0:22:03 working at the leading edge of AI and ML?
0:22:08 Yeah, we have very close partnerships with NVIDIA, with Meta,
0:22:12 as well as Mistral and Raka, you know, the large language model providers
0:22:14 who’ve invested in some of them as well.
0:22:19 We basically see our platform as a way where we provide choice,
0:22:24 but we work very closely with our partners in kind of helping us build specific solutions
0:22:29 when it comes to kind of making sure that our RAG solutions are grounded,
0:22:33 making sure that we have the world’s best-class text-to-sequel experience
0:22:35 that requires partnerships.
0:22:36 We work very closely with our partners.
0:22:40 So in terms of openness, openness matters for many of our customers,
0:22:45 understanding what kind of data was used to train a model is important.
0:22:52 We also partner with some of our providers to have high-quality proprietary models as well.
0:22:58 Snowflake, as I understand it, is a global company, has more than around 40 offices worldwide.
0:22:58 That’s correct, yeah.
0:23:00 Right, and how many data centers do you run?
0:23:04 So we’re running on the three clouds, AWS.
0:23:06 Okay, got it.
0:23:06 ZP, yep.
0:23:07 Right, right.
0:23:11 So kind of looking at the present, but looking forward a little bit.
0:23:12 I’m going to put you on the spot now, like I said I would.
0:23:15 We touched on some of these things during the conversation,
0:23:23 but are there major trends you’re seeing in business adoption and sort of real-world use cases
0:23:30 that your customers, Snowflake’s customers are adopting now, or really are there trends
0:23:36 in areas they’re really interested in exploring with the power of AI?
0:23:42 And then kind of piggybacking on that, where do you see the industry headed?
0:23:46 Sort of in broad strokes, if you like, over the next, say, three to five years.
0:23:49 So we’re seeing a lot of exciting use cases.
0:23:55 I’ve mentioned a couple of them, but our partners are, again, building production use cases.
0:24:02 Some of them are our bread and butter, running large-scale analysis across their data inside
0:24:02 Snowflake.
0:24:08 So we’re seeing a lot of super simple, just using English language to be able to create
0:24:13 categorization, extract information, and kind of make sense of a lot of data.
0:24:21 For instance, one of our customers, Sigma, that’s a BI provider, are running analysis on
0:24:26 sales logs from sales transcripts, sales calls.
0:24:29 And if you come out understanding, why do we win?
0:24:31 Why do we lose deals?
0:24:37 So being able to run this now across a large data set of all sales calls in a period of time
0:24:39 is as simple as writing English language.
0:24:40 So that’s fascinating to me.
0:24:41 That’s amazing, right?
0:24:42 Yeah.
0:24:47 And then, as I mentioned, of course, the bread and butter, high-quality chatbots,
0:24:50 as well as being able to talk to your structured data for BI-type use cases.
0:24:53 Those are the use cases that we’re seeing.
0:24:58 What I’m seeing, of course, the world of AI evolves incredibly fast.
0:25:01 Week over week, we get a new announcement, something new, exciting.
0:25:02 I know.
0:25:05 Three to five years, I should have said months or weeks even.
0:25:06 That’s on me.
0:25:08 Also, it feels like a year.
0:25:14 Of course, the next big phase that is coming that’s already getting traction
0:25:15 is the world of agents.
0:25:22 So how many are we seeing the ability to answer questions by looking at documentation,
0:25:23 but being able to take action?
0:25:23 Right.
0:25:28 And these agents are, this agentic systems are coming, this ability to reason,
0:25:33 this ability to self-heal, the ability to take action for agents to talk to each other,
0:25:37 collaborate, and that is the next evolution of the technology.
0:25:37 Right.
0:25:43 Are there any agentic frameworks on Snowflake that customers can access?
0:25:44 So very soon.
0:25:49 Right now, the agentic systems that we’ve built are behind the scenes.
0:25:56 The Text2SQL BI experience uses a series of tools to deliver the product.
0:25:56 Gotcha.
0:25:58 So we will make that available to our customers.
0:25:59 Right.
0:25:59 Very cool.
0:26:03 Looking back on your time at Snowflake, further back,
0:26:06 I won’t prescribe time frame zero.
0:26:07 I learned my lesson.
0:26:13 Is there a particular story, moment, something that springs to mind as an important,
0:26:20 perhaps unexpected learning that’s kind of really impacted how you view your work
0:26:22 and the landscape today?
0:26:27 Maybe a problem that the solution turned out to be something unexpected
0:26:30 or something you thought was going to be hard and turned out to be simple.
0:26:33 So I’ll give two examples.
0:26:38 One is very early on, as we were building Cortex, we talked to a customer.
0:26:41 This customer is a long-time Snowflake customer.
0:26:46 They’ve built a pipeline to take their data out and then get it processed by an LLM running
0:26:49 elsewhere and then get back.
0:26:52 And of course, that pipeline took two months or so to build,
0:26:56 and it was quite expensive to maintain and they were concerned about it.
0:27:02 And our early prototype was able to replace the full thing with literally a single line of code.
0:27:06 So we are on to something.
0:27:12 When you bring compute, when you bring AI right next to where the data is,
0:27:14 makes everything a lot simpler.
0:27:17 And when it’s a lot simpler, it just unlocks a lot of usage.
0:27:20 So I’m super excited about just ease of use, simplicity.
0:27:27 The other example is just realizing how kind of demos are easy to build,
0:27:28 but production systems are hard.
0:27:36 You have, especially when it comes to working with structured data, generating SQL is difficult.
0:27:42 So we work really hard on how do we build a system that together creates a very,
0:27:43 very high quality response.
0:27:49 When you’re essentially asking revenue questions, it’s not enough to be 80% accurate.
0:27:53 So that’s another big important area that we focused on.
0:27:55 All right, getting into the wrap up here.
0:28:02 I always ask this question anyway, but I have a child who seems to be getting older every year,
0:28:07 and now he’s in high school, interested in computers, computer science, physical science.
0:28:12 What advice would you give to either a young person kind of looking out,
0:28:16 maybe on the edge of graduating, a little older, maybe graduating college,
0:28:21 or maybe somebody who’s older and is just interested in AI and sort of keeps hearing
0:28:26 the things that we’ve been talking about, which is both that things are changing so fast,
0:28:30 but also, there are things that we can do in the present moment,
0:28:33 and still plenty of problems to be solved.
0:28:35 So where should they go?
0:28:38 Is studying computer science still a viable path?
0:28:44 Is it better to just dive right into the work world and start working on,
0:28:50 as you said, prototyping is one thing, building a production scale system is quite something else.
0:28:56 What’s the advice that you give to young people or maybe older people looking to
0:28:58 dive further into AI?
0:29:05 So I think everyone has their own unique path, and everyone is drawn to something,
0:29:09 and it’s important to be able to connect to what you’re drawn to,
0:29:12 and it’ll be different for different people.
0:29:16 But I just focus on that, just listening to that inner voice,
0:29:20 which is hard to listen to sometimes, especially given there’s so much noise out there.
0:29:24 But I will say, even though AI sounds intimidating,
0:29:29 or there is this kind of artificial intelligence that sounds very complex,
0:29:34 and it is complex when you start going down the rabbit hole and start doing your research,
0:29:40 however, the use of the AI is going to unlock, it’s incredibly easy.
0:29:45 All of these systems are now an API away, and they’re probably powerful.
0:29:52 So I think creativity is going to determine all sorts of super interesting technologies
0:29:53 to be built next.
0:29:57 So I would say, don’t be intimidated with technology, just dive right in,
0:30:02 and it’s incredibly easy to use, and really looking forward to what’s
0:30:04 to come in the next two years or so.
0:30:07 Love it, love the optimism, more audio only, which is a shame,
0:30:11 because your face lit up, smile like I did when you were talking about that.
0:30:16 Bearish, you alluded earlier to cookbooks and other resources that Snowflake makes available.
0:30:19 Maybe we can divvy this up into two parts.
0:30:23 Potential customers who want to learn more about what Snowflake does,
0:30:26 what the offerings are, how to maybe engage with you.
0:30:30 And then folks, practitioners working in AI wanting to learn more about,
0:30:33 you know, what Snowflake’s been doing, research,
0:30:37 some of the techniques we talked about, where can people go online to learn more?
0:30:43 So our website, snowflake.com, if you are trying to figure out how do I use AI
0:30:48 just in seconds and bring my data, analyze my data, we have a solution for you.
0:30:49 Thanks.
0:30:51 So snowflake.com is the place.
0:30:52 Perfect.
0:30:53 Bearish, thank you.
0:30:54 This was great.
0:30:58 As with many of these conversations of these days, I feel like this was kind of the warm-up,
0:31:02 and we’ll have to get back in touch down the line to really dig into where things are headed.
0:31:07 But the Snowflake story, you know, is a great one, and it seems like it’s just getting started.
0:31:10 So congratulations on the work so far.
0:31:12 All the best to you going forward.
0:31:15 And, you know, look out for my unnamed friend I mentioned earlier,
0:31:16 if you see them around campus.
0:31:17 That sounds great.
0:31:18 Thanks for having me.
0:31:28 You know.
0:31:38 [Music]
0:31:48 [Music]
0:32:08 [Music]
0:32:18 [BLANK_AUDIO]
0:00:13 >> Hello, and welcome to the Nvidia AI podcast.
0:00:15 I’m your host, Noah Kravitz.
0:00:18 Today, I’m joined by Barasch Kultekin,
0:00:20 the head of AI at Snowflake.
0:00:22 At Snowflake, he’s driving the development of
0:00:24 cutting-edge AI and ML products,
0:00:27 including Snowflake Cortex AI,
0:00:29 and ARTIC, their new foundational model.
0:00:33 But Barasch has also had a remarkable journey through AI himself,
0:00:35 having co-founded Google Now and
0:00:38 led AI efforts for Google Assistant.
0:00:40 So we’ve got a ton to talk about background,
0:00:42 the present at Snowflake, and of course,
0:00:44 we’re not going to let Barasch off the hook without asking
0:00:49 him to opine about the future of AI and the hot topic today.
0:00:51 What’s going to happen to all of us in the age of AI?
0:00:53 So I can’t wait to get into it.
0:00:55 Barasch, thank you so much for joining
0:00:57 the Nvidia AI podcast and welcome.
0:00:58 Thanks a lot, Noah.
0:01:01 I sometimes wonder if the pods we tape on a Friday,
0:01:03 let alone today, a Friday afternoon,
0:01:05 have a little bit looser of a feel than some of the other ones.
0:01:08 So we’ll have to do an exit survey at the end here.
0:01:10 But I’m super excited to talk to you.
0:01:12 We were talking offhand.
0:01:13 I’ve been tracking Snowflake,
0:01:18 especially over the past few years as interest in AI and data has exploded.
0:01:21 A good friend of mine has been with Snowflake now for a few years,
0:01:23 so I’ve been tracking her journey as well.
0:01:25 So I’m thrilled to have you here, Barasch,
0:01:28 and to learn more about Snowflake and your own journey.
0:01:32 Maybe we’ll start with Snowflake and we can get into your background as we go.
0:01:36 Can you start by telling the audience a little bit about what Snowflake is,
0:01:39 how long the company’s been around, what you do,
0:01:45 and obviously your role in the burgeoning AI explosion that we’re all part of?
0:01:48 Of course. So Snowflake is the AI data cloud.
0:01:51 We started the journey over a decade ago
0:01:55 focusing on how do we make data a lot more easily accessible,
0:02:00 how do we make data processing a lot more easily accessible to companies.
0:02:02 As the data volumes were growing,
0:02:07 there was a big innovation of separating the storage of data from compute
0:02:12 that allowed a massive unlock in the data space.
0:02:17 And since then, we’ve been evolving from a provider of a data warehouse
0:02:21 to a data cloud where that data can be easily shared.
0:02:23 And now we’re in the third phase of that journey
0:02:29 where you can now unlock a lot more value of that data with AI for a lot more users.
0:02:32 Right. Maybe just to set baselines,
0:02:36 can you quickly just talk about terms like data warehouse, data lake,
0:02:40 the AI data cloud, as you describe Snowflake,
0:02:44 and what those terms mean and maybe how they’ve evolved a little bit?
0:02:48 Of course, customers have a lot of structured data.
0:02:52 They need to have a way to bring all of that data
0:02:57 and then manage it, govern it, be able to run analysis on it.
0:03:02 So data warehouse allows our customers to very efficiently run
0:03:06 massive scale analysis across large volumes of data.
0:03:07 Right.
0:03:11 And that’s for structured data for tables.
0:03:17 With data lake, you expand that to bringing in more unstructured data
0:03:19 as structured data into the mix.
0:03:20 Right.
0:03:24 And so there’s issues at hand of the way that the data has stored itself,
0:03:27 obviously, the physical media it’s stored on.
0:03:31 But then, as with all things technology and certainly all things AI,
0:03:34 machine learning, deep learning, the software plays a role,
0:03:36 the interconnectivity plays a role.
0:03:39 Obviously, if it’s a data cloud, there’s all of the…
0:03:43 Everything that happens between your cloud and my workstation,
0:03:44 wherever in the world I am.
0:03:46 So there’s a lot going on.
0:03:50 So maybe we can dive in a little bit to some of the breakthroughs
0:03:54 that Snowflake has ushered in, is currently working on.
0:03:56 And kind of how data plays a role,
0:04:02 what the data cloud’s role is in the modern ALML stack, if you will.
0:04:03 So let’s talk a little better,
0:04:09 or if you would talk a little bit about how Snowflake works with enterprises
0:04:12 in helping them unlock their data, as you put it.
0:04:13 Yeah.
0:04:17 So first of all, we’re super excited about what AI is capable of,
0:04:18 what AI is bringing.
0:04:20 And of course, when we think about AI,
0:04:25 there is no AI strategy without a data strategy, is what they say.
0:04:27 AI is fueled with data.
0:04:32 So what our customers are interested in is they are trusting Snowflake with their data.
0:04:36 How can they now get the most out of this data
0:04:39 by bringing AI compute right next to where the data is?
0:04:43 So believe deeply in being able to bring compute to data
0:04:48 versus bringing massive amounts of data out into where the compute is.
0:04:52 So the way we work with our customers is that we built a AI platform.
0:04:58 And with this AI platform, our customers can run natural language analysis,
0:05:02 can build chatbots, can talk to their data in natural language.
0:05:06 And we make this platform available, and our customers are using it
0:05:08 to build all sorts of AI applications.
0:05:10 And so the customer stores their data with you,
0:05:13 but then also the training, the inference,
0:05:17 all the compute operations are done on your side as well.
0:05:18 That’s right, exactly.
0:05:21 So basically, our AI platform runs fully inside Snowflake.
0:05:25 So for Snowflake, it’s really important to govern the data,
0:05:28 as well as AI that’s running.
0:05:30 So we run everything inside Snowflake.
0:05:33 So maybe you can get into a little bit of some of the product offerings.
0:05:36 Do we want to start with Snowflake Cortex?
0:05:40 Yeah, so Snowflake Cortex is our managed service.
0:05:44 It is our offering where we’re running a series of large language models
0:05:45 inside Snowflake.
0:05:49 Our customers have super easy access to these large language models.
0:05:55 But the way we think about AI is we want to make AI easy, efficient, and trusted.
0:05:57 With Cortex, it is incredibly easy
0:06:00 because AI is running right next to where the data is.
0:06:02 So our customers don’t have to build data pipelines
0:06:06 to manage data in multiple places and govern it.
0:06:08 We were running it in a secure environment
0:06:11 and we have a series of very efficient models
0:06:13 from our own to a series of partner models.
0:06:14 We make that available.
0:06:17 And then Cortex also makes it very easy for our customers
0:06:21 to build, talk to my data experiences, if you will,
0:06:26 build chatbots, both for documents as well as for structured data.
0:06:30 What are some of the use cases that are kind of most popular, most seen?
0:06:35 I’ve heard people talk about, we’re recording in late June of 2024.
0:06:41 And I’ve heard people reference 2023 as being the year that generative AI
0:06:45 took over all the headlines and everybody was talking a lot,
0:06:47 but not really sure what to do.
0:06:53 And 2024 maybe being the year that businesses start to actually
0:06:57 develop applications or use applications others have developed,
0:07:02 but really start to do things using leveraging AI on their own data
0:07:07 to whatever it is, improve processes, try new ways of working,
0:07:08 all that kind of stuff.
0:07:12 What are you seeing on your end is the things that customers
0:07:15 and enterprise customers, other snowflake customers,
0:07:17 are interested in doing and then maybe on the flip side,
0:07:21 some of the things that, I don’t know, they’re concerned about
0:07:25 or don’t quite understand or still trying to kind of wrap
0:07:26 their collective heads around.
0:07:28 Yeah, so I agree with you.
0:07:32 In 2023 was, I’d say, the year of proof of concepts.
0:07:34 Right, well said.
0:07:38 We got their hands on AI and then started building demos.
0:07:43 And this year we’re starting to see these turn into real production use cases.
0:07:46 I can give a couple of examples.
0:07:50 We’re working with global pharmaceutical company Bayer
0:07:54 in building an experience where Bayer’s internal teams
0:07:56 and sales organizations, marketing organizations,
0:07:59 can ask questions of their structured data.
0:08:03 So Bayer believes that dashboards can only take them so far.
0:08:05 Dashboards tend to be very rigid.
0:08:08 And then first thing that happens when you see a dashboard
0:08:11 is you have three questions, four questions where you want to drill in
0:08:15 and figure out why something isn’t the way you want it to be or where.
0:08:19 So now we give that power to not only the analysts,
0:08:21 but to business users.
0:08:24 So business users can ask questions of their data, drill in in natural language.
0:08:26 And that’s super powerful.
0:08:29 We’ve been working with a series of companies and like Bayer,
0:08:34 they’re finding it very valuable to give democratized access to that kind of data.
0:08:35 Right.
0:08:39 Another interesting one is we’re working with kind of Siemens.
0:08:41 They have a large research organization.
0:08:45 They’ve just recently built a research chatbot
0:08:50 that has 700,000 pages of research that’s now unlocked
0:08:52 and available for this research organization.
0:08:52 Wow.
0:08:57 So, you know, instead of kind of figuring out how and where to get that data
0:09:01 to continue your research now, the team feels a lot more productive.
0:09:04 How many tokens is 700,000 pages?
0:09:05 It’s a lot of tokens, you know.
0:09:06 A lot of tokens.
0:09:08 A lot of tokens.
0:09:12 So when you’re working, and I’m sure the answer is different depending on the customer,
0:09:17 but when you’re working with a customer to do something like take 700,000 pages of documentation
0:09:20 and turn it into something that, you know, the average employee,
0:09:24 the average user can just speak to using natural language.
0:09:30 What’s the process like in terms of what you’re doing on the technical side?
0:09:32 Are you fine tuning the model?
0:09:35 Are you building kind of custom rag pipelines?
0:09:39 And again, you know, I’m sure it’s different with different use cases.
0:09:43 But what are some of the things that Snowflake does with a customer
0:09:48 that, you know, they couldn’t get just by sort of brute force
0:09:51 uploading these documents to a publicly available model?
0:09:57 So when we actually surveyed our customers, as they think about going from, you know,
0:10:01 these demos, proof of concepts to production, usually three big things emerge.
0:10:06 They are concerned about quality hallucinations.
0:10:06 Sure.
0:10:10 The second one is they’re concerned about kind of security of their data,
0:10:13 governance of that system, and finally the cost.
0:10:16 So those are the top three things that always emerge.
0:10:19 And then we try to address these three concerns head on.
0:10:23 Cortex Search is a new product offering that we are just recently releasing.
0:10:30 And we’ve tuned Cortex Search to be the highest quality in terms of a rag solution.
0:10:32 So we implement a custom rag solution.
0:10:36 We have our own embedding model, and we’ve built a hybrid search engine
0:10:38 that can provide high quality.
0:10:42 And we will tune the system so that it knows when not to answer questions,
0:10:44 you know, reducing hallucinations.
0:10:44 Got it.
0:10:49 And hybrid search meaning combines rag functionality with internet search?
0:10:50 Or–
0:10:50 Exactly.
0:10:54 Our hybrid search is basically combining vector search with the traditional keyword
0:10:55 based tech search.
0:10:56 Right, great.
0:10:56 Catch you.
0:10:57 Okay, cool.
0:11:02 On the hallucination point, any interesting learnings or insights when it comes to,
0:11:09 I would assume it’s more sophisticated than manually just line by line telling the model,
0:11:11 you know, don’t say this, don’t say that.
0:11:15 But how do you sort of coax a model into hallucinating less?
0:11:15 Oh, yeah.
0:11:18 So first of all, there is definitely model tuning.
0:11:20 That’s important in this.
0:11:26 But also, we just touched on the hybrid search element.
0:11:30 The nice thing about hybrid search is it can give you meaningful information
0:11:35 about whether the set of documents is relevant to the question.
0:11:36 Okay.
0:11:42 And, you know, usually, LLMs tend to hallucinate when they are not grounded on data.
0:11:46 So the system can know that the match to that question is low.
0:11:49 And rather than trying to answer the question, it should just reject it.
0:11:49 Got it.
0:11:53 The speaking of models, there are a bunch of Snowflake products.
0:11:54 We don’t have time to get into all of them, obviously.
0:11:58 And I’m going to leave room to bring up things that I didn’t ask about.
0:12:00 But I do want to ask you about Arctic.
0:12:05 So Arctic is an LLM that Snowflake built?
0:12:06 Or how would you describe it?
0:12:08 Yes, Arctic is our own language model.
0:12:10 It’s actually a family of language models.
0:12:17 You have the Arctic LLM as well as an embedding model and a document model.
0:12:25 So the LLM is an open source, large language model that is quite unique in its architecture
0:12:32 by combining both a mixture of experts model with a dense architecture.
0:12:36 We were able to have a very efficient and high quality model.
0:12:40 So we focused on what we’re calling enterprise intelligence,
0:12:46 being able to follow instructions, being able to do well in coding and SQL.
0:12:51 And then we were able to achieve the highest benchmarks in these categories
0:12:54 amongst open source models while being incredibly efficient.
0:12:59 We trained Arctic at about one eighth the cost of similar models.
0:13:03 And that means when we train custom models, for instance, for our customers,
0:13:07 we can be very, very cost effective while delivering very high quality.
0:13:10 Understood if these are trade secrets, you don’t want to divulge.
0:13:14 But how did you figure out how to train the model so much more efficiently?
0:13:19 So we actually really pride ourselves in our openness.
0:13:23 We’ve released cookbooks of our not only the model weights,
0:13:27 but also the research insights as well as our kind of data recipes.
0:13:28 Oh, fantastic.
0:13:28 Okay.
0:13:29 All of those are available.
0:13:31 And we’ve shared some of these insights.
0:13:40 It boiled down to having some of the best researchers that have pioneered MOE models,
0:13:43 a mixture of experts models, you know, back in the day,
0:13:48 along with some of the VLLM original team members.
0:13:51 And all working together to develop this architecture.
0:13:52 Very cool.
0:13:55 Again, for folks who might not be familiar and unfamiliar,
0:13:57 but I don’t fully understand how it works.
0:14:01 Someone asked, what is a mixture of experts approach?
0:14:02 What does that mean?
0:14:03 Was it entail?
0:14:05 Why is it different better than other approaches?
0:14:10 There are two major architectures that we’re seeing.
0:14:12 One is what’s called a dense model.
0:14:17 In the dense model, all of the parameters are active and they’re being used
0:14:18 when you’re doing inference.
0:14:21 So also during training, all of these parameters are active.
0:14:26 Mixture of experts model has larger set of parameters,
0:14:28 but only a subset of them gets used.
0:14:32 So you have different number of experts essentially that are,
0:14:35 you know, that are getting activated to answer one question.
0:14:35 Right.
0:14:37 That tends to be very efficient.
0:14:38 Right, got it.
0:14:40 So you can hone in on the accuracy that you’re looking for,
0:14:44 but then also it’s more efficient because you’re turning things on and off
0:14:46 as you need them instead of just leaving all the lights on.
0:14:49 Yeah, it’s efficient to train as well as efficient to run inference.
0:14:53 So it tends to be cheaper because it has lower number of active parameters.
0:14:53 Got it.
0:14:58 Any other specific products, innovations that Snowflake has put out
0:15:01 during your time that you’re particularly excited about?
0:15:05 I am excited about our Cortex Analyst product that we just recently announced.
0:15:15 As I was alluding to, there’s a lot of data gem that is locked in in very large amounts.
0:15:22 Bringing, allowing more people to have easy access to that data is really important.
0:15:27 So far, you know, data teams have to run kind of SQL analysis
0:15:30 to get insights from these data sets.
0:15:33 Large language models have been exciting to see,
0:15:36 hey, can we turn language into SQL?
0:15:38 And turns out that is a really, really difficult task
0:15:42 because, you know, the world of data tends to be massive.
0:15:47 You know, you have tens of thousands of tables, hundreds of thousands of columns,
0:15:50 really complex abbreviations of column names and so forth.
0:15:55 So we work really hard to have the world’s best text to SQL experience.
0:15:56 And then we’ve achieved it.
0:16:01 So we have the state of the art when it comes to text to SQL.
0:16:05 And that now becomes available to business users
0:16:07 who can now ask natural language questions.
0:16:09 And then we turn that into SQL.
0:16:11 We run that SQL and generate an answer.
0:16:15 So something like, how is my revenue growing in this region for this product
0:16:18 now becomes an easy question to ask for a business user.
0:16:19 Right, fantastic.
0:16:22 My guest today is Barish Gultekin.
0:16:27 Barish is the head of AI at Snowflake, the AI data cloud.
0:16:30 And we’ve been talking about, well, the role of data.
0:16:32 We always talk about the role of data on this show
0:16:33 because data is what fuels AI.
0:16:39 But in particular, Snowflake’s approaches to everything
0:16:43 from fine-tuning customer data, unlocking structured and unstructured data
0:16:47 so that the customers, the developers, folks on Snowflake’s ends
0:16:50 can turn the data into insights and applications.
0:16:54 And then also some of the innovative approaches that Snowflake has taken
0:16:58 to fine-tuning models, building models, converting text to SQL
0:16:59 as we were just talking about.
0:17:02 Let’s switch gears for a second, Barish, if we can,
0:17:06 and talk about your background in AI at Google, even before Google.
0:17:10 Have you always been a data nerd to put it that way?
0:17:12 You’ve always been interested in data, computer science.
0:17:14 Where did your journey start?
0:17:19 Yeah, I have been, actually, before it was cool to say AI.
0:17:24 I started this journey at Google a long time ago.
0:17:32 And at some point around 2010, 2011, we started building Google now.
0:17:35 And it was the traditional 20% project at Google,
0:17:39 where we basically thought our phones should do better than what they do today.
0:17:42 They should be able to give us the information we need when we need it.
0:17:47 So it was this proactive assistant, and we built that product.
0:17:52 Even though, at the time, the technology wasn’t quite there where it is now,
0:17:55 we were able to give helpful information like,
0:17:57 “Hey, there’s traffic on your commute,
0:18:01 and you should just take this alternate route or your flight is delayed.”
0:18:03 And all of that information felt magical
0:18:07 because of bringing context with kind of prediction.
0:18:08 Right, right, right.
0:18:10 Even though it was a set of heuristics,
0:18:12 it felt like, “Oh, there’s something intelligent here.”
0:18:15 And that was the beginning, and I loved Indians.
0:18:21 And then after that, I worked on Google Assistant.
0:18:25 And Google Assistant is, again, exciting because it understands language,
0:18:28 it can respond in natural language.
0:18:31 Early on, it is just a series of use cases
0:18:33 that are just coded one by one.
0:18:34 Right, right.
0:18:36 And now we’re at a point where, finally,
0:18:38 computers can understand language,
0:18:41 and you don’t have to kind of code each use case one at a time.
0:18:41 Right, right, right.
0:18:42 Exactly.
0:18:46 When you’re working on all the things that you work on at this scale that you do now,
0:18:49 or you did at Google, and now you do at Snowflake,
0:18:54 how much do you trust the answers given by a generative AI model?
0:18:58 And what sort of your own, I don’t know if it’s a workflow,
0:18:59 so much as just kind of a mental,
0:19:05 like, do you go back and verify results that you’re not sure about?
0:19:07 Have you kind of gotten to, you know,
0:19:11 do you have a feel for when something’s grounded versus hallucinated?
0:19:12 And this is a little bit more of a,
0:19:16 I don’t know, metaphysical question, perhaps, than the other stuff.
0:19:19 But I’m just wondering, someone with as much experience
0:19:22 and day-to-day working with this stuff as much as you do,
0:19:26 what your kind of feel is for where the systems are right now?
0:19:29 I try to see where we are.
0:19:34 This generative ability, the creativity, if you will,
0:19:36 a feature to a certain degree, right?
0:19:40 So the types of use cases that are great are,
0:19:43 when you ask the language models,
0:19:45 to generate something, to generate content.
0:19:48 And if my question is a factual question,
0:19:51 then I know to be careful.
0:19:54 But if it’s more of a help me brainstorm,
0:19:57 let’s think about this, how do you say this differently?
0:20:00 Those are the things where you’re leaning into the creativity,
0:20:03 into the hallucination as a feature, if you will.
0:20:04 Right, right, right.
0:20:08 And so then how does that translate to enterprise customers you’re working with?
0:20:10 I would imagine there’s a, you know,
0:20:15 they sort of run the gamut from folks who are really excited to work with this,
0:20:18 who maybe you have some customers who are a little more reticent,
0:20:20 but feel like they should be,
0:20:23 how do they relate to this whole notion of, you know,
0:20:26 hallucinations being part of the deal?
0:20:31 I think it’s incredibly important to know that,
0:20:35 right now, where the technology is, we need to build systems.
0:20:38 And these systems need to have grounding in them.
0:20:42 So work hard to provide technology to help our customers,
0:20:45 to make their systems, their products, their chatbots,
0:20:48 a lot more grounded with the data that they provide.
0:20:50 If an LLM is provided with grounding,
0:20:54 if an LLM is provided with the data, it does not hallucinate, right?
0:20:58 Only when there is lack of information, it then kind of makes it up sometimes.
0:21:00 So we work hard on those solutions.
0:21:01 We work with our customers.
0:21:05 We also want to make sure our customers are able to evaluate these models.
0:21:08 We’ve acquired a company called Truera just recently.
0:21:14 Truera is a company that focuses on ML, LLM observability,
0:21:19 being able to evaluate whether a chatbot that’s built is grounded,
0:21:23 whether the quality is right, whether the cost is, you know, how they want it.
0:21:26 So those are the technologies, tools that we’d like to offer to our customers.
0:21:28 And we work closely with them.
0:21:31 Right. And so along those lines, so that was an acquisition obviously,
0:21:36 but Snowflake’s partnering, you mentioned that kind of your company’s openness and transparency.
0:21:38 And there seems to be a spirit of that.
0:21:45 And perhaps because everyone’s laser focused now on this, you know, frontier technology that
0:21:50 inherently we’re all sort of figuring it out, whether as a user or developer to some degree.
0:21:56 What’s the nature of some of the other partnerships or sort of what’s Snowflake’s
0:22:00 role in working and partnering with some of the other tech giants and companies out there
0:22:03 working at the leading edge of AI and ML?
0:22:08 Yeah, we have very close partnerships with NVIDIA, with Meta,
0:22:12 as well as Mistral and Raka, you know, the large language model providers
0:22:14 who’ve invested in some of them as well.
0:22:19 We basically see our platform as a way where we provide choice,
0:22:24 but we work very closely with our partners in kind of helping us build specific solutions
0:22:29 when it comes to kind of making sure that our RAG solutions are grounded,
0:22:33 making sure that we have the world’s best-class text-to-sequel experience
0:22:35 that requires partnerships.
0:22:36 We work very closely with our partners.
0:22:40 So in terms of openness, openness matters for many of our customers,
0:22:45 understanding what kind of data was used to train a model is important.
0:22:52 We also partner with some of our providers to have high-quality proprietary models as well.
0:22:58 Snowflake, as I understand it, is a global company, has more than around 40 offices worldwide.
0:22:58 That’s correct, yeah.
0:23:00 Right, and how many data centers do you run?
0:23:04 So we’re running on the three clouds, AWS.
0:23:06 Okay, got it.
0:23:06 ZP, yep.
0:23:07 Right, right.
0:23:11 So kind of looking at the present, but looking forward a little bit.
0:23:12 I’m going to put you on the spot now, like I said I would.
0:23:15 We touched on some of these things during the conversation,
0:23:23 but are there major trends you’re seeing in business adoption and sort of real-world use cases
0:23:30 that your customers, Snowflake’s customers are adopting now, or really are there trends
0:23:36 in areas they’re really interested in exploring with the power of AI?
0:23:42 And then kind of piggybacking on that, where do you see the industry headed?
0:23:46 Sort of in broad strokes, if you like, over the next, say, three to five years.
0:23:49 So we’re seeing a lot of exciting use cases.
0:23:55 I’ve mentioned a couple of them, but our partners are, again, building production use cases.
0:24:02 Some of them are our bread and butter, running large-scale analysis across their data inside
0:24:02 Snowflake.
0:24:08 So we’re seeing a lot of super simple, just using English language to be able to create
0:24:13 categorization, extract information, and kind of make sense of a lot of data.
0:24:21 For instance, one of our customers, Sigma, that’s a BI provider, are running analysis on
0:24:26 sales logs from sales transcripts, sales calls.
0:24:29 And if you come out understanding, why do we win?
0:24:31 Why do we lose deals?
0:24:37 So being able to run this now across a large data set of all sales calls in a period of time
0:24:39 is as simple as writing English language.
0:24:40 So that’s fascinating to me.
0:24:41 That’s amazing, right?
0:24:42 Yeah.
0:24:47 And then, as I mentioned, of course, the bread and butter, high-quality chatbots,
0:24:50 as well as being able to talk to your structured data for BI-type use cases.
0:24:53 Those are the use cases that we’re seeing.
0:24:58 What I’m seeing, of course, the world of AI evolves incredibly fast.
0:25:01 Week over week, we get a new announcement, something new, exciting.
0:25:02 I know.
0:25:05 Three to five years, I should have said months or weeks even.
0:25:06 That’s on me.
0:25:08 Also, it feels like a year.
0:25:14 Of course, the next big phase that is coming that’s already getting traction
0:25:15 is the world of agents.
0:25:22 So how many are we seeing the ability to answer questions by looking at documentation,
0:25:23 but being able to take action?
0:25:23 Right.
0:25:28 And these agents are, this agentic systems are coming, this ability to reason,
0:25:33 this ability to self-heal, the ability to take action for agents to talk to each other,
0:25:37 collaborate, and that is the next evolution of the technology.
0:25:37 Right.
0:25:43 Are there any agentic frameworks on Snowflake that customers can access?
0:25:44 So very soon.
0:25:49 Right now, the agentic systems that we’ve built are behind the scenes.
0:25:56 The Text2SQL BI experience uses a series of tools to deliver the product.
0:25:56 Gotcha.
0:25:58 So we will make that available to our customers.
0:25:59 Right.
0:25:59 Very cool.
0:26:03 Looking back on your time at Snowflake, further back,
0:26:06 I won’t prescribe time frame zero.
0:26:07 I learned my lesson.
0:26:13 Is there a particular story, moment, something that springs to mind as an important,
0:26:20 perhaps unexpected learning that’s kind of really impacted how you view your work
0:26:22 and the landscape today?
0:26:27 Maybe a problem that the solution turned out to be something unexpected
0:26:30 or something you thought was going to be hard and turned out to be simple.
0:26:33 So I’ll give two examples.
0:26:38 One is very early on, as we were building Cortex, we talked to a customer.
0:26:41 This customer is a long-time Snowflake customer.
0:26:46 They’ve built a pipeline to take their data out and then get it processed by an LLM running
0:26:49 elsewhere and then get back.
0:26:52 And of course, that pipeline took two months or so to build,
0:26:56 and it was quite expensive to maintain and they were concerned about it.
0:27:02 And our early prototype was able to replace the full thing with literally a single line of code.
0:27:06 So we are on to something.
0:27:12 When you bring compute, when you bring AI right next to where the data is,
0:27:14 makes everything a lot simpler.
0:27:17 And when it’s a lot simpler, it just unlocks a lot of usage.
0:27:20 So I’m super excited about just ease of use, simplicity.
0:27:27 The other example is just realizing how kind of demos are easy to build,
0:27:28 but production systems are hard.
0:27:36 You have, especially when it comes to working with structured data, generating SQL is difficult.
0:27:42 So we work really hard on how do we build a system that together creates a very,
0:27:43 very high quality response.
0:27:49 When you’re essentially asking revenue questions, it’s not enough to be 80% accurate.
0:27:53 So that’s another big important area that we focused on.
0:27:55 All right, getting into the wrap up here.
0:28:02 I always ask this question anyway, but I have a child who seems to be getting older every year,
0:28:07 and now he’s in high school, interested in computers, computer science, physical science.
0:28:12 What advice would you give to either a young person kind of looking out,
0:28:16 maybe on the edge of graduating, a little older, maybe graduating college,
0:28:21 or maybe somebody who’s older and is just interested in AI and sort of keeps hearing
0:28:26 the things that we’ve been talking about, which is both that things are changing so fast,
0:28:30 but also, there are things that we can do in the present moment,
0:28:33 and still plenty of problems to be solved.
0:28:35 So where should they go?
0:28:38 Is studying computer science still a viable path?
0:28:44 Is it better to just dive right into the work world and start working on,
0:28:50 as you said, prototyping is one thing, building a production scale system is quite something else.
0:28:56 What’s the advice that you give to young people or maybe older people looking to
0:28:58 dive further into AI?
0:29:05 So I think everyone has their own unique path, and everyone is drawn to something,
0:29:09 and it’s important to be able to connect to what you’re drawn to,
0:29:12 and it’ll be different for different people.
0:29:16 But I just focus on that, just listening to that inner voice,
0:29:20 which is hard to listen to sometimes, especially given there’s so much noise out there.
0:29:24 But I will say, even though AI sounds intimidating,
0:29:29 or there is this kind of artificial intelligence that sounds very complex,
0:29:34 and it is complex when you start going down the rabbit hole and start doing your research,
0:29:40 however, the use of the AI is going to unlock, it’s incredibly easy.
0:29:45 All of these systems are now an API away, and they’re probably powerful.
0:29:52 So I think creativity is going to determine all sorts of super interesting technologies
0:29:53 to be built next.
0:29:57 So I would say, don’t be intimidated with technology, just dive right in,
0:30:02 and it’s incredibly easy to use, and really looking forward to what’s
0:30:04 to come in the next two years or so.
0:30:07 Love it, love the optimism, more audio only, which is a shame,
0:30:11 because your face lit up, smile like I did when you were talking about that.
0:30:16 Bearish, you alluded earlier to cookbooks and other resources that Snowflake makes available.
0:30:19 Maybe we can divvy this up into two parts.
0:30:23 Potential customers who want to learn more about what Snowflake does,
0:30:26 what the offerings are, how to maybe engage with you.
0:30:30 And then folks, practitioners working in AI wanting to learn more about,
0:30:33 you know, what Snowflake’s been doing, research,
0:30:37 some of the techniques we talked about, where can people go online to learn more?
0:30:43 So our website, snowflake.com, if you are trying to figure out how do I use AI
0:30:48 just in seconds and bring my data, analyze my data, we have a solution for you.
0:30:49 Thanks.
0:30:51 So snowflake.com is the place.
0:30:52 Perfect.
0:30:53 Bearish, thank you.
0:30:54 This was great.
0:30:58 As with many of these conversations of these days, I feel like this was kind of the warm-up,
0:31:02 and we’ll have to get back in touch down the line to really dig into where things are headed.
0:31:07 But the Snowflake story, you know, is a great one, and it seems like it’s just getting started.
0:31:10 So congratulations on the work so far.
0:31:12 All the best to you going forward.
0:31:15 And, you know, look out for my unnamed friend I mentioned earlier,
0:31:16 if you see them around campus.
0:31:17 That sounds great.
0:31:18 Thanks for having me.
0:31:28 You know.
0:31:38 [Music]
0:31:48 [Music]
0:32:08 [Music]
0:32:18 [BLANK_AUDIO]
Snowflake is using AI to help enterprises transform data into insights and applications. In this episode of NVIDIA’s AI Podcast, host Noah Kravitz and Baris Gultekin, head of AI at Snowflake, discuss how the company’s AI Data Cloud platform enables customers to access and manage data at scale. By separating the storage of data from compute, Snowflake has allowed organizations across the world to connect via cloud technology and work on a unified platform — eliminating data silos and streamlining collaborative workflows.