AI transcript
0:00:15 Hello, and welcome to the NVIDIA AI podcast. I’m your host, Noah Kravitz.
0:00:20 Artificial intelligence is powering a new generation of physically accurate models and
0:00:24 simulations that are changing the way research and planning are done. In the past few months,
0:00:29 we’ve explored industrial simulation on the podcast, talking about how companies like Siemens
0:00:35 are designing and optimizing factories with the help of AI and technologies including digital twins.
0:00:41 Today, we’re exploring one of the most fascinating and arguably important uses of AI for simulation,
0:00:45 climate simulation. Mike Pritchard, Director of Climate Simulation Research at NVIDIA,
0:00:50 is here to talk about the role of AI in climate science and how AI-driven climate modeling is
0:00:56 helping us to understand, predict, and prepare for extreme weather and climate change. Mike,
0:01:00 welcome and thank you so much for joining the AI podcast.
0:01:02 Thanks so much, Noah. It’s great to meet you.
0:01:07 So maybe we can start by having you share a little bit about your own journey and how you
0:01:09 got into working on climate science.
0:01:13 Sure. Well, it was a little windy road for me. You know, I came out of high school,
0:01:19 not sure if I wanted to study jazz music or chemistry, and I landed in an astrophysics program and learned
0:01:23 about the cosmos and had my first summer job doing astrophysics research to realize I don’t really care
0:01:28 about how two galaxies are going to collide in 15 billion years, enough to work on it all summer.
0:01:34 And yeah, so I took a year off traveling halfway through my undergrad. I ended up visiting Bangladesh
0:01:40 and met all these amazing people living so close to sea level rise and thought back to this course I’d
0:01:45 taken on radiative transfer in planetary atmospheres and realized there’s this physics problem that really
0:01:49 matters to the world. So I haven’t looked back since. You know, I did a master’s, tried a couple of
0:01:55 ice sheet simulator to a climate simulator into San Diego to do a PhD studying the daily cycle of rainfall and
0:01:59 how we can simulate rainfall systems and climate models better and
0:02:03 eventually realized there’s a computer problem here. We just don’t, we know the equations we’d like to solve.
0:02:08 The computers aren’t up for it. So any algorithm that could help me get more explicit cloud physics for more
0:02:14 realistic storm simulations into computer models became interesting. And starting in 2017, AI became the best,
0:02:16 the most interesting algorithm to do that.
0:02:23 Right. I have to go back and ask before we continue talking about climate science. Are you a musician? Do you play?
0:02:29 I play super casually. My jazz chops have dwindled, but there’s a lovely sunset gig I play in San Diego with
0:02:34 someone who lets me play not too many chords and improvise. So yeah, I love music and communicating.
0:02:35 What’s your instrument?
0:02:36 That’s our keyboards.
0:02:39 Keyboards. Great. I’m a drummer. We’ll have to get together sometime.
0:02:40 Oh, right on. Yeah.
0:02:47 All right. But climate science, maybe you can give kind of a brief overview of what it means when we talk
0:02:51 about climate science. We’re going to get into the details, but maybe you can kind of give that overview
0:02:58 and then talk about how the technologies and along with it, your perspective on climate science has evolved
0:03:03 over the years. Okay. Yeah. Caught me short if I nerd out too much, but yeah, we’re all familiar
0:03:09 with weather prediction, right? Climate prediction is sort of similar, but weather prediction is a
0:03:12 little different. You know, where the sort of like, how are the next few days going to unfold while the
0:03:15 atmosphere still has memory of its initial conditions, which I’ve observed and I can initialize
0:03:21 the simulator to predict. Climate simulations are solving very similar equations, but we’re talking about
0:03:25 not the weather, not how things are going to unfold tomorrow or the next day, but the climate,
0:03:29 the average of the weather, the statistics of the weather, and those are predictable on different
0:03:34 timescales for different reasons. So, you know, here in San Diego, I know that, you know, May is
0:03:38 going to be warmer than January because there’s more energy coming into the planet. There’s more solar
0:03:43 energy, watts per square meter. So you add energy to a system, it heats up. Same with, you know,
0:03:47 future climate change. You add a bunch of triatomic molecules to the atmosphere. They tend to back
0:03:51 radiate longwave radiation, like little heat lamps in the sky and more energy, the system heaps up.
0:03:56 But the trillion dollar question is how much will it heat up? Because all these hazards scale with
0:04:00 the overall warmth of the planet, it’s like a fever. And that has a lot to do with very complicated
0:04:05 processes you can’t simulate as well, you know, like clouds, things we don’t know the equations for.
0:04:09 Will they dissipate like ice sheets do, revealing darker surfaces and amplifying the warming? Or will
0:04:14 they stick it up and brighten the planet, reflect some watts as the planet warms? And so, yeah,
0:04:18 the business of simulating the future climate just brings supercomputers to their knees,
0:04:22 because not only do we have to cover the whole planet with resolution and simulate a hundred years,
0:04:27 we have to do that dozens of times for different what-if scenarios and hundreds of times to sample
0:04:32 the different possible chaotic outcomes. And so I almost feel like it’s a bit like learning jazz,
0:04:36 actually. It’s like a never-ending enterprise. You’ll never be done. And there’s always a complex
0:04:41 system that’s unsatisfyingly and perfectly represented in the simulators that deserves
0:04:45 your work. I fixate on clouds because I think they’re mission critical, you know.
0:04:54 So you mentioned, I think you said around 2017 is when AI kind of started to become the tool of choice
0:04:59 for simulations and working on client-sides problems. Can you get in a little bit to how
0:05:05 AI is transforming the way that we simulate, the way we understand how climate works on Earth?
0:05:10 Yeah, thanks. Yeah. So for one thing, it’s a way to short-circuit Moore’s law and to bring
0:05:15 resolution into simulators ahead of computational schedule. So if you can take really complicated cloud
0:05:20 physics that you can simulate for pretty convincingly for a small patch of atmosphere for a few-week
0:05:25 interval, you just can’t scale it up to hundreds of years and the whole planet, you can focus AI on
0:05:30 that and try to learn, learn the physics, outsource the physics to, you know, dense linear algebra,
0:05:37 digital sheets of neurons. And once trained, those can be blazing fast. And so that’s where my interest
0:05:41 began in 2017. It was still a bit of a niche enterprise back then, but the idea, can we build multi-scale
0:05:47 simulators that have very extensively simulated regions of high resolution that we can then wrap
0:05:52 AI around and learn the multi-scale relationships, the predictor-corrected relationships. So I had this
0:05:57 wonderful experience in 2018 where that worked better than I thought it should, and it bought me 20x to 100x
0:06:02 faster simulations. And it totally changed my view of how much resolution and complexity we can afford to
0:06:06 simulate today. Fast forward five years, and the whole world was different. Now we’re not talking about
0:06:12 hybrid physics AI simulations, we’re talking about full AI simulators of the atmosphere. And with that
0:06:18 comes a whole new set of disruptions that include huge ensembles for counterfactuals of weather that
0:06:23 are normally very hard to generate with regular physics calculations, include dimensions of interactivity
0:06:29 I’d never dreamed of before, ways to have interactive climate samplers that democratize the business of
0:06:34 looking at climate predictions, ways to paint resolution onto low-resolution blurry predictions.
0:06:38 Yeah, so I feel like AI is now disrupting things across the ecosystem modeling stack.
0:06:45 Mike, when you talk about resolution in climate modeling, what does that mean? Is that the ability
0:06:51 to predict in a more precise time frame? Is it just getting better predictions of what’s going to
0:06:53 happen? What does resolution mean in this context?
0:06:59 Yeah, great question. Yeah. So at the end of the day, predicting the climate using physics equations means
0:07:03 writing those equations and then dividing the world up into a mesh to actually solve the equations
0:07:09 numerically. Okay. And how high resolution that meshes depends on how much computing power we have
0:07:14 and how long we need to simulate the system. So we need to simulate the system for a hundred years,
0:07:18 hundreds of times, and therefore the mesh is not as fine as we would like because the computers are
0:07:22 always so powerful. So a typical government prediction of 50 years from now, the individual
0:07:28 variables that are predicted are on scales of like counties and states, 25 kilometers a side.
0:07:33 You go out and look at satellite observations of regions of the atmosphere that are that large,
0:07:38 you’ll see all this interior complexity from beautiful processes that form clouds and storms.
0:07:43 Those are fundamentally not represented explicitly in standard simulations of climate. So that’s why
0:07:48 resolution matters. It means that some human has had to come up with some cartoon model based on
0:07:53 their notions of how that subgroup complexity works when all we’re resolving is the county scale stuff.
0:08:00 You kind of mentioned the advance in compute power and simulations just now, but can you talk about some
0:08:04 of the breakthroughs that you’ve been involved with, with different teams you’ve worked at over the
0:08:09 years, when it comes to using machine learning, using AI to accelerate these simulations?
0:08:14 Yeah, I’ll mention a few threads of work. So one is where I began as a university professor,
0:08:21 this hybrid physics AI simulation. It’s really difficult. The imperfections of an AI’s fit to a physical
0:08:27 simulation. Once you embed 10,000 copies of that AI emulator inside of a planetary model that’s based
0:08:32 on physics, all the interactions between the AI subunits and the planetary fluid dynamic solver can
0:08:36 lead you out of sample in ways that make the hybrid system difficult to control. So one research
0:08:41 accomplishment I’m really proud of is a benchmarking activity we did at NeurIPS a couple years ago to
0:08:45 democratize this problem, steal it to lots of machine learners, work with the NSF Science and
0:08:50 Technology Center in Colombia and some NVIDIA people to make really nice code and APIs that people could
0:08:54 compete around and a kind of competition around that and eventually figured out some strategies to
0:09:01 get this kind of really brittle physics AI simulator to work. But then I think it’s coming to NVIDIA and
0:09:05 experience a new type of person that trains very ambitious machine learning models that are so large,
0:09:09 they can cover the entire planet. There’s no more physics anymore. There’s no physics solvers. It’s
0:09:14 just AI. You know, it’s like video prediction, predicting the next frame, but instead of three channels of
0:09:18 color, like red, green, and blue that are mostly correlated, but you know, a dozen channels of atmospheric
0:09:22 states, winds near the surface, winds near the upper atmosphere, water vapor species that are very
0:09:27 absolute distributions. But we’re, you know, trying to take the, learn a dynamical operator that predicts
0:09:31 the next time step and then feed that back into itself and auto-aggressively roll out weather
0:09:36 forecasts. That’s how AI, full AI weather forecasting works. I’m not very skeptical of that three years ago.
0:09:41 You know, the world has been convinced that this is the better way to do AI weather forecasts.
0:09:46 They’re more skillful. The most skillful forecasts in the world now come from AI systems that are
0:09:50 indiscutable. We don’t understand fully how they’re making their predictions, but they’re
0:09:55 easy to score and convince yourself they’re better. Yeah. So I, you know, my team of climate demand
0:10:00 experts at NVIDIA, you know, talks to the developers of these AI weather forecasting systems. And then
0:10:05 we started collaborating with generative AI experts who are accustomed to doing amazing things with video
0:10:09 and imagery and asking, could we do similar things with our data? So we wanted to have collectivity with
0:10:14 super resolution. The world has a lot of blurry predictions of the future, but the world needs a lot more
0:10:18 fine scale, super resolved versions of those predictions because, you know, climate impacts
0:10:22 yourself on the scales, kilometers, and infrastructure. And so we asked, could we take some
0:10:28 blurry climate data of one form and use generative AI to turn it into very highly resolved versions of the
0:10:33 same variables, including all the distribution chips involved, but also to synthesize new variables that
0:10:38 you can only observe at high resolution, like radar reflectivity observations of rainfall.
0:10:42 You know, this just blew my mind. It worked so much better than I expected. So now we have this thing
0:10:48 called Cordiff. It does multivariate super resolution and new channel synthesis and has been used in lots
0:10:52 of different settings to take the pain out of generating high resolution state estimates of the
0:10:58 atmosphere. When you talk about predicting weather, predicting climate, is there a percentage number
0:11:04 in terms of accuracy or a way of kind of putting your finger on how, what’s our accuracy like right
0:11:10 now? And how is it maybe changed? And I would assume improved since the advent of, you know,
0:11:13 advanced AI and now generative AI in the process.
0:11:18 Oh, that’s a great question. Yeah. For weather forecasting, it’s very clear that the pace of
0:11:21 increasing in steel is outpacing what we’ve seen with physics models. So something called like the
0:11:26 quiet resolution weather forecasting, you can look back on the last two decades and ask how has weather
0:11:29 forecasting still gotten better over time? And it’s gotten better, but, you know, slowly and
0:11:34 asymptotically, and it seems to be not getting better much for these days. And there’s a version of this
0:11:38 graph. I just saw that one of the leaders at the European Center for Medium Weather Forecasting
0:11:43 presented a conference that shows AI just leapfrogging all that trajectory. Yeah. So the weather
0:11:48 community has really, really seen this. You can measure this. For climate prediction, it’s difficult.
0:11:52 I mean, the answer is out of sample. The answer is in 50 years. We don’t know the answer. So what we’re
0:11:56 trying to predict are like a range of outcomes and concepts that are irrelevant to weather prediction,
0:12:01 like energy conservation, mass conservation, are really important to make sure your model obeys.
0:12:07 So I would say what’s going on now is this whole ecosystem effect where lots of developers are
0:12:13 working to take what’s happened in AI for weather prediction and use it to the benefit of climate
0:12:18 prediction. So one thing you can do is train an emulator, not of weather observations, but of
0:12:22 climate projections themselves. That’s working pretty well. Once you have that climate projections
0:12:27 that are normally difficult to pass around the world on the internet, it becomes very easy to give people
0:12:31 because you just give someone a trained AI model and they inference the climate much faster
0:12:35 than they would read a data set. Yeah. And there are some people who are trying to build
0:12:40 multi-component versions of these pure AI climate simulators that have the same properties we require
0:12:45 of climate prediction models. So correct coupling between the atmosphere and ocean, correct
0:12:51 conservation of mass and energy, correct responses to adding a little more radiative energy to the
0:12:56 system like CO2 gases do. So the community is hard at work trying to understand if we can build
0:13:00 a next generation of fully AI climate models, but it’s work in progress.
0:13:05 That’s great to hear. And so that kind of makes me think on the flip side, what do we do with these
0:13:09 predictions? What do we do with this information with the ability, as you put it, to inference
0:13:13 what the weather, perhaps what the climate might be like in a given amount of time?
0:13:21 We’ve had, I mean, this year already, wildfires in the West, tornadoes and other extreme storms kind
0:13:26 of in the middle and southern part of the U.S. and obviously globally, all kinds of extreme weather
0:13:32 events becoming more frequent in recent times. So how can advances in AI-driven climate modeling
0:13:38 help us better prepare, help communities better prepare for what’s coming and adapt to the change
0:13:38 in climate?
0:13:40 So I’ll give you two answers.
0:13:40 Okay.
0:13:46 One is like a fundamentally new ability to study the statistics of these rare extremes themselves.
0:13:50 By definition, these low likelihood eye impact extremes, they’re rare in the observed record.
0:13:53 We can’t study them in observations because the sample sizes are tiny.
0:13:59 We could generate, you know, hundreds or thousands of counterfactual low likelihood extremes using
0:14:04 physical simulators, but that’s too extensive. So the speed up of AI models, this thousand
0:14:09 X increase in inference speed that we get, has opened up a new science of huge ensemble counterfactual
0:14:13 generation. There’s a pair of papers I love led by Berkeley that we collaborate on with NVIDIA.
0:14:18 They’ve taken our AI model forecast net. They’ve calibrated it carefully so that it produces
0:14:23 the right probability distribution, a trustworthy probability distribution to study heat extremes.
0:14:27 And then they’ve run like 28,000 years worth of equivalent summer 2023.
0:14:31 And so instead of one forecast every day, we’ve got 7,000 every day.
0:14:36 And so you can find so many different counterfactuals of how 2023 could have turned out.
0:14:40 And, you know, this, this is a new type of data we couldn’t generate before where we could sort of
0:14:45 start to have satisfying sample sizes of even rare heat waves to begin to understand their climate
0:14:49 drivers, to inform our understanding of how those statistics are going to change in the future.
0:14:53 So that’s a mature technology that exists. Insurance companies are already starting to use it.
0:14:57 Insurance companies normally just receive private projections, but they’re starting to generate them
0:15:02 internally because it’s so easy with AI. And then the other thing is just improved weather forecasting
0:15:06 itself, more interactive weather forecasting, more steerable forecasting. My team’s just put out
0:15:11 something I’m really excited about called Climate in a Model. It’s opened my mind to the fact that
0:15:15 there’s potential to do so much more than physics simulations, which is, you know, those are a two-stage act.
0:15:20 Generate output, look at output and finding hurricanes you care about is like a needle in a haystack.
0:15:25 But AI lets us say, you know, I’d like a weather for, I’d like to find a hurricane, please. I’d like to
0:15:29 find a hurricane in a future climate. Just give me a sample of that because that’s what I’m interested
0:15:34 in. So this, I think that will increase the ability of everyone to interact with climate predictions.
0:15:40 You know, imagine a future where you can be sitting on the, in Florida, looking at a seed of a potential
0:15:45 future cyclone way out in the Eastern Atlantic and say, give me only weather forecasts that produce a really
0:15:51 bad version of that hurricane, so I can plan for the worst. That’s, that’s normally really hard to do with
0:15:55 normal simulation technology. But, but AI, I totally see that’s going to be possible.
0:16:01 Yeah. And so are, you know, governmental, you mentioned insurance companies, of course, but are governmental
0:16:09 leaders, governmental organizations, disaster response organizations, are they using, or if not, how can they use these
0:16:15 interactive climate simulators to inform their responses and inform, you know, the policy decisions
0:16:20 for the future? I think it’s going to be a process in their understanding their implications. The whole
0:16:25 world is grappling with the interactivity dimension right now. I know, so I have a, we’re fortunate in
0:16:30 the Earth 2 Initiative NVIDIA to have some really inspiring European science advisors. One of them is
0:16:34 named Bjorn Stevens. He gave a wonderful keynote at the International Supercompetitive Conference a couple
0:16:40 weeks ago. And what he sees is a future where there’s a source of credible, appropriate data to
0:16:46 use for future climate prediction, like for a cutting edge, best in class physics model, and then AI on top
0:16:52 of it to broaden people’s reach into what the data produces. And as long as the AI is faithful to the
0:16:58 training data, then it solves a liability concern where, you know, climate actors need to have data
0:17:03 provenance, need to have provenance to credible sources of impacts related decisions. So I think
0:17:07 where, where we’re researching and where we’re working on right now is building that AI on top to
0:17:13 understand, you know, what is it going to be appropriate to be able to ask a trove of simulation
0:17:18 output for in the future? I know like, I know I can ask for tropical cyclones right now, but I don’t
0:17:23 know if I could ask for other things I’d like to ask for, like, hey, show me, show me a counterfactual of
0:17:28 two compound hurricanes pummeling into Bermuda one after the other in today’s climate and in
0:17:33 2070. But as these capabilities emerge, I think we’ll see what the ecosystem wants to do with them.
0:17:38 Yeah. Otherwise, I see, you know, climate tech startups doing sensible things like taking AI for
0:17:43 state estimation to prove their value prop of proprietary data streams, improving state estimates
0:17:48 for forecasting applications. I’m speaking with Mike Pritchard. Mike is the director of climate
0:17:54 simulation research in NVIDIA. And we’ve been talking about the way that AI has really, I hate
0:18:00 to use this word, but disrupted climate science and simulation, weather forecasting as well. Mike,
0:18:04 in the intro, I referenced digital twins, which is something we’ve talked about on the show,
0:18:12 mainly in reference to industrial AI simulating the way that factories operate for planning purposes and
0:18:16 so on. How do digital twins play into climate science?
0:18:21 Yeah. Well, so we already think of our physical simulators of the future of the planet as sort of
0:18:26 digital twins. If they can be used for scenario planning, you can ask what if questions, you know,
0:18:30 what if humans evolve the land surface this way and emit this much guns in the atmosphere versus what
0:18:36 if they follow this mitigation scenario. So AI as a surrogate for physical climate models on top of
0:18:42 physical climate predictions could be viewed as sort of reducing the latency associated with our familiar
0:18:47 digital twins. But I think you’re after something bigger, which is, you know, what I observe in
0:18:51 some of NVIDIA’s connections to industrial infrastructure work is this ability to kind of
0:18:56 backpropagate and differentiate through, ultimately optimize the control of very complex systems.
0:19:02 And so there, I think it’s really exciting to imagine, you know, a digital twin of an electrical grid
0:19:07 that coupled to digital twins of individual power stations, eventually coupled to digital twins,
0:19:12 maybe there are existing AI forecast models, differentiable, like autoregressors of the entire
0:19:16 atmosphere, coupled to digital twins of climate. And you can chain those together and backpropagate
0:19:21 through the whole stack, then maybe we can play games we couldn’t have dreamed of before to sort of,
0:19:26 you know, optimize for resilience in the face of slowly varying low likelihood climate extremes.
0:19:32 So let’s switch gears for a second. I want to talk about a talk that you gave, a recent TED talk,
0:19:37 where you were talking about the importance of AI in climate simulation. Why give a TED talk? What
0:19:43 was the message that you were hoping to leave the audience, the public with from your talk?
0:19:48 I think just that something very exciting is going on in the business of simulating the atmosphere
0:19:53 that nobody would have predicted 10 years ago. This fundamentally new simulation technology,
0:19:56 but there’s properties we don’t even understand yet, but very enticing characteristics like,
0:20:01 yeah, interactivity. I just wanted to get that message out there. So people are aware that,
0:20:07 you know, that that’s a really beneficial use of AI for a really important problem that affects us all
0:20:12 weather prediction and climate prediction. You know, it’s wonderful to see this like cocoon of people
0:20:17 working across academia and government labs and industry, very complimentary skill sets that are
0:20:21 needed to bring that into existence and achieve its full potential. So that was one of the messages.
0:20:27 But yes, I think the, I’ve touched on some of the remarkable characteristics of AI for simulation here,
0:20:32 but not some that I’m recalling. I mentioned this TED talk. I think one of the amazing things is the
0:20:37 ability to use more data. So we have so many observations of the Earth’s system. It’s actually
0:20:42 really hard to incorporate them into physics models because I mentioned humans have to write
0:20:46 their assumptions of how unresolved things work like clouds, and that can easily clash with real
0:20:51 information about clouds. And you have to eventually sort of sacrifice observations you wish you could
0:20:55 use because they just think they shock your model and you can’t use your model for the things you’d
0:21:00 like to use it for anymore. So AI, by having no humans write down anything, the physical equations
0:21:06 are gone. It just opens up this huge filter that we didn’t used to think we could into how many
0:21:11 observations we could actually use to benefit our simulations. And so there’s exciting work going on
0:21:17 at the European Center now that is using only observations, satellite observations and station
0:21:22 observations as the sources of AI prediction. I think that’s the future. And as we, you know,
0:21:27 forecasting is two problems. It’s state estimation. How good do you know your initial condition?
0:21:32 And prediction. How well can you predict from there? And the unification of AI for improving state
0:21:36 estimation and prediction is happening as we speak. And we don’t know the potential. We don’t know what
0:21:41 the predictability limits of the Earth system are going to look like in 10 years. So it’s a very
0:21:44 exciting time to be alive. That’s what I wanted to convey in 10. It’s an exciting time.
0:21:51 It is. Listening you talk about that makes me think of episodes we’ve done involving astronomy
0:21:58 and involving healthcare and medical imaging and guests talking about how there’s so much data
0:22:05 in existing images, medical images taken, you know, years ago, the images coming back from,
0:22:10 you know, the James Webb telescope. And there’s so much data that, you know, kind of to your point,
0:22:15 humans making observations and writing things down and doing calculations can only go so far.
0:22:22 The AI systems can really, you know, dig into these massive data sets and dig into all the data in a
0:22:28 single image. And in the case of medical imaging, remember we had a guest talking about finding a new
0:22:35 biomedical marker just by going through preexisting imagery. Is there a similar phenomena that happens
0:22:41 with weather observations and weather imagery? Are you able to, you know, find things in kind of the
0:22:46 historical observations that, you know, we just didn’t know were there before? And then they’re able to
0:22:53 inform kind of, you know, getting a better future state understanding that that informs your predictions?
0:23:04 It’s happening as we speak, you know, there’s a pair of intellectual paper on the University of Washington that’s claiming to update the long held notions of how long the predictability limit of the atmosphere is.
0:23:08 And that’s causing a lot of controversial chatter in the community. So yeah, I think the questions you’re
0:23:17 speaking to are happening right now in our community. There are some timescales that are very difficult to predict on, like beyond a week or so when the atmosphere loses its memory.
0:23:28 But when there’s other components of the system complex components, the ocean surface, the ecosystems that have memory that should give us some predictability on three, four, even five week timescales.
0:23:36 That’s where I really hope we’ll have new scientific understanding as AI teachers is how much predictability has been hiding on those timescales all this time.
0:23:43 But you mentioned data and I have to say, like, we have a unique problem in the earth system sciences, which is there’s only so many decades of satellite observations.
0:23:51 And so we don’t have the whole internet of images and video to feed off of. At the end of the, you know, at the end, we have like tens of thousands of days of satellite observations.
0:23:56 And so there are still, there are inevitably going to be some phenomena where the sample sizes are just too small.
0:23:59 You know, there’s only been so many El Nino events in those 50 years.
0:24:10 There’s only been so many, let alone future climate change, you know, how the, you know, some things are inherently out of sample of the observations that feed our models, like the slow modes of the ocean dynamics.
0:24:13 And the ocean becomes the most important controls climate change by 2080.
0:24:25 So in my mind, there’s, there’s always going to be a combination of physical simulators that can, you know, are fit for purpose for those deep time projections alongside observations that I agree with you have not been fully exploited by humans.
0:24:29 And what that combination is going to be is really fascinating to think about.
0:24:34 Does synthetic data play a significant role in, in climate science?
0:24:35 Absolutely.
0:24:47 Yeah, we, you know, we can leverage our many simulations of the future or many past forecasts of weather as tree training objects to try to, you know, get the model as ready as it can be.
0:24:52 Use its memory capacity as efficiently as possible for ultimate fine tuning on the observations of high quality.
0:24:53 Yeah, that’s an active area of research.
0:25:00 So looking ahead, and you kind of talked a little bit about some of these things before, so feel free to kind of recap, if you will.
0:25:07 But what are some of the next big things that you, your colleagues, kind of the climate science community are working on?
0:25:16 And specifically with your work at NVIDIA, are there any big research questions that you’re tackling right now or hoping to get to in the days and months to come?
0:25:20 So around the world, I’m really excited by what’s going on with, with ocean emulation.
0:25:24 We’ve seen the atmosphere and the weather community really do great things.
0:25:27 And the ocean simulation community is starting to do great things as well.
0:25:30 And the unification of these is, it’s a really exciting frontier.
0:25:38 You mentioned a minute ago, the ocean, and I don’t want to get the words wrong, but the ocean kind of being the, the primary controller of climate.
0:25:39 Is that how you put it?
0:25:40 Yeah.
0:25:44 You know, the vast majority of the excess heat that we’ve put into the earth system is absorbed by the ocean.
0:25:54 And the memory of where it goes, where the marine heat waves happen in the future will largely stop our experience of, you know, inter-annual heat extremes and challenging extreme events.
0:25:55 Yeah.
0:25:59 So the, the ocean is really important to include in any prediction of climate.
0:25:59 Right.
0:26:03 So the advent of full ocean emulators is just happening this year and last year.
0:26:04 Oh, that’s amazing.
0:26:06 And they’re coupling the full atmosphere emulators.
0:26:07 It’s just happening.
0:26:12 It’s like in the 1980s when people first coupled physics-based atmosphere and ocean models, which used to be separate communities.
0:26:16 And out came the world’s first El Nino predictions and seasonal forecasting capabilities.
0:26:17 Yeah.
0:26:20 This is like climate model development on like, on, on, on an accelerated timeline.
0:26:24 It feels like the last three years are like decades compressed into a few.
0:26:27 At NVIDIA, I’m really excited about this interactivity dimension.
0:26:28 It, it stuns me.
0:26:30 So we have this thing called climate in a bottle.
0:26:33 It’s a paper that our scientists put out a month ago.
0:26:41 And, um, it’s not, it’s not like these other models I’ve described where you’re trying to generate videos and outputs to come inputs.
0:26:43 That feels very weathery to me.
0:26:46 What feels climate-y is saying, hey, here’s the climate boundary conditions.
0:26:49 Here’s what should be controlling weather noisily.
0:26:52 And give me some samples of weather subject to those boundary conditions.
0:27:02 So this climate in a bottle research paper demonstrates a model like this, where the user can pass in just three pieces of information, like time of day, time of year, that controls where the sun is.
0:27:04 And how warm is the ocean?
0:27:07 I mentioned the ocean largely regulates the atmosphere on climate times again.
0:27:12 That’s very low dimensional information, like a text query to a large language model.
0:27:13 200 kilobytes of information.
0:27:23 Out of the other end of this thing comes 600 megabytes across 12 different variables of 13 million pixels of state estimations consistent with those inputs.
0:27:29 And if you look at the climatology of that system, like you generate tens of thousands of samples and say, how’s the seasonal cycle?
0:27:30 How’s the daily cycle?
0:27:33 How are all the modes of variability I care about as a climatologist?
0:27:34 They look good.
0:27:36 And then now here’s the real kicker.
0:27:41 We’ve added the ability to steer this generation to events of interest.
0:27:45 So one of our scientists trained a tropical cyclone classifier alongside the generator.
0:27:50 And now we’re able to pass in a fourth ingredient, a map, of where I would like to see cyclones.
0:27:53 And this is, to me, I’ve never seen anything like this.
0:27:58 You know, I’m used to having to sift through, you know, petabytes of output to find a few tropical cyclones I care about.
0:28:04 I just feel to ask for one and to have the result be consistent with the statiotemporal dynamics of the data it was trained on.
0:28:12 That’s the tip of an iceberg of new interactivity features of queriable time and informatics that I think is going to feel very different than what we’re used to in five years.
0:28:14 So, yeah, that’s where I’m focusing a lot of attention right now.
0:28:15 Amazing.
0:28:22 If listeners, if you’re listening on audio and can’t see the video, Mike’s face is consistently lighting up as he’s talking about some of these things.
0:28:42 Mike, to leave the listeners with kind of a call to action here, whether there are technologists or just concerned citizens who have been listening to the conversation, and they, we, speaking for myself, want to know how we can get involved, how we can support the advancement of climate science, how we can support sustainability through AI.
0:28:43 What advice would you have?
0:28:43 What can we do?
0:28:51 Oh, I think like any concerned citizen, you know, like contact your politicians if you would like to, you know, exert your preferences about where priorities are.
0:28:57 I feel like a lot of priorities of funding are currently in debate and there’s, you know, rational questions of efficiency going on.
0:29:01 But one thing that’s very important to us all is our sustained observations of the Earth system.
0:29:04 Machine learning is nothing without the data it’s trained on.
0:29:20 And so, yeah, if you value the observing systems that have led and the weather modeling agencies that have led to the data sets that are leading to this amazing revolution in AI prediction quality, express that to the people in charge of the resources that allow those observations to exist.
0:29:22 And then, you know, you can tinker.
0:29:27 If you’re a Kaggle competition enthusiast, you know, pick a climate one.
0:29:34 If you’re an ML researcher and you’d like to plug into a benchmark, we’ve got a few climate ones that I think are still unsolved benchmarks that could use your attention.
0:29:35 Yeah.
0:29:37 And I don’t know what else to say to your question.
0:29:39 You know, just maybe have conversations about it.
0:29:40 The planet is warming.
0:29:41 We’re a part of that.
0:29:43 It’s going to be a part of our future, our kids’ future.
0:29:45 A lot of the warming has already happened.
0:29:49 You know, the average lifetime of a carbon dioxide molecule in the atmosphere is 100 years.
0:29:53 And so the past 200 years of emissions will definitely be warming our planet in the future.
0:29:56 And we all need to think about what we do with that.
0:30:05 I think that AI will allow us to appreciate, by being able to interact more natively, with predictions of the future, the complex system that we’re embedded in.
0:30:08 I think just that awareness is good to talk with your friends about.
0:30:13 Yeah, that is a powerful mechanism for understanding things closer to firsthand, for sure.
0:30:20 Mike, for listeners who would like to know more, would like to follow your work and your team’s work, where’s a good place to go online?
0:30:24 NVIDIA Research, or where can we send people to investigate further?
0:30:26 Yeah, look up NVIDIA Earth 2.
0:30:27 Earth 2 is the name of our initiative.
0:30:29 And there’s all sorts of wonderful software in there.
0:30:36 There’s recipes for many of the AI simulation technologies I’ve mentioned, open source recipes that we’re hoping the ecosystem builds upon.
0:30:37 There’s training materials.
0:30:40 So yeah, Earth 2 is the thing to look up.
0:30:41 Fantastic.
0:30:44 Mike Pritchard, again, thank you so much for taking the time to join the podcast.
0:30:47 It’s a fascinating conversation.
0:31:00 And despite kind of the seriousness of climate change and extreme weather and such, you’ve left me with a lot of optimism about the technologies kind of, as you said, you know, the past few years have just been super jam-packed with advancements.
0:31:09 And the ability to use these to help us understand and adapt to the climate is at the top of the list, in my view, of the importance of this work.
0:31:11 So yeah, what a time to be alive, as you said.
0:31:12 Yeah, thanks, Noah.
0:31:21 It’s a real privilege for me to represent a huge team of hardworking, humble, really earnest technologists and engineers and scientists that are working together on these problems.
0:31:25 I’m just a mouthpiece, but yeah, any credit should go to them.
0:31:26 Absolutely.
0:31:27 Well, thank you to your whole team.
0:31:27 Thanks, Noah.
0:32:17 I’m just a mouthpiece, but yeah, any credit should go to them.
0:00:20 Artificial intelligence is powering a new generation of physically accurate models and
0:00:24 simulations that are changing the way research and planning are done. In the past few months,
0:00:29 we’ve explored industrial simulation on the podcast, talking about how companies like Siemens
0:00:35 are designing and optimizing factories with the help of AI and technologies including digital twins.
0:00:41 Today, we’re exploring one of the most fascinating and arguably important uses of AI for simulation,
0:00:45 climate simulation. Mike Pritchard, Director of Climate Simulation Research at NVIDIA,
0:00:50 is here to talk about the role of AI in climate science and how AI-driven climate modeling is
0:00:56 helping us to understand, predict, and prepare for extreme weather and climate change. Mike,
0:01:00 welcome and thank you so much for joining the AI podcast.
0:01:02 Thanks so much, Noah. It’s great to meet you.
0:01:07 So maybe we can start by having you share a little bit about your own journey and how you
0:01:09 got into working on climate science.
0:01:13 Sure. Well, it was a little windy road for me. You know, I came out of high school,
0:01:19 not sure if I wanted to study jazz music or chemistry, and I landed in an astrophysics program and learned
0:01:23 about the cosmos and had my first summer job doing astrophysics research to realize I don’t really care
0:01:28 about how two galaxies are going to collide in 15 billion years, enough to work on it all summer.
0:01:34 And yeah, so I took a year off traveling halfway through my undergrad. I ended up visiting Bangladesh
0:01:40 and met all these amazing people living so close to sea level rise and thought back to this course I’d
0:01:45 taken on radiative transfer in planetary atmospheres and realized there’s this physics problem that really
0:01:49 matters to the world. So I haven’t looked back since. You know, I did a master’s, tried a couple of
0:01:55 ice sheet simulator to a climate simulator into San Diego to do a PhD studying the daily cycle of rainfall and
0:01:59 how we can simulate rainfall systems and climate models better and
0:02:03 eventually realized there’s a computer problem here. We just don’t, we know the equations we’d like to solve.
0:02:08 The computers aren’t up for it. So any algorithm that could help me get more explicit cloud physics for more
0:02:14 realistic storm simulations into computer models became interesting. And starting in 2017, AI became the best,
0:02:16 the most interesting algorithm to do that.
0:02:23 Right. I have to go back and ask before we continue talking about climate science. Are you a musician? Do you play?
0:02:29 I play super casually. My jazz chops have dwindled, but there’s a lovely sunset gig I play in San Diego with
0:02:34 someone who lets me play not too many chords and improvise. So yeah, I love music and communicating.
0:02:35 What’s your instrument?
0:02:36 That’s our keyboards.
0:02:39 Keyboards. Great. I’m a drummer. We’ll have to get together sometime.
0:02:40 Oh, right on. Yeah.
0:02:47 All right. But climate science, maybe you can give kind of a brief overview of what it means when we talk
0:02:51 about climate science. We’re going to get into the details, but maybe you can kind of give that overview
0:02:58 and then talk about how the technologies and along with it, your perspective on climate science has evolved
0:03:03 over the years. Okay. Yeah. Caught me short if I nerd out too much, but yeah, we’re all familiar
0:03:09 with weather prediction, right? Climate prediction is sort of similar, but weather prediction is a
0:03:12 little different. You know, where the sort of like, how are the next few days going to unfold while the
0:03:15 atmosphere still has memory of its initial conditions, which I’ve observed and I can initialize
0:03:21 the simulator to predict. Climate simulations are solving very similar equations, but we’re talking about
0:03:25 not the weather, not how things are going to unfold tomorrow or the next day, but the climate,
0:03:29 the average of the weather, the statistics of the weather, and those are predictable on different
0:03:34 timescales for different reasons. So, you know, here in San Diego, I know that, you know, May is
0:03:38 going to be warmer than January because there’s more energy coming into the planet. There’s more solar
0:03:43 energy, watts per square meter. So you add energy to a system, it heats up. Same with, you know,
0:03:47 future climate change. You add a bunch of triatomic molecules to the atmosphere. They tend to back
0:03:51 radiate longwave radiation, like little heat lamps in the sky and more energy, the system heaps up.
0:03:56 But the trillion dollar question is how much will it heat up? Because all these hazards scale with
0:04:00 the overall warmth of the planet, it’s like a fever. And that has a lot to do with very complicated
0:04:05 processes you can’t simulate as well, you know, like clouds, things we don’t know the equations for.
0:04:09 Will they dissipate like ice sheets do, revealing darker surfaces and amplifying the warming? Or will
0:04:14 they stick it up and brighten the planet, reflect some watts as the planet warms? And so, yeah,
0:04:18 the business of simulating the future climate just brings supercomputers to their knees,
0:04:22 because not only do we have to cover the whole planet with resolution and simulate a hundred years,
0:04:27 we have to do that dozens of times for different what-if scenarios and hundreds of times to sample
0:04:32 the different possible chaotic outcomes. And so I almost feel like it’s a bit like learning jazz,
0:04:36 actually. It’s like a never-ending enterprise. You’ll never be done. And there’s always a complex
0:04:41 system that’s unsatisfyingly and perfectly represented in the simulators that deserves
0:04:45 your work. I fixate on clouds because I think they’re mission critical, you know.
0:04:54 So you mentioned, I think you said around 2017 is when AI kind of started to become the tool of choice
0:04:59 for simulations and working on client-sides problems. Can you get in a little bit to how
0:05:05 AI is transforming the way that we simulate, the way we understand how climate works on Earth?
0:05:10 Yeah, thanks. Yeah. So for one thing, it’s a way to short-circuit Moore’s law and to bring
0:05:15 resolution into simulators ahead of computational schedule. So if you can take really complicated cloud
0:05:20 physics that you can simulate for pretty convincingly for a small patch of atmosphere for a few-week
0:05:25 interval, you just can’t scale it up to hundreds of years and the whole planet, you can focus AI on
0:05:30 that and try to learn, learn the physics, outsource the physics to, you know, dense linear algebra,
0:05:37 digital sheets of neurons. And once trained, those can be blazing fast. And so that’s where my interest
0:05:41 began in 2017. It was still a bit of a niche enterprise back then, but the idea, can we build multi-scale
0:05:47 simulators that have very extensively simulated regions of high resolution that we can then wrap
0:05:52 AI around and learn the multi-scale relationships, the predictor-corrected relationships. So I had this
0:05:57 wonderful experience in 2018 where that worked better than I thought it should, and it bought me 20x to 100x
0:06:02 faster simulations. And it totally changed my view of how much resolution and complexity we can afford to
0:06:06 simulate today. Fast forward five years, and the whole world was different. Now we’re not talking about
0:06:12 hybrid physics AI simulations, we’re talking about full AI simulators of the atmosphere. And with that
0:06:18 comes a whole new set of disruptions that include huge ensembles for counterfactuals of weather that
0:06:23 are normally very hard to generate with regular physics calculations, include dimensions of interactivity
0:06:29 I’d never dreamed of before, ways to have interactive climate samplers that democratize the business of
0:06:34 looking at climate predictions, ways to paint resolution onto low-resolution blurry predictions.
0:06:38 Yeah, so I feel like AI is now disrupting things across the ecosystem modeling stack.
0:06:45 Mike, when you talk about resolution in climate modeling, what does that mean? Is that the ability
0:06:51 to predict in a more precise time frame? Is it just getting better predictions of what’s going to
0:06:53 happen? What does resolution mean in this context?
0:06:59 Yeah, great question. Yeah. So at the end of the day, predicting the climate using physics equations means
0:07:03 writing those equations and then dividing the world up into a mesh to actually solve the equations
0:07:09 numerically. Okay. And how high resolution that meshes depends on how much computing power we have
0:07:14 and how long we need to simulate the system. So we need to simulate the system for a hundred years,
0:07:18 hundreds of times, and therefore the mesh is not as fine as we would like because the computers are
0:07:22 always so powerful. So a typical government prediction of 50 years from now, the individual
0:07:28 variables that are predicted are on scales of like counties and states, 25 kilometers a side.
0:07:33 You go out and look at satellite observations of regions of the atmosphere that are that large,
0:07:38 you’ll see all this interior complexity from beautiful processes that form clouds and storms.
0:07:43 Those are fundamentally not represented explicitly in standard simulations of climate. So that’s why
0:07:48 resolution matters. It means that some human has had to come up with some cartoon model based on
0:07:53 their notions of how that subgroup complexity works when all we’re resolving is the county scale stuff.
0:08:00 You kind of mentioned the advance in compute power and simulations just now, but can you talk about some
0:08:04 of the breakthroughs that you’ve been involved with, with different teams you’ve worked at over the
0:08:09 years, when it comes to using machine learning, using AI to accelerate these simulations?
0:08:14 Yeah, I’ll mention a few threads of work. So one is where I began as a university professor,
0:08:21 this hybrid physics AI simulation. It’s really difficult. The imperfections of an AI’s fit to a physical
0:08:27 simulation. Once you embed 10,000 copies of that AI emulator inside of a planetary model that’s based
0:08:32 on physics, all the interactions between the AI subunits and the planetary fluid dynamic solver can
0:08:36 lead you out of sample in ways that make the hybrid system difficult to control. So one research
0:08:41 accomplishment I’m really proud of is a benchmarking activity we did at NeurIPS a couple years ago to
0:08:45 democratize this problem, steal it to lots of machine learners, work with the NSF Science and
0:08:50 Technology Center in Colombia and some NVIDIA people to make really nice code and APIs that people could
0:08:54 compete around and a kind of competition around that and eventually figured out some strategies to
0:09:01 get this kind of really brittle physics AI simulator to work. But then I think it’s coming to NVIDIA and
0:09:05 experience a new type of person that trains very ambitious machine learning models that are so large,
0:09:09 they can cover the entire planet. There’s no more physics anymore. There’s no physics solvers. It’s
0:09:14 just AI. You know, it’s like video prediction, predicting the next frame, but instead of three channels of
0:09:18 color, like red, green, and blue that are mostly correlated, but you know, a dozen channels of atmospheric
0:09:22 states, winds near the surface, winds near the upper atmosphere, water vapor species that are very
0:09:27 absolute distributions. But we’re, you know, trying to take the, learn a dynamical operator that predicts
0:09:31 the next time step and then feed that back into itself and auto-aggressively roll out weather
0:09:36 forecasts. That’s how AI, full AI weather forecasting works. I’m not very skeptical of that three years ago.
0:09:41 You know, the world has been convinced that this is the better way to do AI weather forecasts.
0:09:46 They’re more skillful. The most skillful forecasts in the world now come from AI systems that are
0:09:50 indiscutable. We don’t understand fully how they’re making their predictions, but they’re
0:09:55 easy to score and convince yourself they’re better. Yeah. So I, you know, my team of climate demand
0:10:00 experts at NVIDIA, you know, talks to the developers of these AI weather forecasting systems. And then
0:10:05 we started collaborating with generative AI experts who are accustomed to doing amazing things with video
0:10:09 and imagery and asking, could we do similar things with our data? So we wanted to have collectivity with
0:10:14 super resolution. The world has a lot of blurry predictions of the future, but the world needs a lot more
0:10:18 fine scale, super resolved versions of those predictions because, you know, climate impacts
0:10:22 yourself on the scales, kilometers, and infrastructure. And so we asked, could we take some
0:10:28 blurry climate data of one form and use generative AI to turn it into very highly resolved versions of the
0:10:33 same variables, including all the distribution chips involved, but also to synthesize new variables that
0:10:38 you can only observe at high resolution, like radar reflectivity observations of rainfall.
0:10:42 You know, this just blew my mind. It worked so much better than I expected. So now we have this thing
0:10:48 called Cordiff. It does multivariate super resolution and new channel synthesis and has been used in lots
0:10:52 of different settings to take the pain out of generating high resolution state estimates of the
0:10:58 atmosphere. When you talk about predicting weather, predicting climate, is there a percentage number
0:11:04 in terms of accuracy or a way of kind of putting your finger on how, what’s our accuracy like right
0:11:10 now? And how is it maybe changed? And I would assume improved since the advent of, you know,
0:11:13 advanced AI and now generative AI in the process.
0:11:18 Oh, that’s a great question. Yeah. For weather forecasting, it’s very clear that the pace of
0:11:21 increasing in steel is outpacing what we’ve seen with physics models. So something called like the
0:11:26 quiet resolution weather forecasting, you can look back on the last two decades and ask how has weather
0:11:29 forecasting still gotten better over time? And it’s gotten better, but, you know, slowly and
0:11:34 asymptotically, and it seems to be not getting better much for these days. And there’s a version of this
0:11:38 graph. I just saw that one of the leaders at the European Center for Medium Weather Forecasting
0:11:43 presented a conference that shows AI just leapfrogging all that trajectory. Yeah. So the weather
0:11:48 community has really, really seen this. You can measure this. For climate prediction, it’s difficult.
0:11:52 I mean, the answer is out of sample. The answer is in 50 years. We don’t know the answer. So what we’re
0:11:56 trying to predict are like a range of outcomes and concepts that are irrelevant to weather prediction,
0:12:01 like energy conservation, mass conservation, are really important to make sure your model obeys.
0:12:07 So I would say what’s going on now is this whole ecosystem effect where lots of developers are
0:12:13 working to take what’s happened in AI for weather prediction and use it to the benefit of climate
0:12:18 prediction. So one thing you can do is train an emulator, not of weather observations, but of
0:12:22 climate projections themselves. That’s working pretty well. Once you have that climate projections
0:12:27 that are normally difficult to pass around the world on the internet, it becomes very easy to give people
0:12:31 because you just give someone a trained AI model and they inference the climate much faster
0:12:35 than they would read a data set. Yeah. And there are some people who are trying to build
0:12:40 multi-component versions of these pure AI climate simulators that have the same properties we require
0:12:45 of climate prediction models. So correct coupling between the atmosphere and ocean, correct
0:12:51 conservation of mass and energy, correct responses to adding a little more radiative energy to the
0:12:56 system like CO2 gases do. So the community is hard at work trying to understand if we can build
0:13:00 a next generation of fully AI climate models, but it’s work in progress.
0:13:05 That’s great to hear. And so that kind of makes me think on the flip side, what do we do with these
0:13:09 predictions? What do we do with this information with the ability, as you put it, to inference
0:13:13 what the weather, perhaps what the climate might be like in a given amount of time?
0:13:21 We’ve had, I mean, this year already, wildfires in the West, tornadoes and other extreme storms kind
0:13:26 of in the middle and southern part of the U.S. and obviously globally, all kinds of extreme weather
0:13:32 events becoming more frequent in recent times. So how can advances in AI-driven climate modeling
0:13:38 help us better prepare, help communities better prepare for what’s coming and adapt to the change
0:13:38 in climate?
0:13:40 So I’ll give you two answers.
0:13:40 Okay.
0:13:46 One is like a fundamentally new ability to study the statistics of these rare extremes themselves.
0:13:50 By definition, these low likelihood eye impact extremes, they’re rare in the observed record.
0:13:53 We can’t study them in observations because the sample sizes are tiny.
0:13:59 We could generate, you know, hundreds or thousands of counterfactual low likelihood extremes using
0:14:04 physical simulators, but that’s too extensive. So the speed up of AI models, this thousand
0:14:09 X increase in inference speed that we get, has opened up a new science of huge ensemble counterfactual
0:14:13 generation. There’s a pair of papers I love led by Berkeley that we collaborate on with NVIDIA.
0:14:18 They’ve taken our AI model forecast net. They’ve calibrated it carefully so that it produces
0:14:23 the right probability distribution, a trustworthy probability distribution to study heat extremes.
0:14:27 And then they’ve run like 28,000 years worth of equivalent summer 2023.
0:14:31 And so instead of one forecast every day, we’ve got 7,000 every day.
0:14:36 And so you can find so many different counterfactuals of how 2023 could have turned out.
0:14:40 And, you know, this, this is a new type of data we couldn’t generate before where we could sort of
0:14:45 start to have satisfying sample sizes of even rare heat waves to begin to understand their climate
0:14:49 drivers, to inform our understanding of how those statistics are going to change in the future.
0:14:53 So that’s a mature technology that exists. Insurance companies are already starting to use it.
0:14:57 Insurance companies normally just receive private projections, but they’re starting to generate them
0:15:02 internally because it’s so easy with AI. And then the other thing is just improved weather forecasting
0:15:06 itself, more interactive weather forecasting, more steerable forecasting. My team’s just put out
0:15:11 something I’m really excited about called Climate in a Model. It’s opened my mind to the fact that
0:15:15 there’s potential to do so much more than physics simulations, which is, you know, those are a two-stage act.
0:15:20 Generate output, look at output and finding hurricanes you care about is like a needle in a haystack.
0:15:25 But AI lets us say, you know, I’d like a weather for, I’d like to find a hurricane, please. I’d like to
0:15:29 find a hurricane in a future climate. Just give me a sample of that because that’s what I’m interested
0:15:34 in. So this, I think that will increase the ability of everyone to interact with climate predictions.
0:15:40 You know, imagine a future where you can be sitting on the, in Florida, looking at a seed of a potential
0:15:45 future cyclone way out in the Eastern Atlantic and say, give me only weather forecasts that produce a really
0:15:51 bad version of that hurricane, so I can plan for the worst. That’s, that’s normally really hard to do with
0:15:55 normal simulation technology. But, but AI, I totally see that’s going to be possible.
0:16:01 Yeah. And so are, you know, governmental, you mentioned insurance companies, of course, but are governmental
0:16:09 leaders, governmental organizations, disaster response organizations, are they using, or if not, how can they use these
0:16:15 interactive climate simulators to inform their responses and inform, you know, the policy decisions
0:16:20 for the future? I think it’s going to be a process in their understanding their implications. The whole
0:16:25 world is grappling with the interactivity dimension right now. I know, so I have a, we’re fortunate in
0:16:30 the Earth 2 Initiative NVIDIA to have some really inspiring European science advisors. One of them is
0:16:34 named Bjorn Stevens. He gave a wonderful keynote at the International Supercompetitive Conference a couple
0:16:40 weeks ago. And what he sees is a future where there’s a source of credible, appropriate data to
0:16:46 use for future climate prediction, like for a cutting edge, best in class physics model, and then AI on top
0:16:52 of it to broaden people’s reach into what the data produces. And as long as the AI is faithful to the
0:16:58 training data, then it solves a liability concern where, you know, climate actors need to have data
0:17:03 provenance, need to have provenance to credible sources of impacts related decisions. So I think
0:17:07 where, where we’re researching and where we’re working on right now is building that AI on top to
0:17:13 understand, you know, what is it going to be appropriate to be able to ask a trove of simulation
0:17:18 output for in the future? I know like, I know I can ask for tropical cyclones right now, but I don’t
0:17:23 know if I could ask for other things I’d like to ask for, like, hey, show me, show me a counterfactual of
0:17:28 two compound hurricanes pummeling into Bermuda one after the other in today’s climate and in
0:17:33 2070. But as these capabilities emerge, I think we’ll see what the ecosystem wants to do with them.
0:17:38 Yeah. Otherwise, I see, you know, climate tech startups doing sensible things like taking AI for
0:17:43 state estimation to prove their value prop of proprietary data streams, improving state estimates
0:17:48 for forecasting applications. I’m speaking with Mike Pritchard. Mike is the director of climate
0:17:54 simulation research in NVIDIA. And we’ve been talking about the way that AI has really, I hate
0:18:00 to use this word, but disrupted climate science and simulation, weather forecasting as well. Mike,
0:18:04 in the intro, I referenced digital twins, which is something we’ve talked about on the show,
0:18:12 mainly in reference to industrial AI simulating the way that factories operate for planning purposes and
0:18:16 so on. How do digital twins play into climate science?
0:18:21 Yeah. Well, so we already think of our physical simulators of the future of the planet as sort of
0:18:26 digital twins. If they can be used for scenario planning, you can ask what if questions, you know,
0:18:30 what if humans evolve the land surface this way and emit this much guns in the atmosphere versus what
0:18:36 if they follow this mitigation scenario. So AI as a surrogate for physical climate models on top of
0:18:42 physical climate predictions could be viewed as sort of reducing the latency associated with our familiar
0:18:47 digital twins. But I think you’re after something bigger, which is, you know, what I observe in
0:18:51 some of NVIDIA’s connections to industrial infrastructure work is this ability to kind of
0:18:56 backpropagate and differentiate through, ultimately optimize the control of very complex systems.
0:19:02 And so there, I think it’s really exciting to imagine, you know, a digital twin of an electrical grid
0:19:07 that coupled to digital twins of individual power stations, eventually coupled to digital twins,
0:19:12 maybe there are existing AI forecast models, differentiable, like autoregressors of the entire
0:19:16 atmosphere, coupled to digital twins of climate. And you can chain those together and backpropagate
0:19:21 through the whole stack, then maybe we can play games we couldn’t have dreamed of before to sort of,
0:19:26 you know, optimize for resilience in the face of slowly varying low likelihood climate extremes.
0:19:32 So let’s switch gears for a second. I want to talk about a talk that you gave, a recent TED talk,
0:19:37 where you were talking about the importance of AI in climate simulation. Why give a TED talk? What
0:19:43 was the message that you were hoping to leave the audience, the public with from your talk?
0:19:48 I think just that something very exciting is going on in the business of simulating the atmosphere
0:19:53 that nobody would have predicted 10 years ago. This fundamentally new simulation technology,
0:19:56 but there’s properties we don’t even understand yet, but very enticing characteristics like,
0:20:01 yeah, interactivity. I just wanted to get that message out there. So people are aware that,
0:20:07 you know, that that’s a really beneficial use of AI for a really important problem that affects us all
0:20:12 weather prediction and climate prediction. You know, it’s wonderful to see this like cocoon of people
0:20:17 working across academia and government labs and industry, very complimentary skill sets that are
0:20:21 needed to bring that into existence and achieve its full potential. So that was one of the messages.
0:20:27 But yes, I think the, I’ve touched on some of the remarkable characteristics of AI for simulation here,
0:20:32 but not some that I’m recalling. I mentioned this TED talk. I think one of the amazing things is the
0:20:37 ability to use more data. So we have so many observations of the Earth’s system. It’s actually
0:20:42 really hard to incorporate them into physics models because I mentioned humans have to write
0:20:46 their assumptions of how unresolved things work like clouds, and that can easily clash with real
0:20:51 information about clouds. And you have to eventually sort of sacrifice observations you wish you could
0:20:55 use because they just think they shock your model and you can’t use your model for the things you’d
0:21:00 like to use it for anymore. So AI, by having no humans write down anything, the physical equations
0:21:06 are gone. It just opens up this huge filter that we didn’t used to think we could into how many
0:21:11 observations we could actually use to benefit our simulations. And so there’s exciting work going on
0:21:17 at the European Center now that is using only observations, satellite observations and station
0:21:22 observations as the sources of AI prediction. I think that’s the future. And as we, you know,
0:21:27 forecasting is two problems. It’s state estimation. How good do you know your initial condition?
0:21:32 And prediction. How well can you predict from there? And the unification of AI for improving state
0:21:36 estimation and prediction is happening as we speak. And we don’t know the potential. We don’t know what
0:21:41 the predictability limits of the Earth system are going to look like in 10 years. So it’s a very
0:21:44 exciting time to be alive. That’s what I wanted to convey in 10. It’s an exciting time.
0:21:51 It is. Listening you talk about that makes me think of episodes we’ve done involving astronomy
0:21:58 and involving healthcare and medical imaging and guests talking about how there’s so much data
0:22:05 in existing images, medical images taken, you know, years ago, the images coming back from,
0:22:10 you know, the James Webb telescope. And there’s so much data that, you know, kind of to your point,
0:22:15 humans making observations and writing things down and doing calculations can only go so far.
0:22:22 The AI systems can really, you know, dig into these massive data sets and dig into all the data in a
0:22:28 single image. And in the case of medical imaging, remember we had a guest talking about finding a new
0:22:35 biomedical marker just by going through preexisting imagery. Is there a similar phenomena that happens
0:22:41 with weather observations and weather imagery? Are you able to, you know, find things in kind of the
0:22:46 historical observations that, you know, we just didn’t know were there before? And then they’re able to
0:22:53 inform kind of, you know, getting a better future state understanding that that informs your predictions?
0:23:04 It’s happening as we speak, you know, there’s a pair of intellectual paper on the University of Washington that’s claiming to update the long held notions of how long the predictability limit of the atmosphere is.
0:23:08 And that’s causing a lot of controversial chatter in the community. So yeah, I think the questions you’re
0:23:17 speaking to are happening right now in our community. There are some timescales that are very difficult to predict on, like beyond a week or so when the atmosphere loses its memory.
0:23:28 But when there’s other components of the system complex components, the ocean surface, the ecosystems that have memory that should give us some predictability on three, four, even five week timescales.
0:23:36 That’s where I really hope we’ll have new scientific understanding as AI teachers is how much predictability has been hiding on those timescales all this time.
0:23:43 But you mentioned data and I have to say, like, we have a unique problem in the earth system sciences, which is there’s only so many decades of satellite observations.
0:23:51 And so we don’t have the whole internet of images and video to feed off of. At the end of the, you know, at the end, we have like tens of thousands of days of satellite observations.
0:23:56 And so there are still, there are inevitably going to be some phenomena where the sample sizes are just too small.
0:23:59 You know, there’s only been so many El Nino events in those 50 years.
0:24:10 There’s only been so many, let alone future climate change, you know, how the, you know, some things are inherently out of sample of the observations that feed our models, like the slow modes of the ocean dynamics.
0:24:13 And the ocean becomes the most important controls climate change by 2080.
0:24:25 So in my mind, there’s, there’s always going to be a combination of physical simulators that can, you know, are fit for purpose for those deep time projections alongside observations that I agree with you have not been fully exploited by humans.
0:24:29 And what that combination is going to be is really fascinating to think about.
0:24:34 Does synthetic data play a significant role in, in climate science?
0:24:35 Absolutely.
0:24:47 Yeah, we, you know, we can leverage our many simulations of the future or many past forecasts of weather as tree training objects to try to, you know, get the model as ready as it can be.
0:24:52 Use its memory capacity as efficiently as possible for ultimate fine tuning on the observations of high quality.
0:24:53 Yeah, that’s an active area of research.
0:25:00 So looking ahead, and you kind of talked a little bit about some of these things before, so feel free to kind of recap, if you will.
0:25:07 But what are some of the next big things that you, your colleagues, kind of the climate science community are working on?
0:25:16 And specifically with your work at NVIDIA, are there any big research questions that you’re tackling right now or hoping to get to in the days and months to come?
0:25:20 So around the world, I’m really excited by what’s going on with, with ocean emulation.
0:25:24 We’ve seen the atmosphere and the weather community really do great things.
0:25:27 And the ocean simulation community is starting to do great things as well.
0:25:30 And the unification of these is, it’s a really exciting frontier.
0:25:38 You mentioned a minute ago, the ocean, and I don’t want to get the words wrong, but the ocean kind of being the, the primary controller of climate.
0:25:39 Is that how you put it?
0:25:40 Yeah.
0:25:44 You know, the vast majority of the excess heat that we’ve put into the earth system is absorbed by the ocean.
0:25:54 And the memory of where it goes, where the marine heat waves happen in the future will largely stop our experience of, you know, inter-annual heat extremes and challenging extreme events.
0:25:55 Yeah.
0:25:59 So the, the ocean is really important to include in any prediction of climate.
0:25:59 Right.
0:26:03 So the advent of full ocean emulators is just happening this year and last year.
0:26:04 Oh, that’s amazing.
0:26:06 And they’re coupling the full atmosphere emulators.
0:26:07 It’s just happening.
0:26:12 It’s like in the 1980s when people first coupled physics-based atmosphere and ocean models, which used to be separate communities.
0:26:16 And out came the world’s first El Nino predictions and seasonal forecasting capabilities.
0:26:17 Yeah.
0:26:20 This is like climate model development on like, on, on, on an accelerated timeline.
0:26:24 It feels like the last three years are like decades compressed into a few.
0:26:27 At NVIDIA, I’m really excited about this interactivity dimension.
0:26:28 It, it stuns me.
0:26:30 So we have this thing called climate in a bottle.
0:26:33 It’s a paper that our scientists put out a month ago.
0:26:41 And, um, it’s not, it’s not like these other models I’ve described where you’re trying to generate videos and outputs to come inputs.
0:26:43 That feels very weathery to me.
0:26:46 What feels climate-y is saying, hey, here’s the climate boundary conditions.
0:26:49 Here’s what should be controlling weather noisily.
0:26:52 And give me some samples of weather subject to those boundary conditions.
0:27:02 So this climate in a bottle research paper demonstrates a model like this, where the user can pass in just three pieces of information, like time of day, time of year, that controls where the sun is.
0:27:04 And how warm is the ocean?
0:27:07 I mentioned the ocean largely regulates the atmosphere on climate times again.
0:27:12 That’s very low dimensional information, like a text query to a large language model.
0:27:13 200 kilobytes of information.
0:27:23 Out of the other end of this thing comes 600 megabytes across 12 different variables of 13 million pixels of state estimations consistent with those inputs.
0:27:29 And if you look at the climatology of that system, like you generate tens of thousands of samples and say, how’s the seasonal cycle?
0:27:30 How’s the daily cycle?
0:27:33 How are all the modes of variability I care about as a climatologist?
0:27:34 They look good.
0:27:36 And then now here’s the real kicker.
0:27:41 We’ve added the ability to steer this generation to events of interest.
0:27:45 So one of our scientists trained a tropical cyclone classifier alongside the generator.
0:27:50 And now we’re able to pass in a fourth ingredient, a map, of where I would like to see cyclones.
0:27:53 And this is, to me, I’ve never seen anything like this.
0:27:58 You know, I’m used to having to sift through, you know, petabytes of output to find a few tropical cyclones I care about.
0:28:04 I just feel to ask for one and to have the result be consistent with the statiotemporal dynamics of the data it was trained on.
0:28:12 That’s the tip of an iceberg of new interactivity features of queriable time and informatics that I think is going to feel very different than what we’re used to in five years.
0:28:14 So, yeah, that’s where I’m focusing a lot of attention right now.
0:28:15 Amazing.
0:28:22 If listeners, if you’re listening on audio and can’t see the video, Mike’s face is consistently lighting up as he’s talking about some of these things.
0:28:42 Mike, to leave the listeners with kind of a call to action here, whether there are technologists or just concerned citizens who have been listening to the conversation, and they, we, speaking for myself, want to know how we can get involved, how we can support the advancement of climate science, how we can support sustainability through AI.
0:28:43 What advice would you have?
0:28:43 What can we do?
0:28:51 Oh, I think like any concerned citizen, you know, like contact your politicians if you would like to, you know, exert your preferences about where priorities are.
0:28:57 I feel like a lot of priorities of funding are currently in debate and there’s, you know, rational questions of efficiency going on.
0:29:01 But one thing that’s very important to us all is our sustained observations of the Earth system.
0:29:04 Machine learning is nothing without the data it’s trained on.
0:29:20 And so, yeah, if you value the observing systems that have led and the weather modeling agencies that have led to the data sets that are leading to this amazing revolution in AI prediction quality, express that to the people in charge of the resources that allow those observations to exist.
0:29:22 And then, you know, you can tinker.
0:29:27 If you’re a Kaggle competition enthusiast, you know, pick a climate one.
0:29:34 If you’re an ML researcher and you’d like to plug into a benchmark, we’ve got a few climate ones that I think are still unsolved benchmarks that could use your attention.
0:29:35 Yeah.
0:29:37 And I don’t know what else to say to your question.
0:29:39 You know, just maybe have conversations about it.
0:29:40 The planet is warming.
0:29:41 We’re a part of that.
0:29:43 It’s going to be a part of our future, our kids’ future.
0:29:45 A lot of the warming has already happened.
0:29:49 You know, the average lifetime of a carbon dioxide molecule in the atmosphere is 100 years.
0:29:53 And so the past 200 years of emissions will definitely be warming our planet in the future.
0:29:56 And we all need to think about what we do with that.
0:30:05 I think that AI will allow us to appreciate, by being able to interact more natively, with predictions of the future, the complex system that we’re embedded in.
0:30:08 I think just that awareness is good to talk with your friends about.
0:30:13 Yeah, that is a powerful mechanism for understanding things closer to firsthand, for sure.
0:30:20 Mike, for listeners who would like to know more, would like to follow your work and your team’s work, where’s a good place to go online?
0:30:24 NVIDIA Research, or where can we send people to investigate further?
0:30:26 Yeah, look up NVIDIA Earth 2.
0:30:27 Earth 2 is the name of our initiative.
0:30:29 And there’s all sorts of wonderful software in there.
0:30:36 There’s recipes for many of the AI simulation technologies I’ve mentioned, open source recipes that we’re hoping the ecosystem builds upon.
0:30:37 There’s training materials.
0:30:40 So yeah, Earth 2 is the thing to look up.
0:30:41 Fantastic.
0:30:44 Mike Pritchard, again, thank you so much for taking the time to join the podcast.
0:30:47 It’s a fascinating conversation.
0:31:00 And despite kind of the seriousness of climate change and extreme weather and such, you’ve left me with a lot of optimism about the technologies kind of, as you said, you know, the past few years have just been super jam-packed with advancements.
0:31:09 And the ability to use these to help us understand and adapt to the climate is at the top of the list, in my view, of the importance of this work.
0:31:11 So yeah, what a time to be alive, as you said.
0:31:12 Yeah, thanks, Noah.
0:31:21 It’s a real privilege for me to represent a huge team of hardworking, humble, really earnest technologists and engineers and scientists that are working together on these problems.
0:31:25 I’m just a mouthpiece, but yeah, any credit should go to them.
0:31:26 Absolutely.
0:31:27 Well, thank you to your whole team.
0:31:27 Thanks, Noah.
0:32:17 I’m just a mouthpiece, but yeah, any credit should go to them.
Mike Pritchard, Director of Climate Simulation Research at NVIDIA, discusses how artificial intelligence is enhancing the way we model and predict climate patterns. From developing high-resolution, interactive simulations to accelerating climate forecasts, AI is enabling scientists, policymakers, and businesses to gain a deeper understanding of extreme weather and long-term climate change. Learn more at ai-podcast.nvidia.com.
Leave a Reply
You must be logged in to post a comment.