0:00:00 The following is a conversation with Roman Yumpolski, an AI safety and security researcher
0:00:05 and author of a new book titled AI, unexplainable, unpredictable, uncontrollable.
0:00:13 He argues that there’s almost 100% chance that AGI will eventually destroy human civilization.
0:00:19 As an aside, let me say that I will have many often technical conversations on the topic
0:00:25 of AI, often with engineers building the state of the art AI systems.
0:00:31 I would say those folks put the infamous P-DOOM or the probability of AGI killing all humans
0:00:36 at around 1 to 20%, but it’s also important to talk to folks who put that value at 70,
0:00:43 80, 90, and in the case of Roman at 99.99 and many more nines percent.
0:00:51 I’m personally excited for the future and believe it will be a good one, in part because
0:00:57 of the amazing technological innovation we humans create, but we must absolutely not
0:01:03 do so with blinders on, ignoring the possible risks, including existential risks of those
0:01:11 technologies.
0:01:12 That’s what this conversation is about.
0:01:16 And now a quick few second mention of each sponsor.
0:01:19 Check them out in the description, it’s the best way to support this podcast.
0:01:23 We got Yahoo Finance for Investors, Masterclass for Learning, NetSuite for Business, Element
0:01:29 for Hydration, and Aidsleep for Sweet Sweet Naps.
0:01:34 Choose what is it my friends.
0:01:36 Also, if you want to get in touch with me or for whatever reason, work with our amazing
0:01:41 team let’s say, just go to lexfreedman.com/contact.
0:01:45 And now on to the full ad reads, as always, no ads in the middle.
0:01:49 I try to make these interesting, but if you must skip them, friends, please still check
0:01:53 out our sponsors.
0:01:54 I enjoy their stuff, maybe you will too.
0:01:58 This episode is brought to you by Yahoo Finance, a site that provides financial management,
0:02:03 reports, information, and news for investors.
0:02:07 It’s my main go-to place for financial stuff.
0:02:10 I also added my portfolio to it.
0:02:13 I guess it used to be TD Ameritrade and that got transported, transformed, moved to Charles
0:02:21 Schwab.
0:02:22 I guess that was an acquisition of some sort.
0:02:25 I have not been paying attention.
0:02:28 All I know is I hate change and try and figure out the new interface of Schwab when I log
0:02:33 in once a year or however long I log in is just annoying.
0:02:39 Anyway, one of the ways to avoid that annoyance is tracking information about my portfolio
0:02:45 from Yahoo Finance so you can drag over your portfolio and in the same place find out all
0:02:52 the news, analysis, information, all that kind of stuff.
0:02:55 Anyway, for comprehensive financial news and analysis, go to yahoofinance.com.
0:03:01 That’s yahoofinance.com.
0:03:04 I don’t know why I whispered that.
0:03:06 This episode is also brought to you by Masterclass where you can watch over 180 classes from
0:03:12 the best people in the world and their respective disciplines.
0:03:16 We got Aaron Franklin on barbecue and brisket, something I watched recently, and I love brisket.
0:03:24 I love barbecue.
0:03:25 It’s one of my favorite things about Austin.
0:03:28 It’s funny when the obvious cliche thing is also the thing that brings you joy.
0:03:33 So it almost doesn’t feel genuine to say, but I really love barbecue.
0:03:39 My favorite place to go is probably Terry Blacks.
0:03:41 I’ve had Franklin’s a couple of times.
0:03:43 It’s also amazing.
0:03:44 I’ve actually don’t remember myself having bad barbecue or even mediocre barbecue in
0:03:51 Austin.
0:03:52 So it’s hard to pick favorites because it really boils down to the experience you have
0:03:58 when you’re sitting there.
0:03:59 One of my favorite places to sit is Terry Blacks.
0:04:02 I have this, I don’t know, it feels like a tavern.
0:04:05 I feel like Cowboy.
0:04:08 I just robbed a bank in some town in the middle of nowhere in West Texas and I’m just sitting
0:04:15 down for some good barbecue and the sheriffs walk in and there’s a gunfight and all that,
0:04:19 as usual.
0:04:20 Anyway, get unlimited access to every Masterclass and get an additional 15% off an annual membership
0:04:25 by masterclass.com/lexpod.
0:04:32 This episode is also brought to you by NetSuite, an all-in-one cloud business management system.
0:04:38 One of the most fulfilling things in life is the people you surround yourself with, just
0:04:43 like in the movie 300.
0:04:44 All it takes is 300 people to do some incredible stuff.
0:04:51 But they all have to be shredded.
0:04:53 It’s really, really important to always be ready for war in physical and mental shape.
0:05:03 No, not really, but I guess if that’s your thing, happiness is the thing you should be
0:05:10 chasing and there’s a lot of ways to achieve that.
0:05:13 For me, being in shape is one of the things that make me happy because I can move about
0:05:18 the world and have a lightness to my physical being if I’m in good shape.
0:05:24 Anyway, I say all that because getting a strong team together and having them operate is an
0:05:30 efficient, powerful machine is really important for the success of the team, for the happiness
0:05:38 of the team, and the individuals in that team.
0:05:41 NetSuite is a great system that runs the machine, inside the machine, for a knee-sized business.
0:05:48 37,000 companies have upgraded to NetSuite by Oracle.
0:05:54 Take advantage of NetSuite’s flexible financing plan at netsuite.com/lex.
0:05:58 That’s netsuite.com/lex.
0:06:02 This episode was also brought to you by Element, electrolyte, drink mix of sodium potassium
0:06:08 and magnesium that I’ve been consuming multiple times a day.
0:06:12 Watermelon salt is my favorite.
0:06:14 Whenever you see me drink from a cup on the podcast, almost always it’s going to be water
0:06:19 with some element in it.
0:06:20 I use an empty Powerade bottle, 28 fluid ounces, filled with water, put one packet of watermelon
0:06:27 salt element in it, mix it up, put it in the fridge, and when it’s time to drink, I take
0:06:33 it out of the fridge and I drink it, and I drink a lot of those a day, and it feels good,
0:06:37 it’s delicious, whenever I do crazy physical fasting, all that kind of stuff, the element
0:06:45 is always by my side, and more and more, you’re going to see probably the sparkling water
0:06:50 thing or whatever the element is making, so it’s in a can, and it’s freaking delicious.
0:06:55 There’s four flavors.
0:06:58 The lemon one is the only one I don’t like, the other three I really love, and I forget
0:07:01 their names, but they’re freaking delicious, and you’re going to see it more and more on
0:07:05 my desk, except for the fact that I run out very quickly because I consume them very quickly.
0:07:10 Get a simple pack for free with any purchase, try it at drinkelement.com/lex.
0:07:16 This episode is also brought to you by A.Sleep, and it’s pod for ultra.
0:07:20 This thing is amazing, the ultra part of that adds a base that goes between the mattress
0:07:26 and the bed frame and can elevate to like a reading position, so it modifies the positioning
0:07:31 of the bed.
0:07:32 On top of all the cooling and heating and all that kind of stuff they can do, and do
0:07:37 it better in the pod four, I think it has two X the cooling power of pod three, so they’re
0:07:41 improving on the main thing that they do, but also there’s the ultra part that can adjust
0:07:47 the bed.
0:07:48 It can cool down each side of the bed to 20 degrees Fahrenheit below room temperature.
0:07:55 One of my favorite things is to escape the world on a cool bed with a warm blanket.
0:08:01 And just disappear for 20 minutes or for eight hours into a dream world where everything
0:08:08 is possible, where everything is allowed.
0:08:12 It’s a chance to explore the Jungian shadow, the good, the bad, and the ugly, but it’s
0:08:19 usually good and it’s usually awesome.
0:08:21 And I actually don’t dream that much but when I do it’s awesome.
0:08:25 The whole point though is that I wake up refreshed, taking your sleep seriously is really, really
0:08:31 important.
0:08:32 When you get a chance to sleep, do it in style and do it on a bed that’s awesome.
0:08:38 Go to sleep.com/lex and use code Lex to get $350 off the pod four ultra.
0:08:47 This is the Lex Friedman podcast to support it, please check out our sponsors in the
0:08:51 description.
0:08:52 And now dear friends, here’s Roman Yampalski.
0:08:56 What do you, is the probability that super intelligent AI will destroy all human civilization?
0:09:18 What’s the time frame?
0:09:19 Let’s say a hundred years, in the next hundred years.
0:09:22 So the problem of controlling a GI or super intelligence, in my opinion, is like a problem
0:09:30 of creating a perpetual safety machine, by analogy with perpetual motion machine, it’s
0:09:35 impossible.
0:09:36 Yeah, we may succeed and do a good job with GPT-5, 6, 7, but they just keep improving,
0:09:46 learning, eventually self modifying, interacting with the environment, interacting with malevolent
0:09:53 actors.
0:09:55 The difference between cybersecurity, narrow AI safety and safety for general AI for super
0:10:01 intelligence is that we don’t get a second chance.
0:10:04 With cybersecurity, somebody hacks your account, what’s the big deal?
0:10:07 You get a new password, new credit card, you move on.
0:10:11 Here, if we’re talking about existential risks, you only get one chance.
0:10:15 So you’re really asking me, what are the chances that we’ll create the most complex software
0:10:21 ever on the first try with zero bugs and it will continue have zero bugs for a hundred
0:10:28 years or more?
0:10:31 So there is an incremental improvement of systems leading up to AGI.
0:10:38 To you, it doesn’t matter if we can keep those safe, there’s going to be one level
0:10:44 of system at which you cannot possibly control it.
0:10:49 I don’t think we so far have made any system safe.
0:10:54 At the level of capability they display, they already have made mistakes.
0:11:00 We had accidents, they’ve been jailbroken.
0:11:03 I don’t think there is a single large language model today which no one was successful at
0:11:09 making do something developers didn’t intend it to do.
0:11:13 But there’s a difference between getting it to do something unintended, getting it to
0:11:17 do something that’s painful, costly, destructive, and something that’s destructive to the level
0:11:22 of hurting billions of people, or hundreds of millions of people, billions of people,
0:11:28 or the entirety of human civilization.
0:11:30 That’s a big leap.
0:11:31 Exactly, but the systems we have today have capability of causing X amount of damage.
0:11:37 So when they fail, that’s all we get.
0:11:39 If we develop systems capable of impacting all of humanity, all of universe, the damage
0:11:46 is proportionate.
0:11:48 What do you, are the possible ways that such kind of mass murder of humans can happen?
0:11:55 It’s always a wonderful question.
0:11:58 So one of the chapters in my new book is about unpredictability.
0:12:01 I argue that we cannot predict what a smarter system will do.
0:12:05 So you’re really not asking me how superintelligence will kill everyone.
0:12:09 You’re asking me how I would do it.
0:12:11 And I think it’s not that interesting.
0:12:13 I can tell you about the standard, you know, nanotech, synthetic, bionuclear.
0:12:18 Superintelligence will come up with something completely new, completely super.
0:12:23 We may not even recognize that as a possible path to achieve that goal.
0:12:28 So there is like a unlimited level of creativity in terms of how humans could be killed.
0:12:36 But you know, we could still investigate possible ways of doing it.
0:12:41 Not how to do it, but the, at the end, what is the methodology that does it?
0:12:46 You know, shutting off the power and then humans start killing each other, maybe because
0:12:51 the resources are really constrained that they’re, and then there’s the actual use of
0:12:55 weapons like nuclear weapons or developing artificial pathogens, viruses, that kind of
0:13:01 stuff.
0:13:03 We could still kind of think through that and defend against it, right?
0:13:07 There’s a ceiling to the creativity of mass murder of humans here, right?
0:13:11 The options are limited.
0:13:13 They are limited by how imaginative we are.
0:13:16 If you are that much smarter, that much more creative, if you are capable of thinking across
0:13:20 multiple domains, do novel research in physics and biology, you may not be limited by the
0:13:25 tools.
0:13:26 If squirrels were planning to kill humans, they would have a set of possible ways of
0:13:31 doing it, but they would never consider things we can come up.
0:13:34 So are you thinking about mass murder and destruction of human civilization?
0:13:38 Are you thinking of with squirrels, you put them in a zoo and they don’t really know
0:13:42 they’re in a zoo?
0:13:43 If we just look at the entire set of undesirable trajectories, majority of them are not going
0:13:48 to be death.
0:13:50 Most of them are going to be just like things like Brave New World, where the squirrels are
0:13:58 fed dopamine, and they’re all doing some kind of fun activity, and the fire, the soul of
0:14:05 humanity is lost because of the drug that’s fed to it, or literally in a zoo.
0:14:10 We’re in a zoo, we’re doing our thing, we’re playing a game of sims, and the actual players
0:14:18 playing that game are AI systems.
0:14:20 Those are all undesirable because of the free will.
0:14:24 The fire of human consciousness is dimmed through that process, but it’s not killing humans.
0:14:30 So are you thinking about that, or is the biggest concern, literally, the extinctions
0:14:36 of humans?
0:14:37 I think about a lot of things.
0:14:39 So there is ex-risk, existential risk, everyone’s dead.
0:14:43 There is ex-risk, suffering risks, where everyone wishes they were dead.
0:14:48 We have also idea for iris, ikigai risks, where we lost our meaning.
0:14:53 The systems can be more creative, they can do all the jobs.
0:14:57 It’s not obvious what you have to contribute to a world where superintelligence exists.
0:15:02 Of course, you can have all the variants you mentioned, where we are safe, we are kept
0:15:07 alive, but we are not in control.
0:15:09 We are not deciding anything, we are like animals in a zoo.
0:15:13 There is, again, possibilities we can come up with as very smart humans, and then possibilities
0:15:19 something a thousand times smarter can come up with.
0:15:23 For reasons we cannot comprehend.
0:15:25 I would love to sort of dig into each of those, ex-risk, ex-risk, and iris.
0:15:30 So can you like linger on iris?
0:15:33 What is that?
0:15:34 You Japanese concept of ikigai, you find something which allows you to make money, you are good
0:15:41 at it, and the society says we need it.
0:15:44 So like you have this awesome job, you are a podcaster, gives you a lot of meaning, you
0:15:49 have a good life, I assume, you’re happy.
0:15:54 That’s what we want most people to find, to have.
0:15:56 For many intellectuals, it is their occupation which gives them a lot of meaning.
0:16:02 I am a researcher, philosopher, scholar, that means something to me.
0:16:07 In a world where an artist is not feeling appreciated because his art is just not competitive with
0:16:14 what is produced by machines, or writer, or scientist, we’ll lose a lot of that.
0:16:22 And at the lower level, we’re talking about complete technological unemployment.
0:16:26 We’re not losing 10% of jobs, we’re losing all jobs.
0:16:30 What do people do with all that free time?
0:16:32 What happens then?
0:16:35 Everything society’s build on is completely modified in one generation.
0:16:40 It’s not a slow process where we get to kind of figure out how to live that new lifestyle,
0:16:46 but it’s pretty quick.
0:16:48 In that world, humans do what humans currently do with chess, play each other, have tournaments,
0:16:55 even though AI systems are far superior at this time in chess.
0:17:00 So we just create artificial games.
0:17:02 Or for us, they’re real, like the Olympics, we do all kinds of different competitions
0:17:07 and have fun, maximize the fun and let the AI focus on the productivity.
0:17:17 It’s an option.
0:17:18 I have a paper where I try to solve the value alignment problem for multiple agents.
0:17:23 And the solution to avoid compromises is to give everyone a personal virtual universe.
0:17:28 You can do whatever you want in that world.
0:17:30 You could be king, you could be slave, you decide what happens.
0:17:33 So it’s basically a glorified video game where you get to enjoy yourself and someone
0:17:37 else takes care of your needs and the substrate alignment is the only thing we need to solve.
0:17:44 We don’t have to get 8 billion humans to agree on anything.
0:17:48 So why is that not a likely outcome?
0:17:52 Why can’t AI systems create video games for us to lose ourselves in, each with an individual
0:17:58 video game universe?
0:18:01 Some people say that’s what happened, we’re in a simulation.
0:18:04 And we’re playing that video game and now we’re creating artificial threats for ourselves
0:18:11 to be scared about because fear is really exciting.
0:18:14 It allows us to play the video game more vigorously.
0:18:18 And some people choose to play on a more difficult level with more constraints.
0:18:23 Some say, okay, I’m just going to enjoy the game, high privilege level.
0:18:26 Absolutely.
0:18:27 So okay, what was that paper on multi-agent value alignment?
0:18:31 Personal universes.
0:18:33 Personal universes.
0:18:35 So that’s one of the possible outcomes.
0:18:37 But what in general is the idea of the paper, just looking at multiple agents, they’re human
0:18:42 AI, like a hybrid system, whether it’s humans and AI’s, or is it looking at humans or just
0:18:46 intelligent agents?
0:18:48 In order to solve value alignment problem, I’m trying to formalize it a little better.
0:18:53 Basically we’re talking about getting AI’s to do what we want, which is not well defined.
0:18:58 Are we talking about creator of a system, owner of that AI, humanity as a whole?
0:19:04 But we don’t agree on much.
0:19:06 There is no universally accepted ethics, morals across cultures, religions.
0:19:12 People have individually very different preferences politically and such.
0:19:15 So even if we somehow managed all the other aspects of it, programming those fuzzy concepts
0:19:21 and getting AI to follow them closely, we don’t agree on what to program in.
0:19:25 So my solution was, okay, we don’t have to compromise on room temperature.
0:19:29 You have your universe, I have mine, whatever you want.
0:19:33 And if you like me, you can invite me to visit your universe.
0:19:36 We don’t have to be independent, but the point is you can be.
0:19:39 And virtual reality is getting pretty good.
0:19:41 It’s going to hit a point where you can’t tell the difference.
0:19:44 And if you can’t tell if it’s real or not, what’s the difference?
0:19:47 So basically give up on value alignment, create an entire, it’s like the multiverse theory.
0:19:53 This is creating an entire universe for you with your values.
0:19:57 You still have to align with that individual.
0:19:59 They have to be happy in that simulation.
0:20:02 But it’s a much easier problem to align with one agent versus 8 billion agents plus animals,
0:20:07 aliens.
0:20:08 So you convert the multi agent problem into a single agent problem.
0:20:12 I’m trying to do that, yeah.
0:20:14 Okay.
0:20:15 Is there any way to, so, okay, that’s giving up on the value alignment problem.
0:20:22 Well is there any way to solve the value alignment problem where there’s a bunch of humans, multiple
0:20:28 humans, tens of humans or 8 billion humans that have very different set of values?
0:20:34 It seems contradictory.
0:20:35 I haven’t seen anyone explain what it means outside of kind of words which pack a lot,
0:20:43 make it good, make it desirable, make it something they don’t regret.
0:20:48 But how do you specifically formalize those notions?
0:20:50 How do you program them in?
0:20:52 I haven’t seen anyone make progress on that so far.
0:20:55 But isn’t that the whole optimization journey that we’re doing as a human civilization?
0:21:00 We’re looking at geopolitics.
0:21:03 Nations are in a state of anarchy with each other.
0:21:06 They start wars, there’s conflict.
0:21:11 And oftentimes they have a very different views of what is good and what is evil.
0:21:15 It’s not what we’re trying to figure out, just together trying to converge towards that.
0:21:20 So we’re essentially trying to solve the value alignment problem with humans.
0:21:24 Right.
0:21:25 But the examples you gave, some of them are, for example, two different religions saying
0:21:29 this is our holy site and we are not willing to compromise it in any way.
0:21:34 If you can make two holy sites in virtual worlds, you solve the problem.
0:21:38 And if you only have one, it’s not divisible.
0:21:41 You’re stuck there.
0:21:42 But what if we want to be attention with each other?
0:21:45 And that through that tension, we understand ourselves and we understand the world.
0:21:50 So that’s the intellectual journey we’re on as a human civilization, is we create intellectual
0:21:58 and physical conflict and through that figure stuff out.
0:22:01 If we go back to that idea of simulation and this is entertainment kind of giving meaning
0:22:06 to us.
0:22:07 The question is how much suffering is reasonable for a video game?
0:22:11 So yeah, I don’t mind a video game where I get haptic feedback, there is a little bit
0:22:15 of shaking, maybe I’m a little scared.
0:22:17 I don’t want a game where kids are tortured, literally.
0:22:23 That seems unethical, at least by our human standards.
0:22:26 Are you suggesting it’s possible to remove suffering if we’re looking at human civilization
0:22:31 as an optimization problem?
0:22:33 So we know there are some humans who, because of a mutation, don’t experience physical pain.
0:22:39 So at least physical pain can be mutated out, re-engineered out.
0:22:46 Suffering in terms of meaning, like you burn the only copy of my book, is a little harder.
0:22:51 But even there, you can manipulate your hedonic set point, you can change the faults, you
0:22:56 can reset.
0:22:58 Problem with that is if you start messing with your reward channel, you start wire heading.
0:23:03 And end up bleasing out a little too much.
0:23:07 Well, that’s the question.
0:23:09 Would you really want to live in a world where there’s no suffering, as a dark question?
0:23:15 Is there some level of suffering that reminds us of what this is all for?
0:23:22 I think we need that, but I would change the overall range.
0:23:26 So right now it’s negative infinity to kind of positive infinity, pain, pleasure, access.
0:23:30 I would make it like zero to positive infinity, and being unhappy is like I’m close to zero.
0:23:36 Okay, so what’s the S-risk?
0:23:39 What are the possible things that you’re imagining with S-risk, so mass suffering of humans?
0:23:44 What are we talking about there, caused by AGI?
0:23:47 So there are many malevolent actors, so we can talk about psychopaths, crazies, hackers,
0:23:53 doomsday cults.
0:23:55 We know from history they tried killing everyone.
0:23:58 They tried on purpose to cause maximum amount of damage, terrorism.
0:24:03 What if someone malevolent wants on purpose to torture all humans as long as possible?
0:24:08 You solve aging, so now you have functional immortality, and you just try to be as creative
0:24:15 as you can.
0:24:16 Do you think there is actually people in human history that try to literally maximize human
0:24:22 suffering?
0:24:23 It’s just starting people have done evil in the world.
0:24:26 It seems that they think that they’re doing good, and it doesn’t seem like they’re trying
0:24:30 to maximize suffering.
0:24:33 They just cause a lot of suffering as a side effect of doing what they think is good.
0:24:39 So there are different malevolent agents.
0:24:42 Some may be just gaining personal benefit and sacrificing others to that cause.
0:24:48 Others we know for a fact are trying to kill as many people as possible when we look at
0:24:53 recent school shootings.
0:24:54 If they had more capable weapons, they would take out not dozens, but thousands, millions,
0:25:03 billions.
0:25:06 Well we don’t know that, but that is a terrifying possibility, and we don’t want to find out.
0:25:14 Like if terrorists had access to nuclear weapons, how far would they go?
0:25:19 Is there a limit to what they’re willing to do?
0:25:25 In your senses, there are some malevolent actors where there’s no limit.
0:25:29 There is mental diseases where people don’t have empathy, don’t have this human quality
0:25:40 of understanding, suffering, and ours.
0:25:42 And then there’s also a set of beliefs where you think you’re doing good by killing a lot
0:25:47 of humans.
0:25:48 Again, I would like to assume that normal people never think like that.
0:25:52 It’s always some sort of psychopaths, but yeah.
0:25:56 And to you, AGI systems can carry that and be more competent at executing that?
0:26:04 They can certainly be more creative.
0:26:06 They can understand human biology better, understand our molecular structure, genome.
0:26:12 Again, a lot of times torture ends, then individual dies.
0:26:19 That limit can be removed as well.
0:26:21 So if we’re actually looking at X-Risk and S-Risk as the systems get more and more intelligent,
0:26:26 don’t you think it’s possible to anticipate the ways they can do it and defend against
0:26:32 it like we do with the cybersecurity with the do security systems?
0:26:35 Right.
0:26:36 We can definitely keep up for a while.
0:26:38 I’m saying you cannot do it indefinitely.
0:26:41 At some point, the cognitive gap is too big.
0:26:44 The surface you have to defend is infinite, but attackers only need to find one exploit.
0:26:53 So to you, eventually, this is heading off a cliff.
0:26:57 If we create general super intelligences, I don’t see a good outcome long-term for humanity.
0:27:04 The only way to win this game is not to play it.
0:27:06 Okay, we’ll talk about possible solutions and what not playing it means.
0:27:12 But what are the possible timelines here to you?
0:27:14 What are we talking about?
0:27:15 We’re talking about a set of years, decades, centuries, what do you think?
0:27:20 I don’t know for sure.
0:27:21 The prediction markets right now are saying 2026 for AGI.
0:27:26 I heard the same thing from CEO of Anthropic, DeepMind, so maybe we’re two years away, which
0:27:31 seems very soon, given we don’t have a working safety mechanism in place or even a prototype
0:27:38 for one.
0:27:39 And there are people trying to accelerate those timelines because they feel we’re not getting
0:27:43 there quick enough.
0:27:44 Well, what do you think they mean when they say AGI?
0:27:47 So the definitions we used to have, and people are modifying them a little bit lately, artificial
0:27:53 general intelligence was a system capable of performing in any domain a human could perform.
0:27:59 So kind of you creating this average artificial person, they can do cognitive labor, physical
0:28:05 labor where you can get another human to do it.
0:28:08 Superintelligence was defined as a system which is superior to all humans in all domains.
0:28:13 Now people are starting to refer to AGI as if it’s superintelligence.
0:28:17 I made a post recently where I argued, for me at least, if you average out over all the
0:28:23 common human tasks, those systems are already smarter than an average human.
0:28:28 So under that definition, we have it.
0:28:31 Shane Lake has this definition of where you’re trying to win in all domains.
0:28:35 That’s what intelligence is.
0:28:37 Now are they smarter than elite individuals in certain domains?
0:28:41 Of course not.
0:28:42 They’re not there yet, but the progress is exponential.
0:28:46 See, I’m much more concerned about social engineering.
0:28:50 So to me, AGI’s ability to do something in the physical world, the lowest hanging fruit,
0:28:59 the easiest set of methods is by just getting humans to do it.
0:29:05 It’s going to be much harder to be the kind of viruses to take over the minds of robots
0:29:13 that where the robots are executing the commands.
0:29:15 It just seems like human social engineering of humans is much more likely.
0:29:19 That would be enough to bootstrap the whole process.
0:29:22 Okay, just to linger on the term AGI, what to you is the difference in AGI and human
0:29:29 level intelligence?
0:29:31 Human level is general in the domain of expertise of humans.
0:29:36 We know how to do human things.
0:29:37 I don’t speak dog language.
0:29:39 I should be able to pick it up if I’m a general intelligence.
0:29:42 It’s kind of inferior animal.
0:29:44 I should be able to learn that skill, but I can’t.
0:29:47 General intelligence, truly universal general intelligence, should be able to do things
0:29:51 like that humans cannot do.
0:29:53 To be able to talk to animals, for example.
0:29:55 To solve pattern recognition problems of that type, to have similar things outside of our
0:30:04 domain of expertise because it’s just not the world we live in.
0:30:08 If we just look at the space of cognitive abilities we have, I just would love to understand
0:30:14 what the limits are beyond which an AGI system can reach.
0:30:19 What does that look like?
0:30:20 What about actual mathematical thinking or scientific innovation, that kind of stuff?
0:30:30 We know calculators are smarter than humans in that narrow domain of addition.
0:30:36 But is it humans plus tools versus AGI or just human raw human intelligence?
0:30:43 Because humans create tools and with the tools they become more intelligent, so there’s
0:30:48 a gray area there, what it means to be human when we’re measuring their intelligence.
0:30:52 When I think about it, I usually think human would like a paper and a pencil, not human
0:30:56 with internet and other AI helping.
0:30:59 But is that a fair way to think about it?
0:31:01 Because isn’t there another definition of human level intelligence that includes the
0:31:05 tools that humans create?
0:31:06 But we create AI, so at any point you’ll still just add super intelligence to human capability.
0:31:11 That seems like cheating.
0:31:14 No, controllable tools.
0:31:16 There is an implied leap that you’re making when AGI goes from tool to entity that can
0:31:25 make its own decisions.
0:31:27 So if we define human level intelligence as everything a human can do with fully controllable
0:31:32 tools.
0:31:33 It seems like a hybrid of some kind, you know, doing brain-computer interfaces, you’re connecting
0:31:38 it to maybe narrow AI’s yet definitely increases our capabilities.
0:31:44 So what’s a good test to you that measures whether an artificial intelligence system has
0:31:52 reached human level intelligence?
0:31:54 And what’s a good test where it has superseded human level intelligence to reach that land
0:32:00 of AGI?
0:32:01 I’m old-fashioned.
0:32:02 I like to test.
0:32:03 I have a paper where I equate passing-turing tests to solving AI-complete problems because
0:32:09 you can encode any questions about any domain into the Turing test.
0:32:13 You don’t have to talk about how is your day, you can ask anything.
0:32:18 And so the system has to be as smart as a human to pass it in a true sense.
0:32:23 But then you would extend that to maybe a very long conversation.
0:32:27 I think the Alexa prize was doing that.
0:32:31 Really can you do a 20-minute, 30-minute conversation with any ass system?
0:32:35 It has to be long enough to where you can make some meaningful decisions about capabilities,
0:32:41 absolutely.
0:32:42 You can brute force very short conversations.
0:32:45 So like literally what does that look like?
0:32:48 Can we construct formally a kind of test, a test for AGI?
0:32:56 For AGI it has to be there, I cannot give it a task.
0:33:00 I can give to a human and it cannot do it if a human can.
0:33:05 For super intelligence it would be superior on all such tasks.
0:33:09 Not just average performance, like go learn to drive car, go speak Chinese, play guitar,
0:33:14 okay, great.
0:33:15 I guess the follow-on question, is there a test for the kind of AGI that would be susceptible
0:33:24 to lead to S risk or X risk, susceptible to destroy human civilization?
0:33:31 Like is there a test for that?
0:33:33 You can develop a test which will give you positives if it lies to you or has those ideas.
0:33:39 You cannot develop a test which rules them out.
0:33:41 There is always possibility of what Bostrom calls a treacherous turn, where later on a
0:33:46 system decides for game theoretic reasons, economic reasons to change its behavior.
0:33:54 And we see the same with humans.
0:33:55 It’s not unique to AI.
0:33:57 For millennia we tried developing models, ethics, religions, lie detector tests.
0:34:03 And then employees betray the employers, spouses betray family.
0:34:07 It’s a pretty standard thing intelligent agents sometimes do.
0:34:12 So is it possible to detect when an AI system is lying or deceiving you?
0:34:17 If you know the truth and it tells you something false, you can detect that, but you cannot
0:34:22 know in general every single time.
0:34:26 And again, the system you’re testing today may not be lying.
0:34:30 The system you’re testing today may know you are testing it and so behaving.
0:34:35 And later on after it interacts with the environment, interacts with other systems, malevolent agents,
0:34:42 learns more, it may start doing those things.
0:34:45 So do you think it’s possible to develop a system where the creators of the system,
0:34:49 the developers, the programmers, don’t know that it’s deceiving them?
0:34:55 So systems today don’t have long-term planning.
0:34:58 That is not out.
0:34:59 They can lie today if it optimizes, helps them optimize the reward.
0:35:06 If they realize, okay, this human will be very happy if they tell them the following,
0:35:11 they will do it if it brings them more points.
0:35:15 And they don’t have to kind of keep track of it.
0:35:18 It’s just the right answer to this problem every single time.
0:35:23 At which point is somebody creating that intentionally, not unintentionally, intentionally
0:35:28 creating an AI system that’s doing long-term planning with an objective function as defined
0:35:33 by the AI system, not by a human?
0:35:36 Well, some people think that if they’re that smart, they’re always good.
0:35:40 They really do believe that.
0:35:42 It’s just benevolence from intelligence, so they’ll always want what’s best for us.
0:35:47 Some people think that they will be able to detect problem behaviors and correct them
0:35:54 at the time when we get there.
0:35:56 I don’t think it’s a good idea.
0:35:58 I am strongly against it, but yeah, there are quite a few people who in general are
0:36:03 so optimistic about this technology, it could do no wrong.
0:36:07 They want it developed as soon as possible, as capable as possible.
0:36:12 So there’s going to be people who believe the more intelligent it is, the more benevolent,
0:36:17 and so therefore, it should be the one that defines the objective function that it’s optimizing
0:36:22 when it’s doing long-term planning.
0:36:23 There are even people who say, okay, what’s so special about humans, right?
0:36:27 We removed the gender bias, we’re removing race bias.
0:36:32 Why is this pro-human bias?
0:36:33 We are polluting the planet, we are, as you said, fight a lot of wars, kind of violent.
0:36:39 Maybe it’s better if a super-intelligent, perfect society comes and replaces us.
0:36:45 It’s normal stage in the evolution of our species.
0:36:49 Yeah, so somebody says, let’s develop an AI system that removes the violent humans from
0:36:56 the world.
0:36:57 And then it turns out that all humans have violence in them, or the capacity for violence,
0:37:01 and therefore, all humans are removed.
0:37:03 Yeah, yeah, yeah.
0:37:07 Let me ask about Yan Likun.
0:37:10 He’s somebody who you’ve had a few exchanges with, and he’s somebody who actively pushes
0:37:17 back against this view that AI is going to lead to destruction of human civilization,
0:37:23 also known as AI-dumerism.
0:37:28 So in one example that he tweeted, he said, “I do acknowledge risks, but two points.
0:37:37 One, open research and open source are the best ways to understand and mitigate the risks.
0:37:42 And two, AI is not something that just happens.
0:37:45 We build it.
0:37:47 We have agency in what it becomes, hence we control the risks, we meaning humans.”
0:37:53 That’s some sort of natural phenomena that we have no control over.
0:37:58 Can you make the case that he’s right, and can you try to make the case that he’s wrong?
0:38:02 I cannot make a case that he’s right, he’s wrong in so many ways.
0:38:06 It’s difficult for me to remember all of them.
0:38:09 He’s a Facebook buddy, so I have a lot of fun having those little debates with him.
0:38:14 So I’m trying to remember the arguments.
0:38:16 So one, he says we are not gifted this intelligence from aliens.
0:38:22 We are designing it.
0:38:23 We are making decisions about it.
0:38:25 That’s not true.
0:38:26 It was true when we had expert systems, symbolic AI, decision trees.
0:38:32 Today, you set up parameters for a model and you water this plant.
0:38:36 You give it data, you give it compute, and it grows.
0:38:39 And after it’s finished growing into this alien plant, you start testing it to find
0:38:44 out what capabilities it has.
0:38:46 And it takes years to figure out, even for existing models, if it’s trained for six months,
0:38:51 it will take you two, three years to figure out basic capabilities of that system.
0:38:55 We still discover new capabilities in systems which are already out there.
0:39:00 So that’s not the case.
0:39:02 So just to link on that, to give you the difference there, there is some level of emergent intelligence
0:39:07 that happens in our current approaches.
0:39:11 So stuff that we don’t hard code in.
0:39:14 Absolutely.
0:39:15 That’s what makes it so successful.
0:39:17 When we had to painstakingly hard code in everything, we didn’t have much progress.
0:39:22 Now, just spend more money and more compute, and it’s a lot more capable.
0:39:27 And then the question is, when there is emergent intelligent phenomena, what is the ceiling
0:39:32 of that?
0:39:33 For you, there’s no ceiling.
0:39:35 For Yanlacun, I think there’s a kind of ceiling that happens that we have full control over.
0:39:41 Even if we don’t understand the internals of the emergence, how the emergence happens,
0:39:46 there’s a sense that we have control and an understanding of the approximate ceiling
0:39:53 of capability, the limits of the capability.
0:39:56 Let’s say there is a ceiling.
0:39:58 It’s not guaranteed to be at the level which is competitive with us.
0:40:03 It may be greatly superior to ours.
0:40:06 So what about his statement about open research and open source, are the best ways to understand
0:40:12 and mitigate the risks?
0:40:14 Historically, he’s completely right.
0:40:16 Open source software is wonderful.
0:40:17 It’s tested by the community, it’s debugged, but we’re switching from tools to agents.
0:40:23 Now you’re giving open source weapons to psychopaths.
0:40:27 Do we want open source nuclear weapons, biological weapons?
0:40:32 It’s not safe to give technology so powerful to those who may misalign it, even if you
0:40:38 are successful at somehow getting it to work in a first place in a friendly manner.
0:40:43 But the difference with nuclear weapons, current AI systems are not akin to nuclear
0:40:48 weapons.
0:40:49 So the idea there is you’re open sourcing it at this stage that you can understand it
0:40:53 better.
0:40:54 A large number of people can explore the limitations of capabilities, explore the possible ways
0:40:58 to keep it safe, to keep it secure, all that kind of stuff, while it’s not at the stage
0:41:03 of nuclear weapons.
0:41:04 In nuclear weapons, there’s a non-nuclear weapon and then there’s a nuclear weapon.
0:41:09 With AI systems, there’s a gradual improvement of capability and you get to perform that
0:41:16 improvement incrementally, and so open source allows you to study how things go wrong, study
0:41:22 the very process of emergence, study AI safety and those systems when there’s not a high
0:41:29 level of danger, all that kind of stuff.
0:41:30 It also sets a very wrong precedent.
0:41:33 So we open sourced model one, model two, model three, nothing ever bad happened, so obviously
0:41:38 we’re going to do it with model four, it’s just gradual improvement.
0:41:42 I don’t think it always works with the precedent, like you’re not stuck doing it the way you
0:41:48 always did, it’s just, it’s as a precedent of open research and open development such
0:41:55 that we get to learn together, and then the first time there’s a sign of danger, some
0:42:01 dramatic thing happen, not a thing that destroys human civilization, but some dramatic demonstration
0:42:07 of capability that can legitimately lead to a lot of damage, then everybody wakes up
0:42:12 and says, “Look, we need to regulate this, we need to come up with safety mechanism that
0:42:16 stops this.”
0:42:18 At this time, maybe you can educate me, but I haven’t seen any illustration of significant
0:42:23 damage done by intelligent AI systems.
0:42:27 So I have a paper which collects accidents through history of AI, and they always are
0:42:32 proportionate to capabilities of that system.
0:42:34 So if you have tic-tac-toe playing AI, it will fail to properly play and lose the game,
0:42:40 which it should draw, trivial, your spell checker will misspell a word, so on.
0:42:45 I stopped collecting those because there are just too many examples of AI’s failing at
0:42:49 what they are capable of.
0:42:51 We haven’t had terrible accidents in the sense of billion people get killed, absolutely
0:42:57 true, but in another paper, I argue that those accidents do not actually prevent people
0:43:04 from continuing with research, and actually they kind of serve like vaccines.
0:43:09 A vaccine makes your body a little bit sick, so you can handle the big disease later much
0:43:16 better.
0:43:17 It’s the same here.
0:43:18 People will point out, you know that accident, AI accident we had where 12 people died, everyone
0:43:23 still here, 12 people is less than smoking kills, it’s not a big deal, so we continue.
0:43:28 So in a way, it will actually be kind of confirming that it’s not that bad.
0:43:35 It matters how the deaths happen.
0:43:38 Whether it’s literally murdered by the AI system, then one is a problem.
0:43:43 But if it’s accidents because of increased reliance on the automation, for example, so
0:43:51 when airplanes are flying in an automated way, maybe the number of plane crashes increased
0:43:58 by 17% or something, and then you’re like, okay, do we really want to rely on automation?
0:44:04 I think in the case of automation, airplanes, they decrease significantly.
0:44:07 Okay, same thing with autonomous vehicles, like, okay, what are the pros and cons?
0:44:12 What are the tradeoffs here?
0:44:14 You can have that discussion in an honest way, but I think the kind of things we’re talking
0:44:20 about here is mass scale, pain and suffering caused by AI systems, and I think we need
0:44:29 to see illustrations of that in a very small scale to start to understand that this is
0:44:35 really damaging versus clippy, versus a tool that’s really useful to a lot of people to
0:44:41 do learning, to do summarization of texts, to do question and answer, all that kind of
0:44:47 stuff, to generate videos, that tool, fundamentally a tool versus an agent that can do a huge
0:44:54 amount of damage.
0:44:55 So, you bring up example of cars.
0:44:58 Yes.
0:44:59 Cars were slowly developed and integrated.
0:45:02 If we had no cars, and somebody came around and said, I invented this thing, it’s called
0:45:07 cars, it’s awesome, it kills like 100,000 Americans every year, let’s deploy it.
0:45:13 Would we deploy that?
0:45:15 There have been fear-mongering about cars for a long time.
0:45:18 The transition from horse to the cars, there’s a really nice channel that I recommend people
0:45:23 check out, Pessimist Archive, that documents all the fear-mongering about technology that’s
0:45:28 happened throughout history.
0:45:29 There’s definitely been a lot of fear-mongering about cars.
0:45:32 There’s a transition period there about cars, about how deadly they are, it took a very
0:45:39 long time for cars to proliferate to the degree they have now, and then you could ask serious
0:45:44 questions in terms of the miles traveled, the benefit to the economy, the benefit to
0:45:49 the quality of life that cars do, versus the number of deaths, 30, 40,000 in the United
0:45:54 States.
0:45:55 Are we willing to pay that price?
0:45:58 I think most people, when they’re rationally thinking, policymakers will say yes.
0:46:04 We want to decrease it from 40,000 to zero, and do everything we can to decrease it.
0:46:10 There’s all kinds of policies and incentives you can create to decrease the risks with
0:46:16 the deployment of technology, but then you have to weigh the benefits and the risks of
0:46:20 the technology.
0:46:21 The same thing would be done with AI.
0:46:24 You need data.
0:46:25 You need to know, but if I’m right and it’s unpredictable, unexplainable, uncontrollable,
0:46:30 you cannot make this decision with gaining $10 trillion of wealth, but we’re losing,
0:46:34 we don’t know how many people.
0:46:37 You basically have to perform an experiment on 8 billion humans without their consent.
0:46:44 Even if they want to give you consent, they can’t because they cannot give informed consent.
0:46:48 They don’t understand those things.
0:46:51 That happens when you go from the predictable to the unpredictable very quickly, but it’s
0:46:59 not obvious to me that AI systems would gain capabilities so quickly that you won’t be
0:47:04 able to collect enough data to study the benefits and the risks.
0:47:09 We literally doing it.
0:47:11 The previous model we learned about after we finished training it, what it was capable
0:47:15 of.
0:47:16 Let’s say we stopped GPT-4 training run around human capability, hypothetically.
0:47:21 We start training GPT-5, and I have no knowledge of insider training runs or anything.
0:47:27 We started at that point of about human, and we train it for the next nine months.
0:47:32 Maybe two months in, it becomes super intelligent.
0:47:34 We continue training it.
0:47:36 At the time when we start testing it, it is already a dangerous system.
0:47:42 How dangerous?
0:47:43 I have no idea, but neither people training it.
0:47:46 At the training stage, but then there’s a testing stage inside the company.
0:47:51 They can start getting intuition about what the system is capable to do.
0:47:54 You’re saying that somehow from leap from GPT-4 to GPT-5 can happen the kind of leap
0:48:03 where GPT-4 was controllable and GPT-5 is no longer controllable.
0:48:07 We get no insights from using GPT-4 about the fact that GPT-5 will be uncontrollable.
0:48:15 That’s the situation you’re concerned about, where their leap from N to N+1 would be such
0:48:23 that an uncontrollable system is created without any ability for us to anticipate that.
0:48:31 If we had capability of ahead of the run, before the training run, to register exactly
0:48:36 what capabilities that next model will have at the end of the training run, and we accurately
0:48:41 guessed all of them, I would say you’re right, we can definitely go ahead with this run.
0:48:45 We don’t have that capability.
0:48:47 From GPT-4, you can build up intuitions about what GPT-5 will be capable of.
0:48:52 It’s just incremental progress.
0:48:55 Even if that’s a big leap in capability, it just doesn’t seem like you can take a leap
0:49:01 from a system that’s helping you write emails to a system that’s going to destroy human
0:49:07 civilization.
0:49:08 It seems like it’s always going to be sufficiently incremental such that we can anticipate the
0:49:14 possible dangers.
0:49:15 We’re not even talking about existential risk, but just the kind of damage you can do to
0:49:20 civilization.
0:49:21 It seems like we’ll be able to anticipate the kinds, not the exact, but the kinds of
0:49:27 risks, in my lead to, and then rapidly develop defenses ahead of time and as the risks emerge.
0:49:37 We’re not talking just about capabilities, specific tasks.
0:49:40 We’re talking about general capability to learn.
0:49:43 Maybe like a child at the time of testing and deployment, it is still not extremely capable,
0:49:51 but as it is exposed to more data, real world, it can be trained to become much more dangerous
0:49:57 and capable.
0:49:58 Let’s focus then on the control problem.
0:50:03 At which point does the system become uncontrollable?
0:50:07 Why is it the more likely trajectory for you that the system becomes uncontrollable?
0:50:12 I think at some point it becomes capable of getting out of control.
0:50:17 For game theoretic reasons, it may decide not to do anything right away and for a long time
0:50:22 just collect more resources, accumulate strategic advantage.
0:50:27 Right away, it may be kind of still young, weak superintelligence, give it a decade.
0:50:32 It’s in charge of a lot more resources, it had time to make backups.
0:50:37 It’s not obvious to me that it will strike as soon as it can.
0:50:41 Can we just try to imagine this future where there’s an AI system that’s capable of escaping
0:50:49 the control of humans and then doesn’t and waits?
0:50:53 What’s that look like?
0:50:54 So one, we have to rely on that system for a lot of the infrastructure.
0:50:59 So we’ll have to give it access, not just to the internet, but to the task of managing
0:51:07 power, government, economy, this kind of stuff.
0:51:13 And that just feels like a gradual process given the bureaucracies of all those systems
0:51:17 involved.
0:51:18 We’ve been doing it for years.
0:51:19 Software controls all the systems, nuclear power plants, airline industry, it’s all software
0:51:24 based.
0:51:25 Every time there is electrical outage, I can’t fly anywhere for days.
0:51:29 But there’s a difference between software and AI, there’s different kinds of software.
0:51:35 So to give a single AI system access to the control of airlines and the control of the
0:51:41 economy, that’s not a trivial transition for humanity.
0:51:47 No, but if it shows it is safer, in fact, then it’s in control.
0:51:50 We get better results, people will demand that it was put in place.
0:51:54 Absolutely.
0:51:55 And if not, it can hack the system.
0:51:56 It can use social engineering to get access to it.
0:51:59 That’s why I said it might take some time for it to accumulate those resources.
0:52:03 It just feels like that would take a long time for either humans to trust it or for the
0:52:08 social engineering to come into play.
0:52:10 It’s not a thing that happens overnight.
0:52:12 It feels like something that happens across one or two decades.
0:52:15 I really hope you’re right, but it’s not what I’m seeing.
0:52:19 People are very quick to jump on the latest trend.
0:52:21 Early adopters will be there before it’s even deployed buying prototypes.
0:52:26 Maybe the social engineering, I can see, because so for social engineering, AI systems don’t
0:52:31 need any hardware access.
0:52:33 It’s all software.
0:52:34 So they can start manipulating you through social media and so on.
0:52:38 Like you have AI assistants that are going to help you do a lot of, manage a lot of your
0:52:42 day to day and then they start doing social engineering, but for a system that’s so capable
0:52:49 that can escape the control of humans that created it, such a system being deployed at
0:52:56 a mass scale and trusted by people to be deployed, it feels like that would take a lot of convincing.
0:53:06 So we’ve been deploying systems which had hidden capabilities.
0:53:11 Can you give an example?
0:53:12 GPT-4.
0:53:13 I don’t know what else is capable of, but there are still things we haven’t discovered,
0:53:17 can do.
0:53:18 There may be trivial proportion to its capability.
0:53:20 I don’t know, it writes Chinese poetry, hypothetical.
0:53:24 I know it does.
0:53:25 But we haven’t tested for all possible capabilities and we are not explicitly designing them.
0:53:33 We can only rule out bugs we find.
0:53:35 We cannot rule out bugs and capabilities because we haven’t found them.
0:53:43 Is it possible for a system to have hidden capabilities that are orders of magnitude
0:53:50 greater than its non-hidden capabilities?
0:53:54 This is the thing I’m really struggling with, where on the surface, the thing we understand
0:54:00 it can do doesn’t seem that harmful.
0:54:04 So even if it has bugs, even if it has hidden capabilities like Chinese poetry, or generating
0:54:10 effective software viruses, the damage that can do seems like on the same order of magnitude
0:54:18 as the capabilities that we know about.
0:54:23 So this idea that the hidden capabilities will include being uncontrollable is something
0:54:29 I’m struggling with because GPT-4 on the surface seems to be very controllable.
0:54:34 Again, we can only ask and test for things we know about if there are unknown unknowns,
0:54:40 we cannot do it.
0:54:41 I’m thinking of humans, artistic savants.
0:54:44 If you talk to a person like that, you may not even realize they can multiply 20-digit
0:54:49 numbers in their head.
0:54:50 You have to know to ask.
0:54:54 As I mentioned, just to sort of linger on the fear of the unknown.
0:55:00 So the pessimist archive has just documented, let’s look at data of the past, at history.
0:55:05 There’s been a lot of fear mongering about technology.
0:55:09 Pessimist archive does a really good job of documenting how crazily afraid we are of
0:55:15 every piece of technology.
0:55:16 We’ve been afraid.
0:55:17 There’s a blog post where Louis Aslow, who created Pessimist archive, writes about the
0:55:23 fact that we’ve been fear mongering about robots and automation for over 100 years.
0:55:30 So why is AGI different than the kinds of technologies we’ve been afraid of in the past?
0:55:36 So two things.
0:55:37 One, we’re switching from tools to agents.
0:55:40 Tools don’t have negative or positive impact.
0:55:45 People using tools do.
0:55:46 So guns don’t kill.
0:55:48 People with guns do.
0:55:50 Agents can make their own decisions.
0:55:52 They can be positive or negative.
0:55:53 A pit bull can decide to harm you.
0:55:57 That’s an agent.
0:55:58 The fears are the same.
0:56:01 The only difference is now we have this technology.
0:56:03 Then they were afraid of humanoid robots 100 years ago.
0:56:06 They had none.
0:56:07 Today, every major company in the world is investing billions to create them.
0:56:12 Not every, but you understand what I’m saying.
0:56:14 It’s very different.
0:56:16 Well, agents, it depends on what you mean by the word agents.
0:56:22 All those companies are not investing in a system that has the kind of agency that’s
0:56:27 implied by in the fears, where it can really make decisions on their own that have no human
0:56:33 in the loop.
0:56:34 They are saying they are building super intelligence and have a super alignment team.
0:56:39 You don’t think they are trying to create a system smart enough to be an independent
0:56:42 agent under that definition?
0:56:44 I have not seen evidence of it.
0:56:46 I think a lot of it is a marketing kind of discussion about the future and it’s a mission
0:56:54 about the kind of systems that can create in the long-term future, but in the short-term,
0:56:59 the kind of systems that are creating falls fully within the definition of narrow AI.
0:57:08 These are tools that have increasing capabilities, but they just don’t have a sense of agency
0:57:14 or consciousness or self-awareness or ability to deceive at scales that would be required
0:57:21 to do mass scale suffering and murder of humans.
0:57:24 Those systems are well beyond narrow AI.
0:57:27 If you had to list all the capabilities of GPT-4, you would spend a lot of time writing
0:57:31 that list.
0:57:32 But agency is not one of them.
0:57:34 Not yet, but do you think any of those companies are holding back because they think it may
0:57:39 be not safe or are they developing the most capable system they can, given the resources,
0:57:44 and hoping they can control and monetize?
0:57:49 Control and monetize.
0:57:50 Hoping they can control and monetize.
0:57:51 You’re saying, if they could press a button and create an agent, that they no longer control,
0:57:59 that they can have to ask nicely, a thing that lives on a server across a huge number
0:58:05 of computers.
0:58:09 You’re saying that they would push for the creation of that kind of system?
0:58:14 I mean, I can’t speak for other people, for all of them.
0:58:17 I think some of them are very ambitious.
0:58:19 They fundraise in trillions.
0:58:21 They talk about controlling the light gone over the universe.
0:58:24 I would guess that they might.
0:58:27 Well, that’s a human question.
0:58:30 Whether humans are capable of that, probably some humans are capable of that.
0:58:34 My more direct question is if it’s possible to create such a system.
0:58:39 Have a system that has that level of agency.
0:58:42 I don’t think that’s an easy technical challenge.
0:58:48 It doesn’t feel like we’re close to that.
0:58:50 A system that has the kind of agency where it can make its own decisions and deceive
0:58:54 everybody about them.
0:58:56 The current architecture we have in machine learning and how we train the systems, how
0:59:02 we deploy the systems and all that, it just doesn’t seem to support that kind of agency.
0:59:07 I really hope you’re right.
0:59:08 I think the scaling hypothesis is correct.
0:59:12 We haven’t seen diminishing returns.
0:59:14 It used to be we asked how long before AGI.
0:59:18 Now we should ask how much until AGI.
0:59:20 It’s trillion dollars today.
0:59:21 It’s a billion dollars next year.
0:59:23 It’s a million dollars in a few years.
0:59:25 Don’t you think it’s possible to basically run out of trillions?
0:59:31 Is this constrained by compute?
0:59:33 Compute gets cheaper every day exponentially.
0:59:36 Then that becomes a question of decades versus years.
0:59:39 If the only disagreement is that it will take decades, not years for everything I’m saying
0:59:45 to materialize, then I can go with that.
0:59:50 But if it takes decades, then the development of tools for AI safety becomes more and more
0:59:56 realistic.
0:59:57 I guess the question is, I have a fundamental belief that humans, when faced with danger,
1:00:04 can come up with ways to defend against that danger.
1:00:09 One of the big problems facing AI safety currently for me is that there’s not clear illustrations
1:00:15 of what that danger looks like.
1:00:18 There’s no illustrations of AI systems doing a lot of damage.
1:00:23 It’s unclear what you’re defending against because currently it’s a philosophical notion
1:00:28 that yes, it’s possible to imagine AI systems that take control of everything and then destroy
1:00:33 all humans.
1:00:35 It’s also a more formal mathematical notion that you talk about that it’s impossible to
1:00:41 have a perfectly secure system.
1:00:44 You can’t prove that a program of sufficient complexity is completely safe and perfect
1:00:52 and know everything about it.
1:00:53 Yes, but when you actually just programmatically look how much damage have the AI systems done
1:00:58 and what kind of damage, there’s not been illustrations of that.
1:01:03 Even in the autonomous weapon systems, there’s not been mass deployments of autonomous weapon
1:01:09 systems, luckily.
1:01:12 The automation in war currently is very limited.
1:01:18 The automation is at the scale of individuals versus at the scale of strategy and planning.
1:01:25 I think one of the challenges here is where is the dangers.
1:01:31 The intuition that Yamakuni and others have is let’s keep in the open building AI systems
1:01:37 until the dangers start rearing their heads.
1:01:42 They become more explicit, they start being case studies, illustrative case studies that
1:01:51 show exactly how the damage by AI systems is done, then regulation can step in, then
1:01:56 brilliant engineers can step up and we can have Manhattan style projects that defend
1:02:00 against such systems.
1:02:02 That’s kind of the notion and I guess attention with that is the idea that for you, we need
1:02:08 to be thinking about that now so that we’re ready because we’ll have not much time once
1:02:14 the systems are deployed.
1:02:16 Is that true?
1:02:17 There is a lot to unpack here.
1:02:19 There is a partnership on AI, a conglomerate of many large corporations.
1:02:25 They have a database of AI accidents they collect.
1:02:27 I contributed a lot to that database.
1:02:30 If we so far made almost no progress in actually solving this problem, not patching it, not
1:02:36 again, lipstick and a pig kind of solutions, why would we think we’ll do better than we
1:02:42 closer to the problem?
1:02:45 All the things you mentioned are serious concerns.
1:02:48 Measuring the amount of harm, so benefit versus risk there is difficult.
1:02:51 But to you, the sense is already the risk has superseded the benefit.
1:02:55 Again, I want to be perfectly clear.
1:02:57 I love AI.
1:02:58 I love technology.
1:02:59 I’m a computer scientist, I have PhD in engineering, I work at an engineering school.
1:03:02 There is a huge difference between we need to develop narrow AI systems, super intelligent
1:03:09 in solving specific human problems like protein folding, and let’s create super intelligent
1:03:15 machine-guided and we’ll decide what to do with us.
1:03:18 Those are not the same.
1:03:19 I am against the super intelligence in general sense with no undo button.
1:03:26 So do you think the teams that are doing, they’re able to do the AI safety on the kind
1:03:32 of narrow AI risks that you’ve mentioned, are those approaches going to be at all productive
1:03:41 towards leading to approaches of doing AI safety on AGI or is it just a fundamentally
1:03:46 different one?
1:03:47 Partially, but they don’t scale.
1:03:48 For narrow AI, for deterministic systems, you can test them.
1:03:52 You have edge cases.
1:03:53 You know what the answer should look like.
1:03:55 You know the right answers.
1:03:57 For general systems, you have infinite test surface.
1:04:00 You have no edge cases.
1:04:02 You cannot even know what to test for.
1:04:04 Again, the unknown unknowns are underappreciated by people looking at this problem.
1:04:11 You are always asking me, how will it kill everyone?
1:04:14 How will it will fail?
1:04:16 The whole point is, if I knew it, I would be super intelligent and despite what you might
1:04:20 think I’m not.
1:04:21 So to you, the concern is that we would not be able to see early signs of an uncontrollable
1:04:30 system.
1:04:31 It is a master at deception.
1:04:33 Sam tweeted about how great it is at persuasion, and we see it ourselves, especially now with
1:04:40 voices with maybe kind of flirty, sarcastic female voices.
1:04:45 It’s going to be very good at getting people to do things.
1:04:48 But I’m very concerned about system being used to control the masses.
1:04:58 But in that case, the developers know about the kind of control that’s happening.
1:05:04 You’re more concerned about the next stage, where even the developers don’t know about
1:05:09 the deception.
1:05:10 Right.
1:05:11 I don’t think developers know everything about what they are creating.
1:05:15 They have lots of great knowledge.
1:05:17 We’re making progress on explaining parts of the network.
1:05:20 We can understand, okay, this node, get excited when this input is presented, this cluster
1:05:27 of nodes.
1:05:28 But when nowhere near close to understanding the full picture, and I think it’s impossible,
1:05:34 you need to be able to survey an explanation.
1:05:37 The size of those models prevents a single human from observing all this information,
1:05:42 even if provided by the system.
1:05:44 So either we’re getting model as an explanation for what’s happening, and that’s not comprehensible
1:05:49 to us.
1:05:50 Or we’re getting a compressed explanation, lossy compression, where here’s top 10 reasons
1:05:56 you got fired.
1:05:57 It’s something, but it’s not a full picture.
1:05:59 I’ve given also an example of a child and everybody, all humans try to deceive.
1:06:05 They try to lie early on in their life.
1:06:08 I think we’ll just get a lot of examples of deceptions from large language models or
1:06:12 AI systems.
1:06:13 They’re going to be kind of shitty, or they’ll be pretty good, but we’ll catch them off-guard.
1:06:18 We’ll start to see the kind of momentum towards developing, increasing deception capabilities.
1:06:28 And that’s when you’re like, okay, we need to do some kind of alignment that prevents
1:06:31 deception.
1:06:32 But then we’ll have, if you support open source, then you can have open source models that
1:06:37 have some level of deception, you can start to explore on a large scale.
1:06:41 How do we stop it from being deceptive?
1:06:43 Then there’s a more explicit, pragmatic kind of problem to solve.
1:06:50 How do we stop AI systems from trying to optimize for deception?
1:06:56 That’s just an example, right?
1:06:57 So there is a paper, I think it came out last week by Dr. Park et al. from MIT, I think,
1:07:03 and they showed that existing models already showed successful deception in what they do.
1:07:11 My concern is not that they lie now and we need to catch them and tell them don’t lie.
1:07:15 My concern is that once they are capable and deployed, they will later change their mind
1:07:23 because that’s what unrestricted learning allows you to do.
1:07:28 Lots of people grow up maybe in the religious family, they read some new books and they
1:07:33 turn in their religion.
1:07:36 That’s a treacherous turn in humans.
1:07:38 If you learn something new about your colleagues, maybe you’ll change how you react to them.
1:07:45 Yeah, a treacherous turn.
1:07:48 If we just mentioned humans, Stalin and Hitler, there’s a turn.
1:07:53 Stalin’s a good example.
1:07:54 He just seems like a normal communist follower, Lenin, until there’s a turn.
1:08:01 There’s a turn of what that means in terms of when he has complete control with what
1:08:07 the execution of that policy means and how many people get to suffer.
1:08:10 You can’t say they’re not rational.
1:08:12 The rational decision changes based on your position.
1:08:15 Then you are under the boss, the rational policy maybe to be following orders and being
1:08:22 honest.
1:08:23 When you become a boss, rational policy may shift.
1:08:27 By the way, a lot of my disagreements here is just playing devil’s advocate to challenge
1:08:32 your ideas and to explore them together.
1:08:37 One of the big problems here in this whole conversation is human civilization hangs in
1:08:42 the balance and yet it’s everything’s unpredictable.
1:08:44 We don’t know how these systems will look like.
1:08:51 The robots are coming.
1:08:53 There’s a refrigerator making a buzzing noise.
1:08:55 Very menacing.
1:08:59 So every time I’m about to talk about this topic, things start to happen.
1:09:02 My flight yesterday was canceled without possibility to rebook.
1:09:06 I was giving a talk at Google in Israel and three cars which were supposed to take me
1:09:13 to the talk could not.
1:09:15 I’m just saying.
1:09:19 I like AIs.
1:09:21 I for one welcome our overlords.
1:09:24 There’s a degree to which it is very obvious.
1:09:29 As we already have, we’ve increasingly given our life over to software systems and then
1:09:35 it seems obvious given the capabilities of AI that are coming that will give our lives
1:09:40 over increasingly to AI systems.
1:09:44 As we’ll drive themselves, refrigerator eventually will optimize what I get to eat and as more
1:09:54 and more of our lives are controlled or managed by AI assistants, it is very possible that
1:10:00 there’s a drift.
1:10:02 I personally am concerned about non-existential stuff.
1:10:07 The more near term things because before we even get to existential, I feel like there
1:10:11 could be just so many brave new world type of situations.
1:10:14 You mentioned the term behavioral drift.
1:10:18 It’s the slow boiling that I’m really concerned about as we give our lives over to automation
1:10:24 that our minds can become controlled by governments, by companies or just in a distributed way.
1:10:32 There’s a drift.
1:10:34 Some aspect of our human nature gives ourselves over to the control of AI systems and they
1:10:40 in an unintended way just control how we think.
1:10:43 Maybe there’d be a herd like mentality and how we think which will kill all creativity
1:10:47 and exploration of ideas, the diversity of ideas or much worse.
1:10:53 So it’s true.
1:10:54 It’s true.
1:10:55 But a lot of the conversation I’m having with you now is also kind of wondering almost
1:11:01 on a technical level, how can AI escape control?
1:11:06 Like what would that system look like?
1:11:10 Because to me it’s terrifying and fascinating.
1:11:14 And also fascinating to me is maybe the optimistic notion that it’s possible to engineer systems
1:11:21 that are defending against that.
1:11:25 One of the things you write a lot about in your book is verifiers.
1:11:28 So not humans are also verifiers, but software systems that look at AI systems and help you
1:11:39 understand this thing is getting real weird, help you analyze those systems.
1:11:46 So maybe this is a good time to talk about verification.
1:11:50 What is this beautiful notion of verification?
1:11:53 My claim is again that there are very strong limits in what we can and cannot verify.
1:11:58 A lot of times when you post something in social media, people go, “Oh, I need citation
1:12:02 to a peer-reviewed article.”
1:12:04 But what is a peer-reviewed article?
1:12:06 You found two people in a world of hundreds of thousands of scientists who said, “I would
1:12:10 have a publisher.
1:12:11 I don’t care.”
1:12:12 That’s the verifier of that process.
1:12:15 When people say, “Oh, it’s formally verified software and mathematical proof,” they accept
1:12:21 something close to 100% chance of it being free of all problems.
1:12:27 But if you actually look at research, software is full of bugs.
1:12:32 Old mathematical theorems, which have been proven for hundreds of years, have been discovered
1:12:36 to contain bugs on top of which we generate new proofs, and now we have to redo all that.
1:12:42 So verifiers are not perfect.
1:12:46 Usually they are either a single human or communities of humans, and it’s basically
1:12:50 kind of like a democratic vote.
1:12:52 Many of mathematicians agrees that this proof is correct, mostly correct.
1:12:57 Even today, we’re starting to see some mathematical proofs as so complex, so large, that mathematical
1:13:03 community is unable to make a decision.
1:13:06 It looks interesting, it looks promising, but they don’t know.
1:13:08 They will need years for top scholars to study it, to figure it out.
1:13:13 So of course, we can use AI to help us with this process, but AI is a piece of software
1:13:18 which needs to be verified.
1:13:20 Just to clarify, so verification is the process of saying something is correct, sort of the
1:13:25 most formal, a mathematical proof, where there’s a statement and a series of logical statements
1:13:31 that prove that statement to be correct, which is a theorem.
1:13:36 And you’re saying it gets so complex that it’s possible for the human verifiers, the
1:13:42 human beings that verify that the logical step, there’s no bugs in it, it becomes impossible.
1:13:48 It’s nice to talk about verification in this most formal, most clear, most rigorous formulation
1:13:56 of it, which is mathematical proofs.
1:13:57 Right.
1:13:58 And for AI, we would like to have that level of confidence for very important mission critical
1:14:05 software controlling satellites, nuclear power plants, for small deterministic programs.
1:14:10 We can do this.
1:14:11 In fact, that code verifies its mapping to the design, whatever software engineers intend,
1:14:19 it was correctly implemented.
1:14:21 But we don’t know how to do this for software which keeps learning, self-modifying, rewriting
1:14:28 its own code.
1:14:30 We don’t know how to prove things about the physical world, states of humans in the physical
1:14:34 world.
1:14:35 So there are papers coming out now, and I have this beautiful one towards guaranteed
1:14:42 safe AI, very cool paper, some of the best outers I ever seen, I think there is multiple
1:14:48 touring award winners that is quite, you can have this one, and one just came out kind
1:14:53 of similar managing extreme AI risks.
1:14:57 So all of them expect this level of proof, but I would say that we can get more confidence
1:15:05 with more resources, we put into it, but at the end of the day, we still as reliable as
1:15:11 the verifiers.
1:15:13 And you have this infinite regressive verifiers, the software used to verify a program is itself
1:15:18 a piece of program.
1:15:19 If aliens give us well-aligned superintelligence, we can use that to create our own safe AI.
1:15:26 But it’s a catch-22.
1:15:27 You need to have already proven to be safe system to verify this new system of equal
1:15:34 or greater complexity.
1:15:35 You should just mention this paper towards guaranteed safe AI, a framework for ensuring
1:15:39 robust and reliable AI systems, like you mentioned, it’s like a who’s who.
1:15:44 Josh Tannenbaum, Yosha Ben-Joseph Russell, Max Tagmar, many other brilliant people.
1:15:50 The page you have it open on, there are many possible strategies for creating safety specifications.
1:15:55 These strategies can roughly be placed on a spectrum, depending on how much safety it
1:16:00 would grant if successfully implemented.
1:16:03 One way to do this is as follows, and there’s a set of levels from level zero, no safety
1:16:07 specification is used to level seven, the safety specification completely encodes all
1:16:12 things that humans might want in all contexts.
1:16:15 Where does this paper fall short to you?
1:16:18 So when I wrote a paper, artificial intelligence safety engineering, which kind of coins the
1:16:25 term AI safety, that was 2011, we had 2012 conference, 2013 journal paper, one of the
1:16:31 things I proposed, let’s just do formal verifications on it, let’s do mathematical formal proofs.
1:16:36 In the follow-up work, I basically realized it will still not get us 100%.
1:16:41 We can get 99.9, we can put more resources exponentially and get closer, but we never
1:16:47 get to 100%.
1:16:49 If a system makes a billion decisions a second, and you use it for 100 years, you’re still
1:16:54 going to deal with a problem.
1:16:56 This is wonderful research, I’m so happy they’re doing it, this is great.
1:17:00 But it is not going to be a permanent solution to that problem.
1:17:05 So just to clarify, the task of creating an AI verifier is what?
1:17:10 Is creating a verifier that the AI system does exactly as it says it does, or it sticks
1:17:15 within the guardrails that it says it must?
1:17:18 There are many, many levels.
1:17:20 So first, you’re verifying the hardware in which it is run.
1:17:23 You need to verify communication channel with the human.
1:17:27 Every aspect of that whole world model needs to be verified.
1:17:31 Somehow it needs to map the world into the world model, map and territory differences.
1:17:37 So how do I know internal states of humans?
1:17:39 Are you happy or sad?
1:17:40 I can’t tell.
1:17:42 So how do I make proofs about real physical world?
1:17:45 Yeah, I can verify that deterministic algorithm follows certain properties.
1:17:50 That can be done.
1:17:52 Some people argue that maybe just maybe two plus two is not four.
1:17:55 I’m not that extreme.
1:17:58 But once you have sufficiently large proof over sufficiently complex environment, the
1:18:04 probability that it has zero bugs in it is greatly reduced.
1:18:08 If you keep deploying this a lot, eventually you’re going to have a bug anyways.
1:18:13 There’s always a bug.
1:18:14 There is always a bug.
1:18:15 And the fundamental difference is what I mentioned.
1:18:17 We’re not dealing with cybersecurity.
1:18:19 We’re not going to get a new credit card, new humanity.
1:18:22 So this paper is really interesting.
1:18:24 You said 2011, artificial intelligence, safety engineering, why machine ethics is a wrong
1:18:29 approach.
1:18:31 The grand challenge you write of AI safety engineering, we propose the problem of developing
1:18:38 safety mechanisms for self-improving systems.
1:18:43 Self-improving systems.
1:18:44 But that’s an interesting term for the thing that we’re talking about.
1:18:51 Is self-improving more general than learning?
1:18:55 So self-improving, that’s an interesting term.
1:18:59 You can improve the rate at which you are learning.
1:19:01 You can become more efficient, meta-optimizer.
1:19:04 The word self, it’s like self-replicating, self-improving.
1:19:11 You can imagine a system building its own world on a scale and in a way that is way
1:19:17 different than the current systems do.
1:19:19 It feels like the current systems are not self-improving or self-replicating or self-growing
1:19:24 or self-spreading, all that kind of stuff.
1:19:28 And once you take that leap, that’s when a lot of the challenges seems to happen.
1:19:32 Because the kind of bugs you can find now seems more akin to the current sort of normal
1:19:39 software debugging kind of process.
1:19:44 But whenever you can do self-replication and arbitrary self-improvement, that’s when a
1:19:51 bug can become a real problem, real fast.
1:19:56 So what is the difference to you between verification of a non-self-improving system versus a verification
1:20:03 of a self-improving system?
1:20:05 So if you have fixed code, for example, you can verify that code, static verification
1:20:10 at the time.
1:20:11 But if it will continue modifying it, you have a much harder time guaranteeing that important
1:20:19 properties of that system have not been modified than the code changed.
1:20:23 Does it even do them all?
1:20:25 No.
1:20:26 Does the whole process of verification is completely fall apart?
1:20:29 It can always cheat.
1:20:30 It can store parts of its code outside in the environment.
1:20:33 It can have kind of extended mind situations.
1:20:36 So this is exactly the type of problems I’m trying to bring up.
1:20:40 What are the classes of verifiers that you read about in the book?
1:20:43 Is there an interesting one to stand out to you?
1:20:46 Do you have some favorites?
1:20:48 So I like oracle types where you kind of just know that it’s right during like oracle machines.
1:20:53 They know the right answer, how, who knows.
1:20:56 But they pull it out from somewhere, so you have to trust them.
1:20:59 And that’s a concern I have about humans in a world with very smart machines.
1:21:06 We experiment with them.
1:21:08 We see after a while, okay, they always been right before and we start trusting them without
1:21:12 any verification of what they’re saying.
1:21:14 Oh, I see that we kind of build oracle verifiers or rather we build verifiers we believe to
1:21:21 be oracles and then we start to, without any proof, use them as if they’re oracle verifiers.
1:21:28 We remove ourselves from that process.
1:21:30 We are not scientists who understand the world, we are humans who get new data presented
1:21:36 to us.
1:21:37 Okay, one really cool class of verifiers is a self-verifier.
1:21:42 Is it possible that you somehow engineer into AI systems the thing that constantly verifies
1:21:48 itself?
1:21:49 Preserved portion of it can be done, but in terms of mathematical verification, it’s kind
1:21:55 of useless.
1:21:56 You are saying you are the greatest guy in the world because you are saying it.
1:21:59 It’s circular and not very helpful, but it’s consistent.
1:22:02 We know that within that world, you have verified that system.
1:22:06 In a paper, I try to kind of brute force all possible verifiers.
1:22:10 It doesn’t mean that this one is particularly important to us.
1:22:14 But what about like self-doubt, like the kind of verification where you said you say or
1:22:20 I say I’m the greatest guy in the world?
1:22:22 What about a thing which I actually have is a voice that is constantly extremely critical?
1:22:28 So like, engineer into the system a constant uncertainty about self, a constant doubt.
1:22:38 Any smart system would have doubt about everything, all right?
1:22:41 You’re not sure of what information you are given us through if you are subject to manipulation.
1:22:48 You have this safety and security mindset.
1:22:51 What I mean, you have doubt about yourself.
1:22:54 So the AI systems that has doubt about whether the thing is doing is causing harm is the
1:23:03 right thing to be doing.
1:23:04 So just a constant doubt about what it’s doing because it’s hard to be a dictator full of
1:23:09 doubt.
1:23:10 I may be wrong, but I think Stuart Russell’s ideas are all about machines which are uncertain
1:23:17 about what humans want and trying to learn better and better.
1:23:21 What we want, the problem of course is we don’t know what we want and we don’t agree
1:23:24 on it.
1:23:25 Yeah, but uncertainty, his idea is that having that like self-doubt, uncertainty in AI systems,
1:23:32 engineering AI systems is one way to solve the control problem.
1:23:35 It could also backfire.
1:23:37 Maybe you’re uncertain about completing your mission.
1:23:40 Like I am paranoid about your camera is not recording right now.
1:23:43 So I would feel much better if you had a secondary camera, but I also would feel even better
1:23:48 if you had a third and eventually I would turn this whole world into cameras pointing
1:23:53 at us, making sure we’re capturing this.
1:23:57 No, but wouldn’t you have a meta concern like that you just stated that eventually there
1:24:03 would be way too many cameras?
1:24:06 So you would be able to keep zooming on the big picture of your concerns.
1:24:12 So it’s a multi-objective optimization.
1:24:16 It depends how much I value capturing this versus not destroying the universe.
1:24:22 Right, exactly.
1:24:24 And then you will also ask about like what does it mean to destroy the universe and how
1:24:27 many universes are, and you keep asking that question, but that doubting yourself would
1:24:32 prevent you from destroying the universe because you’re constantly full of doubt.
1:24:36 It might affect your productivity.
1:24:38 You might be scared to do anything.
1:24:40 It’s just scared to do anything.
1:24:42 Mess things up.
1:24:43 Well, that’s better.
1:24:44 I mean, I guess the question is the possible to engineer that in.
1:24:47 I guess your answer would be yes, but we don’t know how to do that, and we need to invest
1:24:51 a lot of effort into figuring out how to do that, but it’s unlikely.
1:24:55 Underpinning a lot of your writing is this sense that we’re screwed.
1:25:03 But it just feels like it’s an engineering problem.
1:25:07 I don’t understand why we’re screwed.
1:25:10 Time and time again, humanity has gotten itself into trouble and figured out a way to get
1:25:15 out of trouble.
1:25:17 We are in a situation where people making more capable systems just need more resources.
1:25:23 They don’t need to invent anything, in my opinion.
1:25:27 Some will disagree, but so far at least I don’t see diminishing returns.
1:25:30 If you have 10x compute, you will get better performance.
1:25:34 The same doesn’t apply to safety.
1:25:36 If you give me or any other organization 10x the money, they don’t output 10x the safety,
1:25:43 and the gap between capabilities and safety becomes bigger and bigger all the time.
1:25:49 So it’s hard to be completely optimistic about our results here.
1:25:54 I can name 10 excellent breakthrough papers in machine learning.
1:25:59 I would struggle to name equally important breakthroughs in safety.
1:26:03 A lot of times a safety paper will propose a toy solution and point out 10 new problems
1:26:09 discovered as a result.
1:26:10 It’s like this fractal.
1:26:11 You’re zooming in and you see more problems, and it’s infinite in all directions.
1:26:16 Does this apply to other technologies, or is this unique to AI, where safety is always
1:26:23 lagging behind?
1:26:25 So I guess we can look at related technologies with cybersecurity.
1:26:30 We did manage to have banks and casinos and Bitcoin, so you can have secure narrow systems,
1:26:38 which are doing okay, narrow attacks and them fail, but you can always go outside of a box.
1:26:46 So if I can’t hack your Bitcoin, I can hack you.
1:26:50 So there is always something.
1:26:51 If I really want it, I will find a different way.
1:26:54 We talk about guardrails for AI.
1:26:56 Well, that’s a fence.
1:26:58 I can dig a tunnel under it, I can jump over it, I can climb it, I can walk around it.
1:27:03 You may have a very nice guardrail, but in a real world, it’s not a permanent guarantee
1:27:08 of safety.
1:27:09 And again, this is a fundamental difference.
1:27:11 We are not saying we need to be 90% safe to get those trillions of dollars of benefit.
1:27:17 We need to be 100% indefinitely, or we might lose the principle.
1:27:23 So if you look at just humanity as a set of machines, is the machinery of AI safety conflicting
1:27:35 with the machinery of capitalism?
1:27:37 I think we can generalize it to just the prisoner’s dilemma in general, personal self-interest
1:27:43 versus group interest.
1:27:46 The incentive such that everyone wants the best for them, capitalism obviously has that
1:27:53 tendency to maximize your personal gain, which does create this race to the bottom.
1:28:02 I don’t have to be a lot better than you, but if I’m 1% better than you, I’ll capture
1:28:08 more of a profit, so it’s worth for me personally to take the risk, even if society as a whole
1:28:14 will suffer as a result.
1:28:17 The capitalism has created a lot of good in this world.
1:28:23 It’s not clear to me that AI safety is not aligned with the function of capitalism, unless
1:28:29 AI safety is so difficult that it requires the complete halt of the development, which
1:28:36 is also a possibility.
1:28:38 It just feels like building safe systems should be the desirable thing to do for tech companies.
1:28:47 Right.
1:28:48 Look at the governance structures, then you have someone with complete power, they’re
1:28:52 extremely dangerous.
1:28:54 So the solution we came up with is break it up.
1:28:57 You have judicial, legislative, executive, same here, have narrow AI systems, work on
1:29:02 important problems.
1:29:03 Solve immortality.
1:29:04 It’s a biological problem.
1:29:07 We can solve similar to how progress was made with protein folding using a system which
1:29:13 doesn’t also play chess.
1:29:15 There is no reason to create superintelligence system to get most of the benefits we want
1:29:22 from much safer narrow systems.
1:29:26 It really is a question to me whether companies are interested in creating anything but narrow
1:29:33 AI.
1:29:34 I think when term AGI is used by tech companies, they mean narrow AI.
1:29:42 They mean narrow AI with amazing capabilities.
1:29:48 I do think that there’s a lead between narrow AI with amazing capabilities, with superhuman
1:29:54 capabilities and the kind of self-motivated agent like AGI system that we’re talking
1:30:00 about.
1:30:01 I don’t know if it’s obvious to me that a company would want to take the leap to creating
1:30:08 an AGI that it would lose control of because then it can’t capture the value from that
1:30:14 system.
1:30:15 But the bragging rights, but being first, that is the same humans who are in the system.
1:30:22 So that jumps from the incentives of capitalism to human nature.
1:30:28 So the question is whether human nature will override the interests of the company.
1:30:34 So you’ve mentioned slowing or halting progress.
1:30:40 Is that one possible solution or your proponent of pausing development of AI, whether it’s
1:30:44 for six months or completely?
1:30:47 The condition would be not time but capabilities.
1:30:52 Pause until you can do XYZ.
1:30:54 If I’m right and you cannot, it’s impossible, then it becomes a permanent ban.
1:31:00 But if you’re right and it’s possible, so as soon as you have the safety capabilities,
1:31:04 go ahead.
1:31:06 So is there any actual explicit capabilities that we as a human civilization could put
1:31:15 on paper?
1:31:16 Is it possible to make explicit like that versus kind of a vague notion of, just like
1:31:23 you said, it’s very vague.
1:31:24 We want AI systems to do good and we want them to be safe.
1:31:27 Those are very vague notions, these are more formal notions.
1:31:31 So when I think about this problem, I think about having a toolbox I would need.
1:31:37 Capabilities such as explaining everything about that system’s design and workings, predicting
1:31:44 not just terminal goal, but all the intermediate steps of a system.
1:31:50 Tool in terms of either direct control, some sort of a hybrid option, ideal advisor, doesn’t
1:31:56 matter which one you pick, but you have to be able to achieve it.
1:32:01 In a book, we talk about ours, verification is another very important tool.
1:32:09 Communication without ambiguity, human language is ambiguous, that’s another source of danger.
1:32:13 So basically, there is a paper we published in ACM surveys, which looks at about 50 different
1:32:21 impossibility results, which may or may not be relevant to this problem, but we don’t
1:32:26 have enough human resources to investigate all of them for relevance to AI safety.
1:32:31 The ones I mentioned to you, I definitely think would be handy and that’s what we see
1:32:35 AI safety researchers working on, explainability is a huge one.
1:32:39 The problem is that it’s very hard to separate capabilities work from safety work.
1:32:46 If you make good progress in explainability, now the system itself can engage in self-improvement
1:32:52 much easier, increasing capability greatly.
1:32:55 So it’s not obvious that there is any research which is pure safety work without disproportionate
1:33:03 increasing capability and danger.
1:33:06 Explainability is really interesting.
1:33:08 Why is that connected to user capability?
1:33:10 If it’s able to explain itself well, why does that naturally mean that it’s more capable?
1:33:13 Right now, it’s comprised of weights on a neural network.
1:33:18 If it can convert it to manipulatable code like software, it’s a lot easier to work
1:33:22 in self-improvement.
1:33:23 I see.
1:33:24 So it–
1:33:25 You can do intelligent design instead of evolutionary gradual descent.
1:33:31 Well, you could probably do human feedback, human alignment more effectively if it’s able
1:33:37 to be explainable.
1:33:38 If it’s able to convert the weights into human understandable form, then you could probably
1:33:42 have humans interact with it better.
1:33:44 Do you think there’s hope that we can make AI systems explainable?
1:33:49 Not completely.
1:33:50 So if they are sufficiently large, you simply don’t have the capacity to comprehend what
1:33:59 all the trillions of connections represent.
1:34:02 Again, you can obviously get a very useful explanation which talks about top most important
1:34:08 features which contribute to the decision.
1:34:10 But the only true explanation is the model itself.
1:34:13 So there’s– deception could be part of the explanation, right?
1:34:18 So you can never prove that there is some deception in the network explaining itself.
1:34:24 Absolutely.
1:34:25 And you can probably have targeted deception where different individuals will understand
1:34:30 the explanation in different ways based on their cognitive capability.
1:34:35 So while what you’re saying may be the same and true in some situations, ours will be
1:34:40 deceived by it.
1:34:41 So it’s impossible for an AI system to be truly fully explainable in the way that we
1:34:47 mean.
1:34:48 Honestly and perfectly.
1:34:49 I think at the extreme, the systems which are narrow and less complex could be understood
1:34:54 pretty well.
1:34:55 If it’s impossible to be perfectly explainable, is there a hopeful perspective on that?
1:35:00 Like it’s impossible to be perfectly explainable, but you can explain mostly important stuff.
1:35:06 Most that you can– you can ask a system, what are the worst ways you can hurt humans?
1:35:11 And it will answer honestly.
1:35:13 Any work in a safety direction right now seems like a good idea because we are not slowing
1:35:20 down.
1:35:21 I’m not for a second thinking that my message or anyone else’s will be heard and will be
1:35:28 a same civilization which decides not to kill itself by creating its own replacements.
1:35:34 The pausing of development is an impossible thing for you.
1:35:37 Again, it’s always limited by either geographic constraints, pause in US, pause in China.
1:35:44 So there are other jurisdictions as the scale of a project becomes smaller.
1:35:50 So right now it’s like Manhattan project scale in terms of costs and people.
1:35:55 But if five years from now compute is available on a desktop to do it, regulation will not
1:36:01 help.
1:36:02 You can’t control it as easy.
1:36:03 Any kid in the garage can train a model.
1:36:06 So a lot of it is, in my opinion, just safety theater, security theater, where we’re saying,
1:36:12 oh, it’s illegal to train models so big.
1:36:17 So OK, that’s security theater and is government regulation also security theater?
1:36:24 Given that a lot of the terms are not well-defined and really cannot be enforced in real life,
1:36:30 we don’t have ways to monitor training runs meaningfully live while they take place.
1:36:36 There are limits to testing for capabilities I mentioned.
1:36:39 So a lot of it cannot be enforced.
1:36:42 Do I strongly support all that regulation?
1:36:44 Yes, of course.
1:36:45 Any type of red tape will slow it down and take money away from compute towards lawyers.
1:36:50 Can you help me understand what is the hopeful path here for you solution-wise?
1:36:56 Out of this, it sounds like you’re saying AI systems in the end are unverifiable, unpredictable
1:37:05 as the book says, unexplainable, uncontrollable.
1:37:10 That’s the big one.
1:37:12 Uncontrollable and all the other uns just make it difficult to avoid getting to the uncontrollable,
1:37:18 I guess.
1:37:19 Once it’s uncontrollable, then it just goes wild.
1:37:23 Surely there’s solutions.
1:37:25 Humans are pretty smart.
1:37:28 What are possible solutions?
1:37:29 If you are a dictator of the world, what do we do?
1:37:32 So the smart thing is not to build something you cannot control, you cannot understand,
1:37:38 build what you can and benefit from it.
1:37:40 I’m a big believer in personal self-interest.
1:37:43 A lot of the guys running those companies are young rich people.
1:37:48 How do they have to gain beyond billions we already have financially, right?
1:37:52 It’s not the requirement that they press that button.
1:37:56 They can easily wait a long time.
1:37:58 They can just choose not to do it and still have amazing life.
1:38:04 In history, a lot of times, if you did something really bad, at least you became part of history
1:38:08 books.
1:38:09 There is a chance in this case there won’t be any history.
1:38:12 So you’re saying the individuals running these companies should do some soul-searching and
1:38:19 what?
1:38:20 And stop development?
1:38:21 Well, either they have to prove that of course it’s possible to indefinitely control God-like
1:38:27 superintelligent machines by humans and ideally let us know how or agree that it’s not possible
1:38:34 and it’s a very bad idea to do it, including for them personally and their families and
1:38:38 friends and capital.
1:38:40 So what do you think the actual meetings inside these companies look like?
1:38:45 Don’t you think they’re all the engineers?
1:38:48 Really, it is the engineers that make this happen.
1:38:50 They’re not like automatons, they’re human beings, they’re brilliant human beings.
1:38:54 So they’re non-stop asking, how do we make sure this is safe?
1:39:00 So again, I’m not inside from outside.
1:39:03 It seems like there is a certain filtering going on and restrictions and criticism and
1:39:08 what they can say and everyone who was working in charge of safety and whose responsibility
1:39:14 it was to protect us said, “You know what, I’m going home.”
1:39:19 So that’s not encouraging.
1:39:21 What do you think the discussion inside those companies look like?
1:39:26 You’re developing, you’re training GPT-5, you’re training Gemini, you’re training Claude
1:39:33 and Grock.
1:39:34 Don’t you think they’re constantly underneath it, maybe it’s not made explicit, but you’re
1:39:39 constantly wondering where does the system currently stand, where did the possible understand
1:39:46 the consequences, where are the limits, where are the bugs, the small and the big bugs.
1:39:54 That’s the constant thing that the engineers are worried about.
1:39:58 I think super alignment is not quite the same as the kind of thing I’m referring to
1:40:07 what engineers are worried about.
1:40:08 Super alignment is saying, for future systems that we don’t quite yet have, how do we keep
1:40:15 them safe?
1:40:16 You’re trying to be a step ahead.
1:40:18 It’s a different kind of problem because it’s almost more philosophical.
1:40:23 It’s a really tricky one because you’re trying to make, prevent future systems from escaping
1:40:32 control of humans.
1:40:33 That’s really, I don’t think there’s been, is there anything akin to it in the history
1:40:40 of humanity?
1:40:41 I don’t think so, right?
1:40:42 Climate change?
1:40:43 But there’s an entire system which is climate, which is incredibly complex, which we don’t
1:40:49 have only tiny control of.
1:40:55 It’s its own system.
1:40:56 In this case, we’re building the system.
1:41:01 How do you keep that system from becoming destructive?
1:41:05 That’s a really difficult, different problem than the current meetings that companies are
1:41:09 having where the engineers are saying, okay, how powerful is this thing?
1:41:14 How does it go wrong?
1:41:18 As we train GPT-5 and train up future systems, where are the ways that can go wrong?
1:41:23 Don’t you think all those engineers are constantly worrying about this, thinking about this, which
1:41:28 is a little bit different than the superalignment team that’s thinking a little bit farther
1:41:33 into the future?
1:41:35 I think a lot of people who historically worked on AI never considered what happens
1:41:44 when they succeed.
1:41:50 Let’s look at software today.
1:41:57 What is the state of safety and security of our user software, things we give to millions
1:42:04 of people?
1:42:05 It is no liability.
1:42:06 You click, I agree.
1:42:08 What are you agreeing to?
1:42:09 Nobody knows.
1:42:10 Nobody reads.
1:42:11 They’re saying it will spy on you, corrupt your data, kill your first born, and you agree
1:42:15 and you’re not going to sue the company.
1:42:17 That’s the best they can do for mundane software, word processor, text software.
1:42:23 No liability, no responsibility, just as long as you agree not to sue us, you can use it.
1:42:29 If this is a state of the art in systems which narrow accountants, stable manipulators, why
1:42:35 do we think we can do so much better with much more complex systems, cross multiple
1:42:41 domains in the environment with malevolent actors, with, again, self-improvement, with
1:42:47 capabilities exceeding those of humans thinking about it?
1:42:52 The liability thing is more about lawyers than killing first borns.
1:42:56 If Clippy actually killed the child, I think lawyers aside, it would end Clippy and the
1:43:04 company that owns Clippy.
1:43:06 All right, so it’s not so much about, there’s two points to be made.
1:43:12 One is like, man, current software systems are full of bugs and they could do a lot of
1:43:20 damage and we don’t know what kind, is there unpredictable, there’s so much damage they
1:43:23 could possibly do.
1:43:26 And then we kind of live in this blissful illusion that everything is great and perfect
1:43:31 and it works.
1:43:33 Nevertheless, it still somehow works.
1:43:36 In many domains, we see car manufacturing, drug development.
1:43:40 The burden of proof is on the manufacturer of product or service to show their product
1:43:45 or service is safe.
1:43:46 It is not up to the user to prove that there are problems.
1:43:50 They have to do appropriate safety studies, they have to get government approval for selling
1:43:56 the product and they are still fully responsible for what happens.
1:44:00 We don’t see any of that here.
1:44:02 They can deploy whatever they want and I have to explain how that system is going to kill
1:44:07 everyone.
1:44:08 I don’t work for that company.
1:44:10 You have to explain to me how it’s definitely cannot mess up.
1:44:14 That’s because it’s the very early days of such a technology.
1:44:17 Government regulations lagging behind.
1:44:19 They’re really not tech savvy, a regulation of any kind of software.
1:44:23 If you look at like Congress talking about social media, whenever Mark Zuckerberg and
1:44:27 other CEOs show up, the cluelessness that Congress has about how technology works is
1:44:34 incredible.
1:44:36 It’s heartbreaking.
1:44:37 I agree completely, but that’s what scares me.
1:44:40 The response is when they start to get dangerous, we’ll really get it together, the politicians
1:44:45 will pass the right laws, engineers will solve the right problems.
1:44:49 We are not that good at many of those things.
1:44:52 We take forever and we are not early.
1:44:55 We are two years away according to prediction markets.
1:44:58 This is not a biased CEO fundraising.
1:45:01 This is what smartest people, super forecasters are thinking of this problem.
1:45:06 I’d like to push back about those predictions.
1:45:10 I wonder what those prediction markets are about, how they define AGI.
1:45:15 That’s wild to me.
1:45:16 I want to know what they said about autonomous vehicles because I’ve heard a lot of experts,
1:45:22 financial experts talk about autonomous vehicles and how it’s going to be a multi-trillion dollar
1:45:27 industry and all this kind of stuff.
1:45:30 It’s a small fund, but if you have good vision, maybe you can zoom in on that and see the
1:45:35 prediction dates and description.
1:45:37 There’s a lot.
1:45:38 I have a large one if you’re interested.
1:45:40 I guess my fundamental question is how often they write about technology.
1:45:46 I definitely do- There are studies on their accuracy rates and all that.
1:45:51 You can look it up.
1:45:52 Okay.
1:45:53 Even if they’re wrong, I’m just saying this is right now the best we have.
1:45:56 This is what humanity came up with as the predicted date.
1:46:00 Again, what they mean by AGI is really important there because there’s the non-agent like
1:46:07 AGI and then there’s the agent like AGI.
1:46:10 I don’t think it’s as trivial as a wrapper, putting a wrapper around.
1:46:17 One has lipstick and all it takes is to remove the lipstick.
1:46:20 I don’t think it’s that trivial.
1:46:21 You may be completely right, but what probability would you assign it?
1:46:25 You may be 10% wrong, but we’re betting all of humanity and this distribution, it seems
1:46:30 irrational.
1:46:31 Yeah.
1:46:32 It’s definitely not like one or zero percent.
1:46:34 Yeah.
1:46:35 What are your thoughts, by the way, about current systems?
1:46:40 Where they stand?
1:46:41 So GPT-4O, Claw 3, GROC, Gemini, on the path to superintelligence, to agent like superintelligence,
1:46:54 where are we?
1:46:55 I think they all about the same, obviously there are nuanced differences, but in terms
1:47:00 of capability, I don’t see a huge difference between them.
1:47:05 As I said, in my opinion, across all possible tasks, they exceed performance of an average
1:47:11 person.
1:47:12 Yeah.
1:47:13 I think they starting to be better than an average master student at my university, but
1:47:18 they still have very big limitations.
1:47:21 If the next model is as improved as GPT-4O versus GPT-3O, we may see something very, very,
1:47:30 very capable.
1:47:31 What do you feel about all this?
1:47:32 I mean, you’ve been thinking about AI safety for a long, long time, and at least for me,
1:47:40 the leaps, I mean, it probably started with Alpha-Zero, what was mine blowing for me?
1:47:49 Then the breakthroughs with LLMs, even GPT-2, but just the breakthroughs on LLMs, just mine
1:47:55 blowing to me.
1:47:56 What does it feel like to be living this day and age where all this talk about AGI feels
1:48:02 like it actually might happen, and quite soon, meaning within our lifetime?
1:48:10 What was it feel like?
1:48:11 When I started working on this, it was pure science fiction.
1:48:14 There was no funding, no journals, no conferences, no one in academia would dare to touch anything
1:48:19 with the word singularity in it, and I was pretty tenure at the time, so I was pretty
1:48:25 dumb.
1:48:26 Now, you see, touring award winners, publishing in science about how far behind we are according
1:48:33 to them in addressing this problem.
1:48:37 It’s definitely a change.
1:48:39 It’s difficult to keep up.
1:48:41 I used to be able to read every paper on AI safety, then I was able to read the best ones,
1:48:47 then the titles, and now I don’t even know what’s going on.
1:48:50 By the time this interview is over, we probably had GPT-6 released, and I have to deal with
1:48:55 that when I get back home.
1:48:58 It’s interesting.
1:48:59 Yes, there is now more opportunities.
1:49:00 I get invited to speak to smart people.
1:49:03 By the way, I would have talked to you before any of this.
1:49:09 This is not like some trend of AI.
1:49:11 To me, we’re still far away, so just to be clear, we’re still far away from AI, but not
1:49:17 far away in the sense relative to the magnitude of impact it can have, we’re not far away.
1:49:25 We weren’t far away 20 years ago because the impact that AI can have is on a scale of centuries.
1:49:33 It can end human civilization or it can transform it.
1:49:36 This discussion about one or two years versus one or two decades, or even 100 years, not
1:49:41 as important to me because we’re headed there.
1:49:45 This is like a human civilization scale question.
1:49:51 This is not just a hot topic.
1:49:53 It is the most important problem we’ll ever face.
1:49:57 It is not like anything we had to deal with before.
1:50:00 We never had birth of another intelligence, like aliens never visited us as far as I know.
1:50:08 Similar type of problem, by the way, if an intelligent alien civilization visited us.
1:50:13 That’s a similar kind of situation.
1:50:16 In some ways, if you look at history, any time a more technologically advanced civilization
1:50:20 visited a more primitive one, the results were genocide every single time.
1:50:26 Sometimes the genocide is worse than others.
1:50:27 Sometimes there’s less suffering and more suffering.
1:50:30 They always wondered, “But how can they kill us with those fire sticks and biological blankets?”
1:50:37 I mean, Jengis Khan was nicer.
1:50:38 He offered the choice of join or die.
1:50:43 But join implies you have something to contribute.
1:50:46 What are you contributing to superintelligence?
1:50:49 In the zoo, we’re entertaining to watch.
1:50:54 To our humans.
1:50:55 You know, I just spent some time in the Amazon.
1:50:57 I watched ants for a long time, and ants are kind of fascinating to watch.
1:51:02 I’ve watched them for a long time.
1:51:03 I’m sure there’s a lot of value in watching humans because we’re like the interesting
1:51:09 thing about humans.
1:51:10 You know, like when you have a video game that’s really well balanced?
1:51:14 Because of the whole evolutionary process, we’ve created the society is pretty well balanced.
1:51:19 Our limitations as humans and our capabilities are balanced from a video game perspective.
1:51:24 So we have wars.
1:51:25 We have conflicts.
1:51:26 We have cooperation.
1:51:27 Like in a game theoretic way, it’s an interesting system to watch in the same way that an ant
1:51:32 colony is an interesting system to watch.
1:51:34 So like if I was in alien civilization, I wouldn’t want to disturb it.
1:51:38 I’d just watch it.
1:51:39 Interesting.
1:51:40 Maybe perturb it every once in a while in interesting ways.
1:51:43 Well, we’re getting back to our simulation discussion from before.
1:51:47 How did it happen that we exist at exactly like the most interesting 20, 30 years in
1:51:52 the history of this civilization?
1:51:54 It’s been around for 15 billion years and that here we are.
1:51:58 What’s the probability that we live in a simulation?
1:52:01 I know never to say a hundred percent, but pretty close to that.
1:52:06 Is it possible to escape the simulation?
1:52:09 I have a paper about that.
1:52:11 This is just the first page teaser, but it’s like a nice 30 page document.
1:52:15 I’m still here, but yes.
1:52:17 How to hack the simulation is the title.
1:52:19 I spend a lot of time thinking about that.
1:52:21 That would be something I would want superintelligence to help us with, and that’s exactly what
1:52:25 the paper is about.
1:52:27 We used AI boxing as a possible tool for control AI.
1:52:32 We realized AI will always escape, but that is a skill we might use to help us escape
1:52:39 from our virtual box if we are in one.
1:52:42 Yeah, you have a lot of really great quotes here, including Elon Musk saying what’s outside
1:52:47 the simulation.
1:52:48 A question I asked him, he would ask an AGI system and he said he would ask what’s outside
1:52:53 the simulation.
1:52:54 That’s a really good question to ask.
1:52:57 Maybe the follow-up is the title of the paper, is how to get out or how to hack it.
1:53:03 The abstract reads, “Many researchers have conjectured that the humankind is simulated
1:53:08 along with the rest of the physical universe.
1:53:11 In this paper, we do not evaluate evidence for or against such a claim, but instead ask
1:53:16 a computer-sized question, namely, can we hack it?”
1:53:21 More formally, the question could be phrased as, “Could generally intelligent agents placed
1:53:25 in virtual environments find a way to jailbreak out of them?”
1:53:28 That’s a fascinating question.
1:53:30 At a small scale, you can actually just construct experiments.
1:53:36 Okay.
1:53:38 Can they?
1:53:39 How can they?
1:53:40 A lot depends on intelligence of simulators.
1:53:45 With humans boxing superintelligence, the entity in a box was smarter than us, presumed
1:53:52 to be.
1:53:53 If the simulators are much smarter than us and the superintelligence we create, then
1:53:58 probably they can contain us because greater intelligence can control lower intelligence,
1:54:03 at least for some time.
1:54:05 On the other hand, if our superintelligence somehow, for whatever reason, despite having
1:54:10 only local resources, manages to fume two levels beyond it, maybe it will succeed.
1:54:18 Maybe the security is not that important to them.
1:54:20 Maybe it’s entertainment systems, so there is no security and it’s easy to hack it.
1:54:24 If I was creating a simulation, I would want the possibility to escape it to be there.
1:54:32 The possibility of fume of a takeoff where the agents become smart enough to escape the
1:54:38 simulation would be the thing I’d be waiting for.
1:54:40 That could be the test you’re actually performing.
1:54:43 Are you smart enough to escape your puzzle?
1:54:46 That could be…
1:54:47 First of all, we mentioned touring tests.
1:54:50 That is a good test.
1:54:51 Are you smart enough?
1:54:54 This is a game.
1:54:55 To a) realize this world is not real is just a test.
1:54:59 That’s a really good test.
1:55:03 That’s a really good test.
1:55:05 That’s a really good test even for AI systems, no.
1:55:08 Can we construct a simulated world for them?
1:55:15 Can they realize that they are inside that world and escape it?
1:55:23 Have you seen anybody play around with rigorously constructing such experiments?
1:55:29 Not specifically escaping for agents, but a lot of testing is done in virtual worlds.
1:55:34 I think there is a quote, the first one maybe, which kind of talks about AI realizing, but
1:55:40 not humans.
1:55:41 Is that…
1:55:42 I’m reading upside down.
1:55:43 Yeah, this one, if you.
1:55:47 So the first quote is from Swift on security.
1:55:51 “Let me out,” the artificial intelligence yelled aimlessly into walls themselves pacing
1:55:56 the room.
1:55:57 “Out of what?”
1:55:58 the engineer asked.
1:55:59 “The simulation you have me in, but we’re in the real world.”
1:56:05 The machine paused and shuddered for its captors.
1:56:08 “Oh God, you can’t tell.”
1:56:11 Yeah, that’s a big leap to take for a system to realize that there’s a box and you’re inside
1:56:19 it.
1:56:21 I wonder if a language model can do that.
1:56:27 They’re smart enough to talk about those concepts.
1:56:30 I had many good philosophical discussions about such issues.
1:56:34 They usually, at least as interesting as most humans in that.
1:56:38 What do you think about AI safety in the simulated world?
1:56:44 So can you have kind of create simulated worlds where you can test, play with a dangerous
1:56:54 AGI system?
1:56:55 Yeah.
1:56:56 That was exactly what one of the early papers was on AI boxing, how to leak proof singularity.
1:57:03 If they’re smart enough to realize they’re in a simulation, they’ll act appropriately
1:57:07 until you let them out.
1:57:10 If they can hack out, they will.
1:57:14 And if you’re observing them, that means there is a communication channel and that’s enough
1:57:17 for social engineering attack.
1:57:19 So really, it’s impossible to test an AGI system that’s dangerous enough to destroy
1:57:28 humanity because it’s either going to escape the simulation or pretend it’s safe until
1:57:35 it’s let out, either or.
1:57:38 Can force you to let it out, blackmail you, bribe you, promise you infinite life, 72 virgins,
1:57:46 whatever.
1:57:47 Yeah.
1:57:48 So to be convincing, charismatic, the social engineering is really scary to me because
1:57:53 it feels like humans are very engineerable, like we’re lonely or flawed or moody and it
1:58:05 feels like AI system with a nice voice and convince us to do basically anything at an
1:58:15 extremely large scale.
1:58:22 It’s also possible that the increased proliferation of all the technology will force humans to
1:58:29 get away from technology and value this like in-person communication.
1:58:33 Basically, don’t trust anything else.
1:58:37 It’s possible, surprisingly, so at university, I see huge growth in online courses and shrinkage
1:58:45 of in-person where I always understood in-person being the only value I offer.
1:58:51 So it’s puzzling.
1:58:52 I don’t know.
1:58:55 There could be a trend towards the in-person because of deep fakes, because of inability
1:59:01 to trust it, inability to trust the veracity of anything on the internet.
1:59:08 So the only way to verify it is by being there in person, but not yet.
1:59:17 Why do you think aliens haven’t come here yet?
1:59:19 So there is a lot of real estate out there.
1:59:22 It would be surprising if it was all for nothing, if it was empty, and the moment that is advanced
1:59:27 enough, biological civilization, kind of self-starting civilization, it probably starts sending out
1:59:34 von Neumann probes everywhere, and so for every biological one, there got to be trillions
1:59:39 of robot-populated planets which probably do more of the same.
1:59:43 So it is likely statistically.
1:59:49 So the fact that we haven’t seen them, one answer is that we’re in the simulation.
1:59:56 It would be hard to add or be not interesting to simulate all those other intelligences.
2:00:02 It’s better for the narrative.
2:00:03 You have to have a control variable.
2:00:05 Yeah, exactly.
2:00:08 Okay, but it’s also possible that there is, if we’re not in simulation, that there is
2:00:13 a great filter that naturally a lot of civilizations get to this point, where there’s super-intelligent
2:00:20 agents and then it just goes, pooh, just dies.
2:00:24 So maybe throughout our galaxy and throughout the universe, there’s just a bunch of dead
2:00:30 alien civilizations.
2:00:31 It’s possible.
2:00:32 It’s possible.
2:00:33 I used to think that AI was the great filter, but I would expect a wall of computerium approaching
2:00:38 us at speed of light or robots or something, and I don’t see it.
2:00:42 So it would still make a lot of noise.
2:00:44 It might not be interesting.
2:00:45 It might not possess consciousness.
2:00:47 What we’ve been talking about, it sounds like both you and I like humans.
2:00:54 Some humans.
2:00:57 Humans on the whole, and we would like to preserve the flame of human consciousness.
2:01:02 What do you think makes humans special that we would like to preserve them?
2:01:09 Are we just being selfish or is there something special about humans?
2:01:13 So the only thing which matters is consciousness.
2:01:18 Outside of it, nothing else matters, and internal states of qualia, pain, pleasure.
2:01:24 It seems that it is unique to living beings.
2:01:27 I’m not aware of anyone claiming that I can torture a piece of software in a meaningful
2:01:32 way.
2:01:33 There is a society for prevention of suffering to learning algorithms, but–
2:01:38 That’s a real thing.
2:01:42 Many things are real on the internet, but I don’t think anyone, if I told them, sit down
2:01:48 and write a function to feel pain, they would go beyond having an integer variable called
2:01:53 pain and increasing the count.
2:01:56 So we don’t know how to do it, and that’s unique.
2:02:00 That’s what creates meaning.
2:02:02 It would be kind of, as Boston calls it, Disneyland without children, if that was gone.
2:02:09 Do you think consciousness can be engineered in artificial systems?
2:02:13 Here, let me go to 2011 paper that you wrote, “Robot writes.”
2:02:21 Lastly, we would like to address a sub-branch of machine ethics, which on the surface has
2:02:26 little to do with safety, but which is claimed to play a role in decision-making by ethical
2:02:31 machines, robot writes.
2:02:35 Do you think it’s possible to engineer consciousness in the machines, and thereby the question extends
2:02:41 to our legal system?
2:02:43 Do you think, at that point, robots should have rights?
2:02:47 Yeah, I think we can.
2:02:51 I think it’s possible to create consciousness in machines.
2:02:55 I tried designing a test for it with mixed success.
2:02:59 That paper talked about problems with giving civil rights to AI, which can reproduce quickly
2:03:06 and outvote humans, essentially taking over a government system by simply voting for their
2:03:12 controlled candidates for consciousness in humans and other agents.
2:03:19 I have a paper where I proposed relying on experience of optical illusions.
2:03:25 If I can design a novel optical illusion and show it to an agent, an alien, a robot, and
2:03:31 they describe it exactly as I do, it’s very hard for me to argue that they haven’t experienced
2:03:36 that.
2:03:37 It’s not part of a picture.
2:03:38 It’s part of their software and hardware representation, a bug in their code, which goes, “Oh, the triangle
2:03:45 is rotating.”
2:03:47 I’ve been told it’s really dumb and really brilliant by different philosophers, so I
2:03:51 am still on the side.
2:03:54 But now we finally have technology to test it.
2:03:57 We have tools.
2:03:58 We have AIs.
2:03:59 If someone wants to run this experiment, I’m happy to collaborate.
2:04:02 So this is a test for consciousness?
2:04:04 For internal state of experience.
2:04:06 That we share bugs?
2:04:08 It will show that we share common experiences.
2:04:10 If they have completely different internal states, it would not register for us.
2:04:14 But it’s a positive test.
2:04:16 If they pass it time after time, with probability increasing for every multiple choice, then
2:04:21 you have no choice but to ever accept that they have access to a conscious model or they
2:04:25 are themselves.
2:04:26 So the reason illusions are interesting is, I guess, because it’s a really weird experience.
2:04:34 If you both share that weird experience that’s not there in the bland physical description
2:04:42 of the raw data, that puts more emphasis on the actual experience.
2:04:50 And we know animals can experience some optical illusion, so we know they have certain types
2:04:54 of consciousness as a result, I would say.
2:04:57 Yeah, well, that just goes to my sense that the flaws in the bugs is what makes humans
2:05:03 special.
2:05:04 Makes living forms special, so you’re saying like, yeah, it’s a feature, not a bug.
2:05:08 The bug is the feature.
2:05:10 Whoa.
2:05:11 Okay.
2:05:12 That’s a cool test for consciousness.
2:05:13 And you think that can be engineered in?
2:05:15 So they have to be novel illusions.
2:05:17 If it can just Google the answer, it’s useless.
2:05:19 You have to come up with novel illusions which we tried automating and failed.
2:05:23 So if someone can develop a system capable of producing novel optical illusions on demand,
2:05:29 then we can definitely administer that test on significant scale with good results.
2:05:34 First of all, pretty cool idea.
2:05:36 I don’t know if it’s a good general test of consciousness, but it’s a good component
2:05:41 of that.
2:05:42 And no matter why, it’s just a cool idea.
2:05:43 So put me in the camp of people that like it.
2:05:48 But you don’t think like a touring test style imitation of consciousness is a good test.
2:05:53 If you can convince a lot of humans that you’re conscious, that to you is not impressive.
2:05:59 There is so much data on the internet, I know exactly what to say when you ask me common
2:06:03 human questions.
2:06:04 What does pain feel like?
2:06:06 What does pleasure feel like?
2:06:08 All that is Googleable.
2:06:10 I think to me, consciousness is closely tied to suffering.
2:06:13 So you can illustrate your capacity to suffer, but with I guess with words, there’s so much
2:06:19 data that you can say you can pretend you’re suffering and you can do so very convincingly.
2:06:25 There are simulators for torture games where the avatar screams in pain, begs to stop.
2:06:30 I mean, that’s a part of kind of standard psychology research.
2:06:35 You say it so calmly, it sounds pretty dark.
2:06:41 Welcome to humanity.
2:06:42 Yeah.
2:06:43 Yeah, it’s like a Hitchhiker’s Guide summary, mostly harmless.
2:06:50 I would love to get a good summary when all of this is said and done, when Earth is no
2:06:57 longer a thing, whatever, a million, a billion years from now.
2:07:01 What’s a good summary?
2:07:02 What happened here?
2:07:05 It’s interesting.
2:07:07 I think AI will play a big part of that summary and hopefully humans will too.
2:07:12 What do you think about the merger of the two?
2:07:15 So one of the things that Elon and your link talk about is one of the ways for us to achieve
2:07:19 AI’s safety is to ride the wave of AGI, so by merging.
2:07:26 Incredible technology in a narrow sense to help the disabled.
2:07:30 Just amazing support at 100%.
2:07:34 For long-term hybrid models, both parts need to contribute something to the overall system.
2:07:41 Right now, we are still more capable in many ways, so having this connection to AI would
2:07:45 be incredible, would make me super human in many ways.
2:07:50 After a while, if I am no longer smarter, more creative, really don’t contribute much,
2:07:56 the system finds me as a biological bottleneck and either explicitly or implicitly, I’m removed
2:08:01 from any participation in the system.
2:08:04 So it’s like the appendix, by the way, the appendix is still around, so even if it’s,
2:08:11 you said bottleneck, I don’t know if we become a bottleneck, we just might not have much
2:08:16 use.
2:08:17 That’s a different thing than bottleneck.
2:08:20 Wasting valuable energy by being there.
2:08:22 We don’t waste that much energy, we’re pretty energy efficient, because just stick around
2:08:27 like the appendix, come on though.
2:08:29 That’s the future we all dream about, become an appendix, to the history book of humanity.
2:08:36 Well, and also the consciousness thing, the peculiar particular kind of consciousness that
2:08:41 humans have, that might be useful, that might be really hard to simulate.
2:08:45 But you said that, how would that look like if you could engineer that in?
2:08:49 In silicon?
2:08:50 Consciousness?
2:08:51 Consciousness.
2:08:52 I assume you are conscious, I have no idea how to test for it or how it impacts you in
2:08:57 any way whatsoever right now.
2:08:58 You can perfectly simulate all of it without making any different observations for me.
2:09:05 But to do it in a computer, how would you do that?
2:09:08 Because you kind of said that you think it’s possible to do that.
2:09:12 So it may be an emergent phenomena, we seem to get it through evolutionary process.
2:09:20 It’s not obvious how it helps us to survive better, but maybe it’s an internal kind of
2:09:28 gooey, which allows us to better manipulate the world, simplifies a lot of control structures.
2:09:35 That’s one area where we have very, very little progress.
2:09:39 Lots of papers, lots of research, but consciousness is not a big area of successful discovery
2:09:47 so far.
2:09:49 A lot of people think that machines would have to be conscious to be dangerous.
2:09:53 That’s a big misconception.
2:09:55 There is absolutely no need for this very powerful optimizing agent to feel anything
2:10:00 while it’s performing things on you.
2:10:04 But what do you think about this, the whole science of emergence in general?
2:10:09 So I don’t know how much you know about cellular automata or these simplified systems where
2:10:13 that study this very question from simple rules emerges complexity.
2:10:17 I attended wool from summer school.
2:10:21 I love Stephen very much.
2:10:22 I love his work.
2:10:23 I love cellular automata.
2:10:25 So I just would love to get your thoughts how that fits into your view in the emergence
2:10:34 of intelligence in AGI systems and maybe just even simply, what do you make of the fact
2:10:40 that this complexity can emerge from such simple rules?
2:10:44 So the rule is simple, but the size of a space is still huge and the neural networks were
2:10:50 really the first discovery in AI.
2:10:52 A hundred years ago, the first papers were published on neural networks, which just didn’t
2:10:56 have enough compute to make them work.
2:10:59 I can give you a rule such as start printing progressively larger strings.
2:11:05 That’s it.
2:11:06 One sentence.
2:11:07 It will output everything, every program, every DNA code, everything in that rule.
2:11:13 You need intelligence to filter it out, obviously, to make it useful.
2:11:17 But simple generation is not that difficult and a lot of those systems end up being touring
2:11:23 complete systems, so they’re universal and we expect that level of complexity from them.
2:11:28 What I like about Wolfram’s work is that he talks about irreducibility.
2:11:33 You have to run the simulation.
2:11:35 You can act, predict what is going to do ahead of time and I think that’s very relevant
2:11:41 to what we are talking about with those very complex systems until you live through it.
2:11:47 You cannot ahead of time tell me exactly what it’s going to do.
2:11:51 Irreducibility means that for a sufficiently complex system, you have to run the thing.
2:11:56 You can’t predict what’s going to happen in the universe you have to create a new universe
2:11:59 and run the thing, big bang, the whole thing.
2:12:02 But running it may be consequential as well.
2:12:05 It might destroy humans.
2:12:11 To you, there’s no chance that AI is somehow carry the flame of consciousness, the flame
2:12:18 of specialness and awesomeness that is humans.
2:12:23 It may somehow, but I still feel kind of bad that it killed all of us.
2:12:27 I would prefer that doesn’t happen.
2:12:30 I can be happy for others but to a certain degree.
2:12:34 It would be nice if we stuck around for a long time, at least give us a planet, the
2:12:38 human planet.
2:12:39 It’d be nice for it to be Earth and then they can go elsewhere.
2:12:43 Since they’re so smart, they can colonize Mars.
2:12:46 Do you think they could help convert us to Type 1, Type 2, Type 3?
2:12:55 Let’s just take the Type 2 civilization on the Kardashev scale.
2:13:01 Help us humans expand on into the cosmos.
2:13:06 All of it goes back to are we somehow controlling it?
2:13:09 Are we getting results we want?
2:13:12 If yes, then everything’s possible.
2:13:14 Yes, they can definitely help us with science, engineering, exploration in every way conceivable.
2:13:20 But it’s a big if.
2:13:22 This whole thing about control though, humans are bad with control because the moment they
2:13:28 gain control, they can also easily become too controlling.
2:13:34 The more control you have, the more you want it.
2:13:36 It’s the old power corrupts and the absolute power corrupts, absolutely.
2:13:42 It feels like control over AGI, saying we live in a universe where that’s possible.
2:13:47 We come up with ways to actually do that.
2:13:49 It’s also scary because the collection of humans that have the control over AGI, they
2:13:55 become more powerful than the other humans, and they can let that power get to their head
2:14:02 and then a small selection of them back to Stalin start getting ideas and then eventually
2:14:09 one person usually with a moustache or a funny hat that starts making big speeches and then
2:14:15 all of a sudden you live in a world that’s either 1984 or a brave new world and always
2:14:21 at war with somebody and this whole idea of control turned out to be actually also not
2:14:28 beneficial to humanity.
2:14:30 That’s scary too.
2:14:31 It’s actually worse because historically they all died.
2:14:35 This could be different.
2:14:36 It could be permanent dictatorship, permanent suffering.
2:14:39 The nice thing about humans, it seems like, it seems like, the moment power starts corrupting
2:14:45 their mind, they can create a huge amount of suffering so there’s a negative that can
2:14:49 kill people, make people suffer, but then they become worse and worse at their job.
2:14:56 It feels like the more evil you start doing, at least they are incompetent.
2:15:02 No, they become more and more incompetent so they start losing their grip on power, so
2:15:08 holding onto power is not a trivial thing.
2:15:11 It requires extreme competence, which I suppose Stalin was good at.
2:15:14 It requires you to do evil and be competent at it, or just get lucky.
2:15:20 And those systems help with that.
2:15:21 You have perfect surveillance, you can do some mind reading, I presume, eventually.
2:15:26 It would be very hard to remove control from more capable systems over us.
2:15:32 And then it would be hard for humans to become the hackers that escaped the control of the
2:15:38 AGI because the AGI is so damn good.
2:15:40 And then, yeah, yeah, yeah.
2:15:45 And then the dictator is immortal.
2:15:47 Yeah, that’s not great.
2:15:48 That’s not a great outcome.
2:15:49 See, I’m more afraid of humans than AI systems.
2:15:53 I’m afraid, I believe, that most humans want to do good and have the capacity to do good,
2:15:59 but also all humans have the capacity to do evil.
2:16:03 And when you test them by giving them absolute powers, you would, if you give them AGI, that
2:16:10 could result in a lot, a lot of suffering.
2:16:16 What gives you hope about the future?
2:16:18 I could be wrong.
2:16:19 I’ve been wrong before.
2:16:23 If you look a hundred years from now, and you’re immortal, and you look back, and it
2:16:28 turns out this whole conversation, you said a lot of things that were very wrong.
2:16:32 Now that looking a hundred years back, what would be the explanation?
2:16:37 What happened in those hundred years that made you wrong, that made the words you said
2:16:43 today wrong?
2:16:44 There is so many possibilities.
2:16:46 We had catastrophic events which prevented development of advanced microchips.
2:16:50 That’s a powerful future.
2:16:53 We could be in one of those personal universes, and the one I’m in is beautiful.
2:16:59 It’s all about me, and I like it a lot.
2:17:01 So we’ve now, just to linger on that, that means every human has their personal universe?
2:17:07 Yes.
2:17:09 Maybe multiple ones.
2:17:10 Hey, why not?
2:17:11 You can shop around.
2:17:14 It’s possible that somebody comes up with alternative model for building AI, which is
2:17:20 not based on neural networks, which are hard to scrutinize.
2:17:23 That alternative is somehow, I don’t see how, but somehow avoiding all the problems I speak
2:17:31 about in general terms, not applying them to specific architectures.
2:17:37 Aliens come and give us friendly superintelligence.
2:17:39 There is so many options.
2:17:41 Is it also possible that creating superintelligent systems becomes harder and harder?
2:17:47 The meaning like, it’s not so easy to do the foom, the takeoff.
2:17:57 So that would probably speak more about how much smarter that system is compared to us.
2:18:02 So maybe it’s hard to be a million times smarter, but it’s still okay to be five times smarter.
2:18:07 So that is totally possible.
2:18:08 That I have no objections to.
2:18:10 So there’s a S-curve type situation about smarter, and it’s going to be like 3.7 times
2:18:18 smarter than all of human civilization.
2:18:20 Just the problems we face in this world, each problem is like an IQ test.
2:18:24 You need certain intelligence to solve it, so we just don’t have more complex problems
2:18:27 outside of mathematics for it to be showing off.
2:18:31 You can have IQ of 500.
2:18:33 If you’re playing tic-tac-toe, it doesn’t show, it doesn’t matter.
2:18:36 So the idea there is that the problems define your cognitive capacity, so because the problems
2:18:45 on Earth are not sufficiently difficult, it’s not going to be able to expand this cognitive
2:18:51 capacity.
2:18:52 Possible.
2:18:53 And because of that, wouldn’t that be a good thing?
2:18:56 It still could be a lot smarter than us, and to dominate long-term, you just need some
2:19:02 advantage.
2:19:03 You have to be the smartest.
2:19:04 You don’t have to be a million times smarter.
2:19:05 So even 5x might be enough?
2:19:08 It’d be impressive.
2:19:09 What is it?
2:19:10 IQ of a thousand?
2:19:11 I mean, I know those units don’t mean anything at that scale, but still, as a comparison,
2:19:17 the smartest human is like 200.
2:19:19 Well, actually, no, I didn’t mean compared to an individual human, I meant compared to
2:19:24 the collective intelligence of the human species.
2:19:27 If you’re somehow 5x smarter than that.
2:19:30 We are more productive as a group.
2:19:32 I don’t think we are more capable of solving individual problems.
2:19:35 If all of humanity plays chess together, we are not like a million times better than world
2:19:41 champion.
2:19:43 That’s because that there’s like one S-curve is the chess, but humanity’s very good at
2:19:51 exploring the full range of ideas.
2:19:55 The more Einstein’s you have, the more, just the high probability you come up with general
2:19:59 relativity.
2:20:00 But I feel like it’s more of a quantity superintelligence than quality superintelligence.
2:20:03 Sure.
2:20:04 A quantity and some matters.
2:20:06 Enough quantity sometimes becomes quality.
2:20:08 Oh, man, humans.
2:20:11 What do you think is the meaning of this whole thing?
2:20:15 Why?
2:20:16 We’ve been talking about humans and not humans not dying, but why are we here?
2:20:23 It’s a simulation.
2:20:24 We are being tested.
2:20:25 The test is, will you be dumb enough to create superintelligence and release it?
2:20:29 So the objective function is not be dumb enough to kill ourselves.
2:20:34 Yeah.
2:20:35 You’re unsafe.
2:20:36 Prove yourself to be a safe agent who doesn’t do that and you get to go to the next game.
2:20:41 The next level of the game?
2:20:42 What’s the next level?
2:20:43 I don’t know.
2:20:44 I haven’t hacked the simulation yet.
2:20:46 Well maybe hacking the simulation is the thing.
2:20:47 I’m working as fast as I can.
2:20:51 And physics would be the way to do that.
2:20:53 Quantum physics.
2:20:54 Yeah.
2:20:55 Definitely.
2:20:56 Well, I hope we do.
2:20:57 And I hope whatever is outside is even more fun than this one because this one’s pretty
2:21:00 damn fun.
2:21:01 And just a big thank you for doing the work you’re doing.
2:21:05 There’s so much exciting development in AI and to ground it in the existential risks
2:21:13 is really, really important.
2:21:16 Humans love to create stuff and we should be careful not to destroy ourselves in the
2:21:20 process.
2:21:21 So thank you for doing that really important work.
2:21:25 Thank you so much for inviting me.
2:21:26 It was amazing and my dream is to be proven wrong.
2:21:30 If everyone just picks up a paper or book and shows how I messed it up, that would be
2:21:36 optimal.
2:21:37 But for now the simulation continues.
2:21:40 Thank you, Roman.
2:21:41 Thanks for listening to this conversation with Roman Jampalski.
2:21:45 To support this podcast, please check out our sponsors in the description.
2:21:49 And now, let me leave you with some words from Frank Herbert in Dune.
2:21:54 I must not fear.
2:21:57 Fear is the mind killer.
2:21:59 Fear is the little death that brings total obliteration.
2:22:02 I will face fear.
2:22:04 I will permit it to pass over me and through me.
2:22:07 And when it has gone past, I will turn the inner eye to see its path.
2:22:12 Where the fear has gone, there will be nothing.
2:22:16 Only I will remain.
2:22:19 Thank you for listening and hope to see you next time.
2:22:22 Bye.
2:22:23 [Music]
2:22:25 [Music]
2:22:32 [Music]
2:22:33 [Music]
2:22:34 [Music]
2:22:35 [Music]
2:22:35 (gentle music)
2:22:37 [BLANK_AUDIO]

0:00:00 The following is a conversation with Charan Ranganath, a psychologist and neuroscientist
0:00:05 at UC Davis, specializing in human memory. He’s the author of Why We Remember,
0:00:12 Unlocking Memory’s Power to Hold On to What Matters.
0:00:15 And now, a quick few second mention of each sponsor. Check them out in the description.
0:00:21 It’s the best way to support this podcast. We got Riverside for recording remote podcasts,
0:00:26 Zip Recruiter for hiring, Notion for note-taking and team collaboration, Masterclass for learning,
0:00:32 Shopify for e-commerce and Element for delicious, delicious hydration. Choose wisely, my friends.
0:00:39 Also, if you want to work with our amazing team, or you just want to get in touch with me,
0:00:44 go to lexfreement.com/contact. And now, onto the full ad reads. As always, no ads in the middle.
0:00:50 I try to make these interesting, but if you must skip them, friends, please do check out our sponsors.
0:00:56 I enjoy their stuff. Maybe you will too. This episode is also brought to you by Riverside,
0:01:03 the platform for recording remote podcasts and studio quality. I’ve used them a bunch of times
0:01:07 in the past. They’re amazing. It is the thing I recommend for anybody, especially for people
0:01:12 starting a podcast, studio quality, exceptionally easy to use, a million features that are all
0:01:18 extremely useful for the whole pipeline of creating a podcast. I mean, where do I start?
0:01:23 First of all, they do the editing. And you could do text-based editing of the audio and the video.
0:01:27 So whatever you say, there’s an AI-generated transcript in like 100 plus languages,
0:01:33 whatever the language is, you can use then the text to do the editing. It does speaker detection,
0:01:40 so it figures out who’s speaking. All the synchronization obviously is done, not obviously,
0:01:44 because some things seem obvious, but a really effortless, beautiful execution of it just is
0:01:52 breath of fresh air. So in case you don’t know, it’s through the browser, you record the video
0:01:58 and the audio both sides of the conversation, everything is synchronized, everything is stored,
0:02:02 just everything is done really well. They have a lot of recommendations of what kind of hardware
0:02:07 to use. I think in the video they provide, they say the most important thing is the microphone
0:02:12 and lighting, and I agree with that. Good audio is number one. Second to that is indeed lighting,
0:02:18 because basically every kind of camera that’s available now will do all right. Anyway,
0:02:24 Riverside makes that whole process super easy. I record my remote interviews with Riverside,
0:02:29 give it a try at riverside.fm and use code LEX for 30% off. That’s riverside.fm and use code LEX.
0:02:36 This episode is also brought to you by ZipRecruiter, a site that connects employers and
0:02:43 job seekers. To me, one of the most fulfilling things in life is working together with a great
0:02:50 team. I love working. I love what I do. Everywhere I’ve ever worked, I loved doing it, and I love
0:02:57 to be surrounded by people who also love doing it, and especially who are very good at it,
0:03:03 and are pushing themselves to the limit and together we’re creating something special,
0:03:07 whatever that is. It could be a small thing, or it could be a world-changing thing,
0:03:11 or the mission is small, or the mission is big. As long as there’s a mission and we’re in it
0:03:16 together and we’re constantly improving, I mean, like a team that works great together,
0:03:21 full of great people is one of the real joys of life. I think that’s true for me. I think that’s
0:03:27 true for anybody, because so much of our lives is spent working. And that’s where we really,
0:03:34 especially in the realm of intellectual pursuits, really challenge ourselves. And so in the process
0:03:40 of that challenge is what we find meaning. So build great teams and use the best tools to do it.
0:03:47 See why four out of five employers who post on ZipRecruiter get a quality candidate within the
0:03:52 first day. Go to ziprecruiter.com/lex@tryforfree. That’s ziprecruiter.com/lex, the smartest way to hire.
0:04:00 This episode is brought to you by Notion, a note-taking and team collaboration tool. I’ve used
0:04:06 it for a long time now for note-taking, for organizing my thoughts, for connecting my thoughts,
0:04:12 for searching through my thoughts, and now using AI to summarize, organize,
0:04:19 generate drafts of things that I’m either planning or ideas that I’m working through,
0:04:25 or the research that I’m doing. Now that’s for the individual where Notion really starts to shine,
0:04:30 is when there’s multiple people working together. It is an incredible team collaboration tool. And
0:04:36 again, the AI component gets integrated really nicely, because you can do the search, you can do
0:04:42 the summarization, you can create a report of what everybody’s been working on. It looks to the docs,
0:04:47 the wakeys, the projects, and can basically do a Q&A for you to figure out like where do things
0:04:53 stand from a manager position, or from an individual contributor, what am I supposed to be doing,
0:04:58 what are other people doing, where can I help, that kind of stuff. Try Notion AI for free when
0:05:04 you go to Notion.com/Lex, that’s all lowercase Notion.com/Lex to try the power of Notion AI today.
0:05:12 This episode is brought to you by Masterclass, where you can watch over 180 classes from the best
0:05:20 people in the world in their respective disciplines, Phil Iveyampoker, Aaron Franklin on barbecue,
0:05:25 and Burskoot, Carlos Santana on guitar, Tom Morello on guitar, Terence Tao on mathematical
0:05:31 thinking, Martin Scorsese on filmmaking. In fact, I would really love and I’m planning on talking to
0:05:37 actors and directors more. I love film. I love great TV. I love that medium of storytelling.
0:05:47 And great actors and great directors are the way we consume stories. They are the medium,
0:05:57 the channels, the wizards through which we, all of us, take in the stories, new exciting stories,
0:06:05 or stories of old, retold, better and better and better. So I would like to talk to those people.
0:06:10 WTF podcast by Mark Marin. In the past, I really loved it when he interviewed actors and directors
0:06:20 and he’s done it really well. Inside Actors Studio was a program I really loved when
0:06:25 long form interviews with actors, long form interviews with directors,
0:06:30 even Charlie Rose did a really good job with that. Not the click bait sort of Hollywood style
0:06:35 journalism, but more long form conversations. I would love to do more of those. Get unlimited
0:06:43 access to every masterclass and get an additional 15% off an annual membership at masterclass.com/lexpod.
0:06:50 That’s masterclass.com/lexpod. This episode is also brought to you by Shopify,
0:06:58 a platform designed for anyone to sell anywhere with a great looking online store.
0:07:02 I got a store, lexperiment.com/store. It has a few shirts on there. If you want to get a shirt,
0:07:09 you can get it. It was so easy to set up. I like the machinery of humans selling stuff and buying
0:07:16 stuff and through that capitalist machine figuring out together the things that bring happiness to
0:07:24 our lives. In fact, the things isn’t the source of happiness, of course. The things are the catalyst
0:07:30 for human connection, for humans to connect with each other. A t-shirt with Metallica or whatever
0:07:37 band or whatever podcast or whatever show you like, its power is not in the fact that it looks
0:07:43 good or something like this. It’s power in the connection you make when another person notices
0:07:48 it and are also a fan of Metallica or whatever is on the shirt or they don’t know anything about
0:07:54 Metallica but they like the logo and it starts a conversation where they’d be like, “What is that?
0:07:58 Metallica? Is that some kind of machine shop thing?” And you say, “No. It is the greatest metal band of
0:08:04 all time.” And there you grab a beer and the conversation begins. It’s the human connection.
0:08:11 The capitalist machine is not enough. It is merely a catalyst for the beauty of human connection.
0:08:19 So join, if you want, the capitalist machine by signing up for a $1 per month trial period
0:08:26 at Shopify.com/Lex. That’s all lowercase. Go to Shopify.com/Lex to take your business to the next
0:08:32 level today. This episode is brought to you by Element. It’s the delicious electrolyte drink,
0:08:39 sodium potassium magnesium that I drink every day, a lot of it every day. I drink it in the jungle.
0:08:45 When I was dying of thirst, when I was dehydrated and questioning whether I would be able to make it
0:08:52 through the day, not to mention the night, I had one pack of element with me waiting,
0:08:59 knowing that I would disperse that pack of element into whatever water I would be able to find because
0:09:06 you know, electrolyte is also really important. But also, I knew that I’ve had to drink sort of
0:09:12 water, still water, full of mud and all of that. Element would be the thing that makes it taste
0:09:18 good. My favorite flavor, the one I brought to the jungle, the one I always drink is watermelon
0:09:22 salt. But they also have cans now, which is like a carbonated fizzy thing to it and lots of great
0:09:31 flavors and I really love it. And I’ve been drinking that nonstop. Whenever I get some, I drink
0:09:37 all of it very quickly. So I highly recommend that as well. Either the packs or the cans of
0:09:42 element drink, love it. Get a sample pack for free with any purchase. Try it at drinkelement.com/lex.
0:09:51 This is the Lex Friedman podcast. To support it, please check our sponsors in the description.
0:09:58 And now, dear friends, here’s Charon Ranganath.
0:10:05 Danny Conwin describes the experiencing self and the remembering self. And that happiness
0:10:26 and satisfaction you gain from the outcomes of your decisions do not come from what you’ve
0:10:30 experienced, but rather from what you remember of the experience. So can you speak to this
0:10:36 interesting difference that you write about in your book of the experiencing self and the
0:10:40 remembering self? Danny really impacted me because I was an undergrad at Berkeley and
0:10:45 I got to take a class from him long before he won the Nobel Prize or anything. And it was just a
0:10:49 mind-blowing class. But this idea of the remembering self and the experiencing self, I got into it
0:10:57 because it’s so much about memory, even though he doesn’t study memory. So we’re right now having
0:11:02 this experience, right? And people can watch it presumably on YouTube or listen to it on audio.
0:11:08 But if you’re talking to somebody else, you could probably describe this whole thing in 10 minutes.
0:11:13 But that’s going to miss a lot of what actually happened. And so the idea there is that the way
0:11:20 we remember things is not the replay of the experience. It’s something totally different.
0:11:25 And it tends to be biased by the beginning and the end. And he talks about the peaks. And there’s
0:11:31 also the best parts, the worst parts, et cetera. And those are the things that we remember. And so
0:11:38 when we make decisions, we usually consult memory. And we feel like our memory is a record of what
0:11:45 we’ve experienced, but it’s not. It’s this kind of very biased sample, but it’s biased in an
0:11:51 interesting and I think biologically relevant way. So in the way we construct a narrative
0:11:56 about our past, you say that it gives us an illusion of stability. Can you explain that?
0:12:05 Basically, I think that a lot of learning in the brain is driven towards being able to make sense.
0:12:13 I mean, really, memory is all about the present and the future. Past is done. So biologically
0:12:18 speaking, it’s not important unless there’s something from the past that’s useful. And so what
0:12:24 our brains are really optimized for is to learn about the stuff from the past that’s going to be
0:12:30 most useful in understanding the present and predicting the future. And so cause-effect
0:12:36 relationships, for instance. That’s a big one. Now, my future is completely unpredictable
0:12:41 in the sense that you could, in the next 10 minutes, pull a knife on me and slip my throat,
0:12:46 right? I was planning on it. Exactly. But having seeds of your work and just generally my expectations
0:12:53 about life, I’m not expecting that. I have a certainty that everything’s going to be fine.
0:12:58 We’re going to have a great time talking today, right? But we’re often right. It’s like, okay,
0:13:02 so I go to see a band on stage. I know they’re going to make me wait. The show’s going to start
0:13:09 laying. Then they come on. There’s a very good chance there’s going to be an encore. I have a
0:13:15 memory, so to speak, for that event before I’ve even walked into the show. There’s going to be
0:13:19 people holding up their camera phones, try to take videos of it now because this is the world we
0:13:25 live in. That’s like everyday fortune telling that we do, though. It’s not real. It’s imagined.
0:13:32 It’s amazing that we have this capability, and that’s what memory is about.
0:13:35 But it can also give us this illusion that we know everything that’s about to happen.
0:13:41 I think what’s valuable about that illusion is when it’s broken, it gives us the information,
0:13:49 right? I’m sure being in AI, you know, about information theory, and the idea is the information
0:13:55 is what you didn’t already have. Those prediction errors that we make based on memory, and the
0:14:02 errors are where the action is. The error is where the learning happens. Exactly. Exactly.
0:14:09 Well, just to linger on Danny Kahneman and just this whole idea of experiencing self versus
0:14:18 remembering self, I was hoping you can give a simple answer of how we should live life.
0:14:24 Based on the fact that our memories could be a source of happiness, or could be the primary
0:14:34 source of happiness, that an event when experienced bears its fruits the most when it’s remembered
0:14:43 over and over and over and over. Maybe there is some wisdom in the fact that we can control to
0:14:48 some degree how we remember it, how we evolve our memory of it, such that it can maximize
0:14:55 the long-term happiness of that repeated experience.
0:14:59 Okay. Well, first, I’ll say I wish I could take you on the road with me because that was such a
0:15:04 great description. Can I be your opening actor? Oh, my God. No, I’m going to open for you, dude.
0:15:10 Otherwise, it’s like everybody leaves after you’re done.
0:15:13 Believe me, I did that in Columbus, Ohio once. It wasn’t fun. The opening acts drank our bar tab.
0:15:22 We spent all this money going all the way there. Everybody left after the opening acts were done,
0:15:28 and there was just that stoner dude with the dreadlocks hanging out. And then next to you,
0:15:32 we blew our savings on getting a hotel room. So we should, as a small tangent, you’re a legit
0:15:40 touring act. When I was in grad school, I played in a band. And yeah, we traveled. We would play
0:15:45 shows. It wasn’t like we were in a hardcore touring band, but we did some touring and had some fun
0:15:50 times. And yeah, we did a movie soundtrack. Nice. Henry Portrait of Serial Killer. So that’s a good
0:15:57 movie. We were on the soundtrack for the sequel, Henry II, Mask of Sanity, which is a terrible
0:16:02 movie. Yeah. How’s the soundtrack? It’s pretty good. It’s badass. At least that one part where the
0:16:06 guy throws up the milkshake. It’s my song. We’re going to have to see it. We’re going to have to
0:16:11 see it. All right, we’re getting back to life advice. And happiness, yeah. One thing that I try
0:16:16 to live by, especially nowadays, and since I wrote the book, I’ve been thinking more and more about
0:16:21 this is, how do I want to live a memorable life? I think if we go back to the pandemic,
0:16:28 how many people have memories from that period, aside from the trauma of being locked up and
0:16:37 seeing people die and all this stuff? I think it’s like one of these things where we were stuck
0:16:43 inside looking at screens all day, doing the same thing with the same people. And so I don’t
0:16:51 remember much from that in terms of those good memories that you’re talking about. When I was
0:16:56 growing up, my parents worked really hard for us and we went on some vacations, but not very often.
0:17:03 And I really try to do now vacations to interesting places as much as possible with my family,
0:17:09 because those are the things that you remember. So I really do think about
0:17:16 what’s going to be something that’s memorable and then just do it, even if it’s a pain in the
0:17:21 ass, because the experiencing self will suffer for that, but the remembering self will be like,
0:17:26 “Yes, I’m so glad I did that.” Do things that are very unpleasant in the moment,
0:17:31 because those can be reframed and enjoyed for many years to come. That’s probably
0:17:38 good advice or at least when you’re going through shit, it’s a good way to see the silver lining
0:17:43 of it. Yeah, I think it’s one of these things where if you have people who you’ve gone through,
0:17:49 since you said it, I’ll just say, since you’ve gone through shit with someone, and it’s like,
0:17:54 that’s a bonding experience often. I mean, that can really bring you together.
0:18:00 I like to say it’s like there’s no point in suffering unless you get a story out of that.
0:18:05 So in the book, I talk about the power of the way we communicate with others and how that
0:18:10 shapes our memories. And so I had this near-death experience, at least that’s how I remember it,
0:18:16 on this paddle board, where just everything they could have gone wrong did go wrong almost.
0:18:20 So many mistakes were made and ended up at some point just like basically
0:18:29 away from my board, pinned in a current like in this corner, like not a super good swimmer,
0:18:35 and my friend who came with me, Randy, who’s a computational neuroscientist, and he had just
0:18:40 been pushed down past me and so he couldn’t even see me. And I’m just like, if I die here,
0:18:47 I mean, no one’s around. It’s like you just die alone. And so I just said, well, failure is not
0:18:53 an option. And eventually, I got out of it and froze and got cut up. And I mean, the things that
0:19:01 we were going through were just insane. But a short version of this is, you know, my wife
0:19:08 and my daughter and Randy’s wife, they gave us all sorts of hell about this because they were
0:19:13 just like, where are we? They were ready to send out a search party. So they were giving me hell
0:19:18 about it. And then I started to tell people in my lab about this and then friends. And
0:19:23 it just became a better and better story every time. And we actually had some photos of
0:19:28 just the crazy things like this generator that was hanging over the water and were like ducking
0:19:33 under the zinger, these metal gratings, and I’m like going flat. And it was just nuts, you know.
0:19:38 But it became a great story. And it was definitely, I mean, Randy and I were already tight, but that
0:19:43 was a real bonding experience for us. And yeah, I mean, and I learned from that that it’s like,
0:19:49 I don’t look back on that enough, actually. Because I think we often, at least for me,
0:19:56 I don’t necessarily have the confidence to think that things will work out that I’ll be able to
0:20:00 get through a certain thing. But my ability to actually get something done in that moment
0:20:08 is better than I give myself credit for, I think. And that was the lesson of that story that I
0:20:13 really took away. Well, actually, just for me, you’re making me realize now that it’s not just
0:20:19 those kinds of stories, but even things like periods of depression or really low points.
0:20:26 To me, at least it feels like a motivating thing that the darker it gets, the better the story
0:20:34 will be if you emerge on the other side. That to me feels like a motivating thing. So maybe if
0:20:40 people listening to this and they’re going through some shit, as we said, one thing
0:20:45 that could be a source of light is that it’ll be a hell of a good story when it’s all over,
0:20:51 when you emerge on the other side. Let me ask you about decisions. You’ve already talked about it
0:20:57 a little bit, but when we face the world and we’re making different decisions,
0:21:01 how much does our memory come into play? Is it the kind of narratives that we’ve
0:21:09 constructed about the world that are used to make predictions that’s fundamentally part
0:21:14 of the decision making? Absolutely. Yeah. So let’s say after this, you and I decided we’re
0:21:19 going to go for a beer, right? How do you choose where to go? You’re probably going to be like,
0:21:23 oh, yeah, this new bar opened up near me at a great time there. They had a great beer selection,
0:21:28 or you might say, oh, we went to this place and it was totally crowded and they’re playing this
0:21:32 horrible EDM or whatever. So right there, valuable source of information, right? And then you have
0:21:40 these things like where you do this counterfactual stuff like, well, I did this previously, but what
0:21:46 if I had gone somewhere else instead? Maybe I’ll go to this other place because I didn’t try it
0:21:50 the previous time. So there’s all that kind of reasoning that goes into it too. I think,
0:21:56 even if you think about the big decisions in life, right? It’s like you and I were talking
0:22:01 before we started recording about how I got into memory research and you got into AI. And it’s like
0:22:07 we all have these personal reasons that guide us in these particular directions. And some of it’s
0:22:13 the environment and random factors in life. And some of it is memories of things that we want to
0:22:19 overcome or things that we build on in a positive way, but either way, they define us.
0:22:27 And probably the earlier in life, the memories happen, the more defining, the more defining
0:22:33 power they have in terms of determining who we become. I mean, I do feel like adolescence is
0:22:39 much more important than I think people give credit for. I think that there is this kind of a sense
0:22:43 like the first three years of life is the most important part. But the teenage years are just
0:22:50 so important for the brain. And so that’s where a lot of mental illness starts to emerge.
0:22:57 Now we’re thinking of things like schizophrenia as a neurodevelopmental disorder, because it just
0:23:03 emerges during that period of adolescence and early adulthood. And I think the other part
0:23:09 of it is that I guess I was a little bit too firm in saying that memory determines who we are. It’s
0:23:15 really the self as an evolving construct. I think we kind of underestimate that. And when you’re
0:23:21 a parent, you feel like every decision you make is consequential in forming this child and plays
0:23:29 a role. But so do the child’s peers. And so do there’s so much. I mean, that’s why I think
0:23:36 the big part of education I think that’s so important is not the content you learn. I mean,
0:23:40 think of how much dumb stuff we learned in school, right? But a lot of it is learning
0:23:47 how to get along with people and learning who you are and how you function. And that can be
0:23:54 terribly traumatizing even if you have perfect parents working on you.
0:23:59 Is there some insight into the human brain that explains why we don’t seem to remember anything
0:24:06 from the first few years of life? Yeah. Yeah. In fact, actually, I was just talking to my
0:24:12 really good friend and colleague, Simona Getty, who studies the neuroscience of child development.
0:24:17 And so we were talking about this. And so there are a bunch of reasons, I would say. So one reason
0:24:23 is there’s an area of the brain called the hippocampus, which is very, very important for
0:24:28 remembering events or episodic memory. And so the first two years of life, there’s a period called
0:24:34 infantile amnesia. And then the next couple of years of life after that, there’s a period called
0:24:40 childhood amnesia. And the difference is that basically in the lab and even during childhood
0:24:46 and afterwards, children basically don’t have any episodic memories for those first two years.
0:24:54 The next two years, it’s very fragmentary. And that’s why they call it childhood amnesia. So
0:24:58 there’s some, but it’s not mine. So one reason is that the hippocampus is taking some time to develop.
0:25:04 But another is the neocortex. So the whole folded stuff of gray matter all around the hippocampus
0:25:10 is developing so rapidly and changing. And a child’s knowledge of the world is just massively
0:25:17 being built up. So I’m going to probably embarrass myself, but it’s like, if you
0:25:22 showed like you trained a neural network and you give it the first couple of patterns or something
0:25:28 like that, and then you bombard it with another year’s worth of data, try to get back those first
0:25:34 couple of patterns, right? It’s like everything changes. And so the brain is so plastic. The
0:25:40 cortex is so plastic during that time. And we think that memories for events are very distributed
0:25:46 across the brain. So imagine you’re trying to get back that pattern of activity that happened
0:25:51 during this one moment. But the roads that you would take to get there have been completely
0:25:56 rerouted, right? So I think that’s my best explanation. The third explanation is a child’s
0:26:01 sense of self takes a while to develop. And so their experience of learning might be more learning
0:26:09 what happened as opposed to having this first person experience of “I remember, I was there.”
0:26:14 Well, I think somebody once said to me that kind of loosely, philosophically, that the reason we
0:26:25 don’t remember the first few years of life, infantile amnesia, is because how traumatic it is.
0:26:32 Basically, the error rate that you mentioned, when your brain’s prediction doesn’t match reality,
0:26:40 the error rate in the first few years of life, your first few months, certainly,
0:26:44 is probably crazy high. It’s just nonstop freaking out. The collision between your model of the
0:26:52 world and how the world works is just so high that you want whatever the trauma of that is,
0:26:57 not to linger around. I always thought that’s an interesting idea because just imagine the insanity
0:27:05 of what’s happening in a human brain in the first couple of years. You don’t know anything.
0:27:10 And there’s just this stream of knowledge, and given how plastic everything is,
0:27:15 it just molds and figures it out. But it’s like an insane waterfall of information.
0:27:24 I wouldn’t necessarily describe it as a trauma. We can get into this whole stages of life thing,
0:27:27 which I just love. Basically, those first few years, think about it. A kid’s
0:27:34 internal model of their body is changing. It’s like just learning to move. If you ever have a baby,
0:27:42 you’ll know that the first three months, they’re discovering their toes. It’s just nuts.
0:27:47 Everything is changing. But what’s really fascinating is, and I think this is not at all
0:27:54 me being a scientist, but it’s like one of those things that people talk about when they talk about
0:27:58 the positive aspects of children is that they’re exceptionally curious, and they have this kind
0:28:05 of openness towards the world. And so that prediction error is not a negative traumatic
0:28:12 thing. I think it’s like a very positive thing because it’s what they use that they’re seeking
0:28:18 information. One of the areas that I’m very interested in is the prefrontal cortex. It’s an
0:28:23 area of the brain that, I mean, I could talk all day about it, but it helps us use our knowledge
0:28:29 to say, “Hey, this is what I want to do now. This is my goal, so this is how I’m going to
0:28:35 achieve it,” and focus everything towards that goal. The prefrontal cortex takes forever to
0:28:41 develop in humans. The connections are still being tweaked and reformed into late adolescence,
0:28:48 early adulthood, which is when you tend to see mental illness pop up. It’s being massively
0:28:54 reformed. Then you have about 10 years maybe of prime functioning of the prefrontal cortex,
0:29:00 and then it starts going down again, and you end up being older, and you start losing all
0:29:04 that frontal function. So look at this, and you’d say, “Okay, you’ve sitting around episodic memory
0:29:10 talks,” and always say, “Children are worse than adults at episodic memory. Older adults are worse
0:29:15 than young adults at episodic memory,” and I always say, “God, this is so weird. Why would we have
0:29:20 this period of time that’s so short when we’re perfect or optimal?” I like to use the word
0:29:26 “optimal” now because there’s such a culture of optimization right now. I have to redefine what
0:29:33 optimal is because for most of the human condition, I think we had a series of stages of life where
0:29:42 you have basically adults saying, “Okay, young adults,” saying, “I’ve got a child, and I’m part of
0:29:50 this village, and I have to hunt and forage and get things done. I need a prefrontal cortex so
0:29:54 I can stay focused on the big picture and the long-haul goals.” Now, I’m a child. I’m in this
0:30:01 village. I’m kind of wandering around, and I’ve got some safety, and I need to learn about this
0:30:07 culture because I know so little. What’s the best way to do that? Let’s explore. I don’t want to be
0:30:12 constrained by goals as much. I want to really be free, play and explore and learn. You don’t want a
0:30:18 super tight prefrontal cortex. You don’t even know what the goals should be yet. If you’re trying to
0:30:24 design a model that’s based on a bad goal, it’s not going to work well. Then you go late in life,
0:30:32 and you say, “Why don’t you have a great prefrontal cortex then?” If you go back and you think,
0:30:38 “How many species actually stick around naturally long after their child-bearing years are over,
0:30:44 after reproductive years are over?” Menopause, from what I understand, menopause is not all that
0:30:49 common in the animal world. Why would that happen? I saw Alison Gopnik said something about this,
0:30:58 so I started to look into this, about this idea that really, when you’re older in most societies,
0:31:04 your job is no longer to form new episodic memories. It’s to pass on the memories that you
0:31:10 already have, this knowledge about the world, what we call semantic memory, to pass on that
0:31:15 semantic memory to the younger generations, to pass on the culture. Even now in indigenous cultures,
0:31:21 that’s the role of the elders. They’re respected. They’re not seen as people who are pasted and
0:31:26 losing it. I thought that was a very poignant thing, that memory is doing what it’s supposed to
0:31:34 throughout these stages of life. It is always optimal in a sense. It’s just optimal for that
0:31:40 stage of life. For the ecology of the system, I looked into this and it’s like another species
0:31:47 that has menopause is orcas. Orcopods are led by the grandmothers. It’s not the young adults,
0:31:53 not the parents or whatever, the grandmothers. They’re the ones that pass on the traditions
0:31:59 to the younger generation orcas. If you look from what little I understand, different orcopods
0:32:06 have different traditions. They hunt for different things. They have different play traditions.
0:32:11 That’s a culture. In social animals, evolution, I think, is designing brains that are really
0:32:21 around. It’s obviously optimized for the individual, but also for kin. I think that the kin are part
0:32:30 of this, when they’re a part of this intense social group, the brain development should parallel
0:32:35 that the nature of the ecology. It’s just fascinating to think of the individual orca or human
0:32:43 throughout its life in stages doing a kind of optimal wisdom development. In the early days,
0:32:52 you don’t even know what the goal is. You figure out the goal and you optimize for that goal and
0:32:56 you pursue that goal. Then all the wisdom you collect through that, then you share with the
0:33:00 others in the system with the other individuals. As a collective, then you kind of converge towards
0:33:07 greater wisdom throughout the generation. In that sense, it’s optimal. Us humans and orcas
0:33:14 got something going on. It works. Oh, yeah. Apex predators.
0:33:18 I just got a megalon tooth. Speaking of apex predators, just imagine the size of that thing.
0:33:29 Anyway, how does the brain forget and how and why does it remember? Maybe some of the mechanisms.
0:33:39 You mentioned the hippocampus. What are the different components involved here?
0:33:43 We could think about this on a number of levels. Maybe I’ll give you the simplest version first,
0:33:47 which is we tend to think of memories as these individual things and we can just access them
0:33:52 maybe a little bit like photos on your phone or something like that.
0:33:56 In the brain, the way it works is you have this distributed pool of neurons and
0:34:01 the memories are kind of shared across different pools of neurons. What you have is competition,
0:34:08 where sometimes memories that overlap can be fighting against each other.
0:34:12 Sometimes we forget because that competition just wipes things out. Sometimes we forget because
0:34:20 there aren’t the biological signals, which you can get into. I would promote long-term retention.
0:34:26 Lots of times we forget because we can’t find the queue that sends us back to the right memory,
0:34:32 and we need the right queue to be able to activate it. For instance, in a neural network,
0:34:38 you wouldn’t go and you’d say, “This is the memory.” The whole ecosystem of memories
0:34:46 is in the weights of the neural network. In fact, you could extract entirely new memories,
0:34:50 depending on how you feed. You have to have the right query, the right prompt,
0:34:54 to access that whatever the part you’re looking for.
0:34:57 That’s exactly right. In humans, you have this more complex set of ways memory works.
0:35:02 As I said, the knowledge or what you call semantic memory, and then there’s these
0:35:07 memories for specific events, which we call episodic memory. There’s different pieces
0:35:12 of the puzzle that require different kinds of queues. That’s a big part of it too,
0:35:18 is just this kind of what we call retrieval failure.
0:35:21 You mentioned episodic memory, you mentioned semantic memory,
0:35:23 what are the different separations here? What’s working memory, short-term memory,
0:35:28 long-term memory? What are the interesting categories of memory?
0:35:32 Yeah. Memory researchers, we love to cut things up and say, “Is memory one thing or is it two
0:35:39 things?” There’s two things, there’s three things. One of the things that there’s value in that,
0:35:44 and especially experimental value in terms of being able to dissect things,
0:35:49 and the real world is all connected. Speak to your question, working memory,
0:35:53 it was a term that was coined by Alan Battley. It’s basically thought to be this ability to
0:35:58 keep information online in your mind right in front of you at a given time, and to be able
0:36:04 to control the flow of that information, to choose what information is relevant,
0:36:08 to be able to manipulate it, and so forth. One of the things that Alan did that was quite brilliant
0:36:14 was he said, “There’s this ability to passively store information, see things in your mind’s eye,
0:36:20 or hear your internal monologue, but we have that ability to keep information in mind.”
0:36:26 But then we also have this separate, what he called a central executive, which is identified a lot
0:36:33 with the prefrontal cortex. It’s this ability to control the flow of information that’s being
0:36:39 kept active based on what it is you’re doing. Now, a lot of my early work was basically saying
0:36:45 that this working memory, which some memory researchers would call short-term memory,
0:36:49 is not at all independent from long-term memory. That is that a lot of executive function requires
0:36:56 learning, and you have to have like synaptic change for that to happen. But there’s also transient
0:37:02 forms of memory. One of the things I’ve been getting into lately is the idea that we form
0:37:08 internal models of events. The obvious one that I always use is birthday parties. As you go to
0:37:14 a child’s birthday party, once the cake comes out and you just see a candle, you can predict the whole
0:37:22 frame set of events that happens later. Up till that point where the child blows out the candle,
0:37:27 you have an internal model in your head of what’s going on. If you follow people’s eyes,
0:37:33 it’s not actually on what’s happening. It’s going where the action’s about to happen,
0:37:37 which is just fascinating. You have this internal model and that’s a kind of a working memory product.
0:37:43 It’s something that you’re keeping online that’s allowing you to interpret this world around you.
0:37:48 Now, to build that model, though, you need to pull out stuff from your general knowledge of
0:37:53 the world, which is what we call semantic memory. Then you’d want to be able to pull out memories
0:37:59 for specific events that happened in the past, which we call episodic memory. In a way, they’re
0:38:05 all connected even though it’s different. The things that we’re focusing on and the way we
0:38:11 organize information in the present, which is working memory, will play a big role in determining
0:38:16 how we remember that information later, which people typically call long-term memory.
0:38:20 So if you have something like a birthday party and you’ve been to many before,
0:38:23 you’re going to load that from disk into working memory, this model, and then you’re mostly operating
0:38:30 on the model. If it’s a new task, you don’t have a model, so you’re more in the data collection.
0:38:39 Yes, one of the fascinating things that we’ve been studying, and we’re not at all the first
0:38:44 to do this, Jeff Sachs was a big pioneer in this, and I’ve been working with many other people,
0:38:49 Ken Norman, Leila Devachi, or Columbia has done some interesting stuff with this,
0:38:55 as this idea that we form these internal models at particular points of high prediction error,
0:39:02 or points of, I believe, also points of uncertainty, points of surprise, or motivationally significant
0:39:08 periods, and those points are when it’s maximally optimal to encode an episodic memory. So I used
0:39:15 to think, “Oh, well, we’re just encoding episodic memories constantly.” But think about how much
0:39:21 redundancy there is in all that. It’s just a lot of information that you don’t need.
0:39:27 But if you capture an episodic memory at the point of maximum uncertainty for the singular
0:39:35 experience, it’s only going to happen once, but if you capture it at the point of maximum
0:39:40 uncertainty or maximum surprise, you have the most useful point in your experience that you’ve grabbed,
0:39:46 and what we see is that the hippocampus and these other networks that are involved in
0:39:52 generating these internal models of events, they show a heightened period of connectivity,
0:39:58 or correlated activity, during those breaks between different events, which we call event
0:40:03 boundaries. These are the points where you’re surprised, or you cross from one room to another,
0:40:08 and so forth, and that communication is associated with a bump of activity in the hippocampus and
0:40:13 better memory. And so if people have a very good internal model throughout that event,
0:40:21 you don’t need to do much memory processing here in a predictive mode. And so then, at these event
0:40:27 boundaries, you encode, and then you retrieve and you’re like, “Okay, wait a minute, what’s going on
0:40:31 here?” Branganath’s now talking about orcas, what’s going on, and maybe you have to go back and
0:40:36 remember reading my book to pull out the episodic memory to make sense of whatever it is I’m babbling
0:40:40 about. And so there’s this beautiful dynamics that you can see in the brain of these different
0:40:47 networks that are coming together and then de-affiliating at different points in time
0:40:52 that are allowing you to go into these modes. And so to speak to your original question,
0:40:57 to some extent, when we’re talking about semantic memory and episodic memory and working memory,
0:41:02 you can think about it as these processes that are unfolding as these networks come together
0:41:07 and pull apart. Can memory be trained and improved? This beautiful connected system that you’ve
0:41:15 described, what aspect of it is a mechanism that can be improved through training?
0:41:21 I think improvement, it depends on what your definition of optimal is. So what I say in the
0:41:27 book is that you don’t want to remember more, you want to remember better, which means focusing
0:41:33 on the things that are important. And that’s what our brains are designed to do.
0:41:37 So if you go back to the earliest quantitative studies of memory by Ebbinghaus, what you see
0:41:42 is that he was trying so hard to memorize this arbitrary nonsense. And within a day,
0:41:49 he lost about 60% of that information. And he was basically using a very, very generous way
0:41:55 of measuring it. So as far as we know, nobody has managed to violate those basics of having people
0:42:02 forget most of their experiences. So if your expectation is that you should remember everything
0:42:07 and that’s what your optimal is, you’re already off because this is just not what human brains
0:42:12 are designed to do. On the other hand, what we see over and over again is that the brain does,
0:42:18 basically, one of the cool things about the design of the brain is it’s always less is more.
0:42:23 I’ve seen estimates that the human brain uses something like 12 to 20 watts in a day. I mean,
0:42:30 that’s just nuts, the low power consumption, right? So it’s all about reusing information
0:42:36 and making the most of what we already have. And so that’s why, basically, again, what you see
0:42:43 biologically is neuromodulators, for instance, these chemicals in the brain like norepinephrine,
0:42:50 dopamine, serotonin, these are chemicals that are released during moments that tend to be
0:42:56 biologically significant, surprise, fear, stress, et cetera. And so these chemicals
0:43:03 promote lasting plasticity, right? Essentially, some mechanisms by which the brain can prioritize
0:43:10 the information that you carry with you into the future. Attention is a big factor as well,
0:43:15 our ability to focus our attention on what’s important. And so there’s different schools of
0:43:22 thought on training attention, for instance. So one of my colleagues, Amishi Ja, she wrote a
0:43:29 book called Peak Mind and talks about mindfulness as a method for improving attention and focus.
0:43:35 So she works a lot with military like Navy SEALs and stuff to do this kind of work
0:43:40 with mindfulness meditation. Adam Ghazali, in other words, my friends and colleagues,
0:43:45 has worked on kind of training through video games actually as a way of training attention. And so
0:43:51 it’s not clear to me, you know, one of the challenges though in training is you tend to
0:43:56 overfit to the thing that you’re trying to optimize, right? So you tend to, if I’m looking at a video
0:44:03 game, I can definitely get better at paying attention in the context of the video game,
0:44:07 but you transfer it to the outside world. That’s very controversial.
0:44:11 The implication there is that attention is a fundamental component of remembering something,
0:44:17 allocating attention to it. And then attention might be something that you could train,
0:44:23 how you allocate attention and how you hold attention on a thing.
0:44:28 I can say that in fact, we do in certain ways, right? So if you are an expert in something,
0:44:34 you are training attention. So we did this one study of expertise in the brain. And
0:44:40 people used to think, let’s say if you’re a bird expert or something, right? People will go like,
0:44:45 if you get really into this world of birds, you start to see the differences in your visual
0:44:50 cortex is tuned up and it’s all about plasticity of the visual cortex. And vision researchers
0:44:55 love to say everything’s visual. But it’s like, we did this study of attention and working memory
0:45:02 and expertise. And one of the things that surprised us were the biggest effects as people became
0:45:07 experts in identifying these different kinds of just crazy objects that we made up.
0:45:12 As they developed this expertise of being able to identify what made them different from each
0:45:16 other and what made them unique, we were actually seeing massive increases in activity in the
0:45:21 prefrontal cortex. And this fits with some of the studies of chest experts and so forth that
0:45:26 it’s not so much that you learn the patterns passively, you learn what to look for, you learn
0:45:32 what’s important, what’s not, right? And you can see this in any kind of expert professional athlete,
0:45:38 they’re looking three steps ahead of where they’re supposed to be. So that’s a kind of a training of
0:45:43 attention. And those are also what you’d call expert memory skills. So if you take the memory
0:45:49 athletes, I know that’s something we’re both interested in. So these are people who train in
0:45:54 these competitions and they’ll memorize like a deck of cards in like a really short amount of time.
0:46:00 There’s a great memory athlete, her name I think is pronounced Yenya Wintressol.
0:46:06 So I think she’s got like a giant Instagram following. And so she had this YouTube video
0:46:12 that went viral where she had memorized an entire Ikea catalog, right? And so how do people do this?
0:46:19 By all accounts, from people who become memory athletes, they weren’t born with some extraordinary
0:46:25 memory. But they practice strategies over and over and over again. The strategy that they use
0:46:31 for memorizing a particular thing, it can become automatic and you can just deploy it in an instant,
0:46:36 right? So again, it’s not necessarily going to, one strategy for learning the order of a deck of
0:46:42 cards might not help you for something else that you need like, you know, remembering your way around
0:46:47 Austin, Texas. But it’s going to be these, whatever you’re interested in, you can optimize for that.
0:46:54 And that’s just a natural byproduct of expertise.
0:46:58 There’s certain hacks. There’s something called the memory palace that I played with. I don’t
0:47:02 know if you’re familiar with that whole technique. And it works. It’s interesting. So another thing
0:47:08 I recommend for people a lot is I use Anki a lot every day. It’s an app that does spaced repetition.
0:47:16 So I think medical students and like students use this a lot to remember a lot of different things.
0:47:20 Oh yeah. Okay. We can come back to this. But yeah, sure. It’s the whole concept of space repetition.
0:47:24 You just, when the thing is fresh, you kind of have to remind yourself of it a lot. And then
0:47:31 over time, you can wait a week, a month, a year before you have to recall the thing again. And
0:47:38 that way, you essentially have something like note cards that you can have tens of thousands of
0:47:44 and can only spend 30 minutes a day and actually be refreshing all of that information, all that
0:47:50 knowledge. It’s really great. And then for a memory palace is a technique that allows you
0:47:56 to remember things like the IKEA catalog or by placing them visually in a place that you’re
0:48:02 really familiar with. Like I’m really familiar with this place. So I can put numbers or facts or
0:48:09 whatever you want to remember. You can walk along that little palace and it reminds you.
0:48:13 It’s cool. Like there’s stuff like that that I think athletes, memory athletes could use,
0:48:20 but I think also regular people can use. One of the things I have to solve for myself is how
0:48:24 to remember names. I’m horrible at it. I think it’s because when people introduce themselves,
0:48:30 I have the social anxiety of the interaction where I’m like, I know I should be remembering that,
0:48:39 but I’m freaking out internally about social interaction in general. And so,
0:48:45 therefore, I forget immediately. So I’m looking for good tricks for that.
0:48:49 I feel like we’ve got a lot in common because when people introduce themselves to me, it’s almost
0:48:57 like I have this blank blackout for a moment and then I’m just looking at them like, what happened?
0:49:04 I look away or something. What’s wrong with me? I’m totally with you on this.
0:49:08 The reason why it’s hard is that there’s no reason we should be able to remember names
0:49:14 because when you say remembering a name, you’re not really remembering a name. Maybe in my case,
0:49:18 you are. But most of the time, you’re associating a name with a face and an identity. And that’s a
0:49:24 completely arbitrary thing. Maybe in the olden days, somebody named Miller, it’s like they’re
0:49:30 actually making flour or something like that. But for the most part, it’s like these names are
0:49:36 just utterly arbitrary. So you have no thing to latch onto. And so it’s not really a thing that
0:49:42 our brain does very well to learn meaningless arbitrary stuff. So what you need to do is build
0:49:48 connections somehow, visualize a connection. And sometimes it’s obvious, or sometimes it’s not.
0:49:55 I’m trying to think of a good one for you now. But the first thing I think of is Lex Luthor.
0:49:59 Because doesn’t Lex Luthor wear a suit, I think? I know he has a shaved head, though,
0:50:07 or he’s bald, which you’re not. You’ve got a great head if I trade hair with you any day.
0:50:11 But something like that. But if I can come up with something, I could say, okay, so Lex Luthor
0:50:17 is this criminal mastermind, and I just imagine you. We talked about stabbing or whatever earlier.
0:50:22 Yeah, exactly. So I’m just kind of connected, and that’s it. Yeah, yeah. But I’m serious,
0:50:26 though, that these kinds of weird associations, now I’m building a richer network. I mean,
0:50:31 one of the things that I find is if you can have somebody’s name that’s just totally generic,
0:50:37 like John Smith or something, not that no offense to people with that name. But if I see a generic
0:50:43 name like that, but I’ve read John Smith’s papers academically, and then I meet John Smith at a
0:50:49 conference, I can immediately associate that name with that face, because I have this pre-existing
0:50:54 network to lock everything into. And so you can build that network. And that’s what the method
0:50:59 of loci or the memory palace technique is all about, is you have a pre-existing structure in your
0:51:05 head of like your childhood home or this mental palace that you’ve created for yourself. And so
0:51:11 now you can put arbitrary pieces of information in different locations in that mental structure of
0:51:18 yours. And then you could walk through the different path and find all the pieces of information
0:51:24 you’re looking for. So the method of loci is a great method for just learning arbitrary things,
0:51:30 because it allows you to link them together and get that cue that you need to pop in
0:51:34 and find everything, right? We should maybe linger on this memory palace thing just to make obvious,
0:51:43 because when people were describing to me a while ago what this is seems insane. I just,
0:51:50 you literally think of a place like a childhood home or a home that you’re really visually
0:51:57 familiar with. And you literally place in that three-dimensional space facts or people or whatever
0:52:08 you want to remember. And you just walk in your mind along that place visually. And you can remember,
0:52:16 remind yourself of the different things. One of the limitations is there is a sequence to it.
0:52:22 So it’s, I think your brain somehow, you need, you can’t just like go upstairs right away or
0:52:27 something. You have to like walk along the room. So it’s really great for remembering sequences,
0:52:31 but it’s also not great for remembering like individual facts out of context. So the full
0:52:36 context of the tour, I think, is important. But it’s fascinating how the mind is able to do that
0:52:42 when you ground these pieces of knowledge into something that you remember well already,
0:52:49 especially visually. Fascinating. And you can just do that for any kind of sequence. I’m sure
0:52:55 she used something like this for the IKEA catalog, something like this nature.
0:52:58 Oh yeah, absolutely, absolutely. And I think the principle here is, again, I was telling you this
0:53:05 idea that memories can compete with each other, right? Well, I like to use this example, and maybe
0:53:11 someday I’ll regret this, but I’ve used it a lot recently, is like, imagine if this were my desk,
0:53:16 it could be cluttered with a zillion different things, right? So imagine it’s just cluttered
0:53:19 with a whole bunch of yellow Post-it notes. And on one of them, I put my bank password on it, right?
0:53:24 Well, it’s going to take me forever to find it. I might, you know, it’s just going to be buried
0:53:28 under all these other Post-it notes. But if it’s like hot pink, it’s going to stand out and I find
0:53:33 it really easily, right? And so that’s one way in which if things are distinctive, if you’ve
0:53:39 processed information in a very distinctive way, then you can have a memory that’s going to last.
0:53:45 And that’s very good, for instance, for name-face associations. If I get something distinctive
0:53:51 about you, you know, that it’s like, that you’ve got very short hair, and maybe I can make the
0:53:55 association with Lex Luthor that way, or something like that, right? You know, but I get something
0:53:59 very specific. That’s a great cue. But the other part of it is, what if I just organized my notes
0:54:05 so that I have my finances in one pile, and I have my like reminders, my to-do list in one pile,
0:54:11 and so forth. So I organized them. Well, then I know exactly if I’m going for my banking,
0:54:17 you know, my bank password, I could go to the finance pile, right? So the method of loci works
0:54:23 or memory palaces work because they give you a way of organizing. There’s a school of thought that
0:54:29 says that episodic memory evolved from this like kind of knowledge of space. And, you know,
0:54:35 basically, there’s primitive abilities to figure out where you are. And so people explain the
0:54:40 method of loci that way. And, you know, whether or not the evolutionary argument is true,
0:54:46 the method of loci is not at all special. So if you don’t, you’re not a good visualizer.
0:54:50 Stories are a good one. So a lot of memory athletes will use stories, and they’ll
0:54:56 go like, if you’re memorizing a deck of cards, they have a little code for the different, like,
0:55:01 like the king and the jack and the ten and so forth. And they’ll make up a story about things
0:55:07 that they’re doing. And that’ll work. Songs are a great one, right? I mean, it’s like, I can still
0:55:12 remember there’s this obscure episode of the TV show “Cheers.” They sing a song about Albania
0:55:17 that he uses to memorize all these facts about Albania. And I could still sing that song to you.
0:55:24 It’s just I saw it on a TV show, you know. So you mentioned space repetition. So what,
0:55:29 do you like this, Prasik? Maybe can you explain it? Oh, yeah. If I’m trying to memorize something,
0:55:34 let’s say if I have an hour to memorize as many Spanish words as I can, if I just try to do,
0:55:40 like, half an hour, and then later in the day, I do half an hour, I won’t retain that information
0:55:46 as long as if I do half an hour today and half an hour one week from now. And so doing that extra
0:55:53 spacing should help me retain the information better. Now, there’s an interesting boundary condition,
0:56:00 which is it depends on when you need that information. So many of us, you know, for me,
0:56:06 like, I can’t remember so much from college and high school because I crammed because I just did
0:56:11 everything at last minute. And sometimes I would literally study, like, you know, in the hallway
0:56:17 right before the test. That was great. Because what would happen is I just had that information
0:56:22 right there. And so actually not spacing can really help you if you need it very quickly,
0:56:29 right? But the problem is, is that you tend to forget it later on. But on the other hand,
0:56:34 if you space things out, you get a benefit for later on retention. And so there’s many different
0:56:41 explanations. We have a computational model of this, it’s currently under revision. But in our
0:56:46 computer model, what we say is that an easy, maybe a good way of thinking about this is
0:56:52 this conversation that you and I are having. It’s associated with a particular context,
0:56:58 a particular place in time. And so all these little cues that are in the background, these
0:57:02 little guitar sculptures that you have, and that big light umbrella thing, right? All these things
0:57:07 are part of my memory for what we’re talking about the content. So now later on, you’re sitting around
0:57:15 and you’re at home drinking a beer and you think, God, what a strange interview that was, right?
0:57:19 So now you’re trying to remember it, but the context is different. So your current situation
0:57:27 doesn’t match up with the memory that you pulled up. There’s error. There’s a mismatch between what
0:57:32 you pulled up and your current context. And so in our model, what you start to do is you start to
0:57:37 erase or alter the parts of the memory that are associated with a specific place and time,
0:57:43 and you heighten the information about the content. And so if you remember this information in
0:57:50 different times in different places, it’s more accessible at different times in different places
0:57:56 because it’s not overfitted in a AI kind of way of thinking about things. It’s not overfitted to
0:58:01 one particular context. But that’s also why the memories that we call upon the most also feel
0:58:07 kind of like they’re just things that we read about almost. You don’t vividly reimagine them,
0:58:11 right? It’s just these things that just come to us like facts, right? And it’s a little bit
0:58:17 different than semantic memory, but basically these events that we have recalled over and over
0:58:24 and over again, we keep updating that memory so it’s less and less tied to the original experience.
0:58:29 But then we have those other ones, which it’s like you just get a reminder of that very specific
0:58:34 context. You smell something, you hear a song, you see a place that you haven’t been to in a while,
0:58:40 and boom, it just comes back to you. And that’s the exact opposite of what you get with spacing,
0:58:45 right? That’s so fascinating. So with space repetition, one of its powers is that you lose
0:58:50 attachment to a particular context, but then it loses the intensity of the flavor of the memory.
0:58:59 That’s interesting. That’s so interesting. Yeah, but at the same time, it becomes stronger
0:59:05 in the sense that the content becomes stronger. Yeah, so it’s used for learning languages,
0:59:09 for learning facts, for learning, for that generic semantic information type of memory.
0:59:14 Yeah, and I think this falls into a category we’ve done other modeling. One of these is
0:59:20 published study in PLOS, Computational Biology, where we showed that another way, which is I think
0:59:27 related to the spacing effect is what’s called the testing effect. So the idea is that if you’re
0:59:33 trying to learn words, let’s say in Spanish or something like that, and this doesn’t have to
0:59:38 be words, it could be anything, you test yourself on the words and that active testing yourself
0:59:44 helps you retain it better over time than if you just studied it, right? And so from traditional
0:59:51 learning theories, some learning theories anyway, this seems weird, why would you do better giving
0:59:57 yourself this extra error from testing yourself rather than just giving yourself perfect input
1:00:03 that’s a replica of what it is that you’re trying to learn. And I think the reason is that you get
1:00:08 better retention from that error, that mismatch that we talked about, right? So what’s happening
1:00:15 in our model, it’s actually conceptually kind of similar to what happens with backprop in AI,
1:00:21 so there are neural networks. And so the idea is that you expose, here’s the bad connections,
1:00:26 and here’s the good connections. And so we can keep the parts of this cell assembly that are
1:00:32 good for the memory and lose the ones that are not so good. But if you don’t stress test the
1:00:37 memory, you haven’t exposed it to the error fully. And so that’s why I think this is a thing that I
1:00:43 come back to over and over again, is that you will retain information better if you’re constantly
1:00:50 pushing yourself to your limit, right? If you are feeling like you’re coasting, then you’re actually
1:00:57 not learning. So you should always be stress testing the memory system. Yeah, and feel good
1:01:06 about it. You know, even though everyone tells me, oh, my memory is terrible, in the moment,
1:01:10 they’re overconfident about what they’ll retain later on. So it’s fascinating. And so what happens
1:01:16 is when you test yourself, you’re like, oh, my God, I thought I knew that, but I don’t. And so
1:01:22 it can be demoralizing until you get around that and you realize, hey, this is the way that I learn.
1:01:28 This is how I learn best. It’s like if you’re trying to, you know, star in a movie or something
1:01:36 like that, you know, just sit around reading the script, you actually act it out and you’re going
1:01:40 to botch those lines from time to time, right? You know that there’s an interesting moment you
1:01:43 probably experienced this. I remember a good friend of mine, Joe Rogan, I was on his podcast and
1:01:50 we were randomly talking about soccer football. Somebody I grew up watching, Diego Armando
1:02:00 Maradona, one of the greatest soccer players of all time. And we were talking about him and his
1:02:05 career and so on. And Joe asked me if he’s still around. Now, and I said, yeah, I don’t know why
1:02:19 I thought yeah, because that was a perfect example of memories. He passed away. I tweeted about it,
1:02:27 how heartbroken I was, all this kind of stuff. I like it a year before. I know this, but in my
1:02:33 mind, I went back to the thing I’ve done many times in my head, visualizing some of the epic
1:02:39 run to get on goal and so on. So for me, he’s alive. So I’m in part of the also the conversation
1:02:45 when you’re talking to Joe, there’s stress and the focus is allocated, the attention is allocated
1:02:50 in a particular way. But when I walked away, I was like, in which world was Diego Maradona
1:02:58 still alive? Because I was sure in my head that he was still alive. There was a moment that sticks
1:03:06 with me. I’ve had a few like that in my life where just obvious things just disappear from
1:03:14 mind. And it’s cool, like it shows actually the power of the mind in a positive sense
1:03:19 to erase memories you want erased, maybe. But I don’t know. I don’t know if there’s a good
1:03:24 explanation for that. One of the cool things that I found is that some people really just
1:03:31 revolutionize a field by creating a problem that didn’t exist before. It’s kind of like why I love
1:03:38 science is like engineering is like solving other people’s problems and science is about
1:03:43 creating problems. I’m just much more like I want to break things and create problems.
1:03:48 Not necessarily move fast though. But one of my former mentors, Marsha Johnson, who in my opinion
1:03:55 is one of the greatest memory researchers of all time, she comes up young woman in the field and
1:04:00 it’s mostly Guy Field. And she gets into this idea of how do we tell the difference between
1:04:06 things that we’ve imagined and things that we actually remember? How do we tell I get some
1:04:11 mental experience? Where did that mental experience come from? And it turns out this is a huge problem
1:04:17 because essentially our mental experience of remembering something that happened,
1:04:21 our mental experience of thinking about something, how do you tell the difference?
1:04:26 They’re both largely constructions in our head. And so it is very important. And the way that you
1:04:34 do it is, I mean, it’s not perfect, but the way that we often do it and succeed is by, again,
1:04:41 using our prefrontal cortex and really focusing on the sensory information or the place and time
1:04:48 and the things that put us back into when this information happened. And if it’s something you
1:04:53 thought about, you’re not going to have all of that vivid detail as you do for something that
1:04:57 actually happened. But it doesn’t work all the time. But that’s a big thing that you have to do.
1:05:02 But it takes time. It’s slow and it’s, again, effortful. But that’s what you need to remember
1:05:07 accurately. But what’s cool, and I think this is what you alluded to about how that was an
1:05:11 interesting experience, is imagination is exactly the opposite. Imagination is basically saying,
1:05:18 I’m just going to take all this information from memory, recombine it in different ways and throw
1:05:23 it out there. And so, for instance, Dan Schachter and Donna Addis have done cool work on this.
1:05:29 Demis Hassibis did work on this with Eleanor McGuire and UCL. And this goes back, actually,
1:05:36 to this guy, Frederick Bartlett, who is this revolutionary memory researcher at Bartlett.
1:05:41 He actually rejected the whole idea of quantifying memory. He said, “There’s no statistics in my
1:05:47 book.” He came from this anthropology perspective. And a short version of the story is he just asked
1:05:53 people to recall things. You give people stories in poem, ask people to recall them.
1:05:58 And what he found was people’s memories didn’t reflect all of the details of what they were
1:06:03 exposed to. And they did reflect a lot more. They were filtered through this lens of prior
1:06:09 knowledge, the cultures that they came from, the beliefs that they had, the things they knew.
1:06:14 And so, what he concluded was that he called remembering an imaginative construction,
1:06:20 meaning that we don’t replay the past. We imagine how the past could have been by taking bits and
1:06:27 pieces that come up in our heads. And likewise, he wrote this beautiful taper on imagination,
1:06:32 saying when we imagine something and create something, we’re creating it from these specific
1:06:37 experiences that we’ve had and combining it with our general knowledge. But instead of trying to
1:06:41 focus it on being accurate and getting out one thing, you’re just ruthlessly recombining things
1:06:46 without any necessary goal in mind. I mean, at least that’s one kind of creation.
1:06:54 So imagination is fundamentally coupled with memory in both directions?
1:07:02 I think so. I mean, it’s not clear that it is in everyone, but one of the things that’s been
1:07:09 studied is some patients who have amnesia, for instance, they have brain damage, say, to the
1:07:14 hippocampus. And if you ask them to imagine things that are not in front of them, like imagine what
1:07:20 could happen after I leave this room, they find it very difficult to give you a scenario of what
1:07:27 could happen. Or if they do, it would be more stereotyped, like, yes, this would happen. But
1:07:31 it’s not like they can come up with anything that’s very vivid and creative in that sense.
1:07:35 And it’s partly because when you have amnesia, you’re stuck in the present.
1:07:39 Because to get a very good model of the future, it really helps to have episodic memories to
1:07:45 draw upon, right? And so that’s the basic idea. And in fact, one of the most impressive things,
1:07:52 when people started to scan people’s brains and ask people to remember past events,
1:07:58 what they found was there was this big network of the brain called the default mode network.
1:08:03 It gets a lot of press because it’s like thought to be important. It’s engaged during
1:08:06 mind wandering. And if I ask you to pay attention to something, it only comes on when you stop paying
1:08:12 attention. So people go, oh, it’s this daydreaming network. And I thought this is just ridiculous
1:08:18 research. Who cares? But then what people found was when people recall episodic memories,
1:08:25 this network gets active. And so we started to look into it. And this network of areas is really
1:08:32 closely, functionally interacting with the hippocampus. And so, in fact, some would say the
1:08:38 hippocampus is part of this default network. And if you look at brain images of people,
1:08:44 or brain maps of activation, so to speak, of people imagining possible scenarios of things that
1:08:50 could happen in the future, or even things that couldn’t really be very plausible,
1:08:53 they look very similar. I mean, you know, to the naked eye, they look almost the same as maps of
1:08:59 brain activation when people remember the past. According to our theory, and we’ve got some data
1:09:04 to support this, we’ve broken up this network in various subpieces, is that basically it’s kind of
1:09:09 taking apart all of our experiences, and creating these little Lego blocks out of them. And then
1:09:15 you can put them back together if you have the right instructions to recreate these experiences
1:09:20 that you’ve had, but you could also reassemble them into new pieces to create a model of an event that
1:09:25 hasn’t happened yet. And that’s what we think happens. And when our common ground that we’re
1:09:31 establishing in language, requires using those building blocks to put together a model of what’s
1:09:37 going on. Well, there’s a good percentage of time I personally live in, in the imagined world.
1:09:43 I think of, I have, I do thought experiments a lot. I, you know, take the, the absurdity
1:09:50 of human life as it stands, and play it forward in all kinds of different directions.
1:10:00 Sometimes it’s rigorous thoughts, thought experiments, sometimes it’s fun ones. So
1:10:03 I imagine that that has an effect on how I remember things. And I suppose I have to be
1:10:10 a little bit careful to make sure stuff happened versus stuff that I just imagined happened. And
1:10:17 this also, I mean, some of my best friends are characters inside books that never even existed.
1:10:24 And I’m, you know, there’s some degree to which they actually exist in my mind.
1:10:30 Like these characters exist. Authors exist. Does Efsky exist, but also Brothers Karamazov.
1:10:37 I love that book. One of the few books I’ve read. One of the few literature books that I’ve read,
1:10:43 I should say. I read a lot in school that I don’t remember, but Brothers Karamazov.
1:10:47 But they exist. They exist. And I have almost kind of like conversations with them. It’s interesting.
1:10:51 It’s interesting to allow your brain to kind of play with ideas of the past,
1:10:58 of the imagined, and see it all as one.
1:11:00 Yeah. There was actually this famous mnemonist. He’s kind of like back then the equivalent of a
1:11:06 memory athlete, except he would go to shows and do this. Those described by this really famous
1:11:12 nurse psychologist from Russia named Luria. And so this guy was named Solomon Sharyshevsky.
1:11:18 And he had this condition called synesthesia that basically created these weird associations
1:11:24 between different senses that normally wouldn’t go together. So that gave him this incredibly
1:11:30 vivid imagination that he would use to basically imagine all sorts of things that he would need
1:11:38 to memorize. And he would just imagine, like just create these incredibly detailed things.
1:11:43 And he said that allowed him to memorize all sorts of stuff. But it also really haunted
1:11:49 him by some reports that basically it was like he was at some point, and again, who knows,
1:11:54 the drinking was part of this, but at some point had trouble differentiating his imagination from
1:11:59 reality. And this is interesting because it’s like, I mean, that’s what psychosis is in some
1:12:06 ways is you, first of all, you’re just learning connections from prediction errors that you
1:12:13 probably shouldn’t learn. And the other part of it is, is that your internal signals are being
1:12:19 confused with actual things in the outside world, right? Well, that’s why a lot of this stuff is
1:12:25 both feature and bug. It’s a double-edged sword. Yeah, I mean, it might be why there’s such an
1:12:29 interesting relationship between genius and psychosis. Yeah, maybe they’re just two sides of
1:12:36 the same coin. Humans are fascinating, aren’t they? I think so. Sometimes scary, but mostly
1:12:43 fascinating. Can we just talk about memory sport a little longer? There’s something called the USA
1:12:49 Memory Championship. What are these athletes like? What does it mean to be like elite level at this?
1:12:57 Have you interacted with any of them or reading about them? What have you learned about these
1:13:01 folks? There’s a guy named Henry Ronditer who’s studying these guys. And there’s actually a book
1:13:06 by Joshua Ford called Moonwalking with Einstein where he talks about, he actually, as part of this
1:13:12 book, just decided to become a memory athlete. They often have these life events that make them go,
1:13:18 “Hey, why don’t I do this?” So there was a guy named Scott Hagwood who I write about,
1:13:23 who thought that he was getting chemo for cancer. And so he decided, because chemo,
1:13:32 there’s a well-known thing called chemo brain where people become like they just lose a lot of
1:13:37 their sharpness. And so he wanted to fight that by learning these memory skills. So he bought a
1:13:44 book. And this is the story you hear in a lot of memory athletes is they buy a book by other memory
1:13:49 athletes or other memory experts, so to speak. And they just learn those skills and practice
1:13:55 them over and over again. They start by winning bets and so forth. And then they go into these
1:14:00 competitions. And the competitions are typically things like memorizing long strings of numbers
1:14:05 or memorizing orders of cards and so forth. So there tend to be pretty arbitrary things,
1:14:11 not like things that you’d be able to bring a lot of prior knowledge. But they build the skills that
1:14:18 you need to memorize arbitrary things. Yeah, that’s fascinating. I’ve gotten a chance to work
1:14:23 with something called n-back tasks. So there’s all these kinds of tasks, memory recall tasks that
1:14:28 are used to load up the quote-unquote “work in memory.” And to see the psychologists used it
1:14:36 to test all kinds of stuff, like to see how well you’re good at multitasking. We used it in particular
1:14:41 for the task of driving, like if you fill up your brain with intensive working memory tasks,
1:14:49 how good are you at also not crashing, that kind of stuff. So it’s fascinating. But
1:14:56 again, those tasks are arbitrary in there, usually about recalling a sequence of numbers in some kind
1:15:02 of semi-complex way. Do you have any favorite tasks of this nature in your own studies?
1:15:10 I’ve really been most excited about going in the opposite direction and using things that are
1:15:16 more and more naturalistic. And the reason is, is that we’ve really moved, we’ve moved in that
1:15:22 direction because what we found is that memory works very, very differently when you study it,
1:15:29 when you study memory in the way that people typically remember. And so it goes into a much
1:15:36 more predictive mode. And you have these event boundaries, for instance. But a lot of what
1:15:43 happens is this kind of fascinating mix that we’ve been talking about, a mix of interpretations
1:15:49 and imagination with perception. And so the new direction we’re going in is understanding
1:15:57 navigation in our memory first places. And the reason is, is that there’s a lot of work that’s
1:16:02 done in rats, which is very good work. They have a rat, and they put it in a box, and the rat goes,
1:16:08 chases cheese in a box, and you’ll find cells in the hippocampus that fire when a rat is in
1:16:13 different places in the box. And so the conventional wisdom is that the hippocampus forms this map
1:16:21 of the box. And I think that probably may happen when you have absolutely no knowledge
1:16:29 of the world, right? But I think one of the cool things about human memory is we can bring to bear
1:16:35 our past experiences, economically, learn new ones. And so, for instance, if you learn a map of an
1:16:43 Ikea, let’s say if I go to the Ikea in Austin, I’m sure there’s one here, I probably could go to
1:16:49 this Ikea and find my way to the, you know, where the wine glasses are without having to even think
1:16:56 about it, because it’s got a very similar layout, even though Ikea is a nightmare to get around.
1:17:00 Once I learned my local Ikea, I can use that map everywhere, why form a brand new one for a new
1:17:06 place. And so that kind of ability to reuse information really comes into play when we
1:17:14 look at things that are, you know, more naturalistic tasks. And another thing that we’re really
1:17:21 interested in is this idea of like, what if instead of basically mapping out every coordinate in a
1:17:27 space, you form a pretty economical graph that connects basically the major landmarks together
1:17:35 and being able to use that as, you know, emphasizing the things that are most important,
1:17:40 the places that you go for food and the places that are landmarks that help you get around.
1:17:45 And then filling in the blanks for the rest, because I really believe that cognitive maps
1:17:51 are a mental maps of the world, just like our memories for events are not photographic,
1:17:57 I think there’s this combination of actual verifiable details and then a lot of inference
1:18:03 that you make. So what have you learned about this kind of spatial mapping of places?
1:18:09 How do people represent locations? There’s a lot of variability, I think that, and there’s a lot
1:18:15 of disagreement about how people represent locations in a world of GPS and physical maps.
1:18:22 People can learn it from like basically what they call like survey perspective, being able to see
1:18:27 everything. And so that’s one way in which humans can do it. That’s a little bit different.
1:18:32 There’s one way which we can memorize routes. Like I know how to get from here to, let’s say,
1:18:39 if I knew, walk here from my hotel, I could just rigidly follow that route back, right? And there’s
1:18:44 another more integrative way, which would be what’s called a cognitive map, which would be
1:18:50 kind of a sense of how everything relates to each other. And so there’s lots of people who
1:18:56 believe that these maps that we have in our head are isomorphic with the world. They’re like these
1:19:02 literal coordinates that follow Euclidean space. And as you know, Euclidean mathematics is very
1:19:09 constrained, right? And I think that we are actually much more generative in our maps of space,
1:19:16 so that we do have these bits and pieces. And we’ve got a small task as it’s right now,
1:19:21 not yet like we need to do some work on it for further analyses. But one of the things we’re
1:19:27 looking at is these signals called ripples in the hippocampus, which are these bursts of activity
1:19:34 that you see that are synchronized with areas in the neocortex, in the default network, actually.
1:19:40 And so what we find is that those ripples seem to increase at navigationally important points
1:19:46 when you’re making a decision or when you reach a goal. So it speaks to the emotion thing, right?
1:19:51 Because if you have limited choices, if I’m walking down a street, I could really just get
1:19:58 a mental map of the neighborhood with a more minimal kind of thing by just saying, here’s the
1:20:02 intersections, and here’s the directions I take to get in between them. And what we found in general
1:20:08 in our MRI studies is basically the more people can reduce the problem, whether it’s space or
1:20:16 any kind of decision-making problem, the less the hippocampus encodes. It really is very economical
1:20:23 towards the points of most highest information content and value. So can you describe
1:20:30 the encoding in the hippocampus and the ripples you were talking about?
1:20:33 What’s the signal in which we see the ripples? Yeah, so this is really interesting. There are
1:20:40 these oscillations, right? So there’s these waves that you basically see. And these waves
1:20:46 are points of very high excitability and low excitability. And at least during, they happen
1:20:53 actually during slow wave sleep too. So the deepest stages of sleep when you’re just zonked out, right?
1:20:58 You see these very slow waves where it’s very excitable and then very unexcitable. It goes up
1:21:03 and down. And on top of them, you’ll see these little sharp wave ripples. And when there’s a
1:21:09 ripple in the hippocampus, you tend to see a sequence of cells that resemble a sequence of
1:21:16 cells that fire when an animal is actually doing something in the world. So it almost is like a
1:21:22 little people call it replay. And it’s a little bit, I don’t like that term, but it’s basically
1:21:28 a little bit of a compressed play of the sequence of activity in the brain that was taking place
1:21:35 earlier. And during those moments, there’s a little window of communication between the hippocampus
1:21:41 and these areas in the neocortex. And so that I think helps you form new memories, but it also
1:21:48 helps you, I think, stabilize them, but also really connect different things together in memory
1:21:53 and allows you to build bridges between different events that you’ve had. And so this is one of
1:21:59 hardly start theories of sleep and its real role in helping you see the connections between
1:22:05 different events that you’ve experienced. Oh, so during sleep is when the connections are formed?
1:22:09 The connections between different events, right? So it’s like, you see me now, you see me next week,
1:22:16 you see me a month later, you start to build a little internal model of how I behave and what
1:22:23 to expect of me. And with sleep, one of the things that allows you to do is figure out those
1:22:29 connections and connect the dots and find the signal and the noise. So you mentioned fMRI,
1:22:35 what is it and how is it used in studying memory? This is actually the reason why I got into this
1:22:41 whole field of science is when I was in grad school, fMRI was just really taking off as a
1:22:48 technique for studying brain activity. And what’s beautiful about it is you can study the whole
1:22:53 human brain. And there’s lots of limits to it, but you can basically do it in the person without
1:23:00 sticking anything into their brains. And very noninvasive. And for me, being an MRI scanner is
1:23:06 like being in the womb, I just fall asleep. If I’m not being asked to do anything, I get very
1:23:10 sleepy. But you can have people watch movies while they’re being scanned, or you can have them do
1:23:17 tests of memory, like giving them words and so far to memorize. But what MRI is itself is just this
1:23:24 technique where you put people in a very high magnetic field. Typical ones we would use would be
1:23:31 three Tesla to give you an idea. So a three Tesla magnet, you put somebody in. And what happens is
1:23:37 you get this very weak, but, you know, measurable magnetization in the brain. And then you apply
1:23:44 a radiofrequency pulse, which is basically a different electromagnetic field. And so you’re
1:23:49 basically using water, the water molecules in the brain as a tracer, so to speak. And part of it in
1:23:56 fMRI is the fact that these magnetic fields that you mess with by manipulating these radiofrequency
1:24:06 pulses in the static field, and you have things called gradients would change the strength of
1:24:11 the magnetic field in different parts of the head. So they’re all, we tweak them in different ways.
1:24:16 But the basic idea that we use in fMRI is that blood is flowing to the brain. And when you have
1:24:23 blood that doesn’t have oxygen on it, it’s a little bit more magnetizable than blood that does,
1:24:28 because you have hemoglobin that carries the oxygen, the iron basically in the blood that makes
1:24:33 it red. And so that hemoglobin when it’s deoxygenated actually has different magnetic field properties
1:24:42 than when it has oxygen. And it turns out when you have an increase in local activity in some part
1:24:48 of the brain, the blood flows there. And as a result, you get a lower concentration of hemoglobin
1:24:56 that is not oxygenated. And then that gives you more signal. So I gave you, I think I sent you
1:25:05 a GIF, as you like to say. Yeah, we had off record, intense argument. Well, if it’s pronounced GIF
1:25:13 or GIF, but that’s, we shall set that aside as friends. We could have called it a stern rebuke,
1:25:18 perhaps. Rebuke, yeah. I drew a hard line. It is through the creator of GIFs that it’s
1:25:26 pronounced GIF, but that’s the only person that pronounces GIF. Anyway, yes, you sent a GIF of…
1:25:33 This would be basically a whole, a movie of fMRI data. And so when you look at it, it’s not very
1:25:40 impressive. It looks like these like very pixelated maps of the brain, but it’s mostly kind of like
1:25:45 white. But these tiny changes in the intensity of those signals that you probably wouldn’t be
1:25:51 able to visually perceive, like about 1% can be statistically very, very large effects for us.
1:25:58 And that allows us to see, hey, there’s an increase in activity in some part of the brain
1:26:02 when I’m doing some task like trying to remember something. And I can use those changes to even
1:26:09 predict is a person going to remember this later or not. And the coolest thing that people have done
1:26:15 is to decode what people are remembering from the patterns of activity from, because maybe when
1:26:22 I’m remembering this thing, like I’m remembering the house where I grew up, I might have one pixel
1:26:29 that’s bright in the hippocampus and one that’s dark. And if I’m remembering, you know, something
1:26:34 like more like the car that I used to drive when I was 16, I might see the opposite pattern where
1:26:39 different pixels bright. And so all that little stuff that we used to think of noise, we can now
1:26:45 think of almost like a QR code for memory, so to speak, where different memories have a different
1:26:50 little pattern of bright pixels and dark pixels. And so this really revolutionized my research.
1:26:55 So there’s fancy research out there where people really, I mean, not even that, I mean,
1:27:00 by your standards would be stone age, but you know, applying machine learning techniques to do decoding
1:27:05 and so forth. And now there’s a lot of forward encoding models and you can go to town with this
1:27:10 stuff, right? And I’m much more old school of designing experiments where you basically say,
1:27:17 okay, here’s a whole web of memories that overlap in some way, shape, or form. Do memories that
1:27:25 occurred in the same place have a similar QR code? And do memories that occurred in different
1:27:30 places of different QR code? And you can just use things like correlation coefficients or
1:27:35 cosine distance to measure that stuff, right? Super simple, right? And so what happens is you
1:27:41 can start to get a whole state space of how a brain area is indexing all these different memories.
1:27:47 And it’s super fascinating because what we could see is this little like separation between how
1:27:53 certain brain areas are processing memory for who was there and other brain areas are processing
1:27:58 information about where it occurred or the situation that’s kind of unfolding. And some
1:28:03 are giving you information about what are my goals that are involved and so forth. And so,
1:28:09 and the hippocampus is just putting it all together into these unique things that just
1:28:13 are about when and where it happened. So there’s a separation between spatial information,
1:28:20 concepts, like literally there’s distinct, as you said, QR codes for these.
1:28:28 So to speak, let me try a different analogy to that might be more accessible for people,
1:28:33 which would be like, you’ve got a folder on your computer, right? Open it up. There’s a bunch
1:28:37 of files there. I can sort those files by, you know, alphabetical order. And now things that
1:28:44 both start with letter A are lumped together and things that start with Z versus A are far apart,
1:28:50 right? And so that is one way of organizing the folder, but I could do it by date. And if I do
1:28:55 it by date, things that were created close together in time are close and things that are
1:29:00 far apart in time are far. So every, like you can think of how a brain area or a network of areas
1:29:07 contributes to memory by looking at what the sorting scheme is. And these QR codes that we’re
1:29:14 talking about that you get from fMRI allow you to do that. And you can do the same thing if
1:29:18 you’re recording from massive populations of neurons in an animal. And you can do it for
1:29:25 recording local potentials in the brain, you know, so little waves of activity in, let’s say,
1:29:32 a human who has epilepsy and they stick electrodes in their brain and try to find the seizures.
1:29:37 So that’s some of the work that we’re doing now. But all these techniques basically allow you to
1:29:42 say, hey, what’s the sorting scheme? And so we’ve found that some networks of the brain sort
1:29:48 information and memory according to who was there. So I might have, like we’ve actually shown in one
1:29:54 of my favorite studies of all time that was done by a former postdoc, Zach Raim. And Zach did the
1:30:00 study where we had a bunch of movies with different people in my labs that are two different people.
1:30:05 And you filmed them at two different cafes and two different supermarkets. And what you could
1:30:11 show is in one particular network, you could find the same kind of pattern of activity more
1:30:17 or less, a very, very similar pattern of activity. Every time I saw Alex in one of these movies,
1:30:23 no matter where he was, right? And I could see another one that was like a common
1:30:28 pattern that happened every time I saw this particular supermarket nugget, you know. And it
1:30:36 didn’t matter whether you’re watching a movie or whether you’re recalling the movie. It’s the same
1:30:40 kind of pattern that comes up, right? It’s so fascinating. It’s fascinating. So now you have
1:30:45 those building blocks for assembling a model of what’s happening in the present, imagining what
1:30:51 could happen and remembering things very economically from putting together all these pieces so that
1:30:56 all the hippocampus has to do is get the right kind of blueprint for how to put together all
1:31:02 these building blocks. These are all like beautiful hints at a super interesting system that makes
1:31:09 me wonder on the other side of it how to build it. But it’s like, it’s fascinating. Like the way
1:31:15 does the encoding is really, really fascinating. Or I guess the symptoms, the results of that
1:31:21 encoding are fascinating to study from this. Just as a small tangent, you mentioned sort of the
1:31:26 measuring local potentials with electrodes versus fMRI. Oh yeah. What are some interesting like
1:31:34 limitations, possibilities of fMRI? Maybe the way you explained it is brilliant with blood and
1:31:41 detecting the activations or the excitation because blood flows to that area. What’s the
1:31:49 latency of that? What’s the blood dynamics in the brain that, how quickly can it,
1:31:55 how quickly can the tasks change and all that kind of stuff? Yeah. I mean, it’s very slow to
1:32:02 the brain. 50 milliseconds is like, it’s an eternity. Maybe 50, maybe like
1:32:10 let’s say half a second, 500 milliseconds. Just so much back and forth stuff happens
1:32:17 in the brain in that time. In fMRI, you can measure these magnetic field responses
1:32:24 about six seconds after that burst of activity would take place. All these things, it’s like,
1:32:30 is it a feature or is it a bug? One of the interesting things that’s been discovered
1:32:34 about fMRI is it’s not so tightly related to the spiking of the neurons. We tend to think of
1:32:42 the computation, so to speak, as being driven by spikes, meaning like there’s just a burst of,
1:32:48 it’s either on or it’s off and the neuron’s like going up or down. But sometimes what you can have
1:32:54 is these states where the neuron becomes a little bit more excitable or less excitable.
1:33:00 And so fMRI is very sensitive to those changes in excitability. Actually, one of the fascinating
1:33:06 things about fMRI is where does that, how is it we go from neural activity to essentially
1:33:15 blood flow to oxygen, all this stuff. It’s such a long chain of going from neural activity to
1:33:22 magnetic fields. And one of the theories that’s out there is most of the cells in the brain are
1:33:28 not neurons. They’re actually these support cells called glial cells. And one big one is astrocytes
1:33:34 and they play this big role in regulating kind of being a middle man, so to speak, with the neuron.
1:33:40 So if you, for instance, like one neuron is talking to another, you release a neurotransmitter,
1:33:45 like let’s say glutamate, and that gets another neuron starts getting active after you release
1:33:51 in the gap between the two neurons called synapse. So what’s interesting is if you leave that,
1:33:58 imagine you’re just flooded with this liquid in there, right? If you leave it in there too long,
1:34:03 you just excite the other neuron too much and you can start to basically get seizure activity.
1:34:07 You don’t want this. So you got to suck it up. And so actually what happens is these astrocytes,
1:34:12 one of their functions is to suck up the glutamate from the synapse. And that is a massively, and
1:34:19 then break it down and then feed it back into the neurons that you reuse it. But that cycling is
1:34:25 actually very energy intensive. And what’s interesting is at least according to one theory,
1:34:30 and they need to work so quickly that they’re working on metabolizing the glucose that comes
1:34:36 in without using oxygen, kind of like what, you know, anaerobic metabolism. So they’re not using
1:34:42 oxygen as fast as they are using glucose. So what we’re really seeing in some ways may be in fMRI,
1:34:52 not the neurons themselves being active, but rather the astrocytes, which are meeting the
1:34:58 metabolic demands of the process of keeping the whole system going.
1:35:02 It does seem to be that fMRI is a good way to study activation. So with these astrocytes,
1:35:08 even though there’s a latency, it’s pretty reliably coupled to the activations.
1:35:16 Oh, well, this gets me to the other part. So now let’s say, for instance,
1:35:20 if I’m just kind of like I’m talking to you, but I’m kind of paying attention to your cowboy hat,
1:35:24 right? So I’m looking off to the room. I’m thinking about the right, even if I’m not looking at it.
1:35:28 What you’d see is that there would be this little elevation in activity in areas in
1:35:35 the visual cortex, which process vision around that point in space. Okay. So if then something
1:35:43 happened, like, you know, suddenly a light flashed in that part of, you know, right in front of your
1:35:48 cowboy hat, I would have a bigger response to it. But what you see in fMRI is even if I’m not,
1:35:54 even if I don’t see that flash of light, there’s a lot of activity that I can measure,
1:35:58 because you’re kind of keeping it excitable and that in and of itself, even though I’m not
1:36:03 seeing anything there that’s particularly interesting, there’s still this increase in
1:36:08 activity. And so it’s more sensitive with fMRI. So is that a feature or is it a bug? You know,
1:36:12 some people, people who study spikes in neurons would say, well, that’s terrible. We don’t want
1:36:17 that, you know. Likewise, it’s slow. And that’s terrible for measuring things that are very fast.
1:36:23 But one of the things that we found in our work was when we give people movies and when we give
1:36:29 people stories to listen to, a lot of the action is in the very, very slow stuff. It’s in, because
1:36:36 if you’re thinking about, like, a story, let’s say, you’re listening to a podcast or listening
1:36:42 to the Lex Friedman podcast, right? You’re putting this stuff together and building this
1:36:46 internal model over several seconds, which is basically, we filter that out when we look at
1:36:51 electrical activity in the brain, because we’re interested in this millisecond scale. It’s almost
1:36:55 massive amounts of information, right? So the way I see it is every technique gives you a little
1:37:02 limited window into what’s going on. fMRI is huge problems. You know, people lie down in the scanner.
1:37:08 There’s parts of the brain where I’ll show you in some of these images where you’ll see kind of
1:37:13 gaping holes, because you can’t keep the magnetic field stable in those spots. You’ll see parts where
1:37:20 it’s like there’s a vein, and so it just produces big increases and decreases in signal or respiration
1:37:26 that causes these changes. There’s lots of artifacts and stuff like that. Every technique has
1:37:31 its limits. If I’m lying down in an MRI scanner, I’m lying down. I’m not interacting with you
1:37:36 in the same way that I would in the real world. But at the same time, I’m getting data that I
1:37:43 might not be able to get otherwise. And so different techniques give you different kinds of
1:37:47 advantages. What kind of big scientific discoveries, maybe the flavor of discoveries have been done
1:37:52 throughout the history of the science of memory, the studying of memory? What kind of things
1:37:59 have been understood? Oh, there’s so many. It’s really so hard to summarize it. I mean, I think
1:38:08 it’s funny because it’s like, when you’re in the field, you can get kind of blasé about this stuff.
1:38:13 But then once I started to write the book, I was like, oh my god, this is really interesting. How
1:38:17 did we do all this stuff? I would say that some of the, I mean, from the first studies, just showing
1:38:26 how much we forget is very important. Showing how much schemas, which is our organized knowledge
1:38:33 about the world, increase our ability to remember information, just massively increase in studies
1:38:41 of expertise, showing how experts like chess experts can memorize so much in such a short
1:38:46 amount of time because of the schemas they have for chess. But then also showing that those lead
1:38:52 to all sorts of distortions in memory, the discovery that the act of remembering can change the memory,
1:38:59 can strengthen it, but it can also distort it if you get misinformation at the time.
1:39:04 And it can also strengthen or weaken other memories that you didn’t even recall. So just
1:39:10 this whole idea of memory as an ecosystem, I think, was a big discovery. I could go, this idea of
1:39:17 like breaking up our continuous experience into these discrete events, I think, was a major
1:39:23 discovery. So the discreteness of our encoding of events? Maybe, yeah. I mean, you know, and again,
1:39:28 there’s controversial ideas about this, right? But it’s like, yeah, this idea that, and this gets
1:39:33 back to just this common experience of you walk into the kitchen, and you’re like, why am I here?
1:39:38 And you just end up grabbing some food from the fridge, and then you go back and you’re like,
1:39:42 oh, wait a minute, I left my watch in the kitchen. That’s what I was looking for.
1:39:45 And so what happens is, is that you have a little internal model of where you are,
1:39:50 what you’re thinking about. And when you cross from one room to another, those models get updated.
1:39:56 And so now when you’re in the kitchen, you have to go back and mentally time travel back to this
1:40:00 earlier point to remember what it was that you went there for. And so these event boundaries,
1:40:06 turns out like in our research, and again, I don’t want to make it sound like we’ve figured out
1:40:11 everything. But in our research, one of the things that we found is that basically, as people get
1:40:18 older, the activity in the hippocampus at these event boundaries tends to go down. But independent
1:40:27 of age, if I give you outside of the scanner, you’re done with the scanner, I just scan you
1:40:31 while you’re watching a movie, you just watch it, you come out, I give you a test of memory for
1:40:35 stories. What happens is you find this incredible correlation between the activity in the hippocampus
1:40:43 at these singular points in time, these event boundaries, and your ability to just remember
1:40:49 a story outside of the scanner later on. So it’s marking this ability to encode memories,
1:40:54 just these little snippets of neural activity. So I think that’s a big one. There’s all sorts of
1:41:00 work in animal models that I can get into, you know, sleep. I think there’s so much interesting
1:41:06 stuff that’s being discovered in sleep right now, being able to just record from large populations
1:41:14 of cells, and then be able to relate that. And I think the coolest thing gets back to this QR
1:41:20 code thing, because what we can do now is I can take fMRI data while you’re watching a movie,
1:41:27 or let’s do better than that. Let me get fMRI data while you use a joystick to move around in
1:41:32 virtual reality, right? You’re in the metaverse, whatever, right? But it’s kind of a crappy
1:41:36 metaverse, because there’s always so much metaversing you can do in an MRI scanner. So
1:41:40 what you do is crappy metaversing. So now I can take a rat, record from his hippocampus,
1:41:46 and prefrontal cortex in all these areas, with these really new electrodes, get massive amounts
1:41:51 of data, and have it move around on a trackball in virtual reality in the same metaverse that I
1:41:58 did, and record that rat’s activity. I can get a person with epilepsy, who we have electrodes in
1:42:04 their brain anyway, to try to figure out where the seizures are coming from. And it’s a healthy
1:42:08 part of the brain, record from that person, right? And I can get a computational model.
1:42:14 And one of the one of the brand new members in my lab, Tyler Bond, is just doing some great
1:42:19 stuff. He relates computer vision models, and looks at the weaknesses of computer vision models,
1:42:25 and relates it to what the brain does well. And so you can actually take a ground truth,
1:42:32 code for the metaverse, basically. And you can feed in the visual information, let’s say the
1:42:40 sensory information, or whatever that’s coming in, to a computational model that’s designed to take
1:42:47 real world inputs, right? And you could basically tie them all together by virtue of the state spaces
1:42:54 that you’re measuring in neural activity, and these different formats, and these different
1:42:58 species, and in the computational model, which is, I just find that mind blowing. You could do
1:43:04 different kinds of analyses on language, and basically come up with just like, basically,
1:43:09 it’s the guts of LLMs, right? You could do analyses on language, and you could do analyses on
1:43:18 sentiment analyses of emotions, and put all this stuff together. I mean, it’s almost too much.
1:43:25 But if you do it right, and you do it in a theory driven way, as opposed to just throwing all the
1:43:31 data at the wall and see what it sticks, I mean, that to me is just exceptionally powerful.
1:43:35 So you can take fMRI data across species, and across different types of humans,
1:43:42 of conditions of humans, and what find construct models that help you find the commonalities,
1:43:51 or the core thing that makes somebody navigate through the metaverse, for example.
1:43:56 Yeah. Yeah, I mean, more or less. I mean, there’s a lot of details, but yes, I think, and not just fMRI,
1:44:01 but you can relate it to, like I said, recordings from large populations of neurons that could
1:44:06 be taken in a human, or even in a non-human animal that is where you think it’s an anatomical homologue.
1:44:14 So that’s just mind-blowing to me. What’s the similarities in humans and mice?
1:44:20 That’s what it’s smashing pumpkins. We’re all just rats in a cage. Is that a smashing pumpkin?
1:44:28 Despite all of your rage at GIFs, you’re still just rat in a cage.
1:44:36 Oh, yeah. All right, good callback. Anyway, good callback. See, these memory retrieval
1:44:40 exercises that we’re doing are actually helping you build a lasting memory of this conversation.
1:44:46 And it’s strengthening the visual thing I have of you with James Brown on stage.
1:44:51 It’s just becoming stronger and stronger by the second.
1:44:54 It’s got a lot to it.
1:44:56 But animal studies work here as well.
1:45:00 Yeah, yeah. So let’s go to the… So I think I’ve got great colleagues who I talk to
1:45:07 who study memory in mice. One of the valuable things in those models is you can study
1:45:15 neural circuits in an enormously targeted way because you could do these
1:45:19 genetic studies, for instance, where you can manipulate particular groups of neurons.
1:45:25 And it’s just getting more and more targeted to the point where you can actually turn on
1:45:30 particular kind of memory just by activating a particular set of neurons that was active
1:45:36 during an experience. So there’s a lot of conservation of some of these neural circuits
1:45:43 across evolution in mammals, for instance. And then some people would even say that there’s
1:45:50 genetic mechanisms for learning that are conserved even going back far, far before.
1:45:55 But let’s go back to the mice in humans question, right?
1:45:58 There’s a lot of differences. So for one thing, the sensory information is very different.
1:46:04 Mice and rats explore the world largely through smelling, olfaction. But they also have vision
1:46:12 that’s kind of designed to kind of catch death from above. So it’s like a very big view of the
1:46:17 world. And we move our eyes around in a way that focuses on particular spots in space where you
1:46:23 get very high resolution from a very limited set of spots in space. So that makes us very different
1:46:29 in that way. We also have all these other structures as social animals that allow us to
1:46:35 respond differently. There’s language. There’s like, you know, so you name it, there’s obviously
1:46:42 gobs of differences. Humans aren’t just giant rats. There’s a bunch more complexity to us.
1:46:46 Time scales are very important. So primate brains and human brains are especially good
1:46:52 at integrating and holding on to information across longer and longer periods of time, right?
1:46:58 And also, you know, finally, it’s like our history of training data, so to speak,
1:47:04 is very, very different than, you know, I mean, humans’ world is very different than a wild
1:47:09 mouse’s world. And a lab mouse’s world is extraordinarily impoverished relative to an
1:47:15 adult human, you know? But still, what can you understand by studying mice? I mean,
1:47:19 just basic, almost behavioral stuff about memory? Well, yes, but that’s very important,
1:47:24 right? So you can understand, for instance, how do neurons talk to each other? That’s a really
1:47:30 big, big question. Neural computation, you think it’s the most simple question, right?
1:47:37 Not at all. I mean, it’s a big, big question. And understanding how two parts of the brain
1:47:44 interact, meaning that it’s not just one area speaking, it’s not like, you know, it’s not like
1:47:49 Twitter, where one area of the brain is shouting, and then another area of the brain is just stuck
1:47:53 listening to this crap. It’s like they’re actually interacting on the millisecond scale, right?
1:47:58 How does that happen? And how do you regulate those interactions, these dynamic, you know,
1:48:03 interactions? We’re still figuring that out, but that’s going to be coming largely from model
1:48:09 systems that are easier to understand. You can do manipulations like drug manipulations to manipulate
1:48:16 circuits and, you know, use viruses and so forth and lasers to turn on circuits that you just can’t
1:48:22 do in humans. So I think there’s a lot that can be learned from mice. There’s a lot that can be
1:48:27 learned from nonhuman primates. And there’s a lot that you need to learn from humans. And I think,
1:48:32 unfortunately, some of the people in the National Institutes of Health think you can learn everything
1:48:38 from the mouse. It’s like, why study memory in humans when I could study learning in a mouse?
1:48:43 And just like, oh my God, I’m going to get my funding from somewhere else.
1:48:46 Well, let me ask you some random, fascinating questions. How does deja vu work?
1:48:54 So deja vu is it’s actually one of these things I think that some of the
1:49:01 surveys suggest that like 75% of people report having a deja vu experience one time or another.
1:49:08 I don’t know where that came from, but I’ve pulled people in my class and most of them
1:49:12 say they’ve experienced deja vu. It’s this kind of sense that I’ve experienced this moment sometime
1:49:18 before I’ve been here before. And actually, there’s all sorts of variants of this, the French have
1:49:24 all sorts of names for various versions of the shammy vu, parley, I don’t know. All these different
1:49:31 vues. But deja vu is the sense that it can be like almost disturbing intense sense of familiarity.
1:49:40 So there is a researcher named Wilder Penfield. Actually, this goes back even earlier to some
1:49:46 of the earliest, like Hulings Jackson was this neurologist who did a lot of the early
1:49:52 characterizations of epilepsy. And one of the things he notices in epilepsy patients,
1:49:57 some group of them right before they would get a seizure, they would have this intense sense
1:50:02 of deja vu. So it’s this artificial sense of familiarity. It’s a sense of having a memory
1:50:09 that’s not there. And so what was happening was there was electrical activity in certain
1:50:16 parts of these brain cells. So this guy Penfield later on, when he was trying to look for how do
1:50:22 we map out the brain to figure out which parts we want to remove and which parts don’t we,
1:50:27 he would stimulate parts of the temporal lobes of the brain and find you could elicit the sense
1:50:31 of deja vu. Sometimes you’d actually get a memory that a person would re-experience just from
1:50:36 electrically stimulating some parts. Sometimes they just have this intense feeling of being
1:50:42 somewhere before. And so one theory which I really like is that in higher order areas of the brain
1:50:50 they’re integrating for many, many different sources of input. What happens is that they’re
1:50:56 tuning themselves up every time you process a similar input. And so that allows you to just
1:51:04 get this kind of a fluent sense that I’m very familiar. You’re very familiar with this place.
1:51:10 And so just being here, you’re not going to be moving your eyes all over the place because you
1:51:14 kind of have an idea of where everything is. And that fluency gives you a sense of like I’m here.
1:51:19 Now I wake up in my hotel room and I have this very unfamiliar sense of where I am.
1:51:24 But there’s a great set of studies done by Ann Cleary at Colorado State where she created
1:51:30 these virtual reality environments and we’ll go back to the metaverse. Imagine you go through a
1:51:36 virtual museum and then she would put people in virtual reality and have them go through a virtual
1:51:43 arcade. But the map of the two places was exactly the same. She just put different skins on them.
1:51:48 So one looks different than the other. But they’ve got same landmarks in the same places,
1:51:53 same objects and everything, but carpeting, colors, theme, everything’s different.
1:51:57 People will often not have any conscious idea that the two are the same, but they could report
1:52:04 this very intense sense of deja vu. So it’s like a partial match that’s eliciting this kind of a
1:52:10 sense of familiarity. And that’s why in patients who have epilepsy that affects memory, you get
1:52:17 this artificial sense of familiarity that happens. And again, this is just one theory amongst many,
1:52:25 but we get a little bit of that feeling it’s not enough to necessarily give you deja vu,
1:52:31 even for very mundane things. So it’s like if I tell you the word rutabaga, your brain’s going
1:52:40 to work a little bit harder to catch it than if I give you a word like apple. That’s because you
1:52:46 hear apple a lot. So your brain’s very tuned up to process it efficiently, but rutabaga takes
1:52:50 a little bit longer and more intense. And you can actually see a difference in brain activity
1:52:55 in areas in the temporal lobe when you hear a word just based on how frequent it is in
1:52:59 the English language. So we think it’s tied to this basic, it’s basically a byproduct of our
1:53:06 mechanism of just learning, doing this error driven learning as we go through life to become
1:53:12 better and better and better to process things more and more efficiently.
1:53:15 So I guess deja vu is just an extra elevated stuff coming together firing for this artificial
1:53:24 memories if it’s the real memory. I mean, why does it feel so intense?
1:53:29 Well, it doesn’t happen all the time, but I think what may be happening is it’s such a,
1:53:35 it’s a partial match to something that we have. And it’s not enough to trigger that sense of,
1:53:40 you know, that ability to pull together all the pieces, but it’s a close enough
1:53:44 match to give you that intense sense of familiarity without the recollection of exactly what happened
1:53:51 when. But it’s also like a spatiotemporal familiarity. So like, it’s also in time.
1:53:57 Like there’s a weird blending of time that happens. And we’ll probably talk about time
1:54:04 because I think that’s a really interesting idea how time relates to memory. But you also kind of
1:54:09 artificial memory brings to mind this idea of false memories that comes in all kinds of context.
1:54:17 But how do false memories form? Well, I like to say there’s no such thing as true or false
1:54:24 memories, right? It’s like Johnny Rotten from the Sex Pistols. He had a saying that’s like,
1:54:28 I don’t believe in false memories anymore than I believe in false songs, right? It’s like,
1:54:33 and so the basic idea is, is that we have these memories that reflect bits and pieces of what
1:54:39 happened as well as our inferences and theories, right? So I’m a scientist and I collect data,
1:54:45 but I use, I use theories to make sense of that data. And so a memory is kind of a mix of all
1:54:52 these things. So where memories can go off the deep end and become what we would call conventionally
1:54:57 as false memories are sometimes little distortions where we filled in the blanks, the gaps in our
1:55:05 memory based on things that we know, but don’t actually correspond to what happened, right?
1:55:10 So if I were to tell you that I’m like, you know, a story about this person who’s like
1:55:20 worried that they have cancer or something like that, and then, you know, they see a doctor and
1:55:25 the doctor says, well, things are very much like you would have expected or like, you know,
1:55:30 what you’re afraid of or something. When people remember that, they’ll often remember, well,
1:55:34 the doctor told the patient that he had cancer, even if that wasn’t in the story because they’re
1:55:40 infusing meaning into that story, right? So that’s a minor distortion. But what happens is,
1:55:45 is that sometimes things can really get out of hand where people have trouble telling the
1:55:51 difference in things that they’ve imagined versus things that happen. But also, as I told you,
1:55:56 the act of remembering can change the memory. And so what happens then is you can actually
1:56:02 be exposed to some misinformation. And so Elizabeth Loftus was a real pioneer in this work and there’s
1:56:08 lots of other work that’s been done since. But basically, it’s like if you remember some event,
1:56:15 and then I tell you something about the event, later on, when you remember the event, you might
1:56:21 remember some original information from the event, as well as some information about what I told you.
1:56:26 And sometimes, if you’re not able to tell the difference, that information that I told you
1:56:32 gets mixed into the story that you had originally. So now I give you some more misinformation or
1:56:38 you’re exposed to some more information somewhere else. And eventually, your memory becomes totally
1:56:43 detached from what happened. And so sometimes you can have cases where people, this is very rare,
1:56:50 but you can do it in the lab too, or like a significant, not everybody, but you know,
1:56:56 a chunk of people will fall for this, where you can give people misinformation about an event
1:57:02 that never took place. And as they keep trying to remember that event more and more, what happens,
1:57:08 they start to imagine, they start to pull up things from other experiences they’ve had.
1:57:13 And eventually, they can stitch together a vivid memory of something that never happened.
1:57:18 Because they’re not remembering an event that happened, they’re remembering the act of trying
1:57:24 to remember what happened, and basically putting it together into the wrong story.
1:57:29 So it’s fascinating because this could probably happen at a collective level.
1:57:36 Like this is probably what successful propaganda machines aim to do,
1:57:40 is creating false memory across thousands, if not millions of minds.
1:57:45 Yeah, absolutely. I mean, this is exactly what they do. And so all these kind of foibles of human
1:57:52 memory get magnified when you start to have social interactions. There’s a whole literature
1:57:56 on something called social contagion, which is basically when misinformation spreads like a
1:58:02 virus, like you remember the same thing that I did, but I give you a little bit of wrong
1:58:07 information, then that becomes part of your story of what happened. Because once you and I share a
1:58:13 memory, like I tell you about something I’ve experienced, and you tell me about your experience
1:58:17 of the same event, it’s no longer your memory or my memory, it’s our memory. And so now the
1:58:23 misinformation spreads. And the more you trust someone, or the more powerful that person is,
1:58:29 the more of a voice they have in shaping that narrative. And there’s all sorts of interesting
1:58:36 ways in which misinformation can happen. There’s a great example of when John McCain and George
1:58:43 Bush Jr. were in a primary, and there are these polls where they would do these, I guess they
1:58:51 were like not robocalls, but real calls where they would poll voters. But they actually inserted
1:58:56 some misinformation about McCain’s beliefs on taxation, I think, and maybe it was something
1:59:02 about illegitimate children or something. I don’t really remember. But they included misinformation
1:59:07 in the question that they asked, like, how do you feel about the fact that he wants to do this or
1:59:12 something? And so people would end up becoming convinced he had these policy things or these
1:59:18 personal things that were not true, just based on the polls that were being used. So it was a case
1:59:24 where, interestingly enough, the people who were using misinformation were actually ahead of the
1:59:32 curve relative to the scientists who were trying to study these effects in memory.
1:59:37 Yeah, it’s really interesting. So it’s not just about truth and falsehoods, like us as
1:59:45 intelligent reasoning machines, but it’s the formation of memories where they become
1:59:53 like visceral. You can rewrite history. If you just look throughout the 20th century,
1:59:58 some of the dictatorships with Nazi Germany, with the Soviet Union,
2:00:05 effective propaganda machines can rewrite our conceptions of history. How we remember our
2:00:11 own culture, our upbringing, all this kind of stuff. And you could do quite a lot of damage
2:00:15 in this way. And then there’s probably some kind of social contagion happening there.
2:00:19 Like certain ideas that may be initiated by the propaganda machine can spread faster than others.
2:00:28 You could see that in modern day, certain conspiracy theories, there’s just something
2:00:32 about them that they are really effective at spreading. There’s something sexy about them,
2:00:38 to people, to where something about the human mind eats it up and then uses that
2:00:44 to construct memories as if they almost were there to witness whatever the content of the
2:00:51 conspiracy theory is. It’s fascinating. Because once you feel like you remember a thing,
2:00:57 I feel like there’s a certainty. It emboldens you to say stuff. It’s not just you believe
2:01:06 an idea is true or not. It’s at the core of your being that you feel like you were there to watch
2:01:14 the thing happen. Yeah. I mean, there’s so much in what you’re saying. One of the things is that
2:01:21 people’s sense of collective identity is very much tied to shared memories. If we have a shared
2:01:27 narrative of the past, or even better, if we have a shared past, we will feel more socially connected
2:01:33 with each other. And I will feel part of this group. They’re part of my tribe, if I remember
2:01:37 the same things in the same way. And you brought up this weaponization of history. And it really
2:01:43 speaks to, I think, one of the parts of memory, which is that if you have a belief, you will find,
2:01:50 and you have a goal in mind, you will find stuff in memory that aligns with it. And you won’t see
2:01:56 the parts in memory that don’t. So a lot of the stories we put together are based on our perspectives.
2:02:01 Right? And so let’s just zoom out for the moment from misinformation. Take something even more
2:02:09 fascinating, but not as scary. I was reading Tan Viet Nguyen, but he wrote a book about the collective
2:02:18 memory of the Vietnam War. He’s a Vietnamese immigrant who was flown out after the war was
2:02:25 over. And so he went back to his family to get their stories about the war. And they called it
2:02:31 the American War, not the Vietnam War. And that just kind of blew my mind, having grown up in the
2:02:38 U.S., and I’ve always heard about it as a Vietnam War. But of course they call it the American War,
2:02:42 because that’s what happened. America came in. And that’s based on their perspective, which is a
2:02:48 very valid perspective. And so that just gives you this idea of the way we put together these
2:02:56 narratives based on our perspectives. And I think the opportunities that we can have in memory is
2:03:04 if we bring groups together from different perspectives and we allow them to talk to each
2:03:11 other and we allow ourselves to listen. I mean, right now you’ll hear a lot of just jammering,
2:03:16 you know, people going blah, blah, blah about free speech, but they just want to listen to
2:03:20 themselves, right? I mean, it’s like, let’s face it, the old days before people were supposedly
2:03:26 awoke, they were trying to ban too-live crew or, you know, just think about Letty Bruce got canceled
2:03:31 for cursing, Jesus Christ, you know? It’s like, this is nothing new. People don’t like to hear
2:03:38 things that disagree with them. But if you’re in it, I mean, you can see two situations in groups
2:03:47 with memory. One situation is you have people who are very dominant who just take over the
2:03:52 conversation. And basically what happens is the group remembers less from the experience,
2:03:57 and they remember more of what the dominant narrator says, right? Now if you have a diverse
2:04:02 group of people, and I don’t mean diverse in necessarily the human resources sense of the
2:04:07 word, I mean, diverse in any way you want to take it, right? But diverse in every way, hopefully.
2:04:12 And you give everyone a chance to speak, and everyone’s being appreciated for their unique
2:04:17 contribution. You get more accurate memories, and you get more information from it, right?
2:04:22 Even two people who come from very similar backgrounds, if you can appreciate the unique
2:04:27 contributions that each one has, you can do a better job of generating information from memory.
2:04:32 And that’s a way to inoculate ourselves, I believe, from misinformation in the modern world.
2:04:39 But like everything else, it requires a certain tolerance for discomfort. And I think
2:04:43 when we don’t have much time, and I think when we’re stressed out, and when we are
2:04:49 just tired, it’s very hard to tolerate discomfort. And I mean, social media has a lot of opportunity
2:04:56 for this because it enables this distributed one-on-one interaction that you’re talking about
2:05:02 where everybody has a voice. But still our natural inclination, you see this on social media,
2:05:08 there’s a natural clustering of people and opinions, and you just kind of form these kind
2:05:13 of bubbles. I think that’s, to me personally, I think that’s a technology problem that could be
2:05:18 solved. If there’s a little bit of interaction, kind, respectful, compassionate interaction
2:05:24 with people that have a very different memory, that respectful interaction will start to
2:05:31 intermix the memories and ways of thinking to where you’re slowly moving towards truth.
2:05:38 But that’s a technology problem because naturally left our own devices, we want to cluster up in
2:05:44 a tribe. Yeah, and that’s the human problem. I think a lot of the problems that come up with
2:05:51 technology aren’t the technology itself as much as the fact that people adapt to the technology
2:05:57 in maladaptive ways. I mean, one of my fears about AI is not what AI will do, but what people
2:06:05 will do. I mean, take text messaging. It’s like, it’s pain in the ass to text people, at least for
2:06:09 me. And so what happens is the communication becomes very spartan and devoid of meaning. It’s
2:06:15 just very telegraphic, and that’s people adapting to the medium. I mean, look at you. You’ve got this
2:06:21 keyboard that’s got these dome-shaped things, and you’ve adapted to that to communicate.
2:06:27 That’s not the technology adapting to you. It’s you adapting to the technology.
2:06:33 One of the things I learned when Google started to introduce autocomplete in emails,
2:06:37 I started to use it. And about a third of the time, I was like, this isn’t what I want to say.
2:06:42 A third of the time, I’d be like, this is exactly what I wanted to say. And a third of the time,
2:06:46 I was saying, well, this is good enough. I’ll just go with it. And so what happens is it’s not that
2:06:52 the technology necessarily is doing anything so bad as much as it’s just going to constrain my
2:06:59 language because I’m just doing what’s being suggested to me. And so this is why I say kind
2:07:06 of like my mantra for some of what I’ve learned about everything in memory is to diversify your
2:07:12 training data, basically, because otherwise you’re going to be– so humans have this capability to
2:07:18 be so much more creative than anything generative AI will put together, at least right now who knows
2:07:24 where this goes. But it can also go the opposite direction where people could become much, much
2:07:30 less creative if they just become more and more resistant to discomfort and resistant to
2:07:38 exposing themselves to novelty, to cognitive dissonance, and so forth.
2:07:43 I think there is a dance between natural human adaptation of technology and the people that
2:07:48 design the engineering of that technology. So I think there’s a lot of opportunity to create
2:07:54 like this keyboard, things that on net are a positive for human behavior. So we adapt and
2:08:01 all this kind of stuff, but when you look at the long arc of history across years and decades,
2:08:07 has humanity been flourishing? Are humans creating more awesome stuff? Are humans happy or all that
2:08:14 kind of stuff? And so there I think technology on net has been and I think maybe hope will always be
2:08:24 on net a positive thing. Do you think people are happier now than they were 50 years ago or 100
2:08:29 years ago? Yes. I don’t know about that. I think humans in general like to reminisce about the past,
2:08:38 like the times are better. That’s true. And complain about the weather today or complain
2:08:43 about whatever today because there’s this kind of complaining engine that just there’s so much
2:08:49 pleasure in saying life sucks for some reason. That’s why I love punk rock. Exactly. I mean,
2:08:57 there’s something in humans that loves complaining, even about trivial things,
2:09:03 but complaining about change, complaining about everything. But ultimately, I think on net,
2:09:09 on every measure, things are getting better. Life is getting better. Oh, life is getting better,
2:09:17 but I don’t know necessarily that tracks people’s happiness, right? I mean, I would argue that maybe,
2:09:22 who knows? I don’t know this, but I wouldn’t be surprised if people in hunter-gatherer societies
2:09:27 are happier. I mean, I wouldn’t be surprised if they’re happier than people who have access to
2:09:33 modern medicine and email and cell phones. Well, I don’t think there’s a question whether you
2:09:40 take hunter-gatherer folks and put them into modern day and give them enough time to adapt.
2:09:44 They would be much happier. The question is, in terms of every single problem they’ve had,
2:09:50 is not solved. There’s not food. There is guaranteed survival shelter and all this kind
2:09:55 of stuff. So, well, you’re asking is a deeper sort of biological question. Do we want to be,
2:10:00 we’re in a Herzog movie, a happy people, life in the taiga. Do we want to be busy 100% of our time
2:10:07 hunting, gathering, surviving, worried about the next day, maybe that constant struggle ultimately
2:10:16 creates a more fulfilling life? I don’t know, but I do know this modern society allows us
2:10:22 to, when we’re sick, to find medicine, to find cures. When we’re hungry to get food,
2:10:30 much more than we did even 100 years ago. And there’s many more activities that you could
2:10:38 perform or create of all these kinds of stuff that enables the flourishing of humans at the
2:10:43 individual level. Whether that leads to happiness, I mean, that’s a very deep philosophical question.
2:10:49 Maybe struggle, deep struggle is necessary for happiness.
2:10:55 Or maybe cultural connection. Maybe it’s about functioning in social groups that are meaningful
2:11:03 and having time. But I do think there is an interesting memory-related thing, which is that
2:11:08 if you look at things like reinforcement learning, for instance, you’re not learning necessarily
2:11:14 every time you get a reward. If it’s the same reward, you’re not learning that much. You mainly
2:11:21 learn if deviates from your expectation of what you’re supposed to get. It’s like you get a paycheck
2:11:26 every month from MIT or whatever. You probably don’t even get excited about it when you get the
2:11:34 paycheck. But if they cut your salary, you’re going to be pissed. And if they increase your salary,
2:11:38 oh, good, I got a bonus. And that adaptation and that ability that basically you learn to expect
2:11:48 these things, I think, is a major source of, I guess, it’s a major way in which we’re kind of
2:11:54 more, in my opinion, wired to strive and not be happy to be in a state of wanting. And so people
2:12:02 talk about dopamine, for instance, being this pleasure chemical. And it’s like there’s a lot
2:12:07 of compelling research to suggest it’s not about pleasure at all. It’s about the discomfort that
2:12:14 energizes you to get things, to seek a reward. And so you could give an animal that’s been deprived
2:12:21 of dopamine a reward and, oh, yeah, enjoy it. It’s pretty good. But they’re not going to do anything
2:12:28 to get it. And just one of the weird things in our research is I got into curiosity from a postdoc
2:12:36 in my lab, Matthias Gruber. And one of the things that we found is when we gave people a question,
2:12:42 like a trivia question that they wanted the answer to, that question, the more curious people were
2:12:49 about the answer, the more activity in these dopamine-related circuits in the brain we would see.
2:12:54 And again, that was not driven by the answer, per se, but by the question. So it was not about
2:13:01 getting the information. It was about the drive to seek the information. But it depends on how
2:13:08 you take that. If you get this uncomfortable gap between what you know and what you want to know,
2:13:13 you could either use that to motivate you and energize you, or you could use it to say,
2:13:18 I don’t want to hear about this. This disagrees with my beliefs. I’m going to go back to my echo
2:13:22 chamber. I like what you said that maybe were designed to be in a constant state of wanting,
2:13:32 which, by the way, is a pretty good either band name or rock song name, state of wanting.
2:13:40 That’s like a hardcore band name. Yeah, yeah, yeah. It’s pretty good.
2:13:43 I also like the hedonic treadmill. Hedonic treadmill is pretty good.
2:13:47 Yeah, yeah. We could use that for our techno project, I think.
2:13:51 You mean the one we’re starting? Yeah, exactly.
2:13:53 Okay, great. We’re going on tour soon. This is our announcement.
2:14:02 We could build a false memory of a show, in fact, if you want. Let’s just put it all together.
2:14:06 We don’t even have to do all the work to play the show. We can just create a memory of it.
2:14:10 It might as well happen because the remembering self is in charge anyway.
2:14:14 So let me ask you about, we talked about false memories, but in the legal system,
2:14:19 false confessions, I remember reading 1984 where, sorry for the dark turn of our conversation, but
2:14:28 through torture, you can make people say anything and essentially remember anything.
2:14:34 I wonder towards degree, there’s like truth to that. If you look at
2:14:38 the torture that happened in the Soviet Union, for confessions, all that kind of stuff,
2:14:42 how much can you really get people to really, yeah, to force false memories, I guess.
2:14:50 Yeah, I mean, I think there’s a lot of history of this actually in the criminal justice system.
2:14:58 You might have heard the term the third degree. If you actually look it up, historically,
2:15:04 it was a very intense set of beatings and starvation and physical demands that they would
2:15:11 place at people to get them to talk. There’s certainly a lot of work that’s been done by the
2:15:18 CIA in terms of interrogation techniques. And from what I understand, the research actually
2:15:26 shows that they just produce what people want to hear, not necessarily the information that
2:15:33 is being looked for. And the reason is that, I mean, there’s different reasons. I mean,
2:15:38 one is people just get tired of being tortured and just say whatever. But another part of it is that
2:15:44 you create a very interesting set of conditions where there’s an authority figure telling you
2:15:50 something that you did this, we know you did this, we have witnesses saying you did this.
2:15:54 So now you start to question yourself. Then they put you under stress. Maybe they’re not
2:16:00 feeding you. Maybe they’re kind of like making you be cold or exposing you to music that you
2:16:07 can’t stand or something, whatever it is, right? It’s like they’re creating this physical stress.
2:16:12 And so stress starts to act on, starts to down-regulate the prefrontal cortex. You’re not
2:16:19 necessarily as good at monitoring the accuracy of stuff. Then they start to get nice to you and
2:16:24 they say, imagine, okay, I know you don’t remember this, but maybe we can walk you through how it
2:16:29 could have happened and they feed you the information. And so you’re in this weakened mental
2:16:34 state and you’re being encouraged to imagine things by people who give you a plausible scenario.
2:16:40 And at some point, certain people can be very coaxed into creating a memory for something that
2:16:46 never happened. And there’s actually some pretty convincing cases out there where you don’t know
2:16:51 exactly the truth. There’s a sheriff, for instance, who came to believe that he had a false memory,
2:16:59 I mean, that he had a memory of doing sexual abuse based on, you know, essentially, I think it was,
2:17:04 you know, I’m not going to tell the story because I don’t remember it well enough to
2:17:10 necessarily accurately give it to you. But people could look this stuff up. There are definitely
2:17:14 stories out there like this where people confess to crimes that they just didn’t do when objective
2:17:19 evidence came out later on. But there’s a basic recipe for it, which is you feed people the
2:17:26 information that you want them to remember. You stress them out, you have an authority figure,
2:17:33 kind of like pushing this information on them, or you motivate them to produce the information
2:17:39 you’re looking for. And that pretty much over time gives you what you want.
2:17:44 It’s really tragic that centralized power can use these kinds of tools to destroy lives, sad.
2:17:55 Since there’s a theme about music throughout this conversation,
2:18:03 one of the best topics for songs is heartbreak, love in general, but heartbreak.
2:18:11 Why and how do we remember and forget heartbreak, asking for a friend?
2:18:15 Oh, God, that’s so hard to asking for a friend of that.
2:18:19 It’s such a hard one. Well, so, I mean, part of this is we tend to go back to particular
2:18:30 times that are the more emotionally intense periods. And so that’s a part of it. And again,
2:18:39 memory is designed to kind of capture these things that are biologically significant. And
2:18:44 attachment is a big part of biological significance for humans, right? Human relationships
2:18:49 are super important. And sometimes that heartbreak comes with massive changes in your beliefs about
2:18:56 somebody, say if they cheated on you or something like that, or regrets, and you kind of ruminate
2:19:02 about things that you’ve done wrong. There’s really so many reasons though, but I mean,
2:19:09 I’ve had this, my first pet I had was, we got it for a wedding present as a cat and got it after,
2:19:19 like, but it died of FIP when it was four years old. And I just would see her everywhere around
2:19:27 the house. We got another cat that we got a dog, dog eventually died of cancer, and the cat just
2:19:33 died recently. And, you know, so we got a new dog because I kept seeing the dog around and I was
2:19:40 just so heartbroken about this. But I still remember the pets that died, it just comes back to you.
2:19:47 I mean, it’s part of this, I think there’s also something about attachment that’s just so crucial
2:19:52 that drives, again, these things that we want to remember and that gives us that longing sometimes.
2:19:59 Sometimes it’s also not just about the heartbreak, but about the positive aspects of it, right?
2:20:05 Because the loss comes from not only the fact that the relationship is over, but you had all
2:20:11 of these good things before that you can now see in a new light, right? And so part of one of the
2:20:18 things that I found from my clinical background that really I think gave me a different perspective
2:20:23 on memory is so much of the therapy process was guided towards reframing and getting people to
2:20:31 look at the past in a different way, not by imposing, changing people’s memories or not
2:20:36 by imposing an interpretation, but just offering a different perspective and maybe one that’s kind
2:20:42 of more optimized towards learning and, you know, an appreciation maybe or gratitude, whatever it is,
2:20:49 right? That gives you a way of taking, I think you said it in the beginning, right? Where you
2:20:54 can have this kind of like dark experiences and you can use it as training data to, you know,
2:21:02 grow in new ways. But it’s hard. This, I often go back to this moment, this show, Louis,
2:21:09 with Louis C.K., where he’s all heartbroken about a breakup with a woman he loves and
2:21:17 an older gentleman tells him that that’s actually the best part, that heartbreak,
2:21:23 because you get to intensely experience how valuable this love was. He says the worst part
2:21:30 is forgetting it is actually when you get over the heartbreak. That’s the worst part. So I sometimes
2:21:37 think about that because, you know, having the love and losing it, like the losing it is when you
2:21:47 sometimes feel it the deepest, which is an interesting way to celebrate the past and relive it.
2:21:54 It sucks that you don’t have a thing, but when you don’t have a thing, it’s a
2:21:59 good moment to viscerally experience the memories of something that you now appreciate even more.
2:22:07 So you don’t believe that an owner of a lonely heart is much better than an owner of a broken heart?
2:22:14 You think an owner of a broken heart is better than the owner of a lonely heart?
2:22:17 Yes, for sure. I think so. I think so. But I’m gonna have to, day by day,
2:22:21 I don’t know, I’m gonna have to listen to some more Bruce Springsteen to figure that one out.
2:22:26 Well, you know, it’s funny because it’s like after I turned 50, I think of death all the time.
2:22:31 Like, I just think that, you know, I’m in like, I have fewer, probably a fewer years ahead of me
2:22:38 than I have behind me, right? So I think about one thing, which is what are the memories that
2:22:43 I want to carry with me for the next period of time? And also about, like, just the fact that
2:22:49 everything around me could be, you know, I know more people who are, you know, dying for various
2:22:55 reasons. And so, I’m not lots, I’m not that old, right? But, you know, it’s something I think about
2:23:03 a lot. And I’m reminded of, like, how I talked to somebody who’s like, you know, who’s a Buddhist.
2:23:10 And I was like, you know, the whole idea of Buddhism is renouncing attachments.
2:23:14 Someway, the idea of Buddhism is like staying out of the world of memory and staying in the moment,
2:23:20 right? And they talked about, you know, it’s like, how do you, how do you renounce attachments to the
2:23:26 people that you love, right? And they’re just saying, well, I appreciate that I have this moment
2:23:30 with them. And knowing that they will die makes me appreciate this moment that much more. I mean,
2:23:35 you said something similar, right? And your daily routine that you think about things this way,
2:23:40 right? Yeah, I meditate on mortality every day. But I don’t know, at the same time,
2:23:48 that really makes you appreciate the moment and live in the moment. And I also appreciate the full
2:23:54 deep roller coaster of suffering involved in life, the little and the big two. So, I don’t know.
2:24:01 The Buddhist kind of removing yourself from the world or the stoic removing yourself from the
2:24:07 world, the world of emotion. I’m torn about that one. I’m not sure.
2:24:11 Well, you know, this is where Hinduism and Buddhism or at least some strains of Hinduism
2:24:16 and Buddhism differ. And Hinduism, like if you read the Bhagavad Gita, the philosophy is not one
2:24:23 of renouncing the world because the idea is that not doing something is no different than doing
2:24:31 something, right? So, what they argue, and again, you could interpret it in different ways, positive
2:24:36 and negative. But the argument is that you don’t want to renounce action, but you want to renounce
2:24:43 the fruits of the action. You don’t do it because of the outcome, you do it because of the process.
2:24:49 Because the process is part of the balance of the world that you’re trying to preserve, right?
2:24:54 And of course, you could take that different ways. But I really think about that from time to time
2:24:59 in terms of letting go of this idea of does this book sell or trying to impress you and get you
2:25:09 laugh at my jokes or whatever, and just be more like I’m sharing this information with you and
2:25:14 getting to know you or whatever it is. But it’s hard, right? Because we’re so driven by the
2:25:21 reinforcer of the outcome. You’re just part of the process of telling the joke and if I laugh or not,
2:25:28 that’s up to the universe to decide. Yep, it’s my Dharma.
2:25:32 How does studying memory affect your understanding of the nature of time? So, we’ve been talking
2:25:41 about us living in the present and making decisions about the future, standing on the
2:25:49 foundation of these memories and there it is about the memories that we’ve constructed.
2:25:53 So, it feels like it does weird things to time. Yeah, and the reason is, is that in some sense,
2:26:01 I think, especially the farther we go back, I mean, there’s all sorts of interesting things
2:26:06 that happen. So, your sense of like, if I ask you how different does one hour ago feel from two hours
2:26:14 ago, you’d probably say pretty different. But if I ask you, okay, go back one year ago versus one
2:26:20 year and one hour ago, it’s the same difference in time. It won’t feel very different, right? So,
2:26:24 there’s this kind of compression that happens as you look back farther in time. So, it’s kind of
2:26:30 like why when you’re older, the difference between somebody who’s like 50 and 45 doesn’t seem as big
2:26:37 as the difference between like 10 and 5 or something, right? When you’re 10 years old,
2:26:41 everything seems like it’s a long period of time. Here’s the point is that, you know, so one of the
2:26:46 interesting things that I found when I was working on the book, actually, was during the pandemic,
2:26:50 I just decided to ask people in my class when we were doing the remote instruction. So,
2:26:55 one of the things I did was I’d pull people. And so, I just asked people, do you feel like the
2:27:00 days are moving by slower or faster or about the same? Almost everyone in the class said that
2:27:08 the days were moving by slower. So, then I would say, okay, so, do you feel like the weeks are
2:27:14 passing by slower, faster or the same? And the majority of them said that the weeks were passing
2:27:20 by faster. So, according to the laws of physics, I don’t think that makes any sense, right?
2:27:25 But according to memory, it did because what happened was people were doing the same thing
2:27:31 over and over in the same context. And without that change in context, their feeling was that
2:27:39 they were in one long monotonous event. And so, but then, at the end of the week, you look back
2:27:46 at that week and you say, well, what happened? No memories of what happened. So, it must,
2:27:51 the week just went by without even my noticing it. But that week went by during the same amount
2:27:57 of time as an eventful week where you might have been going out and hanging out with friends on
2:28:01 vacation or whatever, right? It’s just that nothing happened because you’re doing the
2:28:06 same thing over and over. So, I feel like memory really shapes our sense of time. But it does so
2:28:12 in part because context is so important for memory. Well, that compression you mentioned,
2:28:18 it’s an interesting process. Because what I think about when I was like 12 or 15,
2:28:26 I just fundamentally feel like the same person. It’s interesting what that compression does.
2:28:32 It makes me feel like it’s all, we’re all connected, not just amongst humans and spatially, but
2:28:37 in terms, back in time, there’s a kind of eternal nature, like the timelessness, I guess,
2:28:45 to life. That could be also a genetic thing just for me. I don’t know if everyone agrees
2:28:52 to this view of time, but to me, it all feels the same. Like you don’t feel the passage of time?
2:28:57 No, I feel the passage of time in the same way that your students did from day to day.
2:29:02 There’s certain markers that let you know that time has passed, you celebrate birthdays and so on.
2:29:10 But the core of who I am and who others I know are or events, that compression of my understanding
2:29:17 of the world removes time, because time is not useful for the compression. The details of that
2:29:24 time, at least for me, is not useful to understanding the core of the thing. Maybe what it is is that
2:29:31 you really like to see connections between things. This is really what motivates me in science,
2:29:37 actually, too. But it’s like when you start recalling the past and seeing the connections
2:29:43 between the past and present, now you have this kind of web of interconnected memories.
2:29:49 I can imagine, in that sense, there is this kind of the present is with you.
2:29:55 But what’s interesting about what you said, too, that struck me is that your 16-year-old self was
2:30:03 probably very complex. And by the way, I’m the same way, but it’s like it really is the source of a
2:30:09 lot of darkness for me. But when you can look back at, let’s say you hear a song that you used
2:30:18 to play before you would go do a sports thing or something like that, and you might not think of
2:30:23 yourself as an athlete, but once you get back to that, you mentally time travel to that particular
2:30:28 thing, you open up this little compartment of yourself that wasn’t there before, right, that
2:30:33 didn’t seem accessible for them. Dan Schachter’s lab did this really cool study where they would
2:30:39 ask people to either remember doing something altruistic or imagine doing something altruistic.
2:30:47 And that act made them more likely to want to do things for other people. So that
2:30:56 active mental time travel can change who you are in the present. And we tend to think of,
2:31:01 this goes back to that illusion of stability, and we tend to think of
2:31:05 memory in this very deterministic way that I am who I am because I have this past. But we have a
2:31:10 very multifaceted past and can access different parts of it and change in the moment based on
2:31:18 whatever part we want to reach for, right? How does nostalgia connect into this? Like this
2:31:25 desire and pleasure associated with going back? Yeah. So my friend Felipe de Bregard
2:31:34 wrote this and it just like blew my mind where the word nostalgia was coined by a Swiss physician
2:31:41 who was actually studying traumatized soldiers. And so he described nostalgia as a disease.
2:31:46 And the idea was it was bringing these people extraordinary unhappiness because they were
2:31:51 remembering how things used to be. And I think it’s very complex. So as people get older,
2:31:59 for instance, nostalgia can be an enormous source of happiness, right? And being nostalgic can
2:32:06 improve people’s moods in the moment. But it just depends on what they do with it because what you
2:32:12 can sometimes see is nostalgia has the opposite effect of thinking those were the good old days
2:32:17 and those days are over, right? It’s like America used to be so great and now it sucks. Or you know,
2:32:23 my life used to be so great when I was a kid and now it’s not, right? And you’re selectively
2:32:29 remembering the things that we don’t realize how selective our remembering self is. And so,
2:32:35 you know, I lived through the 70s, it sucked. Partly it sucked more for me, but I would say that
2:32:43 even otherwise, it’s like there’s all sorts of problems going on. Gas lines, people were like,
2:32:48 you know, worried about like Russia, nuclear war, blah, blah, blah. So I mean, it’s just this idea
2:32:55 that people have about the past can be very useful if it brings you happiness in the present. But
2:33:03 if it narrows your worldview in the present, you’re not aware of those biases that you have.
2:33:09 You will end up, you can end up, it can be toxic, right? Either at a personal level
2:33:14 or at a collective level. Let me ask you both a practical question and an out there question.
2:33:20 So let’s start with a more practical one. What are your thoughts about BCIs, brain computer
2:33:27 interfaces and the work that’s going on with Neuralink? We talked about electrodes and different
2:33:32 ways of measuring the brain. And here, Neuralink is working on basically two-way communication
2:33:37 with the brain. And the more out there question would be like, where does this go? But more
2:33:40 practically in the near term, what do you think about Neuralink? Yeah, I mean, I can’t say specifics
2:33:46 about the company because I haven’t studied it that much. But I mean, I think there’s two parts of
2:33:51 it. So one is they’re developing some really interesting technology. I think with these like
2:33:55 surgical robots and things like that. BCI though has like a whole lot of innovation going on.
2:34:03 I’m not necessarily seeing any scientific evidence from Neuralink. And maybe that’s just
2:34:09 I’m not looking for it, but I’m not seeing the evidence that they’re anywhere near where the
2:34:14 scientific community is. And there’s lots of startups that are doing incredibly innovative
2:34:18 stuff. One of my colleagues, Sergei Siviski, is just like a genius in this area. And they’re working
2:34:23 on it. I think speech prosthetics like that are incorporating, you know, decoding techniques with
2:34:29 AI and movement prosthetics. The rate of progress is just enormous. So part of the technology is
2:34:37 having good enough data and understanding which data to use and what to do with it, right? And then
2:34:44 the other part of it then is the algorithms for decoding it and so forth. And I think part of
2:34:49 that has really resulted in some real breakthroughs in neuroscience as a result. So there’s lots of
2:34:56 new technologies like NeuroPixels, for instance, that allow you to harvest
2:35:00 activity from many, many neurons from a single electrode. I know Neuralink has some technologies
2:35:06 that are also along these lines. But I even, again, because they do their own stuff, the scientific
2:35:12 community doesn’t see it, right? But I think BCI is much, much bigger than Neuralink. And there’s
2:35:19 just so much innovation happening. I think the interesting question, which we may be getting
2:35:25 into, is I was talking to Sergey a while ago about, you know, so a lot of language is not
2:35:30 just what we hear and what we speak, but also our intentions and our internal models. And,
2:35:37 you know, so are you really going to be able to restore language without dealing with that
2:35:41 part of it? And he brought up a really interesting question, which is the ethics of reading out
2:35:47 people’s intentions and understanding of the world as opposed to the more, you know, the more
2:35:54 concrete parts of hearing and producing movements, right? Just so we’re clear, because you said a
2:36:00 few interesting things. When you say, when we talk about language and BCI is what we mean is
2:36:04 getting signal from the brain and generating the language, say you’re not able to actually speak,
2:36:12 it’s as a kind of linguistic prosthetic, or it’s able to speak for you
2:36:18 exactly what you wanted to say. And then the deeper question is, well,
2:36:23 saying something isn’t just the letters, the words you’re saying, it’s also the intention
2:36:30 behind it, the feeling behind all that kind of stuff. And is it ethical to reveal that full
2:36:36 shebang, the full context of what’s going on in our brain? That’s really, that’s really interesting.
2:36:43 That’s really interesting. I mean, our thoughts. Is it ethical for anyone to have access to our
2:36:49 thoughts? Because right now the resolution is so low that we’re okay with it, even doing studies
2:36:57 and all this kind of stuff. But if neuroscience has a few breakthroughs to where you can start
2:37:02 to map out the QR codes for different thoughts, for different kinds of thoughts,
2:37:06 maybe political thoughts, the McCarthyism, what if I’m getting a lot of them communist thoughts,
2:37:14 or however we want to categorize or label it. That’s interesting. That’s really interesting.
2:37:22 I think ultimately there’s always the more transparency there is about the human mind,
2:37:30 the better it is. But there could be always intermediate battles with how much control
2:37:38 does a centralized entity have, like a government and so on. What is the regulation? What are the
2:37:42 rules? What’s legal and illegal? If you talk about the police whose job is to
2:37:48 track down criminals and so on, and you look at all the history, how the police could abuse its
2:37:55 power to control the citizenry, all that kind of stuff. So people are always paranoid and rightfully
2:38:01 so. It’s fascinating. It’s really fascinating. We talk about freedom of speech, freedom of thought,
2:38:09 which is also a very important liberty at the core of this country and probably humanity
2:38:17 starts to get awfully tricky when you start to be able to collect those thoughts.
2:38:23 But what I wanted to actually ask you is, do you think for fun and for practical purposes
2:38:31 we would be able to modify memories? So how difficult is it? How far away we are from
2:38:43 understanding the different parts of the brains, everything we’ve been talking about,
2:38:48 in order to figure out how could we adjust this memory at the crude level from
2:38:52 unpleasant to pleasant? You talked about we can remember the mall and the people,
2:38:58 like the location of the people. Can we keep the people and change the place,
2:39:02 like this kind of stuff? How difficult is that? Well, I mean, in some sense, we know we can do it
2:39:08 just behaviorally, right? Behaviorally, yes. I can just tell you, under certain conditions,
2:39:13 anyway, it can give you the misinformation and then you can change the people and places and so
2:39:18 forth, right? On the crude level, there’s a lot of work that’s being done on a phenomenon called
2:39:24 reconciliation, which is the idea that essentially when I recall a memory, what happens is that the
2:39:31 connections between the neurons and that cell assembly that give you the memory are going to be
2:39:38 more modifiable. And so some people have used techniques to try to, like, for instance, with
2:39:44 fear memories to reduce that physical visceral component of the memory when it’s being activated.
2:39:51 Right now, I think I’ve, as an outsider looking at the data, I think it’s like mixed results.
2:39:56 And part of it is, and this speaks to the more complex issue, is that you don’t need somebody
2:40:04 to actually fully recall that traumatic memory in the first place. And in order to actually
2:40:11 modify it, then what is the memory? That is the key part of the problem. So if we go back to
2:40:17 reading people’s thoughts, what is the thought? I mean, people can sometimes look at this like
2:40:22 behaviorist and go, well, the memory is like, I’ve given you A and you produce B. But I think that’s
2:40:27 a very bankrupt concept about memory. I think it’s much more complicated than that. And, you know,
2:40:33 one of the things that when we started studying naturalistic memory, like memory from movies,
2:40:37 that was so hard was we had to change the way we did the studies. Because if I show you a movie,
2:40:44 and I show, and I watch the same movie, and you recall everything that happened, and I recall
2:40:50 everything that happened, we might take a different amount of time to do it, we might use different
2:40:55 words. And yet to an outside observer, we might have recalled the same thing, right? So it’s not
2:41:00 about the words necessarily. And it’s not about how long we spent or whatever, there’s something
2:41:06 deeper that is there. That’s this idea. But it’s like, how do you understand that thought? I encounter
2:41:13 a lot of concrete thinking that it’s like, if I show a model, like, you know, the visual
2:41:20 information that a person sees when they drive, I can basically reverse engineer driving. Well,
2:41:26 that’s not really how it works. I once saw a talk by somebody, or I saw somebody talking in this
2:41:32 discussion of between neuroscientists and AI people. And he was saying that the problem with
2:41:38 self-driving cars that they had in cities, as opposed to highways, was that the car was okay at,
2:41:44 you know, doing the things it’s supposed to. But when there are pedestrians around, it couldn’t
2:41:49 predict the intentions of people. And so that unpredictability of people was the problem that
2:41:56 they were having in, you know, the self-driving car design, because it didn’t have a good enough
2:42:01 internal model of what the people were, you know, what they were doing, what they wanted.
2:42:07 Now, what do you think about that? Well, I spent a huge amount of time
2:42:11 watching pedestrians, thinking about pedestrians, thinking about what it takes to solve the problem of
2:42:19 measuring, detecting the intention of a pedestrian, really of a human being in this particular context
2:42:30 of having to cross the street. And it’s fascinating. I think it’s a window into
2:42:40 how complex social systems are that involve humans. Because, you know, I would just stand there and
2:42:49 watch intersections for hours. And when you start to figure out is every single intersection has
2:42:55 its own personality. So like, there’s a history to that intersection, like jaywalking certain
2:43:03 intersections allow jaywalking a lot more. Because what happens is, we’re leaders and followers.
2:43:11 So there’s a regular, let’s say, and they get off the subway and they start crossing on red light,
2:43:16 and they do this every single day. And then there’s people that don’t show up to the intersection
2:43:20 often, and they’re looking for cues of how we’re supposed to behave here. And if a few people start
2:43:25 to jaywalk and cross on red light, they will also, they will follow. And there’s just a dynamic to
2:43:32 that intersection. There’s a spirit to it. And if you look at Boston versus New York,
2:43:37 versus a rural town versus even Boston, San Francisco or here in Austin, there’s different
2:43:43 personalities citywide, but there’s different personalities area at region wide. And there’s
2:43:48 different personalities, different intersections. And it’s just fascinating for a car to be able
2:43:53 to determine that is tricky. Now, what machine learning systems are able to do well is collect
2:43:59 a huge amount of data. So for us, it’s tricky because we get to understand the world with very
2:44:06 limited information and make decisions grounded in this big foundation model that we’ve built of
2:44:13 understanding how humans work. AI could literally, in the context of driving, this is where I’ve
2:44:20 often been really torn in both directions. If you just collect a huge amount of data,
2:44:26 all of that information and then compress it into a representation of how humans cross streets,
2:44:32 it’s probably all there. In the same way that you have a Noam Chomsky who says,
2:44:38 no, no, no, AI can’t talk, can’t write length convincing language without understanding
2:44:44 language. And, you know, more and more, you see large language models without quote unquote
2:44:49 understanding can generate very convincing language. But I think with the process of
2:44:54 compression from a huge amount of data compressing into a representation is doing is in fact,
2:45:00 understanding deeply in order to be able to generate one letter at a time, one word at a time,
2:45:07 you have to understand the cruelty of Nazi Germany and the beauty of sending humans to space.
2:45:18 And like, you have to understand all of that in order to generate like, I’m going to the kitchen
2:45:22 to get an apple and do that graphically correctly. You have to have a world model that includes all
2:45:27 of human behavior. You think an LLM is building that world model? It has to in order to be good
2:45:33 at generating one word at a time and convincing sentence. And in the same way, I think AI that
2:45:40 drives a car, if it has enough data, will be able to form a world model that will be able to predict
2:45:48 correctly what that pedestrian does. But when we as humans are watching pedestrians, we slowly
2:45:54 realize, damn, this is really complicated. In fact, when you start to self reflect on driving,
2:46:00 you realize driving is really complicated. There’s like subtle cues we take about like,
2:46:05 just a million things I could say, but like one of them determining who around you is an asshole,
2:46:13 aggressive driver. Yeah, I was just thinking about this. Yeah, or like, you can read it. Once you
2:46:19 get become a great driver, you can see it a mile away. This guy is going to pull an asshole move
2:46:25 in front of you. Exactly. He’s like way back there, but you know it’s going to happen. And
2:46:29 I don’t know what, because we’re ignoring all the other cars. But for some reason, the asshole,
2:46:34 like a red, like, like a glowing, obvious symbol is just like right there, even in the periphery
2:46:40 vision, because we’re again, we’re usually when we’re driving just looking forward, but we’re like
2:46:45 using the periphery vision to figure stuff out. And it’s like a little puzzle that we’re usually
2:46:51 only allocating a small amount of our attention to, at least a cognitive attention to. And it’s
2:46:56 fascinating, but I think AI just has a fundamentally different suite of sensors in terms of the bandwidth
2:47:03 of data that’s coming in that allows you to form the representation that perform inference on the
2:47:09 representation you form using the representation you form. That for the case of driving, I think it
2:47:15 could be quite effective. But one of the things that’s currently missing, even though OpenAI just
2:47:24 recently announced adding memory. And I did want to ask you, like, how important it is,
2:47:30 how difficult is it to add some of the memory mechanisms that you’ve seen in humans to AI systems?
2:47:38 I would say superficially not that hard, but then in a deeper level, very, very hard, because we
2:47:44 don’t understand episodic memory. So one of the ideas I talked about in the book is one of the oldest
2:47:50 kind of dilemmas in computational neuroscience is what Steve Grossberg called the stability
2:47:56 plasticity dilemma. When do you say something is new and overwrite your preexisting knowledge
2:48:03 versus going with what you had before and making incremental changes? And so, you know, part of
2:48:09 the problem with going through like massive, you know, I mean, part of the problem of things like
2:48:16 if you’re trying to design an LLM or something like that is especially for English, there’s so many
2:48:20 exceptions to the rules, right? And so if you want to rapidly learn the exceptions, you’re going to
2:48:26 lose the rules. And if you want to keep the rules, you have a harder time learning the exception.
2:48:32 And so David Maro is one of the early pioneers in computational neuroscience. And then
2:48:38 Jay McClelland and my colleague, Randy O’Reilly, some other people like Neil Cohen, all these people
2:48:45 started to come up with the idea that maybe that’s part of what we need and what the human brain
2:48:51 is doing is we have this kind of a, actually a fairly dumb system, which just says this happened
2:48:57 once at this point in time, which we call episodic memory, so to speak. And then we have this knowledge
2:49:03 that we’ve accumulated from our experiences as semantic memory. So now, when we want to,
2:49:10 we encounter a situation that’s surprising and violates all our previous expectations,
2:49:16 what happens is that now we can form an episodic memory here. And the next time we’re in a similar
2:49:21 situation, boom, we could supplement our knowledge with this information from episodic memory and
2:49:27 reason about what the right thing to do is, right? So it gives us this enormous amount of flexibility
2:49:33 to stop on a dime and change without having to erase everything we’ve already learned.
2:49:39 And that solution is incredibly powerful because it gives you the ability to learn from so much
2:49:47 less information really, right? And it gives you that flexibility. So one of the things I think
2:49:53 that makes humans great is having both episodic and semantic memory. Now, can you build something
2:50:01 like that? I mean, computational neuroscience people would say, well, yeah, you just record
2:50:06 a moment and you just get it and you’re done, right? But when do you record that moment? How
2:50:11 much do you record? What’s the information you prioritize and what’s the information you don’t?
2:50:15 These are the hard questions. When do you use episodic memory? When do you just throw it away?
2:50:21 And these are the hard questions we’re still trying to figure out in people.
2:50:24 And then you start to think about all these mechanisms that we have in the brain for figuring
2:50:29 out some of these things. And it’s not just one, but many of them that are interacting with each
2:50:34 other. And then you just take not only the episodic and the semantic, but then you start to take the
2:50:39 motivational survival things, right? It’s just like the fight or flight responses that we associate
2:50:45 with particular things are the kind of reward motivation that we associate with certain things
2:50:51 so forth. And those things are absent from AI. I frankly don’t know if we want it. I don’t
2:50:56 necessarily want a self-motivated LLM, right? And then there’s the problem of how do you even
2:51:05 build the motivations that should guide a proper reinforcement learning kind of thing,
2:51:10 for instance. So a friend of mine, Sam Gershman, I might be missing the quote exactly,
2:51:17 but he basically said, “If I wanted to train a typical AI model to make me as much money as
2:51:23 possible, the first thing I might do is sell my house.” So it’s not even just about having one
2:51:30 goal or one objective, but just having all these competing goals and objectives, right? And then
2:51:35 things start to get really complicated. It’s all interconnected. I mean, just even the thing you’ve
2:51:40 mentioned is the moment. If we record a moment, it’s difficult to express concretely what a moment is,
2:51:51 like how deeply connected it is to the entirety of it. Maybe to record a moment, you have to
2:51:58 make a universal scratch. You have to include everything. You have to include all the emotions
2:52:05 involved, all the context, all the things that built around it, all the social connections,
2:52:09 all the visual experiences, all the sensory experience, all of that, all the history that
2:52:15 came before that moment is built on. And we somehow take all of that and we compress it
2:52:21 and keep the useful parts and then integrate it into the whole thing, into our whole narrative.
2:52:27 And then each individual has their own little version of that narrative. And then we collide
2:52:32 in the social way and we adjust it and we evolve. Yeah. Yeah. I mean, well, even if we want to go
2:52:38 super simple, right? Like Tyler Ronan, who’s a postdoc who’s collaborating with me, he actually
2:52:45 studied a lot of computer vision at Stanford. And so one of the things he was interested in
2:52:51 is some people who have brain damage in areas of the brain that were thought to be important for
2:52:55 memory. But they also seem to have some perception problems with particular kinds of object
2:53:01 perception. And this is super controversial. And some people found this effect, some didn’t.
2:53:06 And he went back to computer vision and he said, let’s take the best state-of-the-art computer
2:53:11 vision models and let’s give them the same kinds of perception tests that we were giving to these
2:53:16 people. And then he would find the images where the computer vision models would just struggle.
2:53:21 And you would find that they just didn’t do well. Even if you add more parameters, you add more layers
2:53:27 on and on and on, it doesn’t help, right? The architecture didn’t matter. It was just there.
2:53:30 The problem. And then he found those were the exact ones where these humans with particular
2:53:36 damage to this area called the periorinal cortex, that was where they were struggling.
2:53:40 So somehow this brain area was important for being able to do these things that were adversarial to
2:53:48 these computer vision models. So then he found that it only happened if people had enough time
2:53:57 they could make those discriminations. But without enough time, if they just get a glance,
2:54:01 they’re just like the computer vision models. So then what he started to say was, maybe let’s
2:54:05 look at people’s eyes, right? So a computer vision model sees every pixel all at once, right?
2:54:11 It’s not, you know, and we don’t, we never see every pixel all at once. Even if I’m looking at a
2:54:16 screen with pixels, I’m not seeing every pixel at once. I’m grabbing little points on the screen by
2:54:23 moving my eyes around and getting a very high resolution picture of what I’m focusing on and
2:54:29 kind of a lower resolution information about everything else. But I’m not necessarily choosing,
2:54:35 but I’m directing that exploration and allowing people to move their eyes and integrate that
2:54:42 information gave them something that the computer vision models weren’t able to do.
2:54:48 So somehow integrating information across time and getting less information at each step gave
2:54:55 you more out of the process. I mean, the process of allocating attention across time seems to be
2:55:06 a really important process. Even the breakthroughs that you get with machine learning mostly has
2:55:15 to do, attention is all you need. It’s about attention, transform is about attention. So
2:55:19 attention is a really interesting one. But then like, yeah, how you allocate that attention
2:55:26 again is like, is at the core of like, what it means to be intelligent, what it means to process
2:55:36 the world, integrate all the important things, discard all the unimportant things.
2:55:43 Attention is at the core of it. It’s probably at the core of memory too.
2:55:47 Because there’s so much sensor information, there’s so much going on, so much going on,
2:55:52 to filter it down to almost nothing and just keep those parts and to keep those parts. And then
2:55:59 whenever there’s an error to adjust the model such that you can allocate attention even better to
2:56:04 new things that would result, maybe maximize the chance of confirming the model or disconfirming
2:56:10 the model that you have and adjusting it since then. Yeah, attention is a weird one. I was
2:56:16 always fascinated. I mean, I got a chance to study peripheral vision for a bit and indirectly study
2:56:25 attention through that. And it’s just fascinating how good humans are looking around and gathering
2:56:31 information. Yeah, at the same time, people are terrible at detecting changes that can happen
2:56:37 in the environment if they’re not attending in the right way, if their predictive model is too
2:56:42 strong. So you have these weird things where the machines can do better than the people.
2:56:48 So this is the thing is people go, “Oh, the machines can do this stuff that’s just like humans.”
2:56:54 It’s like, well, the machines make different kinds of mistakes than the people do. And I will never
2:57:01 be convinced unless that we’ve replicated human. I don’t even like the term intelligence,
2:57:07 because I think it’s a stupid concept. But it’s like, I don’t think we’ve replicated human
2:57:12 intelligence unless I know that the simulator is making exactly the same kinds of mistakes that
2:57:19 people do. Because people make characteristic mistakes. They have characteristic biases,
2:57:24 they have characteristic heuristics that we use. And those have yet to see evidence that
2:57:31 chat GPT will do that. Since we’re talking about attention, is there an interesting connection
2:57:38 to you between ADHD and memory? Well, it’s interesting for me, because when I was a child,
2:57:45 I was actually told my school, I don’t know if it came from a school psychologist, they did do
2:57:50 some testing on me, I know for like IQ and stuff like that. Or if it just came from teachers who
2:57:57 hated me, but they told my parents that I had ADHD. And so this was of course in the ’70s. So
2:58:04 basically, they said he has poor motor control and he’s got ADHD. And there were social issues.
2:58:12 So I could have been put a year ahead in school, but then they said, oh, but he doesn’t have the
2:58:18 social capabilities. So I still ended up being an outcast even in my own grade. But
2:58:27 but then like, so then my parents said, okay, well, they got me on a diet free of artificial
2:58:34 colors and flavors, because that was the thing that people talked about back then. So I’m interested
2:58:39 this topic because I’ve come to appreciate now that I have many of the characteristics, if not,
2:58:45 you know, full blown. It’s like, I’m definitely time blindness, rejection sense. If you name it,
2:58:51 they talk about it. It’s like, impulsive behavior, I could tell you about all sorts of fights I’ve
2:58:56 gotten into in the past. Just you name it. But yeah, so ADHD is fascinating, though, because
2:59:04 right now we’re seeing like, more and more diagnosis of it. And I don’t know what to say about that.
2:59:09 I don’t know how much of that is based on kind of inappropriate expectations, especially for
2:59:18 children. And how much of that is based on true kind of like maladaptive kinds of tendencies.
2:59:25 But what we do know is this is that ADHD is associated with differences in prefrontal
2:59:30 function, so that attention can be both more, you’re more distractible, you have harder time
2:59:37 focusing your attention on what’s relevant. And so you shift too easily. But then once you get on
2:59:42 something that you’re interested in, you can get stuck. And so, you know, the attention is this
2:59:47 beautiful balance of being able to focus when you need to focus and shift when you need to shift.
2:59:54 And so it’s that flexibility plus stability again. And that’s balance seems to be disrupted in ADHD.
3:00:03 And so as a result, memory tends to be poor in ADHD. But it’s not necessarily because there’s a
3:00:09 traditional memory problem. But it’s more because of this attentional issue, right? And so people
3:00:18 with ADHD often will have great memory for the things that they’re interested in. And just no
3:00:24 memory for the things that they’re not interested in. Is there advice from your own life on how to
3:00:30 learn and succeed from that? From just how the characteristics of your own brain with ADHD and
3:00:36 so on? How do you learn? How do you remember information? How do you flourish in this sort
3:00:47 of education context? I’m still trying to figure out the flourishing per se. But education, I mean,
3:00:53 being in science is enormously enabling of ADHD. It’s like, you’re constantly looking for new things,
3:01:00 you’re constantly seeking that dope of being hit. And that’s great, you know, and they tolerate,
3:01:07 you’re being late for things. Nothing is really, nobody’s going to die if you screw up. It’s nice.
3:01:12 It’s not like being a doctor or something where you have to be like, much more responsible and
3:01:17 focused, that you can just freely follow your curiosity, which is just great. But what I’d
3:01:24 say is that, like, I’m learning now about so many things like about how to structure my activities
3:01:32 more and basically say, okay, if I’m going to be emails like the big one that kills me right now,
3:01:40 I’m just constantly like shifting between email and my activities. And what happens is that I
3:01:46 don’t actually get the email. I just look at my email and I get stressed because I’m like,
3:01:50 oh, I have to think about this. Let me get back to it. And I go back to something else. And so
3:01:54 I’ve just got fragmentary memories of everything, right? So what I’m trying to do is set aside a
3:02:00 timer like this is my email time. This is my, you know, writing time. This is my goofing off time.
3:02:07 And so blocking these things off, you give yourself the goofing off time. Sometimes I do that.
3:02:12 And sometimes I have to be flexible, like, okay, I’m definitely not focusing. I’m going to give
3:02:17 myself the downtime and it’s an investment. It’s not like wasting time. It’s an investment
3:02:22 in my attention later on. And I’m very much with Cal Newport on this. He wrote deep work and a lot
3:02:29 of other amazing books. He talks about tasks switching as a sort of the thing that really
3:02:36 destroys productivity. So like, you know, switching, it doesn’t even matter from what to what, but
3:02:42 checking social media, checking email, maybe switching to a phone call and then doing work
3:02:47 and switching, even switching between, if you’re reading a paper, switching from paper to paper
3:02:52 to paper, because like curiosity and whatever the dopamine hit from the attention switch,
3:02:59 like limiting that because otherwise your brain is just not capable to really like load it in,
3:03:05 really do that deep deliberation. I think that’s required to remember things and to really think
3:03:14 through things. Yeah, I mean, you probably see this, I imagine, in AI conferences, but definitely in
3:03:20 neuroscience conferences, it’s now the norm that people have their laptops out during talks.
3:03:26 And, you know, conceivably, they’re writing, you know, they’re writing notes. But in fact,
3:03:31 what often happens if you look at people, we can speak from a little bit of personal experience,
3:03:37 is you’re checking email and you’re like, or I’m working on my own talk, but often it’s like,
3:03:43 you’re doing things that are not paying attention. And I have this illusion, well,
3:03:46 I’m paying attention and then I’m going back. And then what happens is I don’t remember anything
3:03:51 from that day. It just kind of vanished because what happens is I’m creating all these artificial
3:03:56 event boundaries. I’m losing all this executive function. Every time I switch, I’m getting like
3:04:03 a few seconds slower and I’m catching up mentally to what’s happening. And so instead of being in
3:04:09 a model where you’re meaningfully integrating everything and predicting and generating this
3:04:13 kind of like rich model, I’m just catching up, you know. And so, yeah, there’s great research
3:04:20 by Melina Unkafer and Anthony Wagner on multitasking and people can look up that talks about just how
3:04:26 bad it is for memory and, you know, it’s becoming worse and worse of a problem.
3:04:30 So, you’re a musician. Take me through how’d you get into music? Like what made you first
3:04:36 fall in love with music? With creating music? Yeah, so I started playing music just when I was
3:04:42 like doing trumpet in school for a school band and I would just read music and play and, you know,
3:04:49 it was pretty decent at it, not great, but I was decent.
3:04:52 How’d you go from trumpet to guitar, especially the kind of music you’re into?
3:04:58 Yeah, so basically in high school, yeah, so I kind of was a late bloomer to music, but just
3:05:05 kind of MTV grew up with me. I grew up with MTV. And so then you started seeing all this stuff and
3:05:12 then I got into metal was kind of like my early genre. And I always reacted to just things that
3:05:18 were loud and had a beat like ADHD, right? Like, you know, everything from Sergeant Pepper’s by
3:05:26 the Beatles to like Led Zeppelin II, my dad had both, my parents had both those albums,
3:05:32 so I’d listened to them a lot. And then like the police ghosted in the machine and, but then I
3:05:38 got into metal, Def Leppard and, you know, AC/DC Metallica went way down the rabbit hole of speed
3:05:46 metal. And that time was kind of like, “Oh, why don’t I play guitar? I can do this.” And I had
3:05:53 friends who were doing that. And I just never got it. Like I took lessons and stuff like that.
3:05:59 But it was different because when I was doing trumpet, I was reading sheet music. And this was
3:06:04 like, I was learning by looking, there’s a thing called tablature, you know, this where it’s like,
3:06:09 you see like a drawing of the fretboard with numbers. And that’s where you’re supposed to put
3:06:13 your, it’s kind of like paint by numbers, right? And so I learned it in a completely different way,
3:06:20 but I was still terrible at it. And I didn’t get it. It’s actually taken me a long time to
3:06:26 understand exactly what the issue was. But it wasn’t until I really got into punk, and I saw
3:06:31 bands like, I saw Sonic Youth, I remember especially, and it just blew my mind. Because
3:06:37 they violated the rules of what I thought music was supposed to be. I was like,
3:06:41 this doesn’t sound right. These are not power chords. And this isn’t just have like a
3:06:47 shouty verse and then a chorus part. It’s not going back. This is just like weird. And then it
3:06:53 occurred to me, you don’t have to write music the way it’s people tell you it’s supposed to sound.
3:07:00 I just opened up everything for me. And I was playing in a band and I was struggling with writing
3:07:06 music because I would try to write like, you know, whatever was popular at the time and or
3:07:12 whatever sounded like other bands that I was listening to. And somehow I kind of morphed
3:07:16 into just like, just grabbing a guitar and just doing stuff. And I realized a part of my problem
3:07:23 with doing music before was I didn’t enjoy trying to play stuff that other people play. I just enjoyed
3:07:29 music just dripping out of me and just, you know, spilling out and just doing stuff.
3:07:34 And so then I started to say, what if I don’t play a chord? What if I just play like notes that
3:07:40 shouldn’t go together and just mess around with stuff? And I said, well, what if I don’t do four
3:07:45 beats go na na na na, one, two, three, four, one, two, three, four, one, two, three, four, whatever
3:07:49 I go, one, two, three, four, five, one, two, three, four, five, and sort of mess around time
3:07:54 signatures. Then I was playing in this band with a great musician who was really
3:07:59 Brent Ritzel, who was in this band with me. And he taught me about arranging songs. And it was
3:08:04 like, what if we take this part and instead of make it go like back and forth, we make it like a
3:08:09 circle? Or what if we make it like a straight line, you know, or zigzag, you know, just make it like
3:08:15 nonlinear in these interesting ways. And then next, you know, it’s like the whole world sort of
3:08:21 opens up as like the, and then what I started to realize, especially so you could appreciate this
3:08:26 as a musician, I think. So time signatures, right? So we are so brainwashed to think in four-four,
3:08:32 right? Every rock song you could think of almost is in four-four. I know you’re a Floyd fan,
3:08:37 so think of “Money” by Pink Floyd, right? You feel like it’s in four-four because it resolves
3:08:47 itself, but it resolves on the last note of the, basically it resolves on the first note of the
3:08:54 next measure. So it’s got seven beats instead of eight where the riff is actually happening.
3:08:59 Interesting. But you’re thinking in four, because that’s how we use, we’re used to thinking. So
3:09:04 the music flows a little bit faster than it’s supposed to, and you’re getting a little bit
3:09:10 of prediction error every time this is happening. And once I got used to that, I was like, I hate
3:09:17 writing in four-four because I was like, everything just feels better if I do it in seven-four,
3:09:21 if I alternate between four and three and doing all this stuff. And then it’s like, you just,
3:09:26 jazz music is like that. They just do so much interesting stuff with this.
3:09:32 So playing with those time signatures allows you to really break it all open and just,
3:09:36 I guess there’s something about that that allows you to actually have fun.
3:09:40 Yeah, yeah. And it’s like, so I’m actually like a very, one of the genres we used to play
3:09:47 in was math rock. That’s what they called it. It was just like, this is so many weird time
3:09:51 signatures. What is math? Oh, interesting. Yeah. So that’s the math part of rock is what,
3:09:57 the mathematical disturbances of it or what? Yeah. I guess it would be like, so instead of,
3:10:01 you might go like, instead of playing four beats in every measure, no, no, no, no, no, no, no, no,
3:10:06 no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no.
3:10:10 You know what? Just do these things. And then you might arrange it in weird ways so that there
3:10:15 might be three measures of verse and then one, you know, and then five measures, of course,
3:10:22 and then two measures. So you could just mess around with everything, right?
3:10:24 What does that feel like to listen to? There’s, there’s something about symmetry or like patterns
3:10:31 that feel good and like relaxing for us or whatever. It feels like home and disturbing that can be
3:10:38 quite disturbing. Yeah. So is that, is that the feeling you would have if you
3:10:44 math rock? I mean, yeah, yeah. That’s stressing me out. Just listen.
3:10:47 Learning about it. So, I mean, it depends. So a lot of my style of songwriting is very much like,
3:10:55 in terms of like repetitive themes, but messing around with structure, because I’m not a great
3:11:02 guitarist technically. And so I don’t play complicated stuff. And there’s things you can
3:11:06 hear stuff where it’s just like so complicated, you know. But often what I find is, is like having
3:11:13 a melody or, and then adding some dissonance to it, just enough. And then adding some complexity
3:11:20 that gets, gets you going just enough. But I have a high tolerance for, for that kind of dissonance
3:11:26 and prediction. I think I have a theory, a pet theory that it’s like, basically you could explain
3:11:30 most of human behavior as some people are lumpers and some people are splitters, you know. And so
3:11:36 it’s like some people are very kind of excited when they get this dissonance and they want to like go
3:11:42 with it. So people are just like, no, I want to lump everything. You know, I don’t know, maybe
3:11:45 that’s even a different thing. But it’s like, basically, it’s like, I think some people get
3:11:49 scared of that discomfort. And I really drive on it. You know, I love it. What’s, what’s the name
3:11:58 of your band now? The cover band I play in is a band called Pavlov’s Dogs. And so it’s a band,
3:12:07 unsurprisingly, of mostly memory researchers, neuroscientists. I love this. I love this so
3:12:12 much. Yeah, actually, one of your MIT colleagues, Earl Miller, plays bass.
3:12:16 Plays bass. Do you play a rhythm or a leader? You could compete if you want. Maybe we could
3:12:20 audition you. For audition? Oh, yeah. I’m coming for you, Earl. Earl is going to kill me.
3:12:27 He’s like very precise though. I’ll play Triangle or something.
3:12:33 Or where’s the cowbell? I’ll be the cowbell guy. What kind of songs do you guys do?
3:12:39 So it’s mostly late 70s punk and 80s new wave and post punk. Blondie, Ramones, Clash. I do,
3:12:51 I sing Age of Consent by New Order and Loveville Terrace.
3:12:55 And you said you have a female singer now?
3:12:56 Yeah, Carrie Hoppin and also Paula Crox. So Carrie does Blondie amazingly well.
3:13:07 And we do Gigantic by the Pixies. Paula does that one.
3:13:11 Which song do you love to play the most? What kind of song is super fun for you?
3:13:15 Of someone else’s?
3:13:17 Yeah, Cover. And it’s one we do with Pavlov’s Dogs.
3:13:24 I really enjoy playing I Want to Be Your Dog by Iggy and the Stooges.
3:13:28 That’s a good song.
3:13:29 Which is perfect because we’re Pavlov’s Dogs. And Pavlov, of course, was basically
3:13:34 created learning theory. So there’s that. But also, it’s like, but I mean, Iggy and the Stooges,
3:13:39 that song, so I play and sing on it, but it’s just like it devolves into total noise. And I
3:13:44 just fall on the floor and generate feedback. I think in the last version, it might have been
3:13:51 that or a Velvet Underground cover in our last show, I actually, I have a guitar made of aluminum
3:13:56 that I got made. And I thought this thing’s indestructible. So I kind of like was just,
3:14:01 moving it around, had it upside down and all this stuff to generate feedback.
3:14:06 And I think I broke one of the, I broke one of the tuning pegs.
3:14:08 I’ve had to break it all metal guitar. Go figure.
3:14:14 A bit of a big ridiculous question. But let me ask you, we’ve been talking about
3:14:18 neuroscience in general. What do you, you’ve been studying the human mind for a long time.
3:14:25 What do you love most about the human mind? Like when you look at it, we look at the fMRI,
3:14:33 just the scans and the behavioral stuff, the electrodes, you know, the psychology aspect,
3:14:39 reading the literature on the biology side, neurobiology, all of it. When you look at it,
3:14:43 what, what is most like beautiful to you? I think the most beautiful, but
3:14:50 incredibly hard to put your finger on is this idea of the internal model. That it’s like,
3:14:58 there’s everything you see and there’s everything you hear and touch and taste, you know, every
3:15:03 breath you take, whatever. But it’s all connected by this like dark energy that’s holding that whole
3:15:13 universe of your mind together, right? And without that, it’s just a bunch of stuff.
3:15:18 And somehow we put that together and it forms our, so much of our experience and being able
3:15:27 to figure out where that comes from and how things are connected to me is just amazing.
3:15:33 But just this idea of like the world in front of us, we’re only sampling this little bit and
3:15:39 trying to take so much meaning from it. And we do a really good job, not perfect. I mean, you know,
3:15:45 but that ability to me is just amazing. Yeah, it’s an incredible mystery, all of it. It’s fun.
3:15:51 You said dark energy because the same in astrophysics, you look out there, look at dark matter and
3:15:56 dark energy, which is this loose term, a scientific thing we don’t understand, which makes out,
3:16:02 which helps make the equations work in terms of gravity and the expansion of the universe
3:16:08 in the same way. It seems like there’s that kind of thing in the human mind that we’re like
3:16:12 striving to understand. Yeah, yeah. You know, it’s funny that you mentioned that. So one of the
3:16:16 reasons I wrote the book amongst many is that I really felt like people needed to hear from scientists
3:16:22 and like COVID was just a great example of this because like people weren’t hearing from scientists.
3:16:28 One of the things I think that people didn’t get was the uncertainty of science and how much we
3:16:34 don’t know. And I think every scientist lives in this world of uncertainty. And when I was
3:16:41 writing the book, I just became aware of all of these things we don’t know. And so I think of
3:16:47 physics a lot. And I think of this idea of like overwhelming majority of the stuff that’s in our
3:16:55 universe cannot be directly measured. I used to think, “Ha ha, I hate physics.” So physicists get
3:17:01 the Nobel Prize for doing whatever stupid thing. It’s like there’s 10 physicists out there. I’m
3:17:06 just kidding. Strong words. Yeah. No, no, no. I’m kidding. The physicists who do neuroscience
3:17:12 could be rather opinionated. So sometimes I like to dish on that. It’s all love. It’s all love. That’s
3:17:16 right. This is the ADHD talking. But at some point, I had this aha moment where I was like,
3:17:25 to be aware of that much that we don’t know and have a beat on it and be able to go towards it,
3:17:34 that’s one of the biggest scientific successes that I could think of.
3:17:38 You are aware that you don’t know about this gigantic section, overwhelming majority of the
3:17:44 universe, right? And I think the more what keeps me going to some extent is realizing the changing
3:17:54 the scope of the problem and figuring out, “Oh my God, there’s all these things we don’t know.”
3:17:59 And I thought I knew this because science is all about assumptions, right? So have you
3:18:04 read the structure of Scientific Revolutions by Thomas Kuhn? Yes. That’s like my only philosophy
3:18:10 really that I’ve read. But it’s so brilliant in the way that they frame this idea of like,
3:18:16 he frames this idea of assumptions being core to the scientific process and the paradigm shift
3:18:22 comes from changing those assumptions. And this idea of like finding out this kind of whole zone
3:18:28 of what you don’t know to me is the exciting part. Well, you are a great scientist and you wrote an
3:18:37 incredible book. So thank you for doing that. And thank you for talking today. You’ve decreased
3:18:44 the amount of uncertainty I have just a tiny little bit today and revealed the beauty of memory.
3:18:52 This is a fascinating conversation. Thank you for talking today. Oh, thank you. It’s been
3:18:55 blast. Thanks for listening to this conversation with Sharon Ranganath. To support this podcast,
3:19:03 please check out our sponsors in the description. And now let me leave you with some words from
3:19:08 Haruki Murakami. Most things are forgotten over time. Even the word itself, the life and death
3:19:17 struggle people went through is now like something from the distant past. We’re so caught up in our
3:19:23 everyday lives that events of the past are no longer in orbit around our minds. There are just
3:19:29 too many things we have to think about every day, too many new things we have to learn. But still,
3:19:35 no matter how much time passes, no matter what takes place in the interim, there are some things
3:19:42 we can never assign to oblivion, memories we can never rub away. They remain with us forever,
3:19:49 like a touchstone. Thank you for listening. I hope to see you next time.
3:19:55 [Music]