#430 – Charan Ranganath: Human Memory, Imagination, Deja Vu, and False Memories

0:00:00 The following is a conversation with Charan Ranganath, a psychologist and neuroscientist
0:00:05 at UC Davis, specializing in human memory. He’s the author of Why We Remember,
0:00:12 Unlocking Memory’s Power to Hold On to What Matters.
0:00:15 And now, a quick few second mention of each sponsor. Check them out in the description.
0:00:21 It’s the best way to support this podcast. We got Riverside for recording remote podcasts,
0:00:26 Zip Recruiter for hiring, Notion for note-taking and team collaboration, Masterclass for learning,
0:00:32 Shopify for e-commerce and Element for delicious, delicious hydration. Choose wisely, my friends.
0:00:39 Also, if you want to work with our amazing team, or you just want to get in touch with me,
0:00:44 go to lexfreement.com/contact. And now, onto the full ad reads. As always, no ads in the middle.
0:00:50 I try to make these interesting, but if you must skip them, friends, please do check out our sponsors.
0:00:56 I enjoy their stuff. Maybe you will too. This episode is also brought to you by Riverside,
0:01:03 the platform for recording remote podcasts and studio quality. I’ve used them a bunch of times
0:01:07 in the past. They’re amazing. It is the thing I recommend for anybody, especially for people
0:01:12 starting a podcast, studio quality, exceptionally easy to use, a million features that are all
0:01:18 extremely useful for the whole pipeline of creating a podcast. I mean, where do I start?
0:01:23 First of all, they do the editing. And you could do text-based editing of the audio and the video.
0:01:27 So whatever you say, there’s an AI-generated transcript in like 100 plus languages,
0:01:33 whatever the language is, you can use then the text to do the editing. It does speaker detection,
0:01:40 so it figures out who’s speaking. All the synchronization obviously is done, not obviously,
0:01:44 because some things seem obvious, but a really effortless, beautiful execution of it just is
0:01:52 breath of fresh air. So in case you don’t know, it’s through the browser, you record the video
0:01:58 and the audio both sides of the conversation, everything is synchronized, everything is stored,
0:02:02 just everything is done really well. They have a lot of recommendations of what kind of hardware
0:02:07 to use. I think in the video they provide, they say the most important thing is the microphone
0:02:12 and lighting, and I agree with that. Good audio is number one. Second to that is indeed lighting,
0:02:18 because basically every kind of camera that’s available now will do all right. Anyway,
0:02:24 Riverside makes that whole process super easy. I record my remote interviews with Riverside,
0:02:29 give it a try at riverside.fm and use code LEX for 30% off. That’s riverside.fm and use code LEX.
0:02:36 This episode is also brought to you by ZipRecruiter, a site that connects employers and
0:02:43 job seekers. To me, one of the most fulfilling things in life is working together with a great
0:02:50 team. I love working. I love what I do. Everywhere I’ve ever worked, I loved doing it, and I love
0:02:57 to be surrounded by people who also love doing it, and especially who are very good at it,
0:03:03 and are pushing themselves to the limit and together we’re creating something special,
0:03:07 whatever that is. It could be a small thing, or it could be a world-changing thing,
0:03:11 or the mission is small, or the mission is big. As long as there’s a mission and we’re in it
0:03:16 together and we’re constantly improving, I mean, like a team that works great together,
0:03:21 full of great people is one of the real joys of life. I think that’s true for me. I think that’s
0:03:27 true for anybody, because so much of our lives is spent working. And that’s where we really,
0:03:34 especially in the realm of intellectual pursuits, really challenge ourselves. And so in the process
0:03:40 of that challenge is what we find meaning. So build great teams and use the best tools to do it.
0:03:47 See why four out of five employers who post on ZipRecruiter get a quality candidate within the
0:03:52 first day. Go to ziprecruiter.com/lex@tryforfree. That’s ziprecruiter.com/lex, the smartest way to hire.
0:04:00 This episode is brought to you by Notion, a note-taking and team collaboration tool. I’ve used
0:04:06 it for a long time now for note-taking, for organizing my thoughts, for connecting my thoughts,
0:04:12 for searching through my thoughts, and now using AI to summarize, organize,
0:04:19 generate drafts of things that I’m either planning or ideas that I’m working through,
0:04:25 or the research that I’m doing. Now that’s for the individual where Notion really starts to shine,
0:04:30 is when there’s multiple people working together. It is an incredible team collaboration tool. And
0:04:36 again, the AI component gets integrated really nicely, because you can do the search, you can do
0:04:42 the summarization, you can create a report of what everybody’s been working on. It looks to the docs,
0:04:47 the wakeys, the projects, and can basically do a Q&A for you to figure out like where do things
0:04:53 stand from a manager position, or from an individual contributor, what am I supposed to be doing,
0:04:58 what are other people doing, where can I help, that kind of stuff. Try Notion AI for free when
0:05:04 you go to Notion.com/Lex, that’s all lowercase Notion.com/Lex to try the power of Notion AI today.
0:05:12 This episode is brought to you by Masterclass, where you can watch over 180 classes from the best
0:05:20 people in the world in their respective disciplines, Phil Iveyampoker, Aaron Franklin on barbecue,
0:05:25 and Burskoot, Carlos Santana on guitar, Tom Morello on guitar, Terence Tao on mathematical
0:05:31 thinking, Martin Scorsese on filmmaking. In fact, I would really love and I’m planning on talking to
0:05:37 actors and directors more. I love film. I love great TV. I love that medium of storytelling.
0:05:47 And great actors and great directors are the way we consume stories. They are the medium,
0:05:57 the channels, the wizards through which we, all of us, take in the stories, new exciting stories,
0:06:05 or stories of old, retold, better and better and better. So I would like to talk to those people.
0:06:10 WTF podcast by Mark Marin. In the past, I really loved it when he interviewed actors and directors
0:06:20 and he’s done it really well. Inside Actors Studio was a program I really loved when
0:06:25 long form interviews with actors, long form interviews with directors,
0:06:30 even Charlie Rose did a really good job with that. Not the click bait sort of Hollywood style
0:06:35 journalism, but more long form conversations. I would love to do more of those. Get unlimited
0:06:43 access to every masterclass and get an additional 15% off an annual membership at masterclass.com/lexpod.
0:06:50 That’s masterclass.com/lexpod. This episode is also brought to you by Shopify,
0:06:58 a platform designed for anyone to sell anywhere with a great looking online store.
0:07:02 I got a store, lexperiment.com/store. It has a few shirts on there. If you want to get a shirt,
0:07:09 you can get it. It was so easy to set up. I like the machinery of humans selling stuff and buying
0:07:16 stuff and through that capitalist machine figuring out together the things that bring happiness to
0:07:24 our lives. In fact, the things isn’t the source of happiness, of course. The things are the catalyst
0:07:30 for human connection, for humans to connect with each other. A t-shirt with Metallica or whatever
0:07:37 band or whatever podcast or whatever show you like, its power is not in the fact that it looks
0:07:43 good or something like this. It’s power in the connection you make when another person notices
0:07:48 it and are also a fan of Metallica or whatever is on the shirt or they don’t know anything about
0:07:54 Metallica but they like the logo and it starts a conversation where they’d be like, “What is that?
0:07:58 Metallica? Is that some kind of machine shop thing?” And you say, “No. It is the greatest metal band of
0:08:04 all time.” And there you grab a beer and the conversation begins. It’s the human connection.
0:08:11 The capitalist machine is not enough. It is merely a catalyst for the beauty of human connection.
0:08:19 So join, if you want, the capitalist machine by signing up for a $1 per month trial period
0:08:26 at Shopify.com/Lex. That’s all lowercase. Go to Shopify.com/Lex to take your business to the next
0:08:32 level today. This episode is brought to you by Element. It’s the delicious electrolyte drink,
0:08:39 sodium potassium magnesium that I drink every day, a lot of it every day. I drink it in the jungle.
0:08:45 When I was dying of thirst, when I was dehydrated and questioning whether I would be able to make it
0:08:52 through the day, not to mention the night, I had one pack of element with me waiting,
0:08:59 knowing that I would disperse that pack of element into whatever water I would be able to find because
0:09:06 you know, electrolyte is also really important. But also, I knew that I’ve had to drink sort of
0:09:12 water, still water, full of mud and all of that. Element would be the thing that makes it taste
0:09:18 good. My favorite flavor, the one I brought to the jungle, the one I always drink is watermelon
0:09:22 salt. But they also have cans now, which is like a carbonated fizzy thing to it and lots of great
0:09:31 flavors and I really love it. And I’ve been drinking that nonstop. Whenever I get some, I drink
0:09:37 all of it very quickly. So I highly recommend that as well. Either the packs or the cans of
0:09:42 element drink, love it. Get a sample pack for free with any purchase. Try it at drinkelement.com/lex.
0:09:51 This is the Lex Friedman podcast. To support it, please check our sponsors in the description.
0:09:58 And now, dear friends, here’s Charon Ranganath.
0:10:05 Danny Conwin describes the experiencing self and the remembering self. And that happiness
0:10:26 and satisfaction you gain from the outcomes of your decisions do not come from what you’ve
0:10:30 experienced, but rather from what you remember of the experience. So can you speak to this
0:10:36 interesting difference that you write about in your book of the experiencing self and the
0:10:40 remembering self? Danny really impacted me because I was an undergrad at Berkeley and
0:10:45 I got to take a class from him long before he won the Nobel Prize or anything. And it was just a
0:10:49 mind-blowing class. But this idea of the remembering self and the experiencing self, I got into it
0:10:57 because it’s so much about memory, even though he doesn’t study memory. So we’re right now having
0:11:02 this experience, right? And people can watch it presumably on YouTube or listen to it on audio.
0:11:08 But if you’re talking to somebody else, you could probably describe this whole thing in 10 minutes.
0:11:13 But that’s going to miss a lot of what actually happened. And so the idea there is that the way
0:11:20 we remember things is not the replay of the experience. It’s something totally different.
0:11:25 And it tends to be biased by the beginning and the end. And he talks about the peaks. And there’s
0:11:31 also the best parts, the worst parts, et cetera. And those are the things that we remember. And so
0:11:38 when we make decisions, we usually consult memory. And we feel like our memory is a record of what
0:11:45 we’ve experienced, but it’s not. It’s this kind of very biased sample, but it’s biased in an
0:11:51 interesting and I think biologically relevant way. So in the way we construct a narrative
0:11:56 about our past, you say that it gives us an illusion of stability. Can you explain that?
0:12:05 Basically, I think that a lot of learning in the brain is driven towards being able to make sense.
0:12:13 I mean, really, memory is all about the present and the future. Past is done. So biologically
0:12:18 speaking, it’s not important unless there’s something from the past that’s useful. And so what
0:12:24 our brains are really optimized for is to learn about the stuff from the past that’s going to be
0:12:30 most useful in understanding the present and predicting the future. And so cause-effect
0:12:36 relationships, for instance. That’s a big one. Now, my future is completely unpredictable
0:12:41 in the sense that you could, in the next 10 minutes, pull a knife on me and slip my throat,
0:12:46 right? I was planning on it. Exactly. But having seeds of your work and just generally my expectations
0:12:53 about life, I’m not expecting that. I have a certainty that everything’s going to be fine.
0:12:58 We’re going to have a great time talking today, right? But we’re often right. It’s like, okay,
0:13:02 so I go to see a band on stage. I know they’re going to make me wait. The show’s going to start
0:13:09 laying. Then they come on. There’s a very good chance there’s going to be an encore. I have a
0:13:15 memory, so to speak, for that event before I’ve even walked into the show. There’s going to be
0:13:19 people holding up their camera phones, try to take videos of it now because this is the world we
0:13:25 live in. That’s like everyday fortune telling that we do, though. It’s not real. It’s imagined.
0:13:32 It’s amazing that we have this capability, and that’s what memory is about.
0:13:35 But it can also give us this illusion that we know everything that’s about to happen.
0:13:41 I think what’s valuable about that illusion is when it’s broken, it gives us the information,
0:13:49 right? I’m sure being in AI, you know, about information theory, and the idea is the information
0:13:55 is what you didn’t already have. Those prediction errors that we make based on memory, and the
0:14:02 errors are where the action is. The error is where the learning happens. Exactly. Exactly.
0:14:09 Well, just to linger on Danny Kahneman and just this whole idea of experiencing self versus
0:14:18 remembering self, I was hoping you can give a simple answer of how we should live life.
0:14:24 Based on the fact that our memories could be a source of happiness, or could be the primary
0:14:34 source of happiness, that an event when experienced bears its fruits the most when it’s remembered
0:14:43 over and over and over and over. Maybe there is some wisdom in the fact that we can control to
0:14:48 some degree how we remember it, how we evolve our memory of it, such that it can maximize
0:14:55 the long-term happiness of that repeated experience.
0:14:59 Okay. Well, first, I’ll say I wish I could take you on the road with me because that was such a
0:15:04 great description. Can I be your opening actor? Oh, my God. No, I’m going to open for you, dude.
0:15:10 Otherwise, it’s like everybody leaves after you’re done.
0:15:13 Believe me, I did that in Columbus, Ohio once. It wasn’t fun. The opening acts drank our bar tab.
0:15:22 We spent all this money going all the way there. Everybody left after the opening acts were done,
0:15:28 and there was just that stoner dude with the dreadlocks hanging out. And then next to you,
0:15:32 we blew our savings on getting a hotel room. So we should, as a small tangent, you’re a legit
0:15:40 touring act. When I was in grad school, I played in a band. And yeah, we traveled. We would play
0:15:45 shows. It wasn’t like we were in a hardcore touring band, but we did some touring and had some fun
0:15:50 times. And yeah, we did a movie soundtrack. Nice. Henry Portrait of Serial Killer. So that’s a good
0:15:57 movie. We were on the soundtrack for the sequel, Henry II, Mask of Sanity, which is a terrible
0:16:02 movie. Yeah. How’s the soundtrack? It’s pretty good. It’s badass. At least that one part where the
0:16:06 guy throws up the milkshake. It’s my song. We’re going to have to see it. We’re going to have to
0:16:11 see it. All right, we’re getting back to life advice. And happiness, yeah. One thing that I try
0:16:16 to live by, especially nowadays, and since I wrote the book, I’ve been thinking more and more about
0:16:21 this is, how do I want to live a memorable life? I think if we go back to the pandemic,
0:16:28 how many people have memories from that period, aside from the trauma of being locked up and
0:16:37 seeing people die and all this stuff? I think it’s like one of these things where we were stuck
0:16:43 inside looking at screens all day, doing the same thing with the same people. And so I don’t
0:16:51 remember much from that in terms of those good memories that you’re talking about. When I was
0:16:56 growing up, my parents worked really hard for us and we went on some vacations, but not very often.
0:17:03 And I really try to do now vacations to interesting places as much as possible with my family,
0:17:09 because those are the things that you remember. So I really do think about
0:17:16 what’s going to be something that’s memorable and then just do it, even if it’s a pain in the
0:17:21 ass, because the experiencing self will suffer for that, but the remembering self will be like,
0:17:26 “Yes, I’m so glad I did that.” Do things that are very unpleasant in the moment,
0:17:31 because those can be reframed and enjoyed for many years to come. That’s probably
0:17:38 good advice or at least when you’re going through shit, it’s a good way to see the silver lining
0:17:43 of it. Yeah, I think it’s one of these things where if you have people who you’ve gone through,
0:17:49 since you said it, I’ll just say, since you’ve gone through shit with someone, and it’s like,
0:17:54 that’s a bonding experience often. I mean, that can really bring you together.
0:18:00 I like to say it’s like there’s no point in suffering unless you get a story out of that.
0:18:05 So in the book, I talk about the power of the way we communicate with others and how that
0:18:10 shapes our memories. And so I had this near-death experience, at least that’s how I remember it,
0:18:16 on this paddle board, where just everything they could have gone wrong did go wrong almost.
0:18:20 So many mistakes were made and ended up at some point just like basically
0:18:29 away from my board, pinned in a current like in this corner, like not a super good swimmer,
0:18:35 and my friend who came with me, Randy, who’s a computational neuroscientist, and he had just
0:18:40 been pushed down past me and so he couldn’t even see me. And I’m just like, if I die here,
0:18:47 I mean, no one’s around. It’s like you just die alone. And so I just said, well, failure is not
0:18:53 an option. And eventually, I got out of it and froze and got cut up. And I mean, the things that
0:19:01 we were going through were just insane. But a short version of this is, you know, my wife
0:19:08 and my daughter and Randy’s wife, they gave us all sorts of hell about this because they were
0:19:13 just like, where are we? They were ready to send out a search party. So they were giving me hell
0:19:18 about it. And then I started to tell people in my lab about this and then friends. And
0:19:23 it just became a better and better story every time. And we actually had some photos of
0:19:28 just the crazy things like this generator that was hanging over the water and were like ducking
0:19:33 under the zinger, these metal gratings, and I’m like going flat. And it was just nuts, you know.
0:19:38 But it became a great story. And it was definitely, I mean, Randy and I were already tight, but that
0:19:43 was a real bonding experience for us. And yeah, I mean, and I learned from that that it’s like,
0:19:49 I don’t look back on that enough, actually. Because I think we often, at least for me,
0:19:56 I don’t necessarily have the confidence to think that things will work out that I’ll be able to
0:20:00 get through a certain thing. But my ability to actually get something done in that moment
0:20:08 is better than I give myself credit for, I think. And that was the lesson of that story that I
0:20:13 really took away. Well, actually, just for me, you’re making me realize now that it’s not just
0:20:19 those kinds of stories, but even things like periods of depression or really low points.
0:20:26 To me, at least it feels like a motivating thing that the darker it gets, the better the story
0:20:34 will be if you emerge on the other side. That to me feels like a motivating thing. So maybe if
0:20:40 people listening to this and they’re going through some shit, as we said, one thing
0:20:45 that could be a source of light is that it’ll be a hell of a good story when it’s all over,
0:20:51 when you emerge on the other side. Let me ask you about decisions. You’ve already talked about it
0:20:57 a little bit, but when we face the world and we’re making different decisions,
0:21:01 how much does our memory come into play? Is it the kind of narratives that we’ve
0:21:09 constructed about the world that are used to make predictions that’s fundamentally part
0:21:14 of the decision making? Absolutely. Yeah. So let’s say after this, you and I decided we’re
0:21:19 going to go for a beer, right? How do you choose where to go? You’re probably going to be like,
0:21:23 oh, yeah, this new bar opened up near me at a great time there. They had a great beer selection,
0:21:28 or you might say, oh, we went to this place and it was totally crowded and they’re playing this
0:21:32 horrible EDM or whatever. So right there, valuable source of information, right? And then you have
0:21:40 these things like where you do this counterfactual stuff like, well, I did this previously, but what
0:21:46 if I had gone somewhere else instead? Maybe I’ll go to this other place because I didn’t try it
0:21:50 the previous time. So there’s all that kind of reasoning that goes into it too. I think,
0:21:56 even if you think about the big decisions in life, right? It’s like you and I were talking
0:22:01 before we started recording about how I got into memory research and you got into AI. And it’s like
0:22:07 we all have these personal reasons that guide us in these particular directions. And some of it’s
0:22:13 the environment and random factors in life. And some of it is memories of things that we want to
0:22:19 overcome or things that we build on in a positive way, but either way, they define us.
0:22:27 And probably the earlier in life, the memories happen, the more defining, the more defining
0:22:33 power they have in terms of determining who we become. I mean, I do feel like adolescence is
0:22:39 much more important than I think people give credit for. I think that there is this kind of a sense
0:22:43 like the first three years of life is the most important part. But the teenage years are just
0:22:50 so important for the brain. And so that’s where a lot of mental illness starts to emerge.
0:22:57 Now we’re thinking of things like schizophrenia as a neurodevelopmental disorder, because it just
0:23:03 emerges during that period of adolescence and early adulthood. And I think the other part
0:23:09 of it is that I guess I was a little bit too firm in saying that memory determines who we are. It’s
0:23:15 really the self as an evolving construct. I think we kind of underestimate that. And when you’re
0:23:21 a parent, you feel like every decision you make is consequential in forming this child and plays
0:23:29 a role. But so do the child’s peers. And so do there’s so much. I mean, that’s why I think
0:23:36 the big part of education I think that’s so important is not the content you learn. I mean,
0:23:40 think of how much dumb stuff we learned in school, right? But a lot of it is learning
0:23:47 how to get along with people and learning who you are and how you function. And that can be
0:23:54 terribly traumatizing even if you have perfect parents working on you.
0:23:59 Is there some insight into the human brain that explains why we don’t seem to remember anything
0:24:06 from the first few years of life? Yeah. Yeah. In fact, actually, I was just talking to my
0:24:12 really good friend and colleague, Simona Getty, who studies the neuroscience of child development.
0:24:17 And so we were talking about this. And so there are a bunch of reasons, I would say. So one reason
0:24:23 is there’s an area of the brain called the hippocampus, which is very, very important for
0:24:28 remembering events or episodic memory. And so the first two years of life, there’s a period called
0:24:34 infantile amnesia. And then the next couple of years of life after that, there’s a period called
0:24:40 childhood amnesia. And the difference is that basically in the lab and even during childhood
0:24:46 and afterwards, children basically don’t have any episodic memories for those first two years.
0:24:54 The next two years, it’s very fragmentary. And that’s why they call it childhood amnesia. So
0:24:58 there’s some, but it’s not mine. So one reason is that the hippocampus is taking some time to develop.
0:25:04 But another is the neocortex. So the whole folded stuff of gray matter all around the hippocampus
0:25:10 is developing so rapidly and changing. And a child’s knowledge of the world is just massively
0:25:17 being built up. So I’m going to probably embarrass myself, but it’s like, if you
0:25:22 showed like you trained a neural network and you give it the first couple of patterns or something
0:25:28 like that, and then you bombard it with another year’s worth of data, try to get back those first
0:25:34 couple of patterns, right? It’s like everything changes. And so the brain is so plastic. The
0:25:40 cortex is so plastic during that time. And we think that memories for events are very distributed
0:25:46 across the brain. So imagine you’re trying to get back that pattern of activity that happened
0:25:51 during this one moment. But the roads that you would take to get there have been completely
0:25:56 rerouted, right? So I think that’s my best explanation. The third explanation is a child’s
0:26:01 sense of self takes a while to develop. And so their experience of learning might be more learning
0:26:09 what happened as opposed to having this first person experience of “I remember, I was there.”
0:26:14 Well, I think somebody once said to me that kind of loosely, philosophically, that the reason we
0:26:25 don’t remember the first few years of life, infantile amnesia, is because how traumatic it is.
0:26:32 Basically, the error rate that you mentioned, when your brain’s prediction doesn’t match reality,
0:26:40 the error rate in the first few years of life, your first few months, certainly,
0:26:44 is probably crazy high. It’s just nonstop freaking out. The collision between your model of the
0:26:52 world and how the world works is just so high that you want whatever the trauma of that is,
0:26:57 not to linger around. I always thought that’s an interesting idea because just imagine the insanity
0:27:05 of what’s happening in a human brain in the first couple of years. You don’t know anything.
0:27:10 And there’s just this stream of knowledge, and given how plastic everything is,
0:27:15 it just molds and figures it out. But it’s like an insane waterfall of information.
0:27:24 I wouldn’t necessarily describe it as a trauma. We can get into this whole stages of life thing,
0:27:27 which I just love. Basically, those first few years, think about it. A kid’s
0:27:34 internal model of their body is changing. It’s like just learning to move. If you ever have a baby,
0:27:42 you’ll know that the first three months, they’re discovering their toes. It’s just nuts.
0:27:47 Everything is changing. But what’s really fascinating is, and I think this is not at all
0:27:54 me being a scientist, but it’s like one of those things that people talk about when they talk about
0:27:58 the positive aspects of children is that they’re exceptionally curious, and they have this kind
0:28:05 of openness towards the world. And so that prediction error is not a negative traumatic
0:28:12 thing. I think it’s like a very positive thing because it’s what they use that they’re seeking
0:28:18 information. One of the areas that I’m very interested in is the prefrontal cortex. It’s an
0:28:23 area of the brain that, I mean, I could talk all day about it, but it helps us use our knowledge
0:28:29 to say, “Hey, this is what I want to do now. This is my goal, so this is how I’m going to
0:28:35 achieve it,” and focus everything towards that goal. The prefrontal cortex takes forever to
0:28:41 develop in humans. The connections are still being tweaked and reformed into late adolescence,
0:28:48 early adulthood, which is when you tend to see mental illness pop up. It’s being massively
0:28:54 reformed. Then you have about 10 years maybe of prime functioning of the prefrontal cortex,
0:29:00 and then it starts going down again, and you end up being older, and you start losing all
0:29:04 that frontal function. So look at this, and you’d say, “Okay, you’ve sitting around episodic memory
0:29:10 talks,” and always say, “Children are worse than adults at episodic memory. Older adults are worse
0:29:15 than young adults at episodic memory,” and I always say, “God, this is so weird. Why would we have
0:29:20 this period of time that’s so short when we’re perfect or optimal?” I like to use the word
0:29:26 “optimal” now because there’s such a culture of optimization right now. I have to redefine what
0:29:33 optimal is because for most of the human condition, I think we had a series of stages of life where
0:29:42 you have basically adults saying, “Okay, young adults,” saying, “I’ve got a child, and I’m part of
0:29:50 this village, and I have to hunt and forage and get things done. I need a prefrontal cortex so
0:29:54 I can stay focused on the big picture and the long-haul goals.” Now, I’m a child. I’m in this
0:30:01 village. I’m kind of wandering around, and I’ve got some safety, and I need to learn about this
0:30:07 culture because I know so little. What’s the best way to do that? Let’s explore. I don’t want to be
0:30:12 constrained by goals as much. I want to really be free, play and explore and learn. You don’t want a
0:30:18 super tight prefrontal cortex. You don’t even know what the goals should be yet. If you’re trying to
0:30:24 design a model that’s based on a bad goal, it’s not going to work well. Then you go late in life,
0:30:32 and you say, “Why don’t you have a great prefrontal cortex then?” If you go back and you think,
0:30:38 “How many species actually stick around naturally long after their child-bearing years are over,
0:30:44 after reproductive years are over?” Menopause, from what I understand, menopause is not all that
0:30:49 common in the animal world. Why would that happen? I saw Alison Gopnik said something about this,
0:30:58 so I started to look into this, about this idea that really, when you’re older in most societies,
0:31:04 your job is no longer to form new episodic memories. It’s to pass on the memories that you
0:31:10 already have, this knowledge about the world, what we call semantic memory, to pass on that
0:31:15 semantic memory to the younger generations, to pass on the culture. Even now in indigenous cultures,
0:31:21 that’s the role of the elders. They’re respected. They’re not seen as people who are pasted and
0:31:26 losing it. I thought that was a very poignant thing, that memory is doing what it’s supposed to
0:31:34 throughout these stages of life. It is always optimal in a sense. It’s just optimal for that
0:31:40 stage of life. For the ecology of the system, I looked into this and it’s like another species
0:31:47 that has menopause is orcas. Orcopods are led by the grandmothers. It’s not the young adults,
0:31:53 not the parents or whatever, the grandmothers. They’re the ones that pass on the traditions
0:31:59 to the younger generation orcas. If you look from what little I understand, different orcopods
0:32:06 have different traditions. They hunt for different things. They have different play traditions.
0:32:11 That’s a culture. In social animals, evolution, I think, is designing brains that are really
0:32:21 around. It’s obviously optimized for the individual, but also for kin. I think that the kin are part
0:32:30 of this, when they’re a part of this intense social group, the brain development should parallel
0:32:35 that the nature of the ecology. It’s just fascinating to think of the individual orca or human
0:32:43 throughout its life in stages doing a kind of optimal wisdom development. In the early days,
0:32:52 you don’t even know what the goal is. You figure out the goal and you optimize for that goal and
0:32:56 you pursue that goal. Then all the wisdom you collect through that, then you share with the
0:33:00 others in the system with the other individuals. As a collective, then you kind of converge towards
0:33:07 greater wisdom throughout the generation. In that sense, it’s optimal. Us humans and orcas
0:33:14 got something going on. It works. Oh, yeah. Apex predators.
0:33:18 I just got a megalon tooth. Speaking of apex predators, just imagine the size of that thing.
0:33:29 Anyway, how does the brain forget and how and why does it remember? Maybe some of the mechanisms.
0:33:39 You mentioned the hippocampus. What are the different components involved here?
0:33:43 We could think about this on a number of levels. Maybe I’ll give you the simplest version first,
0:33:47 which is we tend to think of memories as these individual things and we can just access them
0:33:52 maybe a little bit like photos on your phone or something like that.
0:33:56 In the brain, the way it works is you have this distributed pool of neurons and
0:34:01 the memories are kind of shared across different pools of neurons. What you have is competition,
0:34:08 where sometimes memories that overlap can be fighting against each other.
0:34:12 Sometimes we forget because that competition just wipes things out. Sometimes we forget because
0:34:20 there aren’t the biological signals, which you can get into. I would promote long-term retention.
0:34:26 Lots of times we forget because we can’t find the queue that sends us back to the right memory,
0:34:32 and we need the right queue to be able to activate it. For instance, in a neural network,
0:34:38 you wouldn’t go and you’d say, “This is the memory.” The whole ecosystem of memories
0:34:46 is in the weights of the neural network. In fact, you could extract entirely new memories,
0:34:50 depending on how you feed. You have to have the right query, the right prompt,
0:34:54 to access that whatever the part you’re looking for.
0:34:57 That’s exactly right. In humans, you have this more complex set of ways memory works.
0:35:02 As I said, the knowledge or what you call semantic memory, and then there’s these
0:35:07 memories for specific events, which we call episodic memory. There’s different pieces
0:35:12 of the puzzle that require different kinds of queues. That’s a big part of it too,
0:35:18 is just this kind of what we call retrieval failure.
0:35:21 You mentioned episodic memory, you mentioned semantic memory,
0:35:23 what are the different separations here? What’s working memory, short-term memory,
0:35:28 long-term memory? What are the interesting categories of memory?
0:35:32 Yeah. Memory researchers, we love to cut things up and say, “Is memory one thing or is it two
0:35:39 things?” There’s two things, there’s three things. One of the things that there’s value in that,
0:35:44 and especially experimental value in terms of being able to dissect things,
0:35:49 and the real world is all connected. Speak to your question, working memory,
0:35:53 it was a term that was coined by Alan Battley. It’s basically thought to be this ability to
0:35:58 keep information online in your mind right in front of you at a given time, and to be able
0:36:04 to control the flow of that information, to choose what information is relevant,
0:36:08 to be able to manipulate it, and so forth. One of the things that Alan did that was quite brilliant
0:36:14 was he said, “There’s this ability to passively store information, see things in your mind’s eye,
0:36:20 or hear your internal monologue, but we have that ability to keep information in mind.”
0:36:26 But then we also have this separate, what he called a central executive, which is identified a lot
0:36:33 with the prefrontal cortex. It’s this ability to control the flow of information that’s being
0:36:39 kept active based on what it is you’re doing. Now, a lot of my early work was basically saying
0:36:45 that this working memory, which some memory researchers would call short-term memory,
0:36:49 is not at all independent from long-term memory. That is that a lot of executive function requires
0:36:56 learning, and you have to have like synaptic change for that to happen. But there’s also transient
0:37:02 forms of memory. One of the things I’ve been getting into lately is the idea that we form
0:37:08 internal models of events. The obvious one that I always use is birthday parties. As you go to
0:37:14 a child’s birthday party, once the cake comes out and you just see a candle, you can predict the whole
0:37:22 frame set of events that happens later. Up till that point where the child blows out the candle,
0:37:27 you have an internal model in your head of what’s going on. If you follow people’s eyes,
0:37:33 it’s not actually on what’s happening. It’s going where the action’s about to happen,
0:37:37 which is just fascinating. You have this internal model and that’s a kind of a working memory product.
0:37:43 It’s something that you’re keeping online that’s allowing you to interpret this world around you.
0:37:48 Now, to build that model, though, you need to pull out stuff from your general knowledge of
0:37:53 the world, which is what we call semantic memory. Then you’d want to be able to pull out memories
0:37:59 for specific events that happened in the past, which we call episodic memory. In a way, they’re
0:38:05 all connected even though it’s different. The things that we’re focusing on and the way we
0:38:11 organize information in the present, which is working memory, will play a big role in determining
0:38:16 how we remember that information later, which people typically call long-term memory.
0:38:20 So if you have something like a birthday party and you’ve been to many before,
0:38:23 you’re going to load that from disk into working memory, this model, and then you’re mostly operating
0:38:30 on the model. If it’s a new task, you don’t have a model, so you’re more in the data collection.
0:38:39 Yes, one of the fascinating things that we’ve been studying, and we’re not at all the first
0:38:44 to do this, Jeff Sachs was a big pioneer in this, and I’ve been working with many other people,
0:38:49 Ken Norman, Leila Devachi, or Columbia has done some interesting stuff with this,
0:38:55 as this idea that we form these internal models at particular points of high prediction error,
0:39:02 or points of, I believe, also points of uncertainty, points of surprise, or motivationally significant
0:39:08 periods, and those points are when it’s maximally optimal to encode an episodic memory. So I used
0:39:15 to think, “Oh, well, we’re just encoding episodic memories constantly.” But think about how much
0:39:21 redundancy there is in all that. It’s just a lot of information that you don’t need.
0:39:27 But if you capture an episodic memory at the point of maximum uncertainty for the singular
0:39:35 experience, it’s only going to happen once, but if you capture it at the point of maximum
0:39:40 uncertainty or maximum surprise, you have the most useful point in your experience that you’ve grabbed,
0:39:46 and what we see is that the hippocampus and these other networks that are involved in
0:39:52 generating these internal models of events, they show a heightened period of connectivity,
0:39:58 or correlated activity, during those breaks between different events, which we call event
0:40:03 boundaries. These are the points where you’re surprised, or you cross from one room to another,
0:40:08 and so forth, and that communication is associated with a bump of activity in the hippocampus and
0:40:13 better memory. And so if people have a very good internal model throughout that event,
0:40:21 you don’t need to do much memory processing here in a predictive mode. And so then, at these event
0:40:27 boundaries, you encode, and then you retrieve and you’re like, “Okay, wait a minute, what’s going on
0:40:31 here?” Branganath’s now talking about orcas, what’s going on, and maybe you have to go back and
0:40:36 remember reading my book to pull out the episodic memory to make sense of whatever it is I’m babbling
0:40:40 about. And so there’s this beautiful dynamics that you can see in the brain of these different
0:40:47 networks that are coming together and then de-affiliating at different points in time
0:40:52 that are allowing you to go into these modes. And so to speak to your original question,
0:40:57 to some extent, when we’re talking about semantic memory and episodic memory and working memory,
0:41:02 you can think about it as these processes that are unfolding as these networks come together
0:41:07 and pull apart. Can memory be trained and improved? This beautiful connected system that you’ve
0:41:15 described, what aspect of it is a mechanism that can be improved through training?
0:41:21 I think improvement, it depends on what your definition of optimal is. So what I say in the
0:41:27 book is that you don’t want to remember more, you want to remember better, which means focusing
0:41:33 on the things that are important. And that’s what our brains are designed to do.
0:41:37 So if you go back to the earliest quantitative studies of memory by Ebbinghaus, what you see
0:41:42 is that he was trying so hard to memorize this arbitrary nonsense. And within a day,
0:41:49 he lost about 60% of that information. And he was basically using a very, very generous way
0:41:55 of measuring it. So as far as we know, nobody has managed to violate those basics of having people
0:42:02 forget most of their experiences. So if your expectation is that you should remember everything
0:42:07 and that’s what your optimal is, you’re already off because this is just not what human brains
0:42:12 are designed to do. On the other hand, what we see over and over again is that the brain does,
0:42:18 basically, one of the cool things about the design of the brain is it’s always less is more.
0:42:23 I’ve seen estimates that the human brain uses something like 12 to 20 watts in a day. I mean,
0:42:30 that’s just nuts, the low power consumption, right? So it’s all about reusing information
0:42:36 and making the most of what we already have. And so that’s why, basically, again, what you see
0:42:43 biologically is neuromodulators, for instance, these chemicals in the brain like norepinephrine,
0:42:50 dopamine, serotonin, these are chemicals that are released during moments that tend to be
0:42:56 biologically significant, surprise, fear, stress, et cetera. And so these chemicals
0:43:03 promote lasting plasticity, right? Essentially, some mechanisms by which the brain can prioritize
0:43:10 the information that you carry with you into the future. Attention is a big factor as well,
0:43:15 our ability to focus our attention on what’s important. And so there’s different schools of
0:43:22 thought on training attention, for instance. So one of my colleagues, Amishi Ja, she wrote a
0:43:29 book called Peak Mind and talks about mindfulness as a method for improving attention and focus.
0:43:35 So she works a lot with military like Navy SEALs and stuff to do this kind of work
0:43:40 with mindfulness meditation. Adam Ghazali, in other words, my friends and colleagues,
0:43:45 has worked on kind of training through video games actually as a way of training attention. And so
0:43:51 it’s not clear to me, you know, one of the challenges though in training is you tend to
0:43:56 overfit to the thing that you’re trying to optimize, right? So you tend to, if I’m looking at a video
0:44:03 game, I can definitely get better at paying attention in the context of the video game,
0:44:07 but you transfer it to the outside world. That’s very controversial.
0:44:11 The implication there is that attention is a fundamental component of remembering something,
0:44:17 allocating attention to it. And then attention might be something that you could train,
0:44:23 how you allocate attention and how you hold attention on a thing.
0:44:28 I can say that in fact, we do in certain ways, right? So if you are an expert in something,
0:44:34 you are training attention. So we did this one study of expertise in the brain. And
0:44:40 people used to think, let’s say if you’re a bird expert or something, right? People will go like,
0:44:45 if you get really into this world of birds, you start to see the differences in your visual
0:44:50 cortex is tuned up and it’s all about plasticity of the visual cortex. And vision researchers
0:44:55 love to say everything’s visual. But it’s like, we did this study of attention and working memory
0:45:02 and expertise. And one of the things that surprised us were the biggest effects as people became
0:45:07 experts in identifying these different kinds of just crazy objects that we made up.
0:45:12 As they developed this expertise of being able to identify what made them different from each
0:45:16 other and what made them unique, we were actually seeing massive increases in activity in the
0:45:21 prefrontal cortex. And this fits with some of the studies of chest experts and so forth that
0:45:26 it’s not so much that you learn the patterns passively, you learn what to look for, you learn
0:45:32 what’s important, what’s not, right? And you can see this in any kind of expert professional athlete,
0:45:38 they’re looking three steps ahead of where they’re supposed to be. So that’s a kind of a training of
0:45:43 attention. And those are also what you’d call expert memory skills. So if you take the memory
0:45:49 athletes, I know that’s something we’re both interested in. So these are people who train in
0:45:54 these competitions and they’ll memorize like a deck of cards in like a really short amount of time.
0:46:00 There’s a great memory athlete, her name I think is pronounced Yenya Wintressol.
0:46:06 So I think she’s got like a giant Instagram following. And so she had this YouTube video
0:46:12 that went viral where she had memorized an entire Ikea catalog, right? And so how do people do this?
0:46:19 By all accounts, from people who become memory athletes, they weren’t born with some extraordinary
0:46:25 memory. But they practice strategies over and over and over again. The strategy that they use
0:46:31 for memorizing a particular thing, it can become automatic and you can just deploy it in an instant,
0:46:36 right? So again, it’s not necessarily going to, one strategy for learning the order of a deck of
0:46:42 cards might not help you for something else that you need like, you know, remembering your way around
0:46:47 Austin, Texas. But it’s going to be these, whatever you’re interested in, you can optimize for that.
0:46:54 And that’s just a natural byproduct of expertise.
0:46:58 There’s certain hacks. There’s something called the memory palace that I played with. I don’t
0:47:02 know if you’re familiar with that whole technique. And it works. It’s interesting. So another thing
0:47:08 I recommend for people a lot is I use Anki a lot every day. It’s an app that does spaced repetition.
0:47:16 So I think medical students and like students use this a lot to remember a lot of different things.
0:47:20 Oh yeah. Okay. We can come back to this. But yeah, sure. It’s the whole concept of space repetition.
0:47:24 You just, when the thing is fresh, you kind of have to remind yourself of it a lot. And then
0:47:31 over time, you can wait a week, a month, a year before you have to recall the thing again. And
0:47:38 that way, you essentially have something like note cards that you can have tens of thousands of
0:47:44 and can only spend 30 minutes a day and actually be refreshing all of that information, all that
0:47:50 knowledge. It’s really great. And then for a memory palace is a technique that allows you
0:47:56 to remember things like the IKEA catalog or by placing them visually in a place that you’re
0:48:02 really familiar with. Like I’m really familiar with this place. So I can put numbers or facts or
0:48:09 whatever you want to remember. You can walk along that little palace and it reminds you.
0:48:13 It’s cool. Like there’s stuff like that that I think athletes, memory athletes could use,
0:48:20 but I think also regular people can use. One of the things I have to solve for myself is how
0:48:24 to remember names. I’m horrible at it. I think it’s because when people introduce themselves,
0:48:30 I have the social anxiety of the interaction where I’m like, I know I should be remembering that,
0:48:39 but I’m freaking out internally about social interaction in general. And so,
0:48:45 therefore, I forget immediately. So I’m looking for good tricks for that.
0:48:49 I feel like we’ve got a lot in common because when people introduce themselves to me, it’s almost
0:48:57 like I have this blank blackout for a moment and then I’m just looking at them like, what happened?
0:49:04 I look away or something. What’s wrong with me? I’m totally with you on this.
0:49:08 The reason why it’s hard is that there’s no reason we should be able to remember names
0:49:14 because when you say remembering a name, you’re not really remembering a name. Maybe in my case,
0:49:18 you are. But most of the time, you’re associating a name with a face and an identity. And that’s a
0:49:24 completely arbitrary thing. Maybe in the olden days, somebody named Miller, it’s like they’re
0:49:30 actually making flour or something like that. But for the most part, it’s like these names are
0:49:36 just utterly arbitrary. So you have no thing to latch onto. And so it’s not really a thing that
0:49:42 our brain does very well to learn meaningless arbitrary stuff. So what you need to do is build
0:49:48 connections somehow, visualize a connection. And sometimes it’s obvious, or sometimes it’s not.
0:49:55 I’m trying to think of a good one for you now. But the first thing I think of is Lex Luthor.
0:49:59 Because doesn’t Lex Luthor wear a suit, I think? I know he has a shaved head, though,
0:50:07 or he’s bald, which you’re not. You’ve got a great head if I trade hair with you any day.
0:50:11 But something like that. But if I can come up with something, I could say, okay, so Lex Luthor
0:50:17 is this criminal mastermind, and I just imagine you. We talked about stabbing or whatever earlier.
0:50:22 Yeah, exactly. So I’m just kind of connected, and that’s it. Yeah, yeah. But I’m serious,
0:50:26 though, that these kinds of weird associations, now I’m building a richer network. I mean,
0:50:31 one of the things that I find is if you can have somebody’s name that’s just totally generic,
0:50:37 like John Smith or something, not that no offense to people with that name. But if I see a generic
0:50:43 name like that, but I’ve read John Smith’s papers academically, and then I meet John Smith at a
0:50:49 conference, I can immediately associate that name with that face, because I have this pre-existing
0:50:54 network to lock everything into. And so you can build that network. And that’s what the method
0:50:59 of loci or the memory palace technique is all about, is you have a pre-existing structure in your
0:51:05 head of like your childhood home or this mental palace that you’ve created for yourself. And so
0:51:11 now you can put arbitrary pieces of information in different locations in that mental structure of
0:51:18 yours. And then you could walk through the different path and find all the pieces of information
0:51:24 you’re looking for. So the method of loci is a great method for just learning arbitrary things,
0:51:30 because it allows you to link them together and get that cue that you need to pop in
0:51:34 and find everything, right? We should maybe linger on this memory palace thing just to make obvious,
0:51:43 because when people were describing to me a while ago what this is seems insane. I just,
0:51:50 you literally think of a place like a childhood home or a home that you’re really visually
0:51:57 familiar with. And you literally place in that three-dimensional space facts or people or whatever
0:52:08 you want to remember. And you just walk in your mind along that place visually. And you can remember,
0:52:16 remind yourself of the different things. One of the limitations is there is a sequence to it.
0:52:22 So it’s, I think your brain somehow, you need, you can’t just like go upstairs right away or
0:52:27 something. You have to like walk along the room. So it’s really great for remembering sequences,
0:52:31 but it’s also not great for remembering like individual facts out of context. So the full
0:52:36 context of the tour, I think, is important. But it’s fascinating how the mind is able to do that
0:52:42 when you ground these pieces of knowledge into something that you remember well already,
0:52:49 especially visually. Fascinating. And you can just do that for any kind of sequence. I’m sure
0:52:55 she used something like this for the IKEA catalog, something like this nature.
0:52:58 Oh yeah, absolutely, absolutely. And I think the principle here is, again, I was telling you this
0:53:05 idea that memories can compete with each other, right? Well, I like to use this example, and maybe
0:53:11 someday I’ll regret this, but I’ve used it a lot recently, is like, imagine if this were my desk,
0:53:16 it could be cluttered with a zillion different things, right? So imagine it’s just cluttered
0:53:19 with a whole bunch of yellow Post-it notes. And on one of them, I put my bank password on it, right?
0:53:24 Well, it’s going to take me forever to find it. I might, you know, it’s just going to be buried
0:53:28 under all these other Post-it notes. But if it’s like hot pink, it’s going to stand out and I find
0:53:33 it really easily, right? And so that’s one way in which if things are distinctive, if you’ve
0:53:39 processed information in a very distinctive way, then you can have a memory that’s going to last.
0:53:45 And that’s very good, for instance, for name-face associations. If I get something distinctive
0:53:51 about you, you know, that it’s like, that you’ve got very short hair, and maybe I can make the
0:53:55 association with Lex Luthor that way, or something like that, right? You know, but I get something
0:53:59 very specific. That’s a great cue. But the other part of it is, what if I just organized my notes
0:54:05 so that I have my finances in one pile, and I have my like reminders, my to-do list in one pile,
0:54:11 and so forth. So I organized them. Well, then I know exactly if I’m going for my banking,
0:54:17 you know, my bank password, I could go to the finance pile, right? So the method of loci works
0:54:23 or memory palaces work because they give you a way of organizing. There’s a school of thought that
0:54:29 says that episodic memory evolved from this like kind of knowledge of space. And, you know,
0:54:35 basically, there’s primitive abilities to figure out where you are. And so people explain the
0:54:40 method of loci that way. And, you know, whether or not the evolutionary argument is true,
0:54:46 the method of loci is not at all special. So if you don’t, you’re not a good visualizer.
0:54:50 Stories are a good one. So a lot of memory athletes will use stories, and they’ll
0:54:56 go like, if you’re memorizing a deck of cards, they have a little code for the different, like,
0:55:01 like the king and the jack and the ten and so forth. And they’ll make up a story about things
0:55:07 that they’re doing. And that’ll work. Songs are a great one, right? I mean, it’s like, I can still
0:55:12 remember there’s this obscure episode of the TV show “Cheers.” They sing a song about Albania
0:55:17 that he uses to memorize all these facts about Albania. And I could still sing that song to you.
0:55:24 It’s just I saw it on a TV show, you know. So you mentioned space repetition. So what,
0:55:29 do you like this, Prasik? Maybe can you explain it? Oh, yeah. If I’m trying to memorize something,
0:55:34 let’s say if I have an hour to memorize as many Spanish words as I can, if I just try to do,
0:55:40 like, half an hour, and then later in the day, I do half an hour, I won’t retain that information
0:55:46 as long as if I do half an hour today and half an hour one week from now. And so doing that extra
0:55:53 spacing should help me retain the information better. Now, there’s an interesting boundary condition,
0:56:00 which is it depends on when you need that information. So many of us, you know, for me,
0:56:06 like, I can’t remember so much from college and high school because I crammed because I just did
0:56:11 everything at last minute. And sometimes I would literally study, like, you know, in the hallway
0:56:17 right before the test. That was great. Because what would happen is I just had that information
0:56:22 right there. And so actually not spacing can really help you if you need it very quickly,
0:56:29 right? But the problem is, is that you tend to forget it later on. But on the other hand,
0:56:34 if you space things out, you get a benefit for later on retention. And so there’s many different
0:56:41 explanations. We have a computational model of this, it’s currently under revision. But in our
0:56:46 computer model, what we say is that an easy, maybe a good way of thinking about this is
0:56:52 this conversation that you and I are having. It’s associated with a particular context,
0:56:58 a particular place in time. And so all these little cues that are in the background, these
0:57:02 little guitar sculptures that you have, and that big light umbrella thing, right? All these things
0:57:07 are part of my memory for what we’re talking about the content. So now later on, you’re sitting around
0:57:15 and you’re at home drinking a beer and you think, God, what a strange interview that was, right?
0:57:19 So now you’re trying to remember it, but the context is different. So your current situation
0:57:27 doesn’t match up with the memory that you pulled up. There’s error. There’s a mismatch between what
0:57:32 you pulled up and your current context. And so in our model, what you start to do is you start to
0:57:37 erase or alter the parts of the memory that are associated with a specific place and time,
0:57:43 and you heighten the information about the content. And so if you remember this information in
0:57:50 different times in different places, it’s more accessible at different times in different places
0:57:56 because it’s not overfitted in a AI kind of way of thinking about things. It’s not overfitted to
0:58:01 one particular context. But that’s also why the memories that we call upon the most also feel
0:58:07 kind of like they’re just things that we read about almost. You don’t vividly reimagine them,
0:58:11 right? It’s just these things that just come to us like facts, right? And it’s a little bit
0:58:17 different than semantic memory, but basically these events that we have recalled over and over
0:58:24 and over again, we keep updating that memory so it’s less and less tied to the original experience.
0:58:29 But then we have those other ones, which it’s like you just get a reminder of that very specific
0:58:34 context. You smell something, you hear a song, you see a place that you haven’t been to in a while,
0:58:40 and boom, it just comes back to you. And that’s the exact opposite of what you get with spacing,
0:58:45 right? That’s so fascinating. So with space repetition, one of its powers is that you lose
0:58:50 attachment to a particular context, but then it loses the intensity of the flavor of the memory.
0:58:59 That’s interesting. That’s so interesting. Yeah, but at the same time, it becomes stronger
0:59:05 in the sense that the content becomes stronger. Yeah, so it’s used for learning languages,
0:59:09 for learning facts, for learning, for that generic semantic information type of memory.
0:59:14 Yeah, and I think this falls into a category we’ve done other modeling. One of these is
0:59:20 published study in PLOS, Computational Biology, where we showed that another way, which is I think
0:59:27 related to the spacing effect is what’s called the testing effect. So the idea is that if you’re
0:59:33 trying to learn words, let’s say in Spanish or something like that, and this doesn’t have to
0:59:38 be words, it could be anything, you test yourself on the words and that active testing yourself
0:59:44 helps you retain it better over time than if you just studied it, right? And so from traditional
0:59:51 learning theories, some learning theories anyway, this seems weird, why would you do better giving
0:59:57 yourself this extra error from testing yourself rather than just giving yourself perfect input
1:00:03 that’s a replica of what it is that you’re trying to learn. And I think the reason is that you get
1:00:08 better retention from that error, that mismatch that we talked about, right? So what’s happening
1:00:15 in our model, it’s actually conceptually kind of similar to what happens with backprop in AI,
1:00:21 so there are neural networks. And so the idea is that you expose, here’s the bad connections,
1:00:26 and here’s the good connections. And so we can keep the parts of this cell assembly that are
1:00:32 good for the memory and lose the ones that are not so good. But if you don’t stress test the
1:00:37 memory, you haven’t exposed it to the error fully. And so that’s why I think this is a thing that I
1:00:43 come back to over and over again, is that you will retain information better if you’re constantly
1:00:50 pushing yourself to your limit, right? If you are feeling like you’re coasting, then you’re actually
1:00:57 not learning. So you should always be stress testing the memory system. Yeah, and feel good
1:01:06 about it. You know, even though everyone tells me, oh, my memory is terrible, in the moment,
1:01:10 they’re overconfident about what they’ll retain later on. So it’s fascinating. And so what happens
1:01:16 is when you test yourself, you’re like, oh, my God, I thought I knew that, but I don’t. And so
1:01:22 it can be demoralizing until you get around that and you realize, hey, this is the way that I learn.
1:01:28 This is how I learn best. It’s like if you’re trying to, you know, star in a movie or something
1:01:36 like that, you know, just sit around reading the script, you actually act it out and you’re going
1:01:40 to botch those lines from time to time, right? You know that there’s an interesting moment you
1:01:43 probably experienced this. I remember a good friend of mine, Joe Rogan, I was on his podcast and
1:01:50 we were randomly talking about soccer football. Somebody I grew up watching, Diego Armando
1:02:00 Maradona, one of the greatest soccer players of all time. And we were talking about him and his
1:02:05 career and so on. And Joe asked me if he’s still around. Now, and I said, yeah, I don’t know why
1:02:19 I thought yeah, because that was a perfect example of memories. He passed away. I tweeted about it,
1:02:27 how heartbroken I was, all this kind of stuff. I like it a year before. I know this, but in my
1:02:33 mind, I went back to the thing I’ve done many times in my head, visualizing some of the epic
1:02:39 run to get on goal and so on. So for me, he’s alive. So I’m in part of the also the conversation
1:02:45 when you’re talking to Joe, there’s stress and the focus is allocated, the attention is allocated
1:02:50 in a particular way. But when I walked away, I was like, in which world was Diego Maradona
1:02:58 still alive? Because I was sure in my head that he was still alive. There was a moment that sticks
1:03:06 with me. I’ve had a few like that in my life where just obvious things just disappear from
1:03:14 mind. And it’s cool, like it shows actually the power of the mind in a positive sense
1:03:19 to erase memories you want erased, maybe. But I don’t know. I don’t know if there’s a good
1:03:24 explanation for that. One of the cool things that I found is that some people really just
1:03:31 revolutionize a field by creating a problem that didn’t exist before. It’s kind of like why I love
1:03:38 science is like engineering is like solving other people’s problems and science is about
1:03:43 creating problems. I’m just much more like I want to break things and create problems.
1:03:48 Not necessarily move fast though. But one of my former mentors, Marsha Johnson, who in my opinion
1:03:55 is one of the greatest memory researchers of all time, she comes up young woman in the field and
1:04:00 it’s mostly Guy Field. And she gets into this idea of how do we tell the difference between
1:04:06 things that we’ve imagined and things that we actually remember? How do we tell I get some
1:04:11 mental experience? Where did that mental experience come from? And it turns out this is a huge problem
1:04:17 because essentially our mental experience of remembering something that happened,
1:04:21 our mental experience of thinking about something, how do you tell the difference?
1:04:26 They’re both largely constructions in our head. And so it is very important. And the way that you
1:04:34 do it is, I mean, it’s not perfect, but the way that we often do it and succeed is by, again,
1:04:41 using our prefrontal cortex and really focusing on the sensory information or the place and time
1:04:48 and the things that put us back into when this information happened. And if it’s something you
1:04:53 thought about, you’re not going to have all of that vivid detail as you do for something that
1:04:57 actually happened. But it doesn’t work all the time. But that’s a big thing that you have to do.
1:05:02 But it takes time. It’s slow and it’s, again, effortful. But that’s what you need to remember
1:05:07 accurately. But what’s cool, and I think this is what you alluded to about how that was an
1:05:11 interesting experience, is imagination is exactly the opposite. Imagination is basically saying,
1:05:18 I’m just going to take all this information from memory, recombine it in different ways and throw
1:05:23 it out there. And so, for instance, Dan Schachter and Donna Addis have done cool work on this.
1:05:29 Demis Hassibis did work on this with Eleanor McGuire and UCL. And this goes back, actually,
1:05:36 to this guy, Frederick Bartlett, who is this revolutionary memory researcher at Bartlett.
1:05:41 He actually rejected the whole idea of quantifying memory. He said, “There’s no statistics in my
1:05:47 book.” He came from this anthropology perspective. And a short version of the story is he just asked
1:05:53 people to recall things. You give people stories in poem, ask people to recall them.
1:05:58 And what he found was people’s memories didn’t reflect all of the details of what they were
1:06:03 exposed to. And they did reflect a lot more. They were filtered through this lens of prior
1:06:09 knowledge, the cultures that they came from, the beliefs that they had, the things they knew.
1:06:14 And so, what he concluded was that he called remembering an imaginative construction,
1:06:20 meaning that we don’t replay the past. We imagine how the past could have been by taking bits and
1:06:27 pieces that come up in our heads. And likewise, he wrote this beautiful taper on imagination,
1:06:32 saying when we imagine something and create something, we’re creating it from these specific
1:06:37 experiences that we’ve had and combining it with our general knowledge. But instead of trying to
1:06:41 focus it on being accurate and getting out one thing, you’re just ruthlessly recombining things
1:06:46 without any necessary goal in mind. I mean, at least that’s one kind of creation.
1:06:54 So imagination is fundamentally coupled with memory in both directions?
1:07:02 I think so. I mean, it’s not clear that it is in everyone, but one of the things that’s been
1:07:09 studied is some patients who have amnesia, for instance, they have brain damage, say, to the
1:07:14 hippocampus. And if you ask them to imagine things that are not in front of them, like imagine what
1:07:20 could happen after I leave this room, they find it very difficult to give you a scenario of what
1:07:27 could happen. Or if they do, it would be more stereotyped, like, yes, this would happen. But
1:07:31 it’s not like they can come up with anything that’s very vivid and creative in that sense.
1:07:35 And it’s partly because when you have amnesia, you’re stuck in the present.
1:07:39 Because to get a very good model of the future, it really helps to have episodic memories to
1:07:45 draw upon, right? And so that’s the basic idea. And in fact, one of the most impressive things,
1:07:52 when people started to scan people’s brains and ask people to remember past events,
1:07:58 what they found was there was this big network of the brain called the default mode network.
1:08:03 It gets a lot of press because it’s like thought to be important. It’s engaged during
1:08:06 mind wandering. And if I ask you to pay attention to something, it only comes on when you stop paying
1:08:12 attention. So people go, oh, it’s this daydreaming network. And I thought this is just ridiculous
1:08:18 research. Who cares? But then what people found was when people recall episodic memories,
1:08:25 this network gets active. And so we started to look into it. And this network of areas is really
1:08:32 closely, functionally interacting with the hippocampus. And so, in fact, some would say the
1:08:38 hippocampus is part of this default network. And if you look at brain images of people,
1:08:44 or brain maps of activation, so to speak, of people imagining possible scenarios of things that
1:08:50 could happen in the future, or even things that couldn’t really be very plausible,
1:08:53 they look very similar. I mean, you know, to the naked eye, they look almost the same as maps of
1:08:59 brain activation when people remember the past. According to our theory, and we’ve got some data
1:09:04 to support this, we’ve broken up this network in various subpieces, is that basically it’s kind of
1:09:09 taking apart all of our experiences, and creating these little Lego blocks out of them. And then
1:09:15 you can put them back together if you have the right instructions to recreate these experiences
1:09:20 that you’ve had, but you could also reassemble them into new pieces to create a model of an event that
1:09:25 hasn’t happened yet. And that’s what we think happens. And when our common ground that we’re
1:09:31 establishing in language, requires using those building blocks to put together a model of what’s
1:09:37 going on. Well, there’s a good percentage of time I personally live in, in the imagined world.
1:09:43 I think of, I have, I do thought experiments a lot. I, you know, take the, the absurdity
1:09:50 of human life as it stands, and play it forward in all kinds of different directions.
1:10:00 Sometimes it’s rigorous thoughts, thought experiments, sometimes it’s fun ones. So
1:10:03 I imagine that that has an effect on how I remember things. And I suppose I have to be
1:10:10 a little bit careful to make sure stuff happened versus stuff that I just imagined happened. And
1:10:17 this also, I mean, some of my best friends are characters inside books that never even existed.
1:10:24 And I’m, you know, there’s some degree to which they actually exist in my mind.
1:10:30 Like these characters exist. Authors exist. Does Efsky exist, but also Brothers Karamazov.
1:10:37 I love that book. One of the few books I’ve read. One of the few literature books that I’ve read,
1:10:43 I should say. I read a lot in school that I don’t remember, but Brothers Karamazov.
1:10:47 But they exist. They exist. And I have almost kind of like conversations with them. It’s interesting.
1:10:51 It’s interesting to allow your brain to kind of play with ideas of the past,
1:10:58 of the imagined, and see it all as one.
1:11:00 Yeah. There was actually this famous mnemonist. He’s kind of like back then the equivalent of a
1:11:06 memory athlete, except he would go to shows and do this. Those described by this really famous
1:11:12 nurse psychologist from Russia named Luria. And so this guy was named Solomon Sharyshevsky.
1:11:18 And he had this condition called synesthesia that basically created these weird associations
1:11:24 between different senses that normally wouldn’t go together. So that gave him this incredibly
1:11:30 vivid imagination that he would use to basically imagine all sorts of things that he would need
1:11:38 to memorize. And he would just imagine, like just create these incredibly detailed things.
1:11:43 And he said that allowed him to memorize all sorts of stuff. But it also really haunted
1:11:49 him by some reports that basically it was like he was at some point, and again, who knows,
1:11:54 the drinking was part of this, but at some point had trouble differentiating his imagination from
1:11:59 reality. And this is interesting because it’s like, I mean, that’s what psychosis is in some
1:12:06 ways is you, first of all, you’re just learning connections from prediction errors that you
1:12:13 probably shouldn’t learn. And the other part of it is, is that your internal signals are being
1:12:19 confused with actual things in the outside world, right? Well, that’s why a lot of this stuff is
1:12:25 both feature and bug. It’s a double-edged sword. Yeah, I mean, it might be why there’s such an
1:12:29 interesting relationship between genius and psychosis. Yeah, maybe they’re just two sides of
1:12:36 the same coin. Humans are fascinating, aren’t they? I think so. Sometimes scary, but mostly
1:12:43 fascinating. Can we just talk about memory sport a little longer? There’s something called the USA
1:12:49 Memory Championship. What are these athletes like? What does it mean to be like elite level at this?
1:12:57 Have you interacted with any of them or reading about them? What have you learned about these
1:13:01 folks? There’s a guy named Henry Ronditer who’s studying these guys. And there’s actually a book
1:13:06 by Joshua Ford called Moonwalking with Einstein where he talks about, he actually, as part of this
1:13:12 book, just decided to become a memory athlete. They often have these life events that make them go,
1:13:18 “Hey, why don’t I do this?” So there was a guy named Scott Hagwood who I write about,
1:13:23 who thought that he was getting chemo for cancer. And so he decided, because chemo,
1:13:32 there’s a well-known thing called chemo brain where people become like they just lose a lot of
1:13:37 their sharpness. And so he wanted to fight that by learning these memory skills. So he bought a
1:13:44 book. And this is the story you hear in a lot of memory athletes is they buy a book by other memory
1:13:49 athletes or other memory experts, so to speak. And they just learn those skills and practice
1:13:55 them over and over again. They start by winning bets and so forth. And then they go into these
1:14:00 competitions. And the competitions are typically things like memorizing long strings of numbers
1:14:05 or memorizing orders of cards and so forth. So there tend to be pretty arbitrary things,
1:14:11 not like things that you’d be able to bring a lot of prior knowledge. But they build the skills that
1:14:18 you need to memorize arbitrary things. Yeah, that’s fascinating. I’ve gotten a chance to work
1:14:23 with something called n-back tasks. So there’s all these kinds of tasks, memory recall tasks that
1:14:28 are used to load up the quote-unquote “work in memory.” And to see the psychologists used it
1:14:36 to test all kinds of stuff, like to see how well you’re good at multitasking. We used it in particular
1:14:41 for the task of driving, like if you fill up your brain with intensive working memory tasks,
1:14:49 how good are you at also not crashing, that kind of stuff. So it’s fascinating. But
1:14:56 again, those tasks are arbitrary in there, usually about recalling a sequence of numbers in some kind
1:15:02 of semi-complex way. Do you have any favorite tasks of this nature in your own studies?
1:15:10 I’ve really been most excited about going in the opposite direction and using things that are
1:15:16 more and more naturalistic. And the reason is, is that we’ve really moved, we’ve moved in that
1:15:22 direction because what we found is that memory works very, very differently when you study it,
1:15:29 when you study memory in the way that people typically remember. And so it goes into a much
1:15:36 more predictive mode. And you have these event boundaries, for instance. But a lot of what
1:15:43 happens is this kind of fascinating mix that we’ve been talking about, a mix of interpretations
1:15:49 and imagination with perception. And so the new direction we’re going in is understanding
1:15:57 navigation in our memory first places. And the reason is, is that there’s a lot of work that’s
1:16:02 done in rats, which is very good work. They have a rat, and they put it in a box, and the rat goes,
1:16:08 chases cheese in a box, and you’ll find cells in the hippocampus that fire when a rat is in
1:16:13 different places in the box. And so the conventional wisdom is that the hippocampus forms this map
1:16:21 of the box. And I think that probably may happen when you have absolutely no knowledge
1:16:29 of the world, right? But I think one of the cool things about human memory is we can bring to bear
1:16:35 our past experiences, economically, learn new ones. And so, for instance, if you learn a map of an
1:16:43 Ikea, let’s say if I go to the Ikea in Austin, I’m sure there’s one here, I probably could go to
1:16:49 this Ikea and find my way to the, you know, where the wine glasses are without having to even think
1:16:56 about it, because it’s got a very similar layout, even though Ikea is a nightmare to get around.
1:17:00 Once I learned my local Ikea, I can use that map everywhere, why form a brand new one for a new
1:17:06 place. And so that kind of ability to reuse information really comes into play when we
1:17:14 look at things that are, you know, more naturalistic tasks. And another thing that we’re really
1:17:21 interested in is this idea of like, what if instead of basically mapping out every coordinate in a
1:17:27 space, you form a pretty economical graph that connects basically the major landmarks together
1:17:35 and being able to use that as, you know, emphasizing the things that are most important,
1:17:40 the places that you go for food and the places that are landmarks that help you get around.
1:17:45 And then filling in the blanks for the rest, because I really believe that cognitive maps
1:17:51 are a mental maps of the world, just like our memories for events are not photographic,
1:17:57 I think there’s this combination of actual verifiable details and then a lot of inference
1:18:03 that you make. So what have you learned about this kind of spatial mapping of places?
1:18:09 How do people represent locations? There’s a lot of variability, I think that, and there’s a lot
1:18:15 of disagreement about how people represent locations in a world of GPS and physical maps.
1:18:22 People can learn it from like basically what they call like survey perspective, being able to see
1:18:27 everything. And so that’s one way in which humans can do it. That’s a little bit different.
1:18:32 There’s one way which we can memorize routes. Like I know how to get from here to, let’s say,
1:18:39 if I knew, walk here from my hotel, I could just rigidly follow that route back, right? And there’s
1:18:44 another more integrative way, which would be what’s called a cognitive map, which would be
1:18:50 kind of a sense of how everything relates to each other. And so there’s lots of people who
1:18:56 believe that these maps that we have in our head are isomorphic with the world. They’re like these
1:19:02 literal coordinates that follow Euclidean space. And as you know, Euclidean mathematics is very
1:19:09 constrained, right? And I think that we are actually much more generative in our maps of space,
1:19:16 so that we do have these bits and pieces. And we’ve got a small task as it’s right now,
1:19:21 not yet like we need to do some work on it for further analyses. But one of the things we’re
1:19:27 looking at is these signals called ripples in the hippocampus, which are these bursts of activity
1:19:34 that you see that are synchronized with areas in the neocortex, in the default network, actually.
1:19:40 And so what we find is that those ripples seem to increase at navigationally important points
1:19:46 when you’re making a decision or when you reach a goal. So it speaks to the emotion thing, right?
1:19:51 Because if you have limited choices, if I’m walking down a street, I could really just get
1:19:58 a mental map of the neighborhood with a more minimal kind of thing by just saying, here’s the
1:20:02 intersections, and here’s the directions I take to get in between them. And what we found in general
1:20:08 in our MRI studies is basically the more people can reduce the problem, whether it’s space or
1:20:16 any kind of decision-making problem, the less the hippocampus encodes. It really is very economical
1:20:23 towards the points of most highest information content and value. So can you describe
1:20:30 the encoding in the hippocampus and the ripples you were talking about?
1:20:33 What’s the signal in which we see the ripples? Yeah, so this is really interesting. There are
1:20:40 these oscillations, right? So there’s these waves that you basically see. And these waves
1:20:46 are points of very high excitability and low excitability. And at least during, they happen
1:20:53 actually during slow wave sleep too. So the deepest stages of sleep when you’re just zonked out, right?
1:20:58 You see these very slow waves where it’s very excitable and then very unexcitable. It goes up
1:21:03 and down. And on top of them, you’ll see these little sharp wave ripples. And when there’s a
1:21:09 ripple in the hippocampus, you tend to see a sequence of cells that resemble a sequence of
1:21:16 cells that fire when an animal is actually doing something in the world. So it almost is like a
1:21:22 little people call it replay. And it’s a little bit, I don’t like that term, but it’s basically
1:21:28 a little bit of a compressed play of the sequence of activity in the brain that was taking place
1:21:35 earlier. And during those moments, there’s a little window of communication between the hippocampus
1:21:41 and these areas in the neocortex. And so that I think helps you form new memories, but it also
1:21:48 helps you, I think, stabilize them, but also really connect different things together in memory
1:21:53 and allows you to build bridges between different events that you’ve had. And so this is one of
1:21:59 hardly start theories of sleep and its real role in helping you see the connections between
1:22:05 different events that you’ve experienced. Oh, so during sleep is when the connections are formed?
1:22:09 The connections between different events, right? So it’s like, you see me now, you see me next week,
1:22:16 you see me a month later, you start to build a little internal model of how I behave and what
1:22:23 to expect of me. And with sleep, one of the things that allows you to do is figure out those
1:22:29 connections and connect the dots and find the signal and the noise. So you mentioned fMRI,
1:22:35 what is it and how is it used in studying memory? This is actually the reason why I got into this
1:22:41 whole field of science is when I was in grad school, fMRI was just really taking off as a
1:22:48 technique for studying brain activity. And what’s beautiful about it is you can study the whole
1:22:53 human brain. And there’s lots of limits to it, but you can basically do it in the person without
1:23:00 sticking anything into their brains. And very noninvasive. And for me, being an MRI scanner is
1:23:06 like being in the womb, I just fall asleep. If I’m not being asked to do anything, I get very
1:23:10 sleepy. But you can have people watch movies while they’re being scanned, or you can have them do
1:23:17 tests of memory, like giving them words and so far to memorize. But what MRI is itself is just this
1:23:24 technique where you put people in a very high magnetic field. Typical ones we would use would be
1:23:31 three Tesla to give you an idea. So a three Tesla magnet, you put somebody in. And what happens is
1:23:37 you get this very weak, but, you know, measurable magnetization in the brain. And then you apply
1:23:44 a radiofrequency pulse, which is basically a different electromagnetic field. And so you’re
1:23:49 basically using water, the water molecules in the brain as a tracer, so to speak. And part of it in
1:23:56 fMRI is the fact that these magnetic fields that you mess with by manipulating these radiofrequency
1:24:06 pulses in the static field, and you have things called gradients would change the strength of
1:24:11 the magnetic field in different parts of the head. So they’re all, we tweak them in different ways.
1:24:16 But the basic idea that we use in fMRI is that blood is flowing to the brain. And when you have
1:24:23 blood that doesn’t have oxygen on it, it’s a little bit more magnetizable than blood that does,
1:24:28 because you have hemoglobin that carries the oxygen, the iron basically in the blood that makes
1:24:33 it red. And so that hemoglobin when it’s deoxygenated actually has different magnetic field properties
1:24:42 than when it has oxygen. And it turns out when you have an increase in local activity in some part
1:24:48 of the brain, the blood flows there. And as a result, you get a lower concentration of hemoglobin
1:24:56 that is not oxygenated. And then that gives you more signal. So I gave you, I think I sent you
1:25:05 a GIF, as you like to say. Yeah, we had off record, intense argument. Well, if it’s pronounced GIF
1:25:13 or GIF, but that’s, we shall set that aside as friends. We could have called it a stern rebuke,
1:25:18 perhaps. Rebuke, yeah. I drew a hard line. It is through the creator of GIFs that it’s
1:25:26 pronounced GIF, but that’s the only person that pronounces GIF. Anyway, yes, you sent a GIF of…
1:25:33 This would be basically a whole, a movie of fMRI data. And so when you look at it, it’s not very
1:25:40 impressive. It looks like these like very pixelated maps of the brain, but it’s mostly kind of like
1:25:45 white. But these tiny changes in the intensity of those signals that you probably wouldn’t be
1:25:51 able to visually perceive, like about 1% can be statistically very, very large effects for us.
1:25:58 And that allows us to see, hey, there’s an increase in activity in some part of the brain
1:26:02 when I’m doing some task like trying to remember something. And I can use those changes to even
1:26:09 predict is a person going to remember this later or not. And the coolest thing that people have done
1:26:15 is to decode what people are remembering from the patterns of activity from, because maybe when
1:26:22 I’m remembering this thing, like I’m remembering the house where I grew up, I might have one pixel
1:26:29 that’s bright in the hippocampus and one that’s dark. And if I’m remembering, you know, something
1:26:34 like more like the car that I used to drive when I was 16, I might see the opposite pattern where
1:26:39 different pixels bright. And so all that little stuff that we used to think of noise, we can now
1:26:45 think of almost like a QR code for memory, so to speak, where different memories have a different
1:26:50 little pattern of bright pixels and dark pixels. And so this really revolutionized my research.
1:26:55 So there’s fancy research out there where people really, I mean, not even that, I mean,
1:27:00 by your standards would be stone age, but you know, applying machine learning techniques to do decoding
1:27:05 and so forth. And now there’s a lot of forward encoding models and you can go to town with this
1:27:10 stuff, right? And I’m much more old school of designing experiments where you basically say,
1:27:17 okay, here’s a whole web of memories that overlap in some way, shape, or form. Do memories that
1:27:25 occurred in the same place have a similar QR code? And do memories that occurred in different
1:27:30 places of different QR code? And you can just use things like correlation coefficients or
1:27:35 cosine distance to measure that stuff, right? Super simple, right? And so what happens is you
1:27:41 can start to get a whole state space of how a brain area is indexing all these different memories.
1:27:47 And it’s super fascinating because what we could see is this little like separation between how
1:27:53 certain brain areas are processing memory for who was there and other brain areas are processing
1:27:58 information about where it occurred or the situation that’s kind of unfolding. And some
1:28:03 are giving you information about what are my goals that are involved and so forth. And so,
1:28:09 and the hippocampus is just putting it all together into these unique things that just
1:28:13 are about when and where it happened. So there’s a separation between spatial information,
1:28:20 concepts, like literally there’s distinct, as you said, QR codes for these.
1:28:28 So to speak, let me try a different analogy to that might be more accessible for people,
1:28:33 which would be like, you’ve got a folder on your computer, right? Open it up. There’s a bunch
1:28:37 of files there. I can sort those files by, you know, alphabetical order. And now things that
1:28:44 both start with letter A are lumped together and things that start with Z versus A are far apart,
1:28:50 right? And so that is one way of organizing the folder, but I could do it by date. And if I do
1:28:55 it by date, things that were created close together in time are close and things that are
1:29:00 far apart in time are far. So every, like you can think of how a brain area or a network of areas
1:29:07 contributes to memory by looking at what the sorting scheme is. And these QR codes that we’re
1:29:14 talking about that you get from fMRI allow you to do that. And you can do the same thing if
1:29:18 you’re recording from massive populations of neurons in an animal. And you can do it for
1:29:25 recording local potentials in the brain, you know, so little waves of activity in, let’s say,
1:29:32 a human who has epilepsy and they stick electrodes in their brain and try to find the seizures.
1:29:37 So that’s some of the work that we’re doing now. But all these techniques basically allow you to
1:29:42 say, hey, what’s the sorting scheme? And so we’ve found that some networks of the brain sort
1:29:48 information and memory according to who was there. So I might have, like we’ve actually shown in one
1:29:54 of my favorite studies of all time that was done by a former postdoc, Zach Raim. And Zach did the
1:30:00 study where we had a bunch of movies with different people in my labs that are two different people.
1:30:05 And you filmed them at two different cafes and two different supermarkets. And what you could
1:30:11 show is in one particular network, you could find the same kind of pattern of activity more
1:30:17 or less, a very, very similar pattern of activity. Every time I saw Alex in one of these movies,
1:30:23 no matter where he was, right? And I could see another one that was like a common
1:30:28 pattern that happened every time I saw this particular supermarket nugget, you know. And it
1:30:36 didn’t matter whether you’re watching a movie or whether you’re recalling the movie. It’s the same
1:30:40 kind of pattern that comes up, right? It’s so fascinating. It’s fascinating. So now you have
1:30:45 those building blocks for assembling a model of what’s happening in the present, imagining what
1:30:51 could happen and remembering things very economically from putting together all these pieces so that
1:30:56 all the hippocampus has to do is get the right kind of blueprint for how to put together all
1:31:02 these building blocks. These are all like beautiful hints at a super interesting system that makes
1:31:09 me wonder on the other side of it how to build it. But it’s like, it’s fascinating. Like the way
1:31:15 does the encoding is really, really fascinating. Or I guess the symptoms, the results of that
1:31:21 encoding are fascinating to study from this. Just as a small tangent, you mentioned sort of the
1:31:26 measuring local potentials with electrodes versus fMRI. Oh yeah. What are some interesting like
1:31:34 limitations, possibilities of fMRI? Maybe the way you explained it is brilliant with blood and
1:31:41 detecting the activations or the excitation because blood flows to that area. What’s the
1:31:49 latency of that? What’s the blood dynamics in the brain that, how quickly can it,
1:31:55 how quickly can the tasks change and all that kind of stuff? Yeah. I mean, it’s very slow to
1:32:02 the brain. 50 milliseconds is like, it’s an eternity. Maybe 50, maybe like
1:32:10 let’s say half a second, 500 milliseconds. Just so much back and forth stuff happens
1:32:17 in the brain in that time. In fMRI, you can measure these magnetic field responses
1:32:24 about six seconds after that burst of activity would take place. All these things, it’s like,
1:32:30 is it a feature or is it a bug? One of the interesting things that’s been discovered
1:32:34 about fMRI is it’s not so tightly related to the spiking of the neurons. We tend to think of
1:32:42 the computation, so to speak, as being driven by spikes, meaning like there’s just a burst of,
1:32:48 it’s either on or it’s off and the neuron’s like going up or down. But sometimes what you can have
1:32:54 is these states where the neuron becomes a little bit more excitable or less excitable.
1:33:00 And so fMRI is very sensitive to those changes in excitability. Actually, one of the fascinating
1:33:06 things about fMRI is where does that, how is it we go from neural activity to essentially
1:33:15 blood flow to oxygen, all this stuff. It’s such a long chain of going from neural activity to
1:33:22 magnetic fields. And one of the theories that’s out there is most of the cells in the brain are
1:33:28 not neurons. They’re actually these support cells called glial cells. And one big one is astrocytes
1:33:34 and they play this big role in regulating kind of being a middle man, so to speak, with the neuron.
1:33:40 So if you, for instance, like one neuron is talking to another, you release a neurotransmitter,
1:33:45 like let’s say glutamate, and that gets another neuron starts getting active after you release
1:33:51 in the gap between the two neurons called synapse. So what’s interesting is if you leave that,
1:33:58 imagine you’re just flooded with this liquid in there, right? If you leave it in there too long,
1:34:03 you just excite the other neuron too much and you can start to basically get seizure activity.
1:34:07 You don’t want this. So you got to suck it up. And so actually what happens is these astrocytes,
1:34:12 one of their functions is to suck up the glutamate from the synapse. And that is a massively, and
1:34:19 then break it down and then feed it back into the neurons that you reuse it. But that cycling is
1:34:25 actually very energy intensive. And what’s interesting is at least according to one theory,
1:34:30 and they need to work so quickly that they’re working on metabolizing the glucose that comes
1:34:36 in without using oxygen, kind of like what, you know, anaerobic metabolism. So they’re not using
1:34:42 oxygen as fast as they are using glucose. So what we’re really seeing in some ways may be in fMRI,
1:34:52 not the neurons themselves being active, but rather the astrocytes, which are meeting the
1:34:58 metabolic demands of the process of keeping the whole system going.
1:35:02 It does seem to be that fMRI is a good way to study activation. So with these astrocytes,
1:35:08 even though there’s a latency, it’s pretty reliably coupled to the activations.
1:35:16 Oh, well, this gets me to the other part. So now let’s say, for instance,
1:35:20 if I’m just kind of like I’m talking to you, but I’m kind of paying attention to your cowboy hat,
1:35:24 right? So I’m looking off to the room. I’m thinking about the right, even if I’m not looking at it.
1:35:28 What you’d see is that there would be this little elevation in activity in areas in
1:35:35 the visual cortex, which process vision around that point in space. Okay. So if then something
1:35:43 happened, like, you know, suddenly a light flashed in that part of, you know, right in front of your
1:35:48 cowboy hat, I would have a bigger response to it. But what you see in fMRI is even if I’m not,
1:35:54 even if I don’t see that flash of light, there’s a lot of activity that I can measure,
1:35:58 because you’re kind of keeping it excitable and that in and of itself, even though I’m not
1:36:03 seeing anything there that’s particularly interesting, there’s still this increase in
1:36:08 activity. And so it’s more sensitive with fMRI. So is that a feature or is it a bug? You know,
1:36:12 some people, people who study spikes in neurons would say, well, that’s terrible. We don’t want
1:36:17 that, you know. Likewise, it’s slow. And that’s terrible for measuring things that are very fast.
1:36:23 But one of the things that we found in our work was when we give people movies and when we give
1:36:29 people stories to listen to, a lot of the action is in the very, very slow stuff. It’s in, because
1:36:36 if you’re thinking about, like, a story, let’s say, you’re listening to a podcast or listening
1:36:42 to the Lex Friedman podcast, right? You’re putting this stuff together and building this
1:36:46 internal model over several seconds, which is basically, we filter that out when we look at
1:36:51 electrical activity in the brain, because we’re interested in this millisecond scale. It’s almost
1:36:55 massive amounts of information, right? So the way I see it is every technique gives you a little
1:37:02 limited window into what’s going on. fMRI is huge problems. You know, people lie down in the scanner.
1:37:08 There’s parts of the brain where I’ll show you in some of these images where you’ll see kind of
1:37:13 gaping holes, because you can’t keep the magnetic field stable in those spots. You’ll see parts where
1:37:20 it’s like there’s a vein, and so it just produces big increases and decreases in signal or respiration
1:37:26 that causes these changes. There’s lots of artifacts and stuff like that. Every technique has
1:37:31 its limits. If I’m lying down in an MRI scanner, I’m lying down. I’m not interacting with you
1:37:36 in the same way that I would in the real world. But at the same time, I’m getting data that I
1:37:43 might not be able to get otherwise. And so different techniques give you different kinds of
1:37:47 advantages. What kind of big scientific discoveries, maybe the flavor of discoveries have been done
1:37:52 throughout the history of the science of memory, the studying of memory? What kind of things
1:37:59 have been understood? Oh, there’s so many. It’s really so hard to summarize it. I mean, I think
1:38:08 it’s funny because it’s like, when you’re in the field, you can get kind of blasé about this stuff.
1:38:13 But then once I started to write the book, I was like, oh my god, this is really interesting. How
1:38:17 did we do all this stuff? I would say that some of the, I mean, from the first studies, just showing
1:38:26 how much we forget is very important. Showing how much schemas, which is our organized knowledge
1:38:33 about the world, increase our ability to remember information, just massively increase in studies
1:38:41 of expertise, showing how experts like chess experts can memorize so much in such a short
1:38:46 amount of time because of the schemas they have for chess. But then also showing that those lead
1:38:52 to all sorts of distortions in memory, the discovery that the act of remembering can change the memory,
1:38:59 can strengthen it, but it can also distort it if you get misinformation at the time.
1:39:04 And it can also strengthen or weaken other memories that you didn’t even recall. So just
1:39:10 this whole idea of memory as an ecosystem, I think, was a big discovery. I could go, this idea of
1:39:17 like breaking up our continuous experience into these discrete events, I think, was a major
1:39:23 discovery. So the discreteness of our encoding of events? Maybe, yeah. I mean, you know, and again,
1:39:28 there’s controversial ideas about this, right? But it’s like, yeah, this idea that, and this gets
1:39:33 back to just this common experience of you walk into the kitchen, and you’re like, why am I here?
1:39:38 And you just end up grabbing some food from the fridge, and then you go back and you’re like,
1:39:42 oh, wait a minute, I left my watch in the kitchen. That’s what I was looking for.
1:39:45 And so what happens is, is that you have a little internal model of where you are,
1:39:50 what you’re thinking about. And when you cross from one room to another, those models get updated.
1:39:56 And so now when you’re in the kitchen, you have to go back and mentally time travel back to this
1:40:00 earlier point to remember what it was that you went there for. And so these event boundaries,
1:40:06 turns out like in our research, and again, I don’t want to make it sound like we’ve figured out
1:40:11 everything. But in our research, one of the things that we found is that basically, as people get
1:40:18 older, the activity in the hippocampus at these event boundaries tends to go down. But independent
1:40:27 of age, if I give you outside of the scanner, you’re done with the scanner, I just scan you
1:40:31 while you’re watching a movie, you just watch it, you come out, I give you a test of memory for
1:40:35 stories. What happens is you find this incredible correlation between the activity in the hippocampus
1:40:43 at these singular points in time, these event boundaries, and your ability to just remember
1:40:49 a story outside of the scanner later on. So it’s marking this ability to encode memories,
1:40:54 just these little snippets of neural activity. So I think that’s a big one. There’s all sorts of
1:41:00 work in animal models that I can get into, you know, sleep. I think there’s so much interesting
1:41:06 stuff that’s being discovered in sleep right now, being able to just record from large populations
1:41:14 of cells, and then be able to relate that. And I think the coolest thing gets back to this QR
1:41:20 code thing, because what we can do now is I can take fMRI data while you’re watching a movie,
1:41:27 or let’s do better than that. Let me get fMRI data while you use a joystick to move around in
1:41:32 virtual reality, right? You’re in the metaverse, whatever, right? But it’s kind of a crappy
1:41:36 metaverse, because there’s always so much metaversing you can do in an MRI scanner. So
1:41:40 what you do is crappy metaversing. So now I can take a rat, record from his hippocampus,
1:41:46 and prefrontal cortex in all these areas, with these really new electrodes, get massive amounts
1:41:51 of data, and have it move around on a trackball in virtual reality in the same metaverse that I
1:41:58 did, and record that rat’s activity. I can get a person with epilepsy, who we have electrodes in
1:42:04 their brain anyway, to try to figure out where the seizures are coming from. And it’s a healthy
1:42:08 part of the brain, record from that person, right? And I can get a computational model.
1:42:14 And one of the one of the brand new members in my lab, Tyler Bond, is just doing some great
1:42:19 stuff. He relates computer vision models, and looks at the weaknesses of computer vision models,
1:42:25 and relates it to what the brain does well. And so you can actually take a ground truth,
1:42:32 code for the metaverse, basically. And you can feed in the visual information, let’s say the
1:42:40 sensory information, or whatever that’s coming in, to a computational model that’s designed to take
1:42:47 real world inputs, right? And you could basically tie them all together by virtue of the state spaces
1:42:54 that you’re measuring in neural activity, and these different formats, and these different
1:42:58 species, and in the computational model, which is, I just find that mind blowing. You could do
1:43:04 different kinds of analyses on language, and basically come up with just like, basically,
1:43:09 it’s the guts of LLMs, right? You could do analyses on language, and you could do analyses on
1:43:18 sentiment analyses of emotions, and put all this stuff together. I mean, it’s almost too much.
1:43:25 But if you do it right, and you do it in a theory driven way, as opposed to just throwing all the
1:43:31 data at the wall and see what it sticks, I mean, that to me is just exceptionally powerful.
1:43:35 So you can take fMRI data across species, and across different types of humans,
1:43:42 of conditions of humans, and what find construct models that help you find the commonalities,
1:43:51 or the core thing that makes somebody navigate through the metaverse, for example.
1:43:56 Yeah. Yeah, I mean, more or less. I mean, there’s a lot of details, but yes, I think, and not just fMRI,
1:44:01 but you can relate it to, like I said, recordings from large populations of neurons that could
1:44:06 be taken in a human, or even in a non-human animal that is where you think it’s an anatomical homologue.
1:44:14 So that’s just mind-blowing to me. What’s the similarities in humans and mice?
1:44:20 That’s what it’s smashing pumpkins. We’re all just rats in a cage. Is that a smashing pumpkin?
1:44:28 Despite all of your rage at GIFs, you’re still just rat in a cage.
1:44:36 Oh, yeah. All right, good callback. Anyway, good callback. See, these memory retrieval
1:44:40 exercises that we’re doing are actually helping you build a lasting memory of this conversation.
1:44:46 And it’s strengthening the visual thing I have of you with James Brown on stage.
1:44:51 It’s just becoming stronger and stronger by the second.
1:44:54 It’s got a lot to it.
1:44:56 But animal studies work here as well.
1:45:00 Yeah, yeah. So let’s go to the… So I think I’ve got great colleagues who I talk to
1:45:07 who study memory in mice. One of the valuable things in those models is you can study
1:45:15 neural circuits in an enormously targeted way because you could do these
1:45:19 genetic studies, for instance, where you can manipulate particular groups of neurons.
1:45:25 And it’s just getting more and more targeted to the point where you can actually turn on
1:45:30 particular kind of memory just by activating a particular set of neurons that was active
1:45:36 during an experience. So there’s a lot of conservation of some of these neural circuits
1:45:43 across evolution in mammals, for instance. And then some people would even say that there’s
1:45:50 genetic mechanisms for learning that are conserved even going back far, far before.
1:45:55 But let’s go back to the mice in humans question, right?
1:45:58 There’s a lot of differences. So for one thing, the sensory information is very different.
1:46:04 Mice and rats explore the world largely through smelling, olfaction. But they also have vision
1:46:12 that’s kind of designed to kind of catch death from above. So it’s like a very big view of the
1:46:17 world. And we move our eyes around in a way that focuses on particular spots in space where you
1:46:23 get very high resolution from a very limited set of spots in space. So that makes us very different
1:46:29 in that way. We also have all these other structures as social animals that allow us to
1:46:35 respond differently. There’s language. There’s like, you know, so you name it, there’s obviously
1:46:42 gobs of differences. Humans aren’t just giant rats. There’s a bunch more complexity to us.
1:46:46 Time scales are very important. So primate brains and human brains are especially good
1:46:52 at integrating and holding on to information across longer and longer periods of time, right?
1:46:58 And also, you know, finally, it’s like our history of training data, so to speak,
1:47:04 is very, very different than, you know, I mean, humans’ world is very different than a wild
1:47:09 mouse’s world. And a lab mouse’s world is extraordinarily impoverished relative to an
1:47:15 adult human, you know? But still, what can you understand by studying mice? I mean,
1:47:19 just basic, almost behavioral stuff about memory? Well, yes, but that’s very important,
1:47:24 right? So you can understand, for instance, how do neurons talk to each other? That’s a really
1:47:30 big, big question. Neural computation, you think it’s the most simple question, right?
1:47:37 Not at all. I mean, it’s a big, big question. And understanding how two parts of the brain
1:47:44 interact, meaning that it’s not just one area speaking, it’s not like, you know, it’s not like
1:47:49 Twitter, where one area of the brain is shouting, and then another area of the brain is just stuck
1:47:53 listening to this crap. It’s like they’re actually interacting on the millisecond scale, right?
1:47:58 How does that happen? And how do you regulate those interactions, these dynamic, you know,
1:48:03 interactions? We’re still figuring that out, but that’s going to be coming largely from model
1:48:09 systems that are easier to understand. You can do manipulations like drug manipulations to manipulate
1:48:16 circuits and, you know, use viruses and so forth and lasers to turn on circuits that you just can’t
1:48:22 do in humans. So I think there’s a lot that can be learned from mice. There’s a lot that can be
1:48:27 learned from nonhuman primates. And there’s a lot that you need to learn from humans. And I think,
1:48:32 unfortunately, some of the people in the National Institutes of Health think you can learn everything
1:48:38 from the mouse. It’s like, why study memory in humans when I could study learning in a mouse?
1:48:43 And just like, oh my God, I’m going to get my funding from somewhere else.
1:48:46 Well, let me ask you some random, fascinating questions. How does deja vu work?
1:48:54 So deja vu is it’s actually one of these things I think that some of the
1:49:01 surveys suggest that like 75% of people report having a deja vu experience one time or another.
1:49:08 I don’t know where that came from, but I’ve pulled people in my class and most of them
1:49:12 say they’ve experienced deja vu. It’s this kind of sense that I’ve experienced this moment sometime
1:49:18 before I’ve been here before. And actually, there’s all sorts of variants of this, the French have
1:49:24 all sorts of names for various versions of the shammy vu, parley, I don’t know. All these different
1:49:31 vues. But deja vu is the sense that it can be like almost disturbing intense sense of familiarity.
1:49:40 So there is a researcher named Wilder Penfield. Actually, this goes back even earlier to some
1:49:46 of the earliest, like Hulings Jackson was this neurologist who did a lot of the early
1:49:52 characterizations of epilepsy. And one of the things he notices in epilepsy patients,
1:49:57 some group of them right before they would get a seizure, they would have this intense sense
1:50:02 of deja vu. So it’s this artificial sense of familiarity. It’s a sense of having a memory
1:50:09 that’s not there. And so what was happening was there was electrical activity in certain
1:50:16 parts of these brain cells. So this guy Penfield later on, when he was trying to look for how do
1:50:22 we map out the brain to figure out which parts we want to remove and which parts don’t we,
1:50:27 he would stimulate parts of the temporal lobes of the brain and find you could elicit the sense
1:50:31 of deja vu. Sometimes you’d actually get a memory that a person would re-experience just from
1:50:36 electrically stimulating some parts. Sometimes they just have this intense feeling of being
1:50:42 somewhere before. And so one theory which I really like is that in higher order areas of the brain
1:50:50 they’re integrating for many, many different sources of input. What happens is that they’re
1:50:56 tuning themselves up every time you process a similar input. And so that allows you to just
1:51:04 get this kind of a fluent sense that I’m very familiar. You’re very familiar with this place.
1:51:10 And so just being here, you’re not going to be moving your eyes all over the place because you
1:51:14 kind of have an idea of where everything is. And that fluency gives you a sense of like I’m here.
1:51:19 Now I wake up in my hotel room and I have this very unfamiliar sense of where I am.
1:51:24 But there’s a great set of studies done by Ann Cleary at Colorado State where she created
1:51:30 these virtual reality environments and we’ll go back to the metaverse. Imagine you go through a
1:51:36 virtual museum and then she would put people in virtual reality and have them go through a virtual
1:51:43 arcade. But the map of the two places was exactly the same. She just put different skins on them.
1:51:48 So one looks different than the other. But they’ve got same landmarks in the same places,
1:51:53 same objects and everything, but carpeting, colors, theme, everything’s different.
1:51:57 People will often not have any conscious idea that the two are the same, but they could report
1:52:04 this very intense sense of deja vu. So it’s like a partial match that’s eliciting this kind of a
1:52:10 sense of familiarity. And that’s why in patients who have epilepsy that affects memory, you get
1:52:17 this artificial sense of familiarity that happens. And again, this is just one theory amongst many,
1:52:25 but we get a little bit of that feeling it’s not enough to necessarily give you deja vu,
1:52:31 even for very mundane things. So it’s like if I tell you the word rutabaga, your brain’s going
1:52:40 to work a little bit harder to catch it than if I give you a word like apple. That’s because you
1:52:46 hear apple a lot. So your brain’s very tuned up to process it efficiently, but rutabaga takes
1:52:50 a little bit longer and more intense. And you can actually see a difference in brain activity
1:52:55 in areas in the temporal lobe when you hear a word just based on how frequent it is in
1:52:59 the English language. So we think it’s tied to this basic, it’s basically a byproduct of our
1:53:06 mechanism of just learning, doing this error driven learning as we go through life to become
1:53:12 better and better and better to process things more and more efficiently.
1:53:15 So I guess deja vu is just an extra elevated stuff coming together firing for this artificial
1:53:24 memories if it’s the real memory. I mean, why does it feel so intense?
1:53:29 Well, it doesn’t happen all the time, but I think what may be happening is it’s such a,
1:53:35 it’s a partial match to something that we have. And it’s not enough to trigger that sense of,
1:53:40 you know, that ability to pull together all the pieces, but it’s a close enough
1:53:44 match to give you that intense sense of familiarity without the recollection of exactly what happened
1:53:51 when. But it’s also like a spatiotemporal familiarity. So like, it’s also in time.
1:53:57 Like there’s a weird blending of time that happens. And we’ll probably talk about time
1:54:04 because I think that’s a really interesting idea how time relates to memory. But you also kind of
1:54:09 artificial memory brings to mind this idea of false memories that comes in all kinds of context.
1:54:17 But how do false memories form? Well, I like to say there’s no such thing as true or false
1:54:24 memories, right? It’s like Johnny Rotten from the Sex Pistols. He had a saying that’s like,
1:54:28 I don’t believe in false memories anymore than I believe in false songs, right? It’s like,
1:54:33 and so the basic idea is, is that we have these memories that reflect bits and pieces of what
1:54:39 happened as well as our inferences and theories, right? So I’m a scientist and I collect data,
1:54:45 but I use, I use theories to make sense of that data. And so a memory is kind of a mix of all
1:54:52 these things. So where memories can go off the deep end and become what we would call conventionally
1:54:57 as false memories are sometimes little distortions where we filled in the blanks, the gaps in our
1:55:05 memory based on things that we know, but don’t actually correspond to what happened, right?
1:55:10 So if I were to tell you that I’m like, you know, a story about this person who’s like
1:55:20 worried that they have cancer or something like that, and then, you know, they see a doctor and
1:55:25 the doctor says, well, things are very much like you would have expected or like, you know,
1:55:30 what you’re afraid of or something. When people remember that, they’ll often remember, well,
1:55:34 the doctor told the patient that he had cancer, even if that wasn’t in the story because they’re
1:55:40 infusing meaning into that story, right? So that’s a minor distortion. But what happens is,
1:55:45 is that sometimes things can really get out of hand where people have trouble telling the
1:55:51 difference in things that they’ve imagined versus things that happen. But also, as I told you,
1:55:56 the act of remembering can change the memory. And so what happens then is you can actually
1:56:02 be exposed to some misinformation. And so Elizabeth Loftus was a real pioneer in this work and there’s
1:56:08 lots of other work that’s been done since. But basically, it’s like if you remember some event,
1:56:15 and then I tell you something about the event, later on, when you remember the event, you might
1:56:21 remember some original information from the event, as well as some information about what I told you.
1:56:26 And sometimes, if you’re not able to tell the difference, that information that I told you
1:56:32 gets mixed into the story that you had originally. So now I give you some more misinformation or
1:56:38 you’re exposed to some more information somewhere else. And eventually, your memory becomes totally
1:56:43 detached from what happened. And so sometimes you can have cases where people, this is very rare,
1:56:50 but you can do it in the lab too, or like a significant, not everybody, but you know,
1:56:56 a chunk of people will fall for this, where you can give people misinformation about an event
1:57:02 that never took place. And as they keep trying to remember that event more and more, what happens,
1:57:08 they start to imagine, they start to pull up things from other experiences they’ve had.
1:57:13 And eventually, they can stitch together a vivid memory of something that never happened.
1:57:18 Because they’re not remembering an event that happened, they’re remembering the act of trying
1:57:24 to remember what happened, and basically putting it together into the wrong story.
1:57:29 So it’s fascinating because this could probably happen at a collective level.
1:57:36 Like this is probably what successful propaganda machines aim to do,
1:57:40 is creating false memory across thousands, if not millions of minds.
1:57:45 Yeah, absolutely. I mean, this is exactly what they do. And so all these kind of foibles of human
1:57:52 memory get magnified when you start to have social interactions. There’s a whole literature
1:57:56 on something called social contagion, which is basically when misinformation spreads like a
1:58:02 virus, like you remember the same thing that I did, but I give you a little bit of wrong
1:58:07 information, then that becomes part of your story of what happened. Because once you and I share a
1:58:13 memory, like I tell you about something I’ve experienced, and you tell me about your experience
1:58:17 of the same event, it’s no longer your memory or my memory, it’s our memory. And so now the
1:58:23 misinformation spreads. And the more you trust someone, or the more powerful that person is,
1:58:29 the more of a voice they have in shaping that narrative. And there’s all sorts of interesting
1:58:36 ways in which misinformation can happen. There’s a great example of when John McCain and George
1:58:43 Bush Jr. were in a primary, and there are these polls where they would do these, I guess they
1:58:51 were like not robocalls, but real calls where they would poll voters. But they actually inserted
1:58:56 some misinformation about McCain’s beliefs on taxation, I think, and maybe it was something
1:59:02 about illegitimate children or something. I don’t really remember. But they included misinformation
1:59:07 in the question that they asked, like, how do you feel about the fact that he wants to do this or
1:59:12 something? And so people would end up becoming convinced he had these policy things or these
1:59:18 personal things that were not true, just based on the polls that were being used. So it was a case
1:59:24 where, interestingly enough, the people who were using misinformation were actually ahead of the
1:59:32 curve relative to the scientists who were trying to study these effects in memory.
1:59:37 Yeah, it’s really interesting. So it’s not just about truth and falsehoods, like us as
1:59:45 intelligent reasoning machines, but it’s the formation of memories where they become
1:59:53 like visceral. You can rewrite history. If you just look throughout the 20th century,
1:59:58 some of the dictatorships with Nazi Germany, with the Soviet Union,
2:00:05 effective propaganda machines can rewrite our conceptions of history. How we remember our
2:00:11 own culture, our upbringing, all this kind of stuff. And you could do quite a lot of damage
2:00:15 in this way. And then there’s probably some kind of social contagion happening there.
2:00:19 Like certain ideas that may be initiated by the propaganda machine can spread faster than others.
2:00:28 You could see that in modern day, certain conspiracy theories, there’s just something
2:00:32 about them that they are really effective at spreading. There’s something sexy about them,
2:00:38 to people, to where something about the human mind eats it up and then uses that
2:00:44 to construct memories as if they almost were there to witness whatever the content of the
2:00:51 conspiracy theory is. It’s fascinating. Because once you feel like you remember a thing,
2:00:57 I feel like there’s a certainty. It emboldens you to say stuff. It’s not just you believe
2:01:06 an idea is true or not. It’s at the core of your being that you feel like you were there to watch
2:01:14 the thing happen. Yeah. I mean, there’s so much in what you’re saying. One of the things is that
2:01:21 people’s sense of collective identity is very much tied to shared memories. If we have a shared
2:01:27 narrative of the past, or even better, if we have a shared past, we will feel more socially connected
2:01:33 with each other. And I will feel part of this group. They’re part of my tribe, if I remember
2:01:37 the same things in the same way. And you brought up this weaponization of history. And it really
2:01:43 speaks to, I think, one of the parts of memory, which is that if you have a belief, you will find,
2:01:50 and you have a goal in mind, you will find stuff in memory that aligns with it. And you won’t see
2:01:56 the parts in memory that don’t. So a lot of the stories we put together are based on our perspectives.
2:02:01 Right? And so let’s just zoom out for the moment from misinformation. Take something even more
2:02:09 fascinating, but not as scary. I was reading Tan Viet Nguyen, but he wrote a book about the collective
2:02:18 memory of the Vietnam War. He’s a Vietnamese immigrant who was flown out after the war was
2:02:25 over. And so he went back to his family to get their stories about the war. And they called it
2:02:31 the American War, not the Vietnam War. And that just kind of blew my mind, having grown up in the
2:02:38 U.S., and I’ve always heard about it as a Vietnam War. But of course they call it the American War,
2:02:42 because that’s what happened. America came in. And that’s based on their perspective, which is a
2:02:48 very valid perspective. And so that just gives you this idea of the way we put together these
2:02:56 narratives based on our perspectives. And I think the opportunities that we can have in memory is
2:03:04 if we bring groups together from different perspectives and we allow them to talk to each
2:03:11 other and we allow ourselves to listen. I mean, right now you’ll hear a lot of just jammering,
2:03:16 you know, people going blah, blah, blah about free speech, but they just want to listen to
2:03:20 themselves, right? I mean, it’s like, let’s face it, the old days before people were supposedly
2:03:26 awoke, they were trying to ban too-live crew or, you know, just think about Letty Bruce got canceled
2:03:31 for cursing, Jesus Christ, you know? It’s like, this is nothing new. People don’t like to hear
2:03:38 things that disagree with them. But if you’re in it, I mean, you can see two situations in groups
2:03:47 with memory. One situation is you have people who are very dominant who just take over the
2:03:52 conversation. And basically what happens is the group remembers less from the experience,
2:03:57 and they remember more of what the dominant narrator says, right? Now if you have a diverse
2:04:02 group of people, and I don’t mean diverse in necessarily the human resources sense of the
2:04:07 word, I mean, diverse in any way you want to take it, right? But diverse in every way, hopefully.
2:04:12 And you give everyone a chance to speak, and everyone’s being appreciated for their unique
2:04:17 contribution. You get more accurate memories, and you get more information from it, right?
2:04:22 Even two people who come from very similar backgrounds, if you can appreciate the unique
2:04:27 contributions that each one has, you can do a better job of generating information from memory.
2:04:32 And that’s a way to inoculate ourselves, I believe, from misinformation in the modern world.
2:04:39 But like everything else, it requires a certain tolerance for discomfort. And I think
2:04:43 when we don’t have much time, and I think when we’re stressed out, and when we are
2:04:49 just tired, it’s very hard to tolerate discomfort. And I mean, social media has a lot of opportunity
2:04:56 for this because it enables this distributed one-on-one interaction that you’re talking about
2:05:02 where everybody has a voice. But still our natural inclination, you see this on social media,
2:05:08 there’s a natural clustering of people and opinions, and you just kind of form these kind
2:05:13 of bubbles. I think that’s, to me personally, I think that’s a technology problem that could be
2:05:18 solved. If there’s a little bit of interaction, kind, respectful, compassionate interaction
2:05:24 with people that have a very different memory, that respectful interaction will start to
2:05:31 intermix the memories and ways of thinking to where you’re slowly moving towards truth.
2:05:38 But that’s a technology problem because naturally left our own devices, we want to cluster up in
2:05:44 a tribe. Yeah, and that’s the human problem. I think a lot of the problems that come up with
2:05:51 technology aren’t the technology itself as much as the fact that people adapt to the technology
2:05:57 in maladaptive ways. I mean, one of my fears about AI is not what AI will do, but what people
2:06:05 will do. I mean, take text messaging. It’s like, it’s pain in the ass to text people, at least for
2:06:09 me. And so what happens is the communication becomes very spartan and devoid of meaning. It’s
2:06:15 just very telegraphic, and that’s people adapting to the medium. I mean, look at you. You’ve got this
2:06:21 keyboard that’s got these dome-shaped things, and you’ve adapted to that to communicate.
2:06:27 That’s not the technology adapting to you. It’s you adapting to the technology.
2:06:33 One of the things I learned when Google started to introduce autocomplete in emails,
2:06:37 I started to use it. And about a third of the time, I was like, this isn’t what I want to say.
2:06:42 A third of the time, I’d be like, this is exactly what I wanted to say. And a third of the time,
2:06:46 I was saying, well, this is good enough. I’ll just go with it. And so what happens is it’s not that
2:06:52 the technology necessarily is doing anything so bad as much as it’s just going to constrain my
2:06:59 language because I’m just doing what’s being suggested to me. And so this is why I say kind
2:07:06 of like my mantra for some of what I’ve learned about everything in memory is to diversify your
2:07:12 training data, basically, because otherwise you’re going to be– so humans have this capability to
2:07:18 be so much more creative than anything generative AI will put together, at least right now who knows
2:07:24 where this goes. But it can also go the opposite direction where people could become much, much
2:07:30 less creative if they just become more and more resistant to discomfort and resistant to
2:07:38 exposing themselves to novelty, to cognitive dissonance, and so forth.
2:07:43 I think there is a dance between natural human adaptation of technology and the people that
2:07:48 design the engineering of that technology. So I think there’s a lot of opportunity to create
2:07:54 like this keyboard, things that on net are a positive for human behavior. So we adapt and
2:08:01 all this kind of stuff, but when you look at the long arc of history across years and decades,
2:08:07 has humanity been flourishing? Are humans creating more awesome stuff? Are humans happy or all that
2:08:14 kind of stuff? And so there I think technology on net has been and I think maybe hope will always be
2:08:24 on net a positive thing. Do you think people are happier now than they were 50 years ago or 100
2:08:29 years ago? Yes. I don’t know about that. I think humans in general like to reminisce about the past,
2:08:38 like the times are better. That’s true. And complain about the weather today or complain
2:08:43 about whatever today because there’s this kind of complaining engine that just there’s so much
2:08:49 pleasure in saying life sucks for some reason. That’s why I love punk rock. Exactly. I mean,
2:08:57 there’s something in humans that loves complaining, even about trivial things,
2:09:03 but complaining about change, complaining about everything. But ultimately, I think on net,
2:09:09 on every measure, things are getting better. Life is getting better. Oh, life is getting better,
2:09:17 but I don’t know necessarily that tracks people’s happiness, right? I mean, I would argue that maybe,
2:09:22 who knows? I don’t know this, but I wouldn’t be surprised if people in hunter-gatherer societies
2:09:27 are happier. I mean, I wouldn’t be surprised if they’re happier than people who have access to
2:09:33 modern medicine and email and cell phones. Well, I don’t think there’s a question whether you
2:09:40 take hunter-gatherer folks and put them into modern day and give them enough time to adapt.
2:09:44 They would be much happier. The question is, in terms of every single problem they’ve had,
2:09:50 is not solved. There’s not food. There is guaranteed survival shelter and all this kind
2:09:55 of stuff. So, well, you’re asking is a deeper sort of biological question. Do we want to be,
2:10:00 we’re in a Herzog movie, a happy people, life in the taiga. Do we want to be busy 100% of our time
2:10:07 hunting, gathering, surviving, worried about the next day, maybe that constant struggle ultimately
2:10:16 creates a more fulfilling life? I don’t know, but I do know this modern society allows us
2:10:22 to, when we’re sick, to find medicine, to find cures. When we’re hungry to get food,
2:10:30 much more than we did even 100 years ago. And there’s many more activities that you could
2:10:38 perform or create of all these kinds of stuff that enables the flourishing of humans at the
2:10:43 individual level. Whether that leads to happiness, I mean, that’s a very deep philosophical question.
2:10:49 Maybe struggle, deep struggle is necessary for happiness.
2:10:55 Or maybe cultural connection. Maybe it’s about functioning in social groups that are meaningful
2:11:03 and having time. But I do think there is an interesting memory-related thing, which is that
2:11:08 if you look at things like reinforcement learning, for instance, you’re not learning necessarily
2:11:14 every time you get a reward. If it’s the same reward, you’re not learning that much. You mainly
2:11:21 learn if deviates from your expectation of what you’re supposed to get. It’s like you get a paycheck
2:11:26 every month from MIT or whatever. You probably don’t even get excited about it when you get the
2:11:34 paycheck. But if they cut your salary, you’re going to be pissed. And if they increase your salary,
2:11:38 oh, good, I got a bonus. And that adaptation and that ability that basically you learn to expect
2:11:48 these things, I think, is a major source of, I guess, it’s a major way in which we’re kind of
2:11:54 more, in my opinion, wired to strive and not be happy to be in a state of wanting. And so people
2:12:02 talk about dopamine, for instance, being this pleasure chemical. And it’s like there’s a lot
2:12:07 of compelling research to suggest it’s not about pleasure at all. It’s about the discomfort that
2:12:14 energizes you to get things, to seek a reward. And so you could give an animal that’s been deprived
2:12:21 of dopamine a reward and, oh, yeah, enjoy it. It’s pretty good. But they’re not going to do anything
2:12:28 to get it. And just one of the weird things in our research is I got into curiosity from a postdoc
2:12:36 in my lab, Matthias Gruber. And one of the things that we found is when we gave people a question,
2:12:42 like a trivia question that they wanted the answer to, that question, the more curious people were
2:12:49 about the answer, the more activity in these dopamine-related circuits in the brain we would see.
2:12:54 And again, that was not driven by the answer, per se, but by the question. So it was not about
2:13:01 getting the information. It was about the drive to seek the information. But it depends on how
2:13:08 you take that. If you get this uncomfortable gap between what you know and what you want to know,
2:13:13 you could either use that to motivate you and energize you, or you could use it to say,
2:13:18 I don’t want to hear about this. This disagrees with my beliefs. I’m going to go back to my echo
2:13:22 chamber. I like what you said that maybe were designed to be in a constant state of wanting,
2:13:32 which, by the way, is a pretty good either band name or rock song name, state of wanting.
2:13:40 That’s like a hardcore band name. Yeah, yeah, yeah. It’s pretty good.
2:13:43 I also like the hedonic treadmill. Hedonic treadmill is pretty good.
2:13:47 Yeah, yeah. We could use that for our techno project, I think.
2:13:51 You mean the one we’re starting? Yeah, exactly.
2:13:53 Okay, great. We’re going on tour soon. This is our announcement.
2:14:02 We could build a false memory of a show, in fact, if you want. Let’s just put it all together.
2:14:06 We don’t even have to do all the work to play the show. We can just create a memory of it.
2:14:10 It might as well happen because the remembering self is in charge anyway.
2:14:14 So let me ask you about, we talked about false memories, but in the legal system,
2:14:19 false confessions, I remember reading 1984 where, sorry for the dark turn of our conversation, but
2:14:28 through torture, you can make people say anything and essentially remember anything.
2:14:34 I wonder towards degree, there’s like truth to that. If you look at
2:14:38 the torture that happened in the Soviet Union, for confessions, all that kind of stuff,
2:14:42 how much can you really get people to really, yeah, to force false memories, I guess.
2:14:50 Yeah, I mean, I think there’s a lot of history of this actually in the criminal justice system.
2:14:58 You might have heard the term the third degree. If you actually look it up, historically,
2:15:04 it was a very intense set of beatings and starvation and physical demands that they would
2:15:11 place at people to get them to talk. There’s certainly a lot of work that’s been done by the
2:15:18 CIA in terms of interrogation techniques. And from what I understand, the research actually
2:15:26 shows that they just produce what people want to hear, not necessarily the information that
2:15:33 is being looked for. And the reason is that, I mean, there’s different reasons. I mean,
2:15:38 one is people just get tired of being tortured and just say whatever. But another part of it is that
2:15:44 you create a very interesting set of conditions where there’s an authority figure telling you
2:15:50 something that you did this, we know you did this, we have witnesses saying you did this.
2:15:54 So now you start to question yourself. Then they put you under stress. Maybe they’re not
2:16:00 feeding you. Maybe they’re kind of like making you be cold or exposing you to music that you
2:16:07 can’t stand or something, whatever it is, right? It’s like they’re creating this physical stress.
2:16:12 And so stress starts to act on, starts to down-regulate the prefrontal cortex. You’re not
2:16:19 necessarily as good at monitoring the accuracy of stuff. Then they start to get nice to you and
2:16:24 they say, imagine, okay, I know you don’t remember this, but maybe we can walk you through how it
2:16:29 could have happened and they feed you the information. And so you’re in this weakened mental
2:16:34 state and you’re being encouraged to imagine things by people who give you a plausible scenario.
2:16:40 And at some point, certain people can be very coaxed into creating a memory for something that
2:16:46 never happened. And there’s actually some pretty convincing cases out there where you don’t know
2:16:51 exactly the truth. There’s a sheriff, for instance, who came to believe that he had a false memory,
2:16:59 I mean, that he had a memory of doing sexual abuse based on, you know, essentially, I think it was,
2:17:04 you know, I’m not going to tell the story because I don’t remember it well enough to
2:17:10 necessarily accurately give it to you. But people could look this stuff up. There are definitely
2:17:14 stories out there like this where people confess to crimes that they just didn’t do when objective
2:17:19 evidence came out later on. But there’s a basic recipe for it, which is you feed people the
2:17:26 information that you want them to remember. You stress them out, you have an authority figure,
2:17:33 kind of like pushing this information on them, or you motivate them to produce the information
2:17:39 you’re looking for. And that pretty much over time gives you what you want.
2:17:44 It’s really tragic that centralized power can use these kinds of tools to destroy lives, sad.
2:17:55 Since there’s a theme about music throughout this conversation,
2:18:03 one of the best topics for songs is heartbreak, love in general, but heartbreak.
2:18:11 Why and how do we remember and forget heartbreak, asking for a friend?
2:18:15 Oh, God, that’s so hard to asking for a friend of that.
2:18:19 It’s such a hard one. Well, so, I mean, part of this is we tend to go back to particular
2:18:30 times that are the more emotionally intense periods. And so that’s a part of it. And again,
2:18:39 memory is designed to kind of capture these things that are biologically significant. And
2:18:44 attachment is a big part of biological significance for humans, right? Human relationships
2:18:49 are super important. And sometimes that heartbreak comes with massive changes in your beliefs about
2:18:56 somebody, say if they cheated on you or something like that, or regrets, and you kind of ruminate
2:19:02 about things that you’ve done wrong. There’s really so many reasons though, but I mean,
2:19:09 I’ve had this, my first pet I had was, we got it for a wedding present as a cat and got it after,
2:19:19 like, but it died of FIP when it was four years old. And I just would see her everywhere around
2:19:27 the house. We got another cat that we got a dog, dog eventually died of cancer, and the cat just
2:19:33 died recently. And, you know, so we got a new dog because I kept seeing the dog around and I was
2:19:40 just so heartbroken about this. But I still remember the pets that died, it just comes back to you.
2:19:47 I mean, it’s part of this, I think there’s also something about attachment that’s just so crucial
2:19:52 that drives, again, these things that we want to remember and that gives us that longing sometimes.
2:19:59 Sometimes it’s also not just about the heartbreak, but about the positive aspects of it, right?
2:20:05 Because the loss comes from not only the fact that the relationship is over, but you had all
2:20:11 of these good things before that you can now see in a new light, right? And so part of one of the
2:20:18 things that I found from my clinical background that really I think gave me a different perspective
2:20:23 on memory is so much of the therapy process was guided towards reframing and getting people to
2:20:31 look at the past in a different way, not by imposing, changing people’s memories or not
2:20:36 by imposing an interpretation, but just offering a different perspective and maybe one that’s kind
2:20:42 of more optimized towards learning and, you know, an appreciation maybe or gratitude, whatever it is,
2:20:49 right? That gives you a way of taking, I think you said it in the beginning, right? Where you
2:20:54 can have this kind of like dark experiences and you can use it as training data to, you know,
2:21:02 grow in new ways. But it’s hard. This, I often go back to this moment, this show, Louis,
2:21:09 with Louis C.K., where he’s all heartbroken about a breakup with a woman he loves and
2:21:17 an older gentleman tells him that that’s actually the best part, that heartbreak,
2:21:23 because you get to intensely experience how valuable this love was. He says the worst part
2:21:30 is forgetting it is actually when you get over the heartbreak. That’s the worst part. So I sometimes
2:21:37 think about that because, you know, having the love and losing it, like the losing it is when you
2:21:47 sometimes feel it the deepest, which is an interesting way to celebrate the past and relive it.
2:21:54 It sucks that you don’t have a thing, but when you don’t have a thing, it’s a
2:21:59 good moment to viscerally experience the memories of something that you now appreciate even more.
2:22:07 So you don’t believe that an owner of a lonely heart is much better than an owner of a broken heart?
2:22:14 You think an owner of a broken heart is better than the owner of a lonely heart?
2:22:17 Yes, for sure. I think so. I think so. But I’m gonna have to, day by day,
2:22:21 I don’t know, I’m gonna have to listen to some more Bruce Springsteen to figure that one out.
2:22:26 Well, you know, it’s funny because it’s like after I turned 50, I think of death all the time.
2:22:31 Like, I just think that, you know, I’m in like, I have fewer, probably a fewer years ahead of me
2:22:38 than I have behind me, right? So I think about one thing, which is what are the memories that
2:22:43 I want to carry with me for the next period of time? And also about, like, just the fact that
2:22:49 everything around me could be, you know, I know more people who are, you know, dying for various
2:22:55 reasons. And so, I’m not lots, I’m not that old, right? But, you know, it’s something I think about
2:23:03 a lot. And I’m reminded of, like, how I talked to somebody who’s like, you know, who’s a Buddhist.
2:23:10 And I was like, you know, the whole idea of Buddhism is renouncing attachments.
2:23:14 Someway, the idea of Buddhism is like staying out of the world of memory and staying in the moment,
2:23:20 right? And they talked about, you know, it’s like, how do you, how do you renounce attachments to the
2:23:26 people that you love, right? And they’re just saying, well, I appreciate that I have this moment
2:23:30 with them. And knowing that they will die makes me appreciate this moment that much more. I mean,
2:23:35 you said something similar, right? And your daily routine that you think about things this way,
2:23:40 right? Yeah, I meditate on mortality every day. But I don’t know, at the same time,
2:23:48 that really makes you appreciate the moment and live in the moment. And I also appreciate the full
2:23:54 deep roller coaster of suffering involved in life, the little and the big two. So, I don’t know.
2:24:01 The Buddhist kind of removing yourself from the world or the stoic removing yourself from the
2:24:07 world, the world of emotion. I’m torn about that one. I’m not sure.
2:24:11 Well, you know, this is where Hinduism and Buddhism or at least some strains of Hinduism
2:24:16 and Buddhism differ. And Hinduism, like if you read the Bhagavad Gita, the philosophy is not one
2:24:23 of renouncing the world because the idea is that not doing something is no different than doing
2:24:31 something, right? So, what they argue, and again, you could interpret it in different ways, positive
2:24:36 and negative. But the argument is that you don’t want to renounce action, but you want to renounce
2:24:43 the fruits of the action. You don’t do it because of the outcome, you do it because of the process.
2:24:49 Because the process is part of the balance of the world that you’re trying to preserve, right?
2:24:54 And of course, you could take that different ways. But I really think about that from time to time
2:24:59 in terms of letting go of this idea of does this book sell or trying to impress you and get you
2:25:09 laugh at my jokes or whatever, and just be more like I’m sharing this information with you and
2:25:14 getting to know you or whatever it is. But it’s hard, right? Because we’re so driven by the
2:25:21 reinforcer of the outcome. You’re just part of the process of telling the joke and if I laugh or not,
2:25:28 that’s up to the universe to decide. Yep, it’s my Dharma.
2:25:32 How does studying memory affect your understanding of the nature of time? So, we’ve been talking
2:25:41 about us living in the present and making decisions about the future, standing on the
2:25:49 foundation of these memories and there it is about the memories that we’ve constructed.
2:25:53 So, it feels like it does weird things to time. Yeah, and the reason is, is that in some sense,
2:26:01 I think, especially the farther we go back, I mean, there’s all sorts of interesting things
2:26:06 that happen. So, your sense of like, if I ask you how different does one hour ago feel from two hours
2:26:14 ago, you’d probably say pretty different. But if I ask you, okay, go back one year ago versus one
2:26:20 year and one hour ago, it’s the same difference in time. It won’t feel very different, right? So,
2:26:24 there’s this kind of compression that happens as you look back farther in time. So, it’s kind of
2:26:30 like why when you’re older, the difference between somebody who’s like 50 and 45 doesn’t seem as big
2:26:37 as the difference between like 10 and 5 or something, right? When you’re 10 years old,
2:26:41 everything seems like it’s a long period of time. Here’s the point is that, you know, so one of the
2:26:46 interesting things that I found when I was working on the book, actually, was during the pandemic,
2:26:50 I just decided to ask people in my class when we were doing the remote instruction. So,
2:26:55 one of the things I did was I’d pull people. And so, I just asked people, do you feel like the
2:27:00 days are moving by slower or faster or about the same? Almost everyone in the class said that
2:27:08 the days were moving by slower. So, then I would say, okay, so, do you feel like the weeks are
2:27:14 passing by slower, faster or the same? And the majority of them said that the weeks were passing
2:27:20 by faster. So, according to the laws of physics, I don’t think that makes any sense, right?
2:27:25 But according to memory, it did because what happened was people were doing the same thing
2:27:31 over and over in the same context. And without that change in context, their feeling was that
2:27:39 they were in one long monotonous event. And so, but then, at the end of the week, you look back
2:27:46 at that week and you say, well, what happened? No memories of what happened. So, it must,
2:27:51 the week just went by without even my noticing it. But that week went by during the same amount
2:27:57 of time as an eventful week where you might have been going out and hanging out with friends on
2:28:01 vacation or whatever, right? It’s just that nothing happened because you’re doing the
2:28:06 same thing over and over. So, I feel like memory really shapes our sense of time. But it does so
2:28:12 in part because context is so important for memory. Well, that compression you mentioned,
2:28:18 it’s an interesting process. Because what I think about when I was like 12 or 15,
2:28:26 I just fundamentally feel like the same person. It’s interesting what that compression does.
2:28:32 It makes me feel like it’s all, we’re all connected, not just amongst humans and spatially, but
2:28:37 in terms, back in time, there’s a kind of eternal nature, like the timelessness, I guess,
2:28:45 to life. That could be also a genetic thing just for me. I don’t know if everyone agrees
2:28:52 to this view of time, but to me, it all feels the same. Like you don’t feel the passage of time?
2:28:57 No, I feel the passage of time in the same way that your students did from day to day.
2:29:02 There’s certain markers that let you know that time has passed, you celebrate birthdays and so on.
2:29:10 But the core of who I am and who others I know are or events, that compression of my understanding
2:29:17 of the world removes time, because time is not useful for the compression. The details of that
2:29:24 time, at least for me, is not useful to understanding the core of the thing. Maybe what it is is that
2:29:31 you really like to see connections between things. This is really what motivates me in science,
2:29:37 actually, too. But it’s like when you start recalling the past and seeing the connections
2:29:43 between the past and present, now you have this kind of web of interconnected memories.
2:29:49 I can imagine, in that sense, there is this kind of the present is with you.
2:29:55 But what’s interesting about what you said, too, that struck me is that your 16-year-old self was
2:30:03 probably very complex. And by the way, I’m the same way, but it’s like it really is the source of a
2:30:09 lot of darkness for me. But when you can look back at, let’s say you hear a song that you used
2:30:18 to play before you would go do a sports thing or something like that, and you might not think of
2:30:23 yourself as an athlete, but once you get back to that, you mentally time travel to that particular
2:30:28 thing, you open up this little compartment of yourself that wasn’t there before, right, that
2:30:33 didn’t seem accessible for them. Dan Schachter’s lab did this really cool study where they would
2:30:39 ask people to either remember doing something altruistic or imagine doing something altruistic.
2:30:47 And that act made them more likely to want to do things for other people. So that
2:30:56 active mental time travel can change who you are in the present. And we tend to think of,
2:31:01 this goes back to that illusion of stability, and we tend to think of
2:31:05 memory in this very deterministic way that I am who I am because I have this past. But we have a
2:31:10 very multifaceted past and can access different parts of it and change in the moment based on
2:31:18 whatever part we want to reach for, right? How does nostalgia connect into this? Like this
2:31:25 desire and pleasure associated with going back? Yeah. So my friend Felipe de Bregard
2:31:34 wrote this and it just like blew my mind where the word nostalgia was coined by a Swiss physician
2:31:41 who was actually studying traumatized soldiers. And so he described nostalgia as a disease.
2:31:46 And the idea was it was bringing these people extraordinary unhappiness because they were
2:31:51 remembering how things used to be. And I think it’s very complex. So as people get older,
2:31:59 for instance, nostalgia can be an enormous source of happiness, right? And being nostalgic can
2:32:06 improve people’s moods in the moment. But it just depends on what they do with it because what you
2:32:12 can sometimes see is nostalgia has the opposite effect of thinking those were the good old days
2:32:17 and those days are over, right? It’s like America used to be so great and now it sucks. Or you know,
2:32:23 my life used to be so great when I was a kid and now it’s not, right? And you’re selectively
2:32:29 remembering the things that we don’t realize how selective our remembering self is. And so,
2:32:35 you know, I lived through the 70s, it sucked. Partly it sucked more for me, but I would say that
2:32:43 even otherwise, it’s like there’s all sorts of problems going on. Gas lines, people were like,
2:32:48 you know, worried about like Russia, nuclear war, blah, blah, blah. So I mean, it’s just this idea
2:32:55 that people have about the past can be very useful if it brings you happiness in the present. But
2:33:03 if it narrows your worldview in the present, you’re not aware of those biases that you have.
2:33:09 You will end up, you can end up, it can be toxic, right? Either at a personal level
2:33:14 or at a collective level. Let me ask you both a practical question and an out there question.
2:33:20 So let’s start with a more practical one. What are your thoughts about BCIs, brain computer
2:33:27 interfaces and the work that’s going on with Neuralink? We talked about electrodes and different
2:33:32 ways of measuring the brain. And here, Neuralink is working on basically two-way communication
2:33:37 with the brain. And the more out there question would be like, where does this go? But more
2:33:40 practically in the near term, what do you think about Neuralink? Yeah, I mean, I can’t say specifics
2:33:46 about the company because I haven’t studied it that much. But I mean, I think there’s two parts of
2:33:51 it. So one is they’re developing some really interesting technology. I think with these like
2:33:55 surgical robots and things like that. BCI though has like a whole lot of innovation going on.
2:34:03 I’m not necessarily seeing any scientific evidence from Neuralink. And maybe that’s just
2:34:09 I’m not looking for it, but I’m not seeing the evidence that they’re anywhere near where the
2:34:14 scientific community is. And there’s lots of startups that are doing incredibly innovative
2:34:18 stuff. One of my colleagues, Sergei Siviski, is just like a genius in this area. And they’re working
2:34:23 on it. I think speech prosthetics like that are incorporating, you know, decoding techniques with
2:34:29 AI and movement prosthetics. The rate of progress is just enormous. So part of the technology is
2:34:37 having good enough data and understanding which data to use and what to do with it, right? And then
2:34:44 the other part of it then is the algorithms for decoding it and so forth. And I think part of
2:34:49 that has really resulted in some real breakthroughs in neuroscience as a result. So there’s lots of
2:34:56 new technologies like NeuroPixels, for instance, that allow you to harvest
2:35:00 activity from many, many neurons from a single electrode. I know Neuralink has some technologies
2:35:06 that are also along these lines. But I even, again, because they do their own stuff, the scientific
2:35:12 community doesn’t see it, right? But I think BCI is much, much bigger than Neuralink. And there’s
2:35:19 just so much innovation happening. I think the interesting question, which we may be getting
2:35:25 into, is I was talking to Sergey a while ago about, you know, so a lot of language is not
2:35:30 just what we hear and what we speak, but also our intentions and our internal models. And,
2:35:37 you know, so are you really going to be able to restore language without dealing with that
2:35:41 part of it? And he brought up a really interesting question, which is the ethics of reading out
2:35:47 people’s intentions and understanding of the world as opposed to the more, you know, the more
2:35:54 concrete parts of hearing and producing movements, right? Just so we’re clear, because you said a
2:36:00 few interesting things. When you say, when we talk about language and BCI is what we mean is
2:36:04 getting signal from the brain and generating the language, say you’re not able to actually speak,
2:36:12 it’s as a kind of linguistic prosthetic, or it’s able to speak for you
2:36:18 exactly what you wanted to say. And then the deeper question is, well,
2:36:23 saying something isn’t just the letters, the words you’re saying, it’s also the intention
2:36:30 behind it, the feeling behind all that kind of stuff. And is it ethical to reveal that full
2:36:36 shebang, the full context of what’s going on in our brain? That’s really, that’s really interesting.
2:36:43 That’s really interesting. I mean, our thoughts. Is it ethical for anyone to have access to our
2:36:49 thoughts? Because right now the resolution is so low that we’re okay with it, even doing studies
2:36:57 and all this kind of stuff. But if neuroscience has a few breakthroughs to where you can start
2:37:02 to map out the QR codes for different thoughts, for different kinds of thoughts,
2:37:06 maybe political thoughts, the McCarthyism, what if I’m getting a lot of them communist thoughts,
2:37:14 or however we want to categorize or label it. That’s interesting. That’s really interesting.
2:37:22 I think ultimately there’s always the more transparency there is about the human mind,
2:37:30 the better it is. But there could be always intermediate battles with how much control
2:37:38 does a centralized entity have, like a government and so on. What is the regulation? What are the
2:37:42 rules? What’s legal and illegal? If you talk about the police whose job is to
2:37:48 track down criminals and so on, and you look at all the history, how the police could abuse its
2:37:55 power to control the citizenry, all that kind of stuff. So people are always paranoid and rightfully
2:38:01 so. It’s fascinating. It’s really fascinating. We talk about freedom of speech, freedom of thought,
2:38:09 which is also a very important liberty at the core of this country and probably humanity
2:38:17 starts to get awfully tricky when you start to be able to collect those thoughts.
2:38:23 But what I wanted to actually ask you is, do you think for fun and for practical purposes
2:38:31 we would be able to modify memories? So how difficult is it? How far away we are from
2:38:43 understanding the different parts of the brains, everything we’ve been talking about,
2:38:48 in order to figure out how could we adjust this memory at the crude level from
2:38:52 unpleasant to pleasant? You talked about we can remember the mall and the people,
2:38:58 like the location of the people. Can we keep the people and change the place,
2:39:02 like this kind of stuff? How difficult is that? Well, I mean, in some sense, we know we can do it
2:39:08 just behaviorally, right? Behaviorally, yes. I can just tell you, under certain conditions,
2:39:13 anyway, it can give you the misinformation and then you can change the people and places and so
2:39:18 forth, right? On the crude level, there’s a lot of work that’s being done on a phenomenon called
2:39:24 reconciliation, which is the idea that essentially when I recall a memory, what happens is that the
2:39:31 connections between the neurons and that cell assembly that give you the memory are going to be
2:39:38 more modifiable. And so some people have used techniques to try to, like, for instance, with
2:39:44 fear memories to reduce that physical visceral component of the memory when it’s being activated.
2:39:51 Right now, I think I’ve, as an outsider looking at the data, I think it’s like mixed results.
2:39:56 And part of it is, and this speaks to the more complex issue, is that you don’t need somebody
2:40:04 to actually fully recall that traumatic memory in the first place. And in order to actually
2:40:11 modify it, then what is the memory? That is the key part of the problem. So if we go back to
2:40:17 reading people’s thoughts, what is the thought? I mean, people can sometimes look at this like
2:40:22 behaviorist and go, well, the memory is like, I’ve given you A and you produce B. But I think that’s
2:40:27 a very bankrupt concept about memory. I think it’s much more complicated than that. And, you know,
2:40:33 one of the things that when we started studying naturalistic memory, like memory from movies,
2:40:37 that was so hard was we had to change the way we did the studies. Because if I show you a movie,
2:40:44 and I show, and I watch the same movie, and you recall everything that happened, and I recall
2:40:50 everything that happened, we might take a different amount of time to do it, we might use different
2:40:55 words. And yet to an outside observer, we might have recalled the same thing, right? So it’s not
2:41:00 about the words necessarily. And it’s not about how long we spent or whatever, there’s something
2:41:06 deeper that is there. That’s this idea. But it’s like, how do you understand that thought? I encounter
2:41:13 a lot of concrete thinking that it’s like, if I show a model, like, you know, the visual
2:41:20 information that a person sees when they drive, I can basically reverse engineer driving. Well,
2:41:26 that’s not really how it works. I once saw a talk by somebody, or I saw somebody talking in this
2:41:32 discussion of between neuroscientists and AI people. And he was saying that the problem with
2:41:38 self-driving cars that they had in cities, as opposed to highways, was that the car was okay at,
2:41:44 you know, doing the things it’s supposed to. But when there are pedestrians around, it couldn’t
2:41:49 predict the intentions of people. And so that unpredictability of people was the problem that
2:41:56 they were having in, you know, the self-driving car design, because it didn’t have a good enough
2:42:01 internal model of what the people were, you know, what they were doing, what they wanted.
2:42:07 Now, what do you think about that? Well, I spent a huge amount of time
2:42:11 watching pedestrians, thinking about pedestrians, thinking about what it takes to solve the problem of
2:42:19 measuring, detecting the intention of a pedestrian, really of a human being in this particular context
2:42:30 of having to cross the street. And it’s fascinating. I think it’s a window into
2:42:40 how complex social systems are that involve humans. Because, you know, I would just stand there and
2:42:49 watch intersections for hours. And when you start to figure out is every single intersection has
2:42:55 its own personality. So like, there’s a history to that intersection, like jaywalking certain
2:43:03 intersections allow jaywalking a lot more. Because what happens is, we’re leaders and followers.
2:43:11 So there’s a regular, let’s say, and they get off the subway and they start crossing on red light,
2:43:16 and they do this every single day. And then there’s people that don’t show up to the intersection
2:43:20 often, and they’re looking for cues of how we’re supposed to behave here. And if a few people start
2:43:25 to jaywalk and cross on red light, they will also, they will follow. And there’s just a dynamic to
2:43:32 that intersection. There’s a spirit to it. And if you look at Boston versus New York,
2:43:37 versus a rural town versus even Boston, San Francisco or here in Austin, there’s different
2:43:43 personalities citywide, but there’s different personalities area at region wide. And there’s
2:43:48 different personalities, different intersections. And it’s just fascinating for a car to be able
2:43:53 to determine that is tricky. Now, what machine learning systems are able to do well is collect
2:43:59 a huge amount of data. So for us, it’s tricky because we get to understand the world with very
2:44:06 limited information and make decisions grounded in this big foundation model that we’ve built of
2:44:13 understanding how humans work. AI could literally, in the context of driving, this is where I’ve
2:44:20 often been really torn in both directions. If you just collect a huge amount of data,
2:44:26 all of that information and then compress it into a representation of how humans cross streets,
2:44:32 it’s probably all there. In the same way that you have a Noam Chomsky who says,
2:44:38 no, no, no, AI can’t talk, can’t write length convincing language without understanding
2:44:44 language. And, you know, more and more, you see large language models without quote unquote
2:44:49 understanding can generate very convincing language. But I think with the process of
2:44:54 compression from a huge amount of data compressing into a representation is doing is in fact,
2:45:00 understanding deeply in order to be able to generate one letter at a time, one word at a time,
2:45:07 you have to understand the cruelty of Nazi Germany and the beauty of sending humans to space.
2:45:18 And like, you have to understand all of that in order to generate like, I’m going to the kitchen
2:45:22 to get an apple and do that graphically correctly. You have to have a world model that includes all
2:45:27 of human behavior. You think an LLM is building that world model? It has to in order to be good
2:45:33 at generating one word at a time and convincing sentence. And in the same way, I think AI that
2:45:40 drives a car, if it has enough data, will be able to form a world model that will be able to predict
2:45:48 correctly what that pedestrian does. But when we as humans are watching pedestrians, we slowly
2:45:54 realize, damn, this is really complicated. In fact, when you start to self reflect on driving,
2:46:00 you realize driving is really complicated. There’s like subtle cues we take about like,
2:46:05 just a million things I could say, but like one of them determining who around you is an asshole,
2:46:13 aggressive driver. Yeah, I was just thinking about this. Yeah, or like, you can read it. Once you
2:46:19 get become a great driver, you can see it a mile away. This guy is going to pull an asshole move
2:46:25 in front of you. Exactly. He’s like way back there, but you know it’s going to happen. And
2:46:29 I don’t know what, because we’re ignoring all the other cars. But for some reason, the asshole,
2:46:34 like a red, like, like a glowing, obvious symbol is just like right there, even in the periphery
2:46:40 vision, because we’re again, we’re usually when we’re driving just looking forward, but we’re like
2:46:45 using the periphery vision to figure stuff out. And it’s like a little puzzle that we’re usually
2:46:51 only allocating a small amount of our attention to, at least a cognitive attention to. And it’s
2:46:56 fascinating, but I think AI just has a fundamentally different suite of sensors in terms of the bandwidth
2:47:03 of data that’s coming in that allows you to form the representation that perform inference on the
2:47:09 representation you form using the representation you form. That for the case of driving, I think it
2:47:15 could be quite effective. But one of the things that’s currently missing, even though OpenAI just
2:47:24 recently announced adding memory. And I did want to ask you, like, how important it is,
2:47:30 how difficult is it to add some of the memory mechanisms that you’ve seen in humans to AI systems?
2:47:38 I would say superficially not that hard, but then in a deeper level, very, very hard, because we
2:47:44 don’t understand episodic memory. So one of the ideas I talked about in the book is one of the oldest
2:47:50 kind of dilemmas in computational neuroscience is what Steve Grossberg called the stability
2:47:56 plasticity dilemma. When do you say something is new and overwrite your preexisting knowledge
2:48:03 versus going with what you had before and making incremental changes? And so, you know, part of
2:48:09 the problem with going through like massive, you know, I mean, part of the problem of things like
2:48:16 if you’re trying to design an LLM or something like that is especially for English, there’s so many
2:48:20 exceptions to the rules, right? And so if you want to rapidly learn the exceptions, you’re going to
2:48:26 lose the rules. And if you want to keep the rules, you have a harder time learning the exception.
2:48:32 And so David Maro is one of the early pioneers in computational neuroscience. And then
2:48:38 Jay McClelland and my colleague, Randy O’Reilly, some other people like Neil Cohen, all these people
2:48:45 started to come up with the idea that maybe that’s part of what we need and what the human brain
2:48:51 is doing is we have this kind of a, actually a fairly dumb system, which just says this happened
2:48:57 once at this point in time, which we call episodic memory, so to speak. And then we have this knowledge
2:49:03 that we’ve accumulated from our experiences as semantic memory. So now, when we want to,
2:49:10 we encounter a situation that’s surprising and violates all our previous expectations,
2:49:16 what happens is that now we can form an episodic memory here. And the next time we’re in a similar
2:49:21 situation, boom, we could supplement our knowledge with this information from episodic memory and
2:49:27 reason about what the right thing to do is, right? So it gives us this enormous amount of flexibility
2:49:33 to stop on a dime and change without having to erase everything we’ve already learned.
2:49:39 And that solution is incredibly powerful because it gives you the ability to learn from so much
2:49:47 less information really, right? And it gives you that flexibility. So one of the things I think
2:49:53 that makes humans great is having both episodic and semantic memory. Now, can you build something
2:50:01 like that? I mean, computational neuroscience people would say, well, yeah, you just record
2:50:06 a moment and you just get it and you’re done, right? But when do you record that moment? How
2:50:11 much do you record? What’s the information you prioritize and what’s the information you don’t?
2:50:15 These are the hard questions. When do you use episodic memory? When do you just throw it away?
2:50:21 And these are the hard questions we’re still trying to figure out in people.
2:50:24 And then you start to think about all these mechanisms that we have in the brain for figuring
2:50:29 out some of these things. And it’s not just one, but many of them that are interacting with each
2:50:34 other. And then you just take not only the episodic and the semantic, but then you start to take the
2:50:39 motivational survival things, right? It’s just like the fight or flight responses that we associate
2:50:45 with particular things are the kind of reward motivation that we associate with certain things
2:50:51 so forth. And those things are absent from AI. I frankly don’t know if we want it. I don’t
2:50:56 necessarily want a self-motivated LLM, right? And then there’s the problem of how do you even
2:51:05 build the motivations that should guide a proper reinforcement learning kind of thing,
2:51:10 for instance. So a friend of mine, Sam Gershman, I might be missing the quote exactly,
2:51:17 but he basically said, “If I wanted to train a typical AI model to make me as much money as
2:51:23 possible, the first thing I might do is sell my house.” So it’s not even just about having one
2:51:30 goal or one objective, but just having all these competing goals and objectives, right? And then
2:51:35 things start to get really complicated. It’s all interconnected. I mean, just even the thing you’ve
2:51:40 mentioned is the moment. If we record a moment, it’s difficult to express concretely what a moment is,
2:51:51 like how deeply connected it is to the entirety of it. Maybe to record a moment, you have to
2:51:58 make a universal scratch. You have to include everything. You have to include all the emotions
2:52:05 involved, all the context, all the things that built around it, all the social connections,
2:52:09 all the visual experiences, all the sensory experience, all of that, all the history that
2:52:15 came before that moment is built on. And we somehow take all of that and we compress it
2:52:21 and keep the useful parts and then integrate it into the whole thing, into our whole narrative.
2:52:27 And then each individual has their own little version of that narrative. And then we collide
2:52:32 in the social way and we adjust it and we evolve. Yeah. Yeah. I mean, well, even if we want to go
2:52:38 super simple, right? Like Tyler Ronan, who’s a postdoc who’s collaborating with me, he actually
2:52:45 studied a lot of computer vision at Stanford. And so one of the things he was interested in
2:52:51 is some people who have brain damage in areas of the brain that were thought to be important for
2:52:55 memory. But they also seem to have some perception problems with particular kinds of object
2:53:01 perception. And this is super controversial. And some people found this effect, some didn’t.
2:53:06 And he went back to computer vision and he said, let’s take the best state-of-the-art computer
2:53:11 vision models and let’s give them the same kinds of perception tests that we were giving to these
2:53:16 people. And then he would find the images where the computer vision models would just struggle.
2:53:21 And you would find that they just didn’t do well. Even if you add more parameters, you add more layers
2:53:27 on and on and on, it doesn’t help, right? The architecture didn’t matter. It was just there.
2:53:30 The problem. And then he found those were the exact ones where these humans with particular
2:53:36 damage to this area called the periorinal cortex, that was where they were struggling.
2:53:40 So somehow this brain area was important for being able to do these things that were adversarial to
2:53:48 these computer vision models. So then he found that it only happened if people had enough time
2:53:57 they could make those discriminations. But without enough time, if they just get a glance,
2:54:01 they’re just like the computer vision models. So then what he started to say was, maybe let’s
2:54:05 look at people’s eyes, right? So a computer vision model sees every pixel all at once, right?
2:54:11 It’s not, you know, and we don’t, we never see every pixel all at once. Even if I’m looking at a
2:54:16 screen with pixels, I’m not seeing every pixel at once. I’m grabbing little points on the screen by
2:54:23 moving my eyes around and getting a very high resolution picture of what I’m focusing on and
2:54:29 kind of a lower resolution information about everything else. But I’m not necessarily choosing,
2:54:35 but I’m directing that exploration and allowing people to move their eyes and integrate that
2:54:42 information gave them something that the computer vision models weren’t able to do.
2:54:48 So somehow integrating information across time and getting less information at each step gave
2:54:55 you more out of the process. I mean, the process of allocating attention across time seems to be
2:55:06 a really important process. Even the breakthroughs that you get with machine learning mostly has
2:55:15 to do, attention is all you need. It’s about attention, transform is about attention. So
2:55:19 attention is a really interesting one. But then like, yeah, how you allocate that attention
2:55:26 again is like, is at the core of like, what it means to be intelligent, what it means to process
2:55:36 the world, integrate all the important things, discard all the unimportant things.
2:55:43 Attention is at the core of it. It’s probably at the core of memory too.
2:55:47 Because there’s so much sensor information, there’s so much going on, so much going on,
2:55:52 to filter it down to almost nothing and just keep those parts and to keep those parts. And then
2:55:59 whenever there’s an error to adjust the model such that you can allocate attention even better to
2:56:04 new things that would result, maybe maximize the chance of confirming the model or disconfirming
2:56:10 the model that you have and adjusting it since then. Yeah, attention is a weird one. I was
2:56:16 always fascinated. I mean, I got a chance to study peripheral vision for a bit and indirectly study
2:56:25 attention through that. And it’s just fascinating how good humans are looking around and gathering
2:56:31 information. Yeah, at the same time, people are terrible at detecting changes that can happen
2:56:37 in the environment if they’re not attending in the right way, if their predictive model is too
2:56:42 strong. So you have these weird things where the machines can do better than the people.
2:56:48 So this is the thing is people go, “Oh, the machines can do this stuff that’s just like humans.”
2:56:54 It’s like, well, the machines make different kinds of mistakes than the people do. And I will never
2:57:01 be convinced unless that we’ve replicated human. I don’t even like the term intelligence,
2:57:07 because I think it’s a stupid concept. But it’s like, I don’t think we’ve replicated human
2:57:12 intelligence unless I know that the simulator is making exactly the same kinds of mistakes that
2:57:19 people do. Because people make characteristic mistakes. They have characteristic biases,
2:57:24 they have characteristic heuristics that we use. And those have yet to see evidence that
2:57:31 chat GPT will do that. Since we’re talking about attention, is there an interesting connection
2:57:38 to you between ADHD and memory? Well, it’s interesting for me, because when I was a child,
2:57:45 I was actually told my school, I don’t know if it came from a school psychologist, they did do
2:57:50 some testing on me, I know for like IQ and stuff like that. Or if it just came from teachers who
2:57:57 hated me, but they told my parents that I had ADHD. And so this was of course in the ’70s. So
2:58:04 basically, they said he has poor motor control and he’s got ADHD. And there were social issues.
2:58:12 So I could have been put a year ahead in school, but then they said, oh, but he doesn’t have the
2:58:18 social capabilities. So I still ended up being an outcast even in my own grade. But
2:58:27 but then like, so then my parents said, okay, well, they got me on a diet free of artificial
2:58:34 colors and flavors, because that was the thing that people talked about back then. So I’m interested
2:58:39 this topic because I’ve come to appreciate now that I have many of the characteristics, if not,
2:58:45 you know, full blown. It’s like, I’m definitely time blindness, rejection sense. If you name it,
2:58:51 they talk about it. It’s like, impulsive behavior, I could tell you about all sorts of fights I’ve
2:58:56 gotten into in the past. Just you name it. But yeah, so ADHD is fascinating, though, because
2:59:04 right now we’re seeing like, more and more diagnosis of it. And I don’t know what to say about that.
2:59:09 I don’t know how much of that is based on kind of inappropriate expectations, especially for
2:59:18 children. And how much of that is based on true kind of like maladaptive kinds of tendencies.
2:59:25 But what we do know is this is that ADHD is associated with differences in prefrontal
2:59:30 function, so that attention can be both more, you’re more distractible, you have harder time
2:59:37 focusing your attention on what’s relevant. And so you shift too easily. But then once you get on
2:59:42 something that you’re interested in, you can get stuck. And so, you know, the attention is this
2:59:47 beautiful balance of being able to focus when you need to focus and shift when you need to shift.
2:59:54 And so it’s that flexibility plus stability again. And that’s balance seems to be disrupted in ADHD.
3:00:03 And so as a result, memory tends to be poor in ADHD. But it’s not necessarily because there’s a
3:00:09 traditional memory problem. But it’s more because of this attentional issue, right? And so people
3:00:18 with ADHD often will have great memory for the things that they’re interested in. And just no
3:00:24 memory for the things that they’re not interested in. Is there advice from your own life on how to
3:00:30 learn and succeed from that? From just how the characteristics of your own brain with ADHD and
3:00:36 so on? How do you learn? How do you remember information? How do you flourish in this sort
3:00:47 of education context? I’m still trying to figure out the flourishing per se. But education, I mean,
3:00:53 being in science is enormously enabling of ADHD. It’s like, you’re constantly looking for new things,
3:01:00 you’re constantly seeking that dope of being hit. And that’s great, you know, and they tolerate,
3:01:07 you’re being late for things. Nothing is really, nobody’s going to die if you screw up. It’s nice.
3:01:12 It’s not like being a doctor or something where you have to be like, much more responsible and
3:01:17 focused, that you can just freely follow your curiosity, which is just great. But what I’d
3:01:24 say is that, like, I’m learning now about so many things like about how to structure my activities
3:01:32 more and basically say, okay, if I’m going to be emails like the big one that kills me right now,
3:01:40 I’m just constantly like shifting between email and my activities. And what happens is that I
3:01:46 don’t actually get the email. I just look at my email and I get stressed because I’m like,
3:01:50 oh, I have to think about this. Let me get back to it. And I go back to something else. And so
3:01:54 I’ve just got fragmentary memories of everything, right? So what I’m trying to do is set aside a
3:02:00 timer like this is my email time. This is my, you know, writing time. This is my goofing off time.
3:02:07 And so blocking these things off, you give yourself the goofing off time. Sometimes I do that.
3:02:12 And sometimes I have to be flexible, like, okay, I’m definitely not focusing. I’m going to give
3:02:17 myself the downtime and it’s an investment. It’s not like wasting time. It’s an investment
3:02:22 in my attention later on. And I’m very much with Cal Newport on this. He wrote deep work and a lot
3:02:29 of other amazing books. He talks about tasks switching as a sort of the thing that really
3:02:36 destroys productivity. So like, you know, switching, it doesn’t even matter from what to what, but
3:02:42 checking social media, checking email, maybe switching to a phone call and then doing work
3:02:47 and switching, even switching between, if you’re reading a paper, switching from paper to paper
3:02:52 to paper, because like curiosity and whatever the dopamine hit from the attention switch,
3:02:59 like limiting that because otherwise your brain is just not capable to really like load it in,
3:03:05 really do that deep deliberation. I think that’s required to remember things and to really think
3:03:14 through things. Yeah, I mean, you probably see this, I imagine, in AI conferences, but definitely in
3:03:20 neuroscience conferences, it’s now the norm that people have their laptops out during talks.
3:03:26 And, you know, conceivably, they’re writing, you know, they’re writing notes. But in fact,
3:03:31 what often happens if you look at people, we can speak from a little bit of personal experience,
3:03:37 is you’re checking email and you’re like, or I’m working on my own talk, but often it’s like,
3:03:43 you’re doing things that are not paying attention. And I have this illusion, well,
3:03:46 I’m paying attention and then I’m going back. And then what happens is I don’t remember anything
3:03:51 from that day. It just kind of vanished because what happens is I’m creating all these artificial
3:03:56 event boundaries. I’m losing all this executive function. Every time I switch, I’m getting like
3:04:03 a few seconds slower and I’m catching up mentally to what’s happening. And so instead of being in
3:04:09 a model where you’re meaningfully integrating everything and predicting and generating this
3:04:13 kind of like rich model, I’m just catching up, you know. And so, yeah, there’s great research
3:04:20 by Melina Unkafer and Anthony Wagner on multitasking and people can look up that talks about just how
3:04:26 bad it is for memory and, you know, it’s becoming worse and worse of a problem.
3:04:30 So, you’re a musician. Take me through how’d you get into music? Like what made you first
3:04:36 fall in love with music? With creating music? Yeah, so I started playing music just when I was
3:04:42 like doing trumpet in school for a school band and I would just read music and play and, you know,
3:04:49 it was pretty decent at it, not great, but I was decent.
3:04:52 How’d you go from trumpet to guitar, especially the kind of music you’re into?
3:04:58 Yeah, so basically in high school, yeah, so I kind of was a late bloomer to music, but just
3:05:05 kind of MTV grew up with me. I grew up with MTV. And so then you started seeing all this stuff and
3:05:12 then I got into metal was kind of like my early genre. And I always reacted to just things that
3:05:18 were loud and had a beat like ADHD, right? Like, you know, everything from Sergeant Pepper’s by
3:05:26 the Beatles to like Led Zeppelin II, my dad had both, my parents had both those albums,
3:05:32 so I’d listened to them a lot. And then like the police ghosted in the machine and, but then I
3:05:38 got into metal, Def Leppard and, you know, AC/DC Metallica went way down the rabbit hole of speed
3:05:46 metal. And that time was kind of like, “Oh, why don’t I play guitar? I can do this.” And I had
3:05:53 friends who were doing that. And I just never got it. Like I took lessons and stuff like that.
3:05:59 But it was different because when I was doing trumpet, I was reading sheet music. And this was
3:06:04 like, I was learning by looking, there’s a thing called tablature, you know, this where it’s like,
3:06:09 you see like a drawing of the fretboard with numbers. And that’s where you’re supposed to put
3:06:13 your, it’s kind of like paint by numbers, right? And so I learned it in a completely different way,
3:06:20 but I was still terrible at it. And I didn’t get it. It’s actually taken me a long time to
3:06:26 understand exactly what the issue was. But it wasn’t until I really got into punk, and I saw
3:06:31 bands like, I saw Sonic Youth, I remember especially, and it just blew my mind. Because
3:06:37 they violated the rules of what I thought music was supposed to be. I was like,
3:06:41 this doesn’t sound right. These are not power chords. And this isn’t just have like a
3:06:47 shouty verse and then a chorus part. It’s not going back. This is just like weird. And then it
3:06:53 occurred to me, you don’t have to write music the way it’s people tell you it’s supposed to sound.
3:07:00 I just opened up everything for me. And I was playing in a band and I was struggling with writing
3:07:06 music because I would try to write like, you know, whatever was popular at the time and or
3:07:12 whatever sounded like other bands that I was listening to. And somehow I kind of morphed
3:07:16 into just like, just grabbing a guitar and just doing stuff. And I realized a part of my problem
3:07:23 with doing music before was I didn’t enjoy trying to play stuff that other people play. I just enjoyed
3:07:29 music just dripping out of me and just, you know, spilling out and just doing stuff.
3:07:34 And so then I started to say, what if I don’t play a chord? What if I just play like notes that
3:07:40 shouldn’t go together and just mess around with stuff? And I said, well, what if I don’t do four
3:07:45 beats go na na na na, one, two, three, four, one, two, three, four, one, two, three, four, whatever
3:07:49 I go, one, two, three, four, five, one, two, three, four, five, and sort of mess around time
3:07:54 signatures. Then I was playing in this band with a great musician who was really
3:07:59 Brent Ritzel, who was in this band with me. And he taught me about arranging songs. And it was
3:08:04 like, what if we take this part and instead of make it go like back and forth, we make it like a
3:08:09 circle? Or what if we make it like a straight line, you know, or zigzag, you know, just make it like
3:08:15 nonlinear in these interesting ways. And then next, you know, it’s like the whole world sort of
3:08:21 opens up as like the, and then what I started to realize, especially so you could appreciate this
3:08:26 as a musician, I think. So time signatures, right? So we are so brainwashed to think in four-four,
3:08:32 right? Every rock song you could think of almost is in four-four. I know you’re a Floyd fan,
3:08:37 so think of “Money” by Pink Floyd, right? You feel like it’s in four-four because it resolves
3:08:47 itself, but it resolves on the last note of the, basically it resolves on the first note of the
3:08:54 next measure. So it’s got seven beats instead of eight where the riff is actually happening.
3:08:59 Interesting. But you’re thinking in four, because that’s how we use, we’re used to thinking. So
3:09:04 the music flows a little bit faster than it’s supposed to, and you’re getting a little bit
3:09:10 of prediction error every time this is happening. And once I got used to that, I was like, I hate
3:09:17 writing in four-four because I was like, everything just feels better if I do it in seven-four,
3:09:21 if I alternate between four and three and doing all this stuff. And then it’s like, you just,
3:09:26 jazz music is like that. They just do so much interesting stuff with this.
3:09:32 So playing with those time signatures allows you to really break it all open and just,
3:09:36 I guess there’s something about that that allows you to actually have fun.
3:09:40 Yeah, yeah. And it’s like, so I’m actually like a very, one of the genres we used to play
3:09:47 in was math rock. That’s what they called it. It was just like, this is so many weird time
3:09:51 signatures. What is math? Oh, interesting. Yeah. So that’s the math part of rock is what,
3:09:57 the mathematical disturbances of it or what? Yeah. I guess it would be like, so instead of,
3:10:01 you might go like, instead of playing four beats in every measure, no, no, no, no, no, no, no, no,
3:10:06 no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no.
3:10:10 You know what? Just do these things. And then you might arrange it in weird ways so that there
3:10:15 might be three measures of verse and then one, you know, and then five measures, of course,
3:10:22 and then two measures. So you could just mess around with everything, right?
3:10:24 What does that feel like to listen to? There’s, there’s something about symmetry or like patterns
3:10:31 that feel good and like relaxing for us or whatever. It feels like home and disturbing that can be
3:10:38 quite disturbing. Yeah. So is that, is that the feeling you would have if you
3:10:44 math rock? I mean, yeah, yeah. That’s stressing me out. Just listen.
3:10:47 Learning about it. So, I mean, it depends. So a lot of my style of songwriting is very much like,
3:10:55 in terms of like repetitive themes, but messing around with structure, because I’m not a great
3:11:02 guitarist technically. And so I don’t play complicated stuff. And there’s things you can
3:11:06 hear stuff where it’s just like so complicated, you know. But often what I find is, is like having
3:11:13 a melody or, and then adding some dissonance to it, just enough. And then adding some complexity
3:11:20 that gets, gets you going just enough. But I have a high tolerance for, for that kind of dissonance
3:11:26 and prediction. I think I have a theory, a pet theory that it’s like, basically you could explain
3:11:30 most of human behavior as some people are lumpers and some people are splitters, you know. And so
3:11:36 it’s like some people are very kind of excited when they get this dissonance and they want to like go
3:11:42 with it. So people are just like, no, I want to lump everything. You know, I don’t know, maybe
3:11:45 that’s even a different thing. But it’s like, basically, it’s like, I think some people get
3:11:49 scared of that discomfort. And I really drive on it. You know, I love it. What’s, what’s the name
3:11:58 of your band now? The cover band I play in is a band called Pavlov’s Dogs. And so it’s a band,
3:12:07 unsurprisingly, of mostly memory researchers, neuroscientists. I love this. I love this so
3:12:12 much. Yeah, actually, one of your MIT colleagues, Earl Miller, plays bass.
3:12:16 Plays bass. Do you play a rhythm or a leader? You could compete if you want. Maybe we could
3:12:20 audition you. For audition? Oh, yeah. I’m coming for you, Earl. Earl is going to kill me.
3:12:27 He’s like very precise though. I’ll play Triangle or something.
3:12:33 Or where’s the cowbell? I’ll be the cowbell guy. What kind of songs do you guys do?
3:12:39 So it’s mostly late 70s punk and 80s new wave and post punk. Blondie, Ramones, Clash. I do,
3:12:51 I sing Age of Consent by New Order and Loveville Terrace.
3:12:55 And you said you have a female singer now?
3:12:56 Yeah, Carrie Hoppin and also Paula Crox. So Carrie does Blondie amazingly well.
3:13:07 And we do Gigantic by the Pixies. Paula does that one.
3:13:11 Which song do you love to play the most? What kind of song is super fun for you?
3:13:15 Of someone else’s?
3:13:17 Yeah, Cover. And it’s one we do with Pavlov’s Dogs.
3:13:24 I really enjoy playing I Want to Be Your Dog by Iggy and the Stooges.
3:13:28 That’s a good song.
3:13:29 Which is perfect because we’re Pavlov’s Dogs. And Pavlov, of course, was basically
3:13:34 created learning theory. So there’s that. But also, it’s like, but I mean, Iggy and the Stooges,
3:13:39 that song, so I play and sing on it, but it’s just like it devolves into total noise. And I
3:13:44 just fall on the floor and generate feedback. I think in the last version, it might have been
3:13:51 that or a Velvet Underground cover in our last show, I actually, I have a guitar made of aluminum
3:13:56 that I got made. And I thought this thing’s indestructible. So I kind of like was just,
3:14:01 moving it around, had it upside down and all this stuff to generate feedback.
3:14:06 And I think I broke one of the, I broke one of the tuning pegs.
3:14:08 I’ve had to break it all metal guitar. Go figure.
3:14:14 A bit of a big ridiculous question. But let me ask you, we’ve been talking about
3:14:18 neuroscience in general. What do you, you’ve been studying the human mind for a long time.
3:14:25 What do you love most about the human mind? Like when you look at it, we look at the fMRI,
3:14:33 just the scans and the behavioral stuff, the electrodes, you know, the psychology aspect,
3:14:39 reading the literature on the biology side, neurobiology, all of it. When you look at it,
3:14:43 what, what is most like beautiful to you? I think the most beautiful, but
3:14:50 incredibly hard to put your finger on is this idea of the internal model. That it’s like,
3:14:58 there’s everything you see and there’s everything you hear and touch and taste, you know, every
3:15:03 breath you take, whatever. But it’s all connected by this like dark energy that’s holding that whole
3:15:13 universe of your mind together, right? And without that, it’s just a bunch of stuff.
3:15:18 And somehow we put that together and it forms our, so much of our experience and being able
3:15:27 to figure out where that comes from and how things are connected to me is just amazing.
3:15:33 But just this idea of like the world in front of us, we’re only sampling this little bit and
3:15:39 trying to take so much meaning from it. And we do a really good job, not perfect. I mean, you know,
3:15:45 but that ability to me is just amazing. Yeah, it’s an incredible mystery, all of it. It’s fun.
3:15:51 You said dark energy because the same in astrophysics, you look out there, look at dark matter and
3:15:56 dark energy, which is this loose term, a scientific thing we don’t understand, which makes out,
3:16:02 which helps make the equations work in terms of gravity and the expansion of the universe
3:16:08 in the same way. It seems like there’s that kind of thing in the human mind that we’re like
3:16:12 striving to understand. Yeah, yeah. You know, it’s funny that you mentioned that. So one of the
3:16:16 reasons I wrote the book amongst many is that I really felt like people needed to hear from scientists
3:16:22 and like COVID was just a great example of this because like people weren’t hearing from scientists.
3:16:28 One of the things I think that people didn’t get was the uncertainty of science and how much we
3:16:34 don’t know. And I think every scientist lives in this world of uncertainty. And when I was
3:16:41 writing the book, I just became aware of all of these things we don’t know. And so I think of
3:16:47 physics a lot. And I think of this idea of like overwhelming majority of the stuff that’s in our
3:16:55 universe cannot be directly measured. I used to think, “Ha ha, I hate physics.” So physicists get
3:17:01 the Nobel Prize for doing whatever stupid thing. It’s like there’s 10 physicists out there. I’m
3:17:06 just kidding. Strong words. Yeah. No, no, no. I’m kidding. The physicists who do neuroscience
3:17:12 could be rather opinionated. So sometimes I like to dish on that. It’s all love. It’s all love. That’s
3:17:16 right. This is the ADHD talking. But at some point, I had this aha moment where I was like,
3:17:25 to be aware of that much that we don’t know and have a beat on it and be able to go towards it,
3:17:34 that’s one of the biggest scientific successes that I could think of.
3:17:38 You are aware that you don’t know about this gigantic section, overwhelming majority of the
3:17:44 universe, right? And I think the more what keeps me going to some extent is realizing the changing
3:17:54 the scope of the problem and figuring out, “Oh my God, there’s all these things we don’t know.”
3:17:59 And I thought I knew this because science is all about assumptions, right? So have you
3:18:04 read the structure of Scientific Revolutions by Thomas Kuhn? Yes. That’s like my only philosophy
3:18:10 really that I’ve read. But it’s so brilliant in the way that they frame this idea of like,
3:18:16 he frames this idea of assumptions being core to the scientific process and the paradigm shift
3:18:22 comes from changing those assumptions. And this idea of like finding out this kind of whole zone
3:18:28 of what you don’t know to me is the exciting part. Well, you are a great scientist and you wrote an
3:18:37 incredible book. So thank you for doing that. And thank you for talking today. You’ve decreased
3:18:44 the amount of uncertainty I have just a tiny little bit today and revealed the beauty of memory.
3:18:52 This is a fascinating conversation. Thank you for talking today. Oh, thank you. It’s been
3:18:55 blast. Thanks for listening to this conversation with Sharon Ranganath. To support this podcast,
3:19:03 please check out our sponsors in the description. And now let me leave you with some words from
3:19:08 Haruki Murakami. Most things are forgotten over time. Even the word itself, the life and death
3:19:17 struggle people went through is now like something from the distant past. We’re so caught up in our
3:19:23 everyday lives that events of the past are no longer in orbit around our minds. There are just
3:19:29 too many things we have to think about every day, too many new things we have to learn. But still,
3:19:35 no matter how much time passes, no matter what takes place in the interim, there are some things
3:19:42 we can never assign to oblivion, memories we can never rub away. They remain with us forever,
3:19:49 like a touchstone. Thank you for listening. I hope to see you next time.
3:19:55 [Music]