AI transcript
0:00:02 (dramatic music)
0:00:08 Today on Freakin’ Amic Radio, a very special episode,
0:00:11 a conversation about the late Daniel Kahneman,
0:00:13 whose insights into human behavior
0:00:15 have been threaded through this show for years,
0:00:20 ideas like confirmation bias and loss aversion
0:00:22 and the planning fallacy.
0:00:23 During this conversation,
0:00:25 we also learned about a research paradigm
0:00:29 that Kahneman embraced called adversarial collaboration,
0:00:32 which means working shoulder to shoulder with your rivals.
0:00:37 He felt that this is the right way to do science.
0:00:40 Kahneman was a phenomenally influential psychologist
0:00:43 who won a Nobel Prize in Economics,
0:00:46 wrote the bestselling book, Thinking Fast and Slow,
0:00:50 and left behind an army of collaborators,
0:00:53 mentees, and admirers.
0:00:55 With them, we will take a careful look
0:00:58 at the life and mind of Danny Kahneman starting now.
0:01:01 (upbeat music)
0:01:14 This is Freakin’ Amic Radio,
0:01:18 the podcast that explores the hidden side of everything
0:01:20 with your host, Stephen Dovner.
0:01:23 (upbeat music)
0:01:29 Last month in a sunlit auditorium
0:01:31 overlooking the Chicago River,
0:01:34 there was a gathering of psychologists, economists,
0:01:36 and other social scientists.
0:01:38 This was the Behavioral Decision Research
0:01:40 in Management Conference.
0:01:43 The keynote event was supposed to be
0:01:45 a conversation with Danny Kahneman,
0:01:47 facilitated by Richard Thaler,
0:01:50 his longtime friend and collaborator.
0:01:53 Thaler is the University of Chicago Economist
0:01:55 who helped turn Kahneman’s insights
0:01:59 into the field now known as behavioral economics.
0:02:02 But when Kahneman died in March at age 90,
0:02:05 Thaler came up with a new plan for the conference.
0:02:09 Now it would pay tribute to Danny Kahneman.
0:02:12 Freakin’ Amic Radio was lucky enough to be asked along
0:02:15 to moderate a couple of panel discussions
0:02:17 about his life and work.
0:02:18 The episode you’re about to hear
0:02:21 is a condensed version of those conversations.
0:02:24 This all took place at the downtown outpost
0:02:26 of the University of Chicago’s business school
0:02:29 in front of a couple hundred attendees.
0:02:32 Some of the panelists had known Danny Kahneman
0:02:34 for many decades.
0:02:37 For instance, the psychologist Maya Bar-Hillel,
0:02:40 her father was a philosophy professor
0:02:42 at Hebrew University in Jerusalem,
0:02:45 where Kahneman got his undergraduate degree.
0:02:49 – My father taught Danny and gave him a lot of grief,
0:02:52 but my father apparently gave just about everybody
0:02:54 a lot of grief.
0:02:57 He was a tough-minded philosopher.
0:02:59 – And when Kahneman became a professor,
0:03:03 one of his students was Maya Bar-Hillel,
0:03:05 from generation to generation.
0:03:10 – I met Danny in the first week of my first year
0:03:11 at the Hebrew University.
0:03:15 He gave the introductory statistics course.
0:03:18 He looked at us and he pronounced right away,
0:03:21 you are the creme de la creme.
0:03:25 He said it in French and we were.
0:03:28 (audience laughing)
0:03:30 – Kahneman had grown up in France
0:03:32 during the Nazi occupation.
0:03:36 He survived, barely, and lived for many years in Israel
0:03:41 before coming to the US to get his PhD at UC Berkeley.
0:03:43 His research partner, Amos Tversky,
0:03:46 was another Israeli psychology professor
0:03:47 who moved to the States.
0:03:50 Tversky died young in 1996,
0:03:53 too early to share in what would have surely been
0:03:56 a joint Nobel Prize.
0:03:58 Tversky was regarded as perhaps
0:04:01 even more brilliant than Kahneman.
0:04:03 The two of them published many papers
0:04:05 on judgment and decision-making,
0:04:07 but not just in psychology journals.
0:04:12 – It always struck me just in the story
0:04:14 of how Kahneman and Tversky research
0:04:17 was taken hold of by Thaler and others
0:04:20 and turned into what we now call behavioral economics.
0:04:22 That it was very, very important
0:04:23 that these were two psychologists
0:04:26 who were also very mathematically fluent.
0:04:28 If that hadn’t been the case
0:04:30 and if the publication hadn’t been in,
0:04:32 I guess, econometrica and so on,
0:04:35 was there a pretty good possibility for a counterfactual
0:04:36 where all that research might have stayed
0:04:39 within the realm of psychology and never trickled over
0:04:41 and we might not have what we think
0:04:42 of as behavioral economics?
0:04:44 – It was not an accident.
0:04:48 It’s not how fortunate that they went to econometrica.
0:04:50 They realized that their work
0:04:53 was attended to primarily by psychologists
0:04:56 and in fact, they both considered themselves
0:04:59 all their lives as psychologists,
0:05:02 but they also realized that their research
0:05:06 was perhaps more important outside of psychology.
0:05:10 So the decision to publish their paper in econometrica
0:05:11 was a deliberate move.
0:05:15 It was a strategic move to get the attention.
0:05:17 They believed that that was the ticket
0:05:18 and without the ticket,
0:05:20 they would not have been playing in that field.
0:05:24 – A lot of the early Kahneman-Tversky work
0:05:26 centered around an observation
0:05:29 that may seem obvious in retrospect,
0:05:32 but at the time had not been explored with much rigor.
0:05:36 The idea was that we are all constantly making decisions,
0:05:39 personal, professional, political decisions.
0:05:41 And then later, if we ask ourselves
0:05:44 why we made a given decision,
0:05:47 which by the way, we usually don’t ask,
0:05:50 we might tell ourselves a story about making the decision,
0:05:55 but these stories are often not quite true.
0:05:56 Why?
0:05:59 One reason is that we want to appear to others
0:06:03 and maybe even ourselves as smarter than we are.
0:06:05 Here is Richard Taylor.
0:06:09 – I met Danny and Amos in 1977
0:06:13 and it was a transformative year for me.
0:06:17 I decided to change jobs and took a job at Cornell
0:06:19 and decided to offer a course
0:06:22 in the thing I was now fascinated by
0:06:26 and called it behavioral decision theory,
0:06:30 which is kind of what the name of this field used to be.
0:06:32 I got about eight students.
0:06:36 So I had to think of something to do
0:06:39 to increase the enrollment in the class.
0:06:44 And so I changed the name to managerial decision making.
0:06:49 50 students show up.
0:06:54 I began the class by asking how many of you
0:06:58 signed up for this class because of the name?
0:07:00 No one raised their hand.
0:07:03 I said, well, actually nearly all of you.
0:07:08 That’s one illustration of what Danny’s talking about.
0:07:12 No one thinks they would be stupid enough
0:07:15 to sign up for a class based on the name.
0:07:17 – I believe it was in thinking fast and slow
0:07:19 where Danny wrote, not only are we blind,
0:07:21 but we are blind to our blindness.
0:07:26 – Yeah, and you know, maybe one of the many secrets
0:07:31 of the Conor Monteverski collaboration was
0:07:34 they were not blind to those.
0:07:39 And what they had was a mistake detection facility.
0:07:45 They had some radar where they could anticipate
0:07:48 what the mistake is.
0:07:52 And because they had this mistake detection facility
0:07:57 somewhere, they were able to figure out things.
0:08:00 And then they were pretty good at predicting
0:08:04 what people other than Amos and Danny would do.
0:08:09 To me, the big point, the aha point that I got
0:08:12 from reading their judgment papers
0:08:15 was the idea of predictable bias.
0:08:17 – May I add something to that?
0:08:18 – Yes, please.
0:08:20 – Not only predictable.
0:08:23 I think now I’m gonna say something that is perhaps,
0:08:25 I can’t say that I heard them say it,
0:08:28 but I hope they would both agree,
0:08:30 that the errors are not just predictable.
0:08:33 The errors are smart.
0:08:36 Our stupid mistakes are evidence
0:08:38 of the wonderful human mind.
0:08:42 That’s how normal cognition functions
0:08:45 and normal cognition is amazing.
0:08:46 – I love that.
0:08:48 It does make me wanna ask any, all of you,
0:08:50 a question that’s fairly heretical,
0:08:52 which is, you know, as a layperson,
0:08:56 I read these findings, they’re attractive,
0:08:59 they’re believable, they’re identifiable.
0:09:01 But the thing that I always struggle with
0:09:02 or would like to understand better
0:09:06 is how you all can feel so confident
0:09:08 that they’re generalizable.
0:09:10 There must be people in the world
0:09:12 who are not susceptible to loss aversion
0:09:14 or recency bias or many, many of them.
0:09:18 So really the question is, how much variance is there
0:09:20 and how do you measure the variance?
0:09:23 – You know, Amos used to have a joke
0:09:28 that there were species that didn’t exhibit loss aversion
0:09:30 and they’re now extinct.
0:09:33 (audience laughing)
0:09:38 – Think about what Maya Bar-Huelel was saying there
0:09:42 about the smart errors we all make.
0:09:45 It would be easy to overlook the baseline insight
0:09:46 that Danny Kahneman offered,
0:09:51 that people make thinking mistakes all the time.
0:09:52 Now, most of us, upon hearing this,
0:09:54 might say, no kidding.
0:09:59 Our species is highly fallible, who doesn’t know that?
0:10:03 But it is our fallibility, Kahneman realized,
0:10:07 that makes us interesting and worthy of inspection.
0:10:12 He accepted that humans are capable of wonderful things
0:10:14 as well as terrible ones,
0:10:17 but that we are overconfident in our abilities,
0:10:20 that we often have poor self-control,
0:10:24 and that we employ an arsenal of mental shortcuts
0:10:28 or heuristics to make decisions or reach judgments
0:10:30 that often turn out poorly.
0:10:33 And even when presented with evidence of our mistakes,
0:10:36 we usually fail to change our minds.
0:10:38 Here is Eldar Shafir,
0:10:40 who runs the Kahneman-Triesman Center
0:10:43 for Behavioral Science and Public Policy at Princeton.
0:10:47 He and Kahneman used to co-teach a class.
0:10:49 – One lecture that I love that Danny used to give
0:10:52 in our course, we would talk about people’s
0:10:55 not good sense about conditional probabilities.
0:11:01 And then we had a clip of the O.J. Simpson trial
0:11:05 where Dershowitz explains to the jury
0:11:08 that the probability that a beaten woman
0:11:11 is gonna be killed by her beating partner
0:11:14 is extremely low, which is true.
0:11:16 However, we’re not predicting the chance
0:11:17 that Nicole Simpson will be killed.
0:11:19 She has been killed.
0:11:22 The probability that a beaten woman who has been killed
0:11:25 was killed by her partner is immensely high.
0:11:28 That nuance of not being sensitive to these conditionals
0:11:30 has major implications.
0:11:37 He was so genuinely curious and intellectually alive,
0:11:40 he presented the same way to the Swedish monarchy
0:11:42 and to an undergraduate.
0:11:43 He just wanted to think about it.
0:11:46 He wanted to struggle with the question.
0:11:49 He wanted to listen and think of the best theory.
0:11:50 That was what’s so impressive.
0:11:52 When we talk together, we took turns lecturing.
0:11:56 Here was Danny, I mean, before and after the Nobel,
0:11:58 and every class he would come and sit down
0:11:59 with the students and listen.
0:12:00 He could easily not have shown up.
0:12:01 It’s other people’s lecturing.
0:12:03 He was always there, asking questions,
0:12:05 answering questions, devoted to understanding.
0:12:07 Always had a new thought about something
0:12:10 that we have done on slides for three years in a row.
0:12:12 (gentle music)
0:12:14 And many things were written about Kahneman.
0:12:16 I’m always amused by the fact that they say
0:12:19 he was aware of these quirks and fallacies,
0:12:21 but he was aware that he has them too.
0:12:23 And that sort of implies that since he was aware,
0:12:25 it wasn’t as big a deal.
0:12:28 There’s a couple of recorded interviews with Danny
0:12:30 during COVID, where he predicts
0:12:33 that COVID means life imprisonment
0:12:36 for the more advanced in age.
0:12:37 And he was wrong.
0:12:39 He lived a couple of wonderful years after COVID.
0:12:41 There was a sense in which he really sank deep
0:12:45 into the same error that he was able to predict
0:12:46 and acknowledge.
0:12:48 The other amazing thing about Danny
0:12:52 is when he won the Nobel Prize, you have to give a lecture.
0:12:56 And he gave a lecture and decided he was going
0:13:01 to give a new interpretation of all of his work with Amos.
0:13:06 Using this two system approach.
0:13:10 I kept saying, Danny, you have two months to do this.
0:13:11 Why are you starting over?
0:13:15 They gave you a Nobel Prize for what you did.
0:13:18 You don’t need a new reinterpretation,
0:13:23 but that lecture in Stockholm was the beginning
0:13:26 of what led to thinking fast and slow
0:13:31 because he was reinterpreting all the heuristics
0:13:36 and biases stuff in the lens of system one
0:13:40 and system two, which is your immediate reaction
0:13:43 versus when you sit down and think about it.
0:13:44 – What this story suggests to me,
0:13:47 but tell me if I’m wrong, is that he understood
0:13:51 that having an audience like the Nobel committee audience
0:13:54 and the King or Queen or whatever of Sweden
0:13:56 and that his voice was now amplified
0:13:58 that he wanted to take the work that he’d done
0:14:01 and make it accessible, make it known
0:14:03 to people who have some levers of power.
0:14:07 He wanted to exploit a good opportunity.
0:14:09 – I don’t like the word exploit.
0:14:12 – Exploit for pro-social reasons, a good opportunity.
0:14:14 – Thank you, thank you.
0:14:17 – Yeah, I think he had an audience
0:14:19 and wanted to get it out there,
0:14:24 but it was torture for him, absolute torture.
0:14:29 I mean, during the writing of Thinking Fast and Slow.
0:14:31 – Which was a long gestation.
0:14:34 – Long gestation.
0:14:38 And he decided absolutely positively
0:14:41 that he was quitting two dozen times.
0:14:44 – You know, Danny in some sense really was a bon vivant.
0:14:45 I mean, he enjoyed life.
0:14:47 He loved a good restaurant.
0:14:49 He loved a good vacation.
0:14:51 And I think he loved getting the prize.
0:14:53 – I don’t think he changed.
0:14:57 I mean, yes, the world was paying more attention to him,
0:14:59 but he was always Danny.
0:15:00 – He changed.
0:15:01 – He was happier.
0:15:02 – Okay.
0:15:02 (audience laughing)
0:15:03 – No, I would like to say–
0:15:06 – If Maya says that, I accept.
0:15:10 – I would like to say in what way he relaxed.
0:15:11 – That was relaxed?
0:15:13 (audience laughing)
0:15:15 – Yes, yes.
0:15:16 – What do you mean by that, Maya?
0:15:17 – He had made it.
0:15:20 There was a common misperception
0:15:23 that the Tversky-Kahaneman collaboration
0:15:26 was not symmetrical.
0:15:31 The world seemed to think that Amos was the lead
0:15:34 and Danny was an outside visitor.
0:15:39 It was a symmetrical and equal collaboration,
0:15:41 but the world didn’t know it.
0:15:44 It was very important to Danny
0:15:48 to be able to do good work after Amos dies
0:15:51 so that people won’t continue with this error
0:15:53 that he was less than equal.
0:15:57 So the prize told the world,
0:16:00 this man is noble worthy,
0:16:02 and he did excellent work
0:16:04 because he had an excellent mind,
0:16:08 but also it was important for him
0:16:13 that Kahaneman not die the name together with Tversky,
0:16:14 and it didn’t.
0:16:17 – As I said earlier,
0:16:19 the main ideas of Danny Kahneman
0:16:20 have been threaded through the fabric
0:16:22 of Freakonomics Radio over the years.
0:16:25 Episode 323, for instance,
0:16:27 which was about the planning fallacy.
0:16:30 It’s called Here’s Why Your Projects Are Always Late
0:16:31 and What to Do About It.
0:16:34 Then there’s episode 271,
0:16:36 The Men Who Started a Thinking Revolution.
0:16:38 That was an interview with Michael Lewis
0:16:40 about the book he wrote
0:16:42 on the Kahneman-Tversky partnership.
0:16:45 Lewis’ book is called The Undoing Project,
0:16:47 a friendship that changed our minds.
0:16:50 So yes, you may be familiar
0:16:51 with Kahneman’s greatest hits,
0:16:55 but one of the conversations we had at this conference
0:16:58 went into an area I knew very little about,
0:17:00 and I bet you don’t either.
0:17:02 That’s coming up after the break.
0:17:02 I’m Stephen Dubner,
0:17:04 and this is Freakonomics Radio.
0:17:07 (dramatic music)
0:17:15 – Good morning.
0:17:17 Good morning.
0:17:21 We have a session this morning,
0:17:24 as you know, on adversarial collaboration.
0:17:26 – Day two of the conference,
0:17:28 with a panel devoted to a different way
0:17:32 of doing business in the behavioral sciences.
0:17:34 Let’s begin with a short clip
0:17:36 from the late Danny Kahneman.
0:17:39 This was recorded in 2022.
0:17:43 – Controversy is a terrible way to advance science.
0:17:45 It is normally conducted as a contest,
0:17:47 and with the aim is to embarrass.
0:17:49 The feature that makes most critiques
0:17:52 intellectually useless is a focused
0:17:54 on the weakest argument of the adversary.
0:17:56 It is common for critics
0:17:59 to include a summary caricature of the target position,
0:18:02 refute the weakest argument in that caricature,
0:18:04 and declare the total destruction
0:18:06 of the adversary’s position.
0:18:10 It’s rare for anyone to concede anything.
0:18:14 Doing angry science is a demeaning experience.
0:18:17 – Doing angry science, Kahneman came to believe,
0:18:19 was a terrible thing.
0:18:21 He knew this firsthand.
0:18:26 Before the accolades, the prizes, the acceptance,
0:18:30 Kahneman and Tversky came under a great deal of criticism.
0:18:32 – Kurt Gigerendzer published his first critique
0:18:34 of our work 37 years ago,
0:18:36 and he’s still not done with me.
0:18:39 – And here again is Richard Thaler.
0:18:41 – I’ve certainly had my share
0:18:44 of angry science exchanges.
0:18:48 They’re no fun, and there’s never any light, only heat.
0:18:50 – And that’s why Kahneman came up with a different model,
0:18:54 what he called an adversarial collaboration.
0:18:58 – He felt that this is the right way to do science.
0:19:03 All of the collaborations shed more light than heat.
0:19:08 The typical debates in academia are exactly the opposite.
0:19:11 So I think Danny has a good point.
0:19:13 – Kahneman was trying to solve a fundamental problem
0:19:15 in social science research.
0:19:18 It’s natural for scientists to disagree with one another,
0:19:21 but there’s no clear mechanism
0:19:23 for resolving those disagreements.
0:19:28 So what if researchers who disagree work together
0:19:31 in good faith to resolve the issue?
0:19:34 And what if they take on a neutral third party
0:19:36 to serve as an arbiter?
0:19:40 That is what Kahneman called an adversarial collaboration.
0:19:43 He participated in a variety of them over the years.
0:19:45 The conversation you are about to hear
0:19:48 includes some of the key players.
0:19:50 Let’s start with introductions.
0:19:51 – I’m Tom Gilovich.
0:19:55 I’m a social psychologist at Cornell University.
0:19:58 – I’m Barbara Millers from the University of Pennsylvania.
0:20:02 And I was fortunate to be the arbiter
0:20:07 in two adversarial collaborations with Danny, 20 years apart.
0:20:07 – I’m Matt Killingsworth.
0:20:10 I’m also at the University of Pennsylvania.
0:20:12 And I study human happiness and all being,
0:20:13 like what makes life worth living
0:20:15 and how do we collect data to try to understand
0:20:17 what makes life better?
0:20:18 – My name is Shane Frederick.
0:20:21 I am a professor at the Yale School of Management.
0:20:23 (audience applauding)
0:20:25 – Tom Gilovich, let’s start with you.
0:20:26 Let’s hear basically the story
0:20:29 of the adversarial collaboration with Danny.
0:20:31 It began quite early in your career.
0:20:33 Give us the whole story.
0:20:36 – I had published with Vicky Medvick a chapter
0:20:38 in a book on counterfactual thinking.
0:20:42 When I wrote the chapter, I don’t know if I was aware
0:20:45 that Danny was going to write the wrap up chapter
0:20:47 where he comments on all of the other ones.
0:20:50 So I don’t know if his voice was in my head
0:20:52 as we’re writing the chapter.
0:20:53 – What’s the chapter about?
0:20:54 What’s it cover?
0:20:56 – A lot of it covers the subject of regret,
0:20:58 what we regret most in life.
0:21:01 Amos and Danny had published a study showing
0:21:04 you regret things of action more than inaction.
0:21:07 The example they used is imagine you own stock
0:21:10 in one company, you think, okay, that’s run the course.
0:21:12 You switch to another and it tanks,
0:21:14 you lose a certain amount of money.
0:21:17 Versus you’re thinking about buying this stock,
0:21:20 you decide not to, it takes off
0:21:21 and you lose the same amount of money.
0:21:23 Which would you regret more?
0:21:25 And almost everyone anticipates
0:21:28 that you’d regret the action the most.
0:21:30 Nonetheless, if you ask people,
0:21:32 what are the biggest regrets in your life?
0:21:35 What dominates are things they didn’t do?
0:21:37 And so a lot of the chapter was
0:21:40 how do you reconcile those two things?
0:21:43 – In approaching this riddle or puzzle in your mind,
0:21:47 how did you measure what was your actual research?
0:21:50 Our research was what’s responsible for that difference?
0:21:52 How do things change over time?
0:21:57 Precisely because your regrets of action really hurt.
0:21:58 You do things about them.
0:22:00 Sometimes you change your life accordingly
0:22:04 or you certainly engage in lots of psychological work
0:22:07 to achieve some level of peace.
0:22:11 Because the inaction regrets gnaw at you less powerfully,
0:22:13 you sort of leave them alone.
0:22:16 The fun of that research was tracking down
0:22:19 all of the different psychological processes
0:22:24 that make those regrets of inaction hang around
0:22:27 or in some cases even get more intense.
0:22:31 Whereas the regrets of action dissipate.
0:22:32 – How do you do that though?
0:22:33 Is it empirical?
0:22:34 Is it theoretical?
0:22:35 – It’s empirical.
0:22:37 For example, one of the mechanisms
0:22:39 that we thought was especially interesting
0:22:42 is when you don’t do something
0:22:44 that you later on end up regretting.
0:22:46 There are reasons for that.
0:22:48 I just don’t have the bandwidth right now
0:22:50 to take this project on, et cetera.
0:22:53 Those are compelling reasons in the moment
0:22:55 for why you’re not doing this thing.
0:22:58 You move along in time, you look back and you think,
0:23:00 wait a minute, I could have done that.
0:23:02 So we would do studies where we’d ask
0:23:05 Cornell current students, recent alums,
0:23:07 or much older alums.
0:23:10 Imagine there’s a class that you always wanted to take
0:23:12 but maybe you were a little afraid to.
0:23:14 Suppose we added that to your curriculum
0:23:15 this semester, the students are like,
0:23:18 oh, I can barely hang on right now.
0:23:20 You add that, it’s a catastrophe.
0:23:23 You ask the people who graduated many years ago,
0:23:25 they think, oh, that’s a piece of cake.
0:23:27 I wouldn’t have interfered with my GPA, my social life,
0:23:29 the amount of sleep I get, et cetera.
0:23:31 – Okay, so you write up the findings.
0:23:35 At what point then and in what way do you hear from Danny?
0:23:37 – It’s sort of a highlight of my career.
0:23:41 I was on sabbatical, check in your voice messages
0:23:44 back at Cornell and I get this message.
0:23:47 Tom, this is Danny Kahneman.
0:23:50 I’ve just read your paper with Medvic, it’s brilliant.
0:23:54 It made my day just calling to say thank you.
0:23:56 What a piece of work.
0:24:01 And then I stupidly press star delete
0:24:03 instead of saving that message forever
0:24:04 to play to my grandkids.
0:24:08 So I’m just on a high for a while.
0:24:11 Two weeks later, there’s another phone call
0:24:13 from Danny Kahneman, this time live.
0:24:17 Oh, great, my new best friend is gonna tell me about…
0:24:20 Tom, I’ve been thinking more about your chapter.
0:24:24 It’s all wrong.
0:24:28 I’ve heard you’re planning to publish this.
0:24:30 You can’t do that.
0:24:33 If you do, people will go after you.
0:24:35 And if no one does, I’m gonna go after you.
0:24:39 What happened to Brilliant?
0:24:42 Did he, in that conversation, give you his argument
0:24:44 for why you thought now you were wrong?
0:24:47 Yes, he said that there really isn’t a change
0:24:51 in the intensity of either action or inaction regrets.
0:24:54 There’s just a substitution of the kinds of things
0:24:55 that you’re regretting.
0:24:59 And the regrets of inaction really aren’t serious regrets.
0:25:01 You might not even call them regrets.
0:25:02 They’re more wistful.
0:25:06 Like, oh, I wish I had learned to speak Esperanto.
0:25:07 Said no one ever.
0:25:09 Yeah. (laughing)
0:25:11 All right, so what happens next?
0:25:13 Well, lots of panic for a few days.
0:25:16 Vicki Medvick and I, what do we do?
0:25:18 It can’t be worse than this.
0:25:22 And I think, I forget what there had been an example
0:25:24 of an adversarial collaboration.
0:25:25 I think it was Vicki’s idea.
0:25:27 Hey, we should negotiate with him.
0:25:30 Let’s see if we can do an adversarial collaboration.
0:25:33 So we pitched it and he was very receptive.
0:25:35 This was in what year, Tom?
0:25:38 Chapter came out in ’95, so it was a little before that.
0:25:42 So this was pre-zoom, certainly, early internet.
0:25:43 How are you communicating?
0:25:45 And how did you negotiate the negotiation?
0:25:47 Then how did the actual negotiation happen?
0:25:49 I’m glad you asked that question
0:25:51 ’cause it allows me to bring up a point about Danny
0:25:55 that there’s been all this talk about his many qualities
0:25:57 and they’ve all been well said.
0:25:59 One thing kind of gets left out.
0:26:02 I mean, to be at that level of success,
0:26:05 you have to be brilliant and creative
0:26:07 and many people have spoken about that.
0:26:10 You just have to be a phenomenally hard worker.
0:26:13 And man, he was on task all the time.
0:26:15 It was amazingly easy to get that guy on the phone.
0:26:17 I could get him on the phone more readily
0:26:19 than I could get my co-author, Vicki, on the phone.
0:26:21 And something seemed wrong about that.
0:26:23 He was always ready to engage
0:26:26 and so a lot of it happened over the phone.
0:26:29 And the idea was, no, these regrets of inaction
0:26:32 that you say are important in people’s later lives.
0:26:34 They’re just sort of wistful regrets
0:26:38 and the intense hot emotions that come from action.
0:26:39 Those are different kinds of things.
0:26:42 So we designed some studies where we asked people
0:26:45 to think about the biggest regret of action or inaction.
0:26:47 You designed the studies together.
0:26:48 Then we ran them together, yes.
0:26:52 So think of your biggest regrets of action or inaction
0:26:56 from the recent past or the distant past.
0:26:58 How many of these emotions do they lead you to feel?
0:27:02 We had a set of five hot emotions,
0:27:05 a set of five wistful emotions.
0:27:08 We thought that those long-term regrets of inaction,
0:27:09 some of them are wistful.
0:27:12 I wish I had learned to play the piano,
0:27:14 but some of them are really intense.
0:27:17 You met the right person at the wrong time
0:27:20 and so you let it pass and now you’re looking back like,
0:27:22 oh, my life would have been much better
0:27:25 if I had acted on that.
0:27:30 We also asked people to rate whether a set of five,
0:27:32 I think we called them emotions of despair.
0:27:35 I’m depressed when I think about this.
0:27:37 I feel empty when I think about this.
0:27:40 And lo and behold, Danny was right.
0:27:43 A lot of long-term regrets of inaction
0:27:46 are kind of wistful, but we were right too.
0:27:50 They also produced these big, powerful feelings of,
0:27:53 oh, my life is not what it could have been.
0:27:56 I feel empty and depressed thinking about these things
0:28:00 that I didn’t do that now seem so easy to have done.
0:28:02 – Did Danny, as a result,
0:28:04 acknowledge that you were partially right
0:28:06 as much as he acknowledged that he was partially right?
0:28:08 – Yeah, yeah, and that’s in the paper.
0:28:09 – So truly a happy ending.
0:28:10 – I believe so, yeah.
0:28:13 Certainly more happy than that second phone call
0:28:14 that I got to.
0:28:15 (audience laughing)
0:28:17 – You then went on to collaborate with Danny
0:28:18 several times, correct?
0:28:19 – Yes.
0:28:22 – I’m just curious now that Danny is gone
0:28:24 and since your initial contact
0:28:27 was this adversarial collaboration about regret,
0:28:30 is there anything that you now regret
0:28:33 not having worked on with Danny?
0:28:36 – I wouldn’t say that there’s a particular topic in mind,
0:28:40 but to have someone in your life like that
0:28:42 and not take full advantage of it,
0:28:44 I wish I had just reached out to him more
0:28:47 to have more contact with him.
0:28:50 One of the nice things about this event here
0:28:54 is this family feeling of all the people
0:28:56 he reached, some of whom I know well,
0:28:59 some of whom I didn’t really know at all until now.
0:29:01 They feel like family.
0:29:03 So, you know, often with families,
0:29:06 you wish you had spent more time with them.
0:29:08 – Not my family, but, you know.
0:29:13 (audience laughing and applauding)
0:29:16 – The adversarial collaboration with Tom Gilovich
0:29:19 was one of the earliest ones that Danny Kahneman undertook.
0:29:22 Let’s take a look now at one of the most recent
0:29:25 with Matt Killingsworth from the University of Pennsylvania.
0:29:28 He studied engineering as an undergraduate
0:29:32 and got his PhD in psychology in 2012.
0:29:37 Danny Kahneman got his psychology PhD in 1961.
0:29:39 So, how did Matt Killingsworth end up
0:29:41 in an adversarial collaboration
0:29:43 with this giant in the field?
0:29:46 – Danny and Angus Deaton, both Nobel Prize winners,
0:29:49 had this conclusion that there was sort of this plateau
0:29:52 in people’s happiness once they reached $75,000 in income
0:29:54 and then I’d published a paper
0:29:56 that basically showed something completely different
0:29:59 and we tried to figure out who’s right and what’s the truth.
0:30:01 – You say it with this sort of sang froide
0:30:04 that, you know, we published something completely different,
0:30:06 but as you noted, you were publishing a paper
0:30:08 in opposition to the findings of not one,
0:30:10 but two Nobel laureates.
0:30:12 Were you as calm and cool about it in the moment
0:30:15 when you decided to take up this route
0:30:16 as you seem to be now?
0:30:18 – I mean, partly Danny is to blame,
0:30:22 although he didn’t learn that before he passed away.
0:30:24 I had written this as like a sub point to a sub point
0:30:26 and another paper and actually reading
0:30:29 Michael Lewis’s book about Danny and Angus,
0:30:32 they were talking about how they took on existing ideas
0:30:35 and that was an important part of their intellectual journey.
0:30:37 And so I sort of rolled the dice and thought,
0:30:38 “Well, I’ll try it.”
0:30:40 But yeah, there was definitely some part
0:30:42 in the back of my mind of like, how is this gonna go?
0:30:45 I do have to ask at this particular conference,
0:30:47 which is held in celebration
0:30:49 of Danny Kahneman’s work in life,
0:30:51 do you feel a little bit like, you know,
0:30:52 the wolf in the hen house
0:30:55 or that someone’s gonna come up and shiv you in the back
0:30:56 at the buffet?
0:30:57 – I hope not.
0:31:00 No, I mean, we really had a wonderful collaboration
0:31:03 and I think part of what characterized that is,
0:31:05 we both just wanted to figure out the truth.
0:31:07 I don’t think anyone was attached
0:31:10 to any particular version of reality.
0:31:13 My sense from him is that he was a little bit irritated,
0:31:14 not a lot.
0:31:15 – A lot is a lot.
0:31:17 I mean, he wanted to have a good conversation,
0:31:18 but he wasn’t–
0:31:22 – For the record, that was Shane Frederick saying a lot.
0:31:24 (laughing)
0:31:26 Barb Millers, let’s bring you in here.
0:31:30 How did you get attached to this adversarial collaboration?
0:31:32 – Well, I was talking to Danny on the phone,
0:31:35 not too long after Matt’s paper came out
0:31:37 and he said, “Oh, by the way,
0:31:40 “do you happen to know a guy by the name
0:31:42 “of Matt Killingsworth?”
0:31:45 And I said, “Well, as a matter of fact,
0:31:47 “I do, and if you’d like me to be the arbiter,
0:31:49 “I’d be happy to do so.”
0:31:51 (laughing)
0:31:54 In adversarial collaborations,
0:31:57 the arbiter is the research assistant,
0:32:03 the tiebreaker, and occasionally the therapist.
0:32:07 – So Matt, how would you characterize the response
0:32:10 to that original Kahneman-Deaton finding
0:32:13 about the $75,000 happiness cutoff,
0:32:16 whether inside academia or beyond?
0:32:17 – I mean, it was very influential,
0:32:20 and I mean, it’s probably one of the most visible,
0:32:22 kind of an exciting findings.
0:32:26 I think a lot of us feel like money is sort of overemphasized
0:32:29 in our daily lives, and it kind of gives a justification
0:32:31 for maybe carrying a little bit less about it
0:32:33 and focusing more on all the other stuff
0:32:35 that’s also really important for happiness,
0:32:37 particularly because money is quantitative,
0:32:39 you can kind of think of them as points.
0:32:40 I’m like, “Well, I wanna get more,”
0:32:42 but you can easily imagine a sort of trap
0:32:44 where you’re just continuously trying to get
0:32:46 more and more and more of those points,
0:32:47 ignoring all of this other stuff,
0:32:48 and wouldn’t it be nice
0:32:50 if we could kind of step out of that cycle?
0:32:52 At least that’s part of the reason
0:32:54 that I think that that was so attractive.
0:32:56 – So Matt, you’ve rehearsed very nicely
0:32:59 the original finding, now bring you into the story here.
0:33:00 What are you doing at the moment?
0:33:03 – Sure, so partly also due to Danny,
0:33:05 my research program is really centered
0:33:08 on large-scale experience sampling,
0:33:10 and to get at what that matters in the original study,
0:33:12 those data were collected, how?
0:33:14 In the original study, they basically asked,
0:33:16 “Did you experience a lot of the following emotion
0:33:18 “yesterday, yes or no?”
0:33:20 And then it had a series of emotions
0:33:24 like sadness, happiness, stress, et cetera.
0:33:26 People would either say, “I did or I didn’t.”
0:33:27 – And the technology or mechanism
0:33:29 of harvesting those data was what?
0:33:31 – I wasn’t involved in that data collection.
0:33:34 I believe those were verbal phone calls,
0:33:36 and then sort of interviewing people over the phone.
0:33:38 – Does anyone here know anything different,
0:33:40 or does that sound right, as far as we know?
0:33:41 – That’s right.
0:33:44 That was said with the voice of a referee, I have to say.
0:33:45 (audience laughs)
0:33:47 Okay, Matt, continue, please.
0:33:50 So in my study, I’m essentially measuring
0:33:52 people’s experience right in the moment.
0:33:55 They’re carrying around on their phone an app,
0:33:57 and I’m beeping them at random times.
0:33:59 I’m asking them, “How do you feel right now?”
0:34:00 And they’re responding on a scale
0:34:03 that ranges from very bad to very good,
0:34:04 and the scale is continuous.
0:34:06 So there are a couple of things
0:34:09 that distinguish that from the earlier paper.
0:34:10 One is that it’s right at the time
0:34:11 that people are feeling it,
0:34:14 as opposed to the day as a whole, retrospectively.
0:34:16 And the other is that gradations of feelings
0:34:17 are on a continuous scale,
0:34:20 as opposed to something that’s binary or dichotomous.
0:34:22 – And the scale is what to what?
0:34:24 It’s zero to 10, zero to 100.
0:34:26 – Very bad to very good, and it’s just continuous.
0:34:27 – So you can rake in anywhere you want.
0:34:29 – There are hundreds of unique values.
0:34:30 – Got it.
0:34:31 – And describe the differences
0:34:34 between the pools of research subjects
0:34:35 in the original case and in your case?
0:34:38 – In the original case, it was a survey by Gallup.
0:34:40 It was either representative or at least an attempt
0:34:43 to be representative of whatever random digit dialing was.
0:34:46 That’s a relative strength of their paper
0:34:47 and of that study in general.
0:34:50 My sample, in contrast, was really a convenient sample.
0:34:52 It’s turned out to be an amazing sample
0:34:53 of people that have really beautiful results,
0:34:56 but it was essentially whoever was willing to sign up
0:34:58 to try to understand their own happiness.
0:35:01 – What was the recruiting mechanism?
0:35:03 – Largely thanks to folks like you,
0:35:06 people in a hearing about it in the media,
0:35:07 on the radio, reading about it.
0:35:09 I think that’s interesting.
0:35:10 I’d like to learn about that for myself.
0:35:12 I’m willing to contribute to science.
0:35:14 And it turns out that when I look at,
0:35:15 what’s the distribution of incomes?
0:35:17 For example, in my data,
0:35:19 it’s almost a perfect match for the US census.
0:35:21 I can replicate all kinds of things
0:35:23 that we’ve seen in the literature for decades.
0:35:25 So it turns out to be really good,
0:35:27 but that’s kind of a lucky coincidence.
0:35:32 – Okay, so you gather, assemble, analyze your data,
0:35:34 talk about from the analysis point
0:35:36 to writing up the findings.
0:35:38 What I find when I look at this relationship,
0:35:40 plotting people’s happiness in the moment
0:35:43 versus their income, it just keeps going up.
0:35:45 This sort of critical point
0:35:47 where they had found in the earlier paper,
0:35:50 this flatlining, I really see no difference at all.
0:35:52 And when I write it up in the ultimate paper,
0:35:55 I compare, well, what’s the slope below the point
0:35:56 where they said it stops increasing?
0:35:57 And the point above that,
0:35:59 the slopes differ by less than 1%.
0:36:02 I really see no evidence for a difference at all.
0:36:05 Were you initially looking for that plateau in the data?
0:36:06 – I was really just trying to understand
0:36:08 what’s the relationship between these things
0:36:10 that are obviously important.
0:36:13 We’ve never had, in the moment, continuous data.
0:36:15 And so I want to see, what does this look like?
0:36:17 And it turns out, well, it doesn’t look like
0:36:18 what we thought it looked like.
0:36:23 – When you strike at a king, you must kill him.
0:36:26 That’s from Ralph Waldo Emerson.
0:36:28 Matt Killingsworth did strike
0:36:31 at the king of his realm, Danny Kahneman.
0:36:33 But what happened next?
0:36:35 It’s coming up after the break.
0:36:38 I’m Stephen Dovner, and this is Freakonomics Rating.
0:36:48 (upbeat music)
0:36:52 We’ve been hearing from Matt Killingsworth
0:36:55 about his research on the relationship
0:36:57 between income and happiness.
0:37:00 Research that disputed elements of earlier research
0:37:03 on the same topic by Daniel Kahneman.
0:37:05 – So you write up the paper,
0:37:09 and the paper, not quite directly,
0:37:11 but quasi-directly, addresses the fact
0:37:14 that your finding is contra
0:37:18 to a significant earlier finding by significant scholars.
0:37:20 What happens next?
0:37:22 – There’s a fair amount of attention about it and so forth.
0:37:24 Maybe a month or two afterwards,
0:37:29 I get a note from Barb that she’s been chatting with Danny.
0:37:31 – He’s talking about Barb Mellors,
0:37:33 a longtime friend of Danny Kahneman,
0:37:36 as well as a University of Pennsylvania colleague
0:37:37 of Matt Killingsworth.
0:37:39 – I think both of our attitudes
0:37:41 was we just wanna find out the truth.
0:37:43 I have no personal attachment
0:37:45 to what I found particularly,
0:37:48 and I don’t, other than his initial perhaps irritation,
0:37:50 I don’t really think Danny did.
0:37:52 It was really just what do we think is going on.
0:37:55 – Did you envision that perhaps you would then collaborate
0:37:56 with Danny on a joint study?
0:37:58 – That certainly seemed to be the case
0:37:59 once we got into it,
0:38:02 but these kinds of data took me many years
0:38:04 to collect that data for them.
0:38:06 There were a client on an external organization
0:38:07 with a lot of resources,
0:38:10 and so we ended up doing it by going back
0:38:11 and looking at my data, which existed,
0:38:13 which made it tractable.
0:38:15 I can sort of cut to the chase a little bit
0:38:16 of how we did that if you want.
0:38:18 – I wanna go back to Barb for just a second here.
0:38:21 Barb, if you could give Danny’s perspective
0:38:23 or participation up to the point here
0:38:25 where we’re about to cut to the data.
0:38:29 – The starting point for Danny was the assumption
0:38:32 that both data sets were valid.
0:38:35 So how could they be so different?
0:38:39 There must be a resolution in the data somewhere,
0:38:42 and so the task was going to be,
0:38:44 Matt became the research assistant
0:38:48 and started doing all the re-analyses on his data.
0:38:51 – How would you characterize the tenor
0:38:54 of this adversarial collaboration?
0:38:57 – Really civil and nice,
0:38:59 and just the way it should go.
0:39:01 – So Matt, you said you could cut to the chase.
0:39:03 We’re at the chase now, cut to it.
0:39:05 – Essentially, the resolution was
0:39:07 if we look at the range
0:39:10 within my continuous happiness data,
0:39:11 what if we look at the low end,
0:39:13 which is the part where their measure
0:39:14 would have been sensitive,
0:39:16 can we find a similar pattern?
0:39:18 And lo and behold, when you zero in on that,
0:39:20 rather than looking at the average trend,
0:39:21 which keeps going up,
0:39:24 you get not exactly, but very, very close
0:39:25 to what they found,
0:39:27 certainly much closer to what they found
0:39:29 than to what the average trend looks like.
0:39:31 We both found that pretty convincing evidence
0:39:34 that what they found is true,
0:39:36 there’s nothing wrong with the analysis at all,
0:39:38 but it was really a question of,
0:39:40 how generally applicable is that?
0:39:42 So from low to medium incomes,
0:39:46 the unhappiest part of the distribution rises a lot,
0:39:47 and then from medium to high incomes,
0:39:51 the unhappiest part of the distribution barely changes.
0:39:55 But the rest of the distribution is rising steadily,
0:39:58 and in fact, at the high end of the happiness distribution,
0:39:59 you get an inversion of that.
0:40:01 So rather than a rise and then a plateau,
0:40:05 you have sort of a shallow slope and then an acceleration.
0:40:07 – So in my lay mind, as I’m trying to process all this,
0:40:12 what I’m envisioning is that there is a cohort of people
0:40:15 who are kind of cranky,
0:40:19 and they may experience a little bit less crankiness
0:40:20 as income rises,
0:40:24 but there is a range in which their temperament
0:40:26 may be overwhelms their income.
0:40:28 – If anything, there’s one of the steepest slopes
0:40:31 for the unhappiest people in the range of low income.
0:40:33 So getting out of poverty, if you’re really miserable,
0:40:35 is at least correlationally.
0:40:36 – But that’s almost a different story,
0:40:39 ’cause escaping poverty versus going from middle to upper.
0:40:43 – If you aren’t poor and you’re really miserable,
0:40:45 at least if you sort of extrapolate from there,
0:40:46 it seems to not matter past that.
0:40:49 And probably as we speculate in the paper,
0:40:51 at that point, if you have a decent amount of money
0:40:52 and you’re really unhappy,
0:40:54 whatever’s making you unhappy probably isn’t due
0:40:55 to the lack of resources.
0:40:58 It’s something else going on in your life.
0:41:00 Maybe it’s challenges in a family relationship,
0:41:02 or maybe you’re depressed, or whatever it is,
0:41:05 it’s something going on that perhaps money
0:41:07 isn’t going to make a difference.
0:41:09 – Richard, Taylor, let me ask you a question.
0:41:12 You’ve just heard Matt’s presentation of his side
0:41:13 of the original research,
0:41:15 and then the adversarial collaboration,
0:41:16 and then the conclusion,
0:41:19 all of which I found honestly to be really fascinating.
0:41:21 You know the literature well.
0:41:24 You too have a Nobel Prize,
0:41:26 although I have heard that was a clerical error.
0:41:27 (audience laughing)
0:41:30 How would you, looking at this from above,
0:41:32 Barbara was there as an arbiter,
0:41:34 Matt was one side,
0:41:37 Danny was another side, but is not present,
0:41:39 if you could give an ultimate proclamation on,
0:41:41 not just where this finding arrived,
0:41:43 but what the adversarial collaboration produced here,
0:41:45 what would you say?
0:41:47 – Look, I think this is the way it should work.
0:41:49 This is a good story,
0:41:52 and the world would be a better place,
0:41:54 academia would be a much better place
0:41:58 if there were more of these kinds of collaborations.
0:42:01 – Would it affect what we sometimes
0:42:04 call the replication crisis?
0:42:06 – Uh, oof.
0:42:08 – Barbara says, Barbara’s nodding her head, no,
0:42:10 you say oof.
0:42:12 – Yeah, let’s move on to Shane.
0:42:15 (audience laughing)
0:42:19 – The discomfort you were hearing there
0:42:21 about the replication crisis,
0:42:24 that discomfort is related to a two-part series
0:42:27 we recently produced on academic fraud,
0:42:31 episodes 572 and 573.
0:42:33 And while we’re on the topic of discomfort,
0:42:35 let me add one thing I’m thinking about
0:42:36 as we’re in the middle of this conversation
0:42:39 about adversarial collaborations.
0:42:42 Danny Kahneman is said to have introduced this concept
0:42:45 into the realm of behavioral research,
0:42:47 but if you’re not a behavioralist,
0:42:49 you could look at this as yet another example
0:42:53 of how academic researchers discover something
0:42:56 that people in the real world have been doing forever.
0:43:00 There are all sorts of arbiters and referees out there,
0:43:05 all sorts of processes for compromise and negotiation,
0:43:08 even war games to test one plan against another
0:43:10 and come up with the best solution.
0:43:15 In the same vein, some people look at the big insights
0:43:17 that have come out of this behavioralist research
0:43:22 as essentially ESOP’s fables with a little bit of math.
0:43:24 Earlier this year,
0:43:27 we made a series about the late physicist Richard Feynman.
0:43:30 He didn’t think the social sciences like psychology
0:43:34 and even economics should be called science at all.
0:43:38 He thought they were just too squishy to deserve that name.
0:43:41 And as for psychologists discovering the idea
0:43:43 of adversarial collaboration,
0:43:47 well, Feynman was a junior member of the Manhattan Project,
0:43:50 which in its quest to build an atomic bomb
0:43:53 brought together the most brilliant
0:43:57 and argumentative group of scientists ever assembled.
0:43:59 That was adversarial, and yes,
0:44:03 it was in the end a successful collaboration.
0:44:06 Anyway, back to Chicago.
0:44:09 As Richard Thaler suggests, let’s move on.
0:44:12 Shane Frederick, he is a behavioral scientist at Yale
0:44:16 who collaborated with Danny Kahneman on several projects
0:44:18 and is cited throughout Kahneman’s book,
0:44:20 Thinking Fast and Slow.
0:44:22 Kahneman saw Frederick as a protege
0:44:24 and they had a close relationship,
0:44:27 sometimes a bit too close.
0:44:29 – So I’m at 10 o’clock on a Monday,
0:44:31 maybe like third quarter of Monday football.
0:44:32 I get a call.
0:44:35 – Shane, you said, are you watching the game?
0:44:36 – It was one of the great times I wasn’t.
0:44:38 I said, no, fantastic.
0:44:39 I want to talk about heuristics.
0:44:42 (audience laughing)
0:44:45 Another time, it was like two o’clock in the morning,
0:44:47 going back and forth, back and forth, back and forth.
0:44:49 And thanks, I’m getting sleepy.
0:44:52 And so, go to bed and I wake up in the morning
0:44:54 in the very next email from Danny,
0:44:56 “Shane, don’t play dead on me.”
0:44:59 (audience laughing)
0:45:01 – One idea they worked on together
0:45:04 was the cognitive reflection test, or CRT.
0:45:06 It is meant to measure a person’s ability
0:45:09 to override their gut instinct
0:45:11 and think more carefully about a problem.
0:45:13 Here’s a famous example.
0:45:16 A bat and a ball cost $1.10 in total.
0:45:19 The bat costs $1 more than the ball.
0:45:22 How much does the ball cost?
0:45:26 If your first inclination is to say, 10 cents,
0:45:30 congratulations, you were like just about everyone else.
0:45:32 What the CRT measures is,
0:45:35 can you slow down and actually calculate the answer
0:45:37 rather than just go with your gut?
0:45:42 If you can, you will find that the answer is five cents.
0:45:47 Remember, the bat costs $1 more than the ball.
0:45:49 Shane Frederick has spent much of his career
0:45:52 designing such tests.
0:45:53 – Some of the items are novel, some are invented,
0:45:56 many of them I didn’t, I collect, some I’m revised.
0:45:59 One goes back to at least 1919, their variance of it.
0:46:00 – They were used then in an academic setting,
0:46:01 or like an employment?
0:46:03 – No, just like in books and riddles,
0:46:04 stuff that Maya would read.
0:46:05 (audience laughing)
0:46:06 – But they weren’t stumpers.
0:46:08 – No, they weren’t stumpers.
0:46:13 – I think I must tell the audience what a stumper is, okay?
0:46:16 – That again is Maya Bar-Huelal.
0:46:19 – A one-way ride costs $10,
0:46:22 a round trip costs $20,
0:46:26 a passenger hands the cashier $20,
0:46:28 saying absolutely nothing.
0:46:30 The cashier knows right away
0:46:32 that the passenger wants a round trip
0:46:35 rather than a one-way and change.
0:46:39 And the question is, how did the cashier know this?
0:46:42 – Does anyone in the audience who does not know the answer
0:46:46 to this stumper previously wanna take a guess to the answer?
0:46:50 – If you’re stumped good,
0:46:52 because that’s why they’re called stumpers.
0:46:57 – A stumper is unlike one of Shane Frederick’s CRT questions
0:46:59 in that your intuition doesn’t produce
0:47:01 an obvious but wrong answer.
0:47:04 – It only has to do with my love of riddles,
0:47:07 which I share with my partner in this research,
0:47:08 Shane Frederick.
0:47:12 And we really came to this out of love of riddles,
0:47:15 but we’re both professional psychologists.
0:47:19 So we felt like we had to approach it
0:47:21 from the point of view of psychology.
0:47:22 Now, Danny.
0:47:23 – Wait, wait.
0:47:24 – Oh.
0:47:25 – Answer please.
0:47:27 – Oh, no way.
0:47:29 No, I’m not gonna answer.
0:47:32 That’s not what I was paid for.
0:47:34 (audience laughs)
0:47:36 – All right, I will answer.
0:47:39 He handed the ticket agent two 10s.
0:47:41 – Ah, very nice.
0:47:43 – Did they pay you for that?
0:47:45 (audience laughs)
0:47:47 – I’m getting paid the same as you, Maya.
0:47:50 (audience laughs)
0:47:51 – But there was something more at stake here
0:47:53 than answering a riddle.
0:47:55 Barr Hillel, along with a co-author,
0:47:58 wrote a paper arguing that cognitive reflection tests
0:48:02 are not a good measurement of anything beyond math skills.
0:48:07 And this argument led to an adversarial collaboration.
0:48:09 – When I heard that Shane and Maya and Danny
0:48:12 were having an adversarial collaboration,
0:48:14 I didn’t know who was on which team.
0:48:18 Maya and Danny go back 60 years.
0:48:21 Shane is like his son for Danny.
0:48:22 – The results of this collaboration
0:48:24 have not yet been published,
0:48:26 but Shane Frederick says that he and Kahneman
0:48:30 essentially proved that the CRT is a good measure
0:48:34 of cognitive abilities beyond just math skills.
0:48:35 – It seemed to do quite well,
0:48:37 and it’s doing well consistently.
0:48:38 Time preferences, risk preferences,
0:48:39 other sorts of things.
0:48:41 – And how would you characterize the nature
0:48:42 of the collaboration?
0:48:44 How adversarial was it?
0:48:48 – It wasn’t so adversarial between the opposing groups.
0:48:50 It’s just like everybody’s fighting with everybody else,
0:48:51 but everything.
0:48:54 (audience laughs)
0:48:56 – More like a family gathering, right?
0:48:57 (audience laughs)
0:49:00 – Does this suggest that an adversarial collaboration
0:49:02 is not as useful as one might hope
0:49:04 if the actual adversaries are not often
0:49:06 in the ring with the collaborators?
0:49:09 – I mean, I don’t think it will work.
0:49:10 Do you, Barb?
0:49:13 – Do these things work?
0:49:15 Well, what does work mean?
0:49:16 It depends on your definition.
0:49:20 If you think people change their minds all the way,
0:49:21 no, it doesn’t work.
0:49:26 If they change their minds a bit, that’s good.
0:49:29 And it speeds up science too.
0:49:31 Somebody’s looking over your shoulder
0:49:34 and making sure you’re not making mistakes,
0:49:36 you’re defining variables more precisely,
0:49:38 you’re designing an experiment
0:49:41 that gets right at the core issue.
0:49:42 It’s the way to go.
0:49:45 (piano music)
0:49:47 – Well, I could listen to the five of you talk
0:49:49 for five hours.
0:49:50 There are sessions that need to happen.
0:49:52 This room is turning over.
0:49:53 I thank you so much.
0:49:54 This was a great conversation.
0:49:58 (audience applauds)
0:50:00 – Sorry, there is a clause in the contract
0:50:02 that Thaler must always have every last word, so.
0:50:05 – I was just gonna say something nice about you,
0:50:06 but if you insist, I’ll skip it.
0:50:09 (audience laughs)
0:50:13 – And that was, again, Richard Thaler.
0:50:16 I would like to thank him for putting this event together
0:50:18 and inviting us to join.
0:50:21 Thanks also to the other participants.
0:50:23 It was a pretty wonderful event
0:50:26 and I’m glad we were able to share it with you.
0:50:29 And if you need more Richard Thaler in your life,
0:50:31 and who doesn’t need more Thaler in their lives,
0:50:34 we are going to publish a bonus episode very soon,
0:50:36 an update of a great conversation
0:50:38 I had with Thaler a few years ago.
0:50:42 It’s called People Aren’t Dumb, The World is Hard.
0:50:44 So keep your ears out for that.
0:50:47 Meanwhile, on the next regularly scheduled episode
0:50:51 of Freakonomics Radio, a close-up look at an industry
0:50:54 that’s all about close-up looking.
0:50:58 I like for my glasses to have a bit of pizzazz,
0:51:00 especially if you’re wearing them every day.
0:51:03 – We’re gonna see half of the global population
0:51:05 being myopic by 2015.
0:51:07 – When you are a vertically integrated player,
0:51:10 you master every step within the value chain.
0:51:14 – The margins, even by luxury good standards, are obscene.
0:51:17 – Eyewear is a $150 billion industry
0:51:21 and what you see is not quite what you get.
0:51:23 That’s next time on the show.
0:51:25 Until then, take care of yourself,
0:51:28 and if you can, someone else too.
0:51:33 Freakonomics Radio is produced by Stitcher and Renbud Radio.
0:51:36 You can find our entire archive on any podcast app
0:51:38 also at Freakonomics.com,
0:51:41 where we publish transcripts and show notes.
0:51:43 This episode was produced by Zach Lipinski,
0:51:46 with live recording by Greg Rippen.
0:51:49 Special thanks to conference organizers Amy Boonstra,
0:51:51 Mark Tomelko, and Chris Partridge,
0:51:54 as well as the Black Oak AV team.
0:51:57 Our staff also includes Alina Cullman, Augusta Chapman,
0:51:59 Dalvin Abouaji, Eleanor Osborn,
0:52:02 Elsa Hernandez, Gabriel Roth, Jasmine Klinger,
0:52:04 Jeremy Johnston, Julie Canfer,
0:52:06 Lyric Bowditch, Morgan Levy, Neil Coruth,
0:52:10 Rebecca Lee Douglas, Sarah Lilly, and Teo Jacobs.
0:52:12 Our theme song is “Mr. Fortune”
0:52:13 by the Hitchhikers.
0:52:16 Our composer is Luis Guerra.
0:52:18 As always, thank you for listening.
0:52:20 – I think my mind was wandering.
0:52:23 I have that effect on people, I’ve been told.
0:52:25 (audience laughing)
0:52:28 (upbeat music)
0:52:31 – The Freakonomics Radio Network,
0:52:33 the hidden side of everything.
0:52:36 (upbeat music)
0:52:37 Stitcher.
0:52:40 (gentle music)
0:00:08 Today on Freakin’ Amic Radio, a very special episode,
0:00:11 a conversation about the late Daniel Kahneman,
0:00:13 whose insights into human behavior
0:00:15 have been threaded through this show for years,
0:00:20 ideas like confirmation bias and loss aversion
0:00:22 and the planning fallacy.
0:00:23 During this conversation,
0:00:25 we also learned about a research paradigm
0:00:29 that Kahneman embraced called adversarial collaboration,
0:00:32 which means working shoulder to shoulder with your rivals.
0:00:37 He felt that this is the right way to do science.
0:00:40 Kahneman was a phenomenally influential psychologist
0:00:43 who won a Nobel Prize in Economics,
0:00:46 wrote the bestselling book, Thinking Fast and Slow,
0:00:50 and left behind an army of collaborators,
0:00:53 mentees, and admirers.
0:00:55 With them, we will take a careful look
0:00:58 at the life and mind of Danny Kahneman starting now.
0:01:01 (upbeat music)
0:01:14 This is Freakin’ Amic Radio,
0:01:18 the podcast that explores the hidden side of everything
0:01:20 with your host, Stephen Dovner.
0:01:23 (upbeat music)
0:01:29 Last month in a sunlit auditorium
0:01:31 overlooking the Chicago River,
0:01:34 there was a gathering of psychologists, economists,
0:01:36 and other social scientists.
0:01:38 This was the Behavioral Decision Research
0:01:40 in Management Conference.
0:01:43 The keynote event was supposed to be
0:01:45 a conversation with Danny Kahneman,
0:01:47 facilitated by Richard Thaler,
0:01:50 his longtime friend and collaborator.
0:01:53 Thaler is the University of Chicago Economist
0:01:55 who helped turn Kahneman’s insights
0:01:59 into the field now known as behavioral economics.
0:02:02 But when Kahneman died in March at age 90,
0:02:05 Thaler came up with a new plan for the conference.
0:02:09 Now it would pay tribute to Danny Kahneman.
0:02:12 Freakin’ Amic Radio was lucky enough to be asked along
0:02:15 to moderate a couple of panel discussions
0:02:17 about his life and work.
0:02:18 The episode you’re about to hear
0:02:21 is a condensed version of those conversations.
0:02:24 This all took place at the downtown outpost
0:02:26 of the University of Chicago’s business school
0:02:29 in front of a couple hundred attendees.
0:02:32 Some of the panelists had known Danny Kahneman
0:02:34 for many decades.
0:02:37 For instance, the psychologist Maya Bar-Hillel,
0:02:40 her father was a philosophy professor
0:02:42 at Hebrew University in Jerusalem,
0:02:45 where Kahneman got his undergraduate degree.
0:02:49 – My father taught Danny and gave him a lot of grief,
0:02:52 but my father apparently gave just about everybody
0:02:54 a lot of grief.
0:02:57 He was a tough-minded philosopher.
0:02:59 – And when Kahneman became a professor,
0:03:03 one of his students was Maya Bar-Hillel,
0:03:05 from generation to generation.
0:03:10 – I met Danny in the first week of my first year
0:03:11 at the Hebrew University.
0:03:15 He gave the introductory statistics course.
0:03:18 He looked at us and he pronounced right away,
0:03:21 you are the creme de la creme.
0:03:25 He said it in French and we were.
0:03:28 (audience laughing)
0:03:30 – Kahneman had grown up in France
0:03:32 during the Nazi occupation.
0:03:36 He survived, barely, and lived for many years in Israel
0:03:41 before coming to the US to get his PhD at UC Berkeley.
0:03:43 His research partner, Amos Tversky,
0:03:46 was another Israeli psychology professor
0:03:47 who moved to the States.
0:03:50 Tversky died young in 1996,
0:03:53 too early to share in what would have surely been
0:03:56 a joint Nobel Prize.
0:03:58 Tversky was regarded as perhaps
0:04:01 even more brilliant than Kahneman.
0:04:03 The two of them published many papers
0:04:05 on judgment and decision-making,
0:04:07 but not just in psychology journals.
0:04:12 – It always struck me just in the story
0:04:14 of how Kahneman and Tversky research
0:04:17 was taken hold of by Thaler and others
0:04:20 and turned into what we now call behavioral economics.
0:04:22 That it was very, very important
0:04:23 that these were two psychologists
0:04:26 who were also very mathematically fluent.
0:04:28 If that hadn’t been the case
0:04:30 and if the publication hadn’t been in,
0:04:32 I guess, econometrica and so on,
0:04:35 was there a pretty good possibility for a counterfactual
0:04:36 where all that research might have stayed
0:04:39 within the realm of psychology and never trickled over
0:04:41 and we might not have what we think
0:04:42 of as behavioral economics?
0:04:44 – It was not an accident.
0:04:48 It’s not how fortunate that they went to econometrica.
0:04:50 They realized that their work
0:04:53 was attended to primarily by psychologists
0:04:56 and in fact, they both considered themselves
0:04:59 all their lives as psychologists,
0:05:02 but they also realized that their research
0:05:06 was perhaps more important outside of psychology.
0:05:10 So the decision to publish their paper in econometrica
0:05:11 was a deliberate move.
0:05:15 It was a strategic move to get the attention.
0:05:17 They believed that that was the ticket
0:05:18 and without the ticket,
0:05:20 they would not have been playing in that field.
0:05:24 – A lot of the early Kahneman-Tversky work
0:05:26 centered around an observation
0:05:29 that may seem obvious in retrospect,
0:05:32 but at the time had not been explored with much rigor.
0:05:36 The idea was that we are all constantly making decisions,
0:05:39 personal, professional, political decisions.
0:05:41 And then later, if we ask ourselves
0:05:44 why we made a given decision,
0:05:47 which by the way, we usually don’t ask,
0:05:50 we might tell ourselves a story about making the decision,
0:05:55 but these stories are often not quite true.
0:05:56 Why?
0:05:59 One reason is that we want to appear to others
0:06:03 and maybe even ourselves as smarter than we are.
0:06:05 Here is Richard Taylor.
0:06:09 – I met Danny and Amos in 1977
0:06:13 and it was a transformative year for me.
0:06:17 I decided to change jobs and took a job at Cornell
0:06:19 and decided to offer a course
0:06:22 in the thing I was now fascinated by
0:06:26 and called it behavioral decision theory,
0:06:30 which is kind of what the name of this field used to be.
0:06:32 I got about eight students.
0:06:36 So I had to think of something to do
0:06:39 to increase the enrollment in the class.
0:06:44 And so I changed the name to managerial decision making.
0:06:49 50 students show up.
0:06:54 I began the class by asking how many of you
0:06:58 signed up for this class because of the name?
0:07:00 No one raised their hand.
0:07:03 I said, well, actually nearly all of you.
0:07:08 That’s one illustration of what Danny’s talking about.
0:07:12 No one thinks they would be stupid enough
0:07:15 to sign up for a class based on the name.
0:07:17 – I believe it was in thinking fast and slow
0:07:19 where Danny wrote, not only are we blind,
0:07:21 but we are blind to our blindness.
0:07:26 – Yeah, and you know, maybe one of the many secrets
0:07:31 of the Conor Monteverski collaboration was
0:07:34 they were not blind to those.
0:07:39 And what they had was a mistake detection facility.
0:07:45 They had some radar where they could anticipate
0:07:48 what the mistake is.
0:07:52 And because they had this mistake detection facility
0:07:57 somewhere, they were able to figure out things.
0:08:00 And then they were pretty good at predicting
0:08:04 what people other than Amos and Danny would do.
0:08:09 To me, the big point, the aha point that I got
0:08:12 from reading their judgment papers
0:08:15 was the idea of predictable bias.
0:08:17 – May I add something to that?
0:08:18 – Yes, please.
0:08:20 – Not only predictable.
0:08:23 I think now I’m gonna say something that is perhaps,
0:08:25 I can’t say that I heard them say it,
0:08:28 but I hope they would both agree,
0:08:30 that the errors are not just predictable.
0:08:33 The errors are smart.
0:08:36 Our stupid mistakes are evidence
0:08:38 of the wonderful human mind.
0:08:42 That’s how normal cognition functions
0:08:45 and normal cognition is amazing.
0:08:46 – I love that.
0:08:48 It does make me wanna ask any, all of you,
0:08:50 a question that’s fairly heretical,
0:08:52 which is, you know, as a layperson,
0:08:56 I read these findings, they’re attractive,
0:08:59 they’re believable, they’re identifiable.
0:09:01 But the thing that I always struggle with
0:09:02 or would like to understand better
0:09:06 is how you all can feel so confident
0:09:08 that they’re generalizable.
0:09:10 There must be people in the world
0:09:12 who are not susceptible to loss aversion
0:09:14 or recency bias or many, many of them.
0:09:18 So really the question is, how much variance is there
0:09:20 and how do you measure the variance?
0:09:23 – You know, Amos used to have a joke
0:09:28 that there were species that didn’t exhibit loss aversion
0:09:30 and they’re now extinct.
0:09:33 (audience laughing)
0:09:38 – Think about what Maya Bar-Huelel was saying there
0:09:42 about the smart errors we all make.
0:09:45 It would be easy to overlook the baseline insight
0:09:46 that Danny Kahneman offered,
0:09:51 that people make thinking mistakes all the time.
0:09:52 Now, most of us, upon hearing this,
0:09:54 might say, no kidding.
0:09:59 Our species is highly fallible, who doesn’t know that?
0:10:03 But it is our fallibility, Kahneman realized,
0:10:07 that makes us interesting and worthy of inspection.
0:10:12 He accepted that humans are capable of wonderful things
0:10:14 as well as terrible ones,
0:10:17 but that we are overconfident in our abilities,
0:10:20 that we often have poor self-control,
0:10:24 and that we employ an arsenal of mental shortcuts
0:10:28 or heuristics to make decisions or reach judgments
0:10:30 that often turn out poorly.
0:10:33 And even when presented with evidence of our mistakes,
0:10:36 we usually fail to change our minds.
0:10:38 Here is Eldar Shafir,
0:10:40 who runs the Kahneman-Triesman Center
0:10:43 for Behavioral Science and Public Policy at Princeton.
0:10:47 He and Kahneman used to co-teach a class.
0:10:49 – One lecture that I love that Danny used to give
0:10:52 in our course, we would talk about people’s
0:10:55 not good sense about conditional probabilities.
0:11:01 And then we had a clip of the O.J. Simpson trial
0:11:05 where Dershowitz explains to the jury
0:11:08 that the probability that a beaten woman
0:11:11 is gonna be killed by her beating partner
0:11:14 is extremely low, which is true.
0:11:16 However, we’re not predicting the chance
0:11:17 that Nicole Simpson will be killed.
0:11:19 She has been killed.
0:11:22 The probability that a beaten woman who has been killed
0:11:25 was killed by her partner is immensely high.
0:11:28 That nuance of not being sensitive to these conditionals
0:11:30 has major implications.
0:11:37 He was so genuinely curious and intellectually alive,
0:11:40 he presented the same way to the Swedish monarchy
0:11:42 and to an undergraduate.
0:11:43 He just wanted to think about it.
0:11:46 He wanted to struggle with the question.
0:11:49 He wanted to listen and think of the best theory.
0:11:50 That was what’s so impressive.
0:11:52 When we talk together, we took turns lecturing.
0:11:56 Here was Danny, I mean, before and after the Nobel,
0:11:58 and every class he would come and sit down
0:11:59 with the students and listen.
0:12:00 He could easily not have shown up.
0:12:01 It’s other people’s lecturing.
0:12:03 He was always there, asking questions,
0:12:05 answering questions, devoted to understanding.
0:12:07 Always had a new thought about something
0:12:10 that we have done on slides for three years in a row.
0:12:12 (gentle music)
0:12:14 And many things were written about Kahneman.
0:12:16 I’m always amused by the fact that they say
0:12:19 he was aware of these quirks and fallacies,
0:12:21 but he was aware that he has them too.
0:12:23 And that sort of implies that since he was aware,
0:12:25 it wasn’t as big a deal.
0:12:28 There’s a couple of recorded interviews with Danny
0:12:30 during COVID, where he predicts
0:12:33 that COVID means life imprisonment
0:12:36 for the more advanced in age.
0:12:37 And he was wrong.
0:12:39 He lived a couple of wonderful years after COVID.
0:12:41 There was a sense in which he really sank deep
0:12:45 into the same error that he was able to predict
0:12:46 and acknowledge.
0:12:48 The other amazing thing about Danny
0:12:52 is when he won the Nobel Prize, you have to give a lecture.
0:12:56 And he gave a lecture and decided he was going
0:13:01 to give a new interpretation of all of his work with Amos.
0:13:06 Using this two system approach.
0:13:10 I kept saying, Danny, you have two months to do this.
0:13:11 Why are you starting over?
0:13:15 They gave you a Nobel Prize for what you did.
0:13:18 You don’t need a new reinterpretation,
0:13:23 but that lecture in Stockholm was the beginning
0:13:26 of what led to thinking fast and slow
0:13:31 because he was reinterpreting all the heuristics
0:13:36 and biases stuff in the lens of system one
0:13:40 and system two, which is your immediate reaction
0:13:43 versus when you sit down and think about it.
0:13:44 – What this story suggests to me,
0:13:47 but tell me if I’m wrong, is that he understood
0:13:51 that having an audience like the Nobel committee audience
0:13:54 and the King or Queen or whatever of Sweden
0:13:56 and that his voice was now amplified
0:13:58 that he wanted to take the work that he’d done
0:14:01 and make it accessible, make it known
0:14:03 to people who have some levers of power.
0:14:07 He wanted to exploit a good opportunity.
0:14:09 – I don’t like the word exploit.
0:14:12 – Exploit for pro-social reasons, a good opportunity.
0:14:14 – Thank you, thank you.
0:14:17 – Yeah, I think he had an audience
0:14:19 and wanted to get it out there,
0:14:24 but it was torture for him, absolute torture.
0:14:29 I mean, during the writing of Thinking Fast and Slow.
0:14:31 – Which was a long gestation.
0:14:34 – Long gestation.
0:14:38 And he decided absolutely positively
0:14:41 that he was quitting two dozen times.
0:14:44 – You know, Danny in some sense really was a bon vivant.
0:14:45 I mean, he enjoyed life.
0:14:47 He loved a good restaurant.
0:14:49 He loved a good vacation.
0:14:51 And I think he loved getting the prize.
0:14:53 – I don’t think he changed.
0:14:57 I mean, yes, the world was paying more attention to him,
0:14:59 but he was always Danny.
0:15:00 – He changed.
0:15:01 – He was happier.
0:15:02 – Okay.
0:15:02 (audience laughing)
0:15:03 – No, I would like to say–
0:15:06 – If Maya says that, I accept.
0:15:10 – I would like to say in what way he relaxed.
0:15:11 – That was relaxed?
0:15:13 (audience laughing)
0:15:15 – Yes, yes.
0:15:16 – What do you mean by that, Maya?
0:15:17 – He had made it.
0:15:20 There was a common misperception
0:15:23 that the Tversky-Kahaneman collaboration
0:15:26 was not symmetrical.
0:15:31 The world seemed to think that Amos was the lead
0:15:34 and Danny was an outside visitor.
0:15:39 It was a symmetrical and equal collaboration,
0:15:41 but the world didn’t know it.
0:15:44 It was very important to Danny
0:15:48 to be able to do good work after Amos dies
0:15:51 so that people won’t continue with this error
0:15:53 that he was less than equal.
0:15:57 So the prize told the world,
0:16:00 this man is noble worthy,
0:16:02 and he did excellent work
0:16:04 because he had an excellent mind,
0:16:08 but also it was important for him
0:16:13 that Kahaneman not die the name together with Tversky,
0:16:14 and it didn’t.
0:16:17 – As I said earlier,
0:16:19 the main ideas of Danny Kahneman
0:16:20 have been threaded through the fabric
0:16:22 of Freakonomics Radio over the years.
0:16:25 Episode 323, for instance,
0:16:27 which was about the planning fallacy.
0:16:30 It’s called Here’s Why Your Projects Are Always Late
0:16:31 and What to Do About It.
0:16:34 Then there’s episode 271,
0:16:36 The Men Who Started a Thinking Revolution.
0:16:38 That was an interview with Michael Lewis
0:16:40 about the book he wrote
0:16:42 on the Kahneman-Tversky partnership.
0:16:45 Lewis’ book is called The Undoing Project,
0:16:47 a friendship that changed our minds.
0:16:50 So yes, you may be familiar
0:16:51 with Kahneman’s greatest hits,
0:16:55 but one of the conversations we had at this conference
0:16:58 went into an area I knew very little about,
0:17:00 and I bet you don’t either.
0:17:02 That’s coming up after the break.
0:17:02 I’m Stephen Dubner,
0:17:04 and this is Freakonomics Radio.
0:17:07 (dramatic music)
0:17:15 – Good morning.
0:17:17 Good morning.
0:17:21 We have a session this morning,
0:17:24 as you know, on adversarial collaboration.
0:17:26 – Day two of the conference,
0:17:28 with a panel devoted to a different way
0:17:32 of doing business in the behavioral sciences.
0:17:34 Let’s begin with a short clip
0:17:36 from the late Danny Kahneman.
0:17:39 This was recorded in 2022.
0:17:43 – Controversy is a terrible way to advance science.
0:17:45 It is normally conducted as a contest,
0:17:47 and with the aim is to embarrass.
0:17:49 The feature that makes most critiques
0:17:52 intellectually useless is a focused
0:17:54 on the weakest argument of the adversary.
0:17:56 It is common for critics
0:17:59 to include a summary caricature of the target position,
0:18:02 refute the weakest argument in that caricature,
0:18:04 and declare the total destruction
0:18:06 of the adversary’s position.
0:18:10 It’s rare for anyone to concede anything.
0:18:14 Doing angry science is a demeaning experience.
0:18:17 – Doing angry science, Kahneman came to believe,
0:18:19 was a terrible thing.
0:18:21 He knew this firsthand.
0:18:26 Before the accolades, the prizes, the acceptance,
0:18:30 Kahneman and Tversky came under a great deal of criticism.
0:18:32 – Kurt Gigerendzer published his first critique
0:18:34 of our work 37 years ago,
0:18:36 and he’s still not done with me.
0:18:39 – And here again is Richard Thaler.
0:18:41 – I’ve certainly had my share
0:18:44 of angry science exchanges.
0:18:48 They’re no fun, and there’s never any light, only heat.
0:18:50 – And that’s why Kahneman came up with a different model,
0:18:54 what he called an adversarial collaboration.
0:18:58 – He felt that this is the right way to do science.
0:19:03 All of the collaborations shed more light than heat.
0:19:08 The typical debates in academia are exactly the opposite.
0:19:11 So I think Danny has a good point.
0:19:13 – Kahneman was trying to solve a fundamental problem
0:19:15 in social science research.
0:19:18 It’s natural for scientists to disagree with one another,
0:19:21 but there’s no clear mechanism
0:19:23 for resolving those disagreements.
0:19:28 So what if researchers who disagree work together
0:19:31 in good faith to resolve the issue?
0:19:34 And what if they take on a neutral third party
0:19:36 to serve as an arbiter?
0:19:40 That is what Kahneman called an adversarial collaboration.
0:19:43 He participated in a variety of them over the years.
0:19:45 The conversation you are about to hear
0:19:48 includes some of the key players.
0:19:50 Let’s start with introductions.
0:19:51 – I’m Tom Gilovich.
0:19:55 I’m a social psychologist at Cornell University.
0:19:58 – I’m Barbara Millers from the University of Pennsylvania.
0:20:02 And I was fortunate to be the arbiter
0:20:07 in two adversarial collaborations with Danny, 20 years apart.
0:20:07 – I’m Matt Killingsworth.
0:20:10 I’m also at the University of Pennsylvania.
0:20:12 And I study human happiness and all being,
0:20:13 like what makes life worth living
0:20:15 and how do we collect data to try to understand
0:20:17 what makes life better?
0:20:18 – My name is Shane Frederick.
0:20:21 I am a professor at the Yale School of Management.
0:20:23 (audience applauding)
0:20:25 – Tom Gilovich, let’s start with you.
0:20:26 Let’s hear basically the story
0:20:29 of the adversarial collaboration with Danny.
0:20:31 It began quite early in your career.
0:20:33 Give us the whole story.
0:20:36 – I had published with Vicky Medvick a chapter
0:20:38 in a book on counterfactual thinking.
0:20:42 When I wrote the chapter, I don’t know if I was aware
0:20:45 that Danny was going to write the wrap up chapter
0:20:47 where he comments on all of the other ones.
0:20:50 So I don’t know if his voice was in my head
0:20:52 as we’re writing the chapter.
0:20:53 – What’s the chapter about?
0:20:54 What’s it cover?
0:20:56 – A lot of it covers the subject of regret,
0:20:58 what we regret most in life.
0:21:01 Amos and Danny had published a study showing
0:21:04 you regret things of action more than inaction.
0:21:07 The example they used is imagine you own stock
0:21:10 in one company, you think, okay, that’s run the course.
0:21:12 You switch to another and it tanks,
0:21:14 you lose a certain amount of money.
0:21:17 Versus you’re thinking about buying this stock,
0:21:20 you decide not to, it takes off
0:21:21 and you lose the same amount of money.
0:21:23 Which would you regret more?
0:21:25 And almost everyone anticipates
0:21:28 that you’d regret the action the most.
0:21:30 Nonetheless, if you ask people,
0:21:32 what are the biggest regrets in your life?
0:21:35 What dominates are things they didn’t do?
0:21:37 And so a lot of the chapter was
0:21:40 how do you reconcile those two things?
0:21:43 – In approaching this riddle or puzzle in your mind,
0:21:47 how did you measure what was your actual research?
0:21:50 Our research was what’s responsible for that difference?
0:21:52 How do things change over time?
0:21:57 Precisely because your regrets of action really hurt.
0:21:58 You do things about them.
0:22:00 Sometimes you change your life accordingly
0:22:04 or you certainly engage in lots of psychological work
0:22:07 to achieve some level of peace.
0:22:11 Because the inaction regrets gnaw at you less powerfully,
0:22:13 you sort of leave them alone.
0:22:16 The fun of that research was tracking down
0:22:19 all of the different psychological processes
0:22:24 that make those regrets of inaction hang around
0:22:27 or in some cases even get more intense.
0:22:31 Whereas the regrets of action dissipate.
0:22:32 – How do you do that though?
0:22:33 Is it empirical?
0:22:34 Is it theoretical?
0:22:35 – It’s empirical.
0:22:37 For example, one of the mechanisms
0:22:39 that we thought was especially interesting
0:22:42 is when you don’t do something
0:22:44 that you later on end up regretting.
0:22:46 There are reasons for that.
0:22:48 I just don’t have the bandwidth right now
0:22:50 to take this project on, et cetera.
0:22:53 Those are compelling reasons in the moment
0:22:55 for why you’re not doing this thing.
0:22:58 You move along in time, you look back and you think,
0:23:00 wait a minute, I could have done that.
0:23:02 So we would do studies where we’d ask
0:23:05 Cornell current students, recent alums,
0:23:07 or much older alums.
0:23:10 Imagine there’s a class that you always wanted to take
0:23:12 but maybe you were a little afraid to.
0:23:14 Suppose we added that to your curriculum
0:23:15 this semester, the students are like,
0:23:18 oh, I can barely hang on right now.
0:23:20 You add that, it’s a catastrophe.
0:23:23 You ask the people who graduated many years ago,
0:23:25 they think, oh, that’s a piece of cake.
0:23:27 I wouldn’t have interfered with my GPA, my social life,
0:23:29 the amount of sleep I get, et cetera.
0:23:31 – Okay, so you write up the findings.
0:23:35 At what point then and in what way do you hear from Danny?
0:23:37 – It’s sort of a highlight of my career.
0:23:41 I was on sabbatical, check in your voice messages
0:23:44 back at Cornell and I get this message.
0:23:47 Tom, this is Danny Kahneman.
0:23:50 I’ve just read your paper with Medvic, it’s brilliant.
0:23:54 It made my day just calling to say thank you.
0:23:56 What a piece of work.
0:24:01 And then I stupidly press star delete
0:24:03 instead of saving that message forever
0:24:04 to play to my grandkids.
0:24:08 So I’m just on a high for a while.
0:24:11 Two weeks later, there’s another phone call
0:24:13 from Danny Kahneman, this time live.
0:24:17 Oh, great, my new best friend is gonna tell me about…
0:24:20 Tom, I’ve been thinking more about your chapter.
0:24:24 It’s all wrong.
0:24:28 I’ve heard you’re planning to publish this.
0:24:30 You can’t do that.
0:24:33 If you do, people will go after you.
0:24:35 And if no one does, I’m gonna go after you.
0:24:39 What happened to Brilliant?
0:24:42 Did he, in that conversation, give you his argument
0:24:44 for why you thought now you were wrong?
0:24:47 Yes, he said that there really isn’t a change
0:24:51 in the intensity of either action or inaction regrets.
0:24:54 There’s just a substitution of the kinds of things
0:24:55 that you’re regretting.
0:24:59 And the regrets of inaction really aren’t serious regrets.
0:25:01 You might not even call them regrets.
0:25:02 They’re more wistful.
0:25:06 Like, oh, I wish I had learned to speak Esperanto.
0:25:07 Said no one ever.
0:25:09 Yeah. (laughing)
0:25:11 All right, so what happens next?
0:25:13 Well, lots of panic for a few days.
0:25:16 Vicki Medvick and I, what do we do?
0:25:18 It can’t be worse than this.
0:25:22 And I think, I forget what there had been an example
0:25:24 of an adversarial collaboration.
0:25:25 I think it was Vicki’s idea.
0:25:27 Hey, we should negotiate with him.
0:25:30 Let’s see if we can do an adversarial collaboration.
0:25:33 So we pitched it and he was very receptive.
0:25:35 This was in what year, Tom?
0:25:38 Chapter came out in ’95, so it was a little before that.
0:25:42 So this was pre-zoom, certainly, early internet.
0:25:43 How are you communicating?
0:25:45 And how did you negotiate the negotiation?
0:25:47 Then how did the actual negotiation happen?
0:25:49 I’m glad you asked that question
0:25:51 ’cause it allows me to bring up a point about Danny
0:25:55 that there’s been all this talk about his many qualities
0:25:57 and they’ve all been well said.
0:25:59 One thing kind of gets left out.
0:26:02 I mean, to be at that level of success,
0:26:05 you have to be brilliant and creative
0:26:07 and many people have spoken about that.
0:26:10 You just have to be a phenomenally hard worker.
0:26:13 And man, he was on task all the time.
0:26:15 It was amazingly easy to get that guy on the phone.
0:26:17 I could get him on the phone more readily
0:26:19 than I could get my co-author, Vicki, on the phone.
0:26:21 And something seemed wrong about that.
0:26:23 He was always ready to engage
0:26:26 and so a lot of it happened over the phone.
0:26:29 And the idea was, no, these regrets of inaction
0:26:32 that you say are important in people’s later lives.
0:26:34 They’re just sort of wistful regrets
0:26:38 and the intense hot emotions that come from action.
0:26:39 Those are different kinds of things.
0:26:42 So we designed some studies where we asked people
0:26:45 to think about the biggest regret of action or inaction.
0:26:47 You designed the studies together.
0:26:48 Then we ran them together, yes.
0:26:52 So think of your biggest regrets of action or inaction
0:26:56 from the recent past or the distant past.
0:26:58 How many of these emotions do they lead you to feel?
0:27:02 We had a set of five hot emotions,
0:27:05 a set of five wistful emotions.
0:27:08 We thought that those long-term regrets of inaction,
0:27:09 some of them are wistful.
0:27:12 I wish I had learned to play the piano,
0:27:14 but some of them are really intense.
0:27:17 You met the right person at the wrong time
0:27:20 and so you let it pass and now you’re looking back like,
0:27:22 oh, my life would have been much better
0:27:25 if I had acted on that.
0:27:30 We also asked people to rate whether a set of five,
0:27:32 I think we called them emotions of despair.
0:27:35 I’m depressed when I think about this.
0:27:37 I feel empty when I think about this.
0:27:40 And lo and behold, Danny was right.
0:27:43 A lot of long-term regrets of inaction
0:27:46 are kind of wistful, but we were right too.
0:27:50 They also produced these big, powerful feelings of,
0:27:53 oh, my life is not what it could have been.
0:27:56 I feel empty and depressed thinking about these things
0:28:00 that I didn’t do that now seem so easy to have done.
0:28:02 – Did Danny, as a result,
0:28:04 acknowledge that you were partially right
0:28:06 as much as he acknowledged that he was partially right?
0:28:08 – Yeah, yeah, and that’s in the paper.
0:28:09 – So truly a happy ending.
0:28:10 – I believe so, yeah.
0:28:13 Certainly more happy than that second phone call
0:28:14 that I got to.
0:28:15 (audience laughing)
0:28:17 – You then went on to collaborate with Danny
0:28:18 several times, correct?
0:28:19 – Yes.
0:28:22 – I’m just curious now that Danny is gone
0:28:24 and since your initial contact
0:28:27 was this adversarial collaboration about regret,
0:28:30 is there anything that you now regret
0:28:33 not having worked on with Danny?
0:28:36 – I wouldn’t say that there’s a particular topic in mind,
0:28:40 but to have someone in your life like that
0:28:42 and not take full advantage of it,
0:28:44 I wish I had just reached out to him more
0:28:47 to have more contact with him.
0:28:50 One of the nice things about this event here
0:28:54 is this family feeling of all the people
0:28:56 he reached, some of whom I know well,
0:28:59 some of whom I didn’t really know at all until now.
0:29:01 They feel like family.
0:29:03 So, you know, often with families,
0:29:06 you wish you had spent more time with them.
0:29:08 – Not my family, but, you know.
0:29:13 (audience laughing and applauding)
0:29:16 – The adversarial collaboration with Tom Gilovich
0:29:19 was one of the earliest ones that Danny Kahneman undertook.
0:29:22 Let’s take a look now at one of the most recent
0:29:25 with Matt Killingsworth from the University of Pennsylvania.
0:29:28 He studied engineering as an undergraduate
0:29:32 and got his PhD in psychology in 2012.
0:29:37 Danny Kahneman got his psychology PhD in 1961.
0:29:39 So, how did Matt Killingsworth end up
0:29:41 in an adversarial collaboration
0:29:43 with this giant in the field?
0:29:46 – Danny and Angus Deaton, both Nobel Prize winners,
0:29:49 had this conclusion that there was sort of this plateau
0:29:52 in people’s happiness once they reached $75,000 in income
0:29:54 and then I’d published a paper
0:29:56 that basically showed something completely different
0:29:59 and we tried to figure out who’s right and what’s the truth.
0:30:01 – You say it with this sort of sang froide
0:30:04 that, you know, we published something completely different,
0:30:06 but as you noted, you were publishing a paper
0:30:08 in opposition to the findings of not one,
0:30:10 but two Nobel laureates.
0:30:12 Were you as calm and cool about it in the moment
0:30:15 when you decided to take up this route
0:30:16 as you seem to be now?
0:30:18 – I mean, partly Danny is to blame,
0:30:22 although he didn’t learn that before he passed away.
0:30:24 I had written this as like a sub point to a sub point
0:30:26 and another paper and actually reading
0:30:29 Michael Lewis’s book about Danny and Angus,
0:30:32 they were talking about how they took on existing ideas
0:30:35 and that was an important part of their intellectual journey.
0:30:37 And so I sort of rolled the dice and thought,
0:30:38 “Well, I’ll try it.”
0:30:40 But yeah, there was definitely some part
0:30:42 in the back of my mind of like, how is this gonna go?
0:30:45 I do have to ask at this particular conference,
0:30:47 which is held in celebration
0:30:49 of Danny Kahneman’s work in life,
0:30:51 do you feel a little bit like, you know,
0:30:52 the wolf in the hen house
0:30:55 or that someone’s gonna come up and shiv you in the back
0:30:56 at the buffet?
0:30:57 – I hope not.
0:31:00 No, I mean, we really had a wonderful collaboration
0:31:03 and I think part of what characterized that is,
0:31:05 we both just wanted to figure out the truth.
0:31:07 I don’t think anyone was attached
0:31:10 to any particular version of reality.
0:31:13 My sense from him is that he was a little bit irritated,
0:31:14 not a lot.
0:31:15 – A lot is a lot.
0:31:17 I mean, he wanted to have a good conversation,
0:31:18 but he wasn’t–
0:31:22 – For the record, that was Shane Frederick saying a lot.
0:31:24 (laughing)
0:31:26 Barb Millers, let’s bring you in here.
0:31:30 How did you get attached to this adversarial collaboration?
0:31:32 – Well, I was talking to Danny on the phone,
0:31:35 not too long after Matt’s paper came out
0:31:37 and he said, “Oh, by the way,
0:31:40 “do you happen to know a guy by the name
0:31:42 “of Matt Killingsworth?”
0:31:45 And I said, “Well, as a matter of fact,
0:31:47 “I do, and if you’d like me to be the arbiter,
0:31:49 “I’d be happy to do so.”
0:31:51 (laughing)
0:31:54 In adversarial collaborations,
0:31:57 the arbiter is the research assistant,
0:32:03 the tiebreaker, and occasionally the therapist.
0:32:07 – So Matt, how would you characterize the response
0:32:10 to that original Kahneman-Deaton finding
0:32:13 about the $75,000 happiness cutoff,
0:32:16 whether inside academia or beyond?
0:32:17 – I mean, it was very influential,
0:32:20 and I mean, it’s probably one of the most visible,
0:32:22 kind of an exciting findings.
0:32:26 I think a lot of us feel like money is sort of overemphasized
0:32:29 in our daily lives, and it kind of gives a justification
0:32:31 for maybe carrying a little bit less about it
0:32:33 and focusing more on all the other stuff
0:32:35 that’s also really important for happiness,
0:32:37 particularly because money is quantitative,
0:32:39 you can kind of think of them as points.
0:32:40 I’m like, “Well, I wanna get more,”
0:32:42 but you can easily imagine a sort of trap
0:32:44 where you’re just continuously trying to get
0:32:46 more and more and more of those points,
0:32:47 ignoring all of this other stuff,
0:32:48 and wouldn’t it be nice
0:32:50 if we could kind of step out of that cycle?
0:32:52 At least that’s part of the reason
0:32:54 that I think that that was so attractive.
0:32:56 – So Matt, you’ve rehearsed very nicely
0:32:59 the original finding, now bring you into the story here.
0:33:00 What are you doing at the moment?
0:33:03 – Sure, so partly also due to Danny,
0:33:05 my research program is really centered
0:33:08 on large-scale experience sampling,
0:33:10 and to get at what that matters in the original study,
0:33:12 those data were collected, how?
0:33:14 In the original study, they basically asked,
0:33:16 “Did you experience a lot of the following emotion
0:33:18 “yesterday, yes or no?”
0:33:20 And then it had a series of emotions
0:33:24 like sadness, happiness, stress, et cetera.
0:33:26 People would either say, “I did or I didn’t.”
0:33:27 – And the technology or mechanism
0:33:29 of harvesting those data was what?
0:33:31 – I wasn’t involved in that data collection.
0:33:34 I believe those were verbal phone calls,
0:33:36 and then sort of interviewing people over the phone.
0:33:38 – Does anyone here know anything different,
0:33:40 or does that sound right, as far as we know?
0:33:41 – That’s right.
0:33:44 That was said with the voice of a referee, I have to say.
0:33:45 (audience laughs)
0:33:47 Okay, Matt, continue, please.
0:33:50 So in my study, I’m essentially measuring
0:33:52 people’s experience right in the moment.
0:33:55 They’re carrying around on their phone an app,
0:33:57 and I’m beeping them at random times.
0:33:59 I’m asking them, “How do you feel right now?”
0:34:00 And they’re responding on a scale
0:34:03 that ranges from very bad to very good,
0:34:04 and the scale is continuous.
0:34:06 So there are a couple of things
0:34:09 that distinguish that from the earlier paper.
0:34:10 One is that it’s right at the time
0:34:11 that people are feeling it,
0:34:14 as opposed to the day as a whole, retrospectively.
0:34:16 And the other is that gradations of feelings
0:34:17 are on a continuous scale,
0:34:20 as opposed to something that’s binary or dichotomous.
0:34:22 – And the scale is what to what?
0:34:24 It’s zero to 10, zero to 100.
0:34:26 – Very bad to very good, and it’s just continuous.
0:34:27 – So you can rake in anywhere you want.
0:34:29 – There are hundreds of unique values.
0:34:30 – Got it.
0:34:31 – And describe the differences
0:34:34 between the pools of research subjects
0:34:35 in the original case and in your case?
0:34:38 – In the original case, it was a survey by Gallup.
0:34:40 It was either representative or at least an attempt
0:34:43 to be representative of whatever random digit dialing was.
0:34:46 That’s a relative strength of their paper
0:34:47 and of that study in general.
0:34:50 My sample, in contrast, was really a convenient sample.
0:34:52 It’s turned out to be an amazing sample
0:34:53 of people that have really beautiful results,
0:34:56 but it was essentially whoever was willing to sign up
0:34:58 to try to understand their own happiness.
0:35:01 – What was the recruiting mechanism?
0:35:03 – Largely thanks to folks like you,
0:35:06 people in a hearing about it in the media,
0:35:07 on the radio, reading about it.
0:35:09 I think that’s interesting.
0:35:10 I’d like to learn about that for myself.
0:35:12 I’m willing to contribute to science.
0:35:14 And it turns out that when I look at,
0:35:15 what’s the distribution of incomes?
0:35:17 For example, in my data,
0:35:19 it’s almost a perfect match for the US census.
0:35:21 I can replicate all kinds of things
0:35:23 that we’ve seen in the literature for decades.
0:35:25 So it turns out to be really good,
0:35:27 but that’s kind of a lucky coincidence.
0:35:32 – Okay, so you gather, assemble, analyze your data,
0:35:34 talk about from the analysis point
0:35:36 to writing up the findings.
0:35:38 What I find when I look at this relationship,
0:35:40 plotting people’s happiness in the moment
0:35:43 versus their income, it just keeps going up.
0:35:45 This sort of critical point
0:35:47 where they had found in the earlier paper,
0:35:50 this flatlining, I really see no difference at all.
0:35:52 And when I write it up in the ultimate paper,
0:35:55 I compare, well, what’s the slope below the point
0:35:56 where they said it stops increasing?
0:35:57 And the point above that,
0:35:59 the slopes differ by less than 1%.
0:36:02 I really see no evidence for a difference at all.
0:36:05 Were you initially looking for that plateau in the data?
0:36:06 – I was really just trying to understand
0:36:08 what’s the relationship between these things
0:36:10 that are obviously important.
0:36:13 We’ve never had, in the moment, continuous data.
0:36:15 And so I want to see, what does this look like?
0:36:17 And it turns out, well, it doesn’t look like
0:36:18 what we thought it looked like.
0:36:23 – When you strike at a king, you must kill him.
0:36:26 That’s from Ralph Waldo Emerson.
0:36:28 Matt Killingsworth did strike
0:36:31 at the king of his realm, Danny Kahneman.
0:36:33 But what happened next?
0:36:35 It’s coming up after the break.
0:36:38 I’m Stephen Dovner, and this is Freakonomics Rating.
0:36:48 (upbeat music)
0:36:52 We’ve been hearing from Matt Killingsworth
0:36:55 about his research on the relationship
0:36:57 between income and happiness.
0:37:00 Research that disputed elements of earlier research
0:37:03 on the same topic by Daniel Kahneman.
0:37:05 – So you write up the paper,
0:37:09 and the paper, not quite directly,
0:37:11 but quasi-directly, addresses the fact
0:37:14 that your finding is contra
0:37:18 to a significant earlier finding by significant scholars.
0:37:20 What happens next?
0:37:22 – There’s a fair amount of attention about it and so forth.
0:37:24 Maybe a month or two afterwards,
0:37:29 I get a note from Barb that she’s been chatting with Danny.
0:37:31 – He’s talking about Barb Mellors,
0:37:33 a longtime friend of Danny Kahneman,
0:37:36 as well as a University of Pennsylvania colleague
0:37:37 of Matt Killingsworth.
0:37:39 – I think both of our attitudes
0:37:41 was we just wanna find out the truth.
0:37:43 I have no personal attachment
0:37:45 to what I found particularly,
0:37:48 and I don’t, other than his initial perhaps irritation,
0:37:50 I don’t really think Danny did.
0:37:52 It was really just what do we think is going on.
0:37:55 – Did you envision that perhaps you would then collaborate
0:37:56 with Danny on a joint study?
0:37:58 – That certainly seemed to be the case
0:37:59 once we got into it,
0:38:02 but these kinds of data took me many years
0:38:04 to collect that data for them.
0:38:06 There were a client on an external organization
0:38:07 with a lot of resources,
0:38:10 and so we ended up doing it by going back
0:38:11 and looking at my data, which existed,
0:38:13 which made it tractable.
0:38:15 I can sort of cut to the chase a little bit
0:38:16 of how we did that if you want.
0:38:18 – I wanna go back to Barb for just a second here.
0:38:21 Barb, if you could give Danny’s perspective
0:38:23 or participation up to the point here
0:38:25 where we’re about to cut to the data.
0:38:29 – The starting point for Danny was the assumption
0:38:32 that both data sets were valid.
0:38:35 So how could they be so different?
0:38:39 There must be a resolution in the data somewhere,
0:38:42 and so the task was going to be,
0:38:44 Matt became the research assistant
0:38:48 and started doing all the re-analyses on his data.
0:38:51 – How would you characterize the tenor
0:38:54 of this adversarial collaboration?
0:38:57 – Really civil and nice,
0:38:59 and just the way it should go.
0:39:01 – So Matt, you said you could cut to the chase.
0:39:03 We’re at the chase now, cut to it.
0:39:05 – Essentially, the resolution was
0:39:07 if we look at the range
0:39:10 within my continuous happiness data,
0:39:11 what if we look at the low end,
0:39:13 which is the part where their measure
0:39:14 would have been sensitive,
0:39:16 can we find a similar pattern?
0:39:18 And lo and behold, when you zero in on that,
0:39:20 rather than looking at the average trend,
0:39:21 which keeps going up,
0:39:24 you get not exactly, but very, very close
0:39:25 to what they found,
0:39:27 certainly much closer to what they found
0:39:29 than to what the average trend looks like.
0:39:31 We both found that pretty convincing evidence
0:39:34 that what they found is true,
0:39:36 there’s nothing wrong with the analysis at all,
0:39:38 but it was really a question of,
0:39:40 how generally applicable is that?
0:39:42 So from low to medium incomes,
0:39:46 the unhappiest part of the distribution rises a lot,
0:39:47 and then from medium to high incomes,
0:39:51 the unhappiest part of the distribution barely changes.
0:39:55 But the rest of the distribution is rising steadily,
0:39:58 and in fact, at the high end of the happiness distribution,
0:39:59 you get an inversion of that.
0:40:01 So rather than a rise and then a plateau,
0:40:05 you have sort of a shallow slope and then an acceleration.
0:40:07 – So in my lay mind, as I’m trying to process all this,
0:40:12 what I’m envisioning is that there is a cohort of people
0:40:15 who are kind of cranky,
0:40:19 and they may experience a little bit less crankiness
0:40:20 as income rises,
0:40:24 but there is a range in which their temperament
0:40:26 may be overwhelms their income.
0:40:28 – If anything, there’s one of the steepest slopes
0:40:31 for the unhappiest people in the range of low income.
0:40:33 So getting out of poverty, if you’re really miserable,
0:40:35 is at least correlationally.
0:40:36 – But that’s almost a different story,
0:40:39 ’cause escaping poverty versus going from middle to upper.
0:40:43 – If you aren’t poor and you’re really miserable,
0:40:45 at least if you sort of extrapolate from there,
0:40:46 it seems to not matter past that.
0:40:49 And probably as we speculate in the paper,
0:40:51 at that point, if you have a decent amount of money
0:40:52 and you’re really unhappy,
0:40:54 whatever’s making you unhappy probably isn’t due
0:40:55 to the lack of resources.
0:40:58 It’s something else going on in your life.
0:41:00 Maybe it’s challenges in a family relationship,
0:41:02 or maybe you’re depressed, or whatever it is,
0:41:05 it’s something going on that perhaps money
0:41:07 isn’t going to make a difference.
0:41:09 – Richard, Taylor, let me ask you a question.
0:41:12 You’ve just heard Matt’s presentation of his side
0:41:13 of the original research,
0:41:15 and then the adversarial collaboration,
0:41:16 and then the conclusion,
0:41:19 all of which I found honestly to be really fascinating.
0:41:21 You know the literature well.
0:41:24 You too have a Nobel Prize,
0:41:26 although I have heard that was a clerical error.
0:41:27 (audience laughing)
0:41:30 How would you, looking at this from above,
0:41:32 Barbara was there as an arbiter,
0:41:34 Matt was one side,
0:41:37 Danny was another side, but is not present,
0:41:39 if you could give an ultimate proclamation on,
0:41:41 not just where this finding arrived,
0:41:43 but what the adversarial collaboration produced here,
0:41:45 what would you say?
0:41:47 – Look, I think this is the way it should work.
0:41:49 This is a good story,
0:41:52 and the world would be a better place,
0:41:54 academia would be a much better place
0:41:58 if there were more of these kinds of collaborations.
0:42:01 – Would it affect what we sometimes
0:42:04 call the replication crisis?
0:42:06 – Uh, oof.
0:42:08 – Barbara says, Barbara’s nodding her head, no,
0:42:10 you say oof.
0:42:12 – Yeah, let’s move on to Shane.
0:42:15 (audience laughing)
0:42:19 – The discomfort you were hearing there
0:42:21 about the replication crisis,
0:42:24 that discomfort is related to a two-part series
0:42:27 we recently produced on academic fraud,
0:42:31 episodes 572 and 573.
0:42:33 And while we’re on the topic of discomfort,
0:42:35 let me add one thing I’m thinking about
0:42:36 as we’re in the middle of this conversation
0:42:39 about adversarial collaborations.
0:42:42 Danny Kahneman is said to have introduced this concept
0:42:45 into the realm of behavioral research,
0:42:47 but if you’re not a behavioralist,
0:42:49 you could look at this as yet another example
0:42:53 of how academic researchers discover something
0:42:56 that people in the real world have been doing forever.
0:43:00 There are all sorts of arbiters and referees out there,
0:43:05 all sorts of processes for compromise and negotiation,
0:43:08 even war games to test one plan against another
0:43:10 and come up with the best solution.
0:43:15 In the same vein, some people look at the big insights
0:43:17 that have come out of this behavioralist research
0:43:22 as essentially ESOP’s fables with a little bit of math.
0:43:24 Earlier this year,
0:43:27 we made a series about the late physicist Richard Feynman.
0:43:30 He didn’t think the social sciences like psychology
0:43:34 and even economics should be called science at all.
0:43:38 He thought they were just too squishy to deserve that name.
0:43:41 And as for psychologists discovering the idea
0:43:43 of adversarial collaboration,
0:43:47 well, Feynman was a junior member of the Manhattan Project,
0:43:50 which in its quest to build an atomic bomb
0:43:53 brought together the most brilliant
0:43:57 and argumentative group of scientists ever assembled.
0:43:59 That was adversarial, and yes,
0:44:03 it was in the end a successful collaboration.
0:44:06 Anyway, back to Chicago.
0:44:09 As Richard Thaler suggests, let’s move on.
0:44:12 Shane Frederick, he is a behavioral scientist at Yale
0:44:16 who collaborated with Danny Kahneman on several projects
0:44:18 and is cited throughout Kahneman’s book,
0:44:20 Thinking Fast and Slow.
0:44:22 Kahneman saw Frederick as a protege
0:44:24 and they had a close relationship,
0:44:27 sometimes a bit too close.
0:44:29 – So I’m at 10 o’clock on a Monday,
0:44:31 maybe like third quarter of Monday football.
0:44:32 I get a call.
0:44:35 – Shane, you said, are you watching the game?
0:44:36 – It was one of the great times I wasn’t.
0:44:38 I said, no, fantastic.
0:44:39 I want to talk about heuristics.
0:44:42 (audience laughing)
0:44:45 Another time, it was like two o’clock in the morning,
0:44:47 going back and forth, back and forth, back and forth.
0:44:49 And thanks, I’m getting sleepy.
0:44:52 And so, go to bed and I wake up in the morning
0:44:54 in the very next email from Danny,
0:44:56 “Shane, don’t play dead on me.”
0:44:59 (audience laughing)
0:45:01 – One idea they worked on together
0:45:04 was the cognitive reflection test, or CRT.
0:45:06 It is meant to measure a person’s ability
0:45:09 to override their gut instinct
0:45:11 and think more carefully about a problem.
0:45:13 Here’s a famous example.
0:45:16 A bat and a ball cost $1.10 in total.
0:45:19 The bat costs $1 more than the ball.
0:45:22 How much does the ball cost?
0:45:26 If your first inclination is to say, 10 cents,
0:45:30 congratulations, you were like just about everyone else.
0:45:32 What the CRT measures is,
0:45:35 can you slow down and actually calculate the answer
0:45:37 rather than just go with your gut?
0:45:42 If you can, you will find that the answer is five cents.
0:45:47 Remember, the bat costs $1 more than the ball.
0:45:49 Shane Frederick has spent much of his career
0:45:52 designing such tests.
0:45:53 – Some of the items are novel, some are invented,
0:45:56 many of them I didn’t, I collect, some I’m revised.
0:45:59 One goes back to at least 1919, their variance of it.
0:46:00 – They were used then in an academic setting,
0:46:01 or like an employment?
0:46:03 – No, just like in books and riddles,
0:46:04 stuff that Maya would read.
0:46:05 (audience laughing)
0:46:06 – But they weren’t stumpers.
0:46:08 – No, they weren’t stumpers.
0:46:13 – I think I must tell the audience what a stumper is, okay?
0:46:16 – That again is Maya Bar-Huelal.
0:46:19 – A one-way ride costs $10,
0:46:22 a round trip costs $20,
0:46:26 a passenger hands the cashier $20,
0:46:28 saying absolutely nothing.
0:46:30 The cashier knows right away
0:46:32 that the passenger wants a round trip
0:46:35 rather than a one-way and change.
0:46:39 And the question is, how did the cashier know this?
0:46:42 – Does anyone in the audience who does not know the answer
0:46:46 to this stumper previously wanna take a guess to the answer?
0:46:50 – If you’re stumped good,
0:46:52 because that’s why they’re called stumpers.
0:46:57 – A stumper is unlike one of Shane Frederick’s CRT questions
0:46:59 in that your intuition doesn’t produce
0:47:01 an obvious but wrong answer.
0:47:04 – It only has to do with my love of riddles,
0:47:07 which I share with my partner in this research,
0:47:08 Shane Frederick.
0:47:12 And we really came to this out of love of riddles,
0:47:15 but we’re both professional psychologists.
0:47:19 So we felt like we had to approach it
0:47:21 from the point of view of psychology.
0:47:22 Now, Danny.
0:47:23 – Wait, wait.
0:47:24 – Oh.
0:47:25 – Answer please.
0:47:27 – Oh, no way.
0:47:29 No, I’m not gonna answer.
0:47:32 That’s not what I was paid for.
0:47:34 (audience laughs)
0:47:36 – All right, I will answer.
0:47:39 He handed the ticket agent two 10s.
0:47:41 – Ah, very nice.
0:47:43 – Did they pay you for that?
0:47:45 (audience laughs)
0:47:47 – I’m getting paid the same as you, Maya.
0:47:50 (audience laughs)
0:47:51 – But there was something more at stake here
0:47:53 than answering a riddle.
0:47:55 Barr Hillel, along with a co-author,
0:47:58 wrote a paper arguing that cognitive reflection tests
0:48:02 are not a good measurement of anything beyond math skills.
0:48:07 And this argument led to an adversarial collaboration.
0:48:09 – When I heard that Shane and Maya and Danny
0:48:12 were having an adversarial collaboration,
0:48:14 I didn’t know who was on which team.
0:48:18 Maya and Danny go back 60 years.
0:48:21 Shane is like his son for Danny.
0:48:22 – The results of this collaboration
0:48:24 have not yet been published,
0:48:26 but Shane Frederick says that he and Kahneman
0:48:30 essentially proved that the CRT is a good measure
0:48:34 of cognitive abilities beyond just math skills.
0:48:35 – It seemed to do quite well,
0:48:37 and it’s doing well consistently.
0:48:38 Time preferences, risk preferences,
0:48:39 other sorts of things.
0:48:41 – And how would you characterize the nature
0:48:42 of the collaboration?
0:48:44 How adversarial was it?
0:48:48 – It wasn’t so adversarial between the opposing groups.
0:48:50 It’s just like everybody’s fighting with everybody else,
0:48:51 but everything.
0:48:54 (audience laughs)
0:48:56 – More like a family gathering, right?
0:48:57 (audience laughs)
0:49:00 – Does this suggest that an adversarial collaboration
0:49:02 is not as useful as one might hope
0:49:04 if the actual adversaries are not often
0:49:06 in the ring with the collaborators?
0:49:09 – I mean, I don’t think it will work.
0:49:10 Do you, Barb?
0:49:13 – Do these things work?
0:49:15 Well, what does work mean?
0:49:16 It depends on your definition.
0:49:20 If you think people change their minds all the way,
0:49:21 no, it doesn’t work.
0:49:26 If they change their minds a bit, that’s good.
0:49:29 And it speeds up science too.
0:49:31 Somebody’s looking over your shoulder
0:49:34 and making sure you’re not making mistakes,
0:49:36 you’re defining variables more precisely,
0:49:38 you’re designing an experiment
0:49:41 that gets right at the core issue.
0:49:42 It’s the way to go.
0:49:45 (piano music)
0:49:47 – Well, I could listen to the five of you talk
0:49:49 for five hours.
0:49:50 There are sessions that need to happen.
0:49:52 This room is turning over.
0:49:53 I thank you so much.
0:49:54 This was a great conversation.
0:49:58 (audience applauds)
0:50:00 – Sorry, there is a clause in the contract
0:50:02 that Thaler must always have every last word, so.
0:50:05 – I was just gonna say something nice about you,
0:50:06 but if you insist, I’ll skip it.
0:50:09 (audience laughs)
0:50:13 – And that was, again, Richard Thaler.
0:50:16 I would like to thank him for putting this event together
0:50:18 and inviting us to join.
0:50:21 Thanks also to the other participants.
0:50:23 It was a pretty wonderful event
0:50:26 and I’m glad we were able to share it with you.
0:50:29 And if you need more Richard Thaler in your life,
0:50:31 and who doesn’t need more Thaler in their lives,
0:50:34 we are going to publish a bonus episode very soon,
0:50:36 an update of a great conversation
0:50:38 I had with Thaler a few years ago.
0:50:42 It’s called People Aren’t Dumb, The World is Hard.
0:50:44 So keep your ears out for that.
0:50:47 Meanwhile, on the next regularly scheduled episode
0:50:51 of Freakonomics Radio, a close-up look at an industry
0:50:54 that’s all about close-up looking.
0:50:58 I like for my glasses to have a bit of pizzazz,
0:51:00 especially if you’re wearing them every day.
0:51:03 – We’re gonna see half of the global population
0:51:05 being myopic by 2015.
0:51:07 – When you are a vertically integrated player,
0:51:10 you master every step within the value chain.
0:51:14 – The margins, even by luxury good standards, are obscene.
0:51:17 – Eyewear is a $150 billion industry
0:51:21 and what you see is not quite what you get.
0:51:23 That’s next time on the show.
0:51:25 Until then, take care of yourself,
0:51:28 and if you can, someone else too.
0:51:33 Freakonomics Radio is produced by Stitcher and Renbud Radio.
0:51:36 You can find our entire archive on any podcast app
0:51:38 also at Freakonomics.com,
0:51:41 where we publish transcripts and show notes.
0:51:43 This episode was produced by Zach Lipinski,
0:51:46 with live recording by Greg Rippen.
0:51:49 Special thanks to conference organizers Amy Boonstra,
0:51:51 Mark Tomelko, and Chris Partridge,
0:51:54 as well as the Black Oak AV team.
0:51:57 Our staff also includes Alina Cullman, Augusta Chapman,
0:51:59 Dalvin Abouaji, Eleanor Osborn,
0:52:02 Elsa Hernandez, Gabriel Roth, Jasmine Klinger,
0:52:04 Jeremy Johnston, Julie Canfer,
0:52:06 Lyric Bowditch, Morgan Levy, Neil Coruth,
0:52:10 Rebecca Lee Douglas, Sarah Lilly, and Teo Jacobs.
0:52:12 Our theme song is “Mr. Fortune”
0:52:13 by the Hitchhikers.
0:52:16 Our composer is Luis Guerra.
0:52:18 As always, thank you for listening.
0:52:20 – I think my mind was wandering.
0:52:23 I have that effect on people, I’ve been told.
0:52:25 (audience laughing)
0:52:28 (upbeat music)
0:52:31 – The Freakonomics Radio Network,
0:52:33 the hidden side of everything.
0:52:36 (upbeat music)
0:52:37 Stitcher.
0:52:40 (gentle music)
Daniel Kahneman left his mark on academia (and the real world) in countless ways. A group of his friends and colleagues recently gathered in Chicago to reflect on this legacy — and we were there, with microphones.
- SOURCES:
- Maya Bar-Hillel, professor emeritus of psychology at the Hebrew University of Jerusalem.
- Shane Frederick, professor of marketing at the Yale School of Management.
- Thomas Gilovich, professor of psychology at Cornell University.
- Matt Killingsworth, senior fellow at the Wharton School of the University of Pennsylvania.
- Barbara Mellers, professor of psychology at the University of Pennsylvania.
- Eldar Shafir, director of the Kahneman-Treisman Center for Behavioral Science & Public Policy at Princeton University.
- Richard Thaler, professor of behavioral science and economics at the University of Chicago.
- RESOURCES:
- “Experienced Well-Being Rises With Income, Even Above $75,000 Per Year,” by Matthew A. Killingsworth (PNAS, 2021).
- “The False Allure of Fast Lures,” by Yigal Attali and Maya Bar-Hillel (Judgment and Decision Making, 2020).
- “Learning Psychology From Riddles: The Case of Stumpers,” by Maya Bar-Hillel and Tom Noah (Judgment and Decision Making, 2018).
- Thinking, Fast and Slow, by Daniel Kahneman (2011).
- “High Income Improves Evaluation of Life but Not Emotional Well-Being,” by Daniel Kahneman and Angus Deaton (PNAS, 2010).
- “Varieties of Regret: A Debate and Partial Resolution,” by Thomas Gilovich, Victoria Husted Medvec, and Daniel Kahneman (Psychological Review, 1998).
- “Some Counterfactual Determinants of Satisfaction and Regret,” by Thomas Gilovich and Victoria Husted Medvec (What Might Have Been: The Social Psychology of Counterfactual Thinking, 1995).
- EXTRAS:
- “Remembering Daniel Kahneman,” by People I (Mostly) Admire (2024).
- “Academic Fraud,” series by Freakonomics Radio (2021).
- “Here’s Why All Your Projects Are Always Late — and What to Do About It,” by Freakonomics Radio (2018).
- “The Men Who Started a Thinking Revolution,” by Freakonomics Radio (2017).