Freakonomics Radio
AI transcript
(dramatic music)
Today on Freakin’ Amic Radio, a very special episode,
a conversation about the late Daniel Kahneman,
whose insights into human behavior
have been threaded through this show for years,
ideas like confirmation bias and loss aversion
and the planning fallacy.
During this conversation,
we also learned about a research paradigm
that Kahneman embraced called adversarial collaboration,
which means working shoulder to shoulder with your rivals.
He felt that this is the right way to do science.
Kahneman was a phenomenally influential psychologist
who won a Nobel Prize in Economics,
wrote the bestselling book, Thinking Fast and Slow,
and left behind an army of collaborators,
mentees, and admirers.
With them, we will take a careful look
at the life and mind of Danny Kahneman starting now.
(upbeat music)
This is Freakin’ Amic Radio,
the podcast that explores the hidden side of everything
with your host, Stephen Dovner.
(upbeat music)
Last month in a sunlit auditorium
overlooking the Chicago River,
there was a gathering of psychologists, economists,
and other social scientists.
This was the Behavioral Decision Research
in Management Conference.
The keynote event was supposed to be
a conversation with Danny Kahneman,
facilitated by Richard Thaler,
his longtime friend and collaborator.
Thaler is the University of Chicago Economist
who helped turn Kahneman’s insights
into the field now known as behavioral economics.
But when Kahneman died in March at age 90,
Thaler came up with a new plan for the conference.
Now it would pay tribute to Danny Kahneman.
Freakin’ Amic Radio was lucky enough to be asked along
to moderate a couple of panel discussions
about his life and work.
The episode you’re about to hear
is a condensed version of those conversations.
This all took place at the downtown outpost
of the University of Chicago’s business school
in front of a couple hundred attendees.
Some of the panelists had known Danny Kahneman
for many decades.
For instance, the psychologist Maya Bar-Hillel,
her father was a philosophy professor
at Hebrew University in Jerusalem,
where Kahneman got his undergraduate degree.
– My father taught Danny and gave him a lot of grief,
but my father apparently gave just about everybody
a lot of grief.
He was a tough-minded philosopher.
– And when Kahneman became a professor,
one of his students was Maya Bar-Hillel,
from generation to generation.
– I met Danny in the first week of my first year
at the Hebrew University.
He gave the introductory statistics course.
He looked at us and he pronounced right away,
you are the creme de la creme.
He said it in French and we were.
(audience laughing)
– Kahneman had grown up in France
during the Nazi occupation.
He survived, barely, and lived for many years in Israel
before coming to the US to get his PhD at UC Berkeley.
His research partner, Amos Tversky,
was another Israeli psychology professor
who moved to the States.
Tversky died young in 1996,
too early to share in what would have surely been
a joint Nobel Prize.
Tversky was regarded as perhaps
even more brilliant than Kahneman.
The two of them published many papers
on judgment and decision-making,
but not just in psychology journals.
– It always struck me just in the story
of how Kahneman and Tversky research
was taken hold of by Thaler and others
and turned into what we now call behavioral economics.
That it was very, very important
that these were two psychologists
who were also very mathematically fluent.
If that hadn’t been the case
and if the publication hadn’t been in,
I guess, econometrica and so on,
was there a pretty good possibility for a counterfactual
where all that research might have stayed
within the realm of psychology and never trickled over
and we might not have what we think
of as behavioral economics?
– It was not an accident.
It’s not how fortunate that they went to econometrica.
They realized that their work
was attended to primarily by psychologists
and in fact, they both considered themselves
all their lives as psychologists,
but they also realized that their research
was perhaps more important outside of psychology.
So the decision to publish their paper in econometrica
was a deliberate move.
It was a strategic move to get the attention.
They believed that that was the ticket
and without the ticket,
they would not have been playing in that field.
– A lot of the early Kahneman-Tversky work
centered around an observation
that may seem obvious in retrospect,
but at the time had not been explored with much rigor.
The idea was that we are all constantly making decisions,
personal, professional, political decisions.
And then later, if we ask ourselves
why we made a given decision,
which by the way, we usually don’t ask,
we might tell ourselves a story about making the decision,
but these stories are often not quite true.
Why?
One reason is that we want to appear to others
and maybe even ourselves as smarter than we are.
Here is Richard Taylor.
– I met Danny and Amos in 1977
and it was a transformative year for me.
I decided to change jobs and took a job at Cornell
and decided to offer a course
in the thing I was now fascinated by
and called it behavioral decision theory,
which is kind of what the name of this field used to be.
I got about eight students.
So I had to think of something to do
to increase the enrollment in the class.
And so I changed the name to managerial decision making.
50 students show up.
I began the class by asking how many of you
signed up for this class because of the name?
No one raised their hand.
I said, well, actually nearly all of you.
That’s one illustration of what Danny’s talking about.
No one thinks they would be stupid enough
to sign up for a class based on the name.
– I believe it was in thinking fast and slow
where Danny wrote, not only are we blind,
but we are blind to our blindness.
– Yeah, and you know, maybe one of the many secrets
of the Conor Monteverski collaboration was
they were not blind to those.
And what they had was a mistake detection facility.
They had some radar where they could anticipate
what the mistake is.
And because they had this mistake detection facility
somewhere, they were able to figure out things.
And then they were pretty good at predicting
what people other than Amos and Danny would do.
To me, the big point, the aha point that I got
from reading their judgment papers
was the idea of predictable bias.
– May I add something to that?
– Yes, please.
– Not only predictable.
I think now I’m gonna say something that is perhaps,
I can’t say that I heard them say it,
but I hope they would both agree,
that the errors are not just predictable.
The errors are smart.
Our stupid mistakes are evidence
of the wonderful human mind.
That’s how normal cognition functions
and normal cognition is amazing.
– I love that.
It does make me wanna ask any, all of you,
a question that’s fairly heretical,
which is, you know, as a layperson,
I read these findings, they’re attractive,
they’re believable, they’re identifiable.
But the thing that I always struggle with
or would like to understand better
is how you all can feel so confident
that they’re generalizable.
There must be people in the world
who are not susceptible to loss aversion
or recency bias or many, many of them.
So really the question is, how much variance is there
and how do you measure the variance?
– You know, Amos used to have a joke
that there were species that didn’t exhibit loss aversion
and they’re now extinct.
(audience laughing)
– Think about what Maya Bar-Huelel was saying there
about the smart errors we all make.
It would be easy to overlook the baseline insight
that Danny Kahneman offered,
that people make thinking mistakes all the time.
Now, most of us, upon hearing this,
might say, no kidding.
Our species is highly fallible, who doesn’t know that?
But it is our fallibility, Kahneman realized,
that makes us interesting and worthy of inspection.
He accepted that humans are capable of wonderful things
as well as terrible ones,
but that we are overconfident in our abilities,
that we often have poor self-control,
and that we employ an arsenal of mental shortcuts
or heuristics to make decisions or reach judgments
that often turn out poorly.
And even when presented with evidence of our mistakes,
we usually fail to change our minds.
Here is Eldar Shafir,
who runs the Kahneman-Triesman Center
for Behavioral Science and Public Policy at Princeton.
He and Kahneman used to co-teach a class.
– One lecture that I love that Danny used to give
in our course, we would talk about people’s
not good sense about conditional probabilities.
And then we had a clip of the O.J. Simpson trial
where Dershowitz explains to the jury
that the probability that a beaten woman
is gonna be killed by her beating partner
is extremely low, which is true.
However, we’re not predicting the chance
that Nicole Simpson will be killed.
She has been killed.
The probability that a beaten woman who has been killed
was killed by her partner is immensely high.
That nuance of not being sensitive to these conditionals
has major implications.
He was so genuinely curious and intellectually alive,
he presented the same way to the Swedish monarchy
and to an undergraduate.
He just wanted to think about it.
He wanted to struggle with the question.
He wanted to listen and think of the best theory.
That was what’s so impressive.
When we talk together, we took turns lecturing.
Here was Danny, I mean, before and after the Nobel,
and every class he would come and sit down
with the students and listen.
He could easily not have shown up.
It’s other people’s lecturing.
He was always there, asking questions,
answering questions, devoted to understanding.
Always had a new thought about something
that we have done on slides for three years in a row.
(gentle music)
And many things were written about Kahneman.
I’m always amused by the fact that they say
he was aware of these quirks and fallacies,
but he was aware that he has them too.
And that sort of implies that since he was aware,
it wasn’t as big a deal.
There’s a couple of recorded interviews with Danny
during COVID, where he predicts
that COVID means life imprisonment
for the more advanced in age.
And he was wrong.
He lived a couple of wonderful years after COVID.
There was a sense in which he really sank deep
into the same error that he was able to predict
and acknowledge.
The other amazing thing about Danny
is when he won the Nobel Prize, you have to give a lecture.
And he gave a lecture and decided he was going
to give a new interpretation of all of his work with Amos.
Using this two system approach.
I kept saying, Danny, you have two months to do this.
Why are you starting over?
They gave you a Nobel Prize for what you did.
You don’t need a new reinterpretation,
but that lecture in Stockholm was the beginning
of what led to thinking fast and slow
because he was reinterpreting all the heuristics
and biases stuff in the lens of system one
and system two, which is your immediate reaction
versus when you sit down and think about it.
– What this story suggests to me,
but tell me if I’m wrong, is that he understood
that having an audience like the Nobel committee audience
and the King or Queen or whatever of Sweden
and that his voice was now amplified
that he wanted to take the work that he’d done
and make it accessible, make it known
to people who have some levers of power.
He wanted to exploit a good opportunity.
– I don’t like the word exploit.
– Exploit for pro-social reasons, a good opportunity.
– Thank you, thank you.
– Yeah, I think he had an audience
and wanted to get it out there,
but it was torture for him, absolute torture.
I mean, during the writing of Thinking Fast and Slow.
– Which was a long gestation.
– Long gestation.
And he decided absolutely positively
that he was quitting two dozen times.
– You know, Danny in some sense really was a bon vivant.
I mean, he enjoyed life.
He loved a good restaurant.
He loved a good vacation.
And I think he loved getting the prize.
– I don’t think he changed.
I mean, yes, the world was paying more attention to him,
but he was always Danny.
– He changed.
– He was happier.
– Okay.
(audience laughing)
– No, I would like to say–
– If Maya says that, I accept.
– I would like to say in what way he relaxed.
– That was relaxed?
(audience laughing)
– Yes, yes.
– What do you mean by that, Maya?
– He had made it.
There was a common misperception
that the Tversky-Kahaneman collaboration
was not symmetrical.
The world seemed to think that Amos was the lead
and Danny was an outside visitor.
It was a symmetrical and equal collaboration,
but the world didn’t know it.
It was very important to Danny
to be able to do good work after Amos dies
so that people won’t continue with this error
that he was less than equal.
So the prize told the world,
this man is noble worthy,
and he did excellent work
because he had an excellent mind,
but also it was important for him
that Kahaneman not die the name together with Tversky,
and it didn’t.
– As I said earlier,
the main ideas of Danny Kahneman
have been threaded through the fabric
of Freakonomics Radio over the years.
Episode 323, for instance,
which was about the planning fallacy.
It’s called Here’s Why Your Projects Are Always Late
and What to Do About It.
Then there’s episode 271,
The Men Who Started a Thinking Revolution.
That was an interview with Michael Lewis
about the book he wrote
on the Kahneman-Tversky partnership.
Lewis’ book is called The Undoing Project,
a friendship that changed our minds.
So yes, you may be familiar
with Kahneman’s greatest hits,
but one of the conversations we had at this conference
went into an area I knew very little about,
and I bet you don’t either.
That’s coming up after the break.
I’m Stephen Dubner,
and this is Freakonomics Radio.
(dramatic music)
– Good morning.
Good morning.
We have a session this morning,
as you know, on adversarial collaboration.
– Day two of the conference,
with a panel devoted to a different way
of doing business in the behavioral sciences.
Let’s begin with a short clip
from the late Danny Kahneman.
This was recorded in 2022.
– Controversy is a terrible way to advance science.
It is normally conducted as a contest,
and with the aim is to embarrass.
The feature that makes most critiques
intellectually useless is a focused
on the weakest argument of the adversary.
It is common for critics
to include a summary caricature of the target position,
refute the weakest argument in that caricature,
and declare the total destruction
of the adversary’s position.
It’s rare for anyone to concede anything.
Doing angry science is a demeaning experience.
– Doing angry science, Kahneman came to believe,
was a terrible thing.
He knew this firsthand.
Before the accolades, the prizes, the acceptance,
Kahneman and Tversky came under a great deal of criticism.
– Kurt Gigerendzer published his first critique
of our work 37 years ago,
and he’s still not done with me.
– And here again is Richard Thaler.
– I’ve certainly had my share
of angry science exchanges.
They’re no fun, and there’s never any light, only heat.
– And that’s why Kahneman came up with a different model,
what he called an adversarial collaboration.
– He felt that this is the right way to do science.
All of the collaborations shed more light than heat.
The typical debates in academia are exactly the opposite.
So I think Danny has a good point.
– Kahneman was trying to solve a fundamental problem
in social science research.
It’s natural for scientists to disagree with one another,
but there’s no clear mechanism
for resolving those disagreements.
So what if researchers who disagree work together
in good faith to resolve the issue?
And what if they take on a neutral third party
to serve as an arbiter?
That is what Kahneman called an adversarial collaboration.
He participated in a variety of them over the years.
The conversation you are about to hear
includes some of the key players.
Let’s start with introductions.
– I’m Tom Gilovich.
I’m a social psychologist at Cornell University.
– I’m Barbara Millers from the University of Pennsylvania.
And I was fortunate to be the arbiter
in two adversarial collaborations with Danny, 20 years apart.
– I’m Matt Killingsworth.
I’m also at the University of Pennsylvania.
And I study human happiness and all being,
like what makes life worth living
and how do we collect data to try to understand
what makes life better?
– My name is Shane Frederick.
I am a professor at the Yale School of Management.
(audience applauding)
– Tom Gilovich, let’s start with you.
Let’s hear basically the story
of the adversarial collaboration with Danny.
It began quite early in your career.
Give us the whole story.
– I had published with Vicky Medvick a chapter
in a book on counterfactual thinking.
When I wrote the chapter, I don’t know if I was aware
that Danny was going to write the wrap up chapter
where he comments on all of the other ones.
So I don’t know if his voice was in my head
as we’re writing the chapter.
– What’s the chapter about?
What’s it cover?
– A lot of it covers the subject of regret,
what we regret most in life.
Amos and Danny had published a study showing
you regret things of action more than inaction.
The example they used is imagine you own stock
in one company, you think, okay, that’s run the course.
You switch to another and it tanks,
you lose a certain amount of money.
Versus you’re thinking about buying this stock,
you decide not to, it takes off
and you lose the same amount of money.
Which would you regret more?
And almost everyone anticipates
that you’d regret the action the most.
Nonetheless, if you ask people,
what are the biggest regrets in your life?
What dominates are things they didn’t do?
And so a lot of the chapter was
how do you reconcile those two things?
– In approaching this riddle or puzzle in your mind,
how did you measure what was your actual research?
Our research was what’s responsible for that difference?
How do things change over time?
Precisely because your regrets of action really hurt.
You do things about them.
Sometimes you change your life accordingly
or you certainly engage in lots of psychological work
to achieve some level of peace.
Because the inaction regrets gnaw at you less powerfully,
you sort of leave them alone.
The fun of that research was tracking down
all of the different psychological processes
that make those regrets of inaction hang around
or in some cases even get more intense.
Whereas the regrets of action dissipate.
– How do you do that though?
Is it empirical?
Is it theoretical?
– It’s empirical.
For example, one of the mechanisms
that we thought was especially interesting
is when you don’t do something
that you later on end up regretting.
There are reasons for that.
I just don’t have the bandwidth right now
to take this project on, et cetera.
Those are compelling reasons in the moment
for why you’re not doing this thing.
You move along in time, you look back and you think,
wait a minute, I could have done that.
So we would do studies where we’d ask
Cornell current students, recent alums,
or much older alums.
Imagine there’s a class that you always wanted to take
but maybe you were a little afraid to.
Suppose we added that to your curriculum
this semester, the students are like,
oh, I can barely hang on right now.
You add that, it’s a catastrophe.
You ask the people who graduated many years ago,
they think, oh, that’s a piece of cake.
I wouldn’t have interfered with my GPA, my social life,
the amount of sleep I get, et cetera.
– Okay, so you write up the findings.
At what point then and in what way do you hear from Danny?
– It’s sort of a highlight of my career.
I was on sabbatical, check in your voice messages
back at Cornell and I get this message.
Tom, this is Danny Kahneman.
I’ve just read your paper with Medvic, it’s brilliant.
It made my day just calling to say thank you.
What a piece of work.
And then I stupidly press star delete
instead of saving that message forever
to play to my grandkids.
So I’m just on a high for a while.
Two weeks later, there’s another phone call
from Danny Kahneman, this time live.
Oh, great, my new best friend is gonna tell me about…
Tom, I’ve been thinking more about your chapter.
It’s all wrong.
I’ve heard you’re planning to publish this.
You can’t do that.
If you do, people will go after you.
And if no one does, I’m gonna go after you.
What happened to Brilliant?
Did he, in that conversation, give you his argument
for why you thought now you were wrong?
Yes, he said that there really isn’t a change
in the intensity of either action or inaction regrets.
There’s just a substitution of the kinds of things
that you’re regretting.
And the regrets of inaction really aren’t serious regrets.
You might not even call them regrets.
They’re more wistful.
Like, oh, I wish I had learned to speak Esperanto.
Said no one ever.
Yeah. (laughing)
All right, so what happens next?
Well, lots of panic for a few days.
Vicki Medvick and I, what do we do?
It can’t be worse than this.
And I think, I forget what there had been an example
of an adversarial collaboration.
I think it was Vicki’s idea.
Hey, we should negotiate with him.
Let’s see if we can do an adversarial collaboration.
So we pitched it and he was very receptive.
This was in what year, Tom?
Chapter came out in ’95, so it was a little before that.
So this was pre-zoom, certainly, early internet.
How are you communicating?
And how did you negotiate the negotiation?
Then how did the actual negotiation happen?
I’m glad you asked that question
’cause it allows me to bring up a point about Danny
that there’s been all this talk about his many qualities
and they’ve all been well said.
One thing kind of gets left out.
I mean, to be at that level of success,
you have to be brilliant and creative
and many people have spoken about that.
You just have to be a phenomenally hard worker.
And man, he was on task all the time.
It was amazingly easy to get that guy on the phone.
I could get him on the phone more readily
than I could get my co-author, Vicki, on the phone.
And something seemed wrong about that.
He was always ready to engage
and so a lot of it happened over the phone.
And the idea was, no, these regrets of inaction
that you say are important in people’s later lives.
They’re just sort of wistful regrets
and the intense hot emotions that come from action.
Those are different kinds of things.
So we designed some studies where we asked people
to think about the biggest regret of action or inaction.
You designed the studies together.
Then we ran them together, yes.
So think of your biggest regrets of action or inaction
from the recent past or the distant past.
How many of these emotions do they lead you to feel?
We had a set of five hot emotions,
a set of five wistful emotions.
We thought that those long-term regrets of inaction,
some of them are wistful.
I wish I had learned to play the piano,
but some of them are really intense.
You met the right person at the wrong time
and so you let it pass and now you’re looking back like,
oh, my life would have been much better
if I had acted on that.
We also asked people to rate whether a set of five,
I think we called them emotions of despair.
I’m depressed when I think about this.
I feel empty when I think about this.
And lo and behold, Danny was right.
A lot of long-term regrets of inaction
are kind of wistful, but we were right too.
They also produced these big, powerful feelings of,
oh, my life is not what it could have been.
I feel empty and depressed thinking about these things
that I didn’t do that now seem so easy to have done.
– Did Danny, as a result,
acknowledge that you were partially right
as much as he acknowledged that he was partially right?
– Yeah, yeah, and that’s in the paper.
– So truly a happy ending.
– I believe so, yeah.
Certainly more happy than that second phone call
that I got to.
(audience laughing)
– You then went on to collaborate with Danny
several times, correct?
– Yes.
– I’m just curious now that Danny is gone
and since your initial contact
was this adversarial collaboration about regret,
is there anything that you now regret
not having worked on with Danny?
– I wouldn’t say that there’s a particular topic in mind,
but to have someone in your life like that
and not take full advantage of it,
I wish I had just reached out to him more
to have more contact with him.
One of the nice things about this event here
is this family feeling of all the people
he reached, some of whom I know well,
some of whom I didn’t really know at all until now.
They feel like family.
So, you know, often with families,
you wish you had spent more time with them.
– Not my family, but, you know.
(audience laughing and applauding)
– The adversarial collaboration with Tom Gilovich
was one of the earliest ones that Danny Kahneman undertook.
Let’s take a look now at one of the most recent
with Matt Killingsworth from the University of Pennsylvania.
He studied engineering as an undergraduate
and got his PhD in psychology in 2012.
Danny Kahneman got his psychology PhD in 1961.
So, how did Matt Killingsworth end up
in an adversarial collaboration
with this giant in the field?
– Danny and Angus Deaton, both Nobel Prize winners,
had this conclusion that there was sort of this plateau
in people’s happiness once they reached $75,000 in income
and then I’d published a paper
that basically showed something completely different
and we tried to figure out who’s right and what’s the truth.
– You say it with this sort of sang froide
that, you know, we published something completely different,
but as you noted, you were publishing a paper
in opposition to the findings of not one,
but two Nobel laureates.
Were you as calm and cool about it in the moment
when you decided to take up this route
as you seem to be now?
– I mean, partly Danny is to blame,
although he didn’t learn that before he passed away.
I had written this as like a sub point to a sub point
and another paper and actually reading
Michael Lewis’s book about Danny and Angus,
they were talking about how they took on existing ideas
and that was an important part of their intellectual journey.
And so I sort of rolled the dice and thought,
“Well, I’ll try it.”
But yeah, there was definitely some part
in the back of my mind of like, how is this gonna go?
I do have to ask at this particular conference,
which is held in celebration
of Danny Kahneman’s work in life,
do you feel a little bit like, you know,
the wolf in the hen house
or that someone’s gonna come up and shiv you in the back
at the buffet?
– I hope not.
No, I mean, we really had a wonderful collaboration
and I think part of what characterized that is,
we both just wanted to figure out the truth.
I don’t think anyone was attached
to any particular version of reality.
My sense from him is that he was a little bit irritated,
not a lot.
– A lot is a lot.
I mean, he wanted to have a good conversation,
but he wasn’t–
– For the record, that was Shane Frederick saying a lot.
(laughing)
Barb Millers, let’s bring you in here.
How did you get attached to this adversarial collaboration?
– Well, I was talking to Danny on the phone,
not too long after Matt’s paper came out
and he said, “Oh, by the way,
“do you happen to know a guy by the name
“of Matt Killingsworth?”
And I said, “Well, as a matter of fact,
“I do, and if you’d like me to be the arbiter,
“I’d be happy to do so.”
(laughing)
In adversarial collaborations,
the arbiter is the research assistant,
the tiebreaker, and occasionally the therapist.
– So Matt, how would you characterize the response
to that original Kahneman-Deaton finding
about the $75,000 happiness cutoff,
whether inside academia or beyond?
– I mean, it was very influential,
and I mean, it’s probably one of the most visible,
kind of an exciting findings.
I think a lot of us feel like money is sort of overemphasized
in our daily lives, and it kind of gives a justification
for maybe carrying a little bit less about it
and focusing more on all the other stuff
that’s also really important for happiness,
particularly because money is quantitative,
you can kind of think of them as points.
I’m like, “Well, I wanna get more,”
but you can easily imagine a sort of trap
where you’re just continuously trying to get
more and more and more of those points,
ignoring all of this other stuff,
and wouldn’t it be nice
if we could kind of step out of that cycle?
At least that’s part of the reason
that I think that that was so attractive.
– So Matt, you’ve rehearsed very nicely
the original finding, now bring you into the story here.
What are you doing at the moment?
– Sure, so partly also due to Danny,
my research program is really centered
on large-scale experience sampling,
and to get at what that matters in the original study,
those data were collected, how?
In the original study, they basically asked,
“Did you experience a lot of the following emotion
“yesterday, yes or no?”
And then it had a series of emotions
like sadness, happiness, stress, et cetera.
People would either say, “I did or I didn’t.”
– And the technology or mechanism
of harvesting those data was what?
– I wasn’t involved in that data collection.
I believe those were verbal phone calls,
and then sort of interviewing people over the phone.
– Does anyone here know anything different,
or does that sound right, as far as we know?
– That’s right.
That was said with the voice of a referee, I have to say.
(audience laughs)
Okay, Matt, continue, please.
So in my study, I’m essentially measuring
people’s experience right in the moment.
They’re carrying around on their phone an app,
and I’m beeping them at random times.
I’m asking them, “How do you feel right now?”
And they’re responding on a scale
that ranges from very bad to very good,
and the scale is continuous.
So there are a couple of things
that distinguish that from the earlier paper.
One is that it’s right at the time
that people are feeling it,
as opposed to the day as a whole, retrospectively.
And the other is that gradations of feelings
are on a continuous scale,
as opposed to something that’s binary or dichotomous.
– And the scale is what to what?
It’s zero to 10, zero to 100.
– Very bad to very good, and it’s just continuous.
– So you can rake in anywhere you want.
– There are hundreds of unique values.
– Got it.
– And describe the differences
between the pools of research subjects
in the original case and in your case?
– In the original case, it was a survey by Gallup.
It was either representative or at least an attempt
to be representative of whatever random digit dialing was.
That’s a relative strength of their paper
and of that study in general.
My sample, in contrast, was really a convenient sample.
It’s turned out to be an amazing sample
of people that have really beautiful results,
but it was essentially whoever was willing to sign up
to try to understand their own happiness.
– What was the recruiting mechanism?
– Largely thanks to folks like you,
people in a hearing about it in the media,
on the radio, reading about it.
I think that’s interesting.
I’d like to learn about that for myself.
I’m willing to contribute to science.
And it turns out that when I look at,
what’s the distribution of incomes?
For example, in my data,
it’s almost a perfect match for the US census.
I can replicate all kinds of things
that we’ve seen in the literature for decades.
So it turns out to be really good,
but that’s kind of a lucky coincidence.
– Okay, so you gather, assemble, analyze your data,
talk about from the analysis point
to writing up the findings.
What I find when I look at this relationship,
plotting people’s happiness in the moment
versus their income, it just keeps going up.
This sort of critical point
where they had found in the earlier paper,
this flatlining, I really see no difference at all.
And when I write it up in the ultimate paper,
I compare, well, what’s the slope below the point
where they said it stops increasing?
And the point above that,
the slopes differ by less than 1%.
I really see no evidence for a difference at all.
Were you initially looking for that plateau in the data?
– I was really just trying to understand
what’s the relationship between these things
that are obviously important.
We’ve never had, in the moment, continuous data.
And so I want to see, what does this look like?
And it turns out, well, it doesn’t look like
what we thought it looked like.
– When you strike at a king, you must kill him.
That’s from Ralph Waldo Emerson.
Matt Killingsworth did strike
at the king of his realm, Danny Kahneman.
But what happened next?
It’s coming up after the break.
I’m Stephen Dovner, and this is Freakonomics Rating.
(upbeat music)
We’ve been hearing from Matt Killingsworth
about his research on the relationship
between income and happiness.
Research that disputed elements of earlier research
on the same topic by Daniel Kahneman.
– So you write up the paper,
and the paper, not quite directly,
but quasi-directly, addresses the fact
that your finding is contra
to a significant earlier finding by significant scholars.
What happens next?
– There’s a fair amount of attention about it and so forth.
Maybe a month or two afterwards,
I get a note from Barb that she’s been chatting with Danny.
– He’s talking about Barb Mellors,
a longtime friend of Danny Kahneman,
as well as a University of Pennsylvania colleague
of Matt Killingsworth.
– I think both of our attitudes
was we just wanna find out the truth.
I have no personal attachment
to what I found particularly,
and I don’t, other than his initial perhaps irritation,
I don’t really think Danny did.
It was really just what do we think is going on.
– Did you envision that perhaps you would then collaborate
with Danny on a joint study?
– That certainly seemed to be the case
once we got into it,
but these kinds of data took me many years
to collect that data for them.
There were a client on an external organization
with a lot of resources,
and so we ended up doing it by going back
and looking at my data, which existed,
which made it tractable.
I can sort of cut to the chase a little bit
of how we did that if you want.
– I wanna go back to Barb for just a second here.
Barb, if you could give Danny’s perspective
or participation up to the point here
where we’re about to cut to the data.
– The starting point for Danny was the assumption
that both data sets were valid.
So how could they be so different?
There must be a resolution in the data somewhere,
and so the task was going to be,
Matt became the research assistant
and started doing all the re-analyses on his data.
– How would you characterize the tenor
of this adversarial collaboration?
– Really civil and nice,
and just the way it should go.
– So Matt, you said you could cut to the chase.
We’re at the chase now, cut to it.
– Essentially, the resolution was
if we look at the range
within my continuous happiness data,
what if we look at the low end,
which is the part where their measure
would have been sensitive,
can we find a similar pattern?
And lo and behold, when you zero in on that,
rather than looking at the average trend,
which keeps going up,
you get not exactly, but very, very close
to what they found,
certainly much closer to what they found
than to what the average trend looks like.
We both found that pretty convincing evidence
that what they found is true,
there’s nothing wrong with the analysis at all,
but it was really a question of,
how generally applicable is that?
So from low to medium incomes,
the unhappiest part of the distribution rises a lot,
and then from medium to high incomes,
the unhappiest part of the distribution barely changes.
But the rest of the distribution is rising steadily,
and in fact, at the high end of the happiness distribution,
you get an inversion of that.
So rather than a rise and then a plateau,
you have sort of a shallow slope and then an acceleration.
– So in my lay mind, as I’m trying to process all this,
what I’m envisioning is that there is a cohort of people
who are kind of cranky,
and they may experience a little bit less crankiness
as income rises,
but there is a range in which their temperament
may be overwhelms their income.
– If anything, there’s one of the steepest slopes
for the unhappiest people in the range of low income.
So getting out of poverty, if you’re really miserable,
is at least correlationally.
– But that’s almost a different story,
’cause escaping poverty versus going from middle to upper.
– If you aren’t poor and you’re really miserable,
at least if you sort of extrapolate from there,
it seems to not matter past that.
And probably as we speculate in the paper,
at that point, if you have a decent amount of money
and you’re really unhappy,
whatever’s making you unhappy probably isn’t due
to the lack of resources.
It’s something else going on in your life.
Maybe it’s challenges in a family relationship,
or maybe you’re depressed, or whatever it is,
it’s something going on that perhaps money
isn’t going to make a difference.
– Richard, Taylor, let me ask you a question.
You’ve just heard Matt’s presentation of his side
of the original research,
and then the adversarial collaboration,
and then the conclusion,
all of which I found honestly to be really fascinating.
You know the literature well.
You too have a Nobel Prize,
although I have heard that was a clerical error.
(audience laughing)
How would you, looking at this from above,
Barbara was there as an arbiter,
Matt was one side,
Danny was another side, but is not present,
if you could give an ultimate proclamation on,
not just where this finding arrived,
but what the adversarial collaboration produced here,
what would you say?
– Look, I think this is the way it should work.
This is a good story,
and the world would be a better place,
academia would be a much better place
if there were more of these kinds of collaborations.
– Would it affect what we sometimes
call the replication crisis?
– Uh, oof.
– Barbara says, Barbara’s nodding her head, no,
you say oof.
– Yeah, let’s move on to Shane.
(audience laughing)
– The discomfort you were hearing there
about the replication crisis,
that discomfort is related to a two-part series
we recently produced on academic fraud,
episodes 572 and 573.
And while we’re on the topic of discomfort,
let me add one thing I’m thinking about
as we’re in the middle of this conversation
about adversarial collaborations.
Danny Kahneman is said to have introduced this concept
into the realm of behavioral research,
but if you’re not a behavioralist,
you could look at this as yet another example
of how academic researchers discover something
that people in the real world have been doing forever.
There are all sorts of arbiters and referees out there,
all sorts of processes for compromise and negotiation,
even war games to test one plan against another
and come up with the best solution.
In the same vein, some people look at the big insights
that have come out of this behavioralist research
as essentially ESOP’s fables with a little bit of math.
Earlier this year,
we made a series about the late physicist Richard Feynman.
He didn’t think the social sciences like psychology
and even economics should be called science at all.
He thought they were just too squishy to deserve that name.
And as for psychologists discovering the idea
of adversarial collaboration,
well, Feynman was a junior member of the Manhattan Project,
which in its quest to build an atomic bomb
brought together the most brilliant
and argumentative group of scientists ever assembled.
That was adversarial, and yes,
it was in the end a successful collaboration.
Anyway, back to Chicago.
As Richard Thaler suggests, let’s move on.
Shane Frederick, he is a behavioral scientist at Yale
who collaborated with Danny Kahneman on several projects
and is cited throughout Kahneman’s book,
Thinking Fast and Slow.
Kahneman saw Frederick as a protege
and they had a close relationship,
sometimes a bit too close.
– So I’m at 10 o’clock on a Monday,
maybe like third quarter of Monday football.
I get a call.
– Shane, you said, are you watching the game?
– It was one of the great times I wasn’t.
I said, no, fantastic.
I want to talk about heuristics.
(audience laughing)
Another time, it was like two o’clock in the morning,
going back and forth, back and forth, back and forth.
And thanks, I’m getting sleepy.
And so, go to bed and I wake up in the morning
in the very next email from Danny,
“Shane, don’t play dead on me.”
(audience laughing)
– One idea they worked on together
was the cognitive reflection test, or CRT.
It is meant to measure a person’s ability
to override their gut instinct
and think more carefully about a problem.
Here’s a famous example.
A bat and a ball cost $1.10 in total.
The bat costs $1 more than the ball.
How much does the ball cost?
If your first inclination is to say, 10 cents,
congratulations, you were like just about everyone else.
What the CRT measures is,
can you slow down and actually calculate the answer
rather than just go with your gut?
If you can, you will find that the answer is five cents.
Remember, the bat costs $1 more than the ball.
Shane Frederick has spent much of his career
designing such tests.
– Some of the items are novel, some are invented,
many of them I didn’t, I collect, some I’m revised.
One goes back to at least 1919, their variance of it.
– They were used then in an academic setting,
or like an employment?
– No, just like in books and riddles,
stuff that Maya would read.
(audience laughing)
– But they weren’t stumpers.
– No, they weren’t stumpers.
– I think I must tell the audience what a stumper is, okay?
– That again is Maya Bar-Huelal.
– A one-way ride costs $10,
a round trip costs $20,
a passenger hands the cashier $20,
saying absolutely nothing.
The cashier knows right away
that the passenger wants a round trip
rather than a one-way and change.
And the question is, how did the cashier know this?
– Does anyone in the audience who does not know the answer
to this stumper previously wanna take a guess to the answer?
– If you’re stumped good,
because that’s why they’re called stumpers.
– A stumper is unlike one of Shane Frederick’s CRT questions
in that your intuition doesn’t produce
an obvious but wrong answer.
– It only has to do with my love of riddles,
which I share with my partner in this research,
Shane Frederick.
And we really came to this out of love of riddles,
but we’re both professional psychologists.
So we felt like we had to approach it
from the point of view of psychology.
Now, Danny.
– Wait, wait.
– Oh.
– Answer please.
– Oh, no way.
No, I’m not gonna answer.
That’s not what I was paid for.
(audience laughs)
– All right, I will answer.
He handed the ticket agent two 10s.
– Ah, very nice.
– Did they pay you for that?
(audience laughs)
– I’m getting paid the same as you, Maya.
(audience laughs)
– But there was something more at stake here
than answering a riddle.
Barr Hillel, along with a co-author,
wrote a paper arguing that cognitive reflection tests
are not a good measurement of anything beyond math skills.
And this argument led to an adversarial collaboration.
– When I heard that Shane and Maya and Danny
were having an adversarial collaboration,
I didn’t know who was on which team.
Maya and Danny go back 60 years.
Shane is like his son for Danny.
– The results of this collaboration
have not yet been published,
but Shane Frederick says that he and Kahneman
essentially proved that the CRT is a good measure
of cognitive abilities beyond just math skills.
– It seemed to do quite well,
and it’s doing well consistently.
Time preferences, risk preferences,
other sorts of things.
– And how would you characterize the nature
of the collaboration?
How adversarial was it?
– It wasn’t so adversarial between the opposing groups.
It’s just like everybody’s fighting with everybody else,
but everything.
(audience laughs)
– More like a family gathering, right?
(audience laughs)
– Does this suggest that an adversarial collaboration
is not as useful as one might hope
if the actual adversaries are not often
in the ring with the collaborators?
– I mean, I don’t think it will work.
Do you, Barb?
– Do these things work?
Well, what does work mean?
It depends on your definition.
If you think people change their minds all the way,
no, it doesn’t work.
If they change their minds a bit, that’s good.
And it speeds up science too.
Somebody’s looking over your shoulder
and making sure you’re not making mistakes,
you’re defining variables more precisely,
you’re designing an experiment
that gets right at the core issue.
It’s the way to go.
(piano music)
– Well, I could listen to the five of you talk
for five hours.
There are sessions that need to happen.
This room is turning over.
I thank you so much.
This was a great conversation.
(audience applauds)
– Sorry, there is a clause in the contract
that Thaler must always have every last word, so.
– I was just gonna say something nice about you,
but if you insist, I’ll skip it.
(audience laughs)
– And that was, again, Richard Thaler.
I would like to thank him for putting this event together
and inviting us to join.
Thanks also to the other participants.
It was a pretty wonderful event
and I’m glad we were able to share it with you.
And if you need more Richard Thaler in your life,
and who doesn’t need more Thaler in their lives,
we are going to publish a bonus episode very soon,
an update of a great conversation
I had with Thaler a few years ago.
It’s called People Aren’t Dumb, The World is Hard.
So keep your ears out for that.
Meanwhile, on the next regularly scheduled episode
of Freakonomics Radio, a close-up look at an industry
that’s all about close-up looking.
I like for my glasses to have a bit of pizzazz,
especially if you’re wearing them every day.
– We’re gonna see half of the global population
being myopic by 2015.
– When you are a vertically integrated player,
you master every step within the value chain.
– The margins, even by luxury good standards, are obscene.
– Eyewear is a $150 billion industry
and what you see is not quite what you get.
That’s next time on the show.
Until then, take care of yourself,
and if you can, someone else too.
Freakonomics Radio is produced by Stitcher and Renbud Radio.
You can find our entire archive on any podcast app
also at Freakonomics.com,
where we publish transcripts and show notes.
This episode was produced by Zach Lipinski,
with live recording by Greg Rippen.
Special thanks to conference organizers Amy Boonstra,
Mark Tomelko, and Chris Partridge,
as well as the Black Oak AV team.
Our staff also includes Alina Cullman, Augusta Chapman,
Dalvin Abouaji, Eleanor Osborn,
Elsa Hernandez, Gabriel Roth, Jasmine Klinger,
Jeremy Johnston, Julie Canfer,
Lyric Bowditch, Morgan Levy, Neil Coruth,
Rebecca Lee Douglas, Sarah Lilly, and Teo Jacobs.
Our theme song is “Mr. Fortune”
by the Hitchhikers.
Our composer is Luis Guerra.
As always, thank you for listening.
– I think my mind was wandering.
I have that effect on people, I’ve been told.
(audience laughing)
(upbeat music)
– The Freakonomics Radio Network,
the hidden side of everything.
(upbeat music)
Stitcher.
(gentle music)
Today on Freakin’ Amic Radio, a very special episode,
a conversation about the late Daniel Kahneman,
whose insights into human behavior
have been threaded through this show for years,
ideas like confirmation bias and loss aversion
and the planning fallacy.
During this conversation,
we also learned about a research paradigm
that Kahneman embraced called adversarial collaboration,
which means working shoulder to shoulder with your rivals.
He felt that this is the right way to do science.
Kahneman was a phenomenally influential psychologist
who won a Nobel Prize in Economics,
wrote the bestselling book, Thinking Fast and Slow,
and left behind an army of collaborators,
mentees, and admirers.
With them, we will take a careful look
at the life and mind of Danny Kahneman starting now.
(upbeat music)
This is Freakin’ Amic Radio,
the podcast that explores the hidden side of everything
with your host, Stephen Dovner.
(upbeat music)
Last month in a sunlit auditorium
overlooking the Chicago River,
there was a gathering of psychologists, economists,
and other social scientists.
This was the Behavioral Decision Research
in Management Conference.
The keynote event was supposed to be
a conversation with Danny Kahneman,
facilitated by Richard Thaler,
his longtime friend and collaborator.
Thaler is the University of Chicago Economist
who helped turn Kahneman’s insights
into the field now known as behavioral economics.
But when Kahneman died in March at age 90,
Thaler came up with a new plan for the conference.
Now it would pay tribute to Danny Kahneman.
Freakin’ Amic Radio was lucky enough to be asked along
to moderate a couple of panel discussions
about his life and work.
The episode you’re about to hear
is a condensed version of those conversations.
This all took place at the downtown outpost
of the University of Chicago’s business school
in front of a couple hundred attendees.
Some of the panelists had known Danny Kahneman
for many decades.
For instance, the psychologist Maya Bar-Hillel,
her father was a philosophy professor
at Hebrew University in Jerusalem,
where Kahneman got his undergraduate degree.
– My father taught Danny and gave him a lot of grief,
but my father apparently gave just about everybody
a lot of grief.
He was a tough-minded philosopher.
– And when Kahneman became a professor,
one of his students was Maya Bar-Hillel,
from generation to generation.
– I met Danny in the first week of my first year
at the Hebrew University.
He gave the introductory statistics course.
He looked at us and he pronounced right away,
you are the creme de la creme.
He said it in French and we were.
(audience laughing)
– Kahneman had grown up in France
during the Nazi occupation.
He survived, barely, and lived for many years in Israel
before coming to the US to get his PhD at UC Berkeley.
His research partner, Amos Tversky,
was another Israeli psychology professor
who moved to the States.
Tversky died young in 1996,
too early to share in what would have surely been
a joint Nobel Prize.
Tversky was regarded as perhaps
even more brilliant than Kahneman.
The two of them published many papers
on judgment and decision-making,
but not just in psychology journals.
– It always struck me just in the story
of how Kahneman and Tversky research
was taken hold of by Thaler and others
and turned into what we now call behavioral economics.
That it was very, very important
that these were two psychologists
who were also very mathematically fluent.
If that hadn’t been the case
and if the publication hadn’t been in,
I guess, econometrica and so on,
was there a pretty good possibility for a counterfactual
where all that research might have stayed
within the realm of psychology and never trickled over
and we might not have what we think
of as behavioral economics?
– It was not an accident.
It’s not how fortunate that they went to econometrica.
They realized that their work
was attended to primarily by psychologists
and in fact, they both considered themselves
all their lives as psychologists,
but they also realized that their research
was perhaps more important outside of psychology.
So the decision to publish their paper in econometrica
was a deliberate move.
It was a strategic move to get the attention.
They believed that that was the ticket
and without the ticket,
they would not have been playing in that field.
– A lot of the early Kahneman-Tversky work
centered around an observation
that may seem obvious in retrospect,
but at the time had not been explored with much rigor.
The idea was that we are all constantly making decisions,
personal, professional, political decisions.
And then later, if we ask ourselves
why we made a given decision,
which by the way, we usually don’t ask,
we might tell ourselves a story about making the decision,
but these stories are often not quite true.
Why?
One reason is that we want to appear to others
and maybe even ourselves as smarter than we are.
Here is Richard Taylor.
– I met Danny and Amos in 1977
and it was a transformative year for me.
I decided to change jobs and took a job at Cornell
and decided to offer a course
in the thing I was now fascinated by
and called it behavioral decision theory,
which is kind of what the name of this field used to be.
I got about eight students.
So I had to think of something to do
to increase the enrollment in the class.
And so I changed the name to managerial decision making.
50 students show up.
I began the class by asking how many of you
signed up for this class because of the name?
No one raised their hand.
I said, well, actually nearly all of you.
That’s one illustration of what Danny’s talking about.
No one thinks they would be stupid enough
to sign up for a class based on the name.
– I believe it was in thinking fast and slow
where Danny wrote, not only are we blind,
but we are blind to our blindness.
– Yeah, and you know, maybe one of the many secrets
of the Conor Monteverski collaboration was
they were not blind to those.
And what they had was a mistake detection facility.
They had some radar where they could anticipate
what the mistake is.
And because they had this mistake detection facility
somewhere, they were able to figure out things.
And then they were pretty good at predicting
what people other than Amos and Danny would do.
To me, the big point, the aha point that I got
from reading their judgment papers
was the idea of predictable bias.
– May I add something to that?
– Yes, please.
– Not only predictable.
I think now I’m gonna say something that is perhaps,
I can’t say that I heard them say it,
but I hope they would both agree,
that the errors are not just predictable.
The errors are smart.
Our stupid mistakes are evidence
of the wonderful human mind.
That’s how normal cognition functions
and normal cognition is amazing.
– I love that.
It does make me wanna ask any, all of you,
a question that’s fairly heretical,
which is, you know, as a layperson,
I read these findings, they’re attractive,
they’re believable, they’re identifiable.
But the thing that I always struggle with
or would like to understand better
is how you all can feel so confident
that they’re generalizable.
There must be people in the world
who are not susceptible to loss aversion
or recency bias or many, many of them.
So really the question is, how much variance is there
and how do you measure the variance?
– You know, Amos used to have a joke
that there were species that didn’t exhibit loss aversion
and they’re now extinct.
(audience laughing)
– Think about what Maya Bar-Huelel was saying there
about the smart errors we all make.
It would be easy to overlook the baseline insight
that Danny Kahneman offered,
that people make thinking mistakes all the time.
Now, most of us, upon hearing this,
might say, no kidding.
Our species is highly fallible, who doesn’t know that?
But it is our fallibility, Kahneman realized,
that makes us interesting and worthy of inspection.
He accepted that humans are capable of wonderful things
as well as terrible ones,
but that we are overconfident in our abilities,
that we often have poor self-control,
and that we employ an arsenal of mental shortcuts
or heuristics to make decisions or reach judgments
that often turn out poorly.
And even when presented with evidence of our mistakes,
we usually fail to change our minds.
Here is Eldar Shafir,
who runs the Kahneman-Triesman Center
for Behavioral Science and Public Policy at Princeton.
He and Kahneman used to co-teach a class.
– One lecture that I love that Danny used to give
in our course, we would talk about people’s
not good sense about conditional probabilities.
And then we had a clip of the O.J. Simpson trial
where Dershowitz explains to the jury
that the probability that a beaten woman
is gonna be killed by her beating partner
is extremely low, which is true.
However, we’re not predicting the chance
that Nicole Simpson will be killed.
She has been killed.
The probability that a beaten woman who has been killed
was killed by her partner is immensely high.
That nuance of not being sensitive to these conditionals
has major implications.
He was so genuinely curious and intellectually alive,
he presented the same way to the Swedish monarchy
and to an undergraduate.
He just wanted to think about it.
He wanted to struggle with the question.
He wanted to listen and think of the best theory.
That was what’s so impressive.
When we talk together, we took turns lecturing.
Here was Danny, I mean, before and after the Nobel,
and every class he would come and sit down
with the students and listen.
He could easily not have shown up.
It’s other people’s lecturing.
He was always there, asking questions,
answering questions, devoted to understanding.
Always had a new thought about something
that we have done on slides for three years in a row.
(gentle music)
And many things were written about Kahneman.
I’m always amused by the fact that they say
he was aware of these quirks and fallacies,
but he was aware that he has them too.
And that sort of implies that since he was aware,
it wasn’t as big a deal.
There’s a couple of recorded interviews with Danny
during COVID, where he predicts
that COVID means life imprisonment
for the more advanced in age.
And he was wrong.
He lived a couple of wonderful years after COVID.
There was a sense in which he really sank deep
into the same error that he was able to predict
and acknowledge.
The other amazing thing about Danny
is when he won the Nobel Prize, you have to give a lecture.
And he gave a lecture and decided he was going
to give a new interpretation of all of his work with Amos.
Using this two system approach.
I kept saying, Danny, you have two months to do this.
Why are you starting over?
They gave you a Nobel Prize for what you did.
You don’t need a new reinterpretation,
but that lecture in Stockholm was the beginning
of what led to thinking fast and slow
because he was reinterpreting all the heuristics
and biases stuff in the lens of system one
and system two, which is your immediate reaction
versus when you sit down and think about it.
– What this story suggests to me,
but tell me if I’m wrong, is that he understood
that having an audience like the Nobel committee audience
and the King or Queen or whatever of Sweden
and that his voice was now amplified
that he wanted to take the work that he’d done
and make it accessible, make it known
to people who have some levers of power.
He wanted to exploit a good opportunity.
– I don’t like the word exploit.
– Exploit for pro-social reasons, a good opportunity.
– Thank you, thank you.
– Yeah, I think he had an audience
and wanted to get it out there,
but it was torture for him, absolute torture.
I mean, during the writing of Thinking Fast and Slow.
– Which was a long gestation.
– Long gestation.
And he decided absolutely positively
that he was quitting two dozen times.
– You know, Danny in some sense really was a bon vivant.
I mean, he enjoyed life.
He loved a good restaurant.
He loved a good vacation.
And I think he loved getting the prize.
– I don’t think he changed.
I mean, yes, the world was paying more attention to him,
but he was always Danny.
– He changed.
– He was happier.
– Okay.
(audience laughing)
– No, I would like to say–
– If Maya says that, I accept.
– I would like to say in what way he relaxed.
– That was relaxed?
(audience laughing)
– Yes, yes.
– What do you mean by that, Maya?
– He had made it.
There was a common misperception
that the Tversky-Kahaneman collaboration
was not symmetrical.
The world seemed to think that Amos was the lead
and Danny was an outside visitor.
It was a symmetrical and equal collaboration,
but the world didn’t know it.
It was very important to Danny
to be able to do good work after Amos dies
so that people won’t continue with this error
that he was less than equal.
So the prize told the world,
this man is noble worthy,
and he did excellent work
because he had an excellent mind,
but also it was important for him
that Kahaneman not die the name together with Tversky,
and it didn’t.
– As I said earlier,
the main ideas of Danny Kahneman
have been threaded through the fabric
of Freakonomics Radio over the years.
Episode 323, for instance,
which was about the planning fallacy.
It’s called Here’s Why Your Projects Are Always Late
and What to Do About It.
Then there’s episode 271,
The Men Who Started a Thinking Revolution.
That was an interview with Michael Lewis
about the book he wrote
on the Kahneman-Tversky partnership.
Lewis’ book is called The Undoing Project,
a friendship that changed our minds.
So yes, you may be familiar
with Kahneman’s greatest hits,
but one of the conversations we had at this conference
went into an area I knew very little about,
and I bet you don’t either.
That’s coming up after the break.
I’m Stephen Dubner,
and this is Freakonomics Radio.
(dramatic music)
– Good morning.
Good morning.
We have a session this morning,
as you know, on adversarial collaboration.
– Day two of the conference,
with a panel devoted to a different way
of doing business in the behavioral sciences.
Let’s begin with a short clip
from the late Danny Kahneman.
This was recorded in 2022.
– Controversy is a terrible way to advance science.
It is normally conducted as a contest,
and with the aim is to embarrass.
The feature that makes most critiques
intellectually useless is a focused
on the weakest argument of the adversary.
It is common for critics
to include a summary caricature of the target position,
refute the weakest argument in that caricature,
and declare the total destruction
of the adversary’s position.
It’s rare for anyone to concede anything.
Doing angry science is a demeaning experience.
– Doing angry science, Kahneman came to believe,
was a terrible thing.
He knew this firsthand.
Before the accolades, the prizes, the acceptance,
Kahneman and Tversky came under a great deal of criticism.
– Kurt Gigerendzer published his first critique
of our work 37 years ago,
and he’s still not done with me.
– And here again is Richard Thaler.
– I’ve certainly had my share
of angry science exchanges.
They’re no fun, and there’s never any light, only heat.
– And that’s why Kahneman came up with a different model,
what he called an adversarial collaboration.
– He felt that this is the right way to do science.
All of the collaborations shed more light than heat.
The typical debates in academia are exactly the opposite.
So I think Danny has a good point.
– Kahneman was trying to solve a fundamental problem
in social science research.
It’s natural for scientists to disagree with one another,
but there’s no clear mechanism
for resolving those disagreements.
So what if researchers who disagree work together
in good faith to resolve the issue?
And what if they take on a neutral third party
to serve as an arbiter?
That is what Kahneman called an adversarial collaboration.
He participated in a variety of them over the years.
The conversation you are about to hear
includes some of the key players.
Let’s start with introductions.
– I’m Tom Gilovich.
I’m a social psychologist at Cornell University.
– I’m Barbara Millers from the University of Pennsylvania.
And I was fortunate to be the arbiter
in two adversarial collaborations with Danny, 20 years apart.
– I’m Matt Killingsworth.
I’m also at the University of Pennsylvania.
And I study human happiness and all being,
like what makes life worth living
and how do we collect data to try to understand
what makes life better?
– My name is Shane Frederick.
I am a professor at the Yale School of Management.
(audience applauding)
– Tom Gilovich, let’s start with you.
Let’s hear basically the story
of the adversarial collaboration with Danny.
It began quite early in your career.
Give us the whole story.
– I had published with Vicky Medvick a chapter
in a book on counterfactual thinking.
When I wrote the chapter, I don’t know if I was aware
that Danny was going to write the wrap up chapter
where he comments on all of the other ones.
So I don’t know if his voice was in my head
as we’re writing the chapter.
– What’s the chapter about?
What’s it cover?
– A lot of it covers the subject of regret,
what we regret most in life.
Amos and Danny had published a study showing
you regret things of action more than inaction.
The example they used is imagine you own stock
in one company, you think, okay, that’s run the course.
You switch to another and it tanks,
you lose a certain amount of money.
Versus you’re thinking about buying this stock,
you decide not to, it takes off
and you lose the same amount of money.
Which would you regret more?
And almost everyone anticipates
that you’d regret the action the most.
Nonetheless, if you ask people,
what are the biggest regrets in your life?
What dominates are things they didn’t do?
And so a lot of the chapter was
how do you reconcile those two things?
– In approaching this riddle or puzzle in your mind,
how did you measure what was your actual research?
Our research was what’s responsible for that difference?
How do things change over time?
Precisely because your regrets of action really hurt.
You do things about them.
Sometimes you change your life accordingly
or you certainly engage in lots of psychological work
to achieve some level of peace.
Because the inaction regrets gnaw at you less powerfully,
you sort of leave them alone.
The fun of that research was tracking down
all of the different psychological processes
that make those regrets of inaction hang around
or in some cases even get more intense.
Whereas the regrets of action dissipate.
– How do you do that though?
Is it empirical?
Is it theoretical?
– It’s empirical.
For example, one of the mechanisms
that we thought was especially interesting
is when you don’t do something
that you later on end up regretting.
There are reasons for that.
I just don’t have the bandwidth right now
to take this project on, et cetera.
Those are compelling reasons in the moment
for why you’re not doing this thing.
You move along in time, you look back and you think,
wait a minute, I could have done that.
So we would do studies where we’d ask
Cornell current students, recent alums,
or much older alums.
Imagine there’s a class that you always wanted to take
but maybe you were a little afraid to.
Suppose we added that to your curriculum
this semester, the students are like,
oh, I can barely hang on right now.
You add that, it’s a catastrophe.
You ask the people who graduated many years ago,
they think, oh, that’s a piece of cake.
I wouldn’t have interfered with my GPA, my social life,
the amount of sleep I get, et cetera.
– Okay, so you write up the findings.
At what point then and in what way do you hear from Danny?
– It’s sort of a highlight of my career.
I was on sabbatical, check in your voice messages
back at Cornell and I get this message.
Tom, this is Danny Kahneman.
I’ve just read your paper with Medvic, it’s brilliant.
It made my day just calling to say thank you.
What a piece of work.
And then I stupidly press star delete
instead of saving that message forever
to play to my grandkids.
So I’m just on a high for a while.
Two weeks later, there’s another phone call
from Danny Kahneman, this time live.
Oh, great, my new best friend is gonna tell me about…
Tom, I’ve been thinking more about your chapter.
It’s all wrong.
I’ve heard you’re planning to publish this.
You can’t do that.
If you do, people will go after you.
And if no one does, I’m gonna go after you.
What happened to Brilliant?
Did he, in that conversation, give you his argument
for why you thought now you were wrong?
Yes, he said that there really isn’t a change
in the intensity of either action or inaction regrets.
There’s just a substitution of the kinds of things
that you’re regretting.
And the regrets of inaction really aren’t serious regrets.
You might not even call them regrets.
They’re more wistful.
Like, oh, I wish I had learned to speak Esperanto.
Said no one ever.
Yeah. (laughing)
All right, so what happens next?
Well, lots of panic for a few days.
Vicki Medvick and I, what do we do?
It can’t be worse than this.
And I think, I forget what there had been an example
of an adversarial collaboration.
I think it was Vicki’s idea.
Hey, we should negotiate with him.
Let’s see if we can do an adversarial collaboration.
So we pitched it and he was very receptive.
This was in what year, Tom?
Chapter came out in ’95, so it was a little before that.
So this was pre-zoom, certainly, early internet.
How are you communicating?
And how did you negotiate the negotiation?
Then how did the actual negotiation happen?
I’m glad you asked that question
’cause it allows me to bring up a point about Danny
that there’s been all this talk about his many qualities
and they’ve all been well said.
One thing kind of gets left out.
I mean, to be at that level of success,
you have to be brilliant and creative
and many people have spoken about that.
You just have to be a phenomenally hard worker.
And man, he was on task all the time.
It was amazingly easy to get that guy on the phone.
I could get him on the phone more readily
than I could get my co-author, Vicki, on the phone.
And something seemed wrong about that.
He was always ready to engage
and so a lot of it happened over the phone.
And the idea was, no, these regrets of inaction
that you say are important in people’s later lives.
They’re just sort of wistful regrets
and the intense hot emotions that come from action.
Those are different kinds of things.
So we designed some studies where we asked people
to think about the biggest regret of action or inaction.
You designed the studies together.
Then we ran them together, yes.
So think of your biggest regrets of action or inaction
from the recent past or the distant past.
How many of these emotions do they lead you to feel?
We had a set of five hot emotions,
a set of five wistful emotions.
We thought that those long-term regrets of inaction,
some of them are wistful.
I wish I had learned to play the piano,
but some of them are really intense.
You met the right person at the wrong time
and so you let it pass and now you’re looking back like,
oh, my life would have been much better
if I had acted on that.
We also asked people to rate whether a set of five,
I think we called them emotions of despair.
I’m depressed when I think about this.
I feel empty when I think about this.
And lo and behold, Danny was right.
A lot of long-term regrets of inaction
are kind of wistful, but we were right too.
They also produced these big, powerful feelings of,
oh, my life is not what it could have been.
I feel empty and depressed thinking about these things
that I didn’t do that now seem so easy to have done.
– Did Danny, as a result,
acknowledge that you were partially right
as much as he acknowledged that he was partially right?
– Yeah, yeah, and that’s in the paper.
– So truly a happy ending.
– I believe so, yeah.
Certainly more happy than that second phone call
that I got to.
(audience laughing)
– You then went on to collaborate with Danny
several times, correct?
– Yes.
– I’m just curious now that Danny is gone
and since your initial contact
was this adversarial collaboration about regret,
is there anything that you now regret
not having worked on with Danny?
– I wouldn’t say that there’s a particular topic in mind,
but to have someone in your life like that
and not take full advantage of it,
I wish I had just reached out to him more
to have more contact with him.
One of the nice things about this event here
is this family feeling of all the people
he reached, some of whom I know well,
some of whom I didn’t really know at all until now.
They feel like family.
So, you know, often with families,
you wish you had spent more time with them.
– Not my family, but, you know.
(audience laughing and applauding)
– The adversarial collaboration with Tom Gilovich
was one of the earliest ones that Danny Kahneman undertook.
Let’s take a look now at one of the most recent
with Matt Killingsworth from the University of Pennsylvania.
He studied engineering as an undergraduate
and got his PhD in psychology in 2012.
Danny Kahneman got his psychology PhD in 1961.
So, how did Matt Killingsworth end up
in an adversarial collaboration
with this giant in the field?
– Danny and Angus Deaton, both Nobel Prize winners,
had this conclusion that there was sort of this plateau
in people’s happiness once they reached $75,000 in income
and then I’d published a paper
that basically showed something completely different
and we tried to figure out who’s right and what’s the truth.
– You say it with this sort of sang froide
that, you know, we published something completely different,
but as you noted, you were publishing a paper
in opposition to the findings of not one,
but two Nobel laureates.
Were you as calm and cool about it in the moment
when you decided to take up this route
as you seem to be now?
– I mean, partly Danny is to blame,
although he didn’t learn that before he passed away.
I had written this as like a sub point to a sub point
and another paper and actually reading
Michael Lewis’s book about Danny and Angus,
they were talking about how they took on existing ideas
and that was an important part of their intellectual journey.
And so I sort of rolled the dice and thought,
“Well, I’ll try it.”
But yeah, there was definitely some part
in the back of my mind of like, how is this gonna go?
I do have to ask at this particular conference,
which is held in celebration
of Danny Kahneman’s work in life,
do you feel a little bit like, you know,
the wolf in the hen house
or that someone’s gonna come up and shiv you in the back
at the buffet?
– I hope not.
No, I mean, we really had a wonderful collaboration
and I think part of what characterized that is,
we both just wanted to figure out the truth.
I don’t think anyone was attached
to any particular version of reality.
My sense from him is that he was a little bit irritated,
not a lot.
– A lot is a lot.
I mean, he wanted to have a good conversation,
but he wasn’t–
– For the record, that was Shane Frederick saying a lot.
(laughing)
Barb Millers, let’s bring you in here.
How did you get attached to this adversarial collaboration?
– Well, I was talking to Danny on the phone,
not too long after Matt’s paper came out
and he said, “Oh, by the way,
“do you happen to know a guy by the name
“of Matt Killingsworth?”
And I said, “Well, as a matter of fact,
“I do, and if you’d like me to be the arbiter,
“I’d be happy to do so.”
(laughing)
In adversarial collaborations,
the arbiter is the research assistant,
the tiebreaker, and occasionally the therapist.
– So Matt, how would you characterize the response
to that original Kahneman-Deaton finding
about the $75,000 happiness cutoff,
whether inside academia or beyond?
– I mean, it was very influential,
and I mean, it’s probably one of the most visible,
kind of an exciting findings.
I think a lot of us feel like money is sort of overemphasized
in our daily lives, and it kind of gives a justification
for maybe carrying a little bit less about it
and focusing more on all the other stuff
that’s also really important for happiness,
particularly because money is quantitative,
you can kind of think of them as points.
I’m like, “Well, I wanna get more,”
but you can easily imagine a sort of trap
where you’re just continuously trying to get
more and more and more of those points,
ignoring all of this other stuff,
and wouldn’t it be nice
if we could kind of step out of that cycle?
At least that’s part of the reason
that I think that that was so attractive.
– So Matt, you’ve rehearsed very nicely
the original finding, now bring you into the story here.
What are you doing at the moment?
– Sure, so partly also due to Danny,
my research program is really centered
on large-scale experience sampling,
and to get at what that matters in the original study,
those data were collected, how?
In the original study, they basically asked,
“Did you experience a lot of the following emotion
“yesterday, yes or no?”
And then it had a series of emotions
like sadness, happiness, stress, et cetera.
People would either say, “I did or I didn’t.”
– And the technology or mechanism
of harvesting those data was what?
– I wasn’t involved in that data collection.
I believe those were verbal phone calls,
and then sort of interviewing people over the phone.
– Does anyone here know anything different,
or does that sound right, as far as we know?
– That’s right.
That was said with the voice of a referee, I have to say.
(audience laughs)
Okay, Matt, continue, please.
So in my study, I’m essentially measuring
people’s experience right in the moment.
They’re carrying around on their phone an app,
and I’m beeping them at random times.
I’m asking them, “How do you feel right now?”
And they’re responding on a scale
that ranges from very bad to very good,
and the scale is continuous.
So there are a couple of things
that distinguish that from the earlier paper.
One is that it’s right at the time
that people are feeling it,
as opposed to the day as a whole, retrospectively.
And the other is that gradations of feelings
are on a continuous scale,
as opposed to something that’s binary or dichotomous.
– And the scale is what to what?
It’s zero to 10, zero to 100.
– Very bad to very good, and it’s just continuous.
– So you can rake in anywhere you want.
– There are hundreds of unique values.
– Got it.
– And describe the differences
between the pools of research subjects
in the original case and in your case?
– In the original case, it was a survey by Gallup.
It was either representative or at least an attempt
to be representative of whatever random digit dialing was.
That’s a relative strength of their paper
and of that study in general.
My sample, in contrast, was really a convenient sample.
It’s turned out to be an amazing sample
of people that have really beautiful results,
but it was essentially whoever was willing to sign up
to try to understand their own happiness.
– What was the recruiting mechanism?
– Largely thanks to folks like you,
people in a hearing about it in the media,
on the radio, reading about it.
I think that’s interesting.
I’d like to learn about that for myself.
I’m willing to contribute to science.
And it turns out that when I look at,
what’s the distribution of incomes?
For example, in my data,
it’s almost a perfect match for the US census.
I can replicate all kinds of things
that we’ve seen in the literature for decades.
So it turns out to be really good,
but that’s kind of a lucky coincidence.
– Okay, so you gather, assemble, analyze your data,
talk about from the analysis point
to writing up the findings.
What I find when I look at this relationship,
plotting people’s happiness in the moment
versus their income, it just keeps going up.
This sort of critical point
where they had found in the earlier paper,
this flatlining, I really see no difference at all.
And when I write it up in the ultimate paper,
I compare, well, what’s the slope below the point
where they said it stops increasing?
And the point above that,
the slopes differ by less than 1%.
I really see no evidence for a difference at all.
Were you initially looking for that plateau in the data?
– I was really just trying to understand
what’s the relationship between these things
that are obviously important.
We’ve never had, in the moment, continuous data.
And so I want to see, what does this look like?
And it turns out, well, it doesn’t look like
what we thought it looked like.
– When you strike at a king, you must kill him.
That’s from Ralph Waldo Emerson.
Matt Killingsworth did strike
at the king of his realm, Danny Kahneman.
But what happened next?
It’s coming up after the break.
I’m Stephen Dovner, and this is Freakonomics Rating.
(upbeat music)
We’ve been hearing from Matt Killingsworth
about his research on the relationship
between income and happiness.
Research that disputed elements of earlier research
on the same topic by Daniel Kahneman.
– So you write up the paper,
and the paper, not quite directly,
but quasi-directly, addresses the fact
that your finding is contra
to a significant earlier finding by significant scholars.
What happens next?
– There’s a fair amount of attention about it and so forth.
Maybe a month or two afterwards,
I get a note from Barb that she’s been chatting with Danny.
– He’s talking about Barb Mellors,
a longtime friend of Danny Kahneman,
as well as a University of Pennsylvania colleague
of Matt Killingsworth.
– I think both of our attitudes
was we just wanna find out the truth.
I have no personal attachment
to what I found particularly,
and I don’t, other than his initial perhaps irritation,
I don’t really think Danny did.
It was really just what do we think is going on.
– Did you envision that perhaps you would then collaborate
with Danny on a joint study?
– That certainly seemed to be the case
once we got into it,
but these kinds of data took me many years
to collect that data for them.
There were a client on an external organization
with a lot of resources,
and so we ended up doing it by going back
and looking at my data, which existed,
which made it tractable.
I can sort of cut to the chase a little bit
of how we did that if you want.
– I wanna go back to Barb for just a second here.
Barb, if you could give Danny’s perspective
or participation up to the point here
where we’re about to cut to the data.
– The starting point for Danny was the assumption
that both data sets were valid.
So how could they be so different?
There must be a resolution in the data somewhere,
and so the task was going to be,
Matt became the research assistant
and started doing all the re-analyses on his data.
– How would you characterize the tenor
of this adversarial collaboration?
– Really civil and nice,
and just the way it should go.
– So Matt, you said you could cut to the chase.
We’re at the chase now, cut to it.
– Essentially, the resolution was
if we look at the range
within my continuous happiness data,
what if we look at the low end,
which is the part where their measure
would have been sensitive,
can we find a similar pattern?
And lo and behold, when you zero in on that,
rather than looking at the average trend,
which keeps going up,
you get not exactly, but very, very close
to what they found,
certainly much closer to what they found
than to what the average trend looks like.
We both found that pretty convincing evidence
that what they found is true,
there’s nothing wrong with the analysis at all,
but it was really a question of,
how generally applicable is that?
So from low to medium incomes,
the unhappiest part of the distribution rises a lot,
and then from medium to high incomes,
the unhappiest part of the distribution barely changes.
But the rest of the distribution is rising steadily,
and in fact, at the high end of the happiness distribution,
you get an inversion of that.
So rather than a rise and then a plateau,
you have sort of a shallow slope and then an acceleration.
– So in my lay mind, as I’m trying to process all this,
what I’m envisioning is that there is a cohort of people
who are kind of cranky,
and they may experience a little bit less crankiness
as income rises,
but there is a range in which their temperament
may be overwhelms their income.
– If anything, there’s one of the steepest slopes
for the unhappiest people in the range of low income.
So getting out of poverty, if you’re really miserable,
is at least correlationally.
– But that’s almost a different story,
’cause escaping poverty versus going from middle to upper.
– If you aren’t poor and you’re really miserable,
at least if you sort of extrapolate from there,
it seems to not matter past that.
And probably as we speculate in the paper,
at that point, if you have a decent amount of money
and you’re really unhappy,
whatever’s making you unhappy probably isn’t due
to the lack of resources.
It’s something else going on in your life.
Maybe it’s challenges in a family relationship,
or maybe you’re depressed, or whatever it is,
it’s something going on that perhaps money
isn’t going to make a difference.
– Richard, Taylor, let me ask you a question.
You’ve just heard Matt’s presentation of his side
of the original research,
and then the adversarial collaboration,
and then the conclusion,
all of which I found honestly to be really fascinating.
You know the literature well.
You too have a Nobel Prize,
although I have heard that was a clerical error.
(audience laughing)
How would you, looking at this from above,
Barbara was there as an arbiter,
Matt was one side,
Danny was another side, but is not present,
if you could give an ultimate proclamation on,
not just where this finding arrived,
but what the adversarial collaboration produced here,
what would you say?
– Look, I think this is the way it should work.
This is a good story,
and the world would be a better place,
academia would be a much better place
if there were more of these kinds of collaborations.
– Would it affect what we sometimes
call the replication crisis?
– Uh, oof.
– Barbara says, Barbara’s nodding her head, no,
you say oof.
– Yeah, let’s move on to Shane.
(audience laughing)
– The discomfort you were hearing there
about the replication crisis,
that discomfort is related to a two-part series
we recently produced on academic fraud,
episodes 572 and 573.
And while we’re on the topic of discomfort,
let me add one thing I’m thinking about
as we’re in the middle of this conversation
about adversarial collaborations.
Danny Kahneman is said to have introduced this concept
into the realm of behavioral research,
but if you’re not a behavioralist,
you could look at this as yet another example
of how academic researchers discover something
that people in the real world have been doing forever.
There are all sorts of arbiters and referees out there,
all sorts of processes for compromise and negotiation,
even war games to test one plan against another
and come up with the best solution.
In the same vein, some people look at the big insights
that have come out of this behavioralist research
as essentially ESOP’s fables with a little bit of math.
Earlier this year,
we made a series about the late physicist Richard Feynman.
He didn’t think the social sciences like psychology
and even economics should be called science at all.
He thought they were just too squishy to deserve that name.
And as for psychologists discovering the idea
of adversarial collaboration,
well, Feynman was a junior member of the Manhattan Project,
which in its quest to build an atomic bomb
brought together the most brilliant
and argumentative group of scientists ever assembled.
That was adversarial, and yes,
it was in the end a successful collaboration.
Anyway, back to Chicago.
As Richard Thaler suggests, let’s move on.
Shane Frederick, he is a behavioral scientist at Yale
who collaborated with Danny Kahneman on several projects
and is cited throughout Kahneman’s book,
Thinking Fast and Slow.
Kahneman saw Frederick as a protege
and they had a close relationship,
sometimes a bit too close.
– So I’m at 10 o’clock on a Monday,
maybe like third quarter of Monday football.
I get a call.
– Shane, you said, are you watching the game?
– It was one of the great times I wasn’t.
I said, no, fantastic.
I want to talk about heuristics.
(audience laughing)
Another time, it was like two o’clock in the morning,
going back and forth, back and forth, back and forth.
And thanks, I’m getting sleepy.
And so, go to bed and I wake up in the morning
in the very next email from Danny,
“Shane, don’t play dead on me.”
(audience laughing)
– One idea they worked on together
was the cognitive reflection test, or CRT.
It is meant to measure a person’s ability
to override their gut instinct
and think more carefully about a problem.
Here’s a famous example.
A bat and a ball cost $1.10 in total.
The bat costs $1 more than the ball.
How much does the ball cost?
If your first inclination is to say, 10 cents,
congratulations, you were like just about everyone else.
What the CRT measures is,
can you slow down and actually calculate the answer
rather than just go with your gut?
If you can, you will find that the answer is five cents.
Remember, the bat costs $1 more than the ball.
Shane Frederick has spent much of his career
designing such tests.
– Some of the items are novel, some are invented,
many of them I didn’t, I collect, some I’m revised.
One goes back to at least 1919, their variance of it.
– They were used then in an academic setting,
or like an employment?
– No, just like in books and riddles,
stuff that Maya would read.
(audience laughing)
– But they weren’t stumpers.
– No, they weren’t stumpers.
– I think I must tell the audience what a stumper is, okay?
– That again is Maya Bar-Huelal.
– A one-way ride costs $10,
a round trip costs $20,
a passenger hands the cashier $20,
saying absolutely nothing.
The cashier knows right away
that the passenger wants a round trip
rather than a one-way and change.
And the question is, how did the cashier know this?
– Does anyone in the audience who does not know the answer
to this stumper previously wanna take a guess to the answer?
– If you’re stumped good,
because that’s why they’re called stumpers.
– A stumper is unlike one of Shane Frederick’s CRT questions
in that your intuition doesn’t produce
an obvious but wrong answer.
– It only has to do with my love of riddles,
which I share with my partner in this research,
Shane Frederick.
And we really came to this out of love of riddles,
but we’re both professional psychologists.
So we felt like we had to approach it
from the point of view of psychology.
Now, Danny.
– Wait, wait.
– Oh.
– Answer please.
– Oh, no way.
No, I’m not gonna answer.
That’s not what I was paid for.
(audience laughs)
– All right, I will answer.
He handed the ticket agent two 10s.
– Ah, very nice.
– Did they pay you for that?
(audience laughs)
– I’m getting paid the same as you, Maya.
(audience laughs)
– But there was something more at stake here
than answering a riddle.
Barr Hillel, along with a co-author,
wrote a paper arguing that cognitive reflection tests
are not a good measurement of anything beyond math skills.
And this argument led to an adversarial collaboration.
– When I heard that Shane and Maya and Danny
were having an adversarial collaboration,
I didn’t know who was on which team.
Maya and Danny go back 60 years.
Shane is like his son for Danny.
– The results of this collaboration
have not yet been published,
but Shane Frederick says that he and Kahneman
essentially proved that the CRT is a good measure
of cognitive abilities beyond just math skills.
– It seemed to do quite well,
and it’s doing well consistently.
Time preferences, risk preferences,
other sorts of things.
– And how would you characterize the nature
of the collaboration?
How adversarial was it?
– It wasn’t so adversarial between the opposing groups.
It’s just like everybody’s fighting with everybody else,
but everything.
(audience laughs)
– More like a family gathering, right?
(audience laughs)
– Does this suggest that an adversarial collaboration
is not as useful as one might hope
if the actual adversaries are not often
in the ring with the collaborators?
– I mean, I don’t think it will work.
Do you, Barb?
– Do these things work?
Well, what does work mean?
It depends on your definition.
If you think people change their minds all the way,
no, it doesn’t work.
If they change their minds a bit, that’s good.
And it speeds up science too.
Somebody’s looking over your shoulder
and making sure you’re not making mistakes,
you’re defining variables more precisely,
you’re designing an experiment
that gets right at the core issue.
It’s the way to go.
(piano music)
– Well, I could listen to the five of you talk
for five hours.
There are sessions that need to happen.
This room is turning over.
I thank you so much.
This was a great conversation.
(audience applauds)
– Sorry, there is a clause in the contract
that Thaler must always have every last word, so.
– I was just gonna say something nice about you,
but if you insist, I’ll skip it.
(audience laughs)
– And that was, again, Richard Thaler.
I would like to thank him for putting this event together
and inviting us to join.
Thanks also to the other participants.
It was a pretty wonderful event
and I’m glad we were able to share it with you.
And if you need more Richard Thaler in your life,
and who doesn’t need more Thaler in their lives,
we are going to publish a bonus episode very soon,
an update of a great conversation
I had with Thaler a few years ago.
It’s called People Aren’t Dumb, The World is Hard.
So keep your ears out for that.
Meanwhile, on the next regularly scheduled episode
of Freakonomics Radio, a close-up look at an industry
that’s all about close-up looking.
I like for my glasses to have a bit of pizzazz,
especially if you’re wearing them every day.
– We’re gonna see half of the global population
being myopic by 2015.
– When you are a vertically integrated player,
you master every step within the value chain.
– The margins, even by luxury good standards, are obscene.
– Eyewear is a $150 billion industry
and what you see is not quite what you get.
That’s next time on the show.
Until then, take care of yourself,
and if you can, someone else too.
Freakonomics Radio is produced by Stitcher and Renbud Radio.
You can find our entire archive on any podcast app
also at Freakonomics.com,
where we publish transcripts and show notes.
This episode was produced by Zach Lipinski,
with live recording by Greg Rippen.
Special thanks to conference organizers Amy Boonstra,
Mark Tomelko, and Chris Partridge,
as well as the Black Oak AV team.
Our staff also includes Alina Cullman, Augusta Chapman,
Dalvin Abouaji, Eleanor Osborn,
Elsa Hernandez, Gabriel Roth, Jasmine Klinger,
Jeremy Johnston, Julie Canfer,
Lyric Bowditch, Morgan Levy, Neil Coruth,
Rebecca Lee Douglas, Sarah Lilly, and Teo Jacobs.
Our theme song is “Mr. Fortune”
by the Hitchhikers.
Our composer is Luis Guerra.
As always, thank you for listening.
– I think my mind was wandering.
I have that effect on people, I’ve been told.
(audience laughing)
(upbeat music)
– The Freakonomics Radio Network,
the hidden side of everything.
(upbeat music)
Stitcher.
(gentle music)
Daniel Kahneman left his mark on academia (and the real world) in countless ways. A group of his friends and colleagues recently gathered in Chicago to reflect on this legacy — and we were there, with microphones.
- SOURCES:
- Maya Bar-Hillel, professor emeritus of psychology at the Hebrew University of Jerusalem.
- Shane Frederick, professor of marketing at the Yale School of Management.
- Thomas Gilovich, professor of psychology at Cornell University.
- Matt Killingsworth, senior fellow at the Wharton School of the University of Pennsylvania.
- Barbara Mellers, professor of psychology at the University of Pennsylvania.
- Eldar Shafir, director of the Kahneman-Treisman Center for Behavioral Science & Public Policy at Princeton University.
- Richard Thaler, professor of behavioral science and economics at the University of Chicago.
- RESOURCES:
- “Experienced Well-Being Rises With Income, Even Above $75,000 Per Year,” by Matthew A. Killingsworth (PNAS, 2021).
- “The False Allure of Fast Lures,” by Yigal Attali and Maya Bar-Hillel (Judgment and Decision Making, 2020).
- “Learning Psychology From Riddles: The Case of Stumpers,” by Maya Bar-Hillel and Tom Noah (Judgment and Decision Making, 2018).
- Thinking, Fast and Slow, by Daniel Kahneman (2011).
- “High Income Improves Evaluation of Life but Not Emotional Well-Being,” by Daniel Kahneman and Angus Deaton (PNAS, 2010).
- “Varieties of Regret: A Debate and Partial Resolution,” by Thomas Gilovich, Victoria Husted Medvec, and Daniel Kahneman (Psychological Review, 1998).
- “Some Counterfactual Determinants of Satisfaction and Regret,” by Thomas Gilovich and Victoria Husted Medvec (What Might Have Been: The Social Psychology of Counterfactual Thinking, 1995).
- EXTRAS:
- “Remembering Daniel Kahneman,” by People I (Mostly) Admire (2024).
- “Academic Fraud,” series by Freakonomics Radio (2021).
- “Here’s Why All Your Projects Are Always Late — and What to Do About It,” by Freakonomics Radio (2018).
- “The Men Who Started a Thinking Revolution,” by Freakonomics Radio (2017).
