Hindsight Bias: When Experience Works Against You

Hindsight Bias: When Experience Works Against You

Learning from the past—it’s the reasonable thing to do to improve and make progress. It turns out though, that its merit may well depend on what exactly we learn, since “looking back” is not without its downfall. Cue in hindsight bias.

 

What is Hindsight Bias?

 

Hindsight bias is the belief that past events were more predictable than they were before they happened. Richard E. Nisbett in The Geography of Thought: How Asians and Westerners Think Differently… and Why, writes:

 

“We tend to have two problems when we try to ‘predict’ the past: (1) believing that, at least in retrospect, it can be seen that events could not have turned out other than they did; and (2) even thinking that in fact one easily could have predicted [in advance] that events would have turned out as they did.”

 

The first pertains to the belief of the “inevitability” of certain events, which influences the mind to think of the cause and effects to describe this inevitability. Then, convinced of this causal relationship, the second follows. Applying the causal relationship, one should have known that the event would certainly happen. The problem with this is, it’s a fallacy. Nisbett continues to describe that certain cultural paradigms have greater tendencies to commit this fallacy, but more on that later in future posts.

 

Hindsight bias is what leads to phrases like “I knew it all along” or “I knew from before that this would happen.” Ironically, the people who commit this fallacy may sound like experts, gifted with certain prescience of events, and may become the go-to sources for future predictions. You’ve seen them on TV—people get over-credited for predictive knowledge because they say so-and-so should have known and done something so this other thing would have happened. The key illusion here is the belief that they understand the past–they have a model to explain the past–which they use to predict the future, while in fact they understand the past much less than they think they do.

 

The Mind That Cannot Revert Itself

 

In Thinking, Fast and Slow, Daniel Kahneman explains some of the mechanism that causes hindsight bias:

 

“A general limitation of the human mind is its imperfect ability to reconstruct past states of knowledge, or beliefs that have changed. Once you adopt a new view of the world (or of any part of it), you immediately lost much of your ability to recall what you used to believe before your mind changed.”

 

He continues to describe a psychology experiment to illustrate this point:

 

“Choosing a topic on which minds are not completely made up–say, the death penalty–the experimenter carefully measures people’s attitudes. Next, the participants see or hear a persuasive pro or con message. Then the experimenter measures people’s attitudes again; they usually are closer to the persuasive message they were exposed to. Finally, the participants report the opinion they held beforehand. This task turns out to be surprisingly difficult. Asked to reconstruct their former beliefs, people retrieve their current ones instead–an instance of substitution–and many cannot believe that they ever felt differently.” 

 

Because the mind has a hard time reproducing its less knowledgeable state, over-simplification of the past or overconfidence happens.

 

“Your inability to reconstruct past beliefs will inevitably cause you to underestimate the extent to which you were surprised by past events.”

 

The less knowledgeable me saw things as complicated. But once I know, I couldn’t believe that a previous me ever thought things as complicated. It seems like there’s a limit to sympathy with my old self.

 

Asymmetric Perception

 

Our relationship with the past and future is very interesting. We know that the future is uncertain. I, right now, have many choices that could result in a plethora of outcomes. But I can only make one choice, and thus as I traverse through time, I will only have one version of history. Looking forward, I see a spectrum of possibilities, but looking back, I see only one. The fallacy arises if I believe that because I only see one history in my past, it is the only one version that could have happened while in fact, if I return to that point, just like my presence, I would have multiple choices as well. Other outcomes were just as probable.

 

This asymmetry of our perceptions with respect to time is a warning when we learn from the past since “the tendency to revise the history of one’s beliefs in light of what actually happened produces a robust cognitive illusion.” The lessons learned, or our model of what happened in the past, may be influenced by this bias, which may produce a story that’s too clean and tidy compared to reality.

 

Hindsight Bias image 1

Event A and B at two different times

Hindsight Bias image 2

Perception at time A. A range of possibilities of outcomes, of which B is an instance.

Hindsight bias

Perception at time B, the belief that A to B is inevitable.

 

 

When Experience Works Against You

 

Our view of the past influences our perception of the future. If our model of the past is too straightforward, we may be tempted to extrapolate the same behavior in the future and oversimplify, which can be serious if you were a decision maker. The failure to consider various possibilities of outcomes can lead to incorrect estimation of risks and failure to mitigate. But even if you weren’t a decision maker, the loss from hindsight bias, to me, is its opportunity cost. The disbelief that the future could not turn out another way may prevent you from exploring creative options.

 

In many areas of life, experience is highly valued in considering someone’s suitability for a job, rightly so. Experience can also be described as the quantity of past events in someone’s repertoire. It’s how much history he has in a particular area. It does not always mean that the person is older, although in most cases it correlates strongly with age.

 

For a job that requires expertise, experience is a definitely an advantage. Sometimes, though, a task requires creativity, innovation, new ways of looking at and doing things. For this type of work, experience may mean that the person’s mental model may be too rigid and solidified to consider alternatives.

 

I’ve been on both sides of this fallacy, both as the guilty party and the victim of hindsight bias. As the guilty one, I’ve said things like, “I knew all along this would work or that wouldn’t work out” and giving suggestions like “I’ve tried that, it didn’t work so don’t bother trying.” The root issue is believing that the cause and effect are clear, discounting other factors that also play big roles. Maybe sometimes it helps the recipients of this ‘advice’, but what if it’s actually misdirection?

 

As a victim of hindsight bias, I’ve been on the other side of experts and been told not to hope too much from a certain pursuit, or that my idea won’t work because it’s always done in certain ways. But because I don’t have much pre-knowledge, I’m less constrained to think in their ways and more motivated to pursue alternate ideas. When it comes to thinking divergently, which I believe is necessary in making progress that’s a leap rather than incremental, there’s an advantage to being young and inexperienced.

 

The Impossibles

 

Is there an escape from hindsight bias? I would think so. A conscientious approach to detect hindsight bias in ourselves may fix it and free us from limiting/false models from the past, and acknowledge the statistical nature of our world. But if Kahneman’s right, that our brain cannot fully sympathize with its less knowledgeable version in the past, maybe we cannot overcome hindsight bias 100% internally.

 

Which is why it’s great that there are other people, and sometimes a new person for in a position or as a leader is the best way to bring in new approaches, new visions, and new solutions. There’s a quote a movie that I love, “We are too young to realize that certain things are impossible…so we will do them anyway.” If we can be the one saying this, then great! But if it so happens that we can’t, then maybe we should at least have someone else in the team who can still say those words, and counteract our hindsight bias.


More on cognitive biases:

Don’t Jump!

When Size Matters

OK, But Not OK. Not OK, But OK

 

Image credit: Uxmas.com

Rock of Ages: Structures That Last

Rock of Ages: Structures That Last

Giza. Chichen Itza. Magelang. Three locations where ancient structures stand for thousands of years. The pyramids of Giza have existed for over 4000 years, built out of quarried stones, stacked and shaped into place. El Castillo, the largest pyramid in the Chichen Itza complex in the Yucatan state of Mexico, is made out of rocks forming a 4-sided structure with stairs on each side. There are sculptures of serpents on the sides of the stairs on the northern side. Candi Borobudur, or the Borobudur Temple, in Magelang, Central Java, Indonesia, is an impressive structure of stone carvings of Buddha facing all corners of the earth. The walls are relief panels with carvings that narrate Buddhist cosmology as well as daily living in 9th century Javanese society.

Borobudur Temple

Candi Borobudur, or Borobudur Temple, in Central Java, Indonesia

 

Each of these structures reflects the worldviews of the civilizations that created them. Both their function and aesthetics are filled with meaning, and somehow this deep meaning is communicated to anyone visiting its perimeters. When you come close to these structures, they generate a reaction in the soul.

Pyramid and sphinx

Pyramid and sphinx of Giza, Egypt. Credit: Freeimages

 

In Structures: Or Why Things Don’t Fall Down, reportedly one of the influential books that Elon Musk read when he first thought about building SpaceX, structural engineer J. E. Gordon reflects on structures and aesthetics in his first chapter:

 

“…although most artefacts are not primarily concerned with making an emotional or aesthetic effect, it is highly important to realize that there can be no such thing as an emotionally neutral statement. This is true whether the medium be speech or writing or painting or technological design. Whether we mean it or not, every single thing we design and make will have some kind of subjective impact, for good or bad, over and above its overtly rational purpose.”

 

Gordon contrasts the artefacts of the eighteenth century, which “seem to many of us to be at least pleasing and sometimes incomparably beautiful,” to the works of modern man—modern is a relative term–which is not so much filled with “active ugliness” as the “prevalence of the dull and the commonplace.”

 

“…man does not live by safety and efficiency alone, and we have to face the fact that, visually, the world is becoming an increasingly depressing place… Far too seldom is the heart rejoiced or does one feel any better or happier for looking at the works of modern man.”

 

It seems to me that the grandeur of exceptional structures lies in the spiritual aspect of the work, which is often the source of excellence that makes them out of the ordinary.

 

Ancient Engineering

 

On why these ancient structures last, I am toying with conjectures here. I’d defer to the civil engineers and architects to supply the scientific explanation.

 

Pyramids and temples were built by their creators to last. Whether it was a tribute to kings, ancestors, or gods, these structures reflected their beliefs about life and the afterlife. This to them was a perpetual reality that would never change. Why would it, if this was how the world worked? The bodies of past Pharaohs were not placed there temporarily, nor Buddhas arbitrarily placed in random directions. They were designed with intention of permanence.

 

If these structures were meant to last, they needed to be built in a certain way, much like the difference in attitude and approach when I know I’m making something for temporary use or for ever. Intent drives content, which in turn informs method.

 

The pyramid-like shape of these 3 structures to me seems very stable, although I would here open myself to counter examples that show other equally stable shapes. Admittedly, these 3 examples are simply based on my personal exposures, a lecture I heard recently on Egyptology, my recent visit to Chichen Itza, and my visit and pride in a notable accomplishment of my home country’s past civilizations.

 

Gordon also writes that pre-metallic structures of the past probably ‘force’ their engineers to think deeply about loads and stresses on the materials at their disposal:

 

“To make structures without the benefit of metals requires an instinct for the distribution and direction of stresses which is by no means always possessed by modern engineers; for the use of metals, which are so conveniently tough and uniform, has taken some of the intuition and also some of the thinking out of engineering.” 

 

Certainly there were numerous ancient structures that didn’t last, whether to due structural or environmental causes. Earthquakes, floods, and everything that could happen over the span of 4000 years might destroy even the most superior knowledge of stress distribution on solids. This, though, only makes our respect and admiration grow for the few structures that have survived history and the “primitive technologies” that created them.

 

Of Matter and Structure

 

There is also, however, something to be said about the materials used in these trans-millennial structures. They were made out of stones, probably because stones were the most readily available materials with load-bearing strength before the discovery, manipulation, and production of metals. Is it also possible that this choice material contributes to their survival?

 

Since metallic structures are relatively younger, it is quite unfair to compare stony and metallic structures on equal grounds. We don’t know (or, rather, I don’t know) whether metals will outlast stones, as in, if two pyramids were built, one with stone and one with metals in the same location, which one would last longer? We only have one version of history, so all we can do is credit the ancient glories to the stone structures.

 

As building materials, stones have notable advantages. They are by nature durable, fireproof, and nonreactive. They are not the easiest to work with due to their weight and non-uniformity, and they don’t provide good insulation (not good for places with extremely hot or cold weather). But their resistance to fire and moisture is a clear strength, unlike wood that is in fact fuel for fire and prone to decay, or metals that bend and melt due to heat. Stones will not rust, since the molecules are mostly oxidized and thus will not react with air or water. Further, it can be restored. Many of the ancient stone structures were once lost and re-discovered after many years of abandonment. But careful restoration could bring their old glories back, provided that enterprising people didn’t chip them away, and moss could be removed from stones. Compare this with common houses in North America, which, HGTV tells me, can undergo so much decay and destruction from being abandoned for a few years.

 

The Character of Stones

 

In the Biblical account, there is a section that flies over human history by giving each era a certain character through material symbols. In the book of Daniel chapter 2, the Babylonian king Nebuchadnezzar had a dream which he forgot, and decreed his counselors to reproduce the dream and interpretation. Failure to do so would result in death. Daniel, a Hebrew prophet in the king’s court prayed to God and was granted the vision and its explanation.

 

The vision was of a statue with a head of gold, breast and arms of silver, belly of brass, legs of iron, and feet of iron and clay. Each section symbolized a kingdom, gold for Babylon, silver for Media and Persia, bronze for Greece, iron for Rome, and iron and clay for divided Rome.

 

At the end of the vision was a part that was a bit strange and discontinuous. A stone “cut out without hands” appeared, stroke the feet of the statue, and crumbled the whole structure down. This stone represented the kingdom of God, hence the discontinuity from the materials in the statue–metals refined by human civilizations–was fitting here.

 

Yet the choice of a stone to represent the kingdom of God was rather anticlimactic, wouldn’t you say? I would probably choose something majestic, like a big giant diamond to crush the entire human civilization. A plain old stone is neither flashy nor valuable.

 

Interestingly, however, the commentary continues to highlight that this last kingdom’s distinct characteristic was that it would last forever. Daniel 2:44 says, “And in the days of these kings shall the God of heaven set up a kingdom, which shall never be destroyed: and the kingdom shall not be left to other people, but it shall break in pieces and consume all these kingdoms, and it shall stand for ever.” The other kingdoms pass away, but this one stays.

 

Perhaps there is something here, given what we know about the material characteristics of rocks and stones. In other places, Biblical references to rocks mean stability, strength and assurance (e.g., building on the rock, rock of salvation, etc). God Himself is referred to as the Rock of Israel.

 

It is not uncommon for the Bible to illustrate the kingdom of God with unusual symbols; a seed, leaven, a net cast into the sea, and numerous other unexpected metaphors. They are often humble, simple, and commonplace. Yet each time, they reveal a secret strength that is not always intuitive to human thinking, like the seed that grows into a tree, where birds can lodge in its branches. The baffling-ness calls its audience to think of reality as God sees it, that not everything is valued according to the marketplace and human commerce.

 

In the choice of a stone in Daniel 2, its durability, strength, and stability certainly reflects a characteristic of God’s kingdom. That sameness yesterday, today, and tomorrow is a key aspect of the Biblical God. I don’t know whether the author of the hymn “Rock of Ages” was thinking along this line, but that phrase is awfully and appropriately fitting.

 

Photo credit: Johnny Loi Photography

Two New Books That Celebrate Women and Science

Two New Books That Celebrate Women and Science

Two new books are released today that celebrate women and science, two subjects that are great on their own, but super fascinating when combined. I’ve been really looking forward to these books! They follow in line with a rapid surge of science writings that have been adding to my library. See if you could share my excitement below.

1. Lab Girl by Hope Jahren

 

Lab Girl by Hope Jahren

 

Lab Girl intrigues me from the get-go. I always love stories of people in academia, those pursuing their passion and interests at full throttle, especially in the sciences. Women in academia is another fascinating layer. From my time in graduate school, I’ve come to see that the relationship between an academic and her subject of study is like a love story–a deep one. Jahren is a geobiologist at the University of Hawaii who studies the world of plants and, it seems, is completely in love with it. Sounds like a very promising read.

 

Excerpt:

 

Rise of the Rocket Girls by Nathalia Holt

 

Rise of the Rocket Girls strikes me, given my recent visit to the Kennedy Space Center and all nerdy things I’ve been consuming. Space travel alone–what humans achieve through ingenuity, science, engineering, and perseverance–is impressive enough. But few of these stories are told in the female perspective, hence the thrill of seeing this book. It’s so refreshing to learn about these women who were just as geeked about going out to space as the men, who were brilliant, and just as dedicated to the ones on the spotlight. History lovers would enjoy this book too.

 

Excerpt:

 

The riveting true story of the women who launched America into space.

 

In the 1940s and 50s, when the newly minted Jet Propulsion Laboratory needed quick-thinking mathematicians to calculate velocities and plot trajectories, they didn’t turn to male graduates. Rather, they recruited an elite group of young women who, with only pencil, paper, and mathematical prowess, transformed rocket design, helped bring about the first American satellites, and made the exploration of the solar system possible.

 

For the first time, Rise of the Rocket Girls tells the stories of these women–known as “human computers”–who broke the boundaries of both gender and science. Based on extensive research and interviews with all the living members of the team, Rise of the Rocket Girls offers a unique perspective on the role of women in science: both where we’ve been, and the far reaches of space to which we’re heading.

 

I am geeked beyond belief! Can you blame me?

 

Will you read these books?