When Size Matters

Jun 18, 2015 | Books, Thinking Better

My reading journey through Thinking, Fast and Slow, Part 2

Over the next year and an half, the North American news media will be inundated with polls. Who leads in what polls and what margin does one have over another. There is nothing wrong with polls, except when we believe strongly in their results without questioning their assumptions.

As the recipients of numeric information, especially when presented as percentages, we should all be aware of the quality of the number. Then, our trust in what the number says should be adjusted accordingly.

7 out of 10 is 70%.

70 out of 100 is also 70%.

And so is 700 out of 1000.

They are all equal to 70%, but we know that the quality of the 70% differs. We know (or should know) that larger sample sizes yield more trustworthy results. This is the more intuitive principle from chapter 10, The Law of Small Numbers. The second, however, is less intuitive, although completely equivalent to the first.

***

The best schools are small.

That is an example Kahneman uses in chapter 10. He then summarizes a research finding on the characteristics of the most successful schools. The study concludes that the most successful schools are small in size, which then motivates the Gates Foundation to make large investments to create small schools. Other organizations and government efforts also follow in the effort.

There are many reasons we can postulate on why small schools are more successful. Most of us would find this storyline perfectly acceptable or even subscribe strongly to the conclusion.

Unfortunately, as Kahneman suggests, this analysis is wrong. If the question were asked, what are the characteristics of the worst schools, they would have found that these schools also tend to be small. And as he mentions in another place, a cause that can explain 2 opposite outcomes means nothing at all.

The reason why small schools can produce two opposite and extreme results is because they have small populations. This smallness in numbers yields more variations, i.e., they swing more. You would most likely get a figure like 98% or 1.7% from small samples, not because they are the best or the worst, but because they are small. It is simply a feature of statistics. Any phenomenon that is very high or very low, best or worst, will most likely come from small sample sizes.

***

“But nooo, there must be a reason why small schools perform better.”

Anyone has that small voice speaking in his or her head? I do. There is something inside of me that needs an explanation of this phenomenon.

Kahneman says this is System 1 talking, which likes to find causal relationships to make a coherent belief system, even though some things are mere randomness. These random things can be marked as causal relationships, which lead to a belief that is in fact, completely spurious.

***

The subject of this chapter is not easy to explain and this post is definitely far from sufficient. It takes time to internalize the principles it describes, but its importance is paramount, if you care about the truthfulness of what you think is true.

Is this cause-and-effect that I believe true, or am I assigning causal relationships to something that is, in fact, random?

***

• Always consider the sample size of any statistical study.
• When someone claims a causal relationship, ask whether the same cause can give alternate/opposing/inconsistent outcomes. Then adjust the trustworthiness of that information accordingly.

Don’t Jump!

Jun 14, 2015 | Books, Thinking Better

My reading journey through Thinking, Fast and Slow, Part 1

I’m thoroughly enjoying my journey through Thinking, Fast and Slow by Daniel Kahneman. This book is a must-read for anyone who cares whether the mechanism by which they think, process information, and synthesize conclusions is valid or not.

Thinking, Fast and Slow is the kind of reading material that sparks fireworks in the readers’ brains. I could not reserve my thoughts only at the end of the book, so this is a first of what I expect will be a series (at least two) of reflections on the chapters.

Among its fascinating chapters—concise chapters I should add (yay)—one of my favorites so far is chapter 7. In this chapter titled “A Machine for Jumping to Conclusions”, Kahneman describes our brain’s tendencies to ignore ambiguities or suppress doubts in the information we are presented with.

One revealing example is the statement “Ann approached the bank.”

Most likely, the image prompted by the statement is a lady with some money business walking toward an edifice we call “banks.” For me, the moment I saw the statement, my mind did not conjure any other image—it was so plain.

Yet, if it was revealed that the statement preceding “Ann approached the bank” was “They were floating gently down the river”, the image would have to change drastically.

What just happened? It turns out that our mind has a mechanism, dubbed System 1 in the book, that likes to jump into conclusion in the absence of a clear context (System 2, on the other hand, is the mechanism responsible for deliberate thinking, logic, and reasoning). When there’s limited information, System 1 likes to generate its own context and is unaware of the ambiguity of the given information. Unless someone had just gone canoeing or read about rivers recently, it was unlikely that that he/she would have automatically applied the river-related meaning to the word ‘bank.’

This is so fascinating to me because it shows how interpretation can reveal more about the interpreter than the facts themselves. I automatically equated the money-related institution to the word ‘bank’ because, well, I’m a 21st century, urban woman who spends more times seeing Chase or Bank of America than canoeing in rivers. Thus my mind did not even search for other possible meaning and resolved the statement accordingly.

But the statement itself provided little information about what it actually meant.

I would admit that when the second statement was given, it disoriented me quite a bit—what was it talking about? I was that unaware of the alternate meaning.

To push it even further, if one insists that the word ‘bank’ means money-bank, he/she would have to get more creative in resolving the coherence between “They were floating gently down the river” and “Ann approached the bank.” Some things need to happen in the middle of the two sentences, and the interpreter would have to insert further speculations when no further information is provided. This may be correct, but without more information, it’s hard to say. It would just be another jump to another conclusion.

What I take from this chapter is that slowing down and suspending judgment is a good idea, especially for high-stake matters. If you don’t slow down, you may not perceive ambiguities and miss the true meaning altogether!