Perhaps I should say most of us are not really skeptics.
The Black Swan by Nassim Nicholas Taleb is such a well-written book. One of his subjects is the confirmation bias.
Consider a scientific theory. Most people coming to this blog are probably interested in science in some shape or form and know that for a theory to be scientific it has to be falsifiable – that is, we have to be able to create some tests in advance and say “if it fails these tests then the theory is not true”.
For example, the pink fairy down the bottom of my garden. I say it exists. You say, “Why can’t I see it?”
I say, “It’s invisible.” You ask, “How do I know it’s there?”
And so, in the end I have to provide some kind of evidence that can be tested. Otherwise it’s not a scientific theory.
Taleb gives a great example of what we all really do in practice, from research by pyschologists. Pay attention to Nicholas..
Subjects were presented with the three number sequence 2, 4, 6 and asked to guess the rule generating it. Their method of guessing was to produce other three-number sequences to which the experimenter would say “yes” or “no” depending on whether the new sequences were consistent with the rule.
What did the subjects do? They tried to guess the rule.. of course! That’s what they should have done.
And then they tested it by.. producing a sequence consistent with their theory. So almost no one worked out that the real rule was simply, “numbers in ascending order”
Perhaps they decided that the rule was a starting number x1, x2, x3. Or perhaps they decided that the rule was to take starting number then add 2, and add 2 again.
And they generated a sample sequence from their theory and told the experimenter.
But what almost no one did was to suggest a sequence inconsistent with their own mental theory – a test which would allow them to more easily falsify their theory.
The scientific method is to find a way to falsify a theory, but unconsciously almost all of us just try to corroborate our own theories.
Don’t look at the people around you.. ask yourself,
Do I try and test my theories by falsifying them?
Do I try and understand what my “opponents” say?
Do I spend time at blogs where I feel uncomfortable with their “false and unwarranted” conclusions of the world? And try and understand why they think what they do?
Become a real skeptic. Try and prove yourself wrong!
The Science of Doom 
In my scientific experience, you generate your hypothesis, you do a quick screen by looking for confirmation, and then you go on to the big test – the attempt to falsify. If you don’t do the last, or at least propose how it might be done, you’re not really doing science. Mind you, in “climate science” you can’t do controlled experiments anyway, and observations seem to be riddled with difficulties, incompetence and dishonesty,so the whole thing is unlikely ever to offer knowledge as secure as is common in other fields.
I’m a scientist that used to believe in AGW. I did what you said. I no longer believe it.
“But what almost no one did was to suggest a sequence inconsistent with their own mental theory – a test which would allow them to more easily falsify their theory.”
Not necessarily. This is an artifact of the nature of the answer, which in this example is contrary to every similar problem the students have been presented with — from SATs to elementary school New Math: “What is the next number in sequence?” — and the answer has been chosen in such a way that that particular question is meaningless.
In this cherry-picked example, every ascending sequence is allowed. It’s not so much a problem of experimental design as it is of Kuhn’s paradigm, and the opening example is purposely misleading, rather like asking “what do these objects have in common”, showing an automobile tire and a basketball, and expecting the answer, “they are both physical objects” — which is both obvious and uninteresting.
Blog author: cut it out with that “two dot” thing you are doing. Try commas, semicolons, colons, dashes, or ellipsi. But there is no such thing as.. that.
Heh, I like how you pluralized ellipsis.
[…] long time ago I wrote The Confirmation Bias – Or Why None of Us are Really Skeptics, with a small insight from Nassim Taleb. Right now I’m rereading The Righteous Mind: Why Good […]
In software testing its said that in mature organizations there are 5 clean tests (satisfy basic assumptions) for every dirty test (bad data, missing values etc). But with mature organizations its inverted….5 dirty for every clean. I guess the obvious question is how you bridge the gap.