The Confirmation Bias: Seeing What We Expect to See

Anecdotes, the topic of the post at this link, are often used as evidence for claims, even though they have weaknesses that limit severely their value as evidence. Anecdotes are important for another topic we’re discussing in my PSY 101 course: the confirmation bias, which is a strong tendency to readily accept evidence that seems to support beliefs we already have (i.e., our preconceptions) and to examine closely evidence that seems to contradict our preconceptions in order to find problems with the disconfirming evidence that allow us to discount it.

Figure 1. Discounting evidence that disconfirms one's preconceptions can lead to the development of bizarre beliefs that people hold fervently.
Figure 1. Discounting evidence that disconfirms one’s preconceptions can lead to the development of bizarre beliefs that people hold fervently.

In everyday life, however, we often do not even become aware of disconfirming evidence. Sometimes, this is because we can easily avoid it (e.g., we might avoid watching certain news programs that we know will make claims that contradict our beliefs). Other times, our cognitive limitations make it difficult for us to notice the disconfirming evidence (i.e., it doesn’t reach the conscious level). One example of the latter involves the belief that washing our cars causes it to rain. Many of us are able to point to occasions (anecdotes) on which it rained after we washed their cars, which seems to be compelling evidence of the truth of this belief. We often fail, however, to notice (and, therefore, don’t remember) occasions on which these events didn’t occur together. In order to get better evidence for the accuracy of the belief, we would need to make the kinds of observations indicated in the following table.

Table 1. Each cell of this table indicates the observations that would need to be made to determine if washing one's car causes it to rain.
Table 1. Each cell of this table indicates the observations that would need to be made to determine if washing one’s car causes it to rain.

In making observations that would allow us to fill in the cells of the table, we force ourselves to pay attention not only to evidence that supports our belief, but also to evidence that might disconfirm that belief:

  • The cell labelled A shows the number of times we washed the car and it rained.
  • The cell labelled B shows the number of times we washed the car and it didn’t rain.
  • The cell labelled C shows the number of times we didn’t wash the car and it rained.
  • The cell labelled D shows the number of times we didn’t wash the car and it didn’t rain.

Let’s say that we make the relevant observations for one year and get the following results.

Table 2. The cells of the table show, for a one-year period, the proportion (percentage) of days we either washed or did not wash the car and it either rained or did not rain.
Table 2. The cells of the table show, for a one-year period, the proportion (percentage) of days we either washed or did not wash the car and it either rained or did not rain.

Was it more likely to rain on the days we washed the car?

  • Cell A shows that, on 20% of the days, we washed the car and  it rained.
  • Cell B shows that, on 80% of the days, we washed the car and it didn’t rain.
  • Cell C shows that, on 20% of the days, we didn’t wash the car and  it rained.
  • Cell D shows that, on 80% of the days, we didn’t wash the car and it didn’t rain.

Thus, regardless of whether we had just washed our car or not, it rained on 20% of the days that year (and it didn’t rain on 80% of the days). In other words, washing our car was not associated with whether or not it rained.

The confirmation bias is caused, in part, by our unconscious tendency to ignore, avoid, or distort information that would show a preconception to be wrong. In the present example, people tend to pay attention only to the first cell of the table and to ignore the rest. This is because, in general, we are much more likely to notice when something happens than when something doesn’t happen. By forcing ourselves to pay attention to all relevant information in such situations, we are more likely to realize when our preconceptions are inaccurate.


To summarize: Cognitive researchers have found that we have an automatic (unconscious) tendency to seek out and readily accept information that agrees with (confirms) our preconceptions, and to ignore, distort, or discount information that contradicts (disconfirms) them. This confirmation bias serves to maintain and strengthen our preconceptions: we are much more likely to perceive and remember experiences that confirm our prior beliefs, and to discount or reinterpret those that disconfirm them. Thus, over time, the confirmation bias results these beliefs becoming so well established in our minds that eventually we consider them to be common sense (i.e., obviously true). If we wish to minimize the effects of the confirmation bias, we must force ourselves to look for and examine closely both confirming and disconfirming evidence.


Brief Test of Critical Thinking

The development of critical-thinking skills is a fundamental goal of educators, who have tried to devise various ways of assessing critical thinking in their students. Below is one of my attempts .

A cat using its advanced critical thinking skills
A cat using its advanced critical thinking skills

Brief Test of Critical Thinking

1. When your gasoline guage is past Empty, and you are five miles from the nearest gas station, your best option is to:

a. press the accelerator to the floor so that you can get to the gas station before you run out of gas.
b. let some air out of the front tires, put the car in neutral, and coast to the gas station.
c. place a piece of duct tape over the guage so that the engine won’t know that there’s no more gas.

2. You’ve just ridden your bike two miles to a liquor store and are now at the drive-thru window buying a gallon jug of снаряда рвоту vodka. After the clerk hands you the jug, you discover that you don’t have the $2.50 it will take to buy it. It would be best to:

a. break out your beer bong, funnel the vodka, and then pedal as fast as possible until you pass out.
b. settle for the two-liter Bag-O-Bourbon for 99¢ (plus tax), which is on sale until Sunday.
c. a and/or b

3. It’s ten minutes before your college-algebra final is scheduled to begin. You haven’t been to class in weeks, flunked the other tests because you couldn’t remember what X stood for, and have never been able to count past G. Which of the following should you do?

a. Show the teacher your grandmother’s obituary from last year and tell him that you’re really late for the wake.
b. Take the final anyways. Who knows? Maybe you’ll ‘ace’ it!! Hey, anything’s possible, right? Ya’ just gotta think positive!!
c. Tell the teacher that he never returned your last two tests but that you know that you “did really good on ’em” …
d. And that he also incorrectly entered 9% for your score on the first test instead of the 99% that you really got …

4. Critical thinking is:

a. good.
b. bad.
c. crunchy.

5. If you’re making crank phone calls, would it make sense to cover the caller ID so that the person answering the phone doesn’t know who you are?

a. yes
b. only on the weekends
c. Did I already include “yes”? I did? OK, thanks.
d. no


Add up the number of items you either answered or thought about answering or would have answered if you could read, add the number of pints in a meter, multiply by Planck’s Constant (no, not that Planck, the other one), and divide by your age when you were 10.

Did you get a number? No? Good!

This baby received a perfect score.
This baby received a perfect score.

Anecdotes & Testimonials: Good Evidence for Claims?

In my PSY 101 class, we recently discussed some problems with using anecdotes and testimonials as evidence for claims. In this post, I want to begin to explore this issue in more depth. In future posts, I will discuss further some of the issues touched upon here.


One afternoon, Eileen Lipsker was sitting in her family room watching Jessica, her red-headed five-year-old daughter, play with her friends. Eileen later reported that she felt “spaced out” and was “thinking of nothing.” Lenore Terr (1994), a psychiatrist who spoke with her on many occasions, described what Eileen said happened next:

Jessica twisted her head to look at her mother. To ask something? Her chin pointed up in inquiry. She looked up and over her shoulder. Her eyes brightened. How odd! The young girl’s body remained stationary, while her head pivoted around and up…. And at exactly that moment Eileen Lipsker remembered something. She remembered it as a picture. She could see her redheaded friend Susan Nason looking up, twisting her head, and trying to catch her eye.

Eileen, eight years old, stood outdoors, on a spot a little above the place where her best friend was sitting. It was 1969, twenty years earlier. The sun was beaming directly into Susan’s eyes. And Eileen could see that Susan was afraid…. [Eileen] looked away from those arresting eyes and saw the silhouette of her father. Both of George Franklin’s hands were raised above his head. He was gripping a rock. He steadied himself to bring it down. His target was Susan. (pp. 2-3)

Eileen told Terr that this is how she first recovered her repressed memory of Susan Nason’s murder by her father. Eileen’s recounting of the memory recovery is an example of an anecdote: a brief story told by an individual about a personal experience. No matter how interesting or compelling an anecdote may be, it doesn’t provide good evidence for a claim because it is based on interpretations and memories of personal experiences. In other words, an anecdote is inadequate evidence for a claim because it does not control for factors that affect how a personal experience is (a) initially perceived and interpreted, and (b) eventually remembered.

Figure 1. An example of an anecdote used to support the claim that extraterrestrials visit earth
Figure 1. An example of an anecdote used to support the claim that extraterrestrials visit earth


Autistic Disorder is a severe mental disorder that develops in children before the age of three years. It has three main symptoms: a severe impairment in social interaction, a severe impairment in the ability to communicate, and a severely restricted range of interests, activities, and behaviors. On occasion, new treatments for autism are announced that seem to offer hope for either a cure or, at least, a dramatic reduction of symptoms. One such well-publicized treatment used injections of secretin–a hormone that assists in the digestion of food. Some have claimed that secretin improves the social and language skills of autistic individuals by affecting specific behaviors such as the amount of eye contact made, the level of awareness of one’s surroundings, the degree of sociability, and the amount of speech. One proponent of secretin therapy provided the following evidence for this claim:

The good news is that confirmatory evidence of the power of secretin keeps coming. A national newspaper told of Florida pediatrician Jeff Bradstreet’s own four-year-old son, Matthew, shocking his parents by holding his first normal conversation with them the day after his first secretin infusion. And Virginia pediatrician Lawrence Leichtman told me of his “miracle case”: a five-year-old who had previously said only two words amazed all in the office by saying, 15 minutes after his infusion, “I am hungry. I want to eat.” Most cases are much less dramatic, but the autism world is excited, and for good reason. (Rimland, 1998, p. 3)

Is this good evidence for the effectiveness of secretin in the treatment of autism? The evidence consists of two testimonials. A testimonial is an anecdote that describes the supposed merits of a product or service. Testimonials are not good evidence for a claim because they are anecdotes and, as stated above, anecdotes don’t control for factors that might distort our observations and interpretations of a personal experience, as well as how we remember it later on. For example, we may misremember exactly what happened during the event, or may have misinterpreted what we observed during the event.

Figure 2. A testimonial from a celebrity about a brand of cigarettes
Figure 2. A testimonial from a celebrity about a brand of cigarettes (circa 1951)

In testimonials about therapeutic treatments, one very important factor that causes distortions of perceptions, interpretations, and memories is people’s expectations for the treatment. These expectations may cause them to conclude that their symptoms improved or disappeared even when they haven’t. This generally happens in one of two ways:

  1. The expectations may cause observers (e,g, patients, family members, doctors) to misperceive or misinterpret the behavior of those receiving a treatment, thereby concluding that the behavior has changed when it really hasn’t.
  2. The expectations may cause observers to experience improvement that has nothing to do with the nature of the treatment itself (e.g., the placebo effect or a self-fulfilling prophecy; see later posts in this series).

Specific examples of some of the problems mentioned here will be described in future posts.

You may contact me at


Pendergrast, M. (1996). Victims of memory: Sex abuse accusations and shattered lives (2nd ed.). Hinesburg, VT: Upper Access.

Rimland, B. (1998). The use of secretin in autism: Some preliminary answers. Autism Research Review International, 12(4), 3. Retrieved January 23, 2013, from

Terr, L. T. (1994). Unchained memories: True stories of traumatic memories, lost and found. New York: BasicBooks.

The Evolution of Domestic Dogs — Part 1

The dog was the first domesticated animal. Domestication is an “evolutionary process [that] has been influenced by humans to meet their needs” (Secretariat, 1992, p. 3). In other words, domestication of a species causes biological changes over generations through selection by humans for favorable traits (i.e., traits that are useful, valuable, aesthetically pleasing, etc.).

Domestication led to extraordinarily large changes in the behavioral characteristics of domestic dogs, as well as in their physical characteristics (which is obvious when one compares the many breeds of dogs). Scientific research on the evolution of dog behavior began in the mid-1800s, most notably in the work of Charles Darwin (Darwin, 1872). In the middle of the twentieth century, a deeper understanding of the evolution of dog behavior was gained by combining behavioral analyses with classical genetic analyses of dog breeds (for a review, see Scott & Fuller, 1965).

Figure 1. An extreme example of the effects of domestication on the physical characteristics of a subspecies of the gray wolf.

Over the past 20 years, archaeological discoveries in combination with the results of highly sophisticated genetic analyses have shed a great deal of light on the evolution of domestic dogs (for a review, see Larson, Karlsson, Perri, et al., 2012). For example, there now is little doubt that domestic dogs evolved from the gray wolf, which is found in many parts of Europe and Asia (Honeycutt, 2010; Wayne & Ostrander, 2007). In fact, domestic dogs (Canis lupus familiaris) are considered to be a subspecies of the gray wolf (Canis lupus lupus). This means that, although dogs and wolves have physical features that often are very different, they can mate and produce fertile offspring.

Nevertheless, there still is much controversy about when and where domestic dogs originated. These disagreements are focused on the answers to two questions: when did domestic dogs “split” from gray wolves and where did this happen? These questions have proved difficult to answer because the results of genetic and archaeological research are complex and, hence, very difficult to interpret.

Genetic research on differences in DNA sequences have led to a wide range of estimates about when dogs and wolves first diverged: sometime between 20,000 to 100,000 years ago. One reason for the wide variation in these estimates is that dogs and wolves probably continued to interbreed, not only over long periods of time but also in many locations (Vilà, Savolainen, Maldonado, et al., 1997).

Archaeological researchers find no clear evidence for the existence of domestic dogs until about 15,000-30,000 years ago (Germonpré, Sablin, Stevens, 2009; Ovodov, Crockford, Kuzmin, et al., 2011). A major difficulty with interpreting archaeological findings, however, is that physical characteristics typically used to distinguish domestic dogs from wolves (e.g., the size and position of the teeth, the size and shape of the skull, etc.) probably varied much more in ancient dog populations than they do today (Larson, Karlsson, Perri, et al., 2012). In addition, nothing is known about the variation of these traits in populations of ancient wolves. In other words, the physical characteristics used to distinguish modern dogs and wolves probably overlapped to a relatively large extent in ancient dogs and wolves, thereby making it very difficult for archaeologists to know if they are looking at the bones and teeth of a wolf or a dog.

Figure 2. The 33,000-year-old skull of a wolf-like animal reputed to be a dog (Ovodov, Crockford, Kuzmin, et al., 2011)
Figure 2. The 33,000-year-old skull of a wolf-like animal reputed to be a dog (Ovodov, Crockford, Kuzmin, et al., 2011)

As of now, both the archeological and genetic evidence allow us to conclude with certainty that domestic dogs existed at least 15,000 years ago (Larson, Karlsson, Perri, et al., 2012). It is still an open question, however, if ancient dog populations leading to modern domestic dogs first diverged from wolves earlier than that.

In the next post, I’ll review what recent research seems to tell us about the genetic differences linked to the many physical and behavioral differences between (a) modern domestic dogs and their ancestral species, and (b) the various breeds of modern domestic dogs.

You may contact me at


Darwin, C. (1872). The expression of the emotions in man and animals. London: John Murray. Retrieved January 21, 2013, at

Germonpré M, Sablin MV, Stevens RE, Hedges REM, Hofreier M, Stiller M, et al. (2009). Fossil dogs and wolves from Palaeolithic sites in Belgium, the Ukraine and Russia: Osteometry, ancient DNA and stable isotopes. Journal of Archaeological Science, 36, 473-490. doi: 10.1016/j.jas.2008.09.033

Honeycutt, R. L. (2010). Unraveling the mysteries of dog evolution. BMC Biology, 8(20). doi:10.1186/1741-7007-8-20

Larson, G., Karlsson, E. K., Perri, A., Webster, M. T., Ho, S. Y. W., Peters, J., et al. (2012). Rethinking dog domestication by integrating genetics, archeology, and biogeography. Proceedings of the National Academy of Sciences, 109, 8878–8883. doi: 10.1073/pnas.1203005109

Ovodov, N. D., Crockford, S. J., Kuzmin, Y. V., Higham, T. F. G., Hodgins, G. W. L., & van der Plicht, J. (2011). A 33,000-year-old incipient dog from the Altai Mountains of Siberia: Evidence of the earliest domestication disrupted by the Last Glacial Maximum. PLoS ONE 6(7): e22821. doi:10.1371/journal.pone.0022821

Scott, J. P., & Fuller, J. L. (1965). Genetics and the social behavior of the dog. Chicago: University of Chicago Press.

Secretariat, C. B. D. (1992). The Convention on Biological Diversity. Retrieved December 30, 2012, from

Vilà, C., Savolainen, P., Maldonado, J. E., Amoim, I. R., Rice, J. E., Honeycutt, R.L., et al. (1997). Multiple and ancient origins of the domestic dog. Science, 276, 1687-1689.

Wayne, R. K., & Ostrander, E. A. (2007). Lessons learned from the dog genome. Trends in Genetics, 23, 557–567. doi: 10.1016/j.tig.2007.08.013

Increasing College Costs Causes Autism

Boing Boing‘s Cory Doctorow reported on a spurious correlation of nearly 1.0 between autism and organic-food sales discovered by Jasonp55 on Skeptic Reddit.

This inspired me to look for other extremely high (albeit spurious) correlations with autism. I discovered a correlation of 0.994 between college costs (tuition + fees) and autism rates between the years 1999 and 2007, inclusive.

In the article that I’m certain to get published in Science, my main conclusion will be this: if we want to slash autism rates, we’ll need to drastically reduce college costs by returning educational funding to the levels of previous decades.

Here’s a graph of the cumulative percentages of the two variables that shows clearly their close association.


You may contact me at

Data Sources
1. Office of Special Education Programs, Data Analysis System (DANS), OMB# 1820-0043: “Children with Disabilities Receiving Special Education Under Part B of the Individuals with Disabilities Education Act”
Table 1-11. Number of children and students served under IDEA, Part B, in the U.S. and outlying areas by age group, year, and disability category: Fall 1999 through fall 2008 (Age Group 6-21)
2. National Center for Education Statistics: Digest of Education Statistics 2010 Tables and Figures
Table 345. Average undergraduate tuition and fees and room and board rates charged for full-time students in degree-granting institutions, by type and control of institution: 1964-65 through 2009-10