In my PSY 101 class, we recently discussed some problems with using anecdotes and testimonials as evidence for claims. In this post, I want to begin to explore this issue in more depth. In future posts, I will discuss further some of the issues touched upon here.
One afternoon, Eileen Lipsker was sitting in her family room watching Jessica, her red-headed five-year-old daughter, play with her friends. Eileen later reported that she felt “spaced out” and was “thinking of nothing.” Lenore Terr (1994), a psychiatrist who spoke with her on many occasions, described what Eileen said happened next:
Jessica twisted her head to look at her mother. To ask something? Her chin pointed up in inquiry. She looked up and over her shoulder. Her eyes brightened. How odd! The young girl’s body remained stationary, while her head pivoted around and up…. And at exactly that moment Eileen Lipsker remembered something. She remembered it as a picture. She could see her redheaded friend Susan Nason looking up, twisting her head, and trying to catch her eye.
Eileen, eight years old, stood outdoors, on a spot a little above the place where her best friend was sitting. It was 1969, twenty years earlier. The sun was beaming directly into Susan’s eyes. And Eileen could see that Susan was afraid…. [Eileen] looked away from those arresting eyes and saw the silhouette of her father. Both of George Franklin’s hands were raised above his head. He was gripping a rock. He steadied himself to bring it down. His target was Susan. (pp. 2-3)
Eileen told Terr that this is how she first recovered her repressed memory of Susan Nason’s murder by her father. Eileen’s recounting of the memory recovery is an example of an anecdote: a brief story told by an individual about a personal experience. No matter how interesting or compelling an anecdote may be, it doesn’t provide good evidence for a claim because it is based on interpretations and memories of personal experiences. In other words, an anecdote is inadequate evidence for a claim because it does not control for factors that affect how a personal experience is (a) initially perceived and interpreted, and (b) eventually remembered.
Autistic Disorder is a severe mental disorder that develops in children before the age of three years. It has three main symptoms: a severe impairment in social interaction, a severe impairment in the ability to communicate, and a severely restricted range of interests, activities, and behaviors. On occasion, new treatments for autism are announced that seem to offer hope for either a cure or, at least, a dramatic reduction of symptoms. One such well-publicized treatment used injections of secretin–a hormone that assists in the digestion of food. Some have claimed that secretin improves the social and language skills of autistic individuals by affecting specific behaviors such as the amount of eye contact made, the level of awareness of one’s surroundings, the degree of sociability, and the amount of speech. One proponent of secretin therapy provided the following evidence for this claim:
The good news is that confirmatory evidence of the power of secretin keeps coming. A national newspaper told of Florida pediatrician Jeff Bradstreet’s own four-year-old son, Matthew, shocking his parents by holding his first normal conversation with them the day after his first secretin infusion. And Virginia pediatrician Lawrence Leichtman told me of his “miracle case”: a five-year-old who had previously said only two words amazed all in the office by saying, 15 minutes after his infusion, “I am hungry. I want to eat.” Most cases are much less dramatic, but the autism world is excited, and for good reason. (Rimland, 1998, p. 3)
Is this good evidence for the effectiveness of secretin in the treatment of autism? The evidence consists of two testimonials. A testimonial is an anecdote that describes the supposed merits of a product or service. Testimonials are not good evidence for a claim because they are anecdotes and, as stated above, anecdotes don’t control for factors that might distort our observations and interpretations of a personal experience, as well as how we remember it later on. For example, we may misremember exactly what happened during the event, or may have misinterpreted what we observed during the event.
In testimonials about therapeutic treatments, one very important factor that causes distortions of perceptions, interpretations, and memories is people’s expectations for the treatment. These expectations may cause them to conclude that their symptoms improved or disappeared even when they haven’t. This generally happens in one of two ways:
- The expectations may cause observers (e,g, patients, family members, doctors) to misperceive or misinterpret the behavior of those receiving a treatment, thereby concluding that the behavior has changed when it really hasn’t.
- The expectations may cause observers to experience improvement that has nothing to do with the nature of the treatment itself (e.g., the placebo effect or a self-fulfilling prophecy; see later posts in this series).
Specific examples of some of the problems mentioned here will be described in future posts.
You may contact me at firstname.lastname@example.org
Pendergrast, M. (1996). Victims of memory: Sex abuse accusations and shattered lives (2nd ed.). Hinesburg, VT: Upper Access.
Rimland, B. (1998). The use of secretin in autism: Some preliminary answers. Autism Research Review International, 12(4), 3. Retrieved January 23, 2013, from http://legacy.autism.com/ari/editorials/ed_secretinfindings.htm
Terr, L. T. (1994). Unchained memories: True stories of traumatic memories, lost and found. New York: BasicBooks.