Quote: Cormac McCarthy

As I've said before, I'm a huge fan of Cormac McCarthy's. The following is one (somewhat uncharacteristic) example of the kind of writing that makes him so awesome. From his 1985 masterpiece Blood Meridian:

The man who believes that the secrets of the world are forever hidden lives in mystery and fear. Superstition will drag him down. The rain will erode the deeds of his life. But the man who sets himself the task of singling out the thread of order from the tapestry will by the decision alone have taken charge of the world and it is only by such taking charge that he will effect a way to dictate the terms of his own fate.

3 Quarks Daily science blogging award

The three winners of 2010 3 Quarks Daily Prize in Science, judged by Richard Dawkins, have been announced. The winning posts:

1. "Gut bacteria in Japanese people borrowed digesting genes from ocean bacteria" by Ed Yong of Not Exactly Rocket Science.
2. "Skullcaps and Genomes" by Carl Zimmer of The Loom.
3. "The Evolution of Chloroplasts" by Margaret Morgan of My Growing Passion.

All three articles are most certainly worth a read, so do check them out.

A slight criticism... As I pointed out in the comments at 3 Quarks Daily, the winning entries this year are rather similar - too similar. Not only are all three on biology, all three two concern horizontal gene transfer. The chances that the 'real' best three science blog posts of the year just happened to be on a single two closely related topics is infinitesimal. Both Dawkins, and the editors who whittled down the entries to the nine semifinalists, frankly, ought to have been more ecumenical. I thought "MSL: Mars Action Hero" deserved to be in the top three...

Anecdotes as evidence

An anecdote is a tale, story or an account of events that is sometimes humorous or meant to convey a moral, but is often taken as evidence. In this latter sense there are two broad categories. Firstly, there is testimony, i.e., inferring that such-and-such happened or is true because someone said so. For simplicity's sake, we can further divide testimony into a bunch of different subtypes, most notably, into eyewitness testimony, expert testimony and hearsay. The second broad category of anecdotal evidence is personal experiences (or, if you will, ‘personal testimony’) that takes the form 'I saw so-and-so, therefore it is reasonable for me to believe such-and-such'. Note the key similarity and key difference between these two categories: in both cases someone's experience (or alleged experience) is taken as evidence for some claim, but, for testimony and not for one's own experiences, one has to believe a particular experience occurred on someone else's say-so. Now, it seems perfectly reasonable to believe some things on testimony or personal experience - indeed life would be impossible without it. I am currently having the personal experience of sitting on my couch with my notebook on my lap while typing this post. Hyperbolic doubt aside, I have no good reasons to doubt that his is what is really going on, and you have little reason to doubt my testimony. On the other hand, however, it is common for people to believe wild or improbable things – that aliens regularly visit earth, that highly diluted substances can cure any illness, etc. – solely on the strength of anecdotes. So what exactly is the evidentiary status of anecdotes?

It is vital, first, to distinguish between two types of proposition that anecdotes can be claimed to be evidence for: causal and observational propositions. A causal claim is of the form “P caused X” (or “P, Q and R caused X”) and an observational proposition is “P happened” (or “P occurred, then Q happen and after that R”). This distinction is important for a simple reason: an anecdote can in isolation (almost) never establish the truth or falsehood of a causal claim, it can only be evidence for observational propositions. Why this is the case will be clearer if we think through some concrete examples.

Consider the claim “Mary cheated on John with Bob”. This kind of proposition is pretty straightforward. I can convince myself it is true simply by determining whether Mary and John are in an exclusive relationship, and, if I see Mary make out with Bob, I can reasonably conclude the proposition is true. Now, obviously, there are a bunch of ways I could get it wrong: maybe the woman involved wasn’t Mary, maybe it was simply a friendly hello kiss, or maybe Mary had broken up with John (so it’s making out, but not cheating). It’s clear, though, that if I’m just a little careful, there are a wide variety of circumstances in which I could be very confident Mary did in fact cheat based on my personal experiences. There are a couple of extra complications when John has to decide whether to believe my testimony – maybe I’m lying, for example – but, again, these are not difficult to understand even if they’re difficult to deal with in practice. In other words, we here have a clear case of an anecdote – both in the sense of personal experience and testimony – that can be a good reason to accept the truth of some proposition. Notice two things, though: the claim is not a causal one, and the plausibility (or prior probability) of it being true is pretty high since we know people do in fact cheat on each other regularly. I’ll explain what the latter means in a bit, but let’s move on to anecdotes as evidence for causal claims.

So consider the causal proposition “I took medicine X, I got better, therefore I got better because of medicine X”. This claim is much more complicated than the one about Mary, and you can’t determine whether it’s true simply by looking or making a few observations. Why? Because causality is counterfactual: to say A caused B, is to say B would not have occurred if A had not occurred. (There is a long-standing and complicated philosophical debate about causality. Take it from me: steer clear and stick to the counterfactual view). So, on our example, when you claim “medicine X caused my recovery” you’re committed to the counterfactual proposition that you wouldn’t have gotten better had you not taken medicine X. And you simply cannot, even in principle, know this. For one thing, you have an immune system (gasp!) which may have fought off your infection irrespective of whether you took the medicine. Alternatively, you could have ingested some other substance - maybe you took medicine Y as well, maybe you ate something therapeutic - that took care of the infection. In other words, there are a whole bunch of things - i.e. confounds - that could have caused your recovery, and a single observation – the source of an anecdote – simply cannot distinguish between them. In general, the only reliable way to establish the truth or falsity of causal claims is to do controlled experiments. Let’s look at this a bit more closely.

Assume we are trying to understand the causal relationship between four dichotomous and independent variables (A-D) and a particular dichotomous outcome (O). Assume also that these four variables exhaust the universe of all variables even conceivably related to the outcome. Our aim is to make either inclusion inferences (i.e. conclude the relevant variable has a causal relationship to the outcome) or exclusion inferences (i.e. that the variable does not have a causal relationship to the outcome). Given these assumptions, inclusion inferences are valid only when, from:

A1   B1   C1   D1  =>  O1
A2   B1   C1   D1  => O2

it is concluded that A1 caused O1. The inference is valid because all the variables except one (A) was controlled – held constant – and given the outcome changed, it follows that A is causally related to O. Exclusion inferences are valid only when, from:

A1   B1   C1    D1   =>  O1
A2   B1   C1    D1   =>  O1

it is concluded A1 is not causally related to O1. Again, the validity of the argument is assured because all the variables save one (A) was controlled. Given that A varies while O does not, it follows that A is not causally related to O. Notice that in both cases we are comparing one outcome with a counterfactual. The process of engineering such comparisons is called experimentation and it is at the very heart of the scientific method. (These, by the way, are versions of Mill's Methods).

If we want to determine whether medicine X can cure some disease, we cannot rely on anecdotes because they don’t allow for counterfactual comparison. Assume, for example, that A1 is taking medicine X, and A2 is not taking it; that B1 is having an immune system, and B2 not; that C1 is taking medicine Y, and C2 not taking it; that D1 is being overweight, D2 is not being overweight; and, finally, that O1 is getting better and O2 is not getting better. When we have a single observation - we know that Thaba over there took medicine X, has a healthy immune system, that he isn't taking medicine Y, that he's rather overweight and that he got better after a few days - all we have is:

A1   B1   C1    D2   =>  O1

We have no proper counterfactual: we have not controlled for variables B, C, or D so we can't make any logical inferences about variable A. (Making this inference is the post hoc fallacy). At best, we can say that taking the medicine is possibly related to getting better, but then the same goes for B, C and D. Note also that adding more anecdotes does not resolve our problem: in the real world there are many more than just four variables so things are much more complicated, experiments involving humans are always possibly confounded by the placebo effect, and, importantly, the variables may interact in complex ways. In the oft repeated phrase, the plural of anecdote is anecdotes, not data. Thousands of anecdotes are no more convincing that a single anecdote. As a result, then, anecdotes cannot in general (that is, barring extreme exceptions) establish the truth or falsity of causal propositions.

As I showed above anecdotes can reasonably be taken as convincing evidence for observational claims. But that does not mean we should believe every anecdote (concerning observational propositions). Bob Carrol of Skepdic (an excellent resource worth referring to often, by the way) nicely enumerates the possible problems:

Anecdotes are unreliable for various reasons. Stories are prone to contamination by beliefs, later experiences, feedback, selective attention to details, and so on. Most stories get distorted in the telling and the retelling. Events get exaggerated. Time sequences get confused. Details get muddled. Memories are imperfect and selective; they are often filled in after the fact. People misinterpret their experiences. Experiences are conditioned by biases, memories, and beliefs, so people's perceptions might not be accurate. Most people aren't expecting to be deceived, so they may not be aware of deceptions that others might engage in. Some people make up stories. Some stories are delusions. Sometimes events are inappropriately deemed psychic simply because they seem improbable when they might not be that improbable after all. In short, anecdotes are inherently problematic and are usually impossible to test for accuracy.

In other words, while anecdotes can be good evidence for believing observational propositions - "x happened" - for the reasons listed above, we certainly can't accept all anecdotes uncritically. So what to do? Life is impossible if we dismiss all anecdotes, but we'll be led astray if we accept all anecdotes. The solution is skepticism: that is, being open minded but then filtering beliefs through a bullshit detector. Doing this is simple in principle, but incredibly difficult in practice, so some examples are in order. (Recommended books: Thinking About Thinking by Anthony Flew, The Demon-Haunted World by Carl Sagan [my review], Truth by Simon Blackburn [my review], and Mistakes Were Made (But Not by Me) by Travis & Aronson [my review]).

Megan Fox: not in Jeff's league. 

One of the most important and useful bullshit detecting skills is weighing up the evidence against the plausibility of the claim (here is an example of me doing this). Determine, firstly, how plausible the claim is given everything else we know. For example, given everything I know about my friend Jeff, people in general and the state of technology, the proposition that he has flown in an airplane at least once is highly plausible. (He is middle-class, airplane tickets are cheap and abundant, I've seen him in other cities, etc.). On the contrary, given everything we know, it is extremely implausible that he once had a threesome with Megan Fox and Jessica Alba. (For one thing, he is short and balding. For another,  he's never been to the US). The threesome claim is an extraordinary one: given what we know about attractive celebrities, balding South African men, sexual psychology, and so on, Jeff having a threesome with Fox and Alba is just not the kind of thing we expect to happen. Having determined the plausibility of a claim, the next step is to assess the strength of the evidence. In our example, all the evidence we have is Jeff's testimony. As Bob Carrol explained above, anecdotes are often unreliable because people suffer from innumerable cognitive biases and, more obviously, they sometimes lie. Since in the Alba-Fox example Jeff has a strong motivation to lie - having it believed is highly status-enhancing - his testimony is further undermined. What we have, then, is an extraordinary claim, the only evidence for which is very weak. It is reasonable, then, to withhold assent until better evidence is provided.

A somewhat more enlightening example is alien abduction. People from all over the world claim to have been abducted by extraterrestrials and then molested, lectured on the necessity of world peace, and so on. So, step one: how plausible are these claims? Given what we know about physics and human psychology, not very. First of all, we currently have no evidence that life - let alone intelligent life - exists anywhere else in the universe. (My gut tells me alien life is abundant, but as Carl Sagan pointed out, we shouldn't think with our guts). Secondly, if intelligent life does exist, the aliens will in all likelihood be tens of light-years or more distant, and, since we have no reason to think faster than light travel is practicable, there is no known way for aliens to get to earth in a reasonable period of time. Step two: what about the evidence? Again we have anecdotes: tales from people who claim to have been abducted. Significantly for this example, there is a highly plausible alternative explanation that undermines the evidentiary status of the accounts, namely, hypnogogia and hympnopompia. Briefly, these are vivid hallucinations that occur as you're falling asleep or waking up that are accompanied by sleep paralysis. Typically, a person wakes up terrified and unable to move, senses (an often malevolent) 'presence' in the room, and may also experience a variety of visual, auditory and proprioceptionary hallucinations. Tellingly, this well-studied phenomenon closely mirrors accounts of alien abduction, which often feature extreme fear, a 'presence' in the room, and being unable to move. Since these experiences are accompanied by visual hallucinations and alien visitation is a common trope in popular culture, the other reported experiences are easily accounted for. Importantly, also, hypnogogia and hypnopompia are common (much more common than claims of alien abduction). Extraordinary claims require extraordinary evidence. Since claims of abduction are extraordinary, anecdotes of being abducted are far from extraordinary evidence, so, until much more evidence is provided, it is reasonable to withhold assent.

So... what exactly is the evidentiary status of anecdotes? In summary: (1) anecdotes on their own can never establish the truth or falsity of causal propositions. (2) While anecdotes can be evidence for observational propositions, the plausibility of the claim must be taken into account. To be believed, highly implausible claims require much, much more than mere anecdotes.

The Randi Prayer

The beautiful Randi Prayer, by Crispian Jago of Science, Reason and Critical Thinking....

Our Conjuror

Who art in Florida

Amazing be thy name

Thy Million-Dollar Challenge

Will not be done

Give us this day

Empirical evidence

And debunk our irrationality

As we debunk those who make unsubstantiated claims to us

And lead us not into woo woo

But deliver us from credulity

For thine is the rationality

The logic and the reason

Until unequivocal evidence proves otherwise

Skeptics in the Pizzeria

The next - and for Angela and I, final - Durban Skeptics in the Pizzeria is scheduled for Thursday, June 17th from 18:30 till late at Pizzetta. (It'll be our final one because we're moving to Joburg at the end of the month). Come join us!

A map:


View Larger Map

Picture: Zapiro's follow-up

In all the hullabaloo over Draw Mohammad Day, I forgot to point to cartoonist Jonathan "Zapiro" Shapiro's follow-up cartoon. You'll recall that this cartoon of Zapiro's - in my view one of the best drawn for Draw Mo Day - caused quite a stir, and Shapiro even received death threats. Anyway, Zapiro followed-up with the following (click for a bigger version):

African science blogrolling for June

The updated African science and skepticism blog roll for June... If you know of blogs not listed here, please let me know. Also: add it to your blog! Do a post like this one! (Email me, and I'll send you the HTML). Note: I remove blogs that have been inactive for more than 6 months, so if you're no longer on the list and have resumed blogging, please email me.

• 01 and the universe
• Acinonyx Scepticus
• Amanuensis
  
• ASSAf Blog
• Botswana Skeptic 
• Bomoko and other nonsense words
  
• Communicating Science, the African Way
  
• Defollyant's AntiBlog
  
• Effortless Incitement
• Ewan’s Corner
• Expensive Beliefs
• Fluxosaurus's Blog **new**
• Geekery
• Grumpy Old Man
• Health Frog  
• Hello Universe, This is Nessie
  
• Ionian Enchantment
• Limbic Nutrition
• Lenny Says 
• McBrolloks
• Meh Blog
  
• Nathan Bond's TART Remarks
  
• Orion Spur
  
• Other Things Amanzi
• Pickled Bushman
• Psychohistorian
  
• Reason Check
  
• Retroid Raving
• Roy Jobson
• Science Blog **new**
• Scorched
• Skeptic South Africa
• Stop Danie Krügel
• Sumbandlila Mission Blog
  
• Synapses
• Tauriq Moosa
• The Joys of Atheism
  
• The Science Of Sport
• The Skeptic Black Sheep
• The Skeptic Detective

Video: Saving Africa's Witch Children

Update: I see the video I embedded has been removed. There is, *cough*cough*, a torrent available...

A documentary (embedded below, or click here) on one of the greatest tragedies of our time: witch hunts in Africa. The only appropriate response is burning indignation. (Note: you can download this video by following the link and creating an account with Vimeo).


Saving Africa's Witch Children from Africa's Witch Children on Vimeo.

See also my posts: "Human Sacrifice in Uganda" and "Witch Trials in Africa".