Friday, July 30, 2010

Solid Rock: Preying on the desperate, ignorant and gullible

Here is some good news: South Africa's Advertising Standards Authority (ASA) is doing its job. A church called "Solid Rock Church of Miracles" (warning: horrid late-90s web design...) ran an ad with an accompanying photograph as 'evidence': "Bring the blind, the lame, Aids and cancer victims. 43 Crutches! 25 Walking sticks! 2 White canes from blind people! Already left behind!". In other words... WE DO MIRACLES! Never mind those doctors! Jebus will take care of you! And if you bring along some cash to stuff into our collection boxes, that would be great... Given that praying really, really hard to an invisible sky wizard cannot, in fact, cure AIDS, cancer, blindness or lameness, taking out such an ad preys on the desperate, ignorant and gullible.

Thus, thankfully, ruled the ASA. The reaction of Solid Rock's pastor - former bike gang-member Johan van Wyk - has been rather amusing and more that a little risible. Despite the fact that the miracles bit is in their name and in their mission statement van Wyk told the ASA he's not "claiming anything in these advertisements. It just encourages people to bring everybody. The crutches etc. are hanging in its church so there is no false claim". Right... there are crutches hanging in my church, nobody ever reads between the lines in advertisements, therefore I am not a lying bastard... But why is van Wyk being so coy? Well, maybe it's the five previous rulings against his church. The ad in question, notes the ASA, bears a "striking resemblance" to other ads for which it has already been sanctioned.  "As with all previous rulings," concluded the ASA, "the current advertisement promotes the church, under the auspices that it can cure various diseases or offer treatment for them. The respondent is clearly continuing to make unsubstantiated healing claims despite an instruction not to do so."

And van Wyk's reaction to the ruling? From the Sunday Independent: 
"Look, for our members and for ourselves, miracles are very real," Van Wyk told the Saturday Star of the divine healing he claims happens at the Northcliff, Joburg, church come weekends. "Every weekend we experience miracles and hear testimonies. For us they're very real ...The person most blind is the one who doesn't want to see. Our business is not about proving miracles; it's to help people."
Aside from the surprisingly post-modernist "for us", this is a fascinating insight into the religious mind. There is no room for careful consideration, for even a modicum of doubt, or even for expertise. ('A doctor attended services?' GREAT!). All we have is assertion, emotion, and a petulant demand to be believed. Think about it. If this guy really can cure AIDS, he has a moral duty to present the kind of evidence that would convince a skeptic. But all we're given is unsubstantiated anecdotes - testimony from the emotional and medically untrained, given to the emotional and medically untrained.

Most frustratingly, the bad publicity has likely done the church good:
The church, it seems, has reaped the benefits. "The lady (Phillips [who laid the complain with the ASA]), instead of stopping us, we've had so much publicity. If people will come because of that, we'll have to see.
"We don't advocate everybody will be healed here. But through all these years, only one person has ever complained. She (Phillips) is harming the sick people, not us."
It's time for the ASA to put some bite in its rulings. Any organization that consistently flouts the ASA's judgments deserves to be punished - as severely as the law allows.

Tuesday, July 27, 2010

Video: Bad Universe

So the most excellent Phil "the Bad Astronomer" Plait  has a new Discovery Channel show coming out called "Bad Universe". It seems to be based on his book Death From the Skies! about all the ways the universe could snuff us out. The first episode is on asteroids; and it looks frekkin awesome. Embedded below, or click here for the direct link.

Monday, July 26, 2010

A decidedly weird response

I forgot to mention the rather weird response David Spurrett and I received from the authors of the WEIRD paper I reported on the other day. For those of you who missed it, David and I basically agreed with Henrich et. al. (pdf) that Western undergraduates are often extreme outliers (i.e. weird), and that it is therefore extremely problematic that the behavioral science literature has relied so heavily on this group. Or, as we put it in our abstract: a literature focused on outliers is flawed. We went on to argue, though, that Henrich and his colleagues missed another big problem: most behavioral scientists are themselves deeply WEIRD - likely even weirder than their subjects.

As usual for Behavioral & Brain Sciences, the authors responded to all the open peer-commentaries, including ours. Here is what Henrich and co had to say about our piece (the weird bit in bold):
Finally, Meadon & Spurrett suggest that one important way of addressing these challenges is to bring more non-WEIRD researchers into the process. Empirical findings should be peer reviewed by researchers who bring different cultural models and implicit expectations to the problem. We agree with all these suggestions: Researchers can view phenomena from a novel perspective, not constrained by their own intuitions, when they study those from other cultures, and can potentially discover phenomena that they otherwise would not see. However, we disagree with an extreme version of this argument, which proposes that researchers should entirely avoid studying people from their own culture. Researcher’s intuitions about the ways people in their own cultures think can be a useful source of understanding in building theories and in honing research instruments. 
Well.... sure. But David and I never suggested anything of the kind, and it's not an idea either of us has ever taken seriously. Indeed, that researchers have some advantages when studying their own cultures was part of our point. There are excellent reasons to think diverse research communities are better, so one reason additional non-WEIRD researchers would be useful is that they often have different biases, so they may spot hidden assumptions, value judgments masquerading as facts or other problems that WEIRD researchers may miss. A Nepalese psychologist peer-reviewing a German study - even on German subjects - may see something someone as WEIRD as a German researcher overlooked, for example. But, equally importantly, a Nepalese researcher often has different knowledge like hard-to-learn cultural sensitivity, in-depth knowledge of both a local language and academic English (vital for accurate translation), or an understanding of some important nuance. So, yes, people studying their own cultures "can be a useful source of understanding in building theories and in honing research instruments". That was part of our point.

Anyway, the rest of Henrich et. al.'s response to our commentary, in which they make several excellent points:
More non-WEIRD researchers should be brought into the discussion, as well as onto collaborative research teams. Research teams themselves that better reflect broad global diversity can more effectively address the challenges delineated by Fessler, Rochat, and Bennis. [other commentators on the paper]. With regard to these points, it is instructive to consider why psychology is more dominated by American research than any other science (May 1997). One possibility is that pursuing a career in psychology is a luxury that people cannot afford until the countries and societies in which they live have achieved sufficient economic evelopment. This may be part of the explanation, although this would not explain why universities in wealthy societies like those of Japan and Western Europe typically have proportionately smaller complements of psychology researchers and majors than do North American universities. Another possibility, which we highlight here, is that the field’s emphasis on WEIRD samples, coupled with the guiding assumption of universal psychological processes, tends to unintentionally marginalize international research. If non-WEIRD researchers are interested in extending findings initially established with WEIRD samples in their home populations, such as findings associated with motivations for self-enhancement, they may well be unable to replicate the American results. The implicit assumption that self-enhancement motivations are similar everywhere would suggest that such failed replications are not due to the nature of the samples studied but instead due to some kind of unspecified deficiency in the methods of the non-WEIRD researchers. American researchers have a distinct advantage in that the field’s key theories were largely constructed on data from American participants, and we suggest that this is likely why American research constitutes 70% of the field’s citations. International research suffers from the disadvantage of trying to extend American-based theories with participants who often have different psychological tendencies, yielding results that are difficult to interpret while embracing an untested assumption of universal psychological processes. In contrast, if the field comes to recognize that psychological phenomena cannot be assumed to be universal until demonstrated as such, then research conducted by non-WEIRD researchers, guided by non-WEIRD intuitions, and studied with non-WEIRD samples, would come to be viewed as particularly important for understanding human psychology.
For more discussion, see the comments on my WEIRD post over on Google Buzz, featuring contributions by David, yours truly and the most excellent Simon Halliday.

Saturday, July 24, 2010

Are most experimental subjects in behavioral science WEIRD?

Note: here is a follow up post.

My supervisor David Spurrett and I have a commentary on an important paper - "The weirdest people in the world?" (pdf) - in the most recent edition of Behavioral & Brain Sciences. The authors of the paper, Canadian psychologists Joseph Henrich, Steven Heine and Ara Norenzayan, argue that most experimental subjects in the behavioral sciences are WEIRD - Western, Educated, Industrialized, Rich, and Democratic - and thus weird - not representative of most human beings. And this, if true, is a very serious problem indeed. Behavioral scientists (anthropologists, psychologists, behavioral economists and so on) are often interested in explaining the brains, minds and behavior of Homo sapiens as a species. (Some scientists, of course, are only interested in understanding specific cultures or what makes us different, but one important goal of the behavioral sciences has long been to explain universal human behavior). As evolutionary psychologists John Tooby and Leda Cosmides have put it, they "seek to characterize the universal, species-typical architecture of [the information-processing mechanisms that generate behavior]".

But... Henrich and his colleagues review a large body of literature that seems to show that, across several domains, Western undergraduates - the workhorses of the behavioral sciences - are extreme outliers. In other words, if they are correct, most of the data behavioral scientists have used to test hypothesis and to drive theorizing derives from subjects who are possibly the least suited for generalizing about the human race. Take as an example the Müller-Lyer illusion. In the diagram below, the lines labeled "a" and "b" are exactly equal in length, but many subjects perceive "b" as longer than "a".

This finding (which goes back all the way to 1889) has been used to make deductions about how the human visual system works. The Wikipedia article on the illusion, for example, states that one possible explanation for the effect is that "the visual system processes that judge depth and distance assume in general that the 'angles in' configuration corresponds to an object which is closer, and the 'angles out' configuration corresponds to an object which is far away". Plausible enough. Except that for some people - San foragers, for example - the illusion does not exist, and in many other non-WEIRD societies the effect size is significantly smaller. Henrich and his colleagues cite the work of Segall et. al. (1966), who worked out the magnitude of the illusion across 16 societies by varying the relative lengths of "a" and "b" and then asking subjects to indicate when they thought the lines were equal. The percentage by which "a" must be longer than "b" before the lines are adjudged equal - what they call the "point of subjective equality" (PSE) - varies substantially between subjects from different cultures - and, importantly, WEIRD-subjects are extreme outliers. The results are summarized in the following graph:

Both WEIRD adults and children (aged 5-11) require "a" to be 18%+ longer than "b" before they're perceived as equal, but for the San and South African miners, the illusion simply does not exist - their PSEs are not statistically distinguishable from 0. Why this difference arises is unknown, but Segall et. al. claim it is due to WEIRD people's visual systems developing differently because modern environments expose them to ("unnatural") shapes like 'carpeted corners', thus calibrating their visual systems in a way that favors the emergence of the illusion. Whatever the true explanation, however, it is clear that it is not permissible to use the existence of the illusion among WEIRD subjects to make inferences about the visual system. This is especially true since the San subjects were hunter-gatherers, just like all people for the vast majority of human evolutionary history. Given that species-typical features of the visual system would have evolved in this period, it is particularly telling that PSE seems to be positively correlated with the 'modernity' of the societies in question. (Warning: this is an "eyeball" observation; I haven't done a proper statistical analysis. Caveat emptor).

This is one example from an extremely long paper, but it conveys a flavor of the kind of evidence the authors present. (For much more, see "We agree it's WEIRD, but is it WEIRD enough?" over at Neuroanthropology). Having read the article very carefully, and despite some concerns, I think Henrich, Heine and Norenzayan are right: the Western undergraduate is often unrepresentative of humanity, and the behavioral science literature needs a lot of fixing as a result. (Most obviously, we need far more large, highly-powered, globally representative, prospectively designed, cross-cultural studies). Serious as this is, unfortunately, it gets worse... Since David and I worked extremely hard to present our argument clearly and concisely in our commentary (pdf - our piece starts on p. 44 of the pdf, paginated by BBS as p. 104), and I doubt I could improve on it, what follows is a slightly edited - simplified and somewhat de-academicized - version of the meat of our argument. (Note: each issue of BBS consists of a "target article" - in this case, Henrich et. al. - and 20 or so short peer-commentaries).

Henrich et al. underplay – to the point of missing – that how the behavioural sciences research community itself is constituted introduces biases. That the subject-pool of behavioural science is so shallow is indeed a serious problem, but so is the fact that the majority of behavioural researchers are themselves deeply WEIRD. People in Western countries have, on average, a remarkably homogeneous set of values compared to the full range of worldwide variability (Inglehart & Welzel 2005), and the data Henrich and his colleagues present suggest similarly population-level homogeneity in cognitive styles. Moreover, academics are more uniform than the populations from which they are drawn, so it is likely behavioral scientists are even WEIRDer than their most common subjects. Henrich and his colleagues review a bunch of studies and experiments that did not strike those who designed and conducted them as focused on outliers. Intelligent scientists acting in good faith conducted, peer-reviewed, and published this research, in many cases honestly believing that it threw light on human nature. This forcefully illustrates the power of the biases on the part of researchers themselves. It also suggests that, besides widening the pool of subjects, there are significant gains to be made by broadening the range of inputs to the scientific process, including in the conception, design, and evaluation of empirical and theoretical work. Given that diverse groups are demonstrably better at some kinds of problem solving, as things stand, the WEIRD-dominated literature is robbed of potentially worthwhile perspectives, critiques, and hypotheses that a truly global research community could provide. Clearly, simply increasing the number of behavioural sciences researchers will, in general, be beneficial. Our key contention, though, is that the marginal benefits of additional Western researchers are much smaller than the marginal benefits of more non-Western researchers, among other things, just because they are non-Western.

The non-Western world, in short, can contribute not only additional subjects to experiment upon – the main focus of the target article’s recommendations – but also additional researchers, with novel perspectives and ideas and who are less affected by WEIRD biases. (Naturally, these researchers will have biases of their own. Our claim is not that there is someone who consistently knows better; it is that diverse groups of investigators can avoid some kinds of error.) Clearly, these researchers will have to be educated, will likely be middle class, and, since science flourishes in politically open societies, they will tend be concentrated in liberal countries. Nevertheless, additional non-Western researchers, even if they are educated and relatively wealthy, could be a boon to the behavioural sciences.

A direct and powerful way to remedy both sources of bias – too many WEIRD subjects and too few non-WEIRD researchers – is to foster research capacity in the non-Western world. Non-WEIRD researchers tend to study non-WEIRD subjects, so increasing their number will deepen the subject pool and widen the range of inputs to the scientific process at the same time. Building research capacity, however, should not merely involve collaborations led by WEIRD researchers; it should aim to generate studies led and initiated by non-Western researchers. Committed and long-term inter-institutional collaboration between Western and non-Western universities focused on remedying the deficits in the behavioral sciences literature should include internships at Western universities for non-Western researchers, stints at non-Western universities for WEIRD researchers, and extensive student exchange programs (especially for graduate students). Unlike many existing scholarship and exchange programs in the sciences, a key point of the necessary programs should be for the learning to proceed in both directions.

---------------------------------- Henrich, J., Heine, S., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33 (2-3), 61-83 DOI: 10.1017/S0140525X0999152X

Meadon, M., & Spurrett, D. (2010). It's not just the subjects – there are too many WEIRD researchers Behavioral and Brain Sciences, 33 (2-3), 104-105 DOI: 10.1017/S0140525X10000208

Tuesday, July 20, 2010

Depictions of violence in rock art

A street fight, via Wikipedia.
To understand the phenomena of murder, war, genocide, and other forms of human intraspecific violence, we need to know whether to invoke evolutionary biological explanations or restrict ourselves primarily to socio-cultural theories. If the incidence of violent conflict was high and recurrent for a substantial period during human evolution, and given that being killed drastically reduced fitness and killing may have increased it, then strong selective pressures would have favored physical and psychological adaptations to violence. Conversely, if interpersonal violence was rare or nonexistent until much more recently – until the rise of agriculture about 10,000 years ago, say – not enough time would have elapsed for natural selection to have forged significant new adaptations, and socio-cultural explanations of violence would thus predominate. (It should be noted, though, that recent human evolution has been very rapid, so this judgement may have to be revised in future as more evidence comes in). More precisely, whether adaptations to violence exist or not depends on the intensity of the selection pressures and their duration, and the intensity of the selection pressures is in turn a function of the frequency of violence and the magnitude of its impact on fitness. Thus, to determine the plausibility of positing traits that are adaptations to violence we need to know: (1) how frequent violence was, (2) whether it was recurrent in human evolutionary history and (3) how large its impact on inclusive fitness was.

Determining (3), of course, depends in part on the values we assign to (1) and (2). Being killed before reproduction obviously reduced fitness to zero, and being killed after reproduction eliminated all the kin altruism the individual would otherwise have engaged in. The impact on fitness of being injured depended on the severity of the injury, but it seems clear it would have been negative and serious. What we need to deduce the magnitude of (3) over human evolutionary history, then, is sound empirical estimates of (1) and (2). Unfortunately, however, these estimates are extremely difficult to make because the available evidence is sparse and often ambiguous. Broadly speaking, there are two lines of evidence available to us: studies of contemporary hunter-gatherers and the paleoanthropological and archeological record. There are several controversies around both lines of evidence, but for this post I'll focus on one type of evidence from the archeological record: depictions of violence in rock art.

A beautiful example of such a depiction is a San pictograph "Veg 'n Vlug" (Afrikaans: "Fight or Flight") that is near Clanwilliam in the Cederberg of South Africa. (Note: I've used the Auto-Level feature in Paint.NET to bring out some of the details):

John Parkington describes (large pdf; pp. 62 - 65) the scene thus:
The fight element is created by painting around a small recess in the rock surface to give the impression of a small cave from which a group of humans peer, one of them shooting arrows. A second group of humans, arranged as a procession and depicted apparently moving along a pair of red lines, face the cave occupants and also shoot arrows. From the ‘cave’ several people, most of them male, flee along more pairs of red parallel lines. One human figure, clearly lying prone is connected by these same lines to a strange seated  figure holding the end of the lines, neither of them directly connected with the cave itself. From the neck of the strange seated figure a single red line leads to another small figure with upraised arms.
Contrary to the hypothesis - favored by neo-Rousseauians like Brian Ferguson - that human evolutionary history was entirely (or largely) peaceful, then, we have at least an existence proof of such violence. Or do we? Ferguson has argued that pictographs seemingly depicting violence should not be interpreted literally, but rather metaphorically. In other words, "Veg 'n Vlug" doesn't depict an actual event, the artists meant something else entirely, or is perhaps an attempt at sympathetic magic. (To be clear, as far as I know Ferguson has never written about this specific pictograph. I'm illustrating the kind of argument he's made about other rock art depictions of violence). And there are certainly aspects of "Veg 'n Vlug" that isn't literal. Parkington continues from the above quotation:
This bald, but reasonably literal description gives no hint of the intriguing and enigmatic details that impart a deeper, but still obscure meaning to this apparently unified composition. Take the double red lines for example. They cannot, as might appear at first glance, be footprints or a path, because they connect the feet of those in the procession to the bow of one of the cave occupants and emerge from the bowstring to enter the mouth (or face) of the bow and arrow-wielding figure. The strange figure reeling in the lines from the feet of the prone, perhaps dead, figure cannot be manipulating footprints or a path in any literal way. It is likely that the double, parallel red lines are painted to illustrate some connectedness between people that is intangible but central to the meaning of the composition. The attachments to feet, hands, equipment and mouth probably indicate the nature of the connection but are not explicit enough to provide a definitive narrative.
So what does this mean? Well, clearly, the pictograph cannot be strictly literal. Perhaps the artist(s) intended to convey some, now obscure, metaphorical meaning. Perhaps aspects of the drawing represent something abstract. Contra Ferguson et. al., though, I don't think it is reasonable to conclude that a metaphorical interpretation obviates a literal interpretation.

Take my avatar and favorite painting, A Philosopher Lecturing on the Orrery by Joseph Wright:

The painting has two, complementary, meanings. Whether Wright had in mind a specific instance of a scientist[1] demonstrating an orrery (a clockwork-driven model of the universe), it's clearly the kind of thing that went on at the time. That is, there certainly were orreries, scientists, and scientists demonstrating orreries and the painting represents an instance of the latter. The painting conveys much more than just 'such-and-such' happened, though. Many metaphorical interpretations are possible, naturally, but Wright seems to have intended it as a celebration of science, of the Enlightenment. The point could be argued, but suppose we agree A Philosopher represents the Enlightenment. Does that mean we have to abandon a literal reading? Insist that the painting tells us nothing about orreries and scientists? Obviously not. Literal and metaphorical representations can, and often do, co-exist.

What this illustrates, I'm suggesting, is that however we interpret the non-literal aspects of "Veg 'n Vlug", we need not abandon a literal reading. In other words, even if there are abstract or metaphorical meanings we can assign to the pictograph, it still depicts two groups engaging in violence. It's necessary to go a step or two further, in fact. Not only can metaphorical and literal readings co-exist, we should apply Occam's Razor and favor a kind of interpretive parsimony: the simplest interpretation - the one that requires the fewest new assumptions - is likely the correct one. And in nearly all cases, the literal interpretation is the simplest.

Whatever metaphorical or abstract readings we assign to any pictograph do not necessarily obviate literal interpretations. And interpretive parsimony - favoring the simplest possible interpretation - cautions against metaphorical readings in the first place, and demands especially strong evidence before we elevate metaphor over straightforward representation. In short, unless we have strong reasons to think otherwise, pictographs like "Veg en Vlug" represent evidence of ancient violence.

[1] The word 'philosopher' at the time had multiple meanings, one of which was what we would now call a scientist.