Psychology is still WEIRD
Psychological science is fraught with problems. One of these problems that has recently attracted widespread attention is the proliferation of false positives, which is rooted in a combination of QRPs (questionable research practices), including “p-hacking” (choosing analytical options on the basis of whether they render significant results) and “HARKing” (hypothesizing after the results are known), and very low statistical power (i.e., too few participants). Overall, psychology has responded vigorously to this problem, although much remains to be done. Numerous reforms have been put in place to encourage open science practices and quality in research.
Another problem that has become widely recognized recently is that psychological research often makes inferences about human beings in general based on studies of a thin slice of humanity. As Henrich, Heine, & Norenzayan (2010) noted in a landmark paper, participants in psychological research are usually drawn from populations that are WEIRD (Western, Educated, Industrialized, Rich, Democratic), which are far from representative of mankind—in fact, they turn out to frequently be rather eccentric, even when it comes to basic cognitive, evolutionary, and social phenomena such as cooperation, reasoning styles, and visual perception (see also this interesting preprint by Schultz, Bahrami-Rad, Beauchamp, & Henrich that very thoroughly discusses the historical origins of WEIRD psychology).
The paper by Henrich and colleagues has racked up almost 5000 Google Scholar citations. Yet a recent paper by Rad, Martingano, and Ginges (2018) suggests that the impact of the Henrich et al. paper on actual research practices in psychology has been minimal, at least as indexed by research published in the high-prestige journal Psychological Science. Rad et al. find that researchers persist in relying on WEIRD samples and show little awareness of the WEIRD problem: “Perhaps the most disturbing aspect of our analysis was the lack of information given about the WEIRDness of samples, and the lack of consideration given to issues of cultural diversity in bounding the conclusions” (p. 11402).
Explaining the persistence of the WEIRD problem
How can it be that psychology has responded so vigorously to the problem with false positives, yet so inadequately to the WEIRD problem? Surely both problems are equally serious, are they not? I can think of at least three possible explanations.
1. First and foremost, the WEIRD problem is a manifestation of a much broader problem. It is a manifestation of the lasting influence of the marriage between logical positivism and behaviorism that shaped psychology for almost half a century. Psychological research was supposed to yield universal facts, just like physics, by employing “neutral”, culture-free materials and methods, a quantitative methodology, and hard-core empiricism. Given the vast historical impact of this ideal, it is no mystery that psychology remains both WEIRD and theoretically unsophisticated. This is simply the implicit paradigm under which psychology has operated for more than a century. While the problem with false positives is a problem signaling a crisis within this paradigm, the WEIRD problem is a meta-problem with the paradigm itself.
2. Second, it is possible that researchers do not realize the severity of the WEIRD problem because they are immersed in a homogeneous community of like-minded individuals with similar concerns, and their exposure to other intellectual cultures is limited. Here it is important to note that the WEIRD problem is not limited to participant selection. It is a problem of testing WEIRD theories on WEIRD samples with WEIRD methods. I personally often find psychological theories and concepts US-centric (e.g., the reification of “liberals” and “conservatives” in political psychology or the pre-occupation with the self and neglect of other aspects of the person’s worldview in personality psychology)—which is not surprising given that most of the leading researchers in psychology are from the United States—and I still live in the broader Western cultural sphere.
3. A third possible explanation for the persistence of the WEIRD problem is that there are many practical difficulties involved in conducting research in non-WEIRD contexts. A lot of things could go wrong. You need high-quality translations of research materials. You also need to obtain a reasonable degree of measurement invariance across languages and populations to be able to make meaningful comparisons between them. Even so, the results may not be at all what you expected. Perhaps the theories and instruments do not perform as they are supposed to do. Of course, on a purely scientific basis such findings would be extremely important. But perhaps researchers still find it is easier to just stick to studying well-known populations under well-known conditions in order to more easily find support for their hypotheses and publish their work.
The WEIRD-problem needs to attain the same status as the false positives-problem in psychology. As Rad, Martingano, and Ginges (2018) suggest, authors need to do a much better job reporting sample characteristics, explicitly tying findings to populations, justifying the sampled population, discussing the generalizability of the findings, and investigating existing diversity in their samples. Journals and funders need to start encouraging these practices. Given all the work involved in conducting non-WEIRD research and the fierce competition over research funding and space in high-impact journals, we are unlikely to see any real change unless the inclusion of non-WEIRD research will give extra points.
When it comes to the problem with WEIRD perspectives, psychology might need to become more open to scholarship born out of non-WEIRD (particularly non-US) contexts. An increased openness to philosophical, meta-theoretical, historical, and anthropological scholarship in general, which is for the most part completely ignored in psychological science today, would be particularly helpful. That would help us both to address the WEIRD-problem and to make psychology a more theoretically sophisticated science.
Could it be that all of this comes from institutionalization and the continual production demands as “proof” that the scientists and philosophers are doing their jobs? It seems like not so long ago scientists and philosophers were quite un-tethered and able to develop their thoughts in a rambling and free-formed process where the only demands came from curiosity (weird thoughts? :-)) stemming from and driven within their own selves. Many appear to have been born in affluent families or families who reluctantly afforded them lots of time for their musings or others who were content in taking on some low-paying and low-demands job which also gave them that free time. Now with the democratization and institutionalization of invention and thought, we seem to have come up with these “standards” to keep a check on you folks. All these checks and balances and formalized methodologies seem like they should lead to better results and cull out the lazy, non-creative types. But is it all really better? It appears that people worry more about getting published or producing things that will sell as opposed to connecting to some internal drive to discover and to feel free to explore the weird notions and ideas in their own minds. Instead maybe they get relegated to some robotic in-step mode to come up with weird topics that are suitably populist and amenably socialist. Maybe this is why also instead of flying cars, we get 140 characters and p-hacking instead of soul searching. There is no longer point in being controversial unless you can get a cult following and generate some income out of it. Instead of weird topics, weird methods, and weird participants, where are all the truly weird scientists and philosophers? And why can’t we set them a bit (a lot) freer. Maybe that will organically help solve some these problems. There’s got to be -some- truly real shit in all that bull shit.