No phrase in modern journalism is more suspicious than “studies show,” unless it’s “research reveals.” Mention “science” and the average scribbler’s eyes grow gauzy as his brain shuts down, and too many readers have the same reaction. This is especially true when the science involved is of the “social” variety: fields like social psychology, sociology, social anthropology, and all their bastard children. Entire mountains of baloney would come tumbling down if reporters applied an ounce of skepticism to the research findings shoveled at them by eager university publicists and happily repeated in print or on the air as settled fact.
Fortunately we don’t have to wait for our journalists to apply the skepticism. In a long-overdue act of self-policing, a brave band of social scientists are doing the job themselves. They know what many journalists—including those who routinely cover the sciences—evidently do not: You can’t really have confidence in the results of an experiment until the results are replicated, preferably by other scientists attempting the same or a similar study. This can take time, but deadlines are deadlines.
Beginning in 2011, under the guidance of the Center for Open Science in Virginia, more than 250 researchers have been trying to reproduce the findings of 100 studies published in journals of social psychology. The journal Nature reported last month that the results, while still preliminary, confirm the skepticism. Of the 100 studies, only 39 could be repeated with similar results. And of those 39 attempts at replication, only 4 produced results that were “virtually identical” to the originals.
Social psychology suffers what one researcher calls a “replicability crisis.” For young researchers all the incentives point away from careful work. A social psychologist without published papers is unlikely to find a job, yet journals and universities like to publicize “counterintuitive” findings and see no value in research that produces inconclusive results. And of course at the other end of the pipeline are reporters and editors desperate to declare on their website or morning show that science has established some new and surprising truth of human nature.
In real life, as distinct from life as it’s simulated in a university psych lab, new truths about human nature are hard to come by. (The old ones can be pretty rough, too.) It’s a great advance in social science that research at last reveals—indeed, that studies show—this to be true.