Back in November, Cornell psychologist Daniel Bem published a study which appeared to show that extra-sensory perception (ESP) was a real phenomenon.
Implausible as it might seem, the results reached the threshold for statistical significance.
But there were always going to be plenty of challengers to the research, and now two statisticians say they’ve got a better way of running the math.
Jeffrey Rouder and Richard Morey from the University of Missouri have applied a relatively new statistical method that quantifies how beliefs should change in light of data. And they conclude that Bem’s findings shouldn’t be enough to sway the beliefs of a skeptic.
They claim that conventional statistical significance testing – p values – are limited in usefulness, and use a new technique called meta-analytical Bayes factor instead.
They say that in order to accurately assess the total evidence in Bem’s data, it’s necessary to combine the evidence across several of his experiments, not look at each one in isolation, as previously.
Using these methods they do, in fact, find that there’s some evidence for ESP – indeed, they say, people should be 40 times more likely to believe in ESP as a result of Bem’s research.
Beliefs are odds, they say. For example, a skeptic might hold odds that ESP is a long shot at a million-to-one, while a believer might believe it is as possible as not, giving odds of one-to-one.
In either case, Rouder and Morey say that Bem’s experiments indicate that these odds should change by a factor of 40 in favor of ESP. Thus the believer should now be 40-to-1 sure of ESP, while the skeptic should be 25,000-to-1 sure against it.
Despite this, Rouder and Morey conclude that ESP doesn’t exist.
“We remain unconvinced of the viability of ESP. There is no plausible mechanism for it, and it seems contradicted by well-substantiated theories in both physics and biology,” they say. “Against this background, a change in odds of 40 is negligible.”