That’s not being perverse. It’s about pausing when “things seem too good to be true” and taking time out to ask if “there has probably been a mistake”. (Gunn, @ 2 mins)
[U]ntil the Romans ultimately removed the right of the Sanhedrin to confer death sentences, a defendant unanimously condemned by the judges would be acquitted [14, Sanhedrin 17a], the Talmud stating ‘If the Sanhedrin unanimously find guilty, he is acquitted. Why? — Because we have learned by tradition that sentence must be postponed till the morrow in hope of finding new points in favour of the defence’.
That practice could be interpreted as the Jewish judges being intuitively aware that suspicions about the process should be raised if the final result appears too perfect . . .
[I]f too many judges agree, the system has failed and should not be considered reliable. (Gunn et al 2016)
Or even more simply,
They intuitively reasoned that when something seems too good to be true, most likely a mistake was made. (Zyga, 2016)
The opening quotation above is from a footnote to a chapter by Gregory Doudna in a newly published volume in honour of Thomas L. Thompson, Biblical Narratives, Archaeology & Historicity: Essays in Honour of Thomas L. Thompson. Doudna’s footnote continues:
I thought of what I have come to call Thompson’s Rule when I encountered this scientific study showing that, as counterintuitive as it sounds, unanimous agreement actually does reduce confidence of correctness in conclusions in a wide variety of disciplines (Gunn et al. 2016).
The paper by Gunn and others is Too good to be true: when overwhelming evidence fails to convince. The argument of the paper (with my bolding in all quotations):
Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith; however, rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence and (iii) cryptographic primality testing. In this paper, we investigate the effects of small error rates in a set of measurements or observations. We find that even with very low systemic failure rates, high confidence is surprisingly difficult to achieve . . . .
Sometimes as we find more and more agreement we can begin to lose confidence in those results. Gunn begins with a simple example in a presentation he gave in 2016 (link is to youtube video). Here is the key slide:
With a noisy voltmeter attempting to measure a very small voltage (nanovoltage) one would expect some variation in each attempted measurement. Without the variation, we can conclude something is wrong rather than that we have a precise measurement.
Another example:
The recent Volkswagen scandal is a good example. The company fraudulently programmed a computer chip to run the engine in a mode that minimized diesel fuel emissions during emission tests. But in reality, the emissions did not meet standards when the cars were running on the road. The low emissions were too consistent and ‘too good to be true.’ The emissions team that outed Volkswagen initially got suspicious when they found that emissions were almost at the same level whether a car was new or five years old! The consistency betrayed the systemic bias introduced by the nefarious computer chip. (Zyga 2016)
Then there was the Phantom of Heilbronn or the serial killer “Woman Without a Face“. Police spent eight to fifteen years searching for a woman whom DNA connected to 40 crime scenes (murders to burglaries) in France, Germany and Austria. Her DNA was identified at six murder scenes. A three million euro reward was offered. It turned out that the swabs used to collect the DNA from the crime scenes had been inadvertently contaminated at their production point by the same woman.
Consider, also, election results. What do we normally suspect when we hear of a dictator receiving over 90% of the vote?
We have all encountered someone who has argued that “all the evidence” supports their new pet hypothesis to explain, say, Christianity’s origins. I have never been able to persuade them, as far as I know, that reading “all the evidence” with a bias they either cannot see or think is entirely valid.
Ironically, scholars like Bart Ehrman who attempt to deny a historical and even slightly significant “Jesus myth” view among scholars are doing their case a disservice. By insisting that there is and that there has been no valid or reasonable contrary view ever raised, such scholars are undermining confidence in the case for the historicity of Jesus. If they could accept the challenges from serious thinkers over the past near two centuries, and acknowledge the ideological pressure inherent in “biblical studies” for academics to conform within certain parameters of orthodox faith, then they could begin to not look quite so like those politicians who claim 90% of the vote, or like those police chasing a phantom woman serial killer for eight years across Europe, of the dishonest VW executives . . . . Continue reading ““When everyone is agreed on something, it is probably wrong” — Thompson’s Rule”