Leave a comment

Why too much evidence can be a bad thing










Under ancient Jewish law, if a suspect on trial was unanimously found guilty by all judges, then the suspect was acquitted. This reasoning sounds counterintuitive, but the legislators of the time had noticed that unanimous agreement often indicates the presence of systemic error in the judicial process, even if the exact nature of the error is yet to be discovered. They intuitively reasoned that when something seems too good to be true, most likely a mistake was made.

In a new paper to be published in The Proceedings of The Royal Society A, a team of researchers, Lachlan J. Gunn, et al., from Australia and France has further investigated this idea, which they call the “paradox of unanimity.”

The researchers demonstrated the paradox in the case of a modern-day police line-up, in which witnesses try to identify the suspect out of a line-up of several people. The researchers showed that, as the group of unanimously agreeing witnesses increases, the chance of them being correct decreases until it is no better than a random guess.

In police line-ups, the systemic error may be any kind of bias, such as how the line-up is presented to the witnesses or a personal bias held by the witnesses themselves. Importantly, the researchers showed that even a tiny bit of bias can have a very large impact on the results overall. Specifically, they show that when only 1% of the line-ups exhibit a bias toward a particular suspect, the probability that the witnesses are correct begins to decrease after only three unanimous identifications. Counterintuitively, if one of the many witnesses were to identify a different suspect, then the probability that the other witnesses were correct would substantially increase.

so…jews argue so much there’s something WRONG if we agree?

When someone’s life or liberty is on the line, then, yes, that is exactly the logic behind this, and that’s it’s better to err on the side of caution rather than condemn an innocent.

You may find this interesting – in any death penalty case (which I’ve explained elsewhere how difficult it is to get the death penalty) there was a panel of 23 judges. The Talmud in Tractate Sanhedrin explains that these 23 are divided into 3 groups. A group of 10 whose responsibility was to find legal loopholes or clear evidence, or faults in the testimony, in order to try and prove the defendant innocent, a group of 10 who would argue against the defendant, acting as a prosecutor, and a group of 3 to make the judgement and decide the ruling. The GOAL was to avoid the death penalty – at almost all costs – and to MAKE argument, because in judaism, when there is argument and debate, we are always more likely to come to the truth.

I saw this, and almost posted it.  The paper is basically an academic paper saying “Yeah, that thing Jews have been doing for 3000 years? They nailed it.  If we all agree on something, or if all our data is the same massive pile, then…well, maybe there’s something wrong with our detectors, there’s too much noise in reality for all our actual measurements to agree.”


Here it is! That rare thing: a statistics paper that cites the Babylonian Talmud.

Yay! Thank you!

this is a great example of the value of historical comparison in social research tbh

It also is incredibly relevant to science because – and I argue this so much my ears bleed- science isn’t about proving things. It’s about disproving. You make a hypothesis and you try to disprove it. Then you refine and refine until no matter what you do, you can’t disprove it. Then you put that up as a Theory until you can find knew information or technology to allow you to disprove it and refine again. This is why science is always undoing things. And I know that makes people skeptical of science, but god damn it people, that’s the only way to gain knowledge. You’re supposed to be skeptical of the results, but the methods are self-correcting. 

Don’t be scared of the method.

Why too much evidence can be a bad thing

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: