DNA and the Prosecutor's Fallacy

DNA fingerprint evidence can close a case. If you have found a DNA match the result is certain ... or is it? Why did the Society for Expert Witnesses call in Queen Mary Computer Scientist, Professor Norman Fenton? He made a special video appearance at Studley Castle, where their conference was being held, to explain clearly what you can deduce from such "fingerprint" evidence. Why? Because lawyers often get it wrong even in high profile cases, claiming cast iron cases when they have none at all. Turns out tools developed for Software Engineers help (oh and the same techniques can help you predict the results of Spurs Matches!) Read on.

The Society was keen to get a proper explanation of the so-called "Prosecutor's fallacy" whereby lawyers make incorrect deductions about probabilities that are known to have a profound impact on juries. The Society's Press Officer Tom Magnum had sought out a number of statisticians and probability experts but was unable to get a simple explanation that he felt could be understood by lawyers and judges. He turned to Norman Fenton after reading an article Norman had written with Martin Neil about the subject of probability fallacies in legal reasoning.

Norman explained the prosecutor's fallacy as follows:

"Suppose a crime has been committed and that the criminal has left some physical evidence, such as some of their blood at the scene. Suppose the blood type is such that only 1 in every 1000 people has the matching type. A suspect X who matches the blood type is put on trial. The prosecutor claims that the probability that an innocent person has the matching blood type is 0.1% (1 in a 1000). Person X has the matching blood type and therefore the probability that X is innocent is just 0.1%.

But the prosecutor's assertion, which sounds convincing and could easily sway a jury, is wrong. And it is easy to see why. Let's suppose that the crime could only have been committed by an adult male and that in the population there are 10 million adult males. Then from this population we would actually expect a very large number of people who have the matching blood type (about 10,000). If there is no evidence other than the blood to link X to the crime then X is no more likely than any of the other 9999 matching blood type men to have committed the crime. This means that the probability X is innocent is actually 99.99% which is rather different to the 0.1% claimed by the prosecution.

So what is the source of the fallacy and why do lawyers so commonly make it? It all boils down to a basic misunderstanding about probability (a misunderstanding which many intelligent people have because this kind of basic probability is never taught at schools). The misunderstanding is to assume that the probability of A given that we know B is true (written P(A|B)) is the same as the probability of B given that we know A is true (written P(B|A)) . In this case let A be the assertion "Person X is innocent" and let B be the assertion "person X has the matching blood type". What we are really want to know is P(A|B) (the probability of innocence given the evidence of matching blood type) and this is what the lawyer claims is equal to 0.1%. But in fact, what we actually know is that P(B|A) (the probability of the matching blood type given innocence) is equal to 0.1%. The lawyer has simply stated the probability P(B|A) and claimed this is actually the probability P(A|B).

The fallacy becomes especially challenging when DNA evidence is used. In such cases P(B|A) can be extremely low, such as 1 in 10 million. When the lawyer wrongly asserts that the probability of innocence is therefore 1 in 10 million it seems especially convincing. But even in this case the probability of innocence is actually very high. Assuming again a population of 10 million people who could have committed the crime. Then it turns out that the probability of more than one person having the matching DNA is still actually quite high - about 0.46. So instead of the claimed 1 in 10 million probability of innocence the real probability is not much less than 1 in 2. In such circumstances the 'beyond reasonable doubt' criteria can hardly be claimed to be met."

So if you now understand this different kind of probability, the kind of probability software engineers use to assess risks, then you should recognize when the Prosecutor is getting it wrong. Where do the Spurs matches come in? Is that a whole new ballgame?