Prior probability

The fallacy of “conversion”

Sometimes a DNA analyst wants to “convert” the likelihood ratio LR supporting some hypothesis H into the posterior (i.e. bottom-line) probability of the hypothesis Pr(H | DNA). That is a common mistake.

notation	meaning
H	hypothesis to be established
LR	likelihood ratio supporting H
Pr(H)	prior probability; probability of H without considering the DNA
Pr(H | DNA)	posterior probability; probability of H taking the DNA into account

For example the hypothesis H that the body X is the person missing from the Smith family is supported by LR=100; the analyst wrongly writes a report that X is 99% to be the missing person — Pr(H | DNA)=99%. That’s wrong because obtaining the conclusion Pr(H)=99% from LR=100 is not just a matter of “converting”. We can convert between Celsius and Fahrenheit; 0°C and 32°F mean the same thing. But LR=100 and Pr(H | DNA)=99% are not the same. The latter includes an additional assumption, an assumption about the value of the prior probability that X is the missing Smith. (In particular if LR=100 then Pr(H | DNA)=99% if and only if Pr(H)=50%.) But when armed with both LR and prior probability, one can derive the posterior probability by applying Bayes’ theorem. (Look it up.)

Evaluating the prior probability

Sometimes I am asked “How does one figure out what number to choose for prior probability?” The answer is that, as a DNA analyst, you don’t.

1. The prior probability is subjective, hence not within the expertise of the DNA analyst.
2. Even more pertinently, the prior probability depends on — is a summary of — the circumstances that are not within the knowledge of the DNA analyst — such as
• how the body turned up — date, location compared to where last seen
• how much it resembles the missing person
• what other possibilities are there for the identity of the body
• all sorts of circumstances

We are not qualified to evaluate subjective "data" and more importantly we are not qualified to judge information that we do not have. Evaluation of non-DNA data, which I typically describe as "subjective" (but it doesn't have to be and to be honest there's some subjectivity about interpreting DNA), is the purview of the court, or sometimes of a lawyer or concerned or curious customer. As specialists in the science and mathematics of DNA we don't have any special qualification to evaluate other kinds of evidence, and in fact we do not even have the other evidence. It would be very wrong and ridiculous for a DNA analyst to testify in court that “Considering both the DNA evidence and also the story that the prosecutor and one of the witnesses told me about the circumstances, it is extremely probable that the prosecutor is correct!”

Mass disaster

However, practically speaking, in a humanitarian context such as mass disaster identification (as opposed to criminal/judicial) the DNA analyst is likely to be asked to wear two hats.

Report language — finessing the choice of a prior probability

Consequently it is wrong for the DNA expert to assume a 50% prior probability or any other prior. However, it is honest to write a report that says

1. The LR=10000 that the body is from the X family.
2. The probability that the body is from the X family depends as well on what prior probability (probability from non-DNA facts and circumstances) is assumed.
3. Assuming a prior of 50%, the body is 99.99% to be from the X family.
4. We can calculate the effect of a different prior probability assumption if requested.
Comments:
• With just 1 & 3, the report is still literally truthful but likely to be misunderstood, as the reader may not realize that
  “Assuming ...” means
  “We assume nothing, but if you (the reader) choose to assume ...”.
• Points 2 and/or 4 are supposed to make it clear that prior probability is not the laboratory’s responsibility or competence.