Thursday, October 1, 2009

Bayes' Rule and Dick Cheney's One Percent Doctrine

I was teaching Bayes' Rule this week, and there's an example that has to do with lie detectors and stealing. The story is that you have employees, of whom 10% are stealing. You also have a lie detector that is 80% effective (i.e. P(positive test given stealing) is 0.8 and P(positive test given not stealing) is 0.2, etc.). So, if you apply Bayes' Rule, you learn that given a positive test (i.e. the lie detector says you are lying/stealing) is just 0.308!
So, what about Cheney's "One Percent Doctrine" for justifying torture? Even worse! Let's suppose that there is a 1% chance that a detainee has information about a terror attack. Now suppose that torture is 90% effective in the sense that if that detainee is conspiring with terrorists he "confesses" and gives information about it with probability 0.9, but also confesses (with whatever made-up details he can think of) with probability 0.1 if he is innocent. (This, I think, is being generous to the effectiveness of torture!) Then, GIVEN a confession, the probability that there was really a plot is just 0.009/(0.009+0.099) = 0.083 - less than 10%!
The conclusion here is that the one percent doctrine logically fails when there is even a small chance of getting false positives from your intelligence-gathering resources and methods. It seems from what you hear that torture leads to sufficiently many false positives that it probably isn't helping, and is probably HURTING our security by leading our enforcement agencies on wild goose chases! Food for thought!

No comments:

Post a Comment