Kudos to Tom for taking a swing at the Bayes problem.
Most doctors answered that there was an 80% chance the woman had cancer.
The correct answer, however, is there is 7.8% chance that the woman actually has cancer. Tom got it right ... well sort of. He answered it correctly and then doubted the answer. Tom has told me before that he has a doubting problem though. ;)
Here is what throws people. The fact of the matter is, in our scenario, the rate of women over forty who actually have breast cancer is very low ... 1 out of 100.
Ignoring mammographies, a forty year old woman has only a one percent probability of having breast cancer. That's quite low.
A mammography gives us evidence to alter that probability, however. The positive mammography increases the likelihood that she has cancer. We need help from Reverend Bayes to determine by how much.
Turns out, the probability increases from 1 percent to nearly 7.8 percent ... a huge increase, ... nearly eight times more likely, but it still seems small to most people who try this test.
Why is the actual probability only 7.8 percent instead of 80 percent (which is what most of the errant doctors answered)?
The answer is, there are a large number of false positives relative to the number of true positives. Stated differently, the probability of a positive being a "true" positive is small relative to the probability of it being a false positive. So while the woman is now in the positive mammography group (bad news), the vast majority of woman in the positive mammography group are false positives. This is, of course, good news for her.
My nephew still does not believe me that 7.8% is right ... so if you doubt me, go read Intuitive Explanation of Bayesian Reasoning and when you are done, you can join the worldwide Bayesian conspiracy along with me ;)
Next up : why all the interest in Bayes all of a sudden?
Jeff:
It's not that I doubt the 7.8% answer -- it's that I reject that answer. What I think is happening here is that you're applying Bayes Theorem to a circumstance in which it does not apply, something Kevin points out in your last thread.
The critical design flaw is that you're conflating the at-large "all women over 40" group with the "women who have had mammograms" group; basically, you're assuming that all women over 40 have mammograms (and that a very large percentage of them had positive mammograms); while I'd like to believe the former is true, I doubt it is. Further, in order to fill in the gaps and apply Bayes Theorem, we need to know what percentage of women who have mammograms have a positive result, irrespective of whether or not they actually have breast cancer -- I suspect we could probably interpolate that number from the other two.
Instead, we get a highly inaccurate result from Bayes Theorem because we haven't given it enough information to yield an accurate one. And, as it turns out, we don't even need Bayes, since the 9.6% figure tells us everything we need to know.
If your 9.6% "false positive" statistic is correct, then 90.4% of women who have positive mammograms are likely to have breast cancer. 90.4% != 7.8% If the 7.8% number is correct, then 92.2% of positive mammograms are false. 9.6% != 92.2% See the problem?
Posted by: tgirsch | June 28, 2006 at 18:11
OK, upon further review, I need to review things further. Muddah, Faddah, kindly disregard this lettah! :)
I see where I have gone astray; namely, that I misread the 9.6% stat. It's not that 9.6% of positive mammographies are false; it's that 9.6% of women without breast cancer who have mammograms will get a positive result. That's a big (and hugely important) difference. So clearly, my 90.4% number is the result of my faulty logic.
What we're left with, if Bayes truly applies and if the statistics are correct, is that mammograms are horribly unreliable. Which sucks. Roughly 10.4% of women who have mammograms will get a positive result, even though roughly 9.6% of these don't have breast cancer. The percentage of positives which are false isn't 9.6% -- it's 92.3%!
Now that I've figured out where I went astray, I'm ready to accept the 7.8% figure.
Mea culpa. :)
Posted by: tgirsch | June 28, 2006 at 18:27
Thanks for participating, Tom.
I honestly expected no one to try ... but you stepped right up to the plate.
I should never have doubted you ... oops, there is that word again ;)
Posted by: Mr. Dawntreader | July 03, 2006 at 16:50
"why all the interest in Bayes all of a sudden?"
I could quibble on "all of a sudden", but wouldn't bother. Indeed I can understand why I wouldn't bother commenting at all, except for the fact that my experience might provide part of an answer.
I think my "Participatory Deliberation" project (http://bentrem.sycks.net/gnodal/ is a set of files I just salvaged) was inspired and energized by my having read John Willinsky on bringing publically funded research more into the public domain. Without contradicting that, I have another source of inspiration: the idea that "public knowledge" can be in-formed by public opinion reliably and credibly if we apply solid web-tech and something like Bayesian filtering / weighting.
If most everybody is wrong about something, i.e. holding false beliefs, then I wanna know how and why ... when the manure piles up it's reasonable to suspect that there might be a pony around. Or something.
Posted by: Ben Tremblay | July 05, 2006 at 13:02