To add comments or start new threads please go to the full version of: Bayesian Reasoning?
PhysOrgForum Science, Physics and Technology Discussion Forums > General Sci-Tech Discussions > Puzzling questions

brent.tc
... Title says it all.
mr_homm
Bayesian reasoning is based on Bayes' Theorem in probability theory. Bayes theorem deals with conditional probability, which is the probability that event A will occur given that you know some other event B has occured. This probability is written as P(A|B), and the individual probabilities are P(A) and P(B). What Bayes' Theorem does is let you turn around the probability P(A|B) to find P(B|A), which is the probability that B occurs given that A has occured.

In order to know this, you need some other information. This is clearest from an example: Suppose you have 16 socks in a drawer, 4 short white socks, 3 long white socks, 7 long black socks, and 2 short black socks. (Of course they don't all match. My sock drawer is like that, isn't yours?)

Let's put this information into a table:

CODE

   |  S  L  |
---|----------|--
W |  4  3  | 7
B  |  2  7  | 9
---|----------|--
   |  6  10 |16


The row totals on the right are the total numbers of white and black socks, while the column totals are the total numbers of short and long socks. The lower right corner is the grand total. The general pattern is that you divide an entry by either its row total or its column total to get conditional probabilities, or by the grand total to get the absolute probability. For instance, 4/6 is the number of white socks among all short socks, which is P(W|S). On the other hand, 4/7 is the number of short socks among all the white socks, which is P(S|W), while 4/16 is the number of socks that are both short & white among all socks, which is P(S&W). You can also divide row or column totals by the grand total to get probabilities. For instance, 10/16 is the total number of long socks out of all socks, which is P(L).

Now, here comes Bayes' Theorem. It is really nothing more than noticing that (entry / row total)*(row total / grand total) = (entry / grand total), which is obvious because the row total cancels out of the numerator and denominator. Of course, the same thing works with column totals. When you write this out in probability notation, it says for example that P(W|L)*P(L) = P(W&L). But of course, since it works the same with row totals, it is also true that P(L|W)*P(W) = P(W&L). Since these are both equal to the same thing, you get P(L|W)*P(W) = P(W|L)*P(L),which is the usual formulation of Bayes' Theorem.

So that's what Bayes' Theorem says mathematically. The big question is, what does this mean, procedurally, for experimental science? Bayes' Theorem is crucial to finding out what we want to know, in the following way: Suppose you've done a series of experiments to measure a physical quantity, such as the charge of an electron. You would like to be able to say something like "there is a 99% chance that the true value lies between these two specific numbers a and b." But you CAN'T say that, because you don't have any information about the probability distribution of the true value; you only have information about how your experimental values are distributed. So the best you can do is to turn things around and say "If the true value is <a or >b then there is a 1% chance that my experimental values would be as I observed them to be. In other words, actual experiments always give you P(my results | truth), but what everyone wants is P(truth | my results).

Bayes' Theorem lets you convert one to the other, but ONLY if you also know P(my results) and P(truth). You can get P(my results) by running the experiments lots of times to collect data and see how the results are distributed, but you CANNOT EVER get P(truth). This would mean you had perfect knowledge of all the ways the universe could possibly be, before even doing your experiment. We're still trying to understand just this one universe, and have no handle at all on how all possible universes might look. So it appears that Bayes' Theorem is a dead end.

That's not quite true, because of the Law of Large Numbers, which says that when you have a very large amount of data and break it into smaller chunks and look at the mean value of each chunk, it turns out that these mean values ALWAYS follow the Gaussian normal distribution very closely. The law only becomes exact for infinite data, but it gives very good agreement with the normal distribution for merely large sets of data. So what you do is, you take all the previous experimenters' results (hopefully, lots of people before you have tried to measure this particular quantity), group them, and look at the mean values they got. This will give P(truth) via the Law of Large Numbers. Then you can apply Bayes' Theorem and get P(truth | my results).

Now what good is that, since you already had P(truth)? Well, what you had was a pretty good estimate of it, not the actual thing. Now that you have done your experiments and actually obtained your results, P(truth | my results) gives a NEW and BETTER estimate of the real value of P(truth). This is a kind of bootstrap process, where EVENTUALLY, after many, many people have separately measured the same quantity, you finally get a very good estimate of the actual precision of your knowledge of the measured value.

So Bayesian inference is an ongoing process in which the first person who does a measurement simply makes an arbitrary assumption about P(truth), which is called the "prior distribution" meaning the distribution you thought the value had before you did your experiments. The first person can just pull that number out of a hat, and usually does just that. It doesn't matter, however, because each person builds on the previous work, and eventually the Law of Large Numbers takes over, and it ceases to matter what the first person in the chain thought.

Bayesian inference is the fundamental reasoning process that makes empirical science work, because it shows how experimentation can gradually approach (though never reach) both precision and certainty.

Hope that helps!

--Stuart Anderson
brent.tc
Thanks much! Before I read your answer, I did a bit of Googling, and came up with this (read if you want to know more about Bayesian Reasoning+)... I can't attest to the accuracy of that article, but I have little doubt that it is inaccurate and further doubt that it would do any harm for any of you reading this to read that... ph34r.gif
mr_homm
I have just read the article you linked to. It is very good, and I CAN attest to its accuracy. It is quite accurate. If you are good at math, the explanation may be TOO long and there may be too many examples. If you are not good at math, it is probably necessary to see many examples in order to pick up on what is going on. The diagrams are something I've never seen before anywhere, and they are a good way of visualizing the ideas, so I'll probably start using something like that when I'm teaching probability to my students.

So, thanks a lot for that really good link, and good luck in your studies.

--Stuart Anderson
brent.tc
QUOTE (mr_homm+Jun 19 2008, 12:22 PM)
So, thanks a lot for that really good link, and good luck in your studies.

biggrin.gif
MjolnirPants
So how come it's suppose to be so unintuitive? I didn't have no trouble figurin out the questions they asked, an I ain't nothin even resemblin a math prodigy...

EDIT: By "they" I meant whoever wrote that page Brent linked to.
Also, I ain't seen what's so counter-intuitive about QM, either. Not that I'm a master o it by any means, an there's likely quite a bit about it that I don't know, but the basic ideas that folk like Brian Greene say are so counterintuitive jes... well, they ain't! They're jes kinda simple an elegant, the same way that the concepts in that link are. Now, I ain't tryin to come across as no kinda prodigy, mind, an I ain't tryin to make myself look good. There's plenty o folk here what could run circles around me in math or QM without so much as breakin a sweat. I jes wanna know why so many folk apparently find these concepts so counterintuitive... Can anybody help me get it?
mr_homm
Well, the whole idea of things being counterintuitive is really rather vague. What it seems to mean is that your intuition rather strongly tells you one thing, but the theory tells you that something entirely different is true. But that can only happen if you have either trained your intuition in the old mode of thinking (as classical physicists had done before QM came along) or are a stubborn cuss with an inborn certainty that whatever you happen to thing is right.

Now most people don't fall into either of these categories, and in my experience as a teacher, people have nearly zero confidence in their intuition about physics or mathematics. These subjects leave most people completely cowed. I don't mean that they are afraid of math or physics exactly, but that they completely abdicate their own point of view whenever math or physics comes along. I continually have to fight with my students to get them to assert anything at all; they are so afraid to be wrong and so sure that the theory is full of "tricks" which will catch them out, that they just passively wait for the truth to be explained.

To people with this attitude, nothing is especially counterintuitive, because they have accepted the false idea that EVERYTHING is counterintuitive in the mathematical sciences, so they shut off their brains. They typically don't have the force of confidence necessary to really pursue high level training in these fields.

At the other end of the spectrum are people who really process new ideas very thoroughly and have no trouble revising their intuition as new information becomes available. They also do not usually feel that things like QM and Bayes' Theorem are strikingly counterintuitive, if they encounter them EARLY ENOUGH in their education.

So who is it that feels these things are strongly counterintuitive, and why do they get so much press? In my opinion, this is coming from professional physicists themselves, parroted by science popularizers who know that if they call a theory "mind blowing" they'll sell more copies.

The root of the problem, I think, lies in how science is taught. Science teaching at the college level usually takes a historical approach, in which the lectures follow the historical development of the subject. Every science has a dividing line, before which it was "old school" and "quaint" and WRONG, and after which it is "new" and "modern" and RIGHT. This dividing line is usually associated with a person, who functions the same way mythological figures do, as a symbolic founder the subject. Just as Finn Mac Cool was the mythological founder of Ireland, Aeneas of Rome, and Abraham of Israel, so Euclid was the founder of mathematics, Darwin of biology, and Newton of physics. (Note: by "mythological," I do not mean non-historical; I'm referring the the psychological impact of these people when I say that they function as mythological figures.)

Everything before the founder is considered prologue, and is not treates quite seriously. Once the founder arrives on the scene, things start moving, the subject grows up and gets serious, it assumes the mantle of Truth. Teachers can't help teaching things this way, because the sciences really have advanced a lot since their beginnings, and some of the early ideas really do look rather misguided in retrospect. Students pick up on these cues, and feel that (in physics for instance) Newton was "the real deal" and everything he said and did was the truth. Again, I'm not talking about a rational assessment of Newton's life and work, which the student is probably not yet equipped to make anyway; I'm talking about the feeling that students get, the feeling that they are learning the Truth. Without specifically intending to do so, most teachers are going well beyond conveying information, and are actually INDOCTRINATING students in the Newtonian world-view.

Now this is not bad and evil; it is an automatic consequence of how people work when they hold a founding figure in reverence; it may even be necessary. After all, it was the Newtonian world view which developed the theories that in turn led to the quantum world view. So perhaps immersing the student in the earlier viewpoint is the only way to walk the student through the reasoning that leads to the new viewpoint. We only have the one timeline of human history to observe, so we can't say whether it is possible to reach the quantum view without passing through the Newtonian view first.

The common (safe) assumption is that we should make each individual pass through the process that occured historically. For a physicst, this means intensive training and (accidental) indoctrination into the Newtonian perspective, until this way of thinking is deeply internalized. Then QM is introduced, forcing a radical departure from that way of thinking. This is where the idea that QM is counterintuitive comes from, in my opinion. Only people whose intuition has first been highly trained and focused on the Newtonian picture have this experience. They they tell everyone else how mind bending it was, and since most everybody else is cowed by physics (and thought it was all EQUALLY bizarre, not just QM), this opinion is accepted; after all, it is the experts' opinion, right? But it's really not their opinion, it's a SYMPTOM of the process by which they became experts, like the dark circles under a grad student's eyes. biggrin.gif

Part of the problem is that when people make a mythological figure out of a founder, there can be only ONE. Psychologically, it makes no sense to have two founders 300 years apart, so the first one must be the REAL founder, and his word is law. Then when it turns out that he's wrong, the second founder must be the real one, and the first founder is relegated to the prologue. Once that happens, REAL physics will be considered to have started with QM, and it will no longer be thought counterintuitive. But our historical teaching method prevents this dialectical process from occuring because we continue to train each generation in the Newtonian outlook before springing QM on them.

So there, that's my opinion on why QM is considered counterintuitive. It seems rather strange, looking back at what I've written, that the answer seems to involve depth psychology, history, pedagogical norms, and the historical dialectec. But my opinions are all like that -- a huge mishmash of stuff from god knows where.

--Stuart Anderson
MjolnirPants
QUOTE (mr_homm+Jun 21 2008, 05:44 AM)
So there, that's my opinion on why QM is considered counterintuitive. It seems rather strange, looking back at what I've written, that the answer seems to involve depth psychology, history, pedagogical norms, and the historical dialectec. But my opinions are all like that -- a huge mishmash of stuff from god knows where.

--Stuart Anderson

Stuart, that was a wonderful explaination, an it makes perfect sense to me.
Ya mentioned that people don't find such concepts counterintuitive when they're exposed early enough, an ya mentioned people do find such concepts counterintuitive when they're exposed to em AFTER bein indoctrinated into prior systems o thought, an I can't think o a better explanation fer meself than that.
I was exposed to the postulates o QM an string theory young, I was in 9th grade when my science teacher took a special interest in me. (I had inadvertantly 're-discovered' tipler's cylinder an (with my teacher's help in doin all the maths involved) submitted it as a science project to a fair. I didn't win, as the judge's recognized it fer what it was, but they gave me due credit fer comin up with it on me own, an I became my teacher's favorite student.) He brought me some books on QM an Relativity, an really tried to teach me this stuff. O course, the next year I didn't have that teacher, I got stuck in biology, instead o earth/space sciences, so my 'formal' physics education ended there, with almost no understandin o the math involved in either theory, but a good grip on the qualitative functionin o them both. I actually didn't know hardly any newtonian physics until I started playin pool, a few years later. (Fer instance, I didn't know that a ball what strikes the cushion will move at the exact opposite angle at which it struck until I got into pool. As incredibly ignorant as that sounds, it's true.)

So I think what a feller could take from what yer sayin, an me own experience is that the method by which students are taught physics these days could use an overhaul. perhaps it could be taught the way the American Bujinkan Budo likes taijutsu to be taught: First the student is given an overview o the art, lastin a few months, durin which time they're taught how it's applicable in today's world, as well as techniques (dealing with opponents armed with firearms, fer instance) that weren't historically a part o it. Then, the student goes through the historical progression o the art, startin with the militant aspect (the oldest) an movin on to the espionage-type stuff, finally endin up studyin Saimenjutsu (self-hypnosis, meditation, a bit o buddhism, etc...), all the while incorporatin elements o modern day combatives, to make sure he or she stays focused on what's useful.
Perhaps physics could work similarly, an accomplish more by it. Teach students a good qualitative understandin o 'counter-intuitive' theories like QM an relativity, then get them into the regular program like we have now, but keepin them in touch with modern theories through a qualitative-only course in each theory, that they'd take right up until they start studyin that theory fer real.
dwk
It's 'counter intuitive', in part, because it allows the interchange of cause and effect, Pr( A | B ) <--> Pr( B | A )
(calculating the probability of a prior event given the certainty of a future event)

Causality tends to frame things the other way round. Physicists HATE it when you do that kind of stuff to causality!
buttershug
I hate to be morbid but would Bayesian Reasoning be at all applicable to those 5 feet that washed up on shore in BC?
Four lefts and one right I think and the right did not match with any of the lefts.
All wearing Nikes which float.
PhysOrg scientific forums are totally dedicated to science, physics, and technology. Besides topical forums such as nanotechnology, quantum physics, silicon and III-V technology, applied physics, materials, space and others, you can also join our news and publications discussions. We also provide an off-topic forum category. If you need specific help on a scientific problem or have a question related to physics or technology, visit the PhysOrg Forums. Here you’ll find experts from various fields online every day.
To quit out of "lo-fi" mode and return to the regular forums, please click here.