Engineers see references to Bayesian
Statistics everywhere. Here is a ten-minute overview of the fundamental
idea. The concept is easy - we do it every day . But there's a catch:
Sometimes the arithmetic can be nasty.
| Situation # 1: Given: The
median height of an average American Male is 5'10". (I don't know if this is accurate; that's not the point.) You are on a
business trip and are scheduled to spend the night at a nice hotel downtown.
Wanted: Estimate the probability
that the first male guest you see in the hotel lobby is over 5'10".
Solution: 50%
(Well, that's certainly self-evident.) |
| Situation # 2: On your way to the hotel you discover that the National Basketball
Player's Association is having a convention in town and the official hotel is the one
where you are to stay, and furthermore, they have reserved all the rooms but yours.
Wanted: Now,
estimate the probability that the first male guest you see in the hotel lobby is over
5'10".
Solution: More
than 50% Maybe even much more, and that's obvious too. |
So what? You just applied Bayesian
updating to improve (update anyway) your prior
probability estimate to produce a posterior probability
estimate. Bayes's Theorem supplies the arithmetic to quantify this
qualitative idea.
How does Bayesian Updating Work?
The idea is simple even if the resulting arithmetic
sometimes can be scary. It's based on joint probability - the
probability of two things happening together.
Consider two events, A and B.
They can be anything. A could be the event, Man
over 5'10" for example, and B could be Plays
for the NBA The whole idea is to consider the joint
probability of both events, A and B, happening
together (a man over 5'10" who plays in the NBA), and then perform some arithmetic on
that relationship to provide a updated (posterior) estimate of a prior
probability statement.
IF:
Prob(A|B) is the conditional
probability of A, given B, and
Prob(A and B) is their joint probability |
Some definitions |
|
|
THEN:
Prob(A|B) x Prob(B) = Prob(A
and B) = Prob(B|A) x Prob(A) |
By the definition of conditional probability |
|
|
So that:
Prob(A|B) = Prob(A and B) / Prob(B)
|
By algebraic manipulation. |
| Numerical Example: Let Prob(A) = 0.5
Let Prob(B) = 0.000001
Let Prob(B|A) = 0.00000198 |
Probability of seeing a man
over 5'10"
Probability of playing for the NBA
Probability of playing for the NBA, given
that you're over 5'10" |
Wanted: an updated (a
posteriori) probability estimate that the first guest
seen will be over 5'10", i.e: Prob(A|B)
| Prob(A|B) =
Prob(A and B) / Prob(B), = Prob(A|B) = Prob(B|A)
x Prob(A) / Prob(B) =
Prob(A|B) = 0.00000198 x 0.5 / 0.000001
=
|
Bayesian updating begins
with the conditional probability, Prob(B|A) as given,
when what is desired is the other conditional orobability, Prob(A|B) |
| Prob(A|B) =
0.00000099 / 0.000001 = 0.99 |
Updated probability
of seeing a man over 5'10" given that he plays for the NBA |
|
A Venn Diagram shows that once the universe has been narrowed to NBA
players (crosshatched area), the fraction of that universe that is taller than 5'10"
is very large.
(In fairness, a warning is in
order: This example is very simple, and real problems seldom provide the required
conditional probabilities - they must be inferred from the marginals - and real problems
are seldom are binary - black or white - but consist of many possible outcomes, with only
one of primary interest.)
So What?
Suppose Event A were your analytical
predictions of some physical phenomenon, and Event B the ex
post facto physical measurements (complete with their
uncertainty). Bayesian updating could be used to improve your
analytics in light of the new experimental information. Note that this is NOT
equivalent to "dialing in a correction" between what was predicted and what was
measured.
|
