Engineers see references to Bayesian
Statistics everywhere. Here is a ten-minute overview of the fundamental
idea. The concept is easy - we do it every day . But there's a catch:
Sometimes the arithmetic can be nasty.
|Situation # 1:
median height of an average American Male is 5'10". (I don't know if this is accurate; that's not the point.) You are on a
business trip and are scheduled to spend the night at a nice hotel downtown.
Wanted: Estimate the probability
that the first male guest you see in the hotel lobby is over 5'10".
(Well, that's certainly self-evident.)
|Situation # 2:
On your way to the hotel you discover that the National Basketball
Player's Association is having a convention in town and the official hotel is the one
where you are to stay, and furthermore, they have reserved all the rooms but yours.
estimate the probability that the first male guest you see in the hotel lobby is over
than 50% Maybe even much more, and that's obvious too.
So what? You just applied Bayesian
updating to improve (update anyway) your prior
probability estimate to produce a posterior probability
estimate. Bayes's Theorem supplies the arithmetic to quantify this
How does Bayesian Updating Work?
The idea is simple even if the resulting arithmetic
sometimes can be scary. It's based on joint probability - the
probability of two things happening together.
Consider two events, A and B.
They can be anything. A could be the event, Man
over 5'10" for example, and B could be Plays
for the NBA The whole idea is to consider the joint
probability of both events, A and B, happening
together (a man over 5'10" who plays in the NBA), and then perform some arithmetic on
that relationship to provide a updated (posterior) estimate of a prior
Prob(A|B) is the conditional
probability of A, given B, and
Prob(A and B) is their joint probability
Prob(A|B) x Prob(B) = Prob(A
and B) = Prob(B|A) x Prob(A)
By the definition of conditional probability
Prob(A|B) = Prob(A and B) / Prob(B)
By algebraic manipulation.
Let Prob(A) = 0.5
Let Prob(B) = 0.000001
Let Prob(B|A) = 0.00000198
Probability of seeing a man
Probability of playing for the NBA
Probability of playing for the NBA, given
that you're over 5'10"
Wanted: an updated (a
posteriori) probability estimate that the first guest
seen will be over 5'10", i.e: Prob(A|B)
Prob(A and B) / Prob(B), =
Prob(A|B) = Prob(B|A)
x Prob(A) / Prob(B) =
Prob(A|B) = 0.00000198 x 0.5 / 0.000001
Bayesian updating begins
with the conditional probability, Prob(B|A) as given,
when what is desired is the other conditional orobability, Prob(A|B)
0.00000099 / 0.000001 = 0.99
of seeing a man over 5'10" given that he plays for the NBA
A Venn Diagram shows that once the universe has been narrowed to NBA
players (crosshatched area), the fraction of that universe that is taller than 5'10"
is very large.
(In fairness, a warning is in
order: This example is very simple, and real problems seldom provide the required
conditional probabilities - they must be inferred from the marginals - and real problems
are seldom binary - black or white - but consist of many possible outcomes, with only
one of primary interest.)
Suppose Event A were your analytical
predictions of some physical phenomenon, and Event B the ex
post facto physical measurements (complete with their
uncertainty). Bayesian updating could be used to improve your
analytics in light of the new experimental information. Note that this is NOT
equivalent to "dialing in a correction" between what was predicted and what was