We engineers often ignore the distinctions between joint, marginal, and conditional probabilities - to our detriment.

*Figure 1 - How the Joint, Marginal, and Conditional
distributions are related.*

**conditional probability**:
where where
* f *
is the probability of

**Joint probability** is the probability of two or more
things happening together.
where
* f *is the probability of

Consider first the idea of a probability ** density**
or

A joint probability density two or more variables is called a
**multivariate distribution**. It is often summarized by a
vector of parameters, which may or may not be sufficient to characterize
the distribution completely. Example, the normal is summarized
(sufficiently) by a mean vector and covariance matrix.

**marginal probability**:
where
* f * is the
probability density of

Note that in general the conditional
probability of *A* given *B* is *not* the same as *B* given
*A*. The probability of both *A* and *B* together is *P(AB)*,
and if both *P(A)* and *P(B)* are non-zero this leads to a statement of
**Bayes Theorem**:

* P(A|B) = P(B|A)
x P(A) / P(B) * and

**P(B|A) = P(A|B) **
**x**** P(B) / P(A) **

Conditional probability is also the basis
for **statistical dependence** and **statistical independence**.

**Independence**: Two variables, *A* and *B*, are independent if their
conditional probability is equal to their unconditional probability. In
other words, A and B are independent if, and only if, P(A|B)=P(A), and
P(B|A)=P(B). In engineering terms, A and B are independent if knowing
something about one tells nothing about the other. This is the origin of
the familiar, but often misused, formula P(AB) = P(A) X P(B), which is
true only when A and B are independent.

**conditional independence**: * A* and

** Prob(A=a, B=b | C=c) = Prob(A=a
| C=c)
x Prob(B=b | C=c)** whenever

So the joint probability of * ABC*, when

Prob(C)