Bayes Theorem

What is it?

The Bayes Theorem is an important rule in Probability for calculating conditional probability of an event given previous information about this event. Thomas Bayes described how to update the probability of an hypothesis $P(H)$ , given new evidence $P(E)$ about the same hypothesis, which can be written as:

$$P(H|E)=\frac{P(E|H)\ast P(H)}{P(E)}$$

where:

• $P(H|E)$ = posterior probability - the probability of $H$ given the evidence $E$.
• $P(E|H)$ = likelihood - the probability of observing the evidence $E$ given $H$ is true.
• $P(H)$ = prior probability - initial belief/ probability of $H$ before seeing new evidence.
• $P(E)$ = marginal probability - the probability of $E$ under all possible hypothesis.

---

Multiple evidences

One can also update the probability of a hypothesis given multiple new evidences. Given the exclusive events $H_1,H_2,\dots ,H_m$ are disjoint and $H_1\cup H_2\cup \cdots \cup H_m=\Omega$ (all outcomes in a sample space), the conditional probability of $H_i$ given an arbitrary event (or evidence) $E$, can be written as:

$$P(H_i|A)=\frac{P(E|H_i)\ast P(H_i)}{P(E|H_1)P(H_1)+P(E|H_2)P(H_2)+\cdots +P(E|H_m)P(H_m)}$$

This is also the traditional form of the Bayes’ formula. It combines the Law of Total Probability applied to the denominator $P(E)$ of the single evidence form. For shortcuts, one can use the highest $P(E|H_i)$ to determine $P(H_i|E)$, even if it’s not academically rigorous.