Prabability By Average
What is the mathematical foundation of prabability by average?
How this lesson unfolds.
Begin with How do empirical averages of binary indicator variables physically build theoretical probability from raw real-world data? Use the picture, intuition and explanation together before moving on.
How do empirical averages of binary indicator variables physically build theoretical probability from raw real-world data?
You cannot directly measure the invisible 'probability' of rain; you record 1 if it rains and 0 if it doesn't across 100 days. The average of those 0s and 1s builds the probability.
- $$I_A = \begin{cases} 1 & A \text{ occurs} \\ 0 & A^c \text{ occurs} \end{cases} \implies E[I_A] = P(A), \quad \bar{I}_N \xrightarrow{a.s.} P(A)$$
- $$Theoretical probability P(A) is an abstract mathematical concept; in real-world data, it is estimated through the empirical average of binary indicator variables because E[I_A] = P(A).$$
- Learning goalProve indicator bridge E[I_A] = P(A) and connect empirical averages to probability.
- Helpful foundationsIndicator functions · Strong Law of Large Numbers · Frequentist definition · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapEmpirical probability is estimated directly via sample averages of binary indicator variables.
- Next connectionMaster the formal distinction between indicator variables and probability measures.
More context & mechanics
Probability theory distinguishes between two worlds:
- The Theoretical World (P): Probability P(A) is an axiomatic parameter defined on a measure space.
- The Empirical World (E): Real-world datasets contain only recorded observations, not probabilities.
The universal mathematical bridge between these two worlds is the Indicator Variable I_A, defined as I_A = 1 if event A occurs, and 0 otherwise. As proven by the fundamental indicator property, the mathematical expectation of an indicator equals the probability of the event: E[I_A] = P(A). When we collect N independent empirical trials and compute the sample average:
barI_N = (1/N) * sum_(i=1)^N I_A^((i)) = (Count of times A occurred) / N.
By the Strong Law of Large Numbers, this sample average converges almost surely to the true theoretical probability P(A) as N -> inf. In practice, every probability ever computed from real-world data is an empirical average of indicator variables.
Digital advertising platforms (Google Ads, Meta) estimate click-through probabilities P(Click) by computing the running empirical average of binary click indicators (I_click in 0, 1) across millions of ad impressions.
Richard von Mises established the Frequentist foundations of probability in 1919 in 'Probability, Statistics and Truth', defining probability as the limiting empirical frequency of binary indicators in an infinite collective.
'Indicator is the Bridge, Average is the Probability' — P(A) = E[I_A]. Average 0s and 1s to find the probability.