Mental Model Behind Mean and Variance

Counts become relative frequencies, relative frequencies act as probabilities, expectation locates the center, and variance averages squared distance from that center. The notation changes, but the weighting idea stays the same.

For a discrete random variable, the population mean mu=E[X]=sum xP(X=x) is a probability-weighted average. Equal probabilities recover the ordinary arithmetic mean.

The datasets 1,2,3,4,5 and -97,3,3,3,103 both have mean 3, yet the second is far more dispersed. Mean answers where the center is; variance answers how far values lie from it.

E is an operator: it takes a valid expression of X and returns its probability-weighted average. It is linear, leaves constants unchanged, and can average X, X squared, or squared deviations.

E[g(X)] is shorthand for sum g(x)P(X=x). For g(X)=X it gives the mean; for g(X)=(X-mu)^2 it gives population variance.

For a dataset of size N, a value occurring f times has relative frequency f/N. These nonnegative weights sum to one and can be used as the empirical probability distribution.

Grouping identical observations replaces repeated additions by frequency weights. Dividing frequencies by N turns them into probabilities, so the arithmetic mean and mean squared deviation become expectations under the empirical distribution.

The denominator does not vanish: f/N becomes P(X=x). Thus (1/N) times a grouped sum is rewritten as a probability-weighted sum.

E[X]=mu and E[mu squared]=mu squared because mu is constant. But E[X squared] averages squared values, whereas E[X] squared squares the average; they are equal only when X has zero variability.

Deviations x-mu sum or average to zero, so they cannot measure spread directly. Absolute deviations and squared deviations both avoid cancellation; variance chooses squares for smooth optimization, useful algebra, and stronger penalties for large deviations.

For a finite population, sigma squared=(1/N)sum(x_i-mu)^2. It is nonnegative and equals zero exactly when every population value is the same. This is a population quantity, not the n-1 sample estimator.

Population variance is sigma squared=E[(X-mu)^2]=sum(x-mu)^2P(X=x). The weights describe the distribution and the squared deviations describe distance from its mean.

Expanding (X-mu)^2, using linearity, E[X]=mu, and the constant rule gives Var(X)=E[X^2]-mu^2. The identity is computationally useful but represents the same squared-distance definition.

Variance is measured in squared units. Standard deviation sigma=sqrt(variance) returns spread to the original data unit, making distance from the mean easier to interpret.

Adding c to every value shifts the mean by c but leaves variance unchanged. Multiplying values by a multiplies the mean by a, variance by a squared, and standard deviation by absolute a. Squaring also makes variance sensitive to outliers.

For any distribution with finite variance and any k>0, at least 1-1/k^2 of probability lies within k standard deviations of the mean. The guarantee is conservative but allows skewness and outliers.

For an approximately normal bell-shaped distribution, about 68%, 95%, and 99.7% of values lie within one, two, and three standard deviations of the mean. This empirical rule is not a guarantee for arbitrary skewed data.