Why We Add Expectations but Multiply Probabilities

When you flip a biased coin with P(Heads) = 0.7, the coin lands either Heads (1) or Tails (0). It never lands on '0.7 Heads'. The probability p ∈ [0, 1] exists solely in the mathematical model prior to the experiment. Once realized, the outcome ω collapses into an indicator state X(ω) ∈ 0, 1. Conflating state with propensity is the root of many probabilistic fallacies.

In machine learning binary classification (e.g. email spam filtering), a logistic regression model outputs a score p̂ = 0.85. However, the incoming email is either spam (y=1) or ham (y=0); evaluating log-loss -(ylog p̂ + (1-y)log(1-p̂)) penalizes the divergence between the true binary state and the probability prediction.

Jacob Bernoulli spent twenty years resolving this distinction in Ars Conjectandi (1713), establishing that while individual events are unpredictable binary realities, their long-run frequencies converge deterministically to p.

'Reality is 0 or 1; Probability is in your Head': You can hold 0 apples or 1 apple; p=0.7 only describes your expectation before opening the box.

Suppose n=5 independent trials are conducted. Instead of treating the count X as an opaque black box, define an indicator Xᵢ for each trial: Xᵢ = 1 if trial i succeeds, and Xᵢ = 0 if it fails. The total count is simply X = X₁ + X₂ + X₃ + X₄ + X₅. This indicator decomposition is the single most powerful algebraic tool in discrete probability, transforming combinatorial counting into linear algebra.

In distributed database monitoring, system architects track node health across an n-node cluster. Let Xᵢ = 1 if node i is responsive and 0 if dead. The number of operational nodes is X = ∑ Xᵢ, enabling instant consensus threshold verification (X > n/2).

Pierre-Simon Laplace systematically employed indicator functions in his 1812 Théorie analytique des probabilités, showing that decomposing complex events into sums of binary states radically simplifies global probability computations.

'The Turnstile Rule': Each success clicks the turnstile (+1); each failure walks past without clicking (+0). Total count is just the sum of clicks.

For an indicator variable Xᵢ, E[Xᵢ] = 1(p) + 0(1-p) = p. Although Xᵢ can only ever be 0 or 1, its expected value is the continuous probability parameter p. When repeated over N independent runs, the empirical mean X̄_N = 1/N∑ Xᵢ^((k)) converges almost surely to p. Expectation is the fulcrum (center of mass) of the probability mass distribution.

In online ad auctions (e.g. Google AdSense / Meta Ads), the expected click-through rate (eCTR) for an impression might be 0.015. A user either clicks (1) or ignores (0), but billing algorithms multiply bid by eCTR to rank auctions on expected value.

Christiaan Huygens introduced the concept of mathematical expectation in 1657 in De Ratiociniis in Ludo Aleae ('On Reasoning in Games of Chance'), formalizing the 'fair price' of an uncertain gamble as its expected payout.

'The Seesaw Fulcrum': A seesaw with a 0 at one end and a 1 at the other balances at p. No brick sits at the fulcrum, but that is where the weight balances.

By definition, E[X₁ + X₂] = ∑_(ω) (X₁(ω) + X₂(ω)) P(ω). Splitting the sum gives ∑ X₁(ω) P(ω) + ∑ X₂(ω) P(ω) = E[X₁] + E[X₂]. At no point in this proof is independence invoked. For n Bernoulli trials with identical marginal success probability p, E[X] = n p, whether the trials are coin flips, dependent card draws, or weather cycles.

In randomized algorithms (e.g. finding the expected number of hash collisions in a hash table with linked lists), item insertions are often interdependent, yet computer scientists compute exact expected chain lengths effortlessly using linearity of expectation.

Andrey Kolmogorov solidified expectation as the Lebesgue integral ∫_Ω X dP in 1933. The linearity of integration immediately guaranteed that expectations add unconditionally across all measure spaces.

'Addition Needs No Permission': Expectations add freely anytime, anywhere, with or without independence. No strings attached!

To realize a specific joint outcome A ∩ B, reality must first pass through checkpoint A (with probability P(A)). Then, inside the surviving universe where A has occurred, reality must pass through checkpoint B (with conditional probability P(B|A)). Hence, P(A ∩ B) = P(A)P(B|A). If the events are independent, P(B|A) = P(B), reducing the formula to P(A)P(B).

In Natural Language Processing (speech recognition and auto-complete LLMs), the probability of a sentence W = (w₁, w₂, ṡ, wₘ) is computed via the chain rule: P(W) = ∏_(t=1)^m P(w_t | w₁, ṡ, w_(t-1)), which formulates the autoregressive loss function of modern Transformers.

Abraham de Moivre first formulated the general multiplication rule for dependent events in the 1718 edition of The Doctrine of Chances, setting the standard for sequential probabilistic reasoning.

'The Tollbooth Chain': To drive from start to finish, you must pay toll #1 (P(A)), then pay toll #2 conditioned on having reached it (P(B|A)). You multiply each gate's passage rate.

Addition and multiplication serve distinct topological roles in probability:

• Multiply along branches (AND): As you move forward through sequential time or nested conditions along a single branch of a tree, multiply the conditional likelihoods.
• Add across branches (OR): When alternative, mutually exclusive paths achieve the goal, sum their branch probabilities.
• Add across components (Expectation): To find the expected value of an aggregate total, simply sum the expected values of its sub-components.

In reliability engineering for cloud microservices with redundancy: to calculate the chance that both Primary AND Secondary servers fail, multiply their failure probabilities; to calculate the total expected hours of downtime across 10 independent servers, add their expected downtimes.

In 1654, Blaise Pascal and Pierre de Fermat established these twin rules while solving the Chevalier de Méré's dice paradoxes, distinguishing between compound rolls (multiplication) and cumulative wins (addition).

'AND = Multiply along the wire; OR = Add across the wires; EXPECTATION = Always Add!'.