Every formula below is taken apart the same way: symbols, meaning, where it comes from, when it applies, how it fails.
The point, before any formula
Test statistica difference, over \(\sqrt{\text{variance}/n}\)
Chebyshevthe variance inequality
Bayesthe multiplication rule, with the denominator expanded
The diagram above is the point to make before any individual formula: almost nothing in this book is free-standing. The test statistic is a variance divided by \(n\), square-rooted, and put under a difference. Chebyshev is the variance inequality. Bayes is the multiplication rule with the denominator expanded. If you learn formulas as isolated strings you’ll reproduce them wrongly under pressure; if you know what each one is made of, you can rebuild it from the pieces. So below, every formula gets the same five-part treatment: symbols, meaning, where it comes from, when it applies, how it fails.
Notation decoder
Learn this first — half the errors in the book’s answer keys come from misreading symbols.
Symbol
Reads as
\(P(A)\)
probability of event A
\(P(A\mid B)\)
probability of A given that B happened
\(A^c\)
complement, “not A”
\(A\cap B\)
intersection, “A and B”
\(A\cup B\)
union, “A or B or both”
\(f(x)\)
density (continuous) or mass (discrete)
\(F(x)\)
cumulative distribution, \(P(X\le x)\)
\(\mu,\ \sigma\)
population mean, population standard deviation
\(\bar x,\ s\)
sample mean, sample standard deviation
\(\sigma_{\bar X}\)
standard error of the mean
\(\alpha,\ \beta\)
Type I error rate, Type II error rate
\(\Phi(z)\)
area under the standard normal from 0 to z
01
Complement
\[P(A^c)=1-P(A)\]
\[A^c \cap B^c = (A \cup B)^c\]
Meaning
The probability of not-A is one minus the probability of A.
De Morgan
By De Morgan’s law, \(A^c \cap B^c = (A \cup B)^c\).
Origin
Directly from the axioms. \(A\) and \(A^c\) are disjoint and their union is the whole sample space, so \(P(A)+P(A^c)=P(\Omega)=1\).
When
Whenever “at least one” or “none” appears. \(P(\text{at least one six in 4 rolls}) = 1-(5/6)^4\) is far cheaper than summing four cases.
Fails
Only valid for a genuine complement. “At least two” is not the complement of “at least three” — the complement of “at least two” is “at most one,” and that’s usually a sum of several terms.
02
Addition rule
\[P(A\cup B)=P(A)+P(B)-P(A\cap B)\]
Meaning
The chance that at least one of two events happens.
Origin
Adding \(P(A)+P(B)\) counts every outcome in the overlap twice — once as an A outcome, once as a B outcome. Subtracting \(P(A\cap B)\) once restores each outcome to a single count. With three events you subtract all three pairwise intersections and add back the triple.
When
Union questions. Reduces to plain addition only when the events are disjoint, in which case \(P(A\cap B)=0\).
Fails
Forgetting the subtraction. Also, the three-event version is not symmetric guesswork — get the signs from the pattern −, +, −.
The probability of B restricted to the world where A is already true. It re-scales the sample space from \(\Omega\) down to \(A\).
Origin
Definition. It is the fraction of A’s probability mass that also lies in B. Equivalently, it’s what you get by dividing the joint probability by the conditioning event’s probability so that probabilities conditional on A still sum to 1.
When
Any “given that,” “if we know,” “among those who” phrasing. Also the engine for sequential draws without replacement.
Fails
Conditioning on a zero-probability event. And the classic inversion: \(P(B\mid A)\ne P(A\mid B)\) — \(P(\text{positive}\mid\text{disease})\) is not \(P(\text{disease}\mid\text{positive})\).
04
Multiplication rule
\[P(A\cap B)=P(A)\,P(B\mid A)=P(B)\,P(A\mid B)\]
Meaning
Joint probability as a chain of conditionals.
Origin
Rearrangement of the definition of conditional probability. For a sequence, extend the chain: \(P(A\cap B\cap C)=P(A)P(B\mid A)P(C\mid A\cap B)\).
When
Ordered draws without replacement, tree diagrams, and as the numerator of Bayes.
Fails
Applying it to unordered selections (that needs the combination count instead), and using it with the wrong conditional order.
05
Independence
\[P(A\cap B)=P(A)P(B)\iff P(A\mid B)=P(A)\]
Meaning
Knowing B tells you nothing about A.
Origin
The multiplication rule collapses when the conditional equals the unconditional probability.
When
Only when the problem states independence (with replacement, separate components, “independently chosen”) or when a joint density factors and the support is a rectangle.
Fails
Assuming independence because events feel unrelated. For three events, pairwise independence is not enough — you also need \(P(A\cap B\cap C)=P(A)P(B)P(C)\), and even then it’s possible to be pairwise independent but not mutually independent.
06
Total probability
\[P(B)=\sum_i P(B\mid A_i)P(A_i)\]
Meaning
The overall probability of B as a weighted average over mutually exclusive, exhaustive cases \(A_i\).
Origin
Partition the sample space into the \(A_i\), write B as the disjoint union of the pieces \(B\cap A_i\), then apply the multiplication rule to each piece and add.
When
Mixed populations — multiple machines, urns, or groups with different rates. This is the denominator of every Bayes problem.
Fails
Cases that overlap or don’t cover everything. The weights must sum to 1.
Updating a prior belief in light of evidence. Numerator: prior × likelihood for the case of interest. Denominator: total probability of the evidence across all cases, which normalizes the posterior so the cases sum to 1.
Origin
Conditional probability with the numerator expanded by the multiplication rule and the denominator by total probability.
When
Diagnosis, defect attribution, source identification — anywhere a base rate combines with a test result.
Fails
Dropping the denominator and reporting the likelihood as the posterior. That error is what makes a 99%-accurate test for a rare condition look impressive when it isn’t.
Ordered arrangements of r from n, and unordered selections of r from n. The second line is the multiset permutation: distinct orders of a fixed collection in which type \(i\) already occurs \(n_i\) times.
Origin
\(n(n-1)\cdots(n-r+1)\) counts a sequence of choices. For combinations, each r-subset was counted \(r!\) times, once per ordering, so divide by \(r!\). For a multiset, \(n!\) would be right if every item were distinct; each group of identical copies can be reordered in \(n_i!\) ways that look the same, so divide those factors out.
When
“In order,” “arrangement,” “password,” “ranking” → permutation. “Hand,” “committee,” “set,” “group” → combination. “Distinct arrangements of the letters in MISSISSIPPI,” or any fixed pile of identical objects → multiset.
Fails
The wrong one of the two. If a problem says “three people are chosen,” the answer has no \(3!\) factor. Repetition is three different counts: \(n^r\) when each of r positions is freely chosen from n types; \(\binom{n+r-1}{r}\) for the unordered version of that; and \(n!/(n_1!\,n_2!\,\cdots\,n_k!)\) only when the supply of each type is fixed and you use all of it.
Probability is area under the density. The CDF accumulates area from the left.
Origin
Definitions; the third line is the fundamental theorem of calculus.
When
Converting between representations. Also \(P(X\gt x)=1-F(x)\).
Fails
Thinking \(f(x)\) is a probability — it can exceed 1, and \(P(X=a)=0\) for continuous X. Also the CDF must be nondecreasing with limits 0 and 1; a piecewise answer that dips or plateaus wrongly is wrong.
Integrate out the variable you don’t care about; divide the joint by the marginal to condition.
Origin
Continuous analogues of summing a joint table by row or column, and of the conditional-probability definition.
When
Going from a joint distribution to a single variable, and then to a conditional.
Fails
Integrating over limits that depend on the other variable without accounting for it, and forgetting that the conditional density must itself integrate to 1 over y.
12
Expectation
\[E(X)=\sum_x x f(x)\quad\text{or}\quad\int x f(x)\,dx\]
Meaning
The probability-weighted average, i.e. the long-run mean of X.
Origin
Weighted-mean definition; the discrete version is literally a weighted average with the probabilities as weights.
When
Any “expected value,” “mean,” “average winnings.”
Fails
Forgetting to weight by the density. Also, \(E(X)\) need not be an attainable value, and need not exist (the Cauchy distribution has no mean).
To average a function of X, weight the function’s values by the probabilities — no need to derive the distribution of \(g(X)\) first.
Origin
Group outcomes by the value of \(g(X)\); the probabilities add up to the density of that value.
When
Computing \(E(X^2)\) for variance, or any polynomial moment.
Fails
Computing \(E[g(X)]\) as \(g(E[X])\). These agree only for linear \(g\).
14
Variance and its shortcut
\[Var(X)=E[(X-\mu)^2]=E(X^2)-[E(X)]^2\]
Meaning
Mean squared deviation from the mean; the spread. Units are squared.
Origin
Expand the square: \(E[X^2-2\mu X+\mu^2] = E(X^2)-2\mu E(X)+\mu^2 = E(X^2)-\mu^2\), using \(\mu=E(X)\) and linearity.
When
Use the shortcut form for computation; use the definition when reasoning about meaning.
Fails
Computing \([E(X)]^2\) and calling it \(E(X^2)\). Also reporting variance when standard deviation was asked.
15
Linearity and variance scaling
\[E(aX+b)=aE(X)+b,\qquad Var(aX+b)=a^2Var(X)\]
Meaning
Shifting adds to the mean but doesn’t change spread; scaling multiplies the mean by \(a\) but the variance by \(a^2\), because variance is in squared units.
Origin
Linearity of expectation is axiomatic; the variance rule follows from the definition because deviations scale by \(\lvert a\rvert\).
When
Rescaling units, converting Celsius to Fahrenheit, standardizing.
Fails
Writing \(a\,Var(X)\). And the shift term vanishes only for variance — the mean keeps \(b\).
No more than \(1/k^2\) of the mass sits at least k standard deviations from the mean, for any distribution.
Origin
Markov’s inequality, \(P(Y\ge a)\le E(Y)/a\) for nonnegative Y, applied to \(Y=(X-\mu)^2\) with \(a=k^2\sigma^2\).
When
Bounds when the distribution is unknown, or as a guaranteed floor.
Fails
Applying it to a known distribution — it’s loose (0.111 at \(k=3\) versus 0.0027 for the normal). Also \(k\gt 1\) is required for the bound to be under 1.
Probability of exactly x successes in n independent trials with success probability p.
Origin
Choose which x trials succeed: \(\binom{n}{x}\) ways. Each specific ordering has probability \(p^xq^{n-x}\). Multiply.
When
Fixed n, two outcomes, constant p, independence.
Fails
Using it without replacement (that’s hypergeometric). Also, the mean and variance are derived, not separate facts — \(E(X)=np\) by linearity over indicator variables, and \(Var=npq\) because the indicators are independent.
Probability of x events when they occur at rate \(\lambda\) per unit interval, independently.
Origin
The limit of the binomial with \(n\to\infty\), \(p\to 0\), \(np=\lambda\) fixed.
When
Counts of rare events over time, area, or volume; binomial approximation when \(n\ge 50\) and \(np\lt 5\).
Fails
Using it when events cluster (contagion), which breaks the independence assumption. And the mean equals the variance — a built-in check on your answer.
Distance from the mean in standard deviations, which puts every normal on one common scale.
Origin
Subtracting the mean gives mean 0; dividing by \(\sigma\) gives variance 1. The transformation is affine, so normality is preserved.
When
Any normal probability, and the normal approximation to binomial or sampling distributions.
Fails
Standardizing with the wrong standard deviation — \(\sigma/\sqrt n\) for a mean, \(\sigma\) for a single observation. And reading \(\Phi(z)\) as the left tail when it’s the area from 0 to z: for a left tail you want \(0.5+\Phi(z)\) when z is negative.
The standard deviation of the sample mean across repeated samples; how far \(\bar X\) typically lands from \(\mu\).
Origin
\(Var(\bar X)=Var\left(\frac1n\sum X_i\right)=\frac{1}{n^2}\cdot n\sigma^2=\frac{\sigma^2}{n}\), using independence. Square root. The correction factor accounts for the dependence introduced by sampling without replacement from a finite population; the proportion version substitutes \(pq\) for \(\sigma^2\).
When
Every inference procedure. The proportion form is just the Bernoulli case, where \(\sigma^2=pq\).
Fails
Writing \(\sigma\) where \(\sigma/\sqrt n\) belongs. Omitting the correction when \(n\) is a large fraction of \(N\) (roughly \(n\gt 0.05N\)).
A range built so that a stated fraction of such ranges, over repeated sampling, contain the parameter. The half-width is the critical value times the standard error.
Origin
The pivot \((\bar x-\mu)/(\sigma/\sqrt n)\) is standard normal. Choose the critical value so the middle \(1-\alpha\) of that distribution is captured, then solve the inequality for \(\mu\). The \(t\) version replaces \(\sigma\) with \(s\) and uses heavier tails, because \(s\) is itself estimated.
When
Estimation with a stated confidence. \(z\) needs known \(\sigma\) and large \(n\); \(t\) for small samples from a normal population.
Fails
Using \(s\) without the \(\sqrt n\). Using \(z\) for \(n\lt 30\) with unknown \(\sigma\). Saying “95% probability that \(\mu\) is in this interval” — the interval is fixed and \(\mu\) is fixed; the probability belongs to the procedure.
The n that makes the half-width equal to a target margin E.
Origin
Set \(z\sigma/\sqrt n=E\) and solve for n.
When
Planning a study.
Fails
Rounding down. Always round up — a smaller n gives a wider interval than requested. And use \(p=0.5\) for the proportion version when no prior estimate exists, since that maximizes \(pq\).
The parameter value under which the observed data are most probable. Maximize the likelihood, or equivalently its log, since log is monotone.
Origin
Definition. The log turns products into sums, which is why it’s always used.
When
Deriving estimators. For Bernoulli it gives \(k/n\); for exponential, \(1/\bar x\).
Fails
Forgetting the support constraint (the uniform MLE is the maximum observation, not a stationary point). And a stationary point can be a minimum or a boundary — check the second derivative or the endpoints.
Each is a signal-to-noise ratio: the observed departure from the hypothesized value, divided by the standard error of that departure.
Origin
Under \(H_0\) the numerator is centred at 0, so the ratio is standardized. The chi-square statistic sums squared standardized residuals across categories.
When
\(z\) with known \(\sigma\) and large n; \(t\) with unknown \(\sigma\); chi-square for goodness of fit with \(k-1\) degrees of freedom, less one per estimated parameter.
Fails
Using \(s\) in a \(z\) statistic. Inverting numerator and denominator. Omitting the subtraction in chi-square, which gives a sum of squares instead of squared deviations.
\(\hat b\) is the covariance of x and y over the variance of x — the slope of the least-squares line. \(\hat a\) forces the line through \((\bar x,\bar y)\).
Origin
Minimize \(\sum(y_i-a-bx_i)^2\), take partials with respect to a and b, set to zero, solve the two normal equations.
When
Linear fit and prediction within the observed range.
Fails
Using the line outside the data range (extrapolation). Also, the slope is not the correlation — \(\hat b = r\,s_y/s_x\), so it changes if you swap the axes while \(r\) doesn’t.
How these formulas actually fail
Three failure patterns, in order of how much they cost:
Assumption violation
Every formula above carries a hidden precondition — independence for variance addition, known \(\sigma\) for \(z\), constant p for binomial, rectangular support for the density factorization. The formula still computes when the assumption breaks; it just computes the wrong thing silently.
Wrong direction of a ratio
Signal over noise, not the reverse. Deviation over standard error, not standard error over deviation. Numerator and denominator in Bayes, not inverted.
Missing the correction or the subtraction
Half-unit corrections, \(-P(A\cap B)\), \(-2\mu E(X)\), \(\sqrt n\). These are all one term, and all of them are the difference between right and wrong.
If you want a real test of understanding rather than recognition, close this and rebuild from memory: the variance shortcut derivation, the SE derivation, and the Bayes derivation from the multiplication rule. Those three are the spine.