PART 1 • FOUNDATIONS • LEVEL 04
Probability & Statistics for Machine Learning
Turn uncertainty into measurable evidence. Learn how models reason about chance, summarize data, estimate unknown quantities and distinguish real signals from random variation.
μ−σ+σ
PRIOR10%+ EVIDENCEPOSTERIOR
LEARNING OUTCOMES
By the End of This Level, You Can
QUICK REVISION
Five Skills Used Throughout This Level
Probability becomes easier when these arithmetic and set ideas are clear.
favourable / total for equally likely outcomes.
0.25 and 25% express the same chance.
A ∪ B means either; A ∩ B means both.
The mean balances all observations but reacts to outliers.
Standard deviation is the square root of variance.
CONCEPT 01
Experiment, Outcome, Sample Space and Event
A probability statement begins by defining exactly what can happen.
A repeatable uncertain process, such as rolling a die.
One observed result, such as rolling 4.
Every possible outcome: {1, 2, 3, 4, 5, 6}.
A subset of outcomes, such as even = {2, 4, 6}.
An experiment is a repeatable uncertain process, an outcome is one possible result, the sample space contains all possible outcomes, and an event is a selected subset. Defining these precisely determines which probabilities are meaningful.
For two coin tosses the sample space is {HH, HT, TH, TT}; the event ‘exactly one head’ is {HT, TH}.
In classification, events may represent positive predictions, real positives or specific error types.
Events must be defined before inspecting results to avoid changing the question after seeing the data.
CONCEPT 02
Core Probability Rules
Use set structure to avoid double-counting outcomes.
Often easier to calculate “not A” first.
Subtract the overlap counted twice.
The events cannot occur together.
Combine every disjoint route to B.
Probabilities range from 0 to 1 and sum to 1 over mutually exclusive complete outcomes. The complement rule handles ‘not A,’ addition handles unions, and multiplication handles joint events through conditional probability. Inclusion–exclusion prevents double-counting overlap.
P(A or B)=P(A)+P(B)-P(A and B). If A and B cannot occur together, the intersection term is zero.
These rules support confusion-matrix rates, probabilistic models and risk calculations.
Do not multiply P(A) and P(B) unless independence is justified.
CONCEPT 03
Conditional Probability and Independence
New evidence changes the relevant sample space.
CONDITIONAL PROBABILITYP(A|B) = P(A∩B) / P(B)
Read as “probability of A given B.” Require P(B) > 0.
INDEPENDENCEP(A∩B) = P(A)P(B)
Knowing B does not change A, so P(A|B) = P(A).
Interview trap: Mutually exclusive events with positive probability are not independent. If one happens, the other becomes impossible.
Conditional probability P(A|B) restricts attention to cases where B occurred. Independence means learning B does not change the probability of A. Conditional independence is weaker and depends on known context; it is central to graphical models and Naive Bayes.
Rain and traffic may be dependent overall but conditionally independent after controlling for location and time in a simplified model.
Feature assumptions in probabilistic classifiers are statements about conditional relationships.
P(A|B) and P(B|A) answer different questions and are rarely equal.
CONCEPT 04
Bayes’ Theorem: Update Belief with Evidence
Bayes reverses a conditional probability by combining prior belief and evidence reliability.
POSTERIORP(A|B)=
P(B|A)LIKELIHOOD
×P(A)PRIOR
÷P(B)EVIDENCE
Belief before observing current evidence.
How expected the evidence is if the hypothesis is true.
Total probability of seeing the observation.
Updated belief after combining prior and evidence.
Bayes’ theorem reverses a conditional probability by combining prior belief, evidence likelihood and overall evidence rate. The posterior is proportional to likelihood times prior. It formalizes how base rates influence conclusions even when a test is accurate.
A rare disease can have many false positives because healthy people greatly outnumber affected people, reducing P(disease|positive).
Bayesian reasoning appears in diagnosis, spam filtering, inference and uncertainty updates.
Ignoring prevalence creates the base-rate fallacy and overstates the meaning of a positive result.
CONCEPT 05
Random Variables, PMF, PDF and CDF
A random variable maps each outcome to a numerical value.
Assigns probability mass to countable values. All masses sum to 1.
Probability is area under a density curve. At one exact point, probability is 0.
Accumulated probability up to x. It never decreases and approaches 1.
A random variable assigns a number to each outcome. A PMF gives probabilities for discrete values; a PDF describes density for continuous values; a CDF gives the probability of being at or below a value. For continuous variables, probability is area over an interval, not density at one point.
The CDF can answer the probability that response time is at most 200 ms regardless of whether the distribution is discrete or continuous.
Models often predict distributions or quantiles rather than a single value.
A PDF value can exceed 1; only its total area must equal 1.
CONCEPT 06
Important Distributions for AI & ML
A distribution describes which values are possible and how likely they are.
Click, churn, fraud or class label. Parameter: p.
Assumes fixed n, constant p and independent trials.
Arrivals, failures or requests with average rate λ.
Useful for random initialization and simple simulation.
Controlled by mean μ and variance σ².
Bernoulli models one binary trial, Binomial counts successes across repeated trials, Gaussian models many symmetric continuous variations, Poisson counts events in an interval, and Exponential models waiting time under a constant event rate. Each distribution carries assumptions that must match the process.
Clicks per minute may be approximated by Poisson only if events are suitably independent and the average rate is stable.
Choosing a loss function often corresponds to an assumed target distribution.
Do not select a distribution only because its curve looks familiar; check support and process assumptions.
CONCEPT 07
Expectation, Variance and Standard Deviation
Expectation describes long-run centre; variance describes squared spread around it.
A probability-weighted average, not necessarily an observable outcome.
Average squared deviation; units are squared.
Spread in the same units as X.
Expectation is the long-run probability-weighted average. Variance measures expected squared deviation from the mean, and standard deviation returns to the original units. Linearity of expectation works even without independence, while variance addition usually needs covariance terms.
Two investments can have equal expected return but very different variance and therefore different risk.
Expected loss is the quantity many learning algorithms seek to minimize.
The mean may be unrepresentative for heavy-tailed or strongly skewed data.
CONCEPT 08
Covariance and Correlation
Both describe how two variables move together, but only correlation is scale-free.
Sign indicates direction; magnitude depends on units and scale.
Standardized linear association from −1 to +1.
−1
negative0
none linear+1
positive
negative0
none linear+1
positive
Critical rule: Correlation does not prove causation, and zero correlation does not guarantee independence or absence of a non-linear relationship.
Covariance measures whether two variables move together in their original scales. Correlation standardizes covariance to a unit-free value from -1 to 1. Both summarize linear association and may miss nonlinear dependence or be distorted by outliers.
Height measured in centimetres and metres changes covariance but not correlation.
Correlation matrices support EDA, feature selection and multicollinearity diagnosis.
Correlation does not establish causation, and zero correlation does not always mean independence.
CONCEPT 09
Population, Sample, LLN and CLT
We observe samples to learn about a larger population.
Parameters such as μ, σ and p describe it.
Statistics such as x̄, s and p̂ summarize it.
Intervals and tests connect sample evidence to population claims.
As sample size grows, the sample average tends toward the expected value.
Under common conditions, standardized sample means approach a normal distribution as n grows.
A population is the complete group of interest; a sample is the observed subset. The law of large numbers explains why sample averages stabilize with more independent observations. The central limit theorem explains why many properly normalized sample means approach a normal distribution under suitable conditions.
Repeatedly averaging larger random samples produces a tighter sampling distribution around the population mean.
Sampling theory supports uncertainty estimates, experiment analysis and model evaluation.
More biased data converges to the wrong quantity; sample size cannot repair unrepresentative collection.
CONCEPT 10
Descriptive Statistics and Robust Summaries
Choose summaries that match the data distribution and decision.
Efficient for symmetric data but sensitive to outliers.
Robust for skewed distributions and extreme observations.
Works with categories and repeated numeric values.
Robust spread of the central 50% of observations.
Descriptive statistics summarize centre, spread, range, quantiles and shape. Mean and standard deviation are sensitive to extreme values; median and interquartile range are more robust. Always pair summaries with distributions and relevant subgroups.
For income data, the median usually represents a typical person better than the mean because the upper tail is long.
Robust summaries guide imputation, scaling and anomaly investigation.
One overall average can hide opposite patterns across groups or time periods.
CONCEPT 11
Estimation and Confidence Intervals
A point estimate gives one value; an interval reports plausible values under repeated-sampling logic.
LOWERx̄UPPER
estimate ± critical value × standard errorCorrect 95% interpretation:If we repeated the sampling method many times, about 95% of the constructed intervals would contain the true parameter. It is not a 95% probability statement about a fixed parameter after the interval is computed.
A point estimate gives one plausible parameter value; a confidence interval describes the long-run performance of an interval-building procedure. Its width reflects sample size, variability and confidence level. Assumptions about sampling and dependence determine whether the interval is valid.
A 95% confidence procedure captures the true parameter in about 95% of repeated compatible studies—not with 95% probability for one fixed frequentist interval.
Intervals communicate uncertainty more honestly than reporting a metric to several decimals.
Narrow intervals are not meaningful when data collection is biased or observations are dependent.
CONCEPT 12
Hypothesis Testing Without Misinterpretation
A test asks whether observed data would be unusually incompatible with a specified null hypothesis.
Define the claim and alternative before inspecting results.
Set the false-positive tolerance, commonly 0.05.
Use an appropriate test statistic and assumptions.
Reject or fail to reject H₀; report effect size and uncertainty.
Probability controlled by α under the null.
Probability β; power equals 1 − β.
Not the probability that H₀ is true.
Hypothesis testing compares observed evidence with a null model. A p-value is the probability, assuming the null and test assumptions, of obtaining a result at least as extreme. It is not the probability that the null is true and does not measure practical importance.
A tiny improvement can be statistically significant with a huge sample but still be operationally useless.
A/B testing requires effect size, uncertainty, power and decision cost in addition to a p-value.
Repeated peeking or testing many hypotheses without correction increases false discoveries.
CONCEPT 13
Likelihood, MLE and MAP
Model training often chooses parameters that make observed data probable.
Treat data as fixed and compare candidate parameter values.
Select parameters that best explain observed data.
Combine likelihood with a prior; regularization often has this interpretation.
Numerical practice: Maximize log-likelihood instead of multiplying many tiny probabilities. Logs convert products into stable sums.
Likelihood treats observed data as fixed and compares parameter values by how well they explain it. Maximum likelihood chooses the highest-likelihood parameters. MAP adds a prior and maximizes the posterior, often producing a regularized solution.
For Bernoulli observations, the maximum-likelihood success probability is the observed fraction of successes.
Cross-entropy training is maximum likelihood for common classification models.
Likelihood is not a probability distribution over parameters unless a prior and normalization create a posterior.
CONCEPT 14
Experiment Design and A/B Testing
Good statistics begins before data collection.
Reduce systematic allocation bias.
Provide a meaningful comparison baseline.
Choose primary metric, α and stopping rule first.
Use enough observations to detect a meaningful effect.
Avoid uncontrolled multiple comparisons.
Statistical significance is not practical importance.
A controlled experiment changes one factor and uses random assignment to balance known and unknown confounders. Define the unit, treatment, primary metric, guardrails, duration and analysis before starting. Practical significance and user harm matter alongside statistical significance.
Randomize users—not page views—when repeated views from the same person could contaminate both variants.
Online ML systems often require experiments to verify that offline metric gains improve real outcomes.
Stopping as soon as significance appears inflates false-positive risk unless a sequential method was planned.
INTERACTIVE LEARNING • CODEBHAVYA PREMIUM VISUALIZER
LIVE🎬 Bayes’ Theorem — Visual Flow
Trace a factory alert where 10% of items are defective, sensitivity is 80% and the false-positive rate is 20%.
STEP 1 OF 7
Begin with 100 items
Use a natural-frequency view to make every probability concrete.
Step 1 of 7
PROGRAM TRACING • TRUE LOOP FLOW
Trace a Monte Carlo Probability Estimate
Watch the cursor revisit the loop and condition while six recorded die rolls estimate P(rolling a six).
Press Next to execute the first statement.
EXPRESSION
—OUTPUT
Waiting for print(...)
Step 0
CORE SIMULATION PATTERN
Estimate Probability from Repeated Trials
Monte Carlo methods approximate an unknown quantity by repeating a random experiment and aggregating outcomes.
rolls = [2, 6, 4, 1, 6, 3]
successes = 0
for roll in rolls:
if roll == 6:
successes += 1
trials = len(rolls)
estimate = successes / trials
print(round(estimate, 2))MONTE CARLO ESTIMATE
p̂ = successes / trials- Trial: one repetition of the experiment.
- Success: the event being estimated.
- Estimate: observed relative frequency.
- LLN: p̂ tends toward p as trials grow.
- Uncertainty: finite simulations still contain sampling error.
PROGRAMMING PROBLEMS • CODEBHAVYA PATTERN
Practise Probability and Statistical Reasoning
Attempt each problem independently. The checker rewards correct formulas, NumPy operations and requested output.
0 / 5Solved independently0 / 500Best score
QUICK KNOWLEDGE CHECK
Test Probability and Statistical Interpretation
Select one answer for every question. Results show your answer, the correct answer and an explanation.
Not checked yet
PLACEMENT CHECKPOINT
How Statistics Appears in Hiring Rounds
Interviewers look for correct interpretation, not only memorized formulas.
Bayes Reasoning
Convert rates into natural frequencies and avoid base-rate neglect.
Distribution Choice
Match Bernoulli, Binomial, Poisson or Normal assumptions to a scenario.
Experiment Analysis
Interpret confidence intervals, p-values, effect size and power.
ML Connection
Explain likelihood, loss functions, calibration and uncertainty.
CodeBhavya interview pattern:Define the quantity → State assumptions → Show the formula → Interpret in context → Mention one misuse.
INTERVIEW PREPARATION
🎤 Probability & Statistics — Interview Questions
Answer aloud before selecting Show Answer for each explanation.
KEY TAKEAWAY
Statistical Reasoning in One View
Events and distributions describe uncertainty.
Centre, spread and association reveal data structure.
Samples estimate populations with measurable uncertainty.
Combine evidence, assumptions and practical importance.
Statistics does not remove uncertainty—it makes uncertainty visible enough to reason and act responsibly.
EXTRA TIPS
Habits of Reliable Statistical Thinkers
Define the sample space and event before calculating probability.
Use natural frequencies such as “8 out of 100” for Bayes problems.
Plot the distribution before reporting only mean and standard deviation.
Check independence, sampling and distribution assumptions explicitly.
Report effect sizes and confidence intervals beside p-values.
Separate statistical significance from practical or business value.
EXTRA PRACTICE QUESTIONS
Strengthen Probability and Statistics
Solve conceptually first, then verify through code or simulation.
- 01
List the sample space for tossing two coins.
- 02
Find the probability of at least one head in two tosses.
- 03
Explain the difference between mutually exclusive and independent events.
- 04
Calculate P(A∪B) when P(A)=0.5, P(B)=0.4 and P(A∩B)=0.2.
- 05
Use Bayes’ theorem for a 1% base rate, 95% sensitivity and 5% false-positive rate.
- 06
Choose a distribution for the number of calls arriving each minute.
- 07
Calculate the expected value of a fair six-sided die.
- 08
Compare mean and median for [2, 3, 4, 5, 100].
- 09
Explain why standard deviation is easier to interpret than variance.
- 10
Describe what correlation 0.8 does and does not prove.
- 11
Interpret a 95% confidence interval for an A/B conversion difference.
- 12
Explain a p-value of 0.03 without saying H₀ has a 3% chance of being true.