PART 2 • CORE MACHINE LEARNING • LEVEL 07

Logistic Regression & Classification

Convert evidence into probabilities, probabilities into decisions and decisions into measurable outcomes. Learn classification mathematics, thresholds and evaluation from first principles.

⏱️ 180–210 min🎯 Beginner → Interview Ready🧪 2 Interactive Labs💼 Placement Metrics
Start Learning →Open Boundary Visualizer
CLASS 0CLASS 1
PROBABILITYp(y=1|x) = σ(z)threshold = 0.50
LEARNING OUTCOMES

By the End of This Level, You Can

01Explain log-odds, sigmoid probability and decision boundaries.
02Derive binary cross-entropy and logistic gradients.
03Choose thresholds using real error costs.
04Calculate confusion-matrix classification metrics.
05Handle imbalance, calibration and multiclass problems.
06Build and diagnose a leakage-safe classifier pipeline.
QUICK REVISION

Six Ideas Before Classification

Classification begins with probability, linear scores and honest evaluation.

BINARY LABELy ∈ {0, 1}

Encode the two outcomes consistently.

PROBABILITY0 ≤ p ≤ 1

Confidence before a decision threshold.

LINEAR SCOREz = wᵀx + b

Weighted evidence before sigmoid.

LOGARITHMlog(p)

Turns likelihood products into sums.

DATA SPLITStratify safely

Preserve class proportions when appropriate.

BUSINESS COSTFP ≠ FN

Different mistakes often have different consequences.

CONCEPT 01

Classification Predicts Categories

A classifier estimates evidence for discrete outcomes instead of predicting an unrestricted number.

🩺Disease / no disease

Prioritize the cost of a missed positive diagnosis.

🛡️Fraud / legitimate

Severe imbalance makes accuracy misleading.

📧Spam / inbox

Thresholds balance unwanted mail and missed spam.

🎓Placed / not placed

Use only evidence available before the outcome.

Classifier outputfeatures → probability → threshold → predicted classEvaluate both probability and decision
DETAILED EXPLANATION

Classification predicts a discrete outcome such as fraud or not fraud. Many classifiers first estimate scores or probabilities and then apply a decision rule. Defining the positive class and cost of each error is essential because category labels alone do not describe operational consequences.

WORKED INTUITION

Spam detection, disease screening and defect recognition are classification tasks, even though their false-positive and false-negative costs differ greatly.

AI / PLACEMENT CONNECTION

Classification interviews commonly test decision policy, not only algorithm syntax.

COMMON MISCONCEPTION

Turning every probability above 0.5 into class 1 is a convention, not a universal rule.

CONCEPT 02

Odds and Log-Odds

Logistic regression models the logarithm of class odds as a linear function.

PROBABILITYp = P(y=1|x)

A value from 0 to 1.

ODDSp / (1−p)

How much more likely class 1 is than class 0.

LOGITlog(p/(1−p)) = wᵀx+b

An unrestricted linear score from −∞ to +∞.

Interpretation: increasing feature xⱼ by one unit multiplies the odds by eʷʲ, holding other modeled features constant.
DETAILED EXPLANATION

Odds compare the probability of an event with the probability of it not occurring: p/(1-p). Log-odds apply the logarithm, mapping probabilities between 0 and 1 onto the complete real line. Logistic regression models log-odds as a linear combination of features.

WORKED INTUITION

Probability 0.8 corresponds to odds 4:1 and log-odds ln(4), approximately 1.386.

AI / PLACEMENT CONNECTION

A one-unit feature increase multiplies the odds by exp(weight) when other features remain fixed.

COMMON MISCONCEPTION

Odds and probability are related but not numerically interchangeable.

CONCEPT 03

The Sigmoid Function

Sigmoid converts any real-valued score into a valid probability.

LOGISTIC FUNCTION
σ(z) = 1 / (1 + e⁻ᶻ)
  • z → −∞: probability approaches 0.
  • z = 0: probability equals 0.5.
  • z → +∞: probability approaches 1.
1.00.50.0z = 0
DETAILED EXPLANATION

The sigmoid function maps any real score to a value between 0 and 1. A score of zero maps to 0.5; large positive scores approach 1 and large negative scores approach 0. Its derivative p(1-p) supports gradient-based learning.

WORKED INTUITION

Scores -2, 0 and 2 produce probabilities of about 0.119, 0.5 and 0.881.

AI / PLACEMENT CONNECTION

Sigmoid is used for independent binary outputs and the final probability in binary logistic regression.

COMMON MISCONCEPTION

Extreme scores can cause numerical overflow or log(0); use stable library implementations and clipped probabilities.

CONCEPT 04

Logistic Model and Decision Boundary

The probability is curved, but the default boundary occurs where the linear score equals zero.

LINEAR SCOREz = w₁x₁ + w₂x₂ + b
→
PROBABILITYp = σ(z)
→
DEFAULT DECISIONp ≥ 0.5 ⇔ z ≥ 0

The equation w₁x₁ + w₂x₂ + b = 0 defines a line in 2D, a plane in 3D and a hyperplane in higher dimensions.

DETAILED EXPLANATION

Logistic regression forms a linear score z=w·x+b and transforms it with sigmoid. The threshold determines the boundary: at threshold 0.5, z=0. In multiple dimensions the score boundary is a line, plane or hyperplane even though the probability changes nonlinearly.

WORKED INTUITION

For one feature with positive weight, larger x values receive larger probabilities and cross the decision threshold at a calculable x.

AI / PLACEMENT CONNECTION

Feature scaling and regularization affect optimization and coefficient stability.

COMMON MISCONCEPTION

The name contains regression because log-odds are modelled continuously, but the usual task is classification.

CONCEPT 05

Binary Cross-Entropy: Learn Probabilities

Log loss strongly penalizes confident wrong probabilities and is the negative log-likelihood for Bernoulli outcomes.

ONE EXAMPLE−[y log(p) + (1−y)log(1−p)]
If y = 1

The loss becomes −log(p); higher predicted probability for class 1 reduces loss.

If y = 0

The loss becomes −log(1−p); lower predicted probability for class 1 reduces loss.

Numerical safetyclip p away from exactly 0 and 1Use stable library implementations
DETAILED EXPLANATION

Binary cross-entropy rewards high probability on the true class and applies a rapidly increasing penalty to confident wrong predictions. It is the negative log-likelihood of Bernoulli outcomes. Averaging it across samples creates a smooth objective for logistic-regression training.

WORKED INTUITION

For a true positive label, predicting 0.9 has low loss while predicting 0.01 has very large loss.

AI / PLACEMENT CONNECTION

Cross-entropy trains probability estimates, whereas accuracy changes only after a thresholded class decision.

COMMON MISCONCEPTION

A lower training cross-entropy does not guarantee calibrated or generalizable probabilities.

CONCEPT 06

Optimization and Regularization

Gradient descent adjusts coefficients using probability error; regularization controls flexibility.

PREDICTpᵢ = σ(wᵀxᵢ+b)
GRADIENTdw = (1/n)Xᵀ(p−y)db = mean(p−y)
UPDATEw ← w − αdwb ← b − αdb
REGULARIZEloss + λ penalty(w)

Scale features before comparing or penalizing coefficients.

DETAILED EXPLANATION

The cross-entropy gradient simplifies to prediction probability minus true label, multiplied by each feature for weight gradients. Gradient descent updates weights and bias across repeated batches. L1 or L2 regularization controls coefficient size and can improve stability under correlated or noisy features.

WORKED INTUITION

A positive example predicted with low probability produces a negative error that pushes its score upward.

AI / PLACEMENT CONNECTION

Tracing one batch reveals how every sample contributes to the final parameter update.

COMMON MISCONCEPTION

Regularization strength must be validated, and preprocessing must be fitted inside the same pipeline.

CONCEPT 07

A Threshold Converts Probability into Action

The default 0.5 threshold is convenient—not automatically optimal.

0.120.380.640.870.50
p < threshold → class 0p ≥ threshold → class 1
Lower threshold

Usually catches more positives: recall rises, but false positives may increase.

Higher threshold

Usually demands stronger evidence: false positives fall, but positives may be missed.

DETAILED EXPLANATION

A threshold converts an estimated probability into an action. Lowering it predicts more positives, usually increasing recall and false positives; raising it predicts fewer positives, often increasing precision while missing more positives. Select it using validation data and explicit costs or constraints.

WORKED INTUITION

A screening system may use a lower threshold to avoid missing disease, followed by a specific confirmatory test.

AI / PLACEMENT CONNECTION

Thresholds can differ by workflow stage but require fairness, calibration and capacity checks.

COMMON MISCONCEPTION

Do not tune the threshold on the final test set or choose it only to maximize accuracy.

CONCEPT 08

The Confusion Matrix

Every binary decision belongs to one of four outcomes.

ACTUAL ↓ / PREDICTED →
POSITIVE
NEGATIVE
POSITIVE
TPCorrectly detected positive
FNPositive was missed
NEGATIVE
FPFalse alarm
TNCorrect rejection
DETAILED EXPLANATION

The confusion matrix counts true positives, true negatives, false positives and false negatives for a defined positive class and threshold. It is the source of most classification metrics. Counts show workload and harm; normalized rates support comparison across datasets.

WORKED INTUITION

In disease screening, a false negative is an affected patient incorrectly predicted negative.

AI / PLACEMENT CONNECTION

Reading the matrix correctly is one of the most common placement calculations.

COMMON MISCONCEPTION

Changing which class is positive changes precision, recall and the interpretation of every error.

CONCEPT 09

Classification Metrics and Their Questions

Start with the decision question, then select a metric.

ACCURACY(TP+TN)/All

How often is the classifier correct overall?

PRECISIONTP/(TP+FP)

When it predicts positive, how often is it right?

RECALLTP/(TP+FN)

Of all real positives, how many were found?

SPECIFICITYTN/(TN+FP)

Of all real negatives, how many were rejected?

F1 SCORE2PR/(P+R)

What is the harmonic balance of precision and recall?

BALANCED ACCURACY(Recall+Specificity)/2

How well are both classes recognized?

DETAILED EXPLANATION

Accuracy asks what fraction was correct. Precision asks how many predicted positives were truly positive. Recall asks how many real positives were found. Specificity asks how many real negatives were rejected. F1 balances precision and recall through their harmonic mean.

WORKED INTUITION

Fraud review capacity may prioritize precision, while safety screening may prioritize recall.

AI / PLACEMENT CONNECTION

Choose metrics from the decision cost, class prevalence and workflow capacity.

COMMON MISCONCEPTION

F1 ignores true negatives and may be inappropriate when correctly rejecting negatives is important.

CONCEPT 10

ROC-AUC and PR-AUC Across Thresholds

Ranking metrics summarize many thresholds but answer different questions.

ROC CURVETPR versus FPR

Useful for ranking; can look optimistic when negatives dominate.

PRECISION–RECALL CURVEPrecision versus recall

Often more informative when the positive class is rare.

Important: AUC does not choose an operating threshold and does not prove probabilities are calibrated.
DETAILED EXPLANATION

ROC curves compare true-positive and false-positive rates across thresholds; ROC-AUC measures ranking across random positive–negative pairs. Precision–recall curves focus on positive predictions and are often more informative when positives are rare. Neither curve selects the final operating threshold by itself.

WORKED INTUITION

Two models can have similar ROC-AUC but very different precision at the recall required by the application.

AI / PLACEMENT CONNECTION

Report the curve and a validated operating point aligned with business constraints.

COMMON MISCONCEPTION

AUC summarizes all thresholds, including many that would never be used in practice.

CONCEPT 11

Class Imbalance and Cost-Sensitive Decisions

A 99% accurate classifier can be useless when positives form only 1% and the model predicts everything negative.

STRATIFYPreserve evaluation composition

Use grouped or time-aware alternatives when the data structure requires them.

WEIGHTIncrease costly-error influence

Class weights change training loss, not the real data distribution.

RESAMPLETraining folds only

Oversampling, undersampling or SMOTE must remain inside validation.

THRESHOLDOptimize operating cost

Select with validation data and deploy an explicit policy.

DETAILED EXPLANATION

Imbalanced learning combines suitable splits, metrics, weighting, training-only resampling and threshold decisions. Class weighting changes the training objective; resampling changes the training distribution; thresholding changes the decision policy. These are related but not equivalent interventions.

WORKED INTUITION

A weighted loss can make rare fraud errors matter more without duplicating transactions.

AI / PLACEMENT CONNECTION

Probability calibration may be affected by resampling and should be checked on representative validation data.

COMMON MISCONCEPTION

Applying SMOTE or oversampling before the split leaks synthetic relatives into evaluation data.

CONCEPT 12

Multiclass Logistic Regression

Extend binary decisions to three or more mutually exclusive classes.

ONE-VS-RESTOne binary model per class

Compare each class against all others, then choose the strongest score.

SOFTMAX / MULTINOMIALJoint class probabilities

Exponentiate class scores and normalize so probabilities sum to one.

pₖ = eᶻᵏ / Σⱼeᶻʲ
MULTILABELSeveral labels may be true

Use independent outputs and thresholds; it is not ordinary multiclass classification.

DETAILED EXPLANATION

One-vs-rest trains one binary classifier per class and selects the strongest score. Multinomial logistic regression uses softmax so class probabilities compete and sum to one. The best choice depends on class relationships, solver support, data size and whether labels are mutually exclusive.

WORKED INTUITION

Handwritten digit recognition has ten mutually exclusive classes and naturally supports a softmax output.

AI / PLACEMENT CONNECTION

Softmax generalizes logistic normalization from two classes to several competing classes.

COMMON MISCONCEPTION

Multiclass and multilabel problems are different: multilabel samples can belong to several classes simultaneously.

CONCEPT 13

Probability Calibration

A calibrated model makes probability statements that match observed frequencies.

PERFECT CALIBRATIONpredicted probabilityobserved frequency
CHECKReliability diagram

Group predictions into probability bins and compare predicted with observed rate.

MEASUREBrier score / log loss

Evaluate probability quality, not only hard classifications.

REPAIRPlatt or isotonic calibration

Fit calibration on held-out or cross-validated predictions.

DETAILED EXPLANATION

A calibrated model assigns probabilities that match observed frequencies: among cases predicted near 0.7, roughly 70% should be positive. Reliability diagrams, Brier score and calibration error assess this property. Platt scaling or isotonic regression can recalibrate using separate validation evidence.

WORKED INTUITION

Two models may rank patients equally well, but only the calibrated model supports meaningful risk communication.

AI / PLACEMENT CONNECTION

Calibration matters when probabilities drive pricing, triage or resource allocation.

COMMON MISCONCEPTION

A model can have excellent AUC and poor calibration because ranking and probability accuracy are different.

CONCEPT 14

The CodeBhavya Classification Workflow

Separate model probability, decision policy and business outcome.

  1. 01
    Frame

    Define positive class, unit, cutoff time and error costs.

  2. 02
    Split

    Protect evidence with stratification, groups or time.

  3. 03
    Baseline

    Compare class and probability baselines.

  4. 04
    Pipeline

    Fit preprocessing inside validation folds.

  5. 05
    Train

    Optimize cross-entropy and tune regularization.

  6. 06
    Threshold

    Select an operating point using validation costs.

  7. 07
    Audit

    Inspect calibration, subgroups and failure cases.

  8. 08
    Monitor

    Track drift, prevalence, metrics and threshold impact.

model = Pipeline([
    ("prepare", preprocess),
    ("classifier", LogisticRegression(class_weight="balanced"))
])
model.fit(X_train, y_train)
probability = model.predict_proba(X_validation)[:, 1]
prediction = (probability >= chosen_threshold).astype(int)
DETAILED EXPLANATION

A responsible classification workflow defines the positive class and action, protects the split, creates a simple baseline, trains a leakage-safe probability model, evaluates ranking and calibration, chooses a threshold from costs, checks subgroups and monitors prevalence and error drift after deployment.

WORKED INTUITION

A fraud system may rank transactions, select a review threshold from daily capacity and maintain a second rule for immediate blocking.

AI / PLACEMENT CONNECTION

This workflow connects algorithm knowledge to placement-level system thinking.

COMMON MISCONCEPTION

An impressive offline score is insufficient when labels are delayed, users adapt or the decision changes the future data.

INTERACTIVE LEARNING • CODEBHAVYA PREMIUM VISUALIZER

🎬 Evidence to Classification — Visual Flow

Trace a classifier from observations and scores to sigmoid probability, threshold and evaluation.

LIVE
STEP 1 OF 7

Observe labeled examples

Training examples show two outcomes across the feature space.

Step 1 of 7
PROGRAM TRACING • TRUE NESTED-LOOP FLOW

Trace Logistic Regression from Scratch

Follow each epoch, sigmoid probability, error, accumulated gradient and parameter update.

CORE ALGORITHM

Batch Logistic Gradient Descent

The nested loop exposes every probability and gradient contribution.

x = [-1.0, 0.0, 1.0]
y = [0, 0, 1]
weight = bias = 0.0
rate = 0.5
for epoch in range(2):
    dw = db = 0.0
    for xi, yi in zip(x, y):
        score = weight * xi + bias
        probability = 1 / (1 + exp(-score))
        error = probability - yi
        dw += error * xi
        db += error
    weight -= rate * dw / len(x)
    bias -= rate * db / len(x)
print(round(weight, 2), round(bias, 2))
TRACE RESULT
0.32 -0.16
  • Sigmoid starts every probability at 0.5.
  • Positive and negative samples push the gradient differently.
  • Each batch update occurs only after all three samples.
  • The weight becomes positive for the increasing class pattern.
PROGRAMMING PROBLEMS • CODEBHAVYA PATTERN

Build Probability and Classification Skills

Attempt each problem independently. Workspaces and model programs remain collapsed initially.

0 / 5Solved independently0 / 500Best score
QUICK KNOWLEDGE CHECK

Test Classification Decisions

Select one answer per question. Results show your answer, the correct answer and an explanation.

Not checked yet
PLACEMENT CHECKPOINT

How Classification Appears in Hiring Rounds

Strong answers connect probability, threshold, metrics and business consequences.

ROUND 01

Math and Intuition

Explain sigmoid, log-odds and cross-entropy.

ROUND 02

Metric Calculation

Derive precision, recall, F1 and specificity from a matrix.

ROUND 03

Scenario Decision

Choose a threshold and metric for fraud, medicine or spam.

ROUND 04

Production Audit

Handle imbalance, calibration, drift and subgroup errors.

CodeBhavya interview pattern:Define positive class → Estimate probability → State threshold → Count outcomes → Choose metric → Explain cost → Validate.
INTERVIEW PREPARATION

🎤 Logistic Regression & Classification — Interview Questions

Answer aloud before selecting Show Answer for each explanation.

KEY TAKEAWAY

Responsible Classification in One View

1Score

Combine features into evidence.

→
2Probability

Map evidence through sigmoid.

→
3Decision

Apply a validated threshold.

→
4Evaluate

Measure the mistakes that matter.

A classifier is not only a model—it is a probability estimate plus a decision policy and its consequences.
EXTRA TIPS

Habits of Strong Classification Practitioners

01

Name the positive class explicitly before discussing precision or recall.

02

Inspect confusion matrices as counts and normalized rates.

03

Select thresholds on validation data—not the final test set.

04

Use PR curves and cost analysis for rare positive outcomes.

05

Check probability calibration before using risk scores operationally.

06

Monitor class prevalence because it can change precision after deployment.

EXTRA PRACTICE QUESTIONS

Strengthen Classification Reasoning

Explain the decision and consequence before writing code.

  1. 01

    Calculate sigmoid probabilities for scores −2, 0 and 2.

  2. 02

    Convert probabilities [0.2, 0.55, 0.8] using thresholds 0.5 and 0.7.

  3. 03

    Compute binary cross-entropy for y=1 and p=0.8.

  4. 04

    Derive logistic-regression gradients using the chain rule.

  5. 05

    Construct a confusion matrix from eight actual and predicted labels.

  6. 06

    Calculate accuracy, precision, recall, specificity and F1.

  7. 07

    Choose the primary metric for cancer screening and justify it.

  8. 08

    Explain why accuracy fails for 1% fraud prevalence.

  9. 09

    Compare ROC-AUC and PR-AUC for rare positives.

  10. 10

    Design a validation procedure for threshold selection.

  11. 11

    Compare one-vs-rest and multinomial logistic regression.

  12. 12

    Plan calibration and drift monitoring for a risk classifier.