PART 1 โข FOUNDATIONS โข LEVEL 02
Python, NumPy & Data Tools
Transform raw values into reliable, model-ready arrays and tables. Learn to reason about shape, vectorize calculations, broadcast safely, explore data and build reproducible workflows.
CODEBHAVYA DATA LAB
>>> X.shape
(4, 3)
>>> X.mean(axis=0)
array([4.5, 5.5, 6.5])
>>> (X - X.mean(0)) / X.std(0)
array([[-1.34, -1.34, -1.34], ...])LISTโARRAYโINSIGHTโMODEL
LEARNING OUTCOMES
By the End of This Level, You Can
QUICK REVISION
Python Skills Used in Every Data Workflow
Recall these language tools before moving into numerical arrays.
Store ordered values; slicing uses start, stop and step.
Represent named fields and key-value records.
Express small transformations clearly, but do not replace vectorization.
Isolate transformations and make pipelines testable.
Use explicit dependencies and consistent package versions.
CONCEPT 01
Why NumPy Is the Numerical Foundation of AI
Python lists are flexible containers. NumPy arrays are compact, typed, multidimensional structures built for numerical computation.
values = [10, 20, 30]
doubled = [x * 2 for x in values]- Can mix object types
- Loops execute at Python level
- No native multidimensional shape
VS
values = np.array([10, 20, 30])
doubled = values * 2- Usually one efficient dtype
- Vectorized compiled operations
- Shape and axis are explicit
Engineering rule: Prefer clear vectorized operations for bulk numerical work, but never sacrifice correctness or create enormous temporary arrays only to avoid a small loop.
NumPy stores homogeneous numerical values in compact multidimensional arrays and executes operations in optimized compiled code. This makes whole-array computation faster and clearer than repeatedly interpreting Python loops. Most ML libraries exchange data using array-like tensors with the same shape-based reasoning.
Adding 10 to one million measurements is expressed as one array operation rather than one million Python statements.
Vectorization is the bridge between mathematical notation and efficient model implementation.
Vectorized code can still create large temporary arrays. Speed does not remove the need to reason about memory.
CONCEPT 02
Anatomy of an ndarray
An array is described by its values, axes, shape, size and data type.
axis 1 โ features
102030401224364814284256
axis 0 โ samples
An ndarray is defined by its shape, number of dimensions, data type, size and memory strides. Shape tells how many values exist along each axis. Strides tell how many bytes must be moved to reach the next element along an axis, which explains why some views are inexpensive.
A dataset with 100 samples and 4 features normally has shape (100, 4): axis 0 selects samples and axis 1 selects features.
Most array errors in interviews and models are shape errors rather than arithmetic errors.
A one-dimensional shape (4,) is not the same as a row matrix (1, 4) or column matrix (4, 1).
CONCEPT 03
Data Types, Memory & Precision
The dtype controls representation, range, precision, memory and compatible operations.
int32 / int64Whole numbersCounts, IDs and encoded categories. Watch integer overflow in limited types.
float32ML default in many modelsLower memory and faster accelerator computation, with reduced precision.
float64Higher precisionCommon NumPy default for decimals and scientific calculations.
boolMasks and conditionsStores True/False values for filtering and logical operations.
array.astype(np.float32)creates a converted array; confirm precision and memory trade-offs before converting.The dtype determines representation, range, precision and memory per value. Integers cannot represent fractions; floating-point numbers approximate real values with finite precision. Smaller types save memory and may accelerate hardware, but they can overflow or lose useful detail.
One million float64 values use about 8 MB, while float32 uses about 4 MB. The smaller type is common in neural networks.
Mixed precision can accelerate training, while stable accumulation may still require a higher-precision type.
Do not compare floating-point results using exact equality after several calculations; use a tolerance such as np.allclose.
CONCEPT 04
Create Arrays Intentionally
Choose a constructor that communicates the structure you need.
np.array(data)Convert an existing sequence.
np.zeros(shape)Initialize with zeros.
np.ones(shape)Initialize with ones.
np.arange(start, stop, step)Even integer-like spacing.
np.linspace(start, stop, count)A fixed number of evenly spaced values.
rng.normal(size=shape)Reproducible random samples through a generator.
Array constructors should communicate intent: array converts known values, zeros and ones initialize storage, arange creates step-based sequences, linspace creates an exact count across an interval, and random generators create controlled samples. Always verify the resulting shape and dtype.
np.linspace(0, 1, 5) gives five evenly spaced values including both endpoints, which is useful for evaluating a model on a grid.
Explicit construction makes experiments reproducible and prevents silent integer or shape assumptions.
Floating steps with arange can produce surprising endpoints because binary floating-point cannot represent every decimal exactly.
CONCEPT 05
Indexing, Slicing & Boolean Masks
Select data by position or condition while keeping shape and copy behaviour visible.
X[1, 2]Row 1, column 2.
X[0:2, :]First two rows, all columns.
X[:, 1]All rows from column 1.
X[X > 30]Flattened values satisfying a condition.
X[X[:, 0] > 11]Rows selected using the first feature.
X[[0, 2]]Copy selected rows by index.
View vs copy: Basic slicing often returns a view that shares memory. Fancy indexing and boolean indexing generally return copies. Use
.copy() when independent data is required.Indexing selects individual positions; slicing selects ranges; boolean masks select values satisfying a condition. Basic slicing usually returns a view that shares memory, while advanced or boolean indexing commonly returns a copy. Combining row and column conditions requires careful shape reasoning.
X[X[:, 2] > 10] selects complete rows whose third feature exceeds 10.
Masks are used for filtering invalid rows, defining subgroups and evaluating model errors on important cases.
Changing a view may change the original array. Use copy explicitly when independent data is required.
CONCEPT 06
Vectorization: Think in Whole Arrays
Express numerical intent over batches instead of repeatedly interpreting Python statements.
scaled = []
for value in values:
scaled.append((value - mean) / std)Correct, but Python coordinates every iteration.
scaled = (values - mean) / stdShorter, clearer and normally much faster for numerical arrays.
Same asymptotic work: O(n)Different constant cost: vectorized operations execute optimized loops outside Python.
Vectorization expresses an operation over an entire axis or array. NumPy then performs the repetitive work in optimized loops. The goal is not merely fewer lines: it is to expose the mathematical structure, reduce interpreter overhead and enable efficient low-level execution.
Predictions for all samples are X @ weights + bias instead of manually computing one weighted sum per row.
Linear regression, neural layers and distance calculations are naturally expressed as vectorized operations.
A dense vectorized expression that duplicates huge arrays may be worse than a clear chunked computation.
CONCEPT 07
Broadcasting Rules
Broadcasting performs elementwise operations on compatible shapes without physically copying every repeated value.
Start from the rightmost dimension and move left.
Two dimensions are compatible when equal.
A dimension of 1 can expand conceptually.
This enables scalars, row vectors and column vectors.
(3, 4) + (4,)โ โ (3, 4)(3, 1) + (1, 4)โ โ (3, 4)(3, 4) + (3,)โ incompatibleBroadcasting aligns shapes from the right. Dimensions are compatible when they are equal or one of them is 1; missing leading dimensions are treated as 1. The smaller operand is conceptually expanded without necessarily copying its data.
Subtracting a feature-mean vector of shape (4,) from data of shape (100, 4) centers every column.
Broadcasting supports normalization, bias addition and batch operations throughout ML.
An operation can broadcast successfully but along the wrong semantic axis. A valid shape is not proof of correct meaning.
CONCEPT 08
Aggregation and the Meaning of axis
The selected axis is reduced; the other axes remain in the result.
123456
X.sum()21Reduce every axis to one scalar.
123456
X.sum(axis=0)[5, 7, 9]Collapse rows; one result per column.
123456
X.sum(axis=1)[6, 15]Collapse columns; one result per row.
Aggregations reduce one or more axes. For a matrix, axis 0 removes the row dimension and summarizes each column; axis 1 removes the column dimension and summarizes each row. keepdims retains a length-one axis so later broadcasting remains explicit.
X.mean(axis=0, keepdims=True) produces one mean per feature with shape (1, features).
Batch losses, feature statistics and normalization depend on choosing the intended reduction axis.
Memorizing โaxis 0 means columnsโ is fragile. Ask which axis disappears from the result.
CONCEPT 09
Reshape, Transpose, Concatenate & Stack
Shape transformations change how values are organized. Always verify the result before supplying it to a model.
x.reshape(2, 3)Change shape while preserving element count.
X.TReverse axes for a 2D matrix.
np.concatenate([A, B], axis=0)Join existing arrays along an existing axis.
np.stack([A, B], axis=0)Create a new axis and combine arrays.
x.reshape(-1, 1)Infer row count and create one feature column.
np.squeeze(x)Remove axes whose length is one.
Reshape changes the logical arrangement without changing element count. Transpose reorders axes. Concatenate joins arrays along an existing axis, while stack creates a new axis. These operations determine how samples, features, channels and time steps are represented.
Stacking three vectors of shape (4,) along axis 0 produces a matrix of shape (3, 4).
Correct shape transformations are essential before feeding image, sequence or tabular batches into a model.
Reshape may return a view or copy depending on memory layout; do not assume it is always free.
CONCEPT 10
Pandas: From Records to Model-Ready Tables
A DataFrame adds named columns and indexes around typed arrays, making tabular analysis expressive.
pd.read_csv(...)Specify paths, types and missing markers.
df.info()Check shape, types, nulls and sample rows.
df.loc[rows, cols]Choose rows and columns explicitly.
fillna / dropnaHandle missingness using domain reasoning.
groupby / aggCompare categories and validate assumptions.
to_csv(...)Save clean artifacts with reproducible code.
| student_id | hours | attendance | score |
|---|---|---|---|
| CB101 | 2.5 | 78 | 52 |
| CB102 | 4.0 | 91 | 73 |
| CB103 | โ | 86 | 68 |
df.isna().sum()reveals one missing value in hours. Investigate why it is missing before choosing a treatment.Pandas adds labelled rows and columns, heterogeneous types, missing-value handling, joins and group operations. A DataFrame should be treated as a table with a schema: each column needs a meaning, unit, valid range and expected availability time.
Group sales by region, aggregate count and mean, then join the summary to a reference table using a validated key.
Most tabular ML pipelines begin with Pandas for inspection and end with numeric arrays for modelling.
Index alignment can silently produce missing values when combining Series with different labels.
CONCEPT 11
Exploratory Data Analysis Is a Reliability Check
EDA is not decoration. It discovers data problems, distributions, relationships and dangerous assumptions before modelling.
Rows, columns, types, keys and units.
Count, pattern and reason for absent values.
Range, centre, spread, skew and unusual values.
Associations, redundancy and possible leakage.
Balance, noise, ambiguity and time behaviour.
Coverage and performance-relevant differences.
Histogram distributionBox plot spread/outliersScatter plot two numeric variablesBar chart category comparisonHeatmap matrix patterns
EDA tests whether the data can support the intended decision. Inspect distributions, missingness, duplicates, class balance, impossible values, target relationships and subgroup differences. Every chart should answer a question or expose an assumption that may affect modelling.
A sudden cluster of zero ages may represent missing values encoded incorrectly, not real newborn customers.
EDA guides cleaning, splitting, baseline selection and later monitoring checks.
Exploring the final test target influences modelling choices and leaks evaluation information.
CONCEPT 12
Reproducible Data Workflows
A result is trustworthy only when another run can recreate it from documented inputs and code.
rng = np.random.default_rng(42)Do not manually edit source files between runs.
Record Python and package versions.
Assert required columns, shapes, ranges and uniqueness.
Convert notebook experiments into reusable transformations.
Save derived datasets separately with lineage.
A reproducible workflow fixes random seeds where appropriate, records versions and parameters, preserves raw data, scripts transformations and validates schemas. Reproducibility means another run can reconstruct the same reasoning and resultโnot only that one random number repeats.
A saved pipeline contains imputation, scaling and model steps so prediction uses exactly the transformations fitted during training.
Reliable experiments and production debugging depend on traceable data and code versions.
A seed alone cannot guarantee identical results across different hardware, parallel execution or library versions.
INTERACTIVE LEARNING โข CODEBHAVYA PREMIUM VISUALIZER
LIVE๐ฌ NumPy Broadcasting โ Visual Flow
Watch a column vector and row vector expand conceptually into a 3 ร 4 result.
STEP 1 OF 7
Create a row vector
Start with four feature values arranged across columns.
Step 1 of 7
PROGRAM TRACING โข LINE BY LINE
Trace Vectorized Standardization
Follow a NumPy array through mean, standard deviation, centring, scaling, masking and output.
Press Next to execute the first statement.
EXPRESSION
โOUTPUT
Waiting for print(...)
Step 0 of 9
CORE DATA PATTERN
Standardize and Select an Array
Standardization expresses each value as the number of standard deviations above or below the mean.
import numpy as np
scores = np.array([40., 55., 70., 85.])
mean = scores.mean()
std = scores.std()
centered = scores - mean
scaled = centered / std
mask = scores >= 70
selected = scaled[mask]
print(np.round(selected, 2))STANDARD SCORE
z = (x โ ฮผ) / ฯ- ฮผ: mean of the feature.
- ฯ: standard deviation of the feature.
- z = 0: the value equals the mean.
- Positive z: the value is above the mean.
- Important: in ML, fit ฮผ and ฯ on training data only.
PROGRAMMING PROBLEMS โข CODEBHAVYA PATTERN
Practise Array and Data Reasoning
Attempt each problem independently. The checker verifies required NumPy/Pandas logic and exact task structure.
0 / 5Solved independently0 / 500Best score
QUICK KNOWLEDGE CHECK
Test Shape, Axis and Data Reasoning
Select one answer for every question. Results show your answer, the correct answer and an explanation.
Not checked yet
PLACEMENT CHECKPOINT
How Data-Stack Skills Appear in Hiring Rounds
Expect shape reasoning, transformations, debugging and compact Pandas tasksโnot only syntax definitions.
Predict the Shape
Trace slicing, reshape, transpose and broadcasting without running code.
Vectorize a Loop
Replace repeated Python operations with a correct array expression.
Clean a Table
Inspect missing values, duplicates, types and invalid ranges.
Explain the Pipeline
Defend transformations, train-only statistics and reproducibility choices.
CodeBhavya interview pattern:State the input shape โ Apply the operation โ State the output shape โ Explain memory/copy behaviour โ Mention one failure case.
INTERVIEW PREPARATION
๐ค Python, NumPy & Data โ Interview Questions
Answer aloud before opening each explanation.
KEY TAKEAWAY
The Data Workflow in One View
Know shape, dtype, missingness and meaning.
Use explicit vectorized and tabular operations.
Check outputs, invariants and train-only statistics.
Preserve code, versions, seeds and lineage.
In AI, most silent failures begin before model trainingโwith misunderstood shape, contaminated data or an irreproducible transformation.
EXTRA TIPS
Habits of Reliable Data Practitioners
Print or assert shape after every important transformation while learning.
Use axis only after saying in words which dimension must disappear.
Avoid chained Pandas assignment; use explicit .loc operations.
Fit scaling, imputation and encoding statistics on training data only.
Use np.allclose for floating-point array comparisons.
Profile before optimizing; readable vectorization is better than clever expressions.
EXTRA PRACTICE QUESTIONS
Strengthen Array and Data Thinking
Predict the result first, then verify it in Python.
- 01
Create a 3 ร 4 array containing the numbers 1 through 12.
- 02
Predict the shape of
X[:, 1:3]whenX.shape == (5, 4). - 03
Replace a loop that squares every value with a vectorized expression.
- 04
Explain why shapes
(5, 3)and(5,)cannot broadcast. - 05
Calculate column means while retaining a 2D shape using
keepdims. - 06
Filter all rows whose second feature is greater than 10.
- 07
Reshape 24 values into every possible two-dimensional shape.
- 08
Demonstrate the difference between concatenate and stack.
- 09
Find missing-value counts and percentages for every DataFrame column.
- 10
Group sales by region and calculate count, mean and maximum.
- 11
Write assertions for required columns, non-negative values and unique IDs.
- 12
Explain why setting a random seed is necessary but not always sufficient for reproducibility.