Computer Vision & Convolutional Neural Networks

From Scene to Numbers

A digital image samples light on a rectangular grid. In machine learning, a batch is commonly represented as N × C × H × W: number of images, channels, height and width. Libraries may also store images as H × W × C, so shape awareness is essential.

• Grayscale: one intensity channel.
• RGB: red, green and blue channels.
• Normalization: rescales values or standardizes each channel.
• Resolution: controls detail, computation and memory.

How a Feature Map Is Calculated

Place the kernel over an image patch, multiply aligned entries, add the products and bias, then move the kernel. Weight sharing makes convolution far more parameter-efficient than connecting every pixel to every output neuron.

Y[i,j] = b + Σu Σv X[i+u,j+v]K[u,v]

• Kernel size: local field inspected in one operation.
• Stride: number of pixels moved between outputs.
• Padding: values added around borders.
• Dilation: spacing between kernel elements.

Why Convolution Alone Is Not Enough

Convolution is linear. Nonlinear activations allow layered feature composition, while pooling or strided convolution reduces spatial size and increases effective context. Modern networks often preserve more spatial detail early and reduce it deliberately later.

• ReLU: clips negative responses and introduces nonlinearity.
• Max pooling: keeps the strongest value in each window.
• Average pooling: summarizes the regional mean.
• Global average pooling: converts each final feature map into one number.

Three Calculations for Every Layer

For each convolution, calculate output shape, trainable parameters and receptive field. A standard convolution with Cin input channels, Cout filters and a K×K kernel contains (K²Cin + 1)Cout parameters when each filter has a bias.

• Output size: depends on input, padding, dilation, kernel and stride.
• Parameters: depend on channels and kernel—not image width and height.
• Receptive field: grows across stacked operations.
• Activation memory: often dominates training memory for large batches.

Learn the Right Invariances

Augmentation generates plausible training variations such as crops, flips, colour shifts or rotations. Transformations must preserve the label and update boxes or masks consistently. Transfer learning starts from features learned on a larger source dataset and adapts them to the target problem.

• Feature extraction: freeze the backbone and train a new task head.
• Fine-tuning: update some or all pretrained layers using a smaller learning rate.
• Domain shift: source and target images may have different visual statistics.
• Leakage safety: split related patients, videos or objects before augmentation.

Three Different Prediction Contracts

• Classification: one or more labels for the complete image.
• Object detection: class scores and bounding boxes for multiple objects.
• Semantic segmentation: a class prediction for every pixel.
• Instance segmentation: separate masks for individual object instances.

Accuracy may suit balanced classification, while detection often uses intersection-over-union and mean average precision. Segmentation commonly uses IoU or Dice score because foreground pixels can be rare.

Practical PyTorch Pattern

model.train()
for images, labels in train_loader:
    images, labels = images.to(device), labels.to(device)
    optimizer.zero_grad()
    logits = model(images)
    loss = criterion(logits, labels)
    loss.backward()
    optimizer.step()

Monitor class-wise errors, confusion matrices, data quality, calibration and robustness to lighting, blur, crop and domain shift. Visual inspection of failures is essential because aggregate metrics can hide systematic problems.