Operating Systems Level 2
PART 1 • EXECUTION FOUNDATIONS

Inside the Kernel’s Protected Gateway

Understand what belongs inside a kernel, why kernels use different structures and exactly how a system call moves from an application to a protected service and back.

Level 02 of 15 Beginner → Intermediate 90–120 minutes Requires Level 1
BY THE END, YOU CAN

Reason beyond definitions

  • Separate the kernel from the complete OS.
  • Compare monolithic, microkernel, modular and hybrid designs.
  • Distinguish API, ABI and system-call interface.
  • Trace validation, dispatch, blocking and return.
  • Explain safety and performance trade-offs.
01 • DEFINE THE CORE

The Kernel Is the Privileged Core—not the Whole Operating System

The kernel stays active throughout system operation and controls resources that applications must not manipulate directly.

P

Process management

Creates execution contexts, schedules CPU time, switches contexts and coordinates termination.

M

Memory management

Maintains address spaces, mappings, permissions, allocation and virtual-memory state.

I

I/O management

Coordinates device drivers, controllers, interrupts, buffering and data transfer.

F

File systems

Maps file names and offsets to persistent storage while checking access permissions.

N

Networking

Implements protocol processing, sockets, routing decisions and interface coordination.

S

Protection

Validates identities, addresses, object handles and requested operations at trust boundaries.

02 • ORGANIZE PRIVILEGE

Kernel Architecture Decides Where Services Run

The central design question is not merely “large or small.” It is which services share a privileged address space and how components communicate.

MO

Monolithic kernel

Major services—including scheduling, memory, file systems and many drivers—execute together in kernel space. Direct calls can be fast, but a faulty privileged component can damage the whole system.

MI

Microkernel

Keeps only essential mechanisms in the kernel and moves more services into isolated user-space servers. Isolation and replaceability improve, while message crossings may add cost.

L

Layered design

Organizes the system into ordered levels where each layer uses lower-layer services. Reasoning is clearer, though strict layering can be difficult and inefficient.

H

Modular / hybrid

Combines a substantial privileged kernel with loadable modules or selected microkernel ideas, seeking practical performance and maintainability.

03 • INTERACTIVE COMPARISON

Explore Four Kernel Structures

Select a structure to see its placement of services and the trade-off an engineer must defend.

MONOLITHIC KERNEL

Fast internal cooperation inside one privileged space

04 • SEPARATE THE INTERFACES

API, ABI and System Call Solve Different Problems

These terms sit near one another in execution, but each is a different contract.

SOURCE CONTRACT

API

Names, types and behaviours a programmer uses in source code, such as a library’s function interface.

BINARY CONTRACT

ABI

Calling convention, register use, object format and binary rules that let compiled parts cooperate.

PROTECTED GATEWAY

System-call interface

Numbered kernel entry points and argument conventions used to request privileged services.

KERNEL WORK

Subsystem

The process, memory, file or network implementation that validates and performs the operation.

Source: printf("Hi") → library formats and buffers text → when required, a write system call requests kernel output. One API call is not guaranteed to equal one system call.
05 • CARRY THE REQUEST SAFELY

How System-Call Arguments Reach the Kernel

The system-call number identifies the operation; arguments describe the object and requested work.

Registers

A small number of values are placed directly in CPU registers. This is efficient but limited by the architecture’s register convention.

Memory block

The process stores arguments in a structure and passes its address. The kernel must safely copy and validate user memory.

User stack

Arguments are placed on the calling process’s stack according to a convention. The kernel cannot blindly trust the supplied values.

06 • INTERACTIVE DEEP TRACE

Follow a System Call One Kernel Decision at a Time

Choose an outcome and step through the register setup, privilege transition, validation, dispatch and return path.

STEP 1 OF 8 USER MODE

Request being prepared
07 • ORGANIZE THE SERVICE SURFACE

Major Categories of System Calls

Category Purpose Representative operations
Process control Create, execute, wait for and terminate processes. fork, execve, wait, exit
File management Open objects and transfer or update stored data. open, read, write, close
Device management Request device operations and control parameters. ioctl, read/write through device descriptors
Information Read or change process and system attributes. getpid, clocks, limits and metadata calls
Communication Create channels and exchange data. pipe, sockets, shared-memory operations
Protection Manage identities, permissions and ownership. permission, credential and access-control operations
08 • INTERACTIVE CLASSIFIER

Does This Operation Require a System Call?

Select an operation to inspect whether it can remain in user space and what condition changes the answer.

09 • DEFEND THE BOUNDARY

Every System Call Is Both a Service and an Attack Surface

Arguments originate in less-trusted user space, so correctness requires validation before privileged work.

01

Identify

Is the call number a supported operation?

02

Validate

Are sizes, flags, handles and pointers well formed?

03

Authorize

May this user and process access this object?

04

Perform

Execute while maintaining locks and invariants.

05

Return

Expose only the intended data and a defined status.

Performance cost

A call includes entry/exit work, validation and possible scheduling or device delay. Avoid unnecessary calls, but never bypass protection for speed.

Blocking versus non-blocking

A blocking call may move the process to waiting until progress is possible. A non-blocking call returns promptly, sometimes reporting “try again.”

10 • CHECK YOUR UNDERSTANDING

Ten Misconception-Specific Checks

Each choice explains the precise idea behind it.

Answered correctly: 0 of 10
11 • EXPLAIN & PREPARE

University and Placement Questions

2-MARK QUESTIONS
  1. Define kernel.
  2. What is an ABI?
  3. What is a system-call number?
  4. What is a loadable kernel module?
  5. Define a blocking call.
5-MARK QUESTIONS
  1. Compare monolithic and microkernel architectures.
  2. Explain a complete system-call lifecycle.
  3. Differentiate API, ABI and system-call interface.
  4. Describe system-call parameter passing.
  5. Classify system calls with examples.
INTERVIEW QUESTIONS
  1. Why can’t the kernel trust a user pointer?
  2. Does printf always make one write call?
  3. Why might a microkernel be slower?
  4. Can a system call cause a context switch?
  5. What happens when a syscall blocks?
Show a strong comparison: monolithic kernel versus microkernel
  1. Define the boundary: which services remain privileged?
  2. Explain communication: direct internal calls versus messages between isolated components.
  3. Compare failure isolation, performance, maintainability and trusted-code size.
  4. Avoid declaring a universal winner; connect the decision to system goals and implementation.
  5. Give examples only after explaining the architectural principle.
Show a strong answer: what if a system call blocks?

The kernel records what the process is waiting for, changes it from running to waiting and lets the scheduler select another ready process. When the event completes, an interrupt or another kernel action makes the process ready again. It resumes only after the scheduler later assigns it CPU time; “ready” does not mean “currently running.”

LEVEL 2 SUMMARY

You Can Now Explain the Protected Kernel Gateway

  • The kernel is the privileged resource-control core within the wider operating system.
  • Kernel architectures trade direct performance against isolation, structure and replaceability.
  • API, ABI and system-call interface are separate contracts at source, binary and privilege boundaries.
  • A call prepares arguments, enters the kernel, validates and dispatches work, then returns or blocks.
  • Kernel code must treat every user-provided pointer, size, flag and handle as untrusted.
COURSE CHECKPOINT

Mark this level after you can trace one successful call and one blocked or rejected call aloud.

Saved only in this browser.