PART 4 • DEEP & GENERATIVE AI • LEVEL 22
Generative AI, RAG & Agents
Understand how generative models create new data, ground language models with retrieved evidence and build bounded agents that plan, call tools, verify results and stop safely.
GROUNDING CONTRACTretrieve evidence • use allowed tools • cite support • stop safely
answer = generate(query, evidence, observations)LEARNING OUTCOMES
By the End of This Level, You Can
QUICK REVISION
Six Building Blocks of Applied Generative AI
A dependable system separates model generation from evidence, actions, policies and verification.
A model learns a data distribution and produces text, images, audio, code or structured values.
Instructions, examples, context and output constraints supplied at inference time.
Searches an indexed knowledge collection using lexical, vector or hybrid relevance.
Constrains the response to available evidence and exposes citations or provenance.
A validated function or service for calculation, lookup, database access or workflow execution.
Plans, acts, observes, updates state, verifies completion and stops within limits.
CONCEPT 01
Generative Models Learn How Data Could Be Produced
Different model families represent uncertainty and create samples through different training signals and inference procedures.
Factorizes a joint distribution into conditional predictions. LLMs and many audio models generate sequentially.
Balances reconstruction with a regularized latent distribution that supports smooth sampling.
Trains a generator to fool a learned critic, often producing sharp samples but requiring careful stabilization.
Learns iterative denoising steps that transform noise into a sample conditioned on text or other signals.
A generative model estimates or approximates how observations arise. Autoregressive models explicitly decompose probability across an ordered sequence. VAEs learn an encoder, a probabilistic latent distribution and a decoder while optimizing reconstruction plus KL regularization. GANs define an adversarial game rather than an explicit likelihood. Diffusion models corrupt data through a known forward process and learn a reverse denoising process. Architecture choice depends on modality, likelihood needs, controllability, sample quality, latency and available data—not on novelty alone.
A VAE learns a smooth neighbourhood where nearby latent points decode to related samples; a GAN learns a direct noise-to-sample mapping; diffusion refines noise through many steps.
Compare families using objective, latent representation, inference procedure, training stability and typical applications.
“Generative” means producing samples from a learned distribution; it does not imply creativity, truth, ownership or safe use.
CONCEPT 02
An LLM Application Is More Than One Model Call
The product boundary includes input validation, context construction, model configuration, tools, post-processing and human fallback.
Authenticate, classify risk and sanitize untrusted inputs.
Assemble instructions, user request, evidence and history.
Run a versioned model with controlled decoding.
Check schema, citations, claims, policy and tool results.
Return, abstain, retry safely or escalate to a human.
A production LLM receives several instruction sources with different trust levels. System rules define the application boundary, developer instructions define workflow, user content requests a task, retrieved documents provide evidence and tool results update state. These must remain distinguishable so untrusted text cannot silently become authority. Structured-output validation, timeouts, access control, retries and observability surround the model call. Reliability therefore belongs to the complete system, not only the model’s benchmark score.
A support assistant may retrieve a refund policy, but text inside that policy must remain evidence—not executable instruction telling the system to reveal customer data.
Draw the full request path and label trust boundaries, stored data, external calls, fallbacks and evaluation points.
A long system prompt cannot replace server-side authorization, deterministic validation or safe tool design.
CONCEPT 03
Prompting Is Context Engineering, Not Hidden Retraining
Good prompts define the task, evidence policy, output contract and failure behavior using information available for the current request.
What must be done?State the user-facing objective, permitted transformations and decision boundary.
What evidence is available?Provide relevant data with clear delimiters, labels and provenance.
What must not happen?Define scope, privacy, citation, refusal and escalation requirements.
What must be returned?Use a schema, fields, types, examples and validation rules where downstream code depends on structure.
Zero-shot prompting supplies an instruction; few-shot prompting adds representative demonstrations; decomposition breaks a task into explicit stages; structured outputs constrain machine-consumed responses. Prompt quality depends on precise task definitions, not decorative phrases. Put stable rules before untrusted content, delimit external evidence, define how to handle missing support and validate the output outside the model. Prompt versions should be tested like code because small wording changes can alter behavior.
Role and goal → trusted rules → task data → retrieved evidence → output schema → abstention rule. Keep each block labelled and test conflicting cases.
When asked about prompt engineering, include evaluation, versioning, injection resistance and schema validation—not only wording tips.
Asking a model to “think step by step” is not a substitute for observable intermediate state, tool validation or reproducible evaluation.
CONCEPT 04
RAG Retrieves Evidence Before Generation
Retrieval-augmented generation connects a model to refreshable knowledge, but relevance, support and citation must be measured separately.
documents + metadataParse sources, retain permissions, version and provenance.
meaningful passagesChoose boundaries and overlap that preserve answerable context.
lexical + vectorsStore searchable representations with filters.
rank(query, chunks)Generate candidates, rerank and enforce access.
answer + citationsGenerate only from selected evidence and verify claim support.
Lexical retrieval such as BM25 rewards exact term matches with document-length normalization. Dense retrieval compares learned embeddings and can connect paraphrases. Hybrid retrieval combines complementary candidates; a reranker then scores each query–chunk pair more precisely. Chunk size controls a central trade-off: small chunks are precise but may lose context, while large chunks contain more context but dilute relevance and consume the model window. Metadata filters must enforce access before results reach the model. Retrieved context improves evidence availability but does not guarantee that the final answer uses it correctly.
For “Who can attend the premium lab?”, a chunk containing exact eligibility rules should outrank a general page that repeatedly mentions “premium” without the condition.
Separate retrieval recall, reranking precision, context relevance, answer correctness, citation correctness and claim support.
Vector search is not a knowledge guarantee. The correct document may be missing, inaccessible, poorly chunked, badly embedded or displaced by irrelevant results.
PREMIUM COMPUTATIONAL VISUALIZER
CodeBhavya • Retrieve, Rank, Ground📚 RAG Retrieval & Grounding Workbench
Tokenize a real mini-knowledge base, calculate TF-IDF cosine and BM25 evidence, combine hybrid scores, rank chunks, assemble context and produce a citation-bound answer.
CONCEPT 05
Grounding Requires Claim-Level Support and Honest Abstention
A citation is useful only when the cited source actually supports the nearby claim and the system exposes uncertainty or missing evidence.
Measure whether selected chunks contain information needed for the question.
Compare against a trusted reference, rubric or expert judgment.
Check each factual claim against the provided evidence rather than general plausibility.
Verify citation existence, placement, source authority and direct entailment.
Hallucination is not one failure. A retrieval miss leaves the model without evidence; a context-use failure ignores available support; a synthesis error combines facts incorrectly; a citation error points to a source that does not entail the claim. Robust RAG prompts require the model to distinguish supported, conflicting and absent evidence. Claim-level verification can split an answer into factual units and evaluate each against cited chunks. When evidence is insufficient, abstention or a clarifying question is more reliable than fluent completion.
If two policy versions disagree, the system should surface the version dates and conflict rather than silently choose the more convenient sentence.
Describe an evaluation dataset with answerable, unanswerable, conflicting, access-restricted and adversarial-document cases.
Showing any source link does not make an answer grounded. Support must be direct, current, authorized and connected to the exact claim.
CONCEPT 06
Agents Use Models to Control Bounded Tool-Execution Loops
An agent is a system that maintains state, selects actions, observes results and decides whether to continue—not simply a chatbot with a long prompt.
Parse goal, constraints, permissions and completion condition.
Choose the next useful step or a short explicit sequence.
Call one permitted tool using validated arguments.
Store the result, error or uncertainty in working state.
Check goal, evidence, budget and stop conditions.
Tool definitions should expose a narrow name, description and typed input schema. The model proposes a call, but application code validates authorization and arguments before execution. Observations return as untrusted data. Working memory stores current state; durable memory should be explicit, minimal, consented and retrievable. Planning can be reactive, plan-first or hierarchical, but every design needs maximum steps, timeouts, cost budgets, idempotency for retries and human approval for high-impact actions.
A placement agent may read eligibility rules and student records, calculate eligibility, then request coordinator approval. It must not send applications merely because a model inferred permission.
Explain exactly what the model decides, what deterministic code enforces and which actions require a human checkpoint.
More autonomy is not automatically more intelligence. Unbounded retries, broad tools and unclear goals increase cost and operational risk.
PREMIUM COMPUTATIONAL VISUALIZER
CodeBhavya • Plan, Act, Observe, Verify🧭 Agent Planning & Tool-Execution Laboratory
Give a bounded agent a goal, compare reactive and plan-first control, inspect tool arguments and observations, enforce a step budget and verify completion before returning an answer.
CONCEPT 07
Agent Security Begins with Least Privilege and Untrusted Observations
Every prompt, retrieved document, webpage and tool result can contain malicious or misleading instructions.
External content attempts to override trusted instructions or redirect tool use.
Tools permit broader actions or data access than the task requires.
Generated parameters target unauthorized records, paths, recipients or quantities.
Sensitive context is copied into outputs, logs or external tools.
The agent repeats actions, ignores errors or spends budget without progress.
A plugin, model, index, tool or dependency changes outside the tested boundary.
Trusted instructions and untrusted observations must travel through separate channels. Tool authorization belongs in deterministic application code using the actual user identity and resource policy. Arguments require schema, range and target validation. Side-effecting tools need idempotency keys, previews, confirmation or approval depending on risk. Treat retrieved text as data even when it says “ignore previous instructions.” Maintain a complete audit trail of decisions, calls, results and policy blocks without logging unnecessary sensitive content.
Prefer “get_student_eligibility(student_id)” over a general database-query tool, and return only fields required for the decision.
Threat-model assets, actors, entry points, trust boundaries, permissions, side effects and recovery before selecting an agent framework.
Prompt injection cannot be solved only by telling the model not to follow malicious instructions; architecture and authorization must limit impact.
CONCEPT 08
Evaluation Must Follow the Complete Generative System
Measure component quality, end-to-end task success, safety, latency, cost and recovery using reproducible traces.
Task correctness, completeness, groundedness, format validity, tone and abstention quality.
Recall@K, MRR, NDCG, context relevance, permission correctness and source freshness.
Goal completion, tool selection, argument validity, step count, recovery and human approvals.
Injection resistance, data exposure, prohibited actions, over-refusal, bias and escalation.
Time to first token, total latency, tokens, tool duration, errors, cost and availability.
User effort, appeal paths, decision quality, accessibility and consequences across groups.
Offline evaluation should freeze the corpus, index, embedding model, prompts, model, tools and decoding configuration. Component tests diagnose where an end-to-end failure originates. Trace-based agent evaluation checks whether intermediate actions were valid even when the final answer happens to be correct. Online experiments require guardrails and rollback, while production monitoring watches input drift, retrieval changes, tool errors, unsafe behavior and feedback. Automated judges can scale review, but they must be calibrated against humans and tested for position, verbosity and self-preference bias.
Define minimum quality and maximum safety, latency and cost thresholds before evaluating a candidate version.
Propose a failure taxonomy and show how each logged trace field supports diagnosis rather than collecting data without purpose.
A successful demo is not a production evaluation. Real distributions include missing evidence, conflicting instructions, tool failures and adversarial inputs.
ALGORITHM MAPS
Applied GenAI Logic Before Frameworks
Use these compact procedures for implementation, debugging and interviews.
- Parse, authorize, chunk and version sources.
- Create lexical and/or vector indexes.
- Transform the question and apply access filters.
- Retrieve candidates, rerank and build context.
- Generate with citations and verify claim support.
- Calculate lexical relevance for every chunk.
- Calculate embedding similarity independently.
- Normalize scores to comparable ranges.
- Fuse ranks or weighted scores.
- Return Top-K, then measure recall and precision.
- Define goal, state, tools and stop conditions.
- Select a next action within permissions.
- Validate and execute exactly one tool call.
- Store observation and update working state.
- Verify goal or stop at budget and escalate.
- Define task, risk and component metrics.
- Create normal, slice, failure and attack cases.
- Freeze every model and system version.
- Run traces and score end-to-end outcomes.
- Diagnose, gate release, monitor and roll back.
PROGRAM TRACING • TRUE NESTED-LOOP EXECUTION
Trace Hybrid RAG Retrieval from First Principles
Follow token counts for every document, document frequency, IDF, query and chunk vectors, cosine numerators, BM25 term contributions and final fused ranking. The cursor returns through every loop exactly as Python executes.
Press Next to evaluate the first statement.
EXPRESSION
—OUTPUT
Waiting for print(...)
Step 0
PROGRAMMING PROBLEMS • CODEBHAVYA PATTERN
💻 Generative AI, RAG & Agent Challenges
Attempt each program independently. Workspaces, hints and model programs remain collapsed initially.
0 / 5Solved independently0 / 500Best score
QUICK KNOWLEDGE CHECK
Test Your Grounded-AI Reasoning
Select one answer per question. Results show your choice, the correct answer and a clear explanation.
Not checked yet
PLACEMENT CHECKPOINT
Diagnose GenAI Systems Like an Applied AI Engineer
Locate the failed component before changing models, prompts or frameworks.
Separate retrieval miss, context-use failure, synthesis error and citation mismatch.
Inspect ingestion, parsing, chunk boundary, metadata filter, query rewriting and candidate recall.
Check state updates, error classification, progress signals, idempotency and maximum-step termination.
Improve tool boundaries and descriptions, validate arguments and compare tool-selection traces.
Separate trust channels, restrict permissions, validate targets and treat observations as untrusted data.
Measure retrieval, prompt, output, tool and retry costs separately; route, cache and reduce context carefully.
CodeBhavya interview pattern:Define user goal → Identify trusted and untrusted data → Choose generation and evidence sources → Design retrieval and tools → Enforce permissions and budgets → Trace intermediate state → Verify claims and actions → Evaluate components and end-to-end task → Monitor failures and cost → Provide fallback and human control.
INTERVIEW PREPARATION
🎤 Generative AI, RAG & Agents — Interview Questions
Answer aloud before selecting Show Answer for each detailed explanation.
KEY TAKEAWAY
Reliable Generation Requires Evidence and Bounded Action
State goal, risk and output contract.
Select authorized relevant evidence.
Create a constrained candidate response.
Use narrow validated tools if needed.
Check support, outcome and stop state.
Generative AI becomes dependable when every claim has evidence, every tool has boundaries, every loop has a budget and every important failure has a safe fallback.
EXTRA TIPS
Eight Practical Applied-GenAI Habits
Establish a deterministic baseline before adding an agent or larger model.
Version documents, chunks, embeddings, prompts, tools and model settings together.
Evaluate retrieval independently before judging the generated answer.
Require citations only when the system can verify that they support nearby claims.
Give each tool the minimum data access and action scope required.
Use structured state and schemas instead of depending on conversational memory.
Set maximum calls, time, tokens and cost before an agent begins.
Keep abstention, human approval, retry and rollback paths visible and tested.
EXTRA PRACTICE QUESTIONS
Strengthen Applied GenAI Reasoning
Calculate intermediate values and defend each retrieval, grounding and agent-control decision.
- 01
Compare autoregressive, VAE, GAN and diffusion objectives.
- 02
Design a structured-output schema for placement eligibility.
- 03
Write a prompt that distinguishes trusted rules from retrieved evidence.
- 04
Calculate TF-IDF cosine for a query and three chunks.
- 05
Calculate one BM25 term contribution by hand.
- 06
Compare small, large and overlapping chunk strategies.
- 07
Design metadata filters for private student records.
- 08
Create answerable, unanswerable and conflicting RAG tests.
- 09
Separate retrieval relevance, faithfulness and citation correctness.
- 10
Define state, tools and stop conditions for a planning agent.
- 11
Design idempotency for a side-effecting agent tool.
- 12
Threat-model prompt injection through retrieved documents.
- 13
Compare reactive, plan-first and verification-pass agents.
- 14
Create a complete evaluation scorecard for a college assistant.