Tuesday, October 13, 2026

🎓 AI Engineering Classroom — Phase 1 Lesson 4 : How Computers Learn — Data → Model → Prediction

Welcome back. We continue exactly according to the syllabus.

So far:

✅ Lesson 1 — What is AI?
✅ Lesson 2 — AI Problem-Solving & AI Engineer Mindset
✅ Lesson 3 — AI Terminology & Core Concepts
▶️ Lesson 4 — How Computers Learn

Today we are going to understand the core engine behind Machine Learning:

How does a computer go from raw data to a trained model that can make predictions?

Don't worry about advanced mathematics yet. We will build the intuition first.

1. The Big Picture

At the highest level:

                 DATA
                  │
                  ▼
            PREPARE DATA
                  │
                  ▼
            TRAINING ALGORITHM
                  │
                  ▼
                MODEL
                  │
                  ▼
              NEW DATA
                  │
                  ▼
              PREDICTION

This looks simple.

But almost everything we eventually learn in Machine Learning fits somewhere inside this pipeline.

2. Let's Start With a Real Problem

Imagine you work for a company that wants to predict whether a customer will leave.

The company has historical data:

Customer	Age	Monthly Spend	Logins	Support Tickets	Churn
A	25	₹1,000	25	1	No
B	45	₹500	2	8	Yes
C	31	₹2,000	30	0	No
D	52	₹600	3	7	Yes
E	28	₹1,500	20	1	No

We want to predict:

Will a new customer churn?

This is where Machine Learning begins.

3. Step 1 — Collect Data

The first requirement is data.

Business
   │
   ▼
Historical Data

Data could come from:

Databases
APIs
CSV files
Data warehouses
Data lakes
Applications
Sensors
Logs
Documents
Images

For our example:

Customer Database
        │
        ▼
Historical Customer Records

This is where your Data Engineering background becomes very useful.

An AI system is only as good as the data pipeline supporting it.

4. Step 2 — Identify Features

Remember from Lesson 3:

A feature is an input variable used by the model.

Our data:

Age
Monthly Spend
Logins
Support Tickets

are features.

We can represent them mathematically as:

X = [
    Age,
    Monthly Spend,
    Logins,
    Support Tickets
]

For Customer A:

X = [25, 1000, 25, 1]

The model receives these values as input.

5. Step 3 — Identify the Label

The thing we want to predict is called the target or label.

Here:

Churn

So:

Features
   │
   ▼
   X
   │
   ▼
MODEL
   │
   ▼
Prediction

And during training we also have:

Actual Answer
   │
   ▼
   y

This gives us the fundamental ML notation:

X = input/features

y = target/label

You will see X and y everywhere in ML.

6. Step 4 — Split the Data

We generally don't want to train and evaluate a model on exactly the same data.

Why?

Because the model could simply memorize the training examples.

Instead:

                 COMPLETE DATASET
                        │
             ┌──────────┼──────────┐
             ▼          ▼          ▼
         Training   Validation    Test
           Data        Data        Data

A common conceptual split might be:

70% → Training
15% → Validation
15% → Test

The exact percentages are not universal.

The important concept is:

Train on one portion and evaluate on data the model didn't train on.

7. Why Can't We Just Train on Everything?

Imagine you give a student exactly the questions that will appear in the final exam.

The student memorizes:

Question 1 → Answer A
Question 2 → Answer C
Question 3 → Answer B

Then you give exactly those questions in the exam.

The student scores 100%.

Does that prove the student understands the subject?

No.

The same thing can happen with ML.

We want:

Generalization

The model should learn the underlying patterns rather than memorize the training examples.

8. What Is Training?

Now we reach the heart of ML.

We have:

Training Data
      │
      ▼
Learning Algorithm
      │
      ▼
Model Parameters

The model initially doesn't know the correct relationship.

Training repeatedly adjusts its parameters to reduce prediction error.

Conceptually:

Input
  │
  ▼
Model
  │
  ▼
Prediction
  │
  ▼
Compare with Actual Answer
  │
  ▼
Calculate Error
  │
  ▼
Adjust Parameters
  │
  └───────────────┐
                  │
                  ▼
              Try Again

This loop happens many times.

9. Prediction

Suppose the model sees:

Age = 50
Monthly Spend = ₹600
Logins = 2
Support Tickets = 8

The model might output:

Churn Probability = 0.91

Meaning:

91% estimated probability of churn

The business could then classify:

Probability > 0.5
        ↓
Likely Churn

The exact threshold depends on the business problem.

10. What Is the Model Actually Learning?

This is a critical question.

The model is learning a relationship between inputs and outputs.

Conceptually:

X → y

For our example:

Customer Characteristics
          │
          ▼
       Churn Risk

The model might discover that combinations such as:

Low Logins
+
High Support Tickets
+
Low Engagement

are associated with higher churn.

We don't necessarily tell it this rule explicitly.

The algorithm learns patterns from data.

11. A Very Simple Example

Let's temporarily forget customers.

Suppose:

Hours Studied → Exam Score

1 → 40
2 → 50
3 → 60
4 → 70
5 → 80

We can observe a relationship:

More study
   ↓
Higher score

We could approximate it with:

Score = 30 + 10 × Hours

So if:

Hours = 6

then:

Score = 30 + (10 × 6)
      = 90

This is an extremely simple model.

Real ML models can learn enormously more complicated relationships.

12. Model Parameters

In our simple equation:

Score = 30 + 10 × Hours

we have:

30 → bias/intercept
10 → weight

The model can learn these values from data.

Initially, imagine:

weight = 0
bias = 0

The predictions would be terrible.

Training adjusts them:

weight → 10
bias   → 30

until predictions become good.

This is the basic idea behind parameter learning.

13. Loss Function

Now we need a way to answer:

"How wrong is the model?"

That's the job of a loss function.

Suppose the actual exam score is:

and the model predicts:

Error:

80 - 70 = 10

A loss function converts prediction error into a numerical value.

Conceptually:

Prediction
     │
     ▼
Compare with Actual
     │
     ▼
Loss

The training process tries to minimize the loss.

14. Think of Loss Like a Score

Imagine:

Loss = 100

Very bad.

After training:

Loss = 50

Better.

After more training:

Loss = 10

Much better.

Ideally:

Loss ↓

But remember:

A low training loss does not automatically mean the model will perform well on new data.

That's where generalization and overfitting become important.

15. Optimization

Now we have:

Model
+
Loss Function

We need a method to improve the model.

That's where optimization algorithms come in.

One extremely important algorithm is:

Gradient Descent

You don't need the mathematics yet.

Think of it like walking down a mountain.

       ●
      / \
     /   \
    /     \
   /       \
  /    ↓    \
 /     ↓     \
/______↓______\
      Minimum

The goal is to move toward a point where the loss is lower.

Conceptually:

Parameters
    │
    ▼
Prediction
    │
    ▼
Loss
    │
    ▼
Gradient
    │
    ▼
Update Parameters
    │
    └──────────────► Repeat

This will become extremely important when we study Deep Learning.

16. Learning Rate

Gradient descent needs to decide:

How big a step should we take?

That's the learning rate.

Imagine walking down a mountain.

Too small

Step
.
.
.
.
.

You eventually reach the bottom, but it takes forever.

Too large

      ↓
   Jump
        ↓
             Jump

You might overshoot the minimum.

So:

Learning Rate
      ↓
Controls size of parameter updates

This is an example of a hyperparameter.

17. Parameters vs Hyperparameters

This distinction is very important.

Parameters

The model learns them.

Examples:

Weights
Biases

Hyperparameters

We choose them before/during training.

Examples:

Learning Rate
Number of Trees
Batch Size
Number of Layers

Conceptually:

                MODEL
                  │
        ┌─────────┴─────────┐
        ▼                   ▼
   Parameters          Hyperparameters
   learned by model    chosen by engineer

We will study this much more deeply later.

18. Epoch

In neural-network training, you'll hear:

Epoch

An epoch means the model has gone through the training dataset once.

For example:

Training Dataset
      │
      ▼
   Epoch 1
      │
      ▼
   Epoch 2
      │
      ▼
   Epoch 3
      │
      ▼
   ...

If we train for:

10 epochs

the model has processed the training dataset approximately ten times, subject to how the training procedure is defined.

19. Batch

Large datasets may be too large to process all at once.

So we divide them into smaller batches.

Suppose:

1,000,000 records

and:

Batch Size = 1,000

The model processes:

Batch 1 → 1,000 records
Batch 2 → 1,000 records
Batch 3 → 1,000 records
...

This is particularly important for deep learning.

20. Epoch vs Batch vs Iteration

These terms often confuse beginners.

Suppose:

Dataset = 10,000 records
Batch size = 1,000

Then approximately:

10 batches = 1 epoch

Each batch produces an update in many common training procedures.

So:

Batch
↓
One chunk of training data

Iteration / Step
↓
One parameter-update step

Epoch
↓
One pass through the training dataset

21. Overfitting

Now we reach one of the most important concepts in ML.

Suppose the model becomes extremely good at the training data.

Training Accuracy = 99%

Sounds great.

But on new data:

Test Accuracy = 65%

Something went wrong.

The model has overfit.

It learned the training data too specifically instead of learning general patterns.

Think of a student who memorizes the textbook answers instead of understanding the subject.

22. Underfitting

The opposite problem is underfitting.

Suppose:

Training Accuracy = 60%
Test Accuracy = 58%

The model isn't learning enough.

Conceptually:

Underfitting
→ Model too simple / insufficiently trained

Good Fit
→ Learns useful general patterns

Overfitting
→ Learns training data too specifically

23. The Goal

We want:

Training Performance
        +
Generalization
        ↓
Good Model

Not:

Training Performance only

The ultimate objective is good performance on unseen data.

24. Validation Data

Remember our dataset:

Training
Validation
Test

Validation data helps us make development decisions.

For example:

Model A → Validation Accuracy 82%
Model B → Validation Accuracy 88%
Model C → Validation Accuracy 85%

We might select Model B.

Then, after we finalize the approach, we use the test set for final evaluation.

This helps keep the test set more independent.

25. Generalization

Generalization means:

The model performs well on new, unseen data.

This is one of the most important concepts in ML.

Imagine:

Training Data
████████████████

Model learns patterns

New Data
████████████████

Model predicts successfully

That's generalization.

26. Classification

Now let's introduce common ML problem types.

Classification

Predict a category.

Examples:

Spam / Not Spam
Fraud / Not Fraud
Churn / No Churn
Cat / Dog
Approved / Rejected

Output:

Class

27. Regression

Regression predicts a continuous numerical value.

Examples:

House Price
Sales Forecast
Temperature
Demand
Revenue

Example:

Input
 ↓
Model
 ↓
₹82,50,000

That's regression.

28. Clustering

Clustering is an unsupervised learning technique.

You don't provide labels.

The algorithm tries to discover groups.

Customer Data
      │
      ▼
   Clustering
      │
      ├── Group A
      ├── Group B
      └── Group C

Example:

Customers
   │
   ▼
Segments

We'll study this in the Machine Learning phase.

29. The Complete ML Learning Loop

Now combine everything.

                    HISTORICAL DATA
                          │
                          ▼
                    DATA PREPARATION
                          │
                          ▼
                    TRAIN / VAL / TEST
                          │
                          ▼
                    SELECT ALGORITHM
                          │
                          ▼
                       MODEL
                          │
                          ▼
                      PREDICTION
                          │
                          ▼
                    CALCULATE LOSS
                          │
                          ▼
                    OPTIMIZE MODEL
                          │
                          ▼
                    UPDATE PARAMETERS
                          │
                          └───────────┐
                                      │
                                      ▼
                                  REPEAT
                                      │
                                      ▼
                              VALIDATE MODEL
                                      │
                                      ▼
                               FINAL TEST
                                      │
                                      ▼
                                  DEPLOY
                                      │
                                      ▼
                              NEW PREDICTIONS

This is the foundation of Machine Learning.

30. But What About Deep Learning?

The same basic concept remains.

Data
 ↓
Neural Network
 ↓
Prediction
 ↓
Loss
 ↓
Backpropagation
 ↓
Gradient Descent
 ↓
Update Weights
 ↓
Repeat

Deep Learning simply involves much more sophisticated models and training processes.

Later we'll unpack:

Neurons
↓
Layers
↓
Weights
↓
Activation Functions
↓
Forward Propagation
↓
Loss
↓
Backpropagation
↓
Gradient Descent

31. What About LLMs?

This is where our roadmap eventually becomes very interesting.

An LLM is also a trained model.

At a very high level:

Massive Dataset
       │
       ▼
Tokenization
       │
       ▼
Neural Network
       │
       ▼
Training
       │
       ▼
Foundation Model
       │
       ▼
Inference
       │
       ▼
Generated Tokens

The fundamental learning principles we are discussing today form the foundation beneath modern LLMs.

Later we'll study:

Neural Networks
       ↓
Deep Learning
       ↓
Attention
       ↓
Transformers
       ↓
Large Language Models
       ↓
Generative AI

So don't underestimate today's lesson.

32. A Critical Distinction: Learning vs Memorization

Suppose we give a model:

2 + 2 = 4
3 + 3 = 6
4 + 4 = 8

We don't want the model merely to memorize those exact examples.

We want it to learn a general relationship.

Then:

7 + 7

should produce:

This is the essence of generalization.

Of course, real neural networks and LLMs are considerably more complicated than this simple example.

33. Data Quality Matters

Here's a very important AI Engineering principle:

Garbage in, garbage out.

Suppose your training data contains:

Incorrect labels
Duplicates
Missing values
Bad measurements
Biased samples
Corrupted records

Your model can learn from those problems.

So:

Bad Data
   ↓
Bad Training
   ↓
Bad Model
   ↓
Bad Predictions

This is why Data Engineering is extremely important in AI Engineering.

34. AI Engineering Connection

Imagine your future production system:

                    DATA SOURCES
                         │
                         ▼
                  DATA PIPELINES
                         │
                         ▼
                   DATA QUALITY
                         │
                         ▼
                    ML / AI MODEL
                         │
                         ▼
                     INFERENCE
                         │
                         ▼
                  APPLICATION/API
                         │
                         ▼
                       USER

Your Data Engineering skills will eventually connect directly with:

Feature pipelines
Training data
Data quality
Feature stores
Vector databases
RAG ingestion
Model monitoring
AI observability

That's why we're not treating AI Engineering as just "learn prompts."

35. A Real Example

Let's build a simple mental model for predicting house prices.

Data

Size
Bedrooms
Location
Age

Target

Price

Training

Historical Houses
        │
        ▼
ML Algorithm
        │
        ▼
Learn Parameters

Model

House Features
      │
      ▼
   MODEL
      │
      ▼
Predicted Price

New house

Size = 2000 sqft
Bedrooms = 3
Location = Bangalore
Age = 5

Model:

₹1.25 crore

We then compare predictions against actual values and evaluate how well the model performs.

36. Now Think Like an AI Engineer

A beginner thinks:

"Which model should I use?"

An AI Engineer first asks:

What is the business problem?
        ↓
What is the target?
        ↓
What data do we have?
        ↓
Is the data reliable?
        ↓
What type of ML problem is this?
        ↓
How will we evaluate it?
        ↓
How will we deploy it?
        ↓
How will we monitor it?

This mindset is far more important than memorizing algorithms.

🧠 Lesson 4 — Core Mental Model

Memorize this:

              DATA
                │
                ▼
             FEATURES
                │
                ▼
          LEARNING ALGORITHM
                │
                ▼
              MODEL
                │
                ▼
          NEW INPUT DATA
                │
                ▼
            PREDICTION
                │
                ▼
          EVALUATE RESULT

During training:

Prediction
    │
    ▼
Compare with Actual
    │
    ▼
Loss
    │
    ▼
Parameter Update
    │
    └──────────────► Repeat

The fundamental objective:

Learn patterns from training data that generalize to unseen data.

📚 Lesson 4 Vocabulary

You should now know:

Concept	Meaning
Feature	Input variable
Label/Target	Value we want to predict
Training	Learning model parameters from data
Validation	Data used during development/tuning
Test	Unseen data for final evaluation
Model	Learned representation used for prediction
Loss	Numerical measure of prediction error
Optimization	Process of improving model parameters
Gradient Descent	Common optimization method
Learning Rate	Size of parameter updates
Epoch	One pass through training data
Batch	Subset of training data
Parameter	Value learned by model
Hyperparameter	Setting chosen by engineer
Classification	Predict a category
Regression	Predict a continuous value
Clustering	Discover groups in data
Overfitting	Model fits training data too specifically
Underfitting	Model fails to learn enough
Generalization	Performing well on unseen data

🎯 Phase 1 Progress

PHASE 1 — AI FUNDAMENTALS

✅ Lesson 1 — What is AI?
✅ Lesson 2 — AI Problem-Solving & AI Engineer Mindset
✅ Lesson 3 — AI Terminology & Core Concepts
✅ Lesson 4 — Data → Model → Prediction

⬜ Lesson 5 — Types of Machine Learning
⬜ Lesson 6 — AI Project Lifecycle
⬜ Lesson 7 — AI Engineering vs ML Engineering vs Data Engineering
⬜ Phase 1 Assessment

We are not moving to Python yet. Python comes after the AI fundamentals portion of the roadmap.

📝 Lesson 4 Classroom Test

Answer these in your own words. Don't worry about perfect answers.

Q1

What is the difference between a feature and a label?

Q2

Why do we split data into training, validation, and test sets?

Q3

What does a model actually learn during training?

Q4

What is a loss function?

Q5

What does gradient descent do?

Q6

What is the difference between a parameter and a hyperparameter?

Q7

Explain overfitting using a real-life example.

Q8

What is the difference between classification and regression?

Q9

Suppose we have:

Customer Age
Monthly Spend
Number of Logins
Support Tickets

and want to predict:

Customer Churn

Identify:

Features =
Target =

Q10 — Most Important

Explain this entire process in your own words:

Raw Data
   ↓
Features
   ↓
Training
   ↓
Model
   ↓
New Data
   ↓
Prediction
   ↓
Evaluation

Once you've answered these, we'll continue to Lesson 5 — Types of Machine Learning, where you'll learn Supervised Learning, Unsupervised Learning, Semi-Supervised Learning, Self-Supervised Learning, and Reinforcement Learning and understand where each is used in real AI systems.

🎓 AI Engineering Classroom — Phase 1 Lesson 3: AI Terminology & Core Concepts

Lesson 3: AI Terminology & Core Concepts

Excellent. We continue exactly in sequence.

So far:

Lesson 1: What is AI?
Lesson 2: AI Problem-Solving & AI Engineer Mindset
Lesson 3: AI Terminology & Core Concepts ← Today

Today is important because from this point onward, you will start seeing these terms everywhere in AI documentation, architecture diagrams, interviews, research papers, and projects.

Our goal is not to memorize definitions. You should understand how the concepts connect.

1. The AI Vocabulary Map

First, let's see the major terms together:

                         ARTIFICIAL INTELLIGENCE
                                  │
                    ┌─────────────┴─────────────┐
                    │                           │
              Traditional AI              Machine Learning
                                                │
                                         ┌──────┴──────┐
                                         │             │
                                   Classical ML   Deep Learning
                                                       │
                                                       ▼
                                                  Transformers
                                                       │
                                                       ▼
                                             Foundation Models
                                                       │
                                      ┌────────────────┼───────────────┐
                                      │                │               │
                                     LLMs          Vision Models    Multimodal
                                      │
                                      ▼
                                Generative AI
                                      │
                     ┌────────────────┼─────────────────┐
                     │                │                 │
                   RAG             Agents          Fine-Tuning
                     │                │
                     │                ▼
                     │          Agentic AI
                     │
                     ▼
               AI Applications

This diagram will become much clearer as we progress.

2. What Is a Model?

This is one of the most fundamental concepts.

A model is a mathematical/computational system that has learned patterns from data or has been designed to represent some process.

Think of a model as:

DATA
 │
 ▼
Learning / Training
 │
 ▼
MODEL
 │
 ▼
Prediction / Generation

For example, suppose we train a model to predict house prices.

Training data:

House Size → Price

1000 sqft → ₹50 lakh
1500 sqft → ₹70 lakh
2000 sqft → ₹95 lakh

The model learns a relationship.

Then:

1800 sqft
   │
   ▼
 Model
   │
   ▼
₹85 lakh

The model is the learned representation of the patterns.

3. Training vs Inference

You will hear these two words constantly.

Training

Training means teaching the model from data.

Training Data
     │
     ▼
 Learning Algorithm
     │
     ▼
    Model

For example:

Millions of documents
        │
        ▼
      Training
        │
        ▼
      LLM

Training can require enormous computational resources.

Inference

Inference means using an already-trained model to produce an output.

User Input
    │
    ▼
Trained Model
    │
    ▼
Output

Example:

User:
"Explain RAG."

      ↓

LLM

      ↓

Explanation

That's inference.

Remember

Training = Learn

Inference = Use what was learned

This distinction will become extremely important when we discuss:

Model serving
GPUs
Cost
Fine-tuning
LLM APIs
MLOps
Deployment

4. Parameters

A model contains parameters.

Parameters are values learned during training.

Very simplified:

Input
  │
  ▼
Parameters
  │
  ▼
Output

Modern neural networks may contain:

Millions
Billions
Trillions

of parameters.

When you hear:

"This is a 7-billion-parameter model."

It means the model contains roughly 7 billion learned parameter values.

Important:

More parameters does not automatically mean a model is better for every task.

Model quality also depends on:

Training data
Architecture
Training method
Alignment
Inference strategy
Context handling
Evaluation

5. What Is a Dataset?

A dataset is structured or unstructured data used for analysis or machine learning.

Examples:

Customer Data
Sales Data
Images
Audio
Documents
Sensor Data
Text
Code

For ML, datasets are often divided into:

Dataset
 │
 ├── Training Set
 │
 ├── Validation Set
 │
 └── Test Set

Training Set

Used to learn.

Validation Set

Used to tune decisions during development.

Test Set

Used to evaluate final performance on unseen data.

We will study this deeply in the Machine Learning phase.

6. Features and Labels

These are especially important in classical ML.

Suppose we want to predict whether a customer will churn.

Our dataset could look like:

Age	Monthly Spend	Logins	Support Tickets	Churn
25	1000	20	1	No
42	500	2	5	Yes
31	800	15	2	No

The features are:

Age
Monthly Spend
Logins
Support Tickets

The label/target is:

Churn

So:

Features
   │
   ▼
ML Model
   │
   ▼
Label / Prediction

7. What Is an Algorithm?

An algorithm is a defined procedure for solving a problem.

Examples:

Sorting algorithm
Search algorithm
Gradient descent
Decision Tree
K-Means

A useful distinction:

Algorithm
    +
Data
    ↓
Learning Process
    ↓
Model

Don't confuse:

Algorithm ≠ Model

For example:

Gradient descent is an optimization algorithm.
A trained neural network is a model.

We will revisit this later.

8. Neural Network

A neural network is a computational model inspired loosely by biological neural systems.

Simplified:

Input
 │
 ▼
○ ○ ○ ○
 │ │ │ │
 ▼ ▼ ▼ ▼
○ ○ ○ ○
 │ │ │ │
 ▼ ▼ ▼ ▼
Output

The network contains:

Neurons/nodes
Weights
Biases
Activation functions
Layers

During training, the model adjusts its parameters to reduce error.

9. Weights and Biases

Consider:

Input
  │
  ▼
Weight
  │
  ▼
Calculation
  │
  ▼
Output

A simplified neuron:

output = activation(weight × input + bias)

Don't worry if the mathematics isn't completely clear yet.

We will properly learn:

Vectors
Matrices
Functions
Derivatives
Gradients
Backpropagation

during Phase 3.

For now, remember:

Weights and biases are learnable parameters of neural networks.

10. What Is a Token?

Now we enter LLM territory.

LLMs generally don't process text exactly as humans do.

Text is converted into tokens.

For example:

"AI is powerful"

might conceptually become:

["AI", " is", " powerful"]

Actual tokenization depends on the tokenizer.

A token may be:

A whole word
Part of a word
Punctuation
A special token

Therefore:

Token ≠ necessarily word

This matters because API pricing, context limits, and model processing often depend on tokens.

11. Tokenization

Tokenization converts text into tokens.

Text
 │
 ▼
Tokenizer
 │
 ▼
Tokens
 │
 ▼
Token IDs

Conceptually:

"Hello AI"
      │
      ▼
["Hello", " AI"]
      │
      ▼
[1234, 5678]

The actual IDs depend on the tokenizer.

The model processes numerical representations rather than raw human-readable text.

12. Context Window

An LLM cannot necessarily process unlimited information in one request.

The amount of input/output context a model can handle is called its context window.

Conceptually:

┌──────────────────────────────┐
│        Context Window        │
│                              │
│ System Instructions          │
│ User Question                │
│ Conversation                 │
│ Retrieved Documents          │
│ Tool Results                 │
│ Other Context                │
│                              │
└──────────────────────────────┘

This becomes very important for RAG and agents.

Suppose you have:

10 million documents

You don't simply send all 10 million documents to the LLM.

Instead:

10M Documents
      │
      ▼
 Retrieval
      │
      ▼
Relevant Information
      │
      ▼
Context Window
      │
      ▼
LLM

That is one of the core ideas behind RAG.

13. Prompt

A prompt is the input/context provided to a model to guide its response.

Simple example:

"Explain machine learning in simple language."

But professional AI systems often use much richer prompts.

System Instructions
+
User Request
+
Context
+
Examples
+
Retrieved Data
+
Tool Results

Therefore, don't think of a prompt only as:

"Write me a poem."

In AI engineering, prompts can become part of the application's architecture.

14. System Prompt / System Instructions

A system instruction establishes high-level behaviour for the model.

Conceptually:

System:
You are a financial assistant.
Use only provided company data.
Do not invent information.

User:
What is the GST status?

The model receives both pieces of information.

In production systems, system instructions are important for:

Behaviour
Constraints
Output format
Safety
Tool usage
Role definition

15. Temperature

You will frequently encounter the term temperature.

Temperature influences the randomness of model generation.

Conceptually:

Low temperature
      ↓
More predictable / focused

High temperature
      ↓
More varied / creative

For example:

Low temperature

Useful for:

Structured extraction
Classification
Deterministic-style outputs
Some business workflows

Higher temperature

Potentially useful for:

Brainstorming
Creative writing
Idea generation

Important:

Temperature does not magically make a model more intelligent.

It changes the generation behaviour.

16. Hallucination

This is one of the most important GenAI concepts.

A hallucination occurs when a model generates information that is incorrect, unsupported, or fabricated while presenting it as if it were true.

Example:

User:

"What is our company's 2026 leave policy?"

The model doesn't actually have access to the company's policy.

It might nevertheless produce a convincing answer.

That's dangerous.

This is why enterprise AI systems use techniques such as:

RAG
+
Grounding
+
Tool Calls
+
Validation
+
Guardrails
+
Human Review

17. Grounding

Grounding means connecting the model's response to reliable information.

For example:

User Question
      │
      ▼
Company Knowledge Base
      │
      ▼
Relevant Policy
      │
      ▼
LLM
      │
      ▼
Grounded Answer

RAG is one important way to ground an LLM.

But grounding can also involve:

Databases
APIs
Search
Tools
Structured data

18. Embedding

An embedding converts information into a numerical vector representing semantic characteristics.

Conceptually:

Text
 │
 ▼
Embedding Model
 │
 ▼
Vector

For example:

"How do I reset my password?"

becomes something like:

[0.12, -0.45, 0.87, ...]

The exact numbers are not meaningful to humans individually.

What matters is the relationship between vectors.

For example:

"How do I reset my password?"
                │
                ▼
             Vector A

"I forgot my password."
                │
                ▼
             Vector B

A good embedding model should place semantically similar texts relatively close together.

This becomes the foundation of semantic search.

19. Vector Database

A vector database stores and retrieves vectors efficiently.

Conceptually:

Documents
   │
   ▼
Embeddings
   │
   ▼
Vector Database
   │
   ▼
Similarity Search

Example:

User:
"I forgot my password."

       │
       ▼

Embedding

       │
       ▼

Vector Search

       │
       ▼

Relevant document:
"Password Reset Procedure"

This is the foundation of many RAG systems.

20. RAG

Now connect the previous concepts.

Documents
    │
    ▼
Chunking
    │
    ▼
Embeddings
    │
    ▼
Vector Database

Then at query time:

User Question
      │
      ▼
Query Embedding
      │
      ▼
Vector Search
      │
      ▼
Relevant Chunks
      │
      ▼
LLM Context
      │
      ▼
Generated Answer

That's RAG.

Notice how many concepts from today's lesson are already connected:

Tokens
Embeddings
Vectors
Context
LLM
Grounding
Retrieval

21. Fine-Tuning

Another term you will encounter frequently.

Fine-tuning means taking a pre-trained model and training it further on a specific dataset/task.

Conceptually:

Pretrained Model
       │
       ▼
Domain-Specific Training
       │
       ▼
Fine-Tuned Model

For example:

General Language Model
        │
        ▼
Medical Training Data
        │
        ▼
Domain-Adapted Model

But remember:

Fine-tuning is not the same thing as RAG.

RAG

Provides information at inference time.

LLM + External Knowledge

Fine-tuning

Changes the model's learned parameters.

Base Model
    +
Training
    ↓
Modified Model

We'll study when to use each later.

22. Inference

Let's revisit this because it is extremely important for LLM engineering.

Suppose a model has already been trained.

You send:

"What is RAG?"

The model processes the input and generates output.

That process is inference.

Prompt
  │
  ▼
Tokenization
  │
  ▼
Model
  │
  ▼
Token Generation
  │
  ▼
Output

Modern LLMs typically generate output token by token.

We'll study this much more deeply when we reach Transformers.

23. API

An API allows software systems to communicate.

For example:

Your Application
      │
      │ API Request
      ▼
   LLM Service
      │
      │ API Response
      ▼
Your Application

This is why you can build an application that uses an external AI model without training the model yourself.

This distinction is important:

Using an LLM
       ≠
Training an LLM

As an AI Engineer, you'll often integrate models rather than train massive foundation models from scratch.

24. Function Calling / Tool Calling

An LLM by itself produces language.

Tool calling gives it access to external capabilities.

Example:

User:
"What is today's sales?"

       │
       ▼

      LLM
       │
       ▼
Decides it needs database
       │
       ▼
SQL Tool
       │
       ▼
Database
       │
       ▼
Result
       │
       ▼
LLM
       │
       ▼
Answer

This is one of the building blocks of AI Agents.

25. Memory

Agents may need to maintain information across interactions.

Conceptually:

Conversation 1
      │
      ▼
    Memory
      │
      ▼
Conversation 2
      │
      ▼
Uses previous information

Memory can take many forms:

Conversation history
User profile
Database
Vector store
Application state

Important:

Memory is an architectural concept, not simply "the LLM remembers everything."

26. Agent State

An agent performing a multi-step task needs to know:

What has happened?
What is currently happening?
What remains?
What tools were used?
What results were returned?

This information is the state of the workflow.

Example:

Task:
Investigate failed pipeline

State:
├── Pipeline identified
├── Job failed
├── Logs retrieved
├── Error found
├── Database checked
└── Root cause pending

Agent frameworks such as LangGraph make stateful workflows easier to build.

27. Workflow vs Agent

This distinction is very important.

Workflow

The steps are mostly predefined.

Step 1
  ↓
Step 2
  ↓
Step 3
  ↓
Step 4

Example:

Extract PDF
    ↓
Chunk
    ↓
Embed
    ↓
Store

Agent

The system has more flexibility in deciding what to do next.

Goal
 │
 ▼
Agent
 │
 ├── Tool A?
 ├── Tool B?
 ├── Search?
 ├── Ask human?
 └── Continue?

Professional insight

You should not automatically use an agent when a deterministic workflow is sufficient.

Agents introduce:

Complexity
Cost
Latency
More failure modes
Security considerations
Evaluation challenges

Use them where their flexibility creates value.

28. AI Agent vs Agentic Workflow

A useful distinction:

AI Agent

An individual intelligent component capable of deciding actions.

Agentic Workflow

A larger system in which one or more agents operate within a structured process.

Example:

             Agentic Workflow
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
   Research      Analysis      Review
     Agent         Agent        Agent
        │           │           │
        └───────────┼───────────┘
                    ▼
                 Output

29. Multimodal AI

So far we've mostly discussed text.

Modern AI can work with multiple modalities.

              AI Model
                 │
      ┌──────────┼──────────┐
      ▼          ▼          ▼
     Text       Image      Audio
      │          │          │
      └──────────┼──────────┘
                 ▼
             Understanding

Multimodal systems may work with:

Text
Images
Audio
Video
Documents

Example:

Upload an invoice image and ask:

"Extract the invoice number, vendor, date, and total amount."

The system combines vision + language capabilities.

30. Foundation Model

A foundation model is a large, broadly trained model that can be adapted or used for many downstream tasks.

Conceptually:

Massive Training Data
        │
        ▼
Foundation Model
        │
 ┌──────┼────────┬─────────┐
 ▼      ▼        ▼         ▼
Chat   RAG     Coding    Vision

LLMs are one important category of foundation models.

Foundation models can also exist for:

Vision
Audio
Multimodal tasks

31. Open-Source vs Proprietary Models

You'll encounter both.

Proprietary / Hosted

You typically access them through an API or managed service.

Advantages may include:

Easy integration
Strong capabilities
Managed infrastructure

Open Models

Models whose weights or implementation are made available under particular licenses.

Advantages can include:

Greater control
Self-hosting options
Customization
Potentially lower marginal costs at scale

But you must consider:

License
Infrastructure
Security
Performance
Maintenance

This will become important in AI Architecture.

32. AI Pipeline — Putting Everything Together

Let's create a conceptual modern AI application.

                       USER
                         │
                         ▼
                    APPLICATION
                         │
                         ▼
                      API
                         │
                         ▼
                      AGENT
                         │
             ┌───────────┼───────────┐
             ▼           ▼           ▼
            RAG        TOOLS       MEMORY
             │           │           │
             ▼           ▼           ▼
          Vector DB   APIs/SQL    Database
             │           │
             └─────┬─────┘
                   ▼
                  LLM
                   │
                   ▼
              Evaluation
                   │
                   ▼
                Response

This is the type of architecture we will eventually build ourselves.

But notice:

We haven't learned enough yet.

That's why we're following the syllabus step by step.

🧠 33. The Most Important Vocabulary From Lesson 3

You should now understand these terms:

Term	Meaning
Model	Learned/computational representation used to generate predictions or outputs
Training	Learning from data
Inference	Using a trained model
Parameter	Learned value inside a model
Dataset	Collection of data
Feature	Input variable used by ML
Label	Target/output in supervised learning
Algorithm	Procedure for solving a computational problem
Neural Network	Layered computational model
Token	Unit of text processed by an LLM
Tokenization	Converting text into tokens
Context Window	Information the model can process in a request
Prompt	Instructions/input provided to a model
Temperature	Controls generation randomness
Hallucination	Unsupported or fabricated model output
Grounding	Connecting outputs to reliable information
Embedding	Numerical representation of information
Vector DB	Database optimized for vector similarity search
RAG	Retrieval + generation
Fine-tuning	Further training of a pretrained model
API	Interface allowing systems to communicate
Tool Calling	Model requesting an external function/tool
Agent	AI system capable of using tools/actions
State	Information representing current workflow/task status
Memory	Persisted information used across interactions
Workflow	Predetermined sequence of steps
Agentic AI	Goal-oriented AI systems with planning/action capabilities
Multimodal AI	AI working across multiple data modalities
Foundation Model	Broadly trained model usable across many tasks

🔗 34. How Today's Concepts Connect

This is the mental map I want you to remember:

                         DATA
                           │
                           ▼
                       TRAINING
                           │
                           ▼
                         MODEL
                           │
                           ▼
                       INFERENCE
                           │
                           ▼
                        LLM
                           │
              ┌────────────┼─────────────┐
              ▼            ▼             ▼
           Prompt        RAG          Tools
              │            │             │
              │            ▼             ▼
              │       Embeddings      APIs/SQL
              │            │             │
              └────────────┼─────────────┘
                           ▼
                         AGENT
                           │
                           ▼
                      Agentic AI
                           │
                           ▼
                  Production System
                           │
             ┌─────────────┼─────────────┐
             ▼             ▼             ▼
          Security      Evaluation    Monitoring

🎯 Lesson 3 Checkpoint

Before moving to Lesson 4, you should be comfortable explaining these five differences:

1. Training vs Inference

Training = learning
Inference = using

2. RAG vs Fine-Tuning

RAG
→ gives the model external information

Fine-tuning
→ changes/adapts the model through additional training

3. Workflow vs Agent

Workflow
→ predefined steps

Agent
→ can decide actions dynamically

4. LLM vs Agent

LLM
→ generates/understands language

Agent
→ LLM + tools + state + actions + orchestration

5. Prompt vs Context

Prompt
→ instructions/input

Context
→ broader information available to the model

📝 LESSON 3 — CLASSROOM TEST

No internet. Answer from your understanding.

Q1

What is the difference between training and inference?

Q2

What is a parameter in a machine-learning model?

Q3

Why does an LLM use tokens instead of simply processing words?

Q4

What is a context window, and why does it matter for RAG?

Q5

What is the difference between RAG and fine-tuning?

Q6

What is the difference between an AI Agent and an LLM?

Q7

Why would you use a tool/API with an LLM?

Q8

What is hallucination, and why is it dangerous in enterprise AI?

Q9

What is the difference between a workflow and an agent?

Q10 — Architecture Question

Suppose you have:

1,000 company PDFs + an LLM + a user asking questions about those PDFs.

Explain the complete flow from:

User Question
      ↓
      ?
      ↓
      ?
      ↓
      ?
      ↓
LLM
      ↓
Answer

Fill in the missing components and explain what each component does.

🚦Progress

PHASE 1 — AI FUNDAMENTALS

✅ Lesson 1 — What is AI?
✅ Lesson 2 — AI Problem-Solving & AI Engineer Mindset
▶️ Lesson 3 — AI Terminology & Core Concepts
⬜ Lesson 4 — How Computers Learn: Data → Model → Prediction
⬜ Lesson 5 — Types of Machine Learning
⬜ Lesson 6 — AI Project Lifecycle
⬜ Lesson 7 — AI Engineering vs ML Engineering vs Data Engineering
⬜ Phase 1 Assessment

Do not worry about getting every answer correct. Your answers will tell me exactly where your understanding needs reinforcement.

After you answer the 10 questions, I'll review them as your mentor and then move to Lesson 4, without skipping ahead to Python, ML, RAG, or Agents.

Monday, August 24, 2026

🎓 AI Engineering Classroom — Phase 1 Lesson 2

🎓 AI Engineering Classroom — Phase 1

Lesson 2: AI Problem-Solving & The AI Engineer Mindset

Welcome back.

In Lesson 1, we learned the basic AI hierarchy:


AI
│
├── Machine Learning
│
├── Deep Learning
│
├── Generative AI
│
├── RAG
│
└── AI Agents / Agentic AI

Today, we learn one of the most important skills of an AI Engineer:

Given a real-world problem, how do you decide what technology or architecture to use?

This is what separates an AI Engineer from someone who simply knows how to use ChatGPT or call an LLM API.

1. The AI Engineer Mindset

Imagine a client comes to you and says:

"We want AI."

This is not a technical requirement.

You should never immediately say:

"Let's build an LLM."

Instead, you ask:


What problem are we solving?
        │
        ▼
Who has the problem?
        │
        ▼
What data is available?
        │
        ▼
What output is required?
        │
        ▼
How accurate must it be?
        │
        ▼
Does the system need real-time information?
        │
        ▼
Does it need to take actions?
        │
        ▼
What are the security requirements?
        │
        ▼
What is the cost?
        │
        ▼
What is the simplest technology that solves it?

That last question is extremely important.

The best AI Engineer does not always choose the most advanced AI technology.

The best AI Engineer chooses the simplest reliable solution that solves the business problem.

2. The AI Solution Ladder

When you receive a problem, think through this ladder:


                 BUSINESS PROBLEM
                        │
                        ▼
               Can rules solve it?
                    /       \
                  YES        NO
                  │           │
                  ▼           ▼
            Traditional    Can ML solve it?
            Programming       /      \
                            YES       NO
                             │         │
                             ▼         ▼
                            ML      Can GenAI solve it?
                                      /       \
                                    YES        NO
                                     │          │
                                     ▼          ▼
                                   GenAI    Does it need
                                            external data?
                                             /      \
                                           YES       NO
                                            │         │
                                            ▼         ▼
                                           RAG      LLM
                                            │
                                            ▼
                                     Does it need
                                       actions?
                                      /      \
                                    YES       NO
                                     │         │
                                     ▼         ▼
                                    Agent    RAG/LLM

This is a simplified decision framework.

Let's understand each level.

3. Level 1 — Traditional Programming

Suppose your business requirement is:

"If an invoice amount is greater than £10,000, send it for manager approval."

Do you need AI?

No.

You can write:


if invoice_amount > 10000:
    send_for_approval()

Architecture:


Input
  │
  ▼
Business Rules
  │
  ▼
Output

This is deterministic.

The same input produces the same result.

Use traditional programming when:

Rules are clearly defined.
Logic is deterministic.
No learning is required.
No natural-language understanding is required.

Examples:


Tax calculation rules
Invoice validation
Password validation
Data transformation
ETL pipelines
Database CRUD operations
API routing

4. Level 2 — Machine Learning

Now imagine:

"Predict which customers are likely to leave the company."

Can we write a simple rule?

Maybe:


If customer hasn't logged in for 60 days → Churn

But real customer behaviour may depend on:

Purchase history
Login frequency
Customer support interactions
Contract value
Product usage
Complaints
Payment history

The relationship may be too complex to manually define.

This is where ML can help.


Historical Customer Data
          │
          ▼
    ML Training
          │
          ▼
     ML Model
          │
          ▼
New Customer
          │
          ▼
Churn Probability

Example:


Customer A → 0.12 churn probability
Customer B → 0.84 churn probability
Customer C → 0.67 churn probability

This is a prediction problem.

Use ML when:

You have historical data.
Patterns are difficult to express as rules.
You need predictions.
You need classification or regression.

Examples:


Fraud Detection
Customer Churn
Demand Forecasting
Credit Risk
Recommendation
Predictive Maintenance

5. Level 3 — Deep Learning

Suppose you want to detect defects in factory products.

You have millions of images.

Traditional ML may struggle to manually define all the visual features.

Deep Learning can learn complex representations from images.


Factory Image
      │
      ▼
CNN / Vision Model
      │
      ▼
Defect Detected?
      │
      ▼
Yes / No

Or:


Medical Image
      │
      ▼
Deep Learning Model
      │
      ▼
Classification

Use Deep Learning when:

Data is complex.
You have large datasets.
You are working with images, audio, video, or complex language.
Neural networks are appropriate.

Typical applications:


Computer Vision
Speech Recognition
Natural Language Processing
Image Generation
Video Understanding

6. Level 4 — Generative AI

Now consider:

"Generate a professional email based on these bullet points."

This is not a traditional prediction problem.

You want the system to generate content.


Bullet Points
     │
     ▼
Generative AI
     │
     ▼
Professional Email

Or:

"Summarise this 50-page document."


Document
   │
   ▼
LLM
   │
   ▼
Summary

Or:

"Explain this SQL query in simple language."


SQL
 │
 ▼
LLM
 │
 ▼
Explanation

Use GenAI when:

The input is natural language.
The output is generated content.
You need summarisation.
You need translation.
You need text generation.
You need code generation.
You need conversational interfaces.

7. Level 5 — RAG

Now imagine a company's internal knowledge base.


100,000 Documents
       │
       ├── HR Policies
       ├── Finance Policies
       ├── Legal Documents
       ├── Technical Documentation
       └── Business Procedures

An employee asks:

"What is the company's maternity leave policy?"

A general LLM may not know your company's private policy.

We need to retrieve the relevant information.


User Question
      │
      ▼
Search Knowledge Base
      │
      ▼
Relevant Documents
      │
      ▼
LLM
      │
      ▼
Grounded Answer

This is RAG.

Use RAG when:

The information is private.
The information changes frequently.
The model doesn't know the information.
You need answers grounded in documents.
You need citations or source references.

Examples:


Company Knowledge Assistant
Legal Document Assistant
Financial Policy Assistant
Technical Documentation Assistant
Customer Support Knowledge Base

8. Level 6 — AI Agents

Now imagine the user asks:

"Check why yesterday's AWS Glue pipeline failed and tell me what caused it."

The system needs to:

Check AWS.
Find the failed job.
Read logs.
Identify the error.
Query the database.
Compare expected and actual results.
Explain the root cause.

A simple LLM cannot directly do all this.

We need an agent with tools.


                    User
                     │
                     ▼
                 AI Agent
                     │
       ┌─────────────┼─────────────┐
       ▼             ▼             ▼
   AWS Tool       SQL Tool      Python Tool
       │             │             │
       ▼             ▼             ▼
   AWS Logs      Database       Analysis
       │             │             │
       └─────────────┼─────────────┘
                     ▼
                  LLM
                     │
                     ▼
               Final Answer

Use Agents when:

The system needs to use tools.
The task has multiple steps.
The next action depends on previous results.
The system needs to interact with APIs.
The system needs to perform actions.

9. Level 7 — Agentic AI

Now let's make the problem more complex.

User says:

"Investigate yesterday's failed data pipeline, identify the root cause, fix the SQL transformation, test it, and create a report."

Now the system may need to:


Understand Goal
     │
     ▼
Create Plan
     │
     ▼
Inspect Pipeline
     │
     ▼
Analyze Logs
     │
     ▼
Query Database
     │
     ▼
Identify Problem
     │
     ▼
Generate Fix
     │
     ▼
Test Fix
     │
     ▼
Ask Human Approval
     │
     ▼
Apply Fix
     │
     ▼
Generate Report

This is approaching an agentic workflow.

The system is no longer simply answering a question.

It is trying to achieve a goal.

10. The Most Important Decision

Let's take a real-world example.

Your client, LocalCA, provides:

Tax filing
GST services
Compliance services
Legal services
Financial services

Suppose they say:

"We want an AI system for our customers."

Don't immediately build an agent.

Break down the problem.

Requirement A

"Calculate GST based on predefined rules."

Possible solution:


Traditional Programming

Why?

Because rules are known and deterministic.

Requirement B

"Predict which customers are likely to miss their tax filing deadlines."

Possible solution:


Machine Learning

Why?

Because historical customer behaviour can be used to predict risk.

Requirement C

"Answer questions about GST regulations."

Possible solution:

RAG

Why?

Because the system needs access to regulatory and company-specific knowledge.

Requirement D

"Read a customer's uploaded financial documents and summarise them."

Possible solution:


Generative AI
+
Document Processing

Requirement E

"Check a customer's profile, identify pending compliance tasks, look up relevant regulations, and prepare a recommended action plan."

Possible solution:


AI Agent
+
RAG
+
Tools

Requirement F

"Monitor customer deadlines, check status, prepare filings, request missing documents, and notify customers."

Possible solution:


Agentic AI
+
RAG
+
Tools
+
Workflow Engine
+
Human Approval

Now you are thinking like an AI Engineer.

11. The AI Engineer's Core Question

Whenever you receive a problem, ask:

Question 1

What exactly is the business problem?

Question 2

Is the problem deterministic?

If yes:


Traditional Programming

Question 3

Do we need prediction?

If yes:


Machine Learning

Question 4

Is the data complex?

If yes, consider:


Deep Learning

Question 5

Do we need content generation?

If yes:


Generative AI

Question 6

Does the model need external/private knowledge?

If yes:

RAG

Question 7

Does the system need to use tools or APIs?

If yes:


AI Agent

Question 8

Does it need to perform multi-step tasks toward a goal?

If yes:


Agentic AI

12. AI Is Not Always the Answer

This is a very important professional lesson.

Suppose you have:

"Get customer information by customer ID."

Don't use:


LLM
+
RAG
+
Agent

Just use:


SELECT *
FROM customers
WHERE customer_id = ?;

Simple.

Fast.

Reliable.

Cheap.

This is good engineering.

13. The AI Engineer's Golden Rule

Use AI where AI provides value.

Don't add AI just because AI is fashionable.

A production AI Engineer must think about:


Business Value
      +
Accuracy
      +
Cost
      +
Latency
      +
Security
      +
Scalability
      +
Maintainability

A technically impressive system that costs £100,000 per month and solves a problem that could be solved with a £10 database query is a bad engineering solution.

14. AI Solution Decision Framework

Memorise this:


                 START
                   │
                   ▼
             Define Problem
                   │
                   ▼
          Can rules solve it?
              /          \
            YES           NO
             │             │
             ▼             ▼
          RULES      Need prediction?
                          /      \
                        YES       NO
                         │         │
                         ▼         ▼
                        ML      Need generation?
                                    /      \
                                  YES       NO
                                   │         │
                                   ▼         ▼
                                  GenAI   Need external
                                          knowledge?
                                           /    \
                                         YES     NO
                                          │       │
                                          ▼       ▼
                                         RAG     LLM
                                          │
                                          ▼
                                   Need actions/tools?
                                      /        \
                                    YES         NO
                                     │           │
                                     ▼           ▼
                                   AGENT      RAG/LLM
                                     │
                                     ▼
                              Multi-step goal?
                                  /       \
                                YES        NO
                                 │          │
                                 ▼          ▼
                              AGENTIC     AGENT

Again, this is a thinking framework, not a strict technical law. Real systems often combine multiple approaches.

For example:


Enterprise AI System
        │
        ├── Traditional Code
        │
        ├── ML
        │
        ├── LLM
        │
        ├── RAG
        │
        ├── Agents
        │
        └── Human Approval

The real skill is knowing where each component belongs.

15. Your First Architecture Exercise

Let's design a hypothetical system for LocalCA.

Requirement:

"A customer logs into the LocalCA portal and wants to see all their information in one place."

What could the architecture look like?


                   Customer
                      │
                      ▼
                 LocalCA UI
                      │
                      ▼
                  Backend API
                      │
          ┌───────────┼───────────┐
          ▼           ▼           ▼
      Customer DB   GST Data    Tax Data
          │           │           │
          └───────────┼───────────┘
                      ▼
                Customer 360
                      │
                      ▼
                  Dashboard

Do we need AI?

Maybe not.

The core Customer 360 system is primarily:


Database
+
APIs
+
Data Integration
+
UI

Now add:

"The customer wants to ask questions about their tax documents."

Add:


Customer Documents
       │
       ▼
      RAG
       │
       ▼
      LLM
       │
       ▼
   AI Assistant

Now add:

"The customer wants the system to check pending compliance tasks."

Add:


AI Agent
   │
   ├── Customer Database
   ├── Compliance Database
   ├── RAG
   └── Regulatory Knowledge

Now the architecture becomes:


                         LocalCA AI Platform
                                │
                ┌───────────────┼───────────────┐
                ▼               ▼               ▼
          Customer 360       RAG System      AI Agent
                │               │               │
                ▼               ▼               ▼
            Customer DB      Documents        Tools
                                                │
                                      ┌─────────┼─────────┐
                                      ▼         ▼         ▼
                                    SQL       APIs       AWS

This is an example of AI Engineering thinking.

You don't ask:

"Where can I use an LLM?"

You ask:

"Where does AI create meaningful value inside the overall system?"

16. AI Engineer vs AI Researcher

This distinction is useful.

AI Researcher

Focuses on:


New Algorithms
New Architectures
Mathematical Theory
Model Training
Research Papers

AI Engineer

Focuses on:


Business Problem
      ↓
Data
      ↓
Model
      ↓
Application
      ↓
RAG
      ↓
Agents
      ↓
APIs
      ↓
Deployment
      ↓
Evaluation
      ↓
Monitoring

AI Architect

Focuses on:


Business
   +
System Design
   +
Scalability
   +
Security
   +
Cost
   +
Reliability
   +
AI Technology Selection

Your roadmap is primarily targeting:

AI Engineer → Senior AI Engineer → AI Architect

17. Production Mindset

A beginner asks:

"Can I make the AI answer?"

An AI Engineer asks:

"Can I make the AI answer correctly?"

A Senior AI Engineer asks:

"Can I make it answer correctly, reliably, securely, and at scale?"

An AI Architect asks:

"Is this the right architecture for the business, and can we operate it sustainably?"

This progression is important.


Can it work?
     ↓
Does it work correctly?
     ↓
Can it work reliably?
     ↓
Can it scale?
     ↓
Can it be secured?
     ↓
Can we monitor it?
     ↓
Can we afford it?
     ↓
Can we maintain it?

This mindset will be a major theme throughout our course.

🧠 Lesson 2 — Key Takeaways

Remember these seven levels:

Technology	Best suited for
Traditional Programming	Deterministic rules
Machine Learning	Predictions
Deep Learning	Complex patterns
Generative AI	Content generation
RAG	External/private knowledge
AI Agents	Tool use and multi-step actions
Agentic AI	Goal-oriented autonomous workflows

The real-world system may combine all of them.

📝 CLASSROOM EXERCISE — LESSON 2

Now I want you to think like an AI Engineer.

For each scenario, choose the primary technology you would consider first.

Scenario 1

A company wants to calculate employee salaries according to fixed rules.


Traditional Programming
ML
Deep Learning
GenAI
RAG
Agent

Scenario 2

A bank wants to predict which customers are likely to default on loans.


Traditional Programming
ML
Deep Learning
GenAI
RAG
Agent

Scenario 3

A company wants to summarize 200-page legal documents.


Traditional Programming
ML
Deep Learning
GenAI
RAG
Agent

Scenario 4

An employee asks:

"What is our company's remote-work policy?"

The answer is stored in internal company documents.


Traditional Programming
ML
Deep Learning
GenAI
RAG
Agent

Scenario 5

A Data Engineer asks:

"Check today's failed Glue job, inspect the logs, query the affected tables, and explain the root cause."


Traditional Programming
ML
Deep Learning
GenAI
RAG
Agent

Scenario 6

A company wants an autonomous system that monitors pipelines, identifies failures, investigates them, proposes fixes, runs tests, and requests human approval before deployment.

What would you choose?

Think about:


LLM
+
Tools
+
RAG
+
Agents
+
Agentic Workflow
+
Human-in-the-loop

🎯 Your Assignment

Answer all 6 scenarios in this format:


1. Traditional Programming — Because...
2. Machine Learning — Because...
3. ...

For Scenario 6, draw your own simple architecture using text.


DATA
 │