π AI Engineering Classroom — Phase 1
Lesson 2: AI Problem-Solving & The AI Engineer Mindset
Welcome back.
In Lesson 1, we learned the basic AI hierarchy:
AI│├── Machine Learning│├── Deep Learning│├── Generative AI│├── RAG│└── AI Agents / Agentic AI
Today, we learn one of the most important skills of an AI Engineer:
Given a real-world problem, how do you decide what technology or architecture to use?
This is what separates an AI Engineer from someone who simply knows how to use ChatGPT or call an LLM API.
1. The AI Engineer Mindset
Imagine a client comes to you and says:
"We want AI."
This is not a technical requirement.
You should never immediately say:
"Let's build an LLM."
Instead, you ask:
What problem are we solving?│▼Who has the problem?│▼What data is available?│▼What output is required?│▼How accurate must it be?│▼Does the system need real-time information?│▼Does it need to take actions?│▼What are the security requirements?│▼What is the cost?│▼What is the simplest technology that solves it?
That last question is extremely important.
The best AI Engineer does not always choose the most advanced AI technology.
The best AI Engineer chooses the simplest reliable solution that solves the business problem.
2. The AI Solution Ladder
When you receive a problem, think through this ladder:
BUSINESS PROBLEM│▼Can rules solve it?/ \YES NO│ │▼ ▼Traditional Can ML solve it?Programming / \YES NO│ │▼ ▼ML Can GenAI solve it?/ \YES NO│ │▼ ▼GenAI Does it needexternal data?/ \YES NO│ │▼ ▼RAG LLM│▼Does it needactions?/ \YES NO│ │▼ ▼Agent RAG/LLM
This is a simplified decision framework.
Let's understand each level.
3. Level 1 — Traditional Programming
Suppose your business requirement is:
"If an invoice amount is greater than £10,000, send it for manager approval."
Do you need AI?
No.
You can write:
if invoice_amount > 10000:send_for_approval()
Architecture:
Input│▼Business Rules│▼Output
This is deterministic.
The same input produces the same result.
Use traditional programming when:
- Rules are clearly defined.
- Logic is deterministic.
- No learning is required.
- No natural-language understanding is required.
Examples:
Tax calculation rulesInvoice validationPassword validationData transformationETL pipelinesDatabase CRUD operationsAPI routing
4. Level 2 — Machine Learning
Now imagine:
"Predict which customers are likely to leave the company."
Can we write a simple rule?
Maybe:
If customer hasn't logged in for 60 days → Churn
But real customer behaviour may depend on:
- Purchase history
- Login frequency
- Customer support interactions
- Contract value
- Product usage
- Complaints
- Payment history
The relationship may be too complex to manually define.
This is where ML can help.
Historical Customer Data│▼ML Training│▼ML Model│▼New Customer│▼Churn Probability
Example:
Customer A → 0.12 churn probabilityCustomer B → 0.84 churn probabilityCustomer C → 0.67 churn probability
This is a prediction problem.
Use ML when:
- You have historical data.
- Patterns are difficult to express as rules.
- You need predictions.
- You need classification or regression.
Examples:
Fraud DetectionCustomer ChurnDemand ForecastingCredit RiskRecommendationPredictive Maintenance
5. Level 3 — Deep Learning
Suppose you want to detect defects in factory products.
You have millions of images.
Traditional ML may struggle to manually define all the visual features.
Deep Learning can learn complex representations from images.
Factory Image│▼CNN / Vision Model│▼Defect Detected?│▼Yes / No
Or:
Medical Image│▼Deep Learning Model│▼Classification
Use Deep Learning when:
- Data is complex.
- You have large datasets.
- You are working with images, audio, video, or complex language.
- Neural networks are appropriate.
Typical applications:
Computer VisionSpeech RecognitionNatural Language ProcessingImage GenerationVideo Understanding
6. Level 4 — Generative AI
Now consider:
"Generate a professional email based on these bullet points."
This is not a traditional prediction problem.
You want the system to generate content.
Bullet Points│▼Generative AI│▼Professional Email
Or:
"Summarise this 50-page document."
Document│▼LLM│▼Summary
Or:
"Explain this SQL query in simple language."
SQL│▼LLM│▼Explanation
Use GenAI when:
- The input is natural language.
- The output is generated content.
- You need summarisation.
- You need translation.
- You need text generation.
- You need code generation.
- You need conversational interfaces.
7. Level 5 — RAG
Now imagine a company's internal knowledge base.
100,000 Documents│├── HR Policies├── Finance Policies├── Legal Documents├── Technical Documentation└── Business Procedures
An employee asks:
"What is the company's maternity leave policy?"
A general LLM may not know your company's private policy.
We need to retrieve the relevant information.
User Question│▼Search Knowledge Base│▼Relevant Documents│▼LLM│▼Grounded Answer
This is RAG.
Use RAG when:
- The information is private.
- The information changes frequently.
- The model doesn't know the information.
- You need answers grounded in documents.
- You need citations or source references.
Examples:
Company Knowledge AssistantLegal Document AssistantFinancial Policy AssistantTechnical Documentation AssistantCustomer Support Knowledge Base
8. Level 6 — AI Agents
Now imagine the user asks:
"Check why yesterday's AWS Glue pipeline failed and tell me what caused it."
The system needs to:
- Check AWS.
- Find the failed job.
- Read logs.
- Identify the error.
- Query the database.
- Compare expected and actual results.
- Explain the root cause.
A simple LLM cannot directly do all this.
We need an agent with tools.
User│▼AI Agent│┌─────────────┼─────────────┐▼ ▼ ▼AWS Tool SQL Tool Python Tool│ │ │▼ ▼ ▼AWS Logs Database Analysis│ │ │└─────────────┼─────────────┘▼LLM│▼Final Answer
Use Agents when:
- The system needs to use tools.
- The task has multiple steps.
- The next action depends on previous results.
- The system needs to interact with APIs.
- The system needs to perform actions.
9. Level 7 — Agentic AI
Now let's make the problem more complex.
User says:
"Investigate yesterday's failed data pipeline, identify the root cause, fix the SQL transformation, test it, and create a report."
Now the system may need to:
Understand Goal│▼Create Plan│▼Inspect Pipeline│▼Analyze Logs│▼Query Database│▼Identify Problem│▼Generate Fix│▼Test Fix│▼Ask Human Approval│▼Apply Fix│▼Generate Report
This is approaching an agentic workflow.
The system is no longer simply answering a question.
It is trying to achieve a goal.
10. The Most Important Decision
Let's take a real-world example.
Your client, LocalCA, provides:
- Tax filing
- GST services
- Compliance services
- Legal services
- Financial services
Suppose they say:
"We want an AI system for our customers."
Don't immediately build an agent.
Break down the problem.
Requirement A
"Calculate GST based on predefined rules."
Possible solution:
Traditional Programming
Why?
Because rules are known and deterministic.
Requirement B
"Predict which customers are likely to miss their tax filing deadlines."
Possible solution:
Machine Learning
Why?
Because historical customer behaviour can be used to predict risk.
Requirement C
"Answer questions about GST regulations."
Possible solution:
RAG
Why?
Because the system needs access to regulatory and company-specific knowledge.
Requirement D
"Read a customer's uploaded financial documents and summarise them."
Possible solution:
Generative AI+Document Processing
Requirement E
"Check a customer's profile, identify pending compliance tasks, look up relevant regulations, and prepare a recommended action plan."
Possible solution:
AI Agent+RAG+Tools
Requirement F
"Monitor customer deadlines, check status, prepare filings, request missing documents, and notify customers."
Possible solution:
Agentic AI+RAG+Tools+Workflow Engine+Human Approval
Now you are thinking like an AI Engineer.
11. The AI Engineer's Core Question
Whenever you receive a problem, ask:
Question 1
What exactly is the business problem?
Question 2
Is the problem deterministic?
If yes:
Traditional Programming
Question 3
Do we need prediction?
If yes:
Machine Learning
Question 4
Is the data complex?
If yes, consider:
Deep Learning
Question 5
Do we need content generation?
If yes:
Generative AI
Question 6
Does the model need external/private knowledge?
If yes:
RAG
Question 7
Does the system need to use tools or APIs?
If yes:
AI Agent
Question 8
Does it need to perform multi-step tasks toward a goal?
If yes:
Agentic AI
12. AI Is Not Always the Answer
This is a very important professional lesson.
Suppose you have:
"Get customer information by customer ID."
Don't use:
LLM+RAG+Agent
Just use:
SELECT *FROM customersWHERE customer_id = ?;
Simple.
Fast.
Reliable.
Cheap.
This is good engineering.
13. The AI Engineer's Golden Rule
Use AI where AI provides value.
Don't add AI just because AI is fashionable.
A production AI Engineer must think about:
Business Value+Accuracy+Cost+Latency+Security+Scalability+Maintainability
A technically impressive system that costs £100,000 per month and solves a problem that could be solved with a £10 database query is a bad engineering solution.
14. AI Solution Decision Framework
Memorise this:
START│▼Define Problem│▼Can rules solve it?/ \YES NO│ │▼ ▼RULES Need prediction?/ \YES NO│ │▼ ▼ML Need generation?/ \YES NO│ │▼ ▼GenAI Need externalknowledge?/ \YES NO│ │▼ ▼RAG LLM│▼Need actions/tools?/ \YES NO│ │▼ ▼AGENT RAG/LLM│▼Multi-step goal?/ \YES NO│ │▼ ▼AGENTIC AGENT
Again, this is a thinking framework, not a strict technical law. Real systems often combine multiple approaches.
For example:
Enterprise AI System│├── Traditional Code│├── ML│├── LLM│├── RAG│├── Agents│└── Human Approval
The real skill is knowing where each component belongs.
15. Your First Architecture Exercise
Let's design a hypothetical system for LocalCA.
Requirement:
"A customer logs into the LocalCA portal and wants to see all their information in one place."
What could the architecture look like?
Customer│▼LocalCA UI│▼Backend API│┌───────────┼───────────┐▼ ▼ ▼Customer DB GST Data Tax Data│ │ │└───────────┼───────────┘▼Customer 360│▼Dashboard
Do we need AI?
Maybe not.
The core Customer 360 system is primarily:
Database+APIs+Data Integration+UI
Now add:
"The customer wants to ask questions about their tax documents."
Add:
Customer Documents│▼RAG│▼LLM│▼AI Assistant
Now add:
"The customer wants the system to check pending compliance tasks."
Add:
AI Agent│├── Customer Database├── Compliance Database├── RAG└── Regulatory Knowledge
Now the architecture becomes:
LocalCA AI Platform│┌───────────────┼───────────────┐▼ ▼ ▼Customer 360 RAG System AI Agent│ │ │▼ ▼ ▼Customer DB Documents Tools│┌─────────┼─────────┐▼ ▼ ▼SQL APIs AWS
This is an example of AI Engineering thinking.
You don't ask:
"Where can I use an LLM?"
You ask:
"Where does AI create meaningful value inside the overall system?"
16. AI Engineer vs AI Researcher
This distinction is useful.
AI Researcher
Focuses on:
New AlgorithmsNew ArchitecturesMathematical TheoryModel TrainingResearch Papers
AI Engineer
Focuses on:
Business Problem↓Data↓Model↓Application↓RAG↓Agents↓APIs↓Deployment↓Evaluation↓Monitoring
AI Architect
Focuses on:
Business+System Design+Scalability+Security+Cost+Reliability+AI Technology Selection
Your roadmap is primarily targeting:
AI Engineer → Senior AI Engineer → AI Architect
17. Production Mindset
A beginner asks:
"Can I make the AI answer?"
An AI Engineer asks:
"Can I make the AI answer correctly?"
A Senior AI Engineer asks:
"Can I make it answer correctly, reliably, securely, and at scale?"
An AI Architect asks:
"Is this the right architecture for the business, and can we operate it sustainably?"
This progression is important.
Can it work?↓Does it work correctly?↓Can it work reliably?↓Can it scale?↓Can it be secured?↓Can we monitor it?↓Can we afford it?↓Can we maintain it?
This mindset will be a major theme throughout our course.
π§ Lesson 2 — Key Takeaways
Remember these seven levels:
| Technology | Best suited for |
|---|---|
| Traditional Programming | Deterministic rules |
| Machine Learning | Predictions |
| Deep Learning | Complex patterns |
| Generative AI | Content generation |
| RAG | External/private knowledge |
| AI Agents | Tool use and multi-step actions |
| Agentic AI | Goal-oriented autonomous workflows |
The real-world system may combine all of them.
π CLASSROOM EXERCISE — LESSON 2
Now I want you to think like an AI Engineer.
For each scenario, choose the primary technology you would consider first.
Scenario 1
A company wants to calculate employee salaries according to fixed rules.
Traditional ProgrammingMLDeep LearningGenAIRAGAgent
Scenario 2
A bank wants to predict which customers are likely to default on loans.
Traditional ProgrammingMLDeep LearningGenAIRAGAgent
Scenario 3
A company wants to summarize 200-page legal documents.
Traditional ProgrammingMLDeep LearningGenAIRAGAgent
Scenario 4
An employee asks:
"What is our company's remote-work policy?"
The answer is stored in internal company documents.
Traditional ProgrammingMLDeep LearningGenAIRAGAgent
Scenario 5
A Data Engineer asks:
"Check today's failed Glue job, inspect the logs, query the affected tables, and explain the root cause."
Traditional ProgrammingMLDeep LearningGenAIRAGAgent
Scenario 6
A company wants an autonomous system that monitors pipelines, identifies failures, investigates them, proposes fixes, runs tests, and requests human approval before deployment.
What would you choose?
Think about:
LLM+Tools+RAG+Agents+Agentic Workflow+Human-in-the-loop
π― Your Assignment
Answer all 6 scenarios in this format:
1. Traditional Programming — Because...2. Machine Learning — Because...3. ...
For Scenario 6, draw your own simple architecture using text.
DATA│
No comments:
Post a Comment