What is ML? Classical vs Deep Learning
What is Machine Learning? (Classical vs Deep Learning)
Machine Learning (ML) is about teaching computers to generalize from examples instead of following only explicitly coded rules. Instead of writing if statements for every situation, you provide data and let an algorithm learn the mapping between inputs and outputs.
1. Where ML Fits (and Where It Doesn’t)
| Good Fit |
Why |
Not a Great Fit |
Why |
| Email spam detection |
Many labeled examples, fuzzy patterns |
Simple tax calculation |
Deterministic rules clearer |
| Image classification |
Raw pixels: high-dimensional patterns |
Small dataset (<100 examples) |
Model will overfit |
| Recommendation ranking |
User behavior evolves |
Rare, one-off business rule |
Easier to hand-code |
| Predictive maintenance |
Sensor readings → failure risk |
Cryptographic logic |
Needs precise guarantees |
Rule of Thumb: If you can write an exhaustive, stable set of deterministic rules easily—do that first. ML adds maintenance overhead.
2. Key Vocabulary
| Term |
Meaning |
| Instance / Sample |
One row / example in your dataset |
| Features |
Input variables (numeric, categorical, text, pixels) |
| Label / Target |
What you want to predict (price, class) |
| Model |
Learned function mapping features → prediction |
| Training |
Adjusting model parameters using data |
| Generalization |
Performance on unseen data |
| Overfitting |
Memorizing noise instead of pattern |
| Underfitting |
Model too simple; misses structure |
3. Classical ML vs Deep Learning
| Aspect |
Classical ML |
Deep Learning |
| Typical Data |
Tabular (rows/columns) |
Images, audio, natural language, complex sequences |
| Feature Engineering |
Often manual (domain-driven) |
Network learns hierarchical features |
| Training Data Needs |
Works with smaller datasets |
Usually needs large labeled datasets |
| Interpretability |
Often higher (trees, linear models) |
Lower (latent representations) |
| Compute Needs |
Modest |
GPUs / accelerators often required |
Classical approach example: gradient boosted trees predicting loan default probability using credit score, income, age.
Deep learning example: transformer model generating a summary of a news article.
4. Categories of ML Problems
| Category |
Goal |
Example |
Typical Metrics |
| Classification |
Assign label |
Spam vs not spam |
Accuracy, F1, ROC-AUC |
| Regression |
Predict number |
House price |
MAE, RMSE, R² |
| Clustering |
Group similar items |
Customer segments |
Silhouette score (proxy) |
| Ranking |
Order items |
Search results |
NDCG, MAP |
| Recommendation |
Suggest items |
Movies for user |
Hit Rate@K, NDCG@K |
| Anomaly Detection |
Flag unusual |
Fraudulent transaction |
Precision/Recall on anomalies |
| Generation |
Produce content |
Text completion |
Human eval / BLEU |
5. Simplified Workflow (Lifecycle Snapshot)
flowchart LR
A[Define Problem & Metric] --> B[Collect & Label Data]
B --> C[Split Train / Validation / Test]
C --> D[Train Model]
D --> E[Evaluate & Error Analysis]
E --> F[Deploy]
F --> G[Monitor & Iterate]
G --> B
6. A Tiny End-to-End Example (Tabular)
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, classification_report
data = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
data.data, data.target, test_size=0.2, random_state=42, stratify=data.target
)
model = RandomForestClassifier(n_estimators=200, random_state=42)
model.fit(X_train, y_train)
pred = model.predict(X_test)
print('Accuracy:', round(accuracy_score(y_test, pred), 3))
print(classification_report(y_test, pred, target_names=data.target_names))
7. Bias, Variance, and the Sweet Spot
| Situation |
Train Error |
Validation Error |
Likely Issue |
Remedy |
| Both high |
High |
High |
Underfitting |
More features, different model |
| Train low, val high |
Low |
High |
Overfitting |
Regularize, more data, simpler model |
| Both moderate |
Moderate |
Slightly higher |
Reasonable |
Fine-tune hyperparameters |
You want low validation error without a huge gap to train error.
8. Classical vs Deep: When to Choose
| If... |
Prefer Classical |
If... |
Prefer Deep Learning |
| You have < 10k rows |
✅ |
You have millions of images |
✅ |
| Need interpretability for regulators |
✅ |
Need to auto-extract features from raw signals |
✅ |
| Limited compute budget |
✅ |
You can leverage GPUs / pretrained models |
✅ |
Often hybrid workflows emerge: classical models on engineered aggregate features from embeddings produced by deep models.
9. Common Pitfalls (Beginner Edition)
| Pitfall |
Why It Hurts |
Quick Fix |
| Peeking at test set repeatedly |
Inflates reported performance |
Use validation set for iteration |
| Using accuracy with imbalance |
Hides minority failure |
Use precision/recall/PR-AUC |
| Skipping baseline model |
No performance context |
Implement trivial predictor first |
| Ignoring data leakage |
Unrealistic metrics |
Split BEFORE feature engineering |
| Overfitting hyperparameters |
Memorizes validation |
Use nested CV or final holdout |
10. Mini Glossary (Starter)
| Term |
Short Definition |
| Epoch |
One full pass over training data |
| Parameter |
Learned weight (e.g., network layer weight) |
| Hyperparameter |
Setting you choose (learning rate, depth) |
| Embedding |
Dense numeric representation of discrete input |
| Gradient |
Direction of parameter adjustment to reduce loss |
11. Quick Decision Guide
| Goal |
First Thing to Try |
| Numeric target |
Linear regression + baseline mean |
| Category label |
Logistic regression or random forest |
| Text classification |
Pretrained transformer (fine-tune or embeddings + classical) |
| Image classification |
Pretrained CNN fine-tune |
| Time series (short horizon) |
Naïve baseline + gradient boosting |
12. Checklist
13. Further Resources
- scikit-learn (https://scikit-learn.org)
- "The Deep Learning Book" (Goodfellow et al.)
- fast.ai Practical DL course
- Google ML Crash Course
- "Rules of Machine Learning" (Google) – pragmatic guidance