Types of Machine Learning

Supervised Learning: labeled training data — predict output from inputs (classification, regression)
Unsupervised Learning: no labels — find patterns/structure (clustering, dimensionality reduction)
Reinforcement Learning: agent learns by interacting with environment (rewards/penalties)

Key Algorithms

Algorithm	Type	Best For
Linear Regression	Supervised/Regression	Predicting continuous values
Logistic Regression	Supervised/Classification	Binary classification (spam/not spam)
Decision Tree	Both	Interpretable models, rule-based decisions
Random Forest	Both (Ensemble)	High accuracy, handles overfitting
XGBoost / LightGBM	Both (Ensemble)	Structured data, Kaggle competitions
K-Nearest Neighbors	Both	Simple, no training phase
SVM	Both	High-dimensional spaces, text classification
K-Means	Unsupervised	Customer segmentation
PCA	Unsupervised	Dimensionality reduction

Model Evaluation Metrics

Classification:

Accuracy: % correct predictions (misleading on imbalanced data)
Precision: TP / (TP + FP) — how many predicted positives are actually positive
Recall (Sensitivity): TP / (TP + FN) — how many actual positives were caught
F1 Score: harmonic mean of precision and recall
AUC-ROC: area under the ROC curve — measures discrimination ability

Regression:

MAE: Mean Absolute Error
RMSE: Root Mean Squared Error (penalizes large errors)
R²: proportion of variance explained by the model

Bias-Variance Tradeoff

High Bias (Underfitting): model too simple; poor performance on training and test data
High Variance (Overfitting): model too complex; great on training, poor on test
Goal: find the sweet spot with cross-validation

🔑 scikit-learn Workflow: from sklearn.model_selection import train_test_split → fit() → predict() → score(). Always split data before any preprocessing!