Overview of ML
What is ML? How does it work?
Example Applications
Nature of ML Problems
Traditional CS vs ML
ML Flow
ML Advantages & Dis-advantages
Supervised Learning Setup
What is Supervised Learning?
Algorithms vs. Model?
Classification vs. Regression
Rules vs. Learning
Formalizing the setup or Formulation
1. Regression
2. Classification
Feature Space
Label Space
Hypothesis Space
Hypothesis Space
How to choose Hypothesis Space?
How to evaluate Hypothesis Space? or How do we evaluate the performance?
Hypothesis Cont.
Loss Functions
0/1 Loss
Squared loss
Absolute loss
Root Mean Squared Error
The elusive h
How not to reduce the loss?
Generalization
Generalization loss
Training and Test Data, Validation sets
Nearest Neighbors Methods
KNN Algorithm
Basic Idea
Formal Definition
KNN Decision Boundary
- A supervised, non-parametric algorithm
- Used for classification and regression
- An Instance-based learning algorithm
- A lazy learning algorithm
- Characteristics of kNN
- Practical issues
Nearest Neighbors Methods Cont.
Similarity/Distance Metrics
Constraints/Properties on Distance Metrics
Euclidean Distance
Manhatten Distance
Minkowski distance
Chebyshev Distance
Norm of a vector and Its Properties
Cosine Distance
Practical issues in computing distance
Nearest Neighbors Methods Cont.
KNN Algorithm Formulation: Regression vs Classification
Complexity of KNN
Choosing the value of K – The theory
Tuning the hyperparameter K – the Method
KNN – The good, the bad and the ugly