k-Nearest Neighbors (k-NN)

How k-NN Works

When a new data point is encountered, the algorithm:

• Calculates the distance (e.g., Euclidean) from the new point to all training points
• Selects the k nearest neighbors
• For classification: Predicts the majority class among the neighbors
• For regression: Averages the values of the neighbors

It is a lazy learning algorithm, meaning it does not learn an explicit model but stores all training data.

Python Code for k-NN Classification

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

# Load dataset
iris = load_iris()
X = iris.data
y = iris.target

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize and train k-NN classifier
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X_train, y_train)

# Predict and evaluate
y_pred = knn.predict(X_test)
print(f"Accuracy: {accuracy_score(y_test, y_pred):.2f}")