Logistic Regression

How Logistic Regression Works

Logistic Regression uses the logistic (sigmoid) function to map predicted values to probabilities between 0 and 1.

Key points:

• Predicts the probability of the default class (e.g., class 1)
• Uses the sigmoid function: σ(z) = 1 / (1 + e^(-z)), where z = β₀ + β₁x₁ + ... + βₙxₙ
• Outputs probabilities which are then converted to class labels using a threshold (usually 0.5)
• Optimizes parameters using Maximum Likelihood Estimation

Python Code for Logistic Regression

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset and prepare binary classification problem
iris = load_iris()
X = iris.data
y = (iris.target == 2).astype(int)  # Classify if species is Iris-Virginica or not

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize and train Logistic Regression model
model = LogisticRegression()
model.fit(X_train, y_train)

# Predict on test set
y_pred = model.predict(X_test)

# Evaluate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")