An Introduction to Logistic Regression — Supervised Machine Learning

Logistic Regression

Logistic Regression is a type of supervised machine learning algorithm used for binary classification tasks. It predicts the outcome based on probability, for example, the probability of a coin toss resulting in heads or tails, which is 50% for each. The probability ranges from 0 to 1. Logistic regression uses these probabilities to classify outcomes.

How Logistic Regression Works

Logistic regression uses the “Sigmoid Function” function to predict probabilities. In a binary classification problem, there are two classes (outputs), such as 0 and 1, True and False, Male and Female, or Patient vs. Healthy. The probability for a class is either 0 or 1, with a threshold of 0.5. If the probability is below 0.5, the class is 0; if it is 0.5 or above, the class is 1.

Simply put, logistic regression uses the “Sigmoid” function, which is an S-shaped curve, to map the predicted values into discrete classes.

Use Cases for Logistic Regression:

1. Binary Classification
2. Multiclass Classification
3. One-vs-Rest Classification

Advantages of Logistic Regression:

1. Simple and easy to implement
2. Works well with linearly separable data
3. It does not require high computational power

Assumptions to Meet:

1. The independent variables (features) must be independent.
2. The dependent variable (target) must be categorical.
3. There should be no outliers.
4. There should be no high correlation between the independent variables.

Types of Logistic Regression

1. Binary Logistic Regression:

• Purpose: Used for binary classification tasks with only two possible outcomes.
• Examples: Predicting whether an email is spam or not (0 and 1), determining if a patient is healthy or has a disease (True or False).

2. Multinomial Logistic Regression:

• Purpose: Used for multiclass classification tasks with more than two possible outcomes.
• Examples: Classifying types of flowers into species like Setosa, Versicolour, and Virginica.

3. Unordered Logistic Regression:

• Purpose: Used for classification tasks where the classes are not ordered.
• Examples: Categorizing types of fruits into apples, oranges, and bananas, where the categories have no inherent order.

4. Ordinal Logistic Regression:

• Purpose: Used for classification tasks where the classes have a natural order.
• Examples: Classifying customer satisfaction levels into high, medium, and low.

Logistic Regression Implementaion

---

# Importing libraries
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Importing our dataset
df = pd.read_csv("dataset.csv")

# Separate X (independent variables aka features) and y (dependent variable aka target)
X = df.drop('target', axis=1)
y = df['target']

# Splitting the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initializing the model
lr = LogisticRegression()

# Training the model
lr.fit(X_train, y_train)

# Making predictions
y_pred = lr.predict(X_test)

# Calculating accuracy
accuracy = accuracy_score(y_test, y_pred)
print('Accuracy:', accuracy)

---

---

Ready to dive deeper into the world of Data Science? Join me as we explore essential topics like Exploratory Data Analysis (EDA), Data Visualization, and building powerful Machine Learning models. Subscribe to my blog for the latest tutorials, insights, and practical tips.

Follow: X | Facebook | Instagram | Linkedin

---