Perceptron Algorithm
Step through the perceptron learning algorithm as it iteratively discovers a linear decision boundary that separates two classes. Based on my SC2300 coursework implementation.
Green ring = correct prediction, Red ring = misclassified (weight update)
Training Stats
Weights (θ)
Current Step
Configuration
The Algorithm
The perceptron is one of the simplest binary classifiers. It learns a linear decision boundary by iterating through the training data and updating weights whenever it makes a misclassification.
# Perceptron Update Rule
for each sample (x, y):
if y * dot(x, θ) <= 0: # misclassified
θ += x * y # update weights
Key Concepts
Convergence Guarantee
If the data is linearly separable, the perceptron is guaranteed to converge to a separating hyperplane in a finite number of updates. The number of updates is bounded by (R/γ)² where R is the data radius and γ is the margin.
Averaged Perceptron
The averaged variant stores all intermediate weight vectors after each update and returns their average. This typically generalizes better to unseen data by reducing variance in the learned boundary.
Bias Term
By appending a 1 to each feature vector, the bias is absorbed into the weight vector. This allows the decision boundary to be offset from the origin: w₁x₁ + w₂x₂ + b = 0.
Online Learning
The perceptron processes one sample at a time and updates immediately when it makes an error. This makes it an online learning algorithm, suitable for streaming data or when the full dataset doesn't fit in memory.
Implementation Details
| Parameter | Value |
|---|---|
| Classification | Binary (+1 / -1) |
| Features | 2D + bias |
| Update Rule | θ += xy (when y·(x·θ) ≤ 0) |
| Max Epochs | 100 |
| Course | SC2300 Introduction to ML |