# Classification using Perceptrons

We would like a classifier which maps a data point to one of the given classes. Linear regression is not viable here as the classes are NOT REAL!

A (bad) solution is to assign numbers to classes, but this **imposes ordering to the classes**, which is not ideal. We use *perceptrons* for this case, and the classes are modeled as a one-hot encoded vector.

To start off, we shall tackle a **Binary Classification Problem** where the classes are represented by $+1$ and $-1$.

For input attributes $\phi$ and learnt weights $w$, the hyperplane which separates the two classes would be given by $w^\text{T}\phi = 0$. We look at the perceptron algorithm for learning the weights.

That is, $y_{pred}=1$ if $w^\text{T}\phi>0$ and $y_{pred}=-1$ if $w^\text{T}\phi<0$. Do note that $w^\text{T}\phi$ is the perpendicular signed distance from the hyperplane.

## Perceptron Algorithm

We iterate over all the available data. For a given data point $< x_i, y_i >$;

- If classified correctly, do nothing
- Marginally correct weights if incorrectly classified.

Note that $y \phi$ is *unsigned distance* as it is always positive. $\eta$ is the learning rate here.

One of the drawbacks of perceptron algorithm is that it finds *any* hyperplane which correctly classifies the given data, **not the best** hyperplane. SVM is used to find the best hyperplane.