Defining the margin for support vector machines
Step 1. Define the distance between two points in the space;
$d = (x_2 - x_1)$
-------------------------------------------
Step 2. Compute the projection of $d$ onto $p$ vector:
$proj_{d}p = \|x_2 - x_1\|cos(\theta)=\|(x_2 - x_1)\|\frac{\langle x_2 - x_1, p\rangle}{\|(x_2 - x_1)\|\|p\|} = \frac{\langle (x_2 - x_1), p\rangle}{\|p\|}$
- where: $cos(\theta) = \frac{\langle (x_2 - x_1), p\rangle}{\|(x_2 - x_1)\|\|p\|}$
-------------------------------------------
Step 3. Represent $p$ in terms of $w$:
$p \perp w$ thus $p = \lambda w$, for some $\lambda\in R$ and,
$proj_{d}p = \frac{\lambda\langle (x_2 - x_1), w\rangle}{\lambda\|w\|}$.
-------------------------------------------
Step 4. Define the geometrical margin between two hyperplanes for any $x_{i}\in\mathcal{X}$,
$\big[\langle w\cdot x_{2}\rangle + b = 1\big] - \big[
\langle w\cdot x_{1}\rangle + b = -1\big] = \big[\langle w, (x_2,-x_1)\rangle = 2\big]$
$M = proj_{d}p= \frac{\langle (x_2 - x_1), w\rangle}{\|w\|} =\frac{2}{\|w\|}$
-------------------------------------------
Maximize margin $M$
$\mbox{min}_{w,\xi} \rightarrow\frac{2}{\|w\|}+ C\sum_{i=1}^{n}\xi_{i}$
$\mbox{subject to} \rightarrow y_{i}\big(\langle w\cdot x_{i}\rangle + b\big)\geq 1,\forall x_{i} \in\Omega, i = 1,...,n;$
Where:
Set $\Omega = \{x_1,x_2,...,x_n\}$ where $\Omega\subset\mathcal{X}$ then consider $\{x_1,x_2,...,x_k\}\in\omega_{1}$ and $\{x_{k+1},x_{k+2},...,x_{n}\}\in\omega_{2}$ for two class $\omega_{1},\omega_{2}\in\Omega$.
Estimate:
$\;$ $f(x_{i})< -1, \; \forall \;\; x_{i}\in\omega_2$; or in a compact way $y_{i}\big(\langle w\cdot x_{i}\rangle + b\big)\geq 1$ where $y = \{\pm1\}$;
