Corrected typos in sim

doc/src/SupportVMachines/svm.do.txt

@@ -90,14 +90,14 @@ of dimension $n\times p$, where $n$ represents the observations for each feature
 If the above condition is not met for a given vector $\bm{x}_i$ we have 
 !bt
 \[
-b+w_1x_{i1}+w_2x_{i}2+\dots +w_px_{ip} >0,
+b+w_1x_{i1}+w_2x_{i2}+\dots +w_px_{ip} >0,
 \]
 !et
 if our output $y_i=1$.
 In this case we say that $\bm{x}_i$ lies on one of the sides of the hyperplane and if 
 !bt
 \[
-b+w_1x_{i1}+w_2x_{i}2+\dots +w_px_{ip} < 0,
+b+w_1x_{i1}+w_2x_{i2}+\dots +w_px_{ip} < 0,
 \]
 !et
 for the class of observations $y_i=-1$, 
@@ -106,7 +106,7 @@ then $\bm{x}_i$ lies on the other side.
 Equivalently, for the two classes of observations we have 
 !bt
 \[
-y_i\left(b+w_1x_{i1}+w_2x_{i}2+\dots +w_px_{ip}\right) > 0. 
+y_i\left(b+w_1x_{i1}+w_2x_{i2}+\dots +w_px_{ip}\right) > 0. 
 \]
 !et
 
@@ -576,19 +576,26 @@ y_i(\bm{w}^T\bm{z}_i+b)= 1 \hspace{0.1cm}\forall i,
 \]
 !et
 from which we also find $b$.
-To compute $\bm{z}_i^T\bm{Z}_j$ we define the kerne $K(\bm{x}_i,\bm{x}_j)$ as
+To compute $\bm{z}_i^T\bm{z}_j$ we define the kerne $K(\bm{x}_i,\bm{x}_j)$ as
 !bt
 \[
-K(\bm{x}_i,\bm{x}_j)=\bm{z}_i^T\bm{Z}_j= \phi(\bm{x}_i)^T\phi(\bm{x}_j).
+K(\bm{x}_i,\bm{x}_j)=\bm{z}_i^T\bm{z}_j= \phi(\bm{x}_i)^T\phi(\bm{x}_j).
 \]
 !et
 For the above example, the kernel reads
 !bt
 \[
-K(\bm{x}_i,\bm{x}_j)=[x_i^2, y_i^2, \sqrt{2}x_iy_i]^T\begin{bmatrix} x_j^2 \\ y_j^2 \\ \sqrt{2}x_jy_j \end{bmatrix}=x_i^2x_j^2+2x_ixJy_iy_j+y_i^2y_j^2.
+K(\bm{x}_i,\bm{x}_j)=[x_i^2, y_i^2, \sqrt{2}x_iy_i]^T\begin{bmatrix} x_j^2 \\ y_j^2 \\ \sqrt{2}x_jy_j \end{bmatrix}=x_i^2x_j^2+2x_ix_jy_iy_j+y_i^2y_j^2.
 \]
 !et
-We note that this is nothing but the dot product of the two original vectors $(\bm{x}_i^T\bm{x}_j)^2$. Instead of thus computing the product in the Lagrangian of $\bm{x}_i^T\bm{z}_j$ we simply compute the dot product $(\bm{x}_i^T\bm{x}_j)^2$. This leads to the so-called kernel trick and the result leads to the same as if we went through the trouble of performing the transformation $(\phi(\bm{x}_i)^T\phi(\bm{x}_j)$ during the SVM calculations. 
+
+We note that this is nothing but the dot product of the two original
+vectors $(\bm{x}_i^T\bm{x}_j)^2$. Instead of thus computing the
+product in the Lagrangian of $\bm{z}_i^T\bm{z}_j$ we simply compute
+the dot product $(\bm{x}_i^T\bm{x}_j)^2$. This leads to the so-called
+kernel trick and the result leads to the same as if we went through
+the trouble of performing the transformation
+$(\phi(\bm{x}_i)^T\phi(\bm{x}_j)$ during the SVM calculations.
 
 
 !split
@@ -603,11 +610,11 @@ subject to the constraints $\lambda_i\geq 0$, $\sum_i\lambda_iy_i=0$ in terms of
 !bt
 \[
 \frac{1}{2} \bm{\lambda}^T\begin{bmatrix} y_1y_1K(\bm{x}_1,\bm{x}_1) & y_1y_2K(\bm{x}_1,\bm{x}_2) & \dots & \dots & y_1y_nK(\bm{x}_1,\bm{x}_n) \\
-y_2y_1\bm{x}_2^T\bm{x}_1 & y_2y_2\bm{x}_2^T\bm{x}_2 & \dots & \dots & y_1y_n\bm{x}_2^T\bm{x}_n \\
+y_2y_1K(\bm{x}_2,\bm{x}_1) & y_2y_2(\bm{x}_2,\bm{x}_2) & \dots & \dots & y_1y_nK(\bm{x}_2,\bm{x}_n) \\
 \dots & \dots & \dots & \dots & \dots \\
 \dots & \dots & \dots & \dots & \dots \\
 y_ny_1K(\bm{x}_n,\bm{x}_1) & y_ny_2K(\bm{x}_n\bm{x}_2) & \dots & \dots & y_ny_nK(\bm{x}_n,\bm{x}_n) \\
-\end{bmatrix}\bm{\lambda}-\mathbb{I}\bm{\lambda}, 
+\end{bmatrix}\bm{\lambda}-\mathbb{1}\bm{\lambda}, 
 \]
 !et
 subject to $\bm{y}^T\bm{\lambda}=0$. Here we defined the vectors $\bm{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n]$ and 
@@ -618,7 +625,7 @@ We can rewrite this (see the solutions below) in terms of a convex optimization
 !bt
 \begin{align*}
     &\mathrm{min}_{\lambda}\hspace{0.2cm} \frac{1}{2}\bm{\lambda}^T\bm{P}\bm{\lambda}+\bm{q}^T\bm{\lambda},\\ \nonumber
-    &\mathrm{subject to} \hspace{0.2cm} \bm{G}\bm{\lambda} \preceq \bm{h} \hspace{0.2cm} \bm{A}\bm{\lambda}=f.
+    &\mathrm{subject\hspace{0.1cm}to} \hspace{0.2cm} \bm{G}\bm{\lambda} \preceq \bm{h} \hspace{0.2cm} \wedge \bm{A}\bm{\lambda}=f.
 \end{align*}
 !et
 Below we discuss how to solve these equations. Here we note that the matrix $\bm{P}$ has matrix elements $p_{ij}=y_iy_jK(\bm{x}_i,\bm{x}_j)$.
@@ -636,7 +643,7 @@ o Gaussian Radial Basis Function: $K(\bm{x},\bm{y})=\exp{\left(-\gamma\vert\vert
 o Tanh: $K(\bm{x},\bm{y})=\tanh{(\bm{x}^T\bm{y}+\gamma)}$,
 and many other ones.
 
-An important theorem for us is "Mercer's theorem":https://en.wikipedia.org/wiki/Mercer%27s_theorem". 
+An important theorem for us is "Mercer's theorem":"https://en.wikipedia.org/wiki/Mercer%27s_theorem". 
 The theorem states that if a kernel function $K$ is symmetric, continuous and leads to a positive semi-definite matrix $\bm{P}$ then
 there exists a function $\phi$ that maps $\bm{x}_i$ and $\bm{x}_j$  into another space
 (possibly with much higher dimensions) such that
@@ -658,7 +665,7 @@ A mathematical (quadratic) optimization problem, or just optimization problem, h
 !bt
 \begin{align*}
     &\mathrm{min}_{\lambda}\hspace{0.2cm} \frac{1}{2}\bm{\lambda}^T\bm{P}\bm{\lambda}+\bm{q}^T\bm{\lambda},\\ \nonumber
-    &\mathrm{subject to} \hspace{0.2cm} \bm{G}\bm{\lambda} \preceq \bm{h} \hspace{0.2cm} \bm{A}\bm{\lambda}=f.
+    &\mathrm{subject\hspace{0.1cm}to} \hspace{0.2cm} \bm{G}\bm{\lambda} \preceq \bm{h} \wedge  \bm{A}\bm{\lambda}=f.
 \end{align*}
 !et
 subject to some constraints for say a selected set $i=1,2,\dots, n$.
@@ -666,7 +673,7 @@ In our case we are optimizing with respect to the Lagrangian multipliers $\lambd
 vector $\bm{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n]$ is the optimization variable we are dealing with.
 
 In our case we are particularly interested in a class of optimization problems called convex optmization problems. 
-In our disussion on gradient descent methods we discussed at length the definition of a convex function. 
+In our discussion on gradient descent methods we discussed at length the definition of a convex function. 
 
 Convex optimization problems play a central role in applied mathematics and we recommend strongly "Boyd and Vandenberghe's text on the topics":"http://web.stanford.edu/~boyd/cvxbook/".
 
@@ -700,7 +707,7 @@ We remind ourselves about the general problem we want to solve
 !bt
 \begin{align*}
     &\mathrm{min}_{x}\hspace{0.2cm} \frac{1}{2}\bm{x}^T\bm{P}\bm{x}+\bm{q}^T\bm{x},\\ \nonumber
-    &\mathrm{subject to} \hspace{0.2cm} \bm{G}\bm{x} \preceq \bm{h} \hspace{0.2cm} \bm{A}\bm{x}=f.
+    &\mathrm{subject\hspace{0.1cm} to} \hspace{0.2cm} \bm{G}\bm{x} \preceq \bm{h} \wedge  \bm{A}\bm{x}=f.
 \end{align*}
 !et
 
@@ -718,13 +725,13 @@ Let us show how to perform the optmization using a simple case. Assume we want t
 The minimization problem can be rewritten in terms of vectors and matrices as (with $x$ and $y$ being the unknowns)
 !bt
 \[
-frac{1}{2}\begin{bmatrix} x\\ y \end{bmatrix}^T   \begin{bmatrix} 1 & 0\\ 0 & 0  \end{bmatrix}  \begin{bmatrix} x \\ y \end{bmatrix}  + \begin{bmatrix}3\\ 4  \end{bmatrix}^T \begin{bmatrix}x \\ y  \end{bmatrix}.  
+\frac{1}{2}\begin{bmatrix} x\\ y \end{bmatrix}^T   \begin{bmatrix} 1 & 0\\ 0 & 0  \end{bmatrix}  \begin{bmatrix} x \\ y \end{bmatrix}  + \begin{bmatrix}3\\ 4  \end{bmatrix}^T \begin{bmatrix}x \\ y  \end{bmatrix}.  
 \]
 !et
 Similarly, we can now set up the inequalities (we need to change $\geq$ to $\leq$ by multiplying with $-1$ on bot sides) as the following matrix-vector equation
 !bt
 \[
-\begin{bmatrix} -1 & 0 \\ 0 & -1 \\ -1 & -3 \\ 2 & 5 \\ 3 & 4\end{matrix}\begin{bmatrix} x \\ y\end{bmatrix} \preceq \begin{bmatrix}0 \\ 0\\ -15 \\ 100 \\ 80\end{bmatrix}.
+\begin{bmatrix} -1 & 0 \\ 0 & -1 \\ -1 & -3 \\ 2 & 5 \\ 3 & 4\end{bmatrix}\begin{bmatrix} x \\ y\end{bmatrix} \preceq \begin{bmatrix}0 \\ 0\\ -15 \\ 100 \\ 80\end{bmatrix}.
 \]
 !et
 We have collapsed all the inequalities into a single matrix $\bm{G}$. We see also that our matrix