more update

doc/src/DecisionTrees/DecisionTrees.do.txt

@@ -1704,41 +1704,47 @@ at the internal nodes, and the predictions at the terminal nodes.
 
 To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.
 
-For simplicity we assume also that our functions $b(x;\gamma)=\gamma_0+\gamma_1 x$, with $\gamma_0$ and $\gamma_1$ as the parameters to be determined.
+For simplicity we assume also that our functions $b(x;\gamma)=1+\gamma x$. 
 
 This means that for every iteration, we need to optmize
 
 !bt
 \[
-(\beta_m,\gamma_m)  \mathrm{argmin}_{\beta,\gambda}\hspace{0.2cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(\gamma_0+\gamma_1 x_i))^2.
-\[
+(\beta_m,\gamma_m)  \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2.
+\]
 !et
 
-We start our iteration by simply setting $\f_0(x)=0$. 
+We start our iteration by simply setting $f_0(x)=0$. 
 Taking the derivatives  with respect to $\beta$ and $\gamma$ we obtain
 !bt
 \[
-\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(\gamma_0+\gamma_1 x_i)(y_i-\beta(\gamma_0+\gamma_1 x_i))=0,
+\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0,
 \]
 !et
 and
 !bt
 \[
-\frac{\partial {\cal C}}{\partial \gamma_0} =-2\sum_{i}\beta(y_i-\beta(\gamma_0+\gamma_1 x_i))=0,
+\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0.
 \]
 !et
-and
+We can then rewrite these equations as (defining $w_i=1+\gamma x_i)$
 !bt
 \[
-\frac{\partial {\cal C}}{\partial \gamma_1} = =-2\sum_{i}\beta x_i(y_i-\beta(\gamma_0+\gamma_1 x_i))=0.
+\gamma \bm{w}^T(\bm{y}-\beta\gamma \bm{w})=0,
+\]
+!et
+which gives us $\beta = \bm{w}^T\bm{y}/(\bm{w}^T\bm{w})$. Similarly we have 
+!bt
+\[
+\beta\gamma \bm{x}^T(\bm{y}-\beta(1+\gamma \bm{x}))=0,
 \]
 !et
 
+which leads $\gamma =(\bm{x}^T\bm{y}-\beta\bm{x}^T\em{e})/(\beta\bm{x}^T\bm{x})$.  Inserting
+for $\beta$ gives us an equation for $\gamma$.
 
-!split
-=====   Finding the Optimal Parameters =====
-
-With these equations we can then in turn find the parameters $\beta_1$ and $\gamma_0^{1}$ and $\gamma_1^1$ as
+The solution to these two equations gives us in turn $\beta_1$ and $\gamma_1$ leading to the new expression for $f_1(x)$ as
+$f_1(x) = \beta_1(1+\gamma_1x)$. Doing this $M$ times results in our final estimate for the function $f$.