added steepest descent

doc/src/DecisionTrees/DecisionTrees.do.txt

@@ -1960,7 +1960,29 @@ This means that for every iteration, we need to optimize
 \]
 !et
 
+We define a real function $h_m(x)$ that defines our final function $f_M(x)$ as
+!bt
+\[
+f_M(x) = \sum_{m=0}^M h_m(x).
+\]
+!et
 
+In the steepest decent approach we approximate $h_m(x) = -\rho_m g_m(x)$, where $\rho_m$ is a scalar and $g_m(x)$ the gradient defined as
+!bt
+\[
+g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}.
+\]
+!et
+
+With the new gradient we can update $f_m(x) = f_{m-1}(x) -\rho_m g_m(x)$. Using the above squared-error function we see that
+the gradient is $g_m(x_i) = -2(y_i-f(x_i))$.
+
+Choosing $f_0(x)=0$ we obtain $g_m(x) = -2y_i$ and inserting this into the minimization problem for the cost function we have
+!bt
+\[
+(\rho_1)  \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2.
+\]
+!et
 
 
 !split