-
diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html index 284e0001b..42a5f4c77 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html @@ -284,6 +284,28 @@ $$ (\hat{\boldsymbol{f}}) \mathrm{argmin}_{\boldsymbol{f}}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$ +
+We define a real function \( h_m(x) \) that defines our final function \( f_M(x) \) as +$$ +f_M(x) = \sum_{m=0}^M h_m(x). +$$ + +
+In the steepest decent approach we approximate \( h_m(x) = -\rho_m g_m(x) \), where \( \rho_m \) is a scalar and \( g_m(x) \) the gradient defined as +$$ +g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}. +$$ + +
+With the new gradient we can update \( f_m(x) = f_{m-1}(x) -\rho_m g_m(x) \). Using the above squared-error function we see that +the gradient is \( g_m(x_i) = -2(y_i-f(x_i)) \). + +
+Choosing \( f_0(x)=0 \) we obtain \( g_m(x) = -2y_i \) and inserting this into the minimization problem for the cost function we have +$$ +(\rho_1) \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2. +$$ +
diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html index 1de6f6e4b..63f72251e 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html @@ -293,7 +293,7 @@ MathJax.Hub.Config({
-
diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html index af57a8bd3..fd93c2216 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html @@ -148,7 +148,7 @@ MathJax.Hub.Config({
-
@@ -2381,6 +2381,34 @@ $$ (\hat{\boldsymbol{f}}) \mathrm{argmin}_{\boldsymbol{f}}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$
+
+
+We define a real function \( h_m(x) \) that defines our final function \( f_M(x) \) as +
+$$
+f_M(x) = \sum_{m=0}^M h_m(x).
+$$
+
+
+
+In the steepest decent approach we approximate \( h_m(x) = -\rho_m g_m(x) \), where \( \rho_m \) is a scalar and \( g_m(x) \) the gradient defined as +
+$$
+g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}.
+$$
+
+
+
+With the new gradient we can update \( f_m(x) = f_{m-1}(x) -\rho_m g_m(x) \). Using the above squared-error function we see that +the gradient is \( g_m(x_i) = -2(y_i-f(x_i)) \). + +
+Choosing \( f_0(x)=0 \) we obtain \( g_m(x) = -2y_i \) and inserting this into the minimization problem for the cost function we have +
+$$
+(\rho_1) \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2.
+$$
+
diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
index c9df08769..3045d1183 100644
--- a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
+++ b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
@@ -221,7 +221,7 @@ MathJax.Hub.Config({
-
@@ -2348,6 +2348,28 @@ $$
(\hat{\boldsymbol{f}}) \mathrm{argmin}_{\boldsymbol{f}}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f(x_i))^2.
$$
+
+We define a real function \( h_m(x) \) that defines our final function \( f_M(x) \) as +$$ +f_M(x) = \sum_{m=0}^M h_m(x). +$$ + +
+In the steepest decent approach we approximate \( h_m(x) = -\rho_m g_m(x) \), where \( \rho_m \) is a scalar and \( g_m(x) \) the gradient defined as +$$ +g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}. +$$ + +
+With the new gradient we can update \( f_m(x) = f_{m-1}(x) -\rho_m g_m(x) \). Using the above squared-error function we see that +the gradient is \( g_m(x_i) = -2(y_i-f(x_i)) \). + +
+Choosing \( f_0(x)=0 \) we obtain \( g_m(x) = -2y_i \) and inserting this into the minimization problem for the cost function we have +$$ +(\rho_1) \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2. +$$ +
diff --git a/doc/pub/DecisionTrees/html/DecisionTrees.html b/doc/pub/DecisionTrees/html/DecisionTrees.html
index 3ac58f8f9..ab637865b 100644
--- a/doc/pub/DecisionTrees/html/DecisionTrees.html
+++ b/doc/pub/DecisionTrees/html/DecisionTrees.html
@@ -226,7 +226,7 @@ MathJax.Hub.Config({
-
@@ -2353,6 +2353,28 @@ $$
(\hat{\boldsymbol{f}}) \mathrm{argmin}_{\boldsymbol{f}}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f(x_i))^2.
$$
+
+We define a real function \( h_m(x) \) that defines our final function \( f_M(x) \) as +$$ +f_M(x) = \sum_{m=0}^M h_m(x). +$$ + +
+In the steepest decent approach we approximate \( h_m(x) = -\rho_m g_m(x) \), where \( \rho_m \) is a scalar and \( g_m(x) \) the gradient defined as +$$ +g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}. +$$ + +
+With the new gradient we can update \( f_m(x) = f_{m-1}(x) -\rho_m g_m(x) \). Using the above squared-error function we see that +the gradient is \( g_m(x_i) = -2(y_i-f(x_i)) \). + +
+Choosing \( f_0(x)=0 \) we obtain \( g_m(x) = -2y_i \) and inserting this into the minimization problem for the cost function we have +$$ +(\rho_1) \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2. +$$ +
diff --git a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
index 835164d95..333396a8e 100644
--- a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
+++ b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
@@ -10,7 +10,7 @@
" \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
- "Date: **Nov 11, 2019**\n",
+ "Date: **Nov 12, 2019**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -2580,6 +2580,57 @@
"$$"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We define a real function $h_m(x)$ that defines our final function $f_M(x)$ as"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "f_M(x) = \\sum_{m=0}^M h_m(x).\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the steepest decent approach we approximate $h_m(x) = -\\rho_m g_m(x)$, where $\\rho_m$ is a scalar and $g_m(x)$ the gradient defined as"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "g_m(x_i) = \\left[ \\frac{\\partial {\\cal L}(y_i, f(x_i))}{\\partial f(x_i)}\\right]_{f(x_i)=f_{m-1}(x_i)}.\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "With the new gradient we can update $f_m(x) = f_{m-1}(x) -\\rho_m g_m(x)$. Using the above squared-error function we see that\n",
+ "the gradient is $g_m(x_i) = -2(y_i-f(x_i))$.\n",
+ "\n",
+ "Choosing $f_0(x)=0$ we obtain $g_m(x) = -2y_i$ and inserting this into the minimization problem for the cost function we have"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "(\\rho_1) \\mathrm{argmin}_{\\rho}\\hspace{0.1cm} \\sum_{i=0}^{n-1}(y_i+2\\rho y_i)^2.\n",
+ "$$"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
diff --git a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz
index 19258d47e..65c88d19d 100644
Binary files a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz and b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz differ
diff --git a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf
index be995b115..42f87b2c2 100644
Binary files a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf and b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf differ
diff --git a/doc/src/DecisionTrees/DecisionTrees.do.txt b/doc/src/DecisionTrees/DecisionTrees.do.txt
index b019ed3ec..d728c55d8 100644
--- a/doc/src/DecisionTrees/DecisionTrees.do.txt
+++ b/doc/src/DecisionTrees/DecisionTrees.do.txt
@@ -1960,7 +1960,29 @@ This means that for every iteration, we need to optimize
\]
!et
+We define a real function $h_m(x)$ that defines our final function $f_M(x)$ as
+!bt
+\[
+f_M(x) = \sum_{m=0}^M h_m(x).
+\]
+!et
+In the steepest decent approach we approximate $h_m(x) = -\rho_m g_m(x)$, where $\rho_m$ is a scalar and $g_m(x)$ the gradient defined as
+!bt
+\[
+g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}.
+\]
+!et
+
+With the new gradient we can update $f_m(x) = f_{m-1}(x) -\rho_m g_m(x)$. Using the above squared-error function we see that
+the gradient is $g_m(x_i) = -2(y_i-f(x_i))$.
+
+Choosing $f_0(x)=0$ we obtain $g_m(x) = -2y_i$ and inserting this into the minimization problem for the cost function we have
+!bt
+\[
+(\rho_1) \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2.
+\]
+!et
!split