which leads to \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\boldsymbol{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting -for \( \beta \) gives us an equation for \( \gamma \). +for \( \beta \) gives us an equation for \( \gamma \). This is a non-linear equation in the unknown \( \gamma \) and has to be solved numerically.
The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html index 4a6f767cb..ce76f15ab 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html @@ -269,7 +269,7 @@ MathJax.Hub.Config({
-Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function +Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function $$ C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$ diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html index 59ee06a8b..9b9b3510e 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html @@ -2098,7 +2098,7 @@ $$ $$
-We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma x_i) \) with \( \boldsymbol{e} \) being the unit vector)
+We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma \boldsymbol{x}) \) with \( \boldsymbol{e} \) being the unit vector)
$$
\gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0,
@@ -2114,7 +2114,7 @@ $$
which leads to \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\boldsymbol{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting -for \( \beta \) gives us an equation for \( \gamma \). +for \( \beta \) gives us an equation for \( \gamma \). This is a non-linear equation in the unknown \( \gamma \) and has to be solved numerically.
The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as @@ -2376,7 +2376,7 @@ See discussion during lecture November 8.
-Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function +Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function
$$
C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2.
diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
index 27309b5be..d0156f2db 100644
--- a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
+++ b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html
@@ -2094,7 +2094,7 @@ $$
\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0.
$$
-We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma x_i) \) with \( \boldsymbol{e} \) being the unit vector)
+We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma \boldsymbol{x}) \) with \( \boldsymbol{e} \) being the unit vector)
$$
\gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0,
$$
@@ -2106,7 +2106,7 @@ $$
which leads to \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\boldsymbol{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting -for \( \beta \) gives us an equation for \( \gamma \). +for \( \beta \) gives us an equation for \( \gamma \). This is a non-linear equation in the unknown \( \gamma \) and has to be solved numerically.
The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as @@ -2337,7 +2337,7 @@ See discussion during lecture November 8.
-Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function +Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function $$ C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$ diff --git a/doc/pub/DecisionTrees/html/DecisionTrees.html b/doc/pub/DecisionTrees/html/DecisionTrees.html index f596aab80..941aaf5f3 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees.html @@ -2099,7 +2099,7 @@ $$ \frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. $$ -We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma x_i) \) with \( \boldsymbol{e} \) being the unit vector) +We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma \boldsymbol{x}) \) with \( \boldsymbol{e} \) being the unit vector) $$ \gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0, $$ @@ -2111,7 +2111,7 @@ $$
which leads to \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\boldsymbol{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting -for \( \beta \) gives us an equation for \( \gamma \). +for \( \beta \) gives us an equation for \( \gamma \). This is a non-linear equation in the unknown \( \gamma \) and has to be solved numerically.
The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as @@ -2342,7 +2342,7 @@ See discussion during lecture November 8.
-Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function +Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function $$ C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$ diff --git a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb index 4bfc45466..2aeaf36de 100644 --- a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb +++ b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb @@ -2172,7 +2172,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can then rewrite these equations as (defining $\\boldsymbol{w}=\\boldsymbol{e}+\\gamma x_i)$ with $\\boldsymbol{e}$ being the unit vector)" + "We can then rewrite these equations as (defining $\\boldsymbol{w}=\\boldsymbol{e}+\\gamma \\boldsymbol{x})$ with $\\boldsymbol{e}$ being the unit vector)" ] }, { @@ -2205,7 +2205,7 @@ "metadata": {}, "source": [ "which leads to $\\gamma =(\\boldsymbol{x}^T\\boldsymbol{y}-\\beta\\boldsymbol{x}^T\\boldsymbol{e})/(\\beta\\boldsymbol{x}^T\\boldsymbol{x})$. Inserting\n", - "for $\\beta$ gives us an equation for $\\gamma$.\n", + "for $\\beta$ gives us an equation for $\\gamma$. This is a non-linear equation in the unknown $\\gamma$ and has to be solved numerically. \n", "\n", "The solution to these two equations gives us in turn $\\beta_1$ and $\\gamma_1$ leading to the new expression for $f_1(x)$ as\n", "$f_1(x) = \\beta_1(1+\\gamma_1x)$. Doing this $M$ times results in our final estimate for the function $f$. \n", @@ -2569,7 +2569,7 @@ "\n", "## Gradient Boosting, algorithm\n", "\n", - "Suppose we have a cost function $C(f)=\\sum_{i=0}^{n-1}L(y_i, f(x_i))$ where $y_i$ is our target and $f(x_i)$ the function which is meant to model $y_i$. The above cost function could be our standard square-error function" + "Suppose we have a cost function $C(f)=\\sum_{i=0}^{n-1}L(y_i, f(x_i))$ where $y_i$ is our target and $f(x_i)$ the function which is meant to model $y_i$. The above cost function could be our standard squared-error function" ] }, { diff --git a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz index c31a23f48..246b4020e 100644 Binary files a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz and b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz differ diff --git a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf index 93861c875..21d19f2c7 100644 Binary files a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf and b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf differ diff --git a/doc/src/DecisionTrees/DecisionTrees.do.txt b/doc/src/DecisionTrees/DecisionTrees.do.txt index 1f829c9ec..0dfa6de41 100644 --- a/doc/src/DecisionTrees/DecisionTrees.do.txt +++ b/doc/src/DecisionTrees/DecisionTrees.do.txt @@ -1727,7 +1727,7 @@ and \frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. \] !et -We can then rewrite these equations as (defining $\bm{w}=\bm{e}+\gamma x_i)$ with $\bm{e}$ being the unit vector) +We can then rewrite these equations as (defining $\bm{w}=\bm{e}+\gamma \bm{x})$ with $\bm{e}$ being the unit vector) !bt \[ \gamma \bm{w}^T(\bm{y}-\beta\gamma \bm{w})=0, @@ -1741,7 +1741,7 @@ which gives us $\beta = \bm{w}^T\bm{y}/(\bm{w}^T\bm{w})$. Similarly we have !et which leads to $\gamma =(\bm{x}^T\bm{y}-\beta\bm{x}^T\bm{e})/(\beta\bm{x}^T\bm{x})$. Inserting -for $\beta$ gives us an equation for $\gamma$. +for $\beta$ gives us an equation for $\gamma$. This is a non-linear equation in the unknown $\gamma$ and has to be solved numerically. The solution to these two equations gives us in turn $\beta_1$ and $\gamma_1$ leading to the new expression for $f_1(x)$ as $f_1(x) = \beta_1(1+\gamma_1x)$. Doing this $M$ times results in our final estimate for the function $f$. @@ -1953,7 +1953,7 @@ See discussion during lecture November 8. !split ===== Gradient Boosting, algorithm ===== -Suppose we have a cost function $C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i))$ where $y_i$ is our target and $f(x_i)$ the function which is meant to model $y_i$. The above cost function could be our standard square-error function +Suppose we have a cost function $C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i))$ where $y_i$ is our target and $f(x_i)$ the function which is meant to model $y_i$. The above cost function could be our standard squared-error function !bt \[ C(\bm{y},\bm{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2.