correcting typos
This commit is contained in:
@@ -1691,7 +1691,7 @@ o Establish a cost function, here ${\cal C}(\bm{y},\bm{f}) = \frac{1}{n} \sum_{i
|
||||
o Initialize with a guess $f_0(x)$. It could be one or even zero or some random numbers.
|
||||
o For $m=1:M$
|
||||
o minimize $\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2$ wrt $\gamma$ and $\beta$
|
||||
o This gives the optimial values $\beta_m$ and $\gamma_m$
|
||||
o This gives the optimal values $\beta_m$ and $\gamma_m$
|
||||
o Determine then the new values $f_m(x)=f_{m-1}(x) +\beta_m b(x;\gamma_m)$
|
||||
|
||||
We could use any of the algorithms we have discussed till now. If we
|
||||
@@ -1700,13 +1700,13 @@ at the internal nodes, and the predictions at the terminal nodes.
|
||||
|
||||
|
||||
!split
|
||||
===== Squared Error Exampe and Iterative Fitting =====
|
||||
===== Squared-Error Example and Iterative Fitting =====
|
||||
|
||||
To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.
|
||||
|
||||
For simplicity we assume also that our functions $b(x;\gamma)=1+\gamma x$.
|
||||
|
||||
This means that for every iteration, we need to optmize
|
||||
This means that for every iteration, we need to optimize
|
||||
|
||||
!bt
|
||||
\[
|
||||
@@ -1727,7 +1727,7 @@ and
|
||||
\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0.
|
||||
\]
|
||||
!et
|
||||
We can then rewrite these equations as (defining $w_i=1+\gamma x_i)$
|
||||
We can then rewrite these equations as (defining $\bm{w}=\bm{e}+\gamma x_i)$ with $\bm{e}$ being the unit vector)
|
||||
!bt
|
||||
\[
|
||||
\gamma \bm{w}^T(\bm{y}-\beta\gamma \bm{w})=0,
|
||||
@@ -1740,7 +1740,7 @@ which gives us $\beta = \bm{w}^T\bm{y}/(\bm{w}^T\bm{w})$. Similarly we have
|
||||
\]
|
||||
!et
|
||||
|
||||
which leads $\gamma =(\bm{x}^T\bm{y}-\beta\bm{x}^T\em{e})/(\beta\bm{x}^T\bm{x})$. Inserting
|
||||
which leads to $\gamma =(\bm{x}^T\bm{y}-\beta\bm{x}^T\bm{e})/(\beta\bm{x}^T\bm{x})$. Inserting
|
||||
for $\beta$ gives us an equation for $\gamma$.
|
||||
|
||||
The solution to these two equations gives us in turn $\beta_1$ and $\gamma_1$ leading to the new expression for $f_1(x)$ as
|
||||
@@ -1749,7 +1749,7 @@ $f_1(x) = \beta_1(1+\gamma_1x)$. Doing this $M$ times results in our final estim
|
||||
|
||||
|
||||
!split
|
||||
===== Iterative Fitting, Classification, AdaBoost =====
|
||||
===== Iterative Fitting, Classification and AdaBoost =====
|
||||
|
||||
Let us consider a binary classification problem with two outcomes $y_i \in \{-1,1\}$ and $i=0,1,2,\dots,n-1$ as our set of
|
||||
observations. We define a classification function $G(x)$ which produces a prediction taking any of the two values
|
||||
@@ -1940,7 +1940,7 @@ plt.show()
|
||||
!split
|
||||
===== Gradient boosting: Basics =====
|
||||
|
||||
Gradient boosting is again a similar technique to Adapative boosting,
|
||||
Gradient boosting is again a similar technique to Adaptive boosting,
|
||||
it combines so-called weak classifiers or regressors into a strong
|
||||
method via a series of iterations.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user