+The basic idea is to combine weak classifiers in order to create a good +classifier. With a weak classifier we often intend a classifier which +produces results which are only slightly better than we would get by +random guesses. + +
+This is done by applying in an iterative way a weak (or a standard +classifier like decision trees) to modify the data. In each iteration +we emphasize those observations which are misclassified by weighting +them with a factor.
+The algorithm here is rather straightforward. Assume that our weak +classifier is a decision tree and we consider a binary set of outputs +with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of +observations. Our design matrix is given in terms of the +feature/predictor vectors +\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1} \). Finally, we define also a +classifier determined by our data via a function \( G(\boldsymbol{X}) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \). + +
+We can then define the misclassification error \( \mathrm{err} \) as +
+$$
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\boldsymbol{X}_{i*}),
+$$
+
+
+where the function \( I() \) is one if we misclassify and zero if we classify correctly.
+The basic idea is to combine weak classifiers in order to create a good +classifier. With a weak classifier we often intend a classifier which +produces results which are only slightly better than we would get by +random guesses. + +
+This is done by applying in an iterative way a weak (or a standard +classifier like decision trees) to modify the data. In each iteration +we emphasize those observations which are misclassified by weighting +them with a factor. +
+The algorithm here is rather straightforward. Assume that our weak +classifier is a decision tree and we consider a binary set of outputs +with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of +observations. Our design matrix is given in terms of the +feature/predictor vectors +\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1} \). Finally, we define also a +classifier determined by our data via a function \( G(\boldsymbol{X}) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \). + +
+We can then define the misclassification error \( \mathrm{err} \) as +$$ +\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\boldsymbol{X}_{i*}), +$$ + +where the function \( I() \) is one if we misclassify and zero if we classify correctly. +
diff --git a/doc/pub/DecisionTrees/html/DecisionTrees.html b/doc/pub/DecisionTrees/html/DecisionTrees.html
index dc47b1555..f6c752b84 100644
--- a/doc/pub/DecisionTrees/html/DecisionTrees.html
+++ b/doc/pub/DecisionTrees/html/DecisionTrees.html
@@ -1783,11 +1783,40 @@ Example will be added here.
+The basic idea is to combine weak classifiers in order to create a good +classifier. With a weak classifier we often intend a classifier which +produces results which are only slightly better than we would get by +random guesses. + +
+This is done by applying in an iterative way a weak (or a standard +classifier like decision trees) to modify the data. In each iteration +we emphasize those observations which are misclassified by weighting +them with a factor. +
+The algorithm here is rather straightforward. Assume that our weak +classifier is a decision tree and we consider a binary set of outputs +with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of +observations. Our design matrix is given in terms of the +feature/predictor vectors +\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1} \). Finally, we define also a +classifier determined by our data via a function \( G(\boldsymbol{X}) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \). + +
+We can then define the misclassification error \( \mathrm{err} \) as +$$ +\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\boldsymbol{X}_{i*}), +$$ + +where the function \( I() \) is one if we misclassify and zero if we classify correctly. +
diff --git a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
index 3739cfe22..5a6dac0b3 100644
--- a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
+++ b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb
@@ -1825,11 +1825,44 @@
"\n",
"## Boosting, a Bird'e Eye\n",
"\n",
+ "The basic idea is to combine weak classifiers in order to create a good\n",
+ "classifier. With a weak classifier we often intend a classifier which\n",
+ "produces results which are only slightly better than we would get by\n",
+ "random guesses.\n",
"\n",
+ "This is done by applying in an iterative way a weak (or a standard\n",
+ "classifier like decision trees) to modify the data. In each iteration\n",
+ "we emphasize those observations which are misclassified by weighting\n",
+ "them with a factor.\n",
"\n",
"\n",
"## Adaptive boosting: AdaBoost, Basic Algorithm\n",
"\n",
+ "The algorithm here is rather straightforward. Assume that our weak\n",
+ "classifier is a decision tree and we consider a binary set of outputs\n",
+ "with $y_i \\in \\{-1,1\\}$ and $i=0,1,2,\\dots,n-1$ as our set of\n",
+ "observations. Our design matrix is given in terms of the\n",
+ "feature/predictor vectors\n",
+ "$\\boldsymbol{X}=[\\boldsymbol{x}_0\\boldsymbol{x}_1\\dots\\boldsymbol{x}_{p-1}$. Finally, we define also a\n",
+ "classifier determined by our data via a function $G(\\boldsymbol{X})$. This function tells us how well we are able to classify our outputs/targets $\\boldsymbol{y}$. \n",
+ "\n",
+ "We can then define the misclassification error $\\mathrm{err}$ as"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "\\mathrm{err}=\\frac{1}{n}\\sum_{i=0}^{n-1}I(y_i\\ne G(\\boldsymbol{X}_{i*}),\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "where the function $I()$ is one if we misclassify and zero if we classify correctly. \n",
"\n",
"## AdaBoost Examples"
]
diff --git a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz
index cc84654c3..b20c24052 100644
Binary files a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz and b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz differ
diff --git a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf
index 233d1821a..6e245293f 100644
Binary files a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf and b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf differ
diff --git a/doc/src/DecisionTrees/DecisionTrees.do.txt b/doc/src/DecisionTrees/DecisionTrees.do.txt
index a734ad86b..5365d8bde 100644
--- a/doc/src/DecisionTrees/DecisionTrees.do.txt
+++ b/doc/src/DecisionTrees/DecisionTrees.do.txt
@@ -1466,12 +1466,35 @@ Example will be added here.
!split
===== Boosting, a Bird'e Eye =====
+The basic idea is to combine weak classifiers in order to create a good
+classifier. With a weak classifier we often intend a classifier which
+produces results which are only slightly better than we would get by
+random guesses.
+This is done by applying in an iterative way a weak (or a standard
+classifier like decision trees) to modify the data. In each iteration
+we emphasize those observations which are misclassified by weighting
+them with a factor.
!split
===== Adaptive boosting: AdaBoost, Basic Algorithm =====
+The algorithm here is rather straightforward. Assume that our weak
+classifier is a decision tree and we consider a binary set of outputs
+with $y_i \in \{-1,1\}$ and $i=0,1,2,\dots,n-1$ as our set of
+observations. Our design matrix is given in terms of the
+feature/predictor vectors
+$\bm{X}=[\bm{x}_0\bm{x}_1\dots\bm{x}_{p-1}$. Finally, we define also a
+classifier determined by our data via a function $G(\bm{X})$. This function tells us how well we are able to classify our outputs/targets $\bm{y}$.
+
+We can then define the misclassification error $\mathrm{err}$ as
+!bt
+\[
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\bm{X}_{i*}),
+\]
+!et
+where the function $I()$ is one if we misclassify and zero if we classify correctly.
!split
===== AdaBoost Examples =====