material to decision tree

doc/src/DecisionTrees/DecisionTrees.do.txt

@@ -1466,12 +1466,35 @@ Example will be added here.
 !split
 ===== Boosting, a Bird'e Eye =====
 
+The basic idea is to combine weak classifiers in order to create a good
+classifier. With a weak classifier we often intend a classifier which
+produces results which are only slightly better than we would get by
+random guesses.
 
+This is done by applying in an iterative way a weak (or a standard
+classifier like decision trees) to modify the data. In each iteration
+we emphasize those observations which are misclassified by weighting
+them with a factor.
 
 
 !split
 ===== Adaptive boosting: AdaBoost, Basic Algorithm =====
 
+The algorithm here is rather straightforward. Assume that our weak
+classifier is a decision tree and we consider a binary set of outputs
+with $y_i \in \{-1,1\}$ and $i=0,1,2,\dots,n-1$ as our set of
+observations. Our design matrix is given in terms of the
+feature/predictor vectors
+$\bm{X}=[\bm{x}_0\bm{x}_1\dots\bm{x}_{p-1}$. Finally, we define also a
+classifier determined by our data via a function $G(\bm{X})$. This function tells us how well we are able to classify our outputs/targets $\bm{y}$. 
+
+We can then define the misclassification error $\mathrm{err}$ as
+!bt
+\[
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\bm{X}_{i*}),
+\]
+!et
+where the function $I()$ is one if we misclassify and zero if we classify correctly. 
 
 !split
 ===== AdaBoost Examples =====