material to decision tree

2019-11-02 17:47:47 +01:00
parent 1dc910ff05
commit 7e652dabff
7 changed files with 145 additions and 0 deletions
@@ -1800,11 +1800,42 @@ Example will be added here.

 <section>
 <h2 id="___sec42">Boosting, a Bird'e Eye </h2>
+
+<p>
+The basic idea is to combine weak classifiers in order to create a good
+classifier. With a weak classifier we often intend a classifier which
+produces results which are only slightly better than we would get by
+random guesses.
+
+<p>
+This is done by applying in an iterative way a weak (or a standard
+classifier like decision trees) to modify the data. In each iteration
+we emphasize those observations which are misclassified by weighting
+them with a factor.
 </section>


 <section>
 <h2 id="___sec43">Adaptive boosting: AdaBoost, Basic Algorithm </h2>
+
+<p>
+The algorithm here is rather straightforward. Assume that our weak
+classifier is a decision tree and we consider a binary set of outputs
+with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of
+observations. Our design matrix is given in terms of the
+feature/predictor vectors
+\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1} \). Finally, we define also a
+classifier determined by our data via a function \( G(\boldsymbol{X}) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \).
+
+<p>
+We can then define the misclassification error \( \mathrm{err} \) as
+<p>&nbsp;<br>
+$$
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\boldsymbol{X}_{i*}),
+$$
+<p>&nbsp;<br>
+
+where the function \( I() \) is one if we misclassify and zero if we classify correctly.
 </section>


@@ -1778,11 +1778,40 @@ Example will be added here.

 <h2 id="___sec42">Boosting, a Bird'e Eye </h2>

+<p>
+The basic idea is to combine weak classifiers in order to create a good
+classifier. With a weak classifier we often intend a classifier which
+produces results which are only slightly better than we would get by
+random guesses.
+
+<p>
+This is done by applying in an iterative way a weak (or a standard
+classifier like decision trees) to modify the data. In each iteration
+we emphasize those observations which are misclassified by weighting
+them with a factor.
+
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

 <h2 id="___sec43">Adaptive boosting: AdaBoost, Basic Algorithm </h2>

+<p>
+The algorithm here is rather straightforward. Assume that our weak
+classifier is a decision tree and we consider a binary set of outputs
+with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of
+observations. Our design matrix is given in terms of the
+feature/predictor vectors
+\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1} \). Finally, we define also a
+classifier determined by our data via a function \( G(\boldsymbol{X}) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \).
+
+<p>
+We can then define the misclassification error \( \mathrm{err} \) as
+$$
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\boldsymbol{X}_{i*}),
+$$
+
+where the function \( I() \) is one if we misclassify and zero if we classify correctly.
+
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

@@ -1783,11 +1783,40 @@ Example will be added here.

 <h2 id="___sec42">Boosting, a Bird'e Eye </h2>

+<p>
+The basic idea is to combine weak classifiers in order to create a good
+classifier. With a weak classifier we often intend a classifier which
+produces results which are only slightly better than we would get by
+random guesses.
+
+<p>
+This is done by applying in an iterative way a weak (or a standard
+classifier like decision trees) to modify the data. In each iteration
+we emphasize those observations which are misclassified by weighting
+them with a factor.
+
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

 <h2 id="___sec43">Adaptive boosting: AdaBoost, Basic Algorithm </h2>

+<p>
+The algorithm here is rather straightforward. Assume that our weak
+classifier is a decision tree and we consider a binary set of outputs
+with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of
+observations. Our design matrix is given in terms of the
+feature/predictor vectors
+\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1} \). Finally, we define also a
+classifier determined by our data via a function \( G(\boldsymbol{X}) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \).
+
+<p>
+We can then define the misclassification error \( \mathrm{err} \) as
+$$
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\boldsymbol{X}_{i*}),
+$$
+
+where the function \( I() \) is one if we misclassify and zero if we classify correctly.
+
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

@@ -1825,11 +1825,44 @@
    "\n",
    "## Boosting, a Bird'e Eye\n",
    "\n",
+    "The basic idea is to combine weak classifiers in order to create a good\n",
+    "classifier. With a weak classifier we often intend a classifier which\n",
+    "produces results which are only slightly better than we would get by\n",
+    "random guesses.\n",
    "\n",
+    "This is done by applying in an iterative way a weak (or a standard\n",
+    "classifier like decision trees) to modify the data. In each iteration\n",
+    "we emphasize those observations which are misclassified by weighting\n",
+    "them with a factor.\n",
    "\n",
    "\n",
    "## Adaptive boosting: AdaBoost, Basic Algorithm\n",
    "\n",
+    "The algorithm here is rather straightforward. Assume that our weak\n",
+    "classifier is a decision tree and we consider a binary set of outputs\n",
+    "with $y_i \\in \\{-1,1\\}$ and $i=0,1,2,\\dots,n-1$ as our set of\n",
+    "observations. Our design matrix is given in terms of the\n",
+    "feature/predictor vectors\n",
+    "$\\boldsymbol{X}=[\\boldsymbol{x}_0\\boldsymbol{x}_1\\dots\\boldsymbol{x}_{p-1}$. Finally, we define also a\n",
+    "classifier determined by our data via a function $G(\\boldsymbol{X})$. This function tells us how well we are able to classify our outputs/targets $\\boldsymbol{y}$. \n",
+    "\n",
+    "We can then define the misclassification error $\\mathrm{err}$ as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\mathrm{err}=\\frac{1}{n}\\sum_{i=0}^{n-1}I(y_i\\ne G(\\boldsymbol{X}_{i*}),\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where the function $I()$ is one if we misclassify and zero if we classify correctly. \n",
    "\n",
    "## AdaBoost Examples"
   ]
@@ -1466,12 +1466,35 @@ Example will be added here.
 !split
 ===== Boosting, a Bird'e Eye =====

+The basic idea is to combine weak classifiers in order to create a good
+classifier. With a weak classifier we often intend a classifier which
+produces results which are only slightly better than we would get by
+random guesses.

+This is done by applying in an iterative way a weak (or a standard
+classifier like decision trees) to modify the data. In each iteration
+we emphasize those observations which are misclassified by weighting
+them with a factor.


 !split
 ===== Adaptive boosting: AdaBoost, Basic Algorithm =====

+The algorithm here is rather straightforward. Assume that our weak
+classifier is a decision tree and we consider a binary set of outputs
+with $y_i \in \{-1,1\}$ and $i=0,1,2,\dots,n-1$ as our set of
+observations. Our design matrix is given in terms of the
+feature/predictor vectors
+$\bm{X}=[\bm{x}_0\bm{x}_1\dots\bm{x}_{p-1}$. Finally, we define also a
+classifier determined by our data via a function $G(\bm{X})$. This function tells us how well we are able to classify our outputs/targets $\bm{y}$. 
+
+We can then define the misclassification error $\mathrm{err}$ as
+!bt
+\[
+\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(\bm{X}_{i*}),
+\]
+!et
+where the function $I()$ is one if we misclassify and zero if we classify correctly. 

 !split
 ===== AdaBoost Examples =====