lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-11-01 16:50:26 +01:00
parent 8fe7a8ec93
commit ec963b76e0
10 changed files with 5155 additions and 2839 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/jupyter_execute/Clustering.ipynb
+41 -41
View File
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "2150517f",
"id": "04229efd",
"metadata": {},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
@@ -11,7 +11,7 @@
},
{
"cell_type": "markdown",
"id": "f6e0cd96",
"id": "e2e714e4",
"metadata": {},
"source": [
"# Clustering Analysis\n",
@@ -37,7 +37,7 @@
},
{
"cell_type": "markdown",
"id": "e755ea79",
"id": "1466c1cb",
"metadata": {},
"source": [
"## Basic Idea of the K-means Clustering Algorithm\n",
@@ -68,7 +68,7 @@
},
{
"cell_type": "markdown",
"id": "682f37d4",
"id": "30580c15",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
@@ -83,7 +83,7 @@
},
{
"cell_type": "markdown",
"id": "3bd80ebf",
"id": "c430172a",
"metadata": {},
"source": [
"which we wish to group into $K < n$ clusters. For our dissimilarity measure we\n",
@@ -92,7 +92,7 @@
},
{
"cell_type": "markdown",
"id": "c34b3e47",
"id": "885eba7f",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
@@ -108,7 +108,7 @@
},
{
"cell_type": "markdown",
"id": "54897ef5",
"id": "485a4b5c",
"metadata": {},
"source": [
"Next we define the so called *within-cluster point scatter* which gives us a\n",
@@ -118,7 +118,7 @@
},
{
"cell_type": "markdown",
"id": "02ec7cdd",
"id": "1d3c6ca7",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
@@ -135,7 +135,7 @@
},
{
"cell_type": "markdown",
"id": "d15918ea",
"id": "33a14233",
"metadata": {},
"source": [
"where $\\boldsymbol{\\overline{x_k}}$ is the mean vector associated with the $k$-th\n",
@@ -150,7 +150,7 @@
},
{
"cell_type": "markdown",
"id": "8576296d",
"id": "7044f7fc",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
@@ -169,7 +169,7 @@
},
{
"cell_type": "markdown",
"id": "2736464c",
"id": "7d612869",
"metadata": {},
"source": [
"Which is a quantity that is conserved throughout the $k$-means algorithm. It can\n",
@@ -183,7 +183,7 @@
},
{
"cell_type": "markdown",
"id": "bbe04304",
"id": "3dc3f01f",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
@@ -198,7 +198,7 @@
},
{
"cell_type": "markdown",
"id": "1e8910ec",
"id": "f51d30c2",
"metadata": {},
"source": [
"Now we have all the pieces necessary to formally revisit the k-means algorithm.\n",
@@ -209,7 +209,7 @@
},
{
"cell_type": "markdown",
"id": "bb548d32",
"id": "9265c30e",
"metadata": {},
"source": [
"## The K-means Clustering Algorithm\n",
@@ -244,7 +244,7 @@
},
{
"cell_type": "markdown",
"id": "bc569914",
"id": "63dc40f2",
"metadata": {},
"source": [
"## Writing Our Own Code\n",
@@ -255,7 +255,7 @@
},
{
"cell_type": "markdown",
"id": "8601ba47",
"id": "9e110f91",
"metadata": {},
"source": [
"### Basic Python\n",
@@ -277,7 +277,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "3023fb96",
"id": "c5196fbb",
"metadata": {},
"outputs": [],
"source": [
@@ -294,7 +294,7 @@
},
{
"cell_type": "markdown",
"id": "18feed32",
"id": "46f6bf75",
"metadata": {},
"source": [
"Next we define functions, for ease of use later, to generate Gaussians and to\n",
@@ -304,7 +304,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "171b8aa5",
"id": "c5db2b9c",
"metadata": {},
"outputs": [
{
@@ -377,7 +377,7 @@
},
{
"cell_type": "markdown",
"id": "22a5cc09",
"id": "123f8b0a",
"metadata": {},
"source": [
"Now that we are our, albeit very simple, dataset we are ready to start\n",
@@ -387,7 +387,7 @@
{
"cell_type": "code",
"execution_count": 3,
"id": "7ef1f112",
"id": "44d87c0c",
"metadata": {},
"outputs": [],
"source": [
@@ -428,7 +428,7 @@
},
{
"cell_type": "markdown",
"id": "6fe8327e",
"id": "b0bf3a01",
"metadata": {},
"source": [
"Let's plot and see"
@@ -437,7 +437,7 @@
{
"cell_type": "code",
"execution_count": 4,
"id": "79427d57",
"id": "4af50c01",
"metadata": {},
"outputs": [
{
@@ -474,7 +474,7 @@
},
{
"cell_type": "markdown",
"id": "5d14cf57",
"id": "9a370b9d",
"metadata": {},
"source": [
"So what do we have so far? We have 'picked' $k$ centroids at random from our\n",
@@ -492,7 +492,7 @@
{
"cell_type": "code",
"execution_count": 5,
"id": "4c9c9a53",
"id": "7e049bf1",
"metadata": {},
"outputs": [
{
@@ -500,7 +500,7 @@
"output_type": "stream",
"text": [
"Converged at iteration 5\n",
"Runtime: 0.24420595169067383 seconds\n"
"Runtime: 0.2396700382232666 seconds\n"
]
}
],
@@ -557,7 +557,7 @@
},
{
"cell_type": "markdown",
"id": "61f30110",
"id": "2602cc83",
"metadata": {},
"source": [
"And thats it! We now have an extremely barebones, un-optimized k-means\n",
@@ -567,7 +567,7 @@
{
"cell_type": "code",
"execution_count": 6,
"id": "d964670a",
"id": "743467eb",
"metadata": {},
"outputs": [
{
@@ -604,7 +604,7 @@
},
{
"cell_type": "markdown",
"id": "351ef866",
"id": "1f2749e0",
"metadata": {},
"source": [
"Now there are a few glaring improvements to be done here. First of all is\n",
@@ -617,7 +617,7 @@
},
{
"cell_type": "markdown",
"id": "1d80d4b2",
"id": "ac9fa4a5",
"metadata": {},
"source": [
"## Towards a More Numpythonic Code"
@@ -626,7 +626,7 @@
{
"cell_type": "code",
"execution_count": 7,
"id": "a88b26c9",
"id": "9edf7db3",
"metadata": {},
"outputs": [
{
@@ -634,7 +634,7 @@
"output_type": "stream",
"text": [
"Converged at iteration: 5\n",
"Runtime: 0.20550775527954102 seconds\n"
"Runtime: 0.20168113708496094 seconds\n"
]
}
],
@@ -748,7 +748,7 @@
},
{
"cell_type": "markdown",
"id": "7372c09e",
"id": "212c648e",
"metadata": {},
"source": [
"**Note**: the start of the timing is after the random initialization, and first\n",
@@ -769,7 +769,7 @@
{
"cell_type": "code",
"execution_count": 8,
"id": "566f0e04",
"id": "07bfff3f",
"metadata": {},
"outputs": [
{
@@ -777,7 +777,7 @@
"output_type": "stream",
"text": [
"Converged at iteration: 11\n",
"Runtime: 0.38791799545288086 seconds\n",
"Runtime: 0.389873743057251 seconds\n",
" "
]
}
@@ -789,7 +789,7 @@
},
{
"cell_type": "markdown",
"id": "44ae7ea9",
"id": "b7f27a4a",
"metadata": {},
"source": [
"Here we can see the reason for profiling. We now know for certain a lot can be\n",
@@ -801,7 +801,7 @@
{
"cell_type": "code",
"execution_count": 9,
"id": "93b58a7f",
"id": "b9ffa227",
"metadata": {},
"outputs": [],
"source": [
@@ -892,7 +892,7 @@
},
{
"cell_type": "markdown",
"id": "b2ce6830",
"id": "da6c65d3",
"metadata": {},
"source": [
"When working towards becoming a data scientist using Python this last step is\n",
@@ -905,7 +905,7 @@
{
"cell_type": "code",
"execution_count": 10,
"id": "cd0e1669",
"id": "ea31860d",
"metadata": {},
"outputs": [
{
@@ -913,7 +913,7 @@
"output_type": "stream",
"text": [
"Converged at iteration: 5\n",
"Runtime: 0.0023779869079589844 seconds\n"
"Runtime: 0.002451181411743164 seconds\n"
]
}
],
@@ -924,7 +924,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "d86b2a09",
"id": "2a34f7af",
"metadata": {},
"outputs": [],
"source": []
doc/pub/week34/ipynb/week34.ipynb
+1013 -256
View File
File diff suppressed because one or more lines are too long
doc/pub/week38/ipynb/week38.ipynb
+607 -141
View File
File diff suppressed because one or more lines are too long
doc/pub/week44/html/week44-bs.html
+176 -132
View File
@@ -1,31 +1,28 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
HTML file automatically generated from DocOnce source
(https://github.com/doconce/doconce/)
doconce format html week44.do.txt --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=week44-bs --no_mako
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 44: From Decision Trees to Bagging methods">
<title>Week 44: From Decision Trees to Bagging methods</title>
<!-- Bootstrap style: bootstrap -->
<!-- doconce format html week44.do.txt --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=week44-bs --no_mako -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
@@ -35,85 +32,146 @@ Automatically generated HTML file from DocOnce source
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 44', 2, None, '___sec0'),
('Thursday', 2, None, '___sec1'),
('Decision trees, overarching aims', 2, None, '___sec2'),
('Basics of a tree', 2, None, '___sec3'),
('A Sketch of a Tree, Regression problem', 2, None, '___sec4'),
'sections': [('Overview of week 44', 2, None, 'overview-of-week-44'),
('Thursday', 2, None, 'thursday'),
('Decision trees, overarching aims',
2,
None,
'decision-trees-overarching-aims'),
('Basics of a tree', 2, None, 'basics-of-a-tree'),
('A Sketch of a Tree, Regression problem',
2,
None,
'a-sketch-of-a-tree-regression-problem'),
('A Sketch of a Tree, Classification problem',
2,
None,
'___sec5'),
'a-sketch-of-a-tree-classification-problem'),
('A typical Decision Tree with its pertinent Jargon, '
'Classification Problem',
2,
None,
'___sec6'),
('General Features', 2, None, '___sec7'),
('How do we set it up?', 2, None, '___sec8'),
('Decision trees and Regression', 2, None, '___sec9'),
('Building a tree, regression', 2, None, '___sec10'),
'a-typical-decision-tree-with-its-pertinent-jargon-classification-problem'),
('General Features', 2, None, 'general-features'),
('How do we set it up?', 2, None, 'how-do-we-set-it-up'),
('Decision trees and Regression',
2,
None,
'decision-trees-and-regression'),
('Building a tree, regression',
2,
None,
'building-a-tree-regression'),
('A top-down approach, recursive binary splitting',
2,
None,
'___sec11'),
('Making a tree', 2, None, '___sec12'),
('Pruning the tree', 2, None, '___sec13'),
('Cost complexity pruning', 2, None, '___sec14'),
('Schematic Regression Procedure', 2, None, '___sec15'),
('A Classification Tree', 2, None, '___sec16'),
('Growing a classification tree', 2, None, '___sec17'),
('Classification tree, how to split nodes', 2, None, '___sec18'),
('Visualizing the Tree, Classification', 2, None, '___sec19'),
('Visualizing the Tree, The Moons', 2, None, '___sec20'),
('Other ways of visualizing the trees', 2, None, '___sec21'),
('Printing out as text', 2, None, '___sec22'),
('Algorithms for Setting up Decision Trees', 2, None, '___sec23'),
('The CART algorithm for Classification', 2, None, '___sec24'),
('The CART algorithm for Regression', 2, None, '___sec25'),
('Computing the Gini index', 2, None, '___sec26'),
'a-top-down-approach-recursive-binary-splitting'),
('Making a tree', 2, None, 'making-a-tree'),
('Pruning the tree', 2, None, 'pruning-the-tree'),
('Cost complexity pruning', 2, None, 'cost-complexity-pruning'),
('Schematic Regression Procedure',
2,
None,
'schematic-regression-procedure'),
('A Classification Tree', 2, None, 'a-classification-tree'),
('Growing a classification tree',
2,
None,
'growing-a-classification-tree'),
('Classification tree, how to split nodes',
2,
None,
'classification-tree-how-to-split-nodes'),
('Visualizing the Tree, Classification',
2,
None,
'visualizing-the-tree-classification'),
('Visualizing the Tree, The Moons',
2,
None,
'visualizing-the-tree-the-moons'),
('Other ways of visualizing the trees',
2,
None,
'other-ways-of-visualizing-the-trees'),
('Printing out as text', 2, None, 'printing-out-as-text'),
('Algorithms for Setting up Decision Trees',
2,
None,
'algorithms-for-setting-up-decision-trees'),
('The CART algorithm for Classification',
2,
None,
'the-cart-algorithm-for-classification'),
('The CART algorithm for Regression',
2,
None,
'the-cart-algorithm-for-regression'),
('Computing the Gini index', 2, None, 'computing-the-gini-index'),
('Simple Python Code to read in Data and perform Classification',
2,
None,
'___sec27'),
('Computing the Gini Factor', 2, None, '___sec28'),
('Entropy and the ID3 algorithm', 2, None, '___sec29'),
'simple-python-code-to-read-in-data-and-perform-classification'),
('Computing the Gini Factor',
2,
None,
'computing-the-gini-factor'),
('Entropy and the ID3 algorithm',
2,
None,
'entropy-and-the-id3-algorithm'),
('Cancer Data again now with Decision Trees and other Methods',
2,
None,
'___sec30'),
('Another example, the moons again', 2, None, '___sec31'),
('Playing around with regions', 2, None, '___sec32'),
('Regression trees', 2, None, '___sec33'),
('Final regressor code', 2, None, '___sec34'),
('Pros and cons of trees, pros', 2, None, '___sec35'),
('Disadvantages', 2, None, '___sec36'),
'cancer-data-again-now-with-decision-trees-and-other-methods'),
('Another example, the moons again',
2,
None,
'another-example-the-moons-again'),
('Playing around with regions',
2,
None,
'playing-around-with-regions'),
('Regression trees', 2, None, 'regression-trees'),
('Final regressor code', 2, None, 'final-regressor-code'),
('Pros and cons of trees, pros',
2,
None,
'pros-and-cons-of-trees-pros'),
('Disadvantages', 2, None, 'disadvantages'),
('Ensemble Methods: From a Single Tree to Many Trees and Extreme '
'Boosting, Meet the Jungle of Methods',
2,
None,
'___sec37'),
('An Overview of Ensemble Methods', 2, None, '___sec38'),
('Bagging', 2, None, '___sec39'),
('More bagging', 2, None, '___sec40'),
('Simple Voting Example, head or tail', 2, None, '___sec41'),
('Using the Voting Classifier', 2, None, '___sec42'),
'ensemble-methods-from-a-single-tree-to-many-trees-and-extreme-boosting-meet-the-jungle-of-methods'),
('An Overview of Ensemble Methods',
2,
None,
'an-overview-of-ensemble-methods'),
('Bagging', 2, None, 'bagging'),
('More bagging', 2, None, 'more-bagging'),
('Simple Voting Example, head or tail',
2,
None,
'simple-voting-example-head-or-tail'),
('Using the Voting Classifier',
2,
None,
'using-the-voting-classifier'),
('Please, not the moons again! Voting and Bagging',
2,
None,
'___sec43'),
('Bagging Examples', 2, None, '___sec44'),
'please-not-the-moons-again-voting-and-bagging'),
('Bagging Examples', 2, None, 'bagging-examples'),
('Making your own Bootstrap: Changing the Level of the Decision '
'Tree',
2,
None,
'___sec45')]}
'making-your-own-bootstrap-changing-the-level-of-the-decision-tree')]}
end of tocinfo -->
<body>
@@ -133,8 +191,6 @@ MathJax.Hub.Config({
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
@@ -145,58 +201,57 @@ MathJax.Hub.Config({
</button>
<a class="navbar-brand" href="week44-bs.html">Week 44: From Decision Trees to Bagging methods</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week44-bs001.html#___sec0" style="font-size: 80%;">Overview of week 44</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs002.html#___sec1" style="font-size: 80%;">Thursday</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs003.html#___sec2" style="font-size: 80%;">Decision trees, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs004.html#___sec3" style="font-size: 80%;">Basics of a tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs005.html#___sec4" style="font-size: 80%;">A Sketch of a Tree, Regression problem</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs006.html#___sec5" style="font-size: 80%;">A Sketch of a Tree, Classification problem</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs007.html#___sec6" style="font-size: 80%;">A typical Decision Tree with its pertinent Jargon, Classification Problem</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs008.html#___sec7" style="font-size: 80%;">General Features</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs009.html#___sec8" style="font-size: 80%;">How do we set it up?</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs010.html#___sec9" style="font-size: 80%;">Decision trees and Regression</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs011.html#___sec10" style="font-size: 80%;">Building a tree, regression</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs012.html#___sec11" style="font-size: 80%;">A top-down approach, recursive binary splitting</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs013.html#___sec12" style="font-size: 80%;">Making a tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs014.html#___sec13" style="font-size: 80%;">Pruning the tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs015.html#___sec14" style="font-size: 80%;">Cost complexity pruning</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs016.html#___sec15" style="font-size: 80%;">Schematic Regression Procedure</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs017.html#___sec16" style="font-size: 80%;">A Classification Tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs018.html#___sec17" style="font-size: 80%;">Growing a classification tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs019.html#___sec18" style="font-size: 80%;">Classification tree, how to split nodes</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs020.html#___sec19" style="font-size: 80%;">Visualizing the Tree, Classification</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs021.html#___sec20" style="font-size: 80%;">Visualizing the Tree, The Moons</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs022.html#___sec21" style="font-size: 80%;">Other ways of visualizing the trees</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs023.html#___sec22" style="font-size: 80%;">Printing out as text</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs024.html#___sec23" style="font-size: 80%;">Algorithms for Setting up Decision Trees</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs025.html#___sec24" style="font-size: 80%;">The CART algorithm for Classification</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs026.html#___sec25" style="font-size: 80%;">The CART algorithm for Regression</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs027.html#___sec26" style="font-size: 80%;">Computing the Gini index</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs028.html#___sec27" style="font-size: 80%;">Simple Python Code to read in Data and perform Classification</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs029.html#___sec28" style="font-size: 80%;">Computing the Gini Factor</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs030.html#___sec29" style="font-size: 80%;">Entropy and the ID3 algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs031.html#___sec30" style="font-size: 80%;">Cancer Data again now with Decision Trees and other Methods</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs032.html#___sec31" style="font-size: 80%;">Another example, the moons again</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs033.html#___sec32" style="font-size: 80%;">Playing around with regions</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs034.html#___sec33" style="font-size: 80%;">Regression trees</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs035.html#___sec34" style="font-size: 80%;">Final regressor code</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs036.html#___sec35" style="font-size: 80%;">Pros and cons of trees, pros</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs037.html#___sec36" style="font-size: 80%;">Disadvantages</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs038.html#___sec37" style="font-size: 80%;">Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs039.html#___sec38" style="font-size: 80%;">An Overview of Ensemble Methods</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs040.html#___sec39" style="font-size: 80%;">Bagging</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs041.html#___sec40" style="font-size: 80%;">More bagging</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs042.html#___sec41" style="font-size: 80%;">Simple Voting Example, head or tail</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs043.html#___sec42" style="font-size: 80%;">Using the Voting Classifier</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs044.html#___sec43" style="font-size: 80%;">Please, not the moons again! Voting and Bagging</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs045.html#___sec44" style="font-size: 80%;">Bagging Examples</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs046.html#___sec45" style="font-size: 80%;">Making your own Bootstrap: Changing the Level of the Decision Tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs001.html#overview-of-week-44" style="font-size: 80%;">Overview of week 44</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs002.html#thursday" style="font-size: 80%;">Thursday</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs003.html#decision-trees-overarching-aims" style="font-size: 80%;">Decision trees, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs004.html#basics-of-a-tree" style="font-size: 80%;">Basics of a tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs005.html#a-sketch-of-a-tree-regression-problem" style="font-size: 80%;">A Sketch of a Tree, Regression problem</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs006.html#a-sketch-of-a-tree-classification-problem" style="font-size: 80%;">A Sketch of a Tree, Classification problem</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs007.html#a-typical-decision-tree-with-its-pertinent-jargon-classification-problem" style="font-size: 80%;">A typical Decision Tree with its pertinent Jargon, Classification Problem</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs008.html#general-features" style="font-size: 80%;">General Features</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs009.html#how-do-we-set-it-up" style="font-size: 80%;">How do we set it up?</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs010.html#decision-trees-and-regression" style="font-size: 80%;">Decision trees and Regression</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs011.html#building-a-tree-regression" style="font-size: 80%;">Building a tree, regression</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs012.html#a-top-down-approach-recursive-binary-splitting" style="font-size: 80%;">A top-down approach, recursive binary splitting</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs013.html#making-a-tree" style="font-size: 80%;">Making a tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs014.html#pruning-the-tree" style="font-size: 80%;">Pruning the tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs015.html#cost-complexity-pruning" style="font-size: 80%;">Cost complexity pruning</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs016.html#schematic-regression-procedure" style="font-size: 80%;">Schematic Regression Procedure</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs017.html#a-classification-tree" style="font-size: 80%;">A Classification Tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs018.html#growing-a-classification-tree" style="font-size: 80%;">Growing a classification tree</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs019.html#classification-tree-how-to-split-nodes" style="font-size: 80%;">Classification tree, how to split nodes</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs020.html#visualizing-the-tree-classification" style="font-size: 80%;">Visualizing the Tree, Classification</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs021.html#visualizing-the-tree-the-moons" style="font-size: 80%;">Visualizing the Tree, The Moons</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs022.html#other-ways-of-visualizing-the-trees" style="font-size: 80%;">Other ways of visualizing the trees</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs023.html#printing-out-as-text" style="font-size: 80%;">Printing out as text</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs024.html#algorithms-for-setting-up-decision-trees" style="font-size: 80%;">Algorithms for Setting up Decision Trees</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs025.html#the-cart-algorithm-for-classification" style="font-size: 80%;">The CART algorithm for Classification</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs026.html#the-cart-algorithm-for-regression" style="font-size: 80%;">The CART algorithm for Regression</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs027.html#computing-the-gini-index" style="font-size: 80%;">Computing the Gini index</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs028.html#simple-python-code-to-read-in-data-and-perform-classification" style="font-size: 80%;">Simple Python Code to read in Data and perform Classification</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs029.html#computing-the-gini-factor" style="font-size: 80%;">Computing the Gini Factor</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs030.html#entropy-and-the-id3-algorithm" style="font-size: 80%;">Entropy and the ID3 algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs031.html#cancer-data-again-now-with-decision-trees-and-other-methods" style="font-size: 80%;">Cancer Data again now with Decision Trees and other Methods</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs032.html#another-example-the-moons-again" style="font-size: 80%;">Another example, the moons again</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs033.html#playing-around-with-regions" style="font-size: 80%;">Playing around with regions</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs034.html#regression-trees" style="font-size: 80%;">Regression trees</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs035.html#final-regressor-code" style="font-size: 80%;">Final regressor code</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs036.html#pros-and-cons-of-trees-pros" style="font-size: 80%;">Pros and cons of trees, pros</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs037.html#disadvantages" style="font-size: 80%;">Disadvantages</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs038.html#ensemble-methods-from-a-single-tree-to-many-trees-and-extreme-boosting-meet-the-jungle-of-methods" style="font-size: 80%;">Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs039.html#an-overview-of-ensemble-methods" style="font-size: 80%;">An Overview of Ensemble Methods</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs040.html#bagging" style="font-size: 80%;">Bagging</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs041.html#more-bagging" style="font-size: 80%;">More bagging</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs042.html#simple-voting-example-head-or-tail" style="font-size: 80%;">Simple Voting Example, head or tail</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs043.html#using-the-voting-classifier" style="font-size: 80%;">Using the Voting Classifier</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs044.html#please-not-the-moons-again-voting-and-bagging" style="font-size: 80%;">Please, not the moons again! Voting and Bagging</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs045.html#bagging-examples" style="font-size: 80%;">Bagging Examples</a></li>
<!-- navigation toc: --> <li><a href="._week44-bs046.html#making-your-own-bootstrap-changing-the-level-of-the-decision-tree" style="font-size: 80%;">Making your own Bootstrap: Changing the Level of the Decision Tree</a></li>
</ul>
</li>
@@ -204,36 +259,32 @@ MathJax.Hub.Config({
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0000"></a>
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center><h1>Week 44: From Decision Trees to Bagging methods</h1></center> <!-- document title -->
<center>
<h1>Week 44: From Decision Trees to Bagging methods</h1>
</center> <!-- document title -->
<p>
<!-- author(s): Morten Hjorth-Jensen -->
<center>
<b>Morten Hjorth-Jensen</b> [1, 2]
</center>
<p>
<!-- institution(s) -->
<center>
[1] <b>Department of Physics, University of Oslo</b>
</center>
<center>
[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b>
</center>
<br>
<center>
<h4>Nov 1, 2021</h4>
</center> <!-- date -->
<br>
<center>[1] <b>Department of Physics, University of Oslo</b></center>
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 5, 2020</h4></center> <!-- date -->
<br>
<p>
<p><a href="._week44-bs001.html" class="btn btn-primary btn-lg">Read &raquo;</a></p>
@@ -259,25 +310,18 @@ MathJax.Hub.Config({
<li><a href="._week44-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
<a href="https://..."><img width="250" align=right src="https://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2021, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
doc/pub/week44/html/week44-reveal.html
+865 -472
View File
File diff suppressed because it is too large Load Diff
doc/pub/week44/html/week44-solarized.html
+860 -469
View File
File diff suppressed because it is too large Load Diff
doc/pub/week44/html/week44.html
+935 -472
View File
File diff suppressed because it is too large Load Diff
doc/pub/week44/ipynb/ipynb-week44-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week44/ipynb/week44.ipynb
+656 -854
View File
File diff suppressed because one or more lines are too long
doc/src/week44/week44.do.txt
+2 -2
View File
@@ -6,8 +6,8 @@ DATE: today
!split
===== Overview of week 44 =====
* "Thursday: Wrapping up PCA from last week and basics of decision trees, classification and regression algorithms with video of lecture":"https://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/h20/forelesningsvideoer/LectureOctober29.mp4?vrtx=view-as-webpage"
* "Friday: Decision trees, voting models and bagging with video of lecture":"https://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/h20/forelesningsvideoer/LectureOctober30.mp4?vrtx=view-as-webpage"
* Thursday: Wrapping up PCA from last week and basics of decision trees, classification and regression algorithms
* Friday: Decision trees, voting models and bagging
Geron's chapter 6 covers decision trees while ensemble models, voting and bagging are discussed in chapter 7. See also lecture from "STK-IN4300, lecture 7":"https://www.uio.no/studier/emner/matnat/math/STK-IN4300/h20/slides/lecture_7.pdf". Chapter 9.2 of Hastie et al contains also a good discussion.