lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update week 46

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-11-12 18:58:36 +01:00
parent beb104cebf
commit 36decab909
46 changed files with 22068 additions and 9905 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week46/html/._week46-bs006.html
+238 -95
View File
@@ -8,8 +8,8 @@ doconce format html week46.do.txt --html_style=bootstrap --pygments_html_style=d
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/doconce/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 46: Support Vector Machines and Project 3.">
<title>Week 46: Support Vector Machines and Project 3.</title>
<meta name="description" content="Week 46: Decision Trees, Ensemble methods and Random Forests">
<title>Week 46: Decision Trees, Ensemble methods and Random Forests</title>
<!-- Bootstrap style: bootstrap -->
<!-- doconce format html week46.do.txt --html_style=bootstrap --pygments_html_style=default --html_admon=bootstrap_panel --html_output=week46-bs --no_mako -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -36,68 +36,190 @@ doconce format html week46.do.txt --html_style=bootstrap --pygments_html_style=d
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 46', 2, None, 'overview-of-week-46'),
('Support Vector Machines, overarching aims',
'sections': [('Plan for week 46', 2, None, 'plan-for-week-46'),
('Decision trees, overarching aims',
2,
None,
'support-vector-machines-overarching-aims'),
('Hyperplanes and all that', 2, None, 'hyperplanes-and-all-that'),
('What is a hyperplane?', 2, None, 'what-is-a-hyperplane'),
('A $p$-dimensional space of features',
'decision-trees-overarching-aims'),
('Basics of a tree', 2, None, 'basics-of-a-tree'),
('A Sketch of a Tree, Regression problem',
2,
None,
'a-p-dimensional-space-of-features'),
('The two-dimensional case', 2, None, 'the-two-dimensional-case'),
('Getting into the details', 2, None, 'getting-into-the-details'),
('First attempt at a minimization approach',
'a-sketch-of-a-tree-regression-problem'),
('A Sketch of a Tree, Classification problem',
2,
None,
'first-attempt-at-a-minimization-approach'),
('Solving the equations', 2, None, 'solving-the-equations'),
('Code Example', 2, None, 'code-example'),
('Problems with the Simpler Approach',
'a-sketch-of-a-tree-classification-problem'),
('A typical Decision Tree with its pertinent Jargon, '
'Classification Problem',
2,
None,
'problems-with-the-simpler-approach'),
('A better approach', 2, None, 'a-better-approach'),
('A quick Reminder on Lagrangian Multipliers',
'a-typical-decision-tree-with-its-pertinent-jargon-classification-problem'),
('General Features', 2, None, 'general-features'),
('How do we set it up?', 2, None, 'how-do-we-set-it-up'),
('Decision trees and Regression',
2,
None,
'a-quick-reminder-on-lagrangian-multipliers'),
('Adding the Multiplier', 2, None, 'adding-the-multiplier'),
('Setting up the Problem', 2, None, 'setting-up-the-problem'),
('The problem to solve', 2, None, 'the-problem-to-solve'),
('The last steps', 2, None, 'the-last-steps'),
('A soft classifier', 2, None, 'a-soft-classifier'),
('Soft optmization problem', 2, None, 'soft-optmization-problem'),
('Kernels and non-linearity',
'decision-trees-and-regression'),
('Building a tree, regression',
2,
None,
'kernels-and-non-linearity'),
('The equations', 2, None, 'the-equations'),
('The problem to solve', 2, None, 'the-problem-to-solve'),
("Different kernels and Mercer's theorem",
'building-a-tree-regression'),
('A top-down approach, recursive binary splitting',
2,
None,
'different-kernels-and-mercer-s-theorem'),
('The moons example', 2, None, 'the-moons-example'),
('Mathematical optimization of convex functions',
'a-top-down-approach-recursive-binary-splitting'),
('Making a tree', 2, None, 'making-a-tree'),
('Pruning the tree', 2, None, 'pruning-the-tree'),
('Cost complexity pruning', 2, None, 'cost-complexity-pruning'),
('Schematic Regression Procedure',
2,
None,
'mathematical-optimization-of-convex-functions'),
('How do we solve these problems?',
'schematic-regression-procedure'),
('A Classification Tree', 2, None, 'a-classification-tree'),
('Growing a classification tree',
2,
None,
'how-do-we-solve-these-problems'),
('A simple example', 2, None, 'a-simple-example'),
('Back to the more realistic cases',
'growing-a-classification-tree'),
('Classification tree, how to split nodes',
2,
None,
'back-to-the-more-realistic-cases'),
('Support vector machines for regression',
'classification-tree-how-to-split-nodes'),
('Visualizing the Tree, Classification',
2,
None,
'support-vector-machines-for-regression')]}
'visualizing-the-tree-classification'),
('Visualizing the Tree, The Moons',
2,
None,
'visualizing-the-tree-the-moons'),
('Other ways of visualizing the trees',
2,
None,
'other-ways-of-visualizing-the-trees'),
('Printing out as text', 2, None, 'printing-out-as-text'),
('Algorithms for Setting up Decision Trees',
2,
None,
'algorithms-for-setting-up-decision-trees'),
('The CART algorithm for Classification',
2,
None,
'the-cart-algorithm-for-classification'),
('The CART algorithm for Regression',
2,
None,
'the-cart-algorithm-for-regression'),
('Why binary splits?', 2, None, 'why-binary-splits'),
('Computing a Tree using the Gini Index',
2,
None,
'computing-a-tree-using-the-gini-index'),
('The Table', 2, None, 'the-table'),
('Computing the various Gini Indices',
2,
None,
'computing-the-various-gini-indices'),
('Computing the various Gini Indices, Hours slept',
2,
None,
'computing-the-various-gini-indices-hours-slept'),
('Computing the various Gini Indices, Hours studied',
2,
None,
'computing-the-various-gini-indices-hours-studied'),
('A possible code using Scikit-Learn',
2,
None,
'a-possible-code-using-scikit-learn'),
('Further example: Computing the Gini index',
2,
None,
'further-example-computing-the-gini-index'),
('Simple Python Code to read in Data and perform Classification',
2,
None,
'simple-python-code-to-read-in-data-and-perform-classification'),
('Computing the Gini Factor',
2,
None,
'computing-the-gini-factor'),
('Regression trees', 2, None, 'regression-trees'),
('Final regressor code', 2, None, 'final-regressor-code'),
('Pros and cons of trees, pros',
2,
None,
'pros-and-cons-of-trees-pros'),
('Disadvantages', 2, None, 'disadvantages'),
('Ensemble Methods: From a Single Tree to Many Trees and Extreme '
'Boosting, Meet the Jungle of Methods',
2,
None,
'ensemble-methods-from-a-single-tree-to-many-trees-and-extreme-boosting-meet-the-jungle-of-methods'),
('An Overview of Ensemble Methods',
2,
None,
'an-overview-of-ensemble-methods'),
('Why Voting?', 2, None, 'why-voting'),
('Tossing coins', 2, None, 'tossing-coins'),
('Standard imports first', 2, None, 'standard-imports-first'),
('Simple Voting Example, head or tail',
2,
None,
'simple-voting-example-head-or-tail'),
('Using the Voting Classifier',
2,
None,
'using-the-voting-classifier'),
('Voting and Bagging', 2, None, 'voting-and-bagging'),
('Bagging', 2, None, 'bagging'),
('More bagging', 2, None, 'more-bagging'),
('Making your own Bootstrap: Changing the Level of the Decision '
'Tree',
2,
None,
'making-your-own-bootstrap-changing-the-level-of-the-decision-tree'),
('Random forests', 2, None, 'random-forests'),
('Random Forest Algorithm', 2, None, 'random-forest-algorithm'),
('Random Forests Compared with other Methods on the Cancer Data',
2,
None,
'random-forests-compared-with-other-methods-on-the-cancer-data'),
('Compare Bagging on Trees with Random Forests',
2,
None,
'compare-bagging-on-trees-with-random-forests'),
("Boosting, a Bird's Eye View",
2,
None,
'boosting-a-bird-s-eye-view'),
('What is boosting? Additive Modelling/Iterative Fitting',
2,
None,
'what-is-boosting-additive-modelling-iterative-fitting'),
('Iterative Fitting, Regression and Squared-error Cost Function',
2,
None,
'iterative-fitting-regression-and-squared-error-cost-function'),
('Squared-Error Example and Iterative Fitting',
2,
None,
'squared-error-example-and-iterative-fitting'),
('Iterative Fitting, Classification and AdaBoost',
2,
None,
'iterative-fitting-classification-and-adaboost'),
('Adaptive Boosting, AdaBoost',
2,
None,
'adaptive-boosting-adaboost'),
('Building up AdaBoost', 2, None, 'building-up-adaboost'),
('Adaptive boosting: AdaBoost, Basic Algorithm',
2,
None,
'adaptive-boosting-adaboost-basic-algorithm'),
('Basic Steps of AdaBoost', 2, None, 'basic-steps-of-adaboost'),
('AdaBoost Examples', 2, None, 'adaboost-examples')]}
end of tocinfo -->
<body>
@@ -125,42 +247,77 @@ MathJax.Hub.Config({
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week46-bs.html">Week 46: Support Vector Machines and Project 3.</a>
<a class="navbar-brand" href="week46-bs.html">Week 46: Decision Trees, Ensemble methods and Random Forests</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week46-bs001.html#overview-of-week-46" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs002.html#support-vector-machines-overarching-aims" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs003.html#hyperplanes-and-all-that" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs004.html#what-is-a-hyperplane" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs005.html#a-p-dimensional-space-of-features" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="#the-two-dimensional-case" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs007.html#getting-into-the-details" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs008.html#first-attempt-at-a-minimization-approach" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs009.html#solving-the-equations" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs010.html#code-example" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs011.html#problems-with-the-simpler-approach" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs012.html#a-better-approach" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs013.html#a-quick-reminder-on-lagrangian-multipliers" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs014.html#adding-the-multiplier" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs015.html#setting-up-the-problem" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs017.html#the-last-steps" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs018.html#a-soft-classifier" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs019.html#soft-optmization-problem" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs020.html#kernels-and-non-linearity" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs021.html#the-equations" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs023.html#different-kernels-and-mercer-s-theorem" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#the-moons-example" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs025.html#mathematical-optimization-of-convex-functions" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs026.html#how-do-we-solve-these-problems" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs027.html#a-simple-example" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs028.html#back-to-the-more-realistic-cases" style="font-size: 80%;">Back to the more realistic cases</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs029.html#support-vector-machines-for-regression" style="font-size: 80%;">Support vector machines for regression</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs001.html#plan-for-week-46" style="font-size: 80%;">Plan for week 46</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs002.html#decision-trees-overarching-aims" style="font-size: 80%;">Decision trees, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs003.html#basics-of-a-tree" style="font-size: 80%;">Basics of a tree</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs004.html#a-sketch-of-a-tree-regression-problem" style="font-size: 80%;">A Sketch of a Tree, Regression problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs005.html#a-sketch-of-a-tree-classification-problem" style="font-size: 80%;">A Sketch of a Tree, Classification problem</a></li>
<!-- navigation toc: --> <li><a href="#a-typical-decision-tree-with-its-pertinent-jargon-classification-problem" style="font-size: 80%;">A typical Decision Tree with its pertinent Jargon, Classification Problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs007.html#general-features" style="font-size: 80%;">General Features</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs008.html#how-do-we-set-it-up" style="font-size: 80%;">How do we set it up?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs009.html#decision-trees-and-regression" style="font-size: 80%;">Decision trees and Regression</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs010.html#building-a-tree-regression" style="font-size: 80%;">Building a tree, regression</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs011.html#a-top-down-approach-recursive-binary-splitting" style="font-size: 80%;">A top-down approach, recursive binary splitting</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs012.html#making-a-tree" style="font-size: 80%;">Making a tree</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs013.html#pruning-the-tree" style="font-size: 80%;">Pruning the tree</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs014.html#cost-complexity-pruning" style="font-size: 80%;">Cost complexity pruning</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs015.html#schematic-regression-procedure" style="font-size: 80%;">Schematic Regression Procedure</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs016.html#a-classification-tree" style="font-size: 80%;">A Classification Tree</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs017.html#growing-a-classification-tree" style="font-size: 80%;">Growing a classification tree</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs018.html#classification-tree-how-to-split-nodes" style="font-size: 80%;">Classification tree, how to split nodes</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs019.html#visualizing-the-tree-classification" style="font-size: 80%;">Visualizing the Tree, Classification</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs020.html#visualizing-the-tree-the-moons" style="font-size: 80%;">Visualizing the Tree, The Moons</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs021.html#other-ways-of-visualizing-the-trees" style="font-size: 80%;">Other ways of visualizing the trees</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#printing-out-as-text" style="font-size: 80%;">Printing out as text</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs023.html#algorithms-for-setting-up-decision-trees" style="font-size: 80%;">Algorithms for Setting up Decision Trees</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#the-cart-algorithm-for-classification" style="font-size: 80%;">The CART algorithm for Classification</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs025.html#the-cart-algorithm-for-regression" style="font-size: 80%;">The CART algorithm for Regression</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs026.html#why-binary-splits" style="font-size: 80%;">Why binary splits?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs027.html#computing-a-tree-using-the-gini-index" style="font-size: 80%;">Computing a Tree using the Gini Index</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs028.html#the-table" style="font-size: 80%;">The Table</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs029.html#computing-the-various-gini-indices" style="font-size: 80%;">Computing the various Gini Indices</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs030.html#computing-the-various-gini-indices-hours-slept" style="font-size: 80%;">Computing the various Gini Indices, Hours slept</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs031.html#computing-the-various-gini-indices-hours-studied" style="font-size: 80%;">Computing the various Gini Indices, Hours studied</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs032.html#a-possible-code-using-scikit-learn" style="font-size: 80%;">A possible code using Scikit-Learn</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs033.html#further-example-computing-the-gini-index" style="font-size: 80%;">Further example: Computing the Gini index</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs034.html#simple-python-code-to-read-in-data-and-perform-classification" style="font-size: 80%;">Simple Python Code to read in Data and perform Classification</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs035.html#computing-the-gini-factor" style="font-size: 80%;">Computing the Gini Factor</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs036.html#regression-trees" style="font-size: 80%;">Regression trees</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs037.html#final-regressor-code" style="font-size: 80%;">Final regressor code</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs038.html#pros-and-cons-of-trees-pros" style="font-size: 80%;">Pros and cons of trees, pros</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs039.html#disadvantages" style="font-size: 80%;">Disadvantages</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs040.html#ensemble-methods-from-a-single-tree-to-many-trees-and-extreme-boosting-meet-the-jungle-of-methods" style="font-size: 80%;">Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs041.html#an-overview-of-ensemble-methods" style="font-size: 80%;">An Overview of Ensemble Methods</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs042.html#why-voting" style="font-size: 80%;">Why Voting?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs043.html#tossing-coins" style="font-size: 80%;">Tossing coins</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs044.html#standard-imports-first" style="font-size: 80%;">Standard imports first</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs045.html#simple-voting-example-head-or-tail" style="font-size: 80%;">Simple Voting Example, head or tail</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs046.html#using-the-voting-classifier" style="font-size: 80%;">Using the Voting Classifier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs047.html#voting-and-bagging" style="font-size: 80%;">Voting and Bagging</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs048.html#bagging" style="font-size: 80%;">Bagging</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs049.html#more-bagging" style="font-size: 80%;">More bagging</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs050.html#making-your-own-bootstrap-changing-the-level-of-the-decision-tree" style="font-size: 80%;">Making your own Bootstrap: Changing the Level of the Decision Tree</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs051.html#random-forests" style="font-size: 80%;">Random forests</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs052.html#random-forest-algorithm" style="font-size: 80%;">Random Forest Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs053.html#random-forests-compared-with-other-methods-on-the-cancer-data" style="font-size: 80%;">Random Forests Compared with other Methods on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs054.html#compare-bagging-on-trees-with-random-forests" style="font-size: 80%;">Compare Bagging on Trees with Random Forests</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs055.html#boosting-a-bird-s-eye-view" style="font-size: 80%;">Boosting, a Bird's Eye View</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs056.html#what-is-boosting-additive-modelling-iterative-fitting" style="font-size: 80%;">What is boosting? Additive Modelling/Iterative Fitting</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs057.html#iterative-fitting-regression-and-squared-error-cost-function" style="font-size: 80%;">Iterative Fitting, Regression and Squared-error Cost Function</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs058.html#squared-error-example-and-iterative-fitting" style="font-size: 80%;">Squared-Error Example and Iterative Fitting</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs059.html#iterative-fitting-classification-and-adaboost" style="font-size: 80%;">Iterative Fitting, Classification and AdaBoost</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs060.html#adaptive-boosting-adaboost" style="font-size: 80%;">Adaptive Boosting, AdaBoost</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs061.html#building-up-adaboost" style="font-size: 80%;">Building up AdaBoost</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs062.html#adaptive-boosting-adaboost-basic-algorithm" style="font-size: 80%;">Adaptive boosting: AdaBoost, Basic Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs063.html#basic-steps-of-adaboost" style="font-size: 80%;">Basic Steps of AdaBoost</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs064.html#adaboost-examples" style="font-size: 80%;">AdaBoost Examples</a></li>
</ul>
</li>
@@ -171,30 +328,16 @@ MathJax.Hub.Config({
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0006"></a>
<!-- !split -->
<h2 id="the-two-dimensional-case" class="anchor">The two-dimensional case </h2>
<!-- !split -->
<h2 id="a-typical-decision-tree-with-its-pertinent-jargon-classification-problem" class="anchor">A typical Decision Tree with its pertinent Jargon, Classification Problem </h2>
<p>Let us try to develop our intuition about SVMs by limiting ourselves to a two-dimensional
plane. To separate the two classes of data points, there are many
possible lines (hyperplanes if you prefer a more strict naming)
that could be chosen. Our objective is to find a
plane that has the maximum margin, i.e the maximum distance between
data points of both classes. Maximizing the margin distance provides
some reinforcement so that future data points can be classified with
more confidence.
</p>
<br/><br/>
<center>
<p><img src="DataFiles/cancer.png" width="600" align="bottom"></p>
</center>
<br/><br/>
<p>What a linear classifier attempts to accomplish is to split the
feature space into two half spaces by placing a hyperplane between the
data points. This hyperplane will be our decision boundary. All
points on one side of the plane will belong to class one and all points
on the other side of the plane will belong to the second class two.
</p>
<p>Unfortunately there are many ways in which we can place a hyperplane
to divide the data. Below is an example of two candidate hyperplanes
for our data sample.
</p>
<p>This tree was produced using the Wisconsin cancer data (discussed here as well, see code examples below) using <b>Scikit-Learn</b>'s decision tree classifier. Here we have used the so-called <b>gini</b> index (see below) to split the various branches.</p>
<p>
<!-- navigation buttons at the bottom of the page -->
@@ -217,7 +360,7 @@ for our data sample.
<li><a href="._week46-bs014.html">15</a></li>
<li><a href="._week46-bs015.html">16</a></li>
<li><a href="">...</a></li>
<li><a href="._week46-bs029.html">30</a></li>
<li><a href="._week46-bs064.html">65</a></li>
<li><a href="._week46-bs007.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->