Week 45: Decisions Trees, Random Forests, Bagging and Boosting
Contents
Overview of week 45
Decision trees, overarching aims
Basics of a tree
A Sketch of a Tree, Regression problem
A Sketch of a Tree, Classification problem
A typical Decision Tree with its pertinent Jargon, Classification Problem
General Features
How do we set it up?
Decision trees and Regression
Building a tree, regression
A top-down approach, recursive binary splitting
Making a tree
Pruning the tree
Cost complexity pruning
Schematic Regression Procedure
A Classification Tree
Growing a classification tree
Classification tree, how to split nodes
Visualizing the Tree, Classification
Visualizing the Tree, The Moons
Other ways of visualizing the trees
Printing out as text
Algorithms for Setting up Decision Trees
The CART algorithm for Classification
The CART algorithm for Regression
Computing the Gini index
Simple Python Code to read in Data and perform Classification
Computing the Gini Factor
Entropy and the ID3 algorithm
Cancer Data again now with Decision Trees and other Methods
Another example, the moons again
Playing around with regions
Regression trees
Final regressor code
Pros and cons of trees, pros
Disadvantages
Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods
An Overview of Ensemble Methods
Bagging
More bagging
Making your own Bootstrap: Changing the Level of the Decision Tree
Why Voting?
Tossing coins
Standard imports first
Simple Voting Example, head or tail
Using the Voting Classifier
Voting and Bagging
Random forests
Random Forest Algorithm
Random Forests Compared with other Methods on the Cancer Data
Compare Bagging on Trees with Random Forests
Boosting, a Bird's Eye View
What is boosting? Additive Modelling/Iterative Fitting
Iterative Fitting, Regression and Squared-error Cost Function
Squared-Error Example and Iterative Fitting
Iterative Fitting, Classification and AdaBoost
Adaptive Boosting, AdaBoost
Building up AdaBoost
Adaptive boosting: AdaBoost, Basic Algorithm
Basic Steps of AdaBoost
AdaBoost Examples
Gradient boosting: Basics with Steepest Descent/Functional Gradient Descent
The Squared-Error again! Steepest Descent
Steepest Descent Example
Gradient Boosting, algorithm
Gradient Boosting, Examples of Regression
Gradient Boosting, Classification Example
XGBoost: Extreme Gradient Boosting
Regression Case
Xgboost on the Cancer Data
An Overview of Ensemble Methods
«
1
...
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
...
71
»
