lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test

Browse Source
This commit is contained in:
mhjensen
2020-11-06 07:17:20 +01:00
parent 3c529be9eb
commit 79ff3386cf
40 changed files with 656 additions and 631 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week45/html/._week45-bs000.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -235,7 +233,7 @@ MathJax.Hub.Config({
<li><a href="._week45-bs008.html">9</a></li>
<li><a href="._week45-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs001.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -221,7 +219,7 @@ Geron's chapter 7. See also lecture from <a href="https://www.uio.no/studier/emn
<li><a href="._week45-bs009.html">10</a></li>
<li><a href="._week45-bs010.html">11</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs002.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs002.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -217,7 +215,7 @@ We repeat here the voting approach since this will serve as a motivation for boo
<li><a href="._week45-bs010.html">11</a></li>
<li><a href="._week45-bs011.html">12</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs003.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs003.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -231,7 +229,7 @@ Decision trees play an important role as our weak classifier. They serve as the
<li><a href="._week45-bs011.html">12</a></li>
<li><a href="._week45-bs012.html">13</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs004.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs004.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -239,7 +237,7 @@ numbers kicking in.
<li><a href="._week45-bs012.html">13</a></li>
<li><a href="._week45-bs013.html">14</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs005.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs005.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -256,7 +254,7 @@ DATA_ID <span style="color: #666666">=</span> <span style="color: #BA2121">&quot
<li><a href="._week45-bs013.html">14</a></li>
<li><a href="._week45-bs014.html">15</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs006.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs006.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -241,7 +239,7 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._week45-bs014.html">15</a></li>
<li><a href="._week45-bs015.html">16</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs007.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs007.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -265,7 +263,7 @@ voting_clf<span style="color: #666666">.</span>fit(X_train, y_train)
<li><a href="._week45-bs015.html">16</a></li>
<li><a href="._week45-bs016.html">17</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs008.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs008.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -272,7 +270,7 @@ voting_clf<span style="color: #666666">.</span>fit(X_train, y_train)
<li><a href="._week45-bs016.html">17</a></li>
<li><a href="._week45-bs017.html">18</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs009.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs009.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -258,7 +256,7 @@ this setting.
<li><a href="._week45-bs017.html">18</a></li>
<li><a href="._week45-bs018.html">19</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs010.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs010.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -241,7 +239,7 @@ We will grow of forest of say \( B \) trees.
<li><a href="._week45-bs018.html">19</a></li>
<li><a href="._week45-bs019.html">20</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs011.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs011.html
+21 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -263,6 +261,16 @@ plt<span style="color: #666666">.</span>show()
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_cumulative_gain(y_test, y_probas)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
Recall that the cumulative gains curve shows the percentage of the
overall number of cases in a given category <em>gained</em> by targeting a
percentage of the total number of cases.
<p>
Similarly, the receiver operating characteristic curve, or ROC curve,
displays the diagnostic ability of a binary classifier system as its
discrimination threshold is varied. It plots the true positive rate against the false positive rate.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
@@ -289,7 +297,7 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._week45-bs019.html">20</a></li>
<li><a href="._week45-bs020.html">21</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs012.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs012.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -237,7 +235,7 @@ np<span style="color: #666666">.</span>sum(y_pred <span style="color: #666666">=
<li><a href="._week45-bs020.html">21</a></li>
<li><a href="._week45-bs021.html">22</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs013.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs013.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -232,7 +230,7 @@ them with a factor.
<li><a href="._week45-bs021.html">22</a></li>
<li><a href="._week45-bs022.html">23</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs014.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs014.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -267,7 +265,7 @@ In iterative fitting or additive modeling, we minimize the cost function with re
<li><a href="._week45-bs022.html">23</a></li>
<li><a href="._week45-bs023.html">24</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs015.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs015.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -240,7 +238,7 @@ at the internal nodes, and the predictions at the terminal nodes.
<li><a href="._week45-bs023.html">24</a></li>
<li><a href="._week45-bs024.html">25</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs016.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs016.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -263,7 +261,7 @@ The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma
<li><a href="._week45-bs024.html">25</a></li>
<li><a href="._week45-bs025.html">26</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs017.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs017.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -250,7 +248,7 @@ $$
<li><a href="._week45-bs025.html">26</a></li>
<li><a href="._week45-bs026.html">27</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs018.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs018.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -243,7 +241,7 @@ where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).
<li><a href="._week45-bs026.html">27</a></li>
<li><a href="._week45-bs027.html">28</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs019.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs019.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -259,7 +257,7 @@ $$
<li><a href="._week45-bs027.html">28</a></li>
<li><a href="._week45-bs028.html">29</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs020.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs020.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -237,7 +235,7 @@ where the function \( I() \) is one if we misclassify and zero if we classify co
<li><a href="._week45-bs028.html">29</a></li>
<li><a href="._week45-bs029.html">30</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs021.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs021.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -255,7 +253,7 @@ observations that are missed in the previous iterations.
<li><a href="._week45-bs029.html">30</a></li>
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs022.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs022.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -248,7 +246,7 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs023.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs023.html
+10 -14
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -242,8 +240,6 @@ Start by selecting a set of training data \( n \) and assign to each entry a wei
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs024.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs024.html
+10 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -228,7 +226,6 @@ function was the least squares function.
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs025.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs025.html
+10 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -247,7 +245,6 @@ $$
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs026.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs026.html
+10 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -229,7 +227,6 @@ and find a new value for \( \rho_2=-1/2 \) and continue till we have reached \(
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs027.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs027.html
+15 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -194,6 +192,11 @@ MathJax.Hub.Config({
<h2 id="___sec26" class="anchor">Gradient Boosting, algorithm </h2>
<p>
Steepest descent is however not much used, since it only optimizes \( f \) at a fixed set of \( n \) points,
so we do not learn a function that can generalize. However, we can modify the algorithm by
fitting a weak learner to approximate the negative gradient signal.
<p>
Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function
$$
@@ -236,7 +239,6 @@ The way we proceed in an iterative fashion is to
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs028.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs028.html
+59 -16
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -192,11 +190,57 @@ MathJax.Hub.Config({
<a name="part0028"></a>
<!-- !split -->
<h2 id="___sec27" class="anchor">Gradient Boosting Example, Regression </h2>
<h2 id="___sec27" class="anchor">Gradient Boosting, Examples of Regression </h2>
<p>
We discuss here the difference between the steepest descent approach and gradient boosting by repeating our simple regression example above.
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.ensemble</span> <span style="color: #008000; font-weight: bold">import</span> GradientBoostingRegressor
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.metrics</span> <span style="color: #008000; font-weight: bold">import</span> mean_squared_error
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
maxdegree <span style="color: #666666">=</span> <span style="color: #666666">6</span>
<span style="color: #408080; font-style: italic"># Make data set.</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">-3</span>, <span style="color: #666666">3</span>, n)<span style="color: #666666">.</span>reshape(<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>)
y <span style="color: #666666">=</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>x<span style="color: #666666">**2</span>) <span style="color: #666666">+</span> <span style="color: #666666">1.5</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>(x<span style="color: #666666">-2</span>)<span style="color: #666666">**2</span>)<span style="color: #666666">+</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>normal(<span style="color: #666666">0</span>, <span style="color: #666666">0.1</span>, x<span style="color: #666666">.</span>shape)
error <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
bias <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
variance <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
polydegree <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(x, y, test_size<span style="color: #666666">=0.2</span>)
scaler <span style="color: #666666">=</span> StandardScaler()
scaler<span style="color: #666666">.</span>fit(X_train)
X_train_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_train)
X_test_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_test)
<span style="color: #008000; font-weight: bold">for</span> degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">1</span>,maxdegree):
model <span style="color: #666666">=</span> GradientBoostingRegressor(max_depth<span style="color: #666666">=</span>degree, n_estimators<span style="color: #666666">=100</span>, learning_rate<span style="color: #666666">=1.0</span>)
model<span style="color: #666666">.</span>fit(X_train_scaled,y_train)
y_pred <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_test_scaled)
polydegree[degree] <span style="color: #666666">=</span> degree
error[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>mean((y_test <span style="color: #666666">-</span> y_pred)<span style="color: #666666">**2</span>) )
bias[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( (y_test <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(y_pred))<span style="color: #666666">**2</span> )
variance[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>var(y_pred) )
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Max depth:&#39;</span>, degree)
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Error:&#39;</span>, error[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Bias^2:&#39;</span>, bias[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Var:&#39;</span>, variance[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;</span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> &gt;= </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> + </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> = </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121">&#39;</span><span style="color: #666666">.</span>format(error[degree], bias[degree], variance[degree], bias[degree]<span style="color: #666666">+</span>variance[degree]))
plt<span style="color: #666666">.</span>xlim(<span style="color: #666666">1</span>,maxdegree<span style="color: #666666">-1</span>)
plt<span style="color: #666666">.</span>plot(polydegree, error, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Error&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, bias, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;bias&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, variance, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Variance&#39;</span>)
plt<span style="color: #666666">.</span>legend()
save_fig(<span style="color: #BA2121">&quot;gdregression&quot;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
@@ -217,7 +261,6 @@ We discuss here the difference between the steepest descent approach and gradien
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs029.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs029.html
+39 -48
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -192,55 +190,49 @@ MathJax.Hub.Config({
<a name="part0029"></a>
<!-- !split -->
<h2 id="___sec28" class="anchor">Gradient Boosting, Examples of Regression </h2>
<h2 id="___sec28" class="anchor">Gradient Boosting, Classification Example </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.ensemble</span> <span style="color: #008000; font-weight: bold">import</span> GradientBoostingRegressor
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.datasets</span> <span style="color: #008000; font-weight: bold">import</span> load_breast_cancer
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.metrics</span> <span style="color: #008000; font-weight: bold">import</span> mean_squared_error
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.ensemble</span> <span style="color: #008000; font-weight: bold">import</span> GradientBoostingClassifier
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> cross_validate
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
maxdegree <span style="color: #666666">=</span> <span style="color: #666666">6</span>
<span style="color: #408080; font-style: italic"># Load the data</span>
cancer <span style="color: #666666">=</span> load_breast_cancer()
<span style="color: #408080; font-style: italic"># Make data set.</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">-3</span>, <span style="color: #666666">3</span>, n)<span style="color: #666666">.</span>reshape(<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>)
y <span style="color: #666666">=</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>x<span style="color: #666666">**2</span>) <span style="color: #666666">+</span> <span style="color: #666666">1.5</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>(x<span style="color: #666666">-2</span>)<span style="color: #666666">**2</span>)<span style="color: #666666">+</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>normal(<span style="color: #666666">0</span>, <span style="color: #666666">0.1</span>, x<span style="color: #666666">.</span>shape)
error <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
bias <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
variance <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
polydegree <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(x, y, test_size<span style="color: #666666">=0.2</span>)
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(cancer<span style="color: #666666">.</span>data,cancer<span style="color: #666666">.</span>target,random_state<span style="color: #666666">=0</span>)
<span style="color: #008000">print</span>(X_train<span style="color: #666666">.</span>shape)
<span style="color: #008000">print</span>(X_test<span style="color: #666666">.</span>shape)
<span style="color: #408080; font-style: italic">#now scale the data</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
scaler <span style="color: #666666">=</span> StandardScaler()
scaler<span style="color: #666666">.</span>fit(X_train)
X_train_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_train)
X_test_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_test)
<span style="color: #008000; font-weight: bold">for</span> degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">1</span>,maxdegree):
model <span style="color: #666666">=</span> GradientBoostingRegressor(max_depth<span style="color: #666666">=</span>degree, n_estimators<span style="color: #666666">=100</span>, learning_rate<span style="color: #666666">=1.0</span>)
model<span style="color: #666666">.</span>fit(X_train_scaled,y_train)
y_pred <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_test_scaled)
polydegree[degree] <span style="color: #666666">=</span> degree
error[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>mean((y_test <span style="color: #666666">-</span> y_pred)<span style="color: #666666">**2</span>) )
bias[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( (y_test <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(y_pred))<span style="color: #666666">**2</span> )
variance[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>var(y_pred) )
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Max depth:&#39;</span>, degree)
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Error:&#39;</span>, error[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Bias^2:&#39;</span>, bias[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Var:&#39;</span>, variance[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;</span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> &gt;= </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> + </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> = </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121">&#39;</span><span style="color: #666666">.</span>format(error[degree], bias[degree], variance[degree], bias[degree]<span style="color: #666666">+</span>variance[degree]))
gd_clf <span style="color: #666666">=</span> GradientBoostingClassifier(max_depth<span style="color: #666666">=3</span>, n_estimators<span style="color: #666666">=100</span>, learning_rate<span style="color: #666666">=1.0</span>)
gd_clf<span style="color: #666666">.</span>fit(X_train_scaled, y_train)
<span style="color: #408080; font-style: italic">#Cross validation</span>
accuracy <span style="color: #666666">=</span> cross_validate(gd_clf,X_test_scaled,y_test,cv<span style="color: #666666">=10</span>)[<span style="color: #BA2121">&#39;test_score&#39;</span>]
<span style="color: #008000">print</span>(accuracy)
<span style="color: #008000">print</span>(<span style="color: #BA2121">&quot;Test set accuracy with Random Forests and scaled data: </span><span style="color: #BB6688; font-weight: bold">{:.2f}</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">.</span>format(gd_clf<span style="color: #666666">.</span>score(X_test_scaled,y_test)))
plt<span style="color: #666666">.</span>xlim(<span style="color: #666666">1</span>,maxdegree<span style="color: #666666">-1</span>)
plt<span style="color: #666666">.</span>plot(polydegree, error, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Error&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, bias, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;bias&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, variance, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Variance&#39;</span>)
plt<span style="color: #666666">.</span>legend()
save_fig(<span style="color: #BA2121">&quot;gdregression&quot;</span>)
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
y_pred <span style="color: #666666">=</span> gd_clf<span style="color: #666666">.</span>predict(X_test_scaled)
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_confusion_matrix(y_test, y_pred, normalize<span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">True</span>)
save_fig(<span style="color: #BA2121">&quot;gdclassiffierconfusion&quot;</span>)
plt<span style="color: #666666">.</span>show()
y_probas <span style="color: #666666">=</span> gd_clf<span style="color: #666666">.</span>predict_proba(X_test_scaled)
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_roc(y_test, y_probas)
save_fig(<span style="color: #BA2121">&quot;gdclassiffierroc&quot;</span>)
plt<span style="color: #666666">.</span>show()
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_cumulative_gain(y_test, y_probas)
save_fig(<span style="color: #BA2121">&quot;gdclassiffiercgain&quot;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
@@ -262,7 +254,6 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs030.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs030.html
+24 -54
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -192,51 +190,24 @@ MathJax.Hub.Config({
<a name="part0030"></a>
<!-- !split -->
<h2 id="___sec29" class="anchor">Gradient Boosting, Classification Example </h2>
<h2 id="___sec29" class="anchor">XGBoost: Extreme Gradient Boosting </h2>
<p>
<a href="https://github.com/dmlc/xgboost" target="_self">XGBoost</a> or Extreme Gradient
Boosting, is an optimized distributed gradient boosting library
designed to be highly efficient, flexible and portable. It implements
machine learning algorithms under the Gradient Boosting
framework. XGBoost provides a parallel tree boosting that solve many
data science problems in a fast and accurate way. See the <a href="https://arxiv.org/abs/1603.02754" target="_self">article by Chen and Guestrin</a>.
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.datasets</span> <span style="color: #008000; font-weight: bold">import</span> load_breast_cancer
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.ensemble</span> <span style="color: #008000; font-weight: bold">import</span> GradientBoostingClassifier
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> cross_validate
<p>
The authors design and build a highly scalable end-to-end tree
boosting system. It has a theoretically justified weighted quantile
sketch for efficient proposal calculation. It introduces a novel sparsity-aware algorithm for parallel tree learning and an effective cache-aware block structure for out-of-core tree learning.
<span style="color: #408080; font-style: italic"># Load the data</span>
cancer <span style="color: #666666">=</span> load_breast_cancer()
<p>
It is now the algorithm which wins essentially all ML competitions!!!
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(cancer<span style="color: #666666">.</span>data,cancer<span style="color: #666666">.</span>target,random_state<span style="color: #666666">=0</span>)
<span style="color: #008000">print</span>(X_train<span style="color: #666666">.</span>shape)
<span style="color: #008000">print</span>(X_test<span style="color: #666666">.</span>shape)
<span style="color: #408080; font-style: italic">#now scale the data</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
scaler <span style="color: #666666">=</span> StandardScaler()
scaler<span style="color: #666666">.</span>fit(X_train)
X_train_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_train)
X_test_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_test)
gd_clf <span style="color: #666666">=</span> GradientBoostingClassifier(max_depth<span style="color: #666666">=3</span>, n_estimators<span style="color: #666666">=100</span>, learning_rate<span style="color: #666666">=1.0</span>)
gd_clf<span style="color: #666666">.</span>fit(X_train_scaled, y_train)
<span style="color: #408080; font-style: italic">#Cross validation</span>
accuracy <span style="color: #666666">=</span> cross_validate(gd_clf,X_test_scaled,y_test,cv<span style="color: #666666">=10</span>)[<span style="color: #BA2121">&#39;test_score&#39;</span>]
<span style="color: #008000">print</span>(accuracy)
<span style="color: #008000">print</span>(<span style="color: #BA2121">&quot;Test set accuracy with Random Forests and scaled data: </span><span style="color: #BB6688; font-weight: bold">{:.2f}</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">.</span>format(gd_clf<span style="color: #666666">.</span>score(X_test_scaled,y_test)))
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
y_pred <span style="color: #666666">=</span> gd_clf<span style="color: #666666">.</span>predict(X_test_scaled)
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_confusion_matrix(y_test, y_pred, normalize<span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">True</span>)
save_fig(<span style="color: #BA2121">&quot;gdclassiffierconfusion&quot;</span>)
plt<span style="color: #666666">.</span>show()
y_probas <span style="color: #666666">=</span> gd_clf<span style="color: #666666">.</span>predict_proba(X_test_scaled)
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_roc(y_test, y_probas)
save_fig(<span style="color: #BA2121">&quot;gdclassiffierroc&quot;</span>)
plt<span style="color: #666666">.</span>show()
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_cumulative_gain(y_test, y_probas)
save_fig(<span style="color: #BA2121">&quot;gdclassiffiercgain&quot;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
@@ -255,7 +226,6 @@ plt<span style="color: #666666">.</span>show()
<li class="active"><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs031.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs031.html
+57 -26
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -192,24 +190,58 @@ MathJax.Hub.Config({
<a name="part0031"></a>
<!-- !split -->
<h2 id="___sec30" class="anchor">XGBoost: Extreme Gradient Boosting </h2>
<h2 id="___sec30" class="anchor">Regression Case </h2>
<p>
<a href="https://github.com/dmlc/xgboost" target="_self">XGBoost</a> or Extreme Gradient
Boosting, is an optimized distributed gradient boosting library
designed to be highly efficient, flexible and portable. It implements
machine learning algorithms under the Gradient Boosting
framework. XGBoost provides a parallel tree boosting that solve many
data science problems in a fast and accurate way. See the <a href="https://arxiv.org/abs/1603.02754" target="_self">article by Chen and Guestrin</a>.
<p>
The authors design and build a highly scalable end-to-end tree
boosting system. It has a theoretically justified weighted quantile
sketch for efficient proposal calculation. It introduces a novel sparsity-aware algorithm for parallel tree learning and an effective cache-aware block structure for out-of-core tree learning.
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">xgboost</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">xgb</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.metrics</span> <span style="color: #008000; font-weight: bold">import</span> mean_squared_error
<p>
It is now the algorithm which wins essentially all ML competitions!!!
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
maxdegree <span style="color: #666666">=</span> <span style="color: #666666">6</span>
<span style="color: #408080; font-style: italic"># Make data set.</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">-3</span>, <span style="color: #666666">3</span>, n)<span style="color: #666666">.</span>reshape(<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>)
y <span style="color: #666666">=</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>x<span style="color: #666666">**2</span>) <span style="color: #666666">+</span> <span style="color: #666666">1.5</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>(x<span style="color: #666666">-2</span>)<span style="color: #666666">**2</span>)<span style="color: #666666">+</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>normal(<span style="color: #666666">0</span>, <span style="color: #666666">0.1</span>, x<span style="color: #666666">.</span>shape)
error <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
bias <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
variance <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
polydegree <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(x, y, test_size<span style="color: #666666">=0.2</span>)
scaler <span style="color: #666666">=</span> StandardScaler()
scaler<span style="color: #666666">.</span>fit(X_train)
X_train_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_train)
X_test_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_test)
<span style="color: #008000; font-weight: bold">for</span> degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(maxdegree):
model <span style="color: #666666">=</span> xgb<span style="color: #666666">.</span>XGBRegressor(objective <span style="color: #666666">=</span><span style="color: #BA2121">&#39;reg:squarederror&#39;</span>, colsaobjective <span style="color: #666666">=</span><span style="color: #BA2121">&#39;reg:squarederror&#39;</span>, colsample_bytree <span style="color: #666666">=</span> <span style="color: #666666">0.3</span>, learning_rate <span style="color: #666666">=</span> <span style="color: #666666">0.1</span>,max_depth <span style="color: #666666">=</span> degree, alpha <span style="color: #666666">=</span> <span style="color: #666666">10</span>, n_estimators <span style="color: #666666">=</span> <span style="color: #666666">200</span>)
model<span style="color: #666666">.</span>fit(X_train_scaled,y_train)
y_pred <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_test_scaled)
polydegree[degree] <span style="color: #666666">=</span> degree
error[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>mean((y_test <span style="color: #666666">-</span> y_pred)<span style="color: #666666">**2</span>) )
bias[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( (y_test <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(y_pred))<span style="color: #666666">**2</span> )
variance[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>var(y_pred) )
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Max depth:&#39;</span>, degree)
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Error:&#39;</span>, error[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Bias^2:&#39;</span>, bias[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Var:&#39;</span>, variance[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;</span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> &gt;= </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> + </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> = </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121">&#39;</span><span style="color: #666666">.</span>format(error[degree], bias[degree], variance[degree], bias[degree]<span style="color: #666666">+</span>variance[degree]))
plt<span style="color: #666666">.</span>xlim(<span style="color: #666666">1</span>,maxdegree<span style="color: #666666">-1</span>)
plt<span style="color: #666666">.</span>plot(polydegree, error, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Error&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, bias, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;bias&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, variance, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Variance&#39;</span>)
plt<span style="color: #666666">.</span>legend()
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
@@ -227,7 +259,6 @@ It is now the algorithm which wins essentially all ML competitions!!!
<li><a href="._week45-bs030.html">31</a></li>
<li class="active"><a href="._week45-bs031.html">32</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/._week45-bs032.html
+53 -49
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -192,59 +190,67 @@ MathJax.Hub.Config({
<a name="part0032"></a>
<!-- !split -->
<h2 id="___sec31" class="anchor">Regression Case </h2>
<h2 id="___sec31" class="anchor">Xgboost on the Cancer Data </h2>
<p>
As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">xgboost</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">xgb</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.datasets</span> <span style="color: #008000; font-weight: bold">import</span> load_breast_cancer
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> LabelEncoder
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> cross_validate
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.metrics</span> <span style="color: #008000; font-weight: bold">import</span> mean_squared_error
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">xgboost</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">xgb</span>
<span style="color: #408080; font-style: italic"># Load the data</span>
cancer <span style="color: #666666">=</span> load_breast_cancer()
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
maxdegree <span style="color: #666666">=</span> <span style="color: #666666">6</span>
<span style="color: #408080; font-style: italic"># Make data set.</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linspace(<span style="color: #666666">-3</span>, <span style="color: #666666">3</span>, n)<span style="color: #666666">.</span>reshape(<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>)
y <span style="color: #666666">=</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>x<span style="color: #666666">**2</span>) <span style="color: #666666">+</span> <span style="color: #666666">1.5</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>exp(<span style="color: #666666">-</span>(x<span style="color: #666666">-2</span>)<span style="color: #666666">**2</span>)<span style="color: #666666">+</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>normal(<span style="color: #666666">0</span>, <span style="color: #666666">0.1</span>, x<span style="color: #666666">.</span>shape)
error <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
bias <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
variance <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
polydegree <span style="color: #666666">=</span> np<span style="color: #666666">.</span>zeros(maxdegree)
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(x, y, test_size<span style="color: #666666">=0.2</span>)
X_train, X_test, y_train, y_test <span style="color: #666666">=</span> train_test_split(cancer<span style="color: #666666">.</span>data,cancer<span style="color: #666666">.</span>target,random_state<span style="color: #666666">=0</span>)
<span style="color: #008000">print</span>(X_train<span style="color: #666666">.</span>shape)
<span style="color: #008000">print</span>(X_test<span style="color: #666666">.</span>shape)
<span style="color: #408080; font-style: italic">#now scale the data</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
scaler <span style="color: #666666">=</span> StandardScaler()
scaler<span style="color: #666666">.</span>fit(X_train)
X_train_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_train)
X_test_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>transform(X_test)
<span style="color: #008000; font-weight: bold">for</span> degree <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(maxdegree):
model <span style="color: #666666">=</span> xgb<span style="color: #666666">.</span>XGBRegressor(objective <span style="color: #666666">=</span><span style="color: #BA2121">&#39;reg:squarederror&#39;</span>, colsaobjective <span style="color: #666666">=</span><span style="color: #BA2121">&#39;reg:squarederror&#39;</span>, colsample_bytree <span style="color: #666666">=</span> <span style="color: #666666">0.3</span>, learning_rate <span style="color: #666666">=</span> <span style="color: #666666">0.1</span>,max_depth <span style="color: #666666">=</span> degree, alpha <span style="color: #666666">=</span> <span style="color: #666666">10</span>, n_estimators <span style="color: #666666">=</span> <span style="color: #666666">200</span>)
xg_clf <span style="color: #666666">=</span> xgb<span style="color: #666666">.</span>XGBClassifier()
xg_clf<span style="color: #666666">.</span>fit(X_train_scaled,y_train)
model<span style="color: #666666">.</span>fit(X_train_scaled,y_train)
y_pred <span style="color: #666666">=</span> model<span style="color: #666666">.</span>predict(X_test_scaled)
polydegree[degree] <span style="color: #666666">=</span> degree
error[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>mean((y_test <span style="color: #666666">-</span> y_pred)<span style="color: #666666">**2</span>) )
bias[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( (y_test <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(y_pred))<span style="color: #666666">**2</span> )
variance[degree] <span style="color: #666666">=</span> np<span style="color: #666666">.</span>mean( np<span style="color: #666666">.</span>var(y_pred) )
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Max depth:&#39;</span>, degree)
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Error:&#39;</span>, error[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Bias^2:&#39;</span>, bias[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;Var:&#39;</span>, variance[degree])
<span style="color: #008000">print</span>(<span style="color: #BA2121">&#39;</span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> &gt;= </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> + </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121"> = </span><span style="color: #BB6688; font-weight: bold">{}</span><span style="color: #BA2121">&#39;</span><span style="color: #666666">.</span>format(error[degree], bias[degree], variance[degree], bias[degree]<span style="color: #666666">+</span>variance[degree]))
y_test <span style="color: #666666">=</span> xg_clf<span style="color: #666666">.</span>predict(X_test_scaled)
plt<span style="color: #666666">.</span>xlim(<span style="color: #666666">1</span>,maxdegree<span style="color: #666666">-1</span>)
plt<span style="color: #666666">.</span>plot(polydegree, error, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Error&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, bias, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;bias&#39;</span>)
plt<span style="color: #666666">.</span>plot(polydegree, variance, label<span style="color: #666666">=</span><span style="color: #BA2121">&#39;Variance&#39;</span>)
plt<span style="color: #666666">.</span>legend()
<span style="color: #008000">print</span>(<span style="color: #BA2121">&quot;Test set accuracy with Random Forests and scaled data: </span><span style="color: #BB6688; font-weight: bold">{:.2f}</span><span style="color: #BA2121">&quot;</span><span style="color: #666666">.</span>format(xg_clf<span style="color: #666666">.</span>score(X_test_scaled,y_test)))
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scikitplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">skplt</span>
y_pred <span style="color: #666666">=</span> xg_clf<span style="color: #666666">.</span>predict(X_test_scaled)
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_confusion_matrix(y_test, y_pred, normalize<span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">True</span>)
save_fig(<span style="color: #BA2121">&quot;xdclassiffierconfusion&quot;</span>)
plt<span style="color: #666666">.</span>show()
y_probas <span style="color: #666666">=</span> xg_clf<span style="color: #666666">.</span>predict_proba(X_test_scaled)
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_roc(y_test, y_probas)
save_fig(<span style="color: #BA2121">&quot;xdclassiffierroc&quot;</span>)
plt<span style="color: #666666">.</span>show()
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_cumulative_gain(y_test, y_probas)
save_fig(<span style="color: #BA2121">&quot;gdclassiffiercgain&quot;</span>)
plt<span style="color: #666666">.</span>show()
xgb<span style="color: #666666">.</span>plot_tree(xg_clf,num_trees<span style="color: #666666">=0</span>)
plt<span style="color: #666666">.</span>rcParams[<span style="color: #BA2121">&#39;figure.figsize&#39;</span>] <span style="color: #666666">=</span> [<span style="color: #666666">50</span>, <span style="color: #666666">10</span>]
save_fig(<span style="color: #BA2121">&quot;xgtree&quot;</span>)
plt<span style="color: #666666">.</span>show()
xgb<span style="color: #666666">.</span>plot_importance(xg_clf)
plt<span style="color: #666666">.</span>rcParams[<span style="color: #BA2121">&#39;figure.figsize&#39;</span>] <span style="color: #666666">=</span> [<span style="color: #666666">5</span>, <span style="color: #666666">5</span>]
save_fig(<span style="color: #BA2121">&quot;xgparams&quot;</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
@@ -260,8 +266,6 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._week45-bs030.html">31</a></li>
<li><a href="._week45-bs031.html">32</a></li>
<li class="active"><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs033.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/week45-bs.html
+11 -13
View File
@@ -95,18 +95,17 @@ Automatically generated HTML file from DocOnce source
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -171,12 +170,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week45-bs025.html#___sec24" style="font-size: 80%;">The Squared-Error again! Steepest Descent</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs026.html#___sec25" style="font-size: 80%;">Steepest Descent Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs027.html#___sec26" style="font-size: 80%;">Gradient Boosting, algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting Example, Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs033.html#___sec32" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs028.html#___sec27" style="font-size: 80%;">Gradient Boosting, Examples of Regression</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs029.html#___sec28" style="font-size: 80%;">Gradient Boosting, Classification Example</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs030.html#___sec29" style="font-size: 80%;">XGBoost: Extreme Gradient Boosting</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs031.html#___sec30" style="font-size: 80%;">Regression Case</a></li>
<!-- navigation toc: --> <li><a href="._week45-bs032.html#___sec31" style="font-size: 80%;">Xgboost on the Cancer Data</a></li>
</ul>
</li>
@@ -235,7 +233,7 @@ MathJax.Hub.Config({
<li><a href="._week45-bs008.html">9</a></li>
<li><a href="._week45-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week45-bs033.html">34</a></li>
<li><a href="._week45-bs032.html">33</a></li>
<li><a href="._week45-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week45/html/week45-reveal.html
+19 -13
View File
@@ -578,6 +578,15 @@ plt.show()
skplt.metrics.plot_cumulative_gain(y_test, y_probas)
plt.show()
</pre></div>
<p>
Recall that the cumulative gains curve shows the percentage of the
overall number of cases in a given category <em>gained</em> by targeting a
percentage of the total number of cases.
<p>
Similarly, the receiver operating characteristic curve, or ROC curve,
displays the diagnostic ability of a binary classifier system as its
discrimination threshold is varied. It plots the true positive rate against the false positive rate.
</section>
@@ -1109,6 +1118,11 @@ and find a new value for \( \rho_2=-1/2 \) and continue till we have reached \(
<section>
<h2 id="___sec26">Gradient Boosting, algorithm </h2>
<p>
Steepest descent is however not much used, since it only optimizes \( f \) at a fixed set of \( n \) points,
so we do not learn a function that can generalize. However, we can modify the algorithm by
fitting a weak learner to approximate the negative gradient signal.
<p>
Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function
<p>&nbsp;<br>
@@ -1135,15 +1149,7 @@ The way we proceed in an iterative fashion is to
<section>
<h2 id="___sec27">Gradient Boosting Example, Regression </h2>
<p>
We discuss here the difference between the steepest descent approach and gradient boosting by repeating our simple regression example above.
</section>
<section>
<h2 id="___sec28">Gradient Boosting, Examples of Regression </h2>
<h2 id="___sec27">Gradient Boosting, Examples of Regression </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1198,7 +1204,7 @@ plt.show()
<section>
<h2 id="___sec29">Gradient Boosting, Classification Example </h2>
<h2 id="___sec28">Gradient Boosting, Classification Example </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1247,7 +1253,7 @@ plt.show()
<section>
<h2 id="___sec30">XGBoost: Extreme Gradient Boosting </h2>
<h2 id="___sec29">XGBoost: Extreme Gradient Boosting </h2>
<p>
<a href="https://github.com/dmlc/xgboost" target="_blank">XGBoost</a> or Extreme Gradient
@@ -1268,7 +1274,7 @@ It is now the algorithm which wins essentially all ML competitions!!!
<section>
<h2 id="___sec31">Regression Case </h2>
<h2 id="___sec30">Regression Case </h2>
<p>
@@ -1324,7 +1330,7 @@ plt.show()
<section>
<h2 id="___sec32">Xgboost on the Cancer Data </h2>
<h2 id="___sec31">Xgboost on the Cancer Data </h2>
<p>
As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now.
doc/pub/week45/html/week45-solarized.html
+25 -19
View File
@@ -89,18 +89,17 @@ div { text-align: justify; text-justify: inter-word; }
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -554,6 +553,16 @@ plt.show()
skplt.metrics.plot_cumulative_gain(y_test, y_probas)
plt.show()
</pre></div>
<p>
Recall that the cumulative gains curve shows the percentage of the
overall number of cases in a given category <em>gained</em> by targeting a
percentage of the total number of cases.
<p>
Similarly, the receiver operating characteristic curve, or ROC curve,
displays the diagnostic ability of a binary classifier system as its
discrimination threshold is varied. It plots the true positive rate against the false positive rate.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1021,6 +1030,11 @@ and find a new value for \( \rho_2=-1/2 \) and continue till we have reached \(
<h2 id="___sec26">Gradient Boosting, algorithm </h2>
<p>
Steepest descent is however not much used, since it only optimizes \( f \) at a fixed set of \( n \) points,
so we do not learn a function that can generalize. However, we can modify the algorithm by
fitting a weak learner to approximate the negative gradient signal.
<p>
Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function
$$
@@ -1045,15 +1059,7 @@ The way we proceed in an iterative fashion is to
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec27">Gradient Boosting Example, Regression </h2>
<p>
We discuss here the difference between the steepest descent approach and gradient boosting by repeating our simple regression example above.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec28">Gradient Boosting, Examples of Regression </h2>
<h2 id="___sec27">Gradient Boosting, Examples of Regression </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1107,7 +1113,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec29">Gradient Boosting, Classification Example </h2>
<h2 id="___sec28">Gradient Boosting, Classification Example </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1155,7 +1161,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec30">XGBoost: Extreme Gradient Boosting </h2>
<h2 id="___sec29">XGBoost: Extreme Gradient Boosting </h2>
<p>
<a href="https://github.com/dmlc/xgboost" target="_blank">XGBoost</a> or Extreme Gradient
@@ -1176,7 +1182,7 @@ It is now the algorithm which wins essentially all ML competitions!!!
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec31">Regression Case </h2>
<h2 id="___sec30">Regression Case </h2>
<p>
@@ -1231,7 +1237,7 @@ plt.show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec32">Xgboost on the Cancer Data </h2>
<h2 id="___sec31">Xgboost on the Cancer Data </h2>
<p>
As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now.
doc/pub/week45/html/week45.html
+25 -19
View File
@@ -94,18 +94,17 @@ div { text-align: justify; text-justify: inter-word; }
'___sec24'),
('Steepest Descent Example', 2, None, '___sec25'),
('Gradient Boosting, algorithm', 2, None, '___sec26'),
('Gradient Boosting Example, Regression', 2, None, '___sec27'),
('Gradient Boosting, Examples of Regression',
2,
None,
'___sec28'),
'___sec27'),
('Gradient Boosting, Classification Example',
2,
None,
'___sec29'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec30'),
('Regression Case', 2, None, '___sec31'),
('Xgboost on the Cancer Data', 2, None, '___sec32')]}
'___sec28'),
('XGBoost: Extreme Gradient Boosting', 2, None, '___sec29'),
('Regression Case', 2, None, '___sec30'),
('Xgboost on the Cancer Data', 2, None, '___sec31')]}
end of tocinfo -->
<body>
@@ -559,6 +558,16 @@ plt<span style="color: #666666">.</span>show()
skplt<span style="color: #666666">.</span>metrics<span style="color: #666666">.</span>plot_cumulative_gain(y_test, y_probas)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
Recall that the cumulative gains curve shows the percentage of the
overall number of cases in a given category <em>gained</em> by targeting a
percentage of the total number of cases.
<p>
Similarly, the receiver operating characteristic curve, or ROC curve,
displays the diagnostic ability of a binary classifier system as its
discrimination threshold is varied. It plots the true positive rate against the false positive rate.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1026,6 +1035,11 @@ and find a new value for \( \rho_2=-1/2 \) and continue till we have reached \(
<h2 id="___sec26">Gradient Boosting, algorithm </h2>
<p>
Steepest descent is however not much used, since it only optimizes \( f \) at a fixed set of \( n \) points,
so we do not learn a function that can generalize. However, we can modify the algorithm by
fitting a weak learner to approximate the negative gradient signal.
<p>
Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function
$$
@@ -1050,15 +1064,7 @@ The way we proceed in an iterative fashion is to
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec27">Gradient Boosting Example, Regression </h2>
<p>
We discuss here the difference between the steepest descent approach and gradient boosting by repeating our simple regression example above.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec28">Gradient Boosting, Examples of Regression </h2>
<h2 id="___sec27">Gradient Boosting, Examples of Regression </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -1112,7 +1118,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec29">Gradient Boosting, Classification Example </h2>
<h2 id="___sec28">Gradient Boosting, Classification Example </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -1160,7 +1166,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec30">XGBoost: Extreme Gradient Boosting </h2>
<h2 id="___sec29">XGBoost: Extreme Gradient Boosting </h2>
<p>
<a href="https://github.com/dmlc/xgboost" target="_blank">XGBoost</a> or Extreme Gradient
@@ -1181,7 +1187,7 @@ It is now the algorithm which wins essentially all ML competitions!!!
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec31">Regression Case </h2>
<h2 id="___sec30">Regression Case </h2>
<p>
@@ -1236,7 +1242,7 @@ plt<span style="color: #666666">.</span>show()
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec32">Xgboost on the Cancer Data </h2>
<h2 id="___sec31">Xgboost on the Cancer Data </h2>
<p>
As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now.
doc/pub/week45/ipynb/ipynb-week45-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week45/ipynb/week45.ipynb
+13 -5
View File
@@ -463,6 +463,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Recall that the cumulative gains curve shows the percentage of the\n",
"overall number of cases in a given category *gained* by targeting a\n",
"percentage of the total number of cases.\n",
"\n",
"Similarly, the receiver operating characteristic curve, or ROC curve,\n",
"displays the diagnostic ability of a binary classifier system as its\n",
"discrimination threshold is varied. It plots the true positive rate against the false positive rate.\n",
"\n",
"\n",
"## Compare Bagging on Trees with Random Forests"
]
},
@@ -1198,6 +1207,10 @@
"\n",
"## Gradient Boosting, algorithm\n",
"\n",
"Steepest descent is however not much used, since it only optimizes $f$ at a fixed set of $n$ points,\n",
"so we do not learn a function that can generalize. However, we can modify the algorithm by\n",
"fitting a weak learner to approximate the negative gradient signal. \n",
"\n",
"Suppose we have a cost function $C(f)=\\sum_{i=0}^{n-1}L(y_i, f(x_i))$ where $y_i$ is our target and $f(x_i)$ the function which is meant to model $y_i$. The above cost function could be our standard squared-error function"
]
},
@@ -1228,11 +1241,6 @@
"\n",
"4. The final estimate is then $f_M(x) = \\sum_{m=1}^M\\nu h_m(u_m,x)$.\n",
"\n",
"## Gradient Boosting Example, Regression\n",
"\n",
"We discuss here the difference between the steepest descent approach and gradient boosting by repeating our simple regression example above. \n",
"\n",
"\n",
"## Gradient Boosting, Examples of Regression"
]
},
doc/src/week45/week45.do.txt
+13 -4
View File
@@ -377,6 +377,15 @@ plt.show()
!ec
Recall that the cumulative gains curve shows the percentage of the
overall number of cases in a given category *gained* by targeting a
percentage of the total number of cases.
Similarly, the receiver operating characteristic curve, or ROC curve,
displays the diagnostic ability of a binary classifier system as its
discrimination threshold is varied. It plots the true positive rate against the false positive rate.
!split
===== Compare Bagging on Trees with Random Forests =====
!bc pycod
@@ -811,6 +820,10 @@ and find a new value for $\rho_2=-1/2$ and continue till we have reached $m=M$.
!split
===== Gradient Boosting, algorithm =====
Steepest descent is however not much used, since it only optimizes $f$ at a fixed set of $n$ points,
so we do not learn a function that can generalize. However, we can modify the algorithm by
fitting a weak learner to approximate the negative gradient signal.
Suppose we have a cost function $C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i))$ where $y_i$ is our target and $f(x_i)$ the function which is meant to model $y_i$. The above cost function could be our standard squared-error function
!bt
\[
@@ -826,10 +839,6 @@ o For $m=1:M$, we
o update the estimate $f_m(x) = f_{m-1}(x)+\nu h_m(u_m,x)$;
o The final estimate is then $f_M(x) = \sum_{m=1}^M\nu h_m(u_m,x)$.
!split
===== Gradient Boosting Example, Regression =====
We discuss here the difference between the steepest descent approach and gradient boosting by repeating our simple regression example above.
!split