From ddec5a439427014825662978fc4ccd6bc32eab7b Mon Sep 17 00:00:00 2001 From: mhjensen Date: Sun, 10 Nov 2019 13:47:21 +0100 Subject: [PATCH] more update --- .../html/._DecisionTrees-bs000.html | 56 ++++---- .../html/._DecisionTrees-bs001.html | 56 ++++---- .../html/._DecisionTrees-bs002.html | 56 ++++---- .../html/._DecisionTrees-bs003.html | 56 ++++---- .../html/._DecisionTrees-bs004.html | 56 ++++---- .../html/._DecisionTrees-bs005.html | 56 ++++---- .../html/._DecisionTrees-bs006.html | 56 ++++---- .../html/._DecisionTrees-bs007.html | 56 ++++---- .../html/._DecisionTrees-bs008.html | 56 ++++---- .../html/._DecisionTrees-bs009.html | 56 ++++---- .../html/._DecisionTrees-bs010.html | 56 ++++---- .../html/._DecisionTrees-bs011.html | 56 ++++---- .../html/._DecisionTrees-bs012.html | 56 ++++---- .../html/._DecisionTrees-bs013.html | 56 ++++---- .../html/._DecisionTrees-bs014.html | 56 ++++---- .../html/._DecisionTrees-bs015.html | 56 ++++---- .../html/._DecisionTrees-bs016.html | 56 ++++---- .../html/._DecisionTrees-bs017.html | 56 ++++---- .../html/._DecisionTrees-bs018.html | 56 ++++---- .../html/._DecisionTrees-bs019.html | 56 ++++---- .../html/._DecisionTrees-bs020.html | 56 ++++---- .../html/._DecisionTrees-bs021.html | 56 ++++---- .../html/._DecisionTrees-bs022.html | 56 ++++---- .../html/._DecisionTrees-bs023.html | 56 ++++---- .../html/._DecisionTrees-bs024.html | 56 ++++---- .../html/._DecisionTrees-bs025.html | 56 ++++---- .../html/._DecisionTrees-bs026.html | 56 ++++---- .../html/._DecisionTrees-bs027.html | 56 ++++---- .../html/._DecisionTrees-bs028.html | 56 ++++---- .../html/._DecisionTrees-bs029.html | 56 ++++---- .../html/._DecisionTrees-bs030.html | 56 ++++---- .../html/._DecisionTrees-bs031.html | 56 ++++---- .../html/._DecisionTrees-bs032.html | 56 ++++---- .../html/._DecisionTrees-bs033.html | 56 ++++---- .../html/._DecisionTrees-bs034.html | 56 ++++---- .../html/._DecisionTrees-bs035.html | 56 ++++---- .../html/._DecisionTrees-bs036.html | 56 ++++---- .../html/._DecisionTrees-bs037.html | 56 ++++---- .../html/._DecisionTrees-bs038.html | 56 ++++---- .../html/._DecisionTrees-bs039.html | 56 ++++---- .../html/._DecisionTrees-bs040.html | 56 ++++---- .../html/._DecisionTrees-bs041.html | 56 ++++---- .../html/._DecisionTrees-bs042.html | 56 ++++---- .../html/._DecisionTrees-bs043.html | 56 ++++---- .../html/._DecisionTrees-bs044.html | 56 ++++---- .../html/._DecisionTrees-bs045.html | 56 ++++---- .../html/._DecisionTrees-bs046.html | 56 ++++---- .../html/._DecisionTrees-bs047.html | 56 ++++---- .../html/._DecisionTrees-bs048.html | 84 ++++++------ .../html/._DecisionTrees-bs049.html | 87 +++++++----- .../html/._DecisionTrees-bs050.html | 89 ++++++------- .../html/._DecisionTrees-bs051.html | 97 ++++++++------ .../html/._DecisionTrees-bs052.html | 101 ++++++-------- .../html/._DecisionTrees-bs053.html | 97 ++++++++------ .../html/._DecisionTrees-bs054.html | 108 +++++++-------- .../html/._DecisionTrees-bs055.html | 88 +++++------- .../html/._DecisionTrees-bs056.html | 83 ++++++------ .../html/._DecisionTrees-bs057.html | 125 +++++++++++------- .../html/._DecisionTrees-bs058.html | 119 ++++++++--------- .../html/._DecisionTrees-bs059.html | 110 ++++++--------- .../html/._DecisionTrees-bs060.html | 115 ++++++++++------ .../DecisionTrees/html/DecisionTrees-bs.html | 56 ++++---- .../html/DecisionTrees-reveal.html | 58 ++++---- .../html/DecisionTrees-solarized.html | 89 +++++++------ doc/pub/DecisionTrees/html/DecisionTrees.html | 89 +++++++------ .../DecisionTrees/ipynb/DecisionTrees.ipynb | 36 +++-- .../ipynb/ipynb-DecisionTrees-src.tar.gz | Bin 294061 -> 294061 bytes .../pdf/DecisionTrees-minted.pdf | Bin 541553 -> 542453 bytes doc/src/DecisionTrees/DecisionTrees.do.txt | 30 +++-- 69 files changed, 2155 insertions(+), 2194 deletions(-) diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html index 7b1ab4223..39a7c6c43 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs000.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -311,7 +309,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs001.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs001.html index eafffa067..020ba0890 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs001.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs001.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -316,7 +314,7 @@ given some assumptions, make predictions about the target feature value
  • 10
  • 11
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs002.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs002.html index 4ee550be1..f39b144d4 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs002.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs002.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -294,7 +292,7 @@ This tree was produced using the Wisconsin cancer data (discussed here as well,
  • 11
  • 12
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs003.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs003.html index 1ad13205c..615f518a3 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs003.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs003.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -302,7 +300,7 @@ node.
  • 12
  • 13
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs004.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs004.html index c1ac1b1e5..d10bc9026 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs004.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs004.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -303,7 +301,7 @@ Then we are essentially done!
  • 13
  • 14
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs005.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs005.html index e83300f6b..6110977b0 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs005.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs005.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -382,7 +380,7 @@ plt.show()
  • 14
  • 15
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs006.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs006.html index 516738052..08579e547 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs006.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs006.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -315,7 +313,7 @@ within box \( j \).
  • 15
  • 16
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs007.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs007.html index 38e49d9b5..973d9cfd9 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs007.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs007.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -307,7 +305,7 @@ better tree in some future step.
  • 16
  • 17
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs008.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs008.html index eb2b695f5..3f4594277 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs008.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs008.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -340,7 +338,7 @@ region contains more than five observations.
  • 17
  • 18
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs009.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs009.html index f36e75ae0..2704094d8 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs009.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs009.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -309,7 +307,7 @@ parameter \( \alpha \).
  • 18
  • 19
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs010.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs010.html index c739a708d..88f46e5ae 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs010.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs010.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -322,7 +320,7 @@ subtree corresponding to \( \alpha \).
  • 19
  • 20
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs011.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs011.html index 304e80744..c70fa55d6 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs011.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs011.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -318,7 +316,7 @@ MathJax.Hub.Config({
  • 20
  • 21
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs012.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs012.html index 9797836d6..21388431c 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs012.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs012.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -310,7 +308,7 @@ fall into that region.
  • 21
  • 22
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs013.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs013.html index a5c538e37..af177e3f3 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs013.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs013.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -315,7 +313,7 @@ than is the classification error rate.
  • 22
  • 23
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs014.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs014.html index 89b8e2882..3559b2d81 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs014.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs014.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -341,7 +339,7 @@ $$
  • 23
  • 24
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs015.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs015.html index 0e396da70..85fa0c3e7 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs015.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs015.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -332,7 +330,7 @@ os.system(cmd)
  • 24
  • 25
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs016.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs016.html index 9b7bfa0bd..4bb186d4f 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs016.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs016.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -323,7 +321,7 @@ os.system(cmd)
  • 25
  • 26
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs017.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs017.html index c036e8e25..bd7b75001 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs017.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs017.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -306,7 +304,7 @@ We discuss both algorithms with applications here. The popular library Scikit
  • 26
  • 27
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs018.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs018.html index 3ed123bea..5c166ac71 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs018.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs018.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -296,7 +294,7 @@ MathJax.Hub.Config({
  • 27
  • 28
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs019.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs019.html index d5b6d555b..a96f1adc7 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs019.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs019.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -296,7 +294,7 @@ MathJax.Hub.Config({
  • 28
  • 29
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs020.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs020.html index 28a4ca907..0e40e84e6 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs020.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs020.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -336,7 +334,7 @@ The table here summarizes the various attributes and
  • 29
  • 30
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs021.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs021.html index a28360fbc..f41b68e3f 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs021.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs021.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -367,7 +365,7 @@ os.system(cmd)
  • 30
  • 31
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs022.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs022.html index 379cb05a9..0ef377ff9 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs022.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs022.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -369,7 +367,7 @@ split = get_split(dataset)
  • 31
  • 32
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs023.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs023.html index 4bfee693c..9c22332ff 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs023.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs023.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -327,7 +325,7 @@ attributes at each step while growing the tree.
  • 32
  • 33
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs024.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs024.html index aac18ce99..66e3f6c03 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs024.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs024.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -487,7 +485,7 @@ MathJax.Hub.Config({
  • 33
  • 34
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs025.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs025.html index f908376e9..7d8cd178a 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs025.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs025.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -340,7 +338,7 @@ deep_tree_clf.fit(X_train_scaled, y_train)
  • 34
  • 35
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs026.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs026.html index 4329762e6..33eb58512 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs026.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs026.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -363,7 +361,7 @@ plt.show()
  • 35
  • 36
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs027.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs027.html index cfbb5765b..24477fabf 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs027.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs027.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -319,7 +317,7 @@ plt.show()
  • 36
  • 37
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs028.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs028.html index 35edf1d2d..db67c522c 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs028.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs028.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -313,7 +311,7 @@ tree_reg.fit(X, y)
  • 37
  • 38
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs029.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs029.html index 13bc32c58..8b4f0cb2d 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs029.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs029.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -369,7 +367,7 @@ plt.show()
  • 38
  • 39
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs030.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs030.html index dc6422ca8..7cdf1a908 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs030.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs030.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -305,7 +303,7 @@ MathJax.Hub.Config({
  • 39
  • 40
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs031.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs031.html index 8d2c52dbb..743fe7e25 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs031.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs031.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -310,7 +308,7 @@ trees can be substantially improved.
  • 40
  • 41
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs032.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs032.html index eccae9298..42a5b09a5 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs032.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs032.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -318,7 +316,7 @@ We discuss these methods here.
  • 41
  • 42
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs033.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs033.html index 9d85bacaa..38eb671aa 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs033.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs033.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -299,7 +297,7 @@ MathJax.Hub.Config({
  • 42
  • 43
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs034.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs034.html index 7ac72b06b..223ffdebf 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs034.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs034.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -310,7 +308,7 @@ learning method.
  • 43
  • 44
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs035.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs035.html index eb7537487..0faebab83 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs035.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs035.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -320,7 +318,7 @@ predictor, averaged over all \( B \) trees.
  • 44
  • 45
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs036.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs036.html index 67e4be8eb..c63bad3ce 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs036.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs036.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -312,7 +310,7 @@ plt.show()
  • 45
  • 46
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs037.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs037.html index 7414b5019..dadb9d6b0 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs037.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs037.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -342,7 +340,7 @@ voting_clf.fit(X_train, y_train)
  • 46
  • 47
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs038.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs038.html index d07b2de26..2fbf3909c 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs038.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs038.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -350,7 +348,7 @@ voting_clf.fit(X_train, y_train)
  • 47
  • 48
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs039.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs039.html index 4a72f3034..43b55b06b 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs039.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs039.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -353,7 +351,7 @@ plt.show()
  • 48
  • 49
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs040.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs040.html index 515190eb1..4cb7a033c 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs040.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs040.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -354,7 +352,7 @@ plt.show()
  • 49
  • 50
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs041.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs041.html index e5f1f6fb0..403714f96 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs041.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs041.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -335,7 +333,7 @@ this setting.
  • 50
  • 51
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs042.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs042.html index 73442fc1b..134430144 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs042.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs042.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -317,7 +315,7 @@ We will grow of forest of say \( M \) trees.
  • 51
  • 52
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs043.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs043.html index 453b9a6c4..52938a941 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs043.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs043.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -364,7 +362,7 @@ plt.show()
  • 52
  • 53
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs044.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs044.html index e1c3aaa31..0a0c842a0 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs044.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs044.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -313,7 +311,7 @@ np.sum(y_pred =
  • 53
  • 54
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs045.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs045.html index 0205b16f6..cf134e215 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs045.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs045.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -308,7 +306,7 @@ them with a factor.
  • 54
  • 55
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs046.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs046.html index d4943b91a..ac4baa3f5 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs046.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs046.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -343,7 +341,7 @@ In iterative fitting or additive modeling, we minimize the cost function with re
  • 55
  • 56
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs047.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs047.html index ebea16cc7..c0336091e 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs047.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs047.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -316,7 +314,7 @@ at the internal nodes, and the predictions at the terminal nodes.
  • 56
  • 57
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs048.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs048.html index 3f4459f88..2f0910c8f 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs048.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs048.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -274,33 +272,45 @@ MathJax.Hub.Config({ To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.

    -For simplicity we assume also that our functions \( b(x;\gamma)=\gamma_0+\gamma_1 x \), with \( \gamma_0 \) and \( \gamma_1 \) as the parameters to be determined. +For simplicity we assume also that our functions \( b(x;\gamma)=1+\gamma x \).

    This means that for every iteration, we need to optmize $$ -(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\gambda}\hspace{0.2cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(\gamma_0+\gamma_1 x_i))^2. -\[ +(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2. $$

    -We start our iteration by simply setting \( \f_0(x)=0 \). +We start our iteration by simply setting \( f_0(x)=0 \). Taking the derivatives with respect to \( \beta \) and \( \gamma \) we obtain $$ -\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(\gamma_0+\gamma_1 x_i)(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0, $$ and $$ -\frac{\partial {\cal C}}{\partial \gamma_0} =-2\sum_{i}\beta(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. $$ -and +We can then rewrite these equations as (defining \( w_i=1+\gamma x_i) \) $$ -\frac{\partial {\cal C}}{\partial \gamma_1} = =-2\sum_{i}\beta x_i(y_i-\beta(\gamma_0+\gamma_1 x_i))=0. +\gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0, $$ +which gives us \( \beta = \boldsymbol{w}^T\boldsymbol{y}/(\boldsymbol{w}^T\boldsymbol{w}) \). Similarly we have +$$ +\beta\gamma \boldsymbol{x}^T(\boldsymbol{y}-\beta(1+\gamma \boldsymbol{x}))=0, +$$ + +

    +which leads \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\em{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting +for \( \beta \) gives us an equation for \( \gamma \). + +

    +The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as +\( f_1(x) = \beta_1(1+\gamma_1x) \). Doing this \( M \) times results in our final estimate for the function \( f \). +

    @@ -327,7 +337,7 @@ $$

  • 57
  • 58
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs049.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs049.html index 3094ac20d..3c9f43bb2 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs049.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs049.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,10 +266,37 @@ MathJax.Hub.Config({ -

    Finding the Optimal Parameters

    +

    Iterative Fitting, Classification, AdaBoost

    -With these equations we can then in turn find the parameters \( \beta_1 \) and \( \gamma_0^{1} \) and \( \gamma_1^1 \) as +Let us consider a binary classification problem with two outcomes \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of +observations. We define a classification function \( G(x) \) which produces a prediction taking any of the two values +\( \{-1,1\} \). + +

    +The error rate of the training sample is then + +$$ +\mathrm{\overline{err}}=\frac{1}{n} \sum_{i=0}^{n-1} I(y_i\ne G(x_i)). +$$ + +

    +The iterative procedure starts with defining a weak classifier whose +error rate is barely better than random guessing. The iterative +procedure in boosting is to sequentially apply a the weak +classification algorithm to repeatedly modified versions of the data +producing a sequence of weak classifiers \( G_m(x) \). + +

    +Here we will express our function \( f(x) \) in terms of \( G(x) \). That is +$$ +f_M(x) = \sum_{i=1}^M \beta_m b(x;\gamma_m), +$$ + +will be a function of +$$ +G_M(x) = \mathrm{sign} \sum_{i=1}^M \alpha_m G_m(x). +$$

    @@ -299,7 +324,7 @@ With these equations we can then in turn find the parameters \( \beta_1 \) and \

  • 58
  • 59
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs050.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs050.html index bdcc83fe6..c1ce98823 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs050.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs050.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,37 +266,30 @@ MathJax.Hub.Config({ -

    Iterative Fitting, Classification, AdaBoost

    +

    Adaptive Boosting, AdaBoost

    -Let us consider a binary classification problem with two outcomes \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of -observations. We define a classification function \( G(x) \) which produces a prediction taking any of the two values -\( \{-1,1\} \). - -

    -The error rate of the training sample is then - +In our iterative procedure we define thus $$ -\mathrm{\overline{err}}=\frac{1}{n} \sum_{i=0}^{n-1} I(y_i\ne G(x_i)). +f_m(x) = f_{m-1}(x)+\beta_mG_m(x). $$

    -The iterative procedure starts with defining a weak classifier whose -error rate is barely better than random guessing. The iterative -procedure in boosting is to sequentially apply a the weak -classification algorithm to repeatedly modified versions of the data -producing a sequence of weak classifiers \( G_m(x) \). +The simplest possible cost function which leads (also simple from a computational point of view) to the AdaBoost algorithm is the +exponential cost/loss function defined as +$$ +C(\boldsymbol{y},\boldsymbol{f}) = \sum_{i=0}^{n-1}\exp{(-y_i(f_{m-1}(x_i)+\beta G(x_i))}. +$$

    -Here we will express our function \( f(x) \) in terms of \( G(x) \). That is +We optimize \( \beta \) and \( G \) for each value of \( m=1:M \) as we did in the regression case. +This is normally done in two steps. Let us however first rewrite the cost function as + $$ -f_M(x) = \sum_{i=1}^M \beta_m b(x;\gamma_m), +C(\boldsymbol{y},\boldsymbol{f}) = \sum_{i=0}^{n-1}w_i^{m}\exp{(-y_i\beta G(x_i))}, $$ -will be a function of -$$ -G_M(x) = \mathrm{sign} \sum_{i=1}^M \alpha_m G_m(x). -$$ +where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).

    @@ -326,7 +317,7 @@ $$

  • 59
  • 60
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs051.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs051.html index 447919a1e..368345bff 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs051.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs051.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,32 +266,47 @@ MathJax.Hub.Config({ -

    Adaptive Boosting, AdaBoost

    +

    Building up AdaBoost

    -In our iterative procedure we define thus +First, for any \( \beta > 0 \), we optimize \( G \) by setting $$ -f_m(x) = f_{m-1}(x)+\beta_mG_m(x). +G_m(x) = \mathrm{sign} \sum_{i=0}^{n-1} w_i^m I(y_i \ne G_(x_i)), $$ +which is the classifier that minimizes the weighted error rate in predicting \( y \). +

    -The simplest possible cost function which leads (also simple from a computational point of view) to the AdaBoost algorithm is the -exponential cost/loss function defined as +We can do this by rewriting $$ -C(\boldsymbol{y},\boldsymbol{f}) = \sum_{i=0}^{n-1}\exp{(-y_i(f_{m-1}(x_i)+\beta G(x_i))}. +\exp{-(\beta)}\sum_{y_i=G(x_i)}w_i^m+\exp{(\beta)}\sum_{y_i\ne G(x_i)}w_i^m, $$ -

    -We optimize \( \beta \) and \( G \) for each value of \( m=1:M \) as we did in the regression case. -This is normally done in two steps. Let us however first rewrite the cost function as - +which can be rewritten as $$ -C(\boldsymbol{y},\boldsymbol{f}) = \sum_{i=0}^{n-1}w_i^{m}\exp{(-y_i\beta G(x_i))}, +(\exp{(\beta)}-\exp{-(\beta)})\sum_{i=0}^{n-1}w_i^mI(y_i\ne G(x_i))+\exp{(-\beta)}\sum_{i=0}^{n-1}w_i^m=0, $$ -where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \). +which leads to +$$ +\beta_m = \frac{1}{2}\log{\frac{1-\mathrm{\overline{err}}}{\mathrm{\overline{err}}}}, +$$ + +where we have redefined the error as +$$ +\mathrm{\overline{err}}_m=\frac{1}{n}\frac{\sum_{i=0}^{n-1}w_i^mI(y_i\ne G(x_i)}{\sum_{i=0}^{n-1}w_i^m}, +$$ + +which leads to an update of +$$ +f_m(x) = f_{m-1}(x) +\beta_m G_m(x). +$$ + +This leads to the new weights +$$ +w_i^{m+1} = w_i^m \exp{(-y_i\beta_m G_m(x_i))} +$$ -

      @@ -319,7 +332,7 @@ where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).
    • 60
    • 61
    • ...
      • -
    • 63
      • +
    • 62
    • »
    diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs052.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs052.html index 9cfb5899f..2e71bf81d 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs052.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs052.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,47 +266,26 @@ MathJax.Hub.Config({ -

    Building up AdaBoost

    +

    Adaptive boosting: AdaBoost, Basic Algorithm

    -First, for any \( \beta > 0 \), we optimize \( G \) by setting -$$ -G_m(x) = \mathrm{sign} \sum_{i=0}^{n-1} w_i^m I(y_i \ne G_(x_i)), -$$ - -which is the classifier that minimizes the weighted error rate in predicting \( y \). +The algorithm here is rather straightforward. Assume that our weak +classifier is a decision tree and we consider a binary set of outputs +with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of +observations. Our design matrix is given in terms of the +feature/predictor vectors +\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1}] \). Finally, we define also a +classifier determined by our data via a function \( G(x) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \).

    -We can do this by rewriting +We have already defined the misclassification error \( \mathrm{err} \) as $$ -\exp{-(\beta)}\sum_{y_i=G(x_i)}w_i^m+\exp{(\beta)}\sum_{y_i\ne G(x_i)}w_i^m, +\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(x_i)), $$ -which can be rewritten as -$$ -(\exp{(\beta)}-\exp{-(\beta)})\sum_{i=0}^{n-1}w_i^mI(y_i\ne G(x_i))+\exp{(-\beta)}\sum_{i=0}^{n-1}w_i^m=0, -$$ - -which leads to -$$ -\beta_m = \frac{1}{2}\log{\frac{1-\mathrm{\overline{err}}}{\mathrm{\overline{err}}}}, -$$ - -where we have redefined the error as -$$ -\mathrm{\overline{err}}_m=\frac{1}{n}\frac{\sum_{i=0}^{n-1}w_i^mI(y_i\ne G(x_i)}{\sum_{i=0}^{n-1}w_i^m}, -$$ - -which leads to an update of -$$ -f_m(x) = f_{m-1}(x) +\beta_m G_m(x). -$$ - -This leads to the new weights -$$ -w_i^{m+1} = w_i^m \exp{(-y_i\beta_m G_m(x_i))} -$$ +where the function \( I() \) is one if we misclassify and zero if we classify correctly. +

    diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs053.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs053.html index 040d28dd3..bd612b14e 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs053.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs053.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,24 +266,42 @@ MathJax.Hub.Config({ -

    Adaptive boosting: AdaBoost, Basic Algorithm

    +

    Basic Steps of AdaBoost

    -The algorithm here is rather straightforward. Assume that our weak -classifier is a decision tree and we consider a binary set of outputs -with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of -observations. Our design matrix is given in terms of the -feature/predictor vectors -\( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1}] \). Finally, we define also a -classifier determined by our data via a function \( G(x) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \). +With the above definitions we are now ready to set up the algorithm for AdaBoost. +The basic idea is to set up weights which will be used to scale the correctly classified and the misclassified cases. + +

      +
    1. We start by initializing all weights to \( w_i = 1/n \), with \( i=0,1,2,\dots n-1 \). It is to see then that \( \sum_{i=0}^{n-1}w_i = 1 \).
      2. +
    2. We rewrite the misclassification error as
      3. +
    -

    -We have already defined the misclassification error \( \mathrm{err} \) as $$ -\mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(x_i)), +\mathrm{\overline{err}}_m=\frac{\sum_{i=0}^{n-1}w_i^m I(y_i\ne G(x_i))}{\sum_{i=0}^{n-1}w_i}, $$ -where the function \( I() \) is one if we misclassify and zero if we classify correctly. + +

      +
    1. Then we start looping over all attempts at classifying, namely we start an iterative process for \( m=1:M \), where \( M \) is the final number of classifications. Our given classifier could for example be a plain decision tree. + +
        +
      1. Fit then a given classifier to the training using the weights \( w_i \).
        2. +
      2. Compute then \( \mathrm{err} \) and figure out which events are classified properly and which are classified wrongly.
        3. +
      3. Define a quantity \( \alpha_{m} = \log{(1-\mathrm{\overline{err}}_m)/\mathrm{\overline{err}}_m} \)
        4. +
      4. Set the new weights to \( w_i = w_i\times \exp{(\alpha_m I(y_i\ne G(x_i)} \).
        5. +
      + +
    2. Compute the new classifier \( G(x)= \sum_{i=0}^{n-1}\alpha_m I(y_i\ne G(x_i) \).
      3. +
    + +For the iterations with \( m \le 2 \) the weights are modified +individually at each steps. The obersvations which were misclassified +at iteration \( m-1 \) have a weight which is larger than those which were +classified properly. As this proceeds, the observations which were +difficult to classifiy correctly are given a larger influence. Each +new classification step \( m \) is then forced to concentrate on those +observations that are missed in the previous iterations.

    @@ -311,7 +327,6 @@ where the function \( I() \) is one if we misclassify and zero if we classify co

  • 60
  • 61
  • 62
    • -
  • 63
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs054.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs054.html index 11e78a670..45e976c1e 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs054.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs054.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,43 +266,36 @@ MathJax.Hub.Config({ -

    Basic Steps of AdaBoost

    +

    AdaBoost Examples

    -With the above definitions we are now ready to set up the algorithm for AdaBoost. -The basic idea is to set up weights which will be used to scale the correctly classified and the misclassified cases. +Using Scikit-Learn it is easy to appply the adaptive boosting algorithm, as done here. -

      -
    1. We start by initializing all weights to \( w_i = 1/n \), with \( i=0,1,2,\dots n-1 \). It is to see then that \( \sum_{i=0}^{n-1}w_i = 1 \).
      2. -
    2. We rewrite the misclassification error as
      3. -
    +

    -$$ -\mathrm{\overline{err}}_m=\frac{\sum_{i=0}^{n-1}w_i^m I(y_i\ne G(x_i))}{\sum_{i=0}^{n-1}w_i}, -$$ + +

    from sklearn.ensemble import AdaBoostClassifier
 
+ada_clf = AdaBoostClassifier(
+    DecisionTreeClassifier(max_depth=1), n_estimators=200,
+    algorithm="SAMME.R", learning_rate=0.5, random_state=42)
+ada_clf.fit(X_train, y_train)
 
-
      
-
    1.  Then we start looping over all attempts at classifying, namely we start an iterative process for \( m=1:M \), where \( M \) is the final number of classifications. Our given classifier could for example be a plain decision tree.
-
-
        
- 
      1.  Fit then a given classifier to the training using the weights \( w_i \).
        2. 
- 
      2.  Compute then \( \mathrm{err} \) and figure out which events are classified properly and which are classified wrongly.
        3. 
- 
      3.  Define a quantity \( \alpha_{m} = \log{(1-\mathrm{\overline{err}}_m)/\mathrm{\overline{err}}_m} \)
        4. 
- 
      4.  Set the new weights to \( w_i = w_i\times \exp{(\alpha_m I(y_i\ne G(x_i)} \).
        5. 
-
      
-
-
    2.  Compute the new classifier \( G(x)= \sum_{i=0}^{n-1}\alpha_m I(y_i\ne G(x_i) \).
      3. 
-
    
-
-For the iterations with \( m \le 2 \) the weights are modified
-individually at each steps. The obersvations which were misclassified
-at iteration \( m-1 \) have a weight which is larger than those which were
-classified properly. As this proceeds, the observations which were
-difficult to classifiy correctly are given a larger influence. Each
-new classification step \( m \) is then forced to concentrate on those
-observations that are missed in the previous iterations.
+from sklearn.ensemble import AdaBoostClassifier
 
+ada_clf = AdaBoostClassifier(
+    DecisionTreeClassifier(max_depth=1), n_estimators=200,
+    algorithm="SAMME.R", learning_rate=0.5, random_state=42)
+ada_clf.fit(X_train_scaled, y_train)
+y_pred = ada_clf.predict(X_test_scaled)
+skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
+plt.show()
+y_probas = ada_clf.predict_proba(X_test_scaled)
+skplt.metrics.plot_roc(y_test, y_probas)
+plt.show()
+skplt.metrics.plot_cumulative_gain(y_test, y_probas)
+plt.show()
+

    @@ -328,7 +319,6 @@ observations that are missed in the previous iterations.

  • 60
  • 61
  • 62
    • -
  • 63
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html index 93e36c0a6..835ee26e1 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs055.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,36 +266,21 @@ MathJax.Hub.Config({ -

    AdaBoost Examples

    +

    Gradient boosting: Basics

    -Using Scikit-Learn it is easy to appply the adaptive boosting algorithm, as done here. +Gradient boosting is again a similar technique to Adapative boosting, +it combines so-called weak classifiers or regressors into a strong +method via a series of iterations.

    +In order to understand the method, let us illustrate its basics by +bringing back the essential steps in linear regression, where our cost +function was the least squares function. - -

    from sklearn.ensemble import AdaBoostClassifier
+
    
+See discussion during lecture November 8.
 
-ada_clf = AdaBoostClassifier(
-    DecisionTreeClassifier(max_depth=1), n_estimators=200,
-    algorithm="SAMME.R", learning_rate=0.5, random_state=42)
-ada_clf.fit(X_train, y_train)
-
-from sklearn.ensemble import AdaBoostClassifier
-
-ada_clf = AdaBoostClassifier(
-    DecisionTreeClassifier(max_depth=1), n_estimators=200,
-    algorithm="SAMME.R", learning_rate=0.5, random_state=42)
-ada_clf.fit(X_train_scaled, y_train)
-y_pred = ada_clf.predict(X_test_scaled)
-skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
-plt.show()
-y_probas = ada_clf.predict_proba(X_test_scaled)
-skplt.metrics.plot_roc(y_test, y_probas)
-plt.show()
-skplt.metrics.plot_cumulative_gain(y_test, y_probas)
-plt.show()
-

    @@ -320,7 +303,6 @@ plt.show()

  • 60
  • 61
  • 62
    • -
  • 63
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html index eb94bb397..14623a2c8 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs056.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,22 +266,30 @@ MathJax.Hub.Config({ -

    Gradient boosting: Basics

    +

    Gradient Boosting, algorithm

    -Gradient boosting is again a similar technique to Adapative boosting, -it combines so-called weak classifiers or regressors into a strong -method via a series of iterations. +Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function +$$ +C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2. +$$

    -In order to understand the method, let us illustrate its basics by -bringing back the essential steps in linear regression, where our cost -function was the least squares function. +The way we proceed in an iterative fashion is to -

    -See discussion during lecture November 8. +

      +
    1. Initialize our estimate \( f_0(x) \).
      2. +
    2. For \( m=1:M \), we + +
        +
      1. compute the negative gradient vector \( \boldsymbol{u}_m = -\partial C(\boldsymbol{y},\boldsymbol{f})/\partial \boldsymbol{f}(x) \) at \( f(x) = f_{m-1}(x) \);
        2. +
      2. fit the so-called base-learner to the negative gradient \( h_m(u_m,x) \);
        3. +
      3. update the estimate \( f_m(x) = f_{m-1}(x)+\nu h_m(u_m,x) \);
        4. +
      + +
    3. The final estimate is then \( f_M(x) = \sum_{m=1}^M\nu h_m(u_m,x) \).
      4. +
    -

      @@ -304,7 +310,6 @@ See discussion during lecture November 8.
    • 60
    • 61
    • 62
      • -
    • 63
    • »
    diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs057.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs057.html index 8975c7810..e35744ad7 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs057.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs057.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,30 +266,58 @@ MathJax.Hub.Config({ -

    Gradient Boosting, algorithm

    - +

    Gradient Boosting, Examples of Regression

    -Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function -$$ -C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2. -$$ + +

    import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.ensemble import GradientBoostingRegressor
+from sklearn.preprocessing import StandardScaler
+import scikitplot as skplt
+from sklearn.metrics import mean_squared_error
+
+n = 100
+maxdegree = 6
+
+# Make data set.
+x = np.linspace(-3, 3, n).reshape(-1, 1)
+y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
+
+error = np.zeros(maxdegree)
+bias = np.zeros(maxdegree)
+variance = np.zeros(maxdegree)
+polydegree = np.zeros(maxdegree)
+X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+for degree in range(1,maxdegree):
+    model = GradientBoostingRegressor(max_depth=degree, n_estimators=100, learning_rate=1.0)  
+    model.fit(X_train_scaled,y_train)
+    y_pred = model.predict(X_test_scaled)
+    polydegree[degree] = degree
+    error[degree] = np.mean( np.mean((y_test - y_pred)**2) )
+    bias[degree] = np.mean( (y_test - np.mean(y_pred))**2 )
+    variance[degree] = np.mean( np.var(y_pred) )
+    print('Max depth:', degree)
+    print('Error:', error[degree])
+    print('Bias^2:', bias[degree])
+    print('Var:', variance[degree])
+    print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))
+
+plt.xlim(1,maxdegree-1)
+plt.plot(polydegree, error, label='Error')
+plt.plot(polydegree, bias, label='bias')
+plt.plot(polydegree, variance, label='Variance')
+plt.legend()
+save_fig("gdregression")
+plt.show()
+

    -The way we proceed in an iterative fashion is to - -

      -
    1. Initialize our estimate \( f_0(x) \).
      2. -
    2. For \( m=1:M \), we - -
        -
      1. compute the negative gradient vector \( \boldsymbol{u}_m = -\partial C(\boldsymbol{y},\boldsymbol{f})/\partial \boldsymbol{f}(x) \) at \( f(x) = f_{m-1}(x) \);
        2. -
      2. fit the so-called base-learner to the negative gradient \( h_m(u_m,x) \);
        3. -
      3. update the estimate \( f_m(x) = f_{m-1}(x)+\nu h_m(u_m,x) \);
        4. -
      - -
    3. The final estimate is then \( f_M(x) = \sum_{m=1}^M\nu h_m(u_m,x) \).
      4. -
    -

      @@ -311,7 +337,6 @@ The way we proceed in an iterative fashion is to
    • 60
    • 61
    • 62
      • -
    • 63
    • »
    diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs058.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs058.html index ddda5b9cf..2cda0d1a6 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs058.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs058.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,55 +266,49 @@ MathJax.Hub.Config({ -

    Gradient Boosting, Examples of Regression

    +

    Gradient Boosting, Classification Example 

    import matplotlib.pyplot as plt
 import numpy as np
-from sklearn.model_selection import train_test_split
-from sklearn.ensemble import GradientBoostingRegressor
-from sklearn.preprocessing import StandardScaler
+from sklearn.model_selection import  train_test_split 
+from sklearn.datasets import load_breast_cancer
 import scikitplot as skplt
-from sklearn.metrics import mean_squared_error
+from sklearn.ensemble import GradientBoostingClassifier
+from sklearn.model_selection import cross_validate
 
-n = 100
-maxdegree = 6
+# Load the data
+cancer = load_breast_cancer()
 
-# Make data set.
-x = np.linspace(-3, 3, n).reshape(-1, 1)
-y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
-
-error = np.zeros(maxdegree)
-bias = np.zeros(maxdegree)
-variance = np.zeros(maxdegree)
-polydegree = np.zeros(maxdegree)
-X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
+X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+#now scale the data
+from sklearn.preprocessing import StandardScaler
 scaler = StandardScaler()
 scaler.fit(X_train)
 X_train_scaled = scaler.transform(X_train)
 X_test_scaled = scaler.transform(X_test)
 
-for degree in range(1,maxdegree):
-    model = GradientBoostingRegressor(max_depth=degree, n_estimators=100, learning_rate=1.0)  
-    model.fit(X_train_scaled,y_train)
-    y_pred = model.predict(X_test_scaled)
-    polydegree[degree] = degree
-    error[degree] = np.mean( np.mean((y_test - y_pred)**2) )
-    bias[degree] = np.mean( (y_test - np.mean(y_pred))**2 )
-    variance[degree] = np.mean( np.var(y_pred) )
-    print('Max depth:', degree)
-    print('Error:', error[degree])
-    print('Bias^2:', bias[degree])
-    print('Var:', variance[degree])
-    print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))
+gd_clf = GradientBoostingClassifier(max_depth=3, n_estimators=100, learning_rate=1.0)  
+gd_clf.fit(X_train_scaled, y_train)
+#Cross validation
+accuracy = cross_validate(gd_clf,X_test_scaled,y_test,cv=10)['test_score']
+print(accuracy)
+print("Test set accuracy with Random Forests and scaled data: {:.2f}".format(gd_clf.score(X_test_scaled,y_test)))
 
-plt.xlim(1,maxdegree-1)
-plt.plot(polydegree, error, label='Error')
-plt.plot(polydegree, bias, label='bias')
-plt.plot(polydegree, variance, label='Variance')
-plt.legend()
-save_fig("gdregression")
+import scikitplot as skplt
+y_pred = gd_clf.predict(X_test_scaled)
+skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
+save_fig("gdclassiffierconfusion")
+plt.show()
+y_probas = gd_clf.predict_proba(X_test_scaled)
+skplt.metrics.plot_roc(y_test, y_probas)
+save_fig("gdclassiffierroc")
+plt.show()
+skplt.metrics.plot_cumulative_gain(y_test, y_probas)
+save_fig("gdclassiffiercgain")
 plt.show()

    @@ -338,7 +330,6 @@ plt.show()

  • 60
  • 61
  • 62
    • -
  • 63
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs059.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs059.html index f99146173..02d92efc6 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs059.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs059.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,51 +266,24 @@ MathJax.Hub.Config({ -

    Gradient Boosting, Classification Example

    +

    XGBoost: Extreme Gradient Boosting

    +

    +XGBoost or Extreme Gradient +Boosting, is an optimized distributed gradient boosting library +designed to be highly efficient, flexible and portable. It implements +machine learning algorithms under the Gradient Boosting +framework. XGBoost provides a parallel tree boosting that solve many +data science problems in a fast and accurate way. See the article by Chen and Guestrin. - -

    import matplotlib.pyplot as plt
-import numpy as np
-from sklearn.model_selection import  train_test_split 
-from sklearn.datasets import load_breast_cancer
-import scikitplot as skplt
-from sklearn.ensemble import GradientBoostingClassifier
-from sklearn.model_selection import cross_validate
+
    
+The authors design and build a highly scalable end-to-end tree
+boosting system. It has  a theoretically justified weighted quantile
+sketch for efficient proposal calculation. It introduces a novel sparsity-aware algorithm for parallel tree learning and an effective cache-aware block structure for out-of-core tree learning.
 
-# Load the data
-cancer = load_breast_cancer()
+
    
+It is now the algorithm which wins essentially all ML competitions!!!
 
-X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
-print(X_train.shape)
-print(X_test.shape)
-#now scale the data
-from sklearn.preprocessing import StandardScaler
-scaler = StandardScaler()
-scaler.fit(X_train)
-X_train_scaled = scaler.transform(X_train)
-X_test_scaled = scaler.transform(X_test)
-
-gd_clf = GradientBoostingClassifier(max_depth=3, n_estimators=100, learning_rate=1.0)  
-gd_clf.fit(X_train_scaled, y_train)
-#Cross validation
-accuracy = cross_validate(gd_clf,X_test_scaled,y_test,cv=10)['test_score']
-print(accuracy)
-print("Test set accuracy with Random Forests and scaled data: {:.2f}".format(gd_clf.score(X_test_scaled,y_test)))
-
-import scikitplot as skplt
-y_pred = gd_clf.predict(X_test_scaled)
-skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
-save_fig("gdclassiffierconfusion")
-plt.show()
-y_probas = gd_clf.predict_proba(X_test_scaled)
-skplt.metrics.plot_roc(y_test, y_probas)
-save_fig("gdclassiffierroc")
-plt.show()
-skplt.metrics.plot_cumulative_gain(y_test, y_probas)
-save_fig("gdclassiffiercgain")
-plt.show()
-

    @@ -331,7 +302,6 @@ plt.show()

  • 60
  • 61
  • 62
    • -
  • 63
  • »
    • diff --git a/doc/pub/DecisionTrees/html/._DecisionTrees-bs060.html b/doc/pub/DecisionTrees/html/._DecisionTrees-bs060.html index 864e4c3c6..5af631131 100644 --- a/doc/pub/DecisionTrees/html/._DecisionTrees-bs060.html +++ b/doc/pub/DecisionTrees/html/._DecisionTrees-bs060.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -268,24 +266,58 @@ MathJax.Hub.Config({ -

    XGBoost: Extreme Gradient Boosting

    +

    Regression Case

    -XGBoost or Extreme Gradient -Boosting, is an optimized distributed gradient boosting library -designed to be highly efficient, flexible and portable. It implements -machine learning algorithms under the Gradient Boosting -framework. XGBoost provides a parallel tree boosting that solve many -data science problems in a fast and accurate way. See the article by Chen and Guestrin. -

    -The authors design and build a highly scalable end-to-end tree -boosting system. It has a theoretically justified weighted quantile -sketch for efficient proposal calculation. It introduces a novel sparsity-aware algorithm for parallel tree learning and an effective cache-aware block structure for out-of-core tree learning. + +

    import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+import xgboost as xgb
+from sklearn.preprocessing import StandardScaler
+import scikitplot as skplt
+from sklearn.metrics import mean_squared_error
 
-
    
-It is now the algorithm which wins essentially all ML competitions!!!
+n = 100
+maxdegree = 6
 
+# Make data set.
+x = np.linspace(-3, 3, n).reshape(-1, 1)
+y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
+
+error = np.zeros(maxdegree)
+bias = np.zeros(maxdegree)
+variance = np.zeros(maxdegree)
+polydegree = np.zeros(maxdegree)
+X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+for degree in range(maxdegree):
+    model =  xgb.XGBRegressor(objective ='reg:squarederror', colsaobjective ='reg:squarederror', colsample_bytree = 0.3, learning_rate = 0.1,max_depth = degree, alpha = 10, n_estimators = 200)
+
+    model.fit(X_train_scaled,y_train)
+    y_pred = model.predict(X_test_scaled)
+    polydegree[degree] = degree
+    error[degree] = np.mean( np.mean((y_test - y_pred)**2) )
+    bias[degree] = np.mean( (y_test - np.mean(y_pred))**2 )
+    variance[degree] = np.mean( np.var(y_pred) )
+    print('Max depth:', degree)
+    print('Error:', error[degree])
+    print('Bias^2:', bias[degree])
+    print('Var:', variance[degree])
+    print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))
+
+plt.xlim(1,maxdegree-1)
+plt.plot(polydegree, error, label='Error')
+plt.plot(polydegree, bias, label='bias')
+plt.plot(polydegree, variance, label='Variance')
+plt.legend()
+plt.show()
+

    @@ -303,7 +335,6 @@ It is now the algorithm which wins essentially all ML competitions!!!

  • 60
  • 61
  • 62
    • -
  • 63
  • »
    • diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html index 7b1ab4223..39a7c6c43 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html @@ -128,32 +128,31 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -239,20 +238,19 @@ MathJax.Hub.Config({
  • What is boosting? Additive Modelling/Iterative Fitting
  • Iterative Fitting, Regression and Squared-error Cost Function
  • Squared Error Exampe and Iterative Fitting
    • -
  • Finding the Optimal Parameters
    • -
  • Iterative Fitting, Classification, AdaBoost
    • -
  • Adaptive Boosting, AdaBoost
    • -
  • Building up AdaBoost
    • -
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • -
  • Basic Steps of AdaBoost
    • -
  • AdaBoost Examples
    • -
  • Gradient boosting: Basics
    • -
  • Gradient Boosting, algorithm
    • -
  • Gradient Boosting, Examples of Regression
    • -
  • Gradient Boosting, Classification Example
    • -
  • XGBoost: Extreme Gradient Boosting
    • -
  • Regression Case
    • -
  • Xgboost on the Cancer Data
    • +
  • Iterative Fitting, Classification, AdaBoost
    • +
  • Adaptive Boosting, AdaBoost
    • +
  • Building up AdaBoost
    • +
  • Adaptive boosting: AdaBoost, Basic Algorithm
    • +
  • Basic Steps of AdaBoost
    • +
  • AdaBoost Examples
    • +
  • Gradient boosting: Basics
    • +
  • Gradient Boosting, algorithm
    • +
  • Gradient Boosting, Examples of Regression
    • +
  • Gradient Boosting, Classification Example
    • +
  • XGBoost: Extreme Gradient Boosting
    • +
  • Regression Case
    • +
  • Xgboost on the Cancer Data
    • @@ -311,7 +309,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
    • -
  • 63
    • +
  • 62
  • »
    • diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html index 31cd8c807..5c102282c 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-reveal.html @@ -2071,53 +2071,59 @@ at the internal nodes, and the predictions at the terminal nodes. To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.

    -For simplicity we assume also that our functions \( b(x;\gamma)=\gamma_0+\gamma_1 x \), with \( \gamma_0 \) and \( \gamma_1 \) as the parameters to be determined. +For simplicity we assume also that our functions \( b(x;\gamma)=1+\gamma x \).

    This means that for every iteration, we need to optmize

     
    $$ -(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\gambda}\hspace{0.2cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(\gamma_0+\gamma_1 x_i))^2. -\[ +(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2. $$

     

    -We start our iteration by simply setting \( \f_0(x)=0 \). +We start our iteration by simply setting \( f_0(x)=0 \). Taking the derivatives with respect to \( \beta \) and \( \gamma \) we obtain

     
    $$ -\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(\gamma_0+\gamma_1 x_i)(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0, $$

     
    and

     
    $$ -\frac{\partial {\cal C}}{\partial \gamma_0} =-2\sum_{i}\beta(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. $$

     
    -and +We can then rewrite these equations as (defining \( w_i=1+\gamma x_i) \)

     
    $$ -\frac{\partial {\cal C}}{\partial \gamma_1} = =-2\sum_{i}\beta x_i(y_i-\beta(\gamma_0+\gamma_1 x_i))=0. +\gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0, $$

     
    - - -

    -

    Finding the Optimal Parameters

    +which gives us \( \beta = \boldsymbol{w}^T\boldsymbol{y}/(\boldsymbol{w}^T\boldsymbol{w}) \). Similarly we have +

     
    +$$ +\beta\gamma \boldsymbol{x}^T(\boldsymbol{y}-\beta(1+\gamma \boldsymbol{x}))=0, +$$ +

     

    -With these equations we can then in turn find the parameters \( \beta_1 \) and \( \gamma_0^{1} \) and \( \gamma_1^1 \) as +which leads \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\em{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting +for \( \beta \) gives us an equation for \( \gamma \). + +

    +The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as +\( f_1(x) = \beta_1(1+\gamma_1x) \). Doing this \( M \) times results in our final estimate for the function \( f \).

    -

    Iterative Fitting, Classification, AdaBoost

    +

    Iterative Fitting, Classification, AdaBoost

    Let us consider a binary classification problem with two outcomes \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of @@ -2158,7 +2164,7 @@ $$

    -

    Adaptive Boosting, AdaBoost

    +

    Adaptive Boosting, AdaBoost

    In our iterative procedure we define thus @@ -2192,7 +2198,7 @@ where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).

    -

    Building up AdaBoost

    +

    Building up AdaBoost

    First, for any \( \beta > 0 \), we optimize \( G \) by setting @@ -2250,7 +2256,7 @@ $$

    -

    Adaptive boosting: AdaBoost, Basic Algorithm

    +

    Adaptive boosting: AdaBoost, Basic Algorithm

    The algorithm here is rather straightforward. Assume that our weak @@ -2274,7 +2280,7 @@ where the function \( I() \) is one if we misclassify and zero if we classify co

    -

    Basic Steps of AdaBoost

    +

    Basic Steps of AdaBoost

    With the above definitions we are now ready to set up the algorithm for AdaBoost. @@ -2315,7 +2321,7 @@ observations that are missed in the previous iterations.

    -

    AdaBoost Examples

    +

    AdaBoost Examples

    Using Scikit-Learn it is easy to appply the adaptive boosting algorithm, as done here. @@ -2349,7 +2355,7 @@ plt.show()

    -

    Gradient boosting: Basics

    +

    Gradient boosting: Basics

    Gradient boosting is again a similar technique to Adapative boosting, @@ -2367,7 +2373,7 @@ See discussion during lecture November 8.

    -

    Gradient Boosting, algorithm

    +

    Gradient Boosting, algorithm

    Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function @@ -2395,7 +2401,7 @@ The way we proceed in an iterative fashion is to

    -

    Gradient Boosting, Examples of Regression

    +

    Gradient Boosting, Examples of Regression

    @@ -2450,7 +2456,7 @@ plt.show()

    -

    Gradient Boosting, Classification Example

    +

    Gradient Boosting, Classification Example

    @@ -2499,7 +2505,7 @@ plt.show()

    -

    XGBoost: Extreme Gradient Boosting

    +

    XGBoost: Extreme Gradient Boosting

    XGBoost or Extreme Gradient @@ -2520,7 +2526,7 @@ It is now the algorithm which wins essentially all ML competitions!!!

    -

    Regression Case

    +

    Regression Case

    @@ -2576,7 +2582,7 @@ plt.show()

    -

    Xgboost on the Cancer Data

    +

    Xgboost on the Cancer Data

    As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now. diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html index e0c9995ac..acfe40fa5 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-solarized.html @@ -148,32 +148,31 @@ div { text-align: justify; text-justify: inter-word; } 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -2074,45 +2073,49 @@ at the internal nodes, and the predictions at the terminal nodes. To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.

    -For simplicity we assume also that our functions \( b(x;\gamma)=\gamma_0+\gamma_1 x \), with \( \gamma_0 \) and \( \gamma_1 \) as the parameters to be determined. +For simplicity we assume also that our functions \( b(x;\gamma)=1+\gamma x \).

    This means that for every iteration, we need to optmize $$ -(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\gambda}\hspace{0.2cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(\gamma_0+\gamma_1 x_i))^2. -\[ +(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2. $$

    -We start our iteration by simply setting \( \f_0(x)=0 \). +We start our iteration by simply setting \( f_0(x)=0 \). Taking the derivatives with respect to \( \beta \) and \( \gamma \) we obtain $$ -\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(\gamma_0+\gamma_1 x_i)(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0, $$ and $$ -\frac{\partial {\cal C}}{\partial \gamma_0} =-2\sum_{i}\beta(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. $$ -and +We can then rewrite these equations as (defining \( w_i=1+\gamma x_i) \) $$ -\frac{\partial {\cal C}}{\partial \gamma_1} = =-2\sum_{i}\beta x_i(y_i-\beta(\gamma_0+\gamma_1 x_i))=0. +\gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0, $$ +which gives us \( \beta = \boldsymbol{w}^T\boldsymbol{y}/(\boldsymbol{w}^T\boldsymbol{w}) \). Similarly we have +$$ +\beta\gamma \boldsymbol{x}^T(\boldsymbol{y}-\beta(1+\gamma \boldsymbol{x}))=0, +$$ + +

    +which leads \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\em{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting +for \( \beta \) gives us an equation for \( \gamma \). + +

    +The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as +\( f_1(x) = \beta_1(1+\gamma_1x) \). Doing this \( M \) times results in our final estimate for the function \( f \). +











    -

    Finding the Optimal Parameters

    - -

    -With these equations we can then in turn find the parameters \( \beta_1 \) and \( \gamma_0^{1} \) and \( \gamma_1^1 \) as - -

    -









    - -

    Iterative Fitting, Classification, AdaBoost

    +

    Iterative Fitting, Classification, AdaBoost

    Let us consider a binary classification problem with two outcomes \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of @@ -2147,7 +2150,7 @@ $$











    -

    Adaptive Boosting, AdaBoost

    +

    Adaptive Boosting, AdaBoost

    In our iterative procedure we define thus @@ -2175,7 +2178,7 @@ where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).











    -

    Building up AdaBoost

    +

    Building up AdaBoost

    First, for any \( \beta > 0 \), we optimize \( G \) by setting @@ -2218,7 +2221,7 @@ $$









    -

    Adaptive boosting: AdaBoost, Basic Algorithm

    +

    Adaptive boosting: AdaBoost, Basic Algorithm

    The algorithm here is rather straightforward. Assume that our weak @@ -2240,7 +2243,7 @@ where the function \( I() \) is one if we misclassify and zero if we classify co











    -

    Basic Steps of AdaBoost

    +

    Basic Steps of AdaBoost

    With the above definitions we are now ready to set up the algorithm for AdaBoost. @@ -2280,7 +2283,7 @@ observations that are missed in the previous iterations.











    -

    AdaBoost Examples

    +

    AdaBoost Examples

    Using Scikit-Learn it is easy to appply the adaptive boosting algorithm, as done here. @@ -2313,7 +2316,7 @@ plt.show()











    -

    Gradient boosting: Basics

    +

    Gradient boosting: Basics

    Gradient boosting is again a similar technique to Adapative boosting, @@ -2331,7 +2334,7 @@ See discussion during lecture November 8.











    -

    Gradient Boosting, algorithm

    +

    Gradient Boosting, algorithm

    Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function @@ -2357,7 +2360,7 @@ The way we proceed in an iterative fashion is to









    -

    Gradient Boosting, Examples of Regression

    +

    Gradient Boosting, Examples of Regression

    @@ -2411,7 +2414,7 @@ plt.show()











    -

    Gradient Boosting, Classification Example

    +

    Gradient Boosting, Classification Example

    @@ -2459,7 +2462,7 @@ plt.show()











    -

    XGBoost: Extreme Gradient Boosting

    +

    XGBoost: Extreme Gradient Boosting

    XGBoost or Extreme Gradient @@ -2480,7 +2483,7 @@ It is now the algorithm which wins essentially all ML competitions!!!











    -

    Regression Case

    +

    Regression Case

    @@ -2535,7 +2538,7 @@ plt.show()











    -

    Xgboost on the Cancer Data

    +

    Xgboost on the Cancer Data

    As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now. diff --git a/doc/pub/DecisionTrees/html/DecisionTrees.html b/doc/pub/DecisionTrees/html/DecisionTrees.html index d038bafd1..1c1b06da5 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees.html @@ -153,32 +153,31 @@ div { text-align: justify; text-justify: inter-word; } 2, None, '___sec47'), - ('Finding the Optimal Parameters', 2, None, '___sec48'), ('Iterative Fitting, Classification, AdaBoost', 2, None, - '___sec49'), - ('Adaptive Boosting, AdaBoost', 2, None, '___sec50'), - ('Building up AdaBoost', 2, None, '___sec51'), + '___sec48'), + ('Adaptive Boosting, AdaBoost', 2, None, '___sec49'), + ('Building up AdaBoost', 2, None, '___sec50'), ('Adaptive boosting: AdaBoost, Basic Algorithm', 2, None, - '___sec52'), - ('Basic Steps of AdaBoost', 2, None, '___sec53'), - ('AdaBoost Examples', 2, None, '___sec54'), - ('Gradient boosting: Basics', 2, None, '___sec55'), - ('Gradient Boosting, algorithm', 2, None, '___sec56'), + '___sec51'), + ('Basic Steps of AdaBoost', 2, None, '___sec52'), + ('AdaBoost Examples', 2, None, '___sec53'), + ('Gradient boosting: Basics', 2, None, '___sec54'), + ('Gradient Boosting, algorithm', 2, None, '___sec55'), ('Gradient Boosting, Examples of Regression', 2, None, - '___sec57'), + '___sec56'), ('Gradient Boosting, Classification Example', 2, None, - '___sec58'), - ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec59'), - ('Regression Case', 2, None, '___sec60'), - ('Xgboost on the Cancer Data', 2, None, '___sec61')]} + '___sec57'), + ('XGBoost: Extreme Gradient Boosting', 2, None, '___sec58'), + ('Regression Case', 2, None, '___sec59'), + ('Xgboost on the Cancer Data', 2, None, '___sec60')]} end of tocinfo --> @@ -2079,45 +2078,49 @@ at the internal nodes, and the predictions at the terminal nodes. To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.

    -For simplicity we assume also that our functions \( b(x;\gamma)=\gamma_0+\gamma_1 x \), with \( \gamma_0 \) and \( \gamma_1 \) as the parameters to be determined. +For simplicity we assume also that our functions \( b(x;\gamma)=1+\gamma x \).

    This means that for every iteration, we need to optmize $$ -(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\gambda}\hspace{0.2cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(\gamma_0+\gamma_1 x_i))^2. -\[ +(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2. $$

    -We start our iteration by simply setting \( \f_0(x)=0 \). +We start our iteration by simply setting \( f_0(x)=0 \). Taking the derivatives with respect to \( \beta \) and \( \gamma \) we obtain $$ -\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(\gamma_0+\gamma_1 x_i)(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0, $$ and $$ -\frac{\partial {\cal C}}{\partial \gamma_0} =-2\sum_{i}\beta(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. $$ -and +We can then rewrite these equations as (defining \( w_i=1+\gamma x_i) \) $$ -\frac{\partial {\cal C}}{\partial \gamma_1} = =-2\sum_{i}\beta x_i(y_i-\beta(\gamma_0+\gamma_1 x_i))=0. +\gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0, $$ +which gives us \( \beta = \boldsymbol{w}^T\boldsymbol{y}/(\boldsymbol{w}^T\boldsymbol{w}) \). Similarly we have +$$ +\beta\gamma \boldsymbol{x}^T(\boldsymbol{y}-\beta(1+\gamma \boldsymbol{x}))=0, +$$ + +

    +which leads \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\em{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting +for \( \beta \) gives us an equation for \( \gamma \). + +

    +The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as +\( f_1(x) = \beta_1(1+\gamma_1x) \). Doing this \( M \) times results in our final estimate for the function \( f \). +











    -

    Finding the Optimal Parameters

    - -

    -With these equations we can then in turn find the parameters \( \beta_1 \) and \( \gamma_0^{1} \) and \( \gamma_1^1 \) as - -

    -









    - -

    Iterative Fitting, Classification, AdaBoost

    +

    Iterative Fitting, Classification, AdaBoost

    Let us consider a binary classification problem with two outcomes \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of @@ -2152,7 +2155,7 @@ $$











    -

    Adaptive Boosting, AdaBoost

    +

    Adaptive Boosting, AdaBoost

    In our iterative procedure we define thus @@ -2180,7 +2183,7 @@ where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).











    -

    Building up AdaBoost

    +

    Building up AdaBoost

    First, for any \( \beta > 0 \), we optimize \( G \) by setting @@ -2223,7 +2226,7 @@ $$









    -

    Adaptive boosting: AdaBoost, Basic Algorithm

    +

    Adaptive boosting: AdaBoost, Basic Algorithm

    The algorithm here is rather straightforward. Assume that our weak @@ -2245,7 +2248,7 @@ where the function \( I() \) is one if we misclassify and zero if we classify co











    -

    Basic Steps of AdaBoost

    +

    Basic Steps of AdaBoost

    With the above definitions we are now ready to set up the algorithm for AdaBoost. @@ -2285,7 +2288,7 @@ observations that are missed in the previous iterations.











    -

    AdaBoost Examples

    +

    AdaBoost Examples

    Using Scikit-Learn it is easy to appply the adaptive boosting algorithm, as done here. @@ -2318,7 +2321,7 @@ plt.show()











    -

    Gradient boosting: Basics

    +

    Gradient boosting: Basics

    Gradient boosting is again a similar technique to Adapative boosting, @@ -2336,7 +2339,7 @@ See discussion during lecture November 8.











    -

    Gradient Boosting, algorithm

    +

    Gradient Boosting, algorithm

    Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard square-error function @@ -2362,7 +2365,7 @@ The way we proceed in an iterative fashion is to









    -

    Gradient Boosting, Examples of Regression

    +

    Gradient Boosting, Examples of Regression

    @@ -2416,7 +2419,7 @@ plt.show()











    -

    Gradient Boosting, Classification Example

    +

    Gradient Boosting, Classification Example

    @@ -2464,7 +2467,7 @@ plt.show()











    -

    XGBoost: Extreme Gradient Boosting

    +

    XGBoost: Extreme Gradient Boosting

    XGBoost or Extreme Gradient @@ -2485,7 +2488,7 @@ It is now the algorithm which wins essentially all ML competitions!!!











    -

    Regression Case

    +

    Regression Case

    @@ -2540,7 +2543,7 @@ plt.show()











    -

    Xgboost on the Cancer Data

    +

    Xgboost on the Cancer Data

    As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now. diff --git a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb index 752b3f244..a1baa3fae 100644 --- a/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb +++ b/doc/pub/DecisionTrees/ipynb/DecisionTrees.ipynb @@ -2121,7 +2121,7 @@ "\n", "To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.\n", "\n", - "For simplicity we assume also that our functions $b(x;\\gamma)=\\gamma_0+\\gamma_1 x$, with $\\gamma_0$ and $\\gamma_1$ as the parameters to be determined.\n", + "For simplicity we assume also that our functions $b(x;\\gamma)=1+\\gamma x$. \n", "\n", "This means that for every iteration, we need to optmize" ] @@ -2131,7 +2131,7 @@ "metadata": {}, "source": [ "$$\n", - "(\\beta_m,\\gamma_m) \\mathrm{argmin}_{\\beta,\\gambda}\\hspace{0.2cm} \\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta b(x;\\gamma))^2\\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta(\\gamma_0+\\gamma_1 x_i))^2.\n", + "(\\beta_m,\\gamma_m) \\mathrm{argmin}_{\\beta,\\lambda}\\hspace{0.1cm} \\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta b(x;\\gamma))^2=\\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta(1+\\gamma x_i))^2.\n", "$$" ] }, @@ -2139,7 +2139,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We start our iteration by simply setting $\\f_0(x)=0$. \n", + "We start our iteration by simply setting $f_0(x)=0$. \n", "Taking the derivatives with respect to $\\beta$ and $\\gamma$ we obtain" ] }, @@ -2148,7 +2148,7 @@ "metadata": {}, "source": [ "$$\n", - "\\frac{\\partial {\\cal C}}{\\partial \\beta} = -2\\sum_{i}(\\gamma_0+\\gamma_1 x_i)(y_i-\\beta(\\gamma_0+\\gamma_1 x_i))=0,\n", + "\\frac{\\partial {\\cal C}}{\\partial \\beta} = -2\\sum_{i}(1+\\gamma x_i)(y_i-\\beta(1+\\gamma x_i))=0,\n", "$$" ] }, @@ -2164,7 +2164,7 @@ "metadata": {}, "source": [ "$$\n", - "\\frac{\\partial {\\cal C}}{\\partial \\gamma_0} =-2\\sum_{i}\\beta(y_i-\\beta(\\gamma_0+\\gamma_1 x_i))=0,\n", + "\\frac{\\partial {\\cal C}}{\\partial \\gamma} =-2\\sum_{i}\\beta x_i(y_i-\\beta(1+\\gamma x_i))=0.\n", "$$" ] }, @@ -2172,7 +2172,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "and" + "We can then rewrite these equations as (defining $w_i=1+\\gamma x_i)$" ] }, { @@ -2180,7 +2180,7 @@ "metadata": {}, "source": [ "$$\n", - "\\frac{\\partial {\\cal C}}{\\partial \\gamma_1} = =-2\\sum_{i}\\beta x_i(y_i-\\beta(\\gamma_0+\\gamma_1 x_i))=0.\n", + "\\gamma \\boldsymbol{w}^T(\\boldsymbol{y}-\\beta\\gamma \\boldsymbol{w})=0,\n", "$$" ] }, @@ -2188,9 +2188,27 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Finding the Optimal Parameters\n", + "which gives us $\\beta = \\boldsymbol{w}^T\\boldsymbol{y}/(\\boldsymbol{w}^T\\boldsymbol{w})$. Similarly we have" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\beta\\gamma \\boldsymbol{x}^T(\\boldsymbol{y}-\\beta(1+\\gamma \\boldsymbol{x}))=0,\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "which leads $\\gamma =(\\boldsymbol{x}^T\\boldsymbol{y}-\\beta\\boldsymbol{x}^T\\em{e})/(\\beta\\boldsymbol{x}^T\\boldsymbol{x})$. Inserting\n", + "for $\\beta$ gives us an equation for $\\gamma$.\n", "\n", - "With these equations we can then in turn find the parameters $\\beta_1$ and $\\gamma_0^{1}$ and $\\gamma_1^1$ as\n", + "The solution to these two equations gives us in turn $\\beta_1$ and $\\gamma_1$ leading to the new expression for $f_1(x)$ as\n", + "$f_1(x) = \\beta_1(1+\\gamma_1x)$. Doing this $M$ times results in our final estimate for the function $f$. \n", "\n", "\n", "\n", diff --git a/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz b/doc/pub/DecisionTrees/ipynb/ipynb-DecisionTrees-src.tar.gz index d5bddb5be43f0d564e5674b233dd4fc570a3ca48..2a8f44d75cb205f2c21babd75ba5699cea908703 100644 GIT binary patch delta 30 mcmZ4cQ*iB1L3a6W4u<_~CmPvX*%@2enOfPIx3aS=s{sJ3tO>~g delta 30 mcmZ4cQ*iB1L3a6W4hF;L#~ayO*%@2enOfPIx3aS=s{sJ8;|d!9 diff --git a/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf b/doc/pub/DecisionTrees/pdf/DecisionTrees-minted.pdf index 7ab42a65921393ea5bd37192d61ecfdb94ff1ff7..bfe73b849079c96ba2037aa1e7520dbfc4fb2397 100644 GIT binary patch delta 48770 zcmZUZQ*fXS(4}K@Vw)4&w(U%8+j(Q#PA0Z(+cqYeiEZq+|JGjpRoypz-Ccd2I_EZw zaLK2rlr{1@9rIj6+~N%{m~J+ z;ZTP$@gx*7d2!_}ZK}~Kw9ALGVbsQOUh0N6^m%Wx(D^%lWGNZ1$EKGh-nErR_gmv| zw0cUgHga-u=4*oAl1P@b&}s@O8`a<4x(ov}cCHZInn<)>oj%k2#E458w)*S1&Frsl zI6^1hWfc?6Lbuj;m0V?=!L#!29aX(&hmJpu+N`TSkvcYT7Gvns6yg-juWK=8IuGPq z{5I99D%pz#+b!noAD=ocvxZu`cG=$T-Qv>ewuow-G`i01&qk$_%7uU=2o2H@2#t02 z?N<^u>_L>oLcANB*?S0_s1(zkjC2UpxR5L8ZE|X+;r-rx$a`|sNa^|?wqga1Z6}iE zCvo!^4i(4DVhs`Hj2(Y5ajQ5a=uUT(sA?#bg@lr{eznD`AFlV(N9(hu^!riC)HSrE z7ix}tUj5{};#SJj->?D{x}KeVt=ljn?&eqq;?2q=)Pj+wv|(=pz^CIE-~*eBC*fi7 z$A55vG&X{_!OHw_s!unZpLzTNj{`?fUWuw{Rp=U3j_RN+Xj`QpL^+;pL@a}H&X>2d zm~?n?vXS!9OMDu;a`XkNDFo;ehm5ZF*{jQq(hPPb0NhqDj>W7t;;W&A{DN_1)J zx#Tkv?Y@Wi2UnF6bkd|Kv5t8QDRHD?7Ds)Ah+!Re8+=Bqsd>7QLd#BwXfB0Dmc|h; z4)5bq>2HLrvA+N}QWnq{oR-e>mkgYApkszPq`?ARx)TCyO9!fGmy#U3>vX@LxgAWQ zEH=k4gy)k8n_F;g{52=BsVPRUH!X3>B}UI*8qcE6X^U+~&dWONpf7v1XRAIh^y*q*|B=HY(nUKmFmPi@zS zYd%`}lAE6C6#zsfBzqw!Xsj2Szf6>c3Q^d~Ay7)J$wEm@m3oa_3viO^+lvuPSCOpz zNg&@yfd}^OT#{UbXU7MGjpt!y?>nrC2p(#tMvDV)Rx2+DZfI;mM05^S&m-LnQC0nf zSZFMQMm9y>llwUel9&9aQ5(Po`oJA`G5KW>X@&@^!wrm;O#h^QUBem-khon{05(v1 zejYxLUzLYgpxGtdznE(1{_5J*`M=T|R3FR{kpSyTHJEeme^-YG_&DwDn_m8%3HJU% zq4Ld33mO^d{Va@W!@d>D7%Ext6hGv`zZ@7fN_6yerL#>C3xVW>$^of>elni!mu73sDbuM8HD-?td8=O#5LqT^5_6 zfWr*h4edV66})V|7{>gA;~A&Lh}dkO&ICcaf4l+lz8TRafXrYz?3QZ1K1x#zf(W}| zE>uEI_BOt0N>M)7V1jbOX6p#c$ufQOC>to2caDm=F{HNGB?mafl?@1VF}iwTy3Xqd zT@VxIMvQ|6nS^*fGB+^5T!5+GsGv8VvQu(IXjXcPYm`NuY}aiqm4^FxJ~9@C|Kx#%C28tiYcC37SfqSsheS49lmXVdc2S!YznzZ18<;r1h-p$~gw|8}R%54{xsJv9 z7jVVEFs0@iRo^t%t$3jPePMELEf^;fuE0&Zl#QL zW|sd**j}cxk~4uog;Gt{1E0i|vzr2Um0Olng3tWepK%g@*D_#MD#Sn3rU3p#PWBr% zf=I^FXHuo{x#|T|q_cLX{lFd#Bb6CF%A0^wVi^#9m76Ovw0e!CmBY-8m{n9eRoCVu zyG;K_ubey#!#>){JXBcPZ<2=AYfjKO?!V4%b~*RjaE_NzK0pH&p%Bwx>FOVgO0YRD zYJk;*wsi$X#|rOw&^A;%p#VUtW;U1gw3252Vv!{0;g?xyCapU~>Qw!Lku9z_c&e$n zLCs@-pBd29KA0ucK!fkc6YPJY7YbNvMTQ^@am2S5h3c>Qr*V_6F8QEzU0O_ns8e|R z7UW5?c)H2QBZZxcqZN;P;aBYacMpWlzJyl~Ns$5C#cjeF8>-(3UkH%eh5#??=z(aL z59}tswE9)dSuZxyvwJaVv4*4C%eP{-^b*CL-vA2slSx}%?7X{@OF`G8PY&!LNmvIX zG4EcBO>!8b`zANNBa%AYY%)1gBE->m&Q!{`{@Iv?&;_X!AZ$7^n`SxwNx^8620~wW zCSoofCM81qghFy%qy+Hf>?iLSSQiwX7jh$*U!;Iy!Z(s*@)2I5h=t4eR@6|=wgw5i z*p{H*_%bl83Zsc}5slA7vF3idTq_zwvdUDg(~TH0kZWsW-}# z>-H|YZMmK?1N~rwbUGf_F2QSR*SouF02({5_T!BH%?r^n~2k4wo!S!l)@k?Y@&ESrmr7~}zji&9Ylvkh3&rqXQcnLqZ zwd%7~jsrRf5=b^t{ys#ekfVtSeG=ANbrn=GiHoAJ3WkRRLw+QMws+?)ImtH4@^j%Kia9y8XHP1&S&Uco|*LXb{Jha^={< zvf|IkfojjewL|uN5)8>^XSutdDxhkc z&*5*m0%0Ln7T(UpsVPALnFOL;#>*U0!xvW8uAFdg6)1bKo*sYJS@koE9~_L?5sK18 zm>REITBNnEqJ`eYo}+>C%6W3ZCnZ>=vvAOLHi2-G)rUT<&#$|Gi^%&RPHbF0c#Xv6pMDkas&wE1zw4_Gb}oj%WykC0t_woq<{4(e zcnjtG02|Nk=r5Xmt&=(bvU%6)Y#kg$ylv;^BqDqrKi`4kqy4~^ZG73;AsA>Ezc~3G z^E;IGn1#*LIW^tcq?v8DXbAG2w>&onNV|TXE#~28p8qYPHU1%>%M%jLPKQ?T&;Q(> z2#Mo23lS4e4roPY-2q}>WMe$OWbr|S2{dXi;o8i7Wr&iBxRUV!`O~= zv2#Hfo5|k&W z#^>_ylxLF=@VD%fy;kR?P zK`L{sqYv9+fvE~pIfuT@FsyyH_WN>>}dp|2GkA5y-t)~tF)x^_7jJT@m6F%z+au{9hY9~`5cnZ1RpB{2&#=YNQV z1VIDn$i^Rvq4ZwY-7KJH@A?Z0MGup@gAUk{Cj@-opz}e@(x$KP9ri;#hN2wl@d#K3~r0+sJ65 z@aBJOe^Vl$_K-wz{o4fXQxzwhf-#P@9{a%27vtT~4S8h-k47}A)h%B4+&TAcv%lizG@HVOU73U^K@_%2@y= zx`>?4){rid;bmYB6@$J(g|bIDy~%AhR^C98?J$LD>c?oLnsEWHt=F8~0yfv_u=Z*A zC$2VPoQgB{2#kf17HZhPUL|Ag(}X(h=_+BObS8802dSOt|sW~D-Og;paN_?unp;SRmU_z2nT5xIX%M82U(I7`<+o#ho#2T2_r7o0P ze{e7lDv7q$#d>}TiL+OGD-BUhOH;RACP5kEn(7{Qb!tshTao7RPT5 z4fR`p3@E+y%Xt0Too;@*Xq%V;irzcdREy}z@1Qh`s>%y^n4cGuEdVh2QGM>*)015U zTFzp_jv0i#E^o!99On~zj$md{)l*1GPm&rIUqFOnckg9zguEMiQ-kEaTr;w_p8^Rs zlg*NWhdHj%c3fMYE<9Fx(5AU@_4xgbUqeae}L@yZwri- z7(xp|0LvR|c#w2B8*sE086?CvyIh>(oQI5Rgz|7$ZCMXOX$?I^i%6NDW(rMhvlxcL znGK^N{iodXIPGB4>xS{~0rI9*AX zyoXk27-BxqdqikcL(FApm4{T%dfUF6nX4zKGpD8{aMecl-K}ecvU5$d*INX$KjjEe zWF3A9qac@>Wsi_Z2caE4I8Jtv8Iu=q63leac+G05Plrz6YqGKo+ow`F>t*{7yp-ua zM3**$6csaX>ZcqUY_ND-5htQi+2Nr7B3G|e`{$~2Kc$C+qL+{uCj1f9@1$uE(6Xpn zf&LyTv`zBIZ@kTCw|)#b(tBg9B+(HSLPs4HiziCvRx8-2aY2M^%L{XW?ynKXs2B-NhVh__ z#BODd0;t7XwhI|1yvv;1c&5F*w;}jKAOFM(RYV5R+0PC@Yj45mNnF#V;q-1x;-V}2 z7dEFeC~_gENCa7y{<#{e$lZmGfRPsuN>#hbpPz=eFD~1b5^}`;iroGyl~*C4^IwA- z5a?4py5|i0cr9~`P^Lv^L0NjaXjWs6oUA?r=U1(YS6e;w0hk71=WQ(8LuT^A_{X>s zTKNqqg!QIhJSSBFZjD3GxC@Y4gYGy_Qw&d?_6LC}Xc!wV8g%z2T!m=ggy1j1$X0Y0 z_qgx54cQFzFhf_&9wFQwkqSv(*0C-Uj70aDo{R=)635cwuYK9|$4-N3sv&lz0|@be zm(aSP7;?CkyTYf3#&1Co=W>~v3*MACb=;j=aqcMUr@xdsV+{Oqcp%9fR#28LM9;8i zo6=!xYN1sEt1P~C6Eq@{O05LKiNEStsO=NB@AZA&_Z>#WI~h0-(bX@Jy@+a~wkR0% z+U)v#+iV(K(^p86`_Ee)quY0mqrswp2*lm~ktd8FXrmIdEcHsli*wM!peqtfjRCn5 zPcUMj7~Pf!4?H&6DP}gWF>YJ+l2Qw5QsT(O&?7uA>aK1`)`ZpcE0{3?t!X>B|PJu88@>ObYD2XMZRNDJg0XhX3rR4Tx|7ym1Eu zWSPA7ki7|dYE8orQIiLyGP#Cdq4vTcSQa%8cAh)n6hjKgN7<@L5qQ%sMr|lOnl-=* zz&x&mud#Y{v1l^GzQ8^Pw@dIx^%jk^EDRU1seed7)YDmK1l)+X8WCik5j4B}bN)H1^zU0Z4q;U6LiL2X2g6Yh zO~x;%g+xF4bmej|G;A)m|1DZrnVFc=ZwtVv)Az|BAb=f?AO6pE^F=Wy)PMAC6T;Eg zA?WMZ8E^|A7x~5o8tE@OJSlS5uj=+3q-4@e^PogW>l4~C4B%Qmy%V9~y|%L$Zta6TS$GcIP|hoX?`s~uEvqNl%IUNyYaYPYj8d@cGAzJ` zB#L$y%R?Hbnc|M+{+ciQJ;ldceDj;E8|KxA=t41w6_!lO{UT34Z&J*^H;p6$r0Q$w zlXy8@bH}GK+3Zq3T%Q~mvtjXuR75NaVaPiUhXK=2Cc|QsL65rDwLEd95*k%ZgFY?`165wOmRhM&`-?O6bofBRJ!c)A+&Ky|B^$(MX{cg zy^Gk}$1{d$r1OqDJ#kJybq>*Ba^{*7f^oSB2`2CatqZT<6?n*RXPUYVZmvqLY3t5C zih<`;&8O_7uUD3bh~|p4mAa0EZ0mo#o!LySJQh|`nLWs#7@9R zq^tKsG1tDf_m8pTpF>9iK)r03^`qm%1^{?RZcYih5=j%+(T;y(^)NM%%TY8hLdn{A zo|R7pte z!rMo+bCdQd2d8h1`Jf*96^2tz&A?ihpIUR7<8_)SZ1?iD`f+dlr?fa5E)QRp$2|mC zQ5leOcf-q1u&BgdPN@f3DV)Tzdt-5vlrlvSh-_ZRvO9A(5@0yU-_>M`vw<`FoT`;I z&WFp*ate}-V9L12i)R9sLm7F<^0LixXvs(Uv5HU*Kv*#Oz2z>yq5Fnmhf8*&6ZWUxa(1KFO;f~o<-J`o#c_~)k~0#6Gm!QxmJR?TCNiTG5A5ep#^D|AgqVaT&y(|X34s}{92AR`<6uEi z=3BZ??vDF=lMx{ykTIu7G}Z+$1`oop+!j1N9`rCp`u1hac*$Z?FMXiULsBU^~3@|+h&W^nXKM!3v03%Pi0EXp!XIA{bZdgdz) z6jwEFv9FwO0i`;F6RgF0u|$7t4ANf&O*K^{kQ9&j4p$XmzkXo7^_LjZ9-5CH>kgt* ze!50r2c&Bl^IQu=A)qU@xVVs1Op#6fXl|acoVSu4K`_5o@%iJA?S==re{no+L25U| z>l!n>!WN-2OD8`2h%%6ZvjQdey?}hO_rB|3($iG)>EvZR`ZklcG*# zt|-;>i-`7EkDA9nW!aj{TXrOd=(FCio&97MffA0nr3!g!uUg=XLT~bNO&feDhi~lK zguLbQ0^C-tG5pstbgHXRF$zmCaJXb`ZZLJVk?H2^m)UK zB$$aAolQwxSz1ks;BCGyE3bVj)zoUR zfw-T)u8whkQV0o1p^hYP4Gzpg&%LHd3AarLyFL+c5XoBPi&`AQsTt62*$0VM)}36r z9|7L^XALwG-zQ+_?t5ym269T7A2p(@MvxB<*`?0fh4J6o&6ops_)+$L^#5&}GU7p5 zZKs{zQc01TMusNgQwo*@GfZxG0L3c|@W6U-2&ws7Z5+ehC$LT)HS1!(%Sy;`t%a9m zv*5f3FP1vHez-K=W*Lu9;8AtFkRg3vb!g>@z=`N!n>Jn@=`9@&tM)by`_6~rX_Fy( zg@9bYM(1xiE|wylB4ZHoLUp=2j!w;Yr;7VuqBouZ)0~Gm_h6B^PDAU#-zl%N;mIr`W1MQW z&g~ApYD-qv4H1W469OsBVe+tkWf}(&G~(eweV7LFCNby!e1gsF zZEE=Hx;S8r8JGG?k1rk$EN0b~=bzF*2plrysSE8pJY|CntmJ6g`C)1jAjttr_HU7r zGN$K@B#~CQ)vM?}M z(*9+`kH$UXrUb|3^M0rV7HxuTH3K2Q|E6V$d!YJ{ui=N^u_wn_a+ei$QB`TDXhwA& zbe`{aL~nBtW}vPH$ez?EFqp3KsWb08^VOcN!McLTNZ>RnlJzaP^O=Rr!=|cu#d*m} z$UPx=otVsn(6rpn^8!6UcZ*?^#sg_yQO0ER#W&x{G!zix+#f7@^7ra|WFTDu0d>&0 zVSD3v@1X)Mn>&kNG3&2B4s3ob@zs8~iPqX;FoMu7xby>GxVah=P-2k*T9^>ON=6;& zo-be2vB{&?(DR7H=G~gYZDm)IWr=@ zOpHDWe8AUxtX}FV!u)I@)F%g*bVI^P*KfDH?P-g(_-M4&Ju^a@zMPwDhl(7Oldg`} z6T`sQU^jvPt%b=AEF%%E3YJ8kF2V>wK?`Jl%?&+kJ@jVnjw_sWlj@Kzb0cBYXWnzp z6kDm=M_Z4o5(gd_#n`i6y}jB>Lb4}D!%X7e-`lm5Jk?x$=n=== zF<#Xtb^o#|vodqWqpK5sCv0JfWo9A-^?ede8QQ7*4_9Duv;J?cVBzLWzbOEt0d{n> z5bT7+!-*{;~Yex{{M&&HH#$dlYV{C_TBc z4>&Rlvwc+d@Kc{*)26bzJ0K%wsI&ljUd!+vONECr&rg z)k+~he&6pg0+(1+`B*IS7S~mufbRfz{-whtZvJp}J^1;V`|QJX+}(z=s;mAZ#c`qS zPSogZc&dE?ePw6m+SBFyWBg}K7q@Ctqt4#`>^lBWVfe z5%yW75a%>HYSfPUCR*N~2@q8HyU@u}7U}mhG<+d-EcH!O>A^ z*MfU{de+~BLr@-Bw95B{0X;2T3&@cCvVN6$$0-!wCBwNk9%-z99Ks7FvbTaW!Ns3Z zqo-I!ns)#67N5%FF7M4Jbj)&K)=<`*yyB zBcg%kj^#r9W@`$6;=pFQdh(8N5hovaWSX$CBDY0q-D`TK9V{w_Xnv8rS-*a;CG>&E zdp!;++0Wt&#|SFcA&7NhH_OSN{G>)-JG7$Hc8fwyvZ*c37cC? zNChGhibD7kcBLCNa_Z@Vp@qe=wxnZ*`Ro#+O?J(lQA~N_0aMb&!yKunf|p^JVXQ3l zC`JO9kz%+qrFB3kX(luZrrv_}`?`V2e5Kb=%+x*$oOHiL^yhRYz6I_&Ej(X1^YQ8Y zxHh~@Ai1MIB(`|?CXqdaW6DWS&FB#XiJB<_<%y2qYW9dpxAw<#w>IBs_Q-b(XY`n7 zlV>64P+^J}@B+h7DkQHG#B6LY@$%wdyHP!`Nt!WiE%G`;vQA|Kl zM)D6P@=waSE)YbQnW0##?#Z(?kZO>5)Y=Z3ghO5QK$FgX0GA?>C@{P2HU%k!(GqMQ z8Y=e<2%e0XtDL1Qkzc5NuY4;l|5RN{MIzfZoOG6XN^MpRp0K{TFK_>$)I0FEK4{4| z%p1%3X>({APcbOh_Jm=|d8LPoa-OCLECOlP0mr7cvk`Jhx#!=F8uUe}x0K6S46iOB zhtIqHChX0(rQ)-Ffz_6C$6ou|paYYw(mvN0Fr}Qr^v2a5sDOEi%6B<=Ow#jgw`_S_u0XBDT(fNa(xMEPLq0 zwaN7(+uKTuof39dbJIz&nw5r?ZBHP(UTLis zpm(S`*^jxzo{U7p#ut-j0F}%jhjB$6(Kq(uKXq-T-7QcDX(7m~Il9<+HtyO=*?C>x zT4L;%FiJAo&o3o0Q#40aH@cS#tPB#*Pg_~D?t7`B+k6Z3L|%?wNG$D0GfR;9SKSnE zw(y(8ViA`w2EA*?{%|bjucM>-#nWs*a0w$}X{M+4o6FV((ZYUJ9$wq&eNd)6Lx}>~ z2CTsBuMT^!qBHo{3$~lDWwbh^J7lM0R(oUA?HQ=|uUSMeK-G1nk=Nl*9&O&1%%;-u zTtIykj#xwHj0^{zSDw?hk7xr!6E(k6=IJ!$g+YO03~kZjEp z^4~xT$QDVT8|mtKsjrxELfW)1_E(%Qy3H8F;^6s-nO01_k7)W96}9iUyQ2%E%-T)4 z>ALHijHx~_LcquR{k%A`kLPFOZKfX>hz+d1h;LIzRbbevY;Ltg>Ws^Ro9kE0c%sDj zwCk^%7gJ7iq>-+~?;~P{5sI<`HZj7&R(o-Dv5eizHTzF?Tj4_6JYI~1cs3gXgzR-F znZ|~ZGgR#UGp8Zh@9hoIb0 z-xFV|Twf;vFSuYsyEU`~KM9ZsH&L)5W;PX59S`C-HJaydmzKqv|9kNUC^6b`3^u3JEzmb-cR=%wH7N?5$ZLGL@s0+JZy3F??3IG8;k?Ul~%nUx&9k z>^KnL6Z(QbJ0K=m1Sh$HLo-~=@!!P{Qg=>diNo=71ny z*V#w#Qz6rmsN9@jJ1CH#1&-jcBPdU5(aIF0fRB7P{Purb?Fx$yr{WX0tK$6fW+>?q z*wNq8b|K2I0+n4r>*WG;OaOS2Sj-psB*~i%^Ihqw2SV8L{!Sh+ttjtY7|Enpnfh-5 zTGq}m_e9L_XaW#}UJ5M?86n-?2>1bmPdRQPwPB*&mOm-rSt6m z&)KinMA!@|k4fh~;EbdK#EMk!-ggDLB2cO#h3j{iK)@lhR#alOcMP9Y?u~i0ye=`> zk-MjlZYQ45`@tf%uENjbKN)Zo48>*`V=_vFS&Be(5#Zq?Mr(+Jp)VWfzOs8x060P+(kl_eaFrT>JB?ow9p z-i;D=woGgaY8LG{EEA<(EAakOT7se~KefI?ZyGDm@PQ9(fROT^*SXgfaT>{-B+I`r zZER5T-B2CnCtL){bq7pWHcwRh>3$fXW*MgKGe!T9xWDIC&OaLNS06ySNBE|pA>5Pl zPhynYhKoW|{fwz7C#ukPnY}9kEusrM4=VxJD4Re3 z9hBgQ;1uMa82Zkp6mdEeox??!o$VcFsDiET8FOR7lYw+B`c}WtHM5q0my96bHT)7# z*(7q+e;($Qn+)y95(=D<%k3KSjj!3HbmyK>MUe%rC%4+EWO-hO2Zvd)R)3A!F9uo) zzSlJrhjLzinr%cMc<7NqfjUyxihNP_^5d0wexyW*C{L+eE~R!2V{AXdKxi52*Vg>b zQi~9*X^3$}R80QkG_Rm_)dM2lJUXD8>FRnx;(b{zabW91c;?vsoXEEQs(v9+G}emq zAIk!02lJyCWibXIOr{G*Jj9Yh%w_$wl3kMbm#AE#gaHT`)vu~|*9l4y4(R=-8A{$r z2QKykSFmQEvN0_wmXw6DQ4mG>Nm@B5Ol1g_ONKMg&=^OC8>_PNH^ z$IGDv4Jr3*kiW{2D0{lPUy!r*Rp`?Xj3LkxkuktHxc*<<`&dicVVeWxzqq&3ZnNb` zM0hQfm*wKH)B=dO^?$B_fMd(H{;1Bx3aZchIq%T#RUK`K3otKQ1TAv*ocnB1Jm@ZB zB!;|^rqlB&3&!1BI%=)ND7N-(C_W6iC_bs=7}!jR^z41r7mJ)>o0l<7K>y3P6W=!~ zi7PB!<*`Z%kimKK5sj{ml+jl=57xz~b+gdmi=i#r+H7`R=WLQ~+4#5XuN}$9Ce~>4 z2z8~cS|^72MEyC!t8}HHpXiuIf&btXb;U1@T3kJD8b=O0rH0~E3MVcGtw>Jg%aT^? z#Y7^KjCCZP2F}T7rUYjGEU)^Fv8?m{)uDtD=(bp zhxp(huojv1<=(sB2}o;IFopFCJ((lpu08zww|?24If0iO<-_g++;Gj)ee!f^eChSg zN)~a^Vrk)ZX(@Bqig(||UFc|)F(Xz^R4jy#3RG+~Xn&3mSFKg2t60EBzlUa;|yLcY8(BT9}^fn zTe)}Dzu@A=4QukC=8)mTPU$r7bV<$`T8caKb#PQOiQf#;g)o{ziUdL5jg3LZBl=%R z1>=3+lxBPWM1Prxlmr@z=Lur&|0@E?<6p@{20--$;PzitL1y#898WqNn|q}S&_N*k z`OIa6YNtYuBcFF5UCnGKlET-Yo5+1*YjVO74Lf)J6fGPa9uGQM2rjee`31A|Z8~Q; z9-{9Q{?T_5(K52D?FWO7GIvzXA~$ue3hvog-hLS{-Hv<{#@Ko+1$eIY{~w zfRkSE*EU{(*0jqrp&kuIvUmav7eu)pVGaw^B$wGf?DQ#x=pW`dshjR4lxt7K9r7cE zs|?Rmp*i^MrBx;7^Qacv1YppBLF2SbqfX|3T zuEn}CH(Q2QPJOb^#Ss!O(`Lm@r=3P2Oy4@9IbNRSw@4dTh-Nc5HI9xleu6CX$^`0V z%jqWj3Rl$O6G#Bq9oeNJ%L5A!g07W9mJnY*Cz6!Qp2d&K`sbdSiUTX>4GoVkv>=g~ z=tlg_C6Kf2VpPI0_1Xd*aT$sOz`>j)Um)jqhWdEJrJeRGp3R@*RfqcXJlQ^M1WFnH zDFxbO3+_VaG-3O4^lT`qbG<1|{1?Q}UMt>BC6yN}Y8b5Z0S*6(f6SCnwY)D;i%4JX ztdEpu+2b)DG=*_fG=&?*hV;jcDv-%66#QK#YuVr@$VZ4L@qhTXZOZD10Y8YhRx8AI zqiU9C$pAs4N9tQwO&X*TSl#S}YKIK~FsG0V{ims__ zCGUo_B0$;lkp^0aUS@eY@#Rmjh$<&dS14klNq-OuN!mqxPj*ov_N{VTow@Cb){G6k zLC4b}=E_S}@c}F0 z{_r@e*6$@0Y4Svf7?hZduJd#*pHYFYXpqV2)JqsnF$@z|=;k!0gxjR8G!azy=ZXdA z`-_cfRu?ixQ^}*Emh|cWU0V zyo17{(yv75^Oq}B?E*uc?_Y{>A_OSm>Z^=>gO*=%{S0?EgQH$3o1>&cd7?AOe8_&c?;{ zKPER2>j9~jvh&OkTi`0i>MDiEFI6TL?#ji)$~25Xu(%`jlU}B55$3;8)hn z_4~;8Zp)Q^)9SR>lmC-|PyYRd$LiEzLB$X~xUh~8Mj{NBJuz}Vwvd9nyf)@I^!_2* z{=wnln3>T+XxF;%_i?}3LNE{aP$EpjcL6ZYEj+;R%;8l(#jlDO3Bu>m0uBNRB4or| z5P?lZ6p$Z}{D6%5j~prix@&+0gq^V8g&gcGW{Nc2^~NX@}xx7PJs=o1m2}3dU@^946z03nvmvL9(@Xw7-F@#J2{~o9=myXoPuug-UGzB z)#TjmAXko9HUS*VIA=#tp&fa>_1RSv+zM$8pa4w)A+qpVK{JOuEZuelYU?7C{(-=VD%Q5f+ z->)y8&8aUKeHlTi+M)Y1u73pzf$Uw+^=XtolLix`y61hDes`GtqL#yqJ~Kr>>tE$nmy0ZkBT$)z*NBJ zHOT?UhX;e(QIGd;kS8E|Cqyw~?(Pfz<(vLhkmQ@Qm z7-VbclJC28dunav*dKj4;Tb&eWlzoTuK_q+baU{BLw+Oti$Ad`3ZU^^cmf4?cLV!n z7S8)~kiP&$zCF6+(V+2l3dy}UpDkd!!`t-05jh2`$F<~!wn2{U-gqJ&z-|3q1 zmJk-SJKz}&I?($gktQG=I?%NP8FV}S3Os-$*AdvuD5nls+i&}a40Z?ZNngOQ58%CA z(bof_1uy|?kAlQ7K>zs{(Sk3?ulVl(A&|OLq89@S-A*3vo#zKhw&%}};BP*d0wv^f zimPeP;^QU5?8*I7A|kW$osPBe7s%UAMbbIKzjZW9op#oclab$i7%wR7^h-PqsE;m@ zp6PuzwX>@MIdiMF3%u+yF}|gpuxREDlgIobbWSbj)n}7^!0*LH;|CQG&<0Y$>a{C3 z!DPZy&*@{&Zh_2(8uD2k1AlFEzla%)uCaRH+epUA z=~=2p(f7#as4YR&uD91vjZTZ`4yA*sie+A2Pb69k9Nr{{w$0EAwO?@{quQ&R7}WkF z4bXY)kkIwI02i_)WqO5eiTwB3ExBN(LMm0!dU2$NC*9ns#nNLx+6AqyPizsV%~|sy zSpnl#a*?x!&mW2(PKTP+wl5}|9z$J#VV{^9-S_~(&%zUXjN!NkJP|0KWtE70-Vy_} zIECO9ddgm*aqnvH7qwf`Vw5Hc%b@h1?B)*R_cGLtz=Fu52eps!zgjcP5c1WIzI(0c z{l`2!--#L}tbulawMA0oV$D1AZ167;c%@ZPmF{!{K=_(jpPa7w=4)}zpxHsd_K#$` zYp=Vwf0D641uP-*Pnop#u$$88+!{Y{9;3zxJ%b?u#NLx%t6D=vRalEr?oLQJIxSt4 zW=?JjK=cFkb#T^h5pq)5u6W8LvQE`0PsgcIAB?lg%EdrttM?R5J zHyh(YSbG?Z5i(8iU&k!-;pQ2k)NMWs8YKtyfH3^j&0CwS4k5aDe!q7Gn)#*Voy4vT zRD?94;Fw?a<~Eg?Vo6lT_%~k!Mv>5*=!4C|!IkXO!L;AQSa}mV?H%5exmDN(9Umwz zB==ql)3Q0bwTF3&wK7do2<($X+zkDxG~gTW{**@%0JS3}59%X!0i&5=&NG%LYcfFTLN8TXH zsTkSWDS3j2s5fY-`00_Z!F;-#sa8SiMUu;5Ql*ua@e5`ZmyG-4F_#D0QJx5f=1q=# z)55*{PE$i~4+jY^_UN{l3>FVCt3Vgqa!>URwxoVFT^JPEB#j(EdD0cyvuM4aT&?%S zHJ-Fuc)7ZPqAI1i`Gm%@o4cu1TIM&q%_Zp{-p5rtBkN-qx5znygLC727TkYq^H|6f z@!c*ju;eVpdLH6x^wrcNd9t&X&*8)*w?YOE%{+)gF8k$*i>^>*V%`n>rH^kA>*Xs^ z(lC9;@7xMy!u8;uf6AA7eJDva0lQw7Gmqn_sPJ53DoINJ2ocMxV2`Bx)(zC7i+67? z4kc{*NJJZE1KGn$F6>_;rYvQ1w4#Y6qqXvGoVg>{#OixeA!Ta5 zEUvaBT)%Bv~Q=AIN|~=$$uix& zd|%X`A&U!eaY}$wJel77r;Qa-__e=kgh?Gi?$2Vc_c0$lop*pGDQ}*$$?#9FwGZL-*)TzM z*f&J}c@BI6D7agOctTh;SyJee?kG1N`ZWF>mBuNeJ9Xl39^xDon8+qKu2=g7M3bDUoOGN^u@}wYteQ?)=-GDPRZ4D0p%X z0?6yC!7+e)tz#y~5sg`TA6YoozD*6436VJ$8@bko2m367r)9Z4Vk435dK}zoxy`jk z&bB-GE0;sxUH#G_**i6xDbjq3U^n~Y%ZmrVc(5$5c;@B{|^9KK&8KpYJWwYeoDF>fQ8P_77S`9 z)bD?tb;j;DjFyRJi6lbL+ZsWR{3%bBvOa zu_=&XsCfUBslY-Su1lJeo+}cltv$oE==B(uWnhGQrjj!%*ls{dCVE`IofuFsWK%Ec z{^_Y6J12b@h$yIIrY`WslDaKhe=7fudSfxe%A3~9n3j3#*>zDn$At~UW%h)* zzwQS$Wr3L^tP|TMVimAFeI?_Ys+^eETJUNrNUh|^0NKZ%=Te2iSt^U|xfsSw1 z9{_AVOMk;6I2ur-n1eknA2lX63}K4+*f^AN{Z)`|{n;@>gEyq~Vu!J#$3F)pq;xWx zHOg#S)=mXX^&qzf7S_A%;i2i%{7UNlle1||`eeNZW$XvyZ%#qJO{-lWBV-l^-dHaP z7z?h{?xtDNY)MtHN*v4klGd<2@}UpIVf#_Y41a=~#N}N;V|~UB_7PC4t9HFbO+WJJ zs{MhkOaddjYL*dvWveRX73+?U6D#$d6p1&Q0)n2j2B0Qtv+v5)#yg(?8w2hKJ$pRP z6Ws;%l$3S$9{@}el+2_xZ}U{|-yYpLXOVt3ig#HN{W;K;={zbCqshen7o1~So(h9; zM}I;f$^ESnfxdB-UhY0>@hgtaqU*&>vF)9&rN`Y^#Y2F%LR;CG@5!e-)YoV`<5uph zIOfaj)jsw5b;R988D`*523+poAL88+#qnro+4=t)X1xc|V zL70qf-d<_-seta}*Sh?RCQ>{fePb-7w+Ec%VJ*H9Ox(ByEW$_9?Gm5p6lCpmu&QY_ z%f=dAG<|y`|E7w-dPbki_eoeIQ?LYCZ*wr!F6BG43~Z?G#gOZ=UR;E_bHxfN^?xp- z{7)8ZsDo%z{Hx<&Z{vzW@!`}LMv*})*lEwCKp-Oi7pi=PyJk*<%l%x|7dnQuTUL?c zv9G2d>E<(uNPbH)rew6FSC8+wZdiNS*$?{t?fhGtqh_7tPs& z0R8E%@eR3C^7paNr;^@e%g&)GTQVY)A=7})3Rm|F|L>YH!_;(5NdT5l{J^ii4mD+i z@vxcJi1Or}q4P>N1IG{VQeisWyybKO!Qpab0X1LY7-OM}(|pO7eHj=2vwv{>KxQYA zfWnDviDX6IK7Vy_30~K>We@?yi)l0o^A1Uh8r~09m#oVRScNclChh8qB}e|dY%ifU zu~`P2wreRV?TtKCChF8daC>}u5+;yho+j_U*F7MC2Z3;2}W|<|a9#56{p+63>IuD}Sf#2h281YWB#& z>a%4LvWU7<>N7idhjVg(>04&fkP=B;;xMJq0is5fBgz!nMM-Q|hN@0{cAsS~e0rFz z+jb|Wfa|TGVcgNTS)9%4x>qBkbE6Hzj6Y@LNHCSL&rD4TuD!g;_V7hB8gYEJ+^Dow zTWvSP`?ZgX`WR=@o`0xBg47CwmeEBPo0kOJpyfq$il!K+V*(tK$3Lvptflw6aK`3o z2rbgCQ)F6&Jt*e%LHqBNA#TXMJR94pk5?=h_)1+tMw(hOAACu8TJ(MjG`yHhKPC|c zhrgfL!VW;u#l_KrSpjkw<14I+txr;|l7xpQm3vdnvl%dqZhu!Bs>oNPHo;bnF@2UE z9F3Glrr@Wr*t}>_BORaPP5l@@WmM``tmjklhekR;_U9szR6kR{;)hq(C)ml1C^{a- z0`r6GXs3R)Fdd(;ig(flB&uqcbC}L^7q;-z<#@mU-y%4o}k<8EWzAzu#L zRY&-PCx9w|xqrfO+@TC8b$Y*;$--#B5_7)i<)x4h;S3=Ani@F7U8dHd<1`C7D*LFl z^7f6Qaw~<=UR+@^+fEc&w zktz10_ZeDi2ktt=90JUH6M{;g z2MS!-VM=-LAMhrNy(KR|i8L%_>{GF>n0Mrfv?P1NJrfyi&$fvjA=Oeh_f|&V;P^4I z$KIffIe(MdgLLBecPr%>?VI(_yllquo7>RbYlY{6GRoonBd+wsd!O;OD}!`$U2MJm zPF{kNe?IUUQG!>aKe4ORMXlGEcf)cN1!~w7Iqh5u3J>dj2<~cfp6CtRP%(|r(+d4H z7E0`+Da)#mTnMbAUBUM%?&PIaN$uZ~d)uvRV}H&w!qpcR^KecsgL)ofH@}095)?cB zQZ-qt7*U4LKZ})A47#+&6+y#%u(Jlqb6>s_*WIrV<#`}?VT&pXV^A$;dB(sK4(3*Y z;;TMf5qt3ojn1@aOHmrwqy)9G<&zPivg=aM^fzcc<$Yy`OQ;Mn=fLQ8Y|`MRlch}f z9DhSoO;Iy4R_5q1J!JV}C6hh~{&lW9?w%P*c8+^+Gu{2*M8Hr-gJ3q@^lVX{)J6FaDKuzb3$(?c}d)gMSOhf;c+ z5n{Vul=w{w4_Ib@Lj413-oH^ssj9$516BHWGkEduo44>#D&5AM`%|n z#zS!+higJi6|F%y>iUbv=cB^cb|9+i?{VC0-}8o>9P`jOTr7{+IGyvU2sYUT4?ESK z|0;E(|2%k|LBQF?$EtaQs0zniAb;I0+^ap4V6;lDoCksiHmucSjp^~`#yO=_9AEo! z(O+<7E)Fqd9>(Xc)GRdx4dZIK2;Z?TH3tWxc%%r_8cID+vuUm|f3%oxeBC&Z9IDvY zcK%Spyyo_oRxW{^jv>BFL?Utf8s~Qtg2;TizN&1tvV(&dca5&$ykh(zB7d3DNHL6} zi(ey@4Z}v}9r**1Hu}@A=Irj@B_*7a%&{2K#y-TI#CYl0@f}eIPYoqteDUi)oSPKc z&;G$q!23(5_?v@+L~42133fy&GM{;{puNI^=sV5QCVDmhT#)ZGtzvXTk_U6QG6NnD zcpm^`U5=nO5yzG`aS0V9Y=7^LhxFl!`x#`$J4ufE1WPO2nq;o~Ah{#X?}crQt9`2J z`L8IO3WUsnev3jtk3 z^239@pCu`%UOw9*7&`nKk1e?|^sP)h7BXot(be&6>Y5`Au`yZ_H}=Nol1w3{1aYWr zwL>hihF*d{5$pTDS{v5?jy5X0W73yJTgmvv4ilu8p5`;0^Np0j$YA}&2F_Pl`Qo(X z^k&rx4cW}e#mrDr(|UG?lcc5i*~ zqepJ@+Jb)#lUBbYHAYdXX}B6#LknZz_@gYhh}C2 zL0)Cun{>ZYdMk2BA|~`ELo!n)03Lirb>95_-l1%|?0@q+IwhMWJLVleay6qnlSq!| z-%vmXDp5CKMMR?s{HQ(3N@b`DZXuDZ<{gxcpLvtsNUmL8^v7{JH8hB~WO7*2rL4^M z2E++sqlv!()CDHz3T+QjR0`a+HOd;(nyZ!`#|CZtcERY&`<^M8n3?{G|Iec1m6Gqv#*SG^o+@EKADITV6DRcBZL#&0e85Qpx^amW2qtVXm zDXRB`5naz*HZqJ}D0fjL0yi+QljBis;;RHy0GS=MLr#kj&5BXHn#$pstRn}P;9L4| zibu!wMM}9WjH|Qy2i@m2W8$(bJ*|WUG*d=&Sbw}Kmo=Zxc6@cX%EBZyOV_0i(fj71yg{9W+A=q*!8R#si+Tm8_SfN9zb4S9@P zqJK_nxc8&xgV#NIFrSOLr+C`0K7yvc{pz*K(qD0WxUbS=h^nki&yg)xpiD@HbBimw z>%%1Fs0(-~e9@JbxZtz-oR(vxq67knU7iQeFYJMtp4d=YJzyiH>F}&d^SSm->Z*i)1ZZyb&K4N4g$g z?Nbg9&3wLf{4mO4&Ap@aIi5x5EW!lUhnO{?Tx%?FICItx^iUI{E}>CA@_p>i(xr@l zM9n)S+7-{JqW7DgA4OX!8b%KIuDr?YI?=sp%n8f7!cK$=Ou6Tpxwuyhr>}%L7=JE{ z_npbK{J0)1h}|IU@q+fe{Y~!7NOLh_@>+;Ge9rN&3h%J}>1L9n1CCAG4>>X26DzNn zWXwy_raEn}N6C4FT!g~(QTr#`eyq9{Z+PpHZ=nd6?czZxjD;(~fq$WCgfz#VysXf)7Oeqf=8&Q8ThD{_6GnzEjsn25 z)+)VQE|@5}T3FKYy3*2G9-NIrn)TvKwQUHLsyXYHQEkL7n=pE^~BLXmaCUbP2A{@o zqmt||wmq-vnC;7bx;l$0qkmNDkH86$i_!Y2Q`#S7AY)IZyjGi$S#wD5vC5VLL$ll|s{N+x!1pQ>7@8m&q<6 zp8(?t#_uQh#T9g!A8whAt_g7uv%QN?-QwiRA*`!ThbzYn`mz|)t$&sI@qBYq%PRz_Ol&NY8-Z&=6MjD^d6Mn~vq{wZIAyXAM3 zWa{P#eVkqvFptsXDTvNyTWXNuTiB}ahZc#{W4kPD+S!$O8jZoLUl3x_ko z=Ao-_(%!1P7A5J}v^80VT+v=_qLiXfHh`BQ1!zYg$wDt(OvQM_JVSWC1c$(T<_8LoHQgscpOtg0F< z?cy@eCreUn^$Kl{8v68Y5AYDW4#*VsM7L#;;n`+8`3Zl(TQYGg!2ViH_;`aKw9K;bye77hua$iYRa%)5`E6lTHjSp{ZhnGutr1ryf|oWZs? zU+qPfr&3r1=&2Yxg0B`iNvSfvoN*a)%jr>Sy_R{eOAPtiumAvr<(ITR2F6*_U^D$J zIM$6oa<-N^v?JwkM$t^k{UVI`?iW^CN?0MGY{sQ=r?*KR@~vCr7B_Oi=s0>>L) zT*sbeHh+CvpC*(uGq>voEWZ$0Vt-4Q88M1AfPXv#p#L!D?A(gJoHTtCH!CI3G5C-< zzE2EIrJ9@0%+x-=1-sg2^gJw#(2u=kCG3cYCf5S_kN+g!S)3z05YB9wv=+E*69%z~#I@hEd=|+a=TomA)`i0P1%7Pn| zu9_M7rxS_`-3r7c6KA?BgIqYVRrZ1-g9SE|p?MhB8;;Gt=?Z3GY{a#!f zcYq*?)w&~x^~@|rorW7w=UTADGkojS{2wp+bCZ|hVgwWcHkaT=1r)amV+0Ck0XUc7 zMg%{B9zd*!{>UVHBu#wS|3oYJ;XYp?P#p-+fO^A)jsO-Y1Yiwz0J+!! zpmqQ~un~ViUsqm77oe!4sjsEW3WNvN^>TBAdi)O;SzSGSMGkHP zv4yu2<^cXDz{BbQ)EsTV5Kr)flLGV~r7OIZa3{DN_HSbFM!+6oy8L4f@C1YZBaH*d z^S7^>mX;d8732tkfgvD>4crh0f_ZrYtbWVjZ?G-%KLo)5SuYQdhaBqvQhEG4&A+J2 zK;eItnYs7}g1rBG#Xt}*Prtu<^PgqgKp~!vo-ogUB7y;SjxOK__MQ)G<_P&MQb1Dz9m4&*x!dZJ_KD1BH z5#A>l)WetS{}^y*2-F+m|9@%i93i%L537G*>*dC!4{>z&0xQe?j}Kgg`A23Ch5@(% zV0Qr6$Hsx{A@%Rcc@XnFh~X^>^ml{00qj67p5Q=7J23o#>F)`81_r=9yug9}e=GhI zVe$w8Y#nW2@S%rq9L(R*l_7RefXF{$IFo;={%0Imc!BW!#0uZEwor(RFTfUThsl4X z0foVbgXRCvp8Ky{3SKTQ8X#9N%m3ck|F#CXI=cA&FZTasX9#}a&7uMIa0R*im(I~s z!O;h7tK|r@armdn|H%IdB@MB60RuRB_<`Jfg1rA|^dEMc3w*5MN0Z~j;r5^x{4X7R zVr`ruU{6l~Klg7H7~YxxVh>-A2kw6WE(Ki;d38djPM1AOPgy z0rJJGL0Js$YAU*(>@*l(x;8OX61OQyBe-J#T+8+duss0BE1GqH)jf8mtTsnV$5Im;d z9|X5F`Zwa`hC`5xn*-?IA`!U6`Y!_!IE3E||CaE>!`Z<1?jH#cH@qV7Ul5+&?k@gj^PhO|3@#6&@JA0X z!u1b)paA*9i1z_{{iTNUar=LxhR+(88^{CZ2y(G?wEI)oZ#me*5o-IFE1ah1UyXK7T=Y=Y4;}|D1KLhpXWC$;SO> z`2XjQ{Ec;C9#Chnp`$JQ7s6jA@D1VN=wr$aKbCmla`@|C-^~9b0pow46ZJ1|85yXL zKPMkAfRk5%`(X)%;Uml)_#d7&|6EqT4@CGL{TF_?DF9%w57-8Cb_#0qG~6k@A@W_2 ze8ET=Dx(PSX9cN*kxDYkZ01NS6}epA8XZ`&FtjNoi>VN*rX*?}^ePlmY{VE&;&Qjt zoLc$q)K*(+9TcP z^2n|86VZ3X_#(-hIF|2O+*?TT4rYn>U1GTG@7*~wRkyGp!u@||h02SZL&{>9L?lGE<7CsWd3;Y0pAjI?KW4%ku-VjD816&=-iFzrr~T@LE1LTC zov`=v(|!fRxwSUHGr#S*KqIzgMyQHTcW@3}oCJ0ELdxyZ3*X1pKd#7Enrmc)9eml? z`>}jxFKtm7dX0ZZ?ylWMGjZ$$u&3h_`{j$pD}o{inpJUhCf zoIz6fPBbo5Lhkrx^Cvor(#?>n$&5U2)Qod4bEi~sP3JEL5fP`ozVu0jo(l#7x-i=4 zyR5hT>?kVR1!CXR$|tAKovB;LdGg|2Cxl8~kjRU!SC@a=*?wz#*_;7%!8563@O=v#9fODr3JVq9*U&$4JVIv7*JWrDj5yk3+g0!20p3Xifhu@F-7y0c_g zHt>VQM^Jxi&$MVdokX|XUnC$^+noesuTD$=DAyn;ZsBWt5`xwZ2r=vWRd=;%ib~xV zf?tF>WExc0-}^3kRcLtV4|g2nx~3wGW%bRraviNCyg8WWCmvHNsD z^%Qh$pf)svHKJr>2Wyr|p7(J0X6yi&HD=vQeMnBpV?utOVvkqSur3!UI+pjREXBT$#bs6sDhivl3~Y{F`85;-)4flv zvQlrNdk!r_oW%Z^mHs(vg&Rj>{1wCVsHNJg$~3VUT!4I5vK8U*z$ax4;8aQtRjsC- zwOnNd7X{jfcclcyV^fjzib|EqgWC7DtD}EvqAcc?gC9|*1HZho2q#UIS|^!Sw{iR`0kRMLSc7LVw5$auc+D>I%I@i|ygCNO^< zw@!prZxZE9%lIl|*yn29&y-#?fC%lj(IMamL&FK9l6z=DcqGW)fs`h zE}~SwW<_ewjHry|zE_2j0lZvEFD8H6)(zC9 z#zOrG$opxh){(X2tOLm9DG1fL=8bpzR=$-^2<5&Q-u3wILKgcZa>=%Wi75?J#59H?qm^bf3(cPO& z$h_($Ya9?h-mu6-$$Ca-M(yai_03Ugv1^Xj0w5sG{Z5wo z&FcfQm*h|E$>n8|BHrWG6+}|o7wU6>P|_!4wv0FVMPSc-w9Dkujr7aygG*g8a2XWz zjK&cU$B7G9VXway^gjvv9+m1@*&bAz`AZ;sNNtnsn^W62=KL4xwkLn2CSpj9LdsSO zB1r=@zKj|}M8}4{pI9I7cofG38>DqIk9SfX68>UkE^uM^Vr@(gOhldv^mKo*0lw_7 z!mgcotM}Ro{MO0^mndpc@2sy4jannquaS3WqKMeqh3Qa-^;SB!j4tjHgK-{r&B)~y zR5Rl^KXQvlovi?I0KI<;5viD=I8`$3a5rD*b>?kOFq146c7lDpZaIX{uF=TF~{x zLv49l0`psDUs)!Bin3p+i}XbEQzT=^a1881C`)1*3M#eI4ebL{DpwWmBHrZso7F6m z{VZ7JGz^{l__TiZEL6ax5N~@*W@v)DE|m&@TVW8WpBEHh=nGsTP%Abu0bQpT>8Gq}2CU_m;0PV~*>SdTMQTLT&q9$id}E>FoZ4$Us=lYK z-eB|I=}9dQwXqX_vfwd|E+|H1w*WHX4<xkq8wZB}1x;gg? zGAOmq6CDQyzm0P}Y!2Kz5-9$vtoBg|Bc&-TIkJD4rS^SJfNYRk7tgQC!~#=Y_7&Sn zhdPSqQk8&QV5&Lsn&lc>bofC$&n5HLPM7ggb{dO{X>?CGrjM)%jP_ggn(l?U4zmbpEyQhwdV$2Xjz=Has#`6?KE^Gf+XXd7)o}3Pl#4tD;`+rH?jc=TiJr@U zpbQAzgtZ8_HPasYEXpgSq3Y+T^9~|PhdqB<4Nn-Vr=%K6RLDbGo6%bq+NVBDSy|Ei z+Fd*y*z?Ay{ch6KtDNW>nENH7EzhqtOi@O~}*1Jb{?XlQbo9d(*146!qatD7O zwn_4-l@bLse+gFgxvE zG)e#T(9nJ6z$NB|Mw;v@3%ykB_)wKd5P{T(`TdH1gU}XB=mzHipSm?9)*MDC~}pSoqc zzc+XFTE+iLnO>`OF1o_>TD?7cFn2+)VUSj&1lw>;4CS-0TzT^G*RK@edK zw`h%=nQi28JKvljci+pzMmK+zsuj)@E-o?R$Z3%vRd9xEe<0x-<(zE%9twIJ)PHBCZMf*HMJ&n1{}(6yotN$8V# zWsel_fz+vaWKJ7<=)|_7*iL&gx&CNRgN19Oq+RLp#-+PBE5zl?qO^Zr)S^Drh<0UG zvEsz~%p=D`|D5Rf!qdxYqIv9m>@`1bj6cZ(5PSuRWYyB#j zCxOCtm)U&kNs8}Zruu)RzMK?!EYkAf5#Au%q*~8&MLw#X(t0`ej zGseAg9JUp8X-Ohy-{R+gy-IZ!POV53*W0c>+00KuRaW~UAE^c08|9DEu2B@abK`#3 z8H1PPxsF#+indTsf*(O(Am)VrzFWUm5&cKhVVHyM!5;e(@>72^r95ZA6NzSd{MW4a zuXFlXST9IEKYhm6@Uc=^)hj2DVf&K|tf7jl|NSv3NAi!Fd&elj=U0|VDpgPU_BR*i znF{NceK`)WqIp*7zCCs79#j$Jx@9Wi0$ngVPhR8}hrT`tcz3$ju)TDpH(1#5OTXG& zs>o5S->Jc33)O#v0H9~z{XS>{m<2?`FWbQ9^_7$-wmn_{oe<2}S`^Y!wx_cGW%t($yLbQ-#jn#F$@t5%{W?@&Btk&7cp8yr2$ znw%W&Pdl(_C;}(Cafy-^NX1|z#iiN^#80;E}9Md!~Pm$;ZGhvtE!ldWiMy<)hK;=Ia^#@>Wi(h%SMz;Y}u-GftclKfwFV;=a!Z zdN~c3&|YzfIT?HhTG3WlJaIzV<*HV@xAJfbfBsq(^(IP*ga>_#h(3!no=<>|*r^FUn}(HA4&%yKe};4_4+)j4}0Wx%QO2%r-$A_dDlXPMbn^ zU9x}KYuR0p|>mMc(90-}c>awM-x0W5e0EHn;fs z-NhAIVTem~Vpq)joEtCPj1@mo_991J4O%u&q;Rpk0JG;swKDsu| zSA56z1MfyfK48~IjiKM2m~^`Fn_+)lam;FUGLn;Umyog#W5Jj7k74blBA9IC;*Wd` z5R*mq!#LCyaz+Q5#x-}AhgUFiLV0Fy94N!66BYT-8@rZ%G38?StC6~z^$%#4^Y(wm z(M^67nbd8WK{G@;dk(Vx)q_tw5G}Apdd1A#LmsFnv3f>p*RSQbjfV!xCa>M5H@ZHT z_Au5LQIV9i?8xW*8qlR&TA141iEv|QJQ}@KePDNv@+Har=||3;Xyd)7pH30gBL?oY z&>CqwG`f`Gp%c`B53@Wz~L=eT3K+w;4J3$HxKwJ9R&o*^`OeMq9qxZ;HEEu_ ztd*G}0uNlRqG8q&KvupuJ|^jTF`Dh3N>p)3ZhEGZ0=dTQ2YnG|HyuIZPkMTvB8*UV zVkp7-#9G^rfg5k!z8*+y!pv>0E)}jxkcS6Je%*TTfqayDKke2yKDW)XVEAlalQ^LT zvR^L4`^1a8Fk#txBAuDhxBGt`Iq!A7!iXo*fO1*1m4SmtqE8hD=lZX!Q&}flzcPSR zRbRh0g2FdfhK?RCnE1oiUU>(`hCT@T5r=w)@&vQHC|i*6XC~|3ZIX0`2rl)iOTm zaiq|?=Ze2s_B%9!qaew`wIMmdW}@ZN8cravK0IOKq{^^!ujD%kFQMwTA7qGG7ID9u za!})0?L6opE&?vv*s1>y)q7<0X_3>1*CZRd|td2&ujX4*VuY_(kLa92#ckuH^bK_;L`aXX~VzbVvHz_wP!P0IX zot|=%^gRBtQ!BcoVrGffgp-xRP@0GOfh!tEi3e(TBGRjtk`R$#l2lG`iE*zdeRhy#o;C4%nd+8gkH5;k5=Ay~9g$JrIe zteR32eQ_E|!@!?sQuu&Uzxf(^wA{;z*{nsi;buTv7W0J$kNHA@EQKsO2yZ_{XedQR z`2$WMa_$`Eh7i+Go(uz1n~7_xO}+l2Mm&KSM%&ipkF|ft1+aYw3VQ5XGJ>JleJy8O zpx|C(yNTqOCbg1+ti#+$D=y4EvvCxegFz()&(qQkr-9}aMn3@6;B-D{s$sL^Sbj3_ zF^qhLC{dn)V1mta3I?GEHYt8ix=xFN8wYx3Ea)nhG59?cBQDc zuio*&8eyYx)`zkiiZ)bk+6V>P1ey6Q(4}_MS0sP?8Zo3tM@(9nKApE@*ctnw{hgU( z*df?+#-jQm`0nzw-jo&0UzIyUOh!{G_4;4Sfhx|nEowJ;tTw{;RmCZJ-XXQeJhG=f zEK=vDo#M(j>(65?hlSO8L~ua_Cq|ZP{eo6MQzBk*l{xi|jdBFZgLg1^)cjSEFQsl$l@6ltH#9KVUznPw;MkndTfvXFi9)n)1j}#?&Y?*$BI^)Z)p}myjI( zO(|qf0C9SUP!uk4TZn7yaP@EscR|rGFM@x_YrYYgSl1mxP3Dhvk{KbN%?z^vk^U$%tueKJ6D_frR#G)3uq+N@7uIkcUcLMF0T~q_|1jWq@r2DGI%c_0g5k~OY4Gu!{dB;uwoL0;V-Tj$H2DFp?M&OPKX>BZobaO-NQz(WOZs%BQM4l2};Ij(Q!#6unhO zs_ln8kA8Xaj0_otU!EQ##_bU&{C}qD$|JgtdF&&^Y1yyly2sjuJYQr8oEO8fDBrrs zVv1hxdkVw0RN2Z&gv=V+3A2w$_ep>IaD>(N^~r%!D|uW$S;e?5me8~dxfnu}aVhk7 zm7*)GE=})6Q3(wmJ(u32C#dAf8FzXX6ji+aWXa0w837(gLY}FSR3?R^;bS2!flDl{ zNy>f=i)R^7iWto=6%h#EyesHSN07;S>o8$=6j#IVo0dfmIU5Y!Boizn=;?pBG=E}N zDL=!>$6e3|$*oN*^i8WdqQ|UxeHf?lOd#ma>m3+jbeK!LfZ+y87qN>Li#YLOr26Lq znH(aQ?_;ZLSEN!+7va~VIb^fzs6y78r4FW;$k)D)=q@n)9oVZ1~n=jhG+O!AXJ z{*>1;VQt$Z?J|+j!8e20dwwcYjI0=krX|v=y#B#kpHV$3A5B@%ofdyBb|uhDz7jNc zb)0EH=l${yv8*k3mIL3*v>7YM^TcthX}&PY0+j!zo~CVpl~uTm5>=v$sd_Mvdt!yJ zSwOZ-8$a5rW~l0xYFU6ob@hV{PjI-%Dqrl zoK9DqG{xdyZ@RLBV_89Z>dj3RKjM>f81G9Zd7JRO=n?JQd#A*!EBmje`U(it9)&O* zFXQ*G@Q~-HJnBW8^sGA^P_x0Ln=iw`>`NsbkJ_=QTrt~Q5OsfMR4h7gb#QE+_VI&@yzVI~npdx!ie)H+cQq5)} z_9gEYZ$Qvoa;ZTTnT~Dvu#5u+jS$C|;ZK@SwYTRsPJ3EEttzKG%LkNqxWwL#RmHBJ zjM*$>#;s**bFqJVMe#6GuVNV@bR=1zj=nwmSjt59YI*SqKjUQJd5_SfB^laT4^JQGN+T`MTTlDxH`%YSB`X9Yij0*VwhYmEa~a zH|$~dMKSLXxqW(E>xJpz@iWazo$*UfWnYCpQ7GVIP?nJj5<7A4VBnMMRjy9g@{Vu* zsDMuJ9m#*TR{x=u$Z02eNk2&i;^|8qN{3zrFoevf^5>6O?<@D4Fch|dPT>tDkI3Al zl=fw$o^{6IfOt;TKr)V}RUu>I`ubXA2Aj3JCr?=m2%COY+N3G4BC9doT{k?|fEq{H9=a~E#z?+%t*Pdir+kI($u0ygxH%$4Q> zXBF<#S8M$^*2CMyGh1xkWeBkDg?UV--ZgS4E$jle!Z2GKj&Z-vE=}_uTjYhR{hGP} zyzGBxnxTK|xbQe57m9_=_G{_8S_o9BkTm*~Y37Q_+*;LPad8*ntLpW%nh}2)Z#I82%8q1zlTfI=M(tw)rhqp(({=DT z%0mOU3?@Y!Wl+eUSTk&sBV?+k2fQ+DDU-m~Bk4XpG&;IN*YFmBy~T51&4u-I*L}8t z;8-HOb5-pR%9oUNQ@w3)87A0K$>^e@Y4a_<{)nO@f)~FH0M?CPl&As9wh!w=5yPwlnKJ@!_y||b_;*PYc&n$M6GneMmTEi$>g~?egAVP*@I5huSj^s z%7G&EieDhq3d1pCw2C|aE%K!=t^N?yRT;Pe^2=0wrgk=jwMU&{vsKaZ47p+%i$v>V zOsLIBEn-dqG3J-rb-a|k`0(etntdjkbN#6GFZIw0vyWJSI8vMwZg?!kn6!WH+X1N8 z*(3XTcH^JEZ`rM$3VOwK>KZ!{xblOvh-C>n@ug0$=FhryT6>O?bi)~5c#1gFkh6%X z>wv?3Kdv6-PYZ8}9ka~=)R>tDS+1>w+`qGQZv;D^_UpIaXwBr1wItzn=^(di@^mSK z9AKpTGue0+>mRE!p`M>J2;F~db6xv4kJI9`7M%^A7yPgWfrO(Y(m%1k!MLSqA8vcZ zVkNSvB{RuK=`|j5#$jWofqEW|$TTfq+q&B9PW3(hbJ3kVBAJh*i{MY#D0p<8w4sCkTz+`x21IX_vXB z9r`QS1*R9cuU4q#|DJy@De>LNCpomfcQTvSES${k(ig{=!c=>5=FcJ---g{0vEK(A zAGW_4<*;VeFVA*}U-_nGU=Vk39$pK{AC<|FG?1k**Y*1hK?mA;Q5<6v-~~BfXGkE& z>=N=LvtG7)QrV^iU7xs1_6^mu70iIa6A z^MDw%up(r$BrJb(tz5`Vu_|gaA%)VjtL9tVxs5$%)p)DZFUVM5U`zK?6slkByLy;Q z;|4Ixk+BAnH33)tElQi%>O_TX1-{*BLOE))=hyUCkrqVk#-o8ntref=f=(nz1ym9g zu270MKI;*`_F*Uzo)_p~pFfcE|Au`8XmG#WR0-?7r)DO38%Rd;W+;u=k0| z3+YdrOFx7*rd9RlJ-+vwSz6Re%X&EVEeL)mVW;T9T&B~m8o>U8{^-LwTFa7 z$y^&cM;v45E;Y&J2DHql!MZrsWXx`YF#W0;53?UXVnDcs=DiDHkE5HDTi0$c$>_D! zJR})6Y{P#RZ12Tdz#R4Mjii(Kk#Uca^R_cJfcq+!LA8H4t$lJ2D+I8@n=8SW! zRDBzzlU$uZIWMNWzh1M*cWIJ-Htq1JcbV)?xIPbdAuXD2qH?YejaGoKvrg&!b&I0uoTIJfW?6PB_KYe%kQv zQ`&z-h3pcrhQnJJ1*3c0F{kRK82z`n_61|<*p`$ z4LhR9jylZ5A7UW$M-66m+@d>M5w@P-sTotfGi1BRg2NmOF@7cxiP{z?-BDjPf%6C% z+AeAg#hkO*pjP*oqN^0V=tiINiGA+nH)DT>2ErUJ7%X!sq(kktg>S?>IOU3N9pQh^ zyu>Y!XkSr`t2P@pUlWsY#?!A9^VU%B-{%r-KT0aa(SL8~Jf1yhyE)v-s@3969p$t- zOI6d{b!PH~GU6-F!uzJpdVgWnHg4&b4hei~^{{@6ZyAp}PG`MP-sR$RbAcHH6LcC`?`Nu`rv*}< z&%2#zj7z`Q4yY#T-~FW)l3BZZOUPG|4z*&P#7NHVKrj-EH^eH!Tz!Ft;-j@!<*kX(??>op<$2 zEA$w5gK>#Ik|KlyC0=?uvpjFI&s%|J`A6ic*nrQ&o=*FzY*ct1cgmk=%f+~3(urS5 z1^xRSp1Fzs@zV5#=@0oA;NGc|_CLO1G%&XR(%f2fogl3dLD~PeR~VU#iSz%W-PpeH zq`(*14T7DMlPl#04(12o>dw3PN+h6_R~dIns)HXxmv2&If|ie;xsM;?%o)R1PZQ54 z92`}tEdHHT>XA0%qAD(a^}6)>-R=6_7XUs8j~%Cvc@6HK{i2^jb4Es0;rGItLamLN zJeU|I_uz7og8TnU67a%DJH?@~1n_v~8*zfbCnUw??j36k^cevj=$HkF0|PZ6hFZL+ zjKLkdyVZ{l_=1c|xqHx!JJ_5tx$Y1bzMxe2NGx}*fq|gkR~QU~9(p5J2Y6wuIEP__ z1HPf@F4Wp}b0(po0=Od5vKOp#Wi8Y&e}cJM4ZHYs6+z(lVNQPK68D0f%OU53e5Zw~ z4J9?QfbbUsF>?T9dN))D2GRd8GqG|oV?d#RWq7r>899FL=9^iWRg=S&ioj2@g0u!x z<(k@AxxH9JjUfNy2?v8RgLt2vz|X1T4}ilOmboZ%G(aY^kX!0$3qazzXRNX z2)gi}<(`4FGc{CB1T5s2D|nzC6b5?@oG?td7Z!;cKuX2J@7?F!{W1k>yrC%t9D+zB zfoC(`1~uiqVCX>y_JYCn3_(zQd4gHsg%gPT*wUcq7Dd;IppzK%_X6iH_9m$`aF?$@#^=NZ_ghj^94O{7oXDq4=*e!hTW>DK{s(p@t|*g zL22$muW#u@g7`swd$9EW8I=6Uf`*Q&C49-?;CtF7`P-7|akoebpo0$|#9V^-F9u`` zuwfZ*LJ$HCpCnSkcSD!3Eq4I<97GqmQQ(JUFKW<0w?>Hnmo30uD&Gh4CZAtYc%kC*qemHA=_7{8L zFG2bd97=kqy-x-GA;2F$4uLQd>4o6hy;~Mo2gr6l8#aJDb+w_Fe~|&8Oj0<)=e^pzkRbC|0lyt{l=1TFlW$C{4la1Dqyjgflu zLNq0T)V1%#bbdCQGi-GN-Yo^E2r*VgjqeaU6&FOJ{TR`75Pw>wJoV+O8*4ZFw{O3o z>gFv3xwAF>Cq-Ct2l~VwQi4)ka}G2yB%jO|2A3T`j|gb2zN&B^O~^U`V9vfsxBf>t z&J4xf^ecqlRG0|MKlu5*?C6s;Oie*@QAf1?$1l)})ev+X5!0sBCo|d;-OF6p_ z$`#KSK|gJ;{;q!@r9Iz?K~Qrm8_iyio9M@V5O0xXm>7qU57ndIs8zdp_~FX>d%}Xo zX6;iXq-x9`qC7-Nu6RjUAfnYfHS$8A#lQp8OP7re%v*YzQFG9W{j>MRLlo_f#K4cz zbT|Q48IKFKp zH+y0`<0y7qWRK9QEjMLY3HWA_ch|avq>I zUO4(IR_63u82}Gds)`?srAm8#%uBdRl9?Vz79j*;e5NQQ<&qfJj39%G=Q6a5~K}eUJG{4wK!TpC2>$L$DxwTiVx|H3p?EN5|-3{;RW~8i>0~$0p%@W4wWQkc=YD2-W}f&$_xg zdTgF($ZNaW5E{4FUR1J^o`L%CXHU%ds~g|TZXXKk6e|SpZkdgMX>CtV4cmYTYroPn zV;X9)cI_PT04ph)*`(H*pb?=&p#uL@kX>?Pxyw&+>#_K?$b+Rq$=8B6t&S#2&k!&^ zA)u{81K7g#hO?Wdhi9`hdh%^(!Fg|ZDGqxuYcX#(;ytA$1A@v4vyXW#tzATNwi@zb zLUh)XqT5%ntv-@YFpfXdo6of^qfG|CU0g&ijvS`k zCX+29@L;7S9P;vOh?!v7C8r9nXk(f)0^AQRn5{#s`fMnx^1A616I*_WhgfzM(^rhe z#UW6*7h9j<3>OH(jp`AfWCAN-Pwb-!tCIBEW^d%^J$=KiBns{JmF=Ik*&!18(H z96E_-{Fr+iPRJCxQG#zf8->m|eCRwZosX+xefJ|89IcF3!Fa(5>Opea%5L6(7VuWQ zWfl`o`WX-z5Ka2o*Fs|%u}d^@!NcEtg~i3MZgt)v(s`jRJ6OpMlAW-5d^C*`Bb9Mf zt_X)~T(}y+D&ppeX0)cwf9G1(>L&^Frfj|%e)l`*V54(efzYl2)F50sKj!3mEXuY+ zk2;WR-Vz;sjz6G5pfzG?{VJ9K9dOfk(V8b!WVqgStjn;3IVAF4KE3t9xtYU!jO*R`?C%C}+`pgDBL>kUc>EDth z;xq2g$%FSHYm#3ko$!;Q_Py2UR) zaorGPZ>SXPv}2+W_Wb`GMq`m zdVU$H(W_5k32aIlY($ z((_U7Brl^|2i=q!tJ0BBV>qA1d=~G9Pk$qCx^$|p*TlqE z+%oazmeq&&UF-gt@LGuR;LvNSf2LcwozMns5Dhd6aDF zjiSB2#h<-z`7KObUd?Ro^y_WIm0BYmnXD@tPw%@B?Ap8|)3(ZL6&2n)4~k>2W%4}p zN97Kf@gKiqJQBw<=F4K|M+m$)tKS7Lz=L**v zn#fZRAKsj$14rf$4puf|{Dx}47dl12;{+ocFDBpO2E#9_zbl%mW=Dh*V;v62=;}kNO6uZ_mX0>p04@-I|2n z(8cg*i&K7mV$ywIR$2p@rXsy&_$l$7ha1;umq2xGu}9Dv+6{hb zC^mj}sNpx7hi@&N4)|lRL(VGaQR?U)3}=m^F?K|Sz$N0wNU}yc?gPX7JG-yDPW&I|xMvhe9I3D}7(Z3MWR$*mGv* zgFFhWK>5-Nz2RY(N)MZ6nA|^nuTIZ=l1=v0G4iaV2@Hz7nGd$1onIBf^Sz~Lg=$-qBd{wEm${gO*U6Pod{q$u;ne}|l{E5`_k0fN23Z1B^2=4ta zZql0C`m!_;Nt@o-6BMKi`*jlBPhvI9YYH(O07=$#98x6B=&S0xzgSeq?(JAxb%&5^ zK(nGO(+_>OymHF3^t=t!O3$G-C_OKyeZ+%ATbZhlqu_x9LDIU;rkS1Gf=FC_bwp4K ziS?gI`mY~`;g;;-TbCJrsCj&Uon0&Hq8iSA50KwKc;}JCKpzISd8#lJ=Cm(I67yMl z0UmFw43__xPFRCsE6hmT{;ZSh9y&|29skUKP%bos)|oR_dBx& zOo`hR6^zzOo*0!w+b9=K{Dk5hVxxL~?kQdz(zYE1YG+GV6~nhU?4bl=mJHHH*U-tc zC}<2P&()2=V;kX^-7iyRH^u+C*8 zp#`{4uZ!K$57M0)bbYf~wf}POT{y`SOnoyWoITbzoooi1Dpwo>b3TINi}d@3sfAjVQ`3D6UpJ?s;r=?D@;AF0wMGW(yRZdmvP@UYv$mqCkRB>CtMw6$lF< zj3jfp2~Q65+7{&E?-}YBH8z#IlAzWfUqTudHkQ{z4Zt+`w4Tg%u6u(PTixn|FMum z{uP@uY1^04N8n5czKYxV=uvHU8c^rw!g?dU_CL+ennja&PY0)M{+IPFBE-hovH$np zz?@$a9Gr6lbsfAblD(*F&Yc;Omb`Ebno2MR(W&U)mXG7ssW4=Oe@|fD-v#pFIT%Eg8?RgTsJB7a(aMN!E1Y;RdzEJA zq{REci3ptr90hilc~DW0904!y!no*DS?2Z$m4>yl1)Ie?FY&CjX^xB^+a>FB#>2|v zwfq4BY+(Wd12C>6zeS!VPryYZA%i^Bye{2`e!g00+-e*Y%(^%Cql#D-Ho-eW{<#2*9MDA0l;<2qrr0wn^p0i8I`k{94t!s&DY#_DTO00nvl<|1nw-u;)82xM z-K$h?w$)t216KTecZ-r>Se-rtH`?m3`9)%gFg1RtIuqgj4h-H?$xfMhZ%~bGvdQ>2VD6R*FGdTe)}ec`nLZTorKu zinVnAW+B8JEj*1P2LhM85yV3pHt@CY+_)z;Y;Bc)>podgO$Of7;0up? zFB>~6(l19Y)qai}&HtXEpZ=^xL$wvU^OVV`NlND+LN)Q|*5gZhThgqmv9@&ounTC0 zN@m}VB*)HsvISi3=)vuGO&QeCmLmnjcT>`XsS29;pgTVHFM)MJq%%pb#E4DTke(1B z%fnl=P9H*&?U5Bp{R*^3*}qiDoOO&~7+is3ja%@oEK!AzCGycpJ$I|7Rd+|~iZA$c z4k1|R4(Hi$1DAgr>??eGHnEFp4&BJPIjo(03;G?0kv!7|!0x|um^ICrb2j4h>TJtZ z$!2@C*aXjCUMlzXpqa&zcbKK)WmD?1LvHN4@AZ1XlUEsUf4}okLWzO zh_mzojo$*v%tKruwtW?;<_Cv|ZAb4g1#I{R#s|kATCu((`^Sdsds+M-(wgu7ZevYs zOcPQgg2^bz#zHqtiN(&k{!b)lT}iJx=Go~=`vcPR&>rv`A8J3%;4nwP@6+HI90Z*X z*-*=ToTN}{p;w{sQqpy-_f2CX>e^t!$F?`m+c4X4T2^U8ACf@C;iTCr^`Hj+%knaqpSh4kGt+0SCNV)r(v=y-U!+t6fA-% zZy8to%-PY(ntOV#J)6N-jP{j9ildyapT1l*PK__Oa~kv)N>*$VQR0vp6>;GuaIe`n zJsp%uN(NDii7@I~@h8X)<@nkW4e^6#64}u+=771)+9M`Pp(@jb3H97e7%@r$xtHlp zQnG%G9-@6@+y`~T6jyZR5i*drsG?h^V)P)>(APJVsgEFrqirS^T6$g*v z>CQ9>AH?oqrNUeGe!?M?r`sUUe)|a zG3q_|%TUsF@lo*jf3CxIz0(^|1Tik3WNQpUdSEF{0kp2^-Lx4p@*|W#A1c}-Cm=`f z7`VD;ifWtWJr&GiyPQZ?r_yaRWk^lZwNq?nWVH1)|H2;62GB4g2*D9 zGre8@Nz{To$!F6 zr<(<%%P%p?$zL&dC%?;oTq`c&SMWCKUhU1mp=4VSo7cFB(b%P?^%Aj81?Zc{$ z+e2AYcT|$Nq^&*8X|gMF28xWK78rnh_q9Z&abKbej&a|1mx$*Cz_I$v%PA;uNy}_b z2DAU6TF6UK>@`N{u~TBh3HV~XC@hjeo zv~{J%JEmC8-t63_$|t{Z&5+mzme+gwrK{iA<=l3}`1`O?H>9j~MqgZHlk;tfmSc8h zSatdv61}F(_7k|buyX1j%l#>&7u5*sutg=sz<{fcbN=R;lb!t+;2+BHjPoMZa7Qve zIpq`bvL{lk;cb@RY{Cpq`)pto$`~G6_K80@)EQcpMGg$$9LqK=1zV4rtzUqOp% zr%2(tp0F;XoBP>X%3xFEattArEu|0AxOg=I?qZMr->K8`3Jj)Hi^0^WS1N)Wa}+9q z7LF#P)WdY2U%3ZTKsYk9t|2Nm;RK@RJ@(QKU*|+@fam92Hj%qBH<`2&-!PO zPd*0M{Jk(V|C;x$CQLGxs!R_(Ci8gbJdY>65r@8M>F8@r17^>@!%hoA?(#j@Jf4jL ztGks5(S$NX7fs=IK7ocJwl$Y|C^FYPpB$nRgVCM3E|$xxRF73UQg0{mTV--KC{|YM zspV%9nRQ?!NYv=-3!<8|Wntm$^IJ2<5PS+h!~Z!}`yT`kZzZHySPGB< zNrU+RA$N#bnc10BX2BqS06ObVcw_Ot#~Swq1Tm-W+wABuf1+_$MWjf__UkabK*}(~ z3F3A@awl(=uer1ka{Ds0liWt|gviP(>g^~Sm>cSa_FW6`_T~0POfqfZa(qmbSoztf zDSS$X$pf!9JkapaT=q$`11*0YHP{L>dQ-Ggp!2^{cr}MonDLKL09^bf6y&_#%ReKJ z!`d1js0^7+;NTD;6T-AB?y|!*lMyO~BqnA_)eks*CaE0WG&w0K4Fgyln(F+??qv_8 zuo-z|QyC5Ajb_Mr(I?UuH1`#=q)q5N>&1n0da9TC>vGGuz}leuYFi` zL4*+CU^^Lc_?Z5DLFiNgE205^jwrNuq*e!!>$GY4ET{MIRmT;jyQNkGaehr^gAHnG z)jE}p1#7l#pc84Tq$`JkH};^@t}5raNLtQ#o#e~=4|q|NVp({y=W@nz#(<&!#tm%y zBdO^C9~B)F0qm9*fGk6C$tn8aiRz&}{}~{+eDhthMoYgZEZ+|RLawC%U( z3uzWiHpgmMeeDwXe3mz}m&vGm{q9^qIAeY{lyNtvw0!TYFQ;B%=LmIicyoGqwwkf{ zb5V$*az*9!u0ic?N2uU2fX$7@<`tQ`P;-K}=H22|5ZJc-VIuut#U$W0UzP5jdOYS= zu;rFgVb*Zp(v#(`+w0g88k|+t-mr)os#?6FQqytxX6(2?H$)}5DD-5+?aA6>v*j28 zLnk}bJ;Is@`LY*fzKcf;?#w1Q496@LirMaqFr>E?VJusG*hV;F<6>PS=tsj6qhM`B zHOzFr0B8$1P4TZ{4x5oRFwjk(OPXEk5N0S%(w`=)yW_JY$2%a4GwED^?|qX~^Um$( zDd$u4)c)=(H?_#0Z1GkM&bQbu``LU2@|xOxrYq`vsT<_9?Tqs9&$(`5-TK!Uo$M{A z8J28@{ahVhnw-rgr+S9UxeMVKzQ}n5&e(;?1i)y9-@sDEeyCHvkBxzEwWdfAs@+nM z4zuplT^u8_*;8Xh;Vs9O*!^{7(Pwk_6fmi`H*UreA5YnyPr4X&B+k#M8mTHi-m0Hi zHpRN8>$r!km*M!=7N>pvH8sSbE%TK-pZnKVZ``uHC~Ms5l_x$7YH&@G$>Z`t>ZLDm zn7Zk~b(HaI#&~USlhe$6XIc+;#3nElr-d|ax5$ZlKh-gE(EUX~sl+JGK}NOC`M{p( zn+V%0RB>*rHMQ!buUhHK4Rzzhdd5wvw!Ym66ZAT!xj6G3?%}+o*_3q>!3e&S-TR8^ zKO1fR!x5&t<~VWTV~~ax(iG5vS>^zMIfiSm^rdccYgaervj%#cdE#LTF@^2rY}feF zWOCQoOXMG6cGeWr0*};`1Ks50-`DYl{+1HLDJ-J);Ha|git9z^2W+I3UGM5Wi=n}Q zV5SE-|18$tk-|UHOG8J^D=^(0SfMmgWqzsI@A-uM9>grAp%N9RZ?auVfcHC)`DJ~o zR6m~hR6g@)=WgS)i9ISg8#uj?)FULCKAu=4s_T5M`g1fVodBC6HDX_xl*UF#Sa4m} zKep%eQkKOKvv}Z8hCCx8VGHM1D|*6`XlPQkjIq^GwLCRhtN&vqep0?=_1SSU@2^9w zF4UDX-Cf>(YM^x&VH}ijM{SuE}3tMMJ{Dnat*?KMbJvXzA4%x`&E}dh2 zX&^n0=crd*MxI*HCZWz2WT?+xnHXRW_jSDQ3;ajN4oOn+e(LvUq$C+w_1JL<=2T(S zC_=wa@d@!|xo?W8r4y@!nNeCpDF{dp+e9B}G7tQQ{|Ftpcn}3s{bPmWZ-2W|Y(?7u zKm1aba4%uhP4f@XrNXh83s{{2MijTAF)3mi&X^Nit%5Nev^=KvxF#4lCQp)Ii1A#G z|Xf$4gFN#Pq&LnV5ZK*XNgu!6g#|$=w`#ZN;f8El;j< zEkpzfqDUx!0RQu^PNOwhqQ;Z<)0Buqgl682AR7g!yEx62A1t>K;4y@~+SV;5KLZ+X zZQ5u;2UY_+Bcn7RnJudD;+O|9Gf$9OD>IG3t%ZWwmg>2%%Cnq<*XR%92BHs{7O+z| zHoF_|O7mW8)^$Pv=j-MRm(8kkHklKGtjwZdt*3ytF+R4nKVFXpxUF7HseBky=cj@V zp{%NC^e&p$Ay$gNE9Qhrl`R*y^h0ggW`&_m5Kpz!TDY$)rCw{u#??-1HInoUO0-m; zxY^WVUu$TgOG;R`Sl5l+uZr6^)^keu0D(ud7^{Qx&R<0U)9s&uEghdS=0(zt=fWb9 z_yXO~(t<7V=u9tM70RWr-~2CF*b2Dlw{+@#y*+>Oga{(%zK1QZDtx6#N+Cx5Hjz1e z*zMvf*PVOb^}Hafdl(Z#>qd3)n7Q$Eo1uc-o=DalCJ!(2ss}Zvsrt?N-jXSpb!7(^ zc{P*)ZymJ{+vEuwgmgi%znpsiDs^Sp@)mq=;8X_rzZ{ldLoZX6Us-7WhDe&-_oBK?@^ z^5{XCgTM!dG?o%tyYzVcCnDCTi{}tfilF%L*r+zZYb04Td2utmFAqx5yL`Pm{j4B) z=5frDNid`RrL(F@(Uw@-q?4*-Kvkk0XJ(GdgU*9<*WrANX?$yt&Rf$U1pbs|P9ZS4 zqfD?{z5HfV9G?2%XzjsAV|qZXIWN2|e(_d~=Lu`Fo3*I4%7GVv8RQ`?4|K zTfjFu9lGT|Pu3O$C37~1)l9iUcQ(sT)Iv4@9{6=o>gLl`>>Naxlgdtl$js^B5d=dC zGV*}9Sab5=O2O?t=>&`~*Bv3ncW(h10sAr(DJC8aCHN!csBT>ogWc5tX{f8L!`ute zagl7o3vD7c3nb+ub?TY8o?8!WpA(R3w?aFy>^%u4`Qzb>U*QW*<$^RaWgb!%W^{3icN~OvKM{HGkJVI*A=o4S z->57qg7>n#5VGcD(&X?Z`!tg`_r5sFw_k7DhpXWlFNJCskJ&d|Zv4AuJdY#*T^&*E zR9c`+%Ks>4N(;pdX?0%DAf`jyqISG zc3hv5(X-y@i*PxDuQ*B8GH@B(E;*O0Jm$wNkqZ0JT%BwMmE<{Fo2av4CSjd{rlCDy zknXNG&%G4~)t2_ahQT(5VLnD^UfFt&vqBV8wd@}NDNZ9hGZz`D z>3PZG%f(WPoxRhhCQXk4XM;M;R!1zH+@}$s&E|cbdsU+Vf;1+<>>u>E#57U9w;d+ictqgP zON;X?p0L4XF#eSCE1@d$`(e(t2SGO$8HL~V*U@<36RvdVKaAljrj>l9hrSfs46OMO zp~DEH3bdiq8+kAtCE*UCyd_M^_U7La_v?R`+=$M&T=yL29?_OVI;n_7@*d)fb*sIucoZ4?5kUn(1 zM7{_SlrEn|dsR@JDH4<+l@D|a$S>A$D^d1j<+zwzl>;9-@A+nDs##@i+#1K5#@wH| zm{w)#{wHXH?O;l8sA2k@zK>3yhfpNmSfb&`hy6e?O4!_2TCf<}8)ukH8%BO8$Z4Hd zes;O6>B2d;NVnFR7aku&ge}C^nBIuSP(J*$3;DI>%ruObNOLwUE zQG<1d0n`pjLu?${)boN4RwRloKM0E8E%vzgO&fqcD#QXFuV)hezKmK~yj{Ak<}#D; zT|S!x29p$#8ICR4>0^NAZNM^YNATwP_cJelF5v?m##oWjEF59KX`-OX_SlMnQ%7H9 z-hYjqAk~AX11_HY8E?&1nyx7o1hQw=S9PfuK#%9m{f=p@l-}QC5FuG@goMXu7O1gk zR!OC8IJl<2X%^cCVxRhguUpFJX2+BZymjhommXfRi#I`s`(hjc{6aS`Xd84X`EKS| zyhDjYR@`a6seW&*{IiN}r%c?{7}pecSQl7j>vugpA2){3YYvq4?IVlNx6uw$AL_cW zz{Kg*RrhG?=iy-kmObI+1^s3fNAWfxQ1))yoN`gM?o9#bOgzr47_W~*Q4&f-k&k%1 z9Q-B*^>$OC0j*$w;Iom1>>+_OA)tLLK{I@{@>h=|Wu&c*=VHv3;%$gd{LO)h>!nC% z7OZMwk$Bs1v#9&^!I&9O)9XX-_nU9Os8MEF>|NxL?bf}G*HRzsq|i)DvAyBb66Kwb zTV3abic86Sd#C9WtTSy%ibm(inZaEiw9@Al=AB-={l@4g$mw^gwv^pQDD?kQhA~>w zSic|@pe+Bf5o4r$H4s{Ay`aRnLD~KbA%<&$idF*U_%EKg5(=8Z9PA4|{J+V?H@47X zW}sgo(qGxqEiwzx7VQ5W2EAnz4u&4}|BUdB0LJATC`ZdzH*Z)lMy`~1CYW$Y7FG^M zrj$r#7{dR#t!9RaNeRe+#K&Q0hhq@8baHkfX5--aKdxqCW-jKGzqw#PfGRB=mHlB< zzo(kik;B5#4K^0FGF3+{&3;@>Lwl*qK*oNmc!s(|h}(08CrS2#cvHh=wGK(ECa>;~ z_Qg%o$RIRv(flxEuoze+>Hb;N0eF-Ull1m z2@N>Rt`X!i>@kD_LO%F`{4W%7R_6pNnZ`e10TzrP!ud@B1pz?+11Emf?;UulR0xj* zvJFRL0x5{<3OXqeSZqTwB@g4BWOWB)p#)-@WVGHW%3owL4cQ=(KnS@YR4OBwR$V%F zIPYo@Ho_I0b(mBf#?HK^{RqnbpBR%trzB!jP<|N7Ftv?)h<`tuWMC2u5j;pTp&D_i zpvv~)mYhO3fBa1*muKGDoPz!Hm~7URxDQ#-x7Rv;WD>cDk!SSGzSIwW~sD)7D&VyBlf){7-9 zoOjpa@2p{Pl zp8zO%FhUCOv(a`~NmIHE0N;3bgBgC`k6Y7Iz1t}FIQtH4!fZi`K=+r>u7+FJ>9J)_ zcc*{PPYv%y&SnkHaPWL7IVr8Jx+n)xJsL|F^2(`LS+JW;8nSQ9|^I$*=q9A72!ven=!&v1ePyssQZAqLcUnpp{#uyK5QLGrbk z+tU}B=!h*LdVMFS%e++I=|~tQq5I=c?h->Jm9JNwJa}@&Djfjk!XKMy$8Gb^8lQiQ zr{@F!3w^cp1Rgls)6ktAyiK@!|JI)8f;l77ZKn2G7ii5}KMU8WxpU+HapBIABP?fj zN}N6crf3O%c1QhdD398pjUb`t$gXT6NJ0djX zRj$QA&FiUKzvuPccmC2GjuAefgA>OwWC@G9)k+Nom1Z9S za{I~Pi(Iy`=3DN=2t~ee=k*BXc$HT%_6du$=PxT73(U^ke{(FZZZi7)p7?>Gy4Dk@ z{mS5*8++el&tM$K=jfb>1n#vHp(%d+%@8IpDV1%!*ryjMcPu<#6D{b(in&g)^75AVX+SG21@3_5#pizj3nA!cV zP6w25m)1to2WqgQ#pAqZh&#-?gX6o=^--L$^p5YI$>Zh|$ zr)OW-_-XqI@34v!i-U|M0p80eU`GDevnimwQ5|4pjNR|n=YQF_tS3GLhqA8GHC4f6*qN~CtrV_8HDA%mLD1DPu@(tiY(V@; zuoXPt5)l0ZzYb3)%VyaETIC%Qn-`9q&XV<~Il(3eam&EKrxd!V1?elLAX~(BGu4H6 zcoZyb2?|ClO0jjBl((JRdtIOgMSajVh*IYv%3XO>2jszB(c|{{R-^}1(Y|-Nx_1>q87zRI?g<|*v-b$}>$Ui<3mY!y482d8p zyYtjxHJebK$Lu^zo$8iH0P}$Y)cvXL&TD)~F>W3P?M^kp+I1CTjF@;2ZD-{TScN z;h(ZM@_DLAC`XMi4;;`rMp8qUS~TzRjlDU$45-^8;;gvn=T#v9Dk2aV;@_??TFY(K z1%4pgoi)jJh5TNe!tnj@S7_NSH_9R~SCk=WgkxLm#H`c6q-{3|L|yR;k+F1sa9 z2;mo0$WLhC6pkDUAm>feOk&k_JfVpVs9MsstO|v+Q$JOvfeSh6hlG%j5|PLy`zZs$ zr5n5jtC?=yjG4URseC3_baVR0G2q|?ik?h@!{IxZ6gXBwVvrF;u9|^e!5j}w)Ylu0l z#H|jQHKpiG3TA<}Vlfdp4!aqXRM6t2$sgd~gvghjV6HXQCIm4}9uYo}8x$t4A%q6| zZdflB)f`Zew}F5&KPbLb$D@VztGQ$=@QQ~eeP7Yg8g zf|vC(tJSyRTN4L}gt2KPCYk!M)MZ9al*m3_Zj=aktB$%aY0{?4s~NVH`@o}|(vXdj z$w7HjWz8v*&7njEC){|ftfLc+0K2Psi+yt5Bh-%HmQ}zFi$*3DTpAmbs7<|NzU9wm zwKUd*pB?jq9E*w_c+KKwI*@Xt4eWqC+9ht{lUJ`WsTh9T?v@EQtI3Cdbmwosqk8@pS;fQ4Hum@a>$=3;i}vSR@L&u>+hkW@GOze3USsu>}Y- z6&h}`4Pr`*PCvnOT0~XhOzl$6nP3P~vIStU;F^Tm{R%#V9;$ii5ZEN z;TWXs%OJT%q7Go#>gltCM3efA@&tx!cYAF z-{osMSyMZ67Ykwz7B1qHn_Ms|fMB6R0bL2*|Ax!m+)QEtGNmNfzrB_>`N`YJGmMIu z_@7`BhlNRAU2W|j(XxdDvM>4%Hk~c44SZe8oPfXP%6d7|u=o?tRIGCv4@(D&!O|V? zbpr~_kBX(h40iR+rRUx`h znjpo_e;gh95YV~)bP{QZ{ogs^VG)zUGQ91~SCUB_BXevlIi!V|k_3O4Co%nITCGPN z-^GR=&jTyZQ{FiGr8n~hPNP^iR{>uL>)15Xx$w^PPjrA&R)l_t3&FHNcL(O=t$&$~ z3JNqvtixVf;5CXsy9i^s03t~D!bCYUc79M$gM9C{Sfl;K@Ir_oMiAmg@bUwky8f_^ z9UX1~4T313Vi4+steK%Wsli;1_AQrD6^sm5hX^|Ky<)8pDZ6mA{nWJK-VOE{HGQs{ zAs8f$txIH&e3m*(M&Rs5-_9C zzEoy9{7nsRp5$gTTQ9ah>3^#cok|u~>OZ0xQlMS6R1*}D7Lg{`CiPoHS z*Rqg)TPPLd>8zvquo-#xPnC7^M zy=8D^s1JSC8_%ahw2iP=$uRzIwiNm0O~-C4U3)iOOb4q^X|tAGo8v=vPj)tO^l8TL zV>aPq<}G0Bgv#Jyq$0cL-m`Wf7yT{OdWnh)3dzaI#MNn)94-9EiCN>9mT-s9K^Tz#3uW1L-cjxRqj;iqjJ8U}sE zWTar&#Fp+Z>ByrAN!-c8#FL`#P2>KvfG^XtEiPS*jsx)ifv=P)B(8k!Ng^vmkb^vymn_R1%NkCo(f@s- zz2plT%$WL5f39M33L>lxp7~!L=5eh(ugROgFBOi$hoWs;p0{3Kqa1}nn%tS1PnOvG ze%3{|u6=87p%)dO*iOn8V6$(xAgd;UwU3HWT3W ziX(pHwNKgFK>6ixLM1i{tC@*K2pS zsF%X4Ws|uccmJ+Ce(~B9q9VXO6Pkk{}{+We2s(Hth7&rAG)}+@JJiqkun#ig`5~Oc?IR zb5gX3Kcfyr@{7y_0B;WTdZNELuJFc8(6fQ00ACA28qgO zr@69Dt7s3Z5~5humqA%Kg5EK@jMYHoS9&WqJaM8~@H=T~2dcEwVdF_ziUcL06)uqm zUF`{~7=Nm0SMg~O!MepH1S#Wny5fIW43EseCztqM6z%i?tqCV1AdZ{t)i5PL9+~TP zEM5ABz%1r#3avP@!oVWhk$Gue1x$%)w zTU*6w;E@{w3}Pk<&_g0dDj*=#9_ksed#qICz#sBvylM^zx=;#OGY>0w>GBuh=07k-R3Xd< zZ~=mxMy^%I7js(mY_sloP0X~vbLWC?}HW+8nax5&lRSWUWi+NtqmgKZPR z%4H-Vg~v1ogv7_F7z!8t*VM%x_pj6q%4pd-+s_oGeLl*clJNO1KU2!%aaUKetuDum}>&0_~$o=hv82 zGtK4S_@`4jOe?}_2ow^N@dk1UmOwZO5BgiLi5Ww@Y3?i$ZXlhD)$4kCuvMinaOy}3 zdbWYfY(f;}8G;0!PyOV!!dKQF+KhC6(?@S|s63o;!t`B>c z6YZ&m5tjhulvl`cGQBIw)qeR9K_D2wSAsT~z^9^4;eP$s;N*#UtBJStuR?u%%1Z?p z9BvuWeISY&T-v}oW(iQ4xS$5;S&$)1C>-)(zvNgz;-+ShCkRB4_ zp(!wQ52;hc9W1akFvJy(mJ#dK%a9323_1IAO9mQlv7C zFu)<%v2cL%_;@eK0F8clPn5RhJwOrmzUbqM4ct2o5!YHS`zcW4)P0YC*$Y3ZEg9Mv8Rf8jnelDv_Hv5qR51929Yo)mE})W?C^O|+{LHR=Z#$3lfKBwSx_%D~E) z7qoMY2?u5X(f8{g+NtVmMLZ|0^6S2jyjI}4O=S1=F$*+auzEd8xsrhGh5Rn^Dm9kUy&99&m!`jcbBoQO; zH9h2>WbgH{LqJ&70p5qi2P6rClIaUkb&1DKVG{V#1_K6BO9!xui<4#gaml=>kOxvG9 z!-Q6C;G zs;yYnOEuH0X`K?(#Xpy z5TaTy1uF)ecQgjd)(RqDB_NA35++N%abBB_?!t;$LzC)%$6Yt(aUxJurIHjG)#oz#O%U~6g8r# z(!&57$t9UnzU(T^PHdqdJDYo{T|cqsr_MTKOZj7-)vjF!_j=vxd5^=cn@Ptx_z!(r z5xP(7!%YnTTnvfV6A;{p(E$oi%@D;-?KTNm+IBR5N9yPz>aXlI61cv64|4$ptur$3CI(_aGdgX7BSOGpnzu&fH8<%tL7CwAJVCI6>I!&OAGyNq^|gh_ z6*oQWn#ep`LX5E}k#ul*m9al23c7;a}cRnKaU&OKdI_Pwz z3J_FG4lW`_A_rq@7+zi&23a$E3s*}bW@fhZLR>Ilyd-lTBH+t+^ z+nPq*Kc-wQV|0y;XSq?J2*b>zEXRb6XYNm{7gbyKM{uBN{%OqZ>4!U$b9!U>PCiK?1q9GUL_LY3A7N#| z&@2+E=qm9?rgWAjM#KV$+B0h(1(sBitw?|@mUYFElWW#>KyJgh>PV+g63DDhu! z1lv~qUBV~{kJB&Rsf{g^I$QM{#F3j9wKoRk*7!k^9>pR{G?D)G{;zC3sO*PA7$;>s zzyr*;&AIwvXI#~1)ss-`8^Y8XNrWP6f4-jM%vCj zIb&S#O<9Fa@kgmc`obUz*{m#gS@+~5eDquHKP%T!DHJ5KMqia#VdTRYqEw$&+nxhl z`F@Fu2FSjWiy03OUM zC4Mk%fFJEc`JV928u6r41>4VNPosP0i5>PzEL>XlR^uN852aCK0Z*1+Hu|oqp-wTn zIF`jrxsJLf`4?IA(-vJwwazfxr8Vv}w!>7?_ORxXR~<@jg*P%B>|`rJgu#X>gm*!T1mXy(FX@G~r41IYj9UkbQZmeTDf(WtYhC{*>_zA71{nJ-PO-uq~&)xx| zvAuV?fBL-yS@Lwy$#JR1h@c5Q@v8?DO`J|U7r&0ptO zQ$D2&;>ZXnRv3z;VeDmFW6Dqd1Qr-V1`V;lUAld9qt+FE(7j{B7Y1aK6 z9e`X&Rq!i``%Q`PScF^2!~p__FvD;rZXAUh2^+&kuVoPx{a%%EP)%}N&8Kmtl~WS# zwTuM5$x~f}Ol*2iu8LW82t(zOLVX2t+_WYJ}cF8b4Z3b98zxVLQ$yFlj zg)3Kz7dL1FP>LcD zpnHY>aKAUK(FE{VX*zB*R&JrZbd(pdj@FCH2R8i8LE*|p4f2OvitktTHeUaQ8PlZV z%FDsP7y|WmXMV8nrq_Tl`R{`u1&HMjUlc~8N4EO*l6vvjP;8;g^%#fd==+YbWaiNa=#g8pNJtj7__ zZ)D8OhxhQ-jt%S96NhVD5d|qQf74H$egeV+2*C7ys`q51WNz%TS^Z!!e~CkM+ZpO) z@bS&0`I$7U<-{j4rWJh3ubD)?gU2~wZFpr-6y60cyS|`;;T91C|r4NANl%r(zUaXhj=bz^~+RT zE0Atf^$L@9;z3G;5b?kG2aA*K|K%SRMkdz(zCw?+bsVybtRG4KOf%+QWHtj9bh*YbRWh}XCis9bGg$=*<>>Oxn}#H z`v+Ig686bK=;9ei2i*J0%p<79bg0Udm{UhC-T)o8m3Lw3%-#HGUN@7}-L$8}OGeVA zGJv*vpSiMKRcZd2J*chQ>{8hV`;W#(9-!lG|B4XP;+OA`h)l+lBtUbMmr0o=ck0O6 z@8w-4jrM+yFezq&MBxVijC&uXSqe^LLc!wqMHY(7*mzwAjg|psPHcrW1ie7aSkuVt z=Iz;sj#DM{A>a~-eK#=}B*FyjIj&&a8Gzu#gj?nn!eWAb1U(D3&N-Z6{>EfRL)f(9 zKt?tt%b62fu3m*~h;S2_&XVU-KxmRe3Ux*Cb1xgl#WQ3CN;l zrjX5}6|_^AwZbx~r%i`|az!qA@Dn9uJq#0&5*w3DrETjB_li-h)Xb}nya4wmBeXNo zgs4DHh@4M`7ml8Yk>7km$|sqMv~UCA0i6~J!Jm;uHnRwVNrV223k<9`ZAtrv@eVDn z65y#LJ7aDP&zyWmg=3yffHmybR1whh{f zXn>bE>yfCxjldR#7em;I3V53uukI{&?%U2!Oz2>BEJRI>UWX8}^Kbqij-ReB9s?jp zEqCGq+q&WlxD~2^;HIK6CwvYDn|N>cDONDXm9gU0FC;hUVJ~*&i_m(nZ2%G7tpAxszmrZI3r*5Mlxn{1gsaLo&qzNZ2%iSU0QXWx3ux!gAJZ``pUr z1yxB9oeCK%Naz{pcQ_W`YLrYl*t%M7n5Z1bM$Mm=1}dr!(esEn=0asgGT&JIgN$V{ ze8y6I{rtyOfxA#aiZXBhK5@YbMmtozz?S))3ksC1}vy3cQLhBgNs;Baj<{v8@?Q_YHP0nLL^>^`=?qqYP z^WkpCZsQ%kt*c&)3UK48{WGSZ`**VSZ)H(+7H|iRPeOLAXT2gpXvhp&VDw_0fx3p9 zjspjDI-m)Uag!CvwPmOIxe=y01fTv=ZiDwjO%$;5&V&N``g>jbpDl$*UXe7W z6~Ls`!iFRo*bASsX&uwjWok#WC0GpRF40nT%#qlk&MWXxg4oqN~F zWH4nJTU1HVq@XBt8JYPVq@rnIj&qO9zg7*?nmFx^UqPP8H zLABmWjUE9yRbtUxvEhW;#&y=kvz`jw{w5dm-_hVVK-n6)e|U*><@m&eETa;dN=V|O|WVFpcw~<>J;rFvK{&rXqF}L+Y8+?d;7di8e z4w7n~y|9wi7#hCP3l-gNP?|-koT->pKIr`bDEd^Gyos!ElL%U4Ow2i?>wEQLH#8FS z%QM>2>u9?18*j|v!=n|oYPGj9`Vx`Hj^H&bi;m(9+u7ht(;2M`dS*21=3PS0%4cj3h)EOUco6x-^JbY~$v0 zb%OZk@3-s+K2^>?kbn=h-~-kKoIa$4^G^GG*Y_&8#Swu&9&$+3(8!vOfB3vaH~`+w z0H{>&ik>O<#<`#*XoPHu#?cRRjO&->zfy|$c()g%w9<1$Ej28%1V9F5?p)|)H& z?c>{QWn2XhU&lKW9nm?>vX~WrCtB#l^+pUx$bhAx*Lab-J(W;ViKue9+(?(Fwr(zl zK|URJ-Ty7+6k&`w>fZC$F|I>Xh{7A)1Fq5>>mLT3_$87=lpck+KM{0dN}J~LADy-; z9TMy~2JlzbTr7AV_3P;4wXTaWgwZ|f2+%f5HkL;%N-|)68>m40}rrp z;_HP#h%z23o$i&_cG`T;mliL}X1wCQu%e4IK_xSba8vn$ePmu7gw7X=QL(=D0ERJ3 zf4P31db5Fox@va{k)!Qj{K1sijiTiU*DTpReY46F#8l{q$^$Pic#il7JM{88ai5ZL zFzoO0pdO}CyzIt|zahY1bqe-!*yUgn^&zB;H5sazW3+$W8k)~!;s*^l=7%ffte5Bo zKXqU)?di@eu7lY33nUXELb_JP0IfzInXMYff4-V7CmuS5qL8OBz+?xU=O(E7bmUoG zT+@4N&rS10^i%CtxV_+CKaVszy1)W!BjzuUT(FKrf?e5`A@=nZ-_ye3{!=-_oXF^$ zV`dQ!Jfk~> zDLRF-ZRVppTPEXYenP}$(Fp7fjx|rK2V3<-GAv;VnE2M|RF#=C!+29a{!%qcBneGs zC!WPek+RE)t~#iwqx#PPM>!UL(o{J=QHOY(OeJK&-rMehwI-Y8R`003Y=2ev#IM?d zSEi&JiJG^eo1vNf%g}M^+Tf)AjcP&-iaXEiLKq(E{5)S;)>s%-bY7=_OnE|n?wD|g z7gd!v$Q0EMGX(t>{vcc&tkM7;tAvLF17ff9!E`|*Qs(Hwj~*e{ z{RaGV-k2>a{8-dZP*;-NQyhJg$efC$JuF4|M+jaswptR_Yb9$~^wRqf<>a?W0o&Hd74?ZyM#>hO0zf4YNLWe!o@hT_p!-%1}I(_Z=+)1FFk zJ5>9IVfoJG1tY+`wQFS&QSVzm_@Yqv-`}cU!7CcGD#-6sd$m=gv3)l|+Wq6G-9HdY z^HYJD1JqzTTr4wX<8|r5<+5RiPm9Kfbg2Y%twW9h zcdhPc`{+FmGmk^@>0xkg7t<8lPgmbl+e|{m%-OUh#=6ftwmgACvLA#zVn_@B#ec5< z5B@WAas6-nzy1&YZ;AgG|KUVwPe-v^q2?*F!KgP!=nud#EI$Qhnq@^X!&44S0pI=> z^l0G(QmspV)Gs3>V=i2;kGU+)gwo3=eHe`b@$! zZO>t8fY-F>eJ_7ddNp&_c#ev#Rk}z?JiIB`w26hv74v=hFFs7?4u|jet?yaB?SJf# zGWKO{H!VN13HzFK|toUd(-f$xO|i50ZSQ68Jxdb8JvKn#g8IS-;8RW!ms+$EG|I(*q{HJ z<_=O;i+CguV4r~f zxyL2Z=Q4BV;bkv7qg5^<{+@TK26N9Z2%R%-FAkM(B`c&CM)6u!4c?WBA6|bp$^suF zlfxJ5Hdw0OF^?!JPn5IvCuna7#;!DrD2}Xf`4C@YWYi{U^pdhi0jV5`a(%y@OwM$N zPs#xGX?z;XOI;r^MTP8af|P+9P}$)j70xEi8sbVuX0x++Ss$D8;eVjGe3 zCux_H*pZG|AI&o%bGm2fp7z)(*fr;YJv%(%O81Yo$|5R>%2&Fz`XlQcyLvTbFN9NWz_d|hlX*-a z8D}4aEK|cvIPU#)ePi27N)D2+*#J`m>+x^Td)?)XX3Ap#(Y`T`wKDgZJb`d;536sd z-F{i=L!8c}Ra5nr^_ne5u1}wXdmA#7=Eu_?RHrO%D6 z+^)v3!jaL0izb;`6)7#mUgMwJ#<6Uav$xpM9g5pf%h<=xsf?Oz@bI|-!a{n;*>!a& z&0m8y`%bKlC89*JzMdO-D%lheD&Y|_3iLFKm}t>$6sH9=&-D3v03*q6Mk(xHe|ST1 zL5c2@R+2b@WGLPx^lz*rQ7<6-fl(KBq+U~0a#%Db{cj~VvNE~Tq{S-hbT?dOj4F)Z z=4O7N{!$d>$lg6{-laMMsC`Jn2e;|(ySje!xKBA0l8L}Q@5ToeLR#i zreYLknh?yzm(FS(Kt|k+oWv1@F$af8mW#voh;QF!hlim4r%!!(oxkB$>$7j$L!H{X z;YLW&YV4ANR3jLre2%9b`?}OMld8jsn|liOvS{dwwzmp+RmUv8>s}T-R5nj_>o0`| zH+lk^Fa@)YYyU?x_NF0AQRI`T=sH9&-<$c??BB74v(@Bwz|J&X^(xuEZwoHsuy=B! zh)Jv%xjzzYx4a9ovBq(9#-DZk?fib#`gU`cAUzMe?~aX|o)&y#uM_gHWKH!yK+xIX zLYifv330Y~eO#}Q)k`CvQ$?|>eUkrFS z*v`AGcK0ag3t+x~2HH?RRTo)a9-E^+Rn8$2c(#==01Kj8Wtauui^Zg9nLrC1#2ZDr zYV$+F9fh?T+$i8#2J6#_?31P5IK5-a4@gs8E#j{AJl;ESKALK&Q)3%#uN~Wq8?0I( zc?dW<-`<_s6UmJu(#CPWgF602TwL&*R+r+CAWM54hON%aR(oE`Uy|l}H;2^h?dyuf zldx4b0Pi-=_~7@Wi^@pyWou+Ps)!!f)N-8PUm zZnS%nh{1S>@klWJp6cl4yQCxD+}6A4-?Scmp_2`+ng18`|fq-=#2Yp2m3B=ltOJXWF<6^2B>UIL6IC z00p*~`To^X!_f+3Ob92&!R8?eGzUb&F3(Q*_fGEX{^t8_PkhPtIB3bG#2=0c>aaZT zksQhEEYkVMx4pG;C29^T2t|Jho46F)Z!Ym7Ji=TTmy;Y6N%cp_Q^jOpM#c?Os0iLQ zEq!ZSC7?{7w|mS>jtHZ^{q&_Gv?Sm#0H2&$gmkM0>R+xU(F7(93N5!!DS`HrGi*{M zK@!FHB`OA2^P}Stt=`qY5#0U1yh0g8@}&1+2eZfMRcT zBr9h9N8c61N}-ZvZT^C=T;O{&+G9y|K0$&K`R)$orPQ&IHokTJwI77}9*E(d7sX=z zTxW1G)%Z98bHR%mV0bLl(3Ubw6V(%GskOw03jTh-bQiY@5MFmI7@|-VTlmUuyVKW=3VkA7$xfh_rC%GGPsfu?qC_XvYW*}Ig@+d$5sz{c%xN2*C zdXuseKxsKhOU)Q?GxB+{oh9^O) zE@yqnC!D2%#60A=8Q}9k){C*$5=M)8@8A`}tU--q&Uo@8u@vkm$R97ZvHiO$+vv*FgMPy~?Wkv7|u@m-uKNW8f?##k$)WyM<){FSn ze@XFq`O5;&)jHstP0ghezzH&)nrO89EWt%z;YA z)LF#s6#AaV-Lkd3QzBTH``X(r-_>Xw>_BTP_~>-~xh%#VfzSkyGo$OYyuF>&e^M8J zr!q&15P`jZUxKvKK3vXP*nKg(WiOC!k)Rg8OBffz{?SJk z0HH&hnC2!DFG^afANPDTt>p_aKLRo>-%t9Ly1-A0oF5I~kuYskStpZG^;5xKG>=qQ?NHMaZo?>zg*ZU; z5eNDYrNc4#A&AF5DU}_1UpXVsE^6tg%vt>@Ne=eq4)TW7j_$l(dV3qnFiZIF;O3nWFJxy|9BZ0x}gH#xj)YVt&TC4&mhZOxufPz>weLIo4EQ zDE4R3umjrjPiB%-0tGfIBmLUI>77i@F!|SEv_P6bJ_2NZFCx@K7v@$V`mF1x<-|?t$n)7CoxuMPFJT&5Ty+ z(U2shcNQz9KrZ&g-U5xBmQEhzrZ#c>;cg}85&KZ;r_MLsm?X7#pPnSoyvb>w=}%u_KF5;JKir|=%kD2*!HQHf9z6!a%LjWR0yLM0GaU{@Z!vS+G*>dz)}}_cpXiKOa;sspNZ1%mRZkx-_T|Kya^&tj_4C$mljfoi{-TrRCam&+Wh*W zev6oTZn`DNMq_naHBwQS-nHf=haU-T|lDg@eCxy)yswaKG$dH({9uD?_HVYDCd z{?h6&vpateV7!im2gricJ!F>=*4?=2U_5U7g)2%=m?D#*5W(a+b$HB1#N-4brx>Kc zMQ&XW#(EGOVBZqU`fPTn90Y^%ZKPt{m;wCVx=ZLZA6<8CS05O8bE z1DKoyK&QzSkBmjZKKe+qQKt$j_KeIl{@TYkYi7^d+gO0A^NBY`>@4uIHm`47*FGFT zBxR-@9`8~7nXT8QFdrDw>8gU8IN!A4O?YozyNd-*6uSM)>)r4ZB?_og(%lV_!NBB4 z4M^+h^O7+0tdjxg-q>DWIHRbGx3RqmwuQj!*=4W3boB@@E!!+DF83ua>lHbtQ%NZX zsVxOo{gl`&UG#JaHI#jiisv&lpt_8vavD*-u20~*lPFc(&9lQQg{-;9+797g3!JHI zq*hvjD>Abd^A@BTUnHY$TGh4KR9%9X!lwHGp+yi{finO^9EyjUQ%{bdk3fwN16+}P zQZZvr`CnSg!pf;lad)2mhf1+1&A6J&AuC{$#Xa#bJA$mysZ&GxGABNtjKWB&6-~Cb z8XqXtOQvT28LsiQ`Lkh>N&5}R|4FvMQ{tLxhf)oTx5=kmdq9Wu6X~Do%TG}kZ)y1 zIK~M_yxNy$4TOH6V(gZ1pzsZ$>zzw>>OeCFaBNqb_nA%mJ;wH(Ugv~Rk!kgWLv}dO z82uuL;$-dX_x8sgg&m6;y1PX|wgc3~SnaceDZv1IwT(!+bm381pdN-&U@MSsmJ@|( zn#}~#NKSSzy;e9E5Moe2`eu-(7q;;_W0Hr03AV8UTo@4Aow)l)K${Xc{3$3SnX-P; zROQN2ReY+n%Mr31g~%f0avNJVmHN~ixnGuGWrhuwD zbBjv$PjACZdtXV9-c{|02Pah) za4&P+lDV}Aw5iv2cYSNqlyzwjg-?A?85thuA~~|FG&1zh4zkxBqgHK6o+}JtxIzPZ ze>yps)Z8yir@z5w@$Q{p^e!eCN}d8AUv<0SaQ8>PYgF3K5#e3A1PC=B%~K~8G~vA`Jh6U`RF5A2CvNSi3(SADhLtTHIUgML4-+dhufrkPBh>$LLaoCY(bqzu0hAy5s9@+B z>$m>ngSdB@SQjM#v3WoONkRdkq=H3Cfdm2x5h)%xkW7l96hOTQ3Ib<+0kNT>oJjYV z1vo#r3$3qr?$|xOqVz-Wg9MO~4SNE@L67i^=%Jvnpy$E5HU{x--9rNKfj+39Ku%7+ z>(FwPTkY-TlR!Z7^76h60TLSiDbcZTm<;`>`{uCo!5xFTyoQSXb!EWXh7#-ldhSu{ zf#GbCYXRO*N{$=~^{{Y`keozBj3t+PF+3{Z@B5q(aa6d8vV?b*oL|=9~E(1^uAmoQ62wzvpzZfYY6d;2#^vqe#Dz2O$WdMm;3l-pK*N z96PUv6B9OvQi090Sby>P(aW@08wCL57<5c0iO3M%1@spb=oaZsfIq|B-sDi_FjiWd_zy~A@`*| zp$7`s>Z2rtcnlJPDx*|NkSKIVFO0<`JhJ3_Ukw8bz4gGC$lt^M`$x&g_6&kXQqLcA z(u;}w)ywyL`}PNJ-7Qq*Q#`%~yvKmsa~j5y zg|K7=0RC7gS3mxA;*dcX(JME8m4dk~PqJYxH8*{eV9$B^>H*alZlt@v6a^ELWEjdW zY0CCb@-Nk^S+`c|%JbX({4S06@!zTDS%0C#)Y?oEeb-@Uoa>htKg#FiX}iqj8FnR{ zXA9B*2HR>oX;lrUoB_(%LZO`1T2OAO(u?eKfQrV>ZpxU7RTP>kt}8<-!55D&AD<^5QBxl$W^NywihI%)F?JkX>OE@4$fQtI&Z_?vlP+%f@u@4k z9iE$*g>#n6?quOVF1Bl@i-|XT5Xas!0wAfEjt)3n9N5R{_Wzbq@^8dChr12C)FDuc zVb7~k!%T;VeQm)+T`9>l^wh#R`YQ7+ghXC7Vq9S-{Frq0MbH#F|Ni-V0Kn)T zOdq==*bKoQOYBZm%_Ne_BxMT<$C2}EjAO-Fj}EdXzS6vf?-}-iMRd<|La?|t;up;? zRMyb-XqM%eFh4|G^#65#fKEGUR-So@v4`hYE26sJog<|g$9o?cl+pNyZLH}?7o^)d zWLWD}qakfwR+G{*rRRf1C2B-93^>n+@byfwmL5(^hT+IqvE)q0J1ZNsyXm*vn11-D zwFV;_gWneYIH13bbevd^_G|S_glQrGp0ehS32U%ZeJL0JK-p}Lb{gXdi8~2;3wnmlHh;FW4-T9Xm zpYhl*g+q_HW6L%ZvQG4_2f*~gUIG(y{ZAR&{ViUkUKV9u-kI{?i255BALL~dn#W2} zgfkKyxF~rOtWLKpoUSnhZDIOZ(9wn|H&fa1ODmw9!~x774T=? z2cIACXX(>X20!r1ox+CHThwd*$BS(YeJT6zpb%-jHqy*%`E)WIfaT~o;%29|XyTF4 zKs++-`fH~L)o$A&kB0#NWt=|Oh_^D#|ISdg4b{&0Z^@)#LFM_;7%+v*nW8i8A}#m2b|#a{q88+r|~y5SPqeO2nRf5 z$R-}Ukcj;p`Gjiv_ef&fh^UXTS=$z_)jeshm1&L1>J?xJiT;CgI^b*Oj%W05Hti!nc~$nl?pA7qAuU zddnP|?fN_RctqAd7aWsVHD>2Q_V@kCMYctq(#f}a!t$QNPrO_^rbdcd{e^d9<4Erl zY9s4XB67zSTXWlkkB{ydLAMxa?v0oJW|kG);aWrKfO{St6Jq31{%KdUB^>(kEQ2rD zYtoI<_n-NF1MqwZ?q*0%ic0iu5?=jn<$X9JR0~8)PeDsnJSdhG-@3)68mc%;PBO(0 zY1L0mW6HDM37#E)HWBgqyF#K{!e1Hq_VnR(HCHp9&1`F%NgnjkT;~UL+Crq}sQryQ zUfj43!LZtzC~tR38SCuu@`dd`O^hhy)Iuq1DZaCt1aK!fC17}w#5hbv-1})aKpJng z7c;2Ke*it7$?Od;(GFsm9LYjtebyE!2eBuhBc=1-^Um5kpi)82ndJvaA8YvG=sdQX z_q`l=GQXf-;AVB&Pnl51pQD6DW6QDDQH~k>X1HxBQp7G-EmELvhJQW{w>?7p-0OO9 zBmfBA0FVd%n+=1}>-vZ#)T3V$P>u1!Au=KFX+3z*tFk3vJ$?k24?U7UpC_o{UrM$1 z22yOvr?caHr>E55L}~95XAEgJ$}aIWP5S;}vf25L6O$AjMroLt=Gc9qLhNt%4vn-{ z|N0Nj9Fty1)M&;dpz|5`5FyDwx_33&qSRu60O=0BJ^Qv&=IG1Yvb1@(>P;gjSW{<3 zTPLzp4~WR*#1R=~L$gJ{B0s0G#>J;sm<0~yBX>z8GZ&5Pl$na(xMGlyA{T77=eHs( zXt^X+@K3QHO66&rY4~$e?&R)DKxtcii?gs}?+$T>ijG+ivoy@JBg&jgDlw2VQ5c77 zfH6BDJXxVQ#YlFP;vWJ1d`Aele{*jI4~#hDo5FBK#*Dq}T`*elX}N!2*>Q_savG9p zJIgFGjj!<#h=|(l2+V(e)HXw`ZA`$-4hUV3lpjOs*Zv^{w7e~4h}GXc5l8EP^C!I# zuYBY8(t11o zHI*Zzwm?|P**0@~ci>yJu#$@O^*$(GjR$q-Mjp*<6ZCTsHtlma{w&T)U4#ysd*ZAzl)ySGpGPN<=fX$ z{cm9_Q+YIErW3FFpK_1sdMgda9g2TQ$n0&72=#qY@8RpCmHNT8QFy(WCAqcTTXNKe zQ#1rf71_+jen&aguC+?v5w~({!!=jxsQIxcpgr#Pg-IsSM9}#*(ZT)O2XXv5&F|bj zYMsX%cKSGHXfh9~IA+7%csZ5X(^Z@I*hRR0ulal3#b=&;1Up1j z(f7NeWGbZB4HggA+uCe|EZ=gQdI!RfeFHlQLTEM+^S%R4A=@f>vRl5vTwLp%| zIs5WFP63>gMX#!S$yx9=+ef5Ta)!yS^;%X|Z#@s4g*I&v+!o&=PQub6)Rb#R#CWTW zL(b`i?miBBW^UGC5f-ea&ZNvMwKi@Z5zIx|v?`@9$5D-5wx54UrFiv+h&8i-dVl?s z$aibC>sMn?`px(iS@ccJs(F3$&bNQ)ZC@ztpTfpvnCqER0Mhe9rV;6H7x@M@F^Y&y zBjgwSIpRwgE=$rU$e*^42Yc27jZf_W8)@>}dTk^QC+o8MZ3! ziG>C6wT~~wE|GX<1HQkm2aTS_XZwxFe!auOKIZ9k9`%1{3C)6#WlXWfrX}H4SXm*1 zvIW-nF(EFQqYE2N+sAvI_+xW)B-ZKI$#S1Xy{Kl5B=+8@LOoEsd3N@dAAYc55k2V) zG1bwP%kd}UZ!x?Ss((6>aYQB#j(j)1i5rAwK!C3cw*kCmjxV<A*K)F#nnJCB;luL+pcRcv@&0-Y}BbP6YTKw&@v$4rZ)(NxtP!0J z$t*(%j<_?D?tUWDs|eB0b+h*kIDRIPbcyFP@&r+Z$>UIKfL^CP=Yiua4$`(OblE-^ z79BRs3GHlj9q)-)SGR~V)D6EH3n%r{k!M#*Dgf2euMqhZbqLU_r}b|tq;?tDS@D05 z@b*Q-+?!R%q@9C0%xz<$g~Wb&rjevuge)ZzoWV&fk~p^|5W~R6+g^p{c`n~d8SK@C z^W!O;*`tfYnKa7SRG9c9!F=j4B8_`1l21QkFjy3BDnAA_K7rZU1LeeM96GhK0*xEK z3p{rqAW?-{aba~iH);zo$WteLiXneD6>r=Ns2%4FHv_%YwU7c_etp9Xi*(Pol&JaS zl75~X-*P`UUs11LMW5lp>TWHk0z_5@xT2s<0#h?un+5mG4MuTYZS0AT-{LGpnzhTY z2AKFGESfmWyD&CxMM*%g$7cZ&n4|E=ZPNTC6WKgrm2(vDhfr zO72%WK^|@CcR!fv><=cMMJqYV46|P=eDz#0VJXD^xV_`zGsjSKgmf55Iec2#xA=_7 zsn}Go6>46wv*^C6PRGd^tgwGaaB~x%gFT{j&87$ur0{<5QpPJv^fpb54fbRGM>#kZ z2zsk@XfL08uL28=%#F$OBD^yV>z)*d%RM2clHNEHeeLPP(^1iD2ML<0g*ZNrg}mWL z=RC}HH^?Cew`)EP@dl^xL5HT=Rfz}Vr@`w?V(v~LyUtH!WhC}I`A&bqZq11_vrSsX z97tkd-Bv5sjPd^L7j9W~=RX2?8P9mL7KfO!4&rlHs+Sr=h6%LYL~q%bnnHuoypn}# zOk~y4?V75s2p2QVt{VoD!j=14Pwz=vRo`6c<`O&Tn-IB0y&`Q}KWu8%o<9(6XEgPE*5U47Qraca3Wq6uEGOw1c&pPb4^(ZoNu^gYi;2c#{ohsT!TaCPzsPQ%@MqI|V)k^K1=_<% z_X)wBB5OP{Sa-IL-wQ8WoRXQ^s9d3=m_9z69!hL{LlS=zWPY6cxIBy{Z?|Ct%|Pr? zPfDD!?vszMn#zM}iz`VI6+|?GGbmYMpf(Lr_&Q*Vu%O!ebDcCdHAfcUQSw%d#$g_X zgK)`dA0)>AYI7md4;65VS0z|cL{*j?*I=g4g6ut}aH$raqvaCl816n))|g0HF0=)> zDiwa;opoXy4v0&NJ z#Y5O-TC6P=SG6=w7I1s&LJ0{y=E^u}4@N1So?%2m2?&T;PdEgn-b34tPoJs8#cKC@ z(bqnkB+PgM}4{L_8KJOqS8s{7#{ z`zc@6`5?TrES>OO^}t{Z2R3Ap(t&OXn=JWo%O)T(BW?D{cdR^&?yD2Eqw;}ISXH!S zv?^=CCZw(t7NEEc>3f!z10g=8T^kHn$vx$7$)d&$C&IFl#{piz!dhR!e&28o1J1c^ z{o;R(;%%#TKZWYi?Fkf@+9aBUvHFXj5rq#%69mz_RiBiiD|v)Pvs<-OH+&LEOp#eV zKkJXamY*nGUZWP8q*X5q&iGpm&<2p()-;gr2vBwK5jrPdl^Fj#E`V{5dkLFk>`MnKXv!(=~*q^3o~_+#gapV zOTX+?g$N~9P33#Q_<_0NtNUraYY<|F8LpExp4e@f)?5D%t&i+A;wwhbtq=UGSS0rn87EMwW$q^R!q2^$VvQ=>$1cUboSmEvgp{o9(Y`TyWGINpoA< zIcgb%WNwN$m*J@IZ{Ac=hxQG+jXBCmMbf+Ko3^X2Y!DaF$<*+ zJ$^f3vKpT{CM*O=qZ0fW``R?Jws){DDSwgB^j^afr<7avwvFah+b$1uax=dPH)wCX zTt5-ob^O6O;y4ZRuHDbDW%E0ufu)U?g6)>V)(uEhpf{8U&&HOXc%Oe7+g7n9+DNdd z?d?B@cD5MbYu&0J?KKXHY6Av0`6)cF$$g(TpN*)5BV^KOuA3H0@+Qwyu~5$P zX*RLWdBcy$)#6Vo{i4baqqXLiWF1CoHU|d&!bMcC`wU*5@Gl5p&%9ms;v@I`x;GQm z2A%V*Ir7>o)9bYx^uvGuq!OzZTOhIy1hb=`$ZBx0DbZo*uBm(%9wU(cih-0IMMmsP z367&gG~8;Pvsfir!*cGfUOn0HpTN z?3LP48I5VC3*e@?mvlqiyj!wehoQ>n0p+}^650msQIv{6Z9};v%{M77kUH0=g`y+t zDNIihUBcZ&W-W=!S6ZZUQM#iTxsCmmDIdC4zfGIj!~uuqQDM{~OsfejalX=-@-e0L z{jK?op6KkK@?C#-LC<@_S)th3Ygevwbv2~HBQ)AW3gHB!2^IUQzLqykh zNw49dDxHSiejy^E%lB4dYNG{uKk-A)f>`ma=7C3~`@QaQ7t;whEy~01vyY7N*TV-S zD}%`IpFoC|oeT5dY=*s~G89a_69SNHWlozE$9!>RM z&5q>L+cGx%1lcDXR$0yzdA7W*U*6+-DplD$mfjmPURu9EfOmexiT4+d+6F51x5+Z`pG1e4#7ug{OCt zeZB_qWor?XV`|*KI`K&@u;+)jL`6PB13sF-mPBiDwZ4{ECaM4NsuMNqqOuJn1g!;I zPtw=_2(#?*yVM7l1~ZAKF!>4{%2FI%9#7=9rIaSmjMr($=(3jyEj=aM6xOyDe(dc# zI-6m>vB^^f7X6nCleW4Go@)m#BeLOQZ+fy{vscQte0`o8BvyqR>bBJ;q{o;fcA{g}7J{C$(X+1?~R;+GGK0m*Kbu z69P0hml2-@6t`4m1PW&XIhPTi1r)b)%mfi}0Wp^mp9K@Qm^}rN3js5i5uXJV5HdJ0 z3NK7$ZfA68G9WoNF_%tB1r`J_IXNNNtcu$C47U=dybxSz3-pjHP^(w@?LANz4vdJ=ymkDWbI%! zU?mt7&IRP=6$5B!YJ%X7s&J4C#1_cQr3ZHKashb&fZTk%yf{ou@*ZFi90G$XfZ$*; z08j|-2!GHAyTQS(Hee3`!bB8@37`yyf;|v=JAjQZKoblH8Th(^fdE#}Z=eJ7gmc+| zJP~Rz)ByqovmspMVQ#)25C=#2eGGmsuKP&$-g4XkHIS_{%-ho$0sukn0BYQt+yE_@ zH$n&ju)?4K8?YnD#U22&2N-~j0fzdDdinrmJ%4RO9ep-#L|}a{H#eBa|8SAlH!xJ@ z1SrUA87Kn4Mw|d;Lw$q$PXjO%!QX)spk;v2-{(OX-n(lm8ps-$=qLht?rQ)50Z+jm zo{;;r|HX|N!3^++8p74y1LpcW0f5yJ4tEpd;qmtN=63M%gmc3@9Jt+Fe)BVMgm?nH zVSgUZ0K~Tk*aiH%F{CX6GJot zejn52A9H{w82le;96_GHeKmA+Gytw32ow&6f}pktLpTWTKkxrAZ8EBr3piH2g(iif&aF?k0-03AqEf<6#xJc@B2Ab zgxbl&TwM{YJ#p^arvO3p2@dn{<@rAb+!+e^;m zh@%N|f4JT2h5k#2m{?n9DA>~zAi(=u1x9q{zt|&|!-^Pki zTbLaL>Hy#q6as)eJV3rUyod?m6BGpa0}+F72ln|rLI56aC=8Bp0l0a=0|E9h51jjD z77_yR$lZ(nLBaqY`F|r}Ab>~t58?;#sQy6$03NkJND#oI{s$pqYWzWnn3{i(2!KcH z-$;ZHQR5$kh-vT#AuNsmjeq!f5fJ3!<_P+?NE9Kl`O82Q0TK7Yza;{QaJGoO`$q!g zMdY*p3nEH#_!9xhi-4fN9S|$a<8XgdAx^44UWmAke?dg;kiQ_Jc&EQ0qI%~)@emnY z?$`d09>Lo654@)U{lkdw9(w(yM(}a_qecuDj~mDX4gtB?LG1q&_J3Or_JF|b{&Gdo z^!zI_q6klj!(X)uAeef(fIR;y4Ty*X{|h24{w9NDKWU>~q8&fGN2Ry^D( z<9%dhkYe#@6*`kB_kJ~nl(E_iw7KljHtGioFTXK>rAtDaLUNc(U>Yi77C~vD&@yAD zaB`R1rRLO{FF)*bW!6AJnnBbAin5;yjJOTMax`Z{syZ0hc7N0gicHD{#&cR;w9o+z zKkdqkwsl@6@QvINpQJOGSG9aCME5S81U8a+xRChF5*^$3M~*L*zKLU9&*9xhiFY)Ae%CEQz)|PUm94&w3l$kWD^XqK8djCS zAw@zyqJ7Q%t$#x&{GcKYF~mvH^nFeFwUp>f`)2d%!AkR6 zIT1%+c8)<@pSepr^v55KM{lp)#j^431@UI$pARaQNq@#;4%}JTug=WbHW~%95H>v> zzoUD4eDiP?MQNRMLb&_^q+n|w3r*!_Slx71kuPf2`6El0OxfG6^T(p1P6q=SQ%e08 zkBJ$=9!1~g6bf*lsqGX?tf#-4n)&HW(>4Ko8SgqNT%Jg-D7Nc zPb6q-f4$&mdH?CVzm<*gm*zOuSatx_gu)KLRq0rCFpiYVBySaXtu#jzhH4%Z9%W@} zDXijkYsH~zU@{Hsp;cyH>_I)HX^^g&_ zfPdt0)+spYWk&o7;0mbL0L-|B252+%c4}z;tp0eo)&E8>Q+?%bk&^6H^T;!GlO~gk zWFQKiz_*U`j|oN@-^}05aIss3q>x8H*tcesTu)_49FrV)Wgk(>&nVl5@(N&Ag>?PR z`40AG>EbpOrf3Wm|SI!_RTEYZUP&--eY3JsNjQ69aPxy?_m!ec*Eerz!^pCnZlGGFOj9hIuYabyEN_c^ zHgYzmHCf;N5>jSJSWecLOYGqW@!;fk4uXyc56~m z^n~SoM;WyjFz{wm;2!S7OUjFou^E$*5|{M}B2(q0r>dd9`kX@1l)NVm#ri}AQBJ3LH5Fc5KFtlC#((iMo15d*GS@tv zuYiDb_gi_Ef~-TzX7p#)hvP zf-79H2_7pM7*8M{O^}tW!m~aX57LLNN2Pk!bOhCBp9|&=Yiv<|accjO@ zfzl|fYON%iG(_jiq<=L`dh*1#lZ|%Qqbw%aD7}kiqKo>7@*+4rcjaIf_Bj4_1DVdE6^N}9gFWLMmhPX>? zrUim;#x7|v?aY&+$Zen|dol75RyQ(A+2gWQ>GY$$BH`9gihm1|Co!0~-u*MVidclw zL$H|=mKKOxCZ$0P-vPHg60~sR4&r}K@uGeC15wo=&HP0?L>&~ zje2PdmVsob9e=PTu&8A&%_<30nwzF6+8-@I^#WUtb7&9xusr5{ag9#K6NkW*npLIS zh=P27^S6tX`^BqVPeSKg#ox`Hg$kOM5bjLN4Nvkmq*4>@D9y&@PS9&uIt52z<`7RC zB$9ij1%4${&-!p1u)4rOnfIYlX_|9u-~URz^wm|L%73Qe43RrD<=tTQ%WL!WTpzA! zhL?C`qm`)PKH!NH#f%UQ8;74_&hE~tuW_fZBz)vh>z`BdItHT5vOQoc)Al8J8m-m8 zGA+>j{d~f{0BkxPmOu41#_x=Mu^jKwA?BHDRC1UdY2Ov(xhH?f;vKO&$*tBxV~O&- zMwUJGfPZ+KH`wCi%#@CY#`vi}W$?IGHw-(nPY{)4lNp)(!MAA1F1RUQscZOxOS^sF z3R7efdA$q;M+r+mhE@hR0+uzpNc+vEyK-)8m0fe8TvBcJiJ{0P?3^Oy>KL_3T^t^i z*iLh_=lkMo5nrY|`B@&V%B=elP5CB`@pQu6F+wx+Gb+T(^(7~zd3Bwno@icB zNAe=f&3RDhu}a$l=}Ay%g=Gr5wFvv7>}Oc17;g$Q z;|TUX8^3)CcyNGPF-!rqf0~fI#a5p8R;O8HP^`v;zDhaAOy$D=Yx~v^HCsve3r`ca z<8YJ(zLoJW6ccl?t{$6RSt(UqV?~LtpMPjj)Vc^{kiYxc!RMi-6FiqLIO~kzy-4EM z@diAZdazxdv0t)Smn(o2?3 zDIvkok&#BNt_&%ABr>kKHv3DR@>Z$+7V`Zr_u{%+)-h&NX$TA%vYP#oQ(mG#w10lH ztJV~bG5xx#6P&KX8OBIW-5Oqp#S$?TYY96jWNX@0u@W&0dHw^RxcAhU5bMbBhR|e} zAzFT1u$V^B6uB0>BO4d1lo+yo=w~T(AqH8#T4zG8Xp!qgpHWXldt{c?SwKL0(fh$x zJ7u(0J_ffyp z61hWmE`qRPs(Q*%riksOL~C7Lt>Rgeu9LU+u6^Un!%~72ypjo3q>?qoKNru@lM#A}wFYFMX>_%6li6h2Z zQY$T@rpzzqViHJQHt^+PTCV#n9QVGK{hrxXL zJMH`jWQPmv(TN85+3z;ODw=#oM0YVvihUt#hq5?5f}Nrpp5krt&ntaw$Ld85wW;HT zgr?dnTsxlYh7fOAgE+HBL0$a=mmt#)?h&$h{YZ7(57HS`soh_DnXuc@E^L#N=xkz7 z*PL<+#V&N)`l-pffq(mLgAT;sNi*5=hfL5Q*u_%;CN`KUz)Eo1e6o zypC#N!foY7eQ7zB?I9&cbf74M-DGg8E^jLjT$YI3NePB=JP+wxu2G)aR@-8J^p@2i zDwlw`LPag3kAEf^&!~6<#yXmb(n9mKG0L^T1AN7XOfJq-?M+9N$NWp+`+=d2p+C6^ z{_$BQ?awxWpQUpzI6BYR!TRrS&}zP5UBG>@ZtO!g0fJGY?E|3^ zFL(UTkcSyRIaVr$LIY4&kJ7enl!Zt<`ONjLxHFkYl7CkKFG(8@@D`tfFlYuOYxSO~ zW=PKqq0wH)2N(m%&%fgPd^9ued@ti;B;zuO7jVlVmRfS1r@asTG4xbruc9}TkXU3* zDRZU+U1;H@_?5*x6K4ZrWeWL8o_+$+t zs_(z}IDekL^;bJ~T+=M4AttLTZI&*(lS8t}krEHlJHZ={gdi3R^+$1!lkniS{+FMc zFX~u1;_rmxEY;$60{8LEqEV}|)Z_i%b*qU#_r6TXl<>#cavIB05F)qUDKi()A_Dg8 zD!#O?9h+Xu^z73;ccFOhbOOkYSptT5E`DJXXMd042pjDjX)}TKj~232m|!=71BeNP zxW(P-Pp88>aQN}<3QgC@>7!}R2k1IG9Swg-zg=`ids;)|UJ_%W6tC!d^?-f`ua-yk zi}>k?fOe6Ur@;`Wnn_=v8SgV3*of9+uaG63#6x={>ehfCJn41P6Ai1bpILZJvN&6` zwtql6Guh^AE0zw5dNse&AZhP_O2?s6+LQJ~@+K`TnoqHRD%(pGEaM&zlH{SY`c$nn z$nUF>Ntq(84F4V)l%k?CB6wU=mQw?ad!U^f*S?!COz zts;s*-4?m?qz>PiFaeJaC8HQ%z!NGnSwc8#uyQr12bqPUc|)0{6*gIp@mlcIW`92s zdp=~7+Pm+q>{`SUE}ydAqNhq}?Z;3x4uzi0jRlIQ75;KAm6SpuNGvDPrCGOfOlPTO zVTsj#95Z2z^I7&RobVdUBrI0~jo|%LE83MFN9{&ClA_vG0%bHOSC)O-cs$K^#v9xC zuFNjvG7fSBCyb!wi*`5aU@l8TNq>O!y0;Qofv-!^ZtyK7)i?DS5@ouuDSEGkc@@dt z{bdpnH}m>6PQkhF$h4;&4drK8Cdu9_@pSJ3Rzs>VKB2Og*nNU^gvJ-&xQ)6UQ)X2T ziPjH!#>Fjygo0c3S&B(f*1PJynJ%|lbO=4;Ssx=)1wgiho49LuP?9 z+3!Y8vVPJU6hJPr@Jl;_t-(lPlc@Lu*XWOi+!0cLeg;atYfY!Q)Ueh?@G4%E(75xU zbG|L1*auYZt3zstZ6+-=)%)ZJtKD>C=~H9Q)z;Jko*pbGx0ZmpP8hvU&T9BYSRG~o zffW#g+(?3cg@3LWZ5Lb5e14h~eoIW5b`d|7z!)rKUTPAAvD9vRQ76!s-Wz`5G-v!L*;3n{xFQ71)IXo*qrA*-oPenbXM4D8D3Uw>X43MZTq+|d5CzNXko zUkmfYu2q`eN)iG^7nua*EL~$1vPO@{6O57x_uw8{b6OW3w+Uta5r=8ks(w8F>JCrWs> zA1bRgKllJT-+%h{5aDft6w*?~yVtp#g?{G=8 z@=L4z8u7N@--;VhONL+(8K1Jt<^uR}E<(lWX+h6WM|J%bssG#oW+LHw_i-}+ zh@*!kBSxo`$Dcx}qK9r&ag<47Px6XB#OVBaw14j&5)E@+YBA)@|8gfvgfZ#I%*3*! zn@@z?rXxA_fwZ{#xFeN=+m4LlgN)=(qSt!9C+p9O@*v7&cQ!TovBrSS1?8Sw?~Hx& z6$_0Dd8kzXoy~SNk~V(McbkF5+GVU9B$gu!14vC_S|UwyO8*737YUB=6oU%$o@TC% z1b=Gb_D)Jk#Q{?$M&JXQkIhIS*JT;wFVeldEKC=V@ny3Xl+%##DrO@VL&B8#_lD6V z3}X3vw>6clHWdQKhDMo)8G?*1SWKp@B5AfW!k~P^OAU(QIu|sOhbbDa_OH9b!U={Q zN3cK4aJ5g!UYYK&JJCKGFf#l4W4HJP3xD!xU2a`~lx0TQ$t-JBPe{PTEU&<)_{8a_ zQXX}XC&$+r=dvGW+G6`NFcuZs4`rAJL|CcUE*!KWV9Eg@jqHDYtEkE`N_OV+GO3Mz)rESBls7{!Xe{`L!g z5J6?=Tp49r$81!oS#X!d|1oVJIe!uVqj)Dp*pI7t`k#gGEtu&}r(9dNy)5mu^1%6T z^$1PjE}1I1jt>cVQ0)bu-I3LV+UvW$x!E=^4m`qZoUx|g!ih^(L%YbBkF@`wp&1OP zvID|Q34D*RY)Mi~w;tFjeC|@+RAOcV4vv2`SnV`t&|V8TJ!Eijt;pEhkAE~|oJ6=m#)L%?T#w%Hz>sljphm&st(b(<#YRSCXJ&a88GBvY>!fimdJwIlL}t*DfiZp% zmt^vVGBZ+(3TE{)=>CONk^J4`YAckVBN2L3*}M_ik(8+A*1R)pyB+Bq*weo3Z?P@K*zFYj9#alDfPW#)^4s+=!0VX>n zov*Ub_uSj*mP?r6v8_oFuP^h`MKlyGGv3#ZNEbkhea=fy4u3pSQ=&hWX6j2>bKgj; zA$R}!e2j|pYm~?0Q-G-u(`psFP<=vV@LjOhNAxND&m_Gy-9x=|PC|q^>&Z8?BI)VR zkA&Y`lkA84{j9XR~)j+nn6U-jKuU*JK8!&MR< z&(>-I=5Q9mwK`XAzi(C#bdWNn0pGMO5HYfb+7dJIgy)zVrV1R8!{}@IzYF zU7~DKnP`#u;9SV-t?9B`n?`y*NVjpWF7tSusDC^sB;{u9LoBj16&5H#fpUDs!D*|$ zbY8+lIF`UV(6YpYp%H$>b>sK>^n?{3Xk4VPdvm&Ec*Z`=2aA3|d5T0Qb1T)eW8xvu zz1-?2r>XqaiV*7b2GvPk%~z-_oALeNg|%ayX#cfdBU(e2b8X3_u$;IBb=efUw-e*o zU4KrOj>9S;3TYc=zQ59#&C06~Grvl4)lZz2f%a|Gx@dX1^I9L~g>k}aWsd}d@IP)} zn2{}a7f7V#7&be7StCk(ADV6sh6-n?aI z!28Pv*BPVU05i@|RjJQ-$I{v%YUQ{ zdg#&X^O_}b^*`Od-Wh`kR~Y*?x5Ca0Nz#Jo1J9c>)D5l}NwiDPEkxT%n&Q9WW_c`x zF(#S?j#bQUoXu2FENg_<&QqmuUFD0k2o1z;-#F(-B-ga@ul^`KP!$Kw`l96(nT@&0Y65&J&qd>FvA)nUO;N%7e2nC zjP!p&Z2?$e>~=AL+Z|ko{(qE=L*6Hk)yf-^80)MUXFJxqDCYq>)h`~oHqmfm%`%HY zx6Lk8`e;PV=3?A1`-o7)Pe`M!cQ#6n#J*_PkdP2;A*aihPrJ-zr?1Iz^lyY{;)|}GDj5oN-BhY>}&OcLk2j#!EgqG|O$-j6vWH0~ zX<`u{N)@AN`03IOK7V=Cn(SJC-crY*;I*1dD-c0DBNHbi}kEz)x4|p*L9__TB(f4+daJ{yqHT}$YW+4s<>C89%_QDQc`N;l~MT!rr(nfF2 z?zvLjWrX?s)HVuU)-PVv5A(W@g#ZK0eigE&^MW)SNuHp$Bh$>nx8>oDG)^y~zC=DQ zEgXsKj;&x{eSc}%jm9NrY3LZ-Y11PP+UAdWo?TPIVR>Bf{^LULOx)_s;4G_LJ}1eT z1&TiX#pdHQT!-qPEnjk}Lhn54o40RP-yK%nKD1>sdZ1ZEUm8VBOUlo<#wWiCY7-DY zKfBz*nv0Hs=6=r$fMa(_(&8?zc5zd^wxCN(v_7kYS%3D+F>%$G&@*djp_fshT0i|M zmkkvQ!DUsA67M$;PD5@xHSWTwQPZvYKH1VtU%Ti!&CsYjT$*uC{jJ5O?b_kyJU$_} zd8EPL8F>6q6OG>l+Z;}8aNb_hLRFme0{_QH)Y8U`6;>V8@35$IvJ+^G)FXtii&RODI=Iy<0cW`zu)!7 zC5r2pUHHS671u`={gHCN8eYqQEa!(XU}*SEK5^X?rkIzZjyiYn9CTuqmH4opp{4K0b2T3Sb^*EPB{+7k+4XFvlA9PJL>BpyAr&u)pjBpm*}! zD}Q?`loU5bcVirb_{XJg3AV?UVr(Zu!@p#uvAiQsL<49AmW4kDiu(AqE+UAlAsN$22T4G z0u`lMC52?ptciL|KC_l#B=Q*mW1Q$s6MvV9QX)6BRdTMpy5^aOdT#Evz?1RF`SoiY zP0A~mndKZ5N}upHlxnmX*cJ8P8KUnQ_4!iVNI4})jd@8e9=)a_tW7bc82c7X&~LRO zRq_aCSiGT;)L;GTEvY_sT5SVhI^576Nn1_|CX5R$J}+KkSJ>vfWovu>QGtUwk$>rN z+?;JRHO`Nh=}v|GS0k*miDX_#FBklCcsVZfC+Wb}e9y=>#^4l(KPUCW`+dE+mP8M% zbIs27qCNnJswU7~-g0OJy?3DOIB;srd9Q=V97az)3zm}ZnoUXUex02u@S#nw95DMM zq~TV5M&_;=!Ea#}N8T@d*UsKkiUp=x6ZnJeZ>doJi(^`hSRW^OFeZ z;CU9mjRJCbNt-rUYsV7}yQ1Q-sp26iXDeM;;dQWQC&%!jD6Qy|&Lr0LVt>|3kHsJ} zOkFF_P@IlqnYBhu@@Xa&T}hu0a(l!#v$Jx6P6=pO-I3DP=ywMDkIaTGC;KXQ*&C*) zjh=|-$MoG~q?Y7d!-9eu_%_Y|!i6&Up?v>K_i2dPgEx6^LWYQ{m5X(D2qY z1p58KW1T=9EQSDqNlGaJMsPTS;HpnRc%YFA5Hk?v z?Sa4nFiB}CDOy26O&o|IpntI#Ed&8n0$}n4SHK+fB7mMwAPyj!DAEc7x*!I`5%tc1 zlRp3l2?z^+FAxTZAbtW;yLg0yN5Q9QvKvANLCf3Uzheo*)j$+6_AV-moyfq{N z0|e3?>x*|s0|<;WU?2&X1dOr1L?Idw!D0X>&=uj~0$^PL3(yL%G=JANGY53dOe{^! zMJ0)W&Aq+6u(o`)3FU-ldW8>$msbZ=@SDgAV4Y0Y_AwUOmvLG;plkSRzOy z0`60NAT?ijAsG+KVNiEC3Y;L@vfw>lv8v&ocGO~X`Nmhic+WuYq40l;Zg~Xu`U7b`!_5s~{aA4%iK2KNY|MdT%fl`Dpo4mbB=OxU zU)Yc+7Lt1$;i&3M*b%okP#J=lS|!X$zwr=azq0?nsWaf)At4z7gD9!)^v?g;83j?U z^?w1=zs4_6-3PP=up;h7+~1``ht6|8ew!(?fu8;=ALu^lCMh*iARq$qAC;0C9g+T; z(;%=(gK{VcUq=0RVn}*c_*qFRM>~fPzIW>vc_EK3Q8C^<@-y8*1mCYLJw5%rfAH&# z?4x0Eh^ThT93D0Xm=q6%%!SvNt@ekDV9wF;H&G?_!zY&0>KR{r=a5EDjb4*d)>>7%{r;fDDxz0le65hLLP5iHyJ9H?x-zemR6;1(|6*B#ERO zS4-mPq2|Z@j6iOR5~^0+u`)}O7!l^gr;fx3z^h6;W z71w1V-0+e!=fQK%*O9yaHS>&w!+7MuXl0F7SDX;+#g>WVx`4-vmapL$UpPSm<1jNy z(ltwYeZ%^p;rg6!AJ3P;l~q}> z+{$i^NLeW?|D@k936}+pP5V@_08V<}D7$JwM%CB%Nm}=^%2XrNd=_M%Z(>a&kxO-6 z6MeWxuBfJfKkd(2rS6u^;XJN}IGo?2P zT>1hTZQ<*|mrP6*mtncNs&D-smloaJ=O1xf`1H&-X(;KLt5SZbn>9{Y(P@$xZZq>PVl9Km zk&SK8F^^9oLF4K8DvMTH*Z;)JByS;6&Q~+F=B@nY(vC~tyuhi+<1F5DS-4213oKck zr>a`_b7Al}1>8|X9?6@Y846S>zVPOUugoV(a*MJ=R=D_Qg=6apZToDp`m@Q|XJB06 z(lS#ir>>bEiZRR5L=qUko zp?J=BW_)T{-4t(QclLyP?6b)*>mE5d zEX=MMY4(|TWA%MPsetev2yZ><=thWROJyLwZt&OefPatn*s%`R@Sc-laC2S2uQ)BV z|ENPXQy2UC68(hXE?v%Hc~d%9&>XF*aXeEiq21NLxFU0mC3Ya}?vXlL&f=WMObrR} z1D3M39QE795_ zKAEc%Saq}skUA6&_~(hk=$%9nub!38ZU0`NQq03=!L(*ZsYim=R& z>EpPi7U@-ABmKM1rcGn%g{p1^WFyHQ^7f9H8ygBzs!=DvD(AE7t5sTuDc2>=<6dZj)9b8R6$9!7nV_osOSK_7F zy7&>%;<-H4+7u72kNjzxT6)4!pH6_{&935hlRMbr+uwan-h#QBV%&-z|LL6WnhQC7 z@$;YPL5%qHzYGv{p&H@g3-&*7gbZfuum!m0w>iIn(}yDT7tePg#evbaDBO}CR`H72 zk0+C((n%xGa@O!`&x$IqBc7%{jFJiL@kIFeW`2k*Xl!qp#{b4A49r4yA-;B(Gu|+R zqnqK!6l5WSE*yIx1YBM_`*YFPVLgePFtlR}&r!3|ylu_wII4I}@tEN?O9CSwYr5sj zc?zU~Dw-v=e@%409-2lO!TUHq>84>hjPoScJ=(E6-R| zhRZ;&TaN=ZRJ2vowR4@YuLpzM4T^{tD|@y|swjgMH?Shz$KBq$Xv{&G{uS-BErv6@ zWD7Ksja2FrHCCx}4f!(dOfE6gK=gF(w_IYN!*A*H)vk-P9W;g3`WLp{gGu$~9Gm?3 z%k*SGWGS6>`6DMlPV2i&+D(rI5MJxOWzMZJTADRh3^9YRne~kb)LHUs>PmlFqU+r1KzSiLMv2a8itOs*4V`(et8a6>&ZC^ST}d>c1HxH zR?RrwrY<&PVO_j!{5noA-*&US?>qrVB`y3UtLJ%Jc4M74y^_@NBzUpe{2h08Iq~Rn z*H?&6xs2{XHZ*{Q}SB6(FPTUUf>9#468A-6%~_QS$^&nUM5x-B?c$N zNg}&wJiJxi-GlP)9Y>p4bi?1m`F8lK>NAiJKMXxQq2~K6sF|kfVOOt0-; zy{U`;W)QE>O(XIaX_!@`8i$+{BQ3^i*NlaQ{}pCex`eabc5`~Pz;TGXl+bCC$5gjF zwYDiRT6OO^k|eN~cx|XH@L)L!@sPoG;=ptn_GM8&7?HE|G16-7VI>cPeLOS9+pYZN zVRaz6yk?1sFe)xLpK8;WZWjaGO+6#!KmUPPVR@{_R@}`hhLet1Qln#Cpw{PG>`M{B z{U*Yeg=u9{f4Sdkh)~Q~pPM(p?m8 zw(kXsv(Z*T*SwkOw5h^#kQMtVk#v>k!H~kv*kKy0^zmvUlP=<%Unlzqvfk8e@?APx zc+5vm^VP9H?9XxRoem-~sp_K>l(DG{8}{82>dh6IZ~APr@1bWs4WR#IF@(1DhwdQf ztxa3>(rk>+1+mdJwj)?KfNef$g2( zUS=UX>;lra{J`*s{Sn8sZ5Ko5W7@Uw!{}}>EU|3;x>~8H?^a;0C&fNA^oLf)D>)kM zsGpLeM;c9?LUHMUFKdvJin^2yQ`Ln(+=8#>?dZzoW81|pGt9?HADm+wjurEEzviRy zkoYvmi+hpuCQBXA{Mj6UhiP7+Z;VinTwu5RyhPSJxFHX7tWJcUX}9GQ+JY;1 zh)!l+zo|Ui_WD^j`r3@9r;sW$K|44mQ@99a`6+1Q+*T?(B zehS{Yzj-Yw2%JslTC6LY$&p$1#stagiq~4n0~Fnu)vFH({K5h6ubfdd&Fx7u35Aa} zntJB%y8c9FU7>6zz2v=R>E!y+bK#?3*idG(oK?eHb^K5Ye&ieqtUvM*tUu`WNJd(o zgSD1)BQl_SwYnumQ7Q8uL3~)G)gF8pd3V7Lk$hf6vNLj&3-!S&*n(ZxkA7fu?fFEe z14;pU{O*9{ox!GKYsofgwk^AjTXv6+xo2SbZ#T;7Rhuf?g&co$IdPd>x(=8I-{1Gp zOU<=AiBKrSE1O=6pmmi-PX-zRrttGGeR~BdI?5%SjV*hp)4$Ua7 z2r356tM-m9XfD=0`ZY+Nt#E(A?FCa>1*l=0)Ngnx-QPT2E72apR+2)gMrWfr5iE^pJmN{Y6XIOaFTq$UFJID*m^nSwNSL>)5utw zVOZcK0M^*`#cv`Efj5i&xX9enP+HOoP#*r0F36tQ{OZwG;vPI!wexxwxu|^DTEk!7}WTeCcx zK*uR?E*qvq|D!!`g3pL;mUAUjq{{>`0R|~VBVF;s$Aafxq~YJP$FfLNX$nr8tQU)_<9`gzRYQuklczdhd}Y4Ebu}LjKHLYM`-KYXqGy5Y z)r5|i)pQ-I0JTQW)&anngAOy`vp$Itu6%6rA7ahfs%&#SWV8}_bqB$QAZg__$E`Vw zjy#l;X5@XUSxo0*nuW@)^C%l3Ri}3Q^bq;;ICg4s0^}VK+mrp;?NoO$9Kk_jtn3=S zSgWfU+n~U%(&$MReiySHN`#*L0TJk0h@;JRX4Q7=31w>oy<8HT8Tx@m2XL> z=&+b5Kkp<|cLA?(%4pf)fvg@2+K7l5K=xn+PB*OY2F7vWu@IrtbQ;Rzgdrx)&?Vqn zE8BcXeTmV=dqH3ae~GCsOl3EBTU9&bc&+wfvu2U7e;}h%6d+OYcSuXwM+(?}5{9zm zUL92b`s3{tu-Pn3y;%}VA4+qQuUVcu9~&YH-+W+m=h7^jOc(ODcvuRo22gNc(C=d%K$lsA$o0U-NR}PoQXOy7<5a6Tlycvk{ z+Y!i5)0(;?xv|{8LcndVngYg6y2eNA?4?97HmBSOJDQMaXf?g|ISQxlZPU1_5*_Iqqu-{SzJ06NwokOic&SI;T(8()mtGV&(7J^z zIkW{*Jv2rK8Or>d4>~`C9>*O_9K{0-*vfyMLvd>lP3NR0)Y%_F6ahyaZ_y{MGK~)j z04%{0<8BH3O@Z1G6M$*NKRC7#2nJk8N!p6ENbzfbZ7vc}4@e_qd58Yea!#4@T)E!!Rrcj9Q+%9~zRwwI0>~sLB1COb|i| zztSDdkm*!;qv|C|-hTREKXIDVL-Ivk18a@)Fw zPku4jFGBBh@R**86=&#w<>@@>oKzrUYb7n0w_F#=^3UA(n|6qI;L0B|jBnESxW z!uSvCo0|6cQvit*_Af%%QwN-jTLq<0C8WT_1`ThtM*4LssC@cwcT zWWDkHpsPvVwQqe0DqxVin?JYOo&WX0Jnj^LTGyuKA2W1%F~~KD{^oQduA0c>Xf#$e z^Opd9+3!dm70m-$$3fuSols41%jtA`I+F=@?=i%w@xV8I%lzSVt8}9_c(Y7Db?XBeNY?6t5(%3 zHQa1-ddL@>fPFQMuojYZR~OSU5r@wXRz#po)BGR^IQ-Ki&hYGpiCxuRTPHj!@1@Px zJ*nZzt*A3LQ?+%oGl-CQhx{>BY0u>Y|N7iTn3dP~8&J6YiQwOk_WxVXY3ESKYQ|Xo9>2Jun=!qEX z`^6Gup6V(byg0XK(;;69ISm!;IaqB!6?xCU2C@oi`G%P8p2Oa~p}_w4bVlRyFv`h5VJ(L9{*FDcHc|XL0{$EeM0+BYUPqW->}HaK`_Kos<;8cIrPM17Keb0cqy6O=CEAc zgA^X?PPI$amE!D};wilD#!S_hZ1h*DiG54;63cD_E2%ljzo8;WU{WK?O{G-C=V5)1E(4A~diOlipO)_V$9ijygEUa0vWlVb^7{c1_;JXi zoII0r6jDcPep9j)Upnh}e5PIkt~CuJR7;98GPV?a9vO3{VkYJV+9d85+6&6VDXDu+ zwC*+jjw{HgB@nv4d5{K{6@)f+tNBqnEHjd!4(sTfD;V?X(9^(+Ci=* zVZW6kFf@p#A`byG4m_=I{0SH46Ez{2hefpXgU-wBHPB%PDs}0O8`*t3IOwp3gD4h+ zm)o)xRyn-X z12!C|%GJXC>#DbdJ2zjNOXU>O{Gnr03&`Hxw=C@4;;v&`h5>mBvm)a$=J?fUyjn&#iTU zKY-2*bDZ`<@zgL6(Nz8Zd~o(c$~wUKa(XLUE_>3hjq`#e0K0Uh`NXvCH^BOq&J*0Y zh6miZf+*hkq&B2kVPmP1YRK60s?T9e?D>gw#8Th=c{LZ&Owo~5qdC0z0Y)Z&k%K~!cp72uVN$D zIqy;62!!vXHT^!N@^z*-j7>ePB```3f8UR@*hmZ$pEs3+@6Q}-JL5Ti)h@_9VX(*h z(y_;5%TL;D%BNfa#u!33;X0L9i}-zZNgw@P&01QtAzF-V1irZB4xiSF#H($5d)AHs zl-3t+_%-d$yEjjuT}wbiZN$HyHwjo*b-QB-K>VWKF&n$(-YnOLC;=<%dz)$+fwl56 zq?@ukxoz=h@?wXvmdgk7NR^?*GI@?jl@P+%OD=%UlqV1AegfdkC+v&kY|{Z2p1Rcp z0upWgPBNr;$S)2Yhd;^okYjT!|5T0OcXQo1V|3{;yGKSHr;k0U3%w7lfD-mpJqM{a z@QM7XgCt7XwRPIy;5{R_IZqt1AFbuED^f+SFxvHisb+$Ylx#l%U#6AyEB3=VRCBv- zi1ca6UVs_-pnGMj?kK5w@MJWl)5NA-n76YfCBKvtb#_++Yi9Ra91yW3!k5mtHH|S> z2W(bEsShwKL_U@Tqc=lA}mI8UBde|S4o6kS)L*+Nz=bNmp%B6*o`@6p%?KE_^s z3|Td^T+vep$Bsc_I`h>Xg1p4xa~~l~U%TQZ{YvEXgEKuy0l4z@1HQ85HUY(_`Bf^o zvo2?MFa3vmFMwxLKHo>fr}NXvwT-!7nA+kaiVQgClola1m6LQW-p!?K^hmE{Sw^xN;VnnF%EkK(yVPvJNt;V~Mm>=@80rJz{Af=bE*)(OH}9uQEy*gh`QuVFU3$LCWEY zG~=+fW2a*M*eC01fvz(^vBXAI7+nrcCsaQvp*6Au`B{`dJbn75#Px+)5jq+H8d^3j z<^bl}#Ae>ob**8kybwB|Rko+#7Jp?sl2(iDngVl2Z$fB%%&DXi0WC(;oE&TCtU1V% zPP+vuT!E@tK7bwXiD0Q+!plVvvd{%AZm8}D<-31UBEVXoUP1$CL?7Wqe3?OO++HAz z_-o#%fSIAXoUh@oq-cG5lf1vHS-Ppr_L$u?yrVd-zJaNnhl#UQG9RJR>}|GZF{0O6 zDP{SDb9Y^sMkA(e3l!tk_M{n|H5Hxy$$m%`GuHcq%->)K)mZNUd44bACS%%}td`zH zE=!$|`CjnBo=k%qqw|HooL5&M~Y-WMhfx(+$-lBSmWn$|{=cS?7 z27mgqJR6eyS=!%QwCr|KjE%<7?W(Lh`l)}|QursZUU~5@5OubrzvfnHR;FX5j99sH zDh!`XXwf5fOKEhkE?nHs;#b^Cz!4Od!xOB}kj@8k?4s%*BIvk9{7WpDYEVn*4O+P9 zMT-Kj7f!kcKAL=TQb@2c(WZ12Al(1YqQHH2!V{`Uz6=b`wh3Bd4ht9fP?U z+ln+W-HI8(_g!dQZGQ3b15b0kltsJM3f_2TW1T%y3VB^@_*bvl^X1FFC74Jnwf#u0 z1y3z^?>Ga*K#gK-A^Gfc2~VYoMQ$PMwZUFpv#wTmp(oj(wTTFGiZGK_+rO^7`X7mG z8;IkAYojzsE;uXf0+u6?sCrfGnYq4;w@fd<_o|XG=W<>1p}U@Y6mDL1&hjgJpRvG= zw6(U(_jEgkX8Lj9>!b;*zZ&z&S5vxlJni+U=@snp=v!J9kxx{`rAn$)Y-W9|t}QY> zr3DOlnED>}F9~c4#W723@^ed{(qSaxKzH>DS)`#q(FW?53!eQ1fh&j$R+Z{}u{T`6 z!FEDpQ44QqVd>>Tv`qRK)b7m$VId`Fw9MPUIY`EP-h8M>?&{FS*@{D5Z3ms*A`Azs zWpWoVcKW!g>!G##QMk}Xj3Ufhs@lSO9hGx#qnB7>_QA*9T+#f~Ljtm>q!V#S94j^o zEp7-WxWa)6WMp{?TdlM$2lZ} zpjL%=EEig(agx4*5sbIva3)@S>7OHm}BCP`c>lGQN_rJ zBz=WD!?`=2G!y5$IsV$ed#-g@kzqQ|xf^0r%VK9Z<_4j;-Ga|#vDEr@sM1NmG)hGQ zw(@6vvDy^*w-GxA1j@LEjlTmKA(ql)kfXnXlY@)oZOJ~dX=0a;oyiR(M4{JsZDMo9 zP)L+W0-m_*8o6UmByB^};ap;K1QSs|j3b$TeiKS%7on;4^9*R4TEe52 zSSZCUWChr++Tt2JbKxCXt+>&F?BVEU;RI%pEe&CoaW2$#8NqJFp5~*}3Jn|bhdMdU zeK&OS4w`DZZ!*$mJTB)c;A{sA75KQyou?R}80lpeWE@VmrDqZYbwd(p5)?)2{}c2^m1{Bn<8 z>Wq(Sw-D#vWoKqdvAVc7cH~c&yexUf?$k_zY&s8fIkY!}RrL^Hu-EwM>gnu!#C|=t zV~j&fM&_49JHMhpv!(s~%-E=NFQ!hnvOu|8{h^AF!t$Be$I}~6LUOTrPt$c<^VZfD zUo$3fYx~Rg8fp8AaALX`Qw;cM?fc~WjOG)SLi%+X;02rS)GwJ-DS3B$C3!Z9_K~UV zUDTxrRN5}{amfK9E?TjxUN6LYR-2|ghAO*M$SsW8B~#YJxs-iYedbOrw6-ZYxs)=P zMa;i?H7clE!ua{Be!XMmJs^j5d3~I{C2v;~*F6g+Kr1r5bK?3_ArOrb-u6TXN+kax z_JobeD#tvtdQ|F+YraLVxn4vQx;}>qo>wx2T)Vm;e0T!K*o8ieO%&_+f>6afe(3^!~g&C;SAfc`hE0 zueM97kDz^k*Cm*Us-7;2*lf|3H@(wK_1-b*v(F#$^~+e-%GVozyX^U?UgA`SZ4P zbhcTwe*ojo{>=Zknk_qb^S7F+7UXhzik8xKni?E?3oc(mqtI%snkFlMIY4|eGAAIz*QEUd{99MA;+Gpyo(j`~*B zwZ8+$#o^$DWfZe;baEzQ=lWk_FA*yf8^?d!|5j>gEAJ1X`aV~u48IhNY_PMcmZ~_c zn!ua~gLp|>5(c4&QQn!@c!0k?o)V0jvem&~wEgK*YG?m=T;|*41B+$q3zHyIoa!y6 z5VL<_8=wZwj|JS&1dqYgk3 zgCE~G7EzskuoV+bQ$>#p9;q62$IgHyBgz0ysaKz*-@xxx>OHI#6uF;JK|j6csNpeF z9Ku1gdqV5+x*Z3&6%uy1&5*fUz_L?4aKF^e? zE#uddsxHy=mrVk?ir$UF0{Poe(B`sPu07A@a9zqqMktlG4;W$m{) zON7oF^7wD>J4o4?i*@p)`2dQ3OkRbG@)R{cU;rJ@fL$gJO*zf;hZ#{P_`puDlueVL zY4cz6-4=qw&O(&0WckWG!P~4n(#NXg{jMJ+bl?h}yAI}xqa$$b9_*UG z`|aj-zBtis^yijf>}vny=+DD@&953awXO;7N7Q4vDE`maEB(U^55syP*|mPpL6<$@ zV&4_+`A*Gpv?|wx?Vq@A5Vk*Lj@4j=(>nb^dqd~v1(tYMDBWiyCxL;EzhLVmo6stS zsZ_h)SX_3;=a@=*;)Nw%k`sD5XLnsy%?+E1d|dynb&_YvsYS1?TrG*)QNLi*#M zOxjU4Cg)UCXUPgP+8M8~8Ot)Vxoc`0`jXz}GlnbD<{tN-4Zq%2_mX&V zzC__qEm|n~?ekPWV1YBQypBAMFQSFF?q3()H=uX~1&5o5{V$$cbXjLFOsbdMDHi&9 zeh#A$y10{Ry5nY^m2LC8F-7&O-ky4ivFFZLZ|+T3Cobz9x)`F6m+d1%`hTO2G-2@@ zJxfL~BF-!`{lSNaf|3LAKicvppmo-n#r+cSh$a3hZL;wZ!0C<@MoU7h>Bu32+ zI75872KJes|gZ2#5wCk#h(mI;glMuT^zlsD`f4oamZWs+EijBO5f zZZ7=A!Sbp*7ocGE9isHBC34B|ukD}j1E=Bw0uXBVrPr|Kt2J4ul6~b2aK#0WGBA{d zvhyqtrNe8mlxas3G8v^-HX0^`P&#xxdVyNn(b;-ggG62jM?8c-9(wRInobO1;E5kt z+8M(eqXW{tnQ7-jk5hLuiUuzbH%7)1qLehi5hI=Jf`InmnWQa?uITC)`0li93ZxdU6VvnWq25@bRV*16RmS;C;{k z2(~z)hfbT!*y)fgk;8k+leQbwluHw5ya)DSE(vqt1+k;ct(Uif6yKRD`t7s3|J|~t zY2#`#5(PflE-v1CL{Tl($`@oqh1dNB_=BELcntR|6FAg4%-m!D9kxu{jZKTX;gWUY zdba+^8}2Rg9>z`?459AqPOMfsbPjUXxH!Q zICfE=C4Ff-8SL2t;AeO8YlHv9?_u)-OM&()1%RyI1(5b7HPoguSkGO7X+JYlgd7=~ zO9jugz~rO?exLKRip;=oBht&TpSprpdf7(jj$);;X3n_ASrZ~|>t=o_B8gXDja%3G zodMhEPaDFR>p=1vL4~%2Ms%ZY_Fm3zcFM#wP@2)REEid~By+<+{~Y|H;(c|$9nJw& zuh~f8AsXtlXm*v&yvz*??j6m!evb$6d&S`7-!?pbh??N1{KeQ8LH=fyvjamk4Qj{ z*mqkvokc6Pc7J>2^)gu}ZQ%8SGKYIILhoz*bX#NH^N+8L(@8aizAt809WcEnzdL#H z`Q0rOn9zCtnq77v4|V-5V~4!)e%7NTJ9!mEj$*qS90as8Rg4c-7)|+u4gb{m$c|QJ zaM?eh-g^!mg}@wtV$g-iVAMcV0x2M!6iyY_L*j7H634Pux`BVXYTLq}fL+I2&|by4 z85Am*Rn*^z>bY)UGWTcia{=W4qNVUl*!b~z7-H}?dx8&6AQ{LgEp~()Tt62&B;0saoX zoWm%URgFA;*xtNsuox=Pa7*gM5bilB?nLz{t~qtEp14t`)h=4*rV0-=T(@bDX9SwZ z3<)pv=E5*VsSCLScD-c41pPE}60yd*R<*O=!PFMUEd-{iCbV(I&|9mxjdVRZ@%j$R(Pp<>JId%+{o0els&xpg!-YVJwxvM1T7EI}oaRUJMdD z5)3d8wIQJ%6%&!LCjb204Fv4@0kLu;w>k<8!)sfMzI}!x%=Lr_%^tbKHk1foWj8qe z4d#mADYr)q-tE=>f*XX$;n)3itZ3l?mwA``rH>F>PRs2DcRcOw6?Z(H?G5*dGCTXq zif3850fK*Gl$w-4+wBXXOZuKzjn%v#;fW4PZ%35-GcS--`!gU{OJ!XsW43KRtniKP z21Z_cG^TL4r0Vqb-v^SKM8e8FDk=e%!O_b1m4(dBF3pfNZNE2r#qS#e6lR1gFRj~Dk^tH_zc9wM-bDUHgRklm0`V~>&--#yJ$RU)V#-+j0fN;5; zHbuGLUU2pi(t{J0+j9HwLk{FiGFX;FQ)DZ-?iVrvgGp}3oQ4+WZvApKOpRrrGkHc; z`m*@qARY5fKOw8|&#Dv`%m#TT@(?RZ+T%;)UGw;7!d(L`Gxz)WIpJH=Rz0Yys8RmJKfJs4m5*k6h$uy8y1Yi6-X!kcg9&ljy(G6Y>RVYZ(Kyp4)40 zr|XG^l&l^wfYtYq$s^xLu*207dEOQ z)+b*p?~Yxq+nsJlTfiai<4@0HXH>bKQGgI-Ea5UkZr;T-w*4?S0Z-;q8L z_&#L#e(1{9_Vg8erp4$uKS(U~`BI ziOH!8Q7e;(JjR=5V(u6gucR5%;&P5s@yW@^$qEH*KyZ})1Ks8(shOvtssBX$zqkbb zH_xAgSBP1NNe+r1Lj-)AVvyqhUjTU!fAaFY*(l4vP q0LxzyE8hLz^c5qa@5v`9##EAY@`M;P0UQ%M2NNtgxv0Dt?EeAh=Vto= diff --git a/doc/src/DecisionTrees/DecisionTrees.do.txt b/doc/src/DecisionTrees/DecisionTrees.do.txt index fd1221da7..583e64200 100644 --- a/doc/src/DecisionTrees/DecisionTrees.do.txt +++ b/doc/src/DecisionTrees/DecisionTrees.do.txt @@ -1704,41 +1704,47 @@ at the internal nodes, and the predictions at the terminal nodes. To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function. -For simplicity we assume also that our functions $b(x;\gamma)=\gamma_0+\gamma_1 x$, with $\gamma_0$ and $\gamma_1$ as the parameters to be determined. +For simplicity we assume also that our functions $b(x;\gamma)=1+\gamma x$. This means that for every iteration, we need to optmize !bt \[ -(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\gambda}\hspace{0.2cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(\gamma_0+\gamma_1 x_i))^2. -\[ +(\beta_m,\gamma_m) \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2. +\] !et -We start our iteration by simply setting $\f_0(x)=0$. +We start our iteration by simply setting $f_0(x)=0$. Taking the derivatives with respect to $\beta$ and $\gamma$ we obtain !bt \[ -\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(\gamma_0+\gamma_1 x_i)(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0, \] !et and !bt \[ -\frac{\partial {\cal C}}{\partial \gamma_0} =-2\sum_{i}\beta(y_i-\beta(\gamma_0+\gamma_1 x_i))=0, +\frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. \] !et -and +We can then rewrite these equations as (defining $w_i=1+\gamma x_i)$ !bt \[ -\frac{\partial {\cal C}}{\partial \gamma_1} = =-2\sum_{i}\beta x_i(y_i-\beta(\gamma_0+\gamma_1 x_i))=0. +\gamma \bm{w}^T(\bm{y}-\beta\gamma \bm{w})=0, +\] +!et +which gives us $\beta = \bm{w}^T\bm{y}/(\bm{w}^T\bm{w})$. Similarly we have +!bt +\[ +\beta\gamma \bm{x}^T(\bm{y}-\beta(1+\gamma \bm{x}))=0, \] !et +which leads $\gamma =(\bm{x}^T\bm{y}-\beta\bm{x}^T\em{e})/(\beta\bm{x}^T\bm{x})$. Inserting +for $\beta$ gives us an equation for $\gamma$. -!split -===== Finding the Optimal Parameters ===== - -With these equations we can then in turn find the parameters $\beta_1$ and $\gamma_0^{1}$ and $\gamma_1^1$ as +The solution to these two equations gives us in turn $\beta_1$ and $\gamma_1$ leading to the new expression for $f_1(x)$ as +$f_1(x) = \beta_1(1+\gamma_1x)$. Doing this $M$ times results in our final estimate for the function $f$.