From 0b667b89bfc71869fa488b02bab76093270fe4be Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Fri, 4 Nov 2022 15:09:23 +0100 Subject: [PATCH] added video --- doc/pub/week44/html/._week44-bs001.html | 3 + doc/pub/week44/html/week44-reveal.html | 5 + doc/pub/week44/html/week44-solarized.html | 3 + doc/pub/week44/html/week44.html | 3 + .../ipynb/Results/FigureFiles/baggingboot.png | Bin 25308 -> 27292 bytes doc/pub/week44/ipynb/ipynb-week44-src.tar.gz | Bin 294283 -> 294283 bytes doc/pub/week44/ipynb/week44.ipynb | 1205 ++++++++++------- doc/src/week44/week44.do.txt | 1 + 8 files changed, 705 insertions(+), 515 deletions(-) diff --git a/doc/pub/week44/html/._week44-bs001.html b/doc/pub/week44/html/._week44-bs001.html index 91470901e..c0c5388b9 100644 --- a/doc/pub/week44/html/._week44-bs001.html +++ b/doc/pub/week44/html/._week44-bs001.html @@ -340,6 +340,9 @@ MathJax.Hub.Config({
  • Note: Thursday's lecture is digital only due to High-school post-education day
  • Friday: Decision trees and ensemble models (bagging and random forests)
  • +
    diff --git a/doc/pub/week44/html/week44-reveal.html b/doc/pub/week44/html/week44-reveal.html index eb36876f1..c63319700 100644 --- a/doc/pub/week44/html/week44-reveal.html +++ b/doc/pub/week44/html/week44-reveal.html @@ -207,6 +207,11 @@ MathJax.Hub.Config({

  • Friday: Decision trees and ensemble models (bagging and random forests)
  • + +

    diff --git a/doc/pub/week44/html/week44-solarized.html b/doc/pub/week44/html/week44-solarized.html index 121c3d6d6..5d8f605a6 100644 --- a/doc/pub/week44/html/week44-solarized.html +++ b/doc/pub/week44/html/week44-solarized.html @@ -299,6 +299,9 @@ MathJax.Hub.Config({
  • Note: Thursday's lecture is digital only due to High-school post-education day
  • Friday: Decision trees and ensemble models (bagging and random forests)
  • +
    Videos diff --git a/doc/pub/week44/html/week44.html b/doc/pub/week44/html/week44.html index f9ed34672..cc37a5b48 100644 --- a/doc/pub/week44/html/week44.html +++ b/doc/pub/week44/html/week44.html @@ -376,6 +376,9 @@ MathJax.Hub.Config({
  • Note: Thursday's lecture is digital only due to High-school post-education day
  • Friday: Decision trees and ensemble models (bagging and random forests)
  • +
    Videos diff --git a/doc/pub/week44/ipynb/Results/FigureFiles/baggingboot.png b/doc/pub/week44/ipynb/Results/FigureFiles/baggingboot.png index 71702772afe8a2e580461e25e663e6c1a3209268..5a9dfa0705cad42a69a42017e56eccbfd00cad96 100644 GIT binary patch literal 27292 zcmeFZbyQW|*EYP*Idpd;rHC}rodO0@64D`{bW3vpX(24G$C8fJtLO}S| z#^3Y2&-?!Kj&F?b&yT@9@IHI*wO7n_&1+tBt`n(oUxg5l4iADL!rQ7!S`dUpLl9yH z2Mc^6JTS2U{)oHZ(s$Q(vUY!J?rH_8nY%mNJGt9Gv0(PJa&>#+%uGWpmQoOIxQ! zcE_oLUF}Df+EL7BipL3`xV1<~X&$+lQ&5m5(qecRnZBj9YQywkFx4+t?jt+kA!M>6TjR}-d^_JTUg@#dfO-Yv`73X zg^LtJ9wG%le+Y9c@#U1jPkl5ZJ|ZHbtB)TdgD=ox$+3bT*;YsuTwSoCfg-`>GXfMd zxSH_Z9J&gw?g~{Y6M75jrUr(@75q>Ww#xWm1X%5B|^V(q^ zE_fJX-Wmpe3H9r|1U5V+w`$glL@EW7dy>T6*z@x9cMr#%*lyk=vl}UD2d$J^b`sr7 z7U{A2+Z=*DA3`kS{Fa)G><76MYzAG+su=Plb;dZ*XoUm4W6dhN&5B!+w6DnLvND(5 zxm#V2ZdQ&L8CDX-$((Q;ZjOh4eR&-P73gwxJf}q!>PCP6es8(vXSTYEx3~D`&!2q` z2``P2<9#k-qQ-;kivbT6uhGiQi-yy!R^z9Wq2PWaw;o{!M@he9XPaXbRBaUNyH$wvTeRk9oS{sbsVIvtt|;XJ$~+Bc5m3aunaj= z_W5)Cix0?vZANA*Ao|AU0vP3e!x!`qk8XNTk6xP z$mKL&qM)FlR8BW@a~3i(GKq4y%YVYo&ivXsJAG?uOlAcUB_BQ@L$HY*g-we`Boiwu z3oYxp9DFy+M=SY>c36Y{PHkv2mfK9$#Q0xK{M^DQ+f)Nc%s{_*1nSeNlFyX@?26vVFfZZPI4j6{Q?$KS=#q7%+2 z>z;R^;YF=2F-ogE3i;Wdsbw;QpT zZ3)#cvnFI^V=Hx>lt_{Az?1acaBk?>nylUZ##L-~e)fk(%8Q?lj;_T2)bnhf>7uDG z;85^;vS=UIb}HC4@0oE*3<^ue?5B9~jDzm6hZyl5t-GBSwb>w)qbyJ>iVd?2>AWN&}fkR0K6W>rCZPhrXaSaM&e^umc{AeCa)eM zGBOgh+1kz5_lnimC#ht2xzQ z+avKrvaqtQb%JeG^5iFb-Jhjo90CFiNJvP?3`{Q#zZu%Ce%!H^FeWBORb9Pvra44Y zODp*KbAn|n9mar*)9Lvj>?;7DC@2Q^ot{ZW!Okd$iC)x^p*K}BoX4`rHj1%{ zhfJPC!=}_{ad;@hce{35=4{>c!c`_hbrae`JOs zpC;mJN*WpFZ{zLEtRGA?rXROP=$LvhmaCry5Dg+yW!#jG(0H!1o8Xi8MmF%eC<0El?P#ic5|J%!}s#~IR z(vuz!AAfhu=Dm4uia7jE0mlikx{LEu-gW7NL2WUg-#OhpJUoTe(}dngJ4kvad;)KD z1NlhHgbvJZBmRRf=eZ-dy2D?_@vavcl;a5q2=G=?a_Rma zHe@DFz%ky`z7O0)|EjyHegkck9ARw1yHK;&E#vKg9ueC{*E^p4k+p|0x+%0rRyyJQp(oB#a-mo&@ z)@nz649G?5lF*rUKr6U!42(n{5TIojbek;7eXB1Cpo1m!``Kr08dCD-{A!Rbj|fA9 z{^BOg1xd>z5&h(MKC`^OxGp*g;EQW3sbyujT){^UiLVZq9o@&NFzcrTD9-D6+g16T(@ zpEx5UqyMo`)mmPDiNkosRKVQ@ggx@`)tsf=v4)W-H!^VAgQ&Lel_CiS0Bfhj=MfQ7 zw@J{-e8va2Gq5DQjcx-xf?d(uw~>nBM6=D8{`Tq;X+D!@IDrzOA&KH8Waw*ImP%;qtde}n6Qk~-UI?S;GmF`Mulcc2lrrGU(KW}1xy`4)68M>`KeuNA^> zhqOXFpPj)%`oCmXbA1x?PRjcx00&fZ&#>*}cEc%AnRSnXkr4yfVLQ>LXGls)%ACSN zOz5od;*9uwCtTL-+p8O1v!T?^8>6KrzDx@b!HT}hf^Z^QoBaaw2*B2zZ(a}cD%Ak* zaKS6rCtofc*<$hb_O9CPpx^CRB%WPNGz~K;?o5;REwLZFDI_9-gti(l0=~Z$#5VQY z#cVh~u`4jCr*o>?#kvxBwtIz(`hWmSE-oI*B1_N0z=>+{Xl@=mG}OVZeJqs5aCyR zaDYc6(JeBHd^+VFjwm6vATPQ8Oe7Vh$vTWAitK0A%?@t2RayFj$43QB;>6 zd3t6rR~zOj=V!~Z^O4un9+ywWvnZOC8|*I2UeGT0rnUlK?*!wy+e4c_dyB_w`Nht_ zG#F(3Xr=>Bd9<{&eAWx&u>~L>g+ZL%hFn^G)`XdwtXVV9)Fm(A>ldUOkK6e)Oikk= z$*<21s4%s|6a$;duVlz4H#hef@RKBN9H#`F*k*Lj&*#c)?h=dy{5fL?c85xM>zW1H!@*(M3xpl@2Yw3sQj=l4h&FC4k3x>iFVRRaa$7`?Y4{Tkp zKJu$xYZ}uc$ID-@f4~pG@WU`bHkQvfpf~Rk+!lI&NIW3r&TwePnIDk81w;5E2U))o zlB8`NDLU&72CK9X4L-eQz(C2^-VUM>>-$E?uGARbY`8T9+Aib{O9 zEp(lbC`5t1`BR>cQBKajLNf@&-g6sP5WuFhx3_1&1Ng*I{JVEiF7q88YvMdq_z<&% z1l^raKiC$pZkt2DKjx4^SJAmN{I4Z!>|VLvQyoN+*dYKV^!Wim<9Sz?>Lmf#Ute3> znGI)}8#Z)mpKFgMWnv-*zEd+(*&OgO-u3(MWfG&Q2UBt0V&cEEic+uyKNEKM0SqA zPkv~)OM)>dJnX$VJn%DO!Bsq9H`;XO@6q(rv8Bv7nc#MdX7E38yYrp%fGer!>QY_t z+mW9D)7AC;ySsPV@89lSi>@1!wbX%tO-?rGYG{y51zz~Wxa`#US7ge2nGvR?yz2`~ zOLH&PWtx}Y2gdfvUNA^L-D-O$Y?sol2fzXz2>4Lyeu`KSA-xDQCnp}7#=z!#5(1Dz z<62MizBK86+wvsIr$uGDFC^4icRqgM6})5*Anw8;BBHVhaA%6I$E-TPHuGz-*P{%N zpz2SbZ#q%@>`LAkPQ0KmIq3l=At9ky;JFkQAsxo+JE1il5+5B@_wPKsx&7^#LB>{p z&2{xOr8{@<8vY&{eoqoWK`?J@Yj1~R|H9$M)vRO?{sv#1HeR5ZhaoxwECzAcHY;gB zKno2k;U)JtttoCimykKzQDl&C$3g3(j!)K$Gpnj- z5MMrm0dNjy2k4s`)mf5W=vT#-x(9%(>|uZX`n785o!!oq@3uKm4!U}J+Tl3vV8pc5 zy3D-io$&EaE3u}A26P!g6T~Rb?=g_G->aHi2l#u@Z0J-UT4vMN^5|x>ZFvQr4v0J4 z!(<^intGD!mG=Mywco7;t$R~e5B~sPqLL!*J8b1EN8kmbf5sE;9AU7b?K0US9&k37 z-<~~Qc+BFNdczV3Uu&7j#h|V_uKjOZhy8X>I{*RL8MTZv-~3g6Jkod`+SaD@Jw==x zp2R7?eYLuSVMEyI3ege1;WM4eyM3@H=T-gp^k^RJg0{PS_0PoJmgKU3ngj9=0(w!S zVQmfy-neUIxEr3e$7#s%Ky}YrM&Q{y0i^faJ_K-8JFYbj(5Od%;$Ay#lNL6Hklv6f zpeyegc-poEt&S%r9`R9vD5ypekm@Qxs<*1={WyLPEf}(Kxvm>$_~sTZnh6awVuw;=IvC4s^sY(Q|z{M*oQosE8kiY)c;xuuAQJ z7v$jPZhLX<3W421WW6bf)A z$pZ$?*LaPmliMX0?RaO67e^QI$;r|1G)_L9#)KyPk0x=5i92Den5=y|fA(j8wA$@@ zn%5kzQH^`ib*{R(^9yu3W;zXv0h zO=tv!20Q!;K)coyaW{Vd6L%#QmCCW)OzFxASNwp#>!$OiomUiXZ8>3e3*6f9vv2=w zzi+;>@h=Y?^}KpT0r0EKJ2u^*u%pPhmfx%i;?^&LZrr$`U*mpbZGHVA+bH1%<^-X` zA?t}XWyz6)S&Fu1#X65wgvuU`i0jWJ8H%-6=ylS`i;w=`*Ba6SUe zl`T7`qo)JdtCDoLjSs?Uyn=ArZ=X=aeiYpvO^t%AzbC<>gA|B2^}jqOeRT8fxWlyn zk?|v-$4xi*CrNpKb{iM4>@aJOqAXMv!A65EJku9=E+}(6gW@)->MwI*=f;i4M#hxO zEqXIceXUl#k!b$IOu9i*CkzaapLJL^KMnY|E?>VaEa&y%LIA&oEu+9x!dqGtDp`sa zD_X;epN7fG)e-bcA5DaS7o?}BH(B{dw+DZ^n+Q{^&3@*@RQA`x#?&;062VZ@V3zfa zv5nW|%&qzVZ-Szh>+h9;-o;Iab^X4{im|aV-hJ4iWL8vA@im^gK8>CfQ4uVu-G{;> z5j!(~I0qjVNb6yN6<`(hV~2DfJdi6MseHGcq^GGlo8S<4MLG(`(!kU4=_0X~KZ^jZ zo-1c|7piSyG_jDvnGz+-ClNndfPD6F__ya9lk*wZZr5vQr=YiqWU^YEOo zkWhL}4IOY24xSoAZ&t+L~R&aU!C2yJ5UKw#Pc8$J%Xm zGV=i^a1%kag4Q#;olKo|-uoI49z5F|ueAL2<;j#6*9T8{Q-QE8mf5%WO{=wy(>1Uk z))>ix{(h|8h@yU3bOUROhDh~e(>Elu+~(&|+cL}VcTF>gEm}t!8;Dvz1pDMJ0E9VT zkLq-iozWIX|BV&F>sNw;8992Ml{RqhIo&@fVG*^tLm2w|F6Fdv!^U)2Lw zg>nfZ=veO8uHuk^#_Yc9(yWQsNCBPYKnkmAs{g)9d=o_5Qa*POy3aL6a1Tb$I7Er4 zQSSAUeJMFl3?P<%NCSPMFLJ&uwyeRwfeYVLj|TTB#X`&o+bBr;VSal70;K3ZOtpa< zUl}mlM~E5!sG%yU#)@BtQv#ZOme2;zW(xKS1Hn6CINBP-r-6L$4yx`>8AI;RG4*wb zYf~+OwgPUofNyojh{V;D=R|PpN`AYyWWq}W%S-g2$8(R7yt0HsHEHYF@wZ7c?bdKw z(wg7~Tj2&ZgwlzyF(fs7J`7nLV!)c#R0}j}c=g4hWN^uKSfe^11C0#chn_E+0W?q= zybB)FmSmJ4#kd?-C>YnjX`Zx7aO0K}K>!U*lNfYg&HOgn2=j89jj6t^Du*tDj^yixKb_)#M8 zs5TbjqiaEh(S4!^Rw5PQ|D?EB@xZl~~Nci&dj}A)t z6k)VI67K00_-ZNVpmcw^bg9SHo?sXl6sB}e}Y|DHzfHbBwXet1mAVnGFKLqP&bX^4^ z0|hNz-KC1M7Dog52W7IUMzfo}eGptlB7y?1fT!XlxS|7TZ;RE?cDeZ$rb2_|~*vAhbp||(+8~nsAEG;v%a$%`JEm_3l@%WD)THGL&Gdnx0kv{}z*G=$a zP?RFo6O}P3b?SpvoCL8bei?+T%rwYUxIHf#KQxK zE;jKMI56h79|gZ9h-hlQFp#DK$=snOPwO&TL7QGB4-es~fKvx0Wn~KxIf1B{*K=bu z=*iD4@sYx!q7EQDB!ECQJ)Id2!{LPI4#-xpNJ`S1o153~>KbE)L7a^JW>6jb60e>FXzr z8#I7tAZNqYks*ggS8?`xN$4BeSu-F3)mb?m5Dj<)9NBNaars}_=yS?xz3|YA90Uc% zvi-6?`k$>4@BX8OAb4!DEwA}58*nBDnJsiBY&x#%1BQx}^YfDehVB9u(sM+c^ED7y zJgdE*7v+i!SRmIpQ_KF-vmP7d42TxZZ@%3D45jw_|3>Z>zP{v@4lUH>8ZI`C`|?Go zt)rvj^p{OvT0002;$OWQtQQ?%#G@pkV7p(rf-|>*o6^_a4Evw(O8AZ@URRE5GVagN5=c95}}Neo?@o+qotO|AdeR_?$pQ#5+`8~INO1!wF|;% zR&H(rkbnHBH&v^x%BrD_@xTqMCm!u2}dX?+-S&?um5Y5s=~VY>GE^=nX&&;gjE z%W~>diILJVdyw-cF-WAu|M$*Im)`}c+VB@+zS`ATBQ$#9E7=xpAqwWFhZrAv(Ds*4Hm1-eG z2aLpwpGL|}X6ESy{)QhVO(RCqQH9&#=yYIAUX8N->HAz7()H$jB7j6 zg3$=uN|TUWy|1?{Y13J`a!K+A89|%MtS;jO6MjU@|M-M2=xoAxDb&tB)3W9>y1{@C zQHBqh+2Fr9q!?g57A=|8FRhfagRbh0yE@ zDUJgK4BIpW@BWcTQGV!z6JV46C239rx8=jAmx9u_w?~iJ?ki+~-sZ>xm0QPABKVmd ziXAs=;@KCL9!CA`U#jN_-z9{aB5qL7gL4 za)$^FCn8BTN5FVL#|O>%0}q{k>pzs?11}h#72IDTQ?l59v;c$xCpT5DlK^|Qb?J57 z!fClIwM*@UR=U3=pruIA5X~L{sY@0-2`r`iDp_SuQ!M+;-G9^`Zqs8k22=mx|KdG> za{H$y)5&$}qsyu2^VaC<|M4OCb7R%@=5!rATGrrK&_B*95eH27-;y_J{4=~FKvp~) zVlcb@QTSV6l)VEOMtX=OwcfGb=JbEDPb%1b>kpdc%yPtxHybX^Thj&*Zp_|&h)>K$ zI1aPxAIoIGYHDcO)JI!D+&OvSlf9*vG15E@bPDqu3wO@0k7WAxp05UlymHOJau0&( zKE!mig%)q#09T=Z#+BL%jZ#0WxI#cTBYerr?LY&;uz&Hm(mw|nm~_y;eldcuhPyT0vaMXqAdX0pE>%Ly)7Vh_dy#Vy|G%9|086oiwElvgFqz4wt%yc%3 zEg}=Qs!JnIIe8L46yD+u5r~pvXCuOox5%B9;>qq*Q+Ogc3aoi9pCN#svF4To0{ZcP zCb8EX+TfZnBE)z+K>F-oBoP8$hNnW9K1*g+CSfD_-<+2jNN?;fak~@wU_s#lCnb!w zfCCXZ7#bKvUAy}RPO#XP)4jZ|uB@ya)-3_5p^EGF@ZePY%F5%%CVJGRUo>3^(aNF9-F6ynvLguCBuAS-AAc$p2snt^@>HT>sX8F*0Rk zESQnObSEMzss%0sbX&f^>_g3G(gv&PfHBN6T^wV}oNfqC1e|Vq?Y2|9?DvQ&K6ZIc z_@3=Gk_FZF)+s7fcT<6Iz}BNAa_Xu(KPVZ4H-fWr9G) zX9CpEh&j(dczA8>+8}*EcFw?lKLrQs>gzjdGh7vifdDInSM(t${;3&zZ#(RD<+o z8_eO3CZ9_Fhr{veZ34BktE;OCtmDq{e4OP}oi_)_Cxci=0AvD}eXie6X{vTxHWNv9 z0eK&@y@jv5m9g#^yAmquGi$>L66lSoEQKPg5L|k4bRj+9y9i^Tj3Vpt?}I+Sj z0p`yiRfvqLz`RkeYyNK*GrGXa5i=IAI|Z;M!m`D{_-x|Kb_YF2@mRWl@4g1!%^^k<{|%9hnR8=p zFpF!6olTx8B6!WGU@G8qI37%z8lEFc_+ETP7}tqSMuIu$=UDVA&%NQLxO8hnTzhNv z2k1*e98l2mYx+ssX5X>QRy}=D^>OBPuWhY%6>Vc6la{t!7wAo z0}gK}hE7h-9?$bP-ucnF)JC_JHc;{s8o|`%O8}Z)Vf2zBw)O`WM3NkAHwMq@0wA5I zoGQ=;D5YurM*o78nk?e%n@`>0wSSEwK%^&X}PB1p!9e5JLfYW}V zD&=}5y>(wSdxq&KRwY=5q`d>{tgnNuYM<#eLHp6hp&P89I01*!0P`S!lg<~j3L^DubggLNnX^r9wHt7}JAFr0P1@~3la z*W*#lnZN$j{PQu}rK?T1MmO!LfbFiWgTa4xjKv&HJJt|B_v!QUa}K0?!CIbEP(HKD z)9JEaQ!nOP`fa{g1yrJ7$TvkQur-Kl25VV{fb!fyHc&wac_9mRN*oCYZf*6~2pp}$Q9Dp0KxL`Oc9?df^CMjUwmXwwZ-O!}3fU!{=3jFZ ziE^O?yBR@ZhSn{MDqm?*@yM}PT@K`=e_*X9P~=)MZ&=m^oE;(v(33LdUCSsT-}c(K=qUKkXU#_#r4v*>?9wUvZ1@?b-UhlflZ-P2@Mb9u5(q^satLo`m09 zRm?po%7pK(RUzhhEcGlffdn11rjO0$RsMcb)>%plHxdtgyz2^kE!qdpY#Dmi9u`@r^|?eM->)*Z(TQ}jH9;IUuSg9{Hl`_QwUjYWfk!E!eA%*q4+7j8PpcF zdh2vanMR_PxxlnHU*kZcO~}#t*kKZOINBD$EH7`8oZKvCYV?OtW}?S!M8AKTr57Q+ z9ut#D=Z%;QTT_?((U38@Sfk*y3+-l9+0=K~dWpUCcAAqyL0`{4XHf}?H4$MJoxu;Fe`B$=Pj7YLLlR&wG)D@e-Y zKS}x%GBQ^(=RN*bh-9~oq7|w0oH>fgz-r7YWejXC3pmQ*Y`uLIL9fy-_i3fJS)T+W z<%aNV?MsUp$ya)j%7*)=IyXKhNr@2%zF-sZ*T0JOEVuYL<$#Jl)~KK*2>-R1a|2m8 zpKeCRz}Dyc@mjIFC`EWZ*{_CKXjEa5%72roH5C^6f2I?Z_#t4#iC_(R5HXyqVOKGCFZZj}pCm6v zcjopW*N=yKZ~=H~?h z?HPdBI?1WxyWxpg{-)N&D+KI@g_Q%EHMj->CJk)&i{JLi#!ag)=or4`x5QC%x^NWZ zCFZRe7E&`7v>XRzsKyXC0|WDWZM;DcnqiAcbzil~m>k>8tSW2VYK<_gM)}OS2wB_v zYWDQyCyu&;H%?&Ieu6~k$qvpvI)>rRZT9G)*m+DWhR1mP)iZW`)1Ow0L`?*T62 z@5KUqY5!x41O9S{RA#*w$d^n9_i3R83fkQkn=!DYpNDP&A^9T?Pf-+MM?}`Rj)26< zV44Gx0s}GR5yK(+R`a1<&;!mOE#6Qq-ViO`TWBLr8QqAuMXks_zuA*_T9&B8MO`yf zM{)3{ijauNH5Ws(nX_%B4SjtIyGAOupOeMOC2i%;o`Ij+2O3kG#F@@$9o}VqLc9^P z!w8P=4V-wE0#g)fFH(=&{rz!W)@cwzSyYAJiv^?63Ic2wro( zKYz{dtqwt%4nedQuJdFG&V&f>l`>@|4@r&(BM;k_hNFi7nBuAI=WG6)IsY8J2EidO z&Il)I5ux*a(LQa~&rc-y<+$}y+oLk6^Wk+2-^+8pTI@lRpo(5tF}tTq(EcnagXTR= zR29hx5#gr}EI%)PCVFBccU*Pt@UZ!Ib=-hSs0eV7v0t)^B<=aN>tR8XOM4Xwp%M@o>~`hkuA+b1p6 zt8;Q=TDZ)wRhgTu;)j!uG)M&F;z; zhyJzBI)TzE2S2F}_Gd=B)deDw_Y;$xubLLz5`SSD>Sk45IIMl+V-=?^b}h5m(Q0=x zd9e~XvtkZAbSn&Nlz04M2yyc{BT4XXq=ohZ=rWF$#|E=Q8UFFsqZ`9EOXka~H*Hko zh;(OrBXO(t*ANlI2S$WXd>17&AGb|SxNb-r#)3so&6qS-eTFr{9%^~AqYAe6(!qrQ z*xD9~C|;eRSuGv^XkV9_uF?Ly%&&`A$>fTOU`N950JVI%^Xv(dBui7Fm{6`tIVF1p zgV~544UQ9RURXU*Ljgk*9oo^E)el0v>6VcN+xArw^K*KVC>K;46JBguAHeDc8O`T$ z#MEYeZ*SU^5XRpX;JSua857q!m_aK2OF)C9ZC)WlBE7qB_J0@;(O>*yKXHC{BIhHJ zAj<&pSvp|D8%P0?G>)6hwf&>kF>oHWBZxPZ@=rp1rfz%(Frb}2S0_~BWG^?J)4l^ z|9bS^nT{v!`KdNbZ=xtRmvK5$G&ZWEud~g3j@iG`_Fiw|h(-78-Q(lVyS&2YA#6f0 z3!r))^As!H^(C#!JKIAd(}I@LyI6w{?F-Y~00aZ@;rEoo=^(;s-&tW!M15k$$tfRw z7dRmn@Ov;x)B1DO*~*Q4fl53X0SQWQvO;8NXElL=aUtPXO*Ey)sPhb~jKr(Bs4TYt zD=l2>PfD{}w^967s$Oxa4*EiGXEx#I!0Uf z*H2{6Y>{p8IfQ6-x!4p7wmmMxz@xA{N{i9E6sYMW4CC%V^h8Dpl^ypOeo|PZ_NEz3& z!<$%5jysWz97l#T)B8aYk7fq$f#t?0@Q4x9eB?e%1y&pQ(OezEun=C2fx$D%8GB?j zjuxvzzLJ;5ejUGqTy4GhrYC|bnQ=*3fJ7qLTrrb|jA!`G{$|NI<|rm27+S7}5QkZv zo(OdR+40_iy=T)SMF1yb99S9#jCOYZz2NkufVVqT}T--EA=ykd!z& z*g|U2ELwGu>(6p1ISL$`h}G0;0Eio6u76bJ$s)jQU0|6_64IY8G#yDZ17PXhfP}qC zGWj(QM@zXWKsQI?B3gJXEnI+4!03`T+&8PlKb|$hlsVretFWRJw_CYNW_E_w0BU2E zbdK$7*N+mdCOBH;3K=6_TY{7r2==mL#oW)s!AbE=SGLhs6~A_@bOmHa14*%Yh2C!o z@ZXE6{YmC>dhGSgXGZ!>^7qk1!=QSbZ>7uj;&^$+vZjSvvEUGevx62**45CXQ*(WQ zKtO}l)U;Fn^fh?T0H5>SeUBTW!s>d1Q9LXTsRmHQ>E=!jv&|?y@HIi^YmWW_!OURC zMeu~<@2n0Wm*X%X)mWmIU>BDvVPT>KhZJqQN_>u-B5IULByRTIdt!zLA!VP~2W$SY zU_ywSH)0!wYP;)?{O680tzc$NE~#;gcW2oJtw1QH&2j$pNRu!|B!xG(8MtGNd74UeB;dncy{osaXy_O^{1oA(l0d+JN;(lKj z{q!Q0Hl%Kfunaz|O{B=(98nBP;Vfzn`z~cKZZsn`!Y{|$u2b4PN6ajbRwDr_egHRa z$yhs+N<~&@QTMyb>bfv>mX~G|Xbl^}=_IdI_0;ecP&-4PXPev?UX&x;3aI4AF6QUA z`#p|@&_$2GjFqmt(*l;)BzpZ$O13*VjsP%FpP!fL9VC7~QNs2BOrEXnzONc+6~KTZ zTSz=bZhJ`%GeOJ`3$vsw!%ocqL7*O$L&ZvnU`DS!g7lc*_r-HYgoJ*X6;Rtz=C8Lo zKhCjJ(n)3G5NXli3u^~CESfc-ciTPFz1F$VY@YOjBFrxA8Jm3IA`4%RgEwXQsBZ-l z_eAdbZ5(-L;wumQ9~O>9^#yRYo&5C%%+qJX8w3=o!hRrPlKvt1&L=UWi1a-ZU)!5j z%;VB_@$xNA$U;(4J3PrYZ8mdpXv_9%*Cg$iJ*%lV{F*Mc#Eu7xZ8vC0U?dab?v=;W+ny;MfsIL`B z%j~(xxSbucrYQ|D-3z2sRYiTrn#sH-mwEbf5mp=_tfq7MEXeVKI-Lj=6@Y9oSiL-8 zmH-#?@VUFpiKFA8@F`TR|H8k|K~8T&{e?AhMEctOizD3hjW18T6+vh(6NxBke0wg$ zzao)dN;P_h-}K{TPa9lVm^o%OCAq>=gksjC)p#LU_kX8Ma zmSe>X_=UspaQi=R2f=?amI?mZhPaE*_)EE6m+d#9U%uXvuRXW^H74B}(>e>_2Y-2_aDdOEkR+W%koL zjD^25lY4NSwBJ_V1+%3D*0P`J_SL2yC**8v&^uPM?!Tab zcT)?zMUxLCg7oebddZ|K5T=}789fM`;spZMhM3ZzKrp1ghCZ=9pN?050SZ1AZY6xR zP@5SxMNAiCk+CBct-Wd=G{}=cZiMML7?1Fr5<7egsFA31U&jK}Q;HcqAEi1};jX9OB96|3h28rsNb*AW7_b37tC+b)=J@&qn19_aR~_YG}>I(`w|8$mp-*?Xachq0D?!DqD9YQXLGeFSwU z)l2yLhbye47$)=u`TtYrs5ee2I^A%Y*|{KZ^y}cp`k;&)U~6Z$@J3hOU4E#GuVILn zbf*n6<3!y1gwEBkD}GWV`Wg}ugmBYiFoHh>rRy_`Dc2E4d8QH(95{CEemt?A*&v!( zpq{I*P9BMWgK0!X(%P2(iO2~rZg$R6?U@)O&98ElX$XQtmMctBrHf0J%LHrSSAXd; zPAs|Po%WAWN2R%Q(<3j}@)3#+G+Fuk2up;SaP(}OBc{FX?v$g$I~%)q)1wv*ur650+X%{3X5Y$ON0kQYR%{F(T%6HyMbe*D;uDO)N1+mqaGFk zehQrah(OEGd6P-^X(7gRN9paogV*wZwHmH81bl#Uw9Z=)WCjvaQl5iS zRyuGLZke(Y90>-+*jzi{a8G7Y5s_ub^Jb>=9c*yk;ipwSd?F@U)Tskhr#SB|+}1DD zbw6NR5tY=il-}9mPCu?%_scaR2pV8ZRx}?}6(Biy^l**y9Z|$ERpYN@qA*9MN~6Br zQRLto;c#)4(fs0#x31l&WCe?R+WgO-q`kq8Vsyl)^(q^0ql?tt-%#6sHoT8KkUQpQ z_RI0o*!hxImu1l4AU9>QE9eeRlC>@)Tb(ErHACdj=!f1!A9=i1JBsb3lymqwAB$D| zhN1i#kKzY52HSVG)Qfl0_fqT%O&p$#_ubHePMYQPU+Aw>i%|3{Bdo3meRz-lkU6(s zl^J8ph!-+HfgMn5%&p?lAU0*~q1`XIAgZ?fR93_Hh0!7%LC6Pmypl^@e}B-Ndl$5g zeJhzVr`L_pD|Yo)k-Npq2-$m&5Eg`i`Kwg>NFlY^J3o8B)Z9p~e*13nDV&`-R^Wf@1nm3b?Kv2f&1ojUwGxuGWu^$+)fToeXumnl2zI)Ofsw| zB6>@7Sag!%XSw6jkZ|F&MOx3s+s`+I=495Z#E<5){KdtF1+#lb3+75QvQ!It>d#DA zF}J2B7PGhYI=6X2#;?j@Tx7!iry6{O2R@mRBI|o-|H#H>9wb)dzfJ^xE%?8D;R-lJ z^dl=|%rZ_o3|PQTsD=&4q9>G%;Em8Er|axt6N^qn28(@(mvb(U6Oo6)bzhphlombD z3Ra_NRi4xnlwx--I`wWY$czjlSs!ThbW4Nh6WvK(*U_hm(ApQFPhz zVRYwRi{i|I>!NLL4tdP1@r8}Pb&DCKouNR+mCoT-bT7IM)9zBVCQ1Ex? znh+BMpR6ipc^Z$6&Df@e+M{j_eN#dj{GdF13kO4;G_Sx2lv!1)d>6Qi(0}+ zm$|2!`DS_>Csh@Lf8+SDtMR-ls=DPHOqm)byFl$2`~?A^j4~rH56{(mpYI@sUUaUg zcB^Ho-q)o^c%*956MP*4NGW&1-+!`H4300u-&@0LT=T4|s!DfU0P+m_q$>s0V6<*< ziQ45Vw9$#EDeeC;dDpeH`lkEjHo0OPfAYIzsc!X3vF%m+Q`6oWjNOwy9^xfD!gwKV z0{_;f-}ipH1`duwudGp_D(!tHg-;2+)AA6zn8D1$ayq6YAsK8G5?lHE(#`>?A9KHc z$z2taV>|b9JBxQY%fXGc$;NTAE1(X%`b8>y$oX~wLez-Ni7^+k%@J9Udn;@8H(lcC zW3#UByxZgf-e(QZ`+g&PzkO4s^m3^X-9I|)9w)SKTk@BziYbX z#8BBCzi>ig-9I?5Hug_YxF!?1Wu&kOPiscGd;z9=n8c>l za=WqTl~mSv+2a$uTj`?`{kr%;le{A16$GJ);cX|JRh~t)+@oMCGugG&dio$dBL3yz z39d+04f)lZY#U8Y{rf^2o1`A=|EIe%52y0&`u-M~Qph}oqzn}*nP(x9dB~8VWD48N zHf9JZLm5($p-kJ5GEZfeP-2^vZJsmFs;4b>-#yMcXX7TSIVGFZeLVC8PUH_^I~T4>9#-)#W%{6P7U{}E@H8?!Ko%k6{GK6 zAoq^a?Hz6xSvGP?d|0i{N2gIm_UOpi(DsA7Idp}}0dHMha$0tJVr9Q~7Jk}HaQ7UF zGOED{yNM^SCB(#x#!hq3Y1v5$nm;wPZ(UYa{6U*)vAsqu?Ro`dudc>A2Df-O-?2K}n+g4N9|F74>86qMq zvx1>J66xwSEJvrga?H5T_u>(4e9@h-FZ0kqj0c7bDiG9ts80cZ*4R;>fRP|%QBQOj zs9Qt-E`d4hNnV=+)o>jHC-?nrFJ2kU4{F1dqTSmvhQ)}x`C1!s~1D>nX_$6X+ zj!o%b8Srj6+w;t|q+iuB1z|S^jGnwD&cfb|ED50#6V-Dkx5j?0w<{CxyB>MHAsrtn zC}kl|t9`97F&27vvIJ4@$;$iue0T4>DiD7%8fY$=u>0JgjS6OLMQdD0+OFC)rRiQB zWvGn#+H6O^pRLHH$V;)e;v$wmq0ci`qRJ?1*n|cd3*!>OmL(&F14q6Mm{AdbAP{k= zA%jm}-$(^FeU9`=i%T{z=lj%>CDYtuEv;mwz>O-p{Mp^P4VUuB&5q{b!xK$YFGcqDpZh&2Rk}N2J({7zj+b2_nBEt4V0C2o zkZ@G5q{BbJ-#!ndGA&RaTta!=fV4xmURKfa$72=@3tpa^RnHkO7bU9BGEGxVFL&ic zFWW91?*_4B}RXevx@5Gl$Nj#X&6yVsH(Ra5-}X4hMOCgSkFs`--`IK z+%g-m(4>vg4=y1vGu*ZY6#W2kL*^87=unsoB@$S#v*Nl}eqTgo$? z*x-&Vk?}}Zv(z6(l(N;#pF^_A%Fk{xo3MCZAf6>Ix);+Lh7<0!2``ifs*p$%^e*tjC@8u zwM$O5nQzwR!!c9#NX6JoIqpo*)8gpycp=L1*dAp}MV!>vyFz%lmeGc8_AxaaQ=(PZ z3mdhF_R{iKm-a9n3>``I7&l*$GhCog zoMa9qw(YWgep2M9&R8bLO_u4mPcbPRq^*~yI6v^vz(|e5tTJ9uN_4L$Y!KqTcQReL ztKY2r=j5Dfpy<>3iRYi6GYb&c|G``=HhsrcCCTDLzGbdacj|7uwOrx%@wsfQ5GUqH zK~ya>Eb`6IvaP6ARR0CGfd_vu>xG6}-QtS&=_>>IuQ%rBSIL6{Kj5s33eoJF^b1YV z+%lD{hHp;tDuqU$B(xY7W@Nhhv?gMY6Hr@529I>~vIf}#C2z8X6~*2Wy6SRXn@cB_ z_9;=0;^3XhOSjZobK2`79?^(uIUh+>ON^n43ryow5W3>V^Dy*;LY#Q6#;Nv#H(fu% zGu|o}?R|G#s4k`fGc-wPu^kr_cI2~TwqI7jQ+ED1H~|;_$U@MA{H&tJjO))kR_<2) zW@mbrt>~@O%S-#MD)iHuLT1RG(_SWfW_i})o#VSN{+e255Bx&+-c^~4Y|}e5pq$c&eAC>lPFgn!J7_8+DOjyREKe*88Tg9tCv@-f)qj=Q z5o5l8+{Nc znxj5NW#$6@e0yY9>!yaU$}$InKTLjrR%xYrEk0_zsJ}w4LMPUGzVmRI2~Sk!WyJ8s zFeg2_F7zyh{@FupeZd^s=EH!4=xu*iAM!?{qJ>BI6DQ>ekbgfTxo?F3%77B9O z9r^KcEk)xG&3fi(omfK&!*T{#LlQnT5)@s>>x7;A%q`ayNrOfy+&W*rY$B)84I2%ytS=r@ z>5-yV?5?!5Md4LusaY>bJ-Rd5@@hJ^)Hmv`-}?^_(J)(zxm?CI31H{FbIIAF4_>vs zx+>twm=RGI{*D?67lcHqcZ7FDb=;ACa6D)w=*Bl&Uhp>BI-;;7PX^N@r`28TZ!+(r zoiiE{QBUkk1WWd6`k1epDIcdq5u!E6v(mdRYQ*-0N1b+^ujWfMvm6z=a_)-9?ZF7s zL=U{-fKVc#clIUPzGlP$x?YI zy?08UcMjIws46PVc3(8tRyhgRGISvJ#AJNmOo!iS|3EB3L`uI^(ntE9w_nAFgkNFx z?3xY*9=D_=-9$4R)$2-}`$~PJvj^@q7rnn@dLjf$sdc@9aE*`eaoI69`D8TlbXG0Y zPP##tHtasP5@iSvor0|s$z1;Y?@c}ZD!N;fxB892u*f$dg2LhwJ(RC=$Z5(_xv8?b z5s1TXXG6=Zdqp?(n{N2i%BTF&WvZDj59O>?dK%Mz82w_NdvRihZiCJ-Si|$E3gN4| zZ_dYbLhH`j-;VGr)kcI-g||-Ov=LfW1TC8HG!f=_2)gN^H+~` zvUH9^Xe30dAG!6`{jFmkWehO~Cl5hMuMrHn=bRIDI)z+__z~&{)Zeh{5DSPZw8y^~ zW}P@GcDjc;TszW2uez4MyNIAN)g)(|j(z7?F1WCbeQa`y9Oe^=CR)S1tJkwHjn6^x& zSUf%BfhP8wc(94GKUGCW9Ix|Cih}7Ny*Y{q!os)tY^@9hTSI+}?Zo?<>x7$h3k@r) zdC7LDCAw-OAZ z1vN;d)rjrwd#Jmpg@jU%RP}Qr`1p1MDpsn;0#gB9{2wKA=9wKVWvlO2PEdq~mW1+L zqT|%0o{@|xlYAFO@{kKQX`K=i1MNP;Fh1A#%Z#NIVcIIx^o&y!Zc#A|x#Sx*Ouk7F zbwL0sYJ{N;&6GbAdWMtx{83&DUa7%_nJCkD88DfV&5_8Jft_3A3Py|IN}9T8=SV}W zHmp=AZZ_h&3>0{EHdUF1c?&Kmflq=UK+w2jtfu2JG1lv+%b@qqV#hDGp=$X1bDKDH z9}Dy1lMb1|ybk=kA4USwj(JYPKP%deqIa98TIgRM<*HX2g(z2u+wqY>r}~wl_ST%% z2RKLf(m~{5@PfhCF0VF4Ae4f~97*Q*`}Uw5wQ4{S7#Ohscd0;S}p8E*IAw?>pYRs};|51s1+@1AsFFsXB z!?B_xXt7&Gynl{D;K^g9Dk+A)?! z_ZYkXH)5tNU+maygvQpc$`o=nrl6K0C_6jy2)~O5_!?5q3WK})6r6Et{gC(>E)KGF zLU~=zL%b{e6hVqYLgl{*@s_6hHxN7&W%En@MTC%Gd1#cc1aKaN-wlFmjaoXmr_-+| zbqLTv=|;%5#{{Yafs8Q@SU^PnTm;B2kAEV%L6Q9*5O^x{sz9MLjBYrN2zH=M^9V$= zEWkYe1js~-t1o?47va9BcZec5NQ26b5uk=j9U9a{MnZs3X1+4k)VTxo10@fSs~2xq zlP0N6c?=<`8U!p@>$fN9y?PfFh_h<~K~5^pE;cD?*2HtHF#@2EO&T)4i7HG^p!&<4 z*fZ6Vf{;H0t;7;K>B(O}!NW%4s?}1M>FmUn?L`IQK;tsUPpW~OuQAfs}2ZWtqtH&1BK6>-!41(GDRlRls5j=xThXsX@IRBEm zKx!`n)!4}dEV8gJ8${oal#`ID2d<^SKh^tY`_uYBu=50dR`PnKXvN=gEbW(Hyuha4G&FT=V28Aqln zLCO*U6B6=%R`dbFa(OJ&R`oro zPTq_12-fAh=IdJx@8}9m`#aZgg#Y;My05sk?W9eLoSuLx`R!ka`Lc^3M?7#6i8yEk)eD!L0FLcux;aDP4s)>oa0L2AlAwI#_u?{A)-_CRi%huiXHEz## z8YFkLA|q}z#1xEypFYXXY+#UM5I~jXR8sJ93Cl|QLUJtNU@HK)VHAnpKtf&Dw%tvD zm48Tk2Ums$F!9*ygZ-?bjhiWw5wqP{T*AV1P`6f6P#{F$ZLmhh0K5POIy3onuQbZP z00o5fa3DdJ;v`)*0PzRpTRN|?h;xFLjihn$@%SeqKR&&={%jd%V1~Dr*<(U2VUxSM zzS55R4>I9!I29EYME*$4zz_zkw8PRY7`rpR{wPvsNA3HX(S=vyVX@JmNZI|?W6L7H zLp{r*&av!R5C^H1mDLdx$Tn{SP8NJ;C?FW>%mqcKP0P`8nmn}iP>#o7%l(v;W%|ti ztR@C2Wz0@~-%#h~=FXqO_4JTxX=&BNZjiiIUg%+>ETglOl@gnfaQ4iZGotK!Yqf2l z)D>{OGFjXLFbD)C?oc)|a%$sNc`2B@BgI1>i~(5`FuA#I=TfZQ355xi2dmexh!014C0SvvxNToZ1(Os_k`TGQFOD}~`WhA|OcDa#80|8kf^iZ(K6#!V? zfp52=?hPC?Rwsrcv+V3_6beycep$yz5y{Ew>CstQT4Jhf{C0dRVA6k8<1 zcmO|q6Fb?nfN;nWo-b7PzcR}BSAV&*gf;E%wQl=eY@gyi#WWmn@Eh^6K$|l|Q0&MK z4VcndlNPfrJ!09_a-eVy`>u6t!!ZL7cXu(H+G~M;$wqa2G&Ii5%L5$09N4HpV(!5U znE+`3Rfb)%YeC4AGW&Q+ugQ1=gl{P}?P|7ISt{LKik7K)Pe4d)eh5lrajK?9y>ppV zMd~c;dw%iLDD7yf7Z-&_UI$2RWCBr)1Hh3W1$8P}*j^b2U|Epr+g16`y8?1l9Pz9C z-;)z>W5~c+oWAY5at%R5Ui<=ERKK zEBM!sf2L>E0hPZt)zOe}!B|sAXZ%$FaISqp5DBj40mF?xL=<}<2N(-DVq#)3efw3S znLiWq-%@;*#HyF;kL4%8ZB_I8Wjf%2*M`)N-e~g@Q=VGb+RIf$>*A=~>yR@@ko$!; z3Is6;n9Sv77AFo75r&~MrxxIm{#rb}Nd{jlJ2+f`%R&wX0NpW_=dd#gs%StyUw<4p zBIUbtSOl^!`Fp*)li&tFKmXoA;9FC-w_i*0@+dq<+!>Z&y3cTP%K?%PV9^LbUGQ!z z73iQ5Li&#Bb`%1u2Fk}7;4jW!%@z$J-NHr9w_3kQOl8V};cyk{ z-5D<5ZG4#5Tu#>VpnW01@|qvcM}bN^;2j(i!YM4Q2Q0uKwX-y^M}`2Y{ulsiAX~m0 z2aBH|aA*i#N}gvEcT<5wVlq?pv7w`*=6FEF(siat&DR9&mGk=*07|4DbjzVY_9cfS zQsiZ20|nh?G%cV_NZ#EY&X@S*3P;!UPEC97;6wK8THGcsPr^V0syGG8#k!o5k|r@< zCgoLB!jWba{Lh0wo>=$a?q*hp=#tpic(Fj(SpS!wG1PuQlIA|OJoqQx4Fw<9uiaEE IkT(zdKY@5Tz5oCK literal 25308 zcmeFZcTiMO^EP;A28J9ZNd`qhvP8*I5hO>EtSCWpM#2CR1;Kz|AcN!}L6A6rfQl%R zb5wH9neSZm{pzjVUv1UaZtY)dVWIclIrp4CeY&51x_ctEG;WX((i1`uL~>I_Ne6W!JN?kX}q z_mdysXj3VZQ&3Q7f7eo~hLpe`Dl=jX{97;#5f>2=!PzPRQGy3r=+JWP?Cj}u&5#QG z)Ll9#65O^TMzO*lkG6y^g4ozYv(Vay4Z7x_?pm=1l@hN=jCIh0G_4U|R}V zquA~2?BZU(CJ1F0U2LL*BvUgpacGdd zyKjf9O*QFRGr8`~Zb~IVl%+!uuS(tBzn4S;kMQfZc#_6D2@gWix4d1-X$D=3+jE0u zmxbxoGoMh#pJ|iv#I~=tkr95DPUQFRw`_#z`*@TE!IAp;cAlIlXpN)YOgh?h4u6%GD~`;m;ByB66sWp7d0# zt@d7BVfsHSadqxBw;4HdoDrYdB=wn5kA72f9E97XTJn_a zD>8kuwDi#D6N6(Ff-iw+e}Df=M@P$Rp$A%8T9|L&zG3JS_MJo3ry&n!9w0e)Bl_ShInZ-nnsD9iRiVb~9Z+@-#mO4W2=2($+u*$>l$ z)rg0WPFXH16x$ve?k)|MeYp3Pl2Ou=IGA-Jzqr1s+GDZAVOSKxsAe_YajFXii<|B> zG;hJBHNkwi+cx$7tz%P(Wi!%jF$j+tdm&beQov!lHHJ>i(d5xMjWUF?z^Bq5aQ4Y0 zk_U2#?&;ApaqAHI^7CgnXH?b98~eC7Z|FrW2s{WiW{_MUnD zVC`VBxSd29nm9RJCG#eQ>U_5en`=_FtRUi`biy%)=MkI+pL62}PTL|WhCLs3BuUbG zjr)s17Jt6Q%P0JrcTW_sL|(d;h{;k*nEE*E*j#GejzpT<*)pEB0Bi*I0 zc*cjMpa_j*#b1^nVX%F1Nj9dY@rhDB5aYClMB;X(@ zUjzD$!Xy1MO`@--XK}DKpD5|Y7e>Mm_V_W*(pVkCPpM7fqMDUzjJ3U;oin(PvGuKE zzMk>HZ5#@v0)k;IE(qm}BFACtQf33k&cKbW!$QEKCr8_RGL+W&_ZNyP6F#6LmM2I1 z-g9Z;$XBM-?p&|QF^C=#-$O5>j z>SzWr(kC>$cAfz#GX85*Px!+MrKqI6`6tNN`XOfSO697Huz}oNt{i8Vk&)?YNyU%I zqnFF?jZ%I>!zLtjP9ll)!u4n@P~0EYW8j>Z*!*ITW|E|djEn^Kz^QKApsd|#uoO4r zMr@c_@eQC0Gv8l_bai!kCe&zNfh}Nq(Pds<65qOigG1sDL&CpjXYUS{S`XCv2wi{5 z7zqx&RmU3<-XgEPHOtz~ACjFm!c_+g`FBh_4|kRmrF^b}KWo2kuuNtpm^HoPy(18C zY7eBYPO@r4rzgK_Tbo`1FCw+!$4g7m; zK05`TU*a{lOAsqRKbf+!a@Fa{q2S$CV$S$)sf%VVE!3Rc+$acERQvuS2T#e}@jzKL zDPsiq&2lAvYX_|3i5h<$l~euxB^n;1&Y9B7^SuHsxt2w4lSazC=xu?I`yxOio{%eJ zsngT~dVJ-`hssq00%T@7-iDjjcv|kRj8+}*3_pAEA_SNXvD*e|S`++%`$XU!ZHxx}Ioc@1Kg4PX=-aYlh%A%uYScQ}2N77_JE7yD>;rDdWq1QmQ9B4!Z1 zmCO?5Ane!CIz0bH)KxbtFm8!3EFvQPD@TF}GEKg4a_(1#%Uma$x0E{n?V)N9ZrDbd z)%miqu^}4vJDjw%wS$I?=WCWvj~hF>9z83!w-u=s2Amr z*F6#&aTH4qJn`A@$_qT%Oi?%E=jJBz8uPilQoE_abGsC$xNe6YWkE|Nl8w>T(FyLu z>Khm!@y+b}lO3Q&`*Ymf+&`;hIMiNYABwiePEA?)FvIqUlb)KEcDnv>rTM4qX>^5s z|J0uZ^VYiEvEh540*((Fmp!N93%@dMtnUmv!H$c>=>D&C<-Og8(-@$Fey7K~drM_M z1-#cLoa?9w({czfis7U!K`6Wl&l=SG)xk16NZ{U=I8vwPQ~HzshqXy%4vP7`O*X=4 zsiT8T6+}sCA3<41?_?og zQ}$RmjQD&oSUW6RGNab%b&JWSP&z4ZflKOd9=rXxcEgcX!So61fg~!bq~opF^g@1d zE3jq);OzPw{Qml4y<1aua&LXQ!xPorxiVTK==xjb=y)}-?cFuEH{V%A?5G}!G6d{R z;_YvHu8c&2#ZSi@nl&bQjSVhby>sR!5z-1%tkEq$yI4IXenXO4!dBK;DC#2#!fuCFzd-JS2s$|)%LoGKqu z;<;=*;@+*kag{k15B$V%^D*qQRJs09Eim~+2_auGi8(TgIt)Dq?%(R$ORmnv?W&z2 z`*pD63i-XBIY<7wBm+L>l%fB8V7g(?JLF0fy&+q}&c3!(#XMBR)ORkhZa-LsHWoNj zOg5(GQZmxjeeFTit#zUhXb7}cPHOmPUSEs6u&emZ9hL`SHvyun$Ff?cmo8~AVUHlie!5h{Fk zg#?buo&cAbwvZ_Grkx(R(1lhUd2BdK0NuQm79Mz=f=9D1wrlhTjJ-K^V8_5+Tzhq{ zwz%Ph9-z3x)dr`2`xJoKUtHE)QCa6C1a4cQ*&gyYZDcGiZ(Lr#o(DWCRWoe1f65#Z z0~4V>Q@20!mQK(Lhp#HtL5#;)A_>AUOZ&R-{kTo#LySr5OhuPuYvUEFA`)(V{<=_& zPZI@#$li*6vnN|y?zO;1Bf1?#8OHo~O-%xiw_!g7!WEJ_v)DAnaI58^3Pk%;;OJqu zw^FkzlllI=Rnubv=uI**U?K=lzNtyWAraQ6Y9-D_3u?!zm!7 za?F?8B58N*5t&Bzxyn&b2EN)gg@+?m6tKMq5aj7#*#p=UA&Ot5q>#m{h8EW!5C$F& zj|V$)16vNz2F=ducr38q1h9nz;Y29L`YGeJU^3Ywat1NS&#f^GAN&tJ3yeM}BsT!) zEp@!r?Y;Hu#)rVufP&j)==0~#Lz$g#5)$x$0|Z8n(L7+EgBXnhw&G&Hs?KmOyD7#o zBab{nk_FR4{k^eUgp(&?cbw^*j$*j8=PSaZ-67Z}+p|BcTZpyhsB0^8rK}CLX=|g+nY<5eNajD?~xx z44ghVkR>vFbV?dL<@HPaq}dm zc~Z{L&kt*F;{Hqd!RQZg2x4i?{V2l8Si*YkLWh^be%TNkz1K{Uaos~%DtV&Lwmm$=`aGxh+os2m0+?U}(ckzEv+RvzIWZP?eXaBEP9n~JSU z90n(qFMmt~#&4zm&=Iyhfj~eo?pdF1n;v|S(n7>vi-F${oQtxbpA-$RanJ9UcGGeL z17JT~y4A%d*V|ayK1ZIjy!yJtaa2O;^k@^&p4+>CnGI&eF2>WVwzgCpB?C=|6%YoT z>N8ncSrt`$BC)Wb7Euu5mXG5_uatc}U4?0ikAx(q+#+41F964Y{7J+i!|DG5W&g_fN z!NHLz^netaAF2m_k4n;W2@5={a~qS_1USPE0mmoDhtt55!I3t6C>oCb@DG$&D!w&S z%#>1Hs2fii8-b+OTN#`n$nn!38elP|suzozflz~hBY!wv&jd~>zs5W9 zUo%%g&5@%a<~<;M>+By}*CQ8f`vB;Yco z1P30{zFXMse)I8}{`!^b#kHF8<1-U=J(@0vNz!uy7$b+xMfgK|z~9EgUS(~fF;Vbt zm|sk)S#jHJO5jt#5ZrnzN)m!c)(Qe~Z9~I0`bJXn&wU@cvM=$_5Z$_T?mO8fY>qxe zmV)k)fUq9i4Nb8vz1|*uxQW7<4<4(jDk&{nR}KRLC8FN{@QnTt?@{GF9{s)QRCevY z;zfH%QQiS!MZ4oCwb#Qz8K)}j1)l3=jg_!Pj8IlZpD}*vL?{c+i!&;Up13fjWC}uY z&Wqd6!W!+M^Jky>2Kxab5e#*OMxsNJg+1^z6v}{2;=H(zvD|Sygqe=!J z0x+O#WyStd)jIPD6`+$8<k0LN8aSzrstd{s^2AgkeN-9dKN_^es^A z>|8!SR0Vj9eC`o0=HL->)6*HO`YkIyOiy)ncTc*0urx2%&F?MPmn1@CwQeqOlZ0&J z8W|92Q~7+ufIvp~N<1-54{r&DO3uYYq{Sa?A05w9WchQnkOP+=B?@FUwMsl>iFg>laqoW8)}57P5+nPyeE{wmJ; z`udkIU$6l00+#D(XWcVa)5fiJT&!#Uxfuz=xsS?VHa*w1siAE3w_!&|emr+R5(D(x z`sLXLLARM(z;(ZG*#_ga#KkXnxxCEuI@@NMT;>ty4Wg2LnOZ`EvPu1>ayAXqDg=c;&OKD0jpW3~6D5YTsAT z`2DpI0`l?k;k3DZM_Po%2xm-Q8>28eJ71M z*wX_!4wS&`Vt2MCSz-VnAJa24l}KGW3LpKrHv46g4AQ$o4IE4;U=Gm09pgZpRD|Fs z@z#6HHsGPD2yxvdOl`)ZO;FxE7Osx;f0gdblQ&S6*QyyYvlQJGeT%H$fL8py&s(@? z&NomEm;%eJfEmFu-Q)Rx`bv_RGs<90IU*hvZzt zH356vIi1oo|swTr209B2E0Az%tz|MxiQ=0ZUXYmPV@!zvF zPdN61BL^WItUgiCD@2$(BsZ=%M)DYF96LXVz7g}UY?oj6zw$@{SVwAi#Lau@LtBZ* zqJbZPZ{mPQ5CDPF5ePILT;M1<{Oi5vEPz#Rq}8)U=sMZfmf9K$gj!dSGxQL{iJ4H{ z9vtk;moHN*{Nm%|A@cID;=)B{ac4*^fv*Kb*iUb;_)`I9wYfN8o7?Yby-nrTt$OWWjWm#UoeoZH0r}t!$83eScStY~E ztqZ;Rf_6Qe_wL_EVx)jie1JV54M&X!o=U?RjJ!(Samp%fLjho_yl~FaPwDuPZ?3a~ zXa$ZEVcrFBn=rTsJXH%&*WliK!*+k@2cRqf1uyRU8y^4{2y&bK(E<*TAc!gfnnupg z&l^?PQ3sstKX^D;syx;!idN>5I(fnU)%;L;Ar6S(@RS!L(n8mAnt{6+t+q)DQ0h} zWXyFc5PRU_;=)G^FviRS4@INt1d*^!0G$4xKS1kcj<HY2{~p z)IL-?tpIKNi^IMDxD1`F3qZtl@Sr#dm4yp2hRpGovp?I%rRvI41x?3){PHpe3~faN zhE}s1B3B~Efg?4rE0)mG!>@mR!mm(7NtsAL^{qSvr1j-TDh!CegM`@~H`B%vewcEO zA_ptKl!BB&d)ICJHc3XpL*T3(?QK?uU}z5{c};kcanAhzkKtr zF0TPk7JmN_dGCx^|NB2Xh#?tCqbxWAyG(ffbfh=^Gc8Tyx7NO@_9?PK2H_0WB2WLV zo)o;xhD6SGdQn}J_e+s86}to;WcR{#gKcnLF*>gQY`4w^6avm}vlpT< z9GY-`)MR=E7NR_Oe5x2x{u1J=k-Ek!{{*Ly(2RmJuS7ZLOb%+`4azA<8%iib&f}TX zZE_qlJlN@5g5~iALl<|SwQ@6v;XErsNj1GXn<*NM*fYXAsdFrX`I}ML2V-zu^EStv z&|@?HNa4aRF9`%V@7~xGyj!VZP|2lE=M1X>c=IfC-JJWH9xKDOPL6}M^2$i>J0J8{ zmn|XU%Yh^o9uyOWtDBnH2I+=YjL_fNoQ-y?4iQSfCXQ(Tin+bvfjQ%5BP7lPIdi>~ zv-&7JR>c-k%&|sshS(PFfoZ-PF;OHBBw^J=z(St|1VZH*WI>mtH1K0x|2}8}9_%xD z&{=N}#RAE>KWhFfIX3wEEkgl-rl&+yy=&Q@8V<2*3BW?72DdJOyatNi{eTA_dGP+j z8CLUj4K>%ZMool2}tAth6Vr&dSLV96AB9pTe`B;1OahlQ(>H$nF%Ky zKy*p?XV^B`k7VN4rLy0P(BK(kCJ}k13)jYGsN8lBaHAirS_p|5#mWC+naOQmnB5nv zNp7CRg3wSKI&M)G+Yj;%bEHL(Ar{#)qN+5ipeu zHuWIN8ql4-Xy-oU8*d*2M?0x{alNM({uICo?(`oNA_(X1_v(9F-I{+o@&50^Cl3C@ zFBf7bKrFilsLZ&xZ@D#3dy4~lBUg(k{I8}8Aw0`^*xXxvHh4~XC!fA7J+qGzHvqRd zR$_?)@iE{Xm<5gj*9r3eH{~LTKCWDweRlz*EdcG5C}K}jzdyt94kR=ni?TsD1$q}` zOc0a&?9YR}a4&eFJ75-&ec>X=jDQr&lg1D{#Vo>tmyDth6QvrmfIr8*<27FI4`Ej8 z4@oO3D(D0(AdtbqfP4{1#>0#VOz$gh8yH_6q?LO+PdMWpO2QW_R%;E0O;02jL6oGY zBfozAs{Z~k8{{_Ky-fqvCjjdlBmHP+g5`7%PZT6%GJZvH>As&BKRxk!V(l)@SYpJ3 zi1oC8FEqz|ooT8^wW^Liio4leogLWAmX`o3K%l&#W~Rc^wAe6DyW2(>&ZWU@C7k#L zRG8AwWa;(K=QPW!V*a!zG~c`WHMXFjfF{978(1Sws^Ae8);mb}^gW1p10_CxPFen{ z82Dl+xNoh(V-zra5Eew@adC01{Mr4Bf`ICS*?rzuE?=K>2)yms$f&B?<5|ta1-2Pk zdjXD!Jc_`q{ztl>rhl8izpJ{j=-k#hWeP&&_L~2Y{zDz+C>uVsRM6^eNBpM08-W8h zB3mPg5ab|XIB($#a$r}5gaGKtq3M_QUbB!>v#cDB0kGvRNbZ7I1P9t$8axNWj#fi+ ze`@FBWPnF@mWN$Hdg$?~U|jFz zihdYujszSo@BMzExds@zFMmPNm3Trx;cisJfC!XAgoCu%U62bLD77XEI9@cL1}sG@ z*i5F9q}k+0q#`0BiQ+EoGDmBQFmO#2wx#^m-u}~<#6pFON=}Exwl?o|k%bWoHswjI zAo~Lqm^Uy3ny(c;@3vhRmxd~cU}VNyIl|6Az3x4+?#&vNE~TE|o1U5qAFV3z8UgEX zRB+J($+`Oxf$CKjPQ?+QxpFs**mSE=*;_$czDpK+><*^>t~s1LSA4t0fI4L zLeVEhCfq@mogG|-XoSFX$}$8#CZDF{75e+P?Ft;3e;&+=81pQ%s}Dd2PXP;qqGAqb zfVV(8Jdx-+JZs=lGOgcTCQ;kPyR{Ne_m3`*%&W$yiU#Jqnj5@OP1cY;d5}Tib{hHb z6UgU`I3VNzH#YmBf5C++J17_TJLGqgQ zq;dWT80bX^K-W=<667gpxre@L;(sG~^1DzpO~*U*3>U1%gd%9H4VxH_Z4ba}dMG0! zZK7C1Vy|4-xiNrEp>Z`72tt$?>M#ml!F}h*uVK)uoN56k(lZJMC)%f4laVVZp4Yts zqtsD~$7gPl6Ne7UP#$81kQVt+K^CISo9Rpflqiex&bgqM0bSl1-PrA2ocm{z@n8hg zxAms43L4Q1G!*!YJY2v!FW!w1zGo-^zbz7scr&aRUGkO7A{Sj!-Wo(-!~GYoIffww z-t1{25}0wM9DPc;G-C8ZXdo@4AI4`^nGP)Qc@M4DA|h6?n%^1q7Zs)fwnUwZkk!9L zcx+DCS@tg&VFOnkehg72o`GozFz^uAn<{gOzc9lFESVutZ9?b~>e8V-MAU9Sxg8bL z55&lB2g~tAK(a94;TT0WevjRB(c;DxA2EAZQ1nD=5*DYV1r#m1&V@+}7dG?NN-og3 z68B=t9>3{~Gs!`L1!|XN2OaYU`J73sf(1zB+gcKu&tZi8F>ern(K*xLp=YNO*w$Sf z4eP&VNF2PkXN{`FXnd+aP}8FC3^U#U6V2#xgX9SY@`K*!ZrnQ^mD9Mhl;ctL_V#Od zE`T)rrBaB}IZ&G>S;2pv)6dFbhLT$#g9C>gY48^P>q3=})uugQVA=zU4}awWM13L| z%sBb?k_Nb>>PWy?i~gr;j{xT31{iPfC#H=<^Pi361e1$TBmoe#m?%e_M{}nVhmWT? zw4YnFQWVzmGVrE&R%kv1aBtZ`WzV;@xNv$tNr^jrxN(MId*Z>UWzp~Z5qa^1_p;{@ zC1-f5f((d8Qcca`@2)KaqOPMWWEXcJ_OF;=S-x%%vT+l%5Ch@O{vG6|BYb#zNWrXm z_y`1cp11z&NkP9_MeBS!a~XCuIfNJhUqGRY0-#;oC757}*<-P6P zC=!iJNr|;?e`UEj(?NI5_2T^>?;=I)`^u~S1FJyA8Vsvg{I@7TrHTNsb;i)Q$wKS?J7(--Aa+E+Z;2x zfjdu#hFo>0*N@lj77q@9ebxVro32{;WOqk? za+|dRiS??v^d}03*A>)daQz_i#nq!PuGXDOm1S6>t&nwK0Y+YMSPf*w1>o5{|KYzq4p|b-ttfUU%K=R~6BE=Pp)uWEz zL6T1B{&y@$2o0Om{}&t*ZMa^8V@t5KN?cu1Kw}l6+YE2ZgL5JX0g!C5;Wf9m=74Yt zC+!!jL8^!SDYJAG(1Nt;YWm*ZURU1aQO6q|9wJiAnq|YcOYaeUzh8F5$;an-%&-)} z>V@-F<7v1K8z7Dv&p){!^@kNm$&u~9iKC$7q2>RSMEt`@AQ2im_^=gZi{L~nfSANA zGK`oWj$1l9I-ULN-F6iM^f8FolDIp_elOe%ZLYYa0zB*@oPo%30C|0lBnfJ85?K7! zm6~2tK|H(+S&$Wi&q47YEA}+!stIEpdBHkIKR^vk;A@ZxuOLO9rlm8P` zHqSF2HXtJp1b8Y^ha@xULX4O=${>GIwziuA2&m~r(3#H z7(d4p2gEdIEnNatVrR>VIi+%LvBc`rx<4b%Aa(k2KJtMRV;G=ZWT_0N7aIJ6`ekGZ zV-rkU04d)hfyNeHO2Z>hz=<$RU7Ed5dkzQD&Ht?_aHR%mNPH`#d$~pTE$Kvm;30iz ztvyf}?OVE_^1SDIFm#U?@zkF!3H^id*VL4>y33aakBei+AL%p${Gf4aeaj8Cl)LMn z)N3tZQSRs}Ir9oajnlA+CtOtpf%L6gnnBvxD=)8-w_F9Y=R7vy0&urK zuLV~gt4{l1x)hP?JrRkT<1dPsxu||3ELB2C1v*R!+^`Thc|9| zFsbG(i(3~df`9RNILbjdDh zyJr{#ICkc-;{V#s_`FE1?dUO)=@j@vTNFX61ocA;(HTOdeEQ+Zs3>td9X;oce=SW)Zlagur4Y_ge z(Rs|-cBO@Lk-omf`XFVw_1|uAQs^f1T+n<$_{*pf8ZHw}VY!jWLtsyNytZ(}4Bx zT1q!2jkn9rK20^c5`QE#{`0NEGnOhQ( zIrEif+VF2K<-a(%35==v<`aUs(V0`F2-~?A=}umB`6Uu1r!T2?b`p$Aia$p;Z48yw zJH_Y(>4l(e35GDvoa)_dy{Ync!NRr$`CXW0j%>3`bkK=ebX10FcaooL< z)=XOoFlGQH{_3W}v!;{DA7!3j^`48S4p*~ZYd_CT2;lmLcbX2N7FXs^maL}TPnqPY z#8*2S3WU`$s=0Et*CTO~=v&39Da*UoA4cc1&+dZVB4=oWjN$~dMIl(`%4wV&W%t>xlIe)<5*O@-Id4|#SW~KXC)qK0 zX98^nshHi8qY;x%V*a|UlXwSnRdny9qwd67U{xM9VUB(mp3}5(u5mU)_IFS5(FdzE zDuq;?&6$etfv!gCIB4FtVtl?u4`B?c*bc@jw@&=KmA-TiHcKDg#EA3s0{)iLY#UPwqrGdmpt$5E;}Q7Nq4CKHstf z3w;)!nGkGN%6z*X+b_vQDp7HD&et)PHcNz*i~K;r`DO)EHm{C~jDyT06)K$6Z0+zR zD(QpF&le<>W8eMu=32^{yW?TEs}xTHbfk?oynVfQ(p7I5spznpDgMdR`Bnqf#e=86 z!j!a?T&$UP;uBHGaoZ=-bE3K=#_tkrmB3Sa!zP^hE~z66a<|oopX$m7E62oX<;lpoxbaFw?&#uO-b&+~~ z=LdVifFHRBU#{p{-sV;3<&s+_f&$dW`fL1QMW6qyr~mj9G&&yyj&)M|1=-!${%g`D z?bBxCoxHr4!>`%ecs-i5*{f~ze$exV0BO;gld4DP{K${3d0P3#NiQKX?tbTiLbSUC zzmT>>U4OS&p-}O$-P4ZwBC=2s(wD|doP`9$kq>xw;_xoYW@v`N4|FT+#<^S zdhkbGP3Z%lTJ{1@F`jxBgBl4H{9&bhQ; z^zL!Y|Iz(~r^?fTt^`%x;+5GHv-eeFH?PJ3>n?H!UDJLcywNpZmo$94dtvtW+rjq= zaeSd#+|-?%)EiZ&cyA2?_&PSD*fk8)vX}21Smx;A&&AWt58K78Tr8Eo0}eGJaT%Xx%a+AC;3&XuH}@EG}!0ls`5a)w}JYeqHnu- z`xRa$X6YuJydem52<@1+-3ciyv(+b@>rtKSA^dFbm%ZxVHBTC{>TaeG((QvDFE*Yx zbUJpcGd?dI;ZwVv9W#=AK+j=FG%XR|WN4p7^mW_%nUzPCw*KPd;nS?C!_swcru6?Zpk6Do5bgs+n@`qXOJ}U**;jmadv!9 z5%P#$QiK)goZtcaod7y_I4A>J8AqmCRP5wr}=-W<3oL?Uq1NWb3C2 z7UPaWXEF!~TtFOmlYlLu-C+s#Ot?Y%I0j`E!+)`F0Pg3CJzl8wbP7krAxae4eWAh{ zT=6Q3#41KQf5UKd4JzfM$|`z>K&jB!-Io*5Eo?v4y7C2WRme;vdCn#Le(!#;ec3ai z&G0R&>08esub81+J9FNU!LdC1xAZ#r7_*FEjY6e&*Mr0>C|&HbRzC^zk@-(cj30$r zYUDo;lc=DSh^I8JKuYKzOBEU)Z2uto9NDxAyrDVcm9~WNZ!n5@$4sCzuxP7v>aLh@ zGU+A?ynT5qTWC|1F_nJLlu>9;>4A|CuN|Y~MH~IO6biZx^u3F$-I-55{p3N*p|PZq zbBL`$UF_^nX!a+v=+61BgdJbyCBD3-))dte+n>3ZKHpq4$tAM+Nc-^Z&tGJ|G97r> zxK+DU^(@rvWTG41+~G5c>Z*tGEDMfI$mczS|~XL6!c`Ge2jPO3H>-)wN&e+a;AqT)L=&<3o(~My zLkPw*9b%Q!f<6*reJ^!o!4Vc89G!H}Ws&1U3@i7%U=9>+<{EA6{fSuco|rS3Sk9F* z^~h9%XQUmCD3P3V<63(g3ndY7j^X|`2Ng0ms?XB0S>WYQ3Z7x#sq3_LX756QDa47I z*Z%n#e1$|881?n{Rtgup=FEu6$bljNWcnwZ0911~Sr? zMq^hv1@Gh$+>)g&Nq(h)mvAK3b(T8^MRNnD9QPnM8T3_tXyHoGkvT$HlKpSq6$uvb zTIT%T@+sn40)muiZAW-w?UhObC{>SO2WzP5z66zx;T>o~BOcf{Z!yQGwDW)s$E8%B zul^&wijpmOR%E*4d%^Tdx}zGo@|fJmy8i~&nNPZ}R-QAGWf?e4b6>1lG(cRq53hyl zreii$b>34%ZyThJjFZ7vVZzUumhc*8rgc8JoVv#_v8Q2cqzmdaFCpeUWI7iWQu~#= zItR+!+LZ;<)cgazDD^XmAhCLaqZGuRrO4W35~iEbL`t@cBg5uGl$-g-1DbHQ!#)@%9 zIg8>~M>MFc&Qlm)OLRJ-v$%$ckhdGe!Hw%NJYWF27}~Vu7WDE2qcd9XI&dZETusna z5_a7b`*k(5j1n=dkCYBa*hy<`gp|h;$|f)ew3^8k6J^Q`9I~g*;>+3Fn7@Xwi_cav z9A&tc$u@6dT`2`{kANcK z4=&T!LB$QsJ!#79Uj$Wm{QP%67_Duz^G}0ZR}09IT2>hUs&q2{{zildt`r6(Xm7s_ za|FtJ*I^qHpc=FI8DwRqbW4io@-Y zxLG3K_)wh-C9iU}b;varF2onQ2W~EM>dAeB#>efPkH-sK;%Jpqry|@Rq+6LUo869E z$#IXA>*Hiyv|>G^WZm?WJICL#>6_`HvGl5G3T#EG2x&nuQiXvhv}vr~xT(BRCvN97 zQIJ*)-}7>2_@7MS!uL39)py2j@GeU#qT@QtpLGFYs`O~gTL9Z%eQ~bWCaQxi>nu|_<=Gh&?$`z zx7GnVc>>f*w@I$nhJb3tPlIr|BVY9#yPl!p-Ige76-~`HP>;0^3fu3tzv88n^yGmo zR>$gKRtpedf{r8Nbi`;-Y)A`_$XB;r7ZZUU+*{NHl)7(p$oDjb0M~>XZWZo6l5BOiO zp;W#jp1UrDALz+#sSIx%ggcJ!(5t^lsX?X;UkNHj$qoM0NhG{(Pe@F6V(F*4xRcGP z%GPfiU;mp?q~68;SU8*7x%X!Kr-vl1io8cr#I4+;uZMrW;S~@S``aMZQh7iE1G0;`0M+ z!c|DCd{W0foLYSQg0pKSKV1NS?~us*KD7$>SGwx<=R+u7>O=@_BX8AZ8DH0R;uyrLTtD4!7RXq$=-1{L z65f5KcS3~NZ)c&5)--KnIhJRBtEP?XnBk-T9&>zG!}?QlDiWmSyREfw#R|SXsX_|6 zUAY>uLFtEe ze$dm1fB;?F98MPAXn`#h4>g!(RSr8XJ#1`nqCB3i zPh-pYGo+^dQlUYXc<4$d{k?0f2Q2#A(?5*o6A3)`y8xeJPD)SG4p-JhW5hQ%EF2~np*sa?li zwygenTcK&1>oM7)i8GJJUd35esx7^=3!`mK zrXLmcskOyj8WStR6Rx*i*t@=YH0{#0X(N?IOJEkOmv7=LCt;o6qiR_^yS5ha6aExS5!BbobHE<7tOnecgHbp2QYh89kTnwt^22EUJWd zo(DuL1#^nxSCH8AF00!MjCyQ~ygQ-S3?;1E_~7YOf5$J-OA9pl4c2(^qsf>V$xk2) z(3rAuG=rP~HPtTre-c2l#~iM*0Yw9F(;VasQdt%GOQu6#e7a~S!z6>GPhT>tlEeur zRWoPz<8}#bzw0!8w^E|o&=Y%DL?-c}=D7b|?sUlG&s6ho-VV8%V>f%}20I=EO2^0_ zIN$pIN`_TreVc?1C##rEF=YLI_I|(iUU1^|gR6RW48+HcMcMt--d&ca zcQp#yNXbgIlwmt|5bnT{JQemZdAQo{ti_piNAuDwGNdTd-8Qp~Za#$FG|Kl%jJvdj z!u7>|3gujyVHUz=n)z{Mxi6bZ%=#@J9YSKk6OS~F=QrOp@!26Ql$CK^-Q3>Z(u6yW zE6a-lKYc7Zgqu)bKS*u_iHSeh0mkrVM0iI1(5Mwbcj!iv!$5IX=hn7))}7W~eFVq2 zTI{HkN_)<>T&(4GkiaH#9@QYiA~kbvdsK>Q6-Q&$!8c0tZXDmlj}24&6&Dr64+Cx4 z@_1CmiRGs)qCtHky+eB8^6Mr#{#do|J24M>W7{|c1(di88~d5Qgg)aV(4M7ye86(` z=Cx;hBp$k;QU*~z08KxP5gIC60R`~Rpjd_-G#rDD$b;MMm45rKQYU-U4@`^H7G%wQ zBhuaT3V%Q1B2GTyhf#mO&AP_?WVQ(6b@L+1a%R7aD^%#wV)NL#X(NA^sP)b4$WnjW z$Jr_UAL*}azqBJkDOiQ9SKH;@7hql5Che%u0ub5M6}_T<>scyfE!_j#+hkq^Rx!wM)j+Ez49@madEi<P(Kn7WSiDTm}S~M?8o9W{PtpZgp)8IR1;3hzz&y{)6Ame0Q_))*zlbbNK z$MDY3HLSCFN;GqVJFmglz(sqapGz-{>+NFXR-e@m)oFKe)jW0Y0i8-pY}6u!7%``@ zXz*@w^v7+m`D3p=6ZZe^)<&TC=(F23hzU zSpeL_0VeHL^LVY#CY<%YaN{ZKNDSg2V`{8|` z{u3|KT2y{}R8818ZjvhPJY12dO$^FIpoIR=-9R6f;*-5ASC-=;%xUsIzu8Gj!m}K; z=!a)jB*W8O69SaV6e%?RQMoMYEh(PI1o1nnzvF4@it^>|NjKeg<8e|y0EgKSp| z2mHBP&PX3h^y_*cy%Df#vbocI+EXibX&~H+ViXHLqY; z@soD1gFw>?AH{kH5gy&X*mPjtcegh-|M#o9=B}{GUD^|7+IuFQ3;o*My?fo(feK{) zWVIi5NeYfEIW+o{TQ4L?A>)Gs`QVUV{8AM8fTtgWndBBd5FP?5Jy-f+u1UG->ftxf z-VrVwTpH6V{PXMQjq2^M$%t3U#FaCB*zZvGqI3 zuhsWPt{rKNh_7AGpTHkHf!9pdk#Lbd5)T`g(kAeqEmd^z3P`O#=j&i`ue%)_B<+Xgg}x`0zbeD0Y$z*3tkHIUZVMk zZp)Rki-X6yqE*8sMi{K5uf6Wevt8vjHFn-g&mt7L{JTTZX{@i1Hxu%G42CvBCCk~_ z75LnazTlmc8@wWAMvihyd;AQJt&-buDyLJ-k;AdolqzM>Uu>|`@bzc5n*Lw#&~nB~e&ajHgmWI8)3ZwwQ5=R0c< zrQt=TsU8PqU=_bBIxpPBQ@>Jadwi{-97{BI45#$4<%mIf-L#5wEis~(lP1ndGgVBB z5QMHs*Wbg#x(zz8lLYUx&QE*3jZ=?0UPYOtci6NTJ%wLC$TRVZumc>RFfS>o_jL z!o(?;$3G**Q_K+|y}=`PJT)U46HD*fJra15!ID~dT;KEtmOq;-A^xr5Zf`;O>}EmX z`T`NvS6Sf|u3qI4eO~EnH2Y^y-m%y$Yl6w_q6fE)f3eh;H?Ptz9mA)*p z@^Idd}kYRBJ2FDfqOTG+gLn2;Bsi|S`OoAcHf>2=?V?SrHAX|O^%T}P9IO5 z-jw2^mEj+~W8%B5Vk>XU^jb_?#WWdbQQkd%B~i+I?V>y8BxWwfSX>uQa?P)i&oyVA zlT+`?QKb-;9(MC{L4(7mgI>}uMNd8r|HejRx)u7{)68~d(cxcJDw7vK-1gPh2^ScT0rWf~=%-K;Z)kEj0xo^Jq2T`fph~DXV z@3bA>X<6S%(M5yP$+;ItqdU9|a_Y)n$P6s6M!h$-lDFOObIiC`)>JhJ@7eUlttUVs zXPxyV$zej*hlHPz;^k^Le}HQo&9F#4uvHCh97mD`Orr(Q zmOJrsZ;sy^z5ixGw6xMb|E<;V?M>}4Qi;ZHsu1)-O5P$*-xY(%w`d~eyvV9BbBXUF zYujv_l~4JZFMGB_3kC;XYMT8~qev;eO?u?~m~=bujNsH3Vu)DpR94PtqqAW-=p1un zdLP;rwwN>rT&`Vq&Xu(|NKArU$En!uv88}$au^)#mWL7(K!WCqufd$zZp;RUNs?6Jz0i*0!FqFZJtQ{q}rw^BJ*-zxE& zS>0O#kAO#0dG`sPbRsqCyZb|n3vmToi~Hj>?cL-8g2fl&B&SFHRL*>OXusS>i(_hL zQ;pihKg)xf4d`F_0iQyDk>UidA?@=aE>zN+rm>2q$?BbYso{oxySRx418pPaenio& z4Fw`wSd1AzHic$lFD-+S~S|O{z#xg(J?Me&Or+ zEtLL9;!&D7??!$_eHl(#?s|WE^W6TUyS=CELZ_|yT^H1;JS()K$N28&w4zPe@4tWI z`d~(CEm!cI=xX<}57UJR;yx>^m9JydMp)4amL;S1Ke28*V)+_m9xrF^%?Q(Y%uUatmYWTX5@LO)a z>~UUso&AF~yUZd(pAcEmFZ1}Qt+S54Va`#k(ZJ!=CY!iq^S$47w6e5@r&en};D^HT z@1$w*f1MV%NdW3XKd8xlTULxw5%56`ct@1kc$GOllmhdXUX)xk=0b_e+Qd)dexj6i~u*2L3zIn zVCdkf)bsK2k$caH7U#VQCI8Yo_t9Si3FWX@>`%iAputf|N!+NOaH!@SpToa)(7n2J z5$!Ro^!{9OW88p*LMq0JQyn|nZyd6@8P;cK?+&32jj(kM5HWNH+TX)RM{|jI09%C$ zk}*x6f;@pH2SeKc%{QZd4m2!DZ{OnAfhq!m2@{T789@G zvAyrE-TV-7uMS*p;r-rc2d`HnU16^a_0?nJU|U;TX?69Q7t0e9Cqu%_X1{0n`nWJE z9(g&vWs`RC9azxROKj;LTSAT2C5OmwUH+(`4k)qhP`p^}_lxUeaoJRpFfjDd9^z2% zsX%4%Pe6&S3;-qF1$qj5P7QHDD<#ZQT}F+kw9X5NJTFDO@&cBq!L1o8n$ z-)TJneFbhVfU>c`Brzd{=2O&PKqEDeFSfpfU9yl4WBnk6dWFX>AguxRazT-luI|j2 zb3m5`LrlJ_kxahpQYVZIMkmS%$v*|A`_KJ>11aJ-2oG9cykL@6ef1EidiVPd6#(=V z78iT=mDqjlZ`_RjFI-Oz1sf2(1R}17vt@N?*NyTx0QA&d_#AVh)Gsh(kOJk5Xe<0# zu7rstd}X%Z6J&f2puI%kA94ypoD+;a7z&-4LXV7jWcN=gqpf7jen>H#d0WK+Y}V5> zc^iKSK_e^?fJHw}^3V3OgVNU^)pb_nCS6(hq18r2b;&cMtU7eEYH-^0VBFi}Y0 zF4G()TK@`!bdJ`FTG72#mLf}p{2OB=*L*&Xgz+4n?|GGP`RuGs+`#3Jr~#A{{u)ND zTLIie77P46wQi&76AxVtC_k!x{p-zMGKl#v$OOXz`u{)pqXS!^hk^0yq=TtP#-&~< zzo>{UQDon9JA0Stswx%G578wr{xVgG(EqafB~yq2?Ut4G01y<^xZc5B?~JN~8ejc> zaPQQjmUm~~f{13@<<-H{NUMsHeqg~|p6TO4`XVs?)yckGa`4M}GjAhV?DqxW{E;l* zGCeT-j*X3_+_;ehLo2%Ok6%yj^Guh}voT(uwjk>h2!y685X~7HTkCTD#eo0 z(yQs|(clp+0QD9X@L7~c1Dn`5*aCtlt{%la`+8qNFtF*(Jtrf~7>JxeryAf$D7Rk@ zZ77gS@NjheeXFi>^%=+14^XDAR8&-yaO#2M&7ipfWh)3tLA&$Xlj@S*%=f?FT&?kd z989KJziQ5*UyEK;fpb?`M#ex~C#)WqOefCLf^wHMc#^Gd9f=1+#nKlC?GquKnQ~9p zi_hc7g6dFm^__n+4^vHK+lv=^1cD^+pPhp(bLa67T<`dSTPOS2K~ZiEh<{l*0|ZSc zI?+z$T2FzaN&ogW^RVaN@3&!$ci#6O)u-AjfOzoO4GjVCn1SM__n!En^DKdZfud?> z98yD?V|bcJn`42Nl(DRw&j;oOj7;IycMKsiM*{cMt;)&+5C|1)KXvdiWAow-DF25> zjnN;!JVPI^`UH9`@Jxb0x-4NrAp}>OnIW(mmXf~m;qRG@hR87 zJOCWgWW>#85M?GhEUe@l2h$*KeWCe3kBW0a!>=3^z+6ZPhr#SfW+Q)l>52qcm=V+_ z66ZI+eH@3dWJWHu$A?VKKn3J`ZAlb4h>3A9D^>}qeQ-H!t?oT5E*J7SuXGrf-a*6? z!Jk>}#i11K?{bKssb61uiAG=M$wddh+{smWO3!Kt7=w|`GatlejPL!8{;2t?tA)v+ zd7JR^SaI;e6sDMgLj*bd5QYk!aOSq~7I}Gj%nbRynysTFhaP`QW~S(*b)YL~K9IMp z?f~%#a*-e-sm0+HEYh8YXj1xCeb45q#nPt-y-uH&(b5vb_^TMZoZD5W?|gcf`Q`xx3paUef}%$Rc@hP8qk< zD&v4|s&BXxL-S{TR2{;Z51oa$HrVPw_nQ38(~f_`PSx;}KS1i}OhsG*1gvdrkjxc1 z+mMY!4Wi-5z}RgxCy1WIB>^ft#FS)eU)DqjE^~^x6VUVGxL|&WRzE{7ZI}ry=PRl} zg8t)z!&~#yL*=kiKp8zG8ObLzHeo*qYdd=4zQ=K3HgyN*U>t=)S*VMX1S|!&xe%-4 z)?H|#7nz_Fx8UH{@Mf;3pkNmajRBCdhgH`YC`peTIl=~P@8Ta9KaV1LAJPS#z8S)% znyclR+d7Etpe!5tiVNn7kL8I9o0@y r?0)iwFfS8+^2q\n", @@ -12,8 +14,10 @@ }, { "cell_type": "markdown", - "id": "8da83506", - "metadata": {}, + "id": "3896f306", + "metadata": { + "editable": true + }, "source": [ "# Week 44: Decision Trees, Ensemble methods and Random Forests\n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", @@ -25,8 +29,10 @@ }, { "cell_type": "markdown", - "id": "0d38a800", - "metadata": {}, + "id": "7f793430", + "metadata": { + "editable": true + }, "source": [ "## Overview of week 44\n", "\n", @@ -38,6 +44,8 @@ "\n", "* Friday: Decision trees and ensemble models (bagging and random forests)\n", "\n", + " * [Video of lecture](https://youtu.be/9QcU8VcXxRU)\n", + "\n", "**Videos.**\n", "\n", "1. [Video on Decision trees](https://www.youtube.com/watch?v=RmajweUFKvM&ab_channel=Simplilearn)\n", @@ -49,8 +57,10 @@ }, { "cell_type": "markdown", - "id": "d31105a9", - "metadata": {}, + "id": "8f731670", + "metadata": { + "editable": true + }, "source": [ "## Digression First\n", "\n", @@ -65,8 +75,10 @@ }, { "cell_type": "markdown", - "id": "07fe25d8", - "metadata": {}, + "id": "5204298c", + "metadata": { + "editable": true + }, "source": [ "## Decision trees, overarching aims\n", "\n", @@ -94,8 +106,10 @@ }, { "cell_type": "markdown", - "id": "2d6686ff", - "metadata": {}, + "id": "13462217", + "metadata": { + "editable": true + }, "source": [ "## Basics of a tree\n", "\n", @@ -112,8 +126,10 @@ }, { "cell_type": "markdown", - "id": "ce62a1a1", - "metadata": {}, + "id": "1dcb8496", + "metadata": { + "editable": true + }, "source": [ "## A Sketch of a Tree, Regression problem\n", "\n", @@ -124,8 +140,10 @@ }, { "cell_type": "markdown", - "id": "cf8c3236", - "metadata": {}, + "id": "65cc5600", + "metadata": { + "editable": true + }, "source": [ "## A Sketch of a Tree, Classification problem\n", "\n", @@ -135,8 +153,10 @@ }, { "cell_type": "markdown", - "id": "d450c9c5", - "metadata": {}, + "id": "503138f3", + "metadata": { + "editable": true + }, "source": [ "## A typical Decision Tree with its pertinent Jargon, Classification Problem\n", "\n", @@ -151,8 +171,10 @@ }, { "cell_type": "markdown", - "id": "6065bf9d", - "metadata": {}, + "id": "931cadfd", + "metadata": { + "editable": true + }, "source": [ "## General Features\n", "\n", @@ -172,8 +194,10 @@ }, { "cell_type": "markdown", - "id": "dab43c19", - "metadata": {}, + "id": "80af3452", + "metadata": { + "editable": true + }, "source": [ "## How do we set it up?\n", "\n", @@ -193,8 +217,10 @@ }, { "cell_type": "markdown", - "id": "f35f5834", - "metadata": {}, + "id": "99d353f2", + "metadata": { + "editable": true + }, "source": [ "## Decision trees and Regression" ] @@ -202,8 +228,11 @@ { "cell_type": "code", "execution_count": 1, - "id": "6caa26f7", - "metadata": {}, + "id": "c42147eb", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -300,8 +329,10 @@ }, { "cell_type": "markdown", - "id": "9eb4d7cf", - "metadata": {}, + "id": "5c54b1f4", + "metadata": { + "editable": true + }, "source": [ "## Building a tree, regression\n", "\n", @@ -320,8 +351,10 @@ }, { "cell_type": "markdown", - "id": "53c984bd", - "metadata": {}, + "id": "a7fe1577", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sum_{j=1}^J\\sum_{i\\in R_j}(y_i-\\overline{y}_{R_j})^2,\n", @@ -330,8 +363,10 @@ }, { "cell_type": "markdown", - "id": "a9906c4c", - "metadata": {}, + "id": "76468ec2", + "metadata": { + "editable": true + }, "source": [ "where $\\overline{y}_{R_j}$ is the mean response for the training observations \n", "within box $j$." @@ -339,8 +374,10 @@ }, { "cell_type": "markdown", - "id": "96557142", - "metadata": {}, + "id": "908a9534", + "metadata": { + "editable": true + }, "source": [ "## A top-down approach, recursive binary splitting\n", "\n", @@ -359,8 +396,10 @@ }, { "cell_type": "markdown", - "id": "f30bcd98", - "metadata": {}, + "id": "b8082569", + "metadata": { + "editable": true + }, "source": [ "## Making a tree\n", "\n", @@ -370,8 +409,10 @@ }, { "cell_type": "markdown", - "id": "ea89bbb5", - "metadata": {}, + "id": "c03ba845", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\left\\{X\\vert x_j < s\\right\\},\n", @@ -380,16 +421,20 @@ }, { "cell_type": "markdown", - "id": "1516b6cc", - "metadata": {}, + "id": "923b79d6", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "893e24a3", - "metadata": {}, + "id": "ddb1c706", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\left\\{X\\vert x_j \\geq s\\right\\},\n", @@ -398,16 +443,20 @@ }, { "cell_type": "markdown", - "id": "c4b356f7", - "metadata": {}, + "id": "974a1a13", + "metadata": { + "editable": true + }, "source": [ "so that we obtain the lowest MSE, that is" ] }, { "cell_type": "markdown", - "id": "be0d9aca", - "metadata": {}, + "id": "0879e55b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sum_{i:x_i\\in R_j}(y_i-\\overline{y}_{R_1})^2+\\sum_{i:x_i\\in R_2}(y_i-\\overline{y}_{R_2})^2,\n", @@ -416,8 +465,10 @@ }, { "cell_type": "markdown", - "id": "d4f3358d", - "metadata": {}, + "id": "8d20dc23", + "metadata": { + "editable": true + }, "source": [ "which we want to minimize by considering all predictors\n", "$x_1,x_2,\\dots,x_p$. We consider also all possible values of $s$ for\n", @@ -447,8 +498,10 @@ }, { "cell_type": "markdown", - "id": "7a339714", - "metadata": {}, + "id": "b8e00a18", + "metadata": { + "editable": true + }, "source": [ "## Pruning the tree\n", "\n", @@ -469,8 +522,10 @@ }, { "cell_type": "markdown", - "id": "33f066ed", - "metadata": {}, + "id": "de4b7a52", + "metadata": { + "editable": true + }, "source": [ "## Cost complexity pruning\n", "\n", @@ -479,8 +534,10 @@ }, { "cell_type": "markdown", - "id": "753ee3d1", - "metadata": {}, + "id": "9e950ab1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sum_{m=1}^{\\overline{T}}\\sum_{i:x_i\\in R_m}(y_i-\\overline{y}_{R_m})^2+\\alpha\\overline{T},\n", @@ -489,8 +546,10 @@ }, { "cell_type": "markdown", - "id": "55e0c324", - "metadata": {}, + "id": "5eae9abd", + "metadata": { + "editable": true + }, "source": [ "is as small as possible. Here $\\overline{T}$ is \n", "the number of terminal nodes of the tree $T$ , $R_m$ is the\n", @@ -515,8 +574,10 @@ }, { "cell_type": "markdown", - "id": "813adcf0", - "metadata": {}, + "id": "abe1c0ed", + "metadata": { + "editable": true + }, "source": [ "## Schematic Regression Procedure\n", "\n", @@ -539,8 +600,10 @@ }, { "cell_type": "markdown", - "id": "c48b2ef5", - "metadata": {}, + "id": "93731a82", + "metadata": { + "editable": true + }, "source": [ "## A Classification Tree\n", "\n", @@ -560,8 +623,10 @@ }, { "cell_type": "markdown", - "id": "fddbe38b", - "metadata": {}, + "id": "f86f2095", + "metadata": { + "editable": true + }, "source": [ "## Growing a classification tree\n", "\n", @@ -585,8 +650,10 @@ }, { "cell_type": "markdown", - "id": "a3fba89d", - "metadata": {}, + "id": "6e26c38e", + "metadata": { + "editable": true + }, "source": [ "## Classification tree, how to split nodes\n", "\n", @@ -602,8 +669,10 @@ }, { "cell_type": "markdown", - "id": "22166b43", - "metadata": {}, + "id": "72e74778", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p_{mk} = \\frac{1}{N_m}\\sum_{x_i\\in R_m}I(y_i=k).\n", @@ -612,8 +681,10 @@ }, { "cell_type": "markdown", - "id": "c2e4074c", - "metadata": {}, + "id": "ce861605", + "metadata": { + "editable": true + }, "source": [ "We let $p_{mk}$ represent the majority class of observations in region\n", "$m$. The three most common ways of splitting a node are given by\n", @@ -623,8 +694,10 @@ }, { "cell_type": "markdown", - "id": "b8e1e02c", - "metadata": {}, + "id": "470fddfc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p_{mk} = \\frac{1}{N_m}\\sum_{x_i\\in R_m}I(y_i\\ne k) = 1-p_{mk}.\n", @@ -633,16 +706,20 @@ }, { "cell_type": "markdown", - "id": "25506a4a", - "metadata": {}, + "id": "7bf4f22b", + "metadata": { + "editable": true + }, "source": [ "* Gini index $g$" ] }, { "cell_type": "markdown", - "id": "8e859f7d", - "metadata": {}, + "id": "d09f93cf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g = \\sum_{k=1}^K p_{mk}(1-p_{mk}).\n", @@ -651,16 +728,20 @@ }, { "cell_type": "markdown", - "id": "75118af9", - "metadata": {}, + "id": "47a9f98d", + "metadata": { + "editable": true + }, "source": [ "* Information entropy or just entropy $s$" ] }, { "cell_type": "markdown", - "id": "7ad7a4a9", - "metadata": {}, + "id": "6f55e4f8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "s = -\\sum_{k=1}^K p_{mk}\\log{p_{mk}}.\n", @@ -669,8 +750,10 @@ }, { "cell_type": "markdown", - "id": "80392de1", - "metadata": {}, + "id": "b24a20a2", + "metadata": { + "editable": true + }, "source": [ "## Visualizing the Tree, Classification" ] @@ -678,8 +761,11 @@ { "cell_type": "code", "execution_count": 2, - "id": "448f2c77", - "metadata": {}, + "id": "bc028f62", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import os\n", @@ -719,8 +805,10 @@ }, { "cell_type": "markdown", - "id": "976d6f54", - "metadata": {}, + "id": "55cb02a7", + "metadata": { + "editable": true + }, "source": [ "## Visualizing the Tree, The Moons" ] @@ -728,8 +816,11 @@ { "cell_type": "code", "execution_count": 3, - "id": "245a63bb", - "metadata": {}, + "id": "43c8786b", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Common imports\n", @@ -760,8 +851,10 @@ }, { "cell_type": "markdown", - "id": "96f4a124", - "metadata": {}, + "id": "685cb93a", + "metadata": { + "editable": true + }, "source": [ "## Other ways of visualizing the trees\n", "\n", @@ -771,8 +864,11 @@ { "cell_type": "code", "execution_count": 4, - "id": "89e96fb7", - "metadata": {}, + "id": "f86f0669", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_iris\n", @@ -786,8 +882,10 @@ }, { "cell_type": "markdown", - "id": "3f85a09f", - "metadata": {}, + "id": "80b1c9ce", + "metadata": { + "editable": true + }, "source": [ "## Printing out as text\n", "\n", @@ -798,8 +896,11 @@ { "cell_type": "code", "execution_count": 5, - "id": "b6c15b05", - "metadata": {}, + "id": "82a35973", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_iris\n", @@ -814,8 +915,10 @@ }, { "cell_type": "markdown", - "id": "64b8d1dd", - "metadata": {}, + "id": "285c0f97", + "metadata": { + "editable": true + }, "source": [ "## Algorithms for Setting up Decision Trees\n", "\n", @@ -832,8 +935,10 @@ }, { "cell_type": "markdown", - "id": "696574e0", - "metadata": {}, + "id": "2769cab5", + "metadata": { + "editable": true + }, "source": [ "## The CART algorithm for Classification\n", "\n", @@ -847,8 +952,10 @@ }, { "cell_type": "markdown", - "id": "69cfbe7f", - "metadata": {}, + "id": "67932aa2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(k,t_k) = \\frac{m_{\\mathrm{left}}}{m}G_{\\mathrm{left}}+ \\frac{m_{\\mathrm{right}}}{m}G_{\\mathrm{right}},\n", @@ -857,8 +964,10 @@ }, { "cell_type": "markdown", - "id": "68b5217a", - "metadata": {}, + "id": "06167e51", + "metadata": { + "editable": true + }, "source": [ "where $G_{\\mathrm{left/right}}$ measures the impurity of the left/right subset and $m_{\\mathrm{left/right}}$\n", " is the number of instances in the left/right subset\n", @@ -872,8 +981,10 @@ }, { "cell_type": "markdown", - "id": "d7eeda5d", - "metadata": {}, + "id": "20620ae7", + "metadata": { + "editable": true + }, "source": [ "## The CART algorithm for Regression\n", "\n", @@ -883,8 +994,10 @@ }, { "cell_type": "markdown", - "id": "f0978df9", - "metadata": {}, + "id": "b434cc65", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(k,t_k) = \\frac{m_{\\mathrm{left}}}{m}\\mathrm{MSE}_{\\mathrm{left}}+ \\frac{m_{\\mathrm{right}}}{m}\\mathrm{MSE}_{\\mathrm{right}}.\n", @@ -893,16 +1006,20 @@ }, { "cell_type": "markdown", - "id": "40902e6c", - "metadata": {}, + "id": "2b09a9ef", + "metadata": { + "editable": true + }, "source": [ "Here the MSE for a specific node is defined as" ] }, { "cell_type": "markdown", - "id": "091dc8c0", - "metadata": {}, + "id": "bb805fe4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{MSE}_{\\mathrm{node}}=\\frac{1}{m_\\mathrm{node}}\\sum_{i\\in \\mathrm{node}}(\\overline{y}_{\\mathrm{node}}-y_i)^2,\n", @@ -911,16 +1028,20 @@ }, { "cell_type": "markdown", - "id": "49903d3e", - "metadata": {}, + "id": "31ba0b95", + "metadata": { + "editable": true + }, "source": [ "with" ] }, { "cell_type": "markdown", - "id": "b4bff5f6", - "metadata": {}, + "id": "457cdfae", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\overline{y}_{\\mathrm{node}}=\\frac{1}{m_\\mathrm{node}}\\sum_{i\\in \\mathrm{node}}y_i,\n", @@ -929,8 +1050,10 @@ }, { "cell_type": "markdown", - "id": "cd13ac16", - "metadata": {}, + "id": "6fee2e9b", + "metadata": { + "editable": true + }, "source": [ "the mean value of all observations in a specific node.\n", "\n", @@ -940,8 +1063,10 @@ }, { "cell_type": "markdown", - "id": "ec454ba0", - "metadata": {}, + "id": "ffb1412b", + "metadata": { + "editable": true + }, "source": [ "## Why binary splits?\n", "\n", @@ -953,8 +1078,10 @@ }, { "cell_type": "markdown", - "id": "e1419e8a", - "metadata": {}, + "id": "d1ac8567", + "metadata": { + "editable": true + }, "source": [ "## Computing a Tree using the Gini Index\n", "\n", @@ -977,8 +1104,10 @@ }, { "cell_type": "markdown", - "id": "acf7278e", - "metadata": {}, + "id": "982a3f12", + "metadata": { + "editable": true + }, "source": [ "## The Table\n", "\n", @@ -1003,8 +1132,10 @@ }, { "cell_type": "markdown", - "id": "cf09301a", - "metadata": {}, + "id": "ebcecdc0", + "metadata": { + "editable": true + }, "source": [ "## Computing the various Gini Indices\n", "\n", @@ -1018,8 +1149,10 @@ }, { "cell_type": "markdown", - "id": "ac1071fd", - "metadata": {}, + "id": "14ef43ec", + "metadata": { + "editable": true + }, "source": [ "## Computing the various Gini Indices, Hours slept\n", "\n", @@ -1030,8 +1163,10 @@ }, { "cell_type": "markdown", - "id": "ae964258", - "metadata": {}, + "id": "37ea3246", + "metadata": { + "editable": true + }, "source": [ "## Computing the various Gini Indices, Hours studied\n", "\n", @@ -1044,8 +1179,10 @@ }, { "cell_type": "markdown", - "id": "c50f3fa1", - "metadata": {}, + "id": "4b5d78ee", + "metadata": { + "editable": true + }, "source": [ "## A possible code using Scikit-Learn" ] @@ -1053,8 +1190,11 @@ { "cell_type": "code", "execution_count": 6, - "id": "7550c1fe", - "metadata": {}, + "id": "da857ad8", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Common imports\n", @@ -1121,8 +1261,10 @@ }, { "cell_type": "markdown", - "id": "73371791", - "metadata": {}, + "id": "149b6737", + "metadata": { + "editable": true + }, "source": [ "## Further example: Computing the Gini index\n", "\n", @@ -1163,8 +1305,10 @@ }, { "cell_type": "markdown", - "id": "07b54f10", - "metadata": {}, + "id": "9f2c2b49", + "metadata": { + "editable": true + }, "source": [ "## Simple Python Code to read in Data and perform Classification" ] @@ -1172,8 +1316,11 @@ { "cell_type": "code", "execution_count": 7, - "id": "727954e5", - "metadata": {}, + "id": "cf2c645b", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Common imports\n", @@ -1247,8 +1394,10 @@ }, { "cell_type": "markdown", - "id": "ca1867f9", - "metadata": {}, + "id": "cf269868", + "metadata": { + "editable": true + }, "source": [ "## Computing the Gini Factor\n", "\n", @@ -1263,8 +1412,11 @@ { "cell_type": "code", "execution_count": 8, - "id": "17d5a20a", - "metadata": {}, + "id": "e01adb33", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Split a dataset based on an attribute and an attribute value\n", @@ -1331,8 +1483,10 @@ }, { "cell_type": "markdown", - "id": "8400e1fe", - "metadata": {}, + "id": "a1de546e", + "metadata": { + "editable": true + }, "source": [ "## Regression trees" ] @@ -1340,8 +1494,11 @@ { "cell_type": "code", "execution_count": 9, - "id": "8f030fe1", - "metadata": {}, + "id": "fcdcb884", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Quadratic training set + noise\n", @@ -1355,8 +1512,11 @@ { "cell_type": "code", "execution_count": 10, - "id": "7de07d7a", - "metadata": {}, + "id": "759e0cf1", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.tree import DecisionTreeRegressor\n", @@ -1367,8 +1527,10 @@ }, { "cell_type": "markdown", - "id": "1d114a7d", - "metadata": {}, + "id": "3ecc42b9", + "metadata": { + "editable": true + }, "source": [ "## Final regressor code" ] @@ -1376,8 +1538,11 @@ { "cell_type": "code", "execution_count": 11, - "id": "0cf732d7", - "metadata": {}, + "id": "60a03548", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.tree import DecisionTreeRegressor\n", @@ -1423,8 +1588,11 @@ { "cell_type": "code", "execution_count": 12, - "id": "7713257a", - "metadata": {}, + "id": "a783779b", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "tree_reg1 = DecisionTreeRegressor(random_state=42)\n", @@ -1459,8 +1627,10 @@ }, { "cell_type": "markdown", - "id": "3e0b8e66", - "metadata": {}, + "id": "2745b639", + "metadata": { + "editable": true + }, "source": [ "## Pros and cons of trees, pros\n", "\n", @@ -1481,8 +1651,10 @@ }, { "cell_type": "markdown", - "id": "b73d29d8", - "metadata": {}, + "id": "18106586", + "metadata": { + "editable": true + }, "source": [ "## Disadvantages\n", "\n", @@ -1507,8 +1679,10 @@ }, { "cell_type": "markdown", - "id": "86083d18", - "metadata": {}, + "id": "4193991b", + "metadata": { + "editable": true + }, "source": [ "## Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods\n", "\n", @@ -1536,8 +1710,10 @@ }, { "cell_type": "markdown", - "id": "66d972ae", - "metadata": {}, + "id": "86e2b66a", + "metadata": { + "editable": true + }, "source": [ "## An Overview of Ensemble Methods\n", "\n", @@ -1550,8 +1726,10 @@ }, { "cell_type": "markdown", - "id": "ca6bf352", - "metadata": {}, + "id": "6480a2d8", + "metadata": { + "editable": true + }, "source": [ "## Why Voting?\n", "\n", @@ -1572,8 +1750,10 @@ }, { "cell_type": "markdown", - "id": "94d89cf7", - "metadata": {}, + "id": "8f8e6b9b", + "metadata": { + "editable": true + }, "source": [ "## Tossing coins\n", "\n", @@ -1601,8 +1781,10 @@ }, { "cell_type": "markdown", - "id": "ec65ac92", - "metadata": {}, + "id": "be6951af", + "metadata": { + "editable": true + }, "source": [ "## Standard imports first" ] @@ -1610,8 +1792,11 @@ { "cell_type": "code", "execution_count": 13, - "id": "6904dbea", - "metadata": {}, + "id": "f8aedad8", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Common imports\n", @@ -1655,8 +1840,10 @@ }, { "cell_type": "markdown", - "id": "5ea8eea1", - "metadata": {}, + "id": "3232d714", + "metadata": { + "editable": true + }, "source": [ "## Simple Voting Example, head or tail" ] @@ -1664,8 +1851,11 @@ { "cell_type": "code", "execution_count": 14, - "id": "ed3edd5f", - "metadata": {}, + "id": "7d33cecb", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -1695,8 +1885,10 @@ }, { "cell_type": "markdown", - "id": "913397e8", - "metadata": {}, + "id": "59f9c81a", + "metadata": { + "editable": true + }, "source": [ "## Using the Voting Classifier\n", "\n", @@ -1706,8 +1898,11 @@ { "cell_type": "code", "execution_count": 15, - "id": "d06e933d", - "metadata": {}, + "id": "08866444", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", @@ -1756,8 +1951,10 @@ }, { "cell_type": "markdown", - "id": "6f392050", - "metadata": {}, + "id": "291eda26", + "metadata": { + "editable": true + }, "source": [ "## Voting and Bagging" ] @@ -1765,8 +1962,11 @@ { "cell_type": "code", "execution_count": 16, - "id": "6f49c5ce", - "metadata": {}, + "id": "a220ad51", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", @@ -1792,8 +1992,11 @@ { "cell_type": "code", "execution_count": 17, - "id": "fafb0ab3", - "metadata": {}, + "id": "475e945d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.metrics import accuracy_score\n", @@ -1807,8 +2010,11 @@ { "cell_type": "code", "execution_count": 18, - "id": "58bfd587", - "metadata": {}, + "id": "a0443168", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "log_clf = LogisticRegression(random_state=42)\n", @@ -1824,8 +2030,11 @@ { "cell_type": "code", "execution_count": 19, - "id": "a5e2060a", - "metadata": {}, + "id": "e96466b9", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.metrics import accuracy_score\n", @@ -1838,8 +2047,10 @@ }, { "cell_type": "markdown", - "id": "bc47caa6", - "metadata": {}, + "id": "95d87928", + "metadata": { + "editable": true + }, "source": [ "## Bagging\n", "\n", @@ -1858,8 +2069,10 @@ }, { "cell_type": "markdown", - "id": "7a7ce88c", - "metadata": {}, + "id": "09eac3f8", + "metadata": { + "editable": true + }, "source": [ "## More bagging\n", "\n", @@ -1888,8 +2101,10 @@ }, { "cell_type": "markdown", - "id": "3a2bfa3b", - "metadata": {}, + "id": "97b706bf", + "metadata": { + "editable": true + }, "source": [ "## Making your own Bootstrap: Changing the Level of the Decision Tree\n", "\n", @@ -1899,173 +2114,13 @@ }, { "cell_type": "code", - "execution_count": 42, - "id": "f5f20a2c", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Polynomial degree: 1\n", - "Error: 0.1077869092797025\n", - "Bias^2: 0.09665219931376287\n", - "Var: 0.01113470996593965\n", - "0.1077869092797025 >= 0.09665219931376287 + 0.01113470996593965 = 0.10778690927970253\n", - "Polynomial degree: 2\n", - "Error: 0.07073328352158705\n", - "Bias^2: 0.05660265596808508\n", - "Var: 0.01413062755350194\n", - "0.07073328352158705 >= 0.05660265596808508 + 0.01413062755350194 = 0.07073328352158702\n", - "Polynomial degree: 3\n", - "Error: 0.03491402051096134\n", - "Bias^2: 0.025645913680458925\n", - "Var: 0.009268106830502446\n", - "0.03491402051096134 >= 0.025645913680458925 + 0.009268106830502446 = 0.034914020510961374\n", - "Polynomial degree: 4\n", - "Error: 0.03122805721680159\n", - "Bias^2: 0.020459888416421317\n", - "Var: 0.01076816880038027\n", - "0.03122805721680159 >= 0.020459888416421317 + 0.01076816880038027 = 0.031228057216801587\n", - "Polynomial degree: 5\n", - "Error: 0.029360432630668315\n", - "Bias^2: 0.01933417827965857\n", - "Var: 0.010026254351009753\n", - "0.029360432630668315 >= 0.01933417827965857 + 0.010026254351009753 = 0.029360432630668322\n", - "Polynomial degree: 6\n", - "Error: 0.030277866384743817\n", - "Bias^2: 0.020595346780014352\n", - "Var: 0.00968251960472946\n", - "0.030277866384743817 >= 0.020595346780014352 + 0.00968251960472946 = 0.030277866384743814\n", - "Polynomial degree: 7\n", - "Error: 0.030892885889348087\n", - "Bias^2: 0.02014767854486778\n", - "Var: 0.010745207344480309\n", - "0.030892885889348087 >= 0.02014767854486778 + 0.010745207344480309 = 0.03089288588934809\n", - "Polynomial degree: 8\n", - "Error: 0.03110939965212513\n", - "Bias^2: 0.020817454714933955\n", - "Var: 0.010291944937191167\n", - "0.03110939965212513 >= 0.020817454714933955 + 0.010291944937191167 = 0.03110939965212512\n", - "Polynomial degree: 9\n", - "Error: 0.03129578341811742\n", - "Bias^2: 0.020886306078419455\n", - "Var: 0.01040947733969797\n", - "0.03129578341811742 >= 0.020886306078419455 + 0.01040947733969797 = 0.03129578341811742\n", - "Polynomial degree: 10\n", - "Error: 0.031606699705781\n", - "Bias^2: 0.021032809061206647\n", - "Var: 0.010573890644574355\n", - "0.031606699705781 >= 0.021032809061206647 + 0.010573890644574355 = 0.031606699705781005\n", - "Polynomial degree: 11\n", - "Error: 0.03148677989704399\n", - "Bias^2: 0.021152336559874207\n", - "Var: 0.010334443337169787\n", - "0.03148677989704399 >= 0.021152336559874207 + 0.010334443337169787 = 0.031486779897043994\n", - "Polynomial degree: 12\n", - "Error: 0.032116346271443774\n", - "Bias^2: 0.021860481703725315\n", - "Var: 0.010255864567718462\n", - "0.032116346271443774 >= 0.021860481703725315 + 0.010255864567718462 = 0.032116346271443774\n", - "Polynomial degree: 13\n", - "Error: 0.03209100097235015\n", - "Bias^2: 0.02177267731801543\n", - "Var: 0.01031832365433474\n", - "0.03209100097235015 >= 0.02177267731801543 + 0.01031832365433474 = 0.03209100097235017\n", - "Polynomial degree: 14\n", - "Error: 0.031537537813106746\n", - "Bias^2: 0.02082938849956035\n", - "Var: 0.010708149313546408\n", - "0.031537537813106746 >= 0.02082938849956035 + 0.010708149313546408 = 0.03153753781310675\n", - "Polynomial degree: 15\n", - "Error: 0.03145124683797013\n", - "Bias^2: 0.021111527370668133\n", - "Var: 0.010339719467301988\n", - "0.03145124683797013 >= 0.021111527370668133 + 0.010339719467301988 = 0.03145124683797012\n", - "Polynomial degree: 16\n", - "Error: 0.03209105344712722\n", - "Bias^2: 0.021338345379319522\n", - "Var: 0.010752708067807702\n", - "0.03209105344712722 >= 0.021338345379319522 + 0.010752708067807702 = 0.032091053447127225\n", - "Polynomial degree: 17\n", - "Error: 0.03196914233431665\n", - "Bias^2: 0.02166629218762961\n", - "Var: 0.010302850146687045\n", - "0.03196914233431665 >= 0.02166629218762961 + 0.010302850146687045 = 0.03196914233431666\n", - "Polynomial degree: 18\n", - "Error: 0.03165248155309402\n", - "Bias^2: 0.02132799291304245\n", - "Var: 0.010324488640051558\n", - "0.03165248155309402 >= 0.02132799291304245 + 0.010324488640051558 = 0.03165248155309401\n", - "Polynomial degree: 19\n", - "Error: 0.03235209768543048\n", - "Bias^2: 0.022074472952423024\n", - "Var: 0.010277624733007463\n", - "0.03235209768543048 >= 0.022074472952423024 + 0.010277624733007463 = 0.032352097685430486\n", - "Polynomial degree: 20\n", - "Error: 0.03174154546980151\n", - "Bias^2: 0.021139865993692355\n", - "Var: 0.010601679476109165\n", - "0.03174154546980151 >= 0.021139865993692355 + 0.010601679476109165 = 0.03174154546980152\n", - "Polynomial degree: 21\n", - "Error: 0.03249366272788966\n", - "Bias^2: 0.022392964161879643\n", - "Var: 0.010100698566010037\n", - "0.03249366272788966 >= 0.022392964161879643 + 0.010100698566010037 = 0.03249366272788968\n", - "Polynomial degree: 22\n", - "Error: 0.03100387126675639\n", - "Bias^2: 0.02122260145527001\n", - "Var: 0.009781269811486361\n", - "0.03100387126675639 >= 0.02122260145527001 + 0.009781269811486361 = 0.03100387126675637\n", - "Polynomial degree: 23\n", - "Error: 0.0320452032176362\n", - "Bias^2: 0.021849700000696155\n", - "Var: 0.010195503216940061\n", - "0.0320452032176362 >= 0.021849700000696155 + 0.010195503216940061 = 0.032045203217636216\n", - "Polynomial degree: 24\n", - "Error: 0.0315431807559056\n", - "Bias^2: 0.021311765078499363\n", - "Var: 0.01023141567740624\n", - "0.0315431807559056 >= 0.021311765078499363 + 0.01023141567740624 = 0.031543180755905606\n", - "Polynomial degree: 25\n", - "Error: 0.03167739763326645\n", - "Bias^2: 0.020906889239215253\n", - "Var: 0.010770508394051193\n", - "0.03167739763326645 >= 0.020906889239215253 + 0.010770508394051193 = 0.03167739763326645\n", - "Polynomial degree: 26\n", - "Error: 0.03091217303915449\n", - "Bias^2: 0.021114234118115666\n", - "Var: 0.009797938921038828\n", - "0.03091217303915449 >= 0.021114234118115666 + 0.009797938921038828 = 0.030912173039154493\n", - "Polynomial degree: 27\n", - "Error: 0.03181795943603612\n", - "Bias^2: 0.0217612448640803\n", - "Var: 0.01005671457195582\n", - "0.03181795943603612 >= 0.0217612448640803 + 0.01005671457195582 = 0.03181795943603612\n", - "Polynomial degree: 28\n", - "Error: 0.031719847034916156\n", - "Bias^2: 0.021338073424295095\n", - "Var: 0.01038177361062107\n", - "0.031719847034916156 >= 0.021338073424295095 + 0.01038177361062107 = 0.03171984703491616\n", - "Polynomial degree: 29\n", - "Error: 0.03138781693308297\n", - "Bias^2: 0.021135473862578174\n", - "Var: 0.010252343070504813\n", - "0.03138781693308297 >= 0.021135473862578174 + 0.010252343070504813 = 0.031387816933082985\n", - "Simple tree: 0.6586458006727302\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj0AAAGeCAYAAAB/8bDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABedklEQVR4nO3deXwTdcI/8M/kTpomvWihpS1HOeXyxOK60EUEVFbBZRHEXQQvFB9Zn4UVFdDVfcBr1fUBfR6PtY8IeIH+RNCFCniwoKy63AoFSqFI76Rpmnt+f0wyTdq0NKVt2ubz5jWvmcyR+SYdkk++8/3OCKIoiiAiIiLq5hTRLgARERFRR2DoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMYeoiIiCgmqKJdgPbm8/lQUlKC+Ph4CIIQ7eIQERFRC4iiiJqaGqSnp0OhaJs6mm4fekpKSpCZmRntYhAREVErFBcXo3fv3m3yXN0+9MTHxwOQ3jSTyRTl0hAREVFLWK1WZGZmyt/jbaHbh57AKS2TycTQQ0RE1MW0ZdMUNmQmIiKimMDQQ0RERDGBoYeIiIhiQrdv00NERLFDFEV4PB54vd5oF4XOQ6lUQqVSdejlZBh6iIioW3C5XDh79izsdnu0i0ItZDAY0KtXL2g0mg7ZH0MPERF1eT6fDydOnIBSqUR6ejo0Gg0vSNuJiaIIl8uFsrIynDhxAgMGDGizCxA2h6GHiIi6PJfLBZ/Ph8zMTBgMhmgXh1pAr9dDrVajqKgILpcLOp2u3ffJhsxERNRtdERtAbWdjv578eggIiKimMDQQ0RERDGBoYeIiIhiAkMPERFRFM2ZMweCIOCee+5ptOzee++FIAiYM2cOAKC0tBR33303srKyoNVq0bNnT0ycOBH//Oc/5W369OkDQRAaDStXruyol9RpsfcWERFRlGVmZmL9+vV4/vnnodfrAQAOhwPr1q1DVlaWvN7NN98Mt9uN/Px89OvXD+fOnUNBQQEqKytDnu/Pf/4z7rzzzpB5bXm38q6KoYeIiLodURRR547OVZn1amXE1wi65JJLcPz4cWzYsAG33norAGDDhg3IzMxEv379AADV1dX46quvsGPHDowdOxYAkJ2djSuuuKLR88XHx6Nnz54X+Eq6n5gJPTUON0ymaJeCiIg6Qp3bi6HLPovKvg/9eSIMmsi/Xm+//Xb8/e9/l0PPG2+8gblz52LHjh0AAKPRCKPRiA8//BBXXnkltFptWxY7JsRMm55TFbwsORERdV633XYbvvrqK5w8eRJFRUX4+uuvMXv2bHm5SqXCm2++ifz8fCQkJOCqq67Cww8/jH379jV6rj/96U9ySAoMgfAUy2KmpudkRS1yo10IIiLqEHq1Eof+PDFq+26NlJQUXH/99cjPz4coirj++uuRkpISss7NN9+M66+/Hl9++SX++c9/4tNPP8XTTz+N1157TW7sDACLFi0KeQwAGRkZrSpXdxIzoaeINT1ERDFDEIRWnWKKtrlz52LBggUAgFWrVoVdR6fTYcKECZgwYQKWLVuGO+64A8uXLw8JOSkpKcjJyemIIncpMXN662RFbbSLQERE1KxJkybB5XLB5XJh4sSW1VQNHToUtbX8jmuJrheDW4k1PURE1NkplUocPnxYng5WUVGB6dOnY+7cuRgxYgTi4+Oxd+9ePP3007jxxhtD1q2pqcHPP/8cMs9gMMAU4z16Yij01EIUxYi7ERIREXWkpoKJ0WjE6NGj8fzzz6OwsBButxuZmZm488478fDDD4esu2zZMixbtixk3t13341XXnml3crdFQiiKIrRLkR7slqtMJvNyFz4Lr574tdIMbKLHxFRd+NwOHDixAn07dsXOp0u2sWhFmru7xb4/rZYLG1WQxUzbXoA4EQ5z3kSERHFqtgKPWUMPURERLEqpkLPcdb0EBERxayYCj0nym3RLgIRERFFSYyFHtb0EBERxaqYCj0nK+zw+rp1ZzUiIiJqQsyEHrVSAZfHh5LqumgXhYiIiKIgZkJPZpIeAE9xERERxaqYCT3ZyQYADD1ERNR5jBs3DgsXLmxyeZ8+ffDCCy90WHm6u5i5DUWf5DjsPFHL0ENERF3Gt99+i7i4uGgXo9uIuKbHZrNh4cKFSE9Ph06nw6hRo7B+/frzbnf69GksXLgQY8eORUJCAgRBwJtvvtnk+tu2bUNubi4MBgNSUlIwZ84clJaWRlpcWXYSa3qIiKhr6dGjBwwGQ7SL0W1EHHqmTZuG/Px8LF++HFu2bMHll1+OmTNnYu3atc1ud+zYMbz99tvQaDS47rrrml13586dmDx5MtLS0vDRRx/hxRdfxLZt2zB+/Hg4nc5IiwxAqukBGHqIiKhz8Xg8WLBgARISEpCcnIxHH30UgdtiNjy99de//hXDhw9HXFwcMjMzce+998Jmq78GXVFREaZMmYLExETExcXhoosuwubNmzv6JXVaEZ3e2rx5M7Zu3Yq1a9di5syZAIC8vDwUFRVh0aJFmDFjBpRKZdhtf/nLX6KsrAwAsHfvXqxbt67J/SxatAgDBw7E+++/D5VKKmLfvn1x1VVX4Y033sD8+fMjKTYAIDtFSsqnq+xwerzQqsKXk4iIugFRBNz26OxbbQAEocWr5+fnY968edizZw/27t2Lu+66C9nZ2bjzzjsbratQKPC3v/0Nffr0wYkTJ3Dvvfdi8eLFWL16NQDgvvvug8vlwhdffIG4uDgcOnQIRqOxzV5aVxdR6Nm4cSOMRiOmT58eMv/222/HrFmzsGfPHowZMybstgpFyyqVzpw5g2+//RYrVqyQAw8AjBkzBgMHDsTGjRtbFXpSjFoYtSrYnB4UV9qRkxof8XMQEVEX4bYD/5UenX0/XAJoWt4OJzMzE88//zwEQcCgQYOwf/9+PP/882FDT3Cj5759++KJJ57A/Pnz5dBz6tQp3HzzzRg+fDgAoF+/fhf2WrqZiE5vHThwAEOGDAkJIwAwYsQIefmFCjxH4Dkb7qe1+xAEAX1TpIPwOG88SkREncSVV14JIahmKDc3F0ePHoXX62207vbt2zFhwgRkZGQgPj4ev/vd71BRUYHaWul77T/+4z/w5JNP4qqrrsLy5cuxb9++DnsdXUFENT0VFRVhU2NSUpK8/EIFniPwnA33c759OJ3OkHY/VqtVnu6bEof9Zyxs10NE1N2pDVKNS7T23Q6Kiopw3XXX4Z577sETTzyBpKQkfPXVV5g3bx7cbjcA4I477sDEiRPxySef4B//+AdWrFiB5557Dvfff3+7lKmribjLutDMecrmlrXVfs63jxUrVuDxxx8PuyxQ08PQQ0TUzQlCRKeYomn37t2NHg8YMKBRG9m9e/fC4/Hgueeek5uMvPvuu42eLzMzE/fccw/uueceLFmyBK+++ipDj19Ep7eSk5PD1rRUVlYCCF87E6nk5GQA4WuNKisrz7uPJUuWwGKxyENxcbG8rF8P/+kthh4iIuokiouL8eCDD+LHH3/EunXr8NJLL+GBBx5otF7//v3h8Xjw0ksv4fjx43jrrbfwyiuvhKyzcOFCfPbZZzhx4gS+++47fP755xgyZEhHvZROL6LQM3z4cBw+fBgejydk/v79+wEAw4YNu+ACBZ4j8JwN93O+fWi1WphMppAhgN3WiYios/nd736Huro6XHHFFbjvvvtw//3346677mq03qhRo/DXv/4VTz31FIYNG4a3334bK1asCFnH6/Xivvvuw5AhQzBp0iQMGjRIbuRMgCAGLgbQAlu2bMF1112H9evXY8aMGfL8yZMnY9++fTh16lSTXdaD7d27F5dffjn+/ve/Y86cOY2Wjx49Gna7HT/88IP8fLt370Zubi5efvll3HPPPS0tMqxWK8xmMywWC0S1HiMf/wcAYP9j1yJep27x8xARUeflcDhw4sQJ9O3bFzqdLtrFoRZq7u8W/P0dXIFxISJq0zN58mRMmDAB8+fPh9VqRU5ODtatW4dPP/0Ua9askQPKvHnzkJ+fj8LCQmRnZ8vbv//++wCA48ePA5DCT+D6Ab/5zW/k9Z566ilMmDAB06dPx7333ovS0lI89NBDGDZsGG6//fZWv1izXo0UowblNheKKuwYlmFu9XMRERFR1xJxQ+YNGzbgkUcewbJly1BZWYnBgwdj3bp1uOWWW+R1vF4vvF4vGlYiNby+z6pVq7Bq1SoACFl33Lhx2Lx5M5YtW4YpU6bAYDDghhtuwDPPPAOtVhtpkUP0TYlDuc2F4+W1DD1EREQxJKLTW11Rw+qxxe//G+/uPY0/XDMQD1wzINrFIyKiNsDTW11TR5/eivjeW11d3xTpdNqJctt51iQiIqLuJAZDD3twERERxaKYCz3B1+rp5mf2iIiIKEjshB5/wMlKMkAQgBqHBxW1rigXioiIiDpK7IQeexUAQKdWIiNBD4CnuIiIiGJJ7ISe6pPypNyuh3dbJyIiihmxE3qqiuTJQOjhPbiIiKg7EgQBH374YbSL0enETuipbhx6TjL0EBFRlEyZMgXXXHNN2GX//Oc/IQgCvvvuu1Y999mzZzF58uQLKV63FDuhJ0xND9v0EBFRtMybNw+ff/45ioqKGi174403MGrUKFxyySURPafLJXXQ6dmz5wXfwaA7ip3QE1TT0y9wgcKKWvh87LZOREQd74YbbkBqairefPPNkPl2ux3vvPMObrrpJsycORO9e/eGwWDA8OHDsW7dupB1x40bhwULFuDBBx9ESkoKJkyYAKDx6a0//elPGDhwIAwGA/r164elS5fC7XbLyx977DGMGjUKb731Fvr06QOz2YxbbrkFNTU18jo+nw9PPfUUcnJyoNVqkZWVhb/85S/y8jNnzmDGjBlITExEcnIybrzxRpw8ebLt3rA2EDuhp+qkPJmRqIdaKcDl8aHEUhe9MhERUbsQRRF2tz0qQ0uvAadSqfC73/0Ob775Zsg27733HlwuF+644w5ceuml2LRpEw4cOIC77roLt912G/bs2RPyPPn5+VCpVPj666/xP//zP2H3FR8fjzfffBOHDh3Ciy++iFdffRXPP/98yDqFhYX48MMPsWnTJmzatAk7d+7EypUr5eVLlizBU089haVLl+LQoUNYu3Yt0tLSAEhBLS8vD0ajEV988QW++uorGI1GTJo0Sa596gxi595bD8XD9PhZQCOd2rrmrztxrNSGt+ZdgasH9IhyKYmI6EI0vIeT3W3H6LWjo1KWPbP2wKA2tGjdI0eOYMiQIfj888+Rl5cHABg7diwyMjKwdu3aRutff/31GDJkCJ599lkAUk2PxWLB999/H7KeIAjYuHEjbrrpprD7feaZZ/DOO+9g7969AKSanmeeeQY///wz4uPjAQCLFy/GF198gd27d6OmpgY9evTAf//3f+OOO+5o9HxvvPEGnn76aRw+fBiCIACQTrUlJCTgww8/xLXXXhu2HB19762I77LepVWdBNIuAiC16zlWasOJ8lqGHiIiiorBgwdjzJgxeOONN5CXl4fCwkJ8+eWX+Mc//gGv14uVK1finXfewZkzZ+B0OuF0OhEXFxfyHJdddtl59/P+++/jhRdewLFjx2Cz2eDxeBoFiT59+siBBwB69eqF0tJSAMDhw4fhdDoxfvz4sM//r3/9C8eOHQvZHpBCTWFhYYvei44QW6Gn8oQcevoFuq3zWj1ERN2OXqXHnll7zr9iO+07EvPmzcOCBQuwatUq/P3vf0d2djbGjx+PZ555Bs8//zxeeOEFDB8+HHFxcVi4cGGj00UNQ1BDu3fvxi233ILHH38cEydOhNlsxvr16/Hcc8+FrKdWq0MeC4IAn88nvSZ986/J5/Ph0ksvxdtvv91oWY8enadiIbZCT9UJeZI9uIiIui9BEFp8iinafvvb3+KBBx7A2rVrkZ+fjzvvvBOCIODLL7/EjTfeiNmzZwOQgsXRo0cxZMiQiJ7/66+/RnZ2Nh555BF5XrgeY80ZMGAA9Ho9CgoKwp7euuSSS/DOO+8gNTW1zU5FtYfYacgMSDU9fgw9RETUGRiNRsyYMQMPP/wwSkpKMGfOHABATk4Otm7dil27duHw4cO4++678fPPP0f8/Dk5OTh16hTWr1+PwsJC/O1vf8PGjRsjeg6dToc//elPWLx4Mf7v//4PhYWF2L17N15//XUAwK233oqUlBTceOON+PLLL3HixAns3LkTDzzwAE6fPh1xmdtLbIWeMDU9p6vscHl80SoRERER5s2bh6qqKlxzzTXIysoCACxduhSXXHIJJk6ciHHjxqFnz55NNkxuzo033og//OEPWLBgAUaNGoVdu3Zh6dKlET/P0qVL8Z//+Z9YtmwZhgwZghkzZshtfgwGA7744gtkZWVh2rRpGDJkCObOnYu6urpOVfMTW723evYDHvgBgNSdcdjyz1Dr8mLbg2ORk2qMbkGJiKjVmusFRJ1XR/feiq2aHksx4PUAkM739u3BU1xERESxInZCj0ID+DxS8PHrG7gyc7ktWqUiIiKiDhI7oSchWxqzBxcREVFMip3Qk+gPPUE9uHitHiIiotgRe6GHNT1EREQxKfZCT1BNTx9/6CmtccLm9ESjVERE1Ia6eYfkbqej/16xE3rkNj0n5VlmvRopRg0A4CRre4iIuqzALRTsdnuUS0KRCPy9Gt4Co73Ezm0oEvtI48oTgCgC/rvA9k2JQ7nNhePltRiWYY5e+YiIqNWUSiUSEhJCLpYXuNs3dT6iKMJut6O0tBQJCQlQKpUdst/YCT3m3gAEwF0L2EqB+DQAUuj59mQVTrAxMxFRl9azZ08AkIMPdX4JCQny360jxE7oUWkBcyZgOSU1ZvaHnj5yY2Zeq4eIqCsTBAG9evVCamoq3G53tItD56FWqzushicgdkIPACT1kUJP5Qkg60oA9d3WT1TwPDARUXegVCo7/MuUuobYacgMAIl9pXFIt3X/VZnLbGz1T0RE1I3FVuhJ8oeeoG7r2ckGCAJgdXhQWeuKUsGIiIiovcVW6AlT06NTK5Fu1gPgRQqJiIi6s9gKPWFqegCgn/9u68cZeoiIiLqt2Ao9gZoeezngrJFn83YURERE3V9shR6dCTAkS9OVYe7BxWv1EBERdVuxFXqAJnpwsaaHiIiou4u90BOmXU+/QLf1ilr4fOy2TkRE1B3FXugJU9OTkaiHWinA5fGhxFIXpYIRERFRe4q90BOmpkepEJCVZAAAnCznlZmJiIi6o9gLPWFqeoCgKzPzHlxERETdUuyFnkBNj+U04Km/AjOv1UNERNS9xV7oMaYBagMg+oDqU/Js9uAiIiLq3mIv9AgCu60TERHFoNgLPUAT3dal0FNcaYfL44tGqYiIiKgdxWboSewjjYNqenrEaxGnUcInAqcq2YOLiIiou4nN0BOmpkcQBPTtwVNcRERE3VVshh52WyciIoo5sRl6AjU9VScBX337HTZmJiIi6r5iM/SYMwFBCXgcgO1neXbfFOmqzAw9RERE3U/Eocdms2HhwoVIT0+HTqfDqFGjsH79+hZtW1paijlz5iAlJQUGgwG5ubkoKChotJ7T6cQzzzyDYcOGIS4uDmlpaZg8eTJ27doVaXHDU6qBhExpujK423rg9BZDDxERUXcTceiZNm0a8vPzsXz5cmzZsgWXX345Zs6cibVr1za7ndPpxPjx41FQUIAXX3wRH330EdLS0jBp0iTs3LkzZN0777wTDz30EG666SZ8/PHHWLVqFcrKyjB27Fh88803kRY5vHDX6kmWTm+dszpR6/S0zX6IiIioU1BFsvLmzZuxdetWrF27FjNnzgQA5OXloaioCIsWLcKMGTOgVCrDbvv666/jwIED2LVrF3Jzc+VtR44cicWLF2PPnj0ApHC0du1azJo1C08++aS8/VVXXYX09HS8/fbbuOKKK1r1YkMk9QWObw+p6TEb1EiO06Ci1oUT5bUYlmG+8P0QERFRpxBRTc/GjRthNBoxffr0kPm33347SkpK5ODS1LaDBg2SAw8AqFQqzJ49G9988w3OnDkjFUihgEKhgNkcGjhMJhMUCgV0Ol0kRW5akz242JiZiIioO4oo9Bw4cABDhgyBShVaQTRixAh5eXPbBtYLt+3BgwcBAGq1Gvfeey/y8/Px4Ycfwmq14uTJk7jzzjthNptx5513RlLkpsnX6jkeMpuhh4iIqHuK6PRWRUUF+vXr12h+UlKSvLy5bQPrnW/b559/HmazGTfffDN8/i7lWVlZ+Pzzz5GTk9NsGZ1OJ5xOp/zYarWGXzGx8QUKAfAChURERN1UxA2ZBUFo1bJItv3LX/6CZ599Fo899hi2b9+Ojz76CIMGDcKECRPw/fffN7uPFStWwGw2y0NmZmb4FQM1PY5qoK5Knh24B9dxhh4iIqJuJaLQk5ycHLY2p7KyEgDC1uREuu3hw4exbNkyPP7441i6dCnGjRuHX//61/jkk0+QkJCABx98sNkyLlmyBBaLRR6Ki4vDr6iJA4xp/kKE6bZeZoMois3ui4iIiLqOiELP8OHDcfjwYXg8od259+/fDwAYNmxYs9sG1mtu23//+98QRRGXX355yHpqtRojR45stt0QAGi1WphMppChSWEaM2cnGyAIgNXhQWWtq9l9ERERUdcRUeiZOnUqbDYbPvjgg5D5+fn5SE9Px+jRo5vd9siRIyE9vDweD9asWYPRo0cjPT0dAOTx7t27Q7Z3Op347rvv0Lt370iK3LwwNx7VqZVIN+sBsF0PERFRdxJRQ+bJkydjwoQJmD9/PqxWK3JycrBu3Tp8+umnWLNmjXyNnnnz5iE/Px+FhYXIzs4GAMydOxerVq3C9OnTsXLlSqSmpmL16tX48ccfsW3bNnkfv/jFL3D55Zfjscceg91uxy9/+UtYLBa89NJLOHHiBN566622e/XNdFs/U12HE+W1uKxP06fsiIiIqOuIKPQAwIYNG/DII49g2bJlqKysxODBg7Fu3Trccsst8jperxderzekTYxWq0VBQQEWL16M+++/H3a7HaNGjcKWLVswduxYeT2FQoGtW7fimWeewXvvvYdnn30WRqMRQ4cOxebNmzF58uQLfMlB5JqekyGz+6bE4atj5azpISIi6kYEsZu31rVarTCbzbBYLI3b9xR/C7x+DWDKAB48JM9+46sT+POmQ5g8rCdenn1pB5eYiIiImv3+bqXYvMt6QKCmx1oCuB3ybF6rh4iIqPuJ7dBjSAY08QBEoLpInt0v6KrMPl+3rggjIiKKGbEdegQBSOojTQf14MpI0EOtFOD0+HDW6gi/LREREXUpsR16gLA9uFRKBbKSDACAE2U8xUVERNQdMPSEuVYPEHRl5nJbR5eIiIiI2gFDT2L4u63368F7cBEREXUnDD1JTV+gEGAPLiIiou6CoSepnzSuKgJ8Xnl2n2Qp9Jxk6CEiIuoWGHpMGYBCDfjcgPWMPDtwequ4qg4ujy9apSMiIqI2wtCjUAKJ0v3Bghszp8ZrYdAo4fWJKK6yR6lwRERE1FYYeoCw3dYFQahv18Nu60RERF0eQw/QTLd1NmYmIiLqLhh6gLA1PUD97SjYbZ2IiKjrY+gBmq7pkW88ygsUEhERdXUMPUBQTc9JQKy/wWj9VZlZ00NERNTVMfQA9b23nFbAXinP7uu/Vs85qxO1Tk80SkZERERthKEHANR6ID5dmg5q12M2qJEcpwHA2h4iIqKujqEn4Dw9uE5WMPQQERF1ZQw9AU3ceLQPr9VDRETULTD0BCT1kca88SgREVG3xNATkBj+9Bav1UNERNQ9MPQEyHdbD3+tnuNlNohB3dmJiIioa2HoCQg0ZLadA1z1tTp9kuMgCIDV4UFlrStKhSMiIqILxdAToE8EdAnSdNVJebZOrUS6WQ+Ap7iIiIi6MoaeYE10W++fKl2ZubCUt6MgIiLqqhh6gvHGo0RERN0WQ0+wpmp6ghozExERUdfE0BOsqZqeHtLpreO8QCEREVGXxdATrImann7+mp6iSjtcHl9Hl4qIiIjaAENPsEBNj6UY8NbfVb2nSQeDRgmvT8SpSnuUCkdEREQXgqEnWHwvQKkFfB4p+PgJgiDX9rBdDxERUdfE0BNMoQAS+0jTjXpw+dv1sAcXERFRl8TQ01BS+Luts6aHiIioa2PoaaipG4/6e3AVsgcXERFRl8TQ01CgpifoVhQAr9VDRETU1TH0NBS423qDmp6+/qsyV9ndqOKNR4mIiLochp6GEoNqekRRnm3QqJBu1gEAjpeztoeIiKirYehpKCELEBSAuxawlYYsYrseIiKirouhpyGVBjD1lqarwt+Dq5DteoiIiLochp5wkvpI4yZ6cPEeXERERF0PQ084Td54lD24iIiIuiqGnnCavPGoVNNzqtIOt5c3HiUiIupKGHrCaaKmp5dJB51aAbdXRDFvPEpERNSlMPSE00RNj0Ih1N+Di+16iIiIuhSGnnACNT32csBZE7JIbtfDa/UQERF1KQw94ehMgCFZmmYPLiIiom6BoacpieHvtl5/Dy6GHiIioq6EoacpSU10W08JXJWZp7eIiIi6EoaepiQ21W1dqumpqHXBYnd3dKmIiIiolRh6mhK423qDmp44rQo9TdKNRwvZmJmIiKjLiDj02Gw2LFy4EOnp6dDpdBg1ahTWr1/fom1LS0sxZ84cpKSkwGAwIDc3FwUFBWHXra2txbJlyzBw4EBotVokJycjLy8PR48ejbTIrSN3Wz/ZaFE/tushIiLqclSRbjBt2jR8++23WLlyJQYOHIi1a9di5syZ8Pl8mDVrVpPbOZ1OjB8/HtXV1XjxxReRmpqKVatWYdKkSdi2bRvGjh0rr2uz2ZCXl4eSkhI89NBDGDFiBCwWC3bt2gW7vYMuChg4vWU9DXhc0o1I/fr1iMOuwgq26yEiIupCIgo9mzdvxtatW+WgAwB5eXkoKirCokWLMGPGDCiVyrDbvv766zhw4AB27dqF3NxceduRI0di8eLF2LNnj7zuo48+isOHD2Pfvn3o16+fPP/Xv/51xC+w1YypgDoOcNcC1aeAlBx5Uf0FChl6iIiIuoqITm9t3LgRRqMR06dPD5l/++23o6SkJCS4hNt20KBBcuABAJVKhdmzZ+Obb77BmTNnAAB2ux2vvfYapk+fHhJ4OpwgAIl9pOkG7Xr6p/JaPURERF1NRKHnwIEDGDJkCFSq0AqiESNGyMub2zawXrhtDx48CAD417/+hdraWgwYMADz589HYmIiNBoNLrvsMnzyySfnLaPT6YTVag0ZWq2pG4+mSG16iirs8PrE1j8/ERERdZiIQk9FRQWSkpIazQ/Mq6iouOBtAzU+Tz31FPbv34//+7//w8aNG2EymTBlyhR89tlnzZZxxYoVMJvN8pCZmdmyFxdOEzU9GQl6aFUKuLw+nK7ijUeJiIi6goh7bwmC0KplLd3W5/MBADQaDbZs2YIpU6bg+uuvx6ZNm9CrVy888cQTze5jyZIlsFgs8lBcXNzs+s1q5sajff21PWzMTERE1DVEFHqSk5PD1uZUVlYCQNianEi3TU6W7nk1ZswYxMfHy+sZDAaMHTsW3333XbNl1Gq1MJlMIUOrJYa/KjPAbutERERdTUShZ/jw4Th8+DA8Hk/I/P379wMAhg0b1uy2gfWa2zZcu58AURShUHTg9RTlW1GcBPw1UAH9ewRuR8HQQ0RE1BVElCCmTp0Km82GDz74IGR+fn4+0tPTMXr06Ga3PXLkSEgPL4/HgzVr1mD06NFIT08HAPTq1Qu5ubn4+uuvQxoh2+127Ny5E1deeWUkRb4w5kxAUAIeB1BzNmRRfU0PT28RERF1BRGFnsmTJ2PChAmYP38+Xn31VWzfvh133XUXPv30Uzz99NPyNXrmzZsHlUqFoqIiedu5c+fioosuwvTp07F27Vps27YNv/3tb/Hjjz/iqaeeCtnPs88+i5qaGkycOBEffvghPvroI0yaNAnl5eXnbdPTppRqIMHfELqJG48eL2dNDxERUVcQ8bmiDRs24LbbbsOyZcswadIk7NmzB+vWrcOtt94qr+P1euH1eiGK9d25tVotCgoKkJeXh/vvvx9TpkzB2bNnsWXLlpCrMQNSe56CggJotVrceuutmDVrFtRqNXbs2BFynZ8OcZ4bj5bVOGF18MajREREnZ0gBieTbshqtcJsNsNisbSuUfOmPwB73wCu/k9g/LKQRVf8ZRtKa5z48L6rMCozoW0KTERERBf+/R0G77J+PoG7rVc214OL7XqIiIg6O4ae82m22zpvR0FERNRVMPScTxMXKATqb0fBCxQSERF1fgw955PYF4AAOKoBW2nIov6s6SEiIuoyGHrOR2Oob9dzLvSGqoHQc6KiljceJSIi6uQYeloi7SJpfO5QyOyMRD00KgVcHh9KquuiUDAiIiJqKYaelgiEntLQ0KNUCOiTbADAdj1ERESdHUNPS6QOlcYNTm8B9Vdm5j24iIiIOjeGnpYI1PSU/Qh4Q2+2ymv1EBERdQ0MPS2R2BdQG6Qbj1YeD1nEHlxERERdA0NPSygUQOoQabr0YMgiuaannDU9REREnRlDT0vJ7XpCGzMHrsp8zuqEzelpuBURERF1Egw9LSV3Ww+t6THr1UgxagCwXQ8REVFnxtDTUnK39YONFgV6cLFdDxERUefF0NNSqf7QU3UScIbW6PRPZQ8uIiKizo6hp6XikgFjmjRdejhkkXytnnLW9BAREXVWDD2RaOIUV6AHV2Epa3qIiIg6K4aeSMg9uBqGHqmm52RFLXy88SgREVGnxNATibRh0rhBt/XMRD3USgEOtw8lFt54lIiIqDNi6IlEmr+mp/QgINbX6KiUCmQnBxozs10PERFRZ8TQE4mUQYCgBOqqgJqzIYv6pbAHFxERUWfG0BMJtQ5IzpGmm7gyM++2TkRE1Dkx9EQq+BRXEN6Di4iIqHNj6IlUavjbUfBu60RERJ0bQ0+k5HtwhZ7e6u+v6TlrcaCWNx4lIiLqdBh6IhU4vVV2BPC65dkJBg2S4qQbj57glZmJiIg6HYaeSJmzAI0R8LmBimMhiwI9uArZg4uIiKjTYeiJlELRzJWZea0eIiKizoqhpzXSwoceuTEzT28RERF1Ogw9rRG4HUVp+Gv18AKFREREnQ9DT2vIp7cahp7601u88SgREVHnwtDTGoHTW5ZTgMMiz85KMkClEFDn9uJnqyNKhSMiIqJwGHpaQ58ImDKk6dLD8my1UoGsJAMANmYmIiLqbBh6WqvJHlyBxsxs10NERNSZMPS0VpM9uNhtnYiIqDNi6GmtJntw8QKFREREnRFDT2sF9+AS63tq9eONR4mIiDolhp7WShkIKFSA0wJYTsuzA7eiOFNdhzqXN1qlIyIiogYYelpLpZGCDxByiispToMEgxoAbzxKRETUmTD0XAj5FNcBeZYgCLzxKBERUSfE0HMh0i6Sxo2uzMx2PURERJ0NQ8+FCISeJnpw8Vo9REREnQdDz4UInN4q/wnwuOTZ/VJY00NERNTZMPRcCHNvQGsGfB4p+PjlpAYuUGiDKPLGo0RERJ0BQ8+FEISwV2bOSoqDUiGg1uXFOaszSoUjIiKiYAw9F0pu11MfejQqBTIT9QCk2h4iIiKKPoaeCxV8ZeYggR5chbxWDxERUafA0HOh5G7rTd14lDU9REREnQFDz4VKHSKNa0qAuip5tlzTwx5cREREnULEocdms2HhwoVIT0+HTqfDqFGjsH79+hZtW1paijlz5iAlJQUGgwG5ubkoKChodpu6ujoMHDgQgiDg2WefjbS47U9nBsxZ0nTQKa7AVZlZ00NERNQ5RBx6pk2bhvz8fCxfvhxbtmzB5ZdfjpkzZ2Lt2rXNbud0OjF+/HgUFBTgxRdfxEcffYS0tDRMmjQJO3fubHK7pUuXora2k9eWhOnBFajpOVNdB4ebNx4lIiKKNlUkK2/evBlbt27F2rVrMXPmTABAXl4eioqKsGjRIsyYMQNKpTLstq+//joOHDiAXbt2ITc3V9525MiRWLx4Mfbs2dNom2+++QYvvfQS3n77bUyfPj3S19Zx0i4Cfvo0pAdXilGDeJ0KNQ4PTlbUYnBPUxQLSERERBHV9GzcuBFGo7FRALn99ttRUlISNrgEbzto0CA58ACASqXC7Nmz8c033+DMmTMh67tcLsydOxf33XcfLrvsskiK2fFSG9f0CIKA/rwHFxERUacRUeg5cOAAhgwZApUqtIJoxIgR8vLmtg2sF27bgwdDez/9+c9/Rm1tLZ544olIihgd8rV6DgM+nzw7cA+uwlK26yEiIoq2iE5vVVRUoF+/fo3mJyUlycub2zaw3vm2/eGHH/D000/j448/RlxcHMrKylpcRqfTCaez/irIVqu1xdu2WnIOoNQALhtgOQUk9gGA+poeXquHiIgo6iJuyCwIQquWtXRbj8eDuXPnYsaMGZg4cWKkxcOKFStgNpvlITMzM+LniJhSDaQMkqaDGzOzBxcREVGnEVHoSU5ODlubU1lZCQBha3Ii3faFF17A8ePHsXz5clRXV6O6ulqurXE4HKiurobX23RvqCVLlsBischDcXFxy1/ghZAvUhjUbT2oTQ9vPEpERBRdEYWe4cOH4/Dhw/B4PCHz9+/fDwAYNmxYs9sG1mtu2wMHDsBisWDAgAFITExEYmIiRo4cCUDqvp6YmBj2eQK0Wi1MJlPI0CEC3daDenBlJxugEIAapwdlNt54lIiIKJoiCj1Tp06FzWbDBx98EDI/Pz8f6enpGD16dLPbHjlyJKSHl8fjwZo1azB69Gikp6cDAB566CFs3749ZFi3bh0A4J577sH27duRk5MTSbE7Rmrj21Ho1Er0TjQAAApL2a6HiIgomiJqyDx58mRMmDAB8+fPh9VqRU5ODtatW4dPP/0Ua9aska/RM2/ePOTn56OwsBDZ2dkAgLlz52LVqlWYPn06Vq5cidTUVKxevRo//vgjtm3bJu9j8ODBGDx4cMh+T548CQDo378/xo0bdwEvtx0FTm9VFAJuB6DWAZB6cJ2qtON4uQ25/ZOjWEAiIqLYFnFD5g0bNuC2227DsmXLMGnSJOzZswfr1q3DrbfeKq/j9Xrh9XpD2rFotVoUFBQgLy8P999/P6ZMmYKzZ89iy5YtGDt2bNu8mmiK7wnoEwHRC5T/KM/ul8Jr9RAREXUGgtjNW9harVaYzWZYLJb2b9/z9+uBoq+Am14GRs0CAKzZXYRHPzyAvEE98Pfbr2jf/RMREXUT7fH9zbust6W0xu16+vNu60RERJ0CQ09bCnPj0f7+qzKfrrLD6eGNR4mIiKKFoactpfm77JfWX6unR7wWRq0KPhEoqrBHqWBERETE0NOWevh7ndnOAbXlAKQrTQfuwcUrMxMREUUPQ09b0hrl+26Fux0F2/UQERFFD0NPWwtziqu+MTNreoiIiKKFoaetpTZuzBx8Dy4iIiKKDoaethamB1dwm55uflkkIiKiTouhp60FTm+VHQF8Uhf1vilxUCoEWB0eLHznB1TVuqJYQCIiotjE0NPWkvoBKh3gtgNVJwFINx5dMnkwFALw0Q8luPaFL7D10LnolpOIiCjGMPS0NYUS6DFImg46xXXH1f3wwfwx6N8jDmU1Ttz5f3vx4Ls/wGJ3R6mgREREsYWhpz2E6cEFABdnJeKT/7gad/2yHwQB2PDdGVz7wk5sP1IahUISERHFFoae9iD34DrQaJFOrcTD1w3B+/fkom9KHM5Znbj9zW+x+P1/w+pgrQ8REVF7YehpD/KNRw81ucql2UnY/B9XY94v+kIQgHf3nsbE57/AFz+VdVAhiYiIYgtDT3sIhJ7K44Cr6ftt6TVKLL1hKN65KxfZyQactTjwuze+wZIN+2FzejqosERERLFBELv5hWOsVivMZjMsFgtMJlPH7fjp/oC9HLjzcyDj0vOubnd58PSnP+LNXScBABkJejz9mxG4KielnQtKRJ2Z0+NFSbUDxZV2nLXUwahVo6dZh15mHVLjtVAp2++3q9cnoqS6Dqcq7SiqsKOoshanKuw4VWlHgkGNX+T0wNUDUjC0lwkKhdBu5aDY1B7f3ww97SX/18CJncCv/xu45LYWb/bPwgosev/fOF1VBwC47cpsPDR5MOK0qvYqKXVRVocbJdV1OFvtQImlTp6udXkgQIAgQBrkaQECAvP8j4OXo359vUaJNJMOaSatPO4Rr4NJp4IgdN4vN7vLg5Jqh/ReWOpwptqBcpsTKoUAlUIBtUqARqkImVYrFVApBaiVCmlZ0HRgmUalgFGrQrxOhXidGga1ss2+5D1eH85aHCiusuN0VR1OV9pRXFWH01V2FFfW4VyNA019SisEIDVeh14JUgjqZdbL455mHdITdOhhbD4YOdze+lBTUStPn6q043SVHW7v+b8ikuI0uConBVcPkIZeZn1r344W8Xh9OF5ei4MlFljsbiQbtegRr0WKUYseRi1M+s59nFLLMPS0QtRCz6dLgN2rgdHzgckrI9q01unBii2HsWb3KQBAZpIez/xmJK7sl9weJaUL5Pb6YHN4IAJQ+78w1UoFlBfwpehwe3HW4sDZ6jqUWOq/xOu/0B1ROQWqUyukEBSvQ6pJi9T4+mAU/NiobfsvHa9PRFmNE2eq60Lej8Djkuo6VHXQJSAEATBqVTDp1IjXqUICUbxOBaOufpm0XA2DRonSGgeKK+sDTXGVHWctDnh9zX8M69VK9E7UIz1BLwe7c1YHPOfZDmgcjHqa9KhxuOWam3NWZ7Pba5QK9E7So09yHLKSDMhONiAz0YAz1XX48mgZ/llYgVqXN2SbnFQjfpGTgl8OTMHovskX9KPN7vLg8NkaHDprxaESCw6VWHHk5xo4Pb5my5xi1NQHIf9YmqerXxavRXyYY1UURbi9ItxeH1wenzT2+uD2iqGPPf55Xi8EQUBmoh69Ew3QqZWtfr2REEXp/8RP52w4WlqDo6U2nKqwQ4QIpUIBtUKAUiHI4V2pEKBWKKBUClArBKiUCukHgVL6UaDyz1MrBcT5j+nAcWzSq+XHHfX6GHpaIWqh57u3gP+3AOj7S+D3H7fqKb46Wo4/fbAPZ6qlWp85Y/pg8aRBMGhY63Oh3F4fnB4fHG4vHG4vap1e1DjcqHF6UOPwwObwoMbhhs3/uKbBY2nsRo3D0+SHr0IAVA1qD9QKAWqVvwZBIdUgqBT1QclSJ9XeVLTwqt0JBjV6mfXISPD/yk/QIV6nBkQRIgBRlD4YRQA+/zQC8yFCFP3z/dPSMhE2pxelNQ6UWp04Z5W+YK2Olocsg0aJRIMGav8HrUohhUCV/LjB/MBjpQCl/8NXqRDg8frk0PezpWVf8katCukJOqQnSCGhh1ErfYn5RP+XlC902v/l1ty00+NDrf9v35IyREqjVCAjUY/e/i/NzCQ9MhMN6J2oR2aSAclxmkZfzD6fiHKbUwrHljr/2CGH5bOWlgejeJ0K2ckGZCfFISvZgOwkgzROjkNPk67ZAO/2+vD9qWp8dbQMXxwtx77T1QjepVop4JKsRH8tUA8MyzA3+XwVNicOllhx6KwVB0usOFhiwYny2rA1XXEaJYamm5Bi1KLC5kK5zYmyGidqIvwxoFEpYNKp4fHVB5yW1G41RRCAXiYdspPj0CdFeg+zk/zjZEOrAqAoivjZ6sDRczYcLbXh6LkaeRzJ/8u2olEqYNJLId/kD/smvQrx2tCAlJ6gx8SLerZ6Pww9rRC10HPmO+DVPMCQDCwqlP4ntEKNw43/2nwE676Ran0yEvSYenEGpoxMx6Ce8W1Z4qgSRREurw8Olw91bq80uLyoc3tQFzTP4fKGLA+EFofbB4fHC6d/HJjn9Pjg9K8jhxyP77y/rDsDvVqJXgk6ZCTUn7LISND7f7HrkZ6g69AA7HB7pRBU4/AHISdKrQ6U1tQHo1Jr5F86kVAqBPQ0Se9JINj0SpBCXyDkmHTqdtu/KIpwuH2ocbpDw7B/2uoPwsGhuMbhQY3Tg1qnBylGjT/M+INNklRrkhqvbZc2MV6fiAqbEyUWB37214ydszoQp5VCTlaSAX2S45BgULdZzZzF7sauwnJ8eawcX/xUJp+qD0gwqHFV/xT8YkAKEvTqoJBjabLWqUe8FhelmzC0lwkXpZsxNN2E7CRD2PfM4fairMaJcpsT5TaXPF0/LzDtanFtqUKQwlHwaU/psfSDRatSwOUVUVxpP+9zphi16JNcH4Kyk6W/QXayAWa9GiUWB346V4NjQbU3x87Zmvx/pRCAPslxyEk1YkCaEf1SjFApBXi8Ijw+KcB5fVKA9/pEeIKmpWXSOh5f/Ty3Vwr5VocH1jq3fGzbnJ4mT7WGM6K3Gf9vwS9avkEDDD2tELXQ47ID/5UOQAT+8ycgPu2Cnm7nT2V46IN9OGtxyPMGpBpxw4h03DCyF/r77+Temfh8IsprnThb7Qj5NRr41V5a44Td5fGHGy+ilUM0KgXitdIpifpTFWrE+6t3jf5fMvWnMBo81qph1KkgAHD7P0DcHp887fHXFrg8gQ+h0OngGoV4rVoOOmZ9230RdSS7y4NSqxNVdhd8ogiP/0PX4wse++ofe5uY7xOhEOAPeFLISY1vvtaBOhdRFFFUYceXx8rx5U/SqbDzheK+KXEYKgccE4amm5Aar2uX8tW5vCi3OVHj8ECjEoLCjEIOM5GcqhZFEZW1Lpz0t486WWHHKf+4qKL2vKdf1UqhyVompUJAn2QDBqbFY0CqETn+cd+UuA473eTzibC5/AE/EIbq3KhxumGtk4K+1f9DwFrnQWaSAQ9NHtzq/TH0tELUQg8AvHQpUHEMuG0j0P9XF/x0dpcHWw+dw6Z9Z7HzxzK4vPWnVYb0MuGGEb0wZUQ6spINF7yv8xFFERW1LvxsqW9jUmKRwkygYe05q6NV1cQqhdSQVq9WymOdOsxjjQI6lTRPp1ZCq1JAq1ZCFzTWqeuXSdMKaFXSWKdWQqNUsNcJUQfxeH349+lqfHm0HF8fK4fD7cPQXlKwuSjdhMG9TDB2404bljo3TlXYcbKiFkUVtf7G49Lj0hqplkutFNA3JQ4DUuPl2puBafHokxwHjSq2rjLD0NMKUQ0979wGHP5/wLV/AcYsaNOnttS5/QGoBF8dLQ85bz+ytxk3jEjH9SN6IT3hwnpReLw+FFXacfRcjb+xnHQe+Xh5LVzNNCQMCDSiDPQkCe1dokWcViWFGbUSOn+gUbdjF1wios7I7vKgwuZCT7OOn4F+DD2tENXQs2MlsGMFMHIWMPXldttNVa0Lnx38GR/vK8E/CytCThNdmp2IG0b0wvXDeyHV1HQVscfrw6lKuxRsztXgp0C4KasNqVEKJgjS+en0Bl1kA+1Nepr1SI3X8j8wERFFjKGnFaIaeg5/DLwzG+g1Erj7iw7ZZVmNE58eOIuP953Ftycr5UZnggBc0ScJN4xMx+V9ElFUUV9789N5am70aiUGpBmRkypVsw5MMyKnRzx6mnUxV91KREQdg6GnFaIaeioKgZcuAZRa4OESQNmx56p/tjiwef9ZbNpXgu9OVZ93fZ1agQGp8RiQZsSAVCncDEyLR0aCnu1eiIioQ7XH93f3bTHWGST2BdQGwG2X7sPVY2CH7r6nWYe5v+iLub/oi9NVdnyy7yw27TuLE+W16JNiwMDUeOSkGTEwNR4D0+LRO5HhhoiIui+GnvakUACpQ4Az/wJKD3Z46AnWO9GAu8f2x91j+0etDERERNHEBhntLXWoND53MLrlICIiinEMPe0t7SJpfO5QdMtBREQU4xh62lsg9JSypoeIiCiaGHraW6o/9FSdBJw1US0KERFRLGPoaW9xyYDRf5fZ0iPRLQsREVEMY+jpCGn+xsw8xUVERBQ1DD0dgT24iIiIoo6hpyOkDZPG7MFFREQUNQw9HSFweuvcAaB73/WDiIio02Lo6Qgpg6T7bzmqgW3LGXyIiIiigKGnI6h1wLVPStNfvwh88iDgC39HcyIiImofDD0dZfRdwJQXAQjA3jeAD+8BvJ5ol4qIiChmMPR0pEvnADe/BihUwL53gPd+D3ic0S4VERFRTGDo6WjDfwPMWCO18TmyCVg7A3DVRrtURERE3R5DTzQMmgzc+h6gjgOObwfemgbUVUe7VERERN0aQ0+09BsL/O4jQGcGincD+VOA2vJol4qIiKjbYuiJpszLgTmfAHE9gJ/3AX+fDFhLol0qIiKibomhJ9p6Dgdu3wKYMoDyn4A3JgGVJ6JdKiIiom6HoaczSBkAzP0USOwLVBdJwYd3ZCeizqi2HPjpM6BwO9siUpejinYByC8hSwo+b00FSg8Bb14HzN4ApI+KdsmIKJpEEagtk2qALcXS6fDUoYCxR/vv2+eTaqCL99QPFcdC10kZCGRcWj+kDQNUmvYvG1ErCKLYve+JYLVaYTabYbFYYDKZol2c87NXAmtuBkq+A7QmYNa7QHZutEtFnZHHJX0JVhcBVUX1Y7dduhaUQukfq5p5rA6/XJ8A9BgC9BgEaI3RfqXRI4rSJSVctYAmThoEoe3343VLf8vKE0DVCf/4ZP3gsjXexpAshZ8eg4HUIdJ06mBAn9j6crhqgTPfBYWcb6Tb5zSUMlAqc1WYU/FKLdBrBJBxmRSCel8q1WJf6PvmqgUsZwDracByun7a4wSMadIQ37N+HN9T+gxtj79XZ+Os8b8npwHbOUBjlI4PQ5I01id1ySDaHt/fEYcem82GRx99FO+++y4qKysxePBgPPTQQ7jlllvOu21paSkWL16MTZs2wW63Y+TIkXjyyScxfvx4eR2r1YqXXnoJW7duxZEjR2Cz2dC3b1/Mnj0bDzzwAHQ6XUQvsMuFHgBwWIF1twBFXwMqPXDL20DO+PNvR92LzwfUnG0cagLjmhJA7IDbmSRk+79Qh9SPUwYAKm377/tC+HxSAHTVAk6r9P/KaZHGDkv9vEbTwevUAKK3/jkFpdTjUmfyj/2D1hz6ONw6aoP09wwJNv6x5XTofhoRAHNvabCd87f7a+KjO76XPwj5Q1DqUH94jW+8ruVMaC3Oz/sBX4Mrxav0QO/LgMwrgMwrpWlDkrSstgI48y//sFca11U13o8+yR+ALquvEQo8ByAF+JoSqTyW0/5gcwawnqn/Mg8Xvs5HpQfi0wBjz/Dj+F7StCGp84Yjn1f6m1tOA9Wn6t8PeTglHavno4mvD0HBgSg4GAUv08RJ758yeieEOkXoufbaa/Htt99i5cqVGDhwINauXYvXXnsNb7/9NmbNmtXkdk6nE5dddhmqq6uxcuVKpKamYtWqVfjkk0+wbds2jB07FgBw4MAB5OXl4bbbbsO4ceNgNBrx5ZdfYuXKlbjqqquwdetWCBEcnF0y9ACAyw68+zvg2FZAqQF+8wYwZEq0S0WA9Ovf55F+YXpd0uBxSr98fW7/PLd/cPnnBT0OWc9Tv47HJX24BUKNpVha1hyVHkjMloJJYKwzS+XzeaQPTHk6+LG76eVet1SO0sNAbWn4/QpKIDknNAilDgWS+ko1RpHwOKVw4ayRajScNv+4RgosgeDirpOm3Xbp/4fbP89lr5/vrqvfxuOIrBzRptIBiX2kWpGkvqHTCVmhIdNll047lR4Gyg5L49LD0jHTFHOWFIJSBkrhq/ib8OvHpwNZo4FM/9BzOKBUt+w1iCJQebw+CJ3eK/VMDXccJ/aVaqWsZwBbKZoMccG0JqnThznDP86U3hfbOWmo+VkabOekINtSglIqiyHJ/+UfGCc2eOwfB9ZtafD3ef3Hb139sRk4XgODxyGVORD8LMXSYC1pHETD0SVI70d8mnR82Cukoa7ywn4cKdSAWi8NKl3QtL5+uqllpgxgxPRW7zrqoWfz5s24/vrrsXbtWsycOVOef+211+LgwYM4deoUlMrwH3irV6/Gfffdh127diE3Vzpd4/F4MHLkSBiNRuzZswcAUFsrXZ04Li4uZPtnn30WixYtwpdffolf/OIXLX6BXTb0ANKX4IY7gEMfSf8pb1oNjDx/jVq3JorSB6SlWPrVYy0BvE7pV73orf/iFr3+sa/xvJDl/nFwcPE66wNNSLBx1S9ryQd0WxCU0q/7xGzpSzAheJwtte9oz1+oteX1X6ilh+qnnU38slTppC/V1KGAKV36YHfapA/zkEBjA1w10tjnbr/yAwAE6csyUPPSomlz6Hy1XvpiclhCB6dVqoGQ51mbWMcifanpk/yBpm/9OLGPNG3sCSgusG+JwwqU/Sj9rcqO+P9mRwDbz028NQop1GQGhRxz77Y9pjxO4NwB4HRQbVDDdkGAdFpMDjP+Wq3AdCDo6Mwt36/LLr3umnMNxkHBqOZnKRS0ljrOH4QSpZo0jwNwBwUajz/QnO/Hy/kISuk9SMisf2/MvaUga+4tvTfhavIA6bPRaZGaTtgrgsZBoShkXuWFB6WA9EuAu7a3evOoh54777wT69evR1VVFVSq+iqvdevWYdasWfj6668xZsyYsNtOmDABxcXFOHIktFfSihUr8PDDD+P06dPIyMhoct9ffPEFxo4d2yhwnU+XDj2AVBPw8X8AP7wtPZ7wZ2DkTMCY2rHlEEWpKv7UHuliihWF0gdQXApgSKkfG5JC50V6CsTrkX6FBkJNdbFUfVvt/9VTXSwFj85EUEgf2EqN9Is4MCjU/nkq/1gjtZcJXi94HYVaes+Ca21MGVGtXg5LFKWwGRKEDklftp661j+v2iC1RdDGS+2INPH+djQG6ctFY/D/koyTxhqDtI3aIK2n1gc9NtSvp9Z3jlMXXk/0/pb2yvpaobKfpFMYWaOldjfRaLNlrwRKvpcCgTkDMPWWjv1o/J08LqmheODLv64qdDoQAoKX1VW1PhTINSIG/7Sh/jjVxEk/Fsy9pVobsz/kxPeMvAb1QoiiFFblWqngGqm68zwOWj8hC8hb0upitMf3d0T/Aw8cOIAhQ4aEBB4AGDFihLy8qdBz4MABXH311Y3mB7Y9ePBgs6Hn888/BwBcdNFFkRS561OqgF//t/RFsOcVYOsyaTBlAOkXS0PGJUCvUaHnyC+UxwWc/bcUcIr3SGGnqVMdzdHEA3HJ4YORxij90gqEmepTUlV3s20bIIWM+F7+D4QM/5easr4hbmBaUDQxz99oN3ieSlsfTFRaKcSoNP4wo248T+VfV6ntfKGkvQmC9L6bM4AB19TP93mlRreB2qDaMn948QcZjTHosSlo2j/uyA/1aIjmcWJIAvpcJQ2dgSGp87RTVGnqj+eWCq49CQQjly000KiDAk3gdI9Kd+G1eR1BEPzlj6wNbVcQ0f/CiooK9OvXr9H8pKQkeXlz2wbWi3Tbffv24emnn8bUqVPlkNQUp9MJp7O+JsBqjeC8bmelUACTVkpf8t+/Jf2itvob+R3ZVL9eYt/6EJR+MdBrZNNVng3ZK+sbM57aI/Uea9gmQqmRwlXWaKlbqrNGqg6tLQfs5f5xULWpzyOdwnDVSF+GLX69av+HUKZU25Hg/8WTkCVNmzJa3saAOo5CCST3l4YhN0S7NETtR6GQTmldSE85ioqIf3o014j4fA2MW7PtyZMnccMNNyAzMxOvvfbaecu3YsUKPP744+ddr8sRBGDMAmlw2qRamJLvpXBS8r3UeLDK3xvk4IbARv5raPhDUPolQM9h0q+NimP+gOOvySn/qfE+9UlA1pX15/rTL2558hdFqa1DbUVQIAoKRrXlUmiKT5PCjNkfaBKypC6n3f1XPxERdbiIQk9ycnLYGpnKSqkhWLianAvZtqioCHl5eVCpVCgoKGj2+QOWLFmCBx98UH5stVqRmZl53u26FK2xcVV1XRVQ8kN9CDrzvdTts/xHafj3Omk9QSnV/jR17Y1At9SsK6XeOa09xy4IQb+Eclr3HERERG0ootAzfPhwrFu3Dh6PJ6Rdz/79+wEAw4YNa3bbwHrBmtq2qKgI48aNgyiK2LFjB3r37t2iMmq1Wmi1nfz6Ie1Bnwj0z5OGAFupvzboe+mCYyXfSe0sHNVSW5SMS6QanKwrgd5XSG1viIiIuqmIem9t2bIF1113HdavX48ZM2bI8ydPnox9+/Y122X95Zdfxr333ovdu3dj9OjRAKQu66NGjYLRaMTu3bvldU+dOoWxY8fC6/Vix44dYdsRtVSX773VlkRRagdUWy5dV6WzX1yOiIhiVtR7b02ePBkTJkzA/PnzYbVakZOTg3Xr1uHTTz/FmjVr5MAzb9485Ofno7CwENnZ2QCAuXPnYtWqVZg+fbp8ccLVq1fjxx9/xLZt2+R9lJaWIi8vD2fPnsXrr7+O0tJSlJbW9xrq3bt3i2t9qAEh6KquREREMSbihswbNmzAI488gmXLlsm3oVi3bl3IbSi8Xi+8Xi+CK5G0Wi0KCgqwePFi3H///bDb7Rg1ahS2bNkiX40ZAA4dOoTjx48DAGbPnt1o/8uXL8djjz0WabGJiIgoxvGGo0RERNTptMf3dxe4ShIRERHRhWPoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMYeoiIiCgmMPQQERFRTGDoISIiopjA0HMeNpcNbxx4AxuObkA3v2MHERFRtxbxDUdjhdvrxns/vYdX/v0KqpxVAICD5QexZPQSqBR824iIiLoafns3IIoi/lH0D/ztu7/hVM0pAEB6XDrO1p7Fuz+9i1J7KZ4e+zT0Kn2US0pERESR4OmtIHt/3ovZm2fjjzv/iFM1p5CkS8LSK5di07RN+Ou4v0Kj0GDH6R2447M7UOmojHZxiYiIKAKC2M0bqrTk1vTHq4/j+e+ex47iHQAAvUqPORfNwe8v+j3i1HHyet+Xfo8FBQtgdVmRFZ+FV655BZmmzA54FURERLGlJd/fkYrp0FNqL8XqH1Zj47GN8Ik+KAUlbh5wM+aPmo8UfUrY5ztuOY75W+ejpLYESbokrBq/CsNShnXESyEiIooZDD2tEO5Ns7ls+PvBv+OtQ2+hzlMHAPhV5q/wwKUPoJ+533mfs8xehvsK7sPhysPQq/R4duyz+GXvX7br6yAiIoolDD2tEPym6Y16vP/T+3jl36/IbXJG9hiJ/7zsP3Fx6sURPW+tuxYP7ngQu0p2QSkosSx3GaYNmNYeL4GIiCjmMPS0QuBN27BvA14/9jqKrEUAgGxTNhZeshDjs8ZDEIRWPbfb58Zjux7D/yv8fwCA+SPnY/7I+a1+PiIiIpIw9LRC4E0b8vIQKPVKJOmSMH/kfNw88GaoFeoLfn5RFPHfP/w3/nff/wIApuZMxdLcpW3y3ERERLGqPUJPzFynR6/UY+7IuZhz0ZyQHlkXShAE3H/x/UgzpOEve/6Cjcc2orSuFH8d+1cY1IY22w8AOL1O2Fw2JOmSukVtksPjgFap7RavhTont9cNu8cOo9oIpUIZ7eJQNySKIqwuKxweB1INqfw86+Ripqan8Gwh+vU8fyPlC7GjeAcW7VwEh9eBoclDsWr8qiZ7gbWEKIo4Vn0Mu0p24Z8l/8Tec3vh9DqhV+mRYcxA7/jeyIzPRG+jNM6Mz0S6MR0apabtXlQbEEURZ2vP4nDlYRyuOIwjlUdwuOIwSutKoVKokKhNRKJOGpJ0SUjSJcnzknRJ9cu0STBpTVAIbXt5Ka/PC5fPBZfXBafXCZc3aDpovtPrhNvrrp/2uUPmqxQq6FV6GFQG6NV66FX1g0FlqH/sX9ZVagNFUUSdpw4WpwUWlwUWpwXVzmpYnBZYXdaQx8HzXD4X4tXxMGvNMGlMMGlN0lhjanZenDqu0RdHcBmqndWNBovTgipHVaPlte5aAIBCUMCsMYccZw2Pu0RdIhK10nSCLqHL/H0CvD4vbG6bNLhsqHHVoMZVA5vbJo9tLmm5VqlFij4FyfpkJOuS5ekkXRKvOO/nE32odlajzF6G8rpylNpLUV5XjrI66XGZvUyednqdAIA4dRwGJAzAgERpGJg4EAMSB8CkaZtailjD01ut0B5vWnP2le3DgoIFqHJWIcOYgZeveRl9zX1bvH1FXQV2n90tB52yurKI9i9AQM+4nlIYahCKesf3hllrjvQlRcQn+lBkLZKDzeFKabA4LW3y/EpBiQRtgvxFFXg9bp8bbp8bHp8HHp9HnpbHXjc8YoOxf7mI6PwXUCvUIcFIr9JDo9RAKSihUqigEBRQKpRQCdK0PC94eZhpAPCKXvhEH7w+/1isH4uiGPI4eBwYHB5HSMhx+9wd9r4oBSXiNfEwaUzQqrRymAp8sXSUeE28HI4StAkQBEE6VkRADPwT5an6+eHm+Y8xAQIUggIKQQFBEKBA/bRSUErT/nUEQZD/rsHz6tx1qHH7A43Lhhq3NLZ77Bf8mgUISNAmSGGoQSAKnk7Rp8CsNXd4MPT6vLC6rCFBt85TBx988vvuE6Vpn+gLeSxP+/9GwcsdXgfK7eUorStFuV0KNhV1FfCInhaXTSko4RW9YZf1jOsph6FAEOpr6gu1sn3eP1EUpTMDgcAbdJw0DMLhlts9dggQoFaqoVY0GBrOU6qhUqjCrpMel44Zg2e0+nUw9LRCR4ceACiyFmH+tvkorilGgjYBL/3qJYxKHRV2XZfXhe9Lv5dDzuHKwyHLdUodLu15Kcb0GoMx6WOQbcpGSW0JimuKcbrmdP3YJo0DXfCbEq+JR3pcOsxaM+I18SGDSWOSptWN5xnUhkY1LG6fG8erj+NQxSEp5FQexo+VP4b98FUJKuQk5mBw0mAMSRqCoclD0dfcF3WeOlQ6KlHlqJLHVU5pWn7sH2rcNZH9IVpBISigVWqhUWqgVUhjjVIjz5MfN1imVqjh8XlQ56lDnacOdo9dnq7z1KHOXT+vqQ/Gzk6tUMOsNSNBmyDXzAQeB2ppguepFWrUuGrk2h+rywqr0yrXBIWb5/K5zluGwP4C4TcwHTyYtVKtToI2AXqVHlaXNeR4qnRUospZFXrc+Y+9amc1fKKvg97VtqdT6mDUGGFUGxGviYdRbYRREzStNqLOW4eKugpU1FWgvK4cFY4KVDoqI37dOqUOceq48Pvxl0EuR5gyiaKIKmeDGjpHgxq8oOVWp7XDf6Qk6ZKQok9BD30PaWzogR76HvI4RZ+CFH0KlAoliixFOFp9FD9V/YSjVUdxtOooSmpLwj6vSqFCX3PfkDDUx9QHHp8Hdo8ddrc9/Ng/Xeepk+fVumtD5te4ajr0h0pThqcMx9rr17Z6e4aeVohG6AGkGpsFBQtwoOIAtEotnv7l0/hV1q8giiKOW45jV8ku7CrZhX+d+1ejoDI4aTBy03MxJn0MLk69GFqltkX7FEURFY6K0DBUU4zTNmlcXlfe6tejEBTyB5ZJY4IIEYXVhWH/Y+mUOgxMGoghSUOkIXkIchJyLvi0m8vrCglFVQ7pC0ohKKBW1P/aaGocdlqphkpQyQGmvav2RVGE2+cODUdBgcjtc8u1Mx7R02jaK3rrx0HTHp9HWs/ngSAIITUHTY4V9Y8brq9RakIChEljgl6lb/f2Cg6PIyQIObyOkFBjUBnavQyB2oTgcGRxWiBChBD4J0hjAPJ02HnB8wXINQwNayGCa+F88IXM94m+kHl6lV4KF2p/kAiajlfHt7r2wOvzotpZjQqHPwjVSUEoMB0IR4H50aohBSCfNk3QJiBOHSfXggVqz0LG/tqzpqYDnx/BISYQapJ1yRdcG1PjqsGx6mP4qfInHK2WgtBPVT/B5ra10bvRNAFC+NDpnzZpTI3CaLwmHnHqOIiQPqvcXndITXrw4+DlLp+r0bI0Qxp+f9HvW11+hp5WiFboAQC7245FXyzCF6e/gEJQIC8zD/vL96PUXhqyXoo+BWPSxyA3PRdX9rrygtoBna88Z2xncM5+Tj7fb3VZ5engITDf6rI2+4shXh2PwclS7c3gpMEYmjwU2aZstgsg6sYC7YcathUKfhw4XRIyHdTmyOa2QYDQuKZOF1Rb5z+1GKi5C9QkdrX2Vg2Jooifa3+WaoSCaoZO15yGVqWFQWWQBrVBbiMYeBynipPnG9RSW8HgxwaVQQ4w4WrouxKGnlaIZugBAI/Pgyd3P4kPjn4gz9Mqtbgs7TLkpuciNz0XAxIGdNoW/4Fzww3DkMfnQU5iDnobe3fashNR5xU4ldaVv5SpfbHLehekUqiwPHc5hiYPxRnbGVzZ60pcknZJi09ZRZsgCNCpdNCpdOhh6BHt4hBRN8GwQ9HA0NMBBEHAbwf9NtrFICIiimmM2kRERBQTGHqIiIgoJjD0EBERUUxg6CEiIqKYwNBDREREMYGhh4iIiGICQw8RERHFBIYeIiIiigkMPURERBQTGHqIiIgoJjD0EBERUUxg6CEiIqKYwNBDREREMYGhh4iIiGICQw8RERHFBIYeIiIiigkMPURERBQTGHqIiIgoJjD0EBERUUxg6CEiIqKYwNBDREREMSHi0GOz2bBw4UKkp6dDp9Nh1KhRWL9+fYu2LS0txZw5c5CSkgKDwYDc3FwUFBSEXXfbtm3Izc2FwWBASkoK5syZg9LS0kiLS0RERASgFaFn2rRpyM/Px/Lly7FlyxZcfvnlmDlzJtauXdvsdk6nE+PHj0dBQQFefPFFfPTRR0hLS8OkSZOwc+fOkHV37tyJyZMnIy0tDR999BFefPFFbNu2DePHj4fT6Yy0yEREREQQRFEUW7ry5s2bcf3112Pt2rWYOXOmPP/aa6/FwYMHcerUKSiVyrDbrl69Gvfddx927dqF3NxcAIDH48HIkSNhNBqxZ88eed0rrrgCtbW1+Pe//w2VSgUA2LVrF6666iqsXr0a8+fPb/ELtFqtMJvNsFgsMJlMLd6OiIiIoqc9vr8jqunZuHEjjEYjpk+fHjL/9ttvR0lJSUhwCbftoEGD5MADACqVCrNnz8Y333yDM2fOAADOnDmDb7/9FrfddpsceABgzJgxGDhwIDZu3BhJkYmIiIgARBh6Dhw4gCFDhoSEEQAYMWKEvLy5bQPrhdv24MGDIc/R1LrN7QOQTqNZrdaQgYiIiCii0FNRUYGkpKRG8wPzKioqLnjbwLipdZvbBwCsWLECZrNZHjIzM5tdn4iIiGJDxA2ZBUFo1bJIt21q3fPtY8mSJbBYLPJQXFzc7PpEREQUG1TnX6VecnJy2JqWyspKAOFrZyLdNjk5GUD4WqPKyspm9wEAWq0WWq222XWIiIgo9kRU0zN8+HAcPnwYHo8nZP7+/fsBAMOGDWt228B6zW0bGDe1bnP7ICIiImpKRKFn6tSpsNls+OCDD0Lm5+fnIz09HaNHj2522yNHjoT08PJ4PFizZg1Gjx6N9PR0AEBGRgauuOIKrFmzBl6vV1539+7d+PHHHzFt2rRIikxEREQEIMLr9ADSNXn27t2Lp556Cjk5OVi3bh1effVVrFmzBrfeeisAYN68ecjPz0dhYSGys7MBSL2qLr30UlitVqxcuRKpqalYvXo1Pv74Y2zbtg1jx46V97Fjxw5MmDABU6ZMwb333ovS0lI89NBDMJvN2Lt3b0Snr3idHiIioq4n6tfpAYANGzbgtttuw7JlyzBp0iTs2bMH69atkwMPAHi9Xni9XgTnKa1Wi4KCAuTl5eH+++/HlClTcPbsWWzZsiUk8ADAuHHjsHnzZpw9exZTpkzB/fffj7y8PBQUFLC9DhEREbVKxDU9XQ1reoiIiLqeTlHTQ0RERNQVMfQQERFRTGDoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMiust6VxS49qLVao1ySYiIiKilAt/bbXkN5W4fempqagAAmZmZUS4JERERRaqiogJms7lNnqvb34bC5/OhpKQE8fHxEAQh2sWJOqvViszMTBQXF/O2HG2I72vb43vaPvi+tj2+p+3DYrEgKysLVVVVSEhIaJPn7PY1PQqFAr179452MTodk8nE/5ztgO9r2+N72j74vrY9vqftQ6Fou+bHbMhMREREMYGhh4iIiGICQ0+M0Wq1WL58ObRabbSL0q3wfW17fE/bB9/Xtsf3tH20x/va7RsyExEREQGs6SEiIqIYwdBDREREMYGhh4iIiGICQ08M2LFjBwRBCDvs3r072sXrEmpqarB48WJce+216NGjBwRBwGOPPRZ23e+++w7XXHMNjEYjEhISMG3aNBw/frxjC9xFtPR9nTNnTtjjd/DgwR1f6E7s888/x9y5czF48GDExcUhIyMDN954I/71r381WpfHacu19H3lcRqZH374Addffz2ysrKg1+uRlJSE3NxcrFmzptG6bXW8dvuLE1K9//qv/0JeXl7IvGHDhkWpNF1LRUUF/vd//xcjR47ETTfdhNdeey3sekeOHMG4ceMwatQovPvuu3A4HFi2bBmuvvpq/PDDD+jRo0cHl7xza+n7CgB6vR6ff/55o3lU7+WXX0ZFRQUeeOABDB06FGVlZXjuuedw5ZVX4rPPPsOvfvUrADxOI9XS9xXgcRqJ6upqZGZmYubMmcjIyEBtbS3efvtt3HbbbTh58iQeffRRAG18vIrU7W3fvl0EIL733nvRLkqX5fP5RJ/PJ4qiKJaVlYkAxOXLlzdab/r06WJKSoposVjkeSdPnhTVarW4ePHijipul9HS9/X3v/+9GBcX18Gl63rOnTvXaF5NTY2YlpYmjh8/Xp7H4zQyLX1feZy2jdGjR4uZmZny47Y8Xnl6i6gFAtXUzfF4PNi0aRNuvvnmkEvRZ2dnIy8vDxs3bmzvYnY5LXlfqeVSU1MbzTMajRg6dCiKi4sB8DhtjZa8r9R2UlJSoFJJJ6La+nhl6Ikh9913H1QqFUwmEyZOnIivvvoq2kXqVgoLC1FXV4cRI0Y0WjZixAgcO3YMDocjCiXrHurq6tCzZ08olUr07t0bCxYsQGVlZbSL1elZLBZ89913uOiiiwDwOG0rDd/XAB6nkfP5fPB4PCgrK8Pq1avx2Wef4U9/+hOAtj9e2aYnBpjNZjzwwAMYN24ckpOTcezYMTzzzDMYN24cPvnkE0ycODHaRewWKioqAABJSUmNliUlJUEURVRVVaFXr14dXbQub+TIkRg5cqTcBm3nzp14/vnnUVBQgG+//RZGozHKJey87rvvPtTW1uKRRx4BwOO0rTR8XwEep61177334n/+538AABqNBn/7299w9913A2j745WhJwZcfPHFuPjii+XHV199NaZOnYrhw4dj8eLFDD1trLnTNTyV0zp/+MMfQh5PmDABF198MX7zm9/g1VdfbbScJEuXLsXbb7+Nl156CZdeemnIMh6nrdfU+8rjtHUefvhh3HHHHSgtLcXHH3+MBQsWoLa2Fn/84x/lddrqeGXoiVEJCQm44YYb8Morr6Curo69C9pAcnIygPpfJsEqKyshCAISEhI6uFTd19SpUxEXF8fLLjTh8ccfx5NPPom//OUvWLBggTyfx+mFaep9bQqP0/PLyspCVlYWAOC6664DACxZsgS///3v2/x4ZZueGCb6b7vGX3Vto3///tDr9di/f3+jZfv370dOTg50Ol0UStZ9iaIIhYIfYw09/vjjeOyxx/DYY4/h4YcfDlnG47T1mntfm8PjNDJXXHEFPB4Pjh8/3ubHK/8KMaqqqgqbNm3CqFGj+AHXRlQqFaZMmYINGzagpqZGnn/q1Cls374d06ZNi2Lpup/3338fdrsdV155ZbSL0qk88cQTeOyxx/Doo49i+fLljZbzOG2d872vTeFxGrnt27dDoVCgX79+bX688i7rMWDWrFnIysrCZZddhpSUFBw9ehTPPfccCgsLsWXLFlxzzTXRLmKXsGXLFtTW1qKmpgZz587F9OnT8dvf/haAVCVrMBhw5MgRXH755bjkkkvw0EMPyRfRqqys5EXfmnC+97WsrAyzZs3CLbfcgpycHAiCgJ07d+KFF15A//79sWfPHsTFxUX5VXQOzz33HP74xz9i0qRJYb+YA1+8PE4j05L3taioiMdphO666y6YTCZcccUVSEtLQ3l5Od577z288847WLRoEZ5++mkAbXy8XvBVhKjTW7FihThq1CjRbDaLSqVS7NGjhzh16lTxm2++iXbRupTs7GwRQNjhxIkT8np79+4Vx48fLxoMBtFkMok33XSTeOzYsegVvJM73/taWVkpTp06VezTp4+o1+tFjUYjDhgwQFy8eLFYXV0d7eJ3KmPHjm3yvWz4cc/jtOVa8r7yOI3cG2+8IV599dViSkqKqFKpxISEBHHs2LHiW2+91WjdtjpeWdNDREREMYFteoiIiCgmMPQQERFRTGDoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMYeoiIiCgmMPQQERFRTGDoISIiopjA0ENEREQx4f8Dsnuo/FrkZW8AAAAASUVORK5CYII=\n", - "text/plain": [ - "
    " - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "execution_count": 20, + "id": "18fc9dab", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], "source": [ "\n", "import matplotlib.pyplot as plt\n", @@ -2077,7 +2132,7 @@ "\n", "n = 100\n", "n_boostraps = 100\n", - "maxdepth = 30\n", + "maxdepth = 8\n", "\n", "# Make data set.\n", "x = np.linspace(-3, 3, n).reshape(-1, 1)\n", @@ -2129,8 +2184,10 @@ }, { "cell_type": "markdown", - "id": "79f3b548", - "metadata": {}, + "id": "c4507499", + "metadata": { + "editable": true + }, "source": [ "## Random forests\n", "\n", @@ -2150,8 +2207,10 @@ }, { "cell_type": "markdown", - "id": "f51ddbb5", - "metadata": {}, + "id": "30c44f26", + "metadata": { + "editable": true + }, "source": [ "$$\n", "m\\approx \\sqrt{p}.\n", @@ -2160,8 +2219,10 @@ }, { "cell_type": "markdown", - "id": "5307d349", - "metadata": {}, + "id": "182dde82", + "metadata": { + "editable": true + }, "source": [ "In building a random forest, at\n", "each split in the tree, the algorithm is not even allowed to consider\n", @@ -2183,8 +2244,10 @@ }, { "cell_type": "markdown", - "id": "f5df6740", - "metadata": {}, + "id": "441e79b9", + "metadata": { + "editable": true + }, "source": [ "## Random Forest Algorithm\n", "The algorithm described here can be applied to both classification and regression problems.\n", @@ -2207,8 +2270,10 @@ }, { "cell_type": "markdown", - "id": "e1f7886c", - "metadata": {}, + "id": "906bdc82", + "metadata": { + "editable": true + }, "source": [ "## Random Forests Compared with other Methods on the Cancer Data" ] @@ -2216,8 +2281,11 @@ { "cell_type": "code", "execution_count": 21, - "id": "4e31e8c8", - "metadata": {}, + "id": "d837183f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", @@ -2285,8 +2353,10 @@ }, { "cell_type": "markdown", - "id": "3de40f94", - "metadata": {}, + "id": "837cb618", + "metadata": { + "editable": true + }, "source": [ "Recall that the cumulative gains curve shows the percentage of the\n", "overall number of cases in a given category *gained* by targeting a\n", @@ -2299,8 +2369,10 @@ }, { "cell_type": "markdown", - "id": "9bff02d3", - "metadata": {}, + "id": "bb3be0d1", + "metadata": { + "editable": true + }, "source": [ "## Compare Bagging on Trees with Random Forests" ] @@ -2308,8 +2380,11 @@ { "cell_type": "code", "execution_count": 22, - "id": "23f8734b", - "metadata": {}, + "id": "51a8089f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "bag_clf = BaggingClassifier(\n", @@ -2320,8 +2395,11 @@ { "cell_type": "code", "execution_count": 23, - "id": "59bd532d", - "metadata": {}, + "id": "ad835b2a", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "bag_clf.fit(X_train, y_train)\n", @@ -2335,8 +2413,10 @@ }, { "cell_type": "markdown", - "id": "e24309a7", - "metadata": {}, + "id": "3b670a57", + "metadata": { + "editable": true + }, "source": [ "## Boosting, a Bird's Eye View\n", "\n", @@ -2353,8 +2433,10 @@ }, { "cell_type": "markdown", - "id": "8f84b7a8", - "metadata": {}, + "id": "fca2f8ea", + "metadata": { + "editable": true + }, "source": [ "## What is boosting? Additive Modelling/Iterative Fitting\n", "\n", @@ -2365,8 +2447,10 @@ }, { "cell_type": "markdown", - "id": "a8144faa", - "metadata": {}, + "id": "f083d00f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f_M(x) = \\sum_{i=1}^M \\beta_m b(x;\\gamma_m),\n", @@ -2375,8 +2459,10 @@ }, { "cell_type": "markdown", - "id": "9f523032", - "metadata": {}, + "id": "5e67f8ab", + "metadata": { + "editable": true + }, "source": [ "where $\\beta_m$ are the expansion parameters to be determined in a\n", "minimization process and $b(x;\\gamma_m)$ are some simple functions of\n", @@ -2390,8 +2476,10 @@ }, { "cell_type": "markdown", - "id": "b0d77841", - "metadata": {}, + "id": "0a3b13c4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(t) = \\frac{1}{1+\\exp{(-t)}},\n", @@ -2400,8 +2488,10 @@ }, { "cell_type": "markdown", - "id": "3e086177", - "metadata": {}, + "id": "b4cd2760", + "metadata": { + "editable": true + }, "source": [ "where $t=\\gamma_0+\\gamma_1 x$ and the parameters $\\gamma_0$ and\n", "$\\gamma_1$ were determined by the Logistic Regression fitting\n", @@ -2412,8 +2502,10 @@ }, { "cell_type": "markdown", - "id": "15ad95dc", - "metadata": {}, + "id": "cbba0135", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(\\boldsymbol{y},\\boldsymbol{f}) = \\frac{1}{n} \\sum_{i=0}^{n-1}(y_i-f(x_i))^2.\n", @@ -2422,8 +2514,10 @@ }, { "cell_type": "markdown", - "id": "6287183a", - "metadata": {}, + "id": "9a94bd46", + "metadata": { + "editable": true + }, "source": [ "In this case the function $f(x)$ was replaced by the design matrix\n", "$\\boldsymbol{X}$ and the unknown linear regression parameters $\\boldsymbol{\\beta}$,\n", @@ -2433,8 +2527,10 @@ }, { "cell_type": "markdown", - "id": "27de8464", - "metadata": {}, + "id": "7ddc4cca", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{\\beta}=\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n", @@ -2443,16 +2539,20 @@ }, { "cell_type": "markdown", - "id": "7d8e41cb", - "metadata": {}, + "id": "ab04ee35", + "metadata": { + "editable": true + }, "source": [ "In iterative fitting or additive modeling, we minimize the cost function with respect to the parameters $\\beta_m$ and $\\gamma_m$." ] }, { "cell_type": "markdown", - "id": "f45f6f2a", - "metadata": {}, + "id": "f1065910", + "metadata": { + "editable": true + }, "source": [ "## Iterative Fitting, Regression and Squared-error Cost Function\n", "\n", @@ -2477,8 +2577,10 @@ }, { "cell_type": "markdown", - "id": "af260098", - "metadata": {}, + "id": "e225cab9", + "metadata": { + "editable": true + }, "source": [ "## Squared-Error Example and Iterative Fitting\n", "\n", @@ -2491,8 +2593,10 @@ }, { "cell_type": "markdown", - "id": "f1a85e34", - "metadata": {}, + "id": "86833101", + "metadata": { + "editable": true + }, "source": [ "$$\n", "(\\beta_m,\\gamma_m) = \\mathrm{argmin}_{\\beta,\\lambda}\\hspace{0.1cm} \\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta b(x;\\gamma))^2=\\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta(1+\\gamma x_i))^2.\n", @@ -2501,8 +2605,10 @@ }, { "cell_type": "markdown", - "id": "a809e54d", - "metadata": {}, + "id": "09b937d0", + "metadata": { + "editable": true + }, "source": [ "We start our iteration by simply setting $f_0(x)=0$. \n", "Taking the derivatives with respect to $\\beta$ and $\\gamma$ we obtain" @@ -2510,8 +2616,10 @@ }, { "cell_type": "markdown", - "id": "21b8fb9c", - "metadata": {}, + "id": "0aa5e94a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial {\\cal C}}{\\partial \\beta} = -2\\sum_{i}(1+\\gamma x_i)(y_i-\\beta(1+\\gamma x_i))=0,\n", @@ -2520,16 +2628,20 @@ }, { "cell_type": "markdown", - "id": "d40d3542", - "metadata": {}, + "id": "cef4a5df", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "30b388a2", - "metadata": {}, + "id": "a87160e1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial {\\cal C}}{\\partial \\gamma} =-2\\sum_{i}\\beta x_i(y_i-\\beta(1+\\gamma x_i))=0.\n", @@ -2538,16 +2650,20 @@ }, { "cell_type": "markdown", - "id": "8f5986c5", - "metadata": {}, + "id": "053210aa", + "metadata": { + "editable": true + }, "source": [ "We can then rewrite these equations as (defining $\\boldsymbol{w}=\\boldsymbol{e}+\\gamma \\boldsymbol{x})$ with $\\boldsymbol{e}$ being the unit vector)" ] }, { "cell_type": "markdown", - "id": "605abac6", - "metadata": {}, + "id": "20c89e86", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\gamma \\boldsymbol{w}^T(\\boldsymbol{y}-\\beta\\gamma \\boldsymbol{w})=0,\n", @@ -2556,16 +2672,20 @@ }, { "cell_type": "markdown", - "id": "f67ecd40", - "metadata": {}, + "id": "2da6a7a0", + "metadata": { + "editable": true + }, "source": [ "which gives us $\\beta = \\boldsymbol{w}^T\\boldsymbol{y}/(\\boldsymbol{w}^T\\boldsymbol{w})$. Similarly we have" ] }, { "cell_type": "markdown", - "id": "85880a97", - "metadata": {}, + "id": "e539d7b7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\beta\\gamma \\boldsymbol{x}^T(\\boldsymbol{y}-\\beta(1+\\gamma \\boldsymbol{x}))=0,\n", @@ -2574,8 +2694,10 @@ }, { "cell_type": "markdown", - "id": "e801e0c6", - "metadata": {}, + "id": "b9d48dde", + "metadata": { + "editable": true + }, "source": [ "which leads to $\\gamma =(\\boldsymbol{x}^T\\boldsymbol{y}-\\beta\\boldsymbol{x}^T\\boldsymbol{e})/(\\beta\\boldsymbol{x}^T\\boldsymbol{x})$. Inserting\n", "for $\\beta$ gives us an equation for $\\gamma$. This is a non-linear equation in the unknown $\\gamma$ and has to be solved numerically. \n", @@ -2586,8 +2708,10 @@ }, { "cell_type": "markdown", - "id": "05e0fc0e", - "metadata": {}, + "id": "d731116e", + "metadata": { + "editable": true + }, "source": [ "## Iterative Fitting, Classification and AdaBoost\n", "\n", @@ -2600,8 +2724,10 @@ }, { "cell_type": "markdown", - "id": "d1c5a166", - "metadata": {}, + "id": "243e0159", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{\\overline{err}}=\\frac{1}{n} \\sum_{i=0}^{n-1} I(y_i\\ne G(x_i)).\n", @@ -2610,8 +2736,10 @@ }, { "cell_type": "markdown", - "id": "4476c4c7", - "metadata": {}, + "id": "55a6a0d1", + "metadata": { + "editable": true + }, "source": [ "The iterative procedure starts with defining a weak classifier whose\n", "error rate is barely better than random guessing. The iterative\n", @@ -2624,8 +2752,10 @@ }, { "cell_type": "markdown", - "id": "8b35c8d2", - "metadata": {}, + "id": "1b2d75e9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f_M(x) = \\sum_{i=1}^M \\beta_m b(x;\\gamma_m),\n", @@ -2634,16 +2764,20 @@ }, { "cell_type": "markdown", - "id": "a533e45a", - "metadata": {}, + "id": "f6cee5c0", + "metadata": { + "editable": true + }, "source": [ "will be a function of" ] }, { "cell_type": "markdown", - "id": "b1e33a99", - "metadata": {}, + "id": "66050e09", + "metadata": { + "editable": true + }, "source": [ "$$\n", "G_M(x) = \\mathrm{sign} \\sum_{i=1}^M \\alpha_m G_m(x).\n", @@ -2652,8 +2786,10 @@ }, { "cell_type": "markdown", - "id": "4199e7a8", - "metadata": {}, + "id": "56a630c2", + "metadata": { + "editable": true + }, "source": [ "## Adaptive Boosting, AdaBoost\n", "\n", @@ -2662,8 +2798,10 @@ }, { "cell_type": "markdown", - "id": "db2fda6f", - "metadata": {}, + "id": "5333aca1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f_m(x) = f_{m-1}(x)+\\beta_mG_m(x).\n", @@ -2672,8 +2810,10 @@ }, { "cell_type": "markdown", - "id": "dbb7d266", - "metadata": {}, + "id": "58533db8", + "metadata": { + "editable": true + }, "source": [ "The simplest possible cost function which leads (also simple from a computational point of view) to the AdaBoost algorithm is the\n", "exponential cost/loss function defined as" @@ -2681,8 +2821,10 @@ }, { "cell_type": "markdown", - "id": "70431c63", - "metadata": {}, + "id": "c8f78e0d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(\\boldsymbol{y},\\boldsymbol{f}) = \\sum_{i=0}^{n-1}\\exp{(-y_i(f_{m-1}(x_i)+\\beta G(x_i))}.\n", @@ -2691,8 +2833,10 @@ }, { "cell_type": "markdown", - "id": "9e678e2c", - "metadata": {}, + "id": "80ed0e5b", + "metadata": { + "editable": true + }, "source": [ "We optimize $\\beta$ and $G$ for each value of $m=1:M$ as we did in the regression case.\n", "This is normally done in two steps. Let us however first rewrite the cost function as" @@ -2700,8 +2844,10 @@ }, { "cell_type": "markdown", - "id": "d2957ac5", - "metadata": {}, + "id": "2c203531", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(\\boldsymbol{y},\\boldsymbol{f}) = \\sum_{i=0}^{n-1}w_i^{m}\\exp{(-y_i\\beta G(x_i))},\n", @@ -2710,16 +2856,20 @@ }, { "cell_type": "markdown", - "id": "abebd5ea", - "metadata": {}, + "id": "5d0b9e28", + "metadata": { + "editable": true + }, "source": [ "where we have defined $w_i^m= \\exp{(-y_if_{m-1}(x_i))}$." ] }, { "cell_type": "markdown", - "id": "9d7d7c9f", - "metadata": {}, + "id": "2bd73caf", + "metadata": { + "editable": true + }, "source": [ "## Building up AdaBoost\n", "\n", @@ -2728,8 +2878,10 @@ }, { "cell_type": "markdown", - "id": "9fdaa6db", - "metadata": {}, + "id": "d1ca1da0", + "metadata": { + "editable": true + }, "source": [ "$$\n", "G_m(x) = \\mathrm{sign} \\sum_{i=0}^{n-1} w_i^m I(y_i \\ne G_(x_i)),\n", @@ -2738,8 +2890,10 @@ }, { "cell_type": "markdown", - "id": "f4cbd741", - "metadata": {}, + "id": "ac0baf63", + "metadata": { + "editable": true + }, "source": [ "which is the classifier that minimizes the weighted error rate in predicting $y$.\n", "\n", @@ -2748,8 +2902,10 @@ }, { "cell_type": "markdown", - "id": "8310198c", - "metadata": {}, + "id": "97d3112a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\exp{-(\\beta)}\\sum_{y_i=G(x_i)}w_i^m+\\exp{(\\beta)}\\sum_{y_i\\ne G(x_i)}w_i^m,\n", @@ -2758,16 +2914,20 @@ }, { "cell_type": "markdown", - "id": "b37fe48c", - "metadata": {}, + "id": "9ea22020", + "metadata": { + "editable": true + }, "source": [ "which can be rewritten as" ] }, { "cell_type": "markdown", - "id": "9e909739", - "metadata": {}, + "id": "5fae693c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "(\\exp{(\\beta)}-\\exp{-(\\beta)})\\sum_{i=0}^{n-1}w_i^mI(y_i\\ne G(x_i))+\\exp{(-\\beta)}\\sum_{i=0}^{n-1}w_i^m=0,\n", @@ -2776,16 +2936,20 @@ }, { "cell_type": "markdown", - "id": "d8ad59f0", - "metadata": {}, + "id": "e7721f66", + "metadata": { + "editable": true + }, "source": [ "which leads to" ] }, { "cell_type": "markdown", - "id": "ba2d48e0", - "metadata": {}, + "id": "beecad56", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\beta_m = \\frac{1}{2}\\log{\\frac{1-\\mathrm{\\overline{err}}}{\\mathrm{\\overline{err}}}},\n", @@ -2794,16 +2958,20 @@ }, { "cell_type": "markdown", - "id": "5e1dcb00", - "metadata": {}, + "id": "5aa189b2", + "metadata": { + "editable": true + }, "source": [ "where we have redefined the error as" ] }, { "cell_type": "markdown", - "id": "45c824f3", - "metadata": {}, + "id": "fa4335a9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{\\overline{err}}_m=\\frac{1}{n}\\frac{\\sum_{i=0}^{n-1}w_i^mI(y_i\\ne G(x_i)}{\\sum_{i=0}^{n-1}w_i^m},\n", @@ -2812,16 +2980,20 @@ }, { "cell_type": "markdown", - "id": "c916fad2", - "metadata": {}, + "id": "d0360603", + "metadata": { + "editable": true + }, "source": [ "which leads to an update of" ] }, { "cell_type": "markdown", - "id": "070d7540", - "metadata": {}, + "id": "50af7ce6", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f_m(x) = f_{m-1}(x) +\\beta_m G_m(x).\n", @@ -2830,16 +3002,20 @@ }, { "cell_type": "markdown", - "id": "4b1b22a8", - "metadata": {}, + "id": "f0fafe85", + "metadata": { + "editable": true + }, "source": [ "This leads to the new weights" ] }, { "cell_type": "markdown", - "id": "cc2bf04a", - "metadata": {}, + "id": "47a5a700", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_i^{m+1} = w_i^m \\exp{(-y_i\\beta_m G_m(x_i))}\n", @@ -2848,8 +3024,10 @@ }, { "cell_type": "markdown", - "id": "ea775ce7", - "metadata": {}, + "id": "a98e7bee", + "metadata": { + "editable": true + }, "source": [ "## Adaptive boosting: AdaBoost, Basic Algorithm\n", "\n", @@ -2866,8 +3044,10 @@ }, { "cell_type": "markdown", - "id": "ae10c0a4", - "metadata": {}, + "id": "f7884f46", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{err}=\\frac{1}{n}\\sum_{i=0}^{n-1}I(y_i\\ne G(x_i)),\n", @@ -2876,16 +3056,20 @@ }, { "cell_type": "markdown", - "id": "dfe87ab2", - "metadata": {}, + "id": "a2ac5c4b", + "metadata": { + "editable": true + }, "source": [ "where the function $I()$ is one if we misclassify and zero if we classify correctly." ] }, { "cell_type": "markdown", - "id": "875649dd", - "metadata": {}, + "id": "8a029ef0", + "metadata": { + "editable": true + }, "source": [ "## Basic Steps of AdaBoost\n", "\n", @@ -2898,8 +3082,10 @@ }, { "cell_type": "markdown", - "id": "5aac54e6", - "metadata": {}, + "id": "29784fc8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{\\overline{err}}_m=\\frac{\\sum_{i=0}^{n-1}w_i^m I(y_i\\ne G(x_i))}{\\sum_{i=0}^{n-1}w_i},\n", @@ -2908,8 +3094,10 @@ }, { "cell_type": "markdown", - "id": "a3e84a60", - "metadata": {}, + "id": "5ba646f1", + "metadata": { + "editable": true + }, "source": [ "1. Then we start looping over all attempts at classifying, namely we start an iterative process for $m=1:M$, where $M$ is the final number of classifications. Our given classifier could for example be a plain decision tree.\n", "\n", @@ -2934,8 +3122,10 @@ }, { "cell_type": "markdown", - "id": "d80edac5", - "metadata": {}, + "id": "5e96905a", + "metadata": { + "editable": true + }, "source": [ "## AdaBoost Examples\n", "\n", @@ -2945,8 +3135,11 @@ { "cell_type": "code", "execution_count": 24, - "id": "04a2ab5b", - "metadata": {}, + "id": "82cfb9f3", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.ensemble import AdaBoostClassifier\n", @@ -2973,25 +3166,7 @@ ] } ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.10" - } - }, + "metadata": {}, "nbformat": 4, "nbformat_minor": 5 } diff --git a/doc/src/week44/week44.do.txt b/doc/src/week44/week44.do.txt index 2b93ae0ab..39b7256ea 100644 --- a/doc/src/week44/week44.do.txt +++ b/doc/src/week44/week44.do.txt @@ -10,6 +10,7 @@ DATE: today * "Video of lecture":"https://youtu.be/7jexGH5SOOE" * _Note_: Thursday's lecture is digital only due to "High-school post-education day":"https://www.uio.no/om/samarbeid/skole/fagped-dag/" * Friday: Decision trees and ensemble models (bagging and random forests) + * "Video of lecture":"https://youtu.be/9QcU8VcXxRU" !bblock Videos o "Video on Decision trees":"https://www.youtube.com/watch?v=RmajweUFKvM&ab_channel=Simplilearn"