From 18f4b3687400d18d86017db7f48de9aba8695ab0 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Fri, 4 Nov 2022 08:25:55 +0100 Subject: [PATCH] update --- .../ipynb/Results/FigureFiles/baggingboot.png | Bin 27457 -> 25308 bytes doc/pub/week44/ipynb/week44.ipynb | 873 +++++++----------- 2 files changed, 350 insertions(+), 523 deletions(-) diff --git a/doc/pub/week44/ipynb/Results/FigureFiles/baggingboot.png b/doc/pub/week44/ipynb/Results/FigureFiles/baggingboot.png index 72fdc991dd966cc6dabab7ac7ea43aa5a79c2ba5..71702772afe8a2e580461e25e663e6c1a3209268 100644 GIT binary patch literal 25308 zcmeFZcTiMO^EP;A28J9ZNd`qhvP8*I5hO>EtSCWpM#2CR1;Kz|AcN!}L6A6rfQl%R zb5wH9neSZm{pzjVUv1UaZtY)dVWIclIrp4CeY&51x_ctEG;WX((i1`uL~>I_Ne6W!JN?kX}q z_mdysXj3VZQ&3Q7f7eo~hLpe`Dl=jX{97;#5f>2=!PzPRQGy3r=+JWP?Cj}u&5#QG z)Ll9#65O^TMzO*lkG6y^g4ozYv(Vay4Z7x_?pm=1l@hN=jCIh0G_4U|R}V zquA~2?BZU(CJ1F0U2LL*BvUgpacGdd zyKjf9O*QFRGr8`~Zb~IVl%+!uuS(tBzn4S;kMQfZc#_6D2@gWix4d1-X$D=3+jE0u zmxbxoGoMh#pJ|iv#I~=tkr95DPUQFRw`_#z`*@TE!IAp;cAlIlXpN)YOgh?h4u6%GD~`;m;ByB66sWp7d0# zt@d7BVfsHSadqxBw;4HdoDrYdB=wn5kA72f9E97XTJn_a zD>8kuwDi#D6N6(Ff-iw+e}Df=M@P$Rp$A%8T9|L&zG3JS_MJo3ry&n!9w0e)Bl_ShInZ-nnsD9iRiVb~9Z+@-#mO4W2=2($+u*$>l$ z)rg0WPFXH16x$ve?k)|MeYp3Pl2Ou=IGA-Jzqr1s+GDZAVOSKxsAe_YajFXii<|B> zG;hJBHNkwi+cx$7tz%P(Wi!%jF$j+tdm&beQov!lHHJ>i(d5xMjWUF?z^Bq5aQ4Y0 zk_U2#?&;ApaqAHI^7CgnXH?b98~eC7Z|FrW2s{WiW{_MUnD zVC`VBxSd29nm9RJCG#eQ>U_5en`=_FtRUi`biy%)=MkI+pL62}PTL|WhCLs3BuUbG zjr)s17Jt6Q%P0JrcTW_sL|(d;h{;k*nEE*E*j#GejzpT<*)pEB0Bi*I0 zc*cjMpa_j*#b1^nVX%F1Nj9dY@rhDB5aYClMB;X(@ zUjzD$!Xy1MO`@--XK}DKpD5|Y7e>Mm_V_W*(pVkCPpM7fqMDUzjJ3U;oin(PvGuKE zzMk>HZ5#@v0)k;IE(qm}BFACtQf33k&cKbW!$QEKCr8_RGL+W&_ZNyP6F#6LmM2I1 z-g9Z;$XBM-?p&|QF^C=#-$O5>j z>SzWr(kC>$cAfz#GX85*Px!+MrKqI6`6tNN`XOfSO697Huz}oNt{i8Vk&)?YNyU%I zqnFF?jZ%I>!zLtjP9ll)!u4n@P~0EYW8j>Z*!*ITW|E|djEn^Kz^QKApsd|#uoO4r zMr@c_@eQC0Gv8l_bai!kCe&zNfh}Nq(Pds<65qOigG1sDL&CpjXYUS{S`XCv2wi{5 z7zqx&RmU3<-XgEPHOtz~ACjFm!c_+g`FBh_4|kRmrF^b}KWo2kuuNtpm^HoPy(18C zY7eBYPO@r4rzgK_Tbo`1FCw+!$4g7m; zK05`TU*a{lOAsqRKbf+!a@Fa{q2S$CV$S$)sf%VVE!3Rc+$acERQvuS2T#e}@jzKL zDPsiq&2lAvYX_|3i5h<$l~euxB^n;1&Y9B7^SuHsxt2w4lSazC=xu?I`yxOio{%eJ zsngT~dVJ-`hssq00%T@7-iDjjcv|kRj8+}*3_pAEA_SNXvD*e|S`++%`$XU!ZHxx}Ioc@1Kg4PX=-aYlh%A%uYScQ}2N77_JE7yD>;rDdWq1QmQ9B4!Z1 zmCO?5Ane!CIz0bH)KxbtFm8!3EFvQPD@TF}GEKg4a_(1#%Uma$x0E{n?V)N9ZrDbd z)%miqu^}4vJDjw%wS$I?=WCWvj~hF>9z83!w-u=s2Amr z*F6#&aTH4qJn`A@$_qT%Oi?%E=jJBz8uPilQoE_abGsC$xNe6YWkE|Nl8w>T(FyLu z>Khm!@y+b}lO3Q&`*Ymf+&`;hIMiNYABwiePEA?)FvIqUlb)KEcDnv>rTM4qX>^5s z|J0uZ^VYiEvEh540*((Fmp!N93%@dMtnUmv!H$c>=>D&C<-Og8(-@$Fey7K~drM_M z1-#cLoa?9w({czfis7U!K`6Wl&l=SG)xk16NZ{U=I8vwPQ~HzshqXy%4vP7`O*X=4 zsiT8T6+}sCA3<41?_?og zQ}$RmjQD&oSUW6RGNab%b&JWSP&z4ZflKOd9=rXxcEgcX!So61fg~!bq~opF^g@1d zE3jq);OzPw{Qml4y<1aua&LXQ!xPorxiVTK==xjb=y)}-?cFuEH{V%A?5G}!G6d{R z;_YvHu8c&2#ZSi@nl&bQjSVhby>sR!5z-1%tkEq$yI4IXenXO4!dBK;DC#2#!fuCFzd-JS2s$|)%LoGKqu z;<;=*;@+*kag{k15B$V%^D*qQRJs09Eim~+2_auGi8(TgIt)Dq?%(R$ORmnv?W&z2 z`*pD63i-XBIY<7wBm+L>l%fB8V7g(?JLF0fy&+q}&c3!(#XMBR)ORkhZa-LsHWoNj zOg5(GQZmxjeeFTit#zUhXb7}cPHOmPUSEs6u&emZ9hL`SHvyun$Ff?cmo8~AVUHlie!5h{Fk zg#?buo&cAbwvZ_Grkx(R(1lhUd2BdK0NuQm79Mz=f=9D1wrlhTjJ-K^V8_5+Tzhq{ zwz%Ph9-z3x)dr`2`xJoKUtHE)QCa6C1a4cQ*&gyYZDcGiZ(Lr#o(DWCRWoe1f65#Z z0~4V>Q@20!mQK(Lhp#HtL5#;)A_>AUOZ&R-{kTo#LySr5OhuPuYvUEFA`)(V{<=_& zPZI@#$li*6vnN|y?zO;1Bf1?#8OHo~O-%xiw_!g7!WEJ_v)DAnaI58^3Pk%;;OJqu zw^FkzlllI=Rnubv=uI**U?K=lzNtyWAraQ6Y9-D_3u?!zm!7 za?F?8B58N*5t&Bzxyn&b2EN)gg@+?m6tKMq5aj7#*#p=UA&Ot5q>#m{h8EW!5C$F& zj|V$)16vNz2F=ducr38q1h9nz;Y29L`YGeJU^3Ywat1NS&#f^GAN&tJ3yeM}BsT!) zEp@!r?Y;Hu#)rVufP&j)==0~#Lz$g#5)$x$0|Z8n(L7+EgBXnhw&G&Hs?KmOyD7#o zBab{nk_FR4{k^eUgp(&?cbw^*j$*j8=PSaZ-67Z}+p|BcTZpyhsB0^8rK}CLX=|g+nY<5eNajD?~xx z44ghVkR>vFbV?dL<@HPaq}dm zc~Z{L&kt*F;{Hqd!RQZg2x4i?{V2l8Si*YkLWh^be%TNkz1K{Uaos~%DtV&Lwmm$=`aGxh+os2m0+?U}(ckzEv+RvzIWZP?eXaBEP9n~JSU z90n(qFMmt~#&4zm&=Iyhfj~eo?pdF1n;v|S(n7>vi-F${oQtxbpA-$RanJ9UcGGeL z17JT~y4A%d*V|ayK1ZIjy!yJtaa2O;^k@^&p4+>CnGI&eF2>WVwzgCpB?C=|6%YoT z>N8ncSrt`$BC)Wb7Euu5mXG5_uatc}U4?0ikAx(q+#+41F964Y{7J+i!|DG5W&g_fN z!NHLz^netaAF2m_k4n;W2@5={a~qS_1USPE0mmoDhtt55!I3t6C>oCb@DG$&D!w&S z%#>1Hs2fii8-b+OTN#`n$nn!38elP|suzozflz~hBY!wv&jd~>zs5W9 zUo%%g&5@%a<~<;M>+By}*CQ8f`vB;Yco z1P30{zFXMse)I8}{`!^b#kHF8<1-U=J(@0vNz!uy7$b+xMfgK|z~9EgUS(~fF;Vbt zm|sk)S#jHJO5jt#5ZrnzN)m!c)(Qe~Z9~I0`bJXn&wU@cvM=$_5Z$_T?mO8fY>qxe zmV)k)fUq9i4Nb8vz1|*uxQW7<4<4(jDk&{nR}KRLC8FN{@QnTt?@{GF9{s)QRCevY z;zfH%QQiS!MZ4oCwb#Qz8K)}j1)l3=jg_!Pj8IlZpD}*vL?{c+i!&;Up13fjWC}uY z&Wqd6!W!+M^Jky>2Kxab5e#*OMxsNJg+1^z6v}{2;=H(zvD|Sygqe=!J z0x+O#WyStd)jIPD6`+$8<k0LN8aSzrstd{s^2AgkeN-9dKN_^es^A z>|8!SR0Vj9eC`o0=HL->)6*HO`YkIyOiy)ncTc*0urx2%&F?MPmn1@CwQeqOlZ0&J z8W|92Q~7+ufIvp~N<1-54{r&DO3uYYq{Sa?A05w9WchQnkOP+=B?@FUwMsl>iFg>laqoW8)}57P5+nPyeE{wmJ; z`udkIU$6l00+#D(XWcVa)5fiJT&!#Uxfuz=xsS?VHa*w1siAE3w_!&|emr+R5(D(x z`sLXLLARM(z;(ZG*#_ga#KkXnxxCEuI@@NMT;>ty4Wg2LnOZ`EvPu1>ayAXqDg=c;&OKD0jpW3~6D5YTsAT z`2DpI0`l?k;k3DZM_Po%2xm-Q8>28eJ71M z*wX_!4wS&`Vt2MCSz-VnAJa24l}KGW3LpKrHv46g4AQ$o4IE4;U=Gm09pgZpRD|Fs z@z#6HHsGPD2yxvdOl`)ZO;FxE7Osx;f0gdblQ&S6*QyyYvlQJGeT%H$fL8py&s(@? z&NomEm;%eJfEmFu-Q)Rx`bv_RGs<90IU*hvZzt zH356vIi1oo|swTr209B2E0Az%tz|MxiQ=0ZUXYmPV@!zvF zPdN61BL^WItUgiCD@2$(BsZ=%M)DYF96LXVz7g}UY?oj6zw$@{SVwAi#Lau@LtBZ* zqJbZPZ{mPQ5CDPF5ePILT;M1<{Oi5vEPz#Rq}8)U=sMZfmf9K$gj!dSGxQL{iJ4H{ z9vtk;moHN*{Nm%|A@cID;=)B{ac4*^fv*Kb*iUb;_)`I9wYfN8o7?Yby-nrTt$OWWjWm#UoeoZH0r}t!$83eScStY~E ztqZ;Rf_6Qe_wL_EVx)jie1JV54M&X!o=U?RjJ!(Samp%fLjho_yl~FaPwDuPZ?3a~ zXa$ZEVcrFBn=rTsJXH%&*WliK!*+k@2cRqf1uyRU8y^4{2y&bK(E<*TAc!gfnnupg z&l^?PQ3sstKX^D;syx;!idN>5I(fnU)%;L;Ar6S(@RS!L(n8mAnt{6+t+q)DQ0h} zWXyFc5PRU_;=)G^FviRS4@INt1d*^!0G$4xKS1kcj<HY2{~p z)IL-?tpIKNi^IMDxD1`F3qZtl@Sr#dm4yp2hRpGovp?I%rRvI41x?3){PHpe3~faN zhE}s1B3B~Efg?4rE0)mG!>@mR!mm(7NtsAL^{qSvr1j-TDh!CegM`@~H`B%vewcEO zA_ptKl!BB&d)ICJHc3XpL*T3(?QK?uU}z5{c};kcanAhzkKtr zF0TPk7JmN_dGCx^|NB2Xh#?tCqbxWAyG(ffbfh=^Gc8Tyx7NO@_9?PK2H_0WB2WLV zo)o;xhD6SGdQn}J_e+s86}to;WcR{#gKcnLF*>gQY`4w^6avm}vlpT< z9GY-`)MR=E7NR_Oe5x2x{u1J=k-Ek!{{*Ly(2RmJuS7ZLOb%+`4azA<8%iib&f}TX zZE_qlJlN@5g5~iALl<|SwQ@6v;XErsNj1GXn<*NM*fYXAsdFrX`I}ML2V-zu^EStv z&|@?HNa4aRF9`%V@7~xGyj!VZP|2lE=M1X>c=IfC-JJWH9xKDOPL6}M^2$i>J0J8{ zmn|XU%Yh^o9uyOWtDBnH2I+=YjL_fNoQ-y?4iQSfCXQ(Tin+bvfjQ%5BP7lPIdi>~ zv-&7JR>c-k%&|sshS(PFfoZ-PF;OHBBw^J=z(St|1VZH*WI>mtH1K0x|2}8}9_%xD z&{=N}#RAE>KWhFfIX3wEEkgl-rl&+yy=&Q@8V<2*3BW?72DdJOyatNi{eTA_dGP+j z8CLUj4K>%ZMool2}tAth6Vr&dSLV96AB9pTe`B;1OahlQ(>H$nF%Ky zKy*p?XV^B`k7VN4rLy0P(BK(kCJ}k13)jYGsN8lBaHAirS_p|5#mWC+naOQmnB5nv zNp7CRg3wSKI&M)G+Yj;%bEHL(Ar{#)qN+5ipeu zHuWIN8ql4-Xy-oU8*d*2M?0x{alNM({uICo?(`oNA_(X1_v(9F-I{+o@&50^Cl3C@ zFBf7bKrFilsLZ&xZ@D#3dy4~lBUg(k{I8}8Aw0`^*xXxvHh4~XC!fA7J+qGzHvqRd zR$_?)@iE{Xm<5gj*9r3eH{~LTKCWDweRlz*EdcG5C}K}jzdyt94kR=ni?TsD1$q}` zOc0a&?9YR}a4&eFJ75-&ec>X=jDQr&lg1D{#Vo>tmyDth6QvrmfIr8*<27FI4`Ej8 z4@oO3D(D0(AdtbqfP4{1#>0#VOz$gh8yH_6q?LO+PdMWpO2QW_R%;E0O;02jL6oGY zBfozAs{Z~k8{{_Ky-fqvCjjdlBmHP+g5`7%PZT6%GJZvH>As&BKRxk!V(l)@SYpJ3 zi1oC8FEqz|ooT8^wW^Liio4leogLWAmX`o3K%l&#W~Rc^wAe6DyW2(>&ZWU@C7k#L zRG8AwWa;(K=QPW!V*a!zG~c`WHMXFjfF{978(1Sws^Ae8);mb}^gW1p10_CxPFen{ z82Dl+xNoh(V-zra5Eew@adC01{Mr4Bf`ICS*?rzuE?=K>2)yms$f&B?<5|ta1-2Pk zdjXD!Jc_`q{ztl>rhl8izpJ{j=-k#hWeP&&_L~2Y{zDz+C>uVsRM6^eNBpM08-W8h zB3mPg5ab|XIB($#a$r}5gaGKtq3M_QUbB!>v#cDB0kGvRNbZ7I1P9t$8axNWj#fi+ ze`@FBWPnF@mWN$Hdg$?~U|jFz zihdYujszSo@BMzExds@zFMmPNm3Trx;cisJfC!XAgoCu%U62bLD77XEI9@cL1}sG@ z*i5F9q}k+0q#`0BiQ+EoGDmBQFmO#2wx#^m-u}~<#6pFON=}Exwl?o|k%bWoHswjI zAo~Lqm^Uy3ny(c;@3vhRmxd~cU}VNyIl|6Az3x4+?#&vNE~TE|o1U5qAFV3z8UgEX zRB+J($+`Oxf$CKjPQ?+QxpFs**mSE=*;_$czDpK+><*^>t~s1LSA4t0fI4L zLeVEhCfq@mogG|-XoSFX$}$8#CZDF{75e+P?Ft;3e;&+=81pQ%s}Dd2PXP;qqGAqb zfVV(8Jdx-+JZs=lGOgcTCQ;kPyR{Ne_m3`*%&W$yiU#Jqnj5@OP1cY;d5}Tib{hHb z6UgU`I3VNzH#YmBf5C++J17_TJLGqgQ zq;dWT80bX^K-W=<667gpxre@L;(sG~^1DzpO~*U*3>U1%gd%9H4VxH_Z4ba}dMG0! zZK7C1Vy|4-xiNrEp>Z`72tt$?>M#ml!F}h*uVK)uoN56k(lZJMC)%f4laVVZp4Yts zqtsD~$7gPl6Ne7UP#$81kQVt+K^CISo9Rpflqiex&bgqM0bSl1-PrA2ocm{z@n8hg zxAms43L4Q1G!*!YJY2v!FW!w1zGo-^zbz7scr&aRUGkO7A{Sj!-Wo(-!~GYoIffww z-t1{25}0wM9DPc;G-C8ZXdo@4AI4`^nGP)Qc@M4DA|h6?n%^1q7Zs)fwnUwZkk!9L zcx+DCS@tg&VFOnkehg72o`GozFz^uAn<{gOzc9lFESVutZ9?b~>e8V-MAU9Sxg8bL z55&lB2g~tAK(a94;TT0WevjRB(c;DxA2EAZQ1nD=5*DYV1r#m1&V@+}7dG?NN-og3 z68B=t9>3{~Gs!`L1!|XN2OaYU`J73sf(1zB+gcKu&tZi8F>ern(K*xLp=YNO*w$Sf z4eP&VNF2PkXN{`FXnd+aP}8FC3^U#U6V2#xgX9SY@`K*!ZrnQ^mD9Mhl;ctL_V#Od zE`T)rrBaB}IZ&G>S;2pv)6dFbhLT$#g9C>gY48^P>q3=})uugQVA=zU4}awWM13L| z%sBb?k_Nb>>PWy?i~gr;j{xT31{iPfC#H=<^Pi361e1$TBmoe#m?%e_M{}nVhmWT? zw4YnFQWVzmGVrE&R%kv1aBtZ`WzV;@xNv$tNr^jrxN(MId*Z>UWzp~Z5qa^1_p;{@ zC1-f5f((d8Qcca`@2)KaqOPMWWEXcJ_OF;=S-x%%vT+l%5Ch@O{vG6|BYb#zNWrXm z_y`1cp11z&NkP9_MeBS!a~XCuIfNJhUqGRY0-#;oC757}*<-P6P zC=!iJNr|;?e`UEj(?NI5_2T^>?;=I)`^u~S1FJyA8Vsvg{I@7TrHTNsb;i)Q$wKS?J7(--Aa+E+Z;2x zfjdu#hFo>0*N@lj77q@9ebxVro32{;WOqk? za+|dRiS??v^d}03*A>)daQz_i#nq!PuGXDOm1S6>t&nwK0Y+YMSPf*w1>o5{|KYzq4p|b-ttfUU%K=R~6BE=Pp)uWEz zL6T1B{&y@$2o0Om{}&t*ZMa^8V@t5KN?cu1Kw}l6+YE2ZgL5JX0g!C5;Wf9m=74Yt zC+!!jL8^!SDYJAG(1Nt;YWm*ZURU1aQO6q|9wJiAnq|YcOYaeUzh8F5$;an-%&-)} z>V@-F<7v1K8z7Dv&p){!^@kNm$&u~9iKC$7q2>RSMEt`@AQ2im_^=gZi{L~nfSANA zGK`oWj$1l9I-ULN-F6iM^f8FolDIp_elOe%ZLYYa0zB*@oPo%30C|0lBnfJ85?K7! zm6~2tK|H(+S&$Wi&q47YEA}+!stIEpdBHkIKR^vk;A@ZxuOLO9rlm8P` zHqSF2HXtJp1b8Y^ha@xULX4O=${>GIwziuA2&m~r(3#H z7(d4p2gEdIEnNatVrR>VIi+%LvBc`rx<4b%Aa(k2KJtMRV;G=ZWT_0N7aIJ6`ekGZ zV-rkU04d)hfyNeHO2Z>hz=<$RU7Ed5dkzQD&Ht?_aHR%mNPH`#d$~pTE$Kvm;30iz ztvyf}?OVE_^1SDIFm#U?@zkF!3H^id*VL4>y33aakBei+AL%p${Gf4aeaj8Cl)LMn z)N3tZQSRs}Ir9oajnlA+CtOtpf%L6gnnBvxD=)8-w_F9Y=R7vy0&urK zuLV~gt4{l1x)hP?JrRkT<1dPsxu||3ELB2C1v*R!+^`Thc|9| zFsbG(i(3~df`9RNILbjdDh zyJr{#ICkc-;{V#s_`FE1?dUO)=@j@vTNFX61ocA;(HTOdeEQ+Zs3>td9X;oce=SW)Zlagur4Y_ge z(Rs|-cBO@Lk-omf`XFVw_1|uAQs^f1T+n<$_{*pf8ZHw}VY!jWLtsyNytZ(}4Bx zT1q!2jkn9rK20^c5`QE#{`0NEGnOhQ( zIrEif+VF2K<-a(%35==v<`aUs(V0`F2-~?A=}umB`6Uu1r!T2?b`p$Aia$p;Z48yw zJH_Y(>4l(e35GDvoa)_dy{Ync!NRr$`CXW0j%>3`bkK=ebX10FcaooL< z)=XOoFlGQH{_3W}v!;{DA7!3j^`48S4p*~ZYd_CT2;lmLcbX2N7FXs^maL}TPnqPY z#8*2S3WU`$s=0Et*CTO~=v&39Da*UoA4cc1&+dZVB4=oWjN$~dMIl(`%4wV&W%t>xlIe)<5*O@-Id4|#SW~KXC)qK0 zX98^nshHi8qY;x%V*a|UlXwSnRdny9qwd67U{xM9VUB(mp3}5(u5mU)_IFS5(FdzE zDuq;?&6$etfv!gCIB4FtVtl?u4`B?c*bc@jw@&=KmA-TiHcKDg#EA3s0{)iLY#UPwqrGdmpt$5E;}Q7Nq4CKHstf z3w;)!nGkGN%6z*X+b_vQDp7HD&et)PHcNz*i~K;r`DO)EHm{C~jDyT06)K$6Z0+zR zD(QpF&le<>W8eMu=32^{yW?TEs}xTHbfk?oynVfQ(p7I5spznpDgMdR`Bnqf#e=86 z!j!a?T&$UP;uBHGaoZ=-bE3K=#_tkrmB3Sa!zP^hE~z66a<|oopX$m7E62oX<;lpoxbaFw?&#uO-b&+~~ z=LdVifFHRBU#{p{-sV;3<&s+_f&$dW`fL1QMW6qyr~mj9G&&yyj&)M|1=-!${%g`D z?bBxCoxHr4!>`%ecs-i5*{f~ze$exV0BO;gld4DP{K${3d0P3#NiQKX?tbTiLbSUC zzmT>>U4OS&p-}O$-P4ZwBC=2s(wD|doP`9$kq>xw;_xoYW@v`N4|FT+#<^S zdhkbGP3Z%lTJ{1@F`jxBgBl4H{9&bhQ; z^zL!Y|Iz(~r^?fTt^`%x;+5GHv-eeFH?PJ3>n?H!UDJLcywNpZmo$94dtvtW+rjq= zaeSd#+|-?%)EiZ&cyA2?_&PSD*fk8)vX}21Smx;A&&AWt58K78Tr8Eo0}eGJaT%Xx%a+AC;3&XuH}@EG}!0ls`5a)w}JYeqHnu- z`xRa$X6YuJydem52<@1+-3ciyv(+b@>rtKSA^dFbm%ZxVHBTC{>TaeG((QvDFE*Yx zbUJpcGd?dI;ZwVv9W#=AK+j=FG%XR|WN4p7^mW_%nUzPCw*KPd;nS?C!_swcru6?Zpk6Do5bgs+n@`qXOJ}U**;jmadv!9 z5%P#$QiK)goZtcaod7y_I4A>J8AqmCRP5wr}=-W<3oL?Uq1NWb3C2 z7UPaWXEF!~TtFOmlYlLu-C+s#Ot?Y%I0j`E!+)`F0Pg3CJzl8wbP7krAxae4eWAh{ zT=6Q3#41KQf5UKd4JzfM$|`z>K&jB!-Io*5Eo?v4y7C2WRme;vdCn#Le(!#;ec3ai z&G0R&>08esub81+J9FNU!LdC1xAZ#r7_*FEjY6e&*Mr0>C|&HbRzC^zk@-(cj30$r zYUDo;lc=DSh^I8JKuYKzOBEU)Z2uto9NDxAyrDVcm9~WNZ!n5@$4sCzuxP7v>aLh@ zGU+A?ynT5qTWC|1F_nJLlu>9;>4A|CuN|Y~MH~IO6biZx^u3F$-I-55{p3N*p|PZq zbBL`$UF_^nX!a+v=+61BgdJbyCBD3-))dte+n>3ZKHpq4$tAM+Nc-^Z&tGJ|G97r> zxK+DU^(@rvWTG41+~G5c>Z*tGEDMfI$mczS|~XL6!c`Ge2jPO3H>-)wN&e+a;AqT)L=&<3o(~My zLkPw*9b%Q!f<6*reJ^!o!4Vc89G!H}Ws&1U3@i7%U=9>+<{EA6{fSuco|rS3Sk9F* z^~h9%XQUmCD3P3V<63(g3ndY7j^X|`2Ng0ms?XB0S>WYQ3Z7x#sq3_LX756QDa47I z*Z%n#e1$|881?n{Rtgup=FEu6$bljNWcnwZ0911~Sr? zMq^hv1@Gh$+>)g&Nq(h)mvAK3b(T8^MRNnD9QPnM8T3_tXyHoGkvT$HlKpSq6$uvb zTIT%T@+sn40)muiZAW-w?UhObC{>SO2WzP5z66zx;T>o~BOcf{Z!yQGwDW)s$E8%B zul^&wijpmOR%E*4d%^Tdx}zGo@|fJmy8i~&nNPZ}R-QAGWf?e4b6>1lG(cRq53hyl zreii$b>34%ZyThJjFZ7vVZzUumhc*8rgc8JoVv#_v8Q2cqzmdaFCpeUWI7iWQu~#= zItR+!+LZ;<)cgazDD^XmAhCLaqZGuRrO4W35~iEbL`t@cBg5uGl$-g-1DbHQ!#)@%9 zIg8>~M>MFc&Qlm)OLRJ-v$%$ckhdGe!Hw%NJYWF27}~Vu7WDE2qcd9XI&dZETusna z5_a7b`*k(5j1n=dkCYBa*hy<`gp|h;$|f)ew3^8k6J^Q`9I~g*;>+3Fn7@Xwi_cav z9A&tc$u@6dT`2`{kANcK z4=&T!LB$QsJ!#79Uj$Wm{QP%67_Duz^G}0ZR}09IT2>hUs&q2{{zildt`r6(Xm7s_ za|FtJ*I^qHpc=FI8DwRqbW4io@-Y zxLG3K_)wh-C9iU}b;varF2onQ2W~EM>dAeB#>efPkH-sK;%Jpqry|@Rq+6LUo869E z$#IXA>*Hiyv|>G^WZm?WJICL#>6_`HvGl5G3T#EG2x&nuQiXvhv}vr~xT(BRCvN97 zQIJ*)-}7>2_@7MS!uL39)py2j@GeU#qT@QtpLGFYs`O~gTL9Z%eQ~bWCaQxi>nu|_<=Gh&?$`z zx7GnVc>>f*w@I$nhJb3tPlIr|BVY9#yPl!p-Ige76-~`HP>;0^3fu3tzv88n^yGmo zR>$gKRtpedf{r8Nbi`;-Y)A`_$XB;r7ZZUU+*{NHl)7(p$oDjb0M~>XZWZo6l5BOiO zp;W#jp1UrDALz+#sSIx%ggcJ!(5t^lsX?X;UkNHj$qoM0NhG{(Pe@F6V(F*4xRcGP z%GPfiU;mp?q~68;SU8*7x%X!Kr-vl1io8cr#I4+;uZMrW;S~@S``aMZQh7iE1G0;`0M+ z!c|DCd{W0foLYSQg0pKSKV1NS?~us*KD7$>SGwx<=R+u7>O=@_BX8AZ8DH0R;uyrLTtD4!7RXq$=-1{L z65f5KcS3~NZ)c&5)--KnIhJRBtEP?XnBk-T9&>zG!}?QlDiWmSyREfw#R|SXsX_|6 zUAY>uLFtEe ze$dm1fB;?F98MPAXn`#h4>g!(RSr8XJ#1`nqCB3i zPh-pYGo+^dQlUYXc<4$d{k?0f2Q2#A(?5*o6A3)`y8xeJPD)SG4p-JhW5hQ%EF2~np*sa?li zwygenTcK&1>oM7)i8GJJUd35esx7^=3!`mK zrXLmcskOyj8WStR6Rx*i*t@=YH0{#0X(N?IOJEkOmv7=LCt;o6qiR_^yS5ha6aExS5!BbobHE<7tOnecgHbp2QYh89kTnwt^22EUJWd zo(DuL1#^nxSCH8AF00!MjCyQ~ygQ-S3?;1E_~7YOf5$J-OA9pl4c2(^qsf>V$xk2) z(3rAuG=rP~HPtTre-c2l#~iM*0Yw9F(;VasQdt%GOQu6#e7a~S!z6>GPhT>tlEeur zRWoPz<8}#bzw0!8w^E|o&=Y%DL?-c}=D7b|?sUlG&s6ho-VV8%V>f%}20I=EO2^0_ zIN$pIN`_TreVc?1C##rEF=YLI_I|(iUU1^|gR6RW48+HcMcMt--d&ca zcQp#yNXbgIlwmt|5bnT{JQemZdAQo{ti_piNAuDwGNdTd-8Qp~Za#$FG|Kl%jJvdj z!u7>|3gujyVHUz=n)z{Mxi6bZ%=#@J9YSKk6OS~F=QrOp@!26Ql$CK^-Q3>Z(u6yW zE6a-lKYc7Zgqu)bKS*u_iHSeh0mkrVM0iI1(5Mwbcj!iv!$5IX=hn7))}7W~eFVq2 zTI{HkN_)<>T&(4GkiaH#9@QYiA~kbvdsK>Q6-Q&$!8c0tZXDmlj}24&6&Dr64+Cx4 z@_1CmiRGs)qCtHky+eB8^6Mr#{#do|J24M>W7{|c1(di88~d5Qgg)aV(4M7ye86(` z=Cx;hBp$k;QU*~z08KxP5gIC60R`~Rpjd_-G#rDD$b;MMm45rKQYU-U4@`^H7G%wQ zBhuaT3V%Q1B2GTyhf#mO&AP_?WVQ(6b@L+1a%R7aD^%#wV)NL#X(NA^sP)b4$WnjW z$Jr_UAL*}azqBJkDOiQ9SKH;@7hql5Che%u0ub5M6}_T<>scyfE!_j#+hkq^Rx!wM)j+Ez49@madEi<P(Kn7WSiDTm}S~M?8o9W{PtpZgp)8IR1;3hzz&y{)6Ame0Q_))*zlbbNK z$MDY3HLSCFN;GqVJFmglz(sqapGz-{>+NFXR-e@m)oFKe)jW0Y0i8-pY}6u!7%``@ zXz*@w^v7+m`D3p=6ZZe^)<&TC=(F23hzU zSpeL_0VeHL^LVY#CY<%YaN{ZKNDSg2V`{8|` z{u3|KT2y{}R8818ZjvhPJY12dO$^FIpoIR=-9R6f;*-5ASC-=;%xUsIzu8Gj!m}K; z=!a)jB*W8O69SaV6e%?RQMoMYEh(PI1o1nnzvF4@it^>|NjKeg<8e|y0EgKSp| z2mHBP&PX3h^y_*cy%Df#vbocI+EXibX&~H+ViXHLqY; z@soD1gFw>?AH{kH5gy&X*mPjtcegh-|M#o9=B}{GUD^|7+IuFQ3;o*My?fo(feK{) zWVIi5NeYfEIW+o{TQ4L?A>)Gs`QVUV{8AM8fTtgWndBBd5FP?5Jy-f+u1UG->ftxf z-VrVwTpH6V{PXMQjq2^M$%t3U#FaCB*zZvGqI3 zuhsWPt{rKNh_7AGpTHkHf!9pdk#Lbd5)T`g(kAeqEmd^z3P`O#=j&i`ue%)_B<+Xgg}x`0zbeD0Y$z*3tkHIUZVMk zZp)Rki-X6yqE*8sMi{K5uf6Wevt8vjHFn-g&mt7L{JTTZX{@i1Hxu%G42CvBCCk~_ z75LnazTlmc8@wWAMvihyd;AQJt&-buDyLJ-k;AdolqzM>Uu>|`@bzc5n*Lw#&~nB~e&ajHgmWI8)3ZwwQ5=R0c< zrQt=TsU8PqU=_bBIxpPBQ@>Jadwi{-97{BI45#$4<%mIf-L#5wEis~(lP1ndGgVBB z5QMHs*Wbg#x(zz8lLYUx&QE*3jZ=?0UPYOtci6NTJ%wLC$TRVZumc>RFfS>o_jL z!o(?;$3G**Q_K+|y}=`PJT)U46HD*fJra15!ID~dT;KEtmOq;-A^xr5Zf`;O>}EmX z`T`NvS6Sf|u3qI4eO~EnH2Y^y-m%y$Yl6w_q6fE)f3eh;H?Ptz9mA)*p z@^Idd}kYRBJ2FDfqOTG+gLn2;Bsi|S`OoAcHf>2=?V?SrHAX|O^%T}P9IO5 z-jw2^mEj+~W8%B5Vk>XU^jb_?#WWdbQQkd%B~i+I?V>y8BxWwfSX>uQa?P)i&oyVA zlT+`?QKb-;9(MC{L4(7mgI>}uMNd8r|HejRx)u7{)68~d(cxcJDw7vK-1gPh2^ScT0rWf~=%-K;Z)kEj0xo^Jq2T`fph~DXV z@3bA>X<6S%(M5yP$+;ItqdU9|a_Y)n$P6s6M!h$-lDFOObIiC`)>JhJ@7eUlttUVs zXPxyV$zej*hlHPz;^k^Le}HQo&9F#4uvHCh97mD`Orr(Q zmOJrsZ;sy^z5ixGw6xMb|E<;V?M>}4Qi;ZHsu1)-O5P$*-xY(%w`d~eyvV9BbBXUF zYujv_l~4JZFMGB_3kC;XYMT8~qev;eO?u?~m~=bujNsH3Vu)DpR94PtqqAW-=p1un zdLP;rwwN>rT&`Vq&Xu(|NKArU$En!uv88}$au^)#mWL7(K!WCqufd$zZp;RUNs?6Jz0i*0!FqFZJtQ{q}rw^BJ*-zxE& zS>0O#kAO#0dG`sPbRsqCyZb|n3vmToi~Hj>?cL-8g2fl&B&SFHRL*>OXusS>i(_hL zQ;pihKg)xf4d`F_0iQyDk>UidA?@=aE>zN+rm>2q$?BbYso{oxySRx418pPaenio& z4Fw`wSd1AzHic$lFD-+S~S|O{z#xg(J?Me&Or+ zEtLL9;!&D7??!$_eHl(#?s|WE^W6TUyS=CELZ_|yT^H1;JS()K$N28&w4zPe@4tWI z`d~(CEm!cI=xX<}57UJR;yx>^m9JydMp)4amL;S1Ke28*V)+_m9xrF^%?Q(Y%uUatmYWTX5@LO)a z>~UUso&AF~yUZd(pAcEmFZ1}Qt+S54Va`#k(ZJ!=CY!iq^S$47w6e5@r&en};D^HT z@1$w*f1MV%NdW3XKd8xlTULxw5%56`ct@1kc$GOllmhdXUX)xk=0b_e+Qd)dexj6i~u*2L3zIn zVCdkf)bsK2k$caH7U#VQCI8Yo_t9Si3FWX@>`%iAputf|N!+NOaH!@SpToa)(7n2J z5$!Ro^!{9OW88p*LMq0JQyn|nZyd6@8P;cK?+&32jj(kM5HWNH+TX)RM{|jI09%C$ zk}*x6f;@pH2SeKc%{QZd4m2!DZ{OnAfhq!m2@{T789@G zvAyrE-TV-7uMS*p;r-rc2d`HnU16^a_0?nJU|U;TX?69Q7t0e9Cqu%_X1{0n`nWJE z9(g&vWs`RC9azxROKj;LTSAT2C5OmwUH+(`4k)qhP`p^}_lxUeaoJRpFfjDd9^z2% zsX%4%Pe6&S3;-qF1$qj5P7QHDD<#ZQT}F+kw9X5NJTFDO@&cBq!L1o8n$ z-)TJneFbhVfU>c`Brzd{=2O&PKqEDeFSfpfU9yl4WBnk6dWFX>AguxRazT-luI|j2 zb3m5`LrlJ_kxahpQYVZIMkmS%$v*|A`_KJ>11aJ-2oG9cykL@6ef1EidiVPd6#(=V z78iT=mDqjlZ`_RjFI-Oz1sf2(1R}17vt@N?*NyTx0QA&d_#AVh)Gsh(kOJk5Xe<0# zu7rstd}X%Z6J&f2puI%kA94ypoD+;a7z&-4LXV7jWcN=gqpf7jen>H#d0WK+Y}V5> zc^iKSK_e^?fJHw}^3V3OgVNU^)pb_nCS6(hq18r2b;&cMtU7eEYH-^0VBFi}Y0 zF4G()TK@`!bdJ`FTG72#mLf}p{2OB=*L*&Xgz+4n?|GGP`RuGs+`#3Jr~#A{{u)ND zTLIie77P46wQi&76AxVtC_k!x{p-zMGKl#v$OOXz`u{)pqXS!^hk^0yq=TtP#-&~< zzo>{UQDon9JA0Stswx%G578wr{xVgG(EqafB~yq2?Ut4G01y<^xZc5B?~JN~8ejc> zaPQQjmUm~~f{13@<<-H{NUMsHeqg~|p6TO4`XVs?)yckGa`4M}GjAhV?DqxW{E;l* zGCeT-j*X3_+_;ehLo2%Ok6%yj^Guh}voT(uwjk>h2!y685X~7HTkCTD#eo0 z(yQs|(clp+0QD9X@L7~c1Dn`5*aCtlt{%la`+8qNFtF*(Jtrf~7>JxeryAf$D7Rk@ zZ77gS@NjheeXFi>^%=+14^XDAR8&-yaO#2M&7ipfWh)3tLA&$Xlj@S*%=f?FT&?kd z989KJziQ5*UyEK;fpb?`M#ex~C#)WqOefCLf^wHMc#^Gd9f=1+#nKlC?GquKnQ~9p zi_hc7g6dFm^__n+4^vHK+lv=^1cD^+pPhp(bLa67T<`dSTPOS2K~ZiEh<{l*0|ZSc zI?+z$T2FzaN&ogW^RVaN@3&!$ci#6O)u-AjfOzoO4GjVCn1SM__n!En^DKdZfud?> z98yD?V|bcJn`42Nl(DRw&j;oOj7;IycMKsiM*{cMt;)&+5C|1)KXvdiWAow-DF25> zjnN;!JVPI^`UH9`@Jxb0x-4NrAp}>OnIW(mmXf~m;qRG@hR87 zJOCWgWW>#85M?GhEUe@l2h$*KeWCe3kBW0a!>=3^z+6ZPhr#SfW+Q)l>52qcm=V+_ z66ZI+eH@3dWJWHu$A?VKKn3J`ZAlb4h>3A9D^>}qeQ-H!t?oT5E*J7SuXGrf-a*6? z!Jk>}#i11K?{bKssb61uiAG=M$wddh+{smWO3!Kt7=w|`GatlejPL!8{;2t?tA)v+ zd7JR^SaI;e6sDMgLj*bd5QYk!aOSq~7I}Gj%nbRynysTFhaP`QW~S(*b)YL~K9IMp z?f~%#a*-e-sm0+HEYh8YXj1xCeb45q#nPt-y-uH&(b5vb_^TMZoZD5W?|gcf`Q`xx3paUef}%$Rc@hP8qk< zD&v4|s&BXxL-S{TR2{;Z51oa$HrVPw_nQ38(~f_`PSx;}KS1i}OhsG*1gvdrkjxc1 z+mMY!4Wi-5z}RgxCy1WIB>^ft#FS)eU)DqjE^~^x6VUVGxL|&WRzE{7ZI}ry=PRl} zg8t)z!&~#yL*=kiKp8zG8ObLzHeo*qYdd=4zQ=K3HgyN*U>t=)S*VMX1S|!&xe%-4 z)?H|#7nz_Fx8UH{@Mf;3pkNmajRBCdhgH`YC`peTIl=~P@8Ta9KaV1LAJPS#z8S)% znyclR+d7Etpe!5tiVNn7kL8I9o0@y r?0)iwFfS8+^2qqG{$eBxC+VS3`Wa#(59Dp5{*2hz z%3D>}t*=VbmOZ#~<(xZh*~fFQutfNhZNl50q$SIi&m*0(LBP99~GN*jM0r59xkdV(w&^AG|40N)VfXsf7EOA ztx=cZPl^(Q>*)(muw_IE4>wnI0+SV83l2IWK-@d&wet`1u@Cc830-tAjtgp{t zW@ctLHZ>}WopsgR2JPwLk%Ab2Jo~8QUWm=g9x-yvul~z1&y#|21WIUJ5rH)%I5Cr((XD|upEfo&vhn+?~I`G)YFJ#>$=GEy7EeG_7y>e-~$>p)o)IlPJafw{YgSh0L4( zF9Q}OvRWm04&+OlV?!zwr$SN%4ooxxPi8Lt})6~3<3J(u&i4lw} zwpyBHwpmIq_#71U$k9;%pMrb3KU@12Ev@;+v{*r(!$Lx+?f5&K;aLw)&yW*}K=uU0 z*JM5NOvYkHN^CexMg=@CQsGvX+HHD&{PA_-kDs8^8bP#sHeU5Di7A;<&P>?AWf771_oSW=p5%{w@zNr_{mXI8X?H#L_& z@K8B5U)i|is9T!Yff9wQHj#A>f`Jf7#Lggzum40+Os(y+OAZbIMYgo1=u9A75;kt{fV!5 z)FS~mTurjVxy6Ipld#&{O&;?%N9U^D99#Wv!N9ezvcJ6LNgJ@=U3mtkj4HENd-=}j z_cv_VE#Tt+455_*94X&;B!b7Oe0uuz zS%PaV-j{E_?MlS|`t|GK-ju&&G+!`S+JMmjhi;qc*8KAVy9W;*xO;d6uTliCCupOZ z@4IFaWL`9OtHZQ#;cMLZBc}88%H_*VbWBWXmEVZrxRa!!Z8uwaijEJK^m={}v6il3 zo*va+gYa+&m9Vowo_>$H!zU{K{BNW0zg+(Kywl>BM|Zb;bxnHc$;A{cF1FFY*bVQ)!ZRh79q>Z z?)^O_{P1WgU8ey_F!TYilek-(T8rr!OSfAGOkEZD{UW18!Le=$6IHVJrfA*Twamb& z`7LEy=FSXFczK-(UccTyR{zDKd#=Fc3afF zP)=@o`VS7PWuah&@87?lA1>nf@ZkeCOJqz;+ur(=UT6II)lRd}qI`=5uH zc$qmx%9Xsljw2QGW^vN!&A}j&T5r!hx4|Uie0zz4+fZ|3x`|V-(_J}N=kDvqOAr8U zMywRSPUYm}%#W7wCrQWZ4F1#;dHR&Y>A(^UsI@aeVgde+=IH6^SuXYZ)_4(Yws5p} zBtD~%m6cV>)xjqtBYSh;{P567vOJHgMp#MukN`bDR7@OA{v>395B5>Sr&*ijOxfGt zG~i63iU;P~%L75JkvzFBC&!Kj8BJW;^%tJ_Uu%uvGC&rIvM!0vHyXNfnh#A(aaqndTz#`1%O- z`t|FzWArMA?b^6(XvL&$zJ`Uq2Tm62JvYUQ0d=hpzjn&2Mfm_06Rh zjQ+?Hn7k*Dh%z4h`C+y-QaiV3HQ2Z-@!9C+W8YS_jJWJb9+R`>a=@UF)?Ex&%z zaB=ngF-s}t_MOdVT+}3GvtC(!Jz2F8TTQv*5cr4VMUQB{9xqDKXu)Uy*I# ziZ zLPB;A%bn1qY?>&@Ep)uRVIj;aId6Y|^Kq7tJqzo8^5h9|Vesm-Sl>$L`}{Q`8^hP} zOBaSyGs`17Iyy>c5bx+rz!N?>vS&J?Ta_N|43+NBMwXr&qp?{G@?P4Z{}c@T z)FWJ9UuROu|CFuWFe@l*zx5WJWpAZqwZG8prd*=rja#?kYhA?e`ZuVaO|)9-h1^R~ zGO}V#IS$8-@wVrOLZG%qDF2v;HXff8Tc{SPQu^avEf*J;!=s~*sTHsj2?rNV!p?6p3dR0sktPJ|)lRaRUDBExD|ViAwMTn>6)y$^33*7d+TH`T0q6pX#ZtX77@sraqO zO{ay4o%T0okc>8%XFy0!{%gc$?3uYa`@+Hk>TvAjh&r=$1Gi7r0-umpa(s8pB{OUH z#+zpCeALcDmlu)*!DVFS#R@KwM50kWdWd@8_B7f~Y5- zNUOFMy2u1<*@Eb#Y5Djf$T)PeoY2-GENYASF@5R|zJRKJrz*0>I3MUS5#vFc=A)B} zDxAOT`X=FJ0JRH6zmL~3n(*YbvNFC%GcEDcv|H-Qp~V`kpEM;G*dADY zJz(`B%Ud)-GrQ2bCStv?@q8e*a*4H3`4|O{an$N8uJG~RO*n9vZ**!#u@CLD2-7{& z<8KrsBn>qL!YzROY$rd^*#cPgi56>_yQ_l2c=+&P7`G8Qgg8daaDSl#Z}@v>>@X~h zbaHBnRZkGMnHgqtN9$M4=HxSG<=o9PL;led)Xml#BJGI?6bNLP1>Zc}&UBdpq&qnm z>oOZ{Gdi)|r_o+8{3cY*bHsQKUeH1U*~!uF;gZIQxr$Cp*j+eWFJy{~-SiM4o@~yx zGA^czl5He2yN=S*^!taJm_^62YuB-GQ;FZb7YJ1Zm(7?H4*>ZD?FPS3*90RPt~sPt zeiSY+==0+lwRNMHOYHcez$mmoO$g&o=YOi1&;&c>G#QN=EpzBXvIL<>y!m>V1BT+0 zH&(1wPlfhBJf`sPEPX+W_e8@j_F0*#NUhwwdS!6X3braI7e2vSXKQ-$wdv;Gn<*_y z*wi^@1UP(c2j4zZ&3D&xD=gkk$oGs}ZPbsPh zoHC7|5`*`C+#$M8paoUJ8%@S)ET;j>JHa-JE9|E*P5wE9>$O{aoUR{u@=)V{YNHR$ zmOnLpKt^Pc#n@si_?h}_SLgR@{wOd6lm0xwM!f@lfAbcDeX&OAlGCZ_q^Vogy z=kDJDp=`RAtYeFsLYG)$R}^g#SkoqipMsYkS707iLqt~dgFnv>Skp9i`u#JRrVU>iqQEyf699gV}+c=38e*GVa_E}zj!kY zC?AK6H1!u%{QYQmW8h8Yd_ywy&Y$MpVfeIYiS-|V`cB#E9Q^zgt^g+{S64M?@6`#w z%GzFKv+2ngygzc>GaI9y>{~uXyS9bLM`%JSs#XNMa|_?R=_W0F^hL$E$m7N%Pnjes zDo8Czx`Mc4t&(rhmmzR4{~fX1+YAgo5Smc?HWvYijxGW+}ypqEWoVIXIomC$aV@n_DP@q3|J(IdcGg)?_=K4*# z8Yr+-2i31fw$?**{kxpm+;*Xx{QQyV)rwfgjKcSj#dGw!Xc0hx1Rwxc!|HTGLf7Ck z5Ov#2R?Ah!iyuCEv_Jpdzdz><9=P~a+3x66E6M2Oqk^3v1OBgMl~ag>zX*VW7^uaPE8{YzELI2qOYd0&?J9*dk|=inH~)g{0u z=e&OXy6dN?3w*7Muf;>GY|?Kw^2~n@fy_k#cv5#D=M6wqd{-8=Vi|~XP>%+i?8Dao zF152zclg6qnTO4G`?iOM^M0Mty^nR(0GNtqLR4^7R{=`=0*v;{U-9jJOgs%D5H4UoH_yu84r-#t!!vWEjZ=2G5=UaKJp=d)r^DKM^kW#+W{MpsuZ z{UulV-Xxy=&a$$mAKpn(bS6zjgcy28sUGwN`8S~OvvQEyXevHzu!4$fTE91q zOi{y$i`seb#`qv^fqoAIHq9H3+2y!~uj zbaP&xx&$C+fVmSMBa8C_6NLebFEpLb1HDfk>5{CIa|1kdp(*7@K=G{4Oi za&mDfx#374CIp0sH)W`m_-yk83J_Djq!3yhzc znzCuve`+d_;Ns;iZ0ys3a@%Iqj*hi_j|TVy8UcYQ*x0p$rOXoMxuvD0oszm&RST9Q zi=1p(^SjfQd_|K!jC<2T(i?(`Q2o+0h)>9)7{g2W`*eTTe@`28_^dm7)YcLhy}|wA zqqpdOt`bvdDMha#Bw&6xPu^UeRvJJFIM-}ZaOOW0QVRj&kpb=>XHe$1UXHWdoUOAz zLg)jS|N5{+`%Ek2e9P1~;C{j?EAL++B*YuW9xius5IWkD`}X~NnYE3PT?fm!c)fle z+jc(pnZ1|$&bsWmebj77Ahk_~lUga9x)|fMqYMQ{3%C&8eVDjX&g=6Ay=hvI@ddUz z9)W%R?9Uuaetr{mpfKhJ=~q~eHsbFQ7W*Wk>VGi$(7a9A%Z2_5fgTmKoX0}I?A zcOyWN@>$*i9s-FDX)p791RZv=z#Ff*9GYFcCZM$3GWiWt>3okr?rc-cGmJg-w{%KA zr`~B1U{O<^_)#{c$R!~WH$|m@1%-I+pP_;z#Y}a}KZC&#GKuYFpFTG(-`6?b2(4Iy zNYI&~#t%+i1944exz7%)41olsP^E**&H3@KH1a=TyK~2TwH)mSA%*>I$MpyvlQtj? zcpY}q^3R35Z^LB4=4hue=fyX6*~M`lwDDOYf4aWz?Kg;k+>1VZk@5ne@K6M}h^)|@ zH8y%EEp5>3eYDpG!`PJ%4^o ztq*77Z*Lp2ZBqO|0v}pUz*K75S7I);SD`p`Ok)~zj;xP_>c9I zx{_23wq$op1`z_7^QM!-NTaz{Tz(_nz$++1Z?0%R@|0Tmv`~>6{PEb1=oxmFPu9MY zosp@EqZ1R;i2Wy47k)$QTYzeA>~iZsF-%JrIN5kKXEk9oU;DV=ed@!A-U~C?6y%%r zpM}HLW_0*8h{Y_EGn8-B$dU^Q3CX9vX#G=NTfFjPj{Sn$r%x9?fBu~OA{}{4=jnzh zgR4SIaz<#jZP)XBO0hSGB$4}bSvv;=!QP_uJ!#5>*E7%f%%#3ae>p15(zi_`j8vj7$8mG(Ujl^y ztnKmsKpJecienma`SH=Y^A!GJrvOI49aod6w^)joz((&}B_1BP8N1*RVu>`9=jBn& z)eBfUba%Q>tV|@Q<}XRbzuBBrL~MeFhQalXcFz*L`K6-OOA)7_PX8Tt#kJX?)~BZw z7Y(f(1`}SSke7B$iB-CqD%Z8bQ2j3zel&*7bTTDa!xdKr_IeF6nTD&;sk@g5XWI*079QHh8GmM`p&7ls zisrxVk(I@kI~7)~ZH>_$D;x{)4rsz1}|jcYg;RZVdM7@t10RTJgMT{>2P5t$p?>%XX| zl{D&RjBQ8c|_!BF@;3)*NtD;jR@{6?#Wh6XIt9tFkVNvfzWd$!u2 zo@rP8%O9&|=pJx>x6ZqNF*@+t03bI65j0%1UbW`i>tR~%XAn~b%6p_khTQOVOMDs$MK z1q}VB>GN&J?H)yejmDb-=;hpD(;EB>q-=3pCbjw=6m-RjeJ2!B)OXCyav@)Z*+&{E zuDSHBOnIq!#2O+U9|gCDtM^?2T-I`WzrSV;hVVg;Zk51f_=?&G6-}Lnal`z>#fsO0Z@F zLH2P4v;4Jh@uQ=q%0PTZ+Td1$knaZMkMG87tD$0!4{g|UPAjo^hRrbkck)jjtBW0u zORpW{yJA0aBDl-^tjRW|?e%QOQtK*j)aRx$#XjB1XED)=P(#LB(p@4L!c|$yh_N*# zL^paI7%qK@1EK^@Url~ytCYp_iFf6alMzym zmnmahJpbUgz9Q`|=k~;a64keb^3+ID!~rf=bDiM4yKFj1u(wT^+ok$>{YXXTW_RpJ zLzkV!0`+NF4`h$WRJx2=J#j#%*2PHsz}mCOj^#n#b>U;{k4wb%t4;-x8U#_?JOKa` zzK{`B`k2ogT~a4$qkOl$8WBDFdCO2aC~~N7l5;$z-T7v7hFTS31@CY*A7}5^;I|hg zDwdrm3!dV0AjJ83i=oz^N=&aby0GT2L!X5?$}Ux1>cY*D)!e3lJ-pT9c4WH`W!Jb% z`fzK#8$B%@1pmNJS&TH%7u1Dj-wfbFLOtw_0z;s{xNy{YwgBh%zjtujeioE7IU(A_G3Z5<)eTk(w9z98+xh~8Sa zZbi^k2hs)!{3VLr!8Fp~R)=#e>gpsM2AiYAM66z(K-BZzalrgmuuDe&o4-@UAiAF` z!sWSBqt+!s=c*MJId`S!+_vr{3f65C@<(&%xBMwE6E#agMWi%yXyUnJEoq`@khQ>O zn`>{rHK&BFw*EUqd6?nV7kGTalYg_o>}-(;^ad;N4WB8w`ucjw+U}N&oVKF^U&&x_ z4srQE(zN2u9 z1j1%dUr9#UV~(NbccKjJw{qxsD(VwEx{1ewxP^jCvf-++OTCXwnRqgTPT_OJp|+Jylzi)$L{a9zZI;!v=e^nEp5*+wqC#M z884<&MWRHTvfR{F#n;}#(mVRNl?G$SRZUxlrzU$2#w%x}cFp1ous_#(f{bUa*ig)w zN+$kmfKvTPr{q*bafaxA`y2&|M_?{OeVZ>z@QrcIs~jjW7G__dS_ZOWeR1Uenj#cI z4pbxzmwZy^Gl_-0eJ@-Ll~jnfTT+C-{_?-ft|(vVd-4!f!jYOVCc-vNNB8ozVfNT z&PGEv$7$?Zm%65a>gwzyg@#^sb~aSF9Z*|>a=}7OJLvOgME>x>$A<=LNoY6#{jD?| z2FbEBUW^do>H(iAmwIgUIbkZLWgQvxSM2`NnfgUzL0y`ljKtX=gU8yKsO5*w2j8LG@3ywKa-1nU)FxU zErGOL2n4q-0m?&Iv+qy+uL*2-%kfhBJ#27ZtHLpa2I0xkstYjvUy!DT&5CLSr(QLv zC-5k90UwXr%~2&ji+rJOl5%KCg_Do!KfyZ8?uSJZlU%N?QDeax5B5-LY|K#Iy$8f= z$}&K}w*Llwr>PWVG$Qaj0V*52O^`wWEYv!cVw%Pb0oLX zMhvUI;fgu7??|a#?$BYVy8VZ(ElZ?D*jdlr&ZT`lj+}DrY(*}But;FIUTa04aqSCCZd378959@%emELLeF#=5oMmxs+7 zsJ0e+Qv3IT;uo>CwH>|5!jjRS`8<(N=$w|-_3cY(F73liw?TjtW-Eth_ySL&fHvDv z<@0H{E`1LG79>QN1sl-(9l)OcGxurRYoNgc=w=HKKi1xrA1OIR!D8S ziFT^q_vY=}w^z-^E8j`DxXo1Ypo#^T{_WmZ-d?5p<q7FY_48VVfWOjzA+4xKf0bg4M{C*;@_f8{rkQoQMBzV`|Wa zR#XUUX=xb~bmf&1>T`OvbY&ed=XQN*M9OPl8J8Ve-LUvm34rL(uf7u-W^QW#0~ZiHreF%Jpoh*JSf`5@`{R7eba}9-1^KTd&-eR$*ALo@Nma0#->n7+fPM-y zH0Y$5%)mq|E8QJ{CGV(K%mOj2TrhV_)>Uyx;-}a)HNPlN zFd3NPi`ql?sfe}boC=TPHvrC^waR>-cvNZnaPlJf1UWJ zJgV&R?z(1nb-stGrW-8dR%M^p`jHx)dufF}Pl~bBw4N zz~*Aklm~t_;~;jkd?zuV%28&_~X?PA|?4_DctNkEXEam*Fmoq~14v00R) zMt0Dnr1pQ#0|4Zc=xgsgltk~LC`J*>AL-@}@x!NSY%$v?$SnAKzMaTpE&*oK=B0L7I zV^KJzSJx?986%C|p|al}hXz9+;JBW*F{-hfDHw6(@^V94Jm+aM(GT{pK@MX}=NbC` zu!5#A9wS|3bYd;2PsoN+=}}!FE7un!X9Tj50PqdAn&Tt3anyXo@qUHrA%Wb~m-;_c zlW*ndqBxYT6K@&v@FU|rh4HMn`5twZYw~IiNWkpwruJApJXbjp@)nZ>PYM1_9!DXdaaAK*K7uz+A4KXVLp-)*p$v;Kt0{*yh zh3w|vB`Ma6(%F~W_s?-#4jWP`J@bYFnO-De#_>^S3eY|bB9BP@2p}XM(X#2IsXZfnNg>MhX5@1 zm6Y?Q&Wst~(>Y~9#>cR=n;>SlU|99Y09z7o z{*J5p@+CPm*luuhhk)u4vR?yqlUVc&Wo54bq-KRy(HU&)-$fR>(8}6m-h~dA8Aw+^ zN%jELhKMQ{q069e{^rQ~hv&*v)hiEXyzy~=IIQ5O#cEFsZBaPV(K8y1Hm3zpEhnIO zAiqUG6$^r$>o;z^W6Ui54E=I8=x)(8ym|8mX#&D`A;dQxG1GNgT2F-dh8|VH8%X}G z0XH5Y$Q%ed0U*GUhO$&NUm&osF(#SjK6)5NUWJLFlCI*8hpGXwzPe~GobgyjYqRm! zo~OpwqIFkb+I#QuN8;a^8p43+>tMi*+1-DO3Y{Gt9dRl7|0fdMN7d%xLrf?1&Wsy@ z5P~p5qwaBW1+T4{0&98>53gj}EVyj}YpsoFyz%{4i?3z`xrD)CTBO65Z4x|0KWp5V$bU*r>+pqjG~Yk1O-ERlAziF&sit%)|{O z{p&#ymW{UMqfQ^tMr=6{cjhkkCDoES@eQDnpFDn?4XVZ$nd+^eZEXN4`$BIz1*n_o z&8dQDpFuZRVD&NyiQ@Ezf>e^%ji1Ev%U<}@5Y^K}0^Y!ZF{{H$1C!=VV91&!JRH%Msq$FIRXr5pF69tdomoHyR8-SE% zIVk^ewc`3bUmMpEgo*H1#xb(Rc}`RiK7F?gai9`DHTd{y5Jwt<2n1>J8yA$cj{PSv zucN|4bRw^`wfCp|g?&`pfXs#VNP$fFXB++Vz$<* z*xxff2lL0mxp)f{!0o~(!$x4c0syxy;nGB7_E5f&1;1RsaRGs$)@N(SS(42Xbz^;SlIRV~a_x_zS{3 zgAs7N`SqoA#`)yP2I+1IpX^JQ@6S?%Fez74n3O&My_)x@Pt#3wv1x0&YV=%7I97ZQ zrOu{BnXk_X4hIQFULn@hUJ8`gN$;Z40;5~p4Ez5nhgXK;=xF|hUXFGx z(f1i{W>$ROOXDB+uj`m3E)*PH=1h2H!@F?Ork_tGSfdubxUu^#{U5RYQbomj7PG5t zlVvPQNwb;6bd3%A>z`M5F8~om8^wnxG7#ApeYA1=dw=0&HEe!DfMWXXJ4RAju>``a!mXtmKIL9N zk$MX5Yb3})><}71jUB=0^5VN%7y`Y2X`H}+Xq>>DBzVqdgm9etVgq+|Vi5%Pyvu!@ zaa!_hbe%b2`IcU%S@TyEZrm4jnKP>BLIeAOX3chX9%*`L>`E2_4ZLo!lQY#%c1mo0 zVTSopL_c{T`i!@X+WD{pw&!0&%nA@OJvx?oP*x$ldxGH}_JabmjD=Noj~01=&+mYE zAJ@&@IBI)+gsIT;FRLk<{vod*72 zkk-FYQBw@_R&gEG2#>eiMFxDX#&X&fN_x$%{dlH)CK`5z1tzG{dhjTk@dh1M$Vn?q z5a|Wt9YtcTt=NV!Zen~jZs+P(B# zFq*@5<~tVYlEeyU%zP&!2oyO=Op>G>S0a0=JJZFO#$sNdR+6&MF}`g4wxax5L2-eu zg$e`R7ZIH-PyX1;bc_pngbS(6Ajw4Fw!AuntE>9V~;saW`*2W$L*!M5X-zFv>Z!Q#SMeQ}@L{p00 zGeB6oz|Y7WW=PK19JO0HRd&0vZF2KE48qggYa4@>$%@gR3~~P5NxTgfIm3Df)6;3C zSV6T@Cye8&IeV*=0)qWPSqKbprZa!b;7HRcX->~gbU%f(XQ?WyXq@^i`bEtlPL=XK z>G-z_{83+{7ctq^n_K&-q)x5mDquwUsEG85HtEU+f_x^|x~WRG78qH2>#ca+bq&Om zTJq4RcVT3qRFKUkS0%tYnTb$8VS%Cya>)<-ENba(U{xt3j>kn_hMXD!OEe5y`l8_B zYE#BS0^vlTa$-Gw3V&vrMCzbcyw0GVmDx;rEs)~2Bt-Y`HYj`_Up%pR#%g=jnu!Yv z%so(q*vd(uj(58-oCQq>hwmdKzbLj$i^bH(j@gTBlxr0w6CKhBUBI~bm@V1K=mHik zQ@?wc@;!e&D8W}@!)CKzP9(ZizTvqq#ph_we`}SUUm^QHO4s4oi>IBH>2LC=I#q7r z!XxeE9%6Q>%9aNKA+N1}y!vnvU$e#67b7gkmV#c_;uOcra>9ILUG7|n#_L(CNhw=v zG77M&UzV>kZ|~XGyGr%Y+@9NnW&bbZ`dAj@YWt-t_%*W+WUf-~w^~WY2@WmGhqnB; zb*w3a>*PHeR=2MwSYy}w^DdM-UX2Q#W1;Bd_WLHC8o?WNmXO%6J~^JG2yRgP0B;nK z_`^89bMn6DY=sN%2M9p&OzqnSQm&c7pj*pX+r5btpsZ7t7IRPf-ga4oYR&vn?%VfA+nM3s zUzn4+#yb{rmqqz%cE0N^^ICffe{KU@4L!VFLAslnM)xE6dRopiT$31+oD3fv!PfPI z!Z^d>td~g1%5~ZE0=7<%gwP8pJARt0{MJm$5Y9j;N>vA&fM z82IOo>8IG%Umz!XOHTN3Doh6Q03&an{8`wfqKu=Dsb?}=NM?nX`5we;k>tv$$-`}? zDRjZ~JAaqyGKCVd;TwKupI0GD3<4d}vtqhA{m$JATh^2k7Dm7ML+ZoR-rGzZ94Hm0 zd8STteGuB^pmi_4{Gi{_(mnJllrI9Ib(m#z4+fw2=b&aUcw9a&lbbit#IjgF``c1a z)-BkDaxNj0dovczhiHJ|o&qVQbh9~8>1O`?k=gQvlyGbk!N{J6ZpoSvweFp~wC=QX zKI&zSnvM!v@$O8XQVLAj|9MlYWtmq&QHr*ulxQd2Ga!O(LnT?geOaC)4{b|cHhSPk z=~acZYZ>RTNeXyG6kA*h`L0{LBLzw9xaCTvJ(%0y-|JoZ!(f}iZ2y(h8@Ika@tb1E z)=@g962k#XAucy*LfGks9)}3Gb#P_eIoPl6X?LSTgBg~^Cui3GXk^e;_B9|!Q7@(Y zdb6R3*`DA0qk1QqR6LO*v)o7Q#m8r6{^S^4ArzmQmm%*x*XweV!MnUd0VVO6Iu19) zo^*5-BL(McTd?VF=r{@Z4o!mYK+(iD5v4@yB;}1GzDG}Z*6H}w6$wkpyGej z8J_mA0j_3s-s0;{U*__;`206V_&%w!vb|_D}?h#lZ)rorZfgs&NUoml)S3VDj1W{xI)}XE~ ziuPV_$mRq=gUWnZgcgP|-K#>WmUFj7`p{|e|Jvj5w?lnv3v6sr9W#3KJ81(Se12|= zG8~Pndvv)^z^Ezotq~mwuTd>q&COlE|ldnTTG? z;Ib#bzq;cbyfCz-CpkFY$-)c+RR+kd+fR;ng{5Ky#6T%&I~%TB2m%#WKIHZp+<3_` zop{L3&i-~NpBcHP0rx`NRcN790tKhnZHbU9qap6j&d#?TaTw5Mm|Ab3rn`PUPRVW? zgMGlCy1KfeFcDpGp6UrSc4;7tioo5lx0F=X-x@NzW+)iQYS^;)@s}^UQ6>>SU1^8D zbNjs}Gg96t7opIpJ&W++1BOI^Dmuii=5kuN^rHC6WpCMn2Xi*WGZokR{y47`;hZW{HG&2 z4pZWdXC!3HPp85zMbx{`Mq3TtqIK7Q;7mlJ&hVwYLNE;lmokE&IA#OR8R^rcDd*b4 zWduaARk)N?`Z__@;cAvSu2Nct!#SQ+6YFme6JAeu{Mjaol59^jQsn~a zfA-5tOjtM*G2GEIu25mlz)rK27* z9v^Ps>{IAKXGifQ#=XH~6y0wQb$DRT8=YZCay3_iMa}MUn{p*@-y@b0XxgOns>`r! z28+ky`OMjE4&$b_$7V(c{u%d==UA3#yR#K52T8kcMnV7BuGc6i?9#~X8i@t;kQ?s! zx5pomtFd4`uNnhx*log<>L$1fH&EdMY9H)auK)3VA4*hO8s|E1|53&TXz`It3vOQp z&C%qbL1x)>%K^#`RpXj@OyGP>YKURFi<#^YJdlFBU4f57ir!z2DCw$P;4~$DQq@kxHl`KD zvr=kTN4A>c!OK}`J=$9uQ2LRs@7y|T@5#7CPvWWxK`~+Df34|PikDcRL(|ZssG&CB z4?VHjU||#}=(DtHY@cTAnYKsDV23*Zhd6N48SjSY&`TB`6}4 zZ7RJ@t`BmoR`3@-qIy-0#Ty>up;cJk5$%aF=S&`j)+@BWM_y=ruoB^79D{MTzG$r7 z*G8pnn{v)Mro!;JuT)cQ&e_JPfPy#L#jibf^SwaZ!`|TCh&l6^-4ezmv_ruRF*UV> z`o0<3F5@jxVT;|SIiA8OyIuQ{at^M2E8fSjA@(`-n>OLY%yFZQyu0n~=~wy8n-d6= z2n*~kCJ}lO?#{7<>DutM6G;una;U24j>MvK;|>lASDe66zJ#$62?k0`+{HEp6NlR} zl##81vVRGQw9}iA;2|tyhr`I_$3zh8d1N?%HXrrVa3DuypzgI0cj2E$dwaCJg;iF4 z0fA!=Z6{7zTH!kUd}qApz+Dk>_A;~`BTS&f42_sML3o9Q?+Js9%$W=5VXKPqcxJXamOdwUDBNm& z|J4TU{#P3?fvm)2ObEI}o~=4J@7~RB>}%)xm+FpBrvpnN6Q^eOy$k6w`r54UIVrD! z6qt&@|0c3489IbZe3)oDDEB!$h7uC zduV(LM}@d@Quof5ekNd7^+ssT!ZwIWNK8R2w~2)7+tN@AgPe}Rd>QW1Zm{fPW;9pA zmAN3G*)CI1MDjp=d<1fDBCndNoHxzMApRbxAiJyKwxl&);)>mSLh+(P&l_2f= z7i0JT^DVrY-~zbP&j#&K(1S+V+?fA=?%zce_<`&ek+{K6GVFnh9e(Ek?kCW%?gOu( zy3u<-<;+X`_dm5Wm|Z`xK79M^t>PYb9c+B5$3H4MM3{fq`EajPrK&YXaOlqq(~DkM zuz>x(%$?m`y1RD+Az5woYx@Hm^Z+gb!R^(?61;Pj!2ZCA^B~&D=7Qt%gtTmCleHTT zZnEnc4~D`lMPOFydT?OIEp;*8=(W0B!J1Y}EtdwrrB|M%(ah(oJsEF#bO*ry)EojvzsSKS>(G4=GRIj64PcFLu zjQ@BM*wz#us0JMNysA6e(EPUSKML~HsuS4e=$*%ygJ!EGA914PjwE&n@1C?+5CS{ zTgiQHbbJC2{saut(lNdN->@y3_(3|i4svCgo}`#amEIPe{yK+JIZdcA!oBQ#nn2cP z$)`G_7TYz8xhE~oQJ#dtsnokaOTEx4W6}X80UtiF($Ep=jN+nSJQBU|Nfona>2!;K zg%!BC!0#WV*iPXzuKdhhY2>@Ib1R@wIPhoG&(gU?&)x)SH=!-P4zK1PYV|uoLv~g* zscHPh7hTEUagkcoksk0Y`}a(KeM%iAh<~zxCl+}l=kBMyv+r-VO5KJYLnB2+1uIAa zaYr{W%*myk#ST2&1=4fKsTcn5L9QEqabpyK;#*W`2iiboQ1SKIP^pXqz4x8@*i< zgK@vY^N6&|`2v@lLC6sLPWIH$gj-C29eKo-C%+H|H5EG}jVzJ;%<8HHv{`W#*0Fw~ zw~UGZDA!JweH;iaW;Jp4HaW-BUG}aV|DZGb#Z$-h^4s=Dgu2peJMAW$b#1I{Z0CMr zn`@+^3kzn-&J1EUFB*OEJu;Y>imZw`A4=dAIcgTo26h$vQsMBJg@ve6HkpAs?vb9C zXn#=Uq~ON^XMKlJM;B_B;_iiqV`rU;^4s$E(0ic+d^~g}I*SJ>hf!M`sJ!wS#J z;+Y3!8-Jc~_vB|zbZKewP8?*L)>!C(;E_I}>->UPfcYiai}!mgy&h;xP)-8Bw_@_-&!)HrMV)8Rj+zl||odGL{o4 zEd}Qqk1h6pFMWL?x4B^-_v`RT?S(u_;%knRE8;rECiLeWuJoH4VoBAUcg4n2>@EE) zgn!WSU7eu%sD_bKF;OtcpWCI@D^h4W?TCq(X}EKq_&D?ErmU{xVb->sh$1sE%UidtRr==i1cP*hug9PG)AHt5n80>!gDE7c%D$hSm(^!gXg0M`^@E|M_j0?@+yIj z0}RgsQomUL*rJIGSmpMV&VD#5s0$~Ui&oq`LEiF0_jFnBSj`!g@jG2V=`E?=^SEyt zl{!8n^rPp!FCbn&QM_Mk4LF+^e33D&wlWHMI10=ERyf5 zm87I{;h5h^{g-XIe%op*#v;BOR<=1cLBle;XD$2*Ec?|Ou#N*!7!z!hD&EFA>et{$ zG(63Wmqv=<0IHd7ac)f2)@jTgrrX&0^X|rCOz_dCUlrI!m(Zl3Lf+Fkxh_lbFZrkG z{*HOfG*t14!o!rHyfxbUOLSCjfO)cOx(&MhNZ^cL+Q$k%6E^veC7O7n zc3oQ8gZ7GT0)T#jw@T35!D-av-RD|R1poT7irM=6!Ea>oh-cBTpI}EYeuOKFqhotQ z>ycAU);ijGN=51GMxV+ZIWJ>eruE=kD1M^3anaitWB2xbO)iTZZ_xp^3!=hi>f!uh zE=y_$>k_s%KB9X|93N+1JR{%L%O4bebo2)<&`4B$A_&kTH54rBu^I>|`*<(HnRB&B z|F~yfx!vmB?9Sakg1goRw@uet9w5davaEu6KiTBY_d(yuHXWOgy$JUNbNh-V;~RQU zzm9UT8OE^WUY~3-XFn0J$j=(dtsAtrnOGU%H7ZLLkw-yEo>~~w7wO4(glS${HHu@RyFeZ(Zgr&A7vc~Sk1_(YRnz@5Lj4#Un;LsPU$$L zB|W^XUFf)Q7P$IoG_kRUg04& zD<`obt%lis26x;zNH1R+aOPjZ3e_rjAW(i)W$jQ^j%pIBsc<>rfk<4t@>4svzc68YFhd!`qJ;W3V3@sAp|w zWT02y^=u2j#Y{%MN4}~D=gw_LkJ_UTOoO@J_)qc5?{E74ukOw>Dyn5o!+-)R>7XPL zFbe`ALCHxJi3b%Fk0Mcmft*{Apa(@HC@2{Wh={0&4H5+fnwG4#NDw50B*~It-t9ee z=gzFTGwaUJ`Nv{)_ujj9)mLAA^}bJeI6XS8o+GVmmNTr^M!l$3ToJNLjIJ^3R?1h< zJa*+;l3PS~mACkpcG;bw$<+r`UpKC~lzO2`KhS8QV@g#%r*Z$~zBN78TyIW@_lwKB z)flV@o6t2avdob>-M3cHGx3b`_xsHU<$1PU7)m;`jawZ-e0MFg$adkY+|0(eUJ3^` zTt8W8eqZ4!8xcKUq!n z?0hCor&5>xJ_3F6@YoKIvcc%lsw9KiF3Z^^qsFYmGva?Lp1d=vNRd$xyme`|Uc>7` zrcb4kTTfReCl_PG(1DywN8c0=U5fJC%DdMrpHfTR>??YXB~N>23d0i=xMC*gUnz|| zQ!7YD0F-E_}K&h&_L0P2})&K0rADX^}qhCcyfqhk0jRU4c(-*5Ps?go0zrAR_ zu+rUE?O2gkB+(q{!K6CKq2=dsIWFQ1!vm>avczt+Q|ITrYLm?;TtD3ua+>x?I)B=t zGK1p)XN#t&6@2}8=L##UxIpE29fh^R2j3k_^`^=v=7@o2$@fe#JLIHXQM?x$rH`|g zM7(ECBZIlG$2w*DM~awlg)9{%POn;vICiD75Eh(cAos6}@*0S#-_L z?J?ywEI*h|_mnPL_XqR$oD-Q@U9;08esB5Z^{JwTuZ5(- zE}w=`ih1U&Sk~4oo08HV(-$TcHi-e(OgmrHyGL7Tx+$d+&+x@F4qOz>s3- z5ij3(+5Ihd+Cu!2MhXtGs%MrQ?Ei?L=IwUB(&=m(Up``|P~+92H^U!p-?%Y5-)8Sf z6Q$kU+5Mf}C*w?JOAq(mT%46!?d^D&ha5*6Zuh?LDlu8EM&R@MHob8{Zn!@7Cx}E6 zDKg$COV;Ya3$fPtw-oE=kV^f6`9B^&Q$N z;dOi5*rc=iv_pU0v*#;zm9!-j%}1dF3nq*&k5AD*Es#xWDo1)TJ3X68SI$J8Xk_Vi23%Ff5}Yh*jXCXdU>WT+p&$`+}ynF#=&(q zC~tgyx!0U+F+{6HRU_aCotp4cjUdaYMQQxnM0uam;b5fGJG+iCI)Au-pqxsTu`FE; z2v*)imDomU>oq8^iMpOKHs(PImRRndr#;THc*kI6W%V5%MX{4F+)X@}7w4Xt7Dn^z zIsJiH`THsvY9e*t*x<-TTo#k4g@vRORe8RCwBBEK>AJnv>t;@3TbcqV=ibxV!C)mw zUHl?`jdf$BTnV%!_TP%)mKJ8ZyN$*i?t;KV$i`utV*^8g>8)cf?X`mmQPs?KH2wC< zAZ^1?es{7EFm|BW8R)F6ZuxUK$?y01yOl%2WCfB8oy!9k<>pSE0*VKX#|HXu_4an5 zY({6?G!(Rw-eVTIb2DvTn}PN76mww=gYEOY+1#p^&GxLDWJ7^9aD#2GXwF!0N+R{M zM1dfAzP9V`MYFA|Do-sveEy$pq*Z4vhhnzBx$ZuCk2M>V>>*-6n7B+K3@Lz{e?h7QIbcPfn=meFsP3^FMxy4!Hw zI6?5pe_MsJk$No;%+%kBS2hy`h8|~5P3kT;uK0(JDkl#vH#A4Rx_(SgYyABut){3~ z)L*R=z|8$0X3zgphdSAsMfrE1=fNHgRpZS}avN((53uAHQreotWsL<-Q|>cfjV-Vg zO>b6o2ZOSvS1Kk@M_clllU_IXIE6+~n9VaXze=kbHx^A&dn95OT6kZO?@yoJBQ&`Z1P-D`d%MN| zsyHY0I`zE2jp0VV*eC&EJ@Z>=Ut(1+D-zll!14$b9+YH+LlhXiwl8E9{bcQ%=-z^v zK^Lp!B|(RtnFf_O_jEKx?=wecy;{M_)lrVi^`F^{nv-57*f2-O31%GS5qC_L$=s1_ zE>gcYj+g5_i{4ObXnj)u#p#{Wl9BQ6_NQ+=9`)qD2+|1K6~KR<63QkzAdb58`;*Tu z&WR+1K4#Kb0a-?YyL4rCsXSq*JB6 zZTI%{4~-L9vpr%rczGyK-iq8k#5v9nTx{2E*W6)E#={>)pLWR12^vuE?UX8eBhY6c zY_p?NQh;Q_&C-2#{K4U)VMV6ZygwYn*M@JE-eS1(*rBKJZ*it)`#yW|$$0w{rswBP zJEYmuuJ+XQFjQOqsPEoLb>Ryy>X$bgo44oLgr%Wl>R9RX5Ul(ZqfVn~0HQs99z*Qm z)rU9OLnPcc7SaqRnytfa1!_yG=UVHrTsYZZR2*GXZ0h}zM$qBET{zh02bbnJ#$|7(fE zs}J32b`aMrPbx0SK?C+gHa&Y?)K(N!>C)%Ct^3de>UN>u)v9igh2P^JA|t7@d=-f*mbezrfWm-vkM%9b zBf=`e=tuRxr+sbnSMzTM)nq{JI#VZGjh{)EB|&qp3eIeZIlVRK3AbU!2OHN!1yW2w zrI&J7zG+>hT@-OYk|SG~Y3$KRfGE`Gzx83WGHtJK-Sa~{hLWp%8IFsy-0H4z*=xtU z+iBy^P~cp}oY>2b*)~hNN>>f%JQ_R-Ms&J5Zr$pj)eW{Ad)(s$wAgj*q9#1|Y_65B z8V~JG%(=G{?4)U0^0AK5Q{m!t0@to)C+_>?`l-Xdxi{A$zvq?rZi;kn08(hYOg`MB756`O;9WF(`%2Rx zhX*D)k!@X;zL$N|1q~Bgw&vVa`SK&%?Tg>fhc@F@n$j~ftpoa6yE+Pt^slL0W%X%x zI+lX?J513qSY`5|$*!BCam2tkt>Z9~wYSxVZ|~ag?i`3(@&6Y#cg4RZtY$gm2|toMpoidq0WCtf z8Fh4ZMXPg>i)`!4QhrLu&c82scGerNF?B@elV+ry2d}Xb;1JF5>Z+=TJ~KZ6B9V4z z*~cUPhq}H#D>kZZd#mm`R=+rtAE|nI{QJT-rC0IZvR&l*g417 z@RUiik`K~}|4(ULw^2tE_GBc%u*78#_SYrRzx+Cq?eH}k3=iyT#^l;l!v%GY9O(x< zsmzv4|I)UWpU}|~xgV@v5(;i0c6WAmK6>=%Hy$42$XqZ%wsGtK>xtY}I`BXUehQze zuQ#&1JP@}CqAdYG>h?kzep?p_lGVScs)gJ#0kRG)Ju2p(n_7F<@fVUNZF3t%?3*8c z*@}-Eo@kON)~0A17(51SWUdb;pT9mD!k@4}era>_G&A-Sl~?57eg6D;TMWSm8b2gN zX8aAnxjWzUyTbR_H{X1I_8&F^L(YH_dZ>H@NuM4ws<<;tE0oHCS8`hj_^9=cqhnvn zv60V{&mQ2?U2z-8o*8-P4)xpB@FfR(Z^P6L?v^b`Pfa79xpROmuD%b-i)J{8u;7@e zv6#Gm;sMXk!Ui6mzMsDl`!M$ZvLo;F3)6&fK_zKug9X^NzP&;wWdjx-EC0SHq3y(0 zR&f7#>=50Rp?H4+^9WMu14xEJCrv=ti~!?P+Wq@cIuN0BpgG2l*+ZbL1BQA+2~->; zM4-FrySwj`v90L<9nFg9+ykF>Itha9cpu(Yf<~bwMEVoEddZ}|jR3;VLHsd3GgH!2 zR1gkEf_85TcFJ1*nZB3Q%klP?t4_WocmAKzE}t*b6Id!p{at$Ab272=y#*lSFjU6c z`ugI=b415af2Mqa#lj|K-~0UCkVPd z1PQ=3-L)k5&7AEytcq_XCbFuZIg?n4L=6N-0M^%XaL6fL+l`tmIwV5?7EL}Q%ugiY z@dM@-K2ZA1H}zJQQ@Ohnmt`4g?WxZ5U%3qY6G~GbJvxf#DH?TgBDS22GlqVvf(v1( z`b02PjO9au{9}{1bnDJ@dL>g84)r$@PPR4)91)gl2q$LBO(6(CR z@_$`mz{yB*$#h-X-+l&Hw~#8D6F7&K$g8MdlL@^upw-%VCtW5&65IL{Z(_8J^{+- zna}@d#|&pNuVa2M52W^M!EM4iv}o#+9-z&!g3n6_G(POaTd65oS#p9ewt7!4@I|8~ zPT=v6)%!0;0rx`y(oScedpF)qB>~DxFl=XW?|GfqHr0fmi>IX>*#PV*yd+Od3uVEk z^_{2`X5Flsh%h{X_h8|BzL6l^wwu>M#e z)WFE@|M}IaABKQkJ9hjD2&4;6MIfid4Fb{E(RqDyot!_cqZt2g@TJG!s*-^)A_$`_ z9(s}qn2R~NxhVv5P;c*>asVmI5%p`r|Hxw9Et^I=FP~(9P~Q!HER%(PtmM1aqe2~U zkno;?yxuwhN*mA9i2|_ zgC3*zp>MB2tpV<2yhy?d-0nLWR$Lx{K!j_Q^;AZ|#2nu2qFjYyMOL#t;F1LL7lsoD z&mA<>WV|qD3E3J}N!KCW04A6)Gb7@WyGgKyaDl3*2VCO$5BCF`UGzQh^hbeo1get) z7YTuYWZ{+QApk*eF|TH3P7ut}fLSY_U**z1^crROtB6~SP@Y&*TwILh<77<9Vi4xj zgzG|=-Vgur*r(^V*14AK-EcljD}_+kaB%b^a`qFy2u!kmf@~NJAs-+ZBb*iF?zlnt z^wG>Uuu~iV{biJyU z*bZJk{VFDBJcuNc3v%}fW%A5{lb(u9FP-5zDWc~aEt(9+p%@hyfT57Q@tONC<&1-D zxyd-A!5au9jQgkgZvFgzets1Yc1`vT@V&hPfl1X9wDd=i;PLLBVje_)jNe!3ux#S= z#)hr}Oay)1K9ZvQcMCx3`Ee?Rt(H_pQvuLc7{c9I7_n@r z84yDN82iV|8ykfA1IBYpmTHGZghnuIWO2a9?K^j@J-Li}l6Op%Z#cq-VnUSDoFFYld2y_a$>LXVpzJ$o>T#Ez4|s5J3gBVc6d(m^dQ>DSy4`P~mM#=6 z*@`Q4f@LTPd->#&Trc3q`{>S6Lb!MBl)3InuNUV7^l6;D%Qiq=dz$UQ^$*t?YRV$m z$tcGq2vWEJR3k=V5zaI?4jA<)u}=R)qosr7isHb2rN>%QbtB)V`Zjz-S<9$nv9%DHd*mmgCB@Oae5Ed3N3N2>lC;Ggspv&Z&&z`<=$LqntHo1R<7 zIH4OW*zf_foCv-Q+yl-jSd}Lssel(r2VxqWVz=VP^~dobf7$4NFY#NNFm?Hsu#&Qf z*s-36=P)u%lq)(hpZG#B=Q}@;MsPAP%ajcVKVGwD&3z;?bAA3(AGJUO;F?X$CG?h0 zK&u4w6b}`GTO(khZo^Gds`KBsV}v}#EJfraU{I_DfbYafQwqSJ?{Drc)`rq4|7Xj^ z+VFD);Ei5E&URs_okRo*%n%yDhRs%+FKZk>&P8NT$p6s(SCDGQ3T9cwz2PW$1yp}B zJ=bg#*)F1aT>oRLUW~@du5Zi72=Hqd?yD|1Bt#nB@)$xy3bOJF|CI%E%yv7HR$KAG z&nAs1V+VEFCP2KE9LR>4HXz7HQ~mc;@2Kb_M(t`sOU!c9UN2e f|NUU3<;}?q5#R1cZ&Al_R&+X=28Ui~SX}ux{fgDH diff --git a/doc/pub/week44/ipynb/week44.ipynb b/doc/pub/week44/ipynb/week44.ipynb index c3f9643e3..5f6c07b59 100644 --- a/doc/pub/week44/ipynb/week44.ipynb +++ b/doc/pub/week44/ipynb/week44.ipynb @@ -3,9 +3,7 @@ { "cell_type": "markdown", "id": "1e9d457a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "\n", @@ -15,9 +13,7 @@ { "cell_type": "markdown", "id": "8da83506", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "# Week 44: Decision Trees, Ensemble methods and Random Forests\n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", @@ -30,9 +26,7 @@ { "cell_type": "markdown", "id": "0d38a800", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Overview of week 44\n", "\n", @@ -56,9 +50,7 @@ { "cell_type": "markdown", "id": "d31105a9", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Digression First\n", "\n", @@ -74,9 +66,7 @@ { "cell_type": "markdown", "id": "07fe25d8", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Decision trees, overarching aims\n", "\n", @@ -105,9 +95,7 @@ { "cell_type": "markdown", "id": "2d6686ff", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Basics of a tree\n", "\n", @@ -125,9 +113,7 @@ { "cell_type": "markdown", "id": "ce62a1a1", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## A Sketch of a Tree, Regression problem\n", "\n", @@ -139,9 +125,7 @@ { "cell_type": "markdown", "id": "cf8c3236", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## A Sketch of a Tree, Classification problem\n", "\n", @@ -152,9 +136,7 @@ { "cell_type": "markdown", "id": "d450c9c5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## A typical Decision Tree with its pertinent Jargon, Classification Problem\n", "\n", @@ -170,9 +152,7 @@ { "cell_type": "markdown", "id": "6065bf9d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## General Features\n", "\n", @@ -193,9 +173,7 @@ { "cell_type": "markdown", "id": "dab43c19", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## How do we set it up?\n", "\n", @@ -216,9 +194,7 @@ { "cell_type": "markdown", "id": "f35f5834", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Decision trees and Regression" ] @@ -227,10 +203,7 @@ "cell_type": "code", "execution_count": 1, "id": "6caa26f7", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", @@ -328,9 +301,7 @@ { "cell_type": "markdown", "id": "9eb4d7cf", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Building a tree, regression\n", "\n", @@ -350,9 +321,7 @@ { "cell_type": "markdown", "id": "53c984bd", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\sum_{j=1}^J\\sum_{i\\in R_j}(y_i-\\overline{y}_{R_j})^2,\n", @@ -362,9 +331,7 @@ { "cell_type": "markdown", "id": "a9906c4c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where $\\overline{y}_{R_j}$ is the mean response for the training observations \n", "within box $j$." @@ -373,9 +340,7 @@ { "cell_type": "markdown", "id": "96557142", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## A top-down approach, recursive binary splitting\n", "\n", @@ -395,9 +360,7 @@ { "cell_type": "markdown", "id": "f30bcd98", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Making a tree\n", "\n", @@ -408,9 +371,7 @@ { "cell_type": "markdown", "id": "ea89bbb5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\left\\{X\\vert x_j < s\\right\\},\n", @@ -420,9 +381,7 @@ { "cell_type": "markdown", "id": "1516b6cc", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "and" ] @@ -430,9 +389,7 @@ { "cell_type": "markdown", "id": "893e24a3", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\left\\{X\\vert x_j \\geq s\\right\\},\n", @@ -442,9 +399,7 @@ { "cell_type": "markdown", "id": "c4b356f7", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "so that we obtain the lowest MSE, that is" ] @@ -452,9 +407,7 @@ { "cell_type": "markdown", "id": "be0d9aca", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\sum_{i:x_i\\in R_j}(y_i-\\overline{y}_{R_1})^2+\\sum_{i:x_i\\in R_2}(y_i-\\overline{y}_{R_2})^2,\n", @@ -464,9 +417,7 @@ { "cell_type": "markdown", "id": "d4f3358d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which we want to minimize by considering all predictors\n", "$x_1,x_2,\\dots,x_p$. We consider also all possible values of $s$ for\n", @@ -497,9 +448,7 @@ { "cell_type": "markdown", "id": "7a339714", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Pruning the tree\n", "\n", @@ -521,9 +470,7 @@ { "cell_type": "markdown", "id": "33f066ed", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Cost complexity pruning\n", "\n", @@ -533,9 +480,7 @@ { "cell_type": "markdown", "id": "753ee3d1", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\sum_{m=1}^{\\overline{T}}\\sum_{i:x_i\\in R_m}(y_i-\\overline{y}_{R_m})^2+\\alpha\\overline{T},\n", @@ -545,9 +490,7 @@ { "cell_type": "markdown", "id": "55e0c324", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "is as small as possible. Here $\\overline{T}$ is \n", "the number of terminal nodes of the tree $T$ , $R_m$ is the\n", @@ -573,9 +516,7 @@ { "cell_type": "markdown", "id": "813adcf0", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Schematic Regression Procedure\n", "\n", @@ -599,9 +540,7 @@ { "cell_type": "markdown", "id": "c48b2ef5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## A Classification Tree\n", "\n", @@ -622,9 +561,7 @@ { "cell_type": "markdown", "id": "fddbe38b", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Growing a classification tree\n", "\n", @@ -649,9 +586,7 @@ { "cell_type": "markdown", "id": "a3fba89d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Classification tree, how to split nodes\n", "\n", @@ -668,9 +603,7 @@ { "cell_type": "markdown", "id": "22166b43", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "p_{mk} = \\frac{1}{N_m}\\sum_{x_i\\in R_m}I(y_i=k).\n", @@ -680,9 +613,7 @@ { "cell_type": "markdown", "id": "c2e4074c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "We let $p_{mk}$ represent the majority class of observations in region\n", "$m$. The three most common ways of splitting a node are given by\n", @@ -693,9 +624,7 @@ { "cell_type": "markdown", "id": "b8e1e02c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "p_{mk} = \\frac{1}{N_m}\\sum_{x_i\\in R_m}I(y_i\\ne k) = 1-p_{mk}.\n", @@ -705,9 +634,7 @@ { "cell_type": "markdown", "id": "25506a4a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "* Gini index $g$" ] @@ -715,9 +642,7 @@ { "cell_type": "markdown", "id": "8e859f7d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "g = \\sum_{k=1}^K p_{mk}(1-p_{mk}).\n", @@ -727,9 +652,7 @@ { "cell_type": "markdown", "id": "75118af9", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "* Information entropy or just entropy $s$" ] @@ -737,9 +660,7 @@ { "cell_type": "markdown", "id": "7ad7a4a9", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "s = -\\sum_{k=1}^K p_{mk}\\log{p_{mk}}.\n", @@ -749,9 +670,7 @@ { "cell_type": "markdown", "id": "80392de1", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Visualizing the Tree, Classification" ] @@ -760,10 +679,7 @@ "cell_type": "code", "execution_count": 2, "id": "448f2c77", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "import os\n", @@ -804,9 +720,7 @@ { "cell_type": "markdown", "id": "976d6f54", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Visualizing the Tree, The Moons" ] @@ -815,10 +729,7 @@ "cell_type": "code", "execution_count": 3, "id": "245a63bb", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "# Common imports\n", @@ -850,9 +761,7 @@ { "cell_type": "markdown", "id": "96f4a124", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Other ways of visualizing the trees\n", "\n", @@ -863,10 +772,7 @@ "cell_type": "code", "execution_count": 4, "id": "89e96fb7", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import load_iris\n", @@ -881,9 +787,7 @@ { "cell_type": "markdown", "id": "3f85a09f", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Printing out as text\n", "\n", @@ -895,10 +799,7 @@ "cell_type": "code", "execution_count": 5, "id": "b6c15b05", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import load_iris\n", @@ -914,9 +815,7 @@ { "cell_type": "markdown", "id": "64b8d1dd", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Algorithms for Setting up Decision Trees\n", "\n", @@ -934,9 +833,7 @@ { "cell_type": "markdown", "id": "696574e0", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## The CART algorithm for Classification\n", "\n", @@ -951,9 +848,7 @@ { "cell_type": "markdown", "id": "69cfbe7f", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "C(k,t_k) = \\frac{m_{\\mathrm{left}}}{m}G_{\\mathrm{left}}+ \\frac{m_{\\mathrm{right}}}{m}G_{\\mathrm{right}},\n", @@ -963,9 +858,7 @@ { "cell_type": "markdown", "id": "68b5217a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where $G_{\\mathrm{left/right}}$ measures the impurity of the left/right subset and $m_{\\mathrm{left/right}}$\n", " is the number of instances in the left/right subset\n", @@ -980,9 +873,7 @@ { "cell_type": "markdown", "id": "d7eeda5d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## The CART algorithm for Regression\n", "\n", @@ -993,9 +884,7 @@ { "cell_type": "markdown", "id": "f0978df9", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "C(k,t_k) = \\frac{m_{\\mathrm{left}}}{m}\\mathrm{MSE}_{\\mathrm{left}}+ \\frac{m_{\\mathrm{right}}}{m}\\mathrm{MSE}_{\\mathrm{right}}.\n", @@ -1005,9 +894,7 @@ { "cell_type": "markdown", "id": "40902e6c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "Here the MSE for a specific node is defined as" ] @@ -1015,9 +902,7 @@ { "cell_type": "markdown", "id": "091dc8c0", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\mathrm{MSE}_{\\mathrm{node}}=\\frac{1}{m_\\mathrm{node}}\\sum_{i\\in \\mathrm{node}}(\\overline{y}_{\\mathrm{node}}-y_i)^2,\n", @@ -1027,9 +912,7 @@ { "cell_type": "markdown", "id": "49903d3e", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "with" ] @@ -1037,9 +920,7 @@ { "cell_type": "markdown", "id": "b4bff5f6", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\overline{y}_{\\mathrm{node}}=\\frac{1}{m_\\mathrm{node}}\\sum_{i\\in \\mathrm{node}}y_i,\n", @@ -1049,9 +930,7 @@ { "cell_type": "markdown", "id": "cd13ac16", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "the mean value of all observations in a specific node.\n", "\n", @@ -1062,9 +941,7 @@ { "cell_type": "markdown", "id": "ec454ba0", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Why binary splits?\n", "\n", @@ -1077,9 +954,7 @@ { "cell_type": "markdown", "id": "e1419e8a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Computing a Tree using the Gini Index\n", "\n", @@ -1103,9 +978,7 @@ { "cell_type": "markdown", "id": "acf7278e", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## The Table\n", "\n", @@ -1131,9 +1004,7 @@ { "cell_type": "markdown", "id": "cf09301a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Computing the various Gini Indices\n", "\n", @@ -1148,9 +1019,7 @@ { "cell_type": "markdown", "id": "ac1071fd", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Computing the various Gini Indices, Hours slept\n", "\n", @@ -1162,9 +1031,7 @@ { "cell_type": "markdown", "id": "ae964258", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Computing the various Gini Indices, Hours studied\n", "\n", @@ -1178,9 +1045,7 @@ { "cell_type": "markdown", "id": "c50f3fa1", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## A possible code using Scikit-Learn" ] @@ -1189,10 +1054,7 @@ "cell_type": "code", "execution_count": 6, "id": "7550c1fe", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "# Common imports\n", @@ -1260,9 +1122,7 @@ { "cell_type": "markdown", "id": "73371791", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Further example: Computing the Gini index\n", "\n", @@ -1304,9 +1164,7 @@ { "cell_type": "markdown", "id": "07b54f10", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Simple Python Code to read in Data and perform Classification" ] @@ -1315,10 +1173,7 @@ "cell_type": "code", "execution_count": 7, "id": "727954e5", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "# Common imports\n", @@ -1393,9 +1248,7 @@ { "cell_type": "markdown", "id": "ca1867f9", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Computing the Gini Factor\n", "\n", @@ -1411,10 +1264,7 @@ "cell_type": "code", "execution_count": 8, "id": "17d5a20a", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "# Split a dataset based on an attribute and an attribute value\n", @@ -1482,9 +1332,7 @@ { "cell_type": "markdown", "id": "8400e1fe", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Regression trees" ] @@ -1493,10 +1341,7 @@ "cell_type": "code", "execution_count": 9, "id": "8f030fe1", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "# Quadratic training set + noise\n", @@ -1511,10 +1356,7 @@ "cell_type": "code", "execution_count": 10, "id": "7de07d7a", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.tree import DecisionTreeRegressor\n", @@ -1526,9 +1368,7 @@ { "cell_type": "markdown", "id": "1d114a7d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Final regressor code" ] @@ -1537,10 +1377,7 @@ "cell_type": "code", "execution_count": 11, "id": "0cf732d7", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.tree import DecisionTreeRegressor\n", @@ -1587,10 +1424,7 @@ "cell_type": "code", "execution_count": 12, "id": "7713257a", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "tree_reg1 = DecisionTreeRegressor(random_state=42)\n", @@ -1626,9 +1460,7 @@ { "cell_type": "markdown", "id": "3e0b8e66", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Pros and cons of trees, pros\n", "\n", @@ -1650,9 +1482,7 @@ { "cell_type": "markdown", "id": "b73d29d8", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Disadvantages\n", "\n", @@ -1678,9 +1508,7 @@ { "cell_type": "markdown", "id": "86083d18", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods\n", "\n", @@ -1709,9 +1537,7 @@ { "cell_type": "markdown", "id": "66d972ae", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## An Overview of Ensemble Methods\n", "\n", @@ -1725,9 +1551,7 @@ { "cell_type": "markdown", "id": "ca6bf352", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Why Voting?\n", "\n", @@ -1749,9 +1573,7 @@ { "cell_type": "markdown", "id": "94d89cf7", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Tossing coins\n", "\n", @@ -1780,9 +1602,7 @@ { "cell_type": "markdown", "id": "ec65ac92", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Standard imports first" ] @@ -1791,10 +1611,7 @@ "cell_type": "code", "execution_count": 13, "id": "6904dbea", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "# Common imports\n", @@ -1839,9 +1656,7 @@ { "cell_type": "markdown", "id": "5ea8eea1", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Simple Voting Example, head or tail" ] @@ -1850,10 +1665,7 @@ "cell_type": "code", "execution_count": 14, "id": "ed3edd5f", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "\n", @@ -1884,9 +1696,7 @@ { "cell_type": "markdown", "id": "913397e8", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Using the Voting Classifier\n", "\n", @@ -1897,10 +1707,7 @@ "cell_type": "code", "execution_count": 15, "id": "d06e933d", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", @@ -1950,9 +1757,7 @@ { "cell_type": "markdown", "id": "6f392050", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Voting and Bagging" ] @@ -1961,10 +1766,7 @@ "cell_type": "code", "execution_count": 16, "id": "6f49c5ce", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", @@ -1991,10 +1793,7 @@ "cell_type": "code", "execution_count": 17, "id": "fafb0ab3", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import accuracy_score\n", @@ -2009,10 +1808,7 @@ "cell_type": "code", "execution_count": 18, "id": "58bfd587", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "log_clf = LogisticRegression(random_state=42)\n", @@ -2029,10 +1825,7 @@ "cell_type": "code", "execution_count": 19, "id": "a5e2060a", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import accuracy_score\n", @@ -2046,9 +1839,7 @@ { "cell_type": "markdown", "id": "bc47caa6", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Bagging\n", "\n", @@ -2068,9 +1859,7 @@ { "cell_type": "markdown", "id": "7a7ce88c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## More bagging\n", "\n", @@ -2100,9 +1889,7 @@ { "cell_type": "markdown", "id": "3a2bfa3b", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Making your own Bootstrap: Changing the Level of the Decision Tree\n", "\n", @@ -2112,13 +1899,173 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 42, "id": "f5f20a2c", - "metadata": { - "collapsed": false, - "editable": true - }, - "outputs": [], + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Polynomial degree: 1\n", + "Error: 0.1077869092797025\n", + "Bias^2: 0.09665219931376287\n", + "Var: 0.01113470996593965\n", + "0.1077869092797025 >= 0.09665219931376287 + 0.01113470996593965 = 0.10778690927970253\n", + "Polynomial degree: 2\n", + "Error: 0.07073328352158705\n", + "Bias^2: 0.05660265596808508\n", + "Var: 0.01413062755350194\n", + "0.07073328352158705 >= 0.05660265596808508 + 0.01413062755350194 = 0.07073328352158702\n", + "Polynomial degree: 3\n", + "Error: 0.03491402051096134\n", + "Bias^2: 0.025645913680458925\n", + "Var: 0.009268106830502446\n", + "0.03491402051096134 >= 0.025645913680458925 + 0.009268106830502446 = 0.034914020510961374\n", + "Polynomial degree: 4\n", + "Error: 0.03122805721680159\n", + "Bias^2: 0.020459888416421317\n", + "Var: 0.01076816880038027\n", + "0.03122805721680159 >= 0.020459888416421317 + 0.01076816880038027 = 0.031228057216801587\n", + "Polynomial degree: 5\n", + "Error: 0.029360432630668315\n", + "Bias^2: 0.01933417827965857\n", + "Var: 0.010026254351009753\n", + "0.029360432630668315 >= 0.01933417827965857 + 0.010026254351009753 = 0.029360432630668322\n", + "Polynomial degree: 6\n", + "Error: 0.030277866384743817\n", + "Bias^2: 0.020595346780014352\n", + "Var: 0.00968251960472946\n", + "0.030277866384743817 >= 0.020595346780014352 + 0.00968251960472946 = 0.030277866384743814\n", + "Polynomial degree: 7\n", + "Error: 0.030892885889348087\n", + "Bias^2: 0.02014767854486778\n", + "Var: 0.010745207344480309\n", + "0.030892885889348087 >= 0.02014767854486778 + 0.010745207344480309 = 0.03089288588934809\n", + "Polynomial degree: 8\n", + "Error: 0.03110939965212513\n", + "Bias^2: 0.020817454714933955\n", + "Var: 0.010291944937191167\n", + "0.03110939965212513 >= 0.020817454714933955 + 0.010291944937191167 = 0.03110939965212512\n", + "Polynomial degree: 9\n", + "Error: 0.03129578341811742\n", + "Bias^2: 0.020886306078419455\n", + "Var: 0.01040947733969797\n", + "0.03129578341811742 >= 0.020886306078419455 + 0.01040947733969797 = 0.03129578341811742\n", + "Polynomial degree: 10\n", + "Error: 0.031606699705781\n", + "Bias^2: 0.021032809061206647\n", + "Var: 0.010573890644574355\n", + "0.031606699705781 >= 0.021032809061206647 + 0.010573890644574355 = 0.031606699705781005\n", + "Polynomial degree: 11\n", + "Error: 0.03148677989704399\n", + "Bias^2: 0.021152336559874207\n", + "Var: 0.010334443337169787\n", + "0.03148677989704399 >= 0.021152336559874207 + 0.010334443337169787 = 0.031486779897043994\n", + "Polynomial degree: 12\n", + "Error: 0.032116346271443774\n", + "Bias^2: 0.021860481703725315\n", + "Var: 0.010255864567718462\n", + "0.032116346271443774 >= 0.021860481703725315 + 0.010255864567718462 = 0.032116346271443774\n", + "Polynomial degree: 13\n", + "Error: 0.03209100097235015\n", + "Bias^2: 0.02177267731801543\n", + "Var: 0.01031832365433474\n", + "0.03209100097235015 >= 0.02177267731801543 + 0.01031832365433474 = 0.03209100097235017\n", + "Polynomial degree: 14\n", + "Error: 0.031537537813106746\n", + "Bias^2: 0.02082938849956035\n", + "Var: 0.010708149313546408\n", + "0.031537537813106746 >= 0.02082938849956035 + 0.010708149313546408 = 0.03153753781310675\n", + "Polynomial degree: 15\n", + "Error: 0.03145124683797013\n", + "Bias^2: 0.021111527370668133\n", + "Var: 0.010339719467301988\n", + "0.03145124683797013 >= 0.021111527370668133 + 0.010339719467301988 = 0.03145124683797012\n", + "Polynomial degree: 16\n", + "Error: 0.03209105344712722\n", + "Bias^2: 0.021338345379319522\n", + "Var: 0.010752708067807702\n", + "0.03209105344712722 >= 0.021338345379319522 + 0.010752708067807702 = 0.032091053447127225\n", + "Polynomial degree: 17\n", + "Error: 0.03196914233431665\n", + "Bias^2: 0.02166629218762961\n", + "Var: 0.010302850146687045\n", + "0.03196914233431665 >= 0.02166629218762961 + 0.010302850146687045 = 0.03196914233431666\n", + "Polynomial degree: 18\n", + "Error: 0.03165248155309402\n", + "Bias^2: 0.02132799291304245\n", + "Var: 0.010324488640051558\n", + "0.03165248155309402 >= 0.02132799291304245 + 0.010324488640051558 = 0.03165248155309401\n", + "Polynomial degree: 19\n", + "Error: 0.03235209768543048\n", + "Bias^2: 0.022074472952423024\n", + "Var: 0.010277624733007463\n", + "0.03235209768543048 >= 0.022074472952423024 + 0.010277624733007463 = 0.032352097685430486\n", + "Polynomial degree: 20\n", + "Error: 0.03174154546980151\n", + "Bias^2: 0.021139865993692355\n", + "Var: 0.010601679476109165\n", + "0.03174154546980151 >= 0.021139865993692355 + 0.010601679476109165 = 0.03174154546980152\n", + "Polynomial degree: 21\n", + "Error: 0.03249366272788966\n", + "Bias^2: 0.022392964161879643\n", + "Var: 0.010100698566010037\n", + "0.03249366272788966 >= 0.022392964161879643 + 0.010100698566010037 = 0.03249366272788968\n", + "Polynomial degree: 22\n", + "Error: 0.03100387126675639\n", + "Bias^2: 0.02122260145527001\n", + "Var: 0.009781269811486361\n", + "0.03100387126675639 >= 0.02122260145527001 + 0.009781269811486361 = 0.03100387126675637\n", + "Polynomial degree: 23\n", + "Error: 0.0320452032176362\n", + "Bias^2: 0.021849700000696155\n", + "Var: 0.010195503216940061\n", + "0.0320452032176362 >= 0.021849700000696155 + 0.010195503216940061 = 0.032045203217636216\n", + "Polynomial degree: 24\n", + "Error: 0.0315431807559056\n", + "Bias^2: 0.021311765078499363\n", + "Var: 0.01023141567740624\n", + "0.0315431807559056 >= 0.021311765078499363 + 0.01023141567740624 = 0.031543180755905606\n", + "Polynomial degree: 25\n", + "Error: 0.03167739763326645\n", + "Bias^2: 0.020906889239215253\n", + "Var: 0.010770508394051193\n", + "0.03167739763326645 >= 0.020906889239215253 + 0.010770508394051193 = 0.03167739763326645\n", + "Polynomial degree: 26\n", + "Error: 0.03091217303915449\n", + "Bias^2: 0.021114234118115666\n", + "Var: 0.009797938921038828\n", + "0.03091217303915449 >= 0.021114234118115666 + 0.009797938921038828 = 0.030912173039154493\n", + "Polynomial degree: 27\n", + "Error: 0.03181795943603612\n", + "Bias^2: 0.0217612448640803\n", + "Var: 0.01005671457195582\n", + "0.03181795943603612 >= 0.0217612448640803 + 0.01005671457195582 = 0.03181795943603612\n", + "Polynomial degree: 28\n", + "Error: 0.031719847034916156\n", + "Bias^2: 0.021338073424295095\n", + "Var: 0.01038177361062107\n", + "0.031719847034916156 >= 0.021338073424295095 + 0.01038177361062107 = 0.03171984703491616\n", + "Polynomial degree: 29\n", + "Error: 0.03138781693308297\n", + "Bias^2: 0.021135473862578174\n", + "Var: 0.010252343070504813\n", + "0.03138781693308297 >= 0.021135473862578174 + 0.010252343070504813 = 0.031387816933082985\n", + "Simple tree: 0.6586458006727302\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj0AAAGeCAYAAAB/8bDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABedklEQVR4nO3deXwTdcI/8M/kTpomvWihpS1HOeXyxOK60EUEVFbBZRHEXQQvFB9Zn4UVFdDVfcBr1fUBfR6PtY8IeIH+RNCFCniwoKy63AoFSqFI76Rpmnt+f0wyTdq0NKVt2ubz5jWvmcyR+SYdkk++8/3OCKIoiiAiIiLq5hTRLgARERFRR2DoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMYeoiIiCgmqKJdgPbm8/lQUlKC+Ph4CIIQ7eIQERFRC4iiiJqaGqSnp0OhaJs6mm4fekpKSpCZmRntYhAREVErFBcXo3fv3m3yXN0+9MTHxwOQ3jSTyRTl0hAREVFLWK1WZGZmyt/jbaHbh57AKS2TycTQQ0RE1MW0ZdMUNmQmIiKimMDQQ0RERDGBoYeIiIhiQrdv00NERLFDFEV4PB54vd5oF4XOQ6lUQqVSdejlZBh6iIioW3C5XDh79izsdnu0i0ItZDAY0KtXL2g0mg7ZH0MPERF1eT6fDydOnIBSqUR6ejo0Gg0vSNuJiaIIl8uFsrIynDhxAgMGDGizCxA2h6GHiIi6PJfLBZ/Ph8zMTBgMhmgXh1pAr9dDrVajqKgILpcLOp2u3ffJhsxERNRtdERtAbWdjv578eggIiKimMDQQ0RERDGBoYeIiIhiAkMPERFRFM2ZMweCIOCee+5ptOzee++FIAiYM2cOAKC0tBR33303srKyoNVq0bNnT0ycOBH//Oc/5W369OkDQRAaDStXruyol9RpsfcWERFRlGVmZmL9+vV4/vnnodfrAQAOhwPr1q1DVlaWvN7NN98Mt9uN/Px89OvXD+fOnUNBQQEqKytDnu/Pf/4z7rzzzpB5bXm38q6KoYeIiLodURRR547OVZn1amXE1wi65JJLcPz4cWzYsAG33norAGDDhg3IzMxEv379AADV1dX46quvsGPHDowdOxYAkJ2djSuuuKLR88XHx6Nnz54X+Eq6n5gJPTUON0ymaJeCiIg6Qp3bi6HLPovKvg/9eSIMmsi/Xm+//Xb8/e9/l0PPG2+8gblz52LHjh0AAKPRCKPRiA8//BBXXnkltFptWxY7JsRMm55TFbwsORERdV633XYbvvrqK5w8eRJFRUX4+uuvMXv2bHm5SqXCm2++ifz8fCQkJOCqq67Cww8/jH379jV6rj/96U9ySAoMgfAUy2KmpudkRS1yo10IIiLqEHq1Eof+PDFq+26NlJQUXH/99cjPz4coirj++uuRkpISss7NN9+M66+/Hl9++SX++c9/4tNPP8XTTz+N1157TW7sDACLFi0KeQwAGRkZrSpXdxIzoaeINT1ERDFDEIRWnWKKtrlz52LBggUAgFWrVoVdR6fTYcKECZgwYQKWLVuGO+64A8uXLw8JOSkpKcjJyemIIncpMXN662RFbbSLQERE1KxJkybB5XLB5XJh4sSW1VQNHToUtbX8jmuJrheDW4k1PURE1NkplUocPnxYng5WUVGB6dOnY+7cuRgxYgTi4+Oxd+9ePP3007jxxhtD1q2pqcHPP/8cMs9gMMAU4z16Yij01EIUxYi7ERIREXWkpoKJ0WjE6NGj8fzzz6OwsBButxuZmZm488478fDDD4esu2zZMixbtixk3t13341XXnml3crdFQiiKIrRLkR7slqtMJvNyFz4Lr574tdIMbKLHxFRd+NwOHDixAn07dsXOp0u2sWhFmru7xb4/rZYLG1WQxUzbXoA4EQ5z3kSERHFqtgKPWUMPURERLEqpkLPcdb0EBERxayYCj0nym3RLgIRERFFSYyFHtb0EBERxaqYCj0nK+zw+rp1ZzUiIiJqQsyEHrVSAZfHh5LqumgXhYiIiKIgZkJPZpIeAE9xERERxaqYCT3ZyQYADD1ERNR5jBs3DgsXLmxyeZ8+ffDCCy90WHm6u5i5DUWf5DjsPFHL0ENERF3Gt99+i7i4uGgXo9uIuKbHZrNh4cKFSE9Ph06nw6hRo7B+/frzbnf69GksXLgQY8eORUJCAgRBwJtvvtnk+tu2bUNubi4MBgNSUlIwZ84clJaWRlpcWXYSa3qIiKhr6dGjBwwGQ7SL0W1EHHqmTZuG/Px8LF++HFu2bMHll1+OmTNnYu3atc1ud+zYMbz99tvQaDS47rrrml13586dmDx5MtLS0vDRRx/hxRdfxLZt2zB+/Hg4nc5IiwxAqukBGHqIiKhz8Xg8WLBgARISEpCcnIxHH30UgdtiNjy99de//hXDhw9HXFwcMjMzce+998Jmq78GXVFREaZMmYLExETExcXhoosuwubNmzv6JXVaEZ3e2rx5M7Zu3Yq1a9di5syZAIC8vDwUFRVh0aJFmDFjBpRKZdhtf/nLX6KsrAwAsHfvXqxbt67J/SxatAgDBw7E+++/D5VKKmLfvn1x1VVX4Y033sD8+fMjKTYAIDtFSsqnq+xwerzQqsKXk4iIugFRBNz26OxbbQAEocWr5+fnY968edizZw/27t2Lu+66C9nZ2bjzzjsbratQKPC3v/0Nffr0wYkTJ3Dvvfdi8eLFWL16NQDgvvvug8vlwhdffIG4uDgcOnQIRqOxzV5aVxdR6Nm4cSOMRiOmT58eMv/222/HrFmzsGfPHowZMybstgpFyyqVzpw5g2+//RYrVqyQAw8AjBkzBgMHDsTGjRtbFXpSjFoYtSrYnB4UV9qRkxof8XMQEVEX4bYD/5UenX0/XAJoWt4OJzMzE88//zwEQcCgQYOwf/9+PP/882FDT3Cj5759++KJJ57A/Pnz5dBz6tQp3HzzzRg+fDgAoF+/fhf2WrqZiE5vHThwAEOGDAkJIwAwYsQIefmFCjxH4Dkb7qe1+xAEAX1TpIPwOG88SkREncSVV14JIahmKDc3F0ePHoXX62207vbt2zFhwgRkZGQgPj4ev/vd71BRUYHaWul77T/+4z/w5JNP4qqrrsLy5cuxb9++DnsdXUFENT0VFRVhU2NSUpK8/EIFniPwnA33c759OJ3OkHY/VqtVnu6bEof9Zyxs10NE1N2pDVKNS7T23Q6Kiopw3XXX4Z577sETTzyBpKQkfPXVV5g3bx7cbjcA4I477sDEiRPxySef4B//+AdWrFiB5557Dvfff3+7lKmribjLutDMecrmlrXVfs63jxUrVuDxxx8PuyxQ08PQQ0TUzQlCRKeYomn37t2NHg8YMKBRG9m9e/fC4/Hgueeek5uMvPvuu42eLzMzE/fccw/uueceLFmyBK+++ipDj19Ep7eSk5PD1rRUVlYCCF87E6nk5GQA4WuNKisrz7uPJUuWwGKxyENxcbG8rF8P/+kthh4iIuokiouL8eCDD+LHH3/EunXr8NJLL+GBBx5otF7//v3h8Xjw0ksv4fjx43jrrbfwyiuvhKyzcOFCfPbZZzhx4gS+++47fP755xgyZEhHvZROL6LQM3z4cBw+fBgejydk/v79+wEAw4YNu+ACBZ4j8JwN93O+fWi1WphMppAhgN3WiYios/nd736Huro6XHHFFbjvvvtw//3346677mq03qhRo/DXv/4VTz31FIYNG4a3334bK1asCFnH6/Xivvvuw5AhQzBp0iQMGjRIbuRMgCAGLgbQAlu2bMF1112H9evXY8aMGfL8yZMnY9++fTh16lSTXdaD7d27F5dffjn+/ve/Y86cOY2Wjx49Gna7HT/88IP8fLt370Zubi5efvll3HPPPS0tMqxWK8xmMywWC0S1HiMf/wcAYP9j1yJep27x8xARUeflcDhw4sQJ9O3bFzqdLtrFoRZq7u8W/P0dXIFxISJq0zN58mRMmDAB8+fPh9VqRU5ODtatW4dPP/0Ua9askQPKvHnzkJ+fj8LCQmRnZ8vbv//++wCA48ePA5DCT+D6Ab/5zW/k9Z566ilMmDAB06dPx7333ovS0lI89NBDGDZsGG6//fZWv1izXo0UowblNheKKuwYlmFu9XMRERFR1xJxQ+YNGzbgkUcewbJly1BZWYnBgwdj3bp1uOWWW+R1vF4vvF4vGlYiNby+z6pVq7Bq1SoACFl33Lhx2Lx5M5YtW4YpU6bAYDDghhtuwDPPPAOtVhtpkUP0TYlDuc2F4+W1DD1EREQxJKLTW11Rw+qxxe//G+/uPY0/XDMQD1wzINrFIyKiNsDTW11TR5/eivjeW11d3xTpdNqJctt51iQiIqLuJAZDD3twERERxaKYCz3B1+rp5mf2iIiIKEjshB5/wMlKMkAQgBqHBxW1rigXioiIiDpK7IQeexUAQKdWIiNBD4CnuIiIiGJJ7ISe6pPypNyuh3dbJyIiihmxE3qqiuTJQOjhPbiIiKg7EgQBH374YbSL0enETuipbhx6TjL0EBFRlEyZMgXXXHNN2GX//Oc/IQgCvvvuu1Y999mzZzF58uQLKV63FDuhJ0xND9v0EBFRtMybNw+ff/45ioqKGi174403MGrUKFxyySURPafLJXXQ6dmz5wXfwaA7ip3QE1TT0y9wgcKKWvh87LZOREQd74YbbkBqairefPPNkPl2ux3vvPMObrrpJsycORO9e/eGwWDA8OHDsW7dupB1x40bhwULFuDBBx9ESkoKJkyYAKDx6a0//elPGDhwIAwGA/r164elS5fC7XbLyx977DGMGjUKb731Fvr06QOz2YxbbrkFNTU18jo+nw9PPfUUcnJyoNVqkZWVhb/85S/y8jNnzmDGjBlITExEcnIybrzxRpw8ebLt3rA2EDuhp+qkPJmRqIdaKcDl8aHEUhe9MhERUbsQRRF2tz0qQ0uvAadSqfC73/0Ob775Zsg27733HlwuF+644w5ceuml2LRpEw4cOIC77roLt912G/bs2RPyPPn5+VCpVPj666/xP//zP2H3FR8fjzfffBOHDh3Ciy++iFdffRXPP/98yDqFhYX48MMPsWnTJmzatAk7d+7EypUr5eVLlizBU089haVLl+LQoUNYu3Yt0tLSAEhBLS8vD0ajEV988QW++uorGI1GTJo0Sa596gxi595bD8XD9PhZQCOd2rrmrztxrNSGt+ZdgasH9IhyKYmI6EI0vIeT3W3H6LWjo1KWPbP2wKA2tGjdI0eOYMiQIfj888+Rl5cHABg7diwyMjKwdu3aRutff/31GDJkCJ599lkAUk2PxWLB999/H7KeIAjYuHEjbrrpprD7feaZZ/DOO+9g7969AKSanmeeeQY///wz4uPjAQCLFy/GF198gd27d6OmpgY9evTAf//3f+OOO+5o9HxvvPEGnn76aRw+fBiCIACQTrUlJCTgww8/xLXXXhu2HB19762I77LepVWdBNIuAiC16zlWasOJ8lqGHiIiiorBgwdjzJgxeOONN5CXl4fCwkJ8+eWX+Mc//gGv14uVK1finXfewZkzZ+B0OuF0OhEXFxfyHJdddtl59/P+++/jhRdewLFjx2Cz2eDxeBoFiT59+siBBwB69eqF0tJSAMDhw4fhdDoxfvz4sM//r3/9C8eOHQvZHpBCTWFhYYvei44QW6Gn8oQcevoFuq3zWj1ERN2OXqXHnll7zr9iO+07EvPmzcOCBQuwatUq/P3vf0d2djbGjx+PZ555Bs8//zxeeOEFDB8+HHFxcVi4cGGj00UNQ1BDu3fvxi233ILHH38cEydOhNlsxvr16/Hcc8+FrKdWq0MeC4IAn88nvSZ986/J5/Ph0ksvxdtvv91oWY8enadiIbZCT9UJeZI9uIiIui9BEFp8iinafvvb3+KBBx7A2rVrkZ+fjzvvvBOCIODLL7/EjTfeiNmzZwOQgsXRo0cxZMiQiJ7/66+/RnZ2Nh555BF5XrgeY80ZMGAA9Ho9CgoKwp7euuSSS/DOO+8gNTW1zU5FtYfYacgMSDU9fgw9RETUGRiNRsyYMQMPP/wwSkpKMGfOHABATk4Otm7dil27duHw4cO4++678fPPP0f8/Dk5OTh16hTWr1+PwsJC/O1vf8PGjRsjeg6dToc//elPWLx4Mf7v//4PhYWF2L17N15//XUAwK233oqUlBTceOON+PLLL3HixAns3LkTDzzwAE6fPh1xmdtLbIWeMDU9p6vscHl80SoRERER5s2bh6qqKlxzzTXIysoCACxduhSXXHIJJk6ciHHjxqFnz55NNkxuzo033og//OEPWLBgAUaNGoVdu3Zh6dKlET/P0qVL8Z//+Z9YtmwZhgwZghkzZshtfgwGA7744gtkZWVh2rRpGDJkCObOnYu6urpOVfMTW723evYDHvgBgNSdcdjyz1Dr8mLbg2ORk2qMbkGJiKjVmusFRJ1XR/feiq2aHksx4PUAkM739u3BU1xERESxInZCj0ID+DxS8PHrG7gyc7ktWqUiIiKiDhI7oSchWxqzBxcREVFMip3Qk+gPPUE9uHitHiIiotgRe6GHNT1EREQxKfZCT1BNTx9/6CmtccLm9ESjVERE1Ia6eYfkbqej/16xE3rkNj0n5VlmvRopRg0A4CRre4iIuqzALRTsdnuUS0KRCPy9Gt4Co73Ezm0oEvtI48oTgCgC/rvA9k2JQ7nNhePltRiWYY5e+YiIqNWUSiUSEhJCLpYXuNs3dT6iKMJut6O0tBQJCQlQKpUdst/YCT3m3gAEwF0L2EqB+DQAUuj59mQVTrAxMxFRl9azZ08AkIMPdX4JCQny360jxE7oUWkBcyZgOSU1ZvaHnj5yY2Zeq4eIqCsTBAG9evVCamoq3G53tItD56FWqzushicgdkIPACT1kUJP5Qkg60oA9d3WT1TwPDARUXegVCo7/MuUuobYacgMAIl9pXFIt3X/VZnLbGz1T0RE1I3FVuhJ8oeeoG7r2ckGCAJgdXhQWeuKUsGIiIiovcVW6AlT06NTK5Fu1gPgRQqJiIi6s9gKPWFqegCgn/9u68cZeoiIiLqt2Ao9gZoeezngrJFn83YURERE3V9shR6dCTAkS9OVYe7BxWv1EBERdVuxFXqAJnpwsaaHiIiou4u90BOmXU+/QLf1ilr4fOy2TkRE1B3FXugJU9OTkaiHWinA5fGhxFIXpYIRERFRe4q90BOmpkepEJCVZAAAnCznlZmJiIi6o9gLPWFqeoCgKzPzHlxERETdUuyFnkBNj+U04Km/AjOv1UNERNS9xV7oMaYBagMg+oDqU/Js9uAiIiLq3mIv9AgCu60TERHFoNgLPUAT3dal0FNcaYfL44tGqYiIiKgdxWboSewjjYNqenrEaxGnUcInAqcq2YOLiIiou4nN0BOmpkcQBPTtwVNcRERE3VVshh52WyciIoo5sRl6AjU9VScBX337HTZmJiIi6r5iM/SYMwFBCXgcgO1neXbfFOmqzAw9RERE3U/Eocdms2HhwoVIT0+HTqfDqFGjsH79+hZtW1paijlz5iAlJQUGgwG5ubkoKChotJ7T6cQzzzyDYcOGIS4uDmlpaZg8eTJ27doVaXHDU6qBhExpujK423rg9BZDDxERUXcTceiZNm0a8vPzsXz5cmzZsgWXX345Zs6cibVr1za7ndPpxPjx41FQUIAXX3wRH330EdLS0jBp0iTs3LkzZN0777wTDz30EG666SZ8/PHHWLVqFcrKyjB27Fh88803kRY5vHDX6kmWTm+dszpR6/S0zX6IiIioU1BFsvLmzZuxdetWrF27FjNnzgQA5OXloaioCIsWLcKMGTOgVCrDbvv666/jwIED2LVrF3Jzc+VtR44cicWLF2PPnj0ApHC0du1azJo1C08++aS8/VVXXYX09HS8/fbbuOKKK1r1YkMk9QWObw+p6TEb1EiO06Ci1oUT5bUYlmG+8P0QERFRpxBRTc/GjRthNBoxffr0kPm33347SkpK5ODS1LaDBg2SAw8AqFQqzJ49G9988w3OnDkjFUihgEKhgNkcGjhMJhMUCgV0Ol0kRW5akz242JiZiIioO4oo9Bw4cABDhgyBShVaQTRixAh5eXPbBtYLt+3BgwcBAGq1Gvfeey/y8/Px4Ycfwmq14uTJk7jzzjthNptx5513RlLkpsnX6jkeMpuhh4iIqHuK6PRWRUUF+vXr12h+UlKSvLy5bQPrnW/b559/HmazGTfffDN8/i7lWVlZ+Pzzz5GTk9NsGZ1OJ5xOp/zYarWGXzGx8QUKAfAChURERN1UxA2ZBUFo1bJItv3LX/6CZ599Fo899hi2b9+Ojz76CIMGDcKECRPw/fffN7uPFStWwGw2y0NmZmb4FQM1PY5qoK5Knh24B9dxhh4iIqJuJaLQk5ycHLY2p7KyEgDC1uREuu3hw4exbNkyPP7441i6dCnGjRuHX//61/jkk0+QkJCABx98sNkyLlmyBBaLRR6Ki4vDr6iJA4xp/kKE6bZeZoMois3ui4iIiLqOiELP8OHDcfjwYXg8od259+/fDwAYNmxYs9sG1mtu23//+98QRRGXX355yHpqtRojR45stt0QAGi1WphMppChSWEaM2cnGyAIgNXhQWWtq9l9ERERUdcRUeiZOnUqbDYbPvjgg5D5+fn5SE9Px+jRo5vd9siRIyE9vDweD9asWYPRo0cjPT0dAOTx7t27Q7Z3Op347rvv0Lt370iK3LwwNx7VqZVIN+sBsF0PERFRdxJRQ+bJkydjwoQJmD9/PqxWK3JycrBu3Tp8+umnWLNmjXyNnnnz5iE/Px+FhYXIzs4GAMydOxerVq3C9OnTsXLlSqSmpmL16tX48ccfsW3bNnkfv/jFL3D55Zfjscceg91uxy9/+UtYLBa89NJLOHHiBN566622e/XNdFs/U12HE+W1uKxP06fsiIiIqOuIKPQAwIYNG/DII49g2bJlqKysxODBg7Fu3Trccsst8jperxderzekTYxWq0VBQQEWL16M+++/H3a7HaNGjcKWLVswduxYeT2FQoGtW7fimWeewXvvvYdnn30WRqMRQ4cOxebNmzF58uQLfMlB5JqekyGz+6bE4atj5azpISIi6kYEsZu31rVarTCbzbBYLI3b9xR/C7x+DWDKAB48JM9+46sT+POmQ5g8rCdenn1pB5eYiIiImv3+bqXYvMt6QKCmx1oCuB3ybF6rh4iIqPuJ7dBjSAY08QBEoLpInt0v6KrMPl+3rggjIiKKGbEdegQBSOojTQf14MpI0EOtFOD0+HDW6gi/LREREXUpsR16gLA9uFRKBbKSDACAE2U8xUVERNQdMPSEuVYPEHRl5nJbR5eIiIiI2gFDT2L4u63368F7cBEREXUnDD1JTV+gEGAPLiIiou6CoSepnzSuKgJ8Xnl2n2Qp9Jxk6CEiIuoWGHpMGYBCDfjcgPWMPDtwequ4qg4ujy9apSMiIqI2wtCjUAKJ0v3Bghszp8ZrYdAo4fWJKK6yR6lwRERE1FYYeoCw3dYFQahv18Nu60RERF0eQw/QTLd1NmYmIiLqLhh6gLA1PUD97SjYbZ2IiKjrY+gBmq7pkW88ygsUEhERdXUMPUBQTc9JQKy/wWj9VZlZ00NERNTVMfQA9b23nFbAXinP7uu/Vs85qxO1Tk80SkZERERthKEHANR6ID5dmg5q12M2qJEcpwHA2h4iIqKujqEn4Dw9uE5WMPQQERF1ZQw9AU3ceLQPr9VDRETULTD0BCT1kca88SgREVG3xNATkBj+9Bav1UNERNQ9MPQEyHdbD3+tnuNlNohB3dmJiIioa2HoCQg0ZLadA1z1tTp9kuMgCIDV4UFlrStKhSMiIqILxdAToE8EdAnSdNVJebZOrUS6WQ+Ap7iIiIi6MoaeYE10W++fKl2ZubCUt6MgIiLqqhh6gvHGo0RERN0WQ0+wpmp6ghozExERUdfE0BOsqZqeHtLpreO8QCEREVGXxdATrImann7+mp6iSjtcHl9Hl4qIiIjaAENPsEBNj6UY8NbfVb2nSQeDRgmvT8SpSnuUCkdEREQXgqEnWHwvQKkFfB4p+PgJgiDX9rBdDxERUdfE0BNMoQAS+0jTjXpw+dv1sAcXERFRl8TQ01BS+Luts6aHiIioa2PoaaipG4/6e3AVsgcXERFRl8TQ01CgpifoVhQAr9VDRETU1TH0NBS423qDmp6+/qsyV9ndqOKNR4mIiLochp6GEoNqekRRnm3QqJBu1gEAjpeztoeIiKirYehpKCELEBSAuxawlYYsYrseIiKirouhpyGVBjD1lqarwt+Dq5DteoiIiLochp5wkvpI4yZ6cPEeXERERF0PQ084Td54lD24iIiIuiqGnnCavPGoVNNzqtIOt5c3HiUiIupKGHrCaaKmp5dJB51aAbdXRDFvPEpERNSlMPSE00RNj0Ih1N+Di+16iIiIuhSGnnACNT32csBZE7JIbtfDa/UQERF1KQw94ehMgCFZmmYPLiIiom6BoacpieHvtl5/Dy6GHiIioq6EoacpSU10W08JXJWZp7eIiIi6EoaepiQ21W1dqumpqHXBYnd3dKmIiIiolRh6mhK423qDmp44rQo9TdKNRwvZmJmIiKjLiDj02Gw2LFy4EOnp6dDpdBg1ahTWr1/fom1LS0sxZ84cpKSkwGAwIDc3FwUFBWHXra2txbJlyzBw4EBotVokJycjLy8PR48ejbTIrSN3Wz/ZaFE/tushIiLqclSRbjBt2jR8++23WLlyJQYOHIi1a9di5syZ8Pl8mDVrVpPbOZ1OjB8/HtXV1XjxxReRmpqKVatWYdKkSdi2bRvGjh0rr2uz2ZCXl4eSkhI89NBDGDFiBCwWC3bt2gW7vYMuChg4vWU9DXhc0o1I/fr1iMOuwgq26yEiIupCIgo9mzdvxtatW+WgAwB5eXkoKirCokWLMGPGDCiVyrDbvv766zhw4AB27dqF3NxceduRI0di8eLF2LNnj7zuo48+isOHD2Pfvn3o16+fPP/Xv/51xC+w1YypgDoOcNcC1aeAlBx5Uf0FChl6iIiIuoqITm9t3LgRRqMR06dPD5l/++23o6SkJCS4hNt20KBBcuABAJVKhdmzZ+Obb77BmTNnAAB2ux2vvfYapk+fHhJ4OpwgAIl9pOkG7Xr6p/JaPURERF1NRKHnwIEDGDJkCFSq0AqiESNGyMub2zawXrhtDx48CAD417/+hdraWgwYMADz589HYmIiNBoNLrvsMnzyySfnLaPT6YTVag0ZWq2pG4+mSG16iirs8PrE1j8/ERERdZiIQk9FRQWSkpIazQ/Mq6iouOBtAzU+Tz31FPbv34//+7//w8aNG2EymTBlyhR89tlnzZZxxYoVMJvN8pCZmdmyFxdOEzU9GQl6aFUKuLw+nK7ijUeJiIi6goh7bwmC0KplLd3W5/MBADQaDbZs2YIpU6bg+uuvx6ZNm9CrVy888cQTze5jyZIlsFgs8lBcXNzs+s1q5sajff21PWzMTERE1DVEFHqSk5PD1uZUVlYCQNianEi3TU6W7nk1ZswYxMfHy+sZDAaMHTsW3333XbNl1Gq1MJlMIUOrJYa/KjPAbutERERdTUShZ/jw4Th8+DA8Hk/I/P379wMAhg0b1uy2gfWa2zZcu58AURShUHTg9RTlW1GcBPw1UAH9ewRuR8HQQ0RE1BVElCCmTp0Km82GDz74IGR+fn4+0tPTMXr06Ga3PXLkSEgPL4/HgzVr1mD06NFIT08HAPTq1Qu5ubn4+uuvQxoh2+127Ny5E1deeWUkRb4w5kxAUAIeB1BzNmRRfU0PT28RERF1BRGFnsmTJ2PChAmYP38+Xn31VWzfvh133XUXPv30Uzz99NPyNXrmzZsHlUqFoqIiedu5c+fioosuwvTp07F27Vps27YNv/3tb/Hjjz/iqaeeCtnPs88+i5qaGkycOBEffvghPvroI0yaNAnl5eXnbdPTppRqIMHfELqJG48eL2dNDxERUVcQ8bmiDRs24LbbbsOyZcswadIk7NmzB+vWrcOtt94qr+P1euH1eiGK9d25tVotCgoKkJeXh/vvvx9TpkzB2bNnsWXLlpCrMQNSe56CggJotVrceuutmDVrFtRqNXbs2BFynZ8OcZ4bj5bVOGF18MajREREnZ0gBieTbshqtcJsNsNisbSuUfOmPwB73wCu/k9g/LKQRVf8ZRtKa5z48L6rMCozoW0KTERERBf+/R0G77J+PoG7rVc214OL7XqIiIg6O4ae82m22zpvR0FERNRVMPScTxMXKATqb0fBCxQSERF1fgw955PYF4AAOKoBW2nIov6s6SEiIuoyGHrOR2Oob9dzLvSGqoHQc6KiljceJSIi6uQYeloi7SJpfO5QyOyMRD00KgVcHh9KquuiUDAiIiJqKYaelgiEntLQ0KNUCOiTbADAdj1ERESdHUNPS6QOlcYNTm8B9Vdm5j24iIiIOjeGnpYI1PSU/Qh4Q2+2ymv1EBERdQ0MPS2R2BdQG6Qbj1YeD1nEHlxERERdA0NPSygUQOoQabr0YMgiuaannDU9REREnRlDT0vJ7XpCGzMHrsp8zuqEzelpuBURERF1Egw9LSV3Ww+t6THr1UgxagCwXQ8REVFnxtDTUnK39YONFgV6cLFdDxERUefF0NNSqf7QU3UScIbW6PRPZQ8uIiKizo6hp6XikgFjmjRdejhkkXytnnLW9BAREXVWDD2RaOIUV6AHV2Epa3qIiIg6K4aeSMg9uBqGHqmm52RFLXy88SgREVGnxNATibRh0rhBt/XMRD3USgEOtw8lFt54lIiIqDNi6IlEmr+mp/QgINbX6KiUCmQnBxozs10PERFRZ8TQE4mUQYCgBOqqgJqzIYv6pbAHFxERUWfG0BMJtQ5IzpGmm7gyM++2TkRE1Dkx9EQq+BRXEN6Di4iIqHNj6IlUavjbUfBu60RERJ0bQ0+k5HtwhZ7e6u+v6TlrcaCWNx4lIiLqdBh6IhU4vVV2BPC65dkJBg2S4qQbj57glZmJiIg6HYaeSJmzAI0R8LmBimMhiwI9uArZg4uIiKjTYeiJlELRzJWZea0eIiKizoqhpzXSwoceuTEzT28RERF1Ogw9rRG4HUVp+Gv18AKFREREnQ9DT2vIp7cahp7601u88SgREVHnwtDTGoHTW5ZTgMMiz85KMkClEFDn9uJnqyNKhSMiIqJwGHpaQ58ImDKk6dLD8my1UoGsJAMANmYmIiLqbBh6WqvJHlyBxsxs10NERNSZMPS0VpM9uNhtnYiIqDNi6GmtJntw8QKFREREnRFDT2sF9+AS63tq9eONR4mIiDolhp7WShkIKFSA0wJYTsuzA7eiOFNdhzqXN1qlIyIiogYYelpLpZGCDxByiispToMEgxoAbzxKRETUmTD0XAj5FNcBeZYgCLzxKBERUSfE0HMh0i6Sxo2uzMx2PURERJ0NQ8+FCISeJnpw8Vo9REREnQdDz4UInN4q/wnwuOTZ/VJY00NERNTZMPRcCHNvQGsGfB4p+PjlpAYuUGiDKPLGo0RERJ0BQ8+FEISwV2bOSoqDUiGg1uXFOaszSoUjIiKiYAw9F0pu11MfejQqBTIT9QCk2h4iIiKKPoaeCxV8ZeYggR5chbxWDxERUafA0HOh5G7rTd14lDU9REREnQFDz4VKHSKNa0qAuip5tlzTwx5cREREnULEocdms2HhwoVIT0+HTqfDqFGjsH79+hZtW1paijlz5iAlJQUGgwG5ubkoKChodpu6ujoMHDgQgiDg2WefjbS47U9nBsxZ0nTQKa7AVZlZ00NERNQ5RBx6pk2bhvz8fCxfvhxbtmzB5ZdfjpkzZ2Lt2rXNbud0OjF+/HgUFBTgxRdfxEcffYS0tDRMmjQJO3fubHK7pUuXora2k9eWhOnBFajpOVNdB4ebNx4lIiKKNlUkK2/evBlbt27F2rVrMXPmTABAXl4eioqKsGjRIsyYMQNKpTLstq+//joOHDiAXbt2ITc3V9525MiRWLx4Mfbs2dNom2+++QYvvfQS3n77bUyfPj3S19Zx0i4Cfvo0pAdXilGDeJ0KNQ4PTlbUYnBPUxQLSERERBHV9GzcuBFGo7FRALn99ttRUlISNrgEbzto0CA58ACASqXC7Nmz8c033+DMmTMh67tcLsydOxf33XcfLrvsskiK2fFSG9f0CIKA/rwHFxERUacRUeg5cOAAhgwZApUqtIJoxIgR8vLmtg2sF27bgwdDez/9+c9/Rm1tLZ544olIihgd8rV6DgM+nzw7cA+uwlK26yEiIoq2iE5vVVRUoF+/fo3mJyUlycub2zaw3vm2/eGHH/D000/j448/RlxcHMrKylpcRqfTCaez/irIVqu1xdu2WnIOoNQALhtgOQUk9gGA+poeXquHiIgo6iJuyCwIQquWtXRbj8eDuXPnYsaMGZg4cWKkxcOKFStgNpvlITMzM+LniJhSDaQMkqaDGzOzBxcREVGnEVHoSU5ODlubU1lZCQBha3Ii3faFF17A8ePHsXz5clRXV6O6ulqurXE4HKiurobX23RvqCVLlsBischDcXFxy1/ghZAvUhjUbT2oTQ9vPEpERBRdEYWe4cOH4/Dhw/B4PCHz9+/fDwAYNmxYs9sG1mtu2wMHDsBisWDAgAFITExEYmIiRo4cCUDqvp6YmBj2eQK0Wi1MJlPI0CEC3daDenBlJxugEIAapwdlNt54lIiIKJoiCj1Tp06FzWbDBx98EDI/Pz8f6enpGD16dLPbHjlyJKSHl8fjwZo1azB69Gikp6cDAB566CFs3749ZFi3bh0A4J577sH27duRk5MTSbE7Rmrj21Ho1Er0TjQAAApL2a6HiIgomiJqyDx58mRMmDAB8+fPh9VqRU5ODtatW4dPP/0Ua9aska/RM2/ePOTn56OwsBDZ2dkAgLlz52LVqlWYPn06Vq5cidTUVKxevRo//vgjtm3bJu9j8ODBGDx4cMh+T548CQDo378/xo0bdwEvtx0FTm9VFAJuB6DWAZB6cJ2qtON4uQ25/ZOjWEAiIqLYFnFD5g0bNuC2227DsmXLMGnSJOzZswfr1q3DrbfeKq/j9Xrh9XpD2rFotVoUFBQgLy8P999/P6ZMmYKzZ89iy5YtGDt2bNu8mmiK7wnoEwHRC5T/KM/ul8Jr9RAREXUGgtjNW9harVaYzWZYLJb2b9/z9+uBoq+Am14GRs0CAKzZXYRHPzyAvEE98Pfbr2jf/RMREXUT7fH9zbust6W0xu16+vNu60RERJ0CQ09bCnPj0f7+qzKfrrLD6eGNR4mIiKKFoactpfm77JfWX6unR7wWRq0KPhEoqrBHqWBERETE0NOWevh7ndnOAbXlAKQrTQfuwcUrMxMREUUPQ09b0hrl+26Fux0F2/UQERFFD0NPWwtziqu+MTNreoiIiKKFoaetpTZuzBx8Dy4iIiKKDoaethamB1dwm55uflkkIiKiTouhp60FTm+VHQF8Uhf1vilxUCoEWB0eLHznB1TVuqJYQCIiotjE0NPWkvoBKh3gtgNVJwFINx5dMnkwFALw0Q8luPaFL7D10LnolpOIiCjGMPS0NYUS6DFImg46xXXH1f3wwfwx6N8jDmU1Ttz5f3vx4Ls/wGJ3R6mgREREsYWhpz2E6cEFABdnJeKT/7gad/2yHwQB2PDdGVz7wk5sP1IahUISERHFFoae9iD34DrQaJFOrcTD1w3B+/fkom9KHM5Znbj9zW+x+P1/w+pgrQ8REVF7YehpD/KNRw81ucql2UnY/B9XY94v+kIQgHf3nsbE57/AFz+VdVAhiYiIYgtDT3sIhJ7K44Cr6ftt6TVKLL1hKN65KxfZyQactTjwuze+wZIN+2FzejqosERERLFBELv5hWOsVivMZjMsFgtMJlPH7fjp/oC9HLjzcyDj0vOubnd58PSnP+LNXScBABkJejz9mxG4KielnQtKRJ2Z0+NFSbUDxZV2nLXUwahVo6dZh15mHVLjtVAp2++3q9cnoqS6Dqcq7SiqsKOoshanKuw4VWlHgkGNX+T0wNUDUjC0lwkKhdBu5aDY1B7f3ww97SX/18CJncCv/xu45LYWb/bPwgosev/fOF1VBwC47cpsPDR5MOK0qvYqKXVRVocbJdV1OFvtQImlTp6udXkgQIAgQBrkaQECAvP8j4OXo359vUaJNJMOaSatPO4Rr4NJp4IgdN4vN7vLg5Jqh/ReWOpwptqBcpsTKoUAlUIBtUqARqkImVYrFVApBaiVCmlZ0HRgmUalgFGrQrxOhXidGga1ss2+5D1eH85aHCiusuN0VR1OV9pRXFWH01V2FFfW4VyNA019SisEIDVeh14JUgjqZdbL455mHdITdOhhbD4YOdze+lBTUStPn6q043SVHW7v+b8ikuI0uConBVcPkIZeZn1r344W8Xh9OF5ei4MlFljsbiQbtegRr0WKUYseRi1M+s59nFLLMPS0QtRCz6dLgN2rgdHzgckrI9q01unBii2HsWb3KQBAZpIez/xmJK7sl9weJaUL5Pb6YHN4IAJQ+78w1UoFlBfwpehwe3HW4sDZ6jqUWOq/xOu/0B1ROQWqUyukEBSvQ6pJi9T4+mAU/NiobfsvHa9PRFmNE2eq60Lej8Djkuo6VHXQJSAEATBqVTDp1IjXqUICUbxOBaOufpm0XA2DRonSGgeKK+sDTXGVHWctDnh9zX8M69VK9E7UIz1BLwe7c1YHPOfZDmgcjHqa9KhxuOWam3NWZ7Pba5QK9E7So09yHLKSDMhONiAz0YAz1XX48mgZ/llYgVqXN2SbnFQjfpGTgl8OTMHovskX9KPN7vLg8NkaHDprxaESCw6VWHHk5xo4Pb5my5xi1NQHIf9YmqerXxavRXyYY1UURbi9ItxeH1wenzT2+uD2iqGPPf55Xi8EQUBmoh69Ew3QqZWtfr2REEXp/8RP52w4WlqDo6U2nKqwQ4QIpUIBtUKAUiHI4V2pEKBWKKBUClArBKiUCukHgVL6UaDyz1MrBcT5j+nAcWzSq+XHHfX6GHpaIWqh57u3gP+3AOj7S+D3H7fqKb46Wo4/fbAPZ6qlWp85Y/pg8aRBMGhY63Oh3F4fnB4fHG4vHG4vap1e1DjcqHF6UOPwwObwoMbhhs3/uKbBY2nsRo3D0+SHr0IAVA1qD9QKAWqVvwZBIdUgqBT1QclSJ9XeVLTwqt0JBjV6mfXISPD/yk/QIV6nBkQRIgBRlD4YRQA+/zQC8yFCFP3z/dPSMhE2pxelNQ6UWp04Z5W+YK2Olocsg0aJRIMGav8HrUohhUCV/LjB/MBjpQCl/8NXqRDg8frk0PezpWVf8katCukJOqQnSCGhh1ErfYn5RP+XlC902v/l1ty00+NDrf9v35IyREqjVCAjUY/e/i/NzCQ9MhMN6J2oR2aSAclxmkZfzD6fiHKbUwrHljr/2CGH5bOWlgejeJ0K2ckGZCfFISvZgOwkgzROjkNPk67ZAO/2+vD9qWp8dbQMXxwtx77T1QjepVop4JKsRH8tUA8MyzA3+XwVNicOllhx6KwVB0usOFhiwYny2rA1XXEaJYamm5Bi1KLC5kK5zYmyGidqIvwxoFEpYNKp4fHVB5yW1G41RRCAXiYdspPj0CdFeg+zk/zjZEOrAqAoivjZ6sDRczYcLbXh6LkaeRzJ/8u2olEqYNJLId/kD/smvQrx2tCAlJ6gx8SLerZ6Pww9rRC10HPmO+DVPMCQDCwqlP4ntEKNw43/2nwE676Ran0yEvSYenEGpoxMx6Ce8W1Z4qgSRREurw8Olw91bq80uLyoc3tQFzTP4fKGLA+EFofbB4fHC6d/HJjn9Pjg9K8jhxyP77y/rDsDvVqJXgk6ZCTUn7LISND7f7HrkZ6g69AA7HB7pRBU4/AHISdKrQ6U1tQHo1Jr5F86kVAqBPQ0Se9JINj0SpBCXyDkmHTqdtu/KIpwuH2ocbpDw7B/2uoPwsGhuMbhQY3Tg1qnBylGjT/M+INNklRrkhqvbZc2MV6fiAqbEyUWB37214ydszoQp5VCTlaSAX2S45BgULdZzZzF7sauwnJ8eawcX/xUJp+qD0gwqHFV/xT8YkAKEvTqoJBjabLWqUe8FhelmzC0lwkXpZsxNN2E7CRD2PfM4fairMaJcpsT5TaXPF0/LzDtanFtqUKQwlHwaU/psfSDRatSwOUVUVxpP+9zphi16JNcH4Kyk6W/QXayAWa9GiUWB346V4NjQbU3x87Zmvx/pRCAPslxyEk1YkCaEf1SjFApBXi8Ijw+KcB5fVKA9/pEeIKmpWXSOh5f/Ty3Vwr5VocH1jq3fGzbnJ4mT7WGM6K3Gf9vwS9avkEDDD2tELXQ47ID/5UOQAT+8ycgPu2Cnm7nT2V46IN9OGtxyPMGpBpxw4h03DCyF/r77+Temfh8IsprnThb7Qj5NRr41V5a44Td5fGHGy+ilUM0KgXitdIpifpTFWrE+6t3jf5fMvWnMBo81qph1KkgAHD7P0DcHp887fHXFrg8gQ+h0OngGoV4rVoOOmZ9230RdSS7y4NSqxNVdhd8ogiP/0PX4wse++ofe5uY7xOhEOAPeFLISY1vvtaBOhdRFFFUYceXx8rx5U/SqbDzheK+KXEYKgccE4amm5Aar2uX8tW5vCi3OVHj8ECjEoLCjEIOM5GcqhZFEZW1Lpz0t486WWHHKf+4qKL2vKdf1UqhyVompUJAn2QDBqbFY0CqETn+cd+UuA473eTzibC5/AE/EIbq3KhxumGtk4K+1f9DwFrnQWaSAQ9NHtzq/TH0tELUQg8AvHQpUHEMuG0j0P9XF/x0dpcHWw+dw6Z9Z7HzxzK4vPWnVYb0MuGGEb0wZUQ6spINF7yv8xFFERW1LvxsqW9jUmKRwkygYe05q6NV1cQqhdSQVq9WymOdOsxjjQI6lTRPp1ZCq1JAq1ZCFzTWqeuXSdMKaFXSWKdWQqNUsNcJUQfxeH349+lqfHm0HF8fK4fD7cPQXlKwuSjdhMG9TDB2404bljo3TlXYcbKiFkUVtf7G49Lj0hqplkutFNA3JQ4DUuPl2puBafHokxwHjSq2rjLD0NMKUQ0979wGHP5/wLV/AcYsaNOnttS5/QGoBF8dLQ85bz+ytxk3jEjH9SN6IT3hwnpReLw+FFXacfRcjb+xnHQe+Xh5LVzNNCQMCDSiDPQkCe1dokWcViWFGbUSOn+gUbdjF1wios7I7vKgwuZCT7OOn4F+DD2tENXQs2MlsGMFMHIWMPXldttNVa0Lnx38GR/vK8E/CytCThNdmp2IG0b0wvXDeyHV1HQVscfrw6lKuxRsztXgp0C4KasNqVEKJgjS+en0Bl1kA+1Nepr1SI3X8j8wERFFjKGnFaIaeg5/DLwzG+g1Erj7iw7ZZVmNE58eOIuP953Ftycr5UZnggBc0ScJN4xMx+V9ElFUUV9789N5am70aiUGpBmRkypVsw5MMyKnRzx6mnUxV91KREQdg6GnFaIaeioKgZcuAZRa4OESQNmx56p/tjiwef9ZbNpXgu9OVZ93fZ1agQGp8RiQZsSAVCncDEyLR0aCnu1eiIioQ7XH93f3bTHWGST2BdQGwG2X7sPVY2CH7r6nWYe5v+iLub/oi9NVdnyy7yw27TuLE+W16JNiwMDUeOSkGTEwNR4D0+LRO5HhhoiIui+GnvakUACpQ4Az/wJKD3Z46AnWO9GAu8f2x91j+0etDERERNHEBhntLXWoND53MLrlICIiinEMPe0t7SJpfO5QdMtBREQU4xh62lsg9JSypoeIiCiaGHraW6o/9FSdBJw1US0KERFRLGPoaW9xyYDRf5fZ0iPRLQsREVEMY+jpCGn+xsw8xUVERBQ1DD0dgT24iIiIoo6hpyOkDZPG7MFFREQUNQw9HSFweuvcAaB73/WDiIio02Lo6Qgpg6T7bzmqgW3LGXyIiIiigKGnI6h1wLVPStNfvwh88iDgC39HcyIiImofDD0dZfRdwJQXAQjA3jeAD+8BvJ5ol4qIiChmMPR0pEvnADe/BihUwL53gPd+D3ic0S4VERFRTGDo6WjDfwPMWCO18TmyCVg7A3DVRrtURERE3R5DTzQMmgzc+h6gjgOObwfemgbUVUe7VERERN0aQ0+09BsL/O4jQGcGincD+VOA2vJol4qIiKjbYuiJpszLgTmfAHE9gJ/3AX+fDFhLol0qIiKibomhJ9p6Dgdu3wKYMoDyn4A3JgGVJ6JdKiIiom6HoaczSBkAzP0USOwLVBdJwYd3ZCeizqi2HPjpM6BwO9siUpejinYByC8hSwo+b00FSg8Bb14HzN4ApI+KdsmIKJpEEagtk2qALcXS6fDUoYCxR/vv2+eTaqCL99QPFcdC10kZCGRcWj+kDQNUmvYvG1ErCKLYve+JYLVaYTabYbFYYDKZol2c87NXAmtuBkq+A7QmYNa7QHZutEtFnZHHJX0JVhcBVUX1Y7dduhaUQukfq5p5rA6/XJ8A9BgC9BgEaI3RfqXRI4rSJSVctYAmThoEoe3343VLf8vKE0DVCf/4ZP3gsjXexpAshZ8eg4HUIdJ06mBAn9j6crhqgTPfBYWcb6Tb5zSUMlAqc1WYU/FKLdBrBJBxmRSCel8q1WJf6PvmqgUsZwDracByun7a4wSMadIQ37N+HN9T+gxtj79XZ+Os8b8npwHbOUBjlI4PQ5I01id1ySDaHt/fEYcem82GRx99FO+++y4qKysxePBgPPTQQ7jlllvOu21paSkWL16MTZs2wW63Y+TIkXjyyScxfvx4eR2r1YqXXnoJW7duxZEjR2Cz2dC3b1/Mnj0bDzzwAHQ6XUQvsMuFHgBwWIF1twBFXwMqPXDL20DO+PNvR92LzwfUnG0cagLjmhJA7IDbmSRk+79Qh9SPUwYAKm377/tC+HxSAHTVAk6r9P/KaZHGDkv9vEbTwevUAKK3/jkFpdTjUmfyj/2D1hz6ONw6aoP09wwJNv6x5XTofhoRAHNvabCd87f7a+KjO76XPwj5Q1DqUH94jW+8ruVMaC3Oz/sBX4Mrxav0QO/LgMwrgMwrpWlDkrSstgI48y//sFca11U13o8+yR+ALquvEQo8ByAF+JoSqTyW0/5gcwawnqn/Mg8Xvs5HpQfi0wBjz/Dj+F7StCGp84Yjn1f6m1tOA9Wn6t8PeTglHavno4mvD0HBgSg4GAUv08RJ758yeieEOkXoufbaa/Htt99i5cqVGDhwINauXYvXXnsNb7/9NmbNmtXkdk6nE5dddhmqq6uxcuVKpKamYtWqVfjkk0+wbds2jB07FgBw4MAB5OXl4bbbbsO4ceNgNBrx5ZdfYuXKlbjqqquwdetWCBEcnF0y9ACAyw68+zvg2FZAqQF+8wYwZEq0S0WA9Ovf55F+YXpd0uBxSr98fW7/PLd/cPnnBT0OWc9Tv47HJX24BUKNpVha1hyVHkjMloJJYKwzS+XzeaQPTHk6+LG76eVet1SO0sNAbWn4/QpKIDknNAilDgWS+ko1RpHwOKVw4ayRajScNv+4RgosgeDirpOm3Xbp/4fbP89lr5/vrqvfxuOIrBzRptIBiX2kWpGkvqHTCVmhIdNll047lR4Gyg5L49LD0jHTFHOWFIJSBkrhq/ib8OvHpwNZo4FM/9BzOKBUt+w1iCJQebw+CJ3eK/VMDXccJ/aVaqWsZwBbKZoMccG0JqnThznDP86U3hfbOWmo+VkabOekINtSglIqiyHJ/+UfGCc2eOwfB9ZtafD3ef3Hb139sRk4XgODxyGVORD8LMXSYC1pHETD0SVI70d8mnR82Cukoa7ywn4cKdSAWi8NKl3QtL5+uqllpgxgxPRW7zrqoWfz5s24/vrrsXbtWsycOVOef+211+LgwYM4deoUlMrwH3irV6/Gfffdh127diE3Vzpd4/F4MHLkSBiNRuzZswcAUFsrXZ04Li4uZPtnn30WixYtwpdffolf/OIXLX6BXTb0ANKX4IY7gEMfSf8pb1oNjDx/jVq3JorSB6SlWPrVYy0BvE7pV73orf/iFr3+sa/xvJDl/nFwcPE66wNNSLBx1S9ryQd0WxCU0q/7xGzpSzAheJwtte9oz1+oteX1X6ilh+qnnU38slTppC/V1KGAKV36YHfapA/zkEBjA1w10tjnbr/yAwAE6csyUPPSomlz6Hy1XvpiclhCB6dVqoGQ51mbWMcifanpk/yBpm/9OLGPNG3sCSgusG+JwwqU/Sj9rcqO+P9mRwDbz028NQop1GQGhRxz77Y9pjxO4NwB4HRQbVDDdkGAdFpMDjP+Wq3AdCDo6Mwt36/LLr3umnMNxkHBqOZnKRS0ljrOH4QSpZo0jwNwBwUajz/QnO/Hy/kISuk9SMisf2/MvaUga+4tvTfhavIA6bPRaZGaTtgrgsZBoShkXuWFB6WA9EuAu7a3evOoh54777wT69evR1VVFVSq+iqvdevWYdasWfj6668xZsyYsNtOmDABxcXFOHIktFfSihUr8PDDD+P06dPIyMhoct9ffPEFxo4d2yhwnU+XDj2AVBPw8X8AP7wtPZ7wZ2DkTMCY2rHlEEWpKv7UHuliihWF0gdQXApgSKkfG5JC50V6CsTrkX6FBkJNdbFUfVvt/9VTXSwFj85EUEgf2EqN9Is4MCjU/nkq/1gjtZcJXi94HYVaes+Ca21MGVGtXg5LFKWwGRKEDklftp661j+v2iC1RdDGS+2INPH+djQG6ctFY/D/koyTxhqDtI3aIK2n1gc9NtSvp9Z3jlMXXk/0/pb2yvpaobKfpFMYWaOldjfRaLNlrwRKvpcCgTkDMPWWjv1o/J08LqmheODLv64qdDoQAoKX1VW1PhTINSIG/7Sh/jjVxEk/Fsy9pVobsz/kxPeMvAb1QoiiFFblWqngGqm68zwOWj8hC8hb0upitMf3d0T/Aw8cOIAhQ4aEBB4AGDFihLy8qdBz4MABXH311Y3mB7Y9ePBgs6Hn888/BwBcdNFFkRS561OqgF//t/RFsOcVYOsyaTBlAOkXS0PGJUCvUaHnyC+UxwWc/bcUcIr3SGGnqVMdzdHEA3HJ4YORxij90gqEmepTUlV3s20bIIWM+F7+D4QM/5easr4hbmBaUDQxz99oN3ieSlsfTFRaKcSoNP4wo248T+VfV6ntfKGkvQmC9L6bM4AB19TP93mlRreB2qDaMn948QcZjTHosSlo2j/uyA/1aIjmcWJIAvpcJQ2dgSGp87RTVGnqj+eWCq49CQQjly000KiDAk3gdI9Kd+G1eR1BEPzlj6wNbVcQ0f/CiooK9OvXr9H8pKQkeXlz2wbWi3Tbffv24emnn8bUqVPlkNQUp9MJp7O+JsBqjeC8bmelUACTVkpf8t+/Jf2itvob+R3ZVL9eYt/6EJR+MdBrZNNVng3ZK+sbM57aI/Uea9gmQqmRwlXWaKlbqrNGqg6tLQfs5f5xULWpzyOdwnDVSF+GLX69av+HUKZU25Hg/8WTkCVNmzJa3saAOo5CCST3l4YhN0S7NETtR6GQTmldSE85ioqIf3o014j4fA2MW7PtyZMnccMNNyAzMxOvvfbaecu3YsUKPP744+ddr8sRBGDMAmlw2qRamJLvpXBS8r3UeLDK3xvk4IbARv5raPhDUPolQM9h0q+NimP+gOOvySn/qfE+9UlA1pX15/rTL2558hdFqa1DbUVQIAoKRrXlUmiKT5PCjNkfaBKypC6n3f1XPxERdbiIQk9ycnLYGpnKSqkhWLianAvZtqioCHl5eVCpVCgoKGj2+QOWLFmCBx98UH5stVqRmZl53u26FK2xcVV1XRVQ8kN9CDrzvdTts/xHafj3Omk9QSnV/jR17Y1At9SsK6XeOa09xy4IQb+Eclr3HERERG0ootAzfPhwrFu3Dh6PJ6Rdz/79+wEAw4YNa3bbwHrBmtq2qKgI48aNgyiK2LFjB3r37t2iMmq1Wmi1nfz6Ie1Bnwj0z5OGAFupvzboe+mCYyXfSe0sHNVSW5SMS6QanKwrgd5XSG1viIiIuqmIem9t2bIF1113HdavX48ZM2bI8ydPnox9+/Y122X95Zdfxr333ovdu3dj9OjRAKQu66NGjYLRaMTu3bvldU+dOoWxY8fC6/Vix44dYdsRtVSX773VlkRRagdUWy5dV6WzX1yOiIhiVtR7b02ePBkTJkzA/PnzYbVakZOTg3Xr1uHTTz/FmjVr5MAzb9485Ofno7CwENnZ2QCAuXPnYtWqVZg+fbp8ccLVq1fjxx9/xLZt2+R9lJaWIi8vD2fPnsXrr7+O0tJSlJbW9xrq3bt3i2t9qAEh6KquREREMSbihswbNmzAI488gmXLlsm3oVi3bl3IbSi8Xi+8Xi+CK5G0Wi0KCgqwePFi3H///bDb7Rg1ahS2bNkiX40ZAA4dOoTjx48DAGbPnt1o/8uXL8djjz0WabGJiIgoxvGGo0RERNTptMf3dxe4ShIRERHRhWPoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMYeoiIiCgmMPQQERFRTGDoISIiopjA0HMeNpcNbxx4AxuObkA3v2MHERFRtxbxDUdjhdvrxns/vYdX/v0KqpxVAICD5QexZPQSqBR824iIiLoafns3IIoi/lH0D/ztu7/hVM0pAEB6XDrO1p7Fuz+9i1J7KZ4e+zT0Kn2US0pERESR4OmtIHt/3ovZm2fjjzv/iFM1p5CkS8LSK5di07RN+Ou4v0Kj0GDH6R2447M7UOmojHZxiYiIKAKC2M0bqrTk1vTHq4/j+e+ex47iHQAAvUqPORfNwe8v+j3i1HHyet+Xfo8FBQtgdVmRFZ+FV655BZmmzA54FURERLGlJd/fkYrp0FNqL8XqH1Zj47GN8Ik+KAUlbh5wM+aPmo8UfUrY5ztuOY75W+ejpLYESbokrBq/CsNShnXESyEiIooZDD2tEO5Ns7ls+PvBv+OtQ2+hzlMHAPhV5q/wwKUPoJ+533mfs8xehvsK7sPhysPQq/R4duyz+GXvX7br6yAiIoolDD2tEPym6Y16vP/T+3jl36/IbXJG9hiJ/7zsP3Fx6sURPW+tuxYP7ngQu0p2QSkosSx3GaYNmNYeL4GIiCjmMPS0QuBN27BvA14/9jqKrEUAgGxTNhZeshDjs8ZDEIRWPbfb58Zjux7D/yv8fwCA+SPnY/7I+a1+PiIiIpIw9LRC4E0b8vIQKPVKJOmSMH/kfNw88GaoFeoLfn5RFPHfP/w3/nff/wIApuZMxdLcpW3y3ERERLGqPUJPzFynR6/UY+7IuZhz0ZyQHlkXShAE3H/x/UgzpOEve/6Cjcc2orSuFH8d+1cY1IY22w8AOL1O2Fw2JOmSukVtksPjgFap7RavhTont9cNu8cOo9oIpUIZ7eJQNySKIqwuKxweB1INqfw86+Ripqan8Gwh+vU8fyPlC7GjeAcW7VwEh9eBoclDsWr8qiZ7gbWEKIo4Vn0Mu0p24Z8l/8Tec3vh9DqhV+mRYcxA7/jeyIzPRG+jNM6Mz0S6MR0apabtXlQbEEURZ2vP4nDlYRyuOIwjlUdwuOIwSutKoVKokKhNRKJOGpJ0SUjSJcnzknRJ9cu0STBpTVAIbXt5Ka/PC5fPBZfXBafXCZc3aDpovtPrhNvrrp/2uUPmqxQq6FV6GFQG6NV66FX1g0FlqH/sX9ZVagNFUUSdpw4WpwUWlwUWpwXVzmpYnBZYXdaQx8HzXD4X4tXxMGvNMGlMMGlN0lhjanZenDqu0RdHcBmqndWNBovTgipHVaPlte5aAIBCUMCsMYccZw2Pu0RdIhK10nSCLqHL/H0CvD4vbG6bNLhsqHHVoMZVA5vbJo9tLmm5VqlFij4FyfpkJOuS5ekkXRKvOO/nE32odlajzF6G8rpylNpLUV5XjrI66XGZvUyednqdAIA4dRwGJAzAgERpGJg4EAMSB8CkaZtailjD01ut0B5vWnP2le3DgoIFqHJWIcOYgZeveRl9zX1bvH1FXQV2n90tB52yurKI9i9AQM+4nlIYahCKesf3hllrjvQlRcQn+lBkLZKDzeFKabA4LW3y/EpBiQRtgvxFFXg9bp8bbp8bHp8HHp9HnpbHXjc8YoOxf7mI6PwXUCvUIcFIr9JDo9RAKSihUqigEBRQKpRQCdK0PC94eZhpAPCKXvhEH7w+/1isH4uiGPI4eBwYHB5HSMhx+9wd9r4oBSXiNfEwaUzQqrRymAp8sXSUeE28HI4StAkQBEE6VkRADPwT5an6+eHm+Y8xAQIUggIKQQFBEKBA/bRSUErT/nUEQZD/rsHz6tx1qHH7A43Lhhq3NLZ77Bf8mgUISNAmSGGoQSAKnk7Rp8CsNXd4MPT6vLC6rCFBt85TBx988vvuE6Vpn+gLeSxP+/9GwcsdXgfK7eUorStFuV0KNhV1FfCInhaXTSko4RW9YZf1jOsph6FAEOpr6gu1sn3eP1EUpTMDgcAbdJw0DMLhlts9dggQoFaqoVY0GBrOU6qhUqjCrpMel44Zg2e0+nUw9LRCR4ceACiyFmH+tvkorilGgjYBL/3qJYxKHRV2XZfXhe9Lv5dDzuHKwyHLdUodLu15Kcb0GoMx6WOQbcpGSW0JimuKcbrmdP3YJo0DXfCbEq+JR3pcOsxaM+I18SGDSWOSptWN5xnUhkY1LG6fG8erj+NQxSEp5FQexo+VP4b98FUJKuQk5mBw0mAMSRqCoclD0dfcF3WeOlQ6KlHlqJLHVU5pWn7sH2rcNZH9IVpBISigVWqhUWqgVUhjjVIjz5MfN1imVqjh8XlQ56lDnacOdo9dnq7z1KHOXT+vqQ/Gzk6tUMOsNSNBmyDXzAQeB2ppguepFWrUuGrk2h+rywqr0yrXBIWb5/K5zluGwP4C4TcwHTyYtVKtToI2AXqVHlaXNeR4qnRUospZFXrc+Y+9amc1fKKvg97VtqdT6mDUGGFUGxGviYdRbYRREzStNqLOW4eKugpU1FWgvK4cFY4KVDoqI37dOqUOceq48Pvxl0EuR5gyiaKIKmeDGjpHgxq8oOVWp7XDf6Qk6ZKQok9BD30PaWzogR76HvI4RZ+CFH0KlAoliixFOFp9FD9V/YSjVUdxtOooSmpLwj6vSqFCX3PfkDDUx9QHHp8Hdo8ddrc9/Ng/Xeepk+fVumtD5te4ajr0h0pThqcMx9rr17Z6e4aeVohG6AGkGpsFBQtwoOIAtEotnv7l0/hV1q8giiKOW45jV8ku7CrZhX+d+1ejoDI4aTBy03MxJn0MLk69GFqltkX7FEURFY6K0DBUU4zTNmlcXlfe6tejEBTyB5ZJY4IIEYXVhWH/Y+mUOgxMGoghSUOkIXkIchJyLvi0m8vrCglFVQ7pC0ohKKBW1P/aaGocdlqphkpQyQGmvav2RVGE2+cODUdBgcjtc8u1Mx7R02jaK3rrx0HTHp9HWs/ngSAIITUHTY4V9Y8brq9RakIChEljgl6lb/f2Cg6PIyQIObyOkFBjUBnavQyB2oTgcGRxWiBChBD4J0hjAPJ02HnB8wXINQwNayGCa+F88IXM94m+kHl6lV4KF2p/kAiajlfHt7r2wOvzotpZjQqHPwjVSUEoMB0IR4H50aohBSCfNk3QJiBOHSfXggVqz0LG/tqzpqYDnx/BISYQapJ1yRdcG1PjqsGx6mP4qfInHK2WgtBPVT/B5ra10bvRNAFC+NDpnzZpTI3CaLwmHnHqOIiQPqvcXndITXrw4+DlLp+r0bI0Qxp+f9HvW11+hp5WiFboAQC7245FXyzCF6e/gEJQIC8zD/vL96PUXhqyXoo+BWPSxyA3PRdX9rrygtoBna88Z2xncM5+Tj7fb3VZ5engITDf6rI2+4shXh2PwclS7c3gpMEYmjwU2aZstgsg6sYC7YcathUKfhw4XRIyHdTmyOa2QYDQuKZOF1Rb5z+1GKi5C9QkdrX2Vg2Jooifa3+WaoSCaoZO15yGVqWFQWWQBrVBbiMYeBynipPnG9RSW8HgxwaVQQ4w4WrouxKGnlaIZugBAI/Pgyd3P4kPjn4gz9Mqtbgs7TLkpuciNz0XAxIGdNoW/4Fzww3DkMfnQU5iDnobe3fashNR5xU4ldaVv5SpfbHLehekUqiwPHc5hiYPxRnbGVzZ60pcknZJi09ZRZsgCNCpdNCpdOhh6BHt4hBRN8GwQ9HA0NMBBEHAbwf9NtrFICIiimmM2kRERBQTGHqIiIgoJjD0EBERUUxg6CEiIqKYwNBDREREMYGhh4iIiGICQw8RERHFBIYeIiIiigkMPURERBQTGHqIiIgoJjD0EBERUUxg6CEiIqKYwNBDREREMYGhh4iIiGICQw8RERHFBIYeIiIiigkMPURERBQTGHqIiIgoJjD0EBERUUxg6CEiIqKYwNBDREREMSHi0GOz2bBw4UKkp6dDp9Nh1KhRWL9+fYu2LS0txZw5c5CSkgKDwYDc3FwUFBSEXXfbtm3Izc2FwWBASkoK5syZg9LS0kiLS0RERASgFaFn2rRpyM/Px/Lly7FlyxZcfvnlmDlzJtauXdvsdk6nE+PHj0dBQQFefPFFfPTRR0hLS8OkSZOwc+fOkHV37tyJyZMnIy0tDR999BFefPFFbNu2DePHj4fT6Yy0yEREREQQRFEUW7ry5s2bcf3112Pt2rWYOXOmPP/aa6/FwYMHcerUKSiVyrDbrl69Gvfddx927dqF3NxcAIDH48HIkSNhNBqxZ88eed0rrrgCtbW1+Pe//w2VSgUA2LVrF6666iqsXr0a8+fPb/ELtFqtMJvNsFgsMJlMLd6OiIiIoqc9vr8jqunZuHEjjEYjpk+fHjL/9ttvR0lJSUhwCbftoEGD5MADACqVCrNnz8Y333yDM2fOAADOnDmDb7/9FrfddpsceABgzJgxGDhwIDZu3BhJkYmIiIgARBh6Dhw4gCFDhoSEEQAYMWKEvLy5bQPrhdv24MGDIc/R1LrN7QOQTqNZrdaQgYiIiCii0FNRUYGkpKRG8wPzKioqLnjbwLipdZvbBwCsWLECZrNZHjIzM5tdn4iIiGJDxA2ZBUFo1bJIt21q3fPtY8mSJbBYLPJQXFzc7PpEREQUG1TnX6VecnJy2JqWyspKAOFrZyLdNjk5GUD4WqPKyspm9wEAWq0WWq222XWIiIgo9kRU0zN8+HAcPnwYHo8nZP7+/fsBAMOGDWt228B6zW0bGDe1bnP7ICIiImpKRKFn6tSpsNls+OCDD0Lm5+fnIz09HaNHj2522yNHjoT08PJ4PFizZg1Gjx6N9PR0AEBGRgauuOIKrFmzBl6vV1539+7d+PHHHzFt2rRIikxEREQEIMLr9ADSNXn27t2Lp556Cjk5OVi3bh1effVVrFmzBrfeeisAYN68ecjPz0dhYSGys7MBSL2qLr30UlitVqxcuRKpqalYvXo1Pv74Y2zbtg1jx46V97Fjxw5MmDABU6ZMwb333ovS0lI89NBDMJvN2Lt3b0Snr3idHiIioq4n6tfpAYANGzbgtttuw7JlyzBp0iTs2bMH69atkwMPAHi9Xni9XgTnKa1Wi4KCAuTl5eH+++/HlClTcPbsWWzZsiUk8ADAuHHjsHnzZpw9exZTpkzB/fffj7y8PBQUFLC9DhEREbVKxDU9XQ1reoiIiLqeTlHTQ0RERNQVMfQQERFRTGDoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMiust6VxS49qLVao1ySYiIiKilAt/bbXkN5W4fempqagAAmZmZUS4JERERRaqiogJms7lNnqvb34bC5/OhpKQE8fHxEAQh2sWJOqvViszMTBQXF/O2HG2I72vb43vaPvi+tj2+p+3DYrEgKysLVVVVSEhIaJPn7PY1PQqFAr179452MTodk8nE/5ztgO9r2+N72j74vrY9vqftQ6Fou+bHbMhMREREMYGhh4iIiGICQ0+M0Wq1WL58ObRabbSL0q3wfW17fE/bB9/Xtsf3tH20x/va7RsyExEREQGs6SEiIqIYwdBDREREMYGhh4iIiGICQ08M2LFjBwRBCDvs3r072sXrEmpqarB48WJce+216NGjBwRBwGOPPRZ23e+++w7XXHMNjEYjEhISMG3aNBw/frxjC9xFtPR9nTNnTtjjd/DgwR1f6E7s888/x9y5czF48GDExcUhIyMDN954I/71r381WpfHacu19H3lcRqZH374Addffz2ysrKg1+uRlJSE3NxcrFmzptG6bXW8dvuLE1K9//qv/0JeXl7IvGHDhkWpNF1LRUUF/vd//xcjR47ETTfdhNdeey3sekeOHMG4ceMwatQovPvuu3A4HFi2bBmuvvpq/PDDD+jRo0cHl7xza+n7CgB6vR6ff/55o3lU7+WXX0ZFRQUeeOABDB06FGVlZXjuuedw5ZVX4rPPPsOvfvUrADxOI9XS9xXgcRqJ6upqZGZmYubMmcjIyEBtbS3efvtt3HbbbTh58iQeffRRAG18vIrU7W3fvl0EIL733nvRLkqX5fP5RJ/PJ4qiKJaVlYkAxOXLlzdab/r06WJKSoposVjkeSdPnhTVarW4ePHijipul9HS9/X3v/+9GBcX18Gl63rOnTvXaF5NTY2YlpYmjh8/Xp7H4zQyLX1feZy2jdGjR4uZmZny47Y8Xnl6i6gFAtXUzfF4PNi0aRNuvvnmkEvRZ2dnIy8vDxs3bmzvYnY5LXlfqeVSU1MbzTMajRg6dCiKi4sB8DhtjZa8r9R2UlJSoFJJJ6La+nhl6Ikh9913H1QqFUwmEyZOnIivvvoq2kXqVgoLC1FXV4cRI0Y0WjZixAgcO3YMDocjCiXrHurq6tCzZ08olUr07t0bCxYsQGVlZbSL1elZLBZ89913uOiiiwDwOG0rDd/XAB6nkfP5fPB4PCgrK8Pq1avx2Wef4U9/+hOAtj9e2aYnBpjNZjzwwAMYN24ckpOTcezYMTzzzDMYN24cPvnkE0ycODHaRewWKioqAABJSUmNliUlJUEURVRVVaFXr14dXbQub+TIkRg5cqTcBm3nzp14/vnnUVBQgG+//RZGozHKJey87rvvPtTW1uKRRx4BwOO0rTR8XwEep61177334n/+538AABqNBn/7299w9913A2j745WhJwZcfPHFuPjii+XHV199NaZOnYrhw4dj8eLFDD1trLnTNTyV0zp/+MMfQh5PmDABF198MX7zm9/g1VdfbbScJEuXLsXbb7+Nl156CZdeemnIMh6nrdfU+8rjtHUefvhh3HHHHSgtLcXHH3+MBQsWoLa2Fn/84x/lddrqeGXoiVEJCQm44YYb8Morr6Curo69C9pAcnIygPpfJsEqKyshCAISEhI6uFTd19SpUxEXF8fLLjTh8ccfx5NPPom//OUvWLBggTyfx+mFaep9bQqP0/PLyspCVlYWAOC6664DACxZsgS///3v2/x4ZZueGCb6b7vGX3Vto3///tDr9di/f3+jZfv370dOTg50Ol0UStZ9iaIIhYIfYw09/vjjeOyxx/DYY4/h4YcfDlnG47T1mntfm8PjNDJXXHEFPB4Pjh8/3ubHK/8KMaqqqgqbNm3CqFGj+AHXRlQqFaZMmYINGzagpqZGnn/q1Cls374d06ZNi2Lpup/3338fdrsdV155ZbSL0qk88cQTeOyxx/Doo49i+fLljZbzOG2d872vTeFxGrnt27dDoVCgX79+bX688i7rMWDWrFnIysrCZZddhpSUFBw9ehTPPfccCgsLsWXLFlxzzTXRLmKXsGXLFtTW1qKmpgZz587F9OnT8dvf/haAVCVrMBhw5MgRXH755bjkkkvw0EMPyRfRqqys5EXfmnC+97WsrAyzZs3CLbfcgpycHAiCgJ07d+KFF15A//79sWfPHsTFxUX5VXQOzz33HP74xz9i0qRJYb+YA1+8PE4j05L3taioiMdphO666y6YTCZcccUVSEtLQ3l5Od577z288847WLRoEZ5++mkAbXy8XvBVhKjTW7FihThq1CjRbDaLSqVS7NGjhzh16lTxm2++iXbRupTs7GwRQNjhxIkT8np79+4Vx48fLxoMBtFkMok33XSTeOzYsegVvJM73/taWVkpTp06VezTp4+o1+tFjUYjDhgwQFy8eLFYXV0d7eJ3KmPHjm3yvWz4cc/jtOVa8r7yOI3cG2+8IV599dViSkqKqFKpxISEBHHs2LHiW2+91WjdtjpeWdNDREREMYFteoiIiCgmMPQQERFRTGDoISIiopjA0ENEREQxgaGHiIiIYgJDDxEREcUEhh4iIiKKCQw9REREFBMYeoiIiCgmMPQQERFRTGDoISIiopjA0ENEREQx4f8Dsnuo/FrkZW8AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], "source": [ "\n", "import matplotlib.pyplot as plt\n", @@ -2130,7 +2077,7 @@ "\n", "n = 100\n", "n_boostraps = 100\n", - "maxdepth = 8\n", + "maxdepth = 30\n", "\n", "# Make data set.\n", "x = np.linspace(-3, 3, n).reshape(-1, 1)\n", @@ -2183,9 +2130,7 @@ { "cell_type": "markdown", "id": "79f3b548", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Random forests\n", "\n", @@ -2206,9 +2151,7 @@ { "cell_type": "markdown", "id": "f51ddbb5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "m\\approx \\sqrt{p}.\n", @@ -2218,9 +2161,7 @@ { "cell_type": "markdown", "id": "5307d349", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "In building a random forest, at\n", "each split in the tree, the algorithm is not even allowed to consider\n", @@ -2243,9 +2184,7 @@ { "cell_type": "markdown", "id": "f5df6740", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Random Forest Algorithm\n", "The algorithm described here can be applied to both classification and regression problems.\n", @@ -2269,9 +2208,7 @@ { "cell_type": "markdown", "id": "e1f7886c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Random Forests Compared with other Methods on the Cancer Data" ] @@ -2280,10 +2217,7 @@ "cell_type": "code", "execution_count": 21, "id": "4e31e8c8", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", @@ -2352,9 +2286,7 @@ { "cell_type": "markdown", "id": "3de40f94", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "Recall that the cumulative gains curve shows the percentage of the\n", "overall number of cases in a given category *gained* by targeting a\n", @@ -2368,9 +2300,7 @@ { "cell_type": "markdown", "id": "9bff02d3", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Compare Bagging on Trees with Random Forests" ] @@ -2379,10 +2309,7 @@ "cell_type": "code", "execution_count": 22, "id": "23f8734b", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "bag_clf = BaggingClassifier(\n", @@ -2394,10 +2321,7 @@ "cell_type": "code", "execution_count": 23, "id": "59bd532d", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "bag_clf.fit(X_train, y_train)\n", @@ -2412,9 +2336,7 @@ { "cell_type": "markdown", "id": "e24309a7", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Boosting, a Bird's Eye View\n", "\n", @@ -2432,9 +2354,7 @@ { "cell_type": "markdown", "id": "8f84b7a8", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## What is boosting? Additive Modelling/Iterative Fitting\n", "\n", @@ -2446,9 +2366,7 @@ { "cell_type": "markdown", "id": "a8144faa", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "f_M(x) = \\sum_{i=1}^M \\beta_m b(x;\\gamma_m),\n", @@ -2458,9 +2376,7 @@ { "cell_type": "markdown", "id": "9f523032", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where $\\beta_m$ are the expansion parameters to be determined in a\n", "minimization process and $b(x;\\gamma_m)$ are some simple functions of\n", @@ -2475,9 +2391,7 @@ { "cell_type": "markdown", "id": "b0d77841", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\sigma(t) = \\frac{1}{1+\\exp{(-t)}},\n", @@ -2487,9 +2401,7 @@ { "cell_type": "markdown", "id": "3e086177", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where $t=\\gamma_0+\\gamma_1 x$ and the parameters $\\gamma_0$ and\n", "$\\gamma_1$ were determined by the Logistic Regression fitting\n", @@ -2501,9 +2413,7 @@ { "cell_type": "markdown", "id": "15ad95dc", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{y},\\boldsymbol{f}) = \\frac{1}{n} \\sum_{i=0}^{n-1}(y_i-f(x_i))^2.\n", @@ -2513,9 +2423,7 @@ { "cell_type": "markdown", "id": "6287183a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "In this case the function $f(x)$ was replaced by the design matrix\n", "$\\boldsymbol{X}$ and the unknown linear regression parameters $\\boldsymbol{\\beta}$,\n", @@ -2526,9 +2434,7 @@ { "cell_type": "markdown", "id": "27de8464", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\beta}=\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n", @@ -2538,9 +2444,7 @@ { "cell_type": "markdown", "id": "7d8e41cb", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "In iterative fitting or additive modeling, we minimize the cost function with respect to the parameters $\\beta_m$ and $\\gamma_m$." ] @@ -2548,9 +2452,7 @@ { "cell_type": "markdown", "id": "f45f6f2a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Iterative Fitting, Regression and Squared-error Cost Function\n", "\n", @@ -2576,9 +2478,7 @@ { "cell_type": "markdown", "id": "af260098", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Squared-Error Example and Iterative Fitting\n", "\n", @@ -2592,9 +2492,7 @@ { "cell_type": "markdown", "id": "f1a85e34", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "(\\beta_m,\\gamma_m) = \\mathrm{argmin}_{\\beta,\\lambda}\\hspace{0.1cm} \\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta b(x;\\gamma))^2=\\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\\beta(1+\\gamma x_i))^2.\n", @@ -2604,9 +2502,7 @@ { "cell_type": "markdown", "id": "a809e54d", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "We start our iteration by simply setting $f_0(x)=0$. \n", "Taking the derivatives with respect to $\\beta$ and $\\gamma$ we obtain" @@ -2615,9 +2511,7 @@ { "cell_type": "markdown", "id": "21b8fb9c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\frac{\\partial {\\cal C}}{\\partial \\beta} = -2\\sum_{i}(1+\\gamma x_i)(y_i-\\beta(1+\\gamma x_i))=0,\n", @@ -2627,9 +2521,7 @@ { "cell_type": "markdown", "id": "d40d3542", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "and" ] @@ -2637,9 +2529,7 @@ { "cell_type": "markdown", "id": "30b388a2", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\frac{\\partial {\\cal C}}{\\partial \\gamma} =-2\\sum_{i}\\beta x_i(y_i-\\beta(1+\\gamma x_i))=0.\n", @@ -2649,9 +2539,7 @@ { "cell_type": "markdown", "id": "8f5986c5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "We can then rewrite these equations as (defining $\\boldsymbol{w}=\\boldsymbol{e}+\\gamma \\boldsymbol{x})$ with $\\boldsymbol{e}$ being the unit vector)" ] @@ -2659,9 +2547,7 @@ { "cell_type": "markdown", "id": "605abac6", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\gamma \\boldsymbol{w}^T(\\boldsymbol{y}-\\beta\\gamma \\boldsymbol{w})=0,\n", @@ -2671,9 +2557,7 @@ { "cell_type": "markdown", "id": "f67ecd40", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which gives us $\\beta = \\boldsymbol{w}^T\\boldsymbol{y}/(\\boldsymbol{w}^T\\boldsymbol{w})$. Similarly we have" ] @@ -2681,9 +2565,7 @@ { "cell_type": "markdown", "id": "85880a97", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\beta\\gamma \\boldsymbol{x}^T(\\boldsymbol{y}-\\beta(1+\\gamma \\boldsymbol{x}))=0,\n", @@ -2693,9 +2575,7 @@ { "cell_type": "markdown", "id": "e801e0c6", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which leads to $\\gamma =(\\boldsymbol{x}^T\\boldsymbol{y}-\\beta\\boldsymbol{x}^T\\boldsymbol{e})/(\\beta\\boldsymbol{x}^T\\boldsymbol{x})$. Inserting\n", "for $\\beta$ gives us an equation for $\\gamma$. This is a non-linear equation in the unknown $\\gamma$ and has to be solved numerically. \n", @@ -2707,9 +2587,7 @@ { "cell_type": "markdown", "id": "05e0fc0e", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Iterative Fitting, Classification and AdaBoost\n", "\n", @@ -2723,9 +2601,7 @@ { "cell_type": "markdown", "id": "d1c5a166", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\mathrm{\\overline{err}}=\\frac{1}{n} \\sum_{i=0}^{n-1} I(y_i\\ne G(x_i)).\n", @@ -2735,9 +2611,7 @@ { "cell_type": "markdown", "id": "4476c4c7", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "The iterative procedure starts with defining a weak classifier whose\n", "error rate is barely better than random guessing. The iterative\n", @@ -2751,9 +2625,7 @@ { "cell_type": "markdown", "id": "8b35c8d2", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "f_M(x) = \\sum_{i=1}^M \\beta_m b(x;\\gamma_m),\n", @@ -2763,9 +2635,7 @@ { "cell_type": "markdown", "id": "a533e45a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "will be a function of" ] @@ -2773,9 +2643,7 @@ { "cell_type": "markdown", "id": "b1e33a99", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "G_M(x) = \\mathrm{sign} \\sum_{i=1}^M \\alpha_m G_m(x).\n", @@ -2785,9 +2653,7 @@ { "cell_type": "markdown", "id": "4199e7a8", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Adaptive Boosting, AdaBoost\n", "\n", @@ -2797,9 +2663,7 @@ { "cell_type": "markdown", "id": "db2fda6f", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "f_m(x) = f_{m-1}(x)+\\beta_mG_m(x).\n", @@ -2809,9 +2673,7 @@ { "cell_type": "markdown", "id": "dbb7d266", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "The simplest possible cost function which leads (also simple from a computational point of view) to the AdaBoost algorithm is the\n", "exponential cost/loss function defined as" @@ -2820,9 +2682,7 @@ { "cell_type": "markdown", "id": "70431c63", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{y},\\boldsymbol{f}) = \\sum_{i=0}^{n-1}\\exp{(-y_i(f_{m-1}(x_i)+\\beta G(x_i))}.\n", @@ -2832,9 +2692,7 @@ { "cell_type": "markdown", "id": "9e678e2c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "We optimize $\\beta$ and $G$ for each value of $m=1:M$ as we did in the regression case.\n", "This is normally done in two steps. Let us however first rewrite the cost function as" @@ -2843,9 +2701,7 @@ { "cell_type": "markdown", "id": "d2957ac5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{y},\\boldsymbol{f}) = \\sum_{i=0}^{n-1}w_i^{m}\\exp{(-y_i\\beta G(x_i))},\n", @@ -2855,9 +2711,7 @@ { "cell_type": "markdown", "id": "abebd5ea", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where we have defined $w_i^m= \\exp{(-y_if_{m-1}(x_i))}$." ] @@ -2865,9 +2719,7 @@ { "cell_type": "markdown", "id": "9d7d7c9f", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Building up AdaBoost\n", "\n", @@ -2877,9 +2729,7 @@ { "cell_type": "markdown", "id": "9fdaa6db", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "G_m(x) = \\mathrm{sign} \\sum_{i=0}^{n-1} w_i^m I(y_i \\ne G_(x_i)),\n", @@ -2889,9 +2739,7 @@ { "cell_type": "markdown", "id": "f4cbd741", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which is the classifier that minimizes the weighted error rate in predicting $y$.\n", "\n", @@ -2901,9 +2749,7 @@ { "cell_type": "markdown", "id": "8310198c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\exp{-(\\beta)}\\sum_{y_i=G(x_i)}w_i^m+\\exp{(\\beta)}\\sum_{y_i\\ne G(x_i)}w_i^m,\n", @@ -2913,9 +2759,7 @@ { "cell_type": "markdown", "id": "b37fe48c", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which can be rewritten as" ] @@ -2923,9 +2767,7 @@ { "cell_type": "markdown", "id": "9e909739", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "(\\exp{(\\beta)}-\\exp{-(\\beta)})\\sum_{i=0}^{n-1}w_i^mI(y_i\\ne G(x_i))+\\exp{(-\\beta)}\\sum_{i=0}^{n-1}w_i^m=0,\n", @@ -2935,9 +2777,7 @@ { "cell_type": "markdown", "id": "d8ad59f0", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which leads to" ] @@ -2945,9 +2785,7 @@ { "cell_type": "markdown", "id": "ba2d48e0", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\beta_m = \\frac{1}{2}\\log{\\frac{1-\\mathrm{\\overline{err}}}{\\mathrm{\\overline{err}}}},\n", @@ -2957,9 +2795,7 @@ { "cell_type": "markdown", "id": "5e1dcb00", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where we have redefined the error as" ] @@ -2967,9 +2803,7 @@ { "cell_type": "markdown", "id": "45c824f3", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\mathrm{\\overline{err}}_m=\\frac{1}{n}\\frac{\\sum_{i=0}^{n-1}w_i^mI(y_i\\ne G(x_i)}{\\sum_{i=0}^{n-1}w_i^m},\n", @@ -2979,9 +2813,7 @@ { "cell_type": "markdown", "id": "c916fad2", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "which leads to an update of" ] @@ -2989,9 +2821,7 @@ { "cell_type": "markdown", "id": "070d7540", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "f_m(x) = f_{m-1}(x) +\\beta_m G_m(x).\n", @@ -3001,9 +2831,7 @@ { "cell_type": "markdown", "id": "4b1b22a8", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "This leads to the new weights" ] @@ -3011,9 +2839,7 @@ { "cell_type": "markdown", "id": "cc2bf04a", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "w_i^{m+1} = w_i^m \\exp{(-y_i\\beta_m G_m(x_i))}\n", @@ -3023,9 +2849,7 @@ { "cell_type": "markdown", "id": "ea775ce7", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Adaptive boosting: AdaBoost, Basic Algorithm\n", "\n", @@ -3043,9 +2867,7 @@ { "cell_type": "markdown", "id": "ae10c0a4", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\mathrm{err}=\\frac{1}{n}\\sum_{i=0}^{n-1}I(y_i\\ne G(x_i)),\n", @@ -3055,9 +2877,7 @@ { "cell_type": "markdown", "id": "dfe87ab2", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "where the function $I()$ is one if we misclassify and zero if we classify correctly." ] @@ -3065,9 +2885,7 @@ { "cell_type": "markdown", "id": "875649dd", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## Basic Steps of AdaBoost\n", "\n", @@ -3081,9 +2899,7 @@ { "cell_type": "markdown", "id": "5aac54e6", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "$$\n", "\\mathrm{\\overline{err}}_m=\\frac{\\sum_{i=0}^{n-1}w_i^m I(y_i\\ne G(x_i))}{\\sum_{i=0}^{n-1}w_i},\n", @@ -3093,9 +2909,7 @@ { "cell_type": "markdown", "id": "a3e84a60", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "1. Then we start looping over all attempts at classifying, namely we start an iterative process for $m=1:M$, where $M$ is the final number of classifications. Our given classifier could for example be a plain decision tree.\n", "\n", @@ -3121,9 +2935,7 @@ { "cell_type": "markdown", "id": "d80edac5", - "metadata": { - "editable": true - }, + "metadata": {}, "source": [ "## AdaBoost Examples\n", "\n", @@ -3134,10 +2946,7 @@ "cell_type": "code", "execution_count": 24, "id": "04a2ab5b", - "metadata": { - "collapsed": false, - "editable": true - }, + "metadata": {}, "outputs": [], "source": [ "from sklearn.ensemble import AdaBoostClassifier\n", @@ -3164,7 +2973,25 @@ ] } ], - "metadata": {}, + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.10" + } + }, "nbformat": 4, "nbformat_minor": 5 }