lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

corrected slides

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-11-25 06:51:30 +01:00
parent 4bdb70a008
commit 137a1edb6f
7 changed files with 345 additions and 412 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week47/html/week47-bs.html
+69 -71
View File
@@ -54,7 +54,6 @@ doconce format html week47.do.txt --html_style=bootstrap --pygments_html_style=d
None,
'first-attempt-at-a-minimization-approach'),
('Solving the equations', 2, None, 'solving-the-equations'),
('Code Example', 2, None, 'code-example'),
('Problems with the Simpler Approach',
2,
None,
@@ -273,75 +272,74 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week47-bs007.html#getting-into-the-details" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs008.html#first-attempt-at-a-minimization-approach" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs009.html#solving-the-equations" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs010.html#code-example" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs011.html#problems-with-the-simpler-approach" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs012.html#a-better-approach" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs013.html#a-quick-reminder-on-lagrangian-multipliers" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs014.html#adding-the-multiplier" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs015.html#setting-up-the-problem" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs017.html#the-last-steps" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs018.html#a-soft-classifier" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs019.html#soft-optmization-problem" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs020.html#kernels-and-non-linearity" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#the-equations" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs023.html#different-kernels-and-mercer-s-theorem" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs024.html#the-moons-example" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs025.html#mathematical-optimization-of-convex-functions" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs026.html#how-do-we-solve-these-problems" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs027.html#a-simple-example" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs028.html#back-to-the-more-realistic-cases" style="font-size: 80%;">Back to the more realistic cases</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs029.html#summary-of-course" style="font-size: 80%;">Summary of course</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs030.html#what-me-worry-no-final-exam-in-this-course" style="font-size: 80%;">What? Me worry? No final exam in this course!</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs031.html#what-is-the-link-between-artificial-intelligence-and-machine-learning-and-some-general-remarks" style="font-size: 80%;">What is the link between Artificial Intelligence and Machine Learning and some general Remarks</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs032.html#going-back-to-the-beginning-of-the-semester" style="font-size: 80%;">Going back to the beginning of the semester</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs033.html#not-so-sharp-distinctions" style="font-size: 80%;">Not so sharp distinctions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs034.html#topics-we-have-covered-this-year" style="font-size: 80%;">Topics we have covered this year</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs035.html#statistical-analysis-and-optimization-of-data" style="font-size: 80%;">Statistical analysis and optimization of data</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs036.html#machine-learning" style="font-size: 80%;">Machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs037.html#learning-outcomes-and-overarching-aims-of-this-course" style="font-size: 80%;">Learning outcomes and overarching aims of this course</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs038.html#perspective-on-machine-learning" style="font-size: 80%;">Perspective on Machine Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs039.html#machine-learning-research" style="font-size: 80%;">Machine Learning Research</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs040.html#starting-your-machine-learning-project" style="font-size: 80%;">Starting your Machine Learning Project</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs041.html#choose-a-model-and-algorithm" style="font-size: 80%;">Choose a Model and Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs042.html#preparing-your-data" style="font-size: 80%;">Preparing Your Data</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs043.html#which-activation-and-weights-to-choose-in-neural-networks" style="font-size: 80%;">Which Activation and Weights to Choose in Neural Networks</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs044.html#optimization-methods-and-hyperparameters" style="font-size: 80%;">Optimization Methods and Hyperparameters</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs045.html#resampling" style="font-size: 80%;">Resampling</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs046.html#other-courses-on-data-science-and-machine-learning-at-uio" style="font-size: 80%;">Other courses on Data science and Machine Learning at UiO</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs047.html#additional-courses-of-interest" style="font-size: 80%;">Additional courses of interest</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs048.html#what-s-the-future-like" style="font-size: 80%;">What's the future like?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs049.html#types-of-machine-learning-a-repetition" style="font-size: 80%;">Types of Machine Learning, a repetition</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs050.html#why-boltzmann-machines" style="font-size: 80%;">Why Boltzmann machines?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs051.html#boltzmann-machines" style="font-size: 80%;">Boltzmann Machines</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs052.html#some-similarities-and-differences-from-dnns" style="font-size: 80%;">Some similarities and differences from DNNs</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs053.html#boltzmann-machines-bm" style="font-size: 80%;">Boltzmann machines (BM)</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs054.html#a-standard-bm-setup" style="font-size: 80%;">A standard BM setup</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs055.html#the-structure-of-the-rbm-network" style="font-size: 80%;">The structure of the RBM network</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs056.html#the-network" style="font-size: 80%;">The network</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs057.html#goals" style="font-size: 80%;">Goals</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs058.html#joint-distribution" style="font-size: 80%;">Joint distribution</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs059.html#network-elements-the-energy-function" style="font-size: 80%;">Network Elements, the energy function</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs060.html#defining-different-types-of-rbms" style="font-size: 80%;">Defining different types of RBMs</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs061.html#more-about-rbms" style="font-size: 80%;">More about RBMs</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs062.html#autoencoders-overarching-view" style="font-size: 80%;">Autoencoders: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs063.html#bayesian-machine-learning" style="font-size: 80%;">Bayesian Machine Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs064.html#reinforcement-learning" style="font-size: 80%;">Reinforcement Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs065.html#transfer-learning" style="font-size: 80%;">Transfer learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs066.html#adversarial-learning" style="font-size: 80%;">Adversarial learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs067.html#dual-learning" style="font-size: 80%;">Dual learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs068.html#distributed-machine-learning" style="font-size: 80%;">Distributed machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs069.html#meta-learning" style="font-size: 80%;">Meta learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs070.html#the-challenges-facing-machine-learning" style="font-size: 80%;">The Challenges Facing Machine Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs071.html#explainable-machine-learning" style="font-size: 80%;">Explainable machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs072.html#quantum-machine-learning" style="font-size: 80%;">Quantum machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs073.html#quantum-machine-learning-algorithms-based-on-linear-algebra" style="font-size: 80%;">Quantum machine learning algorithms based on linear algebra</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs074.html#quantum-reinforcement-learning" style="font-size: 80%;">Quantum reinforcement learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs075.html#quantum-deep-learning" style="font-size: 80%;">Quantum deep learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs076.html#social-machine-learning" style="font-size: 80%;">Social machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs077.html#the-last-words" style="font-size: 80%;">The last words?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs078.html#best-wishes-to-you-all-and-thanks-so-much-for-your-heroic-efforts-this-semester" style="font-size: 80%;">Best wishes to you all and thanks so much for your heroic efforts this semester</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs010.html#problems-with-the-simpler-approach" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs011.html#a-better-approach" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs012.html#a-quick-reminder-on-lagrangian-multipliers" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs013.html#adding-the-multiplier" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs014.html#setting-up-the-problem" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs016.html#the-last-steps" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs017.html#a-soft-classifier" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs018.html#soft-optmization-problem" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs019.html#kernels-and-non-linearity" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs020.html#the-equations" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#different-kernels-and-mercer-s-theorem" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs023.html#the-moons-example" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs024.html#mathematical-optimization-of-convex-functions" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs025.html#how-do-we-solve-these-problems" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs026.html#a-simple-example" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs027.html#back-to-the-more-realistic-cases" style="font-size: 80%;">Back to the more realistic cases</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs028.html#summary-of-course" style="font-size: 80%;">Summary of course</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs029.html#what-me-worry-no-final-exam-in-this-course" style="font-size: 80%;">What? Me worry? No final exam in this course!</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs030.html#what-is-the-link-between-artificial-intelligence-and-machine-learning-and-some-general-remarks" style="font-size: 80%;">What is the link between Artificial Intelligence and Machine Learning and some general Remarks</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs031.html#going-back-to-the-beginning-of-the-semester" style="font-size: 80%;">Going back to the beginning of the semester</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs032.html#not-so-sharp-distinctions" style="font-size: 80%;">Not so sharp distinctions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs033.html#topics-we-have-covered-this-year" style="font-size: 80%;">Topics we have covered this year</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs034.html#statistical-analysis-and-optimization-of-data" style="font-size: 80%;">Statistical analysis and optimization of data</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs035.html#machine-learning" style="font-size: 80%;">Machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs036.html#learning-outcomes-and-overarching-aims-of-this-course" style="font-size: 80%;">Learning outcomes and overarching aims of this course</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs037.html#perspective-on-machine-learning" style="font-size: 80%;">Perspective on Machine Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs038.html#machine-learning-research" style="font-size: 80%;">Machine Learning Research</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs039.html#starting-your-machine-learning-project" style="font-size: 80%;">Starting your Machine Learning Project</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs040.html#choose-a-model-and-algorithm" style="font-size: 80%;">Choose a Model and Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs041.html#preparing-your-data" style="font-size: 80%;">Preparing Your Data</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs042.html#which-activation-and-weights-to-choose-in-neural-networks" style="font-size: 80%;">Which Activation and Weights to Choose in Neural Networks</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs043.html#optimization-methods-and-hyperparameters" style="font-size: 80%;">Optimization Methods and Hyperparameters</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs044.html#resampling" style="font-size: 80%;">Resampling</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs045.html#other-courses-on-data-science-and-machine-learning-at-uio" style="font-size: 80%;">Other courses on Data science and Machine Learning at UiO</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs046.html#additional-courses-of-interest" style="font-size: 80%;">Additional courses of interest</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs047.html#what-s-the-future-like" style="font-size: 80%;">What's the future like?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs048.html#types-of-machine-learning-a-repetition" style="font-size: 80%;">Types of Machine Learning, a repetition</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs049.html#why-boltzmann-machines" style="font-size: 80%;">Why Boltzmann machines?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs050.html#boltzmann-machines" style="font-size: 80%;">Boltzmann Machines</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs051.html#some-similarities-and-differences-from-dnns" style="font-size: 80%;">Some similarities and differences from DNNs</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs052.html#boltzmann-machines-bm" style="font-size: 80%;">Boltzmann machines (BM)</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs053.html#a-standard-bm-setup" style="font-size: 80%;">A standard BM setup</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs054.html#the-structure-of-the-rbm-network" style="font-size: 80%;">The structure of the RBM network</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs055.html#the-network" style="font-size: 80%;">The network</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs056.html#goals" style="font-size: 80%;">Goals</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs057.html#joint-distribution" style="font-size: 80%;">Joint distribution</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs058.html#network-elements-the-energy-function" style="font-size: 80%;">Network Elements, the energy function</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs059.html#defining-different-types-of-rbms" style="font-size: 80%;">Defining different types of RBMs</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs060.html#more-about-rbms" style="font-size: 80%;">More about RBMs</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs061.html#autoencoders-overarching-view" style="font-size: 80%;">Autoencoders: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs062.html#bayesian-machine-learning" style="font-size: 80%;">Bayesian Machine Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs063.html#reinforcement-learning" style="font-size: 80%;">Reinforcement Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs064.html#transfer-learning" style="font-size: 80%;">Transfer learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs065.html#adversarial-learning" style="font-size: 80%;">Adversarial learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs066.html#dual-learning" style="font-size: 80%;">Dual learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs067.html#distributed-machine-learning" style="font-size: 80%;">Distributed machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs068.html#meta-learning" style="font-size: 80%;">Meta learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs069.html#the-challenges-facing-machine-learning" style="font-size: 80%;">The Challenges Facing Machine Learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs070.html#explainable-machine-learning" style="font-size: 80%;">Explainable machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs071.html#quantum-machine-learning" style="font-size: 80%;">Quantum machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs072.html#quantum-machine-learning-algorithms-based-on-linear-algebra" style="font-size: 80%;">Quantum machine learning algorithms based on linear algebra</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs073.html#quantum-reinforcement-learning" style="font-size: 80%;">Quantum reinforcement learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs074.html#quantum-deep-learning" style="font-size: 80%;">Quantum deep learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs075.html#social-machine-learning" style="font-size: 80%;">Social machine learning</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs076.html#the-last-words" style="font-size: 80%;">The last words?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs077.html#best-wishes-to-you-all-and-thanks-so-much-for-your-heroic-efforts-this-semester" style="font-size: 80%;">Best wishes to you all and thanks so much for your heroic efforts this semester</a></li>
</ul>
</li>
@@ -396,7 +394,7 @@ MathJax.Hub.Config({
<li><a href="._week47-bs008.html">9</a></li>
<li><a href="._week47-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week47-bs078.html">79</a></li>
<li><a href="._week47-bs077.html">78</a></li>
<li><a href="._week47-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week47/html/week47-reveal.html
+8 -29
View File
@@ -379,6 +379,8 @@ $$
\boldsymbol{x}^T\boldsymbol{w}+b=0.
$$
<p>&nbsp;<br>
<p>For figures, see <a href="https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" target="_blank">handwritten notes</a> for Thursday November 25.</p>
</section>
<section>
@@ -469,7 +471,9 @@ f(x) = \boldsymbol{w}^T\boldsymbol{x}+b = 0,
$$
<p>&nbsp;<br>
<p>as the function that determines the line \( L \) that separates two classes (our two features), see the figure here. </p>
<p>as the function that determines the line \( L \) that separates two classes (our two features), see the figures in the <a href="https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" target="_blank">handwritten notes</a> for Thursday November 25.
.
</p>
<p>Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \). </p>
@@ -531,39 +535,13 @@ $$
</section>
<section>
<h2 id="code-example">Code Example </h2>
<h2 id="problems-with-the-simpler-approach">Problems with the Simpler Approach </h2>
<p>The equations we discussed above can be coded rather easily (the
framework is similar to what we developed for logistic
regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way.
regression).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="font-size: 80%; line-height: 125%;">
</pre>
</div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
</div>
</div>
</div>
</div>
</div>
</section>
<section>
<h2 id="problems-with-the-simpler-approach">Problems with the Simpler Approach </h2>
<p>There are however problems with this approach, although it looks
pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
</p>
@@ -743,6 +721,7 @@ $$
<section>
<h2 id="setting-up-the-problem">Setting up the Problem </h2>
<p>In order to solve the above problem, we define the following Lagrangian function to be minimized </p>
<p>&nbsp;<br>
$$
doc/pub/week47/html/week47-solarized.html
+7 -30
View File
@@ -81,7 +81,6 @@ div.toc p,a {
None,
'first-attempt-at-a-minimization-approach'),
('Solving the equations', 2, None, 'solving-the-equations'),
('Code Example', 2, None, 'code-example'),
('Problems with the Simpler Approach',
2,
None,
@@ -476,6 +475,7 @@ $$
\boldsymbol{x}^T\boldsymbol{w}+b=0.
$$
<p>For figures, see <a href="https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" target="_blank">handwritten notes</a> for Thursday November 25.</p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="a-p-dimensional-space-of-features">A \( p \)-dimensional space of features </h2>
@@ -551,7 +551,9 @@ $$
f(x) = \boldsymbol{w}^T\boldsymbol{x}+b = 0,
$$
<p>as the function that determines the line \( L \) that separates two classes (our two features), see the figure here. </p>
<p>as the function that determines the line \( L \) that separates two classes (our two features), see the figures in the <a href="https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" target="_blank">handwritten notes</a> for Thursday November 25.
.
</p>
<p>Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \). </p>
@@ -600,39 +602,13 @@ $$
<p>where \( \eta \) is our by now well-known learning rate. </p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="code-example">Code Example </h2>
<h2 id="problems-with-the-simpler-approach">Problems with the Simpler Approach </h2>
<p>The equations we discussed above can be coded rather easily (the
framework is similar to what we developed for logistic
regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way.
regression).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="line-height: 125%;">
</pre>
</div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
</div>
</div>
</div>
</div>
</div>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="problems-with-the-simpler-approach">Problems with the Simpler Approach </h2>
<p>There are however problems with this approach, although it looks
pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
</p>
@@ -777,6 +753,7 @@ $$
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="setting-up-the-problem">Setting up the Problem </h2>
<p>In order to solve the above problem, we define the following Lagrangian function to be minimized </p>
$$
{\cal L}(\lambda,b,\boldsymbol{w})=\frac{1}{2}\boldsymbol{w}^T\boldsymbol{w}-\sum_{i=1}^n\lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-1\right],
doc/pub/week47/html/week47.html
+7 -30
View File
@@ -158,7 +158,6 @@ div.toc p,a {
None,
'first-attempt-at-a-minimization-approach'),
('Solving the equations', 2, None, 'solving-the-equations'),
('Code Example', 2, None, 'code-example'),
('Problems with the Simpler Approach',
2,
None,
@@ -553,6 +552,7 @@ $$
\boldsymbol{x}^T\boldsymbol{w}+b=0.
$$
<p>For figures, see <a href="https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" target="_blank">handwritten notes</a> for Thursday November 25.</p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="a-p-dimensional-space-of-features">A \( p \)-dimensional space of features </h2>
@@ -628,7 +628,9 @@ $$
f(x) = \boldsymbol{w}^T\boldsymbol{x}+b = 0,
$$
<p>as the function that determines the line \( L \) that separates two classes (our two features), see the figure here. </p>
<p>as the function that determines the line \( L \) that separates two classes (our two features), see the figures in the <a href="https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" target="_blank">handwritten notes</a> for Thursday November 25.
.
</p>
<p>Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \). </p>
@@ -677,39 +679,13 @@ $$
<p>where \( \eta \) is our by now well-known learning rate. </p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="code-example">Code Example </h2>
<h2 id="problems-with-the-simpler-approach">Problems with the Simpler Approach </h2>
<p>The equations we discussed above can be coded rather easily (the
framework is similar to what we developed for logistic
regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way.
regression).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;">
</pre>
</div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
</div>
</div>
</div>
</div>
</div>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="problems-with-the-simpler-approach">Problems with the Simpler Approach </h2>
<p>There are however problems with this approach, although it looks
pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
</p>
@@ -854,6 +830,7 @@ $$
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="setting-up-the-problem">Setting up the Problem </h2>
<p>In order to solve the above problem, we define the following Lagrangian function to be minimized </p>
$$
{\cal L}(\lambda,b,\boldsymbol{w})=\frac{1}{2}\boldsymbol{w}^T\boldsymbol{w}-\sum_{i=1}^n\lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-1\right],
doc/pub/week47/ipynb/ipynb-week47-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week47/ipynb/week47.ipynb
+245 -243
View File
File diff suppressed because it is too large Load Diff
doc/src/week47/week47.do.txt
+9 -9
View File
@@ -162,6 +162,8 @@ We can then rewrite the above equation as
\]
!et
For figures, see "handwritten notes":"https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" for Thursday November 25.
!split
===== A $p$-dimensional space of features =====
@@ -236,7 +238,8 @@ Let us define the function
f(x) = \bm{w}^T\bm{x}+b = 0,
\]
!et
as the function that determines the line $L$ that separates two classes (our two features), see the figure here.
as the function that determines the line $L$ that separates two classes (our two features), see the figures in the "handwritten notes":"https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" for Thursday November 25.
.
Any point defined by $\bm{x}_i$ and $\bm{x}_2$ on the line $L$ will satisfy $\bm{w}^T(\bm{x}_1-\bm{x}_2)=0$.
@@ -292,19 +295,15 @@ and
where $\eta$ is our by now well-known learning rate.
!split
===== Code Example =====
The equations we discussed above can be coded rather easily (the
framework is similar to what we developed for logistic
regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way.
!bc pycod
!ec
!split
===== Problems with the Simpler Approach =====
The equations we discussed above can be coded rather easily (the
framework is similar to what we developed for logistic
regression).
There are however problems with this approach, although it looks
pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
@@ -458,6 +457,7 @@ If we have a set of constraints $\phi_k$ we have the equations
!split
===== Setting up the Problem =====
In order to solve the above problem, we define the following Lagrangian function to be minimized
!bt
\[