lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating week 35

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-08-24 22:52:34 +02:00
parent 7c3a4adcaa
commit 4da18852c4
9 changed files with 174 additions and 114 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week34/html/week34-bs.html
+1 -1
View File
@@ -239,7 +239,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Aug 23, 2021</h4></center> <!-- date -->
<center><h4>Aug 24, 2021</h4></center> <!-- date -->
<br>
<p>
doc/pub/week34/html/week34-reveal.html
+34 -22
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Aug 23, 2021</h4></center> <!-- date -->
<center><h4>Aug 24, 2021</h4></center> <!-- date -->
<br>
<p>
@@ -334,7 +334,7 @@ Projects are handed in using <b>Canvas</b>. We use Github as repository for code
</ol>
<p>
In addition to the electure notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
In addition to the lecture notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
<ol>
<p><li> Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, <a href="https://www.springer.com/gp/book/9780387310732." target="_blank"><tt>https://www.springer.com/gp/book/9780387310732.</tt></a></li>
@@ -374,7 +374,19 @@ Python is the recurring programming language.
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will
This course aims at giving you insights and knowledge about many of
the central algorithms used in Data Analysis and Machine Learning.
The course is project based and through various numerical projects,
normally three, you will be exposed to fundamental research problems
in these fields, with the aim to reproduce state of the art scientific
results. Both supervised and unsupervised methods will be covered. The
emphasis is on a frequentist approach, although we will try to link it
with a Bayesian approach as well. You will learn to develop and
structure large codes for studying different cases where Machine
Learning is applied to, get acquainted with computing facilities and
learn to handle large scientific projects. A good scientific and
ethical conduct is emphasized throughout the course. More
specifically, after this course you will
<ul>
<p><li> Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning;</li>
@@ -386,7 +398,7 @@ This course aims at giving you insights and knowledge about many of the central
<p><li> Learn about support vector machines and kernel transformations;</li>
<p><li> Reduction of data sets, from PCA to clustering;</li>
<p><li> Autoencoders and Reinforcement Learning;</li>
<p><li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later).</li>
<p><li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later) or Julia or other.</li>
</ul>
</div>
</section>
@@ -441,7 +453,7 @@ The following topics will be covered
<p><li> Support vector machines</li>
<p><li> Bayesian linear and logistic regression</li>
<p><li> Boltzmann Machines</li>
<p><li> Unsupervised learning Dimensionality reduction, from PCA to cluster models</li>
<p><li> Unsupervised learning Dimensionality reduction, from PCA to clustering</li>
</ul>
<p>
@@ -462,7 +474,7 @@ Hands-on demonstrations, exercises and projects aim at deepening your understand
<p><li> GIT for version control, and GitHub or GitLab as repositories, highly recommended. This will be discussed during the first exercise session</li>
<p><li> Anaconda and other Python environments, see intro slides and first exercise session</li>
<p><li> Anaconda and other Python environments, see intro slides and links to programming resources at <a href="https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html" target="_blank"><tt>https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html</tt></a></li>
</ul>
</div>
</section>
@@ -476,7 +488,7 @@ The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus
<ol>
<p><li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<p><li> <a href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html" target="_blank">STK-INF3000/4000 Selected Topics in Data Science</a>. The course provides insight into selected contemporary relevant topics within Data Science.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html" target="_blank">IN4080 Natural Language Processing</a>. Probabilistic and machine learning techniques applied to natural language processing.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html" target="_blank">STK-IN4300 Statistical learning methods in Data Science</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</li>
@@ -862,14 +874,14 @@ $$
<p>
<table border="1">
<thead>
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
</thead>
<tbody>
<tr><td align="center"> \( A=A^{T} \) </td> <td align="center"> symmetric </td> <td align="center"> \( a_{ij}=a_{ji} \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{T} \right )^{-1} \) </td> <td align="center"> real orthogonal </td> <td align="center"> \( \sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj} = \delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{ * } \) </td> <td align="center"> real matrix </td> <td align="center"> \( a_{ij}=a_{ij}^{ * } \) </td> </tr>
<tr><td align="center"> \( A=A^{\dagger} \) </td> <td align="center"> hermitian </td> <td align="center"> \( a_{ij}=a_{ji}^{ * } \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{\dagger} \right )^{-1} \) </td> <td align="center"> unitary </td> <td align="center"> \( \sum_k a_{ik}a_{jk}^{ * }=\sum_k a_{ki}^{ * } a_{kj}=\delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{T} \) </td> <td align="center"> symmetric </td> <td align="center"> \( a_{ij}=a_{ji} \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{T}\right )^{-1} \) </td> <td align="center"> real orthogonal </td> <td align="center"> \( \sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj}=\delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{ * } \) </td> <td align="center"> real matrix </td> <td align="center"> \( a_{ij}=a_{ij}^{ < em>} \) </td> </tr>
<tr><td align="center"> \( A=A^{\dagger} \) </td> <td align="center"> hermitian </td> <td align="center"> \( a_{ij}=a_{ji}^{ < /em>} \) </td> </tr>
<tr><td align="center"> \( A=\left(A^{\dagger}\right )^{-1} \) </td> <td align="center"> unitary </td> <td align="center"> \( \sum_k a_{ik}a_{jk}^{ < em>}=\sum_k a_{ki}^{ < /em> } a_{kj}=\delta_{ij} \) </td> </tr>
</tbody>
</table>
@@ -885,17 +897,17 @@ $$
<p><li> Diagonal if \( a_{ij}=0 \) for \( i\ne j \)</li>
<p><li> Upper triangular if \( a_{ij}=0 \) for \( i > j \)</li>
<p><li> Upper triangular if \( a_{ij}=0 \) for \( i>j \)</li>
<p><li> Lower triangular if \( a_{ij}=0 \) for \( i < j \)</li>
<p><li> Upper Hessenberg if \( a_{ij}=0 \) for \( i > j+1 \)</li>
<p><li> Upper Hessenberg if \( a_{ij}=0 \) for \( i>j+1 \)</li>
<p><li> Lower Hessenberg if \( a_{ij}=0 \) for \( i < j+1 \)</li>
<p><li> Tridiagonal if \( a_{ij}=0 \) for \( |i -j| > 1 \)</li>
<p><li> Tridiagonal if \( a_{ij}=0 \) for \( |i -j|>1 \)</li>
<p><li> Lower banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i > j+p \)</li>
<p><li> Lower banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i>j+p \)</li>
<p><li> Upper banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i < j+p \)</li>
@@ -1005,7 +1017,7 @@ or simply write them as double precision numbers (Python uses 64 bits as default
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%;"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>])
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>]))
<span style="color: #658b00">print</span>(x)
</pre></div>
<p>
@@ -1014,7 +1026,7 @@ To check the number of bytes (remember that one byte contains eight bits for dou
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%;"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>])
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>]))
<span style="color: #658b00">print</span>(x.itemsize)
</pre></div>
</section>
@@ -1327,7 +1339,7 @@ pretty close the basic mathematical expressions.
<p>
Our first data set is going to be a classic from nuclear physics, namely all
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves simply as an example here of a data set.
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves merely as an example here of a data set.
<p>
We will show some of the
@@ -1571,7 +1583,7 @@ plt.show()
</pre></div>
<p>
The function <b>coef</b> gives us the parameter \( \beta \) of our fit while <b>intercept</b> yields
\( \alpha \). Depending on the constant in front of the normal distribution, we get values near or far from \( alpha =2 \) and \( \beta =5 \). Try to play around with different parameters in front of the normal distribution. The function <b>meansquarederror</b> gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
\( \alpha \). Depending on the constant in front of the normal distribution, we get values near or far from \( \alpha =2 \) and \( \beta =5 \). Try to play around with different parameters in front of the normal distribution. The function <b>meansquarederror</b> gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
<p>&nbsp;<br>
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
@@ -1642,7 +1654,7 @@ ways of dealing with outliers.
The Huber cost function is defined as
<p>&nbsp;<br>
$$
H_{\delta}(a)={\begin{cases}{\frac {1}{2}}{a^{2}}&{\text{for }}|a|\leq \delta ,\\\delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}}\end{cases}}}.
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}{a^{2}}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}\end{array}\right.
$$
<p>&nbsp;<br>
doc/pub/week34/html/week34-solarized.html
+34 -22
View File
@@ -191,7 +191,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Aug 23, 2021</h4></center> <!-- date -->
<center><h4>Aug 24, 2021</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -366,7 +366,7 @@ Projects are handed in using <b>Canvas</b>. We use Github as repository for code
<li> The lecture notes are collected as a jupyter-book at <a href="https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html." target="_blank"><tt>https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html.</tt></a></li>
</ol>
In addition to the electure notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
In addition to the lecture notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
<ol>
<li> Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, <a href="https://www.springer.com/gp/book/9780387310732." target="_blank"><tt>https://www.springer.com/gp/book/9780387310732.</tt></a></li>
@@ -406,7 +406,19 @@ Python is the recurring programming language.
<p>
<p>
This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will
This course aims at giving you insights and knowledge about many of
the central algorithms used in Data Analysis and Machine Learning.
The course is project based and through various numerical projects,
normally three, you will be exposed to fundamental research problems
in these fields, with the aim to reproduce state of the art scientific
results. Both supervised and unsupervised methods will be covered. The
emphasis is on a frequentist approach, although we will try to link it
with a Bayesian approach as well. You will learn to develop and
structure large codes for studying different cases where Machine
Learning is applied to, get acquainted with computing facilities and
learn to handle large scientific projects. A good scientific and
ethical conduct is emphasized throughout the course. More
specifically, after this course you will
<ul>
<li> Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning;</li>
@@ -418,7 +430,7 @@ This course aims at giving you insights and knowledge about many of the central
<li> Learn about support vector machines and kernel transformations;</li>
<li> Reduction of data sets, from PCA to clustering;</li>
<li> Autoencoders and Reinforcement Learning;</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later).</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later) or Julia or other.</li>
</ul>
</div>
@@ -476,7 +488,7 @@ The following topics will be covered
<li> Support vector machines</li>
<li> Bayesian linear and logistic regression</li>
<li> Boltzmann Machines</li>
<li> Unsupervised learning Dimensionality reduction, from PCA to cluster models</li>
<li> Unsupervised learning Dimensionality reduction, from PCA to clustering</li>
</ul>
Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics.
@@ -497,7 +509,7 @@ Hands-on demonstrations, exercises and projects aim at deepening your understand
<ul>
<li> GIT for version control, and GitHub or GitLab as repositories, highly recommended. This will be discussed during the first exercise session</li>
<li> Anaconda and other Python environments, see intro slides and first exercise session</li>
<li> Anaconda and other Python environments, see intro slides and links to programming resources at <a href="https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html" target="_blank"><tt>https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html</tt></a></li>
</ul>
</div>
@@ -512,7 +524,7 @@ The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus
<ol>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html" target="_blank">STK-INF3000/4000 Selected Topics in Data Science</a>. The course provides insight into selected contemporary relevant topics within Data Science.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html" target="_blank">IN4080 Natural Language Processing</a>. Probabilistic and machine learning techniques applied to natural language processing.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html" target="_blank">STK-IN4300 Statistical learning methods in Data Science</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</li>
@@ -880,14 +892,14 @@ $$
<p>
<table border="1">
<thead>
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
</thead>
<tbody>
<tr><td align="center"> \( A=A^{T} \) </td> <td align="center"> symmetric </td> <td align="center"> \( a_{ij}=a_{ji} \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{T} \right )^{-1} \) </td> <td align="center"> real orthogonal </td> <td align="center"> \( \sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj} = \delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{ * } \) </td> <td align="center"> real matrix </td> <td align="center"> \( a_{ij}=a_{ij}^{ * } \) </td> </tr>
<tr><td align="center"> \( A=A^{\dagger} \) </td> <td align="center"> hermitian </td> <td align="center"> \( a_{ij}=a_{ji}^{ * } \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{\dagger} \right )^{-1} \) </td> <td align="center"> unitary </td> <td align="center"> \( \sum_k a_{ik}a_{jk}^{ * }=\sum_k a_{ki}^{ * } a_{kj}=\delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{T} \) </td> <td align="center"> symmetric </td> <td align="center"> \( a_{ij}=a_{ji} \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{T}\right )^{-1} \) </td> <td align="center"> real orthogonal </td> <td align="center"> \( \sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj}=\delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{ * } \) </td> <td align="center"> real matrix </td> <td align="center"> \( a_{ij}=a_{ij}^{ < em>} \) </td> </tr>
<tr><td align="center"> \( A=A^{\dagger} \) </td> <td align="center"> hermitian </td> <td align="center"> \( a_{ij}=a_{ji}^{ < /em>} \) </td> </tr>
<tr><td align="center"> \( A=\left(A^{\dagger}\right )^{-1} \) </td> <td align="center"> unitary </td> <td align="center"> \( \sum_k a_{ik}a_{jk}^{ < em>}=\sum_k a_{ki}^{ < /em> } a_{kj}=\delta_{ij} \) </td> </tr>
</tbody>
</table>
@@ -901,12 +913,12 @@ $$
<ul>
<li> Diagonal if \( a_{ij}=0 \) for \( i\ne j \)</li>
<li> Upper triangular if \( a_{ij}=0 \) for \( i > j \)</li>
<li> Upper triangular if \( a_{ij}=0 \) for \( i>j \)</li>
<li> Lower triangular if \( a_{ij}=0 \) for \( i < j \)</li>
<li> Upper Hessenberg if \( a_{ij}=0 \) for \( i > j+1 \)</li>
<li> Upper Hessenberg if \( a_{ij}=0 \) for \( i>j+1 \)</li>
<li> Lower Hessenberg if \( a_{ij}=0 \) for \( i < j+1 \)</li>
<li> Tridiagonal if \( a_{ij}=0 \) for \( |i -j| > 1 \)</li>
<li> Lower banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i > j+p \)</li>
<li> Tridiagonal if \( a_{ij}=0 \) for \( |i -j|>1 \)</li>
<li> Lower banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i>j+p \)</li>
<li> Upper banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i < j+p \)</li>
<li> Banded, block upper triangular, block lower triangular....</li>
</ul>
@@ -1007,7 +1019,7 @@ or simply write them as double precision numbers (Python uses 64 bits as default
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%;"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>])
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>]))
<span style="color: #658b00">print</span>(x)
</pre></div>
<p>
@@ -1016,7 +1028,7 @@ To check the number of bytes (remember that one byte contains eight bits for dou
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%;"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>])
x = np.log(np.array([<span style="color: #B452CD">4.0</span>, <span style="color: #B452CD">7.0</span>, <span style="color: #B452CD">8.0</span>]))
<span style="color: #658b00">print</span>(x.itemsize)
</pre></div>
<p>
@@ -1321,7 +1333,7 @@ pretty close the basic mathematical expressions.
<p>
Our first data set is going to be a classic from nuclear physics, namely all
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves simply as an example here of a data set.
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves merely as an example here of a data set.
<p>
We will show some of the
@@ -1556,7 +1568,7 @@ plt.show()
</pre></div>
<p>
The function <b>coef</b> gives us the parameter \( \beta \) of our fit while <b>intercept</b> yields
\( \alpha \). Depending on the constant in front of the normal distribution, we get values near or far from \( alpha =2 \) and \( \beta =5 \). Try to play around with different parameters in front of the normal distribution. The function <b>meansquarederror</b> gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
\( \alpha \). Depending on the constant in front of the normal distribution, we get values near or far from \( \alpha =2 \) and \( \beta =5 \). Try to play around with different parameters in front of the normal distribution. The function <b>meansquarederror</b> gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
@@ -1616,7 +1628,7 @@ ways of dealing with outliers.
<p>
The Huber cost function is defined as
$$
H_{\delta}(a)={\begin{cases}{\frac {1}{2}}{a^{2}}&{\text{for }}|a|\leq \delta ,\\\delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}}\end{cases}}}.
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}{a^{2}}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}\end{array}\right.
$$
Here \( a=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
doc/pub/week34/html/week34.html
+34 -22
View File
@@ -196,7 +196,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Aug 23, 2021</h4></center> <!-- date -->
<center><h4>Aug 24, 2021</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -371,7 +371,7 @@ Projects are handed in using <b>Canvas</b>. We use Github as repository for code
<li> The lecture notes are collected as a jupyter-book at <a href="https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html." target="_blank"><tt>https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html.</tt></a></li>
</ol>
In addition to the electure notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
In addition to the lecture notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
<ol>
<li> Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, <a href="https://www.springer.com/gp/book/9780387310732." target="_blank"><tt>https://www.springer.com/gp/book/9780387310732.</tt></a></li>
@@ -411,7 +411,19 @@ Python is the recurring programming language.
<p>
<p>
This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will
This course aims at giving you insights and knowledge about many of
the central algorithms used in Data Analysis and Machine Learning.
The course is project based and through various numerical projects,
normally three, you will be exposed to fundamental research problems
in these fields, with the aim to reproduce state of the art scientific
results. Both supervised and unsupervised methods will be covered. The
emphasis is on a frequentist approach, although we will try to link it
with a Bayesian approach as well. You will learn to develop and
structure large codes for studying different cases where Machine
Learning is applied to, get acquainted with computing facilities and
learn to handle large scientific projects. A good scientific and
ethical conduct is emphasized throughout the course. More
specifically, after this course you will
<ul>
<li> Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning;</li>
@@ -423,7 +435,7 @@ This course aims at giving you insights and knowledge about many of the central
<li> Learn about support vector machines and kernel transformations;</li>
<li> Reduction of data sets, from PCA to clustering;</li>
<li> Autoencoders and Reinforcement Learning;</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later).</li>
<li> Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later) or Julia or other.</li>
</ul>
</div>
@@ -481,7 +493,7 @@ The following topics will be covered
<li> Support vector machines</li>
<li> Bayesian linear and logistic regression</li>
<li> Boltzmann Machines</li>
<li> Unsupervised learning Dimensionality reduction, from PCA to cluster models</li>
<li> Unsupervised learning Dimensionality reduction, from PCA to clustering</li>
</ul>
Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics.
@@ -502,7 +514,7 @@ Hands-on demonstrations, exercises and projects aim at deepening your understand
<ul>
<li> GIT for version control, and GitHub or GitLab as repositories, highly recommended. This will be discussed during the first exercise session</li>
<li> Anaconda and other Python environments, see intro slides and first exercise session</li>
<li> Anaconda and other Python environments, see intro slides and links to programming resources at <a href="https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html" target="_blank"><tt>https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html</tt></a></li>
</ul>
</div>
@@ -517,7 +529,7 @@ The link here <a href="https://www.mn.uio.no/english/research/about/centre-focus
<ol>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html" target="_blank">STK-INF3000/4000 Selected Topics in Data Science</a>. The course provides insight into selected contemporary relevant topics within Data Science.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html" target="_blank">IN4080 Natural Language Processing</a>. Probabilistic and machine learning techniques applied to natural language processing.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html" target="_blank">STK-IN4300 Statistical learning methods in Data Science</a>. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.</li>
@@ -885,14 +897,14 @@ $$
<p>
<table border="1">
<thead>
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
</thead>
<tbody>
<tr><td align="center"> \( A=A^{T} \) </td> <td align="center"> symmetric </td> <td align="center"> \( a_{ij}=a_{ji} \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{T} \right )^{-1} \) </td> <td align="center"> real orthogonal </td> <td align="center"> \( \sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj} = \delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{ * } \) </td> <td align="center"> real matrix </td> <td align="center"> \( a_{ij}=a_{ij}^{ * } \) </td> </tr>
<tr><td align="center"> \( A=A^{\dagger} \) </td> <td align="center"> hermitian </td> <td align="center"> \( a_{ij}=a_{ji}^{ * } \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{\dagger} \right )^{-1} \) </td> <td align="center"> unitary </td> <td align="center"> \( \sum_k a_{ik}a_{jk}^{ * }=\sum_k a_{ki}^{ * } a_{kj}=\delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{T} \) </td> <td align="center"> symmetric </td> <td align="center"> \( a_{ij}=a_{ji} \) </td> </tr>
<tr><td align="center"> \( A=\left (A^{T}\right )^{-1} \) </td> <td align="center"> real orthogonal </td> <td align="center"> \( \sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj}=\delta_{ij} \) </td> </tr>
<tr><td align="center"> \( A=A^{ * } \) </td> <td align="center"> real matrix </td> <td align="center"> \( a_{ij}=a_{ij}^{ < em>} \) </td> </tr>
<tr><td align="center"> \( A=A^{\dagger} \) </td> <td align="center"> hermitian </td> <td align="center"> \( a_{ij}=a_{ji}^{ < /em>} \) </td> </tr>
<tr><td align="center"> \( A=\left(A^{\dagger}\right )^{-1} \) </td> <td align="center"> unitary </td> <td align="center"> \( \sum_k a_{ik}a_{jk}^{ < em>}=\sum_k a_{ki}^{ < /em> } a_{kj}=\delta_{ij} \) </td> </tr>
</tbody>
</table>
@@ -906,12 +918,12 @@ $$
<ul>
<li> Diagonal if \( a_{ij}=0 \) for \( i\ne j \)</li>
<li> Upper triangular if \( a_{ij}=0 \) for \( i > j \)</li>
<li> Upper triangular if \( a_{ij}=0 \) for \( i>j \)</li>
<li> Lower triangular if \( a_{ij}=0 \) for \( i < j \)</li>
<li> Upper Hessenberg if \( a_{ij}=0 \) for \( i > j+1 \)</li>
<li> Upper Hessenberg if \( a_{ij}=0 \) for \( i>j+1 \)</li>
<li> Lower Hessenberg if \( a_{ij}=0 \) for \( i < j+1 \)</li>
<li> Tridiagonal if \( a_{ij}=0 \) for \( |i -j| > 1 \)</li>
<li> Lower banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i > j+p \)</li>
<li> Tridiagonal if \( a_{ij}=0 \) for \( |i -j|>1 \)</li>
<li> Lower banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i>j+p \)</li>
<li> Upper banded with bandwidth \( p \): \( a_{ij}=0 \) for \( i < j+p \)</li>
<li> Banded, block upper triangular, block lower triangular....</li>
</ul>
@@ -1012,7 +1024,7 @@ or simply write them as double precision numbers (Python uses 64 bits as default
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%;"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>log(np<span style="color: #666666">.</span>array([<span style="color: #666666">4.0</span>, <span style="color: #666666">7.0</span>, <span style="color: #666666">8.0</span>])
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>log(np<span style="color: #666666">.</span>array([<span style="color: #666666">4.0</span>, <span style="color: #666666">7.0</span>, <span style="color: #666666">8.0</span>]))
<span style="color: #008000">print</span>(x)
</pre></div>
<p>
@@ -1021,7 +1033,7 @@ To check the number of bytes (remember that one byte contains eight bits for dou
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%;"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>log(np<span style="color: #666666">.</span>array([<span style="color: #666666">4.0</span>, <span style="color: #666666">7.0</span>, <span style="color: #666666">8.0</span>])
x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>log(np<span style="color: #666666">.</span>array([<span style="color: #666666">4.0</span>, <span style="color: #666666">7.0</span>, <span style="color: #666666">8.0</span>]))
<span style="color: #008000">print</span>(x<span style="color: #666666">.</span>itemsize)
</pre></div>
<p>
@@ -1326,7 +1338,7 @@ pretty close the basic mathematical expressions.
<p>
Our first data set is going to be a classic from nuclear physics, namely all
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves simply as an example here of a data set.
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves merely as an example here of a data set.
<p>
We will show some of the
@@ -1561,7 +1573,7 @@ plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
The function <b>coef</b> gives us the parameter \( \beta \) of our fit while <b>intercept</b> yields
\( \alpha \). Depending on the constant in front of the normal distribution, we get values near or far from \( alpha =2 \) and \( \beta =5 \). Try to play around with different parameters in front of the normal distribution. The function <b>meansquarederror</b> gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
\( \alpha \). Depending on the constant in front of the normal distribution, we get values near or far from \( \alpha =2 \) and \( \beta =5 \). Try to play around with different parameters in front of the normal distribution. The function <b>meansquarederror</b> gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
@@ -1621,7 +1633,7 @@ ways of dealing with outliers.
<p>
The Huber cost function is defined as
$$
H_{\delta}(a)={\begin{cases}{\frac {1}{2}}{a^{2}}&{\text{for }}|a|\leq \delta ,\\\delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}}\end{cases}}}.
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}{a^{2}}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}\end{array}\right.
$$
Here \( a=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
doc/pub/week34/ipynb/Results/FigureFiles/Masses2016.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

doc/pub/week34/ipynb/Results/FigureFiles/Masses2016Trees.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 21 KiB

doc/pub/week34/ipynb/ipynb-week34-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week34/ipynb/week34.ipynb
+37 -25
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Aug 23, 2021**\n",
"Date: **Aug 24, 2021**\n",
"\n",
"Copyright 1999-2021, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -151,7 +151,7 @@
"\n",
"1. The lecture notes are collected as a jupyter-book at <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html.>\n",
"\n",
"In addition to the electure notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.\n",
"In addition to the lecture notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.\n",
"\n",
"1. Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, <https://www.springer.com/gp/book/9780387310732.> \n",
"\n",
@@ -181,7 +181,19 @@
"\n",
"\n",
"\n",
"This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will\n",
"This course aims at giving you insights and knowledge about many of\n",
"the central algorithms used in Data Analysis and Machine Learning.\n",
"The course is project based and through various numerical projects,\n",
"normally three, you will be exposed to fundamental research problems\n",
"in these fields, with the aim to reproduce state of the art scientific\n",
"results. Both supervised and unsupervised methods will be covered. The\n",
"emphasis is on a frequentist approach, although we will try to link it\n",
"with a Bayesian approach as well. You will learn to develop and\n",
"structure large codes for studying different cases where Machine\n",
"Learning is applied to, get acquainted with computing facilities and\n",
"learn to handle large scientific projects. A good scientific and\n",
"ethical conduct is emphasized throughout the course. More\n",
"specifically, after this course you will\n",
"\n",
"* Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning;\n",
"\n",
@@ -201,7 +213,7 @@
"\n",
"* Autoencoders and Reinforcement Learning;\n",
"\n",
"* Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later).\n",
"* Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later) or Julia or other.\n",
"\n",
"\n",
"\n",
@@ -251,7 +263,7 @@
"\n",
"* Boltzmann Machines\n",
"\n",
"* Unsupervised learning Dimensionality reduction, from PCA to cluster models\n",
"* Unsupervised learning Dimensionality reduction, from PCA to clustering\n",
"\n",
"Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics.\n",
"\n",
@@ -264,7 +276,7 @@
"\n",
" * GIT for version control, and GitHub or GitLab as repositories, highly recommended. This will be discussed during the first exercise session\n",
"\n",
" * Anaconda and other Python environments, see intro slides and first exercise session\n",
" * Anaconda and other Python environments, see intro slides and links to programming resources at <https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html>\n",
"\n",
"\n",
"\n",
@@ -278,7 +290,7 @@
"\n",
"1. [STK2100 Machine learning and statistical methods for prediction and classification](http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html). \n",
"\n",
"2. [IN3050 Introduction to Artificial Intelligence and Machine Learning](https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html). Introductory course in machine learning and AI with an algorithmic approach. \n",
"2. [IN3050/4050 Introduction to Artificial Intelligence and Machine Learning](https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html). Introductory course in machine learning and AI with an algorithmic approach. \n",
"\n",
"3. [STK-INF3000/4000 Selected Topics in Data Science](http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html). The course provides insight into selected contemporary relevant topics within Data Science. \n",
"\n",
@@ -638,14 +650,14 @@
"source": [
"<table border=\"1\">\n",
"<thead>\n",
"<tr><th align=\"center\"> Relations </th> <th align=\"center\"> Name </th> <th align=\"center\"> matrix elements </th> </tr>\n",
"<tr><th align=\"center\"> Relations </th> <th align=\"center\"> Name </th> <th align=\"center\"> matrix elements </th> </tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td align=\"center\"> $A=A^{T}$ </td> <td align=\"center\"> symmetric </td> <td align=\"center\"> $a_{ij}=a_{ji}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=\\left (A^{T} \\right )^{-1}$ </td> <td align=\"center\"> real orthogonal </td> <td align=\"center\"> $\\sum_k a_{ik}a_{jk}=\\sum_k a_{ki} a_{kj} = \\delta_{ij}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=A^{ * }$ </td> <td align=\"center\"> real matrix </td> <td align=\"center\"> $a_{ij}=a_{ij}^{ * }$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=A^{\\dagger}$ </td> <td align=\"center\"> hermitian </td> <td align=\"center\"> $a_{ij}=a_{ji}^{ * }$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=\\left (A^{\\dagger} \\right )^{-1}$ </td> <td align=\"center\"> unitary </td> <td align=\"center\"> $\\sum_k a_{ik}a_{jk}^{ * }=\\sum_k a_{ki}^{ * } a_{kj}=\\delta_{ij}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=A^{T}$ </td> <td align=\"center\"> symmetric </td> <td align=\"center\"> $a_{ij}=a_{ji}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=\\left (A^{T}\\right )^{-1}$ </td> <td align=\"center\"> real orthogonal </td> <td align=\"center\"> $\\sum_k a_{ik}a_{jk}=\\sum_k a_{ki} a_{kj}=\\delta_{ij}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=A^{ * }$ </td> <td align=\"center\"> real matrix </td> <td align=\"center\"> $a_{ij}=a_{ij}^{*}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=A^{\\dagger}$ </td> <td align=\"center\"> hermitian </td> <td align=\"center\"> $a_{ij}=a_{ji}^{*}$ </td> </tr>\n",
"<tr><td align=\"center\"> $A=\\left(A^{\\dagger}\\right )^{-1}$ </td> <td align=\"center\"> unitary </td> <td align=\"center\"> $\\sum_k a_{ik}a_{jk}^{*}=\\sum_k a_{ki}^{ * } a_{kj}=\\delta_{ij}$ </td> </tr>\n",
"</tbody>\n",
"</table>\n",
"\n",
@@ -655,19 +667,19 @@
"\n",
" * Diagonal if $a_{ij}=0$ for $i\\ne j$\n",
"\n",
" * Upper triangular if $a_{ij}=0$ for $i > j$\n",
" * Upper triangular if $a_{ij}=0$ for $i>j$\n",
"\n",
" * Lower triangular if $a_{ij}=0$ for $i < j$\n",
" * Lower triangular if $a_{ij}=0$ for $i<j$\n",
"\n",
" * Upper Hessenberg if $a_{ij}=0$ for $i > j+1$\n",
" * Upper Hessenberg if $a_{ij}=0$ for $i>j+1$\n",
"\n",
" * Lower Hessenberg if $a_{ij}=0$ for $i < j+1$\n",
" * Lower Hessenberg if $a_{ij}=0$ for $i<j+1$\n",
"\n",
" * Tridiagonal if $a_{ij}=0$ for $|i -j| > 1$\n",
" * Tridiagonal if $a_{ij}=0$ for $|i -j|>1$\n",
"\n",
" * Lower banded with bandwidth $p$: $a_{ij}=0$ for $i > j+p$\n",
" * Lower banded with bandwidth $p$: $a_{ij}=0$ for $i>j+p$\n",
"\n",
" * Upper banded with bandwidth $p$: $a_{ij}=0$ for $i < j+p$\n",
" * Upper banded with bandwidth $p$: $a_{ij}=0$ for $i<j+p$\n",
"\n",
" * Banded, block upper triangular, block lower triangular....\n",
"\n",
@@ -841,7 +853,7 @@
"outputs": [],
"source": [
"import numpy as np\n",
"x = np.log(np.array([4.0, 7.0, 8.0])\n",
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
"print(x)"
]
},
@@ -862,7 +874,7 @@
"outputs": [],
"source": [
"import numpy as np\n",
"x = np.log(np.array([4.0, 7.0, 8.0])\n",
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
"print(x.itemsize)"
]
},
@@ -1366,7 +1378,7 @@
"pretty close the basic mathematical expressions.\n",
"\n",
"Our first data set is going to be a classic from nuclear physics, namely all\n",
"available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves simply as an example here of a data set. \n",
"available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves merely as an example here of a data set. \n",
"\n",
"We will show some of the\n",
"strengths of packages like **Scikit-Learn** in fitting nuclear binding energies to\n",
@@ -1656,7 +1668,7 @@
"metadata": {},
"source": [
"The function **coef** gives us the parameter $\\beta$ of our fit while **intercept** yields \n",
"$\\alpha$. Depending on the constant in front of the normal distribution, we get values near or far from $alpha =2$ and $\\beta =5$. Try to play around with different parameters in front of the normal distribution. The function **meansquarederror** gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as"
"$\\alpha$. Depending on the constant in front of the normal distribution, we get values near or far from $\\alpha =2$ and $\\beta =5$. Try to play around with different parameters in front of the normal distribution. The function **meansquarederror** gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as"
]
},
{
@@ -1772,7 +1784,7 @@
"metadata": {},
"source": [
"$$\n",
"H_{\\delta}(a)={\\begin{cases}{\\frac {1}{2}}{a^{2}}&{\\text{for }}|a|\\leq \\delta ,\\\\\\delta (|a|-{\\frac {1}{2}}\\delta ),&{\\text{otherwise.}}\\end{cases}}}.\n",
"H_{\\delta}(a)=\\left\\{\\begin{array}\\frac{1}{2}{a^{2}}&{\\text{for }}|a|\\leq \\delta ,\\\\ \\delta (|a|-{\\frac {1}{2}}\\delta ),&{\\text{otherwise.}\\end{array}\\right.\n",
"$$"
]
},
doc/src/week34/week34.do.txt
+34 -22
View File
@@ -113,7 +113,7 @@ Projects are handed in using _Canvas_. We use Github as repository for codes, be
o The lecture notes are collected as a jupyter-book at https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html.
In addition to the electure notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
In addition to the lecture notes, we recommend the books of Bishop and Goodfellow et al. We will follow these texts closely and the weekly reading assignments refer to these two texts. The text by Hastie et al is also widely used in the Machine Learning community. Finally, we also recommend the hands-on text by Geron, see below.
o Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, https://www.springer.com/gp/book/9780387310732.
@@ -148,7 +148,19 @@ Python is the recurring programming language.
!bblock
This course aims at giving you insights and knowledge about many of the central algorithms used in Data Analysis and Machine Learning. The course is project based and through various numerical projects, normally three, you will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. The emphasis is on a frequentist approach, although we will try to link it with a Bayesian approach as well. You will learn to develop and structure large codes for studying different cases where Machine Learning is applied to, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will
This course aims at giving you insights and knowledge about many of
the central algorithms used in Data Analysis and Machine Learning.
The course is project based and through various numerical projects,
normally three, you will be exposed to fundamental research problems
in these fields, with the aim to reproduce state of the art scientific
results. Both supervised and unsupervised methods will be covered. The
emphasis is on a frequentist approach, although we will try to link it
with a Bayesian approach as well. You will learn to develop and
structure large codes for studying different cases where Machine
Learning is applied to, get acquainted with computing facilities and
learn to handle large scientific projects. A good scientific and
ethical conduct is emphasized throughout the course. More
specifically, after this course you will
* Learn about basic data analysis, statistical analysis, Bayesian statistics, Monte Carlo sampling, data optimization and machine learning;
* Be capable of extending the acquired knowledge to other systems and cases;
@@ -159,7 +171,7 @@ This course aims at giving you insights and knowledge about many of the central
* Learn about support vector machines and kernel transformations;
* Reduction of data sets, from PCA to clustering;
* Autoencoders and Reinforcement Learning;
* Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later).
* Work on numerical projects to illustrate the theory. The projects play a central role and you are expected to know modern programming languages like Python or C++ and/or Fortran (Fortran2003 or later) or Julia or other.
!eblock
@@ -198,7 +210,7 @@ The following topics will be covered
* Support vector machines
* Bayesian linear and logistic regression
* Boltzmann Machines
* Unsupervised learning Dimensionality reduction, from PCA to cluster models
* Unsupervised learning Dimensionality reduction, from PCA to clustering
Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics.
@@ -210,7 +222,7 @@ Hands-on demonstrations, exercises and projects aim at deepening your understand
!bblock and discussed at the lab sessions
* GIT for version control, and GitHub or GitLab as repositories, highly recommended. This will be discussed during the first exercise session
* Anaconda and other Python environments, see intro slides and first exercise session
* Anaconda and other Python environments, see intro slides and links to programming resources at URL:"https://computationalscienceuio.github.io/RefreshProgrammingSkills/intro.html"
!eblock
@@ -223,7 +235,7 @@ Hands-on demonstrations, exercises and projects aim at deepening your understand
The link here URL:"https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" gives an excellent overview of courses on Machine learning at UiO.
o "STK2100 Machine learning and statistical methods for prediction and classification":"http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html".
o "IN3050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI with an algorithmic approach.
o "IN3050/4050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI with an algorithmic approach.
o "STK-INF3000/4000 Selected Topics in Data Science":"http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html". The course provides insight into selected contemporary relevant topics within Data Science.
o "IN4080 Natural Language Processing":"https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html". Probabilistic and machine learning techniques applied to natural language processing.
o "STK-IN4300 Statistical learning methods in Data Science":"https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html". An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.
@@ -555,10 +567,10 @@ The inverse of a matrix is defined by
| Relations | Name | matrix elements |
|----------------------------------------------------------------------|
| $A=A^{T}$ | symmetric | $a_{ij}=a_{ji}$ |
| $A=\left (A^{T} \right )^{-1}$ | real orthogonal | $\sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj} = \delta_{ij}$ |
| $A=A^{ * }$ | real matrix | $a_{ij}=a_{ij}^{ * }$ |
| $A=A^{\dagger}$ | hermitian | $a_{ij}=a_{ji}^{ * }$ |
| $A=\left (A^{\dagger} \right )^{-1}$ | unitary | $\sum_k a_{ik}a_{jk}^{ * }=\sum_k a_{ki}^{ * } a_{kj}=\delta_{ij}$ |
| $A=\left (A^{T}\right )^{-1}$ | real orthogonal | $\sum_k a_{ik}a_{jk}=\sum_k a_{ki} a_{kj}=\delta_{ij}$ |
| $A=A^{ * }$ | real matrix | $a_{ij}=a_{ij}^{*}$ |
| $A=A^{\dagger}$ | hermitian | $a_{ij}=a_{ji}^{*}$ |
| $A=\left(A^{\dagger}\right )^{-1}$ | unitary | $\sum_k a_{ik}a_{jk}^{*}=\sum_k a_{ki}^{ * } a_{kj}=\delta_{ij}$ |
|----------------------------------------------------------------------|
!eblock
@@ -567,13 +579,13 @@ The inverse of a matrix is defined by
=== Some famous Matrices ===
* Diagonal if $a_{ij}=0$ for $i\ne j$
* Upper triangular if $a_{ij}=0$ for $i > j$
* Lower triangular if $a_{ij}=0$ for $i < j$
* Upper Hessenberg if $a_{ij}=0$ for $i > j+1$
* Lower Hessenberg if $a_{ij}=0$ for $i < j+1$
* Tridiagonal if $a_{ij}=0$ for $|i -j| > 1$
* Lower banded with bandwidth $p$: $a_{ij}=0$ for $i > j+p$
* Upper banded with bandwidth $p$: $a_{ij}=0$ for $i < j+p$
* Upper triangular if $a_{ij}=0$ for $i>j$
* Lower triangular if $a_{ij}=0$ for $i<j$
* Upper Hessenberg if $a_{ij}=0$ for $i>j+1$
* Lower Hessenberg if $a_{ij}=0$ for $i<j+1$
* Tridiagonal if $a_{ij}=0$ for $|i -j|>1$
* Lower banded with bandwidth $p$: $a_{ij}=0$ for $i>j+p$
* Upper banded with bandwidth $p$: $a_{ij}=0$ for $i<j+p$
* Banded, block upper triangular, block lower triangular....
@@ -645,13 +657,13 @@ print(x)
or simply write them as double precision numbers (Python uses 64 bits as default for floating point type variables), that is
!bc pycod
import numpy as np
x = np.log(np.array([4.0, 7.0, 8.0])
x = np.log(np.array([4.0, 7.0, 8.0]))
print(x)
!ec
To check the number of bytes (remember that one byte contains eight bits for double precision variables), you can use simple use the _itemsize_ functionality (the array $x$ is actually an object which inherits the functionalities defined in Numpy) as
!bc pycod
import numpy as np
x = np.log(np.array([4.0, 7.0, 8.0])
x = np.log(np.array([4.0, 7.0, 8.0]))
print(x.itemsize)
!ec
@@ -914,7 +926,7 @@ into the calculational formats we need for a specific algorithm. And our code is
pretty close the basic mathematical expressions.
Our first data set is going to be a classic from nuclear physics, namely all
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves simply as an example here of a data set.
available data on binding energies. Don't be intimidated if you are not familiar with nuclear physics. It serves merely as an example here of a data set.
We will show some of the
strengths of packages like _Scikit-Learn_ in fitting nuclear binding energies to
@@ -1131,7 +1143,7 @@ plt.show()
!ec
The function _coef_ gives us the parameter $\beta$ of our fit while _intercept_ yields
$\alpha$. Depending on the constant in front of the normal distribution, we get values near or far from $alpha =2$ and $\beta =5$. Try to play around with different parameters in front of the normal distribution. The function _meansquarederror_ gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
$\alpha$. Depending on the constant in front of the normal distribution, we get values near or far from $\alpha =2$ and $\beta =5$. Try to play around with different parameters in front of the normal distribution. The function _meansquarederror_ gives us the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error or loss defined as
!bt
\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
@@ -1193,7 +1205,7 @@ ways of dealing with outliers.
The Huber cost function is defined as
!bt
\[
H_{\delta}(a)={\begin{cases}{\frac {1}{2}}{a^{2}}&{\text{for }}|a|\leq \delta ,\\\delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}}\end{cases}}}.
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}{a^{2}}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-{\frac {1}{2}}\delta ),&{\text{otherwise.}\end{array}\right.
\]
!et
Here $a=\bm{y} - \bm{\tilde{y}}$.