lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-11-23 09:03:57 +01:00
parent 346adf475b
commit a51d2ae0f2
10 changed files with 384 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week47/html/._week47-bs004.html
+34 -1
View File
@@ -391,18 +391,51 @@ MathJax.Hub.Config({
a decision tree wth different depths and perform a bootstrap aggregate (in this case we perform as many bootstraps as data points \( n \)).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<pre style="line-height: 125%;"><span style="color: #408080; font-style: italic"># Common imports</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.pipeline</span> <span style="color: #008000; font-weight: bold">import</span> make_pipeline
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.utils</span> <span style="color: #008000; font-weight: bold">import</span> resample
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeRegressor
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeClassifier
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler, OneHotEncoder
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.compose</span> <span style="color: #008000; font-weight: bold">import</span> ColumnTransformer
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> Image
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">os</span>
<span style="color: #408080; font-style: italic"># Where to save the figures and data files</span>
PROJECT_ROOT_DIR <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Results&quot;</span>
FIGURE_ID <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Results/FigureFiles&quot;</span>
DATA_ID <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;DataFiles/&quot;</span>
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #AA22FF; font-weight: bold">not</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>exists(PROJECT_ROOT_DIR):
os<span style="color: #666666">.</span>mkdir(PROJECT_ROOT_DIR)
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #AA22FF; font-weight: bold">not</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>exists(FIGURE_ID):
os<span style="color: #666666">.</span>makedirs(FIGURE_ID)
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #AA22FF; font-weight: bold">not</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>exists(DATA_ID):
os<span style="color: #666666">.</span>makedirs(DATA_ID)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">image_path</span>(fig_id):
<span style="color: #008000; font-weight: bold">return</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>join(FIGURE_ID, fig_id)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">data_path</span>(dat_id):
<span style="color: #008000; font-weight: bold">return</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>join(DATA_ID, dat_id)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">save_fig</span>(fig_id):
plt<span style="color: #666666">.</span>savefig(image_path(fig_id) <span style="color: #666666">+</span> <span style="color: #BA2121">&quot;.png&quot;</span>, <span style="color: #008000">format</span><span style="color: #666666">=</span><span style="color: #BA2121">&#39;png&#39;</span>)
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
n_boostraps <span style="color: #666666">=</span> <span style="color: #666666">100</span>
doc/pub/week47/html/._week47-bs045.html
+1 -1
View File
@@ -388,7 +388,7 @@ MathJax.Hub.Config({
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio" class="anchor">Other courses on Data science and Machine Learning at UiO </h2>
<ol>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_self">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_self">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a>. Discussed deep learning and generative deep learning.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html" target="_self">FYS5419 Quantum Computing and Quantum Machine Learning</a></li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_self">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_self">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
doc/pub/week47/html/._week47-bs047.html
+1
View File
@@ -403,6 +403,7 @@ networks have been proposed, such as
<li> Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing;</li>
<li> Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing;</li>
<li> Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning.</li>
<li> <b>Generative deep learning</b>! Recent textbook by David Foster (and obviously many other ones) at <a href="https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/" target="_self"><tt>https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/</tt></a>"</li>
</ol>
<p>
<!-- navigation buttons at the bottom of the page -->
doc/pub/week47/html/._week47-bs050.html
+1 -1
View File
@@ -387,7 +387,7 @@ MathJax.Hub.Config({
<!-- !split -->
<h2 id="boltzmann-machines" class="anchor">Boltzmann Machines </h2>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? </p>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? <b>Simplest approach to generative deep learning</b>.</p>
<ul>
<li> Discriminitave methods have several limitations: They are mainly supervised learning methods, thus requiring labeled data. And there are tasks they cannot accomplish, like drawing new examples from an unknown probability distribution.</li>
doc/pub/week47/html/week47-reveal.html
+37 -3
View File
@@ -289,18 +289,51 @@ predictor, averaged over all \( B \) trees.
a decision tree wth different depths and perform a bootstrap aggregate (in this case we perform as many bootstraps as data points \( n \)).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #228B22"># Common imports</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.model_selection</span> <span style="color: #8B008B; font-weight: bold">import</span> train_test_split
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.pipeline</span> <span style="color: #8B008B; font-weight: bold">import</span> make_pipeline
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.utils</span> <span style="color: #8B008B; font-weight: bold">import</span> resample
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.tree</span> <span style="color: #8B008B; font-weight: bold">import</span> DecisionTreeRegressor
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.tree</span> <span style="color: #8B008B; font-weight: bold">import</span> DecisionTreeClassifier
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.model_selection</span> <span style="color: #8B008B; font-weight: bold">import</span> train_test_split
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.preprocessing</span> <span style="color: #8B008B; font-weight: bold">import</span> StandardScaler, OneHotEncoder
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.compose</span> <span style="color: #8B008B; font-weight: bold">import</span> ColumnTransformer
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> Image
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">os</span>
<span style="color: #228B22"># Where to save the figures and data files</span>
PROJECT_ROOT_DIR = <span style="color: #CD5555">&quot;Results&quot;</span>
FIGURE_ID = <span style="color: #CD5555">&quot;Results/FigureFiles&quot;</span>
DATA_ID = <span style="color: #CD5555">&quot;DataFiles/&quot;</span>
<span style="color: #8B008B; font-weight: bold">if</span> <span style="color: #8B008B">not</span> os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
<span style="color: #8B008B; font-weight: bold">if</span> <span style="color: #8B008B">not</span> os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
<span style="color: #8B008B; font-weight: bold">if</span> <span style="color: #8B008B">not</span> os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">image_path</span>(fig_id):
<span style="color: #8B008B; font-weight: bold">return</span> os.path.join(FIGURE_ID, fig_id)
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">data_path</span>(dat_id):
<span style="color: #8B008B; font-weight: bold">return</span> os.path.join(DATA_ID, dat_id)
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">save_fig</span>(fig_id):
plt.savefig(image_path(fig_id) + <span style="color: #CD5555">&quot;.png&quot;</span>, <span style="color: #658b00">format</span>=<span style="color: #CD5555">&#39;png&#39;</span>)
n = <span style="color: #B452CD">100</span>
n_boostraps = <span style="color: #B452CD">100</span>
@@ -1675,7 +1708,7 @@ set of hyperparameters and regularization methods.
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio">Other courses on Data science and Machine Learning at UiO </h2>
<ol>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a></li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a>. Discussed deep learning and generative deep learning.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html" target="_blank">FYS5419 Quantum Computing and Quantum Machine Learning</a></li>
<p><li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<p><li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
@@ -1715,6 +1748,7 @@ networks have been proposed, such as
<p><li> Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing;</li>
<p><li> Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing;</li>
<p><li> Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning.</li>
<p><li> <b>Generative deep learning</b>! Recent textbook by David Foster (and obviously many other ones) at <a href="https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/" target="_blank"><tt>https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/</tt></a>"</li>
</ol>
</section>
@@ -1765,7 +1799,7 @@ One of the major reasons is that they can be stacked layer-wise to build deep ne
<section>
<h2 id="boltzmann-machines">Boltzmann Machines </h2>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? </p>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? <b>Simplest approach to generative deep learning</b>.</p>
<ul>
<p><li> Discriminitave methods have several limitations: They are mainly supervised learning methods, thus requiring labeled data. And there are tasks they cannot accomplish, like drawing new examples from an unknown probability distribution.</li>
doc/pub/week47/html/week47-solarized.html
+37 -3
View File
@@ -411,18 +411,51 @@ predictor, averaged over all \( B \) trees.
a decision tree wth different depths and perform a bootstrap aggregate (in this case we perform as many bootstraps as data points \( n \)).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="line-height: 125%;"><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<pre style="line-height: 125%;"><span style="color: #228B22"># Common imports</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">matplotlib.pyplot</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">plt</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.model_selection</span> <span style="color: #8B008B; font-weight: bold">import</span> train_test_split
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.pipeline</span> <span style="color: #8B008B; font-weight: bold">import</span> make_pipeline
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.utils</span> <span style="color: #8B008B; font-weight: bold">import</span> resample
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.tree</span> <span style="color: #8B008B; font-weight: bold">import</span> DecisionTreeRegressor
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.tree</span> <span style="color: #8B008B; font-weight: bold">import</span> DecisionTreeClassifier
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.model_selection</span> <span style="color: #8B008B; font-weight: bold">import</span> train_test_split
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.preprocessing</span> <span style="color: #8B008B; font-weight: bold">import</span> StandardScaler, OneHotEncoder
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.compose</span> <span style="color: #8B008B; font-weight: bold">import</span> ColumnTransformer
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> Image
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">os</span>
<span style="color: #228B22"># Where to save the figures and data files</span>
PROJECT_ROOT_DIR = <span style="color: #CD5555">&quot;Results&quot;</span>
FIGURE_ID = <span style="color: #CD5555">&quot;Results/FigureFiles&quot;</span>
DATA_ID = <span style="color: #CD5555">&quot;DataFiles/&quot;</span>
<span style="color: #8B008B; font-weight: bold">if</span> <span style="color: #8B008B">not</span> os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
<span style="color: #8B008B; font-weight: bold">if</span> <span style="color: #8B008B">not</span> os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
<span style="color: #8B008B; font-weight: bold">if</span> <span style="color: #8B008B">not</span> os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">image_path</span>(fig_id):
<span style="color: #8B008B; font-weight: bold">return</span> os.path.join(FIGURE_ID, fig_id)
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">data_path</span>(dat_id):
<span style="color: #8B008B; font-weight: bold">return</span> os.path.join(DATA_ID, dat_id)
<span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">save_fig</span>(fig_id):
plt.savefig(image_path(fig_id) + <span style="color: #CD5555">&quot;.png&quot;</span>, <span style="color: #658b00">format</span>=<span style="color: #CD5555">&#39;png&#39;</span>)
n = <span style="color: #B452CD">100</span>
n_boostraps = <span style="color: #B452CD">100</span>
@@ -1655,7 +1688,7 @@ set of hyperparameters and regularization methods.
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio">Other courses on Data science and Machine Learning at UiO </h2>
<ol>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a>. Discussed deep learning and generative deep learning.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html" target="_blank">FYS5419 Quantum Computing and Quantum Machine Learning</a></li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
@@ -1691,6 +1724,7 @@ networks have been proposed, such as
<li> Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing;</li>
<li> Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing;</li>
<li> Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning.</li>
<li> <b>Generative deep learning</b>! Recent textbook by David Foster (and obviously many other ones) at <a href="https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/" target="_blank"><tt>https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/</tt></a>"</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="types-of-machine-learning-a-repetition">Types of Machine Learning, a repetition </h2>
@@ -1734,7 +1768,7 @@ One of the major reasons is that they can be stacked layer-wise to build deep ne
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="boltzmann-machines">Boltzmann Machines </h2>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? </p>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? <b>Simplest approach to generative deep learning</b>.</p>
<ul>
<li> Discriminitave methods have several limitations: They are mainly supervised learning methods, thus requiring labeled data. And there are tasks they cannot accomplish, like drawing new examples from an unknown probability distribution.</li>
doc/pub/week47/html/week47.html
+37 -3
View File
@@ -488,18 +488,51 @@ predictor, averaged over all \( B \) trees.
a decision tree wth different depths and perform a bootstrap aggregate (in this case we perform as many bootstraps as data points \( n \)).
</p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<pre style="line-height: 125%;"><span style="color: #408080; font-style: italic"># Common imports</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.pipeline</span> <span style="color: #008000; font-weight: bold">import</span> make_pipeline
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.utils</span> <span style="color: #008000; font-weight: bold">import</span> resample
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeRegressor
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeClassifier
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.model_selection</span> <span style="color: #008000; font-weight: bold">import</span> train_test_split
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler, OneHotEncoder
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.compose</span> <span style="color: #008000; font-weight: bold">import</span> ColumnTransformer
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> Image
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">os</span>
<span style="color: #408080; font-style: italic"># Where to save the figures and data files</span>
PROJECT_ROOT_DIR <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Results&quot;</span>
FIGURE_ID <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;Results/FigureFiles&quot;</span>
DATA_ID <span style="color: #666666">=</span> <span style="color: #BA2121">&quot;DataFiles/&quot;</span>
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #AA22FF; font-weight: bold">not</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>exists(PROJECT_ROOT_DIR):
os<span style="color: #666666">.</span>mkdir(PROJECT_ROOT_DIR)
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #AA22FF; font-weight: bold">not</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>exists(FIGURE_ID):
os<span style="color: #666666">.</span>makedirs(FIGURE_ID)
<span style="color: #008000; font-weight: bold">if</span> <span style="color: #AA22FF; font-weight: bold">not</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>exists(DATA_ID):
os<span style="color: #666666">.</span>makedirs(DATA_ID)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">image_path</span>(fig_id):
<span style="color: #008000; font-weight: bold">return</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>join(FIGURE_ID, fig_id)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">data_path</span>(dat_id):
<span style="color: #008000; font-weight: bold">return</span> os<span style="color: #666666">.</span>path<span style="color: #666666">.</span>join(DATA_ID, dat_id)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">save_fig</span>(fig_id):
plt<span style="color: #666666">.</span>savefig(image_path(fig_id) <span style="color: #666666">+</span> <span style="color: #BA2121">&quot;.png&quot;</span>, <span style="color: #008000">format</span><span style="color: #666666">=</span><span style="color: #BA2121">&#39;png&#39;</span>)
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
n_boostraps <span style="color: #666666">=</span> <span style="color: #666666">100</span>
@@ -1732,7 +1765,7 @@ set of hyperparameters and regularization methods.
<h2 id="other-courses-on-data-science-and-machine-learning-at-uio">Other courses on Data science and Machine Learning at UiO </h2>
<ol>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a></li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html" target="_blank">FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences</a>. Discussed deep learning and generative deep learning.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html" target="_blank">FYS5419 Quantum Computing and Quantum Machine Learning</a></li>
<li> <a href="http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html" target="_blank">STK2100 Machine learning and statistical methods for prediction and classification</a>.</li>
<li> <a href="https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html" target="_blank">IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning</a>. Introductory course in machine learning and AI with an algorithmic approach.</li>
@@ -1768,6 +1801,7 @@ networks have been proposed, such as
<li> Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing;</li>
<li> Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing;</li>
<li> Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning.</li>
<li> <b>Generative deep learning</b>! Recent textbook by David Foster (and obviously many other ones) at <a href="https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/" target="_blank"><tt>https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/</tt></a>"</li>
</ol>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="types-of-machine-learning-a-repetition">Types of Machine Learning, a repetition </h2>
@@ -1811,7 +1845,7 @@ One of the major reasons is that they can be stacked layer-wise to build deep ne
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="boltzmann-machines">Boltzmann Machines </h2>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? </p>
<p>Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? <b>Simplest approach to generative deep learning</b>.</p>
<ul>
<li> Discriminitave methods have several limitations: They are mainly supervised learning methods, thus requiring labeled data. And there are tasks they cannot accomplish, like drawing new examples from an unknown probability distribution.</li>
doc/pub/week47/ipynb/ipynb-week47-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week47/ipynb/week47.ipynb
+198 -165
View File
File diff suppressed because it is too large Load Diff
doc/src/week47/week47.do.txt
+38 -4
View File
@@ -71,14 +71,46 @@ predictor, averaged over all $B$ trees.
Let us bring up our good old boostrap example from the linear regression lectures. We change the linerar regression algorithm with
a decision tree wth different depths and perform a bootstrap aggregate (in this case we perform as many bootstraps as data points $n$).
!bc pycod
!bc pycod
# Common imports
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.utils import resample
from sklearn.tree import DecisionTreeRegressor
import pandas as pd
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from IPython.display import Image
import os
# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "DataFiles/"
if not os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
if not os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
if not os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
def image_path(fig_id):
return os.path.join(FIGURE_ID, fig_id)
def data_path(dat_id):
return os.path.join(DATA_ID, dat_id)
def save_fig(fig_id):
plt.savefig(image_path(fig_id) + ".png", format='png')
n = 100
n_boostraps = 100
@@ -1155,7 +1187,7 @@ o Jackknife and many other
===== Other courses on Data science and Machine Learning at UiO =====
o "FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences":"https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html"
o "FYS5429 Advanced Machine Learning and Data Analysis for the Physical Sciences":"https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html". Discussed deep learning and generative deep learning.
o "FYS5419 Quantum Computing and Quantum Machine Learning":"https://www.uio.no/studier/emner/matnat/fys/FYS5419/index-eng.html"
o "STK2100 Machine learning and statistical methods for prediction and classification":"http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html".
o "IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI with an algorithmic approach.
@@ -1187,7 +1219,7 @@ networks have been proposed, such as
o Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing;
o Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing;
o Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning.
o _Generative deep learning_! Recent textbook by David Foster (and obviously many other ones) at URL:"https://www.oreilly.com/library/view/generative-deep-learning/9781492041931/""
!split
===== Types of Machine Learning, a repetition =====
@@ -1227,7 +1259,7 @@ Furthermore, they have been used to solve complicated "quantum mechanical many-p
!split
===== Boltzmann Machines =====
Why use a generative model rather than the more well known discriminative deep neural networks (DNN)?
Why use a generative model rather than the more well known discriminative deep neural networks (DNN)? _Simplest approach to generative deep learning_.
* Discriminitave methods have several limitations: They are mainly supervised learning methods, thus requiring labeled data. And there are tasks they cannot accomplish, like drawing new examples from an unknown probability distribution.
@@ -1734,3 +1766,5 @@ FIGURE: [figures/Nebbdyr2.png, width=500 frac=0.6]