update
This commit is contained in:
Morten Hjorth-Jensen
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
File diff suppressed because it is too large
Load Diff
@@ -323,6 +323,13 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
<a class="reference internal nav-link" href="#basic-matrix-features">
|
||||
2.3. Basic Matrix Features
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#some-famous-matrices">
|
||||
2.3.1. Some famous Matrices
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#numpy-and-arrays">
|
||||
@@ -380,6 +387,13 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
<a class="reference internal nav-link" href="#basic-matrix-features">
|
||||
2.3. Basic Matrix Features
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#some-famous-matrices">
|
||||
2.3.1. Some famous Matrices
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#numpy-and-arrays">
|
||||
@@ -412,7 +426,8 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
|
||||
<div>
|
||||
|
||||
<div class="tex2jax_ignore mathjax_ignore section" id="linear-algebra-handling-of-arrays-and-more-python-features">
|
||||
<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)
|
||||
doconce format html linalg.do.txt --><div class="tex2jax_ignore mathjax_ignore section" id="linear-algebra-handling-of-arrays-and-more-python-features">
|
||||
<h1><span class="section-number">2. </span>Linear Algebra, Handling of Arrays and more Python Features<a class="headerlink" href="#linear-algebra-handling-of-arrays-and-more-python-features" title="Permalink to this headline">¶</a></h1>
|
||||
<div class="section" id="introduction">
|
||||
<h2><span class="section-number">2.1. </span>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h2>
|
||||
@@ -478,7 +493,7 @@ matrices and vectors.</p>
|
||||
\[
|
||||
\mathbf{A}^{-1} \cdot \mathbf{A} = I
|
||||
\]</div>
|
||||
<table border="1">
|
||||
<table class="dotable" border="1">
|
||||
<thead>
|
||||
<tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
|
||||
</thead>
|
||||
@@ -489,8 +504,8 @@ matrices and vectors.</p>
|
||||
<tr><td align="center"> $A = A^{\dagger}$ </td> <td align="center"> hermitian </td> <td align="center"> $a_{ij} = a_{ji}^{ * }$ </td> </tr>
|
||||
<tr><td align="center"> $A = \left (A^{\dagger} \right )^{-1}$ </td> <td align="center"> unitary </td> <td align="center"> $\sum_k a_{ik} a_{jk}^{ * } = \sum_k a_{ki}^{ * } a_{kj} = \delta_{ij}$ </td> </tr>
|
||||
</tbody>
|
||||
</table>
|
||||
### Some famous Matrices
|
||||
</table><div class="section" id="some-famous-matrices">
|
||||
<h3><span class="section-number">2.3.1. </span>Some famous Matrices<a class="headerlink" href="#some-famous-matrices" title="Permalink to this headline">¶</a></h3>
|
||||
<ul class="simple">
|
||||
<li><p>Diagonal if <span class="math notranslate nohighlight">\(a_{ij}=0\)</span> for <span class="math notranslate nohighlight">\(i\ne j\)</span></p></li>
|
||||
<li><p>Upper triangular if <span class="math notranslate nohighlight">\(a_{ij}=0\)</span> for <span class="math notranslate nohighlight">\(i > j\)</span></p></li>
|
||||
@@ -512,6 +527,7 @@ matrices and vectors.</p>
|
||||
<li><p><span class="math notranslate nohighlight">\(0\)</span> is not eigenvalue of <span class="math notranslate nohighlight">\(\mathbf{A}\)</span>.</p></li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="numpy-and-arrays">
|
||||
<h2><span class="section-number">2.4. </span>Numpy and arrays<a class="headerlink" href="#numpy-and-arrays" title="Permalink to this headline">¶</a></h2>
|
||||
<p><a class="reference external" href="http://www.numpy.org/">Numpy</a> provides an easy way to handle arrays in Python. The standard way to import this library is as</p>
|
||||
@@ -525,8 +541,8 @@ matrices and vectors.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.8108619 -0.65136857 0.00577441 0.11499517 0.32179365 -0.70598996
|
||||
0.96992454 1.77865169 -0.40599799 0.04284519]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 0.80738685 -0.37307168 -0.28967287 -0.1023111 -0.42394972 -0.04775904
|
||||
-0.83698677 0.60538875 -0.61463451 -1.12069773]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -607,16 +623,13 @@ The attentive reader will also notice that the output is <span class="math notra
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">4.0</span><span class="p">,</span> <span class="mf">7.0</span><span class="p">,</span> <span class="mf">8.0</span><span class="p">])</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">4.0</span><span class="p">,</span> <span class="mf">7.0</span><span class="p">,</span> <span class="mf">8.0</span><span class="p">]))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt"> File</span><span class="nn"> "<ipython-input-6-f6d7a289d493>"</span><span class="gt">, line </span><span class="mi">3</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="o">^</span>
|
||||
<span class="ne">SyntaxError</span>: invalid syntax
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[1.38629436 1.94591015 2.07944154]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -625,11 +638,16 @@ The attentive reader will also notice that the output is <span class="math notra
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">4.0</span><span class="p">,</span> <span class="mf">7.0</span><span class="p">,</span> <span class="mf">8.0</span><span class="p">])</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">4.0</span><span class="p">,</span> <span class="mf">7.0</span><span class="p">,</span> <span class="mf">8.0</span><span class="p">]))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="o">.</span><span class="n">itemsize</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>8
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>Having defined vectors, we are now ready to try out matrices. We can define a <span class="math notranslate nohighlight">\(3 \times 3 \)</span> real matrix <span class="math notranslate nohighlight">\(\hat{A}\)</span>
|
||||
as (recall that we user lowercase letters for vectors and uppercase letters for matrices)</p>
|
||||
@@ -641,6 +659,13 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[1.38629436 1.94591015 2.07944154]
|
||||
[1.09861229 2.30258509 2.39789527]
|
||||
[1.38629436 1.60943791 1.94591015]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>If we use the <strong>shape</strong> function we would get <span class="math notranslate nohighlight">\((3, 3)\)</span> as output, that is verifying that our matrix is a <span class="math notranslate nohighlight">\(3\times 3\)</span> matrix. We can slice the matrix and print for example the first column (Python organized matrix elements in a row-major order, see below) as</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -652,6 +677,11 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[1.38629436 1.09861229 1.38629436]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>We can continue this was by printing out other columns or rows. The example here prints out the second column</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -663,6 +693,11 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[1.09861229 2.30258509 2.39789527]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>Numpy contains many other functionalities that allow us to slice, subdivide etc etc arrays. We strongly recommend that you look up the <a class="reference external" href="http://www.numpy.org/">Numpy website for more details</a>. Useful functions when defining a matrix are the <strong>np.zeros</strong> function which declares a matrix of a given dimension and sets all elements to zero</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -675,6 +710,20 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
|
||||
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>or initializing all elements to</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -687,6 +736,20 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
|
||||
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>or as unitarily distributed random numbers (see the material on random number generators in the statistics part)</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -699,6 +762,30 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.40214433 0.46508305 0.10620135 0.26186844 0.28067036 0.092066
|
||||
0.30552077 0.51265232 0.57673618 0.76674796]
|
||||
[0.92626212 0.11046771 0.14484695 0.03981057 0.09856879 0.81187794
|
||||
0.43135183 0.43552433 0.03794112 0.96606158]
|
||||
[0.76066069 0.84232163 0.38336316 0.53312754 0.35147135 0.05434571
|
||||
0.21472683 0.96527903 0.18912963 0.56302854]
|
||||
[0.21786964 0.08428156 0.90895045 0.70205195 0.70980493 0.90884627
|
||||
0.09085624 0.35892474 0.68246089 0.51561271]
|
||||
[0.15891336 0.25259666 0.75106135 0.07878641 0.96032148 0.31605061
|
||||
0.60236938 0.04368707 0.34568872 0.24276315]
|
||||
[0.98794823 0.88871662 0.16740002 0.21997099 0.18156717 0.11402309
|
||||
0.85758696 0.51389553 0.45290829 0.89274639]
|
||||
[0.21519063 0.20174121 0.32320052 0.34585355 0.93579127 0.07735703
|
||||
0.63860687 0.10302062 0.28047021 0.48212873]
|
||||
[0.50727059 0.89873772 0.28875373 0.65628853 0.87940752 0.68279358
|
||||
0.51004249 0.46567887 0.37415316 0.91383439]
|
||||
[0.90940378 0.25792767 0.17930649 0.24512498 0.10505137 0.0714956
|
||||
0.69493539 0.35203688 0.1583767 0.80747253]
|
||||
[0.98266587 0.39456996 0.48574149 0.461175 0.30384239 0.53423784
|
||||
0.79009329 0.38868469 0.8991514 0.39560937]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>As we will see throughout these lectures, there are several extremely useful functionalities in Numpy.
|
||||
As an example, consider the discussion of the covariance matrix. Suppose we have defined three vectors
|
||||
@@ -754,6 +841,17 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.048653428302840175
|
||||
4.098802859381565
|
||||
-0.13229545716505003
|
||||
[[0.89550839 2.74368436 2.15717291]
|
||||
[2.74368436 9.71131626 6.39311435]
|
||||
[2.15717291 6.39311435 7.55505907]]
|
||||
[15.92603747 0.07903849 2.15680777]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
@@ -773,6 +871,19 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[1. 0. 0. 0.]
|
||||
[0. 1. 0. 0.]
|
||||
[0. 0. 1. 0.]
|
||||
[0. 0. 0. 1.]]
|
||||
(0, 0) 1.0
|
||||
(1, 1) 1.0
|
||||
(2, 2) 1.0
|
||||
(3, 3) 1.0
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/linalg_44_1.png" src="_images/linalg_44_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="other-matrix-and-vector-operations">
|
||||
@@ -798,8 +909,29 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"This is the mean value for each row:</span><span class="si">{</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">keepdims</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span><span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="c1"># We print then the mean value for each row by setting keepdims=False</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"This is the mean value for each row with keepdims false:</span><span class="si">{</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">keepdims</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span><span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># Ravel return a contiguous flattened array.</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The test matrix:[[ 1. 2. 3.]
|
||||
[ 4. 5. 6.]
|
||||
[ 7. 8. 9.]
|
||||
[10. 11. 12.]]
|
||||
This is the total mean summed over all elements:6.5
|
||||
This is the mean for each column:[[5.5 6.5 7.5]]
|
||||
This is the mean value for each row:[[ 2.]
|
||||
[ 5.]
|
||||
[ 8.]
|
||||
[11.]]
|
||||
This is the mean value for each row with keepdims false:[ 2. 5. 8. 11.]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>Another useful function is the <strong>ravel</strong> function, which returns a flattened array as shown in the example here.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># Ravel return a contiguous flattened array.</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Flatten the matrix:</span><span class="si">{</span><span class="n">np</span><span class="o">.</span><span class="n">ravel</span><span class="p">(</span><span class="n">a</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<span class="c1"># It is the same as reshaping the matrix into a one-dimensional array</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Reshape the matrix to a one-dim array:</span><span class="si">{</span><span class="n">a</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
@@ -815,6 +947,15 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Flatten the matrix:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]
|
||||
Reshape the matrix to a one-dim array:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]
|
||||
[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]
|
||||
[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]
|
||||
[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="gaussian-elimination">
|
||||
@@ -1225,6 +1366,12 @@ can be written as a vector with unknown entries</p>
|
||||
\end{bmatrix},
|
||||
\end{split}\]</div>
|
||||
<p>and continue till we have solved all <span class="math notranslate nohighlight">\(n\)</span> sets of linear equations.</p>
|
||||
<p>The calculation of the inverse here assumes that it actually
|
||||
exists. In many machine learning applications there may be strong
|
||||
linear dependencies among the various columns and/or rows. In our
|
||||
discussions of linear regression we will dive into the mathematics of
|
||||
the singular value decomposition, an algorithm which will allow us to calculate the so-called pseudo-inverse.
|
||||
These details will be presented in our linear regression chapter.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1,8 +1,11 @@
|
||||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# <!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)
|
||||
# doconce format html linalg.do.txt -->
|
||||
|
||||
# # Linear Algebra, Handling of Arrays and more Python Features
|
||||
#
|
||||
|
||||
# ## Introduction
|
||||
#
|
||||
# The aim of this set of lectures is to review some central linear algebra algorithms that we will need in our
|
||||
@@ -15,9 +18,8 @@
|
||||
# as writing your own codes for some central ML algorithms. The
|
||||
# latter can be written in a language of your choice, be it Python, Julia, R,
|
||||
# Rust, C++, Fortran etc. In order to avoid confusion however, in these lectures we will limit our
|
||||
# attention to Python, C++ and Fortran.
|
||||
#
|
||||
#
|
||||
# attention to Python, C++ and Fortran.
|
||||
|
||||
# ## Important Matrix and vector handling packages
|
||||
#
|
||||
# There are several central software packages for linear algebra and eigenvalue problems. Several of the more
|
||||
@@ -47,8 +49,7 @@
|
||||
# convenient way to handle and organize arrays. We discuss this library
|
||||
# as well. Before we proceed we believe it may be convenient to repeat some basic features of
|
||||
# matrices and vectors.
|
||||
#
|
||||
#
|
||||
|
||||
# ## Basic Matrix Features
|
||||
#
|
||||
# Matrix properties reminder
|
||||
@@ -74,7 +75,7 @@
|
||||
# \mathbf{A}^{-1} \cdot \mathbf{A} = I
|
||||
# $$
|
||||
|
||||
# <table border="1">
|
||||
# <table class="dotable" border="1">
|
||||
# <thead>
|
||||
# <tr><th align="center"> Relations </th> <th align="center"> Name </th> <th align="center"> matrix elements </th> </tr>
|
||||
# </thead>
|
||||
@@ -86,6 +87,7 @@
|
||||
# <tr><td align="center"> $A = \left (A^{\dagger} \right )^{-1}$ </td> <td align="center"> unitary </td> <td align="center"> $\sum_k a_{ik} a_{jk}^{ * } = \sum_k a_{ki}^{ * } a_{kj} = \delta_{ij}$ </td> </tr>
|
||||
# </tbody>
|
||||
# </table>
|
||||
|
||||
# ### Some famous Matrices
|
||||
#
|
||||
# * Diagonal if $a_{ij}=0$ for $i\ne j$
|
||||
@@ -119,7 +121,7 @@
|
||||
# * $\mathbf{A}$ is a product of elementary matrices.
|
||||
#
|
||||
# * $0$ is not eigenvalue of $\mathbf{A}$.
|
||||
#
|
||||
|
||||
# ## Numpy and arrays
|
||||
# [Numpy](http://www.numpy.org/) provides an easy way to handle arrays in Python. The standard way to import this library is as
|
||||
|
||||
@@ -190,24 +192,24 @@ print(x)
|
||||
|
||||
|
||||
import numpy as np
|
||||
x = np.log(np.array([4.0, 7.0, 8.0])
|
||||
x = np.log(np.array([4.0, 7.0, 8.0]))
|
||||
print(x)
|
||||
|
||||
|
||||
# To check the number of bytes (remember that one byte contains eight bits for double precision variables), you can use simple use the **itemsize** functionality (the array $x$ is actually an object which inherits the functionalities defined in Numpy) as
|
||||
|
||||
# In[ ]:
|
||||
# In[7]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
x = np.log(np.array([4.0, 7.0, 8.0])
|
||||
x = np.log(np.array([4.0, 7.0, 8.0]))
|
||||
print(x.itemsize)
|
||||
|
||||
|
||||
# Having defined vectors, we are now ready to try out matrices. We can define a $3 \times 3 $ real matrix $\hat{A}$
|
||||
# as (recall that we user lowercase letters for vectors and uppercase letters for matrices)
|
||||
|
||||
# In[ ]:
|
||||
# In[8]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
@@ -217,7 +219,7 @@ print(A)
|
||||
|
||||
# If we use the **shape** function we would get $(3, 3)$ as output, that is verifying that our matrix is a $3\times 3$ matrix. We can slice the matrix and print for example the first column (Python organized matrix elements in a row-major order, see below) as
|
||||
|
||||
# In[ ]:
|
||||
# In[9]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
@@ -228,7 +230,7 @@ print(A[:,0])
|
||||
|
||||
# We can continue this was by printing out other columns or rows. The example here prints out the second column
|
||||
|
||||
# In[ ]:
|
||||
# In[10]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
@@ -239,7 +241,7 @@ print(A[1,:])
|
||||
|
||||
# Numpy contains many other functionalities that allow us to slice, subdivide etc etc arrays. We strongly recommend that you look up the [Numpy website for more details](http://www.numpy.org/). Useful functions when defining a matrix are the **np.zeros** function which declares a matrix of a given dimension and sets all elements to zero
|
||||
|
||||
# In[ ]:
|
||||
# In[11]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
@@ -251,7 +253,7 @@ print(A)
|
||||
|
||||
# or initializing all elements to
|
||||
|
||||
# In[ ]:
|
||||
# In[12]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
@@ -263,7 +265,7 @@ print(A)
|
||||
|
||||
# or as unitarily distributed random numbers (see the material on random number generators in the statistics part)
|
||||
|
||||
# In[ ]:
|
||||
# In[13]:
|
||||
|
||||
|
||||
import numpy as np
|
||||
@@ -311,7 +313,7 @@ print(A)
|
||||
# function **np.mean(x)**. We can also extract the eigenvalues of the
|
||||
# covariance matrix through the **np.linalg.eig()** function.
|
||||
|
||||
# In[ ]:
|
||||
# In[14]:
|
||||
|
||||
|
||||
# Importing various packages
|
||||
@@ -331,7 +333,7 @@ Eigvals, Eigvecs = np.linalg.eig(Sigma)
|
||||
print(Eigvals)
|
||||
|
||||
|
||||
# In[ ]:
|
||||
# In[15]:
|
||||
|
||||
|
||||
get_ipython().run_line_magic('matplotlib', 'inline')
|
||||
@@ -353,7 +355,7 @@ plt.show()
|
||||
#
|
||||
# The following examples show how to compute various quantities like the **mean** value of a matrix or a vector and how to use functions like **reshape** and **ravel**. These are all useful quantities when scaling the data and preparing the data for various machine learning algorithms and when calculating quantities like the mean squared error or the variance.
|
||||
|
||||
# In[ ]:
|
||||
# In[16]:
|
||||
|
||||
|
||||
"""
|
||||
@@ -375,6 +377,12 @@ print(f"This is the mean value for each row:{np.mean(a, axis=1, keepdims=True,dt
|
||||
# We print then the mean value for each row by setting keepdims=False
|
||||
print(f"This is the mean value for each row with keepdims false:{np.mean(a, axis=1, keepdims=False,dtype=np.float64)}")
|
||||
|
||||
|
||||
# Another useful function is the **ravel** function, which returns a flattened array as shown in the example here.
|
||||
|
||||
# In[17]:
|
||||
|
||||
|
||||
# Ravel return a contiguous flattened array.
|
||||
print(f"Flatten the matrix:{np.ravel(a)}")
|
||||
# It is the same as reshaping the matrix into a one-dimensional array
|
||||
@@ -467,7 +475,6 @@ print(np.ravel(a.T, order='A'))
|
||||
# We can solve this system of equations recursively starting from $x_n$ (in our case $x_4$) and proceed with
|
||||
# what is called a backward substitution.
|
||||
#
|
||||
#
|
||||
# This process can be expressed mathematically as
|
||||
|
||||
# <!-- Equation labels as ordinary links -->
|
||||
@@ -485,7 +492,6 @@ print(np.ravel(a.T, order='A'))
|
||||
# the result from the $j$th equation. We assume obviously that $a_{11}\ne 0$ and that
|
||||
# $\mathbf{A}$ is not singular.
|
||||
#
|
||||
#
|
||||
# Our actual $4\times 4$ example reads after the first operation
|
||||
|
||||
# $$
|
||||
@@ -574,8 +580,6 @@ print(np.ravel(a.T, order='A'))
|
||||
# We have also set $w_1^{(1)}=w_1$, the original vector element.
|
||||
# We see that the system of unknowns $x_1,\dots,x_n$ is transformed into an $(n-1)\times (n-1)$ problem.
|
||||
#
|
||||
#
|
||||
#
|
||||
# This step is called forward substitution.
|
||||
# Proceeding with these substitutions, we obtain the
|
||||
# general expressions for the new coefficients
|
||||
@@ -613,9 +617,6 @@ print(np.ravel(a.T, order='A'))
|
||||
# one. We are then
|
||||
# adding $10^7+1$. With single precision this results in $10^7$.
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
# * Gaussian elimination, $O(2/3n^3)$ flops, general matrix
|
||||
#
|
||||
# * LU decomposition, upper triangular and lower tridiagonal matrices, $O(2/3n^3)$ flops, general matrix. Get easily the inverse, determinant and can solve linear equations with back-substitution only, $O(n^2)$ flops
|
||||
@@ -752,7 +753,6 @@ print(np.ravel(a.T, order='A'))
|
||||
# and as soon as we have $\mathbf{y}$ we can obtain $\mathbf{x}$
|
||||
# through $\mathbf{Ux}=\mathbf{y}$.
|
||||
#
|
||||
#
|
||||
# For our four-dimentional example this takes the form
|
||||
|
||||
# $$
|
||||
@@ -792,8 +792,6 @@ print(np.ravel(a.T, order='A'))
|
||||
# This example shows the basis for the algorithm
|
||||
# needed to solve the set of $n$ linear equations.
|
||||
#
|
||||
#
|
||||
#
|
||||
# The algorithm goes as follows
|
||||
#
|
||||
# * Set up the matrix $\bf A$ and the vector $\bf w$ with their correct dimensions. This determines the dimensionality of the unknown vector $\bf x$.
|
||||
@@ -801,7 +799,7 @@ print(np.ravel(a.T, order='A'))
|
||||
# * Then LU decompose the matrix $\bf A$ through a call to the function `ludcmp(double a, int n, int indx, double &d)`. This functions returns the LU decomposed matrix $\bf A$, its determinant and the vector indx which keeps track of the number of interchanges of rows. If the determinant is zero, the solution is malconditioned.
|
||||
#
|
||||
# * Thereafter you call the function `lubksb(double a, int n, int indx, double w)` which uses the LU decomposed matrix $\bf A$ and the vector $\bf w$ and returns $\bf x$ in the same place as $\bf w$. Upon exit the original content in $\bf w$ is destroyed. If you wish to keep this information, you should make a backup of it in your calling function.
|
||||
#
|
||||
|
||||
# ### LU Decomposition, the inverse of a matrix
|
||||
#
|
||||
# If the inverse exists then
|
||||
@@ -861,3 +859,10 @@ print(np.ravel(a.T, order='A'))
|
||||
# $$
|
||||
|
||||
# and continue till we have solved all $n$ sets of linear equations.
|
||||
#
|
||||
# The calculation of the inverse here assumes that it actually
|
||||
# exists. In many machine learning applications there may be strong
|
||||
# linear dependencies among the various columns and/or rows. In our
|
||||
# discussions of linear regression we will dive into the mathematics of
|
||||
# the singular value decomposition, an algorithm which will allow us to calculate the so-called pseudo-inverse.
|
||||
# These details will be presented in our linear regression chapter.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
Reference in New Issue
Block a user