Morten Hjorth-Jensen
2135 lines
86 KiB
Plaintext
2135 lines
86 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "43efcac0",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||
"doconce format html linalg.do.txt -->"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "074ac7c2",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"# Linear Algebra, Handling of Arrays and more Python Features"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0be7d59e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Introduction\n",
|
||
"\n",
|
||
"The aim of this set of lectures is to review some central linear algebra algorithms that we will need in our \n",
|
||
"data analysis part and in the construction of Machine Learning algorithms (ML). \n",
|
||
"This will allow us to introduce some central programming features of high-level languages like Python and \n",
|
||
"compiled languages like C++ and/or Fortran. \n",
|
||
"\n",
|
||
"As discussed in the introductory notes, these series of lectures focuses both on using\n",
|
||
"central Python packages like **tensorflow** and **scikit-learn** as well\n",
|
||
"as writing your own codes for some central ML algorithms. The\n",
|
||
"latter can be written in a language of your choice, be it Python, Julia, R,\n",
|
||
"Rust, C++, Fortran etc. In order to avoid confusion however, in these lectures we will limit our\n",
|
||
"attention to Python, C++ and Fortran."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5e77e6c1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Important Matrix and vector handling packages\n",
|
||
"\n",
|
||
"There are several central software packages for linear algebra and eigenvalue problems. Several of the more\n",
|
||
"popular ones have been wrapped into ofter software packages like those from the widely used text **Numerical Recipes**. The original source codes in many of the available packages are often taken from the widely used\n",
|
||
"software package LAPACK, which follows two other popular packages\n",
|
||
"developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly here.\n",
|
||
"\n",
|
||
" * LINPACK: package for linear equations and least square problems.\n",
|
||
"\n",
|
||
" * LAPACK:package for solving symmetric, unsymmetric and generalized eigenvalue problems. From LAPACK's website <http://www.netlib.org> it is possible to download for free all source codes from this library. Both C/C++ and Fortran versions are available.\n",
|
||
"\n",
|
||
" * BLAS (I, II and III): (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. Blas I is vector operations, II vector-matrix operations and III matrix-matrix operations. Highly parallelized and efficient codes, all available for download from <http://www.netlib.org>.\n",
|
||
"\n",
|
||
"When dealing with matrices and vectors a central issue is memory\n",
|
||
"handling and allocation. If our code is written in Python the way we\n",
|
||
"declare these objects and the way they are handled, interpreted and\n",
|
||
"used by say a linear algebra library, requires codes that interface\n",
|
||
"our Python program with such libraries. For Python programmers,\n",
|
||
"**Numpy** is by now the standard Python package for numerical arrays in\n",
|
||
"Python as well as the source of functions which act on these\n",
|
||
"arrays. These functions span from eigenvalue solvers to functions that\n",
|
||
"compute the mean value, variance or the covariance matrix. If you are\n",
|
||
"not familiar with how arrays are handled in say Python or compiled\n",
|
||
"languages like C++ and Fortran, the sections in this chapter may be\n",
|
||
"useful. For C++ programmer, **Armadillo** is widely used library for\n",
|
||
"linear algebra and eigenvalue problems. In addition it offers a\n",
|
||
"convenient way to handle and organize arrays. We discuss this library\n",
|
||
"as well. Before we proceed we believe it may be convenient to repeat some basic features of \n",
|
||
" matrices and vectors."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2eed76da",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Basic Matrix Features\n",
|
||
"\n",
|
||
"Matrix properties reminder"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "109ff37c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A} =\n",
|
||
" \\begin{bmatrix} a_{11} & a_{12} & a_{13} & a_{14} \\\\\n",
|
||
" a_{21} & a_{22} & a_{23} & a_{24} \\\\\n",
|
||
" a_{31} & a_{32} & a_{33} & a_{34} \\\\\n",
|
||
" a_{41} & a_{42} & a_{43} & a_{44}\n",
|
||
" \\end{bmatrix}\\qquad\n",
|
||
"\\mathbf{I} =\n",
|
||
" \\begin{bmatrix} 1 & 0 & 0 & 0 \\\\\n",
|
||
" 0 & 1 & 0 & 0 \\\\\n",
|
||
" 0 & 0 & 1 & 0 \\\\\n",
|
||
" 0 & 0 & 0 & 1\n",
|
||
" \\end{bmatrix}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c87c75d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The inverse of a matrix is defined by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "aab763d1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}^{-1} \\cdot \\mathbf{A} = I\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f382e0f8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<table class=\"dotable\" border=\"1\">\n",
|
||
"<thead>\n",
|
||
"<tr><th align=\"center\"> Relations </th> <th align=\"center\"> Name </th> <th align=\"center\"> matrix elements </th> </tr>\n",
|
||
"</thead>\n",
|
||
"<tbody>\n",
|
||
"<tr><td align=\"center\"> $A = A^{T}$ </td> <td align=\"center\"> symmetric </td> <td align=\"center\"> $a_{ij} = a_{ji}$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = \\left (A^{T} \\right )^{-1}$ </td> <td align=\"center\"> real orthogonal </td> <td align=\"center\"> $\\sum_k a_{ik} a_{jk} = \\sum_k a_{ki} a_{kj} = \\delta_{ij}$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = A^{ * }$ </td> <td align=\"center\"> real matrix </td> <td align=\"center\"> $a_{ij} = a_{ij}^{ * }$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = A^{\\dagger}$ </td> <td align=\"center\"> hermitian </td> <td align=\"center\"> $a_{ij} = a_{ji}^{ * }$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = \\left (A^{\\dagger} \\right )^{-1}$ </td> <td align=\"center\"> unitary </td> <td align=\"center\"> $\\sum_k a_{ik} a_{jk}^{ * } = \\sum_k a_{ki}^{ * } a_{kj} = \\delta_{ij}$ </td> </tr>\n",
|
||
"</tbody>\n",
|
||
"</table>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5d394b4e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"### Some famous Matrices\n",
|
||
"\n",
|
||
" * Diagonal if $a_{ij}=0$ for $i\\ne j$\n",
|
||
"\n",
|
||
" * Upper triangular if $a_{ij}=0$ for $i > j$\n",
|
||
"\n",
|
||
" * Lower triangular if $a_{ij}=0$ for $i < j$\n",
|
||
"\n",
|
||
" * Upper Hessenberg if $a_{ij}=0$ for $i > j+1$\n",
|
||
"\n",
|
||
" * Lower Hessenberg if $a_{ij}=0$ for $i < j+1$\n",
|
||
"\n",
|
||
" * Tridiagonal if $a_{ij}=0$ for $|i -j| > 1$\n",
|
||
"\n",
|
||
" * Lower banded with bandwidth $p$: $a_{ij}=0$ for $i > j+p$\n",
|
||
"\n",
|
||
" * Upper banded with bandwidth $p$: $a_{ij}=0$ for $i < j+p$\n",
|
||
"\n",
|
||
" * Banded, block upper triangular, block lower triangular....\n",
|
||
"\n",
|
||
"Some Equivalent Statements. For an $N\\times N$ matrix $\\mathbf{A}$ the following properties are all equivalent\n",
|
||
"\n",
|
||
" * If the inverse of $\\mathbf{A}$ exists, $\\mathbf{A}$ is nonsingular.\n",
|
||
"\n",
|
||
" * The equation $\\mathbf{Ax}=0$ implies $\\mathbf{x}=0$.\n",
|
||
"\n",
|
||
" * The rows of $\\mathbf{A}$ form a basis of $R^N$.\n",
|
||
"\n",
|
||
" * The columns of $\\mathbf{A}$ form a basis of $R^N$.\n",
|
||
"\n",
|
||
" * $\\mathbf{A}$ is a product of elementary matrices.\n",
|
||
"\n",
|
||
" * $0$ is not eigenvalue of $\\mathbf{A}$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0a85f2b2",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Numpy and arrays\n",
|
||
"[Numpy](http://www.numpy.org/) provides an easy way to handle arrays in Python. The standard way to import this library is as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "7043c92e",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[ 0.80738685 -0.37307168 -0.28967287 -0.1023111 -0.42394972 -0.04775904\n",
|
||
" -0.83698677 0.60538875 -0.61463451 -1.12069773]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"x = np.random.normal(size=n)\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2b5abf29",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have defined a vector $x$ with $n=10$ elements with its values given by the Normal distribution $N(0,1)$.\n",
|
||
"Another alternative is to declare a vector as follows"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "4193cb75",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1 2 3]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.array([1, 2, 3])\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b112c576",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have defined a vector with three elements, with $x_0=1$, $x_1=2$ and $x_2=3$. Note that both Python and C++\n",
|
||
"start numbering array elements from $0$ and on. This means that a vector with $n$ elements has a sequence of entities $x_0, x_1, x_2, \\dots, x_{n-1}$. We could also let (recommended) Numpy to compute the logarithms of a specific array as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "18167a1f",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4, 7, 8]))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ee482a57",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have used Numpy's unary function $np.log$. This function is\n",
|
||
"highly tuned to compute array elements since the code is vectorized\n",
|
||
"and does not require looping. We normaly recommend that you use the\n",
|
||
"Numpy intrinsic functions instead of the corresponding **log** function\n",
|
||
"from Python's **math** module. The looping is done explicitely by the\n",
|
||
"**np.log** function. The alternative, and slower way to compute the\n",
|
||
"logarithms of a vector would be to write"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "d8305632",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1 1 2]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"from math import log\n",
|
||
"x = np.array([4, 7, 8])\n",
|
||
"for i in range(0, len(x)):\n",
|
||
" x[i] = log(x[i])\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a50fea8e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We note that our code is much longer already and we need to import the **log** function from the **math** module. \n",
|
||
"The attentive reader will also notice that the output is $[1, 1, 2]$. Python interprets automacally our numbers as integers (like the **automatic** keyword in C++). To change this we could define our array elements to be double precision numbers as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "e9263103",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4, 7, 8], dtype = np.float64))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d68631ad",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or simply write them as double precision numbers (Python uses 64 bits as default for floating point type variables), that is"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "074cfbda",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dab5cce4",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To check the number of bytes (remember that one byte contains eight bits for double precision variables), you can use simple use the **itemsize** functionality (the array $x$ is actually an object which inherits the functionalities defined in Numpy) as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "9d335488",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"8\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
|
||
"print(x.itemsize)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "64c9f3e5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Having defined vectors, we are now ready to try out matrices. We can define a $3 \\times 3 $ real matrix $\\hat{A}$\n",
|
||
"as (recall that we user lowercase letters for vectors and uppercase letters for matrices)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "a3b32334",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1.38629436 1.94591015 2.07944154]\n",
|
||
" [1.09861229 2.30258509 2.39789527]\n",
|
||
" [1.38629436 1.60943791 1.94591015]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4ff9ce6b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"If we use the **shape** function we would get $(3, 3)$ as output, that is verifying that our matrix is a $3\\times 3$ matrix. We can slice the matrix and print for example the first column (Python organized matrix elements in a row-major order, see below) as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "d19f85e2",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.09861229 1.38629436]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"# print the first column, row-major order and elements start with 0\n",
|
||
"print(A[:,0])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4e7e8796",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We can continue this was by printing out other columns or rows. The example here prints out the second column"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "4733c8b7",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.09861229 2.30258509 2.39789527]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"# print the first column, row-major order and elements start with 0\n",
|
||
"print(A[1,:])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "df9f205b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Numpy contains many other functionalities that allow us to slice, subdivide etc etc arrays. We strongly recommend that you look up the [Numpy website for more details](http://www.numpy.org/). Useful functions when defining a matrix are the **np.zeros** function which declares a matrix of a given dimension and sets all elements to zero"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "19757d00",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to zero\n",
|
||
"A = np.zeros( (n, n) )\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f1911274",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or initializing all elements to"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "4f737773",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to one\n",
|
||
"A = np.ones( (n, n) )\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cd241572",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or as unitarily distributed random numbers (see the material on random number generators in the statistics part)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "54645585",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[0.40214433 0.46508305 0.10620135 0.26186844 0.28067036 0.092066\n",
|
||
" 0.30552077 0.51265232 0.57673618 0.76674796]\n",
|
||
" [0.92626212 0.11046771 0.14484695 0.03981057 0.09856879 0.81187794\n",
|
||
" 0.43135183 0.43552433 0.03794112 0.96606158]\n",
|
||
" [0.76066069 0.84232163 0.38336316 0.53312754 0.35147135 0.05434571\n",
|
||
" 0.21472683 0.96527903 0.18912963 0.56302854]\n",
|
||
" [0.21786964 0.08428156 0.90895045 0.70205195 0.70980493 0.90884627\n",
|
||
" 0.09085624 0.35892474 0.68246089 0.51561271]\n",
|
||
" [0.15891336 0.25259666 0.75106135 0.07878641 0.96032148 0.31605061\n",
|
||
" 0.60236938 0.04368707 0.34568872 0.24276315]\n",
|
||
" [0.98794823 0.88871662 0.16740002 0.21997099 0.18156717 0.11402309\n",
|
||
" 0.85758696 0.51389553 0.45290829 0.89274639]\n",
|
||
" [0.21519063 0.20174121 0.32320052 0.34585355 0.93579127 0.07735703\n",
|
||
" 0.63860687 0.10302062 0.28047021 0.48212873]\n",
|
||
" [0.50727059 0.89873772 0.28875373 0.65628853 0.87940752 0.68279358\n",
|
||
" 0.51004249 0.46567887 0.37415316 0.91383439]\n",
|
||
" [0.90940378 0.25792767 0.17930649 0.24512498 0.10505137 0.0714956\n",
|
||
" 0.69493539 0.35203688 0.1583767 0.80747253]\n",
|
||
" [0.98266587 0.39456996 0.48574149 0.461175 0.30384239 0.53423784\n",
|
||
" 0.79009329 0.38868469 0.8991514 0.39560937]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to random numbers with x \\in [0, 1]\n",
|
||
"A = np.random.rand(n, n)\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e993bf6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"As we will see throughout these lectures, there are several extremely useful functionalities in Numpy.\n",
|
||
"As an example, consider the discussion of the covariance matrix. Suppose we have defined three vectors\n",
|
||
"$\\hat{x}, \\hat{y}, \\hat{z}$ with $n$ elements each. The covariance matrix is defined as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4fe66190",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\hat{\\Sigma} = \\begin{bmatrix} \\sigma_{xx} & \\sigma_{xy} & \\sigma_{xz} \\\\\n",
|
||
" \\sigma_{yx} & \\sigma_{yy} & \\sigma_{yz} \\\\\n",
|
||
" \\sigma_{zx} & \\sigma_{zy} & \\sigma_{zz} \n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cdd238d5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where for example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "41663b63",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\sigma_{xy} =\\frac{1}{n} \\sum_{i=0}^{n-1}(x_i- \\overline{x})(y_i- \\overline{y}).\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e7bd73e9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The Numpy function **np.cov** calculates the covariance elements using the factor $1/(n-1)$ instead of $1/n$ since it assumes we do not have the exact mean values. For a more in-depth discussion of the covariance and covariance matrix and its meaning, we refer you to the lectures on statistics. \n",
|
||
"The following simple function uses the **np.vstack** function which takes each vector of dimension $1\\times n$ and produces a $ 3\\times n$ matrix $\\hat{W}$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c4d0878",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\hat{W} = \\begin{bmatrix} x_0 & y_0 & z_0 \\\\\n",
|
||
" x_1 & y_1 & z_1 \\\\\n",
|
||
" x_2 & y_2 & z_2 \\\\\n",
|
||
" \\dots & \\dots & \\dots \\\\\n",
|
||
" x_{n-2} & y_{n-2} & z_{n-2} \\\\\n",
|
||
" x_{n-1} & y_{n-1} & z_{n-1}\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "32eb00e7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"which in turn is converted into into the $3 times 3$ covariance matrix\n",
|
||
"$\\hat{\\Sigma}$ via the Numpy function **np.cov()**. In our review of\n",
|
||
"statistical functions and quantities we will discuss more about the\n",
|
||
"meaning of the covariance matrix. Here we note that we can calculate\n",
|
||
"the mean value of each set of samples $\\hat{x}$ etc using the Numpy\n",
|
||
"function **np.mean(x)**. We can also extract the eigenvalues of the\n",
|
||
"covariance matrix through the **np.linalg.eig()** function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "6b91d50d",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"0.048653428302840175\n",
|
||
"4.098802859381565\n",
|
||
"-0.13229545716505003\n",
|
||
"[[0.89550839 2.74368436 2.15717291]\n",
|
||
" [2.74368436 9.71131626 6.39311435]\n",
|
||
" [2.15717291 6.39311435 7.55505907]]\n",
|
||
"[15.92603747 0.07903849 2.15680777]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Importing various packages\n",
|
||
"import numpy as np\n",
|
||
"\n",
|
||
"n = 100\n",
|
||
"x = np.random.normal(size=n)\n",
|
||
"print(np.mean(x))\n",
|
||
"y = 4+3*x+np.random.normal(size=n)\n",
|
||
"print(np.mean(y))\n",
|
||
"z = x**3+np.random.normal(size=n)\n",
|
||
"print(np.mean(z))\n",
|
||
"W = np.vstack((x, y, z))\n",
|
||
"Sigma = np.cov(W)\n",
|
||
"print(Sigma)\n",
|
||
"Eigvals, Eigvecs = np.linalg.eig(Sigma)\n",
|
||
"print(Eigvals)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "6a0aa964",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1. 0. 0. 0.]\n",
|
||
" [0. 1. 0. 0.]\n",
|
||
" [0. 0. 1. 0.]\n",
|
||
" [0. 0. 0. 1.]]\n",
|
||
" (0, 0)\t1.0\n",
|
||
" (1, 1)\t1.0\n",
|
||
" (2, 2)\t1.0\n",
|
||
" (3, 3)\t1.0\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABTtUlEQVR4nO29d3hcV53w//nOqPcyapbcNJK75RLjOEpxeqMkYRdIIWTfzW7wksBS9reEZZeXfVlYWJbAyyZgQnkJYAgtkAAJkASiFMXGXe6WRrZsWbKkUe9tzu+PO3c8ViRb8rQ7M+fzPHo0c+femaOjO+fbv0eUUmg0Go0mfrFFegAajUajiSxaEGg0Gk2cowWBRqPRxDlaEGg0Gk2cowWBRqPRxDkJkR7ApeBwONSiRYsiPQyNRqOJKnbv3u1WShVMPR6VgmDRokXs2rUr0sPQaDSaqEJEmqY7rl1DGo1GE+doQaDRaDRxjhYEGo1GE+doQaDRaDRxjhYEGo1GE+cERRCIyPdEpF1EDs7wuojI10WkQUTqRGS932u3isgx72uPBmM8muCytcZFrct93rFal5utNa4IjSh+0HMfWeJl/oNlEXwfuPUCr98GVHp/HgK+CSAiduAJ7+srgHtEZEWQxqQJElVl2Tzy472+L0Sty80jP95LVVl2hEcW+/jP/dYaF99+zXXe3MfiomQl4mX+g1JHoJR6VUQWXeCUO4AfKKPn9XYRyRGREmAR0KCUagQQkae95x4Oxrg0waHa6eDxe9ex5Ye7KcxM5mzfCE9+YAPVTgdgfBnqmnvZstkZ4ZHGHubcP7xtD0uKMvnLiS7+5e3LqHY6fAL58XvXRXqYMYs5///woz1sWJjLn462x+T8h6ugrBQ47fe82XtsuuOXT/cGIvIQhjXBggULQjNKjY+tNS6qyrJ9i3173yj9IxP0jUwA8KM3m1iQl8aprqGY+TJYhalzX1WWgwjsONEFwH8+f5Q/HGzD1THAE/et952nCQ29Q+P0Do/z8tF2wJj/fad6ebOxk8fvXRcT8x+uYLFMc0xd4PhbDyr1pFJqg1JqQ0HBWyqkNUHGZxI3uHnizw189Kf7UMCVznwSbMLzB89yw1dq+NC2PTHzZbAK/u6IobEJbvvaq3QNjrN2fjZpSXby05PY1dRNuSNDz3sI8I8L7G7q5iM/2YtNoLIwg5REG+nJdn53oJV3rSmJmfkPlyBoBub7PS8DWi5wXBNhTJP4b5/ayZf/cAyAR29byra/38QPHtxIkl0YnfBQlpsaM18Gq+DvDrrmS3/idPcwd62bx68fvoqP3VSJe2CMhXmp7D7VzdN/ORXp4cYcpiB+Zk8zD3xvBxNKkZpk59/vWMknbl7CwMgkAvzgzSbeqD8XSI7meEG4BMFzwAe82UObgF6lVCuwE6gUkcUikgTc7T1XYwEqCjMYm/AAcNfaeWzZXOF7LTnRTkl2CgfP9PHjHdO2L9EEQLXTQWVhBu7BcTYsyuWr71tHrcvNN19p5F/evow715WRnGDjX351gNdjZDGyCtVOB4+9Zw3/9PP9jIx7SEu08+0PbADwzf/bFuXiUfC3T+2k1uWO+gSKYKWP/gR4E1gqIs0i8qCIbBGRLd5TngcagQbg28CHAJRSE8AjwB+AI8DPlFKHgjEmTeB8/rdH8Ci4f9NCaurd593w37r/Mt73tvmkJtr4zLOHqG04l1GkF6LAeeVYOztPdrPYkU5jx6AvIP/4vev4+6udfOymJfzdVYvxKPjqS8cBnc0VTM72jeBRMOFRPHjVYqqdjvPm/6cfvIJ183MYnfDw5d8f88XJotU6DlbW0D0XeV0BD8/w2vMYgkJjIV4+0sZz+1u4fHEun7tzFbetLuaRH+/llpVF593wT77ayITHw093ngZBB46DQK3LzcPb9qCAL9y1GoWadqH5p1uW8mp9B7ubuvk/vznEr/e1RPViZBWUUnzzFRd2ET50nZMf7TjFJmf+eVlxIsK3PnAZV3/pz+w93cNHrq+I6nnXlcWaafnR9iYU8Om3G2Udpt96YX6674avdjp48v4N2G3Cy0fbeFgHjoPC/tM95KYnsrwki03leb65r2vuPe88EeHr9xi1md974yTvv3yBnvsg8NSbTTR1DfE3Vy7kEzcv5fF7151XR2PS0D6AADaBH2xvesvr0YQWBBofZrbE6MQkh1v7qHbmMzA64XP1VDsdb6kVuKrSwS0rixgYnWTzkgK9EF0i/pkqq0tzaO4eYfOSAr71aiMw/dwDtPYOk2ATctMS+dGOU1G9GFmFn+88RWqijY/ftBRgWkFsuuE++66VeBTcubZ0WmERLWhBoPFhZkt85Q/Haesb5ZolBRf1Ode63Lzp6iTRJrxw8GzUfhEijX/K6PfeOEFWSiI/3XnqonP/yI/38kD1IrqHxvmnm5dE9WIUSUxB7B4Ypb59kPe9bQH7m3tmVILMeMHdGxewvCSLPae6p7XaogUtCDQ+TM3nu2+cwJGRxJOvNl7Q1WMuRE/ct553rJlHgk14eNsevRBdAv4VrH862s74pOeixWLmYvTh6ytItAsn3INRvRhFElMQf/n3xxib9LBqXtYFlaAtm52+/81frS+lrrmXgozkqK2u14JAcx4L89OZ9CjcA2MX9TmbC1G108E7qkoYHJvk768u1wvRJVLtdOAsSAfgvln4+83FKCctiWuXFnqD+/lRuxhFkmqng6/fvY6f7z5NWW4qX3jh6KzjXXesLcVuE36xpzkMIw0NWhBozuM7Xp/0+y9fcFGfs79WdHVlAVkpCTR0DOiF6BKpdbnZf7qHstxUntl7Zk6W1R1r59HWN8qOxs4QjjC2sdsEj4Lm7uE5Bd5/uaeZqtJsfr33DJMeozFCtKVRa0Gg8VHrcvOjHU0UZSXzuTtXzZgtMR1JCTZuWVnMi4faGBmfDMNoY4tal5sP/WgPkwr+pnrRnOYe4KR7kJREG7/ed+a894ymxSjS/HiHUaW9ZXP5nALvVWXZ1LcP0NY3yusN0VlcpgWBxsfuk90oBW9fPQ8RmTFtcTq21rhwFmbQPzpBzfEOQC9Ec6GuuZd3ry8F4LplhXOae4D1C41K19/sb2FkfDIqF6NIUuty8/zBVlaUZPHobcvnJIirnQ6euG8dAnzh+SNRWVymBYHGx5LiTCY8ihuXF/qOzZS2OJWqsmy+VeMiI9nOb+ta9UI0R7ZsdtLUOcSCvDTKHUacYLZzb577iZuWMDzu4eM/2x+Vi1EkqW3oZNKjeOeaecD0KaMXYvOSQiqLMjh2tn9W8R2roQWBxsfLR9rITEngbYvz5nytoRWtZ3xS8cKBVl1cNkdGxid5w+XmuqUFiEzXlPfiPHjVYhJtwvMHWnVx2RxZmJ8GwOYl5zobz0UQ17rcnOkeBoxmdNGWOacFgQYAj0fxp6MdbF5SQKL90m6LaqeDm1cWMeFRXLu0UC9Ec2DHiS5Gxj1cu6zw4ifPwF9OdoGgi8sugZrjHRRkJrO8JHPO15rW7xfuWg3Au9bMi7p6Di0I4hyzkKbuTC/ugVFuWF54yb79Wpfb1wnz97q4bE78+Wg7KYk2rijPv6TrzcXoPZeV0T00zn/csTLqFqNIMelRvFbvZvOSS7PGzDTqO9aVsiAvjdbe4air59CCIM4xC2meqj2BTSA9KeGSfPv+xWWrSrNYmJ+mF6KLYAphpRR/PtZOtdPBnlPdlySE/StdAcY9KuoWo0ixv7mH3uHx89xCc8E/jfrKinx2NHaxcVFeVKVRa0EQ55hBsef2t1KclcKjzxy4JN++f3HZpsX5NLoHeey9a/RCdAH8N0Bp6hxiYV7aJQfYzcVo5bxsMlMS2N7YOScfd7zh39up5lgHNoHkBFvAWW7VTgf9oxPUnYmu+14LAg2rSrOZ9ChaekcuOcjorxVdXp7P2ISHlES7XogugCmE/+1ZYwuOZ/aeCTjAbrcJly/Oo9alC8suhH9vp5rjHSx2pPPoMwcCznKrdhquvTejbP61IND4CmnuXFsalCDjxkV5xmbrjV3BGF5MU+10UJSVAsADVywMSoB9U3k+TZ1DtPQMB/xesYophD+0bQ/7TvfQ0jMSlCy3/IxklhVn8kZDdLlEg7VD2a0ickxEGkTk0Wle//9EZJ/356CITIpInve1kyJywPvarmCMRzN7al1uvvZSPQL8x11zqyaeiey0RJYXZ7HjRHRpRZGgtsHNSfcgy4ozg5bpc0WUaqXhptrp8Gnwb68qDlqW25UVDnY1dUdVhX3AgkBE7MATwG3ACuAeEVnhf45S6stKqbVKqbXAp4AapZS/unid9/UNgY5HMzfqmnupKEhneUkWGckJcy6kmYnLy/PY3dTN6ET0fBnCTa3LzT94dyJ7/6aFQRHCAMuLs8hJS+RN3XfogtS63PzpaDs2gZePtAdFCG+tcZGXnsjYhIfdTd2+z7F6hX0wLIKNQINSqlEpNQY8DdxxgfPvAX4ShM/VBIG/u2oxJ9yDXLYw13csGEHGTeX5jE54dLD4AtQ19/L+y40sn/ULcoMmhJ98rZGKwozzLIJoWIzCiZnltigvnaqyHJ64b31QhHBVWTbfec3IwHsjivoOBUMQlAKn/Z43e4+9BRFJA24Fful3WAF/FJHdIvLQTB8iIg+JyC4R2dXR0RGEYWsAjrX1Mzg2yYZFuRc/eZZsrXFhZmNv9y5GeiF6K1s2O+kfnSA9yc7SYqOQKRhCuKosm8MtfZzpGeZ011DULEbhpK65l/9791pOdg2ybkFO0ISwWWFvE+GZPc1R0+ojGIJgugoMNcO57wTemOIWulIptR7DtfSwiFwz3YVKqSeVUhuUUhsKCi4t31fzVkzzdf2C4AmCqrJsHn3mAPNzU9lxoksvRBdgz6lu1szPwW67tLYS01HtdPC/37kSgM8+dyhqFqNwsmWzk9y0JEbGPazz3vvBSretdjpYXZrN2b5R7tk4PyrmPRiCoBmY7/e8DGiZ4dy7meIWUkq1eH+3A7/CcDVpwsTupm4KM5Mpy00N2nua2lV7/yjbGzt136EZGBqb4Ehr/3luuWDx3g1lJNmFl4+2675DM7D3dA8A6+bnBPV9a11ujrcPAPDDKOk7FAxBsBOoFJHFIpKEsdg/N/UkEckGNgPP+h1LF5FM8zFwM3AwCGPSzJLdTd1sWJR7yY3OZqLa6eD6ZYVMeBQ3rSjSC9E07D/dy6RHBdUaM3mzsROPgoLMJN13aAb2nurGkRFcJci0fv/jTsMie8+G+VFRYR+wIFBKTQCPAH8AjgA/U0odEpEtIrLF79S7gD8qpQb9jhUBr4vIfuAvwO+UUr8PdEya2dHWN0Jz93BIFqJal9tX1GS2pdacz55Thltu3YKcoL6vuRjdvrqEnqFxHnvvmqhYjMLNvlM9rFuQE1QlyKywv3NtKTlpiQyMTERFq4+g1BEopZ5XSi1RSjmVUp/3HtuqlNrqd873lVJ3T7muUSm1xvuz0rxWE1rM8nozPrBhUV5Qg7nmQvTN+9aTm5bIxkV5eiGahr2nuikvSCcnLSmo72suRretKmZ8UpGTlhQVi1E46R4co9E9GHQhbFbYiwhVZTnsb+6JilYfurI4DjHL63+zv4XkBBt9w+NBDeb6+g5VOFhVmk1b/6heiKaglGLPqZ6QWGPmYrTa+/88ECWLUTjZ19wDwNogxwf8WePdwnJobCJknxEstCCIQ8xg7h8Pt5GfnsRHf7ovqMFc/75DVWXZ1Lf1s35Brl6IOGeNNXUO0TU4xvoFuSFLrS3NSSU/PUkL4GnYe6oHm0BVWU7IPmNNWQ6THsWhlr6QfUaw0IIgTtmwMA+lAms0NxtWl2Yz4VEcPdsfkvePNkxr7Ke7jNIbu42QpdaKCKvLsrUg8OLfcXTvqW6WFGVS19wTsvqWqvnG/3S/NzvJymhBEKf8YvdpPApuXVkc0qyS1V6N64DXFI93TGvse6+fINEufOmFYyFNra0qy6G+vT8q3BOhxhTCb9S72Xe6h3k5qSGtbynMTKEkOyUqBLEWBHFIrcvN5393BIB/vnVp0HrcTMe87BTy0pM4EGX92UNJtdNBTloi45OK928KbY5/VWk2HgWHo8A9EWp8HUd/vIf+kQl2NHaGvL5ljTdgbHW0IIhD6pp7uarSQVqSnUX56UErr58OEWFVqXZP+PN6vZu2vlHWluWEPMff1Hb36/kHDGGwcVEeAHeuKw15fUvV/GyaOofoGRoL6ecEihYEcciWzU66BsdYXpKFzdvaIJRZJVWlRvZENLXlDRW1LjcP/3gPAO+/IngdR2eiMCuF4qwU7ZrzUuty81p9B3aB5w+Etr5la42LBO/3y1SErNpzSwuCOMTjURxp7WflvKywfJ65A9qRVu2eqGvu5f4rFgKwoiQrpNaYyeqy7KjbOjEUmPUt5QUZrCrNDlrH0ZmoKsvmm68Yi35dc4+le25pQRCHnOoaYmB0ghUl4REE5o2v4wSGNTYxqUi0CxWFGUBorbGtNS5yUhNp7Bikb2QcsK5WGmrqmnt5/J51tPQOszwMQvhcJ1L45Z4zlm7+pwVBHGLmNa+cFx7NpCQ7hfz0JA5oPzUAh1v7qCzMJCkh9F+/qrJsfn/oLAAHz/RaWisNNVs2O1lckE7P0DjLvUpQqAvtqp0OKgszOOEetHTzPy0I4pBDLb0k2IQlxRkh/6ytNS7ebOxkdVm2zyKIV43U5HBLHyvC5Jardjr47/esAeDxPzVYWisNB2b2VLjmv9bl5lSXsXf0D7dbtxNpXAgC/0ISk3hejA639lFRmEFygj3kn2XmbuemJVLfPsArR9vjViMFaO8fwT0wGja3HMAtK4vJSLZT6+q0tFYaDsw41TLvRkChxLS+/vHGSgAeub7Csj234kIQmIuR+Q+IZ/MYDNdQODXSx+9dx4uH25n0KD7ytNZIIXwaKRj3++iEh/x03ZL6SGs/8/NSyUxJDPlnmT233r3O2LDRLmLZnltxIQjMxeiDP9zN/d/dEdfmcXv/CB39o2GLD4Ax/3+13vgybFiUF5fzbnLYq5EuD5NFYCo9t60qoXd4nK+9b61ltdJwcLi1L2zWmNlzqyAzmdy0RI6e7bds87+4EARgLEZORwav1bu5b2P8mseHfYHi8Gqkv9nfSqJNfBt6xyuHW/ooy00lOzX0Gimc00pvWG5sElSYlWxZrTTUDI1NcLJzMGxC2EREWFacZel+W0ERBCJyq4gcE5EGEXl0mtevFZFeEdnn/fnMbK8NFrUuN/Xtxj/iBxYO2oQKM05iZgwtL8kKS5zE1Egfv28dq8qyKS9I1xppGBciUytdVmx85jELa6Wh5ujZfpQKnzXmz7KSTI6d7cfjmWk798gSsCAQETvwBMbm8yuAe0RkxTSnvqaUWuv9+T9zvDYgzMXo0duWAfDgVYvjbjEy4ySvHu9gfl4qh1p6wxIn8e1N4HSwtCiTs70jPH5P/GqkJ9yDYY0PmJQXpJNoF460WlcrDTW++EwEBMHy4iyGxyc51TUU9s+eDcGwCDYCDd7dxsaAp4E7wnDtrPEFbdaXASAQd+axGSfZebKLJLstbHES/70JlhZn0j00TkVhRlxppKY1ZmqkK8JkjfmTaLfhLMjg2Nn4re4+0tpHZkpCUPconi3LSowspaMWnf9gCIJS4LTf82bvsalcISL7ReQFEVk5x2sDwlyM0pMTWJCXxrG2+DSP15Tl4FHg6ohMccvSYvPLEF9aqWmN/XZ/CwAj45MRyVpbVmy4J+KVI619LC/OCuoexbOlsjATm2BZiywYgmC6WZ3qCNsDLFRKrQH+B/j1HK41ThR5SER2iciujo6OSx0rS4ri98vwzJ5mAG5fXRKRNELTT328Lb7m37TGtu04RZJd+OxvDkcka21pcRYtvSP0Do2H9XMjiWmNebybI62YF35rDCA1yc4iR3pMWwTNwHy/52VAi/8JSqk+pdSA9/HzQKKIOGZzrd97PKmU2qCU2lBQUHDJg11WnMkJ9yCjE/HVCbPW5eaLLxwF4P+7JbR7EMxEXnoSBZnJcWcRgCEMctMSGZtUESvqMouojsWRIDatsV/vO8PQ2CRJdolYDdFyC2cOBUMQ7AQqRWSxiCQBdwPP+Z8gIsXitcdEZKP3cztnc22wWVKcyYRHccI9GMqPsRx1zb1cu7SA5AQbC/LSwtL1cjqWxqlFVttg7EGwujQ7YkVdVvdThwLzPv/Ms4cAeHrn6YjVEC0rzqSpc4jBUevtFhewIFBKTQCPAH8AjgA/U0odEpEtIrLFe9pfAwdFZD/wdeBuZTDttYGO6UIsLfJqRXG2GG3Z7KR/dBJnQQb2MOxBMBNLizM53tbPpEXT6EJBrcvNh7btQQHv3VAWEWsMoDgrhayUBMtqpaGi2unwZQq9f9PCiNUQLfOOwYoWWVDqCJRSzyulliilnEqpz3uPbVVKbfU+flwptVIptUYptUkpVXuha0PJYoeRRhdvggCgvq3fF7CNFEuLMxmd8NDUGT8WWV1zL1uuNQRuZVFmxKwxs7Ap3u79Wpebfae7yU5N4OmdpyNijW2tcTHktQSOegPGVup3FjeVxSZJCTbKHRlx92XoGxmntXeEyqLQdxy9ED4/dRzN/5bNTt9OVUu8FmmkstbMwial4sMiM2uIHJnJvG1RfsSssaqybP79t4dJSbRx7Gyf5fqdxZ0gACNOYEXzLJTUe//eJYWRtQgqCzMRib8U0uNt/TgykshLT4rYGLbWuEi0CwOjEzR3G62RraSVhoK65l6+9r61tPeNsqQoI2LWmPm5E5OKF4+0Wa7fWVwKgmXFmTR3DzNgwaBNqDjeNgAQcddQapKdRfnpcZdCerxtwLcjWaSoKsvm57uMFOJjZ/stp5WGgi2bnRRnpzDhUT5rOFLWWLXTwbKSTFp6RrjPYu3A41IQmOZ5PC1Gx9v6SU20U5oT/qpKf7bWuMjPSDrPNRTrWqlSiob2Ad99FymqnQ6++r61AHzn9UbLaaWhot6rBFVG2Bqudblp7DBiY1bbpCYuBYHppz4eR+6J4239VBZlYLOFv6rSn6qybA6e6eOEe5CR8cm40Epbe0cYGJ2gMsKCAOCG5UVkpiSwvbErbjapOd7WjwgRtcjM+/yj3k1qHr7Waal+Z3EnCLbWuDjVOURakt0XJ4h1jRQM10SkNVIwtNIPXlOOAv73c4fiQis97ovPRNY1BMa9PjI+iSMjfjapaWgfYEFeGimJod+RbybMfmd3rjU66CQl2C3V7yzuBEFVWTYffnovJdkpceMn7R4co6PfCJZZgXeumQfAT3eejgut1HRNRFoQm/f6zSuK6BuZ4Ot3RyaDJtwcb+uPuFvIf5OarJQE6tut1e8s7gSBGb1v7h5m76nuuNJIreCaAGjtNTJWLluYGxdaqZExlExuBDOG4JxWes2SAsYmPMzPS7WUVhoKxiY8nHAPRjxt2kREqCjMoKF9INJDOY+4EwRgCIO3LcpjeNzDX60vjWkhAHDce9MttYAgqHW5+cen9zEvJ4W89KSI5XWHk+PtA5awxkyttMKrHTe0D1hKKw0FTZ2DTHiUJebfpLIwUwsCK1DrcrP/dA9AxCoNw0l9Wz+ZyQmUZKdEeig+rXRNWY5vIYplrVQpRUNbf8TdQv6YQdN6iy1GoeC4RTKG/KksysA9MEbX4Fikh+Ij7gSB6Sf9jztXAfDeDfNjViM1W/AeO9tPRVEGIhLxwPg5rTSDpk6jC2wsa6VneoYZHJuMeA2BP9mpiRRmJltOKw0F9e1GxpCzwDrz7/TeC1aa/7gTBKZG+s4180hNtKNU7O5WZrbgPdzax9KiTEsFxisKM/AoOOm25tZ9wcLUuq1kEYAx//FgEdS3GRlDqUmRyxiaSqXPIrNO+nrcCQJTI7XZhPKCdBo6YtdPWu108IW7VtE/MsGZnmFLBcZNDc1KWlEwMa0xX2uPooyIW2P+VBZm4GofiPmeQ/Xt/b6F1yrMy04lLcluqXs/7gSBPxXeL0Msk5NmZKq8Vu+2VKqmsyADkdgVBKY19nqDm4LMZA639lnGGgPj3h8YneBs30ikhxIyxifNjCFrWWM2m/Uyh+JbEBRkGD7cGO459OLhswD8TfUiS6VqpibZKctNtZR5HEzMIPgbDZ0kJ9gsZY0B52UOxRqmNdbUOcj4pKKy0FrWGBhrj1lfYgXiWxB4TUaz/0esUety86Ptp0iy2/jMO1ZYLlWzosBaWlGwuaI8H5tAc/ewpawx8MscstBiFCxMa+w3+1sBGBmftJQ1BlBRlMHZvhH6Rqyxf3RQBIGI3Coix0SkQUQeneb1+0SkzvtTKyJr/F47KSIHRGSfiOwKxnhmi/llaOiITa20rrmXpcWZLCk2egxZLVWzojCDRvdgzO5W9vyBVsYnFZuXFFjKGgNwZCSRk5ZIQ0fsCQLzPn/y1UYAvvyHY5ayxuBcOqtVFKGABYGI2IEngNuAFcA9IrJiymkngM1KqSrgc8CTU16/Tim1Vim1IdDxzIWF+enYbWKZf0aw2bLZSefA2Hmpc1YKjFcUZjA24aG5O/Yyh2pdbh595gAAH7ym3HLWmIgYFlkMWgRg3Ofz84xOu/dHcHvKmai0WAppMCyCjUCDUqpRKTUGPA3c4X+CUqpWKdXtfbodKAvC5wZMUoKNhflplvlnBJuhMSNbqMJCOdT+xLKfuq65l79ab9zmzsLIbYhyISqLMmLSIgBDELs6BlmQl2Y5a2xrjYvm7mGSEmy+ez/SMYxgCIJS4LTf82bvsZl4EHjB77kC/igiu0XkoZkuEpGHRGSXiOzq6OgIaMD+xLKf2ox9OC2WPmcSyxWuWzY7UUqRkZxAYWYyYC1rDIzMra7BMToHRiM9lKBS63LzyLa92EW4cXmR5ayxqrJsPvL0Xooyk6lvs0bjy2AIguka3E/r9BWR6zAEwSf9Dl+plFqP4Vp6WESume5apdSTSqkNSqkNBQUFgY7Zh1HhOsT4pCdo72kVXF5tz0pVlf5kpyZSEMMVrg0dAzgLjYpuq7G1xsWENzZjFa00WNQ19/LZO1YyNumhwoLWmDmetr5RdjVZo/FlMARBMzDf73kZ0DL1JBGpAr4D3KGU6jSPK6VavL/bgV9huJrCRkVhBhMeRVNn7GUOudoHsAkscqRFeigzEssWmat9EGdBeqSHMS1VZdlsfcVY9Bs6BiyhlQaLLZud5KQmAvjm32rWWLXTwboFOfSPTPC+DfMjHsMIhiDYCVSKyGIRSQLuBp7zP0FEFgDPAPcrpY77HU8XkUzzMXAzcDAIY5o1FRYL2gSThg6jvD45wTrl9f5srXGRmZJwXoVrrGilZrGWVa2xaqeDb9y3HoCf7TxtCa00mPisYYu6RWtdbg619AGwbUfkt60MWBAopSaAR4A/AEeAnymlDonIFhHZ4j3tM0A+8I0paaJFwOsish/4C/A7pdTvAx3TXCiP4VYHrvZBSzU7m0pVWTavN7jpH52gvX80prRSs2LdqoIAoLrCQWFmMvubey1X5xAoDe0DZKcmkh/hPSCmw7zP//XtywH4QPWiiMcwglJHoJR6Xim1RCnlVEp93ntsq1Jqq/fx3ymlcr0por40UW+m0Rrvz0rz2nCxtcZFXXMPJdkpMecnnfCW11t6IXI6+Jh3D9cvvnA0prRSUyO1siCudbnpGR4nMznBcpk1geLqGMBZkG7J+Ixv28p1pYhAgk0iHsOI68piswLRkZEUc37S5u5hxiY9lhYEAHd493D91d4zMaWVujoGSLAJC/OtGZ8x7/U71syjf3SCr7ynKuJaaTBxdVhXCTIbX6YkGm1WXB2DEY9hxLUgMKP3x9sGONrazyPbYk8jtaqP1KShfQABqkqzY0orbWgfYEF+Gol2a37FTK30umWFABRkpkRcKw0WvcPjdPSPWv7eByh3WKPxpTXv0jBS7XRwhTOfCY/iXWvnxYQQgHMxD6sWk4FXK/3JXsod6WSlJlou3zsQXB2Dlp57Uys1teZGd+S10mDRaPG0aX+cBRk0ugfwRLjNStwLglqXm91NRtHzL/c0x8QiBIZF4MhIJjstMdJDmRHftpXzc3B1xM62leOTHpo6B6NCI12Yn4YIltBKg4XLW0hp5fiMibMwnZFxDy29wxEdR1wLAtNP+qV3rwbg3etKo14jNVvwNrQP+HKorRoA92mlhRm09o4wODoRE1rpqa4hxidVVGikKYl25uem+VyJsYCrY4BEuzA/NzXSQ7koPosswh2Q41oQmBrpbatLyExOwBMD21aaAfBjZ/txevuwWz0AXu4wBNYJd2wU9Z1LHbVmMdlUnAXpEV+IgomrfYBF+ekkWDQ+448pCCItiK0/UyHE1EhFjG0rG93Rv21ltdPB5+9cxeDYJKe7hqIiJdN0oUT6yxAopjXm8uvxZFVrzB+r+KmDhZE6an1rDIx24JkpCRG/9+NaEPjjLMjA1R4bWlFuujW3p5yJhflp2OScbzdaMa2xNxvdFGYmc/BMr+WtMTCKKq3gpw4GRnxmCGdhdFhjImKJtUcLAi/OQmPHoIEY2LbyxcNtADxwxcKoSMlMTrAzPy/6/dT+21Mm2q23PeVMmC6saBbE57anHGLCY8RnosEaA68Sqi0Ca+DzU0fxlwGMwPC27U0k2oT//c6VUZOS6SzIiAk/9RXl+dgFzvRYb3vKmXD6tmyNXkFsWmPPHzC2pxwcnYgKawyMzKH2/lH6I7htpRYEXnxfBnf0fhnACIAvK8nEWWjN7SlnotyRTmNH9Pupf3/wLGOTimsqHVFhjQHkpyeRZQE/dSCY9/k3vR1VH3vxeFRYY2CNzCEtCLz4/NRRnk+9ZbOT7qHx83LYoyEA7izMYHTCw5me6PVT17rc/PMv6wB48GrrbU85EyKCszDyfupAqXY6fC09rLg95UxYIXNICwIvPj91lKcwjk4Y2UJOR3QEy0z8K1yjlbrmXt67wdiaw1mQHjXWGJzLHIpmzPqZstzUqLHGABbkpWG3iRYEVqHckR71FkFT5xAeZf0eQ1MpNwOWUTz/ptWVkmhjXrZRzBQN1hgYgqCtL7J+6kAw62WS7MK1SwuixhrbWuNiV1MXC/PSfBZZJILcWhD44SzI4IR7MKr91OZCWu6ILkGQn55EdmpiVPupwQi4LspPx2azXvvjC2EK4mgN2Nc19/KFu1YxNG503I0Wa8wMcuekJdHojlwHZC0I/CgviH4/telaKY+SqlYTI586+itcXR3R0WPIn601LvqHDUvAdA9FS+qlyZbNTnLTjPoZc7OpaLDGTIF1uLWXhvYBHt62JyJB7qAIAhG5VUSOiUiDiDw6zesiIl/3vl4nIutne204OZdPHb1aqat9gOKsFNKTEyI9lDlTboF86kAYnZikuTv64jNVZdl8/vkj3mSJwahoSzIdphIULa09TKqdDq50OvAoeGdVZDogBywIRMQOPAHcBqwA7hGRFVNOuw2o9P48BHxzDteGjXP51NGrlbrcg1FTVenP1hoXCTY5L5862rTSaI3PVDsdPOHdv/iFg61RUwg3FVf7wHnxmWih1uVm58kuAJ7ZeyYicY1gWAQbgQbvtpNjwNPAHVPOuQP4gTLYDuSISMksrw0b0Z5PrZSisX0g6uIDYGilv60zioEaO6JTK43W+AyYqZfpuDoGo6YQbiqN7kEWOzKiKj5j3uf//Z41ANyxdl5EgtzBEASlwGm/583eY7M5ZzbXAiAiD4nILhHZ1dHREfCgp7K1xsWbjZ04C89VuEabRtoxMEr/6ETUmcZgLESffadhDD7+54ao1EqjNT4Dxr3e2juMTeBH26Mn9dIfV8dA1M292QH55pXF5KUnMelREQlyB0MQTCd+p6bdzHTObK41Dir1pFJqg1JqQ0FBwRyHeHHM6H1GsmERRKdGai5E0aeRAtyxrhTB6JUUjVpptMZnzHv9A5sW4VHwb+9cHhWpl/5Ea/2M2QEZzrUDj0SQOxiCoBmY7/e8DGiZ5TmzuTYsmNH7XSe7ae8fjVj0PhDMjI9o04pMdp7sQsT4QkRTQZBJtMZnTK30llVFAOSkJkVF6qU/0Rqf8afcEblkiWAIgp1ApYgsFpEk4G7guSnnPAd8wJs9tAnoVUq1zvLasFHtdHC9dzPvm1YURZUQAMMiiMZgGZzTStfOz8Vuk6gpCDKJ5viMqZWaYze3DbV66qU/ZsO8aJx/E2dhOp2DY/QMjYX9swMWBEqpCeAR4A/AEeBnSqlDIrJFRLZ4T3seaAQagG8DH7rQtYGO6VKpdbl5vcFYeH53oDVqFiGTRvdA1AXLTEyt9G2LcznpHuLyxflRpZVGc3zGJDc9ibz0pKhMljBbaEerNQz4CeLwZy0GxZmplHoeY7H3P7bV77ECHp7ttZHA1Ej/5551/K/v7+Sm5UVRE7DcWuOiqiybxo5BX0yj1uWmrrk3arQ6c5zN3cOMTXpo7h6i2umw/NybRHt8xsRZkB6V+xK4OqIzPuOPfzvwyxbmhvWzdWWxF1MjvWZJAQvz0hid8ESNRlpVls3D2/ZwqmuI8oLo2Kd4JqK1qC/a4zMm5Y6MqNyXoLEjOuMz/szPTSXRLhERxFoQePGP3psVrtHiJ612Onj0tuUAHDvbFzWWzHT4zOMoaYns26fYLz4TbWnH/jgL03EPjNE7FD3N55RSRupoFMcHABLsNhbmp0dEEGtBMA3OwnROuoeYmPREeiizJjPFMIn/cCg6Uy9NctOTyE9PipqWyGba8e5TXSx2ZLD9RGfUWmPg1xs/SuYfwD0wRv9IdMdnTMod6RGxhrUgmAZnQYbXTx09zef+fLQdgH/YXB6VqZf+lBekR41FYKYdH2juZWLSE9XWGJyLcURDO3CfNWZmDEXRPsUz4SzMoKlziPEwK6FaEEyDFXYMmgu1LjfP7mvBkZ7EJ29bHnWpl1Oxwmbec2H9glw8CurbB6LaGoNzfupo2CDItMZePNwGQM/weFRbY2BYBBMexemuobB+rhYE0+CMst7sdc29lOamsnxeFkDU9GKfCWdBRsTyqS+FZ/edAeCWlcVRb42ZfuposAjM+3zb9ibsNvjss4ei2hqDc5lD4Q4Ya0EwDTlphp86WrTSh64up61vxGfJQHT0Yp8JM/sjGtIYa11uPvfbIwB85IaKqLfGwEwhjY57v9rpoCg7hUkPvH9TdFtjAE7HuRTScKIFwQxEk3vibN8IQ2OTUV1e749/havVqWvu5daVxYgY4452awyMe/9UV/j91JdCrcvN6a4hKosyot4aA8hOS8SREX4lVAuCGXAWRk9hjXnTVER5MZNJWW4qSXZbVLjmtmx2MjbpoTQnldQkOxDd1hgYQdfxyfD7qedKrcvNw9v24FHwjtXzot4aM4Pf5QXh74CsBcEMOAsy6Boco3vQ+n5q058b7QU1Jgl2G4scaVFhEYAhiJ0xIoTBv6jP2oK4rrmXf75lGWDc+9FujZ3rgGwPewdkLQhmwPxiR0M+u6tjkMyUBAoykiM9lKARyU6Mc8HjUTElCLbWuHAPGMqP6ae2akrmls1O0r31MxWF0bNP8UyYgmx7YxfdQ+N8KIwdkLUgmAGzVUA05LObC5FI9DWbmwlnYTqnIpBPPVdaeocZGff4FqJop6osm0/+so6slMSo2JfD1T6ATWBRfmxYw9VOBzcuN9qB37C8MGzBby0IZqAsN40kuy0qtNJY0kjB0Eo9CiY8ilNeP7VVtVLTfRILVa1wTisdGpvgtXq35QvkXB0DzM9LIyXRHumhBIVal5ua48YOjL8/cDZs8Q4tCGbAbhMWR6jcey70j4zT1jcaM/EBMLTSbdubAEPjs7JWei4+EzuCuNrpYFlJJq29I9xn8QK5hvbYUYLM+/wb964nJdHGNUsKwhb81oJgGszovX/mkNU10ljJGAJjIXrsfWsB+H7tSUtrpQ0dA2SnJpKfnhTpoQSNWpfbl7Xyw+1Nls3CmfQoTrgHY8YaMzsgX1lpbBI0NDYZtuC3FgTTYEbvkxNsnOoa4tXjHVojDTM3Li8iLclOravT0m0bXO0DVBTGTnzG1Eo/ckMlAI9cV2HZlMwz3cOMTsROfMa/A3JFYQYN7eHrgByQIBCRPBF5UUTqvb/fspuCiMwXkT+LyBEROSQi/+j32mdF5IyI7PP+3B7IeIKF6Sf94+E2Jj2KD//Euhqpq2OABJuwIC8t0kMJKrUuN2MTHoqyki1dKOTqiB2NFM5ppe9aMw+A5ES7ZVMyTbdtrLiG/KkozOBMzzBDYxNh+bxALYJHgZeVUpXAy97nU5kAPqGUWg5sAh4WkRV+r39VKbXW+xPxncpMqp0O3lFlfBmurMi3pBAA48uwMD+NRHvsGHemVnr9skIGRyd5/B5rFgr1Do3jHhiNqYXI1EpLslNIT7LjCqNWOlca2mNbEED4+p0FunrcATzlffwUcOfUE5RSrUqpPd7H/Rh7E5cG+Lkhp9bl5o+HzgLw56MdlluETAyNNLa+CKZWevWSAgZGJ1hckG5JrbTBrOiOEdeEPyJCRWEG9e39kR7KjLg6BshPTyI3huIzJuY91RCm5n+BCoIipVQrGAs+UHihk0VkEbAO2OF3+BERqROR703nWvK79iER2SUiuzo6OgIc9oUxNdIn7ltPWW4q6xbkWEojNYPZ45MemjoHcRZGfx92f0yt1AyA17dZUyuNZdcEGHGncC1El0JD+0DMxcZMFuWnY7eJdQSBiLwkIgen+bljLh8kIhnAL4GPKqX6vIe/CTiBtUAr8JWZrldKPamU2qCU2lBQUDCXj54zpkZa7XRQWZhB99C4pTRSM5j93L4zjE8qUFg2mB0IlUXh1YrmiqtjgCS7jbLc1EgPJSRUFGbQ1jdK34g1t62MtfoZf5ISbCzMSwvbvZ9wsROUUjfO9JqItIlIiVKqVURKgPYZzkvEEALblFLP+L13m9853wZ+O5fBhwp/zbOiMIM3XJ1cvtg6cQIzmP3QD3YDsG1HE1vvv8wy4wsW+elJ5KQlUm8xQbC1xkVVWTau9kEWOdJIsNuodbmpa+61nNUSCBV+u5WtWzCjsR4ROgdG6R4aj0m3nImzMHxtVgJ1DT0HPOB9/ADw7NQTxMir+y5wRCn12JTXSvye3gUcDHA8QaeyMJOxCQ/N3dbqxFjtdLC61LAA7tlo3fTKQBARKgszLLdJimmRHTrTS4XXLRebFlkmYE2LLNYquqejojCDk52DYdk7PVBB8EXgJhGpB27yPkdE5omImQF0JXA/cP00aaL/JSIHRKQOuA74WIDjCToVRef81Fai1uVmd1M3Gcl2fr672TLxi2BjxYBltdPBV9+3hta+ETr6Ri1d8BYI873twK0kCMz4mH/GUCzFx/yp8LYDbwpDO/CABIFSqlMpdYNSqtL7u8t7vEUpdbv38etKKVFKVU1NE1VK3a+UWu197V1m4NlKmKanldwTpgY6LyeFyxbmRX0f9gtRUZhJ99A4nQOjkR7KeRRlpQCws6nb0gVvgZBgt7HYkW4pQWBaY6/Vd5CSaONU51BMWmMQ3syh2Ek+DxFZKYkUZSVbSiuta+7l63evo7V3hMrC2NgVayasKIgBfltn6Cz3bJxv6YK3QKkozPClyVoB815/6UgbGckJfPjp2LTG4Fy3AC0ILEJlYaaltKItm53Mz0tldMLDEq8f14rplcGgMsz51LOh1uXmO681YhP47LtWxrRF5izM4HTXECPjk5Eeio9qp4PkBBvugbGYtcYAMpITKMlOCUuMTAuCWWD2/VBKRXooPo57YxZmDCNWMStcrSQI6pp7WVWazSJHOskJ9pi2yCoLM/Co8FW4zoaXjrQxMDrJFeX5MWuNmbEQf4sslLEQLQhmQWWR0QmwpXck0kPxYbqqKmM4fQ6MzCGrFTZt2eyka2CMJYWZvmOxapH5/NQWcQ/Vutx8/Kf7AHjwqsUxa42ZsZA0b5uP2obQZqZpQTALzlW4WidOUN82QEl2CpkpiZEeSsixWubQyPgkJzsHWRLj1tjWGhdtfSPY5JxrLtIZOnXNvbzvbfMBWFKUGbPWmPl3vVbvZnBsMuTbVmpBMAusmE9d397vG1esY7UK18aOQTyKmJ//qrJsPv6z/RRkJltmg6Atm51MeBSpiXZfRXesWmPVTge3rSoG4JolBSGNhWhBMAvy0pPIT0+yTC2Bx6NoaB+IebcQGFrppMeIzVhFK/W55WLcIjC10q7BMd5stM62lfVtA1QWZWCzxcYeEDNR63Lzp6NGs4YXj7SF1P2lBcEssVIaXXO3sWF6rLsmwNBKv/PaCcAQBFbQSo+39fu2Mo11qp0Oqkpz6Boc556N8yMuBMCY/8rC2LbG/BtfFmUls2FhbkhjIVoQzIKtNS4ykhOob+v3ZQ5FUis97o1VVMT4lwHOaaUAT//llCW00uNtAyzKTyM5ITY2TL8QtS43R9uMHpE/2h75DJ2eoTHa+0djXgnyb3y5pCiTnhA3vtSCYBZUlWXzZmMnfSMTdPSPRlwrNYurYt01YXJ1ZQGOjCT2nOqxRN54Q/uAr34jljHv8397u7GP1P2bFkQ8Q8dMm15SHNvz779t5dKiTI639XP54vyQxUK0IJgF1U4HH73R2MP1C88fjbhWWt/WT3FWCllxkDEExoLUNzJBepI94nnjI+OTNHUOxnygGM5ppXetL8VuEzyKiGfomNZwPAhik6XFmYxOeDgVwp5DWhDMkrvWlQHw631nIq6V1rcPxI01YGqlf72+lMGxSb707tUR1UpdHQN4FDHvmoBzWmlygp1yRzrH2/ojnqFzvK2fjOQE5mWnRGwM4Wap1/o5djZ0KdRaEMyS+vZ+BFhekhkRrdSsNDyXMZQZ8eyZcGBqpTevNNLoMlMTI6qVmplj8aSRgrEYHQ3hQjRbjrf1U1mUgdHdPj6oLMxERAuCiGNqpSvnZZFgs0WkmtGsNPzN/haGxyex22JzV7KpmFrpsuIswPgyRFIrPd7WT4JNWJQf+xlD/iwrzqS5e5iB0YmIjqO+beC8iu54IDXJzsK8NJ9bLBRoQTALTK10U3k+x9v62bgoL+xaqZk98+lfG3v3/HTn6Yhnz4SToqxkctISOXq27+InhwDTIjveNsBiRzpJCba4sMhMlvoJ4kjhHhilc3Asbtyi/iwpygzpvR+QIBCRPBF5UUTqvb+n3c9ORE56N6DZJyK75np9pPFppSVZjE54ONk5FBGtNB52JZsJEWFpUeTcE6ZFduBMD0uKMiOeORZuloXBT30xTI14aYxnDE3H0uJMTnaGrgtsoBbBo8DLSqlK4GXv85m4zrspzYZLvD7iRPrLYO5KlpmSENO7ks3E8pIsjp/tx+MJfxfYaqeDx96zhra+UdwDsbsr2UyU5qSSnmTnWAQsMtMa84/PxJM1BoYgmPSokHWBDVQQ3AE85X38FHBnmK8PKxWFGdhtEhH3hKmB5mckcUV5fsx2XbwQS4szGRybpLl7OCKfn5ueBMCOE10RzxwLNzabUBkhi8y0xl493kFWSgKu9oG4ssbAqCUAONYWmrUnUEFQZG4v6f1dOMN5CvijiOwWkYcu4XpLkJJopNEdaQ3/l6GuuZevvKeKs30jLC/JitmuixfCdAlEKk7wm7oWAB64YmHE6xkiwbLiTI75VdeHC/NerzneQXpyAo/8JL6sMYBFjnQS7cKxs6Fpc3NRQSAiL4nIwWl+7pjD51yplFoP3AY8LCLXzHWgIvKQiOwSkV0dHR1zvTxoLCvJishCtGWzk5y0JJQyXCQQu10XZ8KnFUVAK611ufnhm02kJtpifleymVhabLQ66OgP//7RmxbnIwKtvSNxZ40BJNptOAsyQuaau6ggUErdqJRaNc3Ps0CbiJQAeH+3z/AeLd7f7cCvgI3el2Z1vffaJ5VSG5RSGwoKCubyNwYVM40uEi2RTUtk5byssH+2FUhPTmBBXlpE3BN1zb0szE9jdVkOIhLnFln45//ZfWcYn1Rcv6wwLq2xrTUu8tKTfC02ILj9zgJ1DT0HPOB9/ADw7NQTRCRdRDLNx8DNwMHZXm81zIDx8Qh8GY609pGZnODrwx6PGIVN4bfIHrq6nObuYVaUnBPC8WaRLYtQCmmty82/PXsIgI/duCQurbGqsmz2nurhTM8w/SPjQc9aC1QQfBG4SUTqgZu8zxGReSLyvPecIuB1EdkP/AX4nVLq9xe63sos8y4ERyIgCA639rGsJDOuqiqnsrw4kxPuwbBvpt7UNcTQ2CQr4tQa21rj4ujZPgoyk30WQbgyd+qae7lheSEJNqGyKCMurbFqp4NHrjOUjn//zaGgZ60FJAiUUp1KqRuUUpXe313e4y1Kqdu9jxuVUmu8PyuVUp+/2PVWZl52CpkpCWFPo/N4FEdb+87TSOONrTUuRIzmZ+HepOZwi/H/jtf5NzN3irOSOdbWF9Y6ii2bnfQOj1NRmEFKotH6O96sMYB3X1bGkqIMfrE7+P3OdGXxHBERlhdncTTMmUOnu4cYHJv0BYrjkaqybL5fexIw/NThXIwOt/b6NNJ4xNTC69sGONLSx8Mh3kN3Koda+uLWGjM54R7EPTDGR66vCHqcRAuCObK1xuVtdRDeTWpMjTSeBUG108ET964HYNv2prAWdR1u6aOiMCMuNqOZiWqng2uWFDCp4LbVxWETAu39I3T0j7JyXvzUDUzFVHoev3cdH795adDjJFoQzJGqsmxeb3AzMDpBc/dw2LTSI6192CQ+y+v9uarSQWFmMntPh3eTmsOtWiOtdbnZccLw3j67tyVswVpTCYrXbDk4f8cyIOhxEi0I5ki108Enb1sGwBdfCN8mNYdb+ygvOOcjjVdqXW56h8dJskvYtk50D4zS1jcat/EBOKeRfuPe9aQn2bmywhG2zJ1D2ho+b8cyk2DGSbQguATec5mxSc3vDrSGTSs90tof1wsRnFuMHrhiIWOTin99x/KwLEZHWr2BYq2RcmWlg5Wl2bT3j4Ytc+dwSx/z81LJTo2PHfkigRYEl8C+0z3YRViQlxbS4haz2Vbv0DhneoZZXpIVd822/DEXozu9u8XZbRKWxSjeM4bgfI20qjSbI619bFyUF5bMncOtfawsid/4QDjQgmCOmFrp1ZX59I+M8/g9oStuMVP2fr77tPeIirtmW/6Yi9GSogySE2zUNfeGNI3QFMSHW/sozUklJy0prgWxyeqybEYnPNS3h6bvjT8DoxOccA/GtTUWDrQgmCOmVnrTymK6h8Ypy00LmVZqBoQee/E4AN+qaYy7ZlvTkWC3sXJeFgdCbAmYgnjXyS6fNRbPgtjE3BMj1PMP59xy8RwoDgdaEMwRUytdU5YDwP7mnpBqpdVOh6+lxAeuWBj3QsCkqiyHgy29TIZwb4Jqp4PH3ruGMz0j9A2Px90eBDOxKD+djOQEDpwJnSAwrbFD3s9YOS9bW2MhRAuCS2RpcSZJCTbqmntC+jm1LjcN7QM4C9LjstnWTFSVZTM0NomrI7TuicwUI0D5l5PxtwfBTNhswqrSLOpCKAhMa+yVY+3kpyfR2BF/exCEEy0ILpFEu40VJVnsD6F5XOty86Fte/AoeN/b5sdls62ZMBeEUAeKf7WnGYAHr1ykBbEfq70B4/FJT0je33SLvtbQSVqyPS73IAgnWhAEwNr5ORw8Ezr3RF1zLw9etRiAdQty47LZ1kwsdmSQnmQPqUVW63Lzs12ncWQk8W/vjM89CGZidVkOYxMe3z7CoWDt/Bw8HsXprmFtjYUYLQgCwHRPNIQoe2LLZifDY5Mk2IRV3vL6eGy2NR12m7CqNDukQrGuuZf05ISQVXNGM2bA+GAI3UPbdjShgHetmaetsRCjBUEAVPkFjEPFvtM9LC/JIjUpviuKp6OqLJvDrX2MTYTGPfGuNfPoHhpn/YIc3zEtiI1Abkv3MJkp5wLGwQ7k1rrcPPbHegA+d8cqbY2FGC0IAqDckU5mcgL7T/eE5P0nPYr9p3tY57cQaQy21rhITrSf554I9mK051Q3AOsX5gbtPWOBqrJsPvz0XubnpnGguTckabV1zb0sLc6gsjCD7LREbY2FGC0IAsBmE1aXhc49Ud/ez+DYpBYE01BVls0P32wC4MCZ0CxGe5p6SEm0xXWPm+kwF+VG9wAHzvSGpCX1Q1eX09gxyGV+QlhbY6EjIEEgInki8qKI1Ht/v0V1EpGlIrLP76dPRD7qfe2zInLG77XbAxlPuNla4yI/PZmjZ/t8O2YFUyvde6oHgHXztUY6lWqng2/etx4BfhSiltR7TnVTVZpDol3rS1Opdjq4flkRHgU3LC8MeiDX1TFA38jEeYJAEzoCvcMfBV5WSlUCL3ufn4dS6phSaq1Sai1wGTCEsYG9yVfN15VSz0+93spUlWXz52PtjE8qjrQGf9emvae6yU1LZGF+WlDeL9aornBQlpfKoZa+oGeVjIxPcqill3ULc4L2nrFErcvt89f/ru5s0H33u5oMt5wWBOEhUEFwB/CU9/FTwJ0XOf8GwKWUagrwcy1BtdPBf757NQBfffF40LXSvad6WLcgN673KL4QtS437v4xAH6wvSmoi9HBM72MTyrWL9AL0VR8LanvW8/SokwqCtODHsjd3dRNXnoSix3pQXtPzcwEKgiKlFKtAN7fhRc5/27gJ1OOPSIidSLyvelcSyYi8pCI7BKRXR0dHYGNOoi8c808MlMSeLXeHRSt1NdxdHic+vYB1s3P0aX102AuRo/ebuwN8YErFgZ1MfIFirUgeAv+m6RsKs/D1THI/717bVBjZXuaulmvlaCwcVFBICIvicjBaX7umMsHiUgS8C7g536Hvwk4gbVAK/CVma5XSj2plNqglNpQUFAwl48OKbUuN2MTnqBtlGKW1j/9l1MAJCXYdGn9NJiL0b0bF5CeZKdzYCwoWSWmIN7T1MOCvDQKMpO1IJ6Cf0vqy8vzGRqbJD05IWiB3M6BURrdg9otFEYuKgiUUjcqpVZN8/Ms0CYiJQDe3+0XeKvbgD1KqTa/925TSk0qpTzAt4GNgf054cXUSv/+6sWMTSo+dlNlwFqpmZHxtZeMHOqtNS5dWj8N5mKUaLfxtsV5vNnYGZSsElMQb2/sZP2CHN1x9CJsXJwHwI7GroDfyyeEvUkSGxblaiEcJgJ1DT0HPOB9/ADw7AXOvYcpbiFTiHi5CzgY4HjCiqmVPlBttIHoH50IilZa7XSQm2Y0O7t/k+44ejGuKM+nsWOQtr6RgN+r2ungs+9cQc/wON1DuuPoxXBkJFNZmMGOE50Bv5cphJ/dd4ZEuzA4OqGFcJgIVBB8EbhJROqBm7zPEZF5IuLLABKRNO/rz0y5/r9E5ICI1AHXAR8LcDxhxdRKCzKTWVqUSW1DcLTSl4600dI7woaFubq0fhaYi/T2xsAXI4CRcaNSueZ4h+5xMwsuL89j54kuJgJsQGdaw78/eJa89CQ+/rP9WgiHiYAEgVKqUyl1g1Kq0vu7y3u8RSl1u995Q0qpfKVU75Tr71dKrVZKVSml3mUGnqOR6op8dp7s8tUTXCq1LjcffXofAP986zJdWj8LVszLIislgTddwREEv9h9GpvAh6+v0IJ4Fmwqz2dwbNK3yXwgrCnLwaMUbX2jWgiHEV0pEySudDoYnfD4sk0ulbrmXjYuziUzOYF1C3J0af0ssNuEjYvzeTMIFsFr9R385WQ31y4t5BM3L9WC+CJsrXFh8yb2mO6hQPz633ntBB4Fd60r1UI4jGhBECQuL8/DbhNqGwJbjD54TTlHW/u5ssLhq2jVpfUXZmuNi5LsFJo6h2jpGQYufTF6/sBZAO5+23xAdxy9GFVl2fzrrw9Rkp3C9saugILrtS4333ilgdREG//111VaCIcRLQiCxLYdp1jsSOMNv5v2UhYjV8cALb0jbF5qnRRZq1NVls2z+84A8KarM6DFKDMlgUS7cGXFOZeEFsQzYwrKrsExXq/v4JFtlx5c33+6h+QEOzcsLyLRbtNCOIxoQRAkqsqyae4eZv/pHvpHxi95MXrlmFEsd80SLQhmi9F36DIE+M7rjQFl+vz5aDuXL84nPTkh+AONUaqdDq5bVsjYpOK6ZQWX7NffVJ5P38g4Ny4vOu+9tRAOPVoQBIlqp4NP3LwEj4JPPXPgkhejmuMdVBRmUJqTGqKRxiZXVjpwFmZwpLWfezdeWpDxdNcQ9e0DXKutsTlR63Kzo7ETuw1+W9d6ya6cl4+0YxP0/EcALQiCyP2bFpFgE35b1zqnjAezkGZ4bJIdJ7rYvKRAF9LMkVqXm7O9Rh3BU2+evKTF6JXjhjV2/bKLdUrRmJiW7xP3reeWlcUkJ9h5ZNul+fVfPtrOhoV55KQlhWCkmguhBUEQ2XOqG5sISXbhh3NogmYW0vy/2hOMTXgoyEzWhTRzwFyMHr93nS/barZBRlMIg+EWWpifxtneES2EZ4l/36F3VM2jb2Scf7i2fM5+/TM9wxxp7eOG5VoIRwItCIKEuRh94uYljE0q7p9DEzQzKPZ/X6rHLvAt3VZiTpiL0bVLC7l1VTF7mnp47L1rZrUYmUL4lWPt1LrcLCvO5JGfaCE8W/z7Dl23tJC0JDuN7sFZ+/VNQfynI0bnmRuWF2lrOAJoQRAkzMXo764upzgrhQPe57PVjNbOz0EpxaTSbSXmiv9idMfaUgZGJxgcnZzVYmQK4Ud+vJeRcQ+1DZ1aCF8iqUl2blxexAsHzzI+yypjUxD/Yk8zi/LTaO8b0dZwBNCCIEiYi5HdJty1vpRX691UFGbMWjP66ovHGZtU/NVlupAmEK5w5lOYmcyvvemks6Ha6SA33ejt9IErtBC+VLbWuKgozKBnaJw3Goz792LafbXTwRfuWsX+073kpCbxyE90b6dIoAVBCPir9WVMehTP7WuZ1fm1DW6+98ZJ5uem8t9/vUYX0gTAt19r5LKFubxyrJ3eoXHg4ovRT3ee5nTXMFeU5/OTnaf1vF8iVWXZ/L83TpCaaPNlD81Guz/c2g/AvuYe3VYiQmhBEAJeOtKGsyCdX+xu9h270GL0wsFWJj2KLdc6ERFdSBMAVWXZvNHgZnxS8fzBiy9GtS43n3n2IIk24Rv3rddCOACqnQ6euG89kwqe23dmVpvaD41N8N3XG0m0Cx/RvZ0ihhYEIaCqLJvW3hGOnu3nUEvvtIuRf7ZK/8gEmckJlGSn+ISFLqS5NKqdDra+/zJsAv/zcv1F6zm2uzqZ9Cj+esN8ctOTtBAOkGqng9tXFTM2qagqy7modv+fzx9lcHSST799BR/XvZ0ihhYEIaDa6eBr71sLwD/9fP+0i5EZJPv9wVaeP3CW6op8/unndTpIFgSqKxxcWeGgpXeETYvz3rIY+QvhRLuNCY9i/YIcLYSDQK3Lzav1bspyU3n1eAd/PHT2LeeY8z8+6eHZfWdYVpTJkqIMtta4tCCOEFoQhIibVxZz2YJcjrT2c1WF4y2LkXnDf+yn+xmb9LC9sUsHyYJErcvNwTO9ODKSeOHQWV460nbe66YQfvV4Bz/Y3kRVWTb/+cJRLYQDxL+e47sPvA0R+PBP3qrdm/P/2IvH6RuZ4J1rS86zmLUgDj9aEISIWpebRvcAaYl2ntvfcp5m5B8vGJ0w9i94QGerBAX/Ste3ry5BKXh42x7fYlTrcvtSfR/64W46+kc50TGohXAQ8C8uW1qcybr5OYxOeHjx8DlBbM7/1963lidrGslPT+K7r5/U8x9hAhIEIvIeETkkIh4R2XCB824VkWMi0iAij/odzxORF0Wk3vs7Jnar9l+M/vUdywH40LY9fOqZOr79motHfryXvPQkHvz+TgAuX5Sng2RBwn8xumVVMckJNkYnPPz3H4755r6qLJt9p3t8mwj9zZWL9CIUBPzrOczngtER9puvNPjmv6LQcANNKkXn4JjOFLIAopS69ItFlgMe4FvAPymldk1zjh04jrFVZTOwE7hHKXVYRP4L6FJKfdErIHKVUp+82Odu2LBB7dr1lo+yDFtrXFSVZftu7k/+oo6f7jpNfnoinYPj/K/qRfxmfwudg2OkJdn59gOGDNX74wafPx1t4++e2kVygo3hcQ8PXrWIgswUvvjCUQDuXFvKq/Udet5DxL//5hD/742T5KQm0jM8zoeudfLykXaOt/WjMDagqTmu5z9ciMhupdRblPaABIHfm7/CzILgCuCzSqlbvM8/BaCU+k8ROQZcq5Rq9W5k/4pSaunFPs/qgmAqE5MebnishqbOId8xm0BKgo3v/M3bfF8A02zW/tHg8omf7+OXu99aYPap25bxwc3O83zbejEKLh6P4sGndvHnY+2+Y3YbeDzwL29fxt9frec/nMwkCMIRIygFTvs9b/YeAygy9yn2/p6x45SIPCQiu0RkV0dHR8gGGwr+crKL/pEJPnhNOVkpCVy2IBePgr+7uvy8G18HyYJPrcvNn4928JHrK8hJS+SyBYb38c618/igd651pkro2H6ik/3NPXz4+goyUxJYlJ/GpAfuXDePv79az79VuOjuGyLyElA8zUufVko9O4vPkGmOzdkMUUo9CTwJhkUw1+sjxVRtx5GZxBd+d9S3J+smZ77WgkLE1LnPTE3wzX3N8Q5qXW7f3Fc735rZpQmMqfOfpeffslxUECilbgzwM5qB+X7PywCz90KbiJT4uYba33J1lOMfvKx1ufnmK438y9uXMemB92wo0yZxCNFzH1n0/EcP4YgRJGAEi28AzmAEi+9VSh0SkS8DnX7B4jyl1D9f7POiLUZgMjWIDDouEC703EcWPf/WICTBYhG5C/gfoADoAfYppW4RkXnAd5RSt3vPux34GmAHvqeU+rz3eD7wM2ABcAp4j1Kq62KfG62CQKPRaCJJSLOGwo0WBBqNRjN3Ipk1pNFoNBoLowWBRqPRxDlaEGg0Gk2cowWBRqPRxDlRGSwWkQ6g6RIvdwBW7O6mxzU39Ljmhh7X3LDquCCwsS1UShVMPRiVgiAQRGTXdFHzSKPHNTf0uOaGHtfcsOq4IDRj064hjUajiXO0INBoNJo4Jx4FwZORHsAM6HHNDT2uuaHHNTesOi4IwdjiLkag0Wg0mvOJR4tAo9FoNH5oQaDRaDRxTkwKAhF5j4gcEhGPiGyY8tqnRKRBRI6JyC0zXJ8nIi+KSL33d24IxvhTEdnn/TkpIvtmOO+kiBzwnhfyTnsi8lkROeM3tttnOO9W7xw2eFuIh3pcXxaRoyJSJyK/EpGcGc4Ly3xd7O8Xg697X68TkfWhGovfZ84XkT+LyBHv/f+P05xzrYj0+v1/PxPqcXk/94L/lwjN11K/edgnIn0i8tEp54RlvkTkeyLSLiIH/Y7Nah0KyndRKRVzP8ByYCnwCrDB7/gKYD+QDCwGXIB9muv/C3jU+/hR4EshHu9XgM/M8NpJwBHGufssxt4SFzrH7p27ciDJO6crQjyum4EE7+MvzfQ/Ccd8zebvB24HXsDYoW8TsCMM/7sSYL33cSbGPiBTx3Ut8Ntw3U+z/b9EYr6m+Z+exSi4Cvt8AdcA64GDfscuug4F67sYkxaBUuqIUurYNC/dATytlBpVSp0AGoCNM5z3lPfxU8CdIRkohiYEvBf4Sag+IwRsBBqUUo1KqTHgaYw5CxlKqT8qpSa8T7dj7HQXKWbz998B/EAZbAdyvLvwhQylVKtSao/3cT9whHP7g1udsM/XFG4AXEqpS+1YEBBKqVeBqXuxzGYdCsp3MSYFwQUoBU77PW9m+i9KkVKqFYwvF1AYwjFdDbQppepneF0BfxSR3SLyUAjH4c8jXvP8ezOYo7Odx1Dxtxja43SEY75m8/dHdI5EZBGwDtgxzctXiMh+EXlBRFaGaUgX+79E+p66m5mVsUjMF8xuHQrKvF10z2KrIiIvAcXTvPRppdSzM102zbGQ5c/Ocoz3cGFr4EqlVIuIFAIvishRr/YQknEB3wQ+hzEvn8NwW/3t1LeY5tqA53E28yUinwYmgG0zvE3Q52u6oU5zbOrfH9Z77bwPFskAfgl8VCnVN+XlPRjujwFv/OfXQGUYhnWx/0sk5ysJeBfwqWlejtR8zZagzFvUCgKl1I2XcFkzMN/veRnQMs15bSJSopRq9Zqn7aEYoxj7Ob8buOwC79Hi/d0uIr/CMAUDWthmO3ci8m3gt9O8NNt5DOq4ROQB4B3ADcrrIJ3mPYI+X9Mwm78/JHN0MUQkEUMIbFNKPTP1dX/BoJR6XkS+ISIOpVRIG6zN4v8SkfnychuwRynVNvWFSM2Xl9msQ0GZt3hzDT0H3C0iySKyGEOy/2WG8x7wPn4AmMnCCJQbgaNKqebpXhSRdBHJNB9jBEwPTndusJjil71rhs/bCVSKyGKvNnU3xpyFcly3Ap8E3qWUGprhnHDN12z+/ueAD3izYTYBvaaZHyq88abvAkeUUo/NcE6x9zxEZCPGGtAZ4nHN5v8S9vnyY0arPBLz5cds1qHgfBdDHQ2PxA/GAtYMjAJtwB/8Xvs0RpT9GHCb3/Hv4M0wAvKBl4F67++8EI3z+8CWKcfmAc97H5djZAHsBw5huEhCPXc/BA4Add4bqmTquLzPb8fISnGFaVwNGL7Qfd6frZGcr+n+fmCL+f/EMNmf8L5+AL/stRCO6SoMt0Cd3zzdPmVcj3jnZj9G0L06DOOa9v8S6fnyfm4axsKe7Xcs7POFIYhagXHv2vXgTOtQKL6LusWERqPRxDnx5hrSaDQazRS0INBoNJo4RwsCjUajiXO0INBoNJo4RwsCjUajiXO0INBoNJo4RwsCjUajiXP+f2uo3Ej9/ocQAAAAAElFTkSuQmCC\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"filenames": {
|
||
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/linalg_44_1.png"
|
||
},
|
||
"needs_background": "light"
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"%matplotlib inline\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"from scipy import sparse\n",
|
||
"eye = np.eye(4)\n",
|
||
"print(eye)\n",
|
||
"sparse_mtx = sparse.csr_matrix(eye)\n",
|
||
"print(sparse_mtx)\n",
|
||
"x = np.linspace(-10,10,100)\n",
|
||
"y = np.sin(x)\n",
|
||
"plt.plot(x,y,marker='x')\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ed00bf63",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Other Matrix and Vector Operations\n",
|
||
"\n",
|
||
"The following examples show how to compute various quantities like the **mean** value of a matrix or a vector and how to use functions like **reshape** and **ravel**. These are all useful quantities when scaling the data and preparing the data for various machine learning algorithms and when calculating quantities like the mean squared error or the variance."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"id": "a12e8986",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"The test matrix:[[ 1. 2. 3.]\n",
|
||
" [ 4. 5. 6.]\n",
|
||
" [ 7. 8. 9.]\n",
|
||
" [10. 11. 12.]]\n",
|
||
"This is the total mean summed over all elements:6.5\n",
|
||
"This is the mean for each column:[[5.5 6.5 7.5]]\n",
|
||
"This is the mean value for each row:[[ 2.]\n",
|
||
" [ 5.]\n",
|
||
" [ 8.]\n",
|
||
" [11.]]\n",
|
||
"This is the mean value for each row with keepdims false:[ 2. 5. 8. 11.]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"\"\"\"\n",
|
||
"Simple code that tests various numpy functions\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"# Simple test-matrix of dim 3 x 4\n",
|
||
"a = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9],[10, 11, 12]],dtype=np.float64)\n",
|
||
"print(f\"The test matrix:{a}\")\n",
|
||
"# This is the total mean summed over all elements, which here has to be 6.5\n",
|
||
"print(f\"This is the total mean summed over all elements:{np.mean(a,dtype=np.float64)}\")\n",
|
||
"# This is the mean for each column, it returns an array with the mean values for each column. It returns a row-like vector\n",
|
||
"print(f\"This is the mean for each column:{np.mean(a, axis=0, keepdims=True,dtype=np.float64)}\")\n",
|
||
"# This is the mean value for each row, it returns an array via the keepdims option which is a column-like vector if\n",
|
||
"# keepdims=True. Else it return a row-like vector\n",
|
||
"# Try setting keepdims=False\n",
|
||
"print(f\"This is the mean value for each row:{np.mean(a, axis=1, keepdims=True,dtype=np.float64)}\")\n",
|
||
"# We print then the mean value for each row by setting keepdims=False\n",
|
||
"print(f\"This is the mean value for each row with keepdims false:{np.mean(a, axis=1, keepdims=False,dtype=np.float64)}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "596d78a6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Another useful function is the **ravel** function, which returns a flattened array as shown in the example here."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "fe01c225",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Flatten the matrix:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n",
|
||
"Reshape the matrix to a one-dim array:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n",
|
||
"[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]\n",
|
||
"[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]\n",
|
||
"[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Ravel return a contiguous flattened array.\n",
|
||
"print(f\"Flatten the matrix:{np.ravel(a)}\")\n",
|
||
"# It is the same as reshaping the matrix into a one-dimensional array\n",
|
||
"print(f\"Reshape the matrix to a one-dim array:{a.reshape(-1)}\")\n",
|
||
"# ‘C’ means to index the elements in row-major, C-style order, with the last axis index changing fastest, back to the first axis index changing slowest.\n",
|
||
"# ‘F’ means to index the elements in column-major, Fortran-style order, with the first index changing fastest, and the last index changing slowest \n",
|
||
"print(np.ravel(a, order='F'))\n",
|
||
"# When order is ‘A’, it will preserve the array’s ‘C’ or ‘F’ ordering\n",
|
||
"# ‘A’ means to read the elements in Fortran-like index order if a is Fortran contiguous in memory, C-like order otherwise.\n",
|
||
"# ‘K’ means to read the elements in the order they occur in memory, except for reversing the data when strides are negative. By default, ‘C’ index order is used.\n",
|
||
"# Transposing it\n",
|
||
"print(np.ravel(a.T))\n",
|
||
"print(np.ravel(a.T, order='A'))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "545df59a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Gaussian Elimination\n",
|
||
"\n",
|
||
"We start with the linear set of equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "796d7554",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}\\mathbf{x} = \\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "35a9c235",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We assume also that the matrix $\\mathbf{A}$ is non-singular and that the\n",
|
||
"matrix elements along the diagonal satisfy $a_{ii} \\ne 0$. Simple $4\\times 4 $ example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "432d4f8e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11}& a_{12} &a_{13}& a_{14}\\\\\n",
|
||
" a_{21}& a_{22} &a_{23}& a_{24}\\\\\n",
|
||
" a_{31}& a_{32} &a_{33}& a_{34}\\\\\n",
|
||
" a_{41}& a_{42} &a_{43}& a_{44}\\\\\n",
|
||
" \\end{bmatrix} \\begin{bmatrix}\n",
|
||
" x_1\\\\\n",
|
||
" x_2\\\\\n",
|
||
" x_3 \\\\\n",
|
||
" x_4 \\\\\n",
|
||
" \\end{bmatrix}\n",
|
||
" =\\begin{bmatrix}\n",
|
||
" w_1\\\\\n",
|
||
" w_2\\\\\n",
|
||
" w_3 \\\\\n",
|
||
" w_4\\\\\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c06fa57b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4e8d9bd5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "112c2488",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4774bcd7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5bf16627",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4da1a1c6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The basic idea of Gaussian elimination is to use the first equation to eliminate the first unknown $x_1$\n",
|
||
"from the remaining $n-1$ equations. Then we use the new second equation to eliminate the second unknown\n",
|
||
"$x_2$ from the remaining $n-2$ equations. With $n-1$ such eliminations\n",
|
||
"we obtain a so-called upper triangular set of equations of the form"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "041684ac",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{11}x_1 +b_{12}x_2 +b_{13}x_3 + b_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ba6214cb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{22}x_2 + b_{23}x_3 + b_{24}x_4=y_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2a1d0555",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{33}x_3 + b_{34}x_4=y_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "51570380",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"eq:gaussbacksub\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"b_{44}x_4=y_4. \\nonumber\n",
|
||
"\\label{eq:gaussbacksub} \\tag{1}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e51b3aa8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We can solve this system of equations recursively starting from $x_n$ (in our case $x_4$) and proceed with\n",
|
||
"what is called a backward substitution. \n",
|
||
"\n",
|
||
"This process can be expressed mathematically as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c78b600",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto1\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" x_m = \\frac{1}{b_{mm}}\\left(y_m-\\sum_{k=m+1}^nb_{mk}x_k\\right)\\quad m=n-1,n-2,\\dots,1.\n",
|
||
"\\label{_auto1} \\tag{2}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1cd1015f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To arrive at such an upper triangular system of equations, we start by eliminating\n",
|
||
"the unknown $x_1$ for $j=2,n$. We achieve this by multiplying the first equation by $a_{j1}/a_{11}$ and then subtract\n",
|
||
"the result from the $j$th equation. We assume obviously that $a_{11}\\ne 0$ and that\n",
|
||
"$\\mathbf{A}$ is not singular.\n",
|
||
"\n",
|
||
"Our actual $4\\times 4$ example reads after the first operation"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "34093ce7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11}& a_{12} &a_{13}& a_{14}\\\\\n",
|
||
" 0& (a_{22}-\\frac{a_{21}a_{12}}{a_{11}}) &(a_{23}-\\frac{a_{21}a_{13}}{a_{11}}) & (a_{24}-\\frac{a_{21}a_{14}}{a_{11}})\\\\\n",
|
||
"0& (a_{32}-\\frac{a_{31}a_{12}}{a_{11}})& (a_{33}-\\frac{a_{31}a_{13}}{a_{11}})& (a_{34}-\\frac{a_{31}a_{14}}{a_{11}})\\\\\n",
|
||
"0&(a_{42}-\\frac{a_{41}a_{12}}{a_{11}}) &(a_{43}-\\frac{a_{41}a_{13}}{a_{11}}) & (a_{44}-\\frac{a_{41}a_{14}}{a_{11}}) \\\\\n",
|
||
" \\end{bmatrix} \\begin{bmatrix}\n",
|
||
" x_1\\\\\n",
|
||
" x_2\\\\\n",
|
||
" x_3 \\\\\n",
|
||
" x_4 \\\\\n",
|
||
" \\end{bmatrix} \n",
|
||
" =\\begin{bmatrix}\n",
|
||
" y_1\\\\\n",
|
||
" w_2^{(2)}\\\\\n",
|
||
" w_3^{(2)} \\\\\n",
|
||
" w_4^{(2)}\\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f269239d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d4477133",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{11}x_1 +b_{12}x_2 +b_{13}x_3 + b_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "25bdfc8c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{22}x_2 + a^{(2)}_{23}x_3 + a^{(2)}_{24}x_4=w^{(2)}_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "34b8408c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{32}x_2 + a^{(2)}_{33}x_3 + a^{(2)}_{34}x_4=w^{(2)}_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b8185f8d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{42}x_2 + a^{(2)}_{43}x_3 + a^{(2)}_{44}x_4=w^{(2)}_4, \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4a218e4e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto2\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation} \n",
|
||
"\\label{_auto2} \\tag{3}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ba424207",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The new coefficients are"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "10f17fb1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto3\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" b_{1k} = a_{1k}^{(1)} \\quad k=1,\\dots,n,\n",
|
||
"\\label{_auto3} \\tag{4}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e22879fc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where each $a_{1k}^{(1)}$ is equal to the original $a_{1k}$ element. The other coefficients are"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3b765484",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto4\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
"a_{jk}^{(2)} = a_{jk}^{(1)}-\\frac{a_{j1}^{(1)}a_{1k}^{(1)}}{a_{11}^{(1)}} \\quad j,k=2,\\dots,n,\n",
|
||
"\\label{_auto4} \\tag{5}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9383783c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"with a new right-hand side given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c0a3994f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto5\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
"y_{1}=w_1^{(1)}, \\quad w_j^{(2)} =w_j^{(1)}-\\frac{a_{j1}^{(1)}w_1^{(1)}}{a_{11}^{(1)}} \\quad j=2,\\dots,n.\n",
|
||
"\\label{_auto5} \\tag{6}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b5fab15e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We have also set $w_1^{(1)}=w_1$, the original vector element.\n",
|
||
"We see that the system of unknowns $x_1,\\dots,x_n$ is transformed into an $(n-1)\\times (n-1)$ problem.\n",
|
||
"\n",
|
||
"This step is called forward substitution.\n",
|
||
"Proceeding with these substitutions, we obtain the\n",
|
||
"general expressions for the new coefficients"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4d2d01dd",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto6\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" a_{jk}^{(m+1)} = a_{jk}^{(m)}-\\frac{a_{jm}^{(m)}a_{mk}^{(m)}}{a_{mm}^{(m)}} \\quad j,k=m+1,\\dots,n,\n",
|
||
"\\label{_auto6} \\tag{7}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8c1e5ff9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"with $m=1,\\dots,n-1$ and a\n",
|
||
"right-hand side given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7165e693",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto7\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" w_j^{(m+1)} =w_j^{(m)}-\\frac{a_{jm}^{(m)}w_m^{(m)}}{a_{mm}^{(m)}}\\quad j=m+1,\\dots,n.\n",
|
||
"\\label{_auto7} \\tag{8}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c6646d7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This set of $n-1$ elimations leads us to an equations which is solved by back substitution.\n",
|
||
"If the arithmetics is exact and the matrix $\\mathbf{A}$ is not singular, then the computed answer will be exact.\n",
|
||
"\n",
|
||
"Even though the matrix elements along the diagonal are not zero,\n",
|
||
"numerically small numbers may appear and subsequent divisions may lead to large numbers, which, if added\n",
|
||
"to a small number may yield losses of precision. Suppose for example that our first division in $(a_{22}-a_{21}a_{12}/a_{11})$\n",
|
||
"results in $-10^{-7}$ and that $a_{22}$ is one.\n",
|
||
"one. We are then\n",
|
||
"adding $10^7+1$. With single precision this results in $10^7$.\n",
|
||
"\n",
|
||
" * Gaussian elimination, $O(2/3n^3)$ flops, general matrix\n",
|
||
"\n",
|
||
" * LU decomposition, upper triangular and lower tridiagonal matrices, $O(2/3n^3)$ flops, general matrix. Get easily the inverse, determinant and can solve linear equations with back-substitution only, $O(n^2)$ flops\n",
|
||
"\n",
|
||
" * Cholesky decomposition. Real symmetric or hermitian positive definite matrix, $O(1/3n^3)$ flops.\n",
|
||
"\n",
|
||
" * Tridiagonal linear systems, important for differential equations. Normally positive definite and non-singular. $O(8n)$ flops for symmetric. Special case of banded matrices.\n",
|
||
"\n",
|
||
" * Singular value decomposition\n",
|
||
"\n",
|
||
" * the QR method will be discussed in chapter 7 in connection with eigenvalue systems. $O(4/3n^3)$ flops.\n",
|
||
"\n",
|
||
"The LU decomposition method means that we can rewrite\n",
|
||
"this matrix as the product of two matrices $\\mathbf{L}$ and $\\mathbf{U}$\n",
|
||
"where"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cc3c0cf7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11} & a_{12} & a_{13} & a_{14} \\\\\n",
|
||
" a_{21} & a_{22} & a_{23} & a_{24} \\\\\n",
|
||
" a_{31} & a_{32} & a_{33} & a_{34} \\\\\n",
|
||
" a_{41} & a_{42} & a_{43} & a_{44}\n",
|
||
" \\end{bmatrix}\n",
|
||
" = \\begin{bmatrix}\n",
|
||
" 1 & 0 & 0 & 0 \\\\\n",
|
||
" l_{21} & 1 & 0 & 0 \\\\\n",
|
||
" l_{31} & l_{32} & 1 & 0 \\\\\n",
|
||
" l_{41} & l_{42} & l_{43} & 1\n",
|
||
" \\end{bmatrix}\n",
|
||
" \\begin{bmatrix}\n",
|
||
" u_{11} & u_{12} & u_{13} & u_{14} \\\\\n",
|
||
" 0 & u_{22} & u_{23} & u_{24} \\\\\n",
|
||
" 0 & 0 & u_{33} & u_{34} \\\\\n",
|
||
" 0 & 0 & 0 & u_{44}\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7256c8d3",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"LU decomposition forms the backbone of other algorithms in linear algebra, such as the\n",
|
||
"solution of linear equations given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "634741e9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8a16abbb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "442b2dd1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2be3e530",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8f743387",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The above set of equations is conveniently solved by using LU decomposition as an intermediate step.\n",
|
||
"\n",
|
||
"The matrix $\\mathbf{A}\\in \\mathbb{R}^{n\\times n}$ has an LU factorization if the determinant\n",
|
||
"is different from zero. If the LU factorization exists and $\\mathbf{A}$ is non-singular, then the LU factorization\n",
|
||
"is unique and the determinant is given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0fe06953",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"det\\{\\mathbf{A}\\}=det\\{\\mathbf{LU}\\}= det\\{\\mathbf{L}\\}det\\{\\mathbf{U}\\}=u_{11}u_{22}\\dots u_{nn}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8232172f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"There are at least three main advantages with LU decomposition compared with standard Gaussian elimination:\n",
|
||
"\n",
|
||
" * It is straightforward to compute the determinant of a matrix\n",
|
||
"\n",
|
||
" * If we have to solve sets of linear equations with the same matrix but with different vectors $\\mathbf{y}$, the number of FLOPS is of the order $n^3$.\n",
|
||
"\n",
|
||
" * The inverse is such an operation \n",
|
||
"\n",
|
||
"With the LU decomposition it is rather\n",
|
||
"simple to solve a system of linear equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "87c2e89a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "50fc7a76",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e0379f33",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "11e67b7e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8cda9070",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This can be written in matrix form as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "908f05f7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{Ax}=\\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "53053832",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where $\\mathbf{A}$ and $\\mathbf{w}$ are known and we have to solve for\n",
|
||
"$\\mathbf{x}$. Using the LU dcomposition we write"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f3314fa8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A} \\mathbf{x} \\equiv \\mathbf{L} \\mathbf{U} \\mathbf{x} =\\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2043fafc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The previous equation can be calculated in two steps"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6fff8406",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{L} \\mathbf{y} = \\mathbf{w};\\qquad \\mathbf{Ux}=\\mathbf{y}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9a60901d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To show that this is correct we use to the LU decomposition\n",
|
||
"to rewrite our system of linear equations as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "daacf54d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LUx}=\\mathbf{w},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "da702290",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and since the determinant of $\\mathbf{L}$ is equal to 1 (by construction\n",
|
||
"since the diagonals of $\\mathbf{L}$ equal 1) we can use the inverse of\n",
|
||
"$\\mathbf{L}$ to obtain"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c2a04b00",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{Ux}=\\mathbf{L^{-1}w}=\\mathbf{y},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d1214f7f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"which yields the intermediate step"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e8572956",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{L^{-1}w}=\\mathbf{y}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6920b812",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and as soon as we have $\\mathbf{y}$ we can obtain $\\mathbf{x}$\n",
|
||
"through $\\mathbf{Ux}=\\mathbf{y}$.\n",
|
||
"\n",
|
||
"For our four-dimentional example this takes the form"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "379ca640",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"y_1=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8c753134",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{21}y_1 + y_2=w_2\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "12e32410",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{31}y_1 + l_{32}y_2 + y_3 =w_3\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "01e9e4ae",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{41}y_1 + l_{42}y_2 + l_{43}y_3 + y_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a27d4632",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4dae7cfb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{11}x_1 +u_{12}x_2 +u_{13}x_3 + u_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5241a055",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{22}x_2 + u_{23}x_3 + u_{24}x_4=y_2\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f590f30b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{33}x_3 + u_{34}x_4=y_3\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dbddd25a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{44}x_4=y_4 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8d5dc72b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This example shows the basis for the algorithm\n",
|
||
"needed to solve the set of $n$ linear equations.\n",
|
||
"\n",
|
||
"The algorithm goes as follows\n",
|
||
"\n",
|
||
" * Set up the matrix $\\bf A$ and the vector $\\bf w$ with their correct dimensions. This determines the dimensionality of the unknown vector $\\bf x$.\n",
|
||
"\n",
|
||
" * Then LU decompose the matrix $\\bf A$ through a call to the function `ludcmp(double a, int n, int indx, double &d)`. This functions returns the LU decomposed matrix $\\bf A$, its determinant and the vector indx which keeps track of the number of interchanges of rows. If the determinant is zero, the solution is malconditioned.\n",
|
||
"\n",
|
||
" * Thereafter you call the function `lubksb(double a, int n, int indx, double w)` which uses the LU decomposed matrix $\\bf A$ and the vector $\\bf w$ and returns $\\bf x$ in the same place as $\\bf w$. Upon exit the original content in $\\bf w$ is destroyed. If you wish to keep this information, you should make a backup of it in your calling function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fcf615b7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"### LU Decomposition, the inverse of a matrix\n",
|
||
"\n",
|
||
"If the inverse exists then"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e2be427",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}^{-1}\\mathbf{A}=\\mathbf{I},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "519b78bb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"the identity matrix. With an LU decomposed matrix we can rewrite the last equation as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a730fd5f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\mathbf{A}^{-1}=\\mathbf{I}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2cc4e115",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"If we assume that the first column (that is column 1) of the inverse matrix\n",
|
||
"can be written as a vector with unknown entries"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "03c4a0ae",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}_1^{-1}= \\begin{bmatrix}\n",
|
||
" a_{11}^{-1} \\\\\n",
|
||
" a_{21}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n1}^{-1} \\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "93ef84cc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"then we have a linear set of equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "04c367a7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\begin{bmatrix}\n",
|
||
" a_{11}^{-1} \\\\\n",
|
||
" a_{21}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n1}^{-1} \\\\\n",
|
||
" \\end{bmatrix} =\\begin{bmatrix}\n",
|
||
" 1 \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8f903ad8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"In a similar way we can compute the unknow entries of the second column,"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c343638b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\begin{bmatrix}\n",
|
||
" a_{12}^{-1} \\\\\n",
|
||
" a_{22}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n2}^{-1} \\\\\n",
|
||
" \\end{bmatrix}=\\begin{bmatrix}\n",
|
||
" 0 \\\\\n",
|
||
" 1 \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1c013532",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and continue till we have solved all $n$ sets of linear equations.\n",
|
||
"\n",
|
||
"The calculation of the inverse here assumes that it actually\n",
|
||
"exists. In many machine learning applications there may be strong\n",
|
||
"linear dependencies among the various columns and/or rows. In our\n",
|
||
"discussions of linear regression we will dive into the mathematics of\n",
|
||
"the singular value decomposition, an algorithm which will allow us to calculate the so-called pseudo-inverse.\n",
|
||
"These details will be presented in our linear regression chapter."
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.9.10"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |