Morten Hjorth-Jensen
2134 lines
108 KiB
Plaintext
2134 lines
108 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "43efcac0",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||
"doconce format html linalg.do.txt -->"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "074ac7c2",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"# Linear Algebra, Handling of Arrays and more Python Features"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0be7d59e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Introduction\n",
|
||
"\n",
|
||
"The aim of this set of lectures is to review some central linear algebra algorithms that we will need in our \n",
|
||
"data analysis part and in the construction of Machine Learning algorithms (ML). \n",
|
||
"This will allow us to introduce some central programming features of high-level languages like Python and \n",
|
||
"compiled languages like C++ and/or Fortran. \n",
|
||
"\n",
|
||
"As discussed in the introductory notes, these series of lectures focuses both on using\n",
|
||
"central Python packages like **tensorflow** and **scikit-learn** as well\n",
|
||
"as writing your own codes for some central ML algorithms. The\n",
|
||
"latter can be written in a language of your choice, be it Python, Julia, R,\n",
|
||
"Rust, C++, Fortran etc. In order to avoid confusion however, in these lectures we will limit our\n",
|
||
"attention to Python, C++ and Fortran."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5e77e6c1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Important Matrix and vector handling packages\n",
|
||
"\n",
|
||
"There are several central software packages for linear algebra and eigenvalue problems. Several of the more\n",
|
||
"popular ones have been wrapped into ofter software packages like those from the widely used text **Numerical Recipes**. The original source codes in many of the available packages are often taken from the widely used\n",
|
||
"software package LAPACK, which follows two other popular packages\n",
|
||
"developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly here.\n",
|
||
"\n",
|
||
" * LINPACK: package for linear equations and least square problems.\n",
|
||
"\n",
|
||
" * LAPACK:package for solving symmetric, unsymmetric and generalized eigenvalue problems. From LAPACK's website <http://www.netlib.org> it is possible to download for free all source codes from this library. Both C/C++ and Fortran versions are available.\n",
|
||
"\n",
|
||
" * BLAS (I, II and III): (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. Blas I is vector operations, II vector-matrix operations and III matrix-matrix operations. Highly parallelized and efficient codes, all available for download from <http://www.netlib.org>.\n",
|
||
"\n",
|
||
"When dealing with matrices and vectors a central issue is memory\n",
|
||
"handling and allocation. If our code is written in Python the way we\n",
|
||
"declare these objects and the way they are handled, interpreted and\n",
|
||
"used by say a linear algebra library, requires codes that interface\n",
|
||
"our Python program with such libraries. For Python programmers,\n",
|
||
"**Numpy** is by now the standard Python package for numerical arrays in\n",
|
||
"Python as well as the source of functions which act on these\n",
|
||
"arrays. These functions span from eigenvalue solvers to functions that\n",
|
||
"compute the mean value, variance or the covariance matrix. If you are\n",
|
||
"not familiar with how arrays are handled in say Python or compiled\n",
|
||
"languages like C++ and Fortran, the sections in this chapter may be\n",
|
||
"useful. For C++ programmer, **Armadillo** is widely used library for\n",
|
||
"linear algebra and eigenvalue problems. In addition it offers a\n",
|
||
"convenient way to handle and organize arrays. We discuss this library\n",
|
||
"as well. Before we proceed we believe it may be convenient to repeat some basic features of \n",
|
||
" matrices and vectors."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2eed76da",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Basic Matrix Features\n",
|
||
"\n",
|
||
"Matrix properties reminder"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "109ff37c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A} =\n",
|
||
" \\begin{bmatrix} a_{11} & a_{12} & a_{13} & a_{14} \\\\\n",
|
||
" a_{21} & a_{22} & a_{23} & a_{24} \\\\\n",
|
||
" a_{31} & a_{32} & a_{33} & a_{34} \\\\\n",
|
||
" a_{41} & a_{42} & a_{43} & a_{44}\n",
|
||
" \\end{bmatrix}\\qquad\n",
|
||
"\\mathbf{I} =\n",
|
||
" \\begin{bmatrix} 1 & 0 & 0 & 0 \\\\\n",
|
||
" 0 & 1 & 0 & 0 \\\\\n",
|
||
" 0 & 0 & 1 & 0 \\\\\n",
|
||
" 0 & 0 & 0 & 1\n",
|
||
" \\end{bmatrix}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c87c75d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The inverse of a matrix is defined by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "aab763d1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}^{-1} \\cdot \\mathbf{A} = I\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f382e0f8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<table class=\"dotable\" border=\"1\">\n",
|
||
"<thead>\n",
|
||
"<tr><th align=\"center\"> Relations </th> <th align=\"center\"> Name </th> <th align=\"center\"> matrix elements </th> </tr>\n",
|
||
"</thead>\n",
|
||
"<tbody>\n",
|
||
"<tr><td align=\"center\"> $A = A^{T}$ </td> <td align=\"center\"> symmetric </td> <td align=\"center\"> $a_{ij} = a_{ji}$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = \\left (A^{T} \\right )^{-1}$ </td> <td align=\"center\"> real orthogonal </td> <td align=\"center\"> $\\sum_k a_{ik} a_{jk} = \\sum_k a_{ki} a_{kj} = \\delta_{ij}$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = A^{ * }$ </td> <td align=\"center\"> real matrix </td> <td align=\"center\"> $a_{ij} = a_{ij}^{ * }$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = A^{\\dagger}$ </td> <td align=\"center\"> hermitian </td> <td align=\"center\"> $a_{ij} = a_{ji}^{ * }$ </td> </tr>\n",
|
||
"<tr><td align=\"center\"> $A = \\left (A^{\\dagger} \\right )^{-1}$ </td> <td align=\"center\"> unitary </td> <td align=\"center\"> $\\sum_k a_{ik} a_{jk}^{ * } = \\sum_k a_{ki}^{ * } a_{kj} = \\delta_{ij}$ </td> </tr>\n",
|
||
"</tbody>\n",
|
||
"</table>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5d394b4e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"### Some famous Matrices\n",
|
||
"\n",
|
||
" * Diagonal if $a_{ij}=0$ for $i\\ne j$\n",
|
||
"\n",
|
||
" * Upper triangular if $a_{ij}=0$ for $i > j$\n",
|
||
"\n",
|
||
" * Lower triangular if $a_{ij}=0$ for $i < j$\n",
|
||
"\n",
|
||
" * Upper Hessenberg if $a_{ij}=0$ for $i > j+1$\n",
|
||
"\n",
|
||
" * Lower Hessenberg if $a_{ij}=0$ for $i < j+1$\n",
|
||
"\n",
|
||
" * Tridiagonal if $a_{ij}=0$ for $|i -j| > 1$\n",
|
||
"\n",
|
||
" * Lower banded with bandwidth $p$: $a_{ij}=0$ for $i > j+p$\n",
|
||
"\n",
|
||
" * Upper banded with bandwidth $p$: $a_{ij}=0$ for $i < j+p$\n",
|
||
"\n",
|
||
" * Banded, block upper triangular, block lower triangular....\n",
|
||
"\n",
|
||
"Some Equivalent Statements. For an $N\\times N$ matrix $\\mathbf{A}$ the following properties are all equivalent\n",
|
||
"\n",
|
||
" * If the inverse of $\\mathbf{A}$ exists, $\\mathbf{A}$ is nonsingular.\n",
|
||
"\n",
|
||
" * The equation $\\mathbf{Ax}=0$ implies $\\mathbf{x}=0$.\n",
|
||
"\n",
|
||
" * The rows of $\\mathbf{A}$ form a basis of $R^N$.\n",
|
||
"\n",
|
||
" * The columns of $\\mathbf{A}$ form a basis of $R^N$.\n",
|
||
"\n",
|
||
" * $\\mathbf{A}$ is a product of elementary matrices.\n",
|
||
"\n",
|
||
" * $0$ is not eigenvalue of $\\mathbf{A}$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0a85f2b2",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Numpy and arrays\n",
|
||
"[Numpy](http://www.numpy.org/) provides an easy way to handle arrays in Python. The standard way to import this library is as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "7043c92e",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[-0.08059005 0.26043697 0.54190252 -0.8321864 1.74960664 0.28855565\n",
|
||
" 1.03029311 -0.54136139 0.94583038 0.99378218]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"x = np.random.normal(size=n)\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2b5abf29",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have defined a vector $x$ with $n=10$ elements with its values given by the Normal distribution $N(0,1)$.\n",
|
||
"Another alternative is to declare a vector as follows"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "4193cb75",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1 2 3]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.array([1, 2, 3])\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b112c576",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have defined a vector with three elements, with $x_0=1$, $x_1=2$ and $x_2=3$. Note that both Python and C++\n",
|
||
"start numbering array elements from $0$ and on. This means that a vector with $n$ elements has a sequence of entities $x_0, x_1, x_2, \\dots, x_{n-1}$. We could also let (recommended) Numpy to compute the logarithms of a specific array as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "18167a1f",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4, 7, 8]))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ee482a57",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Here we have used Numpy's unary function $np.log$. This function is\n",
|
||
"highly tuned to compute array elements since the code is vectorized\n",
|
||
"and does not require looping. We normaly recommend that you use the\n",
|
||
"Numpy intrinsic functions instead of the corresponding **log** function\n",
|
||
"from Python's **math** module. The looping is done explicitely by the\n",
|
||
"**np.log** function. The alternative, and slower way to compute the\n",
|
||
"logarithms of a vector would be to write"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "d8305632",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1 1 2]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"from math import log\n",
|
||
"x = np.array([4, 7, 8])\n",
|
||
"for i in range(0, len(x)):\n",
|
||
" x[i] = log(x[i])\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a50fea8e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We note that our code is much longer already and we need to import the **log** function from the **math** module. \n",
|
||
"The attentive reader will also notice that the output is $[1, 1, 2]$. Python interprets automacally our numbers as integers (like the **automatic** keyword in C++). To change this we could define our array elements to be double precision numbers as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "e9263103",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4, 7, 8], dtype = np.float64))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d68631ad",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or simply write them as double precision numbers (Python uses 64 bits as default for floating point type variables), that is"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "074cfbda",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.94591015 2.07944154]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
|
||
"print(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dab5cce4",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To check the number of bytes (remember that one byte contains eight bits for double precision variables), you can use simple use the **itemsize** functionality (the array $x$ is actually an object which inherits the functionalities defined in Numpy) as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "9d335488",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"8\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"x = np.log(np.array([4.0, 7.0, 8.0]))\n",
|
||
"print(x.itemsize)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "64c9f3e5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Having defined vectors, we are now ready to try out matrices. We can define a $3 \\times 3 $ real matrix $\\hat{A}$\n",
|
||
"as (recall that we user lowercase letters for vectors and uppercase letters for matrices)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "a3b32334",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1.38629436 1.94591015 2.07944154]\n",
|
||
" [1.09861229 2.30258509 2.39789527]\n",
|
||
" [1.38629436 1.60943791 1.94591015]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4ff9ce6b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"If we use the **shape** function we would get $(3, 3)$ as output, that is verifying that our matrix is a $3\\times 3$ matrix. We can slice the matrix and print for example the first column (Python organized matrix elements in a row-major order, see below) as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "d19f85e2",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.38629436 1.09861229 1.38629436]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"# print the first column, row-major order and elements start with 0\n",
|
||
"print(A[:,0])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4e7e8796",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We can continue this was by printing out other columns or rows. The example here prints out the second column"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "4733c8b7",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[1.09861229 2.30258509 2.39789527]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"A = np.log(np.array([ [4.0, 7.0, 8.0], [3.0, 10.0, 11.0], [4.0, 5.0, 7.0] ]))\n",
|
||
"# print the first column, row-major order and elements start with 0\n",
|
||
"print(A[1,:])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "df9f205b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Numpy contains many other functionalities that allow us to slice, subdivide etc etc arrays. We strongly recommend that you look up the [Numpy website for more details](http://www.numpy.org/). Useful functions when defining a matrix are the **np.zeros** function which declares a matrix of a given dimension and sets all elements to zero"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "19757d00",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
|
||
" [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to zero\n",
|
||
"A = np.zeros( (n, n) )\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f1911274",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or initializing all elements to"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "4f737773",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]\n",
|
||
" [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to one\n",
|
||
"A = np.ones( (n, n) )\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cd241572",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or as unitarily distributed random numbers (see the material on random number generators in the statistics part)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "54645585",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[0.77235007 0.27529208 0.31199504 0.17293829 0.82162246 0.13378194\n",
|
||
" 0.90679215 0.39664124 0.3121824 0.13861839]\n",
|
||
" [0.69473608 0.72612916 0.4570065 0.41275555 0.76067335 0.56239325\n",
|
||
" 0.33900003 0.83105136 0.14230327 0.04713857]\n",
|
||
" [0.62377027 0.12392385 0.7500676 0.67969567 0.15971479 0.97072608\n",
|
||
" 0.00183119 0.95291169 0.59353543 0.03550103]\n",
|
||
" [0.99152919 0.13537597 0.88366546 0.73118203 0.82120582 0.53939154\n",
|
||
" 0.01958776 0.59647764 0.17941609 0.34647125]\n",
|
||
" [0.43263402 0.2754374 0.59137018 0.52019078 0.71121535 0.60648493\n",
|
||
" 0.94665557 0.66298436 0.22615136 0.29639686]\n",
|
||
" [0.84424529 0.59603845 0.9219476 0.44909201 0.67715931 0.18908167\n",
|
||
" 0.76516101 0.38007856 0.83478186 0.75271427]\n",
|
||
" [0.53862422 0.11323706 0.15316333 0.34540564 0.81994631 0.52292446\n",
|
||
" 0.26760957 0.02430273 0.03576146 0.67801091]\n",
|
||
" [0.52928925 0.14990609 0.86292532 0.43014974 0.83844809 0.04560463\n",
|
||
" 0.84163178 0.80868063 0.8371938 0.39611129]\n",
|
||
" [0.12607006 0.5113303 0.63901709 0.99976659 0.34756595 0.28622513\n",
|
||
" 0.89290901 0.84314251 0.31916189 0.29920799]\n",
|
||
" [0.23476091 0.40074419 0.21933245 0.48906993 0.19899282 0.06752501\n",
|
||
" 0.85079729 0.64275853 0.33164051 0.08304321]]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"n = 10\n",
|
||
"# define a matrix of dimension 10 x 10 and set all elements to random numbers with x \\in [0, 1]\n",
|
||
"A = np.random.rand(n, n)\n",
|
||
"print(A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e993bf6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"As we will see throughout these lectures, there are several extremely useful functionalities in Numpy.\n",
|
||
"As an example, consider the discussion of the covariance matrix. Suppose we have defined three vectors\n",
|
||
"$\\hat{x}, \\hat{y}, \\hat{z}$ with $n$ elements each. The covariance matrix is defined as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4fe66190",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\hat{\\Sigma} = \\begin{bmatrix} \\sigma_{xx} & \\sigma_{xy} & \\sigma_{xz} \\\\\n",
|
||
" \\sigma_{yx} & \\sigma_{yy} & \\sigma_{yz} \\\\\n",
|
||
" \\sigma_{zx} & \\sigma_{zy} & \\sigma_{zz} \n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cdd238d5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where for example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "41663b63",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\sigma_{xy} =\\frac{1}{n} \\sum_{i=0}^{n-1}(x_i- \\overline{x})(y_i- \\overline{y}).\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e7bd73e9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The Numpy function **np.cov** calculates the covariance elements using the factor $1/(n-1)$ instead of $1/n$ since it assumes we do not have the exact mean values. For a more in-depth discussion of the covariance and covariance matrix and its meaning, we refer you to the lectures on statistics. \n",
|
||
"The following simple function uses the **np.vstack** function which takes each vector of dimension $1\\times n$ and produces a $ 3\\times n$ matrix $\\hat{W}$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c4d0878",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\hat{W} = \\begin{bmatrix} x_0 & y_0 & z_0 \\\\\n",
|
||
" x_1 & y_1 & z_1 \\\\\n",
|
||
" x_2 & y_2 & z_2 \\\\\n",
|
||
" \\dots & \\dots & \\dots \\\\\n",
|
||
" x_{n-2} & y_{n-2} & z_{n-2} \\\\\n",
|
||
" x_{n-1} & y_{n-1} & z_{n-1}\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "32eb00e7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"which in turn is converted into into the $3 times 3$ covariance matrix\n",
|
||
"$\\hat{\\Sigma}$ via the Numpy function **np.cov()**. In our review of\n",
|
||
"statistical functions and quantities we will discuss more about the\n",
|
||
"meaning of the covariance matrix. Here we note that we can calculate\n",
|
||
"the mean value of each set of samples $\\hat{x}$ etc using the Numpy\n",
|
||
"function **np.mean(x)**. We can also extract the eigenvalues of the\n",
|
||
"covariance matrix through the **np.linalg.eig()** function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "6b91d50d",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"-0.04071979724729911\n",
|
||
"4.064820972167253\n",
|
||
"-0.3254718508870977\n",
|
||
"[[0.88727586 2.57584621 2.19767225]\n",
|
||
" [2.57584621 8.44132765 6.34964801]\n",
|
||
" [2.19767225 6.34964801 9.99322469]]\n",
|
||
"[16.34233281 0.08212093 2.89737445]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Importing various packages\n",
|
||
"import numpy as np\n",
|
||
"\n",
|
||
"n = 100\n",
|
||
"x = np.random.normal(size=n)\n",
|
||
"print(np.mean(x))\n",
|
||
"y = 4+3*x+np.random.normal(size=n)\n",
|
||
"print(np.mean(y))\n",
|
||
"z = x**3+np.random.normal(size=n)\n",
|
||
"print(np.mean(z))\n",
|
||
"W = np.vstack((x, y, z))\n",
|
||
"Sigma = np.cov(W)\n",
|
||
"print(Sigma)\n",
|
||
"Eigvals, Eigvecs = np.linalg.eig(Sigma)\n",
|
||
"print(Eigvals)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "6a0aa964",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[[1. 0. 0. 0.]\n",
|
||
" [0. 1. 0. 0.]\n",
|
||
" [0. 0. 1. 0.]\n",
|
||
" [0. 0. 0. 1.]]\n",
|
||
" (0, 0)\t1.0\n",
|
||
" (1, 1)\t1.0\n",
|
||
" (2, 2)\t1.0\n",
|
||
" (3, 3)\t1.0\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAACUyklEQVR4nO29eXgb1b3//5ZkSba870viPc5GQlbiJkBZ7CxAb6FwKaFpKdw0tJTUkHAppL9CCrSEUhqDKW2gZesXKJReStkaSBwChaxkgSRkkR3v+y5bsrXO74/RGY28L1pmRp/X8/iJIo1GZ6zjOe/zWVUcx3EgCIIgCIJQEOpgD4AgCIIgCMLXkMAhCIIgCEJxkMAhCIIgCEJxkMAhCIIgCEJxkMAhCIIgCEJxkMAhCIIgCEJxkMAhCIIgCEJxkMAhCIIgCEJxhAV7AMHA5XKhsbER0dHRUKlUwR4OQRAEQRDjgOM49Pb2IiMjA2r16DaakBQ4jY2NyMzMDPYwCIIgCIKYBHV1dZg+ffqox4SkwImOjgbA/4JiYmKCPBqCIAiCIMaDyWRCZmamsI6PRkgKHOaWiomJIYFDEARBEDJjPOElFGRMEARBEITiIIFDEARBEITiIIFDEARBEITiIIFDEARBEITiIIFDEARBEITiIIFDEARBEITiIIFDEARBEITiIIFDEARBEITiIIFDEARBEITi8KvA+fTTT/Ff//VfyMjIgEqlwttvvz3me/bu3YvFixdDr9djxowZeOmll4Yc88wzzyAnJwfh4eEoLCzEoUOHfD94IuQp3XUOZeXGYV8rKzeidNe5AI+IIAiCGC9+FThmsxkLFizAM888M67jq6qqcM011+CKK67A8ePHcffdd+NHP/oRPvzwQ+GYN954A5s3b8bWrVtx9OhRLFiwAKtXr0Zra6u/LoMIUTRqFbYPI3LKyo3YvuscNGrqRE94Q6KYmCg0Z/yHX3tRXXXVVbjqqqvGffyOHTuQm5uL3//+9wCAOXPm4LPPPkNpaSlWr14NANi+fTs2bNiA2267TXjP+++/jxdeeAH333+/7y+CCFlKigoAANvdN5iSogJB3GxeOVN4nSAYTBQDnvlTuuscDld3Yl9lBzavnOl1fFm5EU4Xh02DnidCB5oz/kNSzTb379+P4uJir+dWr16Nu+++GwBgs9lw5MgRbNmyRXhdrVajuLgY+/fvH/G8VqsVVqtV+L/JZPLtwAnFIhY5T7lvLCvyE4cVN3TjIcTzxTRgx6LMeOyvbMeh6q4h80YslonQRTxnOI7DdxZNx8GqDhw430lzZopIKsi4ubkZqampXs+lpqbCZDKhv78f7e3tcDqdwx7T3Nw84nm3bduG2NhY4SczM9Mv4yeUSXQ4vw9wujgAwL7KDjy+84zXMeS2IhglRQXYcGku/vKfKtz52lEcqu5CXITWa96QJZAQU1JUgM0rZ6J0txHf/N3HOHC+ExFaDfZVduCW5w+iut1Mc2YSSMqC4y+2bNmCzZs3C/83mUwkcoghlLoFivjm8df91Xjo3a+HHPvHvZU4WtOF52+9CM9/VkU3HkKgd8CO/xjbvZ7r7rcD4OfNs5+ch5PjaL4QXmQnGrz+3293AgA+Nbbj8if2AgDNmQkiKYGTlpaGlpYWr+daWloQExODiIgIaDQaaDSaYY9JS0sb8bx6vR56vd4vYyaUw2Bf+P/bX40H/3VKeP3u4gKca+nDByeaAAAHqjox/1cfwsXRjSdUGSyKHU4XNr52DGeaewEAahXg4oAV+Yno6LPhbEsvnByHsEFCmghtKlp78b9vfgnAM2euXZiBGclRvOsKgEoFmjMTRFIuquXLl6O8vNzruV27dmH58uUAAJ1OhyVLlngd43K5UF5eLhxDEJOFmYm37zqH9S8fxgMicbOpuAB3F8/EH9ctxqZiz03GxYEWqxBGnGnHcRy2vnMKn5xrE16/u3gmNq+ciX2VHUiM0gnPO1wcntpN2TGhyOCsKYvNge8+ewB2J4eY8DBsvGIGNq+ciX8db8T+8x3g3MdxHHD3G8eCM2iZ4lcLTl9fHyoqKoT/V1VV4fjx40hISEBWVha2bNmChoYG/PWvfwUA/OQnP8Ef/vAH/PznP8f//M//YM+ePfj73/+O999/XzjH5s2b8cMf/hBLly7FsmXL8OSTT8JsNgtZVQQxFUqKClDR2od3vmwUnttUXIC7ij1BfXcVz8TBKj7DAeAXq7JyI4mcEEQcIPpFdSc+FbmmxFa9A+c7sK+yA4uz4nCioQd2J4fS3UaoVCSOQw2xpfhnV87ADX/ah06zDVqNCqYBB8I0apQUFQhzZkV+IuZNi8Vzn57H28cakRVvwOZVs4J8FfLArwLniy++wBVXXCH8n8XB/PCHP8RLL72EpqYm1NbWCq/n5ubi/fffx6ZNm/DUU09h+vTp+Mtf/iKkiAPATTfdhLa2Njz44INobm7GwoULsXPnziGBxwQxWWo6zMJjnUbtJW4APkB0X2UHbl2Rg7/ur4aLw5A0TyJ0KCkqQGN3P14/XCc8JxY3bL6syE/ERTkJWJQVj+c/q0JSlI7mTQgiFsVf1XfjdBPvzrQ7PXFZg+fMhm/m4dUDNTDbnCjbUyGIIAZlcA6PXwXO5ZdfDo7jRnx9uCrFl19+OY4dG90Mt3HjRmzcuHGqwyOIIZxs6MGX9T0AAK1GBZvT5WWdGZzJYBqw462jDchNNNBiFcJYHS7hsW7Q4uN0eQcUb/vgNPRharT32bBqbqqQnQfQQhUqlBQVwGx14NlPzwvPiefI4DkDAJfNTMYHJ5uhVgFdFpvwPKWOj4ykYnAIIthseesEAGBWWjSMv7laiMlhPvPBN56SKwugUatQ1WHB2mWZXosVERqYBux41+3SDFN7RDFj06CFKlIfJgiir+p7cMfl+QCo1ECoIY7JGiyKB88ZAHhm3WJMj4uAiwP+447zotTx0ZFUFhVBBJPffXgGJxp4683D374AwNBqxoN31v881oDZadE41WhCQ1c/Hrv+QuE12o2HBpvfOA6Hi0NipA5f/LIYT++pGNWaV1JUAIfThbI9FWg2DeD5z6rgdHG0UIUYz7mtNxr1UEvxcKhUKjx/60VY89SnqGgzY8YvPoBjGEsP4YEEDkG4+bqRr3A9IyUKy3IThOfFZuPBaNQqnGo0Qa0C/mNsx+HqTlyUk0Bm4xChrNyI3af5Pnh3XJ7vFTQ8msjZvGoWjK19+PfJZvzuw7P8c7RQhQwPvH0S7X02aFQqHP3lSry8v3pcLu5ZadG4dUUOXvy8Gg4XN8TyQ3hDAocgAHAch2YT387je8uyoFJ5uwlGuokMXsxKd53DN/ISaTceIrT2DgDgXVPXLZomPD+aKGY8873FyP/FB+AAaCibKmQoKzfi/x2oAQCsnJuKWIN2XKKYIY5rHY/lJ5QhgUMQAI7VdeN0kwn6MDVuWDx9Qu8tKSqAqd+Ov3xWhX2VHUKDPLrpKJ8IrQYAcOXsFCRFeRcTHev7/8PHFUKNEydHpQZCBYfThUi9BmarE99ZPDFRXFZuxEv7apAQqUOn2Yar5qVRcsMoUJAxEZIMLrb16gG+XMG3LszAy/urUbprYkXYfvmtuWBGH9qNhwZ2pwtvHW0AAHx36cRavzAX5u2X5gnPbR80JwllclFuAsxWJ+IMWlwxK8XrtZKighFj9sRu73WFWQAAl7vlB82d4SGBQ4Qk4gq0PRY73vuKz4LRhqkmlcnCV7LlH7PdOKFs9pxpRYfZhuRoPS6flTzu94kXql9cMwcXTo8FABTPSaGFKgT4p1sUf+vCdOjCxr8EizM4V1/Atyb65FwbfnRpLjavnEkZnMNALioiJBH7vI/WdMHqcCEpSofXD9VN2L3EFqwffzNPqGtBZmPlMbjv1Jtf8IX9rl88DX/cWznujLnBpQZWzU3FV/U9Qk8zWqiUi8XmwM5TzQCA7yyamCtcPLcuyIhBZkIE6jr78cnZNrrPjABZcIiQpaSoAJuKC7DXXVOivc82aXGzeeVMbLl6DhZkxgEAVtJuXHGIrX6tpgF8fJafN1a7a0JWv8E1TlbO5Xfjn1W0Y/0luVRWQEEMdoV/eKoZFpsTOYkGfGZsm7ArnKFSqXDVvHQAwL9PNvtkrEqEBA4R0qxx3yQAvnLxRHdCg3fjRbPdPnWVinbjCkPcjPWeN7+E08UhIzYcL+2rnlJQ+czUKGQlGGBzuPAfY9vYbyBkg1gUA8A/j/Gu8PTYcJTuNk6pqOOaebww3nOmFVaHc+qDVSDkoiJCmm0fnAYAqFR8L5iJZrIM3m1fOZu33HxmbMfTNy9CuDvLhlAGg9N5G3sGppwxp1KpsHJuKp7/rAofnWrxEt2EvBHPF7PVgc/cAnb/+c4pz5uF0+OQGqNHi8mKzyvaceVs6sc4GLLgECFLWblRcE/dt2a2T7IRLsiIQWqMHv12Jw6c7/DVUAkJcdNFnoypyVj9hmPlXH5x2nO2FQ6na4yjCTnBLH/PfnoezKDrizISarUKa9zBxv8+QW6q4SCBQ4QkLHZGp+H/BC7OT/JyQUxW5KhUKmEn9fGZVp+Nl5AOv3rnFABABY/Vb6oszY5HnEGLbosdh6u7pnw+QlqUFBWAOaPEgepToXTXOVhsvGtq1+kW2EXCuKzcOOn4HiVBAocISZwuDmuXZcLmdCEmPAxzM2IAeHZbU4mdYXE45WdavaqOEvKnrNwoBHWy9FxfBJOHadQocgvjXV+3THmchLTY/tFZT1FHl29EsUatwptH6hGh1aDbYsehqk4A1LRVDMXgECHJppUz8Yc9/E1meX6i181gqruri2ckQR+mRn1XP4ytfZiZGj2l8xHSgC0csRFa9PTbsTw/UbDWTbUsQOmuc7C5O4zvOt2MB741R2gXQk1b5U1ZuRFleyoAACnReqwrzPJJGYnB8WD/PtmEIzVd1CZGBFlwiJDl8wo+RubiGUk+PW+EToMV+YkAgPLT5KZSCk4Xhw2X5qKn3w61Cliawzdk9YXVT6NW4d2vGqFRq1DX2Y+zLb0AaDcud9j3tzyPnyuFeYm4q3imzyx/JUUF+I67B9orB2pJ3AyCBA4RkgzYnThSy8c6rMj3rcAp3XVO2H3vOePtbiDfuHzZtHIm5qTzrsz502IRE64VXhutxP54GCySdp1q8aqxRAuWPGFlJOCOwCnM9Z0oZvz2hguFx2E+iu9RCiRwiJDkSE0XbA4XUmP0yE+O9Om5NWoV9rgDjI/UdKHLbANAu3ElsL+St/p9w22h8yUlRQVYOYeP39q+6xyJGwWwaeVM/OSyfBx1b6aYwAGmLooZOz6pFB47fBTfoxRI4BAhyecV7QD47ClmbfEVbHcGAC6O7xdDu3FlsN+d+v+NPN8LHAD4zfXzAQAcfJeCTgSXr+q7YXW4kBipw4yUKJ+em91XmEt8Vlo0VVAXQQKHCEk+d+/EV/g4/oZRUlSAZbnxAIDNfz9O4kYB1HVaUN/VD41ahYtyEsZ+wyR4/VCd8NhXKehEcDnozm5alpvg082UeNO08coZAIC+AQd1FxdBAocIOXr67ThR3w0AuHiGf3biALB55SwAvBVHp1GTuJE5rHDjhdNjEaX3fQIqW7Bmp/FZd8vzEmihUgBigeNLxG1iFkyPg0atQkN3P25cOp3axLihNHEi5Dh4vgMuDshLikR6bITfPoe5wQDA5nRNuA0EIS2Ye2q5H9xT4t14nEGLB/91CtowjbAbB6gzvRxxOF04Us0LnMJc384bcfxOpD4Mc9KjcbLBhCM1XTRX3JAFhwg59gnuKf9Zb8rKjXh6TwWSo/QAgP+6MJ124zKG4zgcqPRf/I14N744i3dtHqvtwsYrZtBuXMacajTBbHMiJjxMsMz5i6XZvIXoC6qELUAWHELxlLozl9iuhllWVuQn+aWImng33mwawGsHa5ERF0G7cRlT19mPxp4BaDUqLM2J9/n5xfNvVlo0wrVq9A44cL69j+aKjDlYxYviZbkJUPs5e3Jxdjxe2lctZGwRZMEhQgCNWiVYT1pNAzC29kGlAk7U9/glbVu8G1+YGQcAOFbb7dPaF0Rg2X+eF8ULpsfBoPPvvlCrUePC6XEAgKM13X79LMK/sPYJvnZPDceSbF54n2o0wWJz+P3z5ABZcAjFIy5pfqbJBABIitLjT59U+iWzSbwbX+QWOCcaeuBwumg3LiPElr/9g9xT/m6fsDgrHoeqOnG0tgvfFXUvJ+SD08UJAsfXAcbDkREbjrSYcDSbBvBlXQ+W+6FWk9wgCw4REjDryQfuRoltvdaApG3nJ0chWh+GfrtTKL9PyANm+Xtq9zkcOM8vVMvzEwNSsHFxVhwA3vJHyJMzzSaYBhyI0ofhAnczX3+iUqmwxO0+JTcVDwkcImQoKSoAW5ICVdJcrVZhgduKc7yu2++fR/gOJopLdxvRbBqATqPGgfMdAalptMgdaHyutRemAbvfPofwLaWiRAJmvVmSHY8wjTogbVqWuOfNkRoSOAAJHCKE2P7RWbDol0CWNGdxOMdpNy47SooKUOxun2B3uvD0noqAWP6So/XITIgAxwFfkjCWDeJ4v4PnPe6pQLVpYXE4R2q64KJYP4rBIUKDsnIjyvZUAAASI3W4ZXl2wDKahEBjWqhkSXJ0OAC+fUIgCzYuzopHXWc/jtZ049KC5IB8JjE1xPF+EVoNAL4C9uuH6wIijOdmxCBcq0ZPvx3n2/swI8W/qelShyw4hOJhu6crZ/M78QumxeKu4pkBK2m+0B1PUdnWR+4GGcI6woepVULBxkCwSBDG5G6QEyVFBbh1RTb67U4ACJi4AfgMvAXuDDxyUwVI4DzzzDPIyclBeHg4CgsLcejQoRGPvfzyy6FSqYb8XHPNNcIxt95665DX16xZE4hLIWQIS9tmRffmT+MD/gKVtp0U5XE3fFXX49fPInxL6a5zaDFZAQDl91wW0D4/i7NZwb9ucjfIjHnT4oTHgW7TwtxUVPAvAC6qN954A5s3b8aOHTtQWFiIJ598EqtXr8bZs2eRkpIy5Pi33noLNptN+H9HRwcWLFiAG2+80eu4NWvW4MUXXxT+r9fr/XcRhKxhqbxXP/UfAMD8abHCa4G68SzM5N0Nx+u6cEmBfxp8Er6lrNyIp9xCJjo8DFkJBi8XBODf+TMnPQb6MOZuMPu8EzXhP147WAMAUKsC26aldNc5NPb0AwCODMqk8ndpAynidwvO9u3bsWHDBtx2222YO3cuduzYAYPBgBdeeGHY4xMSEpCWlib87Nq1CwaDYYjA0ev1XsfFx/u+uiihHAbsTpxzp2nPEwmcQCEu+EfIA6eLw6q5qQCAuekxQifoQFn++IJ//Fw9Rmm/sqGs3Iij7r/zx/97QUCtfhq1Cm8fawQAnG8zo9NsE8YUiCBnqeFXgWOz2XDkyBEUFxd7PlCtRnFxMfbv3z+uczz//PNYu3YtIiMjvZ7fu3cvUlJSMGvWLNxxxx3o6OgY8RxWqxUmk8nrhwgtzjb3wuHiEG/QYlqc/xpsjsRCUao4x5G7QQ5sWjkTGe65MlgUlxQVBGQnzPpSHSVhLAuYkNCF8UvrBRkxgiAOhMhhn8U4WtPl1Tom1AqN+tVF1d7eDqfTidTUVK/nU1NTcebMmTHff+jQIZw8eRLPP/+81/Nr1qzB9ddfj9zcXFRWVuIXv/gFrrrqKuzfvx8ajWbIebZt24aHHnpoahdDyJoTDXzsy7xpscJOPJBckBEDrUaFDrMN9V39yEwwBHwMxMQ5Kcwb/xdqE8OqKC8SNd5khKKrQS44XRzWX5yL5z+vgk6jFtyKTFgEok1LSVEBPjrVjJONJtz+/76Ai0NIihtA4llUzz//PObPn49ly5Z5Pb927Vp8+9vfxvz583Hdddfhvffew+HDh7F3795hz7Nlyxb09PQIP3V1dQEYPSElTjXyC9X8ILinACBcq8HcdH6RpHRxeeB0cfja3dpjXkZg5w2rp8KEzdmWXvRZHSHrapALm1bOxEW5vCidmRYFrcazxAbK6gcAP1ieDQBwcYEPcpYSfhU4SUlJ0Gg0aGlp8Xq+paUFaWlpo77XbDbj9ddfx/r168f8nLy8PCQlJaGiomLY1/V6PWJiYrx+iNBCbMEJNKy6qScOx3s37u/qpsTkqO4ww2JzIlyrRl5yYAN8mavh2U/PIzo8DBwH/OqdUyHrapATpxp5UXxBenA2UwDwVb0nWzOQpQ2khl8Fjk6nw5IlS1BeXi4853K5UF5ejuXLl4/63jfffBNWqxXf//73x/yc+vp6dHR0ID09fcpjJpSH1eHE2WY+wDgYFhy2G2/t5dONWcsG2o1LG+aempMeE5TviImc3gG+M/Q/jtSTuJEBgsAJsFuTUVZuxKsHa8E88bdfmhewIGep4XcX1ebNm/HnP/8ZL7/8Mk6fPo077rgDZrMZt912GwDglltuwZYtW4a87/nnn8d1112HxETvjqh9fX249957ceDAAVRXV6O8vBzXXnstZsyYgdWrV/v7cggZcq65D3Ynh9gILabHBz7AmC1U/3Y3+jzVaELprnO0G5c4bKEKtHtKTElRgSCu1KrAlTUgJg9zhzOXdCARBxTPcFsdl89IDGgml5Twex2cm266CW1tbXjwwQfR3NyMhQsXYufOnULgcW1tLdRqb5119uxZfPbZZ/joo4+GnE+j0eCrr77Cyy+/jO7ubmRkZGDVqlV45JFHqBYOMSzMPTU/SAHGAL8wcRyH0t1G2BwuPFVuJHEjcYIVYCyGBRQDfDxFoOqpEJOjvc+KFpMVKhVv+Qs0rKhpSVEBjK19MLb24XSTKaBBzlIiIL2oNm7ciI0bNw772nCBwbNmzRoxlTYiIgIffvihL4dHKJxgxt+Iuat4Jp4sN4LjeLcVLVTSheM4j6shSBYcthv/8Tfz8Oyn56FCYAoMEpPna/ecyU2MRKQ+8K0exUHMc9Kj8e6XwJkm3j0finNG0llUBOELTjYEN4OKUeYWNwC/kwo1c7GcqO/qR0+/HVqNCjNTA9+wUOxquP+q2UiI1IEDsK4wKyRdDXKBieK5GcFPZJmTxo/hdFPo1n0jgUMoGpvDFdQAYwZbsNZcwGcPTo+LoIVKwrA4ipmp0ULRtkAidjWoVCrMTuNF1sLMuIBUUSYmhxB/IwWB43aRnW83Y8Dd+DPUCLwNjSACyLmWXticLsSEhyEzIfABxoD3bvzK2SnYeaoZfTYHNhUXkMtBopxsCG6A8eB6KbPSorGvsgNnmnvxwLfmBmVMxNiwuknBcmuKSY3RI86gRbfFjorWvqC76IMBCRxC0ZwMcgVjwHs3PmB3Qq0Cui12rF2WBZVKRbtxCcJ24sEMMBbD3A1nmkPX3SB1zFYHqtrNAPjK5cGGWf4OnO/E6SYTCRyCUAKszH1JUYFXBhUQnDL34s8K12qQkxSJ821mnGnuJcuNRDkp1DKRxqIwO513UTF3KyE9zjSbwHG85SQpShoZvXPSY9wCJzTnDcXgEIqDFdYrKzd6WXCkUliPxVOcpd24JGk1DaCt1wq1ymM5CTYFKdFQqYD2Phva3AUjCWkhBBgHIT18JELd8kcWHEJxMKuIWMwcq+3CC59XS6L2zKzUGHxwohlnaDcuGcRWv5Nu91R+chQidBpJNLeM0GmQmxiJ8+1mnGk2ITk6OWhjIYbn6yCXFRgOFmh8uskEjuOC5qYPFmTBIRRJSVEBfvCNbCG+RSriBuADRgE+AJqQBt5WP3eAsYSsfgC5qaSOp26SdCw4BalRUKuALotdaBUTSpDAIRTL0px44bGUOuoyF5WxpY8CjCUCa6exfdc5vHO8AQDQabZKqp3GrFS2GyeBIzXsTk85CilZcMK1GqFRbCjWwyGBQyiW1w7WAuB7+Eipo25WggERWg2sDheqO8zBHg7hhomcijb+O/nkXLtkxA3gseCEajyF1CgV1bGqaO2DzelCtD4M0+MjUFZuRKm7BESwYRuqUBTGJHAIRVJWbsTBqk4AwC+vmSupZnNqtQozU/ldFbkbpMWtF+cIj7UaabXTECx/rX1wOF1BHg0hdmuy+Js5GTH4w8cVknFrAp44nFAUxhRkTCgOFjcRb9Ciy2LHzNRoXFKQBEA6vXxmpkbjy/oenGnuxdXz04M6FsLDox+cFh7bnZykmltmxhtg0GlgsTlR3WHGjJTAt5AgPIiTGRZnxQHgXVVScmsCfE8qIDRdVCRwCMXhdHEouXIG/vBxBQA+0A6ApDrqzqJUcclRVm7E64fqAACXFiThopwEyQhigLf8zUqLxrHabpxu6iWBIwHEIgcAjtV2S0rcAB4LTmWbGVaHE/owTZBHFDjIRUUojk0rZ+Kq+elwcUBMeBhSoj1Ft0qKCoKa7suY7a5Pca6lL8gjIQCP1W9JdhwAYEZKlFfgsRRcm4C4hhK5NqWCWMyEqaXl1gSAtJhwxEZo4XRxMIbY/YYEDqFIWAp2QWq0JGs/MAtOdYcZ/bbQbIQnJVg7jehwLQBe4ACewGMpWP0AjzAOxXgKqfLYvz1uTYeLk4wYZqhUKsFNFWq1t8hFRSiSilZ+p8KCeaVGcrQeiZE6dJhtMLb24sLpccEeUkjDrHoXP7YHAF85mCGlHXkoZ8RIkbJyI3Z8ch4AMD0+At9dmikptyZjdlqM0JMqlCCBQygSZoqVcpyCuEM0CZzgY7Y60NDdDwAoSJGmMGYWnIbufpgG7IhxW5yIwMPcmkVzUlB+ulVwawLSSWZgFbrnDpNJJYUK3f6GXFSEIjnXyu9wpWrBAcSBxrQblwKVbbwoTorSIT5SF+TRDE+sQYv02HAAwDmaN0GFuTWzEgwAgBnJ0nNrslT24/XdAHjLH8dxkqrQ7U/IgkMoDqvDiZoOCwBvV4PUmJVKAkdKMLfmDIlab9hufHZaNJp6BnC6uRdLcxIAhMZuXGqw3/UPnj8IwJOtCQTfcsMQW5RUADrNNjz27zN49tPzksv28gdkwSEUR1W7GU4Xh+jwMKTG6Md+Q5AQLDjUk0oSGN0CR6qimO3G+6wOAJ4SA6GyG5cqUhfGzKLE7EmhIm4AEjiEAmHxNwUpUZLMoGLMdFtw2nqt6DTbgjwaQpg3EnVrsoXqcHUXAOBMU68gbkJlwZIafVYHmnoGAAAzkqUpjAF+7rBboUaCqez+ggQOoTiMLSz+Rro3nNJd5/D8Z1WC/35w8J9U+tiEEhXuuC0WSyFFSooKcMvybADAFzVdJG6CTKXbepMcrUesQboB32XlRnBuE45Tgqns/oIEDqE4jBI3GQMed4NWw2+rWBwOuRuCw4DdidpOPm5rhkQtOIwHvjVXeCy1flmhhuCekrAoZveUNRekAQAy4yMkVbzSn5DAIRSHuMifVGHuhkp35+qzzeRuCCbn28xwcUBshBbJUdKN2wKAP+2tFB6zfllEcJD6Zkp8T/nJ5fkAAKvDJbkK3f6Csqh8AMtuGG5RouyGwGJzuFDtzqCScoo4wIucs829eP9EE944XAcOIHETJCra5BG3xRas/ORIVLaZccWsZMnUXAlFpB5gzFLZS4oKYBqwAwBae6249eIc4XUlQwLHBzB3A+B9kxGrZyIwCBlU+jCkxYQHezhjcldxAd4/0QQOgE6jpkUqSFQIVj9pLlSA9/3EYnOi8pNKZCYYhN04QCIn0FSKhLEUEW+sY8K1SInWo7XXivNt5pCYKyRwfIC41oDTxeF/Ls7Fy/uryd0QBIwsUDRV2jtxxntfNQqPbU4XysqNNF+CgMfVIF23png3/vcv+K7n59vMePjaecLrRODg623xLmapWnAGk58chdZeKypb+7AwMy7Yw/E7JHB8RElRAY7WdOGpciOe3mOEiyN3QzAQp4hLnbJyI8rKKxAboUVPvx03LplOO/EgIfVYCsB7N57vDmplFgSaL4Gnqp2P24oOD0NytLTjthgzUqKw/3yHMG+UDgUZ+5Drl0wHALg4cjcEC2Or9FPEAW93w+KsOADAwqy4kAn+kxI2hwvV7fxOXA7CGADykyMBAE09AzC7C/8RgUUcfyMHazHgmTckcIgJc/B8h/CYuRuIwOJpsinthUrsbshz78aZX1wqfWxChZoOMxwuDpE6jdDnSerEGXRIdPfLqnKLMyKwVLTKx1rMyE9hlr/QmDPkovIRZeVGvHqwVvj/Ty/PJ3dDgLE5XMLNXuoWHLG7Ic+9qzpP7oagILinUqNlsxMH+HnTYbahsq0P86bFBns4IYcc3JqDYa7Nmg4z7E4XtBpl2zgCcnXPPPMMcnJyEB4ejsLCQhw6dGjEY1966SWoVCqvn/Bw710Vx3F48MEHkZ6ejoiICBQXF8NoDJ61ROxuYJk7xXNTyd0QAEpFv1+2E4/ShyE9Nlw2FYHzktwWHNqJBwU57sQBcRwOzZtgUClDgZMWEw6DTgO7k0Odu7ClkvG7wHnjjTewefNmbN26FUePHsWCBQuwevVqtLa2jviemJgYNDU1CT81NTVerz/++OMoKyvDjh07cPDgQURGRmL16tUYGBjw9+UMi7e7ge3Gyd0QCFiKflm5EedE7qmn91TIpiIw84vXdVpgdTiDPJrQwyhTgZMXYvEUUsLp4oQNiZR7UA1GrVaJ5o3yhbHfXVTbt2/Hhg0bcNtttwEAduzYgffffx8vvPAC7r///mHfo1KpkJaWNuxrHMfhySefxC9/+Utce+21AIC//vWvSE1Nxdtvv421a9f650JGYbC7YV9lB6rayd0QCMQp+svzEgAAdqdLVin6ydF6ROnD0Gd1oLbDIukKzErEKIMaOMORL4rdIgJLXacFNocL4Vo1psVHBHs4EyI/OQonG0yobOvDSqQGezh+xa8WHJvNhiNHjqC4uNjzgWo1iouLsX///hHf19fXh+zsbGRmZuLaa6/FqVOnhNeqqqrQ3Nzsdc7Y2FgUFhaOeE6r1QqTyeT14y9yk+imE2iYpWz/+U4AwKlGk2zEDcAL+lDaVQUbsVvT4XR57cTl4tYEIASnV7X3wUVW4oDCrH55SVGysBKLEVybrcq3/PlV4LS3t8PpdCI11Vslpqamorm5edj3zJo1Cy+88AL+9a9/4ZVXXoHL5cKKFStQX18PAML7JnLObdu2ITY2VvjJzMyc6qWNiNhFRQSOkqICsNtM2AhtM6RMXpJ73rQr/6YTbMRuzbqufmEn/taxetm4NQG+aaJWo8KA3YXGnv5gDyekkHqLhtEYXENJyUguhHr58uW45ZZbsHDhQlx22WV46623kJycjGeffXbS59yyZQt6enqEn7q6Oh+O2Jt8twWnqsNMu6oA8tTuc2C/bYdLfg0I88jdEDCYxW/7rnOCtSY6XIsndxtlZfkL06iRnUiWv2Aga4GT4pkzHKfsNcqvAicpKQkajQYtLS1ez7e0tIwYYzMYrVaLRYsWoaKiAgCE903knHq9HjExMV4//mJafAR0GjVsDhcaumlXFQjKyo0o3c0LmjC1CncXFcgue21wqjjhX5jIeedLvlVGW69VVuKGkU/zJmCIXZsVg3pQycm1mZMYCZUK6Om3o8NsC/Zw/IpfBY5Op8OSJUtQXl4uPOdyuVBeXo7ly5eP6xxOpxMnTpxAeno6ACA3NxdpaWle5zSZTDh48OC4z+lPNGoVshMNACjtNxCwFP3/dleRzkow4O6VM2WXos9SxUNhVyUVSooKwLxRGpX83JqAx/IXCu6GYMNcm0/tPueVIs7uQXJxbYZrNciM59copcfh+N1FtXnzZvz5z3/Gyy+/jNOnT+OOO+6A2WwWsqpuueUWbNmyRTj+4YcfxkcffYTz58/j6NGj+P73v4+amhr86Ec/AsAHZN5999349a9/jXfeeQcnTpzALbfcgoyMDFx33XX+vpxxwXbjVXTT8TssRX+Bu3FcrjuWRW4p+mzcPf12dCp8VyUVysr5nnEA4OTk59YEKJMqkLB7SuluI/qsDmjUKrz3VZOsMjYZ+SGS1OD3NPGbbroJbW1tePDBB9Hc3IyFCxdi586dQpBwbW0t1GqPzurq6sKGDRvQ3NyM+Ph4LFmyBPv27cPcuXOFY37+85/DbDbj9ttvR3d3Ny655BLs3LlzSEHAYMFnUrWQBScAsBT9R977GoBHKADyStGP0GkwLS4CDd39ON9uRmKUPJr3yRW2647QatBvd2JdYZYsK4+HWm+hYFNSVICaDjP+72gDnC4OT5XLK26LkZ8chY/Ptil+3gSkVcPGjRuxcePGYV/bu3ev1/9LS0tRWlo66vlUKhUefvhhPPzww74aok+hTKrAw1o05CZHjnGkdMlLjuQFTlsfLspJCPZwFAsTNz+9PB9/3FsJAPjF1XOQGhMuO5HDXFQtJiv6rA5E6an7jr+ZPy0W/3e0AYB8myp7elIpW+BILotKCbBdFTXBCxyCwEmSr8Ahd0NgYG7NVRfwSQkp0XpE6sNk59YEgNgILZLc1j4KNA4M/zrOB6arVfJtqhwqqeIk9/0AK/bX0N2PfpsTETpNkEekbOxOF2rdfVXkLHCo2F9gYG7Nt4/xu3C5ujUZecmRaO+z4nybGRdOjwv2cBRNWbkRx+q6AQC/+c58tPVaZWf1Azyb8PqufgzYnQjXKnONIguOH0iI1CHOoAUAVHfQYuVv6rv64XRxiNBqkBotjTisyeBpuqnsXZVUOK8Aqx8QOrvxYMNcm7ER/L09JzHSq6aSnCw5bI3iOGV7Gkjg+AmhMi3txv0O6/uVkxQJtUxSNYeDWXBqOyywO11BHo3yqXbf2HNkL3Ao0DgQOF0c7ioqQJ/VAUC+GZulu87h6T0VwwpjOdXzGQ8kcPyEpycV3XT8DROReTJfqNJiwhGh1cDh4lDndrkR/oNZV5ViwaHNlH/ZtHImrl2Y4bEWx3gyHUuKCryaLksZVs9nwO4EAFS28vNGbvV8xgMJHD+RR4HGAaNK2IkbgjySqaFWq4TFlhYr/8JxHKralCFwhKzNdrNsrAhyhYni7EQDVCp5CgFmcTrVyDedrmzrE8SNHFPeR4OCjP2EYDYmgeN3PDtx+fWFGUxeciS+bjK543BSxzyemBwdZht6rQ6oVHz1a7lSuuscVCoI7WEau/uR6b6esnIjnC5ONpYFOVDVLv9kBoAXOZVtffjX8Ua8+2UjOEBx4gYgC47fELuoqPS+f1HKThwQld5vJWHsT1j8TUZshKwzSDRqFZ7cbUR0OL9XZT2SlOhukALVCglMB4CfXcmLGQ7yreczFiRw/ARvwgR6BxyKb2gWTPptTjT2DACQfwwOIGqeSJlUfkUpGVTM3cDuMefbzIp1N0iBKoUEpgPAO182CI/lWs9nLMhF5SfCtRpMj49AXWc/zreZhWJchG9h7qnYCC3iI3VBHs3kKXXvtq+YlQLAOwaHXA2+R0k78ZKiAnxW0YZDVV34zftfw8Up090gBZRQUBTg7yll5RWICQ+DacCBmy7KlGU9n7EgC44foUwq/6OUhYplNnx4qhkAHyPSY7GTq8FPKGknDgDfXZoFAHBxynU3BJsBuxONPf0A+Bo4ckVs4WNNipdmx8uyns9YkMDxI8xlQplU/oO5GuTunmKuhj98XIFIPR8Tsm3naXI1+AnPTly+AcZiDld3Co+V6m4INnWdFnAcEKUPQ1KUfK3FrFVJSVEBshP5+V/dYZZdPZ/xQC4qP8DcDcO1pCd3g29RiskY8JiGman49UN1JG78gMvFoaaDZcPIP/OurNyINw7XAQBUKuBnV85QpLsh2IjLUcg1RRyA19rDLFHV7r8Hpc0XsuD4AeZuOF7XA8ATMEruBt+jNFdDSVEB2PRQq5R3w5ECLb0D6Lc7oVGrMD0+ItjDmRLsnrKpuABR+jBwHPDtBRmKdDcEGxbvJ2f31GDYtdQotKUQWXD8wOCdeG2HBU/uOocny420I/cxSonBYZSVG8EsxC6O/z/NF9/CRHFWggFajbz3eGJ3w0dft+BUownV7RZhzijJ3RBslFIDRwwrjlrdbgHHcbK2TA0HCRw/UVJUAI7jULrbCIeLI3HjB3osdiE9Vgk3HbYb//aCDLzzZSNSY/TkavADgtUvUf7xN4PdDacaTYKlgeaMbxF6lynIgpOZwJcz6bPy5UyUlu0r7+2LxLmreCaYHg5Tq+iG42Oq3Dfy1Bg9IvXy1urizIY7r5gBALDYnNhUXECuBh+jlCabg2EBoyy+iPAtgotKQfNGH6ZBRizvpq1WYDIMCRw/UlZuBDMQO1wcLVI+RugiroAdldjVwFoH9A44cMvyHMVlNgQb5mqQe+bdYDwBo8pbqIJNv82JJndBUSVYi8UIbioFCmN5b3slDNuRX5QTj8PVXViUGUfuBh/DWjSwZoNyRuxqiNBpkBYTjmbTgJC+SfgOQRgrbKEiC47/YKIxJjwM8QZtkEfjW3ISI/F5RYciA43JguMHxO6G6xdPBwDEGbSU2eBjqjqUF/THoMXKPzhdHGo7lTlv2PXUd1lgc7iCPBplIU5mUFogLrP8KbFeGwkcPzBcIaWaDosiCykFmlKRQGQ7cVbLpKzciFK3lUzukLvBPzR09cPu5KALUwuxB0ohOVqPCK0GLo4XOYTvqFJg/A1DyZspclH5geEKKdV1WeB0ceRumCKsxhDHcaIu4gYvq5kSyE5S7k0nmLCFKjvBALXC6lGpVCpkJxpwprkXNR0WoTM9MXWUVo5CDLum6naz4lLFSeD4mbSYcOjC1LA5XGjs7kdmgvxTU4PJ4BpDahXwzpeNKCuvUFQaPllw/IOSFyqAnzdnmntp3viYagXWwGGwVPFeqwOdZhsSFZQqTi4qP6NWq4SsGNqN+4aSogJ8dykf28QXw1OWuAGUbTYONN5uTW+BoyS3JkCWP39RpcAqxoxwrShVXGHzhgROAMgRNTQjfMOS7HjhsRK7J2e7b6SdZhtMA/Ygj0beMLdmWbnRq7WHElunkOXP9/RZHWjrtQJQZgwO4NlQKa0WDgmcAJCVoOx+H8Hg/47UA+BdVErsnsx3LOZNxbUK21UFGhbcz/eH6wYAHK3tUmSndk9vIZozvoIt+gmROsRGKCtFnJGt0J5UFIMTAHLIbOxTysqNOFTdBQD4xdVzYLE5FVljKCfRgPY+K6o7zJg3LTbYw5E1JUUFcLo4POUWwm9+Ua84cQN47jV1nRY4nC6EybzXlhTwNNlUbvxkrkKL/dHsDwDZtKvyGcytkBLNWzeyEgxeO3QlWXJo3viWaxdmCI+V6NYEgNTocOjD1HC4ODR09wd7OLLFK26rzTtFXGlxW4DnXqM01yYJnACQzYKMO81wUQ2cKeF0cdhUXIB+uxOA56ajxBpDOQr1iweLp/dUAABUUKZbE+CTGoR4ChLGk8Yrbsu96OcmKjNuC/AE3Ve5U8WVArmoAsC0+Aho1CoM2F1o7bUiLTY82EOSLZtWzkSX2YbS3fzilCVKu1fajjw7iSw4vqKs3Ih/HmsAAKy6IBUXZMQq0q0J8Lvxcy197niK5GAPR5aIy1Gku+/XZ5p78f6JJkW6NsX977osdiRE6oI8It9AFpwAoNWoMT2eT8NTWhBXMKhxl9pPjdEjXKsJ8mj8B7P8Kc1sHGjYrntRVhwAXgAo1a0JiC1/JIynApsjrMmmUsUNwKeKMyGnpPtNQATOM888g5ycHISHh6OwsBCHDh0a8dg///nPuPTSSxEfH4/4+HgUFxcPOf7WW2+FSqXy+lmzZo2/L2NKUC0c31EjVKNVZsomg2XEtPZaYbE5gjwa+cJapyQY+F0p+1tUolsTUG5GTDC47eIc4bFWo1KkuGHkKHDe+F3gvPHGG9i8eTO2bt2Ko0ePYsGCBVi9ejVaW1uHPX7v3r24+eab8fHHH2P//v3IzMzEqlWr0NDQ4HXcmjVr0NTUJPz87W9/8/elTAmqT+E7mEjMVnBWAwDEGrSIc3cuJmE8eTa5d93M8ieeNyVFBV6tVZSAUHqf7jVT5rF/nxEe252c4qx9YlgGXpWCLH9+Fzjbt2/Hhg0bcNttt2Hu3LnYsWMHDAYDXnjhhWGPf/XVV/HTn/4UCxcuxOzZs/GXv/wFLpcL5eXlXsfp9XqkpaUJP/Hx8cOeTypQZVrfESoCB6DduK9wibqIK93yx/4u6jr7FWedCiRl5Ua8erAWALAwM06xLk2GEu81fhU4NpsNR44cQXFxsecD1WoUFxdj//794zqHxWKB3W5HQkKC1/N79+5FSkoKZs2ahTvuuAMdHR0jnsNqtcJkMnn9BBph8nQqZ/IEi1r37zBLgWXTB5NDGTE+oaV3ADaHC2FqFTLilB3knx4bAZ1GDZuT739HTBwWt3XJjCQAvGhUctwWIPIyKChr069ZVO3t7XA6nUhNTfV6PjU1FWfOnBnhXd7cd999yMjI8BJJa9aswfXXX4/c3FxUVlbiF7/4Ba666irs378fGs3QoNNt27bhoYcemtrFTBG2UNW0WxTXsTXQsMU+OwQalypxVxUMmNVvWnyE4ovfadQqZCZEoLLNjJoOCzX4nQQsbquphxeI2aK4Lfa6Uih1p72vuoBfp8WbqbJyI1+aQ6ZuXEmniT/22GN4/fXXsXfvXoSHe3Zda9euFR7Pnz8fF154IfLz87F3714UFRUNOc+WLVuwefNm4f8mkwmZmZn+HfwglNyxNZBYbKK+MCFkwSHX5tRgAjErRBb7nMRIVLaZUd1hxiUFScEejuxgC/r3/nwAgLe1WGmBxqzmj93pAgD09NvRbbHhr/trhHYmcsWvAicpKQkajQYtLS1ez7e0tCAtLW3U9z7xxBN47LHHsHv3blx44YWjHpuXl4ekpCRUVFQMK3D0ej30+uAKinCtBukx4WjsGUBNp4UEziRhcRSxEVrEGpTZF0YMVTP2Dez3FwqiGCDLn68IhXg/cc2fKH0Y+qwO/HbnGfztUJ3s0+L9aqvV6XRYsmSJV4AwCxhevnz5iO97/PHH8cgjj2Dnzp1YunTpmJ9TX1+Pjo4OpKen+2Tc/iJL2I3TTWeyhMINRwyz4DT29GPAXb2ZmDjDZVApEdZiYLjeQkpsMeBPbA6X4KJSuuWPxRf1WflyFEoQN0AAsqg2b96MP//5z3j55Zdx+vRp3HHHHTCbzbjtttsAALfccgu2bNkiHP/b3/4WDzzwAF544QXk5OSgubkZzc3N6OvrAwD09fXh3nvvxYEDB1BdXY3y8nJce+21mDFjBlavXu3vy5kSniAu2o1PllBzNSRE6hCtDwPHAfVdNG8mC+vIrvR5w9wNx9xd09nfi1JbDPiThu5+uDggXKsWet8pmZKiArDpoVEpo+aP32NwbrrpJrS1teHBBx9Ec3MzFi5ciJ07dwqBx7W1tVCrPTrrT3/6E2w2G/77v//b6zxbt27Fr371K2g0Gnz11Vd4+eWX0d3djYyMDKxatQqPPPJI0N1QY0Fm46kTaq4GlUqF7CQDTjaYUN1uwYyU6GAPSZYIxSEVPm/E7gaA/3t5avc5lO42KmJHHkjEm6lQSAopKzeCxU47Ob7mj9znS0CCjDdu3IiNGzcO+9revXu9/l9dXT3quSIiIvDhhx/6aGSBRaiF00k78cnCYnCyFO5qADzZDdmJkbzAEQljuWc3BJJuiw2mAd70rnQLDsCLHJeLw5PlRlgdLhI3k0S41yi8bhLgsfBdMz8d759oQkZsuCJ6tSk7X1JiULG/qVMTQinizN3Q1sv3wmHXTu6GicF+bynRekTolNu7TMzdIuEbplaGuyHQhEq8H7ufbF45Ez++LA8A4HCnycu95o+k08SVBjOPd5pt6Om3IzZC+VlAvsTudKHBXbhM6a4GYKi7obrD7HUzokVrfIRKgLEY8aLkcCnD3RBoQkXgsJo/JUUF6LbYAPD97zZcmie8LldI4ASQKH0YkqJ0aO+zobbDgvnTY4M9JFnR0MWXnteHhUbQH8CLnPouC/7+RT3+Y2zHf4ztJG4mSK0QS6F8UQx4duQLpsfiy/oeLMuNV4S7IdAIFdMVbi0Wu7njDDrEhIfBNOBAbadF9vOFXFQBgqVvDteygdI3x4d4J64OIffMPatmCY+V3tHYH4TKThzwdjd868IMAEBKdLgi3A2BhONEvctCwFoshl1vrQJiRUngBAgWT2Gx8cGOFE8xcUJtJ854/VCt8FjpHY39QSgJHLG7IUtoumkR6pzI2d0QSFp7rRiwu6BWAdPiIoI9nIDCLFZKyPYlF1WAGBJP0U7xFBOlOoQWKkZZuRGlu41IiNSh02zD9YunkbthgjBraSjsxMXuhsFZmzRfxg8TxRlxEdCFhZYdgAljJVhwSOAEkJKiApxpMuGDk834x5F6cACJmwkQSjtxwNvd8FV9N3afbsXirHjkJEaSyBknA3YnWkx877JQyLwTkxnPX2+3xU5JDRPEUzcptOYM4Pk7UUK2b2hJUwmw4Zt8ZDoHQKdR0wI1AWpDaCcOeLsbWEfoWnI3TAi2C40OD0NcCPQuExOpD0OSu+ddnQJ244EklGrgDIa5qJQwZ8iCE2A+OuVpPGpzuih9c5y4XKKgvxDZiXu5G5jA6SB3w0QQW/1CoRrtYLISItDeZ0VNhwXzplHW5ngJNWuxGCF2q8sCp4uTdXwoWXACSFm5EX/6pBI6Df9rv3VFNmU2jBMW9KdRqzAtPrSC/gBRmw8F7KoCieBqCMGdOIBhszaJsakJsc2UmPTYCGg1KtidnNBsVK6QwAkQ4niK/JQoAMClBcmUvjlO2EKVERcOrSb0pm2myGzMceSaGi+h1NpjODIV5G4IJELGZgjOG41aJcRvyT3QOPRWiiAhjqfIpniKcVEqEn5sR8WabIZa7aDp8RFQqYA+qwOdZluwhyMbQqm1x3AoKWA0UJgG7Oiy2AGETrzfYDIHucTlCsXgBAhxPEXWoJ5UFE8xPKx2EADYHC4AfACc2BoWKoRrNUiLCUdTzwBqOy1IjAqNSs5TJdQtONkKSvkNFGxRT4zUIUofmkukUhpDh+a3F2SUFKXuT8S1g2am8m69uk4LXj1YG5Lp9VkJBkHgLMqKD/ZwJI/D6UJ9l7flL9Rg95rG7n7YHK6Qq+kyGdjGM1RFMeCZN3K34NBsDwJCpUgSOGPCXHjnWvoAAJ+GcC8mpdx0AkVTzwDsTg66MDXSYsKDPZygkBytR7hWDRcHoVEtMTpCYcgQdWsConuNzNcoEjhBIFtUQt1FsTdjIhYzYSHci0kpZmN/4hW35RaCmfERUKtVIRe3BQAqlUoxi1WgqBUsOKFp9QNE2Xcyb9dAAicIZMRFQKNWwepwobXXGuzhSJ7Hd54RHjtCuBeTUgL//AmL2yorN3q1aAjlnm+sWF2tzBerQBHqgemAx4JjGnCg2yLfpAaKwQkCWo0aGXHhqOvsR22nBWmxoWk+Hw9l5Ub8cW8lACApSodblueEbJsCJXX59RfiuK2l2XycUpfZFtI938iCMzE8XcRDV+BE6DRIjtajrdeK2k4L4gy6YA9pUpAFJ0goqWOrv2C77m/NTwfA/85YTE4o1g5ic6bZNIABuzPIo5EubI58UdMFADhW1x2y4gYQuTbJ8jcizLVpdTjR6C5ux4KMQ9G1CSijxAAJnCDBzMaUSTUyrHbQBe4S82yBD9XaQfEGLaLdaassO4gYnpKiAjBnVJg6dOO2AGV1h/YXzLW57YMz4DjAoNMgOUof2q5NBcwbclEFCTIbjw2rHbTlra8AeAf9heKCpVKpkJlgwNdNJtR0WDAjJTrYQ5IsT+0+ByZ/HS4upHu+ie81HMeFZE+usRC7NgH+d/b0ngpybULeXgay4AQJyogZP57OvqHrE2dQ4baxKSs3onS3x31ZcuWMkHRpMlgVbIvNifY++QaM+puSogJcMSsZAHC2pTekxQ2gjHsNCZwgQcX+xg8JHA+Dq2AT3jCXws3LsgAAaTHh2LxqVsjGbQGAPkyDdHcdIDkvVoEgN4kvKMpxgE6jDllxA4iz7+Q7Z0jgBAm2ULX32dBndQR5NNLF7nShsXsAQGhnNTBIGI8Oi9v6Rl4CAIrbYnjiKeTrbggEe8+2AuBjcmxOV0gKYga73zaZBmB1yDOpgWJwgkRMuBZxBi26LXbUdVowJz0m2EOSJI3d/XC6OOjD1Eim/kvIdu+qyLU5PCxu6w97+IVJXG4/tHfjBhw430mWv1EoKzfifDsvAF+49SJ8WdcdsiUpAL4Xl0GngcXmRH1XP/KTo4I9pAlDAieIZCcY0G3pQU0HCZyREPrCJBigDsFMhsGILTguF0e/kxEgt6Y3VENpdJhrM0ytgsPFISvBgMtm8vE4oSpyWBXsM829qO2wkMAhJkZmggFf1veQu2EUaKHyJiMu3KsKNhWJHB6hGi25NQFQH7OxcLo4/PibeXj20/NQq4BpcREAPKIm1Fybpe7U+OxEXuCIM6nKyo1wujjBWiplKAYniHgyqcgvPhJM/GWSwAEAhGnUws2XduMjQ/PGGypLMTqbVs7EyrmpAID02AivruslRQWyWMx9CasL1N7HtxJiLnG51QUigRNEPDcd6vI7ErQTH4qnMi0J4+GwOpxoMvGB6WT542FzprXXin6bPANG/Q1Ziz2woPwjNd0A+A0DEzdySp0ngRNEqAne2NBNZyiZlEk1KvVd/eA4IFKnQWKkPHvo+JrYCC2iw/mIBLLiDA9tprwpKSrA9YunAQDKT7fKTtwAJHCCCsvwqO/qDzkf73jgOI4a3w2D0COGFqphqRW5p6hqLx9P8fSeimELt4Vqn6XhILfmUEqu5MUMB3nWBQqIwHnmmWeQk5OD8PBwFBYW4tChQ6Me/+abb2L27NkIDw/H/Pnz8cEHH3i9znEcHnzwQaSnpyMiIgLFxcUwGuVXryAtJhw6jRoOF4fGbnJTDabLYhdqBE2Pp5sOw1NCnQTOcNTSTtwLFk9hs7sAeFybcoun8DdkLR7KP481CI/lWBfI7wLnjTfewObNm7F161YcPXoUCxYswOrVq9Ha2jrs8fv27cPNN9+M9evX49ixY7juuutw3XXX4eTJk8Ixjz/+OMrKyrBjxw4cPHgQkZGRWL16NQYGBvx9OT5Fo1ZhejwfMEruhqGwG05qjB7hWk2QRyMdmOWP5szw0ELlDYunONfaB0C+8RT+poasxV6UlRvxVLkRMRG8a/OmizJlVw3c7wJn+/bt2LBhA2677TbMnTsXO3bsgMFgwAsvvDDs8U899RTWrFmDe++9F3PmzMEjjzyCxYsX4w9/+AMA3nrz5JNP4pe//CWuvfZaXHjhhfjrX/+KxsZGvP322/6+HJ+TRT2pRoTtNFlxO4J3N3xwogkA0GH2roJN7gYeEjhDKSkqQPGcFADAXw/UkLgZRL/NibZePmOI5g28BPD8abEAgItyEmTX8sSvAsdms+HIkSMoLi72fKBajeLiYuzfv3/Y9+zfv9/reABYvXq1cHxVVRWam5u9jomNjUVhYeGI57RarTCZTF4/UoHSN0eGfOJD0ahVeObjSkS4LVrMHUPuBg/sdyLuPk8At12cC4D6LA0Hu//GhIchzkCB6azlSUlRgdcaJbeWJ34t9Nfe3g6n04nU1FSv51NTU3HmzJlh39Pc3Dzs8c3NzcLr7LmRjhnMtm3b8NBDD03qGvwNFeAaGcpqGApblFh11dpOM3afbqEduRtxYDrtxL3Zc8YTFsDiKUJ9vjCEOUP3GgDwqvszONtXTnMmJLKotmzZgp6eHuGnrq4u2EMSIAvOyNBCNTwlRQWYlRYNALjztWMkbkS09VnRb3d6VaMleAvf859VgRn4fnRprqxcDf6G7jUjI+c1yq8WnKSkJGg0GrS0tHg939LSgrS0tGHfk5aWNurx7N+Wlhakp6d7HbNw4cJhz6nX66HXS6tRIyuFveoC3hIl11LY/oRcVCOzam4qzjb3wuniyN0ggs2ZwdVoQxlxPMVbR+tR3WFB0exUxIRrQ7bP0mCYdSKL4v2GMFx5Abng1zuATqfDkiVLUF5eLjzncrlQXl6O5cuXD/ue5cuXex0PALt27RKOz83NRVpamtcxJpMJBw8eHPGcUoSlbr7/FR8wahpwoMdip1gKN+JqtOSiGsrZ5l4AgEolz/RNf0E78aGI4ykyhd24WXbxFP6E5s3IsDnT3meDWZTUIAf83mxz8+bN+OEPf4ilS5di2bJlePLJJ2E2m3HbbbcBAG655RZMmzYN27ZtAwDcdddduOyyy/D73/8e11xzDV5//XV88cUXeO655wDwHU7vvvtu/PrXv0ZBQQFyc3PxwAMPICMjA9ddd52/L8dniGMpWEv6xz88g1cP1pK7AZ5qtAaqRjuEsnIjPvqat3JmJRhww+LptBN3Q3FbQxFbgrMTDfiP0bOgh/p8YVCK+MjERmgRZ9Ci22JHbacFc9Jjgj2kceN3gXPTTTehra0NDz74IJqbm7Fw4ULs3LlTCBKura2FWu0xJK1YsQKvvfYafvnLX+IXv/gFCgoK8Pbbb2PevHnCMT//+c9hNptx++23o7u7G5dccgl27tyJ8HB5dVYeHDBK4saDkAlD1Wi9YBa+DZfm4s//qUJDVz9+enk+AJDIgXcVY2IoVCRyKC4Xh3p3P0Cy4AxPVoIB3ZYeEjjDsXHjRmzcuHHY1/bu3TvkuRtvvBE33njjiOdTqVR4+OGH8fDDD/tqiEGjpKgAT+4+BxcHaFSqkF6cxJDJeHiYu2HjFTPw8v4a2BwuNPUMCPMm1N0NVMV4dFiMCRWJ9NDSOwCb04UwtQrpsfLaJAeKrAQDvqrvkV22b0AEDjEyZeVGsDXJyXGUuumGBM7weKdvGlDR2oeaDgsyEww0b0DzZiyyqbDoEJg1a1p8BMI0FJg+HHLNpKJvM4gwd8OaC/jMsMz4CErddEOxFGMj15uOv+i3OdFK1WhHhbnuui129PTbgzwaaUCieGzkmklFAidIiFM3f3QpX2HUxUF2pbD9BaWIj40QT9FpHuPI0KCui6rRjkWUPgxJUfzvhtxUPOJ4P2J4MmW6mSIXVZAQp2629vLp0I09/fjxZXnC66EGqw30sytnDNlVUW2goVAVbG88LRpooRqNzAQD2vtsqO20YJ67z1AoQxacscl2tz2p77LA6eJkU8aEBE6QEC/UyVF6IVW8oas/ZGMpWG0gs9WBfrsTKhUwPd7gZe0iPMjVbOwvhFRfKtY2KtkJBhyr7aZMKjckcMYmLSYcWo0KdieHpp5+TI+Xx++KXFQSQKVSidwNoXvTYYXHnv30PAAgIzYCOz6ppFYEIyAInA4LOC70LH6DIbfm+KDYLW+oD9XYaNQqZMbLb96QwJEImeRuAMCLHBZ03djdT+JmFNguqtfqQJclNANGS0XxasO5NUvdtYEID6zLei3FbqF3wI5Osw0AWXDGQo5rFAkciZBNuyoB1kiSA6jP0iiEazVIi+HrdoTqvGFuzbJyo9DPLTvRQC1PRoGK/XlgfzcJkTpEh2uDPBppI0eXOAkciSDUp6CbDj76uhkAX/iQ+iyNjmexCs3dOHNrbt91DtXuv509Z1rJ8jcK7F7T2N0Pu9MV5NEEF3Jrjh85hlGQwJEI4iZ4oUxZuRGnm/hGkk/dvJDS5seAxQ2EcspvSVEBNlyaK2QePv9ZFYmbUUiO0kMfpoaLAxq6+oM9nKBCAcbjh/2O5HSvIYEjEbIFv3joBowyt4JBpwHAZ8OId+gkcoaSTe4GAEDxnFThMbk1R0etVoV0oLE4bksoKEpxW2OSRS4qYrJMi4uAWgUM2F1oc1djDTWcLg4br5gBi80JwPMHxUROKNYGGossKr0PgLfaAIBKBXJrjgM5uht8hThuS2zBobit0cmSYRVsqoMjEXRhaqTHRqChux81nRakxIRe07dNK2fi60YT/vBxBeIMWsRGeIL+aEc+PHI0G/uasnIjPvq6BQCwrjALKdHh1Fl9DELZtcnmxPZd54R7zOGaTrz5RT25NkfBoAtDUpQe7X1W1HVaECuDIpFkwZEQFGjsiUHKJp/4uGACp9k0gAG7M8ijCTxs1z0zNQoAuTXHCwWnF+DuogLBEkHiZnxkJUQAkM8aRQJHQsgxDc/X1Ajl9qka7XhIiNQhSh8GjgPqQzBglLU8CdfycVvk1hwfnntN6M0Zxg1LpguPKW5rfIhjReUACRwJ4SmkFJq7KkBcbp8sOONBXAU7FDPwNrl33cN1ny8pKqDeZSOQ5W5nUdthDtmkhid3e4KJKW5rfMgt25dicCQE66ETioF/DGqYOHGyEgz4uskkG7Oxr+kRBT1Suu/4mB4fAZUKMNuc6DTbkBilD/aQAkpZuRH/d7QBAHDl7BQszIyjuK1xILeCtCRwJER2CAf+MWooBmfChLprk82Z5Gg9DDq6pY0HVgW7qWcANZ2WkBI4LG5raXY8vqjpQlaCwSvwGCCRMxJZMosTpbuBhGDmv/Y+G/qsDkTpQ+vrsTtdaOweAODx9RJjI8ceMb5kcC0TYnxkJhjQ1DOAuk4LFmfFB3s4AYPFbZ1q7AEA5IjittjrhDel7vT5tRdlAvBUwdZq1CgrN8Lp4iTpDg6tFVTixEZoEWfQottiR22HBXMzYoI9pIDS2N0Pp4uDPkyNlOjQ2VFOlVC34FA36MmRnWDAoapO2ezGfQVbiNc8+SkA780UWW6Gh9UO4jgO4Vo1BuwuNHT1450vG4W2KFKEBI7EyE4woNvSg9rO0BM4QgZVggFqKrY1bljsVm2nBS4XF3K/O5bqnENWvwkRyk03OY4jYTwBxC68xEgdBuw2lO0x4q2jDZJOr6csKomRJaThySNK3ZcIGVR0w5kQ6XHh0KhVsDpcaA3BKtjVw2RQEWMTysX+2vqssNicUKn4gGtibFjphQ6zDQAkL24AEjiSQ26FlHwJS49nKazE+NBq1JgWx8+bUHRT1Yosf8TYsF5MnnYNns1UqPRiYnMmIzYC+jBNkEcjH0qKCsAMxGqV9F16JHAkhtjdEGoMV8uEGB+hWpl2wO5Es4kC0ycCi6f46BTf3qLFZMWA3RlSvZjoXjM5ysqNYDHYLg6Srx1EMTgSQ44dW30F+cQnT1aiAagIPXcDu95ofRjiDdoxjiYA73gKnUYNm9OFbR+cxsv7ayTvcvAVbCNAAmf8MAF8/eJpeOtoAxIjdZJPqyeBIzHYTryhqx8OpwthmtAwsomD/ijdd/yw9M3sYbpDSzl901fUiApDqlTKtzz4isF1X0JJ3ACevxNyh48PJm42r5yJby/IwFtHG2C2ObCpuEDSIic0Vk8ZkRYTDl2YGg4XJ9SECQVY0J9aBUyPJ4EzXpi74WQDX9ODLfih4m6gwPTJI46n0KhVklyg/AX7O8mheTMuWO2gkqICTIuPgEatwoDdhbXLsiTd840sOBJDrVYhMz4ClW1m1HZaQsZdw4L+0mMjoAsj3T1eBu/E6zotXrstpS9aFJg+ecTxFE4Xh7Jyo+LnC4Pc4RNDbAXWatSYHh+Bmg4LqtvNkp4ztJJIiFDObqCgv8lTUlSAO6/IBwB0mG0hI24AsuBMFiaCV85NAcD//ra77z9KxzRgR6c71ZkC0yeHXGookcCREMzd0G3hGwfWhpC7gRaqqXHv6tnC47AQcjfUUpuGCSO28K2/JE94fvPKmSEhcticSYzUhVw7HF/BimpWSzxrk75dCTHY3VAbQu4GcjVMDfGi5AgRd4PTxaGuyy1wkmjejBdxPEVzDx/nV9/VjzsuzxdeVzJkLZ467HdXI/GsTb9acDo7O7Fu3TrExMQgLi4O69evR19f36jH/+xnP8OsWbMQERGBrKwslJSUoKenx+s4lUo15Of111/356UEjJKiAly7MAMAsPNkc0iIG4AsOFOBieA56dEAgItnJIbETpxv+MdBp1EjLSY82MORDZtE95OUaD3CtWo4XRwauvpRUlSg6Kw7wOP6J/fU5GG/O6nX3fKrwFm3bh1OnTqFXbt24b333sOnn36K22+/fcTjGxsb0djYiCeeeAInT57ESy+9hJ07d2L9+vVDjn3xxRfR1NQk/Fx33XV+vJLA8rMrZwAAOAA6jVrx4gagarSTRWzhW3NBOgBgWlxESLgbWKDo9IQIRbtv/YlarRL+5qTubvAVNe10r5kqLPuspt0CjpOuxc9vLqrTp09j586dOHz4MJYuXQoAePrpp3H11VfjiSeeQEZGxpD3zJs3D//3f/8n/D8/Px+/+c1v8P3vfx8OhwNhYZ7hxsXFIS0tzV/DDyrvHG8UHtucLsW7G/qsDqG/CVlwJobY3fCv4w0A+N5Mj//3AuF1pVJD8Tc+ITsxEuda+iQfMOorPBYcmjeTJTPBAJUK6LU60GWxIyFSF+whDYvfLDj79+9HXFycIG4AoLi4GGq1GgcPHhz3eXp6ehATE+MlbgDgzjvvRFJSEpYtW4YXXnhB0ipyIpSVG1G2pwLR4fz13nRRpuJ34szMmRCpQ3Q4VaOdCGJ3Q84gs7HS3Q3kavANbDceKhYcITCd5s2kCddqBLewlOeN3yw4zc3NSElJ8f6wsDAkJCSgubl5XOdob2/HI488MsSt9fDDD+PKK6+EwWDARx99hJ/+9Kfo6+tDSUnJsOexWq2wWj1dlk0m0wSvJjCI3Q0HqzrweUUHLspJwLS4CElXi5wq5J7yDUzgtJissNgcMOiUnUNA88Y3sIW+NgQsOAN2J5qE3mU0b6ZCdqIBTT0DqOkwY3FWfLCHMywTtuDcf//9wwb5in/OnDkz5YGZTCZcc801mDt3Ln71q195vfbAAw/g4osvxqJFi3Dffffh5z//OX73u9+NeK5t27YhNjZW+MnMzJzy+PyB2N3AbjqskJKUq0VOhlKRVWpwgLHSa/74i1iDFnHufkyh4G6gbBjfIJeUX19Q32UBxwGROg0SJepWkQvCvGmX7r1mwlu8e+65B7feeuuox+Tl5SEtLQ2tra1ezzscDnR2do4ZO9Pb24s1a9YgOjoa//znP6HVju62KCwsxCOPPAKr1Qq9Xj/k9S1btmDz5s3C/00mkyRFjtidkDvopqM0yw2r+QMATe5U1ewEg5cVi5g42YmR6LZ0o6bDjDnpMcEejt/w6l1GAmdKsN9fXWc/nC5O0QHbnt5lkdS7bIoIlj8Jp4pPWOAkJycjOTl5zOOWL1+O7u5uHDlyBEuWLAEA7NmzBy6XC4WFhSO+z2QyYfXq1dDr9XjnnXcQHj52+ufx48cRHx8/rLgBAL1eP+JrUiVb4X5xcc2frIQIAMDZlj58eKo5JNLi/UVOogFf1nWjWuEWnA6zDX1WB1TUu2zKZMRFQKtRweZ0oamnX9G/T+pB5TvksEb5zUk/Z84crFmzBhs2bMCOHTtgt9uxceNGrF27VsigamhoQFFREf76179i2bJlMJlMWLVqFSwWC1555RWYTCYhXiY5ORkajQbvvvsuWlpa8I1vfAPh4eHYtWsXHn30Ufzv//6vvy4lKOS4C5exNDwl7jYGFzYkcTN15FKfYqqwhSo9JhzhWk2QRyNvNGoVMhMMON9mRk2HRdECh3pQ+Q6h2J+EN1N+rYPz6quvYvbs2SgqKsLVV1+NSy65BM8995zwut1ux9mzZ2Gx8L+go0eP4uDBgzhx4gRmzJiB9PR04aeurg4AoNVq8cwzz2D58uVYuHAhnn32WWzfvh1bt27156UEnCxRGh7rm6JEfnJZvvBYqwmdFgP+IjfJvauSsF98sojjtmrdGVRZFLflE0IlDoddXzZVTJ8ybDPVabbBNGAP8miGx69pFgkJCXjttddGfD0nJ8crvfvyyy8fM917zZo1WLNmjc/GKFXCtRqkx4SjsWcA1R1mJEbJy8U2Xh794LTw2O4MjRYD/kTJFhxx3JbLfZ/IToikuC0fIIfduC+opcB0nxGlD0NSlA7tfTbUdlgwb1pssIc0BGXnkcqcnKRIXuC0W7AkOyHYw/E5ZeVGvLSvGgAwNz0Ga+alKTodPhCwnXhjzwAG7E5FuW/ELk3WlqK+24I3vqgj1+YUyRFlbSoVr95lJHB8QnZiJNr7bKjuMEtS4FA3cQmTrWCzMdt1XzYzCQCQmxQppMMrvbChP4k3aIUikVLObpgsbI6cbuoFAHxe0UHixgco1YIjdms29fC9y7QaFdJjI8it6QOkPm9I4EgYIZ5CopNnKrCaPyygMcd9rUqs+RNIVCqV4nfjYjETpqa4LV8gVMHuNMOloL895tYsKzcK7qnMeAOe+bgC23edU3RKfCBgsUxSdYmTi0rCKDmegtX8WfeXAwA8N1iA3FNTJTvRgBMNPZLdVU2Vx3d6Cok6XBS35QumxfMNSwfsLrT2WpEWq4zu7GK3ZvEcvrK+i+OEmC2aN1MjR+KbcBI4EoYt+lXtZsWmirNsn9wkymrwFUrOiCkrN+KPeysBACnRenz/G9kUt+UDtBo1psdHoKbDguoOs2IEDjC0HEV1h4XEjY+Q+iacXFQShvk3ewf4jq1KY8DuRGNPPwBP3R9i6gg1lCS6q5osLG7rqnl8JfQcitvyKVJfrKZCSVEB2P5QQ25Nn8EKJraYrOi3OYM8mqGQwJEw4VoN0mOl37F1stR28n1hovVh1BfGh7CbTpXCYnBY3NbMVD6DKs8t5Chuyzd4uoorSxgDvDhmFUicbrcmMXXiDDrESDipgQSOxFFywChbgHOTqS+ML8kWUsX7YXVIb1c1WTa53QpM7IutfiVFBV693IiJw7qyK82Cwyx/LKB4/cW5ZPHzIezvUIqbcBI4EkfqQVxTgYk2cYAxMXWSonSI1GnAcXwDRaVRRfPGLwiZVAq61zBx86NLcuF08Sniv7hmDrk1fYiUXZsUZCxxshVswRluJ05MHZVKhezESHzdZEJNhxkzUqKCPSSfwXGcIHDykmne+BK2marpUE7/O+bWXJQVh798VoWsBINXDA65NSdPqdsqlp0wtBZOWbkRThcXdKsqWXAkTo6E1fFUEVxUSVRV1Nco1fLXYbahd4DvIs5cKoRvmB7P97/rszrQoZD+d8yt6bnXeMQ+uTWnBqsxdK6FL7rJBM5gl2AwIQuOxFHqQgV4UsTJ1eB7pGw2ngrMkpkRG6GoNhRSIFyrQUZsBBq6+1HTYUaSgvrfkdXP9wxNvzd79YWTQqYaWXAkDqsU2dNvR5dCdlUAYLE50GwaAEA1cPyBUjOpzgs7cZozvoS1NGClKcTd6JXQ0qCK5o1fKCkqwI+/mQcAqO/ql5S4AUjgSJ4InQZpMcpLFWc30DiDFnEGShH3NUoMGAVEgenk1vQpzN3QO+AA4LH8ScndMBUoMN1/3H/VbOGxViOtGkMkcGSAx02lIIHTQTccf8ICt+u7LLA5XEEeje9g80YcS0FMHVZL6ERDDwDeJS41d8NksTlcqHPXaCEXle95ek+F8NjulFaNIYrBkQE5iZE4cL7Ty2wsd8hk7D9Kd52DWgWEa9UYsLvQ0N0v/J6lkt0wWc63UWC6vygpKoCxpQ/vftWId79sBAfIXtwAQF2XBS4OMOg0SIlWTlyRFGAieFZaNM429+LSgiRJtU4hC44MUGLAKNXA8R8atQqlu42I0vP7l2qFuBtcLk5wuZEFxz9svHIGAIADoNOoJbFITZWqNs9mSgmp71JBbOFj7VPSY8MlVWOIBI4MYLvVKgXFU3hq4NBO3Ncwd0N7Hx+UXtMuveyGydDSO4B+uxMatQrT4yOCPRxF8v5XjcJjm9MliUVqqgjxN2Qt9imsxlBJUYFgIT7fZpZU6xRyUUkYVkhp5dxUAN4WHLm7Gqqoi7hfKSkqwKfn2vBFTRcefu9ruDj5uxvYQpWVYIBWQ3szX1NWbkTZngrERITB1O/AjUunS8rdMFlY5l0e3Wt8injtyU/mLarsb1Qq84XuEhKGZTZ8cKIJANBtsaPbYpO9q6F3wI72PisA2lX5kxuWTAcAuDhluBs8dZPI6udrxBa+RZnxAIBFmfGScjdMlmqK9/M77HfbYbahx2IP8mg8kMCRMMzU9/SeCkTq+aJmv915VvauBhZHkRipQ0y4NsijUS7HaruFx0pwN1S19wEgUewPxO4Glml0vq1PUu6GyUIJDf4nUh+G1Bg+gLvS/XcqBchFJXEGV4v826FaWYsbgG44gaCs3Ii/f1EHAFCpgJ9dMUP27gbm1iRXg+8Ruxvy3O6G8xJzN0wGs5UKigaKvKQotJisqGozY3FWfLCHA4AsOLKgpKgAzBulVkmrkNJkqKagP7/C3A2bigsQG6EFxwFXzU+XvbuBLDiBIT/JY8GROyyZIZ4KivodwfInIQsOCRwZUFZuBLMQuzhpFVKaDFUdZMHxJ8zdcFfxTOGmUylzd4PTxaG2kwLTAwGz4NR19cu+SGQ1JTMEDHEmlVQgF5XEYbvx6xZm4O3jjUiK0sve1UA1cPzL4OyGY7Xdwk1HrnOmoasfdicHXZgaGbGUIu5PUmP0iNRpYLY5UdtpxoyU6GAPadIwqx/VTfI/gzOppABZcCSMOLPh7mJ+0eodsGNTcYGsXQ2sMzrVwPE/YguOnKkSWnsYoJZp9qBcUKlUyBXmjXQWq8ngac5K9xp/w+41Ve1muCRiJSYLjoQRZzY4nC7oNGpYHS5cv3g6VCqVrFwNrKbPD1fkoNPdFZ1ZcORe00fKsF2VlMzGk4GsfoElLykKJxtMsp83noQGsuD4m2lxEdBqVLA6+PYwmQnBF5UkcCSMeMEP06iRk2TAuZY+IZ5CTrCaPs09fEZDSrQekfowLysV4XvyRSm/HMfJtlQ9Zd4FFnGquJyhGjiBI0yjRnZiJCpa+1DVbpaEwCEXlYxgu3E5mo1ZgOtrh2oB8JkwSmgfIHWyEiKhUatgtjnRYrIGeziThgROYBmcKi5Husw2dLmLzpE7PDDkSSwDjwSOjPAIHGlMnolSUlSAb+QlAAC+qO4kcRMAdGFqZLl3UnKbN6WiOLPB/YTKyo0odQfbE75HagvVZGBxW+mx4TDoyFkRCHKFVHFpCGMSODIiP8Ud+Ncq35tOUhRf7VIp7QPkQL5M3Q3MrVm66xzquzxF/uTeqkQOMBdVl8UuxMzJDdZFnOK2Akd+krQyqfwqcDo7O7Fu3TrExMQgLi4O69evR1/f6DfZyy+/HCqVyuvnJz/5idcxtbW1uOaaa2AwGJCSkoJ7770XDofDn5ciCeTsomIcON8BAAhTqxTRPkAO5Ml03jC35lPuOlCROg3+dqiWLH8BwKALQ0ZsOAD5CWOG4NZMJoETKDyxW9K41/jVbrdu3To0NTVh165dsNvtuO2223D77bfjtddeG/V9GzZswMMPPyz832Dw+E+dTieuueYapKWlYd++fWhqasItt9wCrVaLRx991G/XIgXYQtXeZ0WPxY5Yg7z6OD25+xza+/jd4J57LsfbxxtkX9NHDuTLOFW8pKgAlW19+NfxRlhsTpTuNpK4CRB5yVFo7BnA+TYzluYkBHs444Jla5YUFQguqrwkytYMFCxGrqG7H/02JyJ0mqCOx28WnNOnT2Pnzp34y1/+gsLCQlxyySV4+umn8frrr6OxsXHU9xoMBqSlpQk/MTExwmsfffQRvv76a7zyyitYuHAhrrrqKjzyyCN45plnYLPJ05Q6XqL0YUiL4XdVUmpoNh7Kyo14cjdvrdGHqTEtPkLYocu5po8ckHuq+Jx0/u+fA7k1A4lQQ0lG9xrm1iwrNwouqlxyawaMhEgdYiP4jTdrkxFM/CZw9u/fj7i4OCxdulR4rri4GGq1GgcPHhz1va+++iqSkpIwb948bNmyBRaLxeu88+fPR2pqqvDc6tWrYTKZcOrUKd9fiMSQaxyO08Xh2oUZAPgFl91o5Nw+QC4wyx/bVcmNd47zGyKNityagSRPgqX3x0K8aTrX0gsA+I+xndyaAUKlUknKTeU3F1VzczNSUlK8PywsDAkJCWhubh7xfd/73veQnZ2NjIwMfPXVV7jvvvtw9uxZvPXWW8J5xeIGgPD/kc5rtVphtXpSZE0m06SuSQrkJ0fh84oO2cVTbFo5E3/aW4l/oREzUryLbtFNx78kROoQb9Ciy2LH+fY+XJARG+whjZuyciO+buL/Xv/wvUUwtvaRWzNACKniMnNtlhQVoG/Agef+cx4A8NK+ahI3ASQ3KdLdHib482bCAuf+++/Hb3/721GPOX369KQHdPvttwuP58+fj/T0dBQVFaGyshL5+fmTOue2bdvw0EMPTXpMUkLOqeIVbqvTYIFD+J+85CgcqenC+TazbAQOcytoNSrYnRwKUqNw1fx0ACCREwDYTry20wKH04UwjXySbr85M1kQOOTWDCxS6kk14Rl7zz334PTp06P+5OXlIS0tDa2trV7vdTgc6OzsRFpa2rg/r7CwEABQUVEBAEhLS0NLS4vXMez/I513y5Yt6OnpEX7q6urG/flSQ9YCxz3mAhI4AUeOgcZOF4cNl+bC7uQQplYh253uS27NwJARG4FwrRp2J4e6rv5gD2dC/MUtblQqkFszwDDXZqUEBM6ELTjJyclITk4e87jly5eju7sbR44cwZIlSwAAe/bsgcvlEkTLeDh+/DgAID09XTjvb37zG7S2tgousF27diEmJgZz584d9hx6vR56vX7cnyllWAxObYcFdqcLWpnsqjiOE+KGyIITePJkGGi8aeVMfHKuDX/+TxVykyK95jrtyP2PWq1CTmIkzjT34nxbn2yqSJeVG7H3XBsA4M7LZ0AXpiaLXwDJlVB7GL+tjnPmzMGaNWuwYcMGHDp0CJ9//jk2btyItWvXIiODDzZtaGjA7NmzcejQIQBAZWUlHnnkERw5cgTV1dV45513cMstt+Cb3/wmLrzwQgDAqlWrMHfuXPzgBz/Al19+iQ8//BC//OUvceeddypGxIxGWkw4DDoNHC4ONR2Wsd8gEZpNA+izOqAR7cSJwCFXy5/RHShakEqiOBjILQOPuTVZDZ+C1CjK1gwgpbvO4YOvmqBSAb0DDnSIikQGo/q4X7f/r776KmbPno2ioiJcffXVuOSSS/Dcc88Jr9vtdpw9e1bIktLpdNi9ezdWrVqF2bNn45577sENN9yAd999V3iPRqPBe++9B41Gg+XLl+P73/8+brnlFq+6OUpGpVLJcrFi8TfZiQbowuRhdVIS4swGl4xcO0LcVjIJnGAgzBuZpIo7XRw2FRfA7M4WLEiJBkBuzUChUatQtqcCUXreOcSEcbDS9P1a6C8hIWHUon45OTngOM+Ey8zMxCeffDLmebOzs/HBBx/4ZIxyJD85EicaemQpcCj+JjhkJRgQplah3+5Es2kAGXERwR7SuDAygZMaHeSRhCZCLRyZWHA2rZyJVtMASncboVZ5xg+QeyoQsN8xcwmeb+vDgfMdQUvTpw5kMkSw4LTK46YDiBYqEjhBQatRIyvRgPNtZpxvM8tC4HAc53FR0bwJKKwi8GUz+XhLsYtK6hWBjYK1OBLh2uBW0g1FSooK8PGZVhyr68Yv/nkCLg5BS9MnX4EMyXff7OViNgYoRVwKyM212dZnhWnAAbUKsglwVQqsIvBHX/O1xdr7rDAN2GVREZgV+KN7TfC4btE0AMFvqkwWHBniseAEP0p9vAgZVMnkaggWeTLrKl7RQjvxYCF2NRh0GlhsTmz74Az+dqhW8kXzmAVnJgWmB42OPr6wrk6jFtL0gzFnSODIkOxEA9QqwDTgQHufDcnR0s4e6zTbhGh6luZOBA7mbhiuG72U3Q3k1gwug+Mp5CBuAFHmXQptpoJBWbkRZXsqhLnCrH5A4OOgyEUlQ8K1GmQm8B3W5eBuYO6paXERMOhIUwca5m44WtMFwGPBkbq7wdhKroZgU1JUADY91CrpB+pyHIdzbssflRYIPOyeIhbCwUzTp9VGZoh34zUdFlS29eEbeYkApLsbp/ib4DJ4J97YM4AnPjyLP3xcIekdubGFMu+CTVm5ESyz2sUhaK6G8dLeZ0NPvx1qlceVTwQOp4sb9p7C/h/oNH2y4MgMthvv6eddPiyTSsq7cRI4wYftohhSFzeAuLQAuRqCAbun3HRRJgAgSh8m+WJ5zD2VlWCguK0gsGmUe0pJUUHAN98kcGQGW6iO1HQD4F1Uw5kFpQT1oJIGJUUFYPJXo1ZJcq4wKG4ruIjvKb+4eg4AoM/qwE8vz5e0yPHEbZEoJshFJUtKigrQ0N2PNw7X4ZNzbfjkXJtkxQ0AVFDapiQoKzeCGYidLk7S7gZmvZkeT3FbwWCwqyEjNhyNPQO4fFYKwrUayVYEZinilEFFACRwZMt9a2bjjcN8V3StRrq7cbPVgcaeAQAkcIIJ25GvnJuCXV+3IivBIOkGhCzAmKx+wWGwK2FWWjQaewZwttkkyfnCYBYcCjAmAHJRyZZXDtQIj+1OTrImY5bllRSlQ5xBF+TRhCZid8OdV/CLk8XmlHQDQhZgTKJYGsxKiwEAnHVbSKSId+VrclERZMGRJWzByk4woKbTguI5KZLdjVOAcfARuxssNgdUKr4y7c3LsoTXpQYFGEuL2Wn893C2WboCp8NsQ5fFDhVlUBFuSODIDPFu3Gx14NlPzyM9NkLYjQPBFzkslb2kqGBIsTapprIrGfHv2qALQ3aCAdUdFpxt7g36XBkJoQYOuRokwUx3s9Mzzb2SrZ7O4m8y4w2I0FEGFUEuKtkh3o3PTmc3HZOQXSWF3ThLZS8rN3osOMlRkk5lDyVmu90NZ5pNQR7J8JgG7Ggx8aXeyfInDfJTIqFRq9A74ECTO6ZOalRQiwZiEGTBkRni3ficdPdC1cTvqqSyGxcXlos3aAEAJxtN+MeReklne4UKs9OjsfNUM85I1N3AFqq0mHDEhGuDPBoCAPRhGuQlRcLY2oezzb2S7EbvabJJbk2Chyw4MiYvKQpajQq9Vgfqu/qDPRwvSooKUFI0A10WOwCQuJEQLJ5CShacUlGwc8WgUvtl5UaUut2vRPCYxeJwJBpozALTyYJDMEjgyBhdmFrYrZxuks5ixVg1N014rNOoSdxIBOaiMrb0weF0BXk0PGK3prgHFbk1pYPUA42NFJhODIIEjsyZk+YJ/pMabEeuAmBzuiSZjhyKZCUYEKHVwOpwobrDEuzhAPBuyLfr6xYAQG2nRdIVukONWULsljTuNWKrX0efFZ1mG1QqjzAmqx9BAkfmsDgcqVlwysqN+Mi9UK2/JFfSNVdCDbVahZkSdFMxkcNEV/npVhI3EoJZcCpb+2CXgOXP2+rnqXz95/+cJ6sfAYCCjGWPJ5NKGrsqwJPKPi0uHA3dA5ibEYPrF08HAMmksoc6c9Ki8WVdN8429+JbFwZ7NB5+8I1sYY5IuUJ3KDItLgIGnQYWmxPV7WYUpAbXFSROZrhydjIAQKtWk9WPECALjsxhFpzqDjMsNkeQR8PjdHHYVFyAbneA8QUZsQAgqVT2UIcFjJ5uko4wBoDffHBaeCzlCt2hiFqt8qqHIwXYPWXPmTYAwPl2M4kbQoAEjsxJitIjKUoPjpNO8N+mlTNx7cJpMNuc0IepkZ/s6QZdUlRARf4kgBRr4ZSVG/GPI/UAgGvmp5NbU4IwN9U5CWVSlRQVgDmjWIFRggBI4CiCOenS242fauQXztlp0QjT0DSTGmyhqu/qR++APcij8bg1mWVpbkaMV+AxiRxpMEuCSQ1P7T4HZhN2usjqR3iglUcBCAX/JLQbP9XYA4BfqAjpER+pQ2qMHoA0duOsQrfdwQevzptGbk0pMktiqeJl5UaU7uYFTbhWjbuLCkgQEwIUZKwAPBYcKQkcfixz3fE3hPSYnRaDFlMbTjf1Ykl2QlDHsmnlTPRZHSjdzQcYXyASxuRykA7MtVnbaYHZ6kCkPnhLCLP6XTUvDf8+2Yy56TG4e+VMqN3ZVQDNnVCHLDgKQIincLdskAJfu8XWBWTBkSxSK9x2uskEjgPSY8ORFKUP9nCIYUiI1CE5WhqWP2b1Y20j5pPVjxgEWXAUQH6yd8uGzARDUMfT2juAtl4r1CpgThoJHKkibtYqBU428G7NC8jqJ0lK3bVlZqdFo63XirPNvViUFQ+At6Y4XVxAEwjYZ3332f0APG5NgCw3BA9ZcBSALkyN/GS+/4oUgv+Yeyo3KRIROk2QR0OMxGxRZVopWP5ONpDVT8qwwnpmK1+OgvWkCmY7DZeLw9fu+8386SSMCW9I4CiEuRKqaMxuOLQTly6lu85h58lmhKlV6B1woLFnQHgtWGXuWWC6eCdOSAfm+jla2w2Ad20ycROs2jPVHWb0WR0I16oxI5mabBLekMBRCFJyN3gEDu3EpYpGrcJT5UbERmgBAGfcwjhYu/EBu1Motz9vGs0bqVJSVIDvFWYBAPZVdgS9avAJt1tzTnoMlaMghkAzQiF4elJJwUVFsRRSh+3GO8w2ALybKpi78bPNvXC6OCRG6pAWEx7QzyYmxoPfmis8DnY7DRa3NY/uNcQw+FXgdHZ2Yt26dYiJiUFcXBzWr1+Pvr6+EY+vrq6GSqUa9ufNN98Ujhvu9ddff92flyJpSnedw2fGdgBDWzYE2t3QO2AXmiVSDRxpU1JUgEtmJAIAfv/R2aDuxk+K6iapVNQkUco89+l54XGw22kwC858cmsSw+BXgbNu3TqcOnUKu3btwnvvvYdPP/0Ut99++4jHZ2ZmoqmpyevnoYceQlRUFK666iqvY1988UWv46677jp/Xoqk0ahVePbT8zDoNF4tG4LhbmBBzumx4UiI1AXsc4nJ8aNL8wAALg7QadRB242zwHSKv5E27J6yMJP/nhZnxQWtsJ7LxeFUA80bYmT8liZ++vRp7Ny5E4cPH8bSpUsBAE8//TSuvvpqPPHEE8jIyBjyHo1Gg7S0NK/n/vnPf+K73/0uoqK8A8ji4uKGHBuqiLvqArzI+I+xPSg78lNCqi9Zb+TAoapO4bHN6UJZuTEoIucUuRokj9iFmZkQgeNvfAkAQjsNILDp2TWdFvRaHdCFqVGQSgHGxFD8ZsHZv38/4uLiBHEDAMXFxVCr1Th48OC4znHkyBEcP34c69evH/LanXfeiaSkJCxbtgwvvPCCJNJcg0lJUQGWZvM1Kf6/f54ImruBKhjLh7JyI/64txLxBj7Q+LqFGUHZjdudLpx2W/4owFi6sMJ6JUUFWJjJ32tONprwk8vyg1JYTxxgrKUAY2IY/GbBaW5uRkpKiveHhYUhISEBzc3N4zrH888/jzlz5mDFihVezz/88MO48sorYTAY8NFHH+GnP/0p+vr6UFJSMux5rFYrrFar8H+TKfiZRv5g/SW5+KKmK6juBkHgpNNCJWXEu/HaTgv+caQe0+IjgrIbr2zrg83hQnR4GLKCXKSSGBlxEb+cRAPiDFp0W+w402wKqtVvPoliYgQmLHvvv//+EQOB2c+ZM2emPLD+/n689tprw1pvHnjgAVx88cVYtGgR7rvvPvz85z/H7373uxHPtW3bNsTGxgo/mZmZUx6fFDlW1y08Zu4Gf1Mq2vHbHC4YW/md+AUZMUGrp0KMjXg3vthdjfZYbXdQytyzAn9z0ynAWC6oVCosmB4HgJ83wYACjImxmLAF55577sGtt9466jF5eXlIS0tDa2ur1/MOhwOdnZ3jip35xz/+AYvFgltuuWXMYwsLC/HII4/AarVCrx/aw2bLli3YvHmz8H+TyaQ4kVNWbsRzn55HTHgYTAMO3LB4WkB24hpRY7uiOSmwOznERmjx1tF6lO42YnMAS7cT40e8G1+UFQcA+LKuG04XF/DduJDqSwuVrFiUFYdPzrXheF03fhjgz+Y4juYNMSYTFjjJyclITk4e87jly5eju7sbR44cwZIlSwAAe/bsgcvlQmFh4Zjvf/755/Htb397XJ91/PhxxMfHDytuAECv14/4mhIQuxsq2/rwr+ONyEwwBMTdIA5wZjecmPAwQdxQTxjpMzM1GlH6MPRZHTjb3BuQ9H7W16ikqEBUwZj/3GD0NSImzsLMOADAcZHlOFDUdlpgGnBAp1GjICU64J9PyAO/RWbNmTMHa9aswYYNG3Do0CF8/vnn2LhxI9auXStkUDU0NGD27Nk4dOiQ13srKirw6aef4kc/+tGQ87777rv4y1/+gpMnT6KiogJ/+tOf8Oijj+JnP/uZvy5F8ojdDSzQ+EhNV8DcDexzPvq6BQBQ19VP4kZGaNQqLHCn/R6r6wrYZ27fdQ5P7T4nVL6elxEb1L5GxMRgAqeq3Ywud8HIQMHcU7PTo6ELowBjYnj82k381VdfxcaNG1FUVAS1Wo0bbrgBZWVlwut2ux1nz56FxWLxet8LL7yA6dOnY9WqVUPOqdVq8cwzz2DTpk3gOA4zZszA9u3bsWHDBn9eiqQR73QXZ3viKQLpbigpKkDprnPgAGFnTsiHxVnx+LyiA0drurGuMNvvnze4tEG4Vo33TzThSbL8yYY4gw65SZGoajfjeH03rpiVMvabpoDY6ndikHuKrH7EcPhV4CQkJOC1114b8fWcnJxh07sfffRRPProo8O+Z82aNVizZo3Pxqg0ZqVGI1KnQZ/VgXMtvUILB3/zuw/PgH2TThcXtHoqxORgcTjHagNjwQF4kXOmuRcfnGiC1e4icSNDFmXG8QKn1v8CRxzvd1IUYCx20ROEGL8KHCLwhGnUWJQVj88q2nGkpisgAqes3IhnPq4EwKePXr94elAKfxGTZ5G7rsl5t7shPkBVqJOj+M/hENxKysTkWJgVh7eONQQkDkds9dO73VInG3rw6sFaEsbEsJDzUoEsFsXh+Bu2e1qaw39mYW6iEJMTrBLuxMSJj9QhLykSQGCDRt/7qgkAEKZWBay0AeE7xIHGgSi2WlJUgPWX5MLqcAEAiRtiVEjgKJAlARQ4LMDZ4eRvbstyEwAgKPVUiKmxyF0P52iA3FSP/fu00M388P9XTKJYhsxOi4EuTI2efjuq2s0B+UxxGxiy+hGjQQJHgSzKioNKxadStvYO+PWzNq2cifWX5Ao+8cK8BOG1kqICCvqTESwOJxACp6zciB2f8F2pZ6dFIz5SR5Y/GaILUwuF9gJl+fvLf6oAAGpV4AqaEvKEBI4CiQnXYlYqXxviaACsOEdru+BwcZgWF4Hp8VRqX66wisZf1vX43fLmdHG4cDq/MC7PTxSeJ8uf/AhkPZyndp/D1018WYGX/2cZCWJiVCjIWKEszo7HmeZeHKnpwpp56X79rIPn+Y7UhbkJYxxJSJlZaZ4MPGNrL2an+S9AfdPKmXjvq0YAwDfyEr1eI5eDPGBp28MJHH+kbZeVG1G6mxcyOo0aS7MTcGkBXwiWkhqI4SALjkJZkhW4OJxDVW6Bk0cCR87wBf/iAABHa7r9+lmtpgFUtpmhUgHfyE0c+w2E5GBp20zYfN1owoDd6bdijU4Xh6LZfCr6oqw4ROg0AMjqR4wMWXAUCstqOtnA33TCtRq/fM6A3Snc4AppoZItbDe+KCsO+yo7cKy2C98rzALgn934Abconpseg1iD1mfnJQKHOG3boNPAYnPiV++cwuuH6/yS2bRp5Uz89NUjAICLZyQNOxaCEEMWHIWSlWBAUpQONqdLCAD2B8dqu2FzupASrUd2IsXfyBW2G2/o6gfgCTT21258f2UHgKHuKUJeMOuJxeYEAL+JGwBwuThh3lw8g+YNMTYkcBRI6a5zeHpPhRA0KnZTlZUbUer2V/sCj3sqESoV9Q+SK2yhevs4HxdT2WbG4zvPCBVifb1gHTjPL1TLSeDInpKiAkEAq1T+s6Z83WRCl8WOSJ0GF06P88tnEMqCBI4CYbvxATu/q2ICxx+78YNV/EJFAcbyh4kcxh/3VvpF3DT3DKCq3Qy1CriI5o3sYS5MAOA44IkPz/rlc/ZVtgPgN1NaDS1dxNjQLFEgbKH61MjfEI7UdOGp3ed8vhu3OVyCK4MEjjIoKSoA079qP+3GmfXmgoxYxEZQ/I2cYZumTcUFyHG7qP/wcYVf0rY/r+DnzYp8svoR44MEjkIpKSpAyZUzAAAdZhtK/dDI8ERDNwbsLiRG6jAjJcpn5yWCR1m5ESwZxcXxdUd8DYujWE4LlawRN7m8q3gmrpydCoCvNOzr2jQ2h0twhw8OMCaIkSCBo2A2r5oFFhajUal8Im5KRTeuA+76N8tyE6BSqXwe30MEFrZglRTNQJSeT7As3W30+W58/3kWYExWPznD2rSw+0rRHD6Fu8VkxabiAp+mbR+v60a/3YmESJ1QxJQgxoLSxBVMWbkRrP+dk+NQVm6csshh8T2AJ7ZnWW6C126OkB/i76+kqACVbWa8/1UTluXET7mIGktBLykqQEN3P2o7LdCoVbgoJ8EvKehEYBj8nV2Uk4AofRja+6y4fFaKUFPJF3xewbvbl+cnQu3jjD5CuZAFR6GwBeuOy/KFoGJfmI3F/YJY0F9lm9lv2TZEYBi8G181l3c3dFnsUy6ixkRxWbkRB9zuqXnTYvHi59V+SUEngoMuTI1vzuTdR+VnWn16biE9PJ/cU8T4IQuOAhm8G/+qoRufV3Tg0oIkn5Q0LykqQH1XP/7+RR0A4JUDNSRuZM7g3fjls1IQplbB2NqHby/IQE5S5KTPLS4IN9fdCVqrUZEoViBXzk7FByeasedMy5SsuWKrn8XmwLE63lp88YxEsvoR44YsOApk8G6c9aLqHXD4rKS5eNet06hpkVIYsRFaofXGrq9bpny+kqICbCouwNeNfKPEL6q7SNwokMtnJUOl4iuoN/cMTPo8YqvfoapO2J18M9+3jzWQ1Y8YNyRwFMimQQvH6rmpUKn4QL0bl06f8s5nwO7EW0frAQBhahVsThd181UgK+fwbipfCBwAuGxWivBYq/FN0DshLZKi9ELzzY/PTt5NJXaF/2FPBQAgzqD1SzYooVxI4IQAKTHhQvPND082T/l89775JawOF6L0YTj366uEGxGJHGVR7I7D+aKmEx191imf74F/nQTA19exOzmaLwqFNcQsPz21OBxm9fvCncxwqtFE4oaYECRwQoQ189IAAP+eosApKzfi3a+aAADrvpEFtdtPTiJHeUyPN2Buegxc3NSDRn//0VmcqOd7or3yo0KaLwqldNc5NJt4Mfx5RbtQTR2YXJuY5aKgYrL6EROFBE6IsPoCXuAcru5E+xR242arQ6h0e/2i6cLzTOT4svYFEVxKd51DbASfhzDYTTWRxaqs3Iin3W6GzIQIfCM3kUSxQtGoVXjlQA2i9GHotzuFmkeTbRPzIFn9iClAWVQhQmaCAfOnxeJEQw92fd2Cm5dlTeo80xMMcHHA3PQYzErzLrhFuytloVGrsN9dzPE/xjb025yI0GkmXPPI6eKQGR+Buq5+3LgkU6hjwuYLiWLlIM6YA4A9p1txor5nUhlzv/33GZxp7gUAvH3nxdh7ts0nWaBE6EACJ0Qo3XUOMe7d+L9PNnsJnImkXf7THVx8/eJp/hkoIRlKigrAcRxKdxsxYHfhs4p2nG4yTXixumHxdDxVboRKBdywZLrXa7RQKY+SogKcb+vD28cb8f8O1ADAhMVNWbkRf/qkEgAwf1osLpweJ3QQJ5FDjBdyUYUIGrVKaFa3r6IdPRY7gImZjqvbzTha2w21Cvj2ggy/jpeQBncVz8Qid1bMj//fF5Paif/jCF8v6ZIZSZgWF+GPYRISY9v1FwqPw9QTj51xOF2Iczdi/V6hZzNGrnBiIpDACRHYjQEAHC4O5WdahhQEHEzpoPiIfx5rAABcUpCM1w/XUd+pEOHe1bMA8M03Jxro6XRx+McR3ur33aWZfhkfIT3+/J/zwmOHi5tw09bCvER099sRpQ8bspkqKSqgIn/EuCCBE0KUFBUIDQ7v+fuXY+7GxcW2OI7D28d5gROp01CxrRDicHWn8HisQM/BovjzinY09gwgNkILY0svieIQQNwmZrJNW189yLu2vrNoGiL1FElBTA4SOCFG6U0LAQAcIJRCHwlxpsuWt06gpsMCrUaFf59spnoUIUJZuRGlu434b3fsjFo1ek8zsSgGILTzyE0yoGxPBYlihSO2Ct931Wz8z8U5AIDESN2o80YsjFtNA/joFJ+1973CrEmllxMEQEHGIcebX9QLj50uDr/78AzuXT17xOMHZ0XYnRyJmxBhsAuzo8+Kj8+2oSAlasRAT/F8GbA7hYXqeF0PzZsQYHCbmPWX5OHFfdXoMNtwzfz0EWNnmDAGeBHtcHFYnBWHXV+3TChjjyDEkMAJIdiCdVdRAd79shHn28145uNK6MM0oy48s0Xp4JMJGCTkyeDF6v6r5uCTc20wtvZh7UWZIy5Wg0UxMPEsGkKeDI6NiTVocUF6DA5UdeJsSy/Kbl7k9frgDM7tu84hOpxfllKiw6khKzElyEUVIoh345tWzsSF02MBAKphXA5ik3BthwUb/3YMgGdnRcW2QoPBPc0+ONGEuel8N3Bjax/uLva8NtiNwI4DxnaFEspmSQ7fJqaitQ/vfdUoPD84g7OkqACXzUxG74ADALDzFLnCianhN4Hzm9/8BitWrIDBYEBcXNy43sNxHB588EGkp6cjIiICxcXFMBq9F9POzk6sW7cOMTExiIuLw/r169HX1+eHK1AWg3fjeclRAACOA6bFRcDpcqF01zl8788HhJvOgN2J/96xDzZ336k7r5hB1WdDGI1ahZONJoSpVThS04UPT7UMmTMAcKK+Bz955QgAQAV+7tF8CV3uXT0bF89IBAD86p1T+P1HZ4U5I74nvXKgBp+caxPep9OoSdwQU8JvAsdms+HGG2/EHXfcMe73PP744ygrK8OOHTtw8OBBREZGYvXq1RgYGBCOWbduHU6dOoVdu3bhvffew6efforbb7/dH5egKAbvxkuKCvCjS3MBAA3d/chNisLh6k7sq+zAiny+lP53n92P1l4rwtQq9Fkd0LpvOCRyQhP23Tvcrqnf7jyDg1UdXnOmsbsfa5/bLxxD84UAgGd/sBThWjW6LHb8YU+F15wBgNcO1uKXb58Ujtdp1LA5XTRniCmh4jjOrxWTXnrpJdx9993o7u4e9TiO45CRkYF77rkH//u//wsA6OnpQWpqKl566SWsXbsWp0+fxty5c3H48GEsXboUALBz505cffXVqK+vR0bG+IrPmUwmxMbGoqenBzExMWO/QcGs+/MBfF7ZIfw/PzkSlW1mrJqbio9E/YcGm4onUv2YUBa/+/AMnvm4Uvh/SrQerb1W/PTyfPz9izq099kAAHdekS8EsI9Vc4lQPn/5z3n8+v3TXs/deUU+shIMuO//TgjPbSouwF3FM2nOEMMykfVbMkHGVVVVaG5uRnFxsfBcbGwsCgsLsX//fqxduxb79+9HXFycIG4AoLi4GGq1GgcPHsR3vvOdYAxd1rxw20WY9cudwv8r28wAMKq4AahMeihz7+rZePaT84KVprWXb976x70e0fOjS3O9svOo7xRhtjqGPCcWyoBH3ABDg9XpnkNMFMkInObmZgBAamqq1/OpqanCa83NzUhJSfF6PSwsDAkJCcIxw2G1WmG1ejpom0wmXw1b9jz7CV9xVKNWCU0R+6wOdLlbOZAfnBhMWbkRDhcHrUYFu5PD/GmxqO+yCHMmTK3CL6+ZO+R9NI9CF1ZPafPKmbjzihn42WtH8cFJ73u2WNwwSBgTU2FCMTj3338/VCrVqD9nzpzx11gnzbZt2xAbGyv8ZGZSyXjA221Q+ejV2LxyJuq6+oW0cPKDE4MRzxnjb/g5c6KhR+gsr9WoKNOO8GKwq0mjVuGP31+C5e6q6ip37UeVavgikNSagZgsE7Lg3HPPPbj11ltHPSYvL29SA0lLSwMAtLS0ID09XXi+paUFCxcuFI5pbW31ep/D4UBnZ6fw/uHYsmULNm/eLPzfZDKFvMgZzr9dUlSAA+c9QaOvbfiGcBx7nQhdaM4Qk2FwBifAz6X95zuxIj8RF+UkeBX6ozlD+IoJCZzk5GQkJyf7ZSC5ublIS0tDeXm5IGhMJhMOHjwoZGItX74c3d3dOHLkCJYsWQIA2LNnD1wuFwoLC0c8t16vh16v98u45cpINx22UF2Uw++uyA9OMGjOEJNhsPVltOBhmjOEL/FbDE5tbS06OztRW1sLp9OJ48ePAwBmzJiBqCi+Bsvs2bOxbds2fOc734FKpcLdd9+NX//61ygoKEBubi4eeOABZGRk4LrrrgMAzJkzB2vWrMGGDRuwY8cO2O12bNy4EWvXrh13BhXBM5zJd7gFDCA/OMFDc4bwBTRniEDhtzTxW2+9FS+//PKQ5z/++GNcfvnl/IerVHjxxRcFtxfHcdi6dSuee+45dHd345JLLsEf//hHzJzpubF2dnZi48aNePfdd6FWq3HDDTegrKxMEE3jgdLECYIgCEJ+TGT99nsdHClCAocgCIIg5MdE1m/qRUUQBEEQhOIggUMQBEEQhOIggUMQBEEQhOIggUMQBEEQhOIggUMQBEEQhOIggUMQBEEQhOIggUMQBEEQhOIggUMQBEEQhOIggUMQBEEQhOLwWy8qKcOKN5tMpiCPhCAIgiCI8cLW7fE0YQhJgdPb2wsAyMzMDPJICIIgCIKYKL29vYiNjR31mJDsReVyudDY2Ijo6GioVCqfnttkMiEzMxN1dXWK7HNF1yd/lH6NdH3yR+nXqPTrA/x3jRzHobe3FxkZGVCrR4+yCUkLjlqtxvTp0/36GTExMYqduABdnxJQ+jXS9ckfpV+j0q8P8M81jmW5YVCQMUEQBEEQioMEDkEQBEEQioMEjo/R6/XYunUr9Hp9sIfiF+j65I/Sr5GuT/4o/RqVfn2ANK4xJIOMCYIgCIJQNmTBIQiCIAhCcZDAIQiCIAhCcZDAIQiCIAhCcZDAIQiCIAhCcZDAmSC/+c1vsGLFChgMBsTFxQ17TG1tLa655hoYDAakpKTg3nvvhcPhGPW8nZ2dWLduHWJiYhAXF4f169ejr6/PD1cwMfbu3QuVSjXsz+HDh0d83+WXXz7k+J/85CcBHPn4ycnJGTLWxx57bNT3DAwM4M4770RiYiKioqJwww03oKWlJUAjHj/V1dVYv349cnNzERERgfz8fGzduhU2m23U90n9+3vmmWeQk5OD8PBwFBYW4tChQ6Me/+abb2L27NkIDw/H/Pnz8cEHHwRopBNn27ZtuOiiixAdHY2UlBRcd911OHv27Kjveemll4Z8X+Hh4QEa8cT41a9+NWSss2fPHvU9cvr+hrufqFQq3HnnncMeL4fv7tNPP8V//dd/ISMjAyqVCm+//bbX6xzH4cEHH0R6ejoiIiJQXFwMo9E45nkn+nc8UUjgTBCbzYYbb7wRd9xxx7CvO51OXHPNNbDZbNi3bx9efvllvPTSS3jwwQdHPe+6detw6tQp7Nq1C++99x4+/fRT3H777f64hAmxYsUKNDU1ef386Ec/Qm5uLpYuXTrqezds2OD1vscffzxAo544Dz/8sNdYf/azn416/KZNm/Duu+/izTffxCeffILGxkZcf/31ARrt+Dlz5gxcLheeffZZnDp1CqWlpdixYwd+8YtfjPleqX5/b7zxBjZv3oytW7fi6NGjWLBgAVavXo3W1tZhj9+3bx9uvvlmrF+/HseOHcN1112H6667DidPngzwyMfHJ598gjvvvBMHDhzArl27YLfbsWrVKpjN5lHfFxMT4/V91dTUBGjEE+eCCy7wGutnn3024rFy+/4OHz7sdW27du0CANx4440jvkfq353ZbMaCBQvwzDPPDPv6448/jrKyMuzYsQMHDx5EZGQkVq9ejYGBgRHPOdG/40nBEZPixRdf5GJjY4c8/8EHH3BqtZprbm4WnvvTn/7ExcTEcFarddhzff311xwA7vDhw8Jz//73vzmVSsU1NDT4fOxTwWazccnJydzDDz886nGXXXYZd9dddwVmUFMkOzubKy0tHffx3d3dnFar5d58803hudOnT3MAuP379/thhL7l8ccf53Jzc0c9Rsrf37Jly7g777xT+L/T6eQyMjK4bdu2DXv8d7/7Xe6aa67xeq6wsJD78Y9/7Ndx+orW1lYOAPfJJ5+MeMxI9yMpsnXrVm7BggXjPl7u399dd93F5efncy6Xa9jX5fTdcRzHAeD++c9/Cv93uVxcWloa97vf/U54rru7m9Pr9dzf/va3Ec8z0b/jyUAWHB+zf/9+zJ8/H6mpqcJzq1evhslkwqlTp0Z8T1xcnJdFpLi4GGq1GgcPHvT7mCfCO++8g46ODtx2221jHvvqq68iKSkJ8+bNw5YtW2CxWAIwwsnx2GOPITExEYsWLcLvfve7UV2KR44cgd1uR3FxsfDc7NmzkZWVhf379wdiuFOip6cHCQkJYx4nxe/PZrPhyJEjXr97tVqN4uLiEX/3+/fv9zoe4P8m5fBdAfz3BWDM76yvrw/Z2dnIzMzEtddeO+L9RgoYjUZkZGQgLy8P69atQ21t7YjHyvn7s9lseOWVV/A///M/ozZ2ltN3N5iqqio0Nzd7fUexsbEoLCwc8TuazN/xZAjJZpv+pLm52UvcABD+39zcPOJ7UlJSvJ4LCwtDQkLCiO8JFs8//zxWr149ZrPS733ve8jOzkZGRga++uor3HfffTh79izeeuutAI10/JSUlGDx4sVISEjAvn37sGXLFjQ1NWH79u3DHt/c3AydTjckBis1NVVy39dgKioq8PTTT+OJJ54Y9Tipfn/t7e1wOp3D/o2dOXNm2PeM9Dcp9e8KAFwuF+6++25cfPHFmDdv3ojHzZo1Cy+88AIuvPBC9PT04IknnsCKFStw6tQpvzcWniiFhYV46aWXMGvWLDQ1NeGhhx7CpZdeipMnTyI6OnrI8XL+/t5++210d3fj1ltvHfEYOX13w8G+h4l8R5P5O54MJHAA3H///fjtb3876jGnT58eMxBOTkzmmuvr6/Hhhx/i73//+5jnF8cPzZ8/H+np6SgqKkJlZSXy8/MnP/BxMpHr27x5s/DchRdeCJ1Ohx//+MfYtm2bZEupT+b7a2howJo1a3DjjTdiw4YNo7432N8fwXPnnXfi5MmTo8aoAMDy5cuxfPly4f8rVqzAnDlz8Oyzz+KRRx7x9zAnxFVXXSU8vvDCC1FYWIjs7Gz8/e9/x/r164M4Mt/z/PPP46qrrkJGRsaIx8jpu5MbJHAA3HPPPaMqbADIy8sb17nS0tKGRIKz7Jq0tLQR3zM4sMrhcKCzs3PE90yVyVzziy++iMTERHz729+e8OcVFhYC4C0IgVggp/KdFhYWwuFwoLq6GrNmzRryelpaGmw2G7q7u72sOC0tLX77vgYz0etrbGzEFVdcgRUrVuC5556b8OcF+vsbiaSkJGg0miEZa6P97tPS0iZ0vFTYuHGjkHAw0Z28VqvFokWLUFFR4afR+Y64uDjMnDlzxLHK9furqanB7t27J2z1lNN3B3jWtZaWFqSnpwvPt7S0YOHChcO+ZzJ/x5PCZ9E8IcZYQcYtLS3Cc88++ywXExPDDQwMDHsuFmT8xRdfCM99+OGHkgoydrlcXG5uLnfPPfdM6v2fffYZB4D78ssvfTwy3/PKK69warWa6+zsHPZ1FmT8j3/8Q3juzJkzkg0yrq+v5woKCri1a9dyDodjUueQ0ve3bNkybuPGjcL/nU4nN23atFGDjL/1rW95Pbd8+XLJBqm6XC7uzjvv5DIyMrhz585N6hwOh4ObNWsWt2nTJh+Pzvf09vZy8fHx3FNPPTXs63L7/hhbt27l0tLSOLvdPqH3Sf27wwhBxk888YTwXE9Pz7iCjCfydzypsfrsTCFCTU0Nd+zYMe6hhx7ioqKiuGPHjnHHjh3jent7OY7jJ+e8efO4VatWccePH+d27tzJJScnc1u2bBHOcfDgQW7WrFlcfX298NyaNWu4RYsWcQcPHuQ+++wzrqCggLv55psDfn0jsXv3bg4Ad/r06SGv1dfXc7NmzeIOHjzIcRzHVVRUcA8//DD3xRdfcFVVVdy//vUvLi8vj/vmN78Z6GGPyb59+7jS0lLu+PHjXGVlJffKK69wycnJ3C233CIcM/j6OI7jfvKTn3BZWVncnj17uC+++IJbvnw5t3z58mBcwqjU19dzM2bM4IqKirj6+nquqalJ+BEfI6fv7/XXX+f0ej330ksvcV9//TV3++23c3FxcULm4g9+8APu/vvvF47//PPPubCwMO6JJ57gTp8+zW3dupXTarXciRMngnUJo3LHHXdwsbGx3N69e72+L4vFIhwz+Bofeugh7sMPP+QqKyu5I0eOcGvXruXCw8O5U6dOBeMSRuWee+7h9u7dy1VVVXGff/45V1xczCUlJXGtra0cx8n/++M4frHOysri7rvvviGvyfG76+3tFdY6ANz27du5Y8eOcTU1NRzHcdxjjz3GxcXFcf/617+4r776irv22mu53Nxcrr+/XzjHlVdeyT399NPC/8f6O/YFJHAmyA9/+EMOwJCfjz/+WDimurqau+qqq7iIiAguKSmJu+eee7xU/Mcff8wB4KqqqoTnOjo6uJtvvpmLioriYmJiuNtuu00QTVLg5ptv5lasWDHsa1VVVV6/g9raWu6b3/wml5CQwOn1em7GjBncvffey/X09ARwxOPjyJEjXGFhIRcbG8uFh4dzc+bM4R599FEva9vg6+M4juvv7+d++tOfcvHx8ZzBYOC+853veIkGqfDiiy8OO1/Fxls5fn9PP/00l5WVxel0Om7ZsmXcgQMHhNcuu+wy7oc//KHX8X//+9+5mTNncjqdjrvgggu4999/P8AjHj8jfV8vvviicMzga7z77ruF30dqaip39dVXc0ePHg384MfBTTfdxKWnp3M6nY6bNm0ad9NNN3EVFRXC63L//jiOt8AD4M6ePTvkNTl+d2zNGvzDrsPlcnEPPPAAl5qayun1eq6oqGjItWdnZ3Nbt271em60v2NfoOI4jvOdw4sgCIIgCCL4UB0cgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUBwkcgiAIgiAUx/8P37BIdo6sL4kAAAAASUVORK5CYII=",
|
||
"text/plain": [
|
||
"<Figure size 640x480 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"filenames": {
|
||
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/linalg_44_1.png"
|
||
}
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"%matplotlib inline\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"from scipy import sparse\n",
|
||
"eye = np.eye(4)\n",
|
||
"print(eye)\n",
|
||
"sparse_mtx = sparse.csr_matrix(eye)\n",
|
||
"print(sparse_mtx)\n",
|
||
"x = np.linspace(-10,10,100)\n",
|
||
"y = np.sin(x)\n",
|
||
"plt.plot(x,y,marker='x')\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ed00bf63",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Other Matrix and Vector Operations\n",
|
||
"\n",
|
||
"The following examples show how to compute various quantities like the **mean** value of a matrix or a vector and how to use functions like **reshape** and **ravel**. These are all useful quantities when scaling the data and preparing the data for various machine learning algorithms and when calculating quantities like the mean squared error or the variance."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"id": "a12e8986",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"The test matrix:[[ 1. 2. 3.]\n",
|
||
" [ 4. 5. 6.]\n",
|
||
" [ 7. 8. 9.]\n",
|
||
" [10. 11. 12.]]\n",
|
||
"This is the total mean summed over all elements:6.5\n",
|
||
"This is the mean for each column:[[5.5 6.5 7.5]]\n",
|
||
"This is the mean value for each row:[[ 2.]\n",
|
||
" [ 5.]\n",
|
||
" [ 8.]\n",
|
||
" [11.]]\n",
|
||
"This is the mean value for each row with keepdims false:[ 2. 5. 8. 11.]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"\"\"\"\n",
|
||
"Simple code that tests various numpy functions\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"# Simple test-matrix of dim 3 x 4\n",
|
||
"a = np.array([ [1, 2, 3], [4, 5, 6], [7, 8, 9],[10, 11, 12]],dtype=np.float64)\n",
|
||
"print(f\"The test matrix:{a}\")\n",
|
||
"# This is the total mean summed over all elements, which here has to be 6.5\n",
|
||
"print(f\"This is the total mean summed over all elements:{np.mean(a,dtype=np.float64)}\")\n",
|
||
"# This is the mean for each column, it returns an array with the mean values for each column. It returns a row-like vector\n",
|
||
"print(f\"This is the mean for each column:{np.mean(a, axis=0, keepdims=True,dtype=np.float64)}\")\n",
|
||
"# This is the mean value for each row, it returns an array via the keepdims option which is a column-like vector if\n",
|
||
"# keepdims=True. Else it return a row-like vector\n",
|
||
"# Try setting keepdims=False\n",
|
||
"print(f\"This is the mean value for each row:{np.mean(a, axis=1, keepdims=True,dtype=np.float64)}\")\n",
|
||
"# We print then the mean value for each row by setting keepdims=False\n",
|
||
"print(f\"This is the mean value for each row with keepdims false:{np.mean(a, axis=1, keepdims=False,dtype=np.float64)}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "596d78a6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"Another useful function is the **ravel** function, which returns a flattened array as shown in the example here."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "fe01c225",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"editable": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Flatten the matrix:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n",
|
||
"Reshape the matrix to a one-dim array:[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n",
|
||
"[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]\n",
|
||
"[ 1. 4. 7. 10. 2. 5. 8. 11. 3. 6. 9. 12.]\n",
|
||
"[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Ravel return a contiguous flattened array.\n",
|
||
"print(f\"Flatten the matrix:{np.ravel(a)}\")\n",
|
||
"# It is the same as reshaping the matrix into a one-dimensional array\n",
|
||
"print(f\"Reshape the matrix to a one-dim array:{a.reshape(-1)}\")\n",
|
||
"# ‘C’ means to index the elements in row-major, C-style order, with the last axis index changing fastest, back to the first axis index changing slowest.\n",
|
||
"# ‘F’ means to index the elements in column-major, Fortran-style order, with the first index changing fastest, and the last index changing slowest \n",
|
||
"print(np.ravel(a, order='F'))\n",
|
||
"# When order is ‘A’, it will preserve the array’s ‘C’ or ‘F’ ordering\n",
|
||
"# ‘A’ means to read the elements in Fortran-like index order if a is Fortran contiguous in memory, C-like order otherwise.\n",
|
||
"# ‘K’ means to read the elements in the order they occur in memory, except for reversing the data when strides are negative. By default, ‘C’ index order is used.\n",
|
||
"# Transposing it\n",
|
||
"print(np.ravel(a.T))\n",
|
||
"print(np.ravel(a.T, order='A'))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "545df59a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"## Gaussian Elimination\n",
|
||
"\n",
|
||
"We start with the linear set of equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "796d7554",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}\\mathbf{x} = \\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "35a9c235",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We assume also that the matrix $\\mathbf{A}$ is non-singular and that the\n",
|
||
"matrix elements along the diagonal satisfy $a_{ii} \\ne 0$. Simple $4\\times 4 $ example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "432d4f8e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11}& a_{12} &a_{13}& a_{14}\\\\\n",
|
||
" a_{21}& a_{22} &a_{23}& a_{24}\\\\\n",
|
||
" a_{31}& a_{32} &a_{33}& a_{34}\\\\\n",
|
||
" a_{41}& a_{42} &a_{43}& a_{44}\\\\\n",
|
||
" \\end{bmatrix} \\begin{bmatrix}\n",
|
||
" x_1\\\\\n",
|
||
" x_2\\\\\n",
|
||
" x_3 \\\\\n",
|
||
" x_4 \\\\\n",
|
||
" \\end{bmatrix}\n",
|
||
" =\\begin{bmatrix}\n",
|
||
" w_1\\\\\n",
|
||
" w_2\\\\\n",
|
||
" w_3 \\\\\n",
|
||
" w_4\\\\\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c06fa57b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4e8d9bd5",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "112c2488",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4774bcd7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5bf16627",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4da1a1c6",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The basic idea of Gaussian elimination is to use the first equation to eliminate the first unknown $x_1$\n",
|
||
"from the remaining $n-1$ equations. Then we use the new second equation to eliminate the second unknown\n",
|
||
"$x_2$ from the remaining $n-2$ equations. With $n-1$ such eliminations\n",
|
||
"we obtain a so-called upper triangular set of equations of the form"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "041684ac",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{11}x_1 +b_{12}x_2 +b_{13}x_3 + b_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ba6214cb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{22}x_2 + b_{23}x_3 + b_{24}x_4=y_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2a1d0555",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{33}x_3 + b_{34}x_4=y_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "51570380",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"eq:gaussbacksub\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"b_{44}x_4=y_4. \\nonumber\n",
|
||
"\\label{eq:gaussbacksub} \\tag{1}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e51b3aa8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We can solve this system of equations recursively starting from $x_n$ (in our case $x_4$) and proceed with\n",
|
||
"what is called a backward substitution. \n",
|
||
"\n",
|
||
"This process can be expressed mathematically as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c78b600",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto1\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" x_m = \\frac{1}{b_{mm}}\\left(y_m-\\sum_{k=m+1}^nb_{mk}x_k\\right)\\quad m=n-1,n-2,\\dots,1.\n",
|
||
"\\label{_auto1} \\tag{2}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1cd1015f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To arrive at such an upper triangular system of equations, we start by eliminating\n",
|
||
"the unknown $x_1$ for $j=2,n$. We achieve this by multiplying the first equation by $a_{j1}/a_{11}$ and then subtract\n",
|
||
"the result from the $j$th equation. We assume obviously that $a_{11}\\ne 0$ and that\n",
|
||
"$\\mathbf{A}$ is not singular.\n",
|
||
"\n",
|
||
"Our actual $4\\times 4$ example reads after the first operation"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "34093ce7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11}& a_{12} &a_{13}& a_{14}\\\\\n",
|
||
" 0& (a_{22}-\\frac{a_{21}a_{12}}{a_{11}}) &(a_{23}-\\frac{a_{21}a_{13}}{a_{11}}) & (a_{24}-\\frac{a_{21}a_{14}}{a_{11}})\\\\\n",
|
||
"0& (a_{32}-\\frac{a_{31}a_{12}}{a_{11}})& (a_{33}-\\frac{a_{31}a_{13}}{a_{11}})& (a_{34}-\\frac{a_{31}a_{14}}{a_{11}})\\\\\n",
|
||
"0&(a_{42}-\\frac{a_{41}a_{12}}{a_{11}}) &(a_{43}-\\frac{a_{41}a_{13}}{a_{11}}) & (a_{44}-\\frac{a_{41}a_{14}}{a_{11}}) \\\\\n",
|
||
" \\end{bmatrix} \\begin{bmatrix}\n",
|
||
" x_1\\\\\n",
|
||
" x_2\\\\\n",
|
||
" x_3 \\\\\n",
|
||
" x_4 \\\\\n",
|
||
" \\end{bmatrix} \n",
|
||
" =\\begin{bmatrix}\n",
|
||
" y_1\\\\\n",
|
||
" w_2^{(2)}\\\\\n",
|
||
" w_3^{(2)} \\\\\n",
|
||
" w_4^{(2)}\\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f269239d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"or"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d4477133",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"b_{11}x_1 +b_{12}x_2 +b_{13}x_3 + b_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "25bdfc8c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{22}x_2 + a^{(2)}_{23}x_3 + a^{(2)}_{24}x_4=w^{(2)}_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "34b8408c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{32}x_2 + a^{(2)}_{33}x_3 + a^{(2)}_{34}x_4=w^{(2)}_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b8185f8d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a^{(2)}_{42}x_2 + a^{(2)}_{43}x_3 + a^{(2)}_{44}x_4=w^{(2)}_4, \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4a218e4e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto2\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation} \n",
|
||
"\\label{_auto2} \\tag{3}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ba424207",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The new coefficients are"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "10f17fb1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto3\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" b_{1k} = a_{1k}^{(1)} \\quad k=1,\\dots,n,\n",
|
||
"\\label{_auto3} \\tag{4}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e22879fc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where each $a_{1k}^{(1)}$ is equal to the original $a_{1k}$ element. The other coefficients are"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3b765484",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto4\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
"a_{jk}^{(2)} = a_{jk}^{(1)}-\\frac{a_{j1}^{(1)}a_{1k}^{(1)}}{a_{11}^{(1)}} \\quad j,k=2,\\dots,n,\n",
|
||
"\\label{_auto4} \\tag{5}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9383783c",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"with a new right-hand side given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c0a3994f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto5\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
"y_{1}=w_1^{(1)}, \\quad w_j^{(2)} =w_j^{(1)}-\\frac{a_{j1}^{(1)}w_1^{(1)}}{a_{11}^{(1)}} \\quad j=2,\\dots,n.\n",
|
||
"\\label{_auto5} \\tag{6}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b5fab15e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"We have also set $w_1^{(1)}=w_1$, the original vector element.\n",
|
||
"We see that the system of unknowns $x_1,\\dots,x_n$ is transformed into an $(n-1)\\times (n-1)$ problem.\n",
|
||
"\n",
|
||
"This step is called forward substitution.\n",
|
||
"Proceeding with these substitutions, we obtain the\n",
|
||
"general expressions for the new coefficients"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4d2d01dd",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto6\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" a_{jk}^{(m+1)} = a_{jk}^{(m)}-\\frac{a_{jm}^{(m)}a_{mk}^{(m)}}{a_{mm}^{(m)}} \\quad j,k=m+1,\\dots,n,\n",
|
||
"\\label{_auto6} \\tag{7}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8c1e5ff9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"with $m=1,\\dots,n-1$ and a\n",
|
||
"right-hand side given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7165e693",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"<!-- Equation labels as ordinary links -->\n",
|
||
"<div id=\"_auto7\"></div>\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{equation}\n",
|
||
" w_j^{(m+1)} =w_j^{(m)}-\\frac{a_{jm}^{(m)}w_m^{(m)}}{a_{mm}^{(m)}}\\quad j=m+1,\\dots,n.\n",
|
||
"\\label{_auto7} \\tag{8}\n",
|
||
"\\end{equation}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2c6646d7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This set of $n-1$ elimations leads us to an equations which is solved by back substitution.\n",
|
||
"If the arithmetics is exact and the matrix $\\mathbf{A}$ is not singular, then the computed answer will be exact.\n",
|
||
"\n",
|
||
"Even though the matrix elements along the diagonal are not zero,\n",
|
||
"numerically small numbers may appear and subsequent divisions may lead to large numbers, which, if added\n",
|
||
"to a small number may yield losses of precision. Suppose for example that our first division in $(a_{22}-a_{21}a_{12}/a_{11})$\n",
|
||
"results in $-10^{-7}$ and that $a_{22}$ is one.\n",
|
||
"one. We are then\n",
|
||
"adding $10^7+1$. With single precision this results in $10^7$.\n",
|
||
"\n",
|
||
" * Gaussian elimination, $O(2/3n^3)$ flops, general matrix\n",
|
||
"\n",
|
||
" * LU decomposition, upper triangular and lower tridiagonal matrices, $O(2/3n^3)$ flops, general matrix. Get easily the inverse, determinant and can solve linear equations with back-substitution only, $O(n^2)$ flops\n",
|
||
"\n",
|
||
" * Cholesky decomposition. Real symmetric or hermitian positive definite matrix, $O(1/3n^3)$ flops.\n",
|
||
"\n",
|
||
" * Tridiagonal linear systems, important for differential equations. Normally positive definite and non-singular. $O(8n)$ flops for symmetric. Special case of banded matrices.\n",
|
||
"\n",
|
||
" * Singular value decomposition\n",
|
||
"\n",
|
||
" * the QR method will be discussed in chapter 7 in connection with eigenvalue systems. $O(4/3n^3)$ flops.\n",
|
||
"\n",
|
||
"The LU decomposition method means that we can rewrite\n",
|
||
"this matrix as the product of two matrices $\\mathbf{L}$ and $\\mathbf{U}$\n",
|
||
"where"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cc3c0cf7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{bmatrix}\n",
|
||
" a_{11} & a_{12} & a_{13} & a_{14} \\\\\n",
|
||
" a_{21} & a_{22} & a_{23} & a_{24} \\\\\n",
|
||
" a_{31} & a_{32} & a_{33} & a_{34} \\\\\n",
|
||
" a_{41} & a_{42} & a_{43} & a_{44}\n",
|
||
" \\end{bmatrix}\n",
|
||
" = \\begin{bmatrix}\n",
|
||
" 1 & 0 & 0 & 0 \\\\\n",
|
||
" l_{21} & 1 & 0 & 0 \\\\\n",
|
||
" l_{31} & l_{32} & 1 & 0 \\\\\n",
|
||
" l_{41} & l_{42} & l_{43} & 1\n",
|
||
" \\end{bmatrix}\n",
|
||
" \\begin{bmatrix}\n",
|
||
" u_{11} & u_{12} & u_{13} & u_{14} \\\\\n",
|
||
" 0 & u_{22} & u_{23} & u_{24} \\\\\n",
|
||
" 0 & 0 & u_{33} & u_{34} \\\\\n",
|
||
" 0 & 0 & 0 & u_{44}\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7256c8d3",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"LU decomposition forms the backbone of other algorithms in linear algebra, such as the\n",
|
||
"solution of linear equations given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "634741e9",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8a16abbb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "442b2dd1",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2be3e530",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8f743387",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The above set of equations is conveniently solved by using LU decomposition as an intermediate step.\n",
|
||
"\n",
|
||
"The matrix $\\mathbf{A}\\in \\mathbb{R}^{n\\times n}$ has an LU factorization if the determinant\n",
|
||
"is different from zero. If the LU factorization exists and $\\mathbf{A}$ is non-singular, then the LU factorization\n",
|
||
"is unique and the determinant is given by"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0fe06953",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"det\\{\\mathbf{A}\\}=det\\{\\mathbf{LU}\\}= det\\{\\mathbf{L}\\}det\\{\\mathbf{U}\\}=u_{11}u_{22}\\dots u_{nn}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8232172f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"There are at least three main advantages with LU decomposition compared with standard Gaussian elimination:\n",
|
||
"\n",
|
||
" * It is straightforward to compute the determinant of a matrix\n",
|
||
"\n",
|
||
" * If we have to solve sets of linear equations with the same matrix but with different vectors $\\mathbf{y}$, the number of FLOPS is of the order $n^3$.\n",
|
||
"\n",
|
||
" * The inverse is such an operation \n",
|
||
"\n",
|
||
"With the LU decomposition it is rather\n",
|
||
"simple to solve a system of linear equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "87c2e89a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{11}x_1 +a_{12}x_2 +a_{13}x_3 + a_{14}x_4=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "50fc7a76",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{21}x_1 + a_{22}x_2 + a_{23}x_3 + a_{24}x_4=w_2 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e0379f33",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{31}x_1 + a_{32}x_2 + a_{33}x_3 + a_{34}x_4=w_3 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "11e67b7e",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"a_{41}x_1 + a_{42}x_2 + a_{43}x_3 + a_{44}x_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8cda9070",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This can be written in matrix form as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "908f05f7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{Ax}=\\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "53053832",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"where $\\mathbf{A}$ and $\\mathbf{w}$ are known and we have to solve for\n",
|
||
"$\\mathbf{x}$. Using the LU dcomposition we write"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f3314fa8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A} \\mathbf{x} \\equiv \\mathbf{L} \\mathbf{U} \\mathbf{x} =\\mathbf{w}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2043fafc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"The previous equation can be calculated in two steps"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6fff8406",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{L} \\mathbf{y} = \\mathbf{w};\\qquad \\mathbf{Ux}=\\mathbf{y}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9a60901d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"To show that this is correct we use to the LU decomposition\n",
|
||
"to rewrite our system of linear equations as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "daacf54d",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LUx}=\\mathbf{w},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "da702290",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and since the determinant of $\\mathbf{L}$ is equal to 1 (by construction\n",
|
||
"since the diagonals of $\\mathbf{L}$ equal 1) we can use the inverse of\n",
|
||
"$\\mathbf{L}$ to obtain"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c2a04b00",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{Ux}=\\mathbf{L^{-1}w}=\\mathbf{y},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d1214f7f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"which yields the intermediate step"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e8572956",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{L^{-1}w}=\\mathbf{y}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6920b812",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and as soon as we have $\\mathbf{y}$ we can obtain $\\mathbf{x}$\n",
|
||
"through $\\mathbf{Ux}=\\mathbf{y}$.\n",
|
||
"\n",
|
||
"For our four-dimentional example this takes the form"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "379ca640",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"y_1=w_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8c753134",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{21}y_1 + y_2=w_2\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "12e32410",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{31}y_1 + l_{32}y_2 + y_3 =w_3\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "01e9e4ae",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"l_{41}y_1 + l_{42}y_2 + l_{43}y_3 + y_4=w_4. \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a27d4632",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4dae7cfb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{11}x_1 +u_{12}x_2 +u_{13}x_3 + u_{14}x_4=y_1 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5241a055",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{22}x_2 + u_{23}x_3 + u_{24}x_4=y_2\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f590f30b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{33}x_3 + u_{34}x_4=y_3\\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dbddd25a",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"u_{44}x_4=y_4 \\nonumber\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8d5dc72b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"This example shows the basis for the algorithm\n",
|
||
"needed to solve the set of $n$ linear equations.\n",
|
||
"\n",
|
||
"The algorithm goes as follows\n",
|
||
"\n",
|
||
" * Set up the matrix $\\bf A$ and the vector $\\bf w$ with their correct dimensions. This determines the dimensionality of the unknown vector $\\bf x$.\n",
|
||
"\n",
|
||
" * Then LU decompose the matrix $\\bf A$ through a call to the function `ludcmp(double a, int n, int indx, double &d)`. This functions returns the LU decomposed matrix $\\bf A$, its determinant and the vector indx which keeps track of the number of interchanges of rows. If the determinant is zero, the solution is malconditioned.\n",
|
||
"\n",
|
||
" * Thereafter you call the function `lubksb(double a, int n, int indx, double w)` which uses the LU decomposed matrix $\\bf A$ and the vector $\\bf w$ and returns $\\bf x$ in the same place as $\\bf w$. Upon exit the original content in $\\bf w$ is destroyed. If you wish to keep this information, you should make a backup of it in your calling function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fcf615b7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"### LU Decomposition, the inverse of a matrix\n",
|
||
"\n",
|
||
"If the inverse exists then"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e2be427",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}^{-1}\\mathbf{A}=\\mathbf{I},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "519b78bb",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"the identity matrix. With an LU decomposed matrix we can rewrite the last equation as"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a730fd5f",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\mathbf{A}^{-1}=\\mathbf{I}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2cc4e115",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"If we assume that the first column (that is column 1) of the inverse matrix\n",
|
||
"can be written as a vector with unknown entries"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "03c4a0ae",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{A}_1^{-1}= \\begin{bmatrix}\n",
|
||
" a_{11}^{-1} \\\\\n",
|
||
" a_{21}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n1}^{-1} \\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "93ef84cc",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"then we have a linear set of equations"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "04c367a7",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\begin{bmatrix}\n",
|
||
" a_{11}^{-1} \\\\\n",
|
||
" a_{21}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n1}^{-1} \\\\\n",
|
||
" \\end{bmatrix} =\\begin{bmatrix}\n",
|
||
" 1 \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\end{bmatrix}.\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8f903ad8",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"In a similar way we can compute the unknow entries of the second column,"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c343638b",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\mathbf{LU}\\begin{bmatrix}\n",
|
||
" a_{12}^{-1} \\\\\n",
|
||
" a_{22}^{-1} \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" a_{n2}^{-1} \\\\\n",
|
||
" \\end{bmatrix}=\\begin{bmatrix}\n",
|
||
" 0 \\\\\n",
|
||
" 1 \\\\\n",
|
||
" \\dots \\\\\n",
|
||
" 0 \\\\\n",
|
||
" \\end{bmatrix},\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1c013532",
|
||
"metadata": {
|
||
"editable": true
|
||
},
|
||
"source": [
|
||
"and continue till we have solved all $n$ sets of linear equations.\n",
|
||
"\n",
|
||
"The calculation of the inverse here assumes that it actually\n",
|
||
"exists. In many machine learning applications there may be strong\n",
|
||
"linear dependencies among the various columns and/or rows. In our\n",
|
||
"discussions of linear regression we will dive into the mathematics of\n",
|
||
"the singular value decomposition, an algorithm which will allow us to calculate the so-called pseudo-inverse.\n",
|
||
"These details will be presented in our linear regression chapter."
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.9.15"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |